2024-08-19T10:25:10.486 INFO:root:teuthology version: 1.2.0 2024-08-19T10:25:10.506 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-19T10:25:10.595 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373 branch: main description: rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-bitmap} rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_mon_workunits} email: ceph-qa@ceph.com first_in_suite: false job_id: '7859373' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: main ceph: conf: client: debug monc: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 1 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: mon ms inject internal delays: 0.002 ms inject socket failures: 2500 ms type: async mgr: debug mgr: 20 debug monc: 10 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 paxos service trim min: 5 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore write v2: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(TOO_FEW_PGS\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(AUTH_BAD_CAPS\) sha1: 797669399b32d2915e64eb6d4b1257ae3c50f80b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 797669399b32d2915e64eb6d4b1257ae3c50f80b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 964adda23da9f7b057cb34fdc0b82aeb7ed65324 owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.b - mon.c - mon.d - mon.e - osd.0 - osd.1 - osd.2 - - mon.f - mon.g - mon.h - mon.i - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 1858 sha1: 797669399b32d2915e64eb6d4b1257ae3c50f80b sleep_before_teardown: 0 subset: 29813/100000 suite: rados suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 964adda23da9f7b057cb34fdc0b82aeb7ed65324 targets: smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE1sAOXZx6zu9Ll+PlBcKo6ctF8ClSPtWmlSZud5Qvv9kSgU/F0abFurL58XFgl3HDelBhAATs5mojiG4Gyckv4= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE0n+qO+CEheY4270wXhR5+DoAaKoN8bo5sSb/hJKi6wLjrEQLX6MCFnyabola2NeMvtyT8sRGJZ/kqiSALWM2M= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 store_thrash: true thrash_delay: 1 thrash_many: true - workunit: clients: client.0: - mon/pool_ops.sh - mon/crush_ops.sh - mon/osd.sh - mon/caps.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-18_20:00:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-19T10:25:10.595 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa; will attempt to use it 2024-08-19T10:25:10.596 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa/tasks 2024-08-19T10:25:10.596 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-19T10:25:10.598 INFO:teuthology.task.internal:Checking packages... 2024-08-19T10:25:10.617 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '797669399b32d2915e64eb6d4b1257ae3c50f80b' 2024-08-19T10:25:10.617 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-19T10:25:10.617 INFO:teuthology.packaging:ref: None 2024-08-19T10:25:10.617 INFO:teuthology.packaging:tag: None 2024-08-19T10:25:10.617 INFO:teuthology.packaging:branch: main 2024-08-19T10:25:10.617 INFO:teuthology.packaging:sha1: 797669399b32d2915e64eb6d4b1257ae3c50f80b 2024-08-19T10:25:10.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=main 2024-08-19T10:25:10.832 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-5847-g79766939-1jammy 2024-08-19T10:25:10.834 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-19T10:25:10.841 INFO:teuthology.task.internal:no buildpackages task found 2024-08-19T10:25:10.841 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-19T10:25:10.847 INFO:teuthology.task.internal:Saving configuration 2024-08-19T10:25:10.861 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-19T10:25:10.868 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-19T10:25:10.902 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-19 10:19:35.590777', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJGtnz6KypThLrZiCE6p6+nQpD6WWFDezoXoD3qB6SqI'} 2024-08-19T10:25:10.933 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-19 10:19:35.588907', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICaP4srtVzbNW0n6k+BireTarfH8Lp4fcnQqKTe5Q62g'} 2024-08-19T10:25:10.933 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-19T10:25:10.940 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'] 2024-08-19T10:25:10.940 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2024-08-19T10:25:10.940 INFO:teuthology.run_tasks:Running task console_log... 2024-08-19T10:25:11.017 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4d7551ee60>, signals=[15]) 2024-08-19T10:25:11.017 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-19T10:25:11.023 INFO:teuthology.task.internal:Opening connections... 2024-08-19T10:25:11.024 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2024-08-19T10:25:11.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:25:11.115 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-08-19T10:25:11.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:25:11.202 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-19T10:25:11.210 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2024-08-19T10:25:11.215 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2024-08-19T10:25:11.216 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-08-19T10:25:11.263 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-08-19T10:25:11.263 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Ubuntu" 2024-08-19T10:25:11.263 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="22.04" 2024-08-19T10:25:11.263 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-08-19T10:25:11.263 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_CODENAME=jammy 2024-08-19T10:25:11.263 INFO:teuthology.orchestra.run.smithi089.stdout:ID=ubuntu 2024-08-19T10:25:11.263 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE=debian 2024-08-19T10:25:11.263 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-19T10:25:11.264 INFO:teuthology.orchestra.run.smithi089.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-19T10:25:11.264 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-19T10:25:11.264 INFO:teuthology.orchestra.run.smithi089.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-19T10:25:11.264 INFO:teuthology.orchestra.run.smithi089.stdout:UBUNTU_CODENAME=jammy 2024-08-19T10:25:11.264 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-08-19T10:25:11.296 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-08-19T10:25:11.301 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-08-19T10:25:11.301 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-08-19T10:25:11.348 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-08-19T10:25:11.348 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="Ubuntu" 2024-08-19T10:25:11.348 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="22.04" 2024-08-19T10:25:11.348 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-08-19T10:25:11.348 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_CODENAME=jammy 2024-08-19T10:25:11.348 INFO:teuthology.orchestra.run.smithi190.stdout:ID=ubuntu 2024-08-19T10:25:11.348 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE=debian 2024-08-19T10:25:11.348 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-19T10:25:11.348 INFO:teuthology.orchestra.run.smithi190.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-19T10:25:11.349 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-19T10:25:11.349 INFO:teuthology.orchestra.run.smithi190.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-19T10:25:11.349 INFO:teuthology.orchestra.run.smithi190.stdout:UBUNTU_CODENAME=jammy 2024-08-19T10:25:11.349 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-08-19T10:25:11.379 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-19T10:25:11.388 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-19T10:25:11.394 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-19T10:25:11.394 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2024-08-19T10:25:11.397 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-08-19T10:25:11.401 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-19T10:25:11.413 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-19T10:25:11.413 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2024-08-19T10:25:11.444 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-08-19T10:25:11.451 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-19T10:25:11.530 INFO:teuthology.run_tasks:Running task kernel... 2024-08-19T10:25:11.545 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-08-19T10:25:11.545 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.f': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-08-19T10:25:11.545 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-19T10:25:11.546 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-19T10:25:11.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T10:25:11.550 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2024-08-19T10:25:11.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T10:25:11.551 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-08-19T10:25:11.597 INFO:teuthology.orchestra.run.smithi190.stdout:5.15.0-56-generic 2024-08-19T10:25:11.597 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.15.0-56-generic 2024-08-19T10:25:11.597 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get clean 2024-08-19T10:25:11.598 INFO:teuthology.orchestra.run.smithi089.stdout:5.15.0-56-generic 2024-08-19T10:25:11.599 INFO:teuthology.task.kernel:Running kernel on smithi089: 5.15.0-56-generic 2024-08-19T10:25:11.599 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-get clean 2024-08-19T10:25:11.734 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-get update 2024-08-19T10:25:11.736 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2024-08-19T10:25:11.896 INFO:teuthology.orchestra.run.smithi190.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-08-19T10:25:11.993 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-08-19T10:25:12.000 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-08-19T10:25:12.024 INFO:teuthology.orchestra.run.smithi089.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-08-19T10:25:12.065 INFO:teuthology.orchestra.run.smithi089.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-08-19T10:25:12.074 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-08-19T10:25:12.090 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [524 kB] 2024-08-19T10:25:12.159 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,729 kB] 2024-08-19T10:25:12.200 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [287 kB] 2024-08-19T10:25:12.206 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.2 kB] 2024-08-19T10:25:12.207 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.3 kB] 2024-08-19T10:25:12.209 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,247 kB] 2024-08-19T10:25:12.240 INFO:teuthology.orchestra.run.smithi190.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [388 kB] 2024-08-19T10:25:12.244 INFO:teuthology.orchestra.run.smithi190.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [572 B] 2024-08-19T10:25:12.244 INFO:teuthology.orchestra.run.smithi190.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [621 kB] 2024-08-19T10:25:12.249 INFO:teuthology.orchestra.run.smithi190.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [890 kB] 2024-08-19T10:25:12.257 INFO:teuthology.orchestra.run.smithi190.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [175 kB] 2024-08-19T10:25:12.258 INFO:teuthology.orchestra.run.smithi190.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.0 kB] 2024-08-19T10:25:12.259 INFO:teuthology.orchestra.run.smithi190.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-08-19T10:25:12.259 INFO:teuthology.orchestra.run.smithi190.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-08-19T10:25:12.273 INFO:teuthology.orchestra.run.smithi190.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-08-19T10:25:12.274 INFO:teuthology.orchestra.run.smithi190.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-08-19T10:25:12.420 INFO:teuthology.orchestra.run.smithi089.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-08-19T10:25:12.432 INFO:teuthology.orchestra.run.smithi190.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-08-19T10:25:12.458 INFO:teuthology.orchestra.run.smithi089.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,729 kB] 2024-08-19T10:25:12.558 INFO:teuthology.orchestra.run.smithi089.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [684 kB] 2024-08-19T10:25:12.570 INFO:teuthology.orchestra.run.smithi190.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,948 kB] 2024-08-19T10:25:12.764 INFO:teuthology.orchestra.run.smithi089.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,948 kB] 2024-08-19T10:25:12.872 INFO:teuthology.orchestra.run.smithi089.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [524 kB] 2024-08-19T10:25:12.881 INFO:teuthology.orchestra.run.smithi190.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [684 kB] 2024-08-19T10:25:12.889 INFO:teuthology.orchestra.run.smithi089.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [287 kB] 2024-08-19T10:25:12.898 INFO:teuthology.orchestra.run.smithi089.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.2 kB] 2024-08-19T10:25:12.898 INFO:teuthology.orchestra.run.smithi089.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.3 kB] 2024-08-19T10:25:12.899 INFO:teuthology.orchestra.run.smithi089.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,247 kB] 2024-08-19T10:25:12.906 INFO:teuthology.orchestra.run.smithi089.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [345 kB] 2024-08-19T10:25:12.918 INFO:teuthology.orchestra.run.smithi190.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [345 kB] 2024-08-19T10:25:12.919 INFO:teuthology.orchestra.run.smithi089.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-08-19T10:25:12.920 INFO:teuthology.orchestra.run.smithi089.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.9 kB] 2024-08-19T10:25:12.922 INFO:teuthology.orchestra.run.smithi089.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,322 kB] 2024-08-19T10:25:12.933 INFO:teuthology.orchestra.run.smithi190.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-08-19T10:25:12.933 INFO:teuthology.orchestra.run.smithi190.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,322 kB] 2024-08-19T10:25:12.987 INFO:teuthology.orchestra.run.smithi089.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [401 kB] 2024-08-19T10:25:12.999 INFO:teuthology.orchestra.run.smithi190.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.9 kB] 2024-08-19T10:25:13.000 INFO:teuthology.orchestra.run.smithi190.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [401 kB] 2024-08-19T10:25:13.012 INFO:teuthology.orchestra.run.smithi089.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [604 B] 2024-08-19T10:25:13.012 INFO:teuthology.orchestra.run.smithi089.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [388 kB] 2024-08-19T10:25:13.013 INFO:teuthology.orchestra.run.smithi089.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [725 kB] 2024-08-19T10:25:13.015 INFO:teuthology.orchestra.run.smithi190.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [604 B] 2024-08-19T10:25:13.016 INFO:teuthology.orchestra.run.smithi190.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,111 kB] 2024-08-19T10:25:13.020 INFO:teuthology.orchestra.run.smithi089.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [572 B] 2024-08-19T10:25:13.020 INFO:teuthology.orchestra.run.smithi089.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [890 kB] 2024-08-19T10:25:13.027 INFO:teuthology.orchestra.run.smithi089.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,111 kB] 2024-08-19T10:25:13.036 INFO:teuthology.orchestra.run.smithi089.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [621 kB] 2024-08-19T10:25:13.047 INFO:teuthology.orchestra.run.smithi089.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [175 kB] 2024-08-19T10:25:13.049 INFO:teuthology.orchestra.run.smithi190.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [725 kB] 2024-08-19T10:25:13.053 INFO:teuthology.orchestra.run.smithi089.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.0 kB] 2024-08-19T10:25:13.054 INFO:teuthology.orchestra.run.smithi089.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-08-19T10:25:13.054 INFO:teuthology.orchestra.run.smithi089.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-08-19T10:25:13.055 INFO:teuthology.orchestra.run.smithi089.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-08-19T10:25:13.056 INFO:teuthology.orchestra.run.smithi089.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [259 kB] 2024-08-19T10:25:13.062 INFO:teuthology.orchestra.run.smithi089.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [25.9 kB] 2024-08-19T10:25:13.068 INFO:teuthology.orchestra.run.smithi190.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [259 kB] 2024-08-19T10:25:13.070 INFO:teuthology.orchestra.run.smithi089.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-08-19T10:25:13.071 INFO:teuthology.orchestra.run.smithi089.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-08-19T10:25:13.071 INFO:teuthology.orchestra.run.smithi089.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-08-19T10:25:13.074 INFO:teuthology.orchestra.run.smithi190.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [25.9 kB] 2024-08-19T10:25:13.075 INFO:teuthology.orchestra.run.smithi190.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-08-19T10:25:13.076 INFO:teuthology.orchestra.run.smithi190.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-08-19T10:25:13.127 INFO:teuthology.orchestra.run.smithi190.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-08-19T10:25:13.127 INFO:teuthology.orchestra.run.smithi190.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-08-19T10:25:13.127 INFO:teuthology.orchestra.run.smithi190.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-08-19T10:25:13.129 INFO:teuthology.orchestra.run.smithi190.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-08-19T10:25:13.135 INFO:teuthology.orchestra.run.smithi089.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-08-19T10:25:13.144 INFO:teuthology.orchestra.run.smithi089.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-08-19T10:25:13.144 INFO:teuthology.orchestra.run.smithi089.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-08-19T10:25:13.146 INFO:teuthology.orchestra.run.smithi089.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-08-19T10:25:13.147 INFO:teuthology.orchestra.run.smithi089.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-08-19T10:25:13.148 INFO:teuthology.orchestra.run.smithi089.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-08-19T10:25:13.151 INFO:teuthology.orchestra.run.smithi089.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-08-19T10:25:13.152 INFO:teuthology.orchestra.run.smithi089.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-08-19T10:25:13.159 INFO:teuthology.orchestra.run.smithi190.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-08-19T10:25:13.160 INFO:teuthology.orchestra.run.smithi190.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-08-19T10:25:13.160 INFO:teuthology.orchestra.run.smithi190.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-08-19T10:25:13.161 INFO:teuthology.orchestra.run.smithi190.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-08-19T10:25:13.161 INFO:teuthology.orchestra.run.smithi190.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-08-19T10:25:13.208 INFO:teuthology.orchestra.run.smithi190.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-08-19T10:25:13.232 INFO:teuthology.orchestra.run.smithi089.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-08-19T10:25:13.233 INFO:teuthology.orchestra.run.smithi089.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-08-19T10:25:18.584 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 15.5 MB in 2s (7,607 kB/s) 2024-08-19T10:25:18.936 INFO:teuthology.orchestra.run.smithi089.stdout:Fetched 15.5 MB in 3s (5,812 kB/s) 2024-08-19T10:25:19.875 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-08-19T10:25:19.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-19T10:25:19.946 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-08-19T10:25:20.137 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2024-08-19T10:25:20.137 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2024-08-19T10:25:20.291 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-08-19T10:25:20.303 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2024-08-19T10:25:20.303 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-08-19T10:25:20.303 INFO:teuthology.orchestra.run.smithi190.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-08-19T10:25:20.304 INFO:teuthology.orchestra.run.smithi190.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-08-19T10:25:20.304 INFO:teuthology.orchestra.run.smithi190.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-08-19T10:25:20.304 INFO:teuthology.orchestra.run.smithi190.stdout: python2.7-minimal 2024-08-19T10:25:20.304 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-19T10:25:20.305 INFO:teuthology.orchestra.run.smithi190.stdout:The following additional packages will be installed: 2024-08-19T10:25:20.305 INFO:teuthology.orchestra.run.smithi190.stdout: linux-generic linux-headers-5.15.0-118 linux-headers-5.15.0-118-generic 2024-08-19T10:25:20.306 INFO:teuthology.orchestra.run.smithi190.stdout: linux-headers-generic linux-image-5.15.0-118-generic 2024-08-19T10:25:20.306 INFO:teuthology.orchestra.run.smithi190.stdout: linux-modules-5.15.0-118-generic linux-modules-extra-5.15.0-118-generic 2024-08-19T10:25:20.307 INFO:teuthology.orchestra.run.smithi190.stdout:Suggested packages: 2024-08-19T10:25:20.307 INFO:teuthology.orchestra.run.smithi190.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-08-19T10:25:20.307 INFO:teuthology.orchestra.run.smithi190.stdout:Recommended packages: 2024-08-19T10:25:20.307 INFO:teuthology.orchestra.run.smithi190.stdout: thermald 2024-08-19T10:25:20.308 DEBUG:teuthology.orchestra.run.smithi089:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-19T10:25:20.361 INFO:teuthology.orchestra.run.smithi190.stdout:The following NEW packages will be installed: 2024-08-19T10:25:20.361 INFO:teuthology.orchestra.run.smithi190.stdout: linux-headers-5.15.0-118 linux-headers-5.15.0-118-generic 2024-08-19T10:25:20.361 INFO:teuthology.orchestra.run.smithi190.stdout: linux-image-5.15.0-118-generic linux-modules-5.15.0-118-generic 2024-08-19T10:25:20.362 INFO:teuthology.orchestra.run.smithi190.stdout: linux-modules-extra-5.15.0-118-generic 2024-08-19T10:25:20.363 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages will be upgraded: 2024-08-19T10:25:20.363 INFO:teuthology.orchestra.run.smithi190.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-19T10:25:20.370 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-08-19T10:25:20.543 INFO:teuthology.orchestra.run.smithi190.stdout:3 upgraded, 5 newly installed, 0 to remove and 333 not upgraded. 2024-08-19T10:25:20.543 INFO:teuthology.orchestra.run.smithi190.stdout:Need to get 113 MB of archives. 2024-08-19T10:25:20.543 INFO:teuthology.orchestra.run.smithi190.stdout:After this operation, 583 MB of additional disk space will be used. 2024-08-19T10:25:20.543 INFO:teuthology.orchestra.run.smithi190.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-118-generic amd64 5.15.0-118.128 [22.6 MB] 2024-08-19T10:25:20.580 INFO:teuthology.orchestra.run.smithi089.stdout:Building dependency tree... 2024-08-19T10:25:20.581 INFO:teuthology.orchestra.run.smithi089.stdout:Reading state information... 2024-08-19T10:25:20.756 INFO:teuthology.orchestra.run.smithi089.stdout:The following packages were automatically installed and are no longer required: 2024-08-19T10:25:20.757 INFO:teuthology.orchestra.run.smithi089.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-08-19T10:25:20.757 INFO:teuthology.orchestra.run.smithi089.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-08-19T10:25:20.757 INFO:teuthology.orchestra.run.smithi089.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-08-19T10:25:20.758 INFO:teuthology.orchestra.run.smithi089.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-08-19T10:25:20.758 INFO:teuthology.orchestra.run.smithi089.stdout: python2.7-minimal 2024-08-19T10:25:20.758 INFO:teuthology.orchestra.run.smithi089.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-19T10:25:20.758 INFO:teuthology.orchestra.run.smithi089.stdout:The following additional packages will be installed: 2024-08-19T10:25:20.758 INFO:teuthology.orchestra.run.smithi089.stdout: linux-generic linux-headers-5.15.0-118 linux-headers-5.15.0-118-generic 2024-08-19T10:25:20.759 INFO:teuthology.orchestra.run.smithi089.stdout: linux-headers-generic linux-image-5.15.0-118-generic 2024-08-19T10:25:20.759 INFO:teuthology.orchestra.run.smithi089.stdout: linux-modules-5.15.0-118-generic linux-modules-extra-5.15.0-118-generic 2024-08-19T10:25:20.760 INFO:teuthology.orchestra.run.smithi089.stdout:Suggested packages: 2024-08-19T10:25:20.760 INFO:teuthology.orchestra.run.smithi089.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-08-19T10:25:20.760 INFO:teuthology.orchestra.run.smithi089.stdout:Recommended packages: 2024-08-19T10:25:20.761 INFO:teuthology.orchestra.run.smithi089.stdout: thermald 2024-08-19T10:25:20.817 INFO:teuthology.orchestra.run.smithi089.stdout:The following NEW packages will be installed: 2024-08-19T10:25:20.817 INFO:teuthology.orchestra.run.smithi089.stdout: linux-headers-5.15.0-118 linux-headers-5.15.0-118-generic 2024-08-19T10:25:20.818 INFO:teuthology.orchestra.run.smithi089.stdout: linux-image-5.15.0-118-generic linux-modules-5.15.0-118-generic 2024-08-19T10:25:20.818 INFO:teuthology.orchestra.run.smithi089.stdout: linux-modules-extra-5.15.0-118-generic 2024-08-19T10:25:20.819 INFO:teuthology.orchestra.run.smithi089.stdout:The following packages will be upgraded: 2024-08-19T10:25:20.819 INFO:teuthology.orchestra.run.smithi089.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-19T10:25:20.870 INFO:teuthology.orchestra.run.smithi089.stdout:3 upgraded, 5 newly installed, 0 to remove and 333 not upgraded. 2024-08-19T10:25:20.870 INFO:teuthology.orchestra.run.smithi089.stdout:Need to get 113 MB of archives. 2024-08-19T10:25:20.870 INFO:teuthology.orchestra.run.smithi089.stdout:After this operation, 583 MB of additional disk space will be used. 2024-08-19T10:25:20.870 INFO:teuthology.orchestra.run.smithi089.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-118-generic amd64 5.15.0-118.128 [22.6 MB] 2024-08-19T10:25:21.320 INFO:teuthology.orchestra.run.smithi089.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-118-generic amd64 5.15.0-118.128 [11.5 MB] 2024-08-19T10:25:21.421 INFO:teuthology.orchestra.run.smithi089.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-118-generic amd64 5.15.0-118.128 [64.0 MB] 2024-08-19T10:25:21.917 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-118-generic amd64 5.15.0-118.128 [11.5 MB] 2024-08-19T10:25:22.212 INFO:teuthology.orchestra.run.smithi089.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.118.118 [1,694 B] 2024-08-19T10:25:22.212 INFO:teuthology.orchestra.run.smithi089.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.118.118 [2,466 B] 2024-08-19T10:25:22.213 INFO:teuthology.orchestra.run.smithi089.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-118 all 5.15.0-118.128 [12.4 MB] 2024-08-19T10:25:22.311 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-118-generic amd64 5.15.0-118.128 [64.0 MB] 2024-08-19T10:25:22.332 INFO:teuthology.orchestra.run.smithi089.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-118-generic amd64 5.15.0-118.128 [2,876 kB] 2024-08-19T10:25:22.354 INFO:teuthology.orchestra.run.smithi089.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.118.118 [2,322 B] 2024-08-19T10:25:22.743 INFO:teuthology.orchestra.run.smithi089.stdout:Fetched 113 MB in 2s (74.2 MB/s) 2024-08-19T10:25:22.896 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package linux-modules-5.15.0-118-generic. 2024-08-19T10:25:24.438 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.118.118 [1,694 B] 2024-08-19T10:25:24.439 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.118.118 [2,466 B] 2024-08-19T10:25:24.439 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-118 all 5.15.0-118.128 [12.4 MB] 2024-08-19T10:25:24.830 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-118-generic amd64 5.15.0-118.128 [2,876 kB] 2024-08-19T10:25:24.857 INFO:teuthology.orchestra.run.smithi089.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-08-19T10:25:24.861 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../0-linux-modules-5.15.0-118-generic_5.15.0-118.128_amd64.deb ... 2024-08-19T10:25:24.881 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.118.118 [2,322 B] 2024-08-19T10:25:24.926 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-modules-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:25.293 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 113 MB in 5s (25.1 MB/s) 2024-08-19T10:25:25.474 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-modules-5.15.0-118-generic. 2024-08-19T10:25:26.928 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package linux-image-5.15.0-118-generic. 2024-08-19T10:25:26.949 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../1-linux-image-5.15.0-118-generic_5.15.0-118.128_amd64.deb ... 2024-08-19T10:25:27.029 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-image-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:27.440 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-118-generic. 2024-08-19T10:25:27.460 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-118-generic_5.15.0-118.128_amd64.deb ... 2024-08-19T10:25:27.465 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-08-19T10:25:27.470 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../0-linux-modules-5.15.0-118-generic_5.15.0-118.128_amd64.deb ... 2024-08-19T10:25:27.506 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-modules-extra-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:27.538 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-modules-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:29.947 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-image-5.15.0-118-generic. 2024-08-19T10:25:29.968 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../1-linux-image-5.15.0-118-generic_5.15.0-118.128_amd64.deb ... 2024-08-19T10:25:30.048 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-image-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:30.450 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-118-generic. 2024-08-19T10:25:30.470 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-118-generic_5.15.0-118.128_amd64.deb ... 2024-08-19T10:25:30.533 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-modules-extra-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:33.840 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../3-linux-generic_5.15.0.118.118_amd64.deb ... 2024-08-19T10:25:33.967 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-generic (5.15.0.118.118) over (5.15.0.56.54) ... 2024-08-19T10:25:34.292 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.118.118_amd64.deb ... 2024-08-19T10:25:34.421 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-image-generic (5.15.0.118.118) over (5.15.0.56.54) ... 2024-08-19T10:25:34.716 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package linux-headers-5.15.0-118. 2024-08-19T10:25:34.738 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../5-linux-headers-5.15.0-118_5.15.0-118.128_all.deb ... 2024-08-19T10:25:34.782 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-headers-5.15.0-118 (5.15.0-118.128) ... 2024-08-19T10:25:36.239 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../3-linux-generic_5.15.0.118.118_amd64.deb ... 2024-08-19T10:25:36.365 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-generic (5.15.0.118.118) over (5.15.0.56.54) ... 2024-08-19T10:25:36.664 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.118.118_amd64.deb ... 2024-08-19T10:25:36.760 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-image-generic (5.15.0.118.118) over (5.15.0.56.54) ... 2024-08-19T10:25:37.030 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-headers-5.15.0-118. 2024-08-19T10:25:37.052 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../5-linux-headers-5.15.0-118_5.15.0-118.128_all.deb ... 2024-08-19T10:25:37.097 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-headers-5.15.0-118 (5.15.0-118.128) ... 2024-08-19T10:25:40.131 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package linux-headers-5.15.0-118-generic. 2024-08-19T10:25:40.168 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../6-linux-headers-5.15.0-118-generic_5.15.0-118.128_amd64.deb ... 2024-08-19T10:25:40.205 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-headers-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:43.255 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.118.118_amd64.deb ... 2024-08-19T10:25:43.313 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-headers-5.15.0-118-generic. 2024-08-19T10:25:43.350 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../6-linux-headers-5.15.0-118-generic_5.15.0-118.128_amd64.deb ... 2024-08-19T10:25:43.378 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking linux-headers-generic (5.15.0.118.118) over (5.15.0.56.54) ... 2024-08-19T10:25:43.396 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-headers-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:43.687 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-headers-5.15.0-118 (5.15.0-118.128) ... 2024-08-19T10:25:43.800 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-headers-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:43.926 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-headers-generic (5.15.0.118.118) ... 2024-08-19T10:25:44.056 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-image-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:45.369 INFO:teuthology.orchestra.run.smithi089.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-08-19T10:25:45.369 INFO:teuthology.orchestra.run.smithi089.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-08-19T10:25:45.369 INFO:teuthology.orchestra.run.smithi089.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-118-generic 2024-08-19T10:25:45.369 INFO:teuthology.orchestra.run.smithi089.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-118-generic 2024-08-19T10:25:45.542 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-modules-extra-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:46.019 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.118.118_amd64.deb ... 2024-08-19T10:25:46.125 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-headers-generic (5.15.0.118.118) over (5.15.0.56.54) ... 2024-08-19T10:25:46.458 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-headers-5.15.0-118 (5.15.0-118.128) ... 2024-08-19T10:25:46.580 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-headers-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:46.706 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-headers-generic (5.15.0.118.118) ... 2024-08-19T10:25:46.835 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-image-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:46.977 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-modules-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:48.148 INFO:teuthology.orchestra.run.smithi190.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-08-19T10:25:48.148 INFO:teuthology.orchestra.run.smithi190.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-08-19T10:25:48.148 INFO:teuthology.orchestra.run.smithi190.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-118-generic 2024-08-19T10:25:48.149 INFO:teuthology.orchestra.run.smithi190.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-118-generic 2024-08-19T10:25:48.304 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-modules-extra-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:48.703 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-image-generic (5.15.0.118.118) ... 2024-08-19T10:25:48.829 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up linux-generic (5.15.0.118.118) ... 2024-08-19T10:25:48.955 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for linux-image-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:49.027 INFO:teuthology.orchestra.run.smithi089.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-19T10:25:49.028 INFO:teuthology.orchestra.run.smithi089.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-118-generic 2024-08-19T10:25:49.706 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-modules-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:51.081 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-image-generic (5.15.0.118.118) ... 2024-08-19T10:25:51.210 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-generic (5.15.0.118.118) ... 2024-08-19T10:25:51.333 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for linux-image-5.15.0-118-generic (5.15.0-118.128) ... 2024-08-19T10:25:51.405 INFO:teuthology.orchestra.run.smithi190.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-19T10:25:51.405 INFO:teuthology.orchestra.run.smithi190.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-118-generic 2024-08-19T10:26:03.511 INFO:teuthology.orchestra.run.smithi089.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-19T10:26:03.512 INFO:teuthology.orchestra.run.smithi089.stdout:Sourcing file `/etc/default/grub' 2024-08-19T10:26:03.527 INFO:teuthology.orchestra.run.smithi089.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-19T10:26:03.545 INFO:teuthology.orchestra.run.smithi089.stdout:Generating grub configuration file ... 2024-08-19T10:26:03.790 INFO:teuthology.orchestra.run.smithi089.stdout:Found linux image: /boot/vmlinuz-5.15.0-118-generic 2024-08-19T10:26:03.809 INFO:teuthology.orchestra.run.smithi089.stdout:Found initrd image: /boot/initrd.img-5.15.0-118-generic 2024-08-19T10:26:04.110 INFO:teuthology.orchestra.run.smithi089.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-08-19T10:26:04.116 INFO:teuthology.orchestra.run.smithi089.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-08-19T10:26:04.187 INFO:teuthology.orchestra.run.smithi089.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-08-19T10:26:04.192 INFO:teuthology.orchestra.run.smithi089.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-08-19T10:26:04.358 INFO:teuthology.orchestra.run.smithi089.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-08-19T10:26:04.358 INFO:teuthology.orchestra.run.smithi089.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-08-19T10:26:04.358 INFO:teuthology.orchestra.run.smithi089.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-08-19T10:26:04.398 INFO:teuthology.orchestra.run.smithi089.stdout:done 2024-08-19T10:26:04.887 DEBUG:teuthology.orchestra.run.smithi089:> dpkg -s linux-image-generic 2024-08-19T10:26:04.908 INFO:teuthology.orchestra.run.smithi089.stdout:Package: linux-image-generic 2024-08-19T10:26:04.908 INFO:teuthology.orchestra.run.smithi089.stdout:Status: install ok installed 2024-08-19T10:26:04.908 INFO:teuthology.orchestra.run.smithi089.stdout:Priority: optional 2024-08-19T10:26:04.908 INFO:teuthology.orchestra.run.smithi089.stdout:Section: kernel 2024-08-19T10:26:04.908 INFO:teuthology.orchestra.run.smithi089.stdout:Installed-Size: 21 2024-08-19T10:26:04.908 INFO:teuthology.orchestra.run.smithi089.stdout:Maintainer: Ubuntu Kernel Team 2024-08-19T10:26:04.909 INFO:teuthology.orchestra.run.smithi089.stdout:Architecture: amd64 2024-08-19T10:26:04.909 INFO:teuthology.orchestra.run.smithi089.stdout:Source: linux-meta 2024-08-19T10:26:04.909 INFO:teuthology.orchestra.run.smithi089.stdout:Version: 5.15.0.118.118 2024-08-19T10:26:04.909 INFO:teuthology.orchestra.run.smithi089.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-118), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-08-19T10:26:04.909 INFO:teuthology.orchestra.run.smithi089.stdout:Depends: linux-image-5.15.0-118-generic, linux-modules-extra-5.15.0-118-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-19T10:26:04.909 INFO:teuthology.orchestra.run.smithi089.stdout:Recommends: thermald 2024-08-19T10:26:04.909 INFO:teuthology.orchestra.run.smithi089.stdout:Description: Generic Linux kernel image 2024-08-19T10:26:04.909 INFO:teuthology.orchestra.run.smithi089.stdout: This package will always depend on the latest generic kernel image 2024-08-19T10:26:04.909 INFO:teuthology.orchestra.run.smithi089.stdout: available. 2024-08-19T10:26:04.909 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-118-generic 2024-08-19T10:26:04.910 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-19T10:26:04.910 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-19T10:26:04.910 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-08-19T10:26:04.910 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi089.front.sepia.ceph.com, path=None, version=distro) 2024-08-19T10:26:04.910 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-get clean 2024-08-19T10:26:05.075 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-get update 2024-08-19T10:26:05.200 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-08-19T10:26:05.210 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-08-19T10:26:05.228 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-08-19T10:26:05.295 INFO:teuthology.orchestra.run.smithi190.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-19T10:26:05.295 INFO:teuthology.orchestra.run.smithi190.stdout:Sourcing file `/etc/default/grub' 2024-08-19T10:26:05.306 INFO:teuthology.orchestra.run.smithi190.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-19T10:26:05.320 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-08-19T10:26:05.330 INFO:teuthology.orchestra.run.smithi190.stdout:Generating grub configuration file ... 2024-08-19T10:26:05.589 INFO:teuthology.orchestra.run.smithi190.stdout:Found linux image: /boot/vmlinuz-5.15.0-118-generic 2024-08-19T10:26:05.611 INFO:teuthology.orchestra.run.smithi190.stdout:Found initrd image: /boot/initrd.img-5.15.0-118-generic 2024-08-19T10:26:05.882 INFO:teuthology.orchestra.run.smithi190.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-08-19T10:26:05.887 INFO:teuthology.orchestra.run.smithi190.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-08-19T10:26:05.961 INFO:teuthology.orchestra.run.smithi190.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-08-19T10:26:05.966 INFO:teuthology.orchestra.run.smithi190.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-08-19T10:26:06.122 INFO:teuthology.orchestra.run.smithi190.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-08-19T10:26:06.122 INFO:teuthology.orchestra.run.smithi190.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-08-19T10:26:06.122 INFO:teuthology.orchestra.run.smithi190.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-08-19T10:26:06.171 INFO:teuthology.orchestra.run.smithi190.stdout:done 2024-08-19T10:26:06.584 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-08-19T10:26:06.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-19T10:26:06.640 DEBUG:teuthology.orchestra.run.smithi190:> dpkg -s linux-image-generic 2024-08-19T10:26:06.661 INFO:teuthology.orchestra.run.smithi190.stdout:Package: linux-image-generic 2024-08-19T10:26:06.661 INFO:teuthology.orchestra.run.smithi190.stdout:Status: install ok installed 2024-08-19T10:26:06.661 INFO:teuthology.orchestra.run.smithi190.stdout:Priority: optional 2024-08-19T10:26:06.661 INFO:teuthology.orchestra.run.smithi190.stdout:Section: kernel 2024-08-19T10:26:06.661 INFO:teuthology.orchestra.run.smithi190.stdout:Installed-Size: 21 2024-08-19T10:26:06.661 INFO:teuthology.orchestra.run.smithi190.stdout:Maintainer: Ubuntu Kernel Team 2024-08-19T10:26:06.661 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: amd64 2024-08-19T10:26:06.661 INFO:teuthology.orchestra.run.smithi190.stdout:Source: linux-meta 2024-08-19T10:26:06.661 INFO:teuthology.orchestra.run.smithi190.stdout:Version: 5.15.0.118.118 2024-08-19T10:26:06.661 INFO:teuthology.orchestra.run.smithi190.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-118), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-08-19T10:26:06.662 INFO:teuthology.orchestra.run.smithi190.stdout:Depends: linux-image-5.15.0-118-generic, linux-modules-extra-5.15.0-118-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-19T10:26:06.662 INFO:teuthology.orchestra.run.smithi190.stdout:Recommends: thermald 2024-08-19T10:26:06.662 INFO:teuthology.orchestra.run.smithi190.stdout:Description: Generic Linux kernel image 2024-08-19T10:26:06.662 INFO:teuthology.orchestra.run.smithi190.stdout: This package will always depend on the latest generic kernel image 2024-08-19T10:26:06.662 INFO:teuthology.orchestra.run.smithi190.stdout: available. 2024-08-19T10:26:06.662 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-118-generic 2024-08-19T10:26:06.662 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-19T10:26:06.662 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-19T10:26:06.662 INFO:teuthology.task.kernel:Installing distro kernel on mon.f... 2024-08-19T10:26:06.662 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=None, version=distro) 2024-08-19T10:26:06.663 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get clean 2024-08-19T10:26:06.663 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-08-19T10:26:06.861 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2024-08-19T10:26:06.872 INFO:teuthology.orchestra.run.smithi089.stdout:Building dependency tree... 2024-08-19T10:26:06.873 INFO:teuthology.orchestra.run.smithi089.stdout:Reading state information... 2024-08-19T10:26:07.047 INFO:teuthology.orchestra.run.smithi089.stdout:linux-image-generic is already the newest version (5.15.0.118.118). 2024-08-19T10:26:07.047 INFO:teuthology.orchestra.run.smithi089.stdout:The following packages were automatically installed and are no longer required: 2024-08-19T10:26:07.047 INFO:teuthology.orchestra.run.smithi089.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-08-19T10:26:07.047 INFO:teuthology.orchestra.run.smithi089.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-08-19T10:26:07.047 INFO:teuthology.orchestra.run.smithi089.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-08-19T10:26:07.048 INFO:teuthology.orchestra.run.smithi089.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-08-19T10:26:07.048 INFO:teuthology.orchestra.run.smithi089.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-08-19T10:26:07.048 INFO:teuthology.orchestra.run.smithi089.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-19T10:26:07.086 INFO:teuthology.orchestra.run.smithi089.stdout:0 upgraded, 0 newly installed, 0 to remove and 333 not upgraded. 2024-08-19T10:26:07.088 DEBUG:teuthology.orchestra.run.smithi089:> dpkg -s linux-image-generic 2024-08-19T10:26:07.104 INFO:teuthology.orchestra.run.smithi089.stdout:Package: linux-image-generic 2024-08-19T10:26:07.104 INFO:teuthology.orchestra.run.smithi089.stdout:Status: install ok installed 2024-08-19T10:26:07.104 INFO:teuthology.orchestra.run.smithi089.stdout:Priority: optional 2024-08-19T10:26:07.104 INFO:teuthology.orchestra.run.smithi089.stdout:Section: kernel 2024-08-19T10:26:07.104 INFO:teuthology.orchestra.run.smithi089.stdout:Installed-Size: 21 2024-08-19T10:26:07.104 INFO:teuthology.orchestra.run.smithi089.stdout:Maintainer: Ubuntu Kernel Team 2024-08-19T10:26:07.104 INFO:teuthology.orchestra.run.smithi089.stdout:Architecture: amd64 2024-08-19T10:26:07.104 INFO:teuthology.orchestra.run.smithi089.stdout:Source: linux-meta 2024-08-19T10:26:07.105 INFO:teuthology.orchestra.run.smithi089.stdout:Version: 5.15.0.118.118 2024-08-19T10:26:07.105 INFO:teuthology.orchestra.run.smithi089.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-118), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-08-19T10:26:07.105 INFO:teuthology.orchestra.run.smithi089.stdout:Depends: linux-image-5.15.0-118-generic, linux-modules-extra-5.15.0-118-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-19T10:26:07.105 INFO:teuthology.orchestra.run.smithi089.stdout:Recommends: thermald 2024-08-19T10:26:07.105 INFO:teuthology.orchestra.run.smithi089.stdout:Description: Generic Linux kernel image 2024-08-19T10:26:07.105 INFO:teuthology.orchestra.run.smithi089.stdout: This package will always depend on the latest generic kernel image 2024-08-19T10:26:07.105 INFO:teuthology.orchestra.run.smithi089.stdout: available. 2024-08-19T10:26:07.105 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-08-19T10:26:07.109 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-08-19T10:26:07.109 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-08-19T10:26:07.150 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.OcMdDBTUyJ 2024-08-19T10:26:07.150 DEBUG:teuthology.orchestra.run.smithi089:> sudo cp /boot/grub/grub.cfg /tmp/tmp.OcMdDBTUyJ 2024-08-19T10:26:07.183 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-08-19T10:26:07.205 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.OcMdDBTUyJ 2024-08-19T10:26:07.265 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-08-19T10:26:07.332 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.OcMdDBTUyJ is 10KB 2024-08-19T10:26:07.344 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.OcMdDBTUyJ 2024-08-19T10:26:07.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-19T10:26:07.406 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:26:07.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-19T10:26:07.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-19T10:26:07.486 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-118-generic 2024-08-19T10:26:07.486 DEBUG:teuthology.orchestra.run.smithi089:> sudo update-grub 2024-08-19T10:26:08.530 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-08-19T10:26:08.548 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-19T10:26:08.601 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-08-19T10:26:08.643 INFO:teuthology.orchestra.run.smithi089.stderr:Sourcing file `/etc/default/grub' 2024-08-19T10:26:08.645 INFO:teuthology.orchestra.run.smithi089.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-19T10:26:08.647 INFO:teuthology.orchestra.run.smithi089.stderr:Generating grub configuration file ... 2024-08-19T10:26:08.785 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2024-08-19T10:26:08.785 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2024-08-19T10:26:08.820 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-5.15.0-118-generic 2024-08-19T10:26:08.827 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-5.15.0-118-generic 2024-08-19T10:26:08.951 INFO:teuthology.orchestra.run.smithi190.stdout:linux-image-generic is already the newest version (5.15.0.118.118). 2024-08-19T10:26:08.952 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2024-08-19T10:26:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-08-19T10:26:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-08-19T10:26:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-08-19T10:26:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-08-19T10:26:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-08-19T10:26:08.952 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-19T10:26:08.992 INFO:teuthology.orchestra.run.smithi190.stdout:0 upgraded, 0 newly installed, 0 to remove and 333 not upgraded. 2024-08-19T10:26:08.994 DEBUG:teuthology.orchestra.run.smithi190:> dpkg -s linux-image-generic 2024-08-19T10:26:09.042 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-08-19T10:26:09.047 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-08-19T10:26:09.050 INFO:teuthology.orchestra.run.smithi190.stdout:Package: linux-image-generic 2024-08-19T10:26:09.050 INFO:teuthology.orchestra.run.smithi190.stdout:Status: install ok installed 2024-08-19T10:26:09.050 INFO:teuthology.orchestra.run.smithi190.stdout:Priority: optional 2024-08-19T10:26:09.050 INFO:teuthology.orchestra.run.smithi190.stdout:Section: kernel 2024-08-19T10:26:09.050 INFO:teuthology.orchestra.run.smithi190.stdout:Installed-Size: 21 2024-08-19T10:26:09.050 INFO:teuthology.orchestra.run.smithi190.stdout:Maintainer: Ubuntu Kernel Team 2024-08-19T10:26:09.050 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: amd64 2024-08-19T10:26:09.050 INFO:teuthology.orchestra.run.smithi190.stdout:Source: linux-meta 2024-08-19T10:26:09.050 INFO:teuthology.orchestra.run.smithi190.stdout:Version: 5.15.0.118.118 2024-08-19T10:26:09.050 INFO:teuthology.orchestra.run.smithi190.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-118), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-08-19T10:26:09.050 INFO:teuthology.orchestra.run.smithi190.stdout:Depends: linux-image-5.15.0-118-generic, linux-modules-extra-5.15.0-118-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-19T10:26:09.051 INFO:teuthology.orchestra.run.smithi190.stdout:Recommends: thermald 2024-08-19T10:26:09.051 INFO:teuthology.orchestra.run.smithi190.stdout:Description: Generic Linux kernel image 2024-08-19T10:26:09.051 INFO:teuthology.orchestra.run.smithi190.stdout: This package will always depend on the latest generic kernel image 2024-08-19T10:26:09.051 INFO:teuthology.orchestra.run.smithi190.stdout: available. 2024-08-19T10:26:09.051 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-08-19T10:26:09.090 INFO:teuthology.orchestra.run.smithi089.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-08-19T10:26:09.095 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.pzwslnNly9 2024-08-19T10:26:09.095 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub/grub.cfg /tmp/tmp.pzwslnNly9 2024-08-19T10:26:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-08-19T10:26:09.151 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.pzwslnNly9 2024-08-19T10:26:09.212 INFO:teuthology.orchestra.run.smithi089.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-08-19T10:26:09.212 INFO:teuthology.orchestra.run.smithi089.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-08-19T10:26:09.212 INFO:teuthology.orchestra.run.smithi089.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-08-19T10:26:09.226 INFO:teuthology.orchestra.run.smithi089.stderr:done 2024-08-19T10:26:09.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo shutdown -r now 2024-08-19T10:26:09.275 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.pzwslnNly9 is 10KB 2024-08-19T10:26:09.288 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.pzwslnNly9 2024-08-19T10:26:09.293 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-19T10:26:09.346 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:26:09.346 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-19T10:26:09.347 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-19T10:26:09.410 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-118-generic 2024-08-19T10:26:09.410 DEBUG:teuthology.orchestra.run.smithi190:> sudo update-grub 2024-08-19T10:26:10.544 INFO:teuthology.orchestra.run.smithi190.stderr:Sourcing file `/etc/default/grub' 2024-08-19T10:26:10.545 INFO:teuthology.orchestra.run.smithi190.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-19T10:26:10.547 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2024-08-19T10:26:10.723 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-5.15.0-118-generic 2024-08-19T10:26:10.729 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-5.15.0-118-generic 2024-08-19T10:26:10.947 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-08-19T10:26:10.954 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-08-19T10:26:10.998 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-08-19T10:26:11.003 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-08-19T10:26:11.121 INFO:teuthology.orchestra.run.smithi190.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-08-19T10:26:11.121 INFO:teuthology.orchestra.run.smithi190.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-08-19T10:26:11.121 INFO:teuthology.orchestra.run.smithi190.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-08-19T10:26:11.139 INFO:teuthology.orchestra.run.smithi190.stderr:done 2024-08-19T10:26:11.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2024-08-19T10:26:39.265 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-08-19T10:26:39.265 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-08-19T10:26:39.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:26:41.158 INFO:teuthology.task.kernel:Checking client mon.f for new kernel version... 2024-08-19T10:26:41.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-08-19T10:26:41.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:26:57.635 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-08-19T10:26:59.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-08-19T10:27:06.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-08-19T10:27:06.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:27:08.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-08-19T10:27:08.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:27:09.730 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-08-19T10:27:11.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-08-19T10:27:21.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-08-19T10:27:21.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:27:23.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-08-19T10:27:23.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:27:40.127 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-08-19T10:27:42.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-08-19T10:27:55.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-08-19T10:27:55.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:27:55.371 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-08-19T10:27:55.865 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-08-19T10:27:55.865 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-118-generic"... 2024-08-19T10:27:55.866 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2024-08-19T10:27:55.915 INFO:teuthology.orchestra.run.smithi089.stdout:5.15.0-118-generic 2024-08-19T10:27:55.915 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-118-generic vs 5.15.0-118-generic 2024-08-19T10:27:55.915 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-19T10:27:55.915 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-19T10:27:56.916 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-08-19T10:27:56.916 DEBUG:teuthology.orchestra.run.smithi089:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-19T10:27:56.986 INFO:teuthology.orchestra.run.smithi089.stdout:ttyS1 2024-08-19T10:27:57.014 DEBUG:teuthology.parallel:result is None 2024-08-19T10:27:57.175 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-08-19T10:27:57.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:27:57.493 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-08-19T10:27:58.025 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-08-19T10:27:58.026 INFO:teuthology.task.kernel:Checking kernel version of mon.f, want "5.15.0-118-generic"... 2024-08-19T10:27:58.026 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-08-19T10:27:58.072 INFO:teuthology.orchestra.run.smithi190.stdout:5.15.0-118-generic 2024-08-19T10:27:58.072 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-118-generic vs 5.15.0-118-generic 2024-08-19T10:27:58.073 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-19T10:27:58.073 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-19T10:27:59.074 INFO:teuthology.task.kernel:Enabling kdb on mon.f... 2024-08-19T10:27:59.074 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-19T10:27:59.449 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2024-08-19T10:27:59.486 DEBUG:teuthology.parallel:result is None 2024-08-19T10:27:59.487 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-19T10:27:59.494 INFO:teuthology.task.internal:Creating test directory... 2024-08-19T10:27:59.494 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-19T10:27:59.497 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-19T10:27:59.503 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-19T10:27:59.510 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-19T10:27:59.518 INFO:teuthology.task.internal:Creating archive directory... 2024-08-19T10:27:59.519 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-19T10:27:59.545 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-19T10:27:59.573 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-19T10:27:59.580 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-19T10:27:59.580 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-19T10:27:59.597 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-19T10:27:59.614 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T10:27:59.622 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T10:27:59.637 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T10:27:59.644 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T10:27:59.646 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-19T10:27:59.653 INFO:teuthology.task.internal:Configuring sudo... 2024-08-19T10:27:59.653 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-19T10:27:59.670 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-19T10:27:59.703 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-19T10:27:59.712 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-19T10:27:59.712 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-19T10:27:59.729 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-19T10:27:59.752 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T10:27:59.784 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T10:27:59.832 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:27:59.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-19T10:27:59.951 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T10:27:59.957 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T10:28:00.004 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:28:00.004 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-19T10:28:00.075 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2024-08-19T10:28:00.078 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-08-19T10:28:00.154 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-19T10:28:00.161 INFO:teuthology.task.internal:Starting timer... 2024-08-19T10:28:00.161 INFO:teuthology.run_tasks:Running task pcp... 2024-08-19T10:28:00.173 INFO:teuthology.run_tasks:Running task selinux... 2024-08-19T10:28:00.182 DEBUG:teuthology.task.selinux:Excluding smithi089: OS 'ubuntu' does not support SELinux 2024-08-19T10:28:00.182 DEBUG:teuthology.task.selinux:Excluding smithi190: OS 'ubuntu' does not support SELinux 2024-08-19T10:28:00.182 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-19T10:28:00.182 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-08-19T10:28:00.182 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-19T10:28:00.182 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-19T10:28:00.191 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-19T10:28:00.203 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-19T10:28:00.204 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi089.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-19T10:35:20.810 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-08-19T10:35:20.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-08-19T10:35:20.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:35:20.922 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-08-19T10:35:21.003 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-08-19T10:35:21.003 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-08-19T10:35:21.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T10:35:21.094 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-08-19T10:35:21.174 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-08-19T10:35:21.174 INFO:teuthology.run_tasks:Running task clock... 2024-08-19T10:35:21.185 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-19T10:35:21.185 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-19T10:35:21.186 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T10:35:21.188 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-19T10:35:21.188 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T10:35:21.215 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-08-19T10:35:21.215 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: Command line: ntpd -gq 2024-08-19T10:35:21.215 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: ---------------------------------------------------- 2024-08-19T10:35:21.215 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: ntp-4 is maintained by Network Time Foundation, 2024-08-19T10:35:21.215 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-08-19T10:35:21.215 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: corporation. Support and training for ntp-4 are 2024-08-19T10:35:21.216 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: available at https://www.nwtime.org/support 2024-08-19T10:35:21.216 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: ---------------------------------------------------- 2024-08-19T10:35:21.216 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: proto: precision = 0.063 usec (-24) 2024-08-19T10:35:21.216 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: basedate set to 2022-02-04 2024-08-19T10:35:21.216 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: gps base set to 2022-02-06 (week 2196) 2024-08-19T10:35:21.217 INFO:teuthology.orchestra.run.smithi089.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-19T10:35:21.217 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-19T10:35:21.217 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: restrict ::: KOD does nothing without LIMITED. 2024-08-19T10:35:21.217 INFO:teuthology.orchestra.run.smithi089.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-19T10:35:21.218 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: Listen and drop on 0 v6wildcard [::]:123 2024-08-19T10:35:21.219 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-19T10:35:21.219 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: Listen normally on 2 lo 127.0.0.1:123 2024-08-19T10:35:21.219 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: Listen normally on 3 enp3s0f1 172.21.15.89:123 2024-08-19T10:35:21.219 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: Listen normally on 4 lo [::1]:123 2024-08-19T10:35:21.219 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d501%5]:123 2024-08-19T10:35:21.219 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:21 ntpd[18394]: Listening on routing socket on fd #22 for interface updates 2024-08-19T10:35:21.247 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-08-19T10:35:21.247 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: Command line: ntpd -gq 2024-08-19T10:35:21.248 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: ---------------------------------------------------- 2024-08-19T10:35:21.248 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: ntp-4 is maintained by Network Time Foundation, 2024-08-19T10:35:21.248 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-08-19T10:35:21.248 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: corporation. Support and training for ntp-4 are 2024-08-19T10:35:21.248 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: available at https://www.nwtime.org/support 2024-08-19T10:35:21.248 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: ---------------------------------------------------- 2024-08-19T10:35:21.249 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: proto: precision = 0.046 usec (-24) 2024-08-19T10:35:21.249 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: basedate set to 2022-02-04 2024-08-19T10:35:21.249 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: gps base set to 2022-02-06 (week 2196) 2024-08-19T10:35:21.249 INFO:teuthology.orchestra.run.smithi190.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-19T10:35:21.249 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-19T10:35:21.250 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: restrict ::: KOD does nothing without LIMITED. 2024-08-19T10:35:21.250 INFO:teuthology.orchestra.run.smithi190.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-19T10:35:21.251 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: Listen and drop on 0 v6wildcard [::]:123 2024-08-19T10:35:21.251 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-19T10:35:21.251 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: Listen normally on 2 lo 127.0.0.1:123 2024-08-19T10:35:21.251 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: Listen normally on 3 enp3s0f1 172.21.15.190:123 2024-08-19T10:35:21.251 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: Listen normally on 4 lo [::1]:123 2024-08-19T10:35:21.251 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7763%5]:123 2024-08-19T10:35:21.252 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:21 ntpd[18398]: Listening on routing socket on fd #22 for interface updates 2024-08-19T10:35:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:22 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:22 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:22 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:22 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:22 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:22 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:22.248 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:22 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:22.248 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:22 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:22.248 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:22 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:22.248 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:22 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:22.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:22 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:22.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:22 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:22.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:22 ntpd[18398]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:22.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:22 ntpd[18398]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:23.215 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:23 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:23.215 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:23 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:24.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:24 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:24.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:24 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:24.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:24 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:24.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:24 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:24.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:24 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:24.216 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:24 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:24.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:24 ntpd[18398]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:26.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:26.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:26 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:26.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:26.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:26.251 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:26.251 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:26.251 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:26 ntpd[18398]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:28.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.217 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:28.218 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:28.219 INFO:teuthology.orchestra.run.smithi089.stdout:19 Aug 10:35:28 ntpd[18394]: ntpd: time slew -0.002566 s 2024-08-19T10:35:28.219 INFO:teuthology.orchestra.run.smithi089.stdout:ntpd: time slew -0.002566s 2024-08-19T10:35:28.219 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-19T10:35:28.219 INFO:teuthology.orchestra.run.smithi089.stderr:19 Aug 10:35:28 ntpd[18394]: can't open /var/log/ntpstats/loopstats.20240819: Permission denied 2024-08-19T10:35:28.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.249 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:28.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:28.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:28.250 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:28.251 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-19T10:35:28.251 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: can't open /var/log/ntpstats/rawstats.20240819: Permission denied 2024-08-19T10:35:28.251 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-19T10:35:28.251 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: can't open /var/log/ntpstats/peerstats.20240819: Permission denied 2024-08-19T10:35:28.251 INFO:teuthology.orchestra.run.smithi190.stdout:19 Aug 10:35:28 ntpd[18398]: ntpd: time slew -0.002123 s 2024-08-19T10:35:28.251 INFO:teuthology.orchestra.run.smithi190.stdout:ntpd: time slew -0.002123s 2024-08-19T10:35:28.252 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-19T10:35:28.252 INFO:teuthology.orchestra.run.smithi190.stderr:19 Aug 10:35:28 ntpd[18398]: can't open /var/log/ntpstats/loopstats.20240819: Permission denied 2024-08-19T10:35:28.298 INFO:teuthology.orchestra.run.smithi190.stdout: remote refid st t when poll reach delay offset jitter 2024-08-19T10:35:28.298 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================== 2024-08-19T10:35:28.298 INFO:teuthology.orchestra.run.smithi190.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-08-19T10:35:28.298 INFO:teuthology.orchestra.run.smithi190.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-08-19T10:35:28.298 INFO:teuthology.orchestra.run.smithi190.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-08-19T10:35:28.298 INFO:teuthology.orchestra.run.smithi190.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-08-19T10:35:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: remote refid st t when poll reach delay offset jitter 2024-08-19T10:35:28.333 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================== 2024-08-19T10:35:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-08-19T10:35:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-08-19T10:35:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-08-19T10:35:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-08-19T10:35:28.334 INFO:teuthology.run_tasks:Running task install... 2024-08-19T10:35:28.342 DEBUG:teuthology.task.install:project ceph 2024-08-19T10:35:28.342 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '797669399b32d2915e64eb6d4b1257ae3c50f80b'}} 2024-08-19T10:35:28.343 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '797669399b32d2915e64eb6d4b1257ae3c50f80b'} 2024-08-19T10:35:28.343 INFO:teuthology.task.install:Using flavor: default 2024-08-19T10:35:28.349 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-19T10:35:28.349 INFO:teuthology.task.install:extra packages: [] 2024-08-19T10:35:28.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-key list | grep Ceph 2024-08-19T10:35:28.350 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-key list | grep Ceph 2024-08-19T10:35:28.408 INFO:teuthology.orchestra.run.smithi190.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-08-19T10:35:28.432 INFO:teuthology.orchestra.run.smithi089.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-08-19T10:35:28.436 INFO:teuthology.orchestra.run.smithi190.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-19T10:35:28.436 INFO:teuthology.orchestra.run.smithi190.stdout:uid [ unknown] Ceph.com (release key) 2024-08-19T10:35:28.437 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-08-19T10:35:28.438 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=797669399b32d2915e64eb6d4b1257ae3c50f80b 2024-08-19T10:35:28.459 INFO:teuthology.orchestra.run.smithi089.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-19T10:35:28.459 INFO:teuthology.orchestra.run.smithi089.stdout:uid [ unknown] Ceph.com (release key) 2024-08-19T10:35:28.459 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-08-19T10:35:28.460 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=797669399b32d2915e64eb6d4b1257ae3c50f80b 2024-08-19T10:35:28.592 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default/ 2024-08-19T10:35:28.592 INFO:teuthology.task.install.deb:Package version is 19.0.0-5847-g79766939-1jammy 2024-08-19T10:35:28.609 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default/ 2024-08-19T10:35:28.609 INFO:teuthology.task.install.deb:Package version is 19.0.0-5847-g79766939-1jammy 2024-08-19T10:35:28.712 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:35:28.712 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-19T10:35:28.727 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:35:28.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-19T10:35:28.728 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2024-08-19T10:35:28.741 DEBUG:teuthology.orchestra.run.smithi089:> sudo apt-get update 2024-08-19T10:35:28.894 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-08-19T10:35:28.906 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-08-19T10:35:28.909 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-08-19T10:35:28.918 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-08-19T10:35:28.922 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-08-19T10:35:28.935 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-08-19T10:35:28.950 INFO:teuthology.orchestra.run.smithi190.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy InRelease 2024-08-19T10:35:28.962 INFO:teuthology.orchestra.run.smithi089.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy InRelease 2024-08-19T10:35:28.971 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-08-19T10:35:28.984 INFO:teuthology.orchestra.run.smithi089.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-08-19T10:35:28.995 INFO:teuthology.orchestra.run.smithi190.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy Release.gpg 2024-08-19T10:35:29.012 INFO:teuthology.orchestra.run.smithi089.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy Release.gpg 2024-08-19T10:35:29.016 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,422 B] 2024-08-19T10:35:29.021 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-08-19T10:35:29.029 INFO:teuthology.orchestra.run.smithi089.stdout:Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-08-19T10:35:29.033 INFO:teuthology.orchestra.run.smithi089.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,422 B] 2024-08-19T10:35:29.043 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-08-19T10:35:29.060 INFO:teuthology.orchestra.run.smithi089.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-08-19T10:35:29.236 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 28.6 kB in 0s (86.6 kB/s) 2024-08-19T10:35:29.274 INFO:teuthology.orchestra.run.smithi089.stdout:Fetched 28.6 kB in 0s (82.5 kB/s) 2024-08-19T10:35:30.245 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-08-19T10:35:30.261 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-5847-g79766939-1jammy cephadm=19.0.0-5847-g79766939-1jammy ceph-mds=19.0.0-5847-g79766939-1jammy ceph-mgr=19.0.0-5847-g79766939-1jammy ceph-common=19.0.0-5847-g79766939-1jammy ceph-fuse=19.0.0-5847-g79766939-1jammy ceph-test=19.0.0-5847-g79766939-1jammy ceph-volume=19.0.0-5847-g79766939-1jammy radosgw=19.0.0-5847-g79766939-1jammy python3-rados=19.0.0-5847-g79766939-1jammy python3-rgw=19.0.0-5847-g79766939-1jammy python3-cephfs=19.0.0-5847-g79766939-1jammy python3-rbd=19.0.0-5847-g79766939-1jammy libcephfs2=19.0.0-5847-g79766939-1jammy libcephfs-dev=19.0.0-5847-g79766939-1jammy librados2=19.0.0-5847-g79766939-1jammy librbd1=19.0.0-5847-g79766939-1jammy rbd-fuse=19.0.0-5847-g79766939-1jammy 2024-08-19T10:35:30.297 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-08-19T10:35:30.313 DEBUG:teuthology.orchestra.run.smithi089:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-5847-g79766939-1jammy cephadm=19.0.0-5847-g79766939-1jammy ceph-mds=19.0.0-5847-g79766939-1jammy ceph-mgr=19.0.0-5847-g79766939-1jammy ceph-common=19.0.0-5847-g79766939-1jammy ceph-fuse=19.0.0-5847-g79766939-1jammy ceph-test=19.0.0-5847-g79766939-1jammy ceph-volume=19.0.0-5847-g79766939-1jammy radosgw=19.0.0-5847-g79766939-1jammy python3-rados=19.0.0-5847-g79766939-1jammy python3-rgw=19.0.0-5847-g79766939-1jammy python3-cephfs=19.0.0-5847-g79766939-1jammy python3-rbd=19.0.0-5847-g79766939-1jammy libcephfs2=19.0.0-5847-g79766939-1jammy libcephfs-dev=19.0.0-5847-g79766939-1jammy librados2=19.0.0-5847-g79766939-1jammy librbd1=19.0.0-5847-g79766939-1jammy rbd-fuse=19.0.0-5847-g79766939-1jammy 2024-08-19T10:35:30.323 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-08-19T10:35:30.366 INFO:teuthology.orchestra.run.smithi089.stdout:Reading package lists... 2024-08-19T10:35:30.532 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2024-08-19T10:35:30.532 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2024-08-19T10:35:30.550 INFO:teuthology.orchestra.run.smithi089.stdout:Building dependency tree... 2024-08-19T10:35:30.550 INFO:teuthology.orchestra.run.smithi089.stdout:Reading state information... 2024-08-19T10:35:30.726 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2024-08-19T10:35:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-08-19T10:35:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-08-19T10:35:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-08-19T10:35:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-08-19T10:35:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-08-19T10:35:30.727 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-19T10:35:30.727 INFO:teuthology.orchestra.run.smithi190.stdout:The following additional packages will be installed: 2024-08-19T10:35:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-19T10:35:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-08-19T10:35:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-08-19T10:35:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-08-19T10:35:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-08-19T10:35:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-08-19T10:35:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-08-19T10:35:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-08-19T10:35:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-08-19T10:35:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-08-19T10:35:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-08-19T10:35:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-08-19T10:35:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-08-19T10:35:30.730 INFO:teuthology.orchestra.run.smithi190.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-08-19T10:35:30.730 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-08-19T10:35:30.730 INFO:teuthology.orchestra.run.smithi190.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-08-19T10:35:30.730 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-08-19T10:35:30.730 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-08-19T10:35:30.730 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-08-19T10:35:30.730 INFO:teuthology.orchestra.run.smithi190.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-08-19T10:35:30.730 INFO:teuthology.orchestra.run.smithi190.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-08-19T10:35:30.730 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-08-19T10:35:30.730 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-08-19T10:35:30.731 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet zip 2024-08-19T10:35:30.731 INFO:teuthology.orchestra.run.smithi190.stdout:Suggested packages: 2024-08-19T10:35:30.731 INFO:teuthology.orchestra.run.smithi190.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-08-19T10:35:30.731 INFO:teuthology.orchestra.run.smithi190.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-08-19T10:35:30.731 INFO:teuthology.orchestra.run.smithi190.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-08-19T10:35:30.731 INFO:teuthology.orchestra.run.smithi190.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-08-19T10:35:30.731 INFO:teuthology.orchestra.run.smithi190.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-08-19T10:35:30.731 INFO:teuthology.orchestra.run.smithi190.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-19T10:35:30.731 INFO:teuthology.orchestra.run.smithi190.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-08-19T10:35:30.732 INFO:teuthology.orchestra.run.smithi190.stdout:Recommended packages: 2024-08-19T10:35:30.732 INFO:teuthology.orchestra.run.smithi190.stdout: btrfs-tools 2024-08-19T10:35:30.733 INFO:teuthology.orchestra.run.smithi089.stdout:The following packages were automatically installed and are no longer required: 2024-08-19T10:35:30.733 INFO:teuthology.orchestra.run.smithi089.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-08-19T10:35:30.733 INFO:teuthology.orchestra.run.smithi089.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-08-19T10:35:30.733 INFO:teuthology.orchestra.run.smithi089.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-08-19T10:35:30.733 INFO:teuthology.orchestra.run.smithi089.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-08-19T10:35:30.734 INFO:teuthology.orchestra.run.smithi089.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-08-19T10:35:30.734 INFO:teuthology.orchestra.run.smithi089.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-19T10:35:30.734 INFO:teuthology.orchestra.run.smithi089.stdout:The following additional packages will be installed: 2024-08-19T10:35:30.734 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-19T10:35:30.735 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-08-19T10:35:30.735 INFO:teuthology.orchestra.run.smithi089.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-08-19T10:35:30.736 INFO:teuthology.orchestra.run.smithi089.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-08-19T10:35:30.737 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-08-19T10:35:30.737 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-08-19T10:35:30.737 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-08-19T10:35:30.737 INFO:teuthology.orchestra.run.smithi089.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-08-19T10:35:30.737 INFO:teuthology.orchestra.run.smithi089.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-08-19T10:35:30.737 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-08-19T10:35:30.737 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-08-19T10:35:30.737 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet zip 2024-08-19T10:35:30.737 INFO:teuthology.orchestra.run.smithi089.stdout:Suggested packages: 2024-08-19T10:35:30.737 INFO:teuthology.orchestra.run.smithi089.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-08-19T10:35:30.738 INFO:teuthology.orchestra.run.smithi089.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-08-19T10:35:30.738 INFO:teuthology.orchestra.run.smithi089.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-08-19T10:35:30.738 INFO:teuthology.orchestra.run.smithi089.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-08-19T10:35:30.738 INFO:teuthology.orchestra.run.smithi089.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-08-19T10:35:30.738 INFO:teuthology.orchestra.run.smithi089.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-19T10:35:30.738 INFO:teuthology.orchestra.run.smithi089.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-08-19T10:35:30.738 INFO:teuthology.orchestra.run.smithi089.stdout:Recommended packages: 2024-08-19T10:35:30.738 INFO:teuthology.orchestra.run.smithi089.stdout: btrfs-tools 2024-08-19T10:35:30.832 INFO:teuthology.orchestra.run.smithi190.stdout:The following NEW packages will be installed: 2024-08-19T10:35:30.832 INFO:teuthology.orchestra.run.smithi190.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-19T10:35:30.832 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-19T10:35:30.832 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-08-19T10:35:30.832 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-08-19T10:35:30.832 INFO:teuthology.orchestra.run.smithi190.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-08-19T10:35:30.833 INFO:teuthology.orchestra.run.smithi190.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-08-19T10:35:30.833 INFO:teuthology.orchestra.run.smithi190.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-08-19T10:35:30.833 INFO:teuthology.orchestra.run.smithi190.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-08-19T10:35:30.833 INFO:teuthology.orchestra.run.smithi190.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-08-19T10:35:30.833 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-08-19T10:35:30.833 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-08-19T10:35:30.833 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-08-19T10:35:30.833 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-08-19T10:35:30.833 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-08-19T10:35:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-08-19T10:35:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-08-19T10:35:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-08-19T10:35:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-08-19T10:35:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-08-19T10:35:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-08-19T10:35:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-08-19T10:35:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-08-19T10:35:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-08-19T10:35:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-08-19T10:35:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-08-19T10:35:30.835 INFO:teuthology.orchestra.run.smithi190.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-08-19T10:35:30.836 INFO:teuthology.orchestra.run.smithi089.stdout:The following NEW packages will be installed: 2024-08-19T10:35:30.837 INFO:teuthology.orchestra.run.smithi089.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-19T10:35:30.837 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-19T10:35:30.837 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-08-19T10:35:30.837 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-08-19T10:35:30.837 INFO:teuthology.orchestra.run.smithi089.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-08-19T10:35:30.837 INFO:teuthology.orchestra.run.smithi089.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-08-19T10:35:30.838 INFO:teuthology.orchestra.run.smithi089.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-08-19T10:35:30.839 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-08-19T10:35:30.840 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-08-19T10:35:30.840 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-08-19T10:35:30.840 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-08-19T10:35:30.840 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-08-19T10:35:30.840 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-08-19T10:35:30.840 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-08-19T10:35:30.840 INFO:teuthology.orchestra.run.smithi089.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-08-19T10:35:30.897 INFO:teuthology.orchestra.run.smithi089.stdout:0 upgraded, 118 newly installed, 0 to remove and 320 not upgraded. 2024-08-19T10:35:30.897 INFO:teuthology.orchestra.run.smithi089.stdout:Need to get 251 MB of archives. 2024-08-19T10:35:30.897 INFO:teuthology.orchestra.run.smithi089.stdout:After this operation, 932 MB of additional disk space will be used. 2024-08-19T10:35:30.897 INFO:teuthology.orchestra.run.smithi089.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-08-19T10:35:30.970 INFO:teuthology.orchestra.run.smithi190.stdout:0 upgraded, 118 newly installed, 0 to remove and 320 not upgraded. 2024-08-19T10:35:30.970 INFO:teuthology.orchestra.run.smithi190.stdout:Need to get 251 MB of archives. 2024-08-19T10:35:30.970 INFO:teuthology.orchestra.run.smithi190.stdout:After this operation, 932 MB of additional disk space will be used. 2024-08-19T10:35:30.970 INFO:teuthology.orchestra.run.smithi190.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-5847-g79766939-1jammy [3,632 kB] 2024-08-19T10:35:30.979 INFO:teuthology.orchestra.run.smithi089.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-5847-g79766939-1jammy [3,632 kB] 2024-08-19T10:35:30.985 INFO:teuthology.orchestra.run.smithi089.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-08-19T10:35:30.988 INFO:teuthology.orchestra.run.smithi089.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-08-19T10:35:31.005 INFO:teuthology.orchestra.run.smithi089.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-08-19T10:35:31.019 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-08-19T10:35:31.063 INFO:teuthology.orchestra.run.smithi089.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-08-19T10:35:31.068 INFO:teuthology.orchestra.run.smithi089.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-08-19T10:35:31.081 INFO:teuthology.orchestra.run.smithi089.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-08-19T10:35:31.085 INFO:teuthology.orchestra.run.smithi089.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-08-19T10:35:31.086 INFO:teuthology.orchestra.run.smithi089.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-08-19T10:35:31.086 INFO:teuthology.orchestra.run.smithi089.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-08-19T10:35:31.095 INFO:teuthology.orchestra.run.smithi089.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-08-19T10:35:31.097 INFO:teuthology.orchestra.run.smithi089.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-08-19T10:35:31.099 INFO:teuthology.orchestra.run.smithi089.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-08-19T10:35:31.101 INFO:teuthology.orchestra.run.smithi089.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-08-19T10:35:31.102 INFO:teuthology.orchestra.run.smithi089.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-08-19T10:35:31.106 INFO:teuthology.orchestra.run.smithi089.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-08-19T10:35:31.118 INFO:teuthology.orchestra.run.smithi089.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-08-19T10:35:31.120 INFO:teuthology.orchestra.run.smithi089.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-08-19T10:35:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-08-19T10:35:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-08-19T10:35:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-08-19T10:35:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-08-19T10:35:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-08-19T10:35:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-08-19T10:35:31.137 INFO:teuthology.orchestra.run.smithi089.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-08-19T10:35:31.137 INFO:teuthology.orchestra.run.smithi089.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-08-19T10:35:31.154 INFO:teuthology.orchestra.run.smithi089.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-08-19T10:35:31.156 INFO:teuthology.orchestra.run.smithi089.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-08-19T10:35:31.157 INFO:teuthology.orchestra.run.smithi089.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-08-19T10:35:31.157 INFO:teuthology.orchestra.run.smithi089.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-08-19T10:35:31.158 INFO:teuthology.orchestra.run.smithi089.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-08-19T10:35:31.159 INFO:teuthology.orchestra.run.smithi089.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-19T10:35:31.159 INFO:teuthology.orchestra.run.smithi089.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-08-19T10:35:31.159 INFO:teuthology.orchestra.run.smithi089.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-08-19T10:35:31.171 INFO:teuthology.orchestra.run.smithi089.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-08-19T10:35:31.172 INFO:teuthology.orchestra.run.smithi089.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-08-19T10:35:31.173 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-5847-g79766939-1jammy [3,313 kB] 2024-08-19T10:35:31.188 INFO:teuthology.orchestra.run.smithi089.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-08-19T10:35:31.194 INFO:teuthology.orchestra.run.smithi089.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-08-19T10:35:31.194 INFO:teuthology.orchestra.run.smithi089.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-08-19T10:35:31.194 INFO:teuthology.orchestra.run.smithi089.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-08-19T10:35:31.195 INFO:teuthology.orchestra.run.smithi089.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-08-19T10:35:31.195 INFO:teuthology.orchestra.run.smithi089.stdout:Get:43 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-5847-g79766939-1jammy [3,313 kB] 2024-08-19T10:35:31.196 INFO:teuthology.orchestra.run.smithi089.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-08-19T10:35:31.198 INFO:teuthology.orchestra.run.smithi089.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-08-19T10:35:31.199 INFO:teuthology.orchestra.run.smithi089.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-08-19T10:35:31.210 INFO:teuthology.orchestra.run.smithi089.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-08-19T10:35:31.211 INFO:teuthology.orchestra.run.smithi089.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-08-19T10:35:31.228 INFO:teuthology.orchestra.run.smithi089.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-08-19T10:35:31.228 INFO:teuthology.orchestra.run.smithi089.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-08-19T10:35:31.229 INFO:teuthology.orchestra.run.smithi089.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-08-19T10:35:31.232 INFO:teuthology.orchestra.run.smithi089.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-08-19T10:35:31.254 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-5847-g79766939-1jammy [721 kB] 2024-08-19T10:35:31.284 INFO:teuthology.orchestra.run.smithi089.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-08-19T10:35:31.285 INFO:teuthology.orchestra.run.smithi089.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-08-19T10:35:31.286 INFO:teuthology.orchestra.run.smithi089.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-19T10:35:31.287 INFO:teuthology.orchestra.run.smithi089.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-08-19T10:35:31.288 INFO:teuthology.orchestra.run.smithi089.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-08-19T10:35:31.288 INFO:teuthology.orchestra.run.smithi089.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-5847-g79766939-1jammy [721 kB] 2024-08-19T10:35:31.296 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-5847-g79766939-1jammy [356 kB] 2024-08-19T10:35:31.302 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-5847-g79766939-1jammy [32.8 kB] 2024-08-19T10:35:31.302 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-5847-g79766939-1jammy [180 kB] 2024-08-19T10:35:31.305 INFO:teuthology.orchestra.run.smithi089.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-08-19T10:35:31.306 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-5847-g79766939-1jammy [68.9 kB] 2024-08-19T10:35:31.306 INFO:teuthology.orchestra.run.smithi089.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-08-19T10:35:31.307 INFO:teuthology.orchestra.run.smithi089.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-08-19T10:35:31.307 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-5847-g79766939-1jammy [338 kB] 2024-08-19T10:35:31.313 INFO:teuthology.orchestra.run.smithi190.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-5847-g79766939-1jammy [8,017 kB] 2024-08-19T10:35:31.321 INFO:teuthology.orchestra.run.smithi089.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-08-19T10:35:31.321 INFO:teuthology.orchestra.run.smithi089.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-08-19T10:35:31.322 INFO:teuthology.orchestra.run.smithi089.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-08-19T10:35:31.322 INFO:teuthology.orchestra.run.smithi089.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-08-19T10:35:31.322 INFO:teuthology.orchestra.run.smithi089.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-08-19T10:35:31.323 INFO:teuthology.orchestra.run.smithi089.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-08-19T10:35:31.331 INFO:teuthology.orchestra.run.smithi089.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-5847-g79766939-1jammy [356 kB] 2024-08-19T10:35:31.338 INFO:teuthology.orchestra.run.smithi089.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-08-19T10:35:31.338 INFO:teuthology.orchestra.run.smithi089.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-5847-g79766939-1jammy [32.8 kB] 2024-08-19T10:35:31.339 INFO:teuthology.orchestra.run.smithi089.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-08-19T10:35:31.339 INFO:teuthology.orchestra.run.smithi089.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-5847-g79766939-1jammy [180 kB] 2024-08-19T10:35:31.341 INFO:teuthology.orchestra.run.smithi089.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-08-19T10:35:31.342 INFO:teuthology.orchestra.run.smithi089.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-5847-g79766939-1jammy [68.9 kB] 2024-08-19T10:35:31.344 INFO:teuthology.orchestra.run.smithi089.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-5847-g79766939-1jammy [338 kB] 2024-08-19T10:35:31.350 INFO:teuthology.orchestra.run.smithi089.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-5847-g79766939-1jammy [8,017 kB] 2024-08-19T10:35:31.355 INFO:teuthology.orchestra.run.smithi089.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-08-19T10:35:31.356 INFO:teuthology.orchestra.run.smithi089.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-08-19T10:35:31.357 INFO:teuthology.orchestra.run.smithi089.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-08-19T10:35:31.360 INFO:teuthology.orchestra.run.smithi089.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-08-19T10:35:31.362 INFO:teuthology.orchestra.run.smithi089.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-08-19T10:35:31.363 INFO:teuthology.orchestra.run.smithi089.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-08-19T10:35:31.363 INFO:teuthology.orchestra.run.smithi089.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-08-19T10:35:31.372 INFO:teuthology.orchestra.run.smithi089.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-08-19T10:35:31.373 INFO:teuthology.orchestra.run.smithi089.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-08-19T10:35:31.375 INFO:teuthology.orchestra.run.smithi089.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-08-19T10:35:31.389 INFO:teuthology.orchestra.run.smithi089.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-19T10:35:31.390 INFO:teuthology.orchestra.run.smithi089.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-08-19T10:35:31.390 INFO:teuthology.orchestra.run.smithi089.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-08-19T10:35:31.391 INFO:teuthology.orchestra.run.smithi089.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-08-19T10:35:31.392 INFO:teuthology.orchestra.run.smithi089.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-08-19T10:35:31.398 INFO:teuthology.orchestra.run.smithi089.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-08-19T10:35:31.398 INFO:teuthology.orchestra.run.smithi089.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-08-19T10:35:31.406 INFO:teuthology.orchestra.run.smithi089.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-08-19T10:35:31.408 INFO:teuthology.orchestra.run.smithi089.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-08-19T10:35:31.409 INFO:teuthology.orchestra.run.smithi089.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-08-19T10:35:31.441 INFO:teuthology.orchestra.run.smithi190.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-08-19T10:35:31.453 INFO:teuthology.orchestra.run.smithi190.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-08-19T10:35:31.535 INFO:teuthology.orchestra.run.smithi190.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-08-19T10:35:31.558 INFO:teuthology.orchestra.run.smithi190.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-5847-g79766939-1jammy [112 kB] 2024-08-19T10:35:31.560 INFO:teuthology.orchestra.run.smithi190.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-5847-g79766939-1jammy [472 kB] 2024-08-19T10:35:31.665 INFO:teuthology.orchestra.run.smithi190.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-5847-g79766939-1jammy [28.3 MB] 2024-08-19T10:35:31.685 INFO:teuthology.orchestra.run.smithi089.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-5847-g79766939-1jammy [112 kB] 2024-08-19T10:35:31.687 INFO:teuthology.orchestra.run.smithi089.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-5847-g79766939-1jammy [472 kB] 2024-08-19T10:35:31.696 INFO:teuthology.orchestra.run.smithi089.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-5847-g79766939-1jammy [28.3 MB] 2024-08-19T10:35:31.764 INFO:teuthology.orchestra.run.smithi190.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-08-19T10:35:31.774 INFO:teuthology.orchestra.run.smithi190.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-08-19T10:35:31.804 INFO:teuthology.orchestra.run.smithi190.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-08-19T10:35:31.813 INFO:teuthology.orchestra.run.smithi190.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-08-19T10:35:31.814 INFO:teuthology.orchestra.run.smithi190.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-08-19T10:35:31.815 INFO:teuthology.orchestra.run.smithi190.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-08-19T10:35:31.833 INFO:teuthology.orchestra.run.smithi190.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-08-19T10:35:31.837 INFO:teuthology.orchestra.run.smithi190.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-08-19T10:35:31.841 INFO:teuthology.orchestra.run.smithi190.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-08-19T10:35:31.846 INFO:teuthology.orchestra.run.smithi190.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-08-19T10:35:31.926 INFO:teuthology.orchestra.run.smithi190.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-08-19T10:35:31.930 INFO:teuthology.orchestra.run.smithi190.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-08-19T10:35:31.934 INFO:teuthology.orchestra.run.smithi190.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-08-19T10:35:31.937 INFO:teuthology.orchestra.run.smithi190.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-08-19T10:35:31.938 INFO:teuthology.orchestra.run.smithi190.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-08-19T10:35:31.938 INFO:teuthology.orchestra.run.smithi190.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-08-19T10:35:31.940 INFO:teuthology.orchestra.run.smithi190.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-08-19T10:35:31.941 INFO:teuthology.orchestra.run.smithi190.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-08-19T10:35:31.942 INFO:teuthology.orchestra.run.smithi190.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-08-19T10:35:32.009 INFO:teuthology.orchestra.run.smithi190.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-08-19T10:35:32.024 INFO:teuthology.orchestra.run.smithi190.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-08-19T10:35:32.025 INFO:teuthology.orchestra.run.smithi190.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-08-19T10:35:32.025 INFO:teuthology.orchestra.run.smithi190.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-08-19T10:35:32.031 INFO:teuthology.orchestra.run.smithi190.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-08-19T10:35:32.032 INFO:teuthology.orchestra.run.smithi190.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-08-19T10:35:32.091 INFO:teuthology.orchestra.run.smithi190.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-08-19T10:35:32.092 INFO:teuthology.orchestra.run.smithi190.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-08-19T10:35:32.094 INFO:teuthology.orchestra.run.smithi190.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-19T10:35:32.094 INFO:teuthology.orchestra.run.smithi190.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-08-19T10:35:32.172 INFO:teuthology.orchestra.run.smithi190.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-08-19T10:35:32.174 INFO:teuthology.orchestra.run.smithi190.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-08-19T10:35:32.176 INFO:teuthology.orchestra.run.smithi190.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-08-19T10:35:32.177 INFO:teuthology.orchestra.run.smithi190.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-08-19T10:35:32.188 INFO:teuthology.orchestra.run.smithi190.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-08-19T10:35:32.254 INFO:teuthology.orchestra.run.smithi190.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-08-19T10:35:32.254 INFO:teuthology.orchestra.run.smithi190.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-08-19T10:35:32.255 INFO:teuthology.orchestra.run.smithi190.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-08-19T10:35:32.258 INFO:teuthology.orchestra.run.smithi190.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-08-19T10:35:32.262 INFO:teuthology.orchestra.run.smithi190.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-08-19T10:35:32.335 INFO:teuthology.orchestra.run.smithi190.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-08-19T10:35:32.337 INFO:teuthology.orchestra.run.smithi190.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-08-19T10:35:32.339 INFO:teuthology.orchestra.run.smithi190.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-08-19T10:35:32.344 INFO:teuthology.orchestra.run.smithi190.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-08-19T10:35:32.345 INFO:teuthology.orchestra.run.smithi190.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-08-19T10:35:32.417 INFO:teuthology.orchestra.run.smithi190.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-08-19T10:35:32.421 INFO:teuthology.orchestra.run.smithi190.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-08-19T10:35:32.568 INFO:teuthology.orchestra.run.smithi190.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-5847-g79766939-1jammy [5,198 kB] 2024-08-19T10:35:32.584 INFO:teuthology.orchestra.run.smithi190.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-08-19T10:35:32.585 INFO:teuthology.orchestra.run.smithi190.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-08-19T10:35:32.587 INFO:teuthology.orchestra.run.smithi190.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-19T10:35:32.588 INFO:teuthology.orchestra.run.smithi190.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-08-19T10:35:32.590 INFO:teuthology.orchestra.run.smithi190.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-08-19T10:35:32.623 INFO:teuthology.orchestra.run.smithi190.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-08-19T10:35:32.625 INFO:teuthology.orchestra.run.smithi190.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-08-19T10:35:32.626 INFO:teuthology.orchestra.run.smithi190.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-08-19T10:35:32.649 INFO:teuthology.orchestra.run.smithi190.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-5847-g79766939-1jammy [247 kB] 2024-08-19T10:35:32.652 INFO:teuthology.orchestra.run.smithi190.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-5847-g79766939-1jammy [124 kB] 2024-08-19T10:35:32.654 INFO:teuthology.orchestra.run.smithi190.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-5847-g79766939-1jammy [1,560 kB] 2024-08-19T10:35:32.686 INFO:teuthology.orchestra.run.smithi190.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-08-19T10:35:32.686 INFO:teuthology.orchestra.run.smithi190.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-08-19T10:35:32.692 INFO:teuthology.orchestra.run.smithi190.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-5847-g79766939-1jammy [6,293 kB] 2024-08-19T10:35:32.706 INFO:teuthology.orchestra.run.smithi089.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-5847-g79766939-1jammy [5,198 kB] 2024-08-19T10:35:32.750 INFO:teuthology.orchestra.run.smithi190.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-08-19T10:35:32.751 INFO:teuthology.orchestra.run.smithi190.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-08-19T10:35:32.752 INFO:teuthology.orchestra.run.smithi190.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-08-19T10:35:32.753 INFO:teuthology.orchestra.run.smithi190.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-08-19T10:35:32.754 INFO:teuthology.orchestra.run.smithi190.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-08-19T10:35:32.755 INFO:teuthology.orchestra.run.smithi190.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-08-19T10:35:32.762 INFO:teuthology.orchestra.run.smithi190.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-08-19T10:35:32.803 INFO:teuthology.orchestra.run.smithi089.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-5847-g79766939-1jammy [247 kB] 2024-08-19T10:35:32.828 INFO:teuthology.orchestra.run.smithi089.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-5847-g79766939-1jammy [124 kB] 2024-08-19T10:35:32.830 INFO:teuthology.orchestra.run.smithi089.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-5847-g79766939-1jammy [1,560 kB] 2024-08-19T10:35:32.832 INFO:teuthology.orchestra.run.smithi190.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-08-19T10:35:32.835 INFO:teuthology.orchestra.run.smithi190.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-08-19T10:35:32.836 INFO:teuthology.orchestra.run.smithi190.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-08-19T10:35:32.873 INFO:teuthology.orchestra.run.smithi190.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-5847-g79766939-1jammy [23.1 MB] 2024-08-19T10:35:32.880 INFO:teuthology.orchestra.run.smithi089.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-5847-g79766939-1jammy [6,293 kB] 2024-08-19T10:35:32.913 INFO:teuthology.orchestra.run.smithi190.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-08-19T10:35:32.920 INFO:teuthology.orchestra.run.smithi190.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-08-19T10:35:32.922 INFO:teuthology.orchestra.run.smithi190.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-08-19T10:35:32.924 INFO:teuthology.orchestra.run.smithi190.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-08-19T10:35:32.934 INFO:teuthology.orchestra.run.smithi190.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-08-19T10:35:32.936 INFO:teuthology.orchestra.run.smithi190.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-08-19T10:35:32.944 INFO:teuthology.orchestra.run.smithi190.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-08-19T10:35:32.994 INFO:teuthology.orchestra.run.smithi190.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-19T10:35:32.996 INFO:teuthology.orchestra.run.smithi190.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-08-19T10:35:32.997 INFO:teuthology.orchestra.run.smithi190.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-08-19T10:35:33.076 INFO:teuthology.orchestra.run.smithi190.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-08-19T10:35:33.078 INFO:teuthology.orchestra.run.smithi190.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-08-19T10:35:33.088 INFO:teuthology.orchestra.run.smithi089.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-5847-g79766939-1jammy [23.1 MB] 2024-08-19T10:35:33.095 INFO:teuthology.orchestra.run.smithi190.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-08-19T10:35:33.096 INFO:teuthology.orchestra.run.smithi190.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-08-19T10:35:33.097 INFO:teuthology.orchestra.run.smithi190.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-08-19T10:35:33.102 INFO:teuthology.orchestra.run.smithi190.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-08-19T10:35:33.104 INFO:teuthology.orchestra.run.smithi190.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-08-19T10:35:33.687 INFO:teuthology.orchestra.run.smithi190.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-5847-g79766939-1jammy [14.1 kB] 2024-08-19T10:35:33.688 INFO:teuthology.orchestra.run.smithi190.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-5847-g79766939-1jammy [880 kB] 2024-08-19T10:35:33.705 INFO:teuthology.orchestra.run.smithi190.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-5847-g79766939-1jammy [2,437 kB] 2024-08-19T10:35:33.745 INFO:teuthology.orchestra.run.smithi190.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-5847-g79766939-1jammy [801 kB] 2024-08-19T10:35:33.763 INFO:teuthology.orchestra.run.smithi190.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-5847-g79766939-1jammy [156 kB] 2024-08-19T10:35:33.765 INFO:teuthology.orchestra.run.smithi190.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-5847-g79766939-1jammy [61.3 MB] 2024-08-19T10:35:33.943 INFO:teuthology.orchestra.run.smithi089.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-5847-g79766939-1jammy [14.1 kB] 2024-08-19T10:35:33.944 INFO:teuthology.orchestra.run.smithi089.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-5847-g79766939-1jammy [880 kB] 2024-08-19T10:35:33.987 INFO:teuthology.orchestra.run.smithi089.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-5847-g79766939-1jammy [2,437 kB] 2024-08-19T10:35:34.032 INFO:teuthology.orchestra.run.smithi089.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-5847-g79766939-1jammy [801 kB] 2024-08-19T10:35:34.043 INFO:teuthology.orchestra.run.smithi089.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-5847-g79766939-1jammy [156 kB] 2024-08-19T10:35:34.046 INFO:teuthology.orchestra.run.smithi089.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-5847-g79766939-1jammy [61.3 MB] 2024-08-19T10:35:36.310 INFO:teuthology.orchestra.run.smithi190.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-5847-g79766939-1jammy [8,625 kB] 2024-08-19T10:35:36.560 INFO:teuthology.orchestra.run.smithi190.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-5847-g79766939-1jammy [14.2 kB] 2024-08-19T10:35:36.560 INFO:teuthology.orchestra.run.smithi190.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-5847-g79766939-1jammy [55.8 MB] 2024-08-19T10:35:38.006 INFO:teuthology.orchestra.run.smithi089.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-5847-g79766939-1jammy [8,625 kB] 2024-08-19T10:35:38.361 INFO:teuthology.orchestra.run.smithi089.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-5847-g79766939-1jammy [14.2 kB] 2024-08-19T10:35:38.362 INFO:teuthology.orchestra.run.smithi089.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-5847-g79766939-1jammy [55.8 MB] 2024-08-19T10:35:39.807 INFO:teuthology.orchestra.run.smithi190.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-5847-g79766939-1jammy [130 kB] 2024-08-19T10:35:39.810 INFO:teuthology.orchestra.run.smithi190.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-5847-g79766939-1jammy [38.9 kB] 2024-08-19T10:35:39.872 INFO:teuthology.orchestra.run.smithi190.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-5847-g79766939-1jammy [16.0 MB] 2024-08-19T10:35:40.702 INFO:teuthology.orchestra.run.smithi190.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-5847-g79766939-1jammy [91.4 kB] 2024-08-19T10:35:41.271 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 251 MB in 10s (25.4 MB/s) 2024-08-19T10:35:41.432 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-08-19T10:35:41.530 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-08-19T10:35:41.535 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-08-19T10:35:41.631 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-08-19T10:35:41.951 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-19T10:35:41.977 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-08-19T10:35:42.017 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-08-19T10:35:42.139 INFO:teuthology.orchestra.run.smithi089.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-5847-g79766939-1jammy [130 kB] 2024-08-19T10:35:42.142 INFO:teuthology.orchestra.run.smithi089.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-5847-g79766939-1jammy [38.9 kB] 2024-08-19T10:35:42.143 INFO:teuthology.orchestra.run.smithi089.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-5847-g79766939-1jammy [16.0 MB] 2024-08-19T10:35:42.353 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-19T10:35:42.379 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-08-19T10:35:42.420 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-08-19T10:35:42.823 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-19T10:35:42.849 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-08-19T10:35:42.868 INFO:teuthology.orchestra.run.smithi089.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/797669399b32d2915e64eb6d4b1257ae3c50f80b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-5847-g79766939-1jammy [91.4 kB] 2024-08-19T10:35:42.899 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:35:43.284 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-19T10:35:43.311 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-08-19T10:35:43.368 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:35:43.452 INFO:teuthology.orchestra.run.smithi089.stdout:Fetched 251 MB in 12s (20.9 MB/s) 2024-08-19T10:35:43.621 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-08-19T10:35:43.679 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-19T10:35:43.705 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-08-19T10:35:43.721 INFO:teuthology.orchestra.run.smithi089.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-08-19T10:35:43.727 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-08-19T10:35:43.745 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:35:43.811 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-08-19T10:35:44.048 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-08-19T10:35:44.061 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-08-19T10:35:44.098 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-08-19T10:35:44.157 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-19T10:35:44.183 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-08-19T10:35:44.223 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-08-19T10:35:44.559 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-19T10:35:44.586 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-08-19T10:35:44.626 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-08-19T10:35:44.634 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librados2. 2024-08-19T10:35:44.661 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../007-librados2_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:44.701 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librados2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:44.996 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-19T10:35:45.009 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-08-19T10:35:45.046 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:35:45.145 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librbd1. 2024-08-19T10:35:45.172 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../008-librbd1_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:45.212 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librbd1 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:45.408 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-19T10:35:45.434 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-08-19T10:35:45.466 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:35:45.615 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libcephfs2. 2024-08-19T10:35:45.630 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:45.664 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libcephfs2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:45.811 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-19T10:35:45.837 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-08-19T10:35:45.877 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:35:45.917 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rados. 2024-08-19T10:35:45.932 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../010-python3-rados_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:45.975 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rados (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:46.247 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-08-19T10:35:46.273 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-08-19T10:35:46.294 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-19T10:35:46.313 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-08-19T10:35:46.321 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:35:46.361 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-ceph-argparse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:46.613 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cephfs. 2024-08-19T10:35:46.617 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librados2. 2024-08-19T10:35:46.628 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:46.643 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../007-librados2_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:46.672 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cephfs (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:46.675 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librados2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:46.991 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-19T10:35:47.017 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:35:47.058 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-ceph-common (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:47.119 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librbd1. 2024-08-19T10:35:47.146 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../008-librbd1_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:47.186 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librbd1 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:47.369 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-19T10:35:47.395 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-08-19T10:35:47.435 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-08-19T10:35:47.639 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libcephfs2. 2024-08-19T10:35:47.665 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:47.705 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libcephfs2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:47.746 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-prettytable. 2024-08-19T10:35:47.773 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-08-19T10:35:47.813 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-08-19T10:35:47.999 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rados. 2024-08-19T10:35:48.014 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../010-python3-rados_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:48.024 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rbd. 2024-08-19T10:35:48.039 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:48.058 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rados (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:48.074 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rbd (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:48.352 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-19T10:35:48.367 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:35:48.402 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-ceph-argparse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:48.418 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-19T10:35:48.444 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-08-19T10:35:48.485 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-08-19T10:35:48.646 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cephfs. 2024-08-19T10:35:48.661 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:48.704 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cephfs (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:48.779 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-08-19T10:35:48.805 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-08-19T10:35:48.845 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-08-19T10:35:49.008 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-19T10:35:49.033 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:35:49.074 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-ceph-common (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:49.157 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-08-19T10:35:49.173 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-08-19T10:35:49.215 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-08-19T10:35:49.552 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua5.1. 2024-08-19T10:35:49.578 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-08-19T10:35:49.610 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-08-19T10:35:49.746 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-19T10:35:49.773 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-08-19T10:35:49.852 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-08-19T10:35:50.071 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua-any. 2024-08-19T10:35:50.086 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-08-19T10:35:50.129 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua-any (27ubuntu1) ... 2024-08-19T10:35:50.172 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-prettytable. 2024-08-19T10:35:50.187 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-08-19T10:35:50.238 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-08-19T10:35:50.374 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package zip. 2024-08-19T10:35:50.400 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-08-19T10:35:50.440 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking zip (3.0-12build2) ... 2024-08-19T10:35:50.558 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rbd. 2024-08-19T10:35:50.573 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:50.617 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rbd (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:50.727 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package unzip. 2024-08-19T10:35:50.741 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-08-19T10:35:50.776 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-08-19T10:35:50.987 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-19T10:35:51.013 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-08-19T10:35:51.062 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-08-19T10:35:51.122 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package luarocks. 2024-08-19T10:35:51.148 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-08-19T10:35:51.188 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-08-19T10:35:51.342 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-08-19T10:35:51.368 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-08-19T10:35:51.409 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-08-19T10:35:51.521 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librgw2. 2024-08-19T10:35:51.553 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../025-librgw2_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:51.581 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librgw2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:51.712 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-08-19T10:35:51.738 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-08-19T10:35:51.778 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-08-19T10:35:52.090 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package lua5.1. 2024-08-19T10:35:52.105 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-08-19T10:35:52.148 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-08-19T10:35:52.173 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rgw. 2024-08-19T10:35:52.199 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:52.248 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rgw (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:52.468 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package lua-any. 2024-08-19T10:35:52.494 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-08-19T10:35:52.534 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking lua-any (27ubuntu1) ... 2024-08-19T10:35:52.609 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liboath0:amd64. 2024-08-19T10:35:52.635 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-08-19T10:35:52.676 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-08-19T10:35:52.778 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package zip. 2024-08-19T10:35:52.794 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-08-19T10:35:52.837 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking zip (3.0-12build2) ... 2024-08-19T10:35:52.962 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libradosstriper1. 2024-08-19T10:35:52.988 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:53.028 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libradosstriper1 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:53.148 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package unzip. 2024-08-19T10:35:53.175 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-08-19T10:35:53.207 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-08-19T10:35:53.448 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-common. 2024-08-19T10:35:53.474 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../029-ceph-common_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:53.514 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-common (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:53.586 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package luarocks. 2024-08-19T10:35:53.612 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-08-19T10:35:53.652 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-08-19T10:35:54.010 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package librgw2. 2024-08-19T10:35:54.036 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../025-librgw2_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:54.077 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking librgw2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:54.841 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rgw. 2024-08-19T10:35:54.867 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:54.920 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rgw (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:54.953 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-base. 2024-08-19T10:35:54.979 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../030-ceph-base_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:55.029 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-base (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:55.274 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package liboath0:amd64. 2024-08-19T10:35:55.300 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-08-19T10:35:55.340 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-08-19T10:35:55.582 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-08-19T10:35:55.602 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libradosstriper1. 2024-08-19T10:35:55.608 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-08-19T10:35:55.628 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:55.648 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-08-19T10:35:55.668 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libradosstriper1 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:55.951 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cheroot. 2024-08-19T10:35:55.977 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-08-19T10:35:56.018 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-08-19T10:35:56.088 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-common. 2024-08-19T10:35:56.114 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../029-ceph-common_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:56.154 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-common (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:56.287 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-08-19T10:35:56.302 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-08-19T10:35:56.337 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-08-19T10:35:56.633 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.text. 2024-08-19T10:35:56.658 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-08-19T10:35:56.699 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-08-19T10:35:56.960 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-08-19T10:35:56.986 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-08-19T10:35:57.018 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-08-19T10:35:57.314 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-tempora. 2024-08-19T10:35:57.338 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-08-19T10:35:57.378 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-08-19T10:35:57.652 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-base. 2024-08-19T10:35:57.678 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../030-ceph-base_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:35:57.681 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-portend. 2024-08-19T10:35:57.707 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-08-19T10:35:57.727 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-base (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:35:57.750 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-08-19T10:35:58.019 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-08-19T10:35:58.045 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-08-19T10:35:58.078 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-08-19T10:35:58.314 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-08-19T10:35:58.340 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-08-19T10:35:58.380 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-19T10:35:58.381 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-08-19T10:35:58.407 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-08-19T10:35:58.447 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-08-19T10:35:58.675 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cheroot. 2024-08-19T10:35:58.701 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-08-19T10:35:58.734 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-08-19T10:35:58.800 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-natsort. 2024-08-19T10:35:58.827 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-08-19T10:35:58.867 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-08-19T10:35:59.037 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-08-19T10:35:59.064 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-08-19T10:35:59.103 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-08-19T10:35:59.145 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-logutils. 2024-08-19T10:35:59.160 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-08-19T10:35:59.203 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-08-19T10:35:59.406 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.text. 2024-08-19T10:35:59.433 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-08-19T10:35:59.473 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-08-19T10:35:59.506 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-19T10:35:59.532 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-08-19T10:35:59.572 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-08-19T10:35:59.784 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-08-19T10:35:59.811 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-08-19T10:35:59.851 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-08-19T10:35:59.875 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-mako. 2024-08-19T10:35:59.901 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-08-19T10:35:59.994 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-08-19T10:36:00.137 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-tempora. 2024-08-19T10:36:00.152 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-08-19T10:36:00.186 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-08-19T10:36:00.453 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-19T10:36:00.480 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-19T10:36:00.489 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-portend. 2024-08-19T10:36:00.516 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-08-19T10:36:00.519 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-19T10:36:00.556 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-08-19T10:36:00.797 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-19T10:36:00.823 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-08-19T10:36:00.856 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-08-19T10:36:00.876 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-08-19T10:36:00.903 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-08-19T10:36:00.942 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-08-19T10:36:01.183 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webob. 2024-08-19T10:36:01.210 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-08-19T10:36:01.222 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-19T10:36:01.247 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-08-19T10:36:01.258 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-08-19T10:36:01.287 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-08-19T10:36:01.611 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-waitress. 2024-08-19T10:36:01.638 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-08-19T10:36:01.641 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-natsort. 2024-08-19T10:36:01.667 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-08-19T10:36:01.681 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-08-19T10:36:01.707 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-08-19T10:36:01.989 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-tempita. 2024-08-19T10:36:01.994 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-logutils. 2024-08-19T10:36:02.012 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-08-19T10:36:02.020 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-08-19T10:36:02.047 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-08-19T10:36:02.060 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-08-19T10:36:02.325 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-paste. 2024-08-19T10:36:02.351 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-08-19T10:36:02.391 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-08-19T10:36:02.397 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-19T10:36:02.423 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-08-19T10:36:02.463 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-08-19T10:36:02.711 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-19T10:36:02.738 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-08-19T10:36:02.749 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-mako. 2024-08-19T10:36:02.776 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-08-19T10:36:02.778 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-08-19T10:36:02.816 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-08-19T10:36:03.030 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-19T10:36:03.057 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-08-19T10:36:03.097 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-08-19T10:36:03.119 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-19T10:36:03.146 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-19T10:36:03.186 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-19T10:36:03.400 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webtest. 2024-08-19T10:36:03.426 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-08-19T10:36:03.455 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-19T10:36:03.466 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-08-19T10:36:03.482 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-08-19T10:36:03.522 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-08-19T10:36:03.778 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pecan. 2024-08-19T10:36:03.804 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-08-19T10:36:03.824 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-webob. 2024-08-19T10:36:03.844 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-08-19T10:36:03.851 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-08-19T10:36:03.891 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-08-19T10:36:04.958 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-certifi. 2024-08-19T10:36:04.970 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-waitress. 2024-08-19T10:36:04.985 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-08-19T10:36:04.996 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-08-19T10:36:05.024 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-08-19T10:36:05.030 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-08-19T10:36:05.344 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-idna. 2024-08-19T10:36:05.371 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-08-19T10:36:05.418 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-08-19T10:36:05.573 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-tempita. 2024-08-19T10:36:05.599 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-08-19T10:36:05.639 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-08-19T10:36:05.705 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-urllib3. 2024-08-19T10:36:05.732 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-08-19T10:36:05.771 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-08-19T10:36:05.909 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-paste. 2024-08-19T10:36:05.935 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-08-19T10:36:05.966 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-08-19T10:36:06.066 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-requests. 2024-08-19T10:36:06.093 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-08-19T10:36:06.132 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-08-19T10:36:06.287 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-19T10:36:06.313 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-08-19T10:36:06.353 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-08-19T10:36:06.444 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-19T10:36:06.470 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-08-19T10:36:06.510 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-08-19T10:36:06.631 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-19T10:36:06.658 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-08-19T10:36:06.697 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-08-19T10:36:06.755 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-19T10:36:06.770 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:06.813 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-modules-core (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:07.017 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-webtest. 2024-08-19T10:36:07.044 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-08-19T10:36:07.084 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-08-19T10:36:07.176 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-19T10:36:07.203 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:07.243 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:07.398 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pecan. 2024-08-19T10:36:07.422 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-08-19T10:36:07.462 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-08-19T10:36:07.520 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-19T10:36:07.547 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-08-19T10:36:07.578 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-08-19T10:36:07.817 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-certifi. 2024-08-19T10:36:07.843 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-08-19T10:36:07.865 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-openssl. 2024-08-19T10:36:07.883 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-08-19T10:36:07.891 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-08-19T10:36:07.939 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-08-19T10:36:08.186 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-idna. 2024-08-19T10:36:08.213 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-08-19T10:36:08.253 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-08-19T10:36:08.276 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr. 2024-08-19T10:36:08.302 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:08.342 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:08.539 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-urllib3. 2024-08-19T10:36:08.566 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-08-19T10:36:08.605 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-08-19T10:36:08.729 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mon. 2024-08-19T10:36:08.755 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:08.795 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mon (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:08.909 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-requests. 2024-08-19T10:36:08.935 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-08-19T10:36:08.975 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-08-19T10:36:09.295 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-19T10:36:09.307 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-osd. 2024-08-19T10:36:09.321 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-08-19T10:36:09.333 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:09.361 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-08-19T10:36:09.373 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-osd (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:09.623 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-19T10:36:09.638 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:09.681 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-modules-core (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:10.061 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-19T10:36:10.088 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:10.127 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:10.397 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-19T10:36:10.401 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph. 2024-08-19T10:36:10.412 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-08-19T10:36:10.429 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../066-ceph_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:10.455 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-08-19T10:36:10.468 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:10.791 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-openssl. 2024-08-19T10:36:10.795 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-fuse. 2024-08-19T10:36:10.818 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-08-19T10:36:10.823 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:10.858 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-08-19T10:36:10.862 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-fuse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:11.186 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr. 2024-08-19T10:36:11.213 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:11.240 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mds. 2024-08-19T10:36:11.252 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:11.267 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:11.298 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mds (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:11.648 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mon. 2024-08-19T10:36:11.674 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:11.677 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package cephadm. 2024-08-19T10:36:11.703 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../069-cephadm_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:11.705 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mon (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:11.743 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking cephadm (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:12.054 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-asyncssh. 2024-08-19T10:36:12.080 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-08-19T10:36:12.120 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-08-19T10:36:12.309 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-osd. 2024-08-19T10:36:12.335 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:12.375 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-osd (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:12.423 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python-babel-localedata. 2024-08-19T10:36:12.438 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-08-19T10:36:12.481 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-08-19T10:36:13.429 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph. 2024-08-19T10:36:13.445 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../066-ceph_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:13.477 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-babel. 2024-08-19T10:36:13.487 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:13.504 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-08-19T10:36:13.544 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-08-19T10:36:13.806 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-fuse. 2024-08-19T10:36:13.833 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:13.863 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jinja2. 2024-08-19T10:36:13.873 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-fuse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:13.890 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-08-19T10:36:13.930 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-08-19T10:36:14.226 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mds. 2024-08-19T10:36:14.253 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:14.293 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mds (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:14.333 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-19T10:36:14.360 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:14.399 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-cephadm (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:14.704 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package cephadm. 2024-08-19T10:36:14.731 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../069-cephadm_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:14.752 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-19T10:36:14.770 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking cephadm (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:14.779 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-08-19T10:36:14.819 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-19T10:36:15.082 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-asyncssh. 2024-08-19T10:36:15.109 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-08-19T10:36:15.138 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-routes. 2024-08-19T10:36:15.148 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-08-19T10:36:15.165 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-08-19T10:36:15.205 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-08-19T10:36:15.459 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python-babel-localedata. 2024-08-19T10:36:15.486 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-08-19T10:36:15.525 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-08-19T10:36:15.733 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-19T10:36:15.760 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:15.799 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-dashboard (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:16.748 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-babel. 2024-08-19T10:36:16.775 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-08-19T10:36:16.814 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-08-19T10:36:17.125 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-jinja2. 2024-08-19T10:36:17.152 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-08-19T10:36:17.194 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-08-19T10:36:17.545 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-19T10:36:17.572 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:17.611 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-cephadm (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:17.948 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-19T10:36:17.975 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-08-19T10:36:18.015 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-19T10:36:18.318 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-routes. 2024-08-19T10:36:18.345 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-08-19T10:36:18.376 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-08-19T10:36:18.504 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-08-19T10:36:18.521 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-08-19T10:36:18.562 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-08-19T10:36:18.713 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-19T10:36:18.740 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:18.796 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-dashboard (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:19.007 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-joblib. 2024-08-19T10:36:19.034 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-08-19T10:36:19.073 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-08-19T10:36:19.427 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-08-19T10:36:19.454 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-08-19T10:36:19.493 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-08-19T10:36:19.762 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-sklearn. 2024-08-19T10:36:19.778 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-08-19T10:36:19.812 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-08-19T10:36:20.521 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-19T10:36:20.549 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:20.674 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:21.393 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-08-19T10:36:21.410 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-08-19T10:36:21.450 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-08-19T10:36:21.593 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cachetools. 2024-08-19T10:36:21.621 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-08-19T10:36:21.660 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-08-19T10:36:21.904 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-19T10:36:21.905 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-joblib. 2024-08-19T10:36:21.920 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-08-19T10:36:21.932 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-08-19T10:36:21.954 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-08-19T10:36:21.971 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-08-19T10:36:22.291 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-19T10:36:22.308 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-08-19T10:36:22.318 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-08-19T10:36:22.335 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-08-19T10:36:22.357 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-08-19T10:36:22.374 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-08-19T10:36:22.652 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rsa. 2024-08-19T10:36:22.679 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-08-19T10:36:22.685 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-sklearn. 2024-08-19T10:36:22.713 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-08-19T10:36:22.718 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rsa (4.8-1) ... 2024-08-19T10:36:22.752 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-08-19T10:36:23.039 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-google-auth. 2024-08-19T10:36:23.066 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-08-19T10:36:23.105 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-08-19T10:36:23.441 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-19T10:36:23.457 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-08-19T10:36:23.499 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-08-19T10:36:23.536 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-19T10:36:23.564 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:23.688 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:23.794 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-websocket. 2024-08-19T10:36:23.810 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-08-19T10:36:23.852 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-08-19T10:36:24.139 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-19T10:36:24.170 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-08-19T10:36:24.205 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-08-19T10:36:24.583 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-cachetools. 2024-08-19T10:36:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-08-19T10:36:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-08-19T10:36:24.902 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-19T10:36:24.919 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:24.928 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-19T10:36:24.956 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-08-19T10:36:24.960 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:24.994 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-08-19T10:36:25.222 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-19T10:36:25.249 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-08-19T10:36:25.280 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-08-19T10:36:25.323 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-19T10:36:25.351 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-08-19T10:36:25.389 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-08-19T10:36:25.725 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-19T10:36:25.734 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-rsa. 2024-08-19T10:36:25.752 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-08-19T10:36:25.762 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-08-19T10:36:25.791 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-08-19T10:36:25.802 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-rsa (4.8-1) ... 2024-08-19T10:36:26.112 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-google-auth. 2024-08-19T10:36:26.140 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-08-19T10:36:26.179 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-08-19T10:36:26.210 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package jq. 2024-08-19T10:36:26.226 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-08-19T10:36:26.268 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-08-19T10:36:26.499 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-19T10:36:26.521 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package socat. 2024-08-19T10:36:26.527 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-08-19T10:36:26.537 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-08-19T10:36:26.565 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-08-19T10:36:26.571 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-08-19T10:36:26.857 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package xmlstarlet. 2024-08-19T10:36:26.885 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-08-19T10:36:26.924 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-08-19T10:36:27.110 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-websocket. 2024-08-19T10:36:27.138 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-08-19T10:36:27.176 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-08-19T10:36:27.286 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-test. 2024-08-19T10:36:27.303 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../097-ceph-test_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:27.344 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-test (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:27.488 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-19T10:36:27.517 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-08-19T10:36:27.555 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-08-19T10:36:28.135 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-19T10:36:28.151 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:28.185 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:28.513 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-19T10:36:28.541 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-08-19T10:36:28.579 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-08-19T10:36:28.916 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-19T10:36:28.944 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-08-19T10:36:28.982 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-08-19T10:36:29.243 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package jq. 2024-08-19T10:36:29.259 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-08-19T10:36:29.310 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-08-19T10:36:29.538 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package socat. 2024-08-19T10:36:29.565 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-08-19T10:36:29.604 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-08-19T10:36:29.824 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-volume. 2024-08-19T10:36:29.851 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:29.890 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-volume (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:29.908 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package xmlstarlet. 2024-08-19T10:36:29.935 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-08-19T10:36:29.974 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-08-19T10:36:30.203 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-19T10:36:30.219 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:30.261 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libcephfs-dev (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:30.331 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-test. 2024-08-19T10:36:30.347 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../097-ceph-test_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:30.388 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-test (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:30.530 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua-socket:amd64. 2024-08-19T10:36:30.546 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-08-19T10:36:30.579 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-08-19T10:36:30.901 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua-sec:amd64. 2024-08-19T10:36:30.929 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-08-19T10:36:30.967 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-08-19T10:36:31.296 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package nvme-cli. 2024-08-19T10:36:31.324 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-08-19T10:36:31.362 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-08-19T10:36:31.734 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package pkg-config. 2024-08-19T10:36:31.762 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-08-19T10:36:31.792 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-08-19T10:36:32.045 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-08-19T10:36:32.062 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-08-19T10:36:32.103 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-08-19T10:36:32.548 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-iniconfig. 2024-08-19T10:36:32.576 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-08-19T10:36:32.614 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-08-19T10:36:32.918 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package ceph-volume. 2024-08-19T10:36:32.925 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pastescript. 2024-08-19T10:36:32.946 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-5847-g79766939-1jammy_all.deb ... 2024-08-19T10:36:32.954 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-19T10:36:32.984 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking ceph-volume (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:32.992 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-19T10:36:33.280 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-19T10:36:33.296 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:33.312 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pluggy. 2024-08-19T10:36:33.338 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking libcephfs-dev (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:33.340 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-08-19T10:36:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-08-19T10:36:33.650 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package lua-socket:amd64. 2024-08-19T10:36:33.678 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-08-19T10:36:33.689 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-psutil. 2024-08-19T10:36:33.707 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-08-19T10:36:33.718 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-08-19T10:36:33.756 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-08-19T10:36:34.012 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package lua-sec:amd64. 2024-08-19T10:36:34.041 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-08-19T10:36:34.079 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-08-19T10:36:34.092 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-py. 2024-08-19T10:36:34.121 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-08-19T10:36:34.167 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-py (1.10.0-1) ... 2024-08-19T10:36:34.416 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package nvme-cli. 2024-08-19T10:36:34.445 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-08-19T10:36:34.471 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pygments. 2024-08-19T10:36:34.498 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-08-19T10:36:34.499 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-08-19T10:36:34.537 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-08-19T10:36:34.888 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package pkg-config. 2024-08-19T10:36:34.916 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-08-19T10:36:34.954 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-08-19T10:36:34.991 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-19T10:36:35.019 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-08-19T10:36:35.057 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-08-19T10:36:35.249 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-08-19T10:36:35.277 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-08-19T10:36:35.315 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-08-19T10:36:35.368 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-toml. 2024-08-19T10:36:35.396 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-08-19T10:36:35.434 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-08-19T10:36:35.711 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-iniconfig. 2024-08-19T10:36:35.737 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pytest. 2024-08-19T10:36:35.739 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-08-19T10:36:35.754 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-08-19T10:36:35.777 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-08-19T10:36:35.796 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-08-19T10:36:36.072 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pastescript. 2024-08-19T10:36:36.100 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-19T10:36:36.133 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-simplejson. 2024-08-19T10:36:36.138 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-19T10:36:36.160 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-08-19T10:36:36.199 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-08-19T10:36:36.458 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pluggy. 2024-08-19T10:36:36.476 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-19T10:36:36.486 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-08-19T10:36:36.493 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-08-19T10:36:36.524 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-08-19T10:36:36.534 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-08-19T10:36:36.819 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-psutil. 2024-08-19T10:36:36.847 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-08-19T10:36:36.885 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-08-19T10:36:37.272 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package radosgw. 2024-08-19T10:36:37.300 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../116-radosgw_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:37.338 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking radosgw (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:37.497 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-py. 2024-08-19T10:36:37.525 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-08-19T10:36:37.563 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-py (1.10.0-1) ... 2024-08-19T10:36:37.875 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pygments. 2024-08-19T10:36:37.903 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-08-19T10:36:37.941 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-08-19T10:36:38.199 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package rbd-fuse. 2024-08-19T10:36:38.216 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:38.249 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking rbd-fuse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:38.412 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-19T10:36:38.440 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-08-19T10:36:38.479 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-08-19T10:36:38.540 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-08-19T10:36:38.790 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-toml. 2024-08-19T10:36:38.818 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-08-19T10:36:38.822 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-08-19T10:36:38.856 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-08-19T10:36:38.949 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-08-19T10:36:39.134 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-pytest. 2024-08-19T10:36:39.150 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-08-19T10:36:39.184 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-08-19T10:36:39.189 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-08-19T10:36:39.516 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-08-19T10:36:39.529 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package python3-simplejson. 2024-08-19T10:36:39.557 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-08-19T10:36:39.596 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-08-19T10:36:39.873 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-19T10:36:39.889 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-08-19T10:36:39.907 INFO:teuthology.orchestra.run.smithi190.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-08-19T10:36:39.931 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-08-19T10:36:39.967 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up cephadm (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:40.458 INFO:teuthology.orchestra.run.smithi190.stdout:Adding system user cephadm....done 2024-08-19T10:36:40.573 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-08-19T10:36:40.627 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package radosgw. 2024-08-19T10:36:40.656 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../116-radosgw_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:40.693 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking radosgw (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:40.841 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-08-19T10:36:41.092 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-08-19T10:36:41.193 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-08-19T10:36:41.427 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-19T10:36:41.670 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-08-19T10:36:41.688 INFO:teuthology.orchestra.run.smithi089.stdout:Selecting previously unselected package rbd-fuse. 2024-08-19T10:36:41.705 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-5847-g79766939-1jammy_amd64.deb ... 2024-08-19T10:36:41.747 INFO:teuthology.orchestra.run.smithi089.stdout:Unpacking rbd-fuse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:41.813 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-py (1.10.0-1) ... 2024-08-19T10:36:42.034 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-08-19T10:36:42.131 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-08-19T10:36:42.276 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-08-19T10:36:42.402 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-08-19T10:36:42.508 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-08-19T10:36:42.646 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-08-19T10:36:42.750 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-08-19T10:36:42.962 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-08-19T10:36:43.018 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-08-19T10:36:43.261 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-08-19T10:36:43.397 INFO:teuthology.orchestra.run.smithi089.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-08-19T10:36:43.463 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up cephadm (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:43.513 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-08-19T10:36:43.789 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-08-19T10:36:43.963 INFO:teuthology.orchestra.run.smithi089.stdout:Adding system user cephadm....done 2024-08-19T10:36:44.039 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-ceph-argparse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:44.085 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-08-19T10:36:44.366 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-08-19T10:36:44.370 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-08-19T10:36:44.483 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-08-19T10:36:44.609 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-08-19T10:36:44.630 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-08-19T10:36:44.710 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-08-19T10:36:44.757 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-08-19T10:36:44.977 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-08-19T10:36:45.015 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-19T10:36:45.258 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-08-19T10:36:45.352 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-08-19T10:36:45.376 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-py (1.10.0-1) ... 2024-08-19T10:36:45.384 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-08-19T10:36:45.448 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-08-19T10:36:45.574 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-08-19T10:36:45.678 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-08-19T10:36:45.933 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-08-19T10:36:46.034 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-08-19T10:36:46.038 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-08-19T10:36:46.285 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-08-19T10:36:46.306 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-08-19T10:36:46.544 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-08-19T10:36:46.582 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-08-19T10:36:46.854 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up zip (3.0-12build2) ... 2024-08-19T10:36:46.858 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-08-19T10:36:46.980 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-08-19T10:36:47.093 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-08-19T10:36:47.344 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-08-19T10:36:47.620 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-ceph-argparse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:47.623 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-08-19T10:36:47.874 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-08-19T10:36:48.000 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-08-19T10:36:48.126 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-08-19T10:36:48.180 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-08-19T10:36:48.306 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-08-19T10:36:48.419 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-08-19T10:36:48.432 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-08-19T10:36:48.557 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-08-19T10:36:48.812 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-08-19T10:36:48.817 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-08-19T10:36:49.063 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-08-19T10:36:49.298 INFO:teuthology.orchestra.run.smithi089.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-08-19T10:36:49.331 INFO:teuthology.orchestra.run.smithi089.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-08-19T10:36:49.395 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-08-19T10:36:49.431 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-08-19T10:36:49.521 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-08-19T10:36:49.716 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-08-19T10:36:49.889 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-08-19T10:36:50.015 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-08-19T10:36:50.042 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-08-19T10:36:50.283 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-08-19T10:36:50.293 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-08-19T10:36:50.402 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-ceph-common (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:50.551 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-08-19T10:36:50.704 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-08-19T10:36:50.843 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up zip (3.0-12build2) ... 2024-08-19T10:36:50.969 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-08-19T10:36:51.021 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-08-19T10:36:51.680 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-08-19T10:36:51.931 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-08-19T10:36:52.057 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-08-19T10:36:52.152 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-08-19T10:36:52.158 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-08-19T10:36:52.403 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:36:52.442 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-08-19T10:36:52.538 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-toml (0.10.2-1) ... 2024-08-19T10:36:52.805 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-08-19T10:36:52.819 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-08-19T10:36:52.940 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-08-19T10:36:53.066 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-08-19T10:36:53.069 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-08-19T10:36:53.384 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-08-19T10:36:53.497 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-08-19T10:36:53.627 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-08-19T10:36:53.789 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-08-19T10:36:53.877 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:36:54.087 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-08-19T10:36:54.124 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-08-19T10:36:54.350 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-08-19T10:36:54.388 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-08-19T10:36:54.451 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-ceph-common (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:36:54.639 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-08-19T10:36:54.753 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-08-19T10:36:54.890 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-08-19T10:36:55.070 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-08-19T10:36:55.133 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-19T10:36:55.400 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-08-19T10:36:55.670 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-08-19T10:36:55.779 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-08-19T10:36:56.054 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-08-19T10:36:56.180 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-08-19T10:36:56.259 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-08-19T10:36:56.472 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-08-19T10:36:56.503 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:36:56.637 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-toml (0.10.2-1) ... 2024-08-19T10:36:56.913 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-08-19T10:36:56.973 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-08-19T10:36:57.039 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-08-19T10:36:57.165 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-08-19T10:36:57.249 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-08-19T10:36:57.350 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-08-19T10:36:57.458 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-08-19T10:36:57.468 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-08-19T10:36:57.540 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-08-19T10:36:57.709 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-08-19T10:36:57.761 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-08-19T10:36:57.887 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-08-19T10:36:57.952 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:36:58.078 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-08-19T10:36:58.289 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-08-19T10:36:58.388 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-08-19T10:36:58.548 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua-any (27ubuntu1) ... 2024-08-19T10:36:58.648 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-08-19T10:36:58.657 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-portend (3.0.0-1) ... 2024-08-19T10:36:58.915 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-08-19T10:36:58.916 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:36:59.017 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-08-19T10:36:59.183 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-19T10:36:59.318 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-08-19T10:36:59.434 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-08-19T10:36:59.620 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-08-19T10:36:59.708 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-08-19T10:36:59.836 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-08-19T10:36:59.888 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-08-19T10:37:00.014 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rsa (4.8-1) ... 2024-08-19T10:37:00.120 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-08-19T10:37:00.255 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-08-19T10:37:00.298 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-08-19T10:37:00.556 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-08-19T10:37:00.574 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-08-19T10:37:00.891 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-08-19T10:37:00.918 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-08-19T10:37:01.150 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-08-19T10:37:01.259 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-08-19T10:37:01.304 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-19T10:37:01.385 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-08-19T10:37:01.474 INFO:teuthology.orchestra.run.smithi089.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-08-19T10:37:01.588 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-08-19T10:37:01.704 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-08-19T10:37:01.830 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-08-19T10:37:01.924 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-08-19T10:37:02.050 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librados2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:02.176 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libsqlite3-mod-ceph (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:02.215 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-08-19T10:37:02.302 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-08-19T10:37:02.482 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up lua-any (27ubuntu1) ... 2024-08-19T10:37:02.608 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-portend (3.0.0-1) ... 2024-08-19T10:37:02.638 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libcephfs2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:02.756 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libradosstriper1 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:02.851 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-08-19T10:37:02.882 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-08-19T10:37:02.977 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-08-19T10:37:03.142 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librbd1 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:03.268 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-modules-core (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:03.328 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-08-19T10:37:03.386 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-fuse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:03.597 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-19T10:37:03.598 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-19T10:37:03.638 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-08-19T10:37:03.914 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-08-19T10:37:04.040 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rsa (4.8-1) ... 2024-08-19T10:37:04.104 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libcephfs-dev (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:04.230 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rados (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:04.299 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-08-19T10:37:04.356 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librgw2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:04.457 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-08-19T10:37:04.567 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-08-19T10:37:04.895 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-08-19T10:37:05.289 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-19T10:37:05.574 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-08-19T10:37:05.660 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rbd (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:05.786 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up rbd-fuse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:05.893 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-08-19T10:37:05.912 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rgw (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:06.019 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librados2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:06.038 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cephfs (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:06.145 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libsqlite3-mod-ceph (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:06.164 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-common (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:06.271 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-08-19T10:37:06.608 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libcephfs2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:06.690 INFO:teuthology.orchestra.run.smithi190.stdout:Adding group ceph....done 2024-08-19T10:37:06.733 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libradosstriper1 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:06.834 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-08-19T10:37:07.025 INFO:teuthology.orchestra.run.smithi190.stdout:Adding system user ceph....done 2024-08-19T10:37:07.128 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librbd1 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:07.168 INFO:teuthology.orchestra.run.smithi190.stdout:Setting system user ceph properties....done 2024-08-19T10:37:07.186 INFO:teuthology.orchestra.run.smithi190.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-19T10:37:07.254 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-modules-core (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:07.321 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-19T10:37:07.380 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-fuse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:07.591 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-19T10:37:07.592 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-19T10:37:07.627 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-19T10:37:08.065 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up libcephfs-dev (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:08.073 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-test (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:08.199 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up radosgw (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:08.216 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rados (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:08.342 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up librgw2 (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:08.451 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-08-19T10:37:08.662 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-19T10:37:08.663 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-19T10:37:09.118 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-base (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:09.452 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-19T10:37:09.588 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rbd (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:09.714 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up rbd-fuse (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:09.832 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-rgw (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:09.919 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mds (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:09.941 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up python3-cephfs (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:10.067 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-common (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:10.127 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-19T10:37:10.128 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-19T10:37:10.580 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:10.593 INFO:teuthology.orchestra.run.smithi089.stdout:Adding group ceph....done 2024-08-19T10:37:10.777 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-19T10:37:10.777 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-19T10:37:10.961 INFO:teuthology.orchestra.run.smithi089.stdout:Adding system user ceph....done 2024-08-19T10:37:11.138 INFO:teuthology.orchestra.run.smithi089.stdout:Setting system user ceph properties....done 2024-08-19T10:37:11.153 INFO:teuthology.orchestra.run.smithi089.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-19T10:37:11.266 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-osd (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:11.290 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-19T10:37:11.598 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-19T10:37:11.633 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-19T10:37:11.633 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-19T10:37:12.086 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-test (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:12.117 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-k8sevents (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:12.211 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up radosgw (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:12.251 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:12.427 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mon (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:12.639 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-19T10:37:12.639 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-19T10:37:12.682 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-19T10:37:12.682 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-19T10:37:13.104 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-cephadm (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:13.122 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-base (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:13.246 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:13.373 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-dashboard (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:13.484 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-19T10:37:13.523 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-volume (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:13.868 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-08-19T10:37:13.957 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mds (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:13.985 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-08-19T10:37:14.137 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-08-19T10:37:14.182 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-19T10:37:14.182 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-19T10:37:14.617 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:14.831 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-19T10:37:14.831 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-19T10:37:14.862 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-08-19T10:37:15.327 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-osd (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:15.498 INFO:teuthology.orchestra.run.smithi190.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-19T10:37:15.506 DEBUG:teuthology.parallel:result is None 2024-08-19T10:37:15.591 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-19T10:37:15.591 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-19T10:37:16.070 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-k8sevents (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:16.230 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:16.381 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mon (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:16.591 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-19T10:37:16.591 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-19T10:37:17.065 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-cephadm (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:17.216 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:17.334 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-mgr-dashboard (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:17.485 INFO:teuthology.orchestra.run.smithi089.stdout:Setting up ceph-volume (19.0.0-5847-g79766939-1jammy) ... 2024-08-19T10:37:17.829 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-08-19T10:37:17.930 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-08-19T10:37:18.083 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-08-19T10:37:19.535 INFO:teuthology.orchestra.run.smithi089.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-08-19T10:37:20.201 INFO:teuthology.orchestra.run.smithi089.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-19T10:37:20.206 DEBUG:teuthology.parallel:result is None 2024-08-19T10:37:20.207 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=797669399b32d2915e64eb6d4b1257ae3c50f80b 2024-08-19T10:37:20.359 DEBUG:teuthology.orchestra.run.smithi089:> dpkg-query -W -f '${Version}' ceph 2024-08-19T10:37:20.378 INFO:teuthology.orchestra.run.smithi089.stdout:19.0.0-5847-g79766939-1jammy 2024-08-19T10:37:20.378 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5847-g79766939-1jammy 2024-08-19T10:37:20.378 INFO:teuthology.task.install:The correct ceph version 19.0.0-5847-g79766939-1jammy is installed. 2024-08-19T10:37:20.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=797669399b32d2915e64eb6d4b1257ae3c50f80b 2024-08-19T10:37:20.525 DEBUG:teuthology.orchestra.run.smithi190:> dpkg-query -W -f '${Version}' ceph 2024-08-19T10:37:20.544 INFO:teuthology.orchestra.run.smithi190.stdout:19.0.0-5847-g79766939-1jammy 2024-08-19T10:37:20.545 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5847-g79766939-1jammy 2024-08-19T10:37:20.545 INFO:teuthology.task.install:The correct ceph version 19.0.0-5847-g79766939-1jammy is installed. 2024-08-19T10:37:20.547 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-19T10:37:20.547 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:20.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-19T10:37:20.561 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:20.561 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-19T10:37:20.600 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-19T10:37:20.600 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:20.600 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2024-08-19T10:37:20.615 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-19T10:37:20.671 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:20.671 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-08-19T10:37:20.684 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-19T10:37:20.738 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-19T10:37:20.738 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:20.738 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-19T10:37:20.751 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-19T10:37:20.807 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:20.807 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-19T10:37:20.821 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-19T10:37:20.874 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-19T10:37:20.874 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:20.874 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2024-08-19T10:37:20.889 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-19T10:37:20.943 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:20.943 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-08-19T10:37:20.955 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-19T10:37:21.009 INFO:teuthology.run_tasks:Running task ceph... 2024-08-19T10:37:21.098 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-08-19T10:37:21.099 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /var/log/ceph 2024-08-19T10:37:21.101 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2024-08-19T10:37:21.113 INFO:tasks.ceph:Disabling ceph logrotate... 2024-08-19T10:37:21.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-19T10:37:21.158 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-19T10:37:21.170 INFO:tasks.ceph:Creating extra log directories... 2024-08-19T10:37:21.171 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-19T10:37:21.213 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-19T10:37:21.235 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-08-19T10:37:21.235 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(TOO_FEW_PGS\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(AUTH_BAD_CAPS\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-08-19T10:37:21.236 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373', 'branch': 'main', 'description': 'rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-bitmap} rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_mon_workunits}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7859373', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(TOO_FEW_PGS\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(AUTH_BAD_CAPS\\)'], 'sha1': '797669399b32d2915e64eb6d4b1257ae3c50f80b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '797669399b32d2915e64eb6d4b1257ae3c50f80b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '964adda23da9f7b057cb34fdc0b82aeb7ed65324'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 101, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'], ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 1858, 'sha1': '797669399b32d2915e64eb6d4b1257ae3c50f80b', 'sleep_before_teardown': 0, 'subset': '29813/100000', 'suite': 'rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '964adda23da9f7b057cb34fdc0b82aeb7ed65324', 'targets': {'smithi089.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE1sAOXZx6zu9Ll+PlBcKo6ctF8ClSPtWmlSZud5Qvv9kSgU/F0abFurL58XFgl3HDelBhAATs5mojiG4Gyckv4=', 'smithi190.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE0n+qO+CEheY4270wXhR5+DoAaKoN8bo5sSb/hJKi6wLjrEQLX6MCFnyabola2NeMvtyT8sRGJZ/kqiSALWM2M='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 90, 'store_thrash': True, 'thrash_delay': 1, 'thrash_many': True}}, {'workunit': {'clients': {'client.0': ['mon/pool_ops.sh', 'mon/crush_ops.sh', 'mon/osd.sh', 'mon/caps.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '67db276914d29505c9707d04f3a3b8fb9d8aee4b', 'timestamp': '2024-08-18_20:00:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-08-19T10:37:21.236 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-19T10:37:21.270 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-19T10:37:21.275 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/run/ceph 2024-08-19T10:37:21.318 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2024-08-19T10:37:21.331 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:21.331 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2024-08-19T10:37:21.376 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-19T10:37:21.376 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2024-08-19T10:37:21.425 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-19T10:37:21.425 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T10:37:21.425 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 870 Links: 1 2024-08-19T10:37:21.425 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T10:37:21.425 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-08-19 10:31:53.449960334 +0000 2024-08-19T10:37:21.425 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-08-19 10:31:53.201962549 +0000 2024-08-19T10:37:21.425 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-08-19 10:31:53.201962549 +0000 2024-08-19T10:37:21.425 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2024-08-19T10:37:21.426 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-19T10:37:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-08-19T10:37:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-08-19T10:37:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.00045577 s, 1.1 MB/s 2024-08-19T10:37:21.480 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-19T10:37:21.533 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2024-08-19T10:37:21.581 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-19T10:37:21.581 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T10:37:21.581 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-08-19T10:37:21.581 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T10:37:21.581 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-08-19 10:31:53.925956083 +0000 2024-08-19T10:37:21.581 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-08-19 10:31:53.681958262 +0000 2024-08-19T10:37:21.581 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-08-19 10:31:53.681958262 +0000 2024-08-19T10:37:21.581 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2024-08-19T10:37:21.581 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-19T10:37:21.634 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-08-19T10:37:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-08-19T10:37:21.635 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000451826 s, 1.1 MB/s 2024-08-19T10:37:21.636 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-19T10:37:21.686 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2024-08-19T10:37:21.732 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-19T10:37:21.732 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T10:37:21.733 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-08-19T10:37:21.733 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T10:37:21.733 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-08-19 10:31:54.169953902 +0000 2024-08-19T10:37:21.733 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-08-19 10:31:54.165953939 +0000 2024-08-19T10:37:21.733 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-08-19 10:31:54.165953939 +0000 2024-08-19T10:37:21.733 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2024-08-19T10:37:21.734 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-19T10:37:21.786 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-08-19T10:37:21.787 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-08-19T10:37:21.787 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.00042364 s, 1.2 MB/s 2024-08-19T10:37:21.788 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-19T10:37:21.838 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2024-08-19T10:37:21.885 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-19T10:37:21.885 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T10:37:21.885 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 925 Links: 1 2024-08-19T10:37:21.885 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T10:37:21.885 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-08-19 10:31:54.713949044 +0000 2024-08-19T10:37:21.885 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-08-19 10:31:54.709949079 +0000 2024-08-19T10:37:21.885 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-08-19 10:31:54.709949079 +0000 2024-08-19T10:37:21.885 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2024-08-19T10:37:21.886 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-19T10:37:21.939 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-08-19T10:37:21.939 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-08-19T10:37:21.939 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000406227 s, 1.3 MB/s 2024-08-19T10:37:21.940 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-19T10:37:21.991 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-08-19T10:37:21.991 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:21.991 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-08-19T10:37:21.997 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-19T10:37:21.997 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-08-19T10:37:22.043 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-19T10:37:22.043 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T10:37:22.043 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 869 Links: 1 2024-08-19T10:37:22.043 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T10:37:22.043 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-19 10:31:53.478051227 +0000 2024-08-19T10:37:22.043 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-19 10:31:53.254053228 +0000 2024-08-19T10:37:22.043 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-19 10:31:53.254053228 +0000 2024-08-19T10:37:22.044 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-08-19T10:37:22.044 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-19T10:37:22.097 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-19T10:37:22.097 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-19T10:37:22.097 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000453116 s, 1.1 MB/s 2024-08-19T10:37:22.098 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-19T10:37:22.148 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-08-19T10:37:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-19T10:37:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T10:37:22.195 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-08-19T10:37:22.195 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T10:37:22.195 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-19 10:31:53.698049262 +0000 2024-08-19T10:37:22.195 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-19 10:31:53.694049297 +0000 2024-08-19T10:37:22.195 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-19 10:31:53.694049297 +0000 2024-08-19T10:37:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-08-19T10:37:22.195 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-19T10:37:22.249 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-19T10:37:22.249 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-19T10:37:22.249 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.00035865 s, 1.4 MB/s 2024-08-19T10:37:22.250 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-19T10:37:22.300 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-08-19T10:37:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-19T10:37:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T10:37:22.347 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 896 Links: 1 2024-08-19T10:37:22.347 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T10:37:22.347 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-19 10:31:54.482042259 +0000 2024-08-19T10:37:22.347 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-19 10:31:54.214044652 +0000 2024-08-19T10:37:22.347 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-19 10:31:54.214044652 +0000 2024-08-19T10:37:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-08-19T10:37:22.348 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-19T10:37:22.405 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-19T10:37:22.405 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-19T10:37:22.406 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000370716 s, 1.4 MB/s 2024-08-19T10:37:22.407 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-19T10:37:22.456 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-08-19T10:37:22.503 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-19T10:37:22.503 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T10:37:22.503 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-08-19T10:37:22.503 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T10:37:22.503 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-19 10:31:54.990037721 +0000 2024-08-19T10:37:22.503 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-19 10:31:54.742039937 +0000 2024-08-19T10:37:22.503 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-19 10:31:54.742039937 +0000 2024-08-19T10:37:22.503 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-08-19T10:37:22.504 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-19T10:37:22.557 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-19T10:37:22.557 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-19T10:37:22.557 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000386984 s, 1.3 MB/s 2024-08-19T10:37:22.558 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-19T10:37:22.607 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-08-19T10:37:22.608 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-08-19T10:37:22.608 INFO:tasks.ceph:Generating config... 2024-08-19T10:37:22.609 INFO:tasks.ceph:[client] debug monc = 20 2024-08-19T10:37:22.609 INFO:tasks.ceph:[client] debug ms = 1 2024-08-19T10:37:22.609 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-08-19T10:37:22.609 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-08-19T10:37:22.609 INFO:tasks.ceph:[global] ms inject delay max = 1 2024-08-19T10:37:22.609 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2024-08-19T10:37:22.610 INFO:tasks.ceph:[global] ms inject delay type = mon 2024-08-19T10:37:22.610 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2024-08-19T10:37:22.610 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2024-08-19T10:37:22.610 INFO:tasks.ceph:[global] ms type = async 2024-08-19T10:37:22.610 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-08-19T10:37:22.610 INFO:tasks.ceph:[mgr] debug monc = 10 2024-08-19T10:37:22.610 INFO:tasks.ceph:[mgr] debug ms = 1 2024-08-19T10:37:22.610 INFO:tasks.ceph:[mon] debug mon = 20 2024-08-19T10:37:22.610 INFO:tasks.ceph:[mon] debug ms = 1 2024-08-19T10:37:22.610 INFO:tasks.ceph:[mon] debug paxos = 20 2024-08-19T10:37:22.610 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2024-08-19T10:37:22.610 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-08-19T10:37:22.610 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-08-19T10:37:22.610 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-08-19T10:37:22.611 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2024-08-19T10:37:22.611 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2024-08-19T10:37:22.611 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-08-19T10:37:22.611 INFO:tasks.ceph:[mon] paxos service trim min = 5 2024-08-19T10:37:22.611 INFO:tasks.ceph:[osd] bdev async discard = True 2024-08-19T10:37:22.611 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-08-19T10:37:22.611 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-08-19T10:37:22.611 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-08-19T10:37:22.611 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-08-19T10:37:22.611 INFO:tasks.ceph:[osd] bluestore write v2 = True 2024-08-19T10:37:22.611 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-08-19T10:37:22.611 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-08-19T10:37:22.611 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] debug ms = 1 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] debug osd = 20 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-08-19T10:37:22.612 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-08-19T10:37:22.613 INFO:tasks.ceph:Setting up mon.a... 2024-08-19T10:37:22.613 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-08-19T10:37:22.646 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.keyring 2024-08-19T10:37:22.659 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-08-19T10:37:22.732 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-19T10:37:22.788 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.89'), ('mon.b', '[v2:172.21.15.89:3301,v1:172.21.15.89:6790]'), ('mon.c', '[v2:172.21.15.89:3302,v1:172.21.15.89:6791]'), ('mon.d', '[v2:172.21.15.89:3303,v1:172.21.15.89:6792]'), ('mon.e', '[v2:172.21.15.89:3304,v1:172.21.15.89:6793]'), ('mon.f', '172.21.15.190'), ('mon.g', '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]'), ('mon.h', '[v2:172.21.15.190:3302,v1:172.21.15.190:6791]'), ('mon.i', '[v2:172.21.15.190:3303,v1:172.21.15.190:6792]')] 2024-08-19T10:37:22.788 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.89,[v2:172.21.15.89:3301,v1:172.21.15.89:6790],[v2:172.21.15.89:3302,v1:172.21.15.89:6791],[v2:172.21.15.89:3303,v1:172.21.15.89:6792],[v2:172.21.15.89:3304,v1:172.21.15.89:6793],172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790],[v2:172.21.15.190:3302,v1:172.21.15.190:6791],[v2:172.21.15.190:3303,v1:172.21.15.190:6792]', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug monc': 10}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mon.d': {}, 'mon.e': {}, 'mon.f': {}, 'mon.g': {}, 'mon.h': {}, 'mon.i': {}} 2024-08-19T10:37:22.789 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:22.789 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-08-19T10:37:22.837 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.89 --addv b '[v2:172.21.15.89:3301,v1:172.21.15.89:6790]' --addv c '[v2:172.21.15.89:3302,v1:172.21.15.89:6791]' --addv d '[v2:172.21.15.89:3303,v1:172.21.15.89:6792]' --addv e '[v2:172.21.15.89:3304,v1:172.21.15.89:6793]' --add f 172.21.15.190 --addv g '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]' --addv h '[v2:172.21.15.190:3302,v1:172.21.15.190:6791]' --addv i '[v2:172.21.15.190:3303,v1:172.21.15.190:6792]' --print /home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:22.944 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:22.945 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: generated fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:22.945 INFO:teuthology.orchestra.run.smithi089.stdout:setting min_mon_release = quincy 2024-08-19T10:37:22.945 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 0 2024-08-19T10:37:22.945 INFO:teuthology.orchestra.run.smithi089.stdout:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:22.945 INFO:teuthology.orchestra.run.smithi089.stdout:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:22.945 INFO:teuthology.orchestra.run.smithi089.stdout:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:22.945 INFO:teuthology.orchestra.run.smithi089.stdout:min_mon_release 17 (quincy) 2024-08-19T10:37:22.945 INFO:teuthology.orchestra.run.smithi089.stdout:election_strategy: 1 2024-08-19T10:37:22.945 INFO:teuthology.orchestra.run.smithi089.stdout:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:22.945 INFO:teuthology.orchestra.run.smithi089.stdout:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:22.945 INFO:teuthology.orchestra.run.smithi089.stdout:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:22.946 INFO:teuthology.orchestra.run.smithi089.stdout:3: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:22.946 INFO:teuthology.orchestra.run.smithi089.stdout:4: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:22.946 INFO:teuthology.orchestra.run.smithi089.stdout:5: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:22.946 INFO:teuthology.orchestra.run.smithi089.stdout:6: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:22.946 INFO:teuthology.orchestra.run.smithi089.stdout:7: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:22.946 INFO:teuthology.orchestra.run.smithi089.stdout:8: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:22.946 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (9 monitors) 2024-08-19T10:37:22.947 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-08-19T10:37:22.952 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b6e37c79-20e9-449f-b89e-c2165d4f4bed... 2024-08-19T10:37:22.953 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-19T10:37:22.998 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-19T10:37:23.028 INFO:teuthology.orchestra.run.smithi089.stdout:[global] 2024-08-19T10:37:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: chdir = "" 2024-08-19T10:37:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-19T10:37:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: auth supported = cephx 2024-08-19T10:37:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: filestore xattr use omap = true 2024-08-19T10:37:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.028 INFO:teuthology.orchestra.run.smithi089.stdout: mon clock drift allowed = 1.000 2024-08-19T10:37:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: osd crush chooseleaf type = 0 2024-08-19T10:37:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: auth debug = true 2024-08-19T10:37:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on old message = true 2024-08-19T10:37:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on bug = true 2024-08-19T10:37:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-19T10:37:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: mon pg warn max object skew = 0 2024-08-19T10:37:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.029 INFO:teuthology.orchestra.run.smithi089.stdout: # disable pg_autoscaler by default for new pools 2024-08-19T10:37:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-19T10:37:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default size = 2 2024-08-19T10:37:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow primary affinity = true 2024-08-19T10:37:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow pg remap = true 2024-08-19T10:37:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on legacy crush tunables = false 2024-08-19T10:37:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on crush straw calc version zero = false 2024-08-19T10:37:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on no sortbitwise = false 2024-08-19T10:37:23.030 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on osd down out interval zero = false 2024-08-19T10:37:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on too few osds = false 2024-08-19T10:37:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-19T10:37:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-19T10:37:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: mon_allow_pool_size_one = true 2024-08-19T10:37:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-19T10:37:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: osd default data pool replay window = 5 2024-08-19T10:37:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: mon allow pool delete = true 2024-08-19T10:37:23.031 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: mon cluster log file level = debug 2024-08-19T10:37:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: debug asserts on shutdown = true 2024-08-19T10:37:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: mon health detail to clog = false 2024-08-19T10:37:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: mon host = "172.21.15.89,[v2:172.21.15.89:3301,v1:172.21.15.89:6790],[v2:172.21.15.89:3302,v1:172.21.15.89:6791],[v2:172.21.15.89:3303,v1:172.21.15.89:6792],[v2:172.21.15.89:3304,v1:172.21.15.89:6793],172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790],[v2:172.21.15.190:3302,v1:172.21.15.190:6791],[v2:172.21.15.190:3303,v1:172.21.15.190:6792]" 2024-08-19T10:37:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: mon client directed command retry = 5 2024-08-19T10:37:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: mon election default strategy = 1 2024-08-19T10:37:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: ms inject delay max = 1 2024-08-19T10:37:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: ms inject delay probability = 0.005 2024-08-19T10:37:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: ms inject delay type = mon 2024-08-19T10:37:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: ms inject internal delays = 0.002 2024-08-19T10:37:23.032 INFO:teuthology.orchestra.run.smithi089.stdout: ms inject socket failures = 2500 2024-08-19T10:37:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: ms type = async 2024-08-19T10:37:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: fsid = b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.033 INFO:teuthology.orchestra.run.smithi089.stdout:[osd] 2024-08-19T10:37:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: osd journal size = 100 2024-08-19T10:37:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub load threshold = 5.0 2024-08-19T10:37:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub max interval = 600 2024-08-19T10:37:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: osd mclock profile = high_recovery_ops 2024-08-19T10:37:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.033 INFO:teuthology.orchestra.run.smithi089.stdout: osd recover clone overlap = true 2024-08-19T10:37:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: osd recovery max chunk = 1048576 2024-08-19T10:37:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug shutdown = true 2024-08-19T10:37:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug op order = true 2024-08-19T10:37:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug verify stray on activate = true 2024-08-19T10:37:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug trim objects = true 2024-08-19T10:37:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: osd open classes on start = true 2024-08-19T10:37:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug pg log writeout = true 2024-08-19T10:37:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.034 INFO:teuthology.orchestra.run.smithi089.stdout: osd deep scrub update digest min age = 30 2024-08-19T10:37:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: osd map max advance = 10 2024-08-19T10:37:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: journal zero on create = true 2024-08-19T10:37:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: filestore ondisk finisher threads = 3 2024-08-19T10:37:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: filestore apply finisher threads = 3 2024-08-19T10:37:23.035 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: bdev debug aio = true 2024-08-19T10:37:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug misdirected ops = true 2024-08-19T10:37:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: bdev async discard = True 2024-08-19T10:37:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: bdev enable discard = True 2024-08-19T10:37:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore allocator = bitmap 2024-08-19T10:37:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore block size = 96636764160 2024-08-19T10:37:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore fsck on mount = True 2024-08-19T10:37:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore write v2 = True 2024-08-19T10:37:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore zero block detection = True 2024-08-19T10:37:23.036 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluefs = 20 2024-08-19T10:37:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluestore = 20 2024-08-19T10:37:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2024-08-19T10:37:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: debug osd = 20 2024-08-19T10:37:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: debug rocksdb = 10 2024-08-19T10:37:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd backfillfull_ratio = 0.85 2024-08-19T10:37:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd full ratio = 0.9 2024-08-19T10:37:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd nearfull ratio = 0.8 2024-08-19T10:37:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug verify cached snaps = True 2024-08-19T10:37:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug verify missing on start = True 2024-08-19T10:37:23.037 INFO:teuthology.orchestra.run.smithi089.stdout: osd failsafe full ratio = 0.95 2024-08-19T10:37:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: osd mclock override recovery settings = True 2024-08-19T10:37:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: osd objectstore = bluestore 2024-08-19T10:37:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: osd op queue = debug_random 2024-08-19T10:37:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: osd op queue cut off = debug_random 2024-08-19T10:37:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.038 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr] 2024-08-19T10:37:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2024-08-19T10:37:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: debug mgr = 20 2024-08-19T10:37:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2024-08-19T10:37:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2024-08-19T10:37:23.038 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2024-08-19T10:37:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2024-08-19T10:37:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: mgr/telemetry/nag = false 2024-08-19T10:37:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: debug monc = 10 2024-08-19T10:37:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.039 INFO:teuthology.orchestra.run.smithi089.stdout:[mon] 2024-08-19T10:37:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2024-08-19T10:37:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2024-08-19T10:37:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: debug paxos = 20 2024-08-19T10:37:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2024-08-19T10:37:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: mon data avail warn = 5 2024-08-19T10:37:23.039 INFO:teuthology.orchestra.run.smithi089.stdout: mon mgr mkfs grace = 240 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd reporter subtree level = osd 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd prime pg temp = true 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: auth mon ticket ttl = 660 # 11m 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: auth service ticket ttl = 240 # 4m 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: # don't complain about insecure global_id in the test suite 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-19T10:37:23.040 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: # 1m isn't quite enough 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: mon_down_mkfs_grace = 2m 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_filestore_osds = false 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: mon min osdmap epochs = 25 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: mon osdmap full prune interval = 2 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: mon osdmap full prune min = 15 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: mon osdmap full prune txsize = 2 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: mon scrub inject crc mismatch = 0.01 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: mon scrub inject missing keys = 0.05 2024-08-19T10:37:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: mon scrub interval = 300 2024-08-19T10:37:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: paxos service trim min = 5 2024-08-19T10:37:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:23.042 INFO:teuthology.orchestra.run.smithi089.stdout:[client] 2024-08-19T10:37:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: rgw cache enabled = true 2024-08-19T10:37:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable ops log = true 2024-08-19T10:37:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable usage log = true 2024-08-19T10:37:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-19T10:37:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-19T10:37:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: debug monc = 20 2024-08-19T10:37:23.042 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2024-08-19T10:37:23.042 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.a] 2024-08-19T10:37:23.043 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.b] 2024-08-19T10:37:23.043 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.c] 2024-08-19T10:37:23.043 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.d] 2024-08-19T10:37:23.043 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.e] 2024-08-19T10:37:23.043 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.f] 2024-08-19T10:37:23.043 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.g] 2024-08-19T10:37:23.043 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.h] 2024-08-19T10:37:23.043 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.i] 2024-08-19T10:37:23.045 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2024-08-19T10:37:23.045 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2024-08-19T10:37:23.045 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-19T10:37:23.045 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2024-08-19T10:37:23.045 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.045 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2024-08-19T10:37:23.046 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.046 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2024-08-19T10:37:23.046 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.046 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2024-08-19T10:37:23.046 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2024-08-19T10:37:23.046 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.046 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2024-08-19T10:37:23.046 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2024-08-19T10:37:23.046 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.046 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-19T10:37:23.046 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2024-08-19T10:37:23.047 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.047 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2024-08-19T10:37:23.047 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-19T10:37:23.047 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.047 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2024-08-19T10:37:23.047 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.047 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2024-08-19T10:37:23.047 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2024-08-19T10:37:23.047 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2024-08-19T10:37:23.047 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2024-08-19T10:37:23.047 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2024-08-19T10:37:23.048 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2024-08-19T10:37:23.048 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2024-08-19T10:37:23.048 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-19T10:37:23.048 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-19T10:37:23.048 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2024-08-19T10:37:23.048 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.048 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-19T10:37:23.048 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.048 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2024-08-19T10:37:23.048 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.048 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2024-08-19T10:37:23.049 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.049 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2024-08-19T10:37:23.049 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2024-08-19T10:37:23.049 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2024-08-19T10:37:23.049 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.89,[v2:172.21.15.89:3301,v1:172.21.15.89:6790],[v2:172.21.15.89:3302,v1:172.21.15.89:6791],[v2:172.21.15.89:3303,v1:172.21.15.89:6792],[v2:172.21.15.89:3304,v1:172.21.15.89:6793],172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790],[v2:172.21.15.190:3302,v1:172.21.15.190:6791],[v2:172.21.15.190:3303,v1:172.21.15.190:6792]" 2024-08-19T10:37:23.049 INFO:teuthology.orchestra.run.smithi190.stdout: mon client directed command retry = 5 2024-08-19T10:37:23.049 INFO:teuthology.orchestra.run.smithi190.stdout: mon election default strategy = 1 2024-08-19T10:37:23.049 INFO:teuthology.orchestra.run.smithi190.stdout: ms inject delay max = 1 2024-08-19T10:37:23.049 INFO:teuthology.orchestra.run.smithi190.stdout: ms inject delay probability = 0.005 2024-08-19T10:37:23.049 INFO:teuthology.orchestra.run.smithi190.stdout: ms inject delay type = mon 2024-08-19T10:37:23.049 INFO:teuthology.orchestra.run.smithi190.stdout: ms inject internal delays = 0.002 2024-08-19T10:37:23.050 INFO:teuthology.orchestra.run.smithi190.stdout: ms inject socket failures = 2500 2024-08-19T10:37:23.050 INFO:teuthology.orchestra.run.smithi190.stdout: ms type = async 2024-08-19T10:37:23.050 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:23.050 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.050 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2024-08-19T10:37:23.050 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2024-08-19T10:37:23.050 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.050 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2024-08-19T10:37:23.050 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug trim objects = true 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2024-08-19T10:37:23.051 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2024-08-19T10:37:23.052 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.052 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2024-08-19T10:37:23.052 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.052 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2024-08-19T10:37:23.052 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.052 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2024-08-19T10:37:23.052 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.052 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2024-08-19T10:37:23.053 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2024-08-19T10:37:23.053 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.053 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2024-08-19T10:37:23.053 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2024-08-19T10:37:23.053 INFO:teuthology.orchestra.run.smithi190.stdout: bdev async discard = True 2024-08-19T10:37:23.053 INFO:teuthology.orchestra.run.smithi190.stdout: bdev enable discard = True 2024-08-19T10:37:23.053 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore allocator = bitmap 2024-08-19T10:37:23.053 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2024-08-19T10:37:23.053 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2024-08-19T10:37:23.053 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore write v2 = True 2024-08-19T10:37:23.053 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore zero block detection = True 2024-08-19T10:37:23.054 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2024-08-19T10:37:23.054 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2024-08-19T10:37:23.054 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-08-19T10:37:23.054 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2024-08-19T10:37:23.054 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2024-08-19T10:37:23.054 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2024-08-19T10:37:23.054 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2024-08-19T10:37:23.054 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2024-08-19T10:37:23.054 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify cached snaps = True 2024-08-19T10:37:23.054 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify missing on start = True 2024-08-19T10:37:23.055 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2024-08-19T10:37:23.055 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock override recovery settings = True 2024-08-19T10:37:23.055 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2024-08-19T10:37:23.055 INFO:teuthology.orchestra.run.smithi190.stdout: osd op queue = debug_random 2024-08-19T10:37:23.055 INFO:teuthology.orchestra.run.smithi190.stdout: osd op queue cut off = debug_random 2024-08-19T10:37:23.055 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.055 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2024-08-19T10:37:23.055 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-08-19T10:37:23.055 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2024-08-19T10:37:23.056 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-08-19T10:37:23.056 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-08-19T10:37:23.056 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-08-19T10:37:23.056 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-08-19T10:37:23.056 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2024-08-19T10:37:23.056 INFO:teuthology.orchestra.run.smithi190.stdout: debug monc = 10 2024-08-19T10:37:23.056 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.056 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2024-08-19T10:37:23.056 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-08-19T10:37:23.056 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-08-19T10:37:23.056 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2024-08-19T10:37:23.057 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-08-19T10:37:23.057 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2024-08-19T10:37:23.057 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2024-08-19T10:37:23.057 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-08-19T10:37:23.057 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2024-08-19T10:37:23.057 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2024-08-19T10:37:23.057 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-08-19T10:37:23.057 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.057 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-19T10:37:23.057 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2024-08-19T10:37:23.057 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2024-08-19T10:37:23.058 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.058 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2024-08-19T10:37:23.058 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-19T10:37:23.058 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-19T10:37:23.058 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.058 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2024-08-19T10:37:23.058 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2024-08-19T10:37:23.058 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.058 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2024-08-19T10:37:23.058 INFO:teuthology.orchestra.run.smithi190.stdout: mon min osdmap epochs = 25 2024-08-19T10:37:23.059 INFO:teuthology.orchestra.run.smithi190.stdout: mon osdmap full prune interval = 2 2024-08-19T10:37:23.059 INFO:teuthology.orchestra.run.smithi190.stdout: mon osdmap full prune min = 15 2024-08-19T10:37:23.059 INFO:teuthology.orchestra.run.smithi190.stdout: mon osdmap full prune txsize = 2 2024-08-19T10:37:23.059 INFO:teuthology.orchestra.run.smithi190.stdout: mon scrub inject crc mismatch = 0.01 2024-08-19T10:37:23.059 INFO:teuthology.orchestra.run.smithi190.stdout: mon scrub inject missing keys = 0.05 2024-08-19T10:37:23.059 INFO:teuthology.orchestra.run.smithi190.stdout: mon scrub interval = 300 2024-08-19T10:37:23.059 INFO:teuthology.orchestra.run.smithi190.stdout: paxos service trim min = 5 2024-08-19T10:37:23.059 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T10:37:23.059 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2024-08-19T10:37:23.059 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2024-08-19T10:37:23.059 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout: debug monc = 20 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.d] 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.e] 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.f] 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.g] 2024-08-19T10:37:23.060 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.h] 2024-08-19T10:37:23.061 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.i] 2024-08-19T10:37:23.061 INFO:tasks.ceph:Creating admin key on mon.a... 2024-08-19T10:37:23.061 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-08-19T10:37:23.136 INFO:tasks.ceph:Copying monmap to all nodes... 2024-08-19T10:37:23.136 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:23.136 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-08-19T10:37:23.141 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:23.141 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-08-19T10:37:23.188 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi089.front.sepia.ceph.com 2024-08-19T10:37:23.189 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:23.189 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-19T10:37:23.189 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-19T10:37:23.253 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:23.253 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:23.301 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2024-08-19T10:37:23.301 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:23.301 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-19T10:37:23.301 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-19T10:37:23.325 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:23.325 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:23.371 INFO:tasks.ceph:Setting up mon nodes... 2024-08-19T10:37:23.372 INFO:tasks.ceph:Setting up mgr nodes... 2024-08-19T10:37:23.372 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-08-19T10:37:23.455 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-08-19T10:37:23.463 INFO:tasks.ceph:Setting up mds nodes... 2024-08-19T10:37:23.463 INFO:tasks.ceph_client:Setting up client nodes... 2024-08-19T10:37:23.463 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-19T10:37:23.543 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-08-19T10:37:23.548 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-08-19T10:37:23.549 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-08-19T10:37:23.549 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-08-19T10:37:23.563 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-08-19T10:37:23.563 INFO:tasks.ceph:role: osd.0 2024-08-19T10:37:23.563 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com 2024-08-19T10:37:23.563 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-19T10:37:23.623 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T10:37:23.623 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T10:37:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T10:37:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-08-19T10:37:23.624 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T10:37:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2024-08-19T10:37:23.624 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T10:37:23.624 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-19T10:37:23.624 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T10:37:23.624 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T10:37:24.080 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2024-08-19T10:37:24.082 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2024-08-19T10:37:24.082 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-08-19T10:37:24.174 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-08-19T10:37:24.184 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: /sbin/restorecon: command not found 2024-08-19T10:37:24.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T10:37:24.185 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-08-19T10:37:24.238 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-08-19T10:37:24.238 INFO:tasks.ceph:role: osd.1 2024-08-19T10:37:24.238 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com 2024-08-19T10:37:24.239 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-19T10:37:24.300 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T10:37:24.301 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T10:37:24.301 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T10:37:24.301 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-08-19T10:37:24.301 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T10:37:24.301 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2024-08-19T10:37:24.301 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T10:37:24.301 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-19T10:37:24.301 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T10:37:24.301 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T10:37:24.767 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2024-08-19T10:37:24.769 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2024-08-19T10:37:24.770 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-08-19T10:37:24.915 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-08-19T10:37:24.926 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: /sbin/restorecon: command not found 2024-08-19T10:37:24.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T10:37:24.927 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-08-19T10:37:24.980 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-08-19T10:37:24.980 INFO:tasks.ceph:role: osd.2 2024-08-19T10:37:24.980 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com 2024-08-19T10:37:24.980 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-19T10:37:25.044 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T10:37:25.044 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T10:37:25.044 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T10:37:25.044 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-08-19T10:37:25.044 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T10:37:25.044 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2024-08-19T10:37:25.044 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T10:37:25.044 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-19T10:37:25.044 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T10:37:25.045 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T10:37:25.526 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2024-08-19T10:37:25.528 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2024-08-19T10:37:25.529 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-08-19T10:37:25.680 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-08-19T10:37:25.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T10:37:25.693 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: /sbin/restorecon: command not found 2024-08-19T10:37:25.693 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:25.806+0000 7fba6e3717c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-08-19T10:37:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:25.806+0000 7fba6e3717c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-08-19T10:37:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:25.806+0000 7fba6e3717c0 -1 bdev(0x556576d7f800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-08-19T10:37:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:25.806+0000 7fba6e3717c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-08-19T10:37:28.957 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-08-19T10:37:28.971 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:29.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:29.050+0000 7fcebf10c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-08-19T10:37:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:29.050+0000 7fcebf10c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-08-19T10:37:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:29.050+0000 7fcebf10c7c0 -1 bdev(0x55ace4717800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-08-19T10:37:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:29.050+0000 7fcebf10c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-08-19T10:37:32.192 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-08-19T10:37:32.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:32.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:32.330+0000 7f92141267c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-08-19T10:37:32.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:32.330+0000 7f92141267c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-08-19T10:37:32.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:32.330+0000 7f92141267c0 -1 bdev(0x55c0d7fc7800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-08-19T10:37:32.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:32.330+0000 7f92141267c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-08-19T10:37:35.483 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-08-19T10:37:35.536 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-08-19T10:37:35.551 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-08-19T10:37:35.551 INFO:tasks.ceph:role: osd.3 2024-08-19T10:37:35.551 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com 2024-08-19T10:37:35.551 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-19T10:37:35.614 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T10:37:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T10:37:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T10:37:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-08-19T10:37:35.614 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T10:37:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-08-19T10:37:35.614 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T10:37:35.614 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-19T10:37:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T10:37:35.614 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T10:37:36.071 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-08-19T10:37:36.073 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-08-19T10:37:36.073 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-08-19T10:37:36.177 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-08-19T10:37:36.188 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: /sbin/restorecon: command not found 2024-08-19T10:37:36.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T10:37:36.189 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-08-19T10:37:36.242 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-08-19T10:37:36.242 INFO:tasks.ceph:role: osd.4 2024-08-19T10:37:36.242 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com 2024-08-19T10:37:36.242 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-19T10:37:36.306 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T10:37:36.307 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T10:37:36.307 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T10:37:36.307 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-08-19T10:37:36.307 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T10:37:36.307 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-08-19T10:37:36.307 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T10:37:36.307 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-19T10:37:36.307 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T10:37:36.307 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T10:37:36.776 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-08-19T10:37:36.778 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-08-19T10:37:36.778 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-08-19T10:37:36.931 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-08-19T10:37:36.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T10:37:36.943 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: /sbin/restorecon: command not found 2024-08-19T10:37:36.943 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-08-19T10:37:36.998 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-08-19T10:37:36.998 INFO:tasks.ceph:role: osd.5 2024-08-19T10:37:36.998 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com 2024-08-19T10:37:36.998 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-19T10:37:37.062 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-19T10:37:37.062 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-19T10:37:37.062 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-19T10:37:37.062 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-08-19T10:37:37.062 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-19T10:37:37.062 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-08-19T10:37:37.062 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-19T10:37:37.062 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-19T10:37:37.063 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-19T10:37:37.063 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-19T10:37:37.543 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-08-19T10:37:37.545 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-08-19T10:37:37.546 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-08-19T10:37:37.674 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-08-19T10:37:37.686 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: /sbin/restorecon: command not found 2024-08-19T10:37:37.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T10:37:37.687 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:37.813 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:37.809+0000 7f9e6285c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-08-19T10:37:37.813 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:37.813+0000 7f9e6285c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-08-19T10:37:37.814 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:37.813+0000 7f9e6285c7c0 -1 bdev(0x5622643f9800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-08-19T10:37:37.814 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:37.813+0000 7f9e6285c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-08-19T10:37:40.943 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-08-19T10:37:40.957 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:41.040 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:41.037+0000 7f88c4f277c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-08-19T10:37:41.040 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:41.037+0000 7f88c4f277c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-08-19T10:37:41.041 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:41.037+0000 7f88c4f277c0 -1 bdev(0x5642999a3800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-08-19T10:37:41.041 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:41.037+0000 7f88c4f277c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-08-19T10:37:44.196 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-08-19T10:37:44.251 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:44.331 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:44.329+0000 7fd427d687c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-08-19T10:37:44.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:44.329+0000 7fd427d687c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-08-19T10:37:44.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:44.329+0000 7fd427d687c0 -1 bdev(0x56304405d800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-08-19T10:37:44.332 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:44.329+0000 7fd427d687c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-08-19T10:37:47.477 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-08-19T10:37:47.531 INFO:tasks.ceph:Reading keys from all nodes... 2024-08-19T10:37:47.531 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:47.531 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-08-19T10:37:47.545 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:47.545 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-08-19T10:37:47.600 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:47.600 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-08-19T10:37:47.659 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:47.659 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-08-19T10:37:47.672 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:47.672 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-08-19T10:37:47.725 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:47.725 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-08-19T10:37:47.780 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:47.781 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-08-19T10:37:47.833 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:47.834 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-08-19T10:37:47.882 INFO:tasks.ceph:Adding keys to all mons... 2024-08-19T10:37:47.882 DEBUG:teuthology.orchestra.run.smithi089:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-19T10:37:47.885 DEBUG:teuthology.orchestra.run.smithi190:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-19T10:37:47.928 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.0] 2024-08-19T10:37:47.928 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBlIMNmV2lNMBAAYf1JnWq4q5atpL0N9hREHw== 2024-08-19T10:37:47.928 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.1] 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBpIMNmxPlMAxAAYoWKnYdoSOQlLRPxM0nFMA== 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.2] 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBsIMNmEdjxExAANbhB09ELi0684/DOeOgSxw== 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr.x] 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBjIMNm854/GxAANy95i6zqBHIzO0B9oS6V3A== 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.3] 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBxIMNmgVSTMBAAVdWZ9ToNXoXKex3go1jiww== 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.4] 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQB1IMNmJBiBAhAA66J9HGxl2uw6GGRZtaaL3Q== 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.5] 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQB4IMNmr7vkExAAch29rwyi/oa1SHUtMp84Rw== 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2024-08-19T10:37:47.929 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBjIMNms1Y/IBAASPlHuAgdxRnu/lHvLahmQg== 2024-08-19T10:37:47.936 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.0] 2024-08-19T10:37:47.936 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBlIMNmV2lNMBAAYf1JnWq4q5atpL0N9hREHw== 2024-08-19T10:37:47.936 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.1] 2024-08-19T10:37:47.936 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBpIMNmxPlMAxAAYoWKnYdoSOQlLRPxM0nFMA== 2024-08-19T10:37:47.936 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.2] 2024-08-19T10:37:47.936 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBsIMNmEdjxExAANbhB09ELi0684/DOeOgSxw== 2024-08-19T10:37:47.936 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.x] 2024-08-19T10:37:47.936 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBjIMNm854/GxAANy95i6zqBHIzO0B9oS6V3A== 2024-08-19T10:37:47.937 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.3] 2024-08-19T10:37:47.937 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBxIMNmgVSTMBAAVdWZ9ToNXoXKex3go1jiww== 2024-08-19T10:37:47.937 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.4] 2024-08-19T10:37:47.937 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQB1IMNmJBiBAhAA66J9HGxl2uw6GGRZtaaL3Q== 2024-08-19T10:37:47.937 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.5] 2024-08-19T10:37:47.937 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQB4IMNmr7vkExAAch29rwyi/oa1SHUtMp84Rw== 2024-08-19T10:37:47.937 INFO:teuthology.orchestra.run.smithi190.stdout:[client.0] 2024-08-19T10:37:47.937 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBjIMNms1Y/IBAASPlHuAgdxRnu/lHvLahmQg== 2024-08-19T10:37:47.938 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:47.973 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:48.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:48.057 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:48.120 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:48.141 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:48.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-19T10:37:48.225 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-19T10:37:48.281 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:48.309 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:48.353 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:48.397 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:48.442 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:48.481 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-19T10:37:48.529 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-19T10:37:48.565 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-19T10:37:48.617 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-08-19T10:37:48.617 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-08-19T10:37:48.662 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T10:37:48.747 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-08-19T10:37:48.807 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-08-19T10:37:48.864 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T10:37:48.959 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-08-19T10:37:48.970 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-08-19T10:37:49.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T10:37:49.110 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-08-19T10:37:49.120 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-d 2024-08-19T10:37:49.174 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i d --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T10:37:49.262 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-d 2024-08-19T10:37:49.273 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-e 2024-08-19T10:37:49.325 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i e --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T10:37:49.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-e 2024-08-19T10:37:49.467 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-f 2024-08-19T10:37:49.482 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i f --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T10:37:49.582 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-f 2024-08-19T10:37:49.592 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-g 2024-08-19T10:37:49.643 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i g --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T10:37:49.732 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-g 2024-08-19T10:37:49.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-h 2024-08-19T10:37:49.796 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i h --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T10:37:49.882 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-h 2024-08-19T10:37:49.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-i 2024-08-19T10:37:49.943 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i i --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-19T10:37:50.031 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-i 2024-08-19T10:37:50.041 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:50.044 DEBUG:teuthology.orchestra.run.smithi190:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-19T10:37:50.089 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-08-19T10:37:50.090 INFO:tasks.ceph.mon.a:Restarting daemon 2024-08-19T10:37:50.090 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-08-19T10:37:50.093 INFO:tasks.ceph.mon.a:Started 2024-08-19T10:37:50.093 INFO:tasks.ceph.mon.b:Restarting daemon 2024-08-19T10:37:50.093 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-08-19T10:37:50.095 INFO:tasks.ceph.mon.b:Started 2024-08-19T10:37:50.095 INFO:tasks.ceph.mon.c:Restarting daemon 2024-08-19T10:37:50.095 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-08-19T10:37:50.098 INFO:tasks.ceph.mon.c:Started 2024-08-19T10:37:50.098 INFO:tasks.ceph.mon.d:Restarting daemon 2024-08-19T10:37:50.098 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-08-19T10:37:50.100 INFO:tasks.ceph.mon.d:Started 2024-08-19T10:37:50.100 INFO:tasks.ceph.mon.e:Restarting daemon 2024-08-19T10:37:50.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-08-19T10:37:50.102 INFO:tasks.ceph.mon.e:Started 2024-08-19T10:37:50.102 INFO:tasks.ceph.mon.f:Restarting daemon 2024-08-19T10:37:50.102 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-08-19T10:37:50.135 INFO:tasks.ceph.mon.f:Started 2024-08-19T10:37:50.135 INFO:tasks.ceph.mon.g:Restarting daemon 2024-08-19T10:37:50.135 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-08-19T10:37:50.137 INFO:tasks.ceph.mon.g:Started 2024-08-19T10:37:50.138 INFO:tasks.ceph.mon.h:Restarting daemon 2024-08-19T10:37:50.138 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-08-19T10:37:50.140 INFO:tasks.ceph.mon.h:Started 2024-08-19T10:37:50.140 INFO:tasks.ceph.mon.i:Restarting daemon 2024-08-19T10:37:50.140 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-08-19T10:37:50.142 INFO:tasks.ceph.mon.i:Started 2024-08-19T10:37:50.142 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-08-19T10:37:50.142 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-08-19T10:37:50.142 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-08-19T10:37:50.144 INFO:tasks.ceph.mgr.x:Started 2024-08-19T10:37:50.145 DEBUG:tasks.ceph:set 0 configs 2024-08-19T10:37:50.145 INFO:tasks.ceph:Setting crush tunables to default 2024-08-19T10:37:50.145 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd crush tunables default 2024-08-19T10:37:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient: get_monmap_and_config 2024-08-19T10:37:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient: build_initial_monmap 2024-08-19T10:37:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient: monmap: 2024-08-19T10:37:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:37:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:50.285898+0000 2024-08-19T10:37:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:50.285898+0000 2024-08-19T10:37:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:50.285 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:50.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 1 Processor -- start 2024-08-19T10:37:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 1 -- start start 2024-08-19T10:37:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient: init 2024-08-19T10:37:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient: _add_conns ranks=[9,10,8,1,3,2,0,5,4,6,7] 2024-08-19T10:37:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient(hunting): picked mon.noname-a con 0x7f73240a56b0 addr v2:172.21.15.89:3300/0 2024-08-19T10:37:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient(hunting): picked mon.noname-f con 0x7f73240a1660 addr v2:172.21.15.190:3300/0 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 0x7f7324137460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient(hunting): picked mon.noname-e con 0x7f7324072250 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f73240a6510 con 0x7f73240a56b0 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7324074010 con 0x7f73240a1660 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7324072010 con 0x7f7324072250 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732c737640 10 monclient(hunting): authenticate will time out at 920.102177s 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732a4d3640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732acd4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 0x7f7324137460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732a4d3640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42722/0 (socket says 172.21.15.89:42722) 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732a4d3640 1 -- 172.21.15.89:0/30823316 learned_addr learned my addr 172.21.15.89:0/30823316 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732a4d3640 10 monclient(hunting): get_auth_request con 0x7f73240a56b0 auth_method 0 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732a4d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732a4d3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732a4d3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732acd4640 10 monclient(hunting): get_auth_request con 0x7f7324072250 auth_method 0 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732acd4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732acd4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732acd4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732a4d3640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 msgr2=0x7f73240a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732a4d3640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 msgr2=0x7f73240a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:37:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732acd4640 1 -- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 msgr2=0x7f7324137460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732acd4640 1 -- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 msgr2=0x7f7324137460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732acd4640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 0x7f7324137460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f732acd4640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 0x7f7324137460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f7329cd2640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f7329cd2640 10 monclient(hunting): get_auth_request con 0x7f73240a1660 auth_method 0 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f7329cd2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f7329cd2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f7329cd2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f7329cd2640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 msgr2=0x7f73240a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f7329cd2640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 msgr2=0x7f73240a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:37:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f7329cd2640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:37:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.282+0000 7f7329cd2640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:37:50.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f7329cd2640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:50.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f7329cd2640 10 monclient(hunting): get_auth_request con 0x7f73240a1660 auth_method 0 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f7329cd2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f7329cd2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f7329cd2640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f7329cd2640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 msgr2=0x7f73240a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f7329cd2640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 msgr2=0x7f73240a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f7329cd2640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f7329cd2640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732acd4640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 0x7f7324137460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732a4d3640 10 monclient(hunting): get_auth_request con 0x7f73240a56b0 auth_method 0 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732a4d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:50.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732a4d3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732a4d3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732acd4640 10 monclient(hunting): get_auth_request con 0x7f7324072250 auth_method 0 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732acd4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732acd4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732acd4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732a4d3640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 msgr2=0x7f73240a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732a4d3640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 msgr2=0x7f73240a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732acd4640 1 -- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 msgr2=0x7f7324137460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732acd4640 1 -- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 msgr2=0x7f7324137460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:37:50.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732acd4640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 0x7f7324137460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:37:50.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.486+0000 7f732acd4640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 0x7f7324137460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:37:50.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f7329cd2640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:50.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:50.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732acd4640 1 -- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 msgr2=0x7f7324137460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:37:50.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732acd4640 1 -- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 msgr2=0x7f7324137460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:37:50.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732acd4640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 0x7f7324137460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2024-08-19T10:37:50.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732a4d3640 10 monclient(hunting): get_auth_request con 0x7f73240a56b0 auth_method 0 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732a4d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732a4d3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732a4d3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732acd4640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 0x7f7324137460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f7329cd2640 10 monclient(hunting): get_auth_request con 0x7f73240a1660 auth_method 0 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f7329cd2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f7329cd2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f7329cd2640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732a4d3640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 msgr2=0x7f73240a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732a4d3640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 msgr2=0x7f73240a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:37:50.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:37:50.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f7329cd2640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 msgr2=0x7f73240a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:37:50.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f7329cd2640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 msgr2=0x7f73240a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:37:50.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f7329cd2640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:37:50.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:50.890+0000 7f7329cd2640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.89:3300/0 conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 10 monclient(hunting): get_auth_request con 0x7f73240a56b0 auth_method 0 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 10 monclient(hunting): handle_auth_done global_id 4113 payload 274 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 1 -- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 msgr2=0x7f7324137460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 0x7f7324137460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 1 -- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 msgr2=0x7f73240a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:51.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 10 monclient: _finish_hunting 0 2024-08-19T10:37:51.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 1 monclient: found mon.noname-a 2024-08-19T10:37:51.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:51.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:37:51.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.694+0000 7f732a4d3640 1 -- 172.21.15.89:0/30823316 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7324073d30 con 0x7f73240a56b0 2024-08-19T10:37:51.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732a4d3640 1 -- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a56b0 msgr2=0x7f73240a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:51.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732a4d3640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a56b0 0x7f73240a5a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f73240aad60 tx=0x7f732001cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fcc612b26d6630 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:51.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 1 -- 172.21.15.89:0/30823316 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73200ab070 con 0x7f73240a56b0 2024-08-19T10:37:51.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:51.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:37:51.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 10 monclient: dump: 2024-08-19T10:37:51.705 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:37:51.705 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:51.706 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 10 monclient: _finish_auth 0 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 10 monclient: _check_auth_tickets 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 1 -- 172.21.15.89:0/30823316 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73200044a0 con 0x7f73240a56b0 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 5 monclient: authenticate success, global_id 4113 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 10 monclient: get_monmap_and_config success 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 1 -- 172.21.15.89:0/30823316 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f732000f040 con 0x7f73240a56b0 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 4 set_mon_vals no callback set 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:51.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f73294d1640 10 monclient: dump: 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:51.708 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:51.709 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:51.709 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:51.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 10 monclient: shutdown 2024-08-19T10:37:51.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:51.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 1 -- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a56b0 msgr2=0x7f73240a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:51.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a56b0 0x7f73240a5a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f73240aad60 tx=0x7f732001cb30 comp rx=0 tx=0).stop 2024-08-19T10:37:51.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 1 -- 172.21.15.89:0/30823316 shutdown_connections 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7324072250 0x7f7324137460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 1 --2- 172.21.15.89:0/30823316 >> v2:172.21.15.190:3300/0 conn(0x7f73240a1660 0x7f73240a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 1 --2- 172.21.15.89:0/30823316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a56b0 0x7f73240a5a90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 1 -- 172.21.15.89:0/30823316 >> 172.21.15.89:0/30823316 conn(0x7f73240aa2a0 msgr2=0x7f73240a67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 1 -- 172.21.15.89:0/30823316 shutdown_connections 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.702+0000 7f732c737640 1 -- 172.21.15.89:0/30823316 wait complete. 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient: build_initial_monmap 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient: monmap: 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:37:51.711 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 1 Processor -- start 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 1 -- start start 2024-08-19T10:37:51.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient: init 2024-08-19T10:37:51.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:51.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient: _add_conns ranks=[8,3,0,1,4,7,2,5,6] 2024-08-19T10:37:51.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73240a56b0 0x7f7324135ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:51.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient(hunting): picked mon.noname-e con 0x7f73240a56b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:37:51.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73241363e0 0x7f732413b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient(hunting): picked mon.noname-g con 0x7f73241363e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f732413b9a0 0x7f732413bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient(hunting): picked mon.noname-a con 0x7f732413b9a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7324141260 con 0x7f732413b9a0 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f73241410e0 con 0x7f73241363e0 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f73241413e0 con 0x7f73240a56b0 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732c737640 10 monclient(hunting): authenticate will time out at 921.527569s 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732a4d3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73240a56b0 0x7f7324135ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:51.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732acd4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f732413b9a0 0x7f732413bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732a4d3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73240a56b0 0x7f7324135ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:33174/0 (socket says 172.21.15.89:33174) 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732a4d3640 1 -- 172.21.15.89:0/2279008654 learned_addr learned my addr 172.21.15.89:0/2279008654 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732a4d3640 10 monclient(hunting): get_auth_request con 0x7f73240a56b0 auth_method 0 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732a4d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732a4d3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732a4d3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732acd4640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f732413b9a0 0x7f732413bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57890/0 (socket says 172.21.15.89:57890) 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732acd4640 10 monclient(hunting): get_auth_request con 0x7f732413b9a0 auth_method 0 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732acd4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732acd4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.706+0000 7f732acd4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732a4d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732a4d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:51.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732a4d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73241363e0 0x7f732413b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): get_auth_request con 0x7f73241363e0 auth_method 0 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient(hunting): handle_auth_done global_id 4122 payload 824 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 -- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73240a56b0 msgr2=0x7f7324135ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:51.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73240a56b0 0x7f7324135ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 -- 172.21.15.89:0/2279008654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73241363e0 msgr2=0x7f732413b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73241363e0 0x7f732413b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient: _finish_hunting 0 2024-08-19T10:37:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 monclient: found mon.noname-a 2024-08-19T10:37:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:37:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 -- 172.21.15.89:0/2279008654 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732413d1c0 con 0x7f732413b9a0 2024-08-19T10:37:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 -- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f732413b9a0 msgr2=0x7f732413bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f732413b9a0 0x7f732413bde0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f731c009610 tx=0x7f731c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 -- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f732413b9a0 msgr2=0x7f732413bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:37:51.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 -- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f732413b9a0 msgr2=0x7f732413bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f732413b9a0 0x7f732413bde0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f731c009610 tx=0x7f731c007700 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f732413b9a0 0x7f732413bde0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f731c009610 tx=0x7f731c007700 comp rx=0 tx=0).stop 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 10 monclient: ms_handle_reset current mon [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 1 -- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f732413b9a0 msgr2=0x7f732413bde0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f732413b9a0 0x7f732413bde0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 10 monclient: _add_conns ranks=[7,4,2,6,5,1] 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73080035e0 0x7f7308005a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 10 monclient(hunting): picked mon.noname-i con 0x7f73080035e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7308006580 0x7f7308008a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 10 monclient(hunting): picked mon.noname-c con 0x7f7308006580 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73080095b0 0x7f730800ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:51.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 10 monclient(hunting): picked mon.noname-b con 0x7f73080095b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 1 -- 172.21.15.89:0/2279008654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f730800c660 con 0x7f73080095b0 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 1 -- 172.21.15.89:0/2279008654 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f730800c7e0 con 0x7f7308006580 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 1 -- 172.21.15.89:0/2279008654 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f730800c9e0 con 0x7f73080035e0 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f731affd640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7308006580 0x7f7308008a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): get_auth_request con 0x7f7308006580 auth_method 0 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73080095b0 0x7f730800ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient(hunting): get_auth_request con 0x7f73080095b0 auth_method 0 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f7329cd2640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732acd4640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732a4d3640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73080035e0 0x7f7308005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732a4d3640 10 monclient(hunting): get_auth_request con 0x7f73080035e0 auth_method 0 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732a4d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732a4d3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.710+0000 7f732a4d3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 10 monclient(hunting): handle_auth_done global_id 4122 payload 293 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 1 -- 172.21.15.89:0/2279008654 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73080035e0 msgr2=0x7f7308005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73080035e0 0x7f7308005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 1 -- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7308006580 msgr2=0x7f7308008a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7308006580 0x7f7308008a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 10 monclient: _finish_hunting 0 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 1 monclient: found mon.noname-b 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 1 -- 172.21.15.89:0/2279008654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730800cca0 con 0x7f73080095b0 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732a4d3640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73080035e0 0x7f7308005a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:37:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 1 -- 172.21.15.89:0/2279008654 reap_dead start 2024-08-19T10:37:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 1 -- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73080095b0 msgr2=0x7f730800ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732acd4640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73080095b0 0x7f730800ba70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f731c018000 tx=0x7f731c003bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 1 -- 172.21.15.89:0/2279008654 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f731c007e70 con 0x7f73080095b0 2024-08-19T10:37:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 10 monclient: dump: 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 10 monclient: _finish_auth 0 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 10 monclient: _check_auth_tickets 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 1 -- 172.21.15.89:0/2279008654 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731c019070 con 0x7f73080095b0 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732c737640 5 monclient: authenticate success, global_id 4122 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732c737640 10 monclient: _renew_subs 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732c737640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:37:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732c737640 1 -- 172.21.15.89:0/2279008654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73240a2b80 con 0x7f73080095b0 2024-08-19T10:37:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 1 -- 172.21.15.89:0/2279008654 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f731c014410 con 0x7f73080095b0 2024-08-19T10:37:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:37:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f7328cd0640 4 set_mon_vals no callback set 2024-08-19T10:37:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 10 monclient: dump: 2024-08-19T10:37:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:37:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732c737640 10 monclient: _renew_subs 2024-08-19T10:37:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732c737640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:37:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732c737640 1 -- 172.21.15.89:0/2279008654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7324121a10 con 0x7f73080095b0 2024-08-19T10:37:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 1 -- 172.21.15.89:0/2279008654 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f731c013990 con 0x7f73080095b0 2024-08-19T10:37:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f731affd640 1 -- 172.21.15.89:0/2279008654 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f731c014c80 con 0x7f73080095b0 2024-08-19T10:37:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.714+0000 7f732c737640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.718+0000 7f732c737640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.718+0000 7f732c737640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:37:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.718+0000 7f732c737640 1 -- 172.21.15.89:0/2279008654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7324064590 con 0x7f73080095b0 2024-08-19T10:37:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.718+0000 7f731affd640 1 -- 172.21.15.89:0/2279008654 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f731c013c80 con 0x7f73080095b0 2024-08-19T10:37:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.718+0000 7f731affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.718+0000 7f731affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:37:51.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.786+0000 7f732c737640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2024-08-19T10:37:51.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.786+0000 7f732c737640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-08-19T10:37:51.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.786+0000 7f732c737640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:37:51.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.786+0000 7f732c737640 1 -- 172.21.15.89:0/2279008654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f7324137430 con 0x7f73080095b0 2024-08-19T10:37:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.794+0000 7f731affd640 1 -- 172.21.15.89:0/2279008654 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f731c0145f0 con 0x7f73080095b0 2024-08-19T10:37:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.794+0000 7f731affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-08-19T10:37:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.794+0000 7f731affd640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2024-08-19T10:37:51.799 INFO:teuthology.orchestra.run.smithi089.stderr:adjusted tunables profile to default 2024-08-19T10:37:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.798+0000 7f732c737640 10 monclient: shutdown 2024-08-19T10:37:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.798+0000 7f732c737640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.798+0000 7f732c737640 1 -- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73080095b0 msgr2=0x7f730800ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.798+0000 7f732c737640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73080095b0 0x7f730800ba70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f731c018000 tx=0x7f731c003bb0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.798+0000 7f732c737640 1 -- 172.21.15.89:0/2279008654 shutdown_connections 2024-08-19T10:37:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.798+0000 7f732c737640 1 --2- 172.21.15.89:0/2279008654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73080095b0 0x7f730800ba70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.798+0000 7f732c737640 1 -- 172.21.15.89:0/2279008654 >> 172.21.15.89:0/2279008654 conn(0x7f73240aa2a0 msgr2=0x7f7324072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.798+0000 7f732c737640 1 -- 172.21.15.89:0/2279008654 shutdown_connections 2024-08-19T10:37:51.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:51.798+0000 7f732c737640 1 -- 172.21.15.89:0/2279008654 wait complete. 2024-08-19T10:37:51.818 INFO:tasks.ceph:check_enable_crimson: False 2024-08-19T10:37:51.818 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-08-19T10:37:51.819 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:51.819 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-08-19T10:37:51.831 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:51.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-08-19T10:37:51.883 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-08-19T10:37:51.883 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-08-19T10:37:51.939 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:51.940 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-08-19T10:37:51.953 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:51.953 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-08-19T10:37:52.009 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:37:52.009 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-08-19T10:37:52.059 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 2be543b2-eb94-4907-8f21-6e91b7dd9733 0 2024-08-19T10:37:52.219 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient: get_monmap_and_config 2024-08-19T10:37:52.219 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient: build_initial_monmap 2024-08-19T10:37:52.219 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient: monmap: 2024-08-19T10:37:52.219 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:52.221380+0000 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:52.221380+0000 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:52.220 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:52.221 INFO:teuthology.orchestra.run.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:37:52.221 INFO:teuthology.orchestra.run.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:37:52.221 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 1 Processor -- start 2024-08-19T10:37:52.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 1 -- start start 2024-08-19T10:37:52.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient: init 2024-08-19T10:37:52.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:52.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient: _add_conns ranks=[9,7,2,3,1,0,10,4,6,5,8] 2024-08-19T10:37:52.223 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9a3006de10 0x7f9a30135ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient(hunting): picked mon.noname-a con 0x7f9a3006de10 addr v2:172.21.15.89:3300/0 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a30139200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient(hunting): picked mon.noname-i con 0x7f9a30073f00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a30074320 0x7f9a30139740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient(hunting): picked mon.noname-b con 0x7f9a30074320 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f9a3006e1f0 con 0x7f9a3006de10 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3006f190 con 0x7f9a30074320 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9a3006f470 con 0x7f9a30073f00 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:52.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a3637a640 10 monclient(hunting): authenticate will time out at 917.791264s 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a30139200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a30139200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:47740/0 (socket says 172.21.15.190:47740) 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 1 -- 172.21.15.190:0/59504582 learned_addr learned my addr 172.21.15.190:0/59504582 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 10 monclient(hunting): get_auth_request con 0x7f9a30073f00 auth_method 0 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 10 monclient(hunting): handle_auth_done global_id 4120 payload 274 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a34917640 1 --2- 172.21.15.190:0/59504582 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a30074320 0x7f9a30139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 1 -- 172.21.15.190:0/59504582 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a30074320 msgr2=0x7f9a30139740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:52.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2ffff640 1 --2- 172.21.15.190:0/59504582 >> v2:172.21.15.89:3300/0 conn(0x7f9a3006de10 0x7f9a30135ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 1 --2- 172.21.15.190:0/59504582 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a30074320 0x7f9a30139740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 1 -- 172.21.15.190:0/59504582 >> v2:172.21.15.89:3300/0 conn(0x7f9a3006de10 msgr2=0x7f9a30135ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:52.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 1 --2- 172.21.15.190:0/59504582 >> v2:172.21.15.89:3300/0 conn(0x7f9a3006de10 0x7f9a30135ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:37:52.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 1 monclient: found mon.noname-i 2024-08-19T10:37:52.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:52.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:37:52.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2f7fe640 1 -- 172.21.15.190:0/59504582 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a3006f750 con 0x7f9a30073f00 2024-08-19T10:37:52.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2ffff640 10 monclient: get_auth_request con 0x7f9a3006de10 auth_method 0 2024-08-19T10:37:52.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.217+0000 7f9a2ffff640 1 --2- 172.21.15.190:0/59504582 >> v2:172.21.15.89:3300/0 conn(0x7f9a3006de10 0x7f9a30135ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:37:52.235 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.221+0000 7f9a2f7fe640 1 -- 172.21.15.190:0/59504582 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 msgr2=0x7f9a30139200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:52.235 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.221+0000 7f9a2f7fe640 1 --2- 172.21.15.190:0/59504582 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a30139200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009870 tx=0x7f9a2401c420 comp rx=0 tx=0).ready entity=mon.7 client_cookie=a3bc946b54785d12 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:52.235 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 1 -- 172.21.15.190:0/59504582 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a240b0070 con 0x7f9a30073f00 2024-08-19T10:37:52.235 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:52.235 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:37:52.235 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 10 monclient: dump: 2024-08-19T10:37:52.235 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:52.235 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:52.235 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.236 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.236 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:52.236 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.236 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:52.236 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:52.236 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:52.236 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:52.236 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:52.236 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:52.236 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:52.236 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 10 monclient: _finish_auth 0 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 10 monclient: _check_auth_tickets 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 1 -- 172.21.15.190:0/59504582 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a240aa810 con 0x7f9a30073f00 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 5 monclient: authenticate success, global_id 4120 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient: get_monmap_and_config success 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 1 -- 172.21.15.190:0/59504582 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a240a93e0 con 0x7f9a30073f00 2024-08-19T10:37:52.237 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:52.238 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 4 set_mon_vals no callback set 2024-08-19T10:37:52.238 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:37:52.238 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2effd640 10 monclient: dump: 2024-08-19T10:37:52.238 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:52.238 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:52.238 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.238 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.238 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:52.238 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.238 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:52.238 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient: shutdown 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 -- 172.21.15.190:0/59504582 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 msgr2=0x7f9a30139200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:52.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 --2- 172.21.15.190:0/59504582 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a30139200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009870 tx=0x7f9a2401c420 comp rx=0 tx=0).stop 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 -- 172.21.15.190:0/59504582 shutdown_connections 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 --2- 172.21.15.190:0/59504582 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a30074320 0x7f9a30139740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 --2- 172.21.15.190:0/59504582 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a30139200 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 --2- 172.21.15.190:0/59504582 >> v2:172.21.15.89:3300/0 conn(0x7f9a3006de10 0x7f9a30135ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 -- 172.21.15.190:0/59504582 >> 172.21.15.190:0/59504582 conn(0x7f9a300aa240 msgr2=0x7f9a300aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 -- 172.21.15.190:0/59504582 shutdown_connections 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 -- 172.21.15.190:0/59504582 wait complete. 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient: build_initial_monmap 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient: monmap: 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:37:52.240 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 0.000000 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:created 0.000000 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:52.241 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:52.242 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 Processor -- start 2024-08-19T10:37:52.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 -- start start 2024-08-19T10:37:52.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient: init 2024-08-19T10:37:52.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:52.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient: _add_conns ranks=[2,7,6,5,8,1,3,0,4] 2024-08-19T10:37:52.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a3006de10 0x7f9a301d1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient(hunting): picked mon.noname-b con 0x7f9a3006de10 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:37:52.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a301d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient(hunting): picked mon.noname-i con 0x7f9a30073f00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:37:52.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9a30074320 0x7f9a3013a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient(hunting): picked mon.noname-d con 0x7f9a30074320 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:37:52.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9a30142fc0 con 0x7f9a3006de10 2024-08-19T10:37:52.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9a3013a090 con 0x7f9a30074320 2024-08-19T10:37:52.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9a30143140 con 0x7f9a30073f00 2024-08-19T10:37:52.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:52.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a3637a640 10 monclient(hunting): authenticate will time out at 917.798299s 2024-08-19T10:37:52.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a301d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.243 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a301d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:47746/0 (socket says 172.21.15.190:47746) 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 1 -- 172.21.15.190:0/163561310 learned_addr learned my addr 172.21.15.190:0/163561310 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 10 monclient(hunting): get_auth_request con 0x7f9a30073f00 auth_method 0 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2ffff640 1 --2- 172.21.15.190:0/163561310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a3006de10 0x7f9a301d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a34917640 1 --2- 172.21.15.190:0/163561310 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9a30074320 0x7f9a3013a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2ffff640 10 monclient(hunting): get_auth_request con 0x7f9a3006de10 auth_method 0 2024-08-19T10:37:52.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a34917640 10 monclient(hunting): get_auth_request con 0x7f9a30074320 auth_method 0 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a34917640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a34917640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a34917640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 10 monclient(hunting): handle_auth_done global_id 4129 payload 824 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 1 -- 172.21.15.190:0/163561310 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9a30074320 msgr2=0x7f9a3013a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 1 --2- 172.21.15.190:0/163561310 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9a30074320 0x7f9a3013a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 1 -- 172.21.15.190:0/163561310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a3006de10 msgr2=0x7f9a301d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 1 --2- 172.21.15.190:0/163561310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a3006de10 0x7f9a301d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:37:52.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 1 monclient: found mon.noname-i 2024-08-19T10:37:52.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:52.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:37:52.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.225+0000 7f9a2f7fe640 1 -- 172.21.15.190:0/163561310 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a3013bac0 con 0x7f9a30073f00 2024-08-19T10:37:52.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2f7fe640 1 -- 172.21.15.190:0/163561310 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 msgr2=0x7f9a301d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:52.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2f7fe640 1 --2- 172.21.15.190:0/163561310 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a301d1a50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9a240ae040 tx=0x7f9a240191c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:52.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 1 -- 172.21.15.190:0/163561310 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a240b9070 con 0x7f9a30073f00 2024-08-19T10:37:52.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:52.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:37:52.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 10 monclient: dump: 2024-08-19T10:37:52.246 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:52.247 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 10 monclient: _finish_auth 0 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 10 monclient: _check_auth_tickets 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 1 -- 172.21.15.190:0/163561310 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a24004250 con 0x7f9a30073f00 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a3637a640 5 monclient: authenticate success, global_id 4129 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a3637a640 10 monclient: _renew_subs 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a3637a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a3637a640 1 -- 172.21.15.190:0/163561310 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a300a5780 con 0x7f9a30073f00 2024-08-19T10:37:52.248 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:52.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 1 -- 172.21.15.190:0/163561310 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a240b0040 con 0x7f9a30073f00 2024-08-19T10:37:52.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:52.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:37:52.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 10 monclient: dump: 2024-08-19T10:37:52.249 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:52.249 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:52.249 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.249 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.249 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:52.249 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.249 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2e7fc640 4 set_mon_vals no callback set 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a3637a640 10 monclient: _renew_subs 2024-08-19T10:37:52.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a3637a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:37:52.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a3637a640 1 -- 172.21.15.190:0/163561310 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a3013de20 con 0x7f9a30073f00 2024-08-19T10:37:52.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 1 -- 172.21.15.190:0/163561310 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9a24003620 con 0x7f9a30073f00 2024-08-19T10:37:52.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a2cff9640 1 -- 172.21.15.190:0/163561310 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f9a240b4420 con 0x7f9a30073f00 2024-08-19T10:37:52.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a3637a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:52.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a3637a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:52.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a3637a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:37:52.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.229+0000 7f9a3637a640 1 -- 172.21.15.190:0/163561310 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a30135ff0 con 0x7f9a30073f00 2024-08-19T10:37:52.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.233+0000 7f9a2cff9640 1 -- 172.21.15.190:0/163561310 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9a24016130 con 0x7f9a30073f00 2024-08-19T10:37:52.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.233+0000 7f9a2cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:52.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.233+0000 7f9a2cff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:37:52.314 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.313+0000 7f9a3637a640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "2be543b2-eb94-4907-8f21-6e91b7dd9733", "id": 0}] 2024-08-19T10:37:52.314 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.313+0000 7f9a3637a640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "2be543b2-eb94-4907-8f21-6e91b7dd9733", "id": 0}] 2024-08-19T10:37:52.314 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.313+0000 7f9a3637a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:37:52.314 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.313+0000 7f9a3637a640 1 -- 172.21.15.190:0/163561310 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd new", "uuid": "2be543b2-eb94-4907-8f21-6e91b7dd9733", "id": 0} v 0) -- 0x7f9a30064590 con 0x7f9a30073f00 2024-08-19T10:37:52.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a2cff9640 1 -- 172.21.15.190:0/163561310 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2be543b2-eb94-4907-8f21-6e91b7dd9733", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f9a24003a50 con 0x7f9a30073f00 2024-08-19T10:37:52.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a2cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "2be543b2-eb94-4907-8f21-6e91b7dd9733", "id": 0}] 2024-08-19T10:37:52.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a2cff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:37:52.323 INFO:teuthology.orchestra.run.smithi190.stdout:0 2024-08-19T10:37:52.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a3637a640 10 monclient: shutdown 2024-08-19T10:37:52.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a3637a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:52.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a3637a640 1 -- 172.21.15.190:0/163561310 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 msgr2=0x7f9a301d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:52.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a3637a640 1 --2- 172.21.15.190:0/163561310 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a301d1a50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9a240ae040 tx=0x7f9a240191c0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a3637a640 1 -- 172.21.15.190:0/163561310 shutdown_connections 2024-08-19T10:37:52.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a3637a640 1 --2- 172.21.15.190:0/163561310 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9a30074320 0x7f9a3013a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a3637a640 1 --2- 172.21.15.190:0/163561310 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a30073f00 0x7f9a301d1a50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a3637a640 1 --2- 172.21.15.190:0/163561310 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a3006de10 0x7f9a301d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a3637a640 1 -- 172.21.15.190:0/163561310 >> 172.21.15.190:0/163561310 conn(0x7f9a300aa240 msgr2=0x7f9a30134710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:52.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a3637a640 1 -- 172.21.15.190:0/163561310 shutdown_connections 2024-08-19T10:37:52.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.321+0000 7f9a3637a640 1 -- 172.21.15.190:0/163561310 wait complete. 2024-08-19T10:37:52.339 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 224c1713-25c8-4a48-b16e-8929fcb6424b 1 2024-08-19T10:37:52.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.449+0000 7f4985ca4640 10 monclient: get_monmap_and_config 2024-08-19T10:37:52.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.449+0000 7f4985ca4640 10 monclient: build_initial_monmap 2024-08-19T10:37:52.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.449+0000 7f4985ca4640 10 monclient: monmap: 2024-08-19T10:37:52.452 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:52.452 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:52.454285+0000 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:52.454285+0000 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:52.453 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:52.454 INFO:teuthology.orchestra.run.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:37:52.454 INFO:teuthology.orchestra.run.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:37:52.454 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.455 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 1 Processor -- start 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 1 -- start start 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 10 monclient: init 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 10 monclient: _add_conns ranks=[4,6,5,1,2,3,8,9,0,10,7] 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f49800a4740 0x7f49800a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 10 monclient(hunting): picked mon.noname-c con 0x7f49800a4740 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f49800a5060 0x7f498013a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 10 monclient(hunting): picked mon.noname-d con 0x7f49800a5060 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f498013aa50 0x7f498013ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 10 monclient(hunting): picked mon.noname-h con 0x7f498013aa50 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.456 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f49800a0430 con 0x7f49800a4740 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f49800a08b0 con 0x7f498013aa50 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f498006d420 con 0x7f49800a5060 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f4985ca4640 10 monclient(hunting): authenticate will time out at 918.025046s 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f498013aa50 0x7f498013ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f498013aa50 0x7f498013ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:43890/0 (socket says 172.21.15.190:43890) 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 1 -- 172.21.15.190:0/3808372554 learned_addr learned my addr 172.21.15.190:0/3808372554 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 10 monclient(hunting): get_auth_request con 0x7f498013aa50 auth_method 0 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:52.457 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497f7fe640 1 --2- 172.21.15.190:0/3808372554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f49800a4740 0x7f49800a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497effd640 1 --2- 172.21.15.190:0/3808372554 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f49800a5060 0x7f498013a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 10 monclient(hunting): handle_auth_done global_id 4109 payload 274 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 1 -- 172.21.15.190:0/3808372554 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f49800a5060 msgr2=0x7f498013a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 1 --2- 172.21.15.190:0/3808372554 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f49800a5060 0x7f498013a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 1 -- 172.21.15.190:0/3808372554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f49800a4740 msgr2=0x7f49800a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 1 --2- 172.21.15.190:0/3808372554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f49800a4740 0x7f49800a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 1 monclient: found mon.noname-h 2024-08-19T10:37:52.458 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:52.459 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:37:52.459 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497ffff640 1 -- 172.21.15.190:0/3808372554 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49800a21f0 con 0x7f498013aa50 2024-08-19T10:37:52.459 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497f7fe640 10 monclient: get_auth_request con 0x7f49800a4740 auth_method 0 2024-08-19T10:37:52.459 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.453+0000 7f497f7fe640 1 --2- 172.21.15.190:0/3808372554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f49800a4740 0x7f49800a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:37:52.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497ffff640 1 -- 172.21.15.190:0/3808372554 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f498013aa50 msgr2=0x7f498013ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:52.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497ffff640 1 --2- 172.21.15.190:0/3808372554 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f498013aa50 0x7f498013ce40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4978009e80 tx=0x7f497801c9a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=2617b77625bd147b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:52.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 1 -- 172.21.15.190:0/3808372554 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49780ac020 con 0x7f498013aa50 2024-08-19T10:37:52.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:52.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:37:52.460 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 10 monclient: dump: 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:52.461 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 1 -- 172.21.15.190:0/3808372554 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49780091f0 con 0x7f498013aa50 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 1 -- 172.21.15.190:0/3808372554 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49780a6d70 con 0x7f498013aa50 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 5 monclient: authenticate success, global_id 4109 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 10 monclient: get_monmap_and_config success 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 4 set_mon_vals no callback set 2024-08-19T10:37:52.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f497e7fc640 10 monclient: dump: 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:52.463 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 10 monclient: shutdown 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 1 -- 172.21.15.190:0/3808372554 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f498013aa50 msgr2=0x7f498013ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 1 --2- 172.21.15.190:0/3808372554 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f498013aa50 0x7f498013ce40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4978009e80 tx=0x7f497801c9a0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 1 -- 172.21.15.190:0/3808372554 shutdown_connections 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 1 --2- 172.21.15.190:0/3808372554 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f498013aa50 0x7f498013ce40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 1 --2- 172.21.15.190:0/3808372554 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f49800a5060 0x7f498013a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 1 --2- 172.21.15.190:0/3808372554 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f49800a4740 0x7f49800a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 1 -- 172.21.15.190:0/3808372554 >> 172.21.15.190:0/3808372554 conn(0x7f49800a5f50 msgr2=0x7f498006c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:52.465 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 1 -- 172.21.15.190:0/3808372554 shutdown_connections 2024-08-19T10:37:52.465 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 1 -- 172.21.15.190:0/3808372554 wait complete. 2024-08-19T10:37:52.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 10 monclient: build_initial_monmap 2024-08-19T10:37:52.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 10 monclient: monmap: 2024-08-19T10:37:52.469 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:52.469 INFO:teuthology.orchestra.run.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:37:52.469 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 0.000000 2024-08-19T10:37:52.469 INFO:teuthology.orchestra.run.smithi190.stderr:created 0.000000 2024-08-19T10:37:52.469 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:52.469 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.469 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:37:52.469 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 1 Processor -- start 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 1 -- start start 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.457+0000 7f4985ca4640 10 monclient: init 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:52.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient: _add_conns ranks=[3,7,8,4,1,0,6,5,2] 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49800a4740 0x7f4980122fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient(hunting): picked mon.noname-g con 0x7f49800a4740 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f49800a5060 0x7f4980121100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient(hunting): picked mon.noname-i con 0x7f49800a5060 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f498013aa50 0x7f4980139d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient(hunting): picked mon.noname-e con 0x7f498013aa50 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f49800a5440 con 0x7f49800a4740 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f49800a1330 con 0x7f49800a5060 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f498013ca20 con 0x7f498013aa50 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:52.471 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient(hunting): authenticate will time out at 918.032312s 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49800a4740 0x7f4980122fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497effd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f49800a5060 0x7f4980121100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49800a4740 0x7f4980122fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:50094/0 (socket says 172.21.15.190:50094) 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 1 -- 172.21.15.190:0/70019863 learned_addr learned my addr 172.21.15.190:0/70019863 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 10 monclient(hunting): get_auth_request con 0x7f49800a4740 auth_method 0 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497effd640 10 monclient(hunting): get_auth_request con 0x7f49800a5060 auth_method 0 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:52.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497ffff640 1 --2- 172.21.15.190:0/70019863 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f498013aa50 0x7f4980139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497ffff640 10 monclient(hunting): get_auth_request con 0x7f498013aa50 auth_method 0 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 10 monclient(hunting): handle_auth_done global_id 4116 payload 824 2024-08-19T10:37:52.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 1 -- 172.21.15.190:0/70019863 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f498013aa50 msgr2=0x7f4980139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 1 --2- 172.21.15.190:0/70019863 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f498013aa50 0x7f4980139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 1 -- 172.21.15.190:0/70019863 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f49800a5060 msgr2=0x7f4980121100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 1 --2- 172.21.15.190:0/70019863 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f49800a5060 0x7f4980121100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 1 monclient: found mon.noname-g 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 1 -- 172.21.15.190:0/70019863 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4980122460 con 0x7f49800a4740 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 1 -- 172.21.15.190:0/70019863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49800a4740 msgr2=0x7f4980122fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497f7fe640 1 --2- 172.21.15.190:0/70019863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49800a4740 0x7f4980122fc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f497000c7f0 tx=0x7f497000ccb0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 1 -- 172.21.15.190:0/70019863 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4970015070 con 0x7f49800a4740 2024-08-19T10:37:52.474 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 10 monclient: dump: 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:52.475 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 10 monclient: _finish_auth 0 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 10 monclient: _check_auth_tickets 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 5 monclient: authenticate success, global_id 4116 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 1 -- 172.21.15.190:0/70019863 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4970007c40 con 0x7f49800a4740 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 1 -- 172.21.15.190:0/70019863 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4970007df0 con 0x7f49800a4740 2024-08-19T10:37:52.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f497dffb640 4 set_mon_vals no callback set 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4967fff640 10 monclient: dump: 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient: _renew_subs 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.461+0000 7f4985ca4640 1 -- 172.21.15.190:0/70019863 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49800a7db0 con 0x7f49800a4740 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.465+0000 7f4985ca4640 10 monclient: _renew_subs 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.465+0000 7f4985ca4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.465+0000 7f4985ca4640 1 -- 172.21.15.190:0/70019863 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f498013a270 con 0x7f49800a4740 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.465+0000 7f4967fff640 1 -- 172.21.15.190:0/70019863 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4970018070 con 0x7f49800a4740 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.465+0000 7f4967fff640 1 -- 172.21.15.190:0/70019863 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f49700117a0 con 0x7f49800a4740 2024-08-19T10:37:52.478 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.465+0000 7f4985ca4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:52.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.465+0000 7f4985ca4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:52.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.465+0000 7f4985ca4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:37:52.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.465+0000 7f4985ca4640 1 -- 172.21.15.190:0/70019863 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49800a4b20 con 0x7f49800a4740 2024-08-19T10:37:52.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.469+0000 7f4967fff640 1 -- 172.21.15.190:0/70019863 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f497001ad10 con 0x7f49800a4740 2024-08-19T10:37:52.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.469+0000 7f4967fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:52.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.469+0000 7f4967fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:37:52.541 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.537+0000 7f4985ca4640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "224c1713-25c8-4a48-b16e-8929fcb6424b", "id": 1}] 2024-08-19T10:37:52.541 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.537+0000 7f4985ca4640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "224c1713-25c8-4a48-b16e-8929fcb6424b", "id": 1}] 2024-08-19T10:37:52.541 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.537+0000 7f4985ca4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:37:52.541 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.537+0000 7f4985ca4640 1 -- 172.21.15.190:0/70019863 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "224c1713-25c8-4a48-b16e-8929fcb6424b", "id": 1} v 0) -- 0x7f498006ed30 con 0x7f49800a4740 2024-08-19T10:37:52.550 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:52.549+0000 7f497f7fe640 1 --2- 172.21.15.190:0/70019863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49800a4740 0x7f4980122fc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f497000c7f0 tx=0x7f497000ccb0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f49700077d0 mon_command_ack([{"prefix": "osd new", "uuid": "224c1713-25c8-4a48-b16e-8929fcb6424b", "id": 1}]=0 v4) 2024-08-19T10:37:53.379 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4967fff640 1 -- 172.21.15.190:0/70019863 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "224c1713-25c8-4a48-b16e-8929fcb6424b", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f49700077d0 con 0x7f49800a4740 2024-08-19T10:37:53.379 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4967fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "224c1713-25c8-4a48-b16e-8929fcb6424b", "id": 1}] 2024-08-19T10:37:53.379 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4967fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:37:53.379 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-08-19T10:37:53.380 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4985ca4640 10 monclient: shutdown 2024-08-19T10:37:53.380 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4985ca4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:53.380 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4985ca4640 1 -- 172.21.15.190:0/70019863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49800a4740 msgr2=0x7f4980122fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:53.380 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4985ca4640 1 --2- 172.21.15.190:0/70019863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49800a4740 0x7f4980122fc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f497000c7f0 tx=0x7f497000ccb0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.381 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4985ca4640 1 -- 172.21.15.190:0/70019863 shutdown_connections 2024-08-19T10:37:53.381 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4985ca4640 1 --2- 172.21.15.190:0/70019863 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f498013aa50 0x7f4980139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.381 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4985ca4640 1 --2- 172.21.15.190:0/70019863 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f49800a5060 0x7f4980121100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.381 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4985ca4640 1 --2- 172.21.15.190:0/70019863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49800a4740 0x7f4980122fc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.381 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4985ca4640 1 -- 172.21.15.190:0/70019863 >> 172.21.15.190:0/70019863 conn(0x7f49800a5f50 msgr2=0x7f498009f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:53.381 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4985ca4640 1 -- 172.21.15.190:0/70019863 shutdown_connections 2024-08-19T10:37:53.381 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.377+0000 7f4985ca4640 1 -- 172.21.15.190:0/70019863 wait complete. 2024-08-19T10:37:53.396 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d 2 2024-08-19T10:37:53.515 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient: get_monmap_and_config 2024-08-19T10:37:53.515 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient: build_initial_monmap 2024-08-19T10:37:53.515 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient: monmap: 2024-08-19T10:37:53.515 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:53.515 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:53.515 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:53.516969+0000 2024-08-19T10:37:53.515 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:53.516969+0000 2024-08-19T10:37:53.515 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:37:53.516 INFO:teuthology.orchestra.run.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:37:53.517 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:53.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 1 Processor -- start 2024-08-19T10:37:53.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 1 -- start start 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient: init 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient: _add_conns ranks=[7,3,9,0,5,2,6,8,10,4,1] 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a4b30 0x7fb7dc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient(hunting): picked mon.noname-i con 0x7fb7dc0a4b30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7dc0720c0 0x7fb7dc0a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient(hunting): picked mon.noname-g con 0x7fb7dc0720c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb7dc0a5450 0x7fb7dc13d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient(hunting): picked mon.noname-a con 0x7fb7dc0a5450 addr v2:172.21.15.89:3300/0 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fb7dc073ea0 con 0x7fb7dc0a5450 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:53.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb7dc073600 con 0x7fb7dc0720c0 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb7dc0738e0 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e2efa640 10 monclient(hunting): authenticate will time out at 919.087426s 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e0c96640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a4b30 0x7fb7dc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7dc0720c0 0x7fb7dc0a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e0c96640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a4b30 0x7fb7dc0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:47766/0 (socket says 172.21.15.190:47766) 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7dc0720c0 0x7fb7dc0a4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:50100/0 (socket says 172.21.15.190:50100) 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e0c96640 1 -- 172.21.15.190:0/1335982865 learned_addr learned my addr 172.21.15.190:0/1335982865 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e0c96640 10 monclient(hunting): get_auth_request con 0x7fb7dc0a4b30 auth_method 0 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e0c96640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:53.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e0c96640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e0c96640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 10 monclient(hunting): get_auth_request con 0x7fb7dc0720c0 auth_method 0 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e0c96640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e0c96640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e0c96640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e1497640 1 --2- 172.21.15.190:0/1335982865 >> v2:172.21.15.89:3300/0 conn(0x7fb7dc0a5450 0x7fb7dc13d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:53.521 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 10 monclient(hunting): handle_auth_done global_id 4125 payload 274 2024-08-19T10:37:53.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 1 -- 172.21.15.190:0/1335982865 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a4b30 msgr2=0x7fb7dc0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:53.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 1 --2- 172.21.15.190:0/1335982865 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a4b30 0x7fb7dc0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 1 -- 172.21.15.190:0/1335982865 >> v2:172.21.15.89:3300/0 conn(0x7fb7dc0a5450 msgr2=0x7fb7dc13d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:53.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 1 --2- 172.21.15.190:0/1335982865 >> v2:172.21.15.89:3300/0 conn(0x7fb7dc0a5450 0x7fb7dc13d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 10 monclient: _finish_hunting 0 2024-08-19T10:37:53.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 1 monclient: found mon.noname-g 2024-08-19T10:37:53.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:53.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:37:53.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7dbfff640 1 -- 172.21.15.190:0/1335982865 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7dc0a42a0 con 0x7fb7dc0720c0 2024-08-19T10:37:53.522 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.513+0000 7fb7e0c96640 1 --2- 172.21.15.190:0/1335982865 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a4b30 0x7fb7dc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 1 -- 172.21.15.190:0/1335982865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7dc0720c0 msgr2=0x7fb7dc0a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 1 --2- 172.21.15.190:0/1335982865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7dc0720c0 0x7fb7dc0a4f10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80097c0 tx=0x7fb7c801c360 comp rx=0 tx=0).ready entity=mon.3 client_cookie=94004d566fac01dc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 1 -- 172.21.15.190:0/1335982865 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb7c80b0070 con 0x7fb7dc0720c0 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 10 monclient: dump: 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:53.526 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 10 monclient: _finish_auth 0 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:37:53.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 5 monclient: authenticate success, global_id 4125 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 1 -- 172.21.15.190:0/1335982865 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7c80aa810 con 0x7fb7dc0720c0 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient: get_monmap_and_config success 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 4 set_mon_vals no callback set 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 1 -- 172.21.15.190:0/1335982865 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb7c80a9300 con 0x7fb7dc0720c0 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7db7fe640 10 monclient: dump: 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:53.528 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient: shutdown 2024-08-19T10:37:53.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1335982865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7dc0720c0 msgr2=0x7fb7dc0a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 --2- 172.21.15.190:0/1335982865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7dc0720c0 0x7fb7dc0a4f10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80097c0 tx=0x7fb7c801c360 comp rx=0 tx=0).stop 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1335982865 shutdown_connections 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 --2- 172.21.15.190:0/1335982865 >> v2:172.21.15.89:3300/0 conn(0x7fb7dc0a5450 0x7fb7dc13d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 --2- 172.21.15.190:0/1335982865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7dc0720c0 0x7fb7dc0a4f10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 --2- 172.21.15.190:0/1335982865 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a4b30 0x7fb7dc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1335982865 >> 172.21.15.190:0/1335982865 conn(0x7fb7dc0aa240 msgr2=0x7fb7dc0aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1335982865 shutdown_connections 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1335982865 wait complete. 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient: build_initial_monmap 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient: monmap: 2024-08-19T10:37:53.530 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 0.000000 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:created 0.000000 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:53.531 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 Processor -- start 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 -- start start 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient: init 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient: _add_conns ranks=[4,1,7,6,8,2,0,5,3] 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7dc0720c0 0x7fb7dc137a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient(hunting): picked mon.noname-c con 0x7fb7dc0720c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7dc0a4b30 0x7fb7dc137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient(hunting): picked mon.noname-f con 0x7fb7dc0a4b30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a5450 0x7fb7dc06da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:53.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient(hunting): picked mon.noname-i con 0x7fb7dc0a5450 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:37:53.533 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:53.533 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc140b30 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.533 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:53.533 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb7dc1409b0 con 0x7fb7dc0720c0 2024-08-19T10:37:53.533 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:53.533 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb7dc140cb0 con 0x7fb7dc0a5450 2024-08-19T10:37:53.533 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:53.533 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e2efa640 10 monclient(hunting): authenticate will time out at 919.094549s 2024-08-19T10:37:53.533 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7dc0a4b30 0x7fb7dc137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:53.534 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7dc0a4b30 0x7fb7dc137fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:58408/0 (socket says 172.21.15.190:58408) 2024-08-19T10:37:53.534 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 1 -- 172.21.15.190:0/1763704167 learned_addr learned my addr 172.21.15.190:0/1763704167 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:53.534 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 10 monclient(hunting): get_auth_request con 0x7fb7dc0a4b30 auth_method 0 2024-08-19T10:37:53.534 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e1497640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a5450 0x7fb7dc06da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:53.534 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:53.534 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:53.534 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:53.534 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e1497640 10 monclient(hunting): get_auth_request con 0x7fb7dc0a5450 auth_method 0 2024-08-19T10:37:53.534 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e1497640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:53.534 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e1497640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e1497640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e1497640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e1497640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e1497640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e0c96640 1 --2- 172.21.15.190:0/1763704167 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7dc0720c0 0x7fb7dc137a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 10 monclient(hunting): handle_auth_done global_id 4114 payload 824 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 1 -- 172.21.15.190:0/1763704167 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a5450 msgr2=0x7fb7dc06da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 1 --2- 172.21.15.190:0/1763704167 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a5450 0x7fb7dc06da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 1 -- 172.21.15.190:0/1763704167 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7dc0720c0 msgr2=0x7fb7dc137a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:53.535 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 1 --2- 172.21.15.190:0/1763704167 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7dc0720c0 0x7fb7dc137a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 10 monclient: _finish_hunting 0 2024-08-19T10:37:53.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 1 monclient: found mon.noname-f 2024-08-19T10:37:53.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:53.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:37:53.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7dbfff640 1 -- 172.21.15.190:0/1763704167 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7dc0a0070 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e0c96640 10 monclient: get_auth_request con 0x7fb7dc0720c0 auth_method 0 2024-08-19T10:37:53.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.521+0000 7fb7e0c96640 1 --2- 172.21.15.190:0/1763704167 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7dc0720c0 0x7fb7dc137a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:37:53.538 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7dbfff640 1 -- 172.21.15.190:0/1763704167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7dc0a4b30 msgr2=0x7fb7dc137fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:53.538 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7dbfff640 1 --2- 172.21.15.190:0/1763704167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7dc0a4b30 0x7fb7dc137fc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb7c801cc00 tx=0x7fb7c8019100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:53.538 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 1 -- 172.21.15.190:0/1763704167 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb7c80b0040 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.538 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:53.538 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:37:53.538 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 10 monclient: dump: 2024-08-19T10:37:53.538 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:53.539 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 10 monclient: _finish_auth 0 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 10 monclient: _check_auth_tickets 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 1 -- 172.21.15.190:0/1763704167 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7c80b9070 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7e2efa640 5 monclient: authenticate success, global_id 4114 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7e2efa640 10 monclient: _renew_subs 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7e2efa640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1763704167 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7dc0a95b0 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 1 -- 172.21.15.190:0/1763704167 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb7c80044b0 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 10 monclient: dump: 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7e2efa640 10 monclient: _renew_subs 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7daffd640 4 set_mon_vals no callback set 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7e2efa640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1763704167 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7dc124ba0 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7e2efa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7e2efa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7e2efa640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:37:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 1 -- 172.21.15.190:0/1763704167 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb7c8003620 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7d97fa640 1 -- 172.21.15.190:0/1763704167 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fb7c80b48a0 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.525+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1763704167 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7dc0a4f10 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.529+0000 7fb7d97fa640 1 -- 172.21.15.190:0/1763704167 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb7c80c2060 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.529+0000 7fb7d97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:53.543 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.529+0000 7fb7d97fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:37:53.602 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.597+0000 7fb7e2efa640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d", "id": 2}] 2024-08-19T10:37:53.602 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.597+0000 7fb7e2efa640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d", "id": 2}] 2024-08-19T10:37:53.602 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.597+0000 7fb7e2efa640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:37:53.603 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.597+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1763704167 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d", "id": 2} v 0) -- 0x7fb7dc1248a0 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.872 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.869+0000 7fb7d97fa640 1 -- 172.21.15.190:0/1763704167 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fb7c8016070 con 0x7fb7dc0a4b30 2024-08-19T10:37:53.873 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.869+0000 7fb7d97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d", "id": 2}] 2024-08-19T10:37:53.873 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.869+0000 7fb7d97fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:37:53.873 INFO:teuthology.orchestra.run.smithi190.stdout:2 2024-08-19T10:37:53.874 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.873+0000 7fb7e2efa640 10 monclient: shutdown 2024-08-19T10:37:53.875 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.873+0000 7fb7e2efa640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:53.875 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.873+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1763704167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7dc0a4b30 msgr2=0x7fb7dc137fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:53.875 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.873+0000 7fb7e2efa640 1 --2- 172.21.15.190:0/1763704167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7dc0a4b30 0x7fb7dc137fc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb7c801cc00 tx=0x7fb7c8019100 comp rx=0 tx=0).stop 2024-08-19T10:37:53.875 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.873+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1763704167 shutdown_connections 2024-08-19T10:37:53.875 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.873+0000 7fb7e2efa640 1 --2- 172.21.15.190:0/1763704167 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7dc0a5450 0x7fb7dc06da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.875 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.873+0000 7fb7e2efa640 1 --2- 172.21.15.190:0/1763704167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7dc0a4b30 0x7fb7dc137fc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.875 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.873+0000 7fb7e2efa640 1 --2- 172.21.15.190:0/1763704167 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7dc0720c0 0x7fb7dc137a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:53.875 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.873+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1763704167 >> 172.21.15.190:0/1763704167 conn(0x7fb7dc0aa240 msgr2=0x7fb7dc13aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:53.876 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.873+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1763704167 shutdown_connections 2024-08-19T10:37:53.876 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:53.873+0000 7fb7e2efa640 1 -- 172.21.15.190:0/1763704167 wait complete. 2024-08-19T10:37:53.895 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 2dcaafe0-5837-410b-ae99-c7561ef6b150 3 2024-08-19T10:37:54.012 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient: get_monmap_and_config 2024-08-19T10:37:54.012 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient: build_initial_monmap 2024-08-19T10:37:54.012 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient: monmap: 2024-08-19T10:37:54.012 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:54.012 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:54.012 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:54.013693+0000 2024-08-19T10:37:54.012 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:54.013693+0000 2024-08-19T10:37:54.012 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:54.012 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.012 INFO:teuthology.orchestra.run.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:37:54.012 INFO:teuthology.orchestra.run.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:37:54.013 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:54.013 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:54.013 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:54.013 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:54.013 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:54.013 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:54.013 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:54.013 INFO:teuthology.orchestra.run.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:37:54.013 INFO:teuthology.orchestra.run.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:37:54.013 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.016 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 1 Processor -- start 2024-08-19T10:37:54.016 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 1 -- start start 2024-08-19T10:37:54.017 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient: init 2024-08-19T10:37:54.017 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:54.017 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient: _add_conns ranks=[9,5,10,4,1,3,7,6,0,2,8] 2024-08-19T10:37:54.017 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff9f00a3b30 0x7ff9f00a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.017 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient(hunting): picked mon.noname-a con 0x7ff9f00a3b30 addr v2:172.21.15.89:3300/0 2024-08-19T10:37:54.017 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9f00a3f10 0x7ff9f00a4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.017 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient(hunting): picked mon.noname-h con 0x7ff9f00a3f10 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:37:54.017 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9f01364f0 0x7ff9f0136980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.017 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient(hunting): picked mon.noname-f con 0x7ff9f01364f0 addr v2:172.21.15.190:3300/0 2024-08-19T10:37:54.017 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.017 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f00a08b0 con 0x7ff9f00a3b30 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f0074170 con 0x7ff9f01364f0 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff9f00a0630 con 0x7ff9f00a3f10 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.009+0000 7ff9f762c640 10 monclient(hunting): authenticate will time out at 919.583931s 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f4bc7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9f00a3f10 0x7ff9f00a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9f01364f0 0x7ff9f0136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9f01364f0 0x7ff9f0136980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:58424/0 (socket says 172.21.15.190:58424) 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 1 -- 172.21.15.190:0/982964841 learned_addr learned my addr 172.21.15.190:0/982964841 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f4bc7640 10 monclient(hunting): get_auth_request con 0x7ff9f00a3f10 auth_method 0 2024-08-19T10:37:54.018 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f4bc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f4bc7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f4bc7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 10 monclient(hunting): get_auth_request con 0x7ff9f01364f0 auth_method 0 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f4bc7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f4bc7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f4bc7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:54.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f53c8640 1 --2- 172.21.15.190:0/982964841 >> v2:172.21.15.89:3300/0 conn(0x7ff9f00a3b30 0x7ff9f00a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 10 monclient(hunting): handle_auth_done global_id 4123 payload 274 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 1 -- 172.21.15.190:0/982964841 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9f00a3f10 msgr2=0x7ff9f00a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 1 --2- 172.21.15.190:0/982964841 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9f00a3f10 0x7ff9f00a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 1 -- 172.21.15.190:0/982964841 >> v2:172.21.15.89:3300/0 conn(0x7ff9f00a3b30 msgr2=0x7ff9f00a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 1 --2- 172.21.15.190:0/982964841 >> v2:172.21.15.89:3300/0 conn(0x7ff9f00a3b30 0x7ff9f00a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 10 monclient: _finish_hunting 0 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 1 monclient: found mon.noname-f 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f5bc9640 1 -- 172.21.15.190:0/982964841 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f00a1950 con 0x7ff9f01364f0 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f53c8640 10 monclient: get_auth_request con 0x7ff9f00a3b30 auth_method 0 2024-08-19T10:37:54.020 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.013+0000 7ff9f53c8640 1 --2- 172.21.15.190:0/982964841 >> v2:172.21.15.89:3300/0 conn(0x7ff9f00a3b30 0x7ff9f00a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:37:54.025 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 1 -- 172.21.15.190:0/982964841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f01364f0 msgr2=0x7ff9f0136980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:54.025 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 1 --2- 172.21.15.190:0/982964841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f01364f0 0x7ff9f0136980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800d150 tx=0x7ff9e801eab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c88c1be3ad4596 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:54.025 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 1 -- 172.21.15.190:0/982964841 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9e80a7640 con 0x7ff9f01364f0 2024-08-19T10:37:54.025 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:54.025 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 10 monclient: dump: 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 10 monclient: _finish_auth 0 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 1 -- 172.21.15.190:0/982964841 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e80a7800 con 0x7ff9f01364f0 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 1 -- 172.21.15.190:0/982964841 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9e80a7e20 con 0x7ff9f01364f0 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 5 monclient: authenticate success, global_id 4123 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient: get_monmap_and_config success 2024-08-19T10:37:54.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 4 set_mon_vals no callback set 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9e7fff640 10 monclient: dump: 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:54.028 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient: shutdown 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 -- 172.21.15.190:0/982964841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f01364f0 msgr2=0x7ff9f0136980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 --2- 172.21.15.190:0/982964841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f01364f0 0x7ff9f0136980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800d150 tx=0x7ff9e801eab0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 -- 172.21.15.190:0/982964841 shutdown_connections 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 --2- 172.21.15.190:0/982964841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9f01364f0 0x7ff9f0136980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 --2- 172.21.15.190:0/982964841 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9f00a3f10 0x7ff9f00a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 --2- 172.21.15.190:0/982964841 >> v2:172.21.15.89:3300/0 conn(0x7ff9f00a3b30 0x7ff9f00a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 -- 172.21.15.190:0/982964841 >> 172.21.15.190:0/982964841 conn(0x7ff9f00a8f60 msgr2=0x7ff9f00a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 -- 172.21.15.190:0/982964841 shutdown_connections 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 -- 172.21.15.190:0/982964841 wait complete. 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient: build_initial_monmap 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient: monmap: 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 0.000000 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:created 0.000000 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:37:54.030 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 Processor -- start 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 -- start start 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient: init 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient: _add_conns ranks=[4,8,3,5,1,7,6,2,0] 2024-08-19T10:37:54.031 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9f00a3b30 0x7ff9f01d18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient(hunting): picked mon.noname-c con 0x7ff9f00a3b30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9f00a3f10 0x7ff9f01d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient(hunting): picked mon.noname-e con 0x7ff9f00a3f10 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f01364f0 0x7ff9f0139d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient(hunting): picked mon.noname-g con 0x7ff9f01364f0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f006d140 con 0x7ff9f01364f0 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff9f00a4870 con 0x7ff9f00a3b30 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff9f006d2c0 con 0x7ff9f00a3f10 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:54.032 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f762c640 10 monclient(hunting): authenticate will time out at 919.591102s 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f01364f0 0x7ff9f0139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f01364f0 0x7ff9f0139d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:50116/0 (socket says 172.21.15.190:50116) 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 1 -- 172.21.15.190:0/3986937985 learned_addr learned my addr 172.21.15.190:0/3986937985 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 10 monclient(hunting): get_auth_request con 0x7ff9f01364f0 auth_method 0 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f53c8640 1 --2- 172.21.15.190:0/3986937985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9f00a3b30 0x7ff9f01d18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f4bc7640 1 --2- 172.21.15.190:0/3986937985 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9f00a3f10 0x7ff9f01d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:54.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 10 monclient(hunting): handle_auth_done global_id 4134 payload 824 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 1 -- 172.21.15.190:0/3986937985 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9f00a3f10 msgr2=0x7ff9f01d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 1 --2- 172.21.15.190:0/3986937985 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9f00a3f10 0x7ff9f01d1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 1 -- 172.21.15.190:0/3986937985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9f00a3b30 msgr2=0x7ff9f01d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 1 --2- 172.21.15.190:0/3986937985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9f00a3b30 0x7ff9f01d18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 10 monclient: _finish_hunting 0 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 1 monclient: found mon.noname-g 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f5bc9640 1 -- 172.21.15.190:0/3986937985 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f013bea0 con 0x7ff9f01364f0 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f53c8640 10 monclient: get_auth_request con 0x7ff9f00a3b30 auth_method 0 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f53c8640 1 --2- 172.21.15.190:0/3986937985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9f00a3b30 0x7ff9f01d18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:37:54.034 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f4bc7640 10 monclient: get_auth_request con 0x7ff9f00a3f10 auth_method 0 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.017+0000 7ff9f4bc7640 1 --2- 172.21.15.190:0/3986937985 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9f00a3f10 0x7ff9f01d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f5bc9640 1 -- 172.21.15.190:0/3986937985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f01364f0 msgr2=0x7ff9f0139d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f5bc9640 1 --2- 172.21.15.190:0/3986937985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f01364f0 0x7ff9f0139d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff9e801d160 tx=0x7ff9e801b850 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 1 -- 172.21.15.190:0/3986937985 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9e80be070 con 0x7ff9f01364f0 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 10 monclient: dump: 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.035 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:54.036 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:54.036 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:54.036 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:54.036 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:54.036 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:54.036 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:54.036 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:54.036 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f762c640 5 monclient: authenticate success, global_id 4134 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 1 -- 172.21.15.190:0/3986937985 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e80a7640 con 0x7ff9f01364f0 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 1 -- 172.21.15.190:0/3986937985 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9e8003e10 con 0x7ff9f01364f0 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f762c640 10 monclient: _renew_subs 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f762c640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f762c640 1 -- 172.21.15.190:0/3986937985 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f0123c50 con 0x7ff9f01364f0 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e6ffd640 4 set_mon_vals no callback set 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:54.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 10 monclient: dump: 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:54.038 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:54.039 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:54.039 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:54.039 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:54.039 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.039 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f762c640 10 monclient: _renew_subs 2024-08-19T10:37:54.039 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f762c640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:37:54.039 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f762c640 1 -- 172.21.15.190:0/3986937985 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f0124b20 con 0x7ff9f01364f0 2024-08-19T10:37:54.039 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 1 -- 172.21.15.190:0/3986937985 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff9e80039a0 con 0x7ff9f01364f0 2024-08-19T10:37:54.039 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9e5ffb640 1 -- 172.21.15.190:0/3986937985 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7ff9e80045e0 con 0x7ff9f01364f0 2024-08-19T10:37:54.040 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f762c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:54.040 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f762c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:54.040 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f762c640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:37:54.040 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.021+0000 7ff9f762c640 1 -- 172.21.15.190:0/3986937985 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f0136ae0 con 0x7ff9f01364f0 2024-08-19T10:37:54.040 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.025+0000 7ff9e5ffb640 1 -- 172.21.15.190:0/3986937985 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff9f0123c50 con 0x7ff9f01364f0 2024-08-19T10:37:54.040 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.025+0000 7ff9e5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:54.040 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.025+0000 7ff9e5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:37:54.096 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.093+0000 7ff9f762c640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "2dcaafe0-5837-410b-ae99-c7561ef6b150", "id": 3}] 2024-08-19T10:37:54.097 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.093+0000 7ff9f762c640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "2dcaafe0-5837-410b-ae99-c7561ef6b150", "id": 3}] 2024-08-19T10:37:54.097 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.093+0000 7ff9f762c640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:37:54.097 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.093+0000 7ff9f762c640 1 -- 172.21.15.190:0/3986937985 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2dcaafe0-5837-410b-ae99-c7561ef6b150", "id": 3} v 0) -- 0x7ff9f006d4d0 con 0x7ff9f01364f0 2024-08-19T10:37:54.471 INFO:tasks.ceph.mgr.x.smithi190.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-08-19T10:37:54.472 INFO:tasks.ceph.mgr.x.smithi190.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-08-19T10:37:54.472 INFO:tasks.ceph.mgr.x.smithi190.stderr: from numpy import show_config as show_numpy_config 2024-08-19T10:37:54.481 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.477+0000 7ff9e5ffb640 1 -- 172.21.15.190:0/3986937985 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2dcaafe0-5837-410b-ae99-c7561ef6b150", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7ff9e80187c0 con 0x7ff9f01364f0 2024-08-19T10:37:54.481 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.477+0000 7ff9e5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "2dcaafe0-5837-410b-ae99-c7561ef6b150", "id": 3}] 2024-08-19T10:37:54.481 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.477+0000 7ff9e5ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:37:54.481 INFO:teuthology.orchestra.run.smithi190.stdout:3 2024-08-19T10:37:54.483 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.481+0000 7ff9f762c640 10 monclient: shutdown 2024-08-19T10:37:54.483 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.481+0000 7ff9f762c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:54.483 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.481+0000 7ff9f762c640 1 -- 172.21.15.190:0/3986937985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f01364f0 msgr2=0x7ff9f0139d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:54.483 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.481+0000 7ff9f762c640 1 --2- 172.21.15.190:0/3986937985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f01364f0 0x7ff9f0139d00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff9e801d160 tx=0x7ff9e801b850 comp rx=0 tx=0).stop 2024-08-19T10:37:54.483 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.481+0000 7ff9f762c640 1 -- 172.21.15.190:0/3986937985 shutdown_connections 2024-08-19T10:37:54.483 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.481+0000 7ff9f762c640 1 --2- 172.21.15.190:0/3986937985 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9f01364f0 0x7ff9f0139d00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.483 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.481+0000 7ff9f762c640 1 --2- 172.21.15.190:0/3986937985 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9f00a3f10 0x7ff9f01d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.483 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.481+0000 7ff9f762c640 1 --2- 172.21.15.190:0/3986937985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9f00a3b30 0x7ff9f01d18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.484 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.481+0000 7ff9f762c640 1 -- 172.21.15.190:0/3986937985 >> 172.21.15.190:0/3986937985 conn(0x7ff9f00a8f60 msgr2=0x7ff9f013cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:54.484 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.481+0000 7ff9f762c640 1 -- 172.21.15.190:0/3986937985 shutdown_connections 2024-08-19T10:37:54.484 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.481+0000 7ff9f762c640 1 -- 172.21.15.190:0/3986937985 wait complete. 2024-08-19T10:37:54.506 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 5619c403-27fd-48de-8cee-c5e84289cfde 4 2024-08-19T10:37:54.631 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.625+0000 7fa4053f9640 10 monclient: get_monmap_and_config 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.625+0000 7fa4053f9640 10 monclient: build_initial_monmap 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.625+0000 7fa4053f9640 10 monclient: monmap: 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:54.629966+0000 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:54.629966+0000 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:54.632 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.625+0000 7fa4053f9640 1 Processor -- start 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.625+0000 7fa4053f9640 1 -- start start 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.625+0000 7fa4053f9640 10 monclient: init 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 10 monclient: _add_conns ranks=[3,6,4,2,0,9,1,10,7,8,5] 2024-08-19T10:37:54.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4000a3d00 0x7fa40009eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 10 monclient(hunting): picked mon.noname-g con 0x7fa4000a3d00 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa4000a40e0 0x7fa4000a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 10 monclient(hunting): picked mon.noname-d con 0x7fa4000a40e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa400136440 0x7fa400136860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 10 monclient(hunting): picked mon.noname-c con 0x7fa400136440 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa40009e870 con 0x7fa4000a3d00 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa40009eac0 con 0x7fa400136440 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa400069630 con 0x7fa4000a40e0 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:54.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa4053f9640 10 monclient(hunting): authenticate will time out at 920.200653s 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4000a3d00 0x7fa40009eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4000a3d00 0x7fa40009eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:50126/0 (socket says 172.21.15.190:50126) 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 1 -- 172.21.15.190:0/1623837130 learned_addr learned my addr 172.21.15.190:0/1623837130 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 10 monclient(hunting): get_auth_request con 0x7fa4000a3d00 auth_method 0 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 10 monclient(hunting): handle_auth_done global_id 4143 payload 274 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 1 -- 172.21.15.190:0/1623837130 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa4000a40e0 msgr2=0x7fa4000a44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 1 --2- 172.21.15.190:0/1623837130 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa4000a40e0 0x7fa4000a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3ff7fe640 1 --2- 172.21.15.190:0/1623837130 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa400136440 0x7fa400136860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:54.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 1 -- 172.21.15.190:0/1623837130 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa400136440 msgr2=0x7fa400136860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:54.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 1 --2- 172.21.15.190:0/1623837130 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa400136440 0x7fa400136860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 10 monclient: _finish_hunting 0 2024-08-19T10:37:54.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 1 monclient: found mon.noname-g 2024-08-19T10:37:54.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:54.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:37:54.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.629+0000 7fa3feffd640 1 -- 172.21.15.190:0/1623837130 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40006db50 con 0x7fa4000a3d00 2024-08-19T10:37:54.641 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3feffd640 1 -- 172.21.15.190:0/1623837130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4000a3d00 msgr2=0x7fa40009eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:54.641 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3feffd640 1 --2- 172.21.15.190:0/1623837130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4000a3d00 0x7fa40009eee0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00098d0 tx=0x7fa3f001c440 comp rx=0 tx=0).ready entity=mon.3 client_cookie=91dd826083c1728f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:54.641 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 1 -- 172.21.15.190:0/1623837130 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3f00b0070 con 0x7fa4000a3d00 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 10 monclient: dump: 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:54.642 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:54.643 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:54.643 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:54.643 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:54.643 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:54.643 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:54.643 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.643 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 10 monclient: _finish_auth 0 2024-08-19T10:37:54.643 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:37:54.643 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 1 -- 172.21.15.190:0/1623837130 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f00aa810 con 0x7fa4000a3d00 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 5 monclient: authenticate success, global_id 4143 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient: get_monmap_and_config success 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 1 -- 172.21.15.190:0/1623837130 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3f00aae30 con 0x7fa4000a3d00 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 4 set_mon_vals no callback set 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa3fdffb640 10 monclient: dump: 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:54.644 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.645 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.645 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:54.645 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.645 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:54.645 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:54.646 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:54.646 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:54.646 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:54.646 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:54.646 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:54.646 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:54.646 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:54.646 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient: shutdown 2024-08-19T10:37:54.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:54.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 -- 172.21.15.190:0/1623837130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4000a3d00 msgr2=0x7fa40009eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 --2- 172.21.15.190:0/1623837130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4000a3d00 0x7fa40009eee0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00098d0 tx=0x7fa3f001c440 comp rx=0 tx=0).stop 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 -- 172.21.15.190:0/1623837130 shutdown_connections 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 --2- 172.21.15.190:0/1623837130 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa400136440 0x7fa400136860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 --2- 172.21.15.190:0/1623837130 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa4000a40e0 0x7fa4000a44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 --2- 172.21.15.190:0/1623837130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4000a3d00 0x7fa40009eee0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 -- 172.21.15.190:0/1623837130 >> 172.21.15.190:0/1623837130 conn(0x7fa4000a4df0 msgr2=0x7fa4000a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 -- 172.21.15.190:0/1623837130 shutdown_connections 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 -- 172.21.15.190:0/1623837130 wait complete. 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient: build_initial_monmap 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient: monmap: 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:37:54.647 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 0.000000 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:created 0.000000 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:54.648 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 Processor -- start 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 -- start start 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient: init 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient: _add_conns ranks=[5,4,8,6,3,0,7,1,2] 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa4000a3d00 0x7fa40013bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient(hunting): picked mon.noname-h con 0x7fa4000a3d00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4000a40e0 0x7fa40012d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient(hunting): picked mon.noname-c con 0x7fa4000a40e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa400136440 0x7fa40012de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient(hunting): picked mon.noname-e con 0x7fa400136440 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa4000a4b80 con 0x7fa4000a40e0 2024-08-19T10:37:54.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa4000a4a00 con 0x7fa4000a3d00 2024-08-19T10:37:54.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:54.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa40013feb0 con 0x7fa400136440 2024-08-19T10:37:54.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:54.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.633+0000 7fa4053f9640 10 monclient(hunting): authenticate will time out at 920.207912s 2024-08-19T10:37:54.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa4000a3d00 0x7fa40013bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:54.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa4000a3d00 0x7fa40013bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:43902/0 (socket says 172.21.15.190:43902) 2024-08-19T10:37:54.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 1 -- 172.21.15.190:0/2207688002 learned_addr learned my addr 172.21.15.190:0/2207688002 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:54.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 10 monclient(hunting): get_auth_request con 0x7fa4000a3d00 auth_method 0 2024-08-19T10:37:54.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 10 monclient(hunting): handle_auth_done global_id 4127 payload 824 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3ff7fe640 1 --2- 172.21.15.190:0/2207688002 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa400136440 0x7fa40012de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3fe7fc640 1 --2- 172.21.15.190:0/2207688002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4000a40e0 0x7fa40012d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 1 -- 172.21.15.190:0/2207688002 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa400136440 msgr2=0x7fa40012de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 1 --2- 172.21.15.190:0/2207688002 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa400136440 0x7fa40012de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 1 -- 172.21.15.190:0/2207688002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4000a40e0 msgr2=0x7fa40012d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 1 --2- 172.21.15.190:0/2207688002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4000a40e0 0x7fa40012d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:54.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 10 monclient: _finish_hunting 0 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 1 monclient: found mon.noname-h 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 1 -- 172.21.15.190:0/2207688002 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa400073f10 con 0x7fa4000a3d00 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3fe7fc640 10 monclient: get_auth_request con 0x7fa4000a40e0 auth_method 0 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3fe7fc640 1 --2- 172.21.15.190:0/2207688002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4000a40e0 0x7fa40012d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 1 -- 172.21.15.190:0/2207688002 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa4000a3d00 msgr2=0x7fa40013bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3feffd640 1 --2- 172.21.15.190:0/2207688002 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa4000a3d00 0x7fa40013bb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00097c0 tx=0x7fa3f00191e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 1 -- 172.21.15.190:0/2207688002 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3f00ba070 con 0x7fa4000a3d00 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 10 monclient: dump: 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:54.652 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:54.653 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 10 monclient: _finish_auth 0 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa4053f9640 5 monclient: authenticate success, global_id 4127 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 1 -- 172.21.15.190:0/2207688002 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f00b1040 con 0x7fa4000a3d00 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa4053f9640 10 monclient: _renew_subs 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa4053f9640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa4053f9640 1 -- 172.21.15.190:0/2207688002 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40012fd60 con 0x7fa4000a3d00 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 1 -- 172.21.15.190:0/2207688002 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3f00b0070 con 0x7fa4000a3d00 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:37:54.654 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3fd7fa640 4 set_mon_vals no callback set 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 10 monclient: dump: 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:54.655 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa4053f9640 10 monclient: _renew_subs 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa4053f9640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa4053f9640 1 -- 172.21.15.190:0/2207688002 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40012e340 con 0x7fa4000a3d00 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 1 -- 172.21.15.190:0/2207688002 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa3f0004250 con 0x7fa4000a3d00 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa3e77fe640 1 -- 172.21.15.190:0/2207688002 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fa3f0003da0 con 0x7fa4000a3d00 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa4053f9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa4053f9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa4053f9640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:37:54.656 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.637+0000 7fa4053f9640 1 -- 172.21.15.190:0/2207688002 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4000a44c0 con 0x7fa4000a3d00 2024-08-19T10:37:54.657 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.641+0000 7fa3e77fe640 1 -- 172.21.15.190:0/2207688002 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa400073f10 con 0x7fa4000a3d00 2024-08-19T10:37:54.657 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.641+0000 7fa3e77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:54.657 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.641+0000 7fa3e77fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:37:54.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.713+0000 7fa4053f9640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "5619c403-27fd-48de-8cee-c5e84289cfde", "id": 4}] 2024-08-19T10:37:54.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.713+0000 7fa4053f9640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "5619c403-27fd-48de-8cee-c5e84289cfde", "id": 4}] 2024-08-19T10:37:54.714 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.713+0000 7fa4053f9640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:37:54.715 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:54.713+0000 7fa4053f9640 1 -- 172.21.15.190:0/2207688002 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd new", "uuid": "5619c403-27fd-48de-8cee-c5e84289cfde", "id": 4} v 0) -- 0x7fa400064590 con 0x7fa4000a3d00 2024-08-19T10:37:55.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa3e77fe640 1 -- 172.21.15.190:0/2207688002 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5619c403-27fd-48de-8cee-c5e84289cfde", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fa3f0016150 con 0x7fa4000a3d00 2024-08-19T10:37:55.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa3e77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "5619c403-27fd-48de-8cee-c5e84289cfde", "id": 4}] 2024-08-19T10:37:55.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa3e77fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:37:55.471 INFO:teuthology.orchestra.run.smithi190.stdout:4 2024-08-19T10:37:55.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa4053f9640 10 monclient: shutdown 2024-08-19T10:37:55.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa4053f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:55.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa4053f9640 1 -- 172.21.15.190:0/2207688002 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa4000a3d00 msgr2=0x7fa40013bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:55.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa4053f9640 1 --2- 172.21.15.190:0/2207688002 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa4000a3d00 0x7fa40013bb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00097c0 tx=0x7fa3f00191e0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa4053f9640 1 -- 172.21.15.190:0/2207688002 shutdown_connections 2024-08-19T10:37:55.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa4053f9640 1 --2- 172.21.15.190:0/2207688002 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa400136440 0x7fa40012de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa4053f9640 1 --2- 172.21.15.190:0/2207688002 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4000a40e0 0x7fa40012d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa4053f9640 1 --2- 172.21.15.190:0/2207688002 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa4000a3d00 0x7fa40013bb20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa4053f9640 1 -- 172.21.15.190:0/2207688002 >> 172.21.15.190:0/2207688002 conn(0x7fa4000a4df0 msgr2=0x7fa40013c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:55.472 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa4053f9640 1 -- 172.21.15.190:0/2207688002 shutdown_connections 2024-08-19T10:37:55.473 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.469+0000 7fa4053f9640 1 -- 172.21.15.190:0/2207688002 wait complete. 2024-08-19T10:37:55.488 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 82ab73a3-1212-4b0d-ab05-aa9c01dd5586 5 2024-08-19T10:37:55.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient: get_monmap_and_config 2024-08-19T10:37:55.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient: build_initial_monmap 2024-08-19T10:37:55.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient: monmap: 2024-08-19T10:37:55.607 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:55.607 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:55.607 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:55.609034+0000 2024-08-19T10:37:55.607 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:55.609034+0000 2024-08-19T10:37:55.607 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:55.607 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:37:55.608 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:55.611 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 1 Processor -- start 2024-08-19T10:37:55.611 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 1 -- start start 2024-08-19T10:37:55.611 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient: init 2024-08-19T10:37:55.611 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:55.611 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient: _add_conns ranks=[5,4,7,2,9,8,1,10,6,3,0] 2024-08-19T10:37:55.611 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0ccc0a3b30 0x7f0ccc0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.611 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient(hunting): picked mon.noname-h con 0x7f0ccc0a3b30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:37:55.611 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ccc0a3f10 0x7f0ccc0a4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.611 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient(hunting): picked mon.noname-c con 0x7f0ccc0a3f10 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:55.611 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0ccc1364f0 0x7f0ccc136970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient(hunting): picked mon.noname-i con 0x7f0ccc1364f0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f0ccc0a08b0 con 0x7f0ccc0a3f10 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0ccc074170 con 0x7f0ccc0a3b30 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0ccc0a0630 con 0x7f0ccc1364f0 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd4371640 10 monclient(hunting): authenticate will time out at 921.179532s 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd210d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0ccc0a3b30 0x7f0ccc0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd290e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0ccc1364f0 0x7f0ccc136970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd210d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0ccc0a3b30 0x7f0ccc0a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:35682/0 (socket says 172.21.15.190:35682) 2024-08-19T10:37:55.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd210d640 1 -- 172.21.15.190:0/3808506870 learned_addr learned my addr 172.21.15.190:0/3808506870 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd290e640 10 monclient(hunting): get_auth_request con 0x7f0ccc1364f0 auth_method 0 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd290e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd290e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd290e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd210d640 10 monclient(hunting): get_auth_request con 0x7f0ccc0a3b30 auth_method 0 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd210d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd210d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd210d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd290e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd290e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd290e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd210d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:55.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd210d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd210d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.605+0000 7f0cd190c640 1 --2- 172.21.15.190:0/3808506870 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ccc0a3f10 0x7f0ccc0a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd190c640 10 monclient(hunting): get_auth_request con 0x7f0ccc0a3f10 auth_method 0 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd190c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd190c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd190c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd210d640 10 monclient(hunting): handle_auth_done global_id 4136 payload 274 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd210d640 1 -- 172.21.15.190:0/3808506870 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0ccc1364f0 msgr2=0x7f0ccc136970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd210d640 1 --2- 172.21.15.190:0/3808506870 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0ccc1364f0 0x7f0ccc136970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd210d640 1 -- 172.21.15.190:0/3808506870 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ccc0a3f10 msgr2=0x7f0ccc0a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd210d640 1 --2- 172.21.15.190:0/3808506870 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ccc0a3f10 0x7f0ccc0a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd210d640 10 monclient: _finish_hunting 0 2024-08-19T10:37:55.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd210d640 1 monclient: found mon.noname-h 2024-08-19T10:37:55.615 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd210d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:55.615 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd210d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:37:55.615 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd210d640 1 -- 172.21.15.190:0/3808506870 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc0a1950 con 0x7f0ccc0a3b30 2024-08-19T10:37:55.615 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.609+0000 7f0cd290e640 1 --2- 172.21.15.190:0/3808506870 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0ccc1364f0 0x7f0ccc136970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:37:55.619 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd210d640 1 -- 172.21.15.190:0/3808506870 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0ccc0a3b30 msgr2=0x7f0ccc0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd210d640 1 --2- 172.21.15.190:0/3808506870 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0ccc0a3b30 0x7f0ccc0a2a60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0098d0 tx=0x7f0cbc01c440 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a544a337655f099c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 1 -- 172.21.15.190:0/3808506870 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0cbc0b0070 con 0x7f0ccc0a3b30 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 10 monclient: dump: 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 10 monclient: _finish_auth 0 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 10 monclient: _check_auth_tickets 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 1 -- 172.21.15.190:0/3808506870 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cbc0aa810 con 0x7f0ccc0a3b30 2024-08-19T10:37:55.621 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 1 -- 172.21.15.190:0/3808506870 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0cbc0a9300 con 0x7f0ccc0a3b30 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 5 monclient: authenticate success, global_id 4136 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient: get_monmap_and_config success 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 4 set_mon_vals no callback set 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd110b640 10 monclient: dump: 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:55.622 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient: shutdown 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 -- 172.21.15.190:0/3808506870 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0ccc0a3b30 msgr2=0x7f0ccc0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:55.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 --2- 172.21.15.190:0/3808506870 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0ccc0a3b30 0x7f0ccc0a2a60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0098d0 tx=0x7f0cbc01c440 comp rx=0 tx=0).stop 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 -- 172.21.15.190:0/3808506870 shutdown_connections 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 --2- 172.21.15.190:0/3808506870 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0ccc1364f0 0x7f0ccc136970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 --2- 172.21.15.190:0/3808506870 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ccc0a3f10 0x7f0ccc0a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 --2- 172.21.15.190:0/3808506870 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0ccc0a3b30 0x7f0ccc0a2a60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 -- 172.21.15.190:0/3808506870 >> 172.21.15.190:0/3808506870 conn(0x7f0ccc0a8f60 msgr2=0x7f0ccc0a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 -- 172.21.15.190:0/3808506870 shutdown_connections 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 -- 172.21.15.190:0/3808506870 wait complete. 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient: build_initial_monmap 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient: monmap: 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 0 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 0.000000 2024-08-19T10:37:55.624 INFO:teuthology.orchestra.run.smithi190.stderr:created 0.000000 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 Processor -- start 2024-08-19T10:37:55.625 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 -- start start 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient: init 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient: _add_conns ranks=[4,0,1,2,3,7,8,5,6] 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ccc0a3b30 0x7f0ccc134e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient(hunting): picked mon.noname-c con 0x7f0ccc0a3b30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ccc0a3f10 0x7f0ccc135360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient(hunting): picked mon.noname-a con 0x7f0ccc0a3f10 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ccc1364f0 0x7f0ccc13a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient(hunting): picked mon.noname-f con 0x7f0ccc1364f0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc06d140 con 0x7f0ccc0a3f10 2024-08-19T10:37:55.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc0a4870 con 0x7f0ccc1364f0 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f0ccc06d2c0 con 0x7f0ccc0a3b30 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd4371640 10 monclient(hunting): authenticate will time out at 921.186794s 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ccc1364f0 0x7f0ccc13a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ccc1364f0 0x7f0ccc13a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56884/0 (socket says 172.21.15.190:56884) 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 1 -- 172.21.15.190:0/690483780 learned_addr learned my addr 172.21.15.190:0/690483780 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 10 monclient(hunting): get_auth_request con 0x7f0ccc1364f0 auth_method 0 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd210d640 1 --2- 172.21.15.190:0/690483780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ccc0a3b30 0x7f0ccc134e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd190c640 1 --2- 172.21.15.190:0/690483780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ccc0a3f10 0x7f0ccc135360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd210d640 10 monclient(hunting): get_auth_request con 0x7f0ccc0a3b30 auth_method 0 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd210d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd210d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd210d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd190c640 10 monclient(hunting): get_auth_request con 0x7f0ccc0a3f10 auth_method 0 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd190c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd190c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd190c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 10 monclient(hunting): handle_auth_done global_id 4132 payload 824 2024-08-19T10:37:55.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 1 -- 172.21.15.190:0/690483780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ccc0a3b30 msgr2=0x7f0ccc134e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 1 --2- 172.21.15.190:0/690483780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ccc0a3b30 0x7f0ccc134e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 1 -- 172.21.15.190:0/690483780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ccc0a3f10 msgr2=0x7f0ccc135360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 1 --2- 172.21.15.190:0/690483780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ccc0a3f10 0x7f0ccc135360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 10 monclient: _finish_hunting 0 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 1 monclient: found mon.noname-f 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.613+0000 7f0cd290e640 1 -- 172.21.15.190:0/690483780 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc1247d0 con 0x7f0ccc1364f0 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd290e640 1 -- 172.21.15.190:0/690483780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ccc1364f0 msgr2=0x7f0ccc13a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd290e640 1 --2- 172.21.15.190:0/690483780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ccc1364f0 0x7f0ccc13a070 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009370 tx=0x7f0cc400e490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 1 -- 172.21.15.190:0/690483780 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0cc401a070 con 0x7f0ccc1364f0 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:55.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 10 monclient: dump: 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:55.630 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd4371640 5 monclient: authenticate success, global_id 4132 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 1 -- 172.21.15.190:0/690483780 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc400ee50 con 0x7f0ccc1364f0 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd4371640 10 monclient: _renew_subs 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd4371640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd4371640 1 -- 172.21.15.190:0/690483780 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc124d90 con 0x7f0ccc1364f0 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:55.631 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 1 -- 172.21.15.190:0/690483780 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0cc4009e20 con 0x7f0ccc1364f0 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 10 monclient: dump: 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:epoch 1 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:election_strategy: 1 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:55.632 INFO:teuthology.orchestra.run.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd090a640 4 set_mon_vals no callback set 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd4371640 10 monclient: _renew_subs 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd4371640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd4371640 1 -- 172.21.15.190:0/690483780 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc136960 con 0x7f0ccc1364f0 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 1 -- 172.21.15.190:0/690483780 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0cc40098d0 con 0x7f0ccc1364f0 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cc2ffd640 1 -- 172.21.15.190:0/690483780 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f0cc4009bc0 con 0x7f0ccc1364f0 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd4371640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:55.633 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd4371640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:55.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd4371640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:37:55.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.617+0000 7f0cd4371640 1 -- 172.21.15.190:0/690483780 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ccc073400 con 0x7f0ccc1364f0 2024-08-19T10:37:55.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.621+0000 7f0cc2ffd640 1 -- 172.21.15.190:0/690483780 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0cc400b2e0 con 0x7f0ccc1364f0 2024-08-19T10:37:55.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.621+0000 7f0cc2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:55.634 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.621+0000 7f0cc2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:37:55.692 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.689+0000 7f0cd4371640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "82ab73a3-1212-4b0d-ab05-aa9c01dd5586", "id": 5}] 2024-08-19T10:37:55.692 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.689+0000 7f0cd4371640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "82ab73a3-1212-4b0d-ab05-aa9c01dd5586", "id": 5}] 2024-08-19T10:37:55.693 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.689+0000 7f0cd4371640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:37:55.693 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.689+0000 7f0cd4371640 1 -- 172.21.15.190:0/690483780 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "82ab73a3-1212-4b0d-ab05-aa9c01dd5586", "id": 5} v 0) -- 0x7f0ccc06d5c0 con 0x7f0ccc1364f0 2024-08-19T10:37:55.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cc2ffd640 1 -- 172.21.15.190:0/690483780 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "82ab73a3-1212-4b0d-ab05-aa9c01dd5586", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f0cc4015480 con 0x7f0ccc1364f0 2024-08-19T10:37:55.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cc2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "82ab73a3-1212-4b0d-ab05-aa9c01dd5586", "id": 5}] 2024-08-19T10:37:55.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cc2ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:37:55.707 INFO:teuthology.orchestra.run.smithi190.stdout:5 2024-08-19T10:37:55.708 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cd4371640 10 monclient: shutdown 2024-08-19T10:37:55.708 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cd4371640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:55.708 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cd4371640 1 -- 172.21.15.190:0/690483780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ccc1364f0 msgr2=0x7f0ccc13a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:55.708 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cd4371640 1 --2- 172.21.15.190:0/690483780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ccc1364f0 0x7f0ccc13a070 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009370 tx=0x7f0cc400e490 comp rx=0 tx=0).stop 2024-08-19T10:37:55.708 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cd4371640 1 -- 172.21.15.190:0/690483780 shutdown_connections 2024-08-19T10:37:55.708 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cd4371640 1 --2- 172.21.15.190:0/690483780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ccc0a3f10 0x7f0ccc135360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.708 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cd4371640 1 --2- 172.21.15.190:0/690483780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ccc1364f0 0x7f0ccc13a070 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.708 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cd4371640 1 --2- 172.21.15.190:0/690483780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0ccc0a3b30 0x7f0ccc134e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.708 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cd4371640 1 -- 172.21.15.190:0/690483780 >> 172.21.15.190:0/690483780 conn(0x7f0ccc0a8f60 msgr2=0x7f0ccc0a9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:55.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cd4371640 1 -- 172.21.15.190:0/690483780 shutdown_connections 2024-08-19T10:37:55.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T10:37:55.705+0000 7f0cd4371640 1 -- 172.21.15.190:0/690483780 wait complete. 2024-08-19T10:37:55.723 INFO:tasks.ceph.osd.0:Restarting daemon 2024-08-19T10:37:55.723 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-08-19T10:37:55.726 INFO:tasks.ceph.osd.0:Started 2024-08-19T10:37:55.726 INFO:tasks.ceph.osd.1:Restarting daemon 2024-08-19T10:37:55.726 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-08-19T10:37:55.728 INFO:tasks.ceph.osd.1:Started 2024-08-19T10:37:55.729 INFO:tasks.ceph.osd.2:Restarting daemon 2024-08-19T10:37:55.729 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-08-19T10:37:55.731 INFO:tasks.ceph.osd.2:Started 2024-08-19T10:37:55.731 INFO:tasks.ceph.osd.3:Restarting daemon 2024-08-19T10:37:55.731 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-08-19T10:37:55.733 INFO:tasks.ceph.osd.3:Started 2024-08-19T10:37:55.733 INFO:tasks.ceph.osd.4:Restarting daemon 2024-08-19T10:37:55.733 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-08-19T10:37:55.735 INFO:tasks.ceph.osd.4:Started 2024-08-19T10:37:55.735 INFO:tasks.ceph.osd.5:Restarting daemon 2024-08-19T10:37:55.735 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-08-19T10:37:55.737 INFO:tasks.ceph.osd.5:Started 2024-08-19T10:37:55.738 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T10:37:55.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.850+0000 7faab562d640 10 monclient: get_monmap_and_config 2024-08-19T10:37:55.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.850+0000 7faab562d640 10 monclient: build_initial_monmap 2024-08-19T10:37:55.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.850+0000 7faab562d640 10 monclient: monmap: 2024-08-19T10:37:55.858 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:37:55.858 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:55.858 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:55.856018+0000 2024-08-19T10:37:55.858 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:55.856018+0000 2024-08-19T10:37:55.858 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:37:55.859 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:55.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.850+0000 7faab562d640 1 Processor -- start 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.850+0000 7faab562d640 1 -- start start 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.850+0000 7faab562d640 10 monclient: init 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 10 monclient: _add_conns ranks=[9,2,6,1,0,7,3,10,8,5,4] 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faab006c930 0x7faab00a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 10 monclient(hunting): picked mon.noname-a con 0x7faab006c930 addr v2:172.21.15.89:3300/0 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faab00a4070 0x7faab00a4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 10 monclient(hunting): picked mon.noname-b con 0x7faab00a4070 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faab013d980 0x7faab013fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 10 monclient(hunting): picked mon.noname-d con 0x7faab013d980 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7faab00a12d0 con 0x7faab006c930 2024-08-19T10:37:55.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7faab00a4990 con 0x7faab00a4070 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7faab006cd10 con 0x7faab013d980 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faab562d640 10 monclient(hunting): authenticate will time out at 925.672296s 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faab00a4070 0x7faab00a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaeffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faab006c930 0x7faab00a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaf7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faab013d980 0x7faab013fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaeffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faab006c930 0x7faab00a3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57922/0 (socket says 172.21.15.89:57922) 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaf7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faab013d980 0x7faab013fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:59322/0 (socket says 172.21.15.89:59322) 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faab00a4070 0x7faab00a4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49492/0 (socket says 172.21.15.89:49492) 2024-08-19T10:37:55.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaeffd640 1 -- 172.21.15.89:0/3097206728 learned_addr learned my addr 172.21.15.89:0/3097206728 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaeffd640 10 monclient(hunting): get_auth_request con 0x7faab006c930 auth_method 0 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 10 monclient(hunting): get_auth_request con 0x7faab00a4070 auth_method 0 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaf7fe640 10 monclient(hunting): get_auth_request con 0x7faab013d980 auth_method 0 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:55.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 10 monclient(hunting): handle_auth_done global_id 4133 payload 274 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 1 -- 172.21.15.89:0/3097206728 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faab013d980 msgr2=0x7faab013fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 1 --2- 172.21.15.89:0/3097206728 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faab013d980 0x7faab013fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 1 -- 172.21.15.89:0/3097206728 >> v2:172.21.15.89:3300/0 conn(0x7faab006c930 msgr2=0x7faab00a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 1 --2- 172.21.15.89:0/3097206728 >> v2:172.21.15.89:3300/0 conn(0x7faab006c930 0x7faab00a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:37:55.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 1 monclient: found mon.noname-b 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.854+0000 7faaae7fc640 1 -- 172.21.15.89:0/3097206728 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab00743e0 con 0x7faab00a4070 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 1 -- 172.21.15.89:0/3097206728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faab00a4070 msgr2=0x7faab00a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 1 --2- 172.21.15.89:0/3097206728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faab00a4070 0x7faab00a4450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009dc0 tx=0x7faaa001c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9b81086bfb0701 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 1 -- 172.21.15.89:0/3097206728 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faaa00b2070 con 0x7faab00a4070 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 10 monclient: dump: 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.869 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:55.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 10 monclient: _finish_auth 0 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 10 monclient: _check_auth_tickets 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 1 -- 172.21.15.89:0/3097206728 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa00a59b0 con 0x7faab00a4070 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 5 monclient: authenticate success, global_id 4133 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient: get_monmap_and_config success 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 1 -- 172.21.15.89:0/3097206728 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faaa00a5ba0 con 0x7faab00a4070 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 4 set_mon_vals no callback set 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaadffb640 10 monclient: dump: 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:37:55.871 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:55.872 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient: shutdown 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 -- 172.21.15.89:0/3097206728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faab00a4070 msgr2=0x7faab00a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 --2- 172.21.15.89:0/3097206728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faab00a4070 0x7faab00a4450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009dc0 tx=0x7faaa001c810 comp rx=0 tx=0).stop 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 -- 172.21.15.89:0/3097206728 shutdown_connections 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 --2- 172.21.15.89:0/3097206728 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faab013d980 0x7faab013fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 --2- 172.21.15.89:0/3097206728 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faab00a4070 0x7faab00a4450 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 --2- 172.21.15.89:0/3097206728 >> v2:172.21.15.89:3300/0 conn(0x7faab006c930 0x7faab00a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 -- 172.21.15.89:0/3097206728 >> 172.21.15.89:0/3097206728 conn(0x7faab006de40 msgr2=0x7faab006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 -- 172.21.15.89:0/3097206728 shutdown_connections 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 -- 172.21.15.89:0/3097206728 wait complete. 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient: build_initial_monmap 2024-08-19T10:37:55.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient: monmap: 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:55.874 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 Processor -- start 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 -- start start 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient: init 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient: _add_conns ranks=[4,8,5,3,6,7,1,2,0] 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faab006c930 0x7faab01d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient(hunting): picked mon.noname-c con 0x7faab006c930 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faab00a4070 0x7faab01d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient(hunting): picked mon.noname-e con 0x7faab00a4070 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faab013d980 0x7faab013aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:55.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient(hunting): picked mon.noname-h con 0x7faab013d980 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7faab0142ad0 con 0x7faab006c930 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7faab00a0ca0 con 0x7faab013d980 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7faab0142c50 con 0x7faab00a4070 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faab562d640 10 monclient(hunting): authenticate will time out at 925.679545s 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faab00a4070 0x7faab01d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaaeffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faab006c930 0x7faab01d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faab00a4070 0x7faab01d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:33200/0 (socket says 172.21.15.89:33200) 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 1 -- 172.21.15.89:0/2895341980 learned_addr learned my addr 172.21.15.89:0/2895341980 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaaeffd640 10 monclient(hunting): get_auth_request con 0x7faab006c930 auth_method 0 2024-08-19T10:37:55.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaaeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaaeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaaeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 10 monclient(hunting): get_auth_request con 0x7faab00a4070 auth_method 0 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaaeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaaeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaaeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:55.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.858+0000 7faaaf7fe640 1 --2- 172.21.15.89:0/2895341980 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faab013d980 0x7faab013aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaf7fe640 10 monclient(hunting): get_auth_request con 0x7faab013d980 auth_method 0 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 10 monclient(hunting): handle_auth_done global_id 4144 payload 824 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 1 -- 172.21.15.89:0/2895341980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faab00a4070 msgr2=0x7faab01d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 1 --2- 172.21.15.89:0/2895341980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faab00a4070 0x7faab01d15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 1 -- 172.21.15.89:0/2895341980 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faab013d980 msgr2=0x7faab013aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 1 --2- 172.21.15.89:0/2895341980 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faab013d980 0x7faab013aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 10 monclient: _finish_hunting 0 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 1 monclient: found mon.noname-c 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:37:55.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 1 -- 172.21.15.89:0/2895341980 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab013c3d0 con 0x7faab006c930 2024-08-19T10:37:55.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaae7fc640 1 --2- 172.21.15.89:0/2895341980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faab00a4070 0x7faab01d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:37:55.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 1 -- 172.21.15.89:0/2895341980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faab006c930 msgr2=0x7faab01d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:55.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaaeffd640 1 --2- 172.21.15.89:0/2895341980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faab006c930 0x7faab01d0df0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e870 tx=0x7faaa400ed30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:55.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 1 -- 172.21.15.89:0/2895341980 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faaa4017070 con 0x7faab006c930 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 10 monclient: dump: 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:55.881 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:55.882 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:55.882 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:55.882 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:55.882 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:55.882 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:55.882 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:55.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 10 monclient: _finish_auth 0 2024-08-19T10:37:55.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 10 monclient: _check_auth_tickets 2024-08-19T10:37:55.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:55.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 1 -- 172.21.15.89:0/2895341980 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa4013470 con 0x7faab006c930 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 1 -- 172.21.15.89:0/2895341980 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faaa4013a90 con 0x7faab006c930 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 10 monclient: dump: 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:55.883 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faaad7fa640 4 set_mon_vals no callback set 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faab562d640 5 monclient: authenticate success, global_id 4144 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faab562d640 10 monclient: _renew_subs 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faab562d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:37:55.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faab562d640 1 -- 172.21.15.89:0/2895341980 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab0139aa0 con 0x7faab006c930 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faab562d640 10 monclient: _renew_subs 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faab562d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faab562d640 1 -- 172.21.15.89:0/2895341980 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7faab01242d0 con 0x7faab006c930 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 1 -- 172.21.15.89:0/2895341980 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faaa401a070 con 0x7faab006c930 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.862+0000 7faa977fe640 1 -- 172.21.15.89:0/2895341980 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7faaa400f040 con 0x7faab006c930 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.866+0000 7faab562d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.866+0000 7faab562d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.866+0000 7faab562d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.866+0000 7faab562d640 1 -- 172.21.15.89:0/2895341980 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab00a3b30 con 0x7faab006c930 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.866+0000 7faa977fe640 1 -- 172.21.15.89:0/2895341980 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7faaa4021070 con 0x7faab006c930 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.866+0000 7faa977fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:55.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.866+0000 7faa977fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:37:55.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:37:55.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:37:55.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:37:55.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 1 -- 172.21.15.89:0/2895341980 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faab0064590 con 0x7faab006c930 2024-08-19T10:37:55.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faa977fe640 1 -- 172.21.15.89:0/2895341980 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5605 (secure 0 0 0) 0x7faab0139aa0 con 0x7faab006c930 2024-08-19T10:37:55.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faa977fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:37:55.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faa977fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:37:55.934 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:37:55.935 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","created":"2024-08-19T10:37:50.906618+0000","modified":"2024-08-19T10:37:55.695518+0000","last_up_change":"0.000000","last_in_change":"2024-08-19T10:37:55.695518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2be543b2-eb94-4907-8f21-6e91b7dd9733","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"224c1713-25c8-4a48-b16e-8929fcb6424b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2dcaafe0-5837-410b-ae99-c7561ef6b150","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5619c403-27fd-48de-8cee-c5e84289cfde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"82ab73a3-1212-4b0d-ab05-aa9c01dd5586","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T10:37:55.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 10 monclient: shutdown 2024-08-19T10:37:55.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:55.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 1 -- 172.21.15.89:0/2895341980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faab006c930 msgr2=0x7faab01d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:55.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 1 --2- 172.21.15.89:0/2895341980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faab006c930 0x7faab01d0df0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e870 tx=0x7faaa400ed30 comp rx=0 tx=0).stop 2024-08-19T10:37:55.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 1 -- 172.21.15.89:0/2895341980 shutdown_connections 2024-08-19T10:37:55.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 1 --2- 172.21.15.89:0/2895341980 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faab013d980 0x7faab013aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 1 --2- 172.21.15.89:0/2895341980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faab00a4070 0x7faab01d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 1 --2- 172.21.15.89:0/2895341980 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faab006c930 0x7faab01d0df0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:55.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 1 -- 172.21.15.89:0/2895341980 >> 172.21.15.89:0/2895341980 conn(0x7faab006de40 msgr2=0x7faab013f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:55.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 1 -- 172.21.15.89:0/2895341980 shutdown_connections 2024-08-19T10:37:55.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:37:55.930+0000 7faab562d640 1 -- 172.21.15.89:0/2895341980 wait complete. 2024-08-19T10:37:55.951 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-08-19T10:37:55.952 INFO:tasks.ceph:Waiting for OSDs to come up 2024-08-19T10:37:56.556 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-19T10:37:56.678 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.674+0000 7fb977043640 10 monclient: get_monmap_and_config 2024-08-19T10:37:56.678 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.674+0000 7fb977043640 10 monclient: build_initial_monmap 2024-08-19T10:37:56.678 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.674+0000 7fb977043640 10 monclient: monmap: 2024-08-19T10:37:56.678 INFO:teuthology.misc.health.smithi089.stderr:epoch 0 2024-08-19T10:37:56.678 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:56.679452+0000 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:56.679452+0000 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:37:56.679 INFO:teuthology.misc.health.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:37:56.680 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:37:56.682 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.674+0000 7fb977043640 1 Processor -- start 2024-08-19T10:37:56.682 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.674+0000 7fb977043640 1 -- start start 2024-08-19T10:37:56.682 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.674+0000 7fb977043640 10 monclient: init 2024-08-19T10:37:56.682 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:56.682 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 10 monclient: _add_conns ranks=[0,10,4,8,2,7,1,3,5,9,6] 2024-08-19T10:37:56.682 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb9700a6720 addr v1:172.21.15.89:6789/0 2024-08-19T10:37:56.682 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb9700a7210 0x7fb97013e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:56.682 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 10 monclient(hunting): picked mon.noname-f con 0x7fb9700a7210 addr v2:172.21.15.190:3300/0 2024-08-19T10:37:56.682 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9700a75f0 0x7fb970140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:56.682 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 10 monclient(hunting): picked mon.noname-c con 0x7fb9700a75f0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:56.682 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9700a34b0 con 0x7fb9700a6720 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb9700a3290 con 0x7fb9700a7210 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb9700a4df0 con 0x7fb9700a75f0 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb977043640 10 monclient(hunting): authenticate will time out at 926.496217s 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb974ddf640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb9700a6720 0x7fb9700a6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59396/0 (socket says 172.21.15.89:59396) 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb974ddf640 1 -- 172.21.15.89:0/3687720767 learned_addr learned my addr 172.21.15.89:0/3687720767 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9700a75f0 0x7fb970140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 10 monclient(hunting): get_auth_request con 0x7fb9700a75f0 auth_method 0 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:56.683 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb96ffff640 1 --2- 172.21.15.89:0/3687720767 >> v2:172.21.15.190:3300/0 conn(0x7fb9700a7210 0x7fb97013e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb96ffff640 10 monclient(hunting): get_auth_request con 0x7fb9700a7210 auth_method 0 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb96ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb96ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb96ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 10 monclient(hunting): handle_auth_done global_id 4162 payload 274 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 1 -- 172.21.15.89:0/3687720767 >> v2:172.21.15.190:3300/0 conn(0x7fb9700a7210 msgr2=0x7fb97013e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3687720767 >> v2:172.21.15.190:3300/0 conn(0x7fb9700a7210 0x7fb97013e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 1 -- 172.21.15.89:0/3687720767 >> v1:172.21.15.89:6789/0 conn(0x7fb9700a6720 legacy=0x7fb9700a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:56.684 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 10 monclient: _finish_hunting 0 2024-08-19T10:37:56.685 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 1 monclient: found mon.noname-c 2024-08-19T10:37:56.685 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:56.685 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:37:56.685 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.678+0000 7fb9755e0640 1 -- 172.21.15.89:0/3687720767 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9700a8310 con 0x7fb9700a75f0 2024-08-19T10:37:56.688 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 -- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9700a75f0 msgr2=0x7fb970140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:56.688 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9700a75f0 0x7fb970140ef0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb968009d90 tx=0x7fb96801c620 comp rx=0 tx=0).ready entity=mon.4 client_cookie=ac03776aff1f0390 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:56.688 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 -- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9700a75f0 msgr2=0x7fb970140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-08-19T10:37:56.688 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 -- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9700a75f0 msgr2=0x7fb970140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:37:56.688 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 1 -- 172.21.15.89:0/3687720767 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb9680ac070 con 0x7fb9700a75f0 2024-08-19T10:37:56.688 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:56.688 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9700a75f0 0x7fb970140ef0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb968009d90 tx=0x7fb96801c620 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:37:56.688 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9700a75f0 0x7fb970140ef0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb968009d90 tx=0x7fb96801c620 comp rx=0 tx=0).stop 2024-08-19T10:37:56.688 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:37:56.688 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient: dump: 2024-08-19T10:37:56.688 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:56.689 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient: ms_handle_reset current mon [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 1 -- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9700a75f0 msgr2=0x7fb970140ef0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9700a75f0 0x7fb970140ef0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient: _add_conns ranks=[1,3,2,6,0,7,4,8,5] 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9580044a0 0x7fb9580068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient(hunting): picked mon.f con 0x7fb9580044a0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:37:56.690 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9580070b0 0x7fb958009500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient(hunting): picked mon.g con 0x7fb9580070b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb958009de0 0x7fb95800c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient(hunting): picked mon.b con 0x7fb958009de0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 1 -- 172.21.15.89:0/3687720767 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb95800cbb0 con 0x7fb9580044a0 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 1 -- 172.21.15.89:0/3687720767 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb95800cd60 con 0x7fb958009de0 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 1 -- 172.21.15.89:0/3687720767 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb95800cf10 con 0x7fb9580070b0 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96f7fe640 10 monclient(hunting): ms_handle_reset stray mon [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb958009de0 0x7fb95800c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient(hunting): get_auth_request con 0x7fb958009de0 auth_method 0 2024-08-19T10:37:56.691 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb96ffff640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9580070b0 0x7fb958009500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient(hunting): handle_auth_done global_id 4162 payload 293 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb974ddf640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9580044a0 0x7fb9580068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 -- 172.21.15.89:0/3687720767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9580070b0 msgr2=0x7fb958009500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9580070b0 0x7fb958009500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 -- 172.21.15.89:0/3687720767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9580044a0 msgr2=0x7fb9580068b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9580044a0 0x7fb9580068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:56.692 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient: _finish_hunting 0 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 monclient: found mon.b 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 -- 172.21.15.89:0/3687720767 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=2+}) -- 0x7fb95800d1d0 con 0x7fb958009de0 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient: _finish_auth 0 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 10 monclient: _check_auth_tickets 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb977043640 5 monclient: authenticate success, global_id 4162 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb977043640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.682+0000 7fb9755e0640 1 -- 172.21.15.89:0/3687720767 reap_dead start 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.686+0000 7fb9755e0640 1 -- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb958009de0 msgr2=0x7fb95800c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.686+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb958009de0 0x7fb95800c270 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb9700a7ca0 tx=0x7fb968009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cadcd035a26bea4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.686+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb958009de0 0x7fb95800c270 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb9700a7ca0 tx=0x7fb968009750 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fb968003d90 mon_map magic: 0 2024-08-19T10:37:56.693 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.686+0000 7fb96f7fe640 1 -- 172.21.15.89:0/3687720767 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb968003d90 con 0x7fb958009de0 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.686+0000 7fb96f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.686+0000 7fb96f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.686+0000 7fb96f7fe640 10 monclient: dump: 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:56.694 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:56.695 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:56.695 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:56.695 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:56.695 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:56.695 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:37:56.695 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:56.686+0000 7fb977043640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:37:56.865 INFO:tasks.ceph.osd.1.smithi089.stderr:2024-08-19T10:37:56.862+0000 7fe84896d7c0 -1 Falling back to public interface 2024-08-19T10:37:57.127 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-08-19T10:37:57.125+0000 7f66e6af07c0 -1 Falling back to public interface 2024-08-19T10:37:57.313 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb96f7fe640 1 -- 172.21.15.89:0/3687720767 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb968003bb0 con 0x7fb958009de0 2024-08-19T10:37:57.313 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb96f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:57.313 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb977043640 10 monclient: get_monmap_and_config success 2024-08-19T10:37:57.313 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb977043640 4 set_mon_vals no callback set 2024-08-19T10:37:57.313 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb977043640 10 monclient: shutdown 2024-08-19T10:37:57.314 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb977043640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:57.314 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb977043640 1 -- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb958009de0 msgr2=0x7fb95800c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:57.314 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb977043640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb958009de0 0x7fb95800c270 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb9700a7ca0 tx=0x7fb968009750 comp rx=0 tx=0).stop 2024-08-19T10:37:57.314 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb977043640 1 -- 172.21.15.89:0/3687720767 shutdown_connections 2024-08-19T10:37:57.314 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb977043640 1 --2- 172.21.15.89:0/3687720767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb958009de0 0x7fb95800c270 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:57.314 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb977043640 1 -- 172.21.15.89:0/3687720767 >> 172.21.15.89:0/3687720767 conn(0x7fb9700abd20 msgr2=0x7fb9700ac130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:57.315 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb977043640 1 -- 172.21.15.89:0/3687720767 shutdown_connections 2024-08-19T10:37:57.315 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.310+0000 7fb977043640 1 -- 172.21.15.89:0/3687720767 wait complete. 2024-08-19T10:37:57.315 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient: build_initial_monmap 2024-08-19T10:37:57.315 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient: monmap: 2024-08-19T10:37:57.315 INFO:teuthology.misc.health.smithi089.stderr:epoch 0 2024-08-19T10:37:57.315 INFO:teuthology.misc.health.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:37:57.315 INFO:teuthology.misc.health.smithi089.stderr:last_changed 0.000000 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:created 0.000000 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:37:57.316 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:37:57.317 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 1 Processor -- start 2024-08-19T10:37:57.317 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 1 -- start start 2024-08-19T10:37:57.317 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient: init 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient: _add_conns ranks=[8,5,7,3,0,6,2,1,4] 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb958009de0 0x7fb9701d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient(hunting): picked mon.noname-e con 0x7fb958009de0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9701d1370 0x7fb9701d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient(hunting): picked mon.noname-h con 0x7fb9701d1370 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb97013daf0 0x7fb97013df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient(hunting): picked mon.noname-i con 0x7fb97013daf0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb9700a6c80 con 0x7fb9701d1370 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:57.320 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb9700a6b00 con 0x7fb97013daf0 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb958009de0 0x7fb9701d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb9700a6e00 con 0x7fb958009de0 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb958009de0 0x7fb9701d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:33208/0 (socket says 172.21.15.89:33208) 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb977043640 10 monclient(hunting): authenticate will time out at 927.134589s 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 learned_addr learned my addr 172.21.15.89:0/3727764355 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 10 monclient(hunting): get_auth_request con 0x7fb958009de0 auth_method 0 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:57.321 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb96ffff640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9701d1370 0x7fb9701d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb96ffff640 10 monclient(hunting): get_auth_request con 0x7fb9701d1370 auth_method 0 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb96ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb96ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb96ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb97013daf0 0x7fb97013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb9755e0640 10 monclient(hunting): get_auth_request con 0x7fb97013daf0 auth_method 0 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb9755e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb9755e0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:57.322 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb9755e0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb96ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb96ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb96ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 10 monclient(hunting): handle_auth_done global_id 4157 payload 824 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb97013daf0 msgr2=0x7fb97013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb97013daf0 0x7fb97013df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9701d1370 msgr2=0x7fb9701d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9701d1370 0x7fb9701d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 10 monclient: _finish_hunting 0 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 1 monclient: found mon.noname-e 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:37:57.323 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9701228a0 con 0x7fb958009de0 2024-08-19T10:37:57.324 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.314+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb97013daf0 0x7fb97013df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:37:57.328 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb958009de0 msgr2=0x7fb9701d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:57.328 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb974ddf640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb958009de0 0x7fb9701d0cd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb96000d480 tx=0x7fb96000d940 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:57.328 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb960005d00 con 0x7fb958009de0 2024-08-19T10:37:57.328 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:57.328 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:37:57.328 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: dump: 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:57.329 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb960017070 con 0x7fb958009de0 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb9600074e0 con 0x7fb958009de0 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb977043640 5 monclient: authenticate success, global_id 4157 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb977043640 10 monclient: _renew_subs 2024-08-19T10:37:57.330 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb977043640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96effd640 4 set_mon_vals no callback set 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb977043640 1 -- 172.21.15.89:0/3727764355 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb970139f60 con 0x7fb958009de0 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: dump: 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:57.331 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb977043640 10 monclient: _renew_subs 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb977043640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb977043640 1 -- 172.21.15.89:0/3727764355 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9700a2590 con 0x7fb958009de0 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 2) ==== 47504+0+0 (secure 0 0 0) 0x7fb960007070 con 0x7fb958009de0 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb977043640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb977043640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:57.332 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb977043640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb977043640 1 -- 172.21.15.89:0/3727764355 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb970050ce0 con 0x7fb958009de0 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fb96004d760 con 0x7fb958009de0 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb958009de0 msgr2=0x7fb9701d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb958009de0 msgr2=0x7fb9701d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb974ddf640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb958009de0 0x7fb9701d0cd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb96000d480 tx=0x7fb96000d940 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb974ddf640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb958009de0 0x7fb9701d0cd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb96000d480 tx=0x7fb96000d940 comp rx=0 tx=0).stop 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: ms_handle_reset current mon [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: _reopen_session rank -1 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb958009de0 msgr2=0x7fb9701d0cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb958009de0 0x7fb9701d0cd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient: _add_conns ranks=[4,1,5,6,7,0,2,8,3] 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb95c03cb30 0x7fb95c03ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:57.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient(hunting): picked mon.c con 0x7fb95c03cb30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb95c03f780 0x7fb95c041c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient(hunting): picked mon.f con 0x7fb95c03f780 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9701d1370 0x7fb95c044820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient(hunting): picked mon.h con 0x7fb9701d1370 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c045280 con 0x7fb95c03f780 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb95c045400 con 0x7fb95c03cb30 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb95c0455b0 con 0x7fb9701d1370 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb96d7fa640 10 monclient(hunting): _renew_subs 2024-08-19T10:37:57.334 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb974ddf640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb95c03cb30 0x7fb95c03ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb974ddf640 10 monclient(hunting): get_auth_request con 0x7fb95c03cb30 auth_method 0 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb974ddf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb974ddf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.318+0000 7fb974ddf640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96ffff640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb95c03f780 0x7fb95c041c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9701d1370 0x7fb95c044820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96ffff640 10 monclient(hunting): get_auth_request con 0x7fb95c03f780 auth_method 0 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:37:57.335 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 10 monclient(hunting): handle_auth_done global_id 4157 payload 293 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9701d1370 msgr2=0x7fb95c044820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9701d1370 0x7fb95c044820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb95c03f780 msgr2=0x7fb95c041c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb95c03f780 0x7fb95c041c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 10 monclient: _finish_hunting 0 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 1 monclient: found mon.c 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) -- 0x7fb95c0457b0 con 0x7fb95c03cb30 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb970050ce0 con 0x7fb95c03cb30 2024-08-19T10:37:57.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 10 monclient: _finish_auth 0 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 10 monclient: _check_auth_tickets 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb9755e0640 10 monclient: get_auth_request con 0x7fb9701d1370 auth_method 0 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb9755e0640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9701d1370 0x7fb95c044820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 1 -- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb95c03cb30 msgr2=0x7fb95c03ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb974ddf640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb95c03cb30 0x7fb95c03ef20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb960019850 tx=0x7fb960009040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb960019910 con 0x7fb95c03cb30 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96d7fa640 10 monclient: dump: 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:37:57.337 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:37:57.338 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:37:57.339 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96001c540 con 0x7fb95c03cb30 2024-08-19T10:37:57.339 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:37:57.339 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96e7fc640 4 set_mon_vals no callback set 2024-08-19T10:37:57.339 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.322+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mgrmap(e 2) ==== 47504+0+0 (secure 0 0 0) 0x7fb96001c740 con 0x7fb95c03cb30 2024-08-19T10:37:57.339 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.326+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb9701228a0 con 0x7fb95c03cb30 2024-08-19T10:37:57.339 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.326+0000 7fb96d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:37:57.339 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.326+0000 7fb96d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:37:57.366 INFO:tasks.ceph.osd.3.smithi190.stderr:2024-08-19T10:37:57.365+0000 7f7d22b767c0 -1 Falling back to public interface 2024-08-19T10:37:57.368 INFO:tasks.ceph.osd.2.smithi089.stderr:2024-08-19T10:37:57.366+0000 7f870e0d37c0 -1 Falling back to public interface 2024-08-19T10:37:57.396 INFO:tasks.ceph.osd.0.smithi089.stderr:2024-08-19T10:37:57.394+0000 7f55014b07c0 -1 Falling back to public interface 2024-08-19T10:37:57.398 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:37:57.398 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:37:57.398 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:37:57.398 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 1 -- 172.21.15.89:0/3727764355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9701394e0 con 0x7fb95c03cb30 2024-08-19T10:37:57.398 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb96d7fa640 1 -- 172.21.15.89:0/3727764355 <== mon.4 v2:172.21.15.89:3302/0 5 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5605 (secure 0 0 0) 0x7fb96004f8f0 con 0x7fb95c03cb30 2024-08-19T10:37:57.398 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb96d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:37:57.399 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb96d7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:37:57.399 INFO:teuthology.misc.health.smithi089.stdout: 2024-08-19T10:37:57.399 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":8,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","created":"2024-08-19T10:37:50.906618+0000","modified":"2024-08-19T10:37:55.695518+0000","last_up_change":"0.000000","last_in_change":"2024-08-19T10:37:55.695518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2be543b2-eb94-4907-8f21-6e91b7dd9733","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"224c1713-25c8-4a48-b16e-8929fcb6424b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2dcaafe0-5837-410b-ae99-c7561ef6b150","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5619c403-27fd-48de-8cee-c5e84289cfde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"82ab73a3-1212-4b0d-ab05-aa9c01dd5586","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T10:37:57.400 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 10 monclient: shutdown 2024-08-19T10:37:57.400 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:37:57.400 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 1 -- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb95c03cb30 msgr2=0x7fb95c03ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:37:57.400 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 1 --2- 172.21.15.89:0/3727764355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb95c03cb30 0x7fb95c03ef20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb960019850 tx=0x7fb960009040 comp rx=0 tx=0).stop 2024-08-19T10:37:57.400 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb9755e0640 1 -- 172.21.15.89:0/3727764355 reap_dead start 2024-08-19T10:37:57.400 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 1 -- 172.21.15.89:0/3727764355 shutdown_connections 2024-08-19T10:37:57.400 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 1 -- 172.21.15.89:0/3727764355 >> 172.21.15.89:0/3727764355 conn(0x7fb9700abd20 msgr2=0x7fb97009ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:37:57.400 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 1 -- 172.21.15.89:0/3727764355 shutdown_connections 2024-08-19T10:37:57.400 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:37:57.394+0000 7fb977043640 1 -- 172.21.15.89:0/3727764355 wait complete. 2024-08-19T10:37:57.402 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-08-19T10:37:57.401+0000 7f42ecb447c0 -1 Falling back to public interface 2024-08-19T10:37:57.415 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-08-19T10:37:57.461 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-08-19T10:37:57.457+0000 7f3938818640 -1 mgr.server handle_report got status from non-daemon mon.g 2024-08-19T10:37:57.461 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-08-19T10:37:57.457+0000 7f3938818640 -1 mgr.server handle_report got status from non-daemon mon.e 2024-08-19T10:37:57.465 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-08-19T10:37:57.461+0000 7f3938818640 -1 mgr.server handle_report got status from non-daemon mon.d 2024-08-19T10:37:57.465 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-08-19T10:37:57.465+0000 7f3938818640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-19T10:37:57.465 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-08-19T10:37:57.465+0000 7f3938818640 -1 mgr.server handle_report got status from non-daemon mon.f 2024-08-19T10:37:57.468 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-08-19T10:37:57.465+0000 7f3938818640 -1 mgr.server handle_report got status from non-daemon mon.i 2024-08-19T10:37:57.468 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-08-19T10:37:57.465+0000 7f3938818640 -1 mgr.server handle_report got status from non-daemon mon.h 2024-08-19T10:37:57.468 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-08-19T10:37:57.465+0000 7f3938818640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-19T10:38:00.839 INFO:tasks.ceph.osd.1.smithi089.stderr:2024-08-19T10:38:00.834+0000 7fe84896d7c0 -1 osd.1 0 log_to_monitors true 2024-08-19T10:38:01.130 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-08-19T10:38:01.129+0000 7f66e6af07c0 -1 osd.5 0 log_to_monitors true 2024-08-19T10:38:01.311 INFO:tasks.ceph.osd.3.smithi190.stderr:2024-08-19T10:38:01.309+0000 7f7d22b767c0 -1 osd.3 0 log_to_monitors true 2024-08-19T10:38:01.355 INFO:tasks.ceph.osd.2.smithi089.stderr:2024-08-19T10:38:01.350+0000 7f870e0d37c0 -1 osd.2 0 log_to_monitors true 2024-08-19T10:38:01.480 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-08-19T10:38:01.477+0000 7f42ecb447c0 -1 osd.4 0 log_to_monitors true 2024-08-19T10:38:01.488 INFO:tasks.ceph.osd.0.smithi089.stderr:2024-08-19T10:38:01.486+0000 7f55014b07c0 -1 osd.0 0 log_to_monitors true 2024-08-19T10:38:02.410 INFO:tasks.ceph.osd.1.smithi089.stderr:2024-08-19T10:38:02.406+0000 7fe844916640 -1 osd.1 0 waiting for initial osdmap 2024-08-19T10:38:02.435 INFO:tasks.ceph.osd.1.smithi089.stderr:2024-08-19T10:38:02.430+0000 7fe840750640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T10:38:03.119 INFO:tasks.ceph.osd.2.smithi089.stderr:2024-08-19T10:38:03.118+0000 7f870a88e640 -1 osd.2 0 waiting for initial osdmap 2024-08-19T10:38:03.145 INFO:tasks.ceph.osd.2.smithi089.stderr:2024-08-19T10:38:03.142+0000 7f8701eae640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T10:38:03.521 INFO:tasks.ceph.osd.3.smithi190.stderr:2024-08-19T10:38:03.517+0000 7f7d1eb1f640 -1 osd.3 0 waiting for initial osdmap 2024-08-19T10:38:03.536 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-08-19T10:38:03.533+0000 7f66e2a99640 -1 osd.5 0 waiting for initial osdmap 2024-08-19T10:38:03.537 INFO:tasks.ceph.osd.3.smithi190.stderr:2024-08-19T10:38:03.537+0000 7f7d16951640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T10:38:03.566 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-08-19T10:38:03.565+0000 7f66da8cb640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T10:38:03.622 INFO:tasks.ceph.osd.0.smithi089.stderr:2024-08-19T10:38:03.618+0000 7f54fd459640 -1 osd.0 0 waiting for initial osdmap 2024-08-19T10:38:03.631 INFO:tasks.ceph.osd.3.smithi190.stderr:2024-08-19T10:38:03.629+0000 7f7d1255c640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T10:38:03.645 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-08-19T10:38:03.645+0000 7f66d64d6640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T10:38:03.649 INFO:tasks.ceph.osd.0.smithi089.stderr:2024-08-19T10:38:03.646+0000 7f54f9293640 -1 osd.0 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T10:38:04.020 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-19T10:38:04.142 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.138+0000 7ff48f6fc640 10 monclient: get_monmap_and_config 2024-08-19T10:38:04.142 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.138+0000 7ff48f6fc640 10 monclient: build_initial_monmap 2024-08-19T10:38:04.142 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.138+0000 7ff48f6fc640 10 monclient: monmap: 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:epoch 0 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:38:04.143671+0000 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:38:04.143671+0000 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:04.143 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:04.144 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:04.144 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:04.144 INFO:teuthology.misc.health.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:04.144 INFO:teuthology.misc.health.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:04.144 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:04.145 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.138+0000 7ff48f6fc640 1 Processor -- start 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 1 -- start start 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 10 monclient: init 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 10 monclient: _add_conns ranks=[0,9,1,4,10,6,3,5,2,8,7] 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff4880a6720 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff4880a7210 0x7ff48813e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 10 monclient(hunting): picked mon.noname-a con 0x7ff4880a7210 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff4880a75f0 addr v1:172.21.15.190:6789/0 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4880a34b0 con 0x7ff4880a6720 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4880a4df0 con 0x7ff4880a75f0 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:04.146 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff4880a3290 con 0x7ff4880a7210 2024-08-19T10:38:04.147 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:04.147 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48f6fc640 10 monclient(hunting): authenticate will time out at 933.960837s 2024-08-19T10:38:04.147 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48d498640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff4880a6720 0x7ff4880a6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50458/0 (socket says 172.21.15.89:50458) 2024-08-19T10:38:04.147 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48d498640 1 -- 172.21.15.89:0/1739081217 learned_addr learned my addr 172.21.15.89:0/1739081217 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:04.147 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 1 --2- 172.21.15.89:0/1739081217 >> v2:172.21.15.89:3300/0 conn(0x7ff4880a7210 0x7ff48813e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:04.147 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 10 monclient(hunting): get_auth_request con 0x7ff4880a7210 auth_method 0 2024-08-19T10:38:04.147 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:04.147 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:04.147 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:04.148 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48d498640 1 -- 172.21.15.89:0/1739081217 >> v1:172.21.15.89:6789/0 conn(0x7ff4880a6720 legacy=0x7ff4880a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:04.148 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:04.148 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:04.148 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:04.148 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 10 monclient(hunting): handle_auth_done global_id 4212 payload 274 2024-08-19T10:38:04.148 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 1 -- 172.21.15.89:0/1739081217 >> v1:172.21.15.190:6789/0 conn(0x7ff4880a75f0 legacy=0x7ff488140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:04.148 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 1 -- 172.21.15.89:0/1739081217 >> v1:172.21.15.89:6789/0 conn(0x7ff4880a6720 legacy=0x7ff4880a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:04.148 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 10 monclient: _finish_hunting 0 2024-08-19T10:38:04.148 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 1 monclient: found mon.noname-a 2024-08-19T10:38:04.148 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:04.149 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:04.149 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.142+0000 7ff48cc97640 1 -- 172.21.15.89:0/1739081217 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4880a8310 con 0x7ff4880a7210 2024-08-19T10:38:04.151 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff48cc97640 1 -- 172.21.15.89:0/1739081217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4880a7210 msgr2=0x7ff48813e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:04.151 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff48cc97640 1 --2- 172.21.15.89:0/1739081217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4880a7210 0x7ff48813e9a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff478009870 tx=0x7ff47801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cbb102c676c37d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:04.151 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff47ffff640 1 -- 172.21.15.89:0/1739081217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4780b1070 con 0x7ff4880a7210 2024-08-19T10:38:04.151 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff47ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff47ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff47ffff640 10 monclient: dump: 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:04.152 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff47ffff640 10 monclient: _finish_auth 0 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff47ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff47ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff48f6fc640 5 monclient: authenticate success, global_id 4212 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff48f6fc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff47ffff640 1 -- 172.21.15.89:0/1739081217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4780ab820 con 0x7ff4880a7210 2024-08-19T10:38:04.153 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff47ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff48f6fc640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff48f6fc640 4 set_mon_vals no callback set 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff47ffff640 1 -- 172.21.15.89:0/1739081217 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4780aa310 con 0x7ff4880a7210 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.146+0000 7ff47ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff47ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff47ffff640 10 monclient: dump: 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:04.154 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:04.155 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:04.155 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:04.155 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:04.155 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:04.155 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:04.155 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:04.155 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:04.155 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:04.160 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient: shutdown 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 -- 172.21.15.89:0/1739081217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4880a7210 msgr2=0x7ff48813e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 --2- 172.21.15.89:0/1739081217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4880a7210 0x7ff48813e9a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff478009870 tx=0x7ff47801c3b0 comp rx=0 tx=0).stop 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 -- 172.21.15.89:0/1739081217 shutdown_connections 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 --2- 172.21.15.89:0/1739081217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4880a7210 0x7ff48813e9a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 -- 172.21.15.89:0/1739081217 >> 172.21.15.89:0/1739081217 conn(0x7ff4880abd20 msgr2=0x7ff4880ac130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 -- 172.21.15.89:0/1739081217 shutdown_connections 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 -- 172.21.15.89:0/1739081217 wait complete. 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient: build_initial_monmap 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient: monmap: 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:epoch 0 2024-08-19T10:38:04.161 INFO:teuthology.misc.health.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:created 0.000000 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:04.162 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 Processor -- start 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 -- start start 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient: init 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient: _add_conns ranks=[6,2,8,1,5,3,4,0,7] 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4880a6720 0x7ff4881289f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient(hunting): picked mon.noname-d con 0x7ff4880a6720 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4880a7210 0x7ff4881d1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient(hunting): picked mon.noname-b con 0x7ff4880a7210 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff4880a75f0 0x7ff4881263f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient(hunting): picked mon.noname-e con 0x7ff4880a75f0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:04.163 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff488144fb0 con 0x7ff4880a7210 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff488144e30 con 0x7ff4880a6720 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff488145130 con 0x7ff4880a75f0 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48f6fc640 10 monclient(hunting): authenticate will time out at 933.970148s 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff4880a75f0 0x7ff4881263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48cc97640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4880a7210 0x7ff4881d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff4880a75f0 0x7ff4881263f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:52984/0 (socket says 172.21.15.89:52984) 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 1 -- 172.21.15.89:0/2442373466 learned_addr learned my addr 172.21.15.89:0/2442373466 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 10 monclient(hunting): get_auth_request con 0x7ff4880a75f0 auth_method 0 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:04.164 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48d498640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4880a6720 0x7ff4881289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48cc97640 10 monclient(hunting): get_auth_request con 0x7ff4880a7210 auth_method 0 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48cc97640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48cc97640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48cc97640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48d498640 10 monclient(hunting): get_auth_request con 0x7ff4880a6720 auth_method 0 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48d498640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48d498640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48d498640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:04.165 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48cc97640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48cc97640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48cc97640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48d498640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48d498640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48d498640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 10 monclient(hunting): handle_auth_done global_id 4175 payload 824 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 1 -- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4880a6720 msgr2=0x7ff4881289f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4880a6720 0x7ff4881289f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 1 -- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4880a7210 msgr2=0x7ff4881d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4880a7210 0x7ff4881d1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 10 monclient: _finish_hunting 0 2024-08-19T10:38:04.166 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 1 monclient: found mon.noname-e 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48dc99640 1 -- 172.21.15.89:0/2442373466 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff488129380 con 0x7ff4880a75f0 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.150+0000 7ff48cc97640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4880a7210 0x7ff4881d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48dc99640 1 -- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff4880a75f0 msgr2=0x7ff4881263f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48dc99640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff4880a75f0 0x7ff4881263f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff48000b970 tx=0x7ff48000be30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 1 -- 172.21.15.89:0/2442373466 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff480019070 con 0x7ff4880a75f0 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 10 monclient: dump: 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:04.167 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:04.168 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:04.169 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:04.169 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 1 -- 172.21.15.89:0/2442373466 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff480007350 con 0x7ff4880a75f0 2024-08-19T10:38:04.169 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48f6fc640 5 monclient: authenticate success, global_id 4175 2024-08-19T10:38:04.169 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:04.169 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 1 -- 172.21.15.89:0/2442373466 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4800079b0 con 0x7ff4880a75f0 2024-08-19T10:38:04.169 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:04.169 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:04.169 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47effd640 4 set_mon_vals no callback set 2024-08-19T10:38:04.169 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 10 monclient: dump: 2024-08-19T10:38:04.169 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:38:04.169 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:04.170 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:04.170 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:04.170 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:04.170 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:04.170 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:04.170 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:04.170 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:04.170 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:04.170 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:04.170 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48f6fc640 10 monclient: _renew_subs 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48f6fc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48f6fc640 1 -- 172.21.15.89:0/2442373466 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff488127e90 con 0x7ff4880a75f0 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48f6fc640 10 monclient: _renew_subs 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48f6fc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48f6fc640 1 -- 172.21.15.89:0/2442373466 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4881419a0 con 0x7ff4880a75f0 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48f6fc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48f6fc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48f6fc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:04.171 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48f6fc640 1 -- 172.21.15.89:0/2442373466 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff488050ce0 con 0x7ff4880a75f0 2024-08-19T10:38:04.172 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 1 -- 172.21.15.89:0/2442373466 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 4) ==== 47737+0+0 (secure 0 0 0) 0x7ff480006d10 con 0x7ff4880a75f0 2024-08-19T10:38:04.172 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff46c03bd70 0x7ff46c03e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:04.172 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff47dffb640 1 -- 172.21.15.89:0/2442373466 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(11..11 src has 1..11) ==== 2487+0+0 (secure 0 0 0) 0x7ff48004d930 con 0x7ff4880a75f0 2024-08-19T10:38:04.172 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48d498640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff46c03bd70 0x7ff46c03e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:04.172 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.154+0000 7ff48d498640 10 monclient: get_auth_request con 0x7ff46c03bd70 auth_method 0 2024-08-19T10:38:04.173 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.158+0000 7ff48d498640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff46c03bd70 0x7ff46c03e230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff484000f70 tx=0x7ff484009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:04.173 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.158+0000 7ff47dffb640 1 -- 172.21.15.89:0/2442373466 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff480007df0 con 0x7ff4880a75f0 2024-08-19T10:38:04.173 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.158+0000 7ff47dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:04.173 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.158+0000 7ff47dffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:04.332 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.330+0000 7ff48f6fc640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:04.332 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.330+0000 7ff48f6fc640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:04.332 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.330+0000 7ff48f6fc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:04.332 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.330+0000 7ff48f6fc640 1 -- 172.21.15.89:0/2442373466 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4880a6b00 con 0x7ff4880a75f0 2024-08-19T10:38:04.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.330+0000 7ff47dffb640 1 -- 172.21.15.89:0/2442373466 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+6565 (secure 0 0 0) 0x7ff480007540 con 0x7ff4880a75f0 2024-08-19T10:38:04.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.330+0000 7ff47dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:04.333 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.330+0000 7ff47dffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:04.333 INFO:teuthology.misc.health.smithi089.stdout: 2024-08-19T10:38:04.334 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":11,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","created":"2024-08-19T10:37:50.906618+0000","modified":"2024-08-19T10:38:03.389892+0000","last_up_change":"2024-08-19T10:38:03.389892+0000","last_in_change":"2024-08-19T10:37:55.695518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2be543b2-eb94-4907-8f21-6e91b7dd9733","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"224c1713-25c8-4a48-b16e-8929fcb6424b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6801","nonce":375502586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6803","nonce":375502586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6807","nonce":375502586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6805","nonce":375502586}]},"public_addr":"172.21.15.89:6801/375502586","cluster_addr":"172.21.15.89:6803/375502586","heartbeat_back_addr":"172.21.15.89:6807/375502586","heartbeat_front_addr":"172.21.15.89:6805/375502586","state":["exists","up"]},{"osd":2,"uuid":"5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6809","nonce":1060450292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6811","nonce":1060450292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6815","nonce":1060450292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6813","nonce":1060450292}]},"public_addr":"172.21.15.89:6809/1060450292","cluster_addr":"172.21.15.89:6811/1060450292","heartbeat_back_addr":"172.21.15.89:6815/1060450292","heartbeat_front_addr":"172.21.15.89:6813/1060450292","state":["exists","up"]},{"osd":3,"uuid":"2dcaafe0-5837-410b-ae99-c7561ef6b150","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5619c403-27fd-48de-8cee-c5e84289cfde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"82ab73a3-1212-4b0d-ab05-aa9c01dd5586","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T10:38:04.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.330+0000 7ff48f6fc640 1 -- 172.21.15.89:0/2442373466 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff46c03bd70 msgr2=0x7ff46c03e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:04.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.330+0000 7ff48f6fc640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff46c03bd70 0x7ff46c03e230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff484000f70 tx=0x7ff484009040 comp rx=0 tx=0).stop 2024-08-19T10:38:04.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.330+0000 7ff48f6fc640 10 monclient: shutdown 2024-08-19T10:38:04.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.334+0000 7ff48f6fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:04.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.334+0000 7ff48f6fc640 1 -- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff4880a75f0 msgr2=0x7ff4881263f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:04.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.334+0000 7ff48f6fc640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff4880a75f0 0x7ff4881263f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff48000b970 tx=0x7ff48000be30 comp rx=0 tx=0).stop 2024-08-19T10:38:04.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.334+0000 7ff48f6fc640 1 -- 172.21.15.89:0/2442373466 shutdown_connections 2024-08-19T10:38:04.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.334+0000 7ff48f6fc640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff46c03bd70 0x7ff46c03e230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:04.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.334+0000 7ff48f6fc640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff4880a75f0 0x7ff4881263f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:04.336 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.334+0000 7ff48f6fc640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff4880a7210 0x7ff4881d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:04.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.334+0000 7ff48f6fc640 1 --2- 172.21.15.89:0/2442373466 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4880a6720 0x7ff4881289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:04.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.334+0000 7ff48f6fc640 1 -- 172.21.15.89:0/2442373466 >> 172.21.15.89:0/2442373466 conn(0x7ff4880abd20 msgr2=0x7ff4880aaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:04.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.334+0000 7ff48f6fc640 1 -- 172.21.15.89:0/2442373466 shutdown_connections 2024-08-19T10:38:04.337 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:04.334+0000 7ff48f6fc640 1 -- 172.21.15.89:0/2442373466 wait complete. 2024-08-19T10:38:04.351 DEBUG:teuthology.misc:2 of 6 OSDs are up 2024-08-19T10:38:04.489 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-08-19T10:38:04.489+0000 7f42e8aed640 -1 osd.4 0 waiting for initial osdmap 2024-08-19T10:38:04.518 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-08-19T10:38:04.517+0000 7f42e091f640 -1 osd.4 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T10:38:10.956 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-19T10:38:11.078 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.074+0000 7f156e3c2640 10 monclient: get_monmap_and_config 2024-08-19T10:38:11.078 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.074+0000 7f156e3c2640 10 monclient: build_initial_monmap 2024-08-19T10:38:11.078 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.074+0000 7f156e3c2640 10 monclient: monmap: 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:epoch 0 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:38:11.079796+0000 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:38:11.079796+0000 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:11.079 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:11.080 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:11.080 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:11.080 INFO:teuthology.misc.health.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:11.080 INFO:teuthology.misc.health.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:11.080 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:11.082 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.074+0000 7f156e3c2640 1 Processor -- start 2024-08-19T10:38:11.082 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.074+0000 7f156e3c2640 1 -- start start 2024-08-19T10:38:11.082 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.074+0000 7f156e3c2640 10 monclient: init 2024-08-19T10:38:11.082 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:11.082 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 10 monclient: _add_conns ranks=[9,3,0,1,10,2,4,7,6,8,5] 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f15680a6720 0x7f15680a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 10 monclient(hunting): picked mon.noname-a con 0x7f15680a6720 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15680a7040 0x7f15680a7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 10 monclient(hunting): picked mon.noname-g con 0x7f15680a7040 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f15681394e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15680a34b0 con 0x7f15681394e0 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f15680a3290 con 0x7f15680a6720 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f15680a4df0 con 0x7f15680a7040 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156e3c2640 10 monclient(hunting): authenticate will time out at 940.896402s 2024-08-19T10:38:11.083 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f15680a6720 0x7f15680a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156c95f640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f15681394e0 0x7f1568139920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50462/0 (socket says 172.21.15.89:50462) 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f156c95f640 1 -- 172.21.15.89:0/135556624 learned_addr learned my addr 172.21.15.89:0/135556624 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 10 monclient(hunting): get_auth_request con 0x7f15680a6720 auth_method 0 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f15677fe640 1 --2- 172.21.15.89:0/135556624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15680a7040 0x7f15680a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f15677fe640 10 monclient(hunting): get_auth_request con 0x7f15680a7040 auth_method 0 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f15677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.084 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f15677fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.085 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f15677fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.085 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 10 monclient(hunting): handle_auth_done global_id 4248 payload 274 2024-08-19T10:38:11.085 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 1 -- 172.21.15.89:0/135556624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15680a7040 msgr2=0x7f15680a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:11.085 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 1 --2- 172.21.15.89:0/135556624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15680a7040 0x7f15680a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.085 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 1 -- 172.21.15.89:0/135556624 >> v1:172.21.15.89:6789/0 conn(0x7f15681394e0 legacy=0x7f1568139920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:11.085 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:11.085 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 1 monclient: found mon.noname-a 2024-08-19T10:38:11.085 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:11.085 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:11.085 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.078+0000 7f1567fff640 1 -- 172.21.15.89:0/135556624 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15680a8310 con 0x7f15680a6720 2024-08-19T10:38:11.088 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 1 -- 172.21.15.89:0/135556624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 msgr2=0x7f15680a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:11.088 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 1 --2- 172.21.15.89:0/135556624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 0x7f15680a6b00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f15580097c0 tx=0x7f155801c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a45d31293334083 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 1 -- 172.21.15.89:0/135556624 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f15580b0070 con 0x7f15680a6720 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 10 monclient: dump: 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.089 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 10 monclient: _finish_auth 0 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:11.090 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 1 -- 172.21.15.89:0/135556624 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15580aa810 con 0x7f15680a6720 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 5 monclient: authenticate success, global_id 4248 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 1 -- 172.21.15.89:0/135556624 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f15580aae30 con 0x7f15680a6720 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 4 set_mon_vals no callback set 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1566ffd640 10 monclient: dump: 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.091 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.092 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient: shutdown 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 -- 172.21.15.89:0/135556624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 msgr2=0x7f15680a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 --2- 172.21.15.89:0/135556624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 0x7f15680a6b00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f15580097c0 tx=0x7f155801c1f0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 -- 172.21.15.89:0/135556624 shutdown_connections 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 --2- 172.21.15.89:0/135556624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15680a7040 0x7f15680a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 --2- 172.21.15.89:0/135556624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 0x7f15680a6b00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 -- 172.21.15.89:0/135556624 >> 172.21.15.89:0/135556624 conn(0x7f15680abd20 msgr2=0x7f15680ac130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 -- 172.21.15.89:0/135556624 shutdown_connections 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 -- 172.21.15.89:0/135556624 wait complete. 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient: build_initial_monmap 2024-08-19T10:38:11.093 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient: monmap: 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:epoch 0 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:created 0.000000 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:11.094 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 Processor -- start 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 -- start start 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient: init 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient: _add_conns ranks=[0,4,6,2,8,1,3,7,5] 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 0x7f1568137c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient(hunting): picked mon.noname-a con 0x7f15680a6720 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15680a7040 0x7f1568138190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient(hunting): picked mon.noname-c con 0x7f15680a7040 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:11.095 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f15681394e0 0x7f156813d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient(hunting): picked mon.noname-d con 0x7f15681394e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1568144360 con 0x7f15680a6720 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f15681441e0 con 0x7f15680a7040 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f15681444e0 con 0x7f15681394e0 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156e3c2640 10 monclient(hunting): authenticate will time out at 940.903476s 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f15677fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15680a7040 0x7f1568138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 0x7f1568137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 0x7f1568137c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35872/0 (socket says 172.21.15.89:35872) 2024-08-19T10:38:11.096 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 1 -- 172.21.15.89:0/2237635720 learned_addr learned my addr 172.21.15.89:0/2237635720 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 10 monclient(hunting): get_auth_request con 0x7f15680a6720 auth_method 0 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f15677fe640 10 monclient(hunting): get_auth_request con 0x7f15680a7040 auth_method 0 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f15677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f15677fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f15677fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156c95f640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f15681394e0 0x7f156813d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:11.097 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f15677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f15677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f15677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156c95f640 10 monclient(hunting): get_auth_request con 0x7f15681394e0 auth_method 0 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156c95f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156c95f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f156c95f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.082+0000 7f1567fff640 10 monclient(hunting): handle_auth_done global_id 4257 payload 824 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1567fff640 1 -- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f15681394e0 msgr2=0x7f156813d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1567fff640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f15681394e0 0x7f156813d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1567fff640 1 -- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15680a7040 msgr2=0x7f1568138190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1567fff640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15680a7040 0x7f1568138190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.098 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1567fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:11.099 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1567fff640 1 monclient: found mon.noname-a 2024-08-19T10:38:11.099 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1567fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:11.099 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1567fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:11.099 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1567fff640 1 -- 172.21.15.89:0/2237635720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1568122650 con 0x7f15680a6720 2024-08-19T10:38:11.099 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156c95f640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f15681394e0 0x7f156813d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:38:11.101 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1567fff640 1 -- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 msgr2=0x7f1568137c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:11.101 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1567fff640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 0x7f1568137c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f155801ca90 tx=0x7f1558018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:11.101 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 1 -- 172.21.15.89:0/2237635720 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f15580b0040 con 0x7f15680a6720 2024-08-19T10:38:11.101 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.101 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:11.101 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 10 monclient: dump: 2024-08-19T10:38:11.102 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:38:11.102 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.102 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.102 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.102 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.102 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.102 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.102 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.102 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.102 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.102 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 10 monclient: _finish_auth 0 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 1 -- 172.21.15.89:0/2237635720 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15580b9070 con 0x7f15680a6720 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 1 -- 172.21.15.89:0/2237635720 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f15580044b0 con 0x7f15680a6720 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.103 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1565ffb640 4 set_mon_vals no callback set 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 10 monclient: dump: 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:epoch 1 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.104 INFO:teuthology.misc.health.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr: 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156e3c2640 5 monclient: authenticate success, global_id 4257 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156e3c2640 10 monclient: _renew_subs 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156e3c2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156e3c2640 1 -- 172.21.15.89:0/2237635720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1568122930 con 0x7f15680a6720 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156e3c2640 10 monclient: _renew_subs 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156e3c2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156e3c2640 1 -- 172.21.15.89:0/2237635720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15680a2480 con 0x7f15680a6720 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156e3c2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156e3c2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156e3c2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:11.105 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f156e3c2640 1 -- 172.21.15.89:0/2237635720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1568050ce0 con 0x7f15680a6720 2024-08-19T10:38:11.106 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 1 -- 172.21.15.89:0/2237635720 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1558003620 con 0x7f15680a6720 2024-08-19T10:38:11.106 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f154803be00 0x7f154803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.106 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.086+0000 7f1564ff9640 1 -- 172.21.15.89:0/2237635720 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3993+0+0 (secure 0 0 0) 0x7f15580e7e90 con 0x7f15680a6720 2024-08-19T10:38:11.106 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.090+0000 7f15677fe640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f154803be00 0x7f154803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.106 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.090+0000 7f15677fe640 10 monclient: get_auth_request con 0x7f154803be00 auth_method 0 2024-08-19T10:38:11.106 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.090+0000 7f15677fe640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f154803be00 0x7f154803e2c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f156813eb90 tx=0x7f1550008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:11.106 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.094+0000 7f1564ff9640 1 -- 172.21.15.89:0/2237635720 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f15580c2060 con 0x7f15680a6720 2024-08-19T10:38:11.106 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.094+0000 7f1564ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:11.106 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.094+0000 7f1564ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:11.266 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.262+0000 7f156e3c2640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:11.266 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.262+0000 7f156e3c2640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:11.267 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.262+0000 7f156e3c2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:11.267 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.262+0000 7f156e3c2640 1 -- 172.21.15.89:0/2237635720 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15680a6b00 con 0x7f15680a6720 2024-08-19T10:38:11.267 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f1564ff9640 1 -- 172.21.15.89:0/2237635720 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10497 (secure 0 0 0) 0x7f15680a34b0 con 0x7f15680a6720 2024-08-19T10:38:11.267 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f1564ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:11.267 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f1564ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:11.268 INFO:teuthology.misc.health.smithi089.stdout: 2024-08-19T10:38:11.268 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":14,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","created":"2024-08-19T10:37:50.906618+0000","modified":"2024-08-19T10:38:07.114581+0000","last_up_change":"2024-08-19T10:38:04.611102+0000","last_in_change":"2024-08-19T10:37:55.695518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T10:38:04.494964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2be543b2-eb94-4907-8f21-6e91b7dd9733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6817","nonce":2476057681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6819","nonce":2476057681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6823","nonce":2476057681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6821","nonce":2476057681}]},"public_addr":"172.21.15.89:6817/2476057681","cluster_addr":"172.21.15.89:6819/2476057681","heartbeat_back_addr":"172.21.15.89:6823/2476057681","heartbeat_front_addr":"172.21.15.89:6821/2476057681","state":["exists","up"]},{"osd":1,"uuid":"224c1713-25c8-4a48-b16e-8929fcb6424b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6801","nonce":375502586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6803","nonce":375502586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6807","nonce":375502586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6805","nonce":375502586}]},"public_addr":"172.21.15.89:6801/375502586","cluster_addr":"172.21.15.89:6803/375502586","heartbeat_back_addr":"172.21.15.89:6807/375502586","heartbeat_front_addr":"172.21.15.89:6805/375502586","state":["exists","up"]},{"osd":2,"uuid":"5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6809","nonce":1060450292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6811","nonce":1060450292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6815","nonce":1060450292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6813","nonce":1060450292}]},"public_addr":"172.21.15.89:6809/1060450292","cluster_addr":"172.21.15.89:6811/1060450292","heartbeat_back_addr":"172.21.15.89:6815/1060450292","heartbeat_front_addr":"172.21.15.89:6813/1060450292","state":["exists","up"]},{"osd":3,"uuid":"2dcaafe0-5837-410b-ae99-c7561ef6b150","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6811","nonce":2751619171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6813","nonce":2751619171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6817","nonce":2751619171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6815","nonce":2751619171}]},"public_addr":"172.21.15.190:6811/2751619171","cluster_addr":"172.21.15.190:6813/2751619171","heartbeat_back_addr":"172.21.15.190:6817/2751619171","heartbeat_front_addr":"172.21.15.190:6815/2751619171","state":["exists","up"]},{"osd":4,"uuid":"5619c403-27fd-48de-8cee-c5e84289cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6819","nonce":1174298329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6821","nonce":1174298329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6825","nonce":1174298329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6823","nonce":1174298329}]},"public_addr":"172.21.15.190:6819/1174298329","cluster_addr":"172.21.15.190:6821/1174298329","heartbeat_back_addr":"172.21.15.190:6825/1174298329","heartbeat_front_addr":"172.21.15.190:6823/1174298329","state":["exists","up"]},{"osd":5,"uuid":"82ab73a3-1212-4b0d-ab05-aa9c01dd5586","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6803","nonce":2434772607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6805","nonce":2434772607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6809","nonce":2434772607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6807","nonce":2434772607}]},"public_addr":"172.21.15.190:6803/2434772607","cluster_addr":"172.21.15.190:6805/2434772607","heartbeat_back_addr":"172.21.15.190:6809/2434772607","heartbeat_front_addr":"172.21.15.190:6807/2434772607","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.511833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:01.832703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.351820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.269178+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.511490+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.178520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T10:38:11.270 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 -- 172.21.15.89:0/2237635720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f154803be00 msgr2=0x7f154803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.270 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f154803be00 0x7f154803e2c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f156813eb90 tx=0x7f1550008040 comp rx=0 tx=0).stop 2024-08-19T10:38:11.270 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 10 monclient: shutdown 2024-08-19T10:38:11.270 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:11.270 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 -- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 msgr2=0x7f1568137c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.270 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 0x7f1568137c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f155801ca90 tx=0x7f1558018f90 comp rx=0 tx=0).stop 2024-08-19T10:38:11.270 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 -- 172.21.15.89:0/2237635720 shutdown_connections 2024-08-19T10:38:11.270 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f154803be00 0x7f154803e2c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.270 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f15681394e0 0x7f156813d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.270 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f15680a7040 0x7f1568138190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.271 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 --2- 172.21.15.89:0/2237635720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15680a6720 0x7f1568137c50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.271 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 -- 172.21.15.89:0/2237635720 >> 172.21.15.89:0/2237635720 conn(0x7f15680abd20 msgr2=0x7f1568141100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:11.271 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 -- 172.21.15.89:0/2237635720 shutdown_connections 2024-08-19T10:38:11.271 INFO:teuthology.misc.health.smithi089.stderr:2024-08-19T10:38:11.266+0000 7f156e3c2640 1 -- 172.21.15.89:0/2237635720 wait complete. 2024-08-19T10:38:11.285 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-08-19T10:38:11.285 INFO:tasks.ceph:Creating RBD pool 2024-08-19T10:38:11.285 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd pool create rbd 8 2024-08-19T10:38:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.398+0000 7f4941e77640 10 monclient: get_monmap_and_config 2024-08-19T10:38:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.398+0000 7f4941e77640 10 monclient: build_initial_monmap 2024-08-19T10:38:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.398+0000 7f4941e77640 10 monclient: monmap: 2024-08-19T10:38:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:11.404802+0000 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:11.404802+0000 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:11.404 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:11.405 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:11.405 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:11.405 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 1 Processor -- start 2024-08-19T10:38:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 1 -- start start 2024-08-19T10:38:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 10 monclient: init 2024-08-19T10:38:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:11.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 10 monclient: _add_conns ranks=[10,6,3,8,1,7,9,5,2,4,0] 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f493c0a5270 0x7f493c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 10 monclient(hunting): picked mon.noname-f con 0x7f493c0a5270 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f493c0a6d30 0x7f493c0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 10 monclient(hunting): picked mon.noname-d con 0x7f493c0a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f493c1395b0 0x7f493c139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 10 monclient(hunting): picked mon.noname-g con 0x7f493c1395b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f493c0a7650 con 0x7f493c0a5270 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f493c0a8710 con 0x7f493c1395b0 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f493c0a5a40 con 0x7f493c0a6d30 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f4941e77640 10 monclient(hunting): authenticate will time out at 941.221406s 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f493c0a6d30 0x7f493c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f493c0a6d30 0x7f493c0a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46374/0 (socket says 172.21.15.89:46374) 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 1 -- 172.21.15.89:0/3034880573 learned_addr learned my addr 172.21.15.89:0/3034880573 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 10 monclient(hunting): get_auth_request con 0x7f493c0a6d30 auth_method 0 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:11.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493b7fe640 1 --2- 172.21.15.89:0/3034880573 >> v2:172.21.15.190:3300/0 conn(0x7f493c0a5270 0x7f493c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493bfff640 1 --2- 172.21.15.89:0/3034880573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f493c1395b0 0x7f493c139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493bfff640 10 monclient(hunting): get_auth_request con 0x7f493c1395b0 auth_method 0 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493b7fe640 10 monclient(hunting): get_auth_request con 0x7f493c0a5270 auth_method 0 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 10 monclient(hunting): handle_auth_done global_id 4173 payload 274 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 1 -- 172.21.15.89:0/3034880573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f493c1395b0 msgr2=0x7f493c139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:11.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 1 --2- 172.21.15.89:0/3034880573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f493c1395b0 0x7f493c139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 1 -- 172.21.15.89:0/3034880573 >> v2:172.21.15.190:3300/0 conn(0x7f493c0a5270 msgr2=0x7f493c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:11.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 1 --2- 172.21.15.89:0/3034880573 >> v2:172.21.15.190:3300/0 conn(0x7f493c0a5270 0x7f493c0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 10 monclient: _finish_hunting 0 2024-08-19T10:38:11.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 1 monclient: found mon.noname-d 2024-08-19T10:38:11.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:11.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:11.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.402+0000 7f493affd640 1 -- 172.21.15.89:0/3034880573 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c0a81d0 con 0x7f493c0a6d30 2024-08-19T10:38:11.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493affd640 1 -- 172.21.15.89:0/3034880573 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f493c0a6d30 msgr2=0x7f493c0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:11.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493affd640 1 --2- 172.21.15.89:0/3034880573 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f493c0a6d30 0x7f493c0a7110 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4930009870 tx=0x7f493001c420 comp rx=0 tx=0).ready entity=mon.6 client_cookie=fb9c93e05af2749f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 1 -- 172.21.15.89:0/3034880573 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49300b1070 con 0x7f493c0a6d30 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 10 monclient: dump: 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.415 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 1 -- 172.21.15.89:0/3034880573 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49300ab810 con 0x7f493c0a6d30 2024-08-19T10:38:11.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 5 monclient: authenticate success, global_id 4173 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 1 -- 172.21.15.89:0/3034880573 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49300abe30 con 0x7f493c0a6d30 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f493a7fc640 10 monclient: dump: 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.417 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 4 set_mon_vals no callback set 2024-08-19T10:38:11.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 10 monclient: shutdown 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 1 -- 172.21.15.89:0/3034880573 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f493c0a6d30 msgr2=0x7f493c0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 1 --2- 172.21.15.89:0/3034880573 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f493c0a6d30 0x7f493c0a7110 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4930009870 tx=0x7f493001c420 comp rx=0 tx=0).stop 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 1 -- 172.21.15.89:0/3034880573 shutdown_connections 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 1 --2- 172.21.15.89:0/3034880573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f493c1395b0 0x7f493c139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 1 --2- 172.21.15.89:0/3034880573 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f493c0a6d30 0x7f493c0a7110 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 1 --2- 172.21.15.89:0/3034880573 >> v2:172.21.15.190:3300/0 conn(0x7f493c0a5270 0x7f493c0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 1 -- 172.21.15.89:0/3034880573 >> 172.21.15.89:0/3034880573 conn(0x7f493c069a80 msgr2=0x7f493c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 1 -- 172.21.15.89:0/3034880573 shutdown_connections 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.406+0000 7f4941e77640 1 -- 172.21.15.89:0/3034880573 wait complete. 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient: build_initial_monmap 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient: monmap: 2024-08-19T10:38:11.419 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:11.420 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 1 Processor -- start 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 1 -- start start 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient: init 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient: _add_conns ranks=[5,0,8,7,4,6,2,3,1] 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f493c0a5270 0x7f493c13d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient(hunting): picked mon.noname-h con 0x7f493c0a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c0a6d30 0x7f493c13d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient(hunting): picked mon.noname-a con 0x7f493c0a6d30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f493c1395b0 0x7f493c1d1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient(hunting): picked mon.noname-e con 0x7f493c1395b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:11.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f493c0a5650 con 0x7f493c0a6d30 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f493c0a4a70 con 0x7f493c0a5270 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f493c0a57d0 con 0x7f493c1395b0 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f4941e77640 10 monclient(hunting): authenticate will time out at 941.229445s 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f493c1395b0 0x7f493c1d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c0a6d30 0x7f493c13d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c0a6d30 0x7f493c13d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57642/0 (socket says 172.21.15.89:57642) 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 1 -- 172.21.15.89:0/2379230425 learned_addr learned my addr 172.21.15.89:0/2379230425 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:11.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 10 monclient(hunting): get_auth_request con 0x7f493c0a6d30 auth_method 0 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493bfff640 10 monclient(hunting): get_auth_request con 0x7f493c1395b0 auth_method 0 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:11.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493b7fe640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f493c0a5270 0x7f493c13d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 10 monclient(hunting): handle_auth_done global_id 4266 payload 824 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 1 -- 172.21.15.89:0/2379230425 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f493c1395b0 msgr2=0x7f493c1d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f493c1395b0 0x7f493c1d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 1 -- 172.21.15.89:0/2379230425 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f493c0a5270 msgr2=0x7f493c13d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f493c0a5270 0x7f493c13d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 10 monclient: _finish_hunting 0 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 1 monclient: found mon.noname-a 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493affd640 1 -- 172.21.15.89:0/2379230425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c1405f0 con 0x7f493c0a6d30 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493b7fe640 10 monclient: get_auth_request con 0x7f493c0a5270 auth_method 0 2024-08-19T10:38:11.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.410+0000 7f493b7fe640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f493c0a5270 0x7f493c13d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f493affd640 1 -- 172.21.15.89:0/2379230425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c0a6d30 msgr2=0x7f493c13d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f493affd640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c0a6d30 0x7f493c13d920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f493001aad0 tx=0x7f49300191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 1 -- 172.21.15.89:0/2379230425 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49300ba070 con 0x7f493c0a6d30 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 10 monclient: dump: 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.427 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 10 monclient: _finish_auth 0 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:11.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4941e77640 5 monclient: authenticate success, global_id 4266 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 1 -- 172.21.15.89:0/2379230425 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4930004250 con 0x7f493c0a6d30 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 1 -- 172.21.15.89:0/2379230425 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49300b1040 con 0x7f493c0a6d30 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 10 monclient: dump: 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.429 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4939ffb640 4 set_mon_vals no callback set 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4941e77640 10 monclient: _renew_subs 2024-08-19T10:38:11.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4941e77640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4941e77640 1 -- 172.21.15.89:0/2379230425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c13f990 con 0x7f493c0a6d30 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4941e77640 10 monclient: _renew_subs 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4941e77640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4941e77640 1 -- 172.21.15.89:0/2379230425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c0a28e0 con 0x7f493c0a6d30 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4941e77640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4941e77640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4941e77640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4941e77640 1 -- 172.21.15.89:0/2379230425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f493c064590 con 0x7f493c0a6d30 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 1 -- 172.21.15.89:0/2379230425 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4930003620 con 0x7f493c0a6d30 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f492803be00 0x7f492803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f4923fff640 1 -- 172.21.15.89:0/2379230425 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3993+0+0 (secure 0 0 0) 0x7f49300e7a90 con 0x7f493c0a6d30 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f493b7fe640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f492803be00 0x7f492803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.414+0000 7f493b7fe640 10 monclient: get_auth_request con 0x7f492803be00 auth_method 0 2024-08-19T10:38:11.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.418+0000 7f493b7fe640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f492803be00 0x7f492803e2c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f492c0097a0 tx=0x7f492c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:11.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.418+0000 7f4923fff640 1 -- 172.21.15.89:0/2379230425 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4930016130 con 0x7f493c0a6d30 2024-08-19T10:38:11.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.418+0000 7f4923fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:11.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.418+0000 7f4923fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.590+0000 7f4941e77640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-08-19T10:38:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.590+0000 7f4941e77640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-08-19T10:38:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.590+0000 7f4941e77640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:11.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.590+0000 7f4941e77640 1 -- 172.21.15.89:0/2379230425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f493c06c9e0 con 0x7f493c0a6d30 2024-08-19T10:38:11.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.786+0000 7f4923fff640 1 -- 172.21.15.89:0/2379230425 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) ==== 109+0+0 (secure 0 0 0) 0x7f4930004400 con 0x7f493c0a6d30 2024-08-19T10:38:11.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.786+0000 7f4923fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-08-19T10:38:11.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.786+0000 7f4923fff640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2024-08-19T10:38:11.789 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'rbd' created 2024-08-19T10:38:11.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 -- 172.21.15.89:0/2379230425 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f492803be00 msgr2=0x7f492803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f492803be00 0x7f492803e2c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f492c0097a0 tx=0x7f492c008040 comp rx=0 tx=0).stop 2024-08-19T10:38:11.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 10 monclient: shutdown 2024-08-19T10:38:11.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:11.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 -- 172.21.15.89:0/2379230425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c0a6d30 msgr2=0x7f493c13d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c0a6d30 0x7f493c13d920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f493001aad0 tx=0x7f49300191c0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 -- 172.21.15.89:0/2379230425 shutdown_connections 2024-08-19T10:38:11.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f492803be00 0x7f492803e2c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f493c1395b0 0x7f493c1d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f493c0a6d30 0x7f493c13d920 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 --2- 172.21.15.89:0/2379230425 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f493c0a5270 0x7f493c13d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 -- 172.21.15.89:0/2379230425 >> 172.21.15.89:0/2379230425 conn(0x7f493c069a80 msgr2=0x7f493c0ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 -- 172.21.15.89:0/2379230425 shutdown_connections 2024-08-19T10:38:11.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.790+0000 7f4941e77640 1 -- 172.21.15.89:0/2379230425 wait complete. 2024-08-19T10:38:11.817 DEBUG:teuthology.orchestra.run.smithi089:> rbd --cluster ceph pool init rbd 2024-08-19T10:38:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.878+0000 7f38bcef6100 10 monclient: get_monmap_and_config 2024-08-19T10:38:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.878+0000 7f38bcef6100 10 monclient: build_initial_monmap 2024-08-19T10:38:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.878+0000 7f38bcef6100 10 monclient: monmap: 2024-08-19T10:38:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:11.883698+0000 2024-08-19T10:38:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:11.883698+0000 2024-08-19T10:38:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:11.885 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.878+0000 7f38bcef6100 1 Processor -- start 2024-08-19T10:38:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.878+0000 7f38bcef6100 1 -- start start 2024-08-19T10:38:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.878+0000 7f38bcef6100 10 monclient: init 2024-08-19T10:38:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 10 monclient: _reopen_session rank -1 2024-08-19T10:38:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 10 monclient: _add_conns ranks=[7,0,6,8,4,1,10,5,3,9,2] 2024-08-19T10:38:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x55d93730ee50 0x55d93730f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 10 monclient(hunting): picked mon.noname-i con 0x55d93730ee50 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 10 monclient(hunting): picked mon.noname-a-legacy con 0x55d93727a1f0 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 0x55d937316d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 10 monclient(hunting): picked mon.noname-d con 0x55d93727c410 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55d9370ec160 con 0x55d93727a1f0 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x55d937146e10 con 0x55d93727c410 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x55d937157910 con 0x55d93730ee50 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 10 monclient(hunting): _renew_subs 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcef6100 10 monclient(hunting): authenticate will time out at 941.700514s 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bb1ff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55d93727a1f0 0x55d93727bd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54128/0 (socket says 172.21.15.89:54128) 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bb1ff640 1 -- 172.21.15.89:0/2436827872 learned_addr learned my addr 172.21.15.89:0/2436827872 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 1 --2- 172.21.15.89:0/2436827872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 0x55d937316d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 10 monclient(hunting): get_auth_request con 0x55d93727c410 auth_method 0 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bba00640 1 --2- 172.21.15.89:0/2436827872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x55d93730ee50 0x55d93730f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bba00640 10 monclient(hunting): get_auth_request con 0x55d93730ee50 auth_method 0 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bba00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bba00640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bba00640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 10 monclient(hunting): handle_auth_done global_id 4182 payload 274 2024-08-19T10:38:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 1 -- 172.21.15.89:0/2436827872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x55d93730ee50 msgr2=0x55d93730f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 1 --2- 172.21.15.89:0/2436827872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x55d93730ee50 0x55d93730f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 1 -- 172.21.15.89:0/2436827872 >> v1:172.21.15.89:6789/0 conn(0x55d93727a1f0 legacy=0x55d93727bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 10 monclient: _finish_hunting 0 2024-08-19T10:38:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 1 monclient: found mon.noname-d 2024-08-19T10:38:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.882+0000 7f38bcc62640 1 -- 172.21.15.89:0/2436827872 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9370c6c50 con 0x55d93727c410 2024-08-19T10:38:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcc62640 1 -- 172.21.15.89:0/2436827872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 msgr2=0x55d937316d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:11.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcc62640 1 --2- 172.21.15.89:0/2436827872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 0x55d937316d60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00a4d0 tx=0x7f38ac01b000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=263b3796da90bf84 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 1 -- 172.21.15.89:0/2436827872 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38ac0b0070 con 0x55d93727c410 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 10 monclient: dump: 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.895 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 1 -- 172.21.15.89:0/2436827872 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac0a9340 con 0x55d93727c410 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 5 monclient: authenticate success, global_id 4182 2024-08-19T10:38:11.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient: get_monmap_and_config success 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 1 -- 172.21.15.89:0/2436827872 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38ac0a9960 con 0x55d93727c410 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 4 set_mon_vals no callback set 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38ba9fe640 10 monclient: dump: 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.897 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient: shutdown 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 -- 172.21.15.89:0/2436827872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 msgr2=0x55d937316d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 --2- 172.21.15.89:0/2436827872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 0x55d937316d60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00a4d0 tx=0x7f38ac01b000 comp rx=0 tx=0).stop 2024-08-19T10:38:11.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 -- 172.21.15.89:0/2436827872 shutdown_connections 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 --2- 172.21.15.89:0/2436827872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 0x55d937316d60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 --2- 172.21.15.89:0/2436827872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x55d93730ee50 0x55d93730f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 -- 172.21.15.89:0/2436827872 >> 172.21.15.89:0/2436827872 conn(0x55d93725ddf0 msgr2=0x55d93725e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 -- 172.21.15.89:0/2436827872 shutdown_connections 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 -- 172.21.15.89:0/2436827872 wait complete. 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient: build_initial_monmap 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient: monmap: 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:11.899 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 Processor -- start 2024-08-19T10:38:11.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 -- start start 2024-08-19T10:38:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient: init 2024-08-19T10:38:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient: _reopen_session rank -1 2024-08-19T10:38:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient: _add_conns ranks=[3,6,0,5,4,7,2,1,8] 2024-08-19T10:38:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d93727a1f0 0x55d93730e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient(hunting): picked mon.noname-g con 0x55d93727a1f0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 0x55d93730e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient(hunting): picked mon.noname-d con 0x55d93727c410 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d93730ee50 0x55d937322f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient(hunting): picked mon.noname-a con 0x55d93730ee50 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d9370ed860 con 0x55d93730ee50 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55d93714ae50 con 0x55d93727a1f0 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x55d9371c5250 con 0x55d93727c410 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient(hunting): _renew_subs 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcef6100 10 monclient(hunting): authenticate will time out at 941.707823s 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bb1ff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 0x55d93730e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bb1ff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 0x55d93730e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46400/0 (socket says 172.21.15.89:46400) 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bcc62640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d93730ee50 0x55d937322f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bb1ff640 1 -- 172.21.15.89:0/3962507936 learned_addr learned my addr 172.21.15.89:0/3962507936 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bb1ff640 10 monclient(hunting): get_auth_request con 0x55d93727c410 auth_method 0 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bb1ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bb1ff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.886+0000 7f38bb1ff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcc62640 10 monclient(hunting): get_auth_request con 0x55d93730ee50 auth_method 0 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcc62640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcc62640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcc62640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bba00640 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d93727a1f0 0x55d93730e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcc62640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcc62640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcc62640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:11.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bba00640 10 monclient(hunting): get_auth_request con 0x55d93727a1f0 auth_method 0 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bba00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bba00640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bba00640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bba00640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bba00640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bba00640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 10 monclient(hunting): handle_auth_done global_id 4191 payload 824 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 1 -- 172.21.15.89:0/3962507936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d93727a1f0 msgr2=0x55d93730e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d93727a1f0 0x55d93730e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 1 -- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d93730ee50 msgr2=0x55d937322f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d93730ee50 0x55d937322f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 1 monclient: found mon.noname-d 2024-08-19T10:38:11.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d937312d60 con 0x55d93727c410 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 1 -- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 msgr2=0x55d93730e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bb1ff640 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 0x55d93730e580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f38a400bef0 tx=0x7f38a4009650 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38a400ed10 con 0x55d93727c410 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 10 monclient: dump: 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.905 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 10 monclient: _finish_auth 0 2024-08-19T10:38:11.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a400d3e0 con 0x55d93727c410 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcef6100 5 monclient: authenticate success, global_id 4191 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcef6100 10 monclient: _renew_subs 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcef6100 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d93727fd70 con 0x55d93727c410 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38a400da00 con 0x55d93727c410 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38ba1fd640 4 set_mon_vals no callback set 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 10 monclient: dump: 2024-08-19T10:38:11.907 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:11.908 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:11.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcef6100 10 monclient: _renew_subs 2024-08-19T10:38:11.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcef6100 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:11.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x55d9373150c0 con 0x55d93727c410 2024-08-19T10:38:11.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.890+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f38a400dc20 con 0x55d93727c410 2024-08-19T10:38:11.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.894+0000 7f38a3fff640 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f389c03a620 0x7f389c03cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:11.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.894+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(15..15 src has 1..15) ==== 4335+0+0 (secure 0 0 0) 0x7f38a4005050 con 0x55d93727c410 2024-08-19T10:38:11.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.894+0000 7f38bcef6100 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-08-19T10:38:11.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.894+0000 7f38bcef6100 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-08-19T10:38:11.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.894+0000 7f38bcef6100 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:11.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.894+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x55d937315880 con 0x55d93727c410 2024-08-19T10:38:11.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.894+0000 7f38bba00640 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f389c03a620 0x7f389c03cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:11.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.894+0000 7f38bba00640 10 monclient: get_auth_request con 0x7f389c03a620 auth_method 0 2024-08-19T10:38:11.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:11.894+0000 7f38bba00640 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f389c03a620 0x7f389c03cae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009710 tx=0x7f38b0005a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:12.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v16) ==== 141+0+0 (secure 0 0 0) 0x7f38a4012070 con 0x55d93727c410 2024-08-19T10:38:12.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38a3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-08-19T10:38:12.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38a3fff640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2024-08-19T10:38:12.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38bcef6100 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:12.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38ac0a9340 con 0x55d93727c410 2024-08-19T10:38:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f38a4003e20 con 0x55d93727c410 2024-08-19T10:38:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38a3fff640 10 monclient: handle_get_version_reply finishing 1 version 16 2024-08-19T10:38:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38ba1fd640 10 monclient: _renew_subs 2024-08-19T10:38:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38ba1fd640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38ba1fd640 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=16}) -- 0x7f3894002c50 con 0x55d93727c410 2024-08-19T10:38:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 8 ==== osd_map(16..16 src has 1..16) ==== 674+0+0 (secure 0 0 0) 0x7f38a400d590 con 0x55d93727c410 2024-08-19T10:38:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38bcef6100 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x55d93732a2e0 0x55d93732c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) -- 0x55d93732cc10 con 0x55d93732a2e0 2024-08-19T10:38:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38bcc62640 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x55d93732a2e0 0x55d93732c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.814+0000 7f38bcc62640 10 monclient: get_auth_request con 0x55d93732a2e0 auth_method 0 2024-08-19T10:38:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.822+0000 7f38bcc62640 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x55d93732a2e0 0x55d93732c6d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.826+0000 7f38bcc62640 1 -- 172.21.15.89:0/3962507936 <== osd.1 v2:172.21.15.89:6800/375502586 1 ==== osd_op_reply(1 rbd_trash [create] v16'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7f38ac0a95f0 con 0x55d93732a2e0 2024-08-19T10:38:12.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.830+0000 7f38bcef6100 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x55d93737f3c0 0x55d937381850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:12.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.830+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16) -- 0x55d937381db0 con 0x55d93737f3c0 2024-08-19T10:38:12.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.830+0000 7f38bba00640 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x55d93737f3c0 0x55d937381850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:12.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.830+0000 7f38bba00640 10 monclient: get_auth_request con 0x55d93737f3c0 auth_method 0 2024-08-19T10:38:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.834+0000 7f38bba00640 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x55d93737f3c0 0x55d937381850 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.834+0000 7f38bba00640 1 -- 172.21.15.89:0/3962507936 <== osd.0 v2:172.21.15.89:6816/2476057681 1 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f38b0008750 con 0x55d93737f3c0 2024-08-19T10:38:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.834+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16) -- 0x55d93737dea0 con 0x55d93737f3c0 2024-08-19T10:38:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.838+0000 7f38bba00640 1 -- 172.21.15.89:0/3962507936 <== osd.0 v2:172.21.15.89:6816/2476057681 2 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f38b0008750 con 0x55d93737f3c0 2024-08-19T10:38:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.838+0000 7f38b99fc640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:12.838+0000 7f38b99fc640 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7f3898001f10 con 0x55d93727c410 2024-08-19T10:38:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:13.810+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 9 ==== pool_op_reply(tid 4 (0) Success v17) ==== 55+0+0 (secure 0 0 0) 0x7f38a4049020 con 0x55d93727c410 2024-08-19T10:38:13.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:13.810+0000 7f38a3fff640 10 monclient: _renew_subs 2024-08-19T10:38:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:13.810+0000 7f38a3fff640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:13.810+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=17}) -- 0x7f389c0407a0 con 0x55d93727c410 2024-08-19T10:38:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:13.810+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 10 ==== osd_map(17..17 src has 1..17) ==== 626+0+0 (secure 0 0 0) 0x7f38a400e8a0 con 0x55d93727c410 2024-08-19T10:38:13.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:13.810+0000 7f38ba1fd640 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) -- 0x7f3894005960 con 0x55d93737f3c0 2024-08-19T10:38:13.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:13.814+0000 7f38bba00640 1 -- 172.21.15.89:0/3962507936 <== osd.0 v2:172.21.15.89:6816/2476057681 3 ==== osd_op_reply(5 rbd_info [create,write 0~8] v17'1 uv1 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f38b0008750 con 0x55d93737f3c0 2024-08-19T10:38:13.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:13.814+0000 7f38b99fc640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:13.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:13.814+0000 7f38b99fc640 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v17) -- 0x7f3898003360 con 0x55d93727c410 2024-08-19T10:38:14.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.838+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 11 ==== pool_op_reply(tid 6 (0) Success v18) ==== 43+0+0 (secure 0 0 0) 0x7f38a4025020 con 0x55d93727c410 2024-08-19T10:38:14.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.838+0000 7f38a3fff640 10 monclient: _renew_subs 2024-08-19T10:38:14.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.838+0000 7f38a3fff640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:14.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.838+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=18}) -- 0x7f389c040a20 con 0x55d93727c410 2024-08-19T10:38:14.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.838+0000 7f38a3fff640 1 -- 172.21.15.89:0/3962507936 <== mon.6 v2:172.21.15.89:3303/0 12 ==== osd_map(18..18 src has 1..18) ==== 654+0+0 (secure 0 0 0) 0x7f38a4003310 con 0x55d93727c410 2024-08-19T10:38:14.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.838+0000 7f38ba1fd640 1 -- 172.21.15.89:0/3962507936 --> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) -- 0x7f3894005d00 con 0x55d93737f3c0 2024-08-19T10:38:14.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bba00640 1 -- 172.21.15.89:0/3962507936 <== osd.0 v2:172.21.15.89:6816/2476057681 4 ==== osd_op_reply(7 rbd_info [write 0~19] v18'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38b0008750 con 0x55d93737f3c0 2024-08-19T10:38:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x55d93737f3c0 msgr2=0x55d937381850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x55d93737f3c0 0x55d937381850 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x55d93732a2e0 msgr2=0x55d93732c6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x55d93732a2e0 0x55d93732c6d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f389c03a620 msgr2=0x7f389c03cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f389c03a620 0x7f389c03cae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009710 tx=0x7f38b0005a50 comp rx=0 tx=0).stop 2024-08-19T10:38:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 10 monclient: shutdown 2024-08-19T10:38:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcc62640 1 -- 172.21.15.89:0/3962507936 reap_dead start 2024-08-19T10:38:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 msgr2=0x55d93730e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 0x55d93730e580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f38a400bef0 tx=0x7f38a4009650 comp rx=0 tx=0).stop 2024-08-19T10:38:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 shutdown_connections 2024-08-19T10:38:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 --2- 172.21.15.89:0/3962507936 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x55d93727c410 0x55d93730e580 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 >> 172.21.15.89:0/3962507936 conn(0x55d93725ddf0 msgr2=0x55d93727e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 shutdown_connections 2024-08-19T10:38:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.842+0000 7f38bcef6100 1 -- 172.21.15.89:0/3962507936 wait complete. 2024-08-19T10:38:14.853 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-08-19T10:38:14.853 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-08-19T10:38:14.854 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-08-19T10:38:14.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient: get_monmap_and_config 2024-08-19T10:38:14.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient: build_initial_monmap 2024-08-19T10:38:14.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient: monmap: 2024-08-19T10:38:14.984 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:14.984 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:14.984 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:14.985445+0000 2024-08-19T10:38:14.984 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:14.985445+0000 2024-08-19T10:38:14.984 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:14.984 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:14.985 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:14.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 1 Processor -- start 2024-08-19T10:38:14.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 1 -- start start 2024-08-19T10:38:14.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient: init 2024-08-19T10:38:14.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:14.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient: _add_conns ranks=[7,4,0,1,3,10,6,2,8,5,9] 2024-08-19T10:38:14.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8180a5270 0x7fc8180a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:14.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient(hunting): picked mon.noname-i con 0x7fc8180a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:14.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc8180a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient(hunting): picked mon.noname-c con 0x7fc8180a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc8181395b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8180a8710 con 0x7fc8181395b0 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc8180a7650 con 0x7fc8180a6d30 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc8180a5a40 con 0x7fc8180a5270 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81f3ca640 10 monclient(hunting): authenticate will time out at 944.801999s 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc8180a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc8180a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37696/0 (socket says 172.21.15.89:37696) 2024-08-19T10:38:14.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 1 -- 172.21.15.89:0/1185671653 learned_addr learned my addr 172.21.15.89:0/1185671653 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 10 monclient(hunting): get_auth_request con 0x7fc8180a6d30 auth_method 0 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 10 monclient(hunting): handle_auth_done global_id 4225 payload 274 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 1 -- 172.21.15.89:0/1185671653 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8180a5270 msgr2=0x7fc8180a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 1 --2- 172.21.15.89:0/1185671653 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8180a5270 0x7fc8180a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 1 -- 172.21.15.89:0/1185671653 >> v1:172.21.15.89:6789/0 conn(0x7fc8181395b0 legacy=0x7fc818139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 10 monclient: _finish_hunting 0 2024-08-19T10:38:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 1 monclient: found mon.noname-c 2024-08-19T10:38:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.982+0000 7fc81c965640 1 -- 172.21.15.89:0/1185671653 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8180a81d0 con 0x7fc8180a6d30 2024-08-19T10:38:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81c965640 1 -- 172.21.15.89:0/1185671653 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 msgr2=0x7fc8180a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81c965640 1 --2- 172.21.15.89:0/1185671653 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc8180a7110 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc8140098a0 tx=0x7fc81401c260 comp rx=0 tx=0).ready entity=mon.4 client_cookie=325dda8a95f6c27f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 1 -- 172.21.15.89:0/1185671653 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc8140b0070 con 0x7fc8180a6d30 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 10 monclient: dump: 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 10 monclient: _finish_auth 0 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 1 -- 172.21.15.89:0/1185671653 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8140aa810 con 0x7fc8180a6d30 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 5 monclient: authenticate success, global_id 4225 2024-08-19T10:38:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 1 -- 172.21.15.89:0/1185671653 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc8140a93e0 con 0x7fc8180a6d30 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 4 set_mon_vals no callback set 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc80ffff640 10 monclient: dump: 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 10 monclient: shutdown 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 1 -- 172.21.15.89:0/1185671653 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 msgr2=0x7fc8180a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 1 --2- 172.21.15.89:0/1185671653 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc8180a7110 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc8140098a0 tx=0x7fc81401c260 comp rx=0 tx=0).stop 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 1 -- 172.21.15.89:0/1185671653 shutdown_connections 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 1 --2- 172.21.15.89:0/1185671653 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc8180a7110 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 1 --2- 172.21.15.89:0/1185671653 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8180a5270 0x7fc8180a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 1 -- 172.21.15.89:0/1185671653 >> 172.21.15.89:0/1185671653 conn(0x7fc818069a80 msgr2=0x7fc8180abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 1 -- 172.21.15.89:0/1185671653 shutdown_connections 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.986+0000 7fc81f3ca640 1 -- 172.21.15.89:0/1185671653 wait complete. 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient: build_initial_monmap 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient: monmap: 2024-08-19T10:38:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:14.998 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 1 Processor -- start 2024-08-19T10:38:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 1 -- start start 2024-08-19T10:38:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient: init 2024-08-19T10:38:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient: _add_conns ranks=[1,4,2,3,6,7,0,8,5] 2024-08-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8180a5270 0x7fc8181d1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient(hunting): picked mon.noname-f con 0x7fc8180a5270 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc818138650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient(hunting): picked mon.noname-c con 0x7fc8180a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8181395b0 0x7fc81813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient(hunting): picked mon.noname-b con 0x7fc8181395b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc8180a57d0 con 0x7fc8180a5270 2024-08-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc8180a5650 con 0x7fc8181395b0 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc818144240 con 0x7fc8180a6d30 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81f3ca640 10 monclient(hunting): authenticate will time out at 944.809015s 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc818138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc818138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37706/0 (socket says 172.21.15.89:37706) 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 1 -- 172.21.15.89:0/681524252 learned_addr learned my addr 172.21.15.89:0/681524252 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 10 monclient(hunting): get_auth_request con 0x7fc8180a6d30 auth_method 0 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d967640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8181395b0 0x7fc81813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d967640 10 monclient(hunting): get_auth_request con 0x7fc8181395b0 auth_method 0 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d967640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d967640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d967640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d967640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d967640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d967640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d166640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8180a5270 0x7fc8181d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d166640 10 monclient(hunting): get_auth_request con 0x7fc8180a5270 auth_method 0 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d166640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d166640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d166640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 10 monclient(hunting): handle_auth_done global_id 4234 payload 824 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 1 -- 172.21.15.89:0/681524252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8181395b0 msgr2=0x7fc81813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8181395b0 0x7fc81813d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 1 -- 172.21.15.89:0/681524252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8180a5270 msgr2=0x7fc8181d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8180a5270 0x7fc8181d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 10 monclient: _finish_hunting 0 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 1 monclient: found mon.noname-c 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 1 -- 172.21.15.89:0/681524252 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc818140ad0 con 0x7fc8180a6d30 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81d967640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8181395b0 0x7fc81813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 1 -- 172.21.15.89:0/681524252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 msgr2=0x7fc818138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc81c965640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc818138650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc814000c10 tx=0x7fc814019000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc80dffb640 1 -- 172.21.15.89:0/681524252 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc8140b9070 con 0x7fc8180a6d30 2024-08-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc80dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc80dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.990+0000 7fc80dffb640 10 monclient: dump: 2024-08-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:15.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 1 -- 172.21.15.89:0/681524252 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc814004250 con 0x7fc8180a6d30 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81f3ca640 5 monclient: authenticate success, global_id 4234 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81f3ca640 10 monclient: _renew_subs 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81f3ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81f3ca640 1 -- 172.21.15.89:0/681524252 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc818122fe0 con 0x7fc8180a6d30 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 1 -- 172.21.15.89:0/681524252 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc8140b0040 con 0x7fc8180a6d30 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 10 monclient: dump: 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:15.008 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81f3ca640 10 monclient: _renew_subs 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81f3ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80effd640 4 set_mon_vals no callback set 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81f3ca640 1 -- 172.21.15.89:0/681524252 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc818140070 con 0x7fc8180a6d30 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81f3ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81f3ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81f3ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81f3ca640 1 -- 172.21.15.89:0/681524252 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8180a27a0 con 0x7fc8180a6d30 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 1 -- 172.21.15.89:0/681524252 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc8140bc060 con 0x7fc8180a6d30 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc7fc03be00 0x7fc7fc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc80dffb640 1 -- 172.21.15.89:0/681524252 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7fc8140e7cc0 con 0x7fc8180a6d30 2024-08-19T10:38:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81d166640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc7fc03be00 0x7fc7fc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.994+0000 7fc81d166640 10 monclient: get_auth_request con 0x7fc7fc03be00 auth_method 0 2024-08-19T10:38:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.998+0000 7fc81d166640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc7fc03be00 0x7fc7fc03e2c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc808006fd0 tx=0x7fc808008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.998+0000 7fc80dffb640 1 -- 172.21.15.89:0/681524252 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc8180a27a0 con 0x7fc8180a6d30 2024-08-19T10:38:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.998+0000 7fc80dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:14.998+0000 7fc80dffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.166+0000 7fc81f3ca640 10 start_mon_command cmd=[{"prefix": "config log", "num": 1, "format": "json"}] 2024-08-19T10:38:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.166+0000 7fc81f3ca640 10 monclient: _send_command 2 [{"prefix": "config log", "num": 1, "format": "json"}] 2024-08-19T10:38:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.166+0000 7fc81f3ca640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.166+0000 7fc81f3ca640 1 -- 172.21.15.89:0/681524252 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fc8180a67f0 con 0x7fc8180a6d30 2024-08-19T10:38:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.166+0000 7fc80dffb640 1 -- 172.21.15.89:0/681524252 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v5) ==== 86+0+159 (secure 0 0 0) 0x7fc8180a8710 con 0x7fc8180a6d30 2024-08-19T10:38:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.166+0000 7fc80dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config log", "num": 1, "format": "json"}] 2024-08-19T10:38:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.166+0000 7fc80dffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:15.171 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:38:15.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 -- 172.21.15.89:0/681524252 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc7fc03be00 msgr2=0x7fc7fc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc7fc03be00 0x7fc7fc03e2c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc808006fd0 tx=0x7fc808008040 comp rx=0 tx=0).stop 2024-08-19T10:38:15.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 10 monclient: shutdown 2024-08-19T10:38:15.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:15.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 -- 172.21.15.89:0/681524252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 msgr2=0x7fc818138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc818138650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc814000c10 tx=0x7fc814019000 comp rx=0 tx=0).stop 2024-08-19T10:38:15.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 -- 172.21.15.89:0/681524252 shutdown_connections 2024-08-19T10:38:15.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc7fc03be00 0x7fc7fc03e2c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8181395b0 0x7fc81813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8180a6d30 0x7fc818138650 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 --2- 172.21.15.89:0/681524252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc8180a5270 0x7fc8181d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 -- 172.21.15.89:0/681524252 >> 172.21.15.89:0/681524252 conn(0x7fc818069a80 msgr2=0x7fc8180ab210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:15.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 -- 172.21.15.89:0/681524252 shutdown_connections 2024-08-19T10:38:15.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.170+0000 7fc81f3ca640 1 -- 172.21.15.89:0/681524252 wait complete. 2024-08-19T10:38:15.189 INFO:teuthology.orchestra.run.smithi089.stdout:[{"version":5,"timestamp":"2024-08-19T10:38:04.495507+0000","name":"","changes":[{"name":"osd.4/osd_mclock_max_capacity_iops_ssd","new_value":"8942.696311"}]}] 2024-08-19T10:38:15.189 INFO:tasks.ceph_manager:config epoch is 5 2024-08-19T10:38:15.189 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-19T10:38:15.190 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-08-19T10:38:15.190 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-08-19T10:38:15.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.302+0000 7ff21f2c1640 10 monclient: get_monmap_and_config 2024-08-19T10:38:15.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.302+0000 7ff21f2c1640 10 monclient: build_initial_monmap 2024-08-19T10:38:15.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.302+0000 7ff21f2c1640 10 monclient: monmap: 2024-08-19T10:38:15.307 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:15.307 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.307 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:15.308720+0000 2024-08-19T10:38:15.307 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:15.308720+0000 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:15.308 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:15.309 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 1 Processor -- start 2024-08-19T10:38:15.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 1 -- start start 2024-08-19T10:38:15.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 10 monclient: init 2024-08-19T10:38:15.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:15.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 10 monclient: _add_conns ranks=[9,10,8,1,2,4,7,5,0,6,3] 2024-08-19T10:38:15.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff2180a5270 0x7ff2180a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 10 monclient(hunting): picked mon.noname-a con 0x7ff2180a5270 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2180a6d30 0x7ff2180a7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 10 monclient(hunting): picked mon.noname-f con 0x7ff2180a6d30 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2181395b0 0x7ff218139a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 10 monclient(hunting): picked mon.noname-e con 0x7ff2181395b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff2180a7680 con 0x7ff2180a5270 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff2180a8710 con 0x7ff2180a6d30 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff2180a5a40 con 0x7ff2181395b0 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21f2c1640 10 monclient(hunting): authenticate will time out at 945.125261s 2024-08-19T10:38:15.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2181395b0 0x7ff218139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d05d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff2180a5270 0x7ff2180a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2181395b0 0x7ff218139a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47694/0 (socket says 172.21.15.89:47694) 2024-08-19T10:38:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 1 -- 172.21.15.89:0/3481305091 learned_addr learned my addr 172.21.15.89:0/3481305091 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 10 monclient(hunting): get_auth_request con 0x7ff2181395b0 auth_method 0 2024-08-19T10:38:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d05d640 10 monclient(hunting): get_auth_request con 0x7ff2180a5270 auth_method 0 2024-08-19T10:38:15.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d05d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d05d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d05d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d05d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d05d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d05d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21c85c640 1 --2- 172.21.15.89:0/3481305091 >> v2:172.21.15.190:3300/0 conn(0x7ff2180a6d30 0x7ff2180a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21c85c640 10 monclient(hunting): get_auth_request con 0x7ff2180a6d30 auth_method 0 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21c85c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21c85c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21c85c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 10 monclient(hunting): handle_auth_done global_id 4202 payload 274 2024-08-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 1 -- 172.21.15.89:0/3481305091 >> v2:172.21.15.190:3300/0 conn(0x7ff2180a6d30 msgr2=0x7ff2180a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 1 --2- 172.21.15.89:0/3481305091 >> v2:172.21.15.190:3300/0 conn(0x7ff2180a6d30 0x7ff2180a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 1 -- 172.21.15.89:0/3481305091 >> v2:172.21.15.89:3300/0 conn(0x7ff2180a5270 msgr2=0x7ff2180a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 1 --2- 172.21.15.89:0/3481305091 >> v2:172.21.15.89:3300/0 conn(0x7ff2180a5270 0x7ff2180a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 10 monclient: _finish_hunting 0 2024-08-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 1 monclient: found mon.noname-e 2024-08-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d85e640 1 -- 172.21.15.89:0/3481305091 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2180a81d0 con 0x7ff2181395b0 2024-08-19T10:38:15.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.306+0000 7ff21d05d640 1 --2- 172.21.15.89:0/3481305091 >> v2:172.21.15.89:3300/0 conn(0x7ff2180a5270 0x7ff2180a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21d85e640 1 -- 172.21.15.89:0/3481305091 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2181395b0 msgr2=0x7ff218139a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21d85e640 1 --2- 172.21.15.89:0/3481305091 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2181395b0 0x7ff218139a10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff2180acc30 tx=0x7ff20801c3e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f4c9ffa22037980e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 1 -- 172.21.15.89:0/3481305091 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2080b1070 con 0x7ff2181395b0 2024-08-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 10 monclient: dump: 2024-08-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:15.324 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:15.325 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 10 monclient: _finish_auth 0 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 1 -- 172.21.15.89:0/3481305091 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2080ab810 con 0x7ff2181395b0 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 1 -- 172.21.15.89:0/3481305091 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2080abe30 con 0x7ff2181395b0 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 5 monclient: authenticate success, global_id 4202 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 4 set_mon_vals no callback set 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff217fff640 10 monclient: dump: 2024-08-19T10:38:15.326 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:15.327 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 10 monclient: shutdown 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3481305091 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2181395b0 msgr2=0x7ff218139a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 1 --2- 172.21.15.89:0/3481305091 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2181395b0 0x7ff218139a10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff2180acc30 tx=0x7ff20801c3e0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3481305091 shutdown_connections 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 1 --2- 172.21.15.89:0/3481305091 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2181395b0 0x7ff218139a10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 1 --2- 172.21.15.89:0/3481305091 >> v2:172.21.15.190:3300/0 conn(0x7ff2180a6d30 0x7ff2180a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 1 --2- 172.21.15.89:0/3481305091 >> v2:172.21.15.89:3300/0 conn(0x7ff2180a5270 0x7ff2180a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3481305091 >> 172.21.15.89:0/3481305091 conn(0x7ff218069a80 msgr2=0x7ff2180abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3481305091 shutdown_connections 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3481305091 wait complete. 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 10 monclient: build_initial_monmap 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.310+0000 7ff21f2c1640 10 monclient: monmap: 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:15.329 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 1 Processor -- start 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 1 -- start start 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient: init 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient: _add_conns ranks=[5,6,2,7,0,3,1,4,8] 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff2180a5270 0x7ff218134820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient(hunting): picked mon.noname-h con 0x7ff2180a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2180a6d30 0x7ff218134d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient(hunting): picked mon.noname-d con 0x7ff2180a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2181395b0 0x7ff21809cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient(hunting): picked mon.noname-b con 0x7ff2181395b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff2180a5650 con 0x7ff2181395b0 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff21813a360 con 0x7ff2180a5270 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff2180a57d0 con 0x7ff2180a6d30 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient(hunting): authenticate will time out at 945.132632s 2024-08-19T10:38:15.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21c85c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2180a6d30 0x7ff218134d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2181395b0 0x7ff21809cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2181395b0 0x7ff21809cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44338/0 (socket says 172.21.15.89:44338) 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21c85c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2180a6d30 0x7ff218134d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46416/0 (socket says 172.21.15.89:46416) 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 1 -- 172.21.15.89:0/3602293230 learned_addr learned my addr 172.21.15.89:0/3602293230 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 10 monclient(hunting): get_auth_request con 0x7ff2181395b0 auth_method 0 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21c85c640 10 monclient(hunting): get_auth_request con 0x7ff2180a6d30 auth_method 0 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21c85c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21c85c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21c85c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21c85c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21c85c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21c85c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d05d640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff2180a5270 0x7ff218134820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 10 monclient(hunting): handle_auth_done global_id 4178 payload 824 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 1 -- 172.21.15.89:0/3602293230 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2180a6d30 msgr2=0x7ff218134d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2180a6d30 0x7ff218134d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 1 -- 172.21.15.89:0/3602293230 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff2180a5270 msgr2=0x7ff218134820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff2180a5270 0x7ff218134820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 10 monclient: _finish_hunting 0 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 1 monclient: found mon.noname-b 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 1 -- 172.21.15.89:0/3602293230 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2180a0160 con 0x7ff2181395b0 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d05d640 10 monclient: get_auth_request con 0x7ff2180a5270 auth_method 0 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d05d640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff2180a5270 0x7ff218134820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 1 -- 172.21.15.89:0/3602293230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2181395b0 msgr2=0x7ff21809cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21d85e640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2181395b0 0x7ff21809cc00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff2080097c0 tx=0x7ff208019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 1 -- 172.21.15.89:0/3602293230 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2080ba070 con 0x7ff2181395b0 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 10 monclient: dump: 2024-08-19T10:38:15.334 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:15.335 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 1 -- 172.21.15.89:0/3602293230 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2080abe00 con 0x7ff2181395b0 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 1 -- 172.21.15.89:0/3602293230 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2080b1040 con 0x7ff2181395b0 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff215ffb640 10 monclient: dump: 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.336 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:15.337 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff2177fe640 4 set_mon_vals no callback set 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 5 monclient: authenticate success, global_id 4178 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient: _renew_subs 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3602293230 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2180a0440 con 0x7ff2181395b0 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient: _renew_subs 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.314+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3602293230 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21809ef20 con 0x7ff2181395b0 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.318+0000 7ff21f2c1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.318+0000 7ff21f2c1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.318+0000 7ff21f2c1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.318+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3602293230 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2180a7140 con 0x7ff2181395b0 2024-08-19T10:38:15.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.318+0000 7ff215ffb640 1 -- 172.21.15.89:0/3602293230 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff208004250 con 0x7ff2181395b0 2024-08-19T10:38:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.318+0000 7ff215ffb640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff20403be00 0x7ff20403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.318+0000 7ff215ffb640 1 -- 172.21.15.89:0/3602293230 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7ff2080e7c60 con 0x7ff2181395b0 2024-08-19T10:38:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.318+0000 7ff21d05d640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff20403be00 0x7ff20403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.318+0000 7ff21d05d640 10 monclient: get_auth_request con 0x7ff20403be00 auth_method 0 2024-08-19T10:38:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.322+0000 7ff21d05d640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff20403be00 0x7ff20403e2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff2100099c0 tx=0x7ff210008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.322+0000 7ff215ffb640 1 -- 172.21.15.89:0/3602293230 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff2080160f0 con 0x7ff2181395b0 2024-08-19T10:38:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.322+0000 7ff215ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:15.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.322+0000 7ff215ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:15.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.530+0000 7ff21f2c1640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2024-08-19T10:38:15.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.530+0000 7ff21f2c1640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2024-08-19T10:38:15.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.530+0000 7ff21f2c1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:15.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.530+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3602293230 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff218064590 con 0x7ff2181395b0 2024-08-19T10:38:15.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.534+0000 7ff215ffb640 1 -- 172.21.15.89:0/3602293230 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+92159 (secure 0 0 0) 0x7ff2080bcd20 con 0x7ff2181395b0 2024-08-19T10:38:15.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff215ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2024-08-19T10:38:15.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff215ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:15.540 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:38:15.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3602293230 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff20403be00 msgr2=0x7ff20403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff20403be00 0x7ff20403e2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff2100099c0 tx=0x7ff210008040 comp rx=0 tx=0).stop 2024-08-19T10:38:15.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 10 monclient: shutdown 2024-08-19T10:38:15.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:15.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3602293230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2181395b0 msgr2=0x7ff21809cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2181395b0 0x7ff21809cc00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff2080097c0 tx=0x7ff208019180 comp rx=0 tx=0).stop 2024-08-19T10:38:15.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3602293230 shutdown_connections 2024-08-19T10:38:15.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff20403be00 0x7ff20403e2c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2181395b0 0x7ff21809cc00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2180a6d30 0x7ff218134d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 --2- 172.21.15.89:0/3602293230 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff2180a5270 0x7ff218134820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3602293230 >> 172.21.15.89:0/3602293230 conn(0x7ff218069a80 msgr2=0x7ff2180ab2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:15.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3602293230 shutdown_connections 2024-08-19T10:38:15.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.538+0000 7ff21f2c1640 1 -- 172.21.15.89:0/3602293230 wait complete. 2024-08-19T10:38:15.558 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":3725341825},{"type":"v1","addr":"172.21.15.190:6801","nonce":3725341825}]},"active_addr":"172.21.15.190:6801/3725341825","active_change":"2024-08-19T10:37:56.426749+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":3668141574}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":4123525956}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":809273624}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":3945166349}]}]} 2024-08-19T10:38:15.560 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-08-19T10:38:15.560 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-08-19T10:38:15.560 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T10:38:15.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.670+0000 7ff220c57640 10 monclient: get_monmap_and_config 2024-08-19T10:38:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.670+0000 7ff220c57640 10 monclient: build_initial_monmap 2024-08-19T10:38:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.670+0000 7ff220c57640 10 monclient: monmap: 2024-08-19T10:38:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:15.676355+0000 2024-08-19T10:38:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:15.676355+0000 2024-08-19T10:38:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:15.675 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:15.676 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:15.676 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:15.676 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:15.676 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:15.676 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:15.676 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:15.676 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:15.676 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:15.676 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 1 Processor -- start 2024-08-19T10:38:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 1 -- start start 2024-08-19T10:38:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 10 monclient: init 2024-08-19T10:38:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 10 monclient: _add_conns ranks=[10,6,0,8,7,1,2,3,5,9,4] 2024-08-19T10:38:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff21c0a5270 0x7ff21c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 10 monclient(hunting): picked mon.noname-f con 0x7ff21c0a5270 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:15.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff21c0a6d30 0x7ff21c0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 10 monclient(hunting): picked mon.noname-d con 0x7ff21c0a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff21c1395b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21c0a8710 con 0x7ff21c1395b0 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff21c0a7650 con 0x7ff21c0a5270 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff21c0a5a40 con 0x7ff21c0a6d30 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff220c57640 10 monclient(hunting): authenticate will time out at 945.492917s 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff21ad9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff21c1395b0 0x7ff21c139a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54150/0 (socket says 172.21.15.89:54150) 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff21c0a6d30 0x7ff21c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff21ad9d640 1 -- 172.21.15.89:0/4282723223 learned_addr learned my addr 172.21.15.89:0/4282723223 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:15.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 10 monclient(hunting): get_auth_request con 0x7ff21c0a6d30 auth_method 0 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff21a59c640 1 --2- 172.21.15.89:0/4282723223 >> v2:172.21.15.190:3300/0 conn(0x7ff21c0a5270 0x7ff21c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff21a59c640 10 monclient(hunting): get_auth_request con 0x7ff21c0a5270 auth_method 0 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff21a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff21a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff21a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 10 monclient(hunting): handle_auth_done global_id 4209 payload 274 2024-08-19T10:38:15.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 1 -- 172.21.15.89:0/4282723223 >> v2:172.21.15.190:3300/0 conn(0x7ff21c0a5270 msgr2=0x7ff21c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 1 --2- 172.21.15.89:0/4282723223 >> v2:172.21.15.190:3300/0 conn(0x7ff21c0a5270 0x7ff21c0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 1 -- 172.21.15.89:0/4282723223 >> v1:172.21.15.89:6789/0 conn(0x7ff21c1395b0 legacy=0x7ff21c139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 1 monclient: found mon.noname-d 2024-08-19T10:38:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:15.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.674+0000 7ff213fff640 1 -- 172.21.15.89:0/4282723223 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c0a81d0 con 0x7ff21c0a6d30 2024-08-19T10:38:15.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff213fff640 1 -- 172.21.15.89:0/4282723223 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff21c0a6d30 msgr2=0x7ff21c0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff213fff640 1 --2- 172.21.15.89:0/4282723223 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff21c0a6d30 0x7ff21c0a7110 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2040097c0 tx=0x7ff20401c180 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f72276c192bf2811 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 1 -- 172.21.15.89:0/4282723223 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2040b0070 con 0x7ff21c0a6d30 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 10 monclient: dump: 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.685 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 10 monclient: _finish_auth 0 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:15.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 1 -- 172.21.15.89:0/4282723223 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2040aa810 con 0x7ff21c0a6d30 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 5 monclient: authenticate success, global_id 4209 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 1 -- 172.21.15.89:0/4282723223 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2040a9300 con 0x7ff21c0a6d30 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 4 set_mon_vals no callback set 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff219d9b640 10 monclient: dump: 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.687 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 10 monclient: shutdown 2024-08-19T10:38:15.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 1 -- 172.21.15.89:0/4282723223 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff21c0a6d30 msgr2=0x7ff21c0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 1 --2- 172.21.15.89:0/4282723223 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff21c0a6d30 0x7ff21c0a7110 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2040097c0 tx=0x7ff20401c180 comp rx=0 tx=0).stop 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 1 -- 172.21.15.89:0/4282723223 shutdown_connections 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 1 --2- 172.21.15.89:0/4282723223 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff21c0a6d30 0x7ff21c0a7110 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 1 --2- 172.21.15.89:0/4282723223 >> v2:172.21.15.190:3300/0 conn(0x7ff21c0a5270 0x7ff21c0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 1 -- 172.21.15.89:0/4282723223 >> 172.21.15.89:0/4282723223 conn(0x7ff21c069a80 msgr2=0x7ff21c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 1 -- 172.21.15.89:0/4282723223 shutdown_connections 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 1 -- 172.21.15.89:0/4282723223 wait complete. 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 10 monclient: build_initial_monmap 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 10 monclient: monmap: 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:15.689 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:15.690 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 1 Processor -- start 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 1 -- start start 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.678+0000 7ff220c57640 10 monclient: init 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient: _add_conns ranks=[0,1,8,5,4,6,7,2,3] 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff21c0a5270 0x7ff21c1383e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient(hunting): picked mon.noname-a con 0x7ff21c0a5270 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff21c0a6d30 0x7ff21c13d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient(hunting): picked mon.noname-f con 0x7ff21c0a6d30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff21c1395b0 0x7ff21c13dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient(hunting): picked mon.noname-e con 0x7ff21c1395b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c0a57d0 con 0x7ff21c0a5270 2024-08-19T10:38:15.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c0a5650 con 0x7ff21c0a6d30 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff21c144240 con 0x7ff21c1395b0 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient(hunting): authenticate will time out at 945.500134s 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff21c0a5270 0x7ff21c1383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21ad9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff21c1395b0 0x7ff21c13dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff21c0a5270 0x7ff21c1383e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57676/0 (socket says 172.21.15.89:57676) 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 1 -- 172.21.15.89:0/1642062310 learned_addr learned my addr 172.21.15.89:0/1642062310 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 10 monclient(hunting): get_auth_request con 0x7ff21c0a5270 auth_method 0 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21ad9d640 10 monclient(hunting): get_auth_request con 0x7ff21c1395b0 auth_method 0 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff213fff640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff21c0a6d30 0x7ff21c13d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 10 monclient(hunting): handle_auth_done global_id 4293 payload 824 2024-08-19T10:38:15.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 1 -- 172.21.15.89:0/1642062310 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff21c1395b0 msgr2=0x7ff21c13dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff21c1395b0 0x7ff21c13dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 1 -- 172.21.15.89:0/1642062310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff21c0a6d30 msgr2=0x7ff21c13d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff21c0a6d30 0x7ff21c13d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:38:15.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 1 monclient: found mon.noname-a 2024-08-19T10:38:15.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:15.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:15.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 1 -- 172.21.15.89:0/1642062310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c122be0 con 0x7ff21c0a5270 2024-08-19T10:38:15.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff213fff640 10 monclient: get_auth_request con 0x7ff21c0a6d30 auth_method 0 2024-08-19T10:38:15.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff213fff640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff21c0a6d30 0x7ff21c13d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:15.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21ad9d640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff21c1395b0 0x7ff21c13dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:15.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 1 -- 172.21.15.89:0/1642062310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff21c0a5270 msgr2=0x7ff21c1383e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21a59c640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff21c0a5270 0x7ff21c1383e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00d5f0 tx=0x7ff20c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 1 -- 172.21.15.89:0/1642062310 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff20c007380 con 0x7ff21c0a5270 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 10 monclient: dump: 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.697 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 10 monclient: _finish_auth 0 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:15.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 5 monclient: authenticate success, global_id 4293 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient: _renew_subs 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 1 -- 172.21.15.89:0/1642062310 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff20c019070 con 0x7ff21c0a5270 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 1 -- 172.21.15.89:0/1642062310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21c122ec0 con 0x7ff21c0a5270 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 1 -- 172.21.15.89:0/1642062310 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff20c012c70 con 0x7ff21c0a5270 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff21959a640 4 set_mon_vals no callback set 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff212ffd640 10 monclient: dump: 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.699 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:15.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient: _renew_subs 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.682+0000 7ff220c57640 1 -- 172.21.15.89:0/1642062310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21c0a2510 con 0x7ff21c0a5270 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.686+0000 7ff212ffd640 1 -- 172.21.15.89:0/1642062310 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff20c012800 con 0x7ff21c0a5270 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.686+0000 7ff220c57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.686+0000 7ff220c57640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.686+0000 7ff220c57640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.686+0000 7ff220c57640 1 -- 172.21.15.89:0/1642062310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21c0aad90 con 0x7ff21c0a5270 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.686+0000 7ff212ffd640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff1fc03be00 0x7ff1fc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.686+0000 7ff212ffd640 1 -- 172.21.15.89:0/1642062310 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7ff20c00f040 con 0x7ff21c0a5270 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.686+0000 7ff213fff640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff1fc03be00 0x7ff1fc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.686+0000 7ff213fff640 10 monclient: get_auth_request con 0x7ff1fc03be00 auth_method 0 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.686+0000 7ff213fff640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff1fc03be00 0x7ff1fc03e2c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0a9960 tx=0x7ff2040a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:15.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.690+0000 7ff212ffd640 1 -- 172.21.15.89:0/1642062310 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff20c023020 con 0x7ff21c0a5270 2024-08-19T10:38:15.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.690+0000 7ff212ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:15.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.690+0000 7ff212ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.862+0000 7ff220c57640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.862+0000 7ff220c57640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.862+0000 7ff220c57640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.862+0000 7ff220c57640 1 -- 172.21.15.89:0/1642062310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff21c0a67f0 con 0x7ff21c0a5270 2024-08-19T10:38:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.862+0000 7ff212ffd640 1 -- 172.21.15.89:0/1642062310 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12403 (secure 0 0 0) 0x7ff20c012e10 con 0x7ff21c0a5270 2024-08-19T10:38:15.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.862+0000 7ff212ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:15.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.862+0000 7ff212ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:15.866 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:38:15.866 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","created":"2024-08-19T10:37:50.906618+0000","modified":"2024-08-19T10:38:14.819112+0000","last_up_change":"2024-08-19T10:38:04.611102+0000","last_in_change":"2024-08-19T10:37:55.695518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T10:38:04.494964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-19T10:38:11.600504+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2be543b2-eb94-4907-8f21-6e91b7dd9733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6817","nonce":2476057681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6819","nonce":2476057681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6823","nonce":2476057681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6821","nonce":2476057681}]},"public_addr":"172.21.15.89:6817/2476057681","cluster_addr":"172.21.15.89:6819/2476057681","heartbeat_back_addr":"172.21.15.89:6823/2476057681","heartbeat_front_addr":"172.21.15.89:6821/2476057681","state":["exists","up"]},{"osd":1,"uuid":"224c1713-25c8-4a48-b16e-8929fcb6424b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6801","nonce":375502586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6803","nonce":375502586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6807","nonce":375502586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6805","nonce":375502586}]},"public_addr":"172.21.15.89:6801/375502586","cluster_addr":"172.21.15.89:6803/375502586","heartbeat_back_addr":"172.21.15.89:6807/375502586","heartbeat_front_addr":"172.21.15.89:6805/375502586","state":["exists","up"]},{"osd":2,"uuid":"5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6809","nonce":1060450292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6811","nonce":1060450292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6815","nonce":1060450292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6813","nonce":1060450292}]},"public_addr":"172.21.15.89:6809/1060450292","cluster_addr":"172.21.15.89:6811/1060450292","heartbeat_back_addr":"172.21.15.89:6815/1060450292","heartbeat_front_addr":"172.21.15.89:6813/1060450292","state":["exists","up"]},{"osd":3,"uuid":"2dcaafe0-5837-410b-ae99-c7561ef6b150","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6811","nonce":2751619171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6813","nonce":2751619171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6817","nonce":2751619171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6815","nonce":2751619171}]},"public_addr":"172.21.15.190:6811/2751619171","cluster_addr":"172.21.15.190:6813/2751619171","heartbeat_back_addr":"172.21.15.190:6817/2751619171","heartbeat_front_addr":"172.21.15.190:6815/2751619171","state":["exists","up"]},{"osd":4,"uuid":"5619c403-27fd-48de-8cee-c5e84289cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6819","nonce":1174298329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6821","nonce":1174298329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6825","nonce":1174298329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6823","nonce":1174298329}]},"public_addr":"172.21.15.190:6819/1174298329","cluster_addr":"172.21.15.190:6821/1174298329","heartbeat_back_addr":"172.21.15.190:6825/1174298329","heartbeat_front_addr":"172.21.15.190:6823/1174298329","state":["exists","up"]},{"osd":5,"uuid":"82ab73a3-1212-4b0d-ab05-aa9c01dd5586","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6803","nonce":2434772607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6805","nonce":2434772607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6809","nonce":2434772607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6807","nonce":2434772607}]},"public_addr":"172.21.15.190:6803/2434772607","cluster_addr":"172.21.15.190:6805/2434772607","heartbeat_back_addr":"172.21.15.190:6809/2434772607","heartbeat_front_addr":"172.21.15.190:6807/2434772607","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.511833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:01.832703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.351820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.269178+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.511490+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.178520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T10:38:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 -- 172.21.15.89:0/1642062310 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff1fc03be00 msgr2=0x7ff1fc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff1fc03be00 0x7ff1fc03e2c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0a9960 tx=0x7ff2040a6000 comp rx=0 tx=0).stop 2024-08-19T10:38:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 10 monclient: shutdown 2024-08-19T10:38:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 -- 172.21.15.89:0/1642062310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff21c0a5270 msgr2=0x7ff21c1383e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff21c0a5270 0x7ff21c1383e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00d5f0 tx=0x7ff20c00dab0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 -- 172.21.15.89:0/1642062310 shutdown_connections 2024-08-19T10:38:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff1fc03be00 0x7ff1fc03e2c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff21c1395b0 0x7ff21c13dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff21c0a6d30 0x7ff21c13d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 --2- 172.21.15.89:0/1642062310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff21c0a5270 0x7ff21c1383e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:15.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 -- 172.21.15.89:0/1642062310 >> 172.21.15.89:0/1642062310 conn(0x7ff21c069a80 msgr2=0x7ff21c0ab1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:15.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 -- 172.21.15.89:0/1642062310 shutdown_connections 2024-08-19T10:38:15.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:15.866+0000 7ff220c57640 1 -- 172.21.15.89:0/1642062310 wait complete. 2024-08-19T10:38:15.887 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-08-19T10:38:15.887 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T10:38:16.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient: monmap: 2024-08-19T10:38:16.005 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.005 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.006868+0000 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.006868+0000 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.006 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.007 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.007 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 1 Processor -- start 2024-08-19T10:38:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 1 -- start start 2024-08-19T10:38:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient: init 2024-08-19T10:38:16.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient: _add_conns ranks=[5,3,1,4,7,0,8,2,6,10,9] 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe83806e050 0x7fe838139480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient(hunting): picked mon.noname-h con 0x7fe83806e050 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8381399c0 0x7fe838139da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient(hunting): picked mon.noname-g con 0x7fe8381399c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe8380a56b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe83806f3d0 con 0x7fe8380a56b0 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe83806e430 con 0x7fe8381399c0 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe838072320 con 0x7fe83806e050 2024-08-19T10:38:16.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83f125640 10 monclient(hunting): authenticate will time out at 945.823417s 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe837fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8381399c0 0x7fe838139da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83d6c2640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe8380a56b0 0x7fe8380a5ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.89:45886/0 (socket says 172.21.15.89:45886) 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83cec1640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe83806e050 0x7fe838139480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83d6c2640 1 -- 172.21.15.89:0/2785364821 learned_addr learned my addr 172.21.15.89:0/2785364821 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe837fff640 10 monclient(hunting): get_auth_request con 0x7fe8381399c0 auth_method 0 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe837fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe837fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe837fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83cec1640 10 monclient(hunting): get_auth_request con 0x7fe83806e050 auth_method 0 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83cec1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83cec1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.002+0000 7fe83cec1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe83cec1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe83cec1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe83cec1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 10 monclient(hunting): handle_auth_done global_id 4242 payload 274 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 1 -- 172.21.15.89:0/2785364821 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe83806e050 msgr2=0x7fe838139480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 1 --2- 172.21.15.89:0/2785364821 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe83806e050 0x7fe838139480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 1 -- 172.21.15.89:0/2785364821 >> v1:172.21.15.190:6789/0 conn(0x7fe8380a56b0 legacy=0x7fe8380a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 1 monclient: found mon.noname-g 2024-08-19T10:38:16.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:38:16.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.006+0000 7fe837fff640 1 -- 172.21.15.89:0/2785364821 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83806f0f0 con 0x7fe8381399c0 2024-08-19T10:38:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe837fff640 1 -- 172.21.15.89:0/2785364821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8381399c0 msgr2=0x7fe838139da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe837fff640 1 --2- 172.21.15.89:0/2785364821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8381399c0 0x7fe838139da0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe8280097f0 tx=0x7fe82801c1c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c131798a2982168c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 1 -- 172.21.15.89:0/2785364821 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8280b0070 con 0x7fe8381399c0 2024-08-19T10:38:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:38:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 10 monclient: dump: 2024-08-19T10:38:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.017 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 1 -- 172.21.15.89:0/2785364821 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8280aa810 con 0x7fe8381399c0 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 5 monclient: authenticate success, global_id 4242 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 1 -- 172.21.15.89:0/2785364821 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe8280a9300 con 0x7fe8381399c0 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 4 set_mon_vals no callback set 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe8377fe640 10 monclient: dump: 2024-08-19T10:38:16.019 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.020 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient: shutdown 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 -- 172.21.15.89:0/2785364821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8381399c0 msgr2=0x7fe838139da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 --2- 172.21.15.89:0/2785364821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8381399c0 0x7fe838139da0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe8280097f0 tx=0x7fe82801c1c0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 -- 172.21.15.89:0/2785364821 shutdown_connections 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 --2- 172.21.15.89:0/2785364821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8381399c0 0x7fe838139da0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 --2- 172.21.15.89:0/2785364821 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe83806e050 0x7fe838139480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 -- 172.21.15.89:0/2785364821 >> 172.21.15.89:0/2785364821 conn(0x7fe8380aa2a0 msgr2=0x7fe8380a67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 -- 172.21.15.89:0/2785364821 shutdown_connections 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 -- 172.21.15.89:0/2785364821 wait complete. 2024-08-19T10:38:16.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient: monmap: 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.022 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 Processor -- start 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 -- start start 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient: init 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient: _add_conns ranks=[5,7,2,3,6,0,1,4,8] 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe83806e050 0x7fe8381202f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient(hunting): picked mon.noname-h con 0x7fe83806e050 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe8380a56b0 0x7fe838120830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient(hunting): picked mon.noname-i con 0x7fe8380a56b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:16.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8381399c0 0x7fe838120d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient(hunting): picked mon.noname-b con 0x7fe8381399c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fe838144290 con 0x7fe8381399c0 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe838144110 con 0x7fe83806e050 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fe838144410 con 0x7fe8380a56b0 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83f125640 10 monclient(hunting): authenticate will time out at 945.831810s 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83d6c2640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8381399c0 0x7fe838120d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.010+0000 7fe83d6c2640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8381399c0 0x7fe838120d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44350/0 (socket says 172.21.15.89:44350) 2024-08-19T10:38:16.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 1 -- 172.21.15.89:0/3582724763 learned_addr learned my addr 172.21.15.89:0/3582724763 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 10 monclient(hunting): get_auth_request con 0x7fe8381399c0 auth_method 0 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83cec1640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe83806e050 0x7fe8381202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe837fff640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe8380a56b0 0x7fe838120830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83cec1640 10 monclient(hunting): get_auth_request con 0x7fe83806e050 auth_method 0 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83cec1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83cec1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83cec1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe837fff640 10 monclient(hunting): get_auth_request con 0x7fe8380a56b0 auth_method 0 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe837fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe837fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe837fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 10 monclient(hunting): handle_auth_done global_id 4187 payload 824 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 1 -- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe8380a56b0 msgr2=0x7fe838120830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe8380a56b0 0x7fe838120830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 1 -- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe83806e050 msgr2=0x7fe8381202f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe83806e050 0x7fe8381202f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 1 monclient: found mon.noname-b 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:16.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 1 -- 172.21.15.89:0/3582724763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8380a7da0 con 0x7fe8381399c0 2024-08-19T10:38:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 1 -- 172.21.15.89:0/3582724763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8381399c0 msgr2=0x7fe838120d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83d6c2640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8381399c0 0x7fe838120d70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe83000cb20 tx=0x7fe83000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 1 -- 172.21.15.89:0/3582724763 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe83000acb0 con 0x7fe8381399c0 2024-08-19T10:38:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 10 monclient: dump: 2024-08-19T10:38:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.029 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.030 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 1 -- 172.21.15.89:0/3582724763 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe830012870 con 0x7fe8381399c0 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83f125640 5 monclient: authenticate success, global_id 4187 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83f125640 10 monclient: _renew_subs 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83f125640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83f125640 1 -- 172.21.15.89:0/3582724763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe838122550 con 0x7fe8381399c0 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 1 -- 172.21.15.89:0/3582724763 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe830007070 con 0x7fe8381399c0 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 10 monclient: dump: 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.032 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83f125640 10 monclient: _renew_subs 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83f125640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe836ffd640 4 set_mon_vals no callback set 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83f125640 1 -- 172.21.15.89:0/3582724763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe838139100 con 0x7fe8381399c0 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83f125640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83f125640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83f125640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:16.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe83f125640 1 -- 172.21.15.89:0/3582724763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe838139480 con 0x7fe8381399c0 2024-08-19T10:38:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.014+0000 7fe8357fa640 1 -- 172.21.15.89:0/3582724763 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe830012400 con 0x7fe8381399c0 2024-08-19T10:38:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.018+0000 7fe8357fa640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe82403be30 0x7fe82403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.018+0000 7fe8357fa640 1 -- 172.21.15.89:0/3582724763 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7fe83004ef70 con 0x7fe8381399c0 2024-08-19T10:38:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.018+0000 7fe83cec1640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe82403be30 0x7fe82403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.018+0000 7fe83cec1640 10 monclient: get_auth_request con 0x7fe82403be30 auth_method 0 2024-08-19T10:38:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.018+0000 7fe83cec1640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe82403be30 0x7fe82403e2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009a50 tx=0x7fe82c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.022+0000 7fe8357fa640 1 -- 172.21.15.89:0/3582724763 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe8380a7da0 con 0x7fe8381399c0 2024-08-19T10:38:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.022+0000 7fe8357fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.022+0000 7fe8357fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:16.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.194+0000 7fe83f125640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:16.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.194+0000 7fe83f125640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:16.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.194+0000 7fe83f125640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:16.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.194+0000 7fe83f125640 1 -- 172.21.15.89:0/3582724763 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe838064590 con 0x7fe8381399c0 2024-08-19T10:38:16.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.194+0000 7fe8357fa640 1 -- 172.21.15.89:0/3582724763 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12403 (secure 0 0 0) 0x7fe83004dae0 con 0x7fe8381399c0 2024-08-19T10:38:16.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.194+0000 7fe8357fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:16.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.194+0000 7fe8357fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:16.197 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:38:16.198 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":18,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","created":"2024-08-19T10:37:50.906618+0000","modified":"2024-08-19T10:38:14.819112+0000","last_up_change":"2024-08-19T10:38:04.611102+0000","last_in_change":"2024-08-19T10:37:55.695518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T10:38:04.494964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-19T10:38:11.600504+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2be543b2-eb94-4907-8f21-6e91b7dd9733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6817","nonce":2476057681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6819","nonce":2476057681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6823","nonce":2476057681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6821","nonce":2476057681}]},"public_addr":"172.21.15.89:6817/2476057681","cluster_addr":"172.21.15.89:6819/2476057681","heartbeat_back_addr":"172.21.15.89:6823/2476057681","heartbeat_front_addr":"172.21.15.89:6821/2476057681","state":["exists","up"]},{"osd":1,"uuid":"224c1713-25c8-4a48-b16e-8929fcb6424b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6801","nonce":375502586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6803","nonce":375502586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6807","nonce":375502586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6805","nonce":375502586}]},"public_addr":"172.21.15.89:6801/375502586","cluster_addr":"172.21.15.89:6803/375502586","heartbeat_back_addr":"172.21.15.89:6807/375502586","heartbeat_front_addr":"172.21.15.89:6805/375502586","state":["exists","up"]},{"osd":2,"uuid":"5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6809","nonce":1060450292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6811","nonce":1060450292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6815","nonce":1060450292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6813","nonce":1060450292}]},"public_addr":"172.21.15.89:6809/1060450292","cluster_addr":"172.21.15.89:6811/1060450292","heartbeat_back_addr":"172.21.15.89:6815/1060450292","heartbeat_front_addr":"172.21.15.89:6813/1060450292","state":["exists","up"]},{"osd":3,"uuid":"2dcaafe0-5837-410b-ae99-c7561ef6b150","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6811","nonce":2751619171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6813","nonce":2751619171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6817","nonce":2751619171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6815","nonce":2751619171}]},"public_addr":"172.21.15.190:6811/2751619171","cluster_addr":"172.21.15.190:6813/2751619171","heartbeat_back_addr":"172.21.15.190:6817/2751619171","heartbeat_front_addr":"172.21.15.190:6815/2751619171","state":["exists","up"]},{"osd":4,"uuid":"5619c403-27fd-48de-8cee-c5e84289cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6819","nonce":1174298329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6821","nonce":1174298329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6825","nonce":1174298329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6823","nonce":1174298329}]},"public_addr":"172.21.15.190:6819/1174298329","cluster_addr":"172.21.15.190:6821/1174298329","heartbeat_back_addr":"172.21.15.190:6825/1174298329","heartbeat_front_addr":"172.21.15.190:6823/1174298329","state":["exists","up"]},{"osd":5,"uuid":"82ab73a3-1212-4b0d-ab05-aa9c01dd5586","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6803","nonce":2434772607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6805","nonce":2434772607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6809","nonce":2434772607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6807","nonce":2434772607}]},"public_addr":"172.21.15.190:6803/2434772607","cluster_addr":"172.21.15.190:6805/2434772607","heartbeat_back_addr":"172.21.15.190:6809/2434772607","heartbeat_front_addr":"172.21.15.190:6807/2434772607","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.511833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:01.832703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.351820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.269178+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.511490+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.178520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T10:38:16.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 -- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe82403be30 msgr2=0x7fe82403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe82403be30 0x7fe82403e2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009a50 tx=0x7fe82c008040 comp rx=0 tx=0).stop 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 10 monclient: shutdown 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 -- 172.21.15.89:0/3582724763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8381399c0 msgr2=0x7fe838120d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8381399c0 0x7fe838120d70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe83000cb20 tx=0x7fe83000a060 comp rx=0 tx=0).stop 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 -- 172.21.15.89:0/3582724763 shutdown_connections 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe82403be30 0x7fe82403e2f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe8381399c0 0x7fe838120d70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe8380a56b0 0x7fe838120830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 --2- 172.21.15.89:0/3582724763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe83806e050 0x7fe8381202f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 -- 172.21.15.89:0/3582724763 >> 172.21.15.89:0/3582724763 conn(0x7fe8380aa2a0 msgr2=0x7fe8380724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 -- 172.21.15.89:0/3582724763 shutdown_connections 2024-08-19T10:38:16.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.198+0000 7fe83f125640 1 -- 172.21.15.89:0/3582724763 wait complete. 2024-08-19T10:38:16.216 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-08-19T10:38:16.217 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-08-19T10:38:16.218 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-08-19T10:38:16.218 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-08-19T10:38:16.218 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-08-19T10:38:16.219 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-08-19T10:38:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.326+0000 7f865bb55640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.326+0000 7f865bb55640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.326+0000 7f865bb55640 10 monclient: monmap: 2024-08-19T10:38:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.332373+0000 2024-08-19T10:38:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.332373+0000 2024-08-19T10:38:16.331 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.332 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 1 Processor -- start 2024-08-19T10:38:16.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 1 -- start start 2024-08-19T10:38:16.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 10 monclient: init 2024-08-19T10:38:16.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 10 monclient: _add_conns ranks=[8,0,4,6,7,10,5,3,9,2,1] 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f865406e050 0x7f8654136270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 10 monclient(hunting): picked mon.noname-e con 0x7f865406e050 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8654073ee0 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8654074320 0x7f8654139840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 10 monclient(hunting): picked mon.noname-c con 0x7f8654074320 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f86590f0640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8654073ee0 0x7f8654139130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54156/0 (socket says 172.21.15.89:54156) 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f865406f190 con 0x7f8654073ee0 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f86590f0640 1 -- 172.21.15.89:0/1850395380 learned_addr learned my addr 172.21.15.89:0/1850395380 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8654074320 0x7f8654139840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 1 -- 172.21.15.89:0/1850395380 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f865406f470 con 0x7f8654074320 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 1 -- 172.21.15.89:0/1850395380 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f865406f750 con 0x7f865406e050 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865bb55640 10 monclient(hunting): authenticate will time out at 946.148985s 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f86598f1640 1 --2- 172.21.15.89:0/1850395380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f865406e050 0x7f8654136270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 10 monclient(hunting): get_auth_request con 0x7f8654074320 auth_method 0 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f86598f1640 10 monclient(hunting): get_auth_request con 0x7f865406e050 auth_method 0 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f86598f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f86598f1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f86598f1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f86598f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f86598f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f86598f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 10 monclient(hunting): handle_auth_done global_id 4243 payload 274 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 1 -- 172.21.15.89:0/1850395380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f865406e050 msgr2=0x7f8654136270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 1 --2- 172.21.15.89:0/1850395380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f865406e050 0x7f8654136270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 1 -- 172.21.15.89:0/1850395380 >> v1:172.21.15.89:6789/0 conn(0x7f8654073ee0 legacy=0x7f8654139130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 1 monclient: found mon.noname-c 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f865a0f2640 1 -- 172.21.15.89:0/1850395380 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865406e430 con 0x7f8654074320 2024-08-19T10:38:16.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f86598f1640 1 --2- 172.21.15.89:0/1850395380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f865406e050 0x7f8654136270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:16.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f44f3686640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.330+0000 7f44f3686640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient: monmap: 2024-08-19T10:38:16.340 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.336912+0000 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.336912+0000 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.341 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 1 Processor -- start 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 1 -- start start 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient: init 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient: _add_conns ranks=[9,10,4,8,0,7,1,6,3,2,5] 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f44ec0a3d00 0x7f44ec0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient(hunting): picked mon.noname-a con 0x7f44ec0a3d00 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f44ec0a1a00 0x7f44ec0a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient(hunting): picked mon.noname-f con 0x7f44ec0a1a00 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44ec09b1d0 0x7f44ec09b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient(hunting): picked mon.noname-c con 0x7f44ec09b1d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f44ec09d3d0 con 0x7f44ec0a3d00 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f44ec06fd00 con 0x7f44ec0a1a00 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f44ec0a4b60 con 0x7f44ec09b1d0 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f3686640 10 monclient(hunting): authenticate will time out at 946.153393s 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44ec09b1d0 0x7f44ec09b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44ec09b1d0 0x7f44ec09b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37736/0 (socket says 172.21.15.89:37736) 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 1 -- 172.21.15.89:0/971768953 learned_addr learned my addr 172.21.15.89:0/971768953 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 10 monclient(hunting): get_auth_request con 0x7f44ec09b1d0 auth_method 0 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f0c21640 1 --2- 172.21.15.89:0/971768953 >> v2:172.21.15.190:3300/0 conn(0x7f44ec0a1a00 0x7f44ec0a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f0c21640 10 monclient(hunting): get_auth_request con 0x7f44ec0a1a00 auth_method 0 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f0c21640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f0c21640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f0c21640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1422640 1 --2- 172.21.15.89:0/971768953 >> v2:172.21.15.89:3300/0 conn(0x7f44ec0a3d00 0x7f44ec0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865a0f2640 1 -- 172.21.15.89:0/1850395380 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8654074320 msgr2=0x7f8654139840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865a0f2640 1 --2- 172.21.15.89:0/1850395380 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8654074320 0x7f8654139840 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f864c009dc0 tx=0x7f864c01c850 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2533169b65568342 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1422640 10 monclient(hunting): get_auth_request con 0x7f44ec0a3d00 auth_method 0 2024-08-19T10:38:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1422640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1422640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1422640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 10 monclient(hunting): handle_auth_done global_id 4252 payload 274 2024-08-19T10:38:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 1 -- 172.21.15.89:0/971768953 >> v2:172.21.15.190:3300/0 conn(0x7f44ec0a1a00 msgr2=0x7f44ec0a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 1 --2- 172.21.15.89:0/971768953 >> v2:172.21.15.190:3300/0 conn(0x7f44ec0a1a00 0x7f44ec0a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 1 -- 172.21.15.89:0/971768953 >> v2:172.21.15.89:3300/0 conn(0x7f44ec0a3d00 msgr2=0x7f44ec0a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 1 --2- 172.21.15.89:0/971768953 >> v2:172.21.15.89:3300/0 conn(0x7f44ec0a3d00 0x7f44ec0a40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 1 monclient: found mon.noname-c 2024-08-19T10:38:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1c23640 1 -- 172.21.15.89:0/971768953 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec06db50 con 0x7f44ec09b1d0 2024-08-19T10:38:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f0c21640 1 --2- 172.21.15.89:0/971768953 >> v2:172.21.15.190:3300/0 conn(0x7f44ec0a1a00 0x7f44ec0a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:38:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f44f1422640 1 --2- 172.21.15.89:0/971768953 >> v2:172.21.15.89:3300/0 conn(0x7f44ec0a3d00 0x7f44ec0a40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:38:16.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 1 -- 172.21.15.89:0/1850395380 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f864c0ac070 con 0x7f8654074320 2024-08-19T10:38:16.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 10 monclient: dump: 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.351 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.352 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.352 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.352 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.352 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.352 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 1 -- 172.21.15.89:0/1850395380 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f864c0a68b0 con 0x7f8654074320 2024-08-19T10:38:16.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 1 -- 172.21.15.89:0/1850395380 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f864c0a7420 con 0x7f8654074320 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f86588ef640 10 monclient: dump: 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.353 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.354 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.354 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.354 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.354 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.354 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.354 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.354 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865bb55640 5 monclient: authenticate success, global_id 4243 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865bb55640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865bb55640 4 set_mon_vals no callback set 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865bb55640 10 monclient: shutdown 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865bb55640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865bb55640 1 -- 172.21.15.89:0/1850395380 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8654074320 msgr2=0x7f8654139840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865bb55640 1 --2- 172.21.15.89:0/1850395380 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8654074320 0x7f8654139840 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f864c009dc0 tx=0x7f864c01c850 comp rx=0 tx=0).stop 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865bb55640 1 -- 172.21.15.89:0/1850395380 shutdown_connections 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865bb55640 1 --2- 172.21.15.89:0/1850395380 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8654074320 0x7f8654139840 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865bb55640 1 --2- 172.21.15.89:0/1850395380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f865406e050 0x7f8654136270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.334+0000 7f865bb55640 1 -- 172.21.15.89:0/1850395380 >> 172.21.15.89:0/1850395380 conn(0x7f86540aa240 msgr2=0x7f86540aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 1 -- 172.21.15.89:0/1850395380 shutdown_connections 2024-08-19T10:38:16.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 1 -- 172.21.15.89:0/1850395380 wait complete. 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient: monmap: 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.356 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 1 Processor -- start 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 1 -- start start 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient: init 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient: _add_conns ranks=[0,6,8,7,3,2,4,5,1] 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865406e050 0x7f8654132b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient(hunting): picked mon.noname-a con 0x7f865406e050 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:16.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8654073ee0 0x7f86541330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient(hunting): picked mon.noname-d con 0x7f8654073ee0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8654074320 0x7f865413a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient(hunting): picked mon.noname-e con 0x7f8654074320 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8654143f00 con 0x7f865406e050 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8654139f90 con 0x7f8654073ee0 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f8654144080 con 0x7f8654074320 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865bb55640 10 monclient(hunting): authenticate will time out at 946.157648s 2024-08-19T10:38:16.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8654073ee0 0x7f86541330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8654073ee0 0x7f86541330a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46430/0 (socket says 172.21.15.89:46430) 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865a0f2640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8654074320 0x7f865413a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 1 -- 172.21.15.89:0/3999476716 learned_addr learned my addr 172.21.15.89:0/3999476716 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 10 monclient(hunting): get_auth_request con 0x7f8654073ee0 auth_method 0 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865a0f2640 10 monclient(hunting): get_auth_request con 0x7f8654074320 auth_method 0 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865a0f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865a0f2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865a0f2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86598f1640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865406e050 0x7f8654132b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86598f1640 10 monclient(hunting): get_auth_request con 0x7f865406e050 auth_method 0 2024-08-19T10:38:16.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86598f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86598f1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86598f1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86598f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86598f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86598f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865a0f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865a0f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f865a0f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 10 monclient(hunting): handle_auth_done global_id 4218 payload 824 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 1 -- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8654074320 msgr2=0x7f865413a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8654074320 0x7f865413a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 1 -- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865406e050 msgr2=0x7f8654132b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f865406e050 0x7f8654132b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 1 monclient: found mon.noname-d 2024-08-19T10:38:16.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f86590f0640 1 -- 172.21.15.89:0/3999476716 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8654135060 con 0x7f8654073ee0 2024-08-19T10:38:16.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44f1c23640 1 -- 172.21.15.89:0/971768953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44ec09b1d0 msgr2=0x7f44ec09b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44f1c23640 1 --2- 172.21.15.89:0/971768953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44ec09b1d0 0x7f44ec09b660 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009e50 tx=0x7f44e401c8b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=280940534e97bd5b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 1 -- 172.21.15.89:0/971768953 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44e40ac020 con 0x7f44ec09b1d0 2024-08-19T10:38:16.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:16.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 10 monclient: dump: 2024-08-19T10:38:16.362 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.363 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 1 -- 172.21.15.89:0/971768953 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e4009110 con 0x7f44ec09b1d0 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 1 -- 172.21.15.89:0/971768953 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44e40a6d10 con 0x7f44ec09b1d0 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44e3fff640 10 monclient: dump: 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.364 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44f3686640 5 monclient: authenticate success, global_id 4252 2024-08-19T10:38:16.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44f3686640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.338+0000 7f44f3686640 4 set_mon_vals no callback set 2024-08-19T10:38:16.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f86590f0640 1 -- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8654073ee0 msgr2=0x7f86541330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f86590f0640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8654073ee0 0x7f86541330a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f864400c4b0 tx=0x7f864400c970 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 10 monclient: shutdown 2024-08-19T10:38:16.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 1 -- 172.21.15.89:0/971768953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44ec09b1d0 msgr2=0x7f44ec09b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 1 --2- 172.21.15.89:0/971768953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44ec09b1d0 0x7f44ec09b660 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009e50 tx=0x7f44e401c8b0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 1 -- 172.21.15.89:0/3999476716 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8644005d00 con 0x7f8654073ee0 2024-08-19T10:38:16.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 10 monclient: dump: 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.368 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 1 -- 172.21.15.89:0/3999476716 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86440074e0 con 0x7f8654073ee0 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f865bb55640 5 monclient: authenticate success, global_id 4218 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 1 -- 172.21.15.89:0/3999476716 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8644007690 con 0x7f8654073ee0 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864bfff640 4 set_mon_vals no callback set 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 10 monclient: dump: 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.370 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f865bb55640 10 monclient: _renew_subs 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f865bb55640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f865bb55640 1 -- 172.21.15.89:0/3999476716 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86540a7220 con 0x7f8654073ee0 2024-08-19T10:38:16.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f865bb55640 10 monclient: _renew_subs 2024-08-19T10:38:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f865bb55640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f865bb55640 1 -- 172.21.15.89:0/3999476716 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f865413d920 con 0x7f8654073ee0 2024-08-19T10:38:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 1 -- 172.21.15.89:0/971768953 shutdown_connections 2024-08-19T10:38:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 1 --2- 172.21.15.89:0/971768953 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44ec09b1d0 0x7f44ec09b660 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 1 --2- 172.21.15.89:0/971768953 >> v2:172.21.15.190:3300/0 conn(0x7f44ec0a1a00 0x7f44ec0a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 1 --2- 172.21.15.89:0/971768953 >> v2:172.21.15.89:3300/0 conn(0x7f44ec0a3d00 0x7f44ec0a40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 1 -- 172.21.15.89:0/971768953 >> 172.21.15.89:0/971768953 conn(0x7f44ec0a4df0 msgr2=0x7f44ec0a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f865bb55640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f865bb55640 1 -- 172.21.15.89:0/3999476716 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f865413c400 con 0x7f8654073ee0 2024-08-19T10:38:16.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 1 -- 172.21.15.89:0/971768953 shutdown_connections 2024-08-19T10:38:16.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 1 -- 172.21.15.89:0/971768953 wait complete. 2024-08-19T10:38:16.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 1 -- 172.21.15.89:0/3999476716 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8644007070 con 0x7f8654073ee0 2024-08-19T10:38:16.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f863803be00 0x7f863803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 1 -- 172.21.15.89:0/3999476716 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f864404e530 con 0x7f8654073ee0 2024-08-19T10:38:16.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] conn(0x7f8638041c70 0x7f86380440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 1 -- 172.21.15.89:0/3999476716 --> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8638044780 con 0x7f8638041c70 2024-08-19T10:38:16.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 1 -- 172.21.15.89:0/3999476716 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8644055050 con 0x7f8654073ee0 2024-08-19T10:38:16.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f864a7fc640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-08-19T10:38:16.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 10 monclient: monmap: 2024-08-19T10:38:16.375 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.375 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.375 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.375 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.375 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.375 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.376 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.376 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.376 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.376 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.376 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.376 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.376 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.376 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.376 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.376 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f865a0f2640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] conn(0x7f8638041c70 0x7f86380440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f865a0f2640 10 monclient: get_auth_request con 0x7f8638041c70 auth_method 0 2024-08-19T10:38:16.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f86598f1640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f863803be00 0x7f863803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f86598f1640 10 monclient: get_auth_request con 0x7f863803be00 auth_method 0 2024-08-19T10:38:16.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 1 Processor -- start 2024-08-19T10:38:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 1 -- start start 2024-08-19T10:38:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.342+0000 7f44f3686640 10 monclient: init 2024-08-19T10:38:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient: _add_conns ranks=[5,6,2,8,0,4,7,1,3] 2024-08-19T10:38:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44ec09b1d0 0x7f44ec09f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient(hunting): picked mon.noname-h con 0x7f44ec09b1d0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44ec0a1a00 0x7f44ec09fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient(hunting): picked mon.noname-d con 0x7f44ec0a1a00 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44ec0a3d00 0x7f44ec0a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient(hunting): picked mon.noname-b con 0x7f44ec0a3d00 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:16.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec0a5bd0 con 0x7f44ec0a3d00 2024-08-19T10:38:16.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f44ec069630 con 0x7f44ec09b1d0 2024-08-19T10:38:16.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f44ec141140 con 0x7f44ec0a1a00 2024-08-19T10:38:16.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient(hunting): authenticate will time out at 946.164388s 2024-08-19T10:38:16.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1c23640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44ec0a3d00 0x7f44ec0a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44ec0a1a00 0x7f44ec09fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44ec0a1a00 0x7f44ec09fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46438/0 (socket says 172.21.15.89:46438) 2024-08-19T10:38:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1c23640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44ec0a3d00 0x7f44ec0a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44364/0 (socket says 172.21.15.89:44364) 2024-08-19T10:38:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 1 -- 172.21.15.89:0/2785421946 learned_addr learned my addr 172.21.15.89:0/2785421946 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 10 monclient(hunting): get_auth_request con 0x7f44ec0a1a00 auth_method 0 2024-08-19T10:38:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1c23640 10 monclient(hunting): get_auth_request con 0x7f44ec0a3d00 auth_method 0 2024-08-19T10:38:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1c23640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1c23640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1c23640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1422640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44ec09b1d0 0x7f44ec09f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1c23640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1c23640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1c23640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1422640 10 monclient(hunting): get_auth_request con 0x7f44ec09b1d0 auth_method 0 2024-08-19T10:38:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1422640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1422640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f1422640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 10 monclient(hunting): handle_auth_done global_id 4227 payload 824 2024-08-19T10:38:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 1 -- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44ec09b1d0 msgr2=0x7f44ec09f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44ec09b1d0 0x7f44ec09f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 1 -- 172.21.15.89:0/2785421946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44ec0a3d00 msgr2=0x7f44ec0a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44ec0a3d00 0x7f44ec0a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 1 monclient: found mon.noname-d 2024-08-19T10:38:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 1 -- 172.21.15.89:0/2785421946 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec13cb80 con 0x7f44ec0a1a00 2024-08-19T10:38:16.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f86598f1640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f863803be00 0x7f863803e2c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8650009580 tx=0x7f8650009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 1 -- 172.21.15.89:0/2785421946 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44ec0a1a00 msgr2=0x7f44ec09fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f0c21640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44ec0a1a00 0x7f44ec09fd30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00e810 tx=0x7f44dc00ecd0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 1 -- 172.21.15.89:0/2785421946 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44dc017070 con 0x7f44ec0a1a00 2024-08-19T10:38:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 10 monclient: dump: 2024-08-19T10:38:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.385 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.386 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 1 -- 172.21.15.89:0/2785421946 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44dc00cd50 con 0x7f44ec0a1a00 2024-08-19T10:38:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 1 -- 172.21.15.89:0/2785421946 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44dc00f040 con 0x7f44ec0a1a00 2024-08-19T10:38:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 5 monclient: authenticate success, global_id 4227 2024-08-19T10:38:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient: _renew_subs 2024-08-19T10:38:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 1 -- 172.21.15.89:0/2785421946 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec13dbd0 con 0x7f44ec0a1a00 2024-08-19T10:38:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e37fe640 4 set_mon_vals no callback set 2024-08-19T10:38:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44e1ffb640 10 monclient: dump: 2024-08-19T10:38:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.388 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.389 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f865a0f2640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] conn(0x7f8638041c70 0x7f86380440b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient: _renew_subs 2024-08-19T10:38:16.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.346+0000 7f44f3686640 1 -- 172.21.15.89:0/2785421946 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec0a6440 con 0x7f44ec0a1a00 2024-08-19T10:38:16.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44f3686640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44f3686640 1 -- 172.21.15.89:0/2785421946 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44ec09b660 con 0x7f44ec0a1a00 2024-08-19T10:38:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44e1ffb640 1 -- 172.21.15.89:0/2785421946 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f44dc01a070 con 0x7f44ec0a1a00 2024-08-19T10:38:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44e1ffb640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f44d003be00 0x7f44d003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44e1ffb640 1 -- 172.21.15.89:0/2785421946 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f44dc04e3f0 con 0x7f44ec0a1a00 2024-08-19T10:38:16.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f864a7fc640 1 -- 172.21.15.89:0/3999476716 <== osd.2 v2:172.21.15.89:6808/1060450292 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8638044780 con 0x7f8638041c70 2024-08-19T10:38:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44e1ffb640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] conn(0x7f44d0041c80 0x7f44d00440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44e1ffb640 1 -- 172.21.15.89:0/2785421946 --> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44dc01bec0 con 0x7f44d0041c80 2024-08-19T10:38:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44e1ffb640 1 -- 172.21.15.89:0/2785421946 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f44dc055050 con 0x7f44ec0a1a00 2024-08-19T10:38:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44e1ffb640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-08-19T10:38:16.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44f1422640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f44d003be00 0x7f44d003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44f1422640 10 monclient: get_auth_request con 0x7f44d003be00 auth_method 0 2024-08-19T10:38:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44f1c23640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] conn(0x7f44d0041c80 0x7f44d00440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.350+0000 7f44f1c23640 10 monclient: get_auth_request con 0x7f44d0041c80 auth_method 0 2024-08-19T10:38:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.354+0000 7f44f1422640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f44d003be00 0x7f44d003e2c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f44e800d8d0 tx=0x7f44e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.354+0000 7f44f1c23640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] conn(0x7f44d0041c80 0x7f44d00440c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.358+0000 7f44e1ffb640 1 -- 172.21.15.89:0/2785421946 <== osd.3 v2:172.21.15.190:6810/2751619171 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44dc01bec0 con 0x7f44d0041c80 2024-08-19T10:38:16.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 1 -- 172.21.15.89:0/3999476716 --> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8654133d00 con 0x7f8638041c70 2024-08-19T10:38:16.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f864a7fc640 1 -- 172.21.15.89:0/3999476716 <== osd.2 v2:172.21.15.89:6808/1060450292 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8654133d00 con 0x7f8638041c70 2024-08-19T10:38:16.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 1 -- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] conn(0x7f8638041c70 msgr2=0x7f86380440b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] conn(0x7f8638041c70 0x7f86380440b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 1 -- 172.21.15.89:0/3999476716 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f863803be00 msgr2=0x7f863803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f863803be00 0x7f863803e2c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8650009580 tx=0x7f8650009190 comp rx=0 tx=0).stop 2024-08-19T10:38:16.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 10 monclient: shutdown 2024-08-19T10:38:16.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 1 -- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8654073ee0 msgr2=0x7f86541330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 1 --2- 172.21.15.89:0/3999476716 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8654073ee0 0x7f86541330a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f864400c4b0 tx=0x7f864400c970 comp rx=0 tx=0).stop 2024-08-19T10:38:16.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865a0f2640 1 -- 172.21.15.89:0/3999476716 reap_dead start 2024-08-19T10:38:16.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 1 -- 172.21.15.89:0/3999476716 shutdown_connections 2024-08-19T10:38:16.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 1 -- 172.21.15.89:0/3999476716 >> 172.21.15.89:0/3999476716 conn(0x7f86540aa240 msgr2=0x7f8654072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 1 -- 172.21.15.89:0/3999476716 shutdown_connections 2024-08-19T10:38:16.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.378+0000 7f865bb55640 1 -- 172.21.15.89:0/3999476716 wait complete. 2024-08-19T10:38:16.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.382+0000 7f44f3686640 1 -- 172.21.15.89:0/2785421946 --> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f44ec064590 con 0x7f44d0041c80 2024-08-19T10:38:16.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.382+0000 7f44e1ffb640 1 -- 172.21.15.89:0/2785421946 <== osd.3 v2:172.21.15.190:6810/2751619171 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f44ec064590 con 0x7f44d0041c80 2024-08-19T10:38:16.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.382+0000 7f44f3686640 1 -- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] conn(0x7f44d0041c80 msgr2=0x7f44d00440c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.382+0000 7f44f3686640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] conn(0x7f44d0041c80 0x7f44d00440c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.382+0000 7f44f3686640 1 -- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f44d003be00 msgr2=0x7f44d003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.382+0000 7f44f3686640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f44d003be00 0x7f44d003e2c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f44e800d8d0 tx=0x7f44e8008040 comp rx=0 tx=0).stop 2024-08-19T10:38:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.382+0000 7f44f3686640 10 monclient: shutdown 2024-08-19T10:38:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.382+0000 7f44f3686640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.386+0000 7f44f3686640 1 -- 172.21.15.89:0/2785421946 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44ec0a1a00 msgr2=0x7f44ec09fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.386+0000 7f44f3686640 1 --2- 172.21.15.89:0/2785421946 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44ec0a1a00 0x7f44ec09fd30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00e810 tx=0x7f44dc00ecd0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.386+0000 7f44f1c23640 1 -- 172.21.15.89:0/2785421946 reap_dead start 2024-08-19T10:38:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.386+0000 7f44f3686640 1 -- 172.21.15.89:0/2785421946 shutdown_connections 2024-08-19T10:38:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.386+0000 7f44f3686640 1 -- 172.21.15.89:0/2785421946 >> 172.21.15.89:0/2785421946 conn(0x7f44ec0a4df0 msgr2=0x7f44ec0a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.386+0000 7f44f3686640 1 -- 172.21.15.89:0/2785421946 shutdown_connections 2024-08-19T10:38:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.386+0000 7f44f3686640 1 -- 172.21.15.89:0/2785421946 wait complete. 2024-08-19T10:38:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient: monmap: 2024-08-19T10:38:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.394757+0000 2024-08-19T10:38:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.394757+0000 2024-08-19T10:38:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.402 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.402 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient: monmap: 2024-08-19T10:38:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.394757+0000 2024-08-19T10:38:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.394757+0000 2024-08-19T10:38:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.403 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.404 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.404 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 1 Processor -- start 2024-08-19T10:38:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 1 Processor -- start 2024-08-19T10:38:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 1 -- start start 2024-08-19T10:38:16.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient: init 2024-08-19T10:38:16.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 1 -- start start 2024-08-19T10:38:16.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient: init 2024-08-19T10:38:16.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f6621040640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f6621040640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f6621040640 10 monclient: monmap: 2024-08-19T10:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.396169+0000 2024-08-19T10:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.396169+0000 2024-08-19T10:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.407 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.408 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.408 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient: _add_conns ranks=[3,7,6,9,4,5,8,0,2,1,10] 2024-08-19T10:38:16.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c80a3b30 0x7f55c80a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient: _add_conns ranks=[10,5,7,0,1,9,4,6,8,2,3] 2024-08-19T10:38:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient(hunting): picked mon.noname-g con 0x7f55c80a3b30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55c80a3f10 0x7f55c80a4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient(hunting): picked mon.noname-i con 0x7f55c80a3f10 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:16.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f55c81364f0 0x7f55c8136970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa080a6860 0x7ffa080a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient(hunting): picked mon.noname-d con 0x7f55c81364f0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:16.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f55c80a08b0 con 0x7f55c80a3b30 2024-08-19T10:38:16.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f55c8074170 con 0x7f55c81364f0 2024-08-19T10:38:16.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f55c80a0630 con 0x7f55c80a3f10 2024-08-19T10:38:16.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55ce096640 10 monclient(hunting): authenticate will time out at 946.211614s 2024-08-19T10:38:16.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient(hunting): picked mon.noname-f con 0x7ffa080a6860 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:16.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a7180 0x7ffa080a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient(hunting): picked mon.noname-h con 0x7ffa080a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:16.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ffa08139620 0x7ffa08139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient(hunting): picked mon.noname-i con 0x7ffa08139620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:16.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ffa080a8190 con 0x7ffa080a6860 2024-08-19T10:38:16.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ffa080a33d0 con 0x7ffa080a7180 2024-08-19T10:38:16.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ffa080a2ec0 con 0x7ffa08139620 2024-08-19T10:38:16.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7ffa0faa2640 10 monclient(hunting): authenticate will time out at 946.211850s 2024-08-19T10:38:16.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55c7fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f55c81364f0 0x7f55c8136970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55c7fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f55c81364f0 0x7f55c8136970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46446/0 (socket says 172.21.15.89:46446) 2024-08-19T10:38:16.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55c7fff640 1 -- 172.21.15.89:0/3370060548 learned_addr learned my addr 172.21.15.89:0/3370060548 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55c7fff640 10 monclient(hunting): get_auth_request con 0x7f55c81364f0 auth_method 0 2024-08-19T10:38:16.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.390+0000 7f55c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa080a6860 0x7ffa080a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c77fe640 1 --2- 172.21.15.89:0/3370060548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c80a3b30 0x7f55c80a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c6ffd640 1 --2- 172.21.15.89:0/3370060548 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55c80a3f10 0x7f55c80a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c77fe640 10 monclient(hunting): get_auth_request con 0x7f55c80a3b30 auth_method 0 2024-08-19T10:38:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c6ffd640 10 monclient(hunting): get_auth_request con 0x7f55c80a3f10 auth_method 0 2024-08-19T10:38:16.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffa080a6860 0x7ffa080a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:44164/0 (socket says 172.21.15.89:44164) 2024-08-19T10:38:16.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0e03f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ffa08139620 0x7ffa08139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 1 -- 172.21.15.89:0/878488999 learned_addr learned my addr 172.21.15.89:0/878488999 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 10 monclient(hunting): get_auth_request con 0x7ffa080a6860 auth_method 0 2024-08-19T10:38:16.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 1 Processor -- start 2024-08-19T10:38:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d03d640 1 --2- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a7180 0x7ffa080a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d03d640 10 monclient(hunting): get_auth_request con 0x7ffa080a7180 auth_method 0 2024-08-19T10:38:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d03d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d03d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d03d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0e03f640 10 monclient(hunting): get_auth_request con 0x7ffa08139620 auth_method 0 2024-08-19T10:38:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0e03f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0e03f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0e03f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 10 monclient(hunting): handle_auth_done global_id 4236 payload 274 2024-08-19T10:38:16.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 1 -- 172.21.15.89:0/3370060548 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55c80a3f10 msgr2=0x7f55c80a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 1 --2- 172.21.15.89:0/3370060548 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55c80a3f10 0x7f55c80a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 1 -- 172.21.15.89:0/3370060548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c80a3b30 msgr2=0x7f55c80a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 1 --2- 172.21.15.89:0/3370060548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c80a3b30 0x7f55c80a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 1 monclient: found mon.noname-d 2024-08-19T10:38:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c7fff640 1 -- 172.21.15.89:0/3370060548 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c80a1950 con 0x7f55c81364f0 2024-08-19T10:38:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f55c6ffd640 1 --2- 172.21.15.89:0/3370060548 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55c80a3f10 0x7f55c80a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:38:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d03d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d03d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d03d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0e03f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0e03f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0e03f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 10 monclient(hunting): handle_auth_done global_id 4294 payload 274 2024-08-19T10:38:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 1 -- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ffa08139620 msgr2=0x7ffa08139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 1 --2- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ffa08139620 0x7ffa08139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 1 -- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a7180 msgr2=0x7ffa080a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 1 --2- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a7180 0x7ffa080a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 1 monclient: found mon.noname-f 2024-08-19T10:38:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:38:16.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7ffa0d83e640 1 -- 172.21.15.89:0/878488999 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa080a4cd0 con 0x7ffa080a6860 2024-08-19T10:38:16.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 1 -- start start 2024-08-19T10:38:16.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 10 monclient: init 2024-08-19T10:38:16.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 10 monclient: _add_conns ranks=[2,9,6,10,4,3,1,0,8,7,5] 2024-08-19T10:38:16.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f661c0a51d0 0x7f661c0a6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 10 monclient(hunting): picked mon.noname-b con 0x7f661c0a51d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f661c0a55b0 0x7f661c0a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 10 monclient(hunting): picked mon.noname-a con 0x7f661c0a55b0 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f661c0a72e0 0x7f661c143e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 10 monclient(hunting): picked mon.noname-d con 0x7f661c0a72e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f661c0a8480 con 0x7f661c0a55b0 2024-08-19T10:38:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f661c0a81d0 con 0x7f661c0a51d0 2024-08-19T10:38:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f661c0a3dd0 con 0x7f661c0a72e0 2024-08-19T10:38:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f6621040640 10 monclient(hunting): authenticate will time out at 946.213069s 2024-08-19T10:38:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f661c0a55b0 0x7f661c0a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f661c0a55b0 0x7f661c0a6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57710/0 (socket says 172.21.15.89:57710) 2024-08-19T10:38:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 1 -- 172.21.15.89:0/2534277217 learned_addr learned my addr 172.21.15.89:0/2534277217 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 10 monclient(hunting): get_auth_request con 0x7f661c0a55b0 auth_method 0 2024-08-19T10:38:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661ad9d640 1 --2- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f661c0a51d0 0x7f661c0a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 10 monclient(hunting): handle_auth_done global_id 4320 payload 274 2024-08-19T10:38:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 1 -- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f661c0a72e0 msgr2=0x7f661c143e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 1 --2- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f661c0a72e0 0x7f661c143e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 1 -- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f661c0a51d0 msgr2=0x7f661c0a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 1 --2- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f661c0a51d0 0x7f661c0a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 1 monclient: found mon.noname-a 2024-08-19T10:38:16.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:16.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661a59c640 1 -- 172.21.15.89:0/2534277217 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c0a4410 con 0x7f661c0a55b0 2024-08-19T10:38:16.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661ad9d640 10 monclient: get_auth_request con 0x7f661c0a51d0 auth_method 0 2024-08-19T10:38:16.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f661ad9d640 1 --2- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f661c0a51d0 0x7f661c0a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:16.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient: monmap: 2024-08-19T10:38:16.438 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.438 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.438 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.398025+0000 2024-08-19T10:38:16.438 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.398025+0000 2024-08-19T10:38:16.438 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.438 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.438 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.439 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.439 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.439 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.439 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.439 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.439 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.439 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.439 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.439 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.439 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.439 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 1 Processor -- start 2024-08-19T10:38:16.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 1 -- start start 2024-08-19T10:38:16.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient: init 2024-08-19T10:38:16.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient: _add_conns ranks=[6,0,9,1,10,4,5,8,2,3,7] 2024-08-19T10:38:16.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8eb008d320 0x7f8eb008d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient(hunting): picked mon.noname-d con 0x7f8eb008d320 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:16.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8eb0070450 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:16.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8eb00a6f70 0x7f8eb014e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb5212640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8eb0070450 0x7f8eb00a6860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54158/0 (socket says 172.21.15.89:54158) 2024-08-19T10:38:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient(hunting): picked mon.noname-a con 0x7f8eb00a6f70 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb0072560 con 0x7f8eb0070450 2024-08-19T10:38:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb0070240 con 0x7f8eb00a6f70 2024-08-19T10:38:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8eb006fb80 con 0x7f8eb008d320 2024-08-19T10:38:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6a15640 10 monclient(hunting): authenticate will time out at 946.215053s 2024-08-19T10:38:16.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb5a13640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8eb008d320 0x7f8eb008d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8eb00a6f70 0x7f8eb014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb5a13640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8eb008d320 0x7f8eb008d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46462/0 (socket says 172.21.15.89:46462) 2024-08-19T10:38:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 10 monclient(hunting): get_auth_request con 0x7f8eb00a6f70 auth_method 0 2024-08-19T10:38:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb5a13640 1 -- 172.21.15.89:0/3336525591 learned_addr learned my addr 172.21.15.89:0/3336525591 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb5a13640 10 monclient(hunting): get_auth_request con 0x7f8eb008d320 auth_method 0 2024-08-19T10:38:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb5a13640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb5a13640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb5a13640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb5a13640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb5a13640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb5a13640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 10 monclient(hunting): handle_auth_done global_id 4329 payload 274 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 1 -- 172.21.15.89:0/3336525591 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8eb008d320 msgr2=0x7f8eb008d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 1 --2- 172.21.15.89:0/3336525591 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8eb008d320 0x7f8eb008d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 1 -- 172.21.15.89:0/3336525591 >> v1:172.21.15.89:6789/0 conn(0x7f8eb0070450 legacy=0x7f8eb00a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 1 monclient: found mon.noname-a 2024-08-19T10:38:16.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:16.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.394+0000 7f8eb6214640 1 -- 172.21.15.89:0/3336525591 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb006f910 con 0x7f8eb00a6f70 2024-08-19T10:38:16.445 INFO:teuthology.orchestra.run.smithi089.stdout:47244640261 2024-08-19T10:38:16.445 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-19T10:38:16.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c7fff640 1 -- 172.21.15.89:0/3370060548 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f55c81364f0 msgr2=0x7f55c8136970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c7fff640 1 --2- 172.21.15.89:0/3370060548 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f55c81364f0 0x7f55c8136970 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009eb0 tx=0x7f55c001c8f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f293fe08d9c9d490 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 1 -- 172.21.15.89:0/3370060548 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f55c00ac070 con 0x7f55c81364f0 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 10 monclient: dump: 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.447 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 1 -- 172.21.15.89:0/3370060548 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c00090d0 con 0x7f55c81364f0 2024-08-19T10:38:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 1 -- 172.21.15.89:0/3370060548 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f55c00a8d20 con 0x7f55c81364f0 2024-08-19T10:38:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0d83e640 1 -- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa080a6860 msgr2=0x7ffa080a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0d83e640 1 --2- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa080a6860 0x7ffa080a6c40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009870 tx=0x7ffa0401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72e43cb4010c2b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55c67fc640 10 monclient: dump: 2024-08-19T10:38:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.450 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.451 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 5 monclient: authenticate success, global_id 4236 2024-08-19T10:38:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 4 set_mon_vals no callback set 2024-08-19T10:38:16.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 1 -- 172.21.15.89:0/878488999 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa040b0070 con 0x7ffa080a6860 2024-08-19T10:38:16.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 10 monclient: dump: 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.453 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.454 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 1 -- 172.21.15.89:0/878488999 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa040aa810 con 0x7ffa080a6860 2024-08-19T10:38:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 1 -- 172.21.15.89:0/878488999 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa040a9300 con 0x7ffa080a6860 2024-08-19T10:38:16.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:38:16.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0c83c640 10 monclient: dump: 2024-08-19T10:38:16.455 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.455 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.455 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.455 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.455 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.455 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.455 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.456 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.456 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.456 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.456 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.456 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.456 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.456 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.456 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.456 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 10 monclient: shutdown 2024-08-19T10:38:16.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 1 -- 172.21.15.89:0/3370060548 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f55c81364f0 msgr2=0x7f55c8136970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 1 --2- 172.21.15.89:0/3370060548 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f55c81364f0 0x7f55c8136970 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009eb0 tx=0x7f55c001c8f0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 5 monclient: authenticate success, global_id 4294 2024-08-19T10:38:16.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 4 set_mon_vals no callback set 2024-08-19T10:38:16.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f661a59c640 1 -- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f661c0a55b0 msgr2=0x7f661c0a6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f661a59c640 1 --2- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f661c0a55b0 0x7f661c0a6da0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6610009870 tx=0x7f661001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878c2a74684039c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 10 monclient: shutdown 2024-08-19T10:38:16.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 1 -- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa080a6860 msgr2=0x7ffa080a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 1 --2- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa080a6860 0x7ffa080a6c40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009870 tx=0x7ffa0401c440 comp rx=0 tx=0).stop 2024-08-19T10:38:16.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 1 -- 172.21.15.89:0/3370060548 shutdown_connections 2024-08-19T10:38:16.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 1 --2- 172.21.15.89:0/3370060548 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f55c81364f0 0x7f55c8136970 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 1 --2- 172.21.15.89:0/3370060548 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55c80a3f10 0x7f55c80a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 1 --2- 172.21.15.89:0/3370060548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c80a3b30 0x7f55c80a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 1 -- 172.21.15.89:0/3370060548 >> 172.21.15.89:0/3370060548 conn(0x7f55c80a8f60 msgr2=0x7f55c80a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 1 -- 172.21.15.89:0/878488999 shutdown_connections 2024-08-19T10:38:16.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 1 --2- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ffa08139620 0x7ffa08139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 1 --2- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a7180 0x7ffa080a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 1 --2- 172.21.15.89:0/878488999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa080a6860 0x7ffa080a6c40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7ffa0faa2640 1 -- 172.21.15.89:0/878488999 >> 172.21.15.89:0/878488999 conn(0x7ffa08074b40 msgr2=0x7ffa080abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 1 -- 172.21.15.89:0/3370060548 shutdown_connections 2024-08-19T10:38:16.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.398+0000 7f55ce096640 1 -- 172.21.15.89:0/3370060548 wait complete. 2024-08-19T10:38:16.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 1 -- 172.21.15.89:0/878488999 shutdown_connections 2024-08-19T10:38:16.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient: monmap: 2024-08-19T10:38:16.462 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.462 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.462 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.463 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.464 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.464 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 1 Processor -- start 2024-08-19T10:38:16.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 1 -- 172.21.15.89:0/878488999 wait complete. 2024-08-19T10:38:16.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 1 -- start start 2024-08-19T10:38:16.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient: init 2024-08-19T10:38:16.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6214640 1 -- 172.21.15.89:0/3336525591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eb00a6f70 msgr2=0x7f8eb014e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6214640 1 --2- 172.21.15.89:0/3336525591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eb00a6f70 0x7f8eb014e060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0008830 tx=0x7f8ea0024a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583181cfd3be5730 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6619d9b640 1 -- 172.21.15.89:0/2534277217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66100b0070 con 0x7f661c0a55b0 2024-08-19T10:38:16.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6619d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 1 -- 172.21.15.89:0/3336525591 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ea000be60 con 0x7f8eb00a6f70 2024-08-19T10:38:16.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient: _add_conns ranks=[2,5,8,4,3,0,6,1,7] 2024-08-19T10:38:16.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f55c80a3b30 0x7f55c8139f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient(hunting): picked mon.noname-b con 0x7f55c80a3b30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:16.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f55c80a3f10 0x7f55c813a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient(hunting): picked mon.noname-h con 0x7f55c80a3f10 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:16.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55c81364f0 0x7f55c813aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient(hunting): picked mon.noname-e con 0x7f55c81364f0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:16.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f55c80a4870 con 0x7f55c80a3b30 2024-08-19T10:38:16.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f55c813f750 con 0x7f55c80a3f10 2024-08-19T10:38:16.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f55c806d140 con 0x7f55c81364f0 2024-08-19T10:38:16.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55ce096640 10 monclient(hunting): authenticate will time out at 946.221803s 2024-08-19T10:38:16.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient: monmap: 2024-08-19T10:38:16.469 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.469 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.469 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.469 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.469 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.469 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.469 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.470 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.470 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.470 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.470 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.470 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.470 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.470 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.470 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.470 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55c81364f0 0x7f55c813aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55c81364f0 0x7f55c813aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47722/0 (socket says 172.21.15.89:47722) 2024-08-19T10:38:16.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 1 -- 172.21.15.89:0/8947784 learned_addr learned my addr 172.21.15.89:0/8947784 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 10 monclient(hunting): get_auth_request con 0x7f55c81364f0 auth_method 0 2024-08-19T10:38:16.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c6ffd640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f55c80a3f10 0x7f55c813a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c6ffd640 10 monclient(hunting): get_auth_request con 0x7f55c80a3f10 auth_method 0 2024-08-19T10:38:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c77fe640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f55c80a3b30 0x7f55c8139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c77fe640 10 monclient(hunting): get_auth_request con 0x7f55c80a3b30 auth_method 0 2024-08-19T10:38:16.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 1 Processor -- start 2024-08-19T10:38:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 10 monclient(hunting): handle_auth_done global_id 4238 payload 824 2024-08-19T10:38:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 1 -- 172.21.15.89:0/8947784 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f55c80a3f10 msgr2=0x7f55c813a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f55c80a3f10 0x7f55c813a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 1 -- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f55c80a3b30 msgr2=0x7f55c8139f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f55c80a3b30 0x7f55c8139f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 1 monclient: found mon.noname-e 2024-08-19T10:38:16.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f55c7fff640 1 -- 172.21.15.89:0/8947784 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c8072bb0 con 0x7f55c81364f0 2024-08-19T10:38:16.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6619d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:16.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6619d9b640 10 monclient: dump: 2024-08-19T10:38:16.475 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.476 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.477 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6619d9b640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6619d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 1 -- start start 2024-08-19T10:38:16.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient: init 2024-08-19T10:38:16.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6619d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 5 monclient: authenticate success, global_id 4320 2024-08-19T10:38:16.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:16.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6619d9b640 1 -- 172.21.15.89:0/2534277217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66100aa810 con 0x7f661c0a55b0 2024-08-19T10:38:16.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6619d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 4 set_mon_vals no callback set 2024-08-19T10:38:16.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 10 monclient: shutdown 2024-08-19T10:38:16.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 1 -- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f661c0a55b0 msgr2=0x7f661c0a6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 1 --2- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f661c0a55b0 0x7f661c0a6da0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6610009870 tx=0x7f661001c3e0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:16.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 10 monclient: dump: 2024-08-19T10:38:16.479 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.479 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.479 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.479 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.479 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.480 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.480 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.480 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.480 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.480 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.480 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.480 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.480 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.480 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.480 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.480 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 1 -- 172.21.15.89:0/3336525591 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea00adbd0 con 0x7f8eb00a6f70 2024-08-19T10:38:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 1 -- 172.21.15.89:0/3336525591 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ea00add80 con 0x7f8eb00a6f70 2024-08-19T10:38:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb4a11640 10 monclient: dump: 2024-08-19T10:38:16.481 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.482 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.482 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.482 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.482 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.482 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.482 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.482 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.482 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.482 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.482 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.482 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.483 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.483 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.483 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.483 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 5 monclient: authenticate success, global_id 4329 2024-08-19T10:38:16.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 4 set_mon_vals no callback set 2024-08-19T10:38:16.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 10 monclient: shutdown 2024-08-19T10:38:16.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 1 -- 172.21.15.89:0/3336525591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eb00a6f70 msgr2=0x7f8eb014e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 1 --2- 172.21.15.89:0/3336525591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eb00a6f70 0x7f8eb014e060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0008830 tx=0x7f8ea0024a20 comp rx=0 tx=0).stop 2024-08-19T10:38:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 1 -- 172.21.15.89:0/2534277217 shutdown_connections 2024-08-19T10:38:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 1 --2- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f661c0a72e0 0x7f661c143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 1 --2- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f661c0a55b0 0x7f661c0a6da0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 1 --2- 172.21.15.89:0/2534277217 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f661c0a51d0 0x7f661c0a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 1 -- 172.21.15.89:0/2534277217 >> 172.21.15.89:0/2534277217 conn(0x7f661c074b40 msgr2=0x7f661c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 1 -- 172.21.15.89:0/3336525591 shutdown_connections 2024-08-19T10:38:16.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 1 --2- 172.21.15.89:0/3336525591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eb00a6f70 0x7f8eb014e060 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 1 --2- 172.21.15.89:0/3336525591 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8eb008d320 0x7f8eb008d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 1 -- 172.21.15.89:0/3336525591 >> 172.21.15.89:0/3336525591 conn(0x7f8eb00a7f50 msgr2=0x7f8eb00a4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 1 -- 172.21.15.89:0/3336525591 shutdown_connections 2024-08-19T10:38:16.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 1 -- 172.21.15.89:0/3336525591 wait complete. 2024-08-19T10:38:16.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 1 -- 172.21.15.89:0/2534277217 shutdown_connections 2024-08-19T10:38:16.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 1 -- 172.21.15.89:0/2534277217 wait complete. 2024-08-19T10:38:16.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient: _add_conns ranks=[2,3,4,6,5,1,0,7,8] 2024-08-19T10:38:16.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a6860 0x7ffa08134960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient(hunting): picked mon.noname-b con 0x7ffa080a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:16.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa080a7180 0x7ffa08134ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient(hunting): picked mon.noname-g con 0x7ffa080a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:16.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa08139620 0x7ffa0809c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient(hunting): picked mon.noname-c con 0x7ffa08139620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:16.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ffa08143210 con 0x7ffa080a6860 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffa08143090 con 0x7ffa080a7180 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ffa08143390 con 0x7ffa08139620 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0faa2640 10 monclient(hunting): authenticate will time out at 946.223321s 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0e03f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa08139620 0x7ffa0809c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a6860 0x7ffa08134960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a6860 0x7ffa08134960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44400/0 (socket says 172.21.15.89:44400) 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0e03f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa08139620 0x7ffa0809c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37744/0 (socket says 172.21.15.89:37744) 2024-08-19T10:38:16.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 1 -- 172.21.15.89:0/61584284 learned_addr learned my addr 172.21.15.89:0/61584284 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 10 monclient(hunting): get_auth_request con 0x7ffa080a6860 auth_method 0 2024-08-19T10:38:16.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0e03f640 10 monclient(hunting): get_auth_request con 0x7ffa08139620 auth_method 0 2024-08-19T10:38:16.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0e03f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0e03f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0e03f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d03d640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa080a7180 0x7ffa08134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d03d640 10 monclient(hunting): get_auth_request con 0x7ffa080a7180 auth_method 0 2024-08-19T10:38:16.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d03d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d03d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d03d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0e03f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0e03f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0e03f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f6621040640 10 monclient: monmap: 2024-08-19T10:38:16.491 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.491 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.491 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.491 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.491 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.491 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.491 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.491 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.492 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.492 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.492 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.492 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.492 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.492 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.492 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.492 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 10 monclient(hunting): handle_auth_done global_id 4214 payload 824 2024-08-19T10:38:16.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 1 -- 172.21.15.89:0/61584284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa08139620 msgr2=0x7ffa0809c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa08139620 0x7ffa0809c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 1 -- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa080a7180 msgr2=0x7ffa08134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa080a7180 0x7ffa08134ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 1 monclient: found mon.noname-b 2024-08-19T10:38:16.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:16.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7ffa0d83e640 1 -- 172.21.15.89:0/61584284 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa080a0380 con 0x7ffa080a6860 2024-08-19T10:38:16.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.402+0000 7f8eb6a15640 10 monclient: monmap: 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.494 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.495 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.495 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.495 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.495 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.495 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 1 Processor -- start 2024-08-19T10:38:16.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 1 -- start start 2024-08-19T10:38:16.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 10 monclient: init 2024-08-19T10:38:16.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 1 Processor -- start 2024-08-19T10:38:16.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 1 -- start start 2024-08-19T10:38:16.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 10 monclient: init 2024-08-19T10:38:16.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 10 monclient: _add_conns ranks=[4,0,7,8,5,6,2,1,3] 2024-08-19T10:38:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8eb0070450 0x7f8eb01dde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 10 monclient(hunting): picked mon.noname-c con 0x7f8eb0070450 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eb008d320 0x7f8eb014d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 10 monclient(hunting): picked mon.noname-a con 0x7f8eb008d320 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8eb00a6f70 0x7f8eb014d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 10 monclient(hunting): picked mon.noname-i con 0x7f8eb00a6f70 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb008ddc0 con 0x7f8eb008d320 2024-08-19T10:38:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f8eb008dc40 con 0x7f8eb0070450 2024-08-19T10:38:16.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8eb0150dc0 con 0x7f8eb00a6f70 2024-08-19T10:38:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6a15640 10 monclient(hunting): authenticate will time out at 946.224953s 2024-08-19T10:38:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5212640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eb008d320 0x7f8eb014d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5212640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eb008d320 0x7f8eb014d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57726/0 (socket says 172.21.15.89:57726) 2024-08-19T10:38:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5212640 1 -- 172.21.15.89:0/284036485 learned_addr learned my addr 172.21.15.89:0/284036485 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5212640 10 monclient(hunting): get_auth_request con 0x7f8eb008d320 auth_method 0 2024-08-19T10:38:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5212640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5212640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5212640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 10 monclient: _add_conns ranks=[7,3,1,2,0,4,8,6,5] 2024-08-19T10:38:16.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f661c0a51d0 0x7f661c13cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 10 monclient(hunting): picked mon.noname-i con 0x7f661c0a51d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:16.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0a55b0 0x7f661c13d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 10 monclient(hunting): picked mon.noname-g con 0x7f661c0a55b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c0a72e0 0x7f661c13d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 10 monclient(hunting): picked mon.noname-f con 0x7f661c0a72e0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f661c125060 con 0x7f661c0a72e0 2024-08-19T10:38:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f661c124ee0 con 0x7f661c0a55b0 2024-08-19T10:38:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f661c1251e0 con 0x7f661c0a51d0 2024-08-19T10:38:16.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55c7fff640 1 -- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55c81364f0 msgr2=0x7f55c813aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55c7fff640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55c81364f0 0x7f55c813aa10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009040 tx=0x7f55c0019690 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f6621040640 10 monclient(hunting): authenticate will time out at 946.225095s 2024-08-19T10:38:16.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8eb0070450 0x7f8eb01dde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 10 monclient(hunting): get_auth_request con 0x7f8eb0070450 auth_method 0 2024-08-19T10:38:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6214640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8eb00a6f70 0x7f8eb014d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 1 -- 172.21.15.89:0/8947784 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f55c00ac070 con 0x7f55c81364f0 2024-08-19T10:38:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 10 monclient: dump: 2024-08-19T10:38:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.503 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661ad9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f661c0a51d0 0x7f661c13cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661ad9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f661c0a51d0 0x7f661c13cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:56942/0 (socket says 172.21.15.89:56942) 2024-08-19T10:38:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661ad9d640 1 -- 172.21.15.89:0/3278700546 learned_addr learned my addr 172.21.15.89:0/3278700546 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661ad9d640 10 monclient(hunting): get_auth_request con 0x7f661c0a51d0 auth_method 0 2024-08-19T10:38:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c0a72e0 0x7f661c13d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661a59c640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0a55b0 0x7f661c13d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 10 monclient(hunting): get_auth_request con 0x7f661c0a72e0 auth_method 0 2024-08-19T10:38:16.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661a59c640 10 monclient(hunting): get_auth_request con 0x7f661c0a55b0 auth_method 0 2024-08-19T10:38:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6214640 10 monclient(hunting): get_auth_request con 0x7f8eb00a6f70 auth_method 0 2024-08-19T10:38:16.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6214640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6214640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6214640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5212640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5212640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5212640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6214640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6214640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb6214640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 10 monclient(hunting): handle_auth_done global_id 4270 payload 824 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 1 -- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8eb00a6f70 msgr2=0x7f8eb014d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8eb00a6f70 0x7f8eb014d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 1 -- 172.21.15.89:0/284036485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eb008d320 msgr2=0x7f8eb014d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eb008d320 0x7f8eb014d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 1 monclient: found mon.noname-c 2024-08-19T10:38:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5a13640 1 -- 172.21.15.89:0/284036485 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb00a6710 con 0x7f8eb0070450 2024-08-19T10:38:16.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f8eb5212640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8eb008d320 0x7f8eb014d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 1 -- 172.21.15.89:0/8947784 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c00b1450 con 0x7f55c81364f0 2024-08-19T10:38:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55ce096640 5 monclient: authenticate success, global_id 4238 2024-08-19T10:38:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55ce096640 10 monclient: _renew_subs 2024-08-19T10:38:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55ce096640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55ce096640 1 -- 172.21.15.89:0/8947784 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c813bed0 con 0x7f55c81364f0 2024-08-19T10:38:16.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 1 -- 172.21.15.89:0/8947784 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f55c00b1ab0 con 0x7f55c81364f0 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55c5ffb640 4 set_mon_vals no callback set 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 10 monclient: dump: 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.511 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55ce096640 10 monclient: _renew_subs 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55ce096640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55ce096640 1 -- 172.21.15.89:0/8947784 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c811fee0 con 0x7f55c81364f0 2024-08-19T10:38:16.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55ce096640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55ce096640 1 -- 172.21.15.89:0/8947784 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55c806d440 con 0x7f55c81364f0 2024-08-19T10:38:16.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ffa0d83e640 1 -- 172.21.15.89:0/61584284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a6860 msgr2=0x7ffa08134960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ffa0d83e640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a6860 0x7ffa08134960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa040ae040 tx=0x7ffa040191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55c7fff640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55c81364f0 0x7f55c813aa10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009040 tx=0x7f55c0019690 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f55c00e8f80 osd_map(18..18 src has 1..18) 2024-08-19T10:38:16.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 1 -- 172.21.15.89:0/8947784 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f55c00af070 con 0x7f55c81364f0 2024-08-19T10:38:16.514 INFO:teuthology.orchestra.run.smithi089.stdout:51539607559 2024-08-19T10:38:16.515 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-19T10:38:16.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 10 monclient(hunting): handle_auth_done global_id 4303 payload 824 2024-08-19T10:38:16.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 1 -- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f661c0a51d0 msgr2=0x7f661c13cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f661c0a51d0 0x7f661c13cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 1 -- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0a55b0 msgr2=0x7f661c13d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0a55b0 0x7f661c13d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 1 monclient: found mon.noname-f 2024-08-19T10:38:16.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:38:16.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f661b59e640 1 -- 172.21.15.89:0/3278700546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c13f360 con 0x7f661c0a72e0 2024-08-19T10:38:16.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 1 -- 172.21.15.89:0/61584284 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa040b9070 con 0x7ffa080a6860 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 10 monclient: dump: 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.517 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.518 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.518 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.518 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.518 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.518 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.518 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.518 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ffa0faa2640 5 monclient: authenticate success, global_id 4214 2024-08-19T10:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ffa0faa2640 10 monclient: _renew_subs 2024-08-19T10:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ffa0faa2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ffa0faa2640 1 -- 172.21.15.89:0/61584284 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0809e020 con 0x7ffa080a6860 2024-08-19T10:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ffa0faa2640 10 monclient: _renew_subs 2024-08-19T10:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ffa0faa2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ffa0faa2640 1 -- 172.21.15.89:0/61584284 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa080a2320 con 0x7ffa080a6860 2024-08-19T10:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 1 -- 172.21.15.89:0/61584284 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa040aad10 con 0x7ffa080a6860 2024-08-19T10:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 1 -- 172.21.15.89:0/61584284 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa040b0040 con 0x7ffa080a6860 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fe7fc640 10 monclient: dump: 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.520 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.521 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.521 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.521 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.521 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.521 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.521 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ff9fffff640 4 set_mon_vals no callback set 2024-08-19T10:38:16.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ffa0faa2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:16.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7ffa0faa2640 1 -- 172.21.15.89:0/61584284 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9f0000f90 con 0x7ffa080a6860 2024-08-19T10:38:16.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f55b403be00 0x7f55b403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 1 -- 172.21.15.89:0/8947784 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f55c00e8f80 con 0x7f55c81364f0 2024-08-19T10:38:16.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x7f55b4041c70 0x7f55b40440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55affff640 1 -- 172.21.15.89:0/8947784 --> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55c00c5e50 con 0x7f55b4041c70 2024-08-19T10:38:16.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55c6ffd640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x7f55b4041c70 0x7f55b40440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55c6ffd640 10 monclient: get_auth_request con 0x7f55b4041c70 auth_method 0 2024-08-19T10:38:16.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55c77fe640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f55b403be00 0x7f55b403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.406+0000 7f55c77fe640 10 monclient: get_auth_request con 0x7f55b403be00 auth_method 0 2024-08-19T10:38:16.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb5a13640 1 -- 172.21.15.89:0/284036485 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8eb0070450 msgr2=0x7f8eb01dde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb5a13640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8eb0070450 0x7f8eb01dde40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009480 tx=0x7f8ea8007a90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 1 -- 172.21.15.89:0/284036485 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ea800d670 con 0x7f8eb0070450 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 10 monclient: dump: 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.524 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb6a15640 5 monclient: authenticate success, global_id 4270 2024-08-19T10:38:16.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb6a15640 10 monclient: _renew_subs 2024-08-19T10:38:16.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb6a15640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:16.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb6a15640 1 -- 172.21.15.89:0/284036485 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb0147cb0 con 0x7f8eb0070450 2024-08-19T10:38:16.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb6a15640 10 monclient: _renew_subs 2024-08-19T10:38:16.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb6a15640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:16.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb6a15640 1 -- 172.21.15.89:0/284036485 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb00a6ab0 con 0x7f8eb0070450 2024-08-19T10:38:16.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 1 -- 172.21.15.89:0/284036485 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea801a8f0 con 0x7f8eb0070450 2024-08-19T10:38:16.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7ff9fe7fc640 1 -- 172.21.15.89:0/61584284 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ffa04004040 con 0x7ffa080a6860 2024-08-19T10:38:16.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 1 -- 172.21.15.89:0/284036485 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ea800d830 con 0x7f8eb0070450 2024-08-19T10:38:16.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f661b59e640 1 -- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c0a72e0 msgr2=0x7f661c13d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f661b59e640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c0a72e0 0x7f661c13d630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f660c00b410 tx=0x7f660c00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7ff9fe7fc640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff9ec03be00 0x7ff9ec03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 10 monclient: dump: 2024-08-19T10:38:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.528 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.529 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.530 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea7fff640 4 set_mon_vals no callback set 2024-08-19T10:38:16.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb6a15640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:16.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb6a15640 1 -- 172.21.15.89:0/284036485 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e98000f90 con 0x7f8eb0070450 2024-08-19T10:38:16.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 1 -- 172.21.15.89:0/3278700546 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f660c015020 con 0x7f661c0a72e0 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 10 monclient: dump: 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.531 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 1 -- 172.21.15.89:0/3278700546 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f660c011cb0 con 0x7f661c0a72e0 2024-08-19T10:38:16.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 1 -- 172.21.15.89:0/3278700546 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f660c011e60 con 0x7f661c0a72e0 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f66037fe640 10 monclient: dump: 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.533 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.534 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.534 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.534 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.534 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.534 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.534 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.534 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.534 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7ff9fe7fc640 1 -- 172.21.15.89:0/61584284 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7ffa040e7fa0 con 0x7ffa080a6860 2024-08-19T10:38:16.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7ff9fe7fc640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] conn(0x7ff9ec041c70 0x7ff9ec0440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7ff9fe7fc640 1 -- 172.21.15.89:0/61584284 --> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa040b8ec0 con 0x7ff9ec041c70 2024-08-19T10:38:16.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f661959a640 4 set_mon_vals no callback set 2024-08-19T10:38:16.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7ff9fe7fc640 1 -- 172.21.15.89:0/61584284 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7ffa040e8280 con 0x7ffa080a6860 2024-08-19T10:38:16.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7ff9fe7fc640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-08-19T10:38:16.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7ffa0e03f640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] conn(0x7ff9ec041c70 0x7ff9ec0440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7ffa0e03f640 10 monclient: get_auth_request con 0x7ff9ec041c70 auth_method 0 2024-08-19T10:38:16.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f6621040640 5 monclient: authenticate success, global_id 4303 2024-08-19T10:38:16.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f6621040640 10 monclient: _renew_subs 2024-08-19T10:38:16.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f6621040640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:38:16.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f6621040640 1 -- 172.21.15.89:0/3278700546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661c13f640 con 0x7f661c0a72e0 2024-08-19T10:38:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 1 -- 172.21.15.89:0/284036485 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8ea801a480 con 0x7f8eb0070450 2024-08-19T10:38:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f55c77fe640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f55b403be00 0x7f55b403e2c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009b60 tx=0x7f55bc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f6621040640 10 monclient: _renew_subs 2024-08-19T10:38:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f6621040640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:38:16.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f6621040640 1 -- 172.21.15.89:0/3278700546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661c13f080 con 0x7f661c0a72e0 2024-08-19T10:38:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7ffa0d03d640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff9ec03be00 0x7ff9ec03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7ffa0d03d640 10 monclient: get_auth_request con 0x7ff9ec03be00 auth_method 0 2024-08-19T10:38:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8e9403be00 0x7f8e9403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 1 -- 172.21.15.89:0/284036485 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f8ea804ec30 con 0x7f8eb0070450 2024-08-19T10:38:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] conn(0x7f8e94041cf0 0x7f8e94044130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 1 -- 172.21.15.89:0/284036485 --> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e94044800 con 0x7f8e94041cf0 2024-08-19T10:38:16.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 1 -- 172.21.15.89:0/284036485 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8ea80532a0 con 0x7f8eb0070450 2024-08-19T10:38:16.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8ea67fc640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-08-19T10:38:16.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f6621040640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:38:16.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f6621040640 1 -- 172.21.15.89:0/3278700546 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f661c0a6860 con 0x7f661c0a72e0 2024-08-19T10:38:16.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb5212640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8e9403be00 0x7f8e9403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb5212640 10 monclient: get_auth_request con 0x7f8e9403be00 auth_method 0 2024-08-19T10:38:16.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb6214640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] conn(0x7f8e94041cf0 0x7f8e94044130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.410+0000 7f8eb6214640 10 monclient: get_auth_request con 0x7f8e94041cf0 auth_method 0 2024-08-19T10:38:16.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f66037fe640 1 -- 172.21.15.89:0/3278700546 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f660c018020 con 0x7f661c0a72e0 2024-08-19T10:38:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f66037fe640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f660803be00 0x7f660803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f66037fe640 1 -- 172.21.15.89:0/3278700546 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f660c04edc0 con 0x7f661c0a72e0 2024-08-19T10:38:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f66037fe640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x7f6608041c70 0x7f66080440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f66037fe640 1 -- 172.21.15.89:0/3278700546 --> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f660c004520 con 0x7f6608041c70 2024-08-19T10:38:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f66037fe640 1 -- 172.21.15.89:0/3278700546 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f660c04f0a0 con 0x7f661c0a72e0 2024-08-19T10:38:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f66037fe640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-08-19T10:38:16.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f55c6ffd640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x7f55b4041c70 0x7f55b40440b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f661a59c640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x7f6608041c70 0x7f66080440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f661a59c640 10 monclient: get_auth_request con 0x7f6608041c70 auth_method 0 2024-08-19T10:38:16.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f55affff640 1 -- 172.21.15.89:0/8947784 <== osd.0 v2:172.21.15.89:6816/2476057681 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55c00c5e50 con 0x7f55b4041c70 2024-08-19T10:38:16.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f8eb5212640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8e9403be00 0x7f8e9403e2c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0145fc0 tx=0x7f8eac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f661ad9d640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f660803be00 0x7f660803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f661ad9d640 10 monclient: get_auth_request con 0x7f660803be00 auth_method 0 2024-08-19T10:38:16.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7ffa0d03d640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff9ec03be00 0x7ff9ec03e2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa0809fa10 tx=0x7ff9f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7ffa0e03f640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] conn(0x7ff9ec041c70 0x7ff9ec0440b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.414+0000 7f8eb6214640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] conn(0x7f8e94041cf0 0x7f8e94044130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.418+0000 7ff9fe7fc640 1 -- 172.21.15.89:0/61584284 <== osd.4 v2:172.21.15.190:6818/1174298329 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa040b8ec0 con 0x7ff9ec041c70 2024-08-19T10:38:16.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.418+0000 7f661ad9d640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f660803be00 0x7f660803e2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f661400f5e0 tx=0x7f6614007830 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.418+0000 7f661a59c640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x7f6608041c70 0x7f66080440b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.418+0000 7f66037fe640 1 -- 172.21.15.89:0/3278700546 <== osd.1 v2:172.21.15.89:6800/375502586 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f660c004520 con 0x7f6608041c70 2024-08-19T10:38:16.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.418+0000 7f8ea67fc640 1 -- 172.21.15.89:0/284036485 <== osd.5 v2:172.21.15.190:6802/2434772607 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e94044800 con 0x7f8e94041cf0 2024-08-19T10:38:16.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 1 -- 172.21.15.89:0/61584284 --> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff9f0002c80 con 0x7ff9ec041c70 2024-08-19T10:38:16.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ff9fe7fc640 1 -- 172.21.15.89:0/61584284 <== osd.4 v2:172.21.15.190:6818/1174298329 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff9f0002c80 con 0x7ff9ec041c70 2024-08-19T10:38:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 1 -- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] conn(0x7ff9ec041c70 msgr2=0x7ff9ec0440b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] conn(0x7ff9ec041c70 0x7ff9ec0440b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 1 -- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff9ec03be00 msgr2=0x7ff9ec03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff9ec03be00 0x7ff9ec03e2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa0809fa10 tx=0x7ff9f8008040 comp rx=0 tx=0).stop 2024-08-19T10:38:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 10 monclient: shutdown 2024-08-19T10:38:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 1 -- 172.21.15.89:0/61584284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a6860 msgr2=0x7ffa08134960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 1 --2- 172.21.15.89:0/61584284 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a6860 0x7ffa08134960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa040ae040 tx=0x7ffa040191e0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0e03f640 1 -- 172.21.15.89:0/61584284 reap_dead start 2024-08-19T10:38:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 1 -- 172.21.15.89:0/61584284 shutdown_connections 2024-08-19T10:38:16.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 1 -- 172.21.15.89:0/61584284 >> 172.21.15.89:0/61584284 conn(0x7ffa08074b40 msgr2=0x7ffa080ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 1 -- 172.21.15.89:0/61584284 shutdown_connections 2024-08-19T10:38:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7ffa0faa2640 1 -- 172.21.15.89:0/61584284 wait complete. 2024-08-19T10:38:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.430+0000 7f8eb6a15640 1 -- 172.21.15.89:0/284036485 --> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e98002c80 con 0x7f8e94041cf0 2024-08-19T10:38:16.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8ea67fc640 1 -- 172.21.15.89:0/284036485 <== osd.5 v2:172.21.15.190:6802/2434772607 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8e98002c80 con 0x7f8e94041cf0 2024-08-19T10:38:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 1 -- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] conn(0x7f8e94041cf0 msgr2=0x7f8e94044130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] conn(0x7f8e94041cf0 0x7f8e94044130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 1 -- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8e9403be00 msgr2=0x7f8e9403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8e9403be00 0x7f8e9403e2c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0145fc0 tx=0x7f8eac009040 comp rx=0 tx=0).stop 2024-08-19T10:38:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 10 monclient: shutdown 2024-08-19T10:38:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 1 -- 172.21.15.89:0/284036485 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8eb0070450 msgr2=0x7f8eb01dde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 1 --2- 172.21.15.89:0/284036485 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8eb0070450 0x7f8eb01dde40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009480 tx=0x7f8ea8007a90 comp rx=0 tx=0).stop 2024-08-19T10:38:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6214640 1 -- 172.21.15.89:0/284036485 reap_dead start 2024-08-19T10:38:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 1 -- 172.21.15.89:0/284036485 shutdown_connections 2024-08-19T10:38:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 1 -- 172.21.15.89:0/284036485 >> 172.21.15.89:0/284036485 conn(0x7f8eb00a7f50 msgr2=0x7f8eb00a0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 1 -- 172.21.15.89:0/284036485 shutdown_connections 2024-08-19T10:38:16.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.434+0000 7f8eb6a15640 1 -- 172.21.15.89:0/284036485 wait complete. 2024-08-19T10:38:16.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 1 -- 172.21.15.89:0/8947784 --> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f55c8064590 con 0x7f55b4041c70 2024-08-19T10:38:16.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55affff640 1 -- 172.21.15.89:0/8947784 <== osd.0 v2:172.21.15.89:6816/2476057681 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f55c8064590 con 0x7f55b4041c70 2024-08-19T10:38:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 1 -- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x7f55b4041c70 msgr2=0x7f55b40440b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x7f55b4041c70 0x7f55b40440b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 1 -- 172.21.15.89:0/8947784 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f55b403be00 msgr2=0x7f55b403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f55b403be00 0x7f55b403e2c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009b60 tx=0x7f55bc009190 comp rx=0 tx=0).stop 2024-08-19T10:38:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 10 monclient: shutdown 2024-08-19T10:38:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 20 monclient: shutdown canceling and discarding version request 1 2024-08-19T10:38:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 1 -- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55c81364f0 msgr2=0x7f55c813aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 1 --2- 172.21.15.89:0/8947784 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55c81364f0 0x7f55c813aa10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009040 tx=0x7f55c0019690 comp rx=0 tx=0).stop 2024-08-19T10:38:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55c7fff640 1 -- 172.21.15.89:0/8947784 reap_dead start 2024-08-19T10:38:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55affff640 1 -- 172.21.15.89:0/8947784 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (unknown 0 0 0) 0x7f55c00eb510 con 0x7f55c81364f0 2024-08-19T10:38:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55affff640 10 monclient: discarding stray monitor message mon_get_version_reply(handle=1 version=18) 2024-08-19T10:38:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 1 -- 172.21.15.89:0/8947784 shutdown_connections 2024-08-19T10:38:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 1 -- 172.21.15.89:0/8947784 >> 172.21.15.89:0/8947784 conn(0x7f55c80a8f60 msgr2=0x7f55c813ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 1 -- 172.21.15.89:0/8947784 shutdown_connections 2024-08-19T10:38:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.438+0000 7f55ce096640 1 -- 172.21.15.89:0/8947784 wait complete. 2024-08-19T10:38:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 1 -- 172.21.15.89:0/3278700546 --> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f661c072f10 con 0x7f6608041c70 2024-08-19T10:38:16.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f66037fe640 1 -- 172.21.15.89:0/3278700546 <== osd.1 v2:172.21.15.89:6800/375502586 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f661c072f10 con 0x7f6608041c70 2024-08-19T10:38:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 1 -- 172.21.15.89:0/3278700546 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x7f6608041c70 msgr2=0x7f66080440b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x7f6608041c70 0x7f66080440b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 1 -- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f660803be00 msgr2=0x7f660803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f660803be00 0x7f660803e2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f661400f5e0 tx=0x7f6614007830 comp rx=0 tx=0).stop 2024-08-19T10:38:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 10 monclient: shutdown 2024-08-19T10:38:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 1 -- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c0a72e0 msgr2=0x7f661c13d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 1 --2- 172.21.15.89:0/3278700546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f661c0a72e0 0x7f661c13d630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f660c00b410 tx=0x7f660c00b8d0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f661b59e640 1 -- 172.21.15.89:0/3278700546 reap_dead start 2024-08-19T10:38:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 1 -- 172.21.15.89:0/3278700546 shutdown_connections 2024-08-19T10:38:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 1 -- 172.21.15.89:0/3278700546 >> 172.21.15.89:0/3278700546 conn(0x7f661c074b40 msgr2=0x7f661c0aa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 1 -- 172.21.15.89:0/3278700546 shutdown_connections 2024-08-19T10:38:16.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.442+0000 7f6621040640 1 -- 172.21.15.89:0/3278700546 wait complete. 2024-08-19T10:38:16.558 INFO:teuthology.orchestra.run.smithi089.stdout:51539607556 2024-08-19T10:38:16.558 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-19T10:38:16.559 INFO:teuthology.orchestra.run.smithi089.stdout:51539607557 2024-08-19T10:38:16.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-19T10:38:16.560 INFO:teuthology.orchestra.run.smithi089.stdout:51539607558 2024-08-19T10:38:16.560 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-19T10:38:16.561 INFO:teuthology.orchestra.run.smithi089.stdout:47244640262 2024-08-19T10:38:16.561 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-19T10:38:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient: monmap: 2024-08-19T10:38:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.705124+0000 2024-08-19T10:38:16.704 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.705124+0000 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.705 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.706 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 1 Processor -- start 2024-08-19T10:38:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 1 -- start start 2024-08-19T10:38:16.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient: init 2024-08-19T10:38:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient: _add_conns ranks=[3,8,10,7,5,2,9,0,6,4,1] 2024-08-19T10:38:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecdc0a6860 0x7fecdc0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient(hunting): picked mon.noname-g con 0x7fecdc0a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecdc0a7180 0x7fecdc0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient(hunting): picked mon.noname-e con 0x7fecdc0a7180 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:16.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fecdc139620 0x7fecdc139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient(hunting): picked mon.noname-f con 0x7fecdc139620 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fecdc0a8190 con 0x7fecdc139620 2024-08-19T10:38:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fecdc0a33d0 con 0x7fecdc0a6860 2024-08-19T10:38:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fecdc0a2ec0 con 0x7fecdc0a7180 2024-08-19T10:38:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fece1e00640 10 monclient(hunting): authenticate will time out at 946.521585s 2024-08-19T10:38:16.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecdc0a7180 0x7fecdc0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecdc0a7180 0x7fecdc0a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47728/0 (socket says 172.21.15.89:47728) 2024-08-19T10:38:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 1 -- 172.21.15.89:0/3012847286 learned_addr learned my addr 172.21.15.89:0/3012847286 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 10 monclient(hunting): get_auth_request con 0x7fecdc0a7180 auth_method 0 2024-08-19T10:38:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecdbfff640 1 --2- 172.21.15.89:0/3012847286 >> v2:172.21.15.190:3300/0 conn(0x7fecdc139620 0x7fecdc139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecdbfff640 10 monclient(hunting): get_auth_request con 0x7fecdc139620 auth_method 0 2024-08-19T10:38:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecdbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 10 monclient(hunting): handle_auth_done global_id 4247 payload 274 2024-08-19T10:38:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 1 -- 172.21.15.89:0/3012847286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecdc0a6860 msgr2=0x7fecdc0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecdb7fe640 1 --2- 172.21.15.89:0/3012847286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecdc0a6860 0x7fecdc0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 1 --2- 172.21.15.89:0/3012847286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecdc0a6860 0x7fecdc0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 1 -- 172.21.15.89:0/3012847286 >> v2:172.21.15.190:3300/0 conn(0x7fecdc139620 msgr2=0x7fecdc139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 1 --2- 172.21.15.89:0/3012847286 >> v2:172.21.15.190:3300/0 conn(0x7fecdc139620 0x7fecdc139a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 1 monclient: found mon.noname-e 2024-08-19T10:38:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecd3fff640 1 -- 172.21.15.89:0/3012847286 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc0a4cd0 con 0x7fecdc0a7180 2024-08-19T10:38:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecdbfff640 1 --2- 172.21.15.89:0/3012847286 >> v2:172.21.15.190:3300/0 conn(0x7fecdc139620 0x7fecdc139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:38:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecdb7fe640 10 monclient: get_auth_request con 0x7fecdc0a6860 auth_method 0 2024-08-19T10:38:16.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.702+0000 7fecdb7fe640 1 --2- 172.21.15.89:0/3012847286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecdc0a6860 0x7fecdc0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:16.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecd3fff640 1 -- 172.21.15.89:0/3012847286 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecdc0a7180 msgr2=0x7fecdc0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecd3fff640 1 --2- 172.21.15.89:0/3012847286 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecdc0a7180 0x7fecdc0a7560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009870 tx=0x7fecc401c420 comp rx=0 tx=0).ready entity=mon.8 client_cookie=62904815d299840f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 1 -- 172.21.15.89:0/3012847286 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fecc40b0070 con 0x7fecdc0a7180 2024-08-19T10:38:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 10 monclient: dump: 2024-08-19T10:38:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.714 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.715 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.715 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 1 -- 172.21.15.89:0/3012847286 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc40aa810 con 0x7fecdc0a7180 2024-08-19T10:38:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 1 -- 172.21.15.89:0/3012847286 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fecc40a9300 con 0x7fecdc0a7180 2024-08-19T10:38:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fecdaffd640 10 monclient: dump: 2024-08-19T10:38:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.716 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.717 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.718 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 5 monclient: authenticate success, global_id 4247 2024-08-19T10:38:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 4 set_mon_vals no callback set 2024-08-19T10:38:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 10 monclient: shutdown 2024-08-19T10:38:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 1 -- 172.21.15.89:0/3012847286 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecdc0a7180 msgr2=0x7fecdc0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 1 --2- 172.21.15.89:0/3012847286 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecdc0a7180 0x7fecdc0a7560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009870 tx=0x7fecc401c420 comp rx=0 tx=0).stop 2024-08-19T10:38:16.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 1 -- 172.21.15.89:0/3012847286 shutdown_connections 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 1 --2- 172.21.15.89:0/3012847286 >> v2:172.21.15.190:3300/0 conn(0x7fecdc139620 0x7fecdc139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 1 --2- 172.21.15.89:0/3012847286 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecdc0a7180 0x7fecdc0a7560 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 1 --2- 172.21.15.89:0/3012847286 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecdc0a6860 0x7fecdc0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 1 -- 172.21.15.89:0/3012847286 >> 172.21.15.89:0/3012847286 conn(0x7fecdc074b40 msgr2=0x7fecdc0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 1 -- 172.21.15.89:0/3012847286 shutdown_connections 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.706+0000 7fece1e00640 1 -- 172.21.15.89:0/3012847286 wait complete. 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient: monmap: 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.719 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.720 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.721 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient: monmap: 2024-08-19T10:38:16.721 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.721 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.713304+0000 2024-08-19T10:38:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.713304+0000 2024-08-19T10:38:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.722 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.723 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.723 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.723 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.723 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.723 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.723 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 1 Processor -- start 2024-08-19T10:38:16.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 1 -- start start 2024-08-19T10:38:16.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient: init 2024-08-19T10:38:16.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 1 Processor -- start 2024-08-19T10:38:16.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 1 -- start start 2024-08-19T10:38:16.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient: init 2024-08-19T10:38:16.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient: _add_conns ranks=[1,0,4,7,6,2,3,8,5] 2024-08-19T10:38:16.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecdc0a6860 0x7fecdc09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient(hunting): picked mon.noname-f con 0x7fecdc0a6860 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:16.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecdc0a7180 0x7fecdc09cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient(hunting): picked mon.noname-a con 0x7fecdc0a7180 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:16.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient: _add_conns ranks=[2,7,5,6,3,1,8,4,0,9,10] 2024-08-19T10:38:16.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8dac09b8e0 0x7f8dac09bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecd3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecdc0a7180 0x7fecdc09cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecdc139620 0x7fecdc09d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecd3fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecdc0a7180 0x7fecdc09cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57728/0 (socket says 172.21.15.89:57728) 2024-08-19T10:38:16.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecdc0a6860 0x7fecdc09c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8dac09b8e0 0x7f8dac09bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecdc139620 0x7fecdc09d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecdc139620 0x7fecdc09d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37754/0 (socket says 172.21.15.89:37754) 2024-08-19T10:38:16.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8dac09b8e0 0x7f8dac09bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44410/0 (socket says 172.21.15.89:44410) 2024-08-19T10:38:16.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 1 -- 172.21.15.89:0/3980721476 learned_addr learned my addr 172.21.15.89:0/3980721476 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient(hunting): picked mon.noname-b con 0x7f8dac09b8e0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:16.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 1 --2- 172.21.15.89:0/3980721476 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8dac09ddd0 0x7f8dac0a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient(hunting): picked mon.noname-i con 0x7f8dac09ddd0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:16.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 1 --2- 172.21.15.89:0/3980721476 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8dac0a4240 0x7f8dac0a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient(hunting): picked mon.noname-h con 0x7f8dac0a4240 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:16.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient(hunting): picked mon.noname-c con 0x7fecdc139620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:16.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecd3fff640 1 -- 172.21.15.89:0/1613298768 learned_addr learned my addr 172.21.15.89:0/1613298768 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 1 -- 172.21.15.89:0/1613298768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc143210 con 0x7fecdc0a7180 2024-08-19T10:38:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 1 -- 172.21.15.89:0/1613298768 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc143090 con 0x7fecdc0a6860 2024-08-19T10:38:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 1 -- 172.21.15.89:0/1613298768 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fecdc143390 con 0x7fecdc139620 2024-08-19T10:38:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fece1e00640 10 monclient(hunting): authenticate will time out at 946.530541s 2024-08-19T10:38:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdb7fe640 10 monclient(hunting): get_auth_request con 0x7fecdc0a6860 auth_method 0 2024-08-19T10:38:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecd3fff640 10 monclient(hunting): get_auth_request con 0x7fecdc0a7180 auth_method 0 2024-08-19T10:38:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecd3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecd3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecd3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 10 monclient(hunting): get_auth_request con 0x7fecdc139620 auth_method 0 2024-08-19T10:38:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 1 -- 172.21.15.89:0/3980721476 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8dac09d760 con 0x7f8dac09b8e0 2024-08-19T10:38:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 1 -- 172.21.15.89:0/3980721476 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f8dac09d9b0 con 0x7f8dac0a4240 2024-08-19T10:38:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 1 -- 172.21.15.89:0/3980721476 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8dac06fd00 con 0x7f8dac09ddd0 2024-08-19T10:38:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db2e69640 10 monclient(hunting): authenticate will time out at 946.530523s 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 10 monclient(hunting): get_auth_request con 0x7f8dac09b8e0 auth_method 0 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8dabfff640 1 --2- 172.21.15.89:0/3980721476 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8dac09ddd0 0x7f8dac0a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8dabfff640 10 monclient(hunting): get_auth_request con 0x7f8dac09ddd0 auth_method 0 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8dabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8dabfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8dabfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db1406640 1 --2- 172.21.15.89:0/3980721476 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8dac0a4240 0x7f8dac0a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecd3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecd3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecd3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 10 monclient(hunting): handle_auth_done global_id 4279 payload 824 2024-08-19T10:38:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 1 -- 172.21.15.89:0/1613298768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecdc0a6860 msgr2=0x7fecdc09c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecdc0a6860 0x7fecdc09c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 1 -- 172.21.15.89:0/1613298768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecdc0a7180 msgr2=0x7fecdc09cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecdc0a7180 0x7fecdc09cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 1 monclient: found mon.noname-c 2024-08-19T10:38:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:16.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7fecdbfff640 1 -- 172.21.15.89:0/1613298768 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc13f8b0 con 0x7fecdc139620 2024-08-19T10:38:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db1406640 10 monclient(hunting): get_auth_request con 0x7f8dac0a4240 auth_method 0 2024-08-19T10:38:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db1406640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db1406640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db1406640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 10 monclient(hunting): handle_auth_done global_id 4223 payload 274 2024-08-19T10:38:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 1 -- 172.21.15.89:0/3980721476 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8dac09ddd0 msgr2=0x7f8dac0a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 1 --2- 172.21.15.89:0/3980721476 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8dac09ddd0 0x7f8dac0a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 1 -- 172.21.15.89:0/3980721476 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8dac0a4240 msgr2=0x7f8dac0a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 1 --2- 172.21.15.89:0/3980721476 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8dac0a4240 0x7f8dac0a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 1 monclient: found mon.noname-b 2024-08-19T10:38:16.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:16.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.710+0000 7f8db0c05640 1 -- 172.21.15.89:0/3980721476 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac06db50 con 0x7f8dac09b8e0 2024-08-19T10:38:16.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecdbfff640 1 -- 172.21.15.89:0/1613298768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecdc139620 msgr2=0x7fecdc09d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecdbfff640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecdc139620 0x7fecdc09d460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fecd4007db0 tx=0x7fecd400d4a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 1 -- 172.21.15.89:0/1613298768 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fecd401a070 con 0x7fecdc139620 2024-08-19T10:38:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 10 monclient: dump: 2024-08-19T10:38:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.739 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fece1e00640 5 monclient: authenticate success, global_id 4279 2024-08-19T10:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fece1e00640 10 monclient: _renew_subs 2024-08-19T10:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fece1e00640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fece1e00640 1 -- 172.21.15.89:0/1613298768 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecdc09eda0 con 0x7fecdc139620 2024-08-19T10:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 1 -- 172.21.15.89:0/1613298768 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd40039a0 con 0x7fecdc139620 2024-08-19T10:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fece1e00640 10 monclient: _renew_subs 2024-08-19T10:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fece1e00640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fece1e00640 1 -- 172.21.15.89:0/1613298768 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fecdc0a94d0 con 0x7fecdc139620 2024-08-19T10:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 1 -- 172.21.15.89:0/1613298768 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fecd4004030 con 0x7fecdc139620 2024-08-19T10:38:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 10 monclient: dump: 2024-08-19T10:38:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.742 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.743 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.743 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.743 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.743 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.743 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.743 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.743 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.743 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecda7fc640 4 set_mon_vals no callback set 2024-08-19T10:38:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f4e8aaf1640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f4e8aaf1640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f4e8aaf1640 10 monclient: monmap: 2024-08-19T10:38:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.718848+0000 2024-08-19T10:38:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.718848+0000 2024-08-19T10:38:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.744 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.745 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.745 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd1ffb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd1ffb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd1ffb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:16.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd1ffb640 1 -- 172.21.15.89:0/1613298768 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecdc0a6c40 con 0x7fecdc139620 2024-08-19T10:38:16.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f8db0c05640 1 -- 172.21.15.89:0/3980721476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8dac09b8e0 msgr2=0x7f8dac09bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f8db0c05640 1 --2- 172.21.15.89:0/3980721476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8dac09b8e0 0x7f8dac09bcc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8da00098a0 tx=0x7f8da001c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35d96a95d264cf4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f8dab7fe640 1 -- 172.21.15.89:0/3980721476 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8da00b0070 con 0x7f8dac09b8e0 2024-08-19T10:38:16.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f8dab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f8dab7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:16.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f8dab7fe640 10 monclient: dump: 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.748 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.749 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.749 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.749 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f8dab7fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f8dab7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f8dab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f8db2e69640 5 monclient: authenticate success, global_id 4223 2024-08-19T10:38:16.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7f8db2e69640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:16.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 1 -- 172.21.15.89:0/1613298768 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fecd4003b50 con 0x7fecdc139620 2024-08-19T10:38:16.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fecbc03be00 0x7fecbc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.714+0000 7fecd8ff9640 1 -- 172.21.15.89:0/1613298768 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7fecd404e960 con 0x7fecdc139620 2024-08-19T10:38:16.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8dab7fe640 1 -- 172.21.15.89:0/3980721476 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da00aa3a0 con 0x7f8dac09b8e0 2024-08-19T10:38:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8dab7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8dab7fe640 1 -- 172.21.15.89:0/3980721476 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8da00aaa00 con 0x7f8dac09b8e0 2024-08-19T10:38:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8dab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8dab7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8dab7fe640 10 monclient: dump: 2024-08-19T10:38:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.751 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.752 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.752 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8db2e69640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8db2e69640 4 set_mon_vals no callback set 2024-08-19T10:38:16.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7fecdb7fe640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fecbc03be00 0x7fecbc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7fecdb7fe640 10 monclient: get_auth_request con 0x7fecbc03be00 auth_method 0 2024-08-19T10:38:16.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8db2e69640 10 monclient: shutdown 2024-08-19T10:38:16.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8db2e69640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8db2e69640 1 -- 172.21.15.89:0/3980721476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8dac09b8e0 msgr2=0x7f8dac09bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7f8db2e69640 1 --2- 172.21.15.89:0/3980721476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8dac09b8e0 0x7f8dac09bcc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8da00098a0 tx=0x7f8da001c460 comp rx=0 tx=0).stop 2024-08-19T10:38:16.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.718+0000 7fecdb7fe640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fecbc03be00 0x7fecbc03e2c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feccc00a9a0 tx=0x7feccc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 -- 172.21.15.89:0/3980721476 shutdown_connections 2024-08-19T10:38:16.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 --2- 172.21.15.89:0/3980721476 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8dac0a4240 0x7f8dac0a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 --2- 172.21.15.89:0/3980721476 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8dac09ddd0 0x7f8dac0a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 --2- 172.21.15.89:0/3980721476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8dac09b8e0 0x7f8dac09bcc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 -- 172.21.15.89:0/3980721476 >> 172.21.15.89:0/3980721476 conn(0x7f8dac0a4df0 msgr2=0x7f8dac0a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 -- 172.21.15.89:0/3980721476 shutdown_connections 2024-08-19T10:38:16.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 -- 172.21.15.89:0/3980721476 wait complete. 2024-08-19T10:38:16.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7fecd8ff9640 1 -- 172.21.15.89:0/1613298768 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fecd4010070 con 0x7fecdc139620 2024-08-19T10:38:16.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7fecd8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7fecd8ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:16.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient: monmap: 2024-08-19T10:38:16.758 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.758 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.758 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.759 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.760 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 Processor -- start 2024-08-19T10:38:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 -- start start 2024-08-19T10:38:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient: init 2024-08-19T10:38:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient: _add_conns ranks=[8,1,3,6,5,2,4,7,0] 2024-08-19T10:38:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8dac09b8e0 0x7f8dac0934b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient(hunting): picked mon.noname-e con 0x7f8dac09b8e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:16.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac09ddd0 0x7f8dac0939f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient(hunting): picked mon.noname-f con 0x7f8dac09ddd0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0a4240 0x7f8dac093f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient(hunting): picked mon.noname-g con 0x7f8dac0a4240 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac069300 con 0x7f8dac09ddd0 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8dac0a5c00 con 0x7f8dac0a4240 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f8dac142cd0 con 0x7f8dac09b8e0 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db2e69640 10 monclient(hunting): authenticate will time out at 946.542673s 2024-08-19T10:38:16.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8dac09b8e0 0x7f8dac0934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8dac09b8e0 0x7f8dac0934b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47742/0 (socket says 172.21.15.89:47742) 2024-08-19T10:38:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 1 -- 172.21.15.89:0/852176609 learned_addr learned my addr 172.21.15.89:0/852176609 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 10 monclient(hunting): get_auth_request con 0x7f8dac09b8e0 auth_method 0 2024-08-19T10:38:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8dabfff640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac09ddd0 0x7f8dac0939f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db1406640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0a4240 0x7f8dac093f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 10 monclient(hunting): handle_auth_done global_id 4256 payload 824 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 1 -- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0a4240 msgr2=0x7f8dac093f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0a4240 0x7f8dac093f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 1 -- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac09ddd0 msgr2=0x7f8dac0939f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac09ddd0 0x7f8dac0939f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 1 monclient: found mon.noname-e 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db0c05640 1 -- 172.21.15.89:0/852176609 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac096e90 con 0x7f8dac09b8e0 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db1406640 10 monclient: get_auth_request con 0x7f8dac0a4240 auth_method 0 2024-08-19T10:38:16.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8db1406640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0a4240 0x7f8dac093f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:16.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f4e8aaf1640 1 Processor -- start 2024-08-19T10:38:16.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8dabfff640 10 monclient: get_auth_request con 0x7f8dac09ddd0 auth_method 0 2024-08-19T10:38:16.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f8dabfff640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac09ddd0 0x7f8dac0939f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f4e8aaf1640 1 -- start start 2024-08-19T10:38:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.722+0000 7f4e8aaf1640 10 monclient: init 2024-08-19T10:38:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 10 monclient: _add_conns ranks=[2,4,9,8,5,7,0,1,3,6,10] 2024-08-19T10:38:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0242f0 0x7f4e7c0bce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 10 monclient(hunting): picked mon.noname-b con 0x7f4e7c0242f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e7c0246d0 0x7f4e7c01e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 10 monclient(hunting): picked mon.noname-c con 0x7f4e7c0246d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4e7c024b10 0x7f4e7c01edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 10 monclient(hunting): picked mon.noname-a con 0x7f4e7c024b10 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7c0bd380 con 0x7f4e7c024b10 2024-08-19T10:38:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c020150 con 0x7f4e7c0242f0 2024-08-19T10:38:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f4e7c0210e0 con 0x7f4e7c0246d0 2024-08-19T10:38:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8aaf1640 10 monclient(hunting): authenticate will time out at 946.544228s 2024-08-19T10:38:16.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0242f0 0x7f4e7c0bce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0242f0 0x7f4e7c0bce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44418/0 (socket says 172.21.15.89:44418) 2024-08-19T10:38:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 1 -- 172.21.15.89:0/1716917463 learned_addr learned my addr 172.21.15.89:0/1716917463 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 10 monclient(hunting): get_auth_request con 0x7f4e7c0242f0 auth_method 0 2024-08-19T10:38:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8a2f0640 1 --2- 172.21.15.89:0/1716917463 >> v2:172.21.15.89:3300/0 conn(0x7f4e7c024b10 0x7f4e7c01edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8a2f0640 10 monclient(hunting): get_auth_request con 0x7f4e7c024b10 auth_method 0 2024-08-19T10:38:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8a2f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8a2f0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8a2f0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8a2f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8a2f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8a2f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e892ee640 1 --2- 172.21.15.89:0/1716917463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e7c0246d0 0x7f4e7c01e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e892ee640 10 monclient(hunting): get_auth_request con 0x7f4e7c0246d0 auth_method 0 2024-08-19T10:38:16.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e892ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e892ee640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e892ee640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 10 monclient(hunting): handle_auth_done global_id 4232 payload 274 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 1 -- 172.21.15.89:0/1716917463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e7c0246d0 msgr2=0x7f4e7c01e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 1 --2- 172.21.15.89:0/1716917463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e7c0246d0 0x7f4e7c01e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 1 -- 172.21.15.89:0/1716917463 >> v2:172.21.15.89:3300/0 conn(0x7f4e7c024b10 msgr2=0x7f4e7c01edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 1 --2- 172.21.15.89:0/1716917463 >> v2:172.21.15.89:3300/0 conn(0x7f4e7c024b10 0x7f4e7c01edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 1 monclient: found mon.noname-b 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:16.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e89aef640 1 -- 172.21.15.89:0/1716917463 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c0213c0 con 0x7f4e7c0242f0 2024-08-19T10:38:16.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f4e8a2f0640 1 --2- 172.21.15.89:0/1716917463 >> v2:172.21.15.89:3300/0 conn(0x7f4e7c024b10 0x7f4e7c01edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:16.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db0c05640 1 -- 172.21.15.89:0/852176609 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8dac09b8e0 msgr2=0x7f8dac0934b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db0c05640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8dac09b8e0 0x7f8dac0934b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8da00ae040 tx=0x7f8da0019200 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 1 -- 172.21.15.89:0/852176609 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8da00b9070 con 0x7f8dac09b8e0 2024-08-19T10:38:16.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:16.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 10 monclient: dump: 2024-08-19T10:38:16.773 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.774 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.775 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.775 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.775 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 1 -- 172.21.15.89:0/852176609 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da0003ca0 con 0x7f8dac09b8e0 2024-08-19T10:38:16.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 1 -- 172.21.15.89:0/852176609 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8da00b0040 con 0x7f8dac09b8e0 2024-08-19T10:38:16.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:16.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 10 monclient: dump: 2024-08-19T10:38:16.776 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.776 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.776 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.776 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.776 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.776 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.776 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.776 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.776 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.777 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db2e69640 5 monclient: authenticate success, global_id 4256 2024-08-19T10:38:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db2e69640 10 monclient: _renew_subs 2024-08-19T10:38:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db2e69640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db2e69640 1 -- 172.21.15.89:0/852176609 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dac097170 con 0x7f8dac09b8e0 2024-08-19T10:38:16.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db2e69640 10 monclient: _renew_subs 2024-08-19T10:38:16.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db2e69640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db2e69640 1 -- 172.21.15.89:0/852176609 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dac096bb0 con 0x7f8dac09b8e0 2024-08-19T10:38:16.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8daaffd640 4 set_mon_vals no callback set 2024-08-19T10:38:16.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db2e69640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 1 -- 172.21.15.89:0/852176609 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8da0003830 con 0x7f8dac09b8e0 2024-08-19T10:38:16.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db2e69640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db2e69640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8db2e69640 1 -- 172.21.15.89:0/852176609 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d8c005090 con 0x7f8dac09b8e0 2024-08-19T10:38:16.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.726+0000 7f8da97fa640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8d9003be00 0x7f8d9003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f8da97fa640 1 -- 172.21.15.89:0/852176609 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f8da00e7a40 con 0x7f8dac09b8e0 2024-08-19T10:38:16.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f8dabfff640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8d9003be00 0x7f8d9003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f8dabfff640 10 monclient: get_auth_request con 0x7f8d9003be00 auth_method 0 2024-08-19T10:38:16.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e89aef640 1 -- 172.21.15.89:0/1716917463 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0242f0 msgr2=0x7f4e7c0bce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e89aef640 1 --2- 172.21.15.89:0/1716917463 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0242f0 0x7f4e7c0bce40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4e740098a0 tx=0x7f4e7401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfc118cfdd8701b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e88aed640 1 -- 172.21.15.89:0/1716917463 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e740b0070 con 0x7f4e7c0242f0 2024-08-19T10:38:16.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e88aed640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e88aed640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:16.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e88aed640 10 monclient: dump: 2024-08-19T10:38:16.781 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.781 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.781 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.781 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.781 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.781 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.782 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.782 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.782 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.782 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.782 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.782 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.782 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.782 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.782 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.782 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e88aed640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e88aed640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e88aed640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 5 monclient: authenticate success, global_id 4232 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e88aed640 1 -- 172.21.15.89:0/1716917463 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e740aa810 con 0x7f4e7c0242f0 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e88aed640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 4 set_mon_vals no callback set 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 10 monclient: shutdown 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/1716917463 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0242f0 msgr2=0x7f4e7c0bce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 1 --2- 172.21.15.89:0/1716917463 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0242f0 0x7f4e7c0bce40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4e740098a0 tx=0x7f4e7401c400 comp rx=0 tx=0).stop 2024-08-19T10:38:16.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/1716917463 shutdown_connections 2024-08-19T10:38:16.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 1 --2- 172.21.15.89:0/1716917463 >> v2:172.21.15.89:3300/0 conn(0x7f4e7c024b10 0x7f4e7c01edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 1 --2- 172.21.15.89:0/1716917463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e7c0246d0 0x7f4e7c01e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 1 --2- 172.21.15.89:0/1716917463 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e7c0242f0 0x7f4e7c0bce40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/1716917463 >> 172.21.15.89:0/1716917463 conn(0x7f4e7c007780 msgr2=0x7f4e7c007b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/1716917463 shutdown_connections 2024-08-19T10:38:16.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/1716917463 wait complete. 2024-08-19T10:38:16.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.730+0000 7f4e8aaf1640 10 monclient: monmap: 2024-08-19T10:38:16.784 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.785 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.785 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.785 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.785 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.785 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.785 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.785 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.785 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.785 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.785 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.785 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.786 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.786 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.786 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.786 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 1 Processor -- start 2024-08-19T10:38:16.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f8dabfff640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8d9003be00 0x7f8d9003e2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8dac094c10 tx=0x7f8d98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 1 -- start start 2024-08-19T10:38:16.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 10 monclient: init 2024-08-19T10:38:16.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 10 monclient: _add_conns ranks=[7,6,8,3,4,1,2,0,5] 2024-08-19T10:38:16.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e7c0242f0 0x7f4e7c0b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 10 monclient(hunting): picked mon.noname-i con 0x7f4e7c0242f0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4e7c0246d0 0x7f4e7c0b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 10 monclient(hunting): picked mon.noname-d con 0x7f4e7c0246d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e7c024b10 0x7f4e7c0c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 10 monclient(hunting): picked mon.noname-e con 0x7f4e7c024b10 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4e7c0c68c0 con 0x7f4e7c0246d0 2024-08-19T10:38:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4e7c0c6740 con 0x7f4e7c0242f0 2024-08-19T10:38:16.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4e7c0c6a40 con 0x7f4e7c024b10 2024-08-19T10:38:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8aaf1640 10 monclient(hunting): authenticate will time out at 946.553257s 2024-08-19T10:38:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e892ee640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4e7c0246d0 0x7f4e7c0b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e892ee640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4e7c0246d0 0x7f4e7c0b60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46464/0 (socket says 172.21.15.89:46464) 2024-08-19T10:38:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e892ee640 1 -- 172.21.15.89:0/2794270065 learned_addr learned my addr 172.21.15.89:0/2794270065 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e892ee640 10 monclient(hunting): get_auth_request con 0x7f4e7c0246d0 auth_method 0 2024-08-19T10:38:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e892ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e892ee640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e892ee640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e7c024b10 0x7f4e7c0c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 10 monclient(hunting): get_auth_request con 0x7f4e7c024b10 auth_method 0 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e892ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e892ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e892ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e89aef640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e7c0242f0 0x7f4e7c0b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e89aef640 10 monclient(hunting): get_auth_request con 0x7f4e7c0242f0 auth_method 0 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e89aef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e89aef640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e89aef640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e89aef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e89aef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e89aef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 10 monclient(hunting): handle_auth_done global_id 4265 payload 824 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 1 -- 172.21.15.89:0/2794270065 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e7c0242f0 msgr2=0x7f4e7c0b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e7c0242f0 0x7f4e7c0b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 1 -- 172.21.15.89:0/2794270065 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4e7c0246d0 msgr2=0x7f4e7c0b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4e7c0246d0 0x7f4e7c0b60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 1 monclient: found mon.noname-e 2024-08-19T10:38:16.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e8a2f0640 1 -- 172.21.15.89:0/2794270065 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c0c3a60 con 0x7f4e7c024b10 2024-08-19T10:38:16.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f4e892ee640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4e7c0246d0 0x7f4e7c0b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:16.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f8da97fa640 1 -- 172.21.15.89:0/852176609 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8da0016170 con 0x7f8dac09b8e0 2024-08-19T10:38:16.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f8da97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.734+0000 7f8da97fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:16.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e8a2f0640 1 -- 172.21.15.89:0/2794270065 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e7c024b10 msgr2=0x7f4e7c0c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e8a2f0640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e7c024b10 0x7f4e7c0c07a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4e8406c5d0 tx=0x7f4e8406dd10 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 1 -- 172.21.15.89:0/2794270065 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e8409b070 con 0x7f4e7c024b10 2024-08-19T10:38:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 10 monclient: dump: 2024-08-19T10:38:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.795 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.796 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.796 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.796 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.796 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.796 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.796 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.796 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.796 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 1 -- 172.21.15.89:0/2794270065 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e84095800 con 0x7f4e7c024b10 2024-08-19T10:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e8aaf1640 5 monclient: authenticate success, global_id 4265 2024-08-19T10:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e8aaf1640 10 monclient: _renew_subs 2024-08-19T10:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e8aaf1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/2794270065 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c0b6e00 con 0x7f4e7c024b10 2024-08-19T10:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 1 -- 172.21.15.89:0/2794270065 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e840959b0 con 0x7f4e7c024b10 2024-08-19T10:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7bfff640 4 set_mon_vals no callback set 2024-08-19T10:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 10 monclient: dump: 2024-08-19T10:38:16.798 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.798 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.798 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.798 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.798 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.798 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.798 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.798 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.798 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.798 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.798 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e8aaf1640 10 monclient: _renew_subs 2024-08-19T10:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e8aaf1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/2794270065 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c0a1a00 con 0x7f4e7c024b10 2024-08-19T10:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 1 -- 172.21.15.89:0/2794270065 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4e84095390 con 0x7f4e7c024b10 2024-08-19T10:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e6803be30 0x7f4e6803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.738+0000 7f4e7a7fc640 1 -- 172.21.15.89:0/2794270065 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f4e840cc8e0 con 0x7f4e7c024b10 2024-08-19T10:38:16.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.742+0000 7f4e89aef640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e6803be30 0x7f4e6803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.742+0000 7f4e89aef640 10 monclient: get_auth_request con 0x7f4e6803be30 auth_method 0 2024-08-19T10:38:16.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.742+0000 7f4e8aaf1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.742+0000 7f4e8aaf1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.742+0000 7f4e8aaf1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.742+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/2794270065 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7c0bcdd0 con 0x7f4e7c024b10 2024-08-19T10:38:16.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.742+0000 7f4e89aef640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e6803be30 0x7f4e6803e2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0084a0 tx=0x7f4e740a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f4e7a7fc640 1 -- 172.21.15.89:0/2794270065 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4e84069e50 con 0x7f4e7c024b10 2024-08-19T10:38:16.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f4e7a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f4e7a7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:16.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient: monmap: 2024-08-19T10:38:16.806 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.806 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.806 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.750618+0000 2024-08-19T10:38:16.806 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.750618+0000 2024-08-19T10:38:16.806 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.806 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.806 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.806 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.807 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 1 Processor -- start 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 1 -- start start 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient: init 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient: _add_conns ranks=[2,9,3,8,7,10,5,1,4,6,0] 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39a80a6860 0x7f39a80a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient(hunting): picked mon.noname-b con 0x7f39a80a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f39a80a7180 0x7f39a80a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient(hunting): picked mon.noname-a con 0x7f39a80a7180 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a8139620 0x7f39a8139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient(hunting): picked mon.noname-g con 0x7f39a8139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f39a80a8190 con 0x7f39a80a7180 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f39a80a33d0 con 0x7f39a80a6860 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f39a80a2ec0 con 0x7f39a8139620 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39acb06640 10 monclient(hunting): authenticate will time out at 946.567606s 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.746+0000 7f39a659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39a80a6860 0x7f39a80a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a6d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a8139620 0x7f39a8139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f39a80a7180 0x7f39a80a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39a80a6860 0x7f39a80a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44432/0 (socket says 172.21.15.89:44432) 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a659c640 1 -- 172.21.15.89:0/745475152 learned_addr learned my addr 172.21.15.89:0/745475152 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a659c640 10 monclient(hunting): get_auth_request con 0x7f39a80a6860 auth_method 0 2024-08-19T10:38:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a6d9d640 10 monclient(hunting): get_auth_request con 0x7f39a8139620 auth_method 0 2024-08-19T10:38:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 10 monclient(hunting): get_auth_request con 0x7f39a80a7180 auth_method 0 2024-08-19T10:38:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 10 monclient(hunting): handle_auth_done global_id 4365 payload 274 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 1 -- 172.21.15.89:0/745475152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a8139620 msgr2=0x7f39a8139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 1 --2- 172.21.15.89:0/745475152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a8139620 0x7f39a8139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 1 -- 172.21.15.89:0/745475152 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39a80a6860 msgr2=0x7f39a80a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 1 --2- 172.21.15.89:0/745475152 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39a80a6860 0x7f39a80a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 1 monclient: found mon.noname-a 2024-08-19T10:38:16.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:16.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f399ffff640 1 -- 172.21.15.89:0/745475152 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a80a4cd0 con 0x7f39a80a7180 2024-08-19T10:38:16.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.750+0000 7f39a659c640 1 --2- 172.21.15.89:0/745475152 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39a80a6860 0x7f39a80a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:16.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f399ffff640 1 -- 172.21.15.89:0/745475152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a7180 msgr2=0x7f39a80a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f399ffff640 1 --2- 172.21.15.89:0/745475152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a7180 0x7f39a80a7560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39900097c0 tx=0x7f399001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6472774f1c25991d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39a5d9b640 1 -- 172.21.15.89:0/745475152 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39900b0070 con 0x7f39a80a7180 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39a5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39a5d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39a5d9b640 10 monclient: dump: 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39a5d9b640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39a5d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39a5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 5 monclient: authenticate success, global_id 4365 2024-08-19T10:38:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39a5d9b640 1 -- 172.21.15.89:0/745475152 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39900aa810 con 0x7f39a80a7180 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39a5d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 4 set_mon_vals no callback set 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 10 monclient: shutdown 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 1 -- 172.21.15.89:0/745475152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a7180 msgr2=0x7f39a80a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 1 --2- 172.21.15.89:0/745475152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a7180 0x7f39a80a7560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39900097c0 tx=0x7f399001c360 comp rx=0 tx=0).stop 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39a5d9b640 1 -- 172.21.15.89:0/745475152 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39900aae30 con 0x7f39a80a7180 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39a5d9b640 10 monclient: discarding stray monitor message mon_map magic: 0 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 1 -- 172.21.15.89:0/745475152 shutdown_connections 2024-08-19T10:38:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 1 --2- 172.21.15.89:0/745475152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a7180 0x7f39a80a7560 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 1 --2- 172.21.15.89:0/745475152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a8139620 0x7f39a8139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 1 --2- 172.21.15.89:0/745475152 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39a80a6860 0x7f39a80a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 1 -- 172.21.15.89:0/745475152 >> 172.21.15.89:0/745475152 conn(0x7f39a8074b40 msgr2=0x7f39a80abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 1 -- 172.21.15.89:0/745475152 shutdown_connections 2024-08-19T10:38:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.754+0000 7f39acb06640 1 -- 172.21.15.89:0/745475152 wait complete. 2024-08-19T10:38:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient: monmap: 2024-08-19T10:38:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 1 Processor -- start 2024-08-19T10:38:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 1 -- start start 2024-08-19T10:38:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient: init 2024-08-19T10:38:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient: _add_conns ranks=[5,3,0,2,1,6,4,7,8] 2024-08-19T10:38:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39a80a6860 0x7f39a81d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient(hunting): picked mon.noname-h con 0x7f39a80a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a8139620 0x7f39a80a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient(hunting): picked mon.noname-g con 0x7f39a8139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a2530 0x7f39a813dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient(hunting): picked mon.noname-a con 0x7f39a80a2530 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8143210 con 0x7f39a80a2530 2024-08-19T10:38:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f39a8143090 con 0x7f39a8139620 2024-08-19T10:38:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f39a8143390 con 0x7f39a80a6860 2024-08-19T10:38:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39acb06640 10 monclient(hunting): authenticate will time out at 946.577721s 2024-08-19T10:38:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a2530 0x7f39a813dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a2530 0x7f39a813dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57760/0 (socket says 172.21.15.89:57760) 2024-08-19T10:38:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 1 -- 172.21.15.89:0/174529304 learned_addr learned my addr 172.21.15.89:0/174529304 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 10 monclient(hunting): get_auth_request con 0x7f39a80a2530 auth_method 0 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a659c640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39a80a6860 0x7f39a81d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f399ffff640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a8139620 0x7f39a80a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a659c640 10 monclient(hunting): get_auth_request con 0x7f39a80a6860 auth_method 0 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f399ffff640 10 monclient(hunting): get_auth_request con 0x7f39a8139620 auth_method 0 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f399ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f399ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f399ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f399ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f399ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f399ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 10 monclient(hunting): handle_auth_done global_id 4374 payload 824 2024-08-19T10:38:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 1 -- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39a80a6860 msgr2=0x7f39a81d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39a80a6860 0x7f39a81d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 1 -- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a8139620 msgr2=0x7f39a80a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a8139620 0x7f39a80a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 1 monclient: found mon.noname-a 2024-08-19T10:38:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.758+0000 7f39a6d9d640 1 -- 172.21.15.89:0/174529304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a80a2970 con 0x7f39a80a2530 2024-08-19T10:38:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f39a6d9d640 1 -- 172.21.15.89:0/174529304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a2530 msgr2=0x7f39a813dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f39a6d9d640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a2530 0x7f39a813dab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39a000efc0 tx=0x7f39a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient: monmap: 2024-08-19T10:38:16.823 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.823 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.823 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.765900+0000 2024-08-19T10:38:16.823 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.765900+0000 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.825 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f399effd640 1 -- 172.21.15.89:0/174529304 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39a0018020 con 0x7f39a80a2530 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f399effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f399effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f399effd640 10 monclient: dump: 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f399effd640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f399effd640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f399effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f39acb06640 5 monclient: authenticate success, global_id 4374 2024-08-19T10:38:16.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f39acb06640 10 monclient: _renew_subs 2024-08-19T10:38:16.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f39acb06640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:16.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f39acb06640 1 -- 172.21.15.89:0/174529304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a80ac270 con 0x7f39a80a2530 2024-08-19T10:38:16.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f39acb06640 10 monclient: _renew_subs 2024-08-19T10:38:16.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f39acb06640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:16.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f39acb06640 1 -- 172.21.15.89:0/174529304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a813f570 con 0x7f39a80a2530 2024-08-19T10:38:16.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f399effd640 1 -- 172.21.15.89:0/174529304 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a00148f0 con 0x7f39a80a2530 2024-08-19T10:38:16.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f399effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 1 Processor -- start 2024-08-19T10:38:16.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 1 -- start start 2024-08-19T10:38:16.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient: init 2024-08-19T10:38:16.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f39a559a640 4 set_mon_vals no callback set 2024-08-19T10:38:16.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f39acb06640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient: _add_conns ranks=[2,6,8,1,7,10,3,5,9,0,4] 2024-08-19T10:38:16.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b240a67d0 0x7f7b240a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient(hunting): picked mon.noname-b con 0x7f7b240a67d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:16.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 0x7f7b240a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient(hunting): picked mon.noname-d con 0x7f7b240a6bb0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7b24139590 0x7f7b241399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7b24139590 0x7f7b241399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7b24139590 0x7f7b241399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47762/0 (socket says 172.21.15.89:47762) 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 1 -- 172.21.15.89:0/3768302012 learned_addr learned my addr 172.21.15.89:0/3768302012 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient(hunting): picked mon.noname-e con 0x7f7b24139590 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 1 -- 172.21.15.89:0/3768302012 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7b240a8740 con 0x7f7b240a67d0 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 1 -- 172.21.15.89:0/3768302012 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7b240a3380 con 0x7f7b240a6bb0 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 1 -- 172.21.15.89:0/3768302012 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7b240a8250 con 0x7f7b24139590 2024-08-19T10:38:16.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b29600640 10 monclient(hunting): authenticate will time out at 946.582889s 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 10 monclient(hunting): get_auth_request con 0x7f7b24139590 auth_method 0 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b227fc640 1 --2- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 0x7f7b240a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b227fc640 10 monclient(hunting): get_auth_request con 0x7f7b240a6bb0 auth_method 0 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b227fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b227fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b227fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f399effd640 1 -- 172.21.15.89:0/174529304 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39a00133e0 con 0x7f39a80a2530 2024-08-19T10:38:16.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f39acb06640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f39acb06640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:16.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f39acb06640 1 -- 172.21.15.89:0/174529304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f398c005090 con 0x7f39a80a2530 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f399effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f399effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f399effd640 10 monclient: dump: 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.835 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f399effd640 1 -- 172.21.15.89:0/174529304 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f39a001b020 con 0x7f39a80a2530 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f399effd640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398803bde0 0x7f398803e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f399effd640 1 -- 172.21.15.89:0/174529304 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f39a004fab0 con 0x7f39a80a2530 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f39a659c640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398803bde0 0x7f398803e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f39a659c640 10 monclient: get_auth_request con 0x7f398803bde0 auth_method 0 2024-08-19T10:38:16.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 10 monclient(hunting): handle_auth_done global_id 4274 payload 274 2024-08-19T10:38:16.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 1 -- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 msgr2=0x7f7b240a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 1 --2- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 0x7f7b240a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 1 -- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b240a67d0 msgr2=0x7f7b240a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 1 --2- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b240a67d0 0x7f7b240a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 1 monclient: found mon.noname-e 2024-08-19T10:38:16.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.762+0000 7f7b237fe640 1 -- 172.21.15.89:0/3768302012 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b240a4cc0 con 0x7f7b24139590 2024-08-19T10:38:16.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f7b227fc640 1 --2- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 0x7f7b240a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:16.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f0d5bfa0640 10 monclient: get_monmap_and_config 2024-08-19T10:38:16.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f0d5bfa0640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f0d5bfa0640 10 monclient: monmap: 2024-08-19T10:38:16.839 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.839 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.839 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:16.772354+0000 2024-08-19T10:38:16.839 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:16.772354+0000 2024-08-19T10:38:16.839 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:16.840 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:16.841 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.766+0000 7f39a659c640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398803bde0 0x7f398803e2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3998006cb0 tx=0x7f3998009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b237fe640 1 -- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7b24139590 msgr2=0x7f7b241399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b237fe640 1 --2- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7b24139590 0x7f7b241399f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009eb0 tx=0x7f7b1c01c9c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1b7b71279c968b68 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 1 -- 172.21.15.89:0/3768302012 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7b1c0090d0 con 0x7f7b24139590 2024-08-19T10:38:16.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 1 Processor -- start 2024-08-19T10:38:16.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:16.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 1 -- start start 2024-08-19T10:38:16.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 10 monclient: init 2024-08-19T10:38:16.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 10 monclient: dump: 2024-08-19T10:38:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.845 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.846 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.846 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.846 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.846 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.846 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.846 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 5 monclient: authenticate success, global_id 4274 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 1 -- 172.21.15.89:0/3768302012 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b1c0a59d0 con 0x7f7b24139590 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 4 set_mon_vals no callback set 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 1 -- 172.21.15.89:0/3768302012 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7b1c0ad3f0 con 0x7f7b24139590 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b21ffb640 10 monclient: dump: 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.847 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.848 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.848 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.848 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.848 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.848 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.848 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.848 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.848 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.848 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.848 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.848 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.849 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.849 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 10 monclient: shutdown 2024-08-19T10:38:16.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 1 -- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7b24139590 msgr2=0x7f7b241399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 1 --2- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7b24139590 0x7f7b241399f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009eb0 tx=0x7f7b1c01c9c0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 1 -- 172.21.15.89:0/3768302012 shutdown_connections 2024-08-19T10:38:16.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 1 --2- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7b24139590 0x7f7b241399f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 1 --2- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 0x7f7b240a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 1 --2- 172.21.15.89:0/3768302012 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7b240a67d0 0x7f7b240a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 1 -- 172.21.15.89:0/3768302012 >> 172.21.15.89:0/3768302012 conn(0x7f7b24069ae0 msgr2=0x7f7b240abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 10 monclient: _add_conns ranks=[5,4,1,10,2,3,7,8,9,0,6] 2024-08-19T10:38:16.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3b30 0x7f0d540a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 10 monclient(hunting): picked mon.noname-h con 0x7f0d540a3b30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:16.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d540a3f10 0x7f0d540a4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5953b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d540a3f10 0x7f0d540a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5953b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d540a3f10 0x7f0d540a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37760/0 (socket says 172.21.15.89:37760) 2024-08-19T10:38:16.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3b30 0x7f0d540a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3b30 0x7f0d540a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:58624/0 (socket says 172.21.15.89:58624) 2024-08-19T10:38:16.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5953b640 1 -- 172.21.15.89:0/174439348 learned_addr learned my addr 172.21.15.89:0/174439348 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 10 monclient(hunting): picked mon.noname-c con 0x7f0d540a3f10 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:16.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0d541364f0 addr v1:172.21.15.190:6789/0 2024-08-19T10:38:16.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/174439348 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d54074170 con 0x7f0d541364f0 2024-08-19T10:38:16.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/174439348 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f0d540a08b0 con 0x7f0d540a3f10 2024-08-19T10:38:16.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/174439348 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0d540a0630 con 0x7f0d540a3b30 2024-08-19T10:38:16.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5bfa0640 10 monclient(hunting): authenticate will time out at 946.591089s 2024-08-19T10:38:16.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5953b640 10 monclient(hunting): get_auth_request con 0x7f0d540a3f10 auth_method 0 2024-08-19T10:38:16.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5953b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5953b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5953b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 10 monclient(hunting): get_auth_request con 0x7f0d540a3b30 auth_method 0 2024-08-19T10:38:16.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5953b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5953b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5953b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 10 monclient(hunting): handle_auth_done global_id 4316 payload 274 2024-08-19T10:38:16.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 1 -- 172.21.15.89:0/174439348 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d540a3f10 msgr2=0x7f0d540a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:16.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 1 --2- 172.21.15.89:0/174439348 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d540a3f10 0x7f0d540a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 1 -- 172.21.15.89:0/174439348 >> v1:172.21.15.190:6789/0 conn(0x7f0d541364f0 legacy=0x7f0d54136970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T10:38:16.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 1 monclient: found mon.noname-h 2024-08-19T10:38:16.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:38:16.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d59d3c640 1 -- 172.21.15.89:0/174439348 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d540a1950 con 0x7f0d540a3b30 2024-08-19T10:38:16.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f0d5953b640 1 --2- 172.21.15.89:0/174439348 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d540a3f10 0x7f0d540a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:16.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f399effd640 1 -- 172.21.15.89:0/174529304 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f39a0022030 con 0x7f39a80a2530 2024-08-19T10:38:16.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f399effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f399effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:16.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 1 -- 172.21.15.89:0/3768302012 shutdown_connections 2024-08-19T10:38:16.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 1 -- 172.21.15.89:0/3768302012 wait complete. 2024-08-19T10:38:16.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 10 monclient: monmap: 2024-08-19T10:38:16.862 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.862 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.862 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.862 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.862 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.862 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.862 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.862 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.863 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.863 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.863 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.863 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.863 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.863 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.863 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.863 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 1 Processor -- start 2024-08-19T10:38:16.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 1 -- start start 2024-08-19T10:38:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.770+0000 7f7b29600640 10 monclient: init 2024-08-19T10:38:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient: _add_conns ranks=[7,6,1,0,8,4,2,3,5] 2024-08-19T10:38:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7b240a67d0 0x7f7b241d8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient(hunting): picked mon.noname-i con 0x7f7b240a67d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 0x7f7b2409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient(hunting): picked mon.noname-d con 0x7f7b240a6bb0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b24139590 0x7f7b2409cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient(hunting): picked mon.noname-f con 0x7f7b24139590 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b24143390 con 0x7f7b24139590 2024-08-19T10:38:16.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7b24143210 con 0x7f7b240a6bb0 2024-08-19T10:38:16.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7b24143510 con 0x7f7b240a67d0 2024-08-19T10:38:16.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient(hunting): authenticate will time out at 946.592132s 2024-08-19T10:38:16.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 0x7f7b2409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 0x7f7b2409c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46482/0 (socket says 172.21.15.89:46482) 2024-08-19T10:38:16.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 1 -- 172.21.15.89:0/3422645321 learned_addr learned my addr 172.21.15.89:0/3422645321 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 10 monclient(hunting): get_auth_request con 0x7f7b240a6bb0 auth_method 0 2024-08-19T10:38:16.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b22ffd640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7b240a67d0 0x7f7b241d8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b237fe640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b24139590 0x7f7b2409cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b22ffd640 10 monclient(hunting): get_auth_request con 0x7f7b240a67d0 auth_method 0 2024-08-19T10:38:16.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b22ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b22ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b22ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b237fe640 10 monclient(hunting): get_auth_request con 0x7f7b24139590 auth_method 0 2024-08-19T10:38:16.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b237fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b237fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b22ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b22ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b22ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 10 monclient(hunting): handle_auth_done global_id 4263 payload 824 2024-08-19T10:38:16.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 1 -- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7b240a67d0 msgr2=0x7f7b241d8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7b240a67d0 0x7f7b241d8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 1 -- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b24139590 msgr2=0x7f7b2409cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b24139590 0x7f7b2409cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 1 monclient: found mon.noname-d 2024-08-19T10:38:16.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 1 -- 172.21.15.89:0/3422645321 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2409e3d0 con 0x7f7b240a6bb0 2024-08-19T10:38:16.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 1 -- 172.21.15.89:0/3422645321 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 msgr2=0x7f7b2409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b227fc640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 0x7f7b2409c9e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800c750 tx=0x7f7b1800cc10 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 1 -- 172.21.15.89:0/3422645321 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7b1800ce60 con 0x7f7b240a6bb0 2024-08-19T10:38:16.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:16.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 10 monclient: dump: 2024-08-19T10:38:16.869 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.869 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.869 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.869 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.869 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.869 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.870 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.870 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.870 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.870 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.870 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.870 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.870 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.870 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.870 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.870 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 1 -- 172.21.15.89:0/3422645321 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b180113f0 con 0x7f7b240a6bb0 2024-08-19T10:38:16.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d59d3c640 1 -- 172.21.15.89:0/174439348 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3b30 msgr2=0x7f0d540a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d59d3c640 1 --2- 172.21.15.89:0/174439348 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3b30 0x7f0d540a2a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0d500097f0 tx=0x7f0d5001e770 comp rx=0 tx=0).ready entity=mon.5 client_cookie=c055da7887a0261c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 1 -- 172.21.15.89:0/174439348 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0d500ac8e0 con 0x7f0d540a3b30 2024-08-19T10:38:16.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:38:16.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 10 monclient: dump: 2024-08-19T10:38:16.873 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.873 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.873 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.873 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.873 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.873 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.874 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.874 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.874 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.874 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.874 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.874 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.874 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.874 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.874 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.874 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 1 -- 172.21.15.89:0/174439348 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d500b1070 con 0x7f0d540a3b30 2024-08-19T10:38:16.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 1 -- 172.21.15.89:0/174439348 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0d500ab3d0 con 0x7f0d540a3b30 2024-08-19T10:38:16.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:38:16.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d58d3a640 10 monclient: dump: 2024-08-19T10:38:16.875 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.876 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.877 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.877 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.877 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d5bfa0640 5 monclient: authenticate success, global_id 4316 2024-08-19T10:38:16.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d5bfa0640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:16.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d5bfa0640 4 set_mon_vals no callback set 2024-08-19T10:38:16.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d5bfa0640 10 monclient: shutdown 2024-08-19T10:38:16.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d5bfa0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/174439348 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3b30 msgr2=0x7f0d540a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f0d5bfa0640 1 --2- 172.21.15.89:0/174439348 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3b30 0x7f0d540a2a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0d500097f0 tx=0x7f0d5001e770 comp rx=0 tx=0).stop 2024-08-19T10:38:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 1 -- 172.21.15.89:0/3422645321 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7b18011a10 con 0x7f7b240a6bb0 2024-08-19T10:38:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 5 monclient: authenticate success, global_id 4263 2024-08-19T10:38:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient: _renew_subs 2024-08-19T10:38:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 1 -- 172.21.15.89:0/3422645321 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2409ffb0 con 0x7f7b240a6bb0 2024-08-19T10:38:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b217fa640 4 set_mon_vals no callback set 2024-08-19T10:38:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:16.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b0b7fe640 10 monclient: dump: 2024-08-19T10:38:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.881 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.882 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient: _renew_subs 2024-08-19T10:38:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.774+0000 7f7b29600640 1 -- 172.21.15.89:0/3422645321 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b240a0290 con 0x7f7b240a6bb0 2024-08-19T10:38:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f7b29600640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f7b29600640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f7b29600640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f7b29600640 1 -- 172.21.15.89:0/3422645321 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b24064590 con 0x7f7b240a6bb0 2024-08-19T10:38:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f7b0b7fe640 1 -- 172.21.15.89:0/3422645321 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7b18011c70 con 0x7f7b240a6bb0 2024-08-19T10:38:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f7b0b7fe640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7b1003be30 0x7f7b1003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f7b0b7fe640 1 -- 172.21.15.89:0/3422645321 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f7b1804eac0 con 0x7f7b240a6bb0 2024-08-19T10:38:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f7b22ffd640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7b1003be30 0x7f7b1003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.778+0000 7f7b22ffd640 10 monclient: get_auth_request con 0x7f7b1003be30 auth_method 0 2024-08-19T10:38:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.782+0000 7f7b22ffd640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7b1003be30 0x7f7b1003e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009990 tx=0x7f7b14009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.782+0000 7f7b0b7fe640 1 -- 172.21.15.89:0/3422645321 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7b1801e030 con 0x7f7b240a6bb0 2024-08-19T10:38:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.782+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/174439348 shutdown_connections 2024-08-19T10:38:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.782+0000 7f0d5bfa0640 1 --2- 172.21.15.89:0/174439348 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0d540a3f10 0x7f0d540a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.782+0000 7f0d5bfa0640 1 --2- 172.21.15.89:0/174439348 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3b30 0x7f0d540a2a60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.782+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/174439348 >> 172.21.15.89:0/174439348 conn(0x7f0d540a8f60 msgr2=0x7f0d540a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.782+0000 7f7b0b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.782+0000 7f7b0b7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.782+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/174439348 shutdown_connections 2024-08-19T10:38:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.782+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/174439348 wait complete. 2024-08-19T10:38:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient: build_initial_monmap 2024-08-19T10:38:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient: monmap: 2024-08-19T10:38:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:16.891 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 1 Processor -- start 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 1 -- start start 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient: init 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient: _add_conns ranks=[1,5,3,0,7,8,4,2,6] 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d540a3b30 0x7f0d5413c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient(hunting): picked mon.noname-f con 0x7f0d540a3b30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:16.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3f10 0x7f0d541cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient(hunting): picked mon.noname-h con 0x7f0d540a3f10 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d541364f0 0x7f0d541cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient(hunting): picked mon.noname-g con 0x7f0d541364f0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5406d2c0 con 0x7f0d540a3b30 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d59d3c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d540a3b30 0x7f0d5413c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5953b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3f10 0x7f0d541cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d59d3c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d540a3b30 0x7f0d5413c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:44220/0 (socket says 172.21.15.89:44220) 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d59d3c640 1 -- 172.21.15.89:0/4007427447 learned_addr learned my addr 172.21.15.89:0/4007427447 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d541364f0 0x7f0d541cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0d5406d140 con 0x7f0d541364f0 2024-08-19T10:38:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0d5406d440 con 0x7f0d540a3f10 2024-08-19T10:38:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5bfa0640 10 monclient(hunting): authenticate will time out at 946.606828s 2024-08-19T10:38:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d59d3c640 10 monclient(hunting): get_auth_request con 0x7f0d540a3b30 auth_method 0 2024-08-19T10:38:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d59d3c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d59d3c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d59d3c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5953b640 10 monclient(hunting): get_auth_request con 0x7f0d540a3f10 auth_method 0 2024-08-19T10:38:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5953b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5953b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5953b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 10 monclient(hunting): get_auth_request con 0x7f0d541364f0 auth_method 0 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d59d3c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d59d3c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d59d3c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5953b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:16.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5953b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:16.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5953b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:16.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 10 monclient(hunting): handle_auth_done global_id 4296 payload 824 2024-08-19T10:38:16.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 1 -- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3f10 msgr2=0x7f0d541cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3f10 0x7f0d541cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 1 -- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d540a3b30 msgr2=0x7f0d5413c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d540a3b30 0x7f0d5413c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 10 monclient: _finish_hunting 0 2024-08-19T10:38:16.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 1 monclient: found mon.noname-g 2024-08-19T10:38:16.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:16.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:38:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d5a53d640 1 -- 172.21.15.89:0/4007427447 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d540739e0 con 0x7f0d541364f0 2024-08-19T10:38:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.786+0000 7f0d59d3c640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d540a3b30 0x7f0d5413c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d5a53d640 1 -- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d541364f0 msgr2=0x7f0d541cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d5a53d640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d541364f0 0x7f0d541cda30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009ef0 tx=0x7f0d4c00be20 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 1 -- 172.21.15.89:0/4007427447 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0d4c015b90 con 0x7f0d541364f0 2024-08-19T10:38:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:38:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 10 monclient: dump: 2024-08-19T10:38:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.897 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.898 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.898 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.898 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.898 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.898 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.898 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.898 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.898 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.898 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.898 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.898 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.899 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:38:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:38:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 1 -- 172.21.15.89:0/4007427447 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4c015d70 con 0x7f0d541364f0 2024-08-19T10:38:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4bfff640 4 set_mon_vals no callback set 2024-08-19T10:38:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 1 -- 172.21.15.89:0/4007427447 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0d4c017400 con 0x7f0d541364f0 2024-08-19T10:38:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:16.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d4a7fc640 10 monclient: dump: 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:16.900 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:16.901 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d5bfa0640 5 monclient: authenticate success, global_id 4296 2024-08-19T10:38:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d5bfa0640 10 monclient: _renew_subs 2024-08-19T10:38:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d5bfa0640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:38:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d54073490 con 0x7f0d541364f0 2024-08-19T10:38:16.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d5bfa0640 10 monclient: _renew_subs 2024-08-19T10:38:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d5bfa0640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:38:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.790+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5413cd80 con 0x7f0d541364f0 2024-08-19T10:38:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.794+0000 7f0d5bfa0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.794+0000 7f0d5bfa0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.794+0000 7f0d5bfa0640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:38:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.794+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d540a2a60 con 0x7f0d541364f0 2024-08-19T10:38:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.794+0000 7f0d4a7fc640 1 -- 172.21.15.89:0/4007427447 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0d4c015720 con 0x7f0d541364f0 2024-08-19T10:38:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.794+0000 7f0d4a7fc640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d3803be30 0x7f0d3803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.794+0000 7f0d4a7fc640 1 -- 172.21.15.89:0/4007427447 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f0d4c0537b0 con 0x7f0d541364f0 2024-08-19T10:38:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.794+0000 7f0d59d3c640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d3803be30 0x7f0d3803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:16.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.794+0000 7f0d59d3c640 10 monclient: get_auth_request con 0x7f0d3803be30 auth_method 0 2024-08-19T10:38:16.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.798+0000 7f0d59d3c640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d3803be30 0x7f0d3803e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009710 tx=0x7f0d500a9000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:16.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.806+0000 7f0d4a7fc640 1 -- 172.21.15.89:0/4007427447 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0d4c017ac0 con 0x7f0d541364f0 2024-08-19T10:38:16.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.806+0000 7f0d4a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:16.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.806+0000 7f0d4a7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:16.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.930+0000 7f8db2e69640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-08-19T10:38:16.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.930+0000 7f8db2e69640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-08-19T10:38:16.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.930+0000 7f8db2e69640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.930+0000 7f8db2e69640 1 -- 172.21.15.89:0/852176609 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8d8c005380 con 0x7f8dac09b8e0 2024-08-19T10:38:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.930+0000 7f8da97fa640 1 -- 172.21.15.89:0/852176609 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8da00e65b0 con 0x7f8dac09b8e0 2024-08-19T10:38:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.930+0000 7f8da97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-08-19T10:38:16.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.930+0000 7f8da97fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:16.934 INFO:teuthology.orchestra.run.smithi089.stdout:51539607559 2024-08-19T10:38:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 -- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8d9003be00 msgr2=0x7f8d9003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8d9003be00 0x7f8d9003e2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8dac094c10 tx=0x7f8d98008040 comp rx=0 tx=0).stop 2024-08-19T10:38:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 10 monclient: shutdown 2024-08-19T10:38:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 -- 172.21.15.89:0/852176609 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8dac09b8e0 msgr2=0x7f8dac0934b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8dac09b8e0 0x7f8dac0934b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8da00ae040 tx=0x7f8da0019200 comp rx=0 tx=0).stop 2024-08-19T10:38:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 -- 172.21.15.89:0/852176609 shutdown_connections 2024-08-19T10:38:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8dac0a4240 0x7f8dac093f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8dac09ddd0 0x7f8dac0939f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8d9003be00 0x7f8d9003e2c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 --2- 172.21.15.89:0/852176609 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8dac09b8e0 0x7f8dac0934b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 -- 172.21.15.89:0/852176609 >> 172.21.15.89:0/852176609 conn(0x7f8dac0a4df0 msgr2=0x7f8dac0a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 -- 172.21.15.89:0/852176609 shutdown_connections 2024-08-19T10:38:16.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.934+0000 7f8db2e69640 1 -- 172.21.15.89:0/852176609 wait complete. 2024-08-19T10:38:16.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.950+0000 7f4e8aaf1640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-08-19T10:38:16.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.950+0000 7f4e8aaf1640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-08-19T10:38:16.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.950+0000 7f4e8aaf1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:16.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.950+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/2794270065 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4e7c009860 con 0x7f4e7c024b10 2024-08-19T10:38:16.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.950+0000 7f4e7a7fc640 1 -- 172.21.15.89:0/2794270065 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4e840b3090 con 0x7f4e7c024b10 2024-08-19T10:38:16.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e7a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-08-19T10:38:16.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e7a7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:16.956 INFO:teuthology.orchestra.run.smithi089.stdout:51539607556 2024-08-19T10:38:16.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607559 got 51539607559 for osd.3 2024-08-19T10:38:16.958 DEBUG:teuthology.parallel:result is None 2024-08-19T10:38:16.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/2794270065 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e6803be30 msgr2=0x7f4e6803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e6803be30 0x7f4e6803e2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0084a0 tx=0x7f4e740a6000 comp rx=0 tx=0).stop 2024-08-19T10:38:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 10 monclient: shutdown 2024-08-19T10:38:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/2794270065 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e7c024b10 msgr2=0x7f4e7c0c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e7c024b10 0x7f4e7c0c07a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4e8406c5d0 tx=0x7f4e8406dd10 comp rx=0 tx=0).stop 2024-08-19T10:38:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/2794270065 shutdown_connections 2024-08-19T10:38:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e6803be30 0x7f4e6803e2f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e7c024b10 0x7f4e7c0c07a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4e7c0246d0 0x7f4e7c0b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 --2- 172.21.15.89:0/2794270065 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e7c0242f0 0x7f4e7c0b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:16.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/2794270065 >> 172.21.15.89:0/2794270065 conn(0x7f4e7c007780 msgr2=0x7f4e7c023440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:16.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/2794270065 shutdown_connections 2024-08-19T10:38:16.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:16.954+0000 7f4e8aaf1640 1 -- 172.21.15.89:0/2794270065 wait complete. 2024-08-19T10:38:16.973 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607556 for osd.4 2024-08-19T10:38:16.973 DEBUG:teuthology.parallel:result is None 2024-08-19T10:38:17.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-08-19T10:38:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-08-19T10:38:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 -- 172.21.15.89:0/1613298768 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fecdc064590 con 0x7fecdc139620 2024-08-19T10:38:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd8ff9640 1 -- 172.21.15.89:0/1613298768 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fecd4004740 con 0x7fecdc139620 2024-08-19T10:38:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-08-19T10:38:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd8ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:17.004 INFO:teuthology.orchestra.run.smithi089.stdout:47244640261 2024-08-19T10:38:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 -- 172.21.15.89:0/1613298768 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fecbc03be00 msgr2=0x7fecbc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fecbc03be00 0x7fecbc03e2c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feccc00a9a0 tx=0x7feccc008040 comp rx=0 tx=0).stop 2024-08-19T10:38:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 10 monclient: shutdown 2024-08-19T10:38:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 -- 172.21.15.89:0/1613298768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecdc139620 msgr2=0x7fecdc09d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecdc139620 0x7fecdc09d460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fecd4007db0 tx=0x7fecd400d4a0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 -- 172.21.15.89:0/1613298768 shutdown_connections 2024-08-19T10:38:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fecbc03be00 0x7fecbc03e2c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fecdc0a7180 0x7fecdc09cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecdc139620 0x7fecdc09d460 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 --2- 172.21.15.89:0/1613298768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecdc0a6860 0x7fecdc09c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 -- 172.21.15.89:0/1613298768 >> 172.21.15.89:0/1613298768 conn(0x7fecdc074b40 msgr2=0x7fecdc0aaaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 -- 172.21.15.89:0/1613298768 shutdown_connections 2024-08-19T10:38:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.002+0000 7fecd1ffb640 1 -- 172.21.15.89:0/1613298768 wait complete. 2024-08-19T10:38:17.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.2 2024-08-19T10:38:17.028 DEBUG:teuthology.parallel:result is None 2024-08-19T10:38:17.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.042+0000 7f39acb06640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-08-19T10:38:17.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.042+0000 7f39acb06640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-08-19T10:38:17.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.042+0000 7f39acb06640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:17.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.042+0000 7f39acb06640 1 -- 172.21.15.89:0/174529304 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f398c005380 con 0x7f39a80a2530 2024-08-19T10:38:17.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.042+0000 7f399effd640 1 -- 172.21.15.89:0/174529304 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f39a80ac270 con 0x7f39a80a2530 2024-08-19T10:38:17.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.042+0000 7f399effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-08-19T10:38:17.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.042+0000 7f399effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:17.045 INFO:teuthology.orchestra.run.smithi089.stdout:51539607557 2024-08-19T10:38:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 -- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398803bde0 msgr2=0x7f398803e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398803bde0 0x7f398803e2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3998006cb0 tx=0x7f3998009240 comp rx=0 tx=0).stop 2024-08-19T10:38:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 10 monclient: shutdown 2024-08-19T10:38:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 -- 172.21.15.89:0/174529304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a2530 msgr2=0x7f39a813dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a2530 0x7f39a813dab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39a000efc0 tx=0x7f39a000c490 comp rx=0 tx=0).stop 2024-08-19T10:38:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 -- 172.21.15.89:0/174529304 shutdown_connections 2024-08-19T10:38:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398803bde0 0x7f398803e2a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39a80a2530 0x7f39a813dab0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39a8139620 0x7f39a80a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 --2- 172.21.15.89:0/174529304 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39a80a6860 0x7f39a81d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 -- 172.21.15.89:0/174529304 >> 172.21.15.89:0/174529304 conn(0x7f39a8074b40 msgr2=0x7f39a8137a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 -- 172.21.15.89:0/174529304 shutdown_connections 2024-08-19T10:38:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.046+0000 7f39acb06640 1 -- 172.21.15.89:0/174529304 wait complete. 2024-08-19T10:38:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.050+0000 7f7b29600640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-08-19T10:38:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.050+0000 7f7b29600640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-08-19T10:38:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.050+0000 7f7b29600640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.050+0000 7f7b29600640 1 -- 172.21.15.89:0/3422645321 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7b2409f660 con 0x7f7b240a6bb0 2024-08-19T10:38:17.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.050+0000 7f7b0b7fe640 1 -- 172.21.15.89:0/3422645321 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7b18015020 con 0x7f7b240a6bb0 2024-08-19T10:38:17.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.050+0000 7f7b0b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-08-19T10:38:17.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.050+0000 7f7b0b7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:17.054 INFO:teuthology.orchestra.run.smithi089.stdout:51539607558 2024-08-19T10:38:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 -- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7b1003be30 msgr2=0x7f7b1003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7b1003be30 0x7f7b1003e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009990 tx=0x7f7b14009110 comp rx=0 tx=0).stop 2024-08-19T10:38:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 10 monclient: shutdown 2024-08-19T10:38:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 -- 172.21.15.89:0/3422645321 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 msgr2=0x7f7b2409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 0x7f7b2409c9e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800c750 tx=0x7f7b1800cc10 comp rx=0 tx=0).stop 2024-08-19T10:38:17.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 -- 172.21.15.89:0/3422645321 shutdown_connections 2024-08-19T10:38:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7b1003be30 0x7f7b1003e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b24139590 0x7f7b2409cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b240a6bb0 0x7f7b2409c9e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 --2- 172.21.15.89:0/3422645321 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7b240a67d0 0x7f7b241d8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 -- 172.21.15.89:0/3422645321 >> 172.21.15.89:0/3422645321 conn(0x7f7b24069ae0 msgr2=0x7f7b240ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 -- 172.21.15.89:0/3422645321 shutdown_connections 2024-08-19T10:38:17.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.054+0000 7f7b29600640 1 -- 172.21.15.89:0/3422645321 wait complete. 2024-08-19T10:38:17.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.5 2024-08-19T10:38:17.071 DEBUG:teuthology.parallel:result is None 2024-08-19T10:38:17.072 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.0 2024-08-19T10:38:17.072 DEBUG:teuthology.parallel:result is None 2024-08-19T10:38:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-08-19T10:38:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-08-19T10:38:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:38:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0d54064590 con 0x7f0d541364f0 2024-08-19T10:38:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d4a7fc640 1 -- 172.21.15.89:0/4007427447 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0d4c01fc80 con 0x7f0d541364f0 2024-08-19T10:38:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d4a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-08-19T10:38:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d4a7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:17.084 INFO:teuthology.orchestra.run.smithi089.stdout:47244640261 2024-08-19T10:38:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d3803be30 msgr2=0x7f0d3803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d3803be30 0x7f0d3803e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009710 tx=0x7f0d500a9000 comp rx=0 tx=0).stop 2024-08-19T10:38:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 10 monclient: shutdown 2024-08-19T10:38:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d541364f0 msgr2=0x7f0d541cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d541364f0 0x7f0d541cda30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009ef0 tx=0x7f0d4c00be20 comp rx=0 tx=0).stop 2024-08-19T10:38:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 shutdown_connections 2024-08-19T10:38:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d3803be30 0x7f0d3803e2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d541364f0 0x7f0d541cda30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0d540a3f10 0x7f0d541cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 --2- 172.21.15.89:0/4007427447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d540a3b30 0x7f0d5413c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 >> 172.21.15.89:0/4007427447 conn(0x7f0d540a8f60 msgr2=0x7f0d5413de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 shutdown_connections 2024-08-19T10:38:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:17.082+0000 7f0d5bfa0640 1 -- 172.21.15.89:0/4007427447 wait complete. 2024-08-19T10:38:17.101 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640261 for osd.1 2024-08-19T10:38:18.101 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-19T10:38:18.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.226+0000 7f664b02e640 10 monclient: get_monmap_and_config 2024-08-19T10:38:18.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.226+0000 7f664b02e640 10 monclient: build_initial_monmap 2024-08-19T10:38:18.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.226+0000 7f664b02e640 10 monclient: monmap: 2024-08-19T10:38:18.230 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:18.230 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:18.230 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:18.231987+0000 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:18.231987+0000 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:18.231 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:18.232 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:18.232 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:18.232 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:18.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 1 Processor -- start 2024-08-19T10:38:18.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 1 -- start start 2024-08-19T10:38:18.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 10 monclient: init 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 10 monclient: _add_conns ranks=[0,2,8,4,5,10,1,6,3,9,7] 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f664409b8e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664409bcc0 0x7f664409ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 10 monclient(hunting): picked mon.noname-b con 0x7f664409bcc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66440a4410 0x7f66441414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 10 monclient(hunting): picked mon.noname-e con 0x7f66440a4410 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f664406db50 con 0x7f664409b8e0 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f664409d760 con 0x7f664409bcc0 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f664409d9b0 con 0x7f66440a4410 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f664b02e640 10 monclient(hunting): authenticate will time out at 948.049128s 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6648dca640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f664409b8e0 0x7f66440a3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54174/0 (socket says 172.21.15.89:54174) 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6648dca640 1 -- 172.21.15.89:0/3861049672 learned_addr learned my addr 172.21.15.89:0/3861049672 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 1 --2- 172.21.15.89:0/3861049672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664409bcc0 0x7f664409ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f66495cb640 1 --2- 172.21.15.89:0/3861049672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66440a4410 0x7f66441414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 10 monclient(hunting): get_auth_request con 0x7f664409bcc0 auth_method 0 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f66495cb640 10 monclient(hunting): get_auth_request con 0x7f66440a4410 auth_method 0 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f66495cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f66495cb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f66495cb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f66495cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f66495cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f66495cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 10 monclient(hunting): handle_auth_done global_id 4250 payload 274 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 1 -- 172.21.15.89:0/3861049672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66440a4410 msgr2=0x7f66441414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 1 --2- 172.21.15.89:0/3861049672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66440a4410 0x7f66441414c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 1 -- 172.21.15.89:0/3861049672 >> v1:172.21.15.89:6789/0 conn(0x7f664409b8e0 legacy=0x7f66440a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:18.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 1 monclient: found mon.noname-b 2024-08-19T10:38:18.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:18.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:18.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f6643fff640 1 -- 172.21.15.89:0/3861049672 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664406fd00 con 0x7f664409bcc0 2024-08-19T10:38:18.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.230+0000 7f66495cb640 1 --2- 172.21.15.89:0/3861049672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66440a4410 0x7f66441414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f6643fff640 1 -- 172.21.15.89:0/3861049672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664409bcc0 msgr2=0x7f664409ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f6643fff640 1 --2- 172.21.15.89:0/3861049672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664409bcc0 0x7f664409ddd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f66380097f0 tx=0x7f663801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f886a812c0935a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 1 -- 172.21.15.89:0/3861049672 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66380b0070 con 0x7f664409bcc0 2024-08-19T10:38:18.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 10 monclient: dump: 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:18.243 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:18.244 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 1 -- 172.21.15.89:0/3861049672 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66380aa810 con 0x7f664409bcc0 2024-08-19T10:38:18.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 1 -- 172.21.15.89:0/3861049672 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66380aae30 con 0x7f664409bcc0 2024-08-19T10:38:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 5 monclient: authenticate success, global_id 4250 2024-08-19T10:38:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 4 set_mon_vals no callback set 2024-08-19T10:38:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f66437fe640 10 monclient: dump: 2024-08-19T10:38:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:18.245 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:18.246 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 10 monclient: shutdown 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 1 -- 172.21.15.89:0/3861049672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664409bcc0 msgr2=0x7f664409ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 1 --2- 172.21.15.89:0/3861049672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664409bcc0 0x7f664409ddd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f66380097f0 tx=0x7f663801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 1 -- 172.21.15.89:0/3861049672 shutdown_connections 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 1 --2- 172.21.15.89:0/3861049672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66440a4410 0x7f66441414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 1 --2- 172.21.15.89:0/3861049672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f664409bcc0 0x7f664409ddd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 1 -- 172.21.15.89:0/3861049672 >> 172.21.15.89:0/3861049672 conn(0x7f66440a4df0 msgr2=0x7f66440a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 1 -- 172.21.15.89:0/3861049672 shutdown_connections 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 1 -- 172.21.15.89:0/3861049672 wait complete. 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 10 monclient: build_initial_monmap 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 10 monclient: monmap: 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:18.247 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:18.248 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 1 Processor -- start 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 1 -- start start 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.234+0000 7f664b02e640 10 monclient: init 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 10 monclient: _add_conns ranks=[8,0,1,2,4,6,5,3,7] 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f664409b8e0 0x7f6644073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 10 monclient(hunting): picked mon.noname-e con 0x7f664409b8e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664409bcc0 0x7f664409a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 10 monclient(hunting): picked mon.noname-a con 0x7f664409bcc0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66440a4410 0x7f664409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:18.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 10 monclient(hunting): picked mon.noname-f con 0x7f66440a4410 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6644144220 con 0x7f664409bcc0 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6644069300 con 0x7f66440a4410 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f66441443a0 con 0x7f664409b8e0 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f664b02e640 10 monclient(hunting): authenticate will time out at 948.056278s 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f664409b8e0 0x7f6644073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6643fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664409bcc0 0x7f664409a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f664409b8e0 0x7f6644073a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47786/0 (socket says 172.21.15.89:47786) 2024-08-19T10:38:18.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6643fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664409bcc0 0x7f664409a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57780/0 (socket says 172.21.15.89:57780) 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 1 -- 172.21.15.89:0/714902048 learned_addr learned my addr 172.21.15.89:0/714902048 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 10 monclient(hunting): get_auth_request con 0x7f664409b8e0 auth_method 0 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6643fff640 10 monclient(hunting): get_auth_request con 0x7f664409bcc0 auth_method 0 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6643fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6643fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6643fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:18.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6643fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6643fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6643fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f66495cb640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66440a4410 0x7f664409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 10 monclient(hunting): handle_auth_done global_id 4292 payload 824 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 1 -- 172.21.15.89:0/714902048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66440a4410 msgr2=0x7f664409aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66440a4410 0x7f664409aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 1 -- 172.21.15.89:0/714902048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664409bcc0 msgr2=0x7f664409a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664409bcc0 0x7f664409a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 10 monclient: _finish_hunting 0 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 1 monclient: found mon.noname-e 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:18.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:18.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 1 -- 172.21.15.89:0/714902048 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6644074510 con 0x7f664409b8e0 2024-08-19T10:38:18.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6643fff640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664409bcc0 0x7f664409a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:18.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 1 -- 172.21.15.89:0/714902048 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f664409b8e0 msgr2=0x7f6644073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:18.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.238+0000 7f6648dca640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f664409b8e0 0x7f6644073a10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f663400cc40 tx=0x7f663400a430 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:18.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 1 -- 172.21.15.89:0/714902048 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f663400ad50 con 0x7f664409b8e0 2024-08-19T10:38:18.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:18.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:18.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 10 monclient: dump: 2024-08-19T10:38:18.255 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:18.255 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:18.255 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:18.255 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:18.255 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:18.255 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:18.256 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:18.256 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:18.256 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:18.256 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:18.256 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:18.256 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:18.256 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:18.256 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:18.256 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:18.256 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f664b02e640 5 monclient: authenticate success, global_id 4292 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 1 -- 172.21.15.89:0/714902048 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6634012870 con 0x7f664409b8e0 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 1 -- 172.21.15.89:0/714902048 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6634007070 con 0x7f664409b8e0 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f6642ffd640 4 set_mon_vals no callback set 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 10 monclient: dump: 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:18.257 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:18.258 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f664b02e640 10 monclient: _renew_subs 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f664b02e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f664b02e640 1 -- 172.21.15.89:0/714902048 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66440a79a0 con 0x7f664409b8e0 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f664b02e640 10 monclient: _renew_subs 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f664b02e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f664b02e640 1 -- 172.21.15.89:0/714902048 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f66440946c0 con 0x7f664409b8e0 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f664b02e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f664b02e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f664b02e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f664b02e640 1 -- 172.21.15.89:0/714902048 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664409ddd0 con 0x7f664409b8e0 2024-08-19T10:38:18.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 1 -- 172.21.15.89:0/714902048 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6634012400 con 0x7f664409b8e0 2024-08-19T10:38:18.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f663003be00 0x7f663003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:18.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f66417fa640 1 -- 172.21.15.89:0/714902048 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f663404f090 con 0x7f664409b8e0 2024-08-19T10:38:18.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f6643fff640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f663003be00 0x7f663003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:18.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.242+0000 7f6643fff640 10 monclient: get_auth_request con 0x7f663003be00 auth_method 0 2024-08-19T10:38:18.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.246+0000 7f6643fff640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f663003be00 0x7f663003e2c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f66380ab000 tx=0x7f66380a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:18.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.246+0000 7f66417fa640 1 -- 172.21.15.89:0/714902048 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f66340072d0 con 0x7f664409b8e0 2024-08-19T10:38:18.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.246+0000 7f66417fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:18.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.246+0000 7f66417fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:18.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.402+0000 7f664b02e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-08-19T10:38:18.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.402+0000 7f664b02e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-08-19T10:38:18.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.402+0000 7f664b02e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:18.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.402+0000 7f664b02e640 1 -- 172.21.15.89:0/714902048 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f664406e560 con 0x7f664409b8e0 2024-08-19T10:38:18.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f66417fa640 1 -- 172.21.15.89:0/714902048 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6634012c00 con 0x7f664409b8e0 2024-08-19T10:38:18.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f66417fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-08-19T10:38:18.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f66417fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:18.408 INFO:teuthology.orchestra.run.smithi089.stdout:47244640261 2024-08-19T10:38:18.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 -- 172.21.15.89:0/714902048 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f663003be00 msgr2=0x7f663003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:18.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f663003be00 0x7f663003e2c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f66380ab000 tx=0x7f66380a6000 comp rx=0 tx=0).stop 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 10 monclient: shutdown 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 -- 172.21.15.89:0/714902048 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f664409b8e0 msgr2=0x7f6644073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f664409b8e0 0x7f6644073a10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f663400cc40 tx=0x7f663400a430 comp rx=0 tx=0).stop 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 -- 172.21.15.89:0/714902048 shutdown_connections 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66440a4410 0x7f664409aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664409bcc0 0x7f664409a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f663003be00 0x7f663003e2c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 --2- 172.21.15.89:0/714902048 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f664409b8e0 0x7f6644073a10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 -- 172.21.15.89:0/714902048 >> 172.21.15.89:0/714902048 conn(0x7f66440a4df0 msgr2=0x7f66440a0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 -- 172.21.15.89:0/714902048 shutdown_connections 2024-08-19T10:38:18.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:18.406+0000 7f664b02e640 1 -- 172.21.15.89:0/714902048 wait complete. 2024-08-19T10:38:18.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640261 for osd.1 2024-08-19T10:38:19.426 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-19T10:38:19.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.554+0000 7f63bb88b640 10 monclient: get_monmap_and_config 2024-08-19T10:38:19.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.554+0000 7f63bb88b640 10 monclient: build_initial_monmap 2024-08-19T10:38:19.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.554+0000 7f63bb88b640 10 monclient: monmap: 2024-08-19T10:38:19.559 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:19.559 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:19.559 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:19.559846+0000 2024-08-19T10:38:19.559 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:19.559846+0000 2024-08-19T10:38:19.559 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:19.559 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.559 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:19.560 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:19.560 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.554+0000 7f63bb88b640 1 Processor -- start 2024-08-19T10:38:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.554+0000 7f63bb88b640 1 -- start start 2024-08-19T10:38:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.554+0000 7f63bb88b640 10 monclient: init 2024-08-19T10:38:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 10 monclient: _add_conns ranks=[3,9,7,6,0,1,2,5,8,10,4] 2024-08-19T10:38:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b40a5270 0x7f63b40a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 10 monclient(hunting): picked mon.noname-g con 0x7f63b40a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f63b40a6d30 0x7f63b40a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 10 monclient(hunting): picked mon.noname-a con 0x7f63b40a6d30 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63b41395b0 0x7f63b4139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 10 monclient(hunting): picked mon.noname-i con 0x7f63b41395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f63b40a7650 con 0x7f63b40a6d30 2024-08-19T10:38:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f63b40a8710 con 0x7f63b40a5270 2024-08-19T10:38:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f63b40a5a40 con 0x7f63b41395b0 2024-08-19T10:38:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63bb88b640 10 monclient(hunting): authenticate will time out at 949.376471s 2024-08-19T10:38:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f63b40a6d30 0x7f63b40a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f63b40a6d30 0x7f63b40a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57786/0 (socket says 172.21.15.89:57786) 2024-08-19T10:38:19.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 1 -- 172.21.15.89:0/2699265823 learned_addr learned my addr 172.21.15.89:0/2699265823 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 10 monclient(hunting): get_auth_request con 0x7f63b40a6d30 auth_method 0 2024-08-19T10:38:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b9627640 1 --2- 172.21.15.89:0/2699265823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b40a5270 0x7f63b40a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b9e28640 1 --2- 172.21.15.89:0/2699265823 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63b41395b0 0x7f63b4139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b9627640 10 monclient(hunting): get_auth_request con 0x7f63b40a5270 auth_method 0 2024-08-19T10:38:19.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b9627640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:19.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b9627640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:19.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b9627640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:19.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 10 monclient(hunting): handle_auth_done global_id 4392 payload 274 2024-08-19T10:38:19.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 1 -- 172.21.15.89:0/2699265823 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63b41395b0 msgr2=0x7f63b4139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:19.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 1 --2- 172.21.15.89:0/2699265823 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63b41395b0 0x7f63b4139a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 1 -- 172.21.15.89:0/2699265823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b40a5270 msgr2=0x7f63b40a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:19.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 1 --2- 172.21.15.89:0/2699265823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b40a5270 0x7f63b40a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 10 monclient: _finish_hunting 0 2024-08-19T10:38:19.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 1 monclient: found mon.noname-a 2024-08-19T10:38:19.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:19.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:19.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.558+0000 7f63b8e26640 1 -- 172.21.15.89:0/2699265823 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b40a81d0 con 0x7f63b40a6d30 2024-08-19T10:38:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63b8e26640 1 -- 172.21.15.89:0/2699265823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63b40a6d30 msgr2=0x7f63b40a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63b8e26640 1 --2- 172.21.15.89:0/2699265823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63b40a6d30 0x7f63b40a7110 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009870 tx=0x7f63b001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd7312f0a62a62a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 1 -- 172.21.15.89:0/2699265823 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63b00b1070 con 0x7f63b40a6d30 2024-08-19T10:38:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 10 monclient: dump: 2024-08-19T10:38:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:19.577 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:19.578 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:19.579 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 10 monclient: _finish_auth 0 2024-08-19T10:38:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 5 monclient: authenticate success, global_id 4392 2024-08-19T10:38:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 1 -- 172.21.15.89:0/2699265823 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b00ab810 con 0x7f63b40a6d30 2024-08-19T10:38:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:19.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 4 set_mon_vals no callback set 2024-08-19T10:38:19.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 1 -- 172.21.15.89:0/2699265823 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63b00aa300 con 0x7f63b40a6d30 2024-08-19T10:38:19.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:19.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:19.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63abfff640 10 monclient: dump: 2024-08-19T10:38:19.580 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:19.580 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:19.580 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:19.581 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:19.582 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:19.583 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient: shutdown 2024-08-19T10:38:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 -- 172.21.15.89:0/2699265823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63b40a6d30 msgr2=0x7f63b40a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 --2- 172.21.15.89:0/2699265823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63b40a6d30 0x7f63b40a7110 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009870 tx=0x7f63b001c420 comp rx=0 tx=0).stop 2024-08-19T10:38:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 -- 172.21.15.89:0/2699265823 shutdown_connections 2024-08-19T10:38:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 --2- 172.21.15.89:0/2699265823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63b40a6d30 0x7f63b40a7110 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 --2- 172.21.15.89:0/2699265823 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63b41395b0 0x7f63b4139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 --2- 172.21.15.89:0/2699265823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b40a5270 0x7f63b40a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 -- 172.21.15.89:0/2699265823 >> 172.21.15.89:0/2699265823 conn(0x7f63b4069a80 msgr2=0x7f63b40abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:19.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 -- 172.21.15.89:0/2699265823 shutdown_connections 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 -- 172.21.15.89:0/2699265823 wait complete. 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient: build_initial_monmap 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient: monmap: 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:19.584 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 Processor -- start 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 -- start start 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient: init 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient: _add_conns ranks=[6,1,3,7,5,0,4,2,8] 2024-08-19T10:38:19.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63b40a5270 0x7f63b409cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient(hunting): picked mon.noname-d con 0x7f63b40a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40a6d30 0x7f63b409d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient(hunting): picked mon.noname-f con 0x7f63b40a6d30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b41395b0 0x7f63b41d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient(hunting): picked mon.noname-g con 0x7f63b41395b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f63b40a5650 con 0x7f63b40a6d30 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f63b40a4a70 con 0x7f63b41395b0 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f63b40a57d0 con 0x7f63b40a5270 2024-08-19T10:38:19.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63bb88b640 10 monclient(hunting): authenticate will time out at 949.383842s 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63b9627640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63b40a5270 0x7f63b409cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63b9627640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63b40a5270 0x7f63b409cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46486/0 (socket says 172.21.15.89:46486) 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63b9627640 1 -- 172.21.15.89:0/2790616394 learned_addr learned my addr 172.21.15.89:0/2790616394 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63b9627640 10 monclient(hunting): get_auth_request con 0x7f63b40a5270 auth_method 0 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63b9627640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63b9627640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.562+0000 7f63b9627640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:19.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9e28640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b41395b0 0x7f63b41d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9e28640 10 monclient(hunting): get_auth_request con 0x7f63b41395b0 auth_method 0 2024-08-19T10:38:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9e28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9e28640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9e28640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b8e26640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40a6d30 0x7f63b409d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b8e26640 10 monclient(hunting): get_auth_request con 0x7f63b40a6d30 auth_method 0 2024-08-19T10:38:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b8e26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b8e26640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b8e26640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 10 monclient(hunting): handle_auth_done global_id 4272 payload 824 2024-08-19T10:38:19.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 1 -- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b41395b0 msgr2=0x7f63b41d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b41395b0 0x7f63b41d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 1 -- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40a6d30 msgr2=0x7f63b409d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40a6d30 0x7f63b409d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 10 monclient: _finish_hunting 0 2024-08-19T10:38:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 1 monclient: found mon.noname-d 2024-08-19T10:38:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:19.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 1 -- 172.21.15.89:0/2790616394 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b409f830 con 0x7f63b40a5270 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 1 -- 172.21.15.89:0/2790616394 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63b40a5270 msgr2=0x7f63b409cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63b9627640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63b40a5270 0x7f63b409cea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f63a400d620 tx=0x7f63a400dae0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 1 -- 172.21.15.89:0/2790616394 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63a4007380 con 0x7f63b40a5270 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 10 monclient: dump: 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.590 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:19.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 1 -- 172.21.15.89:0/2790616394 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a4019070 con 0x7f63b40a5270 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63bb88b640 5 monclient: authenticate success, global_id 4272 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63bb88b640 10 monclient: _renew_subs 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63bb88b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63bb88b640 1 -- 172.21.15.89:0/2790616394 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b413f880 con 0x7f63b40a5270 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 1 -- 172.21.15.89:0/2790616394 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63a400bca0 con 0x7f63b40a5270 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63ab7fe640 4 set_mon_vals no callback set 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 10 monclient: dump: 2024-08-19T10:38:19.592 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:19.593 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63bb88b640 10 monclient: _renew_subs 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63bb88b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63bb88b640 1 -- 172.21.15.89:0/2790616394 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b409e030 con 0x7f63b40a5270 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63bb88b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63bb88b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63bb88b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63bb88b640 1 -- 172.21.15.89:0/2790616394 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b40a7110 con 0x7f63b40a5270 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.566+0000 7f63a9ffb640 1 -- 172.21.15.89:0/2790616394 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f63a400b830 con 0x7f63b40a5270 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.570+0000 7f63a9ffb640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f639803be00 0x7f639803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.570+0000 7f63a9ffb640 1 -- 172.21.15.89:0/2790616394 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f63a400f040 con 0x7f63b40a5270 2024-08-19T10:38:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.570+0000 7f63b8e26640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f639803be00 0x7f639803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.570+0000 7f63b8e26640 10 monclient: get_auth_request con 0x7f639803be00 auth_method 0 2024-08-19T10:38:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.570+0000 7f63b8e26640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f639803be00 0x7f639803e2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63b40a34f0 tx=0x7f63b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.574+0000 7f63a9ffb640 1 -- 172.21.15.89:0/2790616394 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f63a401cd70 con 0x7f63b40a5270 2024-08-19T10:38:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.574+0000 7f63a9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:19.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.574+0000 7f63a9ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:19.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.730+0000 7f63bb88b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-08-19T10:38:19.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.730+0000 7f63bb88b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-08-19T10:38:19.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.730+0000 7f63bb88b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:19.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.730+0000 7f63bb88b640 1 -- 172.21.15.89:0/2790616394 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f63b4064590 con 0x7f63b40a5270 2024-08-19T10:38:19.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.730+0000 7f63a9ffb640 1 -- 172.21.15.89:0/2790616394 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f63a4013cb0 con 0x7f63b40a5270 2024-08-19T10:38:19.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.730+0000 7f63a9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-08-19T10:38:19.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.730+0000 7f63a9ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:19.735 INFO:teuthology.orchestra.run.smithi089.stdout:47244640262 2024-08-19T10:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 -- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f639803be00 msgr2=0x7f639803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f639803be00 0x7f639803e2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63b40a34f0 tx=0x7f63b0008040 comp rx=0 tx=0).stop 2024-08-19T10:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 10 monclient: shutdown 2024-08-19T10:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 -- 172.21.15.89:0/2790616394 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63b40a5270 msgr2=0x7f63b409cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63b40a5270 0x7f63b409cea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f63a400d620 tx=0x7f63a400dae0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 -- 172.21.15.89:0/2790616394 shutdown_connections 2024-08-19T10:38:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f639803be00 0x7f639803e2c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b41395b0 0x7f63b41d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40a6d30 0x7f63b409d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 --2- 172.21.15.89:0/2790616394 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63b40a5270 0x7f63b409cea0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 -- 172.21.15.89:0/2790616394 >> 172.21.15.89:0/2790616394 conn(0x7f63b4069a80 msgr2=0x7f63b40ab110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 -- 172.21.15.89:0/2790616394 shutdown_connections 2024-08-19T10:38:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.734+0000 7f63bb88b640 1 -- 172.21.15.89:0/2790616394 wait complete. 2024-08-19T10:38:19.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.1 2024-08-19T10:38:19.752 DEBUG:teuthology.parallel:result is None 2024-08-19T10:38:19.752 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-19T10:38:19.752 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T10:38:19.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.854+0000 7f99cd086640 10 monclient: get_monmap_and_config 2024-08-19T10:38:19.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.854+0000 7f99cd086640 10 monclient: build_initial_monmap 2024-08-19T10:38:19.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.854+0000 7f99cd086640 10 monclient: monmap: 2024-08-19T10:38:19.858 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:19.858 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:19.858 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:19.859721+0000 2024-08-19T10:38:19.858 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:19.859721+0000 2024-08-19T10:38:19.858 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:19.858 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:19.859 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.854+0000 7f99cd086640 1 Processor -- start 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.854+0000 7f99cd086640 1 -- start start 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.854+0000 7f99cd086640 10 monclient: init 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 10 monclient: _add_conns ranks=[7,2,5,6,3,9,4,10,1,8,0] 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f99c80a6860 0x7f99c80a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 10 monclient(hunting): picked mon.noname-i con 0x7f99c80a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99c80a7180 0x7f99c80a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 10 monclient(hunting): picked mon.noname-b con 0x7f99c80a7180 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99c8139620 0x7f99c8139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 10 monclient(hunting): picked mon.noname-h con 0x7f99c8139620 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f99c80a8190 con 0x7f99c80a7180 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f99c80a33d0 con 0x7f99c8139620 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f99c80a2ec0 con 0x7f99c80a6860 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99cd086640 10 monclient(hunting): authenticate will time out at 949.676499s 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99c80a7180 0x7f99c80a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99c80a7180 0x7f99c80a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44456/0 (socket says 172.21.15.89:44456) 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 1 -- 172.21.15.89:0/2156014451 learned_addr learned my addr 172.21.15.89:0/2156014451 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 10 monclient(hunting): get_auth_request con 0x7f99c80a7180 auth_method 0 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:19.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c6d9d640 1 --2- 172.21.15.89:0/2156014451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f99c80a6860 0x7f99c80a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c6d9d640 10 monclient(hunting): get_auth_request con 0x7f99c80a6860 auth_method 0 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c759e640 1 --2- 172.21.15.89:0/2156014451 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99c8139620 0x7f99c8139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 10 monclient(hunting): handle_auth_done global_id 4259 payload 274 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 1 -- 172.21.15.89:0/2156014451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f99c80a6860 msgr2=0x7f99c80a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 1 --2- 172.21.15.89:0/2156014451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f99c80a6860 0x7f99c80a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 1 -- 172.21.15.89:0/2156014451 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99c8139620 msgr2=0x7f99c8139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:19.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 1 --2- 172.21.15.89:0/2156014451 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99c8139620 0x7f99c8139a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 10 monclient: _finish_hunting 0 2024-08-19T10:38:19.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 1 monclient: found mon.noname-b 2024-08-19T10:38:19.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:19.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:19.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c659c640 1 -- 172.21.15.89:0/2156014451 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c80a4cd0 con 0x7f99c80a7180 2024-08-19T10:38:19.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c759e640 10 monclient: get_auth_request con 0x7f99c8139620 auth_method 0 2024-08-19T10:38:19.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.858+0000 7f99c759e640 1 --2- 172.21.15.89:0/2156014451 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99c8139620 0x7f99c8139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:19.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c659c640 1 -- 172.21.15.89:0/2156014451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99c80a7180 msgr2=0x7f99c80a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:19.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c659c640 1 --2- 172.21.15.89:0/2156014451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99c80a7180 0x7f99c80a7560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99b00098a0 tx=0x7f99b001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8c4a2d9482fce36 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:19.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 1 -- 172.21.15.89:0/2156014451 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f99b00b0070 con 0x7f99c80a7180 2024-08-19T10:38:19.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:19.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:19.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 10 monclient: dump: 2024-08-19T10:38:19.870 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:19.870 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:19.870 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:19.871 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 10 monclient: _finish_auth 0 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 1 -- 172.21.15.89:0/2156014451 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b00aa810 con 0x7f99c80a7180 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 5 monclient: authenticate success, global_id 4259 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 4 set_mon_vals no callback set 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 1 -- 172.21.15.89:0/2156014451 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f99b00a9300 con 0x7f99c80a7180 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:19.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99c5d9b640 10 monclient: dump: 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:19.873 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:19.874 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:19.874 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:19.874 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:19.874 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 10 monclient: shutdown 2024-08-19T10:38:19.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:19.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 1 -- 172.21.15.89:0/2156014451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99c80a7180 msgr2=0x7f99c80a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:19.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 1 --2- 172.21.15.89:0/2156014451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99c80a7180 0x7f99c80a7560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99b00098a0 tx=0x7f99b001c440 comp rx=0 tx=0).stop 2024-08-19T10:38:19.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 1 -- 172.21.15.89:0/2156014451 shutdown_connections 2024-08-19T10:38:19.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 1 --2- 172.21.15.89:0/2156014451 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99c8139620 0x7f99c8139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 1 --2- 172.21.15.89:0/2156014451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99c80a7180 0x7f99c80a7560 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 1 --2- 172.21.15.89:0/2156014451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f99c80a6860 0x7f99c80a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 1 -- 172.21.15.89:0/2156014451 >> 172.21.15.89:0/2156014451 conn(0x7f99c8074b40 msgr2=0x7f99c80abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 1 -- 172.21.15.89:0/2156014451 shutdown_connections 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 1 -- 172.21.15.89:0/2156014451 wait complete. 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 10 monclient: build_initial_monmap 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.862+0000 7f99cd086640 10 monclient: monmap: 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:19.875 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 1 Processor -- start 2024-08-19T10:38:19.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 1 -- start start 2024-08-19T10:38:19.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient: init 2024-08-19T10:38:19.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:19.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient: _add_conns ranks=[6,5,0,3,8,4,1,7,2] 2024-08-19T10:38:19.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99c80a6860 0x7f99c81d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient(hunting): picked mon.noname-d con 0x7f99c80a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:19.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99c80a7180 0x7f99c80a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient(hunting): picked mon.noname-h con 0x7f99c80a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:19.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c8139620 0x7f99c80a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient(hunting): picked mon.noname-a con 0x7f99c8139620 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:19.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8143210 con 0x7f99c8139620 2024-08-19T10:38:19.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f99c8143090 con 0x7f99c80a7180 2024-08-19T10:38:19.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:19.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f99c8143390 con 0x7f99c80a6860 2024-08-19T10:38:19.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:19.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient(hunting): authenticate will time out at 949.684605s 2024-08-19T10:38:19.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99c80a6860 0x7f99c81d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99c80a6860 0x7f99c81d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46492/0 (socket says 172.21.15.89:46492) 2024-08-19T10:38:19.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 1 -- 172.21.15.89:0/1907987945 learned_addr learned my addr 172.21.15.89:0/1907987945 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:19.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 10 monclient(hunting): get_auth_request con 0x7f99c80a6860 auth_method 0 2024-08-19T10:38:19.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:19.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:19.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:19.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:19.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:19.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:19.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c759e640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c8139620 0x7f99c80a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c759e640 10 monclient(hunting): get_auth_request con 0x7f99c8139620 auth_method 0 2024-08-19T10:38:19.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:19.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c759e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:19.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c759e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:19.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 10 monclient(hunting): handle_auth_done global_id 4281 payload 824 2024-08-19T10:38:19.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 1 -- 172.21.15.89:0/1907987945 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99c80a7180 msgr2=0x7f99c80a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:19.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99c80a7180 0x7f99c80a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 1 -- 172.21.15.89:0/1907987945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c8139620 msgr2=0x7f99c80a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:19.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c8139620 0x7f99c80a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:19.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:38:19.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 1 monclient: found mon.noname-d 2024-08-19T10:38:19.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:19.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:19.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 1 -- 172.21.15.89:0/1907987945 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c80abf00 con 0x7f99c80a6860 2024-08-19T10:38:19.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c759e640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c8139620 0x7f99c80a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:38:19.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 1 -- 172.21.15.89:0/1907987945 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99c80a6860 msgr2=0x7f99c81d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:19.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c6d9d640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99c80a6860 0x7f99c81d16e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99b8007f20 tx=0x7f99b800e570 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:19.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 1 -- 172.21.15.89:0/1907987945 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f99b8013a00 con 0x7f99c80a6860 2024-08-19T10:38:19.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:19.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:19.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 10 monclient: dump: 2024-08-19T10:38:19.890 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:19.891 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:19.892 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:19.893 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:19.893 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:19.893 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:19.893 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 5 monclient: authenticate success, global_id 4281 2024-08-19T10:38:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient: _renew_subs 2024-08-19T10:38:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:19.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 1 -- 172.21.15.89:0/1907987945 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c80abbb0 con 0x7f99c80a6860 2024-08-19T10:38:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 1 -- 172.21.15.89:0/1907987945 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b8019070 con 0x7f99c80a6860 2024-08-19T10:38:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 1 -- 172.21.15.89:0/1907987945 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f99b8013c00 con 0x7f99c80a6860 2024-08-19T10:38:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:19.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99c559a640 4 set_mon_vals no callback set 2024-08-19T10:38:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99bf7fe640 10 monclient: dump: 2024-08-19T10:38:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.896 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:19.897 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:19.897 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:19.897 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:19.897 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:19.897 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:19.898 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:19.898 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:19.898 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:19.898 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:19.898 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:19.898 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:19.899 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient: _renew_subs 2024-08-19T10:38:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.866+0000 7f99cd086640 1 -- 172.21.15.89:0/1907987945 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c80a0710 con 0x7f99c80a6860 2024-08-19T10:38:19.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.870+0000 7f99cd086640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:19.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.870+0000 7f99cd086640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:19.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.870+0000 7f99cd086640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:19.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.870+0000 7f99cd086640 1 -- 172.21.15.89:0/1907987945 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c80a7560 con 0x7f99c80a6860 2024-08-19T10:38:19.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.870+0000 7f99bf7fe640 1 -- 172.21.15.89:0/1907987945 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f99b8013590 con 0x7f99c80a6860 2024-08-19T10:38:19.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.870+0000 7f99bf7fe640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f99a803bd30 0x7f99a803e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:19.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.870+0000 7f99bf7fe640 1 -- 172.21.15.89:0/1907987945 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f99b804f2b0 con 0x7f99c80a6860 2024-08-19T10:38:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.870+0000 7f99c659c640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f99a803bd30 0x7f99a803e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.870+0000 7f99c659c640 10 monclient: get_auth_request con 0x7f99a803bd30 auth_method 0 2024-08-19T10:38:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.874+0000 7f99c659c640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f99a803bd30 0x7f99a803e1f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009870 tx=0x7f99b0004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.874+0000 7f99bf7fe640 1 -- 172.21.15.89:0/1907987945 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f99c80abf00 con 0x7f99c80a6860 2024-08-19T10:38:19.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.874+0000 7f99bf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:19.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:19.874+0000 7f99bf7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:20.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.030+0000 7f99cd086640 1 -- 172.21.15.89:0/1907987945 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99c8064590 con 0x7f99a803bd30 2024-08-19T10:38:20.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99bf7fe640 1 -- 172.21.15.89:0/1907987945 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34115 (secure 0 0 0) 0x7f99c8064590 con 0x7f99a803bd30 2024-08-19T10:38:20.036 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:38:20.036 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-08-19T10:38:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 -- 172.21.15.89:0/1907987945 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f99a803bd30 msgr2=0x7f99a803e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f99a803bd30 0x7f99a803e1f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009870 tx=0x7f99b0004740 comp rx=0 tx=0).stop 2024-08-19T10:38:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 10 monclient: shutdown 2024-08-19T10:38:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 -- 172.21.15.89:0/1907987945 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99c80a6860 msgr2=0x7f99c81d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99c80a6860 0x7f99c81d16e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99b8007f20 tx=0x7f99b800e570 comp rx=0 tx=0).stop 2024-08-19T10:38:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 -- 172.21.15.89:0/1907987945 shutdown_connections 2024-08-19T10:38:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f99a803bd30 0x7f99a803e1f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c8139620 0x7f99c80a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99c80a7180 0x7f99c80a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 --2- 172.21.15.89:0/1907987945 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99c80a6860 0x7f99c81d16e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 -- 172.21.15.89:0/1907987945 >> 172.21.15.89:0/1907987945 conn(0x7f99c8074b40 msgr2=0x7f99c8138ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 -- 172.21.15.89:0/1907987945 shutdown_connections 2024-08-19T10:38:20.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.034+0000 7f99cd086640 1 -- 172.21.15.89:0/1907987945 wait complete. 2024-08-19T10:38:20.053 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-08-19T10:38:18.447904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164320,"kb_used_data":2176,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066720,"statfs":{"total":579820584960,"available":579652321280,"internally_reserved":0,"allocated":2228224,"data_stored":1519228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001890"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.838257+0000","last_change":"2024-08-19T10:38:14.838742+0000","last_active":"2024-08-19T10:38:14.838257+0000","last_peered":"2024-08-19T10:38:14.838257+0000","last_clean":"2024-08-19T10:38:14.838257+0000","last_became_active":"2024-08-19T10:38:12.819810+0000","last_became_peered":"2024-08-19T10:38:12.819810+0000","last_unstale":"2024-08-19T10:38:14.838257+0000","last_undegraded":"2024-08-19T10:38:14.838257+0000","last_fullsized":"2024-08-19T10:38:14.838257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:10:48.036604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021153309999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.849314+0000","last_change":"2024-08-19T10:38:14.849789+0000","last_active":"2024-08-19T10:38:14.849314+0000","last_peered":"2024-08-19T10:38:14.849314+0000","last_clean":"2024-08-19T10:38:14.849314+0000","last_became_active":"2024-08-19T10:38:12.830800+0000","last_became_peered":"2024-08-19T10:38:12.830800+0000","last_unstale":"2024-08-19T10:38:14.849314+0000","last_undegraded":"2024-08-19T10:38:14.849314+0000","last_fullsized":"2024-08-19T10:38:14.849314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:25:14.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081542699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.837210+0000","last_change":"2024-08-19T10:38:14.839689+0000","last_active":"2024-08-19T10:38:14.837210+0000","last_peered":"2024-08-19T10:38:14.837210+0000","last_clean":"2024-08-19T10:38:14.837210+0000","last_became_active":"2024-08-19T10:38:12.825835+0000","last_became_peered":"2024-08-19T10:38:12.825835+0000","last_unstale":"2024-08-19T10:38:14.837210+0000","last_undegraded":"2024-08-19T10:38:14.837210+0000","last_fullsized":"2024-08-19T10:38:14.837210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:13:58.426106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003718322,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.838322+0000","last_change":"2024-08-19T10:38:14.838322+0000","last_active":"2024-08-19T10:38:14.838322+0000","last_peered":"2024-08-19T10:38:14.838322+0000","last_clean":"2024-08-19T10:38:14.838322+0000","last_became_active":"2024-08-19T10:38:12.821074+0000","last_became_peered":"2024-08-19T10:38:12.821074+0000","last_unstale":"2024-08-19T10:38:14.838322+0000","last_undegraded":"2024-08-19T10:38:14.838322+0000","last_fullsized":"2024-08-19T10:38:14.838322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:36:05.704454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010645120000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.844887+0000","last_change":"2024-08-19T10:38:14.840443+0000","last_active":"2024-08-19T10:38:14.844887+0000","last_peered":"2024-08-19T10:38:14.844887+0000","last_clean":"2024-08-19T10:38:14.844887+0000","last_became_active":"2024-08-19T10:38:12.823338+0000","last_became_peered":"2024-08-19T10:38:12.823338+0000","last_unstale":"2024-08-19T10:38:14.844887+0000","last_undegraded":"2024-08-19T10:38:14.844887+0000","last_fullsized":"2024-08-19T10:38:14.844887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:21:50.030940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001115312,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.838406+0000","last_change":"2024-08-19T10:38:14.838406+0000","last_active":"2024-08-19T10:38:14.838406+0000","last_peered":"2024-08-19T10:38:14.838406+0000","last_clean":"2024-08-19T10:38:14.838406+0000","last_became_active":"2024-08-19T10:38:12.820746+0000","last_became_peered":"2024-08-19T10:38:12.820746+0000","last_unstale":"2024-08-19T10:38:14.838406+0000","last_undegraded":"2024-08-19T10:38:14.838406+0000","last_fullsized":"2024-08-19T10:38:14.838406+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:26:36.717033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001410375,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.839239+0000","last_change":"2024-08-19T10:38:14.839751+0000","last_active":"2024-08-19T10:38:14.839239+0000","last_peered":"2024-08-19T10:38:14.839239+0000","last_clean":"2024-08-19T10:38:14.839239+0000","last_became_active":"2024-08-19T10:38:12.819986+0000","last_became_peered":"2024-08-19T10:38:12.819986+0000","last_unstale":"2024-08-19T10:38:14.839239+0000","last_undegraded":"2024-08-19T10:38:14.839239+0000","last_fullsized":"2024-08-19T10:38:14.839239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:41:38.839856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010100350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.837363+0000","last_change":"2024-08-19T10:38:14.838122+0000","last_active":"2024-08-19T10:38:14.837363+0000","last_peered":"2024-08-19T10:38:14.837363+0000","last_clean":"2024-08-19T10:38:14.837363+0000","last_became_active":"2024-08-19T10:38:12.820601+0000","last_became_peered":"2024-08-19T10:38:12.820601+0000","last_unstale":"2024-08-19T10:38:14.837363+0000","last_undegraded":"2024-08-19T10:38:14.837363+0000","last_fullsized":"2024-08-19T10:38:14.837363+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:09:29.225317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002082918,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.835718+0000","last_change":"2024-08-19T10:38:06.130856+0000","last_active":"2024-08-19T10:38:14.835718+0000","last_peered":"2024-08-19T10:38:14.835718+0000","last_clean":"2024-08-19T10:38:14.835718+0000","last_became_active":"2024-08-19T10:38:06.129934+0000","last_became_peered":"2024-08-19T10:38:06.129934+0000","last_unstale":"2024-08-19T10:38:14.835718+0000","last_undegraded":"2024-08-19T10:38:14.835718+0000","last_fullsized":"2024-08-19T10:38:14.835718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_clean_scrub_stamp":"2024-08-19T10:38:04.611102+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:27:58.119234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":646777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607556,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":56428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607559,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":172032,"data_stored":56409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":56428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":172032,"data_stored":56409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":646777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T10:38:20.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T10:38:20.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.166+0000 7f38f2ec9640 10 monclient: get_monmap_and_config 2024-08-19T10:38:20.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.166+0000 7f38f2ec9640 10 monclient: build_initial_monmap 2024-08-19T10:38:20.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.166+0000 7f38f2ec9640 10 monclient: monmap: 2024-08-19T10:38:20.170 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:20.170 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.170 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:20.171637+0000 2024-08-19T10:38:20.170 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:20.171637+0000 2024-08-19T10:38:20.170 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:20.171 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.166+0000 7f38f2ec9640 1 Processor -- start 2024-08-19T10:38:20.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.166+0000 7f38f2ec9640 1 -- start start 2024-08-19T10:38:20.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.166+0000 7f38f2ec9640 10 monclient: init 2024-08-19T10:38:20.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:20.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 10 monclient: _add_conns ranks=[10,7,4,0,6,3,5,2,9,8,1] 2024-08-19T10:38:20.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f38ec0a5270 0x7f38ec0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 10 monclient(hunting): picked mon.noname-f con 0x7f38ec0a5270 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:20.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f38ec0a6d30 0x7f38ec0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 10 monclient(hunting): picked mon.noname-i con 0x7f38ec0a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec1395b0 0x7f38ec139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 10 monclient(hunting): picked mon.noname-c con 0x7f38ec1395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f38ec0a7650 con 0x7f38ec0a5270 2024-08-19T10:38:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f38ec0a8710 con 0x7f38ec1395b0 2024-08-19T10:38:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f38ec0a5a40 con 0x7f38ec0a6d30 2024-08-19T10:38:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f2ec9640 10 monclient(hunting): authenticate will time out at 949.988263s 2024-08-19T10:38:20.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec1395b0 0x7f38ec139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec1395b0 0x7f38ec139a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37798/0 (socket says 172.21.15.89:37798) 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 1 -- 172.21.15.89:0/3905398675 learned_addr learned my addr 172.21.15.89:0/3905398675 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 10 monclient(hunting): get_auth_request con 0x7f38ec1395b0 auth_method 0 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f0c65640 1 --2- 172.21.15.89:0/3905398675 >> v2:172.21.15.190:3300/0 conn(0x7f38ec0a5270 0x7f38ec0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38ebfff640 1 --2- 172.21.15.89:0/3905398675 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f38ec0a6d30 0x7f38ec0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38ebfff640 10 monclient(hunting): get_auth_request con 0x7f38ec0a6d30 auth_method 0 2024-08-19T10:38:20.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38ebfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 10 monclient(hunting): handle_auth_done global_id 4306 payload 274 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 1 -- 172.21.15.89:0/3905398675 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f38ec0a6d30 msgr2=0x7f38ec0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 1 --2- 172.21.15.89:0/3905398675 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f38ec0a6d30 0x7f38ec0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 1 -- 172.21.15.89:0/3905398675 >> v2:172.21.15.190:3300/0 conn(0x7f38ec0a5270 msgr2=0x7f38ec0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 1 --2- 172.21.15.89:0/3905398675 >> v2:172.21.15.190:3300/0 conn(0x7f38ec0a5270 0x7f38ec0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 10 monclient: _finish_hunting 0 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 1 monclient: found mon.noname-c 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:20.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f1466640 1 -- 172.21.15.89:0/3905398675 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec0a81d0 con 0x7f38ec1395b0 2024-08-19T10:38:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f0c65640 10 monclient: get_auth_request con 0x7f38ec0a5270 auth_method 0 2024-08-19T10:38:20.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.170+0000 7f38f0c65640 1 --2- 172.21.15.89:0/3905398675 >> v2:172.21.15.190:3300/0 conn(0x7f38ec0a5270 0x7f38ec0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f1466640 1 -- 172.21.15.89:0/3905398675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec1395b0 msgr2=0x7f38ec139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f1466640 1 --2- 172.21.15.89:0/3905398675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec1395b0 0x7f38ec139a40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009870 tx=0x7f38dc01c420 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f7df7aa4825f7171 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 1 -- 172.21.15.89:0/3905398675 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38dc0b1070 con 0x7f38ec1395b0 2024-08-19T10:38:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 10 monclient: dump: 2024-08-19T10:38:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.182 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:20.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 1 -- 172.21.15.89:0/3905398675 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38dc0ab810 con 0x7f38ec1395b0 2024-08-19T10:38:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 5 monclient: authenticate success, global_id 4306 2024-08-19T10:38:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 4 set_mon_vals no callback set 2024-08-19T10:38:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 1 -- 172.21.15.89:0/3905398675 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38dc0aa300 con 0x7f38ec1395b0 2024-08-19T10:38:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38eb7fe640 10 monclient: dump: 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 10 monclient: shutdown 2024-08-19T10:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 1 -- 172.21.15.89:0/3905398675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec1395b0 msgr2=0x7f38ec139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 1 --2- 172.21.15.89:0/3905398675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec1395b0 0x7f38ec139a40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009870 tx=0x7f38dc01c420 comp rx=0 tx=0).stop 2024-08-19T10:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 1 -- 172.21.15.89:0/3905398675 shutdown_connections 2024-08-19T10:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 1 --2- 172.21.15.89:0/3905398675 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec1395b0 0x7f38ec139a40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 1 --2- 172.21.15.89:0/3905398675 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f38ec0a6d30 0x7f38ec0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 1 --2- 172.21.15.89:0/3905398675 >> v2:172.21.15.190:3300/0 conn(0x7f38ec0a5270 0x7f38ec0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 1 -- 172.21.15.89:0/3905398675 >> 172.21.15.89:0/3905398675 conn(0x7f38ec069a80 msgr2=0x7f38ec0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 1 -- 172.21.15.89:0/3905398675 shutdown_connections 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 1 -- 172.21.15.89:0/3905398675 wait complete. 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 10 monclient: build_initial_monmap 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 10 monclient: monmap: 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.174+0000 7f38f2ec9640 1 Processor -- start 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 1 -- start start 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient: init 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient: _add_conns ranks=[4,8,2,5,1,0,7,6,3] 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec0a5270 0x7f38ec0a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient(hunting): picked mon.noname-c con 0x7f38ec0a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38ec0a6d30 0x7f38ec13e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient(hunting): picked mon.noname-e con 0x7f38ec0a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38ec1395b0 0x7f38ec13ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient(hunting): picked mon.noname-b con 0x7f38ec1395b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f38ec0a5650 con 0x7f38ec1395b0 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f38ec0a4a70 con 0x7f38ec0a5270 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f38ec0a57d0 con 0x7f38ec0a6d30 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient(hunting): authenticate will time out at 949.996680s 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f0c65640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec0a5270 0x7f38ec0a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f0c65640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec0a5270 0x7f38ec0a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37808/0 (socket says 172.21.15.89:37808) 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f0c65640 1 -- 172.21.15.89:0/2268101352 learned_addr learned my addr 172.21.15.89:0/2268101352 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f0c65640 10 monclient(hunting): get_auth_request con 0x7f38ec0a5270 auth_method 0 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f0c65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f0c65640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f0c65640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38ec1395b0 0x7f38ec13ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38ebfff640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38ec0a6d30 0x7f38ec13e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38ebfff640 10 monclient(hunting): get_auth_request con 0x7f38ec0a6d30 auth_method 0 2024-08-19T10:38:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38ebfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 10 monclient(hunting): get_auth_request con 0x7f38ec1395b0 auth_method 0 2024-08-19T10:38:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:20.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f0c65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f0c65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f0c65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 10 monclient(hunting): handle_auth_done global_id 4268 payload 824 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 1 -- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38ec0a6d30 msgr2=0x7f38ec13e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38ec0a6d30 0x7f38ec13e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 1 -- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec0a5270 msgr2=0x7f38ec0a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec0a5270 0x7f38ec0a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 10 monclient: _finish_hunting 0 2024-08-19T10:38:20.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 1 monclient: found mon.noname-b 2024-08-19T10:38:20.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:20.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:20.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 1 -- 172.21.15.89:0/2268101352 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec0aafe0 con 0x7f38ec1395b0 2024-08-19T10:38:20.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38ebfff640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38ec0a6d30 0x7f38ec13e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:20.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 1 -- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38ec1395b0 msgr2=0x7f38ec13ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:20.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f1466640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38ec1395b0 0x7f38ec13ebd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0ac000 tx=0x7f38dc0191c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:20.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 1 -- 172.21.15.89:0/2268101352 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38dc0b1040 con 0x7f38ec1395b0 2024-08-19T10:38:20.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 10 monclient: dump: 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 1 -- 172.21.15.89:0/2268101352 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38dc004250 con 0x7f38ec1395b0 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 5 monclient: authenticate success, global_id 4268 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient: _renew_subs 2024-08-19T10:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 1 -- 172.21.15.89:0/2268101352 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec135800 con 0x7f38ec1395b0 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 1 -- 172.21.15.89:0/2268101352 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38dc004400 con 0x7f38ec1395b0 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38e97fa640 10 monclient: dump: 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient: _renew_subs 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38ea7fc640 4 set_mon_vals no callback set 2024-08-19T10:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.178+0000 7f38f2ec9640 1 -- 172.21.15.89:0/2268101352 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec09f0a0 con 0x7f38ec1395b0 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.182+0000 7f38f2ec9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.182+0000 7f38f2ec9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.182+0000 7f38f2ec9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.182+0000 7f38f2ec9640 1 -- 172.21.15.89:0/2268101352 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ec0a7110 con 0x7f38ec1395b0 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.182+0000 7f38e97fa640 1 -- 172.21.15.89:0/2268101352 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f38dc003620 con 0x7f38ec1395b0 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.182+0000 7f38e97fa640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38d803bdf0 0x7f38d803e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.182+0000 7f38e97fa640 1 -- 172.21.15.89:0/2268101352 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4402+0+0 (secure 0 0 0) 0x7f38dc0e8100 con 0x7f38ec1395b0 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.182+0000 7f38f0c65640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38d803bdf0 0x7f38d803e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.182+0000 7f38f0c65640 10 monclient: get_auth_request con 0x7f38d803bdf0 auth_method 0 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.186+0000 7f38f0c65640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38d803bdf0 0x7f38d803e2b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38e400f5e0 tx=0x7f38e4007830 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:20.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.186+0000 7f38e97fa640 1 -- 172.21.15.89:0/2268101352 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f38dc016130 con 0x7f38ec1395b0 2024-08-19T10:38:20.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.186+0000 7f38e97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:20.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.186+0000 7f38e97fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:20.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.354+0000 7f38f2ec9640 1 -- 172.21.15.89:0/2268101352 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f38ec064590 con 0x7f38d803bdf0 2024-08-19T10:38:20.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.354+0000 7f38e97fa640 1 -- 172.21.15.89:0/2268101352 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34067 (secure 0 0 0) 0x7f38ec064590 con 0x7f38d803bdf0 2024-08-19T10:38:20.359 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:38:20.359 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-08-19T10:38:20.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 -- 172.21.15.89:0/2268101352 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38d803bdf0 msgr2=0x7f38d803e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38d803bdf0 0x7f38d803e2b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38e400f5e0 tx=0x7f38e4007830 comp rx=0 tx=0).stop 2024-08-19T10:38:20.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 10 monclient: shutdown 2024-08-19T10:38:20.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:20.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 -- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38ec1395b0 msgr2=0x7f38ec13ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38ec1395b0 0x7f38ec13ebd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0ac000 tx=0x7f38dc0191c0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 -- 172.21.15.89:0/2268101352 shutdown_connections 2024-08-19T10:38:20.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38ec1395b0 0x7f38ec13ebd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38d803bdf0 0x7f38d803e2b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38ec0a6d30 0x7f38ec13e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 --2- 172.21.15.89:0/2268101352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38ec0a5270 0x7f38ec0a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 -- 172.21.15.89:0/2268101352 >> 172.21.15.89:0/2268101352 conn(0x7f38ec069a80 msgr2=0x7f38ec13ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:20.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 -- 172.21.15.89:0/2268101352 shutdown_connections 2024-08-19T10:38:20.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.358+0000 7f38f2ec9640 1 -- 172.21.15.89:0/2268101352 wait complete. 2024-08-19T10:38:20.376 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-08-19T10:38:20.207548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164320,"kb_used_data":2176,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066720,"statfs":{"total":579820584960,"available":579652321280,"internally_reserved":0,"allocated":2228224,"data_stored":1519228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.398751"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.838257+0000","last_change":"2024-08-19T10:38:14.838742+0000","last_active":"2024-08-19T10:38:14.838257+0000","last_peered":"2024-08-19T10:38:14.838257+0000","last_clean":"2024-08-19T10:38:14.838257+0000","last_became_active":"2024-08-19T10:38:12.819810+0000","last_became_peered":"2024-08-19T10:38:12.819810+0000","last_unstale":"2024-08-19T10:38:14.838257+0000","last_undegraded":"2024-08-19T10:38:14.838257+0000","last_fullsized":"2024-08-19T10:38:14.838257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:10:48.036604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021153309999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.849314+0000","last_change":"2024-08-19T10:38:14.849789+0000","last_active":"2024-08-19T10:38:14.849314+0000","last_peered":"2024-08-19T10:38:14.849314+0000","last_clean":"2024-08-19T10:38:14.849314+0000","last_became_active":"2024-08-19T10:38:12.830800+0000","last_became_peered":"2024-08-19T10:38:12.830800+0000","last_unstale":"2024-08-19T10:38:14.849314+0000","last_undegraded":"2024-08-19T10:38:14.849314+0000","last_fullsized":"2024-08-19T10:38:14.849314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:25:14.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081542699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.837210+0000","last_change":"2024-08-19T10:38:14.839689+0000","last_active":"2024-08-19T10:38:14.837210+0000","last_peered":"2024-08-19T10:38:14.837210+0000","last_clean":"2024-08-19T10:38:14.837210+0000","last_became_active":"2024-08-19T10:38:12.825835+0000","last_became_peered":"2024-08-19T10:38:12.825835+0000","last_unstale":"2024-08-19T10:38:14.837210+0000","last_undegraded":"2024-08-19T10:38:14.837210+0000","last_fullsized":"2024-08-19T10:38:14.837210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:13:58.426106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003718322,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.838322+0000","last_change":"2024-08-19T10:38:14.838322+0000","last_active":"2024-08-19T10:38:14.838322+0000","last_peered":"2024-08-19T10:38:14.838322+0000","last_clean":"2024-08-19T10:38:14.838322+0000","last_became_active":"2024-08-19T10:38:12.821074+0000","last_became_peered":"2024-08-19T10:38:12.821074+0000","last_unstale":"2024-08-19T10:38:14.838322+0000","last_undegraded":"2024-08-19T10:38:14.838322+0000","last_fullsized":"2024-08-19T10:38:14.838322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:36:05.704454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010645120000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.844887+0000","last_change":"2024-08-19T10:38:14.840443+0000","last_active":"2024-08-19T10:38:14.844887+0000","last_peered":"2024-08-19T10:38:14.844887+0000","last_clean":"2024-08-19T10:38:14.844887+0000","last_became_active":"2024-08-19T10:38:12.823338+0000","last_became_peered":"2024-08-19T10:38:12.823338+0000","last_unstale":"2024-08-19T10:38:14.844887+0000","last_undegraded":"2024-08-19T10:38:14.844887+0000","last_fullsized":"2024-08-19T10:38:14.844887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:21:50.030940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001115312,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.838406+0000","last_change":"2024-08-19T10:38:14.838406+0000","last_active":"2024-08-19T10:38:14.838406+0000","last_peered":"2024-08-19T10:38:14.838406+0000","last_clean":"2024-08-19T10:38:14.838406+0000","last_became_active":"2024-08-19T10:38:12.820746+0000","last_became_peered":"2024-08-19T10:38:12.820746+0000","last_unstale":"2024-08-19T10:38:14.838406+0000","last_undegraded":"2024-08-19T10:38:14.838406+0000","last_fullsized":"2024-08-19T10:38:14.838406+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:26:36.717033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001410375,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.839239+0000","last_change":"2024-08-19T10:38:14.839751+0000","last_active":"2024-08-19T10:38:14.839239+0000","last_peered":"2024-08-19T10:38:14.839239+0000","last_clean":"2024-08-19T10:38:14.839239+0000","last_became_active":"2024-08-19T10:38:12.819986+0000","last_became_peered":"2024-08-19T10:38:12.819986+0000","last_unstale":"2024-08-19T10:38:14.839239+0000","last_undegraded":"2024-08-19T10:38:14.839239+0000","last_fullsized":"2024-08-19T10:38:14.839239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:41:38.839856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010100350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.837363+0000","last_change":"2024-08-19T10:38:14.838122+0000","last_active":"2024-08-19T10:38:14.837363+0000","last_peered":"2024-08-19T10:38:14.837363+0000","last_clean":"2024-08-19T10:38:14.837363+0000","last_became_active":"2024-08-19T10:38:12.820601+0000","last_became_peered":"2024-08-19T10:38:12.820601+0000","last_unstale":"2024-08-19T10:38:14.837363+0000","last_undegraded":"2024-08-19T10:38:14.837363+0000","last_fullsized":"2024-08-19T10:38:14.837363+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:09:29.225317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002082918,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.835718+0000","last_change":"2024-08-19T10:38:06.130856+0000","last_active":"2024-08-19T10:38:14.835718+0000","last_peered":"2024-08-19T10:38:14.835718+0000","last_clean":"2024-08-19T10:38:14.835718+0000","last_became_active":"2024-08-19T10:38:06.129934+0000","last_became_peered":"2024-08-19T10:38:06.129934+0000","last_unstale":"2024-08-19T10:38:14.835718+0000","last_undegraded":"2024-08-19T10:38:14.835718+0000","last_fullsized":"2024-08-19T10:38:14.835718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_clean_scrub_stamp":"2024-08-19T10:38:04.611102+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:27:58.119234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":646777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":56428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607559,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":172032,"data_stored":56409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":56428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":172032,"data_stored":56409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640263,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":646777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T10:38:20.377 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-19T10:38:20.377 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-19T10:38:20.377 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-08-19T10:38:20.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-08-19T10:38:20.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.478+0000 7f44b532e640 10 monclient: get_monmap_and_config 2024-08-19T10:38:20.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.478+0000 7f44b532e640 10 monclient: build_initial_monmap 2024-08-19T10:38:20.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.478+0000 7f44b532e640 10 monclient: monmap: 2024-08-19T10:38:20.483 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:20.483 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.483 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:20.484834+0000 2024-08-19T10:38:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:20.484834+0000 2024-08-19T10:38:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:20.484 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:20.485 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:20.485 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 1 Processor -- start 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 1 -- start start 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 10 monclient: init 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 10 monclient: _add_conns ranks=[10,6,4,3,7,1,8,2,5,0,9] 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f44b009eae0 0x7f44b009eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 10 monclient(hunting): picked mon.noname-f con 0x7f44b009eae0 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44b0069070 0x7f44b009dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 10 monclient(hunting): picked mon.noname-d con 0x7f44b0069070 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 0x7f44b00a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 10 monclient(hunting): picked mon.noname-c con 0x7f44b00a3d70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f44b009e200 con 0x7f44b009eae0 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f44b009c840 con 0x7f44b00a3d70 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f44b009d8a0 con 0x7f44b0069070 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44b532e640 10 monclient(hunting): authenticate will time out at 950.301684s 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44b0069070 0x7f44b009dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44b0069070 0x7f44b009dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46506/0 (socket says 172.21.15.89:46506) 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 1 -- 172.21.15.89:0/1388359802 learned_addr learned my addr 172.21.15.89:0/1388359802 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 10 monclient(hunting): get_auth_request con 0x7f44b0069070 auth_method 0 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44af7fe640 1 --2- 172.21.15.89:0/1388359802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 0x7f44b00a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44aeffd640 1 --2- 172.21.15.89:0/1388359802 >> v2:172.21.15.190:3300/0 conn(0x7f44b009eae0 0x7f44b009eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44aeffd640 10 monclient(hunting): get_auth_request con 0x7f44b009eae0 auth_method 0 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44aeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44af7fe640 10 monclient(hunting): get_auth_request con 0x7f44b00a3d70 auth_method 0 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44af7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 10 monclient(hunting): handle_auth_done global_id 4290 payload 274 2024-08-19T10:38:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 1 -- 172.21.15.89:0/1388359802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 msgr2=0x7f44b00a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 1 --2- 172.21.15.89:0/1388359802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 0x7f44b00a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 1 -- 172.21.15.89:0/1388359802 >> v2:172.21.15.190:3300/0 conn(0x7f44b009eae0 msgr2=0x7f44b009eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 1 --2- 172.21.15.89:0/1388359802 >> v2:172.21.15.190:3300/0 conn(0x7f44b009eae0 0x7f44b009eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:38:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 1 monclient: found mon.noname-d 2024-08-19T10:38:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:20.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.482+0000 7f44ae7fc640 1 -- 172.21.15.89:0/1388359802 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b00694f0 con 0x7f44b0069070 2024-08-19T10:38:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44ae7fc640 1 -- 172.21.15.89:0/1388359802 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44b0069070 msgr2=0x7f44b009dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44ae7fc640 1 --2- 172.21.15.89:0/1388359802 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44b0069070 0x7f44b009dcc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009870 tx=0x7f44a401c420 comp rx=0 tx=0).ready entity=mon.6 client_cookie=23a821ebd7f5f872 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 1 -- 172.21.15.89:0/1388359802 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44a40b0070 con 0x7f44b0069070 2024-08-19T10:38:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 10 monclient: dump: 2024-08-19T10:38:20.494 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.495 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 1 -- 172.21.15.89:0/1388359802 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a40aa810 con 0x7f44b0069070 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 5 monclient: authenticate success, global_id 4290 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 1 -- 172.21.15.89:0/1388359802 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44a40a9300 con 0x7f44b0069070 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 4 set_mon_vals no callback set 2024-08-19T10:38:20.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44adffb640 10 monclient: dump: 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.497 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 10 monclient: shutdown 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 1 -- 172.21.15.89:0/1388359802 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44b0069070 msgr2=0x7f44b009dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 1 --2- 172.21.15.89:0/1388359802 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44b0069070 0x7f44b009dcc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009870 tx=0x7f44a401c420 comp rx=0 tx=0).stop 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 1 -- 172.21.15.89:0/1388359802 shutdown_connections 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 1 --2- 172.21.15.89:0/1388359802 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 0x7f44b00a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 1 --2- 172.21.15.89:0/1388359802 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44b0069070 0x7f44b009dcc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 1 --2- 172.21.15.89:0/1388359802 >> v2:172.21.15.190:3300/0 conn(0x7f44b009eae0 0x7f44b009eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 1 -- 172.21.15.89:0/1388359802 >> 172.21.15.89:0/1388359802 conn(0x7f44b00a4e60 msgr2=0x7f44b00a5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 1 -- 172.21.15.89:0/1388359802 shutdown_connections 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 1 -- 172.21.15.89:0/1388359802 wait complete. 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 10 monclient: build_initial_monmap 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.486+0000 7f44b532e640 10 monclient: monmap: 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.499 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 1 Processor -- start 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 1 -- start start 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient: init 2024-08-19T10:38:20.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient: _add_conns ranks=[7,4,0,6,8,3,1,5,2] 2024-08-19T10:38:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44b009eae0 0x7f44b009abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient(hunting): picked mon.noname-i con 0x7f44b009eae0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 0x7f44b00954e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient(hunting): picked mon.noname-c con 0x7f44b00a3d70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44b013fea0 0x7f44b0095a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient(hunting): picked mon.noname-a con 0x7f44b013fea0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:20.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0142ec0 con 0x7f44b013fea0 2024-08-19T10:38:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f44b0142d40 con 0x7f44b00a3d70 2024-08-19T10:38:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f44b0143040 con 0x7f44b009eae0 2024-08-19T10:38:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient(hunting): authenticate will time out at 950.309051s 2024-08-19T10:38:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 0x7f44b00954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 0x7f44b00954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37824/0 (socket says 172.21.15.89:37824) 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 1 -- 172.21.15.89:0/383889998 learned_addr learned my addr 172.21.15.89:0/383889998 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 10 monclient(hunting): get_auth_request con 0x7f44b00a3d70 auth_method 0 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44af7fe640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44b013fea0 0x7f44b0095a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44af7fe640 10 monclient(hunting): get_auth_request con 0x7f44b013fea0 auth_method 0 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44af7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:20.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 10 monclient(hunting): handle_auth_done global_id 4333 payload 824 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44aeffd640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44b009eae0 0x7f44b009abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 1 -- 172.21.15.89:0/383889998 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44b009eae0 msgr2=0x7f44b009abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44b009eae0 0x7f44b009abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 1 -- 172.21.15.89:0/383889998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44b013fea0 msgr2=0x7f44b0095a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44b013fea0 0x7f44b0095a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:38:20.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 1 monclient: found mon.noname-c 2024-08-19T10:38:20.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:20.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:20.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 1 -- 172.21.15.89:0/383889998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b0099200 con 0x7f44b00a3d70 2024-08-19T10:38:20.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 1 -- 172.21.15.89:0/383889998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 msgr2=0x7f44b00954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:20.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ae7fc640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 0x7f44b00954e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f44a40ae040 tx=0x7f44a40191c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:20.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44977fe640 1 -- 172.21.15.89:0/383889998 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44a40b0040 con 0x7f44b00a3d70 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44977fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44977fe640 10 monclient: dump: 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.508 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44977fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44977fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 5 monclient: authenticate success, global_id 4333 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44977fe640 1 -- 172.21.15.89:0/383889998 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a40044b0 con 0x7f44b00a3d70 2024-08-19T10:38:20.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44977fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient: _renew_subs 2024-08-19T10:38:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44977fe640 1 -- 172.21.15.89:0/383889998 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44a4003a90 con 0x7f44b00a3d70 2024-08-19T10:38:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44b532e640 1 -- 172.21.15.89:0/383889998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b00a34b0 con 0x7f44b00a3d70 2024-08-19T10:38:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.490+0000 7f44ad7fa640 4 set_mon_vals no callback set 2024-08-19T10:38:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44977fe640 10 monclient: dump: 2024-08-19T10:38:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.510 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.511 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.512 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.512 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44b532e640 10 monclient: _renew_subs 2024-08-19T10:38:20.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44b532e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:20.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44b532e640 1 -- 172.21.15.89:0/383889998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b0096ea0 con 0x7f44b00a3d70 2024-08-19T10:38:20.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44b532e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:20.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44b532e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:20.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44b532e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44b532e640 1 -- 172.21.15.89:0/383889998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b009eec0 con 0x7f44b00a3d70 2024-08-19T10:38:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44977fe640 1 -- 172.21.15.89:0/383889998 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f44a4003c50 con 0x7f44b00a3d70 2024-08-19T10:38:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44977fe640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f449c03be00 0x7f449c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44977fe640 1 -- 172.21.15.89:0/383889998 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f44a40e7ca0 con 0x7f44b00a3d70 2024-08-19T10:38:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44aeffd640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f449c03be00 0x7f449c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.494+0000 7f44aeffd640 10 monclient: get_auth_request con 0x7f449c03be00 auth_method 0 2024-08-19T10:38:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.498+0000 7f44aeffd640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f449c03be00 0x7f449c03e2c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009790 tx=0x7f44a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.498+0000 7f44977fe640 1 -- 172.21.15.89:0/383889998 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f44a4016130 con 0x7f44b00a3d70 2024-08-19T10:38:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.498+0000 7f44977fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:20.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.498+0000 7f44977fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.686+0000 7f44b532e640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2024-08-19T10:38:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.686+0000 7f44b532e640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2024-08-19T10:38:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.686+0000 7f44b532e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.686+0000 7f44b532e640 1 -- 172.21.15.89:0/383889998 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f44b0064590 con 0x7f44b00a3d70 2024-08-19T10:38:20.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44977fe640 1 -- 172.21.15.89:0/383889998 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f44a40bbca0 con 0x7f44b00a3d70 2024-08-19T10:38:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44977fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2024-08-19T10:38:20.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44977fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:20.692 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:38:20.692 INFO:teuthology.orchestra.run.smithi089.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-19T10:38:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 -- 172.21.15.89:0/383889998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f449c03be00 msgr2=0x7f449c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f449c03be00 0x7f449c03e2c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009790 tx=0x7f44a0008040 comp rx=0 tx=0).stop 2024-08-19T10:38:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 10 monclient: shutdown 2024-08-19T10:38:20.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 -- 172.21.15.89:0/383889998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 msgr2=0x7f44b00954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 0x7f44b00954e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f44a40ae040 tx=0x7f44a40191c0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 -- 172.21.15.89:0/383889998 shutdown_connections 2024-08-19T10:38:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f449c03be00 0x7f449c03e2c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44b013fea0 0x7f44b0095a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44b00a3d70 0x7f44b00954e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 --2- 172.21.15.89:0/383889998 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44b009eae0 0x7f44b009abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 -- 172.21.15.89:0/383889998 >> 172.21.15.89:0/383889998 conn(0x7f44b00a4e60 msgr2=0x7f44b00a04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 -- 172.21.15.89:0/383889998 shutdown_connections 2024-08-19T10:38:20.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.690+0000 7f44b532e640 1 -- 172.21.15.89:0/383889998 wait complete. 2024-08-19T10:38:20.709 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-08-19T10:38:20.710 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-08-19T10:38:20.726 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-08-19T10:38:20.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T10:38:20.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.838+0000 7f849f247640 10 monclient: get_monmap_and_config 2024-08-19T10:38:20.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.838+0000 7f849f247640 10 monclient: build_initial_monmap 2024-08-19T10:38:20.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.838+0000 7f849f247640 10 monclient: monmap: 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:20.844069+0000 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:20.844069+0000 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:20.843 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:20.844 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:20.844 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:20.844 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:20.844 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:20.844 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:20.844 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 1 Processor -- start 2024-08-19T10:38:20.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 1 -- start start 2024-08-19T10:38:20.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 10 monclient: init 2024-08-19T10:38:20.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:20.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 10 monclient: _add_conns ranks=[4,3,1,6,2,7,10,8,5,0,9] 2024-08-19T10:38:20.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84980a5270 0x7f84980a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 10 monclient(hunting): picked mon.noname-c con 0x7f84980a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:20.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84980a6d30 0x7f84980a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 10 monclient(hunting): picked mon.noname-g con 0x7f84980a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:20.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f84981395b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:38:20.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84980a8710 con 0x7f84981395b0 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f84980a7650 con 0x7f84980a6d30 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f84980a5a40 con 0x7f84980a5270 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849f247640 10 monclient(hunting): authenticate will time out at 950.661171s 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84980a5270 0x7f84980a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84980a5270 0x7f84980a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37838/0 (socket says 172.21.15.89:37838) 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 1 -- 172.21.15.89:0/2907888664 learned_addr learned my addr 172.21.15.89:0/2907888664 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 10 monclient(hunting): get_auth_request con 0x7f84980a5270 auth_method 0 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f8497fff640 1 --2- 172.21.15.89:0/2907888664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84980a6d30 0x7f84980a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 10 monclient(hunting): handle_auth_done global_id 4342 payload 274 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 1 -- 172.21.15.89:0/2907888664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84980a6d30 msgr2=0x7f84980a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 1 --2- 172.21.15.89:0/2907888664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84980a6d30 0x7f84980a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 1 -- 172.21.15.89:0/2907888664 >> v1:172.21.15.190:6789/0 conn(0x7f84981395b0 legacy=0x7f8498139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 10 monclient: _finish_hunting 0 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 1 monclient: found mon.noname-c 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:20.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f849cfe3640 1 -- 172.21.15.89:0/2907888664 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84980a81d0 con 0x7f84980a5270 2024-08-19T10:38:20.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f8497fff640 10 monclient: get_auth_request con 0x7f84980a6d30 auth_method 0 2024-08-19T10:38:20.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.842+0000 7f8497fff640 1 --2- 172.21.15.89:0/2907888664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84980a6d30 0x7f84980a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:20.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849cfe3640 1 -- 172.21.15.89:0/2907888664 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84980a5270 msgr2=0x7f84980a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:20.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849cfe3640 1 --2- 172.21.15.89:0/2907888664 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84980a5270 0x7f84980a67f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f84880098a0 tx=0x7f848801c260 comp rx=0 tx=0).ready entity=mon.4 client_cookie=44819a811f87f928 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:20.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 1 -- 172.21.15.89:0/2907888664 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84880b0070 con 0x7f84980a5270 2024-08-19T10:38:20.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:20.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 10 monclient: dump: 2024-08-19T10:38:20.854 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.854 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.854 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.854 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.854 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.854 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.854 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.855 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.855 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.855 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.855 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.855 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.855 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.855 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.855 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.855 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 1 -- 172.21.15.89:0/2907888664 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84880aa810 con 0x7f84980a5270 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 1 -- 172.21.15.89:0/2907888664 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84880aae30 con 0x7f84980a5270 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 5 monclient: authenticate success, global_id 4342 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 4 set_mon_vals no callback set 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f84977fe640 10 monclient: dump: 2024-08-19T10:38:20.856 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.857 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.858 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.858 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 10 monclient: shutdown 2024-08-19T10:38:20.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:20.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 1 -- 172.21.15.89:0/2907888664 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84980a5270 msgr2=0x7f84980a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 1 --2- 172.21.15.89:0/2907888664 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84980a5270 0x7f84980a67f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f84880098a0 tx=0x7f848801c260 comp rx=0 tx=0).stop 2024-08-19T10:38:20.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 1 -- 172.21.15.89:0/2907888664 shutdown_connections 2024-08-19T10:38:20.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 1 --2- 172.21.15.89:0/2907888664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84980a6d30 0x7f84980a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 1 --2- 172.21.15.89:0/2907888664 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84980a5270 0x7f84980a67f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 1 -- 172.21.15.89:0/2907888664 >> 172.21.15.89:0/2907888664 conn(0x7f8498069a80 msgr2=0x7f84980abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:20.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 1 -- 172.21.15.89:0/2907888664 shutdown_connections 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.846+0000 7f849f247640 1 -- 172.21.15.89:0/2907888664 wait complete. 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient: build_initial_monmap 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient: monmap: 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:20.859 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 1 Processor -- start 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 1 -- start start 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient: init 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient: _add_conns ranks=[6,2,7,5,0,4,3,8,1] 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84980a5270 0x7f8498137e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient(hunting): picked mon.noname-d con 0x7f84980a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84980a6d30 0x7f8498138380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient(hunting): picked mon.noname-b con 0x7f84980a6d30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84981395b0 0x7f849813d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient(hunting): picked mon.noname-i con 0x7f84981395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f84980a57d0 con 0x7f84980a6d30 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f84980a5650 con 0x7f84980a5270 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8498144240 con 0x7f84981395b0 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:20.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 10 monclient(hunting): authenticate will time out at 950.668988s 2024-08-19T10:38:20.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84980a6d30 0x7f8498138380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84980a6d30 0x7f8498138380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44486/0 (socket says 172.21.15.89:44486) 2024-08-19T10:38:20.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 1 -- 172.21.15.89:0/1293190503 learned_addr learned my addr 172.21.15.89:0/1293190503 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:20.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 10 monclient(hunting): get_auth_request con 0x7f84980a6d30 auth_method 0 2024-08-19T10:38:20.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849cfe3640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84980a5270 0x7f8498137e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:20.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:20.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849cfe3640 10 monclient(hunting): get_auth_request con 0x7f84980a5270 auth_method 0 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849cfe3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849cfe3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849cfe3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849d7e4640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84981395b0 0x7f849813d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 10 monclient(hunting): handle_auth_done global_id 4277 payload 824 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 1 -- 172.21.15.89:0/1293190503 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84981395b0 msgr2=0x7f849813d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84981395b0 0x7f849813d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 1 -- 172.21.15.89:0/1293190503 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84980a5270 msgr2=0x7f8498137e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84980a5270 0x7f8498137e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 1 monclient: found mon.noname-b 2024-08-19T10:38:20.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:20.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:20.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 1 -- 172.21.15.89:0/1293190503 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8498122850 con 0x7f84980a6d30 2024-08-19T10:38:20.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849d7e4640 10 monclient: get_auth_request con 0x7f84981395b0 auth_method 0 2024-08-19T10:38:20.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849cfe3640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84980a5270 0x7f8498137e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:38:20.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849d7e4640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84981395b0 0x7f849813d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 1 -- 172.21.15.89:0/1293190503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84980a6d30 msgr2=0x7f8498138380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8497fff640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84980a6d30 0x7f8498138380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f848c00c7c0 tx=0x7f848c00cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 1 -- 172.21.15.89:0/1293190503 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f848c0113f0 con 0x7f84980a6d30 2024-08-19T10:38:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 10 monclient: dump: 2024-08-19T10:38:20.866 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.867 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f849f247640 5 monclient: authenticate success, global_id 4277 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 1 -- 172.21.15.89:0/1293190503 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848c0115d0 con 0x7f84980a6d30 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 1 -- 172.21.15.89:0/1293190503 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f848c011bf0 con 0x7f84980a6d30 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f8496ffd640 4 set_mon_vals no callback set 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.850+0000 7f84957fa640 10 monclient: dump: 2024-08-19T10:38:20.868 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:20.869 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:20.870 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:20.870 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:20.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849f247640 10 monclient: _renew_subs 2024-08-19T10:38:20.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849f247640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:20.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849f247640 1 -- 172.21.15.89:0/1293190503 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8498122b30 con 0x7f84980a6d30 2024-08-19T10:38:20.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849f247640 10 monclient: _renew_subs 2024-08-19T10:38:20.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849f247640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:20.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849f247640 1 -- 172.21.15.89:0/1293190503 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8498122e10 con 0x7f84980a6d30 2024-08-19T10:38:20.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849f247640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:20.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849f247640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:20.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849f247640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:20.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f84957fa640 1 -- 172.21.15.89:0/1293190503 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f848c011780 con 0x7f84980a6d30 2024-08-19T10:38:20.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849f247640 1 -- 172.21.15.89:0/1293190503 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84980a67f0 con 0x7f84980a6d30 2024-08-19T10:38:20.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f84957fa640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848403be00 0x7f848403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:20.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f84957fa640 1 -- 172.21.15.89:0/1293190503 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f848c04eac0 con 0x7f84980a6d30 2024-08-19T10:38:20.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849cfe3640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848403be00 0x7f848403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:20.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.854+0000 7f849cfe3640 10 monclient: get_auth_request con 0x7f848403be00 auth_method 0 2024-08-19T10:38:20.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.858+0000 7f849cfe3640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848403be00 0x7f848403e2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84880ab000 tx=0x7f84880a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:20.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.858+0000 7f84957fa640 1 -- 172.21.15.89:0/1293190503 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f848c015020 con 0x7f84980a6d30 2024-08-19T10:38:20.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.858+0000 7f84957fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:20.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:20.858+0000 7f84957fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:21.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.018+0000 7f849f247640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:21.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.018+0000 7f849f247640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:21.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.018+0000 7f849f247640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:21.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.018+0000 7f849f247640 1 -- 172.21.15.89:0/1293190503 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8498064590 con 0x7f84980a6d30 2024-08-19T10:38:21.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.018+0000 7f84957fa640 1 -- 172.21.15.89:0/1293190503 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12360 (secure 0 0 0) 0x7f848c04d650 con 0x7f84980a6d30 2024-08-19T10:38:21.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.018+0000 7f84957fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:38:21.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.018+0000 7f84957fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:21.022 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:38:21.022 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":19,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","created":"2024-08-19T10:37:50.906618+0000","modified":"2024-08-19T10:38:20.196421+0000","last_up_change":"2024-08-19T10:38:04.611102+0000","last_in_change":"2024-08-19T10:37:55.695518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T10:38:04.494964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-19T10:38:11.600504+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2be543b2-eb94-4907-8f21-6e91b7dd9733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6817","nonce":2476057681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6819","nonce":2476057681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6823","nonce":2476057681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6821","nonce":2476057681}]},"public_addr":"172.21.15.89:6817/2476057681","cluster_addr":"172.21.15.89:6819/2476057681","heartbeat_back_addr":"172.21.15.89:6823/2476057681","heartbeat_front_addr":"172.21.15.89:6821/2476057681","state":["exists","up"]},{"osd":1,"uuid":"224c1713-25c8-4a48-b16e-8929fcb6424b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6801","nonce":375502586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6803","nonce":375502586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6807","nonce":375502586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6805","nonce":375502586}]},"public_addr":"172.21.15.89:6801/375502586","cluster_addr":"172.21.15.89:6803/375502586","heartbeat_back_addr":"172.21.15.89:6807/375502586","heartbeat_front_addr":"172.21.15.89:6805/375502586","state":["exists","up"]},{"osd":2,"uuid":"5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6809","nonce":1060450292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6811","nonce":1060450292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6815","nonce":1060450292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6813","nonce":1060450292}]},"public_addr":"172.21.15.89:6809/1060450292","cluster_addr":"172.21.15.89:6811/1060450292","heartbeat_back_addr":"172.21.15.89:6815/1060450292","heartbeat_front_addr":"172.21.15.89:6813/1060450292","state":["exists","up"]},{"osd":3,"uuid":"2dcaafe0-5837-410b-ae99-c7561ef6b150","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6811","nonce":2751619171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6813","nonce":2751619171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6817","nonce":2751619171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6815","nonce":2751619171}]},"public_addr":"172.21.15.190:6811/2751619171","cluster_addr":"172.21.15.190:6813/2751619171","heartbeat_back_addr":"172.21.15.190:6817/2751619171","heartbeat_front_addr":"172.21.15.190:6815/2751619171","state":["exists","up"]},{"osd":4,"uuid":"5619c403-27fd-48de-8cee-c5e84289cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6819","nonce":1174298329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6821","nonce":1174298329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6825","nonce":1174298329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6823","nonce":1174298329}]},"public_addr":"172.21.15.190:6819/1174298329","cluster_addr":"172.21.15.190:6821/1174298329","heartbeat_back_addr":"172.21.15.190:6825/1174298329","heartbeat_front_addr":"172.21.15.190:6823/1174298329","state":["exists","up"]},{"osd":5,"uuid":"82ab73a3-1212-4b0d-ab05-aa9c01dd5586","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6803","nonce":2434772607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6805","nonce":2434772607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6809","nonce":2434772607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6807","nonce":2434772607}]},"public_addr":"172.21.15.190:6803/2434772607","cluster_addr":"172.21.15.190:6805/2434772607","heartbeat_back_addr":"172.21.15.190:6809/2434772607","heartbeat_front_addr":"172.21.15.190:6807/2434772607","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.511833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:01.832703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.351820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.269178+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.511490+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.178520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T10:38:21.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 -- 172.21.15.89:0/1293190503 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848403be00 msgr2=0x7f848403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848403be00 0x7f848403e2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84880ab000 tx=0x7f84880a6000 comp rx=0 tx=0).stop 2024-08-19T10:38:21.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 10 monclient: shutdown 2024-08-19T10:38:21.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:21.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 -- 172.21.15.89:0/1293190503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84980a6d30 msgr2=0x7f8498138380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84980a6d30 0x7f8498138380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f848c00c7c0 tx=0x7f848c00cc80 comp rx=0 tx=0).stop 2024-08-19T10:38:21.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 -- 172.21.15.89:0/1293190503 shutdown_connections 2024-08-19T10:38:21.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848403be00 0x7f848403e2c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84981395b0 0x7f849813d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84980a6d30 0x7f8498138380 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 --2- 172.21.15.89:0/1293190503 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84980a5270 0x7f8498137e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 -- 172.21.15.89:0/1293190503 >> 172.21.15.89:0/1293190503 conn(0x7f8498069a80 msgr2=0x7f84980a9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:21.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 -- 172.21.15.89:0/1293190503 shutdown_connections 2024-08-19T10:38:21.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.022+0000 7f849f247640 1 -- 172.21.15.89:0/1293190503 wait complete. 2024-08-19T10:38:21.040 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-19T10:38:04.494964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-08-19T10:38:11.600504+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-19T10:38:21.040 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-19T10:38:21.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.154+0000 7f0db246b640 10 monclient: get_monmap_and_config 2024-08-19T10:38:21.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.154+0000 7f0db246b640 10 monclient: build_initial_monmap 2024-08-19T10:38:21.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.154+0000 7f0db246b640 10 monclient: monmap: 2024-08-19T10:38:21.158 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:21.158 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.158 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:21.159846+0000 2024-08-19T10:38:21.159 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:21.159846+0000 2024-08-19T10:38:21.159 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:21.159 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.159 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:21.159 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:21.159 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:21.159 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:21.159 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:21.159 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:21.160 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:21.160 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:21.160 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:21.160 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:21.160 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:21.160 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 1 Processor -- start 2024-08-19T10:38:21.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 1 -- start start 2024-08-19T10:38:21.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 10 monclient: init 2024-08-19T10:38:21.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:21.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 10 monclient: _add_conns ranks=[2,5,0,9,6,10,3,7,4,8,1] 2024-08-19T10:38:21.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0dac0a5270 0x7f0dac0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 10 monclient(hunting): picked mon.noname-b con 0x7f0dac0a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:21.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0dac0a6d30 0x7f0dac0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 10 monclient(hunting): picked mon.noname-h con 0x7f0dac0a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:21.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0dac1395b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:21.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dac0a8710 con 0x7f0dac1395b0 2024-08-19T10:38:21.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f0dac0a7650 con 0x7f0dac0a5270 2024-08-19T10:38:21.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0dac0a5a40 con 0x7f0dac0a6d30 2024-08-19T10:38:21.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:21.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db246b640 10 monclient(hunting): authenticate will time out at 950.976868s 2024-08-19T10:38:21.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0dac0a5270 0x7f0dac0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db0a08640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0dac1395b0 0x7f0dac139a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54184/0 (socket says 172.21.15.89:54184) 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0db0a08640 1 -- 172.21.15.89:0/2350278339 learned_addr learned my addr 172.21.15.89:0/2350278339 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 10 monclient(hunting): get_auth_request con 0x7f0dac0a5270 auth_method 0 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0da37fe640 1 --2- 172.21.15.89:0/2350278339 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0dac0a6d30 0x7f0dac0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 10 monclient(hunting): handle_auth_done global_id 4286 payload 274 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 1 -- 172.21.15.89:0/2350278339 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0dac0a6d30 msgr2=0x7f0dac0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 1 --2- 172.21.15.89:0/2350278339 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0dac0a6d30 0x7f0dac0a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 1 -- 172.21.15.89:0/2350278339 >> v1:172.21.15.89:6789/0 conn(0x7f0dac1395b0 legacy=0x7f0dac139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:21.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:21.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 1 monclient: found mon.noname-b 2024-08-19T10:38:21.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:21.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:21.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.158+0000 7f0dabfff640 1 -- 172.21.15.89:0/2350278339 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac0a81d0 con 0x7f0dac0a5270 2024-08-19T10:38:21.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dabfff640 1 -- 172.21.15.89:0/2350278339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0dac0a5270 msgr2=0x7f0dac0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:21.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dabfff640 1 --2- 172.21.15.89:0/2350278339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0dac0a5270 0x7f0dac0a67f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0097f0 tx=0x7f0d9c01c1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce9d00df90d80387 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:21.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 1 -- 172.21.15.89:0/2350278339 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0d9c0b0070 con 0x7f0dac0a5270 2024-08-19T10:38:21.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:21.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 10 monclient: dump: 2024-08-19T10:38:21.168 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.168 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.168 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.168 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.168 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.168 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.168 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.168 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.169 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.169 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.169 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.169 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.169 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.169 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.169 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.169 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:21.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:21.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:21.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 1 -- 172.21.15.89:0/2350278339 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0aa810 con 0x7f0dac0a5270 2024-08-19T10:38:21.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:21.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 1 -- 172.21.15.89:0/2350278339 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0d9c0a93e0 con 0x7f0dac0a5270 2024-08-19T10:38:21.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 5 monclient: authenticate success, global_id 4286 2024-08-19T10:38:21.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:21.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 4 set_mon_vals no callback set 2024-08-19T10:38:21.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:21.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0dab7fe640 10 monclient: dump: 2024-08-19T10:38:21.172 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.172 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.172 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.172 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.173 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.173 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.173 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.173 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.173 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.173 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.173 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.173 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.173 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.174 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.174 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.174 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 10 monclient: shutdown 2024-08-19T10:38:21.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:21.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 1 -- 172.21.15.89:0/2350278339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0dac0a5270 msgr2=0x7f0dac0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 1 --2- 172.21.15.89:0/2350278339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0dac0a5270 0x7f0dac0a67f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0097f0 tx=0x7f0d9c01c1c0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 1 -- 172.21.15.89:0/2350278339 shutdown_connections 2024-08-19T10:38:21.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 1 --2- 172.21.15.89:0/2350278339 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0dac0a6d30 0x7f0dac0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 1 --2- 172.21.15.89:0/2350278339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0dac0a5270 0x7f0dac0a67f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 1 -- 172.21.15.89:0/2350278339 >> 172.21.15.89:0/2350278339 conn(0x7f0dac069a80 msgr2=0x7f0dac0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:21.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 1 -- 172.21.15.89:0/2350278339 shutdown_connections 2024-08-19T10:38:21.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 1 -- 172.21.15.89:0/2350278339 wait complete. 2024-08-19T10:38:21.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 10 monclient: build_initial_monmap 2024-08-19T10:38:21.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 10 monclient: monmap: 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:21.176 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:21.177 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:21.177 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:21.177 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.162+0000 7f0db246b640 1 Processor -- start 2024-08-19T10:38:21.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 1 -- start start 2024-08-19T10:38:21.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient: init 2024-08-19T10:38:21.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:21.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient: _add_conns ranks=[4,0,3,1,2,5,8,7,6] 2024-08-19T10:38:21.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0dac0a5270 0x7f0dac1d1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient(hunting): picked mon.noname-c con 0x7f0dac0a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:21.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dac0a6d30 0x7f0dac138650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient(hunting): picked mon.noname-a con 0x7f0dac0a6d30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:21.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dac1395b0 0x7f0dac13d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient(hunting): picked mon.noname-g con 0x7f0dac1395b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:21.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac0a57d0 con 0x7f0dac0a6d30 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0dac0a5650 con 0x7f0dac1395b0 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f0dac144240 con 0x7f0dac0a5270 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient(hunting): authenticate will time out at 950.984483s 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dac0a6d30 0x7f0dac138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0dac0a5270 0x7f0dac1d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dac0a6d30 0x7f0dac138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57844/0 (socket says 172.21.15.89:57844) 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0dac0a5270 0x7f0dac1d1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37844/0 (socket says 172.21.15.89:37844) 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da37fe640 1 -- 172.21.15.89:0/689957638 learned_addr learned my addr 172.21.15.89:0/689957638 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da37fe640 10 monclient(hunting): get_auth_request con 0x7f0dac0a6d30 auth_method 0 2024-08-19T10:38:21.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 10 monclient(hunting): get_auth_request con 0x7f0dac0a5270 auth_method 0 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:21.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db0a08640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dac1395b0 0x7f0dac13d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db0a08640 10 monclient(hunting): get_auth_request con 0x7f0dac1395b0 auth_method 0 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db0a08640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db0a08640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db0a08640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 10 monclient(hunting): handle_auth_done global_id 4351 payload 824 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 1 -- 172.21.15.89:0/689957638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dac1395b0 msgr2=0x7f0dac13d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dac1395b0 0x7f0dac13d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 1 -- 172.21.15.89:0/689957638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dac0a6d30 msgr2=0x7f0dac138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dac0a6d30 0x7f0dac138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 1 monclient: found mon.noname-c 2024-08-19T10:38:21.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 1 -- 172.21.15.89:0/689957638 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac122fe0 con 0x7f0dac0a5270 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da37fe640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dac0a6d30 0x7f0dac138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 1 -- 172.21.15.89:0/689957638 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0dac0a5270 msgr2=0x7f0dac1d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0dabfff640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0dac0a5270 0x7f0dac1d1230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0a34c0 tx=0x7f0d9c018f60 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 1 -- 172.21.15.89:0/689957638 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0d9c0b9070 con 0x7f0dac0a5270 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 10 monclient: dump: 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.186 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.187 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:21.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:21.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:21.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 1 -- 172.21.15.89:0/689957638 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c004240 con 0x7f0dac0a5270 2024-08-19T10:38:21.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 5 monclient: authenticate success, global_id 4351 2024-08-19T10:38:21.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient: _renew_subs 2024-08-19T10:38:21.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:21.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 1 -- 172.21.15.89:0/689957638 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac0aadd0 con 0x7f0dac0a5270 2024-08-19T10:38:21.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:21.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 1 -- 172.21.15.89:0/689957638 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0d9c0b0040 con 0x7f0dac0a5270 2024-08-19T10:38:21.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:21.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0daaffd640 4 set_mon_vals no callback set 2024-08-19T10:38:21.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0da97fa640 10 monclient: dump: 2024-08-19T10:38:21.189 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.189 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.189 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.189 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.189 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.189 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.189 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient: _renew_subs 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:21.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.166+0000 7f0db246b640 1 -- 172.21.15.89:0/689957638 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac0a71c0 con 0x7f0dac0a5270 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.170+0000 7f0db246b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.170+0000 7f0db246b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.170+0000 7f0db246b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.170+0000 7f0db246b640 1 -- 172.21.15.89:0/689957638 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dac122440 con 0x7f0dac0a5270 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.170+0000 7f0da97fa640 1 -- 172.21.15.89:0/689957638 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0d9c003620 con 0x7f0dac0a5270 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.170+0000 7f0da97fa640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d8c03be00 0x7f0d8c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.170+0000 7f0da97fa640 1 -- 172.21.15.89:0/689957638 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f0d9c0e7d60 con 0x7f0dac0a5270 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.170+0000 7f0da37fe640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d8c03be00 0x7f0d8c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.170+0000 7f0da37fe640 10 monclient: get_auth_request con 0x7f0d8c03be00 auth_method 0 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.174+0000 7f0da37fe640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d8c03be00 0x7f0d8c03e2c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0dac13f8c0 tx=0x7f0d94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.174+0000 7f0da97fa640 1 -- 172.21.15.89:0/689957638 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0d9c0ed020 con 0x7f0dac0a5270 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.174+0000 7f0da97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:21.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.174+0000 7f0da97fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:21.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.334+0000 7f0db246b640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-08-19T10:38:21.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.334+0000 7f0db246b640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-08-19T10:38:21.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.334+0000 7f0db246b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:21.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.334+0000 7f0db246b640 1 -- 172.21.15.89:0/689957638 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0dac0a67f0 con 0x7f0dac0a5270 2024-08-19T10:38:21.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.334+0000 7f0da97fa640 1 -- 172.21.15.89:0/689957638 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f0dac0a8710 con 0x7f0dac0a5270 2024-08-19T10:38:21.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.334+0000 7f0da97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-08-19T10:38:21.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.334+0000 7f0da97fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:21.337 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2024-08-19T10:38:21.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 -- 172.21.15.89:0/689957638 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d8c03be00 msgr2=0x7f0d8c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d8c03be00 0x7f0d8c03e2c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0dac13f8c0 tx=0x7f0d94008040 comp rx=0 tx=0).stop 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 10 monclient: shutdown 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 -- 172.21.15.89:0/689957638 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0dac0a5270 msgr2=0x7f0dac1d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0dac0a5270 0x7f0dac1d1230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0a34c0 tx=0x7f0d9c018f60 comp rx=0 tx=0).stop 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 -- 172.21.15.89:0/689957638 shutdown_connections 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0d8c03be00 0x7f0d8c03e2c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0dac1395b0 0x7f0dac13d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0dac0a6d30 0x7f0dac138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 --2- 172.21.15.89:0/689957638 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0dac0a5270 0x7f0dac1d1230 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 -- 172.21.15.89:0/689957638 >> 172.21.15.89:0/689957638 conn(0x7f0dac069a80 msgr2=0x7f0dac0a9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 -- 172.21.15.89:0/689957638 shutdown_connections 2024-08-19T10:38:21.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.338+0000 7f0db246b640 1 -- 172.21.15.89:0/689957638 wait complete. 2024-08-19T10:38:21.356 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-08-19T10:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.470+0000 7fd7fef3a640 10 monclient: get_monmap_and_config 2024-08-19T10:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.470+0000 7fd7fef3a640 10 monclient: build_initial_monmap 2024-08-19T10:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.470+0000 7fd7fef3a640 10 monclient: monmap: 2024-08-19T10:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:21.475843+0000 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:21.475843+0000 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:21.476 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.470+0000 7fd7fef3a640 1 Processor -- start 2024-08-19T10:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.470+0000 7fd7fef3a640 1 -- start start 2024-08-19T10:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.470+0000 7fd7fef3a640 10 monclient: init 2024-08-19T10:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 10 monclient: _add_conns ranks=[1,7,0,3,10,8,5,6,9,4,2] 2024-08-19T10:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd7f80a5270 addr v1:172.21.15.190:6789/0 2024-08-19T10:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7f80a6f00 0x7fd7f813ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 10 monclient(hunting): picked mon.noname-i con 0x7fd7f80a6f00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd7f80a72e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f80a8710 con 0x7fd7f80a72e0 2024-08-19T10:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f80a5a40 con 0x7fd7f80a5270 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd7f80a81d0 con 0x7fd7f80a6f00 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fef3a640 10 monclient(hunting): authenticate will time out at 951.292644s 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fd4d7640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd7f80a72e0 0x7fd7f8140f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44618/0 (socket says 172.21.15.89:44618) 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7fd4d7640 1 -- 172.21.15.89:0/505618593 learned_addr learned my addr 172.21.15.89:0/505618593 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 1 --2- 172.21.15.89:0/505618593 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7f80a6f00 0x7fd7f813ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 10 monclient(hunting): get_auth_request con 0x7fd7f80a6f00 auth_method 0 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 10 monclient(hunting): handle_auth_done global_id 4309 payload 274 2024-08-19T10:38:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 1 -- 172.21.15.89:0/505618593 >> v1:172.21.15.190:6789/0 conn(0x7fd7f80a5270 legacy=0x7fd7f80a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 1 -- 172.21.15.89:0/505618593 >> v1:172.21.15.89:6789/0 conn(0x7fd7f80a72e0 legacy=0x7fd7f8140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 1 monclient: found mon.noname-i 2024-08-19T10:38:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:38:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.474+0000 7fd7f7fff640 1 -- 172.21.15.89:0/505618593 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f80a4790 con 0x7fd7f80a6f00 2024-08-19T10:38:21.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f7fff640 1 -- 172.21.15.89:0/505618593 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7f80a6f00 msgr2=0x7fd7f813ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:21.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f7fff640 1 --2- 172.21.15.89:0/505618593 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7f80a6f00 0x7fd7f813ea00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0097f0 tx=0x7fd7ec01c190 comp rx=0 tx=0).ready entity=mon.7 client_cookie=318a41be919c32ae server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:21.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 1 -- 172.21.15.89:0/505618593 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7ec0b0070 con 0x7fd7f80a6f00 2024-08-19T10:38:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:38:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 10 monclient: dump: 2024-08-19T10:38:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 1 -- 172.21.15.89:0/505618593 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ec0aa810 con 0x7fd7f80a6f00 2024-08-19T10:38:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 5 monclient: authenticate success, global_id 4309 2024-08-19T10:38:21.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:21.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:21.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 1 -- 172.21.15.89:0/505618593 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7ec0aae30 con 0x7fd7f80a6f00 2024-08-19T10:38:21.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:21.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 4 set_mon_vals no callback set 2024-08-19T10:38:21.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:38:21.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7f77fe640 10 monclient: dump: 2024-08-19T10:38:21.484 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.485 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.486 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.486 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 10 monclient: shutdown 2024-08-19T10:38:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 1 -- 172.21.15.89:0/505618593 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7f80a6f00 msgr2=0x7fd7f813ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 1 --2- 172.21.15.89:0/505618593 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7f80a6f00 0x7fd7f813ea00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0097f0 tx=0x7fd7ec01c190 comp rx=0 tx=0).stop 2024-08-19T10:38:21.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 1 -- 172.21.15.89:0/505618593 shutdown_connections 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 1 --2- 172.21.15.89:0/505618593 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7f80a6f00 0x7fd7f813ea00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 1 -- 172.21.15.89:0/505618593 >> 172.21.15.89:0/505618593 conn(0x7fd7f8069a80 msgr2=0x7fd7f80abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 1 -- 172.21.15.89:0/505618593 shutdown_connections 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.478+0000 7fd7fef3a640 1 -- 172.21.15.89:0/505618593 wait complete. 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient: build_initial_monmap 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient: monmap: 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.491 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 1 Processor -- start 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 1 -- start start 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient: init 2024-08-19T10:38:21.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient: _add_conns ranks=[5,4,0,6,2,1,3,7,8] 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7f80a5270 0x7fd7f8128b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient(hunting): picked mon.noname-h con 0x7fd7f80a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7f80a6f00 0x7fd7f81d1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient(hunting): picked mon.noname-c con 0x7fd7f80a6f00 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f80a72e0 0x7fd7f8126660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient(hunting): picked mon.noname-a con 0x7fd7f80a72e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f8145010 con 0x7fd7f80a72e0 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd7f8144e90 con 0x7fd7f80a6f00 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd7f8145190 con 0x7fd7f80a5270 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fef3a640 10 monclient(hunting): authenticate will time out at 951.301086s 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7f80a6f00 0x7fd7f81d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7f80a6f00 0x7fd7f81d1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33792/0 (socket says 172.21.15.89:33792) 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fd4d7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f80a72e0 0x7fd7f8126660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 1 -- 172.21.15.89:0/773853743 learned_addr learned my addr 172.21.15.89:0/773853743 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 10 monclient(hunting): get_auth_request con 0x7fd7f80a6f00 auth_method 0 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fd4d7640 10 monclient(hunting): get_auth_request con 0x7fd7f80a72e0 auth_method 0 2024-08-19T10:38:21.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fd4d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fd4d7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fd4d7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fd4d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fd4d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fd4d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fccd6640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7f80a5270 0x7fd7f8128b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 10 monclient(hunting): handle_auth_done global_id 4360 payload 824 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 1 -- 172.21.15.89:0/773853743 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7f80a5270 msgr2=0x7fd7f8128b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7f80a5270 0x7fd7f8128b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 1 -- 172.21.15.89:0/773853743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f80a72e0 msgr2=0x7fd7f8126660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f80a72e0 0x7fd7f8126660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 1 monclient: found mon.noname-c 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 1 -- 172.21.15.89:0/773853743 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f80a2630 con 0x7fd7f80a6f00 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fccd6640 10 monclient: get_auth_request con 0x7fd7f80a5270 auth_method 0 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fd4d7640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f80a72e0 0x7fd7f8126660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7fccd6640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7f80a5270 0x7fd7f8128b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 1 -- 172.21.15.89:0/773853743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7f80a6f00 msgr2=0x7fd7f81d1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f7fff640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7f80a6f00 0x7fd7f81d1880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec01ca30 tx=0x7fd7ec018f30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f57fa640 1 -- 172.21.15.89:0/773853743 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7ec0b0040 con 0x7fd7f80a6f00 2024-08-19T10:38:21.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.482+0000 7fd7f57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 10 monclient: dump: 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.497 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 1 -- 172.21.15.89:0/773853743 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ec0b9070 con 0x7fd7f80a6f00 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 1 -- 172.21.15.89:0/773853743 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7ec0044b0 con 0x7fd7f80a6f00 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 10 monclient: dump: 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.499 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f6ffd640 4 set_mon_vals no callback set 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fef3a640 5 monclient: authenticate success, global_id 4360 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fef3a640 10 monclient: _renew_subs 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fef3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fef3a640 1 -- 172.21.15.89:0/773853743 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f812a090 con 0x7fd7f80a6f00 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fef3a640 10 monclient: _renew_subs 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fef3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fef3a640 1 -- 172.21.15.89:0/773853743 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f8129500 con 0x7fd7f80a6f00 2024-08-19T10:38:21.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fef3a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fef3a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fef3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fef3a640 1 -- 172.21.15.89:0/773853743 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f80a67f0 con 0x7fd7f80a6f00 2024-08-19T10:38:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 1 -- 172.21.15.89:0/773853743 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd7ec0b3070 con 0x7fd7f80a6f00 2024-08-19T10:38:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7e403be00 0x7fd7e403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7f57fa640 1 -- 172.21.15.89:0/773853743 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fd7ec0e83e0 con 0x7fd7f80a6f00 2024-08-19T10:38:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fccd6640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7e403be00 0x7fd7e403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.486+0000 7fd7fccd6640 10 monclient: get_auth_request con 0x7fd7e403be00 auth_method 0 2024-08-19T10:38:21.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.490+0000 7fd7fccd6640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7e403be00 0x7fd7e403e2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0007e90 tx=0x7fd7f00073a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:21.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.490+0000 7fd7f57fa640 1 -- 172.21.15.89:0/773853743 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd7f80a2630 con 0x7fd7f80a6f00 2024-08-19T10:38:21.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.490+0000 7fd7f57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:21.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.490+0000 7fd7f57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:21.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-08-19T10:38:21.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-08-19T10:38:21.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:21.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 -- 172.21.15.89:0/773853743 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) -- 0x7fd7f8064590 con 0x7fd7f80a6f00 2024-08-19T10:38:21.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7f57fa640 1 -- 172.21.15.89:0/773853743 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) ==== 92+0+10 (secure 0 0 0) 0x7fd7f80a8710 con 0x7fd7f80a6f00 2024-08-19T10:38:21.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7f57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-08-19T10:38:21.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7f57fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:21.648 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 8 2024-08-19T10:38:21.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 -- 172.21.15.89:0/773853743 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7e403be00 msgr2=0x7fd7e403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7e403be00 0x7fd7e403e2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0007e90 tx=0x7fd7f00073a0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 10 monclient: shutdown 2024-08-19T10:38:21.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:21.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 -- 172.21.15.89:0/773853743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7f80a6f00 msgr2=0x7fd7f81d1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7f80a6f00 0x7fd7f81d1880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec01ca30 tx=0x7fd7ec018f30 comp rx=0 tx=0).stop 2024-08-19T10:38:21.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 -- 172.21.15.89:0/773853743 shutdown_connections 2024-08-19T10:38:21.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7e403be00 0x7fd7e403e2c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7f80a72e0 0x7fd7f8126660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7f80a6f00 0x7fd7f81d1880 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 --2- 172.21.15.89:0/773853743 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7f80a5270 0x7fd7f8128b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 -- 172.21.15.89:0/773853743 >> 172.21.15.89:0/773853743 conn(0x7fd7f8069a80 msgr2=0x7fd7f809dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:21.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 -- 172.21.15.89:0/773853743 shutdown_connections 2024-08-19T10:38:21.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.646+0000 7fd7fef3a640 1 -- 172.21.15.89:0/773853743 wait complete. 2024-08-19T10:38:21.665 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-08-19T10:38:21.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T10:38:21.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient: get_monmap_and_config 2024-08-19T10:38:21.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient: build_initial_monmap 2024-08-19T10:38:21.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient: monmap: 2024-08-19T10:38:21.779 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:21.779 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.779 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:21.775363+0000 2024-08-19T10:38:21.779 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:21.775363+0000 2024-08-19T10:38:21.779 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:21.780 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.780 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:21.780 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:21.780 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:21.780 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:21.780 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:21.781 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:21.781 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:21.781 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:21.781 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:21.781 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:21.782 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:21.782 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 1 Processor -- start 2024-08-19T10:38:21.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 1 -- start start 2024-08-19T10:38:21.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient: init 2024-08-19T10:38:21.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:21.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient: _add_conns ranks=[8,10,6,5,4,1,9,7,3,2,0] 2024-08-19T10:38:21.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f683806c930 0x7f68380a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient(hunting): picked mon.noname-e con 0x7f683806c930 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:21.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68380a3fa0 0x7f68380a4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient(hunting): picked mon.noname-f con 0x7f68380a3fa0 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:21.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f683813d910 0x7f683813fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient(hunting): picked mon.noname-d con 0x7f683813d910 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:21.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f68380744f0 con 0x7f68380a3fa0 2024-08-19T10:38:21.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f68380a48c0 con 0x7f683813d910 2024-08-19T10:38:21.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f683806cd10 con 0x7f683806c930 2024-08-19T10:38:21.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:21.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.770+0000 7f683f5a6640 10 monclient(hunting): authenticate will time out at 951.591865s 2024-08-19T10:38:21.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f683806c930 0x7f68380a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683db43640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f683813d910 0x7f683813fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683db43640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f683813d910 0x7f683813fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36558/0 (socket says 172.21.15.89:36558) 2024-08-19T10:38:21.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f683806c930 0x7f68380a3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:53174/0 (socket says 172.21.15.89:53174) 2024-08-19T10:38:21.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683db43640 1 -- 172.21.15.89:0/1886399288 learned_addr learned my addr 172.21.15.89:0/1886399288 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:21.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683db43640 10 monclient(hunting): get_auth_request con 0x7f683813d910 auth_method 0 2024-08-19T10:38:21.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683db43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:21.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683db43640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:21.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683db43640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:21.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 10 monclient(hunting): get_auth_request con 0x7f683806c930 auth_method 0 2024-08-19T10:38:21.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:21.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:21.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:21.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:21.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:21.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:21.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683db43640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:21.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683db43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:21.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683db43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:21.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683cb41640 1 --2- 172.21.15.89:0/1886399288 >> v2:172.21.15.190:3300/0 conn(0x7f68380a3fa0 0x7f68380a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683cb41640 10 monclient(hunting): get_auth_request con 0x7f68380a3fa0 auth_method 0 2024-08-19T10:38:21.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683cb41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:21.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683cb41640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:21.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683cb41640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:21.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 10 monclient(hunting): handle_auth_done global_id 4310 payload 274 2024-08-19T10:38:21.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 1 -- 172.21.15.89:0/1886399288 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f683813d910 msgr2=0x7f683813fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:21.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 1 --2- 172.21.15.89:0/1886399288 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f683813d910 0x7f683813fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 1 -- 172.21.15.89:0/1886399288 >> v2:172.21.15.190:3300/0 conn(0x7f68380a3fa0 msgr2=0x7f68380a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:21.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 1 --2- 172.21.15.89:0/1886399288 >> v2:172.21.15.190:3300/0 conn(0x7f68380a3fa0 0x7f68380a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 10 monclient: _finish_hunting 0 2024-08-19T10:38:21.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 1 monclient: found mon.noname-e 2024-08-19T10:38:21.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:21.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:21.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.774+0000 7f683d342640 1 -- 172.21.15.89:0/1886399288 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68380a4ae0 con 0x7f683806c930 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683d342640 1 -- 172.21.15.89:0/1886399288 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f683806c930 msgr2=0x7f68380a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683d342640 1 --2- 172.21.15.89:0/1886399288 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f683806c930 0x7f68380a3a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f68340097c0 tx=0x7f683401c360 comp rx=0 tx=0).ready entity=mon.8 client_cookie=cbb0bf0bd2ddbee5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 1 -- 172.21.15.89:0/1886399288 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68340b0070 con 0x7f683806c930 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 10 monclient: dump: 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.800 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 10 monclient: _finish_auth 0 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:21.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 1 -- 172.21.15.89:0/1886399288 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68340aa810 con 0x7f683806c930 2024-08-19T10:38:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 5 monclient: authenticate success, global_id 4310 2024-08-19T10:38:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 1 -- 172.21.15.89:0/1886399288 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68340aae30 con 0x7f683806c930 2024-08-19T10:38:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f682ffff640 10 monclient: dump: 2024-08-19T10:38:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.802 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.803 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 4 set_mon_vals no callback set 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient: shutdown 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 -- 172.21.15.89:0/1886399288 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f683806c930 msgr2=0x7f68380a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 --2- 172.21.15.89:0/1886399288 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f683806c930 0x7f68380a3a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f68340097c0 tx=0x7f683401c360 comp rx=0 tx=0).stop 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 -- 172.21.15.89:0/1886399288 shutdown_connections 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 --2- 172.21.15.89:0/1886399288 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f683813d910 0x7f683813fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 --2- 172.21.15.89:0/1886399288 >> v2:172.21.15.190:3300/0 conn(0x7f68380a3fa0 0x7f68380a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 --2- 172.21.15.89:0/1886399288 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f683806c930 0x7f68380a3a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 -- 172.21.15.89:0/1886399288 >> 172.21.15.89:0/1886399288 conn(0x7f68380a8f60 msgr2=0x7f68380a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 -- 172.21.15.89:0/1886399288 shutdown_connections 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 -- 172.21.15.89:0/1886399288 wait complete. 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient: build_initial_monmap 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient: monmap: 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:21.805 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 Processor -- start 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 -- start start 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient: init 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient: _add_conns ranks=[3,0,1,8,4,6,5,2,7] 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f683806c930 0x7f6838126c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient(hunting): picked mon.noname-g con 0x7f683806c930 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68380a3fa0 0x7f6838124c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient(hunting): picked mon.noname-a con 0x7f68380a3fa0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f683813d910 0x7f683813cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient(hunting): picked mon.noname-f con 0x7f683813d910 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6838142be0 con 0x7f68380a3fa0 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6838142a60 con 0x7f683813d910 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6838142d60 con 0x7f683806c930 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683f5a6640 10 monclient(hunting): authenticate will time out at 951.598829s 2024-08-19T10:38:21.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68380a3fa0 0x7f6838124c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68380a3fa0 0x7f6838124c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33016/0 (socket says 172.21.15.89:33016) 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 1 -- 172.21.15.89:0/2618312507 learned_addr learned my addr 172.21.15.89:0/2618312507 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 10 monclient(hunting): get_auth_request con 0x7f68380a3fa0 auth_method 0 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 10 monclient(hunting): handle_auth_done global_id 4437 payload 824 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 1 -- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f683806c930 msgr2=0x7f6838126c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f683806c930 0x7f6838126c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 1 -- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f683813d910 msgr2=0x7f683813cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f683813d910 0x7f683813cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 10 monclient: _finish_hunting 0 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 1 monclient: found mon.noname-a 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.778+0000 7f683cb41640 1 -- 172.21.15.89:0/2618312507 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68380a4510 con 0x7f68380a3fa0 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683cb41640 1 -- 172.21.15.89:0/2618312507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68380a3fa0 msgr2=0x7f6838124c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683cb41640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68380a3fa0 0x7f6838124c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f682800d830 tx=0x7f682800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 1 -- 172.21.15.89:0/2618312507 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f682800bcc0 con 0x7f68380a3fa0 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 10 monclient: dump: 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.809 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.810 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 1 -- 172.21.15.89:0/2618312507 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6828019070 con 0x7f68380a3fa0 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683f5a6640 5 monclient: authenticate success, global_id 4437 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683f5a6640 10 monclient: _renew_subs 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683f5a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683f5a6640 1 -- 172.21.15.89:0/2618312507 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6838127bb0 con 0x7f68380a3fa0 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 1 -- 172.21.15.89:0/2618312507 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68280148a0 con 0x7f68380a3fa0 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:21.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 10 monclient: dump: 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:21.812 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:21.813 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:21.813 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:21.813 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:21.813 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:21.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682f7fe640 4 set_mon_vals no callback set 2024-08-19T10:38:21.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683f5a6640 10 monclient: _renew_subs 2024-08-19T10:38:21.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683f5a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:21.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683f5a6640 1 -- 172.21.15.89:0/2618312507 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6838128450 con 0x7f68380a3fa0 2024-08-19T10:38:21.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683f5a6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:21.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683f5a6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:21.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683f5a6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:21.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683f5a6640 1 -- 172.21.15.89:0/2618312507 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68380a2250 con 0x7f68380a3fa0 2024-08-19T10:38:21.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 1 -- 172.21.15.89:0/2618312507 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6828014430 con 0x7f68380a3fa0 2024-08-19T10:38:21.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f681c03be00 0x7f681c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:21.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f682dffb640 1 -- 172.21.15.89:0/2618312507 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f682800f040 con 0x7f68380a3fa0 2024-08-19T10:38:21.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683d342640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f681c03be00 0x7f681c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:21.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.782+0000 7f683d342640 10 monclient: get_auth_request con 0x7f681c03be00 auth_method 0 2024-08-19T10:38:21.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.786+0000 7f683d342640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f681c03be00 0x7f681c03e2c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f68340097c0 tx=0x7f6834004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:21.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.786+0000 7f682dffb640 1 -- 172.21.15.89:0/2618312507 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f68380a2250 con 0x7f68380a3fa0 2024-08-19T10:38:21.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.786+0000 7f682dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:21.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.786+0000 7f682dffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:21.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.946+0000 7f683f5a6640 1 -- 172.21.15.89:0/2618312507 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6838064590 con 0x7f681c03be00 2024-08-19T10:38:21.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.946+0000 7f682dffb640 1 -- 172.21.15.89:0/2618312507 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34051 (secure 0 0 0) 0x7f6838064590 con 0x7f681c03be00 2024-08-19T10:38:21.950 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:38:21.951 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-08-19T10:38:21.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 -- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f681c03be00 msgr2=0x7f681c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f681c03be00 0x7f681c03e2c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f68340097c0 tx=0x7f6834004740 comp rx=0 tx=0).stop 2024-08-19T10:38:21.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 10 monclient: shutdown 2024-08-19T10:38:21.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:21.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 -- 172.21.15.89:0/2618312507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68380a3fa0 msgr2=0x7f6838124c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:21.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68380a3fa0 0x7f6838124c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f682800d830 tx=0x7f682800dcf0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 -- 172.21.15.89:0/2618312507 shutdown_connections 2024-08-19T10:38:21.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f681c03be00 0x7f681c03e2c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f683813d910 0x7f683813cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68380a3fa0 0x7f6838124c20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 --2- 172.21.15.89:0/2618312507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f683806c930 0x7f6838126c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:21.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 -- 172.21.15.89:0/2618312507 >> 172.21.15.89:0/2618312507 conn(0x7f68380a8f60 msgr2=0x7f683813e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:21.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 -- 172.21.15.89:0/2618312507 shutdown_connections 2024-08-19T10:38:21.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:21.950+0000 7f683f5a6640 1 -- 172.21.15.89:0/2618312507 wait complete. 2024-08-19T10:38:21.967 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-08-19T10:38:20.448577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164332,"kb_used_data":2188,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066708,"statfs":{"total":579820584960,"available":579652308992,"internally_reserved":0,"allocated":2240512,"data_stored":1523830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.640207"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.838257+0000","last_change":"2024-08-19T10:38:14.838742+0000","last_active":"2024-08-19T10:38:14.838257+0000","last_peered":"2024-08-19T10:38:14.838257+0000","last_clean":"2024-08-19T10:38:14.838257+0000","last_became_active":"2024-08-19T10:38:12.819810+0000","last_became_peered":"2024-08-19T10:38:12.819810+0000","last_unstale":"2024-08-19T10:38:14.838257+0000","last_undegraded":"2024-08-19T10:38:14.838257+0000","last_fullsized":"2024-08-19T10:38:14.838257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:10:48.036604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021153309999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.849314+0000","last_change":"2024-08-19T10:38:14.849789+0000","last_active":"2024-08-19T10:38:14.849314+0000","last_peered":"2024-08-19T10:38:14.849314+0000","last_clean":"2024-08-19T10:38:14.849314+0000","last_became_active":"2024-08-19T10:38:12.830800+0000","last_became_peered":"2024-08-19T10:38:12.830800+0000","last_unstale":"2024-08-19T10:38:14.849314+0000","last_undegraded":"2024-08-19T10:38:14.849314+0000","last_fullsized":"2024-08-19T10:38:14.849314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:25:14.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081542699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.837210+0000","last_change":"2024-08-19T10:38:14.839689+0000","last_active":"2024-08-19T10:38:14.837210+0000","last_peered":"2024-08-19T10:38:14.837210+0000","last_clean":"2024-08-19T10:38:14.837210+0000","last_became_active":"2024-08-19T10:38:12.825835+0000","last_became_peered":"2024-08-19T10:38:12.825835+0000","last_unstale":"2024-08-19T10:38:14.837210+0000","last_undegraded":"2024-08-19T10:38:14.837210+0000","last_fullsized":"2024-08-19T10:38:14.837210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:13:58.426106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003718322,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-19T10:38:20.212249+0000","last_change":"2024-08-19T10:38:14.838322+0000","last_active":"2024-08-19T10:38:20.212249+0000","last_peered":"2024-08-19T10:38:20.212249+0000","last_clean":"2024-08-19T10:38:20.212249+0000","last_became_active":"2024-08-19T10:38:12.821074+0000","last_became_peered":"2024-08-19T10:38:12.821074+0000","last_unstale":"2024-08-19T10:38:20.212249+0000","last_undegraded":"2024-08-19T10:38:20.212249+0000","last_fullsized":"2024-08-19T10:38:20.212249+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:36:05.704454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010645120000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.844887+0000","last_change":"2024-08-19T10:38:14.840443+0000","last_active":"2024-08-19T10:38:14.844887+0000","last_peered":"2024-08-19T10:38:14.844887+0000","last_clean":"2024-08-19T10:38:14.844887+0000","last_became_active":"2024-08-19T10:38:12.823338+0000","last_became_peered":"2024-08-19T10:38:12.823338+0000","last_unstale":"2024-08-19T10:38:14.844887+0000","last_undegraded":"2024-08-19T10:38:14.844887+0000","last_fullsized":"2024-08-19T10:38:14.844887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:21:50.030940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001115312,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-19T10:38:20.212679+0000","last_change":"2024-08-19T10:38:14.838406+0000","last_active":"2024-08-19T10:38:20.212679+0000","last_peered":"2024-08-19T10:38:20.212679+0000","last_clean":"2024-08-19T10:38:20.212679+0000","last_became_active":"2024-08-19T10:38:12.820746+0000","last_became_peered":"2024-08-19T10:38:12.820746+0000","last_unstale":"2024-08-19T10:38:20.212679+0000","last_undegraded":"2024-08-19T10:38:20.212679+0000","last_fullsized":"2024-08-19T10:38:20.212679+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:26:36.717033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001410375,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.839239+0000","last_change":"2024-08-19T10:38:14.839751+0000","last_active":"2024-08-19T10:38:14.839239+0000","last_peered":"2024-08-19T10:38:14.839239+0000","last_clean":"2024-08-19T10:38:14.839239+0000","last_became_active":"2024-08-19T10:38:12.819986+0000","last_became_peered":"2024-08-19T10:38:12.819986+0000","last_unstale":"2024-08-19T10:38:14.839239+0000","last_undegraded":"2024-08-19T10:38:14.839239+0000","last_fullsized":"2024-08-19T10:38:14.839239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:41:38.839856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010100350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.837363+0000","last_change":"2024-08-19T10:38:14.838122+0000","last_active":"2024-08-19T10:38:14.837363+0000","last_peered":"2024-08-19T10:38:14.837363+0000","last_clean":"2024-08-19T10:38:14.837363+0000","last_became_active":"2024-08-19T10:38:12.820601+0000","last_became_peered":"2024-08-19T10:38:12.820601+0000","last_unstale":"2024-08-19T10:38:14.837363+0000","last_undegraded":"2024-08-19T10:38:14.837363+0000","last_fullsized":"2024-08-19T10:38:14.837363+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:09:29.225317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002082918,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.835718+0000","last_change":"2024-08-19T10:38:06.130856+0000","last_active":"2024-08-19T10:38:14.835718+0000","last_peered":"2024-08-19T10:38:14.835718+0000","last_clean":"2024-08-19T10:38:14.835718+0000","last_became_active":"2024-08-19T10:38:06.129934+0000","last_became_peered":"2024-08-19T10:38:06.129934+0000","last_unstale":"2024-08-19T10:38:14.835718+0000","last_undegraded":"2024-08-19T10:38:14.835718+0000","last_fullsized":"2024-08-19T10:38:14.835718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_clean_scrub_stamp":"2024-08-19T10:38:04.611102+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:27:58.119234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":646777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":56428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607560,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":61011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":56428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":172032,"data_stored":56409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640263,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":646777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T10:38:21.968 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T10:38:22.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.078+0000 7f82df6d2640 10 monclient: get_monmap_and_config 2024-08-19T10:38:22.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.078+0000 7f82df6d2640 10 monclient: build_initial_monmap 2024-08-19T10:38:22.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.078+0000 7f82df6d2640 10 monclient: monmap: 2024-08-19T10:38:22.082 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:22.082 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.082 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:22.083681+0000 2024-08-19T10:38:22.082 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:22.083681+0000 2024-08-19T10:38:22.083 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:22.083 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.083 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:22.083 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:22.083 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:22.083 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:22.083 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:22.084 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:22.084 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:22.084 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:22.084 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:22.084 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:22.084 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:22.084 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 1 Processor -- start 2024-08-19T10:38:22.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 1 -- start start 2024-08-19T10:38:22.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 10 monclient: init 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 10 monclient: _add_conns ranks=[0,4,9,5,8,3,6,7,10,1,2] 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f82d80a3d00 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 0x7f82d80a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 10 monclient(hunting): picked mon.noname-c con 0x7f82d80a1a00 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f82d813c910 0x7f82d813ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 10 monclient(hunting): picked mon.noname-a con 0x7f82d813c910 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d806db50 con 0x7f82d80a3d00 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f82d809d3d0 con 0x7f82d813c910 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f82d806fd00 con 0x7f82d80a1a00 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82df6d2640 10 monclient(hunting): authenticate will time out at 951.901047s 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dd46e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f82d80a3d00 0x7f82d80a40e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44626/0 (socket says 172.21.15.89:44626) 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dd46e640 1 -- 172.21.15.89:0/2960214533 learned_addr learned my addr 172.21.15.89:0/2960214533 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 1 --2- 172.21.15.89:0/2960214533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 0x7f82d80a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 10 monclient(hunting): get_auth_request con 0x7f82d80a1a00 auth_method 0 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82ddc6f640 1 --2- 172.21.15.89:0/2960214533 >> v2:172.21.15.89:3300/0 conn(0x7f82d813c910 0x7f82d813ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82ddc6f640 10 monclient(hunting): get_auth_request con 0x7f82d813c910 auth_method 0 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82ddc6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82ddc6f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82ddc6f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82ddc6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82ddc6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82ddc6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 10 monclient(hunting): handle_auth_done global_id 4369 payload 274 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 1 -- 172.21.15.89:0/2960214533 >> v2:172.21.15.89:3300/0 conn(0x7f82d813c910 msgr2=0x7f82d813ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 1 --2- 172.21.15.89:0/2960214533 >> v2:172.21.15.89:3300/0 conn(0x7f82d813c910 0x7f82d813ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 1 -- 172.21.15.89:0/2960214533 >> v1:172.21.15.89:6789/0 conn(0x7f82d80a3d00 legacy=0x7f82d80a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 10 monclient: _finish_hunting 0 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 1 monclient: found mon.noname-c 2024-08-19T10:38:22.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:22.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:22.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82dcc6d640 1 -- 172.21.15.89:0/2960214533 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d809c9c0 con 0x7f82d80a1a00 2024-08-19T10:38:22.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.082+0000 7f82ddc6f640 1 --2- 172.21.15.89:0/2960214533 >> v2:172.21.15.89:3300/0 conn(0x7f82d813c910 0x7f82d813ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82dcc6d640 1 -- 172.21.15.89:0/2960214533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 msgr2=0x7f82d80a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82dcc6d640 1 --2- 172.21.15.89:0/2960214533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 0x7f82d80a47f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82c80097c0 tx=0x7f82c801c360 comp rx=0 tx=0).ready entity=mon.4 client_cookie=6d1e6c059c13c243 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 1 -- 172.21.15.89:0/2960214533 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f82c80b0070 con 0x7f82d80a1a00 2024-08-19T10:38:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 10 monclient: dump: 2024-08-19T10:38:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.093 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.094 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.094 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.094 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.094 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.094 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.094 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.094 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.094 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.095 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.095 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.095 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 10 monclient: _finish_auth 0 2024-08-19T10:38:22.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:22.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:22.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 1 -- 172.21.15.89:0/2960214533 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c80aa810 con 0x7f82d80a1a00 2024-08-19T10:38:22.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 5 monclient: authenticate success, global_id 4369 2024-08-19T10:38:22.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:22.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 4 set_mon_vals no callback set 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 1 -- 172.21.15.89:0/2960214533 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f82c80aae30 con 0x7f82d80a1a00 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82cffff640 10 monclient: dump: 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.096 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 10 monclient: shutdown 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 1 -- 172.21.15.89:0/2960214533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 msgr2=0x7f82d80a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 1 --2- 172.21.15.89:0/2960214533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 0x7f82d80a47f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82c80097c0 tx=0x7f82c801c360 comp rx=0 tx=0).stop 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 1 -- 172.21.15.89:0/2960214533 shutdown_connections 2024-08-19T10:38:22.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 1 --2- 172.21.15.89:0/2960214533 >> v2:172.21.15.89:3300/0 conn(0x7f82d813c910 0x7f82d813ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 1 --2- 172.21.15.89:0/2960214533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 0x7f82d80a47f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 1 -- 172.21.15.89:0/2960214533 >> 172.21.15.89:0/2960214533 conn(0x7f82d80a4df0 msgr2=0x7f82d80a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 1 -- 172.21.15.89:0/2960214533 shutdown_connections 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 1 -- 172.21.15.89:0/2960214533 wait complete. 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 10 monclient: build_initial_monmap 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 10 monclient: monmap: 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.098 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 1 Processor -- start 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 1 -- start start 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.086+0000 7f82df6d2640 10 monclient: init 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:22.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient: _add_conns ranks=[4,8,0,6,3,5,7,2,1] 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 0x7f82d8137d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient(hunting): picked mon.noname-c con 0x7f82d80a1a00 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f82d80a3d00 0x7f82d8073430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient(hunting): picked mon.noname-e con 0x7f82d80a3d00 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d813c910 0x7f82d8073970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient(hunting): picked mon.noname-a con 0x7f82d813c910 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8141be0 con 0x7f82d813c910 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f82d8141a60 con 0x7f82d80a1a00 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f82d8141d60 con 0x7f82d80a3d00 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient(hunting): authenticate will time out at 951.908475s 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d813c910 0x7f82d8073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dcc6d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f82d80a3d00 0x7f82d8073430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dd46e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 0x7f82d8137d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d813c910 0x7f82d8073970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33038/0 (socket says 172.21.15.89:33038) 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 1 -- 172.21.15.89:0/2219048607 learned_addr learned my addr 172.21.15.89:0/2219048607 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 10 monclient(hunting): get_auth_request con 0x7f82d813c910 auth_method 0 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dcc6d640 10 monclient(hunting): get_auth_request con 0x7f82d80a3d00 auth_method 0 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dcc6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dcc6d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dcc6d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dd46e640 10 monclient(hunting): get_auth_request con 0x7f82d80a1a00 auth_method 0 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dd46e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dd46e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dd46e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dcc6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dcc6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dcc6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dd46e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dd46e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dd46e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 10 monclient(hunting): handle_auth_done global_id 4455 payload 824 2024-08-19T10:38:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 1 -- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f82d80a3d00 msgr2=0x7f82d8073430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f82d80a3d00 0x7f82d8073430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 1 -- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 msgr2=0x7f82d8137d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 0x7f82d8137d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 10 monclient: _finish_hunting 0 2024-08-19T10:38:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 1 monclient: found mon.noname-a 2024-08-19T10:38:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 1 -- 172.21.15.89:0/2219048607 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d812edd0 con 0x7f82d813c910 2024-08-19T10:38:22.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82dd46e640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 0x7f82d8137d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:22.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 1 -- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d813c910 msgr2=0x7f82d8073970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ddc6f640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d813c910 0x7f82d8073970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f82d000efc0 tx=0x7f82d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 1 -- 172.21.15.89:0/2219048607 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f82d0018030 con 0x7f82d813c910 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 10 monclient: dump: 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.106 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 1 -- 172.21.15.89:0/2219048607 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d00148f0 con 0x7f82d813c910 2024-08-19T10:38:22.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 5 monclient: authenticate success, global_id 4455 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient: _renew_subs 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 1 -- 172.21.15.89:0/2219048607 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d812f0b0 con 0x7f82d813c910 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 1 -- 172.21.15.89:0/2219048607 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f82d00133e0 con 0x7f82d813c910 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82cdffb640 10 monclient: dump: 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.108 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82ceffd640 4 set_mon_vals no callback set 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient: _renew_subs 2024-08-19T10:38:22.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.090+0000 7f82df6d2640 1 -- 172.21.15.89:0/2219048607 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d812f670 con 0x7f82d813c910 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.094+0000 7f82df6d2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.094+0000 7f82df6d2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.094+0000 7f82df6d2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.094+0000 7f82df6d2640 1 -- 172.21.15.89:0/2219048607 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d80a40e0 con 0x7f82d813c910 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.094+0000 7f82cdffb640 1 -- 172.21.15.89:0/2219048607 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f82d001b070 con 0x7f82d813c910 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.094+0000 7f82cdffb640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82bc03be30 0x7f82bc03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.094+0000 7f82cdffb640 1 -- 172.21.15.89:0/2219048607 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f82d0019070 con 0x7f82d813c910 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.094+0000 7f82dd46e640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82bc03be30 0x7f82bc03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.094+0000 7f82dd46e640 10 monclient: get_auth_request con 0x7f82bc03be30 auth_method 0 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.094+0000 7f82dd46e640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82bc03be30 0x7f82bc03e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f82d4007ca0 tx=0x7f82d4007bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.098+0000 7f82cdffb640 1 -- 172.21.15.89:0/2219048607 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f82d0022020 con 0x7f82d813c910 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.098+0000 7f82cdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:22.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.098+0000 7f82cdffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:22.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.266+0000 7f82df6d2640 1 -- 172.21.15.89:0/2219048607 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82d8064590 con 0x7f82bc03be30 2024-08-19T10:38:22.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.266+0000 7f82cdffb640 1 -- 172.21.15.89:0/2219048607 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34051 (secure 0 0 0) 0x7f82d8064590 con 0x7f82bc03be30 2024-08-19T10:38:22.271 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:38:22.271 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-08-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 -- 172.21.15.89:0/2219048607 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82bc03be30 msgr2=0x7f82bc03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82bc03be30 0x7f82bc03e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f82d4007ca0 tx=0x7f82d4007bf0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 10 monclient: shutdown 2024-08-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 -- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d813c910 msgr2=0x7f82d8073970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d813c910 0x7f82d8073970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f82d000efc0 tx=0x7f82d000c490 comp rx=0 tx=0).stop 2024-08-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 -- 172.21.15.89:0/2219048607 shutdown_connections 2024-08-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82d813c910 0x7f82d8073970 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82bc03be30 0x7f82bc03e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f82d80a3d00 0x7f82d8073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 --2- 172.21.15.89:0/2219048607 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f82d80a1a00 0x7f82d8137d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 -- 172.21.15.89:0/2219048607 >> 172.21.15.89:0/2219048607 conn(0x7f82d80a4df0 msgr2=0x7f82d813bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:22.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 -- 172.21.15.89:0/2219048607 shutdown_connections 2024-08-19T10:38:22.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.270+0000 7f82df6d2640 1 -- 172.21.15.89:0/2219048607 wait complete. 2024-08-19T10:38:22.289 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-08-19T10:38:20.448577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164332,"kb_used_data":2188,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066708,"statfs":{"total":579820584960,"available":579652308992,"internally_reserved":0,"allocated":2240512,"data_stored":1523830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.640207"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.838257+0000","last_change":"2024-08-19T10:38:14.838742+0000","last_active":"2024-08-19T10:38:14.838257+0000","last_peered":"2024-08-19T10:38:14.838257+0000","last_clean":"2024-08-19T10:38:14.838257+0000","last_became_active":"2024-08-19T10:38:12.819810+0000","last_became_peered":"2024-08-19T10:38:12.819810+0000","last_unstale":"2024-08-19T10:38:14.838257+0000","last_undegraded":"2024-08-19T10:38:14.838257+0000","last_fullsized":"2024-08-19T10:38:14.838257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:10:48.036604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021153309999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.849314+0000","last_change":"2024-08-19T10:38:14.849789+0000","last_active":"2024-08-19T10:38:14.849314+0000","last_peered":"2024-08-19T10:38:14.849314+0000","last_clean":"2024-08-19T10:38:14.849314+0000","last_became_active":"2024-08-19T10:38:12.830800+0000","last_became_peered":"2024-08-19T10:38:12.830800+0000","last_unstale":"2024-08-19T10:38:14.849314+0000","last_undegraded":"2024-08-19T10:38:14.849314+0000","last_fullsized":"2024-08-19T10:38:14.849314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:25:14.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081542699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.837210+0000","last_change":"2024-08-19T10:38:14.839689+0000","last_active":"2024-08-19T10:38:14.837210+0000","last_peered":"2024-08-19T10:38:14.837210+0000","last_clean":"2024-08-19T10:38:14.837210+0000","last_became_active":"2024-08-19T10:38:12.825835+0000","last_became_peered":"2024-08-19T10:38:12.825835+0000","last_unstale":"2024-08-19T10:38:14.837210+0000","last_undegraded":"2024-08-19T10:38:14.837210+0000","last_fullsized":"2024-08-19T10:38:14.837210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:13:58.426106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003718322,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-19T10:38:20.212249+0000","last_change":"2024-08-19T10:38:14.838322+0000","last_active":"2024-08-19T10:38:20.212249+0000","last_peered":"2024-08-19T10:38:20.212249+0000","last_clean":"2024-08-19T10:38:20.212249+0000","last_became_active":"2024-08-19T10:38:12.821074+0000","last_became_peered":"2024-08-19T10:38:12.821074+0000","last_unstale":"2024-08-19T10:38:20.212249+0000","last_undegraded":"2024-08-19T10:38:20.212249+0000","last_fullsized":"2024-08-19T10:38:20.212249+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:36:05.704454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010645120000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.844887+0000","last_change":"2024-08-19T10:38:14.840443+0000","last_active":"2024-08-19T10:38:14.844887+0000","last_peered":"2024-08-19T10:38:14.844887+0000","last_clean":"2024-08-19T10:38:14.844887+0000","last_became_active":"2024-08-19T10:38:12.823338+0000","last_became_peered":"2024-08-19T10:38:12.823338+0000","last_unstale":"2024-08-19T10:38:14.844887+0000","last_undegraded":"2024-08-19T10:38:14.844887+0000","last_fullsized":"2024-08-19T10:38:14.844887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:21:50.030940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001115312,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-08-19T10:38:20.212679+0000","last_change":"2024-08-19T10:38:14.838406+0000","last_active":"2024-08-19T10:38:20.212679+0000","last_peered":"2024-08-19T10:38:20.212679+0000","last_clean":"2024-08-19T10:38:20.212679+0000","last_became_active":"2024-08-19T10:38:12.820746+0000","last_became_peered":"2024-08-19T10:38:12.820746+0000","last_unstale":"2024-08-19T10:38:20.212679+0000","last_undegraded":"2024-08-19T10:38:20.212679+0000","last_fullsized":"2024-08-19T10:38:20.212679+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:26:36.717033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001410375,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.839239+0000","last_change":"2024-08-19T10:38:14.839751+0000","last_active":"2024-08-19T10:38:14.839239+0000","last_peered":"2024-08-19T10:38:14.839239+0000","last_clean":"2024-08-19T10:38:14.839239+0000","last_became_active":"2024-08-19T10:38:12.819986+0000","last_became_peered":"2024-08-19T10:38:12.819986+0000","last_unstale":"2024-08-19T10:38:14.839239+0000","last_undegraded":"2024-08-19T10:38:14.839239+0000","last_fullsized":"2024-08-19T10:38:14.839239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:41:38.839856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010100350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.837363+0000","last_change":"2024-08-19T10:38:14.838122+0000","last_active":"2024-08-19T10:38:14.837363+0000","last_peered":"2024-08-19T10:38:14.837363+0000","last_clean":"2024-08-19T10:38:14.837363+0000","last_became_active":"2024-08-19T10:38:12.820601+0000","last_became_peered":"2024-08-19T10:38:12.820601+0000","last_unstale":"2024-08-19T10:38:14.837363+0000","last_undegraded":"2024-08-19T10:38:14.837363+0000","last_fullsized":"2024-08-19T10:38:14.837363+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:09:29.225317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002082918,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-08-19T10:38:14.835718+0000","last_change":"2024-08-19T10:38:06.130856+0000","last_active":"2024-08-19T10:38:14.835718+0000","last_peered":"2024-08-19T10:38:14.835718+0000","last_clean":"2024-08-19T10:38:14.835718+0000","last_became_active":"2024-08-19T10:38:06.129934+0000","last_became_peered":"2024-08-19T10:38:06.129934+0000","last_unstale":"2024-08-19T10:38:14.835718+0000","last_undegraded":"2024-08-19T10:38:14.835718+0000","last_fullsized":"2024-08-19T10:38:14.835718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_clean_scrub_stamp":"2024-08-19T10:38:04.611102+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:27:58.119234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":646777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":56428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607560,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":61011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":56428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":172032,"data_stored":56409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640263,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":646777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T10:38:22.290 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-08-19T10:38:22.291 DEBUG:tasks.mon_thrash:Yielding 2024-08-19T10:38:22.291 INFO:teuthology.run_tasks:Running task workunit... 2024-08-19T10:38:22.306 INFO:tasks.workunit:Pulling workunits from ref 964adda23da9f7b057cb34fdc0b82aeb7ed65324 2024-08-19T10:38:22.306 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-19T10:38:22.306 DEBUG:teuthology.orchestra.run.smithi190:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-19T10:38:22.307 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-08-19T10:38:22.307 INFO:tasks.mon_thrash.mon_thrasher:seed: 1724063902, revive delay: 90.0, thrash delay: 1.0 thrash many: True, maintain quorum: True store thrash: True, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-08-19T10:38:22.307 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:38:22.308 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:38:22.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T10:38:22.314 INFO:teuthology.orchestra.run.smithi190.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-08-19T10:38:22.314 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-08-19T10:38:22.362 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-08-19T10:38:22.362 DEBUG:teuthology.orchestra.run.smithi190:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-08-19T10:38:22.409 INFO:tasks.workunit:timeout=3h 2024-08-19T10:38:22.410 INFO:tasks.workunit:cleanup=True 2024-08-19T10:38:22.410 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 964adda23da9f7b057cb34fdc0b82aeb7ed65324 2024-08-19T10:38:22.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient: get_monmap_and_config 2024-08-19T10:38:22.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient: build_initial_monmap 2024-08-19T10:38:22.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient: monmap: 2024-08-19T10:38:22.420 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:22.420 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.420 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:22.421259+0000 2024-08-19T10:38:22.420 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:22.421259+0000 2024-08-19T10:38:22.420 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:22.420 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.420 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:22.420 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:22.420 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:22.420 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:22.421 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:22.421 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:22.421 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:22.421 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:22.421 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:22.421 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:22.421 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:22.421 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 1 Processor -- start 2024-08-19T10:38:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 1 -- start start 2024-08-19T10:38:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient: init 2024-08-19T10:38:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient: _add_conns ranks=[4,9,10,6,7,2,0,8,3,5,1] 2024-08-19T10:38:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30d009b8e0 0x7f30d009bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient(hunting): picked mon.noname-c con 0x7f30d009b8e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:22.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f30d009ddd0 0x7f30d00a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient(hunting): picked mon.noname-a con 0x7f30d009ddd0 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f30d00a4240 0x7f30d00a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient(hunting): picked mon.noname-f con 0x7f30d00a4240 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f30d009d760 con 0x7f30d009ddd0 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f30d009d9b0 con 0x7f30d00a4240 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f30d006fd00 con 0x7f30d009b8e0 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d77db640 10 monclient(hunting): authenticate will time out at 952.237691s 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30d009b8e0 0x7f30d009bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d4d76640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f30d009ddd0 0x7f30d00a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30d009b8e0 0x7f30d009bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33812/0 (socket says 172.21.15.89:33812) 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 1 -- 172.21.15.89:0/37950094 learned_addr learned my addr 172.21.15.89:0/37950094 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 10 monclient(hunting): get_auth_request con 0x7f30d009b8e0 auth_method 0 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d4d76640 10 monclient(hunting): get_auth_request con 0x7f30d009ddd0 auth_method 0 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d4d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d4d76640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d4d76640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d4d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d4d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d4d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5d78640 1 --2- 172.21.15.89:0/37950094 >> v2:172.21.15.190:3300/0 conn(0x7f30d00a4240 0x7f30d00a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5d78640 10 monclient(hunting): get_auth_request con 0x7f30d00a4240 auth_method 0 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5d78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5d78640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5d78640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 10 monclient(hunting): handle_auth_done global_id 4387 payload 274 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 1 -- 172.21.15.89:0/37950094 >> v2:172.21.15.190:3300/0 conn(0x7f30d00a4240 msgr2=0x7f30d00a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 1 --2- 172.21.15.89:0/37950094 >> v2:172.21.15.190:3300/0 conn(0x7f30d00a4240 0x7f30d00a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 1 -- 172.21.15.89:0/37950094 >> v2:172.21.15.89:3300/0 conn(0x7f30d009ddd0 msgr2=0x7f30d00a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 1 --2- 172.21.15.89:0/37950094 >> v2:172.21.15.89:3300/0 conn(0x7f30d009ddd0 0x7f30d00a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 10 monclient: _finish_hunting 0 2024-08-19T10:38:22.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 1 monclient: found mon.noname-c 2024-08-19T10:38:22.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:22.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:22.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d5577640 1 -- 172.21.15.89:0/37950094 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d006db50 con 0x7f30d009b8e0 2024-08-19T10:38:22.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.418+0000 7f30d4d76640 1 --2- 172.21.15.89:0/37950094 >> v2:172.21.15.89:3300/0 conn(0x7f30d009ddd0 0x7f30d00a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d5577640 1 -- 172.21.15.89:0/37950094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30d009b8e0 msgr2=0x7f30d009bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:22.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d5577640 1 --2- 172.21.15.89:0/37950094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30d009b8e0 0x7f30d009bcc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f30d009e8c0 tx=0x7f30cc01c3b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=df0e7aa9e2f7e86e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 1 -- 172.21.15.89:0/37950094 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30cc0b0070 con 0x7f30d009b8e0 2024-08-19T10:38:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 10 monclient: dump: 2024-08-19T10:38:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.434 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.435 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.436 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.437 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 10 monclient: _finish_auth 0 2024-08-19T10:38:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 1 -- 172.21.15.89:0/37950094 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30cc0aa810 con 0x7f30d009b8e0 2024-08-19T10:38:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 5 monclient: authenticate success, global_id 4387 2024-08-19T10:38:22.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 1 -- 172.21.15.89:0/37950094 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30cc0aae30 con 0x7f30d009b8e0 2024-08-19T10:38:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 4 set_mon_vals no callback set 2024-08-19T10:38:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30c7fff640 10 monclient: dump: 2024-08-19T10:38:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.439 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.440 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.441 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 10 monclient: shutdown 2024-08-19T10:38:22.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 1 -- 172.21.15.89:0/37950094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30d009b8e0 msgr2=0x7f30d009bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 1 --2- 172.21.15.89:0/37950094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30d009b8e0 0x7f30d009bcc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f30d009e8c0 tx=0x7f30cc01c3b0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 1 -- 172.21.15.89:0/37950094 shutdown_connections 2024-08-19T10:38:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 1 --2- 172.21.15.89:0/37950094 >> v2:172.21.15.190:3300/0 conn(0x7f30d00a4240 0x7f30d00a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 1 --2- 172.21.15.89:0/37950094 >> v2:172.21.15.89:3300/0 conn(0x7f30d009ddd0 0x7f30d00a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 1 --2- 172.21.15.89:0/37950094 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30d009b8e0 0x7f30d009bcc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 1 -- 172.21.15.89:0/37950094 >> 172.21.15.89:0/37950094 conn(0x7f30d00a4df0 msgr2=0x7f30d00a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 1 -- 172.21.15.89:0/37950094 shutdown_connections 2024-08-19T10:38:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 1 -- 172.21.15.89:0/37950094 wait complete. 2024-08-19T10:38:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 10 monclient: build_initial_monmap 2024-08-19T10:38:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.422+0000 7f30d77db640 10 monclient: monmap: 2024-08-19T10:38:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:22.443 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:22.444 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:22.445 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 1 Processor -- start 2024-08-19T10:38:22.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 1 -- start start 2024-08-19T10:38:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient: init 2024-08-19T10:38:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient: _add_conns ranks=[0,8,2,6,4,1,7,5,3] 2024-08-19T10:38:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d009b8e0 0x7f30d009a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient(hunting): picked mon.noname-a con 0x7f30d009b8e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:22.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30d009ddd0 0x7f30d009ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient(hunting): picked mon.noname-e con 0x7f30d009ddd0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30d00a4240 0x7f30d0095090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient(hunting): picked mon.noname-b con 0x7f30d00a4240 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30d0069300 con 0x7f30d009b8e0 2024-08-19T10:38:22.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f30d00a5c00 con 0x7f30d00a4240 2024-08-19T10:38:22.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f30d0142cd0 con 0x7f30d009ddd0 2024-08-19T10:38:22.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:22.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient(hunting): authenticate will time out at 952.244804s 2024-08-19T10:38:22.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d009b8e0 0x7f30d009a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d009b8e0 0x7f30d009a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33052/0 (socket says 172.21.15.89:33052) 2024-08-19T10:38:22.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 1 -- 172.21.15.89:0/914242396 learned_addr learned my addr 172.21.15.89:0/914242396 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:22.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 10 monclient(hunting): get_auth_request con 0x7f30d009b8e0 auth_method 0 2024-08-19T10:38:22.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d4d76640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30d009ddd0 0x7f30d009ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5d78640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30d00a4240 0x7f30d0095090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d4d76640 10 monclient(hunting): get_auth_request con 0x7f30d009ddd0 auth_method 0 2024-08-19T10:38:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d4d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d4d76640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d4d76640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5d78640 10 monclient(hunting): get_auth_request con 0x7f30d00a4240 auth_method 0 2024-08-19T10:38:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5d78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5d78640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5d78640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d4d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d4d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d4d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5d78640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5d78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5d78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 10 monclient(hunting): handle_auth_done global_id 4473 payload 824 2024-08-19T10:38:22.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 1 -- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30d009ddd0 msgr2=0x7f30d009ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30d009ddd0 0x7f30d009ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 1 -- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30d00a4240 msgr2=0x7f30d0095090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30d00a4240 0x7f30d0095090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 10 monclient: _finish_hunting 0 2024-08-19T10:38:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 1 monclient: found mon.noname-a 2024-08-19T10:38:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 1 -- 172.21.15.89:0/914242396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d0098640 con 0x7f30d009b8e0 2024-08-19T10:38:22.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 1 -- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d009b8e0 msgr2=0x7f30d009a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d5577640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d009b8e0 0x7f30d009a620 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30cc01cc50 tx=0x7f30cc019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 1 -- 172.21.15.89:0/914242396 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30cc0ba070 con 0x7f30d009b8e0 2024-08-19T10:38:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 10 monclient: dump: 2024-08-19T10:38:22.454 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.455 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 1 -- 172.21.15.89:0/914242396 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30cc0b1040 con 0x7f30d009b8e0 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 5 monclient: authenticate success, global_id 4473 2024-08-19T10:38:22.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient: _renew_subs 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 1 -- 172.21.15.89:0/914242396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d00a34d0 con 0x7f30d009b8e0 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 1 -- 172.21.15.89:0/914242396 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30cc0b0070 con 0x7f30d009b8e0 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c5ffb640 10 monclient: dump: 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.457 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.458 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.462 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient: _renew_subs 2024-08-19T10:38:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30d77db640 1 -- 172.21.15.89:0/914242396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d0098ee0 con 0x7f30d009b8e0 2024-08-19T10:38:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.426+0000 7f30c77fe640 4 set_mon_vals no callback set 2024-08-19T10:38:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.430+0000 7f30d77db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.430+0000 7f30d77db640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.430+0000 7f30d77db640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.430+0000 7f30d77db640 1 -- 172.21.15.89:0/914242396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d00a3d00 con 0x7f30d009b8e0 2024-08-19T10:38:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.430+0000 7f30c5ffb640 1 -- 172.21.15.89:0/914242396 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f30cc0bd060 con 0x7f30d009b8e0 2024-08-19T10:38:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.430+0000 7f30c5ffb640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f30b403be00 0x7f30b403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.430+0000 7f30c5ffb640 1 -- 172.21.15.89:0/914242396 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f30cc0e7ae0 con 0x7f30d009b8e0 2024-08-19T10:38:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.430+0000 7f30d4d76640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f30b403be00 0x7f30b403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.430+0000 7f30d4d76640 10 monclient: get_auth_request con 0x7f30b403be00 auth_method 0 2024-08-19T10:38:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.434+0000 7f30d4d76640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f30b403be00 0x7f30b403e2c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009580 tx=0x7f30c0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.434+0000 7f30c5ffb640 1 -- 172.21.15.89:0/914242396 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f30d0098640 con 0x7f30d009b8e0 2024-08-19T10:38:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.434+0000 7f30c5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.434+0000 7f30c5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:22.471 INFO:tasks.workunit.client.0.smithi190.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-19T10:38:22.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.650+0000 7f30d77db640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:38:22.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.650+0000 7f30d77db640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:38:22.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.650+0000 7f30d77db640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:22.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.650+0000 7f30d77db640 1 -- 172.21.15.89:0/914242396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f30d006e560 con 0x7f30d009b8e0 2024-08-19T10:38:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.650+0000 7f30c5ffb640 1 -- 172.21.15.89:0/914242396 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7f30cc0160f0 con 0x7f30d009b8e0 2024-08-19T10:38:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.650+0000 7f30c5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:38:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.650+0000 7f30c5ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:22.654 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":4,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":31,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:38:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 -- 172.21.15.89:0/914242396 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f30b403be00 msgr2=0x7f30b403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f30b403be00 0x7f30b403e2c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009580 tx=0x7f30c0009190 comp rx=0 tx=0).stop 2024-08-19T10:38:22.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 10 monclient: shutdown 2024-08-19T10:38:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 -- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d009b8e0 msgr2=0x7f30d009a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d009b8e0 0x7f30d009a620 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30cc01cc50 tx=0x7f30cc019150 comp rx=0 tx=0).stop 2024-08-19T10:38:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 -- 172.21.15.89:0/914242396 shutdown_connections 2024-08-19T10:38:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30d00a4240 0x7f30d0095090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f30b403be00 0x7f30b403e2c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30d009ddd0 0x7f30d009ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 --2- 172.21.15.89:0/914242396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d009b8e0 0x7f30d009a620 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 -- 172.21.15.89:0/914242396 >> 172.21.15.89:0/914242396 conn(0x7f30d00a4df0 msgr2=0x7f30d00a05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 -- 172.21.15.89:0/914242396 shutdown_connections 2024-08-19T10:38:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.654+0000 7f30d77db640 1 -- 172.21.15.89:0/914242396 wait complete. 2024-08-19T10:38:22.672 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:38:22.672 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-08-19T10:38:22.672 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:38:22.673 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:38:22.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient: get_monmap_and_config 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient: build_initial_monmap 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient: monmap: 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:22.781254+0000 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:22.781254+0000 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:22.780 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:22.781 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:22.781 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:22.781 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:22.781 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:22.781 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:22.781 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:22.781 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 1 Processor -- start 2024-08-19T10:38:22.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 1 -- start start 2024-08-19T10:38:22.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient: init 2024-08-19T10:38:22.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:22.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient: _add_conns ranks=[2,0,6,1,7,5,10,4,8,9,3] 2024-08-19T10:38:22.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c200a5270 0x7f3c200a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient(hunting): picked mon.noname-b con 0x7f3c200a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:22.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3c200a6d30 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:22.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3c200a7110 0x7f3c20139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient(hunting): picked mon.noname-d con 0x7f3c200a7110 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:22.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c200a8710 con 0x7f3c200a6d30 2024-08-19T10:38:22.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3c200a5a40 con 0x7f3c200a5270 2024-08-19T10:38:22.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f3c200a81d0 con 0x7f3c200a7110 2024-08-19T10:38:22.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:22.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c25b39640 10 monclient(hunting): authenticate will time out at 952.597899s 2024-08-19T10:38:22.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c200a5270 0x7f3c200a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1effd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3c200a6d30 0x7f3c201395b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44632/0 (socket says 172.21.15.89:44632) 2024-08-19T10:38:22.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 10 monclient(hunting): get_auth_request con 0x7f3c200a5270 auth_method 0 2024-08-19T10:38:22.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1effd640 1 -- 172.21.15.89:0/4237792790 learned_addr learned my addr 172.21.15.89:0/4237792790 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:22.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1ffff640 1 --2- 172.21.15.89:0/4237792790 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3c200a7110 0x7f3c20139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1ffff640 10 monclient(hunting): get_auth_request con 0x7f3c200a7110 auth_method 0 2024-08-19T10:38:22.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_done global_id 4304 payload 274 2024-08-19T10:38:22.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 1 -- 172.21.15.89:0/4237792790 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3c200a7110 msgr2=0x7f3c20139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:22.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 1 --2- 172.21.15.89:0/4237792790 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3c200a7110 0x7f3c20139cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 1 -- 172.21.15.89:0/4237792790 >> v1:172.21.15.89:6789/0 conn(0x7f3c200a6d30 legacy=0x7f3c201395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:22.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:38:22.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 1 monclient: found mon.noname-b 2024-08-19T10:38:22.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:22.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:22.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1f7fe640 1 -- 172.21.15.89:0/4237792790 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c200a4790 con 0x7f3c200a5270 2024-08-19T10:38:22.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.778+0000 7f3c1ffff640 1 --2- 172.21.15.89:0/4237792790 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3c200a7110 0x7f3c20139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:22.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1f7fe640 1 -- 172.21.15.89:0/4237792790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c200a5270 msgr2=0x7f3c200a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:22.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1f7fe640 1 --2- 172.21.15.89:0/4237792790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c200a5270 0x7f3c200a67f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c180097f0 tx=0x7f3c1801c190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f42d62d3ceda08f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 1 -- 172.21.15.89:0/4237792790 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c180b0070 con 0x7f3c200a5270 2024-08-19T10:38:22.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:22.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 10 monclient: dump: 2024-08-19T10:38:22.805 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.805 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.805 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.805 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.805 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.806 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.806 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.806 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.806 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.806 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.806 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.806 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.806 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.806 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 1 -- 172.21.15.89:0/4237792790 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c180aa810 con 0x7f3c200a5270 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 1 -- 172.21.15.89:0/4237792790 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c180aae30 con 0x7f3c200a5270 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 5 monclient: authenticate success, global_id 4304 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 4 set_mon_vals no callback set 2024-08-19T10:38:22.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c1e7fc640 10 monclient: dump: 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.808 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 10 monclient: shutdown 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 1 -- 172.21.15.89:0/4237792790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c200a5270 msgr2=0x7f3c200a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 1 --2- 172.21.15.89:0/4237792790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c200a5270 0x7f3c200a67f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c180097f0 tx=0x7f3c1801c190 comp rx=0 tx=0).stop 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 1 -- 172.21.15.89:0/4237792790 shutdown_connections 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 1 --2- 172.21.15.89:0/4237792790 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3c200a7110 0x7f3c20139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 1 --2- 172.21.15.89:0/4237792790 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c200a5270 0x7f3c200a67f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 1 -- 172.21.15.89:0/4237792790 >> 172.21.15.89:0/4237792790 conn(0x7f3c20069a80 msgr2=0x7f3c200abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 1 -- 172.21.15.89:0/4237792790 shutdown_connections 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 1 -- 172.21.15.89:0/4237792790 wait complete. 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 10 monclient: build_initial_monmap 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.782+0000 7f3c25b39640 10 monclient: monmap: 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:22.810 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 1 Processor -- start 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 1 -- start start 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 10 monclient: init 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 10 monclient: _add_conns ranks=[7,3,8,5,4,1,6,0,2] 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c200a5270 0x7f3c20138c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 10 monclient(hunting): picked mon.noname-i con 0x7f3c200a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c200a6d30 0x7f3c20139140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 10 monclient(hunting): picked mon.noname-g con 0x7f3c200a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c200a7110 0x7f3c2013e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 10 monclient(hunting): picked mon.noname-e con 0x7f3c200a7110 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3c200a57d0 con 0x7f3c200a6d30 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f3c200a5650 con 0x7f3c200a5270 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3c2013a200 con 0x7f3c200a7110 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 10 monclient(hunting): authenticate will time out at 952.604780s 2024-08-19T10:38:22.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c200a7110 0x7f3c2013e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c200a7110 0x7f3c2013e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:53196/0 (socket says 172.21.15.89:53196) 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 1 -- 172.21.15.89:0/2517023154 learned_addr learned my addr 172.21.15.89:0/2517023154 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 10 monclient(hunting): get_auth_request con 0x7f3c200a7110 auth_method 0 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1f7fe640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c200a5270 0x7f3c20138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 10 monclient(hunting): handle_auth_done global_id 4337 payload 824 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1effd640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c200a6d30 0x7f3c20139140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 1 -- 172.21.15.89:0/2517023154 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c200a5270 msgr2=0x7f3c20138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c200a5270 0x7f3c20138c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 1 -- 172.21.15.89:0/2517023154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c200a6d30 msgr2=0x7f3c20139140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3c200a6d30 0x7f3c20139140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 1 monclient: found mon.noname-e 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 1 -- 172.21.15.89:0/2517023154 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c20123210 con 0x7f3c200a7110 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 1 -- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c200a7110 msgr2=0x7f3c2013e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1ffff640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c200a7110 0x7f3c2013e330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000e950 tx=0x7f3c1000ee10 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c10017070 con 0x7f3c200a7110 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:22.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 10 monclient: dump: 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.815 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.816 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 10 monclient: _finish_auth 0 2024-08-19T10:38:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1000cc10 con 0x7f3c200a7110 2024-08-19T10:38:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c25b39640 5 monclient: authenticate success, global_id 4337 2024-08-19T10:38:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c1000cdc0 con 0x7f3c200a7110 2024-08-19T10:38:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:22.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c17fff640 10 monclient: dump: 2024-08-19T10:38:22.817 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.817 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.817 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.817 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.817 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.786+0000 7f3c1dffb640 4 set_mon_vals no callback set 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c25b39640 10 monclient: _renew_subs 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c25b39640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c25b39640 1 -- 172.21.15.89:0/2517023154 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c201401a0 con 0x7f3c200a7110 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c25b39640 10 monclient: _renew_subs 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c25b39640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c25b39640 1 -- 172.21.15.89:0/2517023154 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c201237d0 con 0x7f3c200a7110 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c25b39640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c25b39640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f030 0x7f3c2002d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c25b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c25b39640 1 -- 172.21.15.89:0/2517023154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20140690 con 0x7f3c2013f030 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f030 0x7f3c2002d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 10 monclient: get_auth_request con 0x7f3c2013f030 auth_method 0 2024-08-19T10:38:22.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3c100059a0 con 0x7f3c200a7110 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_done global_id 4482 payload 824 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 1 -- 172.21.15.89:0/2517023154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c2013fd00 con 0x7f3c2013f030 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c0403be00 0x7f3c0403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f3c1000f040 con 0x7f3c200a7110 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1effd640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c0403be00 0x7f3c0403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1effd640 10 monclient: get_auth_request con 0x7f3c0403be00 auth_method 0 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 1 -- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f030 msgr2=0x7f3c2002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:22.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c1f7fe640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f030 0x7f3c2002d1d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c200a34f0 tx=0x7f3c18018f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c180b9070 con 0x7f3c2013f030 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 10 monclient: dump: 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.821 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.822 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.822 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.822 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.822 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.822 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.822 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.822 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3c18003830 con 0x7f3c2013f030 2024-08-19T10:38:22.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:22.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f030 msgr2=0x7f3c2002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f030 0x7f3c2002d1d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c200a34f0 tx=0x7f3c18018f30 comp rx=0 tx=0).stop 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f030 msgr2=0x7f3c2002d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.790+0000 7f3c17fff640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f030 0x7f3c2002d1d0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c200a34f0 tx=0x7f3c18018f30 comp rx=0 tx=0).stop 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.794+0000 7f3c1effd640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c0403be00 0x7f3c0403e2c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3c2013e210 tx=0x7f3c08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c25b39640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c25b39640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f410 0x7f3c2002d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c25b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c25b39640 1 -- 172.21.15.89:0/2517023154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3c200a67f0 con 0x7f3c2013f410 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f410 0x7f3c2002d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 10 monclient: get_auth_request con 0x7f3c2013f410 auth_method 0 2024-08-19T10:38:22.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 10 monclient(hunting): handle_auth_done global_id 4491 payload 824 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 1 -- 172.21.15.89:0/2517023154 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3c20064590 con 0x7f3c2013f410 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 1 -- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f410 msgr2=0x7f3c2002d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.798+0000 7f3c1f7fe640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f410 0x7f3c2002d710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c10 tx=0x7f3c180a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c180b0040 con 0x7f3c2013f410 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c17fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c17fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c17fff640 10 monclient: dump: 2024-08-19T10:38:22.824 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.825 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.826 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7f3c180bfe10 con 0x7f3c2013f410 2024-08-19T10:38:22.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c17fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:22.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c17fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:22.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f410 msgr2=0x7f3c2002d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c17fff640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f410 0x7f3c2002d710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c10 tx=0x7f3c180a8000 comp rx=0 tx=0).stop 2024-08-19T10:38:22.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c17fff640 1 -- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f410 msgr2=0x7f3c2002d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:22.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c17fff640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c2013f410 0x7f3c2002d710 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c10 tx=0x7f3c180a8000 comp rx=0 tx=0).stop 2024-08-19T10:38:22.826 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 4, 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 32615, 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:22.827 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:38:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 31, 2024-08-19T10:38:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:22.828 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:22.829 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:22.830 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:22.831 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:22.832 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:22.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:22.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:22.836 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.837 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:22.838 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:22.840 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:22.843 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:22.845 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.846 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:22.847 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:22.849 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:22.850 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:22.851 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:22.851 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c25b39640 1 -- 172.21.15.89:0/2517023154 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c0403be00 msgr2=0x7f3c0403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c25b39640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c0403be00 0x7f3c0403e2c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3c2013e210 tx=0x7f3c08009040 comp rx=0 tx=0).stop 2024-08-19T10:38:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c25b39640 10 monclient: shutdown 2024-08-19T10:38:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c25b39640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c25b39640 1 -- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c200a7110 msgr2=0x7f3c2013e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c25b39640 1 --2- 172.21.15.89:0/2517023154 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c200a7110 0x7f3c2013e330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000e950 tx=0x7f3c1000ee10 comp rx=0 tx=0).stop 2024-08-19T10:38:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c1ffff640 1 -- 172.21.15.89:0/2517023154 reap_dead start 2024-08-19T10:38:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c1ffff640 1 -- 172.21.15.89:0/2517023154 reap_dead start 2024-08-19T10:38:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c25b39640 1 -- 172.21.15.89:0/2517023154 shutdown_connections 2024-08-19T10:38:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c25b39640 1 -- 172.21.15.89:0/2517023154 >> 172.21.15.89:0/2517023154 conn(0x7f3c20069a80 msgr2=0x7f3c200aa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:22.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c25b39640 1 -- 172.21.15.89:0/2517023154 shutdown_connections 2024-08-19T10:38:22.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.802+0000 7f3c25b39640 1 -- 172.21.15.89:0/2517023154 wait complete. 2024-08-19T10:38:22.853 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:38:22.853 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:38:22.853 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:38:22.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.962+0000 7f78b0e70640 10 monclient: get_monmap_and_config 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.962+0000 7f78b0e70640 10 monclient: build_initial_monmap 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.962+0000 7f78b0e70640 10 monclient: monmap: 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:22.967321+0000 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:22.967321+0000 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:22.966 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:22.967 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:22.967 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:22.967 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:22.967 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:22.967 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:22.967 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:22.967 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:22.967 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.962+0000 7f78b0e70640 1 Processor -- start 2024-08-19T10:38:22.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.962+0000 7f78b0e70640 1 -- start start 2024-08-19T10:38:22.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.962+0000 7f78b0e70640 10 monclient: init 2024-08-19T10:38:22.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:22.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 10 monclient: _add_conns ranks=[4,3,5,2,10,0,9,6,8,1,7] 2024-08-19T10:38:22.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac06c930 0x7f78ac06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 10 monclient(hunting): picked mon.noname-c con 0x7f78ac06c930 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78ac09d340 0x7f78ac06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 10 monclient(hunting): picked mon.noname-g con 0x7f78ac09d340 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f78ac137500 0x7f78ac137900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 10 monclient(hunting): picked mon.noname-h con 0x7f78ac137500 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac0a54f0 con 0x7f78ac09d340 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f78ac0a3e00 con 0x7f78ac06c930 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f78ac06d790 con 0x7f78ac137500 2024-08-19T10:38:22.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78b0e70640 10 monclient(hunting): authenticate will time out at 952.784591s 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac06c930 0x7f78ac06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac06c930 0x7f78ac06cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33818/0 (socket says 172.21.15.89:33818) 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 1 -- 172.21.15.89:0/938811481 learned_addr learned my addr 172.21.15.89:0/938811481 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 10 monclient(hunting): get_auth_request con 0x7f78ac06c930 auth_method 0 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aad9d640 1 --2- 172.21.15.89:0/938811481 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f78ac137500 0x7f78ac137900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78a1d9b640 1 --2- 172.21.15.89:0/938811481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78ac09d340 0x7f78ac06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78a1d9b640 10 monclient(hunting): get_auth_request con 0x7f78ac09d340 auth_method 0 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78a1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aad9d640 10 monclient(hunting): get_auth_request con 0x7f78ac137500 auth_method 0 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 10 monclient(hunting): handle_auth_done global_id 4396 payload 274 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 1 -- 172.21.15.89:0/938811481 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f78ac137500 msgr2=0x7f78ac137900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 1 --2- 172.21.15.89:0/938811481 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f78ac137500 0x7f78ac137900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 1 -- 172.21.15.89:0/938811481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78ac09d340 msgr2=0x7f78ac06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 1 --2- 172.21.15.89:0/938811481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78ac09d340 0x7f78ac06d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 10 monclient: _finish_hunting 0 2024-08-19T10:38:22.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 1 monclient: found mon.noname-c 2024-08-19T10:38:22.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:22.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:22.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.966+0000 7f78aa59c640 1 -- 172.21.15.89:0/938811481 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac09fbf0 con 0x7f78ac06c930 2024-08-19T10:38:22.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78aa59c640 1 -- 172.21.15.89:0/938811481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac06c930 msgr2=0x7f78ac06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:22.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78aa59c640 1 --2- 172.21.15.89:0/938811481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac06c930 0x7f78ac06cd10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f789c0098a0 tx=0x7f789c01c3b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=b7b95a1bf4b329bd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 1 -- 172.21.15.89:0/938811481 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f789c0b0070 con 0x7f78ac06c930 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 10 monclient: dump: 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.977 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 10 monclient: _finish_auth 0 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 1 -- 172.21.15.89:0/938811481 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789c0aa810 con 0x7f78ac06c930 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:22.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 1 -- 172.21.15.89:0/938811481 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f789c0aae30 con 0x7f78ac06c930 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 5 monclient: authenticate success, global_id 4396 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 4 set_mon_vals no callback set 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78a9d9b640 10 monclient: dump: 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.979 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient: shutdown 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 -- 172.21.15.89:0/938811481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac06c930 msgr2=0x7f78ac06cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 --2- 172.21.15.89:0/938811481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac06c930 0x7f78ac06cd10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f789c0098a0 tx=0x7f789c01c3b0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 -- 172.21.15.89:0/938811481 shutdown_connections 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 --2- 172.21.15.89:0/938811481 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f78ac137500 0x7f78ac137900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 --2- 172.21.15.89:0/938811481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78ac09d340 0x7f78ac06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 --2- 172.21.15.89:0/938811481 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac06c930 0x7f78ac06cd10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 -- 172.21.15.89:0/938811481 >> 172.21.15.89:0/938811481 conn(0x7f78ac0a8f60 msgr2=0x7f78ac0a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 -- 172.21.15.89:0/938811481 shutdown_connections 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 -- 172.21.15.89:0/938811481 wait complete. 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient: build_initial_monmap 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient: monmap: 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:22.981 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 Processor -- start 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 -- start start 2024-08-19T10:38:22.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient: init 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient: _add_conns ranks=[3,5,4,8,7,0,2,6,1] 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78ac06c930 0x7f78ac13c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient(hunting): picked mon.noname-g con 0x7f78ac06c930 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f78ac09d340 0x7f78ac136530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient(hunting): picked mon.noname-h con 0x7f78ac09d340 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac137500 0x7f78ac11f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient(hunting): picked mon.noname-c con 0x7f78ac137500 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac138250 con 0x7f78ac06c930 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f78ac074420 con 0x7f78ac137500 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f78ac140f70 con 0x7f78ac09d340 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78b0e70640 10 monclient(hunting): authenticate will time out at 952.791804s 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78aad9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac137500 0x7f78ac11f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78aad9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac137500 0x7f78ac11f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33828/0 (socket says 172.21.15.89:33828) 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78aad9d640 1 -- 172.21.15.89:0/1628935343 learned_addr learned my addr 172.21.15.89:0/1628935343 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78aad9d640 10 monclient(hunting): get_auth_request con 0x7f78ac137500 auth_method 0 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78aad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.970+0000 7f78aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a1d9b640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f78ac09d340 0x7f78ac136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78ac06c930 0x7f78ac13c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a1d9b640 10 monclient(hunting): get_auth_request con 0x7f78ac09d340 auth_method 0 2024-08-19T10:38:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient(hunting): get_auth_request con 0x7f78ac06c930 auth_method 0 2024-08-19T10:38:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 10 monclient(hunting): handle_auth_done global_id 4405 payload 824 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 1 -- 172.21.15.89:0/1628935343 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f78ac09d340 msgr2=0x7f78ac136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f78ac09d340 0x7f78ac136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 1 -- 172.21.15.89:0/1628935343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78ac06c930 msgr2=0x7f78ac13c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78ac06c930 0x7f78ac13c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 10 monclient: _finish_hunting 0 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 1 monclient: found mon.noname-c 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:22.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 1 -- 172.21.15.89:0/1628935343 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac0a96d0 con 0x7f78ac137500 2024-08-19T10:38:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 1 -- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac137500 msgr2=0x7f78ac11f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aad9d640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac137500 0x7f78ac11f380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009fd0 tx=0x7f78a400e4b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78a4014890 con 0x7f78ac137500 2024-08-19T10:38:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 10 monclient: dump: 2024-08-19T10:38:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.992 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:22.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78b0e70640 5 monclient: authenticate success, global_id 4405 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a4019070 con 0x7f78ac137500 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78a4013380 con 0x7f78ac137500 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78b0e70640 10 monclient: _renew_subs 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78b0e70640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a959a640 4 set_mon_vals no callback set 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78b0e70640 1 -- 172.21.15.89:0/1628935343 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ac121c10 con 0x7f78ac137500 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 10 monclient: dump: 2024-08-19T10:38:22.994 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.995 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78b0e70640 10 monclient: _renew_subs 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78b0e70640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78b0e70640 1 -- 172.21.15.89:0/1628935343 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ac1206f0 con 0x7f78ac137500 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78b0e70640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78b0e70640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac136a70 0x7f78ac11f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78b0e70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78b0e70640 1 -- 172.21.15.89:0/1628935343 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac13d840 con 0x7f78ac136a70 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac136a70 0x7f78ac11f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient: get_auth_request con 0x7f78ac136a70 auth_method 0 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:22.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f78a4014420 con 0x7f78ac137500 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 10 monclient(hunting): handle_auth_done global_id 4313 payload 824 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78aa59c640 1 -- 172.21.15.89:0/1628935343 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78ac09e9e0 con 0x7f78ac136a70 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f788c03be00 0x7f788c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.974+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f78a404f090 con 0x7f78ac137500 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a1d9b640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f788c03be00 0x7f788c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a1d9b640 10 monclient: get_auth_request con 0x7f788c03be00 auth_method 0 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78aa59c640 1 -- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac136a70 msgr2=0x7f78ac11f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78aa59c640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac136a70 0x7f78ac11f8c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f789c0ad000 tx=0x7f789c0191b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f789c0b9070 con 0x7f78ac136a70 2024-08-19T10:38:22.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a37fe640 10 monclient: dump: 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:22.998 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f789c0b1040 con 0x7f78ac136a70 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a37fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac136a70 msgr2=0x7f78ac11f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a37fe640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac136a70 0x7f78ac11f8c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f789c0ad000 tx=0x7f789c0191b0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac136a70 msgr2=0x7f78ac11f8c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a37fe640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac136a70 0x7f78ac11f8c0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f789c0ad000 tx=0x7f789c0191b0 comp rx=0 tx=0).stop 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.978+0000 7f78a1d9b640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f788c03be00 0x7f788c03e2c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f78ac13c450 tx=0x7f7894008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:22.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac11fe00 0x7f78ac072e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 1 -- 172.21.15.89:0/1628935343 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac120210 con 0x7f78ac11fe00 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac11fe00 0x7f78ac072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 10 monclient: get_auth_request con 0x7f78ac11fe00 auth_method 0 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 10 monclient(hunting): handle_auth_done global_id 4322 payload 824 2024-08-19T10:38:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 1 -- 172.21.15.89:0/1628935343 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f78ac064590 con 0x7f78ac11fe00 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 1 -- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac11fe00 msgr2=0x7f78ac072e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aa59c640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac11fe00 0x7f78ac072e20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f789c01c1e0 tx=0x7f789c0a8000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f789c0b0050 con 0x7f78ac11fe00 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78a37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78a37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78a37fe640 10 monclient: dump: 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7f789c0c2070 con 0x7f78ac11fe00 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78a37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78a37fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac11fe00 msgr2=0x7f78ac072e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78a37fe640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac11fe00 0x7f78ac072e20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f789c01c1e0 tx=0x7f789c0a8000 comp rx=0 tx=0).stop 2024-08-19T10:38:23.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78a37fe640 1 -- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac11fe00 msgr2=0x7f78ac072e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:23.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78a37fe640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78ac11fe00 0x7f78ac072e20 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f789c01c1e0 tx=0x7f789c0a8000 comp rx=0 tx=0).stop 2024-08-19T10:38:23.003 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 4, 2024-08-19T10:38:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 32815, 2024-08-19T10:38:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:23.003 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 32, 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:23.004 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:23.005 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.006 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:23.007 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.008 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.009 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.010 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.012 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.013 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.014 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.015 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.017 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.020 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.021 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.022 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.023 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.024 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.025 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:23.026 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:23.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 1 -- 172.21.15.89:0/1628935343 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f788c03be00 msgr2=0x7f788c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f788c03be00 0x7f788c03e2c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f78ac13c450 tx=0x7f7894008040 comp rx=0 tx=0).stop 2024-08-19T10:38:23.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 10 monclient: shutdown 2024-08-19T10:38:23.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:23.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 1 -- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac137500 msgr2=0x7f78ac11f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 1 --2- 172.21.15.89:0/1628935343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78ac137500 0x7f78ac11f380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009fd0 tx=0x7f78a400e4b0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aad9d640 1 -- 172.21.15.89:0/1628935343 reap_dead start 2024-08-19T10:38:23.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78aad9d640 1 -- 172.21.15.89:0/1628935343 reap_dead start 2024-08-19T10:38:23.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 1 -- 172.21.15.89:0/1628935343 shutdown_connections 2024-08-19T10:38:23.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 1 -- 172.21.15.89:0/1628935343 >> 172.21.15.89:0/1628935343 conn(0x7f78ac0a8f60 msgr2=0x7f78ac09ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:23.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 1 -- 172.21.15.89:0/1628935343 shutdown_connections 2024-08-19T10:38:23.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:22.986+0000 7f78b0e70640 1 -- 172.21.15.89:0/1628935343 wait complete. 2024-08-19T10:38:23.029 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:38:23.029 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:38:23.030 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:38:23.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient: get_monmap_and_config 2024-08-19T10:38:23.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient: build_initial_monmap 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient: monmap: 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:23.145431+0000 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:23.145431+0000 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:23.145 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:23.146 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:23.146 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 1 Processor -- start 2024-08-19T10:38:23.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 1 -- start start 2024-08-19T10:38:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient: init 2024-08-19T10:38:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient: _add_conns ranks=[2,7,9,10,4,5,1,6,0,8,3] 2024-08-19T10:38:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38980a6860 0x7f38980a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient(hunting): picked mon.noname-b con 0x7f38980a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f38980a7180 0x7f38980a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient(hunting): picked mon.noname-i con 0x7f38980a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3898139620 0x7f3898139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient(hunting): picked mon.noname-a con 0x7f3898139620 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f38980a8190 con 0x7f3898139620 2024-08-19T10:38:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f38980a33d0 con 0x7f38980a6860 2024-08-19T10:38:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f38980a2ec0 con 0x7f38980a7180 2024-08-19T10:38:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f389dcb1640 10 monclient(hunting): authenticate will time out at 952.962503s 2024-08-19T10:38:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3898139620 0x7f3898139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f38977fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38980a6860 0x7f38980a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3898139620 0x7f3898139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33088/0 (socket says 172.21.15.89:33088) 2024-08-19T10:38:23.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 1 -- 172.21.15.89:0/3417570488 learned_addr learned my addr 172.21.15.89:0/3417570488 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f38977fe640 10 monclient(hunting): get_auth_request con 0x7f38980a6860 auth_method 0 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f38977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f38977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f38977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 10 monclient(hunting): get_auth_request con 0x7f3898139620 auth_method 0 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f38977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f38977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f38977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3896ffd640 1 --2- 172.21.15.89:0/3417570488 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f38980a7180 0x7f38980a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3896ffd640 10 monclient(hunting): get_auth_request con 0x7f38980a7180 auth_method 0 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3896ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3896ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3896ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 10 monclient(hunting): handle_auth_done global_id 4500 payload 274 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 1 -- 172.21.15.89:0/3417570488 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f38980a7180 msgr2=0x7f38980a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 1 --2- 172.21.15.89:0/3417570488 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f38980a7180 0x7f38980a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 1 -- 172.21.15.89:0/3417570488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38980a6860 msgr2=0x7f38980a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 1 --2- 172.21.15.89:0/3417570488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38980a6860 0x7f38980a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:23.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 1 monclient: found mon.noname-a 2024-08-19T10:38:23.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:23.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:23.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f3897fff640 1 -- 172.21.15.89:0/3417570488 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38980a4cd0 con 0x7f3898139620 2024-08-19T10:38:23.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.142+0000 7f38977fe640 1 --2- 172.21.15.89:0/3417570488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38980a6860 0x7f38980a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:23.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f3897fff640 1 -- 172.21.15.89:0/3417570488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898139620 msgr2=0x7f3898139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f3897fff640 1 --2- 172.21.15.89:0/3417570488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898139620 0x7f3898139a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3890009e50 tx=0x7f389001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4a521a4180bec9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 1 -- 172.21.15.89:0/3417570488 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38900090d0 con 0x7f3898139620 2024-08-19T10:38:23.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:23.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 10 monclient: dump: 2024-08-19T10:38:23.161 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.161 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.161 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.162 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.162 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.162 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.162 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.162 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.162 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.163 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.163 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.163 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.163 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.163 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.163 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 10 monclient: _finish_auth 0 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 10 monclient: _check_auth_tickets 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 1 -- 172.21.15.89:0/3417570488 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38900a59a0 con 0x7f3898139620 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 1 -- 172.21.15.89:0/3417570488 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38900ad3f0 con 0x7f3898139620 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f389dcb1640 5 monclient: authenticate success, global_id 4500 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f389dcb1640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f389dcb1640 4 set_mon_vals no callback set 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.146+0000 7f38967fc640 10 monclient: dump: 2024-08-19T10:38:23.164 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.165 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.165 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.165 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.165 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.165 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.165 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient: shutdown 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 -- 172.21.15.89:0/3417570488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898139620 msgr2=0x7f3898139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 --2- 172.21.15.89:0/3417570488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898139620 0x7f3898139a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3890009e50 tx=0x7f389001c8f0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 -- 172.21.15.89:0/3417570488 shutdown_connections 2024-08-19T10:38:23.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 --2- 172.21.15.89:0/3417570488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3898139620 0x7f3898139a80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 --2- 172.21.15.89:0/3417570488 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f38980a7180 0x7f38980a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 --2- 172.21.15.89:0/3417570488 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38980a6860 0x7f38980a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 -- 172.21.15.89:0/3417570488 >> 172.21.15.89:0/3417570488 conn(0x7f3898074b40 msgr2=0x7f38980abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:23.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 -- 172.21.15.89:0/3417570488 shutdown_connections 2024-08-19T10:38:23.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 -- 172.21.15.89:0/3417570488 wait complete. 2024-08-19T10:38:23.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient: build_initial_monmap 2024-08-19T10:38:23.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient: monmap: 2024-08-19T10:38:23.167 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:23.168 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 Processor -- start 2024-08-19T10:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 -- start start 2024-08-19T10:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient: init 2024-08-19T10:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient: _add_conns ranks=[8,4,7,1,5,3,6,2,0] 2024-08-19T10:38:23.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38980a6860 0x7f3898134830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient(hunting): picked mon.noname-e con 0x7f38980a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:23.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38980a7180 0x7f3898134d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient(hunting): picked mon.noname-c con 0x7f38980a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:23.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3898139620 0x7f389809c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient(hunting): picked mon.noname-i con 0x7f3898139620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:23.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3898143210 con 0x7f38980a7180 2024-08-19T10:38:23.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f3898143090 con 0x7f3898139620 2024-08-19T10:38:23.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3898143390 con 0x7f38980a6860 2024-08-19T10:38:23.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:23.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f389dcb1640 10 monclient(hunting): authenticate will time out at 952.969854s 2024-08-19T10:38:23.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38980a6860 0x7f3898134830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f3896ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38980a7180 0x7f3898134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38980a6860 0x7f3898134830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:53212/0 (socket says 172.21.15.89:53212) 2024-08-19T10:38:23.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f3896ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38980a7180 0x7f3898134d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33834/0 (socket says 172.21.15.89:33834) 2024-08-19T10:38:23.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 1 -- 172.21.15.89:0/950607203 learned_addr learned my addr 172.21.15.89:0/950607203 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:23.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 10 monclient(hunting): get_auth_request con 0x7f38980a6860 auth_method 0 2024-08-19T10:38:23.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f3896ffd640 10 monclient(hunting): get_auth_request con 0x7f38980a7180 auth_method 0 2024-08-19T10:38:23.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f3896ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f3896ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f3896ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f3896ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f3896ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f3896ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f3897fff640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3898139620 0x7f389809c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 10 monclient(hunting): handle_auth_done global_id 4346 payload 824 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 1 -- 172.21.15.89:0/950607203 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3898139620 msgr2=0x7f389809c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3898139620 0x7f389809c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 1 -- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38980a7180 msgr2=0x7f3898134d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38980a7180 0x7f3898134d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 10 monclient: _finish_hunting 0 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 1 monclient: found mon.noname-e 2024-08-19T10:38:23.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:23.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:23.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.150+0000 7f38977fe640 1 -- 172.21.15.89:0/950607203 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38980a0030 con 0x7f38980a6860 2024-08-19T10:38:23.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f38977fe640 1 -- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38980a6860 msgr2=0x7f3898134830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f38977fe640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38980a6860 0x7f3898134830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f388800ebe0 tx=0x7f388800c520 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3888017070 con 0x7f38980a6860 2024-08-19T10:38:23.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:23.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 10 monclient: dump: 2024-08-19T10:38:23.174 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.175 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 10 monclient: _finish_auth 0 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388800ce40 con 0x7f38980a6860 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3888013d50 con 0x7f38980a6860 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f389dcb1640 5 monclient: authenticate success, global_id 4346 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f389dcb1640 10 monclient: _renew_subs 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f389dcb1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f389dcb1640 1 -- 172.21.15.89:0/950607203 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38980a0310 con 0x7f38980a6860 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3895ffb640 4 set_mon_vals no callback set 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:23.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 10 monclient: dump: 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.177 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.178 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f389dcb1640 10 monclient: _renew_subs 2024-08-19T10:38:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f389dcb1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f389dcb1640 1 -- 172.21.15.89:0/950607203 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f389809e020 con 0x7f38980a6860 2024-08-19T10:38:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f389dcb1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f389dcb1640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809cf20 0x7f389809d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f389dcb1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f389dcb1640 1 -- 172.21.15.89:0/950607203 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f38980a0d10 con 0x7f389809cf20 2024-08-19T10:38:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f388801a070 con 0x7f38980a6860 2024-08-19T10:38:23.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3896ffd640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809cf20 0x7f389809d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3896ffd640 10 monclient: get_auth_request con 0x7f389809cf20 auth_method 0 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3896ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3896ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3896ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3896ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3896ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3896ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f387803be30 0x7f387803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f388800f040 con 0x7f38980a6860 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3896ffd640 10 monclient(hunting): handle_auth_done global_id 4423 payload 824 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3896ffd640 1 -- 172.21.15.89:0/950607203 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38980a7560 con 0x7f389809cf20 2024-08-19T10:38:23.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3897fff640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f387803be30 0x7f387803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.154+0000 7f3897fff640 10 monclient: get_auth_request con 0x7f387803be30 auth_method 0 2024-08-19T10:38:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f3896ffd640 1 -- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809cf20 msgr2=0x7f389809d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f3896ffd640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809cf20 0x7f389809d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f389813f5a0 tx=0x7f3880010c80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3880018020 con 0x7f389809cf20 2024-08-19T10:38:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f388ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f388ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f388ffff640 10 monclient: dump: 2024-08-19T10:38:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.180 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3880004450 con 0x7f389809cf20 2024-08-19T10:38:23.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f388ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:23.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f388ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:23.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809cf20 msgr2=0x7f389809d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f388ffff640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809cf20 0x7f389809d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f389813f5a0 tx=0x7f3880010c80 comp rx=0 tx=0).stop 2024-08-19T10:38:23.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809cf20 msgr2=0x7f389809d300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:23.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f388ffff640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809cf20 0x7f389809d300 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f389813f5a0 tx=0x7f3880010c80 comp rx=0 tx=0).stop 2024-08-19T10:38:23.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.158+0000 7f3897fff640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f387803be30 0x7f387803e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f38980a9a20 tx=0x7f38900a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f389dcb1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f389dcb1640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809d840 0x7f389813aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f389dcb1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f389dcb1640 1 -- 172.21.15.89:0/950607203 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3898064590 con 0x7f389809d840 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f3896ffd640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809d840 0x7f389813aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f3896ffd640 10 monclient: get_auth_request con 0x7f389809d840 auth_method 0 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f3896ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f3896ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f3896ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f3896ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f3896ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f3896ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f3896ffd640 10 monclient(hunting): handle_auth_done global_id 4432 payload 824 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.162+0000 7f3896ffd640 1 -- 172.21.15.89:0/950607203 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f389809f790 con 0x7f389809d840 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f3896ffd640 1 -- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809d840 msgr2=0x7f389813aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f3896ffd640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809d840 0x7f389813aa80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3880019000 tx=0x7f38800046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3880018040 con 0x7f389809d840 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f388ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f388ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f388ffff640 10 monclient: dump: 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.185 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9433 (secure 0 0 0) 0x7f3880003620 con 0x7f389809d840 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f388ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:23.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f388ffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:23.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809d840 msgr2=0x7f389813aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f388ffff640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809d840 0x7f389813aa80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3880019000 tx=0x7f38800046d0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f388ffff640 1 -- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809d840 msgr2=0x7f389813aa80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:23.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f388ffff640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f389809d840 0x7f389813aa80 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3880019000 tx=0x7f38800046d0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.187 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 4, 2024-08-19T10:38:23.187 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 32983, 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 32, 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.188 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.189 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.190 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:23.191 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:23.192 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.193 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:23.195 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.198 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:23.200 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:23.201 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:23.202 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.203 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.205 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.206 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.208 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.209 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.210 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:23.211 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:23.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f389dcb1640 1 -- 172.21.15.89:0/950607203 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f387803be30 msgr2=0x7f387803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f389dcb1640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f387803be30 0x7f387803e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f38980a9a20 tx=0x7f38900a7070 comp rx=0 tx=0).stop 2024-08-19T10:38:23.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f389dcb1640 10 monclient: shutdown 2024-08-19T10:38:23.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f389dcb1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:23.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f389dcb1640 1 -- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38980a6860 msgr2=0x7f3898134830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f389dcb1640 1 --2- 172.21.15.89:0/950607203 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f38980a6860 0x7f3898134830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f388800ebe0 tx=0x7f388800c520 comp rx=0 tx=0).stop 2024-08-19T10:38:23.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f3897fff640 1 -- 172.21.15.89:0/950607203 reap_dead start 2024-08-19T10:38:23.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f3897fff640 1 -- 172.21.15.89:0/950607203 reap_dead start 2024-08-19T10:38:23.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f389dcb1640 1 -- 172.21.15.89:0/950607203 shutdown_connections 2024-08-19T10:38:23.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f389dcb1640 1 -- 172.21.15.89:0/950607203 >> 172.21.15.89:0/950607203 conn(0x7f3898074b40 msgr2=0x7f38980ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:23.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f389dcb1640 1 -- 172.21.15.89:0/950607203 shutdown_connections 2024-08-19T10:38:23.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.166+0000 7f389dcb1640 1 -- 172.21.15.89:0/950607203 wait complete. 2024-08-19T10:38:23.214 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:38:23.214 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:38:23.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:38:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient: get_monmap_and_config 2024-08-19T10:38:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient: build_initial_monmap 2024-08-19T10:38:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient: monmap: 2024-08-19T10:38:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.320 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:23.321785+0000 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:23.321785+0000 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:23.321 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:23.322 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:23.322 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 1 Processor -- start 2024-08-19T10:38:23.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 1 -- start start 2024-08-19T10:38:23.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient: init 2024-08-19T10:38:23.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:23.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient: _add_conns ranks=[5,6,2,4,7,8,1,3,9,0,10] 2024-08-19T10:38:23.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f92240a6860 0x7f92240a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient(hunting): picked mon.noname-h con 0x7f92240a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:23.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 0x7f92240a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient(hunting): picked mon.noname-d con 0x7f92240a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:23.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9224139620 0x7f9224139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient(hunting): picked mon.noname-b con 0x7f9224139620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:23.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f92240a8190 con 0x7f9224139620 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f92240a33d0 con 0x7f92240a6860 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f92240a2ec0 con 0x7f92240a7180 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f922a771640 10 monclient(hunting): authenticate will time out at 953.138076s 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9224139620 0x7f9224139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f92237fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 0x7f92240a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9224139620 0x7f9224139a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37868/0 (socket says 172.21.15.89:37868) 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 1 -- 172.21.15.89:0/1305736767 learned_addr learned my addr 172.21.15.89:0/1305736767 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:23.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 10 monclient(hunting): get_auth_request con 0x7f9224139620 auth_method 0 2024-08-19T10:38:23.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f92237fe640 10 monclient(hunting): get_auth_request con 0x7f92240a7180 auth_method 0 2024-08-19T10:38:23.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f92237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f92237fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f92237fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f92237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f92237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f92237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 10 monclient(hunting): handle_auth_done global_id 4340 payload 274 2024-08-19T10:38:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9223fff640 1 --2- 172.21.15.89:0/1305736767 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f92240a6860 0x7f92240a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 1 -- 172.21.15.89:0/1305736767 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 msgr2=0x7f92240a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 1 --2- 172.21.15.89:0/1305736767 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 0x7f92240a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 1 -- 172.21.15.89:0/1305736767 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f92240a6860 msgr2=0x7f92240a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 1 --2- 172.21.15.89:0/1305736767 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f92240a6860 0x7f92240a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 10 monclient: _finish_hunting 0 2024-08-19T10:38:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 1 monclient: found mon.noname-b 2024-08-19T10:38:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.318+0000 7f9228d0e640 1 -- 172.21.15.89:0/1305736767 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92240a4cd0 con 0x7f9224139620 2024-08-19T10:38:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9228d0e640 1 -- 172.21.15.89:0/1305736767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9224139620 msgr2=0x7f9224139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9228d0e640 1 --2- 172.21.15.89:0/1305736767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9224139620 0x7f9224139a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f921c0098a0 tx=0x7f921c01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd37a5da2c10f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 1 -- 172.21.15.89:0/1305736767 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f921c0b0070 con 0x7f9224139620 2024-08-19T10:38:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 10 monclient: dump: 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.336 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.337 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 10 monclient: _finish_auth 0 2024-08-19T10:38:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:38:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 1 -- 172.21.15.89:0/1305736767 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c0aa810 con 0x7f9224139620 2024-08-19T10:38:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 5 monclient: authenticate success, global_id 4340 2024-08-19T10:38:23.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 1 -- 172.21.15.89:0/1305736767 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f921c0a9300 con 0x7f9224139620 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 4 set_mon_vals no callback set 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f9222ffd640 10 monclient: dump: 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.338 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.339 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 10 monclient: shutdown 2024-08-19T10:38:23.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:23.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 1 -- 172.21.15.89:0/1305736767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9224139620 msgr2=0x7f9224139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 1 --2- 172.21.15.89:0/1305736767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9224139620 0x7f9224139a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f921c0098a0 tx=0x7f921c01c440 comp rx=0 tx=0).stop 2024-08-19T10:38:23.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 1 -- 172.21.15.89:0/1305736767 shutdown_connections 2024-08-19T10:38:23.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 1 --2- 172.21.15.89:0/1305736767 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9224139620 0x7f9224139a60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 1 --2- 172.21.15.89:0/1305736767 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 0x7f92240a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 1 --2- 172.21.15.89:0/1305736767 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f92240a6860 0x7f92240a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 1 -- 172.21.15.89:0/1305736767 >> 172.21.15.89:0/1305736767 conn(0x7f9224074b40 msgr2=0x7f92240abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:23.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 1 -- 172.21.15.89:0/1305736767 shutdown_connections 2024-08-19T10:38:23.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.322+0000 7f922a771640 1 -- 172.21.15.89:0/1305736767 wait complete. 2024-08-19T10:38:23.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient: build_initial_monmap 2024-08-19T10:38:23.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient: monmap: 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:23.342 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 1 Processor -- start 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 1 -- start start 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient: init 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient: _add_conns ranks=[5,6,7,8,3,2,0,4,1] 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f92240a6860 0x7f9224134890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient(hunting): picked mon.noname-h con 0x7f92240a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:23.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 0x7f9224134dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient(hunting): picked mon.noname-d con 0x7f92240a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:23.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9224139620 0x7f922409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient(hunting): picked mon.noname-i con 0x7f9224139620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:23.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f9224143210 con 0x7f92240a6860 2024-08-19T10:38:23.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9224143090 con 0x7f92240a7180 2024-08-19T10:38:23.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9224143390 con 0x7f9224139620 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient(hunting): authenticate will time out at 953.144999s 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 0x7f9224134dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 0x7f9224134dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36592/0 (socket says 172.21.15.89:36592) 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 1 -- 172.21.15.89:0/906311698 learned_addr learned my addr 172.21.15.89:0/906311698 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 10 monclient(hunting): get_auth_request con 0x7f92240a7180 auth_method 0 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 10 monclient(hunting): handle_auth_done global_id 4335 payload 824 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 1 -- 172.21.15.89:0/906311698 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9224139620 msgr2=0x7f922409c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9224139620 0x7f922409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 1 -- 172.21.15.89:0/906311698 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f92240a6860 msgr2=0x7f9224134890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f92240a6860 0x7f9224134890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 10 monclient: _finish_hunting 0 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 1 monclient: found mon.noname-d 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 1 -- 172.21.15.89:0/906311698 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92240a9210 con 0x7f92240a7180 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 1 -- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 msgr2=0x7f9224134dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92237fe640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 0x7f9224134dd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f921800e930 tx=0x7f921800edf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9218017070 con 0x7f92240a7180 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 10 monclient: dump: 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.347 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 10 monclient: _finish_auth 0 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 5 monclient: authenticate success, global_id 4335 2024-08-19T10:38:23.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9218013470 con 0x7f92240a7180 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9218013a90 con 0x7f92240a7180 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient: _renew_subs 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 1 -- 172.21.15.89:0/906311698 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922409fa90 con 0x7f92240a7180 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f92227fc640 4 set_mon_vals no callback set 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f9220ff9640 10 monclient: dump: 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.349 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.350 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.350 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient: _renew_subs 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.326+0000 7f922a771640 1 -- 172.21.15.89:0/906311698 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f92240a2820 con 0x7f92240a7180 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f922a771640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f922a771640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409d6e0 0x7f922402d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f922a771640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f922a771640 1 -- 172.21.15.89:0/906311698 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9224140560 con 0x7f922409d6e0 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f921801a070 con 0x7f92240a7180 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409d6e0 0x7f922402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 10 monclient: get_auth_request con 0x7f922409d6e0 auth_method 0 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f921003bd30 0x7f921003e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f921800f040 con 0x7f92240a7180 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 10 monclient(hunting): handle_auth_done global_id 4344 payload 824 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 1 -- 172.21.15.89:0/906311698 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92240a7560 con 0x7f922409d6e0 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9228d0e640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f921003bd30 0x7f921003e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9228d0e640 10 monclient: get_auth_request con 0x7f921003bd30 auth_method 0 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 1 -- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409d6e0 msgr2=0x7f922402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9223fff640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409d6e0 0x7f922402d1d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f921400ccc0 tx=0x7f921400a660 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f921400ea70 con 0x7f922409d6e0 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 10 monclient: dump: 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.353 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f921400e600 con 0x7f922409d6e0 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409d6e0 msgr2=0x7f922402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409d6e0 0x7f922402d1d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f921400ccc0 tx=0x7f921400a660 comp rx=0 tx=0).stop 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409d6e0 msgr2=0x7f922402d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.330+0000 7f9220ff9640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409d6e0 0x7f922402d1d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.334+0000 7f9228d0e640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f921003bd30 0x7f921003e1f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f921c009870 tx=0x7f921c004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f922a771640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f922a771640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409dac0 0x7f922402d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f922a771640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f922a771640 1 -- 172.21.15.89:0/906311698 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9224064590 con 0x7f922409dac0 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f9223fff640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409dac0 0x7f922402d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f9223fff640 10 monclient: get_auth_request con 0x7f922409dac0 auth_method 0 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f9223fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f9223fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f9223fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f9223fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f9223fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f9223fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f9223fff640 10 monclient(hunting): handle_auth_done global_id 4353 payload 824 2024-08-19T10:38:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.338+0000 7f9223fff640 1 -- 172.21.15.89:0/906311698 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f922406cab0 con 0x7f922409dac0 2024-08-19T10:38:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9223fff640 1 -- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409dac0 msgr2=0x7f922402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9223fff640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409dac0 0x7f922402d710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f921400ccc0 tx=0x7f9214004a50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9214016420 con 0x7f922409dac0 2024-08-19T10:38:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9220ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9220ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9220ff9640 10 monclient: dump: 2024-08-19T10:38:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.358 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7f921401da90 con 0x7f922409dac0 2024-08-19T10:38:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9220ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9220ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409dac0 msgr2=0x7f922402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9220ff9640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409dac0 0x7f922402d710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f921400ccc0 tx=0x7f9214004a50 comp rx=0 tx=0).stop 2024-08-19T10:38:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9220ff9640 1 -- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409dac0 msgr2=0x7f922402d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:23.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9220ff9640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f922409dac0 0x7f922402d710 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f921400ccc0 tx=0x7f9214004a50 comp rx=0 tx=0).stop 2024-08-19T10:38:23.360 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:23.360 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 4, 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 33159, 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:23.361 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 32, 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.362 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.363 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.364 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.365 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:23.366 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.367 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.368 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.369 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.370 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.371 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.372 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.373 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:23.377 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:23.378 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.379 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.381 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.384 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.385 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:23.386 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:23.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f922a771640 1 -- 172.21.15.89:0/906311698 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f921003bd30 msgr2=0x7f921003e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f922a771640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f921003bd30 0x7f921003e1f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f921c009870 tx=0x7f921c004740 comp rx=0 tx=0).stop 2024-08-19T10:38:23.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f922a771640 10 monclient: shutdown 2024-08-19T10:38:23.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f922a771640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:23.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f922a771640 1 -- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 msgr2=0x7f9224134dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f922a771640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 0x7f9224134dd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f921800e930 tx=0x7f921800edf0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f9228d0e640 1 -- 172.21.15.89:0/906311698 reap_dead start 2024-08-19T10:38:23.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f922a771640 1 -- 172.21.15.89:0/906311698 shutdown_connections 2024-08-19T10:38:23.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f922a771640 1 --2- 172.21.15.89:0/906311698 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92240a7180 0x7f9224134dd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f922a771640 1 -- 172.21.15.89:0/906311698 >> 172.21.15.89:0/906311698 conn(0x7f9224074b40 msgr2=0x7f92241388a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:23.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f922a771640 1 -- 172.21.15.89:0/906311698 shutdown_connections 2024-08-19T10:38:23.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.342+0000 7f922a771640 1 -- 172.21.15.89:0/906311698 wait complete. 2024-08-19T10:38:23.389 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:38:23.389 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:38:23.390 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:38:23.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient: get_monmap_and_config 2024-08-19T10:38:23.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient: build_initial_monmap 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient: monmap: 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:23.506082+0000 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:23.506082+0000 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:23.505 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:23.506 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:23.506 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:23.506 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:23.506 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:23.506 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 1 Processor -- start 2024-08-19T10:38:23.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 1 -- start start 2024-08-19T10:38:23.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient: init 2024-08-19T10:38:23.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:23.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient: _add_conns ranks=[2,3,4,1,9,0,6,8,10,5,7] 2024-08-19T10:38:23.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e440a6860 0x7f4e440a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient(hunting): picked mon.noname-b con 0x7f4e440a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:23.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e440a7180 0x7f4e440a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient(hunting): picked mon.noname-g con 0x7f4e440a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:23.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e44139620 0x7f4e44139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient(hunting): picked mon.noname-c con 0x7f4e44139620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4e440a8190 con 0x7f4e440a6860 2024-08-19T10:38:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4e440a33d0 con 0x7f4e440a7180 2024-08-19T10:38:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f4e440a2ec0 con 0x7f4e44139620 2024-08-19T10:38:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4c57e640 10 monclient(hunting): authenticate will time out at 953.322640s 2024-08-19T10:38:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4ab1b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e44139620 0x7f4e44139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e440a6860 0x7f4e440a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4ab1b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e44139620 0x7f4e44139a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33866/0 (socket says 172.21.15.89:33866) 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4ab1b640 1 -- 172.21.15.89:0/3244887535 learned_addr learned my addr 172.21.15.89:0/3244887535 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4ab1b640 10 monclient(hunting): get_auth_request con 0x7f4e44139620 auth_method 0 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4ab1b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4ab1b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4ab1b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 10 monclient(hunting): get_auth_request con 0x7f4e440a6860 auth_method 0 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4ab1b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4ab1b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4ab1b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e49b19640 1 --2- 172.21.15.89:0/3244887535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e440a7180 0x7f4e440a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e49b19640 10 monclient(hunting): get_auth_request con 0x7f4e440a7180 auth_method 0 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e49b19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e49b19640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e49b19640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 10 monclient(hunting): handle_auth_done global_id 4349 payload 274 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 1 -- 172.21.15.89:0/3244887535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e44139620 msgr2=0x7f4e44139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 1 --2- 172.21.15.89:0/3244887535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e44139620 0x7f4e44139a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 1 -- 172.21.15.89:0/3244887535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e440a7180 msgr2=0x7f4e440a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:23.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 1 --2- 172.21.15.89:0/3244887535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e440a7180 0x7f4e440a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 10 monclient: _finish_hunting 0 2024-08-19T10:38:23.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 1 monclient: found mon.noname-b 2024-08-19T10:38:23.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:23.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:23.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4a31a640 1 -- 172.21.15.89:0/3244887535 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e440a4cd0 con 0x7f4e440a6860 2024-08-19T10:38:23.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.502+0000 7f4e4ab1b640 1 --2- 172.21.15.89:0/3244887535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e44139620 0x7f4e44139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:23.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e4a31a640 1 -- 172.21.15.89:0/3244887535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e440a6860 msgr2=0x7f4e440a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e4a31a640 1 --2- 172.21.15.89:0/3244887535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e440a6860 0x7f4e440a6c40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e400097f0 tx=0x7f4e4001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdb96a8d39230bcc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 1 -- 172.21.15.89:0/3244887535 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e400b0070 con 0x7f4e440a6860 2024-08-19T10:38:23.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 10 monclient: dump: 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.518 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 10 monclient: _finish_auth 0 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 10 monclient: _check_auth_tickets 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 1 -- 172.21.15.89:0/3244887535 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e400aa810 con 0x7f4e440a6860 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 1 -- 172.21.15.89:0/3244887535 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e400aae30 con 0x7f4e440a6860 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:23.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e49318640 10 monclient: dump: 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.520 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e4c57e640 5 monclient: authenticate success, global_id 4349 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e4c57e640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.506+0000 7f4e4c57e640 4 set_mon_vals no callback set 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient: shutdown 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 -- 172.21.15.89:0/3244887535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e440a6860 msgr2=0x7f4e440a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 --2- 172.21.15.89:0/3244887535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e440a6860 0x7f4e440a6c40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e400097f0 tx=0x7f4e4001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 -- 172.21.15.89:0/3244887535 shutdown_connections 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 --2- 172.21.15.89:0/3244887535 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4e44139620 0x7f4e44139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 --2- 172.21.15.89:0/3244887535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e440a7180 0x7f4e440a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 --2- 172.21.15.89:0/3244887535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e440a6860 0x7f4e440a6c40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 -- 172.21.15.89:0/3244887535 >> 172.21.15.89:0/3244887535 conn(0x7f4e44074b40 msgr2=0x7f4e440abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:23.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 -- 172.21.15.89:0/3244887535 shutdown_connections 2024-08-19T10:38:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 -- 172.21.15.89:0/3244887535 wait complete. 2024-08-19T10:38:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient: build_initial_monmap 2024-08-19T10:38:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient: monmap: 2024-08-19T10:38:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:23.522 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 Processor -- start 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 -- start start 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient: init 2024-08-19T10:38:23.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient: _add_conns ranks=[0,1,3,2,4,7,8,6,5] 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e440a6860 0x7f4e441d1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient(hunting): picked mon.noname-a con 0x7f4e440a6860 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e440a7180 0x7f4e440a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient(hunting): picked mon.noname-f con 0x7f4e440a7180 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e44139620 0x7f4e440a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient(hunting): picked mon.noname-g con 0x7f4e44139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4e44143210 con 0x7f4e440a6860 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e44143090 con 0x7f4e440a7180 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4e44143390 con 0x7f4e44139620 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:23.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4c57e640 10 monclient(hunting): authenticate will time out at 953.329895s 2024-08-19T10:38:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e440a6860 0x7f4e441d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e440a6860 0x7f4e441d1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33098/0 (socket says 172.21.15.89:33098) 2024-08-19T10:38:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 1 -- 172.21.15.89:0/698962077 learned_addr learned my addr 172.21.15.89:0/698962077 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 10 monclient(hunting): get_auth_request con 0x7f4e440a6860 auth_method 0 2024-08-19T10:38:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4ab1b640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e44139620 0x7f4e440a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 10 monclient(hunting): handle_auth_done global_id 4509 payload 824 2024-08-19T10:38:23.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e49b19640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e440a7180 0x7f4e440a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 1 -- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e44139620 msgr2=0x7f4e440a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e44139620 0x7f4e440a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 1 -- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e440a7180 msgr2=0x7f4e440a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e440a7180 0x7f4e440a1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 10 monclient: _finish_hunting 0 2024-08-19T10:38:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 1 monclient: found mon.noname-a 2024-08-19T10:38:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4a31a640 1 -- 172.21.15.89:0/698962077 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4409e410 con 0x7f4e440a6860 2024-08-19T10:38:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4ab1b640 10 monclient: get_auth_request con 0x7f4e44139620 auth_method 0 2024-08-19T10:38:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.510+0000 7f4e4ab1b640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e44139620 0x7f4e440a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:23.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4a31a640 1 -- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e440a6860 msgr2=0x7f4e441d1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4a31a640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e440a6860 0x7f4e441d1750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e4001c1d0 tx=0x7f4e40019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e400b0040 con 0x7f4e440a6860 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 10 monclient: dump: 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.528 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 10 monclient: _finish_auth 0 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 10 monclient: _check_auth_tickets 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:23.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e400b9070 con 0x7f4e440a6860 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4c57e640 5 monclient: authenticate success, global_id 4509 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4c57e640 10 monclient: _renew_subs 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4c57e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4c57e640 1 -- 172.21.15.89:0/698962077 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4409ef70 con 0x7f4e440a6860 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e400aae60 con 0x7f4e440a6860 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 10 monclient: dump: 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.530 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e48b17640 4 set_mon_vals no callback set 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4c57e640 10 monclient: _renew_subs 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4c57e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:23.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4c57e640 1 -- 172.21.15.89:0/698962077 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e440a01b0 con 0x7f4e440a6860 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4c57e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4c57e640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409c9e0 0x7f4e4409cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4c57e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4c57e640 1 -- 172.21.15.89:0/698962077 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4e440a7560 con 0x7f4e4409c9e0 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e49b19640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409c9e0 0x7f4e4409cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e49b19640 10 monclient: get_auth_request con 0x7f4e4409c9e0 auth_method 0 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e49b19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e49b19640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e49b19640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4e400b3070 con 0x7f4e440a6860 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e49b19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e49b19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e49b19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e49b19640 10 monclient(hunting): handle_auth_done global_id 4355 payload 824 2024-08-19T10:38:23.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e2803be00 0x7f4e2803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e49b19640 1 -- 172.21.15.89:0/698962077 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e44064590 con 0x7f4e4409c9e0 2024-08-19T10:38:23.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f4e400e8370 con 0x7f4e440a6860 2024-08-19T10:38:23.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4ab1b640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e2803be00 0x7f4e2803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.514+0000 7f4e4ab1b640 10 monclient: get_auth_request con 0x7f4e2803be00 auth_method 0 2024-08-19T10:38:23.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e49b19640 1 -- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409c9e0 msgr2=0x7f4e4409cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e49b19640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409c9e0 0x7f4e4409cdc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e4409ea80 tx=0x7f4e3400e990 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e34017070 con 0x7f4e4409c9e0 2024-08-19T10:38:23.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e3affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e3affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:23.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e3affd640 10 monclient: dump: 2024-08-19T10:38:23.538 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.538 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.538 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.538 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.538 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.539 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4e34013700 con 0x7f4e4409c9e0 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e3affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e3affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409c9e0 msgr2=0x7f4e4409cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e3affd640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409c9e0 0x7f4e4409cdc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e4409ea80 tx=0x7f4e3400e990 comp rx=0 tx=0).stop 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409c9e0 msgr2=0x7f4e4409cdc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e3affd640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409c9e0 0x7f4e4409cdc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.518+0000 7f4e4ab1b640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e2803be00 0x7f4e2803e2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009440 tx=0x7f4e3c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e4c57e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e4c57e640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409d300 0x7f4e4413aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e4c57e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e4c57e640 1 -- 172.21.15.89:0/698962077 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4e440a0620 con 0x7f4e4409d300 2024-08-19T10:38:23.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e49b19640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409d300 0x7f4e4413aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e49b19640 10 monclient: get_auth_request con 0x7f4e4409d300 auth_method 0 2024-08-19T10:38:23.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e49b19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e49b19640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e49b19640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e49b19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e49b19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e49b19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e49b19640 10 monclient(hunting): handle_auth_done global_id 4364 payload 824 2024-08-19T10:38:23.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.522+0000 7f4e49b19640 1 -- 172.21.15.89:0/698962077 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4e44139400 con 0x7f4e4409d300 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e49b19640 1 -- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409d300 msgr2=0x7f4e4413aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e49b19640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409d300 0x7f4e4413aa20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4e34019000 tx=0x7f4e34004740 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e3400f040 con 0x7f4e4409d300 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e3affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e3affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e3affd640 10 monclient: dump: 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.542 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7f4e34004020 con 0x7f4e4409d300 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e3affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e3affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409d300 msgr2=0x7f4e4413aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e3affd640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409d300 0x7f4e4413aa20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4e34019000 tx=0x7f4e34004740 comp rx=0 tx=0).stop 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e3affd640 1 -- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409d300 msgr2=0x7f4e4413aa20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e3affd640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e4409d300 0x7f4e4413aa20 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4e34019000 tx=0x7f4e34004740 comp rx=0 tx=0).stop 2024-08-19T10:38:23.544 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 4, 2024-08-19T10:38:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 33319, 2024-08-19T10:38:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:38:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:23.544 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 32, 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.545 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.546 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.547 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:23.548 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.549 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.550 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.551 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.552 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.553 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.554 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.556 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.557 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.559 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.560 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.562 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:23.564 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.566 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:23.567 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:23.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4c57e640 1 -- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e2803be00 msgr2=0x7f4e2803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4c57e640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e2803be00 0x7f4e2803e2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009440 tx=0x7f4e3c00b040 comp rx=0 tx=0).stop 2024-08-19T10:38:23.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4c57e640 10 monclient: shutdown 2024-08-19T10:38:23.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4c57e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:23.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4c57e640 1 -- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e440a6860 msgr2=0x7f4e441d1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4c57e640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e440a6860 0x7f4e441d1750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e4001c1d0 tx=0x7f4e40019140 comp rx=0 tx=0).stop 2024-08-19T10:38:23.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4ab1b640 1 -- 172.21.15.89:0/698962077 reap_dead start 2024-08-19T10:38:23.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4c57e640 1 -- 172.21.15.89:0/698962077 shutdown_connections 2024-08-19T10:38:23.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4c57e640 1 --2- 172.21.15.89:0/698962077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e440a6860 0x7f4e441d1750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4c57e640 1 -- 172.21.15.89:0/698962077 >> 172.21.15.89:0/698962077 conn(0x7f4e44074b40 msgr2=0x7f4e44138910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:23.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4c57e640 1 -- 172.21.15.89:0/698962077 shutdown_connections 2024-08-19T10:38:23.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.526+0000 7f4e4c57e640 1 -- 172.21.15.89:0/698962077 wait complete. 2024-08-19T10:38:23.570 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:38:23.570 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:38:23.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:38:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient: get_monmap_and_config 2024-08-19T10:38:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient: build_initial_monmap 2024-08-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient: monmap: 2024-08-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:23.687194+0000 2024-08-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:23.687194+0000 2024-08-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:23.687 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:23.687 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:23.687 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:23.687 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:23.687 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:23.687 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:23.687 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:23.687 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 1 Processor -- start 2024-08-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 1 -- start start 2024-08-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient: init 2024-08-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient: _add_conns ranks=[0,5,7,1,10,6,3,4,2,8,9] 2024-08-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa8240a5270 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa8240a6f00 0x7fa82413ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient(hunting): picked mon.noname-h con 0x7fa8240a6f00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:23.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8240a72e0 0x7fa824140f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient(hunting): picked mon.noname-i con 0x7fa8240a72e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:23.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8240a8710 con 0x7fa8240a5270 2024-08-19T10:38:23.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa8240a5a40 con 0x7fa8240a6f00 2024-08-19T10:38:23.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa8240a81d0 con 0x7fa8240a72e0 2024-08-19T10:38:23.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa8291e8640 10 monclient(hunting): authenticate will time out at 953.503716s 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa822d9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa8240a5270 0x7fa8240a67f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44648/0 (socket says 172.21.15.89:44648) 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.682+0000 7fa822d9d640 1 -- 172.21.15.89:0/1551269913 learned_addr learned my addr 172.21.15.89:0/1551269913 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82259c640 1 --2- 172.21.15.89:0/1551269913 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa8240a6f00 0x7fa82413ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82259c640 10 monclient(hunting): get_auth_request con 0x7fa8240a6f00 auth_method 0 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 1 --2- 172.21.15.89:0/1551269913 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8240a72e0 0x7fa824140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 10 monclient(hunting): get_auth_request con 0x7fa8240a72e0 auth_method 0 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 10 monclient(hunting): handle_auth_done global_id 4327 payload 274 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 1 -- 172.21.15.89:0/1551269913 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa8240a6f00 msgr2=0x7fa82413ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 1 --2- 172.21.15.89:0/1551269913 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa8240a6f00 0x7fa82413ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 1 -- 172.21.15.89:0/1551269913 >> v1:172.21.15.89:6789/0 conn(0x7fa8240a5270 legacy=0x7fa8240a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 10 monclient: _finish_hunting 0 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 1 monclient: found mon.noname-i 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:38:23.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82359e640 1 -- 172.21.15.89:0/1551269913 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8240a4790 con 0x7fa8240a72e0 2024-08-19T10:38:23.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.686+0000 7fa82259c640 1 --2- 172.21.15.89:0/1551269913 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa8240a6f00 0x7fa82413ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:23.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa82359e640 1 -- 172.21.15.89:0/1551269913 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8240a72e0 msgr2=0x7fa824140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa82359e640 1 --2- 172.21.15.89:0/1551269913 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8240a72e0 0x7fa824140f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009dc0 tx=0x7fa81c01c640 comp rx=0 tx=0).ready entity=mon.7 client_cookie=dd2dbdca575aef66 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 1 -- 172.21.15.89:0/1551269913 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa81c0ac070 con 0x7fa8240a72e0 2024-08-19T10:38:23.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 10 monclient: dump: 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.703 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 10 monclient: _finish_auth 0 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 1 -- 172.21.15.89:0/1551269913 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c0a68a0 con 0x7fa8240a72e0 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 1 -- 172.21.15.89:0/1551269913 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa81c0a7410 con 0x7fa8240a72e0 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa821d9b640 10 monclient: dump: 2024-08-19T10:38:23.704 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.705 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 5 monclient: authenticate success, global_id 4327 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 4 set_mon_vals no callback set 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 10 monclient: shutdown 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 1 -- 172.21.15.89:0/1551269913 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8240a72e0 msgr2=0x7fa824140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 1 --2- 172.21.15.89:0/1551269913 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8240a72e0 0x7fa824140f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009dc0 tx=0x7fa81c01c640 comp rx=0 tx=0).stop 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 1 -- 172.21.15.89:0/1551269913 shutdown_connections 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 1 --2- 172.21.15.89:0/1551269913 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa8240a6f00 0x7fa82413ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 1 --2- 172.21.15.89:0/1551269913 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8240a72e0 0x7fa824140f50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 1 -- 172.21.15.89:0/1551269913 >> 172.21.15.89:0/1551269913 conn(0x7fa824069a80 msgr2=0x7fa8240abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 1 -- 172.21.15.89:0/1551269913 shutdown_connections 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 1 -- 172.21.15.89:0/1551269913 wait complete. 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 10 monclient: build_initial_monmap 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 10 monclient: monmap: 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:23.707 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 1 Processor -- start 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 1 -- start start 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.690+0000 7fa8291e8640 10 monclient: init 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient: _add_conns ranks=[8,7,1,0,2,4,3,6,5] 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8240a5270 0x7fa82413b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient(hunting): picked mon.noname-e con 0x7fa8240a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8240a6f00 0x7fa82413bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient(hunting): picked mon.noname-i con 0x7fa8240a6f00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:23.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8240a72e0 0x7fa82413c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient(hunting): picked mon.noname-f con 0x7fa8240a72e0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:23.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa824143e30 con 0x7fa8240a72e0 2024-08-19T10:38:23.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa824143cb0 con 0x7fa8240a6f00 2024-08-19T10:38:23.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa824143fb0 con 0x7fa8240a5270 2024-08-19T10:38:23.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:23.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient(hunting): authenticate will time out at 953.512104s 2024-08-19T10:38:23.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8240a5270 0x7fa82413b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8240a5270 0x7fa82413b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:53240/0 (socket says 172.21.15.89:53240) 2024-08-19T10:38:23.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 1 -- 172.21.15.89:0/3582476986 learned_addr learned my addr 172.21.15.89:0/3582476986 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:23.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 10 monclient(hunting): get_auth_request con 0x7fa8240a5270 auth_method 0 2024-08-19T10:38:23.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 10 monclient(hunting): handle_auth_done global_id 4373 payload 824 2024-08-19T10:38:23.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa82359e640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8240a72e0 0x7fa82413c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 1 -- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8240a6f00 msgr2=0x7fa82413bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8240a6f00 0x7fa82413bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 1 -- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8240a72e0 msgr2=0x7fa82413c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8240a72e0 0x7fa82413c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:38:23.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 1 monclient: found mon.noname-e 2024-08-19T10:38:23.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:23.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:23.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 1 -- 172.21.15.89:0/3582476986 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82413efd0 con 0x7fa8240a5270 2024-08-19T10:38:23.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 1 -- 172.21.15.89:0/3582476986 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8240a5270 msgr2=0x7fa82413b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa822d9d640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8240a5270 0x7fa82413b5d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa81800b430 tx=0x7fa81800b8f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa818015020 con 0x7fa8240a5270 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 10 monclient: dump: 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.713 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa818011cb0 con 0x7fa8240a5270 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 5 monclient: authenticate success, global_id 4373 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient: _renew_subs 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:23.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 1 -- 172.21.15.89:0/3582476986 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82413f2b0 con 0x7fa8240a5270 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa818011e60 con 0x7fa8240a5270 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 10 monclient: dump: 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.715 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa82159a640 4 set_mon_vals no callback set 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient: _renew_subs 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 1 -- 172.21.15.89:0/3582476986 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8240a9380 con 0x7fa8240a5270 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:23.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136580 0x7fa824136960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa8291e8640 1 -- 172.21.15.89:0/3582476986 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa82413eb60 con 0x7fa824136580 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa818018020 con 0x7fa8240a5270 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa82259c640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136580 0x7fa824136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa82259c640 10 monclient: get_auth_request con 0x7fa824136580 auth_method 0 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa82259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa82259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa80b7fe640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa81003be00 0x7fa81003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.694+0000 7fa82259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fa81804e9e0 con 0x7fa8240a5270 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa82259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa82259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa82259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa82359e640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa81003be00 0x7fa81003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa82359e640 10 monclient: get_auth_request con 0x7fa81003be00 auth_method 0 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa82259c640 10 monclient(hunting): handle_auth_done global_id 4384 payload 824 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa82259c640 1 -- 172.21.15.89:0/3582476986 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8240a67f0 con 0x7fa824136580 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa82259c640 1 -- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136580 msgr2=0x7fa824136960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa82259c640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136580 0x7fa824136960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa82413d5c0 tx=0x7fa81400eb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa82359e640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa81003be00 0x7fa81003e2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0ab000 tx=0x7fa81c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa814017070 con 0x7fa824136580 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa80b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa80b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.698+0000 7fa80b7fe640 10 monclient: dump: 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.718 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.719 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.702+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa81400ee70 con 0x7fa824136580 2024-08-19T10:38:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.702+0000 7fa80b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.702+0000 7fa80b7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.702+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136580 msgr2=0x7fa824136960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.702+0000 7fa80b7fe640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136580 0x7fa824136960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa82413d5c0 tx=0x7fa81400eb40 comp rx=0 tx=0).stop 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.702+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136580 msgr2=0x7fa824136960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.702+0000 7fa80b7fe640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136580 0x7fa824136960 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa82413d5c0 tx=0x7fa81400eb40 comp rx=0 tx=0).stop 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa8291e8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa8291e8640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136ea0 0x7fa8241377a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa8291e8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa8291e8640 1 -- 172.21.15.89:0/3582476986 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa824064590 con 0x7fa824136ea0 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa82259c640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136ea0 0x7fa8241377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa82259c640 10 monclient: get_auth_request con 0x7fa824136ea0 auth_method 0 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa82259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa82259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa82259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa82259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa82259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa82259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa82259c640 10 monclient(hunting): handle_auth_done global_id 4393 payload 824 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.706+0000 7fa82259c640 1 -- 172.21.15.89:0/3582476986 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa82413dab0 con 0x7fa824136ea0 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa82259c640 1 -- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136ea0 msgr2=0x7fa8241377a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa82259c640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136ea0 0x7fa8241377a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa82413d5c0 tx=0x7fa8140046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa814017070 con 0x7fa824136ea0 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa80b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa80b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa80b7fe640 10 monclient: dump: 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.723 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9433 (secure 0 0 0) 0x7fa814006600 con 0x7fa824136ea0 2024-08-19T10:38:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa80b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa80b7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136ea0 msgr2=0x7fa8241377a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa80b7fe640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136ea0 0x7fa8241377a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa82413d5c0 tx=0x7fa8140046d0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa80b7fe640 1 -- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136ea0 msgr2=0x7fa8241377a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa80b7fe640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa824136ea0 0x7fa8241377a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.725 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:23.725 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 4, 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 33503, 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 32, 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.727 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.728 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:23.729 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:23.730 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.731 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:23.732 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.733 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:23.735 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:23.736 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.738 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.739 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.742 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.743 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.744 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.746 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.748 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:38:23.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.751 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:23.751 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:23.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa8291e8640 1 -- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa81003be00 msgr2=0x7fa81003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa8291e8640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa81003be00 0x7fa81003e2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0ab000 tx=0x7fa81c0a7000 comp rx=0 tx=0).stop 2024-08-19T10:38:23.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa8291e8640 10 monclient: shutdown 2024-08-19T10:38:23.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa8291e8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:23.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa8291e8640 1 -- 172.21.15.89:0/3582476986 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8240a5270 msgr2=0x7fa82413b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa8291e8640 1 --2- 172.21.15.89:0/3582476986 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8240a5270 0x7fa82413b5d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa81800b430 tx=0x7fa81800b8f0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa82359e640 1 -- 172.21.15.89:0/3582476986 reap_dead start 2024-08-19T10:38:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa82359e640 1 -- 172.21.15.89:0/3582476986 reap_dead start 2024-08-19T10:38:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa8291e8640 1 -- 172.21.15.89:0/3582476986 shutdown_connections 2024-08-19T10:38:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa8291e8640 1 -- 172.21.15.89:0/3582476986 >> 172.21.15.89:0/3582476986 conn(0x7fa824069a80 msgr2=0x7fa82409c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa8291e8640 1 -- 172.21.15.89:0/3582476986 shutdown_connections 2024-08-19T10:38:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.710+0000 7fa8291e8640 1 -- 172.21.15.89:0/3582476986 wait complete. 2024-08-19T10:38:23.753 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:38:23.754 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:38:23.754 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:38:23.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient: get_monmap_and_config 2024-08-19T10:38:23.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient: build_initial_monmap 2024-08-19T10:38:23.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient: monmap: 2024-08-19T10:38:23.867 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:23.869079+0000 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:23.869079+0000 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:23.868 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:23.869 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:23.869 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:23.869 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:23.869 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:23.869 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 1 Processor -- start 2024-08-19T10:38:23.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 1 -- start start 2024-08-19T10:38:23.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient: init 2024-08-19T10:38:23.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:23.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient: _add_conns ranks=[2,0,1,3,5,9,10,7,4,6,8] 2024-08-19T10:38:23.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0bb00a3bc0 0x7f0bb00a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient(hunting): picked mon.noname-b con 0x7f0bb00a3bc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0bb00a44e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0bb013bca0 addr v1:172.21.15.190:6789/0 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bb00a14e0 con 0x7f0bb00a44e0 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bb0074140 con 0x7f0bb013bca0 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb009fc30 con 0x7f0bb00a3bc0 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb6c29640 10 monclient(hunting): authenticate will time out at 953.685678s 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0bb00a3bc0 0x7f0bb00a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0bb00a3bc0 0x7f0bb00a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37888/0 (socket says 172.21.15.89:37888) 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0baffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0bb00a44e0 0x7f0bb013a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44656/0 (socket says 172.21.15.89:44656) 2024-08-19T10:38:23.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 1 -- 172.21.15.89:0/2602804770 learned_addr learned my addr 172.21.15.89:0/2602804770 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 10 monclient(hunting): get_auth_request con 0x7f0bb00a3bc0 auth_method 0 2024-08-19T10:38:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_done global_id 4358 payload 274 2024-08-19T10:38:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 1 -- 172.21.15.89:0/2602804770 >> v1:172.21.15.190:6789/0 conn(0x7f0bb013bca0 legacy=0x7f0bb013e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 1 -- 172.21.15.89:0/2602804770 >> v1:172.21.15.89:6789/0 conn(0x7f0bb00a44e0 legacy=0x7f0bb013a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:23.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 10 monclient: _finish_hunting 0 2024-08-19T10:38:23.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 1 monclient: found mon.noname-b 2024-08-19T10:38:23.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:23.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:23.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.866+0000 7f0bb49c5640 1 -- 172.21.15.89:0/2602804770 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb00a0940 con 0x7f0bb00a3bc0 2024-08-19T10:38:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb49c5640 1 -- 172.21.15.89:0/2602804770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0bb00a3bc0 msgr2=0x7f0bb00a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb49c5640 1 --2- 172.21.15.89:0/2602804770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0bb00a3bc0 0x7f0bb00a3fa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009e80 tx=0x7f0ba801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c1fbddb1f54f4ec server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 1 -- 172.21.15.89:0/2602804770 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba80091b0 con 0x7f0bb00a3bc0 2024-08-19T10:38:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 10 monclient: dump: 2024-08-19T10:38:23.877 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.878 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 1 -- 172.21.15.89:0/2602804770 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba80a5b30 con 0x7f0bb00a3bc0 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 5 monclient: authenticate success, global_id 4358 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 1 -- 172.21.15.89:0/2602804770 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba80a5ce0 con 0x7f0bb00a3bc0 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 4 set_mon_vals no callback set 2024-08-19T10:38:23.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0baf7fe640 10 monclient: dump: 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.880 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.881 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 10 monclient: shutdown 2024-08-19T10:38:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 1 -- 172.21.15.89:0/2602804770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0bb00a3bc0 msgr2=0x7f0bb00a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 1 --2- 172.21.15.89:0/2602804770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0bb00a3bc0 0x7f0bb00a3fa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009e80 tx=0x7f0ba801c8e0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 1 -- 172.21.15.89:0/2602804770 shutdown_connections 2024-08-19T10:38:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 1 --2- 172.21.15.89:0/2602804770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0bb00a3bc0 0x7f0bb00a3fa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 1 -- 172.21.15.89:0/2602804770 >> 172.21.15.89:0/2602804770 conn(0x7f0bb006de40 msgr2=0x7f0bb006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 1 -- 172.21.15.89:0/2602804770 shutdown_connections 2024-08-19T10:38:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.870+0000 7f0bb6c29640 1 -- 172.21.15.89:0/2602804770 wait complete. 2024-08-19T10:38:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient: build_initial_monmap 2024-08-19T10:38:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient: monmap: 2024-08-19T10:38:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:23.882 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 1 Processor -- start 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 1 -- start start 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient: init 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient: _add_conns ranks=[7,1,0,6,2,4,8,3,5] 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0bb00a3bc0 0x7f0bb0123800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient(hunting): picked mon.noname-i con 0x7f0bb00a3bc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bb00a44e0 0x7f0bb0123d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient(hunting): picked mon.noname-f con 0x7f0bb00a44e0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb013bca0 0x7f0bb0124280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient(hunting): picked mon.noname-a con 0x7f0bb013bca0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb00a2a90 con 0x7f0bb013bca0 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb00a3210 con 0x7f0bb00a44e0 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0bb00a4900 con 0x7f0bb00a3bc0 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb6c29640 10 monclient(hunting): authenticate will time out at 953.693097s 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb013bca0 0x7f0bb0124280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb013bca0 0x7f0bb0124280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33112/0 (socket says 172.21.15.89:33112) 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 1 -- 172.21.15.89:0/623986018 learned_addr learned my addr 172.21.15.89:0/623986018 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 10 monclient(hunting): get_auth_request con 0x7f0bb013bca0 auth_method 0 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0baffff640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bb00a44e0 0x7f0bb0123d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 10 monclient(hunting): handle_auth_done global_id 4518 payload 824 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 1 -- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0bb00a3bc0 msgr2=0x7f0bb0123800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0bb00a3bc0 0x7f0bb0123800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 1 -- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bb00a44e0 msgr2=0x7f0bb0123d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bb00a44e0 0x7f0bb0123d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 10 monclient: _finish_hunting 0 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 1 monclient: found mon.noname-a 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 1 -- 172.21.15.89:0/623986018 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb013f0b0 con 0x7f0bb013bca0 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0baffff640 10 monclient: get_auth_request con 0x7f0bb00a44e0 auth_method 0 2024-08-19T10:38:23.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0baffff640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bb00a44e0 0x7f0bb0123d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 1 -- 172.21.15.89:0/623986018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb013bca0 msgr2=0x7f0bb0124280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.874+0000 7f0bb51c6640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb013bca0 0x7f0bb0124280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c3c0 tx=0x7f0ba000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba0011c10 con 0x7f0bb013bca0 2024-08-19T10:38:23.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 10 monclient: dump: 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.891 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.892 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba0017070 con 0x7f0bb013bca0 2024-08-19T10:38:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb6c29640 5 monclient: authenticate success, global_id 4518 2024-08-19T10:38:23.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb6c29640 10 monclient: _renew_subs 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb6c29640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb6c29640 1 -- 172.21.15.89:0/623986018 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb00a61f0 con 0x7f0bb013bca0 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba0011df0 con 0x7f0bb013bca0 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0baeffd640 4 set_mon_vals no callback set 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 10 monclient: dump: 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.893 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb6c29640 10 monclient: _renew_subs 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb6c29640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:23.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb6c29640 1 -- 172.21.15.89:0/623986018 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb0126350 con 0x7f0bb013bca0 2024-08-19T10:38:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb6c29640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb6c29640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139380 0x7f0bb0139760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb6c29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb6c29640 1 -- 172.21.15.89:0/623986018 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb0064590 con 0x7f0bb0139380 2024-08-19T10:38:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0ba00117a0 con 0x7f0bb013bca0 2024-08-19T10:38:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb49c5640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139380 0x7f0bb0139760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb49c5640 10 monclient: get_auth_request con 0x7f0bb0139380 auth_method 0 2024-08-19T10:38:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb49c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb49c5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb49c5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0b9403be60 0x7f0b9403e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f0ba004e6c0 con 0x7f0bb013bca0 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0baffff640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0b9403be60 0x7f0b9403e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0baffff640 10 monclient: get_auth_request con 0x7f0b9403be60 auth_method 0 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_done global_id 4359 payload 824 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.878+0000 7f0bb49c5640 1 -- 172.21.15.89:0/623986018 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bb00a3fa0 con 0x7f0bb0139380 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bb49c5640 1 -- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139380 msgr2=0x7f0bb0139760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bb49c5640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139380 0x7f0bb0139760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00a2a00 tx=0x7f0ba8009510 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0baffff640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0b9403be60 0x7f0b9403e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b98007fd0 tx=0x7f0b98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba80bb020 con 0x7f0bb0139380 2024-08-19T10:38:23.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bad7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bad7fa640 10 monclient: dump: 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.897 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.898 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0ba80a5ae0 con 0x7f0bb0139380 2024-08-19T10:38:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bad7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bad7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139380 msgr2=0x7f0bb0139760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bad7fa640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139380 0x7f0bb0139760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00a2a00 tx=0x7f0ba8009510 comp rx=0 tx=0).stop 2024-08-19T10:38:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139380 msgr2=0x7f0bb0139760 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:23.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.882+0000 7f0bad7fa640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139380 0x7f0bb0139760 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb6c29640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb6c29640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139ca0 0x7f0bb013d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb6c29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb6c29640 1 -- 172.21.15.89:0/623986018 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb006ed10 con 0x7f0bb0139ca0 2024-08-19T10:38:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb49c5640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139ca0 0x7f0bb013d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb49c5640 10 monclient: get_auth_request con 0x7f0bb0139ca0 auth_method 0 2024-08-19T10:38:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb49c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:23.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb49c5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb49c5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb49c5640 10 monclient(hunting): handle_auth_done global_id 4368 payload 824 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.890+0000 7f0bb49c5640 1 -- 172.21.15.89:0/623986018 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0bb0072e90 con 0x7f0bb0139ca0 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb49c5640 1 -- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139ca0 msgr2=0x7f0bb013d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb49c5640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139ca0 0x7f0bb013d2d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009e80 tx=0x7f0ba80a7000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba8004470 con 0x7f0bb0139ca0 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bad7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bad7fa640 10 monclient: dump: 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:23.904 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:23.905 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9433 (secure 0 0 0) 0x7f0ba8004000 con 0x7f0bb0139ca0 2024-08-19T10:38:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bad7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bad7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139ca0 msgr2=0x7f0bb013d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bad7fa640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139ca0 0x7f0bb013d2d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009e80 tx=0x7f0ba80a7000 comp rx=0 tx=0).stop 2024-08-19T10:38:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bad7fa640 1 -- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139ca0 msgr2=0x7f0bb013d2d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:23.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bad7fa640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bb0139ca0 0x7f0bb013d2d0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009e80 tx=0x7f0ba80a7000 comp rx=0 tx=0).stop 2024-08-19T10:38:23.906 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:23.906 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:23.906 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 4, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 33663, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 33, 2024-08-19T10:38:23.907 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.908 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.909 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:23.910 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:23.911 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.912 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:23.913 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:23.914 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.915 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.916 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.917 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.918 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:23.919 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.920 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.921 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:23.922 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.924 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.925 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.926 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.927 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:23.928 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:23.929 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:23.930 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:23.930 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:23.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb6c29640 1 -- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0b9403be60 msgr2=0x7f0b9403e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb6c29640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0b9403be60 0x7f0b9403e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b98007fd0 tx=0x7f0b98009040 comp rx=0 tx=0).stop 2024-08-19T10:38:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb6c29640 10 monclient: shutdown 2024-08-19T10:38:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb6c29640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb6c29640 1 -- 172.21.15.89:0/623986018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb013bca0 msgr2=0x7f0bb0124280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb6c29640 1 --2- 172.21.15.89:0/623986018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb013bca0 0x7f0bb0124280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c3c0 tx=0x7f0ba000c880 comp rx=0 tx=0).stop 2024-08-19T10:38:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb51c6640 1 -- 172.21.15.89:0/623986018 reap_dead start 2024-08-19T10:38:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb51c6640 1 -- 172.21.15.89:0/623986018 reap_dead start 2024-08-19T10:38:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb6c29640 1 -- 172.21.15.89:0/623986018 shutdown_connections 2024-08-19T10:38:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb6c29640 1 -- 172.21.15.89:0/623986018 >> 172.21.15.89:0/623986018 conn(0x7f0bb006de40 msgr2=0x7f0bb006d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb6c29640 1 -- 172.21.15.89:0/623986018 shutdown_connections 2024-08-19T10:38:23.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:23.894+0000 7f0bb6c29640 1 -- 172.21.15.89:0/623986018 wait complete. 2024-08-19T10:38:23.933 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:38:23.933 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:38:23.933 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:38:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.038+0000 7f0208a33640 10 monclient: get_monmap_and_config 2024-08-19T10:38:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.038+0000 7f0208a33640 10 monclient: build_initial_monmap 2024-08-19T10:38:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.038+0000 7f0208a33640 10 monclient: monmap: 2024-08-19T10:38:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:24.043729+0000 2024-08-19T10:38:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:24.043729+0000 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:24.043 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 1 Processor -- start 2024-08-19T10:38:24.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 1 -- start start 2024-08-19T10:38:24.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 10 monclient: init 2024-08-19T10:38:24.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:24.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 10 monclient: _add_conns ranks=[0,7,4,5,9,1,3,8,2,10,6] 2024-08-19T10:38:24.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f02040a4d60 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:24.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f02040a5140 0x7f02040a5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 10 monclient(hunting): picked mon.noname-i con 0x7f02040a5140 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 0x7f0204140e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 10 monclient(hunting): picked mon.noname-c con 0x7f020413ea70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020406f2f0 con 0x7f02040a4d60 2024-08-19T10:38:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f020406e940 con 0x7f020413ea70 2024-08-19T10:38:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f020406daf0 con 0x7f02040a5140 2024-08-19T10:38:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0208a33640 10 monclient(hunting): authenticate will time out at 953.860872s 2024-08-19T10:38:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 0x7f0204140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 0x7f0204140e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33872/0 (socket says 172.21.15.89:33872) 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 1 -- 172.21.15.89:0/2378097423 learned_addr learned my addr 172.21.15.89:0/2378097423 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 10 monclient(hunting): get_auth_request con 0x7f020413ea70 auth_method 0 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 10 monclient(hunting): handle_auth_done global_id 4450 payload 274 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 1 -- 172.21.15.89:0/2378097423 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f02040a5140 msgr2=0x7f02040a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0201d9b640 1 --2- 172.21.15.89:0/2378097423 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f02040a5140 0x7f02040a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 1 --2- 172.21.15.89:0/2378097423 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f02040a5140 0x7f02040a5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 1 -- 172.21.15.89:0/2378097423 >> v1:172.21.15.89:6789/0 conn(0x7f02040a4d60 legacy=0x7f020406e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:38:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 1 monclient: found mon.noname-c 2024-08-19T10:38:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.042+0000 7f0202d9d640 1 -- 172.21.15.89:0/2378097423 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020406dcf0 con 0x7f020413ea70 2024-08-19T10:38:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0202d9d640 1 -- 172.21.15.89:0/2378097423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 msgr2=0x7f0204140e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0202d9d640 1 --2- 172.21.15.89:0/2378097423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 0x7f0204140e60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009eb0 tx=0x7f01fc01c6e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=bd79a34403b7cd4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 1 -- 172.21.15.89:0/2378097423 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01fc0ac070 con 0x7f020413ea70 2024-08-19T10:38:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 10 monclient: dump: 2024-08-19T10:38:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.054 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 10 monclient: _finish_auth 0 2024-08-19T10:38:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 10 monclient: _check_auth_tickets 2024-08-19T10:38:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 1 -- 172.21.15.89:0/2378097423 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01fc0091b0 con 0x7f020413ea70 2024-08-19T10:38:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 1 -- 172.21.15.89:0/2378097423 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01fc0a8d30 con 0x7f020413ea70 2024-08-19T10:38:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 5 monclient: authenticate success, global_id 4450 2024-08-19T10:38:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 4 set_mon_vals no callback set 2024-08-19T10:38:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f020159a640 10 monclient: dump: 2024-08-19T10:38:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 10 monclient: shutdown 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 1 -- 172.21.15.89:0/2378097423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 msgr2=0x7f0204140e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 1 --2- 172.21.15.89:0/2378097423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 0x7f0204140e60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009eb0 tx=0x7f01fc01c6e0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 1 -- 172.21.15.89:0/2378097423 shutdown_connections 2024-08-19T10:38:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 1 --2- 172.21.15.89:0/2378097423 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 0x7f0204140e60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 1 --2- 172.21.15.89:0/2378097423 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f02040a5140 0x7f02040a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 1 -- 172.21.15.89:0/2378097423 >> 172.21.15.89:0/2378097423 conn(0x7f02040aa240 msgr2=0x7f02040aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 1 -- 172.21.15.89:0/2378097423 shutdown_connections 2024-08-19T10:38:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 1 -- 172.21.15.89:0/2378097423 wait complete. 2024-08-19T10:38:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 10 monclient: build_initial_monmap 2024-08-19T10:38:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 10 monclient: monmap: 2024-08-19T10:38:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 1 Processor -- start 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 1 -- start start 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 10 monclient: init 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 10 monclient: _add_conns ranks=[5,2,4,7,1,0,6,3,8] 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02040a4d60 0x7f0204135f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.046+0000 7f0208a33640 10 monclient(hunting): picked mon.noname-h con 0x7f02040a4d60 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02040a5140 0x7f0204136490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient(hunting): picked mon.noname-b con 0x7f02040a5140 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 0x7f020413a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient(hunting): picked mon.noname-c con 0x7f020413ea70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f0204143d40 con 0x7f02040a5140 2024-08-19T10:38:24.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f0204143bc0 con 0x7f020413ea70 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0204143ec0 con 0x7f02040a4d60 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient(hunting): authenticate will time out at 953.868057s 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0202d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 0x7f020413a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02040a5140 0x7f0204136490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02040a5140 0x7f0204136490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37898/0 (socket says 172.21.15.89:37898) 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 1 -- 172.21.15.89:0/917851835 learned_addr learned my addr 172.21.15.89:0/917851835 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0202d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 0x7f020413a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33882/0 (socket says 172.21.15.89:33882) 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 10 monclient(hunting): get_auth_request con 0x7f02040a5140 auth_method 0 2024-08-19T10:38:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0202d9d640 10 monclient(hunting): get_auth_request con 0x7f020413ea70 auth_method 0 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0202d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0202d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0202d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0202d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0202d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0202d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f020259c640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02040a4d60 0x7f0204135f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f020259c640 10 monclient(hunting): get_auth_request con 0x7f02040a4d60 auth_method 0 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f020259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f020259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f020259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f020259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f020259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f020259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 10 monclient(hunting): handle_auth_done global_id 4367 payload 824 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 1 -- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02040a4d60 msgr2=0x7f0204135f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02040a4d60 0x7f0204135f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 1 -- 172.21.15.89:0/917851835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 msgr2=0x7f020413a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 0x7f020413a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:38:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 1 monclient: found mon.noname-b 2024-08-19T10:38:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 1 -- 172.21.15.89:0/917851835 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0204120810 con 0x7f02040a5140 2024-08-19T10:38:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0202d9d640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f020413ea70 0x7f020413a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:24.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 1 -- 172.21.15.89:0/917851835 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02040a5140 msgr2=0x7f0204136490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:24.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0201d9b640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02040a5140 0x7f0204136490 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01f800c820 tx=0x7f01f800cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:24.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01f8015070 con 0x7f02040a5140 2024-08-19T10:38:24.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:24.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 10 monclient: dump: 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f800ce10 con 0x7f02040a5140 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01f8011860 con 0x7f02040a5140 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 5 monclient: authenticate success, global_id 4367 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient: _renew_subs 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 1 -- 172.21.15.89:0/917851835 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020413d7f0 con 0x7f02040a5140 2024-08-19T10:38:24.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0200d99640 4 set_mon_vals no callback set 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f01f2ffd640 10 monclient: dump: 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient: _renew_subs 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 1 -- 172.21.15.89:0/917851835 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0204120dd0 con 0x7f02040a5140 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413abc0 0x7f020413afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.050+0000 7f0208a33640 1 -- 172.21.15.89:0/917851835 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0204120c10 con 0x7f020413abc0 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f01f80113f0 con 0x7f02040a5140 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413abc0 0x7f020413afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 10 monclient: get_auth_request con 0x7f020413abc0 auth_method 0 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f01f2ffd640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f01e403be00 0x7f01e403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f01f804e350 con 0x7f02040a5140 2024-08-19T10:38:24.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 10 monclient(hunting): handle_auth_done global_id 4370 payload 824 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 1 -- 172.21.15.89:0/917851835 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0204076cc0 con 0x7f020413abc0 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f0202d9d640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f01e403be00 0x7f01e403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f0202d9d640 10 monclient: get_auth_request con 0x7f01e403be00 auth_method 0 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 1 -- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413abc0 msgr2=0x7f020413afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f020259c640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413abc0 0x7f020413afa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01f400f770 tx=0x7f01f400fc30 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01f400fe40 con 0x7f020413abc0 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f01f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f01f2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f01f2ffd640 10 monclient: dump: 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.077 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.054+0000 7f0202d9d640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f01e403be00 0x7f01e403e2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f02040a7cf0 tx=0x7f01fc0b4040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:24.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.058+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f01f4019070 con 0x7f020413abc0 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.058+0000 7f01f2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.058+0000 7f01f2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.058+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413abc0 msgr2=0x7f020413afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.058+0000 7f01f2ffd640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413abc0 0x7f020413afa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01f400f770 tx=0x7f01f400fc30 comp rx=0 tx=0).stop 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.058+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413abc0 msgr2=0x7f020413afa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.058+0000 7f01f2ffd640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413abc0 0x7f020413afa0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01f400f770 tx=0x7f01f400fc30 comp rx=0 tx=0).stop 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f0208a33640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f0208a33640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413b4e0 0x7f020413f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f0208a33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f0208a33640 1 -- 172.21.15.89:0/917851835 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f02040a5520 con 0x7f020413b4e0 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f020259c640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413b4e0 0x7f020413f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f020259c640 10 monclient: get_auth_request con 0x7f020413b4e0 auth_method 0 2024-08-19T10:38:24.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f020259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f020259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f020259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f020259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f020259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f020259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f020259c640 10 monclient(hunting): handle_auth_done global_id 4379 payload 824 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.062+0000 7f020259c640 1 -- 172.21.15.89:0/917851835 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0204064590 con 0x7f020413b4e0 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f020259c640 1 -- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413b4e0 msgr2=0x7f020413f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f020259c640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413b4e0 0x7f020413f190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f01f400fc70 tx=0x7f01f4003770 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01f401ec30 con 0x7f020413b4e0 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f01f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f01f2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:38:24.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f01f2ffd640 10 monclient: dump: 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.081 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.082 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.082 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.082 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.082 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9433 (secure 0 0 0) 0x7f01f4006c00 con 0x7f020413b4e0 2024-08-19T10:38:24.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f01f2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:24.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f01f2ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:24.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413b4e0 msgr2=0x7f020413f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f01f2ffd640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413b4e0 0x7f020413f190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f01f400fc70 tx=0x7f01f4003770 comp rx=0 tx=0).stop 2024-08-19T10:38:24.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f01f2ffd640 1 -- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413b4e0 msgr2=0x7f020413f190 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:24.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f01f2ffd640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f020413b4e0 0x7f020413f190 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f01f400fc70 tx=0x7f01f4003770 comp rx=0 tx=0).stop 2024-08-19T10:38:24.083 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:24.083 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:24.083 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:24.083 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:24.083 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 4, 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 33839, 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:24.084 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:24.085 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 33, 2024-08-19T10:38:24.085 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:24.085 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:24.085 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:24.085 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:24.085 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:24.085 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:24.085 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:24.085 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:24.085 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:24.086 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:24.087 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:24.088 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:24.089 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:24.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.090 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:24.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.090 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:24.090 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:24.090 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.090 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:24.091 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.094 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:24.095 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:24.096 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.098 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.101 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.102 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.103 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.106 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.106 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.106 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:24.106 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.107 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.107 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.107 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.107 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:24.107 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.107 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.107 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.107 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.107 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:24.107 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:38:24.108 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:24.109 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:24.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0208a33640 1 -- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f01e403be00 msgr2=0x7f01e403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0208a33640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f01e403be00 0x7f01e403e2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f02040a7cf0 tx=0x7f01fc0b4040 comp rx=0 tx=0).stop 2024-08-19T10:38:24.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0208a33640 10 monclient: shutdown 2024-08-19T10:38:24.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0208a33640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:24.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0208a33640 1 -- 172.21.15.89:0/917851835 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02040a5140 msgr2=0x7f0204136490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0208a33640 1 --2- 172.21.15.89:0/917851835 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02040a5140 0x7f0204136490 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01f800c820 tx=0x7f01f800cce0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0202d9d640 1 -- 172.21.15.89:0/917851835 reap_dead start 2024-08-19T10:38:24.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0202d9d640 1 -- 172.21.15.89:0/917851835 reap_dead start 2024-08-19T10:38:24.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0208a33640 1 -- 172.21.15.89:0/917851835 shutdown_connections 2024-08-19T10:38:24.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0208a33640 1 -- 172.21.15.89:0/917851835 >> 172.21.15.89:0/917851835 conn(0x7f02040aa240 msgr2=0x7f02040a7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:24.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0208a33640 1 -- 172.21.15.89:0/917851835 shutdown_connections 2024-08-19T10:38:24.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.066+0000 7f0208a33640 1 -- 172.21.15.89:0/917851835 wait complete. 2024-08-19T10:38:24.112 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:38:24.112 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:38:24.112 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:38:24.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.226+0000 7f8f5357c640 10 monclient: get_monmap_and_config 2024-08-19T10:38:24.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.226+0000 7f8f5357c640 10 monclient: build_initial_monmap 2024-08-19T10:38:24.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 10 monclient: monmap: 2024-08-19T10:38:24.232 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:24.232 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.232 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:24.232916+0000 2024-08-19T10:38:24.232 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:24.232916+0000 2024-08-19T10:38:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:24.234 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:24.235 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:24.235 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 1 Processor -- start 2024-08-19T10:38:24.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 1 -- start start 2024-08-19T10:38:24.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 10 monclient: init 2024-08-19T10:38:24.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:24.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 10 monclient: _add_conns ranks=[7,0,4,3,2,1,6,5,8,10,9] 2024-08-19T10:38:24.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c0a6860 0x7f8f4c0a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 10 monclient(hunting): picked mon.noname-i con 0x7f8f4c0a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:24.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8f4c0a54d0 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:24.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a58b0 0x7f8f4c1408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 10 monclient(hunting): picked mon.noname-c con 0x7f8f4c0a58b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:24.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f4c0a4c90 con 0x7f8f4c0a54d0 2024-08-19T10:38:24.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f8f4c0a7350 con 0x7f8f4c0a58b0 2024-08-19T10:38:24.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8f4c0a7550 con 0x7f8f4c0a6860 2024-08-19T10:38:24.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:24.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f5357c640 10 monclient(hunting): authenticate will time out at 954.049679s 2024-08-19T10:38:24.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f50b17640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8f4c0a54d0 0x7f8f4c0a6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44682/0 (socket says 172.21.15.89:44682) 2024-08-19T10:38:24.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f50b17640 1 -- 172.21.15.89:0/105717458 learned_addr learned my addr 172.21.15.89:0/105717458 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:24.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 1 --2- 172.21.15.89:0/105717458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a58b0 0x7f8f4c1408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 10 monclient(hunting): get_auth_request con 0x7f8f4c0a58b0 auth_method 0 2024-08-19T10:38:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51318640 1 --2- 172.21.15.89:0/105717458 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c0a6860 0x7f8f4c0a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51318640 10 monclient(hunting): get_auth_request con 0x7f8f4c0a6860 auth_method 0 2024-08-19T10:38:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51318640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51318640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51318640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 10 monclient(hunting): handle_auth_done global_id 4468 payload 274 2024-08-19T10:38:24.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 1 -- 172.21.15.89:0/105717458 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c0a6860 msgr2=0x7f8f4c0a81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:24.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 1 --2- 172.21.15.89:0/105717458 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c0a6860 0x7f8f4c0a81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 1 -- 172.21.15.89:0/105717458 >> v1:172.21.15.89:6789/0 conn(0x7f8f4c0a54d0 legacy=0x7f8f4c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:24.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 10 monclient: _finish_hunting 0 2024-08-19T10:38:24.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 1 monclient: found mon.noname-c 2024-08-19T10:38:24.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:24.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:24.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.230+0000 7f8f51b19640 1 -- 172.21.15.89:0/105717458 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c141e30 con 0x7f8f4c0a58b0 2024-08-19T10:38:24.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f51b19640 1 -- 172.21.15.89:0/105717458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a58b0 msgr2=0x7f8f4c1408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:24.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f51b19640 1 --2- 172.21.15.89:0/105717458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a58b0 0x7f8f4c1408e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f440097f0 tx=0x7f8f4401c430 comp rx=0 tx=0).ready entity=mon.4 client_cookie=41e255945d38c2f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:24.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 1 -- 172.21.15.89:0/105717458 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8f440b0070 con 0x7f8f4c0a58b0 2024-08-19T10:38:24.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:24.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 10 monclient: dump: 2024-08-19T10:38:24.262 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.262 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.262 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.262 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.262 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.263 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.264 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.264 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.264 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.264 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.264 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.264 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 10 monclient: _finish_auth 0 2024-08-19T10:38:24.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:24.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:24.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 1 -- 172.21.15.89:0/105717458 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f440a9980 con 0x7f8f4c0a58b0 2024-08-19T10:38:24.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 5 monclient: authenticate success, global_id 4468 2024-08-19T10:38:24.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 1 -- 172.21.15.89:0/105717458 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8f440aa3a0 con 0x7f8f4c0a58b0 2024-08-19T10:38:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f4bfff640 10 monclient: dump: 2024-08-19T10:38:24.266 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.267 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.267 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.267 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.267 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.267 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.268 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.269 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:24.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 4 set_mon_vals no callback set 2024-08-19T10:38:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 10 monclient: shutdown 2024-08-19T10:38:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 1 -- 172.21.15.89:0/105717458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a58b0 msgr2=0x7f8f4c1408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 1 --2- 172.21.15.89:0/105717458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a58b0 0x7f8f4c1408e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f440097f0 tx=0x7f8f4401c430 comp rx=0 tx=0).stop 2024-08-19T10:38:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 1 -- 172.21.15.89:0/105717458 shutdown_connections 2024-08-19T10:38:24.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 1 --2- 172.21.15.89:0/105717458 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a58b0 0x7f8f4c1408e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 1 --2- 172.21.15.89:0/105717458 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c0a6860 0x7f8f4c0a81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 1 -- 172.21.15.89:0/105717458 >> 172.21.15.89:0/105717458 conn(0x7f8f4c069a00 msgr2=0x7f8f4c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 1 -- 172.21.15.89:0/105717458 shutdown_connections 2024-08-19T10:38:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 1 -- 172.21.15.89:0/105717458 wait complete. 2024-08-19T10:38:24.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 10 monclient: build_initial_monmap 2024-08-19T10:38:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.234+0000 7f8f5357c640 10 monclient: monmap: 2024-08-19T10:38:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:24.272 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:24.273 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:24.274 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:24.274 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 1 Processor -- start 2024-08-19T10:38:24.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 1 -- start start 2024-08-19T10:38:24.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 10 monclient: init 2024-08-19T10:38:24.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:24.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 10 monclient: _add_conns ranks=[4,7,3,8,1,5,0,6,2] 2024-08-19T10:38:24.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a54d0 0x7f8f4c1346d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 10 monclient(hunting): picked mon.noname-c con 0x7f8f4c0a54d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:24.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c0a58b0 0x7f8f4c134c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 10 monclient(hunting): picked mon.noname-i con 0x7f8f4c0a58b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:24.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f4c0a6860 0x7f8f4c135150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 10 monclient(hunting): picked mon.noname-g con 0x7f8f4c0a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:24.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4c141a00 con 0x7f8f4c0a6860 2024-08-19T10:38:24.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f8f4c141880 con 0x7f8f4c0a54d0 2024-08-19T10:38:24.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8f4c141b80 con 0x7f8f4c0a58b0 2024-08-19T10:38:24.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:24.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f5357c640 10 monclient(hunting): authenticate will time out at 954.056973s 2024-08-19T10:38:24.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a54d0 0x7f8f4c1346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a54d0 0x7f8f4c1346d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33896/0 (socket says 172.21.15.89:33896) 2024-08-19T10:38:24.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 1 -- 172.21.15.89:0/4201580459 learned_addr learned my addr 172.21.15.89:0/4201580459 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:24.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 10 monclient(hunting): get_auth_request con 0x7f8f4c0a54d0 auth_method 0 2024-08-19T10:38:24.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:24.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:24.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:24.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51b19640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f4c0a6860 0x7f8f4c135150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51b19640 10 monclient(hunting): get_auth_request con 0x7f8f4c0a6860 auth_method 0 2024-08-19T10:38:24.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51b19640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51b19640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51b19640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f50b17640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c0a58b0 0x7f8f4c134c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f50b17640 10 monclient(hunting): get_auth_request con 0x7f8f4c0a58b0 auth_method 0 2024-08-19T10:38:24.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f50b17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f50b17640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f50b17640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51b19640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:24.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51b19640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:24.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51b19640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:24.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 10 monclient(hunting): handle_auth_done global_id 4477 payload 824 2024-08-19T10:38:24.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 1 -- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c0a58b0 msgr2=0x7f8f4c134c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c0a58b0 0x7f8f4c134c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 1 -- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f4c0a6860 msgr2=0x7f8f4c135150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f4c0a6860 0x7f8f4c135150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 10 monclient: _finish_hunting 0 2024-08-19T10:38:24.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 1 monclient: found mon.noname-c 2024-08-19T10:38:24.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:24.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:24.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 1 -- 172.21.15.89:0/4201580459 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c0a19f0 con 0x7f8f4c0a54d0 2024-08-19T10:38:24.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 1 -- 172.21.15.89:0/4201580459 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a54d0 msgr2=0x7f8f4c1346d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:24.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.238+0000 7f8f51318640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a54d0 0x7f8f4c1346d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c007e40 tx=0x7f8f3c00e4b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:24.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8f3c013a00 con 0x7f8f4c0a54d0 2024-08-19T10:38:24.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:24.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 10 monclient: dump: 2024-08-19T10:38:24.285 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.285 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.286 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.286 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.286 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.286 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.286 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.287 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.287 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.287 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.287 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.287 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.287 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.288 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.288 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.288 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:24.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:24.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:24.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f3c019070 con 0x7f8f4c0a54d0 2024-08-19T10:38:24.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:24.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8f3c013c00 con 0x7f8f4c0a54d0 2024-08-19T10:38:24.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:24.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 10 monclient: dump: 2024-08-19T10:38:24.290 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.290 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.290 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.290 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.290 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.291 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.291 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.291 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.291 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.291 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.292 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.292 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.292 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.292 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.292 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.292 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f5357c640 5 monclient: authenticate success, global_id 4477 2024-08-19T10:38:24.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f5357c640 10 monclient: _renew_subs 2024-08-19T10:38:24.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f4b7fe640 4 set_mon_vals no callback set 2024-08-19T10:38:24.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f5357c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:24.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f5357c640 1 -- 172.21.15.89:0/4201580459 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4c13cb30 con 0x7f8f4c0a54d0 2024-08-19T10:38:24.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f5357c640 10 monclient: _renew_subs 2024-08-19T10:38:24.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f5357c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:24.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f5357c640 1 -- 172.21.15.89:0/4201580459 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4c0a12d0 con 0x7f8f4c0a54d0 2024-08-19T10:38:24.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f5357c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:24.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f5357c640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c063b30 0x7f8f4c063f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f5357c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f5357c640 1 -- 172.21.15.89:0/4201580459 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8f4c064590 con 0x7f8f4c063b30 2024-08-19T10:38:24.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8f3c013590 con 0x7f8f4c0a54d0 2024-08-19T10:38:24.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f50b17640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c063b30 0x7f8f4c063f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8f3803be00 0x7f8f3803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f50b17640 10 monclient: get_auth_request con 0x7f8f4c063b30 auth_method 0 2024-08-19T10:38:24.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f50b17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f50b17640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f50b17640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f8f3c04eec0 con 0x7f8f4c0a54d0 2024-08-19T10:38:24.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f50b17640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:24.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f50b17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:24.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f50b17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:24.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f51b19640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8f3803be00 0x7f8f3803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f51b19640 10 monclient: get_auth_request con 0x7f8f3803be00 auth_method 0 2024-08-19T10:38:24.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f50b17640 10 monclient(hunting): handle_auth_done global_id 4354 payload 824 2024-08-19T10:38:24.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.242+0000 7f8f50b17640 1 -- 172.21.15.89:0/4201580459 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f4c076c60 con 0x7f8f4c063b30 2024-08-19T10:38:24.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f50b17640 1 -- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c063b30 msgr2=0x7f8f4c063f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:24.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f50b17640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c063b30 0x7f8f4c063f10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000e500 tx=0x7f8f4000e9c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f51b19640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8f3803be00 0x7f8f3803e2c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0a5440 tx=0x7f8f440a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8f40004e40 con 0x7f8f4c063b30 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f49ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f49ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f49ffb640 10 monclient: dump: 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.298 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8f40018070 con 0x7f8f4c063b30 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f49ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f49ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c063b30 msgr2=0x7f8f4c063f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f49ffb640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c063b30 0x7f8f4c063f10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000e500 tx=0x7f8f4000e9c0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c063b30 msgr2=0x7f8f4c063f10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:24.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.246+0000 7f8f49ffb640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c063b30 0x7f8f4c063f10 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000e500 tx=0x7f8f4000e9c0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f5357c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:24.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f5357c640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c02d1d0 0x7f8f4c02d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f5357c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f5357c640 1 -- 172.21.15.89:0/4201580459 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8f4c13a160 con 0x7f8f4c02d1d0 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f50b17640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c02d1d0 0x7f8f4c02d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f50b17640 10 monclient: get_auth_request con 0x7f8f4c02d1d0 auth_method 0 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f50b17640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f50b17640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f50b17640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f50b17640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f50b17640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.250+0000 7f8f50b17640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f50b17640 10 monclient(hunting): handle_auth_done global_id 4363 payload 824 2024-08-19T10:38:24.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f50b17640 1 -- 172.21.15.89:0/4201580459 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8f4c06ca20 con 0x7f8f4c02d1d0 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f50b17640 1 -- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c02d1d0 msgr2=0x7f8f4c02d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f50b17640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c02d1d0 0x7f8f4c02d5e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000e4d0 tx=0x7f8f40004950 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8f40028440 con 0x7f8f4c02d1d0 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f49ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f49ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f49ffb640 10 monclient: dump: 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:24.303 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9600 (secure 0 0 0) 0x7f8f4001ee50 con 0x7f8f4c02d1d0 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f49ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f49ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c02d1d0 msgr2=0x7f8f4c02d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f49ffb640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c02d1d0 0x7f8f4c02d5e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000e4d0 tx=0x7f8f40004950 comp rx=0 tx=0).stop 2024-08-19T10:38:24.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f49ffb640 1 -- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c02d1d0 msgr2=0x7f8f4c02d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:24.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.254+0000 7f8f49ffb640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f4c02d1d0 0x7f8f4c02d5e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.305 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 4, 2024-08-19T10:38:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 34043, 2024-08-19T10:38:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:24.305 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 33, 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:24.306 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:24.307 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.308 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:24.309 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:24.310 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:24.311 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.312 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.313 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:24.314 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.316 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.317 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.319 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.320 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:24.322 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:24.325 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:24.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:24.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:24.328 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f5357c640 1 -- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8f3803be00 msgr2=0x7f8f3803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f5357c640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8f3803be00 0x7f8f3803e2c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0a5440 tx=0x7f8f440a6000 comp rx=0 tx=0).stop 2024-08-19T10:38:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f5357c640 10 monclient: shutdown 2024-08-19T10:38:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f5357c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f5357c640 1 -- 172.21.15.89:0/4201580459 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a54d0 msgr2=0x7f8f4c1346d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f5357c640 1 --2- 172.21.15.89:0/4201580459 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8f4c0a54d0 0x7f8f4c1346d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c007e40 tx=0x7f8f3c00e4b0 comp rx=0 tx=0).stop 2024-08-19T10:38:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f51b19640 1 -- 172.21.15.89:0/4201580459 reap_dead start 2024-08-19T10:38:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f51b19640 1 -- 172.21.15.89:0/4201580459 reap_dead start 2024-08-19T10:38:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f5357c640 1 -- 172.21.15.89:0/4201580459 shutdown_connections 2024-08-19T10:38:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f5357c640 1 -- 172.21.15.89:0/4201580459 >> 172.21.15.89:0/4201580459 conn(0x7f8f4c069a00 msgr2=0x7f8f4c0ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:24.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f5357c640 1 -- 172.21.15.89:0/4201580459 shutdown_connections 2024-08-19T10:38:24.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:24.258+0000 7f8f5357c640 1 -- 172.21.15.89:0/4201580459 wait complete. 2024-08-19T10:38:24.331 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:38:24.331 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['f', 'i'] 2024-08-19T10:38:24.331 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b', 'h'] 2024-08-19T10:38:24.331 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-08-19T10:38:24.331 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-08-19T10:38:24.332 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-08-19T10:38:24.332 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:38:24.349 INFO:tasks.ceph.mon.f:Stopped 2024-08-19T10:38:24.349 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-08-19T10:38:24.349 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-08-19T10:38:24.349 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-08-19T10:38:24.349 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:38:24.417 INFO:tasks.ceph.mon.i:Stopped 2024-08-19T10:38:24.418 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-08-19T10:38:24.418 INFO:tasks.ceph.mon.b:Sent signal 19 2024-08-19T10:38:24.418 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-08-19T10:38:24.418 INFO:tasks.ceph.mon.h:Sent signal 19 2024-08-19T10:38:24.418 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-08-19T10:38:39.421 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-08-19T10:38:39.422 INFO:tasks.ceph.mon.b:Sent signal 18 2024-08-19T10:38:39.422 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-08-19T10:38:39.422 INFO:tasks.ceph.mon.h:Sent signal 18 2024-08-19T10:38:39.422 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-08-19T10:38:39.422 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:38:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.550+0000 7f916ab13640 10 monclient: get_monmap_and_config 2024-08-19T10:38:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.550+0000 7f916ab13640 10 monclient: build_initial_monmap 2024-08-19T10:38:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 10 monclient: monmap: 2024-08-19T10:38:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:39.555 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:39.556974+0000 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:39.556974+0000 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:39.556 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:39.557 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:39.557 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:39.557 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:39.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 1 Processor -- start 2024-08-19T10:38:39.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 1 -- start start 2024-08-19T10:38:39.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 10 monclient: init 2024-08-19T10:38:39.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:39.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 10 monclient: _add_conns ranks=[1,5,8,7,4,2,3,9,10,0,6] 2024-08-19T10:38:39.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f916409b390 addr v1:172.21.15.190:6789/0 2024-08-19T10:38:39.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f916409ec20 0x7f916413ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:39.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 10 monclient(hunting): picked mon.noname-h con 0x7f916409ec20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 10 monclient(hunting): picked mon.noname-e con 0x7f916413b0a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f916406db50 con 0x7f916409b390 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f916409c900 con 0x7f916409ec20 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f916409be80 con 0x7f916413b0a0 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f916ab13640 10 monclient(hunting): authenticate will time out at 969.373848s 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91688af640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f916409b390 legacy=0x7f916409b770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91690b0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91690b0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:53120/0 (socket says 172.21.15.89:53120) 2024-08-19T10:38:39.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91690b0640 1 -- 172.21.15.89:0/632680780 learned_addr learned my addr 172.21.15.89:0/632680780 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91690b0640 10 monclient(hunting): get_auth_request con 0x7f916413b0a0 auth_method 0 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91690b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91690b0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91690b0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 1 --2- 172.21.15.89:0/632680780 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f916409ec20 0x7f916413ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 10 monclient(hunting): get_auth_request con 0x7f916409ec20 auth_method 0 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91690b0640 1 -- 172.21.15.89:0/632680780 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 msgr2=0x7f916413d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91690b0640 1 -- 172.21.15.89:0/632680780 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 msgr2=0x7f916413d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91690b0640 1 --2- 172.21.15.89:0/632680780 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:39.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f91690b0640 1 --2- 172.21.15.89:0/632680780 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 10 monclient(hunting): handle_auth_done global_id 4388 payload 274 2024-08-19T10:38:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 1 -- 172.21.15.89:0/632680780 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 msgr2=0x7f916413d490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 1 --2- 172.21.15.89:0/632680780 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 1 -- 172.21.15.89:0/632680780 >> v1:172.21.15.190:6789/0 conn(0x7f916409b390 legacy=0x7f916409b770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 1 monclient: found mon.noname-h 2024-08-19T10:38:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:39.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:38:39.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.554+0000 7f9163fff640 1 -- 172.21.15.89:0/632680780 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916409d9a0 con 0x7f916409ec20 2024-08-19T10:38:39.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f9163fff640 1 -- 172.21.15.89:0/632680780 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f916409ec20 msgr2=0x7f916413ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:39.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f9163fff640 1 --2- 172.21.15.89:0/632680780 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f916409ec20 0x7f916413ab60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9158009820 tx=0x7f915801c350 comp rx=0 tx=0).ready entity=mon.5 client_cookie=7af561791bd11ffe server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:39.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 1 -- 172.21.15.89:0/632680780 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91580b0070 con 0x7f916409ec20 2024-08-19T10:38:39.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:39.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:38:39.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 10 monclient: dump: 2024-08-19T10:38:39.567 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:39.567 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:39.567 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:39.567 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:39.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:39.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:39.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:39.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f916ab13640 5 monclient: authenticate success, global_id 4388 2024-08-19T10:38:39.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f916ab13640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:39.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 1 -- 172.21.15.89:0/632680780 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91580aa3a0 con 0x7f916409ec20 2024-08-19T10:38:39.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:39.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f916ab13640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:39.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f916ab13640 4 set_mon_vals no callback set 2024-08-19T10:38:39.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 1 -- 172.21.15.89:0/632680780 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91580aaa40 con 0x7f916409ec20 2024-08-19T10:38:39.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:39.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.558+0000 7f91637fe640 10 monclient: dump: 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:39.570 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient: shutdown 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 -- 172.21.15.89:0/632680780 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f916409ec20 msgr2=0x7f916413ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 --2- 172.21.15.89:0/632680780 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f916409ec20 0x7f916413ab60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9158009820 tx=0x7f915801c350 comp rx=0 tx=0).stop 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 -- 172.21.15.89:0/632680780 shutdown_connections 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 --2- 172.21.15.89:0/632680780 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 --2- 172.21.15.89:0/632680780 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f916409ec20 0x7f916413ab60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 -- 172.21.15.89:0/632680780 >> 172.21.15.89:0/632680780 conn(0x7f91640a4df0 msgr2=0x7f91640a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 -- 172.21.15.89:0/632680780 shutdown_connections 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 -- 172.21.15.89:0/632680780 wait complete. 2024-08-19T10:38:39.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient: build_initial_monmap 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient: monmap: 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:39.572 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 Processor -- start 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 -- start start 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient: init 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient: _add_conns ranks=[7,0,8,5,2,4,6,3,1] 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f916409b390 0x7f9164073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient(hunting): picked mon.noname-i con 0x7f916409b390 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f916409ec20 0x7f916413b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient(hunting): picked mon.noname-a con 0x7f916409ec20 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:39.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient(hunting): picked mon.noname-e con 0x7f916413b0a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f91640a5bd0 con 0x7f916409ec20 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f916409cb40 con 0x7f916409b390 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f916413d9d0 con 0x7f916413b0a0 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f916ab13640 10 monclient(hunting): authenticate will time out at 969.382126s 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f91688af640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f916409b390 msgr2=0x7f9164073ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f91688af640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f916409b390 0x7f9164073ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f916409ec20 0x7f916413b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:39.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f916409ec20 0x7f916413b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37482/0 (socket says 172.21.15.89:37482) 2024-08-19T10:38:39.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 1 -- 172.21.15.89:0/714828254 learned_addr learned my addr 172.21.15.89:0/714828254 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:39.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 10 monclient(hunting): get_auth_request con 0x7f916409ec20 auth_method 0 2024-08-19T10:38:39.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:39.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:39.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:39.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f91690b0640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:39.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f91690b0640 10 monclient(hunting): get_auth_request con 0x7f916413b0a0 auth_method 0 2024-08-19T10:38:39.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f91690b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:39.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f91690b0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:39.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f91690b0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:39.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f91690b0640 1 -- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 msgr2=0x7f916413b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f91690b0640 1 -- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 msgr2=0x7f916413b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f91690b0640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f91690b0640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 10 monclient(hunting): handle_auth_done global_id 4527 payload 824 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 1 -- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 msgr2=0x7f916413b9c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 1 -- 172.21.15.89:0/714828254 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f916409b390 msgr2=0x7f9164073ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f916409b390 0x7f9164073ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 1 monclient: found mon.noname-a 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:39.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:39.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.562+0000 7f9163fff640 1 -- 172.21.15.89:0/714828254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916413cb60 con 0x7f916409ec20 2024-08-19T10:38:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f9163fff640 1 -- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f916409ec20 msgr2=0x7f916413b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f9163fff640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f916409ec20 0x7f916413b480 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f915801c180 tx=0x7f91580190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 1 -- 172.21.15.89:0/714828254 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91580b0040 con 0x7f916409ec20 2024-08-19T10:38:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 10 monclient: dump: 2024-08-19T10:38:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f916ab13640 5 monclient: authenticate success, global_id 4527 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f916ab13640 10 monclient: _renew_subs 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f916ab13640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f916ab13640 1 -- 172.21.15.89:0/714828254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9164073490 con 0x7f916409ec20 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f916ab13640 10 monclient: _renew_subs 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f916ab13640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f916ab13640 1 -- 172.21.15.89:0/714828254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916409bbd0 con 0x7f916409ec20 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 1 -- 172.21.15.89:0/714828254 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91580039f0 con 0x7f916409ec20 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 1 -- 172.21.15.89:0/714828254 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9158003ba0 con 0x7f916409ec20 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f9162ffd640 4 set_mon_vals no callback set 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 10 monclient: dump: 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:39.582 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f916ab13640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:39.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f916ab13640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f916ab13640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 1 -- 172.21.15.89:0/714828254 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f91580ab400 con 0x7f916409ec20 2024-08-19T10:38:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f916ab13640 1 -- 172.21.15.89:0/714828254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9144005090 con 0x7f916409ec20 2024-08-19T10:38:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f914803be00 0x7f914803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91617fa640 1 -- 172.21.15.89:0/714828254 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f91580e81d0 con 0x7f916409ec20 2024-08-19T10:38:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91688af640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f914803be00 0x7f914803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.566+0000 7f91688af640 10 monclient: get_auth_request con 0x7f914803be00 auth_method 0 2024-08-19T10:38:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.570+0000 7f91688af640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f914803be00 0x7f914803e2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9150006fd0 tx=0x7f9150006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.570+0000 7f91617fa640 1 -- 172.21.15.89:0/714828254 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f91580c2020 con 0x7f916409ec20 2024-08-19T10:38:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.570+0000 7f91617fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.570+0000 7f91617fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.798+0000 7f916ab13640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:38:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.798+0000 7f916ab13640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:38:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.798+0000 7f916ab13640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.798+0000 7f916ab13640 1 -- 172.21.15.89:0/714828254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9144005380 con 0x7f916409ec20 2024-08-19T10:38:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.798+0000 7f91617fa640 1 -- 172.21.15.89:0/714828254 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3167 (secure 0 0 0) 0x7f916413cb60 con 0x7f916409ec20 2024-08-19T10:38:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.798+0000 7f91617fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:38:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.798+0000 7f91617fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:39.803 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":8,"quorum":[0,2,3,5,6],"quorum_names":["a","b","g","h","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:38:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 -- 172.21.15.89:0/714828254 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f914803be00 msgr2=0x7f914803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f914803be00 0x7f914803e2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9150006fd0 tx=0x7f9150006d40 comp rx=0 tx=0).stop 2024-08-19T10:38:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 10 monclient: shutdown 2024-08-19T10:38:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 -- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f916409ec20 msgr2=0x7f916413b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f916409ec20 0x7f916413b480 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f915801c180 tx=0x7f91580190f0 comp rx=0 tx=0).stop 2024-08-19T10:38:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 -- 172.21.15.89:0/714828254 shutdown_connections 2024-08-19T10:38:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f914803be00 0x7f914803e2c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f916413b0a0 0x7f916413b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f916409ec20 0x7f916413b480 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 --2- 172.21.15.89:0/714828254 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f916409b390 0x7f9164073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 -- 172.21.15.89:0/714828254 >> 172.21.15.89:0/714828254 conn(0x7f91640a4df0 msgr2=0x7f9164139670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 -- 172.21.15.89:0/714828254 shutdown_connections 2024-08-19T10:38:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:39.802+0000 7f916ab13640 1 -- 172.21.15.89:0/714828254 wait complete. 2024-08-19T10:38:42.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:38:42.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.946+0000 7f8b65ff5640 10 monclient: get_monmap_and_config 2024-08-19T10:38:42.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.946+0000 7f8b65ff5640 10 monclient: build_initial_monmap 2024-08-19T10:38:42.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.946+0000 7f8b65ff5640 10 monclient: monmap: 2024-08-19T10:38:42.949 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:42.949 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:42.949 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:42.950937+0000 2024-08-19T10:38:42.949 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:42.950937+0000 2024-08-19T10:38:42.949 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:42.949 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:42.950 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.946+0000 7f8b65ff5640 1 Processor -- start 2024-08-19T10:38:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.946+0000 7f8b65ff5640 1 -- start start 2024-08-19T10:38:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.946+0000 7f8b65ff5640 10 monclient: init 2024-08-19T10:38:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 10 monclient: _add_conns ranks=[3,0,1,5,4,6,9,7,10,2,8] 2024-08-19T10:38:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6006c930 0x7f8b6006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 10 monclient(hunting): picked mon.noname-g con 0x7f8b6006c930 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8b6009d340 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8b6006d250 addr v1:172.21.15.190:6789/0 2024-08-19T10:38:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b60138320 con 0x7f8b6009d340 2024-08-19T10:38:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b6009fbf0 con 0x7f8b6006d250 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8b600a54f0 con 0x7f8b6006c930 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b65ff5640 10 monclient(hunting): authenticate will time out at 972.768244s 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5ffff640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f8b6006d250 legacy=0x7f8b60137c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b56ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8b6009d340 0x7f8b60137500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45280/0 (socket says 172.21.15.89:45280) 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b56ffd640 1 -- 172.21.15.89:0/1367969812 learned_addr learned my addr 172.21.15.89:0/1367969812 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 1 --2- 172.21.15.89:0/1367969812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6006c930 0x7f8b6006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 10 monclient(hunting): get_auth_request con 0x7f8b6006c930 auth_method 0 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 10 monclient(hunting): handle_auth_done global_id 4386 payload 274 2024-08-19T10:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 1 -- 172.21.15.89:0/1367969812 >> v1:172.21.15.190:6789/0 conn(0x7f8b6006d250 legacy=0x7f8b60137c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 1 -- 172.21.15.89:0/1367969812 >> v1:172.21.15.89:6789/0 conn(0x7f8b6009d340 legacy=0x7f8b60137500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 1 monclient: found mon.noname-g 2024-08-19T10:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:38:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.950+0000 7f8b5f7fe640 1 -- 172.21.15.89:0/1367969812 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b600a3b30 con 0x7f8b6006c930 2024-08-19T10:38:42.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5f7fe640 1 -- 172.21.15.89:0/1367969812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6006c930 msgr2=0x7f8b6006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:42.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5f7fe640 1 --2- 172.21.15.89:0/1367969812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6006c930 0x7f8b6006cd10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8b500097f0 tx=0x7f8b5001c3a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=d7db147300ce3025 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 1 -- 172.21.15.89:0/1367969812 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b500b0070 con 0x7f8b6006c930 2024-08-19T10:38:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:38:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 10 monclient: dump: 2024-08-19T10:38:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 10 monclient: _finish_auth 0 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 10 monclient: _check_auth_tickets 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 1 -- 172.21.15.89:0/1367969812 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b500aa810 con 0x7f8b6006c930 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 1 -- 172.21.15.89:0/1367969812 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b500a93e0 con 0x7f8b6006c930 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 5 monclient: authenticate success, global_id 4386 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 4 set_mon_vals no callback set 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b5effd640 10 monclient: dump: 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:42.960 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:42.961 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 10 monclient: shutdown 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 1 -- 172.21.15.89:0/1367969812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6006c930 msgr2=0x7f8b6006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 1 --2- 172.21.15.89:0/1367969812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6006c930 0x7f8b6006cd10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8b500097f0 tx=0x7f8b5001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 1 -- 172.21.15.89:0/1367969812 shutdown_connections 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 1 --2- 172.21.15.89:0/1367969812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6006c930 0x7f8b6006cd10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 1 -- 172.21.15.89:0/1367969812 >> 172.21.15.89:0/1367969812 conn(0x7f8b600a8f60 msgr2=0x7f8b600a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 1 -- 172.21.15.89:0/1367969812 shutdown_connections 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 1 -- 172.21.15.89:0/1367969812 wait complete. 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 10 monclient: build_initial_monmap 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 10 monclient: monmap: 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:42.962 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:42.963 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:42.964 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:42.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 1 Processor -- start 2024-08-19T10:38:42.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 1 -- start start 2024-08-19T10:38:42.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.954+0000 7f8b65ff5640 10 monclient: init 2024-08-19T10:38:42.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:42.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient: _add_conns ranks=[0,2,3,8,4,1,5,6,7] 2024-08-19T10:38:42.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b6006c930 0x7f8b60136740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:42.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient(hunting): picked mon.noname-a con 0x7f8b6006c930 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:42.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b6006d250 0x7f8b60136c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:42.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient(hunting): picked mon.noname-b con 0x7f8b6006d250 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:42.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6009d340 0x7f8b601371c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:42.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient(hunting): picked mon.noname-g con 0x7f8b6009d340 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:42.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60142f90 con 0x7f8b6006c930 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8b60074420 con 0x7f8b6006d250 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8b60143110 con 0x7f8b6009d340 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient(hunting): authenticate will time out at 972.776266s 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b6006c930 0x7f8b60136740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b6006d250 0x7f8b60136c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b6006c930 0x7f8b60136740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60124/0 (socket says 172.21.15.89:60124) 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 1 -- 172.21.15.89:0/33730738 learned_addr learned my addr 172.21.15.89:0/33730738 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 10 monclient(hunting): get_auth_request con 0x7f8b6006c930 auth_method 0 2024-08-19T10:38:42.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b56ffd640 10 monclient(hunting): get_auth_request con 0x7f8b6006d250 auth_method 0 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b56ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b56ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b56ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b56ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b56ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:42.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5ffff640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6009d340 0x7f8b601371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request con 0x7f8b6009d340 auth_method 0 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 10 monclient(hunting): handle_auth_done global_id 4536 payload 824 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 1 -- 172.21.15.89:0/33730738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6009d340 msgr2=0x7f8b601371c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6009d340 0x7f8b601371c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 1 -- 172.21.15.89:0/33730738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b6006d250 msgr2=0x7f8b60136c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b6006d250 0x7f8b60136c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:42.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 1 monclient: found mon.noname-a 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 1 -- 172.21.15.89:0/33730738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b601204a0 con 0x7f8b6006c930 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b56ffd640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b6006d250 0x7f8b60136c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 1 -- 172.21.15.89:0/33730738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b6006c930 msgr2=0x7f8b60136740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5f7fe640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b6006c930 0x7f8b60136740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b500ae040 tx=0x7f8b50019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 1 -- 172.21.15.89:0/33730738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b500b0040 con 0x7f8b6006c930 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 10 monclient: dump: 2024-08-19T10:38:42.972 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:42.973 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 10 monclient: _finish_auth 0 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 10 monclient: _check_auth_tickets 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 1 -- 172.21.15.89:0/33730738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b500044b0 con 0x7f8b6006c930 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 1 -- 172.21.15.89:0/33730738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b50003a90 con 0x7f8b6006c930 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5cff9640 10 monclient: dump: 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:42.974 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:42.975 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:42.975 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:42.975 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:42.975 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:42.975 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:42.975 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:42.975 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:42.975 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:42.975 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:42.975 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:42.976 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:42.976 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:42.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b5e7fc640 4 set_mon_vals no callback set 2024-08-19T10:38:42.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 5 monclient: authenticate success, global_id 4536 2024-08-19T10:38:42.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient: _renew_subs 2024-08-19T10:38:42.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:42.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 1 -- 172.21.15.89:0/33730738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b600a8a70 con 0x7f8b6006c930 2024-08-19T10:38:42.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient: _renew_subs 2024-08-19T10:38:42.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:42.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.958+0000 7f8b65ff5640 1 -- 172.21.15.89:0/33730738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b600a6410 con 0x7f8b6006c930 2024-08-19T10:38:42.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.962+0000 7f8b65ff5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:42.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.962+0000 7f8b65ff5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:42.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.962+0000 7f8b65ff5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:42.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.962+0000 7f8b65ff5640 1 -- 172.21.15.89:0/33730738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6006cd10 con 0x7f8b6006c930 2024-08-19T10:38:42.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.962+0000 7f8b5cff9640 1 -- 172.21.15.89:0/33730738 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8b50003620 con 0x7f8b6006c930 2024-08-19T10:38:42.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.962+0000 7f8b5cff9640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b4003be00 0x7f8b4003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:42.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.962+0000 7f8b5cff9640 1 -- 172.21.15.89:0/33730738 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f8b500e8710 con 0x7f8b6006c930 2024-08-19T10:38:42.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.962+0000 7f8b56ffd640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b4003be00 0x7f8b4003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:42.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.962+0000 7f8b56ffd640 10 monclient: get_auth_request con 0x7f8b4003be00 auth_method 0 2024-08-19T10:38:42.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.966+0000 7f8b56ffd640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b4003be00 0x7f8b4003e2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b48007ca0 tx=0x7f8b48007bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:42.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.966+0000 7f8b5cff9640 1 -- 172.21.15.89:0/33730738 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8b500160b0 con 0x7f8b6006c930 2024-08-19T10:38:42.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.966+0000 7f8b5cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:42.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:42.966+0000 7f8b5cff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.198+0000 7f8b65ff5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:38:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.198+0000 7f8b65ff5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:38:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.198+0000 7f8b65ff5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:38:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.198+0000 7f8b65ff5640 1 -- 172.21.15.89:0/33730738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8b6006ed40 con 0x7f8b6006c930 2024-08-19T10:38:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.198+0000 7f8b5cff9640 1 -- 172.21.15.89:0/33730738 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3167 (secure 0 0 0) 0x7f8b500a9df0 con 0x7f8b6006c930 2024-08-19T10:38:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.198+0000 7f8b5cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:38:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.198+0000 7f8b5cff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:43.202 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":8,"quorum":[0,2,3,5,6],"quorum_names":["a","b","g","h","d"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:38:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 -- 172.21.15.89:0/33730738 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b4003be00 msgr2=0x7f8b4003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b4003be00 0x7f8b4003e2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b48007ca0 tx=0x7f8b48007bf0 comp rx=0 tx=0).stop 2024-08-19T10:38:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 10 monclient: shutdown 2024-08-19T10:38:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 -- 172.21.15.89:0/33730738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b6006c930 msgr2=0x7f8b60136740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b6006c930 0x7f8b60136740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b500ae040 tx=0x7f8b50019140 comp rx=0 tx=0).stop 2024-08-19T10:38:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 -- 172.21.15.89:0/33730738 shutdown_connections 2024-08-19T10:38:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b4003be00 0x7f8b4003e2c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b6009d340 0x7f8b601371c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b6006d250 0x7f8b60136c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 --2- 172.21.15.89:0/33730738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b6006c930 0x7f8b60136740 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 -- 172.21.15.89:0/33730738 >> 172.21.15.89:0/33730738 conn(0x7f8b600a8f60 msgr2=0x7f8b600733a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 -- 172.21.15.89:0/33730738 shutdown_connections 2024-08-19T10:38:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:43.202+0000 7f8b65ff5640 1 -- 172.21.15.89:0/33730738 wait complete. 2024-08-19T10:38:46.221 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:38:46.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient: get_monmap_and_config 2024-08-19T10:38:46.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient: build_initial_monmap 2024-08-19T10:38:46.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient: monmap: 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:46.354342+0000 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:46.354342+0000 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:46.353 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:46.354 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:46.354 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:46.354 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:46.354 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:46.354 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 1 Processor -- start 2024-08-19T10:38:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 1 -- start start 2024-08-19T10:38:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient: init 2024-08-19T10:38:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient: _add_conns ranks=[6,3,4,1,0,10,2,7,8,9,5] 2024-08-19T10:38:46.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient(hunting): picked mon.noname-d con 0x7f6ab00a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient(hunting): picked mon.noname-g con 0x7f6ab00a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient(hunting): picked mon.noname-c con 0x7f6ab01395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab00a7650 con 0x7f6ab00a6d30 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6ab00a8710 con 0x7f6ab01395b0 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6ab00a5a40 con 0x7f6ab00a5270 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:46.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6ab4962640 10 monclient(hunting): authenticate will time out at 976.170978s 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aaed9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:51802/0 (socket says 172.21.15.89:51802) 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 learned_addr learned my addr 172.21.15.89:0/116227872 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aaed9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58894/0 (socket says 172.21.15.89:58894) 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aae59c640 10 monclient(hunting): get_auth_request con 0x7f6ab00a5270 auth_method 0 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aae59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aae59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aaed9d640 10 monclient(hunting): get_auth_request con 0x7f6ab01395b0 auth_method 0 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aaed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:46.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aaed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aaed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 msgr2=0x7f6ab00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 msgr2=0x7f6ab00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 msgr2=0x7f6ab01399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 msgr2=0x7f6ab01399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aadd9b640 10 monclient(hunting): get_auth_request con 0x7f6ab00a6d30 auth_method 0 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aadd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:46.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aadd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:46.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aadd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:46.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 msgr2=0x7f6ab00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:38:46.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 msgr2=0x7f6ab00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:46.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:46.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.350+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:46.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:46.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aae59c640 10 monclient(hunting): get_auth_request con 0x7f6ab00a5270 auth_method 0 2024-08-19T10:38:46.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:46.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aae59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:46.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aae59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:46.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:46.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 msgr2=0x7f6ab00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:38:46.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 msgr2=0x7f6ab00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aaed9d640 10 monclient(hunting): get_auth_request con 0x7f6ab01395b0 auth_method 0 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aaed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aaed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aaed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aadd9b640 10 monclient(hunting): get_auth_request con 0x7f6ab00a6d30 auth_method 0 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aadd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aadd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aadd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:46.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 msgr2=0x7f6ab01399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:38:46.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 msgr2=0x7f6ab01399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:46.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:46.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:38:46.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 msgr2=0x7f6ab00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:38:46.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 msgr2=0x7f6ab00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:46.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:46.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.554+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:38:46.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:46.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aae59c640 10 monclient(hunting): get_auth_request con 0x7f6ab00a5270 auth_method 0 2024-08-19T10:38:46.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aae59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aae59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aaed9d640 10 monclient(hunting): get_auth_request con 0x7f6ab01395b0 auth_method 0 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aaed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aaed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aaed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 msgr2=0x7f6ab00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 msgr2=0x7f6ab00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:38:46.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 msgr2=0x7f6ab01399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 msgr2=0x7f6ab01399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aadd9b640 10 monclient(hunting): get_auth_request con 0x7f6ab00a6d30 auth_method 0 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aadd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aadd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aadd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 msgr2=0x7f6ab00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 msgr2=0x7f6ab00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:46.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:46.958+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:38:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aae59c640 10 monclient(hunting): get_auth_request con 0x7f6ab00a5270 auth_method 0 2024-08-19T10:38:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aae59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aae59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:47.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aaed9d640 10 monclient(hunting): get_auth_request con 0x7f6ab01395b0 auth_method 0 2024-08-19T10:38:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aaed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aaed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aaed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aadd9b640 10 monclient(hunting): get_auth_request con 0x7f6ab00a6d30 auth_method 0 2024-08-19T10:38:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aadd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aadd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aadd9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 msgr2=0x7f6ab00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:38:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 msgr2=0x7f6ab00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:47.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 msgr2=0x7f6ab01399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:38:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 msgr2=0x7f6ab01399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:38:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:38:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 msgr2=0x7f6ab00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:38:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 msgr2=0x7f6ab00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:47.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:47.762+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:38:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 10 monclient(hunting): tick 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 monclient(hunting): continuing hunt 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 msgr2=0x7f6ab00a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ab00a5270 0x7f6ab00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 msgr2=0x7f6ab01399d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab01395b0 0x7f6ab01399d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 msgr2=0x7f6ab00a7110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab00a6d30 0x7f6ab00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 10 monclient: _add_conns ranks=[5,10,7,1,2,9,8,0] 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 10 monclient(hunting): picked mon.noname-h con 0x7f6a9c003830 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 --2- 172.21.15.89:0/116227872 >> v2:172.21.15.190:3300/0 conn(0x7f6a9c006220 0x7f6a9c0086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 10 monclient(hunting): picked mon.noname-f con 0x7f6a9c006220 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6a9c008d10 0x7f6a9c00b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 10 monclient(hunting): picked mon.noname-i con 0x7f6a9c008d10 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 -- 172.21.15.89:0/116227872 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab00a7650 con 0x7f6a9c006220 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 -- 172.21.15.89:0/116227872 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6ab00a8710 con 0x7f6a9c003830 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 1 -- 172.21.15.89:0/116227872 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6ab00a5a40 con 0x7f6a9c008d10 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aa7fff640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6a9c008d10 msgr2=0x7f6a9c00b1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> v2:172.21.15.190:3300/0 conn(0x7f6a9c006220 msgr2=0x7f6a9c0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:38:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> v2:172.21.15.190:3300/0 conn(0x7f6a9c006220 0x7f6a9c0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6a9c008d10 0x7f6a9c00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aae59c640 10 monclient(hunting): get_auth_request con 0x7f6a9c003830 auth_method 0 2024-08-19T10:38:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aae59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.350+0000 7f6aae59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.354+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 msgr2=0x7f6a9c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:38:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.354+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 msgr2=0x7f6a9c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.354+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.354+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.554+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6a9c008d10 msgr2=0x7f6a9c00b1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:38:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.554+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> v2:172.21.15.190:3300/0 conn(0x7f6a9c006220 msgr2=0x7f6a9c0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:38:49.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.554+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> v2:172.21.15.190:3300/0 conn(0x7f6a9c006220 0x7f6a9c0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:38:49.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.554+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6a9c008d10 0x7f6a9c00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:38:49.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.558+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:49.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.558+0000 7f6aae59c640 10 monclient(hunting): get_auth_request con 0x7f6a9c003830 auth_method 0 2024-08-19T10:38:49.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.558+0000 7f6aae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:49.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.558+0000 7f6aae59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:49.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.558+0000 7f6aae59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:49.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.558+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 msgr2=0x7f6a9c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:38:49.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.558+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 msgr2=0x7f6a9c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:49.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.558+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:49.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.558+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:38:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.958+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6a9c008d10 msgr2=0x7f6a9c00b1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:38:49.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.958+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> v2:172.21.15.190:3300/0 conn(0x7f6a9c006220 msgr2=0x7f6a9c0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:38:49.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.958+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> v2:172.21.15.190:3300/0 conn(0x7f6a9c006220 0x7f6a9c0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:38:49.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.958+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6a9c008d10 0x7f6a9c00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:38:49.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.962+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:49.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.962+0000 7f6aae59c640 10 monclient(hunting): get_auth_request con 0x7f6a9c003830 auth_method 0 2024-08-19T10:38:49.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.962+0000 7f6aae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:49.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.962+0000 7f6aae59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:49.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.962+0000 7f6aae59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:49.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.962+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 msgr2=0x7f6a9c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:38:49.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.962+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 msgr2=0x7f6a9c005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:38:49.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.962+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:49.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:49.962+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:38:50.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.761+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6a9c008d10 msgr2=0x7f6a9c00b1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:38:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.761+0000 7f6aadd9b640 1 -- 172.21.15.89:0/116227872 >> v2:172.21.15.190:3300/0 conn(0x7f6a9c006220 msgr2=0x7f6a9c0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:38:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.761+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/116227872 >> v2:172.21.15.190:3300/0 conn(0x7f6a9c006220 0x7f6a9c0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:38:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.761+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6a9c008d10 0x7f6a9c00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:38:50.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:50.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 10 monclient(hunting): get_auth_request con 0x7f6a9c003830 auth_method 0 2024-08-19T10:38:50.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:50.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 10 monclient(hunting): handle_auth_done global_id 4397 payload 274 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6a9c008d10 msgr2=0x7f6a9c00b1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6a9c008d10 0x7f6a9c00b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> v2:172.21.15.190:3300/0 conn(0x7f6a9c006220 msgr2=0x7f6a9c0086b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> v2:172.21.15.190:3300/0 conn(0x7f6a9c006220 0x7f6a9c0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 10 monclient: _finish_hunting 0 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 1 monclient: found mon.noname-h 2024-08-19T10:38:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aaed9d640 1 -- 172.21.15.89:0/116227872 reap_dead start 2024-08-19T10:38:50.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:50.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:38:50.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.765+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab00a81d0 con 0x7f6a9c003830 2024-08-19T10:38:50.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aae59c640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 msgr2=0x7f6a9c005c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:50.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aae59c640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c003680 tx=0x7f6a9801dd60 comp rx=0 tx=0).ready entity=mon.5 client_cookie=584a04e365e7356c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 1 -- 172.21.15.89:0/116227872 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a980ac070 con 0x7f6a9c003830 2024-08-19T10:38:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:38:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 10 monclient: dump: 2024-08-19T10:38:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 10 monclient: _finish_auth 0 2024-08-19T10:38:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 10 monclient: _check_auth_tickets 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 1 -- 172.21.15.89:0/116227872 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a980aa070 con 0x7f6a9c003830 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 5 monclient: authenticate success, global_id 4397 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 4 set_mon_vals no callback set 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 1 -- 172.21.15.89:0/116227872 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a9800f040 con 0x7f6a9c003830 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6aad59a640 10 monclient: dump: 2024-08-19T10:38:50.779 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:50.780 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:50.781 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:50.781 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:50.781 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:50.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 10 monclient: shutdown 2024-08-19T10:38:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 1 -- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 msgr2=0x7f6a9c005c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c003680 tx=0x7f6a9801dd60 comp rx=0 tx=0).stop 2024-08-19T10:38:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 1 -- 172.21.15.89:0/116227872 shutdown_connections 2024-08-19T10:38:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 1 --2- 172.21.15.89:0/116227872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6a9c003830 0x7f6a9c005c40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 1 -- 172.21.15.89:0/116227872 >> 172.21.15.89:0/116227872 conn(0x7f6ab0069a80 msgr2=0x7f6ab00abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 1 -- 172.21.15.89:0/116227872 shutdown_connections 2024-08-19T10:38:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.773+0000 7f6ab4962640 1 -- 172.21.15.89:0/116227872 wait complete. 2024-08-19T10:38:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient: build_initial_monmap 2024-08-19T10:38:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient: monmap: 2024-08-19T10:38:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 1 Processor -- start 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 1 -- start start 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient: init 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient: _add_conns ranks=[4,8,3,6,0,7,1,5,2] 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a9c003830 0x7f6ab01d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient(hunting): picked mon.noname-c con 0x7f6a9c003830 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab01d2440 0x7f6ab0126f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient(hunting): picked mon.noname-e con 0x7f6ab01d2440 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab01d2840 0x7f6ab0127470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient(hunting): picked mon.noname-g con 0x7f6ab01d2840 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab013a320 con 0x7f6ab01d2840 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6ab013a1a0 con 0x7f6a9c003830 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6ab013c9f0 con 0x7f6ab01d2440 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6ab4962640 10 monclient(hunting): authenticate will time out at 980.597899s 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aadd9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab01d2440 0x7f6ab0126f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aadd9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab01d2440 0x7f6ab0126f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:49054/0 (socket says 172.21.15.89:49054) 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a9c003830 0x7f6ab01d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aadd9b640 1 -- 172.21.15.89:0/1595511598 learned_addr learned my addr 172.21.15.89:0/1595511598 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aadd9b640 10 monclient(hunting): get_auth_request con 0x7f6ab01d2440 auth_method 0 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aadd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:50.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aadd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aadd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 10 monclient(hunting): get_auth_request con 0x7f6a9c003830 auth_method 0 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aaed9d640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab01d2840 0x7f6ab0127470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aadd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aadd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aadd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aaed9d640 10 monclient(hunting): get_auth_request con 0x7f6ab01d2840 auth_method 0 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aaed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aaed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:50.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aaed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aaed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aaed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aaed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 10 monclient(hunting): handle_auth_done global_id 4486 payload 824 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 1 -- 172.21.15.89:0/1595511598 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab01d2440 msgr2=0x7f6ab0126f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab01d2440 0x7f6ab0126f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 1 -- 172.21.15.89:0/1595511598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab01d2840 msgr2=0x7f6ab0127470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab01d2840 0x7f6ab0127470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:50.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 10 monclient: _finish_hunting 0 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 1 monclient: found mon.noname-c 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aae59c640 1 -- 172.21.15.89:0/1595511598 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab0139170 con 0x7f6a9c003830 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.777+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab01d2440 0x7f6ab0126f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aae59c640 1 -- 172.21.15.89:0/1595511598 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a9c003830 msgr2=0x7f6ab01d1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aae59c640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a9c003830 0x7f6ab01d1f00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00b910 tx=0x7f6a98008a40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 1 -- 172.21.15.89:0/1595511598 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a980ac050 con 0x7f6a9c003830 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 10 monclient: dump: 2024-08-19T10:38:50.792 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:50.793 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 1 -- 172.21.15.89:0/1595511598 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a98004410 con 0x7f6a9c003830 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 1 -- 172.21.15.89:0/1595511598 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a980a6d10 con 0x7f6a9c003830 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 10 monclient: dump: 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:50.794 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:50.795 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6ab4962640 5 monclient: authenticate success, global_id 4486 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6ab4962640 10 monclient: _renew_subs 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aacd99640 4 set_mon_vals no callback set 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6ab4962640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6ab4962640 1 -- 172.21.15.89:0/1595511598 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab0128860 con 0x7f6a9c003830 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6ab4962640 10 monclient: _renew_subs 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6ab4962640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6ab4962640 1 -- 172.21.15.89:0/1595511598 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab01293c0 con 0x7f6a9c003830 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6ab4962640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6ab4962640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6ab4962640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:50.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6ab4962640 1 -- 172.21.15.89:0/1595511598 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab013cb70 con 0x7f6a9c003830 2024-08-19T10:38:50.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 1 -- 172.21.15.89:0/1595511598 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6a980a68a0 con 0x7f6a9c003830 2024-08-19T10:38:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9c03cf50 0x7f6a9c03f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aa6ffd640 1 -- 172.21.15.89:0/1595511598 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f6a980b7030 con 0x7f6a9c003830 2024-08-19T10:38:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9c03cf50 0x7f6a9c03f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.781+0000 7f6aadd9b640 10 monclient: get_auth_request con 0x7f6a9c03cf50 auth_method 0 2024-08-19T10:38:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.785+0000 7f6aadd9b640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9c03cf50 0x7f6a9c03f410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0128ed0 tx=0x7f6aa000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.789+0000 7f6aa6ffd640 1 -- 172.21.15.89:0/1595511598 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6a9800a930 con 0x7f6a9c003830 2024-08-19T10:38:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.789+0000 7f6aa6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:50.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:50.789+0000 7f6aa6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.025+0000 7f6ab4962640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:38:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.025+0000 7f6ab4962640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:38:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.025+0000 7f6ab4962640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:51.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.025+0000 7f6ab4962640 1 -- 172.21.15.89:0/1595511598 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6ab0064590 con 0x7f6a9c003830 2024-08-19T10:38:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.025+0000 7f6aa6ffd640 1 -- 172.21.15.89:0/1595511598 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3180 (secure 0 0 0) 0x7f6a980bada0 con 0x7f6a9c003830 2024-08-19T10:38:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.025+0000 7f6aa6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:38:51.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.025+0000 7f6aa6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:51.031 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":10,"quorum":[0,2,3,4,5,6,8],"quorum_names":["a","b","g","c","h","d","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:38:51.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 -- 172.21.15.89:0/1595511598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9c03cf50 msgr2=0x7f6a9c03f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9c03cf50 0x7f6a9c03f410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0128ed0 tx=0x7f6aa000b040 comp rx=0 tx=0).stop 2024-08-19T10:38:51.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 10 monclient: shutdown 2024-08-19T10:38:51.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:51.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 -- 172.21.15.89:0/1595511598 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a9c003830 msgr2=0x7f6ab01d1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a9c003830 0x7f6ab01d1f00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00b910 tx=0x7f6a98008a40 comp rx=0 tx=0).stop 2024-08-19T10:38:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 -- 172.21.15.89:0/1595511598 shutdown_connections 2024-08-19T10:38:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ab01d2840 0x7f6ab0127470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9c03cf50 0x7f6a9c03f410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab01d2440 0x7f6ab0126f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 --2- 172.21.15.89:0/1595511598 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a9c003830 0x7f6ab01d1f00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 -- 172.21.15.89:0/1595511598 >> 172.21.15.89:0/1595511598 conn(0x7f6ab0069a80 msgr2=0x7f6ab009ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 -- 172.21.15.89:0/1595511598 shutdown_connections 2024-08-19T10:38:51.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.029+0000 7f6ab4962640 1 -- 172.21.15.89:0/1595511598 wait complete. 2024-08-19T10:38:51.049 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-08-19T10:38:51.049 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-08-19T10:38:51.050 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:38:51.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient: get_monmap_and_config 2024-08-19T10:38:51.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient: build_initial_monmap 2024-08-19T10:38:51.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient: monmap: 2024-08-19T10:38:51.171 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:51.173064+0000 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:51.173064+0000 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:51.172 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:51.173 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:51.173 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:51.173 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:51.173 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 1 Processor -- start 2024-08-19T10:38:51.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 1 -- start start 2024-08-19T10:38:51.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient: init 2024-08-19T10:38:51.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:51.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient: _add_conns ranks=[9,2,10,8,4,0,7,5,1,6,3] 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa5b00a6860 0x7fa5b00a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient(hunting): picked mon.noname-a con 0x7fa5b00a6860 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b00a7180 0x7fa5b00a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient(hunting): picked mon.noname-b con 0x7fa5b00a7180 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa5b0139620 0x7fa5b0139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient(hunting): picked mon.noname-f con 0x7fa5b0139620 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fa5b00a8190 con 0x7fa5b00a6860 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa5b00a33d0 con 0x7fa5b0139620 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b00a2ec0 con 0x7fa5b00a7180 2024-08-19T10:38:51.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:51.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b6ac2640 10 monclient(hunting): authenticate will time out at 980.989611s 2024-08-19T10:38:51.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b485e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa5b00a6860 0x7fa5b00a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b00a7180 0x7fa5b00a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b00a7180 0x7fa5b00a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:57544/0 (socket says 172.21.15.89:57544) 2024-08-19T10:38:51.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b485e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa5b00a6860 0x7fa5b00a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60162/0 (socket says 172.21.15.89:60162) 2024-08-19T10:38:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 1 -- 172.21.15.89:0/1507940028 learned_addr learned my addr 172.21.15.89:0/1507940028 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 10 monclient(hunting): get_auth_request con 0x7fa5b00a7180 auth_method 0 2024-08-19T10:38:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b485e640 10 monclient(hunting): get_auth_request con 0x7fa5b00a6860 auth_method 0 2024-08-19T10:38:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b485e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b485e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b485e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b505f640 1 -- 172.21.15.89:0/1507940028 >> v2:172.21.15.190:3300/0 conn(0x7fa5b0139620 msgr2=0x7fa5b0139ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b505f640 1 --2- 172.21.15.89:0/1507940028 >> v2:172.21.15.190:3300/0 conn(0x7fa5b0139620 0x7fa5b0139ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b485e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b485e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b485e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 10 monclient(hunting): handle_auth_done global_id 4385 payload 274 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 1 -- 172.21.15.89:0/1507940028 >> v2:172.21.15.190:3300/0 conn(0x7fa5b0139620 msgr2=0x7fa5b0139ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 1 --2- 172.21.15.89:0/1507940028 >> v2:172.21.15.190:3300/0 conn(0x7fa5b0139620 0x7fa5b0139ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 1 -- 172.21.15.89:0/1507940028 >> v2:172.21.15.89:3300/0 conn(0x7fa5b00a6860 msgr2=0x7fa5b00a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 1 --2- 172.21.15.89:0/1507940028 >> v2:172.21.15.89:3300/0 conn(0x7fa5b00a6860 0x7fa5b00a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 10 monclient: _finish_hunting 0 2024-08-19T10:38:51.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 1 monclient: found mon.noname-b 2024-08-19T10:38:51.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:51.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:51.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5affff640 1 -- 172.21.15.89:0/1507940028 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b00a4cd0 con 0x7fa5b00a7180 2024-08-19T10:38:51.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.169+0000 7fa5b485e640 1 --2- 172.21.15.89:0/1507940028 >> v2:172.21.15.89:3300/0 conn(0x7fa5b00a6860 0x7fa5b00a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:51.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5affff640 1 -- 172.21.15.89:0/1507940028 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b00a7180 msgr2=0x7fa5b00a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5affff640 1 --2- 172.21.15.89:0/1507940028 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b00a7180 0x7fa5b00a7560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00a61e0 tx=0x7fa5a001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04661bc585f2c87 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 1 -- 172.21.15.89:0/1507940028 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5a00b0070 con 0x7fa5b00a7180 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 10 monclient: dump: 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.184 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:51.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 1 -- 172.21.15.89:0/1507940028 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a00a4e50 con 0x7fa5b00a7180 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 5 monclient: authenticate success, global_id 4385 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 1 -- 172.21.15.89:0/1507940028 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5a00aac80 con 0x7fa5b00a7180 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5af7fe640 10 monclient: dump: 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.186 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.187 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.187 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.187 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.187 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.187 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.187 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.187 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.187 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.187 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.187 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 4 set_mon_vals no callback set 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 10 monclient: shutdown 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1507940028 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b00a7180 msgr2=0x7fa5b00a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 1 --2- 172.21.15.89:0/1507940028 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b00a7180 0x7fa5b00a7560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00a61e0 tx=0x7fa5a001c400 comp rx=0 tx=0).stop 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1507940028 shutdown_connections 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 1 --2- 172.21.15.89:0/1507940028 >> v2:172.21.15.190:3300/0 conn(0x7fa5b0139620 0x7fa5b0139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 1 --2- 172.21.15.89:0/1507940028 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5b00a7180 0x7fa5b00a7560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 1 --2- 172.21.15.89:0/1507940028 >> v2:172.21.15.89:3300/0 conn(0x7fa5b00a6860 0x7fa5b00a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1507940028 >> 172.21.15.89:0/1507940028 conn(0x7fa5b0074b40 msgr2=0x7fa5b00abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1507940028 shutdown_connections 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.173+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1507940028 wait complete. 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient: build_initial_monmap 2024-08-19T10:38:51.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient: monmap: 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:51.189 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 1 Processor -- start 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 1 -- start start 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient: init 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient: _add_conns ranks=[4,1,5,2,7,8,0,6,3] 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b00a6860 0x7fa5b0131960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient(hunting): picked mon.noname-c con 0x7fa5b00a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b00a7180 0x7fa5b0131ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient(hunting): picked mon.noname-f con 0x7fa5b00a7180 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa5b0139620 0x7fa5b013d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient(hunting): picked mon.noname-h con 0x7fa5b0139620 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b0143210 con 0x7fa5b00a7180 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa5b0143090 con 0x7fa5b00a6860 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa5b0143390 con 0x7fa5b0139620 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b6ac2640 10 monclient(hunting): authenticate will time out at 980.997472s 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5affff640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b00a7180 msgr2=0x7fa5b0131ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5affff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b00a7180 0x7fa5b0131ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:51.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b00a6860 0x7fa5b0131960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b00a6860 0x7fa5b0131960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58942/0 (socket says 172.21.15.89:58942) 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 1 -- 172.21.15.89:0/1230758803 learned_addr learned my addr 172.21.15.89:0/1230758803 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 10 monclient(hunting): get_auth_request con 0x7fa5b00a6860 auth_method 0 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b505f640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa5b0139620 0x7fa5b013d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b505f640 10 monclient(hunting): get_auth_request con 0x7fa5b0139620 auth_method 0 2024-08-19T10:38:51.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b505f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b505f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b505f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 10 monclient(hunting): handle_auth_done global_id 4495 payload 824 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 1 -- 172.21.15.89:0/1230758803 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa5b0139620 msgr2=0x7fa5b013d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa5b0139620 0x7fa5b013d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 1 -- 172.21.15.89:0/1230758803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b00a7180 msgr2=0x7fa5b0131ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5b00a7180 0x7fa5b0131ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 10 monclient: _finish_hunting 0 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 1 monclient: found mon.noname-c 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.177+0000 7fa5b485e640 1 -- 172.21.15.89:0/1230758803 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b009d5f0 con 0x7fa5b00a6860 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b485e640 1 -- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b00a6860 msgr2=0x7fa5b0131960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b485e640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b00a6860 0x7fa5b0131960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400ec10 tx=0x7fa5a400c520 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:38:51.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5a4017070 con 0x7fa5b00a6860 2024-08-19T10:38:51.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:51.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 10 monclient: dump: 2024-08-19T10:38:51.197 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.197 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.197 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.197 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:51.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:51.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:51.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a400ce40 con 0x7fa5b00a6860 2024-08-19T10:38:51.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b6ac2640 5 monclient: authenticate success, global_id 4495 2024-08-19T10:38:51.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b6ac2640 10 monclient: _renew_subs 2024-08-19T10:38:51.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b6ac2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:51.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1230758803 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b00ac2b0 con 0x7fa5b00a6860 2024-08-19T10:38:51.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5a4013d50 con 0x7fa5b00a6860 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5aeffd640 4 set_mon_vals no callback set 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 10 monclient: dump: 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.200 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.201 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.201 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.201 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.201 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.201 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.201 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.201 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.201 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b6ac2640 10 monclient: _renew_subs 2024-08-19T10:38:51.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b6ac2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:51.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1230758803 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b0135870 con 0x7fa5b00a6860 2024-08-19T10:38:51.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b6ac2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:51.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b6ac2640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013dbb0 0x7fa5b013df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b6ac2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1230758803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b00a7560 con 0x7fa5b013dbb0 2024-08-19T10:38:51.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa5a401a070 con 0x7fa5b00a6860 2024-08-19T10:38:51.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5affff640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013dbb0 0x7fa5b013df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5affff640 10 monclient: get_auth_request con 0x7fa5b013dbb0 auth_method 0 2024-08-19T10:38:51.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa59c03be30 0x7fa59c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5affff640 10 monclient(hunting): handle_auth_done global_id 4554 payload 824 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5affff640 1 -- 172.21.15.89:0/1230758803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5b0064590 con 0x7fa5b013dbb0 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fa5a400f040 con 0x7fa5b00a6860 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.181+0000 7fa5b505f640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa59c03be30 0x7fa59c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5b505f640 10 monclient: get_auth_request con 0x7fa59c03be30 auth_method 0 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5affff640 1 -- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013dbb0 msgr2=0x7fa5b013df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5affff640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013dbb0 0x7fa5b013df90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00ad000 tx=0x7fa5a001ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5a00b0040 con 0x7fa5b013dbb0 2024-08-19T10:38:51.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5ad7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5ad7fa640 10 monclient: dump: 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.204 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa5a00a4e50 con 0x7fa5b013dbb0 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5ad7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5ad7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013dbb0 msgr2=0x7fa5b013df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5ad7fa640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013dbb0 0x7fa5b013df90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00ad000 tx=0x7fa5a001ac40 comp rx=0 tx=0).stop 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013dbb0 msgr2=0x7fa5b013df90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:51.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5ad7fa640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013dbb0 0x7fa5b013df90 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00ad000 tx=0x7fa5a001ac40 comp rx=0 tx=0).stop 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.185+0000 7fa5b505f640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa59c03be30 0x7fa59c03e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5b013fbc0 tx=0x7fa5a80093f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5b6ac2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5b6ac2640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013e4d0 0x7fa5b01327b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5b6ac2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1230758803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b013f120 con 0x7fa5b013e4d0 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013e4d0 0x7fa5b01327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 10 monclient: get_auth_request con 0x7fa5b013e4d0 auth_method 0 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 10 monclient(hunting): handle_auth_done global_id 4563 payload 824 2024-08-19T10:38:51.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 1 -- 172.21.15.89:0/1230758803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa5b013fc80 con 0x7fa5b013e4d0 2024-08-19T10:38:51.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 1 -- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013e4d0 msgr2=0x7fa5b01327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5affff640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013e4d0 0x7fa5b01327b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa5a001aab0 tx=0x7fa5a00191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5a00b0040 con 0x7fa5b013e4d0 2024-08-19T10:38:51.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5ad7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:51.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5ad7fa640 10 monclient: dump: 2024-08-19T10:38:51.209 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.209 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.210 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.211 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.211 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9246 (secure 0 0 0) 0x7fa5a00b43f0 con 0x7fa5b013e4d0 2024-08-19T10:38:51.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5ad7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:51.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5ad7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:51.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013e4d0 msgr2=0x7fa5b01327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5ad7fa640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013e4d0 0x7fa5b01327b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa5a001aab0 tx=0x7fa5a00191a0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5ad7fa640 1 -- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013e4d0 msgr2=0x7fa5b01327b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:51.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5ad7fa640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5b013e4d0 0x7fa5b01327b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 10, 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 61007, 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:51.212 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.213 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.215 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.215 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.215 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.215 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.215 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.215 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.215 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.215 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.215 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:51.216 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.217 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:51.218 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:51.219 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:51.219 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.220 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.220 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:51.220 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:51.220 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.220 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.220 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.220 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:51.221 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.222 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.222 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:51.222 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:51.222 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.222 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.222 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.222 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:51.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:51.223 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:51.223 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:51.224 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.225 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:51.229 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:51.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.231 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:51.231 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.231 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.231 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:51.231 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:51.231 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:51.232 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.233 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.234 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.234 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:51.234 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:51.234 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:51.234 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.235 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.236 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:51.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.237 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.237 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.237 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:51.237 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.237 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.237 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:51.237 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:51.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1230758803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa59c03be30 msgr2=0x7fa59c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5b6ac2640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa59c03be30 0x7fa59c03e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5b013fbc0 tx=0x7fa5a80093f0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5b6ac2640 10 monclient: shutdown 2024-08-19T10:38:51.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5b6ac2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:51.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b00a6860 msgr2=0x7fa5b0131960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5b6ac2640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b00a6860 0x7fa5b0131960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400ec10 tx=0x7fa5a400c520 comp rx=0 tx=0).stop 2024-08-19T10:38:51.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.193+0000 7fa5b505f640 1 -- 172.21.15.89:0/1230758803 reap_dead start 2024-08-19T10:38:51.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.197+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1230758803 shutdown_connections 2024-08-19T10:38:51.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.197+0000 7fa5b6ac2640 1 --2- 172.21.15.89:0/1230758803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5b00a6860 0x7fa5b0131960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.197+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1230758803 >> 172.21.15.89:0/1230758803 conn(0x7fa5b0074b40 msgr2=0x7fa5b00a9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:51.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.197+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1230758803 shutdown_connections 2024-08-19T10:38:51.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.197+0000 7fa5b6ac2640 1 -- 172.21.15.89:0/1230758803 wait complete. 2024-08-19T10:38:51.240 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:38:51.240 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-08-19T10:38:51.240 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:38:51.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient: get_monmap_and_config 2024-08-19T10:38:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient: build_initial_monmap 2024-08-19T10:38:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient: monmap: 2024-08-19T10:38:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:51.353404+0000 2024-08-19T10:38:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:51.353404+0000 2024-08-19T10:38:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.352 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:51.353 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 1 Processor -- start 2024-08-19T10:38:51.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 1 -- start start 2024-08-19T10:38:51.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient: init 2024-08-19T10:38:51.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:51.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient: _add_conns ranks=[4,1,8,0,6,3,2,10,5,7,9] 2024-08-19T10:38:51.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 0x7f3cb80a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient(hunting): picked mon.noname-c con 0x7f3cb80a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3cb80a6d30 addr v1:172.21.15.190:6789/0 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3cb80a7110 0x7f3cb8139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient(hunting): picked mon.noname-e con 0x7f3cb80a7110 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cb80a8710 con 0x7f3cb80a6d30 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3cb80a5a40 con 0x7f3cb80a5270 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3cb80a81d0 con 0x7f3cb80a7110 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cbe138640 10 monclient(hunting): authenticate will time out at 981.169985s 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb6ffd640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f3cb80a6d30 legacy=0x7f3cb81395b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 0x7f3cb80a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3cb80a7110 0x7f3cb8139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3cb80a7110 0x7f3cb8139cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:46990/0 (socket says 172.21.15.89:46990) 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 0x7f3cb80a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58222/0 (socket says 172.21.15.89:58222) 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb7fff640 1 -- 172.21.15.89:0/3214240913 learned_addr learned my addr 172.21.15.89:0/3214240913 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb7fff640 10 monclient(hunting): get_auth_request con 0x7f3cb80a7110 auth_method 0 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 10 monclient(hunting): get_auth_request con 0x7f3cb80a5270 auth_method 0 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 10 monclient(hunting): handle_auth_done global_id 4504 payload 274 2024-08-19T10:38:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 1 -- 172.21.15.89:0/3214240913 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3cb80a7110 msgr2=0x7f3cb8139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 1 --2- 172.21.15.89:0/3214240913 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3cb80a7110 0x7f3cb8139cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 1 -- 172.21.15.89:0/3214240913 >> v1:172.21.15.190:6789/0 conn(0x7f3cb80a6d30 legacy=0x7f3cb81395b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:38:51.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 1 monclient: found mon.noname-c 2024-08-19T10:38:51.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:51.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:51.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.349+0000 7f3cb77fe640 1 -- 172.21.15.89:0/3214240913 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb80a4790 con 0x7f3cb80a5270 2024-08-19T10:38:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 1 -- 172.21.15.89:0/3214240913 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 msgr2=0x7f3cb80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 1 --2- 172.21.15.89:0/3214240913 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 0x7f3cb80a67f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0097f0 tx=0x7f3cac01c190 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f9228823afa99818 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 1 -- 172.21.15.89:0/3214240913 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cac0b0070 con 0x7f3cb80a5270 2024-08-19T10:38:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 10 monclient: dump: 2024-08-19T10:38:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.367 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.368 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 10 monclient: _finish_auth 0 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 1 -- 172.21.15.89:0/3214240913 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cac0aa810 con 0x7f3cb80a5270 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 1 -- 172.21.15.89:0/3214240913 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cac0a9300 con 0x7f3cb80a5270 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 5 monclient: authenticate success, global_id 4504 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 4 set_mon_vals no callback set 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb67fc640 10 monclient: dump: 2024-08-19T10:38:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.370 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient: shutdown 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 -- 172.21.15.89:0/3214240913 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 msgr2=0x7f3cb80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 --2- 172.21.15.89:0/3214240913 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 0x7f3cb80a67f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0097f0 tx=0x7f3cac01c190 comp rx=0 tx=0).stop 2024-08-19T10:38:51.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 -- 172.21.15.89:0/3214240913 shutdown_connections 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 --2- 172.21.15.89:0/3214240913 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3cb80a7110 0x7f3cb8139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 --2- 172.21.15.89:0/3214240913 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 0x7f3cb80a67f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 -- 172.21.15.89:0/3214240913 >> 172.21.15.89:0/3214240913 conn(0x7f3cb8069a80 msgr2=0x7f3cb80abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 -- 172.21.15.89:0/3214240913 shutdown_connections 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 -- 172.21.15.89:0/3214240913 wait complete. 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient: build_initial_monmap 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient: monmap: 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.372 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:51.373 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:51.373 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:51.373 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:51.373 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:51.373 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:51.373 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:51.373 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:51.373 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:51.373 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 Processor -- start 2024-08-19T10:38:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 -- start start 2024-08-19T10:38:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient: init 2024-08-19T10:38:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient: _add_conns ranks=[4,0,5,8,7,2,3,1,6] 2024-08-19T10:38:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 0x7f3cb8138f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient(hunting): picked mon.noname-c con 0x7f3cb80a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cb80a6d30 0x7f3cb8139490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient(hunting): picked mon.noname-a con 0x7f3cb80a6d30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3cb80a7110 0x7f3cb813e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient(hunting): picked mon.noname-h con 0x7f3cb80a7110 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:51.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb80a57d0 con 0x7f3cb80a6d30 2024-08-19T10:38:51.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3cb80a5650 con 0x7f3cb80a5270 2024-08-19T10:38:51.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f3cb813a200 con 0x7f3cb80a7110 2024-08-19T10:38:51.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:51.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cbe138640 10 monclient(hunting): authenticate will time out at 981.178759s 2024-08-19T10:38:51.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 0x7f3cb8138f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 0x7f3cb8138f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58224/0 (socket says 172.21.15.89:58224) 2024-08-19T10:38:51.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb6ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cb80a6d30 0x7f3cb8139490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 1 -- 172.21.15.89:0/2413276717 learned_addr learned my addr 172.21.15.89:0/2413276717 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 10 monclient(hunting): get_auth_request con 0x7f3cb80a5270 auth_method 0 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb6ffd640 10 monclient(hunting): get_auth_request con 0x7f3cb80a6d30 auth_method 0 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb7fff640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3cb80a7110 0x7f3cb813e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb7fff640 10 monclient(hunting): get_auth_request con 0x7f3cb80a7110 auth_method 0 2024-08-19T10:38:51.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 10 monclient(hunting): handle_auth_done global_id 4513 payload 824 2024-08-19T10:38:51.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 1 -- 172.21.15.89:0/2413276717 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3cb80a7110 msgr2=0x7f3cb813e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3cb80a7110 0x7f3cb813e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 1 -- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cb80a6d30 msgr2=0x7f3cb8139490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3cb80a6d30 0x7f3cb8139490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 1 monclient: found mon.noname-c 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.357+0000 7f3cb77fe640 1 -- 172.21.15.89:0/2413276717 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb81234b0 con 0x7f3cb80a5270 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb77fe640 1 -- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 msgr2=0x7f3cb8138f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb77fe640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 0x7f3cb8138f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0ae040 tx=0x7f3cac018f30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cac0b0040 con 0x7f3cb80a5270 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:51.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 10 monclient: dump: 2024-08-19T10:38:51.379 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.379 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.379 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.379 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.379 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.379 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.379 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.379 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.379 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.380 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.380 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.380 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.380 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.380 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.380 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.381 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 10 monclient: _finish_auth 0 2024-08-19T10:38:51.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:51.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:51.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cac0044b0 con 0x7f3cb80a5270 2024-08-19T10:38:51.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:51.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cac003a90 con 0x7f3cb80a5270 2024-08-19T10:38:51.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:51.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 10 monclient: dump: 2024-08-19T10:38:51.382 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.382 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.382 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.382 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.382 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.382 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.382 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.382 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.383 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.383 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.383 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.383 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.383 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.383 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.383 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.383 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cbe138640 5 monclient: authenticate success, global_id 4513 2024-08-19T10:38:51.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cbe138640 10 monclient: _renew_subs 2024-08-19T10:38:51.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cbe138640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:51.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb5ffb640 4 set_mon_vals no callback set 2024-08-19T10:38:51.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cbe138640 1 -- 172.21.15.89:0/2413276717 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb8123790 con 0x7f3cb80a5270 2024-08-19T10:38:51.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cbe138640 10 monclient: _renew_subs 2024-08-19T10:38:51.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cbe138640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:51.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cbe138640 1 -- 172.21.15.89:0/2413276717 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb81401c0 con 0x7f3cb80a5270 2024-08-19T10:38:51.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cbe138640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:51.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cbe138640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813e9b0 0x7f3cb813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cbe138640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cbe138640 1 -- 172.21.15.89:0/2413276717 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb80a1fc0 con 0x7f3cb813e9b0 2024-08-19T10:38:51.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb6ffd640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813e9b0 0x7f3cb813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb6ffd640 10 monclient: get_auth_request con 0x7f3cb813e9b0 auth_method 0 2024-08-19T10:38:51.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3cac003c50 con 0x7f3cb80a5270 2024-08-19T10:38:51.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb6ffd640 10 monclient(hunting): handle_auth_done global_id 4394 payload 824 2024-08-19T10:38:51.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3cb6ffd640 1 -- 172.21.15.89:0/2413276717 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3cb80a21f0 con 0x7f3cb813e9b0 2024-08-19T10:38:51.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3ca403be00 0x7f3ca403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.361+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f3cac0e83b0 con 0x7f3cb80a5270 2024-08-19T10:38:51.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3cb7fff640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3ca403be00 0x7f3ca403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3cb7fff640 10 monclient: get_auth_request con 0x7f3ca403be00 auth_method 0 2024-08-19T10:38:51.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3cb6ffd640 1 -- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813e9b0 msgr2=0x7f3cb813ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3cb6ffd640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813e9b0 0x7f3cb813ed90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8003940 tx=0x7f3ca8009ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3ca8016070 con 0x7f3cb813e9b0 2024-08-19T10:38:51.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3c9ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3c9ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3c9ffff640 10 monclient: dump: 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.390 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3ca80128f0 con 0x7f3cb813e9b0 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3c9ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3c9ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813e9b0 msgr2=0x7f3cb813ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3c9ffff640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813e9b0 0x7f3cb813ed90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8003940 tx=0x7f3ca8009ab0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813e9b0 msgr2=0x7f3cb813ed90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.365+0000 7f3c9ffff640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813e9b0 0x7f3cb813ed90 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8003940 tx=0x7f3ca8009ab0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.369+0000 7f3cb7fff640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3ca403be00 0x7f3ca403e2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009d80 tx=0x7f3cb000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.369+0000 7f3cbe138640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.369+0000 7f3cbe138640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813f2d0 0x7f3cb80a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.369+0000 7f3cbe138640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.369+0000 7f3cbe138640 1 -- 172.21.15.89:0/2413276717 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb80a2510 con 0x7f3cb813f2d0 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.369+0000 7f3cb6ffd640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813f2d0 0x7f3cb80a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.369+0000 7f3cb6ffd640 10 monclient: get_auth_request con 0x7f3cb813f2d0 auth_method 0 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.369+0000 7f3cb6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.369+0000 7f3cb6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.369+0000 7f3cb6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cb6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cb6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cb6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cb6ffd640 10 monclient(hunting): handle_auth_done global_id 4403 payload 824 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cb6ffd640 1 -- 172.21.15.89:0/2413276717 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3cb80a67f0 con 0x7f3cb813f2d0 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cb6ffd640 1 -- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813f2d0 msgr2=0x7f3cb80a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cb6ffd640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813f2d0 0x7f3cb80a05d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8003940 tx=0x7f3ca8005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3ca8016040 con 0x7f3cb813f2d0 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3c9ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3c9ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3c9ffff640 10 monclient: dump: 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.393 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9411 (secure 0 0 0) 0x7f3ca8005e20 con 0x7f3cb813f2d0 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3c9ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:51.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3c9ffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:51.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813f2d0 msgr2=0x7f3cb80a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3c9ffff640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813f2d0 0x7f3cb80a05d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8003940 tx=0x7f3ca8005990 comp rx=0 tx=0).stop 2024-08-19T10:38:51.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3c9ffff640 1 -- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813f2d0 msgr2=0x7f3cb80a05d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:51.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3c9ffff640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3cb813f2d0 0x7f3cb80a05d0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8003940 tx=0x7f3ca8005990 comp rx=0 tx=0).stop 2024-08-19T10:38:51.395 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:51.395 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:51.395 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:51.395 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:51.395 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 10, 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 61203, 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:38:51.396 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.397 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.398 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:51.399 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.400 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:51.401 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.402 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:51.403 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:51.403 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.403 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.403 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.403 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:51.403 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:51.403 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.404 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:51.405 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:51.405 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.405 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.405 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.405 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:51.405 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:51.406 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:51.406 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:51.407 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.407 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.407 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:51.407 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:51.407 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.407 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.407 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:51.408 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:51.409 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.410 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:51.414 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:51.415 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.415 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.415 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.415 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:51.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:51.415 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:51.416 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:51.417 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.417 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.417 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.417 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:51.417 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:51.417 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.417 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:51.418 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:51.418 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.418 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.418 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.419 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:51.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.421 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.421 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.421 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:38:51.421 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.421 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.421 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:51.421 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:51.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cbe138640 1 -- 172.21.15.89:0/2413276717 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3ca403be00 msgr2=0x7f3ca403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cbe138640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3ca403be00 0x7f3ca403e2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009d80 tx=0x7f3cb000b040 comp rx=0 tx=0).stop 2024-08-19T10:38:51.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cbe138640 10 monclient: shutdown 2024-08-19T10:38:51.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cbe138640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:51.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cbe138640 1 -- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 msgr2=0x7f3cb8138f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cbe138640 1 --2- 172.21.15.89:0/2413276717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3cb80a5270 0x7f3cb8138f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0ae040 tx=0x7f3cac018f30 comp rx=0 tx=0).stop 2024-08-19T10:38:51.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cb7fff640 1 -- 172.21.15.89:0/2413276717 reap_dead start 2024-08-19T10:38:51.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cb7fff640 1 -- 172.21.15.89:0/2413276717 reap_dead start 2024-08-19T10:38:51.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cbe138640 1 -- 172.21.15.89:0/2413276717 shutdown_connections 2024-08-19T10:38:51.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cbe138640 1 -- 172.21.15.89:0/2413276717 >> 172.21.15.89:0/2413276717 conn(0x7f3cb8069a80 msgr2=0x7f3cb8141230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:51.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cbe138640 1 -- 172.21.15.89:0/2413276717 shutdown_connections 2024-08-19T10:38:51.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.373+0000 7f3cbe138640 1 -- 172.21.15.89:0/2413276717 wait complete. 2024-08-19T10:38:51.423 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:38:51.424 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-08-19T10:38:51.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:38:51.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient: get_monmap_and_config 2024-08-19T10:38:51.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient: build_initial_monmap 2024-08-19T10:38:51.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient: monmap: 2024-08-19T10:38:51.540 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:51.540 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.540 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:51.541462+0000 2024-08-19T10:38:51.540 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:51.541462+0000 2024-08-19T10:38:51.540 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:51.541 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:51.542 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:51.542 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 1 Processor -- start 2024-08-19T10:38:51.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 1 -- start start 2024-08-19T10:38:51.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient: init 2024-08-19T10:38:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient: _add_conns ranks=[10,2,3,8,0,4,6,5,1,9,7] 2024-08-19T10:38:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f20fc0a6860 0x7f20fc0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient(hunting): picked mon.noname-f con 0x7f20fc0a6860 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20fc0a7180 0x7f20fc0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient(hunting): picked mon.noname-b con 0x7f20fc0a7180 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc139620 0x7f20fc139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient(hunting): picked mon.noname-g con 0x7f20fc139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f20fc0a8190 con 0x7f20fc0a6860 2024-08-19T10:38:51.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f20fc0a33d0 con 0x7f20fc0a7180 2024-08-19T10:38:51.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f20fc0a2ec0 con 0x7f20fc139620 2024-08-19T10:38:51.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:51.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f21035ba640 10 monclient(hunting): authenticate will time out at 981.357968s 2024-08-19T10:38:51.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101356640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f20fc0a6860 msgr2=0x7f20fc0a6c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:38:51.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101356640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f20fc0a6860 0x7f20fc0a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:51.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc139620 0x7f20fc139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc139620 0x7f20fc139ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:56010/0 (socket says 172.21.15.89:56010) 2024-08-19T10:38:51.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 1 -- 172.21.15.89:0/208370242 learned_addr learned my addr 172.21.15.89:0/208370242 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:51.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2100b55640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20fc0a7180 0x7f20fc0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 10 monclient(hunting): get_auth_request con 0x7f20fc139620 auth_method 0 2024-08-19T10:38:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2100b55640 10 monclient(hunting): get_auth_request con 0x7f20fc0a7180 auth_method 0 2024-08-19T10:38:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2100b55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2100b55640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2100b55640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 10 monclient(hunting): handle_auth_done global_id 4413 payload 274 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 1 -- 172.21.15.89:0/208370242 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20fc0a7180 msgr2=0x7f20fc0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 1 --2- 172.21.15.89:0/208370242 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20fc0a7180 0x7f20fc0a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 1 -- 172.21.15.89:0/208370242 >> v2:172.21.15.190:3300/0 conn(0x7f20fc0a6860 msgr2=0x7f20fc0a6c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 1 --2- 172.21.15.89:0/208370242 >> v2:172.21.15.190:3300/0 conn(0x7f20fc0a6860 0x7f20fc0a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 10 monclient: _finish_hunting 0 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 1 monclient: found mon.noname-g 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:51.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:38:51.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2101b57640 1 -- 172.21.15.89:0/208370242 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc0a4cd0 con 0x7f20fc139620 2024-08-19T10:38:51.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.537+0000 7f2100b55640 1 --2- 172.21.15.89:0/208370242 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20fc0a7180 0x7f20fc0a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:51.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f2101b57640 1 -- 172.21.15.89:0/208370242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc139620 msgr2=0x7f20fc139ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f2101b57640 1 --2- 172.21.15.89:0/208370242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc139620 0x7f20fc139ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009870 tx=0x7f20ec01c440 comp rx=0 tx=0).ready entity=mon.3 client_cookie=baf750be19d19f8a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 1 -- 172.21.15.89:0/208370242 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20ec0b0070 con 0x7f20fc139620 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 10 monclient: dump: 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.556 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 10 monclient: _finish_auth 0 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 1 -- 172.21.15.89:0/208370242 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ec0a4e50 con 0x7f20fc139620 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:51.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 1 -- 172.21.15.89:0/208370242 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20ec0aac80 con 0x7f20fc139620 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f20f3fff640 10 monclient: dump: 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.558 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 5 monclient: authenticate success, global_id 4413 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 4 set_mon_vals no callback set 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 10 monclient: shutdown 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 1 -- 172.21.15.89:0/208370242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc139620 msgr2=0x7f20fc139ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 1 --2- 172.21.15.89:0/208370242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc139620 0x7f20fc139ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009870 tx=0x7f20ec01c440 comp rx=0 tx=0).stop 2024-08-19T10:38:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 1 -- 172.21.15.89:0/208370242 shutdown_connections 2024-08-19T10:38:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 1 --2- 172.21.15.89:0/208370242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc139620 0x7f20fc139ab0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 1 --2- 172.21.15.89:0/208370242 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20fc0a7180 0x7f20fc0a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 1 --2- 172.21.15.89:0/208370242 >> v2:172.21.15.190:3300/0 conn(0x7f20fc0a6860 0x7f20fc0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 1 -- 172.21.15.89:0/208370242 >> 172.21.15.89:0/208370242 conn(0x7f20fc074b40 msgr2=0x7f20fc0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 1 -- 172.21.15.89:0/208370242 shutdown_connections 2024-08-19T10:38:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.541+0000 7f21035ba640 1 -- 172.21.15.89:0/208370242 wait complete. 2024-08-19T10:38:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient: build_initial_monmap 2024-08-19T10:38:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient: monmap: 2024-08-19T10:38:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:51.560 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:51.561 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 1 Processor -- start 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 1 -- start start 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient: init 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient: _add_conns ranks=[6,3,4,2,7,0,5,1,8] 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20fc0a6860 0x7f20fc1348c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient(hunting): picked mon.noname-d con 0x7f20fc0a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc0a7180 0x7f20fc134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient(hunting): picked mon.noname-g con 0x7f20fc0a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc139620 0x7f20fc09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient(hunting): picked mon.noname-c con 0x7f20fc139620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:51.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f20fc143210 con 0x7f20fc0a7180 2024-08-19T10:38:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f20fc143090 con 0x7f20fc139620 2024-08-19T10:38:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f20fc143390 con 0x7f20fc0a6860 2024-08-19T10:38:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f21035ba640 10 monclient(hunting): authenticate will time out at 981.365126s 2024-08-19T10:38:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20fc0a6860 0x7f20fc1348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101b57640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc139620 0x7f20fc09c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20fc0a6860 0x7f20fc1348c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:40934/0 (socket says 172.21.15.89:40934) 2024-08-19T10:38:51.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 1 -- 172.21.15.89:0/614639511 learned_addr learned my addr 172.21.15.89:0/614639511 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 10 monclient(hunting): get_auth_request con 0x7f20fc0a6860 auth_method 0 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101b57640 10 monclient(hunting): get_auth_request con 0x7f20fc139620 auth_method 0 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101b57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2100b55640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc0a7180 0x7f20fc134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101b57640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101b57640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2100b55640 10 monclient(hunting): get_auth_request con 0x7f20fc0a7180 auth_method 0 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2100b55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2100b55640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2100b55640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101b57640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101b57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101b57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 10 monclient(hunting): handle_auth_done global_id 4362 payload 824 2024-08-19T10:38:51.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc139620 msgr2=0x7f20fc09c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc139620 0x7f20fc09c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc0a7180 msgr2=0x7f20fc134e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc0a7180 0x7f20fc134e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 10 monclient: _finish_hunting 0 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 1 monclient: found mon.noname-d 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 1 -- 172.21.15.89:0/614639511 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc0a03e0 con 0x7f20fc0a6860 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2100b55640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20fc0a7180 0x7f20fc134e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101b57640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc139620 0x7f20fc09c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20fc0a6860 msgr2=0x7f20fc1348c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.545+0000 7f2101356640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20fc0a6860 0x7f20fc1348c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20f400b560 tx=0x7f20f400ba20 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20f4013020 con 0x7f20fc0a6860 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: dump: 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.567 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f4003de0 con 0x7f20fc0a6860 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:51.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20f400f460 con 0x7f20fc0a6860 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: dump: 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.569 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f37fe640 4 set_mon_vals no callback set 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f21035ba640 5 monclient: authenticate success, global_id 4362 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f21035ba640 10 monclient: _renew_subs 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f21035ba640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:51.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f21035ba640 1 -- 172.21.15.89:0/614639511 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20fc140470 con 0x7f20fc0a6860 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f21035ba640 10 monclient: _renew_subs 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f21035ba640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f21035ba640 1 -- 172.21.15.89:0/614639511 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f20fc09fb40 con 0x7f20fc0a6860 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f21035ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f21035ba640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09cf20 0x7f20fc09d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f21035ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f21035ba640 1 -- 172.21.15.89:0/614639511 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f20fc0a7560 con 0x7f20fc09cf20 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09cf20 0x7f20fc09d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 10 monclient: get_auth_request con 0x7f20fc09cf20 auth_method 0 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f20f4016020 con 0x7f20fc0a6860 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 10 monclient(hunting): handle_auth_done global_id 4531 payload 824 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 1 -- 172.21.15.89:0/614639511 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20fc064590 con 0x7f20fc09cf20 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20e003be00 0x7f20e003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f20f404e300 con 0x7f20fc0a6860 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2101b57640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20e003be00 0x7f20e003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2101b57640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20e003be00 msgr2=0x7f20e003e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2101b57640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20e003be00 msgr2=0x7f20e003e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2101b57640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20e003be00 0x7f20e003e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2101b57640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20e003be00 0x7f20e003e2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:51.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09cf20 msgr2=0x7f20fc09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f2100b55640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09cf20 0x7f20fc09d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20fc09e6f0 tx=0x7f20f800e8c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20f8019070 con 0x7f20fc09cf20 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.549+0000 7f20f1ffb640 10 monclient: dump: 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.573 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.553+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f20f8015870 con 0x7f20fc09cf20 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.553+0000 7f20f1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.553+0000 7f20f1ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.553+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09cf20 msgr2=0x7f20fc09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.553+0000 7f20f1ffb640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09cf20 0x7f20fc09d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20fc09e6f0 tx=0x7f20f800e8c0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.553+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09cf20 msgr2=0x7f20fc09d300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:51.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.553+0000 7f20f1ffb640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09cf20 0x7f20fc09d300 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20fc09e6f0 tx=0x7f20f800e8c0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f21035ba640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:51.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f21035ba640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09d840 0x7f20fc13a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f21035ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f21035ba640 1 -- 172.21.15.89:0/614639511 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f20fc0a28d0 con 0x7f20fc09d840 2024-08-19T10:38:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f2100b55640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09d840 0x7f20fc13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f2100b55640 10 monclient: get_auth_request con 0x7f20fc09d840 auth_method 0 2024-08-19T10:38:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f2100b55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f2100b55640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f2100b55640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f2100b55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f2100b55640 10 monclient(hunting): handle_auth_done global_id 4540 payload 824 2024-08-19T10:38:51.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.557+0000 7f2100b55640 1 -- 172.21.15.89:0/614639511 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f20fc06cab0 con 0x7f20fc09d840 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f2100b55640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09d840 msgr2=0x7f20fc13a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f2100b55640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09d840 0x7f20fc13a840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20fc09e6f0 tx=0x7f20f8004a50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20f801fc00 con 0x7f20fc09d840 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f20f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f20f1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f20f1ffb640 10 monclient: dump: 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9244 (secure 0 0 0) 0x7f20f801fc00 con 0x7f20fc09d840 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f20f1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f20f1ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09d840 msgr2=0x7f20fc13a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f20f1ffb640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09d840 0x7f20fc13a840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20fc09e6f0 tx=0x7f20f8004a50 comp rx=0 tx=0).stop 2024-08-19T10:38:51.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f20f1ffb640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09d840 msgr2=0x7f20fc13a840 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:51.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f20f1ffb640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20fc09d840 0x7f20fc13a840 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20fc09e6f0 tx=0x7f20f8004a50 comp rx=0 tx=0).stop 2024-08-19T10:38:51.580 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:51.580 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:51.580 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:51.580 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:51.580 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 10, 2024-08-19T10:38:51.580 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 61379, 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:51.581 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.582 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.583 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:51.584 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.585 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:51.586 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.587 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.590 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:51.591 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.592 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.592 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.592 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:51.592 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:51.592 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.592 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.592 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.593 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.594 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.594 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:51.594 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:51.594 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.594 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.594 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.594 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:51.594 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.595 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.596 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.597 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.598 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:51.602 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.603 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:51.604 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:51.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f21035ba640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20e003be00 msgr2=0x7f20e003e2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:38:51.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f21035ba640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20e003be00 0x7f20e003e2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f21035ba640 10 monclient: shutdown 2024-08-19T10:38:51.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f21035ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:51.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f21035ba640 1 -- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20fc0a6860 msgr2=0x7f20fc1348c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f2101b57640 1 -- 172.21.15.89:0/614639511 reap_dead start 2024-08-19T10:38:51.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f21035ba640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20fc0a6860 0x7f20fc1348c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20f400b560 tx=0x7f20f400ba20 comp rx=0 tx=0).stop 2024-08-19T10:38:51.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f21035ba640 1 -- 172.21.15.89:0/614639511 shutdown_connections 2024-08-19T10:38:51.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f21035ba640 1 --2- 172.21.15.89:0/614639511 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20fc0a6860 0x7f20fc1348c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f21035ba640 1 -- 172.21.15.89:0/614639511 >> 172.21.15.89:0/614639511 conn(0x7f20fc074b40 msgr2=0x7f20fc0ab370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:51.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f21035ba640 1 -- 172.21.15.89:0/614639511 shutdown_connections 2024-08-19T10:38:51.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.561+0000 7f21035ba640 1 -- 172.21.15.89:0/614639511 wait complete. 2024-08-19T10:38:51.607 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:38:51.607 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-08-19T10:38:51.607 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:38:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.717+0000 7f4a1dd18640 10 monclient: get_monmap_and_config 2024-08-19T10:38:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.717+0000 7f4a1dd18640 10 monclient: build_initial_monmap 2024-08-19T10:38:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.717+0000 7f4a1dd18640 10 monclient: monmap: 2024-08-19T10:38:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:51.724842+0000 2024-08-19T10:38:51.723 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:51.724842+0000 2024-08-19T10:38:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:51.724 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:51.725 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:51.725 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 1 Processor -- start 2024-08-19T10:38:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 1 -- start start 2024-08-19T10:38:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 10 monclient: init 2024-08-19T10:38:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 10 monclient: _add_conns ranks=[10,5,1,3,6,9,0,8,4,2,7] 2024-08-19T10:38:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4a180a3b30 0x7f4a180a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 10 monclient(hunting): picked mon.noname-f con 0x7f4a180a3b30 addr v2:172.21.15.190:3300/0 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a180a3f10 0x7f4a180a4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 10 monclient(hunting): picked mon.noname-h con 0x7f4a180a3f10 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4a181364f0 addr v1:172.21.15.190:6789/0 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a18074170 con 0x7f4a181364f0 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f4a180a08b0 con 0x7f4a180a3b30 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4a180a0630 con 0x7f4a180a3f10 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a1dd18640 10 monclient(hunting): authenticate will time out at 981.541613s 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a17fff640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f4a181364f0 legacy=0x7f4a18136980 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a177fe640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f4a180a3b30 msgr2=0x7f4a180a2a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:38:51.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a177fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4a180a3b30 0x7f4a180a2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a180a3f10 0x7f4a180a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a180a3f10 0x7f4a180a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:51862/0 (socket says 172.21.15.89:51862) 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 1 -- 172.21.15.89:0/1017630345 learned_addr learned my addr 172.21.15.89:0/1017630345 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 10 monclient(hunting): get_auth_request con 0x7f4a180a3f10 auth_method 0 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 10 monclient(hunting): handle_auth_done global_id 4424 payload 274 2024-08-19T10:38:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 1 -- 172.21.15.89:0/1017630345 >> v2:172.21.15.190:3300/0 conn(0x7f4a180a3b30 msgr2=0x7f4a180a2a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 1 --2- 172.21.15.89:0/1017630345 >> v2:172.21.15.190:3300/0 conn(0x7f4a180a3b30 0x7f4a180a2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 1 -- 172.21.15.89:0/1017630345 >> v1:172.21.15.190:6789/0 conn(0x7f4a181364f0 legacy=0x7f4a18136980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:38:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 1 monclient: found mon.noname-h 2024-08-19T10:38:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:38:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.721+0000 7f4a16ffd640 1 -- 172.21.15.89:0/1017630345 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a180a1950 con 0x7f4a180a3f10 2024-08-19T10:38:51.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a16ffd640 1 -- 172.21.15.89:0/1017630345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a180a3f10 msgr2=0x7f4a180a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a16ffd640 1 --2- 172.21.15.89:0/1017630345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a180a3f10 0x7f4a180a4330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a180a2120 tx=0x7f4a0c01eaf0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=b19de25da27c0ae8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 1 -- 172.21.15.89:0/1017630345 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a0c0b4070 con 0x7f4a180a3f10 2024-08-19T10:38:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:38:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 10 monclient: dump: 2024-08-19T10:38:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.735 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.736 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 10 monclient: _finish_auth 0 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 10 monclient: _check_auth_tickets 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 1 -- 172.21.15.89:0/1017630345 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a0c0a7b90 con 0x7f4a180a3f10 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 1 -- 172.21.15.89:0/1017630345 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a0c0a7d80 con 0x7f4a180a3f10 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a1dd18640 5 monclient: authenticate success, global_id 4424 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a1dd18640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a1dd18640 4 set_mon_vals no callback set 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a167fc640 10 monclient: dump: 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.739 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a1dd18640 10 monclient: shutdown 2024-08-19T10:38:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a1dd18640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a1dd18640 1 -- 172.21.15.89:0/1017630345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a180a3f10 msgr2=0x7f4a180a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a1dd18640 1 --2- 172.21.15.89:0/1017630345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a180a3f10 0x7f4a180a4330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a180a2120 tx=0x7f4a0c01eaf0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a1dd18640 1 -- 172.21.15.89:0/1017630345 shutdown_connections 2024-08-19T10:38:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a1dd18640 1 --2- 172.21.15.89:0/1017630345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a180a3f10 0x7f4a180a4330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.725+0000 7f4a1dd18640 1 --2- 172.21.15.89:0/1017630345 >> v2:172.21.15.190:3300/0 conn(0x7f4a180a3b30 0x7f4a180a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 1 -- 172.21.15.89:0/1017630345 >> 172.21.15.89:0/1017630345 conn(0x7f4a180a8f60 msgr2=0x7f4a180a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 1 -- 172.21.15.89:0/1017630345 shutdown_connections 2024-08-19T10:38:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 1 -- 172.21.15.89:0/1017630345 wait complete. 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient: build_initial_monmap 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient: monmap: 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 1 Processor -- start 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 1 -- start start 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient: init 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient: _add_conns ranks=[7,2,5,1,3,8,4,6,0] 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a180a3b30 0x7f4a18134a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient(hunting): picked mon.noname-i con 0x7f4a180a3b30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a180a3f10 0x7f4a18134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient(hunting): picked mon.noname-b con 0x7f4a180a3f10 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a181364f0 0x7f4a18135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient(hunting): picked mon.noname-h con 0x7f4a181364f0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1806d2c0 con 0x7f4a180a3f10 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4a1806d140 con 0x7f4a181364f0 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4a1806d440 con 0x7f4a180a3b30 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a1dd18640 10 monclient(hunting): authenticate will time out at 981.549681s 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a177fe640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a180a3b30 msgr2=0x7f4a18134a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a177fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a180a3b30 0x7f4a18134a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a180a3f10 0x7f4a18134f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a180a3f10 0x7f4a18134f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37948/0 (socket says 172.21.15.89:37948) 2024-08-19T10:38:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 1 -- 172.21.15.89:0/2572593451 learned_addr learned my addr 172.21.15.89:0/2572593451 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 10 monclient(hunting): get_auth_request con 0x7f4a180a3f10 auth_method 0 2024-08-19T10:38:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a17fff640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a181364f0 0x7f4a18135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a17fff640 10 monclient(hunting): get_auth_request con 0x7f4a181364f0 auth_method 0 2024-08-19T10:38:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a17fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a17fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a17fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a17fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a17fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a17fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 10 monclient(hunting): handle_auth_done global_id 4421 payload 824 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 1 -- 172.21.15.89:0/2572593451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a180a3b30 msgr2=0x7f4a18134a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a180a3b30 0x7f4a18134a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 1 -- 172.21.15.89:0/2572593451 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a181364f0 msgr2=0x7f4a18135490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a181364f0 0x7f4a18135490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:38:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 1 monclient: found mon.noname-b 2024-08-19T10:38:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:38:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.729+0000 7f4a16ffd640 1 -- 172.21.15.89:0/2572593451 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a18123c10 con 0x7f4a180a3f10 2024-08-19T10:38:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a16ffd640 1 -- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a180a3f10 msgr2=0x7f4a18134f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a16ffd640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a180a3f10 0x7f4a18134f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00b2c0 tx=0x7f4a0c01b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a0c0b4070 con 0x7f4a180a3f10 2024-08-19T10:38:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:38:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 10 monclient: dump: 2024-08-19T10:38:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 10 monclient: _finish_auth 0 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 10 monclient: _check_auth_tickets 2024-08-19T10:38:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a0c0b6070 con 0x7f4a180a3f10 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a1dd18640 5 monclient: authenticate success, global_id 4421 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a1dd18640 10 monclient: _renew_subs 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a1dd18640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a1dd18640 1 -- 172.21.15.89:0/2572593451 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a18073b30 con 0x7f4a180a3f10 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a0c003ca0 con 0x7f4a180a3f10 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a15ffb640 4 set_mon_vals no callback set 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 10 monclient: dump: 2024-08-19T10:38:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a1dd18640 10 monclient: _renew_subs 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a1dd18640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a1dd18640 1 -- 172.21.15.89:0/2572593451 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a18123930 con 0x7f4a180a3f10 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a1dd18640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a1dd18640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a1813b010 0x7f4a18063b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a1dd18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a1dd18640 1 -- 172.21.15.89:0/2572593451 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4a18064590 con 0x7f4a1813b010 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a177fe640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a1813b010 0x7f4a18063b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4a0c003830 con 0x7f4a180a3f10 2024-08-19T10:38:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a177fe640 10 monclient: get_auth_request con 0x7f4a1813b010 auth_method 0 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a177fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a177fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a0403be00 0x7f4a0403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f4a0c0e8e30 con 0x7f4a180a3f10 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a177fe640 10 monclient(hunting): handle_auth_done global_id 4371 payload 824 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a177fe640 1 -- 172.21.15.89:0/2572593451 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a1813bd00 con 0x7f4a1813b010 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a17fff640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a0403be00 0x7f4a0403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.733+0000 7f4a17fff640 10 monclient: get_auth_request con 0x7f4a0403be00 auth_method 0 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f4a177fe640 1 -- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a1813b010 msgr2=0x7f4a18063b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f4a177fe640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a1813b010 0x7f4a18063b30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a080038c0 tx=0x7f4a0800b810 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a08016070 con 0x7f4a1813b010 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f49fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f49fffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f49fffff640 10 monclient: dump: 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4a08003df0 con 0x7f4a1813b010 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f49fffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f49fffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a1813b010 msgr2=0x7f4a18063b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f49fffff640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a1813b010 0x7f4a18063b30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a080038c0 tx=0x7f4a0800b810 comp rx=0 tx=0).stop 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a1813b010 msgr2=0x7f4a18063b30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f49fffff640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a1813b010 0x7f4a18063b30 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a080038c0 tx=0x7f4a0800b810 comp rx=0 tx=0).stop 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.737+0000 7f4a17fff640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a0403be00 0x7f4a0403e2c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a1813c150 tx=0x7f4a100095f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a1dd18640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a1dd18640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a18064070 0x7f4a1802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a1dd18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a1dd18640 1 -- 172.21.15.89:0/2572593451 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4a180a2a60 con 0x7f4a18064070 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a177fe640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a18064070 0x7f4a1802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a177fe640 10 monclient: get_auth_request con 0x7f4a18064070 auth_method 0 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a177fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a177fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a177fe640 10 monclient(hunting): handle_auth_done global_id 4380 payload 824 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.741+0000 7f4a177fe640 1 -- 172.21.15.89:0/2572593451 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4a1809f100 con 0x7f4a18064070 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a177fe640 1 -- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a18064070 msgr2=0x7f4a1802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a177fe640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a18064070 0x7f4a1802d1d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4a080038c0 tx=0x7f4a08005950 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a08016030 con 0x7f4a18064070 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f49fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f49fffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f49fffff640 10 monclient: dump: 2024-08-19T10:38:51.759 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.760 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.761 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9411 (secure 0 0 0) 0x7f4a08005760 con 0x7f4a18064070 2024-08-19T10:38:51.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f49fffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:51.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f49fffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:51.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a18064070 msgr2=0x7f4a1802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f49fffff640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a18064070 0x7f4a1802d1d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4a080038c0 tx=0x7f4a08005950 comp rx=0 tx=0).stop 2024-08-19T10:38:51.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f49fffff640 1 -- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a18064070 msgr2=0x7f4a1802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:51.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f49fffff640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a18064070 0x7f4a1802d1d0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4a080038c0 tx=0x7f4a08005950 comp rx=0 tx=0).stop 2024-08-19T10:38:51.761 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:51.761 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 10, 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 61563, 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:51.762 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.763 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.764 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:51.765 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.766 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:51.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.768 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:51.769 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:51.772 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:51.773 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.774 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.777 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.779 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:51.780 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.781 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:51.782 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:51.782 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.782 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.782 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.782 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:51.782 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:51.782 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.783 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.784 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.785 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.786 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:38:51.786 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.786 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.786 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:51.786 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:51.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a1dd18640 1 -- 172.21.15.89:0/2572593451 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a0403be00 msgr2=0x7f4a0403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a1dd18640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a0403be00 0x7f4a0403e2c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a1813c150 tx=0x7f4a100095f0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a1dd18640 10 monclient: shutdown 2024-08-19T10:38:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a1dd18640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a1dd18640 1 -- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a180a3f10 msgr2=0x7f4a18134f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a1dd18640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a180a3f10 0x7f4a18134f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00b2c0 tx=0x7f4a0c01b890 comp rx=0 tx=0).stop 2024-08-19T10:38:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a17fff640 1 -- 172.21.15.89:0/2572593451 reap_dead start 2024-08-19T10:38:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a1dd18640 1 -- 172.21.15.89:0/2572593451 shutdown_connections 2024-08-19T10:38:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a1dd18640 1 --2- 172.21.15.89:0/2572593451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a180a3f10 0x7f4a18134f50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a1dd18640 1 -- 172.21.15.89:0/2572593451 >> 172.21.15.89:0/2572593451 conn(0x7f4a180a8f60 msgr2=0x7f4a18072e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a1dd18640 1 -- 172.21.15.89:0/2572593451 shutdown_connections 2024-08-19T10:38:51.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.745+0000 7f4a1dd18640 1 -- 172.21.15.89:0/2572593451 wait complete. 2024-08-19T10:38:51.788 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:38:51.789 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-08-19T10:38:51.789 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:38:51.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient: get_monmap_and_config 2024-08-19T10:38:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient: build_initial_monmap 2024-08-19T10:38:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient: monmap: 2024-08-19T10:38:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:51.905430+0000 2024-08-19T10:38:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:51.905430+0000 2024-08-19T10:38:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:51.904 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:51.905 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:51.905 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:51.905 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:51.905 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:51.905 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:51.905 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:51.905 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:51.905 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:51.905 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 1 Processor -- start 2024-08-19T10:38:51.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 1 -- start start 2024-08-19T10:38:51.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient: init 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient: _add_conns ranks=[2,5,9,1,6,7,8,4,10,3,0] 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff12c0a6860 0x7ff12c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient(hunting): picked mon.noname-b con 0x7ff12c0a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff12c0a7180 0x7ff12c0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient(hunting): picked mon.noname-h con 0x7ff12c0a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff12c139620 0x7ff12c139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient(hunting): picked mon.noname-a con 0x7ff12c139620 addr v2:172.21.15.89:3300/0 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff12c0a8190 con 0x7ff12c139620 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c0a33d0 con 0x7ff12c0a6860 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff12c0a2ec0 con 0x7ff12c0a7180 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff132ab1640 10 monclient(hunting): authenticate will time out at 981.721901s 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff12c139620 0x7ff12c139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13084d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff12c0a6860 0x7ff12c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff12c139620 0x7ff12c139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:44032/0 (socket says 172.21.15.89:44032) 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13084d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff12c0a6860 0x7ff12c0a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37956/0 (socket says 172.21.15.89:37956) 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 1 -- 172.21.15.89:0/2099707380 learned_addr learned my addr 172.21.15.89:0/2099707380 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 10 monclient(hunting): get_auth_request con 0x7ff12c139620 auth_method 0 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13084d640 10 monclient(hunting): get_auth_request con 0x7ff12c0a6860 auth_method 0 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13084d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13084d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13084d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff12bfff640 1 --2- 172.21.15.89:0/2099707380 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff12c0a7180 0x7ff12c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff12bfff640 10 monclient(hunting): get_auth_request con 0x7ff12c0a7180 auth_method 0 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff12bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff12bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff12bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13084d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13084d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13084d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 10 monclient(hunting): handle_auth_done global_id 4581 payload 274 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 1 -- 172.21.15.89:0/2099707380 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff12c0a7180 msgr2=0x7ff12c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 1 --2- 172.21.15.89:0/2099707380 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff12c0a7180 0x7ff12c0a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 1 -- 172.21.15.89:0/2099707380 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff12c0a6860 msgr2=0x7ff12c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 1 --2- 172.21.15.89:0/2099707380 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff12c0a6860 0x7ff12c0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 10 monclient: _finish_hunting 0 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 1 monclient: found mon.noname-a 2024-08-19T10:38:51.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:51.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:38:51.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13104e640 1 -- 172.21.15.89:0/2099707380 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0a4cd0 con 0x7ff12c139620 2024-08-19T10:38:51.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.901+0000 7ff13084d640 1 --2- 172.21.15.89:0/2099707380 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff12c0a6860 0x7ff12c0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff13104e640 1 -- 172.21.15.89:0/2099707380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c139620 msgr2=0x7ff12c139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff13104e640 1 --2- 172.21.15.89:0/2099707380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c139620 0x7ff12c139a80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff124009e50 tx=0x7ff12401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb96a2b02ee7ba9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 1 -- 172.21.15.89:0/2099707380 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff1240090d0 con 0x7ff12c139620 2024-08-19T10:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 10 monclient: dump: 2024-08-19T10:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 1 -- 172.21.15.89:0/2099707380 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1240a59a0 con 0x7ff12c139620 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 5 monclient: authenticate success, global_id 4581 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 4 set_mon_vals no callback set 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 1 -- 172.21.15.89:0/2099707380 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff1240ad3f0 con 0x7ff12c139620 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff12b7fe640 10 monclient: dump: 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.924 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 10 monclient: shutdown 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 1 -- 172.21.15.89:0/2099707380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c139620 msgr2=0x7ff12c139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 1 --2- 172.21.15.89:0/2099707380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c139620 0x7ff12c139a80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff124009e50 tx=0x7ff12401c8f0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 1 -- 172.21.15.89:0/2099707380 shutdown_connections 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 1 --2- 172.21.15.89:0/2099707380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c139620 0x7ff12c139a80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 1 --2- 172.21.15.89:0/2099707380 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff12c0a7180 0x7ff12c0a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 1 --2- 172.21.15.89:0/2099707380 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff12c0a6860 0x7ff12c0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 1 -- 172.21.15.89:0/2099707380 >> 172.21.15.89:0/2099707380 conn(0x7ff12c074b40 msgr2=0x7ff12c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 1 -- 172.21.15.89:0/2099707380 shutdown_connections 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.905+0000 7ff132ab1640 1 -- 172.21.15.89:0/2099707380 wait complete. 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient: build_initial_monmap 2024-08-19T10:38:51.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient: monmap: 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:51.927 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 1 Processor -- start 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 1 -- start start 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient: init 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient: _add_conns ranks=[6,0,1,8,4,7,3,2,5] 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff12c0a6860 0x7ff12c134830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient(hunting): picked mon.noname-d con 0x7ff12c0a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c0a7180 0x7ff12c134d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient(hunting): picked mon.noname-a con 0x7ff12c0a7180 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12c139620 0x7ff12c09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient(hunting): picked mon.noname-f con 0x7ff12c139620 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c143210 con 0x7ff12c0a7180 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c143090 con 0x7ff12c139620 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff12c143390 con 0x7ff12c0a6860 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 10 monclient(hunting): authenticate will time out at 981.729112s 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff12c0a6860 0x7ff12c134830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13104e640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12c139620 msgr2=0x7ff12c09c9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13104e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12c139620 0x7ff12c09c9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff12c0a6860 0x7ff12c134830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:40946/0 (socket says 172.21.15.89:40946) 2024-08-19T10:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 1 -- 172.21.15.89:0/1721668909 learned_addr learned my addr 172.21.15.89:0/1721668909 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 10 monclient(hunting): get_auth_request con 0x7ff12c0a6860 auth_method 0 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff12bfff640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c0a7180 0x7ff12c134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff12bfff640 10 monclient(hunting): get_auth_request con 0x7ff12c0a7180 auth_method 0 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff12bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff12bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff12bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 10 monclient(hunting): handle_auth_done global_id 4389 payload 824 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 1 -- 172.21.15.89:0/1721668909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12c139620 msgr2=0x7ff12c09c9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12c139620 0x7ff12c09c9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 1 -- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c0a7180 msgr2=0x7ff12c134d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c0a7180 0x7ff12c134d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 10 monclient: _finish_hunting 0 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 1 monclient: found mon.noname-d 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 1 -- 172.21.15.89:0/1721668909 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0a0030 con 0x7ff12c0a6860 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff12bfff640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12c0a7180 0x7ff12c134d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:38:51.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 1 -- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff12c0a6860 msgr2=0x7ff12c134830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff13084d640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff12c0a6860 0x7ff12c134830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00ebe0 tx=0x7ff11c00c520 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff11c017070 con 0x7ff12c0a6860 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff1297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff1297fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff1297fa640 10 monclient: dump: 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.932 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff1297fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff1297fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff1297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c00ce40 con 0x7ff12c0a6860 2024-08-19T10:38:51.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.909+0000 7ff132ab1640 5 monclient: authenticate success, global_id 4389 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff132ab1640 10 monclient: _renew_subs 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff132ab1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff132ab1640 1 -- 172.21.15.89:0/1721668909 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c0a0310 con 0x7ff12c0a6860 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff11c013d50 con 0x7ff12c0a6860 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 10 monclient: dump: 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.934 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff132ab1640 10 monclient: _renew_subs 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff132ab1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:51.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff132ab1640 1 -- 172.21.15.89:0/1721668909 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c09e020 con 0x7ff12c0a6860 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12affd640 4 set_mon_vals no callback set 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff132ab1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff132ab1640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09cf20 0x7ff12c09d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff132ab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff132ab1640 1 -- 172.21.15.89:0/1721668909 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff12c0a0d10 con 0x7ff12c09cf20 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09cf20 0x7ff12c09d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 10 monclient: get_auth_request con 0x7ff12c09cf20 auth_method 0 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff11c01a070 con 0x7ff12c0a6860 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 10 monclient(hunting): handle_auth_done global_id 4400 payload 824 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff11803be30 0x7ff11803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 1 -- 172.21.15.89:0/1721668909 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff12c0a7560 con 0x7ff12c09cf20 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7ff11c00f040 con 0x7ff12c0a6860 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff13104e640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff11803be30 0x7ff11803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff13104e640 10 monclient: get_auth_request con 0x7ff11803be30 auth_method 0 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 1 -- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09cf20 msgr2=0x7ff12c09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff12bfff640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09cf20 0x7ff12c09d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff12c13f5a0 tx=0x7ff12000f610 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff120014ac0 con 0x7ff12c09cf20 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 10 monclient: dump: 2024-08-19T10:38:51.937 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.938 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.939 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.939 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff120014650 con 0x7ff12c09cf20 2024-08-19T10:38:51.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:51.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:51.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09cf20 msgr2=0x7ff12c09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09cf20 0x7ff12c09d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff12c13f5a0 tx=0x7ff12000f610 comp rx=0 tx=0).stop 2024-08-19T10:38:51.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09cf20 msgr2=0x7ff12c09d300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:51.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.913+0000 7ff1297fa640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09cf20 0x7ff12c09d300 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff12c13f5a0 tx=0x7ff12000f610 comp rx=0 tx=0).stop 2024-08-19T10:38:51.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.917+0000 7ff13104e640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff11803be30 0x7ff11803e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0a9a20 tx=0x7ff1240a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff132ab1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:51.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff132ab1640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09d840 0x7ff12c13aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:51.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff132ab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:51.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff132ab1640 1 -- 172.21.15.89:0/1721668909 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff12c064590 con 0x7ff12c09d840 2024-08-19T10:38:51.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff12bfff640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09d840 0x7ff12c13aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:51.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff12bfff640 10 monclient: get_auth_request con 0x7ff12c09d840 auth_method 0 2024-08-19T10:38:51.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff12bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:51.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff12bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff12bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff12bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff12bfff640 10 monclient(hunting): handle_auth_done global_id 4409 payload 824 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.921+0000 7ff12bfff640 1 -- 172.21.15.89:0/1721668909 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff12c09f790 con 0x7ff12c09d840 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff12bfff640 1 -- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09d840 msgr2=0x7ff12c13aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff12bfff640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09d840 0x7ff12c13aa80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff12c13f5a0 tx=0x7ff1200046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff12002b020 con 0x7ff12c09d840 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff1297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff1297fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff1297fa640 10 monclient: dump: 2024-08-19T10:38:51.942 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:51.943 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:51.944 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9244 (secure 0 0 0) 0x7ff1200133a0 con 0x7ff12c09d840 2024-08-19T10:38:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff1297fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff1297fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09d840 msgr2=0x7ff12c13aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff1297fa640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09d840 0x7ff12c13aa80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff12c13f5a0 tx=0x7ff1200046d0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff1297fa640 1 -- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09d840 msgr2=0x7ff12c13aa80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:51.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff1297fa640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff12c09d840 0x7ff12c13aa80 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff12c13f5a0 tx=0x7ff1200046d0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.944 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 10, 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 61719, 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:51.945 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.946 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.947 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:51.948 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:51.949 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.950 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.951 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:51.952 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.953 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.954 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:51.955 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.956 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.957 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.960 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:51.960 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:51.960 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.960 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.960 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.960 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:51.960 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.962 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:51.963 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.964 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:51.965 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.966 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:51.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.968 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:51.968 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:51.968 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:51.968 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:51.968 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:51.968 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:51.968 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:51.968 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:51.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:51.968 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:51.968 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:51.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff132ab1640 1 -- 172.21.15.89:0/1721668909 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff11803be30 msgr2=0x7ff11803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff132ab1640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff11803be30 0x7ff11803e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0a9a20 tx=0x7ff1240a7070 comp rx=0 tx=0).stop 2024-08-19T10:38:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff132ab1640 10 monclient: shutdown 2024-08-19T10:38:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff132ab1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff132ab1640 1 -- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff12c0a6860 msgr2=0x7ff12c134830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff132ab1640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff12c0a6860 0x7ff12c134830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00ebe0 tx=0x7ff11c00c520 comp rx=0 tx=0).stop 2024-08-19T10:38:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff13104e640 1 -- 172.21.15.89:0/1721668909 reap_dead start 2024-08-19T10:38:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff132ab1640 1 -- 172.21.15.89:0/1721668909 shutdown_connections 2024-08-19T10:38:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff132ab1640 1 --2- 172.21.15.89:0/1721668909 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff12c0a6860 0x7ff12c134830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff132ab1640 1 -- 172.21.15.89:0/1721668909 >> 172.21.15.89:0/1721668909 conn(0x7ff12c074b40 msgr2=0x7ff12c0ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff132ab1640 1 -- 172.21.15.89:0/1721668909 shutdown_connections 2024-08-19T10:38:51.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:51.925+0000 7ff132ab1640 1 -- 172.21.15.89:0/1721668909 wait complete. 2024-08-19T10:38:51.971 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:38:51.971 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-08-19T10:38:51.971 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:38:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.081+0000 7fcfaef1f640 10 monclient: get_monmap_and_config 2024-08-19T10:38:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.081+0000 7fcfaef1f640 10 monclient: build_initial_monmap 2024-08-19T10:38:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.081+0000 7fcfaef1f640 10 monclient: monmap: 2024-08-19T10:38:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:52.088655+0000 2024-08-19T10:38:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:52.088655+0000 2024-08-19T10:38:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:52.087 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:52.088 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 1 Processor -- start 2024-08-19T10:38:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 1 -- start start 2024-08-19T10:38:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 10 monclient: init 2024-08-19T10:38:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 10 monclient: _add_conns ranks=[5,0,3,4,10,1,7,8,6,9,2] 2024-08-19T10:38:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcfa80a6860 0x7fcfa80a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 10 monclient(hunting): picked mon.noname-h con 0x7fcfa80a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcfa80a7180 addr v1:172.21.15.89:6789/0 2024-08-19T10:38:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa80a7560 0x7fcfa8139d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 10 monclient(hunting): picked mon.noname-g con 0x7fcfa80a7560 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:38:52.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa80a33d0 con 0x7fcfa80a7180 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa80a8190 con 0x7fcfa80a7560 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fcfa80a79f0 con 0x7fcfa80a6860 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaef1f640 10 monclient(hunting): authenticate will time out at 981.905213s 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfa7fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcfa80a7180 0x7fcfa8139620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35150/0 (socket says 172.21.15.89:35150) 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfa7fff640 1 -- 172.21.15.89:0/3142617583 learned_addr learned my addr 172.21.15.89:0/3142617583 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaccbb640 1 --2- 172.21.15.89:0/3142617583 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcfa80a6860 0x7fcfa80a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 1 --2- 172.21.15.89:0/3142617583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa80a7560 0x7fcfa8139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaccbb640 10 monclient(hunting): get_auth_request con 0x7fcfa80a6860 auth_method 0 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaccbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaccbb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaccbb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 10 monclient(hunting): get_auth_request con 0x7fcfa80a7560 auth_method 0 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaccbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaccbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfaccbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 10 monclient(hunting): handle_auth_done global_id 4431 payload 274 2024-08-19T10:38:52.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 1 -- 172.21.15.89:0/3142617583 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcfa80a6860 msgr2=0x7fcfa80a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 1 --2- 172.21.15.89:0/3142617583 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcfa80a6860 0x7fcfa80a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 1 -- 172.21.15.89:0/3142617583 >> v1:172.21.15.89:6789/0 conn(0x7fcfa80a7180 legacy=0x7fcfa8139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 10 monclient: _finish_hunting 0 2024-08-19T10:38:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 1 monclient: found mon.noname-g 2024-08-19T10:38:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:38:52.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.085+0000 7fcfad4bc640 1 -- 172.21.15.89:0/3142617583 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa80a2ec0 con 0x7fcfa80a7560 2024-08-19T10:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfad4bc640 1 -- 172.21.15.89:0/3142617583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa80a7560 msgr2=0x7fcfa8139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfad4bc640 1 --2- 172.21.15.89:0/3142617583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa80a7560 0x7fcfa8139d30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009dc0 tx=0x7fcfa001c850 comp rx=0 tx=0).ready entity=mon.3 client_cookie=a4b73213db972a48 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 1 -- 172.21.15.89:0/3142617583 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcfa00a6960 con 0x7fcfa80a7560 2024-08-19T10:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 10 monclient: dump: 2024-08-19T10:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.096 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 10 monclient: _finish_auth 0 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:38:52.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 1 -- 172.21.15.89:0/3142617583 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa00a6b20 con 0x7fcfa80a7560 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 1 -- 172.21.15.89:0/3142617583 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcfa00a6cd0 con 0x7fcfa80a7560 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfa77fe640 10 monclient: dump: 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.098 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 5 monclient: authenticate success, global_id 4431 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 4 set_mon_vals no callback set 2024-08-19T10:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 10 monclient: shutdown 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3142617583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa80a7560 msgr2=0x7fcfa8139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 1 --2- 172.21.15.89:0/3142617583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa80a7560 0x7fcfa8139d30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009dc0 tx=0x7fcfa001c850 comp rx=0 tx=0).stop 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3142617583 shutdown_connections 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 1 --2- 172.21.15.89:0/3142617583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa80a7560 0x7fcfa8139d30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 1 --2- 172.21.15.89:0/3142617583 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcfa80a6860 0x7fcfa80a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3142617583 >> 172.21.15.89:0/3142617583 conn(0x7fcfa8074b40 msgr2=0x7fcfa80abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3142617583 shutdown_connections 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.089+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3142617583 wait complete. 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient: build_initial_monmap 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient: monmap: 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:52.100 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 1 Processor -- start 2024-08-19T10:38:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 1 -- start start 2024-08-19T10:38:52.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient: init 2024-08-19T10:38:52.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient: _add_conns ranks=[7,5,6,3,2,0,4,1,8] 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcfa80a6860 0x7fcfa81d1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient(hunting): picked mon.noname-i con 0x7fcfa80a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcfa80a7180 0x7fcfa81d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient(hunting): picked mon.noname-h con 0x7fcfa80a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcfa80a7560 0x7fcfa813df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient(hunting): picked mon.noname-d con 0x7fcfa80a7560 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fcfa813a3f0 con 0x7fcfa80a7180 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fcfa813a270 con 0x7fcfa80a7560 2024-08-19T10:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fcfa81440a0 con 0x7fcfa80a6860 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaef1f640 10 monclient(hunting): authenticate will time out at 981.913751s 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaccbb640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcfa80a6860 msgr2=0x7fcfa81d1360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfaccbb640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcfa80a6860 0x7fcfa81d1360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcfa80a7560 0x7fcfa813df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcfa80a7560 0x7fcfa813df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:40952/0 (socket says 172.21.15.89:40952) 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 1 -- 172.21.15.89:0/3779607867 learned_addr learned my addr 172.21.15.89:0/3779607867 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 10 monclient(hunting): get_auth_request con 0x7fcfa80a7560 auth_method 0 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfa7fff640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcfa80a7180 0x7fcfa81d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfa7fff640 10 monclient(hunting): get_auth_request con 0x7fcfa80a7180 auth_method 0 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfa7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfa7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfa7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfa7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfa7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfa7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 10 monclient(hunting): handle_auth_done global_id 4398 payload 824 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 1 -- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcfa80a6860 msgr2=0x7fcfa81d1360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcfa80a6860 0x7fcfa81d1360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 1 -- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcfa80a7180 msgr2=0x7fcfa81d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcfa80a7180 0x7fcfa81d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 10 monclient: _finish_hunting 0 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 1 monclient: found mon.noname-d 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.093+0000 7fcfad4bc640 1 -- 172.21.15.89:0/3779607867 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa81404f0 con 0x7fcfa80a7560 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfad4bc640 1 -- 172.21.15.89:0/3779607867 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcfa80a7560 msgr2=0x7fcfa813df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfad4bc640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcfa80a7560 0x7fcfa813df90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcfa001c680 tx=0x7fcfa00195f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcfa00bc070 con 0x7fcfa80a7560 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 10 monclient: dump: 2024-08-19T10:38:52.110 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.111 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 10 monclient: _finish_auth 0 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa00a6e10 con 0x7fcfa80a7560 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcfa0003dc0 con 0x7fcfa80a7560 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaef1f640 5 monclient: authenticate success, global_id 4398 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaef1f640 10 monclient: _renew_subs 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaef1f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3779607867 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa8122fd0 con 0x7fcfa80a7560 2024-08-19T10:38:52.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa6ffd640 4 set_mon_vals no callback set 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 10 monclient: dump: 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaef1f640 10 monclient: _renew_subs 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaef1f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3779607867 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa80a2570 con 0x7fcfa80a7560 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaef1f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaef1f640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813e4d0 0x7fcfa813e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaef1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3779607867 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa813f990 con 0x7fcfa813e4d0 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fcfa00bf070 con 0x7fcfa80a7560 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcf9403be00 0x7fcf9403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaccbb640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813e4d0 0x7fcfa813e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaccbb640 10 monclient: get_auth_request con 0x7fcfa813e4d0 auth_method 0 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7fcfa00e9a80 con 0x7fcfa80a7560 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaccbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaccbb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaccbb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaccbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaccbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaccbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa7fff640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcf9403be00 0x7fcf9403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfa7fff640 10 monclient: get_auth_request con 0x7fcf9403be00 auth_method 0 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaccbb640 10 monclient(hunting): handle_auth_done global_id 4440 payload 824 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.097+0000 7fcfaccbb640 1 -- 172.21.15.89:0/3779607867 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcfa80a2040 con 0x7fcfa813e4d0 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa7fff640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcf9403be00 0x7fcf9403e2c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c007ca0 tx=0x7fcf9c007bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfaccbb640 1 -- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813e4d0 msgr2=0x7fcfa813e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfaccbb640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813e4d0 0x7fcfa813e8b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800e750 tx=0x7fcf9800ec10 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcf98017070 con 0x7fcfa813e4d0 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa57fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa57fa640 10 monclient: dump: 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcf9800ee60 con 0x7fcfa813e4d0 2024-08-19T10:38:52.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813e4d0 msgr2=0x7fcfa813e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa57fa640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813e4d0 0x7fcfa813e8b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800e750 tx=0x7fcf9800ec10 comp rx=0 tx=0).stop 2024-08-19T10:38:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813e4d0 msgr2=0x7fcfa813e8b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:52.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.101+0000 7fcfa57fa640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813e4d0 0x7fcfa813e8b0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800e750 tx=0x7fcf9800ec10 comp rx=0 tx=0).stop 2024-08-19T10:38:52.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.105+0000 7fcfaef1f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.105+0000 7fcfaef1f640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813edf0 0x7fcfa813a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.105+0000 7fcfaef1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.105+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3779607867 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa80a6c40 con 0x7fcfa813edf0 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.105+0000 7fcfaccbb640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813edf0 0x7fcfa813a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.105+0000 7fcfaccbb640 10 monclient: get_auth_request con 0x7fcfa813edf0 auth_method 0 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.105+0000 7fcfaccbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.105+0000 7fcfaccbb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.105+0000 7fcfaccbb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaccbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaccbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaccbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaccbb640 10 monclient(hunting): handle_auth_done global_id 4449 payload 824 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaccbb640 1 -- 172.21.15.89:0/3779607867 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcfa8064590 con 0x7fcfa813edf0 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaccbb640 1 -- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813edf0 msgr2=0x7fcfa813a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaccbb640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813edf0 0x7fcfa813a630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800f010 tx=0x7fcf98004740 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcf98017070 con 0x7fcfa813edf0 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfa57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfa57fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfa57fa640 10 monclient: dump: 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.121 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9578 (secure 0 0 0) 0x7fcf980066b0 con 0x7fcfa813edf0 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfa57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfa57fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:52.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813edf0 msgr2=0x7fcfa813a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfa57fa640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813edf0 0x7fcfa813a630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800f010 tx=0x7fcf98004740 comp rx=0 tx=0).stop 2024-08-19T10:38:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfa57fa640 1 -- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813edf0 msgr2=0x7fcfa813a630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:52.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfa57fa640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfa813edf0 0x7fcfa813a630 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800f010 tx=0x7fcf98004740 comp rx=0 tx=0).stop 2024-08-19T10:38:52.123 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:52.123 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:52.123 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:52.123 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:52.123 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 10, 2024-08-19T10:38:52.123 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 61882, 2024-08-19T10:38:52.123 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:52.124 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:52.125 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.126 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:52.127 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:52.128 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.131 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:52.132 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.134 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.135 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.137 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.138 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.139 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:52.140 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:52.143 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:52.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:52.145 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.147 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:52.147 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcf9403be00 msgr2=0x7fcf9403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaef1f640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcf9403be00 0x7fcf9403e2c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c007ca0 tx=0x7fcf9c007bf0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaef1f640 10 monclient: shutdown 2024-08-19T10:38:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaef1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3779607867 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcfa80a7560 msgr2=0x7fcfa813df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfaef1f640 1 --2- 172.21.15.89:0/3779607867 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcfa80a7560 0x7fcfa813df90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcfa001c680 tx=0x7fcfa00195f0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfad4bc640 1 -- 172.21.15.89:0/3779607867 reap_dead start 2024-08-19T10:38:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.109+0000 7fcfad4bc640 1 -- 172.21.15.89:0/3779607867 reap_dead start 2024-08-19T10:38:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.113+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3779607867 shutdown_connections 2024-08-19T10:38:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.113+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3779607867 >> 172.21.15.89:0/3779607867 conn(0x7fcfa8074b40 msgr2=0x7fcfa8141090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.113+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3779607867 shutdown_connections 2024-08-19T10:38:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.113+0000 7fcfaef1f640 1 -- 172.21.15.89:0/3779607867 wait complete. 2024-08-19T10:38:52.149 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:38:52.149 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-08-19T10:38:52.149 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:38:52.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient: get_monmap_and_config 2024-08-19T10:38:52.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient: build_initial_monmap 2024-08-19T10:38:52.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient: monmap: 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:38:52.266256+0000 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:38:52.266256+0000 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:52.265 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:52.266 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:52.266 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:52.266 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:38:52.266 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:38:52.266 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 1 Processor -- start 2024-08-19T10:38:52.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 1 -- start start 2024-08-19T10:38:52.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient: init 2024-08-19T10:38:52.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:52.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient: _add_conns ranks=[8,1,2,3,5,7,9,0,10,4,6] 2024-08-19T10:38:52.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00040a3c50 0x7f00040a4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient(hunting): picked mon.noname-e con 0x7f00040a3c50 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:38:52.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f00040a4570 addr v1:172.21.15.190:6789/0 2024-08-19T10:38:52.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000413a0e0 0x7f0004141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient(hunting): picked mon.noname-b con 0x7f000413a0e0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00040a5400 con 0x7f00040a4570 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f000406d470 con 0x7f000413a0e0 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f00040a13e0 con 0x7f00040a3c50 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0008d19640 10 monclient(hunting): authenticate will time out at 982.083591s 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0001d9b640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f00040a4570 legacy=0x7f00041399d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f000259c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00040a3c50 0x7f00040a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0002d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000413a0e0 0x7f0004141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f000259c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00040a3c50 0x7f00040a4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47010/0 (socket says 172.21.15.89:47010) 2024-08-19T10:38:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f000259c640 1 -- 172.21.15.89:0/3213683119 learned_addr learned my addr 172.21.15.89:0/3213683119 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f000259c640 10 monclient(hunting): get_auth_request con 0x7f00040a3c50 auth_method 0 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f000259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f000259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f000259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0002d9d640 10 monclient(hunting): get_auth_request con 0x7f000413a0e0 auth_method 0 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0002d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0002d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0002d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f000259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f000259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f000259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0002d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0002d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.261+0000 7f0002d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.265+0000 7f000259c640 10 monclient(hunting): handle_auth_done global_id 4418 payload 274 2024-08-19T10:38:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.265+0000 7f000259c640 1 -- 172.21.15.89:0/3213683119 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000413a0e0 msgr2=0x7f0004141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:38:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.265+0000 7f000259c640 1 --2- 172.21.15.89:0/3213683119 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000413a0e0 0x7f0004141ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.265+0000 7f000259c640 1 -- 172.21.15.89:0/3213683119 >> v1:172.21.15.190:6789/0 conn(0x7f00040a4570 legacy=0x7f00041399d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:38:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.265+0000 7f000259c640 10 monclient: _finish_hunting 0 2024-08-19T10:38:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.265+0000 7f000259c640 1 monclient: found mon.noname-e 2024-08-19T10:38:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.265+0000 7f000259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.265+0000 7f000259c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:38:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.265+0000 7f000259c640 1 -- 172.21.15.89:0/3213683119 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000406cf60 con 0x7f00040a3c50 2024-08-19T10:38:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 1 -- 172.21.15.89:0/3213683119 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00040a3c50 msgr2=0x7f00040a4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 1 --2- 172.21.15.89:0/3213683119 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00040a3c50 0x7f00040a4030 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efff4009e80 tx=0x7efff401c910 comp rx=0 tx=0).ready entity=mon.8 client_cookie=8720cf4bd991e479 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 1 -- 172.21.15.89:0/3213683119 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efff40090d0 con 0x7f00040a3c50 2024-08-19T10:38:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:38:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 10 monclient: dump: 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 10 monclient: _finish_auth 0 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 10 monclient: _check_auth_tickets 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 1 -- 172.21.15.89:0/3213683119 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff40a6970 con 0x7f00040a3c50 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 5 monclient: authenticate success, global_id 4418 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient: get_monmap_and_config success 2024-08-19T10:38:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 4 set_mon_vals no callback set 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 1 -- 172.21.15.89:0/3213683119 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efff40af480 con 0x7f00040a3c50 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000159a640 10 monclient: dump: 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient: shutdown 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 -- 172.21.15.89:0/3213683119 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00040a3c50 msgr2=0x7f00040a4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 --2- 172.21.15.89:0/3213683119 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00040a3c50 0x7f00040a4030 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efff4009e80 tx=0x7efff401c910 comp rx=0 tx=0).stop 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 -- 172.21.15.89:0/3213683119 shutdown_connections 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 --2- 172.21.15.89:0/3213683119 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f000413a0e0 0x7f0004141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 --2- 172.21.15.89:0/3213683119 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00040a3c50 0x7f00040a4030 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 -- 172.21.15.89:0/3213683119 >> 172.21.15.89:0/3213683119 conn(0x7f000406de40 msgr2=0x7f000406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 -- 172.21.15.89:0/3213683119 shutdown_connections 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 -- 172.21.15.89:0/3213683119 wait complete. 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient: build_initial_monmap 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient: monmap: 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:38:52.278 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:38:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:38:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:38:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:38:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:38:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:38:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:38:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:38:52.279 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 Processor -- start 2024-08-19T10:38:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 -- start start 2024-08-19T10:38:52.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient: init 2024-08-19T10:38:52.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient: _reopen_session rank -1 2024-08-19T10:38:52.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient: _add_conns ranks=[4,5,1,8,2,6,0,3,7] 2024-08-19T10:38:52.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00040a3c50 0x7f00041d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient(hunting): picked mon.noname-c con 0x7f00040a3c50 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:38:52.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040a4570 0x7f00041d2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient(hunting): picked mon.noname-h con 0x7f00040a4570 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000413a0e0 0x7f000413d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient(hunting): picked mon.noname-f con 0x7f000413a0e0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f000406d810 con 0x7f000413a0e0 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f000406d690 con 0x7f00040a3c50 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0004144a00 con 0x7f00040a4570 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient(hunting): _renew_subs 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0008d19640 10 monclient(hunting): authenticate will time out at 982.090954s 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00040a3c50 0x7f00041d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0002d9d640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000413a0e0 msgr2=0x7f000413d780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00040a3c50 0x7f00041d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58262/0 (socket says 172.21.15.89:58262) 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 1 -- 172.21.15.89:0/2073643657 learned_addr learned my addr 172.21.15.89:0/2073643657 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 10 monclient(hunting): get_auth_request con 0x7f00040a3c50 auth_method 0 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0002d9d640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000413a0e0 0x7f000413d780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0001d9b640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040a4570 0x7f00041d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 10 monclient(hunting): handle_auth_done global_id 4549 payload 824 2024-08-19T10:38:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040a4570 msgr2=0x7f00041d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040a4570 0x7f00041d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000413a0e0 msgr2=0x7f000413d780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000413a0e0 0x7f000413d780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 10 monclient: _finish_hunting 0 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 1 monclient: found mon.noname-c 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f000259c640 1 -- 172.21.15.89:0/2073643657 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00041412c0 con 0x7f00040a3c50 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0001d9b640 10 monclient: get_auth_request con 0x7f00040a4570 auth_method 0 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.269+0000 7f0001d9b640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040a4570 0x7f00041d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f000259c640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00040a3c50 msgr2=0x7f00041d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f000259c640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00040a3c50 0x7f00041d1c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efff4000f90 tx=0x7efff40196b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:52.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efff4003e10 con 0x7f00040a3c50 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 10 monclient: dump: 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.289 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff40a6d30 con 0x7f00040a3c50 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efff40044b0 con 0x7f00040a3c50 2024-08-19T10:38:52.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 10 monclient: dump: 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.291 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0008d19640 5 monclient: authenticate success, global_id 4549 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0000d99640 4 set_mon_vals no callback set 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0008d19640 10 monclient: _renew_subs 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0008d19640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0008d19640 1 -- 172.21.15.89:0/2073643657 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00040a7220 con 0x7f00040a3c50 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0008d19640 10 monclient: _renew_subs 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0008d19640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:38:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0008d19640 1 -- 172.21.15.89:0/2073643657 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f00040a6d10 con 0x7f00040a3c50 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0008d19640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0008d19640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 0x7f00040719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0008d19640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0008d19640 1 -- 172.21.15.89:0/2073643657 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0004141db0 con 0x7f00040715d0 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efff40a67f0 con 0x7f00040a3c50 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0001d9b640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 0x7f00040719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7effe403be30 0x7effe403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0001d9b640 10 monclient: get_auth_request con 0x7f00040715d0 auth_method 0 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0001d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0001d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0001d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7efff40b6070 con 0x7f00040a3c50 2024-08-19T10:38:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0001d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0001d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0001d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0002d9d640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7effe403be30 0x7effe403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0002d9d640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7effe403be30 msgr2=0x7effe403e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0002d9d640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7effe403be30 msgr2=0x7effe403e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0002d9d640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7effe403be30 0x7effe403e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0002d9d640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7effe403be30 0x7effe403e2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0001d9b640 10 monclient(hunting): handle_auth_done global_id 4469 payload 824 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.273+0000 7f0001d9b640 1 -- 172.21.15.89:0/2073643657 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f00040a4030 con 0x7f00040715d0 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 msgr2=0x7f00040719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 0x7f00040719b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f000413f8b0 tx=0x7efff800cb80 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 msgr2=0x7f00040719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:38:52.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 msgr2=0x7f00040719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 0x7f00040719b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f000413f8b0 tx=0x7efff800cb80 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 0x7f00040719b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f000413f8b0 tx=0x7efff800cb80 comp rx=0 tx=0).stop 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 msgr2=0x7f00040719b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7efff2ffd640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 0x7f00040719b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7efff2ffd640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7effe40415d0 0x7effe40439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 msgr2=0x7f00040719b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7efff2ffd640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f00040715d0 0x7f00040719b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7efff2ffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7effe4044030 con 0x7effe40415d0 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7effe40415d0 0x7effe40439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 10 monclient: get_auth_request con 0x7effe40415d0 auth_method 0 2024-08-19T10:38:52.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 10 monclient(hunting): handle_auth_done global_id 4478 payload 824 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.277+0000 7f0001d9b640 1 -- 172.21.15.89:0/2073643657 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efff400aea0 con 0x7effe40415d0 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7f0001d9b640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7effe40415d0 msgr2=0x7effe40439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7f0001d9b640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7effe40415d0 0x7effe40439f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efff800cbc0 tx=0x7efff8005990 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efff801b430 con 0x7effe40415d0 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7efff2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7efff2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:38:52.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7efff2ffd640 10 monclient: dump: 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.297 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.298 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.298 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efff8016070 con 0x7effe40415d0 2024-08-19T10:38:52.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7efff2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:38:52.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7efff2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:38:52.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7effe40415d0 msgr2=0x7effe40439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:52.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7efff2ffd640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7effe40415d0 0x7effe40439f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efff800cbc0 tx=0x7efff8005990 comp rx=0 tx=0).stop 2024-08-19T10:38:52.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7effe40415d0 msgr2=0x7effe40439f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:52.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.281+0000 7efff2ffd640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7effe40415d0 0x7effe40439f0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efff800cbc0 tx=0x7efff8005990 comp rx=0 tx=0).stop 2024-08-19T10:38:52.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0008d19640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:38:52.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0008d19640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0004071ef0 0x7f000413dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:38:52.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0008d19640 10 monclient(hunting): start opening mon connection 2024-08-19T10:38:52.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0008d19640 1 -- 172.21.15.89:0/2073643657 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0004069f50 con 0x7f0004071ef0 2024-08-19T10:38:52.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0001d9b640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0004071ef0 0x7f000413dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:38:52.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0001d9b640 10 monclient: get_auth_request con 0x7f0004071ef0 auth_method 0 2024-08-19T10:38:52.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0001d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:38:52.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0001d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0001d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0001d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0001d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.285+0000 7f0001d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0001d9b640 10 monclient(hunting): handle_auth_done global_id 4487 payload 824 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0001d9b640 1 -- 172.21.15.89:0/2073643657 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0004064590 con 0x7f0004071ef0 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0001d9b640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0004071ef0 msgr2=0x7f000413dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0001d9b640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0004071ef0 0x7f000413dcc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effe40441b0 tx=0x7efff8016000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efff801b430 con 0x7f0004071ef0 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7efff2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7efff2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7efff2ffd640 10 monclient: dump: 2024-08-19T10:38:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:38:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:38:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:38:52.303 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:38:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9411 (secure 0 0 0) 0x7efff8006460 con 0x7f0004071ef0 2024-08-19T10:38:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7efff2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:38:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7efff2ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:38:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0004071ef0 msgr2=0x7f000413dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7efff2ffd640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0004071ef0 0x7f000413dcc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effe40441b0 tx=0x7efff8016000 comp rx=0 tx=0).stop 2024-08-19T10:38:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7efff2ffd640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0004071ef0 msgr2=0x7f000413dcc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:38:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7efff2ffd640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0004071ef0 0x7f000413dcc0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effe40441b0 tx=0x7efff8016000 comp rx=0 tx=0).stop 2024-08-19T10:38:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0002d9d640 1 -- 172.21.15.89:0/2073643657 reap_dead start 2024-08-19T10:38:52.303 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 10, 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 62062, 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:38:52.304 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:52.305 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:52.306 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:38:52.307 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:38:52.308 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.309 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.310 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:38:52.312 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:38:52.313 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.314 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.315 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.316 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.317 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.318 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:38:52.319 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:38:52.320 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:38:52.323 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:52.325 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:38:52.326 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:38:52.327 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:38:52.327 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:38:52.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:38:52.327 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:38:52.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:38:52.327 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:38:52.327 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:38:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7effe403be30 msgr2=0x7effe403e2f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:38:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7effe403be30 0x7effe403e2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 10 monclient: shutdown 2024-08-19T10:38:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:38:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 1 -- 172.21.15.89:0/2073643657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00040a3c50 msgr2=0x7f00041d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:38:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00040a3c50 0x7f00041d1c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efff4000f90 tx=0x7efff40196b0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 1 -- 172.21.15.89:0/2073643657 shutdown_connections 2024-08-19T10:38:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7effe403be30 0x7effe403e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 1 --2- 172.21.15.89:0/2073643657 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00040a3c50 0x7f00041d1c50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:38:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 1 -- 172.21.15.89:0/2073643657 >> 172.21.15.89:0/2073643657 conn(0x7f000406de40 msgr2=0x7f00040a5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:38:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 1 -- 172.21.15.89:0/2073643657 shutdown_connections 2024-08-19T10:38:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:38:52.289+0000 7f0008d19640 1 -- 172.21.15.89:0/2073643657 wait complete. 2024-08-19T10:38:52.329 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:38:52.330 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-08-19T10:39:16.762 INFO:tasks.workunit.client.0.smithi190.stderr:Note: switching to '964adda23da9f7b057cb34fdc0b82aeb7ed65324'. 2024-08-19T10:39:16.762 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr:state without impacting any branches by switching back to a branch. 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr: git switch -c 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr:Or undo this operation with: 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr: git switch - 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:16.763 INFO:tasks.workunit.client.0.smithi190.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-19T10:39:16.764 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:16.764 INFO:tasks.workunit.client.0.smithi190.stderr:HEAD is now at 964adda23da Merge pull request #59290 from anthonyeleven/mountpoint 2024-08-19T10:39:16.765 DEBUG:teuthology.orchestra.run.smithi190:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-19T10:39:16.771 INFO:tasks.workunit.client.0.smithi190.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-19T10:39:16.774 INFO:tasks.workunit.client.0.smithi190.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-19T10:39:16.774 INFO:tasks.workunit.client.0.smithi190.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-19T10:39:17.664 INFO:tasks.workunit.client.0.smithi190.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-19T10:39:17.738 INFO:tasks.workunit.client.0.smithi190.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-19T10:39:17.805 INFO:tasks.workunit.client.0.smithi190.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-19T10:39:17.806 INFO:tasks.workunit.client.0.smithi190.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-19T10:39:17.806 INFO:tasks.workunit.client.0.smithi190.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-19T10:39:17.851 INFO:tasks.workunit.client.0.smithi190.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-19T10:39:17.855 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T10:39:17.856 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-19T10:39:17.903 INFO:tasks.workunit:Running workunits matching mon/pool_ops.sh on client.0... 2024-08-19T10:39:17.904 INFO:tasks.workunit:Running workunit mon/pool_ops.sh... 2024-08-19T10:39:17.904 DEBUG:teuthology.orchestra.run.smithi190:workunit test mon/pool_ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=964adda23da9f7b057cb34fdc0b82aeb7ed65324 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/pool_ops.sh 2024-08-19T10:39:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:+ TEST_POOL=testpool1234 2024-08-19T10:39:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create testpool1234 8 --autoscale-mode off 2024-08-19T10:39:18.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient: get_monmap_and_config 2024-08-19T10:39:18.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient: build_initial_monmap 2024-08-19T10:39:18.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient: monmap: 2024-08-19T10:39:18.076 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:18.076 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:18.076 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:18.077274+0000 2024-08-19T10:39:18.076 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:18.077274+0000 2024-08-19T10:39:18.076 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:18.076 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:18.076 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:18.077 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:18.077 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:18.077 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:18.077 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:18.077 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:18.077 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:18.077 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:18.077 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:18.077 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:18.077 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:18.077 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:18.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 1 Processor -- start 2024-08-19T10:39:18.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 1 -- start start 2024-08-19T10:39:18.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient: init 2024-08-19T10:39:18.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:18.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient: _add_conns ranks=[10,3,4,5,2,0,1,7,9,6,8] 2024-08-19T10:39:18.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f87201396a0 0x7f8720139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:18.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient(hunting): picked mon.noname-f con 0x7f87201396a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:39:18.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87200a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:18.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient(hunting): picked mon.noname-g con 0x7f8720139fc0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:18.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87200a6e20 0x7f87200a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient(hunting): picked mon.noname-c con 0x7f87200a6e20 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f87200a4f20 con 0x7f87201396a0 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f87200a83e0 con 0x7f8720139fc0 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f87200a2f30 con 0x7f87200a6e20 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8728484640 10 monclient(hunting): authenticate will time out at 1003.647684s 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8726220640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f87201396a0 msgr2=0x7f8720139a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8726220640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f87201396a0 0x7f8720139a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:18.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8725a1f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87200a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8725a1f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87200a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46892/0 (socket says 172.21.15.190:46892) 2024-08-19T10:39:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8725a1f640 1 -- 172.21.15.190:0/3328512614 learned_addr learned my addr 172.21.15.190:0/3328512614 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8725a1f640 10 monclient(hunting): get_auth_request con 0x7f8720139fc0 auth_method 0 2024-08-19T10:39:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8725a1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8725a1f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8725a1f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8725a1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8725a1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.074+0000 7f8725a1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8726a21640 1 --2- 172.21.15.190:0/3328512614 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87200a6e20 0x7f87200a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8725a1f640 10 monclient(hunting): handle_auth_done global_id 4458 payload 274 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8725a1f640 1 -- 172.21.15.190:0/3328512614 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87200a6e20 msgr2=0x7f87200a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8725a1f640 1 --2- 172.21.15.190:0/3328512614 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87200a6e20 0x7f87200a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8725a1f640 1 -- 172.21.15.190:0/3328512614 >> v2:172.21.15.190:3300/0 conn(0x7f87201396a0 msgr2=0x7f8720139a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8725a1f640 1 --2- 172.21.15.190:0/3328512614 >> v2:172.21.15.190:3300/0 conn(0x7f87201396a0 0x7f8720139a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8725a1f640 10 monclient: _finish_hunting 0 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8725a1f640 1 monclient: found mon.noname-g 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8725a1f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8725a1f640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8725a1f640 1 -- 172.21.15.190:0/3328512614 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87200a35e0 con 0x7f8720139fc0 2024-08-19T10:39:18.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8726a21640 10 monclient: get_auth_request con 0x7f87200a6e20 auth_method 0 2024-08-19T10:39:18.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.078+0000 7f8726a21640 1 --2- 172.21.15.190:0/3328512614 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87200a6e20 0x7f87200a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:39:18.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8725a1f640 1 -- 172.21.15.190:0/3328512614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 msgr2=0x7f87200a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:18.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8725a1f640 1 --2- 172.21.15.190:0/3328512614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87200a68e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f87100098a0 tx=0x7f871001c420 comp rx=0 tx=0).ready entity=mon.3 client_cookie=1342e528cc9de3c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:18.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 1 -- 172.21.15.190:0/3328512614 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87100b0070 con 0x7f8720139fc0 2024-08-19T10:39:18.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:18.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:18.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 10 monclient: dump: 2024-08-19T10:39:18.090 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:18.090 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:18.090 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:18.090 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:18.090 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:18.090 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:18.090 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:18.090 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 10 monclient: _finish_auth 0 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 10 monclient: _check_auth_tickets 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:18.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 1 -- 172.21.15.190:0/3328512614 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87100aa810 con 0x7f8720139fc0 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 1 -- 172.21.15.190:0/3328512614 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87100a93e0 con 0x7f8720139fc0 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f872521e640 10 monclient: dump: 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:18.092 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 5 monclient: authenticate success, global_id 4458 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:18.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 4 set_mon_vals no callback set 2024-08-19T10:39:18.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient: shutdown 2024-08-19T10:39:18.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:18.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 -- 172.21.15.190:0/3328512614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 msgr2=0x7f87200a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:18.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 --2- 172.21.15.190:0/3328512614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87200a68e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f87100098a0 tx=0x7f871001c420 comp rx=0 tx=0).stop 2024-08-19T10:39:18.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 -- 172.21.15.190:0/3328512614 shutdown_connections 2024-08-19T10:39:18.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 --2- 172.21.15.190:0/3328512614 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87200a6e20 0x7f87200a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:18.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 --2- 172.21.15.190:0/3328512614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87200a68e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:18.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 --2- 172.21.15.190:0/3328512614 >> v2:172.21.15.190:3300/0 conn(0x7f87201396a0 0x7f8720139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:18.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 -- 172.21.15.190:0/3328512614 >> 172.21.15.190:0/3328512614 conn(0x7f87200abee0 msgr2=0x7f87200ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:18.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 -- 172.21.15.190:0/3328512614 shutdown_connections 2024-08-19T10:39:18.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 -- 172.21.15.190:0/3328512614 wait complete. 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient: build_initial_monmap 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient: monmap: 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:18.095 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:18.096 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:18.096 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:18.096 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:18.096 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:18.096 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:18.096 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:18.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 Processor -- start 2024-08-19T10:39:18.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 -- start start 2024-08-19T10:39:18.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient: init 2024-08-19T10:39:18.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:18.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient: _add_conns ranks=[4,0,3,7,6,1,8,5,2] 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87200a6e20 0x7f87200a2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient(hunting): picked mon.noname-c con 0x7f87200a6e20 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87201396a0 0x7f87200a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient(hunting): picked mon.noname-a con 0x7f87201396a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87201d1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient(hunting): picked mon.noname-g con 0x7f8720139fc0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f872013a3a0 con 0x7f87201396a0 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8720143390 con 0x7f8720139fc0 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f87200a2ad0 con 0x7f87200a6e20 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:18.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8728484640 10 monclient(hunting): authenticate will time out at 1003.654767s 2024-08-19T10:39:18.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87201d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:18.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87201d1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46904/0 (socket says 172.21.15.190:46904) 2024-08-19T10:39:18.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 1 -- 172.21.15.190:0/3335531205 learned_addr learned my addr 172.21.15.190:0/3335531205 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:18.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 10 monclient(hunting): get_auth_request con 0x7f8720139fc0 auth_method 0 2024-08-19T10:39:18.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:18.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:18.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:18.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:18.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:18.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:18.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726220640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87200a6e20 0x7f87200a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8725a1f640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87201396a0 0x7f87200a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726220640 10 monclient(hunting): get_auth_request con 0x7f87200a6e20 auth_method 0 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726220640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726220640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726220640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8725a1f640 10 monclient(hunting): get_auth_request con 0x7f87201396a0 auth_method 0 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8725a1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8725a1f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8725a1f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 10 monclient(hunting): handle_auth_done global_id 4467 payload 824 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 1 -- 172.21.15.190:0/3335531205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87200a6e20 msgr2=0x7f87200a2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87200a6e20 0x7f87200a2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:18.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 1 -- 172.21.15.190:0/3335531205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87201396a0 msgr2=0x7f87200a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87201396a0 0x7f87200a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 10 monclient: _finish_hunting 0 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 1 monclient: found mon.noname-g 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.082+0000 7f8726a21640 1 -- 172.21.15.190:0/3335531205 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872013fb90 con 0x7f8720139fc0 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8726a21640 1 -- 172.21.15.190:0/3335531205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 msgr2=0x7f87201d1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8726a21640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87201d1630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f871800cf20 tx=0x7f871800a7d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 1 -- 172.21.15.190:0/3335531205 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f871801a070 con 0x7f8720139fc0 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:18.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 10 monclient: dump: 2024-08-19T10:39:18.101 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:18.101 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:18.101 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:18.101 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:18.101 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:18.101 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:18.101 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:18.101 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:18.101 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:18.101 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:18.101 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 10 monclient: _finish_auth 0 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 1 -- 172.21.15.190:0/3335531205 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871800dcc0 con 0x7f8720139fc0 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8728484640 5 monclient: authenticate success, global_id 4467 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8728484640 10 monclient: _renew_subs 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8728484640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:18.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8728484640 1 -- 172.21.15.190:0/3335531205 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872009f270 con 0x7f8720139fc0 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 1 -- 172.21.15.190:0/3335531205 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87180137b0 con 0x7f8720139fc0 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8717fff640 4 set_mon_vals no callback set 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 10 monclient: dump: 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:18.103 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8728484640 10 monclient: _renew_subs 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8728484640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8728484640 1 -- 172.21.15.190:0/3335531205 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f87201357f0 con 0x7f8720139fc0 2024-08-19T10:39:18.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8728484640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:18.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8728484640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:18.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8728484640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:18.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8728484640 1 -- 172.21.15.190:0/3335531205 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8720139a80 con 0x7f8720139fc0 2024-08-19T10:39:18.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 1 -- 172.21.15.190:0/3335531205 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8718013340 con 0x7f8720139fc0 2024-08-19T10:39:18.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f870403be00 0x7f870403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:18.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8716ffd640 1 -- 172.21.15.190:0/3335531205 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4374+0+0 (secure 0 0 0) 0x7f871804f0f0 con 0x7f8720139fc0 2024-08-19T10:39:18.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8726220640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f870403be00 0x7f870403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:18.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.086+0000 7f8726220640 10 monclient: get_auth_request con 0x7f870403be00 auth_method 0 2024-08-19T10:39:18.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.090+0000 7f8726220640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f870403be00 0x7f870403e2c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f871c006fd0 tx=0x7f871c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:18.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.090+0000 7f8716ffd640 1 -- 172.21.15.190:0/3335531205 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f871800dae0 con 0x7f8720139fc0 2024-08-19T10:39:18.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.090+0000 7f8716ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:18.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.090+0000 7f8716ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:18.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.262+0000 7f8728484640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-08-19T10:39:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.262+0000 7f8728484640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-08-19T10:39:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.262+0000 7f8728484640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.262+0000 7f8728484640 1 -- 172.21.15.190:0/3335531205 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"} v 0) -- 0x7f87200ac2c0 con 0x7f8720139fc0 2024-08-19T10:39:18.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.906+0000 7f8716ffd640 1 -- 172.21.15.190:0/3335531205 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}]=0 pool 'testpool1234' created v20) ==== 152+0+0 (secure 0 0 0) 0x7f8718013a10 con 0x7f8720139fc0 2024-08-19T10:39:18.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.906+0000 7f8716ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-08-19T10:39:18.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:18.906+0000 7f8716ffd640 10 monclient: _finish_command 2 = system:0 pool 'testpool1234' created 2024-08-19T10:39:19.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.010+0000 7f8728484640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-08-19T10:39:19.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.010+0000 7f8728484640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-08-19T10:39:19.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.010+0000 7f8728484640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:19.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.010+0000 7f8728484640 1 -- 172.21.15.190:0/3335531205 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"} v 0) -- 0x7f87200008e0 con 0x7f8720139fc0 2024-08-19T10:39:19.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8716ffd640 1 -- 172.21.15.190:0/3335531205 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}]=0 pool 'testpool1234' already exists v20) ==== 159+0+0 (secure 0 0 0) 0x7f871801c310 con 0x7f8720139fc0 2024-08-19T10:39:19.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8716ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "testpool1234", "pg_num": 8, "autoscale_mode": "off"}] 2024-08-19T10:39:19.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8716ffd640 10 monclient: _finish_command 3 = system:0 pool 'testpool1234' already exists 2024-08-19T10:39:19.015 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'testpool1234' already exists 2024-08-19T10:39:19.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 -- 172.21.15.190:0/3335531205 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f870403be00 msgr2=0x7f870403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:19.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f870403be00 0x7f870403e2c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f871c006fd0 tx=0x7f871c008040 comp rx=0 tx=0).stop 2024-08-19T10:39:19.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 10 monclient: shutdown 2024-08-19T10:39:19.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:19.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 -- 172.21.15.190:0/3335531205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 msgr2=0x7f87201d1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:19.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87201d1630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f871800cf20 tx=0x7f871800a7d0 comp rx=0 tx=0).stop 2024-08-19T10:39:19.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 -- 172.21.15.190:0/3335531205 shutdown_connections 2024-08-19T10:39:19.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f870403be00 0x7f870403e2c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:19.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8720139fc0 0x7f87201d1630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:19.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87201396a0 0x7f87200a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:19.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 --2- 172.21.15.190:0/3335531205 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87200a6e20 0x7f87200a2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:19.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 -- 172.21.15.190:0/3335531205 >> 172.21.15.190:0/3335531205 conn(0x7f87200abee0 msgr2=0x7f8720134ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:19.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 -- 172.21.15.190:0/3335531205 shutdown_connections 2024-08-19T10:39:19.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.014+0000 7f8728484640 1 -- 172.21.15.190:0/3335531205 wait complete. 2024-08-19T10:39:19.033 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set testpool1234 pg_num_min 2 2024-08-19T10:39:19.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient: get_monmap_and_config 2024-08-19T10:39:19.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient: build_initial_monmap 2024-08-19T10:39:19.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient: monmap: 2024-08-19T10:39:19.154 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:19.155 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:19.155 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:19.155867+0000 2024-08-19T10:39:19.155 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:19.155867+0000 2024-08-19T10:39:19.155 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:19.155 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:19.155 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:19.155 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:19.155 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:19.155 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:19.155 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:19.156 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:19.156 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:19.156 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:19.156 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:19.156 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:19.156 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:19.156 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:19.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 1 Processor -- start 2024-08-19T10:39:19.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 1 -- start start 2024-08-19T10:39:19.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient: init 2024-08-19T10:39:19.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient: _add_conns ranks=[6,4,0,10,9,8,2,3,1,7,5] 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ee40a68e0 0x7f1ee40a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient(hunting): picked mon.noname-d con 0x7f1ee40a68e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a6cc0 0x7f1ee40a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient(hunting): picked mon.noname-c con 0x7f1ee40a6cc0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1ee41396a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ee40a86c0 con 0x7f1ee41396a0 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f1ee40a8410 con 0x7f1ee40a6cc0 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1ee40a63a0 con 0x7f1ee40a68e0 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:19.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1eeb38c640 10 monclient(hunting): authenticate will time out at 1004.725729s 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ee40a68e0 0x7f1ee40a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9929640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1ee41396a0 0x7f1ee4139af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:39180/0 (socket says 172.21.15.190:39180) 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9929640 1 -- 172.21.15.190:0/3205576129 learned_addr learned my addr 172.21.15.190:0/3205576129 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee8927640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a6cc0 0x7f1ee40a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 10 monclient(hunting): get_auth_request con 0x7f1ee40a68e0 auth_method 0 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee8927640 10 monclient(hunting): get_auth_request con 0x7f1ee40a6cc0 auth_method 0 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee8927640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee8927640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:19.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee8927640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee8927640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee8927640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee8927640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 10 monclient(hunting): handle_auth_done global_id 4407 payload 274 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 1 -- 172.21.15.190:0/3205576129 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a6cc0 msgr2=0x7f1ee40a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 1 --2- 172.21.15.190:0/3205576129 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a6cc0 0x7f1ee40a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 1 -- 172.21.15.190:0/3205576129 >> v1:172.21.15.89:6789/0 conn(0x7f1ee41396a0 legacy=0x7f1ee4139af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 10 monclient: _finish_hunting 0 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 1 monclient: found mon.noname-d 2024-08-19T10:39:19.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:19.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:19.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.154+0000 7f1ee9128640 1 -- 172.21.15.190:0/3205576129 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee40a4680 con 0x7f1ee40a68e0 2024-08-19T10:39:19.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1ee9128640 1 -- 172.21.15.190:0/3205576129 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ee40a68e0 msgr2=0x7f1ee40a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:19.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1ee9128640 1 --2- 172.21.15.190:0/3205576129 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ee40a68e0 0x7f1ee40a5760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00097f0 tx=0x7f1ee001c080 comp rx=0 tx=0).ready entity=mon.6 client_cookie=40b4f955cfe81c2f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:19.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 1 -- 172.21.15.190:0/3205576129 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ee00a78c0 con 0x7f1ee40a68e0 2024-08-19T10:39:19.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:19.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:19.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 10 monclient: dump: 2024-08-19T10:39:19.164 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:19.165 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:19.165 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:19.165 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:19.165 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:19.165 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:19.165 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:19.165 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:19.165 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:19.165 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:19.165 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:19.166 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:19.166 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:19.166 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:19.166 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:19.166 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:19.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 10 monclient: _finish_auth 0 2024-08-19T10:39:19.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:19.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 1 -- 172.21.15.190:0/3205576129 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee00a7a80 con 0x7f1ee40a68e0 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 1 -- 172.21.15.190:0/3205576129 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ee00a7c70 con 0x7f1ee40a68e0 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1edbfff640 10 monclient: dump: 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:19.167 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 5 monclient: authenticate success, global_id 4407 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 4 set_mon_vals no callback set 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient: shutdown 2024-08-19T10:39:19.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:19.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3205576129 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ee40a68e0 msgr2=0x7f1ee40a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:19.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 --2- 172.21.15.190:0/3205576129 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ee40a68e0 0x7f1ee40a5760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00097f0 tx=0x7f1ee001c080 comp rx=0 tx=0).stop 2024-08-19T10:39:19.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3205576129 shutdown_connections 2024-08-19T10:39:19.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 --2- 172.21.15.190:0/3205576129 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a6cc0 0x7f1ee40a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:19.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 --2- 172.21.15.190:0/3205576129 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ee40a68e0 0x7f1ee40a5760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:19.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3205576129 >> 172.21.15.190:0/3205576129 conn(0x7f1ee40abee0 msgr2=0x7f1ee40ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:19.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3205576129 shutdown_connections 2024-08-19T10:39:19.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3205576129 wait complete. 2024-08-19T10:39:19.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient: build_initial_monmap 2024-08-19T10:39:19.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient: monmap: 2024-08-19T10:39:19.174 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:19.174 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:19.174 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:19.174 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:19.174 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:19.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 Processor -- start 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 -- start start 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient: init 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient: _add_conns ranks=[4,1,0,5,7,2,6,3,8] 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a68e0 0x7f1ee41380f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient(hunting): picked mon.noname-c con 0x7f1ee40a68e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ee40a6cc0 0x7f1ee4138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient(hunting): picked mon.noname-f con 0x7f1ee40a6cc0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ee41396a0 0x7f1ee413dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient(hunting): picked mon.noname-a con 0x7f1ee41396a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4144cd0 con 0x7f1ee41396a0 2024-08-19T10:39:19.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4144b50 con 0x7f1ee40a6cc0 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f1ee4144e50 con 0x7f1ee40a68e0 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.162+0000 7f1eeb38c640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1eeb38c640 10 monclient(hunting): authenticate will time out at 1004.736013s 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee8927640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ee40a6cc0 msgr2=0x7f1ee4138630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee8927640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ee40a6cc0 0x7f1ee4138630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a68e0 0x7f1ee41380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a68e0 0x7f1ee41380f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:40860/0 (socket says 172.21.15.190:40860) 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 1 -- 172.21.15.190:0/3616506866 learned_addr learned my addr 172.21.15.190:0/3616506866 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 10 monclient(hunting): get_auth_request con 0x7f1ee40a68e0 auth_method 0 2024-08-19T10:39:19.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9929640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ee41396a0 0x7f1ee413dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9929640 10 monclient(hunting): get_auth_request con 0x7f1ee41396a0 auth_method 0 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9929640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9929640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9929640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9929640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9929640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:19.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9929640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 10 monclient(hunting): handle_auth_done global_id 4576 payload 824 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 1 -- 172.21.15.190:0/3616506866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ee40a6cc0 msgr2=0x7f1ee4138630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ee40a6cc0 0x7f1ee4138630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 1 -- 172.21.15.190:0/3616506866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ee41396a0 msgr2=0x7f1ee413dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ee41396a0 0x7f1ee413dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 10 monclient: _finish_hunting 0 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 1 monclient: found mon.noname-c 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 1 -- 172.21.15.190:0/3616506866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee40aadf0 con 0x7f1ee40a68e0 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 1 -- 172.21.15.190:0/3616506866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a68e0 msgr2=0x7f1ee41380f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:19.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ee9128640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a68e0 0x7f1ee41380f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ee001c920 tx=0x7f1ee0019140 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:19.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 1 -- 172.21.15.190:0/3616506866 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ee00bd070 con 0x7f1ee40a68e0 2024-08-19T10:39:19.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:19.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:19.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 10 monclient: dump: 2024-08-19T10:39:19.180 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:19.180 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:19.180 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:19.180 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:19.180 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:19.180 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:19.180 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:39:19.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 1 -- 172.21.15.190:0/3616506866 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee00a78a0 con 0x7f1ee40a68e0 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 1 -- 172.21.15.190:0/3616506866 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ee0004460 con 0x7f1ee40a68e0 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1ed9ffb640 10 monclient: dump: 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:19.182 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1edb7fe640 4 set_mon_vals no callback set 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1eeb38c640 5 monclient: authenticate success, global_id 4576 2024-08-19T10:39:19.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1eeb38c640 10 monclient: _renew_subs 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1eeb38c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3616506866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee4122ad0 con 0x7f1ee40a68e0 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1eeb38c640 10 monclient: _renew_subs 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1eeb38c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.166+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3616506866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee4140dc0 con 0x7f1ee40a68e0 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.170+0000 7f1eeb38c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.170+0000 7f1eeb38c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.170+0000 7f1eeb38c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.170+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3616506866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee40a5760 con 0x7f1ee40a68e0 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.170+0000 7f1ed9ffb640 1 -- 172.21.15.190:0/3616506866 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1ee00a5040 con 0x7f1ee40a68e0 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.170+0000 7f1ed9ffb640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1ec803be00 0x7f1ec803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:19.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.170+0000 7f1ed9ffb640 1 -- 172.21.15.190:0/3616506866 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(20..20 src has 1..20) ==== 4697+0+0 (secure 0 0 0) 0x7f1ee00e8660 con 0x7f1ee40a68e0 2024-08-19T10:39:19.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.170+0000 7f1ee8927640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1ec803be00 0x7f1ec803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:19.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.170+0000 7f1ee8927640 10 monclient: get_auth_request con 0x7f1ec803be00 auth_method 0 2024-08-19T10:39:19.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.174+0000 7f1ee8927640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1ec803be00 0x7f1ec803e2c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ee413f140 tx=0x7f1ed4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:19.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.182+0000 7f1ed9ffb640 1 -- 172.21.15.190:0/3616506866 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1ee40aadf0 con 0x7f1ee40a68e0 2024-08-19T10:39:19.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.182+0000 7f1ed9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:19.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.182+0000 7f1ed9ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:19.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.354+0000 7f1eeb38c640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-08-19T10:39:19.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.354+0000 7f1eeb38c640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-08-19T10:39:19.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.354+0000 7f1eeb38c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:19.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.354+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3616506866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"} v 0) -- 0x7f1ee406c9c0 con 0x7f1ee40a68e0 2024-08-19T10:39:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.906+0000 7f1ed9ffb640 1 -- 172.21.15.190:0/3616506866 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}]=0 set pool 3 pg_num_min to 2 v21) ==== 143+0+0 (secure 0 0 0) 0x7f1ee00ab030 con 0x7f1ee40a68e0 2024-08-19T10:39:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.906+0000 7f1ed9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-08-19T10:39:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:19.906+0000 7f1ed9ffb640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_min to 2 2024-08-19T10:39:20.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.018+0000 7f1eeb38c640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-08-19T10:39:20.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.018+0000 7f1eeb38c640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-08-19T10:39:20.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.018+0000 7f1eeb38c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:20.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.018+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3616506866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"} v 0) -- 0x7f1ee41227f0 con 0x7f1ee40a68e0 2024-08-19T10:39:20.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.930+0000 7f1ed9ffb640 1 -- 172.21.15.190:0/3616506866 <== mon.4 v2:172.21.15.89:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}]=0 set pool 3 pg_num_min to 2 v22) ==== 143+0+0 (secure 0 0 0) 0x7f1ee00160b0 con 0x7f1ee40a68e0 2024-08-19T10:39:20.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.930+0000 7f1ed9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "2"}] 2024-08-19T10:39:20.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.930+0000 7f1ed9ffb640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_min to 2 2024-08-19T10:39:20.935 INFO:tasks.workunit.client.0.smithi190.stderr:set pool 3 pg_num_min to 2 2024-08-19T10:39:20.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3616506866 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1ec803be00 msgr2=0x7f1ec803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:20.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1ec803be00 0x7f1ec803e2c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ee413f140 tx=0x7f1ed4009190 comp rx=0 tx=0).stop 2024-08-19T10:39:20.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 10 monclient: shutdown 2024-08-19T10:39:20.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:20.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3616506866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a68e0 msgr2=0x7f1ee41380f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:20.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a68e0 0x7f1ee41380f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ee001c920 tx=0x7f1ee0019140 comp rx=0 tx=0).stop 2024-08-19T10:39:20.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3616506866 shutdown_connections 2024-08-19T10:39:20.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1ec803be00 0x7f1ec803e2c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:20.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ee41396a0 0x7f1ee413dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:20.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ee40a6cc0 0x7f1ee4138630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:20.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 --2- 172.21.15.190:0/3616506866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1ee40a68e0 0x7f1ee41380f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:20.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3616506866 >> 172.21.15.190:0/3616506866 conn(0x7f1ee40abee0 msgr2=0x7f1ee40a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:20.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3616506866 shutdown_connections 2024-08-19T10:39:20.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:20.934+0000 7f1eeb38c640 1 -- 172.21.15.190:0/3616506866 wait complete. 2024-08-19T10:39:20.955 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool get testpool1234 pg_num_min 2024-08-19T10:39:20.956 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 2 2024-08-19T10:39:21.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.074+0000 7fc4d9b2d640 10 monclient: get_monmap_and_config 2024-08-19T10:39:21.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.074+0000 7fc4d9b2d640 10 monclient: build_initial_monmap 2024-08-19T10:39:21.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.074+0000 7fc4d9b2d640 10 monclient: monmap: 2024-08-19T10:39:21.077 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:21.077 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:21.077 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:21.078288+0000 2024-08-19T10:39:21.077 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:21.078288+0000 2024-08-19T10:39:21.077 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:21.077 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.077 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:21.077 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:21.078 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:21.078 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:21.078 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:21.078 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:21.078 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:21.078 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:21.078 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:21.078 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:21.078 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:21.078 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.074+0000 7fc4d9b2d640 1 Processor -- start 2024-08-19T10:39:21.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.074+0000 7fc4d9b2d640 1 -- start start 2024-08-19T10:39:21.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.074+0000 7fc4d9b2d640 10 monclient: init 2024-08-19T10:39:21.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:21.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 10 monclient: _add_conns ranks=[2,1,9,8,6,4,7,3,5,10,0] 2024-08-19T10:39:21.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4d40a6740 0x7fc4d40a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 10 monclient(hunting): picked mon.noname-b con 0x7fc4d40a6740 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:21.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc4d40a5bc0 addr v1:172.21.15.190:6789/0 2024-08-19T10:39:21.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc4d40a7230 0x7fc4d4140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 10 monclient(hunting): picked mon.noname-a con 0x7fc4d40a7230 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:21.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d40a3740 con 0x7fc4d40a5bc0 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d40a4c20 con 0x7fc4d40a7230 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d40a3530 con 0x7fc4d40a6740 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d9b2d640 10 monclient(hunting): authenticate will time out at 1006.648385s 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d2ffd640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fc4d40a5bc0 legacy=0x7fc4d40a6b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4d40a6740 0x7fc4d40a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d3fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc4d40a7230 0x7fc4d4140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4d40a6740 0x7fc4d40a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:44298/0 (socket says 172.21.15.190:44298) 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d3fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc4d40a7230 0x7fc4d4140ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:50386/0 (socket says 172.21.15.190:50386) 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 1 -- 172.21.15.190:0/3811772398 learned_addr learned my addr 172.21.15.190:0/3811772398 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 10 monclient(hunting): get_auth_request con 0x7fc4d40a6740 auth_method 0 2024-08-19T10:39:21.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d3fff640 10 monclient(hunting): get_auth_request con 0x7fc4d40a7230 auth_method 0 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:21.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 10 monclient(hunting): handle_auth_done global_id 4448 payload 274 2024-08-19T10:39:21.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 1 -- 172.21.15.190:0/3811772398 >> v2:172.21.15.89:3300/0 conn(0x7fc4d40a7230 msgr2=0x7fc4d4140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:21.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 1 --2- 172.21.15.190:0/3811772398 >> v2:172.21.15.89:3300/0 conn(0x7fc4d40a7230 0x7fc4d4140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 1 -- 172.21.15.190:0/3811772398 >> v1:172.21.15.190:6789/0 conn(0x7fc4d40a5bc0 legacy=0x7fc4d40a6b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:21.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:39:21.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 1 monclient: found mon.noname-b 2024-08-19T10:39:21.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:21.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:21.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d37fe640 1 -- 172.21.15.190:0/3811772398 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d40a3240 con 0x7fc4d40a6740 2024-08-19T10:39:21.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.078+0000 7fc4d3fff640 1 --2- 172.21.15.190:0/3811772398 >> v2:172.21.15.89:3300/0 conn(0x7fc4d40a7230 0x7fc4d4140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:21.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d37fe640 1 -- 172.21.15.190:0/3811772398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4d40a6740 msgr2=0x7fc4d40a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:21.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d37fe640 1 --2- 172.21.15.190:0/3811772398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4d40a6740 0x7fc4d40a5680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0098a0 tx=0x7fc4bc01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcf699af8ff0f005 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:21.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 1 -- 172.21.15.190:0/3811772398 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4bc0b1070 con 0x7fc4d40a6740 2024-08-19T10:39:21.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:21.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:21.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 10 monclient: dump: 2024-08-19T10:39:21.086 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:21.086 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:21.086 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.086 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.086 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:21.086 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:21.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 1 -- 172.21.15.190:0/3811772398 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4bc0aa980 con 0x7fc4d40a6740 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 5 monclient: authenticate success, global_id 4448 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 1 -- 172.21.15.190:0/3811772398 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4bc0ab3a0 con 0x7fc4d40a6740 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d27fc640 10 monclient: dump: 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:21.088 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 4 set_mon_vals no callback set 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 10 monclient: shutdown 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3811772398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4d40a6740 msgr2=0x7fc4d40a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 1 --2- 172.21.15.190:0/3811772398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4d40a6740 0x7fc4d40a5680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0098a0 tx=0x7fc4bc01c400 comp rx=0 tx=0).stop 2024-08-19T10:39:21.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3811772398 shutdown_connections 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 1 --2- 172.21.15.190:0/3811772398 >> v2:172.21.15.89:3300/0 conn(0x7fc4d40a7230 0x7fc4d4140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 1 --2- 172.21.15.190:0/3811772398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4d40a6740 0x7fc4d40a5680 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3811772398 >> 172.21.15.190:0/3811772398 conn(0x7fc4d40abd40 msgr2=0x7fc4d40ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3811772398 shutdown_connections 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3811772398 wait complete. 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 10 monclient: build_initial_monmap 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.082+0000 7fc4d9b2d640 10 monclient: monmap: 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:21.090 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.091 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:21.091 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:21.091 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:21.091 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:21.091 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:21.091 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:21.091 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:21.091 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:21.091 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:21.091 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 1 Processor -- start 2024-08-19T10:39:21.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 1 -- start start 2024-08-19T10:39:21.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 10 monclient: init 2024-08-19T10:39:21.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:21.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 10 monclient: _add_conns ranks=[1,3,8,7,2,4,0,5,6] 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d40a6740 0x7fc4d4139020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 10 monclient(hunting): picked mon.noname-f con 0x7fc4d40a6740 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d40a7230 0x7fc4d4139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 10 monclient(hunting): picked mon.noname-g con 0x7fc4d40a7230 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4d413e820 0x7fc4d413ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 10 monclient(hunting): picked mon.noname-e con 0x7fc4d413e820 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4143990 con 0x7fc4d40a6740 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d4143810 con 0x7fc4d40a7230 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc4d4143b10 con 0x7fc4d413e820 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:21.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d9b2d640 10 monclient(hunting): authenticate will time out at 1006.656884s 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d37fe640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d40a6740 msgr2=0x7fc4d4139020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d40a6740 0x7fc4d4139020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d40a7230 0x7fc4d4139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d40a7230 0x7fc4d4139560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46908/0 (socket says 172.21.15.190:46908) 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 1 -- 172.21.15.190:0/3281608156 learned_addr learned my addr 172.21.15.190:0/3281608156 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 10 monclient(hunting): get_auth_request con 0x7fc4d40a7230 auth_method 0 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d3fff640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4d413e820 0x7fc4d413ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d3fff640 10 monclient(hunting): get_auth_request con 0x7fc4d413e820 auth_method 0 2024-08-19T10:39:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 10 monclient(hunting): handle_auth_done global_id 4476 payload 824 2024-08-19T10:39:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 1 -- 172.21.15.190:0/3281608156 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4d413e820 msgr2=0x7fc4d413ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4d413e820 0x7fc4d413ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 1 -- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d40a6740 msgr2=0x7fc4d4139020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d40a6740 0x7fc4d4139020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 1 monclient: found mon.noname-g 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 1 -- 172.21.15.190:0/3281608156 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d4123bf0 con 0x7fc4d40a7230 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 1 -- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d40a7230 msgr2=0x7fc4d4139560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4d2ffd640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d40a7230 0x7fc4d4139560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400b580 tx=0x7fc4c400ba40 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.086+0000 7fc4cbfff640 1 -- 172.21.15.190:0/3281608156 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4c4016070 con 0x7fc4d40a7230 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 10 monclient: dump: 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:21.099 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:21.100 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 10 monclient: _finish_auth 0 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 1 -- 172.21.15.190:0/3281608156 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c4003de0 con 0x7fc4d40a7230 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d9b2d640 5 monclient: authenticate success, global_id 4476 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d9b2d640 10 monclient: _renew_subs 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d9b2d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3281608156 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d4123ed0 con 0x7fc4d40a7230 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 1 -- 172.21.15.190:0/3281608156 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4c400f3e0 con 0x7fc4d40a7230 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:21.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d1ffb640 4 set_mon_vals no callback set 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 10 monclient: dump: 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:21.102 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:21.103 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:21.103 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:21.103 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:21.103 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:21.103 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d9b2d640 10 monclient: _renew_subs 2024-08-19T10:39:21.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d9b2d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:21.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3281608156 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d40a90b0 con 0x7fc4d40a7230 2024-08-19T10:39:21.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d9b2d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:21.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d9b2d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:21.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d9b2d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:21.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3281608156 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d40a5680 con 0x7fc4d40a7230 2024-08-19T10:39:21.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 1 -- 172.21.15.190:0/3281608156 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc4c40050d0 con 0x7fc4d40a7230 2024-08-19T10:39:21.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4b403be00 0x7fc4b403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4cbfff640 1 -- 172.21.15.190:0/3281608156 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4713+0+0 (secure 0 0 0) 0x7fc4c404dc10 con 0x7fc4d40a7230 2024-08-19T10:39:21.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d37fe640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4b403be00 0x7fc4b403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:21.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.090+0000 7fc4d37fe640 10 monclient: get_auth_request con 0x7fc4b403be00 auth_method 0 2024-08-19T10:39:21.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.094+0000 7fc4d37fe640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4b403be00 0x7fc4b403e2c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0ae000 tx=0x7fc4bc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:21.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.094+0000 7fc4cbfff640 1 -- 172.21.15.190:0/3281608156 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc4c4018310 con 0x7fc4d40a7230 2024-08-19T10:39:21.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.094+0000 7fc4cbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:21.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.094+0000 7fc4cbfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.266+0000 7fc4d9b2d640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-08-19T10:39:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.266+0000 7fc4d9b2d640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-08-19T10:39:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.266+0000 7fc4d9b2d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.266+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3281608156 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) -- 0x7fc4d406c8d0 con 0x7fc4d40a7230 2024-08-19T10:39:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.266+0000 7fc4cbfff640 1 -- 172.21.15.190:0/3281608156 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=0 v22) ==== 105+0+14 (secure 0 0 0) 0x7fc4c400f580 con 0x7fc4d40a7230 2024-08-19T10:39:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.266+0000 7fc4cbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-08-19T10:39:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.266+0000 7fc4cbfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:39:21.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.362+0000 7fc4d9b2d640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-08-19T10:39:21.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.362+0000 7fc4d9b2d640 10 monclient: _send_command 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-08-19T10:39:21.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.362+0000 7fc4d9b2d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:21.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.362+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3281608156 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) -- 0x7fc4d4139aa0 con 0x7fc4d40a7230 2024-08-19T10:39:21.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4cbfff640 1 -- 172.21.15.190:0/3281608156 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=0 v22) ==== 105+0+14 (secure 0 0 0) 0x7fc4c400e340 con 0x7fc4d40a7230 2024-08-19T10:39:21.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4cbfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-08-19T10:39:21.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4cbfff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:39:21.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4b403be00 msgr2=0x7fc4b403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:21.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4b403be00 0x7fc4b403e2c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0ae000 tx=0x7fc4bc0a7000 comp rx=0 tx=0).stop 2024-08-19T10:39:21.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 10 monclient: shutdown 2024-08-19T10:39:21.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:21.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d40a7230 msgr2=0x7fc4d4139560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:21.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d40a7230 0x7fc4d4139560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400b580 tx=0x7fc4c400ba40 comp rx=0 tx=0).stop 2024-08-19T10:39:21.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3281608156 shutdown_connections 2024-08-19T10:39:21.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4b403be00 0x7fc4b403e2c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4d413e820 0x7fc4d413ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d40a7230 0x7fc4d4139560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 --2- 172.21.15.190:0/3281608156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d40a6740 0x7fc4d4139020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3281608156 >> 172.21.15.190:0/3281608156 conn(0x7fc4d40abd40 msgr2=0x7fc4d40aaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:21.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3281608156 shutdown_connections 2024-08-19T10:39:21.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.366+0000 7fc4d9b2d640 1 -- 172.21.15.190:0/3281608156 wait complete. 2024-08-19T10:39:21.383 INFO:tasks.workunit.client.0.smithi190.stdout:pg_num_min: 2 2024-08-19T10:39:21.384 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set testpool1234 pg_num_max 33 2024-08-19T10:39:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient: get_monmap_and_config 2024-08-19T10:39:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient: build_initial_monmap 2024-08-19T10:39:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient: monmap: 2024-08-19T10:39:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:21.507788+0000 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:21.507788+0000 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:21.508 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 1 Processor -- start 2024-08-19T10:39:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 1 -- start start 2024-08-19T10:39:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient: init 2024-08-19T10:39:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient: _add_conns ranks=[8,10,2,0,4,1,5,3,7,9,6] 2024-08-19T10:39:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f271c0a58d0 0x7f271c0a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient(hunting): picked mon.noname-e con 0x7f271c0a58d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f271c06e7e0 0x7f271c0a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient(hunting): picked mon.noname-f con 0x7f271c06e7e0 addr v2:172.21.15.190:3300/0 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f271c1396a0 0x7f271c139b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient(hunting): picked mon.noname-b con 0x7f271c1396a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f271c06e0e0 con 0x7f271c06e7e0 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f271c0a6730 con 0x7f271c1396a0 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f271c06fab0 con 0x7f271c0a58d0 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f272225f640 10 monclient(hunting): authenticate will time out at 1007.077574s 2024-08-19T10:39:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271affd640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f271c06e7e0 msgr2=0x7f271c0a61f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271affd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f271c06e7e0 0x7f271c0a61f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f271c1396a0 0x7f271c139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f271c0a58d0 0x7f271c0a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f271c1396a0 0x7f271c139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:44306/0 (socket says 172.21.15.190:44306) 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 1 -- 172.21.15.190:0/2501544082 learned_addr learned my addr 172.21.15.190:0/2501544082 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 10 monclient(hunting): get_auth_request con 0x7f271c1396a0 auth_method 0 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271b7fe640 10 monclient(hunting): get_auth_request con 0x7f271c0a58d0 auth_method 0 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 10 monclient(hunting): handle_auth_done global_id 4457 payload 274 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 1 -- 172.21.15.190:0/2501544082 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f271c0a58d0 msgr2=0x7f271c0a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 1 --2- 172.21.15.190:0/2501544082 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f271c0a58d0 0x7f271c0a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 1 -- 172.21.15.190:0/2501544082 >> v2:172.21.15.190:3300/0 conn(0x7f271c06e7e0 msgr2=0x7f271c0a61f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 1 --2- 172.21.15.190:0/2501544082 >> v2:172.21.15.190:3300/0 conn(0x7f271c06e7e0 0x7f271c0a61f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 1 monclient: found mon.noname-b 2024-08-19T10:39:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:21.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:21.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.506+0000 7f271bfff640 1 -- 172.21.15.190:0/2501544082 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c06de60 con 0x7f271c1396a0 2024-08-19T10:39:21.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271bfff640 1 -- 172.21.15.190:0/2501544082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f271c1396a0 msgr2=0x7f271c139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:21.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271bfff640 1 --2- 172.21.15.190:0/2501544082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f271c1396a0 0x7f271c139b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2714009e50 tx=0x7f271401c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56aebdf279b75f1c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:21.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 1 -- 172.21.15.190:0/2501544082 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f27140090d0 con 0x7f271c1396a0 2024-08-19T10:39:21.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:21.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:21.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 10 monclient: dump: 2024-08-19T10:39:21.517 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:21.517 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:21.517 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.517 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.517 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:21.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f272225f640 5 monclient: authenticate success, global_id 4457 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f272225f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 1 -- 172.21.15.190:0/2501544082 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27140a5ad0 con 0x7f271c1396a0 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 1 -- 172.21.15.190:0/2501544082 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f27140a5c80 con 0x7f271c1396a0 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f271a7fc640 10 monclient: dump: 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.519 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f272225f640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:21.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.510+0000 7f272225f640 4 set_mon_vals no callback set 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient: shutdown 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 -- 172.21.15.190:0/2501544082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f271c1396a0 msgr2=0x7f271c139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 --2- 172.21.15.190:0/2501544082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f271c1396a0 0x7f271c139b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2714009e50 tx=0x7f271401c910 comp rx=0 tx=0).stop 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 -- 172.21.15.190:0/2501544082 shutdown_connections 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 --2- 172.21.15.190:0/2501544082 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f271c1396a0 0x7f271c139b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 --2- 172.21.15.190:0/2501544082 >> v2:172.21.15.190:3300/0 conn(0x7f271c06e7e0 0x7f271c0a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 --2- 172.21.15.190:0/2501544082 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f271c0a58d0 0x7f271c0a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 -- 172.21.15.190:0/2501544082 >> 172.21.15.190:0/2501544082 conn(0x7f271c0aa4c0 msgr2=0x7f271c0a69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 -- 172.21.15.190:0/2501544082 shutdown_connections 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 -- 172.21.15.190:0/2501544082 wait complete. 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient: build_initial_monmap 2024-08-19T10:39:21.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient: monmap: 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:21.522 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 Processor -- start 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 -- start start 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient: init 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient: _add_conns ranks=[1,7,3,5,8,0,6,4,2] 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f271c06e7e0 0x7f271c1d1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient(hunting): picked mon.noname-f con 0x7f271c06e7e0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f271c0a58d0 0x7f271c138780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient(hunting): picked mon.noname-i con 0x7f271c0a58d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271c1396a0 0x7f271c120510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient(hunting): picked mon.noname-g con 0x7f271c1396a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f271c0a7420 con 0x7f271c06e7e0 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f271c06f460 con 0x7f271c1396a0 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f271c06e2f0 con 0x7f271c0a58d0 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f272225f640 10 monclient(hunting): authenticate will time out at 1007.085798s 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271affd640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f271c0a58d0 msgr2=0x7f271c138780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271affd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f271c0a58d0 0x7f271c138780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271b7fe640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f271c06e7e0 msgr2=0x7f271c1d1130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:21.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f271c06e7e0 0x7f271c1d1130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271c1396a0 0x7f271c120510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271c1396a0 0x7f271c120510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46922/0 (socket says 172.21.15.190:46922) 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 1 -- 172.21.15.190:0/1140435564 learned_addr learned my addr 172.21.15.190:0/1140435564 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 10 monclient(hunting): get_auth_request con 0x7f271c1396a0 auth_method 0 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 10 monclient(hunting): handle_auth_done global_id 4485 payload 824 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 1 -- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f271c0a58d0 msgr2=0x7f271c138780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f271c0a58d0 0x7f271c138780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 1 -- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f271c06e7e0 msgr2=0x7f271c1d1130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:21.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f271c06e7e0 0x7f271c1d1130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:21.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:21.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 1 monclient: found mon.noname-g 2024-08-19T10:39:21.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:21.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:21.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.514+0000 7f271bfff640 1 -- 172.21.15.190:0/1140435564 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c0a6c50 con 0x7f271c1396a0 2024-08-19T10:39:21.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f271bfff640 1 -- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271c1396a0 msgr2=0x7f271c120510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:21.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f271bfff640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271c1396a0 0x7f271c120510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2714009040 tx=0x7f27140196b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:21.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 1 -- 172.21.15.190:0/1140435564 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f27140bc020 con 0x7f271c1396a0 2024-08-19T10:39:21.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:21.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:21.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 10 monclient: dump: 2024-08-19T10:39:21.528 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:21.528 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:21.528 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:21.529 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 10 monclient: _finish_auth 0 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 1 -- 172.21.15.190:0/1140435564 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2714003dc0 con 0x7f271c1396a0 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 1 -- 172.21.15.190:0/1140435564 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f27140b7450 con 0x7f271c1396a0 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f272225f640 5 monclient: authenticate success, global_id 4485 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f272225f640 10 monclient: _renew_subs 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f272225f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f272225f640 1 -- 172.21.15.190:0/1140435564 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c13fa80 con 0x7f271c1396a0 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f27197fa640 4 set_mon_vals no callback set 2024-08-19T10:39:21.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:21.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 10 monclient: dump: 2024-08-19T10:39:21.531 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:21.531 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:21.531 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.531 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:21.531 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:21.531 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:21.531 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:21.531 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:21.531 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:21.531 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f272225f640 10 monclient: _renew_subs 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f272225f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f272225f640 1 -- 172.21.15.190:0/1140435564 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c1238a0 con 0x7f271c1396a0 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f272225f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f272225f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f272225f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f272225f640 1 -- 172.21.15.190:0/1140435564 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c0a8080 con 0x7f271c1396a0 2024-08-19T10:39:21.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 1 -- 172.21.15.190:0/1140435564 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f27140bf020 con 0x7f271c1396a0 2024-08-19T10:39:21.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f270403be00 0x7f270403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:21.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f2713fff640 1 -- 172.21.15.190:0/1140435564 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4713+0+0 (secure 0 0 0) 0x7f27140b3040 con 0x7f271c1396a0 2024-08-19T10:39:21.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f271b7fe640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f270403be00 0x7f270403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:21.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.518+0000 7f271b7fe640 10 monclient: get_auth_request con 0x7f270403be00 auth_method 0 2024-08-19T10:39:21.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.522+0000 7f271b7fe640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f270403be00 0x7f270403e2c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f270c0094e0 tx=0x7f270c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:21.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.522+0000 7f2713fff640 1 -- 172.21.15.190:0/1140435564 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f271c0a8080 con 0x7f271c1396a0 2024-08-19T10:39:21.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.522+0000 7f2713fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:21.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.522+0000 7f2713fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:21.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.694+0000 7f272225f640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-08-19T10:39:21.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.694+0000 7f272225f640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-08-19T10:39:21.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.694+0000 7f272225f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:21.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.694+0000 7f272225f640 1 -- 172.21.15.190:0/1140435564 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"} v 0) -- 0x7f271c139b50 con 0x7f271c1396a0 2024-08-19T10:39:21.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.930+0000 7f2713fff640 1 -- 172.21.15.190:0/1140435564 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}]=0 set pool 3 pg_num_max to 33 v23) ==== 145+0+0 (secure 0 0 0) 0x7f27140c6030 con 0x7f271c1396a0 2024-08-19T10:39:21.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.930+0000 7f2713fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-08-19T10:39:21.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:21.930+0000 7f2713fff640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_max to 33 2024-08-19T10:39:22.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.034+0000 7f272225f640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-08-19T10:39:22.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.034+0000 7f272225f640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-08-19T10:39:22.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.034+0000 7f272225f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:22.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.034+0000 7f272225f640 1 -- 172.21.15.190:0/1140435564 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"} v 0) -- 0x7f271c0008e0 con 0x7f271c1396a0 2024-08-19T10:39:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.942+0000 7f2713fff640 1 -- 172.21.15.190:0/1140435564 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}]=0 set pool 3 pg_num_max to 33 v24) ==== 145+0+0 (secure 0 0 0) 0x7f2714016620 con 0x7f271c1396a0 2024-08-19T10:39:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.942+0000 7f2713fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "33"}] 2024-08-19T10:39:22.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.942+0000 7f2713fff640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_max to 33 2024-08-19T10:39:22.950 INFO:tasks.workunit.client.0.smithi190.stderr:set pool 3 pg_num_max to 33 2024-08-19T10:39:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 -- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f270403be00 msgr2=0x7f270403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f270403be00 0x7f270403e2c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f270c0094e0 tx=0x7f270c009190 comp rx=0 tx=0).stop 2024-08-19T10:39:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 10 monclient: shutdown 2024-08-19T10:39:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 -- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271c1396a0 msgr2=0x7f271c120510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271c1396a0 0x7f271c120510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2714009040 tx=0x7f27140196b0 comp rx=0 tx=0).stop 2024-08-19T10:39:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 -- 172.21.15.190:0/1140435564 shutdown_connections 2024-08-19T10:39:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f270403be00 0x7f270403e2c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f271c1396a0 0x7f271c120510 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f271c0a58d0 0x7f271c138780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 --2- 172.21.15.190:0/1140435564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f271c06e7e0 0x7f271c1d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 -- 172.21.15.190:0/1140435564 >> 172.21.15.190:0/1140435564 conn(0x7f271c0aa4c0 msgr2=0x7f271c0a47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 -- 172.21.15.190:0/1140435564 shutdown_connections 2024-08-19T10:39:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:22.946+0000 7f272225f640 1 -- 172.21.15.190:0/1140435564 wait complete. 2024-08-19T10:39:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool get testpool1234 pg_num_max 2024-08-19T10:39:22.969 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 33 2024-08-19T10:39:23.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.086+0000 7f6ab54bf640 10 monclient: get_monmap_and_config 2024-08-19T10:39:23.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.086+0000 7f6ab54bf640 10 monclient: build_initial_monmap 2024-08-19T10:39:23.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.086+0000 7f6ab54bf640 10 monclient: monmap: 2024-08-19T10:39:23.089 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:23.089 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:23.090731+0000 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:23.090731+0000 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:23.090 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:23.091 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:23.091 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:23.091 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:23.091 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 1 Processor -- start 2024-08-19T10:39:23.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 1 -- start start 2024-08-19T10:39:23.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 10 monclient: init 2024-08-19T10:39:23.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:23.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 10 monclient: _add_conns ranks=[7,10,8,0,5,4,6,2,1,3,9] 2024-08-19T10:39:23.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ab0139980 0x7f6ab0139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 10 monclient(hunting): picked mon.noname-i con 0x7f6ab0139980 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:23.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6ab013a2a0 0x7f6ab00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 10 monclient(hunting): picked mon.noname-f con 0x7f6ab013a2a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:39:23.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab00a5c70 0x7f6ab00a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 10 monclient(hunting): picked mon.noname-e con 0x7f6ab00a5c70 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:23.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab009ca60 con 0x7f6ab013a2a0 2024-08-19T10:39:23.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6ab00a3640 con 0x7f6ab0139980 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6ab00a8470 con 0x7f6ab00a5c70 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6ab54bf640 10 monclient(hunting): authenticate will time out at 1008.660859s 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aa67fc640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f6ab013a2a0 msgr2=0x7f6ab00a5730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaeffd640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ab0139980 msgr2=0x7f6ab0139d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aa67fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6ab013a2a0 0x7f6ab00a5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaeffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ab0139980 0x7f6ab0139d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab00a5c70 0x7f6ab00a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab00a5c70 0x7f6ab00a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:42886/0 (socket says 172.21.15.190:42886) 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 1 -- 172.21.15.190:0/3981216439 learned_addr learned my addr 172.21.15.190:0/3981216439 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:23.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 10 monclient(hunting): get_auth_request con 0x7f6ab00a5c70 auth_method 0 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 10 monclient(hunting): handle_auth_done global_id 4445 payload 274 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 1 -- 172.21.15.190:0/3981216439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ab0139980 msgr2=0x7f6ab0139d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 1 --2- 172.21.15.190:0/3981216439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ab0139980 0x7f6ab0139d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 1 -- 172.21.15.190:0/3981216439 >> v2:172.21.15.190:3300/0 conn(0x7f6ab013a2a0 msgr2=0x7f6ab00a5730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 1 --2- 172.21.15.190:0/3981216439 >> v2:172.21.15.190:3300/0 conn(0x7f6ab013a2a0 0x7f6ab00a5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:39:23.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 1 monclient: found mon.noname-e 2024-08-19T10:39:23.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:23.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:39:23.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.090+0000 7f6aaf7fe640 1 -- 172.21.15.190:0/3981216439 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab00a3850 con 0x7f6ab00a5c70 2024-08-19T10:39:23.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aaf7fe640 1 -- 172.21.15.190:0/3981216439 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab00a5c70 msgr2=0x7f6ab00a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:23.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aaf7fe640 1 --2- 172.21.15.190:0/3981216439 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab00a5c70 0x7f6ab00a67e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009e80 tx=0x7f6aa801c650 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d728b98908ba29ca server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:23.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 1 -- 172.21.15.190:0/3981216439 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6aa80ac070 con 0x7f6ab00a5c70 2024-08-19T10:39:23.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:39:23.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 10 monclient: dump: 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.100 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 1 -- 172.21.15.190:0/3981216439 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa80091b0 con 0x7f6ab00a5c70 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6ab54bf640 5 monclient: authenticate success, global_id 4445 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6ab54bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 1 -- 172.21.15.190:0/3981216439 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6aa80a8cf0 con 0x7f6ab00a5c70 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6aae7fc640 10 monclient: dump: 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.102 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6ab54bf640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.094+0000 7f6ab54bf640 4 set_mon_vals no callback set 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient: shutdown 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 -- 172.21.15.190:0/3981216439 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab00a5c70 msgr2=0x7f6ab00a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 --2- 172.21.15.190:0/3981216439 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab00a5c70 0x7f6ab00a67e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009e80 tx=0x7f6aa801c650 comp rx=0 tx=0).stop 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 -- 172.21.15.190:0/3981216439 shutdown_connections 2024-08-19T10:39:23.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 --2- 172.21.15.190:0/3981216439 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ab00a5c70 0x7f6ab00a67e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 --2- 172.21.15.190:0/3981216439 >> v2:172.21.15.190:3300/0 conn(0x7f6ab013a2a0 0x7f6ab00a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 --2- 172.21.15.190:0/3981216439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ab0139980 0x7f6ab0139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 -- 172.21.15.190:0/3981216439 >> 172.21.15.190:0/3981216439 conn(0x7f6ab00abde0 msgr2=0x7f6ab00ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 -- 172.21.15.190:0/3981216439 shutdown_connections 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 -- 172.21.15.190:0/3981216439 wait complete. 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient: build_initial_monmap 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient: monmap: 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:23.104 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:23.105 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:23.105 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:23.105 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:23.105 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:23.105 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:23.105 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:23.105 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:23.105 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 Processor -- start 2024-08-19T10:39:23.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 -- start start 2024-08-19T10:39:23.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient: init 2024-08-19T10:39:23.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:23.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient: _add_conns ranks=[1,2,4,8,5,7,3,0,6] 2024-08-19T10:39:23.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab00a5c70 0x7f6ab01d1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient(hunting): picked mon.noname-f con 0x7f6ab00a5c70 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:23.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ab0139980 0x7f6ab013fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient(hunting): picked mon.noname-b con 0x7f6ab0139980 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab0140160 0x7f6ab009f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient(hunting): picked mon.noname-c con 0x7f6ab0140160 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0143270 con 0x7f6ab00a5c70 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab01430f0 con 0x7f6ab0139980 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6ab01433f0 con 0x7f6ab0140160 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaeffd640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab00a5c70 msgr2=0x7f6ab01d1580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6ab54bf640 10 monclient(hunting): authenticate will time out at 1008.669843s 2024-08-19T10:39:23.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab00a5c70 0x7f6ab01d1580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:23.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaf7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab0140160 0x7f6ab009f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ab0139980 0x7f6ab013fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ab0139980 0x7f6ab013fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:44322/0 (socket says 172.21.15.190:44322) 2024-08-19T10:39:23.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaf7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab0140160 0x7f6ab009f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:40868/0 (socket says 172.21.15.190:40868) 2024-08-19T10:39:23.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 1 -- 172.21.15.190:0/2209916787 learned_addr learned my addr 172.21.15.190:0/2209916787 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:23.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 10 monclient(hunting): get_auth_request con 0x7f6ab0139980 auth_method 0 2024-08-19T10:39:23.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:23.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:23.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaf7fe640 10 monclient(hunting): get_auth_request con 0x7f6ab0140160 auth_method 0 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 10 monclient(hunting): handle_auth_done global_id 4466 payload 824 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 1 -- 172.21.15.190:0/2209916787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab0140160 msgr2=0x7f6ab009f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:23.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab0140160 0x7f6ab009f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 1 -- 172.21.15.190:0/2209916787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab00a5c70 msgr2=0x7f6ab01d1580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab00a5c70 0x7f6ab01d1580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 10 monclient: _finish_hunting 0 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 1 monclient: found mon.noname-b 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aa67fc640 1 -- 172.21.15.190:0/2209916787 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab00a9230 con 0x7f6ab0139980 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.098+0000 7f6aaf7fe640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab0140160 0x7f6ab009f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa67fc640 1 -- 172.21.15.190:0/2209916787 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ab0139980 msgr2=0x7f6ab013fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa67fc640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ab0139980 0x7f6ab013fc20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800b580 tx=0x7f6a9800ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 1 -- 172.21.15.190:0/2209916787 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a98017070 con 0x7f6ab0139980 2024-08-19T10:39:23.113 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 10 monclient: dump: 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.114 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 10 monclient: _finish_auth 0 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 1 -- 172.21.15.190:0/2209916787 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a98003de0 con 0x7f6ab0139980 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6ab54bf640 5 monclient: authenticate success, global_id 4466 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6ab54bf640 10 monclient: _renew_subs 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6ab54bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:23.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6ab54bf640 1 -- 172.21.15.190:0/2209916787 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab013b3e0 con 0x7f6ab0139980 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aad7fa640 4 set_mon_vals no callback set 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6ab54bf640 10 monclient: _renew_subs 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6ab54bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6ab54bf640 1 -- 172.21.15.190:0/2209916787 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab00a1080 con 0x7f6ab0139980 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 1 -- 172.21.15.190:0/2209916787 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a98005540 con 0x7f6ab0139980 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 10 monclient: dump: 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.116 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6ab54bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:23.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6ab54bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:23.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6ab54bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:23.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6ab54bf640 1 -- 172.21.15.190:0/2209916787 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab0139de0 con 0x7f6ab0139980 2024-08-19T10:39:23.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 1 -- 172.21.15.190:0/2209916787 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6a9800f070 con 0x7f6ab0139980 2024-08-19T10:39:23.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9003be00 0x7f6a9003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aa7fff640 1 -- 172.21.15.190:0/2209916787 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7f6a9804df80 con 0x7f6ab0139980 2024-08-19T10:39:23.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aaeffd640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9003be00 0x7f6a9003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.102+0000 7f6aaeffd640 10 monclient: get_auth_request con 0x7f6a9003be00 auth_method 0 2024-08-19T10:39:23.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.106+0000 7f6aaeffd640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9003be00 0x7f6a9003e2c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0003990 tx=0x7f6aa0006cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:23.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.118+0000 7f6aa7fff640 1 -- 172.21.15.190:0/2209916787 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6ab00a9230 con 0x7f6ab0139980 2024-08-19T10:39:23.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.118+0000 7f6aa7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:23.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.118+0000 7f6aa7fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:23.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.274+0000 7f6ab54bf640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-08-19T10:39:23.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.274+0000 7f6ab54bf640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-08-19T10:39:23.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.274+0000 7f6ab54bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:23.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.274+0000 7f6ab54bf640 1 -- 172.21.15.190:0/2209916787 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) -- 0x7f6ab01d1ac0 con 0x7f6ab0139980 2024-08-19T10:39:23.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.274+0000 7f6aa7fff640 1 -- 172.21.15.190:0/2209916787 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=0 v24) ==== 105+0+15 (secure 0 0 0) 0x7f6a9800d040 con 0x7f6ab0139980 2024-08-19T10:39:23.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.274+0000 7f6aa7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-08-19T10:39:23.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.274+0000 7f6aa7fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:39:23.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.366+0000 7f6ab54bf640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-08-19T10:39:23.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.366+0000 7f6ab54bf640 10 monclient: _send_command 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-08-19T10:39:23.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.366+0000 7f6ab54bf640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:23.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.366+0000 7f6ab54bf640 1 -- 172.21.15.190:0/2209916787 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) -- 0x7f6ab00008e0 con 0x7f6ab0139980 2024-08-19T10:39:23.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.366+0000 7f6aa7fff640 1 -- 172.21.15.190:0/2209916787 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=0 v24) ==== 105+0+15 (secure 0 0 0) 0x7f6a9803f070 con 0x7f6ab0139980 2024-08-19T10:39:23.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.366+0000 7f6aa7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-08-19T10:39:23.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.366+0000 7f6aa7fff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:39:23.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 -- 172.21.15.190:0/2209916787 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9003be00 msgr2=0x7f6a9003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:23.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9003be00 0x7f6a9003e2c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0003990 tx=0x7f6aa0006cf0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 10 monclient: shutdown 2024-08-19T10:39:23.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:23.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 -- 172.21.15.190:0/2209916787 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ab0139980 msgr2=0x7f6ab013fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:23.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ab0139980 0x7f6ab013fc20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800b580 tx=0x7f6a9800ba40 comp rx=0 tx=0).stop 2024-08-19T10:39:23.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 -- 172.21.15.190:0/2209916787 shutdown_connections 2024-08-19T10:39:23.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a9003be00 0x7f6a9003e2c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6ab0139980 0x7f6ab013fc20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ab0140160 0x7f6ab009f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 --2- 172.21.15.190:0/2209916787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ab00a5c70 0x7f6ab01d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 -- 172.21.15.190:0/2209916787 >> 172.21.15.190:0/2209916787 conn(0x7f6ab00abde0 msgr2=0x7f6ab00ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:23.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 -- 172.21.15.190:0/2209916787 shutdown_connections 2024-08-19T10:39:23.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.370+0000 7f6ab54bf640 1 -- 172.21.15.190:0/2209916787 wait complete. 2024-08-19T10:39:23.385 INFO:tasks.workunit.client.0.smithi190.stdout:pg_num_max: 33 2024-08-19T10:39:23.386 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool set testpool1234 pg_num_min 9 2024-08-19T10:39:23.386 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:23.386 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set testpool1234 pg_num_min 9 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient: get_monmap_and_config 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient: build_initial_monmap 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient: monmap: 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:23.504193+0000 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:23.504193+0000 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:23.503 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:23.504 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:23.504 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:23.504 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:23.504 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:23.504 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:23.504 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:23.504 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:23.504 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:23.504 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 1 Processor -- start 2024-08-19T10:39:23.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 1 -- start start 2024-08-19T10:39:23.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient: init 2024-08-19T10:39:23.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:23.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient: _add_conns ranks=[0,8,5,1,9,3,4,10,7,6,2] 2024-08-19T10:39:23.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f21b41396a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f21b413a190 0x7f21b413eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient(hunting): picked mon.noname-e con 0x7f21b413a190 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 0x7f21b4141490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient(hunting): picked mon.noname-h con 0x7f21b413f0a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21b40a83e0 con 0x7f21b41396a0 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f21b40a4f20 con 0x7f21b413f0a0 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f21b40a2f30 con 0x7f21b413a190 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21ba177640 10 monclient(hunting): authenticate will time out at 1009.074535s 2024-08-19T10:39:23.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 0x7f21b4141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 0x7f21b4141490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:40016/0 (socket says 172.21.15.190:40016) 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 1 -- 172.21.15.190:0/3117573233 learned_addr learned my addr 172.21.15.190:0/3117573233 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 10 monclient(hunting): get_auth_request con 0x7f21b413f0a0 auth_method 0 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b2ffd640 1 --2- 172.21.15.190:0/3117573233 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f21b413a190 0x7f21b413eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 10 monclient(hunting): handle_auth_done global_id 4496 payload 274 2024-08-19T10:39:23.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 1 -- 172.21.15.190:0/3117573233 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f21b413a190 msgr2=0x7f21b413eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:23.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 1 --2- 172.21.15.190:0/3117573233 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f21b413a190 0x7f21b413eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 1 -- 172.21.15.190:0/3117573233 >> v1:172.21.15.89:6789/0 conn(0x7f21b41396a0 legacy=0x7f21b4139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:23.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:23.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 1 monclient: found mon.noname-h 2024-08-19T10:39:23.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:23.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:23.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b3fff640 1 -- 172.21.15.190:0/3117573233 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b40a35e0 con 0x7f21b413f0a0 2024-08-19T10:39:23.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b2ffd640 10 monclient: get_auth_request con 0x7f21b413a190 auth_method 0 2024-08-19T10:39:23.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.502+0000 7f21b2ffd640 1 --2- 172.21.15.190:0/3117573233 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f21b413a190 0x7f21b413eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:39:23.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b3fff640 1 -- 172.21.15.190:0/3117573233 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 msgr2=0x7f21b4141490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:23.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b3fff640 1 --2- 172.21.15.190:0/3117573233 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 0x7f21b4141490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009eb0 tx=0x7f21ac01c8f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=32662178d4a74ef9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:23.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 1 -- 172.21.15.190:0/3117573233 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21ac0091b0 con 0x7f21b413f0a0 2024-08-19T10:39:23.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:23.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 10 monclient: dump: 2024-08-19T10:39:23.511 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.512 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.512 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.512 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.512 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.512 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.512 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.512 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.512 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.512 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.512 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.512 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.513 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.513 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.513 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.513 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:23.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:23.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:23.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 1 -- 172.21.15.190:0/3117573233 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ac0a6950 con 0x7f21b413f0a0 2024-08-19T10:39:23.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 1 -- 172.21.15.190:0/3117573233 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21ac0af480 con 0x7f21b413f0a0 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21ba177640 5 monclient: authenticate success, global_id 4496 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21ba177640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21ba177640 4 set_mon_vals no callback set 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21b27fc640 10 monclient: dump: 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.514 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21ba177640 10 monclient: shutdown 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21ba177640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21ba177640 1 -- 172.21.15.190:0/3117573233 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 msgr2=0x7f21b4141490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.506+0000 7f21ba177640 1 --2- 172.21.15.190:0/3117573233 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 0x7f21b4141490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009eb0 tx=0x7f21ac01c8f0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 -- 172.21.15.190:0/3117573233 shutdown_connections 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 --2- 172.21.15.190:0/3117573233 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 0x7f21b4141490 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 --2- 172.21.15.190:0/3117573233 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f21b413a190 0x7f21b413eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 -- 172.21.15.190:0/3117573233 >> 172.21.15.190:0/3117573233 conn(0x7f21b40abee0 msgr2=0x7f21b40ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 -- 172.21.15.190:0/3117573233 shutdown_connections 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 -- 172.21.15.190:0/3117573233 wait complete. 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient: build_initial_monmap 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient: monmap: 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:23.516 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 Processor -- start 2024-08-19T10:39:23.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 -- start start 2024-08-19T10:39:23.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient: init 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient: _add_conns ranks=[6,0,5,3,7,4,2,1,8] 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f21b41396a0 0x7f21b41d1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient(hunting): picked mon.noname-d con 0x7f21b41396a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21b413a190 0x7f21b41d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient(hunting): picked mon.noname-a con 0x7f21b413a190 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 0x7f21b409f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient(hunting): picked mon.noname-h con 0x7f21b413f0a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f21b41441f0 con 0x7f21b413a190 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f21b40a78f0 con 0x7f21b413f0a0 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f21b4144370 con 0x7f21b41396a0 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21ba177640 10 monclient(hunting): authenticate will time out at 1009.081748s 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 0x7f21b409f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 0x7f21b409f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:40026/0 (socket says 172.21.15.190:40026) 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 1 -- 172.21.15.190:0/1452484390 learned_addr learned my addr 172.21.15.190:0/1452484390 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 10 monclient(hunting): get_auth_request con 0x7f21b413f0a0 auth_method 0 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:23.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b2ffd640 1 --2- 172.21.15.190:0/1452484390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21b413a190 0x7f21b41d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b37fe640 1 --2- 172.21.15.190:0/1452484390 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f21b41396a0 0x7f21b41d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b2ffd640 10 monclient(hunting): get_auth_request con 0x7f21b413a190 auth_method 0 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 10 monclient(hunting): handle_auth_done global_id 4505 payload 824 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 1 -- 172.21.15.190:0/1452484390 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f21b41396a0 msgr2=0x7f21b41d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 1 --2- 172.21.15.190:0/1452484390 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f21b41396a0 0x7f21b41d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 1 -- 172.21.15.190:0/1452484390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21b413a190 msgr2=0x7f21b41d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 1 --2- 172.21.15.190:0/1452484390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21b413a190 0x7f21b41d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:23.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 1 monclient: found mon.noname-h 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b3fff640 1 -- 172.21.15.190:0/1452484390 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b41228b0 con 0x7f21b413f0a0 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b37fe640 10 monclient: get_auth_request con 0x7f21b41396a0 auth_method 0 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.510+0000 7f21b37fe640 1 --2- 172.21.15.190:0/1452484390 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f21b41396a0 0x7f21b41d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21b3fff640 1 -- 172.21.15.190:0/1452484390 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 msgr2=0x7f21b409f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21b3fff640 1 --2- 172.21.15.190:0/1452484390 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f21b413f0a0 0x7f21b409f5f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f21ac01c720 tx=0x7f21ac019690 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 1 -- 172.21.15.190:0/1452484390 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21ac0be070 con 0x7f21b413f0a0 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 10 monclient: dump: 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.524 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 10 monclient: _finish_auth 0 2024-08-19T10:39:23.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 1 -- 172.21.15.190:0/1452484390 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ac0039a0 con 0x7f21b413f0a0 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 1 -- 172.21.15.190:0/1452484390 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f21ac004040 con 0x7f21b413f0a0 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 10 monclient: dump: 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.526 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21b1ffb640 4 set_mon_vals no callback set 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21ba177640 5 monclient: authenticate success, global_id 4505 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21ba177640 10 monclient: _renew_subs 2024-08-19T10:39:23.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21ba177640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21ba177640 1 -- 172.21.15.190:0/1452484390 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b40a7030 con 0x7f21b413f0a0 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21ba177640 10 monclient: _renew_subs 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21ba177640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21ba177640 1 -- 172.21.15.190:0/1452484390 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b40a1b00 con 0x7f21b413f0a0 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21ba177640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21ba177640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21ba177640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 1 -- 172.21.15.190:0/1452484390 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f21ac003b50 con 0x7f21b413f0a0 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21ba177640 1 -- 172.21.15.190:0/1452484390 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b4139a80 con 0x7f21b413f0a0 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 1 --2- 172.21.15.190:0/1452484390 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f21a003be30 0x7f21a003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f219bfff640 1 -- 172.21.15.190:0/1452484390 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7f21ac0b5030 con 0x7f21b413f0a0 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21b37fe640 1 --2- 172.21.15.190:0/1452484390 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f21a003be30 0x7f21a003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.514+0000 7f21b37fe640 10 monclient: get_auth_request con 0x7f21a003be30 auth_method 0 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.518+0000 7f21b37fe640 1 --2- 172.21.15.190:0/1452484390 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f21a003be30 0x7f21a003e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f21a8003e40 tx=0x7f21a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:23.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.518+0000 7f219bfff640 1 -- 172.21.15.190:0/1452484390 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f21ac016600 con 0x7f21b413f0a0 2024-08-19T10:39:23.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.518+0000 7f219bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:23.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.518+0000 7f219bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:23.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.686+0000 7f21ba177640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-08-19T10:39:23.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.686+0000 7f21ba177640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-08-19T10:39:23.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.686+0000 7f21ba177640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:23.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.686+0000 7f21ba177640 1 -- 172.21.15.190:0/1452484390 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"} v 0) -- 0x7f21b40a1400 con 0x7f21b413f0a0 2024-08-19T10:39:23.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.690+0000 7f219bfff640 1 -- 172.21.15.190:0/1452484390 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}]=-22 specified pg_num_min 9 > pg_num 8 v24) ==== 150+0+0 (secure 0 0 0) 0x7f21ac0b7370 con 0x7f21b413f0a0 2024-08-19T10:39:23.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.690+0000 7f219bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "9"}] 2024-08-19T10:39:23.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.690+0000 7f219bfff640 10 monclient: _finish_command 2 = mon:22 specified pg_num_min 9 > pg_num 8 2024-08-19T10:39:23.692 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: specified pg_num_min 9 > pg_num 8 2024-08-19T10:39:23.697 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:23.697 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool set testpool1234 pg_num_max 7 2024-08-19T10:39:23.697 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:23.697 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set testpool1234 pg_num_max 7 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient: get_monmap_and_config 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient: build_initial_monmap 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient: monmap: 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:23.815730+0000 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:23.815730+0000 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:23.815 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 1 Processor -- start 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 1 -- start start 2024-08-19T10:39:23.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient: init 2024-08-19T10:39:23.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient: _add_conns ranks=[3,6,7,1,5,0,9,2,4,10,8] 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40a68e0 0x7fb1a40a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient(hunting): picked mon.noname-g con 0x7fb1a40a68e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1a40a6cc0 0x7fb1a40a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient(hunting): picked mon.noname-d con 0x7fb1a40a6cc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb1a41396a0 0x7fb1a4139ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient(hunting): picked mon.noname-i con 0x7fb1a41396a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a40a86c0 con 0x7fb1a40a68e0 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb1a40a8410 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb1a40a63a0 con 0x7fb1a41396a0 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1aa2fb640 10 monclient(hunting): authenticate will time out at 1009.385653s 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a8898640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb1a41396a0 msgr2=0x7fb1a4139ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a8898640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb1a41396a0 0x7fb1a4139ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40a68e0 0x7fb1a40a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40a68e0 0x7fb1a40a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46926/0 (socket says 172.21.15.190:46926) 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 1 -- 172.21.15.190:0/466638909 learned_addr learned my addr 172.21.15.190:0/466638909 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 10 monclient(hunting): get_auth_request con 0x7fb1a40a68e0 auth_method 0 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:23.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a37fe640 1 --2- 172.21.15.190:0/466638909 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1a40a6cc0 0x7fb1a40a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 10 monclient(hunting): handle_auth_done global_id 4494 payload 274 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 1 -- 172.21.15.190:0/466638909 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb1a41396a0 msgr2=0x7fb1a4139ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 1 --2- 172.21.15.190:0/466638909 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb1a41396a0 0x7fb1a4139ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 1 -- 172.21.15.190:0/466638909 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1a40a6cc0 msgr2=0x7fb1a40a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 1 --2- 172.21.15.190:0/466638909 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1a40a6cc0 0x7fb1a40a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 1 monclient: found mon.noname-g 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a3fff640 1 -- 172.21.15.190:0/466638909 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a40a4680 con 0x7fb1a40a68e0 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a37fe640 10 monclient: get_auth_request con 0x7fb1a40a6cc0 auth_method 0 2024-08-19T10:39:23.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.814+0000 7fb1a37fe640 1 --2- 172.21.15.190:0/466638909 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1a40a6cc0 0x7fb1a40a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:39:23.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a3fff640 1 -- 172.21.15.190:0/466638909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40a68e0 msgr2=0x7fb1a40a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:23.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a3fff640 1 --2- 172.21.15.190:0/466638909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40a68e0 0x7fb1a40a5760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb1940098a0 tx=0x7fb19401c140 comp rx=0 tx=0).ready entity=mon.3 client_cookie=7d2576b7d89641cd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:23.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 1 -- 172.21.15.190:0/466638909 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1940ab070 con 0x7fb1a40a68e0 2024-08-19T10:39:23.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 10 monclient: dump: 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.826 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 5 monclient: authenticate success, global_id 4494 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 1 -- 172.21.15.190:0/466638909 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1940a5970 con 0x7fb1a40a68e0 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:23.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 4 set_mon_vals no callback set 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 1 -- 172.21.15.190:0/466638909 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1940a6400 con 0x7fb1a40a68e0 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1a2ffd640 10 monclient: dump: 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.828 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 10 monclient: shutdown 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 1 -- 172.21.15.190:0/466638909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40a68e0 msgr2=0x7fb1a40a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 1 --2- 172.21.15.190:0/466638909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40a68e0 0x7fb1a40a5760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb1940098a0 tx=0x7fb19401c140 comp rx=0 tx=0).stop 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 1 -- 172.21.15.190:0/466638909 shutdown_connections 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 1 --2- 172.21.15.190:0/466638909 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1a40a6cc0 0x7fb1a40a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 1 --2- 172.21.15.190:0/466638909 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb1a41396a0 0x7fb1a4139ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 1 --2- 172.21.15.190:0/466638909 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40a68e0 0x7fb1a40a5760 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 1 -- 172.21.15.190:0/466638909 >> 172.21.15.190:0/466638909 conn(0x7fb1a40abee0 msgr2=0x7fb1a40ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 1 -- 172.21.15.190:0/466638909 shutdown_connections 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.818+0000 7fb1aa2fb640 1 -- 172.21.15.190:0/466638909 wait complete. 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient: build_initial_monmap 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient: monmap: 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:23.830 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 1 Processor -- start 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 1 -- start start 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient: init 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient: _add_conns ranks=[8,4,1,6,0,5,7,2,3] 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb1a40a68e0 0x7fb1a41d2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient(hunting): picked mon.noname-e con 0x7fb1a40a68e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a6cc0 0x7fb1a40a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient(hunting): picked mon.noname-c con 0x7fb1a40a6cc0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a41396a0 0x7fb1a40a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient(hunting): picked mon.noname-f con 0x7fb1a41396a0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4143cc0 con 0x7fb1a41396a0 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb1a4143b40 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb1a4143e40 con 0x7fb1a40a68e0 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:23.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1aa2fb640 10 monclient(hunting): authenticate will time out at 1009.392992s 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a8898640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a41396a0 msgr2=0x7fb1a40a25b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a8898640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a41396a0 0x7fb1a40a25b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb1a40a68e0 0x7fb1a41d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a6cc0 0x7fb1a40a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a3fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb1a40a68e0 0x7fb1a41d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:42892/0 (socket says 172.21.15.190:42892) 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a3fff640 1 -- 172.21.15.190:0/3495310045 learned_addr learned my addr 172.21.15.190:0/3495310045 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a3fff640 10 monclient(hunting): get_auth_request con 0x7fb1a40a68e0 auth_method 0 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 10 monclient(hunting): get_auth_request con 0x7fb1a40a6cc0 auth_method 0 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:23.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 10 monclient(hunting): handle_auth_done global_id 4594 payload 824 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 1 -- 172.21.15.190:0/3495310045 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb1a40a68e0 msgr2=0x7fb1a41d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 1 --2- 172.21.15.190:0/3495310045 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb1a40a68e0 0x7fb1a41d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 1 -- 172.21.15.190:0/3495310045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a41396a0 msgr2=0x7fb1a40a25b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 1 --2- 172.21.15.190:0/3495310045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a41396a0 0x7fb1a40a25b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:39:23.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 1 monclient: found mon.noname-c 2024-08-19T10:39:23.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:23.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:23.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.822+0000 7fb1a37fe640 1 -- 172.21.15.190:0/3495310045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a413f7e0 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a37fe640 1 -- 172.21.15.190:0/3495310045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a6cc0 msgr2=0x7fb1a40a2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:23.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a37fe640 1 --2- 172.21.15.190:0/3495310045 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1a40a6cc0 0x7fb1a40a2070 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb19800c7f0 tx=0x7fb19800ccb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:23.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 1 -- 172.21.15.190:0/3495310045 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb198015070 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 10 monclient: dump: 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.837 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 10 monclient: _finish_auth 0 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 1 -- 172.21.15.190:0/3495310045 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb198007ca0 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1aa2fb640 5 monclient: authenticate success, global_id 4594 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1aa2fb640 10 monclient: _renew_subs 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1aa2fb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1aa2fb640 1 -- 172.21.15.190:0/3495310045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a409d270 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 1 -- 172.21.15.190:0/3495310045 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb198007e50 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 10 monclient: dump: 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:23.839 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a27fc640 4 set_mon_vals no callback set 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1aa2fb640 10 monclient: _renew_subs 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1aa2fb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1aa2fb640 1 -- 172.21.15.190:0/3495310045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a4140340 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1aa2fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1aa2fb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1aa2fb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1aa2fb640 1 -- 172.21.15.190:0/3495310045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a40a73d0 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 1 -- 172.21.15.190:0/3495310045 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb198018070 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 1 --2- 172.21.15.190:0/3495310045 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb19003bfa0 0x7fb19003e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:23.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a0ff9640 1 -- 172.21.15.190:0/3495310045 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7fb19804e8e0 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a3fff640 1 --2- 172.21.15.190:0/3495310045 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb19003bfa0 0x7fb19003e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:23.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.826+0000 7fb1a3fff640 10 monclient: get_auth_request con 0x7fb19003bfa0 auth_method 0 2024-08-19T10:39:23.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.830+0000 7fb1a3fff640 1 --2- 172.21.15.190:0/3495310045 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb19003bfa0 0x7fb19003e460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb19401c180 tx=0x7fb1940a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:23.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.838+0000 7fb1a0ff9640 1 -- 172.21.15.190:0/3495310045 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb1a40a73d0 con 0x7fb1a40a6cc0 2024-08-19T10:39:23.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.838+0000 7fb1a0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:23.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.838+0000 7fb1a0ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:23.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.998+0000 7fb1aa2fb640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-08-19T10:39:24.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.998+0000 7fb1aa2fb640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-08-19T10:39:24.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.998+0000 7fb1aa2fb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:24.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:23.998+0000 7fb1aa2fb640 1 -- 172.21.15.190:0/3495310045 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"} v 0) -- 0x7fb1a40a70a0 con 0x7fb1a40a6cc0 2024-08-19T10:39:24.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.318+0000 7fb1a0ff9640 1 -- 172.21.15.190:0/3495310045 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}]=-22 specified pg_num_max 7 < pg_num 8 v24) ==== 150+0+0 (secure 0 0 0) 0x7fb198018360 con 0x7fb1a40a6cc0 2024-08-19T10:39:24.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.318+0000 7fb1a0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "7"}] 2024-08-19T10:39:24.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.318+0000 7fb1a0ff9640 10 monclient: _finish_command 2 = mon:22 specified pg_num_max 7 < pg_num 8 2024-08-19T10:39:24.318 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: specified pg_num_max 7 < pg_num 8 2024-08-19T10:39:24.323 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:24.323 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool set testpool1234 pg_num 1 2024-08-19T10:39:24.323 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:24.323 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set testpool1234 pg_num 1 2024-08-19T10:39:24.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient: get_monmap_and_config 2024-08-19T10:39:24.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient: build_initial_monmap 2024-08-19T10:39:24.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient: monmap: 2024-08-19T10:39:24.447 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:24.447 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:24.447 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:24.448358+0000 2024-08-19T10:39:24.447 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:24.448358+0000 2024-08-19T10:39:24.447 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:24.447 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:24.448 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:24.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 1 Processor -- start 2024-08-19T10:39:24.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 1 -- start start 2024-08-19T10:39:24.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient: init 2024-08-19T10:39:24.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:24.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient: _add_conns ranks=[6,7,4,0,3,1,9,10,2,5,8] 2024-08-19T10:39:24.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81e80a65d0 0x7f81e80a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:24.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient(hunting): picked mon.noname-d con 0x7f81e80a65d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:24.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81e809d480 0x7f81e80a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:24.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient(hunting): picked mon.noname-i con 0x7f81e809d480 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:24.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81e809e950 0x7f81e8139390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:24.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient(hunting): picked mon.noname-c con 0x7f81e809e950 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f81e80a7430 con 0x7f81e809e950 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f81e80a55a0 con 0x7f81e80a65d0 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f81e809dc90 con 0x7f81e809d480 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eda5b640 10 monclient(hunting): authenticate will time out at 1010.019117s 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81e7fff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81e809d480 msgr2=0x7f81e80a6ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81e7fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81e809d480 0x7f81e80a6ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81ed25a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81e809e950 0x7f81e8139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eca59640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81e80a65d0 0x7f81e80a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81ed25a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81e809e950 0x7f81e8139390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:40900/0 (socket says 172.21.15.190:40900) 2024-08-19T10:39:24.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eca59640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81e80a65d0 0x7f81e80a69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:46430/0 (socket says 172.21.15.190:46430) 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81ed25a640 1 -- 172.21.15.190:0/1991772096 learned_addr learned my addr 172.21.15.190:0/1991772096 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81ed25a640 10 monclient(hunting): get_auth_request con 0x7f81e809e950 auth_method 0 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81ed25a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81ed25a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81ed25a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eca59640 10 monclient(hunting): get_auth_request con 0x7f81e80a65d0 auth_method 0 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eca59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eca59640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eca59640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81ed25a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81ed25a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81ed25a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eca59640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eca59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:24.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.446+0000 7f81eca59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:24.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.450+0000 7f81eca59640 10 monclient(hunting): handle_auth_done global_id 4416 payload 274 2024-08-19T10:39:24.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.450+0000 7f81eca59640 1 -- 172.21.15.190:0/1991772096 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81e809d480 msgr2=0x7f81e80a6ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:24.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.450+0000 7f81eca59640 1 --2- 172.21.15.190:0/1991772096 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81e809d480 0x7f81e80a6ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:24.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.450+0000 7f81eca59640 1 -- 172.21.15.190:0/1991772096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81e809e950 msgr2=0x7f81e8139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:24.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.450+0000 7f81eca59640 1 --2- 172.21.15.190:0/1991772096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81e809e950 0x7f81e8139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:24.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.450+0000 7f81eca59640 10 monclient: _finish_hunting 0 2024-08-19T10:39:24.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.450+0000 7f81eca59640 1 monclient: found mon.noname-d 2024-08-19T10:39:24.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.450+0000 7f81eca59640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:24.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.450+0000 7f81eca59640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:24.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.450+0000 7f81eca59640 1 -- 172.21.15.190:0/1991772096 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e809d150 con 0x7f81e80a65d0 2024-08-19T10:39:24.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.450+0000 7f81ed25a640 1 --2- 172.21.15.190:0/1991772096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81e809e950 0x7f81e8139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:24.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 1 -- 172.21.15.190:0/1991772096 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81e80a65d0 msgr2=0x7f81e80a69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:24.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 1 --2- 172.21.15.190:0/1991772096 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81e80a65d0 0x7f81e80a69b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0097f0 tx=0x7f81dc01c190 comp rx=0 tx=0).ready entity=mon.6 client_cookie=3a9f6483828317f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:24.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 1 -- 172.21.15.190:0/1991772096 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81dc0b0070 con 0x7f81e80a65d0 2024-08-19T10:39:24.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:24.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:24.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 10 monclient: dump: 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:24.457 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 1 -- 172.21.15.190:0/1991772096 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81dc0a9980 con 0x7f81e80a65d0 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 1 -- 172.21.15.190:0/1991772096 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81dc0aa3a0 con 0x7f81e80a65d0 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e77fe640 10 monclient: dump: 2024-08-19T10:39:24.458 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:24.459 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:24.459 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:24.461 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:24.461 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:24.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 5 monclient: authenticate success, global_id 4416 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 4 set_mon_vals no callback set 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient: shutdown 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 -- 172.21.15.190:0/1991772096 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81e80a65d0 msgr2=0x7f81e80a69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 --2- 172.21.15.190:0/1991772096 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81e80a65d0 0x7f81e80a69b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0097f0 tx=0x7f81dc01c190 comp rx=0 tx=0).stop 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 -- 172.21.15.190:0/1991772096 shutdown_connections 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 --2- 172.21.15.190:0/1991772096 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81e809e950 0x7f81e8139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 --2- 172.21.15.190:0/1991772096 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81e809d480 0x7f81e80a6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 --2- 172.21.15.190:0/1991772096 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81e80a65d0 0x7f81e80a69b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 -- 172.21.15.190:0/1991772096 >> 172.21.15.190:0/1991772096 conn(0x7f81e80abbd0 msgr2=0x7f81e80abfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 -- 172.21.15.190:0/1991772096 shutdown_connections 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 -- 172.21.15.190:0/1991772096 wait complete. 2024-08-19T10:39:24.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient: build_initial_monmap 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient: monmap: 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:24.464 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:24.465 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:24.465 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:24.465 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:24.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 Processor -- start 2024-08-19T10:39:24.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 -- start start 2024-08-19T10:39:24.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient: init 2024-08-19T10:39:24.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:24.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient: _add_conns ranks=[2,8,0,5,3,4,6,1,7] 2024-08-19T10:39:24.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81e809d480 0x7f81e8133680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:24.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient(hunting): picked mon.noname-b con 0x7f81e809d480 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:24.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f81e809e950 0x7f81e8133bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:24.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient(hunting): picked mon.noname-e con 0x7f81e809e950 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:24.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81e80a65d0 0x7f81e809edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:24.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient(hunting): picked mon.noname-a con 0x7f81e80a65d0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:24.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8143200 con 0x7f81e80a65d0 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f81e8143080 con 0x7f81e809d480 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f81e8143380 con 0x7f81e809e950 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eda5b640 10 monclient(hunting): authenticate will time out at 1010.027019s 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81ed25a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81e80a65d0 0x7f81e809edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81e809d480 0x7f81e8133680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81ed25a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81e80a65d0 0x7f81e809edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:50400/0 (socket says 172.21.15.190:50400) 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81ed25a640 1 -- 172.21.15.190:0/1725556203 learned_addr learned my addr 172.21.15.190:0/1725556203 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81ed25a640 10 monclient(hunting): get_auth_request con 0x7f81e80a65d0 auth_method 0 2024-08-19T10:39:24.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81ed25a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81ed25a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81ed25a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 10 monclient(hunting): get_auth_request con 0x7f81e809d480 auth_method 0 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e7fff640 1 --2- 172.21.15.190:0/1725556203 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f81e809e950 0x7f81e8133bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e7fff640 10 monclient(hunting): get_auth_request con 0x7f81e809e950 auth_method 0 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81ed25a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81ed25a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:24.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81ed25a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 10 monclient(hunting): handle_auth_done global_id 4475 payload 824 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 1 -- 172.21.15.190:0/1725556203 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f81e809e950 msgr2=0x7f81e8133bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.454+0000 7f81eca59640 1 --2- 172.21.15.190:0/1725556203 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f81e809e950 0x7f81e8133bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eca59640 1 -- 172.21.15.190:0/1725556203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81e80a65d0 msgr2=0x7f81e809edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eca59640 1 --2- 172.21.15.190:0/1725556203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81e80a65d0 0x7f81e809edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eca59640 10 monclient: _finish_hunting 0 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eca59640 1 monclient: found mon.noname-b 2024-08-19T10:39:24.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eca59640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eca59640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eca59640 1 -- 172.21.15.190:0/1725556203 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e80a1880 con 0x7f81e809d480 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eca59640 1 -- 172.21.15.190:0/1725556203 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81e809d480 msgr2=0x7f81e8133680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eca59640 1 --2- 172.21.15.190:0/1725556203 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81e809d480 0x7f81e8133680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f81e80a5720 tx=0x7f81dc018f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 1 -- 172.21.15.190:0/1725556203 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81dc0b0070 con 0x7f81e809d480 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 10 monclient: dump: 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:24.473 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:24.474 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 10 monclient: _finish_auth 0 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eda5b640 5 monclient: authenticate success, global_id 4475 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eda5b640 10 monclient: _renew_subs 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eda5b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eda5b640 1 -- 172.21.15.190:0/1725556203 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e80aaf10 con 0x7f81e809d480 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eda5b640 10 monclient: _renew_subs 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eda5b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 1 -- 172.21.15.190:0/1725556203 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81dc0b1040 con 0x7f81e809d480 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eda5b640 1 -- 172.21.15.190:0/1725556203 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e80a15a0 con 0x7f81e809d480 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:24.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 1 -- 172.21.15.190:0/1725556203 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81dc0044a0 con 0x7f81e809d480 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e57fa640 10 monclient: dump: 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:24.476 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81e67fc640 4 set_mon_vals no callback set 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eda5b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eda5b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eda5b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:24.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.458+0000 7f81eda5b640 1 -- 172.21.15.190:0/1725556203 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e81354e0 con 0x7f81e809d480 2024-08-19T10:39:24.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.462+0000 7f81eca59640 1 --2- 172.21.15.190:0/1725556203 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f81e809d480 0x7f81e8133680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f81e80a5720 tx=0x7f81dc018f30 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f81dc004030 mgrmap(e 5) 2024-08-19T10:39:24.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.462+0000 7f81e57fa640 1 -- 172.21.15.190:0/1725556203 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f81dc004030 con 0x7f81e809d480 2024-08-19T10:39:24.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.462+0000 7f81e57fa640 1 --2- 172.21.15.190:0/1725556203 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f81cc03be30 0x7f81cc03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:24.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.462+0000 7f81e7fff640 1 --2- 172.21.15.190:0/1725556203 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f81cc03be30 0x7f81cc03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:24.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.462+0000 7f81e7fff640 10 monclient: get_auth_request con 0x7f81cc03be30 auth_method 0 2024-08-19T10:39:24.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.462+0000 7f81e7fff640 1 --2- 172.21.15.190:0/1725556203 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f81cc03be30 0x7f81cc03e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f81e813f1c0 tx=0x7f81d4009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:24.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:24.474+0000 7f81e57fa640 1 -- 172.21.15.190:0/1725556203 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7f81dc0e9060 con 0x7f81e809d480 2024-08-19T10:39:25.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.290+0000 7f81e57fa640 1 -- 172.21.15.190:0/1725556203 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f81dc0160d0 con 0x7f81e809d480 2024-08-19T10:39:25.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.290+0000 7f81e57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:25.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.290+0000 7f81e57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:25.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.470+0000 7f81eda5b640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-08-19T10:39:25.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.470+0000 7f81eda5b640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-08-19T10:39:25.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.470+0000 7f81eda5b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:25.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.470+0000 7f81eda5b640 1 -- 172.21.15.190:0/1725556203 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"} v 0) -- 0x7f81e806c8d0 con 0x7f81e809d480 2024-08-19T10:39:25.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.470+0000 7f81e57fa640 1 -- 172.21.15.190:0/1725556203 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}]=-22 specified pg_num 1 < pg_num_min 2 v24) ==== 146+0+0 (secure 0 0 0) 0x7f81dc0c7df0 con 0x7f81e809d480 2024-08-19T10:39:25.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.474+0000 7f81e57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "1"}] 2024-08-19T10:39:25.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.474+0000 7f81e57fa640 10 monclient: _finish_command 2 = mon:22 specified pg_num 1 < pg_num_min 2 2024-08-19T10:39:25.474 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: specified pg_num 1 < pg_num_min 2 2024-08-19T10:39:25.480 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:25.480 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool set testpool1234 pg_num 44 2024-08-19T10:39:25.480 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:25.480 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set testpool1234 pg_num 44 2024-08-19T10:39:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient: get_monmap_and_config 2024-08-19T10:39:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient: build_initial_monmap 2024-08-19T10:39:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient: monmap: 2024-08-19T10:39:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:25.603664+0000 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:25.603664+0000 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:25.604 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 1 Processor -- start 2024-08-19T10:39:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 1 -- start start 2024-08-19T10:39:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient: init 2024-08-19T10:39:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient: _add_conns ranks=[6,9,0,5,10,7,1,4,3,2,8] 2024-08-19T10:39:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a58d0 0x7f40cc0a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient(hunting): picked mon.noname-d con 0x7f40cc0a58d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40cc0722b0 0x7f40cc0a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient(hunting): picked mon.noname-a con 0x7f40cc0722b0 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f40cc137680 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40cc0a40f0 con 0x7f40cc137680 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f40cc072730 con 0x7f40cc0722b0 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f40cc0a6730 con 0x7f40cc0a58d0 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40d2234640 10 monclient(hunting): authenticate will time out at 1011.173586s 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a58d0 0x7f40cc0a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40caffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40cc0722b0 0x7f40cc0a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a58d0 0x7f40cc0a5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:49298/0 (socket says 172.21.15.190:49298) 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 1 -- 172.21.15.190:0/1543063069 learned_addr learned my addr 172.21.15.190:0/1543063069 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40caffd640 1 --2- 172.21.15.190:0/1543063069 >> v2:172.21.15.89:3300/0 conn(0x7f40cc0722b0 0x7f40cc0a61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:36516/0 (socket says 172.21.15.190:36516) 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 10 monclient(hunting): get_auth_request con 0x7f40cc0a58d0 auth_method 0 2024-08-19T10:39:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40caffd640 10 monclient(hunting): get_auth_request con 0x7f40cc0722b0 auth_method 0 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40caffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40caffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40caffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40caffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40caffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40caffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 10 monclient(hunting): handle_auth_done global_id 4425 payload 274 2024-08-19T10:39:25.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 1 -- 172.21.15.190:0/1543063069 >> v2:172.21.15.89:3300/0 conn(0x7f40cc0722b0 msgr2=0x7f40cc0a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:25.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 1 --2- 172.21.15.190:0/1543063069 >> v2:172.21.15.89:3300/0 conn(0x7f40cc0722b0 0x7f40cc0a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:25.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 1 -- 172.21.15.190:0/1543063069 >> v1:172.21.15.89:6789/0 conn(0x7f40cc137680 legacy=0x7f40cc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:25.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:39:25.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 1 monclient: found mon.noname-d 2024-08-19T10:39:25.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:25.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:25.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.602+0000 7f40cb7fe640 1 -- 172.21.15.190:0/1543063069 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc073d90 con 0x7f40cc0a58d0 2024-08-19T10:39:25.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.606+0000 7f40cb7fe640 1 -- 172.21.15.190:0/1543063069 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a58d0 msgr2=0x7f40cc0a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:25.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.606+0000 7f40cb7fe640 1 --2- 172.21.15.190:0/1543063069 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a58d0 0x7f40cc0a5cb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009e50 tx=0x7f40c401c710 comp rx=0 tx=0).ready entity=mon.6 client_cookie=98e1da9dc42a7409 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:25.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.606+0000 7f40ca7fc640 1 -- 172.21.15.190:0/1543063069 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40c40090d0 con 0x7f40cc0a58d0 2024-08-19T10:39:25.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.606+0000 7f40ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:25.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40ca7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:25.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40ca7fc640 10 monclient: dump: 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:25.612 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40ca7fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40ca7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40ca7fc640 1 -- 172.21.15.190:0/1543063069 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c40ae070 con 0x7f40cc0a58d0 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40ca7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 5 monclient: authenticate success, global_id 4425 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40ca7fc640 1 -- 172.21.15.190:0/1543063069 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40c40a9d40 con 0x7f40cc0a58d0 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 4 set_mon_vals no callback set 2024-08-19T10:39:25.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40ca7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40ca7fc640 10 monclient: dump: 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:25.614 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient: shutdown 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 -- 172.21.15.190:0/1543063069 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a58d0 msgr2=0x7f40cc0a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 --2- 172.21.15.190:0/1543063069 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a58d0 0x7f40cc0a5cb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009e50 tx=0x7f40c401c710 comp rx=0 tx=0).stop 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 -- 172.21.15.190:0/1543063069 shutdown_connections 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 --2- 172.21.15.190:0/1543063069 >> v2:172.21.15.89:3300/0 conn(0x7f40cc0722b0 0x7f40cc0a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 --2- 172.21.15.190:0/1543063069 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a58d0 0x7f40cc0a5cb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 -- 172.21.15.190:0/1543063069 >> 172.21.15.190:0/1543063069 conn(0x7f40cc0aa4c0 msgr2=0x7f40cc0a69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:25.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 -- 172.21.15.190:0/1543063069 shutdown_connections 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 -- 172.21.15.190:0/1543063069 wait complete. 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient: build_initial_monmap 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient: monmap: 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:25.616 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:25.617 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:25.617 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:25.617 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:25.617 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 Processor -- start 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 -- start start 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient: init 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient: _add_conns ranks=[7,5,2,4,1,0,3,8,6] 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f40cc0722b0 0x7f40cc136490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient(hunting): picked mon.noname-i con 0x7f40cc0722b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f40cc0a58d0 0x7f40cc13b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient(hunting): picked mon.noname-h con 0x7f40cc0a58d0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc137680 0x7f40cc13bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient(hunting): picked mon.noname-b con 0x7f40cc137680 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:25.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc074610 con 0x7f40cc137680 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f40cc074490 con 0x7f40cc0a58d0 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f40cc142100 con 0x7f40cc0722b0 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40d2234640 10 monclient(hunting): authenticate will time out at 1011.182347s 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40cb7fe640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f40cc0722b0 msgr2=0x7f40cc136490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40cb7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f40cc0722b0 0x7f40cc136490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f40cc0a58d0 0x7f40cc13b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f40cc0a58d0 0x7f40cc13b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:58090/0 (socket says 172.21.15.190:58090) 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 1 -- 172.21.15.190:0/2474300224 learned_addr learned my addr 172.21.15.190:0/2474300224 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 10 monclient(hunting): get_auth_request con 0x7f40cc0a58d0 auth_method 0 2024-08-19T10:39:25.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40cbfff640 1 --2- 172.21.15.190:0/2474300224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc137680 0x7f40cc13bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40cbfff640 10 monclient(hunting): get_auth_request con 0x7f40cc137680 auth_method 0 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40cbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:25.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 10 monclient(hunting): handle_auth_done global_id 4514 payload 824 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 1 -- 172.21.15.190:0/2474300224 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f40cc0722b0 msgr2=0x7f40cc136490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 1 --2- 172.21.15.190:0/2474300224 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f40cc0722b0 0x7f40cc136490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 1 -- 172.21.15.190:0/2474300224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc137680 msgr2=0x7f40cc13bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 1 --2- 172.21.15.190:0/2474300224 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc137680 0x7f40cc13bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 10 monclient: _finish_hunting 0 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 1 monclient: found mon.noname-h 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.610+0000 7f40caffd640 1 -- 172.21.15.190:0/2474300224 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc0a3900 con 0x7f40cc0a58d0 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40caffd640 1 -- 172.21.15.190:0/2474300224 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f40cc0a58d0 msgr2=0x7f40cc13b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40caffd640 1 --2- 172.21.15.190:0/2474300224 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f40cc0a58d0 0x7f40cc13b6e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f40b400e8d0 tx=0x7f40b400ed90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 1 -- 172.21.15.190:0/2474300224 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40b4017070 con 0x7f40cc0a58d0 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:25.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 10 monclient: dump: 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:25.626 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:25.627 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:25.627 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:25.627 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:25.627 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:25.627 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 10 monclient: _finish_auth 0 2024-08-19T10:39:25.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:25.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:25.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40d2234640 5 monclient: authenticate success, global_id 4514 2024-08-19T10:39:25.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 1 -- 172.21.15.190:0/2474300224 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b4013470 con 0x7f40cc0a58d0 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40d2234640 10 monclient: _renew_subs 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40d2234640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40d2234640 1 -- 172.21.15.190:0/2474300224 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc1209d0 con 0x7f40cc0a58d0 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 1 -- 172.21.15.190:0/2474300224 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40b4013a90 con 0x7f40cc0a58d0 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 10 monclient: dump: 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.628 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c97fa640 4 set_mon_vals no callback set 2024-08-19T10:39:25.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40d2234640 10 monclient: _renew_subs 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40d2234640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40d2234640 1 -- 172.21.15.190:0/2474300224 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc120f90 con 0x7f40cc0a58d0 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40d2234640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40d2234640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40d2234640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40d2234640 1 -- 172.21.15.190:0/2474300224 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40cc137f90 con 0x7f40cc0a58d0 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 1 -- 172.21.15.190:0/2474300224 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f40b401a070 con 0x7f40cc0a58d0 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 1 --2- 172.21.15.190:0/2474300224 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f40ac03be30 0x7f40ac03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40c3fff640 1 -- 172.21.15.190:0/2474300224 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7f40b400f040 con 0x7f40cc0a58d0 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40cb7fe640 1 --2- 172.21.15.190:0/2474300224 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f40ac03be30 0x7f40ac03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.614+0000 7f40cb7fe640 10 monclient: get_auth_request con 0x7f40ac03be30 auth_method 0 2024-08-19T10:39:25.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.618+0000 7f40cb7fe640 1 --2- 172.21.15.190:0/2474300224 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f40ac03be30 0x7f40ac03e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f40c40aa000 tx=0x7f40c40a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:25.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.618+0000 7f40c3fff640 1 -- 172.21.15.190:0/2474300224 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f40cc0a3900 con 0x7f40cc0a58d0 2024-08-19T10:39:25.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.618+0000 7f40c3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:25.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.618+0000 7f40c3fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.786+0000 7f40d2234640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-08-19T10:39:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.786+0000 7f40d2234640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-08-19T10:39:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.786+0000 7f40d2234640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.786+0000 7f40d2234640 1 -- 172.21.15.190:0/2474300224 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"} v 0) -- 0x7f40cc137b10 con 0x7f40cc0a58d0 2024-08-19T10:39:25.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.798+0000 7f40c3fff640 1 -- 172.21.15.190:0/2474300224 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}]=-22 specified pg_num 44 < pg_num_max 33 v24) ==== 149+0+0 (secure 0 0 0) 0x7f40b4011350 con 0x7f40cc0a58d0 2024-08-19T10:39:25.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.798+0000 7f40c3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num", "val": "44"}] 2024-08-19T10:39:25.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.798+0000 7f40c3fff640 10 monclient: _finish_command 2 = mon:22 specified pg_num 44 < pg_num_max 33 2024-08-19T10:39:25.800 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: specified pg_num 44 < pg_num_max 33 2024-08-19T10:39:25.806 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:25.807 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set testpool1234 pg_num_min 0 2024-08-19T10:39:25.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient: get_monmap_and_config 2024-08-19T10:39:25.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient: build_initial_monmap 2024-08-19T10:39:25.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient: monmap: 2024-08-19T10:39:25.929 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:25.929 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:25.929 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:25.929496+0000 2024-08-19T10:39:25.929 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:25.929496+0000 2024-08-19T10:39:25.929 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:25.929 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.929 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:25.929 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:25.929 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:25.929 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:25.930 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:25.930 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:25.930 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:25.930 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:25.930 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:25.930 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:25.930 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:25.930 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 1 Processor -- start 2024-08-19T10:39:25.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 1 -- start start 2024-08-19T10:39:25.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient: init 2024-08-19T10:39:25.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:25.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient: _add_conns ranks=[4,1,7,9,6,0,3,5,8,10,2] 2024-08-19T10:39:25.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ea81396a0 0x7f7ea8139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient(hunting): picked mon.noname-c con 0x7f7ea81396a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:25.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7ea8139fc0 addr v1:172.21.15.190:6789/0 2024-08-19T10:39:25.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ea80a6ff0 0x7f7ea80a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient(hunting): picked mon.noname-i con 0x7f7ea80a6ff0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:25.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea80a83e0 con 0x7f7ea8139fc0 2024-08-19T10:39:25.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:25.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f7ea80a4f20 con 0x7f7ea81396a0 2024-08-19T10:39:25.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:25.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7ea80a2f30 con 0x7f7ea80a6ff0 2024-08-19T10:39:25.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:25.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eafc5e640 10 monclient(hunting): authenticate will time out at 1011.499489s 2024-08-19T10:39:25.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eae1fb640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ea80a6ff0 msgr2=0x7f7ea80a73f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:25.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7ead1f9640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f7ea8139fc0 legacy=0x7f7ea80a68e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:39:25.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7eae1fb640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ea80a6ff0 0x7f7ea80a73f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:25.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7ead9fa640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ea81396a0 0x7f7ea8139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:25.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7ead9fa640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ea81396a0 0x7f7ea8139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:50298/0 (socket says 172.21.15.190:50298) 2024-08-19T10:39:25.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7ead9fa640 1 -- 172.21.15.190:0/3035323641 learned_addr learned my addr 172.21.15.190:0/3035323641 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:25.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7ead9fa640 10 monclient(hunting): get_auth_request con 0x7f7ea81396a0 auth_method 0 2024-08-19T10:39:25.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7ead9fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:25.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7ead9fa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:25.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7ead9fa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:25.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7ead9fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:25.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.926+0000 7f7ead9fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:25.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.930+0000 7f7ead9fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:25.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.930+0000 7f7ead9fa640 10 monclient(hunting): handle_auth_done global_id 4612 payload 274 2024-08-19T10:39:25.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.930+0000 7f7ead9fa640 1 -- 172.21.15.190:0/3035323641 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ea80a6ff0 msgr2=0x7f7ea80a73f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:25.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.930+0000 7f7ead9fa640 1 --2- 172.21.15.190:0/3035323641 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ea80a6ff0 0x7f7ea80a73f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:25.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.930+0000 7f7ead9fa640 1 -- 172.21.15.190:0/3035323641 >> v1:172.21.15.190:6789/0 conn(0x7f7ea8139fc0 legacy=0x7f7ea80a68e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:25.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.930+0000 7f7ead9fa640 10 monclient: _finish_hunting 0 2024-08-19T10:39:25.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.930+0000 7f7ead9fa640 1 monclient: found mon.noname-c 2024-08-19T10:39:25.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.930+0000 7f7ead9fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:25.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.930+0000 7f7ead9fa640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:25.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.930+0000 7f7ead9fa640 1 -- 172.21.15.190:0/3035323641 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea80a35e0 con 0x7f7ea81396a0 2024-08-19T10:39:25.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7ead9fa640 1 -- 172.21.15.190:0/3035323641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ea81396a0 msgr2=0x7f7ea8139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:25.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7ead9fa640 1 --2- 172.21.15.190:0/3035323641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ea81396a0 0x7f7ea8139a80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00098d0 tx=0x7f7ea001c460 comp rx=0 tx=0).ready entity=mon.4 client_cookie=75e8cc2835c57e66 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:25.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 1 -- 172.21.15.190:0/3035323641 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7ea00b0070 con 0x7f7ea81396a0 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 10 monclient: dump: 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:25.941 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:25.942 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:25.942 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:25.942 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:25.942 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:25.942 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:25.942 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:25.942 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 10 monclient: _finish_auth 0 2024-08-19T10:39:25.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 10 monclient: _check_auth_tickets 2024-08-19T10:39:25.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:25.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 1 -- 172.21.15.190:0/3035323641 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea00aa810 con 0x7f7ea81396a0 2024-08-19T10:39:25.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 5 monclient: authenticate success, global_id 4612 2024-08-19T10:39:25.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:25.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:25.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:25.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 4 set_mon_vals no callback set 2024-08-19T10:39:25.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 1 -- 172.21.15.190:0/3035323641 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7ea00aae30 con 0x7f7ea81396a0 2024-08-19T10:39:25.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:25.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:25.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eac9f8640 10 monclient: dump: 2024-08-19T10:39:25.943 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:25.944 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:25.945 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:25.945 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:25.945 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient: shutdown 2024-08-19T10:39:25.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:25.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3035323641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ea81396a0 msgr2=0x7f7ea8139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:25.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 --2- 172.21.15.190:0/3035323641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ea81396a0 0x7f7ea8139a80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00098d0 tx=0x7f7ea001c460 comp rx=0 tx=0).stop 2024-08-19T10:39:25.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3035323641 shutdown_connections 2024-08-19T10:39:25.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 --2- 172.21.15.190:0/3035323641 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ea80a6ff0 0x7f7ea80a73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:25.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 --2- 172.21.15.190:0/3035323641 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ea81396a0 0x7f7ea8139a80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:25.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3035323641 >> 172.21.15.190:0/3035323641 conn(0x7f7ea80abee0 msgr2=0x7f7ea80ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:25.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3035323641 shutdown_connections 2024-08-19T10:39:25.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3035323641 wait complete. 2024-08-19T10:39:25.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient: build_initial_monmap 2024-08-19T10:39:25.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient: monmap: 2024-08-19T10:39:25.946 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:25.946 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:25.946 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:25.946 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:25.946 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:25.946 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.947 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:25.947 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:25.947 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:25.947 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:25.947 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:25.947 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:25.947 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:25.947 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:25.947 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:25.947 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 Processor -- start 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 -- start start 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient: init 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient: _add_conns ranks=[6,0,2,7,8,5,1,4,3] 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7ea80a6ff0 0x7f7ea81d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient(hunting): picked mon.noname-d con 0x7f7ea80a6ff0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea81396a0 0x7f7ea8138950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient(hunting): picked mon.noname-a con 0x7f7ea81396a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ea8139fc0 0x7f7ea813dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient(hunting): picked mon.noname-b con 0x7f7ea8139fc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea813a3a0 con 0x7f7ea81396a0 2024-08-19T10:39:25.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:25.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea80a7930 con 0x7f7ea8139fc0 2024-08-19T10:39:25.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:25.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7ea8140110 con 0x7f7ea80a6ff0 2024-08-19T10:39:25.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:25.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.934+0000 7f7eafc5e640 10 monclient(hunting): authenticate will time out at 1011.507772s 2024-08-19T10:39:25.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead1f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea81396a0 0x7f7ea8138950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:25.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead9fa640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7ea80a6ff0 0x7f7ea81d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:25.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead1f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea81396a0 0x7f7ea8138950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:36518/0 (socket says 172.21.15.190:36518) 2024-08-19T10:39:25.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead1f9640 1 -- 172.21.15.190:0/3605976794 learned_addr learned my addr 172.21.15.190:0/3605976794 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:25.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead1f9640 10 monclient(hunting): get_auth_request con 0x7f7ea81396a0 auth_method 0 2024-08-19T10:39:25.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead1f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:25.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead1f9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:25.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead1f9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:25.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ea8139fc0 0x7f7ea813dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:25.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 10 monclient(hunting): get_auth_request con 0x7f7ea8139fc0 auth_method 0 2024-08-19T10:39:25.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:25.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:25.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:25.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead9fa640 10 monclient(hunting): get_auth_request con 0x7f7ea80a6ff0 auth_method 0 2024-08-19T10:39:25.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead9fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:25.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead9fa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:25.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead9fa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:25.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead1f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:25.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead1f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:25.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead1f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:25.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:25.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:25.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:25.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead9fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:25.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead9fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:25.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead9fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:25.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 10 monclient(hunting): handle_auth_done global_id 4493 payload 824 2024-08-19T10:39:25.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 1 -- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7ea80a6ff0 msgr2=0x7f7ea81d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:25.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7ea80a6ff0 0x7f7ea81d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:25.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 1 -- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea81396a0 msgr2=0x7f7ea8138950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:25.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea81396a0 0x7f7ea8138950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:25.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 10 monclient: _finish_hunting 0 2024-08-19T10:39:25.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 1 monclient: found mon.noname-b 2024-08-19T10:39:25.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:25.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:25.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 1 -- 172.21.15.190:0/3605976794 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea813f5b0 con 0x7f7ea8139fc0 2024-08-19T10:39:25.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ead1f9640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea81396a0 0x7f7ea8138950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:25.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 1 -- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ea8139fc0 msgr2=0x7f7ea813dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:25.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eae1fb640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ea8139fc0 0x7f7ea813dc30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800bdd0 tx=0x7f7e98009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:25.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 1 -- 172.21.15.190:0/3605976794 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e9800ce00 con 0x7f7ea8139fc0 2024-08-19T10:39:25.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:25.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:25.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 10 monclient: dump: 2024-08-19T10:39:25.955 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:25.955 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:25.955 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.956 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.956 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:25.956 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.956 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:25.956 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:25.956 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:25.956 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:25.956 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:25.956 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:25.956 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:25.956 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:25.957 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eafc5e640 5 monclient: authenticate success, global_id 4493 2024-08-19T10:39:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eafc5e640 10 monclient: _renew_subs 2024-08-19T10:39:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eafc5e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3605976794 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea8122a60 con 0x7f7ea8139fc0 2024-08-19T10:39:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eafc5e640 10 monclient: _renew_subs 2024-08-19T10:39:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eafc5e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3605976794 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea81d21d0 con 0x7f7ea8139fc0 2024-08-19T10:39:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 1 -- 172.21.15.190:0/3605976794 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e98010030 con 0x7f7ea8139fc0 2024-08-19T10:39:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 1 -- 172.21.15.190:0/3605976794 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e9801b060 con 0x7f7ea8139fc0 2024-08-19T10:39:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea7fff640 4 set_mon_vals no callback set 2024-08-19T10:39:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.938+0000 7f7ea67fc640 10 monclient: dump: 2024-08-19T10:39:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:25.959 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:25.959 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.959 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:25.959 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:25.959 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:25.959 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:25.959 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:25.959 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:25.959 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:25.959 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:25.959 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:25.960 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:25.960 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:25.960 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:25.960 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:25.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.942+0000 7f7eafc5e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:25.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.942+0000 7f7eafc5e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:25.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.942+0000 7f7eafc5e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:25.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.942+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3605976794 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e88005090 con 0x7f7ea8139fc0 2024-08-19T10:39:25.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.942+0000 7f7ea67fc640 1 -- 172.21.15.190:0/3605976794 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7e98006150 con 0x7f7ea8139fc0 2024-08-19T10:39:25.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.942+0000 7f7ea67fc640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e9403be00 0x7f7e9403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:25.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.942+0000 7f7ea67fc640 1 -- 172.21.15.190:0/3605976794 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 4729+0+0 (secure 0 0 0) 0x7f7e9804f380 con 0x7f7ea8139fc0 2024-08-19T10:39:25.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.942+0000 7f7ead9fa640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e9403be00 0x7f7e9403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:25.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.942+0000 7f7ead9fa640 10 monclient: get_auth_request con 0x7f7e9403be00 auth_method 0 2024-08-19T10:39:25.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.946+0000 7f7ead9fa640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e9403be00 0x7f7e9403e2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00ab000 tx=0x7f7ea00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:25.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.954+0000 7f7ea67fc640 1 -- 172.21.15.190:0/3605976794 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7e98013cf0 con 0x7f7ea8139fc0 2024-08-19T10:39:25.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.954+0000 7f7ea67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:25.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:25.954+0000 7f7ea67fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:26.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:26.122+0000 7f7eafc5e640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-08-19T10:39:26.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:26.122+0000 7f7eafc5e640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-08-19T10:39:26.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:26.122+0000 7f7eafc5e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:26.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:26.122+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3605976794 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"} v 0) -- 0x7f7e880056f0 con 0x7f7ea8139fc0 2024-08-19T10:39:26.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:26.822+0000 7f7ea67fc640 1 -- 172.21.15.190:0/3605976794 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}]=0 set pool 3 pg_num_min to 0 v25) ==== 143+0+0 (secure 0 0 0) 0x7f7e880056f0 con 0x7f7ea8139fc0 2024-08-19T10:39:26.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:26.822+0000 7f7ea67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-08-19T10:39:26.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:26.822+0000 7f7ea67fc640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_min to 0 2024-08-19T10:39:26.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:26.926+0000 7f7eafc5e640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-08-19T10:39:26.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:26.926+0000 7f7eafc5e640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-08-19T10:39:26.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:26.926+0000 7f7eafc5e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:26.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:26.926+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3605976794 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"} v 0) -- 0x7f7e88005ac0 con 0x7f7ea8139fc0 2024-08-19T10:39:27.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.894+0000 7f7ea67fc640 1 -- 172.21.15.190:0/3605976794 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}]=0 set pool 3 pg_num_min to 0 v26) ==== 143+0+0 (secure 0 0 0) 0x7f7e9804ce00 con 0x7f7ea8139fc0 2024-08-19T10:39:27.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.894+0000 7f7ea67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_min", "val": "0"}] 2024-08-19T10:39:27.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.894+0000 7f7ea67fc640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_min to 0 2024-08-19T10:39:27.895 INFO:tasks.workunit.client.0.smithi190.stderr:set pool 3 pg_num_min to 0 2024-08-19T10:39:27.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3605976794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e9403be00 msgr2=0x7f7e9403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:27.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e9403be00 0x7f7e9403e2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00ab000 tx=0x7f7ea00a6000 comp rx=0 tx=0).stop 2024-08-19T10:39:27.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 10 monclient: shutdown 2024-08-19T10:39:27.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:27.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ea8139fc0 msgr2=0x7f7ea813dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:27.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ea8139fc0 0x7f7ea813dc30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800bdd0 tx=0x7f7e98009ed0 comp rx=0 tx=0).stop 2024-08-19T10:39:27.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3605976794 shutdown_connections 2024-08-19T10:39:27.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e9403be00 0x7f7e9403e2c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:27.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ea8139fc0 0x7f7ea813dc30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:27.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea81396a0 0x7f7ea8138950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:27.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 --2- 172.21.15.190:0/3605976794 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7ea80a6ff0 0x7f7ea81d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:27.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3605976794 >> 172.21.15.190:0/3605976794 conn(0x7f7ea80abee0 msgr2=0x7f7ea80aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:27.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3605976794 shutdown_connections 2024-08-19T10:39:27.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:27.898+0000 7f7eafc5e640 1 -- 172.21.15.190:0/3605976794 wait complete. 2024-08-19T10:39:27.922 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool get testpool1234 pg_num_min 2024-08-19T10:39:27.922 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:27.922 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool get testpool1234 pg_num_min 2024-08-19T10:39:28.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.042+0000 7f2a9917a640 10 monclient: get_monmap_and_config 2024-08-19T10:39:28.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.042+0000 7f2a9917a640 10 monclient: build_initial_monmap 2024-08-19T10:39:28.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.042+0000 7f2a9917a640 10 monclient: monmap: 2024-08-19T10:39:28.045 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:28.045 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:28.045 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:28.046323+0000 2024-08-19T10:39:28.046 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:28.046323+0000 2024-08-19T10:39:28.046 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:28.046 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.046 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:28.046 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:28.046 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:28.046 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:28.046 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:28.046 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:28.046 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:28.047 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:28.047 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:28.047 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:28.047 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:28.047 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.042+0000 7f2a9917a640 1 Processor -- start 2024-08-19T10:39:28.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.042+0000 7f2a9917a640 1 -- start start 2024-08-19T10:39:28.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.042+0000 7f2a9917a640 10 monclient: init 2024-08-19T10:39:28.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:28.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 10 monclient: _add_conns ranks=[0,4,7,8,9,5,3,2,6,1,10] 2024-08-19T10:39:28.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2a94139980 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:28.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a940a5730 0x7f2a940a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 10 monclient(hunting): picked mon.noname-c con 0x7f2a940a5730 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:28.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2a9413e8c0 0x7f2a94140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 10 monclient(hunting): picked mon.noname-i con 0x7f2a9413e8c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:28.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a940a8470 con 0x7f2a94139980 2024-08-19T10:39:28.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:28.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f2a9409ca60 con 0x7f2a940a5730 2024-08-19T10:39:28.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:28.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2a940a3640 con 0x7f2a9413e8c0 2024-08-19T10:39:28.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:28.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9917a640 10 monclient(hunting): authenticate will time out at 1013.616229s 2024-08-19T10:39:28.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9359e640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2a9413e8c0 msgr2=0x7f2a94140cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:28.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a9359e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2a9413e8c0 0x7f2a94140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:28.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a940a5730 0x7f2a940a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a92d9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2a94139980 0x7f2a94139d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:33566/0 (socket says 172.21.15.190:33566) 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a92d9d640 1 -- 172.21.15.190:0/3595611654 learned_addr learned my addr 172.21.15.190:0/3595611654 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 10 monclient(hunting): get_auth_request con 0x7f2a940a5730 auth_method 0 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 10 monclient(hunting): handle_auth_done global_id 4621 payload 274 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 1 -- 172.21.15.190:0/3595611654 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2a9413e8c0 msgr2=0x7f2a94140cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:28.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 1 --2- 172.21.15.190:0/3595611654 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2a9413e8c0 0x7f2a94140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 1 -- 172.21.15.190:0/3595611654 >> v1:172.21.15.89:6789/0 conn(0x7f2a94139980 legacy=0x7f2a94139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:28.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:39:28.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 1 monclient: found mon.noname-c 2024-08-19T10:39:28.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:28.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:28.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.046+0000 7f2a8a59c640 1 -- 172.21.15.190:0/3595611654 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a940a3850 con 0x7f2a940a5730 2024-08-19T10:39:28.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a8a59c640 1 -- 172.21.15.190:0/3595611654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a940a5730 msgr2=0x7f2a940a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:28.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a8a59c640 1 --2- 172.21.15.190:0/3595611654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a940a5730 0x7f2a940a5b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009820 tx=0x7f2a7c01c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=6f0e5c40f35da7f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:28.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 1 -- 172.21.15.190:0/3595611654 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a7c0b0070 con 0x7f2a940a5730 2024-08-19T10:39:28.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:28.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 10 monclient: dump: 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:28.055 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 10 monclient: _finish_auth 0 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 10 monclient: _check_auth_tickets 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 1 -- 172.21.15.190:0/3595611654 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a7c0aa810 con 0x7f2a940a5730 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 1 -- 172.21.15.190:0/3595611654 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a7c0aae30 con 0x7f2a940a5730 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9259c640 10 monclient: dump: 2024-08-19T10:39:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:28.057 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:28.057 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.057 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.057 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:28.057 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.057 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:28.057 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:28.057 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:28.057 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:28.058 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:28.058 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:28.058 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:28.058 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:28.058 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:28.058 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 5 monclient: authenticate success, global_id 4621 2024-08-19T10:39:28.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:28.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 4 set_mon_vals no callback set 2024-08-19T10:39:28.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 10 monclient: shutdown 2024-08-19T10:39:28.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 1 -- 172.21.15.190:0/3595611654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a940a5730 msgr2=0x7f2a940a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 1 --2- 172.21.15.190:0/3595611654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a940a5730 0x7f2a940a5b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009820 tx=0x7f2a7c01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 1 -- 172.21.15.190:0/3595611654 shutdown_connections 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 1 --2- 172.21.15.190:0/3595611654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a940a5730 0x7f2a940a5b10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 1 --2- 172.21.15.190:0/3595611654 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2a9413e8c0 0x7f2a94140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 1 -- 172.21.15.190:0/3595611654 >> 172.21.15.190:0/3595611654 conn(0x7f2a940abde0 msgr2=0x7f2a940ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 1 -- 172.21.15.190:0/3595611654 shutdown_connections 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 1 -- 172.21.15.190:0/3595611654 wait complete. 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 10 monclient: build_initial_monmap 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.050+0000 7f2a9917a640 10 monclient: monmap: 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 1 Processor -- start 2024-08-19T10:39:28.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 1 -- start start 2024-08-19T10:39:28.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient: init 2024-08-19T10:39:28.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient: _add_conns ranks=[5,2,7,6,1,4,0,8,3] 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a940a5730 0x7f2a941327f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient(hunting): picked mon.noname-h con 0x7f2a940a5730 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a94139980 0x7f2a94132d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient(hunting): picked mon.noname-b con 0x7f2a94139980 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2a9413e8c0 0x7f2a9413b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient(hunting): picked mon.noname-i con 0x7f2a9413e8c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f2a94143b90 con 0x7f2a94139980 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2a94143a10 con 0x7f2a940a5730 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:28.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2a94143d10 con 0x7f2a9413e8c0 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient(hunting): authenticate will time out at 1013.624661s 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9359e640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2a9413e8c0 msgr2=0x7f2a9413b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9359e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2a9413e8c0 0x7f2a9413b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a940a5730 0x7f2a941327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a940a5730 0x7f2a941327f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:58102/0 (socket says 172.21.15.190:58102) 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 1 -- 172.21.15.190:0/2093381031 learned_addr learned my addr 172.21.15.190:0/2093381031 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 10 monclient(hunting): get_auth_request con 0x7f2a940a5730 auth_method 0 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:28.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8a59c640 1 --2- 172.21.15.190:0/2093381031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a94139980 0x7f2a94132d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8a59c640 10 monclient(hunting): get_auth_request con 0x7f2a94139980 auth_method 0 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 10 monclient(hunting): handle_auth_done global_id 4523 payload 824 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 1 -- 172.21.15.190:0/2093381031 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2a9413e8c0 msgr2=0x7f2a9413b540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 1 --2- 172.21.15.190:0/2093381031 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2a9413e8c0 0x7f2a9413b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 1 -- 172.21.15.190:0/2093381031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a94139980 msgr2=0x7f2a94132d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 1 --2- 172.21.15.190:0/2093381031 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a94139980 0x7f2a94132d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:39:28.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 1 monclient: found mon.noname-h 2024-08-19T10:39:28.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:28.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:28.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 1 -- 172.21.15.190:0/2093381031 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9413d5c0 con 0x7f2a940a5730 2024-08-19T10:39:28.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 1 -- 172.21.15.190:0/2093381031 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a940a5730 msgr2=0x7f2a941327f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:28.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a92d9d640 1 --2- 172.21.15.190:0/2093381031 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a940a5730 0x7f2a941327f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400c4a0 tx=0x7f2a8400c960 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:28.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 1 -- 172.21.15.190:0/2093381031 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a84017070 con 0x7f2a940a5730 2024-08-19T10:39:28.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:28.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:28.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 10 monclient: dump: 2024-08-19T10:39:28.068 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:28.069 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:28.070 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:28.070 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:28.070 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 10 monclient: _finish_auth 0 2024-08-19T10:39:28.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:28.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:28.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 1 -- 172.21.15.190:0/2093381031 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a84005d00 con 0x7f2a940a5730 2024-08-19T10:39:28.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 5 monclient: authenticate success, global_id 4523 2024-08-19T10:39:28.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient: _renew_subs 2024-08-19T10:39:28.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:28.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 1 -- 172.21.15.190:0/2093381031 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a94134750 con 0x7f2a940a5730 2024-08-19T10:39:28.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:28.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 1 -- 172.21.15.190:0/2093381031 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a840074e0 con 0x7f2a940a5730 2024-08-19T10:39:28.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:28.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:28.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a91d9b640 4 set_mon_vals no callback set 2024-08-19T10:39:28.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a8bfff640 10 monclient: dump: 2024-08-19T10:39:28.071 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:28.071 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:28.071 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.071 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.071 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient: _renew_subs 2024-08-19T10:39:28.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.054+0000 7f2a9917a640 1 -- 172.21.15.190:0/2093381031 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9409d320 con 0x7f2a940a5730 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.058+0000 7f2a9917a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.058+0000 7f2a9917a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.058+0000 7f2a9917a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.058+0000 7f2a9917a640 1 -- 172.21.15.190:0/2093381031 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a940a7360 con 0x7f2a940a5730 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.058+0000 7f2a8bfff640 1 -- 172.21.15.190:0/2093381031 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2a84007070 con 0x7f2a940a5730 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.058+0000 7f2a8bfff640 1 --2- 172.21.15.190:0/2093381031 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2a7803be00 0x7f2a7803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.058+0000 7f2a8bfff640 1 -- 172.21.15.190:0/2093381031 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(26..26 src has 1..26) ==== 4713+0+0 (secure 0 0 0) 0x7f2a8404dde0 con 0x7f2a940a5730 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.058+0000 7f2a8a59c640 1 --2- 172.21.15.190:0/2093381031 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2a7803be00 0x7f2a7803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.058+0000 7f2a8a59c640 10 monclient: get_auth_request con 0x7f2a7803be00 auth_method 0 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.058+0000 7f2a8a59c640 1 --2- 172.21.15.190:0/2093381031 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2a7803be00 0x7f2a7803e2c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0ab000 tx=0x7f2a7c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.062+0000 7f2a8bfff640 1 -- 172.21.15.190:0/2093381031 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f2a94134750 con 0x7f2a940a5730 2024-08-19T10:39:28.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.062+0000 7f2a8bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:28.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.062+0000 7f2a8bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:28.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.230+0000 7f2a9917a640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-08-19T10:39:28.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.230+0000 7f2a9917a640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-08-19T10:39:28.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.230+0000 7f2a9917a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:28.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.230+0000 7f2a9917a640 1 -- 172.21.15.190:0/2093381031 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"} v 0) -- 0x7f2a940a5b90 con 0x7f2a940a5730 2024-08-19T10:39:28.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.234+0000 7f2a8bfff640 1 -- 172.21.15.190:0/2093381031 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}]=-2 option 'pg_num_min' is not set on pool 'testpool1234' v26) ==== 158+0+0 (secure 0 0 0) 0x7f2a84011bf0 con 0x7f2a940a5730 2024-08-19T10:39:28.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.234+0000 7f2a8bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_min"}] 2024-08-19T10:39:28.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.234+0000 7f2a8bfff640 10 monclient: _finish_command 2 = mon:2 option 'pg_num_min' is not set on pool 'testpool1234' 2024-08-19T10:39:28.235 INFO:tasks.workunit.client.0.smithi190.stderr:Error ENOENT: option 'pg_num_min' is not set on pool 'testpool1234' 2024-08-19T10:39:28.241 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:28.242 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set testpool1234 pg_num_max 0 2024-08-19T10:39:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient: get_monmap_and_config 2024-08-19T10:39:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient: build_initial_monmap 2024-08-19T10:39:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient: monmap: 2024-08-19T10:39:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:28.361526+0000 2024-08-19T10:39:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:28.361526+0000 2024-08-19T10:39:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 1 Processor -- start 2024-08-19T10:39:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 1 -- start start 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient: init 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient: _add_conns ranks=[6,5,8,9,1,0,10,7,3,2,4] 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f699c1396a0 0x7f699c139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient(hunting): picked mon.noname-d con 0x7f699c1396a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c139fc0 0x7f699c0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient(hunting): picked mon.noname-h con 0x7f699c139fc0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f699c0a6e20 0x7f699c0a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient(hunting): picked mon.noname-e con 0x7f699c0a6e20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f699c0a4f20 con 0x7f699c139fc0 2024-08-19T10:39:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f699c0a83e0 con 0x7f699c1396a0 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f699c0a2f30 con 0x7f699c0a6e20 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f69a10b0640 10 monclient(hunting): authenticate will time out at 1013.931323s 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c139fc0 0x7f699c0a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c139fc0 0x7f699c0a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:58112/0 (socket says 172.21.15.190:58112) 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 1 -- 172.21.15.190:0/303568925 learned_addr learned my addr 172.21.15.190:0/303568925 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 10 monclient(hunting): get_auth_request con 0x7f699c139fc0 auth_method 0 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:28.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699ad9d640 1 --2- 172.21.15.190:0/303568925 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f699c1396a0 0x7f699c139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699b59e640 1 --2- 172.21.15.190:0/303568925 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f699c0a6e20 0x7f699c0a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699ad9d640 10 monclient(hunting): get_auth_request con 0x7f699c1396a0 auth_method 0 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699b59e640 10 monclient(hunting): get_auth_request con 0x7f699c0a6e20 auth_method 0 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699b59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699b59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 10 monclient(hunting): handle_auth_done global_id 4532 payload 274 2024-08-19T10:39:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 1 -- 172.21.15.190:0/303568925 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f699c0a6e20 msgr2=0x7f699c0a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 1 --2- 172.21.15.190:0/303568925 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f699c0a6e20 0x7f699c0a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 1 -- 172.21.15.190:0/303568925 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f699c1396a0 msgr2=0x7f699c139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 1 --2- 172.21.15.190:0/303568925 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f699c1396a0 0x7f699c139a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:39:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 1 monclient: found mon.noname-h 2024-08-19T10:39:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.358+0000 7f699a59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.362+0000 7f699a59c640 1 -- 172.21.15.190:0/303568925 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c0a35e0 con 0x7f699c139fc0 2024-08-19T10:39:28.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 1 -- 172.21.15.190:0/303568925 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c139fc0 msgr2=0x7f699c0a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:28.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 1 --2- 172.21.15.190:0/303568925 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c139fc0 0x7f699c0a68e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f69900098a0 tx=0x7f699001c440 comp rx=0 tx=0).ready entity=mon.5 client_cookie=1ae0f0cde03f62c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:28.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 1 -- 172.21.15.190:0/303568925 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69900b0070 con 0x7f699c139fc0 2024-08-19T10:39:28.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:28.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:28.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 10 monclient: dump: 2024-08-19T10:39:28.374 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:28.375 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 10 monclient: _finish_auth 0 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 1 -- 172.21.15.190:0/303568925 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69900aa810 con 0x7f699c139fc0 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 1 -- 172.21.15.190:0/303568925 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69900aae30 con 0x7f699c139fc0 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 5 monclient: authenticate success, global_id 4532 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 4 set_mon_vals no callback set 2024-08-19T10:39:28.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:28.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:28.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f6999d9b640 10 monclient: dump: 2024-08-19T10:39:28.377 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:28.377 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:28.377 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.377 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.377 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:28.377 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.377 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:28.377 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:28.377 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:28.378 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:28.378 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:28.378 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:28.378 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:28.378 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:28.378 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:28.378 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient: shutdown 2024-08-19T10:39:28.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:28.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 -- 172.21.15.190:0/303568925 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c139fc0 msgr2=0x7f699c0a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 --2- 172.21.15.190:0/303568925 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c139fc0 0x7f699c0a68e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f69900098a0 tx=0x7f699001c440 comp rx=0 tx=0).stop 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 -- 172.21.15.190:0/303568925 shutdown_connections 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 --2- 172.21.15.190:0/303568925 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f699c0a6e20 0x7f699c0a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 --2- 172.21.15.190:0/303568925 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c139fc0 0x7f699c0a68e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 --2- 172.21.15.190:0/303568925 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f699c1396a0 0x7f699c139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 -- 172.21.15.190:0/303568925 >> 172.21.15.190:0/303568925 conn(0x7f699c0abee0 msgr2=0x7f699c0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 -- 172.21.15.190:0/303568925 shutdown_connections 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 -- 172.21.15.190:0/303568925 wait complete. 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient: build_initial_monmap 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient: monmap: 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:28.379 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:28.380 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 Processor -- start 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 -- start start 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient: init 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient: _add_conns ranks=[3,5,0,1,6,8,4,7,2] 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f699c0a6e20 0x7f699c1348b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient(hunting): picked mon.noname-g con 0x7f699c0a6e20 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c1396a0 0x7f699c134df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient(hunting): picked mon.noname-h con 0x7f699c1396a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f699c139fc0 0x7f699c09ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient(hunting): picked mon.noname-a con 0x7f699c139fc0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:28.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f699c13a3a0 con 0x7f699c139fc0 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f699c143390 con 0x7f699c0a6e20 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f699c1356f0 con 0x7f699c1396a0 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f69a10b0640 10 monclient(hunting): authenticate will time out at 1013.938793s 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699ad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f699c0a6e20 0x7f699c1348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c1396a0 0x7f699c134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699ad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f699c0a6e20 0x7f699c1348b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46412/0 (socket says 172.21.15.190:46412) 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699ad9d640 1 -- 172.21.15.190:0/3446707538 learned_addr learned my addr 172.21.15.190:0/3446707538 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699ad9d640 10 monclient(hunting): get_auth_request con 0x7f699c0a6e20 auth_method 0 2024-08-19T10:39:28.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 10 monclient(hunting): get_auth_request con 0x7f699c1396a0 auth_method 0 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:28.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699b59e640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f699c139fc0 0x7f699c09ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699b59e640 10 monclient(hunting): get_auth_request con 0x7f699c139fc0 auth_method 0 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699b59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699b59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 10 monclient(hunting): handle_auth_done global_id 4541 payload 824 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 1 -- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f699c0a6e20 msgr2=0x7f699c1348b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f699c0a6e20 0x7f699c1348b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 1 -- 172.21.15.190:0/3446707538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f699c139fc0 msgr2=0x7f699c09ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f699c139fc0 0x7f699c09ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 1 monclient: found mon.noname-h 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:28.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699a59c640 1 -- 172.21.15.190:0/3446707538 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c0a0120 con 0x7f699c1396a0 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.366+0000 7f699ad9d640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f699c0a6e20 0x7f699c1348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f699a59c640 1 -- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c1396a0 msgr2=0x7f699c134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f699a59c640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c1396a0 0x7f699c134df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69900ad000 tx=0x7f69900191e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 1 -- 172.21.15.190:0/3446707538 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69900b0070 con 0x7f699c1396a0 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 10 monclient: dump: 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.385 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:28.386 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 1 -- 172.21.15.190:0/3446707538 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69900b1040 con 0x7f699c1396a0 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69a10b0640 5 monclient: authenticate success, global_id 4541 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69a10b0640 10 monclient: _renew_subs 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69a10b0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69a10b0640 1 -- 172.21.15.190:0/3446707538 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c0a0400 con 0x7f699c1396a0 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 1 -- 172.21.15.190:0/3446707538 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69900044b0 con 0x7f699c1396a0 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:28.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 10 monclient: dump: 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:28.388 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f6998d99640 4 set_mon_vals no callback set 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69a10b0640 10 monclient: _renew_subs 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69a10b0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69a10b0640 1 -- 172.21.15.190:0/3446707538 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c09fe40 con 0x7f699c1396a0 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69a10b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69a10b0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69a10b0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69a10b0640 1 -- 172.21.15.190:0/3446707538 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699c139a80 con 0x7f699c1396a0 2024-08-19T10:39:28.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 1 -- 172.21.15.190:0/3446707538 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6990004040 con 0x7f699c1396a0 2024-08-19T10:39:28.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698803be00 0x7f698803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:28.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f69837fe640 1 -- 172.21.15.190:0/3446707538 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(26..26 src has 1..26) ==== 4713+0+0 (secure 0 0 0) 0x7f69900e8f10 con 0x7f699c1396a0 2024-08-19T10:39:28.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f699ad9d640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698803be00 0x7f698803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:28.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.370+0000 7f699ad9d640 10 monclient: get_auth_request con 0x7f698803be00 auth_method 0 2024-08-19T10:39:28.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.374+0000 7f699ad9d640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698803be00 0x7f698803e2c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f698c007e80 tx=0x7f698c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:28.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.374+0000 7f69837fe640 1 -- 172.21.15.190:0/3446707538 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6990016150 con 0x7f699c1396a0 2024-08-19T10:39:28.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.374+0000 7f69837fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:28.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.374+0000 7f69837fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:28.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.546+0000 7f69a10b0640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-08-19T10:39:28.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.546+0000 7f69a10b0640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-08-19T10:39:28.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.546+0000 7f69a10b0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:28.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:28.546+0000 7f69a10b0640 1 -- 172.21.15.190:0/3446707538 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"} v 0) -- 0x7f699c06c9c0 con 0x7f699c1396a0 2024-08-19T10:39:29.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:29.366+0000 7f69837fe640 1 -- 172.21.15.190:0/3446707538 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}]=0 set pool 3 pg_num_max to 0 v27) ==== 143+0+0 (secure 0 0 0) 0x7f69900c7df0 con 0x7f699c1396a0 2024-08-19T10:39:29.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:29.366+0000 7f69837fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-08-19T10:39:29.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:29.366+0000 7f69837fe640 10 monclient: _finish_command 2 = system:0 set pool 3 pg_num_max to 0 2024-08-19T10:39:29.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:29.470+0000 7f69a10b0640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-08-19T10:39:29.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:29.470+0000 7f69a10b0640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-08-19T10:39:29.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:29.470+0000 7f69a10b0640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:29.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:29.470+0000 7f69a10b0640 1 -- 172.21.15.190:0/3446707538 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"} v 0) -- 0x7f699c135330 con 0x7f699c1396a0 2024-08-19T10:39:30.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.370+0000 7f69837fe640 1 -- 172.21.15.190:0/3446707538 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}]=0 set pool 3 pg_num_max to 0 v28) ==== 143+0+0 (secure 0 0 0) 0x7f6990003d90 con 0x7f699c1396a0 2024-08-19T10:39:30.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.370+0000 7f69837fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "testpool1234", "var": "pg_num_max", "val": "0"}] 2024-08-19T10:39:30.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.370+0000 7f69837fe640 10 monclient: _finish_command 3 = system:0 set pool 3 pg_num_max to 0 2024-08-19T10:39:30.371 INFO:tasks.workunit.client.0.smithi190.stderr:set pool 3 pg_num_max to 0 2024-08-19T10:39:30.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 -- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698803be00 msgr2=0x7f698803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:30.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698803be00 0x7f698803e2c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f698c007e80 tx=0x7f698c008040 comp rx=0 tx=0).stop 2024-08-19T10:39:30.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 10 monclient: shutdown 2024-08-19T10:39:30.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:30.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 -- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c1396a0 msgr2=0x7f699c134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:30.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c1396a0 0x7f699c134df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69900ad000 tx=0x7f69900191e0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 -- 172.21.15.190:0/3446707538 shutdown_connections 2024-08-19T10:39:30.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698803be00 0x7f698803e2c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f699c139fc0 0x7f699c09ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f699c1396a0 0x7f699c134df0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 --2- 172.21.15.190:0/3446707538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f699c0a6e20 0x7f699c1348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 -- 172.21.15.190:0/3446707538 >> 172.21.15.190:0/3446707538 conn(0x7f699c0abee0 msgr2=0x7f699c0ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:30.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 -- 172.21.15.190:0/3446707538 shutdown_connections 2024-08-19T10:39:30.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.374+0000 7f69a10b0640 1 -- 172.21.15.190:0/3446707538 wait complete. 2024-08-19T10:39:30.397 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool get testpool1234 pg_num_max 2024-08-19T10:39:30.398 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:30.398 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool get testpool1234 pg_num_max 2024-08-19T10:39:30.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient: get_monmap_and_config 2024-08-19T10:39:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient: build_initial_monmap 2024-08-19T10:39:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient: monmap: 2024-08-19T10:39:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:30.521975+0000 2024-08-19T10:39:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:30.521975+0000 2024-08-19T10:39:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:30.522 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 1 Processor -- start 2024-08-19T10:39:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 1 -- start start 2024-08-19T10:39:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient: init 2024-08-19T10:39:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient: _add_conns ranks=[8,5,3,4,1,7,2,10,0,9,6] 2024-08-19T10:39:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39e0139980 0x7f39e0139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient(hunting): picked mon.noname-e con 0x7f39e0139980 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39e013a2a0 0x7f39e00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient(hunting): picked mon.noname-h con 0x7f39e013a2a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39e00a5c70 0x7f39e00a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient(hunting): picked mon.noname-g con 0x7f39e00a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:30.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:30.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f39e009ca60 con 0x7f39e00a5c70 2024-08-19T10:39:30.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:30.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f39e00a3640 con 0x7f39e013a2a0 2024-08-19T10:39:30.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:30.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f39e00a8470 con 0x7f39e0139980 2024-08-19T10:39:30.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:30.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.518+0000 7f39e523b640 10 monclient(hunting): authenticate will time out at 1016.091959s 2024-08-19T10:39:30.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39e013a2a0 0x7f39e00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:30.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39e013a2a0 0x7f39e00a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:58124/0 (socket says 172.21.15.190:58124) 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 1 -- 172.21.15.190:0/247344190 learned_addr learned my addr 172.21.15.190:0/247344190 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 10 monclient(hunting): get_auth_request con 0x7f39e013a2a0 auth_method 0 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39df59e640 1 --2- 172.21.15.190:0/247344190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39e00a5c70 0x7f39e00a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39df59e640 10 monclient(hunting): get_auth_request con 0x7f39e00a5c70 auth_method 0 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39df59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39df59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39df59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39df59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39df59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39df59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39ded9d640 1 --2- 172.21.15.190:0/247344190 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39e0139980 0x7f39e0139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39ded9d640 10 monclient(hunting): get_auth_request con 0x7f39e0139980 auth_method 0 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39ded9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39ded9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39ded9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 10 monclient(hunting): handle_auth_done global_id 4550 payload 274 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 1 -- 172.21.15.190:0/247344190 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39e0139980 msgr2=0x7f39e0139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 1 --2- 172.21.15.190:0/247344190 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39e0139980 0x7f39e0139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 1 -- 172.21.15.190:0/247344190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39e00a5c70 msgr2=0x7f39e00a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 1 --2- 172.21.15.190:0/247344190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39e00a5c70 0x7f39e00a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 10 monclient: _finish_hunting 0 2024-08-19T10:39:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 1 monclient: found mon.noname-h 2024-08-19T10:39:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39de59c640 1 -- 172.21.15.190:0/247344190 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e00a3850 con 0x7f39e013a2a0 2024-08-19T10:39:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.522+0000 7f39df59e640 1 --2- 172.21.15.190:0/247344190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39e00a5c70 0x7f39e00a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39de59c640 1 -- 172.21.15.190:0/247344190 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39e013a2a0 msgr2=0x7f39e00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39de59c640 1 --2- 172.21.15.190:0/247344190 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39e013a2a0 0x7f39e00a5730 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39c80098d0 tx=0x7f39c801c150 comp rx=0 tx=0).ready entity=mon.5 client_cookie=ea5aa6cd2192e0c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 1 -- 172.21.15.190:0/247344190 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39c80b0070 con 0x7f39e013a2a0 2024-08-19T10:39:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 10 monclient: dump: 2024-08-19T10:39:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:30.535 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:30.535 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:30.535 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:30.535 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:30.535 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:30.535 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:30.535 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:30.535 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:30.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 10 monclient: _finish_auth 0 2024-08-19T10:39:30.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 10 monclient: _check_auth_tickets 2024-08-19T10:39:30.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:30.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 1 -- 172.21.15.190:0/247344190 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c80aa810 con 0x7f39e013a2a0 2024-08-19T10:39:30.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 5 monclient: authenticate success, global_id 4550 2024-08-19T10:39:30.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:30.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:30.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:30.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 1 -- 172.21.15.190:0/247344190 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39c80aae30 con 0x7f39e013a2a0 2024-08-19T10:39:30.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:30.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 4 set_mon_vals no callback set 2024-08-19T10:39:30.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:30.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ddd9b640 10 monclient: dump: 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:30.537 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:30.538 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:30.538 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:30.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient: shutdown 2024-08-19T10:39:30.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:30.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 -- 172.21.15.190:0/247344190 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39e013a2a0 msgr2=0x7f39e00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:30.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 --2- 172.21.15.190:0/247344190 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39e013a2a0 0x7f39e00a5730 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39c80098d0 tx=0x7f39c801c150 comp rx=0 tx=0).stop 2024-08-19T10:39:30.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 -- 172.21.15.190:0/247344190 shutdown_connections 2024-08-19T10:39:30.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 --2- 172.21.15.190:0/247344190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39e00a5c70 0x7f39e00a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 --2- 172.21.15.190:0/247344190 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39e013a2a0 0x7f39e00a5730 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 --2- 172.21.15.190:0/247344190 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39e0139980 0x7f39e0139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 -- 172.21.15.190:0/247344190 >> 172.21.15.190:0/247344190 conn(0x7f39e00abde0 msgr2=0x7f39e00ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 -- 172.21.15.190:0/247344190 shutdown_connections 2024-08-19T10:39:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 -- 172.21.15.190:0/247344190 wait complete. 2024-08-19T10:39:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient: build_initial_monmap 2024-08-19T10:39:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient: monmap: 2024-08-19T10:39:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:30.540 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 Processor -- start 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 -- start start 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient: init 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient: _add_conns ranks=[3,1,2,6,7,5,0,8,4] 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39e00a5c70 0x7f39e00a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient(hunting): picked mon.noname-g con 0x7f39e00a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39e0139980 0x7f39e013f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient(hunting): picked mon.noname-f con 0x7f39e0139980 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39e013bb20 0x7f39e013fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient(hunting): picked mon.noname-b con 0x7f39e013bb20 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0143270 con 0x7f39e0139980 2024-08-19T10:39:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f39e01430f0 con 0x7f39e013bb20 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f39e01433f0 con 0x7f39e00a5c70 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39de59c640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39e0139980 msgr2=0x7f39e013f860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39de59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39e0139980 0x7f39e013f860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39e523b640 10 monclient(hunting): authenticate will time out at 1016.099421s 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39e00a5c70 0x7f39e00a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39e00a5c70 0x7f39e00a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46428/0 (socket says 172.21.15.190:46428) 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 1 -- 172.21.15.190:0/3435958610 learned_addr learned my addr 172.21.15.190:0/3435958610 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 10 monclient(hunting): get_auth_request con 0x7f39e00a5c70 auth_method 0 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 10 monclient(hunting): handle_auth_done global_id 4521 payload 824 2024-08-19T10:39:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 1 -- 172.21.15.190:0/3435958610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39e013bb20 msgr2=0x7f39e013fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 1 --2- 172.21.15.190:0/3435958610 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39e013bb20 0x7f39e013fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 1 -- 172.21.15.190:0/3435958610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39e0139980 msgr2=0x7f39e013f860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 1 --2- 172.21.15.190:0/3435958610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39e0139980 0x7f39e013f860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 10 monclient: _finish_hunting 0 2024-08-19T10:39:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 1 monclient: found mon.noname-g 2024-08-19T10:39:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.526+0000 7f39ded9d640 1 -- 172.21.15.190:0/3435958610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e00a1c20 con 0x7f39e00a5c70 2024-08-19T10:39:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.530+0000 7f39ded9d640 1 -- 172.21.15.190:0/3435958610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39e00a5c70 msgr2=0x7f39e00a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.530+0000 7f39ded9d640 1 --2- 172.21.15.190:0/3435958610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39e00a5c70 0x7f39e00a11c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39d000d800 tx=0x7f39d000dcc0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.530+0000 7f39d77fe640 1 -- 172.21.15.190:0/3435958610 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:39:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 1 -- 172.21.15.190:0/3435958610 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39d000bcc0 con 0x7f39e00a5c70 2024-08-19T10:39:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 10 monclient: dump: 2024-08-19T10:39:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39e523b640 5 monclient: authenticate success, global_id 4521 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39e523b640 10 monclient: _renew_subs 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 1 -- 172.21.15.190:0/3435958610 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d0019070 con 0x7f39e00a5c70 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39e523b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39e523b640 1 -- 172.21.15.190:0/3435958610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e00a9050 con 0x7f39e00a5c70 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 1 -- 172.21.15.190:0/3435958610 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39d00148a0 con 0x7f39e00a5c70 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 10 monclient: dump: 2024-08-19T10:39:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:30.550 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39dd59a640 4 set_mon_vals no callback set 2024-08-19T10:39:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39e523b640 10 monclient: _renew_subs 2024-08-19T10:39:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39e523b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39e523b640 1 -- 172.21.15.190:0/3435958610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e013ca80 con 0x7f39e00a5c70 2024-08-19T10:39:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39e523b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39e523b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39e523b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39e523b640 1 -- 172.21.15.190:0/3435958610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e0139d60 con 0x7f39e00a5c70 2024-08-19T10:39:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 1 -- 172.21.15.190:0/3435958610 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f39d0014430 con 0x7f39e00a5c70 2024-08-19T10:39:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 1 --2- 172.21.15.190:0/3435958610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f39c003be00 0x7f39c003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39d77fe640 1 -- 172.21.15.190:0/3435958610 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 4697+0+0 (secure 0 0 0) 0x7f39d000f040 con 0x7f39e00a5c70 2024-08-19T10:39:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39de59c640 1 --2- 172.21.15.190:0/3435958610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f39c003be00 0x7f39c003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.534+0000 7f39de59c640 10 monclient: get_auth_request con 0x7f39c003be00 auth_method 0 2024-08-19T10:39:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.538+0000 7f39de59c640 1 --2- 172.21.15.190:0/3435958610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f39c003be00 0x7f39c003e2c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f39c80098d0 tx=0x7f39c80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.538+0000 7f39d77fe640 1 -- 172.21.15.190:0/3435958610 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f39d0053070 con 0x7f39e00a5c70 2024-08-19T10:39:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.538+0000 7f39d77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.538+0000 7f39d77fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:30.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.706+0000 7f39e523b640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-08-19T10:39:30.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.706+0000 7f39e523b640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-08-19T10:39:30.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.706+0000 7f39e523b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:30.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.706+0000 7f39e523b640 1 -- 172.21.15.190:0/3435958610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"} v 0) -- 0x7f39e013cc80 con 0x7f39e00a5c70 2024-08-19T10:39:30.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.886+0000 7f39d77fe640 1 -- 172.21.15.190:0/3435958610 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}]=-2 option 'pg_num_max' is not set on pool 'testpool1234' v28) ==== 158+0+0 (secure 0 0 0) 0x7f39e00a9050 con 0x7f39e00a5c70 2024-08-19T10:39:30.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.886+0000 7f39d77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "testpool1234", "var": "pg_num_max"}] 2024-08-19T10:39:30.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:30.886+0000 7f39d77fe640 10 monclient: _finish_command 2 = mon:2 option 'pg_num_max' is not set on pool 'testpool1234' 2024-08-19T10:39:30.886 INFO:tasks.workunit.client.0.smithi190.stderr:Error ENOENT: option 'pg_num_max' is not set on pool 'testpool1234' 2024-08-19T10:39:30.891 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:30.891 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete testpool1234 testpool1234 --yes-i-really-really-mean-it 2024-08-19T10:39:31.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient: get_monmap_and_config 2024-08-19T10:39:31.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient: build_initial_monmap 2024-08-19T10:39:31.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient: monmap: 2024-08-19T10:39:31.015 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:31.015 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:31.015 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:31.016127+0000 2024-08-19T10:39:31.015 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:31.016127+0000 2024-08-19T10:39:31.015 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:31.016 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 1 Processor -- start 2024-08-19T10:39:31.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 1 -- start start 2024-08-19T10:39:31.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient: init 2024-08-19T10:39:31.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:31.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient: _add_conns ranks=[0,3,2,9,5,6,8,7,4,10,1] 2024-08-19T10:39:31.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd9901396a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:31.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd99013a190 0x7fd99013eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient(hunting): picked mon.noname-g con 0x7fd99013a190 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:31.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd99013f0a0 0x7fd990141490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient(hunting): picked mon.noname-b con 0x7fd99013f0a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:31.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9900a83e0 con 0x7fd9901396a0 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd9900a4f20 con 0x7fd99013f0a0 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd9900a2f30 con 0x7fd99013a190 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd9959b5640 10 monclient(hunting): authenticate will time out at 1016.586284s 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd99013a190 0x7fd99013eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd99013a190 0x7fd99013eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46430/0 (socket says 172.21.15.190:46430) 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 1 -- 172.21.15.190:0/3499108338 learned_addr learned my addr 172.21.15.190:0/3499108338 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 10 monclient(hunting): get_auth_request con 0x7fd99013a190 auth_method 0 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:31.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98f7fe640 1 --2- 172.21.15.190:0/3499108338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd99013f0a0 0x7fd990141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 10 monclient(hunting): handle_auth_done global_id 4530 payload 274 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 1 -- 172.21.15.190:0/3499108338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd99013f0a0 msgr2=0x7fd990141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 1 --2- 172.21.15.190:0/3499108338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd99013f0a0 0x7fd990141490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 1 -- 172.21.15.190:0/3499108338 >> v1:172.21.15.89:6789/0 conn(0x7fd9901396a0 legacy=0x7fd990139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 1 monclient: found mon.noname-g 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:31.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:31.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98e7fc640 1 -- 172.21.15.190:0/3499108338 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9900a35e0 con 0x7fd99013a190 2024-08-19T10:39:31.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98f7fe640 10 monclient: get_auth_request con 0x7fd99013f0a0 auth_method 0 2024-08-19T10:39:31.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.014+0000 7fd98f7fe640 1 --2- 172.21.15.190:0/3499108338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd99013f0a0 0x7fd990141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:39:31.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98e7fc640 1 -- 172.21.15.190:0/3499108338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd99013a190 msgr2=0x7fd99013eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:31.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98e7fc640 1 --2- 172.21.15.190:0/3499108338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd99013a190 0x7fd99013eb60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9840098a0 tx=0x7fd98401c440 comp rx=0 tx=0).ready entity=mon.3 client_cookie=e64d7e443a070a06 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:31.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 1 -- 172.21.15.190:0/3499108338 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd9840b0070 con 0x7fd99013a190 2024-08-19T10:39:31.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:31.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:31.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 10 monclient: dump: 2024-08-19T10:39:31.026 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:31.026 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:31.026 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.026 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.026 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:31.026 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 10 monclient: _finish_auth 0 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:39:31.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 1 -- 172.21.15.190:0/3499108338 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9840aa810 con 0x7fd99013a190 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd9959b5640 5 monclient: authenticate success, global_id 4530 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd9959b5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 1 -- 172.21.15.190:0/3499108338 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd9840aae30 con 0x7fd99013a190 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd98dffb640 10 monclient: dump: 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.028 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd9959b5640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:31.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd9959b5640 4 set_mon_vals no callback set 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd9959b5640 10 monclient: shutdown 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd9959b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd9959b5640 1 -- 172.21.15.190:0/3499108338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd99013a190 msgr2=0x7fd99013eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.018+0000 7fd9959b5640 1 --2- 172.21.15.190:0/3499108338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd99013a190 0x7fd99013eb60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9840098a0 tx=0x7fd98401c440 comp rx=0 tx=0).stop 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 -- 172.21.15.190:0/3499108338 shutdown_connections 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 --2- 172.21.15.190:0/3499108338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd99013f0a0 0x7fd990141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 --2- 172.21.15.190:0/3499108338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd99013a190 0x7fd99013eb60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 -- 172.21.15.190:0/3499108338 >> 172.21.15.190:0/3499108338 conn(0x7fd9900abee0 msgr2=0x7fd9900ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 -- 172.21.15.190:0/3499108338 shutdown_connections 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 -- 172.21.15.190:0/3499108338 wait complete. 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient: build_initial_monmap 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient: monmap: 2024-08-19T10:39:31.030 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:31.031 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:31.031 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:31.031 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:31.031 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:31.031 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.031 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:31.031 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:31.031 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:31.032 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:31.032 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:31.032 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:31.032 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:31.032 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:31.032 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:31.032 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 Processor -- start 2024-08-19T10:39:31.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 -- start start 2024-08-19T10:39:31.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient: init 2024-08-19T10:39:31.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:31.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient: _add_conns ranks=[2,0,5,1,6,4,7,3,8] 2024-08-19T10:39:31.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9901396a0 0x7fd9901d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient(hunting): picked mon.noname-b con 0x7fd9901396a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:31.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd99013a190 0x7fd9901d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient(hunting): picked mon.noname-a con 0x7fd99013a190 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:31.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd99013f0a0 0x7fd99013c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient(hunting): picked mon.noname-h con 0x7fd99013f0a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:31.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd9901441f0 con 0x7fd99013a190 2024-08-19T10:39:31.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd9900a78f0 con 0x7fd9901396a0 2024-08-19T10:39:31.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd990144370 con 0x7fd99013f0a0 2024-08-19T10:39:31.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:31.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd9959b5640 10 monclient(hunting): authenticate will time out at 1016.593622s 2024-08-19T10:39:31.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd99013f0a0 0x7fd99013c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd99013f0a0 0x7fd99013c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:58138/0 (socket says 172.21.15.190:58138) 2024-08-19T10:39:31.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 1 -- 172.21.15.190:0/1756982394 learned_addr learned my addr 172.21.15.190:0/1756982394 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:31.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 10 monclient(hunting): get_auth_request con 0x7fd99013f0a0 auth_method 0 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98e7fc640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd99013a190 0x7fd9901d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98effd640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9901396a0 0x7fd9901d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98e7fc640 10 monclient(hunting): get_auth_request con 0x7fd99013a190 auth_method 0 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:31.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98effd640 10 monclient(hunting): get_auth_request con 0x7fd9901396a0 auth_method 0 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 10 monclient(hunting): handle_auth_done global_id 4559 payload 824 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 1 -- 172.21.15.190:0/1756982394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9901396a0 msgr2=0x7fd9901d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9901396a0 0x7fd9901d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 1 -- 172.21.15.190:0/1756982394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd99013a190 msgr2=0x7fd9901d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd99013a190 0x7fd9901d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 1 monclient: found mon.noname-h 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:31.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:31.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98f7fe640 1 -- 172.21.15.190:0/1756982394 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99009cde0 con 0x7fd99013f0a0 2024-08-19T10:39:31.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98e7fc640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd99013a190 0x7fd9901d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:39:31.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.022+0000 7fd98effd640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9901396a0 0x7fd9901d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:39:31.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd98f7fe640 1 -- 172.21.15.190:0/1756982394 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd99013f0a0 msgr2=0x7fd99013c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:31.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd98f7fe640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd99013f0a0 0x7fd99013c6a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd98800cd50 tx=0x7fd98800a650 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:31.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 1 -- 172.21.15.190:0/1756982394 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd98800fb00 con 0x7fd99013f0a0 2024-08-19T10:39:31.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:31.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:31.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 10 monclient: dump: 2024-08-19T10:39:31.039 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:31.039 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:31.039 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.040 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.040 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:31.040 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.040 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:31.040 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:31.040 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:31.041 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:31.041 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:31.041 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:31.041 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:31.041 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:31.041 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:31.041 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 10 monclient: _finish_auth 0 2024-08-19T10:39:31.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:31.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:31.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd9959b5640 5 monclient: authenticate success, global_id 4559 2024-08-19T10:39:31.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 1 -- 172.21.15.190:0/1756982394 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98800fce0 con 0x7fd99013f0a0 2024-08-19T10:39:31.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd9959b5640 10 monclient: _renew_subs 2024-08-19T10:39:31.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd9959b5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:31.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd9959b5640 1 -- 172.21.15.190:0/1756982394 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99009e030 con 0x7fd99013f0a0 2024-08-19T10:39:31.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:31.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 1 -- 172.21.15.190:0/1756982394 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd988018870 con 0x7fd99013f0a0 2024-08-19T10:39:31.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:31.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:31.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd98cff9640 4 set_mon_vals no callback set 2024-08-19T10:39:31.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 10 monclient: dump: 2024-08-19T10:39:31.043 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:31.043 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:31.043 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.043 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.043 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:31.043 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.043 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:31.043 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:31.043 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:31.043 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:31.044 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:31.044 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:31.044 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:31.044 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:31.044 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:31.044 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd9959b5640 10 monclient: _renew_subs 2024-08-19T10:39:31.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd9959b5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:31.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd9959b5640 1 -- 172.21.15.190:0/1756982394 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99009e310 con 0x7fd99013f0a0 2024-08-19T10:39:31.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd9959b5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:31.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd9959b5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:31.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd9959b5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:31.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd9959b5640 1 -- 172.21.15.190:0/1756982394 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd990139a80 con 0x7fd99013f0a0 2024-08-19T10:39:31.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 1 -- 172.21.15.190:0/1756982394 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd988018400 con 0x7fd99013f0a0 2024-08-19T10:39:31.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd97c03be30 0x7fd97c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd977fff640 1 -- 172.21.15.190:0/1756982394 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(28..28 src has 1..28) ==== 4697+0+0 (secure 0 0 0) 0x7fd98804f730 con 0x7fd99013f0a0 2024-08-19T10:39:31.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd98effd640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd97c03be30 0x7fd97c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.026+0000 7fd98effd640 10 monclient: get_auth_request con 0x7fd97c03be30 auth_method 0 2024-08-19T10:39:31.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.030+0000 7fd98effd640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd97c03be30 0x7fd97c03e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd980003ec0 tx=0x7fd980009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:31.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.030+0000 7fd977fff640 1 -- 172.21.15.190:0/1756982394 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd988014030 con 0x7fd99013f0a0 2024-08-19T10:39:31.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.030+0000 7fd977fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:31.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.030+0000 7fd977fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:31.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.198+0000 7fd9959b5640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:31.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.198+0000 7fd9959b5640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:31.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.198+0000 7fd9959b5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:31.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.198+0000 7fd9959b5640 1 -- 172.21.15.190:0/1756982394 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd9900a1f70 con 0x7fd99013f0a0 2024-08-19T10:39:31.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.474+0000 7fd977fff640 1 -- 172.21.15.190:0/1756982394 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}]=0 pool 'testpool1234' removed v29) ==== 176+0+0 (secure 0 0 0) 0x7fd98800ee20 con 0x7fd99013f0a0 2024-08-19T10:39:31.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.474+0000 7fd977fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:31.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.474+0000 7fd977fff640 10 monclient: _finish_command 2 = system:0 pool 'testpool1234' removed 2024-08-19T10:39:31.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.582+0000 7fd9959b5640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:31.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.582+0000 7fd9959b5640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:31.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.582+0000 7fd9959b5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:31.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.582+0000 7fd9959b5640 1 -- 172.21.15.190:0/1756982394 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd99013eae0 con 0x7fd99013f0a0 2024-08-19T10:39:31.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.582+0000 7fd977fff640 1 -- 172.21.15.190:0/1756982394 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}]=0 pool 'testpool1234' does not exist v29) ==== 183+0+0 (secure 0 0 0) 0x7fd988016300 con 0x7fd99013f0a0 2024-08-19T10:39:31.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.582+0000 7fd977fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "testpool1234", "pool2": "testpool1234", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:31.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.582+0000 7fd977fff640 10 monclient: _finish_command 3 = system:0 pool 'testpool1234' does not exist 2024-08-19T10:39:31.586 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'testpool1234' does not exist 2024-08-19T10:39:31.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 -- 172.21.15.190:0/1756982394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd97c03be30 msgr2=0x7fd97c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:31.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd97c03be30 0x7fd97c03e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd980003ec0 tx=0x7fd980009040 comp rx=0 tx=0).stop 2024-08-19T10:39:31.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 10 monclient: shutdown 2024-08-19T10:39:31.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:31.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 -- 172.21.15.190:0/1756982394 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd99013f0a0 msgr2=0x7fd99013c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:31.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd99013f0a0 0x7fd99013c6a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd98800cd50 tx=0x7fd98800a650 comp rx=0 tx=0).stop 2024-08-19T10:39:31.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 -- 172.21.15.190:0/1756982394 shutdown_connections 2024-08-19T10:39:31.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd97c03be30 0x7fd97c03e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd99013f0a0 0x7fd99013c6a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd99013a190 0x7fd9901d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 --2- 172.21.15.190:0/1756982394 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9901396a0 0x7fd9901d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 -- 172.21.15.190:0/1756982394 >> 172.21.15.190:0/1756982394 conn(0x7fd9900abee0 msgr2=0x7fd9900aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:31.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 -- 172.21.15.190:0/1756982394 shutdown_connections 2024-08-19T10:39:31.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.586+0000 7fd9959b5640 1 -- 172.21.15.190:0/1756982394 wait complete. 2024-08-19T10:39:31.604 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool create foo 123 123 invalid foo-profile foo-rule 2024-08-19T10:39:31.604 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:31.604 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create foo 123 123 invalid foo-profile foo-rule 2024-08-19T10:39:31.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient: get_monmap_and_config 2024-08-19T10:39:31.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient: build_initial_monmap 2024-08-19T10:39:31.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient: monmap: 2024-08-19T10:39:31.731 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:31.731 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:31.731 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:31.732129+0000 2024-08-19T10:39:31.731 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:31.732129+0000 2024-08-19T10:39:31.731 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:31.731 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.731 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:31.731 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:31.731 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:31.732 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:31.732 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:31.732 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:31.732 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:31.732 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:31.732 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:31.732 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:31.732 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:31.732 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 1 Processor -- start 2024-08-19T10:39:31.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 1 -- start start 2024-08-19T10:39:31.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient: init 2024-08-19T10:39:31.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:31.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient: _add_conns ranks=[0,6,4,10,5,3,9,7,1,2,8] 2024-08-19T10:39:31.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc0781399f0 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:31.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0780a5640 0x7fc07813e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient(hunting): picked mon.noname-d con 0x7fc0780a5640 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc07813ee20 0x7fc078141210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient(hunting): picked mon.noname-c con 0x7fc07813ee20 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc07813a4e0 con 0x7fc0781399f0 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc0780a8340 con 0x7fc07813ee20 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fc0780a4580 con 0x7fc0780a5640 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07cdb0640 10 monclient(hunting): authenticate will time out at 1017.302127s 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0780a5640 0x7fc07813e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07659c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc0781399f0 0x7fc078139dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:33586/0 (socket says 172.21.15.190:33586) 2024-08-19T10:39:31.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc076d9d640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc07813ee20 0x7fc078141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc07659c640 1 -- 172.21.15.190:0/2673232245 learned_addr learned my addr 172.21.15.190:0/2673232245 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 10 monclient(hunting): get_auth_request con 0x7fc0780a5640 auth_method 0 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc076d9d640 10 monclient(hunting): get_auth_request con 0x7fc07813ee20 auth_method 0 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc076d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc076d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc076d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:31.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc076d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc076d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc076d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 10 monclient(hunting): handle_auth_done global_id 4452 payload 274 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 1 -- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc07813ee20 msgr2=0x7fc078141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc07813ee20 0x7fc078141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 1 -- 172.21.15.190:0/2673232245 >> v1:172.21.15.89:6789/0 conn(0x7fc0781399f0 legacy=0x7fc078139dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 1 monclient: found mon.noname-d 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc075d9b640 1 -- 172.21.15.190:0/2673232245 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0780a4860 con 0x7fc0780a5640 2024-08-19T10:39:31.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.730+0000 7fc076d9d640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc07813ee20 0x7fc078141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:31.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.734+0000 7fc075d9b640 1 -- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0780a5640 msgr2=0x7fc07813e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:31.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.734+0000 7fc075d9b640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0780a5640 0x7fc07813e8e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc0600097f0 tx=0x7fc06001c3a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f63dda4154946a27 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:31.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc075d9b640 1 -- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0780a5640 msgr2=0x7fc07813e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-08-19T10:39:31.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc075d9b640 1 -- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0780a5640 msgr2=0x7fc07813e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:39:31.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc075d9b640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0780a5640 0x7fc07813e8e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc0600097f0 tx=0x7fc06001c3a0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:39:31.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc075d9b640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0780a5640 0x7fc07813e8e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc0600097f0 tx=0x7fc06001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 10 monclient: ms_handle_reset current mon [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:31.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 1 -- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0780a5640 msgr2=0x7fc07813e8e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0780a5640 0x7fc07813e8e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 10 monclient: _add_conns ranks=[9,7,2,3,5,8,1,10] 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 1 --2- 172.21.15.190:0/2673232245 >> v2:172.21.15.89:3300/0 conn(0x7fc064003640 0x7fc064005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 10 monclient(hunting): picked mon.noname-a con 0x7fc064003640 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0640062b0 0x7fc064008740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 10 monclient(hunting): picked mon.noname-i con 0x7fc0640062b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc064009080 0x7fc06400b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 10 monclient(hunting): picked mon.noname-b con 0x7fc064009080 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 1 -- 172.21.15.190:0/2673232245 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fc06400bec0 con 0x7fc064003640 2024-08-19T10:39:31.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 1 -- 172.21.15.190:0/2673232245 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc06400c040 con 0x7fc064009080 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 1 -- 172.21.15.190:0/2673232245 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc06400c240 con 0x7fc0640062b0 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 10 monclient(hunting): ms_handle_reset stray mon [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc075d9b640 1 -- 172.21.15.190:0/2673232245 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0640062b0 msgr2=0x7fc064008740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc075d9b640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0640062b0 0x7fc064008740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07659c640 1 --2- 172.21.15.190:0/2673232245 >> v2:172.21.15.89:3300/0 conn(0x7fc064003640 0x7fc064005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc064009080 0x7fc06400b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07659c640 10 monclient(hunting): get_auth_request con 0x7fc064003640 auth_method 0 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:31.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:31.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:39:31.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 10 monclient(hunting): get_auth_request con 0x7fc064009080 auth_method 0 2024-08-19T10:39:31.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:31.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:31.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:39:31.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07659c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 10 monclient(hunting): handle_auth_done global_id 4452 payload 293 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 1 -- 172.21.15.190:0/2673232245 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0640062b0 msgr2=0x7fc064008740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0640062b0 0x7fc064008740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 1 -- 172.21.15.190:0/2673232245 >> v2:172.21.15.89:3300/0 conn(0x7fc064003640 msgr2=0x7fc064005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 1 --2- 172.21.15.190:0/2673232245 >> v2:172.21.15.89:3300/0 conn(0x7fc064003640 0x7fc064005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 1 monclient: found mon.noname-b 2024-08-19T10:39:31.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:31.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:31.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 1 -- 172.21.15.190:0/2673232245 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06400c500 con 0x7fc064009080 2024-08-19T10:39:31.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 1 -- 172.21.15.190:0/2673232245 reap_dead start 2024-08-19T10:39:31.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 1 -- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc064009080 msgr2=0x7fc06400b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:31.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc076d9d640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc064009080 0x7fc06400b540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc07000e320 tx=0x7fc060009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eb135310d742322 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:31.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.738+0000 7fc07559a640 1 -- 172.21.15.190:0/2673232245 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 1 -- 172.21.15.190:0/2673232245 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc070018020 con 0x7fc064009080 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 10 monclient: dump: 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:31.749 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 10 monclient: _finish_auth 0 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 10 monclient: _check_auth_tickets 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 1 -- 172.21.15.190:0/2673232245 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc070014b00 con 0x7fc064009080 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:31.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 1 -- 172.21.15.190:0/2673232245 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc070009de0 con 0x7fc064009080 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07559a640 10 monclient: dump: 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:31.751 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 5 monclient: authenticate success, global_id 4452 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 4 set_mon_vals no callback set 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 10 monclient: shutdown 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 1 -- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc064009080 msgr2=0x7fc06400b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc064009080 0x7fc06400b540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc07000e320 tx=0x7fc060009190 comp rx=0 tx=0).stop 2024-08-19T10:39:31.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 1 -- 172.21.15.190:0/2673232245 shutdown_connections 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 1 --2- 172.21.15.190:0/2673232245 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc064009080 0x7fc06400b540 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 1 -- 172.21.15.190:0/2673232245 >> 172.21.15.190:0/2673232245 conn(0x7fc078074c10 msgr2=0x7fc0780abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 1 -- 172.21.15.190:0/2673232245 shutdown_connections 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 1 -- 172.21.15.190:0/2673232245 wait complete. 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 10 monclient: build_initial_monmap 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 10 monclient: monmap: 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.753 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 1 Processor -- start 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 1 -- start start 2024-08-19T10:39:31.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.742+0000 7fc07cdb0640 10 monclient: init 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 10 monclient: _add_conns ranks=[0,4,6,5,1,3,2,7,8] 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc064009080 0x7fc0781d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 10 monclient(hunting): picked mon.noname-a con 0x7fc064009080 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc078138980 0x7fc07809fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 10 monclient(hunting): picked mon.noname-c con 0x7fc078138980 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc078138de0 0x7fc0780a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 10 monclient(hunting): picked mon.noname-d con 0x7fc078138de0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc078139dd0 con 0x7fc064009080 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc0780a5a20 con 0x7fc078138980 2024-08-19T10:39:31.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fc078139f50 con 0x7fc078138de0 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07cdb0640 10 monclient(hunting): authenticate will time out at 1017.316255s 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc064009080 0x7fc0781d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc076d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc078138de0 0x7fc0780a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc078138980 0x7fc07809fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc064009080 0x7fc0781d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:36554/0 (socket says 172.21.15.190:36554) 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07659c640 1 -- 172.21.15.190:0/444319722 learned_addr learned my addr 172.21.15.190:0/444319722 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07659c640 10 monclient(hunting): get_auth_request con 0x7fc064009080 auth_method 0 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:31.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc076d9d640 10 monclient(hunting): get_auth_request con 0x7fc078138de0 auth_method 0 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc076d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc076d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc076d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 10 monclient(hunting): get_auth_request con 0x7fc078138980 auth_method 0 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc076d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc076d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc076d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc07659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:31.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 10 monclient(hunting): handle_auth_done global_id 4639 payload 824 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 1 -- 172.21.15.190:0/444319722 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc078138de0 msgr2=0x7fc0780a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 1 --2- 172.21.15.190:0/444319722 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc078138de0 0x7fc0780a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 1 -- 172.21.15.190:0/444319722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc064009080 msgr2=0x7fc0781d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 1 --2- 172.21.15.190:0/444319722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc064009080 0x7fc0781d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 1 monclient: found mon.noname-c 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:31.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 1 -- 172.21.15.190:0/444319722 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0780a1750 con 0x7fc078138980 2024-08-19T10:39:31.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 1 -- 172.21.15.190:0/444319722 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc078138980 msgr2=0x7fc07809fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.746+0000 7fc075d9b640 1 --2- 172.21.15.190:0/444319722 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc078138980 0x7fc07809fce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0780a4750 tx=0x7fc060019140 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 1 -- 172.21.15.190:0/444319722 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0600b0070 con 0x7fc078138980 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 10 monclient: dump: 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.761 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 10 monclient: _finish_auth 0 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 10 monclient: _check_auth_tickets 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:31.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07cdb0640 5 monclient: authenticate success, global_id 4639 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07cdb0640 10 monclient: _renew_subs 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07cdb0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07cdb0640 1 -- 172.21.15.190:0/444319722 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc078139470 con 0x7fc078138980 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 1 -- 172.21.15.190:0/444319722 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0600bb070 con 0x7fc078138980 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07cdb0640 10 monclient: _renew_subs 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc074d99640 4 set_mon_vals no callback set 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07cdb0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07cdb0640 1 -- 172.21.15.190:0/444319722 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc078122530 con 0x7fc078138980 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 1 -- 172.21.15.190:0/444319722 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc060003df0 con 0x7fc078138980 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:31.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 10 monclient: dump: 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:31.764 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07cdb0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07cdb0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07cdb0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07cdb0640 1 -- 172.21.15.190:0/444319722 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0780a68e0 con 0x7fc078138980 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 1 -- 172.21.15.190:0/444319722 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc0600c0460 con 0x7fc078138980 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 1 --2- 172.21.15.190:0/444319722 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc05803be00 0x7fc05803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc06effd640 1 -- 172.21.15.190:0/444319722 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(29..29 src has 1..29) ==== 4346+0+0 (secure 0 0 0) 0x7fc0600e9a60 con 0x7fc078138980 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07659c640 1 --2- 172.21.15.190:0/444319722 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc05803be00 0x7fc05803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.750+0000 7fc07659c640 10 monclient: get_auth_request con 0x7fc05803be00 auth_method 0 2024-08-19T10:39:31.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.754+0000 7fc07659c640 1 --2- 172.21.15.190:0/444319722 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc05803be00 0x7fc05803e2c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc068007aa0 tx=0x7fc068007dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:31.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.762+0000 7fc06effd640 1 -- 172.21.15.190:0/444319722 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc0600160b0 con 0x7fc078138980 2024-08-19T10:39:31.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.762+0000 7fc06effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:31.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:31.762+0000 7fc06effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:31.929 INFO:tasks.workunit.client.0.smithi190.stderr:foo-rule not valid: foo-rule not one of 'true', 'false' 2024-08-19T10:39:31.929 INFO:tasks.workunit.client.0.smithi190.stderr:Invalid command: unused arguments: ['foo-rule'] 2024-08-19T10:39:31.929 INFO:tasks.workunit.client.0.smithi190.stderr:osd pool create [] [] [] [] [] [] [] [] [] [] [--bulk] [] [] [--yes-i-really-mean-it] : create pool 2024-08-19T10:39:31.929 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: invalid command 2024-08-19T10:39:31.931 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:31.931 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create foo 123 123 replicated 2024-08-19T10:39:32.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.038+0000 7f0b6fd70640 10 monclient: get_monmap_and_config 2024-08-19T10:39:32.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.038+0000 7f0b6fd70640 10 monclient: build_initial_monmap 2024-08-19T10:39:32.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.038+0000 7f0b6fd70640 10 monclient: monmap: 2024-08-19T10:39:32.041 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:32.041 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:32.041 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:32.042258+0000 2024-08-19T10:39:32.041 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:32.042258+0000 2024-08-19T10:39:32.041 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:32.041 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.041 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:32.042 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:32.042 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:32.042 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:32.042 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:32.042 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:32.042 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:32.042 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:32.042 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:32.042 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:32.042 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:32.043 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.038+0000 7f0b6fd70640 1 Processor -- start 2024-08-19T10:39:32.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.038+0000 7f0b6fd70640 1 -- start start 2024-08-19T10:39:32.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.038+0000 7f0b6fd70640 10 monclient: init 2024-08-19T10:39:32.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:32.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 10 monclient: _add_conns ranks=[5,7,10,0,4,3,1,2,9,6,8] 2024-08-19T10:39:32.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b680a68e0 0x7f0b680a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 10 monclient(hunting): picked mon.noname-h con 0x7f0b680a68e0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:32.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b680a6cc0 0x7f0b680a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 10 monclient(hunting): picked mon.noname-i con 0x7f0b680a6cc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0b681396a0 0x7f0b68139af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 10 monclient(hunting): picked mon.noname-f con 0x7f0b681396a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0b680a86c0 con 0x7f0b681396a0 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0b680a8410 con 0x7f0b680a68e0 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0b680a63a0 con 0x7f0b680a6cc0 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6fd70640 10 monclient(hunting): authenticate will time out at 1017.612120s 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6d30b640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b680a6cc0 msgr2=0x7f0b680a70a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b680a68e0 0x7f0b680a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:32.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6e30d640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f0b681396a0 msgr2=0x7f0b68139af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6d30b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b680a6cc0 0x7f0b680a70a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b680a68e0 0x7f0b680a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:58148/0 (socket says 172.21.15.190:58148) 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 1 -- 172.21.15.190:0/4072431019 learned_addr learned my addr 172.21.15.190:0/4072431019 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 10 monclient(hunting): get_auth_request con 0x7f0b680a68e0 auth_method 0 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6e30d640 1 --2- 172.21.15.190:0/4072431019 >> v2:172.21.15.190:3300/0 conn(0x7f0b681396a0 0x7f0b68139af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 10 monclient(hunting): handle_auth_done global_id 4568 payload 274 2024-08-19T10:39:32.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 1 -- 172.21.15.190:0/4072431019 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b680a6cc0 msgr2=0x7f0b680a70a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 1 --2- 172.21.15.190:0/4072431019 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b680a6cc0 0x7f0b680a70a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 1 -- 172.21.15.190:0/4072431019 >> v2:172.21.15.190:3300/0 conn(0x7f0b681396a0 msgr2=0x7f0b68139af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 1 --2- 172.21.15.190:0/4072431019 >> v2:172.21.15.190:3300/0 conn(0x7f0b681396a0 0x7f0b68139af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 10 monclient: _finish_hunting 0 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 1 monclient: found mon.noname-h 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.042+0000 7f0b6db0c640 1 -- 172.21.15.190:0/4072431019 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b680a4680 con 0x7f0b680a68e0 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6db0c640 1 -- 172.21.15.190:0/4072431019 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b680a68e0 msgr2=0x7f0b680a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6db0c640 1 --2- 172.21.15.190:0/4072431019 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b680a68e0 0x7f0b680a5760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009e50 tx=0x7f0b6001c9c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=4ac026ddef70298 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 1 -- 172.21.15.190:0/4072431019 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b600091b0 con 0x7f0b680a68e0 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:32.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 10 monclient: dump: 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:32.052 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 10 monclient: _finish_auth 0 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 10 monclient: _check_auth_tickets 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 5 monclient: authenticate success, global_id 4568 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 1 -- 172.21.15.190:0/4072431019 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b600a5c30 con 0x7f0b680a68e0 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:32.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 4 set_mon_vals no callback set 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 1 -- 172.21.15.190:0/4072431019 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b600a5de0 con 0x7f0b680a68e0 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6cb0a640 10 monclient: dump: 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:32.054 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:32.055 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:32.055 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:32.055 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:32.055 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:32.055 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:32.055 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:32.055 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient: shutdown 2024-08-19T10:39:32.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:32.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 -- 172.21.15.190:0/4072431019 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b680a68e0 msgr2=0x7f0b680a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:32.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 --2- 172.21.15.190:0/4072431019 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b680a68e0 0x7f0b680a5760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009e50 tx=0x7f0b6001c9c0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 -- 172.21.15.190:0/4072431019 shutdown_connections 2024-08-19T10:39:32.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 --2- 172.21.15.190:0/4072431019 >> v2:172.21.15.190:3300/0 conn(0x7f0b681396a0 0x7f0b68139af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 --2- 172.21.15.190:0/4072431019 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b680a6cc0 0x7f0b680a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 --2- 172.21.15.190:0/4072431019 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b680a68e0 0x7f0b680a5760 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 -- 172.21.15.190:0/4072431019 >> 172.21.15.190:0/4072431019 conn(0x7f0b680abee0 msgr2=0x7f0b680ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:32.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 -- 172.21.15.190:0/4072431019 shutdown_connections 2024-08-19T10:39:32.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 -- 172.21.15.190:0/4072431019 wait complete. 2024-08-19T10:39:32.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient: build_initial_monmap 2024-08-19T10:39:32.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient: monmap: 2024-08-19T10:39:32.060 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:32.060 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:32.060 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:32.061 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 Processor -- start 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 -- start start 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient: init 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient: _add_conns ranks=[0,7,5,2,6,1,8,4,3] 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b680a6cc0 0x7f0b681d1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient(hunting): picked mon.noname-a con 0x7f0b680a6cc0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b681396a0 0x7f0b680a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient(hunting): picked mon.noname-i con 0x7f0b681396a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b681d1a60 0x7f0b680a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient(hunting): picked mon.noname-h con 0x7f0b681d1a60 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:32.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0b680a5ca0 con 0x7f0b680a6cc0 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0b680a75e0 con 0x7f0b681d1a60 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0b680a2b20 con 0x7f0b681396a0 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6fd70640 10 monclient(hunting): authenticate will time out at 1017.619365s 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6d30b640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b681396a0 msgr2=0x7f0b680a20a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6d30b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b681396a0 0x7f0b680a20a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b681d1a60 0x7f0b680a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b681d1a60 0x7f0b680a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:58150/0 (socket says 172.21.15.190:58150) 2024-08-19T10:39:32.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 1 -- 172.21.15.190:0/394492546 learned_addr learned my addr 172.21.15.190:0/394492546 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 10 monclient(hunting): get_auth_request con 0x7f0b681d1a60 auth_method 0 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 10 monclient(hunting): handle_auth_done global_id 4577 payload 824 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6db0c640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b680a6cc0 0x7f0b681d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 1 -- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b681396a0 msgr2=0x7f0b680a20a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b681396a0 0x7f0b680a20a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 1 -- 172.21.15.190:0/394492546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b680a6cc0 msgr2=0x7f0b681d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b680a6cc0 0x7f0b681d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 10 monclient: _finish_hunting 0 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 1 monclient: found mon.noname-h 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.046+0000 7f0b6e30d640 1 -- 172.21.15.190:0/394492546 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6809db40 con 0x7f0b681d1a60 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6e30d640 1 -- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b681d1a60 msgr2=0x7f0b680a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6e30d640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b681d1a60 0x7f0b680a25e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400b480 tx=0x7f0b6400b940 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 1 -- 172.21.15.190:0/394492546 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b64003de0 con 0x7f0b681d1a60 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 10 monclient: dump: 2024-08-19T10:39:32.065 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:32.066 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6fd70640 5 monclient: authenticate success, global_id 4577 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 1 -- 172.21.15.190:0/394492546 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b64015070 con 0x7f0b681d1a60 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6fd70640 10 monclient: _renew_subs 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 1 -- 172.21.15.190:0/394492546 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0b64010480 con 0x7f0b681d1a60 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6fd70640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:32.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6fd70640 1 -- 172.21.15.190:0/394492546 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6813f880 con 0x7f0b681d1a60 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5bfff640 4 set_mon_vals no callback set 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 10 monclient: dump: 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:32.068 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6fd70640 10 monclient: _renew_subs 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6fd70640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6fd70640 1 -- 172.21.15.190:0/394492546 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6809d580 con 0x7f0b681d1a60 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6fd70640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6fd70640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:32.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6fd70640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:32.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6fd70640 1 -- 172.21.15.190:0/394492546 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6809d780 con 0x7f0b681d1a60 2024-08-19T10:39:32.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 1 -- 172.21.15.190:0/394492546 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0b64005070 con 0x7f0b681d1a60 2024-08-19T10:39:32.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0b4c03be00 0x7f0b4c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b5a7fc640 1 -- 172.21.15.190:0/394492546 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(29..29 src has 1..29) ==== 4346+0+0 (secure 0 0 0) 0x7f0b6404e7d0 con 0x7f0b681d1a60 2024-08-19T10:39:32.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6db0c640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0b4c03be00 0x7f0b4c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:32.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.050+0000 7f0b6db0c640 10 monclient: get_auth_request con 0x7f0b4c03be00 auth_method 0 2024-08-19T10:39:32.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.054+0000 7f0b6db0c640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0b4c03be00 0x7f0b4c03e2c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0b600ab000 tx=0x7f0b600a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:32.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.058+0000 7f0b5a7fc640 1 -- 172.21.15.190:0/394492546 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0b6809d780 con 0x7f0b681d1a60 2024-08-19T10:39:32.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.058+0000 7f0b5a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:32.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.058+0000 7f0b5a7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:32.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.226+0000 7f0b6fd70640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-08-19T10:39:32.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.226+0000 7f0b6fd70640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-08-19T10:39:32.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.226+0000 7f0b6fd70640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:32.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.226+0000 7f0b6fd70640 1 -- 172.21.15.190:0/394492546 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"} v 0) -- 0x7f0b680a70a0 con 0x7f0b681d1a60 2024-08-19T10:39:32.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.506+0000 7f0b5a7fc640 1 -- 172.21.15.190:0/394492546 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}]=0 pool 'foo' created v30) ==== 154+0+0 (secure 0 0 0) 0x7f0b6400d360 con 0x7f0b681d1a60 2024-08-19T10:39:32.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.506+0000 7f0b5a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-08-19T10:39:32.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.506+0000 7f0b5a7fc640 10 monclient: _finish_command 2 = system:0 pool 'foo' created 2024-08-19T10:39:32.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.626+0000 7f0b6fd70640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-08-19T10:39:32.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.626+0000 7f0b6fd70640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-08-19T10:39:32.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.626+0000 7f0b6fd70640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:32.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.626+0000 7f0b6fd70640 1 -- 172.21.15.190:0/394492546 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"} v 0) -- 0x7f0b680008e0 con 0x7f0b681d1a60 2024-08-19T10:39:32.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.626+0000 7f0b5a7fc640 1 -- 172.21.15.190:0/394492546 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}]=0 pool 'foo' already exists v30) ==== 161+0+0 (secure 0 0 0) 0x7f0b6404e420 con 0x7f0b681d1a60 2024-08-19T10:39:32.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.626+0000 7f0b5a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123, "pgp_num": 123, "pool_type": "replicated"}] 2024-08-19T10:39:32.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.626+0000 7f0b5a7fc640 10 monclient: _finish_command 3 = system:0 pool 'foo' already exists 2024-08-19T10:39:32.629 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'foo' already exists 2024-08-19T10:39:32.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 -- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0b4c03be00 msgr2=0x7f0b4c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:32.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0b4c03be00 0x7f0b4c03e2c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0b600ab000 tx=0x7f0b600a7070 comp rx=0 tx=0).stop 2024-08-19T10:39:32.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 10 monclient: shutdown 2024-08-19T10:39:32.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:32.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 -- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b681d1a60 msgr2=0x7f0b680a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:32.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b681d1a60 0x7f0b680a25e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400b480 tx=0x7f0b6400b940 comp rx=0 tx=0).stop 2024-08-19T10:39:32.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 -- 172.21.15.190:0/394492546 shutdown_connections 2024-08-19T10:39:32.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0b4c03be00 0x7f0b4c03e2c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0b681d1a60 0x7f0b680a25e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0b681396a0 0x7f0b680a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 --2- 172.21.15.190:0/394492546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b680a6cc0 0x7f0b681d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 -- 172.21.15.190:0/394492546 >> 172.21.15.190:0/394492546 conn(0x7f0b680abee0 msgr2=0x7f0b68134f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:32.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 -- 172.21.15.190:0/394492546 shutdown_connections 2024-08-19T10:39:32.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.630+0000 7f0b6fd70640 1 -- 172.21.15.190:0/394492546 wait complete. 2024-08-19T10:39:32.647 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create fooo 123 123 erasure default 2024-08-19T10:39:32.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient: get_monmap_and_config 2024-08-19T10:39:32.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient: build_initial_monmap 2024-08-19T10:39:32.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient: monmap: 2024-08-19T10:39:32.771 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:32.771 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:32.771 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:32.772330+0000 2024-08-19T10:39:32.771 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:32.772330+0000 2024-08-19T10:39:32.771 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:32.772 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.772 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:32.772 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:32.772 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:32.772 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:32.772 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:32.772 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:32.772 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:32.772 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:32.772 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:32.773 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:32.773 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:32.773 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 1 Processor -- start 2024-08-19T10:39:32.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 1 -- start start 2024-08-19T10:39:32.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient: init 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient: _add_conns ranks=[2,9,6,10,3,8,5,0,7,4,1] 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90141399f0 0x7f90140a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient(hunting): picked mon.noname-b con 0x7f90141399f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9014139dd0 0x7f90140a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient(hunting): picked mon.noname-a con 0x7f9014139dd0 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f901413a1b0 0x7f90140a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient(hunting): picked mon.noname-d con 0x7f901413a1b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f90140a4eb0 con 0x7f9014139dd0 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f90140a5b80 con 0x7f90141399f0 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f90140a8340 con 0x7f901413a1b0 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9018d16640 10 monclient(hunting): authenticate will time out at 1018.342224s 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9014139dd0 0x7f90140a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9014139dd0 0x7f90140a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:36572/0 (socket says 172.21.15.190:36572) 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 1 -- 172.21.15.190:0/2138719515 learned_addr learned my addr 172.21.15.190:0/2138719515 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 10 monclient(hunting): get_auth_request con 0x7f9014139dd0 auth_method 0 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f901259c640 1 --2- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90141399f0 0x7f90140a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:32.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:32.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f901259c640 10 monclient(hunting): get_auth_request con 0x7f90141399f0 auth_method 0 2024-08-19T10:39:32.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f901259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:32.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f901259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:32.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f901259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:32.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9012d9d640 1 --2- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f901413a1b0 0x7f90140a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:32.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9012d9d640 10 monclient(hunting): get_auth_request con 0x7f901413a1b0 auth_method 0 2024-08-19T10:39:32.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9012d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:32.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9012d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:32.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9012d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:32.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:32.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:32.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:32.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f901259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:32.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f901259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:32.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f901259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:32.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9012d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:32.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9012d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:32.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9012d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:32.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 10 monclient(hunting): handle_auth_done global_id 4689 payload 274 2024-08-19T10:39:32.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 1 -- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f901413a1b0 msgr2=0x7f90140a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:32.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 1 --2- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f901413a1b0 0x7f90140a6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 1 -- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90141399f0 msgr2=0x7f90140a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:32.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 1 --2- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90141399f0 0x7f90140a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:39:32.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 1 monclient: found mon.noname-a 2024-08-19T10:39:32.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:32.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:39:32.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.770+0000 7f9011d9b640 1 -- 172.21.15.190:0/2138719515 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90140a4580 con 0x7f9014139dd0 2024-08-19T10:39:32.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9011d9b640 1 -- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9014139dd0 msgr2=0x7f90140a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:32.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9011d9b640 1 --2- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9014139dd0 0x7f90140a68e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90080098a0 tx=0x7f900801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78d49881b7b46e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:32.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 1 -- 172.21.15.190:0/2138719515 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90080b0070 con 0x7f9014139dd0 2024-08-19T10:39:32.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:32.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:39:32.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 10 monclient: dump: 2024-08-19T10:39:32.785 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:32.785 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:32.785 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.785 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.785 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:32.785 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.785 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 10 monclient: _finish_auth 0 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 10 monclient: _check_auth_tickets 2024-08-19T10:39:32.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 1 -- 172.21.15.190:0/2138719515 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90080aa810 con 0x7f9014139dd0 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 1 -- 172.21.15.190:0/2138719515 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90080a9300 con 0x7f9014139dd0 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f901159a640 10 monclient: dump: 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:32.787 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.788 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:32.788 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:32.788 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:32.788 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:32.788 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:32.788 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:32.788 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:32.788 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:32.788 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:32.788 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 5 monclient: authenticate success, global_id 4689 2024-08-19T10:39:32.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:32.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 4 set_mon_vals no callback set 2024-08-19T10:39:32.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient: shutdown 2024-08-19T10:39:32.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:32.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 -- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9014139dd0 msgr2=0x7f90140a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:32.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 --2- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9014139dd0 0x7f90140a68e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90080098a0 tx=0x7f900801c3d0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 -- 172.21.15.190:0/2138719515 shutdown_connections 2024-08-19T10:39:32.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 --2- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f901413a1b0 0x7f90140a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 --2- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9014139dd0 0x7f90140a68e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 --2- 172.21.15.190:0/2138719515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f90141399f0 0x7f90140a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 -- 172.21.15.190:0/2138719515 >> 172.21.15.190:0/2138719515 conn(0x7f9014074c10 msgr2=0x7f90140abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 -- 172.21.15.190:0/2138719515 shutdown_connections 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 -- 172.21.15.190:0/2138719515 wait complete. 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient: build_initial_monmap 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient: monmap: 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:32.790 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 Processor -- start 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 -- start start 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient: init 2024-08-19T10:39:32.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient: _add_conns ranks=[4,2,8,7,3,6,1,0,5] 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90141399f0 0x7f901409efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient(hunting): picked mon.noname-c con 0x7f90141399f0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9014139dd0 0x7f901409f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient(hunting): picked mon.noname-b con 0x7f9014139dd0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f901413a1b0 0x7f901409fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient(hunting): picked mon.noname-e con 0x7f901413a1b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9014143110 con 0x7f9014139dd0 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f90140a3fd0 con 0x7f90141399f0 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:32.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9014143290 con 0x7f901413a1b0 2024-08-19T10:39:32.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:32.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.778+0000 7f9018d16640 10 monclient(hunting): authenticate will time out at 1018.351698s 2024-08-19T10:39:32.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f901259c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90141399f0 0x7f901409efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:32.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9014139dd0 0x7f901409f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:32.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f901259c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90141399f0 0x7f901409efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:50342/0 (socket says 172.21.15.190:50342) 2024-08-19T10:39:32.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f901259c640 1 -- 172.21.15.190:0/4186657319 learned_addr learned my addr 172.21.15.190:0/4186657319 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:32.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f901259c640 10 monclient(hunting): get_auth_request con 0x7f90141399f0 auth_method 0 2024-08-19T10:39:32.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f901259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:32.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f901259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:32.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f901259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 10 monclient(hunting): get_auth_request con 0x7f9014139dd0 auth_method 0 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9012d9d640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f901413a1b0 0x7f901409fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9012d9d640 10 monclient(hunting): get_auth_request con 0x7f901413a1b0 auth_method 0 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9012d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9012d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9012d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f901259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f901259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f901259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:32.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9012d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9012d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9012d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 10 monclient(hunting): handle_auth_done global_id 4529 payload 824 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 1 -- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f901413a1b0 msgr2=0x7f901409fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f901413a1b0 0x7f901409fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 1 -- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90141399f0 msgr2=0x7f901409efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90141399f0 0x7f901409efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 1 monclient: found mon.noname-b 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:32.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:32.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 1 -- 172.21.15.190:0/4186657319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90140a0980 con 0x7f9014139dd0 2024-08-19T10:39:32.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 1 -- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9014139dd0 msgr2=0x7f901409f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9011d9b640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9014139dd0 0x7f901409f4f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f90140a3b50 tx=0x7f9008019170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 1 -- 172.21.15.190:0/4186657319 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90080b0070 con 0x7f9014139dd0 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 10 monclient: dump: 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.798 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 10 monclient: _finish_auth 0 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:39:32.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 1 -- 172.21.15.190:0/4186657319 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90080b1040 con 0x7f9014139dd0 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 1 -- 172.21.15.190:0/4186657319 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90080044a0 con 0x7f9014139dd0 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9018d16640 5 monclient: authenticate success, global_id 4529 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9018d16640 10 monclient: _renew_subs 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9018d16640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9018d16640 1 -- 172.21.15.190:0/4186657319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f901413aa50 con 0x7f9014139dd0 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9010d99640 4 set_mon_vals no callback set 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9002ffd640 10 monclient: dump: 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:32.800 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:32.801 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9018d16640 10 monclient: _renew_subs 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9018d16640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.782+0000 7f9018d16640 1 -- 172.21.15.190:0/4186657319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f901413b010 con 0x7f9014139dd0 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.786+0000 7f9018d16640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.786+0000 7f9018d16640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.786+0000 7f9018d16640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.786+0000 7f9018d16640 1 -- 172.21.15.190:0/4186657319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90140a5640 con 0x7f9014139dd0 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.786+0000 7f9002ffd640 1 -- 172.21.15.190:0/4186657319 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f90080bc060 con 0x7f9014139dd0 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.786+0000 7f9002ffd640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ff403be00 0x7f8ff403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.786+0000 7f9002ffd640 1 -- 172.21.15.190:0/4186657319 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 4688+0+0 (secure 0 0 0) 0x7f90080e7eb0 con 0x7f9014139dd0 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.786+0000 7f901259c640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ff403be00 0x7f8ff403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:32.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.786+0000 7f901259c640 10 monclient: get_auth_request con 0x7f8ff403be00 auth_method 0 2024-08-19T10:39:32.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.786+0000 7f901259c640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ff403be00 0x7f8ff403e2c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9004009c20 tx=0x7f9004009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:32.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.798+0000 7f9002ffd640 1 -- 172.21.15.190:0/4186657319 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9008016110 con 0x7f9014139dd0 2024-08-19T10:39:32.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.798+0000 7f9002ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:32.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.798+0000 7f9002ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:32.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.970+0000 7f9018d16640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-08-19T10:39:32.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.970+0000 7f9018d16640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-08-19T10:39:32.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.970+0000 7f9018d16640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:32.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:32.970+0000 7f9018d16640 1 -- 172.21.15.190:0/4186657319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"} v 0) -- 0x7f90140a3500 con 0x7f9014139dd0 2024-08-19T10:39:34.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.538+0000 7f9002ffd640 1 -- 172.21.15.190:0/4186657319 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}]=0 pool 'fooo' created v32) ==== 188+0+0 (secure 0 0 0) 0x7f9008003d00 con 0x7f9014139dd0 2024-08-19T10:39:34.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.538+0000 7f9002ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-08-19T10:39:34.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.538+0000 7f9002ffd640 10 monclient: _finish_command 2 = system:0 pool 'fooo' created 2024-08-19T10:39:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.662+0000 7f9018d16640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-08-19T10:39:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.662+0000 7f9018d16640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-08-19T10:39:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.662+0000 7f9018d16640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:34.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.662+0000 7f9018d16640 1 -- 172.21.15.190:0/4186657319 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"} v 0) -- 0x7f90140008e0 con 0x7f9014139dd0 2024-08-19T10:39:34.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.666+0000 7f9002ffd640 1 -- 172.21.15.190:0/4186657319 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}]=0 pool 'fooo' already exists v32) ==== 195+0+0 (secure 0 0 0) 0x7f90080bc6f0 con 0x7f9014139dd0 2024-08-19T10:39:34.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.666+0000 7f9002ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "fooo", "pg_num": 123, "pgp_num": 123, "pool_type": "erasure", "erasure_code_profile": "default"}] 2024-08-19T10:39:34.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.666+0000 7f9002ffd640 10 monclient: _finish_command 3 = system:0 pool 'fooo' already exists 2024-08-19T10:39:34.668 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'fooo' already exists 2024-08-19T10:39:34.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 -- 172.21.15.190:0/4186657319 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ff403be00 msgr2=0x7f8ff403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:34.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ff403be00 0x7f8ff403e2c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9004009c20 tx=0x7f9004009190 comp rx=0 tx=0).stop 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 10 monclient: shutdown 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 -- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9014139dd0 msgr2=0x7f901409f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9014139dd0 0x7f901409f4f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f90140a3b50 tx=0x7f9008019170 comp rx=0 tx=0).stop 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 -- 172.21.15.190:0/4186657319 shutdown_connections 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ff403be00 0x7f8ff403e2c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f901413a1b0 0x7f901409fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9014139dd0 0x7f901409f4f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 --2- 172.21.15.190:0/4186657319 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f90141399f0 0x7f901409efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 -- 172.21.15.190:0/4186657319 >> 172.21.15.190:0/4186657319 conn(0x7f9014074c10 msgr2=0x7f9014134df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 -- 172.21.15.190:0/4186657319 shutdown_connections 2024-08-19T10:39:34.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.670+0000 7f9018d16640 1 -- 172.21.15.190:0/4186657319 wait complete. 2024-08-19T10:39:34.686 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create foooo 123 2024-08-19T10:39:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.802+0000 7f1cf54a5640 10 monclient: get_monmap_and_config 2024-08-19T10:39:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.802+0000 7f1cf54a5640 10 monclient: build_initial_monmap 2024-08-19T10:39:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.802+0000 7f1cf54a5640 10 monclient: monmap: 2024-08-19T10:39:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:34.807326+0000 2024-08-19T10:39:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:34.807326+0000 2024-08-19T10:39:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:34.808 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:34.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 1 Processor -- start 2024-08-19T10:39:34.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 1 -- start start 2024-08-19T10:39:34.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 10 monclient: init 2024-08-19T10:39:34.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:34.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 10 monclient: _add_conns ranks=[8,9,6,7,10,2,5,3,4,1,0] 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1cf0139980 0x7f1cf0139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 10 monclient(hunting): picked mon.noname-e con 0x7f1cf0139980 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1cf013a2a0 0x7f1cf00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 10 monclient(hunting): picked mon.noname-a con 0x7f1cf013a2a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1cf00a5c70 0x7f1cf00a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 10 monclient(hunting): picked mon.noname-d con 0x7f1cf00a5c70 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf009ca60 con 0x7f1cf013a2a0 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1cf00a3640 con 0x7f1cf00a5c70 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f1cf00a8470 con 0x7f1cf0139980 2024-08-19T10:39:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cf54a5640 10 monclient(hunting): authenticate will time out at 1020.377141s 2024-08-19T10:39:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1cf00a5c70 0x7f1cf00a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cee7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1cf013a2a0 0x7f1cf00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1ceeffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1cf0139980 0x7f1cf0139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1cf00a5c70 0x7f1cf00a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:49354/0 (socket says 172.21.15.190:49354) 2024-08-19T10:39:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cee7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1cf013a2a0 0x7f1cf00a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:36578/0 (socket says 172.21.15.190:36578) 2024-08-19T10:39:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 1 -- 172.21.15.190:0/4252691834 learned_addr learned my addr 172.21.15.190:0/4252691834 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 10 monclient(hunting): get_auth_request con 0x7f1cf00a5c70 auth_method 0 2024-08-19T10:39:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cee7fc640 10 monclient(hunting): get_auth_request con 0x7f1cf013a2a0 auth_method 0 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cee7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1ceeffd640 10 monclient(hunting): get_auth_request con 0x7f1cf0139980 auth_method 0 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1ceeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1ceeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1ceeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1ceeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1ceeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1ceeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 10 monclient(hunting): handle_auth_done global_id 4479 payload 274 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 1 -- 172.21.15.190:0/4252691834 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1cf0139980 msgr2=0x7f1cf0139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 1 --2- 172.21.15.190:0/4252691834 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1cf0139980 0x7f1cf0139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 1 -- 172.21.15.190:0/4252691834 >> v2:172.21.15.89:3300/0 conn(0x7f1cf013a2a0 msgr2=0x7f1cf00a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 1 --2- 172.21.15.190:0/4252691834 >> v2:172.21.15.89:3300/0 conn(0x7f1cf013a2a0 0x7f1cf00a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 1 monclient: found mon.noname-d 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.806+0000 7f1cef7fe640 1 -- 172.21.15.190:0/4252691834 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf00a3850 con 0x7f1cf00a5c70 2024-08-19T10:39:34.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cef7fe640 1 -- 172.21.15.190:0/4252691834 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1cf00a5c70 msgr2=0x7f1cf00a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:34.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cef7fe640 1 --2- 172.21.15.190:0/4252691834 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1cf00a5c70 0x7f1cf00a67e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009e50 tx=0x7f1ce801c630 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d5e96bfd47bf5bb0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:34.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 1 -- 172.21.15.190:0/4252691834 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ce80ac070 con 0x7f1cf00a5c70 2024-08-19T10:39:34.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:34.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:34.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 10 monclient: dump: 2024-08-19T10:39:34.818 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:34.818 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:34.818 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:34.818 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:34.818 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:34.818 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:34.818 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:34.818 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 10 monclient: _finish_auth 0 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 10 monclient: _check_auth_tickets 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:34.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 1 -- 172.21.15.190:0/4252691834 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce80090d0 con 0x7f1cf00a5c70 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 1 -- 172.21.15.190:0/4252691834 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ce8007cd0 con 0x7f1cf00a5c70 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 5 monclient: authenticate success, global_id 4479 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 4 set_mon_vals no callback set 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cedffb640 10 monclient: dump: 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:34.820 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 10 monclient: shutdown 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 1 -- 172.21.15.190:0/4252691834 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1cf00a5c70 msgr2=0x7f1cf00a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:34.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 1 --2- 172.21.15.190:0/4252691834 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1cf00a5c70 0x7f1cf00a67e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009e50 tx=0x7f1ce801c630 comp rx=0 tx=0).stop 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 1 -- 172.21.15.190:0/4252691834 shutdown_connections 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 1 --2- 172.21.15.190:0/4252691834 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1cf00a5c70 0x7f1cf00a67e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 1 --2- 172.21.15.190:0/4252691834 >> v2:172.21.15.89:3300/0 conn(0x7f1cf013a2a0 0x7f1cf00a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 1 --2- 172.21.15.190:0/4252691834 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1cf0139980 0x7f1cf0139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 1 -- 172.21.15.190:0/4252691834 >> 172.21.15.190:0/4252691834 conn(0x7f1cf00abde0 msgr2=0x7f1cf00ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 1 -- 172.21.15.190:0/4252691834 shutdown_connections 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.810+0000 7f1cf54a5640 1 -- 172.21.15.190:0/4252691834 wait complete. 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient: build_initial_monmap 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient: monmap: 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:34.822 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:34.823 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:34.823 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:34.823 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:34.823 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:34.823 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:34.823 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:34.823 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:34.823 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:34.823 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:34.823 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 1 Processor -- start 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 1 -- start start 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient: init 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient: _add_conns ranks=[8,3,5,7,1,6,2,4,0] 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1cf00a5c70 0x7f1cf00a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient(hunting): picked mon.noname-e con 0x7f1cf00a5c70 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf0139980 0x7f1cf013fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient(hunting): picked mon.noname-g con 0x7f1cf0139980 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:34.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1cf013bbd0 0x7f1cf013a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient(hunting): picked mon.noname-h con 0x7f1cf013bbd0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf0143270 con 0x7f1cf0139980 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1cf01430f0 con 0x7f1cf013bbd0 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f1cf01433f0 con 0x7f1cf00a5c70 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cf54a5640 10 monclient(hunting): authenticate will time out at 1020.385507s 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf0139980 0x7f1cf013fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf0139980 0x7f1cf013fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46454/0 (socket says 172.21.15.190:46454) 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 1 -- 172.21.15.190:0/788490771 learned_addr learned my addr 172.21.15.190:0/788490771 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:34.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 10 monclient(hunting): get_auth_request con 0x7f1cf0139980 auth_method 0 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cef7fe640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1cf013bbd0 0x7f1cf013a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cef7fe640 10 monclient(hunting): get_auth_request con 0x7f1cf013bbd0 auth_method 0 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cef7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:34.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1ceeffd640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1cf00a5c70 0x7f1cf00a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1ceeffd640 10 monclient(hunting): get_auth_request con 0x7f1cf00a5c70 auth_method 0 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1ceeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1ceeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1ceeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 10 monclient(hunting): handle_auth_done global_id 4539 payload 824 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 1 -- 172.21.15.190:0/788490771 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1cf00a5c70 msgr2=0x7f1cf00a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1cf00a5c70 0x7f1cf00a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 1 -- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1cf013bbd0 msgr2=0x7f1cf013a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1cf013bbd0 0x7f1cf013a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:39:34.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 1 monclient: found mon.noname-g 2024-08-19T10:39:34.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:34.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:34.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.814+0000 7f1cee7fc640 1 -- 172.21.15.190:0/788490771 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf00a1c30 con 0x7f1cf0139980 2024-08-19T10:39:34.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cee7fc640 1 -- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf0139980 msgr2=0x7f1cf013fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cee7fc640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf0139980 0x7f1cf013fb40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e8a0 tx=0x7f1ce400eca0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 1 -- 172.21.15.190:0/788490771 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ce4017070 con 0x7f1cf0139980 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 10 monclient: dump: 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:34.830 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:34.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 1 -- 172.21.15.190:0/788490771 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce4013470 con 0x7f1cf0139980 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cf54a5640 5 monclient: authenticate success, global_id 4539 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cf54a5640 10 monclient: _renew_subs 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cf54a5640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cf54a5640 1 -- 172.21.15.190:0/788490771 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf00a68b0 con 0x7f1cf0139980 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 1 -- 172.21.15.190:0/788490771 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ce4013a90 con 0x7f1cf0139980 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 10 monclient: dump: 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:34.832 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cecff9640 4 set_mon_vals no callback set 2024-08-19T10:39:34.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cf54a5640 10 monclient: _renew_subs 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cf54a5640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cf54a5640 1 -- 172.21.15.190:0/788490771 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf013cb30 con 0x7f1cf0139980 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cf54a5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cf54a5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 1 -- 172.21.15.190:0/788490771 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1ce401a070 con 0x7f1cf0139980 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cf54a5640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cf54a5640 1 -- 172.21.15.190:0/788490771 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf0139d60 con 0x7f1cf0139980 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1cdc03be00 0x7f1cdc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1cd77fe640 1 -- 172.21.15.190:0/788490771 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(32..32 src has 1..32) ==== 5130+0+0 (secure 0 0 0) 0x7f1ce400f040 con 0x7f1cf0139980 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1ceeffd640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1cdc03be00 0x7f1cdc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.818+0000 7f1ceeffd640 10 monclient: get_auth_request con 0x7f1cdc03be00 auth_method 0 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.822+0000 7f1ceeffd640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1cdc03be00 0x7f1cdc03e2c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009af0 tx=0x7f1ce0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:34.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.822+0000 7f1cd77fe640 1 -- 172.21.15.190:0/788490771 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1ce4021070 con 0x7f1cf0139980 2024-08-19T10:39:34.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.822+0000 7f1cd77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:34.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.822+0000 7f1cd77fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:34.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.994+0000 7f1cf54a5640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-08-19T10:39:34.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.994+0000 7f1cf54a5640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-08-19T10:39:34.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.994+0000 7f1cf54a5640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:34.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:34.994+0000 7f1cf54a5640 1 -- 172.21.15.190:0/788490771 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "foooo", "pg_num": 123} v 0) -- 0x7f1cf006c8d0 con 0x7f1cf0139980 2024-08-19T10:39:35.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.538+0000 7f1cd77fe640 1 -- 172.21.15.190:0/788490771 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}]=0 pool 'foooo' created v33) ==== 115+0+0 (secure 0 0 0) 0x7f1ce4011350 con 0x7f1cf0139980 2024-08-19T10:39:35.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.538+0000 7f1cd77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-08-19T10:39:35.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.538+0000 7f1cd77fe640 10 monclient: _finish_command 2 = system:0 pool 'foooo' created 2024-08-19T10:39:35.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.702+0000 7f1cf54a5640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-08-19T10:39:35.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.702+0000 7f1cf54a5640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-08-19T10:39:35.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.702+0000 7f1cf54a5640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:35.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.702+0000 7f1cf54a5640 1 -- 172.21.15.190:0/788490771 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "foooo", "pg_num": 123} v 0) -- 0x7f1cf013cd30 con 0x7f1cf0139980 2024-08-19T10:39:35.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cd77fe640 1 -- 172.21.15.190:0/788490771 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}]=0 pool 'foooo' already exists v33) ==== 122+0+0 (secure 0 0 0) 0x7f1ce401ae20 con 0x7f1cf0139980 2024-08-19T10:39:35.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cd77fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foooo", "pg_num": 123}] 2024-08-19T10:39:35.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cd77fe640 10 monclient: _finish_command 3 = system:0 pool 'foooo' already exists 2024-08-19T10:39:35.887 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'foooo' already exists 2024-08-19T10:39:35.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 -- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1cdc03be00 msgr2=0x7f1cdc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:35.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1cdc03be00 0x7f1cdc03e2c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009af0 tx=0x7f1ce0008040 comp rx=0 tx=0).stop 2024-08-19T10:39:35.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 10 monclient: shutdown 2024-08-19T10:39:35.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:35.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 -- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf0139980 msgr2=0x7f1cf013fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:35.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf0139980 0x7f1cf013fb40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e8a0 tx=0x7f1ce400eca0 comp rx=0 tx=0).stop 2024-08-19T10:39:35.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 -- 172.21.15.190:0/788490771 shutdown_connections 2024-08-19T10:39:35.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1cf013bbd0 0x7f1cf013a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:35.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf0139980 0x7f1cf013fb40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:35.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1cdc03be00 0x7f1cdc03e2c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:35.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 --2- 172.21.15.190:0/788490771 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1cf00a5c70 0x7f1cf00a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:35.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 -- 172.21.15.190:0/788490771 >> 172.21.15.190:0/788490771 conn(0x7f1cf00abde0 msgr2=0x7f1cf00ab460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:35.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 -- 172.21.15.190:0/788490771 shutdown_connections 2024-08-19T10:39:35.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:35.886+0000 7f1cf54a5640 1 -- 172.21.15.190:0/788490771 wait complete. 2024-08-19T10:39:35.912 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create foo 123 2024-08-19T10:39:36.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.030+0000 7fa23194f640 10 monclient: get_monmap_and_config 2024-08-19T10:39:36.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.030+0000 7fa23194f640 10 monclient: build_initial_monmap 2024-08-19T10:39:36.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.030+0000 7fa23194f640 10 monclient: monmap: 2024-08-19T10:39:36.033 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:36.033 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:36.034374+0000 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:36.034374+0000 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:36.034 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:36.035 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:36.035 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:36.035 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:36.035 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.030+0000 7fa23194f640 1 Processor -- start 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.030+0000 7fa23194f640 1 -- start start 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.030+0000 7fa23194f640 10 monclient: init 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 10 monclient: _add_conns ranks=[9,6,8,2,7,10,1,5,0,4,3] 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa22c0a6740 0x7fa22c0a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 10 monclient(hunting): picked mon.noname-a con 0x7fa22c0a6740 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a5bc0 0x7fa22c0a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 10 monclient(hunting): picked mon.noname-d con 0x7fa22c0a5bc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa22c0a7060 0x7fa22c139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 10 monclient(hunting): picked mon.noname-e con 0x7fa22c0a7060 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fa22c0a4c20 con 0x7fa22c0a6740 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa22c0a3530 con 0x7fa22c0a5bc0 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa22c0a3740 con 0x7fa22c0a7060 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa23194f640 10 monclient(hunting): authenticate will time out at 1021.604410s 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa22c0a6740 0x7fa22c0a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a5bc0 0x7fa22c0a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa22c0a7060 0x7fa22c139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa22c0a6740 0x7fa22c0a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:37238/0 (socket says 172.21.15.190:37238) 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22affd640 1 -- 172.21.15.190:0/3089535017 learned_addr learned my addr 172.21.15.190:0/3089535017 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:36.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22affd640 10 monclient(hunting): get_auth_request con 0x7fa22c0a6740 auth_method 0 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a5bc0 0x7fa22c0a6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:58286/0 (socket says 172.21.15.190:58286) 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22b7fe640 10 monclient(hunting): get_auth_request con 0x7fa22c0a7060 auth_method 0 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 10 monclient(hunting): get_auth_request con 0x7fa22c0a5bc0 auth_method 0 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:36.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 10 monclient(hunting): handle_auth_done global_id 4488 payload 274 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 1 -- 172.21.15.190:0/3089535017 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa22c0a7060 msgr2=0x7fa22c139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 1 --2- 172.21.15.190:0/3089535017 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa22c0a7060 0x7fa22c139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 1 -- 172.21.15.190:0/3089535017 >> v2:172.21.15.89:3300/0 conn(0x7fa22c0a6740 msgr2=0x7fa22c0a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:36.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 1 --2- 172.21.15.190:0/3089535017 >> v2:172.21.15.89:3300/0 conn(0x7fa22c0a6740 0x7fa22c0a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:39:36.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 1 monclient: found mon.noname-d 2024-08-19T10:39:36.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:36.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:36.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.034+0000 7fa22a7fc640 1 -- 172.21.15.190:0/3089535017 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c0a74d0 con 0x7fa22c0a5bc0 2024-08-19T10:39:36.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa22a7fc640 1 -- 172.21.15.190:0/3089535017 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a5bc0 msgr2=0x7fa22c0a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:36.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa22a7fc640 1 --2- 172.21.15.190:0/3089535017 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a5bc0 0x7fa22c0a6b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2200097f0 tx=0x7fa22001c360 comp rx=0 tx=0).ready entity=mon.6 client_cookie=8ff78c7f4a763a93 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 1 -- 172.21.15.190:0/3089535017 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa2200b0070 con 0x7fa22c0a5bc0 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 10 monclient: dump: 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:36.046 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 10 monclient: _finish_auth 0 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:36.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 1 -- 172.21.15.190:0/3089535017 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2200aa810 con 0x7fa22c0a5bc0 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 1 -- 172.21.15.190:0/3089535017 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa2200a9300 con 0x7fa22c0a5bc0 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa229ffb640 10 monclient: dump: 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:36.048 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 5 monclient: authenticate success, global_id 4488 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 4 set_mon_vals no callback set 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 10 monclient: shutdown 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:36.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 1 -- 172.21.15.190:0/3089535017 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a5bc0 msgr2=0x7fa22c0a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 1 --2- 172.21.15.190:0/3089535017 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a5bc0 0x7fa22c0a6b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2200097f0 tx=0x7fa22001c360 comp rx=0 tx=0).stop 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 1 -- 172.21.15.190:0/3089535017 shutdown_connections 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 1 --2- 172.21.15.190:0/3089535017 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa22c0a7060 0x7fa22c139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 1 --2- 172.21.15.190:0/3089535017 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a5bc0 0x7fa22c0a6b20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 1 --2- 172.21.15.190:0/3089535017 >> v2:172.21.15.89:3300/0 conn(0x7fa22c0a6740 0x7fa22c0a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 1 -- 172.21.15.190:0/3089535017 >> 172.21.15.190:0/3089535017 conn(0x7fa22c0abd40 msgr2=0x7fa22c0ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 1 -- 172.21.15.190:0/3089535017 shutdown_connections 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 1 -- 172.21.15.190:0/3089535017 wait complete. 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 10 monclient: build_initial_monmap 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.038+0000 7fa23194f640 10 monclient: monmap: 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:36.050 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 1 Processor -- start 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 1 -- start start 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 10 monclient: init 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 10 monclient: _add_conns ranks=[6,1,4,0,2,7,3,5,8] 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a6740 0x7fa22c09ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 10 monclient(hunting): picked mon.noname-d con 0x7fa22c0a6740 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa22c0a7060 0x7fa22c09cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 10 monclient(hunting): picked mon.noname-f con 0x7fa22c0a7060 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa22c09d4e0 0x7fa22c1334a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 10 monclient(hunting): picked mon.noname-c con 0x7fa22c09d4e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c143450 con 0x7fa22c0a7060 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa22c1432d0 con 0x7fa22c09d4e0 2024-08-19T10:39:36.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa22c1435d0 con 0x7fa22c0a6740 2024-08-19T10:39:36.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:36.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa23194f640 10 monclient(hunting): authenticate will time out at 1021.612619s 2024-08-19T10:39:36.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22a7fc640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa22c0a7060 msgr2=0x7fa22c09cfa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:36.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa22c0a7060 0x7fa22c09cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:36.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa22c09d4e0 0x7fa22c1334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a6740 0x7fa22c09ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a6740 0x7fa22c09ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:58288/0 (socket says 172.21.15.190:58288) 2024-08-19T10:39:36.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa22c09d4e0 0x7fa22c1334a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:50896/0 (socket says 172.21.15.190:50896) 2024-08-19T10:39:36.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 1 -- 172.21.15.190:0/1902366843 learned_addr learned my addr 172.21.15.190:0/1902366843 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 10 monclient(hunting): get_auth_request con 0x7fa22c0a6740 auth_method 0 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22b7fe640 10 monclient(hunting): get_auth_request con 0x7fa22c09d4e0 auth_method 0 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:36.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 10 monclient(hunting): handle_auth_done global_id 4497 payload 824 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 1 -- 172.21.15.190:0/1902366843 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa22c09d4e0 msgr2=0x7fa22c1334a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa22c09d4e0 0x7fa22c1334a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 1 -- 172.21.15.190:0/1902366843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa22c0a7060 msgr2=0x7fa22c09cfa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa22c0a7060 0x7fa22c09cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 10 monclient: _finish_hunting 0 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 1 monclient: found mon.noname-d 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:36.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 1 -- 172.21.15.190:0/1902366843 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c09f870 con 0x7fa22c0a6740 2024-08-19T10:39:36.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 1 -- 172.21.15.190:0/1902366843 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a6740 msgr2=0x7fa22c09ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:36.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.042+0000 7fa22affd640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a6740 0x7fa22c09ca60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00e8f0 tx=0x7fa21c00edb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:36.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 1 -- 172.21.15.190:0/1902366843 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa21c017070 con 0x7fa22c0a6740 2024-08-19T10:39:36.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:36.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:36.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 10 monclient: dump: 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:36.058 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 10 monclient: _finish_auth 0 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 1 -- 172.21.15.190:0/1902366843 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa21c013470 con 0x7fa22c0a6740 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa23194f640 5 monclient: authenticate success, global_id 4497 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 1 -- 172.21.15.190:0/1902366843 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa21c013620 con 0x7fa22c0a6740 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa23194f640 10 monclient: _renew_subs 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa23194f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa23194f640 1 -- 172.21.15.190:0/1902366843 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c09fb50 con 0x7fa22c0a6740 2024-08-19T10:39:36.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:36.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:36.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 10 monclient: dump: 2024-08-19T10:39:36.060 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:36.060 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:36.060 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.060 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.060 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:36.060 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.060 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa2297fa640 4 set_mon_vals no callback set 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa23194f640 10 monclient: _renew_subs 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa23194f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:36.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa23194f640 1 -- 172.21.15.190:0/1902366843 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c09e810 con 0x7fa22c0a6740 2024-08-19T10:39:36.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa23194f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:36.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa23194f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:36.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa23194f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:36.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa23194f640 1 -- 172.21.15.190:0/1902366843 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa22c0a5680 con 0x7fa22c0a6740 2024-08-19T10:39:36.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 1 -- 172.21.15.190:0/1902366843 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa21c01a070 con 0x7fa22c0a6740 2024-08-19T10:39:36.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa21803be30 0x7fa21803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa213fff640 1 -- 172.21.15.190:0/1902366843 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(33..33 src has 1..33) ==== 5474+0+0 (secure 0 0 0) 0x7fa21c00f040 con 0x7fa22c0a6740 2024-08-19T10:39:36.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa22a7fc640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa21803be30 0x7fa21803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.046+0000 7fa22a7fc640 10 monclient: get_auth_request con 0x7fa21803be30 auth_method 0 2024-08-19T10:39:36.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.050+0000 7fa22a7fc640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa21803be30 0x7fa21803e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa2200ad000 tx=0x7fa2200a4f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:36.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.058+0000 7fa213fff640 1 -- 172.21.15.190:0/1902366843 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fa21c04ce30 con 0x7fa22c0a6740 2024-08-19T10:39:36.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.058+0000 7fa213fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:36.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.058+0000 7fa213fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:36.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.230+0000 7fa23194f640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-08-19T10:39:36.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.230+0000 7fa23194f640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-08-19T10:39:36.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.230+0000 7fa23194f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:36.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.230+0000 7fa23194f640 1 -- 172.21.15.190:0/1902366843 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123} v 0) -- 0x7fa22c13f830 con 0x7fa22c0a6740 2024-08-19T10:39:36.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.230+0000 7fa213fff640 1 -- 172.21.15.190:0/1902366843 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}]=0 pool 'foo' already exists v33) ==== 118+0+0 (secure 0 0 0) 0x7fa21c04d1b0 con 0x7fa22c0a6740 2024-08-19T10:39:36.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.230+0000 7fa213fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-08-19T10:39:36.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.230+0000 7fa213fff640 10 monclient: _finish_command 2 = system:0 pool 'foo' already exists 2024-08-19T10:39:36.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.334+0000 7fa23194f640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-08-19T10:39:36.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.334+0000 7fa23194f640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-08-19T10:39:36.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.334+0000 7fa23194f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:36.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.334+0000 7fa23194f640 1 -- 172.21.15.190:0/1902366843 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd pool create", "pool": "foo", "pg_num": 123} v 0) -- 0x7fa22c0a0510 con 0x7fa22c0a6740 2024-08-19T10:39:36.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.334+0000 7fa213fff640 1 -- 172.21.15.190:0/1902366843 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}]=0 pool 'foo' already exists v33) ==== 118+0+0 (secure 0 0 0) 0x7fa21c0123d0 con 0x7fa22c0a6740 2024-08-19T10:39:36.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.334+0000 7fa213fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "foo", "pg_num": 123}] 2024-08-19T10:39:36.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.334+0000 7fa213fff640 10 monclient: _finish_command 3 = system:0 pool 'foo' already exists 2024-08-19T10:39:36.338 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'foo' already exists 2024-08-19T10:39:36.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 -- 172.21.15.190:0/1902366843 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa21803be30 msgr2=0x7fa21803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:36.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa21803be30 0x7fa21803e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa2200ad000 tx=0x7fa2200a4f90 comp rx=0 tx=0).stop 2024-08-19T10:39:36.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 10 monclient: shutdown 2024-08-19T10:39:36.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:36.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 -- 172.21.15.190:0/1902366843 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a6740 msgr2=0x7fa22c09ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:36.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a6740 0x7fa22c09ca60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00e8f0 tx=0x7fa21c00edb0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 -- 172.21.15.190:0/1902366843 shutdown_connections 2024-08-19T10:39:36.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa21803be30 0x7fa21803e2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa22c09d4e0 0x7fa22c1334a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa22c0a7060 0x7fa22c09cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 --2- 172.21.15.190:0/1902366843 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa22c0a6740 0x7fa22c09ca60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 -- 172.21.15.190:0/1902366843 >> 172.21.15.190:0/1902366843 conn(0x7fa22c0abd40 msgr2=0x7fa22c136770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:36.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 -- 172.21.15.190:0/1902366843 shutdown_connections 2024-08-19T10:39:36.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.338+0000 7fa23194f640 1 -- 172.21.15.190:0/1902366843 wait complete. 2024-08-19T10:39:36.356 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set foo size 1 --yes-i-really-mean-it 2024-08-19T10:39:36.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient: get_monmap_and_config 2024-08-19T10:39:36.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient: build_initial_monmap 2024-08-19T10:39:36.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient: monmap: 2024-08-19T10:39:36.480 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:36.481 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:36.481 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:36.481247+0000 2024-08-19T10:39:36.481 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:36.481247+0000 2024-08-19T10:39:36.481 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:36.481 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.481 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:36.482 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:36.482 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:36.482 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:36.482 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:36.482 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:36.482 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:36.483 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:36.483 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:36.483 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:36.483 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:36.483 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 1 Processor -- start 2024-08-19T10:39:36.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 1 -- start start 2024-08-19T10:39:36.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient: init 2024-08-19T10:39:36.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:36.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient: _add_conns ranks=[9,6,5,2,4,10,7,8,3,0,1] 2024-08-19T10:39:36.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f964c1396a0 0x7f964c139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient(hunting): picked mon.noname-a con 0x7f964c1396a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:36.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f964c139fc0 0x7f964c0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient(hunting): picked mon.noname-d con 0x7f964c139fc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:36.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f964c0a6e20 0x7f964c0a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient(hunting): picked mon.noname-h con 0x7f964c0a6e20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:36.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f964c0a4f20 con 0x7f964c1396a0 2024-08-19T10:39:36.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f964c0a83e0 con 0x7f964c0a6e20 2024-08-19T10:39:36.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f964c0a2f30 con 0x7f964c139fc0 2024-08-19T10:39:36.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:36.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f9651ded640 10 monclient(hunting): authenticate will time out at 1022.051135s 2024-08-19T10:39:36.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f964c0a6e20 0x7f964c0a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f964c0a6e20 0x7f964c0a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:53492/0 (socket says 172.21.15.190:53492) 2024-08-19T10:39:36.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 1 -- 172.21.15.190:0/3493984697 learned_addr learned my addr 172.21.15.190:0/3493984697 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:36.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 10 monclient(hunting): get_auth_request con 0x7f964c0a6e20 auth_method 0 2024-08-19T10:39:36.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:36.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:36.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:36.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:36.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:36.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:36.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964affd640 1 --2- 172.21.15.190:0/3493984697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f964c139fc0 0x7f964c0a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964affd640 10 monclient(hunting): get_auth_request con 0x7f964c139fc0 auth_method 0 2024-08-19T10:39:36.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:36.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:36.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:36.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964b7fe640 1 --2- 172.21.15.190:0/3493984697 >> v2:172.21.15.89:3300/0 conn(0x7f964c1396a0 0x7f964c139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964b7fe640 10 monclient(hunting): get_auth_request con 0x7f964c1396a0 auth_method 0 2024-08-19T10:39:36.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:36.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:36.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:36.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:36.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:36.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:36.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 10 monclient(hunting): handle_auth_done global_id 4595 payload 274 2024-08-19T10:39:36.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 1 -- 172.21.15.190:0/3493984697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f964c139fc0 msgr2=0x7f964c0a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:36.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 1 --2- 172.21.15.190:0/3493984697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f964c139fc0 0x7f964c0a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 1 -- 172.21.15.190:0/3493984697 >> v2:172.21.15.89:3300/0 conn(0x7f964c1396a0 msgr2=0x7f964c139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:36.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 1 --2- 172.21.15.190:0/3493984697 >> v2:172.21.15.89:3300/0 conn(0x7f964c1396a0 0x7f964c139a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:36.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 1 monclient: found mon.noname-h 2024-08-19T10:39:36.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:36.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:36.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.478+0000 7f964bfff640 1 -- 172.21.15.190:0/3493984697 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964c0a35e0 con 0x7f964c0a6e20 2024-08-19T10:39:36.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964bfff640 1 -- 172.21.15.190:0/3493984697 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f964c0a6e20 msgr2=0x7f964c0a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:36.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964bfff640 1 --2- 172.21.15.190:0/3493984697 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f964c0a6e20 0x7f964c0a7220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9644009e80 tx=0x7f964401c8d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=97587b3506a2231b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:36.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 1 -- 172.21.15.190:0/3493984697 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f96440090d0 con 0x7f964c0a6e20 2024-08-19T10:39:36.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:36.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:36.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 10 monclient: dump: 2024-08-19T10:39:36.503 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:36.503 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:36.504 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 1 -- 172.21.15.190:0/3493984697 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96440a5980 con 0x7f964c0a6e20 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 1 -- 172.21.15.190:0/3493984697 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f96440ad3f0 con 0x7f964c0a6e20 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 5 monclient: authenticate success, global_id 4595 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 4 set_mon_vals no callback set 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:36.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964a7fc640 10 monclient: dump: 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:36.506 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:36.507 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:36.507 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:36.507 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:36.507 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient: shutdown 2024-08-19T10:39:36.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:36.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 -- 172.21.15.190:0/3493984697 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f964c0a6e20 msgr2=0x7f964c0a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:36.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 --2- 172.21.15.190:0/3493984697 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f964c0a6e20 0x7f964c0a7220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9644009e80 tx=0x7f964401c8d0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 -- 172.21.15.190:0/3493984697 shutdown_connections 2024-08-19T10:39:36.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 --2- 172.21.15.190:0/3493984697 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f964c0a6e20 0x7f964c0a7220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 --2- 172.21.15.190:0/3493984697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f964c139fc0 0x7f964c0a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 --2- 172.21.15.190:0/3493984697 >> v2:172.21.15.89:3300/0 conn(0x7f964c1396a0 0x7f964c139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 -- 172.21.15.190:0/3493984697 >> 172.21.15.190:0/3493984697 conn(0x7f964c0abee0 msgr2=0x7f964c0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 -- 172.21.15.190:0/3493984697 shutdown_connections 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 -- 172.21.15.190:0/3493984697 wait complete. 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient: build_initial_monmap 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient: monmap: 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.508 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 Processor -- start 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 -- start start 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient: init 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:36.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient: _add_conns ranks=[1,3,8,6,5,7,0,4,2] 2024-08-19T10:39:36.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f964c0a6e20 0x7f964c1348e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient(hunting): picked mon.noname-f con 0x7f964c0a6e20 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:36.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f964c1396a0 0x7f964c134e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient(hunting): picked mon.noname-g con 0x7f964c1396a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:36.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f964c139fc0 0x7f964c09ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient(hunting): picked mon.noname-e con 0x7f964c139fc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:36.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f964c13a3a0 con 0x7f964c0a6e20 2024-08-19T10:39:36.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f964c143390 con 0x7f964c1396a0 2024-08-19T10:39:36.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:36.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f964c135780 con 0x7f964c139fc0 2024-08-19T10:39:36.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:36.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f9651ded640 10 monclient(hunting): authenticate will time out at 1022.058596s 2024-08-19T10:39:36.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964b7fe640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f964c0a6e20 msgr2=0x7f964c1348e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:36.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f964c0a6e20 0x7f964c1348e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:36.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f964c1396a0 0x7f964c134e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f964c1396a0 0x7f964c134e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47404/0 (socket says 172.21.15.190:47404) 2024-08-19T10:39:36.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 1 -- 172.21.15.190:0/3026968407 learned_addr learned my addr 172.21.15.190:0/3026968407 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:36.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 10 monclient(hunting): get_auth_request con 0x7f964c1396a0 auth_method 0 2024-08-19T10:39:36.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:36.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:36.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:36.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:36.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:36.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:36.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 10 monclient(hunting): handle_auth_done global_id 4548 payload 824 2024-08-19T10:39:36.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964bfff640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f964c139fc0 0x7f964c09ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 1 -- 172.21.15.190:0/3026968407 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f964c139fc0 msgr2=0x7f964c09ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:36.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f964c139fc0 0x7f964c09ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 1 -- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f964c0a6e20 msgr2=0x7f964c1348e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:36.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f964c0a6e20 0x7f964c1348e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:36.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 10 monclient: _finish_hunting 0 2024-08-19T10:39:36.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 1 monclient: found mon.noname-g 2024-08-19T10:39:36.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:36.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:36.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.486+0000 7f964affd640 1 -- 172.21.15.190:0/3026968407 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964c09e3a0 con 0x7f964c1396a0 2024-08-19T10:39:36.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f964affd640 1 -- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f964c1396a0 msgr2=0x7f964c134e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:36.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f964affd640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f964c1396a0 0x7f964c134e20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f964000e870 tx=0x7f964000ec70 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:36.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 1 -- 172.21.15.190:0/3026968407 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9640017070 con 0x7f964c1396a0 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 10 monclient: dump: 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:36.514 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:36.515 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:36.515 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:36.515 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:36.515 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:36.515 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:36.515 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:36.515 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 10 monclient: _finish_auth 0 2024-08-19T10:39:36.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:36.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:36.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 1 -- 172.21.15.190:0/3026968407 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9640013470 con 0x7f964c1396a0 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 1 -- 172.21.15.190:0/3026968407 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9640013a90 con 0x7f964c1396a0 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9651ded640 5 monclient: authenticate success, global_id 4548 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9651ded640 10 monclient: _renew_subs 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9651ded640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9651ded640 1 -- 172.21.15.190:0/3026968407 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964c0a06e0 con 0x7f964c1396a0 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f96497fa640 4 set_mon_vals no callback set 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 10 monclient: dump: 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:36.516 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:36.517 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.517 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:36.517 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:36.517 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:36.517 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:36.517 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:36.517 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:36.517 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:36.517 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:36.517 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:36.517 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9651ded640 10 monclient: _renew_subs 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9651ded640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9651ded640 1 -- 172.21.15.190:0/3026968407 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f964c135360 con 0x7f964c1396a0 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9651ded640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9651ded640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9651ded640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9651ded640 1 -- 172.21.15.190:0/3026968407 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f964c139a80 con 0x7f964c1396a0 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 1 -- 172.21.15.190:0/3026968407 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f964001a070 con 0x7f964c1396a0 2024-08-19T10:39:36.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f963803be00 0x7f963803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:36.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f9633fff640 1 -- 172.21.15.190:0/3026968407 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(33..33 src has 1..33) ==== 5474+0+0 (secure 0 0 0) 0x7f964000f040 con 0x7f964c1396a0 2024-08-19T10:39:36.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f964b7fe640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f963803be00 0x7f963803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:36.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.490+0000 7f964b7fe640 10 monclient: get_auth_request con 0x7f963803be00 auth_method 0 2024-08-19T10:39:36.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.494+0000 7f964b7fe640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f963803be00 0x7f963803e2c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f963c000f00 tx=0x7f963c006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:36.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.494+0000 7f9633fff640 1 -- 172.21.15.190:0/3026968407 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9640021070 con 0x7f964c1396a0 2024-08-19T10:39:36.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.494+0000 7f9633fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:36.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.494+0000 7f9633fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:36.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.705+0000 7f9651ded640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-08-19T10:39:36.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.705+0000 7f9651ded640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-08-19T10:39:36.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.705+0000 7f9651ded640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:36.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:36.705+0000 7f9651ded640 1 -- 172.21.15.190:0/3026968407 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true} v 0) -- 0x7f964c0a0080 con 0x7f964c1396a0 2024-08-19T10:39:37.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:37.909+0000 7f9633fff640 1 -- 172.21.15.190:0/3026968407 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}]=0 set pool 4 size to 1 v35) ==== 152+0+0 (secure 0 0 0) 0x7f9640013620 con 0x7f964c1396a0 2024-08-19T10:39:37.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:37.909+0000 7f9633fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-08-19T10:39:37.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:37.909+0000 7f9633fff640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 1 2024-08-19T10:39:38.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.037+0000 7f9651ded640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-08-19T10:39:38.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.037+0000 7f9651ded640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-08-19T10:39:38.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.037+0000 7f9651ded640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:38.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.037+0000 7f9651ded640 1 -- 172.21.15.190:0/3026968407 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true} v 0) -- 0x7f964c0008e0 con 0x7f964c1396a0 2024-08-19T10:39:38.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.913+0000 7f9633fff640 1 -- 172.21.15.190:0/3026968407 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}]=0 set pool 4 size to 1 v36) ==== 152+0+0 (secure 0 0 0) 0x7f964001b750 con 0x7f964c1396a0 2024-08-19T10:39:38.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.913+0000 7f9633fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "1", "yes_i_really_mean_it": true}] 2024-08-19T10:39:38.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.913+0000 7f9633fff640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 1 2024-08-19T10:39:38.920 INFO:tasks.workunit.client.0.smithi190.stderr:set pool 4 size to 1 2024-08-19T10:39:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.925+0000 7f9651ded640 1 -- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f963803be00 msgr2=0x7f963803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.925+0000 7f9651ded640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f963803be00 0x7f963803e2c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f963c000f00 tx=0x7f963c006a30 comp rx=0 tx=0).stop 2024-08-19T10:39:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.925+0000 7f9651ded640 10 monclient: shutdown 2024-08-19T10:39:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.925+0000 7f9651ded640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.925+0000 7f9651ded640 1 -- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f964c1396a0 msgr2=0x7f964c134e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.925+0000 7f9651ded640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f964c1396a0 0x7f964c134e20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f964000e870 tx=0x7f964000ec70 comp rx=0 tx=0).stop 2024-08-19T10:39:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.929+0000 7f9651ded640 1 -- 172.21.15.190:0/3026968407 shutdown_connections 2024-08-19T10:39:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.929+0000 7f9651ded640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f963803be00 0x7f963803e2c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.929+0000 7f9651ded640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f964c139fc0 0x7f964c09ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.929+0000 7f9651ded640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f964c1396a0 0x7f964c134e20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.929+0000 7f9651ded640 1 --2- 172.21.15.190:0/3026968407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f964c0a6e20 0x7f964c1348e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.929+0000 7f9651ded640 1 -- 172.21.15.190:0/3026968407 >> 172.21.15.190:0/3026968407 conn(0x7f964c0abee0 msgr2=0x7f964c0ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.933+0000 7f9651ded640 1 -- 172.21.15.190:0/3026968407 shutdown_connections 2024-08-19T10:39:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:38.933+0000 7f9651ded640 1 -- 172.21.15.190:0/3026968407 wait complete. 2024-08-19T10:39:38.965 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_config_value foo min_size 1 2024-08-19T10:39:38.965 INFO:tasks.workunit.client.0.smithi190.stderr:+ local pool_name config_opt expected_val val 2024-08-19T10:39:38.965 INFO:tasks.workunit.client.0.smithi190.stderr:+ pool_name=foo 2024-08-19T10:39:38.966 INFO:tasks.workunit.client.0.smithi190.stderr:+ config_opt=min_size 2024-08-19T10:39:38.966 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_val=1 2024-08-19T10:39:38.966 INFO:tasks.workunit.client.0.smithi190.stderr:++ get_config_value_or_die foo min_size 2024-08-19T10:39:38.966 INFO:tasks.workunit.client.0.smithi190.stderr:++ local pool_name config_opt raw val 2024-08-19T10:39:38.966 INFO:tasks.workunit.client.0.smithi190.stderr:++ pool_name=foo 2024-08-19T10:39:38.966 INFO:tasks.workunit.client.0.smithi190.stderr:++ config_opt=min_size 2024-08-19T10:39:38.966 INFO:tasks.workunit.client.0.smithi190.stderr:+++ ceph osd pool get foo min_size 2024-08-19T10:39:39.390 INFO:tasks.workunit.client.0.smithi190.stderr:++ raw='min_size: 1' 2024-08-19T10:39:39.390 INFO:tasks.workunit.client.0.smithi190.stderr:++ [[ 0 -ne 0 ]] 2024-08-19T10:39:39.390 INFO:tasks.workunit.client.0.smithi190.stderr:+++ sed -e 's/[{} "]//g' 2024-08-19T10:39:39.391 INFO:tasks.workunit.client.0.smithi190.stderr:+++ echo min_size: 1 2024-08-19T10:39:39.391 INFO:tasks.workunit.client.0.smithi190.stderr:++ raw=min_size:1 2024-08-19T10:39:39.392 INFO:tasks.workunit.client.0.smithi190.stderr:+++ echo min_size:1 2024-08-19T10:39:39.392 INFO:tasks.workunit.client.0.smithi190.stderr:+++ cut -f2 -d: 2024-08-19T10:39:39.393 INFO:tasks.workunit.client.0.smithi190.stderr:++ val=1 2024-08-19T10:39:39.394 INFO:tasks.workunit.client.0.smithi190.stderr:++ echo 1 2024-08-19T10:39:39.394 INFO:tasks.workunit.client.0.smithi190.stderr:++ return 0 2024-08-19T10:39:39.394 INFO:tasks.workunit.client.0.smithi190.stderr:+ val=1 2024-08-19T10:39:39.394 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 1 != \1 ]] 2024-08-19T10:39:39.394 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set foo size 4 2024-08-19T10:39:39.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient: get_monmap_and_config 2024-08-19T10:39:39.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient: build_initial_monmap 2024-08-19T10:39:39.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient: monmap: 2024-08-19T10:39:39.515 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:39.515 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:39.516256+0000 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:39.516256+0000 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:39.516 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:39.517 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:39.517 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:39.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 1 Processor -- start 2024-08-19T10:39:39.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 1 -- start start 2024-08-19T10:39:39.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient: init 2024-08-19T10:39:39.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:39.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient: _add_conns ranks=[3,8,9,2,10,4,7,0,6,1,5] 2024-08-19T10:39:39.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9800a68e0 0x7fb9800a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:39.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient(hunting): picked mon.noname-g con 0x7fb9800a68e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:39.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb9800a6cc0 0x7fb9800a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient(hunting): picked mon.noname-e con 0x7fb9800a6cc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb9801396a0 0x7fb980139af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient(hunting): picked mon.noname-a con 0x7fb9801396a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fb9800a86c0 con 0x7fb9801396a0 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb9800a8410 con 0x7fb9800a68e0 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb9800a63a0 con 0x7fb9800a6cc0 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb988029640 10 monclient(hunting): authenticate will time out at 1025.087045s 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9800a68e0 0x7fb9800a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:39.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9800a68e0 0x7fb9800a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47410/0 (socket says 172.21.15.190:47410) 2024-08-19T10:39:39.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 1 -- 172.21.15.190:0/2820124238 learned_addr learned my addr 172.21.15.190:0/2820124238 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:39.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 10 monclient(hunting): get_auth_request con 0x7fb9800a68e0 auth_method 0 2024-08-19T10:39:39.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:39.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:39.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:39.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:39.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:39.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:39.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb9865c6640 1 --2- 172.21.15.190:0/2820124238 >> v2:172.21.15.89:3300/0 conn(0x7fb9801396a0 0x7fb980139af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:39.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb9855c4640 1 --2- 172.21.15.190:0/2820124238 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb9800a6cc0 0x7fb9800a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:39.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 10 monclient(hunting): handle_auth_done global_id 4557 payload 274 2024-08-19T10:39:39.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 1 -- 172.21.15.190:0/2820124238 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb9800a6cc0 msgr2=0x7fb9800a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:39.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 1 --2- 172.21.15.190:0/2820124238 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb9800a6cc0 0x7fb9800a70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:39.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 1 -- 172.21.15.190:0/2820124238 >> v2:172.21.15.89:3300/0 conn(0x7fb9801396a0 msgr2=0x7fb980139af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:39.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 1 --2- 172.21.15.190:0/2820124238 >> v2:172.21.15.89:3300/0 conn(0x7fb9801396a0 0x7fb980139af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:39.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 10 monclient: _finish_hunting 0 2024-08-19T10:39:39.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 1 monclient: found mon.noname-g 2024-08-19T10:39:39.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:39.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:39.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb985dc5640 1 -- 172.21.15.190:0/2820124238 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9800a4680 con 0x7fb9800a68e0 2024-08-19T10:39:39.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb9865c6640 10 monclient: get_auth_request con 0x7fb9801396a0 auth_method 0 2024-08-19T10:39:39.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.513+0000 7fb9865c6640 1 --2- 172.21.15.190:0/2820124238 >> v2:172.21.15.89:3300/0 conn(0x7fb9801396a0 0x7fb980139af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:39:39.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.517+0000 7fb985dc5640 1 -- 172.21.15.190:0/2820124238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9800a68e0 msgr2=0x7fb9800a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:39.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.517+0000 7fb985dc5640 1 --2- 172.21.15.190:0/2820124238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9800a68e0 0x7fb9800a5760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009870 tx=0x7fb97c01c4b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=510e403b38fe1bcb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:39.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.517+0000 7fb984dc3640 1 -- 172.21.15.190:0/2820124238 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb97c0ab860 con 0x7fb9800a68e0 2024-08-19T10:39:39.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:39.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:39.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 10 monclient: dump: 2024-08-19T10:39:39.527 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:39.527 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:39.527 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:39.527 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:39.527 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:39.527 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:39.527 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:39.527 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 10 monclient: _finish_auth 0 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 10 monclient: _check_auth_tickets 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 1 -- 172.21.15.190:0/2820124238 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c0b0070 con 0x7fb9800a68e0 2024-08-19T10:39:39.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 1 -- 172.21.15.190:0/2820124238 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb97c0aa430 con 0x7fb9800a68e0 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 5 monclient: authenticate success, global_id 4557 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 4 set_mon_vals no callback set 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb984dc3640 10 monclient: dump: 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:39.529 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient: shutdown 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:39.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 -- 172.21.15.190:0/2820124238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9800a68e0 msgr2=0x7fb9800a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 --2- 172.21.15.190:0/2820124238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9800a68e0 0x7fb9800a5760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009870 tx=0x7fb97c01c4b0 comp rx=0 tx=0).stop 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 -- 172.21.15.190:0/2820124238 shutdown_connections 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 --2- 172.21.15.190:0/2820124238 >> v2:172.21.15.89:3300/0 conn(0x7fb9801396a0 0x7fb980139af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 --2- 172.21.15.190:0/2820124238 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb9800a6cc0 0x7fb9800a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 --2- 172.21.15.190:0/2820124238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9800a68e0 0x7fb9800a5760 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 -- 172.21.15.190:0/2820124238 >> 172.21.15.190:0/2820124238 conn(0x7fb9800abee0 msgr2=0x7fb9800ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 -- 172.21.15.190:0/2820124238 shutdown_connections 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 -- 172.21.15.190:0/2820124238 wait complete. 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient: build_initial_monmap 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient: monmap: 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:39.531 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:39.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 Processor -- start 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 -- start start 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient: init 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient: _add_conns ranks=[4,7,6,2,0,5,3,1,8] 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9800a68e0 0x7fb980133a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient(hunting): picked mon.noname-c con 0x7fb9800a68e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb9800a6cc0 0x7fb980133fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient(hunting): picked mon.noname-i con 0x7fb9800a6cc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb9801396a0 0x7fb98009ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient(hunting): picked mon.noname-d con 0x7fb9801396a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb9800a5ca0 con 0x7fb9800a68e0 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:39.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb9800a75e0 con 0x7fb9801396a0 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb98013f7b0 con 0x7fb9800a6cc0 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb988029640 10 monclient(hunting): authenticate will time out at 1025.094381s 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9855c4640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb9800a6cc0 msgr2=0x7fb980133fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9855c4640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb9800a6cc0 0x7fb980133fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb985dc5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9800a68e0 0x7fb980133a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb9801396a0 0x7fb98009ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb985dc5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9800a68e0 0x7fb980133a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:50908/0 (socket says 172.21.15.190:50908) 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb985dc5640 1 -- 172.21.15.190:0/418276799 learned_addr learned my addr 172.21.15.190:0/418276799 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb985dc5640 10 monclient(hunting): get_auth_request con 0x7fb9800a68e0 auth_method 0 2024-08-19T10:39:39.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb985dc5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:39.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb985dc5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:39.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb985dc5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:39.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 10 monclient(hunting): get_auth_request con 0x7fb9801396a0 auth_method 0 2024-08-19T10:39:39.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:39.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:39.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:39.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb985dc5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:39.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb985dc5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:39.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb985dc5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:39.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 10 monclient(hunting): handle_auth_done global_id 4524 payload 824 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 1 -- 172.21.15.190:0/418276799 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb9800a6cc0 msgr2=0x7fb980133fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb9800a6cc0 0x7fb980133fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 1 -- 172.21.15.190:0/418276799 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9800a68e0 msgr2=0x7fb980133a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9800a68e0 0x7fb980133a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 10 monclient: _finish_hunting 0 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 1 monclient: found mon.noname-d 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:39.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.521+0000 7fb9865c6640 1 -- 172.21.15.190:0/418276799 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb980139c10 con 0x7fb9801396a0 2024-08-19T10:39:39.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9865c6640 1 -- 172.21.15.190:0/418276799 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb9801396a0 msgr2=0x7fb98009ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:39.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9865c6640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb9801396a0 0x7fb98009ca60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb97800cc90 tx=0x7fb97800a650 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:39.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 1 -- 172.21.15.190:0/418276799 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb97800fb00 con 0x7fb9801396a0 2024-08-19T10:39:39.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:39.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:39.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 10 monclient: dump: 2024-08-19T10:39:39.539 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:39.539 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:39.539 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:39.539 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:39.539 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:39.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb988029640 5 monclient: authenticate success, global_id 4524 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb988029640 10 monclient: _renew_subs 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb988029640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb988029640 1 -- 172.21.15.190:0/418276799 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98009ee50 con 0x7fb9801396a0 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 1 -- 172.21.15.190:0/418276799 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97800fca0 con 0x7fb9801396a0 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 1 -- 172.21.15.190:0/418276799 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb97800fe50 con 0x7fb9801396a0 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:39.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 10 monclient: dump: 2024-08-19T10:39:39.542 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:39.542 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:39.542 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:39.542 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:39.542 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:39.542 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:39.542 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:39.542 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:39.542 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:39.542 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:39.543 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb977fff640 4 set_mon_vals no callback set 2024-08-19T10:39:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb988029640 10 monclient: _renew_subs 2024-08-19T10:39:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb988029640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb988029640 1 -- 172.21.15.190:0/418276799 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98013ef60 con 0x7fb9801396a0 2024-08-19T10:39:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb988029640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb988029640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb988029640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb988029640 1 -- 172.21.15.190:0/418276799 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9800a70a0 con 0x7fb9801396a0 2024-08-19T10:39:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 1 -- 172.21.15.190:0/418276799 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb97800f690 con 0x7fb9801396a0 2024-08-19T10:39:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.525+0000 7fb9767fc640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb96403bfa0 0x7fb96403e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.529+0000 7fb9767fc640 1 -- 172.21.15.190:0/418276799 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(36..36 src has 1..36) ==== 5474+0+0 (secure 0 0 0) 0x7fb97804fc20 con 0x7fb9801396a0 2024-08-19T10:39:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.529+0000 7fb985dc5640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb96403bfa0 0x7fb96403e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.529+0000 7fb985dc5640 10 monclient: get_auth_request con 0x7fb96403bfa0 auth_method 0 2024-08-19T10:39:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.529+0000 7fb985dc5640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb96403bfa0 0x7fb96403e460 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb97c01c2e0 tx=0x7fb97c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:39.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.541+0000 7fb9767fc640 1 -- 172.21.15.190:0/418276799 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb97804fe20 con 0x7fb9801396a0 2024-08-19T10:39:39.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.541+0000 7fb9767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:39.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.541+0000 7fb9767fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:39.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.709+0000 7fb988029640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-08-19T10:39:39.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.709+0000 7fb988029640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-08-19T10:39:39.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.709+0000 7fb988029640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:39.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.709+0000 7fb988029640 1 -- 172.21.15.190:0/418276799 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"} v 0) -- 0x7fb98006c9c0 con 0x7fb9801396a0 2024-08-19T10:39:39.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.913+0000 7fb9767fc640 1 -- 172.21.15.190:0/418276799 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}]=0 set pool 4 size to 4 v37) ==== 122+0+0 (secure 0 0 0) 0x7fb978014030 con 0x7fb9801396a0 2024-08-19T10:39:39.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.913+0000 7fb9767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-08-19T10:39:39.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:39.913+0000 7fb9767fc640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 4 2024-08-19T10:39:40.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.045+0000 7fb988029640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-08-19T10:39:40.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.045+0000 7fb988029640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-08-19T10:39:40.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.045+0000 7fb988029640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:40.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.045+0000 7fb988029640 1 -- 172.21.15.190:0/418276799 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"} v 0) -- 0x7fb9800a0120 con 0x7fb9801396a0 2024-08-19T10:39:40.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.913+0000 7fb9767fc640 1 -- 172.21.15.190:0/418276799 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}]=0 set pool 4 size to 4 v38) ==== 122+0+0 (secure 0 0 0) 0x7fb978017b40 con 0x7fb9801396a0 2024-08-19T10:39:40.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.913+0000 7fb9767fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "4"}] 2024-08-19T10:39:40.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.913+0000 7fb9767fc640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 4 2024-08-19T10:39:40.917 INFO:tasks.workunit.client.0.smithi190.stderr:set pool 4 size to 4 2024-08-19T10:39:40.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.921+0000 7fb988029640 1 -- 172.21.15.190:0/418276799 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb96403bfa0 msgr2=0x7fb96403e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:40.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.921+0000 7fb988029640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb96403bfa0 0x7fb96403e460 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb97c01c2e0 tx=0x7fb97c0a7000 comp rx=0 tx=0).stop 2024-08-19T10:39:40.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.921+0000 7fb988029640 10 monclient: shutdown 2024-08-19T10:39:40.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.921+0000 7fb988029640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:40.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.921+0000 7fb988029640 1 -- 172.21.15.190:0/418276799 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb9801396a0 msgr2=0x7fb98009ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:40.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.921+0000 7fb988029640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb9801396a0 0x7fb98009ca60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb97800cc90 tx=0x7fb97800a650 comp rx=0 tx=0).stop 2024-08-19T10:39:40.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.925+0000 7fb988029640 1 -- 172.21.15.190:0/418276799 shutdown_connections 2024-08-19T10:39:40.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.925+0000 7fb988029640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb96403bfa0 0x7fb96403e460 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:40.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.925+0000 7fb988029640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb9801396a0 0x7fb98009ca60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:40.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.925+0000 7fb988029640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb9800a6cc0 0x7fb980133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:40.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.925+0000 7fb988029640 1 --2- 172.21.15.190:0/418276799 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb9800a68e0 0x7fb980133a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:40.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.925+0000 7fb988029640 1 -- 172.21.15.190:0/418276799 >> 172.21.15.190:0/418276799 conn(0x7fb9800abee0 msgr2=0x7fb9801346c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:40.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.925+0000 7fb988029640 1 -- 172.21.15.190:0/418276799 shutdown_connections 2024-08-19T10:39:40.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:40.925+0000 7fb988029640 1 -- 172.21.15.190:0/418276799 wait complete. 2024-08-19T10:39:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_config_value foo min_size 2 2024-08-19T10:39:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:+ local pool_name config_opt expected_val val 2024-08-19T10:39:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:+ pool_name=foo 2024-08-19T10:39:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:+ config_opt=min_size 2024-08-19T10:39:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_val=2 2024-08-19T10:39:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:++ get_config_value_or_die foo min_size 2024-08-19T10:39:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:++ local pool_name config_opt raw val 2024-08-19T10:39:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:++ pool_name=foo 2024-08-19T10:39:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:++ config_opt=min_size 2024-08-19T10:39:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:+++ ceph osd pool get foo min_size 2024-08-19T10:39:41.417 INFO:tasks.workunit.client.0.smithi190.stderr:++ raw='min_size: 2' 2024-08-19T10:39:41.418 INFO:tasks.workunit.client.0.smithi190.stderr:++ [[ 0 -ne 0 ]] 2024-08-19T10:39:41.418 INFO:tasks.workunit.client.0.smithi190.stderr:+++ echo min_size: 2 2024-08-19T10:39:41.418 INFO:tasks.workunit.client.0.smithi190.stderr:+++ sed -e 's/[{} "]//g' 2024-08-19T10:39:41.419 INFO:tasks.workunit.client.0.smithi190.stderr:++ raw=min_size:2 2024-08-19T10:39:41.420 INFO:tasks.workunit.client.0.smithi190.stderr:+++ echo min_size:2 2024-08-19T10:39:41.420 INFO:tasks.workunit.client.0.smithi190.stderr:+++ cut -f2 -d: 2024-08-19T10:39:41.421 INFO:tasks.workunit.client.0.smithi190.stderr:++ val=2 2024-08-19T10:39:41.421 INFO:tasks.workunit.client.0.smithi190.stderr:++ echo 2 2024-08-19T10:39:41.421 INFO:tasks.workunit.client.0.smithi190.stderr:++ return 0 2024-08-19T10:39:41.421 INFO:tasks.workunit.client.0.smithi190.stderr:+ val=2 2024-08-19T10:39:41.421 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 2 != \2 ]] 2024-08-19T10:39:41.421 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set foo size 10 2024-08-19T10:39:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient: get_monmap_and_config 2024-08-19T10:39:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient: build_initial_monmap 2024-08-19T10:39:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient: monmap: 2024-08-19T10:39:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:41.528401+0000 2024-08-19T10:39:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:41.528401+0000 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:41.529 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 1 Processor -- start 2024-08-19T10:39:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 1 -- start start 2024-08-19T10:39:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient: init 2024-08-19T10:39:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient: _add_conns ranks=[1,3,5,9,7,6,8,10,2,4,0] 2024-08-19T10:39:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fad4c0a4720 addr v1:172.21.15.190:6789/0 2024-08-19T10:39:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad4c0a3010 0x7fad4c13c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient(hunting): picked mon.noname-g con 0x7fad4c0a3010 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fad4c0a2950 0x7fad4c13ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient(hunting): picked mon.noname-h con 0x7fad4c0a2950 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad4c09cc70 con 0x7fad4c0a4720 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fad4c0a2740 con 0x7fad4c0a3010 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad51bb9640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fad4c0a4720 legacy=0x7fad4c0a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fad4c0a2e00 con 0x7fad4c0a2950 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad53e1d640 10 monclient(hunting): authenticate will time out at 1027.099039s 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad513b8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad4c0a3010 0x7fad4c13c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad513b8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad4c0a3010 0x7fad4c13c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47426/0 (socket says 172.21.15.190:47426) 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fad4c0a2950 0x7fad4c13ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad513b8640 1 -- 172.21.15.190:0/19620635 learned_addr learned my addr 172.21.15.190:0/19620635 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad513b8640 10 monclient(hunting): get_auth_request con 0x7fad4c0a3010 auth_method 0 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad513b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad513b8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad513b8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 10 monclient(hunting): get_auth_request con 0x7fad4c0a2950 auth_method 0 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad513b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad513b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad513b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 10 monclient(hunting): handle_auth_done global_id 4622 payload 274 2024-08-19T10:39:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 1 -- 172.21.15.190:0/19620635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad4c0a3010 msgr2=0x7fad4c13c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 1 --2- 172.21.15.190:0/19620635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad4c0a3010 0x7fad4c13c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 1 -- 172.21.15.190:0/19620635 >> v1:172.21.15.190:6789/0 conn(0x7fad4c0a4720 legacy=0x7fad4c0a4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 10 monclient: _finish_hunting 0 2024-08-19T10:39:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 1 monclient: found mon.noname-h 2024-08-19T10:39:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad523ba640 1 -- 172.21.15.190:0/19620635 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c0a6210 con 0x7fad4c0a2950 2024-08-19T10:39:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.525+0000 7fad513b8640 1 --2- 172.21.15.190:0/19620635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad4c0a3010 0x7fad4c13c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad523ba640 1 -- 172.21.15.190:0/19620635 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fad4c0a2950 msgr2=0x7fad4c13ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad523ba640 1 --2- 172.21.15.190:0/19620635 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fad4c0a2950 0x7fad4c13ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad44009dc0 tx=0x7fad4401c850 comp rx=0 tx=0).ready entity=mon.5 client_cookie=3be4003a7c06c124 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 1 -- 172.21.15.190:0/19620635 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fad440ac070 con 0x7fad4c0a2950 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 10 monclient: dump: 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 10 monclient: _finish_auth 0 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 10 monclient: _check_auth_tickets 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 1 -- 172.21.15.190:0/19620635 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad440a68b0 con 0x7fad4c0a2950 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad53e1d640 5 monclient: authenticate success, global_id 4622 2024-08-19T10:39:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad53e1d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad53e1d640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad53e1d640 4 set_mon_vals no callback set 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 1 -- 172.21.15.190:0/19620635 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fad440a7420 con 0x7fad4c0a2950 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.529+0000 7fad50bb7640 10 monclient: dump: 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient: shutdown 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 -- 172.21.15.190:0/19620635 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fad4c0a2950 msgr2=0x7fad4c13ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 --2- 172.21.15.190:0/19620635 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fad4c0a2950 0x7fad4c13ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad44009dc0 tx=0x7fad4401c850 comp rx=0 tx=0).stop 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 -- 172.21.15.190:0/19620635 shutdown_connections 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 --2- 172.21.15.190:0/19620635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad4c0a3010 0x7fad4c13c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 --2- 172.21.15.190:0/19620635 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fad4c0a2950 0x7fad4c13ee80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 -- 172.21.15.190:0/19620635 >> 172.21.15.190:0/19620635 conn(0x7fad4c0a9310 msgr2=0x7fad4c0a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 -- 172.21.15.190:0/19620635 shutdown_connections 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 -- 172.21.15.190:0/19620635 wait complete. 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient: build_initial_monmap 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient: monmap: 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:41.543 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 Processor -- start 2024-08-19T10:39:41.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 -- start start 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient: init 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient: _add_conns ranks=[1,6,4,0,7,8,3,5,2] 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad4c0a2950 0x7fad4c1d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient(hunting): picked mon.noname-f con 0x7fad4c0a2950 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad4c0a4720 0x7fad4c13db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient(hunting): picked mon.noname-d con 0x7fad4c0a4720 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad4c13e0c0 0x7fad4c0a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient(hunting): picked mon.noname-c con 0x7fad4c13e0c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c141d60 con 0x7fad4c0a2950 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fad4c141be0 con 0x7fad4c13e0c0 2024-08-19T10:39:41.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fad4c141ee0 con 0x7fad4c0a4720 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad53e1d640 10 monclient(hunting): authenticate will time out at 1027.105987s 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad51bb9640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad4c0a2950 msgr2=0x7fad4c1d0ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad51bb9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad4c0a2950 0x7fad4c1d0ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad4c13e0c0 0x7fad4c0a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad513b8640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad4c0a4720 0x7fad4c13db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad4c13e0c0 0x7fad4c0a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:50932/0 (socket says 172.21.15.190:50932) 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 1 -- 172.21.15.190:0/4235815465 learned_addr learned my addr 172.21.15.190:0/4235815465 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 10 monclient(hunting): get_auth_request con 0x7fad4c13e0c0 auth_method 0 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:41.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad513b8640 10 monclient(hunting): get_auth_request con 0x7fad4c0a4720 auth_method 0 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad513b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad513b8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad513b8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad513b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad513b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad513b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 10 monclient(hunting): handle_auth_done global_id 4684 payload 824 2024-08-19T10:39:41.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 1 -- 172.21.15.190:0/4235815465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad4c0a4720 msgr2=0x7fad4c13db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:41.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad4c0a4720 0x7fad4c13db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:41.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 1 -- 172.21.15.190:0/4235815465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad4c0a2950 msgr2=0x7fad4c1d0ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:41.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad4c0a2950 0x7fad4c1d0ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:41.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 10 monclient: _finish_hunting 0 2024-08-19T10:39:41.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 1 monclient: found mon.noname-c 2024-08-19T10:39:41.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:41.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:41.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.533+0000 7fad523ba640 1 -- 172.21.15.190:0/4235815465 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c12a630 con 0x7fad4c13e0c0 2024-08-19T10:39:41.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad523ba640 1 -- 172.21.15.190:0/4235815465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad4c13e0c0 msgr2=0x7fad4c0a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:41.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad523ba640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad4c13e0c0 0x7fad4c0a0210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad4401c680 tx=0x7fad440195f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:41.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 1 -- 172.21.15.190:0/4235815465 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fad440ac070 con 0x7fad4c13e0c0 2024-08-19T10:39:41.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:41.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:41.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 10 monclient: dump: 2024-08-19T10:39:41.550 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:41.551 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 1 -- 172.21.15.190:0/4235815465 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad44003e10 con 0x7fad4c13e0c0 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 1 -- 172.21.15.190:0/4235815465 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fad440044b0 con 0x7fad4c13e0c0 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 10 monclient: dump: 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:41.552 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:41.553 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad437fe640 4 set_mon_vals no callback set 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad53e1d640 5 monclient: authenticate success, global_id 4684 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad53e1d640 10 monclient: _renew_subs 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad53e1d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad53e1d640 1 -- 172.21.15.190:0/4235815465 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c128fb0 con 0x7fad4c13e0c0 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad53e1d640 10 monclient: _renew_subs 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad53e1d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad53e1d640 1 -- 172.21.15.190:0/4235815465 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c0a1cb0 con 0x7fad4c13e0c0 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad53e1d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad53e1d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad53e1d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:41.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad53e1d640 1 -- 172.21.15.190:0/4235815465 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c0008e0 con 0x7fad4c13e0c0 2024-08-19T10:39:41.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 1 -- 172.21.15.190:0/4235815465 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fad440af070 con 0x7fad4c13e0c0 2024-08-19T10:39:41.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fad3003be00 0x7fad3003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:41.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad427fc640 1 -- 172.21.15.190:0/4235815465 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(38..38 src has 1..38) ==== 5474+0+0 (secure 0 0 0) 0x7fad440b7070 con 0x7fad4c13e0c0 2024-08-19T10:39:41.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad51bb9640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fad3003be00 0x7fad3003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:41.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.537+0000 7fad51bb9640 10 monclient: get_auth_request con 0x7fad3003be00 auth_method 0 2024-08-19T10:39:41.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.541+0000 7fad51bb9640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fad3003be00 0x7fad3003e2c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fad48003990 tx=0x7fad48006cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:41.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.549+0000 7fad427fc640 1 -- 172.21.15.190:0/4235815465 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fad44016560 con 0x7fad4c13e0c0 2024-08-19T10:39:41.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.549+0000 7fad427fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:41.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.549+0000 7fad427fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:41.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.721+0000 7fad53e1d640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-08-19T10:39:41.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.721+0000 7fad53e1d640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-08-19T10:39:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.721+0000 7fad53e1d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:41.721+0000 7fad53e1d640 1 -- 172.21.15.190:0/4235815465 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"} v 0) -- 0x7fad4c13e4c0 con 0x7fad4c13e0c0 2024-08-19T10:39:42.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:42.249+0000 7fad427fc640 1 -- 172.21.15.190:0/4235815465 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}]=0 set pool 4 size to 10 v39) ==== 124+0+0 (secure 0 0 0) 0x7fad440b9370 con 0x7fad4c13e0c0 2024-08-19T10:39:42.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:42.249+0000 7fad427fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-08-19T10:39:42.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:42.249+0000 7fad427fc640 10 monclient: _finish_command 2 = system:0 set pool 4 size to 10 2024-08-19T10:39:42.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:42.405+0000 7fad53e1d640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-08-19T10:39:42.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:42.405+0000 7fad53e1d640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-08-19T10:39:42.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:42.405+0000 7fad53e1d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:42.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:42.405+0000 7fad53e1d640 1 -- 172.21.15.190:0/4235815465 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"} v 0) -- 0x7fad4c0a4b00 con 0x7fad4c13e0c0 2024-08-19T10:39:43.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.629+0000 7fad427fc640 1 -- 172.21.15.190:0/4235815465 <== mon.4 v2:172.21.15.89:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}]=0 set pool 4 size to 10 v40) ==== 124+0+0 (secure 0 0 0) 0x7fad440b1e10 con 0x7fad4c13e0c0 2024-08-19T10:39:43.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.629+0000 7fad427fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "10"}] 2024-08-19T10:39:43.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.629+0000 7fad427fc640 10 monclient: _finish_command 3 = system:0 set pool 4 size to 10 2024-08-19T10:39:43.635 INFO:tasks.workunit.client.0.smithi190.stderr:set pool 4 size to 10 2024-08-19T10:39:43.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 -- 172.21.15.190:0/4235815465 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fad3003be00 msgr2=0x7fad3003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:43.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fad3003be00 0x7fad3003e2c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fad48003990 tx=0x7fad48006cf0 comp rx=0 tx=0).stop 2024-08-19T10:39:43.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 10 monclient: shutdown 2024-08-19T10:39:43.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:43.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 -- 172.21.15.190:0/4235815465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad4c13e0c0 msgr2=0x7fad4c0a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:43.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad4c13e0c0 0x7fad4c0a0210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad4401c680 tx=0x7fad440195f0 comp rx=0 tx=0).stop 2024-08-19T10:39:43.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 -- 172.21.15.190:0/4235815465 shutdown_connections 2024-08-19T10:39:43.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fad3003be00 0x7fad3003e2c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:43.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad4c0a4720 0x7fad4c13db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:43.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fad4c13e0c0 0x7fad4c0a0210 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:43.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 --2- 172.21.15.190:0/4235815465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad4c0a2950 0x7fad4c1d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:43.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 -- 172.21.15.190:0/4235815465 >> 172.21.15.190:0/4235815465 conn(0x7fad4c0a9310 msgr2=0x7fad4c070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:43.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 -- 172.21.15.190:0/4235815465 shutdown_connections 2024-08-19T10:39:43.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:43.649+0000 7fad53e1d640 1 -- 172.21.15.190:0/4235815465 wait complete. 2024-08-19T10:39:43.692 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_config_value foo min_size 5 2024-08-19T10:39:43.692 INFO:tasks.workunit.client.0.smithi190.stderr:+ local pool_name config_opt expected_val val 2024-08-19T10:39:43.692 INFO:tasks.workunit.client.0.smithi190.stderr:+ pool_name=foo 2024-08-19T10:39:43.692 INFO:tasks.workunit.client.0.smithi190.stderr:+ config_opt=min_size 2024-08-19T10:39:43.692 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_val=5 2024-08-19T10:39:43.692 INFO:tasks.workunit.client.0.smithi190.stderr:++ get_config_value_or_die foo min_size 2024-08-19T10:39:43.692 INFO:tasks.workunit.client.0.smithi190.stderr:++ local pool_name config_opt raw val 2024-08-19T10:39:43.692 INFO:tasks.workunit.client.0.smithi190.stderr:++ pool_name=foo 2024-08-19T10:39:43.692 INFO:tasks.workunit.client.0.smithi190.stderr:++ config_opt=min_size 2024-08-19T10:39:43.692 INFO:tasks.workunit.client.0.smithi190.stderr:+++ ceph osd pool get foo min_size 2024-08-19T10:39:44.141 INFO:tasks.workunit.client.0.smithi190.stderr:++ raw='min_size: 5' 2024-08-19T10:39:44.142 INFO:tasks.workunit.client.0.smithi190.stderr:++ [[ 0 -ne 0 ]] 2024-08-19T10:39:44.142 INFO:tasks.workunit.client.0.smithi190.stderr:+++ sed -e 's/[{} "]//g' 2024-08-19T10:39:44.142 INFO:tasks.workunit.client.0.smithi190.stderr:+++ echo min_size: 5 2024-08-19T10:39:44.144 INFO:tasks.workunit.client.0.smithi190.stderr:++ raw=min_size:5 2024-08-19T10:39:44.145 INFO:tasks.workunit.client.0.smithi190.stderr:+++ echo min_size:5 2024-08-19T10:39:44.145 INFO:tasks.workunit.client.0.smithi190.stderr:+++ cut -f2 -d: 2024-08-19T10:39:44.147 INFO:tasks.workunit.client.0.smithi190.stderr:++ val=5 2024-08-19T10:39:44.147 INFO:tasks.workunit.client.0.smithi190.stderr:++ echo 5 2024-08-19T10:39:44.147 INFO:tasks.workunit.client.0.smithi190.stderr:++ return 0 2024-08-19T10:39:44.147 INFO:tasks.workunit.client.0.smithi190.stderr:+ val=5 2024-08-19T10:39:44.147 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 5 != \5 ]] 2024-08-19T10:39:44.148 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool set foo size 0 2024-08-19T10:39:44.148 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:44.148 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set foo size 0 2024-08-19T10:39:44.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient: get_monmap_and_config 2024-08-19T10:39:44.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient: build_initial_monmap 2024-08-19T10:39:44.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient: monmap: 2024-08-19T10:39:44.267 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:44.267 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.267 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:44.267675+0000 2024-08-19T10:39:44.267 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:44.267675+0000 2024-08-19T10:39:44.268 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:44.268 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.268 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:44.268 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:44.268 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:44.268 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:44.268 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:44.268 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:44.269 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:44.269 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:44.269 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:44.269 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:44.269 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:44.269 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 1 Processor -- start 2024-08-19T10:39:44.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 1 -- start start 2024-08-19T10:39:44.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient: init 2024-08-19T10:39:44.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:44.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient: _add_conns ranks=[6,1,0,10,3,9,7,4,8,2,5] 2024-08-19T10:39:44.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa6800a3d80 0x7fa68009cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient(hunting): picked mon.noname-d con 0x7fa6800a3d80 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:44.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa6800a4160 addr v1:172.21.15.190:6789/0 2024-08-19T10:39:44.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa68013c920 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:44.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa680069c10 con 0x7fa68013c920 2024-08-19T10:39:44.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa68009c5f0 con 0x7fa6800a4160 2024-08-19T10:39:44.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa68006d760 con 0x7fa6800a3d80 2024-08-19T10:39:44.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:44.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa684c98640 10 monclient(hunting): authenticate will time out at 1029.837518s 2024-08-19T10:39:44.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67dd9b640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fa6800a4160 legacy=0x7fa6800a4540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:39:44.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67ed9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa68013c920 0x7fa68013ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:37498/0 (socket says 172.21.15.190:37498) 2024-08-19T10:39:44.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67ed9d640 1 -- 172.21.15.190:0/1920825140 learned_addr learned my addr 172.21.15.190:0/1920825140 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:39:44.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 1 --2- 172.21.15.190:0/1920825140 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa6800a3d80 0x7fa68009cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 10 monclient(hunting): get_auth_request con 0x7fa6800a3d80 auth_method 0 2024-08-19T10:39:44.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:44.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:44.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:44.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:44.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:44.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:44.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 10 monclient(hunting): handle_auth_done global_id 4542 payload 274 2024-08-19T10:39:44.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 1 -- 172.21.15.190:0/1920825140 >> v1:172.21.15.190:6789/0 conn(0x7fa6800a4160 legacy=0x7fa6800a4540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:44.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 1 -- 172.21.15.190:0/1920825140 >> v1:172.21.15.89:6789/0 conn(0x7fa68013c920 legacy=0x7fa68013ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:44.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:39:44.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 1 monclient: found mon.noname-d 2024-08-19T10:39:44.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:44.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:44.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.265+0000 7fa67e59c640 1 -- 172.21.15.190:0/1920825140 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6800a5cb0 con 0x7fa6800a3d80 2024-08-19T10:39:44.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67e59c640 1 -- 172.21.15.190:0/1920825140 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa6800a3d80 msgr2=0x7fa68009cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:44.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67e59c640 1 --2- 172.21.15.190:0/1920825140 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa6800a3d80 0x7fa68009cfd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa678009dc0 tx=0x7fa67801c850 comp rx=0 tx=0).ready entity=mon.6 client_cookie=6ebf98080e2fc428 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:44.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 1 -- 172.21.15.190:0/1920825140 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa6780ac070 con 0x7fa6800a3d80 2024-08-19T10:39:44.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:44.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 10 monclient: dump: 2024-08-19T10:39:44.288 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.288 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.288 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.288 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.288 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.288 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.289 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.289 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.289 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.289 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.289 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.290 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.290 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.290 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.290 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.291 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 10 monclient: _finish_auth 0 2024-08-19T10:39:44.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:39:44.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:44.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 1 -- 172.21.15.190:0/1920825140 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6780a6990 con 0x7fa6800a3d80 2024-08-19T10:39:44.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa684c98640 5 monclient: authenticate success, global_id 4542 2024-08-19T10:39:44.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:44.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 1 -- 172.21.15.190:0/1920825140 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa6780a7420 con 0x7fa6800a3d80 2024-08-19T10:39:44.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:44.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa67d59a640 10 monclient: dump: 2024-08-19T10:39:44.292 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.292 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.292 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.293 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.293 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.293 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.293 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.293 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.293 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.293 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.293 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.294 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.294 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.294 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.294 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.294 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa684c98640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:44.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.269+0000 7fa684c98640 4 set_mon_vals no callback set 2024-08-19T10:39:44.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient: shutdown 2024-08-19T10:39:44.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:44.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 -- 172.21.15.190:0/1920825140 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa6800a3d80 msgr2=0x7fa68009cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:44.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 --2- 172.21.15.190:0/1920825140 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa6800a3d80 0x7fa68009cfd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa678009dc0 tx=0x7fa67801c850 comp rx=0 tx=0).stop 2024-08-19T10:39:44.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 -- 172.21.15.190:0/1920825140 shutdown_connections 2024-08-19T10:39:44.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 --2- 172.21.15.190:0/1920825140 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa6800a3d80 0x7fa68009cfd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 -- 172.21.15.190:0/1920825140 >> 172.21.15.190:0/1920825140 conn(0x7fa6800a4e70 msgr2=0x7fa6800a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:44.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 -- 172.21.15.190:0/1920825140 shutdown_connections 2024-08-19T10:39:44.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 -- 172.21.15.190:0/1920825140 wait complete. 2024-08-19T10:39:44.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient: build_initial_monmap 2024-08-19T10:39:44.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient: monmap: 2024-08-19T10:39:44.296 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:44.296 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:44.297 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:44.297 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:44.297 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:44.297 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.297 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:44.297 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:44.297 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:44.298 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:44.298 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:44.298 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:44.298 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:44.298 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:44.298 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:44.298 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 Processor -- start 2024-08-19T10:39:44.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 -- start start 2024-08-19T10:39:44.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient: init 2024-08-19T10:39:44.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:44.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient: _add_conns ranks=[8,5,3,1,7,2,0,4,6] 2024-08-19T10:39:44.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa6800a3d80 0x7fa68009f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient(hunting): picked mon.noname-e con 0x7fa6800a3d80 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:44.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa6800a4160 0x7fa68009fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient(hunting): picked mon.noname-h con 0x7fa6800a4160 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:44.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa68013c920 0x7fa6800a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient(hunting): picked mon.noname-g con 0x7fa68013c920 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:44.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa680142dd0 con 0x7fa68013c920 2024-08-19T10:39:44.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa680142c50 con 0x7fa6800a4160 2024-08-19T10:39:44.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa680050c20 con 0x7fa6800a3d80 2024-08-19T10:39:44.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:44.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa684c98640 10 monclient(hunting): authenticate will time out at 1029.845967s 2024-08-19T10:39:44.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa68013c920 0x7fa6800a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa68013c920 0x7fa6800a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47454/0 (socket says 172.21.15.190:47454) 2024-08-19T10:39:44.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 1 -- 172.21.15.190:0/550588778 learned_addr learned my addr 172.21.15.190:0/550588778 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:44.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 10 monclient(hunting): get_auth_request con 0x7fa68013c920 auth_method 0 2024-08-19T10:39:44.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:44.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:44.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:44.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67dd9b640 1 --2- 172.21.15.190:0/550588778 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa6800a4160 0x7fa68009fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67dd9b640 10 monclient(hunting): get_auth_request con 0x7fa6800a4160 auth_method 0 2024-08-19T10:39:44.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:44.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:44.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:44.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:44.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:44.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:44.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:44.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:44.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:44.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67e59c640 1 --2- 172.21.15.190:0/550588778 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa6800a3d80 0x7fa68009f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67e59c640 10 monclient(hunting): get_auth_request con 0x7fa6800a3d80 auth_method 0 2024-08-19T10:39:44.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:44.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:44.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:44.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 10 monclient(hunting): handle_auth_done global_id 4602 payload 824 2024-08-19T10:39:44.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 1 -- 172.21.15.190:0/550588778 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa6800a3d80 msgr2=0x7fa68009f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:44.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 1 --2- 172.21.15.190:0/550588778 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa6800a3d80 0x7fa68009f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 1 -- 172.21.15.190:0/550588778 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa6800a4160 msgr2=0x7fa68009fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:44.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 1 --2- 172.21.15.190:0/550588778 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa6800a4160 0x7fa68009fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:39:44.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 1 monclient: found mon.noname-g 2024-08-19T10:39:44.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:44.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:44.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.273+0000 7fa67ed9d640 1 -- 172.21.15.190:0/550588778 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6800a0f70 con 0x7fa68013c920 2024-08-19T10:39:44.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa67ed9d640 1 -- 172.21.15.190:0/550588778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa68013c920 msgr2=0x7fa6800a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:44.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa67ed9d640 1 --2- 172.21.15.190:0/550588778 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa68013c920 0x7fa6800a02f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa67000b3c0 tx=0x7fa67000b880 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:44.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 1 -- 172.21.15.190:0/550588778 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa670015020 con 0x7fa68013c920 2024-08-19T10:39:44.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:44.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 10 monclient: dump: 2024-08-19T10:39:44.307 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.307 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.307 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.307 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.307 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.307 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.308 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.308 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.308 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.308 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.308 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.308 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.308 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.309 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.309 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.309 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 10 monclient: _finish_auth 0 2024-08-19T10:39:44.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 10 monclient: _check_auth_tickets 2024-08-19T10:39:44.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:44.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 1 -- 172.21.15.190:0/550588778 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa670011cb0 con 0x7fa68013c920 2024-08-19T10:39:44.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:44.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 1 -- 172.21.15.190:0/550588778 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa670011e60 con 0x7fa68013c920 2024-08-19T10:39:44.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:44.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 10 monclient: dump: 2024-08-19T10:39:44.310 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.311 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.311 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.311 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.311 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.311 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.311 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.311 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.312 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.312 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.312 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.312 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.312 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.312 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.313 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.313 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa67cd99640 4 set_mon_vals no callback set 2024-08-19T10:39:44.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa684c98640 5 monclient: authenticate success, global_id 4602 2024-08-19T10:39:44.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa684c98640 10 monclient: _renew_subs 2024-08-19T10:39:44.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa684c98640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:44.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa684c98640 1 -- 172.21.15.190:0/550588778 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6800a37e0 con 0x7fa68013c920 2024-08-19T10:39:44.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa684c98640 10 monclient: _renew_subs 2024-08-19T10:39:44.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa684c98640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:44.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa684c98640 1 -- 172.21.15.190:0/550588778 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa680073970 con 0x7fa68013c920 2024-08-19T10:39:44.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa684c98640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:44.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa684c98640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:44.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa684c98640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:44.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa684c98640 1 -- 172.21.15.190:0/550588778 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa68009cfd0 con 0x7fa68013c920 2024-08-19T10:39:44.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 1 -- 172.21.15.190:0/550588778 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa670018020 con 0x7fa68013c920 2024-08-19T10:39:44.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 1 --2- 172.21.15.190:0/550588778 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa66003be00 0x7fa66003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa66effd640 1 -- 172.21.15.190:0/550588778 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(40..40 src has 1..40) ==== 5474+0+0 (secure 0 0 0) 0x7fa67004e560 con 0x7fa68013c920 2024-08-19T10:39:44.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa67e59c640 1 --2- 172.21.15.190:0/550588778 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa66003be00 0x7fa66003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.277+0000 7fa67e59c640 10 monclient: get_auth_request con 0x7fa66003be00 auth_method 0 2024-08-19T10:39:44.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.281+0000 7fa67e59c640 1 --2- 172.21.15.190:0/550588778 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa66003be00 0x7fa66003e2c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa6800708f0 tx=0x7fa6780a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:44.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.281+0000 7fa66effd640 1 -- 172.21.15.190:0/550588778 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fa67001f070 con 0x7fa68013c920 2024-08-19T10:39:44.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.281+0000 7fa66effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:44.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.281+0000 7fa66effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:44.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.453+0000 7fa684c98640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-08-19T10:39:44.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.453+0000 7fa684c98640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-08-19T10:39:44.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.453+0000 7fa684c98640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:44.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.453+0000 7fa684c98640 1 -- 172.21.15.190:0/550588778 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"} v 0) -- 0x7fa680139e00 con 0x7fa68013c920 2024-08-19T10:39:44.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.453+0000 7fa66effd640 1 -- 172.21.15.190:0/550588778 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}]=-22 pool size must be between 1 and 10 v40) ==== 136+0+0 (secure 0 0 0) 0x7fa67000f2e0 con 0x7fa68013c920 2024-08-19T10:39:44.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.453+0000 7fa66effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "0"}] 2024-08-19T10:39:44.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.453+0000 7fa66effd640 10 monclient: _finish_command 2 = mon:22 pool size must be between 1 and 10 2024-08-19T10:39:44.457 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: pool size must be between 1 and 10 2024-08-19T10:39:44.461 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:44.461 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool set foo size 20 2024-08-19T10:39:44.461 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:44.462 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set foo size 20 2024-08-19T10:39:44.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient: get_monmap_and_config 2024-08-19T10:39:44.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient: build_initial_monmap 2024-08-19T10:39:44.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient: monmap: 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:44.583632+0000 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:44.583632+0000 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:44.583 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:44.584 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:44.584 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:44.584 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:44.584 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:44.584 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 1 Processor -- start 2024-08-19T10:39:44.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 1 -- start start 2024-08-19T10:39:44.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient: init 2024-08-19T10:39:44.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:44.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient: _add_conns ranks=[4,2,0,9,3,1,6,7,8,5,10] 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8c1396a0 0x7fea8c139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient(hunting): picked mon.noname-c con 0x7fea8c1396a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea8c139fc0 0x7fea8c0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient(hunting): picked mon.noname-b con 0x7fea8c139fc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fea8c0a6e20 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea8c0a83e0 con 0x7fea8c0a6e20 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fea8c0a4f20 con 0x7fea8c139fc0 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fea8c0a2f30 con 0x7fea8c1396a0 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:44.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea91b86640 10 monclient(hunting): authenticate will time out at 1030.153651s 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8b7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8c1396a0 0x7fea8c139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea8c139fc0 0x7fea8c0a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8bfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fea8c0a6e20 0x7fea8c0a7220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:37512/0 (socket says 172.21.15.190:37512) 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8bfff640 1 -- 172.21.15.190:0/234486282 learned_addr learned my addr 172.21.15.190:0/234486282 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 10 monclient(hunting): get_auth_request con 0x7fea8c139fc0 auth_method 0 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8b7fe640 10 monclient(hunting): get_auth_request con 0x7fea8c1396a0 auth_method 0 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:44.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 10 monclient(hunting): handle_auth_done global_id 4565 payload 274 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 1 -- 172.21.15.190:0/234486282 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8c1396a0 msgr2=0x7fea8c139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 1 --2- 172.21.15.190:0/234486282 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8c1396a0 0x7fea8c139a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 1 -- 172.21.15.190:0/234486282 >> v1:172.21.15.89:6789/0 conn(0x7fea8c0a6e20 legacy=0x7fea8c0a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 10 monclient: _finish_hunting 0 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 1 monclient: found mon.noname-b 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:44.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:44.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.581+0000 7fea8affd640 1 -- 172.21.15.190:0/234486282 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c0a35e0 con 0x7fea8c139fc0 2024-08-19T10:39:44.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8affd640 1 -- 172.21.15.190:0/234486282 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea8c139fc0 msgr2=0x7fea8c0a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8affd640 1 --2- 172.21.15.190:0/234486282 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea8c139fc0 0x7fea8c0a68e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea780097f0 tx=0x7fea7801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b77820d48189044f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 1 -- 172.21.15.190:0/234486282 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fea780b0070 con 0x7fea8c139fc0 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 10 monclient: dump: 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.592 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.593 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.593 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.593 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.593 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.593 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.593 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.593 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.593 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.593 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 1 -- 172.21.15.190:0/234486282 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea780aa810 con 0x7fea8c139fc0 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 1 -- 172.21.15.190:0/234486282 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fea780a9300 con 0x7fea8c139fc0 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea8a7fc640 10 monclient: dump: 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.594 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea91b86640 5 monclient: authenticate success, global_id 4565 2024-08-19T10:39:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea91b86640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.585+0000 7fea91b86640 4 set_mon_vals no callback set 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient: shutdown 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 -- 172.21.15.190:0/234486282 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea8c139fc0 msgr2=0x7fea8c0a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 --2- 172.21.15.190:0/234486282 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea8c139fc0 0x7fea8c0a68e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea780097f0 tx=0x7fea7801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 -- 172.21.15.190:0/234486282 shutdown_connections 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 --2- 172.21.15.190:0/234486282 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fea8c139fc0 0x7fea8c0a68e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 --2- 172.21.15.190:0/234486282 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea8c1396a0 0x7fea8c139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 -- 172.21.15.190:0/234486282 >> 172.21.15.190:0/234486282 conn(0x7fea8c0abee0 msgr2=0x7fea8c0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 -- 172.21.15.190:0/234486282 shutdown_connections 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 -- 172.21.15.190:0/234486282 wait complete. 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient: build_initial_monmap 2024-08-19T10:39:44.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient: monmap: 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:44.598 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:44.598 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 Processor -- start 2024-08-19T10:39:44.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 -- start start 2024-08-19T10:39:44.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient: init 2024-08-19T10:39:44.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:44.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient: _add_conns ranks=[7,0,8,6,1,3,2,5,4] 2024-08-19T10:39:44.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea8c0a6e20 0x7fea8c1d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient(hunting): picked mon.noname-i con 0x7fea8c0a6e20 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:44.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8c1396a0 0x7fea8c138900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient(hunting): picked mon.noname-a con 0x7fea8c1396a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:44.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea8c139fc0 0x7fea8c13da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient(hunting): picked mon.noname-e con 0x7fea8c139fc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:44.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c13a3a0 con 0x7fea8c1396a0 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fea8c0a7930 con 0x7fea8c0a6e20 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fea8c13f540 con 0x7fea8c139fc0 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8b7fe640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea8c0a6e20 msgr2=0x7fea8c1d13b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea91b86640 10 monclient(hunting): authenticate will time out at 1030.162382s 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8b7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea8c0a6e20 0x7fea8c1d13b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8c1396a0 0x7fea8c138900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea8c139fc0 0x7fea8c13da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8c1396a0 0x7fea8c138900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:37290/0 (socket says 172.21.15.190:37290) 2024-08-19T10:39:44.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 1 -- 172.21.15.190:0/2324545641 learned_addr learned my addr 172.21.15.190:0/2324545641 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 10 monclient(hunting): get_auth_request con 0x7fea8c1396a0 auth_method 0 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8bfff640 10 monclient(hunting): get_auth_request con 0x7fea8c139fc0 auth_method 0 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:44.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 10 monclient(hunting): handle_auth_done global_id 4734 payload 824 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 1 -- 172.21.15.190:0/2324545641 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea8c139fc0 msgr2=0x7fea8c13da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 1 --2- 172.21.15.190:0/2324545641 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea8c139fc0 0x7fea8c13da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 1 -- 172.21.15.190:0/2324545641 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea8c0a6e20 msgr2=0x7fea8c1d13b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 1 --2- 172.21.15.190:0/2324545641 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea8c0a6e20 0x7fea8c1d13b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 10 monclient: _finish_hunting 0 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 1 monclient: found mon.noname-a 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.589+0000 7fea8affd640 1 -- 172.21.15.190:0/2324545641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c122530 con 0x7fea8c1396a0 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea8affd640 1 -- 172.21.15.190:0/2324545641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8c1396a0 msgr2=0x7fea8c138900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea8affd640 1 --2- 172.21.15.190:0/2324545641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8c1396a0 0x7fea8c138900 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fea7801c1d0 tx=0x7fea78019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:44.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 1 -- 172.21.15.190:0/2324545641 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fea780b9020 con 0x7fea8c1396a0 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 10 monclient: dump: 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.606 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 10 monclient: _finish_auth 0 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 1 -- 172.21.15.190:0/2324545641 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea780044b0 con 0x7fea8c1396a0 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 1 -- 172.21.15.190:0/2324545641 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fea780b0040 con 0x7fea8c1396a0 2024-08-19T10:39:44.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea91b86640 5 monclient: authenticate success, global_id 4734 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea91b86640 10 monclient: _renew_subs 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea91b86640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea91b86640 1 -- 172.21.15.190:0/2324545641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8c123290 con 0x7fea8c1396a0 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 10 monclient: dump: 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.608 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.609 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.609 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.609 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.609 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.609 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.609 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.609 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.609 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.609 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.610 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea91b86640 10 monclient: _renew_subs 2024-08-19T10:39:44.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea91b86640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:44.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea91b86640 1 -- 172.21.15.190:0/2324545641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8c1229f0 con 0x7fea8c1396a0 2024-08-19T10:39:44.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea89ffb640 4 set_mon_vals no callback set 2024-08-19T10:39:44.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea91b86640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:44.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea91b86640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:44.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea91b86640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:44.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea91b86640 1 -- 172.21.15.190:0/2324545641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8c13a520 con 0x7fea8c1396a0 2024-08-19T10:39:44.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.593+0000 7fea83fff640 1 -- 172.21.15.190:0/2324545641 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fea780bc020 con 0x7fea8c1396a0 2024-08-19T10:39:44.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.597+0000 7fea83fff640 1 --2- 172.21.15.190:0/2324545641 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fea7403be00 0x7fea7403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.597+0000 7fea83fff640 1 -- 172.21.15.190:0/2324545641 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5474+0+0 (secure 0 0 0) 0x7fea780e8d90 con 0x7fea8c1396a0 2024-08-19T10:39:44.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.597+0000 7fea8b7fe640 1 --2- 172.21.15.190:0/2324545641 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fea7403be00 0x7fea7403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.597+0000 7fea8b7fe640 10 monclient: get_auth_request con 0x7fea7403be00 auth_method 0 2024-08-19T10:39:44.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.597+0000 7fea8b7fe640 1 --2- 172.21.15.190:0/2324545641 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fea7403be00 0x7fea7403e2c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0094e0 tx=0x7fea7c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:44.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.609+0000 7fea83fff640 1 -- 172.21.15.190:0/2324545641 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fea8c13a520 con 0x7fea8c1396a0 2024-08-19T10:39:44.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.609+0000 7fea83fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:44.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.609+0000 7fea83fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:44.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.781+0000 7fea91b86640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-08-19T10:39:44.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.781+0000 7fea91b86640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-08-19T10:39:44.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.781+0000 7fea91b86640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:44.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.781+0000 7fea91b86640 1 -- 172.21.15.190:0/2324545641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"} v 0) -- 0x7fea8c0a25b0 con 0x7fea8c1396a0 2024-08-19T10:39:44.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.785+0000 7fea83fff640 1 -- 172.21.15.190:0/2324545641 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}]=-22 pool size must be between 1 and 10 v40) ==== 137+0+0 (secure 0 0 0) 0x7fea8c0a25b0 con 0x7fea8c1396a0 2024-08-19T10:39:44.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.785+0000 7fea83fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "foo", "var": "size", "val": "20"}] 2024-08-19T10:39:44.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.785+0000 7fea83fff640 10 monclient: _finish_command 2 = mon:22 pool size must be between 1 and 10 2024-08-19T10:39:44.787 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: pool size must be between 1 and 10 2024-08-19T10:39:44.791 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:44.791 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool delete foo 2024-08-19T10:39:44.791 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:44.791 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete foo 2024-08-19T10:39:44.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.901+0000 7f083b020640 10 monclient: get_monmap_and_config 2024-08-19T10:39:44.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.901+0000 7f083b020640 10 monclient: build_initial_monmap 2024-08-19T10:39:44.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.901+0000 7f083b020640 10 monclient: monmap: 2024-08-19T10:39:44.905 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:44.906 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.906 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:44.906477+0000 2024-08-19T10:39:44.906 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:44.906477+0000 2024-08-19T10:39:44.906 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:44.906 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.906 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:44.906 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:44.906 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:44.906 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:44.906 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:44.906 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:44.907 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:44.907 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:44.907 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:44.907 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:44.907 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:44.908 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 1 Processor -- start 2024-08-19T10:39:44.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 1 -- start start 2024-08-19T10:39:44.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 10 monclient: init 2024-08-19T10:39:44.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:44.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 10 monclient: _add_conns ranks=[0,2,1,7,8,9,4,10,3,5,6] 2024-08-19T10:39:44.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0834135220 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:44.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083409eb70 0x7f083413e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 10 monclient(hunting): picked mon.noname-b con 0x7f083409eb70 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:44.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f083413ec80 addr v1:172.21.15.190:6789/0 2024-08-19T10:39:44.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f083409d8a0 con 0x7f0834135220 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f083409db80 con 0x7f083413ec80 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f083409cb60 con 0x7f083409eb70 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f083b020640 10 monclient(hunting): authenticate will time out at 1030.477249s 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f08395bd640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f083413ec80 legacy=0x7f0834141070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0838dbc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0834135220 0x7f0834135600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:37526/0 (socket says 172.21.15.190:37526) 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0838dbc640 1 -- 172.21.15.190:0/928806838 learned_addr learned my addr 172.21.15.190:0/928806838 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 1 --2- 172.21.15.190:0/928806838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083409eb70 0x7f083413e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 10 monclient(hunting): get_auth_request con 0x7f083409eb70 auth_method 0 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:44.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:44.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:44.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:44.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:44.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 10 monclient(hunting): handle_auth_done global_id 4574 payload 274 2024-08-19T10:39:44.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 1 -- 172.21.15.190:0/928806838 >> v1:172.21.15.190:6789/0 conn(0x7f083413ec80 legacy=0x7f0834141070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:44.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 1 -- 172.21.15.190:0/928806838 >> v1:172.21.15.89:6789/0 conn(0x7f0834135220 legacy=0x7f0834135600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:44.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:44.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 1 monclient: found mon.noname-b 2024-08-19T10:39:44.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:44.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:44.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.905+0000 7f0833fff640 1 -- 172.21.15.190:0/928806838 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083409de60 con 0x7f083409eb70 2024-08-19T10:39:44.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f0833fff640 1 -- 172.21.15.190:0/928806838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083409eb70 msgr2=0x7f083413e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:44.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f0833fff640 1 --2- 172.21.15.190:0/928806838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083409eb70 0x7f083413e740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08200097f0 tx=0x7f082001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b43ffc53d8a0b25d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:44.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 1 -- 172.21.15.190:0/928806838 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08200b0070 con 0x7f083409eb70 2024-08-19T10:39:44.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:44.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 10 monclient: dump: 2024-08-19T10:39:44.916 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.916 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.916 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.916 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.916 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.916 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.916 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.916 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.917 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.917 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.917 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.917 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.917 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.917 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.917 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.918 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:44.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:44.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:44.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f083b020640 5 monclient: authenticate success, global_id 4574 2024-08-19T10:39:44.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f083b020640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:44.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 1 -- 172.21.15.190:0/928806838 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08200aa3a0 con 0x7f083409eb70 2024-08-19T10:39:44.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:44.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 1 -- 172.21.15.190:0/928806838 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08200aaa00 con 0x7f083409eb70 2024-08-19T10:39:44.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f08337fe640 10 monclient: dump: 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.919 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f083b020640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.909+0000 7f083b020640 4 set_mon_vals no callback set 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient: shutdown 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 -- 172.21.15.190:0/928806838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083409eb70 msgr2=0x7f083413e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 --2- 172.21.15.190:0/928806838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083409eb70 0x7f083413e740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08200097f0 tx=0x7f082001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 -- 172.21.15.190:0/928806838 shutdown_connections 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 --2- 172.21.15.190:0/928806838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083409eb70 0x7f083413e740 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 -- 172.21.15.190:0/928806838 >> 172.21.15.190:0/928806838 conn(0x7f08340a4de0 msgr2=0x7f08340a51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 -- 172.21.15.190:0/928806838 shutdown_connections 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 -- 172.21.15.190:0/928806838 wait complete. 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient: build_initial_monmap 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient: monmap: 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:44.921 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.922 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:44.922 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:44.922 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:44.922 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:44.922 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:44.922 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:44.922 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:44.922 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:44.923 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:44.923 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 Processor -- start 2024-08-19T10:39:44.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 -- start start 2024-08-19T10:39:44.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient: init 2024-08-19T10:39:44.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:44.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient: _add_conns ranks=[1,0,6,7,4,2,5,8,3] 2024-08-19T10:39:44.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f083409eb70 0x7f0834070c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient(hunting): picked mon.noname-f con 0x7f083409eb70 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:44.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0834135220 0x7f0834071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient(hunting): picked mon.noname-a con 0x7f0834135220 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:44.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f083413ec80 0x7f08340716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient(hunting): picked mon.noname-d con 0x7f083413ec80 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:44.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0834144fb0 con 0x7f0834135220 2024-08-19T10:39:44.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f083406b6c0 con 0x7f083409eb70 2024-08-19T10:39:44.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:44.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f0834145130 con 0x7f083413ec80 2024-08-19T10:39:44.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:44.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f083b020640 10 monclient(hunting): authenticate will time out at 1030.486751s 2024-08-19T10:39:44.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0838dbc640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f083409eb70 msgr2=0x7f0834070c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:44.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0838dbc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f083409eb70 0x7f0834070c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:44.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0834135220 0x7f0834071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f08395bd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f083413ec80 0x7f08340716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0834135220 0x7f0834071170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:37298/0 (socket says 172.21.15.190:37298) 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 1 -- 172.21.15.190:0/1133628300 learned_addr learned my addr 172.21.15.190:0/1133628300 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 10 monclient(hunting): get_auth_request con 0x7f0834135220 auth_method 0 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f08395bd640 10 monclient(hunting): get_auth_request con 0x7f083413ec80 auth_method 0 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f08395bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f08395bd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f08395bd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:44.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:44.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f08395bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:44.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f08395bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:44.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f08395bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:44.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 10 monclient(hunting): handle_auth_done global_id 4743 payload 824 2024-08-19T10:39:44.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 1 -- 172.21.15.190:0/1133628300 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f083413ec80 msgr2=0x7f08340716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:44.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 1 --2- 172.21.15.190:0/1133628300 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f083413ec80 0x7f08340716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 1 -- 172.21.15.190:0/1133628300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f083409eb70 msgr2=0x7f0834070c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:44.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 1 --2- 172.21.15.190:0/1133628300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f083409eb70 0x7f0834070c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:44.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:44.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 1 monclient: found mon.noname-a 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f0833fff640 1 -- 172.21.15.190:0/1133628300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0834141630 con 0x7f0834135220 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.913+0000 7f08395bd640 1 --2- 172.21.15.190:0/1133628300 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f083413ec80 0x7f08340716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f0833fff640 1 -- 172.21.15.190:0/1133628300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0834135220 msgr2=0x7f0834071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f0833fff640 1 --2- 172.21.15.190:0/1133628300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0834135220 0x7f0834071170 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f08200ae040 tx=0x7f0820019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 1 -- 172.21.15.190:0/1133628300 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08200b0040 con 0x7f0834135220 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 10 monclient: dump: 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.937 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.938 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.938 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.938 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.938 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.938 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.938 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.938 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.938 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.938 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 10 monclient: _finish_auth 0 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 10 monclient: _check_auth_tickets 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f083b020640 5 monclient: authenticate success, global_id 4743 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f083b020640 10 monclient: _renew_subs 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f083b020640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:44.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f083b020640 1 -- 172.21.15.190:0/1133628300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0834073540 con 0x7f0834135220 2024-08-19T10:39:44.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f083b020640 10 monclient: _renew_subs 2024-08-19T10:39:44.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f083b020640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:44.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f083b020640 1 -- 172.21.15.190:0/1133628300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0834073f00 con 0x7f0834135220 2024-08-19T10:39:44.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 1 -- 172.21.15.190:0/1133628300 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08200039f0 con 0x7f0834135220 2024-08-19T10:39:44.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:44.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 1 -- 172.21.15.190:0/1133628300 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0820003ba0 con 0x7f0834135220 2024-08-19T10:39:44.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:44.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:39:44.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f08317fa640 10 monclient: dump: 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:44.941 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:44.942 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:44.942 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:44.942 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:44.942 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:44.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f0832ffd640 4 set_mon_vals no callback set 2024-08-19T10:39:44.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f083b020640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:44.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f083b020640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:44.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f083b020640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:44.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.917+0000 7f083b020640 1 -- 172.21.15.190:0/1133628300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0814005090 con 0x7f0834135220 2024-08-19T10:39:44.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.921+0000 7f08317fa640 1 -- 172.21.15.190:0/1133628300 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f08200bd450 con 0x7f0834135220 2024-08-19T10:39:44.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.921+0000 7f08317fa640 1 --2- 172.21.15.190:0/1133628300 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f081803be00 0x7f081803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:44.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.921+0000 7f08317fa640 1 -- 172.21.15.190:0/1133628300 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5474+0+0 (secure 0 0 0) 0x7f08200b9070 con 0x7f0834135220 2024-08-19T10:39:44.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.921+0000 7f0838dbc640 1 --2- 172.21.15.190:0/1133628300 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f081803be00 0x7f081803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:44.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.921+0000 7f0838dbc640 10 monclient: get_auth_request con 0x7f081803be00 auth_method 0 2024-08-19T10:39:44.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.921+0000 7f0838dbc640 1 --2- 172.21.15.190:0/1133628300 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f081803be00 0x7f081803e2c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0828004820 tx=0x7f0828007b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:44.944 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.933+0000 7f08317fa640 1 -- 172.21.15.190:0/1133628300 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f08200c2020 con 0x7f0834135220 2024-08-19T10:39:44.944 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.933+0000 7f08317fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:44.944 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:44.933+0000 7f08317fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:45.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.101+0000 7f083b020640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo"}] 2024-08-19T10:39:45.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.101+0000 7f083b020640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo"}] 2024-08-19T10:39:45.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.101+0000 7f083b020640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:45.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.101+0000 7f083b020640 1 -- 172.21.15.190:0/1133628300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo"} v 0) -- 0x7f0814005650 con 0x7f0834135220 2024-08-19T10:39:45.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.101+0000 7f08317fa640 1 -- 172.21.15.190:0/1133628300 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v40) ==== 274+0+0 (secure 0 0 0) 0x7f08200160b0 con 0x7f0834135220 2024-08-19T10:39:45.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.101+0000 7f08317fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo"}] 2024-08-19T10:39:45.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.101+0000 7f08317fa640 10 monclient: _finish_command 2 = mon:1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-08-19T10:39:45.107 INFO:tasks.workunit.client.0.smithi190.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-08-19T10:39:45.110 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:45.110 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool delete foo foo 2024-08-19T10:39:45.110 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:45.110 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete foo foo 2024-08-19T10:39:45.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient: get_monmap_and_config 2024-08-19T10:39:45.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient: build_initial_monmap 2024-08-19T10:39:45.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient: monmap: 2024-08-19T10:39:45.223 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:45.223 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.223 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:45.224333+0000 2024-08-19T10:39:45.223 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:45.224333+0000 2024-08-19T10:39:45.223 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:45.224 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 1 Processor -- start 2024-08-19T10:39:45.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 1 -- start start 2024-08-19T10:39:45.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient: init 2024-08-19T10:39:45.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:45.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient: _add_conns ranks=[8,2,10,5,4,7,9,6,1,0,3] 2024-08-19T10:39:45.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48a80a22f0 0x7f48a80a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient(hunting): picked mon.noname-e con 0x7f48a80a22f0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:45.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a41c0 0x7f48a80a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient(hunting): picked mon.noname-b con 0x7f48a80a41c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f48a813aa60 0x7f48a813ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient(hunting): picked mon.noname-f con 0x7f48a813aa60 addr v2:172.21.15.190:3300/0 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f48a809e0a0 con 0x7f48a813aa60 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f48a80a4b00 con 0x7f48a80a41c0 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f48a809f770 con 0x7f48a80a22f0 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48acaff640 10 monclient(hunting): authenticate will time out at 1030.795165s 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a6d9d640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f48a813aa60 msgr2=0x7f48a813ce50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a6d9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f48a813aa60 0x7f48a813ce50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:45.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a659c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48a80a22f0 0x7f48a80a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a5d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a41c0 0x7f48a80a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a659c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48a80a22f0 0x7f48a80a3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:56712/0 (socket says 172.21.15.190:56712) 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a5d9b640 10 monclient(hunting): get_auth_request con 0x7f48a80a41c0 auth_method 0 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a659c640 1 -- 172.21.15.190:0/3098513162 learned_addr learned my addr 172.21.15.190:0/3098513162 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a659c640 10 monclient(hunting): get_auth_request con 0x7f48a80a22f0 auth_method 0 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:45.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.221+0000 7f48a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.225+0000 7f48a5d9b640 10 monclient(hunting): handle_auth_done global_id 4583 payload 274 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.225+0000 7f48a5d9b640 1 -- 172.21.15.190:0/3098513162 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48a80a22f0 msgr2=0x7f48a80a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.225+0000 7f48a5d9b640 1 --2- 172.21.15.190:0/3098513162 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48a80a22f0 0x7f48a80a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.225+0000 7f48a5d9b640 1 -- 172.21.15.190:0/3098513162 >> v2:172.21.15.190:3300/0 conn(0x7f48a813aa60 msgr2=0x7f48a813ce50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.225+0000 7f48a5d9b640 1 --2- 172.21.15.190:0/3098513162 >> v2:172.21.15.190:3300/0 conn(0x7f48a813aa60 0x7f48a813ce50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.225+0000 7f48a5d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.225+0000 7f48a5d9b640 1 monclient: found mon.noname-b 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.225+0000 7f48a5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:45.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.225+0000 7f48a5d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:45.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.225+0000 7f48a5d9b640 1 -- 172.21.15.190:0/3098513162 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a809e7f0 con 0x7f48a80a41c0 2024-08-19T10:39:45.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.225+0000 7f48a659c640 1 --2- 172.21.15.190:0/3098513162 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48a80a22f0 0x7f48a80a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:45.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a5d9b640 1 -- 172.21.15.190:0/3098513162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a41c0 msgr2=0x7f48a80a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:45.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a5d9b640 1 --2- 172.21.15.190:0/3098513162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a41c0 0x7f48a80a45c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f489c00ba60 tx=0x7f489c01eae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f940c0252f0e8d26 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:45.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 1 -- 172.21.15.190:0/3098513162 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f489c0b4070 con 0x7f48a80a41c0 2024-08-19T10:39:45.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:45.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 10 monclient: dump: 2024-08-19T10:39:45.232 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.233 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 10 monclient: _finish_auth 0 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 10 monclient: _check_auth_tickets 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 1 -- 172.21.15.190:0/3098513162 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489c0a7b90 con 0x7f48a80a41c0 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 1 -- 172.21.15.190:0/3098513162 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f489c0a7dc0 con 0x7f48a80a41c0 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a559a640 10 monclient: dump: 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.234 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.235 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 5 monclient: authenticate success, global_id 4583 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 4 set_mon_vals no callback set 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient: shutdown 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 -- 172.21.15.190:0/3098513162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a41c0 msgr2=0x7f48a80a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 --2- 172.21.15.190:0/3098513162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a41c0 0x7f48a80a45c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f489c00ba60 tx=0x7f489c01eae0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 -- 172.21.15.190:0/3098513162 shutdown_connections 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 --2- 172.21.15.190:0/3098513162 >> v2:172.21.15.190:3300/0 conn(0x7f48a813aa60 0x7f48a813ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 --2- 172.21.15.190:0/3098513162 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a41c0 0x7f48a80a45c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 --2- 172.21.15.190:0/3098513162 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48a80a22f0 0x7f48a80a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 -- 172.21.15.190:0/3098513162 >> 172.21.15.190:0/3098513162 conn(0x7f48a80a8b60 msgr2=0x7f48a80a4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:45.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 -- 172.21.15.190:0/3098513162 shutdown_connections 2024-08-19T10:39:45.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 -- 172.21.15.190:0/3098513162 wait complete. 2024-08-19T10:39:45.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient: build_initial_monmap 2024-08-19T10:39:45.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient: monmap: 2024-08-19T10:39:45.241 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:45.241 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:45.241 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:45.241 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:45.241 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:45.241 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.241 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:45.241 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:45.241 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:45.241 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 Processor -- start 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 -- start start 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient: init 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient: _add_conns ranks=[2,6,7,0,3,4,5,1,8] 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a22f0 0x7f48a80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient(hunting): picked mon.noname-b con 0x7f48a80a22f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f48a80a41c0 0x7f48a81cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient(hunting): picked mon.noname-d con 0x7f48a80a41c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48a813aa60 0x7f48a81cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient(hunting): picked mon.noname-i con 0x7f48a813aa60 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f48a80a01f0 con 0x7f48a80a22f0 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f48a8074ac0 con 0x7f48a80a41c0 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f48a80a0370 con 0x7f48a813aa60 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48acaff640 10 monclient(hunting): authenticate will time out at 1030.803458s 2024-08-19T10:39:45.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a6d9d640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48a813aa60 msgr2=0x7f48a81cd680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a6d9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48a813aa60 0x7f48a81cd680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a22f0 0x7f48a80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a22f0 0x7f48a80732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:40920/0 (socket says 172.21.15.190:40920) 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a659c640 1 -- 172.21.15.190:0/3789194857 learned_addr learned my addr 172.21.15.190:0/3789194857 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a659c640 10 monclient(hunting): get_auth_request con 0x7f48a80a22f0 auth_method 0 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a5d9b640 1 --2- 172.21.15.190:0/3789194857 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f48a80a41c0 0x7f48a81cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a5d9b640 10 monclient(hunting): get_auth_request con 0x7f48a80a41c0 auth_method 0 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.229+0000 7f48a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 10 monclient(hunting): handle_auth_done global_id 4592 payload 824 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 1 -- 172.21.15.190:0/3789194857 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48a813aa60 msgr2=0x7f48a81cd680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 1 --2- 172.21.15.190:0/3789194857 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48a813aa60 0x7f48a81cd680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 1 -- 172.21.15.190:0/3789194857 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f48a80a41c0 msgr2=0x7f48a81cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 1 --2- 172.21.15.190:0/3789194857 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f48a80a41c0 0x7f48a81cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 10 monclient: _finish_hunting 0 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 1 monclient: found mon.noname-b 2024-08-19T10:39:45.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 1 -- 172.21.15.190:0/3789194857 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a811eba0 con 0x7f48a80a22f0 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 1 -- 172.21.15.190:0/3789194857 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a22f0 msgr2=0x7f48a80732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a659c640 1 --2- 172.21.15.190:0/3789194857 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48a80a22f0 0x7f48a80732a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f489800e9d0 tx=0x7f489800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 1 -- 172.21.15.190:0/3789194857 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4898017070 con 0x7f48a80a22f0 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 10 monclient: dump: 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.246 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 10 monclient: _finish_auth 0 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 1 -- 172.21.15.190:0/3789194857 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489800cc70 con 0x7f48a80a22f0 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 1 -- 172.21.15.190:0/3789194857 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f489800ce20 con 0x7f48a80a22f0 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48a4d99640 4 set_mon_vals no callback set 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f4896ffd640 10 monclient: dump: 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.248 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48acaff640 5 monclient: authenticate success, global_id 4592 2024-08-19T10:39:45.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48acaff640 10 monclient: _renew_subs 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48acaff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48acaff640 1 -- 172.21.15.190:0/3789194857 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a80a63b0 con 0x7f48a80a22f0 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48acaff640 10 monclient: _renew_subs 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48acaff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.233+0000 7f48acaff640 1 -- 172.21.15.190:0/3789194857 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a80a1e30 con 0x7f48a80a22f0 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.237+0000 7f48acaff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.237+0000 7f48acaff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.237+0000 7f48acaff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.237+0000 7f48acaff640 1 -- 172.21.15.190:0/3789194857 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a80a3c80 con 0x7f48a80a22f0 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.237+0000 7f4896ffd640 1 -- 172.21.15.190:0/3789194857 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f489801a070 con 0x7f48a80a22f0 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.237+0000 7f4896ffd640 1 --2- 172.21.15.190:0/3789194857 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f488803be00 0x7f488803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.237+0000 7f4896ffd640 1 -- 172.21.15.190:0/3789194857 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(40..40 src has 1..40) ==== 5474+0+0 (secure 0 0 0) 0x7f489800f040 con 0x7f48a80a22f0 2024-08-19T10:39:45.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.237+0000 7f48a5d9b640 1 --2- 172.21.15.190:0/3789194857 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f488803be00 0x7f488803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.237+0000 7f48a5d9b640 10 monclient: get_auth_request con 0x7f488803be00 auth_method 0 2024-08-19T10:39:45.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.237+0000 7f48a5d9b640 1 --2- 172.21.15.190:0/3789194857 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f488803be00 0x7f488803e2c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f48a809e3a0 tx=0x7f489c0a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:45.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.249+0000 7f4896ffd640 1 -- 172.21.15.190:0/3789194857 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4898021070 con 0x7f48a80a22f0 2024-08-19T10:39:45.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.249+0000 7f4896ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:45.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.249+0000 7f4896ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.413+0000 7f48acaff640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-08-19T10:39:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.413+0000 7f48acaff640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-08-19T10:39:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.413+0000 7f48acaff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.413+0000 7f48acaff640 1 -- 172.21.15.190:0/3789194857 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"} v 0) -- 0x7f48a813ae40 con 0x7f48a80a22f0 2024-08-19T10:39:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.417+0000 7f4896ffd640 1 -- 172.21.15.190:0/3789194857 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v40) ==== 290+0+0 (secure 0 0 0) 0x7f489801bdc0 con 0x7f48a80a22f0 2024-08-19T10:39:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.417+0000 7f4896ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo"}] 2024-08-19T10:39:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.417+0000 7f4896ffd640 10 monclient: _finish_command 2 = mon:1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-08-19T10:39:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool foo. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-08-19T10:39:45.424 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:45.424 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool delete foo foo --force 2024-08-19T10:39:45.424 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:45.424 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete foo foo --force 2024-08-19T10:39:45.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient: get_monmap_and_config 2024-08-19T10:39:45.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient: build_initial_monmap 2024-08-19T10:39:45.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient: monmap: 2024-08-19T10:39:45.544 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:45.544 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.544 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:45.545307+0000 2024-08-19T10:39:45.544 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:45.545307+0000 2024-08-19T10:39:45.545 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:45.545 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.545 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:45.545 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:45.545 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:45.545 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:45.545 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:45.545 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:45.545 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:45.545 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:45.545 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:45.546 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:45.546 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:45.546 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 1 Processor -- start 2024-08-19T10:39:45.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 1 -- start start 2024-08-19T10:39:45.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient: init 2024-08-19T10:39:45.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:45.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient: _add_conns ranks=[2,8,4,0,1,6,9,5,10,3,7] 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68481396a0 0x7f6848139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient(hunting): picked mon.noname-b con 0x7f68481396a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6848139fc0 0x7f68480a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient(hunting): picked mon.noname-e con 0x7f6848139fc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68480a6e20 0x7f68480a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient(hunting): picked mon.noname-c con 0x7f68480a6e20 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f68480a4f20 con 0x7f68481396a0 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f68480a83e0 con 0x7f68480a6e20 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f68480a2f30 con 0x7f6848139fc0 2024-08-19T10:39:45.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:45.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f684d309640 10 monclient(hunting): authenticate will time out at 1031.115220s 2024-08-19T10:39:45.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68477fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68480a6e20 0x7f68480a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68467fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6848139fc0 0x7f68480a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f6846ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68481396a0 0x7f6848139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68467fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6848139fc0 0x7f68480a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:46520/0 (socket says 172.21.15.190:46520) 2024-08-19T10:39:45.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68477fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68480a6e20 0x7f68480a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:35432/0 (socket says 172.21.15.190:35432) 2024-08-19T10:39:45.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68467fc640 1 -- 172.21.15.190:0/3346741561 learned_addr learned my addr 172.21.15.190:0/3346741561 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:45.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f6846ffd640 10 monclient(hunting): get_auth_request con 0x7f68481396a0 auth_method 0 2024-08-19T10:39:45.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f6846ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f6846ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f6846ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68467fc640 10 monclient(hunting): get_auth_request con 0x7f6848139fc0 auth_method 0 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68467fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68467fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68467fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68477fe640 10 monclient(hunting): get_auth_request con 0x7f68480a6e20 auth_method 0 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68477fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f68477fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f6846ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f6846ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:45.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.541+0000 7f6846ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:45.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f68467fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:45.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f68467fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:45.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f68467fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:45.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f68477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:45.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f68477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:45.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f68477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:45.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f6846ffd640 10 monclient(hunting): handle_auth_done global_id 4601 payload 274 2024-08-19T10:39:45.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f6846ffd640 1 -- 172.21.15.190:0/3346741561 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6848139fc0 msgr2=0x7f68480a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:45.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f6846ffd640 1 --2- 172.21.15.190:0/3346741561 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6848139fc0 0x7f68480a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f6846ffd640 1 -- 172.21.15.190:0/3346741561 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68480a6e20 msgr2=0x7f68480a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:45.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f6846ffd640 1 --2- 172.21.15.190:0/3346741561 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68480a6e20 0x7f68480a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f6846ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:39:45.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f6846ffd640 1 monclient: found mon.noname-b 2024-08-19T10:39:45.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f6846ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:45.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f6846ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:45.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.545+0000 7f6846ffd640 1 -- 172.21.15.190:0/3346741561 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68480a35e0 con 0x7f68481396a0 2024-08-19T10:39:45.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6846ffd640 1 -- 172.21.15.190:0/3346741561 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68481396a0 msgr2=0x7f6848139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:45.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6846ffd640 1 --2- 172.21.15.190:0/3346741561 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68481396a0 0x7f6848139a80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6838009dc0 tx=0x7f683801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f60a7fe27e83d51 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:45.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 1 -- 172.21.15.190:0/3346741561 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68380ac070 con 0x7f68481396a0 2024-08-19T10:39:45.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:45.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 10 monclient: dump: 2024-08-19T10:39:45.558 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.559 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 10 monclient: _finish_auth 0 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 1 -- 172.21.15.190:0/3346741561 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68380a68b0 con 0x7f68481396a0 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 5 monclient: authenticate success, global_id 4601 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 1 -- 172.21.15.190:0/3346741561 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68380a6aa0 con 0x7f68481396a0 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 4 set_mon_vals no callback set 2024-08-19T10:39:45.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:45.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6845ffb640 10 monclient: dump: 2024-08-19T10:39:45.561 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.561 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.561 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.561 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.561 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.561 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.562 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.562 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.562 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.562 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.562 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.562 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.562 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.562 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.562 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.562 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient: shutdown 2024-08-19T10:39:45.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:45.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 -- 172.21.15.190:0/3346741561 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68481396a0 msgr2=0x7f6848139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:45.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 --2- 172.21.15.190:0/3346741561 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68481396a0 0x7f6848139a80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6838009dc0 tx=0x7f683801c850 comp rx=0 tx=0).stop 2024-08-19T10:39:45.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 -- 172.21.15.190:0/3346741561 shutdown_connections 2024-08-19T10:39:45.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 --2- 172.21.15.190:0/3346741561 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f68480a6e20 0x7f68480a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 --2- 172.21.15.190:0/3346741561 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6848139fc0 0x7f68480a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 --2- 172.21.15.190:0/3346741561 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68481396a0 0x7f6848139a80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 -- 172.21.15.190:0/3346741561 >> 172.21.15.190:0/3346741561 conn(0x7f68480abee0 msgr2=0x7f68480ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:45.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 -- 172.21.15.190:0/3346741561 shutdown_connections 2024-08-19T10:39:45.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 -- 172.21.15.190:0/3346741561 wait complete. 2024-08-19T10:39:45.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient: build_initial_monmap 2024-08-19T10:39:45.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient: monmap: 2024-08-19T10:39:45.564 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:45.564 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:45.564 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:45.564 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:45.564 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:45.564 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.564 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:45.564 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:45.564 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:45.564 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 Processor -- start 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 -- start start 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient: init 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient: _add_conns ranks=[1,3,6,7,0,2,8,5,4] 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68480a6e20 0x7f6848134910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient(hunting): picked mon.noname-f con 0x7f68480a6e20 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:45.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68481396a0 0x7f6848134e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient(hunting): picked mon.noname-g con 0x7f68481396a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:45.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6848139fc0 0x7f684809ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient(hunting): picked mon.noname-d con 0x7f6848139fc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:45.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f684813a3a0 con 0x7f68480a6e20 2024-08-19T10:39:45.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6848143390 con 0x7f68481396a0 2024-08-19T10:39:45.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f68481356f0 con 0x7f6848139fc0 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6846ffd640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68480a6e20 msgr2=0x7f6848134910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f684d309640 10 monclient(hunting): authenticate will time out at 1031.123406s 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f6846ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68480a6e20 0x7f6848134910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68481396a0 0x7f6848134e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68481396a0 0x7f6848134e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42256/0 (socket says 172.21.15.190:42256) 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 1 -- 172.21.15.190:0/507015489 learned_addr learned my addr 172.21.15.190:0/507015489 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 10 monclient(hunting): get_auth_request con 0x7f68481396a0 auth_method 0 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68477fe640 1 --2- 172.21.15.190:0/507015489 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6848139fc0 0x7f684809ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68477fe640 10 monclient(hunting): get_auth_request con 0x7f6848139fc0 auth_method 0 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68477fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68477fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 10 monclient(hunting): handle_auth_done global_id 4611 payload 824 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 1 -- 172.21.15.190:0/507015489 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6848139fc0 msgr2=0x7f684809ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 1 --2- 172.21.15.190:0/507015489 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6848139fc0 0x7f684809ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 1 -- 172.21.15.190:0/507015489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68480a6e20 msgr2=0x7f6848134910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:45.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 1 --2- 172.21.15.190:0/507015489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68480a6e20 0x7f6848134910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 10 monclient: _finish_hunting 0 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 1 monclient: found mon.noname-g 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.549+0000 7f68467fc640 1 -- 172.21.15.190:0/507015489 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68480ac530 con 0x7f68481396a0 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f68467fc640 1 -- 172.21.15.190:0/507015489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68481396a0 msgr2=0x7f6848134e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f68467fc640 1 --2- 172.21.15.190:0/507015489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68481396a0 0x7f6848134e50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f683000e7f0 tx=0x7f683000ecb0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 1 -- 172.21.15.190:0/507015489 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6830017070 con 0x7f68481396a0 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 10 monclient: dump: 2024-08-19T10:39:45.572 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.573 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 1 -- 172.21.15.190:0/507015489 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6830013470 con 0x7f68481396a0 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f684d309640 5 monclient: authenticate success, global_id 4611 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f684d309640 10 monclient: _renew_subs 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f684d309640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f684d309640 1 -- 172.21.15.190:0/507015489 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6848140870 con 0x7f68481396a0 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:45.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 1 -- 172.21.15.190:0/507015489 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6830013a90 con 0x7f68481396a0 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 10 monclient: dump: 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.575 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f6844ff9640 4 set_mon_vals no callback set 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f684d309640 10 monclient: _renew_subs 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f684d309640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f684d309640 1 -- 172.21.15.190:0/507015489 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f684809fd40 con 0x7f68481396a0 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f684d309640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:45.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f684d309640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:45.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f684d309640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:45.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f684d309640 1 -- 172.21.15.190:0/507015489 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6848139a80 con 0x7f68481396a0 2024-08-19T10:39:45.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 1 -- 172.21.15.190:0/507015489 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f683001a070 con 0x7f68481396a0 2024-08-19T10:39:45.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 1 --2- 172.21.15.190:0/507015489 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f682803be30 0x7f682803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f683f7fe640 1 -- 172.21.15.190:0/507015489 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(40..40 src has 1..40) ==== 5474+0+0 (secure 0 0 0) 0x7f683000f040 con 0x7f68481396a0 2024-08-19T10:39:45.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f6846ffd640 1 --2- 172.21.15.190:0/507015489 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f682803be30 0x7f682803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.553+0000 7f6846ffd640 10 monclient: get_auth_request con 0x7f682803be30 auth_method 0 2024-08-19T10:39:45.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.557+0000 7f6846ffd640 1 --2- 172.21.15.190:0/507015489 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f682803be30 0x7f682803e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6838009dc0 tx=0x7f68380a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:45.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.561+0000 7f683f7fe640 1 -- 172.21.15.190:0/507015489 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f683001a360 con 0x7f68481396a0 2024-08-19T10:39:45.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.561+0000 7f683f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:45.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.561+0000 7f683f7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:45.722 INFO:tasks.workunit.client.0.smithi190.stderr:--force not valid: --force not one of 'true', 'false' 2024-08-19T10:39:45.722 INFO:tasks.workunit.client.0.smithi190.stderr:Invalid command: unused arguments: ['--force'] 2024-08-19T10:39:45.722 INFO:tasks.workunit.client.0.smithi190.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-08-19T10:39:45.722 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: invalid command 2024-08-19T10:39:45.725 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:45.725 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool delete foo fooo --yes-i-really-mean-it 2024-08-19T10:39:45.726 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:45.726 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete foo fooo --yes-i-really-mean-it 2024-08-19T10:39:45.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient: get_monmap_and_config 2024-08-19T10:39:45.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient: build_initial_monmap 2024-08-19T10:39:45.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient: monmap: 2024-08-19T10:39:45.844 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:45.844 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.844 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:45.845088+0000 2024-08-19T10:39:45.844 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:45.845088+0000 2024-08-19T10:39:45.844 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:45.844 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.845 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:45.845 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:45.845 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:45.845 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:45.845 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:45.845 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:45.845 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:45.845 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:45.845 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:45.845 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:45.845 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:45.846 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 1 Processor -- start 2024-08-19T10:39:45.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 1 -- start start 2024-08-19T10:39:45.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient: init 2024-08-19T10:39:45.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:45.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient: _add_conns ranks=[9,5,10,7,3,6,8,1,4,2,0] 2024-08-19T10:39:45.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f07e00a36a0 0x7f07e00a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient(hunting): picked mon.noname-a con 0x7f07e00a36a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:45.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f07e00a19d0 0x7f07e00a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient(hunting): picked mon.noname-h con 0x7f07e00a19d0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:45.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f07e0137470 0x7f07e0137900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient(hunting): picked mon.noname-f con 0x7f07e0137470 addr v2:172.21.15.190:3300/0 2024-08-19T10:39:45.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f07e00a4500 con 0x7f07e00a36a0 2024-08-19T10:39:45.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f07e009d050 con 0x7f07e0137470 2024-08-19T10:39:45.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f07e006f250 con 0x7f07e00a19d0 2024-08-19T10:39:45.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:45.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07e5cfa640 10 monclient(hunting): authenticate will time out at 1031.414946s 2024-08-19T10:39:45.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07dffff640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f07e0137470 msgr2=0x7f07e0137900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:45.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07dffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f07e0137470 0x7f07e0137900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:45.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f07e00a19d0 0x7f07e00a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f07e00a19d0 0x7f07e00a3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36834/0 (socket says 172.21.15.190:36834) 2024-08-19T10:39:45.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 1 -- 172.21.15.190:0/1025262539 learned_addr learned my addr 172.21.15.190:0/1025262539 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:45.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 10 monclient(hunting): get_auth_request con 0x7f07e00a19d0 auth_method 0 2024-08-19T10:39:45.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:45.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07df7fe640 1 --2- 172.21.15.190:0/1025262539 >> v2:172.21.15.89:3300/0 conn(0x7f07e00a36a0 0x7f07e00a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 10 monclient(hunting): handle_auth_done global_id 4649 payload 274 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 1 -- 172.21.15.190:0/1025262539 >> v2:172.21.15.190:3300/0 conn(0x7f07e0137470 msgr2=0x7f07e0137900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 1 --2- 172.21.15.190:0/1025262539 >> v2:172.21.15.190:3300/0 conn(0x7f07e0137470 0x7f07e0137900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 1 -- 172.21.15.190:0/1025262539 >> v2:172.21.15.89:3300/0 conn(0x7f07e00a36a0 msgr2=0x7f07e00a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 1 --2- 172.21.15.190:0/1025262539 >> v2:172.21.15.89:3300/0 conn(0x7f07e00a36a0 0x7f07e00a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 10 monclient: _finish_hunting 0 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 1 monclient: found mon.noname-h 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:45.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07deffd640 1 -- 172.21.15.190:0/1025262539 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e009dec0 con 0x7f07e00a19d0 2024-08-19T10:39:45.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07df7fe640 10 monclient: get_auth_request con 0x7f07e00a36a0 auth_method 0 2024-08-19T10:39:45.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.841+0000 7f07df7fe640 1 --2- 172.21.15.190:0/1025262539 >> v2:172.21.15.89:3300/0 conn(0x7f07e00a36a0 0x7f07e00a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:39:45.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.845+0000 7f07deffd640 1 -- 172.21.15.190:0/1025262539 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f07e00a19d0 msgr2=0x7f07e00a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:45.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.845+0000 7f07deffd640 1 --2- 172.21.15.190:0/1025262539 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f07e00a19d0 0x7f07e00a3fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f07e00a18e0 tx=0x7f07d001c400 comp rx=0 tx=0).ready entity=mon.5 client_cookie=331b87805c4ca45e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:45.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 1 -- 172.21.15.190:0/1025262539 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f07d00b0070 con 0x7f07e00a19d0 2024-08-19T10:39:45.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:45.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 10 monclient: dump: 2024-08-19T10:39:45.858 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.858 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.858 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.858 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.858 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.859 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.859 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.859 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.859 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.859 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.859 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.859 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.859 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.859 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 5 monclient: authenticate success, global_id 4649 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 1 -- 172.21.15.190:0/1025262539 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d00aa810 con 0x7f07e00a19d0 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 4 set_mon_vals no callback set 2024-08-19T10:39:45.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 1 -- 172.21.15.190:0/1025262539 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f07d00a93e0 con 0x7f07e00a19d0 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07de7fc640 10 monclient: dump: 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.862 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient: shutdown 2024-08-19T10:39:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 -- 172.21.15.190:0/1025262539 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f07e00a19d0 msgr2=0x7f07e00a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 --2- 172.21.15.190:0/1025262539 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f07e00a19d0 0x7f07e00a3fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f07e00a18e0 tx=0x7f07d001c400 comp rx=0 tx=0).stop 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 -- 172.21.15.190:0/1025262539 shutdown_connections 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 --2- 172.21.15.190:0/1025262539 >> v2:172.21.15.190:3300/0 conn(0x7f07e0137470 0x7f07e0137900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 --2- 172.21.15.190:0/1025262539 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f07e00a19d0 0x7f07e00a3fc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 --2- 172.21.15.190:0/1025262539 >> v2:172.21.15.89:3300/0 conn(0x7f07e00a36a0 0x7f07e00a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 -- 172.21.15.190:0/1025262539 >> 172.21.15.190:0/1025262539 conn(0x7f07e00a4790 msgr2=0x7f07e00a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 -- 172.21.15.190:0/1025262539 shutdown_connections 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 -- 172.21.15.190:0/1025262539 wait complete. 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient: build_initial_monmap 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient: monmap: 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:45.864 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 Processor -- start 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 -- start start 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient: init 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient: _add_conns ranks=[2,1,4,6,7,5,3,8,0] 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07e00a19d0 0x7f07e0072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient(hunting): picked mon.noname-b con 0x7f07e00a19d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00a36a0 0x7f07e011e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient(hunting): picked mon.noname-f con 0x7f07e00a36a0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07e0137470 0x7f07e011e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient(hunting): picked mon.noname-c con 0x7f07e0137470 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f07e009ca60 con 0x7f07e00a36a0 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f07e00a2050 con 0x7f07e00a19d0 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f07e01410f0 con 0x7f07e0137470 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07e5cfa640 10 monclient(hunting): authenticate will time out at 1031.422321s 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07deffd640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00a36a0 msgr2=0x7f07e011e2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07deffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00a36a0 0x7f07e011e2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07e0137470 0x7f07e011e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07df7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07e00a19d0 0x7f07e0072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07e0137470 0x7f07e011e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:35446/0 (socket says 172.21.15.190:35446) 2024-08-19T10:39:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 1 -- 172.21.15.190:0/3397235145 learned_addr learned my addr 172.21.15.190:0/3397235145 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 10 monclient(hunting): get_auth_request con 0x7f07e0137470 auth_method 0 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07df7fe640 10 monclient(hunting): get_auth_request con 0x7f07e00a19d0 auth_method 0 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07df7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 10 monclient(hunting): handle_auth_done global_id 4711 payload 824 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 1 -- 172.21.15.190:0/3397235145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07e00a19d0 msgr2=0x7f07e0072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 1 --2- 172.21.15.190:0/3397235145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07e00a19d0 0x7f07e0072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 1 -- 172.21.15.190:0/3397235145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00a36a0 msgr2=0x7f07e011e2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 1 --2- 172.21.15.190:0/3397235145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00a36a0 0x7f07e011e2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 1 monclient: found mon.noname-c 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07dffff640 1 -- 172.21.15.190:0/3397235145 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e01209a0 con 0x7f07e0137470 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.849+0000 7f07df7fe640 1 --2- 172.21.15.190:0/3397235145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f07e00a19d0 0x7f07e0072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:45.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07dffff640 1 -- 172.21.15.190:0/3397235145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07e0137470 msgr2=0x7f07e011e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07dffff640 1 --2- 172.21.15.190:0/3397235145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f07e0137470 0x7f07e011e820 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07d800be60 tx=0x7f07d800bf50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 1 -- 172.21.15.190:0/3397235145 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f07d8015850 con 0x7f07e0137470 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 10 monclient: dump: 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.869 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 10 monclient: _finish_auth 0 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 1 -- 172.21.15.190:0/3397235145 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d8010030 con 0x7f07e0137470 2024-08-19T10:39:45.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 1 -- 172.21.15.190:0/3397235145 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f07d80060d0 con 0x7f07e0137470 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07e5cfa640 5 monclient: authenticate success, global_id 4711 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07e5cfa640 10 monclient: _renew_subs 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07e5cfa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07e5cfa640 1 -- 172.21.15.190:0/3397235145 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e0076d80 con 0x7f07e0137470 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07ddffb640 4 set_mon_vals no callback set 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07c7fff640 10 monclient: dump: 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.871 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:45.872 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:45.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07e5cfa640 10 monclient: _renew_subs 2024-08-19T10:39:45.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07e5cfa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:45.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07e5cfa640 1 -- 172.21.15.190:0/3397235145 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e01206c0 con 0x7f07e0137470 2024-08-19T10:39:45.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07e5cfa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:45.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07e5cfa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:45.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07e5cfa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:39:45.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.853+0000 7f07e5cfa640 1 -- 172.21.15.190:0/3397235145 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e0137900 con 0x7f07e0137470 2024-08-19T10:39:45.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.857+0000 7f07c7fff640 1 -- 172.21.15.190:0/3397235145 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f07d80153e0 con 0x7f07e0137470 2024-08-19T10:39:45.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.857+0000 7f07c7fff640 1 --2- 172.21.15.190:0/3397235145 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f07cc03be00 0x7f07cc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:45.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.857+0000 7f07c7fff640 1 -- 172.21.15.190:0/3397235145 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(40..40 src has 1..40) ==== 5474+0+0 (secure 0 0 0) 0x7f07d804f810 con 0x7f07e0137470 2024-08-19T10:39:45.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.857+0000 7f07df7fe640 1 --2- 172.21.15.190:0/3397235145 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f07cc03be00 0x7f07cc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:45.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.857+0000 7f07df7fe640 10 monclient: get_auth_request con 0x7f07cc03be00 auth_method 0 2024-08-19T10:39:45.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.857+0000 7f07df7fe640 1 --2- 172.21.15.190:0/3397235145 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f07cc03be00 0x7f07cc03e2c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f07d400af30 tx=0x7f07d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:45.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.869+0000 7f07c7fff640 1 -- 172.21.15.190:0/3397235145 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f07d801d300 con 0x7f07e0137470 2024-08-19T10:39:45.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.869+0000 7f07c7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:45.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:45.869+0000 7f07c7fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:46.042 INFO:tasks.workunit.client.0.smithi190.stderr:--yes-i-really-mean-it not valid: --yes-i-really-mean-it not one of 'true', 'false' 2024-08-19T10:39:46.043 INFO:tasks.workunit.client.0.smithi190.stderr:Invalid command: unused arguments: ['--yes-i-really-mean-it'] 2024-08-19T10:39:46.043 INFO:tasks.workunit.client.0.smithi190.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-08-19T10:39:46.043 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: invalid command 2024-08-19T10:39:46.046 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:46.046 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd pool delete foo --yes-i-really-mean-it foo 2024-08-19T10:39:46.046 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:39:46.046 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete foo --yes-i-really-mean-it foo 2024-08-19T10:39:46.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient: get_monmap_and_config 2024-08-19T10:39:46.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient: build_initial_monmap 2024-08-19T10:39:46.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient: monmap: 2024-08-19T10:39:46.167 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:46.167 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:46.167 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:46.167774+0000 2024-08-19T10:39:46.167 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:46.167774+0000 2024-08-19T10:39:46.167 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:46.167 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.167 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:46.167 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:46.167 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:46.167 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:46.168 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:46.168 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:46.168 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:46.168 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:46.168 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:46.168 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:46.168 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:46.168 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 1 Processor -- start 2024-08-19T10:39:46.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 1 -- start start 2024-08-19T10:39:46.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient: init 2024-08-19T10:39:46.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:46.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient: _add_conns ranks=[9,0,10,1,6,2,4,5,7,3,8] 2024-08-19T10:39:46.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbfb00a3d80 0x7fbfb00a4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient(hunting): picked mon.noname-a con 0x7fbfb00a3d80 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:46.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbfb009f0c0 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:46.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbfb0069770 0x7fbfb00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient(hunting): picked mon.noname-f con 0x7fbfb0069770 addr v2:172.21.15.190:3300/0 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfb009eca0 con 0x7fbfb009f0c0 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fbfb009e6e0 con 0x7fbfb00a3d80 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbfb00a4be0 con 0x7fbfb0069770 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb789e640 10 monclient(hunting): authenticate will time out at 1031.737602s 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb5e3b640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fbfb0069770 msgr2=0x7fbfb00a46a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb5e3b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbfb0069770 0x7fbfb00a46a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb563a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbfb00a3d80 0x7fbfb00a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:46.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb563a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbfb00a3d80 0x7fbfb00a4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:38410/0 (socket says 172.21.15.190:38410) 2024-08-19T10:39:46.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb563a640 1 -- 172.21.15.190:0/1521308130 learned_addr learned my addr 172.21.15.190:0/1521308130 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:46.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb563a640 10 monclient(hunting): get_auth_request con 0x7fbfb00a3d80 auth_method 0 2024-08-19T10:39:46.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb563a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:46.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb563a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:46.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb563a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:46.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb563a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:46.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb563a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:46.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.165+0000 7fbfb563a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:46.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.169+0000 7fbfb4e39640 1 -- 172.21.15.190:0/1521308130 >> v1:172.21.15.89:6789/0 conn(0x7fbfb009f0c0 legacy=0x7fbfb009b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:46.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.169+0000 7fbfb563a640 10 monclient(hunting): handle_auth_done global_id 4752 payload 274 2024-08-19T10:39:46.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.169+0000 7fbfb563a640 1 -- 172.21.15.190:0/1521308130 >> v2:172.21.15.190:3300/0 conn(0x7fbfb0069770 msgr2=0x7fbfb00a46a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:46.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.169+0000 7fbfb563a640 1 --2- 172.21.15.190:0/1521308130 >> v2:172.21.15.190:3300/0 conn(0x7fbfb0069770 0x7fbfb00a46a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.169+0000 7fbfb563a640 1 -- 172.21.15.190:0/1521308130 >> v1:172.21.15.89:6789/0 conn(0x7fbfb009f0c0 legacy=0x7fbfb009b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:46.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.169+0000 7fbfb563a640 10 monclient: _finish_hunting 0 2024-08-19T10:39:46.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.169+0000 7fbfb563a640 1 monclient: found mon.noname-a 2024-08-19T10:39:46.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.169+0000 7fbfb563a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:46.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.169+0000 7fbfb563a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:39:46.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.169+0000 7fbfb563a640 1 -- 172.21.15.190:0/1521308130 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb009e400 con 0x7fbfb00a3d80 2024-08-19T10:39:46.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb563a640 1 -- 172.21.15.190:0/1521308130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfb00a3d80 msgr2=0x7fbfb00a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:46.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb563a640 1 --2- 172.21.15.190:0/1521308130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfb00a3d80 0x7fbfb00a4160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbfb006e7b0 tx=0x7fbfa801d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27121ab08ee4b3e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:46.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 1 -- 172.21.15.190:0/1521308130 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbfa80091b0 con 0x7fbfb00a3d80 2024-08-19T10:39:46.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:46.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:39:46.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 10 monclient: dump: 2024-08-19T10:39:46.178 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:46.178 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:46.179 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 10 monclient: _finish_auth 0 2024-08-19T10:39:46.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:46.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:46.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 5 monclient: authenticate success, global_id 4752 2024-08-19T10:39:46.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:46.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 1 -- 172.21.15.190:0/1521308130 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa80a5c50 con 0x7fbfb00a3d80 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 4 set_mon_vals no callback set 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 1 -- 172.21.15.190:0/1521308130 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbfa80a5e00 con 0x7fbfb00a3d80 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfa7fff640 10 monclient: dump: 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.182 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 10 monclient: shutdown 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 1 -- 172.21.15.190:0/1521308130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfb00a3d80 msgr2=0x7fbfb00a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 1 --2- 172.21.15.190:0/1521308130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfb00a3d80 0x7fbfb00a4160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbfb006e7b0 tx=0x7fbfa801d440 comp rx=0 tx=0).stop 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 1 -- 172.21.15.190:0/1521308130 shutdown_connections 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 1 --2- 172.21.15.190:0/1521308130 >> v2:172.21.15.190:3300/0 conn(0x7fbfb0069770 0x7fbfb00a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 1 --2- 172.21.15.190:0/1521308130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfb00a3d80 0x7fbfb00a4160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 1 -- 172.21.15.190:0/1521308130 >> 172.21.15.190:0/1521308130 conn(0x7fbfb00a4e70 msgr2=0x7fbfb00a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 1 -- 172.21.15.190:0/1521308130 shutdown_connections 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 1 -- 172.21.15.190:0/1521308130 wait complete. 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 10 monclient: build_initial_monmap 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 10 monclient: monmap: 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:46.184 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:46.185 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.173+0000 7fbfb789e640 1 Processor -- start 2024-08-19T10:39:46.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 1 -- start start 2024-08-19T10:39:46.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient: init 2024-08-19T10:39:46.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:46.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient: _add_conns ranks=[2,3,5,8,1,0,4,6,7] 2024-08-19T10:39:46.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfb0069770 0x7fbfb013bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient(hunting): picked mon.noname-b con 0x7fbfb0069770 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:46.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfb009f0c0 0x7fbfb013c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient(hunting): picked mon.noname-g con 0x7fbfb009f0c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:46.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbfb013c5c0 0x7fbfb01cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient(hunting): picked mon.noname-h con 0x7fbfb013c5c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb0140ac0 con 0x7fbfb0069770 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb0140940 con 0x7fbfb009f0c0 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fbfb0140c40 con 0x7fbfb013c5c0 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient(hunting): authenticate will time out at 1031.748524s 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbfb013c5c0 0x7fbfb01cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb4e39640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfb009f0c0 0x7fbfb013c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbfb013c5c0 0x7fbfb01cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36842/0 (socket says 172.21.15.190:36842) 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 1 -- 172.21.15.190:0/366189507 learned_addr learned my addr 172.21.15.190:0/366189507 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:46.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 10 monclient(hunting): get_auth_request con 0x7fbfb013c5c0 auth_method 0 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb4e39640 10 monclient(hunting): get_auth_request con 0x7fbfb009f0c0 auth_method 0 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb4e39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb4e39640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb4e39640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb4e39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb4e39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb4e39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:46.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb563a640 1 --2- 172.21.15.190:0/366189507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfb0069770 0x7fbfb013bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb563a640 10 monclient(hunting): get_auth_request con 0x7fbfb0069770 auth_method 0 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb563a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb563a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb563a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 10 monclient(hunting): handle_auth_done global_id 4658 payload 824 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 1 -- 172.21.15.190:0/366189507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfb009f0c0 msgr2=0x7fbfb013c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 1 --2- 172.21.15.190:0/366189507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfb009f0c0 0x7fbfb013c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 1 -- 172.21.15.190:0/366189507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfb0069770 msgr2=0x7fbfb013bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 1 --2- 172.21.15.190:0/366189507 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbfb0069770 0x7fbfb013bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 10 monclient: _finish_hunting 0 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 1 monclient: found mon.noname-h 2024-08-19T10:39:46.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 1 -- 172.21.15.190:0/366189507 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb012ef60 con 0x7fbfb013c5c0 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb4e39640 1 --2- 172.21.15.190:0/366189507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfb009f0c0 0x7fbfb013c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 1 -- 172.21.15.190:0/366189507 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbfb013c5c0 msgr2=0x7fbfb01cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb5e3b640 1 --2- 172.21.15.190:0/366189507 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbfb013c5c0 0x7fbfb01cee70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00b440 tx=0x7fbfac00b900 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 1 -- 172.21.15.190:0/366189507 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbfac016070 con 0x7fbfb013c5c0 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 10 monclient: dump: 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.194 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 1 -- 172.21.15.190:0/366189507 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfac003de0 con 0x7fbfb013c5c0 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 5 monclient: authenticate success, global_id 4658 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient: _renew_subs 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 1 -- 172.21.15.190:0/366189507 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb012f240 con 0x7fbfb013c5c0 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 1 -- 172.21.15.190:0/366189507 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbfac0103a0 con 0x7fbfb013c5c0 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa5ffb640 10 monclient: dump: 2024-08-19T10:39:46.196 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:46.197 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:46.197 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.197 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.197 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:46.197 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.197 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:46.197 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:46.197 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:46.197 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:46.197 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfa77fe640 4 set_mon_vals no callback set 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient: _renew_subs 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.177+0000 7fbfb789e640 1 -- 172.21.15.190:0/366189507 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb012ec80 con 0x7fbfb013c5c0 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.181+0000 7fbfb789e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.181+0000 7fbfb789e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.181+0000 7fbfb789e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:46.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.181+0000 7fbfb789e640 1 -- 172.21.15.190:0/366189507 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb009b0c0 con 0x7fbfb013c5c0 2024-08-19T10:39:46.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.181+0000 7fbfa5ffb640 1 -- 172.21.15.190:0/366189507 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbfac005070 con 0x7fbfb013c5c0 2024-08-19T10:39:46.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.181+0000 7fbfa5ffb640 1 --2- 172.21.15.190:0/366189507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf9403be00 0x7fbf9403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.181+0000 7fbfa5ffb640 1 -- 172.21.15.190:0/366189507 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(40..40 src has 1..40) ==== 5474+0+0 (secure 0 0 0) 0x7fbfac04deb0 con 0x7fbfb013c5c0 2024-08-19T10:39:46.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.181+0000 7fbfb563a640 1 --2- 172.21.15.190:0/366189507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf9403be00 0x7fbf9403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:46.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.181+0000 7fbfb563a640 10 monclient: get_auth_request con 0x7fbf9403be00 auth_method 0 2024-08-19T10:39:46.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.181+0000 7fbfb563a640 1 --2- 172.21.15.190:0/366189507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf9403be00 0x7fbf9403e2c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00705b0 tx=0x7fbfa8004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:46.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.185+0000 7fbfa5ffb640 1 -- 172.21.15.190:0/366189507 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fbfac00d280 con 0x7fbfb013c5c0 2024-08-19T10:39:46.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.185+0000 7fbfa5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:46.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.185+0000 7fbfa5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:46.360 INFO:tasks.workunit.client.0.smithi190.stderr:foo not valid: foo not one of 'true', 'false' 2024-08-19T10:39:46.360 INFO:tasks.workunit.client.0.smithi190.stderr:Invalid command: unused arguments: ['foo'] 2024-08-19T10:39:46.360 INFO:tasks.workunit.client.0.smithi190.stderr:osd pool delete [] [--yes-i-really-really-mean-it] [--yes-i-really-really-mean-it-not-faking] : delete pool 2024-08-19T10:39:46.360 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: invalid command 2024-08-19T10:39:46.363 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:39:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete foooo foooo --yes-i-really-really-mean-it 2024-08-19T10:39:46.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient: get_monmap_and_config 2024-08-19T10:39:46.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient: build_initial_monmap 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient: monmap: 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:46.487778+0000 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:46.487778+0000 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:46.487 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:46.488 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:46.488 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:46.488 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:46.488 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:46.488 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:46.488 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 1 Processor -- start 2024-08-19T10:39:46.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 1 -- start start 2024-08-19T10:39:46.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient: init 2024-08-19T10:39:46.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:46.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient: _add_conns ranks=[6,8,7,4,1,10,0,3,5,2,9] 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac9c0a68e0 0x7fac9c0a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient(hunting): picked mon.noname-d con 0x7fac9c0a68e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fac9c0a6cc0 0x7fac9c0a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient(hunting): picked mon.noname-e con 0x7fac9c0a6cc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac9c1396a0 0x7fac9c139ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient(hunting): picked mon.noname-i con 0x7fac9c1396a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fac9c0a86c0 con 0x7fac9c0a68e0 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fac9c0a8410 con 0x7fac9c1396a0 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fac9c0a63a0 con 0x7fac9c0a6cc0 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca2b5e640 10 monclient(hunting): authenticate will time out at 1032.057615s 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca10fb640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac9c1396a0 msgr2=0x7fac9c139ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca10fb640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac9c1396a0 0x7fac9c139ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca08fa640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac9c0a68e0 0x7fac9c0a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fac9c0a6cc0 0x7fac9c0a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca08fa640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac9c0a68e0 0x7fac9c0a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:51922/0 (socket says 172.21.15.190:51922) 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fac9c0a6cc0 0x7fac9c0a70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:46530/0 (socket says 172.21.15.190:46530) 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca08fa640 1 -- 172.21.15.190:0/1144010385 learned_addr learned my addr 172.21.15.190:0/1144010385 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca08fa640 10 monclient(hunting): get_auth_request con 0x7fac9c0a68e0 auth_method 0 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca08fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca08fa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca08fa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:46.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 10 monclient(hunting): get_auth_request con 0x7fac9c0a6cc0 auth_method 0 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca08fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca08fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca08fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 10 monclient(hunting): handle_auth_done global_id 4580 payload 274 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 1 -- 172.21.15.190:0/1144010385 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac9c1396a0 msgr2=0x7fac9c139ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac9c1396a0 0x7fac9c139ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 1 -- 172.21.15.190:0/1144010385 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac9c0a68e0 msgr2=0x7fac9c0a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:46.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac9c0a68e0 0x7fac9c0a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:46.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 1 monclient: found mon.noname-e 2024-08-19T10:39:46.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:46.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:39:46.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7fac9bfff640 1 -- 172.21.15.190:0/1144010385 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c0a4680 con 0x7fac9c0a6cc0 2024-08-19T10:39:46.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.485+0000 7faca08fa640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac9c0a68e0 0x7fac9c0a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:46.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.489+0000 7fac9bfff640 1 -- 172.21.15.190:0/1144010385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fac9c0a6cc0 msgr2=0x7fac9c0a70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:46.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.489+0000 7fac9bfff640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fac9c0a6cc0 0x7fac9c0a70a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009dc0 tx=0x7fac8c01c550 comp rx=0 tx=0).ready entity=mon.8 client_cookie=4d386835afa2d5c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:46.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9bfff640 1 -- 172.21.15.190:0/1144010385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fac9c0a6cc0 msgr2=0x7fac9c0a70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9bfff640 1 -- 172.21.15.190:0/1144010385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fac9c0a6cc0 msgr2=0x7fac9c0a70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9bfff640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fac9c0a6cc0 0x7fac9c0a70a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009dc0 tx=0x7fac8c01c550 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9bfff640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fac9c0a6cc0 0x7fac9c0a70a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009dc0 tx=0x7fac8c01c550 comp rx=0 tx=0).stop 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: ms_handle_reset current mon [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 1 -- 172.21.15.190:0/1144010385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fac9c0a6cc0 msgr2=0x7fac9c0a70a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fac9c0a6cc0 0x7fac9c0a70a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: _add_conns ranks=[5,0,9,4,10,1,3,2] 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac84003640 0x7fac84005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient(hunting): picked mon.noname-h con 0x7fac84003640 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fac840062b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 1 --2- 172.21.15.190:0/1144010385 >> v2:172.21.15.89:3300/0 conn(0x7fac8400a220 0x7fac8400c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient(hunting): picked mon.noname-a con 0x7fac8400a220 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 1 -- 172.21.15.190:0/1144010385 --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac8400cfb0 con 0x7fac840062b0 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 1 -- 172.21.15.190:0/1144010385 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fac8400e220 con 0x7fac8400a220 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 1 -- 172.21.15.190:0/1144010385 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fac8400e3f0 con 0x7fac84003640 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient(hunting): ms_handle_reset stray mon [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac84003640 0x7fac84005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 10 monclient(hunting): get_auth_request con 0x7fac84003640 auth_method 0 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:46.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca10fb640 1 --2- 172.21.15.190:0/1144010385 >> v2:172.21.15.89:3300/0 conn(0x7fac8400a220 0x7fac8400c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca10fb640 10 monclient(hunting): get_auth_request con 0x7fac8400a220 auth_method 0 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca10fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca10fb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca10fb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 10 monclient(hunting): handle_auth_done global_id 4580 payload 293 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 1 -- 172.21.15.190:0/1144010385 >> v2:172.21.15.89:3300/0 conn(0x7fac8400a220 msgr2=0x7fac8400c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 1 --2- 172.21.15.190:0/1144010385 >> v2:172.21.15.89:3300/0 conn(0x7fac8400a220 0x7fac8400c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 1 -- 172.21.15.190:0/1144010385 >> v1:172.21.15.89:6789/0 conn(0x7fac840062b0 legacy=0x7fac84008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:46.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 10 monclient: _finish_hunting 0 2024-08-19T10:39:46.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 1 monclient: found mon.noname-h 2024-08-19T10:39:46.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:46.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:46.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 1 -- 172.21.15.190:0/1144010385 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8400e6b0 con 0x7fac84003640 2024-08-19T10:39:46.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca10fb640 1 -- 172.21.15.190:0/1144010385 reap_dead start 2024-08-19T10:39:46.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 1 -- 172.21.15.190:0/1144010385 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac84003640 msgr2=0x7fac84005a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:46.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca08fa640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac84003640 0x7fac84005a90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fac90009d30 tx=0x7fac8c009750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=803cbd5726748f3b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:46.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 1 -- 172.21.15.190:0/1144010385 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac90017070 con 0x7fac84003640 2024-08-19T10:39:46.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:46.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: dump: 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:46.506 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 1 -- 172.21.15.190:0/1144010385 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9000ee50 con 0x7fac84003640 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 1 -- 172.21.15.190:0/1144010385 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac90009930 con 0x7fac84003640 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca2b5e640 5 monclient: authenticate success, global_id 4580 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca2b5e640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca2b5e640 4 set_mon_vals no callback set 2024-08-19T10:39:46.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7fac9b7fe640 10 monclient: dump: 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:46.508 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca2b5e640 10 monclient: shutdown 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca2b5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca2b5e640 1 -- 172.21.15.190:0/1144010385 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac84003640 msgr2=0x7fac84005a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.493+0000 7faca2b5e640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac84003640 0x7fac84005a90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fac90009d30 tx=0x7fac8c009750 comp rx=0 tx=0).stop 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 -- 172.21.15.190:0/1144010385 shutdown_connections 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 --2- 172.21.15.190:0/1144010385 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac84003640 0x7fac84005a90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 -- 172.21.15.190:0/1144010385 >> 172.21.15.190:0/1144010385 conn(0x7fac9c0abee0 msgr2=0x7fac9c0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:46.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 -- 172.21.15.190:0/1144010385 shutdown_connections 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 -- 172.21.15.190:0/1144010385 wait complete. 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient: build_initial_monmap 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient: monmap: 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:46.510 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 Processor -- start 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 -- start start 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient: init 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient: _add_conns ranks=[1,7,0,8,4,3,6,2,5] 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84003640 0x7fac9c09ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient(hunting): picked mon.noname-f con 0x7fac84003640 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:46.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac9c0a28e0 0x7fac9c09cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient(hunting): picked mon.noname-i con 0x7fac9c0a28e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09d4e0 0x7fac9c1d8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient(hunting): picked mon.noname-a con 0x7fac9c09d4e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c0a7220 con 0x7fac9c09d4e0 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c0a70a0 con 0x7fac84003640 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fac9c0a73a0 con 0x7fac9c0a28e0 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca2b5e640 10 monclient(hunting): authenticate will time out at 1032.069769s 2024-08-19T10:39:46.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7fac9bfff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac9c0a28e0 msgr2=0x7fac9c09cfa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca08fa640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84003640 msgr2=0x7fac9c09ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca08fa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84003640 0x7fac9c09ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7fac9bfff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac9c0a28e0 0x7fac9c09cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09d4e0 0x7fac9c1d8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09d4e0 0x7fac9c1d8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:38442/0 (socket says 172.21.15.190:38442) 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 1 -- 172.21.15.190:0/1722372677 learned_addr learned my addr 172.21.15.190:0/1722372677 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 10 monclient(hunting): get_auth_request con 0x7fac9c09d4e0 auth_method 0 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:46.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 10 monclient(hunting): handle_auth_done global_id 4770 payload 824 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 1 -- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac9c0a28e0 msgr2=0x7fac9c09cfa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac9c0a28e0 0x7fac9c09cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 1 -- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84003640 msgr2=0x7fac9c09ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84003640 0x7fac9c09ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 10 monclient: _finish_hunting 0 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 1 monclient: found mon.noname-a 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.497+0000 7faca10fb640 1 -- 172.21.15.190:0/1722372677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c0a9370 con 0x7fac9c09d4e0 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca10fb640 1 -- 172.21.15.190:0/1722372677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09d4e0 msgr2=0x7fac9c1d8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:46.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca10fb640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09d4e0 0x7fac9c1d8920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fac94006fd0 tx=0x7fac9400d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 1 -- 172.21.15.190:0/1722372677 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac94003d90 con 0x7fac9c09d4e0 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 10 monclient: dump: 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:46.515 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 10 monclient: _finish_auth 0 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 10 monclient: _check_auth_tickets 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 1 -- 172.21.15.190:0/1722372677 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac94017070 con 0x7fac9c09d4e0 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca2b5e640 5 monclient: authenticate success, global_id 4770 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca2b5e640 10 monclient: _renew_subs 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 1 -- 172.21.15.190:0/1722372677 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac94012420 con 0x7fac9c09d4e0 2024-08-19T10:39:46.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca2b5e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca2b5e640 1 -- 172.21.15.190:0/1722372677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c140050 con 0x7fac9c09d4e0 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac9a7fc640 4 set_mon_vals no callback set 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 10 monclient: dump: 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:46.517 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca2b5e640 10 monclient: _renew_subs 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca2b5e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca2b5e640 1 -- 172.21.15.190:0/1722372677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c09fe00 con 0x7fac9c09d4e0 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca2b5e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca2b5e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca2b5e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:46.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca2b5e640 1 -- 172.21.15.190:0/1722372677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c0a5760 con 0x7fac9c09d4e0 2024-08-19T10:39:46.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 1 -- 172.21.15.190:0/1722372677 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fac94005af0 con 0x7fac9c09d4e0 2024-08-19T10:39:46.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8803be30 0x7fac8803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:46.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7fac997fa640 1 -- 172.21.15.190:0/1722372677 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5474+0+0 (secure 0 0 0) 0x7fac9400f040 con 0x7fac9c09d4e0 2024-08-19T10:39:46.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca08fa640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8803be30 0x7fac8803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:46.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.501+0000 7faca08fa640 10 monclient: get_auth_request con 0x7fac8803be30 auth_method 0 2024-08-19T10:39:46.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.505+0000 7faca08fa640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8803be30 0x7fac8803e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fac90009d30 tx=0x7fac90008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:46.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.513+0000 7fac997fa640 1 -- 172.21.15.190:0/1722372677 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fac9404c330 con 0x7fac9c09d4e0 2024-08-19T10:39:46.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.513+0000 7fac997fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:46.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.513+0000 7fac997fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:46.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.717+0000 7faca2b5e640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:46.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.717+0000 7faca2b5e640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:46.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.717+0000 7faca2b5e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:46.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:46.717+0000 7faca2b5e640 1 -- 172.21.15.190:0/1722372677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7fac9c09fa80 con 0x7fac9c09d4e0 2024-08-19T10:39:47.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:47.917+0000 7fac997fa640 1 -- 172.21.15.190:0/1722372677 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}]=0 pool 'foooo' removed v41) ==== 155+0+0 (secure 0 0 0) 0x7fac94005070 con 0x7fac9c09d4e0 2024-08-19T10:39:47.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:47.917+0000 7fac997fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:47.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:47.917+0000 7fac997fa640 10 monclient: _finish_command 2 = system:0 pool 'foooo' removed 2024-08-19T10:39:48.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.057+0000 7faca2b5e640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:48.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.057+0000 7faca2b5e640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:48.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.057+0000 7faca2b5e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:48.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.057+0000 7faca2b5e640 1 -- 172.21.15.190:0/1722372677 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7fac9c0008e0 con 0x7fac9c09d4e0 2024-08-19T10:39:48.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.057+0000 7fac997fa640 1 -- 172.21.15.190:0/1722372677 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}]=0 pool 'foooo' does not exist v41) ==== 162+0+0 (secure 0 0 0) 0x7fac94019540 con 0x7fac9c09d4e0 2024-08-19T10:39:48.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.057+0000 7fac997fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foooo", "pool2": "foooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:48.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.057+0000 7fac997fa640 10 monclient: _finish_command 3 = system:0 pool 'foooo' does not exist 2024-08-19T10:39:48.060 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'foooo' does not exist 2024-08-19T10:39:48.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 -- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8803be30 msgr2=0x7fac8803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:48.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8803be30 0x7fac8803e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fac90009d30 tx=0x7fac90008030 comp rx=0 tx=0).stop 2024-08-19T10:39:48.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 10 monclient: shutdown 2024-08-19T10:39:48.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:48.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 -- 172.21.15.190:0/1722372677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09d4e0 msgr2=0x7fac9c1d8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:48.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09d4e0 0x7fac9c1d8920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fac94006fd0 tx=0x7fac9400d7e0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 -- 172.21.15.190:0/1722372677 shutdown_connections 2024-08-19T10:39:48.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8803be30 0x7fac8803e2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac9c09d4e0 0x7fac9c1d8920 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac9c0a28e0 0x7fac9c09cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 --2- 172.21.15.190:0/1722372677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac84003640 0x7fac9c09ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 -- 172.21.15.190:0/1722372677 >> 172.21.15.190:0/1722372677 conn(0x7fac9c0abee0 msgr2=0x7fac9c136b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:48.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 -- 172.21.15.190:0/1722372677 shutdown_connections 2024-08-19T10:39:48.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.061+0000 7faca2b5e640 1 -- 172.21.15.190:0/1722372677 wait complete. 2024-08-19T10:39:48.077 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-08-19T10:39:48.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient: get_monmap_and_config 2024-08-19T10:39:48.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient: build_initial_monmap 2024-08-19T10:39:48.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient: monmap: 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:48.185165+0000 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:48.185165+0000 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:48.185 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:48.186 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:48.186 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:48.186 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:48.186 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:48.186 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:48.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 1 Processor -- start 2024-08-19T10:39:48.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 1 -- start start 2024-08-19T10:39:48.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient: init 2024-08-19T10:39:48.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:48.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient: _add_conns ranks=[2,3,9,4,1,10,0,5,6,7,8] 2024-08-19T10:39:48.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39940a3a00 0x7f39940a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:48.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient(hunting): picked mon.noname-b con 0x7f39940a3a00 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:48.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940a1f50 0x7f39940a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:48.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient(hunting): picked mon.noname-g con 0x7f39940a1f50 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:48.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f399413b7f0 0x7f399413dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:48.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient(hunting): picked mon.noname-a con 0x7f399413b7f0 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:48.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:48.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f39940a1d10 con 0x7f399413b7f0 2024-08-19T10:39:48.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:48.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f39940a4860 con 0x7f39940a3a00 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f399409da20 con 0x7f39940a1f50 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f399ab41640 10 monclient(hunting): authenticate will time out at 1033.755140s 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940a1f50 0x7f39940a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940a1f50 0x7f39940a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42272/0 (socket says 172.21.15.190:42272) 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 1 -- 172.21.15.190:0/2125396393 learned_addr learned my addr 172.21.15.190:0/2125396393 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 10 monclient(hunting): get_auth_request con 0x7f39940a1f50 auth_method 0 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:48.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:48.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:48.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:48.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f39988dd640 1 --2- 172.21.15.190:0/2125396393 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39940a3a00 0x7f39940a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:48.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f39990de640 1 --2- 172.21.15.190:0/2125396393 >> v2:172.21.15.89:3300/0 conn(0x7f399413b7f0 0x7f399413dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:48.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f39988dd640 10 monclient(hunting): get_auth_request con 0x7f39940a3a00 auth_method 0 2024-08-19T10:39:48.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f39988dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:48.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f39988dd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:48.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f39988dd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:48.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 10 monclient(hunting): handle_auth_done global_id 4629 payload 274 2024-08-19T10:39:48.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 1 -- 172.21.15.190:0/2125396393 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39940a3a00 msgr2=0x7f39940a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:48.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 1 --2- 172.21.15.190:0/2125396393 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39940a3a00 0x7f39940a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 1 -- 172.21.15.190:0/2125396393 >> v2:172.21.15.89:3300/0 conn(0x7f399413b7f0 msgr2=0x7f399413dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:48.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 1 --2- 172.21.15.190:0/2125396393 >> v2:172.21.15.89:3300/0 conn(0x7f399413b7f0 0x7f399413dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:48.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 1 monclient: found mon.noname-g 2024-08-19T10:39:48.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:48.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:48.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f3993fff640 1 -- 172.21.15.190:0/2125396393 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399409d180 con 0x7f39940a1f50 2024-08-19T10:39:48.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f39990de640 10 monclient: get_auth_request con 0x7f399413b7f0 auth_method 0 2024-08-19T10:39:48.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.181+0000 7f39990de640 1 --2- 172.21.15.190:0/2125396393 >> v2:172.21.15.89:3300/0 conn(0x7f399413b7f0 0x7f399413dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:39:48.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.185+0000 7f3993fff640 1 -- 172.21.15.190:0/2125396393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940a1f50 msgr2=0x7f39940a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:48.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.185+0000 7f3993fff640 1 --2- 172.21.15.190:0/2125396393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940a1f50 0x7f39940a4320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3988009870 tx=0x7f398801c3f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=aa1215baea5bb4b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:48.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 1 -- 172.21.15.190:0/2125396393 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39880b0070 con 0x7f39940a1f50 2024-08-19T10:39:48.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 10 monclient: dump: 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:48.200 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 5 monclient: authenticate success, global_id 4629 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 1 -- 172.21.15.190:0/2125396393 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39880044a0 con 0x7f39940a1f50 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:48.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:48.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:48.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 4 set_mon_vals no callback set 2024-08-19T10:39:48.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 1 -- 172.21.15.190:0/2125396393 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3988003a90 con 0x7f39940a1f50 2024-08-19T10:39:48.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:48.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:48.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39937fe640 10 monclient: dump: 2024-08-19T10:39:48.202 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:48.202 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:48.202 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:48.202 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:48.203 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:48.203 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:48.203 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:48.203 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:48.203 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:48.203 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:48.204 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:48.204 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:48.204 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:48.204 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:48.205 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:48.205 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:48.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient: shutdown 2024-08-19T10:39:48.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:48.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 -- 172.21.15.190:0/2125396393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940a1f50 msgr2=0x7f39940a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:48.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 --2- 172.21.15.190:0/2125396393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940a1f50 0x7f39940a4320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3988009870 tx=0x7f398801c3f0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 -- 172.21.15.190:0/2125396393 shutdown_connections 2024-08-19T10:39:48.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 --2- 172.21.15.190:0/2125396393 >> v2:172.21.15.89:3300/0 conn(0x7f399413b7f0 0x7f399413dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 --2- 172.21.15.190:0/2125396393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39940a1f50 0x7f39940a4320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 --2- 172.21.15.190:0/2125396393 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39940a3a00 0x7f39940a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 -- 172.21.15.190:0/2125396393 >> 172.21.15.190:0/2125396393 conn(0x7f39940a51d0 msgr2=0x7f399406c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:48.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 -- 172.21.15.190:0/2125396393 shutdown_connections 2024-08-19T10:39:48.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 -- 172.21.15.190:0/2125396393 wait complete. 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient: build_initial_monmap 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient: monmap: 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:48.207 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 Processor -- start 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 -- start start 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient: init 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient: _add_conns ranks=[0,6,3,2,7,5,1,4,8] 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39940a1f50 0x7f3994135bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient(hunting): picked mon.noname-a con 0x7f39940a1f50 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f39940a3a00 0x7f3994136130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:48.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient(hunting): picked mon.noname-d con 0x7f39940a3a00 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399413b7f0 0x7f399409f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient(hunting): picked mon.noname-g con 0x7f399413b7f0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39940a1a60 con 0x7f39940a1f50 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f399409ef80 con 0x7f399413b7f0 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f3994140940 con 0x7f39940a3a00 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f399ab41640 10 monclient(hunting): authenticate will time out at 1033.762378s 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399413b7f0 0x7f399409f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399413b7f0 0x7f399409f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42278/0 (socket says 172.21.15.190:42278) 2024-08-19T10:39:48.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 1 -- 172.21.15.190:0/2224125641 learned_addr learned my addr 172.21.15.190:0/2224125641 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 10 monclient(hunting): get_auth_request con 0x7f399413b7f0 auth_method 0 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39988dd640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39940a1f50 0x7f3994135bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f3993fff640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f39940a3a00 0x7f3994136130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 10 monclient(hunting): handle_auth_done global_id 4638 payload 824 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 1 -- 172.21.15.190:0/2224125641 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f39940a3a00 msgr2=0x7f3994136130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:48.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f39940a3a00 0x7f3994136130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 1 -- 172.21.15.190:0/2224125641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39940a1f50 msgr2=0x7f3994135bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39940a1f50 0x7f3994135bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 10 monclient: _finish_hunting 0 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 1 monclient: found mon.noname-g 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39990de640 1 -- 172.21.15.190:0/2224125641 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39941d10c0 con 0x7f399413b7f0 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39988dd640 10 monclient: get_auth_request con 0x7f39940a1f50 auth_method 0 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.189+0000 7f39988dd640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39940a1f50 0x7f3994135bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39990de640 1 -- 172.21.15.190:0/2224125641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399413b7f0 msgr2=0x7f399409f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39990de640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399413b7f0 0x7f399409f4f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f398c00cd80 tx=0x7f398c00a650 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 1 -- 172.21.15.190:0/2224125641 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f398c00fb00 con 0x7f399413b7f0 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:48.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 10 monclient: dump: 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:48.212 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 10 monclient: _finish_auth 0 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 10 monclient: _check_auth_tickets 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 1 -- 172.21.15.190:0/2224125641 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398c00fce0 con 0x7f399413b7f0 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 1 -- 172.21.15.190:0/2224125641 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f398c018870 con 0x7f399413b7f0 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f399ab41640 5 monclient: authenticate success, global_id 4638 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f399ab41640 10 monclient: _renew_subs 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f399ab41640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f399ab41640 1 -- 172.21.15.190:0/2224125641 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39941232d0 con 0x7f399413b7f0 2024-08-19T10:39:48.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:48.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f3992ffd640 4 set_mon_vals no callback set 2024-08-19T10:39:48.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:48.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 10 monclient: dump: 2024-08-19T10:39:48.214 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:48.214 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:48.214 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:48.214 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:48.214 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:48.214 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:48.215 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:48.215 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:48.215 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:48.215 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:48.215 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:48.215 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:48.215 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:48.215 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:48.215 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:48.215 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f399ab41640 10 monclient: _renew_subs 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f399ab41640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f399ab41640 1 -- 172.21.15.190:0/2224125641 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39940a77a0 con 0x7f399413b7f0 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 1 -- 172.21.15.190:0/2224125641 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f398c018400 con 0x7f399413b7f0 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f399ab41640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f399ab41640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f399ab41640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f399ab41640 1 -- 172.21.15.190:0/2224125641 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39940a7250 con 0x7f399413b7f0 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398003be30 0x7f398003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39917fa640 1 -- 172.21.15.190:0/2224125641 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(41..41 src has 1..41) ==== 5130+0+0 (secure 0 0 0) 0x7f398c04f960 con 0x7f399413b7f0 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39988dd640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398003be30 0x7f398003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:48.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.193+0000 7f39988dd640 10 monclient: get_auth_request con 0x7f398003be30 auth_method 0 2024-08-19T10:39:48.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.197+0000 7f39988dd640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398003be30 0x7f398003e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3984009840 tx=0x7f3984008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:48.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.197+0000 7f39917fa640 1 -- 172.21.15.190:0/2224125641 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f39941d10c0 con 0x7f399413b7f0 2024-08-19T10:39:48.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.197+0000 7f39917fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:48.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.197+0000 7f39917fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:48.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.353+0000 7f399ab41640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:48.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.353+0000 7f399ab41640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:48.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.353+0000 7f399ab41640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:48.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.353+0000 7f399ab41640 1 -- 172.21.15.190:0/2224125641 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f39940a3de0 con 0x7f399413b7f0 2024-08-19T10:39:48.944 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.937+0000 7f39917fa640 1 -- 172.21.15.190:0/2224125641 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' removed v42) ==== 152+0+0 (secure 0 0 0) 0x7f398c014030 con 0x7f399413b7f0 2024-08-19T10:39:48.944 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.937+0000 7f39917fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:48.944 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:48.937+0000 7f39917fa640 10 monclient: _finish_command 2 = system:0 pool 'fooo' removed 2024-08-19T10:39:49.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.089+0000 7f399ab41640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:49.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.089+0000 7f399ab41640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:49.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.089+0000 7f399ab41640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:49.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.089+0000 7f399ab41640 1 -- 172.21.15.190:0/2224125641 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f39940a0a30 con 0x7f399413b7f0 2024-08-19T10:39:49.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f39917fa640 1 -- 172.21.15.190:0/2224125641 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v42) ==== 159+0+0 (secure 0 0 0) 0x7f398c00d370 con 0x7f399413b7f0 2024-08-19T10:39:49.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f39917fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:49.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f39917fa640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-08-19T10:39:49.096 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'fooo' does not exist 2024-08-19T10:39:49.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 -- 172.21.15.190:0/2224125641 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398003be30 msgr2=0x7f398003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:49.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398003be30 0x7f398003e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3984009840 tx=0x7f3984008040 comp rx=0 tx=0).stop 2024-08-19T10:39:49.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 10 monclient: shutdown 2024-08-19T10:39:49.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:49.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 -- 172.21.15.190:0/2224125641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399413b7f0 msgr2=0x7f399409f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:49.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399413b7f0 0x7f399409f4f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f398c00cd80 tx=0x7f398c00a650 comp rx=0 tx=0).stop 2024-08-19T10:39:49.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 -- 172.21.15.190:0/2224125641 shutdown_connections 2024-08-19T10:39:49.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f398003be30 0x7f398003e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:49.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f399413b7f0 0x7f399409f4f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:49.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f39940a3a00 0x7f3994136130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:49.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 --2- 172.21.15.190:0/2224125641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39940a1f50 0x7f3994135bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:49.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 -- 172.21.15.190:0/2224125641 >> 172.21.15.190:0/2224125641 conn(0x7f39940a51d0 msgr2=0x7f39941368a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:49.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 -- 172.21.15.190:0/2224125641 shutdown_connections 2024-08-19T10:39:49.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.093+0000 7f399ab41640 1 -- 172.21.15.190:0/2224125641 wait complete. 2024-08-19T10:39:49.114 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete foo foo --yes-i-really-really-mean-it 2024-08-19T10:39:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient: get_monmap_and_config 2024-08-19T10:39:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient: build_initial_monmap 2024-08-19T10:39:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient: monmap: 2024-08-19T10:39:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:49.233021+0000 2024-08-19T10:39:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:49.233021+0000 2024-08-19T10:39:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:49.233 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 1 Processor -- start 2024-08-19T10:39:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 1 -- start start 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient: init 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient: _add_conns ranks=[1,5,2,7,3,6,4,8,0,10,9] 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f97d80a36a0 addr v1:172.21.15.190:6789/0 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97d80a19d0 0x7f97d813c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient(hunting): picked mon.noname-h con 0x7f97d80a19d0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97d80a4190 0x7f97d813ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient(hunting): picked mon.noname-b con 0x7f97d80a4190 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d809d050 con 0x7f97d80a36a0 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f97d806f250 con 0x7f97d80a4190 2024-08-19T10:39:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d7fff640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f97d80a36a0 legacy=0x7f97d80a3a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:39:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f97d809dec0 con 0x7f97d80a19d0 2024-08-19T10:39:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97de8e4640 10 monclient(hunting): authenticate will time out at 1034.803015s 2024-08-19T10:39:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97d80a19d0 0x7f97d813c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97d80a19d0 0x7f97d813c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36856/0 (socket says 172.21.15.190:36856) 2024-08-19T10:39:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 1 -- 172.21.15.190:0/234365187 learned_addr learned my addr 172.21.15.190:0/234365187 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 10 monclient(hunting): get_auth_request con 0x7f97d80a19d0 auth_method 0 2024-08-19T10:39:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 10 monclient(hunting): handle_auth_done global_id 4667 payload 274 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 1 -- 172.21.15.190:0/234365187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97d80a4190 msgr2=0x7f97d813ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 1 --2- 172.21.15.190:0/234365187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97d80a4190 0x7f97d813ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 1 -- 172.21.15.190:0/234365187 >> v1:172.21.15.190:6789/0 conn(0x7f97d80a36a0 legacy=0x7f97d80a3a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 1 monclient: found mon.noname-h 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.229+0000 7f97d77fe640 1 -- 172.21.15.190:0/234365187 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d809d620 con 0x7f97d80a19d0 2024-08-19T10:39:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.233+0000 7f97d77fe640 1 -- 172.21.15.190:0/234365187 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97d80a19d0 msgr2=0x7f97d813c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.233+0000 7f97d77fe640 1 --2- 172.21.15.190:0/234365187 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97d80a19d0 0x7f97d813c8c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f97c80098a0 tx=0x7f97c801c440 comp rx=0 tx=0).ready entity=mon.5 client_cookie=ce1413e3d7fecfc8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.233+0000 7f97d6ffd640 1 -- 172.21.15.190:0/234365187 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97c80b0070 con 0x7f97d80a19d0 2024-08-19T10:39:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 10 monclient: dump: 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 1 -- 172.21.15.190:0/234365187 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c80aa810 con 0x7f97d80a19d0 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 1 -- 172.21.15.190:0/234365187 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97c80a93e0 con 0x7f97d80a19d0 2024-08-19T10:39:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 5 monclient: authenticate success, global_id 4667 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 4 set_mon_vals no callback set 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d6ffd640 10 monclient: dump: 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:49.246 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:49.246 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:49.246 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:49.246 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:49.246 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:49.246 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:49.246 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:49.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient: shutdown 2024-08-19T10:39:49.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 -- 172.21.15.190:0/234365187 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97d80a19d0 msgr2=0x7f97d813c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 --2- 172.21.15.190:0/234365187 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97d80a19d0 0x7f97d813c8c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f97c80098a0 tx=0x7f97c801c440 comp rx=0 tx=0).stop 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 -- 172.21.15.190:0/234365187 shutdown_connections 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 --2- 172.21.15.190:0/234365187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97d80a4190 0x7f97d813ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 --2- 172.21.15.190:0/234365187 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97d80a19d0 0x7f97d813c8c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 -- 172.21.15.190:0/234365187 >> 172.21.15.190:0/234365187 conn(0x7f97d80a4790 msgr2=0x7f97d80a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 -- 172.21.15.190:0/234365187 shutdown_connections 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 -- 172.21.15.190:0/234365187 wait complete. 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient: build_initial_monmap 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient: monmap: 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:49.247 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:49.248 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 Processor -- start 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 -- start start 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient: init 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient: _add_conns ranks=[3,0,7,6,2,5,4,1,8] 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d80a19d0 0x7f97d8126830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient(hunting): picked mon.noname-g con 0x7f97d80a19d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97d80a36a0 0x7f97d81cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient(hunting): picked mon.noname-a con 0x7f97d80a36a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d80a4190 0x7f97d81243b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient(hunting): picked mon.noname-i con 0x7f97d80a4190 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:49.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8141cf0 con 0x7f97d80a36a0 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f97d8141b70 con 0x7f97d80a19d0 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f97d8141e70 con 0x7f97d80a4190 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97de8e4640 10 monclient(hunting): authenticate will time out at 1034.810648s 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97dce81640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d80a4190 msgr2=0x7f97d81243b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97dce81640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d80a4190 0x7f97d81243b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d80a19d0 0x7f97d8126830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d80a19d0 0x7f97d8126830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42292/0 (socket says 172.21.15.190:42292) 2024-08-19T10:39:49.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 1 -- 172.21.15.190:0/1326772622 learned_addr learned my addr 172.21.15.190:0/1326772622 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 10 monclient(hunting): get_auth_request con 0x7f97d80a19d0 auth_method 0 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d77fe640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97d80a36a0 0x7f97d81cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d77fe640 10 monclient(hunting): get_auth_request con 0x7f97d80a36a0 auth_method 0 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:49.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 10 monclient(hunting): handle_auth_done global_id 4647 payload 824 2024-08-19T10:39:49.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 1 -- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d80a4190 msgr2=0x7f97d81243b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:49.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d80a4190 0x7f97d81243b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:49.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 1 -- 172.21.15.190:0/1326772622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97d80a36a0 msgr2=0x7f97d81cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:49.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97d80a36a0 0x7f97d81cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:49.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:49.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 1 monclient: found mon.noname-g 2024-08-19T10:39:49.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:49.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:49.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d7fff640 1 -- 172.21.15.190:0/1326772622 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d80a3020 con 0x7f97d80a19d0 2024-08-19T10:39:49.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.237+0000 7f97d77fe640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97d80a36a0 0x7f97d81cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:39:49.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d7fff640 1 -- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d80a19d0 msgr2=0x7f97d8126830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:49.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d7fff640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d80a19d0 0x7f97d8126830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97c000b550 tx=0x7f97c000ba10 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:49.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1326772622 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97c00092d0 con 0x7f97d80a19d0 2024-08-19T10:39:49.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:49.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:49.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 10 monclient: dump: 2024-08-19T10:39:49.255 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:49.255 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:49.255 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:49.255 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:49.256 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:49.256 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:49.256 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:49.256 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:49.256 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:49.256 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:49.256 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:49.256 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:49.256 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:49.256 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:49.257 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 10 monclient: _finish_auth 0 2024-08-19T10:39:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:39:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1326772622 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c0015070 con 0x7f97d80a19d0 2024-08-19T10:39:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97de8e4640 5 monclient: authenticate success, global_id 4647 2024-08-19T10:39:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97de8e4640 10 monclient: _renew_subs 2024-08-19T10:39:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97de8e4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97de8e4640 1 -- 172.21.15.190:0/1326772622 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d81271c0 con 0x7f97d80a19d0 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1326772622 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97c0003e00 con 0x7f97d80a19d0 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d67fc640 4 set_mon_vals no callback set 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 10 monclient: dump: 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97de8e4640 10 monclient: _renew_subs 2024-08-19T10:39:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97de8e4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97de8e4640 1 -- 172.21.15.190:0/1326772622 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d8128030 con 0x7f97d80a19d0 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97de8e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97de8e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97de8e4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97de8e4640 1 -- 172.21.15.190:0/1326772622 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d813eb30 con 0x7f97d80a19d0 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1326772622 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f97c00050f0 con 0x7f97d80a19d0 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97b803be00 0x7f97b803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1326772622 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(42..42 src has 1..42) ==== 4780+0+0 (secure 0 0 0) 0x7f97c000d040 con 0x7f97d80a19d0 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d77fe640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97b803be00 0x7f97b803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.241+0000 7f97d77fe640 10 monclient: get_auth_request con 0x7f97b803be00 auth_method 0 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.245+0000 7f97d77fe640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97b803be00 0x7f97b803e2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f97d80a2000 tx=0x7f97c80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:49.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:49.245+0000 7f97d7fff640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d80a19d0 0x7f97d8126830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97c000b550 tx=0x7f97c000ba10 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f97d813eb30 mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) 2024-08-19T10:39:50.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:50.077+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1326772622 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f97d813eb30 con 0x7f97d80a19d0 2024-08-19T10:39:50.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:50.077+0000 7f97d4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:50.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:50.077+0000 7f97d4ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:50.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:50.281+0000 7f97de8e4640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:50.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:50.281+0000 7f97de8e4640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:50.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:50.281+0000 7f97de8e4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:50.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:50.281+0000 7f97de8e4640 1 -- 172.21.15.190:0/1326772622 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f97d81c6d00 con 0x7f97d80a19d0 2024-08-19T10:39:51.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.261+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1326772622 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' removed v44) ==== 149+0+0 (secure 0 0 0) 0x7f97c00104a0 con 0x7f97d80a19d0 2024-08-19T10:39:51.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.261+0000 7f97d4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:51.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.261+0000 7f97d4ff9640 10 monclient: _finish_command 2 = system:0 pool 'foo' removed 2024-08-19T10:39:51.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.425+0000 7f97de8e4640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:51.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.425+0000 7f97de8e4640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:51.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.425+0000 7f97de8e4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:51.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.425+0000 7f97de8e4640 1 -- 172.21.15.190:0/1326772622 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f97d80a3a80 con 0x7f97d80a19d0 2024-08-19T10:39:51.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1326772622 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v44) ==== 156+0+0 (secure 0 0 0) 0x7f97c0005670 con 0x7f97d80a19d0 2024-08-19T10:39:51.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97d4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:51.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97d4ff9640 10 monclient: _finish_command 3 = system:0 pool 'foo' does not exist 2024-08-19T10:39:51.443 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'foo' does not exist 2024-08-19T10:39:51.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 -- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97b803be00 msgr2=0x7f97b803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:51.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97b803be00 0x7f97b803e2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f97d80a2000 tx=0x7f97c80a6000 comp rx=0 tx=0).stop 2024-08-19T10:39:51.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 10 monclient: shutdown 2024-08-19T10:39:51.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:51.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 -- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d80a19d0 msgr2=0x7f97d8126830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:51.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d80a19d0 0x7f97d8126830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97c000b550 tx=0x7f97c000ba10 comp rx=0 tx=0).stop 2024-08-19T10:39:51.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 -- 172.21.15.190:0/1326772622 shutdown_connections 2024-08-19T10:39:51.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97b803be00 0x7f97b803e2c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f97d80a36a0 0x7f97d81cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d80a4190 0x7f97d81243b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 --2- 172.21.15.190:0/1326772622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d80a19d0 0x7f97d8126830 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 -- 172.21.15.190:0/1326772622 >> 172.21.15.190:0/1326772622 conn(0x7f97d80a4790 msgr2=0x7f97d80708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:51.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 -- 172.21.15.190:0/1326772622 shutdown_connections 2024-08-19T10:39:51.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.441+0000 7f97de8e4640 1 -- 172.21.15.190:0/1326772622 wait complete. 2024-08-19T10:39:51.460 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete foo foo --yes-i-really-really-mean-it 2024-08-19T10:39:51.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient: get_monmap_and_config 2024-08-19T10:39:51.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient: build_initial_monmap 2024-08-19T10:39:51.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient: monmap: 2024-08-19T10:39:51.579 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:51.579 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:51.579 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:51.580411+0000 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:51.580411+0000 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:51.580 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:51.581 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:51.581 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:51.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 1 Processor -- start 2024-08-19T10:39:51.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 1 -- start start 2024-08-19T10:39:51.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient: init 2024-08-19T10:39:51.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:51.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient: _add_conns ranks=[4,6,0,1,3,9,5,8,2,10,7] 2024-08-19T10:39:51.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe380a58d0 0x7fbe380a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient(hunting): picked mon.noname-c con 0x7fbe380a58d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe380722b0 0x7fbe380a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient(hunting): picked mon.noname-d con 0x7fbe380722b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbe38137680 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe380a40f0 con 0x7fbe38137680 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbe38072730 con 0x7fbe380a58d0 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fbe380a6730 con 0x7fbe380722b0 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3ec36640 10 monclient(hunting): authenticate will time out at 1037.150237s 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe380a58d0 0x7fbe380a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:51.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3d1d3640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbe38137680 0x7fbe38137b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:39832/0 (socket says 172.21.15.190:39832) 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe2ffff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe380722b0 0x7fbe380a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3d1d3640 1 -- 172.21.15.190:0/2129047839 learned_addr learned my addr 172.21.15.190:0/2129047839 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 10 monclient(hunting): get_auth_request con 0x7fbe380a58d0 auth_method 0 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe2ffff640 10 monclient(hunting): get_auth_request con 0x7fbe380722b0 auth_method 0 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe2ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe2ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe2ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:51.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe2ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe2ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe2ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 10 monclient(hunting): handle_auth_done global_id 4720 payload 274 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 1 -- 172.21.15.190:0/2129047839 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe380722b0 msgr2=0x7fbe380a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 1 --2- 172.21.15.190:0/2129047839 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe380722b0 0x7fbe380a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 1 -- 172.21.15.190:0/2129047839 >> v1:172.21.15.89:6789/0 conn(0x7fbe38137680 legacy=0x7fbe38137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 10 monclient: _finish_hunting 0 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 1 monclient: found mon.noname-c 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:51.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe3c9d2640 1 -- 172.21.15.190:0/2129047839 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe38073d90 con 0x7fbe380a58d0 2024-08-19T10:39:51.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.577+0000 7fbe2ffff640 1 --2- 172.21.15.190:0/2129047839 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe380722b0 0x7fbe380a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:51.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.581+0000 7fbe3c9d2640 1 -- 172.21.15.190:0/2129047839 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe380a58d0 msgr2=0x7fbe380a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:51.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.581+0000 7fbe3c9d2640 1 --2- 172.21.15.190:0/2129047839 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe380a58d0 0x7fbe380a5cb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe340097f0 tx=0x7fbe3401c160 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4000eaca786a84a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:51.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 1 -- 172.21.15.190:0/2129047839 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbe340b1070 con 0x7fbe380a58d0 2024-08-19T10:39:51.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:51.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:51.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 10 monclient: dump: 2024-08-19T10:39:51.592 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:51.592 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:51.592 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:51.593 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:51.593 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:51.593 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:51.593 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:51.593 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:51.593 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:51.593 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:51.593 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:51.593 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 1 -- 172.21.15.190:0/2129047839 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe340a4c90 con 0x7fbe380a58d0 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 5 monclient: authenticate success, global_id 4720 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 1 -- 172.21.15.190:0/2129047839 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbe340aac20 con 0x7fbe380a58d0 2024-08-19T10:39:51.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 4 set_mon_vals no callback set 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2f7fe640 10 monclient: dump: 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:51.595 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient: shutdown 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 -- 172.21.15.190:0/2129047839 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe380a58d0 msgr2=0x7fbe380a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 --2- 172.21.15.190:0/2129047839 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe380a58d0 0x7fbe380a5cb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe340097f0 tx=0x7fbe3401c160 comp rx=0 tx=0).stop 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 -- 172.21.15.190:0/2129047839 shutdown_connections 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 --2- 172.21.15.190:0/2129047839 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe380722b0 0x7fbe380a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 --2- 172.21.15.190:0/2129047839 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe380a58d0 0x7fbe380a5cb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 -- 172.21.15.190:0/2129047839 >> 172.21.15.190:0/2129047839 conn(0x7fbe380aa4c0 msgr2=0x7fbe380a69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 -- 172.21.15.190:0/2129047839 shutdown_connections 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 -- 172.21.15.190:0/2129047839 wait complete. 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient: build_initial_monmap 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient: monmap: 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:51.597 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 Processor -- start 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 -- start start 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient: init 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient: _add_conns ranks=[1,2,8,3,0,6,7,5,4] 2024-08-19T10:39:51.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380722b0 0x7fbe3813da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient(hunting): picked mon.noname-f con 0x7fbe380722b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe380a58d0 0x7fbe381cf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient(hunting): picked mon.noname-b con 0x7fbe380a58d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbe38137680 0x7fbe3813b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient(hunting): picked mon.noname-e con 0x7fbe38137680 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38074610 con 0x7fbe380722b0 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fbe38074490 con 0x7fbe380a58d0 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fbe38142100 con 0x7fbe38137680 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:51.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3ec36640 10 monclient(hunting): authenticate will time out at 1037.158334s 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3c9d2640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380722b0 msgr2=0x7fbe3813da60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3c9d2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380722b0 0x7fbe3813da60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe380a58d0 0x7fbe381cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe380a58d0 0x7fbe381cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:33488/0 (socket says 172.21.15.190:33488) 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 1 -- 172.21.15.190:0/3085599762 learned_addr learned my addr 172.21.15.190:0/3085599762 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 10 monclient(hunting): get_auth_request con 0x7fbe380a58d0 auth_method 0 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3d1d3640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbe38137680 0x7fbe3813b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:51.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:51.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3d1d3640 10 monclient(hunting): get_auth_request con 0x7fbe38137680 auth_method 0 2024-08-19T10:39:51.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3d1d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:51.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3d1d3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:51.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3d1d3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:51.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3d1d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:51.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3d1d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:51.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe3d1d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:51.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 10 monclient(hunting): handle_auth_done global_id 4637 payload 824 2024-08-19T10:39:51.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 1 -- 172.21.15.190:0/3085599762 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbe38137680 msgr2=0x7fbe3813b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:51.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbe38137680 0x7fbe3813b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 1 -- 172.21.15.190:0/3085599762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380722b0 msgr2=0x7fbe3813da60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:51.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380722b0 0x7fbe3813da60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:51.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 1 monclient: found mon.noname-b 2024-08-19T10:39:51.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:51.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:51.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.585+0000 7fbe2ffff640 1 -- 172.21.15.190:0/3085599762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3813e3f0 con 0x7fbe380a58d0 2024-08-19T10:39:51.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2ffff640 1 -- 172.21.15.190:0/3085599762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe380a58d0 msgr2=0x7fbe381cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:51.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2ffff640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe380a58d0 0x7fbe381cf700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800e960 tx=0x7fbe2800ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:51.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 1 -- 172.21.15.190:0/3085599762 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbe28017070 con 0x7fbe380a58d0 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 10 monclient: dump: 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:51.605 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 1 -- 172.21.15.190:0/3085599762 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe2800cc10 con 0x7fbe380a58d0 2024-08-19T10:39:51.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe3ec36640 5 monclient: authenticate success, global_id 4637 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe3ec36640 10 monclient: _renew_subs 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe3ec36640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe3ec36640 1 -- 172.21.15.190:0/3085599762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe380a3810 con 0x7fbe380a58d0 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 1 -- 172.21.15.190:0/3085599762 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbe2800cdc0 con 0x7fbe380a58d0 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 10 monclient: dump: 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:51.607 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe3ec36640 10 monclient: _renew_subs 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe3ec36640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:51.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe3ec36640 1 -- 172.21.15.190:0/3085599762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe38120e70 con 0x7fbe380a58d0 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2effd640 4 set_mon_vals no callback set 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe3ec36640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe3ec36640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe3ec36640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe3ec36640 1 -- 172.21.15.190:0/3085599762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe38137b10 con 0x7fbe380a58d0 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 1 -- 172.21.15.190:0/3085599762 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbe2801a070 con 0x7fbe380a58d0 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.589+0000 7fbe2d7fa640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe2403be30 0x7fbe2403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.593+0000 7fbe2d7fa640 1 -- 172.21.15.190:0/3085599762 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 4438+0+0 (secure 0 0 0) 0x7fbe2800f040 con 0x7fbe380a58d0 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.593+0000 7fbe3c9d2640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe2403be30 0x7fbe2403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.593+0000 7fbe3c9d2640 10 monclient: get_auth_request con 0x7fbe2403be30 auth_method 0 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.593+0000 7fbe3c9d2640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe2403be30 0x7fbe2403e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe380a76c0 tx=0x7fbe340a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.601+0000 7fbe2d7fa640 1 -- 172.21.15.190:0/3085599762 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fbe28011350 con 0x7fbe380a58d0 2024-08-19T10:39:51.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.601+0000 7fbe2d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:51.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.601+0000 7fbe2d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:51.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.773+0000 7fbe3ec36640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:51.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.773+0000 7fbe3ec36640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:51.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.773+0000 7fbe3ec36640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:51.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.773+0000 7fbe3ec36640 1 -- 172.21.15.190:0/3085599762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbe38136b60 con 0x7fbe380a58d0 2024-08-19T10:39:51.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.777+0000 7fbe2d7fa640 1 -- 172.21.15.190:0/3085599762 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v44) ==== 156+0+0 (secure 0 0 0) 0x7fbe2801be20 con 0x7fbe380a58d0 2024-08-19T10:39:51.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.777+0000 7fbe2d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:51.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.777+0000 7fbe2d7fa640 10 monclient: _finish_command 2 = system:0 pool 'foo' does not exist 2024-08-19T10:39:51.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.877+0000 7fbe3ec36640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:51.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.877+0000 7fbe3ec36640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:51.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.877+0000 7fbe3ec36640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:51.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.877+0000 7fbe3ec36640 1 -- 172.21.15.190:0/3085599762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbe380008e0 con 0x7fbe380a58d0 2024-08-19T10:39:51.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.877+0000 7fbe2d7fa640 1 -- 172.21.15.190:0/3085599762 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}]=0 pool 'foo' does not exist v44) ==== 156+0+0 (secure 0 0 0) 0x7fbe2804c7d0 con 0x7fbe380a58d0 2024-08-19T10:39:51.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.877+0000 7fbe2d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "foo", "pool2": "foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:51.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.877+0000 7fbe2d7fa640 10 monclient: _finish_command 3 = system:0 pool 'foo' does not exist 2024-08-19T10:39:51.882 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'foo' does not exist 2024-08-19T10:39:51.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 -- 172.21.15.190:0/3085599762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe2403be30 msgr2=0x7fbe2403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:51.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe2403be30 0x7fbe2403e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe380a76c0 tx=0x7fbe340a6070 comp rx=0 tx=0).stop 2024-08-19T10:39:51.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 10 monclient: shutdown 2024-08-19T10:39:51.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:51.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 -- 172.21.15.190:0/3085599762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe380a58d0 msgr2=0x7fbe381cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:51.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe380a58d0 0x7fbe381cf700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800e960 tx=0x7fbe2800ee20 comp rx=0 tx=0).stop 2024-08-19T10:39:51.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 -- 172.21.15.190:0/3085599762 shutdown_connections 2024-08-19T10:39:51.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe2403be30 0x7fbe2403e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbe38137680 0x7fbe3813b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe380a58d0 0x7fbe381cf700 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 --2- 172.21.15.190:0/3085599762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380722b0 0x7fbe3813da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:51.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 -- 172.21.15.190:0/3085599762 >> 172.21.15.190:0/3085599762 conn(0x7fbe380aa4c0 msgr2=0x7fbe38072c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:51.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 -- 172.21.15.190:0/3085599762 shutdown_connections 2024-08-19T10:39:51.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:51.881+0000 7fbe3ec36640 1 -- 172.21.15.190:0/3085599762 wait complete. 2024-08-19T10:39:51.899 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-08-19T10:39:52.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient: get_monmap_and_config 2024-08-19T10:39:52.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient: build_initial_monmap 2024-08-19T10:39:52.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient: monmap: 2024-08-19T10:39:52.020 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:52.020 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.020 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:52.021339+0000 2024-08-19T10:39:52.020 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:52.021339+0000 2024-08-19T10:39:52.020 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:52.021 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 1 Processor -- start 2024-08-19T10:39:52.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 1 -- start start 2024-08-19T10:39:52.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient: init 2024-08-19T10:39:52.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient: _add_conns ranks=[4,2,1,10,7,9,8,0,5,6,3] 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e2c1396a0 0x7f6e2c139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient(hunting): picked mon.noname-c con 0x7f6e2c1396a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e2c139fc0 0x7f6e2c0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient(hunting): picked mon.noname-b con 0x7f6e2c139fc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6e2c0a6e20 addr v1:172.21.15.190:6789/0 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2c0a83e0 con 0x7f6e2c0a6e20 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2c0a4f20 con 0x7f6e2c139fc0 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6e2c0a2f30 con 0x7f6e2c1396a0 2024-08-19T10:39:52.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:52.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e33570640 10 monclient(hunting): authenticate will time out at 1037.591228s 2024-08-19T10:39:52.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e31b0d640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f6e2c0a6e20 legacy=0x7f6e2c0a7220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:39:52.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e30b0b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e2c139fc0 0x7f6e2c0a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e3130c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e2c1396a0 0x7f6e2c139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e30b0b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e2c139fc0 0x7f6e2c0a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:33492/0 (socket says 172.21.15.190:33492) 2024-08-19T10:39:52.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e30b0b640 1 -- 172.21.15.190:0/3859167397 learned_addr learned my addr 172.21.15.190:0/3859167397 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:52.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e30b0b640 10 monclient(hunting): get_auth_request con 0x7f6e2c139fc0 auth_method 0 2024-08-19T10:39:52.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e30b0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e30b0b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e30b0b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e3130c640 10 monclient(hunting): get_auth_request con 0x7f6e2c1396a0 auth_method 0 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e3130c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e3130c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e3130c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e30b0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e30b0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e30b0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e3130c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e3130c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:52.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.017+0000 7f6e3130c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:52.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.021+0000 7f6e30b0b640 10 monclient(hunting): handle_auth_done global_id 4646 payload 274 2024-08-19T10:39:52.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.021+0000 7f6e30b0b640 1 -- 172.21.15.190:0/3859167397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e2c1396a0 msgr2=0x7f6e2c139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:52.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.021+0000 7f6e30b0b640 1 --2- 172.21.15.190:0/3859167397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e2c1396a0 0x7f6e2c139a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.021+0000 7f6e30b0b640 1 -- 172.21.15.190:0/3859167397 >> v1:172.21.15.190:6789/0 conn(0x7f6e2c0a6e20 legacy=0x7f6e2c0a7220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:52.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.021+0000 7f6e30b0b640 10 monclient: _finish_hunting 0 2024-08-19T10:39:52.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.021+0000 7f6e30b0b640 1 monclient: found mon.noname-b 2024-08-19T10:39:52.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.021+0000 7f6e30b0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:52.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.021+0000 7f6e30b0b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:52.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.021+0000 7f6e30b0b640 1 -- 172.21.15.190:0/3859167397 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c0a35e0 con 0x7f6e2c139fc0 2024-08-19T10:39:52.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e30b0b640 1 -- 172.21.15.190:0/3859167397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e2c139fc0 msgr2=0x7f6e2c0a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:52.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e30b0b640 1 --2- 172.21.15.190:0/3859167397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e2c139fc0 0x7f6e2c0a68e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0098a0 tx=0x7f6e1c01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8300a3c331fe7149 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:52.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 1 -- 172.21.15.190:0/3859167397 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6e1c0b0070 con 0x7f6e2c139fc0 2024-08-19T10:39:52.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:52.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 10 monclient: dump: 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.033 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 10 monclient: _finish_auth 0 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 1 -- 172.21.15.190:0/3859167397 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1c0aa810 con 0x7f6e2c139fc0 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 1 -- 172.21.15.190:0/3859167397 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6e1c0aae30 con 0x7f6e2c139fc0 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e23fff640 10 monclient: dump: 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.034 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.035 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.036 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 5 monclient: authenticate success, global_id 4646 2024-08-19T10:39:52.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:52.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 4 set_mon_vals no callback set 2024-08-19T10:39:52.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 10 monclient: shutdown 2024-08-19T10:39:52.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:52.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 1 -- 172.21.15.190:0/3859167397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e2c139fc0 msgr2=0x7f6e2c0a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:52.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 1 --2- 172.21.15.190:0/3859167397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e2c139fc0 0x7f6e2c0a68e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0098a0 tx=0x7f6e1c01c440 comp rx=0 tx=0).stop 2024-08-19T10:39:52.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 1 -- 172.21.15.190:0/3859167397 shutdown_connections 2024-08-19T10:39:52.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 1 --2- 172.21.15.190:0/3859167397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6e2c139fc0 0x7f6e2c0a68e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 1 --2- 172.21.15.190:0/3859167397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6e2c1396a0 0x7f6e2c139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 1 -- 172.21.15.190:0/3859167397 >> 172.21.15.190:0/3859167397 conn(0x7f6e2c0abee0 msgr2=0x7f6e2c0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:52.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 1 -- 172.21.15.190:0/3859167397 shutdown_connections 2024-08-19T10:39:52.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 1 -- 172.21.15.190:0/3859167397 wait complete. 2024-08-19T10:39:52.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 10 monclient: build_initial_monmap 2024-08-19T10:39:52.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.025+0000 7f6e33570640 10 monclient: monmap: 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:52.038 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 1 Processor -- start 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 1 -- start start 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient: init 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient: _add_conns ranks=[0,7,5,3,1,6,8,4,2] 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e2c0a6e20 0x7f6e2c1d1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient(hunting): picked mon.noname-a con 0x7f6e2c0a6e20 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6e2c1396a0 0x7f6e2c1389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient(hunting): picked mon.noname-i con 0x7f6e2c1396a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:52.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6e2c139fc0 0x7f6e2c13dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient(hunting): picked mon.noname-h con 0x7f6e2c139fc0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c13a3a0 con 0x7f6e2c0a6e20 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6e2c0a7930 con 0x7f6e2c139fc0 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6e2c0ac3a0 con 0x7f6e2c1396a0 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient(hunting): authenticate will time out at 1037.600854s 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e30b0b640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6e2c1396a0 msgr2=0x7f6e2c1389f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e30b0b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6e2c1396a0 0x7f6e2c1389f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6e2c139fc0 0x7f6e2c13dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6e2c139fc0 0x7f6e2c13dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36864/0 (socket says 172.21.15.190:36864) 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 1 -- 172.21.15.190:0/842525308 learned_addr learned my addr 172.21.15.190:0/842525308 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 10 monclient(hunting): get_auth_request con 0x7f6e2c139fc0 auth_method 0 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e3130c640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e2c0a6e20 0x7f6e2c1d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e3130c640 10 monclient(hunting): get_auth_request con 0x7f6e2c0a6e20 auth_method 0 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e3130c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e3130c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e3130c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:52.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 10 monclient(hunting): handle_auth_done global_id 4676 payload 824 2024-08-19T10:39:52.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 1 -- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6e2c1396a0 msgr2=0x7f6e2c1389f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:52.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6e2c1396a0 0x7f6e2c1389f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 1 -- 172.21.15.190:0/842525308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e2c0a6e20 msgr2=0x7f6e2c1d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:52.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e2c0a6e20 0x7f6e2c1d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 10 monclient: _finish_hunting 0 2024-08-19T10:39:52.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 1 monclient: found mon.noname-h 2024-08-19T10:39:52.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:52.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:52.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 1 -- 172.21.15.190:0/842525308 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c123380 con 0x7f6e2c139fc0 2024-08-19T10:39:52.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 1 -- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6e2c139fc0 msgr2=0x7f6e2c13dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:52.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e31b0d640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6e2c139fc0 0x7f6e2c13dbc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400be90 tx=0x7f6e2400bf80 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:52.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e21ffb640 1 -- 172.21.15.190:0/842525308 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6e24015850 con 0x7f6e2c139fc0 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e21ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e21ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e21ffb640 10 monclient: dump: 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.045 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.046 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.046 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.046 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.046 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.046 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.046 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.046 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e21ffb640 10 monclient: _finish_auth 0 2024-08-19T10:39:52.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e21ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:39:52.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e21ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:52.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e21ffb640 1 -- 172.21.15.190:0/842525308 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e24010030 con 0x7f6e2c139fc0 2024-08-19T10:39:52.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 5 monclient: authenticate success, global_id 4676 2024-08-19T10:39:52.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient: _renew_subs 2024-08-19T10:39:52.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:52.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e33570640 1 -- 172.21.15.190:0/842525308 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c0a2690 con 0x7f6e2c139fc0 2024-08-19T10:39:52.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.029+0000 7f6e21ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:52.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e21ffb640 1 -- 172.21.15.190:0/842525308 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6e240060d0 con 0x7f6e2c139fc0 2024-08-19T10:39:52.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e21ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e21ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e237fe640 4 set_mon_vals no callback set 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e21ffb640 10 monclient: dump: 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.048 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e33570640 10 monclient: _renew_subs 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e33570640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e33570640 1 -- 172.21.15.190:0/842525308 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c1230a0 con 0x7f6e2c139fc0 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e33570640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e33570640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e33570640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e33570640 1 -- 172.21.15.190:0/842525308 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c13a520 con 0x7f6e2c139fc0 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e21ffb640 1 -- 172.21.15.190:0/842525308 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6e240153e0 con 0x7f6e2c139fc0 2024-08-19T10:39:52.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e21ffb640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6e1003be00 0x7f6e1003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e21ffb640 1 -- 172.21.15.190:0/842525308 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(44..44 src has 1..44) ==== 4438+0+0 (secure 0 0 0) 0x7f6e2404ee20 con 0x7f6e2c139fc0 2024-08-19T10:39:52.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e3130c640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6e1003be00 0x7f6e1003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.033+0000 7f6e3130c640 10 monclient: get_auth_request con 0x7f6e1003be00 auth_method 0 2024-08-19T10:39:52.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.037+0000 7f6e3130c640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6e1003be00 0x7f6e1003e2c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009af0 tx=0x7f6e28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:52.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.037+0000 7f6e21ffb640 1 -- 172.21.15.190:0/842525308 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6e2c13a520 con 0x7f6e2c139fc0 2024-08-19T10:39:52.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.037+0000 7f6e21ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:52.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.037+0000 7f6e21ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:52.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.205+0000 7f6e33570640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.205+0000 7f6e33570640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.205+0000 7f6e33570640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:52.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.205+0000 7f6e33570640 1 -- 172.21.15.190:0/842525308 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6e2c0ac5d0 con 0x7f6e2c139fc0 2024-08-19T10:39:52.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.209+0000 7f6e21ffb640 1 -- 172.21.15.190:0/842525308 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v44) ==== 159+0+0 (secure 0 0 0) 0x7f6e2c0ac5d0 con 0x7f6e2c139fc0 2024-08-19T10:39:52.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.209+0000 7f6e21ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.209+0000 7f6e21ffb640 10 monclient: _finish_command 2 = system:0 pool 'fooo' does not exist 2024-08-19T10:39:52.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.309+0000 7f6e33570640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.309+0000 7f6e33570640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.309+0000 7f6e33570640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:52.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.309+0000 7f6e33570640 1 -- 172.21.15.190:0/842525308 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6e2c139a80 con 0x7f6e2c139fc0 2024-08-19T10:39:52.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.309+0000 7f6e21ffb640 1 -- 172.21.15.190:0/842525308 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v44) ==== 159+0+0 (secure 0 0 0) 0x7f6e2403f020 con 0x7f6e2c139fc0 2024-08-19T10:39:52.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.309+0000 7f6e21ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.309+0000 7f6e21ffb640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-08-19T10:39:52.314 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'fooo' does not exist 2024-08-19T10:39:52.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 -- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6e1003be00 msgr2=0x7f6e1003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6e1003be00 0x7f6e1003e2c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009af0 tx=0x7f6e28008040 comp rx=0 tx=0).stop 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 10 monclient: shutdown 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 -- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6e2c139fc0 msgr2=0x7f6e2c13dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6e2c139fc0 0x7f6e2c13dbc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400be90 tx=0x7f6e2400bf80 comp rx=0 tx=0).stop 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 -- 172.21.15.190:0/842525308 shutdown_connections 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6e1003be00 0x7f6e1003e2c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6e2c139fc0 0x7f6e2c13dbc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6e2c1396a0 0x7f6e2c1389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 --2- 172.21.15.190:0/842525308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e2c0a6e20 0x7f6e2c1d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 -- 172.21.15.190:0/842525308 >> 172.21.15.190:0/842525308 conn(0x7f6e2c0abee0 msgr2=0x7f6e2c0aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 -- 172.21.15.190:0/842525308 shutdown_connections 2024-08-19T10:39:52.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.313+0000 7f6e33570640 1 -- 172.21.15.190:0/842525308 wait complete. 2024-08-19T10:39:52.334 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete fooo fooo --yes-i-really-really-mean-it 2024-08-19T10:39:52.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient: get_monmap_and_config 2024-08-19T10:39:52.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient: build_initial_monmap 2024-08-19T10:39:52.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient: monmap: 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:52.455724+0000 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:52.455724+0000 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:52.455 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:52.456 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:52.456 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:52.456 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:52.456 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:52.456 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:52.456 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 1 Processor -- start 2024-08-19T10:39:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 1 -- start start 2024-08-19T10:39:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient: init 2024-08-19T10:39:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient: _add_conns ranks=[1,6,4,10,7,0,5,9,2,8,3] 2024-08-19T10:39:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f194c09f950 addr v1:172.21.15.190:6789/0 2024-08-19T10:39:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0440 0x7f194c13b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient(hunting): picked mon.noname-d con 0x7f194c0a0440 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f194c0a0820 0x7f194c13dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient(hunting): picked mon.noname-c con 0x7f194c0a0820 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f194c0a1f10 con 0x7f194c09f950 2024-08-19T10:39:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f194c06f990 con 0x7f194c0a0820 2024-08-19T10:39:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f194c0a1670 con 0x7f194c0a0440 2024-08-19T10:39:52.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:52.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1950a04640 10 monclient(hunting): authenticate will time out at 1038.025500s 2024-08-19T10:39:52.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f194a59c640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f194c09f950 legacy=0x7f194c09fd30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:39:52.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0440 0x7f194c13b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0440 0x7f194c13b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:51958/0 (socket says 172.21.15.190:51958) 2024-08-19T10:39:52.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f194ad9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f194c0a0820 0x7f194c13dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 1 -- 172.21.15.190:0/951412435 learned_addr learned my addr 172.21.15.190:0/951412435 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:52.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 10 monclient(hunting): get_auth_request con 0x7f194c0a0440 auth_method 0 2024-08-19T10:39:52.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f194ad9d640 10 monclient(hunting): get_auth_request con 0x7f194c0a0820 auth_method 0 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f194ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f194ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f194ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f194ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f194ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f194ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:52.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 10 monclient(hunting): handle_auth_done global_id 4596 payload 274 2024-08-19T10:39:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 1 -- 172.21.15.190:0/951412435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f194c0a0820 msgr2=0x7f194c13dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 1 --2- 172.21.15.190:0/951412435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f194c0a0820 0x7f194c13dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 1 -- 172.21.15.190:0/951412435 >> v1:172.21.15.190:6789/0 conn(0x7f194c09f950 legacy=0x7f194c09fd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:39:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 1 monclient: found mon.noname-d 2024-08-19T10:39:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f1949d9b640 1 -- 172.21.15.190:0/951412435 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194c06d9e0 con 0x7f194c0a0440 2024-08-19T10:39:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.453+0000 7f194ad9d640 1 --2- 172.21.15.190:0/951412435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f194c0a0820 0x7f194c13dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f1949d9b640 1 -- 172.21.15.190:0/951412435 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0440 msgr2=0x7f194c13b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f1949d9b640 1 --2- 172.21.15.190:0/951412435 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0440 0x7f194c13b7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f19440098d0 tx=0x7f194401c4d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=6d4fcc71eeb12753 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 1 -- 172.21.15.190:0/951412435 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f19440b0070 con 0x7f194c0a0440 2024-08-19T10:39:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 10 monclient: dump: 2024-08-19T10:39:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 10 monclient: _finish_auth 0 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 10 monclient: _check_auth_tickets 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 1 -- 172.21.15.190:0/951412435 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19440aa810 con 0x7f194c0a0440 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:52.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 1 -- 172.21.15.190:0/951412435 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f19440a9300 con 0x7f194c0a0440 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f194959a640 10 monclient: dump: 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.469 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.470 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.470 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.470 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.470 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.470 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f1950a04640 5 monclient: authenticate success, global_id 4596 2024-08-19T10:39:52.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f1950a04640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:52.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.457+0000 7f1950a04640 4 set_mon_vals no callback set 2024-08-19T10:39:52.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient: shutdown 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 -- 172.21.15.190:0/951412435 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0440 msgr2=0x7f194c13b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 --2- 172.21.15.190:0/951412435 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0440 0x7f194c13b7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f19440098d0 tx=0x7f194401c4d0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 -- 172.21.15.190:0/951412435 shutdown_connections 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 --2- 172.21.15.190:0/951412435 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f194c0a0820 0x7f194c13dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 --2- 172.21.15.190:0/951412435 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0440 0x7f194c13b7e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 -- 172.21.15.190:0/951412435 >> 172.21.15.190:0/951412435 conn(0x7f194c0a8450 msgr2=0x7f194c0a8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 -- 172.21.15.190:0/951412435 shutdown_connections 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 -- 172.21.15.190:0/951412435 wait complete. 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient: build_initial_monmap 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient: monmap: 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:52.471 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:52.472 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:52.473 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 Processor -- start 2024-08-19T10:39:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 -- start start 2024-08-19T10:39:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient: init 2024-08-19T10:39:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient: _add_conns ranks=[2,3,6,0,5,8,4,7,1] 2024-08-19T10:39:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f194c09f950 0x7f194c13c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient(hunting): picked mon.noname-b con 0x7f194c09f950 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f194c0a0440 0x7f194c13ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient(hunting): picked mon.noname-g con 0x7f194c0a0440 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0820 0x7f194c092400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient(hunting): picked mon.noname-d con 0x7f194c0a0820 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f194c06ca50 con 0x7f194c09f950 2024-08-19T10:39:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f194c06c8d0 con 0x7f194c0a0440 2024-08-19T10:39:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f194c06cbd0 con 0x7f194c0a0820 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1950a04640 10 monclient(hunting): authenticate will time out at 1038.034284s 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f194c0a0440 0x7f194c13ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f194c0a0440 0x7f194c13ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42302/0 (socket says 172.21.15.190:42302) 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 1 -- 172.21.15.190:0/2050641112 learned_addr learned my addr 172.21.15.190:0/2050641112 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 10 monclient(hunting): get_auth_request con 0x7f194c0a0440 auth_method 0 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f194a59c640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f194c09f950 0x7f194c13c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f194ad9d640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0820 0x7f194c092400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f194a59c640 10 monclient(hunting): get_auth_request con 0x7f194c09f950 auth_method 0 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f194a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f194a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f194a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f194ad9d640 10 monclient(hunting): get_auth_request con 0x7f194c0a0820 auth_method 0 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f194ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f194ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f194ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 10 monclient(hunting): handle_auth_done global_id 4656 payload 824 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 1 -- 172.21.15.190:0/2050641112 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0820 msgr2=0x7f194c092400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0820 0x7f194c092400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 1 -- 172.21.15.190:0/2050641112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f194c09f950 msgr2=0x7f194c13c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f194c09f950 0x7f194c13c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:39:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 1 monclient: found mon.noname-g 2024-08-19T10:39:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.461+0000 7f1949d9b640 1 -- 172.21.15.190:0/2050641112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194c072750 con 0x7f194c0a0440 2024-08-19T10:39:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1949d9b640 1 -- 172.21.15.190:0/2050641112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f194c0a0440 msgr2=0x7f194c13ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1949d9b640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f194c0a0440 0x7f194c13ce00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f19440097c0 tx=0x7f1944019270 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 1 -- 172.21.15.190:0/2050641112 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f19440b9070 con 0x7f194c0a0440 2024-08-19T10:39:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 10 monclient: dump: 2024-08-19T10:39:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 10 monclient: _finish_auth 0 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 10 monclient: _check_auth_tickets 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1950a04640 5 monclient: authenticate success, global_id 4656 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1950a04640 10 monclient: _renew_subs 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1950a04640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1950a04640 1 -- 172.21.15.190:0/2050641112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194c071700 con 0x7f194c0a0440 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1950a04640 10 monclient: _renew_subs 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1950a04640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1950a04640 1 -- 172.21.15.190:0/2050641112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f194c072470 con 0x7f194c0a0440 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 1 -- 172.21.15.190:0/2050641112 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1944004040 con 0x7f194c0a0440 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 1 -- 172.21.15.190:0/2050641112 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f19440b0040 con 0x7f194c0a0440 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193bfff640 4 set_mon_vals no callback set 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 10 monclient: dump: 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1950a04640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1950a04640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1950a04640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f1950a04640 1 -- 172.21.15.190:0/2050641112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1930005090 con 0x7f194c0a0440 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 1 -- 172.21.15.190:0/2050641112 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f19440041f0 con 0x7f194c0a0440 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f192c03be60 0x7f192c03e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f193affd640 1 -- 172.21.15.190:0/2050641112 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 4438+0+0 (secure 0 0 0) 0x7f19440e7980 con 0x7f194c0a0440 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f194a59c640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f192c03be60 0x7f192c03e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.465+0000 7f194a59c640 10 monclient: get_auth_request con 0x7f192c03be60 auth_method 0 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.469+0000 7f194a59c640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f192c03be60 0x7f192c03e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f193c003d10 tx=0x7f193c0094d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.469+0000 7f193affd640 1 -- 172.21.15.190:0/2050641112 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1944016150 con 0x7f194c0a0440 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.469+0000 7f193affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.469+0000 7f193affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:52.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.641+0000 7f1950a04640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.641+0000 7f1950a04640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.641+0000 7f1950a04640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:52.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.641+0000 7f1950a04640 1 -- 172.21.15.190:0/2050641112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1930005380 con 0x7f194c0a0440 2024-08-19T10:39:52.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.645+0000 7f193affd640 1 -- 172.21.15.190:0/2050641112 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v44) ==== 159+0+0 (secure 0 0 0) 0x7f19440bd3f0 con 0x7f194c0a0440 2024-08-19T10:39:52.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.645+0000 7f193affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.645+0000 7f193affd640 10 monclient: _finish_command 2 = system:0 pool 'fooo' does not exist 2024-08-19T10:39:52.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.737+0000 7f1950a04640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.737+0000 7f1950a04640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.737+0000 7f1950a04640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:52.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.737+0000 7f1950a04640 1 -- 172.21.15.190:0/2050641112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1930005c50 con 0x7f194c0a0440 2024-08-19T10:39:52.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f193affd640 1 -- 172.21.15.190:0/2050641112 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}]=0 pool 'fooo' does not exist v44) ==== 159+0+0 (secure 0 0 0) 0x7f19440b3d70 con 0x7f194c0a0440 2024-08-19T10:39:52.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f193affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fooo", "pool2": "fooo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:52.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f193affd640 10 monclient: _finish_command 3 = system:0 pool 'fooo' does not exist 2024-08-19T10:39:52.747 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'fooo' does not exist 2024-08-19T10:39:52.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 -- 172.21.15.190:0/2050641112 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f192c03be60 msgr2=0x7f192c03e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:52.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f192c03be60 0x7f192c03e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f193c003d10 tx=0x7f193c0094d0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 10 monclient: shutdown 2024-08-19T10:39:52.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:52.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 -- 172.21.15.190:0/2050641112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f194c0a0440 msgr2=0x7f194c13ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:52.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f194c0a0440 0x7f194c13ce00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f19440097c0 tx=0x7f1944019270 comp rx=0 tx=0).stop 2024-08-19T10:39:52.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 -- 172.21.15.190:0/2050641112 shutdown_connections 2024-08-19T10:39:52.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f192c03be60 0x7f192c03e320 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f194c0a0820 0x7f194c092400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f194c0a0440 0x7f194c13ce00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 --2- 172.21.15.190:0/2050641112 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f194c09f950 0x7f194c13c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 -- 172.21.15.190:0/2050641112 >> 172.21.15.190:0/2050641112 conn(0x7f194c0a8450 msgr2=0x7f194c0973b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:52.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 -- 172.21.15.190:0/2050641112 shutdown_connections 2024-08-19T10:39:52.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.741+0000 7f1950a04640 1 -- 172.21.15.190:0/2050641112 wait complete. 2024-08-19T10:39:52.760 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool delete fuggg fuggg --yes-i-really-really-mean-it 2024-08-19T10:39:52.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.873+0000 7f85d7500640 10 monclient: get_monmap_and_config 2024-08-19T10:39:52.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.873+0000 7f85d7500640 10 monclient: build_initial_monmap 2024-08-19T10:39:52.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.873+0000 7f85d7500640 10 monclient: monmap: 2024-08-19T10:39:52.878 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:52.878 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.878 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:52.879475+0000 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:52.879475+0000 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:52.879 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:52.880 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:52.880 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 1 Processor -- start 2024-08-19T10:39:52.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 1 -- start start 2024-08-19T10:39:52.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 10 monclient: init 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 10 monclient: _add_conns ranks=[3,7,9,8,5,10,4,1,2,0,6] 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d00a3d80 0x7f85d00a4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 10 monclient(hunting): picked mon.noname-g con 0x7f85d00a3d80 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d009f0c0 0x7f85d00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 10 monclient(hunting): picked mon.noname-i con 0x7f85d009f0c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f85d009b250 0x7f85d009b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 10 monclient(hunting): picked mon.noname-a con 0x7f85d009b250 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f85d0069750 con 0x7f85d009b250 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f85d00a4be0 con 0x7f85d00a3d80 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f85d0069950 con 0x7f85d009f0c0 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d7500640 10 monclient(hunting): authenticate will time out at 1038.449417s 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d4a9b640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d009f0c0 msgr2=0x7f85d00a46a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d4a9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d009f0c0 0x7f85d00a46a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d00a3d80 0x7f85d00a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d00a3d80 0x7f85d00a4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42306/0 (socket says 172.21.15.190:42306) 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 1 -- 172.21.15.190:0/812781192 learned_addr learned my addr 172.21.15.190:0/812781192 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 10 monclient(hunting): get_auth_request con 0x7f85d00a3d80 auth_method 0 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:52.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 10 monclient(hunting): handle_auth_done global_id 4665 payload 274 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 1 -- 172.21.15.190:0/812781192 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d009f0c0 msgr2=0x7f85d00a46a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 1 --2- 172.21.15.190:0/812781192 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d009f0c0 0x7f85d00a46a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 1 -- 172.21.15.190:0/812781192 >> v2:172.21.15.89:3300/0 conn(0x7f85d009b250 msgr2=0x7f85d009b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 1 --2- 172.21.15.190:0/812781192 >> v2:172.21.15.89:3300/0 conn(0x7f85d009b250 0x7f85d009b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 10 monclient: _finish_hunting 0 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 1 monclient: found mon.noname-g 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:52.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:52.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.877+0000 7f85d529c640 1 -- 172.21.15.190:0/812781192 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d009e390 con 0x7f85d00a3d80 2024-08-19T10:39:52.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d529c640 1 -- 172.21.15.190:0/812781192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d00a3d80 msgr2=0x7f85d00a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:52.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d529c640 1 --2- 172.21.15.190:0/812781192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d00a3d80 0x7f85d00a4160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009870 tx=0x7f85c801c470 comp rx=0 tx=0).ready entity=mon.3 client_cookie=f17919c8c53718ee server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:52.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 1 -- 172.21.15.190:0/812781192 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f85c80b0070 con 0x7f85d00a3d80 2024-08-19T10:39:52.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:52.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 10 monclient: dump: 2024-08-19T10:39:52.888 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.888 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.888 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 10 monclient: _finish_auth 0 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 5 monclient: authenticate success, global_id 4665 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 1 -- 172.21.15.190:0/812781192 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c80aa810 con 0x7f85d00a3d80 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 4 set_mon_vals no callback set 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 1 -- 172.21.15.190:0/812781192 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f85c80aae30 con 0x7f85d00a3d80 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85c7fff640 10 monclient: dump: 2024-08-19T10:39:52.890 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.891 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 10 monclient: shutdown 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 1 -- 172.21.15.190:0/812781192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d00a3d80 msgr2=0x7f85d00a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 1 --2- 172.21.15.190:0/812781192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d00a3d80 0x7f85d00a4160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009870 tx=0x7f85c801c470 comp rx=0 tx=0).stop 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 1 -- 172.21.15.190:0/812781192 shutdown_connections 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 1 --2- 172.21.15.190:0/812781192 >> v2:172.21.15.89:3300/0 conn(0x7f85d009b250 0x7f85d009b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 1 --2- 172.21.15.190:0/812781192 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d009f0c0 0x7f85d00a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 1 --2- 172.21.15.190:0/812781192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d00a3d80 0x7f85d00a4160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 1 -- 172.21.15.190:0/812781192 >> 172.21.15.190:0/812781192 conn(0x7f85d00a4e70 msgr2=0x7f85d00a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 1 -- 172.21.15.190:0/812781192 shutdown_connections 2024-08-19T10:39:52.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 1 -- 172.21.15.190:0/812781192 wait complete. 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 10 monclient: build_initial_monmap 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.881+0000 7f85d7500640 10 monclient: monmap: 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:52.893 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 1 Processor -- start 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 1 -- start start 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient: init 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient: _add_conns ranks=[1,3,7,4,0,2,6,8,5] 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d009b250 0x7f85d013d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient(hunting): picked mon.noname-f con 0x7f85d009b250 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:52.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d009f0c0 0x7f85d013ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient(hunting): picked mon.noname-g con 0x7f85d009f0c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a3d80 0x7f85d013e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient(hunting): picked mon.noname-i con 0x7f85d00a3d80 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85d009da30 con 0x7f85d009b250 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f85d00a5cb0 con 0x7f85d009f0c0 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f85d009cbf0 con 0x7f85d00a3d80 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient(hunting): authenticate will time out at 1038.456734s 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d529c640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d009b250 msgr2=0x7f85d013d990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d5a9d640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a3d80 msgr2=0x7f85d013e410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:52.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d5a9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a3d80 0x7f85d013e410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d529c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d009b250 0x7f85d013d990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d009f0c0 0x7f85d013ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d009f0c0 0x7f85d013ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42316/0 (socket says 172.21.15.190:42316) 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 1 -- 172.21.15.190:0/465736427 learned_addr learned my addr 172.21.15.190:0/465736427 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 10 monclient(hunting): get_auth_request con 0x7f85d009f0c0 auth_method 0 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 10 monclient(hunting): handle_auth_done global_id 4674 payload 824 2024-08-19T10:39:52.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 1 -- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a3d80 msgr2=0x7f85d013e410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:52.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a3d80 0x7f85d013e410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 1 -- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d009b250 msgr2=0x7f85d013d990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:52.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d009b250 0x7f85d013d990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:52.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 10 monclient: _finish_hunting 0 2024-08-19T10:39:52.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 1 monclient: found mon.noname-g 2024-08-19T10:39:52.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:52.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:52.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 1 -- 172.21.15.190:0/465736427 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d01371d0 con 0x7f85d009f0c0 2024-08-19T10:39:52.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 1 -- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d009f0c0 msgr2=0x7f85d013ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:52.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d4a9b640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d009f0c0 0x7f85d013ded0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f85c000b450 tx=0x7f85c000b910 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:52.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 1 -- 172.21.15.190:0/465736427 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f85c0003db0 con 0x7f85d009f0c0 2024-08-19T10:39:52.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 10 monclient: dump: 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.900 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 5 monclient: authenticate success, global_id 4674 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 1 -- 172.21.15.190:0/465736427 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c0015070 con 0x7f85d009f0c0 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 1 -- 172.21.15.190:0/465736427 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f85c0010400 con 0x7f85d009f0c0 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient: _renew_subs 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:52.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 1 -- 172.21.15.190:0/465736427 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d009b6d0 con 0x7f85d009f0c0 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c5ffb640 10 monclient: dump: 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:52.902 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85c77fe640 4 set_mon_vals no callback set 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient: _renew_subs 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.885+0000 7f85d7500640 1 -- 172.21.15.190:0/465736427 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d013bc70 con 0x7f85d009f0c0 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.889+0000 7f85d7500640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:52.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.889+0000 7f85d7500640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:52.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.889+0000 7f85d7500640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:52.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.889+0000 7f85d7500640 1 -- 172.21.15.190:0/465736427 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d009cfd0 con 0x7f85d009f0c0 2024-08-19T10:39:52.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.889+0000 7f85d4a9b640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d009f0c0 0x7f85d013ded0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f85c000b450 tx=0x7f85c000b910 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f85c000f070 mgrmap(e 5) 2024-08-19T10:39:52.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.889+0000 7f85c5ffb640 1 -- 172.21.15.190:0/465736427 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f85c000f070 con 0x7f85d009f0c0 2024-08-19T10:39:52.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.889+0000 7f85c5ffb640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85b403be00 0x7f85b403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:52.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.889+0000 7f85d529c640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85b403be00 0x7f85b403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:52.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.889+0000 7f85d529c640 10 monclient: get_auth_request con 0x7f85b403be00 auth_method 0 2024-08-19T10:39:52.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.889+0000 7f85d529c640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85b403be00 0x7f85b403e2c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f85c80ab000 tx=0x7f85c80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:52.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:52.893+0000 7f85c5ffb640 1 -- 172.21.15.190:0/465736427 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 4438+0+0 (secure 0 0 0) 0x7f85c004e180 con 0x7f85d009f0c0 2024-08-19T10:39:53.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.513+0000 7f85c5ffb640 1 -- 172.21.15.190:0/465736427 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f85c000d040 con 0x7f85d009f0c0 2024-08-19T10:39:53.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.513+0000 7f85c5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:53.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.513+0000 7f85c5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:53.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.693+0000 7f85d7500640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:53.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.693+0000 7f85d7500640 10 monclient: _send_command 2 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:53.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.693+0000 7f85d7500640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:53.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.693+0000 7f85d7500640 1 -- 172.21.15.190:0/465736427 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true} v 0) -- 0x7f85d013a750 con 0x7f85d009f0c0 2024-08-19T10:39:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.693+0000 7f85c5ffb640 1 -- 172.21.15.190:0/465736427 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}]=0 pool 'fuggg' does not exist v44) ==== 162+0+0 (secure 0 0 0) 0x7f85c000f360 con 0x7f85d009f0c0 2024-08-19T10:39:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.693+0000 7f85c5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.693+0000 7f85c5ffb640 10 monclient: _finish_command 2 = system:0 pool 'fuggg' does not exist 2024-08-19T10:39:53.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.797+0000 7f85d7500640 10 start_mon_command cmd=[{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:53.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.797+0000 7f85d7500640 10 monclient: _send_command 3 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:53.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.797+0000 7f85d7500640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:53.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.797+0000 7f85d7500640 1 -- 172.21.15.190:0/465736427 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true} v 0) -- 0x7f85d00008e0 con 0x7f85d009f0c0 2024-08-19T10:39:53.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.797+0000 7f85c5ffb640 1 -- 172.21.15.190:0/465736427 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}]=0 pool 'fuggg' does not exist v44) ==== 162+0+0 (secure 0 0 0) 0x7f85c0018c60 con 0x7f85d009f0c0 2024-08-19T10:39:53.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.797+0000 7f85c5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool delete", "pool": "fuggg", "pool2": "fuggg", "yes_i_really_really_mean_it": true}] 2024-08-19T10:39:53.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.797+0000 7f85c5ffb640 10 monclient: _finish_command 3 = system:0 pool 'fuggg' does not exist 2024-08-19T10:39:53.801 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'fuggg' does not exist 2024-08-19T10:39:53.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 -- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85b403be00 msgr2=0x7f85b403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:53.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85b403be00 0x7f85b403e2c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f85c80ab000 tx=0x7f85c80a6000 comp rx=0 tx=0).stop 2024-08-19T10:39:53.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 10 monclient: shutdown 2024-08-19T10:39:53.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:53.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 -- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d009f0c0 msgr2=0x7f85d013ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:53.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d009f0c0 0x7f85d013ded0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f85c000b450 tx=0x7f85c000b910 comp rx=0 tx=0).stop 2024-08-19T10:39:53.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 -- 172.21.15.190:0/465736427 shutdown_connections 2024-08-19T10:39:53.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85b403be00 0x7f85b403e2c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:53.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a3d80 0x7f85d013e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:53.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85d009f0c0 0x7f85d013ded0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:53.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 --2- 172.21.15.190:0/465736427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d009b250 0x7f85d013d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:53.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 -- 172.21.15.190:0/465736427 >> 172.21.15.190:0/465736427 conn(0x7f85d00a4e70 msgr2=0x7f85d009fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:53.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 -- 172.21.15.190:0/465736427 shutdown_connections 2024-08-19T10:39:53.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:53.801+0000 7f85d7500640 1 -- 172.21.15.190:0/465736427 wait complete. 2024-08-19T10:39:53.818 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo OK 2024-08-19T10:39:53.819 INFO:tasks.workunit.client.0.smithi190.stdout:OK 2024-08-19T10:39:53.819 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-19T10:39:53.820 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-19T10:39:53.875 INFO:tasks.workunit:Running workunits matching mon/crush_ops.sh on client.0... 2024-08-19T10:39:53.877 INFO:tasks.workunit:Running workunit mon/crush_ops.sh... 2024-08-19T10:39:53.877 DEBUG:teuthology.orchestra.run.smithi190:workunit test mon/crush_ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=964adda23da9f7b057cb34fdc0b82aeb7ed65324 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/crush_ops.sh 2024-08-19T10:39:53.927 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush dump 2024-08-19T10:39:54.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient: get_monmap_and_config 2024-08-19T10:39:54.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient: build_initial_monmap 2024-08-19T10:39:54.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient: monmap: 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:54.047711+0000 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:54.047711+0000 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:54.047 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:54.048 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:54.048 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:54.048 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:54.048 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 1 Processor -- start 2024-08-19T10:39:54.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 1 -- start start 2024-08-19T10:39:54.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient: init 2024-08-19T10:39:54.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient: _add_conns ranks=[5,4,7,3,2,9,10,6,0,1,8] 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1b200a6740 0x7f1b200a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient(hunting): picked mon.noname-h con 0x7f1b200a6740 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b200a5bc0 0x7f1b200a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient(hunting): picked mon.noname-c con 0x7f1b200a5bc0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1b200a7060 0x7f1b20139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient(hunting): picked mon.noname-i con 0x7f1b200a7060 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f1b200a4c20 con 0x7f1b200a5bc0 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1b200a3530 con 0x7f1b200a6740 2024-08-19T10:39:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f1b200a3740 con 0x7f1b200a7060 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b252c8640 10 monclient(hunting): authenticate will time out at 1039.617571s 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1f7fe640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1b200a7060 msgr2=0x7f1b20139590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1b200a6740 0x7f1b200a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1b200a6740 0x7f1b200a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36870/0 (socket says 172.21.15.190:36870) 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 1 -- 172.21.15.190:0/643386041 learned_addr learned my addr 172.21.15.190:0/643386041 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 10 monclient(hunting): get_auth_request con 0x7f1b200a6740 auth_method 0 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1f7fe640 1 --2- 172.21.15.190:0/643386041 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1b200a7060 0x7f1b20139590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1e7fc640 1 --2- 172.21.15.190:0/643386041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b200a5bc0 0x7f1b200a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1e7fc640 10 monclient(hunting): get_auth_request con 0x7f1b200a5bc0 auth_method 0 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 10 monclient(hunting): handle_auth_done global_id 4685 payload 274 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 1 -- 172.21.15.190:0/643386041 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1b200a7060 msgr2=0x7f1b20139590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 1 --2- 172.21.15.190:0/643386041 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1b200a7060 0x7f1b20139590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 1 -- 172.21.15.190:0/643386041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b200a5bc0 msgr2=0x7f1b200a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 1 --2- 172.21.15.190:0/643386041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b200a5bc0 0x7f1b200a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 10 monclient: _finish_hunting 0 2024-08-19T10:39:54.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 1 monclient: found mon.noname-h 2024-08-19T10:39:54.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:54.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:54.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.045+0000 7f1b1effd640 1 -- 172.21.15.190:0/643386041 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b200a3240 con 0x7f1b200a6740 2024-08-19T10:39:54.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1effd640 1 -- 172.21.15.190:0/643386041 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1b200a6740 msgr2=0x7f1b200a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:54.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1effd640 1 --2- 172.21.15.190:0/643386041 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1b200a6740 0x7f1b200a5680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1b180098a0 tx=0x7f1b1801c460 comp rx=0 tx=0).ready entity=mon.5 client_cookie=e16a5ded96d36882 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:54.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 1 -- 172.21.15.190:0/643386041 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1b180b0070 con 0x7f1b200a6740 2024-08-19T10:39:54.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:54.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 10 monclient: dump: 2024-08-19T10:39:54.059 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.059 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.060 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 10 monclient: _finish_auth 0 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 1 -- 172.21.15.190:0/643386041 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b180aa810 con 0x7f1b200a6740 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 1 -- 172.21.15.190:0/643386041 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1b180aae30 con 0x7f1b200a6740 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 5 monclient: authenticate success, global_id 4685 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 4 set_mon_vals no callback set 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:54.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b1dffb640 10 monclient: dump: 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.062 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 10 monclient: shutdown 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 1 -- 172.21.15.190:0/643386041 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1b200a6740 msgr2=0x7f1b200a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 1 --2- 172.21.15.190:0/643386041 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1b200a6740 0x7f1b200a5680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1b180098a0 tx=0x7f1b1801c460 comp rx=0 tx=0).stop 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 1 -- 172.21.15.190:0/643386041 shutdown_connections 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 1 --2- 172.21.15.190:0/643386041 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1b200a7060 0x7f1b20139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 1 --2- 172.21.15.190:0/643386041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1b200a5bc0 0x7f1b200a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 1 --2- 172.21.15.190:0/643386041 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1b200a6740 0x7f1b200a5680 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 1 -- 172.21.15.190:0/643386041 >> 172.21.15.190:0/643386041 conn(0x7f1b200abd40 msgr2=0x7f1b200ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:54.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 1 -- 172.21.15.190:0/643386041 shutdown_connections 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 1 -- 172.21.15.190:0/643386041 wait complete. 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 10 monclient: build_initial_monmap 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.049+0000 7f1b252c8640 10 monclient: monmap: 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:54.064 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 1 Processor -- start 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 1 -- start start 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 10 monclient: init 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 10 monclient: _add_conns ranks=[6,2,1,0,7,4,8,5,3] 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1b200a6740 0x7f1b201358a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 10 monclient(hunting): picked mon.noname-d con 0x7f1b200a6740 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:54.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b200a7060 0x7f1b20135de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 10 monclient(hunting): picked mon.noname-b con 0x7f1b200a7060 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b201d1660 0x7f1b201d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 10 monclient(hunting): picked mon.noname-f con 0x7f1b201d1660 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20143450 con 0x7f1b201d1660 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1b201432d0 con 0x7f1b200a7060 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1b201435d0 con 0x7f1b200a6740 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b252c8640 10 monclient(hunting): authenticate will time out at 1039.624752s 2024-08-19T10:39:54.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1f7fe640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b201d1660 msgr2=0x7f1b201d1a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b201d1660 0x7f1b201d1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1b200a6740 0x7f1b201358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1e7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b200a7060 0x7f1b20135de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1b200a6740 0x7f1b201358a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:51984/0 (socket says 172.21.15.190:51984) 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 1 -- 172.21.15.190:0/4257864512 learned_addr learned my addr 172.21.15.190:0/4257864512 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 10 monclient(hunting): get_auth_request con 0x7f1b200a6740 auth_method 0 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1e7fc640 10 monclient(hunting): get_auth_request con 0x7f1b200a7060 auth_method 0 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 10 monclient(hunting): handle_auth_done global_id 4614 payload 824 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 1 -- 172.21.15.190:0/4257864512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b200a7060 msgr2=0x7f1b20135de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b200a7060 0x7f1b20135de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 1 -- 172.21.15.190:0/4257864512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b201d1660 msgr2=0x7f1b201d1a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b201d1660 0x7f1b201d1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 10 monclient: _finish_hunting 0 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 1 monclient: found mon.noname-d 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1effd640 1 -- 172.21.15.190:0/4257864512 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b200a8f90 con 0x7f1b200a6740 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.053+0000 7f1b1e7fc640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b200a7060 0x7f1b20135de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b1effd640 1 -- 172.21.15.190:0/4257864512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1b200a6740 msgr2=0x7f1b201358a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b1effd640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1b200a6740 0x7f1b201358a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1b180097c0 tx=0x7f1b18019200 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 1 -- 172.21.15.190:0/4257864512 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1b180b9070 con 0x7f1b200a6740 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 10 monclient: dump: 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.069 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.070 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 1 -- 172.21.15.190:0/4257864512 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b18004250 con 0x7f1b200a6740 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b252c8640 5 monclient: authenticate success, global_id 4614 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b252c8640 10 monclient: _renew_subs 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b252c8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b1d7fa640 4 set_mon_vals no callback set 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b252c8640 1 -- 172.21.15.190:0/4257864512 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2009f7d0 con 0x7f1b200a6740 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 1 -- 172.21.15.190:0/4257864512 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1b180b0040 con 0x7f1b200a6740 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 10 monclient: dump: 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.072 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.073 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.073 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.073 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.073 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.073 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b252c8640 10 monclient: _renew_subs 2024-08-19T10:39:54.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b252c8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:54.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b252c8640 1 -- 172.21.15.190:0/4257864512 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b200a1290 con 0x7f1b200a6740 2024-08-19T10:39:54.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b252c8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:54.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b252c8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:54.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b252c8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:54.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b252c8640 1 -- 172.21.15.190:0/4257864512 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b200a5680 con 0x7f1b200a6740 2024-08-19T10:39:54.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 1 -- 172.21.15.190:0/4257864512 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1b18003620 con 0x7f1b200a6740 2024-08-19T10:39:54.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1b0003be00 0x7f1b0003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b177fe640 1 -- 172.21.15.190:0/4257864512 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(44..44 src has 1..44) ==== 4438+0+0 (secure 0 0 0) 0x7f1b180e8420 con 0x7f1b200a6740 2024-08-19T10:39:54.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b1e7fc640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1b0003be00 0x7f1b0003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.057+0000 7f1b1e7fc640 10 monclient: get_auth_request con 0x7f1b0003be00 auth_method 0 2024-08-19T10:39:54.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.061+0000 7f1b1e7fc640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1b0003be00 0x7f1b0003e2c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b200a2e30 tx=0x7f1b08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:54.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.069+0000 7f1b177fe640 1 -- 172.21.15.190:0/4257864512 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1b18016170 con 0x7f1b200a6740 2024-08-19T10:39:54.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.069+0000 7f1b177fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:54.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.069+0000 7f1b177fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:54.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.241+0000 7f1b252c8640 10 start_mon_command cmd=[{"prefix": "osd crush dump"}] 2024-08-19T10:39:54.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.241+0000 7f1b252c8640 10 monclient: _send_command 2 [{"prefix": "osd crush dump"}] 2024-08-19T10:39:54.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.241+0000 7f1b252c8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:54.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.241+0000 7f1b252c8640 1 -- 172.21.15.190:0/4257864512 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush dump"} v 0) -- 0x7f1b200a1790 con 0x7f1b200a6740 2024-08-19T10:39:54.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.241+0000 7f1b177fe640 1 -- 172.21.15.190:0/4257864512 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush dump"}]=0 v44) ==== 62+0+7331 (secure 0 0 0) 0x7f1b180aa7e0 con 0x7f1b200a6740 2024-08-19T10:39:54.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.241+0000 7f1b177fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush dump"}] 2024-08-19T10:39:54.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.241+0000 7f1b177fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:39:54.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.345+0000 7f1b252c8640 10 start_mon_command cmd=[{"prefix": "osd crush dump"}] 2024-08-19T10:39:54.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.345+0000 7f1b252c8640 10 monclient: _send_command 3 [{"prefix": "osd crush dump"}] 2024-08-19T10:39:54.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.345+0000 7f1b252c8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:39:54.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.345+0000 7f1b252c8640 1 -- 172.21.15.190:0/4257864512 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush dump"} v 0) -- 0x7f1b200a0330 con 0x7f1b200a6740 2024-08-19T10:39:54.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.345+0000 7f1b177fe640 1 -- 172.21.15.190:0/4257864512 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush dump"}]=0 v44) ==== 62+0+7331 (secure 0 0 0) 0x7f1b180e81a0 con 0x7f1b200a6740 2024-08-19T10:39:54.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.345+0000 7f1b177fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush dump"}] 2024-08-19T10:39:54.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.345+0000 7f1b177fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:39:54.349 INFO:tasks.workunit.client.0.smithi190.stdout:{ 2024-08-19T10:39:54.349 INFO:tasks.workunit.client.0.smithi190.stdout: "devices": [ 2024-08-19T10:39:54.349 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.349 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 0, 2024-08-19T10:39:54.349 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "osd.0", 2024-08-19T10:39:54.349 INFO:tasks.workunit.client.0.smithi190.stdout: "class": "ssd" 2024-08-19T10:39:54.349 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.349 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 1, 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "osd.1", 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: "class": "ssd" 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 2, 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "osd.2", 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: "class": "ssd" 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 3, 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "osd.3", 2024-08-19T10:39:54.350 INFO:tasks.workunit.client.0.smithi190.stdout: "class": "ssd" 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 4, 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "osd.4", 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: "class": "ssd" 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 5, 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "osd.5", 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: "class": "ssd" 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: "types": [ 2024-08-19T10:39:54.351 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 0, 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "osd" 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 1, 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "host" 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 2, 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "chassis" 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.352 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 3, 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "rack" 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 4, 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "row" 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 5, 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "pdu" 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 6, 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "pod" 2024-08-19T10:39:54.353 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 7, 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "room" 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 8, 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "datacenter" 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 9, 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "zone" 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.354 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 10, 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "region" 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 11, 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "root" 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: "buckets": [ 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: "id": -1, 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "default", 2024-08-19T10:39:54.355 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 11, 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: "type_name": "root", 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 34560, 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: "alg": "straw2", 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: "hash": "rjenkins1", 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: "items": [ 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: "id": -3, 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 17280, 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 0 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: "id": -5, 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 17280, 2024-08-19T10:39:54.356 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 1 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: "id": -2, 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "default~ssd", 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 11, 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: "type_name": "root", 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 34560, 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: "alg": "straw2", 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: "hash": "rjenkins1", 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: "items": [ 2024-08-19T10:39:54.357 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: "id": -4, 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 17280, 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 0 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: "id": -6, 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 17280, 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 1 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.358 INFO:tasks.workunit.client.0.smithi190.stdout: "id": -3, 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "smithi190", 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 1, 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: "type_name": "host", 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 17280, 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: "alg": "straw2", 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: "hash": "rjenkins1", 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: "items": [ 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 5, 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 0 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.359 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 3, 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 1 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 4, 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 2 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: "id": -4, 2024-08-19T10:39:54.360 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "smithi190~ssd", 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 1, 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: "type_name": "host", 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 17280, 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: "alg": "straw2", 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: "hash": "rjenkins1", 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: "items": [ 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 5, 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 0 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.361 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 3, 2024-08-19T10:39:54.362 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.362 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 1 2024-08-19T10:39:54.362 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.362 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.362 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 4, 2024-08-19T10:39:54.362 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.362 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 2 2024-08-19T10:39:54.362 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.362 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:39:54.362 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.362 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: "id": -5, 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "smithi089", 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 1, 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: "type_name": "host", 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 17280, 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: "alg": "straw2", 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: "hash": "rjenkins1", 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: "items": [ 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 2, 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.363 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 0 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 1, 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 1 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 0, 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 2 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.364 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: "id": -6, 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: "name": "smithi089~ssd", 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: "type_id": 1, 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: "type_name": "host", 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 17280, 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: "alg": "straw2", 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: "hash": "rjenkins1", 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: "items": [ 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 2, 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 0 2024-08-19T10:39:54.365 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 1, 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 1 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: "id": 0, 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: "weight": 5760, 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: "pos": 2 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.366 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: "rules": [ 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: "rule_id": 0, 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: "rule_name": "replicated_rule", 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: "type": 1, 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: "steps": [ 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "take", 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: "item": -1, 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: "item_name": "default" 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "choose_firstn", 2024-08-19T10:39:54.367 INFO:tasks.workunit.client.0.smithi190.stdout: "num": 0, 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: "type": "osd" 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "emit" 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: "rule_id": 1, 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: "rule_name": "erasure-code", 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: "type": 3, 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: "steps": [ 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.368 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "set_chooseleaf_tries", 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: "num": 5 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "set_choose_tries", 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: "num": 100 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "take", 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: "item": -1, 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: "item_name": "default" 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.369 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "choose_indep", 2024-08-19T10:39:54.370 INFO:tasks.workunit.client.0.smithi190.stdout: "num": 0, 2024-08-19T10:39:54.370 INFO:tasks.workunit.client.0.smithi190.stdout: "type": "osd" 2024-08-19T10:39:54.370 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.370 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.370 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "emit" 2024-08-19T10:39:54.370 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.370 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:39:54.370 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.370 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:39:54.370 INFO:tasks.workunit.client.0.smithi190.stdout: "tunables": { 2024-08-19T10:39:54.370 INFO:tasks.workunit.client.0.smithi190.stdout: "choose_local_tries": 0, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "choose_local_fallback_tries": 0, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "choose_total_tries": 50, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "chooseleaf_descend_once": 1, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "chooseleaf_vary_r": 1, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "chooseleaf_stable": 1, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "msr_descents": 100, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "msr_collision_tries": 100, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "straw_calc_version": 1, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "allowed_bucket_algs": 54, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "profile": "jewel", 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "optimal_tunables": 1, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "legacy_tunables": 0, 2024-08-19T10:39:54.371 INFO:tasks.workunit.client.0.smithi190.stdout: "minimum_required_version": "jewel", 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: "require_feature_tunables": 1, 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: "require_feature_tunables2": 1, 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: "has_v2_rules": 1, 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: "require_feature_tunables3": 1, 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: "has_v3_rules": 0, 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: "has_v4_buckets": 1, 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: "require_feature_tunables5": 1, 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: "has_v5_rules": 0, 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: "has_msr_rules": 0 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: "choose_args": {} 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout:} 2024-08-19T10:39:54.372 INFO:tasks.workunit.client.0.smithi190.stdout: 2024-08-19T10:39:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 -- 172.21.15.190:0/4257864512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1b0003be00 msgr2=0x7f1b0003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1b0003be00 0x7f1b0003e2c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b200a2e30 tx=0x7f1b08008040 comp rx=0 tx=0).stop 2024-08-19T10:39:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 10 monclient: shutdown 2024-08-19T10:39:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 -- 172.21.15.190:0/4257864512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1b200a6740 msgr2=0x7f1b201358a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1b200a6740 0x7f1b201358a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1b180097c0 tx=0x7f1b18019200 comp rx=0 tx=0).stop 2024-08-19T10:39:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 -- 172.21.15.190:0/4257864512 shutdown_connections 2024-08-19T10:39:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1b0003be00 0x7f1b0003e2c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b201d1660 0x7f1b201d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1b200a7060 0x7f1b20135de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 --2- 172.21.15.190:0/4257864512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1b200a6740 0x7f1b201358a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 -- 172.21.15.190:0/4257864512 >> 172.21.15.190:0/4257864512 conn(0x7f1b200abd40 msgr2=0x7f1b2013a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 -- 172.21.15.190:0/4257864512 shutdown_connections 2024-08-19T10:39:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.349+0000 7f1b252c8640 1 -- 172.21.15.190:0/4257864512 wait complete. 2024-08-19T10:39:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule dump 2024-08-19T10:39:54.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.481+0000 7fc630593640 10 monclient: get_monmap_and_config 2024-08-19T10:39:54.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.481+0000 7fc630593640 10 monclient: build_initial_monmap 2024-08-19T10:39:54.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.481+0000 7fc630593640 10 monclient: monmap: 2024-08-19T10:39:54.485 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:54.485 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.485 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:54.486046+0000 2024-08-19T10:39:54.485 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:54.486046+0000 2024-08-19T10:39:54.485 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:54.485 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:54.486 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.481+0000 7fc630593640 1 Processor -- start 2024-08-19T10:39:54.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.481+0000 7fc630593640 1 -- start start 2024-08-19T10:39:54.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.481+0000 7fc630593640 10 monclient: init 2024-08-19T10:39:54.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:54.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 10 monclient: _add_conns ranks=[2,10,4,7,8,0,3,5,6,9,1] 2024-08-19T10:39:54.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc628069410 0x7fc628074b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 10 monclient(hunting): picked mon.noname-b con 0x7fc628069410 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:54.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc6280a3c80 0x7fc6280a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 10 monclient(hunting): picked mon.noname-f con 0x7fc6280a3c80 addr v2:172.21.15.190:3300/0 2024-08-19T10:39:54.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6280a4600 0x7fc62813da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 10 monclient(hunting): picked mon.noname-c con 0x7fc6280a4600 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:54.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc6280697f0 con 0x7fc6280a3c80 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc6280a4a80 con 0x7fc628069410 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc6280a20b0 con 0x7fc6280a4600 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc630593640 10 monclient(hunting): authenticate will time out at 1040.056174s 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62db2e640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fc6280a3c80 msgr2=0x7fc6280a40c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62db2e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc6280a3c80 0x7fc6280a40c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62e32f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc628069410 0x7fc628074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6280a4600 0x7fc62813da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62e32f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc628069410 0x7fc628074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:33530/0 (socket says 172.21.15.190:33530) 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62e32f640 1 -- 172.21.15.190:0/3726557650 learned_addr learned my addr 172.21.15.190:0/3726557650 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62e32f640 10 monclient(hunting): get_auth_request con 0x7fc628069410 auth_method 0 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62e32f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62e32f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62e32f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 10 monclient(hunting): get_auth_request con 0x7fc6280a4600 auth_method 0 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62e32f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62e32f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62e32f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:54.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 10 monclient(hunting): handle_auth_done global_id 4756 payload 274 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 1 -- 172.21.15.190:0/3726557650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc628069410 msgr2=0x7fc628074b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 1 --2- 172.21.15.190:0/3726557650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc628069410 0x7fc628074b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 1 -- 172.21.15.190:0/3726557650 >> v2:172.21.15.190:3300/0 conn(0x7fc6280a3c80 msgr2=0x7fc6280a40c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 1 --2- 172.21.15.190:0/3726557650 >> v2:172.21.15.190:3300/0 conn(0x7fc6280a3c80 0x7fc6280a40c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 10 monclient: _finish_hunting 0 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 1 monclient: found mon.noname-c 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62eb30640 1 -- 172.21.15.190:0/3726557650 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62809d500 con 0x7fc6280a4600 2024-08-19T10:39:54.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.485+0000 7fc62e32f640 1 --2- 172.21.15.190:0/3726557650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc628069410 0x7fc628074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:54.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62eb30640 1 -- 172.21.15.190:0/3726557650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6280a4600 msgr2=0x7fc62813da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:54.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62eb30640 1 --2- 172.21.15.190:0/3726557650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6280a4600 0x7fc62813da80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc620009dc0 tx=0x7fc62001c800 comp rx=0 tx=0).ready entity=mon.4 client_cookie=b5ca1124f331ab5b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:54.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 1 -- 172.21.15.190:0/3726557650 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc6200b2070 con 0x7fc6280a4600 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 10 monclient: dump: 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.497 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 10 monclient: _finish_auth 0 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 10 monclient: _check_auth_tickets 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 1 -- 172.21.15.190:0/3726557650 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6200a5ab0 con 0x7fc6280a4600 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 1 -- 172.21.15.190:0/3726557650 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc6200a5ce0 con 0x7fc6280a4600 2024-08-19T10:39:54.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc62d32d640 10 monclient: dump: 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.499 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 5 monclient: authenticate success, global_id 4756 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 4 set_mon_vals no callback set 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 10 monclient: shutdown 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 1 -- 172.21.15.190:0/3726557650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6280a4600 msgr2=0x7fc62813da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 1 --2- 172.21.15.190:0/3726557650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6280a4600 0x7fc62813da80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc620009dc0 tx=0x7fc62001c800 comp rx=0 tx=0).stop 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 1 -- 172.21.15.190:0/3726557650 shutdown_connections 2024-08-19T10:39:54.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 1 --2- 172.21.15.190:0/3726557650 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc6280a4600 0x7fc62813da80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 1 --2- 172.21.15.190:0/3726557650 >> v2:172.21.15.190:3300/0 conn(0x7fc6280a3c80 0x7fc6280a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 1 --2- 172.21.15.190:0/3726557650 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc628069410 0x7fc628074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 1 -- 172.21.15.190:0/3726557650 >> 172.21.15.190:0/3726557650 conn(0x7fc6280a4d70 msgr2=0x7fc6280a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 1 -- 172.21.15.190:0/3726557650 shutdown_connections 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 1 -- 172.21.15.190:0/3726557650 wait complete. 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 10 monclient: build_initial_monmap 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 10 monclient: monmap: 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:54.501 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 1 Processor -- start 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 1 -- start start 2024-08-19T10:39:54.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 10 monclient: init 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.489+0000 7fc630593640 10 monclient: _add_conns ranks=[5,0,2,1,8,3,7,4,6] 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc628069410 0x7fc62809f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient(hunting): picked mon.noname-h con 0x7fc628069410 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6280a3c80 0x7fc62809fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient(hunting): picked mon.noname-a con 0x7fc6280a3c80 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6280a4600 0x7fc6280a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient(hunting): picked mon.noname-b con 0x7fc6280a4600 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc6280a24b0 con 0x7fc6280a3c80 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc6280a2330 con 0x7fc6280a4600 2024-08-19T10:39:54.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc6281407e0 con 0x7fc628069410 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient(hunting): authenticate will time out at 1040.064075s 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc628069410 0x7fc62809f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc628069410 0x7fc62809f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36884/0 (socket says 172.21.15.190:36884) 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 1 -- 172.21.15.190:0/1635895687 learned_addr learned my addr 172.21.15.190:0/1635895687 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 10 monclient(hunting): get_auth_request con 0x7fc628069410 auth_method 0 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:54.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62db2e640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6280a3c80 0x7fc62809fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62eb30640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6280a4600 0x7fc6280a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62db2e640 10 monclient(hunting): get_auth_request con 0x7fc6280a3c80 auth_method 0 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62db2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62db2e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62db2e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 10 monclient(hunting): handle_auth_done global_id 4694 payload 824 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 1 -- 172.21.15.190:0/1635895687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6280a4600 msgr2=0x7fc6280a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6280a4600 0x7fc6280a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 1 -- 172.21.15.190:0/1635895687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6280a3c80 msgr2=0x7fc62809fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6280a3c80 0x7fc62809fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 10 monclient: _finish_hunting 0 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 1 monclient: found mon.noname-h 2024-08-19T10:39:54.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:54.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:54.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 1 -- 172.21.15.190:0/1635895687 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc628124790 con 0x7fc628069410 2024-08-19T10:39:54.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 1 -- 172.21.15.190:0/1635895687 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc628069410 msgr2=0x7fc62809f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:54.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62e32f640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc628069410 0x7fc62809f770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc62400e9d0 tx=0x7fc62400ee90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:54.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 1 -- 172.21.15.190:0/1635895687 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc624017070 con 0x7fc628069410 2024-08-19T10:39:54.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:54.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 10 monclient: dump: 2024-08-19T10:39:54.508 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.508 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.509 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 10 monclient: _finish_auth 0 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 10 monclient: _check_auth_tickets 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 1 -- 172.21.15.190:0/1635895687 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62400cc70 con 0x7fc628069410 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 5 monclient: authenticate success, global_id 4694 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient: _renew_subs 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 1 -- 172.21.15.190:0/1635895687 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc628125b50 con 0x7fc628069410 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 1 -- 172.21.15.190:0/1635895687 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc62400ce20 con 0x7fc628069410 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc62cb2c640 4 set_mon_vals no callback set 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc61effd640 10 monclient: dump: 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.511 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.512 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.512 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.512 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.512 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.512 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient: _renew_subs 2024-08-19T10:39:54.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:54.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 1 -- 172.21.15.190:0/1635895687 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc628125870 con 0x7fc628069410 2024-08-19T10:39:54.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:54.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:54.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:54.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.493+0000 7fc630593640 1 -- 172.21.15.190:0/1635895687 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc628073870 con 0x7fc628069410 2024-08-19T10:39:54.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.497+0000 7fc61effd640 1 -- 172.21.15.190:0/1635895687 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc62401a070 con 0x7fc628069410 2024-08-19T10:39:54.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.497+0000 7fc61effd640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc60c03be00 0x7fc60c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.497+0000 7fc61effd640 1 -- 172.21.15.190:0/1635895687 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(44..44 src has 1..44) ==== 4438+0+0 (secure 0 0 0) 0x7fc62400f040 con 0x7fc628069410 2024-08-19T10:39:54.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.497+0000 7fc62db2e640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc60c03be00 0x7fc60c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.497+0000 7fc62db2e640 10 monclient: get_auth_request con 0x7fc60c03be00 auth_method 0 2024-08-19T10:39:54.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.497+0000 7fc62db2e640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc60c03be00 0x7fc60c03e2c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6280a0ed0 tx=0x7fc618008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:54.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.501+0000 7fc61effd640 1 -- 172.21.15.190:0/1635895687 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc628073870 con 0x7fc628069410 2024-08-19T10:39:54.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.501+0000 7fc61effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:54.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.501+0000 7fc61effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:54.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.673+0000 7fc630593640 10 start_mon_command cmd=[{"prefix": "osd crush rule dump"}] 2024-08-19T10:39:54.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.673+0000 7fc630593640 10 monclient: _send_command 2 [{"prefix": "osd crush rule dump"}] 2024-08-19T10:39:54.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.673+0000 7fc630593640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:54.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.673+0000 7fc630593640 1 -- 172.21.15.190:0/1635895687 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rule dump"} v 0) -- 0x7fc628124f20 con 0x7fc628069410 2024-08-19T10:39:54.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.673+0000 7fc61effd640 1 -- 172.21.15.190:0/1635895687 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule dump"}]=0 v44) ==== 67+0+1058 (secure 0 0 0) 0x7fc624021070 con 0x7fc628069410 2024-08-19T10:39:54.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.673+0000 7fc61effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule dump"}] 2024-08-19T10:39:54.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.673+0000 7fc61effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:39:54.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 10 start_mon_command cmd=[{"prefix": "osd crush rule dump"}] 2024-08-19T10:39:54.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 10 monclient: _send_command 3 [{"prefix": "osd crush rule dump"}] 2024-08-19T10:39:54.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:54.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 1 -- 172.21.15.190:0/1635895687 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rule dump"} v 0) -- 0x7fc6280008e0 con 0x7fc628069410 2024-08-19T10:39:54.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc61effd640 1 -- 172.21.15.190:0/1635895687 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule dump"}]=0 v44) ==== 67+0+1058 (secure 0 0 0) 0x7fc62401bdc0 con 0x7fc628069410 2024-08-19T10:39:54.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc61effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule dump"}] 2024-08-19T10:39:54.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc61effd640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:39:54.776 INFO:tasks.workunit.client.0.smithi190.stdout:[ 2024-08-19T10:39:54.776 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.776 INFO:tasks.workunit.client.0.smithi190.stdout: "rule_id": 0, 2024-08-19T10:39:54.776 INFO:tasks.workunit.client.0.smithi190.stdout: "rule_name": "replicated_rule", 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: "type": 1, 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: "steps": [ 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "take", 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: "item": -1, 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: "item_name": "default" 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "choose_firstn", 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: "num": 0, 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: "type": "osd" 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.777 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "emit" 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: "rule_id": 1, 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: "rule_name": "erasure-code", 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: "type": 3, 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: "steps": [ 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "set_chooseleaf_tries", 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: "num": 5 2024-08-19T10:39:54.778 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "set_choose_tries", 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: "num": 100 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "take", 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: "item": -1, 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: "item_name": "default" 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "choose_indep", 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: "num": 0, 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: "type": "osd" 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:39:54.779 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:39:54.780 INFO:tasks.workunit.client.0.smithi190.stdout: "op": "emit" 2024-08-19T10:39:54.780 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.780 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:39:54.780 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:39:54.780 INFO:tasks.workunit.client.0.smithi190.stdout:] 2024-08-19T10:39:54.780 INFO:tasks.workunit.client.0.smithi190.stdout: 2024-08-19T10:39:54.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 1 -- 172.21.15.190:0/1635895687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc60c03be00 msgr2=0x7fc60c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc60c03be00 0x7fc60c03e2c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6280a0ed0 tx=0x7fc618008040 comp rx=0 tx=0).stop 2024-08-19T10:39:54.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 10 monclient: shutdown 2024-08-19T10:39:54.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:54.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 1 -- 172.21.15.190:0/1635895687 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc628069410 msgr2=0x7fc62809f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc628069410 0x7fc62809f770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc62400e9d0 tx=0x7fc62400ee90 comp rx=0 tx=0).stop 2024-08-19T10:39:54.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 1 -- 172.21.15.190:0/1635895687 shutdown_connections 2024-08-19T10:39:54.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc60c03be00 0x7fc60c03e2c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc6280a4600 0x7fc6280a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc6280a3c80 0x7fc62809fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 1 --2- 172.21.15.190:0/1635895687 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc628069410 0x7fc62809f770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.773+0000 7fc630593640 1 -- 172.21.15.190:0/1635895687 >> 172.21.15.190:0/1635895687 conn(0x7fc6280a4d70 msgr2=0x7fc628070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:54.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.777+0000 7fc630593640 1 -- 172.21.15.190:0/1635895687 shutdown_connections 2024-08-19T10:39:54.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.777+0000 7fc630593640 1 -- 172.21.15.190:0/1635895687 wait complete. 2024-08-19T10:39:54.793 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule ls 2024-08-19T10:39:54.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient: get_monmap_and_config 2024-08-19T10:39:54.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient: build_initial_monmap 2024-08-19T10:39:54.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient: monmap: 2024-08-19T10:39:54.899 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:54.899 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:54.900477+0000 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:54.900477+0000 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:54.900 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:54.901 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:54.901 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:54.901 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:54.901 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 1 Processor -- start 2024-08-19T10:39:54.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 1 -- start start 2024-08-19T10:39:54.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient: init 2024-08-19T10:39:54.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:54.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient: _add_conns ranks=[1,2,8,10,7,6,9,3,5,0,4] 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f76b00a6850 addr v1:172.21.15.190:6789/0 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b00a6c30 0x7f76b00a7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient(hunting): picked mon.noname-b con 0x7f76b00a6c30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f76b013ebb0 0x7f76b0140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient(hunting): picked mon.noname-e con 0x7f76b013ebb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b00a5f00 con 0x7f76b00a6850 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f76b00a3640 con 0x7f76b00a6c30 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f76b00a3850 con 0x7f76b013ebb0 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b6dde640 10 monclient(hunting): authenticate will time out at 1040.470217s 2024-08-19T10:39:54.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b4b7a640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f76b00a6850 legacy=0x7f76b00a57f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b00a6c30 0x7f76b00a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b537b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f76b013ebb0 0x7f76b0140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b00a6c30 0x7f76b00a7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:33550/0 (socket says 172.21.15.190:33550) 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 1 -- 172.21.15.190:0/969581404 learned_addr learned my addr 172.21.15.190:0/969581404 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 10 monclient(hunting): get_auth_request con 0x7f76b00a6c30 auth_method 0 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b537b640 10 monclient(hunting): get_auth_request con 0x7f76b013ebb0 auth_method 0 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b537b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b537b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b537b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:54.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b537b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b537b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b537b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 10 monclient(hunting): handle_auth_done global_id 4682 payload 274 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 1 -- 172.21.15.190:0/969581404 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f76b013ebb0 msgr2=0x7f76b0140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 1 --2- 172.21.15.190:0/969581404 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f76b013ebb0 0x7f76b0140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 1 -- 172.21.15.190:0/969581404 >> v1:172.21.15.190:6789/0 conn(0x7f76b00a6850 legacy=0x7f76b00a57f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 1 monclient: found mon.noname-b 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:54.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:54.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76affff640 1 -- 172.21.15.190:0/969581404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b00a46e0 con 0x7f76b00a6c30 2024-08-19T10:39:54.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.897+0000 7f76b537b640 1 --2- 172.21.15.190:0/969581404 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f76b013ebb0 0x7f76b0140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.901+0000 7f76affff640 1 -- 172.21.15.190:0/969581404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b00a6c30 msgr2=0x7f76b00a7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.901+0000 7f76affff640 1 --2- 172.21.15.190:0/969581404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b00a6c30 0x7f76b00a7010 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f76a40098a0 tx=0x7f76a401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6357bf8c1e63b09d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 1 -- 172.21.15.190:0/969581404 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76a40b0070 con 0x7f76b00a6c30 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 10 monclient: dump: 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.911 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 1 -- 172.21.15.190:0/969581404 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a40aa810 con 0x7f76b00a6c30 2024-08-19T10:39:54.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 5 monclient: authenticate success, global_id 4682 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 1 -- 172.21.15.190:0/969581404 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76a40aae30 con 0x7f76b00a6c30 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76af7fe640 10 monclient: dump: 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.913 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.914 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.914 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.914 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.914 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.914 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.914 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.914 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.914 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 4 set_mon_vals no callback set 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient: shutdown 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 -- 172.21.15.190:0/969581404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b00a6c30 msgr2=0x7f76b00a7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 --2- 172.21.15.190:0/969581404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b00a6c30 0x7f76b00a7010 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f76a40098a0 tx=0x7f76a401c440 comp rx=0 tx=0).stop 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 -- 172.21.15.190:0/969581404 shutdown_connections 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 --2- 172.21.15.190:0/969581404 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f76b013ebb0 0x7f76b0140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 --2- 172.21.15.190:0/969581404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76b00a6c30 0x7f76b00a7010 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 -- 172.21.15.190:0/969581404 >> 172.21.15.190:0/969581404 conn(0x7f76b00abe50 msgr2=0x7f76b00ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 -- 172.21.15.190:0/969581404 shutdown_connections 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 -- 172.21.15.190:0/969581404 wait complete. 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient: build_initial_monmap 2024-08-19T10:39:54.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient: monmap: 2024-08-19T10:39:54.916 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:54.916 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:54.916 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:54.916 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:54.916 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:54.916 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.916 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:54.916 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:54.916 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:54.916 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:54.916 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 Processor -- start 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 -- start start 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient: init 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient: _add_conns ranks=[0,3,8,1,2,6,7,4,5] 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b00a6850 0x7f76b013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient(hunting): picked mon.noname-a con 0x7f76b00a6850 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b00a6c30 0x7f76b0136930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient(hunting): picked mon.noname-g con 0x7f76b00a6c30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:54.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f76b013ebb0 0x7f76b0136e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient(hunting): picked mon.noname-e con 0x7f76b013ebb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:39:54.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f76b0143e80 con 0x7f76b00a6850 2024-08-19T10:39:54.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f76b0143d00 con 0x7f76b00a6c30 2024-08-19T10:39:54.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:54.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f76b0144000 con 0x7f76b013ebb0 2024-08-19T10:39:54.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:54.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b6dde640 10 monclient(hunting): authenticate will time out at 1040.478770s 2024-08-19T10:39:54.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b00a6c30 0x7f76b0136930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b00a6c30 0x7f76b0136930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42332/0 (socket says 172.21.15.190:42332) 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 1 -- 172.21.15.190:0/4073597633 learned_addr learned my addr 172.21.15.190:0/4073597633 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 10 monclient(hunting): get_auth_request con 0x7f76b00a6c30 auth_method 0 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b4b7a640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b00a6850 0x7f76b013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b537b640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f76b013ebb0 0x7f76b0136e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b4b7a640 10 monclient(hunting): get_auth_request con 0x7f76b00a6850 auth_method 0 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b4b7a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b4b7a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b4b7a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b537b640 10 monclient(hunting): get_auth_request con 0x7f76b013ebb0 auth_method 0 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b537b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b537b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76b537b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 10 monclient(hunting): handle_auth_done global_id 4683 payload 824 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 1 -- 172.21.15.190:0/4073597633 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f76b013ebb0 msgr2=0x7f76b0136e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f76b013ebb0 0x7f76b0136e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 1 -- 172.21.15.190:0/4073597633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b00a6850 msgr2=0x7f76b013cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b00a6850 0x7f76b013cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 1 monclient: found mon.noname-g 2024-08-19T10:39:54.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:54.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:54.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.905+0000 7f76affff640 1 -- 172.21.15.190:0/4073597633 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b013c400 con 0x7f76b00a6c30 2024-08-19T10:39:54.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76affff640 1 -- 172.21.15.190:0/4073597633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b00a6c30 msgr2=0x7f76b0136930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:54.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76affff640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b00a6c30 0x7f76b0136930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f76a401cce0 tx=0x7f76a40191e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:54.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 1 -- 172.21.15.190:0/4073597633 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76a40ba070 con 0x7f76b00a6c30 2024-08-19T10:39:54.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 10 monclient: dump: 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.924 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 10 monclient: _finish_auth 0 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 1 -- 172.21.15.190:0/4073597633 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a40b1040 con 0x7f76b00a6c30 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b6dde640 5 monclient: authenticate success, global_id 4683 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b6dde640 10 monclient: _renew_subs 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b6dde640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b6dde640 1 -- 172.21.15.190:0/4073597633 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b013db50 con 0x7f76b00a6c30 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:54.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 1 -- 172.21.15.190:0/4073597633 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76a40b0070 con 0x7f76b00a6c30 2024-08-19T10:39:54.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:54.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:54.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 10 monclient: dump: 2024-08-19T10:39:54.926 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:54.926 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:54.926 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.926 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:54.926 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:54.926 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:54.927 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:54.927 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:54.927 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:54.927 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:54.927 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:54.927 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:54.928 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:54.928 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:54.928 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:54.928 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:54.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b6dde640 10 monclient: _renew_subs 2024-08-19T10:39:54.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b6dde640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:54.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b6dde640 1 -- 172.21.15.190:0/4073597633 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b013c120 con 0x7f76b00a6c30 2024-08-19T10:39:54.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76aeffd640 4 set_mon_vals no callback set 2024-08-19T10:39:54.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 1 -- 172.21.15.190:0/4073597633 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f76a4004250 con 0x7f76b00a6c30 2024-08-19T10:39:54.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b6dde640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:54.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b6dde640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:54.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b6dde640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:54.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f769c03be00 0x7f769c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:54.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b6dde640 1 -- 172.21.15.190:0/4073597633 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b00a7010 con 0x7f76b00a6c30 2024-08-19T10:39:54.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76ad7fa640 1 -- 172.21.15.190:0/4073597633 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 4438+0+0 (secure 0 0 0) 0x7f76a40a4e00 con 0x7f76b00a6c30 2024-08-19T10:39:54.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b4b7a640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f769c03be00 0x7f769c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:54.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.909+0000 7f76b4b7a640 10 monclient: get_auth_request con 0x7f769c03be00 auth_method 0 2024-08-19T10:39:54.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.913+0000 7f76b4b7a640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f769c03be00 0x7f769c03e2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76a0006fd0 tx=0x7f76a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:54.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.913+0000 7f76ad7fa640 1 -- 172.21.15.190:0/4073597633 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f76a4016150 con 0x7f76b00a6c30 2024-08-19T10:39:54.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.913+0000 7f76ad7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:54.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:54.913+0000 7f76ad7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:55.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.073+0000 7f76b6dde640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:39:55.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.073+0000 7f76b6dde640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:39:55.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.073+0000 7f76b6dde640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:55.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.073+0000 7f76b6dde640 1 -- 172.21.15.190:0/4073597633 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f76b00a9030 con 0x7f76b00a6c30 2024-08-19T10:39:55.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.073+0000 7f76ad7fa640 1 -- 172.21.15.190:0/4073597633 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v44) ==== 65+0+29 (secure 0 0 0) 0x7f76a4003620 con 0x7f76b00a6c30 2024-08-19T10:39:55.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.073+0000 7f76ad7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:39:55.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.073+0000 7f76ad7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:39:55.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.165+0000 7f76b6dde640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:39:55.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.165+0000 7f76b6dde640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:39:55.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.165+0000 7f76b6dde640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:55.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.165+0000 7f76b6dde640 1 -- 172.21.15.190:0/4073597633 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f76b013d5e0 con 0x7f76b00a6c30 2024-08-19T10:39:55.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76ad7fa640 1 -- 172.21.15.190:0/4073597633 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v44) ==== 65+0+29 (secure 0 0 0) 0x7f76a40b4360 con 0x7f76b00a6c30 2024-08-19T10:39:55.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76ad7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:39:55.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76ad7fa640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:39:55.172 INFO:tasks.workunit.client.0.smithi190.stdout:replicated_rule 2024-08-19T10:39:55.172 INFO:tasks.workunit.client.0.smithi190.stdout:erasure-code 2024-08-19T10:39:55.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 -- 172.21.15.190:0/4073597633 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f769c03be00 msgr2=0x7f769c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:55.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f769c03be00 0x7f769c03e2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76a0006fd0 tx=0x7f76a0008040 comp rx=0 tx=0).stop 2024-08-19T10:39:55.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 10 monclient: shutdown 2024-08-19T10:39:55.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:55.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 -- 172.21.15.190:0/4073597633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b00a6c30 msgr2=0x7f76b0136930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:55.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b00a6c30 0x7f76b0136930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f76a401cce0 tx=0x7f76a40191e0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 -- 172.21.15.190:0/4073597633 shutdown_connections 2024-08-19T10:39:55.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f769c03be00 0x7f769c03e2c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f76b013ebb0 0x7f76b0136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b00a6c30 0x7f76b0136930 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 --2- 172.21.15.190:0/4073597633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b00a6850 0x7f76b013cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 -- 172.21.15.190:0/4073597633 >> 172.21.15.190:0/4073597633 conn(0x7f76b00abe50 msgr2=0x7f76b013fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:55.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 -- 172.21.15.190:0/4073597633 shutdown_connections 2024-08-19T10:39:55.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.169+0000 7f76b6dde640 1 -- 172.21.15.190:0/4073597633 wait complete. 2024-08-19T10:39:55.186 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule list 2024-08-19T10:39:55.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.301+0000 7f4f92303640 10 monclient: get_monmap_and_config 2024-08-19T10:39:55.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.301+0000 7f4f92303640 10 monclient: build_initial_monmap 2024-08-19T10:39:55.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.301+0000 7f4f92303640 10 monclient: monmap: 2024-08-19T10:39:55.305 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:55.305 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:55.305 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:55.305951+0000 2024-08-19T10:39:55.305 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:55.305951+0000 2024-08-19T10:39:55.305 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:55.305 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.305 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:55.305 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:55.305 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:55.306 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:55.306 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:55.306 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:55.306 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:55.306 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:55.306 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:55.306 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:55.306 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:55.306 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.301+0000 7f4f92303640 1 Processor -- start 2024-08-19T10:39:55.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.301+0000 7f4f92303640 1 -- start start 2024-08-19T10:39:55.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.301+0000 7f4f92303640 10 monclient: init 2024-08-19T10:39:55.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:55.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 10 monclient: _add_conns ranks=[4,7,10,0,9,1,6,2,3,8,5] 2024-08-19T10:39:55.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f8c0a6740 0x7f4f8c0a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 10 monclient(hunting): picked mon.noname-c con 0x7f4f8c0a6740 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4f8c0a5bc0 0x7f4f8c0a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 10 monclient(hunting): picked mon.noname-i con 0x7f4f8c0a5bc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4f8c0a7060 0x7f4f8c139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 10 monclient(hunting): picked mon.noname-f con 0x7f4f8c0a7060 addr v2:172.21.15.190:3300/0 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f4f8c0a4c20 con 0x7f4f8c0a7060 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f4f8c0a3530 con 0x7f4f8c0a6740 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4f8c0a3740 con 0x7f4f8c0a5bc0 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:55.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f92303640 10 monclient(hunting): authenticate will time out at 1040.876369s 2024-08-19T10:39:55.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8b7fe640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4f8c0a5bc0 msgr2=0x7f4f8c0a6b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:55.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f908a0640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f4f8c0a7060 msgr2=0x7f4f8c139590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:55.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8b7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4f8c0a5bc0 0x7f4f8c0a6b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:55.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f908a0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4f8c0a7060 0x7f4f8c139590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:55.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f8c0a6740 0x7f4f8c0a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:55.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f8c0a6740 0x7f4f8c0a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:35500/0 (socket says 172.21.15.190:35500) 2024-08-19T10:39:55.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 1 -- 172.21.15.190:0/1106841956 learned_addr learned my addr 172.21.15.190:0/1106841956 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:55.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 10 monclient(hunting): get_auth_request con 0x7f4f8c0a6740 auth_method 0 2024-08-19T10:39:55.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:55.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:55.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_done global_id 4765 payload 274 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 1 -- 172.21.15.190:0/1106841956 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4f8c0a5bc0 msgr2=0x7f4f8c0a6b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 1 --2- 172.21.15.190:0/1106841956 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4f8c0a5bc0 0x7f4f8c0a6b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 1 -- 172.21.15.190:0/1106841956 >> v2:172.21.15.190:3300/0 conn(0x7f4f8c0a7060 msgr2=0x7f4f8c139590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 1 --2- 172.21.15.190:0/1106841956 >> v2:172.21.15.190:3300/0 conn(0x7f4f8c0a7060 0x7f4f8c139590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 1 monclient: found mon.noname-c 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:55.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:55.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.305+0000 7f4f8bfff640 1 -- 172.21.15.190:0/1106841956 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c0a3240 con 0x7f4f8c0a6740 2024-08-19T10:39:55.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8bfff640 1 -- 172.21.15.190:0/1106841956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f8c0a6740 msgr2=0x7f4f8c0a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:55.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8bfff640 1 --2- 172.21.15.190:0/1106841956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f8c0a6740 0x7f4f8c0a5680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009870 tx=0x7f4f7c01c460 comp rx=0 tx=0).ready entity=mon.4 client_cookie=50df47ad2239988f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:55.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 1 -- 172.21.15.190:0/1106841956 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4f7c0ab870 con 0x7f4f8c0a6740 2024-08-19T10:39:55.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:55.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:55.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 10 monclient: dump: 2024-08-19T10:39:55.317 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:55.317 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:55.317 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.317 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.317 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:55.317 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.317 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:55.317 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 10 monclient: _finish_auth 0 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 10 monclient: _check_auth_tickets 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:55.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 1 -- 172.21.15.190:0/1106841956 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f7c0b0070 con 0x7f4f8c0a6740 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 1 -- 172.21.15.190:0/1106841956 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4f7c0aa440 con 0x7f4f8c0a6740 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f8affd640 10 monclient: dump: 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:55.319 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 5 monclient: authenticate success, global_id 4765 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 4 set_mon_vals no callback set 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 10 monclient: shutdown 2024-08-19T10:39:55.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 1 -- 172.21.15.190:0/1106841956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f8c0a6740 msgr2=0x7f4f8c0a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 1 --2- 172.21.15.190:0/1106841956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f8c0a6740 0x7f4f8c0a5680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009870 tx=0x7f4f7c01c460 comp rx=0 tx=0).stop 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 1 -- 172.21.15.190:0/1106841956 shutdown_connections 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 1 --2- 172.21.15.190:0/1106841956 >> v2:172.21.15.190:3300/0 conn(0x7f4f8c0a7060 0x7f4f8c139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 1 --2- 172.21.15.190:0/1106841956 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4f8c0a5bc0 0x7f4f8c0a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 1 --2- 172.21.15.190:0/1106841956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4f8c0a6740 0x7f4f8c0a5680 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 1 -- 172.21.15.190:0/1106841956 >> 172.21.15.190:0/1106841956 conn(0x7f4f8c0abd40 msgr2=0x7f4f8c0ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 1 -- 172.21.15.190:0/1106841956 shutdown_connections 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 1 -- 172.21.15.190:0/1106841956 wait complete. 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 10 monclient: build_initial_monmap 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.309+0000 7f4f92303640 10 monclient: monmap: 2024-08-19T10:39:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 1 Processor -- start 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 1 -- start start 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 10 monclient: init 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 10 monclient: _add_conns ranks=[6,1,0,8,2,4,3,7,5] 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4f8c0a6740 0x7f4f8c09cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 10 monclient(hunting): picked mon.noname-d con 0x7f4f8c0a6740 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f8c0a7060 0x7f4f8c0a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 10 monclient(hunting): picked mon.noname-f con 0x7f4f8c0a7060 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8c0a2860 0x7f4f8c1d1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 10 monclient(hunting): picked mon.noname-a con 0x7f4f8c0a2860 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:55.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c143450 con 0x7f4f8c0a2860 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1432d0 con 0x7f4f8c0a7060 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4f8c1435d0 con 0x7f4f8c0a6740 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f92303640 10 monclient(hunting): authenticate will time out at 1040.884720s 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f8b7fe640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f8c0a7060 msgr2=0x7f4f8c0a2320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f8b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f8c0a7060 0x7f4f8c0a2320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f8bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4f8c0a6740 0x7f4f8c09cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8c0a2860 0x7f4f8c1d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8c0a2860 0x7f4f8c1d1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:38490/0 (socket says 172.21.15.190:38490) 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 1 -- 172.21.15.190:0/2204940581 learned_addr learned my addr 172.21.15.190:0/2204940581 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:55.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 10 monclient(hunting): get_auth_request con 0x7f4f8c0a2860 auth_method 0 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f8bfff640 10 monclient(hunting): get_auth_request con 0x7f4f8c0a6740 auth_method 0 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f8bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:55.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 10 monclient(hunting): handle_auth_done global_id 4815 payload 824 2024-08-19T10:39:55.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 1 -- 172.21.15.190:0/2204940581 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4f8c0a6740 msgr2=0x7f4f8c09cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:55.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4f8c0a6740 0x7f4f8c09cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 1 -- 172.21.15.190:0/2204940581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f8c0a7060 msgr2=0x7f4f8c0a2320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:55.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f8c0a7060 0x7f4f8c0a2320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 10 monclient: _finish_hunting 0 2024-08-19T10:39:55.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 1 monclient: found mon.noname-a 2024-08-19T10:39:55.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:55.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:39:55.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.313+0000 7f4f908a0640 1 -- 172.21.15.190:0/2204940581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c13f7c0 con 0x7f4f8c0a2860 2024-08-19T10:39:55.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f908a0640 1 -- 172.21.15.190:0/2204940581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8c0a2860 msgr2=0x7f4f8c1d1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:55.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f908a0640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8c0a2860 0x7f4f8c1d1870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400be60 tx=0x7f4f8400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 1 -- 172.21.15.190:0/2204940581 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4f840158e0 con 0x7f4f8c0a2860 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 10 monclient: dump: 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:55.329 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 10 monclient: _finish_auth 0 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 1 -- 172.21.15.190:0/2204940581 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f84010070 con 0x7f4f8c0a2860 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:55.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f92303640 5 monclient: authenticate success, global_id 4815 2024-08-19T10:39:55.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f92303640 10 monclient: _renew_subs 2024-08-19T10:39:55.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f92303640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:55.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f92303640 1 -- 172.21.15.190:0/2204940581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8c09e3b0 con 0x7f4f8c0a2860 2024-08-19T10:39:55.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f89ffb640 4 set_mon_vals no callback set 2024-08-19T10:39:55.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f92303640 10 monclient: _renew_subs 2024-08-19T10:39:55.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f92303640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:55.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f92303640 1 -- 172.21.15.190:0/2204940581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8c0a9020 con 0x7f4f8c0a2860 2024-08-19T10:39:55.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 1 -- 172.21.15.190:0/2204940581 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4f84015ae0 con 0x7f4f8c0a2860 2024-08-19T10:39:55.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 10 monclient: dump: 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:55.332 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f92303640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f92303640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f92303640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f92303640 1 -- 172.21.15.190:0/2204940581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f8c0a5680 con 0x7f4f8c0a2860 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 1 -- 172.21.15.190:0/2204940581 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4f84015470 con 0x7f4f8c0a2860 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f7803be30 0x7f4f7803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f88ff9640 1 -- 172.21.15.190:0/2204940581 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4438+0+0 (secure 0 0 0) 0x7f4f8404f500 con 0x7f4f8c0a2860 2024-08-19T10:39:55.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f8bfff640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f7803be30 0x7f4f7803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:55.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.317+0000 7f4f8bfff640 10 monclient: get_auth_request con 0x7f4f7803be30 auth_method 0 2024-08-19T10:39:55.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.321+0000 7f4f88ff9640 1 -- 172.21.15.190:0/2204940581 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:39:55.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.321+0000 7f4f8bfff640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f7803be30 0x7f4f7803e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009870 tx=0x7f4f7c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:55.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.329+0000 7f4f88ff9640 1 -- 172.21.15.190:0/2204940581 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4f8404d0a0 con 0x7f4f8c0a2860 2024-08-19T10:39:55.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.329+0000 7f4f88ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:55.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.329+0000 7f4f88ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:55.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.501+0000 7f4f92303640 10 start_mon_command cmd=[{"prefix": "osd crush rule list"}] 2024-08-19T10:39:55.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.501+0000 7f4f92303640 10 monclient: _send_command 2 [{"prefix": "osd crush rule list"}] 2024-08-19T10:39:55.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.501+0000 7f4f92303640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:55.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.501+0000 7f4f92303640 1 -- 172.21.15.190:0/2204940581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule list"} v 0) -- 0x7f4f8c09d9f0 con 0x7f4f8c0a2860 2024-08-19T10:39:55.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.505+0000 7f4f88ff9640 1 -- 172.21.15.190:0/2204940581 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule list"}]=0 v45) ==== 67+0+29 (secure 0 0 0) 0x7f4f8c09d9f0 con 0x7f4f8c0a2860 2024-08-19T10:39:55.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.505+0000 7f4f88ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule list"}] 2024-08-19T10:39:55.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.505+0000 7f4f88ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:39:55.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.605+0000 7f4f92303640 10 start_mon_command cmd=[{"prefix": "osd crush rule list"}] 2024-08-19T10:39:55.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.605+0000 7f4f92303640 10 monclient: _send_command 3 [{"prefix": "osd crush rule list"}] 2024-08-19T10:39:55.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.605+0000 7f4f92303640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:55.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.605+0000 7f4f92303640 1 -- 172.21.15.190:0/2204940581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule list"} v 0) -- 0x7f4f8c0008e0 con 0x7f4f8c0a2860 2024-08-19T10:39:55.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.605+0000 7f4f88ff9640 1 -- 172.21.15.190:0/2204940581 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule list"}]=0 v45) ==== 67+0+29 (secure 0 0 0) 0x7f4f8404f280 con 0x7f4f8c0a2860 2024-08-19T10:39:55.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.605+0000 7f4f88ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule list"}] 2024-08-19T10:39:55.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.605+0000 7f4f88ff9640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:39:55.611 INFO:tasks.workunit.client.0.smithi190.stdout:replicated_rule 2024-08-19T10:39:55.611 INFO:tasks.workunit.client.0.smithi190.stdout:erasure-code 2024-08-19T10:39:55.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 -- 172.21.15.190:0/2204940581 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f7803be30 msgr2=0x7f4f7803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:55.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f7803be30 0x7f4f7803e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009870 tx=0x7f4f7c0a7000 comp rx=0 tx=0).stop 2024-08-19T10:39:55.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 10 monclient: shutdown 2024-08-19T10:39:55.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:55.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 -- 172.21.15.190:0/2204940581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8c0a2860 msgr2=0x7f4f8c1d1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:55.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8c0a2860 0x7f4f8c1d1870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400be60 tx=0x7f4f8400bf50 comp rx=0 tx=0).stop 2024-08-19T10:39:55.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 -- 172.21.15.190:0/2204940581 shutdown_connections 2024-08-19T10:39:55.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f7803be30 0x7f4f7803e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f8c0a2860 0x7f4f8c1d1870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f8c0a7060 0x7f4f8c0a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 --2- 172.21.15.190:0/2204940581 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4f8c0a6740 0x7f4f8c09cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 -- 172.21.15.190:0/2204940581 >> 172.21.15.190:0/2204940581 conn(0x7f4f8c0abd40 msgr2=0x7f4f8c13a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:55.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 -- 172.21.15.190:0/2204940581 shutdown_connections 2024-08-19T10:39:55.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.609+0000 7f4f92303640 1 -- 172.21.15.190:0/2204940581 wait complete. 2024-08-19T10:39:55.627 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule create-simple foo default host 2024-08-19T10:39:55.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient: get_monmap_and_config 2024-08-19T10:39:55.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient: build_initial_monmap 2024-08-19T10:39:55.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient: monmap: 2024-08-19T10:39:55.744 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:55.745554+0000 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:55.745554+0000 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:55.745 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:55.746 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:55.746 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:55.746 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:55.746 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 1 Processor -- start 2024-08-19T10:39:55.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 1 -- start start 2024-08-19T10:39:55.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient: init 2024-08-19T10:39:55.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:55.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient: _add_conns ranks=[6,0,4,3,10,8,1,5,9,7,2] 2024-08-19T10:39:55.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f86140a46b0 0x7f86140a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient(hunting): picked mon.noname-d con 0x7f86140a46b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:55.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f86140a28e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f86140a4fd0 0x7f8614137b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient(hunting): picked mon.noname-c con 0x7f86140a4fd0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86140a26a0 con 0x7f86140a28e0 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f86140a2d60 con 0x7f86140a4fd0 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f86140a2fa0 con 0x7f86140a46b0 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.741+0000 7f861b277640 10 monclient(hunting): authenticate will time out at 1041.315720s 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619814640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f86140a4fd0 0x7f8614137b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f86140a46b0 0x7f86140a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8618812640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f86140a28e0 0x7f8614137470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:56144/0 (socket says 172.21.15.190:56144) 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8618812640 1 -- 172.21.15.190:0/3154551917 learned_addr learned my addr 172.21.15.190:0/3154551917 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:39:55.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619814640 10 monclient(hunting): get_auth_request con 0x7f86140a4fd0 auth_method 0 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619814640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619814640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619814640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 10 monclient(hunting): get_auth_request con 0x7f86140a46b0 auth_method 0 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619814640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619814640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619814640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:55.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:55.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 10 monclient(hunting): handle_auth_done global_id 4632 payload 274 2024-08-19T10:39:55.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 1 -- 172.21.15.190:0/3154551917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f86140a4fd0 msgr2=0x7f8614137b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:55.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 1 --2- 172.21.15.190:0/3154551917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f86140a4fd0 0x7f8614137b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 1 -- 172.21.15.190:0/3154551917 >> v1:172.21.15.89:6789/0 conn(0x7f86140a28e0 legacy=0x7f8614137470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:55.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 10 monclient: _finish_hunting 0 2024-08-19T10:39:55.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 1 monclient: found mon.noname-d 2024-08-19T10:39:55.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:55.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:55.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.745+0000 7f8619013640 1 -- 172.21.15.190:0/3154551917 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861409cc70 con 0x7f86140a46b0 2024-08-19T10:39:55.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f8619013640 1 -- 172.21.15.190:0/3154551917 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f86140a46b0 msgr2=0x7f86140a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:55.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f8619013640 1 --2- 172.21.15.190:0/3154551917 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f86140a46b0 0x7f86140a4a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f86040097f0 tx=0x7f860401c340 comp rx=0 tx=0).ready entity=mon.6 client_cookie=6a729bba6ad328d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:55.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 1 -- 172.21.15.190:0/3154551917 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f86040b0070 con 0x7f86140a46b0 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 10 monclient: dump: 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:55.754 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 10 monclient: _finish_auth 0 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 1 -- 172.21.15.190:0/3154551917 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86040aa810 con 0x7f86140a46b0 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:55.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 1 -- 172.21.15.190:0/3154551917 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f86040a4e60 con 0x7f86140a46b0 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f860bfff640 10 monclient: dump: 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:55.756 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 5 monclient: authenticate success, global_id 4632 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 4 set_mon_vals no callback set 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 10 monclient: shutdown 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 1 -- 172.21.15.190:0/3154551917 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f86140a46b0 msgr2=0x7f86140a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 1 --2- 172.21.15.190:0/3154551917 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f86140a46b0 0x7f86140a4a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f86040097f0 tx=0x7f860401c340 comp rx=0 tx=0).stop 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 1 -- 172.21.15.190:0/3154551917 shutdown_connections 2024-08-19T10:39:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 1 --2- 172.21.15.190:0/3154551917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f86140a4fd0 0x7f8614137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 1 --2- 172.21.15.190:0/3154551917 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f86140a46b0 0x7f86140a4a90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 1 -- 172.21.15.190:0/3154551917 >> 172.21.15.190:0/3154551917 conn(0x7f86140a92a0 msgr2=0x7f86140a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 1 -- 172.21.15.190:0/3154551917 shutdown_connections 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 1 -- 172.21.15.190:0/3154551917 wait complete. 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 10 monclient: build_initial_monmap 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 10 monclient: monmap: 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.758 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 1 Processor -- start 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 1 -- start start 2024-08-19T10:39:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.749+0000 7f861b277640 10 monclient: init 2024-08-19T10:39:55.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:55.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient: _add_conns ranks=[7,5,3,0,2,6,1,8,4] 2024-08-19T10:39:55.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f86140a28e0 0x7f8614136f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient(hunting): picked mon.noname-i con 0x7f86140a28e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:55.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f86140a46b0 0x7f8614137450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient(hunting): picked mon.noname-h con 0x7f86140a46b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140a4fd0 0x7f861413cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient(hunting): picked mon.noname-g con 0x7f86140a4fd0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f861409fda0 con 0x7f86140a4fd0 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f861409fc20 con 0x7f86140a46b0 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f86141380c0 con 0x7f86140a28e0 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient(hunting): authenticate will time out at 1041.324191s 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619013640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f86140a28e0 msgr2=0x7f8614136f10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619013640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f86140a28e0 0x7f8614136f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140a4fd0 0x7f861413cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:55.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8618812640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f86140a46b0 0x7f8614137450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140a4fd0 0x7f861413cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:41418/0 (socket says 172.21.15.190:41418) 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 1 -- 172.21.15.190:0/1222082996 learned_addr learned my addr 172.21.15.190:0/1222082996 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 10 monclient(hunting): get_auth_request con 0x7f86140a4fd0 auth_method 0 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8618812640 10 monclient(hunting): get_auth_request con 0x7f86140a46b0 auth_method 0 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8618812640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8618812640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8618812640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:55.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8618812640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8618812640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8618812640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 10 monclient(hunting): handle_auth_done global_id 4692 payload 824 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 1 -- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f86140a28e0 msgr2=0x7f8614136f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f86140a28e0 0x7f8614136f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 1 -- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f86140a46b0 msgr2=0x7f8614137450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f86140a46b0 0x7f8614137450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 10 monclient: _finish_hunting 0 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 1 monclient: found mon.noname-g 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 1 -- 172.21.15.190:0/1222082996 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86140a0500 con 0x7f86140a4fd0 2024-08-19T10:39:55.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 1 -- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140a4fd0 msgr2=0x7f861413cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8619814640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140a4fd0 0x7f861413cee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f860c00efc0 tx=0x7f860c00c490 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 1 -- 172.21.15.190:0/1222082996 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f860c018020 con 0x7f86140a4fd0 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 10 monclient: dump: 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:55.767 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 10 monclient: _finish_auth 0 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 1 -- 172.21.15.190:0/1222082996 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c0148f0 con 0x7f86140a4fd0 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 5 monclient: authenticate success, global_id 4692 2024-08-19T10:39:55.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient: _renew_subs 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 1 -- 172.21.15.190:0/1222082996 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86140a8100 con 0x7f86140a4fd0 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 1 -- 172.21.15.190:0/1222082996 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f860c0133e0 con 0x7f86140a4fd0 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f8609ffb640 10 monclient: dump: 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:55.769 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient: _renew_subs 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f860b7fe640 4 set_mon_vals no callback set 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.753+0000 7f861b277640 1 -- 172.21.15.190:0/1222082996 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f86140a4080 con 0x7f86140a4fd0 2024-08-19T10:39:55.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f861b277640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f861b277640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f861b277640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f861b277640 1 -- 172.21.15.190:0/1222082996 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86140a4a90 con 0x7f86140a4fd0 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f8609ffb640 1 -- 172.21.15.190:0/1222082996 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f860c01b020 con 0x7f86140a4fd0 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f8609ffb640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85f803be00 0x7f85f803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f8609ffb640 1 -- 172.21.15.190:0/1222082996 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(45..45 src has 1..45) ==== 4438+0+0 (secure 0 0 0) 0x7f860c04f360 con 0x7f86140a4fd0 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f8619013640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85f803be00 0x7f85f803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f8619013640 10 monclient: get_auth_request con 0x7f85f803be00 auth_method 0 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f8619013640 1 -- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85f803be00 msgr2=0x7f85f803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f8619013640 1 -- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85f803be00 msgr2=0x7f85f803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f8619013640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85f803be00 0x7f85f803e2c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f86040ab000 tx=0x7f86040a6000 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.757+0000 7f8619013640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85f803be00 0x7f85f803e2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.761+0000 7f8609ffb640 1 -- 172.21.15.190:0/1222082996 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f860c022030 con 0x7f86140a4fd0 2024-08-19T10:39:55.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.761+0000 7f8609ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:55.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.761+0000 7f8609ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:55.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.917+0000 7f861b277640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:55.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.917+0000 7f861b277640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:55.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.917+0000 7f861b277640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:55.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.917+0000 7f861b277640 1 -- 172.21.15.190:0/1222082996 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) -- 0x7f86140a20a0 con 0x7f86140a4fd0 2024-08-19T10:39:55.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.961+0000 7f8619013640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85f803be00 0x7f85f803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:55.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.961+0000 7f8619013640 10 monclient: get_auth_request con 0x7f85f803be00 auth_method 0 2024-08-19T10:39:55.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:55.965+0000 7f8619013640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85f803be00 0x7f85f803e2c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8604000c10 tx=0x7f8604009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:56.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.277+0000 7f8609ffb640 1 -- 172.21.15.190:0/1222082996 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 v46) ==== 126+0+0 (secure 0 0 0) 0x7f860c010040 con 0x7f86140a4fd0 2024-08-19T10:39:56.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.277+0000 7f8609ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:56.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.277+0000 7f8609ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:39:56.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.381+0000 7f861b277640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:56.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.381+0000 7f861b277640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:56.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.381+0000 7f861b277640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:56.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.381+0000 7f861b277640 1 -- 172.21.15.190:0/1222082996 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) -- 0x7f86140008e0 con 0x7f86140a4fd0 2024-08-19T10:39:56.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f8609ffb640 1 -- 172.21.15.190:0/1222082996 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v46) ==== 149+0+0 (secure 0 0 0) 0x7f860c04f0d0 con 0x7f86140a4fd0 2024-08-19T10:39:56.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f8609ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:56.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f8609ffb640 10 monclient: _finish_command 3 = system:0 rule foo already exists 2024-08-19T10:39:56.387 INFO:tasks.workunit.client.0.smithi190.stderr:rule foo already exists 2024-08-19T10:39:56.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 -- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85f803be00 msgr2=0x7f85f803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:56.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85f803be00 0x7f85f803e2c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8604000c10 tx=0x7f8604009190 comp rx=0 tx=0).stop 2024-08-19T10:39:56.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 10 monclient: shutdown 2024-08-19T10:39:56.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:56.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 -- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140a4fd0 msgr2=0x7f861413cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:56.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140a4fd0 0x7f861413cee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f860c00efc0 tx=0x7f860c00c490 comp rx=0 tx=0).stop 2024-08-19T10:39:56.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 -- 172.21.15.190:0/1222082996 shutdown_connections 2024-08-19T10:39:56.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85f803be00 0x7f85f803e2c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f86140a46b0 0x7f8614137450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86140a4fd0 0x7f861413cee0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 --2- 172.21.15.190:0/1222082996 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f86140a28e0 0x7f8614136f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 -- 172.21.15.190:0/1222082996 >> 172.21.15.190:0/1222082996 conn(0x7f86140a92a0 msgr2=0x7f86140730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:56.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 -- 172.21.15.190:0/1222082996 shutdown_connections 2024-08-19T10:39:56.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.385+0000 7f861b277640 1 -- 172.21.15.190:0/1222082996 wait complete. 2024-08-19T10:39:56.404 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule create-simple foo default host 2024-08-19T10:39:56.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.517+0000 7f02028fa640 10 monclient: get_monmap_and_config 2024-08-19T10:39:56.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.517+0000 7f02028fa640 10 monclient: build_initial_monmap 2024-08-19T10:39:56.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.517+0000 7f02028fa640 10 monclient: monmap: 2024-08-19T10:39:56.522 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:56.522 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:56.523 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:56.522683+0000 2024-08-19T10:39:56.523 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:56.522683+0000 2024-08-19T10:39:56.523 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:56.523 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.523 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:56.523 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:56.524 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:56.524 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:56.524 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:56.524 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:56.524 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:56.525 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:56.525 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:56.525 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:56.525 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:56.525 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.517+0000 7f02028fa640 1 Processor -- start 2024-08-19T10:39:56.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.517+0000 7f02028fa640 1 -- start start 2024-08-19T10:39:56.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.517+0000 7f02028fa640 10 monclient: init 2024-08-19T10:39:56.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:56.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 10 monclient: _add_conns ranks=[6,4,5,1,9,0,8,3,10,2,7] 2024-08-19T10:39:56.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f01fc1396a0 0x7f01fc139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 10 monclient(hunting): picked mon.noname-d con 0x7f01fc1396a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:56.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01fc139fc0 0x7f01fc0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 10 monclient(hunting): picked mon.noname-c con 0x7f01fc139fc0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:56.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc0a6e20 0x7f01fc0a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 10 monclient(hunting): picked mon.noname-h con 0x7f01fc0a6e20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:56.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f01fc0a4f20 con 0x7f01fc139fc0 2024-08-19T10:39:56.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f01fc0a83e0 con 0x7f01fc0a6e20 2024-08-19T10:39:56.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f01fc0a2f30 con 0x7f01fc1396a0 2024-08-19T10:39:56.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:56.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f02028fa640 10 monclient(hunting): authenticate will time out at 1042.092628s 2024-08-19T10:39:56.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc0a6e20 0x7f01fc0a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:56.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc0a6e20 0x7f01fc0a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48040/0 (socket says 172.21.15.190:48040) 2024-08-19T10:39:56.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 1 -- 172.21.15.190:0/3771968401 learned_addr learned my addr 172.21.15.190:0/3771968401 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:56.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 10 monclient(hunting): get_auth_request con 0x7f01fc0a6e20 auth_method 0 2024-08-19T10:39:56.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:56.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:56.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:56.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:56.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:56.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:56.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f01fbfff640 1 --2- 172.21.15.190:0/3771968401 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f01fc1396a0 0x7f01fc139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:56.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f01f37fe640 1 --2- 172.21.15.190:0/3771968401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01fc139fc0 0x7f01fc0a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:56.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f01f37fe640 10 monclient(hunting): get_auth_request con 0x7f01fc139fc0 auth_method 0 2024-08-19T10:39:56.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f01f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:56.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f01f37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:56.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f01f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:56.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f01fbfff640 10 monclient(hunting): get_auth_request con 0x7f01fc1396a0 auth_method 0 2024-08-19T10:39:56.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f01fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:56.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f01fbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:56.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f01fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:56.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 10 monclient(hunting): handle_auth_done global_id 4712 payload 274 2024-08-19T10:39:56.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 1 -- 172.21.15.190:0/3771968401 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f01fc1396a0 msgr2=0x7f01fc139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:56.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 1 --2- 172.21.15.190:0/3771968401 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f01fc1396a0 0x7f01fc139a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 1 -- 172.21.15.190:0/3771968401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01fc139fc0 msgr2=0x7f01fc0a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:56.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 1 --2- 172.21.15.190:0/3771968401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01fc139fc0 0x7f01fc0a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 10 monclient: _finish_hunting 0 2024-08-19T10:39:56.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 1 monclient: found mon.noname-h 2024-08-19T10:39:56.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:56.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:56.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.521+0000 7f0200e97640 1 -- 172.21.15.190:0/3771968401 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01fc0a35e0 con 0x7f01fc0a6e20 2024-08-19T10:39:56.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f0200e97640 1 -- 172.21.15.190:0/3771968401 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc0a6e20 msgr2=0x7f01fc0a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:56.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f0200e97640 1 --2- 172.21.15.190:0/3771968401 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc0a6e20 0x7f01fc0a7220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009eb0 tx=0x7f01f401c8f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=aadf61d6b3a5c7f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:56.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 1 -- 172.21.15.190:0/3771968401 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01f40090d0 con 0x7f01fc0a6e20 2024-08-19T10:39:56.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:56.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:56.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 10 monclient: dump: 2024-08-19T10:39:56.552 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:56.552 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:56.552 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.552 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.552 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:56.553 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.553 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:56.553 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:56.553 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:56.553 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:56.553 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:56.554 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:56.554 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:56.554 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:56.554 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:56.554 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:56.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:56.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:56.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 1 -- 172.21.15.190:0/3771968401 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f40a6950 con 0x7f01fc0a6e20 2024-08-19T10:39:56.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:56.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 1 -- 172.21.15.190:0/3771968401 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01f40af480 con 0x7f01fc0a6e20 2024-08-19T10:39:56.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 5 monclient: authenticate success, global_id 4712 2024-08-19T10:39:56.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:56.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 4 set_mon_vals no callback set 2024-08-19T10:39:56.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:56.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:56.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f01fb7fe640 10 monclient: dump: 2024-08-19T10:39:56.557 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:56.557 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:56.557 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.557 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.557 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:56.558 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.558 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:56.558 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:56.558 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:56.558 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:56.559 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:56.559 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:56.559 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:56.559 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:56.559 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:56.560 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 10 monclient: shutdown 2024-08-19T10:39:56.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:56.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 1 -- 172.21.15.190:0/3771968401 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc0a6e20 msgr2=0x7f01fc0a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:56.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 1 --2- 172.21.15.190:0/3771968401 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc0a6e20 0x7f01fc0a7220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009eb0 tx=0x7f01f401c8f0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 1 -- 172.21.15.190:0/3771968401 shutdown_connections 2024-08-19T10:39:56.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 1 --2- 172.21.15.190:0/3771968401 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc0a6e20 0x7f01fc0a7220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 1 --2- 172.21.15.190:0/3771968401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f01fc139fc0 0x7f01fc0a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 1 --2- 172.21.15.190:0/3771968401 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f01fc1396a0 0x7f01fc139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 1 -- 172.21.15.190:0/3771968401 >> 172.21.15.190:0/3771968401 conn(0x7f01fc0abee0 msgr2=0x7f01fc0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:56.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 1 -- 172.21.15.190:0/3771968401 shutdown_connections 2024-08-19T10:39:56.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 1 -- 172.21.15.190:0/3771968401 wait complete. 2024-08-19T10:39:56.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 10 monclient: build_initial_monmap 2024-08-19T10:39:56.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 10 monclient: monmap: 2024-08-19T10:39:56.562 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:56.562 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:56.563 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:56.563 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:56.563 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:56.563 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.563 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:56.564 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:56.564 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:56.564 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:56.564 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:56.564 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:56.564 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:56.565 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:56.565 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:56.565 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 1 Processor -- start 2024-08-19T10:39:56.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 1 -- start start 2024-08-19T10:39:56.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.525+0000 7f02028fa640 10 monclient: init 2024-08-19T10:39:56.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:56.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient: _add_conns ranks=[6,5,7,1,4,8,0,2,3] 2024-08-19T10:39:56.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f01fc0a6e20 0x7f01fc134ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient(hunting): picked mon.noname-d con 0x7f01fc0a6e20 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:56.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc1396a0 0x7f01fc135020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient(hunting): picked mon.noname-h con 0x7f01fc1396a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:56.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f01fc139fc0 0x7f01fc09ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient(hunting): picked mon.noname-i con 0x7f01fc139fc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:56.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f01fc13a3a0 con 0x7f01fc1396a0 2024-08-19T10:39:56.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f01fc143390 con 0x7f01fc0a6e20 2024-08-19T10:39:56.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f01fc09fae0 con 0x7f01fc139fc0 2024-08-19T10:39:56.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:56.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient(hunting): authenticate will time out at 1042.100432s 2024-08-19T10:39:56.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f0200e97640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f01fc139fc0 msgr2=0x7f01fc09ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:56.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f0200e97640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f01fc139fc0 0x7f01fc09ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:56.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc1396a0 0x7f01fc135020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:56.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc1396a0 0x7f01fc135020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48044/0 (socket says 172.21.15.190:48044) 2024-08-19T10:39:56.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 1 -- 172.21.15.190:0/1364577141 learned_addr learned my addr 172.21.15.190:0/1364577141 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:56.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 10 monclient(hunting): get_auth_request con 0x7f01fc1396a0 auth_method 0 2024-08-19T10:39:56.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:56.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:56.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:56.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:56.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:56.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:56.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01fbfff640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f01fc0a6e20 0x7f01fc134ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:56.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 10 monclient(hunting): handle_auth_done global_id 4721 payload 824 2024-08-19T10:39:56.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 1 -- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f01fc139fc0 msgr2=0x7f01fc09ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:56.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f01fc139fc0 0x7f01fc09ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 1 -- 172.21.15.190:0/1364577141 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f01fc0a6e20 msgr2=0x7f01fc134ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:56.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f01fc0a6e20 0x7f01fc134ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:39:56.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 1 monclient: found mon.noname-h 2024-08-19T10:39:56.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:56.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:39:56.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 1 -- 172.21.15.190:0/1364577141 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01fc0ac4a0 con 0x7f01fc1396a0 2024-08-19T10:39:56.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01fbfff640 10 monclient: get_auth_request con 0x7f01fc0a6e20 auth_method 0 2024-08-19T10:39:56.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01fbfff640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f01fc0a6e20 0x7f01fc134ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:39:56.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 1 -- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc1396a0 msgr2=0x7f01fc135020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:56.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f37fe640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc1396a0 0x7f01fc135020 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f01e400d5f0 tx=0x7f01e400dab0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:56.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 1 -- 172.21.15.190:0/1364577141 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01e4007380 con 0x7f01fc1396a0 2024-08-19T10:39:56.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:56.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:39:56.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 10 monclient: dump: 2024-08-19T10:39:56.575 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:56.575 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:56.575 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.575 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.575 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:56.576 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.576 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:56.576 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:56.576 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:56.576 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:56.577 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:56.577 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:56.577 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:56.577 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:56.577 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:56.577 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 10 monclient: _finish_auth 0 2024-08-19T10:39:56.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 10 monclient: _check_auth_tickets 2024-08-19T10:39:56.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:56.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 5 monclient: authenticate success, global_id 4721 2024-08-19T10:39:56.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 1 -- 172.21.15.190:0/1364577141 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e4019070 con 0x7f01fc1396a0 2024-08-19T10:39:56.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:56.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 1 -- 172.21.15.190:0/1364577141 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f01e4012c70 con 0x7f01fc1396a0 2024-08-19T10:39:56.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient: _renew_subs 2024-08-19T10:39:56.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:56.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 1 -- 172.21.15.190:0/1364577141 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01fc140820 con 0x7f01fc1396a0 2024-08-19T10:39:56.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01faffd640 4 set_mon_vals no callback set 2024-08-19T10:39:56.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:56.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:39:56.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f01f97fa640 10 monclient: dump: 2024-08-19T10:39:56.580 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:56.580 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:56.581 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.581 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.581 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:56.581 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.581 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:56.581 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:56.582 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:56.582 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:56.582 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:56.582 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:56.582 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:56.583 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:56.583 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:56.583 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient: _renew_subs 2024-08-19T10:39:56.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:56.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.529+0000 7f02028fa640 1 -- 172.21.15.190:0/1364577141 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f01fc0a1050 con 0x7f01fc1396a0 2024-08-19T10:39:56.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.533+0000 7f02028fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:56.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.533+0000 7f02028fa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:56.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.533+0000 7f02028fa640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:56.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.533+0000 7f02028fa640 1 -- 172.21.15.190:0/1364577141 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01fc13a520 con 0x7f01fc1396a0 2024-08-19T10:39:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.533+0000 7f01f97fa640 1 -- 172.21.15.190:0/1364577141 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f01e4012800 con 0x7f01fc1396a0 2024-08-19T10:39:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.533+0000 7f01f97fa640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f01dc03be00 0x7f01dc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.533+0000 7f01f97fa640 1 -- 172.21.15.190:0/1364577141 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(46..46 src has 1..46) ==== 4497+0+0 (secure 0 0 0) 0x7f01e400f040 con 0x7f01fc1396a0 2024-08-19T10:39:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.533+0000 7f01fbfff640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f01dc03be00 0x7f01dc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.533+0000 7f01fbfff640 10 monclient: get_auth_request con 0x7f01dc03be00 auth_method 0 2024-08-19T10:39:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.533+0000 7f01fbfff640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f01dc03be00 0x7f01dc03e2c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009840 tx=0x7f01ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.537+0000 7f01f97fa640 1 -- 172.21.15.190:0/1364577141 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f01fc13a520 con 0x7f01fc1396a0 2024-08-19T10:39:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.537+0000 7f01f97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:56.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.537+0000 7f01f97fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:56.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.709+0000 7f02028fa640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:56.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.709+0000 7f02028fa640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:56.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.709+0000 7f02028fa640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:56.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.709+0000 7f02028fa640 1 -- 172.21.15.190:0/1364577141 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) -- 0x7f01fc139a80 con 0x7f01fc1396a0 2024-08-19T10:39:56.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.709+0000 7f01f97fa640 1 -- 172.21.15.190:0/1364577141 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v46) ==== 149+0+0 (secure 0 0 0) 0x7f01e4012e10 con 0x7f01fc1396a0 2024-08-19T10:39:56.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.713+0000 7f01f97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:56.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.713+0000 7f01f97fa640 10 monclient: _finish_command 2 = system:0 rule foo already exists 2024-08-19T10:39:56.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.813+0000 7f02028fa640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:56.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.813+0000 7f02028fa640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:56.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.813+0000 7f02028fa640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:39:56.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.813+0000 7f02028fa640 1 -- 172.21.15.190:0/1364577141 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"} v 0) -- 0x7f01fc0008e0 con 0x7f01fc1396a0 2024-08-19T10:39:56.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.817+0000 7f01f97fa640 1 -- 172.21.15.190:0/1364577141 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}]=0 rule foo already exists v46) ==== 149+0+0 (secure 0 0 0) 0x7f01e4011340 con 0x7f01fc1396a0 2024-08-19T10:39:56.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.817+0000 7f01f97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo", "root": "default", "type": "host"}] 2024-08-19T10:39:56.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.817+0000 7f01f97fa640 10 monclient: _finish_command 3 = system:0 rule foo already exists 2024-08-19T10:39:56.820 INFO:tasks.workunit.client.0.smithi190.stderr:rule foo already exists 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.817+0000 7f02028fa640 1 -- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f01dc03be00 msgr2=0x7f01dc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.817+0000 7f02028fa640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f01dc03be00 0x7f01dc03e2c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009840 tx=0x7f01ec008040 comp rx=0 tx=0).stop 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 10 monclient: shutdown 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 1 -- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc1396a0 msgr2=0x7f01fc135020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc1396a0 0x7f01fc135020 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f01e400d5f0 tx=0x7f01e400dab0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 1 -- 172.21.15.190:0/1364577141 shutdown_connections 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f01dc03be00 0x7f01dc03e2c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f01fc139fc0 0x7f01fc09ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f01fc1396a0 0x7f01fc135020 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 1 --2- 172.21.15.190:0/1364577141 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f01fc0a6e20 0x7f01fc134ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 1 -- 172.21.15.190:0/1364577141 >> 172.21.15.190:0/1364577141 conn(0x7f01fc0abee0 msgr2=0x7f01fc0ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:56.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 1 -- 172.21.15.190:0/1364577141 shutdown_connections 2024-08-19T10:39:56.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.821+0000 7f02028fa640 1 -- 172.21.15.190:0/1364577141 wait complete. 2024-08-19T10:39:56.838 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule create-simple bar default host 2024-08-19T10:39:56.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient: get_monmap_and_config 2024-08-19T10:39:56.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient: build_initial_monmap 2024-08-19T10:39:56.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient: monmap: 2024-08-19T10:39:56.960 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:56.961407+0000 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:56.961407+0000 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:56.961 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:56.962 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:56.962 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:56.962 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:56.962 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 1 Processor -- start 2024-08-19T10:39:56.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 1 -- start start 2024-08-19T10:39:56.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient: init 2024-08-19T10:39:56.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:56.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient: _add_conns ranks=[9,7,4,10,5,6,1,8,3,2,0] 2024-08-19T10:39:56.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd8941396a0 0x7fd894139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient(hunting): picked mon.noname-a con 0x7fd8941396a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:56.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd894139fc0 0x7fd8940a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient(hunting): picked mon.noname-i con 0x7fd894139fc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:56.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8940a6e20 0x7fd8940a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient(hunting): picked mon.noname-c con 0x7fd8940a6e20 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fd8940a4f20 con 0x7fd8941396a0 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd8940a83e0 con 0x7fd8940a6e20 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd8940a2f30 con 0x7fd894139fc0 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd89a808640 10 monclient(hunting): authenticate will time out at 1042.531267s 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd88bfff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd894139fc0 msgr2=0x7fd8940a68e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd88bfff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd894139fc0 0x7fd8940a68e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd898da5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8940a6e20 0x7fd8940a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd893fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd8941396a0 0x7fd894139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:56.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd898da5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8940a6e20 0x7fd8940a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:49542/0 (socket says 172.21.15.190:49542) 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd898da5640 1 -- 172.21.15.190:0/3109394398 learned_addr learned my addr 172.21.15.190:0/3109394398 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd898da5640 10 monclient(hunting): get_auth_request con 0x7fd8940a6e20 auth_method 0 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd898da5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd898da5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd898da5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd893fff640 10 monclient(hunting): get_auth_request con 0x7fd8941396a0 auth_method 0 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd893fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd893fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd893fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd898da5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd898da5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:56.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd898da5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.957+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd898da5640 10 monclient(hunting): handle_auth_done global_id 4792 payload 274 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd898da5640 1 -- 172.21.15.190:0/3109394398 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd894139fc0 msgr2=0x7fd8940a68e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd898da5640 1 --2- 172.21.15.190:0/3109394398 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd894139fc0 0x7fd8940a68e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd898da5640 1 -- 172.21.15.190:0/3109394398 >> v2:172.21.15.89:3300/0 conn(0x7fd8941396a0 msgr2=0x7fd894139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd898da5640 1 --2- 172.21.15.190:0/3109394398 >> v2:172.21.15.89:3300/0 conn(0x7fd8941396a0 0x7fd894139a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd898da5640 10 monclient: _finish_hunting 0 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd898da5640 1 monclient: found mon.noname-c 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd898da5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd898da5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:56.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.961+0000 7fd898da5640 1 -- 172.21.15.190:0/3109394398 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8940a35e0 con 0x7fd8940a6e20 2024-08-19T10:39:56.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd898da5640 1 -- 172.21.15.190:0/3109394398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8940a6e20 msgr2=0x7fd8940a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:56.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd898da5640 1 --2- 172.21.15.190:0/3109394398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8940a6e20 0x7fd8940a7220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009e80 tx=0x7fd88c01c8f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8cc30e7c80ce416b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:56.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 1 -- 172.21.15.190:0/3109394398 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd88c0090d0 con 0x7fd8940a6e20 2024-08-19T10:39:56.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:56.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:56.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 10 monclient: dump: 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:56.970 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 1 -- 172.21.15.190:0/3109394398 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd88c0a59a0 con 0x7fd8940a6e20 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 1 -- 172.21.15.190:0/3109394398 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd88c0ad3f0 con 0x7fd8940a6e20 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd8937fe640 10 monclient: dump: 2024-08-19T10:39:56.971 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:56.972 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 5 monclient: authenticate success, global_id 4792 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 4 set_mon_vals no callback set 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient: shutdown 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 -- 172.21.15.190:0/3109394398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8940a6e20 msgr2=0x7fd8940a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 --2- 172.21.15.190:0/3109394398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8940a6e20 0x7fd8940a7220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009e80 tx=0x7fd88c01c8f0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 -- 172.21.15.190:0/3109394398 shutdown_connections 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 --2- 172.21.15.190:0/3109394398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8940a6e20 0x7fd8940a7220 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 --2- 172.21.15.190:0/3109394398 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd894139fc0 0x7fd8940a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 --2- 172.21.15.190:0/3109394398 >> v2:172.21.15.89:3300/0 conn(0x7fd8941396a0 0x7fd894139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 -- 172.21.15.190:0/3109394398 >> 172.21.15.190:0/3109394398 conn(0x7fd8940abee0 msgr2=0x7fd8940ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 -- 172.21.15.190:0/3109394398 shutdown_connections 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 -- 172.21.15.190:0/3109394398 wait complete. 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient: build_initial_monmap 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient: monmap: 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:56.974 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:56.975 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:56.975 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:56.975 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:56.975 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:56.975 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:56.975 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:56.975 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 Processor -- start 2024-08-19T10:39:56.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 -- start start 2024-08-19T10:39:56.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient: init 2024-08-19T10:39:56.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:56.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient: _add_conns ranks=[6,0,1,2,7,4,8,5,3] 2024-08-19T10:39:56.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8940a6e20 0x7fd894134c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient(hunting): picked mon.noname-d con 0x7fd8940a6e20 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941396a0 0x7fd89409ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient(hunting): picked mon.noname-a con 0x7fd8941396a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd894139fc0 0x7fd89409cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient(hunting): picked mon.noname-f con 0x7fd894139fc0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd89413a3a0 con 0x7fd8941396a0 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd894143390 con 0x7fd894139fc0 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd89409ffa0 con 0x7fd8940a6e20 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:56.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd89a808640 10 monclient(hunting): authenticate will time out at 1042.539652s 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd898da5640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd894139fc0 msgr2=0x7fd89409cfa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.965+0000 7fd898da5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd894139fc0 0x7fd89409cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd893fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8940a6e20 0x7fd894134c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941396a0 0x7fd89409ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd893fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8940a6e20 0x7fd894134c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:56582/0 (socket says 172.21.15.190:56582) 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd893fff640 1 -- 172.21.15.190:0/3051597320 learned_addr learned my addr 172.21.15.190:0/3051597320 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd893fff640 10 monclient(hunting): get_auth_request con 0x7fd8940a6e20 auth_method 0 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd893fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd893fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd893fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 10 monclient(hunting): get_auth_request con 0x7fd8941396a0 auth_method 0 2024-08-19T10:39:56.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:56.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:56.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:56.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:56.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:56.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd893fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:56.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:56.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:56.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:56.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 10 monclient(hunting): handle_auth_done global_id 4833 payload 824 2024-08-19T10:39:56.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 1 -- 172.21.15.190:0/3051597320 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8940a6e20 msgr2=0x7fd894134c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:56.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8940a6e20 0x7fd894134c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 1 -- 172.21.15.190:0/3051597320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd894139fc0 msgr2=0x7fd89409cfa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd894139fc0 0x7fd89409cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 1 monclient: found mon.noname-a 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 1 -- 172.21.15.190:0/3051597320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8940a04d0 con 0x7fd8941396a0 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd893fff640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8940a6e20 0x7fd894134c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 1 -- 172.21.15.190:0/3051597320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941396a0 msgr2=0x7fd89409ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd88bfff640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941396a0 0x7fd89409ca60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00b430 tx=0x7fd87c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 1 -- 172.21.15.190:0/3051597320 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd87c016070 con 0x7fd8941396a0 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:56.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 10 monclient: dump: 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:56.984 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 10 monclient: _finish_auth 0 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 10 monclient: _check_auth_tickets 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 1 -- 172.21.15.190:0/3051597320 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd87c003de0 con 0x7fd8941396a0 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 1 -- 172.21.15.190:0/3051597320 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd87c0103a0 con 0x7fd8941396a0 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:39:56.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd8917fa640 10 monclient: dump: 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:56.986 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.969+0000 7fd892ffd640 4 set_mon_vals no callback set 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd89a808640 5 monclient: authenticate success, global_id 4833 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd89a808640 10 monclient: _renew_subs 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd89a808640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd89a808640 1 -- 172.21.15.190:0/3051597320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd894140840 con 0x7fd8941396a0 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd89a808640 10 monclient: _renew_subs 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd89a808640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd89a808640 1 -- 172.21.15.190:0/3051597320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8940a11b0 con 0x7fd8941396a0 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd89a808640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd89a808640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd89a808640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:56.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd89a808640 1 -- 172.21.15.190:0/3051597320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89413a520 con 0x7fd8941396a0 2024-08-19T10:39:56.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd8917fa640 1 -- 172.21.15.190:0/3051597320 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd87c005070 con 0x7fd8941396a0 2024-08-19T10:39:56.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd8917fa640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd87403be00 0x7fd87403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:56.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd8917fa640 1 -- 172.21.15.190:0/3051597320 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4497+0+0 (secure 0 0 0) 0x7fd87c04dfb0 con 0x7fd8941396a0 2024-08-19T10:39:56.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd893fff640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd87403be00 0x7fd87403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:56.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.973+0000 7fd893fff640 10 monclient: get_auth_request con 0x7fd87403be00 auth_method 0 2024-08-19T10:39:56.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.977+0000 7fd893fff640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd87403be00 0x7fd87403e2c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd884011310 tx=0x7fd884009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:56.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.985+0000 7fd8917fa640 1 -- 172.21.15.190:0/3051597320 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd87c07ca40 con 0x7fd8941396a0 2024-08-19T10:39:56.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.985+0000 7fd8917fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:56.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:56.985+0000 7fd8917fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:57.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.157+0000 7fd89a808640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-08-19T10:39:57.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.157+0000 7fd89a808640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-08-19T10:39:57.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.157+0000 7fd89a808640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:57.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.157+0000 7fd89a808640 1 -- 172.21.15.190:0/3051597320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"} v 0) -- 0x7fd894139a80 con 0x7fd8941396a0 2024-08-19T10:39:57.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.397+0000 7fd8917fa640 1 -- 172.21.15.190:0/3051597320 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}]=0 v47) ==== 126+0+0 (secure 0 0 0) 0x7fd89413a520 con 0x7fd8941396a0 2024-08-19T10:39:57.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.397+0000 7fd8917fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-08-19T10:39:57.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.397+0000 7fd8917fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:39:57.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.501+0000 7fd89a808640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-08-19T10:39:57.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.501+0000 7fd89a808640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-08-19T10:39:57.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.501+0000 7fd89a808640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:39:57.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.501+0000 7fd89a808640 1 -- 172.21.15.190:0/3051597320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"} v 0) -- 0x7fd8940008e0 con 0x7fd8941396a0 2024-08-19T10:39:57.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd8917fa640 1 -- 172.21.15.190:0/3051597320 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}]=0 rule bar already exists v47) ==== 149+0+0 (secure 0 0 0) 0x7fd87c01f020 con 0x7fd8941396a0 2024-08-19T10:39:57.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd8917fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "bar", "root": "default", "type": "host"}] 2024-08-19T10:39:57.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd8917fa640 10 monclient: _finish_command 3 = system:0 rule bar already exists 2024-08-19T10:39:57.507 INFO:tasks.workunit.client.0.smithi190.stderr:rule bar already exists 2024-08-19T10:39:57.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 -- 172.21.15.190:0/3051597320 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd87403be00 msgr2=0x7fd87403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:57.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd87403be00 0x7fd87403e2c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd884011310 tx=0x7fd884009190 comp rx=0 tx=0).stop 2024-08-19T10:39:57.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 10 monclient: shutdown 2024-08-19T10:39:57.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:57.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 -- 172.21.15.190:0/3051597320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941396a0 msgr2=0x7fd89409ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:57.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941396a0 0x7fd89409ca60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00b430 tx=0x7fd87c00b8f0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 -- 172.21.15.190:0/3051597320 shutdown_connections 2024-08-19T10:39:57.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd87403be00 0x7fd87403e2c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd894139fc0 0x7fd89409cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8941396a0 0x7fd89409ca60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 --2- 172.21.15.190:0/3051597320 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8940a6e20 0x7fd894134c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 -- 172.21.15.190:0/3051597320 >> 172.21.15.190:0/3051597320 conn(0x7fd8940abee0 msgr2=0x7fd8940ab400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:57.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 -- 172.21.15.190:0/3051597320 shutdown_connections 2024-08-19T10:39:57.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.505+0000 7fd89a808640 1 -- 172.21.15.190:0/3051597320 wait complete. 2024-08-19T10:39:57.524 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm-device-class all 2024-08-19T10:39:57.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.637+0000 7efe7afa2640 10 monclient: get_monmap_and_config 2024-08-19T10:39:57.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.637+0000 7efe7afa2640 10 monclient: build_initial_monmap 2024-08-19T10:39:57.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.637+0000 7efe7afa2640 10 monclient: monmap: 2024-08-19T10:39:57.642 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:57.642 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:57.642 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:57.642303+0000 2024-08-19T10:39:57.642 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:57.642303+0000 2024-08-19T10:39:57.642 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:57.642 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:57.642 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:57.642 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:57.642 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:57.642 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:57.642 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:57.643 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:57.643 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:57.643 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:57.643 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:57.643 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:57.643 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:57.643 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:57.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.637+0000 7efe7afa2640 1 Processor -- start 2024-08-19T10:39:57.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.637+0000 7efe7afa2640 1 -- start start 2024-08-19T10:39:57.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.637+0000 7efe7afa2640 10 monclient: init 2024-08-19T10:39:57.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:57.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 10 monclient: _add_conns ranks=[6,9,4,5,0,8,2,3,7,1,10] 2024-08-19T10:39:57.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efe740a5940 0x7efe740720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:57.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 10 monclient(hunting): picked mon.noname-d con 0x7efe740a5940 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:57.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe740a5d20 0x7efe740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:57.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 10 monclient(hunting): picked mon.noname-a con 0x7efe740a5d20 addr v2:172.21.15.89:3300/0 2024-08-19T10:39:57.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe74139710 0x7efe74139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:57.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 10 monclient(hunting): picked mon.noname-c con 0x7efe74139710 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:57.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:57.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7efe7406df10 con 0x7efe740a5d20 2024-08-19T10:39:57.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:57.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7efe7406db00 con 0x7efe74139710 2024-08-19T10:39:57.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:57.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7efe7406efb0 con 0x7efe740a5940 2024-08-19T10:39:57.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:57.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7afa2640 10 monclient(hunting): authenticate will time out at 1043.212278s 2024-08-19T10:39:57.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe78d3e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efe740a5940 0x7efe740720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:57.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe74139710 0x7efe74139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:57.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe73fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe740a5d20 0x7efe740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:57.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe74139710 0x7efe74139bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:49556/0 (socket says 172.21.15.190:49556) 2024-08-19T10:39:57.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe78d3e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efe740a5940 0x7efe740720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:56590/0 (socket says 172.21.15.190:56590) 2024-08-19T10:39:57.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 1 -- 172.21.15.190:0/4179035308 learned_addr learned my addr 172.21.15.190:0/4179035308 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:57.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 10 monclient(hunting): get_auth_request con 0x7efe74139710 auth_method 0 2024-08-19T10:39:57.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:57.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:57.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:57.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe78d3e640 10 monclient(hunting): get_auth_request con 0x7efe740a5940 auth_method 0 2024-08-19T10:39:57.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe78d3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:57.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe78d3e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:57.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe78d3e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:57.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe73fff640 10 monclient(hunting): get_auth_request con 0x7efe740a5d20 auth_method 0 2024-08-19T10:39:57.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:57.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe73fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:57.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe73fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:57.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:57.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:57.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:57.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:57.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:57.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:57.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe78d3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:57.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe78d3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:57.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe78d3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:57.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 10 monclient(hunting): handle_auth_done global_id 4801 payload 274 2024-08-19T10:39:57.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 1 -- 172.21.15.190:0/4179035308 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efe740a5940 msgr2=0x7efe740720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:57.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 1 --2- 172.21.15.190:0/4179035308 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efe740a5940 0x7efe740720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 1 -- 172.21.15.190:0/4179035308 >> v2:172.21.15.89:3300/0 conn(0x7efe740a5d20 msgr2=0x7efe740a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:57.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 1 --2- 172.21.15.190:0/4179035308 >> v2:172.21.15.89:3300/0 conn(0x7efe740a5d20 0x7efe740a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 10 monclient: _finish_hunting 0 2024-08-19T10:39:57.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 1 monclient: found mon.noname-c 2024-08-19T10:39:57.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:57.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:39:57.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.641+0000 7efe7953f640 1 -- 172.21.15.190:0/4179035308 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7406f290 con 0x7efe74139710 2024-08-19T10:39:57.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7953f640 1 -- 172.21.15.190:0/4179035308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe74139710 msgr2=0x7efe74139bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:57.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7953f640 1 --2- 172.21.15.190:0/4179035308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe74139710 0x7efe74139bc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009e80 tx=0x7efe6c01c8d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=199e3e1d618768f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:57.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 1 -- 172.21.15.190:0/4179035308 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe6c0090d0 con 0x7efe74139710 2024-08-19T10:39:57.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:57.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:39:57.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 10 monclient: dump: 2024-08-19T10:39:57.659 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:57.660 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:57.661 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:57.661 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:57.661 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:57.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 10 monclient: _finish_auth 0 2024-08-19T10:39:57.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 10 monclient: _check_auth_tickets 2024-08-19T10:39:57.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:57.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 1 -- 172.21.15.190:0/4179035308 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c0a5ba0 con 0x7efe74139710 2024-08-19T10:39:57.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:57.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 1 -- 172.21.15.190:0/4179035308 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe6c0a5d90 con 0x7efe74139710 2024-08-19T10:39:57.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:57.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:39:57.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe737fe640 10 monclient: dump: 2024-08-19T10:39:57.662 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:57.662 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:57.662 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:57.662 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:57.662 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:57.662 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 5 monclient: authenticate success, global_id 4801 2024-08-19T10:39:57.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 4 set_mon_vals no callback set 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 10 monclient: shutdown 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 1 -- 172.21.15.190:0/4179035308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe74139710 msgr2=0x7efe74139bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 1 --2- 172.21.15.190:0/4179035308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe74139710 0x7efe74139bc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009e80 tx=0x7efe6c01c8d0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 1 -- 172.21.15.190:0/4179035308 shutdown_connections 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 1 --2- 172.21.15.190:0/4179035308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe74139710 0x7efe74139bc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 1 --2- 172.21.15.190:0/4179035308 >> v2:172.21.15.89:3300/0 conn(0x7efe740a5d20 0x7efe740a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 1 --2- 172.21.15.190:0/4179035308 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efe740a5940 0x7efe740720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 1 -- 172.21.15.190:0/4179035308 >> 172.21.15.190:0/4179035308 conn(0x7efe740aa530 msgr2=0x7efe740a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 1 -- 172.21.15.190:0/4179035308 shutdown_connections 2024-08-19T10:39:57.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 1 -- 172.21.15.190:0/4179035308 wait complete. 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 10 monclient: build_initial_monmap 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 10 monclient: monmap: 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 1 Processor -- start 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 1 -- start start 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.645+0000 7efe7afa2640 10 monclient: init 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient: _add_conns ranks=[5,3,0,6,1,4,8,2,7] 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe740a5940 0x7efe741225a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient(hunting): picked mon.noname-h con 0x7efe740a5940 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:39:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe740a5d20 0x7efe741d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient(hunting): picked mon.noname-g con 0x7efe740a5d20 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:39:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe74139710 0x7efe741d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient(hunting): picked mon.noname-a con 0x7efe74139710 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:39:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efe740a6640 con 0x7efe74139710 2024-08-19T10:39:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7efe7406e5f0 con 0x7efe740a5d20 2024-08-19T10:39:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7efe74123500 con 0x7efe740a5940 2024-08-19T10:39:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:57.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient(hunting): authenticate will time out at 1043.220328s 2024-08-19T10:39:57.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe740a5d20 0x7efe741d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:57.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe78d3e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe740a5940 0x7efe741225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:57.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe740a5d20 0x7efe741d16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:41434/0 (socket says 172.21.15.190:41434) 2024-08-19T10:39:57.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe78d3e640 10 monclient(hunting): get_auth_request con 0x7efe740a5940 auth_method 0 2024-08-19T10:39:57.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe78d3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:57.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 1 -- 172.21.15.190:0/3364672926 learned_addr learned my addr 172.21.15.190:0/3364672926 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:57.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe78d3e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:57.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe78d3e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:57.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 10 monclient(hunting): get_auth_request con 0x7efe740a5d20 auth_method 0 2024-08-19T10:39:57.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe78d3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe78d3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe78d3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7953f640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe74139710 0x7efe741d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7953f640 10 monclient(hunting): get_auth_request con 0x7efe74139710 auth_method 0 2024-08-19T10:39:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7953f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7953f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7953f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 10 monclient(hunting): handle_auth_done global_id 4701 payload 824 2024-08-19T10:39:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 1 -- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe740a5940 msgr2=0x7efe741225a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe740a5940 0x7efe741225a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 1 -- 172.21.15.190:0/3364672926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe74139710 msgr2=0x7efe741d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe74139710 0x7efe741d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 10 monclient: _finish_hunting 0 2024-08-19T10:39:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 1 monclient: found mon.noname-g 2024-08-19T10:39:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:39:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 1 -- 172.21.15.190:0/3364672926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe74122f30 con 0x7efe740a5d20 2024-08-19T10:39:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe78d3e640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe740a5940 0x7efe741225a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 1 -- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe740a5d20 msgr2=0x7efe741d16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe73fff640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe740a5d20 0x7efe741d16b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00e930 tx=0x7efe5c00edf0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 1 -- 172.21.15.190:0/3364672926 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe5c017070 con 0x7efe740a5d20 2024-08-19T10:39:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:39:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 10 monclient: dump: 2024-08-19T10:39:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:57.673 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 10 monclient: _finish_auth 0 2024-08-19T10:39:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 10 monclient: _check_auth_tickets 2024-08-19T10:39:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 1 -- 172.21.15.190:0/3364672926 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5c013470 con 0x7efe740a5d20 2024-08-19T10:39:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 5 monclient: authenticate success, global_id 4701 2024-08-19T10:39:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient: _renew_subs 2024-08-19T10:39:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 1 -- 172.21.15.190:0/3364672926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe74123da0 con 0x7efe740a5d20 2024-08-19T10:39:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 1 -- 172.21.15.190:0/3364672926 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe5c013a90 con 0x7efe740a5d20 2024-08-19T10:39:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:39:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe717fa640 10 monclient: dump: 2024-08-19T10:39:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe72ffd640 4 set_mon_vals no callback set 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient: _renew_subs 2024-08-19T10:39:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:57.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.649+0000 7efe7afa2640 1 -- 172.21.15.190:0/3364672926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efe741c80a0 con 0x7efe740a5d20 2024-08-19T10:39:57.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.653+0000 7efe7afa2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:57.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.653+0000 7efe7afa2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:57.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.653+0000 7efe7afa2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:57.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.653+0000 7efe7afa2640 1 -- 172.21.15.190:0/3364672926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe740a6100 con 0x7efe740a5d20 2024-08-19T10:39:57.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.653+0000 7efe717fa640 1 -- 172.21.15.190:0/3364672926 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efe5c01a070 con 0x7efe740a5d20 2024-08-19T10:39:57.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.653+0000 7efe717fa640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe5403be00 0x7efe5403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:57.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.653+0000 7efe717fa640 1 -- 172.21.15.190:0/3364672926 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(47..47 src has 1..47) ==== 4556+0+0 (secure 0 0 0) 0x7efe5c00f040 con 0x7efe740a5d20 2024-08-19T10:39:57.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.653+0000 7efe78d3e640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe5403be00 0x7efe5403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:57.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.653+0000 7efe78d3e640 10 monclient: get_auth_request con 0x7efe5403be00 auth_method 0 2024-08-19T10:39:57.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.653+0000 7efe78d3e640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe5403be00 0x7efe5403e2c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efe64009f50 tx=0x7efe64009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:57.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.657+0000 7efe717fa640 1 -- 172.21.15.190:0/3364672926 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7efe5c013620 con 0x7efe740a5d20 2024-08-19T10:39:57.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.657+0000 7efe717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:57.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.657+0000 7efe717fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:57.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.829+0000 7efe7afa2640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-08-19T10:39:57.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.829+0000 7efe7afa2640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-08-19T10:39:57.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.829+0000 7efe7afa2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:57.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:57.829+0000 7efe7afa2640 1 -- 172.21.15.190:0/3364672926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["all"]} v 0) -- 0x7efe7406ff20 con 0x7efe740a5d20 2024-08-19T10:39:58.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:58.405+0000 7efe717fa640 1 -- 172.21.15.190:0/3364672926 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["all"]}]=0 done removing class of osd(s): 0,1,2,3,4,5 v48) ==== 131+0+0 (secure 0 0 0) 0x7efe5c013cb0 con 0x7efe740a5d20 2024-08-19T10:39:58.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:58.405+0000 7efe717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-08-19T10:39:58.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:58.405+0000 7efe717fa640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0,1,2,3,4,5 2024-08-19T10:39:58.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:58.513+0000 7efe7afa2640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-08-19T10:39:58.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:58.513+0000 7efe7afa2640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-08-19T10:39:58.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:58.513+0000 7efe7afa2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:39:58.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:58.513+0000 7efe7afa2640 1 -- 172.21.15.190:0/3364672926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["all"]} v 0) -- 0x7efe7413faa0 con 0x7efe740a5d20 2024-08-19T10:39:59.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.429+0000 7efe717fa640 1 -- 172.21.15.190:0/3364672926 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["all"]}]=0 osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): v49) ==== 282+0+0 (secure 0 0 0) 0x7efe5c0232f0 con 0x7efe740a5d20 2024-08-19T10:39:59.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.429+0000 7efe717fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["all"]}] 2024-08-19T10:39:59.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.429+0000 7efe717fa640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): 2024-08-19T10:39:59.434 INFO:tasks.workunit.client.0.smithi190.stderr:osd.0 belongs to no class, osd.1 belongs to no class, osd.2 belongs to no class, osd.3 belongs to no class, osd.4 belongs to no class, osd.5 belongs to no class, done removing class of osd(s): 2024-08-19T10:39:59.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 -- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe5403be00 msgr2=0x7efe5403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:59.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe5403be00 0x7efe5403e2c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efe64009f50 tx=0x7efe64009190 comp rx=0 tx=0).stop 2024-08-19T10:39:59.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 10 monclient: shutdown 2024-08-19T10:39:59.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:59.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 -- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe740a5d20 msgr2=0x7efe741d16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:59.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe740a5d20 0x7efe741d16b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00e930 tx=0x7efe5c00edf0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 -- 172.21.15.190:0/3364672926 shutdown_connections 2024-08-19T10:39:59.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe5403be00 0x7efe5403e2c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe74139710 0x7efe741d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe740a5d20 0x7efe741d16b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 --2- 172.21.15.190:0/3364672926 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe740a5940 0x7efe741225a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 -- 172.21.15.190:0/3364672926 >> 172.21.15.190:0/3364672926 conn(0x7efe740aa530 msgr2=0x7efe740a4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:59.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 -- 172.21.15.190:0/3364672926 shutdown_connections 2024-08-19T10:39:59.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.433+0000 7efe7afa2640 1 -- 172.21.15.190:0/3364672926 wait complete. 2024-08-19T10:39:59.455 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush set-device-class ssd osd.0 2024-08-19T10:39:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.573+0000 7efeec23f640 10 monclient: get_monmap_and_config 2024-08-19T10:39:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.573+0000 7efeec23f640 10 monclient: build_initial_monmap 2024-08-19T10:39:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.573+0000 7efeec23f640 10 monclient: monmap: 2024-08-19T10:39:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:39:59.578215+0000 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:39:59.578215+0000 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:59.579 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:59.579 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:39:59.579 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:39:59.579 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.573+0000 7efeec23f640 1 Processor -- start 2024-08-19T10:39:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.573+0000 7efeec23f640 1 -- start start 2024-08-19T10:39:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.573+0000 7efeec23f640 10 monclient: init 2024-08-19T10:39:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 10 monclient: _add_conns ranks=[7,6,2,3,5,4,9,8,0,1,10] 2024-08-19T10:39:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efee4139980 0x7efee4139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 10 monclient(hunting): picked mon.noname-i con 0x7efee4139980 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:39:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee413a2a0 0x7efee40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 10 monclient(hunting): picked mon.noname-d con 0x7efee413a2a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee40a5c70 0x7efee40a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 10 monclient(hunting): picked mon.noname-b con 0x7efee40a5c70 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7efee409ca60 con 0x7efee40a5c70 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7efee40a3640 con 0x7efee413a2a0 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7efee40a8470 con 0x7efee4139980 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeec23f640 10 monclient(hunting): authenticate will time out at 1045.148378s 2024-08-19T10:39:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee9fdb640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efee4139980 msgr2=0x7efee4139d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee9fdb640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efee4139980 0x7efee4139d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee413a2a0 0x7efee40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeea7dc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee40a5c70 0x7efee40a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee413a2a0 0x7efee40a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:56600/0 (socket says 172.21.15.190:56600) 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeea7dc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee40a5c70 0x7efee40a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:39280/0 (socket says 172.21.15.190:39280) 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 1 -- 172.21.15.190:0/2602193329 learned_addr learned my addr 172.21.15.190:0/2602193329 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 10 monclient(hunting): get_auth_request con 0x7efee413a2a0 auth_method 0 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeea7dc640 10 monclient(hunting): get_auth_request con 0x7efee40a5c70 auth_method 0 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeea7dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeea7dc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeea7dc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeea7dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeea7dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeea7dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 10 monclient(hunting): handle_auth_done global_id 4668 payload 274 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 1 -- 172.21.15.190:0/2602193329 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efee4139980 msgr2=0x7efee4139d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 1 --2- 172.21.15.190:0/2602193329 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efee4139980 0x7efee4139d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 1 -- 172.21.15.190:0/2602193329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee40a5c70 msgr2=0x7efee40a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 1 --2- 172.21.15.190:0/2602193329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee40a5c70 0x7efee40a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 10 monclient: _finish_hunting 0 2024-08-19T10:39:59.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 1 monclient: found mon.noname-d 2024-08-19T10:39:59.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:59.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:39:59.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efee97da640 1 -- 172.21.15.190:0/2602193329 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee40a3850 con 0x7efee413a2a0 2024-08-19T10:39:59.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.577+0000 7efeea7dc640 1 --2- 172.21.15.190:0/2602193329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee40a5c70 0x7efee40a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:39:59.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee97da640 1 -- 172.21.15.190:0/2602193329 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee413a2a0 msgr2=0x7efee40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:59.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee97da640 1 --2- 172.21.15.190:0/2602193329 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee413a2a0 0x7efee40a5730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efed40098d0 tx=0x7efed401c170 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d62947e207bad97e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:59.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 1 -- 172.21.15.190:0/2602193329 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efed40b0070 con 0x7efee413a2a0 2024-08-19T10:39:59.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:59.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:39:59.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 10 monclient: dump: 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 10 monclient: _finish_auth 0 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 10 monclient: _check_auth_tickets 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 1 -- 172.21.15.190:0/2602193329 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed40aa810 con 0x7efee413a2a0 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 1 -- 172.21.15.190:0/2602193329 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efed40aae30 con 0x7efee413a2a0 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efee8fd9640 10 monclient: dump: 2024-08-19T10:39:59.588 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:59.589 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 5 monclient: authenticate success, global_id 4668 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 10 monclient: get_monmap_and_config success 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 4 set_mon_vals no callback set 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 10 monclient: shutdown 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 1 -- 172.21.15.190:0/2602193329 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee413a2a0 msgr2=0x7efee40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 1 --2- 172.21.15.190:0/2602193329 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee413a2a0 0x7efee40a5730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efed40098d0 tx=0x7efed401c170 comp rx=0 tx=0).stop 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 1 -- 172.21.15.190:0/2602193329 shutdown_connections 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 1 --2- 172.21.15.190:0/2602193329 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee413a2a0 0x7efee40a5730 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 1 --2- 172.21.15.190:0/2602193329 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee40a5c70 0x7efee40a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 1 --2- 172.21.15.190:0/2602193329 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efee4139980 0x7efee4139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 1 -- 172.21.15.190:0/2602193329 >> 172.21.15.190:0/2602193329 conn(0x7efee40abde0 msgr2=0x7efee40ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:39:59.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 1 -- 172.21.15.190:0/2602193329 shutdown_connections 2024-08-19T10:39:59.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.581+0000 7efeec23f640 1 -- 172.21.15.190:0/2602193329 wait complete. 2024-08-19T10:39:59.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient: build_initial_monmap 2024-08-19T10:39:59.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient: monmap: 2024-08-19T10:39:59.595 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:39:59.595 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:39:59.595 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:39:59.595 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:39:59.595 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:39:59.595 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:59.595 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:39:59.595 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:39:59.595 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:39:59.595 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:39:59.596 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:39:59.596 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:39:59.596 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:39:59.596 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:39:59.596 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:39:59.596 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:59.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 1 Processor -- start 2024-08-19T10:39:59.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 1 -- start start 2024-08-19T10:39:59.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient: init 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient: _reopen_session rank -1 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient: _add_conns ranks=[4,2,6,0,1,7,8,5,3] 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee40a5c70 0x7efee413fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient(hunting): picked mon.noname-c con 0x7efee40a5c70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4139980 0x7efee4140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient(hunting): picked mon.noname-b con 0x7efee4139980 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee41d17a0 0x7efee409f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient(hunting): picked mon.noname-d con 0x7efee41d17a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7efee4143270 con 0x7efee4139980 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:59.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7efee41430f0 con 0x7efee40a5c70 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7efee41433f0 con 0x7efee41d17a0 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient(hunting): _renew_subs 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeec23f640 10 monclient(hunting): authenticate will time out at 1045.157350s 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4139980 0x7efee4140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee9fdb640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee40a5c70 0x7efee413fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeea7dc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee41d17a0 0x7efee409f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4139980 0x7efee4140280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:39294/0 (socket says 172.21.15.190:39294) 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 1 -- 172.21.15.190:0/2709266365 learned_addr learned my addr 172.21.15.190:0/2709266365 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 10 monclient(hunting): get_auth_request con 0x7efee4139980 auth_method 0 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:59.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee9fdb640 10 monclient(hunting): get_auth_request con 0x7efee40a5c70 auth_method 0 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee9fdb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee9fdb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee9fdb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeea7dc640 10 monclient(hunting): get_auth_request con 0x7efee41d17a0 auth_method 0 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeea7dc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeea7dc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeea7dc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee9fdb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:59.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee9fdb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee9fdb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeea7dc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeea7dc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efeea7dc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 10 monclient(hunting): handle_auth_done global_id 4700 payload 824 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 1 -- 172.21.15.190:0/2709266365 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee41d17a0 msgr2=0x7efee409f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee41d17a0 0x7efee409f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 1 -- 172.21.15.190:0/2709266365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee40a5c70 msgr2=0x7efee413fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee40a5c70 0x7efee413fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 10 monclient: _finish_hunting 0 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 1 monclient: found mon.noname-b 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:39:59.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.585+0000 7efee97da640 1 -- 172.21.15.190:0/2709266365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee41357d0 con 0x7efee4139980 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efee97da640 1 -- 172.21.15.190:0/2709266365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4139980 msgr2=0x7efee4140280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efee97da640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4139980 0x7efee4140280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efed40097c0 tx=0x7efed4018f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 1 -- 172.21.15.190:0/2709266365 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efed40b9020 con 0x7efee4139980 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 10 monclient: dump: 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:59.601 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 10 monclient: _finish_auth 0 2024-08-19T10:39:59.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 1 -- 172.21.15.190:0/2709266365 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed40044b0 con 0x7efee4139980 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeec23f640 5 monclient: authenticate success, global_id 4700 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeec23f640 10 monclient: _renew_subs 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeec23f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeec23f640 1 -- 172.21.15.190:0/2709266365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee40a5970 con 0x7efee4139980 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 1 -- 172.21.15.190:0/2709266365 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efed40b0040 con 0x7efee4139980 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efedbfff640 4 set_mon_vals no callback set 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 10 monclient: dump: 2024-08-19T10:39:59.603 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:39:59.604 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeec23f640 10 monclient: _renew_subs 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeec23f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeec23f640 1 -- 172.21.15.190:0/2709266365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efee413b520 con 0x7efee4139980 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeec23f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeec23f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeec23f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeec23f640 1 -- 172.21.15.190:0/2709266365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee413af60 con 0x7efee4139980 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 1 -- 172.21.15.190:0/2709266365 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efed40bc020 con 0x7efee4139980 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efec803bdf0 0x7efec803e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:39:59.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efeda7fc640 1 -- 172.21.15.190:0/2709266365 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(49..49 src has 1..49) ==== 4252+0+0 (secure 0 0 0) 0x7efed40e8900 con 0x7efee4139980 2024-08-19T10:39:59.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efee9fdb640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efec803bdf0 0x7efec803e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:39:59.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.589+0000 7efee9fdb640 10 monclient: get_auth_request con 0x7efec803bdf0 auth_method 0 2024-08-19T10:39:59.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.593+0000 7efee9fdb640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efec803bdf0 0x7efec803e2b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efee0003990 tx=0x7efee0009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:39:59.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.601+0000 7efeda7fc640 1 -- 172.21.15.190:0/2709266365 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7efee413af60 con 0x7efee4139980 2024-08-19T10:39:59.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.601+0000 7efeda7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:39:59.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.601+0000 7efeda7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:39:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.781+0000 7efeec23f640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-08-19T10:39:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.781+0000 7efeec23f640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-08-19T10:39:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.781+0000 7efeec23f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:39:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:39:59.781+0000 7efeec23f640 1 -- 172.21.15.190:0/2709266365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]} v 0) -- 0x7efee413b320 con 0x7efee4139980 2024-08-19T10:40:00.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:00.432+0000 7efeda7fc640 1 -- 172.21.15.190:0/2709266365 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'ssd' v50) ==== 135+0+0 (secure 0 0 0) 0x7efed40c3030 con 0x7efee4139980 2024-08-19T10:40:00.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:00.432+0000 7efeda7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-08-19T10:40:00.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:00.432+0000 7efeda7fc640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'ssd' 2024-08-19T10:40:00.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:00.536+0000 7efeec23f640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-08-19T10:40:00.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:00.540+0000 7efeec23f640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-08-19T10:40:00.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:00.540+0000 7efeec23f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:00.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:00.540+0000 7efeec23f640 1 -- 172.21.15.190:0/2709266365 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]} v 0) -- 0x7efee4139d60 con 0x7efee4139980 2024-08-19T10:40:01.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.460+0000 7efeda7fc640 1 -- 172.21.15.190:0/2709266365 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}]=0 osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' v51) ==== 237+0+0 (secure 0 0 0) 0x7efed4016170 con 0x7efee4139980 2024-08-19T10:40:01.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.460+0000 7efeda7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["osd.0"]}] 2024-08-19T10:40:01.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.460+0000 7efeda7fc640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-08-19T10:40:01.465 INFO:tasks.workunit.client.0.smithi190.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-08-19T10:40:01.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 -- 172.21.15.190:0/2709266365 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efec803bdf0 msgr2=0x7efec803e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:01.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efec803bdf0 0x7efec803e2b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efee0003990 tx=0x7efee0009110 comp rx=0 tx=0).stop 2024-08-19T10:40:01.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 10 monclient: shutdown 2024-08-19T10:40:01.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:01.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 -- 172.21.15.190:0/2709266365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4139980 msgr2=0x7efee4140280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:01.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4139980 0x7efee4140280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efed40097c0 tx=0x7efed4018f10 comp rx=0 tx=0).stop 2024-08-19T10:40:01.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 -- 172.21.15.190:0/2709266365 shutdown_connections 2024-08-19T10:40:01.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efec803bdf0 0x7efec803e2b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:01.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efee41d17a0 0x7efee409f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:01.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efee4139980 0x7efee4140280 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efed40097c0 tx=0x7efed4018f10 comp rx=0 tx=0).stop 2024-08-19T10:40:01.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 --2- 172.21.15.190:0/2709266365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efee40a5c70 0x7efee413fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:01.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 -- 172.21.15.190:0/2709266365 >> 172.21.15.190:0/2709266365 conn(0x7efee40abde0 msgr2=0x7efee41350a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:01.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 -- 172.21.15.190:0/2709266365 shutdown_connections 2024-08-19T10:40:01.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.464+0000 7efeec23f640 1 -- 172.21.15.190:0/2709266365 wait complete. 2024-08-19T10:40:01.486 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush set-device-class hdd osd.1 2024-08-19T10:40:01.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.600+0000 7fa540fb2640 10 monclient: get_monmap_and_config 2024-08-19T10:40:01.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.600+0000 7fa540fb2640 10 monclient: build_initial_monmap 2024-08-19T10:40:01.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.600+0000 7fa540fb2640 10 monclient: monmap: 2024-08-19T10:40:01.606 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:01.606 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:01.606 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:01.606740+0000 2024-08-19T10:40:01.606 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:01.606740+0000 2024-08-19T10:40:01.606 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:01.606 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:01.606 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:01.607 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:01.607 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:01.607 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:01.607 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:01.607 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:01.607 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:01.607 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:01.607 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:01.607 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:01.607 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:01.607 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:01.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.600+0000 7fa540fb2640 1 Processor -- start 2024-08-19T10:40:01.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 1 -- start start 2024-08-19T10:40:01.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 10 monclient: init 2024-08-19T10:40:01.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:01.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 10 monclient: _add_conns ranks=[5,3,2,8,1,10,7,0,4,6,9] 2024-08-19T10:40:01.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:01.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 10 monclient(hunting): picked mon.noname-h con 0x7fa53c139980 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:01.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c13a2a0 0x7fa53c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:01.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 10 monclient(hunting): picked mon.noname-g con 0x7fa53c13a2a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:01.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa53c0a5c70 0x7fa53c0a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:01.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 10 monclient(hunting): picked mon.noname-b con 0x7fa53c0a5c70 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:01.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa53c09ca60 con 0x7fa53c0a5c70 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa53c0a3640 con 0x7fa53c13a2a0 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa53c0a8470 con 0x7fa53c139980 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa540fb2640 10 monclient(hunting): authenticate will time out at 1047.176802s 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c13a2a0 0x7fa53c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48052/0 (socket says 172.21.15.190:48052) 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c13a2a0 0x7fa53c0a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:41442/0 (socket says 172.21.15.190:41442) 2024-08-19T10:40:01.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53a59c640 1 -- 172.21.15.190:0/4288641305 learned_addr learned my addr 172.21.15.190:0/4288641305 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53a59c640 10 monclient(hunting): get_auth_request con 0x7fa53c139980 auth_method 0 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 10 monclient(hunting): get_auth_request con 0x7fa53c13a2a0 auth_method 0 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:01.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53ad9d640 1 --2- 172.21.15.190:0/4288641305 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa53c0a5c70 0x7fa53c0a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53ad9d640 10 monclient(hunting): get_auth_request con 0x7fa53c0a5c70 auth_method 0 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 10 monclient(hunting): handle_auth_done global_id 4710 payload 274 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 1 -- 172.21.15.190:0/4288641305 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 msgr2=0x7fa53c139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 1 --2- 172.21.15.190:0/4288641305 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 1 -- 172.21.15.190:0/4288641305 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa53c0a5c70 msgr2=0x7fa53c0a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 1 --2- 172.21.15.190:0/4288641305 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa53c0a5c70 0x7fa53c0a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:40:01.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 1 monclient: found mon.noname-g 2024-08-19T10:40:01.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:01.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:01.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa531d9b640 1 -- 172.21.15.190:0/4288641305 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c0a3850 con 0x7fa53c13a2a0 2024-08-19T10:40:01.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.604+0000 7fa53a59c640 1 --2- 172.21.15.190:0/4288641305 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:01.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa531d9b640 1 -- 172.21.15.190:0/4288641305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c13a2a0 msgr2=0x7fa53c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:01.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa531d9b640 1 --2- 172.21.15.190:0/4288641305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c13a2a0 0x7fa53c0a5730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa524009820 tx=0x7fa52401c090 comp rx=0 tx=0).ready entity=mon.3 client_cookie=2a6023ac720c4421 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:01.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 1 -- 172.21.15.190:0/4288641305 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5240b0070 con 0x7fa53c13a2a0 2024-08-19T10:40:01.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:01.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:01.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 10 monclient: dump: 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:01.620 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 10 monclient: _finish_auth 0 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 5 monclient: authenticate success, global_id 4710 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 1 -- 172.21.15.190:0/4288641305 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5240aa810 con 0x7fa53c13a2a0 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:01.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 4 set_mon_vals no callback set 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 1 -- 172.21.15.190:0/4288641305 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5240aae30 con 0x7fa53c13a2a0 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa539d9b640 10 monclient: dump: 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient: shutdown 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 -- 172.21.15.190:0/4288641305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c13a2a0 msgr2=0x7fa53c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 --2- 172.21.15.190:0/4288641305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c13a2a0 0x7fa53c0a5730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa524009820 tx=0x7fa52401c090 comp rx=0 tx=0).stop 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 -- 172.21.15.190:0/4288641305 shutdown_connections 2024-08-19T10:40:01.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 --2- 172.21.15.190:0/4288641305 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa53c0a5c70 0x7fa53c0a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:01.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 --2- 172.21.15.190:0/4288641305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa53c13a2a0 0x7fa53c0a5730 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:01.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 --2- 172.21.15.190:0/4288641305 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:01.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 -- 172.21.15.190:0/4288641305 >> 172.21.15.190:0/4288641305 conn(0x7fa53c0abde0 msgr2=0x7fa53c0ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:01.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 -- 172.21.15.190:0/4288641305 shutdown_connections 2024-08-19T10:40:01.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 -- 172.21.15.190:0/4288641305 wait complete. 2024-08-19T10:40:01.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient: build_initial_monmap 2024-08-19T10:40:01.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient: monmap: 2024-08-19T10:40:01.625 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:01.625 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:01.625 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:01.625 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:01.625 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:01.625 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:01.625 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:01.625 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:01.625 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 Processor -- start 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 -- start start 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient: init 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient: _add_conns ranks=[1,5,4,6,3,2,8,7,0] 2024-08-19T10:40:01.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c0a5c70 0x7fa53c0a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:01.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient(hunting): picked mon.noname-f con 0x7fa53c0a5c70 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:01.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c0a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:01.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient(hunting): picked mon.noname-h con 0x7fa53c139980 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:01.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa53c0a0df0 0x7fa53c1d1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:01.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient(hunting): picked mon.noname-c con 0x7fa53c0a0df0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:01.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:01.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c143270 con 0x7fa53c0a5c70 2024-08-19T10:40:01.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:01.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa53c1430f0 con 0x7fa53c0a0df0 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa53c1433f0 con 0x7fa53c139980 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa540fb2640 10 monclient(hunting): authenticate will time out at 1047.183872s 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa53a59c640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c0a5c70 msgr2=0x7fa53c0a0370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.608+0000 7fa53a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c0a5c70 0x7fa53c0a0370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c0a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c0a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48054/0 (socket says 172.21.15.190:48054) 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 1 -- 172.21.15.190:0/4195683998 learned_addr learned my addr 172.21.15.190:0/4195683998 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 10 monclient(hunting): get_auth_request con 0x7fa53c139980 auth_method 0 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:01.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 10 monclient(hunting): handle_auth_done global_id 4748 payload 824 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 1 -- 172.21.15.190:0/4195683998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa53c0a0df0 msgr2=0x7fa53c1d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa53c0a0df0 0x7fa53c1d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 1 -- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c0a5c70 msgr2=0x7fa53c0a0370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c0a5c70 0x7fa53c0a0370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 1 monclient: found mon.noname-h 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:01.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 1 -- 172.21.15.190:0/4195683998 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c0776f0 con 0x7fa53c139980 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 1 -- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 msgr2=0x7fa53c0a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa531d9b640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c0a08b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa5240ae040 tx=0x7fa524018e30 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 1 -- 172.21.15.190:0/4195683998 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5240b0070 con 0x7fa53c139980 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 10 monclient: dump: 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:01.630 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:01.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa540fb2640 5 monclient: authenticate success, global_id 4748 2024-08-19T10:40:01.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 1 -- 172.21.15.190:0/4195683998 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5240b1040 con 0x7fa53c139980 2024-08-19T10:40:01.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa540fb2640 10 monclient: _renew_subs 2024-08-19T10:40:01.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa540fb2640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:01.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa540fb2640 1 -- 172.21.15.190:0/4195683998 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53c13bee0 con 0x7fa53c139980 2024-08-19T10:40:01.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:01.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 1 -- 172.21.15.190:0/4195683998 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa524004460 con 0x7fa53c139980 2024-08-19T10:40:01.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:01.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:01.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa53959a640 4 set_mon_vals no callback set 2024-08-19T10:40:01.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa5337fe640 10 monclient: dump: 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:01.633 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa540fb2640 10 monclient: _renew_subs 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa540fb2640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa540fb2640 1 -- 172.21.15.190:0/4195683998 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53c13aca0 con 0x7fa53c139980 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa540fb2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa540fb2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa540fb2640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.612+0000 7fa540fb2640 1 -- 172.21.15.190:0/4195683998 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa53c135660 con 0x7fa53c139980 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.616+0000 7fa5337fe640 1 -- 172.21.15.190:0/4195683998 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa5240bc030 con 0x7fa53c139980 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.616+0000 7fa5337fe640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa51c03be00 0x7fa51c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:01.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.616+0000 7fa5337fe640 1 -- 172.21.15.190:0/4195683998 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(51..51 src has 1..51) ==== 4476+0+0 (secure 0 0 0) 0x7fa5240ba020 con 0x7fa53c139980 2024-08-19T10:40:01.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.616+0000 7fa53a59c640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa51c03be00 0x7fa51c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:01.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.616+0000 7fa53a59c640 10 monclient: get_auth_request con 0x7fa51c03be00 auth_method 0 2024-08-19T10:40:01.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.616+0000 7fa53a59c640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa51c03be00 0x7fa51c03e2c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009630 tx=0x7fa52c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:01.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.620+0000 7fa5337fe640 1 -- 172.21.15.190:0/4195683998 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fa5240c3060 con 0x7fa53c139980 2024-08-19T10:40:01.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.620+0000 7fa5337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:01.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.620+0000 7fa5337fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:01.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.792+0000 7fa540fb2640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-08-19T10:40:01.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.792+0000 7fa540fb2640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-08-19T10:40:01.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.792+0000 7fa540fb2640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:01.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:01.792+0000 7fa540fb2640 1 -- 172.21.15.190:0/4195683998 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]} v 0) -- 0x7fa53c139d60 con 0x7fa53c139980 2024-08-19T10:40:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:02.460+0000 7fa5337fe640 1 -- 172.21.15.190:0/4195683998 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}]=0 set osd(s) 1 to class 'hdd' v52) ==== 135+0+0 (secure 0 0 0) 0x7fa53c0776f0 con 0x7fa53c139980 2024-08-19T10:40:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:02.460+0000 7fa5337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-08-19T10:40:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:02.460+0000 7fa5337fe640 10 monclient: _finish_command 2 = system:0 set osd(s) 1 to class 'hdd' 2024-08-19T10:40:02.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:02.568+0000 7fa540fb2640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-08-19T10:40:02.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:02.568+0000 7fa540fb2640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-08-19T10:40:02.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:02.568+0000 7fa540fb2640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:02.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:02.568+0000 7fa540fb2640 1 -- 172.21.15.190:0/4195683998 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]} v 0) -- 0x7fa53c0a1cd0 con 0x7fa53c139980 2024-08-19T10:40:03.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.472+0000 7fa5337fe640 1 -- 172.21.15.190:0/4195683998 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}]=0 osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' v53) ==== 237+0+0 (secure 0 0 0) 0x7fa524016090 con 0x7fa53c139980 2024-08-19T10:40:03.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.472+0000 7fa5337fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["osd.1"]}] 2024-08-19T10:40:03.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.472+0000 7fa5337fe640 10 monclient: _finish_command 3 = system:0 osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' 2024-08-19T10:40:03.476 INFO:tasks.workunit.client.0.smithi190.stderr:osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. set osd(s) to class 'hdd' 2024-08-19T10:40:03.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 -- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa51c03be00 msgr2=0x7fa51c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa51c03be00 0x7fa51c03e2c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009630 tx=0x7fa52c009240 comp rx=0 tx=0).stop 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 10 monclient: shutdown 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 -- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 msgr2=0x7fa53c0a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c0a08b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa5240ae040 tx=0x7fa524018e30 comp rx=0 tx=0).stop 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 -- 172.21.15.190:0/4195683998 shutdown_connections 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa51c03be00 0x7fa51c03e2c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53c139980 0x7fa53c0a08b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa53c0a0df0 0x7fa53c1d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 --2- 172.21.15.190:0/4195683998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa53c0a5c70 0x7fa53c0a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 -- 172.21.15.190:0/4195683998 >> 172.21.15.190:0/4195683998 conn(0x7fa53c0abde0 msgr2=0x7fa53c138cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:03.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 -- 172.21.15.190:0/4195683998 shutdown_connections 2024-08-19T10:40:03.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.476+0000 7fa540fb2640 1 -- 172.21.15.190:0/4195683998 wait complete. 2024-08-19T10:40:03.504 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule create-replicated foo-ssd default host ssd 2024-08-19T10:40:03.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.624+0000 7f3fa29d2640 10 monclient: get_monmap_and_config 2024-08-19T10:40:03.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.624+0000 7f3fa29d2640 10 monclient: build_initial_monmap 2024-08-19T10:40:03.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.624+0000 7f3fa29d2640 10 monclient: monmap: 2024-08-19T10:40:03.631 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:03.631 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:03.631 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:03.631066+0000 2024-08-19T10:40:03.631 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:03.631066+0000 2024-08-19T10:40:03.631 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:03.631 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:03.631 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:03.631 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:03.631 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:03.631 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:03.632 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:03.632 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:03.632 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:03.632 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:03.632 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:03.632 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:03.632 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:03.632 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:03.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 1 Processor -- start 2024-08-19T10:40:03.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 1 -- start start 2024-08-19T10:40:03.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 10 monclient: init 2024-08-19T10:40:03.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:03.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 10 monclient: _add_conns ranks=[4,6,7,5,2,0,8,10,3,1,9] 2024-08-19T10:40:03.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1399f0 0x7f3f9c0a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:03.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 10 monclient(hunting): picked mon.noname-c con 0x7f3f9c1399f0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:03.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c139dd0 0x7f3f9c0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:03.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 10 monclient(hunting): picked mon.noname-d con 0x7f3f9c139dd0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3f9c13a1b0 0x7f3f9c0a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 10 monclient(hunting): picked mon.noname-i con 0x7f3f9c13a1b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3f9c0a4eb0 con 0x7f3f9c1399f0 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f3f9c0a5b80 con 0x7f3f9c139dd0 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f3f9c0a8340 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa29d2640 10 monclient(hunting): authenticate will time out at 1049.200936s 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa0f6f640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3f9c13a1b0 msgr2=0x7f3f9c0a6e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3fa0f6f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3f9c13a1b0 0x7f3f9c0a6e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:03.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c139dd0 0x7f3f9c0a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c139dd0 0x7f3f9c0a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:56624/0 (socket says 172.21.15.190:56624) 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 1 -- 172.21.15.190:0/123946804 learned_addr learned my addr 172.21.15.190:0/123946804 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 10 monclient(hunting): get_auth_request con 0x7f3f9c139dd0 auth_method 0 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f9bfff640 1 --2- 172.21.15.190:0/123946804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1399f0 0x7f3f9c0a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f9bfff640 10 monclient(hunting): get_auth_request con 0x7f3f9c1399f0 auth_method 0 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f9bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:03.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:03.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:03.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:03.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:03.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:03.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:03.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 10 monclient(hunting): handle_auth_done global_id 4686 payload 274 2024-08-19T10:40:03.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 1 -- 172.21.15.190:0/123946804 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3f9c13a1b0 msgr2=0x7f3f9c0a6e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:03.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 1 --2- 172.21.15.190:0/123946804 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3f9c13a1b0 0x7f3f9c0a6e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:03.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 1 -- 172.21.15.190:0/123946804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1399f0 msgr2=0x7f3f9c0a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:03.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 1 --2- 172.21.15.190:0/123946804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1399f0 0x7f3f9c0a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:03.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:03.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 1 monclient: found mon.noname-d 2024-08-19T10:40:03.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:03.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:03.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.628+0000 7f3f93fff640 1 -- 172.21.15.190:0/123946804 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c0a4580 con 0x7f3f9c139dd0 2024-08-19T10:40:03.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f93fff640 1 -- 172.21.15.190:0/123946804 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c139dd0 msgr2=0x7f3f9c0a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:03.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f93fff640 1 --2- 172.21.15.190:0/123946804 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c139dd0 0x7f3f9c0a68e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3f840098a0 tx=0x7f3f8401c460 comp rx=0 tx=0).ready entity=mon.6 client_cookie=4a16a6088183aa8d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:03.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 1 -- 172.21.15.190:0/123946804 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f840b0070 con 0x7f3f9c139dd0 2024-08-19T10:40:03.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:03.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:03.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 10 monclient: dump: 2024-08-19T10:40:03.647 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:03.647 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:03.647 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:03.647 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:03.647 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:03.647 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:03.647 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:03.647 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:03.647 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:03.647 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:03.647 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 1 -- 172.21.15.190:0/123946804 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f840aa810 con 0x7f3f9c139dd0 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 1 -- 172.21.15.190:0/123946804 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f840a9300 con 0x7f3f9c139dd0 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3fa29d2640 5 monclient: authenticate success, global_id 4686 2024-08-19T10:40:03.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3fa29d2640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:03.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3fa29d2640 4 set_mon_vals no callback set 2024-08-19T10:40:03.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:03.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:03.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.632+0000 7f3f9b7fe640 10 monclient: dump: 2024-08-19T10:40:03.649 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:03.649 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:03.649 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:03.649 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:03.649 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:03.649 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:03.649 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient: shutdown 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 -- 172.21.15.190:0/123946804 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c139dd0 msgr2=0x7f3f9c0a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:03.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 --2- 172.21.15.190:0/123946804 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c139dd0 0x7f3f9c0a68e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3f840098a0 tx=0x7f3f8401c460 comp rx=0 tx=0).stop 2024-08-19T10:40:03.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 -- 172.21.15.190:0/123946804 shutdown_connections 2024-08-19T10:40:03.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 --2- 172.21.15.190:0/123946804 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3f9c13a1b0 0x7f3f9c0a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:03.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 --2- 172.21.15.190:0/123946804 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c139dd0 0x7f3f9c0a68e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:03.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 --2- 172.21.15.190:0/123946804 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f9c1399f0 0x7f3f9c0a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:03.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 -- 172.21.15.190:0/123946804 >> 172.21.15.190:0/123946804 conn(0x7f3f9c074c10 msgr2=0x7f3f9c0abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:03.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 -- 172.21.15.190:0/123946804 shutdown_connections 2024-08-19T10:40:03.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 -- 172.21.15.190:0/123946804 wait complete. 2024-08-19T10:40:03.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient: build_initial_monmap 2024-08-19T10:40:03.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient: monmap: 2024-08-19T10:40:03.652 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:03.652 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:03.652 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:03.652 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:03.652 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:03.652 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:03.652 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:03.652 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:03.652 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 Processor -- start 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 -- start start 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient: init 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient: _add_conns ranks=[6,8,3,0,7,2,4,5,1] 2024-08-19T10:40:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c1399f0 0x7f3f9c1d1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient(hunting): picked mon.noname-d con 0x7f3f9c1399f0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f9c139dd0 0x7f3f9c13fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient(hunting): picked mon.noname-e con 0x7f3f9c139dd0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f9c13a1b0 0x7f3f9c140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient(hunting): picked mon.noname-g con 0x7f3f9c13a1b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9c143290 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f3f9c143110 con 0x7f3f9c1399f0 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3f9c143410 con 0x7f3f9c139dd0 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:03.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa29d2640 10 monclient(hunting): authenticate will time out at 1049.209865s 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f9c13a1b0 0x7f3f9c140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f9c13a1b0 0x7f3f9c140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:41454/0 (socket says 172.21.15.190:41454) 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 1 -- 172.21.15.190:0/414443134 learned_addr learned my addr 172.21.15.190:0/414443134 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 10 monclient(hunting): get_auth_request con 0x7f3f9c13a1b0 auth_method 0 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f9bfff640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c1399f0 0x7f3f9c1d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f9bfff640 10 monclient(hunting): get_auth_request con 0x7f3f9c1399f0 auth_method 0 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f9bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:03.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f93fff640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f9c139dd0 0x7f3f9c13fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f93fff640 10 monclient(hunting): get_auth_request con 0x7f3f9c139dd0 auth_method 0 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f93fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f93fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3f93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 10 monclient(hunting): handle_auth_done global_id 4719 payload 824 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 1 -- 172.21.15.190:0/414443134 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f9c139dd0 msgr2=0x7f3f9c13fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:03.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f9c139dd0 0x7f3f9c13fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:03.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 1 -- 172.21.15.190:0/414443134 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c1399f0 msgr2=0x7f3f9c1d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:03.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c1399f0 0x7f3f9c1d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:03.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 10 monclient: _finish_hunting 0 2024-08-19T10:40:03.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 1 monclient: found mon.noname-g 2024-08-19T10:40:03.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:03.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:03.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.636+0000 7f3fa0f6f640 1 -- 172.21.15.190:0/414443134 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c0ab710 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa0f6f640 1 -- 172.21.15.190:0/414443134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f9c13a1b0 msgr2=0x7f3f9c140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:03.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa0f6f640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f9c13a1b0 0x7f3f9c140150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400be60 tx=0x7f3f9400bf50 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:03.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 1 -- 172.21.15.190:0/414443134 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f94015850 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 10 monclient: dump: 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:03.658 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 10 monclient: _finish_auth 0 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 10 monclient: _check_auth_tickets 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 1 -- 172.21.15.190:0/414443134 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f94010030 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa29d2640 5 monclient: authenticate success, global_id 4719 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa29d2640 10 monclient: _renew_subs 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa29d2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:03.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa29d2640 1 -- 172.21.15.190:0/414443134 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9c13b6b0 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 1 -- 172.21.15.190:0/414443134 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f940060d0 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f9affd640 4 set_mon_vals no callback set 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 10 monclient: dump: 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:03.660 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa29d2640 10 monclient: _renew_subs 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa29d2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa29d2640 1 -- 172.21.15.190:0/414443134 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9c0a0d00 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa29d2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:03.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa29d2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:03.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa29d2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:03.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3fa29d2640 1 -- 172.21.15.190:0/414443134 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f9c0a5640 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 1 -- 172.21.15.190:0/414443134 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3f94003d10 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f7c03be00 0x7f3f7c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:03.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f997fa640 1 -- 172.21.15.190:0/414443134 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 4708+0+0 (secure 0 0 0) 0x7f3f9404ed40 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f9bfff640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f7c03be00 0x7f3f7c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:03.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.640+0000 7f3f9bfff640 10 monclient: get_auth_request con 0x7f3f7c03be00 auth_method 0 2024-08-19T10:40:03.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.644+0000 7f3f9bfff640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f7c03be00 0x7f3f7c03e2c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009b60 tx=0x7f3f8c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:03.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.644+0000 7f3f997fa640 1 -- 172.21.15.190:0/414443134 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3f9c0ab710 con 0x7f3f9c13a1b0 2024-08-19T10:40:03.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.644+0000 7f3f997fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:03.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.644+0000 7f3f997fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:03.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.812+0000 7f3fa29d2640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-08-19T10:40:03.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.812+0000 7f3fa29d2640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-08-19T10:40:03.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.812+0000 7f3fa29d2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:03.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:03.812+0000 7f3fa29d2640 1 -- 172.21.15.190:0/414443134 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"} v 0) -- 0x7f3f9c13ae10 con 0x7f3f9c13a1b0 2024-08-19T10:40:04.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.500+0000 7f3f997fa640 1 -- 172.21.15.190:0/414443134 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}]=0 v54) ==== 150+0+0 (secure 0 0 0) 0x7f3f94013cf0 con 0x7f3f9c13a1b0 2024-08-19T10:40:04.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.500+0000 7f3f997fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-08-19T10:40:04.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.500+0000 7f3f997fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:04.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.600+0000 7f3fa29d2640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-08-19T10:40:04.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.600+0000 7f3fa29d2640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-08-19T10:40:04.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.600+0000 7f3fa29d2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:04.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.600+0000 7f3fa29d2640 1 -- 172.21.15.190:0/414443134 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"} v 0) -- 0x7f3f9c0a3500 con 0x7f3f9c13a1b0 2024-08-19T10:40:04.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3f997fa640 1 -- 172.21.15.190:0/414443134 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}]=0 rule foo-ssd already exists v54) ==== 177+0+0 (secure 0 0 0) 0x7f3f94015a10 con 0x7f3f9c13a1b0 2024-08-19T10:40:04.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3f997fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-ssd", "root": "default", "type": "host", "class": "ssd"}] 2024-08-19T10:40:04.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3f997fa640 10 monclient: _finish_command 3 = system:0 rule foo-ssd already exists 2024-08-19T10:40:04.607 INFO:tasks.workunit.client.0.smithi190.stderr:rule foo-ssd already exists 2024-08-19T10:40:04.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 -- 172.21.15.190:0/414443134 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f7c03be00 msgr2=0x7f3f7c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:04.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f7c03be00 0x7f3f7c03e2c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009b60 tx=0x7f3f8c009190 comp rx=0 tx=0).stop 2024-08-19T10:40:04.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 10 monclient: shutdown 2024-08-19T10:40:04.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:04.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 -- 172.21.15.190:0/414443134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f9c13a1b0 msgr2=0x7f3f9c140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:04.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f9c13a1b0 0x7f3f9c140150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400be60 tx=0x7f3f9400bf50 comp rx=0 tx=0).stop 2024-08-19T10:40:04.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 -- 172.21.15.190:0/414443134 shutdown_connections 2024-08-19T10:40:04.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f9c13a1b0 0x7f3f9c140150 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:04.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f7c03be00 0x7f3f7c03e2c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:04.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f9c139dd0 0x7f3f9c13fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:04.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 --2- 172.21.15.190:0/414443134 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3f9c1399f0 0x7f3f9c1d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:04.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 -- 172.21.15.190:0/414443134 >> 172.21.15.190:0/414443134 conn(0x7f3f9c074c10 msgr2=0x7f3f9c135070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:04.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 -- 172.21.15.190:0/414443134 shutdown_connections 2024-08-19T10:40:04.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.604+0000 7f3fa29d2640 1 -- 172.21.15.190:0/414443134 wait complete. 2024-08-19T10:40:04.624 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule create-replicated foo-hdd default host hdd 2024-08-19T10:40:04.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.740+0000 7fd036739640 10 monclient: get_monmap_and_config 2024-08-19T10:40:04.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.740+0000 7fd036739640 10 monclient: build_initial_monmap 2024-08-19T10:40:04.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.740+0000 7fd036739640 10 monclient: monmap: 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:04.747099+0000 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:04.747099+0000 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:04.747 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:04.748 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:04.748 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:04.748 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:04.748 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:04.748 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:04.748 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:04.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 1 Processor -- start 2024-08-19T10:40:04.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 1 -- start start 2024-08-19T10:40:04.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 10 monclient: init 2024-08-19T10:40:04.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:04.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 10 monclient: _add_conns ranks=[0,7,1,2,6,4,10,3,9,8,5] 2024-08-19T10:40:04.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd0300a4720 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:04.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0300a5210 0x7fd03013b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:04.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 10 monclient(hunting): picked mon.noname-i con 0x7fd0300a5210 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:04.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd03013be10 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:04.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0300a0470 con 0x7fd0300a4720 2024-08-19T10:40:04.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0300a08b0 con 0x7fd03013be10 2024-08-19T10:40:04.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:04.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd0300a0b30 con 0x7fd0300a5210 2024-08-19T10:40:04.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:04.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd036739640 10 monclient(hunting): authenticate will time out at 1050.317128s 2024-08-19T10:40:04.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd02f7fe640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0300a5210 msgr2=0x7fd03013b8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:04.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd02f7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0300a5210 0x7fd03013b8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:04.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd034cd6640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fd03013be10 legacy=0x7fd03013e200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:40:04.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd02ffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd0300a4720 0x7fd0300a4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:56146/0 (socket says 172.21.15.190:56146) 2024-08-19T10:40:04.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd02ffff640 1 -- 172.21.15.190:0/11925742 learned_addr learned my addr 172.21.15.190:0/11925742 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:40:04.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.744+0000 7fd02ffff640 1 -- 172.21.15.190:0/11925742 >> v1:172.21.15.89:6789/0 conn(0x7fd0300a4720 legacy=0x7fd0300a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:04.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1377050190 0 0) 0x7fd0300a0470 con 0x7fd0300a4720 2024-08-19T10:40:04.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): _negotiate 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): my global_id is 4860 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd018003530 con 0x7fd0300a4720 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 752968864 0 0) 0x7fd018003530 con 0x7fd0300a4720 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0300a5210 msgr2=0x7fd03013b8d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 --2- 172.21.15.190:0/11925742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0300a5210 0x7fd03013b8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 >> v1:172.21.15.190:6789/0 conn(0x7fd03013be10 legacy=0x7fd03013e200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient: _finish_hunting 0 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 monclient: found mon.noname-a-legacy 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.89:6789/0 2024-08-19T10:40:04.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0300a2470 con 0x7fd0300a4720 2024-08-19T10:40:04.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient: _finish_auth 0 2024-08-19T10:40:04.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:04.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:04.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 3927028378 0 0) 0x7fd020001ce0 con 0x7fd0300a4720 2024-08-19T10:40:04.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd036739640 5 monclient: authenticate success, global_id 4860 2024-08-19T10:40:04.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd036739640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:04.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:04.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-08-19T10:40:04.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient: dump: 2024-08-19T10:40:04.756 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:04.756 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:04.757 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] but i'm connected to v1:172.21.15.89:6789/0, reconnecting 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 >> v1:172.21.15.89:6789/0 conn(0x7fd0300a4720 legacy=0x7fd0300a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient: _add_conns ranks=[2,0,7,1,8,4,6,3,5] 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 --2- 172.21.15.190:0/11925742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd018004210 0x7fd018004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): picked mon.b con 0x7fd018004210 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 --2- 172.21.15.190:0/11925742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd018004f00 0x7fd0180099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): picked mon.a con 0x7fd018004f00 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 --2- 172.21.15.190:0/11925742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0300a5210 0x7fd01800bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): picked mon.i con 0x7fd0300a5210 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:04.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd01800c5e0 con 0x7fd018004f00 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd01800c760 con 0x7fd018004210 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd01800c8e0 con 0x7fd0300a5210 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02effd640 10 monclient(hunting): _finish_auth 0 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd036739640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd034cd6640 1 -- 172.21.15.190:0/11925742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0300a5210 msgr2=0x7fd01800bf10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd034cd6640 1 --2- 172.21.15.190:0/11925742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0300a5210 0x7fd01800bf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 1 --2- 172.21.15.190:0/11925742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd018004210 0x7fd018004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient(hunting): get_auth_request con 0x7fd018004210 auth_method 0 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02f7fe640 1 --2- 172.21.15.190:0/11925742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd018004f00 0x7fd0180099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:04.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02f7fe640 10 monclient(hunting): get_auth_request con 0x7fd018004f00 auth_method 0 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:40:04.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient(hunting): handle_auth_done global_id 4860 payload 293 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 1 -- 172.21.15.190:0/11925742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0300a5210 msgr2=0x7fd01800bf10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 1 --2- 172.21.15.190:0/11925742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0300a5210 0x7fd01800bf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 1 -- 172.21.15.190:0/11925742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd018004f00 msgr2=0x7fd0180099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 1 --2- 172.21.15.190:0/11925742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd018004f00 0x7fd0180099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 1 monclient: found mon.b 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 1 -- 172.21.15.190:0/11925742 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd018003530 con 0x7fd018004210 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient: _finish_auth 0 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:04.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.748+0000 7fd02ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02ffff640 1 -- 172.21.15.190:0/11925742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd018004210 msgr2=0x7fd018004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02ffff640 1 --2- 172.21.15.190:0/11925742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd018004210 0x7fd018004810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd02000d160 tx=0x7fd02001cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9497f9acb8b416f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd020008d70 con 0x7fd018004210 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02effd640 10 monclient: dump: 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:04.767 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd020005d50 con 0x7fd018004210 2024-08-19T10:40:04.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02effd640 1 -- 172.21.15.190:0/11925742 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd020008620 con 0x7fd018004210 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd02effd640 10 monclient: dump: 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:04.769 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 4 set_mon_vals no callback set 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 10 monclient: shutdown 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:04.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 1 -- 172.21.15.190:0/11925742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd018004210 msgr2=0x7fd018004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 1 --2- 172.21.15.190:0/11925742 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd018004210 0x7fd018004810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd02000d160 tx=0x7fd02001cb80 comp rx=0 tx=0).stop 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd034cd6640 1 -- 172.21.15.190:0/11925742 reap_dead start 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 1 -- 172.21.15.190:0/11925742 shutdown_connections 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 1 -- 172.21.15.190:0/11925742 >> 172.21.15.190:0/11925742 conn(0x7fd03006dfe0 msgr2=0x7fd0300a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 1 -- 172.21.15.190:0/11925742 shutdown_connections 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 1 -- 172.21.15.190:0/11925742 wait complete. 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 10 monclient: build_initial_monmap 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.752+0000 7fd036739640 10 monclient: monmap: 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:04.771 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 1 Processor -- start 2024-08-19T10:40:04.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 1 -- start start 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient: init 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient: _add_conns ranks=[1,2,5,8,0,4,7,3,6] 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd018004210 0x7fd03006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient(hunting): picked mon.noname-f con 0x7fd018004210 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd03009ecd0 0x7fd0301355b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient(hunting): picked mon.noname-b con 0x7fd03009ecd0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd030135af0 0x7fd0301d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient(hunting): picked mon.noname-h con 0x7fd030135af0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd0300a2ea0 con 0x7fd018004210 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:04.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd03006a400 con 0x7fd03009ecd0 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd0300a3020 con 0x7fd030135af0 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient(hunting): authenticate will time out at 1050.328718s 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02ffff640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd018004210 msgr2=0x7fd03006c8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd018004210 0x7fd03006c8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd030135af0 0x7fd0301d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd030135af0 0x7fd0301d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48066/0 (socket says 172.21.15.190:48066) 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 1 -- 172.21.15.190:0/3420281982 learned_addr learned my addr 172.21.15.190:0/3420281982 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 10 monclient(hunting): get_auth_request con 0x7fd030135af0 auth_method 0 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:04.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02f7fe640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd03009ecd0 0x7fd0301355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02f7fe640 10 monclient(hunting): get_auth_request con 0x7fd03009ecd0 auth_method 0 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 10 monclient(hunting): handle_auth_done global_id 4757 payload 824 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 1 -- 172.21.15.190:0/3420281982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd03009ecd0 msgr2=0x7fd0301355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd03009ecd0 0x7fd0301355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:04.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 1 -- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd018004210 msgr2=0x7fd03006c8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd018004210 0x7fd03006c8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 10 monclient: _finish_hunting 0 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 1 monclient: found mon.noname-h 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 1 -- 172.21.15.190:0/3420281982 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03009e850 con 0x7fd030135af0 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 1 -- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd030135af0 msgr2=0x7fd0301d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd034cd6640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd030135af0 0x7fd0301d0f70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd0300a10b0 tx=0x7fd028009e50 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 1 -- 172.21.15.190:0/3420281982 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd028006a40 con 0x7fd030135af0 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 10 monclient: dump: 2024-08-19T10:40:04.776 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:04.777 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 10 monclient: _finish_auth 0 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 10 monclient: _check_auth_tickets 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 1 -- 172.21.15.190:0/3420281982 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd028014340 con 0x7fd030135af0 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 1 -- 172.21.15.190:0/3420281982 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd028014960 con 0x7fd030135af0 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 5 monclient: authenticate success, global_id 4757 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient: _renew_subs 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 1 -- 172.21.15.190:0/3420281982 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0300a5810 con 0x7fd030135af0 2024-08-19T10:40:04.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02cff9640 10 monclient: dump: 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:04.779 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd02dffb640 4 set_mon_vals no callback set 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient: _renew_subs 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.756+0000 7fd036739640 1 -- 172.21.15.190:0/3420281982 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fd030123f80 con 0x7fd030135af0 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.760+0000 7fd036739640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.760+0000 7fd036739640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.760+0000 7fd036739640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.760+0000 7fd036739640 1 -- 172.21.15.190:0/3420281982 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0300a4b00 con 0x7fd030135af0 2024-08-19T10:40:04.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.760+0000 7fd02cff9640 1 -- 172.21.15.190:0/3420281982 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd028003d10 con 0x7fd030135af0 2024-08-19T10:40:04.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.760+0000 7fd02cff9640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd01c03be00 0x7fd01c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:04.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.760+0000 7fd02cff9640 1 -- 172.21.15.190:0/3420281982 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(54..54 src has 1..54) ==== 4771+0+0 (secure 0 0 0) 0x7fd02804f0a0 con 0x7fd030135af0 2024-08-19T10:40:04.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.760+0000 7fd02ffff640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd01c03be00 0x7fd01c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:04.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.760+0000 7fd02ffff640 10 monclient: get_auth_request con 0x7fd01c03be00 auth_method 0 2024-08-19T10:40:04.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.760+0000 7fd02ffff640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd01c03be00 0x7fd01c03e2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd0200014e0 tx=0x7fd020001870 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:04.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.764+0000 7fd02cff9640 1 -- 172.21.15.190:0/3420281982 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd0280145a0 con 0x7fd030135af0 2024-08-19T10:40:04.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.764+0000 7fd02cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:04.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.764+0000 7fd02cff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:04.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.936+0000 7fd036739640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-08-19T10:40:04.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.936+0000 7fd036739640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-08-19T10:40:04.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.936+0000 7fd036739640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:04.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:04.936+0000 7fd036739640 1 -- 172.21.15.190:0/3420281982 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"} v 0) -- 0x7fd030139640 con 0x7fd030135af0 2024-08-19T10:40:05.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.224+0000 7fd02cff9640 1 -- 172.21.15.190:0/3420281982 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}]=0 v55) ==== 150+0+0 (secure 0 0 0) 0x7fd0280065d0 con 0x7fd030135af0 2024-08-19T10:40:05.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.224+0000 7fd02cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-08-19T10:40:05.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.224+0000 7fd02cff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:05.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.332+0000 7fd036739640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-08-19T10:40:05.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.332+0000 7fd036739640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-08-19T10:40:05.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.332+0000 7fd036739640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:05.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.332+0000 7fd036739640 1 -- 172.21.15.190:0/3420281982 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"} v 0) -- 0x7fd030136020 con 0x7fd030135af0 2024-08-19T10:40:05.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.332+0000 7fd02cff9640 1 -- 172.21.15.190:0/3420281982 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}]=0 rule foo-hdd already exists v55) ==== 177+0+0 (secure 0 0 0) 0x7fd028007290 con 0x7fd030135af0 2024-08-19T10:40:05.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.332+0000 7fd02cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-replicated", "name": "foo-hdd", "root": "default", "type": "host", "class": "hdd"}] 2024-08-19T10:40:05.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.332+0000 7fd02cff9640 10 monclient: _finish_command 3 = system:0 rule foo-hdd already exists 2024-08-19T10:40:05.338 INFO:tasks.workunit.client.0.smithi190.stderr:rule foo-hdd already exists 2024-08-19T10:40:05.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 -- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd01c03be00 msgr2=0x7fd01c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:05.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd01c03be00 0x7fd01c03e2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd0200014e0 tx=0x7fd020001870 comp rx=0 tx=0).stop 2024-08-19T10:40:05.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 10 monclient: shutdown 2024-08-19T10:40:05.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:05.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 -- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd030135af0 msgr2=0x7fd0301d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:05.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd030135af0 0x7fd0301d0f70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd0300a10b0 tx=0x7fd028009e50 comp rx=0 tx=0).stop 2024-08-19T10:40:05.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 -- 172.21.15.190:0/3420281982 shutdown_connections 2024-08-19T10:40:05.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd01c03be00 0x7fd01c03e2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd030135af0 0x7fd0301d0f70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd03009ecd0 0x7fd0301355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 --2- 172.21.15.190:0/3420281982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd018004210 0x7fd03006c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 -- 172.21.15.190:0/3420281982 >> 172.21.15.190:0/3420281982 conn(0x7fd03006dfe0 msgr2=0x7fd030072d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:05.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 -- 172.21.15.190:0/3420281982 shutdown_connections 2024-08-19T10:40:05.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.336+0000 7fd036739640 1 -- 172.21.15.190:0/3420281982 wait complete. 2024-08-19T10:40:05.355 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule ls-by-class ssd 2024-08-19T10:40:05.355 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep foo-ssd 2024-08-19T10:40:05.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient: get_monmap_and_config 2024-08-19T10:40:05.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient: build_initial_monmap 2024-08-19T10:40:05.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient: monmap: 2024-08-19T10:40:05.475 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:05.476056+0000 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:05.476056+0000 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:05.476 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:05.477 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:05.477 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:05.477 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 1 Processor -- start 2024-08-19T10:40:05.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 1 -- start start 2024-08-19T10:40:05.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient: init 2024-08-19T10:40:05.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:05.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient: _add_conns ranks=[0,9,5,6,1,2,8,3,10,4,7] 2024-08-19T10:40:05.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0acc0a46b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:05.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0acc09fd00 0x7f0acc13b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient(hunting): picked mon.noname-a con 0x7f0acc09fd00 addr v2:172.21.15.89:3300/0 2024-08-19T10:40:05.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0acc0a51a0 0x7f0acc13dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient(hunting): picked mon.noname-h con 0x7f0acc0a51a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0acc0a0410 con 0x7f0acc0a46b0 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f0acc0a0ad0 con 0x7f0acc09fd00 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0acc0a2dd0 con 0x7f0acc0a51a0 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad376f640 10 monclient(hunting): authenticate will time out at 1051.046907s 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0acc0a51a0 0x7f0acc13dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0acc0a51a0 0x7f0acc13dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54718/0 (socket says 172.21.15.190:54718) 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 1 -- 172.21.15.190:0/410404405 learned_addr learned my addr 172.21.15.190:0/410404405 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 10 monclient(hunting): get_auth_request con 0x7f0acc0a51a0 auth_method 0 2024-08-19T10:40:05.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad0d0a640 1 --2- 172.21.15.190:0/410404405 >> v2:172.21.15.89:3300/0 conn(0x7f0acc09fd00 0x7f0acc13b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad0d0a640 10 monclient(hunting): get_auth_request con 0x7f0acc09fd00 auth_method 0 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad0d0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad0d0a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad0d0a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 10 monclient(hunting): handle_auth_done global_id 4766 payload 274 2024-08-19T10:40:05.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 1 -- 172.21.15.190:0/410404405 >> v2:172.21.15.89:3300/0 conn(0x7f0acc09fd00 msgr2=0x7f0acc13b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:05.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 1 --2- 172.21.15.190:0/410404405 >> v2:172.21.15.89:3300/0 conn(0x7f0acc09fd00 0x7f0acc13b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 1 -- 172.21.15.190:0/410404405 >> v1:172.21.15.89:6789/0 conn(0x7f0acc0a46b0 legacy=0x7f0acc0a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:05.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 10 monclient: _finish_hunting 0 2024-08-19T10:40:05.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 1 monclient: found mon.noname-h 2024-08-19T10:40:05.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:05.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:05.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.472+0000 7f0ad1d0c640 1 -- 172.21.15.190:0/410404405 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc0a2410 con 0x7f0acc0a51a0 2024-08-19T10:40:05.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ad1d0c640 1 -- 172.21.15.190:0/410404405 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0acc0a51a0 msgr2=0x7f0acc13dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:05.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ad1d0c640 1 --2- 172.21.15.190:0/410404405 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0acc0a51a0 0x7f0acc13dc60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009e50 tx=0x7f0ac401c8f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a56596b14b7cd902 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:05.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 1 -- 172.21.15.190:0/410404405 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ac40090d0 con 0x7f0acc0a51a0 2024-08-19T10:40:05.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:05.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:05.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 10 monclient: dump: 2024-08-19T10:40:05.484 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:05.485 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 10 monclient: _finish_auth 0 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 1 -- 172.21.15.190:0/410404405 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac40a59a0 con 0x7f0acc0a51a0 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ad376f640 5 monclient: authenticate success, global_id 4766 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ad376f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ad376f640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 1 -- 172.21.15.190:0/410404405 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0ac40ad3f0 con 0x7f0acc0a51a0 2024-08-19T10:40:05.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ad376f640 4 set_mon_vals no callback set 2024-08-19T10:40:05.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:05.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:05.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.476+0000 7f0ac3fff640 10 monclient: dump: 2024-08-19T10:40:05.487 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:05.487 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:05.487 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.487 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.487 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:05.487 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.487 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:05.487 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient: shutdown 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 -- 172.21.15.190:0/410404405 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0acc0a51a0 msgr2=0x7f0acc13dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 --2- 172.21.15.190:0/410404405 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0acc0a51a0 0x7f0acc13dc60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009e50 tx=0x7f0ac401c8f0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 -- 172.21.15.190:0/410404405 shutdown_connections 2024-08-19T10:40:05.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 --2- 172.21.15.190:0/410404405 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0acc0a51a0 0x7f0acc13dc60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 --2- 172.21.15.190:0/410404405 >> v2:172.21.15.89:3300/0 conn(0x7f0acc09fd00 0x7f0acc13b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 -- 172.21.15.190:0/410404405 >> 172.21.15.190:0/410404405 conn(0x7f0acc0a92a0 msgr2=0x7f0acc0a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 -- 172.21.15.190:0/410404405 shutdown_connections 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 -- 172.21.15.190:0/410404405 wait complete. 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient: build_initial_monmap 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient: monmap: 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.489 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:05.490 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:05.490 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:05.490 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:05.490 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:05.490 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:05.490 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:05.490 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:05.490 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:05.490 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 Processor -- start 2024-08-19T10:40:05.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 -- start start 2024-08-19T10:40:05.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient: init 2024-08-19T10:40:05.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:05.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient: _add_conns ranks=[7,0,8,3,2,4,1,5,6] 2024-08-19T10:40:05.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0acc09fd00 0x7f0acc1d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient(hunting): picked mon.noname-i con 0x7f0acc09fd00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0acc0a46b0 0x7f0acc1347a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient(hunting): picked mon.noname-a con 0x7f0acc0a46b0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0acc0a51a0 0x7f0acc09ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient(hunting): picked mon.noname-e con 0x7f0acc0a51a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc06ca50 con 0x7f0acc0a46b0 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0acc06c8d0 con 0x7f0acc09fd00 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f0acc06cbd0 con 0x7f0acc0a51a0 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad376f640 10 monclient(hunting): authenticate will time out at 1051.054393s 2024-08-19T10:40:05.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad150b640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0acc09fd00 msgr2=0x7f0acc1d0c60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad150b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0acc09fd00 0x7f0acc1d0c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad1d0c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0acc0a51a0 0x7f0acc09ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0acc0a46b0 0x7f0acc1347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad1d0c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0acc0a51a0 0x7f0acc09ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:35364/0 (socket says 172.21.15.190:35364) 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad1d0c640 1 -- 172.21.15.190:0/245312830 learned_addr learned my addr 172.21.15.190:0/245312830 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad1d0c640 10 monclient(hunting): get_auth_request con 0x7f0acc0a51a0 auth_method 0 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad1d0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad1d0c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad1d0c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 10 monclient(hunting): get_auth_request con 0x7f0acc0a46b0 auth_method 0 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:05.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:05.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad1d0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:05.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad1d0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:05.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad1d0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:05.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:05.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:05.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:05.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 10 monclient(hunting): handle_auth_done global_id 4878 payload 824 2024-08-19T10:40:05.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 1 -- 172.21.15.190:0/245312830 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0acc0a51a0 msgr2=0x7f0acc09ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:05.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0acc0a51a0 0x7f0acc09ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 1 -- 172.21.15.190:0/245312830 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0acc09fd00 msgr2=0x7f0acc1d0c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:05.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0acc09fd00 0x7f0acc1d0c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 10 monclient: _finish_hunting 0 2024-08-19T10:40:05.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 1 monclient: found mon.noname-a 2024-08-19T10:40:05.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:05.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:05.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.480+0000 7f0ad0d0a640 1 -- 172.21.15.190:0/245312830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc120140 con 0x7f0acc0a46b0 2024-08-19T10:40:05.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad0d0a640 1 -- 172.21.15.190:0/245312830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0acc0a46b0 msgr2=0x7f0acc1347a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:05.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad0d0a640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0acc0a46b0 0x7f0acc1347a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00d5f0 tx=0x7f0abc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:05.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 1 -- 172.21.15.190:0/245312830 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0abc018020 con 0x7f0acc0a46b0 2024-08-19T10:40:05.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 10 monclient: dump: 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:05.500 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:05.501 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:05.501 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:05.501 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:05.501 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:05.501 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:05.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:05.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:05.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad376f640 5 monclient: authenticate success, global_id 4878 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 1 -- 172.21.15.190:0/245312830 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0abc014ca0 con 0x7f0acc0a46b0 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac37fe640 4 set_mon_vals no callback set 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 1 -- 172.21.15.190:0/245312830 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0abc014e50 con 0x7f0acc0a46b0 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ac1ffb640 10 monclient: dump: 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.502 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad376f640 10 monclient: _renew_subs 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad376f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:05.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad376f640 1 -- 172.21.15.190:0/245312830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0acc0a57c0 con 0x7f0acc0a46b0 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad376f640 10 monclient: _renew_subs 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad376f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad376f640 1 -- 172.21.15.190:0/245312830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0acc09e730 con 0x7f0acc0a46b0 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad376f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad376f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad376f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.484+0000 7f0ad376f640 1 -- 172.21.15.190:0/245312830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0acc0a59f0 con 0x7f0acc0a46b0 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.488+0000 7f0ac1ffb640 1 -- 172.21.15.190:0/245312830 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0abc01b020 con 0x7f0acc0a46b0 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.488+0000 7f0ac1ffb640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0ab003bfd0 0x7f0ab003e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.488+0000 7f0ac1ffb640 1 -- 172.21.15.190:0/245312830 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4834+0+0 (secure 0 0 0) 0x7f0abc00f040 con 0x7f0acc0a46b0 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.488+0000 7f0ad150b640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0ab003bfd0 0x7f0ab003e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.488+0000 7f0ad150b640 10 monclient: get_auth_request con 0x7f0ab003bfd0 auth_method 0 2024-08-19T10:40:05.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.488+0000 7f0ad150b640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0ab003bfd0 0x7f0ab003e490 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8003780 tx=0x7f0ac8007a60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:05.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.500+0000 7f0ac1ffb640 1 -- 172.21.15.190:0/245312830 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0acc0a59f0 con 0x7f0acc0a46b0 2024-08-19T10:40:05.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.500+0000 7f0ac1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:05.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.500+0000 7f0ac1ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:05.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.672+0000 7f0ad376f640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:05.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.672+0000 7f0ad376f640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:05.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.672+0000 7f0ad376f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:05.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.672+0000 7f0ad376f640 1 -- 172.21.15.190:0/245312830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) -- 0x7f0acc11f5e0 con 0x7f0acc0a46b0 2024-08-19T10:40:05.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.672+0000 7f0ac1ffb640 1 -- 172.21.15.190:0/245312830 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v55) ==== 90+0+8 (secure 0 0 0) 0x7f0abc022020 con 0x7f0acc0a46b0 2024-08-19T10:40:05.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.672+0000 7f0ac1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:05.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.672+0000 7f0ac1ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:05.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.776+0000 7f0ad376f640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:05.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.776+0000 7f0ad376f640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:05.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.776+0000 7f0ad376f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:05.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.776+0000 7f0ad376f640 1 -- 172.21.15.190:0/245312830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) -- 0x7f0acc11f310 con 0x7f0acc0a46b0 2024-08-19T10:40:05.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.776+0000 7f0ac1ffb640 1 -- 172.21.15.190:0/245312830 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v55) ==== 90+0+8 (secure 0 0 0) 0x7f0abc01c3b0 con 0x7f0acc0a46b0 2024-08-19T10:40:05.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.776+0000 7f0ac1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:05.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.776+0000 7f0ac1ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:40:05.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 -- 172.21.15.190:0/245312830 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0ab003bfd0 msgr2=0x7f0ab003e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:05.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0ab003bfd0 0x7f0ab003e490 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8003780 tx=0x7f0ac8007a60 comp rx=0 tx=0).stop 2024-08-19T10:40:05.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 10 monclient: shutdown 2024-08-19T10:40:05.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:05.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 -- 172.21.15.190:0/245312830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0acc0a46b0 msgr2=0x7f0acc1347a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:05.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0acc0a46b0 0x7f0acc1347a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00d5f0 tx=0x7f0abc00dab0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 -- 172.21.15.190:0/245312830 shutdown_connections 2024-08-19T10:40:05.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0ab003bfd0 0x7f0ab003e490 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0acc0a51a0 0x7f0acc09ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0acc0a46b0 0x7f0acc1347a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 --2- 172.21.15.190:0/245312830 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0acc09fd00 0x7f0acc1d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 -- 172.21.15.190:0/245312830 >> 172.21.15.190:0/245312830 conn(0x7f0acc0a92a0 msgr2=0x7f0acc0a6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:05.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 -- 172.21.15.190:0/245312830 shutdown_connections 2024-08-19T10:40:05.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.780+0000 7f0ad376f640 1 -- 172.21.15.190:0/245312830 wait complete. 2024-08-19T10:40:05.799 INFO:tasks.workunit.client.0.smithi190.stdout:foo-ssd 2024-08-19T10:40:05.800 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule ls-by-class ssd 2024-08-19T10:40:05.800 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false grep foo-hdd 2024-08-19T10:40:05.800 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:40:05.800 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep foo-hdd 2024-08-19T10:40:05.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.904+0000 7fb102847640 10 monclient: get_monmap_and_config 2024-08-19T10:40:05.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.904+0000 7fb102847640 10 monclient: build_initial_monmap 2024-08-19T10:40:05.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient: monmap: 2024-08-19T10:40:05.911 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:05.911 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:05.911 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:05.911568+0000 2024-08-19T10:40:05.911 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:05.911568+0000 2024-08-19T10:40:05.911 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:05.912 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.912 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:05.912 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:05.912 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:05.912 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:05.912 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:05.912 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:05.912 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:05.912 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:05.912 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:05.912 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:05.913 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:05.913 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 1 Processor -- start 2024-08-19T10:40:05.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 1 -- start start 2024-08-19T10:40:05.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient: init 2024-08-19T10:40:05.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:05.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient: _add_conns ranks=[10,4,3,7,8,0,9,1,2,6,5] 2024-08-19T10:40:05.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb0fc0a3610 0x7fb0fc0a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient(hunting): picked mon.noname-f con 0x7fb0fc0a3610 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:05.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0fc0a3f30 0x7fb0fc1373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient(hunting): picked mon.noname-c con 0x7fb0fc0a3f30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:05.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0fc137920 0x7fb0fc137dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient(hunting): picked mon.noname-g con 0x7fb0fc137920 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:05.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:05.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb0fc070830 con 0x7fb0fc0a3610 2024-08-19T10:40:05.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:05.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb0fc074380 con 0x7fb0fc137920 2024-08-19T10:40:05.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:05.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb0fc070170 con 0x7fb0fc0a3f30 2024-08-19T10:40:05.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:05.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb102847640 10 monclient(hunting): authenticate will time out at 1051.481348s 2024-08-19T10:40:05.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb0fbfff640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fb0fc0a3610 msgr2=0x7fb0fc0a39f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:05.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb0fbfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb0fc0a3610 0x7fb0fc0a39f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:05.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0fc137920 0x7fb0fc137dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:05.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0fc137920 0x7fb0fc137dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:56924/0 (socket says 172.21.15.190:56924) 2024-08-19T10:40:05.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 1 -- 172.21.15.190:0/3212334141 learned_addr learned my addr 172.21.15.190:0/3212334141 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:05.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 10 monclient(hunting): get_auth_request con 0x7fb0fc137920 auth_method 0 2024-08-19T10:40:05.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:05.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:05.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:05.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:05.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:05.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:05.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb0fb7fe640 1 --2- 172.21.15.190:0/3212334141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0fc0a3f30 0x7fb0fc1373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:05.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb0fb7fe640 10 monclient(hunting): get_auth_request con 0x7fb0fc0a3f30 auth_method 0 2024-08-19T10:40:05.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb0fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:05.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb0fb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:05.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb0fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:05.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 10 monclient(hunting): handle_auth_done global_id 4728 payload 274 2024-08-19T10:40:05.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 1 -- 172.21.15.190:0/3212334141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0fc0a3f30 msgr2=0x7fb0fc1373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:05.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 1 --2- 172.21.15.190:0/3212334141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0fc0a3f30 0x7fb0fc1373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 1 -- 172.21.15.190:0/3212334141 >> v2:172.21.15.190:3300/0 conn(0x7fb0fc0a3610 msgr2=0x7fb0fc0a39f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:05.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 1 --2- 172.21.15.190:0/3212334141 >> v2:172.21.15.190:3300/0 conn(0x7fb0fc0a3610 0x7fb0fc0a39f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 10 monclient: _finish_hunting 0 2024-08-19T10:40:05.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 1 monclient: found mon.noname-g 2024-08-19T10:40:05.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:05.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:05.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.908+0000 7fb100de4640 1 -- 172.21.15.190:0/3212334141 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc0740a0 con 0x7fb0fc137920 2024-08-19T10:40:05.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb100de4640 1 -- 172.21.15.190:0/3212334141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0fc137920 msgr2=0x7fb0fc137dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:05.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb100de4640 1 --2- 172.21.15.190:0/3212334141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0fc137920 0x7fb0fc137dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009e80 tx=0x7fb0f401c8f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=34d5f01abfb5f24d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:05.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 1 -- 172.21.15.190:0/3212334141 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0f40090d0 con 0x7fb0fc137920 2024-08-19T10:40:05.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:05.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 10 monclient: dump: 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:05.927 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 10 monclient: _finish_auth 0 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 1 -- 172.21.15.190:0/3212334141 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f40a5ab0 con 0x7fb0fc137920 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 5 monclient: authenticate success, global_id 4728 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 1 -- 172.21.15.190:0/3212334141 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0f40a5c60 con 0x7fb0fc137920 2024-08-19T10:40:05.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 4 set_mon_vals no callback set 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb0faffd640 10 monclient: dump: 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:05.929 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 10 monclient: shutdown 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 1 -- 172.21.15.190:0/3212334141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0fc137920 msgr2=0x7fb0fc137dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 1 --2- 172.21.15.190:0/3212334141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0fc137920 0x7fb0fc137dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009e80 tx=0x7fb0f401c8f0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 1 -- 172.21.15.190:0/3212334141 shutdown_connections 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 1 --2- 172.21.15.190:0/3212334141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0fc137920 0x7fb0fc137dd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 1 --2- 172.21.15.190:0/3212334141 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0fc0a3f30 0x7fb0fc1373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 1 --2- 172.21.15.190:0/3212334141 >> v2:172.21.15.190:3300/0 conn(0x7fb0fc0a3610 0x7fb0fc0a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 1 -- 172.21.15.190:0/3212334141 >> 172.21.15.190:0/3212334141 conn(0x7fb0fc0a4700 msgr2=0x7fb0fc0a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 1 -- 172.21.15.190:0/3212334141 shutdown_connections 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 1 -- 172.21.15.190:0/3212334141 wait complete. 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 10 monclient: build_initial_monmap 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 10 monclient: monmap: 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:05.931 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 1 Processor -- start 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 1 -- start start 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.912+0000 7fb102847640 10 monclient: init 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 10 monclient: _add_conns ranks=[2,7,0,3,6,5,4,1,8] 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0fc0a3610 0x7fb0fc11e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 10 monclient(hunting): picked mon.noname-b con 0x7fb0fc0a3610 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:05.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb0fc0a3f30 0x7fb0fc11e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 10 monclient(hunting): picked mon.noname-i con 0x7fb0fc0a3f30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0fc137920 0x7fb0fc11ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 10 monclient(hunting): picked mon.noname-a con 0x7fb0fc137920 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc0a4310 con 0x7fb0fc137920 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb0fc070a30 con 0x7fb0fc0a3610 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb0fc0a0490 con 0x7fb0fc0a3f30 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb102847640 10 monclient(hunting): authenticate will time out at 1051.488343s 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fb7fe640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb0fc0a3f30 msgr2=0x7fb0fc11e790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:05.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fb7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb0fc0a3f30 0x7fb0fc11e790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0fc0a3610 0x7fb0fc11e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb100de4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0fc137920 0x7fb0fc11ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0fc0a3610 0x7fb0fc11e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:49596/0 (socket says 172.21.15.190:49596) 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 1 -- 172.21.15.190:0/208734672 learned_addr learned my addr 172.21.15.190:0/208734672 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 10 monclient(hunting): get_auth_request con 0x7fb0fc0a3610 auth_method 0 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb100de4640 10 monclient(hunting): get_auth_request con 0x7fb0fc137920 auth_method 0 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb100de4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb100de4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:05.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb100de4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb100de4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb100de4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb100de4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 10 monclient(hunting): handle_auth_done global_id 4727 payload 824 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 1 -- 172.21.15.190:0/208734672 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb0fc0a3f30 msgr2=0x7fb0fc11e790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb0fc0a3f30 0x7fb0fc11e790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 1 -- 172.21.15.190:0/208734672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0fc137920 msgr2=0x7fb0fc11ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0fc137920 0x7fb0fc11ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:05.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 1 monclient: found mon.noname-b 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb0fbfff640 1 -- 172.21.15.190:0/208734672 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc0a3aa0 con 0x7fb0fc0a3610 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.916+0000 7fb100de4640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0fc137920 0x7fb0fc11ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0fbfff640 1 -- 172.21.15.190:0/208734672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0fc0a3610 msgr2=0x7fb0fc11e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0fbfff640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0fc0a3610 0x7fb0fc11e250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00b550 tx=0x7fb0ec00ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 1 -- 172.21.15.190:0/208734672 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0ec003de0 con 0x7fb0fc0a3610 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 10 monclient: dump: 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:05.936 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 10 monclient: _finish_auth 0 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb102847640 5 monclient: authenticate success, global_id 4727 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb102847640 10 monclient: _renew_subs 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb102847640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb102847640 1 -- 172.21.15.190:0/208734672 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc121610 con 0x7fb0fc0a3610 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb102847640 10 monclient: _renew_subs 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb102847640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb102847640 1 -- 172.21.15.190:0/208734672 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc0a09c0 con 0x7fb0fc0a3610 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 1 -- 172.21.15.190:0/208734672 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ec015070 con 0x7fb0fc0a3610 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 1 -- 172.21.15.190:0/208734672 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0ec005560 con 0x7fb0fc0a3610 2024-08-19T10:40:05.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0fa7fc640 4 set_mon_vals no callback set 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 10 monclient: dump: 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:05.939 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb102847640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb102847640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb102847640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb102847640 1 -- 172.21.15.190:0/208734672 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0dc005090 con 0x7fb0fc0a3610 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 1 -- 172.21.15.190:0/208734672 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb0ec0050f0 con 0x7fb0fc0a3610 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0e803be00 0x7fb0e803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0f8ff9640 1 -- 172.21.15.190:0/208734672 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 4834+0+0 (secure 0 0 0) 0x7fb0ec04f230 con 0x7fb0fc0a3610 2024-08-19T10:40:05.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0fb7fe640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0e803be00 0x7fb0e803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:05.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.920+0000 7fb0fb7fe640 10 monclient: get_auth_request con 0x7fb0e803be00 auth_method 0 2024-08-19T10:40:05.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.924+0000 7fb0fb7fe640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0e803be00 0x7fb0e803e2c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0000f00 tx=0x7fb0f0006e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:05.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.932+0000 7fb0f8ff9640 1 -- 172.21.15.190:0/208734672 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb0fc0a3aa0 con 0x7fb0fc0a3610 2024-08-19T10:40:05.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.932+0000 7fb0f8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:05.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:05.932+0000 7fb0f8ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:06.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.092+0000 7fb102847640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:06.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.092+0000 7fb102847640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:06.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.092+0000 7fb102847640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:06.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.092+0000 7fb102847640 1 -- 172.21.15.190:0/208734672 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) -- 0x7fb0dc005650 con 0x7fb0fc0a3610 2024-08-19T10:40:06.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.092+0000 7fb0f8ff9640 1 -- 172.21.15.190:0/208734672 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v55) ==== 90+0+8 (secure 0 0 0) 0x7fb0ec04cc40 con 0x7fb0fc0a3610 2024-08-19T10:40:06.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.092+0000 7fb0f8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:06.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.092+0000 7fb0f8ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:06.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.184+0000 7fb102847640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:06.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.184+0000 7fb102847640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:06.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.184+0000 7fb102847640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:06.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.184+0000 7fb102847640 1 -- 172.21.15.190:0/208734672 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "ssd"} v 0) -- 0x7fb0dc005b40 con 0x7fb0fc0a3610 2024-08-19T10:40:06.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.184+0000 7fb0f8ff9640 1 -- 172.21.15.190:0/208734672 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "ssd"}]=0 v55) ==== 90+0+8 (secure 0 0 0) 0x7fb0ec005a50 con 0x7fb0fc0a3610 2024-08-19T10:40:06.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.184+0000 7fb0f8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "ssd"}] 2024-08-19T10:40:06.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.184+0000 7fb0f8ff9640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:40:06.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 -- 172.21.15.190:0/208734672 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0e803be00 msgr2=0x7fb0e803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:06.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0e803be00 0x7fb0e803e2c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0000f00 tx=0x7fb0f0006e20 comp rx=0 tx=0).stop 2024-08-19T10:40:06.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 10 monclient: shutdown 2024-08-19T10:40:06.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:06.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 -- 172.21.15.190:0/208734672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0fc0a3610 msgr2=0x7fb0fc11e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:06.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0fc0a3610 0x7fb0fc11e250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00b550 tx=0x7fb0ec00ba10 comp rx=0 tx=0).stop 2024-08-19T10:40:06.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 -- 172.21.15.190:0/208734672 shutdown_connections 2024-08-19T10:40:06.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0e803be00 0x7fb0e803e2c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0fc137920 0x7fb0fc11ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb0fc0a3f30 0x7fb0fc11e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 --2- 172.21.15.190:0/208734672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0fc0a3610 0x7fb0fc11e250 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 -- 172.21.15.190:0/208734672 >> 172.21.15.190:0/208734672 conn(0x7fb0fc0a4700 msgr2=0x7fb0fc09ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:06.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 -- 172.21.15.190:0/208734672 shutdown_connections 2024-08-19T10:40:06.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.188+0000 7fb102847640 1 -- 172.21.15.190:0/208734672 wait complete. 2024-08-19T10:40:06.206 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:40:06.207 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep foo-hdd 2024-08-19T10:40:06.207 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule ls-by-class hdd 2024-08-19T10:40:06.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 10 monclient: get_monmap_and_config 2024-08-19T10:40:06.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 10 monclient: build_initial_monmap 2024-08-19T10:40:06.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 10 monclient: monmap: 2024-08-19T10:40:06.322 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:06.322 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:06.322 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:06.322225+0000 2024-08-19T10:40:06.322 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:06.322225+0000 2024-08-19T10:40:06.322 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:06.322 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.322 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:06.323 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:06.323 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:06.323 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:06.323 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:06.323 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:06.323 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:06.323 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:06.323 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:06.323 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:06.323 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:06.323 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 1 Processor -- start 2024-08-19T10:40:06.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 1 -- start start 2024-08-19T10:40:06.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 10 monclient: init 2024-08-19T10:40:06.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:06.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 10 monclient: _add_conns ranks=[6,0,1,10,8,2,5,4,3,7,9] 2024-08-19T10:40:06.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ae4139980 0x7f8ae40a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 10 monclient(hunting): picked mon.noname-d con 0x7f8ae4139980 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:06.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8ae4139d60 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:06.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8ae413a140 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:06.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ae40a8620 con 0x7f8ae4139d60 2024-08-19T10:40:06.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.316+0000 7f8aeb291640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ae40a5b50 con 0x7f8ae413a140 2024-08-19T10:40:06.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8aeb291640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:06.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8aeb291640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8ae40a7600 con 0x7f8ae4139980 2024-08-19T10:40:06.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8aeb291640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:06.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8aeb291640 10 monclient(hunting): authenticate will time out at 1051.892018s 2024-08-19T10:40:06.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae982e640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f8ae413a140 legacy=0x7f8ae40a6ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ae4139980 0x7f8ae40a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae882c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8ae4139d60 0x7f8ae40a67e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:46502/0 (socket says 172.21.15.190:46502) 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae882c640 1 -- 172.21.15.190:0/935514499 learned_addr learned my addr 172.21.15.190:0/935514499 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 10 monclient(hunting): get_auth_request con 0x7f8ae4139980 auth_method 0 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 10 monclient(hunting): handle_auth_done global_id 4704 payload 274 2024-08-19T10:40:06.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 1 -- 172.21.15.190:0/935514499 >> v1:172.21.15.190:6789/0 conn(0x7f8ae413a140 legacy=0x7f8ae40a6ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:06.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 1 -- 172.21.15.190:0/935514499 >> v1:172.21.15.89:6789/0 conn(0x7f8ae4139d60 legacy=0x7f8ae40a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:06.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 10 monclient: _finish_hunting 0 2024-08-19T10:40:06.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 1 monclient: found mon.noname-d 2024-08-19T10:40:06.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:06.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:06.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.320+0000 7f8ae902d640 1 -- 172.21.15.190:0/935514499 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae40a45b0 con 0x7f8ae4139980 2024-08-19T10:40:06.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8ae902d640 1 -- 172.21.15.190:0/935514499 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ae4139980 msgr2=0x7f8ae40a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:06.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8ae902d640 1 --2- 172.21.15.190:0/935514499 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ae4139980 0x7f8ae40a5610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00097f0 tx=0x7f8ae001c370 comp rx=0 tx=0).ready entity=mon.6 client_cookie=17c8800f797d4d07 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:06.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 1 -- 172.21.15.190:0/935514499 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ae00b0070 con 0x7f8ae4139980 2024-08-19T10:40:06.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:06.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:06.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 10 monclient: dump: 2024-08-19T10:40:06.334 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:06.334 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:06.335 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:06.336 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:06.336 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 10 monclient: _finish_auth 0 2024-08-19T10:40:06.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:06.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:06.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 1 -- 172.21.15.190:0/935514499 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae00aa810 con 0x7f8ae4139980 2024-08-19T10:40:06.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:06.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 1 -- 172.21.15.190:0/935514499 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ae00aae30 con 0x7f8ae4139980 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8adbfff640 10 monclient: dump: 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:06.337 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 5 monclient: authenticate success, global_id 4704 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 4 set_mon_vals no callback set 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 10 monclient: shutdown 2024-08-19T10:40:06.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 1 -- 172.21.15.190:0/935514499 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ae4139980 msgr2=0x7f8ae40a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 1 --2- 172.21.15.190:0/935514499 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ae4139980 0x7f8ae40a5610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00097f0 tx=0x7f8ae001c370 comp rx=0 tx=0).stop 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 1 -- 172.21.15.190:0/935514499 shutdown_connections 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 1 --2- 172.21.15.190:0/935514499 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ae4139980 0x7f8ae40a5610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 1 -- 172.21.15.190:0/935514499 >> 172.21.15.190:0/935514499 conn(0x7f8ae40abde0 msgr2=0x7f8ae40ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 1 -- 172.21.15.190:0/935514499 shutdown_connections 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 1 -- 172.21.15.190:0/935514499 wait complete. 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 10 monclient: build_initial_monmap 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.324+0000 7f8aeb291640 10 monclient: monmap: 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:06.339 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 1 Processor -- start 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 1 -- start start 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 10 monclient: init 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 10 monclient: _add_conns ranks=[0,2,7,6,8,5,1,3,4] 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae4139980 0x7f8ae4140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 10 monclient(hunting): picked mon.noname-a con 0x7f8ae4139980 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ae4139d60 0x7f8ae4138690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 10 monclient(hunting): picked mon.noname-b con 0x7f8ae4139d60 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ae413a140 0x7f8ae41d1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 10 monclient(hunting): picked mon.noname-i con 0x7f8ae413a140 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae40a34d0 con 0x7f8ae4139980 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:06.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae40a3350 con 0x7f8ae4139d60 2024-08-19T10:40:06.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:06.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8ae4145440 con 0x7f8ae413a140 2024-08-19T10:40:06.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:06.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8aeb291640 10 monclient(hunting): authenticate will time out at 1051.900867s 2024-08-19T10:40:06.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae982e640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ae413a140 msgr2=0x7f8ae41d1440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:06.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae982e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ae413a140 0x7f8ae41d1440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:06.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae902d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae4139980 0x7f8ae4140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:06.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ae4139d60 0x7f8ae4138690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:06.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae902d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae4139980 0x7f8ae4140a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:36194/0 (socket says 172.21.15.190:36194) 2024-08-19T10:40:06.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae902d640 1 -- 172.21.15.190:0/1839417631 learned_addr learned my addr 172.21.15.190:0/1839417631 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:06.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae902d640 10 monclient(hunting): get_auth_request con 0x7f8ae4139980 auth_method 0 2024-08-19T10:40:06.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae902d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:06.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae902d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:06.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae902d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:06.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 10 monclient(hunting): get_auth_request con 0x7f8ae4139d60 auth_method 0 2024-08-19T10:40:06.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:06.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:06.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:06.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:06.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae902d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae902d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae902d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 10 monclient(hunting): handle_auth_done global_id 4736 payload 824 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 1 -- 172.21.15.190:0/1839417631 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ae413a140 msgr2=0x7f8ae41d1440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ae413a140 0x7f8ae41d1440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 1 -- 172.21.15.190:0/1839417631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae4139980 msgr2=0x7f8ae4140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae4139980 0x7f8ae4140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 10 monclient: _finish_hunting 0 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 1 monclient: found mon.noname-b 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:06.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:06.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.328+0000 7f8ae882c640 1 -- 172.21.15.190:0/1839417631 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae4141390 con 0x7f8ae4139d60 2024-08-19T10:40:06.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ae882c640 1 -- 172.21.15.190:0/1839417631 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ae4139d60 msgr2=0x7f8ae4138690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:06.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ae882c640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ae4139d60 0x7f8ae4138690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c440 tx=0x7f8ad400c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:06.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 1 -- 172.21.15.190:0/1839417631 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ad4017070 con 0x7f8ae4139d60 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 10 monclient: dump: 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:06.347 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 1 -- 172.21.15.190:0/1839417631 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad4005d00 con 0x7f8ae4139d60 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 1 -- 172.21.15.190:0/1839417631 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ad4007540 con 0x7f8ae4139d60 2024-08-19T10:40:06.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8adb7fe640 4 set_mon_vals no callback set 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 10 monclient: dump: 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:06.349 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8aeb291640 5 monclient: authenticate success, global_id 4736 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8aeb291640 10 monclient: _renew_subs 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8aeb291640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8aeb291640 1 -- 172.21.15.190:0/1839417631 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae4142200 con 0x7f8ae4139d60 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8aeb291640 10 monclient: _renew_subs 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8aeb291640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:06.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8aeb291640 1 -- 172.21.15.190:0/1839417631 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae413f8c0 con 0x7f8ae4139d60 2024-08-19T10:40:06.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8aeb291640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:06.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8aeb291640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:06.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8aeb291640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:06.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8aeb291640 1 -- 172.21.15.190:0/1839417631 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae40a2470 con 0x7f8ae4139d60 2024-08-19T10:40:06.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 1 -- 172.21.15.190:0/1839417631 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8ad4007070 con 0x7f8ae4139d60 2024-08-19T10:40:06.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac803be00 0x7f8ac803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ad9ffb640 1 -- 172.21.15.190:0/1839417631 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 4834+0+0 (secure 0 0 0) 0x7f8ad404e1e0 con 0x7f8ae4139d60 2024-08-19T10:40:06.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ae902d640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac803be00 0x7f8ac803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:06.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.332+0000 7f8ae902d640 10 monclient: get_auth_request con 0x7f8ac803be00 auth_method 0 2024-08-19T10:40:06.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.336+0000 7f8ae902d640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac803be00 0x7f8ac803e2c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8ae001c3b0 tx=0x7f8ae00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:06.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.344+0000 7f8ad9ffb640 1 -- 172.21.15.190:0/1839417631 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8ad4010300 con 0x7f8ae4139d60 2024-08-19T10:40:06.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.344+0000 7f8ad9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:06.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.344+0000 7f8ad9ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:06.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.504+0000 7f8aeb291640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:06.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.504+0000 7f8aeb291640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:06.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.504+0000 7f8aeb291640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:06.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.504+0000 7f8aeb291640 1 -- 172.21.15.190:0/1839417631 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) -- 0x7f8ae406c8d0 con 0x7f8ae4139d60 2024-08-19T10:40:06.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.504+0000 7f8ad9ffb640 1 -- 172.21.15.190:0/1839417631 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v55) ==== 90+0+8 (secure 0 0 0) 0x7f8ad40200a0 con 0x7f8ae4139d60 2024-08-19T10:40:06.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.504+0000 7f8ad9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:06.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.504+0000 7f8ad9ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.600+0000 7f8aeb291640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.600+0000 7f8aeb291640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.600+0000 7f8aeb291640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.600+0000 7f8aeb291640 1 -- 172.21.15.190:0/1839417631 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) -- 0x7f8ae40a5610 con 0x7f8ae4139d60 2024-08-19T10:40:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.600+0000 7f8ad9ffb640 1 -- 172.21.15.190:0/1839417631 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v55) ==== 90+0+8 (secure 0 0 0) 0x7f8ad400ecf0 con 0x7f8ae4139d60 2024-08-19T10:40:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.600+0000 7f8ad9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.600+0000 7f8ad9ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:40:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 -- 172.21.15.190:0/1839417631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac803be00 msgr2=0x7f8ac803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac803be00 0x7f8ac803e2c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8ae001c3b0 tx=0x7f8ae00a6000 comp rx=0 tx=0).stop 2024-08-19T10:40:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 10 monclient: shutdown 2024-08-19T10:40:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 -- 172.21.15.190:0/1839417631 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ae4139d60 msgr2=0x7f8ae4138690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ae4139d60 0x7f8ae4138690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c440 tx=0x7f8ad400c900 comp rx=0 tx=0).stop 2024-08-19T10:40:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 -- 172.21.15.190:0/1839417631 shutdown_connections 2024-08-19T10:40:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac803be00 0x7f8ac803e2c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ae413a140 0x7f8ae41d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ae4139d60 0x7f8ae4138690 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 --2- 172.21.15.190:0/1839417631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae4139980 0x7f8ae4140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 -- 172.21.15.190:0/1839417631 >> 172.21.15.190:0/1839417631 conn(0x7f8ae40abde0 msgr2=0x7f8ae409cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 -- 172.21.15.190:0/1839417631 shutdown_connections 2024-08-19T10:40:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.604+0000 7f8aeb291640 1 -- 172.21.15.190:0/1839417631 wait complete. 2024-08-19T10:40:06.621 INFO:tasks.workunit.client.0.smithi190.stdout:foo-hdd 2024-08-19T10:40:06.622 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule ls-by-class hdd 2024-08-19T10:40:06.622 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false grep foo-ssd 2024-08-19T10:40:06.622 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:40:06.622 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep foo-ssd 2024-08-19T10:40:06.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient: get_monmap_and_config 2024-08-19T10:40:06.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient: build_initial_monmap 2024-08-19T10:40:06.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient: monmap: 2024-08-19T10:40:06.739 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:06.739 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:06.739 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:06.739753+0000 2024-08-19T10:40:06.739 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:06.739753+0000 2024-08-19T10:40:06.739 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:06.740 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 1 Processor -- start 2024-08-19T10:40:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 1 -- start start 2024-08-19T10:40:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient: init 2024-08-19T10:40:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient: _add_conns ranks=[10,2,6,4,0,7,1,5,3,9,8] 2024-08-19T10:40:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f42280a46b0 0x7f42280a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient(hunting): picked mon.noname-f con 0x7f42280a46b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42280a4fd0 0x7f4228139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient(hunting): picked mon.noname-b con 0x7f42280a4fd0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4228139da0 0x7f422813a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient(hunting): picked mon.noname-d con 0x7f4228139da0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f42280a0210 con 0x7f42280a46b0 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f422806cea0 con 0x7f42280a4fd0 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f422806ca60 con 0x7f4228139da0 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f422d341640 10 monclient(hunting): authenticate will time out at 1052.309608s 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f4226ffd640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f42280a46b0 msgr2=0x7f42280a4a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f4226ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f42280a46b0 0x7f42280a4a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4228139da0 0x7f422813a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4228139da0 0x7f422813a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:51448/0 (socket says 172.21.15.190:51448) 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42267fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42280a4fd0 0x7f4228139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 1 -- 172.21.15.190:0/3007834287 learned_addr learned my addr 172.21.15.190:0/3007834287 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 10 monclient(hunting): get_auth_request con 0x7f4228139da0 auth_method 0 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42267fc640 10 monclient(hunting): get_auth_request con 0x7f42280a4fd0 auth_method 0 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42267fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42267fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42267fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:06.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:06.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42267fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:06.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42267fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:06.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42267fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:06.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 10 monclient(hunting): handle_auth_done global_id 4713 payload 274 2024-08-19T10:40:06.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 1 -- 172.21.15.190:0/3007834287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42280a4fd0 msgr2=0x7f4228139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:06.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 1 --2- 172.21.15.190:0/3007834287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42280a4fd0 0x7f4228139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 1 -- 172.21.15.190:0/3007834287 >> v2:172.21.15.190:3300/0 conn(0x7f42280a46b0 msgr2=0x7f42280a4a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:06.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 1 --2- 172.21.15.190:0/3007834287 >> v2:172.21.15.190:3300/0 conn(0x7f42280a46b0 0x7f42280a4a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:06.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 1 monclient: found mon.noname-d 2024-08-19T10:40:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42277fe640 1 -- 172.21.15.190:0/3007834287 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42280a61a0 con 0x7f4228139da0 2024-08-19T10:40:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.736+0000 7f42267fc640 1 --2- 172.21.15.190:0/3007834287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42280a4fd0 0x7f4228139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:06.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f42277fe640 1 -- 172.21.15.190:0/3007834287 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4228139da0 msgr2=0x7f422813a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f42277fe640 1 --2- 172.21.15.190:0/3007834287 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4228139da0 0x7f422813a230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4218009870 tx=0x7f421801c440 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e567816fc72b362d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 1 -- 172.21.15.190:0/3007834287 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42180b0070 con 0x7f4228139da0 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 10 monclient: dump: 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 1 -- 172.21.15.190:0/3007834287 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42180a4e50 con 0x7f4228139da0 2024-08-19T10:40:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 1 -- 172.21.15.190:0/3007834287 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42180aac80 con 0x7f4228139da0 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f422d341640 5 monclient: authenticate success, global_id 4713 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f422d341640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f422d341640 4 set_mon_vals no callback set 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.740+0000 7f4225ffb640 10 monclient: dump: 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:06.753 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient: shutdown 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:06.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 -- 172.21.15.190:0/3007834287 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4228139da0 msgr2=0x7f422813a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 --2- 172.21.15.190:0/3007834287 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4228139da0 0x7f422813a230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4218009870 tx=0x7f421801c440 comp rx=0 tx=0).stop 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 -- 172.21.15.190:0/3007834287 shutdown_connections 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 --2- 172.21.15.190:0/3007834287 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4228139da0 0x7f422813a230 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 --2- 172.21.15.190:0/3007834287 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42280a4fd0 0x7f4228139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 --2- 172.21.15.190:0/3007834287 >> v2:172.21.15.190:3300/0 conn(0x7f42280a46b0 0x7f42280a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 -- 172.21.15.190:0/3007834287 >> 172.21.15.190:0/3007834287 conn(0x7f42280a92a0 msgr2=0x7f42280a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 -- 172.21.15.190:0/3007834287 shutdown_connections 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 -- 172.21.15.190:0/3007834287 wait complete. 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient: build_initial_monmap 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient: monmap: 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:06.755 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:06.756 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 Processor -- start 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 -- start start 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient: init 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient: _add_conns ranks=[6,5,3,4,2,7,1,0,8] 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f42280a46b0 0x7f422813a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient(hunting): picked mon.noname-d con 0x7f42280a46b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42280a4fd0 0x7f422813adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient(hunting): picked mon.noname-h con 0x7f42280a4fd0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4228139da0 0x7f422813b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient(hunting): picked mon.noname-g con 0x7f4228139da0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f42280a53b0 con 0x7f4228139da0 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f422806d120 con 0x7f42280a4fd0 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f422813dcf0 con 0x7f42280a46b0 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f422d341640 10 monclient(hunting): authenticate will time out at 1052.317745s 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4228139da0 0x7f422813b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42267fc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42280a4fd0 0x7f422813adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4228139da0 0x7f422813b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:56940/0 (socket says 172.21.15.190:56940) 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 1 -- 172.21.15.190:0/57321493 learned_addr learned my addr 172.21.15.190:0/57321493 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 10 monclient(hunting): get_auth_request con 0x7f4228139da0 auth_method 0 2024-08-19T10:40:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42267fc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42280a4fd0 0x7f422813adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54734/0 (socket says 172.21.15.190:54734) 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42267fc640 10 monclient(hunting): get_auth_request con 0x7f42280a4fd0 auth_method 0 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42267fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42267fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42267fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42267fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42267fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42267fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f4226ffd640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f42280a46b0 0x7f422813a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f4226ffd640 10 monclient(hunting): get_auth_request con 0x7f42280a46b0 auth_method 0 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f4226ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f4226ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f4226ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 10 monclient(hunting): handle_auth_done global_id 4737 payload 824 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 1 -- 172.21.15.190:0/57321493 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f42280a46b0 msgr2=0x7f422813a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f42280a46b0 0x7f422813a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 1 -- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42280a4fd0 msgr2=0x7f422813adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42280a4fd0 0x7f422813adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 1 monclient: found mon.noname-g 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:06.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42277fe640 1 -- 172.21.15.190:0/57321493 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422813dfd0 con 0x7f4228139da0 2024-08-19T10:40:06.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.744+0000 7f42267fc640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42280a4fd0 0x7f422813adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:06.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f42277fe640 1 -- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4228139da0 msgr2=0x7f422813b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f42277fe640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4228139da0 0x7f422813b2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f421801cce0 tx=0x7f42180191e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 1 -- 172.21.15.190:0/57321493 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42180b0070 con 0x7f4228139da0 2024-08-19T10:40:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 10 monclient: dump: 2024-08-19T10:40:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 1 -- 172.21.15.190:0/57321493 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42180ba070 con 0x7f4228139da0 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f422d341640 5 monclient: authenticate success, global_id 4737 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f422d341640 10 monclient: _renew_subs 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f422d341640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f422d341640 1 -- 172.21.15.190:0/57321493 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422813fe90 con 0x7f4228139da0 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 1 -- 172.21.15.190:0/57321493 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42180b58b0 con 0x7f4228139da0 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 10 monclient: dump: 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f42257fa640 4 set_mon_vals no callback set 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f422d341640 10 monclient: _renew_subs 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f422d341640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f422d341640 1 -- 172.21.15.190:0/57321493 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f422813e2b0 con 0x7f4228139da0 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f422d341640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f422d341640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f422d341640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f422d341640 1 -- 172.21.15.190:0/57321493 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42280a4a90 con 0x7f4228139da0 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 1 -- 172.21.15.190:0/57321493 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f42180b5440 con 0x7f4228139da0 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f421403be00 0x7f421403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f420f7fe640 1 -- 172.21.15.190:0/57321493 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 4834+0+0 (secure 0 0 0) 0x7f42180e8770 con 0x7f4228139da0 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f4226ffd640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f421403be00 0x7f421403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.748+0000 7f4226ffd640 10 monclient: get_auth_request con 0x7f421403be00 auth_method 0 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.752+0000 7f4226ffd640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f421403be00 0x7f421403e2c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4220006fd0 tx=0x7f4220008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.752+0000 7f420f7fe640 1 -- 172.21.15.190:0/57321493 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4218016150 con 0x7f4228139da0 2024-08-19T10:40:06.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.752+0000 7f420f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:06.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.752+0000 7f420f7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:06.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.912+0000 7f422d341640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:06.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.912+0000 7f422d341640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:06.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.912+0000 7f422d341640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:06.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.912+0000 7f422d341640 1 -- 172.21.15.190:0/57321493 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) -- 0x7f422813f1b0 con 0x7f4228139da0 2024-08-19T10:40:06.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.912+0000 7f420f7fe640 1 -- 172.21.15.190:0/57321493 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v55) ==== 90+0+8 (secure 0 0 0) 0x7f42180aad70 con 0x7f4228139da0 2024-08-19T10:40:06.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.912+0000 7f420f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:06.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:06.912+0000 7f420f7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:07.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.008+0000 7f422d341640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:07.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.008+0000 7f422d341640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:07.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.008+0000 7f422d341640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:07.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.008+0000 7f422d341640 1 -- 172.21.15.190:0/57321493 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule ls-by-class", "class": "hdd"} v 0) -- 0x7f422813fa40 con 0x7f4228139da0 2024-08-19T10:40:07.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.008+0000 7f420f7fe640 1 -- 172.21.15.190:0/57321493 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls-by-class", "class": "hdd"}]=0 v55) ==== 90+0+8 (secure 0 0 0) 0x7f42180b4c90 con 0x7f4228139da0 2024-08-19T10:40:07.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.008+0000 7f420f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls-by-class", "class": "hdd"}] 2024-08-19T10:40:07.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.008+0000 7f420f7fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:40:07.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 -- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f421403be00 msgr2=0x7f421403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:07.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f421403be00 0x7f421403e2c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4220006fd0 tx=0x7f4220008040 comp rx=0 tx=0).stop 2024-08-19T10:40:07.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 10 monclient: shutdown 2024-08-19T10:40:07.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:07.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 -- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4228139da0 msgr2=0x7f422813b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:07.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4228139da0 0x7f422813b2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f421801cce0 tx=0x7f42180191e0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 -- 172.21.15.190:0/57321493 shutdown_connections 2024-08-19T10:40:07.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f421403be00 0x7f421403e2c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4228139da0 0x7f422813b2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42280a4fd0 0x7f422813adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 --2- 172.21.15.190:0/57321493 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f42280a46b0 0x7f422813a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 -- 172.21.15.190:0/57321493 >> 172.21.15.190:0/57321493 conn(0x7f42280a92a0 msgr2=0x7f42280a8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:07.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 -- 172.21.15.190:0/57321493 shutdown_connections 2024-08-19T10:40:07.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.012+0000 7f422d341640 1 -- 172.21.15.190:0/57321493 wait complete. 2024-08-19T10:40:07.029 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:40:07.029 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd erasure-code-profile set ec-foo-ssd crush-device-class=ssd m=2 k=2 2024-08-19T10:40:07.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.140+0000 7f9062dbc640 10 monclient: get_monmap_and_config 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.140+0000 7f9062dbc640 10 monclient: build_initial_monmap 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.140+0000 7f9062dbc640 10 monclient: monmap: 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:07.147283+0000 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:07.147283+0000 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:07.147 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:07.148 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:07.148 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:07.148 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:07.148 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:07.148 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:07.148 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:07.148 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:07.148 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:07.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 1 Processor -- start 2024-08-19T10:40:07.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 1 -- start start 2024-08-19T10:40:07.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 10 monclient: init 2024-08-19T10:40:07.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:07.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 10 monclient: _add_conns ranks=[5,7,2,0,8,10,3,6,1,9,4] 2024-08-19T10:40:07.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a2080 0x7f905c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:07.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 10 monclient(hunting): picked mon.noname-h con 0x7f905c0a2080 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:07.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f905c0a1700 0x7f905c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:07.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 10 monclient(hunting): picked mon.noname-i con 0x7f905c0a1700 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:07.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f905c09fb70 0x7f905c06ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:07.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 10 monclient(hunting): picked mon.noname-b con 0x7f905c09fb70 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:07.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:07.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f905c09f9c0 con 0x7f905c09fb70 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f905c0a29a0 con 0x7f905c0a2080 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f905c0a0510 con 0x7f905c0a1700 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9062dbc640 10 monclient(hunting): authenticate will time out at 1052.717532s 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f905bfff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f905c0a1700 msgr2=0x7f905c06c8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f905bfff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f905c0a1700 0x7f905c06c8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a2080 0x7f905c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a2080 0x7f905c0a2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54750/0 (socket says 172.21.15.190:54750) 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 1 -- 172.21.15.190:0/4096324093 learned_addr learned my addr 172.21.15.190:0/4096324093 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 10 monclient(hunting): get_auth_request con 0x7f905c0a2080 auth_method 0 2024-08-19T10:40:07.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 10 monclient(hunting): handle_auth_done global_id 4784 payload 274 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 1 -- 172.21.15.190:0/4096324093 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f905c0a1700 msgr2=0x7f905c06c8d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 1 --2- 172.21.15.190:0/4096324093 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f905c0a1700 0x7f905c06c8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 1 -- 172.21.15.190:0/4096324093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f905c09fb70 msgr2=0x7f905c06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 1 --2- 172.21.15.190:0/4096324093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f905c09fb70 0x7f905c06ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 10 monclient: _finish_hunting 0 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 1 monclient: found mon.noname-h 2024-08-19T10:40:07.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:07.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:07.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.144+0000 7f9060b58640 1 -- 172.21.15.190:0/4096324093 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c0a07f0 con 0x7f905c0a2080 2024-08-19T10:40:07.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9060b58640 1 -- 172.21.15.190:0/4096324093 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a2080 msgr2=0x7f905c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:07.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9060b58640 1 --2- 172.21.15.190:0/4096324093 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a2080 0x7f905c0a2460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f904c009e80 tx=0x7f904c01c920 comp rx=0 tx=0).ready entity=mon.5 client_cookie=e99fbe3a46886c8a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:07.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 1 -- 172.21.15.190:0/4096324093 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f904c0091b0 con 0x7f905c0a2080 2024-08-19T10:40:07.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 10 monclient: dump: 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:07.159 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 1 -- 172.21.15.190:0/4096324093 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904c0a5a80 con 0x7f905c0a2080 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 1 -- 172.21.15.190:0/4096324093 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f904c0a5c30 con 0x7f905c0a2080 2024-08-19T10:40:07.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 5 monclient: authenticate success, global_id 4784 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 4 set_mon_vals no callback set 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f905b7fe640 10 monclient: dump: 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:07.161 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 10 monclient: shutdown 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 1 -- 172.21.15.190:0/4096324093 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a2080 msgr2=0x7f905c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 1 --2- 172.21.15.190:0/4096324093 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a2080 0x7f905c0a2460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f904c009e80 tx=0x7f904c01c920 comp rx=0 tx=0).stop 2024-08-19T10:40:07.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 1 -- 172.21.15.190:0/4096324093 shutdown_connections 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 1 --2- 172.21.15.190:0/4096324093 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f905c09fb70 0x7f905c06ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 1 --2- 172.21.15.190:0/4096324093 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f905c0a1700 0x7f905c06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 1 --2- 172.21.15.190:0/4096324093 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a2080 0x7f905c0a2460 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 1 -- 172.21.15.190:0/4096324093 >> 172.21.15.190:0/4096324093 conn(0x7f905c0a8450 msgr2=0x7f905c0a8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 1 -- 172.21.15.190:0/4096324093 shutdown_connections 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 1 -- 172.21.15.190:0/4096324093 wait complete. 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 10 monclient: build_initial_monmap 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.148+0000 7f9062dbc640 10 monclient: monmap: 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:07.163 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 1 Processor -- start 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 1 -- start start 2024-08-19T10:40:07.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient: init 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient: _add_conns ranks=[8,5,2,4,0,1,6,7,3] 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f905c09fb70 0x7f905c1d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient(hunting): picked mon.noname-e con 0x7f905c09fb70 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a1700 0x7f905c09ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient(hunting): picked mon.noname-h con 0x7f905c0a1700 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f905c0a2080 0x7f905c071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient(hunting): picked mon.noname-b con 0x7f905c0a2080 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f905c142fb0 con 0x7f905c0a2080 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f905c142e30 con 0x7f905c0a1700 2024-08-19T10:40:07.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:07.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f905c143130 con 0x7f905c09fb70 2024-08-19T10:40:07.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:07.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient(hunting): authenticate will time out at 1052.724596s 2024-08-19T10:40:07.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a1700 0x7f905c09ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:07.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a1700 0x7f905c09ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54752/0 (socket says 172.21.15.190:54752) 2024-08-19T10:40:07.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 1 -- 172.21.15.190:0/36200356 learned_addr learned my addr 172.21.15.190:0/36200356 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:07.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 10 monclient(hunting): get_auth_request con 0x7f905c0a1700 auth_method 0 2024-08-19T10:40:07.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:07.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:07.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9060b58640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f905c09fb70 0x7f905c1d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 10 monclient(hunting): handle_auth_done global_id 4793 payload 824 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 1 -- 172.21.15.190:0/36200356 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f905c09fb70 msgr2=0x7f905c1d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f905c09fb70 0x7f905c1d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 1 -- 172.21.15.190:0/36200356 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f905c0a2080 msgr2=0x7f905c071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9061359640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f905c0a2080 0x7f905c071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f905c0a2080 0x7f905c071310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 1 monclient: found mon.noname-h 2024-08-19T10:40:07.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 1 -- 172.21.15.190:0/36200356 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c073e40 con 0x7f905c0a1700 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 1 -- 172.21.15.190:0/36200356 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a1700 msgr2=0x7f905c09ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905bfff640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a1700 0x7f905c09ab20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f905000b580 tx=0x7f905000ba40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 1 -- 172.21.15.190:0/36200356 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9050016070 con 0x7f905c0a1700 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 10 monclient: dump: 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:07.168 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 10 monclient: _finish_auth 0 2024-08-19T10:40:07.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 10 monclient: _check_auth_tickets 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 1 -- 172.21.15.190:0/36200356 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9050003de0 con 0x7f905c0a1700 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 5 monclient: authenticate success, global_id 4793 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient: _renew_subs 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 1 -- 172.21.15.190:0/36200356 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905c074a10 con 0x7f905c0a1700 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 1 -- 172.21.15.190:0/36200356 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f905000f3e0 con 0x7f905c0a1700 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f90597fa640 10 monclient: dump: 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:07.170 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:07.171 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:07.172 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:07.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient: _renew_subs 2024-08-19T10:40:07.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f905affd640 4 set_mon_vals no callback set 2024-08-19T10:40:07.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:07.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.152+0000 7f9062dbc640 1 -- 172.21.15.190:0/36200356 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f905c0737d0 con 0x7f905c0a1700 2024-08-19T10:40:07.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.156+0000 7f9062dbc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:07.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.156+0000 7f9062dbc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:07.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.156+0000 7f9062dbc640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:07.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.156+0000 7f9062dbc640 1 -- 172.21.15.190:0/36200356 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905c06de00 con 0x7f905c0a1700 2024-08-19T10:40:07.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.156+0000 7f90597fa640 1 -- 172.21.15.190:0/36200356 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f90500050d0 con 0x7f905c0a1700 2024-08-19T10:40:07.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.156+0000 7f90597fa640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f904803be30 0x7f904803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:07.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.156+0000 7f90597fa640 1 -- 172.21.15.190:0/36200356 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(55..55 src has 1..55) ==== 4834+0+0 (secure 0 0 0) 0x7f905004dd70 con 0x7f905c0a1700 2024-08-19T10:40:07.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.156+0000 7f9060b58640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f904803be30 0x7f904803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:07.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.156+0000 7f9060b58640 10 monclient: get_auth_request con 0x7f904803be30 auth_method 0 2024-08-19T10:40:07.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.156+0000 7f9060b58640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f904803be30 0x7f904803e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f904c009e50 tx=0x7f904c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:07.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.160+0000 7f90597fa640 1 -- 172.21.15.190:0/36200356 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f905c073e40 con 0x7f905c0a1700 2024-08-19T10:40:07.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.160+0000 7f90597fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:07.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.160+0000 7f90597fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:07.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.320+0000 7f9062dbc640 10 start_mon_command cmd=[{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-08-19T10:40:07.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.320+0000 7f9062dbc640 10 monclient: _send_command 2 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-08-19T10:40:07.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.320+0000 7f9062dbc640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:07.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.320+0000 7f9062dbc640 1 -- 172.21.15.190:0/36200356 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]} v 0) -- 0x7f905c0a4320 con 0x7f905c0a1700 2024-08-19T10:40:07.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.984+0000 7f90597fa640 1 -- 172.21.15.190:0/36200356 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}]=0 v56) ==== 151+0+0 (secure 0 0 0) 0x7f905000f640 con 0x7f905c0a1700 2024-08-19T10:40:07.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.984+0000 7f90597fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-08-19T10:40:07.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:07.984+0000 7f90597fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:08.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.092+0000 7f9062dbc640 10 start_mon_command cmd=[{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-08-19T10:40:08.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.092+0000 7f9062dbc640 10 monclient: _send_command 3 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-08-19T10:40:08.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.092+0000 7f9062dbc640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:08.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.092+0000 7f9062dbc640 1 -- 172.21.15.190:0/36200356 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]} v 0) -- 0x7f905c13da10 con 0x7f905c0a1700 2024-08-19T10:40:08.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.096+0000 7f90597fa640 1 -- 172.21.15.190:0/36200356 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}]=0 v56) ==== 151+0+0 (secure 0 0 0) 0x7f9050018310 con 0x7f905c0a1700 2024-08-19T10:40:08.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.096+0000 7f90597fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd erasure-code-profile set", "name": "ec-foo-ssd", "profile": ["crush-device-class=ssd", "m=2", "k=2"]}] 2024-08-19T10:40:08.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.096+0000 7f90597fa640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:40:08.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 -- 172.21.15.190:0/36200356 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f904803be30 msgr2=0x7f904803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:08.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f904803be30 0x7f904803e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f904c009e50 tx=0x7f904c0a7070 comp rx=0 tx=0).stop 2024-08-19T10:40:08.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 10 monclient: shutdown 2024-08-19T10:40:08.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:08.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 -- 172.21.15.190:0/36200356 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a1700 msgr2=0x7f905c09ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:08.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a1700 0x7f905c09ab20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f905000b580 tx=0x7f905000ba40 comp rx=0 tx=0).stop 2024-08-19T10:40:08.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 -- 172.21.15.190:0/36200356 shutdown_connections 2024-08-19T10:40:08.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f905c0a2080 0x7f905c071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:08.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f905c0a1700 0x7f905c09ab20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:08.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f904803be30 0x7f904803e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:08.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 --2- 172.21.15.190:0/36200356 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f905c09fb70 0x7f905c1d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:08.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 -- 172.21.15.190:0/36200356 >> 172.21.15.190:0/36200356 conn(0x7f905c0a8450 msgr2=0x7f905c13af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:08.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 -- 172.21.15.190:0/36200356 shutdown_connections 2024-08-19T10:40:08.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.100+0000 7f9062dbc640 1 -- 172.21.15.190:0/36200356 wait complete. 2024-08-19T10:40:08.118 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create ec-foo 2 erasure ec-foo-ssd 2024-08-19T10:40:08.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient: get_monmap_and_config 2024-08-19T10:40:08.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient: build_initial_monmap 2024-08-19T10:40:08.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient: monmap: 2024-08-19T10:40:08.241 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:08.241 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:08.241 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:08.241071+0000 2024-08-19T10:40:08.241 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:08.241071+0000 2024-08-19T10:40:08.241 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:08.241 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:08.241 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:08.241 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:08.241 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:08.241 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:08.242 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:08.242 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:08.242 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:08.242 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:08.242 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:08.242 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:08.242 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:08.242 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:08.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 1 Processor -- start 2024-08-19T10:40:08.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 1 -- start start 2024-08-19T10:40:08.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient: init 2024-08-19T10:40:08.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:08.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient: _add_conns ranks=[3,2,8,9,10,1,0,6,5,4,7] 2024-08-19T10:40:08.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf9c1396a0 0x7fcf9c139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient(hunting): picked mon.noname-g con 0x7fcf9c1396a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf9c139fc0 0x7fcf9c0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient(hunting): picked mon.noname-b con 0x7fcf9c139fc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcf9c0a6e20 0x7fcf9c0a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient(hunting): picked mon.noname-e con 0x7fcf9c0a6e20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9c0a4f20 con 0x7fcf9c139fc0 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9c0a83e0 con 0x7fcf9c1396a0 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fcf9c0a2f30 con 0x7fcf9c0a6e20 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa2a7c640 10 monclient(hunting): authenticate will time out at 1053.810978s 2024-08-19T10:40:08.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf9c1396a0 0x7fcf9c139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf9c1396a0 0x7fcf9c139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:56956/0 (socket says 172.21.15.190:56956) 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 1 -- 172.21.15.190:0/2377935879 learned_addr learned my addr 172.21.15.190:0/2377935879 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 10 monclient(hunting): get_auth_request con 0x7fcf9c1396a0 auth_method 0 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa1019640 1 --2- 172.21.15.190:0/2377935879 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcf9c0a6e20 0x7fcf9c0a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcf9bfff640 1 --2- 172.21.15.190:0/2377935879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf9c139fc0 0x7fcf9c0a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:08.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa1019640 10 monclient(hunting): get_auth_request con 0x7fcf9c0a6e20 auth_method 0 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa1019640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa1019640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa1019640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 10 monclient(hunting): handle_auth_done global_id 4746 payload 274 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 1 -- 172.21.15.190:0/2377935879 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcf9c0a6e20 msgr2=0x7fcf9c0a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 1 --2- 172.21.15.190:0/2377935879 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcf9c0a6e20 0x7fcf9c0a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 1 -- 172.21.15.190:0/2377935879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf9c139fc0 msgr2=0x7fcf9c0a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 1 --2- 172.21.15.190:0/2377935879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf9c139fc0 0x7fcf9c0a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 10 monclient: _finish_hunting 0 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 1 monclient: found mon.noname-g 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcfa0818640 1 -- 172.21.15.190:0/2377935879 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c0a35e0 con 0x7fcf9c1396a0 2024-08-19T10:40:08.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcf9bfff640 10 monclient: get_auth_request con 0x7fcf9c139fc0 auth_method 0 2024-08-19T10:40:08.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.236+0000 7fcf9bfff640 1 --2- 172.21.15.190:0/2377935879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf9c139fc0 0x7fcf9c0a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:08.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.240+0000 7fcfa0818640 1 -- 172.21.15.190:0/2377935879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf9c1396a0 msgr2=0x7fcf9c139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:08.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.240+0000 7fcfa0818640 1 --2- 172.21.15.190:0/2377935879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf9c1396a0 0x7fcf9c139a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0098d0 tx=0x7fcf8c01c440 comp rx=0 tx=0).ready entity=mon.3 client_cookie=7463660081ef7d46 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:08.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 1 -- 172.21.15.190:0/2377935879 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcf8c0b0070 con 0x7fcf9c1396a0 2024-08-19T10:40:08.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:08.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:08.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 10 monclient: dump: 2024-08-19T10:40:08.251 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:08.251 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:08.251 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:08.252 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 1 -- 172.21.15.190:0/2377935879 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf8c0aa810 con 0x7fcf9c1396a0 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 5 monclient: authenticate success, global_id 4746 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 1 -- 172.21.15.190:0/2377935879 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcf8c0a9300 con 0x7fcf9c1396a0 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 4 set_mon_vals no callback set 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9b7fe640 10 monclient: dump: 2024-08-19T10:40:08.253 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:08.254 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient: shutdown 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/2377935879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf9c1396a0 msgr2=0x7fcf9c139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 --2- 172.21.15.190:0/2377935879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf9c1396a0 0x7fcf9c139a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0098d0 tx=0x7fcf8c01c440 comp rx=0 tx=0).stop 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/2377935879 shutdown_connections 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 --2- 172.21.15.190:0/2377935879 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcf9c0a6e20 0x7fcf9c0a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 --2- 172.21.15.190:0/2377935879 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcf9c139fc0 0x7fcf9c0a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 --2- 172.21.15.190:0/2377935879 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf9c1396a0 0x7fcf9c139a80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/2377935879 >> 172.21.15.190:0/2377935879 conn(0x7fcf9c0abee0 msgr2=0x7fcf9c0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/2377935879 shutdown_connections 2024-08-19T10:40:08.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/2377935879 wait complete. 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient: build_initial_monmap 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient: monmap: 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:08.256 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 Processor -- start 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 -- start start 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient: init 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient: _add_conns ranks=[4,5,7,8,1,3,2,6,0] 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf9c0a6e20 0x7fcf9c134970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient(hunting): picked mon.noname-c con 0x7fcf9c0a6e20 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcf9c1396a0 0x7fcf9c134eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient(hunting): picked mon.noname-h con 0x7fcf9c1396a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:08.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcf9c139fc0 0x7fcf9c09ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient(hunting): picked mon.noname-i con 0x7fcf9c139fc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fcf9c13a3a0 con 0x7fcf9c0a6e20 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fcf9c143390 con 0x7fcf9c1396a0 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fcf9c0ac5f0 con 0x7fcf9c139fc0 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa2a7c640 10 monclient(hunting): authenticate will time out at 1053.818157s 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa1019640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcf9c139fc0 msgr2=0x7fcf9c09ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa1019640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcf9c139fc0 0x7fcf9c09ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcf9c1396a0 0x7fcf9c134eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:08.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcf9c1396a0 0x7fcf9c134eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54766/0 (socket says 172.21.15.190:54766) 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 1 -- 172.21.15.190:0/1549811667 learned_addr learned my addr 172.21.15.190:0/1549811667 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 10 monclient(hunting): get_auth_request con 0x7fcf9c1396a0 auth_method 0 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa0818640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf9c0a6e20 0x7fcf9c134970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa0818640 10 monclient(hunting): get_auth_request con 0x7fcf9c0a6e20 auth_method 0 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa0818640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa0818640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcfa0818640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:08.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 10 monclient(hunting): handle_auth_done global_id 4802 payload 824 2024-08-19T10:40:08.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 1 -- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcf9c139fc0 msgr2=0x7fcf9c09ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:08.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcf9c139fc0 0x7fcf9c09ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:08.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 1 -- 172.21.15.190:0/1549811667 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf9c0a6e20 msgr2=0x7fcf9c134970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:08.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf9c0a6e20 0x7fcf9c134970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:08.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:08.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 1 monclient: found mon.noname-h 2024-08-19T10:40:08.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:08.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:08.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.244+0000 7fcf9bfff640 1 -- 172.21.15.190:0/1549811667 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c140940 con 0x7fcf9c1396a0 2024-08-19T10:40:08.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf9bfff640 1 -- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcf9c1396a0 msgr2=0x7fcf9c134eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:08.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf9bfff640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcf9c1396a0 0x7fcf9c134eb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000c7f0 tx=0x7fcf9000ccb0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:08.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 1 -- 172.21.15.190:0/1549811667 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcf90015070 con 0x7fcf9c1396a0 2024-08-19T10:40:08.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:08.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:08.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 10 monclient: dump: 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:08.263 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 10 monclient: _finish_auth 0 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 10 monclient: _check_auth_tickets 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 1 -- 172.21.15.190:0/1549811667 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf90007c40 con 0x7fcf9c1396a0 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa2a7c640 5 monclient: authenticate success, global_id 4802 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa2a7c640 10 monclient: _renew_subs 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa2a7c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/1549811667 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c0a0750 con 0x7fcf9c1396a0 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 1 -- 172.21.15.190:0/1549811667 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcf90007df0 con 0x7fcf9c1396a0 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:08.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf9affd640 4 set_mon_vals no callback set 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 10 monclient: dump: 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:08.265 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa2a7c640 10 monclient: _renew_subs 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa2a7c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/1549811667 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c0a1130 con 0x7fcf9c1396a0 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa2a7c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa2a7c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa2a7c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/1549811667 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9c0ac7f0 con 0x7fcf9c1396a0 2024-08-19T10:40:08.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 1 -- 172.21.15.190:0/1549811667 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fcf90018070 con 0x7fcf9c1396a0 2024-08-19T10:40:08.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcf8803be00 0x7fcf8803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:08.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcf997fa640 1 -- 172.21.15.190:0/1549811667 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(56..56 src has 1..56) ==== 5131+0+0 (secure 0 0 0) 0x7fcf9004e9d0 con 0x7fcf9c1396a0 2024-08-19T10:40:08.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa0818640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcf8803be00 0x7fcf8803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:08.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.248+0000 7fcfa0818640 10 monclient: get_auth_request con 0x7fcf8803be00 auth_method 0 2024-08-19T10:40:08.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.252+0000 7fcfa0818640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcf8803be00 0x7fcf8803e2c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0098a0 tx=0x7fcf8c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:08.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.252+0000 7fcf997fa640 1 -- 172.21.15.190:0/1549811667 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fcf9c0ac7f0 con 0x7fcf9c1396a0 2024-08-19T10:40:08.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.252+0000 7fcf997fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:08.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.252+0000 7fcf997fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:08.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.420+0000 7fcfa2a7c640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-08-19T10:40:08.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.420+0000 7fcfa2a7c640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-08-19T10:40:08.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.420+0000 7fcfa2a7c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:08.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:08.420+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/1549811667 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"} v 0) -- 0x7fcf9c139a80 con 0x7fcf9c1396a0 2024-08-19T10:40:10.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.008+0000 7fcf997fa640 1 -- 172.21.15.190:0/1549811667 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}]=0 pool 'ec-foo' created v58) ==== 177+0+0 (secure 0 0 0) 0x7fcf90022d90 con 0x7fcf9c1396a0 2024-08-19T10:40:10.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.008+0000 7fcf997fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-08-19T10:40:10.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.008+0000 7fcf997fa640 10 monclient: _finish_command 2 = system:0 pool 'ec-foo' created 2024-08-19T10:40:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.112+0000 7fcfa2a7c640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-08-19T10:40:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.112+0000 7fcfa2a7c640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-08-19T10:40:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.112+0000 7fcfa2a7c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.112+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/1549811667 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"} v 0) -- 0x7fcf9c0008e0 con 0x7fcf9c1396a0 2024-08-19T10:40:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcf997fa640 1 -- 172.21.15.190:0/1549811667 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}]=0 pool 'ec-foo' already exists v58) ==== 184+0+0 (secure 0 0 0) 0x7fcf900113f0 con 0x7fcf9c1396a0 2024-08-19T10:40:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcf997fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "ec-foo", "pg_num": 2, "pool_type": "erasure", "erasure_code_profile": "ec-foo-ssd"}] 2024-08-19T10:40:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcf997fa640 10 monclient: _finish_command 3 = system:0 pool 'ec-foo' already exists 2024-08-19T10:40:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'ec-foo' already exists 2024-08-19T10:40:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcf8803be00 msgr2=0x7fcf8803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcf8803be00 0x7fcf8803e2c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0098a0 tx=0x7fcf8c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:40:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 10 monclient: shutdown 2024-08-19T10:40:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcf9c1396a0 msgr2=0x7fcf9c134eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcf9c1396a0 0x7fcf9c134eb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000c7f0 tx=0x7fcf9000ccb0 comp rx=0 tx=0).stop 2024-08-19T10:40:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/1549811667 shutdown_connections 2024-08-19T10:40:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcf8803be00 0x7fcf8803e2c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcf9c139fc0 0x7fcf9c09ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcf9c1396a0 0x7fcf9c134eb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 --2- 172.21.15.190:0/1549811667 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcf9c0a6e20 0x7fcf9c134970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/1549811667 >> 172.21.15.190:0/1549811667 conn(0x7fcf9c0abee0 msgr2=0x7fcf9c1379a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/1549811667 shutdown_connections 2024-08-19T10:40:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.116+0000 7fcfa2a7c640 1 -- 172.21.15.190:0/1549811667 wait complete. 2024-08-19T10:40:10.137 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool rm ec-foo ec-foo --yes-i-really-really-mean-it 2024-08-19T10:40:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient: get_monmap_and_config 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient: build_initial_monmap 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient: monmap: 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:10.264310+0000 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:10.264310+0000 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:10.265 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:10.265 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:10.265 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:10.265 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:10.265 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:10.265 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:10.265 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:10.265 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:10.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 1 Processor -- start 2024-08-19T10:40:10.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 1 -- start start 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient: init 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient: _add_conns ranks=[0,3,9,5,4,7,6,2,10,1,8] 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcb741396a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb7413a190 0x7fcb7413eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient(hunting): picked mon.noname-g con 0x7fcb7413a190 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb7413f0a0 0x7fcb74141490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient(hunting): picked mon.noname-a con 0x7fcb7413f0a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb740a83e0 con 0x7fcb741396a0 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fcb740a4f20 con 0x7fcb7413f0a0 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcb740a2f30 con 0x7fcb7413a190 2024-08-19T10:40:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb7b3ca640 10 monclient(hunting): authenticate will time out at 1055.834188s 2024-08-19T10:40:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb7413a190 0x7fcb7413eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb7413a190 0x7fcb7413eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:56958/0 (socket says 172.21.15.190:56958) 2024-08-19T10:40:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 1 -- 172.21.15.190:0/1421232884 learned_addr learned my addr 172.21.15.190:0/1421232884 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 10 monclient(hunting): get_auth_request con 0x7fcb7413a190 auth_method 0 2024-08-19T10:40:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 10 monclient(hunting): handle_auth_done global_id 4755 payload 274 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb79967640 1 --2- 172.21.15.190:0/1421232884 >> v2:172.21.15.89:3300/0 conn(0x7fcb7413f0a0 0x7fcb74141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 1 -- 172.21.15.190:0/1421232884 >> v2:172.21.15.89:3300/0 conn(0x7fcb7413f0a0 msgr2=0x7fcb74141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 1 --2- 172.21.15.190:0/1421232884 >> v2:172.21.15.89:3300/0 conn(0x7fcb7413f0a0 0x7fcb74141490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 1 -- 172.21.15.190:0/1421232884 >> v1:172.21.15.89:6789/0 conn(0x7fcb741396a0 legacy=0x7fcb74139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 10 monclient: _finish_hunting 0 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 1 monclient: found mon.noname-g 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.260+0000 7fcb78965640 1 -- 172.21.15.190:0/1421232884 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb740a35e0 con 0x7fcb7413a190 2024-08-19T10:40:10.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb78965640 1 -- 172.21.15.190:0/1421232884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb7413a190 msgr2=0x7fcb7413eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:10.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb78965640 1 --2- 172.21.15.190:0/1421232884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb7413a190 0x7fcb7413eb60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009870 tx=0x7fcb7001c420 comp rx=0 tx=0).ready entity=mon.3 client_cookie=bf201eb82f041c12 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:10.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 1 -- 172.21.15.190:0/1421232884 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb700b0070 con 0x7fcb7413a190 2024-08-19T10:40:10.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:10.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:10.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 10 monclient: dump: 2024-08-19T10:40:10.275 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:10.275 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:10.275 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:10.275 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:10.275 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:10.275 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:10.275 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:10.275 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:10.275 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:10.275 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:10.276 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:10.276 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:10.276 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:10.276 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:10.276 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:10.276 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:10.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 10 monclient: _finish_auth 0 2024-08-19T10:40:10.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:10.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:10.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 1 -- 172.21.15.190:0/1421232884 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb700aa810 con 0x7fcb7413a190 2024-08-19T10:40:10.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb7b3ca640 5 monclient: authenticate success, global_id 4755 2024-08-19T10:40:10.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb7b3ca640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:10.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:10.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb7b3ca640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:10.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 1 -- 172.21.15.190:0/1421232884 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb700a93e0 con 0x7fcb7413a190 2024-08-19T10:40:10.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:10.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb7b3ca640 4 set_mon_vals no callback set 2024-08-19T10:40:10.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:10.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.264+0000 7fcb6bfff640 10 monclient: dump: 2024-08-19T10:40:10.277 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:10.277 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:10.278 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient: shutdown 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1421232884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb7413a190 msgr2=0x7fcb7413eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 --2- 172.21.15.190:0/1421232884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb7413a190 0x7fcb7413eb60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009870 tx=0x7fcb7001c420 comp rx=0 tx=0).stop 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1421232884 shutdown_connections 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 --2- 172.21.15.190:0/1421232884 >> v2:172.21.15.89:3300/0 conn(0x7fcb7413f0a0 0x7fcb74141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 --2- 172.21.15.190:0/1421232884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb7413a190 0x7fcb7413eb60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1421232884 >> 172.21.15.190:0/1421232884 conn(0x7fcb740abee0 msgr2=0x7fcb740ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1421232884 shutdown_connections 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1421232884 wait complete. 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient: build_initial_monmap 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient: monmap: 2024-08-19T10:40:10.279 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:10.280 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:10.280 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:10.280 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:10.280 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:10.280 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:10.280 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:10.280 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:10.280 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:10.280 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:10.280 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 Processor -- start 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 -- start start 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient: init 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient: _add_conns ranks=[7,2,5,3,6,4,8,1,0] 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcb741396a0 0x7fcb741338a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient(hunting): picked mon.noname-i con 0x7fcb741396a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:10.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7413a190 0x7fcb74133de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient(hunting): picked mon.noname-b con 0x7fcb7413a190 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb7413f0a0 0x7fcb7413c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient(hunting): picked mon.noname-h con 0x7fcb7413f0a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fcb741441f0 con 0x7fcb7413a190 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fcb740a78f0 con 0x7fcb7413f0a0 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fcb74144370 con 0x7fcb741396a0 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb7b3ca640 10 monclient(hunting): authenticate will time out at 1055.842083s 2024-08-19T10:40:10.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79166640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcb741396a0 msgr2=0x7fcb741338a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79166640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcb741396a0 0x7fcb741338a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb7413f0a0 0x7fcb7413c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb7413f0a0 0x7fcb7413c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54774/0 (socket says 172.21.15.190:54774) 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 1 -- 172.21.15.190:0/1493283579 learned_addr learned my addr 172.21.15.190:0/1493283579 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 10 monclient(hunting): get_auth_request con 0x7fcb7413f0a0 auth_method 0 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb78965640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7413a190 0x7fcb74133de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 10 monclient(hunting): handle_auth_done global_id 4811 payload 824 2024-08-19T10:40:10.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 1 -- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcb741396a0 msgr2=0x7fcb741338a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:10.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcb741396a0 0x7fcb741338a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:10.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 1 -- 172.21.15.190:0/1493283579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7413a190 msgr2=0x7fcb74133de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:10.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7413a190 0x7fcb74133de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:10.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 10 monclient: _finish_hunting 0 2024-08-19T10:40:10.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 1 monclient: found mon.noname-h 2024-08-19T10:40:10.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:10.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:10.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.268+0000 7fcb79967640 1 -- 172.21.15.190:0/1493283579 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb740ac3e0 con 0x7fcb7413f0a0 2024-08-19T10:40:10.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb79967640 1 -- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb7413f0a0 msgr2=0x7fcb7413c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb79967640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb7413f0a0 0x7fcb7413c8a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00cc30 tx=0x7fcb6c00a5c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 1 -- 172.21.15.190:0/1493283579 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb6c00ce40 con 0x7fcb7413f0a0 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 10 monclient: dump: 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:10.287 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:10.288 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:10.288 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:10.288 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:10.288 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:10.288 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:10.288 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:10.288 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:10.288 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:10.288 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 1 -- 172.21.15.190:0/1493283579 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb6c00fa70 con 0x7fcb7413f0a0 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 1 -- 172.21.15.190:0/1493283579 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb6c00fc20 con 0x7fcb7413f0a0 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb7b3ca640 5 monclient: authenticate success, global_id 4811 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 10 monclient: dump: 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:10.289 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:10.290 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb6b7fe640 4 set_mon_vals no callback set 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb7b3ca640 10 monclient: _renew_subs 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb7b3ca640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1493283579 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb740a7030 con 0x7fcb7413f0a0 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb7b3ca640 10 monclient: _renew_subs 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb7b3ca640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1493283579 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7409e4e0 con 0x7fcb7413f0a0 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb7b3ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb7b3ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb7b3ca640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1493283579 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb74139a80 con 0x7fcb7413f0a0 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 1 -- 172.21.15.190:0/1493283579 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fcb6c00f600 con 0x7fcb7413f0a0 2024-08-19T10:40:10.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcb5803be00 0x7fcb5803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:10.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb69ffb640 1 -- 172.21.15.190:0/1493283579 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(59..59 src has 1..59) ==== 5572+0+0 (secure 0 0 0) 0x7fcb6c009030 con 0x7fcb7413f0a0 2024-08-19T10:40:10.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb79166640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcb5803be00 0x7fcb5803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:10.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.272+0000 7fcb79166640 10 monclient: get_auth_request con 0x7fcb5803be00 auth_method 0 2024-08-19T10:40:10.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.276+0000 7fcb79166640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcb5803be00 0x7fcb5803e2c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb64003780 tx=0x7fcb64007a60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:10.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.276+0000 7fcb69ffb640 1 -- 172.21.15.190:0/1493283579 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fcb6c01d030 con 0x7fcb7413f0a0 2024-08-19T10:40:10.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.276+0000 7fcb69ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:10.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.276+0000 7fcb69ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:10.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.448+0000 7fcb7b3ca640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:10.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.448+0000 7fcb7b3ca640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:10.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.448+0000 7fcb7b3ca640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:10.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:10.448+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1493283579 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcb7409cce0 con 0x7fcb7413f0a0 2024-08-19T10:40:11.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.268+0000 7fcb69ffb640 1 -- 172.21.15.190:0/1493283579 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}]=0 pool 'ec-foo' removed v60) ==== 154+0+0 (secure 0 0 0) 0x7fcb6c014030 con 0x7fcb7413f0a0 2024-08-19T10:40:11.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.268+0000 7fcb69ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:11.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.268+0000 7fcb69ffb640 10 monclient: _finish_command 2 = system:0 pool 'ec-foo' removed 2024-08-19T10:40:11.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.364+0000 7fcb7b3ca640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:11.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.364+0000 7fcb7b3ca640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:11.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.364+0000 7fcb7b3ca640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:11.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.364+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1493283579 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcb7409d8f0 con 0x7fcb7413f0a0 2024-08-19T10:40:11.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.368+0000 7fcb69ffb640 1 -- 172.21.15.190:0/1493283579 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}]=0 pool 'ec-foo' does not exist v60) ==== 161+0+0 (secure 0 0 0) 0x7fcb6c00d2e0 con 0x7fcb7413f0a0 2024-08-19T10:40:11.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.368+0000 7fcb69ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "ec-foo", "pool2": "ec-foo", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:11.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.368+0000 7fcb69ffb640 10 monclient: _finish_command 3 = system:0 pool 'ec-foo' does not exist 2024-08-19T10:40:11.373 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'ec-foo' does not exist 2024-08-19T10:40:11.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcb5803be00 msgr2=0x7fcb5803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:11.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcb5803be00 0x7fcb5803e2c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb64003780 tx=0x7fcb64007a60 comp rx=0 tx=0).stop 2024-08-19T10:40:11.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 10 monclient: shutdown 2024-08-19T10:40:11.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:11.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb7413f0a0 msgr2=0x7fcb7413c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:11.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb7413f0a0 0x7fcb7413c8a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00cc30 tx=0x7fcb6c00a5c0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1493283579 shutdown_connections 2024-08-19T10:40:11.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcb5803be00 0x7fcb5803e2c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb7413f0a0 0x7fcb7413c8a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb7413a190 0x7fcb74133de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 --2- 172.21.15.190:0/1493283579 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcb741396a0 0x7fcb741338a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1493283579 >> 172.21.15.190:0/1493283579 conn(0x7fcb740abee0 msgr2=0x7fcb74134860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:11.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1493283579 shutdown_connections 2024-08-19T10:40:11.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.372+0000 7fcb7b3ca640 1 -- 172.21.15.190:0/1493283579 wait complete. 2024-08-19T10:40:11.390 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule ls 2024-08-19T10:40:11.390 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep foo 2024-08-19T10:40:11.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient: get_monmap_and_config 2024-08-19T10:40:11.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient: build_initial_monmap 2024-08-19T10:40:11.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient: monmap: 2024-08-19T10:40:11.508 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:11.508 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:11.508805+0000 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:11.508805+0000 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:11.509 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:11.510 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:11.510 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:11.510 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 1 Processor -- start 2024-08-19T10:40:11.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 1 -- start start 2024-08-19T10:40:11.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient: init 2024-08-19T10:40:11.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:11.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient: _add_conns ranks=[5,8,1,6,2,0,10,9,4,3,7] 2024-08-19T10:40:11.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb8c0a6740 0x7ffb8c0a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient(hunting): picked mon.noname-h con 0x7ffb8c0a6740 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:11.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a5bc0 0x7ffb8c0a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient(hunting): picked mon.noname-e con 0x7ffb8c0a5bc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:11.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ffb8c0a7060 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:11.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8c0a3740 con 0x7ffb8c0a7060 2024-08-19T10:40:11.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:11.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ffb8c0a4c20 con 0x7ffb8c0a6740 2024-08-19T10:40:11.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:11.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ffb8c0a3530 con 0x7ffb8c0a5bc0 2024-08-19T10:40:11.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:11.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb91798640 10 monclient(hunting): authenticate will time out at 1057.079127s 2024-08-19T10:40:11.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8b7fe640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7ffb8c0a7060 legacy=0x7ffb8c139590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:40:11.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb8c0a6740 0x7ffb8c0a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:11.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb8c0a6740 0x7ffb8c0a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54782/0 (socket says 172.21.15.190:54782) 2024-08-19T10:40:11.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 1 -- 172.21.15.190:0/1619032647 learned_addr learned my addr 172.21.15.190:0/1619032647 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:11.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 10 monclient(hunting): get_auth_request con 0x7ffb8c0a6740 auth_method 0 2024-08-19T10:40:11.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:11.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:11.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:11.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:11.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:11.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8a7fc640 1 --2- 172.21.15.190:0/1619032647 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a5bc0 0x7ffb8c0a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8a7fc640 10 monclient(hunting): get_auth_request con 0x7ffb8c0a5bc0 auth_method 0 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 10 monclient(hunting): handle_auth_done global_id 4820 payload 274 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 1 -- 172.21.15.190:0/1619032647 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a5bc0 msgr2=0x7ffb8c0a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 1 --2- 172.21.15.190:0/1619032647 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a5bc0 0x7ffb8c0a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 1 -- 172.21.15.190:0/1619032647 >> v1:172.21.15.190:6789/0 conn(0x7ffb8c0a7060 legacy=0x7ffb8c139590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 10 monclient: _finish_hunting 0 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 1 monclient: found mon.noname-h 2024-08-19T10:40:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:11.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:11.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.504+0000 7ffb8affd640 1 -- 172.21.15.190:0/1619032647 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c0a3240 con 0x7ffb8c0a6740 2024-08-19T10:40:11.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.508+0000 7ffb8affd640 1 -- 172.21.15.190:0/1619032647 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb8c0a6740 msgr2=0x7ffb8c0a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:11.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.508+0000 7ffb8affd640 1 --2- 172.21.15.190:0/1619032647 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb8c0a6740 0x7ffb8c0a5680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0098a0 tx=0x7ffb7c01c440 comp rx=0 tx=0).ready entity=mon.5 client_cookie=6485d9d20d291c0f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:11.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.508+0000 7ffb89ffb640 1 -- 172.21.15.190:0/1619032647 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb7c0b0070 con 0x7ffb8c0a6740 2024-08-19T10:40:11.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:11.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:11.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 10 monclient: dump: 2024-08-19T10:40:11.519 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:11.520 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 1 -- 172.21.15.190:0/1619032647 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c0aa810 con 0x7ffb8c0a6740 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 5 monclient: authenticate success, global_id 4820 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 1 -- 172.21.15.190:0/1619032647 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb7c0aae30 con 0x7ffb8c0a6740 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:11.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 4 set_mon_vals no callback set 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb89ffb640 10 monclient: dump: 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:11.522 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient: shutdown 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 -- 172.21.15.190:0/1619032647 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb8c0a6740 msgr2=0x7ffb8c0a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 --2- 172.21.15.190:0/1619032647 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb8c0a6740 0x7ffb8c0a5680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0098a0 tx=0x7ffb7c01c440 comp rx=0 tx=0).stop 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 -- 172.21.15.190:0/1619032647 shutdown_connections 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 --2- 172.21.15.190:0/1619032647 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a5bc0 0x7ffb8c0a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 --2- 172.21.15.190:0/1619032647 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb8c0a6740 0x7ffb8c0a5680 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 -- 172.21.15.190:0/1619032647 >> 172.21.15.190:0/1619032647 conn(0x7ffb8c0abd40 msgr2=0x7ffb8c0ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:11.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 -- 172.21.15.190:0/1619032647 shutdown_connections 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 -- 172.21.15.190:0/1619032647 wait complete. 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient: build_initial_monmap 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient: monmap: 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:11.524 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 Processor -- start 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 -- start start 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient: init 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:11.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient: _add_conns ranks=[1,8,6,4,0,7,3,5,2] 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb8c0a6740 0x7ffb8c1d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient(hunting): picked mon.noname-f con 0x7ffb8c0a6740 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a7060 0x7ffb8c138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient(hunting): picked mon.noname-e con 0x7ffb8c0a7060 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffb8c139500 0x7ffb8c13e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient(hunting): picked mon.noname-d con 0x7ffb8c139500 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c144460 con 0x7ffb8c0a6740 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ffb8c1442e0 con 0x7ffb8c139500 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:11.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ffb8c1445e0 con 0x7ffb8c0a7060 2024-08-19T10:40:11.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:11.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb91798640 10 monclient(hunting): authenticate will time out at 1057.086199s 2024-08-19T10:40:11.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8affd640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb8c0a6740 msgr2=0x7ffb8c1d1a00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:11.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb8c0a6740 0x7ffb8c1d1a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:11.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8a7fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a7060 0x7ffb8c138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:11.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffb8c139500 0x7ffb8c13e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:11.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8a7fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a7060 0x7ffb8c138fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:35408/0 (socket says 172.21.15.190:35408) 2024-08-19T10:40:11.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffb8c139500 0x7ffb8c13e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:51466/0 (socket says 172.21.15.190:51466) 2024-08-19T10:40:11.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8a7fc640 1 -- 172.21.15.190:0/4205772802 learned_addr learned my addr 172.21.15.190:0/4205772802 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:11.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8a7fc640 10 monclient(hunting): get_auth_request con 0x7ffb8c0a7060 auth_method 0 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 10 monclient(hunting): get_auth_request con 0x7ffb8c139500 auth_method 0 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:11.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 10 monclient(hunting): handle_auth_done global_id 4731 payload 824 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 1 -- 172.21.15.190:0/4205772802 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a7060 msgr2=0x7ffb8c138fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a7060 0x7ffb8c138fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 1 -- 172.21.15.190:0/4205772802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb8c0a6740 msgr2=0x7ffb8c1d1a00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb8c0a6740 0x7ffb8c1d1a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 1 monclient: found mon.noname-d 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8b7fe640 1 -- 172.21.15.190:0/4205772802 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c0a91b0 con 0x7ffb8c139500 2024-08-19T10:40:11.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.512+0000 7ffb8a7fc640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a7060 0x7ffb8c138fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb8b7fe640 1 -- 172.21.15.190:0/4205772802 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffb8c139500 msgr2=0x7ffb8c13e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb8b7fe640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffb8c139500 0x7ffb8c13e4e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400bdd0 tx=0x7ffb84009ed0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 1 -- 172.21.15.190:0/4205772802 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb8400ce00 con 0x7ffb8c139500 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 10 monclient: dump: 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.532 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:11.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 1 -- 172.21.15.190:0/4205772802 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb84010030 con 0x7ffb8c139500 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb91798640 5 monclient: authenticate success, global_id 4731 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 1 -- 172.21.15.190:0/4205772802 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb84015cc0 con 0x7ffb8c139500 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb88ff9640 4 set_mon_vals no callback set 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb737fe640 10 monclient: dump: 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.534 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb91798640 10 monclient: _renew_subs 2024-08-19T10:40:11.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb91798640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:11.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb91798640 1 -- 172.21.15.190:0/4205772802 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8c123790 con 0x7ffb8c139500 2024-08-19T10:40:11.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb91798640 10 monclient: _renew_subs 2024-08-19T10:40:11.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb91798640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:11.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb91798640 1 -- 172.21.15.190:0/4205772802 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8c0a1f70 con 0x7ffb8c139500 2024-08-19T10:40:11.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb91798640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:11.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb91798640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:11.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb91798640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:11.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.516+0000 7ffb91798640 1 -- 172.21.15.190:0/4205772802 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb8c0a21a0 con 0x7ffb8c139500 2024-08-19T10:40:11.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.520+0000 7ffb737fe640 1 -- 172.21.15.190:0/4205772802 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ffb840153e0 con 0x7ffb8c139500 2024-08-19T10:40:11.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.520+0000 7ffb737fe640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb7803be00 0x7ffb7803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.520+0000 7ffb737fe640 1 -- 172.21.15.190:0/4205772802 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7ffb8404f540 con 0x7ffb8c139500 2024-08-19T10:40:11.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.520+0000 7ffb8affd640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb7803be00 0x7ffb7803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:11.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.520+0000 7ffb8affd640 10 monclient: get_auth_request con 0x7ffb7803be00 auth_method 0 2024-08-19T10:40:11.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.520+0000 7ffb8affd640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb7803be00 0x7ffb7803e2c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0098a0 tx=0x7ffb7c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:11.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.532+0000 7ffb737fe640 1 -- 172.21.15.190:0/4205772802 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ffb8c0a21a0 con 0x7ffb8c139500 2024-08-19T10:40:11.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.532+0000 7ffb737fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:11.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.532+0000 7ffb737fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:11.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.700+0000 7ffb91798640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:11.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.700+0000 7ffb91798640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:11.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.700+0000 7ffb91798640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:11.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.700+0000 7ffb91798640 1 -- 172.21.15.190:0/4205772802 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7ffb8c077ad0 con 0x7ffb8c139500 2024-08-19T10:40:11.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.704+0000 7ffb737fe640 1 -- 172.21.15.190:0/4205772802 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v60) ==== 65+0+60 (secure 0 0 0) 0x7ffb8c077ad0 con 0x7ffb8c139500 2024-08-19T10:40:11.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.704+0000 7ffb737fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:11.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.704+0000 7ffb737fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:11.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.804+0000 7ffb91798640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:11.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.804+0000 7ffb91798640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:11.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.804+0000 7ffb91798640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:11.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.804+0000 7ffb91798640 1 -- 172.21.15.190:0/4205772802 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7ffb8c0a6b20 con 0x7ffb8c139500 2024-08-19T10:40:11.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.804+0000 7ffb737fe640 1 -- 172.21.15.190:0/4205772802 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v60) ==== 65+0+60 (secure 0 0 0) 0x7ffb8c0a91b0 con 0x7ffb8c139500 2024-08-19T10:40:11.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb737fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:11.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb737fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:40:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 -- 172.21.15.190:0/4205772802 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb7803be00 msgr2=0x7ffb7803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb7803be00 0x7ffb7803e2c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0098a0 tx=0x7ffb7c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:40:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 10 monclient: shutdown 2024-08-19T10:40:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 -- 172.21.15.190:0/4205772802 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffb8c139500 msgr2=0x7ffb8c13e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffb8c139500 0x7ffb8c13e4e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400bdd0 tx=0x7ffb84009ed0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 -- 172.21.15.190:0/4205772802 shutdown_connections 2024-08-19T10:40:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffb8c139500 0x7ffb8c13e4e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb7803be00 0x7ffb7803e2c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb8c0a7060 0x7ffb8c138fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 --2- 172.21.15.190:0/4205772802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb8c0a6740 0x7ffb8c1d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 -- 172.21.15.190:0/4205772802 >> 172.21.15.190:0/4205772802 conn(0x7ffb8c0abd40 msgr2=0x7ffb8c140f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 -- 172.21.15.190:0/4205772802 shutdown_connections 2024-08-19T10:40:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.808+0000 7ffb91798640 1 -- 172.21.15.190:0/4205772802 wait complete. 2024-08-19T10:40:11.829 INFO:tasks.workunit.client.0.smithi190.stdout:foo 2024-08-19T10:40:11.829 INFO:tasks.workunit.client.0.smithi190.stdout:foo-ssd 2024-08-19T10:40:11.829 INFO:tasks.workunit.client.0.smithi190.stdout:foo-hdd 2024-08-19T10:40:11.829 INFO:tasks.workunit.client.0.smithi190.stdout:ec-foo 2024-08-19T10:40:11.829 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rename foo foo-asdf 2024-08-19T10:40:11.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient: get_monmap_and_config 2024-08-19T10:40:11.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient: build_initial_monmap 2024-08-19T10:40:11.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient: monmap: 2024-08-19T10:40:11.948 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:11.948 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:11.948 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:11.948108+0000 2024-08-19T10:40:11.948 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:11.948108+0000 2024-08-19T10:40:11.948 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:11.948 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.948 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 1 Processor -- start 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 1 -- start start 2024-08-19T10:40:11.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient: init 2024-08-19T10:40:11.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient: _add_conns ranks=[9,0,8,3,10,7,6,5,1,4,2] 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb540a58d0 0x7fdb540a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient(hunting): picked mon.noname-a con 0x7fdb540a58d0 addr v2:172.21.15.89:3300/0 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdb540722b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 0x7fdb54137d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient(hunting): picked mon.noname-e con 0x7fdb540a61f0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb540a40f0 con 0x7fdb540722b0 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fdb54072730 con 0x7fdb540a58d0 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fdb54073d90 con 0x7fdb540a61f0 2024-08-19T10:40:11.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5bad7640 10 monclient(hunting): authenticate will time out at 1057.518054s 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 0x7fdb54137d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb59873640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb540a58d0 0x7fdb540a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb59072640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdb540722b0 0x7fdb54137680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:46528/0 (socket says 172.21.15.190:46528) 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb59072640 1 -- 172.21.15.190:0/699185419 learned_addr learned my addr 172.21.15.190:0/699185419 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 10 monclient(hunting): get_auth_request con 0x7fdb540a61f0 auth_method 0 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb59873640 10 monclient(hunting): get_auth_request con 0x7fdb540a58d0 auth_method 0 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb59873640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb59873640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:11.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb59873640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb59873640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb59873640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb59873640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 10 monclient(hunting): handle_auth_done global_id 4661 payload 274 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 1 -- 172.21.15.190:0/699185419 >> v2:172.21.15.89:3300/0 conn(0x7fdb540a58d0 msgr2=0x7fdb540a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 1 --2- 172.21.15.190:0/699185419 >> v2:172.21.15.89:3300/0 conn(0x7fdb540a58d0 0x7fdb540a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 1 -- 172.21.15.190:0/699185419 >> v1:172.21.15.89:6789/0 conn(0x7fdb540722b0 legacy=0x7fdb54137680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 10 monclient: _finish_hunting 0 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 1 monclient: found mon.noname-e 2024-08-19T10:40:11.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:11.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:11.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.944+0000 7fdb5a074640 1 -- 172.21.15.190:0/699185419 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb54074070 con 0x7fdb540a61f0 2024-08-19T10:40:11.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb5a074640 1 -- 172.21.15.190:0/699185419 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 msgr2=0x7fdb54137d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:11.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb5a074640 1 --2- 172.21.15.190:0/699185419 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 0x7fdb54137d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdb540a6990 tx=0x7fdb5001c160 comp rx=0 tx=0).ready entity=mon.8 client_cookie=601aaeaf6f1bb668 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 1 -- 172.21.15.190:0/699185419 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb500b1070 con 0x7fdb540a61f0 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 10 monclient: dump: 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:11.957 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:11.958 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:11.958 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:11.958 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:11.958 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:11.958 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:11.958 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:11.958 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:11.958 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 10 monclient: _finish_auth 0 2024-08-19T10:40:11.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 10 monclient: _check_auth_tickets 2024-08-19T10:40:11.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:11.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 1 -- 172.21.15.190:0/699185419 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb500a4c90 con 0x7fdb540a61f0 2024-08-19T10:40:11.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb5bad7640 5 monclient: authenticate success, global_id 4661 2024-08-19T10:40:11.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb5bad7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:11.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:11.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 1 -- 172.21.15.190:0/699185419 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb500aa7b0 con 0x7fdb540a61f0 2024-08-19T10:40:11.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:11.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:11.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb58871640 10 monclient: dump: 2024-08-19T10:40:11.959 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:11.959 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:11.959 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:11.960 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.948+0000 7fdb5bad7640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 4 set_mon_vals no callback set 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient: shutdown 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 -- 172.21.15.190:0/699185419 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 msgr2=0x7fdb54137d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 --2- 172.21.15.190:0/699185419 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 0x7fdb54137d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdb540a6990 tx=0x7fdb5001c160 comp rx=0 tx=0).stop 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 -- 172.21.15.190:0/699185419 shutdown_connections 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 --2- 172.21.15.190:0/699185419 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 0x7fdb54137d90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 --2- 172.21.15.190:0/699185419 >> v2:172.21.15.89:3300/0 conn(0x7fdb540a58d0 0x7fdb540a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 -- 172.21.15.190:0/699185419 >> 172.21.15.190:0/699185419 conn(0x7fdb540aa4c0 msgr2=0x7fdb540a69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 -- 172.21.15.190:0/699185419 shutdown_connections 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 -- 172.21.15.190:0/699185419 wait complete. 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient: build_initial_monmap 2024-08-19T10:40:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient: monmap: 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:11.963 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:11.963 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:11.963 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 Processor -- start 2024-08-19T10:40:11.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 -- start start 2024-08-19T10:40:11.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient: init 2024-08-19T10:40:11.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:11.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient: _add_conns ranks=[7,0,8,2,5,4,1,3,6] 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdb540722b0 0x7fdb5413e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient(hunting): picked mon.noname-i con 0x7fdb540722b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb540a58d0 0x7fdb541cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient(hunting): picked mon.noname-a con 0x7fdb540a58d0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 0x7fdb541cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient(hunting): picked mon.noname-e con 0x7fdb540a61f0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54074610 con 0x7fdb540a58d0 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fdb54074490 con 0x7fdb540722b0 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:11.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fdb541382d0 con 0x7fdb540a61f0 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5bad7640 10 monclient(hunting): authenticate will time out at 1057.526578s 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59873640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdb540722b0 msgr2=0x7fdb5413e300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59873640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdb540722b0 0x7fdb5413e300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59072640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb540a58d0 0x7fdb541cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59072640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb540a58d0 0x7fdb541cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:36220/0 (socket says 172.21.15.190:36220) 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59072640 1 -- 172.21.15.190:0/4240625489 learned_addr learned my addr 172.21.15.190:0/4240625489 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59072640 10 monclient(hunting): get_auth_request con 0x7fdb540a58d0 auth_method 0 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 0x7fdb541cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59072640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59072640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59072640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:11.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 10 monclient(hunting): get_auth_request con 0x7fdb540a61f0 auth_method 0 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59072640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59072640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb59072640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 10 monclient(hunting): handle_auth_done global_id 4670 payload 824 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 1 -- 172.21.15.190:0/4240625489 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdb540722b0 msgr2=0x7fdb5413e300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdb540722b0 0x7fdb5413e300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 1 -- 172.21.15.190:0/4240625489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb540a58d0 msgr2=0x7fdb541cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:11.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb540a58d0 0x7fdb541cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 10 monclient: _finish_hunting 0 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 1 monclient: found mon.noname-e 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.952+0000 7fdb5a074640 1 -- 172.21.15.190:0/4240625489 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb54121430 con 0x7fdb540a61f0 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5a074640 1 -- 172.21.15.190:0/4240625489 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 msgr2=0x7fdb541cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5a074640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 0x7fdb541cf3d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb500ab000 tx=0x7fdb50018f00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 1 -- 172.21.15.190:0/4240625489 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb500b1040 con 0x7fdb540a61f0 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 10 monclient: dump: 2024-08-19T10:40:11.970 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:11.971 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 1 -- 172.21.15.190:0/4240625489 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb50003d60 con 0x7fdb540a61f0 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 1 -- 172.21.15.190:0/4240625489 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb500ac8b0 con 0x7fdb540a61f0 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4a7fc640 10 monclient: dump: 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:11.972 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:11.973 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb4b7fe640 4 set_mon_vals no callback set 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5bad7640 5 monclient: authenticate success, global_id 4670 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5bad7640 10 monclient: _renew_subs 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5bad7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5bad7640 1 -- 172.21.15.190:0/4240625489 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5413e080 con 0x7fdb540a61f0 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5bad7640 10 monclient: _renew_subs 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5bad7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5bad7640 1 -- 172.21.15.190:0/4240625489 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb54121150 con 0x7fdb540a61f0 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5bad7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5bad7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5bad7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:11.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.956+0000 7fdb5bad7640 1 -- 172.21.15.190:0/4240625489 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb540a5cb0 con 0x7fdb540a61f0 2024-08-19T10:40:11.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.960+0000 7fdb4a7fc640 1 -- 172.21.15.190:0/4240625489 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fdb500ac440 con 0x7fdb540a61f0 2024-08-19T10:40:11.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.960+0000 7fdb4a7fc640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdb3803be00 0x7fdb3803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:11.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.960+0000 7fdb4a7fc640 1 -- 172.21.15.190:0/4240625489 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(60..60 src has 1..60) ==== 5217+0+0 (secure 0 0 0) 0x7fdb500e9540 con 0x7fdb540a61f0 2024-08-19T10:40:11.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.960+0000 7fdb59873640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdb3803be00 0x7fdb3803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:11.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.960+0000 7fdb59873640 10 monclient: get_auth_request con 0x7fdb3803be00 auth_method 0 2024-08-19T10:40:11.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.960+0000 7fdb59873640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdb3803be00 0x7fdb3803e2c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009b30 tx=0x7fdb4c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:11.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.972+0000 7fdb4a7fc640 1 -- 172.21.15.190:0/4240625489 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fdb540a40f0 con 0x7fdb540a61f0 2024-08-19T10:40:11.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.972+0000 7fdb4a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:11.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:11.972+0000 7fdb4a7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:12.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:12.148+0000 7fdb5bad7640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:12.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:12.148+0000 7fdb5bad7640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:12.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:12.148+0000 7fdb5bad7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:12.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:12.148+0000 7fdb5bad7640 1 -- 172.21.15.190:0/4240625489 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) -- 0x7fdb540008e0 con 0x7fdb540a61f0 2024-08-19T10:40:13.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.024+0000 7fdb4a7fc640 1 -- 172.21.15.190:0/4240625489 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 v61) ==== 110+0+0 (secure 0 0 0) 0x7fdb500c9dc0 con 0x7fdb540a61f0 2024-08-19T10:40:13.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.024+0000 7fdb4a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:13.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.024+0000 7fdb4a7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:13.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.132+0000 7fdb5bad7640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:13.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.132+0000 7fdb5bad7640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:13.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.132+0000 7fdb5bad7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:13.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.132+0000 7fdb5bad7640 1 -- 172.21.15.190:0/4240625489 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) -- 0x7fdb5406fe50 con 0x7fdb540a61f0 2024-08-19T10:40:13.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.132+0000 7fdb4a7fc640 1 -- 172.21.15.190:0/4240625489 <== mon.8 v2:172.21.15.89:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v61) ==== 139+0+0 (secure 0 0 0) 0x7fdb500ce020 con 0x7fdb540a61f0 2024-08-19T10:40:13.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.132+0000 7fdb4a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:13.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.132+0000 7fdb4a7fc640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo-asdf' 2024-08-19T10:40:13.139 INFO:tasks.workunit.client.0.smithi190.stderr:already renamed to 'foo-asdf' 2024-08-19T10:40:13.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 -- 172.21.15.190:0/4240625489 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdb3803be00 msgr2=0x7fdb3803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:13.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdb3803be00 0x7fdb3803e2c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009b30 tx=0x7fdb4c00b040 comp rx=0 tx=0).stop 2024-08-19T10:40:13.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 10 monclient: shutdown 2024-08-19T10:40:13.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:13.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 -- 172.21.15.190:0/4240625489 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 msgr2=0x7fdb541cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:13.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 0x7fdb541cf3d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb500ab000 tx=0x7fdb50018f00 comp rx=0 tx=0).stop 2024-08-19T10:40:13.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 -- 172.21.15.190:0/4240625489 shutdown_connections 2024-08-19T10:40:13.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdb3803be00 0x7fdb3803e2c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdb540a61f0 0x7fdb541cf3d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb540a58d0 0x7fdb541cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 --2- 172.21.15.190:0/4240625489 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdb540722b0 0x7fdb5413e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 -- 172.21.15.190:0/4240625489 >> 172.21.15.190:0/4240625489 conn(0x7fdb540aa4c0 msgr2=0x7fdb5413f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:13.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 -- 172.21.15.190:0/4240625489 shutdown_connections 2024-08-19T10:40:13.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.136+0000 7fdb5bad7640 1 -- 172.21.15.190:0/4240625489 wait complete. 2024-08-19T10:40:13.156 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rename foo foo-asdf 2024-08-19T10:40:13.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.272+0000 7f17c5c83640 10 monclient: get_monmap_and_config 2024-08-19T10:40:13.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.272+0000 7f17c5c83640 10 monclient: build_initial_monmap 2024-08-19T10:40:13.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 10 monclient: monmap: 2024-08-19T10:40:13.279 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:13.279 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:13.279 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:13.279555+0000 2024-08-19T10:40:13.279 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:13.279555+0000 2024-08-19T10:40:13.279 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:13.279 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.279 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:13.280 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:13.280 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:13.280 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:13.280 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:13.280 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:13.280 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:13.280 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:13.280 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:13.280 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:13.280 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:13.280 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 1 Processor -- start 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 1 -- start start 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 10 monclient: init 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 10 monclient: _add_conns ranks=[6,1,8,4,10,9,7,3,2,5,0] 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c01396a0 0x7f17c0139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 10 monclient(hunting): picked mon.noname-d con 0x7f17c01396a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f17c0139fc0 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17c00a6ff0 0x7f17c00a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 10 monclient(hunting): picked mon.noname-e con 0x7f17c00a6ff0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17c00a83e0 con 0x7f17c0139fc0 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:13.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f17c00a4f20 con 0x7f17c01396a0 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f17c00a2f30 con 0x7f17c00a6ff0 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17c5c83640 10 monclient(hunting): authenticate will time out at 1058.849427s 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17beffd640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f17c0139fc0 legacy=0x7f17c00a68e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c01396a0 0x7f17c0139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17c00a6ff0 0x7f17c00a73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c01396a0 0x7f17c0139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:51476/0 (socket says 172.21.15.190:51476) 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 1 -- 172.21.15.190:0/2308558407 learned_addr learned my addr 172.21.15.190:0/2308558407 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 10 monclient(hunting): get_auth_request con 0x7f17c01396a0 auth_method 0 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:13.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bffff640 10 monclient(hunting): get_auth_request con 0x7f17c00a6ff0 auth_method 0 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 10 monclient(hunting): handle_auth_done global_id 4740 payload 274 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 1 -- 172.21.15.190:0/2308558407 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17c00a6ff0 msgr2=0x7f17c00a73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 1 --2- 172.21.15.190:0/2308558407 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17c00a6ff0 0x7f17c00a73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 1 -- 172.21.15.190:0/2308558407 >> v1:172.21.15.190:6789/0 conn(0x7f17c0139fc0 legacy=0x7f17c00a68e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:13.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:13.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 1 monclient: found mon.noname-d 2024-08-19T10:40:13.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:13.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:13.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bf7fe640 1 -- 172.21.15.190:0/2308558407 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c00a35e0 con 0x7f17c01396a0 2024-08-19T10:40:13.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.276+0000 7f17bffff640 1 --2- 172.21.15.190:0/2308558407 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17c00a6ff0 0x7f17c00a73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:13.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17bf7fe640 1 -- 172.21.15.190:0/2308558407 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c01396a0 msgr2=0x7f17c0139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17bf7fe640 1 --2- 172.21.15.190:0/2308558407 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c01396a0 0x7f17c0139a80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f17b00098d0 tx=0x7f17b001c440 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ba5bacefc80199b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 1 -- 172.21.15.190:0/2308558407 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17b00b0070 con 0x7f17c01396a0 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 10 monclient: dump: 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.288 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17c5c83640 5 monclient: authenticate success, global_id 4740 2024-08-19T10:40:13.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17c5c83640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 1 -- 172.21.15.190:0/2308558407 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b00aa810 con 0x7f17c01396a0 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17c5c83640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17c5c83640 4 set_mon_vals no callback set 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 1 -- 172.21.15.190:0/2308558407 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17b00aae30 con 0x7f17c01396a0 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.280+0000 7f17be7fc640 10 monclient: dump: 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.290 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient: shutdown 2024-08-19T10:40:13.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 -- 172.21.15.190:0/2308558407 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c01396a0 msgr2=0x7f17c0139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 --2- 172.21.15.190:0/2308558407 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c01396a0 0x7f17c0139a80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f17b00098d0 tx=0x7f17b001c440 comp rx=0 tx=0).stop 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 -- 172.21.15.190:0/2308558407 shutdown_connections 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 --2- 172.21.15.190:0/2308558407 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17c00a6ff0 0x7f17c00a73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 --2- 172.21.15.190:0/2308558407 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c01396a0 0x7f17c0139a80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 -- 172.21.15.190:0/2308558407 >> 172.21.15.190:0/2308558407 conn(0x7f17c00abee0 msgr2=0x7f17c00ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 -- 172.21.15.190:0/2308558407 shutdown_connections 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 -- 172.21.15.190:0/2308558407 wait complete. 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient: build_initial_monmap 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient: monmap: 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:13.292 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:13.293 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 Processor -- start 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 -- start start 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient: init 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient: _add_conns ranks=[6,2,7,4,8,5,1,0,3] 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c00a6ff0 0x7f17c0137f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient(hunting): picked mon.noname-d con 0x7f17c00a6ff0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17c0139fc0 0x7f17c0138450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient(hunting): picked mon.noname-b con 0x7f17c0139fc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f17c013d6b0 0x7f17c013da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient(hunting): picked mon.noname-i con 0x7f17c013d6b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:13.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f17c013a3a0 con 0x7f17c0139fc0 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f17c00a7930 con 0x7f17c00a6ff0 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f17c0140170 con 0x7f17c013d6b0 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17c5c83640 10 monclient(hunting): authenticate will time out at 1058.858100s 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bffff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f17c013d6b0 msgr2=0x7f17c013da90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f17c013d6b0 0x7f17c013da90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c00a6ff0 0x7f17c0137f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17beffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17c0139fc0 0x7f17c0138450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:13.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c00a6ff0 0x7f17c0137f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:51492/0 (socket says 172.21.15.190:51492) 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 1 -- 172.21.15.190:0/652247621 learned_addr learned my addr 172.21.15.190:0/652247621 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 10 monclient(hunting): get_auth_request con 0x7f17c00a6ff0 auth_method 0 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17beffd640 10 monclient(hunting): get_auth_request con 0x7f17c0139fc0 auth_method 0 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17beffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17beffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:13.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 10 monclient(hunting): handle_auth_done global_id 4749 payload 824 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 1 -- 172.21.15.190:0/652247621 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f17c013d6b0 msgr2=0x7f17c013da90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f17c013d6b0 0x7f17c013da90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 1 -- 172.21.15.190:0/652247621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17c0139fc0 msgr2=0x7f17c0138450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17c0139fc0 0x7f17c0138450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 1 monclient: found mon.noname-d 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.284+0000 7f17bf7fe640 1 -- 172.21.15.190:0/652247621 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c00ac500 con 0x7f17c00a6ff0 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17bf7fe640 1 -- 172.21.15.190:0/652247621 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c00a6ff0 msgr2=0x7f17c0137f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17bf7fe640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c00a6ff0 0x7f17c0137f10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f17b001c270 tx=0x7f17b00191e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:13.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 1 -- 172.21.15.190:0/652247621 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17b00ba070 con 0x7f17c00a6ff0 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 10 monclient: dump: 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:13.302 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 10 monclient: _finish_auth 0 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 1 -- 172.21.15.190:0/652247621 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b00b1040 con 0x7f17c00a6ff0 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 1 -- 172.21.15.190:0/652247621 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17b00b0070 con 0x7f17c00a6ff0 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17c5c83640 5 monclient: authenticate success, global_id 4749 2024-08-19T10:40:13.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17c5c83640 10 monclient: _renew_subs 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17c5c83640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17c5c83640 1 -- 172.21.15.190:0/652247621 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c01225d0 con 0x7f17c00a6ff0 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17bdffb640 4 set_mon_vals no callback set 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 10 monclient: dump: 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17c5c83640 10 monclient: _renew_subs 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17c5c83640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17c5c83640 1 -- 172.21.15.190:0/652247621 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c0122e70 con 0x7f17c00a6ff0 2024-08-19T10:40:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17c5c83640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:13.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17c5c83640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:13.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17c5c83640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:13.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17c5c83640 1 -- 172.21.15.190:0/652247621 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c0140c30 con 0x7f17c00a6ff0 2024-08-19T10:40:13.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 1 -- 172.21.15.190:0/652247621 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f17b00aacd0 con 0x7f17c00a6ff0 2024-08-19T10:40:13.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f17ac03be00 0x7f17ac03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17a7fff640 1 -- 172.21.15.190:0/652247621 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(61..61 src has 1..61) ==== 5222+0+0 (secure 0 0 0) 0x7f17b00e8070 con 0x7f17c00a6ff0 2024-08-19T10:40:13.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17beffd640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f17ac03be00 0x7f17ac03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:13.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.288+0000 7f17beffd640 10 monclient: get_auth_request con 0x7f17ac03be00 auth_method 0 2024-08-19T10:40:13.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.292+0000 7f17beffd640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f17ac03be00 0x7f17ac03e2c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f17c013f810 tx=0x7f17b4009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:13.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.300+0000 7f17a7fff640 1 -- 172.21.15.190:0/652247621 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f17c00ac500 con 0x7f17c00a6ff0 2024-08-19T10:40:13.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.300+0000 7f17a7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:13.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.300+0000 7f17a7fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:13.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.480+0000 7f17c5c83640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:13.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.480+0000 7f17c5c83640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:13.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.480+0000 7f17c5c83640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:13.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.480+0000 7f17c5c83640 1 -- 172.21.15.190:0/652247621 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) -- 0x7f17c0139a80 con 0x7f17c00a6ff0 2024-08-19T10:40:13.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.484+0000 7f17a7fff640 1 -- 172.21.15.190:0/652247621 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v61) ==== 139+0+0 (secure 0 0 0) 0x7f17b0016150 con 0x7f17c00a6ff0 2024-08-19T10:40:13.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.484+0000 7f17a7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:13.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.484+0000 7f17a7fff640 10 monclient: _finish_command 2 = system:0 already renamed to 'foo-asdf' 2024-08-19T10:40:13.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.588+0000 7f17c5c83640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:13.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.588+0000 7f17c5c83640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:13.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.588+0000 7f17c5c83640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:13.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.588+0000 7f17c5c83640 1 -- 172.21.15.190:0/652247621 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"} v 0) -- 0x7f17c00008e0 con 0x7f17c00a6ff0 2024-08-19T10:40:13.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.592+0000 7f17a7fff640 1 -- 172.21.15.190:0/652247621 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}]=0 already renamed to 'foo-asdf' v61) ==== 139+0+0 (secure 0 0 0) 0x7f17b00aa7e0 con 0x7f17c00a6ff0 2024-08-19T10:40:13.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.592+0000 7f17a7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo", "dstname": "foo-asdf"}] 2024-08-19T10:40:13.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.592+0000 7f17a7fff640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo-asdf' 2024-08-19T10:40:13.597 INFO:tasks.workunit.client.0.smithi190.stderr:already renamed to 'foo-asdf' 2024-08-19T10:40:13.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 -- 172.21.15.190:0/652247621 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f17ac03be00 msgr2=0x7f17ac03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f17ac03be00 0x7f17ac03e2c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f17c013f810 tx=0x7f17b4009110 comp rx=0 tx=0).stop 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 10 monclient: shutdown 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 -- 172.21.15.190:0/652247621 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c00a6ff0 msgr2=0x7f17c0137f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c00a6ff0 0x7f17c0137f10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f17b001c270 tx=0x7f17b00191e0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 -- 172.21.15.190:0/652247621 shutdown_connections 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f17ac03be00 0x7f17ac03e2c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f17c013d6b0 0x7f17c013da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f17c0139fc0 0x7f17c0138450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 --2- 172.21.15.190:0/652247621 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f17c00a6ff0 0x7f17c0137f10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 -- 172.21.15.190:0/652247621 >> 172.21.15.190:0/652247621 conn(0x7f17c00abee0 msgr2=0x7f17c00aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 -- 172.21.15.190:0/652247621 shutdown_connections 2024-08-19T10:40:13.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.596+0000 7f17c5c83640 1 -- 172.21.15.190:0/652247621 wait complete. 2024-08-19T10:40:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rename bar bar-asdf 2024-08-19T10:40:13.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient: get_monmap_and_config 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient: build_initial_monmap 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient: monmap: 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:13.737129+0000 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:13.737129+0000 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:13.737 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:13.738 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:13.738 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:13.738 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:13.738 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:13.738 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:13.738 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:13.738 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:13.738 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 1 Processor -- start 2024-08-19T10:40:13.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 1 -- start start 2024-08-19T10:40:13.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient: init 2024-08-19T10:40:13.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:13.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient: _add_conns ranks=[9,2,0,3,10,6,4,8,5,1,7] 2024-08-19T10:40:13.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b4409b8d0 0x7f8b4409bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient(hunting): picked mon.noname-a con 0x7f8b4409b8d0 addr v2:172.21.15.89:3300/0 2024-08-19T10:40:13.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b4409dfb0 0x7f8b440a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient(hunting): picked mon.noname-b con 0x7f8b4409dfb0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:13.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8b440a42c0 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4409d2f0 con 0x7f8b440a42c0 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f8b4409d8b0 con 0x7f8b4409b8d0 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4409d5d0 con 0x7f8b4409dfb0 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.732+0000 7f8b49230640 10 monclient(hunting): authenticate will time out at 1059.307933s 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b4409dfb0 0x7f8b440a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b42d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8b4409b8d0 0x7f8b4409bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4359e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8b440a42c0 0x7f8b44140530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:46532/0 (socket says 172.21.15.190:46532) 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 10 monclient(hunting): get_auth_request con 0x7f8b4409dfb0 auth_method 0 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:13.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4359e640 1 -- 172.21.15.190:0/3086325174 learned_addr learned my addr 172.21.15.190:0/3086325174 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b42d9d640 10 monclient(hunting): get_auth_request con 0x7f8b4409b8d0 auth_method 0 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b42d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b42d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b42d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b42d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b42d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b42d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:13.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 10 monclient(hunting): handle_auth_done global_id 4763 payload 274 2024-08-19T10:40:13.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 1 -- 172.21.15.190:0/3086325174 >> v2:172.21.15.89:3300/0 conn(0x7f8b4409b8d0 msgr2=0x7f8b4409bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:13.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 1 --2- 172.21.15.190:0/3086325174 >> v2:172.21.15.89:3300/0 conn(0x7f8b4409b8d0 0x7f8b4409bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 1 -- 172.21.15.190:0/3086325174 >> v1:172.21.15.89:6789/0 conn(0x7f8b440a42c0 legacy=0x7f8b44140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:13.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 10 monclient: _finish_hunting 0 2024-08-19T10:40:13.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 1 monclient: found mon.noname-b 2024-08-19T10:40:13.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:13.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:13.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b4259c640 1 -- 172.21.15.190:0/3086325174 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4409c950 con 0x7f8b4409dfb0 2024-08-19T10:40:13.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.736+0000 7f8b42d9d640 1 --2- 172.21.15.190:0/3086325174 >> v2:172.21.15.89:3300/0 conn(0x7f8b4409b8d0 0x7f8b4409bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:13.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b4259c640 1 -- 172.21.15.190:0/3086325174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b4409dfb0 msgr2=0x7f8b440a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:13.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b4259c640 1 --2- 172.21.15.190:0/3086325174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b4409dfb0 0x7f8b440a3d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8b380097c0 tx=0x7f8b3801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110d3d66b43f8645 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:13.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 1 -- 172.21.15.190:0/3086325174 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b380b2070 con 0x7f8b4409dfb0 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 10 monclient: dump: 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:13.746 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 10 monclient: _finish_auth 0 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 1 -- 172.21.15.190:0/3086325174 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b380ab7b0 con 0x7f8b4409dfb0 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 1 -- 172.21.15.190:0/3086325174 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b380abe10 con 0x7f8b4409dfb0 2024-08-19T10:40:13.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b41d9b640 10 monclient: dump: 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:13.748 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 5 monclient: authenticate success, global_id 4763 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 4 set_mon_vals no callback set 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 10 monclient: shutdown 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 1 -- 172.21.15.190:0/3086325174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b4409dfb0 msgr2=0x7f8b440a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 1 --2- 172.21.15.190:0/3086325174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b4409dfb0 0x7f8b440a3d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8b380097c0 tx=0x7f8b3801c3f0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 1 -- 172.21.15.190:0/3086325174 shutdown_connections 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 1 --2- 172.21.15.190:0/3086325174 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b4409dfb0 0x7f8b440a3d80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 1 --2- 172.21.15.190:0/3086325174 >> v2:172.21.15.89:3300/0 conn(0x7f8b4409b8d0 0x7f8b4409bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 1 -- 172.21.15.190:0/3086325174 >> 172.21.15.190:0/3086325174 conn(0x7f8b440a4e70 msgr2=0x7f8b440a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:13.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 1 -- 172.21.15.190:0/3086325174 shutdown_connections 2024-08-19T10:40:13.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 1 -- 172.21.15.190:0/3086325174 wait complete. 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 10 monclient: build_initial_monmap 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 10 monclient: monmap: 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:13.754 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.740+0000 7f8b49230640 1 Processor -- start 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 1 -- start start 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient: init 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient: _add_conns ranks=[3,6,2,8,5,4,1,7,0] 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b4409b8d0 0x7f8b4413d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient(hunting): picked mon.noname-g con 0x7f8b4409b8d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:13.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b4409dfb0 0x7f8b4413dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient(hunting): picked mon.noname-d con 0x7f8b4409dfb0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b440a42c0 0x7f8b44092400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient(hunting): picked mon.noname-b con 0x7f8b440a42c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4409e3d0 con 0x7f8b440a42c0 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8b440a5c80 con 0x7f8b4409b8d0 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8b4409e550 con 0x7f8b4409dfb0 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient(hunting): authenticate will time out at 1059.316653s 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b4409b8d0 0x7f8b4413d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b4409b8d0 0x7f8b4413d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:56964/0 (socket says 172.21.15.190:56964) 2024-08-19T10:40:13.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 1 -- 172.21.15.190:0/4268556132 learned_addr learned my addr 172.21.15.190:0/4268556132 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 10 monclient(hunting): get_auth_request con 0x7f8b4409b8d0 auth_method 0 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4359e640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b440a42c0 0x7f8b44092400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4259c640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b4409dfb0 0x7f8b4413dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4359e640 10 monclient(hunting): get_auth_request con 0x7f8b440a42c0 auth_method 0 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4359e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:13.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4359e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4259c640 10 monclient(hunting): get_auth_request con 0x7f8b4409dfb0 auth_method 0 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 10 monclient(hunting): handle_auth_done global_id 4764 payload 824 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 1 -- 172.21.15.190:0/4268556132 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b4409dfb0 msgr2=0x7f8b4413dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b4409dfb0 0x7f8b4413dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 1 -- 172.21.15.190:0/4268556132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b440a42c0 msgr2=0x7f8b44092400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b440a42c0 0x7f8b44092400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 1 monclient: found mon.noname-g 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:13.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 1 -- 172.21.15.190:0/4268556132 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b440a39c0 con 0x7f8b4409b8d0 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4359e640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b440a42c0 0x7f8b44092400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 1 -- 172.21.15.190:0/4268556132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b4409b8d0 msgr2=0x7f8b4413d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b42d9d640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b4409b8d0 0x7f8b4413d790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400ebe0 tx=0x7f8b3400c520 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 1 -- 172.21.15.190:0/4268556132 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b34017070 con 0x7f8b4409b8d0 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 10 monclient: dump: 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.759 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 1 -- 172.21.15.190:0/4268556132 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3400ce40 con 0x7f8b4409b8d0 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 5 monclient: authenticate success, global_id 4764 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient: _renew_subs 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 1 -- 172.21.15.190:0/4268556132 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b440a3f10 con 0x7f8b4409b8d0 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 1 -- 172.21.15.190:0/4268556132 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b34013d50 con 0x7f8b4409b8d0 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b2b7fe640 10 monclient: dump: 2024-08-19T10:40:13.761 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:13.762 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient: _renew_subs 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b4159a640 4 set_mon_vals no callback set 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.744+0000 7f8b49230640 1 -- 172.21.15.190:0/4268556132 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b440948a0 con 0x7f8b4409b8d0 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.748+0000 7f8b49230640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.748+0000 7f8b49230640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.748+0000 7f8b49230640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.748+0000 7f8b49230640 1 -- 172.21.15.190:0/4268556132 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4409bcb0 con 0x7f8b4409b8d0 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.748+0000 7f8b2b7fe640 1 -- 172.21.15.190:0/4268556132 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8b3401a070 con 0x7f8b4409b8d0 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.748+0000 7f8b2b7fe640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b3003bd60 0x7f8b3003e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:13.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.748+0000 7f8b2b7fe640 1 -- 172.21.15.190:0/4268556132 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 5222+0+0 (secure 0 0 0) 0x7f8b3400f040 con 0x7f8b4409b8d0 2024-08-19T10:40:13.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.748+0000 7f8b4259c640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b3003bd60 0x7f8b3003e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:13.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.748+0000 7f8b4259c640 10 monclient: get_auth_request con 0x7f8b3003bd60 auth_method 0 2024-08-19T10:40:13.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.748+0000 7f8b4259c640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b3003bd60 0x7f8b3003e220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8b440708f0 tx=0x7f8b380a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:13.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.752+0000 7f8b2b7fe640 1 -- 172.21.15.190:0/4268556132 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8b34021070 con 0x7f8b4409b8d0 2024-08-19T10:40:13.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.752+0000 7f8b2b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:13.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.752+0000 7f8b2b7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:13.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.928+0000 7f8b49230640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-08-19T10:40:13.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.928+0000 7f8b49230640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-08-19T10:40:13.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.928+0000 7f8b49230640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:13.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:13.928+0000 7f8b49230640 1 -- 172.21.15.190:0/4268556132 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"} v 0) -- 0x7f8b4406e490 con 0x7f8b4409b8d0 2024-08-19T10:40:14.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.028+0000 7f8b2b7fe640 1 -- 172.21.15.190:0/4268556132 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}]=0 v62) ==== 110+0+0 (secure 0 0 0) 0x7f8b34013700 con 0x7f8b4409b8d0 2024-08-19T10:40:14.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.028+0000 7f8b2b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-08-19T10:40:14.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.028+0000 7f8b2b7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:14.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.132+0000 7f8b49230640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-08-19T10:40:14.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.132+0000 7f8b49230640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-08-19T10:40:14.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.132+0000 7f8b49230640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:14.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.132+0000 7f8b49230640 1 -- 172.21.15.190:0/4268556132 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"} v 0) -- 0x7f8b4409af00 con 0x7f8b4409b8d0 2024-08-19T10:40:14.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.660+0000 7f8b2b7fe640 1 -- 172.21.15.190:0/4268556132 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}]=0 already renamed to 'bar-asdf' v62) ==== 139+0+0 (secure 0 0 0) 0x7f8b3401bdc0 con 0x7f8b4409b8d0 2024-08-19T10:40:14.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.660+0000 7f8b2b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "bar", "dstname": "bar-asdf"}] 2024-08-19T10:40:14.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.660+0000 7f8b2b7fe640 10 monclient: _finish_command 3 = system:0 already renamed to 'bar-asdf' 2024-08-19T10:40:14.667 INFO:tasks.workunit.client.0.smithi190.stderr:already renamed to 'bar-asdf' 2024-08-19T10:40:14.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 -- 172.21.15.190:0/4268556132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b3003bd60 msgr2=0x7f8b3003e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:14.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b3003bd60 0x7f8b3003e220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8b440708f0 tx=0x7f8b380a7070 comp rx=0 tx=0).stop 2024-08-19T10:40:14.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 10 monclient: shutdown 2024-08-19T10:40:14.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:14.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 -- 172.21.15.190:0/4268556132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b4409b8d0 msgr2=0x7f8b4413d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:14.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b4409b8d0 0x7f8b4413d790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400ebe0 tx=0x7f8b3400c520 comp rx=0 tx=0).stop 2024-08-19T10:40:14.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 -- 172.21.15.190:0/4268556132 shutdown_connections 2024-08-19T10:40:14.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b3003bd60 0x7f8b3003e220 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:14.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b4409dfb0 0x7f8b4413dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:14.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b440a42c0 0x7f8b44092400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:14.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 --2- 172.21.15.190:0/4268556132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b4409b8d0 0x7f8b4413d790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:14.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 -- 172.21.15.190:0/4268556132 >> 172.21.15.190:0/4268556132 conn(0x7f8b440a4e70 msgr2=0x7f8b440a21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:14.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 -- 172.21.15.190:0/4268556132 shutdown_connections 2024-08-19T10:40:14.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.664+0000 7f8b49230640 1 -- 172.21.15.190:0/4268556132 wait complete. 2024-08-19T10:40:14.687 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule ls 2024-08-19T10:40:14.687 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep foo-asdf 2024-08-19T10:40:14.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.800+0000 7f9b0f9b8640 10 monclient: get_monmap_and_config 2024-08-19T10:40:14.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.800+0000 7f9b0f9b8640 10 monclient: build_initial_monmap 2024-08-19T10:40:14.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.800+0000 7f9b0f9b8640 10 monclient: monmap: 2024-08-19T10:40:14.807 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:14.807 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:14.807 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:14.806835+0000 2024-08-19T10:40:14.807 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:14.806835+0000 2024-08-19T10:40:14.807 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:14.807 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:14.807 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:14.807 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:14.807 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:14.808 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:14.808 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:14.808 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:14.808 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:14.808 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:14.808 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:14.808 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:14.808 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:14.808 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:14.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 1 Processor -- start 2024-08-19T10:40:14.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 1 -- start start 2024-08-19T10:40:14.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 10 monclient: init 2024-08-19T10:40:14.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:14.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 10 monclient: _add_conns ranks=[1,0,3,5,9,6,2,10,7,4,8] 2024-08-19T10:40:14.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9b08139980 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:14.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9b080a5730 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:14.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b080a5b10 0x7f9b08141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:14.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 10 monclient(hunting): picked mon.noname-g con 0x7f9b080a5b10 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:14.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b080a8470 con 0x7f9b080a5730 2024-08-19T10:40:14.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b080a3850 con 0x7f9b08139980 2024-08-19T10:40:14.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:14.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9b0809ca60 con 0x7f9b080a5b10 2024-08-19T10:40:14.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:14.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0f9b8640 10 monclient(hunting): authenticate will time out at 1060.376775s 2024-08-19T10:40:14.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0d754640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f9b08139980 legacy=0x7f9b08139d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:40:14.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b080a5b10 0x7f9b08141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:14.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b080a5b10 0x7f9b08141ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:56966/0 (socket says 172.21.15.190:56966) 2024-08-19T10:40:14.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 1 -- 172.21.15.190:0/4195743126 learned_addr learned my addr 172.21.15.190:0/4195743126 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:14.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 10 monclient(hunting): get_auth_request con 0x7f9b080a5b10 auth_method 0 2024-08-19T10:40:14.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:14.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:14.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:14.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:14.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:14.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:14.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 10 monclient(hunting): handle_auth_done global_id 4773 payload 274 2024-08-19T10:40:14.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 1 -- 172.21.15.190:0/4195743126 >> v1:172.21.15.190:6789/0 conn(0x7f9b08139980 legacy=0x7f9b08139d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:14.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 1 -- 172.21.15.190:0/4195743126 >> v1:172.21.15.89:6789/0 conn(0x7f9b080a5730 legacy=0x7f9b0813e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:14.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 10 monclient: _finish_hunting 0 2024-08-19T10:40:14.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 1 monclient: found mon.noname-g 2024-08-19T10:40:14.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:14.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:14.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.804+0000 7f9b0df55640 1 -- 172.21.15.190:0/4195743126 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0813a470 con 0x7f9b080a5b10 2024-08-19T10:40:14.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0df55640 1 -- 172.21.15.190:0/4195743126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b080a5b10 msgr2=0x7f9b08141ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:14.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0df55640 1 --2- 172.21.15.190:0/4195743126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b080a5b10 0x7f9b08141ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009e80 tx=0x7f9b0001c910 comp rx=0 tx=0).ready entity=mon.3 client_cookie=f6f3e90d32c715da server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:14.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 1 -- 172.21.15.190:0/4195743126 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b000091b0 con 0x7f9b080a5b10 2024-08-19T10:40:14.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:14.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:14.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 10 monclient: dump: 2024-08-19T10:40:14.817 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:14.817 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:14.817 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:14.818 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:14.818 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:14.818 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:14.818 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:14.818 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:14.818 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:14.818 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:14.818 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:14.818 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:14.818 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:14.819 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:14.819 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:14.819 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:14.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 10 monclient: _finish_auth 0 2024-08-19T10:40:14.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:14.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:14.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 1 -- 172.21.15.190:0/4195743126 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b000a6970 con 0x7f9b080a5b10 2024-08-19T10:40:14.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 5 monclient: authenticate success, global_id 4773 2024-08-19T10:40:14.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:14.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:14.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 1 -- 172.21.15.190:0/4195743126 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b000af480 con 0x7f9b080a5b10 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 4 set_mon_vals no callback set 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9afffff640 10 monclient: dump: 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:14.820 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:14.821 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:14.821 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:14.821 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:14.821 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:14.821 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:14.821 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:14.821 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:14.821 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:14.821 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:14.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 10 monclient: shutdown 2024-08-19T10:40:14.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:14.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/4195743126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b080a5b10 msgr2=0x7f9b08141ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:14.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 1 --2- 172.21.15.190:0/4195743126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b080a5b10 0x7f9b08141ff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009e80 tx=0x7f9b0001c910 comp rx=0 tx=0).stop 2024-08-19T10:40:14.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/4195743126 shutdown_connections 2024-08-19T10:40:14.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 1 --2- 172.21.15.190:0/4195743126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b080a5b10 0x7f9b08141ff0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:14.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/4195743126 >> 172.21.15.190:0/4195743126 conn(0x7f9b080abde0 msgr2=0x7f9b080ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:14.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/4195743126 shutdown_connections 2024-08-19T10:40:14.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/4195743126 wait complete. 2024-08-19T10:40:14.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 10 monclient: build_initial_monmap 2024-08-19T10:40:14.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 10 monclient: monmap: 2024-08-19T10:40:14.822 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:14.822 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:14.823 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:14.823 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:14.823 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:14.823 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:14.823 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:14.823 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:14.823 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:14.823 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:14.823 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:14.823 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:14.823 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:14.824 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:14.824 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:14.824 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:14.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 1 Processor -- start 2024-08-19T10:40:14.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.808+0000 7f9b0f9b8640 1 -- start start 2024-08-19T10:40:14.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 10 monclient: init 2024-08-19T10:40:14.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:14.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 10 monclient: _add_conns ranks=[7,2,6,5,3,4,1,0,8] 2024-08-19T10:40:14.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b080a5730 0x7f9b081d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:14.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 10 monclient(hunting): picked mon.noname-i con 0x7f9b080a5730 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:14.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a5b10 0x7f9b081d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:14.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 10 monclient(hunting): picked mon.noname-b con 0x7f9b080a5b10 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:14.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9b08139980 0x7f9b081d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:14.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0d754640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b080a5730 msgr2=0x7f9b081d0fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:14.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 10 monclient(hunting): picked mon.noname-d con 0x7f9b08139980 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:14.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:14.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9b0806b6c0 con 0x7f9b080a5b10 2024-08-19T10:40:14.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:14.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9b08144d50 con 0x7f9b08139980 2024-08-19T10:40:14.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0d754640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b080a5730 0x7f9b081d0fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:14.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:14.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9b08144ed0 con 0x7f9b080a5730 2024-08-19T10:40:14.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:14.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0f9b8640 10 monclient(hunting): authenticate will time out at 1060.384691s 2024-08-19T10:40:14.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0df55640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9b08139980 0x7f9b081d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:14.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a5b10 0x7f9b081d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:14.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0df55640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9b08139980 0x7f9b081d1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:51518/0 (socket says 172.21.15.190:51518) 2024-08-19T10:40:14.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a5b10 0x7f9b081d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:49692/0 (socket says 172.21.15.190:49692) 2024-08-19T10:40:14.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0df55640 1 -- 172.21.15.190:0/1028946908 learned_addr learned my addr 172.21.15.190:0/1028946908 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:14.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0df55640 10 monclient(hunting): get_auth_request con 0x7f9b08139980 auth_method 0 2024-08-19T10:40:14.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0df55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:14.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0df55640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0df55640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 10 monclient(hunting): get_auth_request con 0x7f9b080a5b10 auth_method 0 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0df55640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0df55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0df55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:14.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 10 monclient(hunting): handle_auth_done global_id 4781 payload 824 2024-08-19T10:40:14.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 1 -- 172.21.15.190:0/1028946908 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b080a5730 msgr2=0x7f9b081d0fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:14.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b080a5730 0x7f9b081d0fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:14.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 1 -- 172.21.15.190:0/1028946908 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9b08139980 msgr2=0x7f9b081d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:14.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9b08139980 0x7f9b081d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:14.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 10 monclient: _finish_hunting 0 2024-08-19T10:40:14.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 1 monclient: found mon.noname-b 2024-08-19T10:40:14.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:14.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:14.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0cf53640 1 -- 172.21.15.190:0/1028946908 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b08141130 con 0x7f9b080a5b10 2024-08-19T10:40:14.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.812+0000 7f9b0df55640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9b08139980 0x7f9b081d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:14.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0cf53640 1 -- 172.21.15.190:0/1028946908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a5b10 msgr2=0x7f9b081d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:14.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0cf53640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a5b10 0x7f9b081d14f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9af800c410 tx=0x7f9af800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 1 -- 172.21.15.190:0/1028946908 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 1 -- 172.21.15.190:0/1028946908 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9af8017070 con 0x7f9b080a5b10 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 10 monclient: dump: 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:14.831 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:14.832 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 1 -- 172.21.15.190:0/1028946908 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af8006c50 con 0x7f9b080a5b10 2024-08-19T10:40:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 1 -- 172.21.15.190:0/1028946908 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9af8006e00 con 0x7f9b080a5b10 2024-08-19T10:40:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0f9b8640 5 monclient: authenticate success, global_id 4781 2024-08-19T10:40:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0f9b8640 10 monclient: _renew_subs 2024-08-19T10:40:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0f9b8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/1028946908 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b08141410 con 0x7f9b080a5b10 2024-08-19T10:40:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9aff7fe640 4 set_mon_vals no callback set 2024-08-19T10:40:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:14.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9afdffb640 10 monclient: dump: 2024-08-19T10:40:14.834 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:14.834 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:14.834 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:14.834 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:14.834 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:14.834 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:14.834 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:14.834 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:14.834 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0f9b8640 10 monclient: _renew_subs 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0f9b8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/1028946908 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0813f0b0 con 0x7f9b080a5b10 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0f9b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:14.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0f9b8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0f9b8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.816+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/1028946908 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b080a2840 con 0x7f9b080a5b10 2024-08-19T10:40:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.820+0000 7f9afdffb640 1 -- 172.21.15.190:0/1028946908 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9af80067e0 con 0x7f9b080a5b10 2024-08-19T10:40:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.820+0000 7f9afdffb640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9aec03be00 0x7f9aec03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.820+0000 7f9afdffb640 1 -- 172.21.15.190:0/1028946908 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5227+0+0 (secure 0 0 0) 0x7f9af804e300 con 0x7f9b080a5b10 2024-08-19T10:40:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.820+0000 7f9b0d754640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9aec03be00 0x7f9aec03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.820+0000 7f9b0d754640 10 monclient: get_auth_request con 0x7f9aec03be00 auth_method 0 2024-08-19T10:40:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.820+0000 7f9b0d754640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9aec03be00 0x7f9aec03e2c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b04003990 tx=0x7f9b04006cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:14.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.832+0000 7f9afdffb640 1 -- 172.21.15.190:0/1028946908 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9b080a2840 con 0x7f9b080a5b10 2024-08-19T10:40:14.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.832+0000 7f9afdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:14.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:14.832+0000 7f9afdffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:15.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.004+0000 7f9b0f9b8640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:15.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.004+0000 7f9b0f9b8640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:15.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.004+0000 7f9b0f9b8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:15.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.004+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/1028946908 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f9b08139d60 con 0x7f9b080a5b10 2024-08-19T10:40:15.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.004+0000 7f9afdffb640 1 -- 172.21.15.190:0/1028946908 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v62) ==== 65+0+70 (secure 0 0 0) 0x7f9af8020020 con 0x7f9b080a5b10 2024-08-19T10:40:15.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.004+0000 7f9afdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:15.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.004+0000 7f9afdffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:15.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.108+0000 7f9b0f9b8640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:15.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.108+0000 7f9b0f9b8640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:15.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.108+0000 7f9b0f9b8640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:15.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.108+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/1028946908 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f9b080008e0 con 0x7f9b080a5b10 2024-08-19T10:40:15.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.108+0000 7f9afdffb640 1 -- 172.21.15.190:0/1028946908 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v62) ==== 65+0+70 (secure 0 0 0) 0x7f9af804e080 con 0x7f9b080a5b10 2024-08-19T10:40:15.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.108+0000 7f9afdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:15.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.108+0000 7f9afdffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:40:15.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/1028946908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9aec03be00 msgr2=0x7f9aec03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:15.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9aec03be00 0x7f9aec03e2c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b04003990 tx=0x7f9b04006cf0 comp rx=0 tx=0).stop 2024-08-19T10:40:15.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 10 monclient: shutdown 2024-08-19T10:40:15.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:15.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/1028946908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a5b10 msgr2=0x7f9b081d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:15.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a5b10 0x7f9b081d14f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9af800c410 tx=0x7f9af800c8d0 comp rx=0 tx=0).stop 2024-08-19T10:40:15.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/1028946908 shutdown_connections 2024-08-19T10:40:15.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9aec03be00 0x7f9aec03e2c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:15.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9b08139980 0x7f9b081d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:15.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b080a5b10 0x7f9b081d14f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:15.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 --2- 172.21.15.190:0/1028946908 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b080a5730 0x7f9b081d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:15.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/1028946908 >> 172.21.15.190:0/1028946908 conn(0x7f9b080abde0 msgr2=0x7f9b080a9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:15.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/1028946908 shutdown_connections 2024-08-19T10:40:15.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.112+0000 7f9b0f9b8640 1 -- 172.21.15.190:0/1028946908 wait complete. 2024-08-19T10:40:15.131 INFO:tasks.workunit.client.0.smithi190.stdout:foo-asdf 2024-08-19T10:40:15.132 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep bar-asdf 2024-08-19T10:40:15.132 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule ls 2024-08-19T10:40:15.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.244+0000 7fc8e14c6640 10 monclient: get_monmap_and_config 2024-08-19T10:40:15.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.244+0000 7fc8e14c6640 10 monclient: build_initial_monmap 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.244+0000 7fc8e14c6640 10 monclient: monmap: 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:15.250048+0000 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:15.250048+0000 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:15.250 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:15.251 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:15.251 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:15.251 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:15.251 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:15.251 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:15.251 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:15.251 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:15.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.244+0000 7fc8e14c6640 1 Processor -- start 2024-08-19T10:40:15.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.244+0000 7fc8e14c6640 1 -- start start 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.244+0000 7fc8e14c6640 10 monclient: init 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 10 monclient: _add_conns ranks=[10,7,1,8,0,4,9,3,6,2,5] 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8dc0a6850 0x7fc8dc0a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 10 monclient(hunting): picked mon.noname-f con 0x7fc8dc0a6850 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8dc0a6c30 0x7fc8dc0a7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 10 monclient(hunting): picked mon.noname-i con 0x7fc8dc0a6c30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc8dc1396a0 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8dc0a3850 con 0x7fc8dc1396a0 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc8dc0a3640 con 0x7fc8dc0a6850 2024-08-19T10:40:15.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:15.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc8dc0a46e0 con 0x7fc8dc0a6c30 2024-08-19T10:40:15.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:15.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8e14c6640 10 monclient(hunting): authenticate will time out at 1060.820499s 2024-08-19T10:40:15.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8daffd640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fc8dc0a6850 msgr2=0x7fc8dc0a57f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:15.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8da7fc640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8dc0a6c30 msgr2=0x7fc8dc0a7010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:15.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8db7fe640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fc8dc1396a0 legacy=0x7fc8dc139b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:40:15.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8daffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8dc0a6850 0x7fc8dc0a57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:15.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.248+0000 7fc8da7fc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8dc0a6c30 0x7fc8dc0a7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:15.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.452+0000 7fc8db7fe640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fc8dc1396a0 legacy=0x7fc8dc139b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:40:15.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.452+0000 7fc8da7fc640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8dc0a6c30 msgr2=0x7fc8dc0a7010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:15.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.452+0000 7fc8daffd640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fc8dc0a6850 msgr2=0x7fc8dc0a57f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:15.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.452+0000 7fc8da7fc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8dc0a6c30 0x7fc8dc0a7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:40:15.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.452+0000 7fc8daffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8dc0a6850 0x7fc8dc0a57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:40:15.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.856+0000 7fc8db7fe640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fc8dc1396a0 legacy=0x7fc8dc139b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:40:15.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.856+0000 7fc8da7fc640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8dc0a6c30 msgr2=0x7fc8dc0a7010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:15.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.856+0000 7fc8daffd640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fc8dc0a6850 msgr2=0x7fc8dc0a57f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:15.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.856+0000 7fc8da7fc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8dc0a6c30 0x7fc8dc0a7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:40:15.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:15.856+0000 7fc8daffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8dc0a6850 0x7fc8dc0a57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:40:16.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:16.660+0000 7fc8db7fe640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fc8dc1396a0 legacy=0x7fc8dc139b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:40:16.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:16.660+0000 7fc8da7fc640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8dc0a6c30 msgr2=0x7fc8dc0a7010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:16.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:16.660+0000 7fc8daffd640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fc8dc0a6850 msgr2=0x7fc8dc0a57f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:16.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:16.660+0000 7fc8da7fc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8dc0a6c30 0x7fc8dc0a7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:40:16.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:16.660+0000 7fc8daffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8dc0a6850 0x7fc8dc0a57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:40:16.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:16.660+0000 7fc8d9ffb640 1 -- DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:40:18.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 10 monclient(hunting): tick 2024-08-19T10:40:18.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:40:18.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 monclient(hunting): continuing hunt 2024-08-19T10:40:18.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:40:18.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8dc0a6c30 msgr2=0x7fc8dc0a7010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:18.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8dc0a6c30 0x7fc8dc0a7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fc8dc0a6850 msgr2=0x7fc8dc0a57f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:18.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc8dc0a6850 0x7fc8dc0a57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fc8dc1396a0 legacy=0x7fc8dc139b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 10 monclient: _add_conns ranks=[8,2,5,9,4,3,6,0] 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc8c8003800 0x7fc8c8005c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 10 monclient(hunting): picked mon.noname-e con 0x7fc8c8003800 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8c80062a0 0x7fc8c8008730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 10 monclient(hunting): picked mon.noname-b con 0x7fc8c80062a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc8c8008e20 0x7fc8c800b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 10 monclient(hunting): picked mon.noname-h con 0x7fc8c8008e20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc8dc0a3640 con 0x7fc8c80062a0 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc8dc0a46e0 con 0x7fc8c8008e20 2024-08-19T10:40:18.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:18.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc8c800ba70 con 0x7fc8c8003800 2024-08-19T10:40:18.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8d8ff9640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:18.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc8c8008e20 0x7fc8c800b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:18.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc8c8008e20 0x7fc8c800b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:55364/0 (socket says 172.21.15.190:55364) 2024-08-19T10:40:18.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 1 -- 172.21.15.190:0/1819229173 learned_addr learned my addr 172.21.15.190:0/1819229173 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:18.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 10 monclient(hunting): get_auth_request con 0x7fc8c8008e20 auth_method 0 2024-08-19T10:40:18.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:18.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:18.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:18.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8daffd640 1 --2- 172.21.15.190:0/1819229173 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc8c8003800 0x7fc8c8005c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:18.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8da7fc640 1 --2- 172.21.15.190:0/1819229173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8c80062a0 0x7fc8c8008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:18.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8daffd640 10 monclient(hunting): get_auth_request con 0x7fc8c8003800 auth_method 0 2024-08-19T10:40:18.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:18.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8daffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:18.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8daffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:18.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8da7fc640 10 monclient(hunting): get_auth_request con 0x7fc8c80062a0 auth_method 0 2024-08-19T10:40:18.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8da7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:18.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8da7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:18.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:18.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8da7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8da7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8da7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 10 monclient(hunting): handle_auth_done global_id 4829 payload 274 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 1 -- 172.21.15.190:0/1819229173 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc8c8003800 msgr2=0x7fc8c8005c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 1 --2- 172.21.15.190:0/1819229173 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc8c8003800 0x7fc8c8005c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 1 -- 172.21.15.190:0/1819229173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8c80062a0 msgr2=0x7fc8c8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:18.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 1 --2- 172.21.15.190:0/1819229173 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc8c80062a0 0x7fc8c8008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:18.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 1 monclient: found mon.noname-h 2024-08-19T10:40:18.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:18.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:18.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.248+0000 7fc8db7fe640 1 -- 172.21.15.190:0/1819229173 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc0a3350 con 0x7fc8c8008e20 2024-08-19T10:40:18.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.252+0000 7fc8db7fe640 1 -- 172.21.15.190:0/1819229173 reap_dead start 2024-08-19T10:40:18.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8db7fe640 1 -- 172.21.15.190:0/1819229173 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc8c8008e20 msgr2=0x7fc8c800b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:18.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8db7fe640 1 --2- 172.21.15.190:0/1819229173 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc8c8008e20 0x7fc8c800b2b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c320 tx=0x7fc8d401d230 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d121d77d4510e9c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:18.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 1 -- 172.21.15.190:0/1819229173 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc8d40b1070 con 0x7fc8c8008e20 2024-08-19T10:40:18.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:18.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:18.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 10 monclient: dump: 2024-08-19T10:40:18.262 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:18.262 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:18.262 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:18.262 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:18.262 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:18.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:18.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:18.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:18.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 1 -- 172.21.15.190:0/1819229173 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8dc0a5f40 con 0x7fc8c8008e20 2024-08-19T10:40:18.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:18.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 1 -- 172.21.15.190:0/1819229173 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc8d40abc60 con 0x7fc8c8008e20 2024-08-19T10:40:18.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8e14c6640 5 monclient: authenticate success, global_id 4829 2024-08-19T10:40:18.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8e14c6640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:18.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8e14c6640 4 set_mon_vals no callback set 2024-08-19T10:40:18.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:18.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:18.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8d9ffb640 10 monclient: dump: 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:18.265 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:18.266 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:18.266 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:18.266 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:18.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8e14c6640 10 monclient: shutdown 2024-08-19T10:40:18.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8e14c6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:18.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8e14c6640 1 -- 172.21.15.190:0/1819229173 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc8c8008e20 msgr2=0x7fc8c800b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:18.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.256+0000 7fc8e14c6640 1 --2- 172.21.15.190:0/1819229173 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc8c8008e20 0x7fc8c800b2b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c320 tx=0x7fc8d401d230 comp rx=0 tx=0).stop 2024-08-19T10:40:18.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 -- 172.21.15.190:0/1819229173 shutdown_connections 2024-08-19T10:40:18.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 --2- 172.21.15.190:0/1819229173 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc8c8008e20 0x7fc8c800b2b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 -- 172.21.15.190:0/1819229173 >> 172.21.15.190:0/1819229173 conn(0x7fc8dc0abe50 msgr2=0x7fc8dc0ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:18.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 -- 172.21.15.190:0/1819229173 shutdown_connections 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 -- 172.21.15.190:0/1819229173 wait complete. 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient: build_initial_monmap 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient: monmap: 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:18.270 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:18.271 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:18.271 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:18.271 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:18.271 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:18.271 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:18.271 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:18.271 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:18.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 Processor -- start 2024-08-19T10:40:18.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 -- start start 2024-08-19T10:40:18.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient: init 2024-08-19T10:40:18.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient: _add_conns ranks=[7,4,6,2,8,3,5,1,0] 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8c8008e20 0x7fc8dc13fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient(hunting): picked mon.noname-i con 0x7fc8c8008e20 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8dc1d17d0 0x7fc8dc1d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient(hunting): picked mon.noname-c con 0x7fc8dc1d17d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc8dc13d7b0 0x7fc8dc13dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient(hunting): picked mon.noname-d con 0x7fc8dc13d7b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc8dc0a5970 con 0x7fc8dc1d17d0 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fc8dc0a57f0 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:18.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc8dc0a5af0 con 0x7fc8c8008e20 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8daffd640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8c8008e20 msgr2=0x7fc8dc13fd80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8e14c6640 10 monclient(hunting): authenticate will time out at 1063.834635s 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8daffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8c8008e20 0x7fc8dc13fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc8dc13d7b0 0x7fc8dc13dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8da7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8dc1d17d0 0x7fc8dc1d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8da7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8dc1d17d0 0x7fc8dc1d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:33372/0 (socket says 172.21.15.190:33372) 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8da7fc640 1 -- 172.21.15.190:0/2457693846 learned_addr learned my addr 172.21.15.190:0/2457693846 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc8dc13d7b0 0x7fc8dc13dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:44760/0 (socket says 172.21.15.190:44760) 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8da7fc640 10 monclient(hunting): get_auth_request con 0x7fc8dc1d17d0 auth_method 0 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8da7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8da7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:18.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8da7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 10 monclient(hunting): get_auth_request con 0x7fc8dc13d7b0 auth_method 0 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8da7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8da7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8da7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 10 monclient(hunting): handle_auth_done global_id 4776 payload 824 2024-08-19T10:40:18.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 1 -- 172.21.15.190:0/2457693846 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8c8008e20 msgr2=0x7fc8dc13fd80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:18.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8c8008e20 0x7fc8dc13fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 1 -- 172.21.15.190:0/2457693846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8dc1d17d0 msgr2=0x7fc8dc1d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:18.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8dc1d17d0 0x7fc8dc1d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:18.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 1 monclient: found mon.noname-d 2024-08-19T10:40:18.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:18.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:18.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8db7fe640 1 -- 172.21.15.190:0/2457693846 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc0a2660 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.260+0000 7fc8da7fc640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8dc1d17d0 0x7fc8dc1d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:18.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8db7fe640 1 -- 172.21.15.190:0/2457693846 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc8dc13d7b0 msgr2=0x7fc8dc13dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:18.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8db7fe640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc8dc13d7b0 0x7fc8dc13dc10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0a5790 tx=0x7fc8d4019fd0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:18.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 1 -- 172.21.15.190:0/2457693846 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc8d40b1040 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:18.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:18.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 10 monclient: dump: 2024-08-19T10:40:18.278 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:18.278 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:18.278 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:18.278 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:18.278 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:18.278 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:18.278 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:18.279 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:18.279 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:18.279 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:18.279 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:18.279 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:18.279 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:18.279 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:18.279 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:18.280 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:18.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:18.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:18.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:18.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 1 -- 172.21.15.190:0/2457693846 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d4003e30 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:18.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 1 -- 172.21.15.190:0/2457693846 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc8d40acd00 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8e14c6640 5 monclient: authenticate success, global_id 4776 2024-08-19T10:40:18.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8e14c6640 10 monclient: _renew_subs 2024-08-19T10:40:18.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8e14c6640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:18.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8e14c6640 1 -- 172.21.15.190:0/2457693846 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8dc122b50 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:18.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:18.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8c37fe640 10 monclient: dump: 2024-08-19T10:40:18.282 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:18.282 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:18.282 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:18.282 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:18.282 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:18.282 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:18.282 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:18.282 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:18.283 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:18.283 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:18.283 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:18.283 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:18.283 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:18.283 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:18.283 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:18.283 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:18.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8d97fa640 4 set_mon_vals no callback set 2024-08-19T10:40:18.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8e14c6640 10 monclient: _renew_subs 2024-08-19T10:40:18.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8e14c6640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:18.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.264+0000 7fc8e14c6640 1 -- 172.21.15.190:0/2457693846 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8dc1c9100 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.268+0000 7fc8e14c6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:18.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.268+0000 7fc8e14c6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:18.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.268+0000 7fc8e14c6640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:18.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.268+0000 7fc8e14c6640 1 -- 172.21.15.190:0/2457693846 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8dc0a7010 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.268+0000 7fc8c37fe640 1 -- 172.21.15.190:0/2457693846 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc8d40c0020 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.268+0000 7fc8c37fe640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc8c803d1a0 0x7fc8c803f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:18.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.268+0000 7fc8c37fe640 1 -- 172.21.15.190:0/2457693846 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(63..63 src has 1..63) ==== 5227+0+0 (secure 0 0 0) 0x7fc8d40be020 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.268+0000 7fc8daffd640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc8c803d1a0 0x7fc8c803f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:18.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.268+0000 7fc8daffd640 10 monclient: get_auth_request con 0x7fc8c803d1a0 auth_method 0 2024-08-19T10:40:18.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.272+0000 7fc8daffd640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc8c803d1a0 0x7fc8c803f660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc000eb0 tx=0x7fc8cc00d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:18.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.280+0000 7fc8c37fe640 1 -- 172.21.15.190:0/2457693846 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc8dc0a2660 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.280+0000 7fc8c37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:18.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.280+0000 7fc8c37fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:18.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.456+0000 7fc8e14c6640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:18.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.456+0000 7fc8e14c6640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:18.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.456+0000 7fc8e14c6640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:18.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.456+0000 7fc8e14c6640 1 -- 172.21.15.190:0/2457693846 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7fc8dc140710 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.456+0000 7fc8c37fe640 1 -- 172.21.15.190:0/2457693846 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v63) ==== 65+0+70 (secure 0 0 0) 0x7fc8d4016f40 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.456+0000 7fc8c37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:18.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.456+0000 7fc8c37fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:18.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.560+0000 7fc8e14c6640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:18.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.560+0000 7fc8e14c6640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:18.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.560+0000 7fc8e14c6640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:18.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.560+0000 7fc8e14c6640 1 -- 172.21.15.190:0/2457693846 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7fc8dc13f220 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.560+0000 7fc8c37fe640 1 -- 172.21.15.190:0/2457693846 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v63) ==== 65+0+70 (secure 0 0 0) 0x7fc8d40e7530 con 0x7fc8dc13d7b0 2024-08-19T10:40:18.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.560+0000 7fc8c37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:18.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.560+0000 7fc8c37fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:40:18.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 -- 172.21.15.190:0/2457693846 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc8c803d1a0 msgr2=0x7fc8c803f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:18.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc8c803d1a0 0x7fc8c803f660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc000eb0 tx=0x7fc8cc00d040 comp rx=0 tx=0).stop 2024-08-19T10:40:18.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 10 monclient: shutdown 2024-08-19T10:40:18.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:18.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 -- 172.21.15.190:0/2457693846 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc8dc13d7b0 msgr2=0x7fc8dc13dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:18.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc8dc13d7b0 0x7fc8dc13dc10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0a5790 tx=0x7fc8d4019fd0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 -- 172.21.15.190:0/2457693846 shutdown_connections 2024-08-19T10:40:18.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc8c803d1a0 0x7fc8c803f660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc8dc13d7b0 0x7fc8dc13dc10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc8dc1d17d0 0x7fc8dc1d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 --2- 172.21.15.190:0/2457693846 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc8c8008e20 0x7fc8dc13fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:18.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 -- 172.21.15.190:0/2457693846 >> 172.21.15.190:0/2457693846 conn(0x7fc8dc0abe50 msgr2=0x7fc8dc0aaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:18.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 -- 172.21.15.190:0/2457693846 shutdown_connections 2024-08-19T10:40:18.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:18.564+0000 7fc8e14c6640 1 -- 172.21.15.190:0/2457693846 wait complete. 2024-08-19T10:40:18.582 INFO:tasks.workunit.client.0.smithi190.stdout:bar-asdf 2024-08-19T10:40:18.583 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rm foo 2024-08-19T10:40:18.583 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 'does not exist' 2024-08-19T10:40:18.999 INFO:tasks.workunit.client.0.smithi190.stdout:2024-08-19T10:40:18.884+0000 7f18857fa640 1 -- 172.21.15.190:0/3935454710 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v63) ==== 103+0+0 (secure 0 0 0) 0x7f187804f080 con 0x7f188809b6a0 2024-08-19T10:40:18.999 INFO:tasks.workunit.client.0.smithi190.stdout:2024-08-19T10:40:18.884+0000 7f18857fa640 10 monclient: _finish_command 2 = system:0 rule foo does not exist 2024-08-19T10:40:18.999 INFO:tasks.workunit.client.0.smithi190.stdout:2024-08-19T10:40:18.979+0000 7f18857fa640 1 -- 172.21.15.190:0/3935454710 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v63) ==== 103+0+0 (secure 0 0 0) 0x7f1878014a40 con 0x7f188809b6a0 2024-08-19T10:40:18.999 INFO:tasks.workunit.client.0.smithi190.stdout:2024-08-19T10:40:18.979+0000 7f18857fa640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-08-19T10:40:19.000 INFO:tasks.workunit.client.0.smithi190.stdout:rule foo does not exist 2024-08-19T10:40:19.000 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rm bar 2024-08-19T10:40:19.000 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 'does not exist' 2024-08-19T10:40:19.426 INFO:tasks.workunit.client.0.smithi190.stdout:2024-08-19T10:40:19.303+0000 7f03697fa640 1 -- 172.21.15.190:0/3628610890 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v63) ==== 103+0+0 (secure 0 0 0) 0x7f036c0a5790 con 0x7f036c13ec30 2024-08-19T10:40:19.426 INFO:tasks.workunit.client.0.smithi190.stdout:2024-08-19T10:40:19.303+0000 7f03697fa640 10 monclient: _finish_command 2 = system:0 rule bar does not exist 2024-08-19T10:40:19.426 INFO:tasks.workunit.client.0.smithi190.stdout:2024-08-19T10:40:19.403+0000 7f03697fa640 1 -- 172.21.15.190:0/3628610890 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v63) ==== 103+0+0 (secure 0 0 0) 0x7f0364016600 con 0x7f036c13ec30 2024-08-19T10:40:19.426 INFO:tasks.workunit.client.0.smithi190.stdout:2024-08-19T10:40:19.403+0000 7f03697fa640 10 monclient: _finish_command 3 = system:0 rule bar does not exist 2024-08-19T10:40:19.426 INFO:tasks.workunit.client.0.smithi190.stdout:rule bar does not exist 2024-08-19T10:40:19.427 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rename foo-asdf foo 2024-08-19T10:40:19.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient: get_monmap_and_config 2024-08-19T10:40:19.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient: build_initial_monmap 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient: monmap: 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:19.544744+0000 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:19.544744+0000 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:19.545 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:19.546 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:19.546 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:19.546 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:19.546 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:19.546 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:19.546 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:19.546 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:19.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 1 Processor -- start 2024-08-19T10:40:19.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 1 -- start start 2024-08-19T10:40:19.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient: init 2024-08-19T10:40:19.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:19.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient: _add_conns ranks=[2,6,0,9,1,7,5,4,8,10,3] 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97d81396a0 0x7f97d8139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient(hunting): picked mon.noname-b con 0x7f97d81396a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f97d8139fc0 0x7f97d80a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient(hunting): picked mon.noname-d con 0x7f97d8139fc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f97d80a6e20 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d80a83e0 con 0x7f97d80a6e20 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f97d80a4f20 con 0x7f97d81396a0 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f97d80a2f30 con 0x7f97d8139fc0 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97de8e1640 10 monclient(hunting): authenticate will time out at 1065.115516s 2024-08-19T10:40:19.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97d81396a0 0x7f97d8139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:19.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.539+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f97d8139fc0 0x7f97d80a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:19.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f97d8139fc0 0x7f97d80a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:44770/0 (socket says 172.21.15.190:44770) 2024-08-19T10:40:19.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 10 monclient(hunting): get_auth_request con 0x7f97d81396a0 auth_method 0 2024-08-19T10:40:19.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:19.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:19.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:19.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d77fe640 1 -- 172.21.15.190:0/1166870187 learned_addr learned my addr 172.21.15.190:0/1166870187 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:19.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97dce7e640 1 --1- 172.21.15.190:0/1166870187 >> v1:172.21.15.89:6789/0 conn(0x7f97d80a6e20 0x7f97d80a7220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:52480/0 (socket says 172.21.15.190:52480) 2024-08-19T10:40:19.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d77fe640 10 monclient(hunting): get_auth_request con 0x7f97d8139fc0 auth_method 0 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 10 monclient(hunting): handle_auth_done global_id 4808 payload 274 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 1 -- 172.21.15.190:0/1166870187 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f97d8139fc0 msgr2=0x7f97d80a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 1 --2- 172.21.15.190:0/1166870187 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f97d8139fc0 0x7f97d80a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:19.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 1 -- 172.21.15.190:0/1166870187 >> v1:172.21.15.89:6789/0 conn(0x7f97d80a6e20 legacy=0x7f97d80a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:19.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:19.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 1 monclient: found mon.noname-b 2024-08-19T10:40:19.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:19.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:19.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.543+0000 7f97d7fff640 1 -- 172.21.15.190:0/1166870187 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d80a35e0 con 0x7f97d81396a0 2024-08-19T10:40:19.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d7fff640 1 -- 172.21.15.190:0/1166870187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97d81396a0 msgr2=0x7f97d8139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:19.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d7fff640 1 --2- 172.21.15.190:0/1166870187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97d81396a0 0x7f97d8139a80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f97c80097f0 tx=0x7f97c801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d57e9c6380befb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:19.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 1 -- 172.21.15.190:0/1166870187 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97c80b0070 con 0x7f97d81396a0 2024-08-19T10:40:19.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:19.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:19.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 10 monclient: dump: 2024-08-19T10:40:19.556 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:19.556 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:19.556 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:19.556 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:19.557 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 5 monclient: authenticate success, global_id 4808 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 1 -- 172.21.15.190:0/1166870187 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c80aa810 con 0x7f97d81396a0 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 4 set_mon_vals no callback set 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 1 -- 172.21.15.190:0/1166870187 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97c80aae30 con 0x7f97d81396a0 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:19.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d6ffd640 10 monclient: dump: 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:19.559 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient: shutdown 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 -- 172.21.15.190:0/1166870187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97d81396a0 msgr2=0x7f97d8139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 --2- 172.21.15.190:0/1166870187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97d81396a0 0x7f97d8139a80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f97c80097f0 tx=0x7f97c801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 -- 172.21.15.190:0/1166870187 shutdown_connections 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 --2- 172.21.15.190:0/1166870187 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f97d8139fc0 0x7f97d80a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:19.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 --2- 172.21.15.190:0/1166870187 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f97d81396a0 0x7f97d8139a80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 -- 172.21.15.190:0/1166870187 >> 172.21.15.190:0/1166870187 conn(0x7f97d80abee0 msgr2=0x7f97d80ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 -- 172.21.15.190:0/1166870187 shutdown_connections 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 -- 172.21.15.190:0/1166870187 wait complete. 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient: build_initial_monmap 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient: monmap: 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:19.561 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 Processor -- start 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 -- start start 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient: init 2024-08-19T10:40:19.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient: _add_conns ranks=[1,3,7,2,5,6,8,4,0] 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97d80a6e20 0x7f97d81d1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient(hunting): picked mon.noname-f con 0x7f97d80a6e20 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d81396a0 0x7f97d81d1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient(hunting): picked mon.noname-g con 0x7f97d81396a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d8139fc0 0x7f97d81d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient(hunting): picked mon.noname-i con 0x7f97d8139fc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f97d813a3a0 con 0x7f97d80a6e20 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f97d80a7930 con 0x7f97d81396a0 2024-08-19T10:40:19.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f97d80a2690 con 0x7f97d8139fc0 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97de8e1640 10 monclient(hunting): authenticate will time out at 1065.123867s 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97dce7e640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d8139fc0 msgr2=0x7f97d81d1bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d7fff640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97d80a6e20 msgr2=0x7f97d81d1130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97dce7e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d8139fc0 0x7f97d81d1bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.547+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97d80a6e20 0x7f97d81d1130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d81396a0 0x7f97d81d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d81396a0 0x7f97d81d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47018/0 (socket says 172.21.15.190:47018) 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 1 -- 172.21.15.190:0/1734019544 learned_addr learned my addr 172.21.15.190:0/1734019544 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 10 monclient(hunting): get_auth_request con 0x7f97d81396a0 auth_method 0 2024-08-19T10:40:19.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 10 monclient(hunting): handle_auth_done global_id 4800 payload 824 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 1 -- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d8139fc0 msgr2=0x7f97d81d1bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d8139fc0 0x7f97d81d1bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 1 -- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97d80a6e20 msgr2=0x7f97d81d1130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97d80a6e20 0x7f97d81d1130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 1 monclient: found mon.noname-g 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:19.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:19.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 1 -- 172.21.15.190:0/1734019544 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d81404d0 con 0x7f97d81396a0 2024-08-19T10:40:19.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 1 -- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d81396a0 msgr2=0x7f97d81d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:19.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d77fe640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d81396a0 0x7f97d81d1670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e7f0 tx=0x7f97cc00ecb0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:19.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1734019544 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97cc017070 con 0x7f97d81396a0 2024-08-19T10:40:19.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:19.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:19.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 10 monclient: dump: 2024-08-19T10:40:19.569 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:19.569 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:19.569 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:19.569 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:19.569 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:19.569 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:19.569 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:19.569 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:19.569 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:19.569 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:19.569 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:19.570 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:19.570 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:19.570 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:19.570 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:19.570 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:19.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 10 monclient: _finish_auth 0 2024-08-19T10:40:19.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:40:19.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:19.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1734019544 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97cc013470 con 0x7f97d81396a0 2024-08-19T10:40:19.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:19.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1734019544 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f97cc013a90 con 0x7f97d81396a0 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97de8e1640 5 monclient: authenticate success, global_id 4800 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97de8e1640 10 monclient: _renew_subs 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97de8e1640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97de8e1640 1 -- 172.21.15.190:0/1734019544 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d8122cd0 con 0x7f97d81396a0 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d5ffb640 4 set_mon_vals no callback set 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 10 monclient: dump: 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:19.571 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97de8e1640 10 monclient: _renew_subs 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97de8e1640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:19.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97de8e1640 1 -- 172.21.15.190:0/1734019544 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d813fdd0 con 0x7f97d81396a0 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97de8e1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97de8e1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97de8e1640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97de8e1640 1 -- 172.21.15.190:0/1734019544 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d8139a80 con 0x7f97d81396a0 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.551+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1734019544 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f97cc01a070 con 0x7f97d81396a0 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.555+0000 7f97d4ff9640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97c403be00 0x7f97c403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.555+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1734019544 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 5227+0+0 (secure 0 0 0) 0x7f97cc00f040 con 0x7f97d81396a0 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.555+0000 7f97d7fff640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97c403be00 0x7f97c403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.555+0000 7f97d7fff640 10 monclient: get_auth_request con 0x7f97c403be00 auth_method 0 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.555+0000 7f97d7fff640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97c403be00 0x7f97c403e2c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f97c80ab000 tx=0x7f97c80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.559+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1734019544 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f97cc01a360 con 0x7f97d81396a0 2024-08-19T10:40:19.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.559+0000 7f97d4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:19.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.559+0000 7f97d4ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:19.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.715+0000 7f97de8e1640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:19.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.715+0000 7f97de8e1640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:19.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.715+0000 7f97de8e1640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:19.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:19.715+0000 7f97de8e1640 1 -- 172.21.15.190:0/1734019544 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) -- 0x7f97d806c9c0 con 0x7f97d81396a0 2024-08-19T10:40:20.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.211+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1734019544 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 v64) ==== 110+0+0 (secure 0 0 0) 0x7f97cc013620 con 0x7f97d81396a0 2024-08-19T10:40:20.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.211+0000 7f97d4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:20.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.211+0000 7f97d4ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.319+0000 7f97de8e1640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.319+0000 7f97de8e1640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.319+0000 7f97de8e1640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.319+0000 7f97de8e1640 1 -- 172.21.15.190:0/1734019544 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) -- 0x7f97d8140aa0 con 0x7f97d81396a0 2024-08-19T10:40:20.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.319+0000 7f97d4ff9640 1 -- 172.21.15.190:0/1734019544 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v65) ==== 134+0+0 (secure 0 0 0) 0x7f97cc00cad0 con 0x7f97d81396a0 2024-08-19T10:40:20.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.319+0000 7f97d4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:20.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.319+0000 7f97d4ff9640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo' 2024-08-19T10:40:20.327 INFO:tasks.workunit.client.0.smithi190.stderr:already renamed to 'foo' 2024-08-19T10:40:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 -- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97c403be00 msgr2=0x7f97c403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97c403be00 0x7f97c403e2c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f97c80ab000 tx=0x7f97c80a6000 comp rx=0 tx=0).stop 2024-08-19T10:40:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 10 monclient: shutdown 2024-08-19T10:40:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 -- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d81396a0 msgr2=0x7f97d81d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d81396a0 0x7f97d81d1670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e7f0 tx=0x7f97cc00ecb0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 -- 172.21.15.190:0/1734019544 shutdown_connections 2024-08-19T10:40:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f97c403be00 0x7f97c403e2c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f97d8139fc0 0x7f97d81d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97d81396a0 0x7f97d81d1670 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 --2- 172.21.15.190:0/1734019544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97d80a6e20 0x7f97d81d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 -- 172.21.15.190:0/1734019544 >> 172.21.15.190:0/1734019544 conn(0x7f97d80abee0 msgr2=0x7f97d80aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 -- 172.21.15.190:0/1734019544 shutdown_connections 2024-08-19T10:40:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.323+0000 7f97de8e1640 1 -- 172.21.15.190:0/1734019544 wait complete. 2024-08-19T10:40:20.343 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rename foo-asdf foo 2024-08-19T10:40:20.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.451+0000 7fca1dfab640 10 monclient: get_monmap_and_config 2024-08-19T10:40:20.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.451+0000 7fca1dfab640 10 monclient: build_initial_monmap 2024-08-19T10:40:20.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient: monmap: 2024-08-19T10:40:20.459 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:20.459 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:20.459 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:20.459588+0000 2024-08-19T10:40:20.459 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:20.459588+0000 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:20.460 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 1 Processor -- start 2024-08-19T10:40:20.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 1 -- start start 2024-08-19T10:40:20.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient: init 2024-08-19T10:40:20.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:20.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient: _add_conns ranks=[5,6,7,10,8,2,3,4,9,1,0] 2024-08-19T10:40:20.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fca181396a0 0x7fca18139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient(hunting): picked mon.noname-h con 0x7fca181396a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca18139fc0 0x7fca180a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient(hunting): picked mon.noname-d con 0x7fca18139fc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca180a6e20 0x7fca180a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient(hunting): picked mon.noname-i con 0x7fca180a6e20 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fca180a4f20 con 0x7fca181396a0 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fca180a83e0 con 0x7fca18139fc0 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fca180a2f30 con 0x7fca180a6e20 2024-08-19T10:40:20.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca1dfab640 10 monclient(hunting): authenticate will time out at 1066.029443s 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca17fff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca180a6e20 msgr2=0x7fca180a7220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca17fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca180a6e20 0x7fca180a7220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fca181396a0 0x7fca18139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fca181396a0 0x7fca18139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:55410/0 (socket says 172.21.15.190:55410) 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 1 -- 172.21.15.190:0/326085578 learned_addr learned my addr 172.21.15.190:0/326085578 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 10 monclient(hunting): get_auth_request con 0x7fca181396a0 auth_method 0 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:20.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca16ffd640 1 --2- 172.21.15.190:0/326085578 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca18139fc0 0x7fca180a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca16ffd640 10 monclient(hunting): get_auth_request con 0x7fca18139fc0 auth_method 0 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca16ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca16ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca16ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 10 monclient(hunting): handle_auth_done global_id 4847 payload 274 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 1 -- 172.21.15.190:0/326085578 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca180a6e20 msgr2=0x7fca180a7220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 1 --2- 172.21.15.190:0/326085578 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca180a6e20 0x7fca180a7220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 1 -- 172.21.15.190:0/326085578 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca18139fc0 msgr2=0x7fca180a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 1 --2- 172.21.15.190:0/326085578 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca18139fc0 0x7fca180a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:20.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 1 monclient: found mon.noname-h 2024-08-19T10:40:20.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:20.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:20.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.455+0000 7fca177fe640 1 -- 172.21.15.190:0/326085578 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca180a35e0 con 0x7fca181396a0 2024-08-19T10:40:20.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca177fe640 1 -- 172.21.15.190:0/326085578 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fca181396a0 msgr2=0x7fca18139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:20.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca177fe640 1 --2- 172.21.15.190:0/326085578 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fca181396a0 0x7fca18139a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca080098a0 tx=0x7fca0801c460 comp rx=0 tx=0).ready entity=mon.5 client_cookie=b9b814fbe1f5d089 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:20.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 1 -- 172.21.15.190:0/326085578 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca080b0070 con 0x7fca181396a0 2024-08-19T10:40:20.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:20.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:20.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 10 monclient: dump: 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:20.473 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 5 monclient: authenticate success, global_id 4847 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 1 -- 172.21.15.190:0/326085578 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca080aa810 con 0x7fca181396a0 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 4 set_mon_vals no callback set 2024-08-19T10:40:20.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 1 -- 172.21.15.190:0/326085578 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca080aae30 con 0x7fca181396a0 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca167fc640 10 monclient: dump: 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:20.475 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 10 monclient: shutdown 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 1 -- 172.21.15.190:0/326085578 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fca181396a0 msgr2=0x7fca18139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 1 --2- 172.21.15.190:0/326085578 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fca181396a0 0x7fca18139a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca080098a0 tx=0x7fca0801c460 comp rx=0 tx=0).stop 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 1 -- 172.21.15.190:0/326085578 shutdown_connections 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 1 --2- 172.21.15.190:0/326085578 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca180a6e20 0x7fca180a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 1 --2- 172.21.15.190:0/326085578 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca18139fc0 0x7fca180a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 1 --2- 172.21.15.190:0/326085578 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fca181396a0 0x7fca18139a80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 1 -- 172.21.15.190:0/326085578 >> 172.21.15.190:0/326085578 conn(0x7fca180abee0 msgr2=0x7fca180ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 1 -- 172.21.15.190:0/326085578 shutdown_connections 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 1 -- 172.21.15.190:0/326085578 wait complete. 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 10 monclient: build_initial_monmap 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.459+0000 7fca1dfab640 10 monclient: monmap: 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:20.477 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 1 Processor -- start 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 1 -- start start 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient: init 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:20.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient: _add_conns ranks=[1,3,4,5,7,8,2,6,0] 2024-08-19T10:40:20.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca180a6e20 0x7fca1813f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient(hunting): picked mon.noname-f con 0x7fca180a6e20 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:20.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca181396a0 0x7fca1809cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient(hunting): picked mon.noname-g con 0x7fca181396a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:20.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca18139fc0 0x7fca1809d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient(hunting): picked mon.noname-c con 0x7fca18139fc0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:20.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca1813a3a0 con 0x7fca180a6e20 2024-08-19T10:40:20.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fca18143390 con 0x7fca181396a0 2024-08-19T10:40:20.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fca180ac2f0 con 0x7fca18139fc0 2024-08-19T10:40:20.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:20.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient(hunting): authenticate will time out at 1066.036601s 2024-08-19T10:40:20.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca177fe640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca180a6e20 msgr2=0x7fca1813f060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:20.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca177fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca180a6e20 0x7fca1813f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:20.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca181396a0 0x7fca1809cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:20.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca181396a0 0x7fca1809cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47024/0 (socket says 172.21.15.190:47024) 2024-08-19T10:40:20.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 1 -- 172.21.15.190:0/1014124067 learned_addr learned my addr 172.21.15.190:0/1014124067 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:20.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 10 monclient(hunting): get_auth_request con 0x7fca181396a0 auth_method 0 2024-08-19T10:40:20.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:20.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:20.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:20.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:20.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:20.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:20.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca17fff640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca18139fc0 0x7fca1809d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:20.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 10 monclient(hunting): handle_auth_done global_id 4809 payload 824 2024-08-19T10:40:20.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 1 -- 172.21.15.190:0/1014124067 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca18139fc0 msgr2=0x7fca1809d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:20.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca18139fc0 0x7fca1809d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 1 -- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca180a6e20 msgr2=0x7fca1813f060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:20.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca180a6e20 0x7fca1813f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 1 monclient: found mon.noname-g 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 1 -- 172.21.15.190:0/1014124067 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1809e2c0 con 0x7fca181396a0 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca17fff640 10 monclient: get_auth_request con 0x7fca18139fc0 auth_method 0 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca17fff640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca18139fc0 0x7fca1809d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 1 -- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca181396a0 msgr2=0x7fca1809cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca16ffd640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca181396a0 0x7fca1809cf70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00d620 tx=0x7fca0c00dae0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 1 -- 172.21.15.190:0/1014124067 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca0c007380 con 0x7fca181396a0 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:20.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 10 monclient: dump: 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:20.483 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 10 monclient: _finish_auth 0 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 1 -- 172.21.15.190:0/1014124067 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c019070 con 0x7fca181396a0 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 5 monclient: authenticate success, global_id 4809 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient: _renew_subs 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 1 -- 172.21.15.190:0/1014124067 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1813faf0 con 0x7fca181396a0 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:20.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 1 -- 172.21.15.190:0/1014124067 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca0c00bca0 con 0x7fca181396a0 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca15ffb640 4 set_mon_vals no callback set 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fc9fffff640 10 monclient: dump: 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:20.485 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient: _renew_subs 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.463+0000 7fca1dfab640 1 -- 172.21.15.190:0/1014124067 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1809f220 con 0x7fca181396a0 2024-08-19T10:40:20.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.467+0000 7fca1dfab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:20.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.467+0000 7fca1dfab640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:20.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.467+0000 7fca1dfab640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:20.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.467+0000 7fca1dfab640 1 -- 172.21.15.190:0/1014124067 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca1809f420 con 0x7fca181396a0 2024-08-19T10:40:20.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.467+0000 7fc9fffff640 1 -- 172.21.15.190:0/1014124067 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fca0c00b830 con 0x7fca181396a0 2024-08-19T10:40:20.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.467+0000 7fc9fffff640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca0403be00 0x7fca0403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.467+0000 7fc9fffff640 1 -- 172.21.15.190:0/1014124067 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(65..65 src has 1..65) ==== 5222+0+0 (secure 0 0 0) 0x7fca0c00f040 con 0x7fca181396a0 2024-08-19T10:40:20.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.467+0000 7fca177fe640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca0403be00 0x7fca0403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:20.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.467+0000 7fca177fe640 10 monclient: get_auth_request con 0x7fca0403be00 auth_method 0 2024-08-19T10:40:20.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.467+0000 7fca177fe640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca0403be00 0x7fca0403e2c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fca080098a0 tx=0x7fca08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:20.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.471+0000 7fc9fffff640 1 -- 172.21.15.190:0/1014124067 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fca1809f420 con 0x7fca181396a0 2024-08-19T10:40:20.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.471+0000 7fc9fffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:20.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.471+0000 7fc9fffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:20.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.643+0000 7fca1dfab640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:20.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.643+0000 7fca1dfab640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:20.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.643+0000 7fca1dfab640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:20.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.643+0000 7fca1dfab640 1 -- 172.21.15.190:0/1014124067 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) -- 0x7fca18139a80 con 0x7fca181396a0 2024-08-19T10:40:20.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.647+0000 7fc9fffff640 1 -- 172.21.15.190:0/1014124067 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v65) ==== 134+0+0 (secure 0 0 0) 0x7fca0c01cd70 con 0x7fca181396a0 2024-08-19T10:40:20.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.647+0000 7fc9fffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:20.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.647+0000 7fc9fffff640 10 monclient: _finish_command 2 = system:0 already renamed to 'foo' 2024-08-19T10:40:20.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.751+0000 7fca1dfab640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:20.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.751+0000 7fca1dfab640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:20.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.751+0000 7fca1dfab640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:20.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.751+0000 7fca1dfab640 1 -- 172.21.15.190:0/1014124067 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"} v 0) -- 0x7fca180008e0 con 0x7fca181396a0 2024-08-19T10:40:20.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.755+0000 7fc9fffff640 1 -- 172.21.15.190:0/1014124067 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}]=0 already renamed to 'foo' v65) ==== 134+0+0 (secure 0 0 0) 0x7fca0c040070 con 0x7fca181396a0 2024-08-19T10:40:20.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.755+0000 7fc9fffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "foo-asdf", "dstname": "foo"}] 2024-08-19T10:40:20.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.755+0000 7fc9fffff640 10 monclient: _finish_command 3 = system:0 already renamed to 'foo' 2024-08-19T10:40:20.761 INFO:tasks.workunit.client.0.smithi190.stderr:already renamed to 'foo' 2024-08-19T10:40:20.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.755+0000 7fca1dfab640 1 -- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca0403be00 msgr2=0x7fca0403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:20.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.755+0000 7fca1dfab640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca0403be00 0x7fca0403e2c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fca080098a0 tx=0x7fca08008040 comp rx=0 tx=0).stop 2024-08-19T10:40:20.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.755+0000 7fca1dfab640 10 monclient: shutdown 2024-08-19T10:40:20.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.755+0000 7fca1dfab640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:20.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.755+0000 7fca1dfab640 1 -- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca181396a0 msgr2=0x7fca1809cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:20.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.755+0000 7fca1dfab640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca181396a0 0x7fca1809cf70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00d620 tx=0x7fca0c00dae0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.759+0000 7fca1dfab640 1 -- 172.21.15.190:0/1014124067 shutdown_connections 2024-08-19T10:40:20.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.759+0000 7fca1dfab640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca0403be00 0x7fca0403e2c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.759+0000 7fca1dfab640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca181396a0 0x7fca1809cf70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.759+0000 7fca1dfab640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fca18139fc0 0x7fca1809d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.759+0000 7fca1dfab640 1 --2- 172.21.15.190:0/1014124067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca180a6e20 0x7fca1813f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.759+0000 7fca1dfab640 1 -- 172.21.15.190:0/1014124067 >> 172.21.15.190:0/1014124067 conn(0x7fca180abee0 msgr2=0x7fca180aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:20.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.759+0000 7fca1dfab640 1 -- 172.21.15.190:0/1014124067 shutdown_connections 2024-08-19T10:40:20.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.759+0000 7fca1dfab640 1 -- 172.21.15.190:0/1014124067 wait complete. 2024-08-19T10:40:20.778 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rename bar-asdf bar 2024-08-19T10:40:20.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient: get_monmap_and_config 2024-08-19T10:40:20.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient: build_initial_monmap 2024-08-19T10:40:20.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient: monmap: 2024-08-19T10:40:20.895 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:20.895 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:20.896 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:20.895815+0000 2024-08-19T10:40:20.896 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:20.895815+0000 2024-08-19T10:40:20.896 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:20.896 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.896 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:20.896 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:20.896 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:20.896 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:20.896 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:20.896 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:20.896 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:20.897 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:20.897 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:20.897 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:20.897 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:20.897 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 1 Processor -- start 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 1 -- start start 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient: init 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient: _add_conns ranks=[3,8,7,0,5,4,10,2,1,6,9] 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900a68e0 0x7fca900a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient(hunting): picked mon.noname-g con 0x7fca900a68e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fca900a6cc0 0x7fca900a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient(hunting): picked mon.noname-e con 0x7fca900a6cc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca901396a0 0x7fca90139ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient(hunting): picked mon.noname-i con 0x7fca901396a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fca900a86c0 con 0x7fca900a68e0 2024-08-19T10:40:20.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fca900a8410 con 0x7fca901396a0 2024-08-19T10:40:20.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fca900a63a0 con 0x7fca900a6cc0 2024-08-19T10:40:20.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:20.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca98084640 10 monclient(hunting): authenticate will time out at 1066.465710s 2024-08-19T10:40:20.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca96621640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca901396a0 msgr2=0x7fca90139ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:20.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca96621640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca901396a0 0x7fca90139ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:20.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900a68e0 0x7fca900a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:20.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900a68e0 0x7fca900a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47040/0 (socket says 172.21.15.190:47040) 2024-08-19T10:40:20.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 1 -- 172.21.15.190:0/3409303281 learned_addr learned my addr 172.21.15.190:0/3409303281 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:20.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 10 monclient(hunting): get_auth_request con 0x7fca900a68e0 auth_method 0 2024-08-19T10:40:20.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:20.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:20.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:20.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:20.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:20.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:20.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca9561f640 1 --2- 172.21.15.190:0/3409303281 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fca900a6cc0 0x7fca900a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:20.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 10 monclient(hunting): handle_auth_done global_id 4818 payload 274 2024-08-19T10:40:20.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 1 -- 172.21.15.190:0/3409303281 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fca900a6cc0 msgr2=0x7fca900a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:20.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 1 --2- 172.21.15.190:0/3409303281 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fca900a6cc0 0x7fca900a70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 1 -- 172.21.15.190:0/3409303281 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca901396a0 msgr2=0x7fca90139ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:20.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 1 --2- 172.21.15.190:0/3409303281 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca901396a0 0x7fca90139ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 10 monclient: _finish_hunting 0 2024-08-19T10:40:20.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 1 monclient: found mon.noname-g 2024-08-19T10:40:20.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:20.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:20.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.891+0000 7fca95e20640 1 -- 172.21.15.190:0/3409303281 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca900a4680 con 0x7fca900a68e0 2024-08-19T10:40:20.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca95e20640 1 -- 172.21.15.190:0/3409303281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900a68e0 msgr2=0x7fca900a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:20.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca95e20640 1 --2- 172.21.15.190:0/3409303281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900a68e0 0x7fca900a5760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0098a0 tx=0x7fca8c01c140 comp rx=0 tx=0).ready entity=mon.3 client_cookie=ade90b55862cc6cb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:20.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 1 -- 172.21.15.190:0/3409303281 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca8c0ab070 con 0x7fca900a68e0 2024-08-19T10:40:20.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:20.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:20.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 10 monclient: dump: 2024-08-19T10:40:20.909 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:20.909 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:20.910 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:20.911 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:20.911 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 10 monclient: _finish_auth 0 2024-08-19T10:40:20.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 10 monclient: _check_auth_tickets 2024-08-19T10:40:20.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:20.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 1 -- 172.21.15.190:0/3409303281 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca8c0a5970 con 0x7fca900a68e0 2024-08-19T10:40:20.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 5 monclient: authenticate success, global_id 4818 2024-08-19T10:40:20.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:20.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:20.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:20.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 1 -- 172.21.15.190:0/3409303281 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca8c0a6400 con 0x7fca900a68e0 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 4 set_mon_vals no callback set 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca94e1e640 10 monclient: dump: 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:20.912 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 10 monclient: shutdown 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 1 -- 172.21.15.190:0/3409303281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900a68e0 msgr2=0x7fca900a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 1 --2- 172.21.15.190:0/3409303281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900a68e0 0x7fca900a5760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0098a0 tx=0x7fca8c01c140 comp rx=0 tx=0).stop 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 1 -- 172.21.15.190:0/3409303281 shutdown_connections 2024-08-19T10:40:20.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 1 --2- 172.21.15.190:0/3409303281 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fca900a6cc0 0x7fca900a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 1 --2- 172.21.15.190:0/3409303281 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca901396a0 0x7fca90139ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 1 --2- 172.21.15.190:0/3409303281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca900a68e0 0x7fca900a5760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 1 -- 172.21.15.190:0/3409303281 >> 172.21.15.190:0/3409303281 conn(0x7fca900abee0 msgr2=0x7fca900ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 1 -- 172.21.15.190:0/3409303281 shutdown_connections 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.895+0000 7fca98084640 1 -- 172.21.15.190:0/3409303281 wait complete. 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient: build_initial_monmap 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient: monmap: 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:20.914 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 1 Processor -- start 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 1 -- start start 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient: init 2024-08-19T10:40:20.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient: _add_conns ranks=[0,1,2,3,6,7,8,4,5] 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca900a68e0 0x7fca901d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient(hunting): picked mon.noname-a con 0x7fca900a68e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca900a6cc0 0x7fca900a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient(hunting): picked mon.noname-f con 0x7fca900a6cc0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca901396a0 0x7fca900a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient(hunting): picked mon.noname-b con 0x7fca901396a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca90143cc0 con 0x7fca900a68e0 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca90143b40 con 0x7fca900a6cc0 2024-08-19T10:40:20.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:20.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fca90143e40 con 0x7fca901396a0 2024-08-19T10:40:20.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:20.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca98084640 10 monclient(hunting): authenticate will time out at 1066.473072s 2024-08-19T10:40:20.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca9561f640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca900a6cc0 msgr2=0x7fca900a1fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:20.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca9561f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca900a6cc0 0x7fca900a1fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:20.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca900a68e0 0x7fca901d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:20.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca96621640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca901396a0 0x7fca900a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:20.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca900a68e0 0x7fca901d23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:33138/0 (socket says 172.21.15.190:33138) 2024-08-19T10:40:20.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca96621640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca901396a0 0x7fca900a2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:60676/0 (socket says 172.21.15.190:60676) 2024-08-19T10:40:20.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 1 -- 172.21.15.190:0/410045049 learned_addr learned my addr 172.21.15.190:0/410045049 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:20.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 10 monclient(hunting): get_auth_request con 0x7fca900a68e0 auth_method 0 2024-08-19T10:40:20.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:20.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:20.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:20.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca96621640 10 monclient(hunting): get_auth_request con 0x7fca901396a0 auth_method 0 2024-08-19T10:40:20.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca96621640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:20.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca96621640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:20.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca96621640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca96621640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca96621640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca96621640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 10 monclient(hunting): handle_auth_done global_id 4941 payload 824 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 1 -- 172.21.15.190:0/410045049 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca901396a0 msgr2=0x7fca900a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca901396a0 0x7fca900a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 1 -- 172.21.15.190:0/410045049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca900a6cc0 msgr2=0x7fca900a1fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca900a6cc0 0x7fca900a1fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 10 monclient: _finish_hunting 0 2024-08-19T10:40:20.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 1 monclient: found mon.noname-a 2024-08-19T10:40:20.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:20.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:20.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca95e20640 1 -- 172.21.15.190:0/410045049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9009d020 con 0x7fca900a68e0 2024-08-19T10:40:20.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.899+0000 7fca96621640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca901396a0 0x7fca900a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:20.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca95e20640 1 -- 172.21.15.190:0/410045049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca900a68e0 msgr2=0x7fca901d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:20.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca95e20640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca900a68e0 0x7fca901d23b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0097c0 tx=0x7fca8c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:20.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 1 -- 172.21.15.190:0/410045049 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:40:20.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 1 -- 172.21.15.190:0/410045049 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca8c0ab070 con 0x7fca900a68e0 2024-08-19T10:40:20.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:20.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 10 monclient: dump: 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:20.923 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:20.924 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:20.924 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:20.924 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:20.924 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:20.924 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:20.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:20.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:20.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca98084640 5 monclient: authenticate success, global_id 4941 2024-08-19T10:40:20.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca98084640 10 monclient: _renew_subs 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca98084640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 1 -- 172.21.15.190:0/410045049 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca8c0b4030 con 0x7fca900a68e0 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca98084640 1 -- 172.21.15.190:0/410045049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9013f820 con 0x7fca900a68e0 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 1 -- 172.21.15.190:0/410045049 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca8c0a5e00 con 0x7fca900a68e0 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca87fff640 4 set_mon_vals no callback set 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca867fc640 10 monclient: dump: 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:20.925 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:20.926 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca98084640 10 monclient: _renew_subs 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca98084640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca98084640 1 -- 172.21.15.190:0/410045049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9009d2e0 con 0x7fca900a68e0 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca98084640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca98084640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca98084640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.903+0000 7fca98084640 1 -- 172.21.15.190:0/410045049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca900a70a0 con 0x7fca900a68e0 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.907+0000 7fca867fc640 1 -- 172.21.15.190:0/410045049 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fca8c0ae070 con 0x7fca900a68e0 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.907+0000 7fca867fc640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca7403be00 0x7fca7403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.907+0000 7fca867fc640 1 -- 172.21.15.190:0/410045049 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5222+0+0 (secure 0 0 0) 0x7fca8c0e7d00 con 0x7fca900a68e0 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.907+0000 7fca9561f640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca7403be00 0x7fca7403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:20.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.907+0000 7fca9561f640 10 monclient: get_auth_request con 0x7fca7403be00 auth_method 0 2024-08-19T10:40:20.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.907+0000 7fca9561f640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca7403be00 0x7fca7403e2c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca9013fe90 tx=0x7fca80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:20.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.919+0000 7fca867fc640 1 -- 172.21.15.190:0/410045049 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fca8c016170 con 0x7fca900a68e0 2024-08-19T10:40:20.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.919+0000 7fca867fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:20.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:20.919+0000 7fca867fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:21.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.091+0000 7fca98084640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-08-19T10:40:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.091+0000 7fca98084640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-08-19T10:40:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.091+0000 7fca98084640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:21.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.091+0000 7fca98084640 1 -- 172.21.15.190:0/410045049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"} v 0) -- 0x7fca900a2a60 con 0x7fca900a68e0 2024-08-19T10:40:21.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.247+0000 7fca867fc640 1 -- 172.21.15.190:0/410045049 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}]=0 v66) ==== 110+0+0 (secure 0 0 0) 0x7fca8c0a6d00 con 0x7fca900a68e0 2024-08-19T10:40:21.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.247+0000 7fca867fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-08-19T10:40:21.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.247+0000 7fca867fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:21.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.347+0000 7fca98084640 10 start_mon_command cmd=[{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-08-19T10:40:21.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.347+0000 7fca98084640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-08-19T10:40:21.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.347+0000 7fca98084640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:21.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.347+0000 7fca98084640 1 -- 172.21.15.190:0/410045049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"} v 0) -- 0x7fca900008e0 con 0x7fca900a68e0 2024-08-19T10:40:21.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.347+0000 7fca867fc640 1 -- 172.21.15.190:0/410045049 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}]=0 already renamed to 'bar' v66) ==== 134+0+0 (secure 0 0 0) 0x7fca8c0c8030 con 0x7fca900a68e0 2024-08-19T10:40:21.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.347+0000 7fca867fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rename", "srcname": "bar-asdf", "dstname": "bar"}] 2024-08-19T10:40:21.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.347+0000 7fca867fc640 10 monclient: _finish_command 3 = system:0 already renamed to 'bar' 2024-08-19T10:40:21.354 INFO:tasks.workunit.client.0.smithi190.stderr:already renamed to 'bar' 2024-08-19T10:40:21.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 -- 172.21.15.190:0/410045049 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca7403be00 msgr2=0x7fca7403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:21.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca7403be00 0x7fca7403e2c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca9013fe90 tx=0x7fca80008040 comp rx=0 tx=0).stop 2024-08-19T10:40:21.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 10 monclient: shutdown 2024-08-19T10:40:21.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:21.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 -- 172.21.15.190:0/410045049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca900a68e0 msgr2=0x7fca901d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:21.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca900a68e0 0x7fca901d23b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0097c0 tx=0x7fca8c019200 comp rx=0 tx=0).stop 2024-08-19T10:40:21.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 -- 172.21.15.190:0/410045049 shutdown_connections 2024-08-19T10:40:21.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fca7403be00 0x7fca7403e2c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fca901396a0 0x7fca900a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca900a6cc0 0x7fca900a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 --2- 172.21.15.190:0/410045049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca900a68e0 0x7fca901d23b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 -- 172.21.15.190:0/410045049 >> 172.21.15.190:0/410045049 conn(0x7fca900abee0 msgr2=0x7fca901350d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:21.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 -- 172.21.15.190:0/410045049 shutdown_connections 2024-08-19T10:40:21.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.351+0000 7fca98084640 1 -- 172.21.15.190:0/410045049 wait complete. 2024-08-19T10:40:21.371 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule ls 2024-08-19T10:40:21.371 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false grep foo-asdf 2024-08-19T10:40:21.371 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:40:21.371 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep foo-asdf 2024-08-19T10:40:21.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.479+0000 7f935bd83640 10 monclient: get_monmap_and_config 2024-08-19T10:40:21.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.479+0000 7f935bd83640 10 monclient: build_initial_monmap 2024-08-19T10:40:21.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.479+0000 7f935bd83640 10 monclient: monmap: 2024-08-19T10:40:21.486 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:21.486 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:21.486 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:21.485748+0000 2024-08-19T10:40:21.486 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:21.485748+0000 2024-08-19T10:40:21.486 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:21.486 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.486 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:21.486 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:21.486 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:21.487 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:21.487 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:21.487 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:21.487 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:21.487 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:21.487 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:21.487 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:21.487 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:21.487 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.479+0000 7f935bd83640 1 Processor -- start 2024-08-19T10:40:21.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.479+0000 7f935bd83640 1 -- start start 2024-08-19T10:40:21.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.479+0000 7f935bd83640 10 monclient: init 2024-08-19T10:40:21.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:21.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 10 monclient: _add_conns ranks=[7,0,6,5,3,2,9,10,4,8,1] 2024-08-19T10:40:21.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f935406c8d0 0x7f935406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 10 monclient(hunting): picked mon.noname-i con 0x7f935406c8d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f935409d530 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f935406d1f0 0x7f9354137c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 10 monclient(hunting): picked mon.noname-d con 0x7f935406d1f0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935409ff00 con 0x7f935409d530 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f93540a5060 con 0x7f935406d1f0 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f93540a5340 con 0x7f935406c8d0 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935bd83640 10 monclient(hunting): authenticate will time out at 1067.056417s 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f9359b1f640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f935406c8d0 msgr2=0x7f935406ccb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:21.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f9359b1f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f935406c8d0 0x7f935406ccb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f935406d1f0 0x7f9354137c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935931e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f935409d530 0x7f9354137580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:52486/0 (socket says 172.21.15.190:52486) 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935931e640 1 -- 172.21.15.190:0/641802445 learned_addr learned my addr 172.21.15.190:0/641802445 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 10 monclient(hunting): get_auth_request con 0x7f935406d1f0 auth_method 0 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 10 monclient(hunting): handle_auth_done global_id 4803 payload 274 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 1 -- 172.21.15.190:0/641802445 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f935406c8d0 msgr2=0x7f935406ccb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 1 --2- 172.21.15.190:0/641802445 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f935406c8d0 0x7f935406ccb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 1 -- 172.21.15.190:0/641802445 >> v1:172.21.15.89:6789/0 conn(0x7f935409d530 legacy=0x7f9354137580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:21.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 10 monclient: _finish_hunting 0 2024-08-19T10:40:21.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 1 monclient: found mon.noname-d 2024-08-19T10:40:21.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:21.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:21.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.483+0000 7f935a320640 1 -- 172.21.15.190:0/641802445 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93540a2910 con 0x7f935406d1f0 2024-08-19T10:40:21.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935a320640 1 -- 172.21.15.190:0/641802445 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f935406d1f0 msgr2=0x7f9354137c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:21.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935a320640 1 --2- 172.21.15.190:0/641802445 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f935406d1f0 0x7f9354137c90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f934c009dc0 tx=0x7f934c01c900 comp rx=0 tx=0).ready entity=mon.6 client_cookie=dcd42d8f68a90cfc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:21.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 1 -- 172.21.15.190:0/641802445 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f934c0b2070 con 0x7f935406d1f0 2024-08-19T10:40:21.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 10 monclient: dump: 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:21.494 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 10 monclient: _finish_auth 0 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 10 monclient: _check_auth_tickets 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 1 -- 172.21.15.190:0/641802445 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f934c0a5b40 con 0x7f935406d1f0 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 5 monclient: authenticate success, global_id 4803 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:21.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 1 -- 172.21.15.190:0/641802445 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f934c0a5cf0 con 0x7f935406d1f0 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 4 set_mon_vals no callback set 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f9358b1d640 10 monclient: dump: 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:21.496 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:21.497 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:21.497 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:21.497 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:21.497 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:21.497 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:21.497 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:21.497 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 10 monclient: shutdown 2024-08-19T10:40:21.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:21.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 1 -- 172.21.15.190:0/641802445 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f935406d1f0 msgr2=0x7f9354137c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:21.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 1 --2- 172.21.15.190:0/641802445 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f935406d1f0 0x7f9354137c90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f934c009dc0 tx=0x7f934c01c900 comp rx=0 tx=0).stop 2024-08-19T10:40:21.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 1 -- 172.21.15.190:0/641802445 shutdown_connections 2024-08-19T10:40:21.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 1 --2- 172.21.15.190:0/641802445 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f935406d1f0 0x7f9354137c90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 1 --2- 172.21.15.190:0/641802445 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f935406c8d0 0x7f935406ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 1 -- 172.21.15.190:0/641802445 >> 172.21.15.190:0/641802445 conn(0x7f935406dde0 msgr2=0x7f935406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:21.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 1 -- 172.21.15.190:0/641802445 shutdown_connections 2024-08-19T10:40:21.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 1 -- 172.21.15.190:0/641802445 wait complete. 2024-08-19T10:40:21.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 10 monclient: build_initial_monmap 2024-08-19T10:40:21.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.487+0000 7f935bd83640 10 monclient: monmap: 2024-08-19T10:40:21.503 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:21.503 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:21.503 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:21.503 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:21.503 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 1 Processor -- start 2024-08-19T10:40:21.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 1 -- start start 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient: init 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient: _add_conns ranks=[3,5,2,8,7,0,6,1,4] 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f935406c8d0 0x7f9354134f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient(hunting): picked mon.noname-g con 0x7f935406c8d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f935406d1f0 0x7f93541354a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient(hunting): picked mon.noname-h con 0x7f935406d1f0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f935409d530 0x7f93541359e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient(hunting): picked mon.noname-b con 0x7f935409d530 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f93541381d0 con 0x7f935409d530 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f93540a3c90 con 0x7f935406c8d0 2024-08-19T10:40:21.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f9354138350 con 0x7f935406d1f0 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient(hunting): authenticate will time out at 1067.064835s 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f935406c8d0 0x7f9354134f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935931e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f935406d1f0 0x7f93541354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f935406c8d0 0x7f9354134f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47044/0 (socket says 172.21.15.190:47044) 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 1 -- 172.21.15.190:0/3139102440 learned_addr learned my addr 172.21.15.190:0/3139102440 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 10 monclient(hunting): get_auth_request con 0x7f935406c8d0 auth_method 0 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935931e640 10 monclient(hunting): get_auth_request con 0x7f935406d1f0 auth_method 0 2024-08-19T10:40:21.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935931e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935931e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935931e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935931e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935931e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935931e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935a320640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f935409d530 0x7f93541359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935a320640 10 monclient(hunting): get_auth_request con 0x7f935409d530 auth_method 0 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935a320640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935a320640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:21.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935a320640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 10 monclient(hunting): handle_auth_done global_id 4827 payload 824 2024-08-19T10:40:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 1 -- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f935406d1f0 msgr2=0x7f93541354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f935406d1f0 0x7f93541354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 1 -- 172.21.15.190:0/3139102440 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f935409d530 msgr2=0x7f93541359e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f935409d530 0x7f93541359e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 10 monclient: _finish_hunting 0 2024-08-19T10:40:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 1 monclient: found mon.noname-g 2024-08-19T10:40:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 1 -- 172.21.15.190:0/3139102440 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9354137bd0 con 0x7f935406c8d0 2024-08-19T10:40:21.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 1 -- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f935406c8d0 msgr2=0x7f9354134f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f9359b1f640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f935406c8d0 0x7f9354134f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f935000b550 tx=0x7f935000ba10 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 1 -- 172.21.15.190:0/3139102440 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93500092d0 con 0x7f935406c8d0 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 10 monclient: dump: 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:21.509 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:21.510 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:21.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 5 monclient: authenticate success, global_id 4827 2024-08-19T10:40:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 1 -- 172.21.15.190:0/3139102440 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9350015070 con 0x7f935406c8d0 2024-08-19T10:40:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient: _renew_subs 2024-08-19T10:40:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 1 -- 172.21.15.190:0/3139102440 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9350003de0 con 0x7f935406c8d0 2024-08-19T10:40:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 1 -- 172.21.15.190:0/3139102440 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93540a8510 con 0x7f935406c8d0 2024-08-19T10:40:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:21.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934bfff640 4 set_mon_vals no callback set 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f934a7fc640 10 monclient: dump: 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:21.512 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:21.513 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient: _renew_subs 2024-08-19T10:40:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.491+0000 7f935bd83640 1 -- 172.21.15.190:0/3139102440 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f93541363d0 con 0x7f935406c8d0 2024-08-19T10:40:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.495+0000 7f935bd83640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.495+0000 7f935bd83640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:21.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.495+0000 7f935bd83640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:21.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.495+0000 7f935bd83640 1 -- 172.21.15.190:0/3139102440 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935406ccb0 con 0x7f935406c8d0 2024-08-19T10:40:21.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.495+0000 7f934a7fc640 1 -- 172.21.15.190:0/3139102440 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f93500050f0 con 0x7f935406c8d0 2024-08-19T10:40:21.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.495+0000 7f934a7fc640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f933803be00 0x7f933803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.495+0000 7f934a7fc640 1 -- 172.21.15.190:0/3139102440 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(66..66 src has 1..66) ==== 5217+0+0 (secure 0 0 0) 0x7f935004e970 con 0x7f935406c8d0 2024-08-19T10:40:21.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.495+0000 7f935931e640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f933803be00 0x7f933803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:21.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.495+0000 7f935931e640 10 monclient: get_auth_request con 0x7f933803be00 auth_method 0 2024-08-19T10:40:21.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.495+0000 7f935931e640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f933803be00 0x7f933803e2c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f93440092c0 tx=0x7f9344009250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:21.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.499+0000 7f934a7fc640 1 -- 172.21.15.190:0/3139102440 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f93500104c0 con 0x7f935406c8d0 2024-08-19T10:40:21.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.499+0000 7f934a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:21.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.499+0000 7f934a7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:21.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.659+0000 7f935bd83640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:21.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.659+0000 7f935bd83640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:21.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.659+0000 7f935bd83640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:21.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.659+0000 7f935bd83640 1 -- 172.21.15.190:0/3139102440 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f93540a64b0 con 0x7f935406c8d0 2024-08-19T10:40:21.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.659+0000 7f934a7fc640 1 -- 172.21.15.190:0/3139102440 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v66) ==== 65+0+60 (secure 0 0 0) 0x7f935004e020 con 0x7f935406c8d0 2024-08-19T10:40:21.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.659+0000 7f934a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:21.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.659+0000 7f934a7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:21.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:21.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:21.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:21.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 -- 172.21.15.190:0/3139102440 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f93540a8a60 con 0x7f935406c8d0 2024-08-19T10:40:21.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f934a7fc640 1 -- 172.21.15.190:0/3139102440 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v66) ==== 65+0+60 (secure 0 0 0) 0x7f935000d360 con 0x7f935406c8d0 2024-08-19T10:40:21.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f934a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:21.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f934a7fc640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:40:21.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 -- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f933803be00 msgr2=0x7f933803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:21.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f933803be00 0x7f933803e2c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f93440092c0 tx=0x7f9344009250 comp rx=0 tx=0).stop 2024-08-19T10:40:21.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 10 monclient: shutdown 2024-08-19T10:40:21.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:21.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 -- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f935406c8d0 msgr2=0x7f9354134f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:21.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f935406c8d0 0x7f9354134f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f935000b550 tx=0x7f935000ba10 comp rx=0 tx=0).stop 2024-08-19T10:40:21.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 -- 172.21.15.190:0/3139102440 shutdown_connections 2024-08-19T10:40:21.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f933803be00 0x7f933803e2c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f935406d1f0 0x7f93541354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f935409d530 0x7f93541359e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 --2- 172.21.15.190:0/3139102440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f935406c8d0 0x7f9354134f60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 -- 172.21.15.190:0/3139102440 >> 172.21.15.190:0/3139102440 conn(0x7f935406dde0 msgr2=0x7f935409e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:21.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 -- 172.21.15.190:0/3139102440 shutdown_connections 2024-08-19T10:40:21.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.755+0000 7f935bd83640 1 -- 172.21.15.190:0/3139102440 wait complete. 2024-08-19T10:40:21.777 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:40:21.777 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule ls 2024-08-19T10:40:21.777 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false grep bar-asdf 2024-08-19T10:40:21.778 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:40:21.778 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep bar-asdf 2024-08-19T10:40:21.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient: get_monmap_and_config 2024-08-19T10:40:21.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient: build_initial_monmap 2024-08-19T10:40:21.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient: monmap: 2024-08-19T10:40:21.892 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:21.892 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:21.892 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:21.892332+0000 2024-08-19T10:40:21.892 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:21.892332+0000 2024-08-19T10:40:21.892 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:21.892 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.892 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:21.893 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:21.893 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:21.893 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:21.893 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:21.893 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:21.893 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:21.893 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:21.893 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:21.893 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:21.893 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:21.893 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 1 Processor -- start 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 1 -- start start 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient: init 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient: _add_conns ranks=[9,1,7,2,3,8,4,0,6,10,5] 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3d080a46b0 0x7f3d080a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient(hunting): picked mon.noname-a con 0x7f3d080a46b0 addr v2:172.21.15.89:3300/0 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3d080a2eb0 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3d080a4fd0 0x7f3d08137bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient(hunting): picked mon.noname-i con 0x7f3d080a4fd0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d080a61d0 con 0x7f3d080a2eb0 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:21.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f3d080a2c70 con 0x7f3d080a46b0 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f3d080a5430 con 0x7f3d080a4fd0 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0feab640 10 monclient(hunting): authenticate will time out at 1067.462746s 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0d446640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f3d080a2eb0 legacy=0x7f3d081374e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0e448640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3d080a4fd0 msgr2=0x7f3d08137bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0e448640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3d080a4fd0 0x7f3d08137bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3d080a46b0 0x7f3d080a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3d080a46b0 0x7f3d080a4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:33154/0 (socket says 172.21.15.190:33154) 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 1 -- 172.21.15.190:0/1493240187 learned_addr learned my addr 172.21.15.190:0/1493240187 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 10 monclient(hunting): get_auth_request con 0x7f3d080a46b0 auth_method 0 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:21.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 10 monclient(hunting): handle_auth_done global_id 4950 payload 274 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 1 -- 172.21.15.190:0/1493240187 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3d080a4fd0 msgr2=0x7f3d08137bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 1 --2- 172.21.15.190:0/1493240187 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3d080a4fd0 0x7f3d08137bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 1 -- 172.21.15.190:0/1493240187 >> v1:172.21.15.190:6789/0 conn(0x7f3d080a2eb0 legacy=0x7f3d081374e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 10 monclient: _finish_hunting 0 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 1 monclient: found mon.noname-a 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:21.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.887+0000 7f3d0dc47640 1 -- 172.21.15.190:0/1493240187 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0806c8d0 con 0x7f3d080a46b0 2024-08-19T10:40:21.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 1 -- 172.21.15.190:0/1493240187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 msgr2=0x7f3d080a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:21.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 1 --2- 172.21.15.190:0/1493240187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 0x7f3d080a4a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009e50 tx=0x7f3cf801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f886ccf3d1dbeb98 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:21.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 1 -- 172.21.15.190:0/1493240187 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cf80091b0 con 0x7f3d080a46b0 2024-08-19T10:40:21.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 10 monclient: dump: 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:21.904 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 10 monclient: _finish_auth 0 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 10 monclient: _check_auth_tickets 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 1 -- 172.21.15.190:0/1493240187 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf80a59c0 con 0x7f3d080a46b0 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 1 -- 172.21.15.190:0/1493240187 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cf80a5bb0 con 0x7f3d080a46b0 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:21.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0cc45640 10 monclient: dump: 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:21.906 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 5 monclient: authenticate success, global_id 4950 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 4 set_mon_vals no callback set 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient: shutdown 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 -- 172.21.15.190:0/1493240187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 msgr2=0x7f3d080a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 --2- 172.21.15.190:0/1493240187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 0x7f3d080a4a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009e50 tx=0x7f3cf801c910 comp rx=0 tx=0).stop 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 -- 172.21.15.190:0/1493240187 shutdown_connections 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 --2- 172.21.15.190:0/1493240187 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3d080a4fd0 0x7f3d08137bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 --2- 172.21.15.190:0/1493240187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 0x7f3d080a4a90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 -- 172.21.15.190:0/1493240187 >> 172.21.15.190:0/1493240187 conn(0x7f3d080a92a0 msgr2=0x7f3d080a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 -- 172.21.15.190:0/1493240187 shutdown_connections 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 -- 172.21.15.190:0/1493240187 wait complete. 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient: build_initial_monmap 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient: monmap: 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:21.908 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 Processor -- start 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 -- start start 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient: init 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient: _add_conns ranks=[8,0,6,2,1,3,7,5,4] 2024-08-19T10:40:21.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3d080a2eb0 0x7f3d080a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient(hunting): picked mon.noname-e con 0x7f3d080a2eb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 0x7f3d0813d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient(hunting): picked mon.noname-a con 0x7f3d080a46b0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3d080a4fd0 0x7f3d08137280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient(hunting): picked mon.noname-d con 0x7f3d080a4fd0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3d080a2a70 con 0x7f3d080a46b0 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f3d08138340 con 0x7f3d080a4fd0 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3d08141fd0 con 0x7f3d080a2eb0 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:21.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0feab640 10 monclient(hunting): authenticate will time out at 1067.470743s 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3d080a2eb0 0x7f3d080a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0e448640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3d080a4fd0 0x7f3d08137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 0x7f3d0813d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3d080a2eb0 0x7f3d080a1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:34816/0 (socket says 172.21.15.190:34816) 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 1 -- 172.21.15.190:0/2220996653 learned_addr learned my addr 172.21.15.190:0/2220996653 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 10 monclient(hunting): get_auth_request con 0x7f3d080a2eb0 auth_method 0 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0e448640 10 monclient(hunting): get_auth_request con 0x7f3d080a4fd0 auth_method 0 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0e448640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0e448640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:21.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0e448640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 10 monclient(hunting): get_auth_request con 0x7f3d080a46b0 auth_method 0 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0dc47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0e448640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0e448640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0e448640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:21.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 10 monclient(hunting): handle_auth_done global_id 4959 payload 824 2024-08-19T10:40:21.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 1 -- 172.21.15.190:0/2220996653 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3d080a2eb0 msgr2=0x7f3d080a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:21.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3d080a2eb0 0x7f3d080a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 1 -- 172.21.15.190:0/2220996653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3d080a4fd0 msgr2=0x7f3d08137280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:21.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3d080a4fd0 0x7f3d08137280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:21.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 10 monclient: _finish_hunting 0 2024-08-19T10:40:21.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 1 monclient: found mon.noname-a 2024-08-19T10:40:21.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:21.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:21.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.895+0000 7f3d0d446640 1 -- 172.21.15.190:0/2220996653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d080a8240 con 0x7f3d080a46b0 2024-08-19T10:40:21.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0d446640 1 -- 172.21.15.190:0/2220996653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 msgr2=0x7f3d0813d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:21.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0d446640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 0x7f3d0813d600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00b400 tx=0x7f3cfc00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:21.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 1 -- 172.21.15.190:0/2220996653 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cfc016070 con 0x7f3d080a46b0 2024-08-19T10:40:21.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 10 monclient: dump: 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:21.916 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 1 -- 172.21.15.190:0/2220996653 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cfc003de0 con 0x7f3d080a46b0 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0feab640 5 monclient: authenticate success, global_id 4959 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0feab640 10 monclient: _renew_subs 2024-08-19T10:40:21.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0feab640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:21.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0feab640 1 -- 172.21.15.190:0/2220996653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d080a8700 con 0x7f3d080a46b0 2024-08-19T10:40:21.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:21.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 1 -- 172.21.15.190:0/2220996653 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3cfc005070 con 0x7f3d080a46b0 2024-08-19T10:40:21.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:21.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:21.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d07fff640 4 set_mon_vals no callback set 2024-08-19T10:40:21.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 10 monclient: dump: 2024-08-19T10:40:21.918 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:21.918 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:21.918 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.918 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:21.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0feab640 10 monclient: _renew_subs 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0feab640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0feab640 1 -- 172.21.15.190:0/2220996653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d080a05e0 con 0x7f3d080a46b0 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0feab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0feab640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0feab640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d0feab640 1 -- 172.21.15.190:0/2220996653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d080a4a90 con 0x7f3d080a46b0 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.899+0000 7f3d067fc640 1 -- 172.21.15.190:0/2220996653 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3cfc005210 con 0x7f3d080a46b0 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.903+0000 7f3d067fc640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3cf403be00 0x7f3cf403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.903+0000 7f3d067fc640 1 -- 172.21.15.190:0/2220996653 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5217+0+0 (secure 0 0 0) 0x7f3cfc04e2d0 con 0x7f3d080a46b0 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.903+0000 7f3d0dc47640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3cf403be00 0x7f3cf403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.903+0000 7f3d0dc47640 10 monclient: get_auth_request con 0x7f3cf403be00 auth_method 0 2024-08-19T10:40:21.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.903+0000 7f3d0dc47640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3cf403be00 0x7f3cf403e2c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80af040 tx=0x7f3cf80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:21.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.915+0000 7f3d067fc640 1 -- 172.21.15.190:0/2220996653 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3cfc0056e0 con 0x7f3d080a46b0 2024-08-19T10:40:21.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.915+0000 7f3d067fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:21.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:21.915+0000 7f3d067fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.083+0000 7f3d0feab640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.083+0000 7f3d0feab640 10 monclient: _send_command 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.083+0000 7f3d0feab640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.083+0000 7f3d0feab640 1 -- 172.21.15.190:0/2220996653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f3d08123cf0 con 0x7f3d080a46b0 2024-08-19T10:40:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.087+0000 7f3d067fc640 1 -- 172.21.15.190:0/2220996653 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v66) ==== 65+0+60 (secure 0 0 0) 0x7f3cfc01f080 con 0x7f3d080a46b0 2024-08-19T10:40:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.087+0000 7f3d067fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.087+0000 7f3d067fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.187+0000 7f3d0feab640 10 start_mon_command cmd=[{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.187+0000 7f3d0feab640 10 monclient: _send_command 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.187+0000 7f3d0feab640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.187+0000 7f3d0feab640 1 -- 172.21.15.190:0/2220996653 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule ls"} v 0) -- 0x7f3d080008e0 con 0x7f3d080a46b0 2024-08-19T10:40:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.187+0000 7f3d067fc640 1 -- 172.21.15.190:0/2220996653 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule ls"}]=0 v66) ==== 65+0+60 (secure 0 0 0) 0x7f3cfc04e080 con 0x7f3d080a46b0 2024-08-19T10:40:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.187+0000 7f3d067fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule ls"}] 2024-08-19T10:40:22.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.187+0000 7f3d067fc640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:40:22.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 -- 172.21.15.190:0/2220996653 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3cf403be00 msgr2=0x7f3cf403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:22.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3cf403be00 0x7f3cf403e2c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80af040 tx=0x7f3cf80a7070 comp rx=0 tx=0).stop 2024-08-19T10:40:22.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 10 monclient: shutdown 2024-08-19T10:40:22.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:22.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 -- 172.21.15.190:0/2220996653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 msgr2=0x7f3d0813d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:22.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 0x7f3d0813d600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00b400 tx=0x7f3cfc00b8c0 comp rx=0 tx=0).stop 2024-08-19T10:40:22.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 -- 172.21.15.190:0/2220996653 shutdown_connections 2024-08-19T10:40:22.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3d080a4fd0 0x7f3d08137280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:22.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d080a46b0 0x7f3d0813d600 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:22.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3cf403be00 0x7f3cf403e2c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:22.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 --2- 172.21.15.190:0/2220996653 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3d080a2eb0 0x7f3d080a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:22.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 -- 172.21.15.190:0/2220996653 >> 172.21.15.190:0/2220996653 conn(0x7f3d080a92a0 msgr2=0x7f3d0813ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:22.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 -- 172.21.15.190:0/2220996653 shutdown_connections 2024-08-19T10:40:22.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.191+0000 7f3d0feab640 1 -- 172.21.15.190:0/2220996653 wait complete. 2024-08-19T10:40:22.212 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:40:22.212 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rm foo 2024-08-19T10:40:22.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient: get_monmap_and_config 2024-08-19T10:40:22.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient: build_initial_monmap 2024-08-19T10:40:22.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient: monmap: 2024-08-19T10:40:22.319 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:22.319 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:22.319866+0000 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:22.319866+0000 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:22.320 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:22.321 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:22.321 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:22.321 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:22.321 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:22.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 1 Processor -- start 2024-08-19T10:40:22.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 1 -- start start 2024-08-19T10:40:22.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient: init 2024-08-19T10:40:22.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:22.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient: _add_conns ranks=[6,0,5,4,2,3,1,9,10,8,7] 2024-08-19T10:40:22.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2be00a56c0 0x7f2be00a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:22.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient(hunting): picked mon.noname-d con 0x7f2be00a56c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:22.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2be00722a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:22.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2be00a61b0 0x7f2be013e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:22.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient(hunting): picked mon.noname-h con 0x7f2be00a61b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:22.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be0072720 con 0x7f2be00722a0 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2be00737c0 con 0x7f2be00a61b0 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f2be00a71e0 con 0x7f2be00a56c0 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2be4b6f640 10 monclient(hunting): authenticate will time out at 1067.889837s 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2be00a61b0 0x7f2be013e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2be00a61b0 0x7f2be013e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:55424/0 (socket says 172.21.15.190:55424) 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 1 -- 172.21.15.190:0/2589498194 learned_addr learned my addr 172.21.15.190:0/2589498194 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 10 monclient(hunting): get_auth_request con 0x7f2be00a61b0 auth_method 0 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:22.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bde59c640 1 --2- 172.21.15.190:0/2589498194 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2be00a56c0 0x7f2be00a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bde59c640 10 monclient(hunting): get_auth_request con 0x7f2be00a56c0 auth_method 0 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bde59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bde59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bde59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 10 monclient(hunting): handle_auth_done global_id 4865 payload 274 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 1 -- 172.21.15.190:0/2589498194 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2be00a56c0 msgr2=0x7f2be00a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 1 --2- 172.21.15.190:0/2589498194 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2be00a56c0 0x7f2be00a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:22.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 1 -- 172.21.15.190:0/2589498194 >> v1:172.21.15.89:6789/0 conn(0x7f2be00722a0 legacy=0x7f2be00a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:22.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 10 monclient: _finish_hunting 0 2024-08-19T10:40:22.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 1 monclient: found mon.noname-h 2024-08-19T10:40:22.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:22.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:22.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.315+0000 7f2bded9d640 1 -- 172.21.15.190:0/2589498194 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be013fcf0 con 0x7f2be00a61b0 2024-08-19T10:40:22.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bded9d640 1 -- 172.21.15.190:0/2589498194 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2be00a61b0 msgr2=0x7f2be013e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:22.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bded9d640 1 --2- 172.21.15.190:0/2589498194 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2be00a61b0 0x7f2be013e7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009e50 tx=0x7f2bd801c910 comp rx=0 tx=0).ready entity=mon.5 client_cookie=3c1c2f2ecc4b718b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:22.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 1 -- 172.21.15.190:0/2589498194 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2bd80a6970 con 0x7f2be00a61b0 2024-08-19T10:40:22.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:22.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:22.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 10 monclient: dump: 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 10 monclient: _finish_auth 0 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 10 monclient: _check_auth_tickets 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 1 -- 172.21.15.190:0/2589498194 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd80a6b30 con 0x7f2be00a61b0 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 5 monclient: authenticate success, global_id 4865 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:22.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 1 -- 172.21.15.190:0/2589498194 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2bd80a6ce0 con 0x7f2be00a61b0 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 4 set_mon_vals no callback set 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2bddd9b640 10 monclient: dump: 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:22.332 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 10 monclient: shutdown 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:22.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 1 -- 172.21.15.190:0/2589498194 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2be00a61b0 msgr2=0x7f2be013e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:22.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 1 --2- 172.21.15.190:0/2589498194 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2be00a61b0 0x7f2be013e7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009e50 tx=0x7f2bd801c910 comp rx=0 tx=0).stop 2024-08-19T10:40:22.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 1 -- 172.21.15.190:0/2589498194 shutdown_connections 2024-08-19T10:40:22.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 1 --2- 172.21.15.190:0/2589498194 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2be00a61b0 0x7f2be013e7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:22.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 1 --2- 172.21.15.190:0/2589498194 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2be00a56c0 0x7f2be00a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:22.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 1 -- 172.21.15.190:0/2589498194 >> 172.21.15.190:0/2589498194 conn(0x7f2be00aa2b0 msgr2=0x7f2be00a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:22.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 1 -- 172.21.15.190:0/2589498194 shutdown_connections 2024-08-19T10:40:22.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.319+0000 7f2be4b6f640 1 -- 172.21.15.190:0/2589498194 wait complete. 2024-08-19T10:40:22.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient: build_initial_monmap 2024-08-19T10:40:22.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient: monmap: 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:22.335 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 1 Processor -- start 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 1 -- start start 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient: init 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient: _add_conns ranks=[1,8,7,5,0,6,2,3,4] 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2be00722a0 0x7f2be01cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient(hunting): picked mon.noname-f con 0x7f2be00722a0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2be00a56c0 0x7f2be01cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient(hunting): picked mon.noname-e con 0x7f2be00a56c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:22.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2be00a61b0 0x7f2be01cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient(hunting): picked mon.noname-i con 0x7f2be00a61b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2be013f670 con 0x7f2be00722a0 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2be013f4f0 con 0x7f2be00a61b0 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2be013f7f0 con 0x7f2be00a56c0 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2be4b6f640 10 monclient(hunting): authenticate will time out at 1067.897304s 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bde59c640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2be00722a0 msgr2=0x7f2be01cec60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bde59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2be00722a0 0x7f2be01cec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:22.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bded9d640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2be00a61b0 msgr2=0x7f2be01cf6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bded9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2be00a61b0 0x7f2be01cf6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2be00a56c0 0x7f2be01cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2be00a56c0 0x7f2be01cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:34826/0 (socket says 172.21.15.190:34826) 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 1 -- 172.21.15.190:0/1957623341 learned_addr learned my addr 172.21.15.190:0/1957623341 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 10 monclient(hunting): get_auth_request con 0x7f2be00a56c0 auth_method 0 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:22.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 10 monclient(hunting): handle_auth_done global_id 4715 payload 824 2024-08-19T10:40:22.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 1 -- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2be00a61b0 msgr2=0x7f2be01cf6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:22.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2be00a61b0 0x7f2be01cf6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:22.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 1 -- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2be00722a0 msgr2=0x7f2be01cec60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:22.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2be00722a0 0x7f2be01cec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:22.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:22.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 1 monclient: found mon.noname-e 2024-08-19T10:40:22.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:22.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:22.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.323+0000 7f2bd7fff640 1 -- 172.21.15.190:0/1957623341 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be01359b0 con 0x7f2be00a56c0 2024-08-19T10:40:22.339 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-08-19T10:40:22.339 INFO:tasks.ceph.mon.f:Restarting daemon 2024-08-19T10:40:22.340 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-08-19T10:40:22.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd7fff640 1 -- 172.21.15.190:0/1957623341 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2be00a56c0 msgr2=0x7f2be01cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:22.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd7fff640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2be00a56c0 0x7f2be01cf1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800c470 tx=0x7f2bc800c930 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:22.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 1 -- 172.21.15.190:0/1957623341 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2bc8016030 con 0x7f2be00a56c0 2024-08-19T10:40:22.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:22.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:22.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 10 monclient: dump: 2024-08-19T10:40:22.343 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:22.343 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:22.343 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:22.343 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:22.343 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:22.343 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:22.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2be4b6f640 5 monclient: authenticate success, global_id 4715 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2be4b6f640 10 monclient: _renew_subs 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2be4b6f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2be4b6f640 1 -- 172.21.15.190:0/1957623341 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be00a8bc0 con 0x7f2be00a56c0 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 1 -- 172.21.15.190:0/1957623341 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc8005020 con 0x7f2be00a56c0 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bdd59a640 4 set_mon_vals no callback set 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2be4b6f640 10 monclient: _renew_subs 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2be4b6f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2be4b6f640 1 -- 172.21.15.190:0/1957623341 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be01356d0 con 0x7f2be00a56c0 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 1 -- 172.21.15.190:0/1957623341 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2bc8012840 con 0x7f2be00a56c0 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:22.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 10 monclient: dump: 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:22.346 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2be4b6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2be4b6f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2be4b6f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2be4b6f640 1 -- 172.21.15.190:0/1957623341 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be0134f40 con 0x7f2be00a56c0 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 1 -- 172.21.15.190:0/1957623341 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2bc80051d0 con 0x7f2be00a56c0 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2bc003be00 0x7f2bc003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bd6ffd640 1 -- 172.21.15.190:0/1957623341 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(66..66 src has 1..66) ==== 5217+0+0 (secure 0 0 0) 0x7f2bc8017070 con 0x7f2be00a56c0 2024-08-19T10:40:22.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bde59c640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2bc003be00 0x7f2bc003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:22.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.327+0000 7f2bde59c640 10 monclient: get_auth_request con 0x7f2bc003be00 auth_method 0 2024-08-19T10:40:22.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.331+0000 7f2bde59c640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2bc003be00 0x7f2bc003e2c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0000f00 tx=0x7f2bd0006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:22.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.339+0000 7f2bd6ffd640 1 -- 172.21.15.190:0/1957623341 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f2bc801f020 con 0x7f2be00a56c0 2024-08-19T10:40:22.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.339+0000 7f2bd6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:22.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.339+0000 7f2bd6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:22.349 INFO:tasks.ceph.mon.f:Started 2024-08-19T10:40:22.349 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-08-19T10:40:22.349 INFO:tasks.ceph.mon.i:Restarting daemon 2024-08-19T10:40:22.349 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-08-19T10:40:22.351 INFO:tasks.ceph.mon.i:Started 2024-08-19T10:40:22.351 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-08-19T10:40:22.351 INFO:tasks.ceph.mon.b:Sent signal 19 2024-08-19T10:40:22.352 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-08-19T10:40:22.352 INFO:tasks.ceph.mon.h:Sent signal 19 2024-08-19T10:40:22.352 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-08-19T10:40:22.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.519+0000 7f2be4b6f640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:22.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.519+0000 7f2be4b6f640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:22.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.519+0000 7f2be4b6f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:22.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:22.519+0000 7f2be4b6f640 1 -- 172.21.15.190:0/1957623341 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) -- 0x7f2be00a1670 con 0x7f2be00a56c0 2024-08-19T10:40:25.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:25.327+0000 7f2bd5ffb640 10 monclient: tick 2024-08-19T10:40:25.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:25.327+0000 7f2bd5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:25.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:25.327+0000 7f2bd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:25.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:25.327+0000 7f2bd5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:26.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:26.323+0000 7f2bd5ffb640 10 monclient: tick 2024-08-19T10:40:26.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:26.323+0000 7f2bd5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:26.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:26.323+0000 7f2bd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:27.323+0000 7f2bd5ffb640 10 monclient: tick 2024-08-19T10:40:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:27.323+0000 7f2bd5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:27.323+0000 7f2bd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:28.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:28.327+0000 7f2bd5ffb640 10 monclient: tick 2024-08-19T10:40:28.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:28.327+0000 7f2bd5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:28.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:28.327+0000 7f2bd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:29.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:29.327+0000 7f2bd5ffb640 10 monclient: tick 2024-08-19T10:40:29.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:29.327+0000 7f2bd5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:29.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:29.327+0000 7f2bd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:30.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:30.323+0000 7f2bd5ffb640 10 monclient: tick 2024-08-19T10:40:30.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:30.323+0000 7f2bd5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:30.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:30.323+0000 7f2bd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:31.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:31.327+0000 7f2bd5ffb640 10 monclient: tick 2024-08-19T10:40:31.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:31.327+0000 7f2bd5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:31.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:31.327+0000 7f2bd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:32.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:32.327+0000 7f2bd5ffb640 10 monclient: tick 2024-08-19T10:40:32.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:32.327+0000 7f2bd5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:32.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:32.327+0000 7f2bd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:33.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.327+0000 7f2bd5ffb640 10 monclient: tick 2024-08-19T10:40:33.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.327+0000 7f2bd5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:33.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.327+0000 7f2bd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:33.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.555+0000 7f2bd6ffd640 1 -- 172.21.15.190:0/1957623341 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 v67) ==== 80+0+0 (secure 0 0 0) 0x7f2bc8011380 con 0x7f2be00a56c0 2024-08-19T10:40:33.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.555+0000 7f2bd6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:33.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.555+0000 7f2bd6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.663+0000 7f2be4b6f640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.663+0000 7f2be4b6f640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:33.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.663+0000 7f2be4b6f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:33.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.663+0000 7f2be4b6f640 1 -- 172.21.15.190:0/1957623341 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) -- 0x7f2be00008e0 con 0x7f2be00a56c0 2024-08-19T10:40:33.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.663+0000 7f2bd6ffd640 1 -- 172.21.15.190:0/1957623341 <== mon.8 v2:172.21.15.89:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v67) ==== 103+0+0 (secure 0 0 0) 0x7f2bc8007540 con 0x7f2be00a56c0 2024-08-19T10:40:33.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.663+0000 7f2bd6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:33.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.663+0000 7f2bd6ffd640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-08-19T10:40:33.666 INFO:tasks.workunit.client.0.smithi190.stderr:rule foo does not exist 2024-08-19T10:40:33.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 -- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2bc003be00 msgr2=0x7f2bc003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:33.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2bc003be00 0x7f2bc003e2c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0000f00 tx=0x7f2bd0006a30 comp rx=0 tx=0).stop 2024-08-19T10:40:33.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 10 monclient: shutdown 2024-08-19T10:40:33.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:33.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 -- 172.21.15.190:0/1957623341 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2be00a56c0 msgr2=0x7f2be01cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:33.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2be00a56c0 0x7f2be01cf1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800c470 tx=0x7f2bc800c930 comp rx=0 tx=0).stop 2024-08-19T10:40:33.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 -- 172.21.15.190:0/1957623341 shutdown_connections 2024-08-19T10:40:33.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2be00a61b0 0x7f2be01cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:33.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2bc003be00 0x7f2bc003e2c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:33.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2be00a56c0 0x7f2be01cf1a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:33.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 --2- 172.21.15.190:0/1957623341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2be00722a0 0x7f2be01cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:33.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 -- 172.21.15.190:0/1957623341 >> 172.21.15.190:0/1957623341 conn(0x7f2be00aa2b0 msgr2=0x7f2be00a7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:33.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 -- 172.21.15.190:0/1957623341 shutdown_connections 2024-08-19T10:40:33.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.667+0000 7f2be4b6f640 1 -- 172.21.15.190:0/1957623341 wait complete. 2024-08-19T10:40:33.683 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rm foo 2024-08-19T10:40:33.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient: get_monmap_and_config 2024-08-19T10:40:33.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient: build_initial_monmap 2024-08-19T10:40:33.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient: monmap: 2024-08-19T10:40:33.805 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:33.805 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:33.805 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:33.804420+0000 2024-08-19T10:40:33.805 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:33.804420+0000 2024-08-19T10:40:33.805 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:33.805 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:33.805 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:33.806 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:33.806 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:33.806 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:33.806 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:33.806 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:33.806 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:33.806 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:33.806 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:33.807 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:33.807 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:33.807 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:33.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 1 Processor -- start 2024-08-19T10:40:33.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 1 -- start start 2024-08-19T10:40:33.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient: init 2024-08-19T10:40:33.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:33.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient: _add_conns ranks=[4,2,1,10,7,0,9,8,5,6,3] 2024-08-19T10:40:33.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31ec132e50 0x7f31ec133230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:33.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient(hunting): picked mon.noname-c con 0x7f31ec132e50 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:33.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31ec09e910 0x7f31ec0a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:33.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient(hunting): picked mon.noname-b con 0x7f31ec09e910 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:33.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f31ec09ecf0 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:33.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31ec09ca60 con 0x7f31ec09ecf0 2024-08-19T10:40:33.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f31ec074df0 con 0x7f31ec09e910 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f31ec09e130 con 0x7f31ec132e50 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f3145640 10 monclient(hunting): authenticate will time out at 1079.375226s 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f16e2640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f31ec09ecf0 0x7f31ec0a41c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:36984/0 (socket says 172.21.15.190:36984) 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f16e2640 1 -- 172.21.15.190:0/3860334627 learned_addr learned my addr 172.21.15.190:0/3860334627 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f0ee1640 1 --2- 172.21.15.190:0/3860334627 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31ec132e50 0x7f31ec133230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f0ee1640 10 monclient(hunting): get_auth_request con 0x7f31ec132e50 auth_method 0 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f0ee1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f0ee1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f0ee1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:33.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f0ee1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:33.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f0ee1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:33.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.803+0000 7f31f0ee1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:33.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.807+0000 7f31f0ee1640 10 monclient(hunting): handle_auth_done global_id 4873 payload 274 2024-08-19T10:40:33.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.807+0000 7f31f0ee1640 1 -- 172.21.15.190:0/3860334627 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31ec09e910 msgr2=0x7f31ec0a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:33.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.807+0000 7f31f0ee1640 1 --2- 172.21.15.190:0/3860334627 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31ec09e910 0x7f31ec0a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:33.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.807+0000 7f31f0ee1640 1 -- 172.21.15.190:0/3860334627 >> v1:172.21.15.190:6789/0 conn(0x7f31ec09ecf0 legacy=0x7f31ec0a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:33.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.807+0000 7f31f0ee1640 10 monclient: _finish_hunting 0 2024-08-19T10:40:33.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.807+0000 7f31f0ee1640 1 monclient: found mon.noname-c 2024-08-19T10:40:33.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.807+0000 7f31f0ee1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:33.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.807+0000 7f31f0ee1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:33.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.807+0000 7f31f0ee1640 1 -- 172.21.15.190:0/3860334627 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ec09db70 con 0x7f31ec132e50 2024-08-19T10:40:33.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f0ee1640 1 -- 172.21.15.190:0/3860334627 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31ec132e50 msgr2=0x7f31ec133230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:33.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f0ee1640 1 --2- 172.21.15.190:0/3860334627 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31ec132e50 0x7f31ec133230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f31e80097f0 tx=0x7f31e801c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=19d1fed06c18b5e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:33.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 1 -- 172.21.15.190:0/3860334627 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31e80b0070 con 0x7f31ec132e50 2024-08-19T10:40:33.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:33.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:33.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 10 monclient: dump: 2024-08-19T10:40:33.819 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:33.819 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:33.819 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:33.819 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:33.820 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:33.820 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:33.820 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:33.820 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:33.820 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:33.820 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:33.821 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:33.821 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:33.821 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:33.821 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:33.821 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:33.821 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:33.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:33.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:33.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:33.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 1 -- 172.21.15.190:0/3860334627 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e80aa810 con 0x7f31ec132e50 2024-08-19T10:40:33.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:33.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 1 -- 172.21.15.190:0/3860334627 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31e80a4e60 con 0x7f31ec132e50 2024-08-19T10:40:33.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:33.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:33.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31e37fe640 10 monclient: dump: 2024-08-19T10:40:33.823 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:33.823 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:33.823 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:33.823 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:33.824 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:33.824 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:33.824 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:33.824 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:33.824 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:33.825 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 5 monclient: authenticate success, global_id 4873 2024-08-19T10:40:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 4 set_mon_vals no callback set 2024-08-19T10:40:33.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 10 monclient: shutdown 2024-08-19T10:40:33.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:33.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 1 -- 172.21.15.190:0/3860334627 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31ec132e50 msgr2=0x7f31ec133230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:33.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 1 --2- 172.21.15.190:0/3860334627 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31ec132e50 0x7f31ec133230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f31e80097f0 tx=0x7f31e801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:40:33.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 1 -- 172.21.15.190:0/3860334627 shutdown_connections 2024-08-19T10:40:33.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 1 --2- 172.21.15.190:0/3860334627 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31ec09e910 0x7f31ec0a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:33.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 1 --2- 172.21.15.190:0/3860334627 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f31ec132e50 0x7f31ec133230 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:33.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 1 -- 172.21.15.190:0/3860334627 >> 172.21.15.190:0/3860334627 conn(0x7f31ec0a4d70 msgr2=0x7f31ec0a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:33.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 1 -- 172.21.15.190:0/3860334627 shutdown_connections 2024-08-19T10:40:33.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 1 -- 172.21.15.190:0/3860334627 wait complete. 2024-08-19T10:40:33.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 10 monclient: build_initial_monmap 2024-08-19T10:40:33.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 10 monclient: monmap: 2024-08-19T10:40:33.827 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:33.827 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:33.827 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:33.827 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:33.827 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:33.827 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:33.827 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:33.827 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:33.828 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:33.828 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:33.828 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:33.828 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:33.829 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:33.829 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:33.829 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:33.829 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:33.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 1 Processor -- start 2024-08-19T10:40:33.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 1 -- start start 2024-08-19T10:40:33.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 10 monclient: init 2024-08-19T10:40:33.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:33.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 10 monclient: _add_conns ranks=[2,1,0,6,7,3,5,8,4] 2024-08-19T10:40:33.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.811+0000 7f31f3145640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31ec09e910 0x7f31ec1d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:33.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 10 monclient(hunting): picked mon.noname-b con 0x7f31ec09e910 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:33.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec09ecf0 0x7f31ec1402d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:33.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 10 monclient(hunting): picked mon.noname-f con 0x7f31ec09ecf0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:33.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ec140810 0x7f31ec13af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:33.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 10 monclient(hunting): picked mon.noname-a con 0x7f31ec140810 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:33.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:33.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec0a4a50 con 0x7f31ec140810 2024-08-19T10:40:33.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:33.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec0a48d0 con 0x7f31ec09ecf0 2024-08-19T10:40:33.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:33.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f31ec143e80 con 0x7f31ec09e910 2024-08-19T10:40:33.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:33.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f3145640 10 monclient(hunting): authenticate will time out at 1079.384055s 2024-08-19T10:40:33.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31e3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec09ecf0 0x7f31ec1402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:33.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31e3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec09ecf0 0x7f31ec1402d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:47796/0 (socket says 172.21.15.190:47796) 2024-08-19T10:40:33.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31e3fff640 1 -- 172.21.15.190:0/3419990395 learned_addr learned my addr 172.21.15.190:0/3419990395 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:33.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31e3fff640 10 monclient(hunting): get_auth_request con 0x7f31ec09ecf0 auth_method 0 2024-08-19T10:40:33.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:33.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31e3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:33.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:33.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31e3fff640 1 -- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec09ecf0 msgr2=0x7f31ec1402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:40:33.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31e3fff640 1 -- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec09ecf0 msgr2=0x7f31ec1402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:40:33.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31e3fff640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec09ecf0 0x7f31ec1402d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:33.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31e3fff640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec09ecf0 0x7f31ec1402d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:33.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ec140810 0x7f31ec13af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:33.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 10 monclient(hunting): get_auth_request con 0x7f31ec140810 auth_method 0 2024-08-19T10:40:33.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:33.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:33.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:33.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:33.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:33.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:33.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 10 monclient(hunting): handle_auth_done global_id 4968 payload 824 2024-08-19T10:40:33.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 1 -- 172.21.15.190:0/3419990395 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31ec09e910 msgr2=0x7f31ec1d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:33.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31ec09e910 0x7f31ec1d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:33.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 1 -- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec09ecf0 msgr2=0x7f31ec1402d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:33.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec09ecf0 0x7f31ec1402d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:33.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 10 monclient: _finish_hunting 0 2024-08-19T10:40:33.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 1 monclient: found mon.noname-a 2024-08-19T10:40:33.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:33.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:33.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 1 -- 172.21.15.190:0/3419990395 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ec0a74c0 con 0x7f31ec140810 2024-08-19T10:40:33.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 1 -- 172.21.15.190:0/3419990395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ec140810 msgr2=0x7f31ec13af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:33.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.815+0000 7f31f16e2640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ec140810 0x7f31ec13af50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f31e400bd60 tx=0x7f31e4009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:33.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 1 -- 172.21.15.190:0/3419990395 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31e4011b90 con 0x7f31ec140810 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 10 monclient: dump: 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:33.842 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 10 monclient: _finish_auth 0 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 10 monclient: _check_auth_tickets 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 1 -- 172.21.15.190:0/3419990395 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e4011d70 con 0x7f31ec140810 2024-08-19T10:40:33.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:33.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 1 -- 172.21.15.190:0/3419990395 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31e4013870 con 0x7f31ec140810 2024-08-19T10:40:33.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:33.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:33.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 10 monclient: dump: 2024-08-19T10:40:33.844 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:33.844 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:33.844 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:33.845 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:33.845 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:33.845 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:33.845 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:33.845 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:33.845 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:33.845 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:33.846 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:33.846 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:33.846 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:33.846 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:33.846 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:33.846 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:33.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f3145640 5 monclient: authenticate success, global_id 4968 2024-08-19T10:40:33.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f3145640 10 monclient: _renew_subs 2024-08-19T10:40:33.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f3145640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:33.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f3145640 1 -- 172.21.15.190:0/3419990395 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ec13eda0 con 0x7f31ec140810 2024-08-19T10:40:33.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e2ffd640 4 set_mon_vals no callback set 2024-08-19T10:40:33.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f3145640 10 monclient: _renew_subs 2024-08-19T10:40:33.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f3145640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:33.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f3145640 1 -- 172.21.15.190:0/3419990395 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ec13eac0 con 0x7f31ec140810 2024-08-19T10:40:33.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f3145640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:33.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f3145640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:33.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f3145640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:33.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f3145640 1 -- 172.21.15.190:0/3419990395 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ec0a7860 con 0x7f31ec140810 2024-08-19T10:40:33.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 1 -- 172.21.15.190:0/3419990395 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f31e4013400 con 0x7f31ec140810 2024-08-19T10:40:33.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31d803be30 0x7f31d803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:33.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31e17fa640 1 -- 172.21.15.190:0/3419990395 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5162+0+0 (secure 0 0 0) 0x7f31e4050430 con 0x7f31ec140810 2024-08-19T10:40:33.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f0ee1640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31d803be30 0x7f31d803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:33.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.819+0000 7f31f0ee1640 10 monclient: get_auth_request con 0x7f31d803be30 auth_method 0 2024-08-19T10:40:33.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.823+0000 7f31f0ee1640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31d803be30 0x7f31d803e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f31e80ab000 tx=0x7f31e8004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:33.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.831+0000 7f31e17fa640 1 -- 172.21.15.190:0/3419990395 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f31ec0a74c0 con 0x7f31ec140810 2024-08-19T10:40:33.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.831+0000 7f31e17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:33.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:33.831+0000 7f31e17fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:34.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.007+0000 7f31f3145640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:34.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.007+0000 7f31f3145640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:34.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.007+0000 7f31f3145640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:34.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.007+0000 7f31f3145640 1 -- 172.21.15.190:0/3419990395 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) -- 0x7f31ec133230 con 0x7f31ec140810 2024-08-19T10:40:34.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.007+0000 7f31e17fa640 1 -- 172.21.15.190:0/3419990395 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v67) ==== 103+0+0 (secure 0 0 0) 0x7f31e404dd00 con 0x7f31ec140810 2024-08-19T10:40:34.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.007+0000 7f31e17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:34.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.007+0000 7f31e17fa640 10 monclient: _finish_command 2 = system:0 rule foo does not exist 2024-08-19T10:40:34.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.111+0000 7f31f3145640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:34.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.111+0000 7f31f3145640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:34.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.111+0000 7f31f3145640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:34.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.111+0000 7f31f3145640 1 -- 172.21.15.190:0/3419990395 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo"} v 0) -- 0x7f31ec140c50 con 0x7f31ec140810 2024-08-19T10:40:34.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.111+0000 7f31e17fa640 1 -- 172.21.15.190:0/3419990395 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo"}]=0 rule foo does not exist v67) ==== 103+0+0 (secure 0 0 0) 0x7f31e404e080 con 0x7f31ec140810 2024-08-19T10:40:34.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.111+0000 7f31e17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo"}] 2024-08-19T10:40:34.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.111+0000 7f31e17fa640 10 monclient: _finish_command 3 = system:0 rule foo does not exist 2024-08-19T10:40:34.115 INFO:tasks.workunit.client.0.smithi190.stderr:rule foo does not exist 2024-08-19T10:40:34.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 -- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31d803be30 msgr2=0x7f31d803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:34.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31d803be30 0x7f31d803e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f31e80ab000 tx=0x7f31e8004740 comp rx=0 tx=0).stop 2024-08-19T10:40:34.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 10 monclient: shutdown 2024-08-19T10:40:34.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:34.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 -- 172.21.15.190:0/3419990395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ec140810 msgr2=0x7f31ec13af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:34.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ec140810 0x7f31ec13af50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f31e400bd60 tx=0x7f31e4009ed0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 -- 172.21.15.190:0/3419990395 shutdown_connections 2024-08-19T10:40:34.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31d803be30 0x7f31d803e2f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ec140810 0x7f31ec13af50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31ec09ecf0 0x7f31ec1402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 --2- 172.21.15.190:0/3419990395 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31ec09e910 0x7f31ec1d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 -- 172.21.15.190:0/3419990395 >> 172.21.15.190:0/3419990395 conn(0x7f31ec0a4d70 msgr2=0x7f31ec135b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:34.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 -- 172.21.15.190:0/3419990395 shutdown_connections 2024-08-19T10:40:34.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.115+0000 7f31f3145640 1 -- 172.21.15.190:0/3419990395 wait complete. 2024-08-19T10:40:34.130 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rm bar 2024-08-19T10:40:34.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient: get_monmap_and_config 2024-08-19T10:40:34.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient: build_initial_monmap 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient: monmap: 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:34.247888+0000 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:34.247888+0000 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:34.248 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:34.249 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:34.249 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:34.249 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:34.249 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:34.249 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:34.249 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 1 Processor -- start 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 1 -- start start 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient: init 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient: _add_conns ranks=[8,2,4,7,5,10,1,0,6,3,9] 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9598139980 0x7f9598139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient(hunting): picked mon.noname-e con 0x7f9598139980 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f959813a2a0 0x7f95980a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient(hunting): picked mon.noname-b con 0x7f959813a2a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95980a5c70 0x7f95980a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient(hunting): picked mon.noname-c con 0x7f95980a5c70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:34.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f959809ca60 con 0x7f959813a2a0 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f95980a3640 con 0x7f95980a5c70 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f95980a8470 con 0x7f9598139980 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f959d764640 10 monclient(hunting): authenticate will time out at 1079.817763s 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95980a5c70 0x7f95980a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9598139980 0x7f9598139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95980a5c70 0x7f95980a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:48606/0 (socket says 172.21.15.190:48606) 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 1 -- 172.21.15.190:0/2124225565 learned_addr learned my addr 172.21.15.190:0/2124225565 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 10 monclient(hunting): get_auth_request con 0x7f95980a5c70 auth_method 0 2024-08-19T10:40:34.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f9596ffd640 10 monclient(hunting): get_auth_request con 0x7f9598139980 auth_method 0 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f9596ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f9596ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f9596ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f9596ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f9596ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f9596ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:34.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 10 monclient(hunting): handle_auth_done global_id 4882 payload 274 2024-08-19T10:40:34.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 1 -- 172.21.15.190:0/2124225565 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9598139980 msgr2=0x7f9598139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:34.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 1 --2- 172.21.15.190:0/2124225565 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9598139980 0x7f9598139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 1 -- 172.21.15.190:0/2124225565 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f959813a2a0 msgr2=0x7f95980a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:34.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 1 --2- 172.21.15.190:0/2124225565 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f959813a2a0 0x7f95980a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:34.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 1 monclient: found mon.noname-c 2024-08-19T10:40:34.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:34.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:34.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.247+0000 7f95977fe640 1 -- 172.21.15.190:0/2124225565 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95980a3850 con 0x7f95980a5c70 2024-08-19T10:40:34.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f95977fe640 1 -- 172.21.15.190:0/2124225565 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95980a5c70 msgr2=0x7f95980a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:34.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f95977fe640 1 --2- 172.21.15.190:0/2124225565 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95980a5c70 0x7f95980a67e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9590009eb0 tx=0x7f959001c600 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4d9d012dc6fe379 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:34.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 1 -- 172.21.15.190:0/2124225565 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f95900ac020 con 0x7f95980a5c70 2024-08-19T10:40:34.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:34.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:34.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 10 monclient: dump: 2024-08-19T10:40:34.259 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:34.259 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:34.259 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.259 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.259 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:34.259 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:34.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 1 -- 172.21.15.190:0/2124225565 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9590009110 con 0x7f95980a5c70 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 1 -- 172.21.15.190:0/2124225565 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f95900a6cf0 con 0x7f95980a5c70 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f9595ffb640 10 monclient: dump: 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:34.261 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f959d764640 5 monclient: authenticate success, global_id 4882 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f959d764640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:34.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.251+0000 7f959d764640 4 set_mon_vals no callback set 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient: shutdown 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 -- 172.21.15.190:0/2124225565 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95980a5c70 msgr2=0x7f95980a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 --2- 172.21.15.190:0/2124225565 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95980a5c70 0x7f95980a67e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9590009eb0 tx=0x7f959001c600 comp rx=0 tx=0).stop 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 -- 172.21.15.190:0/2124225565 shutdown_connections 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 --2- 172.21.15.190:0/2124225565 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f95980a5c70 0x7f95980a67e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 --2- 172.21.15.190:0/2124225565 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f959813a2a0 0x7f95980a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 --2- 172.21.15.190:0/2124225565 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9598139980 0x7f9598139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 -- 172.21.15.190:0/2124225565 >> 172.21.15.190:0/2124225565 conn(0x7f95980abde0 msgr2=0x7f95980ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 -- 172.21.15.190:0/2124225565 shutdown_connections 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 -- 172.21.15.190:0/2124225565 wait complete. 2024-08-19T10:40:34.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient: build_initial_monmap 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient: monmap: 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:34.264 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:34.265 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:34.265 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:34.265 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:34.265 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 Processor -- start 2024-08-19T10:40:34.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 -- start start 2024-08-19T10:40:34.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient: init 2024-08-19T10:40:34.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:34.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient: _add_conns ranks=[0,3,2,8,6,7,1,4,5] 2024-08-19T10:40:34.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95980a5c70 0x7f95981d1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient(hunting): picked mon.noname-a con 0x7f95980a5c70 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:34.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9598139980 0x7f959813f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient(hunting): picked mon.noname-g con 0x7f9598139980 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:34.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f959813fe40 0x7f959813a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient(hunting): picked mon.noname-b con 0x7f959813fe40 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:34.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:34.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9598143270 con 0x7f95980a5c70 2024-08-19T10:40:34.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:34.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f95981430f0 con 0x7f959813fe40 2024-08-19T10:40:34.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:34.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f95981433f0 con 0x7f9598139980 2024-08-19T10:40:34.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:34.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f959d764640 10 monclient(hunting): authenticate will time out at 1079.826236s 2024-08-19T10:40:34.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f95967fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9598139980 0x7f959813f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:34.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f95967fc640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9598139980 msgr2=0x7f959813f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:40:34.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f95967fc640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9598139980 msgr2=0x7f959813f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:40:34.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f95967fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9598139980 0x7f959813f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:34.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f95967fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9598139980 0x7f959813f900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:34.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95980a5c70 0x7f95981d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:34.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95980a5c70 0x7f95981d1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:44406/0 (socket says 172.21.15.190:44406) 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 1 -- 172.21.15.190:0/4178017182 learned_addr learned my addr 172.21.15.190:0/4178017182 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 10 monclient(hunting): get_auth_request con 0x7f95980a5c70 auth_method 0 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 10 monclient(hunting): handle_auth_done global_id 4977 payload 824 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 1 -- 172.21.15.190:0/4178017182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9598139980 msgr2=0x7f959813f900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9598139980 0x7f959813f900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 1 -- 172.21.15.190:0/4178017182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f959813fe40 msgr2=0x7f959813a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:34.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f959813fe40 0x7f959813a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:40:34.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 1 monclient: found mon.noname-a 2024-08-19T10:40:34.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:34.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:34.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.255+0000 7f9596ffd640 1 -- 172.21.15.190:0/4178017182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95980a5790 con 0x7f95980a5c70 2024-08-19T10:40:34.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f9596ffd640 1 -- 172.21.15.190:0/4178017182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95980a5c70 msgr2=0x7f95981d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:34.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f9596ffd640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95980a5c70 0x7f95981d1770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f958c00e9b0 tx=0x7f958c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:34.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 1 -- 172.21.15.190:0/4178017182 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f958c017070 con 0x7f95980a5c70 2024-08-19T10:40:34.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:34.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:34.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 10 monclient: dump: 2024-08-19T10:40:34.272 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:34.272 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:34.272 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.272 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.272 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:34.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:34.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:34.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 1 -- 172.21.15.190:0/4178017182 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958c013470 con 0x7f95980a5c70 2024-08-19T10:40:34.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f959d764640 5 monclient: authenticate success, global_id 4977 2024-08-19T10:40:34.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f959d764640 10 monclient: _renew_subs 2024-08-19T10:40:34.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f959d764640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:34.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f959d764640 1 -- 172.21.15.190:0/4178017182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959813cbd0 con 0x7f95980a5c70 2024-08-19T10:40:34.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:34.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f959d764640 10 monclient: _renew_subs 2024-08-19T10:40:34.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f95957fa640 4 set_mon_vals no callback set 2024-08-19T10:40:34.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f959d764640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:34.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f959d764640 1 -- 172.21.15.190:0/4178017182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959813de10 con 0x7f95980a5c70 2024-08-19T10:40:34.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 1 -- 172.21.15.190:0/4178017182 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f958c013620 con 0x7f95980a5c70 2024-08-19T10:40:34.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:34.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:34.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 10 monclient: dump: 2024-08-19T10:40:34.275 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:34.275 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:34.275 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.275 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f959d764640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:34.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f959d764640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:34.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f959d764640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:34.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f959d764640 1 -- 172.21.15.190:0/4178017182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9598139d60 con 0x7f95980a5c70 2024-08-19T10:40:34.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 1 -- 172.21.15.190:0/4178017182 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f958c01a070 con 0x7f95980a5c70 2024-08-19T10:40:34.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f958403be00 0x7f958403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.259+0000 7f957f7fe640 1 -- 172.21.15.190:0/4178017182 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5162+0+0 (secure 0 0 0) 0x7f958c00f040 con 0x7f95980a5c70 2024-08-19T10:40:34.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.263+0000 7f95967fc640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f958403be00 0x7f958403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:34.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.263+0000 7f95967fc640 10 monclient: get_auth_request con 0x7f958403be00 auth_method 0 2024-08-19T10:40:34.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.263+0000 7f95967fc640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f958403be00 0x7f958403e2c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f959813c6e0 tx=0x7f9588008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:34.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.275+0000 7f957f7fe640 1 -- 172.21.15.190:0/4178017182 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f95980a5790 con 0x7f95980a5c70 2024-08-19T10:40:34.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.275+0000 7f957f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:34.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.275+0000 7f957f7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:34.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.447+0000 7f959d764640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "bar"}] 2024-08-19T10:40:34.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.447+0000 7f959d764640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-08-19T10:40:34.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.447+0000 7f959d764640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:34.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.447+0000 7f959d764640 1 -- 172.21.15.190:0/4178017182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "bar"} v 0) -- 0x7f95980a67e0 con 0x7f95980a5c70 2024-08-19T10:40:34.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.559+0000 7f957f7fe640 1 -- 172.21.15.190:0/4178017182 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 v68) ==== 80+0+0 (secure 0 0 0) 0x7f958c01bdc0 con 0x7f95980a5c70 2024-08-19T10:40:34.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.559+0000 7f957f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-08-19T10:40:34.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.559+0000 7f957f7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.663+0000 7f959d764640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "bar"}] 2024-08-19T10:40:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.663+0000 7f959d764640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-08-19T10:40:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.663+0000 7f959d764640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.663+0000 7f959d764640 1 -- 172.21.15.190:0/4178017182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rule rm", "name": "bar"} v 0) -- 0x7f9598076f00 con 0x7f95980a5c70 2024-08-19T10:40:34.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.663+0000 7f957f7fe640 1 -- 172.21.15.190:0/4178017182 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "bar"}]=0 rule bar does not exist v68) ==== 103+0+0 (secure 0 0 0) 0x7f958c04cfd0 con 0x7f95980a5c70 2024-08-19T10:40:34.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.663+0000 7f957f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "bar"}] 2024-08-19T10:40:34.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.663+0000 7f957f7fe640 10 monclient: _finish_command 3 = system:0 rule bar does not exist 2024-08-19T10:40:34.666 INFO:tasks.workunit.client.0.smithi190.stderr:rule bar does not exist 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 -- 172.21.15.190:0/4178017182 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f958403be00 msgr2=0x7f958403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f958403be00 0x7f958403e2c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f959813c6e0 tx=0x7f9588008040 comp rx=0 tx=0).stop 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 10 monclient: shutdown 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 -- 172.21.15.190:0/4178017182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95980a5c70 msgr2=0x7f95981d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95980a5c70 0x7f95981d1770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f958c00e9b0 tx=0x7f958c00ee70 comp rx=0 tx=0).stop 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 -- 172.21.15.190:0/4178017182 shutdown_connections 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f958403be00 0x7f958403e2c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f959813fe40 0x7f959813a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9598139980 0x7f959813f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 --2- 172.21.15.190:0/4178017182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95980a5c70 0x7f95981d1770 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 -- 172.21.15.190:0/4178017182 >> 172.21.15.190:0/4178017182 conn(0x7f95980abde0 msgr2=0x7f9598138d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:34.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 -- 172.21.15.190:0/4178017182 shutdown_connections 2024-08-19T10:40:34.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.667+0000 7f959d764640 1 -- 172.21.15.190:0/4178017182 wait complete. 2024-08-19T10:40:34.683 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create pinning_pool 1 2024-08-19T10:40:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient: get_monmap_and_config 2024-08-19T10:40:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient: build_initial_monmap 2024-08-19T10:40:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient: monmap: 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:34.787791+0000 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:34.787791+0000 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:34.789 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 1 Processor -- start 2024-08-19T10:40:34.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 1 -- start start 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient: init 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient: _add_conns ranks=[2,3,0,10,8,9,1,7,6,5,4] 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd33c139980 0x7fd33c139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient(hunting): picked mon.noname-b con 0x7fd33c139980 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c13a2a0 0x7fd33c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient(hunting): picked mon.noname-g con 0x7fd33c13a2a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd33c0a5c70 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd33c0a8470 con 0x7fd33c0a5c70 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd33c09ca60 con 0x7fd33c139980 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:34.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd33c0a3640 con 0x7fd33c13a2a0 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd343843640 10 monclient(hunting): authenticate will time out at 1080.357836s 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c13a2a0 0x7fd33c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c13a2a0 0x7fd33c0a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:55642/0 (socket says 172.21.15.190:55642) 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 1 -- 172.21.15.190:0/2630865900 learned_addr learned my addr 172.21.15.190:0/2630865900 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 10 monclient(hunting): get_auth_request con 0x7fd33c13a2a0 auth_method 0 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:34.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 10 monclient(hunting): handle_auth_done global_id 4836 payload 274 2024-08-19T10:40:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 1 -- 172.21.15.190:0/2630865900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd33c139980 msgr2=0x7fd33c139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 1 --2- 172.21.15.190:0/2630865900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd33c139980 0x7fd33c139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 1 -- 172.21.15.190:0/2630865900 >> v1:172.21.15.89:6789/0 conn(0x7fd33c0a5c70 legacy=0x7fd33c0a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 10 monclient: _finish_hunting 0 2024-08-19T10:40:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 1 monclient: found mon.noname-g 2024-08-19T10:40:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.787+0000 7fd340dde640 1 -- 172.21.15.190:0/2630865900 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33c0a3850 con 0x7fd33c13a2a0 2024-08-19T10:40:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd340dde640 1 -- 172.21.15.190:0/2630865900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c13a2a0 msgr2=0x7fd33c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd340dde640 1 --2- 172.21.15.190:0/2630865900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c13a2a0 0x7fd33c0a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0098a0 tx=0x7fd32c01c410 comp rx=0 tx=0).ready entity=mon.3 client_cookie=19fa35018b73733b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 1 -- 172.21.15.190:0/2630865900 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd32c0b0070 con 0x7fd33c13a2a0 2024-08-19T10:40:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 10 monclient: dump: 2024-08-19T10:40:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 10 monclient: _finish_auth 0 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 1 -- 172.21.15.190:0/2630865900 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd32c0aa810 con 0x7fd33c13a2a0 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 1 -- 172.21.15.190:0/2630865900 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd32c0aae30 con 0x7fd33c13a2a0 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd333fff640 10 monclient: dump: 2024-08-19T10:40:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:34.802 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:34.802 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:34.802 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 5 monclient: authenticate success, global_id 4836 2024-08-19T10:40:34.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:34.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 4 set_mon_vals no callback set 2024-08-19T10:40:34.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 10 monclient: shutdown 2024-08-19T10:40:34.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:34.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 1 -- 172.21.15.190:0/2630865900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c13a2a0 msgr2=0x7fd33c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:34.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 1 --2- 172.21.15.190:0/2630865900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c13a2a0 0x7fd33c0a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0098a0 tx=0x7fd32c01c410 comp rx=0 tx=0).stop 2024-08-19T10:40:34.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 1 -- 172.21.15.190:0/2630865900 shutdown_connections 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 1 --2- 172.21.15.190:0/2630865900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd33c13a2a0 0x7fd33c0a5730 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 1 --2- 172.21.15.190:0/2630865900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd33c139980 0x7fd33c139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 1 -- 172.21.15.190:0/2630865900 >> 172.21.15.190:0/2630865900 conn(0x7fd33c0abde0 msgr2=0x7fd33c0ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 1 -- 172.21.15.190:0/2630865900 shutdown_connections 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 1 -- 172.21.15.190:0/2630865900 wait complete. 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 10 monclient: build_initial_monmap 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.791+0000 7fd343843640 10 monclient: monmap: 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:34.803 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 1 Processor -- start 2024-08-19T10:40:34.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 1 -- start start 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 10 monclient: init 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 10 monclient: _add_conns ranks=[6,1,7,4,2,5,3,0,8] 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd33c0a5c70 0x7fd33c0a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 10 monclient(hunting): picked mon.noname-d con 0x7fd33c0a5c70 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c139980 0x7fd33c0a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 10 monclient(hunting): picked mon.noname-f con 0x7fd33c139980 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd33c0a0b30 0x7fd33c1d1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 10 monclient(hunting): picked mon.noname-i con 0x7fd33c0a0b30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c144280 con 0x7fd33c139980 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:34.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd33c144100 con 0x7fd33c0a5c70 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd33c144400 con 0x7fd33c0a0b30 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd343843640 10 monclient(hunting): authenticate will time out at 1080.364935s 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd341de0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd33c0a0b30 0x7fd33c1d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd340dde640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c139980 0x7fd33c0a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd340dde640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c139980 0x7fd33c0a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:47804/0 (socket says 172.21.15.190:47804) 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd341de0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd33c0a0b30 0x7fd33c1d1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44280/0 (socket says 172.21.15.190:44280) 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd340dde640 1 -- 172.21.15.190:0/1572363506 learned_addr learned my addr 172.21.15.190:0/1572363506 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd340dde640 10 monclient(hunting): get_auth_request con 0x7fd33c139980 auth_method 0 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd340dde640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:34.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd340dde640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd340dde640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd341de0640 10 monclient(hunting): get_auth_request con 0x7fd33c0a0b30 auth_method 0 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd341de0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd341de0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd341de0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd340dde640 1 -- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c139980 msgr2=0x7fd33c0a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd340dde640 1 -- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c139980 msgr2=0x7fd33c0a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd341de0640 1 -- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd33c0a0b30 msgr2=0x7fd33c1d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd340dde640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c139980 0x7fd33c0a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd341de0640 1 -- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd33c0a0b30 msgr2=0x7fd33c1d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd341de0640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd33c0a0b30 0x7fd33c1d1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:34.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd341de0640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd33c0a0b30 0x7fd33c1d1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd340dde640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c139980 0x7fd33c0a05f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd33c0a5c70 0x7fd33c0a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 10 monclient(hunting): get_auth_request con 0x7fd33c0a5c70 auth_method 0 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 10 monclient(hunting): handle_auth_done global_id 4830 payload 824 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 1 -- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd33c0a0b30 msgr2=0x7fd33c1d1480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd33c0a0b30 0x7fd33c1d1480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 1 -- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c139980 msgr2=0x7fd33c0a05f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:34.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c139980 0x7fd33c0a05f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:34.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 10 monclient: _finish_hunting 0 2024-08-19T10:40:34.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 1 monclient: found mon.noname-d 2024-08-19T10:40:34.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:34.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:34.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.795+0000 7fd3415df640 1 -- 172.21.15.190:0/1572363506 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33c0a2080 con 0x7fd33c0a5c70 2024-08-19T10:40:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd3415df640 1 -- 172.21.15.190:0/1572363506 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd33c0a5c70 msgr2=0x7fd33c0a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd3415df640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd33c0a5c70 0x7fd33c0a00b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd33800b3f0 tx=0x7fd33800b8b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 1 -- 172.21.15.190:0/1572363506 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd338015020 con 0x7fd33c0a5c70 2024-08-19T10:40:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 10 monclient: dump: 2024-08-19T10:40:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.811 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:34.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 1 -- 172.21.15.190:0/1572363506 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd338011cb0 con 0x7fd33c0a5c70 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd343843640 5 monclient: authenticate success, global_id 4830 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd343843640 10 monclient: _renew_subs 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd343843640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd343843640 1 -- 172.21.15.190:0/1572363506 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33c0a1b60 con 0x7fd33c0a5c70 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 1 -- 172.21.15.190:0/1572363506 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd338011e60 con 0x7fd33c0a5c70 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd3337fe640 4 set_mon_vals no callback set 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 10 monclient: dump: 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:34.813 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:34.814 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd343843640 10 monclient: _renew_subs 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd343843640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd343843640 1 -- 172.21.15.190:0/1572363506 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fd33c13c010 con 0x7fd33c0a5c70 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd343843640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd343843640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd343843640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd343843640 1 -- 172.21.15.190:0/1572363506 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd33c139de0 con 0x7fd33c0a5c70 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 1 -- 172.21.15.190:0/1572363506 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd338018020 con 0x7fd33c0a5c70 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd32003be00 0x7fd32003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd331ffb640 1 -- 172.21.15.190:0/1572363506 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(68..68 src has 1..68) ==== 5107+0+0 (secure 0 0 0) 0x7fd33804f0e0 con 0x7fd33c0a5c70 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd340dde640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd32003be00 0x7fd32003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.799+0000 7fd340dde640 10 monclient: get_auth_request con 0x7fd32003be00 auth_method 0 2024-08-19T10:40:34.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.803+0000 7fd340dde640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd32003be00 0x7fd32003e2c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009870 tx=0x7fd32c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:34.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.811+0000 7fd331ffb640 1 -- 172.21.15.190:0/1572363506 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd33801f020 con 0x7fd33c0a5c70 2024-08-19T10:40:34.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.811+0000 7fd331ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:34.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.811+0000 7fd331ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:34.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.979+0000 7fd343843640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-08-19T10:40:34.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.979+0000 7fd343843640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-08-19T10:40:34.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.979+0000 7fd343843640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:34.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:34.979+0000 7fd343843640 1 -- 172.21.15.190:0/1572363506 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1} v 0) -- 0x7fd33c134770 con 0x7fd33c0a5c70 2024-08-19T10:40:35.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.239+0000 7fd331ffb640 1 -- 172.21.15.190:0/1572363506 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}]=0 pool 'pinning_pool' created v69) ==== 127+0+0 (secure 0 0 0) 0x7fd33c0a2080 con 0x7fd33c0a5c70 2024-08-19T10:40:35.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.239+0000 7fd331ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-08-19T10:40:35.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.239+0000 7fd331ffb640 10 monclient: _finish_command 2 = system:0 pool 'pinning_pool' created 2024-08-19T10:40:35.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.335+0000 7fd343843640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-08-19T10:40:35.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.335+0000 7fd343843640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-08-19T10:40:35.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.335+0000 7fd343843640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:35.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.335+0000 7fd343843640 1 -- 172.21.15.190:0/1572363506 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1} v 0) -- 0x7fd33c0008e0 con 0x7fd33c0a5c70 2024-08-19T10:40:35.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd331ffb640 1 -- 172.21.15.190:0/1572363506 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}]=0 pool 'pinning_pool' already exists v69) ==== 134+0+0 (secure 0 0 0) 0x7fd338010380 con 0x7fd33c0a5c70 2024-08-19T10:40:35.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd331ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "pinning_pool", "pg_num": 1}] 2024-08-19T10:40:35.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd331ffb640 10 monclient: _finish_command 3 = system:0 pool 'pinning_pool' already exists 2024-08-19T10:40:35.340 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'pinning_pool' already exists 2024-08-19T10:40:35.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 -- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd32003be00 msgr2=0x7fd32003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:35.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd32003be00 0x7fd32003e2c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009870 tx=0x7fd32c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:40:35.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 10 monclient: shutdown 2024-08-19T10:40:35.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:35.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 -- 172.21.15.190:0/1572363506 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd33c0a5c70 msgr2=0x7fd33c0a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:35.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd33c0a5c70 0x7fd33c0a00b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd33800b3f0 tx=0x7fd33800b8b0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 -- 172.21.15.190:0/1572363506 shutdown_connections 2024-08-19T10:40:35.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd32003be00 0x7fd32003e2c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd33c0a0b30 0x7fd33c1d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd33c139980 0x7fd33c0a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 --2- 172.21.15.190:0/1572363506 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd33c0a5c70 0x7fd33c0a00b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 -- 172.21.15.190:0/1572363506 >> 172.21.15.190:0/1572363506 conn(0x7fd33c0abde0 msgr2=0x7fd33c0aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:35.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 -- 172.21.15.190:0/1572363506 shutdown_connections 2024-08-19T10:40:35.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.339+0000 7fd343843640 1 -- 172.21.15.190:0/1572363506 wait complete. 2024-08-19T10:40:35.358 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush rule rm replicated_rule 2024-08-19T10:40:35.358 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:40:35.358 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rm replicated_rule 2024-08-19T10:40:35.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.475+0000 7f9eda301640 10 monclient: get_monmap_and_config 2024-08-19T10:40:35.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.475+0000 7f9eda301640 10 monclient: build_initial_monmap 2024-08-19T10:40:35.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.475+0000 7f9eda301640 10 monclient: monmap: 2024-08-19T10:40:35.478 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:35.478 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:35.478 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:35.478394+0000 2024-08-19T10:40:35.478 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:35.478394+0000 2024-08-19T10:40:35.478 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:35.479 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.475+0000 7f9eda301640 1 Processor -- start 2024-08-19T10:40:35.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.475+0000 7f9eda301640 1 -- start start 2024-08-19T10:40:35.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.475+0000 7f9eda301640 10 monclient: init 2024-08-19T10:40:35.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:35.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 10 monclient: _add_conns ranks=[9,6,7,3,10,1,5,8,2,0,4] 2024-08-19T10:40:35.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9ed4139980 0x7f9ed4139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 10 monclient(hunting): picked mon.noname-a con 0x7f9ed4139980 addr v2:172.21.15.89:3300/0 2024-08-19T10:40:35.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9ed413a2a0 0x7f9ed40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 10 monclient(hunting): picked mon.noname-d con 0x7f9ed413a2a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:35.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9ed40a5c70 0x7f9ed40a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 10 monclient(hunting): picked mon.noname-i con 0x7f9ed40a5c70 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f9ed409ca60 con 0x7f9ed4139980 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9ed40a3640 con 0x7f9ed413a2a0 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9ed40a8470 con 0x7f9ed40a5c70 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9eda301640 10 monclient(hunting): authenticate will time out at 1081.048319s 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed889e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9ed40a5c70 0x7f9ed40a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed889e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9ed40a5c70 0x7f9ed40a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:41452/0 (socket says 172.21.15.190:41452) 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed889e640 1 -- 172.21.15.190:0/614468973 learned_addr learned my addr 172.21.15.190:0/614468973 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed889e640 10 monclient(hunting): get_auth_request con 0x7f9ed40a5c70 auth_method 0 2024-08-19T10:40:35.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed889e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed889e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed889e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed889e640 1 -- 172.21.15.190:0/614468973 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9ed40a5c70 msgr2=0x7f9ed40a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed889e640 1 -- 172.21.15.190:0/614468973 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9ed40a5c70 msgr2=0x7f9ed40a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed889e640 1 --2- 172.21.15.190:0/614468973 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9ed40a5c70 0x7f9ed40a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed889e640 1 --2- 172.21.15.190:0/614468973 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9ed40a5c70 0x7f9ed40a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed3fff640 1 --2- 172.21.15.190:0/614468973 >> v2:172.21.15.89:3300/0 conn(0x7f9ed4139980 0x7f9ed4139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 1 --2- 172.21.15.190:0/614468973 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9ed413a2a0 0x7f9ed40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed3fff640 10 monclient(hunting): get_auth_request con 0x7f9ed4139980 auth_method 0 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:35.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:35.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 10 monclient(hunting): get_auth_request con 0x7f9ed413a2a0 auth_method 0 2024-08-19T10:40:35.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:35.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:35.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:35.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:35.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:35.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:35.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 10 monclient(hunting): handle_auth_done global_id 4839 payload 274 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 1 -- 172.21.15.190:0/614468973 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9ed40a5c70 msgr2=0x7f9ed40a67e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 1 --2- 172.21.15.190:0/614468973 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9ed40a5c70 0x7f9ed40a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 1 -- 172.21.15.190:0/614468973 >> v2:172.21.15.89:3300/0 conn(0x7f9ed4139980 msgr2=0x7f9ed4139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 1 --2- 172.21.15.190:0/614468973 >> v2:172.21.15.89:3300/0 conn(0x7f9ed4139980 0x7f9ed4139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 1 monclient: found mon.noname-d 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:35.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ecb7fe640 1 -- 172.21.15.190:0/614468973 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed40a3850 con 0x7f9ed413a2a0 2024-08-19T10:40:35.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.479+0000 7f9ed3fff640 1 --2- 172.21.15.190:0/614468973 >> v2:172.21.15.89:3300/0 conn(0x7f9ed4139980 0x7f9ed4139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:35.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ecb7fe640 1 -- 172.21.15.190:0/614468973 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9ed413a2a0 msgr2=0x7f9ed40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:35.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ecb7fe640 1 --2- 172.21.15.190:0/614468973 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9ed413a2a0 0x7f9ed40a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0097c0 tx=0x7f9ebc01c0a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ae9019c832db8c0f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:35.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 1 -- 172.21.15.190:0/614468973 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ebc0b1070 con 0x7f9ed413a2a0 2024-08-19T10:40:35.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:35.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:35.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 10 monclient: dump: 2024-08-19T10:40:35.494 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:35.494 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:35.494 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.494 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 1 -- 172.21.15.190:0/614468973 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc0a4ce0 con 0x7f9ed413a2a0 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 1 -- 172.21.15.190:0/614468973 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ebc0aac20 con 0x7f9ed413a2a0 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9ed37fe640 10 monclient: dump: 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.496 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 5 monclient: authenticate success, global_id 4839 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 4 set_mon_vals no callback set 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 10 monclient: shutdown 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 1 -- 172.21.15.190:0/614468973 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9ed413a2a0 msgr2=0x7f9ed40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 1 --2- 172.21.15.190:0/614468973 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9ed413a2a0 0x7f9ed40a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0097c0 tx=0x7f9ebc01c0a0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 1 -- 172.21.15.190:0/614468973 shutdown_connections 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 1 --2- 172.21.15.190:0/614468973 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9ed40a5c70 0x7f9ed40a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 1 --2- 172.21.15.190:0/614468973 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9ed413a2a0 0x7f9ed40a5730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 1 --2- 172.21.15.190:0/614468973 >> v2:172.21.15.89:3300/0 conn(0x7f9ed4139980 0x7f9ed4139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 1 -- 172.21.15.190:0/614468973 >> 172.21.15.190:0/614468973 conn(0x7f9ed40abde0 msgr2=0x7f9ed40ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:35.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 1 -- 172.21.15.190:0/614468973 shutdown_connections 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.483+0000 7f9eda301640 1 -- 172.21.15.190:0/614468973 wait complete. 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient: build_initial_monmap 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient: monmap: 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:35.499 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 1 Processor -- start 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 1 -- start start 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient: init 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient: _add_conns ranks=[3,5,4,1,7,6,8,2,0] 2024-08-19T10:40:35.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ed40a5c70 0x7f9ed409fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient(hunting): picked mon.noname-g con 0x7f9ed40a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9ed4139980 0x7f9ed40a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient(hunting): picked mon.noname-h con 0x7f9ed4139980 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed40a06a0 0x7f9ed41d1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient(hunting): picked mon.noname-c con 0x7f9ed40a06a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed4143270 con 0x7f9ed40a5c70 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9ed41430f0 con 0x7f9ed40a06a0 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f9ed41433f0 con 0x7f9ed4139980 2024-08-19T10:40:35.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9eda301640 10 monclient(hunting): authenticate will time out at 1081.057390s 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ed40a5c70 0x7f9ed409fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ed40a5c70 0x7f9ed409fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42452/0 (socket says 172.21.15.190:42452) 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 1 -- 172.21.15.190:0/3793450303 learned_addr learned my addr 172.21.15.190:0/3793450303 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 10 monclient(hunting): get_auth_request con 0x7f9ed40a5c70 auth_method 0 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed889e640 1 --2- 172.21.15.190:0/3793450303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed40a06a0 0x7f9ed41d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 10 monclient(hunting): handle_auth_done global_id 4845 payload 824 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 1 -- 172.21.15.190:0/3793450303 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9ed4139980 msgr2=0x7f9ed40a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 1 --2- 172.21.15.190:0/3793450303 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9ed4139980 0x7f9ed40a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 1 -- 172.21.15.190:0/3793450303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed40a06a0 msgr2=0x7f9ed41d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 1 --2- 172.21.15.190:0/3793450303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed40a06a0 0x7f9ed41d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 1 monclient: found mon.noname-g 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed3fff640 1 -- 172.21.15.190:0/3793450303 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed413b3b0 con 0x7f9ed40a5c70 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed889e640 10 monclient: get_auth_request con 0x7f9ed40a06a0 auth_method 0 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.487+0000 7f9ed889e640 1 --2- 172.21.15.190:0/3793450303 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9ed40a06a0 0x7f9ed41d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed3fff640 1 -- 172.21.15.190:0/3793450303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ed40a5c70 msgr2=0x7f9ed409fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:35.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed3fff640 1 --2- 172.21.15.190:0/3793450303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ed40a5c70 0x7f9ed409fc20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400ec10 tx=0x7f9ec400c520 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 1 -- 172.21.15.190:0/3793450303 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ec4017070 con 0x7f9ed40a5c70 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 10 monclient: dump: 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:35.504 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 10 monclient: _finish_auth 0 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 10 monclient: _check_auth_tickets 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9eda301640 5 monclient: authenticate success, global_id 4845 2024-08-19T10:40:35.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 1 -- 172.21.15.190:0/3793450303 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec400ce40 con 0x7f9ed40a5c70 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 1 -- 172.21.15.190:0/3793450303 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ec4013d50 con 0x7f9ed40a5c70 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed2ffd640 4 set_mon_vals no callback set 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 10 monclient: dump: 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.506 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9eda301640 10 monclient: _renew_subs 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9eda301640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9eda301640 1 -- 172.21.15.190:0/3793450303 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed40a5730 con 0x7f9ed40a5c70 2024-08-19T10:40:35.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9eda301640 10 monclient: _renew_subs 2024-08-19T10:40:35.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9eda301640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:35.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9eda301640 1 -- 172.21.15.190:0/3793450303 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed413ab10 con 0x7f9ed40a5c70 2024-08-19T10:40:35.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9eda301640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:35.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9eda301640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:35.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9eda301640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:35.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9eda301640 1 -- 172.21.15.190:0/3793450303 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed4139d60 con 0x7f9ed40a5c70 2024-08-19T10:40:35.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 1 -- 172.21.15.190:0/3793450303 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9ec401a070 con 0x7f9ed40a5c70 2024-08-19T10:40:35.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 1 --2- 172.21.15.190:0/3793450303 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9eb403be00 0x7f9eb403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ed17fa640 1 -- 172.21.15.190:0/3793450303 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(69..69 src has 1..69) ==== 5458+0+0 (secure 0 0 0) 0x7f9ec400f040 con 0x7f9ed40a5c70 2024-08-19T10:40:35.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ecb7fe640 1 --2- 172.21.15.190:0/3793450303 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9eb403be00 0x7f9eb403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.491+0000 7f9ecb7fe640 10 monclient: get_auth_request con 0x7f9eb403be00 auth_method 0 2024-08-19T10:40:35.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.495+0000 7f9ecb7fe640 1 --2- 172.21.15.190:0/3793450303 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9eb403be00 0x7f9eb403e2c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0097c0 tx=0x7f9ebc004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:35.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.495+0000 7f9ed17fa640 1 -- 172.21.15.190:0/3793450303 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9ec4021070 con 0x7f9ed40a5c70 2024-08-19T10:40:35.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.495+0000 7f9ed17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:35.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.495+0000 7f9ed17fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:35.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.667+0000 7f9eda301640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-08-19T10:40:35.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.667+0000 7f9eda301640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-08-19T10:40:35.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.667+0000 7f9eda301640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:35.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.667+0000 7f9eda301640 1 -- 172.21.15.190:0/3793450303 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rule rm", "name": "replicated_rule"} v 0) -- 0x7f9ed413af10 con 0x7f9ed40a5c70 2024-08-19T10:40:35.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.671+0000 7f9ed17fa640 1 -- 172.21.15.190:0/3793450303 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "replicated_rule"}]=-16 crush rule replicated_rule (0) is in use v69) ==== 132+0+0 (secure 0 0 0) 0x7f9ec401ac80 con 0x7f9ed40a5c70 2024-08-19T10:40:35.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.671+0000 7f9ed17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "replicated_rule"}] 2024-08-19T10:40:35.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.671+0000 7f9ed17fa640 10 monclient: _finish_command 2 = mon:16 crush rule replicated_rule (0) is in use 2024-08-19T10:40:35.672 INFO:tasks.workunit.client.0.smithi190.stderr:Error EBUSY: crush rule replicated_rule (0) is in use 2024-08-19T10:40:35.676 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:40:35.676 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool rm pinning_pool pinning_pool --yes-i-really-really-mean-it 2024-08-19T10:40:35.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient: get_monmap_and_config 2024-08-19T10:40:35.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient: build_initial_monmap 2024-08-19T10:40:35.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient: monmap: 2024-08-19T10:40:35.784 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:35.784 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:35.784 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:35.783795+0000 2024-08-19T10:40:35.784 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:35.783795+0000 2024-08-19T10:40:35.784 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:35.784 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.784 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:35.784 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:35.784 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:35.784 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:35.784 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:35.785 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:35.785 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:35.785 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:35.785 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:35.785 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:35.785 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:35.785 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 1 Processor -- start 2024-08-19T10:40:35.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 1 -- start start 2024-08-19T10:40:35.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient: init 2024-08-19T10:40:35.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:35.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient: _add_conns ranks=[8,10,1,2,4,6,3,0,9,7,5] 2024-08-19T10:40:35.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc0a4720 0x7fe5cc0a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient(hunting): picked mon.noname-e con 0x7fe5cc0a4720 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe5cc0a3010 0x7fe5cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient(hunting): picked mon.noname-f con 0x7fe5cc0a3010 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe5cc0a2950 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5cc09cc70 con 0x7fe5cc0a2950 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe5cc0a2740 con 0x7fe5cc0a3010 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fe5cc0a2dd0 con 0x7fe5cc0a4720 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d2cbb640 10 monclient(hunting): authenticate will time out at 1081.353649s 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5cbfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe5cc0a3010 0x7fe5cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5cbfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe5cc0a3010 0x7fe5cc0a5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60572/0 (socket says 172.21.15.190:60572) 2024-08-19T10:40:35.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5cbfff640 1 -- 172.21.15.190:0/1326384583 learned_addr learned my addr 172.21.15.190:0/1326384583 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5cbfff640 10 monclient(hunting): get_auth_request con 0x7fe5cc0a3010 auth_method 0 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5cbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5cbfff640 1 -- 172.21.15.190:0/1326384583 >> v2:172.21.15.190:3300/0 conn(0x7fe5cc0a3010 msgr2=0x7fe5cc0a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5cbfff640 1 -- 172.21.15.190:0/1326384583 >> v2:172.21.15.190:3300/0 conn(0x7fe5cc0a3010 msgr2=0x7fe5cc0a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5cbfff640 1 --2- 172.21.15.190:0/1326384583 >> v2:172.21.15.190:3300/0 conn(0x7fe5cc0a3010 0x7fe5cc0a5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5cbfff640 1 --2- 172.21.15.190:0/1326384583 >> v2:172.21.15.190:3300/0 conn(0x7fe5cc0a3010 0x7fe5cc0a5040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 1 --2- 172.21.15.190:0/1326384583 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc0a4720 0x7fe5cc0a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 10 monclient(hunting): get_auth_request con 0x7fe5cc0a4720 auth_method 0 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:35.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 10 monclient(hunting): handle_auth_done global_id 4733 payload 274 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 1 -- 172.21.15.190:0/1326384583 >> v2:172.21.15.190:3300/0 conn(0x7fe5cc0a3010 msgr2=0x7fe5cc0a5040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 1 --2- 172.21.15.190:0/1326384583 >> v2:172.21.15.190:3300/0 conn(0x7fe5cc0a3010 0x7fe5cc0a5040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 1 -- 172.21.15.190:0/1326384583 >> v1:172.21.15.190:6789/0 conn(0x7fe5cc0a2950 legacy=0x7fe5cc1374e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 10 monclient: _finish_hunting 0 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 1 monclient: found mon.noname-e 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:35.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.783+0000 7fe5d0a57640 1 -- 172.21.15.190:0/1326384583 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc0a6210 con 0x7fe5cc0a4720 2024-08-19T10:40:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.787+0000 7fe5d0a57640 1 -- 172.21.15.190:0/1326384583 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc0a4720 msgr2=0x7fe5cc0a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.787+0000 7fe5d0a57640 1 --2- 172.21.15.190:0/1326384583 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc0a4720 0x7fe5cc0a4b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009d90 tx=0x7fe5c401c830 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6412902f6d27697a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 1 -- 172.21.15.190:0/1326384583 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5c40b0870 con 0x7fe5cc0a4720 2024-08-19T10:40:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 10 monclient: dump: 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 1 -- 172.21.15.190:0/1326384583 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c40b0a30 con 0x7fe5cc0a4720 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 1 -- 172.21.15.190:0/1326384583 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5c40b0be0 con 0x7fe5cc0a4720 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 5 monclient: authenticate success, global_id 4733 2024-08-19T10:40:35.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 4 set_mon_vals no callback set 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cb7fe640 10 monclient: dump: 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient: shutdown 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/1326384583 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc0a4720 msgr2=0x7fe5cc0a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 --2- 172.21.15.190:0/1326384583 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc0a4720 0x7fe5cc0a4b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009d90 tx=0x7fe5c401c830 comp rx=0 tx=0).stop 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/1326384583 shutdown_connections 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 --2- 172.21.15.190:0/1326384583 >> v2:172.21.15.190:3300/0 conn(0x7fe5cc0a3010 0x7fe5cc0a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 --2- 172.21.15.190:0/1326384583 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc0a4720 0x7fe5cc0a4b00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/1326384583 >> 172.21.15.190:0/1326384583 conn(0x7fe5cc0a9310 msgr2=0x7fe5cc0a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/1326384583 shutdown_connections 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/1326384583 wait complete. 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient: build_initial_monmap 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient: monmap: 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:35.799 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 Processor -- start 2024-08-19T10:40:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 -- start start 2024-08-19T10:40:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient: init 2024-08-19T10:40:35.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient: _add_conns ranks=[6,8,0,2,7,1,4,5,3] 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe5cc0a2950 0x7fe5cc11f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient(hunting): picked mon.noname-d con 0x7fe5cc0a2950 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc1365a0 0x7fe5cc1cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient(hunting): picked mon.noname-e con 0x7fe5cc1365a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5cc136a00 0x7fe5cc1cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient(hunting): picked mon.noname-a con 0x7fe5cc136a00 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc1422f0 con 0x7fe5cc136a00 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fe5cc142170 con 0x7fe5cc0a2950 2024-08-19T10:40:35.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fe5cc142470 con 0x7fe5cc1365a0 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d2cbb640 10 monclient(hunting): authenticate will time out at 1081.362873s 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d0a57640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe5cc0a2950 0x7fe5cc11f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cbfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc1365a0 0x7fe5cc1cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d0a57640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe5cc0a2950 0x7fe5cc11f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:35222/0 (socket says 172.21.15.190:35222) 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d0a57640 1 -- 172.21.15.190:0/3561833257 learned_addr learned my addr 172.21.15.190:0/3561833257 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d0a57640 10 monclient(hunting): get_auth_request con 0x7fe5cc0a2950 auth_method 0 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d0a57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d0a57640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d0a57640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d1258640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5cc136a00 0x7fe5cc1cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cbfff640 10 monclient(hunting): get_auth_request con 0x7fe5cc1365a0 auth_method 0 2024-08-19T10:40:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d1258640 10 monclient(hunting): get_auth_request con 0x7fe5cc136a00 auth_method 0 2024-08-19T10:40:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d1258640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d1258640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d1258640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d0a57640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d0a57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d0a57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d1258640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d1258640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d1258640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.791+0000 7fe5d0a57640 10 monclient(hunting): handle_auth_done global_id 4848 payload 824 2024-08-19T10:40:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d0a57640 1 -- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc1365a0 msgr2=0x7fe5cc1cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d0a57640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc1365a0 0x7fe5cc1cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d0a57640 1 -- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5cc136a00 msgr2=0x7fe5cc1cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d0a57640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5cc136a00 0x7fe5cc1cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d0a57640 10 monclient: _finish_hunting 0 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d0a57640 1 monclient: found mon.noname-d 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d0a57640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d0a57640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d0a57640 1 -- 172.21.15.190:0/3561833257 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc09ec00 con 0x7fe5cc0a2950 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5cbfff640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc1365a0 0x7fe5cc1cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d0a57640 1 -- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe5cc0a2950 msgr2=0x7fe5cc11f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d0a57640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe5cc0a2950 0x7fe5cc11f730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe5c401d0d0 tx=0x7fe5c40195d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 1 -- 172.21.15.190:0/3561833257 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5c4003bb0 con 0x7fe5cc0a2950 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 10 monclient: dump: 2024-08-19T10:40:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 10 monclient: _finish_auth 0 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 10 monclient: _check_auth_tickets 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d2cbb640 5 monclient: authenticate success, global_id 4848 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d2cbb640 10 monclient: _renew_subs 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d2cbb640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/3561833257 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc13d020 con 0x7fe5cc0a2950 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d2cbb640 10 monclient: _renew_subs 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d2cbb640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/3561833257 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc120f30 con 0x7fe5cc0a2950 2024-08-19T10:40:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 1 -- 172.21.15.190:0/3561833257 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c4003d50 con 0x7fe5cc0a2950 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 1 -- 172.21.15.190:0/3561833257 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe5c40044b0 con 0x7fe5cc0a2950 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5c97fa640 10 monclient: dump: 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5caffd640 4 set_mon_vals no callback set 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d2cbb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d2cbb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d2cbb640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.795+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/3561833257 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a4005090 con 0x7fe5cc0a2950 2024-08-19T10:40:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.799+0000 7fe5c97fa640 1 -- 172.21.15.190:0/3561833257 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe5c40c2030 con 0x7fe5cc0a2950 2024-08-19T10:40:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.799+0000 7fe5c97fa640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe5b003be00 0x7fe5b003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.799+0000 7fe5c97fa640 1 -- 172.21.15.190:0/3561833257 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(69..69 src has 1..69) ==== 5458+0+0 (secure 0 0 0) 0x7fe5c40c0020 con 0x7fe5cc0a2950 2024-08-19T10:40:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.799+0000 7fe5cbfff640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe5b003be00 0x7fe5b003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.799+0000 7fe5cbfff640 10 monclient: get_auth_request con 0x7fe5b003be00 auth_method 0 2024-08-19T10:40:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.799+0000 7fe5cbfff640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe5b003be00 0x7fe5b003e2c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0a28e0 tx=0x7fe5b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.811+0000 7fe5c97fa640 1 -- 172.21.15.190:0/3561833257 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe5c40c9060 con 0x7fe5cc0a2950 2024-08-19T10:40:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.811+0000 7fe5c97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.811+0000 7fe5c97fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:35.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.967+0000 7fe5d2cbb640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:35.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.967+0000 7fe5d2cbb640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:35.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.967+0000 7fe5d2cbb640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:35.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:35.967+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/3561833257 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe5a4005380 con 0x7fe5cc0a2950 2024-08-19T10:40:36.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:36.903+0000 7fe5c97fa640 1 -- 172.21.15.190:0/3561833257 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}]=0 pool 'pinning_pool' removed v70) ==== 172+0+0 (secure 0 0 0) 0x7fe5cc09ec00 con 0x7fe5cc0a2950 2024-08-19T10:40:36.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:36.903+0000 7fe5c97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:36.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:36.903+0000 7fe5c97fa640 10 monclient: _finish_command 2 = system:0 pool 'pinning_pool' removed 2024-08-19T10:40:37.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.003+0000 7fe5d2cbb640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:37.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.003+0000 7fe5d2cbb640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:37.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.003+0000 7fe5d2cbb640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:37.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.003+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/3561833257 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe5a4005b30 con 0x7fe5cc0a2950 2024-08-19T10:40:37.352 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-08-19T10:40:37.353 INFO:tasks.ceph.mon.b:Sent signal 18 2024-08-19T10:40:37.353 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-08-19T10:40:37.353 INFO:tasks.ceph.mon.h:Sent signal 18 2024-08-19T10:40:37.353 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:40:37.353 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:40:37.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient: get_monmap_and_config 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient: build_initial_monmap 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient: monmap: 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:37.482233+0000 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:37.482233+0000 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:37.482 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:37.483 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:37.483 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:37.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 1 Processor -- start 2024-08-19T10:40:37.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 1 -- start start 2024-08-19T10:40:37.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient: init 2024-08-19T10:40:37.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:37.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient: _add_conns ranks=[7,3,1,2,10,6,9,0,5,4,8] 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient(hunting): picked mon.noname-i con 0x7fd8680a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 0x7fd8680a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient(hunting): picked mon.noname-g con 0x7fd8680a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd8681395b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8680a8710 con 0x7fd8681395b0 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd8680a7650 con 0x7fd8680a6d30 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd8680a5a40 con 0x7fd8680a5270 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:37.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86f3ae640 10 monclient(hunting): authenticate will time out at 1087.299408s 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86c949640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 0x7fd8680a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86d14a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86d94b640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd8681395b0 0x7fd868139a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.89:55596/0 (socket says 172.21.15.89:55596) 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86d94b640 1 -- 172.21.15.89:0/70722324 learned_addr learned my addr 172.21.15.89:0/70722324 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86c949640 10 monclient(hunting): get_auth_request con 0x7fd8680a6d30 auth_method 0 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86c949640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86c949640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86c949640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86d14a640 10 monclient(hunting): get_auth_request con 0x7fd8680a5270 auth_method 0 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86d14a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86d14a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.478+0000 7fd86d14a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:37.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.482+0000 7fd86c949640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 msgr2=0x7fd8680a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:40:37.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.482+0000 7fd86d14a640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 msgr2=0x7fd8680a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:37.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.482+0000 7fd86c949640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 msgr2=0x7fd8680a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:37.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.482+0000 7fd86d14a640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 msgr2=0x7fd8680a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:37.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.482+0000 7fd86c949640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 0x7fd8680a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:37.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.482+0000 7fd86d14a640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:37.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.482+0000 7fd86c949640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 0x7fd8680a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:37.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.482+0000 7fd86d14a640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:37.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.482+0000 7fd86d94b640 1 -- 172.21.15.89:0/70722324 >> v1:172.21.15.190:6789/0 conn(0x7fd8681395b0 legacy=0x7fd868139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:37.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86d14a640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:37.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86c949640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 0x7fd8680a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:37.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86d14a640 10 monclient(hunting): get_auth_request con 0x7fd8680a5270 auth_method 0 2024-08-19T10:40:37.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86d14a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:37.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86d14a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86d14a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86c949640 10 monclient(hunting): get_auth_request con 0x7fd8680a6d30 auth_method 0 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86c949640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86c949640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86c949640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86c949640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 msgr2=0x7fd8680a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86d14a640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 msgr2=0x7fd8680a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86c949640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 msgr2=0x7fd8680a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86d14a640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 msgr2=0x7fd8680a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86c949640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 0x7fd8680a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86d14a640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:37.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86d14a640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:40:37.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:37.686+0000 7fd86c949640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 0x7fd8680a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:40:37.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.971+0000 7fe5c97fa640 1 -- 172.21.15.190:0/3561833257 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}]=0 pool 'pinning_pool' does not exist v70) ==== 179+0+0 (secure 0 0 0) 0x7fe5c40ac030 con 0x7fe5cc0a2950 2024-08-19T10:40:37.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.971+0000 7fe5c97fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "pinning_pool", "pool2": "pinning_pool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:40:37.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.971+0000 7fe5c97fa640 10 monclient: _finish_command 3 = system:0 pool 'pinning_pool' does not exist 2024-08-19T10:40:37.973 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'pinning_pool' does not exist 2024-08-19T10:40:37.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/3561833257 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe5b003be00 msgr2=0x7fe5b003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:37.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe5b003be00 0x7fe5b003e2c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0a28e0 tx=0x7fe5b4008040 comp rx=0 tx=0).stop 2024-08-19T10:40:37.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 10 monclient: shutdown 2024-08-19T10:40:37.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:37.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe5cc0a2950 msgr2=0x7fe5cc11f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:37.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe5cc0a2950 0x7fe5cc11f730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe5c401d0d0 tx=0x7fe5c40195d0 comp rx=0 tx=0).stop 2024-08-19T10:40:37.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/3561833257 shutdown_connections 2024-08-19T10:40:37.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5cc136a00 0x7fe5cc1cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:37.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe5b003be00 0x7fe5b003e2c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:37.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe5cc1365a0 0x7fe5cc1cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:37.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 --2- 172.21.15.190:0/3561833257 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe5cc0a2950 0x7fe5cc11f730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:37.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/3561833257 >> 172.21.15.190:0/3561833257 conn(0x7fe5cc0a9310 msgr2=0x7fe5cc0a7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:37.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/3561833257 shutdown_connections 2024-08-19T10:40:37.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:37.975+0000 7fe5d2cbb640 1 -- 172.21.15.190:0/3561833257 wait complete. 2024-08-19T10:40:37.997 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush add-bucket foo osd 2024-08-19T10:40:37.997 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:40:37.997 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket foo osd 2024-08-19T10:40:38.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 0x7fd8680a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86d14a640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86d14a640 10 monclient(hunting): get_auth_request con 0x7fd8680a5270 auth_method 0 2024-08-19T10:40:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86d14a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86d14a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86d14a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 10 monclient(hunting): get_auth_request con 0x7fd8680a6d30 auth_method 0 2024-08-19T10:40:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86d14a640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 msgr2=0x7fd8680a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:40:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86d14a640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 msgr2=0x7fd8680a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86d14a640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86d14a640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:40:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 10 monclient(hunting): handle_auth_done global_id 4854 payload 274 2024-08-19T10:40:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 msgr2=0x7fd8680a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 1 -- 172.21.15.89:0/70722324 >> v1:172.21.15.190:6789/0 conn(0x7fd8681395b0 legacy=0x7fd868139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 1 monclient: found mon.noname-g 2024-08-19T10:40:38.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:38.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.090+0000 7fd86c949640 1 -- 172.21.15.89:0/70722324 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8680a81d0 con 0x7fd8680a6d30 2024-08-19T10:40:38.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86c949640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 msgr2=0x7fd8680a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86c949640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 0x7fd8680a7110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd85400e980 tx=0x7fd85401c9d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=d22a8725037667c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 1 -- 172.21.15.89:0/70722324 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8540ab070 con 0x7fd8680a6d30 2024-08-19T10:40:38.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:38.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 10 monclient: dump: 2024-08-19T10:40:38.101 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.101 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.101 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.101 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 1 -- 172.21.15.89:0/70722324 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8540a5bc0 con 0x7fd8680a6d30 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 5 monclient: authenticate success, global_id 4854 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 1 -- 172.21.15.89:0/70722324 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd85400f040 con 0x7fd8680a6d30 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd867fff640 10 monclient: dump: 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.103 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:38.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 4 set_mon_vals no callback set 2024-08-19T10:40:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 10 monclient: shutdown 2024-08-19T10:40:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 1 -- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 msgr2=0x7fd8680a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 0x7fd8680a7110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd85400e980 tx=0x7fd85401c9d0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 1 -- 172.21.15.89:0/70722324 shutdown_connections 2024-08-19T10:40:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8680a6d30 0x7fd8680a7110 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 1 --2- 172.21.15.89:0/70722324 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8680a5270 0x7fd8680a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 1 -- 172.21.15.89:0/70722324 >> 172.21.15.89:0/70722324 conn(0x7fd868069a80 msgr2=0x7fd8680abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:38.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 1 -- 172.21.15.89:0/70722324 shutdown_connections 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.098+0000 7fd86f3ae640 1 -- 172.21.15.89:0/70722324 wait complete. 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient: build_initial_monmap 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient: monmap: 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:38.107 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:38.108 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 1 Processor -- start 2024-08-19T10:40:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 1 -- start start 2024-08-19T10:40:38.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient: init 2024-08-19T10:40:38.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient: _add_conns ranks=[4,0,5,6,2,1,3,8,7] 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8680a5270 0x7fd868138200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient(hunting): picked mon.noname-c con 0x7fd8680a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680a6d30 0x7fd86813d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient(hunting): picked mon.noname-a con 0x7fd8680a6d30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd8681395b0 0x7fd86813d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient(hunting): picked mon.noname-h con 0x7fd8681395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd8680a57d0 con 0x7fd8680a6d30 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd8680a5650 con 0x7fd8680a5270 2024-08-19T10:40:38.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd868144240 con 0x7fd8681395b0 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86f3ae640 10 monclient(hunting): authenticate will time out at 1087.922627s 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8680a5270 0x7fd868138200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8680a5270 0x7fd868138200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37390/0 (socket says 172.21.15.89:37390) 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 1 -- 172.21.15.89:0/4082776354 learned_addr learned my addr 172.21.15.89:0/4082776354 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 10 monclient(hunting): get_auth_request con 0x7fd8680a5270 auth_method 0 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86c949640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680a6d30 0x7fd86813d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86c949640 10 monclient(hunting): get_auth_request con 0x7fd8680a6d30 auth_method 0 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d94b640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd8681395b0 0x7fd86813d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86c949640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86c949640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86c949640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d94b640 10 monclient(hunting): get_auth_request con 0x7fd8681395b0 auth_method 0 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d94b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d94b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d94b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 10 monclient(hunting): handle_auth_done global_id 4891 payload 824 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 1 -- 172.21.15.89:0/4082776354 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd8681395b0 msgr2=0x7fd86813d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd8681395b0 0x7fd86813d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 1 -- 172.21.15.89:0/4082776354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680a6d30 msgr2=0x7fd86813d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680a6d30 0x7fd86813d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 1 monclient: found mon.noname-c 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d14a640 1 -- 172.21.15.89:0/4082776354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd868122a00 con 0x7fd8680a5270 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86d94b640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd8681395b0 0x7fd86813d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.102+0000 7fd86c949640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680a6d30 0x7fd86813d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86d14a640 1 -- 172.21.15.89:0/4082776354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8680a5270 msgr2=0x7fd868138200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86d14a640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8680a5270 0x7fd868138200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd85c007ef0 tx=0x7fd85c010590 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 1 -- 172.21.15.89:0/4082776354 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd85c003e10 con 0x7fd8680a5270 2024-08-19T10:40:38.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 10 monclient: dump: 2024-08-19T10:40:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.117 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 1 -- 172.21.15.89:0/4082776354 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c015650 con 0x7fd8680a5270 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86f3ae640 5 monclient: authenticate success, global_id 4891 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86f3ae640 10 monclient: _renew_subs 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86f3ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86f3ae640 1 -- 172.21.15.89:0/4082776354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd868122ce0 con 0x7fd8680a5270 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 1 -- 172.21.15.89:0/4082776354 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd85c015c70 con 0x7fd8680a5270 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 10 monclient: dump: 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.119 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd866ffd640 4 set_mon_vals no callback set 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86f3ae640 10 monclient: _renew_subs 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86f3ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86f3ae640 1 -- 172.21.15.89:0/4082776354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fd86813fab0 con 0x7fd8680a5270 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86f3ae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86f3ae640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86f3ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd86f3ae640 1 -- 172.21.15.89:0/4082776354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8680a67f0 con 0x7fd8680a5270 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.106+0000 7fd865ffb640 1 -- 172.21.15.89:0/4082776354 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd85c015800 con 0x7fd8680a5270 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.110+0000 7fd865ffb640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd84c03be00 0x7fd84c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.110+0000 7fd865ffb640 1 -- 172.21.15.89:0/4082776354 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(70..70 src has 1..70) ==== 5107+0+0 (secure 0 0 0) 0x7fd85c050750 con 0x7fd8680a5270 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.110+0000 7fd86c949640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd84c03be00 0x7fd84c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.110+0000 7fd86c949640 10 monclient: get_auth_request con 0x7fd84c03be00 auth_method 0 2024-08-19T10:40:38.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.110+0000 7fd86c949640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd84c03be00 0x7fd84c03e2c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd8540ad000 tx=0x7fd8540a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.114+0000 7fd865ffb640 1 -- 172.21.15.89:0/4082776354 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd85c015e10 con 0x7fd8680a5270 2024-08-19T10:40:38.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.114+0000 7fd865ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.114+0000 7fd865ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:38.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.115+0000 7fd0ae00b640 10 monclient: get_monmap_and_config 2024-08-19T10:40:38.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.115+0000 7fd0ae00b640 10 monclient: build_initial_monmap 2024-08-19T10:40:38.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.115+0000 7fd0ae00b640 10 monclient: monmap: 2024-08-19T10:40:38.126 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:38.126 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.126 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:38.118802+0000 2024-08-19T10:40:38.126 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:38.118802+0000 2024-08-19T10:40:38.126 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:38.126 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.126 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:38.126 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:38.126 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.115+0000 7fd0ae00b640 1 Processor -- start 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 1 -- start start 2024-08-19T10:40:38.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 10 monclient: init 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 10 monclient: _add_conns ranks=[2,0,5,3,1,7,9,6,10,4,8] 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0a8139980 0x7fd0a8139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 10 monclient(hunting): picked mon.noname-b con 0x7fd0a8139980 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd0a813a2a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd0a813e8c0 0x7fd0a8140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 10 monclient(hunting): picked mon.noname-h con 0x7fd0a813e8c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a80a8470 con 0x7fd0a813a2a0 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a809ca60 con 0x7fd0a8139980 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd0a80a3640 con 0x7fd0a813e8c0 2024-08-19T10:40:38.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0ae00b640 10 monclient(hunting): authenticate will time out at 1083.688669s 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd0a813e8c0 0x7fd0a8140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd0a813e8c0 0x7fd0a8140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:43416/0 (socket says 172.21.15.190:43416) 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 1 -- 172.21.15.190:0/2403169763 learned_addr learned my addr 172.21.15.190:0/2403169763 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 10 monclient(hunting): get_auth_request con 0x7fd0a813e8c0 auth_method 0 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a77fe640 1 --2- 172.21.15.190:0/2403169763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0a8139980 0x7fd0a8139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a77fe640 10 monclient(hunting): get_auth_request con 0x7fd0a8139980 auth_method 0 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 10 monclient(hunting): handle_auth_done global_id 4883 payload 274 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 1 -- 172.21.15.190:0/2403169763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0a8139980 msgr2=0x7fd0a8139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 1 --2- 172.21.15.190:0/2403169763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0a8139980 0x7fd0a8139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 1 -- 172.21.15.190:0/2403169763 >> v1:172.21.15.89:6789/0 conn(0x7fd0a813a2a0 legacy=0x7fd0a80a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 1 monclient: found mon.noname-h 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.119+0000 7fd0a7fff640 1 -- 172.21.15.190:0/2403169763 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a80a3850 con 0x7fd0a813e8c0 2024-08-19T10:40:38.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a7fff640 1 -- 172.21.15.190:0/2403169763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd0a813e8c0 msgr2=0x7fd0a8140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a7fff640 1 --2- 172.21.15.190:0/2403169763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd0a813e8c0 0x7fd0a8140cb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009e80 tx=0x7fd0a001c8e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=9a04db118183b116 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 1 -- 172.21.15.190:0/2403169763 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd0a00090d0 con 0x7fd0a813e8c0 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 10 monclient: dump: 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.131 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 5 monclient: authenticate success, global_id 4883 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 1 -- 172.21.15.190:0/2403169763 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a00a5b30 con 0x7fd0a813e8c0 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:38.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 1 -- 172.21.15.190:0/2403169763 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd0a00a5ce0 con 0x7fd0a813e8c0 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 4 set_mon_vals no callback set 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a67fc640 10 monclient: dump: 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.133 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient: shutdown 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 -- 172.21.15.190:0/2403169763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd0a813e8c0 msgr2=0x7fd0a8140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 --2- 172.21.15.190:0/2403169763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd0a813e8c0 0x7fd0a8140cb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009e80 tx=0x7fd0a001c8e0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 -- 172.21.15.190:0/2403169763 shutdown_connections 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 --2- 172.21.15.190:0/2403169763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd0a813e8c0 0x7fd0a8140cb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 --2- 172.21.15.190:0/2403169763 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0a8139980 0x7fd0a8139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 -- 172.21.15.190:0/2403169763 >> 172.21.15.190:0/2403169763 conn(0x7fd0a80abde0 msgr2=0x7fd0a80ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 -- 172.21.15.190:0/2403169763 shutdown_connections 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 -- 172.21.15.190:0/2403169763 wait complete. 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient: build_initial_monmap 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient: monmap: 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:38.135 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.136 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:38.136 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:38.136 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:38.136 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:38.136 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:38.136 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:38.136 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:38.136 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:38.136 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:38.136 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 Processor -- start 2024-08-19T10:40:38.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 -- start start 2024-08-19T10:40:38.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient: init 2024-08-19T10:40:38.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:38.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient: _add_conns ranks=[7,4,3,0,6,2,8,1,5] 2024-08-19T10:40:38.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0a8139980 0x7fd0a81d1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient(hunting): picked mon.noname-i con 0x7fd0a8139980 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0a813e8c0 0x7fd0a80a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient(hunting): picked mon.noname-c con 0x7fd0a813e8c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0a80a2450 0x7fd0a8137680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient(hunting): picked mon.noname-g con 0x7fd0a80a2450 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a8143b90 con 0x7fd0a80a2450 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd0a8143a10 con 0x7fd0a813e8c0 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd0a8143d10 con 0x7fd0a8139980 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:38.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0ae00b640 10 monclient(hunting): authenticate will time out at 1083.695785s 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0a80a2450 0x7fd0a8137680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a77fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0a8139980 0x7fd0a81d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0a80a2450 0x7fd0a8137680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42468/0 (socket says 172.21.15.190:42468) 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a7fff640 1 -- 172.21.15.190:0/1036165439 learned_addr learned my addr 172.21.15.190:0/1036165439 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a7fff640 10 monclient(hunting): get_auth_request con 0x7fd0a80a2450 auth_method 0 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a77fe640 10 monclient(hunting): get_auth_request con 0x7fd0a8139980 auth_method 0 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.123+0000 7fd0a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a6ffd640 1 --2- 172.21.15.190:0/1036165439 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0a813e8c0 0x7fd0a80a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a6ffd640 10 monclient(hunting): get_auth_request con 0x7fd0a813e8c0 auth_method 0 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a77fe640 1 -- 172.21.15.190:0/1036165439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0a8139980 msgr2=0x7fd0a81d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a77fe640 1 -- 172.21.15.190:0/1036165439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0a8139980 msgr2=0x7fd0a81d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a77fe640 1 --2- 172.21.15.190:0/1036165439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0a8139980 0x7fd0a81d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a77fe640 1 --2- 172.21.15.190:0/1036165439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0a8139980 0x7fd0a81d1120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:38.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 10 monclient(hunting): handle_auth_done global_id 4863 payload 824 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 1 -- 172.21.15.190:0/1036165439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0a8139980 msgr2=0x7fd0a81d1120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 1 --2- 172.21.15.190:0/1036165439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd0a8139980 0x7fd0a81d1120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 1 -- 172.21.15.190:0/1036165439 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0a813e8c0 msgr2=0x7fd0a80a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 1 --2- 172.21.15.190:0/1036165439 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd0a813e8c0 0x7fd0a80a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 1 monclient: found mon.noname-g 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 1 -- 172.21.15.190:0/1036165439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a809e9e0 con 0x7fd0a80a2450 2024-08-19T10:40:38.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 1 -- 172.21.15.190:0/1036165439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0a80a2450 msgr2=0x7fd0a8137680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a7fff640 1 --2- 172.21.15.190:0/1036165439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0a80a2450 0x7fd0a8137680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009040 tx=0x7fd0a0019680 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 1 -- 172.21.15.190:0/1036165439 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd0a00bc020 con 0x7fd0a80a2450 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 10 monclient: dump: 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.145 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0ae00b640 5 monclient: authenticate success, global_id 4863 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 1 -- 172.21.15.190:0/1036165439 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a0009210 con 0x7fd0a80a2450 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0ae00b640 10 monclient: _renew_subs 2024-08-19T10:40:38.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0ae00b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0ae00b640 1 -- 172.21.15.190:0/1036165439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a81390a0 con 0x7fd0a80a2450 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 1 -- 172.21.15.190:0/1036165439 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd0a0003dc0 con 0x7fd0a80a2450 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0a5ffb640 4 set_mon_vals no callback set 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 10 monclient: dump: 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.147 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0ae00b640 10 monclient: _renew_subs 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0ae00b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd0ae00b640 1 -- 172.21.15.190:0/1036165439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a809e140 con 0x7fd0a80a2450 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 1 -- 172.21.15.190:0/1036165439 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd0a00bf020 con 0x7fd0a80a2450 2024-08-19T10:40:38.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.127+0000 7fd09ffff640 1 --2- 172.21.15.190:0/1036165439 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd08803be30 0x7fd08803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.131+0000 7fd09ffff640 1 -- 172.21.15.190:0/1036165439 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(70..70 src has 1..70) ==== 5107+0+0 (secure 0 0 0) 0x7fd0a00ee060 con 0x7fd0a80a2450 2024-08-19T10:40:38.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.131+0000 7fd0ae00b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.131+0000 7fd0ae00b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.131+0000 7fd0ae00b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:38.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.131+0000 7fd0a77fe640 1 --2- 172.21.15.190:0/1036165439 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd08803be30 0x7fd08803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.131+0000 7fd0ae00b640 1 -- 172.21.15.190:0/1036165439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a8139d60 con 0x7fd0a80a2450 2024-08-19T10:40:38.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.131+0000 7fd0a77fe640 10 monclient: get_auth_request con 0x7fd08803be30 auth_method 0 2024-08-19T10:40:38.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.131+0000 7fd0a77fe640 1 --2- 172.21.15.190:0/1036165439 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd08803be30 0x7fd08803e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b990 tx=0x7fd098008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.135+0000 7fd09ffff640 1 -- 172.21.15.190:0/1036165439 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd0a00c6070 con 0x7fd0a80a2450 2024-08-19T10:40:38.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.135+0000 7fd09ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.135+0000 7fd09ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:38.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.303+0000 7fd0ae00b640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-08-19T10:40:38.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.303+0000 7fd0ae00b640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-08-19T10:40:38.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.303+0000 7fd0ae00b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:40:38.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.303+0000 7fd0ae00b640 1 -- 172.21.15.190:0/1036165439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"} v 0) -- 0x7fd0a809d1e0 con 0x7fd0a80a2450 2024-08-19T10:40:38.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.346+0000 7fd86f3ae640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:40:38.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.346+0000 7fd86f3ae640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:40:38.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.346+0000 7fd86f3ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:38.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.346+0000 7fd86f3ae640 1 -- 172.21.15.89:0/4082776354 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd868064590 con 0x7fd8680a5270 2024-08-19T10:40:38.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd865ffb640 1 -- 172.21.15.89:0/4082776354 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7fd85c04e050 con 0x7fd8680a5270 2024-08-19T10:40:38.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd865ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:40:38.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd865ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:38.353 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":26,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:40:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 -- 172.21.15.89:0/4082776354 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd84c03be00 msgr2=0x7fd84c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd84c03be00 0x7fd84c03e2c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd8540ad000 tx=0x7fd8540a7070 comp rx=0 tx=0).stop 2024-08-19T10:40:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 10 monclient: shutdown 2024-08-19T10:40:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 -- 172.21.15.89:0/4082776354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8680a5270 msgr2=0x7fd868138200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8680a5270 0x7fd868138200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd85c007ef0 tx=0x7fd85c010590 comp rx=0 tx=0).stop 2024-08-19T10:40:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 -- 172.21.15.89:0/4082776354 shutdown_connections 2024-08-19T10:40:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd84c03be00 0x7fd84c03e2c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd8681395b0 0x7fd86813d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8680a6d30 0x7fd86813d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 --2- 172.21.15.89:0/4082776354 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd8680a5270 0x7fd868138200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 -- 172.21.15.89:0/4082776354 >> 172.21.15.89:0/4082776354 conn(0x7fd868069a80 msgr2=0x7fd8680aab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 -- 172.21.15.89:0/4082776354 shutdown_connections 2024-08-19T10:40:38.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.350+0000 7fd86f3ae640 1 -- 172.21.15.89:0/4082776354 wait complete. 2024-08-19T10:40:38.373 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:40:38.373 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:40:38.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:40:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient: get_monmap_and_config 2024-08-19T10:40:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient: build_initial_monmap 2024-08-19T10:40:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient: monmap: 2024-08-19T10:40:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:38.489507+0000 2024-08-19T10:40:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:38.489507+0000 2024-08-19T10:40:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:38.489 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:38.490 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:38.490 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 1 Processor -- start 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 1 -- start start 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient: init 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient: _add_conns ranks=[0,5,8,7,1,6,9,3,4,10,2] 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f716c09b8e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f716c09bcc0 0x7f716c09ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient(hunting): picked mon.noname-h con 0x7f716c09bcc0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c0a4410 0x7f716c1414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient(hunting): picked mon.noname-e con 0x7f716c0a4410 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f716c06db50 con 0x7f716c09b8e0 2024-08-19T10:40:38.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f716c09d760 con 0x7f716c09bcc0 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f716c09d9b0 con 0x7f716c0a4410 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7173892640 10 monclient(hunting): authenticate will time out at 1088.306756s 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c0a4410 0x7f716c1414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f717162e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f716c09b8e0 0x7f716c0a3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58306/0 (socket says 172.21.15.89:58306) 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f717162e640 1 -- 172.21.15.89:0/139868644 learned_addr learned my addr 172.21.15.89:0/139868644 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 10 monclient(hunting): get_auth_request con 0x7f716c0a4410 auth_method 0 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7170e2d640 1 --2- 172.21.15.89:0/139868644 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f716c09bcc0 0x7f716c09ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7170e2d640 10 monclient(hunting): get_auth_request con 0x7f716c09bcc0 auth_method 0 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7170e2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7170e2d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7170e2d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 10 monclient(hunting): handle_auth_done global_id 4751 payload 274 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 1 -- 172.21.15.89:0/139868644 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f716c09bcc0 msgr2=0x7f716c09ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 1 --2- 172.21.15.89:0/139868644 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f716c09bcc0 0x7f716c09ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 1 -- 172.21.15.89:0/139868644 >> v1:172.21.15.89:6789/0 conn(0x7f716c09b8e0 legacy=0x7f716c0a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 1 monclient: found mon.noname-e 2024-08-19T10:40:38.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:38.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.486+0000 7f7171e2f640 1 -- 172.21.15.89:0/139868644 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716c06fd00 con 0x7f716c0a4410 2024-08-19T10:40:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7171e2f640 1 -- 172.21.15.89:0/139868644 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c0a4410 msgr2=0x7f716c1414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7171e2f640 1 --2- 172.21.15.89:0/139868644 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c0a4410 0x7f716c1414c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7164009dc0 tx=0x7f716401c850 comp rx=0 tx=0).ready entity=mon.8 client_cookie=76874c26ff15ba19 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 1 -- 172.21.15.89:0/139868644 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f71640b2070 con 0x7f716c0a4410 2024-08-19T10:40:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 10 monclient: dump: 2024-08-19T10:40:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.500 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 1 -- 172.21.15.89:0/139868644 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71640a5990 con 0x7f716c0a4410 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 1 -- 172.21.15.89:0/139868644 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f71640ab340 con 0x7f716c0a4410 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7163fff640 10 monclient: dump: 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.502 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7173892640 5 monclient: authenticate success, global_id 4751 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7173892640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.490+0000 7f7173892640 4 set_mon_vals no callback set 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient: shutdown 2024-08-19T10:40:38.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 -- 172.21.15.89:0/139868644 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c0a4410 msgr2=0x7f716c1414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 --2- 172.21.15.89:0/139868644 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c0a4410 0x7f716c1414c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7164009dc0 tx=0x7f716401c850 comp rx=0 tx=0).stop 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 -- 172.21.15.89:0/139868644 shutdown_connections 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 --2- 172.21.15.89:0/139868644 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c0a4410 0x7f716c1414c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 --2- 172.21.15.89:0/139868644 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f716c09bcc0 0x7f716c09ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 -- 172.21.15.89:0/139868644 >> 172.21.15.89:0/139868644 conn(0x7f716c0a4df0 msgr2=0x7f716c0a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 -- 172.21.15.89:0/139868644 shutdown_connections 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 -- 172.21.15.89:0/139868644 wait complete. 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient: build_initial_monmap 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient: monmap: 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:38.504 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 Processor -- start 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 -- start start 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient: init 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient: _add_conns ranks=[1,8,7,3,2,0,5,4,6] 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c09b8e0 0x7f716c13da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient(hunting): picked mon.noname-f con 0x7f716c09b8e0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c09bcc0 0x7f716c13dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient(hunting): picked mon.noname-e con 0x7f716c09bcc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f716c0a4410 0x7f716c092380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient(hunting): picked mon.noname-i con 0x7f716c0a4410 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f716c144220 con 0x7f716c09b8e0 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f716c069300 con 0x7f716c0a4410 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f716c1443a0 con 0x7f716c09bcc0 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7173892640 10 monclient(hunting): authenticate will time out at 1088.313990s 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c09bcc0 0x7f716c13dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c09bcc0 0x7f716c13dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:37064/0 (socket says 172.21.15.89:37064) 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 1 -- 172.21.15.89:0/3289298443 learned_addr learned my addr 172.21.15.89:0/3289298443 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 10 monclient(hunting): get_auth_request con 0x7f716c09bcc0 auth_method 0 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7171e2f640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f716c0a4410 0x7f716c092380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f717162e640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c09b8e0 0x7f716c13da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7171e2f640 10 monclient(hunting): get_auth_request con 0x7f716c0a4410 auth_method 0 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7171e2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7171e2f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7171e2f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f717162e640 10 monclient(hunting): get_auth_request con 0x7f716c09b8e0 auth_method 0 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f717162e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f717162e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f717162e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 10 monclient(hunting): handle_auth_done global_id 4760 payload 824 2024-08-19T10:40:38.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 1 -- 172.21.15.89:0/3289298443 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f716c0a4410 msgr2=0x7f716c092380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f716c0a4410 0x7f716c092380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 1 -- 172.21.15.89:0/3289298443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c09b8e0 msgr2=0x7f716c13da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f716c09b8e0 0x7f716c13da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 1 monclient: found mon.noname-e 2024-08-19T10:40:38.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:38.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.494+0000 7f7170e2d640 1 -- 172.21.15.89:0/3289298443 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716c0955c0 con 0x7f716c09bcc0 2024-08-19T10:40:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7170e2d640 1 -- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c09bcc0 msgr2=0x7f716c13dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7170e2d640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c09bcc0 0x7f716c13dfa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f715c00d800 tx=0x7f715c00dcc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f715c00bd40 con 0x7f716c09bcc0 2024-08-19T10:40:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 10 monclient: dump: 2024-08-19T10:40:38.513 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.514 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715c019070 con 0x7f716c09bcc0 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7173892640 5 monclient: authenticate success, global_id 4760 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7173892640 10 monclient: _renew_subs 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7173892640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7173892640 1 -- 172.21.15.89:0/3289298443 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716c094660 con 0x7f716c09bcc0 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f715c0148a0 con 0x7f716c09bcc0 2024-08-19T10:40:38.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 10 monclient: dump: 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.516 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7173892640 10 monclient: _renew_subs 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f71637fe640 4 set_mon_vals no callback set 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7173892640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7173892640 1 -- 172.21.15.89:0/3289298443 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f716c0928c0 con 0x7f716c09bcc0 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7173892640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7173892640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c092b50 0x7f716c092f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7173892640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7173892640 1 -- 172.21.15.89:0/3289298443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f716c064590 con 0x7f716c092b50 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f717162e640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c092b50 0x7f716c092f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f717162e640 10 monclient: get_auth_request con 0x7f716c092b50 auth_method 0 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f717162e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f717162e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f717162e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f717162e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f717162e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f717162e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f715c014430 con 0x7f716c09bcc0 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f717162e640 10 monclient(hunting): handle_auth_done global_id 5013 payload 824 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f717162e640 1 -- 172.21.15.89:0/3289298443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f716c0a1b70 con 0x7f716c092b50 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f715003be00 0x7f715003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(70..70 src has 1..70) ==== 5107+0+0 (secure 0 0 0) 0x7f715c00f040 con 0x7f716c09bcc0 2024-08-19T10:40:38.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7171e2f640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f715003be00 0x7f715003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.498+0000 7f7171e2f640 10 monclient: get_auth_request con 0x7f715003be00 auth_method 0 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f717162e640 1 -- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c092b50 msgr2=0x7f716c092f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f717162e640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c092b50 0x7f716c092f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f716800e500 tx=0x7f716800e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7168004e40 con 0x7f716c092b50 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7161ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7161ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7161ffb640 10 monclient: dump: 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.519 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7168018070 con 0x7f716c092b50 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7161ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7161ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c092b50 msgr2=0x7f716c092f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7161ffb640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c092b50 0x7f716c092f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f716800e500 tx=0x7f716800e9c0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c092b50 msgr2=0x7f716c092f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7161ffb640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c092b50 0x7f716c092f30 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f716800e500 tx=0x7f716800e9c0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.502+0000 7f7171e2f640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f715003be00 0x7f715003e2c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f716c074db0 tx=0x7f71640a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f7173892640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f7173892640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c093470 0x7f716c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f7173892640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f7173892640 1 -- 172.21.15.89:0/3289298443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f716c0a7a80 con 0x7f716c093470 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f717162e640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c093470 0x7f716c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f717162e640 10 monclient: get_auth_request con 0x7f716c093470 auth_method 0 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f717162e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f717162e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f717162e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f717162e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f717162e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f717162e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f717162e640 10 monclient(hunting): handle_auth_done global_id 5022 payload 824 2024-08-19T10:40:38.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.506+0000 7f717162e640 1 -- 172.21.15.89:0/3289298443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f716c0a1fc0 con 0x7f716c093470 2024-08-19T10:40:38.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f717162e640 1 -- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c093470 msgr2=0x7f716c070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f717162e640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c093470 0x7f716c070cc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f716800e4d0 tx=0x7f7168004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7168028440 con 0x7f716c093470 2024-08-19T10:40:38.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7161ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7161ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:38.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7161ffb640 10 monclient: dump: 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.524 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.525 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.525 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.525 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9269 (secure 0 0 0) 0x7f716801ee50 con 0x7f716c093470 2024-08-19T10:40:38.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7161ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:38.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7161ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:38.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c093470 msgr2=0x7f716c070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7161ffb640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c093470 0x7f716c070cc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f716800e4d0 tx=0x7f7168004950 comp rx=0 tx=0).stop 2024-08-19T10:40:38.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7161ffb640 1 -- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c093470 msgr2=0x7f716c070cc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:38.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7161ffb640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f716c093470 0x7f716c070cc0 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f716800e4d0 tx=0x7f7168004950 comp rx=0 tx=0).stop 2024-08-19T10:40:38.525 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 168323, 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:38.526 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:38.527 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:38.528 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:38.529 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:38.530 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.531 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.532 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:38.534 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:38.535 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.537 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:38.538 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.539 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:38.541 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:38.542 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.544 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:38.545 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.546 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:38.547 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:38.548 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:38.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7173892640 1 -- 172.21.15.89:0/3289298443 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f715003be00 msgr2=0x7f715003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7173892640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f715003be00 0x7f715003e2c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f716c074db0 tx=0x7f71640a7070 comp rx=0 tx=0).stop 2024-08-19T10:40:38.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7173892640 10 monclient: shutdown 2024-08-19T10:40:38.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7173892640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:38.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7173892640 1 -- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c09bcc0 msgr2=0x7f716c13dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7173892640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c09bcc0 0x7f716c13dfa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f715c00d800 tx=0x7f715c00dcc0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7171e2f640 1 -- 172.21.15.89:0/3289298443 reap_dead start 2024-08-19T10:40:38.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7173892640 1 -- 172.21.15.89:0/3289298443 shutdown_connections 2024-08-19T10:40:38.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7173892640 1 --2- 172.21.15.89:0/3289298443 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f716c09bcc0 0x7f716c13dfa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7173892640 1 -- 172.21.15.89:0/3289298443 >> 172.21.15.89:0/3289298443 conn(0x7f716c0a4df0 msgr2=0x7f716c0a21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7173892640 1 -- 172.21.15.89:0/3289298443 shutdown_connections 2024-08-19T10:40:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.510+0000 7f7173892640 1 -- 172.21.15.89:0/3289298443 wait complete. 2024-08-19T10:40:38.551 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:38.551 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:40:38.551 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:40:38.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient: get_monmap_and_config 2024-08-19T10:40:38.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient: build_initial_monmap 2024-08-19T10:40:38.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient: monmap: 2024-08-19T10:40:38.665 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:38.665 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.665 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:38.665832+0000 2024-08-19T10:40:38.665 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:38.665832+0000 2024-08-19T10:40:38.665 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:38.665 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:38.666 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 1 Processor -- start 2024-08-19T10:40:38.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 1 -- start start 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient: init 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient: _add_conns ranks=[6,1,9,3,0,2,10,4,7,8,5] 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fde500a58c0 0x7fde500a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient(hunting): picked mon.noname-d con 0x7fde500a58c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fde5006e6c0 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fde500720d0 0x7fde50140af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient(hunting): picked mon.noname-a con 0x7fde500720d0 addr v2:172.21.15.89:3300/0 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde5006f0f0 con 0x7fde5006e6c0 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fde5006f3d0 con 0x7fde500720d0 2024-08-19T10:40:38.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fde5006f6b0 con 0x7fde500a58c0 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde58015640 10 monclient(hunting): authenticate will time out at 1088.482448s 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fde500720d0 0x7fde50140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde55db1640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fde500a58c0 0x7fde500a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fde500720d0 0x7fde50140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39152/0 (socket says 172.21.15.89:39152) 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 1 -- 172.21.15.89:0/1810900001 learned_addr learned my addr 172.21.15.89:0/1810900001 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde55db1640 10 monclient(hunting): get_auth_request con 0x7fde500a58c0 auth_method 0 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde55db1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde55db1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde55db1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 10 monclient(hunting): get_auth_request con 0x7fde500720d0 auth_method 0 2024-08-19T10:40:38.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde55db1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde55db1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde55db1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 10 monclient(hunting): handle_auth_done global_id 5031 payload 274 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 1 -- 172.21.15.89:0/1810900001 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fde500a58c0 msgr2=0x7fde500a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 1 --2- 172.21.15.89:0/1810900001 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fde500a58c0 0x7fde500a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 1 -- 172.21.15.89:0/1810900001 >> v1:172.21.15.190:6789/0 conn(0x7fde5006e6c0 legacy=0x7fde500a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:38.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 1 monclient: found mon.noname-a 2024-08-19T10:40:38.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:38.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde565b2640 1 -- 172.21.15.89:0/1810900001 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5006f990 con 0x7fde500720d0 2024-08-19T10:40:38.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.662+0000 7fde55db1640 1 --2- 172.21.15.89:0/1810900001 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fde500a58c0 0x7fde500a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:38.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde565b2640 1 -- 172.21.15.89:0/1810900001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500720d0 msgr2=0x7fde50140af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde565b2640 1 --2- 172.21.15.89:0/1810900001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500720d0 0x7fde50140af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde48009e50 tx=0x7fde4801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c015664c7bb21e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 1 -- 172.21.15.89:0/1810900001 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde480090d0 con 0x7fde500720d0 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 10 monclient: dump: 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.676 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 1 -- 172.21.15.89:0/1810900001 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde480a5c50 con 0x7fde500720d0 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 1 -- 172.21.15.89:0/1810900001 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde480a5e00 con 0x7fde500720d0 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde54daf640 10 monclient: dump: 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.678 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde58015640 5 monclient: authenticate success, global_id 5031 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde58015640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.666+0000 7fde58015640 4 set_mon_vals no callback set 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient: shutdown 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 -- 172.21.15.89:0/1810900001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500720d0 msgr2=0x7fde50140af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 --2- 172.21.15.89:0/1810900001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500720d0 0x7fde50140af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde48009e50 tx=0x7fde4801c940 comp rx=0 tx=0).stop 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 -- 172.21.15.89:0/1810900001 shutdown_connections 2024-08-19T10:40:38.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 --2- 172.21.15.89:0/1810900001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde500720d0 0x7fde50140af0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 --2- 172.21.15.89:0/1810900001 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fde500a58c0 0x7fde500a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 -- 172.21.15.89:0/1810900001 >> 172.21.15.89:0/1810900001 conn(0x7fde500aa4b0 msgr2=0x7fde500a69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 -- 172.21.15.89:0/1810900001 shutdown_connections 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 -- 172.21.15.89:0/1810900001 wait complete. 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient: build_initial_monmap 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient: monmap: 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.680 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 Processor -- start 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 -- start start 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient: init 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:38.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient: _add_conns ranks=[2,6,5,0,4,3,7,1,8] 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde5006e6c0 0x7fde50128c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient(hunting): picked mon.noname-b con 0x7fde5006e6c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fde500720d0 0x7fde501d1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient(hunting): picked mon.noname-d con 0x7fde500720d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fde500a58c0 0x7fde50126840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient(hunting): picked mon.noname-h con 0x7fde500a58c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fde50143be0 con 0x7fde5006e6c0 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fde50143a60 con 0x7fde500a58c0 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fde50143d60 con 0x7fde500720d0 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde58015640 10 monclient(hunting): authenticate will time out at 1088.489839s 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde5006e6c0 0x7fde50128c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde5006e6c0 0x7fde50128c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47596/0 (socket says 172.21.15.89:47596) 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 1 -- 172.21.15.89:0/300897858 learned_addr learned my addr 172.21.15.89:0/300897858 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 10 monclient(hunting): get_auth_request con 0x7fde5006e6c0 auth_method 0 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde555b0640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fde500720d0 0x7fde501d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde555b0640 10 monclient(hunting): get_auth_request con 0x7fde500720d0 auth_method 0 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde555b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde555b0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde555b0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde555b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde555b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde555b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 10 monclient(hunting): handle_auth_done global_id 4844 payload 824 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde565b2640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fde500a58c0 0x7fde50126840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 1 -- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fde500720d0 msgr2=0x7fde501d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fde500720d0 0x7fde501d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 1 -- 172.21.15.89:0/300897858 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fde500a58c0 msgr2=0x7fde50126840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fde500a58c0 0x7fde50126840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 1 monclient: found mon.noname-b 2024-08-19T10:40:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:38.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.670+0000 7fde55db1640 1 -- 172.21.15.89:0/300897858 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5012a450 con 0x7fde5006e6c0 2024-08-19T10:40:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde55db1640 1 -- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde5006e6c0 msgr2=0x7fde50128c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde55db1640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde5006e6c0 0x7fde50128c50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fde4000e980 tx=0x7fde4000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde40017070 con 0x7fde5006e6c0 2024-08-19T10:40:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 10 monclient: dump: 2024-08-19T10:40:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.689 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.690 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4000cc10 con 0x7fde5006e6c0 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde58015640 5 monclient: authenticate success, global_id 4844 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde58015640 10 monclient: _renew_subs 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde58015640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde58015640 1 -- 172.21.15.89:0/300897858 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde500a35c0 con 0x7fde5006e6c0 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde4000cdc0 con 0x7fde5006e6c0 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 10 monclient: dump: 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.691 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.692 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde47fff640 4 set_mon_vals no callback set 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde58015640 10 monclient: _renew_subs 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde58015640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde58015640 1 -- 172.21.15.89:0/300897858 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fde501295e0 con 0x7fde5006e6c0 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde58015640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde58015640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501d1890 0x7fde50126d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde58015640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde58015640 1 -- 172.21.15.89:0/300897858 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fde50069f50 con 0x7fde501d1890 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fde4001a070 con 0x7fde5006e6c0 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde555b0640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501d1890 0x7fde50126d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde555b0640 10 monclient: get_auth_request con 0x7fde501d1890 auth_method 0 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde555b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde555b0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde555b0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde555b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde555b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde555b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fde3403be30 0x7fde3403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(70..70 src has 1..70) ==== 5107+0+0 (secure 0 0 0) 0x7fde4000f040 con 0x7fde5006e6c0 2024-08-19T10:40:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde555b0640 10 monclient(hunting): handle_auth_done global_id 4853 payload 824 2024-08-19T10:40:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde555b0640 1 -- 172.21.15.89:0/300897858 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde500a8710 con 0x7fde501d1890 2024-08-19T10:40:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde565b2640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fde3403be30 0x7fde3403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.674+0000 7fde565b2640 10 monclient: get_auth_request con 0x7fde3403be30 auth_method 0 2024-08-19T10:40:38.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde555b0640 1 -- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501d1890 msgr2=0x7fde50126d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde555b0640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501d1890 0x7fde50126d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fde4c007fd0 tx=0x7fde4c00fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde4c00d770 con 0x7fde501d1890 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde467fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde467fc640 10 monclient: dump: 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.695 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fde4c019070 con 0x7fde501d1890 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde467fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde467fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501d1890 msgr2=0x7fde50126d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde467fc640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501d1890 0x7fde50126d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fde4c007fd0 tx=0x7fde4c00fc00 comp rx=0 tx=0).stop 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501d1890 msgr2=0x7fde50126d80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde467fc640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501d1890 0x7fde50126d80 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fde4c007fd0 tx=0x7fde4c00fc00 comp rx=0 tx=0).stop 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.678+0000 7fde565b2640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fde3403be30 0x7fde3403e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde5006e310 tx=0x7fde480a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde58015640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde58015640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501272c0 0x7fde50122510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde58015640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde58015640 1 -- 172.21.15.89:0/300897858 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fde501276a0 con 0x7fde501272c0 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde555b0640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501272c0 0x7fde50122510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde555b0640 10 monclient: get_auth_request con 0x7fde501272c0 auth_method 0 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde555b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde555b0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde555b0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde555b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde555b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde555b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde555b0640 10 monclient(hunting): handle_auth_done global_id 4862 payload 824 2024-08-19T10:40:38.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.682+0000 7fde555b0640 1 -- 172.21.15.89:0/300897858 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fde500a5ca0 con 0x7fde501272c0 2024-08-19T10:40:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde555b0640 1 -- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501272c0 msgr2=0x7fde50122510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde555b0640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501272c0 0x7fde50122510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fde4c007fd0 tx=0x7fde4c003770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde4c007030 con 0x7fde501272c0 2024-08-19T10:40:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde467fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde467fc640 10 monclient: dump: 2024-08-19T10:40:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.699 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.700 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9601 (secure 0 0 0) 0x7fde4c00db90 con 0x7fde501272c0 2024-08-19T10:40:38.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde467fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:38.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde467fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:38.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501272c0 msgr2=0x7fde50122510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde467fc640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501272c0 0x7fde50122510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fde4c007fd0 tx=0x7fde4c003770 comp rx=0 tx=0).stop 2024-08-19T10:40:38.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde467fc640 1 -- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501272c0 msgr2=0x7fde50122510 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:38.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde467fc640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde501272c0 0x7fde50122510 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fde4c007fd0 tx=0x7fde4c003770 comp rx=0 tx=0).stop 2024-08-19T10:40:38.701 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:38.701 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 168515, 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:38.702 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:38.703 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:38.704 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:38.705 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:38.706 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:38.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.708 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.709 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:38.710 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.712 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.713 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:38.714 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:38.717 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.718 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:38.719 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.721 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.722 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.723 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.724 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.725 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:38.726 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:38.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde58015640 1 -- 172.21.15.89:0/300897858 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fde3403be30 msgr2=0x7fde3403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde58015640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fde3403be30 0x7fde3403e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde5006e310 tx=0x7fde480a7070 comp rx=0 tx=0).stop 2024-08-19T10:40:38.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde58015640 10 monclient: shutdown 2024-08-19T10:40:38.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde58015640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:38.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde58015640 1 -- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde5006e6c0 msgr2=0x7fde50128c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde58015640 1 --2- 172.21.15.89:0/300897858 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde5006e6c0 0x7fde50128c50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fde4000e980 tx=0x7fde4000ee40 comp rx=0 tx=0).stop 2024-08-19T10:40:38.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde565b2640 1 -- 172.21.15.89:0/300897858 reap_dead start 2024-08-19T10:40:38.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde565b2640 1 -- 172.21.15.89:0/300897858 reap_dead start 2024-08-19T10:40:38.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde58015640 1 -- 172.21.15.89:0/300897858 shutdown_connections 2024-08-19T10:40:38.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde58015640 1 -- 172.21.15.89:0/300897858 >> 172.21.15.89:0/300897858 conn(0x7fde500aa4b0 msgr2=0x7fde500a3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:38.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde58015640 1 -- 172.21.15.89:0/300897858 shutdown_connections 2024-08-19T10:40:38.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.686+0000 7fde58015640 1 -- 172.21.15.89:0/300897858 wait complete. 2024-08-19T10:40:38.728 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:38.728 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:40:38.728 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:40:38.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.767+0000 7fd09ffff640 1 -- 172.21.15.190:0/1036165439 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}]=-22 type 'osd' is for devices, not buckets v70) ==== 136+0+0 (secure 0 0 0) 0x7fd0a0016620 con 0x7fd0a80a2450 2024-08-19T10:40:38.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.767+0000 7fd09ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "osd"}] 2024-08-19T10:40:38.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.767+0000 7fd09ffff640 10 monclient: _finish_command 2 = mon:22 type 'osd' is for devices, not buckets 2024-08-19T10:40:38.769 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: type 'osd' is for devices, not buckets 2024-08-19T10:40:38.775 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:40:38.775 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket foo root 2024-08-19T10:40:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient: get_monmap_and_config 2024-08-19T10:40:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient: build_initial_monmap 2024-08-19T10:40:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient: monmap: 2024-08-19T10:40:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:38.845166+0000 2024-08-19T10:40:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:38.845166+0000 2024-08-19T10:40:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:38.846 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:38.846 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:38.846 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:38.846 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:38.846 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:38.846 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:38.846 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:38.846 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:38.846 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 1 Processor -- start 2024-08-19T10:40:38.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 1 -- start start 2024-08-19T10:40:38.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient: init 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient: _add_conns ranks=[3,0,5,1,6,8,7,10,9,4,2] 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50100a6860 0x7f50100a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient(hunting): picked mon.noname-g con 0x7f50100a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f50100a7180 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f50100a7560 0x7f5010139d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient(hunting): picked mon.noname-h con 0x7f50100a7560 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50100a33d0 con 0x7f50100a7180 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f50100a8190 con 0x7f50100a6860 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f50100a79f0 con 0x7f50100a7560 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f501539d640 10 monclient(hunting): authenticate will time out at 1088.661712s 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500e7fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f50100a7180 0x7f5010139620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58312/0 (socket says 172.21.15.89:58312) 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500e7fc640 1 -- 172.21.15.89:0/2040130890 learned_addr learned my addr 172.21.15.89:0/2040130890 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500effd640 1 --2- 172.21.15.89:0/2040130890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50100a6860 0x7f50100a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 1 --2- 172.21.15.89:0/2040130890 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f50100a7560 0x7f5010139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 10 monclient(hunting): get_auth_request con 0x7f50100a7560 auth_method 0 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500effd640 10 monclient(hunting): get_auth_request con 0x7f50100a6860 auth_method 0 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 10 monclient(hunting): handle_auth_done global_id 4901 payload 274 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 1 -- 172.21.15.89:0/2040130890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50100a6860 msgr2=0x7f50100a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 1 --2- 172.21.15.89:0/2040130890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50100a6860 0x7f50100a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 1 -- 172.21.15.89:0/2040130890 >> v1:172.21.15.89:6789/0 conn(0x7f50100a7180 legacy=0x7f5010139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:38.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 1 monclient: found mon.noname-h 2024-08-19T10:40:38.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:38.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500f7fe640 1 -- 172.21.15.89:0/2040130890 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50100a2ec0 con 0x7f50100a7560 2024-08-19T10:40:38.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.842+0000 7f500effd640 1 --2- 172.21.15.89:0/2040130890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50100a6860 0x7f50100a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:38.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500f7fe640 1 -- 172.21.15.89:0/2040130890 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f50100a7560 msgr2=0x7f5010139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500f7fe640 1 --2- 172.21.15.89:0/2040130890 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f50100a7560 0x7f5010139d30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f50000097f0 tx=0x7f500001c3a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=fb672e6af897fff1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 1 -- 172.21.15.89:0/2040130890 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f50000b0070 con 0x7f50100a7560 2024-08-19T10:40:38.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:38.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 10 monclient: dump: 2024-08-19T10:40:38.854 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.854 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.855 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.855 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.855 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.855 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.855 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.855 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.855 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.855 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.855 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.855 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f501539d640 5 monclient: authenticate success, global_id 4901 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f501539d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 1 -- 172.21.15.89:0/2040130890 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50000aa3a0 con 0x7f50100a7560 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f501539d640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:38.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f501539d640 4 set_mon_vals no callback set 2024-08-19T10:40:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 1 -- 172.21.15.89:0/2040130890 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f50000aaa40 con 0x7f50100a7560 2024-08-19T10:40:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.846+0000 7f500dffb640 10 monclient: dump: 2024-08-19T10:40:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.857 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.858 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient: shutdown 2024-08-19T10:40:38.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 -- 172.21.15.89:0/2040130890 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f50100a7560 msgr2=0x7f5010139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 --2- 172.21.15.89:0/2040130890 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f50100a7560 0x7f5010139d30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f50000097f0 tx=0x7f500001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 -- 172.21.15.89:0/2040130890 shutdown_connections 2024-08-19T10:40:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 --2- 172.21.15.89:0/2040130890 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f50100a7560 0x7f5010139d30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 --2- 172.21.15.89:0/2040130890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50100a6860 0x7f50100a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 -- 172.21.15.89:0/2040130890 >> 172.21.15.89:0/2040130890 conn(0x7f5010074b40 msgr2=0x7f50100abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 -- 172.21.15.89:0/2040130890 shutdown_connections 2024-08-19T10:40:38.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 -- 172.21.15.89:0/2040130890 wait complete. 2024-08-19T10:40:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient: build_initial_monmap 2024-08-19T10:40:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient: monmap: 2024-08-19T10:40:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:38.860 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:38.861 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 Processor -- start 2024-08-19T10:40:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 -- start start 2024-08-19T10:40:38.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient: init 2024-08-19T10:40:38.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:38.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient: _add_conns ranks=[1,7,4,0,8,5,6,2,3] 2024-08-19T10:40:38.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50100a6860 0x7f50101d1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient(hunting): picked mon.noname-f con 0x7f50100a6860 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f50100a7180 0x7f50101d1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient(hunting): picked mon.noname-i con 0x7f50100a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50100a7560 0x7f501013dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient(hunting): picked mon.noname-c con 0x7f50100a7560 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f501013a3f0 con 0x7f50100a6860 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f501013a270 con 0x7f50100a7560 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f50101440a0 con 0x7f50100a7180 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:38.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f501539d640 10 monclient(hunting): authenticate will time out at 1088.670611s 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50100a7560 0x7f501013dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50100a7560 0x7f501013dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37402/0 (socket says 172.21.15.89:37402) 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 1 -- 172.21.15.89:0/2462801579 learned_addr learned my addr 172.21.15.89:0/2462801579 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 10 monclient(hunting): get_auth_request con 0x7f50100a7560 auth_method 0 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500effd640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50100a6860 0x7f50101d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500effd640 10 monclient(hunting): get_auth_request con 0x7f50100a6860 auth_method 0 2024-08-19T10:40:38.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500e7fc640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f50100a7180 0x7f50101d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500e7fc640 10 monclient(hunting): get_auth_request con 0x7f50100a7180 auth_method 0 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 10 monclient(hunting): handle_auth_done global_id 4909 payload 824 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 1 -- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f50100a7180 msgr2=0x7f50101d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f50100a7180 0x7f50101d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 1 -- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50100a6860 msgr2=0x7f50101d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50100a6860 0x7f50101d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 1 monclient: found mon.noname-c 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500f7fe640 1 -- 172.21.15.89:0/2462801579 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50100a25b0 con 0x7f50100a7560 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.850+0000 7f500effd640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50100a6860 0x7f50101d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500f7fe640 1 -- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50100a7560 msgr2=0x7f501013dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500f7fe640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50100a7560 0x7f501013dfb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f500001c1d0 tx=0x7f5000019140 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f50000b0040 con 0x7f50100a7560 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 10 monclient: dump: 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.870 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.871 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50000b9070 con 0x7f50100a7560 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f501539d640 5 monclient: authenticate success, global_id 4909 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f501539d640 10 monclient: _renew_subs 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f501539d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f501539d640 1 -- 172.21.15.89:0/2462801579 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5010123560 con 0x7f50100a7560 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5000003d60 con 0x7f50100a7560 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:38.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 10 monclient: dump: 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.873 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f501539d640 10 monclient: _renew_subs 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f501539d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f501539d640 1 -- 172.21.15.89:0/2462801579 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f50100a2060 con 0x7f50100a7560 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500d7fa640 4 set_mon_vals no callback set 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f501539d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f501539d640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013e4f0 0x7f501013e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f501539d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f501539d640 1 -- 172.21.15.89:0/2462801579 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f50100ac000 con 0x7f501013e4f0 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500effd640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013e4f0 0x7f501013e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500effd640 10 monclient: get_auth_request con 0x7f501013e4f0 auth_method 0 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f50000038f0 con 0x7f50100a7560 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ffc03be00 0x7f4ffc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500effd640 10 monclient(hunting): handle_auth_done global_id 4918 payload 824 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500effd640 1 -- 172.21.15.89:0/2462801579 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50100a6c40 con 0x7f501013e4f0 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(70..70 src has 1..70) ==== 5107+0+0 (secure 0 0 0) 0x7f50000a4e00 con 0x7f50100a7560 2024-08-19T10:40:38.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500e7fc640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ffc03be00 0x7f4ffc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.854+0000 7f500e7fc640 10 monclient: get_auth_request con 0x7f4ffc03be00 auth_method 0 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f500effd640 1 -- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013e4f0 msgr2=0x7f501013e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f500effd640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013e4f0 0x7f501013e8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5008007f30 tx=0x7f5008011660 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f500801a020 con 0x7f501013e4f0 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f4ff77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f4ff77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f4ff77fe640 10 monclient: dump: 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.876 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5008016890 con 0x7f501013e4f0 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f4ff77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f4ff77fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:38.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013e4f0 msgr2=0x7f501013e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f4ff77fe640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013e4f0 0x7f501013e8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5008007f30 tx=0x7f5008011660 comp rx=0 tx=0).stop 2024-08-19T10:40:38.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013e4f0 msgr2=0x7f501013e8d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:38.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f4ff77fe640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013e4f0 0x7f501013e8d0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5008007f30 tx=0x7f5008011660 comp rx=0 tx=0).stop 2024-08-19T10:40:38.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.858+0000 7f500e7fc640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ffc03be00 0x7f4ffc03e2c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5004007fd0 tx=0x7f5004009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.862+0000 7f501539d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:38.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.862+0000 7f501539d640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013ee10 0x7f501013a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.862+0000 7f501539d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.862+0000 7f501539d640 1 -- 172.21.15.89:0/2462801579 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f5010064590 con 0x7f501013ee10 2024-08-19T10:40:38.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013ee10 0x7f501013a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 10 monclient: get_auth_request con 0x7f501013ee10 auth_method 0 2024-08-19T10:40:38.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 10 monclient(hunting): handle_auth_done global_id 4927 payload 824 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 1 -- 172.21.15.89:0/2462801579 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f501006cab0 con 0x7f501013ee10 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 1 -- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013ee10 msgr2=0x7f501013a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500effd640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013ee10 0x7f501013a630 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5008007f30 tx=0x7f5008004a50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f500801a050 con 0x7f501013ee10 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f4ff77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f4ff77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f4ff77fe640 10 monclient: dump: 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.881 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:38.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f5008029430 con 0x7f501013ee10 2024-08-19T10:40:38.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f4ff77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:38.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f4ff77fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:38.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013ee10 msgr2=0x7f501013a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f4ff77fe640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013ee10 0x7f501013a630 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5008007f30 tx=0x7f5008004a50 comp rx=0 tx=0).stop 2024-08-19T10:40:38.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f4ff77fe640 1 -- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013ee10 msgr2=0x7f501013a630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:38.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f4ff77fe640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f501013ee10 0x7f501013a630 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5008007f30 tx=0x7f5008004a50 comp rx=0 tx=0).stop 2024-08-19T10:40:38.883 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:38.883 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:38.883 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 168683, 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:38.884 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:38.885 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:38.886 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:38.887 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:38.888 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:38.889 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.891 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.892 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:38.892 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.892 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.892 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:38.892 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.892 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.892 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.893 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:38.894 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.894 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:38.894 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.894 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.894 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:38.895 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:38.895 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.895 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.895 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.895 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:38.895 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:38.895 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.895 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.896 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.896 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:38.896 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.896 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.896 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:38.896 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.896 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.897 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.898 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:38.899 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:38.899 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.899 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.899 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.899 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:38.899 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:38.899 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.899 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.900 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:38.901 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:38.901 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.901 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.901 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.901 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:38.901 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:38.901 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.901 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:38.902 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:38.905 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:38.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:38.907 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:38.908 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:38.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f501539d640 1 -- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ffc03be00 msgr2=0x7f4ffc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f501539d640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ffc03be00 0x7f4ffc03e2c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5004007fd0 tx=0x7f5004009040 comp rx=0 tx=0).stop 2024-08-19T10:40:38.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f501539d640 10 monclient: shutdown 2024-08-19T10:40:38.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f501539d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:38.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f501539d640 1 -- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50100a7560 msgr2=0x7f501013dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f500f7fe640 1 -- 172.21.15.89:0/2462801579 reap_dead start 2024-08-19T10:40:38.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f501539d640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50100a7560 0x7f501013dfb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f500001c1d0 tx=0x7f5000019140 comp rx=0 tx=0).stop 2024-08-19T10:40:38.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f501539d640 1 -- 172.21.15.89:0/2462801579 shutdown_connections 2024-08-19T10:40:38.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f501539d640 1 --2- 172.21.15.89:0/2462801579 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f50100a7560 0x7f501013dfb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f501539d640 1 -- 172.21.15.89:0/2462801579 >> 172.21.15.89:0/2462801579 conn(0x7f5010074b40 msgr2=0x7f5010140f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:38.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f501539d640 1 -- 172.21.15.89:0/2462801579 shutdown_connections 2024-08-19T10:40:38.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:38.866+0000 7f501539d640 1 -- 172.21.15.89:0/2462801579 wait complete. 2024-08-19T10:40:38.911 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:38.911 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:40:38.911 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:40:38.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.891+0000 7f63c2649640 10 monclient: get_monmap_and_config 2024-08-19T10:40:38.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.891+0000 7f63c2649640 10 monclient: build_initial_monmap 2024-08-19T10:40:38.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.891+0000 7f63c2649640 10 monclient: monmap: 2024-08-19T10:40:38.920 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:38.920 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.920 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:38.894480+0000 2024-08-19T10:40:38.920 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:38.894480+0000 2024-08-19T10:40:38.920 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:38.920 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 1 Processor -- start 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 1 -- start start 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 10 monclient: init 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 10 monclient: _add_conns ranks=[3,7,2,4,8,5,1,6,0,9,10] 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0a5840 0x7f63bc0a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 10 monclient(hunting): picked mon.noname-g con 0x7f63bc0a5840 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc0a6160 0x7f63bc1375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 10 monclient(hunting): picked mon.noname-i con 0x7f63bc0a6160 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63bc137b30 0x7f63bc137fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 10 monclient(hunting): picked mon.noname-b con 0x7f63bc137b30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f63bc072260 con 0x7f63bc137b30 2024-08-19T10:40:38.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f63bc072f80 con 0x7f63bc0a5840 2024-08-19T10:40:38.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f63bc074020 con 0x7f63bc0a6160 2024-08-19T10:40:38.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:38.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c2649640 10 monclient(hunting): authenticate will time out at 1084.465054s 2024-08-19T10:40:38.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bb7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc0a6160 0x7f63bc1375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0a5840 0x7f63bc0a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bb7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc0a6160 0x7f63bc1375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:41480/0 (socket says 172.21.15.190:41480) 2024-08-19T10:40:38.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0a5840 0x7f63bc0a5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42478/0 (socket says 172.21.15.190:42478) 2024-08-19T10:40:38.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bb7fe640 1 -- 172.21.15.190:0/2448617920 learned_addr learned my addr 172.21.15.190:0/2448617920 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:38.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bb7fe640 10 monclient(hunting): get_auth_request con 0x7f63bc0a6160 auth_method 0 2024-08-19T10:40:38.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 10 monclient(hunting): get_auth_request con 0x7f63bc0a5840 auth_method 0 2024-08-19T10:40:38.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bb7fe640 1 -- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc0a6160 msgr2=0x7f63bc1375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bb7fe640 1 -- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc0a6160 msgr2=0x7f63bc1375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bb7fe640 1 --2- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc0a6160 0x7f63bc1375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c0be6640 1 --2- 172.21.15.190:0/2448617920 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63bc137b30 0x7f63bc137fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bb7fe640 1 --2- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc0a6160 0x7f63bc1375f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c0be6640 10 monclient(hunting): get_auth_request con 0x7f63bc137b30 auth_method 0 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c0be6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c0be6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c0be6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c0be6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c0be6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63c0be6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 10 monclient(hunting): handle_auth_done global_id 4881 payload 274 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 1 -- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc0a6160 msgr2=0x7f63bc1375f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 1 --2- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc0a6160 0x7f63bc1375f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 1 -- 172.21.15.190:0/2448617920 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63bc137b30 msgr2=0x7f63bc137fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 1 --2- 172.21.15.190:0/2448617920 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63bc137b30 0x7f63bc137fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 1 monclient: found mon.noname-g 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.895+0000 7f63bbfff640 1 -- 172.21.15.190:0/2448617920 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63bc072460 con 0x7f63bc0a5840 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63bbfff640 1 -- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0a5840 msgr2=0x7f63bc0a5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63bbfff640 1 --2- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0a5840 0x7f63bc0a5c20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0097f0 tx=0x7f63ac01c3a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=6c41bf7d70004a11 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 1 -- 172.21.15.190:0/2448617920 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63ac0b0070 con 0x7f63bc0a5840 2024-08-19T10:40:38.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:40:38.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 10 monclient: dump: 2024-08-19T10:40:38.927 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:38.927 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.927 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.927 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.927 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 1 -- 172.21.15.190:0/2448617920 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ac0aa810 con 0x7f63bc0a5840 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 1 -- 172.21.15.190:0/2448617920 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63ac0a9300 con 0x7f63bc0a5840 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 5 monclient: authenticate success, global_id 4881 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 4 set_mon_vals no callback set 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63baffd640 10 monclient: dump: 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:38.929 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.930 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.930 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.930 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.930 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.930 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.930 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.930 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.930 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.930 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.930 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.930 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 10 monclient: shutdown 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 1 -- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0a5840 msgr2=0x7f63bc0a5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 1 --2- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0a5840 0x7f63bc0a5c20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0097f0 tx=0x7f63ac01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 1 -- 172.21.15.190:0/2448617920 shutdown_connections 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 1 --2- 172.21.15.190:0/2448617920 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63bc137b30 0x7f63bc137fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 1 --2- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc0a6160 0x7f63bc1375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 1 --2- 172.21.15.190:0/2448617920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63bc0a5840 0x7f63bc0a5c20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 1 -- 172.21.15.190:0/2448617920 >> 172.21.15.190:0/2448617920 conn(0x7f63bc0aa430 msgr2=0x7f63bc0a6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:38.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 1 -- 172.21.15.190:0/2448617920 shutdown_connections 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 1 -- 172.21.15.190:0/2448617920 wait complete. 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 10 monclient: build_initial_monmap 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.899+0000 7f63c2649640 10 monclient: monmap: 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:38.932 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 1 Processor -- start 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 1 -- start start 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient: init 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient: _add_conns ranks=[1,8,7,5,3,6,0,2,4] 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0a5840 0x7f63bc1cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient(hunting): picked mon.noname-f con 0x7f63bc0a5840 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:38.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63bc0a6160 0x7f63bc1cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient(hunting): picked mon.noname-e con 0x7f63bc0a6160 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc137b30 0x7f63bc1cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient(hunting): picked mon.noname-i con 0x7f63bc137b30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc0a6540 con 0x7f63bc0a5840 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f63bc074440 con 0x7f63bc137b30 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f63bc135450 con 0x7f63bc0a6160 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient(hunting): authenticate will time out at 1084.472714s 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0a5840 0x7f63bc1cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc137b30 0x7f63bc1cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0a5840 0x7f63bc1cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60590/0 (socket says 172.21.15.190:60590) 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 10 monclient(hunting): get_auth_request con 0x7f63bc137b30 auth_method 0 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bbfff640 1 -- 172.21.15.190:0/3904294661 learned_addr learned my addr 172.21.15.190:0/3904294661 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bbfff640 10 monclient(hunting): get_auth_request con 0x7f63bc0a5840 auth_method 0 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bbfff640 1 -- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0a5840 msgr2=0x7f63bc1cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bbfff640 1 -- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0a5840 msgr2=0x7f63bc1cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:40:38.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bb7fe640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63bc0a6160 0x7f63bc1cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bbfff640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0a5840 0x7f63bc1cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bbfff640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0a5840 0x7f63bc1cf210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bb7fe640 10 monclient(hunting): get_auth_request con 0x7f63bc0a6160 auth_method 0 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 10 monclient(hunting): handle_auth_done global_id 14128 payload 824 2024-08-19T10:40:38.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 1 -- 172.21.15.190:0/3904294661 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63bc0a6160 msgr2=0x7f63bc1cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63bc0a6160 0x7f63bc1cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 1 -- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0a5840 msgr2=0x7f63bc1cf210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0a5840 0x7f63bc1cf210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 10 monclient: _finish_hunting 0 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 1 monclient: found mon.noname-i 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 1 -- 172.21.15.190:0/3904294661 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63bc13ce70 con 0x7f63bc137b30 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 1 -- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc137b30 msgr2=0x7f63bc1cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c0be6640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc137b30 0x7f63bc1cfc90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63b400eba0 tx=0x7f63b400c490 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 1 -- 172.21.15.190:0/3904294661 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63b4018020 con 0x7f63bc137b30 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 10 monclient: dump: 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.938 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 10 monclient: _finish_auth 0 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 1 -- 172.21.15.190:0/3904294661 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b400cd30 con 0x7f63bc137b30 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 1 -- 172.21.15.190:0/3904294661 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63b4014d60 con 0x7f63bc137b30 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:38.939 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:40:38.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63b8ff9640 10 monclient: dump: 2024-08-19T10:40:38.940 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:38.940 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:38.940 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.940 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:38.940 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:38.940 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:38.940 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:38.940 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:38.940 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:38.940 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63ba7fc640 4 set_mon_vals no callback set 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 5 monclient: authenticate success, global_id 14128 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient: _renew_subs 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 1 -- 172.21.15.190:0/3904294661 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63bc0a8d40 con 0x7f63bc137b30 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient: _renew_subs 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:40:38.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.903+0000 7f63c2649640 1 -- 172.21.15.190:0/3904294661 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f63bc13ab10 con 0x7f63bc137b30 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.907+0000 7f63c2649640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.907+0000 7f63c2649640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.907+0000 7f63c2649640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.907+0000 7f63c2649640 1 -- 172.21.15.190:0/3904294661 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63bc0a1330 con 0x7f63bc137b30 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.907+0000 7f63b8ff9640 1 -- 172.21.15.190:0/3904294661 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f63b401b020 con 0x7f63bc137b30 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.907+0000 7f63b8ff9640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63a803be30 0x7f63a803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.907+0000 7f63b8ff9640 1 -- 172.21.15.190:0/3904294661 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(70..70 src has 1..70) ==== 5107+0+0 (secure 0 0 0) 0x7f63b404f6c0 con 0x7f63bc137b30 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.907+0000 7f63bbfff640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63a803be30 0x7f63a803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.907+0000 7f63bbfff640 10 monclient: get_auth_request con 0x7f63a803be30 auth_method 0 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.907+0000 7f63bbfff640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63a803be30 0x7f63a803e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0097f0 tx=0x7f63ac0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.911+0000 7f63b8ff9640 1 -- 172.21.15.190:0/3904294661 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f63b4022030 con 0x7f63bc137b30 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.911+0000 7f63b8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:38.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:38.911+0000 7f63b8ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:39.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient: get_monmap_and_config 2024-08-19T10:40:39.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient: monmap: 2024-08-19T10:40:39.041 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.041 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.041 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:39.041503+0000 2024-08-19T10:40:39.041 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:39.041503+0000 2024-08-19T10:40:39.041 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.041 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.041 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:39.042 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:39.042 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.042 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.042 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.042 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.042 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.042 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.042 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.042 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:39.042 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:39.042 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 1 Processor -- start 2024-08-19T10:40:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 1 -- start start 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient: init 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient: _add_conns ranks=[2,3,5,9,6,7,10,4,0,1,8] 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a5270 0x7f41700a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient(hunting): picked mon.noname-b con 0x7f41700a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41700a6d30 0x7f41700a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient(hunting): picked mon.noname-g con 0x7f41700a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41701395b0 0x7f41701399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient(hunting): picked mon.noname-h con 0x7f41701395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f41700a7650 con 0x7f41700a5270 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f41700a8710 con 0x7f41700a6d30 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f41700a5a40 con 0x7f41701395b0 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f417673f640 10 monclient(hunting): authenticate will time out at 1088.857985s 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a5270 0x7f41700a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a5270 0x7f41700a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47628/0 (socket says 172.21.15.89:47628) 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 1 -- 172.21.15.89:0/1427175907 learned_addr learned my addr 172.21.15.89:0/1427175907 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 10 monclient(hunting): get_auth_request con 0x7f41700a5270 auth_method 0 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416f7fe640 1 --2- 172.21.15.89:0/1427175907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41700a6d30 0x7f41700a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416f7fe640 10 monclient(hunting): get_auth_request con 0x7f41700a6d30 auth_method 0 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f4174cdc640 1 --2- 172.21.15.89:0/1427175907 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41701395b0 0x7f41701399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f4174cdc640 10 monclient(hunting): get_auth_request con 0x7f41701395b0 auth_method 0 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f4174cdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f4174cdc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f4174cdc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 10 monclient(hunting): handle_auth_done global_id 4880 payload 274 2024-08-19T10:40:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 1 -- 172.21.15.89:0/1427175907 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41701395b0 msgr2=0x7f41701399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 1 --2- 172.21.15.89:0/1427175907 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41701395b0 0x7f41701399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 1 -- 172.21.15.89:0/1427175907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41700a6d30 msgr2=0x7f41700a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 1 --2- 172.21.15.89:0/1427175907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41700a6d30 0x7f41700a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 1 monclient: found mon.noname-b 2024-08-19T10:40:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:39.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.038+0000 7f416ffff640 1 -- 172.21.15.89:0/1427175907 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41700a81d0 con 0x7f41700a5270 2024-08-19T10:40:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416ffff640 1 -- 172.21.15.89:0/1427175907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a5270 msgr2=0x7f41700a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416ffff640 1 --2- 172.21.15.89:0/1427175907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a5270 0x7f41700a67f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f41600098a0 tx=0x7f416001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91828510d9d48ab7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 1 -- 172.21.15.89:0/1427175907 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f41600b0070 con 0x7f41700a5270 2024-08-19T10:40:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 10 monclient: dump: 2024-08-19T10:40:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.054 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 1 -- 172.21.15.89:0/1427175907 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41600aa810 con 0x7f41700a5270 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 5 monclient: authenticate success, global_id 4880 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 1 -- 172.21.15.89:0/1427175907 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f41600a9300 con 0x7f41700a5270 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 4 set_mon_vals no callback set 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f416effd640 10 monclient: dump: 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.056 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.057 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 10 monclient: shutdown 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 1 -- 172.21.15.89:0/1427175907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a5270 msgr2=0x7f41700a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 1 --2- 172.21.15.89:0/1427175907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a5270 0x7f41700a67f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f41600098a0 tx=0x7f416001c440 comp rx=0 tx=0).stop 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 1 -- 172.21.15.89:0/1427175907 shutdown_connections 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 1 --2- 172.21.15.89:0/1427175907 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41701395b0 0x7f41701399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 1 --2- 172.21.15.89:0/1427175907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41700a6d30 0x7f41700a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 1 --2- 172.21.15.89:0/1427175907 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a5270 0x7f41700a67f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 1 -- 172.21.15.89:0/1427175907 >> 172.21.15.89:0/1427175907 conn(0x7f4170069a80 msgr2=0x7f41700abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 1 -- 172.21.15.89:0/1427175907 shutdown_connections 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.042+0000 7f417673f640 1 -- 172.21.15.89:0/1427175907 wait complete. 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient: monmap: 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.059 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 1 Processor -- start 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 1 -- start start 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient: init 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient: _add_conns ranks=[8,2,6,5,1,3,4,0,7] 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f41700a5270 0x7f417013dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient(hunting): picked mon.noname-e con 0x7f41700a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a6d30 0x7f417013e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient(hunting): picked mon.noname-b con 0x7f41700a6d30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701395b0 0x7f417013e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient(hunting): picked mon.noname-d con 0x7f41701395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f41700a57d0 con 0x7f41700a6d30 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f41700a5650 con 0x7f41701395b0 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4170143230 con 0x7f41700a5270 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f417673f640 10 monclient(hunting): authenticate will time out at 1088.865311s 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a6d30 0x7f417013e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416ffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f41700a5270 0x7f417013dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a6d30 0x7f417013e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47634/0 (socket says 172.21.15.89:47634) 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 1 -- 172.21.15.89:0/1034379653 learned_addr learned my addr 172.21.15.89:0/1034379653 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f4174cdc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701395b0 0x7f417013e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 10 monclient(hunting): get_auth_request con 0x7f41700a6d30 auth_method 0 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416ffff640 10 monclient(hunting): get_auth_request con 0x7f41700a5270 auth_method 0 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f4174cdc640 10 monclient(hunting): get_auth_request con 0x7f41701395b0 auth_method 0 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f4174cdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f4174cdc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f4174cdc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f4174cdc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f4174cdc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f4174cdc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 10 monclient(hunting): handle_auth_done global_id 4889 payload 824 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 1 -- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f41700a5270 msgr2=0x7f417013dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f41700a5270 0x7f417013dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 1 -- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701395b0 msgr2=0x7f417013e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701395b0 0x7f417013e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 1 monclient: found mon.noname-b 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.046+0000 7f416f7fe640 1 -- 172.21.15.89:0/1034379653 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417009e560 con 0x7f41700a6d30 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416f7fe640 1 -- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a6d30 msgr2=0x7f417013e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416f7fe640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a6d30 0x7f417013e130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f416400e960 tx=0x7f416400ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4164017070 con 0x7f41700a6d30 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 10 monclient: dump: 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.064 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416400cc10 con 0x7f41700a6d30 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f417673f640 5 monclient: authenticate success, global_id 4889 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f417673f640 10 monclient: _renew_subs 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f417673f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f417673f640 1 -- 172.21.15.89:0/1034379653 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4170140530 con 0x7f41700a6d30 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f416400cdc0 con 0x7f41700a6d30 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 10 monclient: dump: 2024-08-19T10:40:39.066 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.067 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416e7fc640 4 set_mon_vals no callback set 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f417673f640 10 monclient: _renew_subs 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f417673f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f417673f640 1 -- 172.21.15.89:0/1034379653 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f41700a71a0 con 0x7f41700a6d30 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f417673f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f417673f640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701d1a80 0x7f41701d1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f417673f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f417673f640 1 -- 172.21.15.89:0/1034379653 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f417009d040 con 0x7f41701d1a80 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701d1a80 0x7f41701d1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f416401a070 con 0x7f41700a6d30 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 10 monclient: get_auth_request con 0x7f41701d1a80 auth_method 0 2024-08-19T10:40:39.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f415c03be00 0x7f415c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 10 monclient(hunting): handle_auth_done global_id 4884 payload 824 2024-08-19T10:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(70..70 src has 1..70) ==== 5107+0+0 (secure 0 0 0) 0x7f416400f040 con 0x7f41700a6d30 2024-08-19T10:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 1 -- 172.21.15.89:0/1034379653 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4170140760 con 0x7f41701d1a80 2024-08-19T10:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f4174cdc640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f415c03be00 0x7f415c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f4174cdc640 10 monclient: get_auth_request con 0x7f415c03be00 auth_method 0 2024-08-19T10:40:39.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 1 -- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701d1a80 msgr2=0x7f41701d1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.050+0000 7f416ffff640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701d1a80 0x7f41701d1e60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f41600ad000 tx=0x7f41600191e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f41600b0040 con 0x7f41701d1a80 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f416cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f416cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f416cff9640 10 monclient: dump: 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.072 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4160003830 con 0x7f41701d1a80 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f416cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f416cff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701d1a80 msgr2=0x7f41701d1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f416cff9640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701d1a80 0x7f41701d1e60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f41600ad000 tx=0x7f41600191e0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701d1a80 msgr2=0x7f41701d1e60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f416cff9640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f41701d1a80 0x7f41701d1e60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.054+0000 7f4174cdc640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f415c03be00 0x7f415c03e2c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f41700acc30 tx=0x7f4168009820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f417673f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f417673f640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f417002d1d0 0x7f417002d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f417673f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f417673f640 1 -- 172.21.15.89:0/1034379653 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f41700a67f0 con 0x7f417002d1d0 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f416ffff640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f417002d1d0 0x7f417002d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f416ffff640 10 monclient: get_auth_request con 0x7f417002d1d0 auth_method 0 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f416ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f416ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f416ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f416ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f416ffff640 10 monclient(hunting): handle_auth_done global_id 4893 payload 824 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.058+0000 7f416ffff640 1 -- 172.21.15.89:0/1034379653 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4170064590 con 0x7f417002d1d0 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416ffff640 1 -- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f417002d1d0 msgr2=0x7f417002d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416ffff640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f417002d1d0 0x7f417002d5e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f416001c480 tx=0x7f41600a8000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f41600b0040 con 0x7f417002d1d0 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416cff9640 10 monclient: dump: 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.075 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f41600c4070 con 0x7f417002d1d0 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416cff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:39.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f417002d1d0 msgr2=0x7f417002d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416cff9640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f417002d1d0 0x7f417002d5e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f416001c480 tx=0x7f41600a8000 comp rx=0 tx=0).stop 2024-08-19T10:40:39.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416cff9640 1 -- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f417002d1d0 msgr2=0x7f417002d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f416cff9640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f417002d1d0 0x7f417002d5e0 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f416001c480 tx=0x7f41600a8000 comp rx=0 tx=0).stop 2024-08-19T10:40:39.077 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:39.077 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:39.077 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:39.077 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:39.077 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:39.077 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 168879, 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.078 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.079 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.080 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:39.081 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.082 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.083 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.084 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:39.087 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.088 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.089 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.091 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.092 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.093 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.094 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.095 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:39.096 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.098 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.099 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:39.100 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:39.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:39.087+0000 7f63c2649640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-08-19T10:40:39.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:39.087+0000 7f63c2649640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-08-19T10:40:39.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:39.087+0000 7f63c2649640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:40:39.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:39.087+0000 7f63c2649640 1 -- 172.21.15.190:0/3904294661 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "root"} v 0) -- 0x7f63bc06fd50 con 0x7f63bc137b30 2024-08-19T10:40:39.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f417673f640 1 -- 172.21.15.89:0/1034379653 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f415c03be00 msgr2=0x7f415c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f417673f640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f415c03be00 0x7f415c03e2c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f41700acc30 tx=0x7f4168009820 comp rx=0 tx=0).stop 2024-08-19T10:40:39.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f417673f640 10 monclient: shutdown 2024-08-19T10:40:39.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f417673f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:39.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f417673f640 1 -- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a6d30 msgr2=0x7f417013e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f417673f640 1 --2- 172.21.15.89:0/1034379653 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41700a6d30 0x7f417013e130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f416400e960 tx=0x7f416400ee20 comp rx=0 tx=0).stop 2024-08-19T10:40:39.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f4174cdc640 1 -- 172.21.15.89:0/1034379653 reap_dead start 2024-08-19T10:40:39.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f4174cdc640 1 -- 172.21.15.89:0/1034379653 reap_dead start 2024-08-19T10:40:39.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f417673f640 1 -- 172.21.15.89:0/1034379653 shutdown_connections 2024-08-19T10:40:39.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f417673f640 1 -- 172.21.15.89:0/1034379653 >> 172.21.15.89:0/1034379653 conn(0x7f4170069a80 msgr2=0x7f4170138b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:39.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f417673f640 1 -- 172.21.15.89:0/1034379653 shutdown_connections 2024-08-19T10:40:39.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.062+0000 7f417673f640 1 -- 172.21.15.89:0/1034379653 wait complete. 2024-08-19T10:40:39.104 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:39.104 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:40:39.104 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:40:39.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient: get_monmap_and_config 2024-08-19T10:40:39.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient: monmap: 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:39.221039+0000 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:39.221039+0000 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.221 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:39.222 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 1 Processor -- start 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 1 -- start start 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient: init 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient: _add_conns ranks=[7,10,5,0,4,8,9,3,6,2,1] 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f17dc06c930 0x7f17dc06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient(hunting): picked mon.noname-i con 0x7f17dc06c930 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17dc09d340 0x7f17dc06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient(hunting): picked mon.noname-f con 0x7f17dc09d340 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f17dc137500 0x7f17dc137960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient(hunting): picked mon.noname-h con 0x7f17dc137500 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f17dc0a54f0 con 0x7f17dc09d340 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f17dc0a3e00 con 0x7f17dc137500 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f17dc06d790 con 0x7f17dc06c930 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e2edb640 10 monclient(hunting): authenticate will time out at 1089.037653s 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f17dc137500 0x7f17dc137960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17dbfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17dc09d340 0x7f17dc06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f17dc137500 0x7f17dc137960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:53784/0 (socket says 172.21.15.89:53784) 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 1 -- 172.21.15.89:0/2619586448 learned_addr learned my addr 172.21.15.89:0/2619586448 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 10 monclient(hunting): get_auth_request con 0x7f17dc137500 auth_method 0 2024-08-19T10:40:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e0c77640 1 --2- 172.21.15.89:0/2619586448 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f17dc06c930 0x7f17dc06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17dbfff640 10 monclient(hunting): get_auth_request con 0x7f17dc09d340 auth_method 0 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17dbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e0c77640 10 monclient(hunting): get_auth_request con 0x7f17dc06c930 auth_method 0 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e0c77640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e0c77640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e0c77640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e0c77640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e0c77640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e0c77640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 10 monclient(hunting): handle_auth_done global_id 4919 payload 274 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 1 -- 172.21.15.89:0/2619586448 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f17dc06c930 msgr2=0x7f17dc06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 1 --2- 172.21.15.89:0/2619586448 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f17dc06c930 0x7f17dc06cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 1 -- 172.21.15.89:0/2619586448 >> v2:172.21.15.190:3300/0 conn(0x7f17dc09d340 msgr2=0x7f17dc06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 1 --2- 172.21.15.89:0/2619586448 >> v2:172.21.15.190:3300/0 conn(0x7f17dc09d340 0x7f17dc06d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 1 monclient: found mon.noname-h 2024-08-19T10:40:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.218+0000 7f17e1478640 1 -- 172.21.15.89:0/2619586448 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc09fbf0 con 0x7f17dc137500 2024-08-19T10:40:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17e1478640 1 -- 172.21.15.89:0/2619586448 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f17dc137500 msgr2=0x7f17dc137960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17e1478640 1 --2- 172.21.15.89:0/2619586448 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f17dc137500 0x7f17dc137960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009870 tx=0x7f17cc01c420 comp rx=0 tx=0).ready entity=mon.5 client_cookie=5282be54c131dece server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 1 -- 172.21.15.89:0/2619586448 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17cc0b0070 con 0x7f17dc137500 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 10 monclient: dump: 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.233 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 1 -- 172.21.15.89:0/2619586448 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17cc0aa810 con 0x7f17dc137500 2024-08-19T10:40:39.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17e2edb640 5 monclient: authenticate success, global_id 4919 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17e2edb640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17e2edb640 4 set_mon_vals no callback set 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 1 -- 172.21.15.89:0/2619586448 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17cc0aaa40 con 0x7f17dc137500 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.222+0000 7f17db7fe640 10 monclient: dump: 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.235 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient: shutdown 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 -- 172.21.15.89:0/2619586448 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f17dc137500 msgr2=0x7f17dc137960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 --2- 172.21.15.89:0/2619586448 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f17dc137500 0x7f17dc137960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009870 tx=0x7f17cc01c420 comp rx=0 tx=0).stop 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 -- 172.21.15.89:0/2619586448 shutdown_connections 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 --2- 172.21.15.89:0/2619586448 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f17dc137500 0x7f17dc137960 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 --2- 172.21.15.89:0/2619586448 >> v2:172.21.15.190:3300/0 conn(0x7f17dc09d340 0x7f17dc06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 --2- 172.21.15.89:0/2619586448 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f17dc06c930 0x7f17dc06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 -- 172.21.15.89:0/2619586448 >> 172.21.15.89:0/2619586448 conn(0x7f17dc0a8f60 msgr2=0x7f17dc0a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 -- 172.21.15.89:0/2619586448 shutdown_connections 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 -- 172.21.15.89:0/2619586448 wait complete. 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient: monmap: 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 Processor -- start 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 -- start start 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient: init 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient: _add_conns ranks=[4,3,8,0,6,5,7,1,2] 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17dc06c930 0x7f17dc1d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient(hunting): picked mon.noname-c con 0x7f17dc06c930 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17dc09d340 0x7f17dc1d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient(hunting): picked mon.noname-g con 0x7f17dc09d340 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc137500 0x7f17dc1d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient(hunting): picked mon.noname-e con 0x7f17dc137500 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f17dc074420 con 0x7f17dc09d340 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f17dc1382b0 con 0x7f17dc06c930 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f17dc140f70 con 0x7f17dc137500 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e2edb640 10 monclient(hunting): authenticate will time out at 1089.046116s 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17dc06c930 0x7f17dc1d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17dc06c930 0x7f17dc1d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37424/0 (socket says 172.21.15.89:37424) 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e1478640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc137500 0x7f17dc1d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 1 -- 172.21.15.89:0/998612904 learned_addr learned my addr 172.21.15.89:0/998612904 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 10 monclient(hunting): get_auth_request con 0x7f17dc06c930 auth_method 0 2024-08-19T10:40:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e1478640 10 monclient(hunting): get_auth_request con 0x7f17dc137500 auth_method 0 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e1478640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e1478640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e1478640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e1478640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e1478640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e1478640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17dbfff640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17dc09d340 0x7f17dc1d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17dbfff640 10 monclient(hunting): get_auth_request con 0x7f17dc09d340 auth_method 0 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17dbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 10 monclient(hunting): handle_auth_done global_id 4936 payload 824 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 1 -- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc137500 msgr2=0x7f17dc1d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc137500 0x7f17dc1d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 1 -- 172.21.15.89:0/998612904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17dc09d340 msgr2=0x7f17dc1d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17dc09d340 0x7f17dc1d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 1 monclient: found mon.noname-c 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17e0c77640 1 -- 172.21.15.89:0/998612904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc0a9840 con 0x7f17dc06c930 2024-08-19T10:40:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.226+0000 7f17dbfff640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17dc09d340 0x7f17dc1d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:40:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e0c77640 1 -- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17dc06c930 msgr2=0x7f17dc1d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e0c77640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17dc06c930 0x7f17dc1d11a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f17d400efc0 tx=0x7f17d400c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17d4018020 con 0x7f17dc06c930 2024-08-19T10:40:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 10 monclient: dump: 2024-08-19T10:40:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d40148f0 con 0x7f17dc06c930 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17d40133e0 con 0x7f17dc06c930 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e2edb640 5 monclient: authenticate success, global_id 4936 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e2edb640 10 monclient: _renew_subs 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e2edb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e2edb640 1 -- 172.21.15.89:0/998612904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc121ec0 con 0x7f17dc06c930 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17da7fc640 4 set_mon_vals no callback set 2024-08-19T10:40:39.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 10 monclient: dump: 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.250 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e2edb640 10 monclient: _renew_subs 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e2edb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e2edb640 1 -- 172.21.15.89:0/998612904 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc073aa0 con 0x7f17dc06c930 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e2edb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e2edb640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1209a0 0x7f17dc133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e2edb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e2edb640 1 -- 172.21.15.89:0/998612904 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f17dc137960 con 0x7f17dc1209a0 2024-08-19T10:40:39.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17dbfff640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1209a0 0x7f17dc133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17dbfff640 10 monclient: get_auth_request con 0x7f17dc1209a0 auth_method 0 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17dbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17dbfff640 10 monclient(hunting): handle_auth_done global_id 4796 payload 824 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f17d401b020 con 0x7f17dc06c930 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17dbfff640 1 -- 172.21.15.89:0/998612904 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17dc064590 con 0x7f17dc1209a0 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f17c003be30 0x7f17c003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(70..70 src has 1..70) ==== 5107+0+0 (secure 0 0 0) 0x7f17d404f4e0 con 0x7f17dc06c930 2024-08-19T10:40:39.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e1478640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f17c003be30 0x7f17c003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.230+0000 7f17e1478640 10 monclient: get_auth_request con 0x7f17c003be30 auth_method 0 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17dbfff640 1 -- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1209a0 msgr2=0x7f17dc133fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17dbfff640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1209a0 0x7f17dc133fb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f17dc072fc0 tx=0x7f17c4011a60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17c400fe70 con 0x7f17dc1209a0 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17d97fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17d97fa640 10 monclient: dump: 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.253 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f17c400fa00 con 0x7f17dc1209a0 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17d97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17d97fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1209a0 msgr2=0x7f17dc133fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17d97fa640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1209a0 0x7f17dc133fb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f17dc072fc0 tx=0x7f17c4011a60 comp rx=0 tx=0).stop 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1209a0 msgr2=0x7f17dc133fb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17d97fa640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1209a0 0x7f17dc133fb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.234+0000 7f17e1478640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f17c003be30 0x7f17c003e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0ad000 tx=0x7f17cc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17e2edb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17e2edb640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1344f0 0x7f17dc134900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17e2edb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17e2edb640 1 -- 172.21.15.89:0/998612904 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f17dc134e40 con 0x7f17dc1344f0 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17dbfff640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1344f0 0x7f17dc134900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17dbfff640 10 monclient: get_auth_request con 0x7f17dc1344f0 auth_method 0 2024-08-19T10:40:39.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17dbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17dbfff640 10 monclient(hunting): handle_auth_done global_id 4805 payload 824 2024-08-19T10:40:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.238+0000 7f17dbfff640 1 -- 172.21.15.89:0/998612904 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f17dc121b40 con 0x7f17dc1344f0 2024-08-19T10:40:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17dbfff640 1 -- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1344f0 msgr2=0x7f17dc134900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17dbfff640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1344f0 0x7f17dc134900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f17dc072fc0 tx=0x7f17c4004740 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17c4004490 con 0x7f17dc1344f0 2024-08-19T10:40:39.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17d97fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17d97fa640 10 monclient: dump: 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.257 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f17c4004020 con 0x7f17dc1344f0 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17d97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17d97fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1344f0 msgr2=0x7f17dc134900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17d97fa640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1344f0 0x7f17dc134900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f17dc072fc0 tx=0x7f17c4004740 comp rx=0 tx=0).stop 2024-08-19T10:40:39.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17d97fa640 1 -- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1344f0 msgr2=0x7f17dc134900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17d97fa640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f17dc1344f0 0x7f17dc134900 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f17dc072fc0 tx=0x7f17c4004740 comp rx=0 tx=0).stop 2024-08-19T10:40:39.259 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 169035, 2024-08-19T10:40:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.274 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:39.275 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.276 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.278 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.279 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:39.280 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.282 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.283 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:39.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.284 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.284 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.284 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:39.284 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.284 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.284 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:39.284 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:39.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e2edb640 1 -- 172.21.15.89:0/998612904 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f17c003be30 msgr2=0x7f17c003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e2edb640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f17c003be30 0x7f17c003e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0ad000 tx=0x7f17cc0a6000 comp rx=0 tx=0).stop 2024-08-19T10:40:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e2edb640 10 monclient: shutdown 2024-08-19T10:40:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e2edb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e2edb640 1 -- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17dc06c930 msgr2=0x7f17dc1d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e2edb640 1 --2- 172.21.15.89:0/998612904 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f17dc06c930 0x7f17dc1d11a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f17d400efc0 tx=0x7f17d400c490 comp rx=0 tx=0).stop 2024-08-19T10:40:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e1478640 1 -- 172.21.15.89:0/998612904 reap_dead start 2024-08-19T10:40:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e1478640 1 -- 172.21.15.89:0/998612904 reap_dead start 2024-08-19T10:40:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e2edb640 1 -- 172.21.15.89:0/998612904 shutdown_connections 2024-08-19T10:40:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e2edb640 1 -- 172.21.15.89:0/998612904 >> 172.21.15.89:0/998612904 conn(0x7f17dc0a8f60 msgr2=0x7f17dc09e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e2edb640 1 -- 172.21.15.89:0/998612904 shutdown_connections 2024-08-19T10:40:39.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.242+0000 7f17e2edb640 1 -- 172.21.15.89:0/998612904 wait complete. 2024-08-19T10:40:39.287 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:39.287 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:40:39.287 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:40:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient: get_monmap_and_config 2024-08-19T10:40:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient: monmap: 2024-08-19T10:40:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:39.405124+0000 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:39.405124+0000 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:39.406 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 1 Processor -- start 2024-08-19T10:40:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 1 -- start start 2024-08-19T10:40:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient: init 2024-08-19T10:40:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient: _add_conns ranks=[4,0,8,1,7,6,3,2,5,10,9] 2024-08-19T10:40:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4440a3af0 0x7fc4440a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient(hunting): picked mon.noname-c con 0x7fc4440a3af0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc4440a4410 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc44413f0a0 0x7fc444141490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient(hunting): picked mon.noname-e con 0x7fc44413f0a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4440a0ed0 con 0x7fc4440a4410 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc4440a02a0 con 0x7fc4440a3af0 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc4440a31e0 con 0x7fc44413f0a0 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc44b794640 10 monclient(hunting): authenticate will time out at 1089.221774s 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4440a3af0 0x7fc4440a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4440a3af0 0x7fc4440a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37426/0 (socket says 172.21.15.89:37426) 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 1 -- 172.21.15.89:0/621437737 learned_addr learned my addr 172.21.15.89:0/621437737 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 10 monclient(hunting): get_auth_request con 0x7fc4440a3af0 auth_method 0 2024-08-19T10:40:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449d31640 1 --2- 172.21.15.89:0/621437737 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc44413f0a0 0x7fc444141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449d31640 10 monclient(hunting): get_auth_request con 0x7fc44413f0a0 auth_method 0 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449d31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449d31640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449d31640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449d31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449d31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449d31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 10 monclient(hunting): handle_auth_done global_id 4945 payload 274 2024-08-19T10:40:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 1 -- 172.21.15.89:0/621437737 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc44413f0a0 msgr2=0x7fc444141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 1 --2- 172.21.15.89:0/621437737 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc44413f0a0 0x7fc444141490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 1 -- 172.21.15.89:0/621437737 >> v1:172.21.15.89:6789/0 conn(0x7fc4440a4410 legacy=0x7fc44413d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 1 monclient: found mon.noname-c 2024-08-19T10:40:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.402+0000 7fc449530640 1 -- 172.21.15.89:0/621437737 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4440a11b0 con 0x7fc4440a3af0 2024-08-19T10:40:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc449530640 1 -- 172.21.15.89:0/621437737 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4440a3af0 msgr2=0x7fc4440a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc449530640 1 --2- 172.21.15.89:0/621437737 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4440a3af0 0x7fc4440a3ed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009e80 tx=0x7fc43c01c910 comp rx=0 tx=0).ready entity=mon.4 client_cookie=3ad13835903e5f98 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 1 -- 172.21.15.89:0/621437737 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc43c0090d0 con 0x7fc4440a3af0 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 10 monclient: dump: 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.416 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 1 -- 172.21.15.89:0/621437737 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc43c0a5a50 con 0x7fc4440a3af0 2024-08-19T10:40:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 5 monclient: authenticate success, global_id 4945 2024-08-19T10:40:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 4 set_mon_vals no callback set 2024-08-19T10:40:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 1 -- 172.21.15.89:0/621437737 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc43c0a5c00 con 0x7fc4440a3af0 2024-08-19T10:40:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc443fff640 10 monclient: dump: 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 10 monclient: shutdown 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 1 -- 172.21.15.89:0/621437737 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4440a3af0 msgr2=0x7fc4440a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 1 --2- 172.21.15.89:0/621437737 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4440a3af0 0x7fc4440a3ed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009e80 tx=0x7fc43c01c910 comp rx=0 tx=0).stop 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 1 -- 172.21.15.89:0/621437737 shutdown_connections 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 1 --2- 172.21.15.89:0/621437737 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc44413f0a0 0x7fc444141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 1 --2- 172.21.15.89:0/621437737 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4440a3af0 0x7fc4440a3ed0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 1 -- 172.21.15.89:0/621437737 >> 172.21.15.89:0/621437737 conn(0x7fc44406de40 msgr2=0x7fc44406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 1 -- 172.21.15.89:0/621437737 shutdown_connections 2024-08-19T10:40:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.406+0000 7fc44b794640 1 -- 172.21.15.89:0/621437737 wait complete. 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient: monmap: 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 1 Processor -- start 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 1 -- start start 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient: init 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient: _add_conns ranks=[7,1,4,2,8,3,0,5,6] 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc4440a3af0 0x7fc44413a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient(hunting): picked mon.noname-i con 0x7fc4440a3af0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4440a4410 0x7fc44413ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient(hunting): picked mon.noname-f con 0x7fc4440a4410 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc44413f0a0 0x7fc44413cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient(hunting): picked mon.noname-c con 0x7fc44413f0a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc44406cbf0 con 0x7fc4440a4410 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc444074400 con 0x7fc44413f0a0 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc4440a1eb0 con 0x7fc4440a3af0 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc44b794640 10 monclient(hunting): authenticate will time out at 1089.229122s 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc44413f0a0 0x7fc44413cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc44413f0a0 0x7fc44413cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37428/0 (socket says 172.21.15.89:37428) 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 1 -- 172.21.15.89:0/4201348590 learned_addr learned my addr 172.21.15.89:0/4201348590 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 10 monclient(hunting): get_auth_request con 0x7fc44413f0a0 auth_method 0 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449530640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc4440a3af0 0x7fc44413a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 10 monclient(hunting): handle_auth_done global_id 4954 payload 824 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc448d2f640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4440a4410 0x7fc44413ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 1 -- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc4440a3af0 msgr2=0x7fc44413a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc4440a3af0 0x7fc44413a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 1 -- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4440a4410 msgr2=0x7fc44413ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4440a4410 0x7fc44413ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 1 monclient: found mon.noname-c 2024-08-19T10:40:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 1 -- 172.21.15.89:0/4201348590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4441248f0 con 0x7fc44413f0a0 2024-08-19T10:40:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 1 -- 172.21.15.89:0/4201348590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc44413f0a0 msgr2=0x7fc44413cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc449d31640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc44413f0a0 0x7fc44413cc10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc43400e820 tx=0x7fc43400ece0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.410+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc434017070 con 0x7fc44413f0a0 2024-08-19T10:40:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 10 monclient: dump: 2024-08-19T10:40:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.429 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc434013470 con 0x7fc44413f0a0 2024-08-19T10:40:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc434013a90 con 0x7fc44413f0a0 2024-08-19T10:40:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc44b794640 5 monclient: authenticate success, global_id 4954 2024-08-19T10:40:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc44b794640 10 monclient: _renew_subs 2024-08-19T10:40:39.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc44b794640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc44b794640 1 -- 172.21.15.89:0/4201348590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc444124bd0 con 0x7fc44413f0a0 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc4437fe640 4 set_mon_vals no callback set 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 10 monclient: dump: 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.430 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc44b794640 10 monclient: _renew_subs 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc44b794640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc44b794640 1 -- 172.21.15.89:0/4201348590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4440a23e0 con 0x7fc44413f0a0 2024-08-19T10:40:39.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc44b794640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc44b794640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413d150 0x7fc44413d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc44b794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc44b794640 1 -- 172.21.15.89:0/4201348590 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc44406cd70 con 0x7fc44413d150 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc43401a070 con 0x7fc44413f0a0 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc449530640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413d150 0x7fc44413d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc449530640 10 monclient: get_auth_request con 0x7fc44413d150 auth_method 0 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc449530640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc449530640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc449530640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc43003be30 0x7fc43003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc449530640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc449530640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(70..70 src has 1..70) ==== 5107+0+0 (secure 0 0 0) 0x7fc43400f040 con 0x7fc44413f0a0 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc449530640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc448d2f640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc43003be30 0x7fc43003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc449530640 10 monclient(hunting): handle_auth_done global_id 14122 payload 824 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc449530640 1 -- 172.21.15.89:0/4201348590 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc444140f30 con 0x7fc44413d150 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.414+0000 7fc448d2f640 10 monclient: get_auth_request con 0x7fc43003be30 auth_method 0 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc449530640 1 -- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413d150 msgr2=0x7fc44413d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc449530640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413d150 0x7fc44413d530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc4440a13c0 tx=0x7fc43c0196b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc43c0bb070 con 0x7fc44413d150 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc441ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc441ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc441ffb640 10 monclient: dump: 2024-08-19T10:40:39.433 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.434 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc448d2f640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc43003be30 0x7fc43003e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc438007fd0 tx=0x7fc438009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc43c004250 con 0x7fc44413d150 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc441ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc441ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413d150 msgr2=0x7fc44413d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc441ffb640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413d150 0x7fc44413d530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc4440a13c0 tx=0x7fc43c0196b0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413d150 msgr2=0x7fc44413d530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.418+0000 7fc441ffb640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413d150 0x7fc44413d530 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc4440a13c0 tx=0x7fc43c0196b0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.422+0000 7fc44b794640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.422+0000 7fc44b794640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413da70 0x7fc444070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.422+0000 7fc44b794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.422+0000 7fc44b794640 1 -- 172.21.15.89:0/4201348590 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc4440a3ed0 con 0x7fc44413da70 2024-08-19T10:40:39.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.422+0000 7fc449530640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413da70 0x7fc444070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.422+0000 7fc449530640 10 monclient: get_auth_request con 0x7fc44413da70 auth_method 0 2024-08-19T10:40:39.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.422+0000 7fc449530640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.422+0000 7fc449530640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.422+0000 7fc449530640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.426+0000 7fc449530640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.426+0000 7fc449530640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.426+0000 7fc449530640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.426+0000 7fc449530640 10 monclient(hunting): handle_auth_done global_id 14131 payload 824 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.426+0000 7fc449530640 1 -- 172.21.15.89:0/4201348590 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc444064590 con 0x7fc44413da70 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.426+0000 7fc449530640 1 -- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413da70 msgr2=0x7fc444070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.426+0000 7fc449530640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413da70 0x7fc444070dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc43c01c740 tx=0x7fc43c0a7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.426+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc43c0a5d60 con 0x7fc44413da70 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.426+0000 7fc441ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.426+0000 7fc441ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.426+0000 7fc441ffb640 10 monclient: dump: 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.440 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7fc43c0c0450 con 0x7fc44413da70 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc441ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:39.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc441ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:39.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413da70 msgr2=0x7fc444070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc441ffb640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413da70 0x7fc444070dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc43c01c740 tx=0x7fc43c0a7000 comp rx=0 tx=0).stop 2024-08-19T10:40:39.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc441ffb640 1 -- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413da70 msgr2=0x7fc444070dc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc441ffb640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc44413da70 0x7fc444070dc0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc43c01c740 tx=0x7fc43c0a7000 comp rx=0 tx=0).stop 2024-08-19T10:40:39.442 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 17019, 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.463 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.464 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:39.465 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.466 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.466 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:39.466 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc44b794640 1 -- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc43003be30 msgr2=0x7fc43003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc44b794640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc43003be30 0x7fc43003e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc438007fd0 tx=0x7fc438009040 comp rx=0 tx=0).stop 2024-08-19T10:40:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc44b794640 10 monclient: shutdown 2024-08-19T10:40:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc44b794640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc44b794640 1 -- 172.21.15.89:0/4201348590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc44413f0a0 msgr2=0x7fc44413cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc44b794640 1 --2- 172.21.15.89:0/4201348590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc44413f0a0 0x7fc44413cc10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc43400e820 tx=0x7fc43400ece0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc449d31640 1 -- 172.21.15.89:0/4201348590 reap_dead start 2024-08-19T10:40:39.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc449d31640 1 -- 172.21.15.89:0/4201348590 reap_dead start 2024-08-19T10:40:39.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc44b794640 1 -- 172.21.15.89:0/4201348590 shutdown_connections 2024-08-19T10:40:39.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc44b794640 1 -- 172.21.15.89:0/4201348590 >> 172.21.15.89:0/4201348590 conn(0x7fc44406de40 msgr2=0x7fc44413f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:39.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc44b794640 1 -- 172.21.15.89:0/4201348590 shutdown_connections 2024-08-19T10:40:39.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.430+0000 7fc44b794640 1 -- 172.21.15.89:0/4201348590 wait complete. 2024-08-19T10:40:39.468 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:39.468 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:40:39.469 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:40:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient: get_monmap_and_config 2024-08-19T10:40:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient: monmap: 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:39.585035+0000 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:39.585035+0000 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.586 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.586 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.586 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.586 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:39.586 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:39.586 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 1 Processor -- start 2024-08-19T10:40:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 1 -- start start 2024-08-19T10:40:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient: init 2024-08-19T10:40:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient: _add_conns ranks=[6,2,8,5,3,10,7,1,0,4,9] 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc43809eae0 0x7fc43809eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient(hunting): picked mon.noname-d con 0x7fc43809eae0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc438069070 0x7fc43809dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient(hunting): picked mon.noname-b con 0x7fc438069070 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4380a3d70 0x7fc4380a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient(hunting): picked mon.noname-e con 0x7fc4380a3d70 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc43809e200 con 0x7fc438069070 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fc43809c840 con 0x7fc43809eae0 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc43809d8a0 con 0x7fc4380a3d70 2024-08-19T10:40:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43f0e2640 10 monclient(hunting): authenticate will time out at 1089.401808s 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4380a3d70 0x7fc4380a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43ce7e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc43809eae0 0x7fc43809eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4380a3d70 0x7fc4380a41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:37100/0 (socket says 172.21.15.89:37100) 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 1 -- 172.21.15.89:0/3799944860 learned_addr learned my addr 172.21.15.89:0/3799944860 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 10 monclient(hunting): get_auth_request con 0x7fc4380a3d70 auth_method 0 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43ce7e640 10 monclient(hunting): get_auth_request con 0x7fc43809eae0 auth_method 0 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43ce7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43ce7e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43ce7e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc437fff640 1 --2- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc438069070 0x7fc43809dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc437fff640 10 monclient(hunting): get_auth_request con 0x7fc438069070 auth_method 0 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc437fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc437fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc437fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43ce7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43ce7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43ce7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc437fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc437fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc437fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 10 monclient(hunting): handle_auth_done global_id 4823 payload 274 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 1 -- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc43809eae0 msgr2=0x7fc43809eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 1 --2- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc43809eae0 0x7fc43809eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 1 -- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc438069070 msgr2=0x7fc43809dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 1 --2- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc438069070 0x7fc43809dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 1 monclient: found mon.noname-e 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43d67f640 1 -- 172.21.15.89:0/3799944860 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4380694f0 con 0x7fc4380a3d70 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc43ce7e640 1 --2- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc43809eae0 0x7fc43809eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.582+0000 7fc437fff640 1 --2- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc438069070 0x7fc43809dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:39.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43d67f640 1 -- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4380a3d70 msgr2=0x7fc4380a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43d67f640 1 --2- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4380a3d70 0x7fc4380a41d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc4280098d0 tx=0x7fc42801c410 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9540200a74f1072c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 1 -- 172.21.15.89:0/3799944860 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4280b0070 con 0x7fc4380a3d70 2024-08-19T10:40:39.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:39.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 10 monclient: dump: 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.599 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 5 monclient: authenticate success, global_id 4823 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 1 -- 172.21.15.89:0/3799944860 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4280044a0 con 0x7fc4380a3d70 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 4 set_mon_vals no callback set 2024-08-19T10:40:39.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 1 -- 172.21.15.89:0/3799944860 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc428003a90 con 0x7fc4380a3d70 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc4377fe640 10 monclient: dump: 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.601 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 10 monclient: shutdown 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4380a3d70 msgr2=0x7fc4380a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 1 --2- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4380a3d70 0x7fc4380a41d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc4280098d0 tx=0x7fc42801c410 comp rx=0 tx=0).stop 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3799944860 shutdown_connections 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 1 --2- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4380a3d70 0x7fc4380a41d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 1 --2- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc438069070 0x7fc43809dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 1 --2- 172.21.15.89:0/3799944860 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc43809eae0 0x7fc43809eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3799944860 >> 172.21.15.89:0/3799944860 conn(0x7fc4380a4e60 msgr2=0x7fc4380a5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3799944860 shutdown_connections 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.586+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3799944860 wait complete. 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient: monmap: 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.603 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 1 Processor -- start 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 1 -- start start 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient: init 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient: _add_conns ranks=[7,1,0,5,3,6,4,2,8] 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc43809eae0 0x7fc438092840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient(hunting): picked mon.noname-i con 0x7fc43809eae0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4380a3d70 0x7fc438092d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient(hunting): picked mon.noname-f con 0x7fc4380a3d70 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4381d12e0 0x7fc4381d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient(hunting): picked mon.noname-a con 0x7fc4381d12e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc438142ec0 con 0x7fc4381d12e0 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc438142d40 con 0x7fc4380a3d70 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc438143040 con 0x7fc43809eae0 2024-08-19T10:40:39.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43f0e2640 10 monclient(hunting): authenticate will time out at 1089.409078s 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4381d12e0 0x7fc4381d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4381d12e0 0x7fc4381d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39156/0 (socket says 172.21.15.89:39156) 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 1 -- 172.21.15.89:0/3618909213 learned_addr learned my addr 172.21.15.89:0/3618909213 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 10 monclient(hunting): get_auth_request con 0x7fc4381d12e0 auth_method 0 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43ce7e640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc43809eae0 0x7fc438092840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc437fff640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4380a3d70 0x7fc438092d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 10 monclient(hunting): handle_auth_done global_id 5040 payload 824 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 1 -- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc43809eae0 msgr2=0x7fc438092840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc43809eae0 0x7fc438092840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 1 -- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4380a3d70 msgr2=0x7fc438092d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4380a3d70 0x7fc438092d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 1 monclient: found mon.noname-a 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 1 -- 172.21.15.89:0/3618909213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc438074840 con 0x7fc4381d12e0 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43ce7e640 10 monclient: get_auth_request con 0x7fc43809eae0 auth_method 0 2024-08-19T10:40:39.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc437fff640 10 monclient: get_auth_request con 0x7fc4380a3d70 auth_method 0 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43ce7e640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc43809eae0 0x7fc438092840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc437fff640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4380a3d70 0x7fc438092d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 1 -- 172.21.15.89:0/3618909213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4381d12e0 msgr2=0x7fc4381d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.590+0000 7fc43d67f640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4381d12e0 0x7fc4381d16e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc42801c240 tx=0x7fc4280191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4280b9070 con 0x7fc4381d12e0 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 10 monclient: dump: 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.608 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc428003d60 con 0x7fc4381d12e0 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4280b0040 con 0x7fc4381d12e0 2024-08-19T10:40:39.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 10 monclient: dump: 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43f0e2640 5 monclient: authenticate success, global_id 5040 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43f0e2640 10 monclient: _renew_subs 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43f0e2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc436ffd640 4 set_mon_vals no callback set 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3618909213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4380a5410 con 0x7fc4381d12e0 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43f0e2640 10 monclient: _renew_subs 2024-08-19T10:40:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43f0e2640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3618909213 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4380951d0 con 0x7fc4381d12e0 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43f0e2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43f0e2640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438069070 0x7fc4381d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43f0e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3618909213 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc4381d2160 con 0x7fc438069070 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc4280038f0 con 0x7fc4381d12e0 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43ce7e640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438069070 0x7fc4381d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43ce7e640 10 monclient: get_auth_request con 0x7fc438069070 auth_method 0 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43ce7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43ce7e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43ce7e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc41803be30 0x7fc41803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43ce7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43ce7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43ce7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5107+0+0 (secure 0 0 0) 0x7fc4280e8190 con 0x7fc4381d12e0 2024-08-19T10:40:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43ce7e640 10 monclient(hunting): handle_auth_done global_id 4908 payload 824 2024-08-19T10:40:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc437fff640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc41803be30 0x7fc41803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc43ce7e640 1 -- 172.21.15.89:0/3618909213 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc43809eec0 con 0x7fc438069070 2024-08-19T10:40:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.594+0000 7fc437fff640 10 monclient: get_auth_request con 0x7fc41803be30 auth_method 0 2024-08-19T10:40:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc43ce7e640 1 -- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438069070 msgr2=0x7fc4381d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc43ce7e640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438069070 0x7fc4381d1c20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc430007f30 tx=0x7fc43000f560 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc43001c070 con 0x7fc438069070 2024-08-19T10:40:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc4357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc4357fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc4357fa640 10 monclient: dump: 2024-08-19T10:40:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.616 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc437fff640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc41803be30 0x7fc41803e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc4380745f0 tx=0x7fc420008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc43000fe40 con 0x7fc438069070 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc4357fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc4357fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438069070 msgr2=0x7fc4381d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc4357fa640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438069070 0x7fc4381d1c20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc430007f30 tx=0x7fc43000f560 comp rx=0 tx=0).stop 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438069070 msgr2=0x7fc4381d1c20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.598+0000 7fc4357fa640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438069070 0x7fc4381d1c20 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc430007f30 tx=0x7fc43000f560 comp rx=0 tx=0).stop 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.602+0000 7fc43f0e2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.602+0000 7fc43f0e2640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438070cb0 0x7fc4380710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.602+0000 7fc43f0e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.602+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3618909213 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc438064590 con 0x7fc438070cb0 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.602+0000 7fc43ce7e640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438070cb0 0x7fc4380710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.602+0000 7fc43ce7e640 10 monclient: get_auth_request con 0x7fc438070cb0 auth_method 0 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.602+0000 7fc43ce7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.602+0000 7fc43ce7e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.602+0000 7fc43ce7e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc43ce7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc43ce7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc43ce7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc43ce7e640 10 monclient(hunting): handle_auth_done global_id 4917 payload 824 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc43ce7e640 1 -- 172.21.15.89:0/3618909213 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc4380932c0 con 0x7fc438070cb0 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc43ce7e640 1 -- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438070cb0 msgr2=0x7fc4380710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc43ce7e640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438070cb0 0x7fc4380710c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc430007f30 tx=0x7fc430004a50 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc43001c040 con 0x7fc438070cb0 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc4357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc4357fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc4357fa640 10 monclient: dump: 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7fc43001e620 con 0x7fc438070cb0 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc4357fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc4357fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438070cb0 msgr2=0x7fc4380710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc4357fa640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438070cb0 0x7fc4380710c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc430007f30 tx=0x7fc430004a50 comp rx=0 tx=0).stop 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc4357fa640 1 -- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438070cb0 msgr2=0x7fc4380710c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.606+0000 7fc4357fa640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc438070cb0 0x7fc4380710c0 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc430007f30 tx=0x7fc430004a50 comp rx=0 tx=0).stop 2024-08-19T10:40:39.621 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:39.621 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:39.621 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:39.621 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:39.621 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:39.621 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 169381, 2024-08-19T10:40:39.621 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:39.621 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:39.621 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:39.622 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.623 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.624 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:39.625 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.626 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:39.628 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.629 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:39.630 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:39.633 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:39.634 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.635 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:39.644 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc41803be30 msgr2=0x7fc41803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43f0e2640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc41803be30 0x7fc41803e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc4380745f0 tx=0x7fc420008040 comp rx=0 tx=0).stop 2024-08-19T10:40:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43f0e2640 10 monclient: shutdown 2024-08-19T10:40:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43f0e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3618909213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4381d12e0 msgr2=0x7fc4381d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43f0e2640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4381d12e0 0x7fc4381d16e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc42801c240 tx=0x7fc4280191b0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43d67f640 1 -- 172.21.15.89:0/3618909213 reap_dead start 2024-08-19T10:40:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3618909213 shutdown_connections 2024-08-19T10:40:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43f0e2640 1 --2- 172.21.15.89:0/3618909213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4381d12e0 0x7fc4381d16e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3618909213 >> 172.21.15.89:0/3618909213 conn(0x7fc4380a4e60 msgr2=0x7fc4380a0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:39.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3618909213 shutdown_connections 2024-08-19T10:40:39.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.610+0000 7fc43f0e2640 1 -- 172.21.15.89:0/3618909213 wait complete. 2024-08-19T10:40:39.647 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:39.647 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:40:39.647 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:40:39.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.762+0000 7fb1b20c9640 10 monclient: get_monmap_and_config 2024-08-19T10:40:39.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.762+0000 7fb1b20c9640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.762+0000 7fb1b20c9640 10 monclient: monmap: 2024-08-19T10:40:39.768 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.768 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.768 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:39.768647+0000 2024-08-19T10:40:39.768 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:39.768647+0000 2024-08-19T10:40:39.768 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.768 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.768 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:39.769 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:39.769 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.769 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.769 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.769 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.769 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.769 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.769 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.769 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:39.769 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:39.769 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 1 Processor -- start 2024-08-19T10:40:39.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 1 -- start start 2024-08-19T10:40:39.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 10 monclient: init 2024-08-19T10:40:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 10 monclient: _add_conns ranks=[6,10,4,0,3,8,5,1,7,9,2] 2024-08-19T10:40:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a5270 0x7fb1ac0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 10 monclient(hunting): picked mon.noname-d con 0x7fb1ac0a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1ac0a6d30 0x7fb1ac0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 10 monclient(hunting): picked mon.noname-f con 0x7fb1ac0a6d30 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 0x7fb1ac139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 10 monclient(hunting): picked mon.noname-c con 0x7fb1ac1395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb1ac0a7650 con 0x7fb1ac0a6d30 2024-08-19T10:40:39.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb1ac0a8710 con 0x7fb1ac1395b0 2024-08-19T10:40:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb1ac0a5a40 con 0x7fb1ac0a5270 2024-08-19T10:40:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1b20c9640 10 monclient(hunting): authenticate will time out at 1089.585219s 2024-08-19T10:40:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1abfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 0x7fb1ac139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a5270 0x7fb1ac0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1abfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 0x7fb1ac139a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37436/0 (socket says 172.21.15.89:37436) 2024-08-19T10:40:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a5270 0x7fb1ac0a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:57344/0 (socket says 172.21.15.89:57344) 2024-08-19T10:40:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1abfff640 1 -- 172.21.15.89:0/2364742864 learned_addr learned my addr 172.21.15.89:0/2364742864 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1abfff640 10 monclient(hunting): get_auth_request con 0x7fb1ac1395b0 auth_method 0 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1abfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1abfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 10 monclient(hunting): get_auth_request con 0x7fb1ac0a5270 auth_method 0 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1aaffd640 1 --2- 172.21.15.89:0/2364742864 >> v2:172.21.15.190:3300/0 conn(0x7fb1ac0a6d30 0x7fb1ac0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1aaffd640 10 monclient(hunting): get_auth_request con 0x7fb1ac0a6d30 auth_method 0 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1aaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_done global_id 4911 payload 274 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 1 -- 172.21.15.89:0/2364742864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 msgr2=0x7fb1ac139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 1 --2- 172.21.15.89:0/2364742864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 0x7fb1ac139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 1 -- 172.21.15.89:0/2364742864 >> v2:172.21.15.190:3300/0 conn(0x7fb1ac0a6d30 msgr2=0x7fb1ac0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 1 --2- 172.21.15.89:0/2364742864 >> v2:172.21.15.190:3300/0 conn(0x7fb1ac0a6d30 0x7fb1ac0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 1 monclient: found mon.noname-d 2024-08-19T10:40:39.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:39.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.766+0000 7fb1ab7fe640 1 -- 172.21.15.89:0/2364742864 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac0a81d0 con 0x7fb1ac0a5270 2024-08-19T10:40:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1ab7fe640 1 -- 172.21.15.89:0/2364742864 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a5270 msgr2=0x7fb1ac0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1ab7fe640 1 --2- 172.21.15.89:0/2364742864 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a5270 0x7fb1ac0a67f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00097c0 tx=0x7fb1a001c360 comp rx=0 tx=0).ready entity=mon.6 client_cookie=bb8ae4a3c928d0f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 1 -- 172.21.15.89:0/2364742864 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1a00b0070 con 0x7fb1ac0a5270 2024-08-19T10:40:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 10 monclient: dump: 2024-08-19T10:40:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.782 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.783 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 5 monclient: authenticate success, global_id 4911 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 1 -- 172.21.15.89:0/2364742864 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a00aa810 con 0x7fb1ac0a5270 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 4 set_mon_vals no callback set 2024-08-19T10:40:39.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 1 -- 172.21.15.89:0/2364742864 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1a00a9300 con 0x7fb1ac0a5270 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1aa7fc640 10 monclient: dump: 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.785 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 10 monclient: shutdown 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 1 -- 172.21.15.89:0/2364742864 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a5270 msgr2=0x7fb1ac0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 1 --2- 172.21.15.89:0/2364742864 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a5270 0x7fb1ac0a67f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00097c0 tx=0x7fb1a001c360 comp rx=0 tx=0).stop 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 1 -- 172.21.15.89:0/2364742864 shutdown_connections 2024-08-19T10:40:39.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 1 --2- 172.21.15.89:0/2364742864 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 0x7fb1ac139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 1 --2- 172.21.15.89:0/2364742864 >> v2:172.21.15.190:3300/0 conn(0x7fb1ac0a6d30 0x7fb1ac0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 1 --2- 172.21.15.89:0/2364742864 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a5270 0x7fb1ac0a67f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 1 -- 172.21.15.89:0/2364742864 >> 172.21.15.89:0/2364742864 conn(0x7fb1ac069a80 msgr2=0x7fb1ac0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:39.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 1 -- 172.21.15.89:0/2364742864 shutdown_connections 2024-08-19T10:40:39.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 1 -- 172.21.15.89:0/2364742864 wait complete. 2024-08-19T10:40:39.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 10 monclient: monmap: 2024-08-19T10:40:39.787 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.787 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:39.787 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:39.787 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 1 Processor -- start 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 1 -- start start 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.770+0000 7fb1b20c9640 10 monclient: init 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient: _add_conns ranks=[3,6,4,5,7,2,1,8,0] 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac0a5270 0x7fb1ac133930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient(hunting): picked mon.noname-g con 0x7fb1ac0a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a6d30 0x7fb1ac133e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient(hunting): picked mon.noname-d con 0x7fb1ac0a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 0x7fb1ac09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient(hunting): picked mon.noname-c con 0x7fb1ac1395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb1ac0a5650 con 0x7fb1ac0a5270 2024-08-19T10:40:39.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb1ac0a4a70 con 0x7fb1ac1395b0 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb1ac0a57d0 con 0x7fb1ac0a6d30 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient(hunting): authenticate will time out at 1089.592281s 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 0x7fb1ac09c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1aaffd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a6d30 0x7fb1ac133e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 0x7fb1ac09c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37446/0 (socket says 172.21.15.89:37446) 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 1 -- 172.21.15.89:0/846773585 learned_addr learned my addr 172.21.15.89:0/846773585 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 10 monclient(hunting): get_auth_request con 0x7fb1ac1395b0 auth_method 0 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1aaffd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a6d30 0x7fb1ac133e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:57348/0 (socket says 172.21.15.89:57348) 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1aaffd640 10 monclient(hunting): get_auth_request con 0x7fb1ac0a6d30 auth_method 0 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1aaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1aaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1aaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1aaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1ab7fe640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac0a5270 0x7fb1ac133930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1ab7fe640 10 monclient(hunting): get_auth_request con 0x7fb1ac0a5270 auth_method 0 2024-08-19T10:40:39.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1ab7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 10 monclient(hunting): handle_auth_done global_id 4972 payload 824 2024-08-19T10:40:39.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 1 -- 172.21.15.89:0/846773585 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a6d30 msgr2=0x7fb1ac133e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a6d30 0x7fb1ac133e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 1 -- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac0a5270 msgr2=0x7fb1ac133930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1ac0a5270 0x7fb1ac133930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 1 monclient: found mon.noname-c 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 1 -- 172.21.15.89:0/846773585 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ac09f440 con 0x7fb1ac1395b0 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1aaffd640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1ac0a6d30 0x7fb1ac133e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 1 -- 172.21.15.89:0/846773585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 msgr2=0x7fb1ac09c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1abfff640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 0x7fb1ac09c9e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400efc0 tx=0x7fb1a400c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1a4003e10 con 0x7fb1ac1395b0 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 10 monclient: dump: 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.793 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.794 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a400cd30 con 0x7fb1ac1395b0 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1a40044a0 con 0x7fb1ac1395b0 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb193fff640 10 monclient: dump: 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.795 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.796 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1a9ffb640 4 set_mon_vals no callback set 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 5 monclient: authenticate success, global_id 4972 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient: _renew_subs 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 1 -- 172.21.15.89:0/846773585 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ac139b20 con 0x7fb1ac1395b0 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient: _renew_subs 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.774+0000 7fb1b20c9640 1 -- 172.21.15.89:0/846773585 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ac09f160 con 0x7fb1ac1395b0 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1b20c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1b20c9640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac09cf20 0x7fb1ac09d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1b20c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1b20c9640 1 -- 172.21.15.89:0/846773585 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb1ac0a6910 con 0x7fb1ac09cf20 2024-08-19T10:40:39.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb1a4012050 con 0x7fb1ac1395b0 2024-08-19T10:40:39.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb193fff640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb19803be30 0x7fb19803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac09cf20 0x7fb1ac09d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 10 monclient: get_auth_request con 0x7fb1ac09cf20 auth_method 0 2024-08-19T10:40:39.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(70..70 src has 1..70) ==== 5107+0+0 (secure 0 0 0) 0x7fb1a404f010 con 0x7fb1ac1395b0 2024-08-19T10:40:39.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1aaffd640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb19803be30 0x7fb19803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1aaffd640 10 monclient: get_auth_request con 0x7fb19803be30 auth_method 0 2024-08-19T10:40:39.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_done global_id 4928 payload 824 2024-08-19T10:40:39.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 1 -- 172.21.15.89:0/846773585 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1ac09d840 con 0x7fb1ac09cf20 2024-08-19T10:40:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 1 -- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac09cf20 msgr2=0x7fb1ac09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1ab7fe640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac09cf20 0x7fb1ac09d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00ad000 tx=0x7fb1a0019100 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb1aaffd640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb19803be30 0x7fb19803e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac13f3f0 tx=0x7fb19c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1a00b0040 con 0x7fb1ac09cf20 2024-08-19T10:40:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb193fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb193fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.778+0000 7fb193fff640 10 monclient: dump: 2024-08-19T10:40:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.803 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.782+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb1a0003a40 con 0x7fb1ac09cf20 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.782+0000 7fb193fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.782+0000 7fb193fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.782+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac09cf20 msgr2=0x7fb1ac09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.782+0000 7fb193fff640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac09cf20 0x7fb1ac09d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00ad000 tx=0x7fb1a0019100 comp rx=0 tx=0).stop 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.782+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac09cf20 msgr2=0x7fb1ac09d300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.782+0000 7fb193fff640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac09cf20 0x7fb1ac09d300 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00ad000 tx=0x7fb1a0019100 comp rx=0 tx=0).stop 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1b20c9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1b20c9640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac137bb0 0x7fb1ac137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1b20c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1b20c9640 1 -- 172.21.15.89:0/846773585 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb1ac0a7110 con 0x7fb1ac137bb0 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1ab7fe640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac137bb0 0x7fb1ac137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1ab7fe640 10 monclient: get_auth_request con 0x7fb1ac137bb0 auth_method 0 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1ab7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1ab7fe640 10 monclient(hunting): handle_auth_done global_id 4937 payload 824 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.786+0000 7fb1ab7fe640 1 -- 172.21.15.89:0/846773585 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb1ac064590 con 0x7fb1ac137bb0 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb1ab7fe640 1 -- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac137bb0 msgr2=0x7fb1ac137fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb1ab7fe640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac137bb0 0x7fb1ac137fc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00097c0 tx=0x7fb1a00a8000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1a00b0040 con 0x7fb1ac137bb0 2024-08-19T10:40:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb193fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb193fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb193fff640 10 monclient: dump: 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7fb1a00bf280 con 0x7fb1ac137bb0 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb193fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb193fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac137bb0 msgr2=0x7fb1ac137fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb193fff640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac137bb0 0x7fb1ac137fc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00097c0 tx=0x7fb1a00a8000 comp rx=0 tx=0).stop 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb193fff640 1 -- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac137bb0 msgr2=0x7fb1ac137fc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.790+0000 7fb193fff640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb1ac137bb0 0x7fb1ac137fc0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00097c0 tx=0x7fb1a00a8000 comp rx=0 tx=0).stop 2024-08-19T10:40:39.808 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:39.808 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:39.808 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:39.808 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:39.808 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:39.808 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 169561, 2024-08-19T10:40:39.808 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:39.808 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:39.808 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:39.808 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:39.809 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:39.809 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:39.809 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:39.809 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:39.809 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:39.809 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:39.809 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.809 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:40:39.810 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.810 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:39.810 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:39.810 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.810 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.810 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.810 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.810 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.810 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.810 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.810 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.811 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.811 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.811 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.811 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.811 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:39.811 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:39.811 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.811 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.811 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.811 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:39.812 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:39.813 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.814 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.815 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.816 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.817 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.818 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:39.820 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.821 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:39.821 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.821 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.821 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.822 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:39.822 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:39.822 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.822 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.822 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.822 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.823 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:39.825 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:39.826 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.827 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.828 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:39.828 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:39.828 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.828 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.828 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:39.829 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:39.832 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.833 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.833 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:39.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.833 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:39.833 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:39.833 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:40:39.833 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:39.833 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.833 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:39.833 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1b20c9640 1 -- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb19803be30 msgr2=0x7fb19803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1b20c9640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb19803be30 0x7fb19803e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac13f3f0 tx=0x7fb19c008040 comp rx=0 tx=0).stop 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1b20c9640 10 monclient: shutdown 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1b20c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1b20c9640 1 -- 172.21.15.89:0/846773585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 msgr2=0x7fb1ac09c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1b20c9640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 0x7fb1ac09c9e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400efc0 tx=0x7fb1a400c490 comp rx=0 tx=0).stop 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1abfff640 1 -- 172.21.15.89:0/846773585 reap_dead start 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1b20c9640 1 -- 172.21.15.89:0/846773585 shutdown_connections 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1b20c9640 1 --2- 172.21.15.89:0/846773585 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1ac1395b0 0x7fb1ac09c9e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1b20c9640 1 -- 172.21.15.89:0/846773585 >> 172.21.15.89:0/846773585 conn(0x7fb1ac069a80 msgr2=0x7fb1ac0ab1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1b20c9640 1 -- 172.21.15.89:0/846773585 shutdown_connections 2024-08-19T10:40:39.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.794+0000 7fb1b20c9640 1 -- 172.21.15.89:0/846773585 wait complete. 2024-08-19T10:40:39.836 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:39.836 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:40:39.836 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:40:39.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient: get_monmap_and_config 2024-08-19T10:40:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient: monmap: 2024-08-19T10:40:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:39.954292+0000 2024-08-19T10:40:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:39.954292+0000 2024-08-19T10:40:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:39.954 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:39.955 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:39.955 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 1 Processor -- start 2024-08-19T10:40:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 1 -- start start 2024-08-19T10:40:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient: init 2024-08-19T10:40:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient: _add_conns ranks=[7,5,9,1,4,8,6,3,10,0,2] 2024-08-19T10:40:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b940a6860 0x7f6b940a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient(hunting): picked mon.noname-i con 0x7f6b940a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b940a7180 0x7f6b940a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient(hunting): picked mon.noname-h con 0x7f6b940a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:39.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6b94139620 0x7f6b94139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient(hunting): picked mon.noname-a con 0x7f6b94139620 addr v2:172.21.15.89:3300/0 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f6b940a8190 con 0x7f6b94139620 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6b940a33d0 con 0x7f6b940a7180 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6b940a2ec0 con 0x7f6b940a6860 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9c731640 10 monclient(hunting): authenticate will time out at 1089.771695s 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9acce640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6b94139620 0x7f6b94139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9acce640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6b94139620 0x7f6b94139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39164/0 (socket says 172.21.15.89:39164) 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9acce640 1 -- 172.21.15.89:0/3869438708 learned_addr learned my addr 172.21.15.89:0/3869438708 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9acce640 10 monclient(hunting): get_auth_request con 0x7f6b94139620 auth_method 0 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9acce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9acce640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.950+0000 7f6b9acce640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b99ccc640 1 --2- 172.21.15.89:0/3869438708 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b940a7180 0x7f6b940a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b99ccc640 10 monclient(hunting): get_auth_request con 0x7f6b940a7180 auth_method 0 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b99ccc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b99ccc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b99ccc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 10 monclient(hunting): handle_auth_done global_id 5049 payload 274 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9a4cd640 1 --2- 172.21.15.89:0/3869438708 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b940a6860 0x7f6b940a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 1 -- 172.21.15.89:0/3869438708 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b940a6860 msgr2=0x7f6b940a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 1 --2- 172.21.15.89:0/3869438708 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b940a6860 0x7f6b940a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 1 -- 172.21.15.89:0/3869438708 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b940a7180 msgr2=0x7f6b940a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 1 --2- 172.21.15.89:0/3869438708 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b940a7180 0x7f6b940a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 1 monclient: found mon.noname-a 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9acce640 1 -- 172.21.15.89:0/3869438708 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b940a4cd0 con 0x7f6b94139620 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9a4cd640 10 monclient: get_auth_request con 0x7f6b940a6860 auth_method 0 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b9a4cd640 1 --2- 172.21.15.89:0/3869438708 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b940a6860 0x7f6b940a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:39.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.954+0000 7f6b99ccc640 1 --2- 172.21.15.89:0/3869438708 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b940a7180 0x7f6b940a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:40:39.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9acce640 1 -- 172.21.15.89:0/3869438708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b94139620 msgr2=0x7f6b94139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9acce640 1 --2- 172.21.15.89:0/3869438708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b94139620 0x7f6b94139a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009e50 tx=0x7f6b8c01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa4afa7597fd6ad server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 1 -- 172.21.15.89:0/3869438708 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b8c0090d0 con 0x7f6b94139620 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 10 monclient: dump: 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.966 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 5 monclient: authenticate success, global_id 5049 2024-08-19T10:40:39.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 1 -- 172.21.15.89:0/3869438708 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8c0a59a0 con 0x7f6b94139620 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 4 set_mon_vals no callback set 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 1 -- 172.21.15.89:0/3869438708 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b8c0ad3f0 con 0x7f6b94139620 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b994cb640 10 monclient: dump: 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.968 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient: shutdown 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:39.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 -- 172.21.15.89:0/3869438708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b94139620 msgr2=0x7f6b94139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 --2- 172.21.15.89:0/3869438708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b94139620 0x7f6b94139a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009e50 tx=0x7f6b8c01c8f0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 -- 172.21.15.89:0/3869438708 shutdown_connections 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 --2- 172.21.15.89:0/3869438708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b94139620 0x7f6b94139a80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 --2- 172.21.15.89:0/3869438708 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b940a7180 0x7f6b940a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 --2- 172.21.15.89:0/3869438708 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b940a6860 0x7f6b940a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 -- 172.21.15.89:0/3869438708 >> 172.21.15.89:0/3869438708 conn(0x7f6b94074b40 msgr2=0x7f6b940abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 -- 172.21.15.89:0/3869438708 shutdown_connections 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 -- 172.21.15.89:0/3869438708 wait complete. 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient: build_initial_monmap 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient: monmap: 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:39.970 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 Processor -- start 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 -- start start 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient: init 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient: _add_conns ranks=[2,6,7,1,8,5,0,4,3] 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b940a6860 0x7f6b941d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient(hunting): picked mon.noname-b con 0x7f6b940a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b940a7180 0x7f6b941d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient(hunting): picked mon.noname-d con 0x7f6b940a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b94139620 0x7f6b9409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient(hunting): picked mon.noname-i con 0x7f6b94139620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6b94143210 con 0x7f6b940a6860 2024-08-19T10:40:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6b94143090 con 0x7f6b940a7180 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6b94143390 con 0x7f6b94139620 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9c731640 10 monclient(hunting): authenticate will time out at 1089.779318s 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9a4cd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b940a6860 0x7f6b941d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b940a7180 0x7f6b941d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9a4cd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b940a6860 0x7f6b941d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:47642/0 (socket says 172.21.15.89:47642) 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9a4cd640 1 -- 172.21.15.89:0/3018196215 learned_addr learned my addr 172.21.15.89:0/3018196215 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 10 monclient(hunting): get_auth_request con 0x7f6b940a7180 auth_method 0 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9a4cd640 10 monclient(hunting): get_auth_request con 0x7f6b940a6860 auth_method 0 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9a4cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9a4cd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9a4cd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9a4cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9a4cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9a4cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9acce640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b94139620 0x7f6b9409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9acce640 10 monclient(hunting): get_auth_request con 0x7f6b94139620 auth_method 0 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9acce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9acce640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9acce640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 10 monclient(hunting): handle_auth_done global_id 4929 payload 824 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 1 -- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b94139620 msgr2=0x7f6b9409c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b94139620 0x7f6b9409c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 1 -- 172.21.15.89:0/3018196215 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b940a6860 msgr2=0x7f6b941d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b940a6860 0x7f6b941d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 10 monclient: _finish_hunting 0 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 1 monclient: found mon.noname-d 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:39.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:39.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b99ccc640 1 -- 172.21.15.89:0/3018196215 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b940a0d70 con 0x7f6b940a7180 2024-08-19T10:40:39.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.958+0000 7f6b9a4cd640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b940a6860 0x7f6b941d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:39.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:39.971+0000 7f63b8ff9640 1 -- 172.21.15.190:0/3904294661 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}]=0 added bucket foo type root to crush map v71) ==== 138+0+0 (secure 0 0 0) 0x7f63b4010040 con 0x7f63bc137b30 2024-08-19T10:40:39.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:39.971+0000 7f63b8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-08-19T10:40:39.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:39.971+0000 7f63b8ff9640 10 monclient: _finish_command 2 = system:0 added bucket foo type root to crush map 2024-08-19T10:40:39.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.962+0000 7f6b99ccc640 1 -- 172.21.15.89:0/3018196215 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b940a7180 msgr2=0x7f6b941d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.962+0000 7f6b99ccc640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b940a7180 0x7f6b941d1a80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400e810 tx=0x7f6b8400ecd0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b84017070 con 0x7f6b940a7180 2024-08-19T10:40:39.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 10 monclient: dump: 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.980 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 10 monclient: _finish_auth 0 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8400cb40 con 0x7f6b940a7180 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b8400f040 con 0x7f6b940a7180 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9c731640 5 monclient: authenticate success, global_id 4929 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9c731640 10 monclient: _renew_subs 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9c731640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:39.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9c731640 1 -- 172.21.15.89:0/3018196215 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b940a2830 con 0x7f6b940a7180 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b98cca640 4 set_mon_vals no callback set 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 10 monclient: dump: 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.982 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9c731640 10 monclient: _renew_subs 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9c731640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9c731640 1 -- 172.21.15.89:0/3018196215 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9409fd80 con 0x7f6b940a7180 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9c731640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9c731640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409e020 0x7f6b9409cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9c731640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9c731640 1 -- 172.21.15.89:0/3018196215 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6b940a0210 con 0x7f6b9409e020 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6b8401a070 con 0x7f6b940a7180 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b7803be00 0x7f6b7803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(71..71 src has 1..71) ==== 5134+0+0 (secure 0 0 0) 0x7f6b8404e550 con 0x7f6b940a7180 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9a4cd640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409e020 0x7f6b9409cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9a4cd640 10 monclient: get_auth_request con 0x7f6b9409e020 auth_method 0 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9a4cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9a4cd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9a4cd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9acce640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b7803be00 0x7f6b7803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9acce640 10 monclient: get_auth_request con 0x7f6b7803be00 auth_method 0 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9a4cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9a4cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.966+0000 7f6b9a4cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b9a4cd640 10 monclient(hunting): handle_auth_done global_id 14155 payload 824 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b9a4cd640 1 -- 172.21.15.89:0/3018196215 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b940a7560 con 0x7f6b9409e020 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b9a4cd640 1 -- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409e020 msgr2=0x7f6b9409cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b9a4cd640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409e020 0x7f6b9409cf20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000fd00 tx=0x7f6b9000d590 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b9acce640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b7803be00 0x7f6b7803e2c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0ab000 tx=0x7f6b8c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b90013ac0 con 0x7f6b9409e020 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b8affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b8affd640 10 monclient: dump: 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.985 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.986 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6b90013650 con 0x7f6b9409e020 2024-08-19T10:40:39.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b8affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b8affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409e020 msgr2=0x7f6b9409cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b8affd640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409e020 0x7f6b9409cf20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000fd00 tx=0x7f6b9000d590 comp rx=0 tx=0).stop 2024-08-19T10:40:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409e020 msgr2=0x7f6b9409cf20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.970+0000 7f6b8affd640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409e020 0x7f6b9409cf20 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000fd00 tx=0x7f6b9000d590 comp rx=0 tx=0).stop 2024-08-19T10:40:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9c731640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9c731640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409d460 0x7f6b9409d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9c731640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9c731640 1 -- 172.21.15.89:0/3018196215 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6b9409ddb0 con 0x7f6b9409d460 2024-08-19T10:40:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9a4cd640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409d460 0x7f6b9409d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:39.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9a4cd640 10 monclient: get_auth_request con 0x7f6b9409d460 auth_method 0 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9a4cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9a4cd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9a4cd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9a4cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9a4cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.978+0000 7f6b9a4cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9a4cd640 10 monclient(hunting): handle_auth_done global_id 14164 payload 824 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9a4cd640 1 -- 172.21.15.89:0/3018196215 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6b94064590 con 0x7f6b9409d460 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9a4cd640 1 -- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409d460 msgr2=0x7f6b9409d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9a4cd640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409d460 0x7f6b9409d870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6b94069d80 tx=0x7f6b900046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b90006040 con 0x7f6b9409d460 2024-08-19T10:40:39.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:39.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b8affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:40:39.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b8affd640 10 monclient: dump: 2024-08-19T10:40:39.991 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:39.991 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:39.991 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.991 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:39.991 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:39.991 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:39.991 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:39.991 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7f6b90013ad0 con 0x7f6b9409d460 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b8affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b8affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409d460 msgr2=0x7f6b9409d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b8affd640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409d460 0x7f6b9409d870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6b94069d80 tx=0x7f6b900046d0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b8affd640 1 -- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409d460 msgr2=0x7f6b9409d870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:39.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b8affd640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b9409d460 0x7f6b9409d870 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6b94069d80 tx=0x7f6b900046d0 comp rx=0 tx=0).stop 2024-08-19T10:40:39.993 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 17571, 2024-08-19T10:40:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:40.005 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.006 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.007 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:40.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:40.009 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:40.010 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:40.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:40.012 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:40.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:40.015 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:40.016 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:40.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9c731640 1 -- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b7803be00 msgr2=0x7f6b7803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9c731640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b7803be00 0x7f6b7803e2c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0ab000 tx=0x7f6b8c0a7070 comp rx=0 tx=0).stop 2024-08-19T10:40:40.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9c731640 10 monclient: shutdown 2024-08-19T10:40:40.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9c731640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:40.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9c731640 1 -- 172.21.15.89:0/3018196215 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b940a7180 msgr2=0x7f6b941d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9c731640 1 --2- 172.21.15.89:0/3018196215 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b940a7180 0x7f6b941d1a80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400e810 tx=0x7f6b8400ecd0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9acce640 1 -- 172.21.15.89:0/3018196215 reap_dead start 2024-08-19T10:40:40.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.982+0000 7f6b9acce640 1 -- 172.21.15.89:0/3018196215 reap_dead start 2024-08-19T10:40:40.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.986+0000 7f6b9c731640 1 -- 172.21.15.89:0/3018196215 shutdown_connections 2024-08-19T10:40:40.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.986+0000 7f6b9c731640 1 -- 172.21.15.89:0/3018196215 >> 172.21.15.89:0/3018196215 conn(0x7f6b94074b40 msgr2=0x7f6b940ab7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:40.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.986+0000 7f6b9c731640 1 -- 172.21.15.89:0/3018196215 shutdown_connections 2024-08-19T10:40:40.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:39.986+0000 7f6b9c731640 1 -- 172.21.15.89:0/3018196215 wait complete. 2024-08-19T10:40:40.019 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:40.019 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-08-19T10:40:40.019 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-08-19T10:40:40.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.079+0000 7f63c2649640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-08-19T10:40:40.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.079+0000 7f63c2649640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-08-19T10:40:40.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.079+0000 7f63c2649640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:40:40.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.079+0000 7f63c2649640 1 -- 172.21.15.190:0/3904294661 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "root"} v 0) -- 0x7f63bc13b930 con 0x7f63bc137b30 2024-08-19T10:40:40.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.079+0000 7f63b8ff9640 1 -- 172.21.15.190:0/3904294661 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}]=0 bucket 'foo' already exists v71) ==== 126+0+0 (secure 0 0 0) 0x7f63b4007d40 con 0x7f63bc137b30 2024-08-19T10:40:40.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.079+0000 7f63b8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "root"}] 2024-08-19T10:40:40.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.079+0000 7f63b8ff9640 10 monclient: _finish_command 3 = system:0 bucket 'foo' already exists 2024-08-19T10:40:40.081 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'foo' already exists 2024-08-19T10:40:40.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 -- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63a803be30 msgr2=0x7f63a803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63a803be30 0x7f63a803e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0097f0 tx=0x7f63ac0a6000 comp rx=0 tx=0).stop 2024-08-19T10:40:40.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 10 monclient: shutdown 2024-08-19T10:40:40.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:40.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 -- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc137b30 msgr2=0x7f63bc1cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc137b30 0x7f63bc1cfc90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63b400eba0 tx=0x7f63b400c490 comp rx=0 tx=0).stop 2024-08-19T10:40:40.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 -- 172.21.15.190:0/3904294661 shutdown_connections 2024-08-19T10:40:40.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63bc137b30 0x7f63bc1cfc90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63a803be30 0x7f63a803e2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63bc0a6160 0x7f63bc1cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 --2- 172.21.15.190:0/3904294661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63bc0a5840 0x7f63bc1cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 -- 172.21.15.190:0/3904294661 >> 172.21.15.190:0/3904294661 conn(0x7f63bc0aa430 msgr2=0x7f63bc0a3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:40.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 -- 172.21.15.190:0/3904294661 shutdown_connections 2024-08-19T10:40:40.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.083+0000 7f63c2649640 1 -- 172.21.15.190:0/3904294661 wait complete. 2024-08-19T10:40:40.100 INFO:tasks.workunit.client.0.smithi190.stderr:++ ceph osd create 2024-08-19T10:40:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.130+0000 7f6aa630f640 10 monclient: get_monmap_and_config 2024-08-19T10:40:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.130+0000 7f6aa630f640 10 monclient: build_initial_monmap 2024-08-19T10:40:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.130+0000 7f6aa630f640 10 monclient: monmap: 2024-08-19T10:40:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:40.135632+0000 2024-08-19T10:40:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:40.135632+0000 2024-08-19T10:40:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:40.135 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:40.136 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:40.136 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.130+0000 7f6aa630f640 1 Processor -- start 2024-08-19T10:40:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.130+0000 7f6aa630f640 1 -- start start 2024-08-19T10:40:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.130+0000 7f6aa630f640 10 monclient: init 2024-08-19T10:40:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 10 monclient: _add_conns ranks=[3,10,6,9,2,4,5,8,7,1,0] 2024-08-19T10:40:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6aa00a5270 0x7f6aa00a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 10 monclient(hunting): picked mon.noname-g con 0x7f6aa00a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6aa00a6d30 0x7f6aa00a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 10 monclient(hunting): picked mon.noname-f con 0x7f6aa00a6d30 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:40.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6aa01395b0 0x7f6aa0139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 10 monclient(hunting): picked mon.noname-d con 0x7f6aa01395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa00a7650 con 0x7f6aa00a6d30 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa00a8710 con 0x7f6aa00a5270 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6aa00a5a40 con 0x7f6aa01395b0 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa630f640 10 monclient(hunting): authenticate will time out at 1089.952672s 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6aa01395b0 0x7f6aa0139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6aa01395b0 0x7f6aa0139a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:57364/0 (socket says 172.21.15.89:57364) 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 1 -- 172.21.15.89:0/476221013 learned_addr learned my addr 172.21.15.89:0/476221013 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:40.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 10 monclient(hunting): get_auth_request con 0x7f6aa01395b0 auth_method 0 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9ffff640 1 --2- 172.21.15.89:0/476221013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6aa00a5270 0x7f6aa00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9ffff640 10 monclient(hunting): get_auth_request con 0x7f6aa00a5270 auth_method 0 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9f7fe640 1 --2- 172.21.15.89:0/476221013 >> v2:172.21.15.190:3300/0 conn(0x7f6aa00a6d30 0x7f6aa00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9f7fe640 10 monclient(hunting): get_auth_request con 0x7f6aa00a6d30 auth_method 0 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6a9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 10 monclient(hunting): handle_auth_done global_id 4938 payload 274 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 1 -- 172.21.15.89:0/476221013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6aa00a5270 msgr2=0x7f6aa00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 1 --2- 172.21.15.89:0/476221013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6aa00a5270 0x7f6aa00a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 1 -- 172.21.15.89:0/476221013 >> v2:172.21.15.190:3300/0 conn(0x7f6aa00a6d30 msgr2=0x7f6aa00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 1 --2- 172.21.15.89:0/476221013 >> v2:172.21.15.190:3300/0 conn(0x7f6aa00a6d30 0x7f6aa00a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 10 monclient: _finish_hunting 0 2024-08-19T10:40:40.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 1 monclient: found mon.noname-d 2024-08-19T10:40:40.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:40.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:40.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.134+0000 7f6aa48ac640 1 -- 172.21.15.89:0/476221013 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa00a81d0 con 0x7f6aa01395b0 2024-08-19T10:40:40.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa48ac640 1 -- 172.21.15.89:0/476221013 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6aa01395b0 msgr2=0x7f6aa0139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:40.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa48ac640 1 --2- 172.21.15.89:0/476221013 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6aa01395b0 0x7f6aa0139a40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009e50 tx=0x7f6a9801c8f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ae6a1a0cb602ad35 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:40.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 1 -- 172.21.15.89:0/476221013 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a980090d0 con 0x7f6aa01395b0 2024-08-19T10:40:40.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:40.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:40.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 10 monclient: dump: 2024-08-19T10:40:40.150 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:40.150 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:40.151 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:40.152 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 10 monclient: _finish_auth 0 2024-08-19T10:40:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 1 -- 172.21.15.89:0/476221013 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a980a5ba0 con 0x7f6aa01395b0 2024-08-19T10:40:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 5 monclient: authenticate success, global_id 4938 2024-08-19T10:40:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 1 -- 172.21.15.89:0/476221013 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a980a5d50 con 0x7f6aa01395b0 2024-08-19T10:40:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 4 set_mon_vals no callback set 2024-08-19T10:40:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9effd640 10 monclient: dump: 2024-08-19T10:40:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:40.154 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient: shutdown 2024-08-19T10:40:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:40.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 -- 172.21.15.89:0/476221013 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6aa01395b0 msgr2=0x7f6aa0139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 --2- 172.21.15.89:0/476221013 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6aa01395b0 0x7f6aa0139a40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009e50 tx=0x7f6a9801c8f0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 -- 172.21.15.89:0/476221013 shutdown_connections 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 --2- 172.21.15.89:0/476221013 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6aa01395b0 0x7f6aa0139a40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 --2- 172.21.15.89:0/476221013 >> v2:172.21.15.190:3300/0 conn(0x7f6aa00a6d30 0x7f6aa00a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 --2- 172.21.15.89:0/476221013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6aa00a5270 0x7f6aa00a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 -- 172.21.15.89:0/476221013 >> 172.21.15.89:0/476221013 conn(0x7f6aa0069a80 msgr2=0x7f6aa00abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 -- 172.21.15.89:0/476221013 shutdown_connections 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 -- 172.21.15.89:0/476221013 wait complete. 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient: build_initial_monmap 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient: monmap: 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:40.155 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:40.156 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 Processor -- start 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 -- start start 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient: init 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient: _add_conns ranks=[4,8,5,0,7,1,2,6,3] 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6aa00a5270 0x7f6aa01d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient(hunting): picked mon.noname-c con 0x7f6aa00a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6aa00a6d30 0x7f6aa00a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient(hunting): picked mon.noname-e con 0x7f6aa00a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6aa01395b0 0x7f6aa00a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient(hunting): picked mon.noname-h con 0x7f6aa01395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6aa00a5650 con 0x7f6aa00a5270 2024-08-19T10:40:40.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6aa00a4a70 con 0x7f6aa01395b0 2024-08-19T10:40:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6aa00a57d0 con 0x7f6aa00a6d30 2024-08-19T10:40:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6aa630f640 10 monclient(hunting): authenticate will time out at 1089.959818s 2024-08-19T10:40:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9ffff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6aa00a5270 0x7f6aa01d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9f7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6aa00a6d30 0x7f6aa00a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9ffff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6aa00a5270 0x7f6aa01d14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:37450/0 (socket says 172.21.15.89:37450) 2024-08-19T10:40:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9ffff640 1 -- 172.21.15.89:0/924015343 learned_addr learned my addr 172.21.15.89:0/924015343 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9ffff640 10 monclient(hunting): get_auth_request con 0x7f6aa00a5270 auth_method 0 2024-08-19T10:40:40.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9f7fe640 10 monclient(hunting): get_auth_request con 0x7f6aa00a6d30 auth_method 0 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.138+0000 7f6a9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:40.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa48ac640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6aa01395b0 0x7f6aa00a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa48ac640 10 monclient(hunting): get_auth_request con 0x7f6aa01395b0 auth_method 0 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa48ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa48ac640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa48ac640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 10 monclient(hunting): handle_auth_done global_id 4981 payload 824 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 1 -- 172.21.15.89:0/924015343 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6aa00a6d30 msgr2=0x7f6aa00a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6aa00a6d30 0x7f6aa00a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 1 -- 172.21.15.89:0/924015343 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6aa01395b0 msgr2=0x7f6aa00a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6aa01395b0 0x7f6aa00a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 1 monclient: found mon.noname-c 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:40.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 1 -- 172.21.15.89:0/924015343 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa0140480 con 0x7f6aa00a5270 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9f7fe640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6aa00a6d30 0x7f6aa00a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 1 -- 172.21.15.89:0/924015343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6aa00a5270 msgr2=0x7f6aa01d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9ffff640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6aa00a5270 0x7f6aa01d14b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000e870 tx=0x7f6a9000ec70 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 1 -- 172.21.15.89:0/924015343 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a90017070 con 0x7f6aa00a5270 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 10 monclient: dump: 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:40.161 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:40.162 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:40.163 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 10 monclient: _finish_auth 0 2024-08-19T10:40:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 10 monclient: _check_auth_tickets 2024-08-19T10:40:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 1 -- 172.21.15.89:0/924015343 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a90013470 con 0x7f6aa00a5270 2024-08-19T10:40:40.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa630f640 5 monclient: authenticate success, global_id 4981 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa630f640 10 monclient: _renew_subs 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa630f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa630f640 1 -- 172.21.15.89:0/924015343 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa00a67f0 con 0x7f6aa00a5270 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 1 -- 172.21.15.89:0/924015343 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a90013a90 con 0x7f6aa00a5270 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 10 monclient: dump: 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:40.164 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:40.165 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9e7fc640 4 set_mon_vals no callback set 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa630f640 10 monclient: _renew_subs 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa630f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa630f640 1 -- 172.21.15.89:0/924015343 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa00a0930 con 0x7f6aa00a5270 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa630f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa630f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa630f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6aa630f640 1 -- 172.21.15.89:0/924015343 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa00a7110 con 0x7f6aa00a5270 2024-08-19T10:40:40.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 1 -- 172.21.15.89:0/924015343 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6a9001a070 con 0x7f6aa00a5270 2024-08-19T10:40:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.142+0000 7f6a9cff9640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a8003be00 0x7f6a8003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.146+0000 7f6a9cff9640 1 -- 172.21.15.89:0/924015343 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(71..71 src has 1..71) ==== 5134+0+0 (secure 0 0 0) 0x7f6a9000f040 con 0x7f6aa00a5270 2024-08-19T10:40:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.146+0000 7f6a9f7fe640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a8003be00 0x7f6a8003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.146+0000 7f6a9f7fe640 10 monclient: get_auth_request con 0x7f6a8003be00 auth_method 0 2024-08-19T10:40:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.146+0000 7f6a9f7fe640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a8003be00 0x7f6a8003e2c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6aa009ec40 tx=0x7f6a88008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.150+0000 7f6a9cff9640 1 -- 172.21.15.89:0/924015343 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6a90021020 con 0x7f6aa00a5270 2024-08-19T10:40:40.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.150+0000 7f6a9cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:40.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.150+0000 7f6a9cff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:40.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient: get_monmap_and_config 2024-08-19T10:40:40.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient: build_initial_monmap 2024-08-19T10:40:40.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient: monmap: 2024-08-19T10:40:40.217 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:40.217 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:40.218 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:40.217091+0000 2024-08-19T10:40:40.218 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:40.217091+0000 2024-08-19T10:40:40.218 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:40.218 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:40.218 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:40.218 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:40.218 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:40.218 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:40.219 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:40.219 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:40.219 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:40.219 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:40.219 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:40.219 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:40.219 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:40.219 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:40.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 1 Processor -- start 2024-08-19T10:40:40.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 1 -- start start 2024-08-19T10:40:40.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient: init 2024-08-19T10:40:40.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:40.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient: _add_conns ranks=[7,0,1,8,6,2,9,3,5,10,4] 2024-08-19T10:40:40.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2740a36a0 0x7ff2740a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient(hunting): picked mon.noname-i con 0x7ff2740a36a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:40.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff2740a3fc0 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:40.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff274137b80 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:40.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff27409d570 con 0x7ff2740a3fc0 2024-08-19T10:40:40.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff27409d850 con 0x7ff274137b80 2024-08-19T10:40:40.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:40.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff27409db30 con 0x7ff2740a36a0 2024-08-19T10:40:40.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:40.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27c377640 10 monclient(hunting): authenticate will time out at 1085.787477s 2024-08-19T10:40:40.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2740a36a0 0x7ff2740a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2740a36a0 0x7ff2740a3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:41496/0 (socket says 172.21.15.190:41496) 2024-08-19T10:40:40.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 1 -- 172.21.15.190:0/1857278484 learned_addr learned my addr 172.21.15.190:0/1857278484 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:40.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 10 monclient(hunting): get_auth_request con 0x7ff2740a36a0 auth_method 0 2024-08-19T10:40:40.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 10 monclient(hunting): handle_auth_done global_id 14173 payload 274 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 1 -- 172.21.15.190:0/1857278484 >> v1:172.21.15.190:6789/0 conn(0x7ff274137b80 legacy=0x7ff27413f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 1 -- 172.21.15.190:0/1857278484 >> v1:172.21.15.89:6789/0 conn(0x7ff2740a3fc0 legacy=0x7ff274137470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 10 monclient: _finish_hunting 0 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 1 monclient: found mon.noname-i 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:40.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:40:40.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.215+0000 7ff27a113640 1 -- 172.21.15.190:0/1857278484 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2740a55a0 con 0x7ff2740a36a0 2024-08-19T10:40:40.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a113640 1 -- 172.21.15.190:0/1857278484 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2740a36a0 msgr2=0x7ff2740a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a113640 1 --2- 172.21.15.190:0/1857278484 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2740a36a0 0x7ff2740a3a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0098a0 tx=0x7ff26c01c380 comp rx=0 tx=0).ready entity=mon.7 client_cookie=cc2b246f1820b700 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 1 -- 172.21.15.190:0/1857278484 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff26c0b0070 con 0x7ff2740a36a0 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 10 monclient: dump: 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:40.231 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:40.232 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:40.232 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:40.232 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:40.232 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:40.232 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:40.232 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:40.232 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:40.232 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:40.232 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:40.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 10 monclient: _finish_auth 0 2024-08-19T10:40:40.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 10 monclient: _check_auth_tickets 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 5 monclient: authenticate success, global_id 14173 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 1 -- 172.21.15.190:0/1857278484 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26c0aa810 con 0x7ff2740a36a0 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 4 set_mon_vals no callback set 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 1 -- 172.21.15.190:0/1857278484 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff26c0a93e0 con 0x7ff2740a36a0 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279111640 10 monclient: dump: 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:40.233 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:40.234 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient: shutdown 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 -- 172.21.15.190:0/1857278484 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2740a36a0 msgr2=0x7ff2740a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 --2- 172.21.15.190:0/1857278484 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2740a36a0 0x7ff2740a3a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0098a0 tx=0x7ff26c01c380 comp rx=0 tx=0).stop 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 -- 172.21.15.190:0/1857278484 shutdown_connections 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 --2- 172.21.15.190:0/1857278484 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2740a36a0 0x7ff2740a3a80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 -- 172.21.15.190:0/1857278484 >> 172.21.15.190:0/1857278484 conn(0x7ff2740a4790 msgr2=0x7ff2740a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 -- 172.21.15.190:0/1857278484 shutdown_connections 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 -- 172.21.15.190:0/1857278484 wait complete. 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient: build_initial_monmap 2024-08-19T10:40:40.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient: monmap: 2024-08-19T10:40:40.236 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:40.236 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:40.236 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:40.236 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:40.236 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:40.236 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:40.236 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:40.236 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:40.236 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:40.236 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:40.236 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 Processor -- start 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 -- start start 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient: init 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient: _add_conns ranks=[4,1,2,0,5,7,8,6,3] 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2740a36a0 0x7ff2741cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient(hunting): picked mon.noname-c con 0x7ff2740a36a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:40.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2740a3fc0 0x7ff2741cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient(hunting): picked mon.noname-f con 0x7ff2740a3fc0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:40.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff274137b80 0x7ff274138e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient(hunting): picked mon.noname-b con 0x7ff274137b80 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:40.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:40.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff27409ef20 con 0x7ff2740a3fc0 2024-08-19T10:40:40.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:40.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff27409eda0 con 0x7ff274137b80 2024-08-19T10:40:40.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:40.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff2740a1df0 con 0x7ff2740a36a0 2024-08-19T10:40:40.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:40.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27c377640 10 monclient(hunting): authenticate will time out at 1085.794548s 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2740a3fc0 0x7ff2741cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2740a3fc0 0x7ff2741cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60598/0 (socket says 172.21.15.190:60598) 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 1 -- 172.21.15.190:0/1712644454 learned_addr learned my addr 172.21.15.190:0/1712644454 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 10 monclient(hunting): get_auth_request con 0x7ff2740a3fc0 auth_method 0 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a113640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2740a36a0 0x7ff2741cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a914640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff274137b80 0x7ff274138e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a113640 10 monclient(hunting): get_auth_request con 0x7ff2740a36a0 auth_method 0 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a113640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a113640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a113640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a914640 10 monclient(hunting): get_auth_request con 0x7ff274137b80 auth_method 0 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a914640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a914640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff27a914640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 10 monclient(hunting): handle_auth_done global_id 14158 payload 824 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 1 -- 172.21.15.190:0/1712644454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2740a36a0 msgr2=0x7ff2741cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2740a36a0 0x7ff2741cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 1 -- 172.21.15.190:0/1712644454 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff274137b80 msgr2=0x7ff274138e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff274137b80 0x7ff274138e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 10 monclient: _finish_hunting 0 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 1 monclient: found mon.noname-f 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.223+0000 7ff279912640 1 -- 172.21.15.190:0/1712644454 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27413c8e0 con 0x7ff2740a3fc0 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff279912640 1 -- 172.21.15.190:0/1712644454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2740a3fc0 msgr2=0x7ff2741cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff279912640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2740a3fc0 0x7ff2741cfb30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c7d0 tx=0x7ff26800cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 1 -- 172.21.15.190:0/1712644454 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff268016020 con 0x7ff2740a3fc0 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 10 monclient: dump: 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:40.241 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:40.242 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 10 monclient: _finish_auth 0 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 1 -- 172.21.15.190:0/1712644454 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff268012420 con 0x7ff2740a3fc0 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 1 -- 172.21.15.190:0/1712644454 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff268012a40 con 0x7ff2740a3fc0 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 10 monclient: dump: 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:40.243 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:40.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27c377640 5 monclient: authenticate success, global_id 14158 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27c377640 10 monclient: _renew_subs 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff278910640 4 set_mon_vals no callback set 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27c377640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27c377640 1 -- 172.21.15.190:0/1712644454 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27413cbc0 con 0x7ff2740a3fc0 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27c377640 10 monclient: _renew_subs 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27c377640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27c377640 1 -- 172.21.15.190:0/1712644454 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27413c600 con 0x7ff2740a3fc0 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27c377640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27c377640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27c377640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27c377640 1 -- 172.21.15.190:0/1712644454 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2740a3a80 con 0x7ff2740a3fc0 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 1 -- 172.21.15.190:0/1712644454 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff268019020 con 0x7ff2740a3fc0 2024-08-19T10:40:40.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff26003be00 0x7ff26003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:40.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff272ffd640 1 -- 172.21.15.190:0/1712644454 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5134+0+0 (secure 0 0 0) 0x7ff26804ebf0 con 0x7ff2740a3fc0 2024-08-19T10:40:40.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27a113640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff26003be00 0x7ff26003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:40.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.227+0000 7ff27a113640 10 monclient: get_auth_request con 0x7ff26003be00 auth_method 0 2024-08-19T10:40:40.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.231+0000 7ff27a113640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff26003be00 0x7ff26003e2c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2740a5510 tx=0x7ff26c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:40.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.231+0000 7ff272ffd640 1 -- 172.21.15.190:0/1712644454 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff268020030 con 0x7ff2740a3fc0 2024-08-19T10:40:40.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.231+0000 7ff272ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:40.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.231+0000 7ff272ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:40.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.386+0000 7f6aa630f640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-08-19T10:40:40.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.386+0000 7f6aa630f640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-08-19T10:40:40.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.386+0000 7f6aa630f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:40.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.386+0000 7f6aa630f640 1 -- 172.21.15.89:0/924015343 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f6aa0064590 con 0x7f6aa00a5270 2024-08-19T10:40:40.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.386+0000 7f6a9cff9640 1 -- 172.21.15.89:0/924015343 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f6a9001ac80 con 0x7f6aa00a5270 2024-08-19T10:40:40.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.386+0000 7f6a9cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-08-19T10:40:40.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.386+0000 7f6a9cff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:40.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=11,logm=67} crc {auth=2220159425,config=2586702516,health=3926953126,kv=3907926731,logm=630195181}) 2024-08-19T10:40:40.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=9,config=2,health=12,kv=11,logm=66} crc {auth=2220159425,config=2586702516,health=3977944326,kv=533376655,logm=3307499425}) 2024-08-19T10:40:40.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=11,logm=67} crc {auth=2220159425,config=2586702516,health=3926953126,kv=3907926731,logm=630195181}) 2024-08-19T10:40:40.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=8,config=1,health=12,kv=12,logm=67} crc {auth=2080160915,config=1747058506,health=3977944326,kv=1101891974,logm=2286992568}) 2024-08-19T10:40:40.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=11,logm=67} crc {auth=2220159425,config=2586702516,health=3926953126,kv=3907926731,logm=630195181}) 2024-08-19T10:40:40.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=2,health=11,kv=11,logm=67} crc {auth=2220159425,config=2586702516,health=3926953126,kv=3907926731,logm=4235076735}) 2024-08-19T10:40:40.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=11,logm=67} crc {auth=2220159425,config=2586702516,health=3926953126,kv=3907926731,logm=630195181}) 2024-08-19T10:40:40.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=2220159425,config=2586702516,health=3977944326,kv=890679220,logm=4257358435}) 2024-08-19T10:40:40.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 -- 172.21.15.89:0/924015343 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a8003be00 msgr2=0x7f6a8003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.396 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a8003be00 0x7f6a8003e2c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6aa009ec40 tx=0x7f6a88008040 comp rx=0 tx=0).stop 2024-08-19T10:40:40.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 10 monclient: shutdown 2024-08-19T10:40:40.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:40.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 -- 172.21.15.89:0/924015343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6aa00a5270 msgr2=0x7f6aa01d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:40.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6aa00a5270 0x7f6aa01d14b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000e870 tx=0x7f6a9000ec70 comp rx=0 tx=0).stop 2024-08-19T10:40:40.397 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=11,logm=67} crc {auth=2220159425,config=2586702516,health=3926953126,kv=3907926731,logm=630195181}) 2024-08-19T10:40:40.397 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=2220159425,config=2586702516,health=3977944326,kv=1101891974,logm=3696256955}) 2024-08-19T10:40:40.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=11,logm=67} crc {auth=2220159425,config=2586702516,health=3926953126,kv=3907926731,logm=630195181}) 2024-08-19T10:40:40.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=8,config=2,health=8,kv=11,logm=71} crc {auth=3525822709,config=2586702516,health=2157493640,kv=2081965194,logm=3027841048}) 2024-08-19T10:40:40.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=11,logm=67} crc {auth=2220159425,config=2586702516,health=3926953126,kv=3907926731,logm=630195181}) 2024-08-19T10:40:40.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 -- 172.21.15.89:0/924015343 shutdown_connections 2024-08-19T10:40:40.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6aa01395b0 0x7f6aa00a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=9,config=2,health=10,kv=10,logm=69} crc {auth=2220159425,config=2586702516,health=1151464652,kv=3395565556,logm=1455045051}) 2024-08-19T10:40:40.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a8003be00 0x7f6a8003e2c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6aa00a6d30 0x7f6aa00a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 --2- 172.21.15.89:0/924015343 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6aa00a5270 0x7f6aa01d14b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:40.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 -- 172.21.15.89:0/924015343 >> 172.21.15.89:0/924015343 conn(0x7f6aa0069a80 msgr2=0x7f6aa01388a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:40.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=11,logm=67} crc {auth=2220159425,config=2586702516,health=3926953126,kv=3907926731,logm=630195181}) 2024-08-19T10:40:40.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=9,config=1,health=12,kv=10,logm=68} crc {auth=2220159425,config=1747058506,health=3977944326,kv=1067870453,logm=1963752987}) 2024-08-19T10:40:40.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 -- 172.21.15.89:0/924015343 shutdown_connections 2024-08-19T10:40:40.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:40.390+0000 7f6aa630f640 1 -- 172.21.15.89:0/924015343 wait complete. 2024-08-19T10:40:40.411 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-08-19T10:40:40.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.415+0000 7ff27c377640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-08-19T10:40:40.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.415+0000 7ff27c377640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-08-19T10:40:40.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.419+0000 7ff27c377640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:40:40.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:40.419+0000 7ff27c377640 1 -- 172.21.15.190:0/1712644454 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7ff27406db60 con 0x7ff2740a3fc0 2024-08-19T10:40:40.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1647765884}) 2024-08-19T10:40:40.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=23450391}) 2024-08-19T10:40:40.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1647765884}) 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3747593652}) 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1647765884}) 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1731295718}) 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1647765884}) 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1552861444}) 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1647765884}) 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=265552446}) 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1647765884}) 2024-08-19T10:40:40.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1949463084}) 2024-08-19T10:40:40.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1647765884}) 2024-08-19T10:40:40.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3968042467}) 2024-08-19T10:40:40.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1647765884}) 2024-08-19T10:40:40.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3846814201}) 2024-08-19T10:40:40.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1724635781}) 2024-08-19T10:40:40.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2441630441}) 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1724635781}) 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=645894401}) 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1724635781}) 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1057514204}) 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1724635781}) 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2475213996}) 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1724635781}) 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1562555844}) 2024-08-19T10:40:40.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1724635781}) 2024-08-19T10:40:40.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3730557814}) 2024-08-19T10:40:40.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1724635781}) 2024-08-19T10:40:40.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2912050201}) 2024-08-19T10:40:40.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:40.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1724635781}) 2024-08-19T10:40:40.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=448551563}) 2024-08-19T10:40:40.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3390658177}) 2024-08-19T10:40:41.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=256839906}) 2024-08-19T10:40:41.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3390658177}) 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3142177212}) 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3390658177}) 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=439999501}) 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3390658177}) 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=339845142}) 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3390658177}) 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1451543976}) 2024-08-19T10:40:41.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.002 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3390658177}) 2024-08-19T10:40:41.002 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1389485728}) 2024-08-19T10:40:41.002 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.002 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3390658177}) 2024-08-19T10:40:41.002 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4056977991}) 2024-08-19T10:40:41.002 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.002 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3390658177}) 2024-08-19T10:40:41.002 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:40.990+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4099680239}) 2024-08-19T10:40:41.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=619057255}) 2024-08-19T10:40:41.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3676305958}) 2024-08-19T10:40:41.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=619057255}) 2024-08-19T10:40:41.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3136969604}) 2024-08-19T10:40:41.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=619057255}) 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3139784088}) 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=619057255}) 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4118952745}) 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=619057255}) 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=131901526}) 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=619057255}) 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1309747710}) 2024-08-19T10:40:41.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=619057255}) 2024-08-19T10:40:41.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=301177926}) 2024-08-19T10:40:41.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=619057255}) 2024-08-19T10:40:41.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1678863040}) 2024-08-19T10:40:41.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:41.011+0000 7ff272ffd640 1 -- 172.21.15.190:0/1712644454 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v72) ==== 58+0+2 (secure 0 0 0) 0x7ff268022300 con 0x7ff2740a3fc0 2024-08-19T10:40:41.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:41.011+0000 7ff272ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-08-19T10:40:41.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:41.011+0000 7ff272ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:41.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:41.123+0000 7ff27c377640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-08-19T10:40:41.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:41.123+0000 7ff27c377640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-08-19T10:40:41.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:41.123+0000 7ff27c377640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:40:41.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:41.123+0000 7ff27c377640 1 -- 172.21.15.190:0/1712644454 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7ff2741d0070 con 0x7ff2740a3fc0 2024-08-19T10:40:41.412 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:40:41.412 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:40:41.504 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.504 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3430316778}) 2024-08-19T10:40:41.504 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1397718870}) 2024-08-19T10:40:41.504 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.504 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3430316778}) 2024-08-19T10:40:41.504 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2997709029}) 2024-08-19T10:40:41.504 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.504 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3430316778}) 2024-08-19T10:40:41.504 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1156625419}) 2024-08-19T10:40:41.504 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.505 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3430316778}) 2024-08-19T10:40:41.505 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=904742096}) 2024-08-19T10:40:41.505 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.505 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3430316778}) 2024-08-19T10:40:41.505 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2920652602}) 2024-08-19T10:40:41.505 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.505 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3430316778}) 2024-08-19T10:40:41.505 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1440934728}) 2024-08-19T10:40:41.506 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.506 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3430316778}) 2024-08-19T10:40:41.506 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=968171817}) 2024-08-19T10:40:41.506 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.506 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3430316778}) 2024-08-19T10:40:41.506 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3380270711}) 2024-08-19T10:40:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient: get_monmap_and_config 2024-08-19T10:40:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient: build_initial_monmap 2024-08-19T10:40:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient: monmap: 2024-08-19T10:40:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:41.542122+0000 2024-08-19T10:40:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:41.542122+0000 2024-08-19T10:40:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:41.543 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:41.543 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:41.543 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:41.543 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:41.543 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:41.543 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 1 Processor -- start 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 1 -- start start 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient: init 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient: _add_conns ranks=[0,3,10,5,1,9,6,8,2,7,4] 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f166c0a5270 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c0a6f00 0x7f166c13ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient(hunting): picked mon.noname-g con 0x7f166c0a6f00 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f166c0a72e0 0x7f166c140f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient(hunting): picked mon.noname-f con 0x7f166c0a72e0 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166c0a8710 con 0x7f166c0a5270 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f166c0a5a40 con 0x7f166c0a72e0 2024-08-19T10:40:41.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f166c0a81d0 con 0x7f166c0a6f00 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f16731f9640 10 monclient(hunting): authenticate will time out at 1091.358857s 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1670f95640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f166c0a5270 0x7f166c0a67f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59276/0 (socket says 172.21.15.89:59276) 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1670f95640 1 -- 172.21.15.89:0/873282109 learned_addr learned my addr 172.21.15.89:0/873282109 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 1 --2- 172.21.15.89:0/873282109 >> v2:172.21.15.190:3300/0 conn(0x7f166c0a72e0 0x7f166c140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 10 monclient(hunting): get_auth_request con 0x7f166c0a72e0 auth_method 0 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f166bfff640 1 --2- 172.21.15.89:0/873282109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c0a6f00 0x7f166c13ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f166bfff640 10 monclient(hunting): get_auth_request con 0x7f166c0a6f00 auth_method 0 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f166bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f166bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f166bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f166bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f166bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f166bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 10 monclient(hunting): handle_auth_done global_id 14167 payload 274 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 1 -- 172.21.15.89:0/873282109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c0a6f00 msgr2=0x7f166c13ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 1 --2- 172.21.15.89:0/873282109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c0a6f00 0x7f166c13ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 1 -- 172.21.15.89:0/873282109 >> v1:172.21.15.89:6789/0 conn(0x7f166c0a5270 legacy=0x7f166c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 10 monclient: _finish_hunting 0 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 1 monclient: found mon.noname-f 2024-08-19T10:40:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:40:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.538+0000 7f1671796640 1 -- 172.21.15.89:0/873282109 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c0a4790 con 0x7f166c0a72e0 2024-08-19T10:40:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1671796640 1 -- 172.21.15.89:0/873282109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a72e0 msgr2=0x7f166c140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1671796640 1 --2- 172.21.15.89:0/873282109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a72e0 0x7f166c140f50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f16640097c0 tx=0x7f166401c150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1204590ceed3b8e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 1 -- 172.21.15.89:0/873282109 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16640b0070 con 0x7f166c0a72e0 2024-08-19T10:40:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:40:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 10 monclient: dump: 2024-08-19T10:40:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 1 -- 172.21.15.89:0/873282109 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16640aa810 con 0x7f166c0a72e0 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 1 -- 172.21.15.89:0/873282109 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16640a9300 con 0x7f166c0a72e0 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166b7fe640 10 monclient: dump: 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 5 monclient: authenticate success, global_id 14167 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 4 set_mon_vals no callback set 2024-08-19T10:40:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient: shutdown 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 -- 172.21.15.89:0/873282109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a72e0 msgr2=0x7f166c140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 --2- 172.21.15.89:0/873282109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a72e0 0x7f166c140f50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f16640097c0 tx=0x7f166401c150 comp rx=0 tx=0).stop 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 -- 172.21.15.89:0/873282109 shutdown_connections 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 --2- 172.21.15.89:0/873282109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f166c0a72e0 0x7f166c140f50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 --2- 172.21.15.89:0/873282109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f166c0a6f00 0x7f166c13ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 -- 172.21.15.89:0/873282109 >> 172.21.15.89:0/873282109 conn(0x7f166c069a80 msgr2=0x7f166c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 -- 172.21.15.89:0/873282109 shutdown_connections 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 -- 172.21.15.89:0/873282109 wait complete. 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient: build_initial_monmap 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient: monmap: 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 Processor -- start 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 -- start start 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient: init 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient: _add_conns ranks=[6,7,0,1,3,8,5,2,4] 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f166c0a5270 0x7f166c128b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient(hunting): picked mon.noname-d con 0x7f166c0a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f166c0a6f00 0x7f166c13f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient(hunting): picked mon.noname-i con 0x7f166c0a6f00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166c0a72e0 0x7f166c126730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient(hunting): picked mon.noname-a con 0x7f166c0a72e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f166c143e30 con 0x7f166c0a72e0 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:41.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f166c143cb0 con 0x7f166c0a5270 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f166c143fb0 con 0x7f166c0a6f00 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f16731f9640 10 monclient(hunting): authenticate will time out at 1091.366757s 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f166c0a5270 0x7f166c128b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f166c0a5270 0x7f166c128b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54340/0 (socket says 172.21.15.89:54340) 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 1 -- 172.21.15.89:0/3875072394 learned_addr learned my addr 172.21.15.89:0/3875072394 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 10 monclient(hunting): get_auth_request con 0x7f166c0a5270 auth_method 0 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1671796640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166c0a72e0 0x7f166c126730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1671796640 10 monclient(hunting): get_auth_request con 0x7f166c0a72e0 auth_method 0 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1671796640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1671796640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1671796640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1671796640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1671796640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1671796640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166bfff640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f166c0a6f00 0x7f166c13f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 10 monclient(hunting): handle_auth_done global_id 4947 payload 824 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 1 -- 172.21.15.89:0/3875072394 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f166c0a6f00 msgr2=0x7f166c13f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f166c0a6f00 0x7f166c13f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 1 -- 172.21.15.89:0/3875072394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166c0a72e0 msgr2=0x7f166c126730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166c0a72e0 0x7f166c126730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 10 monclient: _finish_hunting 0 2024-08-19T10:40:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 1 monclient: found mon.noname-d 2024-08-19T10:40:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1670f95640 1 -- 172.21.15.89:0/3875072394 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c1d1dd0 con 0x7f166c0a5270 2024-08-19T10:40:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166bfff640 10 monclient: get_auth_request con 0x7f166c0a6f00 auth_method 0 2024-08-19T10:40:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f1671796640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166c0a72e0 0x7f166c126730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.546+0000 7f166bfff640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f166c0a6f00 0x7f166c13f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f1670f95640 1 -- 172.21.15.89:0/3875072394 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f166c0a5270 msgr2=0x7f166c128b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f1670f95640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f166c0a5270 0x7f166c128b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f166000b430 tx=0x7f166000b8f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 1 -- 172.21.15.89:0/3875072394 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1660015020 con 0x7f166c0a5270 2024-08-19T10:40:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 10 monclient: dump: 2024-08-19T10:40:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 10 monclient: _finish_auth 0 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 10 monclient: _check_auth_tickets 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 1 -- 172.21.15.89:0/3875072394 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1660011cb0 con 0x7f166c0a5270 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 1 -- 172.21.15.89:0/3875072394 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1660011e60 con 0x7f166c0a5270 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16731f9640 5 monclient: authenticate success, global_id 4947 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16731f9640 10 monclient: _renew_subs 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16731f9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16731f9640 1 -- 172.21.15.89:0/3875072394 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c0ab330 con 0x7f166c0a5270 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 10 monclient: dump: 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f166affd640 4 set_mon_vals no callback set 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16731f9640 10 monclient: _renew_subs 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16731f9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16731f9640 1 -- 172.21.15.89:0/3875072394 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c128010 con 0x7f166c0a5270 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16731f9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16731f9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16731f9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16731f9640 1 -- 172.21.15.89:0/3875072394 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166c0a6200 con 0x7f166c0a5270 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 1 -- 172.21.15.89:0/3875072394 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1660018020 con 0x7f166c0a5270 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f165803be00 0x7f165803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f16697fa640 1 -- 172.21.15.89:0/3875072394 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(72..72 src has 1..72) ==== 5249+0+0 (secure 0 0 0) 0x7f166004eb30 con 0x7f166c0a5270 2024-08-19T10:40:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f166bfff640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f165803be00 0x7f165803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.550+0000 7f166bfff640 10 monclient: get_auth_request con 0x7f165803be00 auth_method 0 2024-08-19T10:40:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.554+0000 7f166bfff640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f165803be00 0x7f165803e2c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f166c128680 tx=0x7f165c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.554+0000 7f16697fa640 1 -- 172.21.15.89:0/3875072394 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f166001f020 con 0x7f166c0a5270 2024-08-19T10:40:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.554+0000 7f16697fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.554+0000 7f16697fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.786+0000 7f16731f9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:40:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.786+0000 7f16731f9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:40:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.786+0000 7f16731f9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:41.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.786+0000 7f16731f9640 1 -- 172.21.15.89:0/3875072394 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f166c0a67f0 con 0x7f166c0a5270 2024-08-19T10:40:41.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.786+0000 7f16697fa640 1 -- 172.21.15.89:0/3875072394 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7f166c1d1dd0 con 0x7f166c0a5270 2024-08-19T10:40:41.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.786+0000 7f16697fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:40:41.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.786+0000 7f16697fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:41.792 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":26,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:40:41.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 -- 172.21.15.89:0/3875072394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f165803be00 msgr2=0x7f165803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f165803be00 0x7f165803e2c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f166c128680 tx=0x7f165c00b040 comp rx=0 tx=0).stop 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 10 monclient: shutdown 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 -- 172.21.15.89:0/3875072394 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f166c0a5270 msgr2=0x7f166c128b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f166c0a5270 0x7f166c128b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f166000b430 tx=0x7f166000b8f0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 -- 172.21.15.89:0/3875072394 shutdown_connections 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f165803be00 0x7f165803e2c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f166c0a72e0 0x7f166c126730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f166c0a6f00 0x7f166c13f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 --2- 172.21.15.89:0/3875072394 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f166c0a5270 0x7f166c128b70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 -- 172.21.15.89:0/3875072394 >> 172.21.15.89:0/3875072394 conn(0x7f166c069a80 msgr2=0x7f166c0aa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:41.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 -- 172.21.15.89:0/3875072394 shutdown_connections 2024-08-19T10:40:41.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.790+0000 7f16731f9640 1 -- 172.21.15.89:0/3875072394 wait complete. 2024-08-19T10:40:41.799 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.799 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592697455}) 2024-08-19T10:40:41.799 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1926943933}) 2024-08-19T10:40:41.799 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.799 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592697455}) 2024-08-19T10:40:41.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4240635852}) 2024-08-19T10:40:41.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592697455}) 2024-08-19T10:40:41.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=347408523}) 2024-08-19T10:40:41.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592697455}) 2024-08-19T10:40:41.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2691537270}) 2024-08-19T10:40:41.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592697455}) 2024-08-19T10:40:41.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3973499933}) 2024-08-19T10:40:41.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592697455}) 2024-08-19T10:40:41.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=645626544}) 2024-08-19T10:40:41.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592697455}) 2024-08-19T10:40:41.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1173250665}) 2024-08-19T10:40:41.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592697455}) 2024-08-19T10:40:41.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=461484858}) 2024-08-19T10:40:41.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623150348}) 2024-08-19T10:40:41.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1835352204}) 2024-08-19T10:40:41.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623150348}) 2024-08-19T10:40:41.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3812473085}) 2024-08-19T10:40:41.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623150348}) 2024-08-19T10:40:41.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=454722289}) 2024-08-19T10:40:41.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623150348}) 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2534805621}) 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623150348}) 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1248821098}) 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623150348}) 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=234405479}) 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623150348}) 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2521236036}) 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623150348}) 2024-08-19T10:40:41.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=753294725}) 2024-08-19T10:40:41.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3157067511}) 2024-08-19T10:40:41.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1089514790}) 2024-08-19T10:40:41.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3157067511}) 2024-08-19T10:40:41.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=987510567}) 2024-08-19T10:40:41.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3157067511}) 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3740255767}) 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3157067511}) 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3639068681}) 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3157067511}) 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2303312847}) 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3157067511}) 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=432987397}) 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3157067511}) 2024-08-19T10:40:41.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2680385053}) 2024-08-19T10:40:41.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3157067511}) 2024-08-19T10:40:41.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.802+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=777914873}) 2024-08-19T10:40:41.811 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2369400523}) 2024-08-19T10:40:41.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1260997288}) 2024-08-19T10:40:41.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2369400523}) 2024-08-19T10:40:41.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=502159468}) 2024-08-19T10:40:41.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2369400523}) 2024-08-19T10:40:41.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2537153329}) 2024-08-19T10:40:41.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2369400523}) 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1467074298}) 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2369400523}) 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=979603154}) 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2369400523}) 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4165086464}) 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2369400523}) 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=18732602}) 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2369400523}) 2024-08-19T10:40:41.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3107538331}) 2024-08-19T10:40:41.815 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:40:41.815 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-08-19T10:40:41.815 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:40:41.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:40:41.818 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.818 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=86,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=3677287309,mdsmap=2043744197,mgr=544454686,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=3616038664}) 2024-08-19T10:40:41.818 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=85,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=2934841947,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=3616038664}) 2024-08-19T10:40:41.818 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.818 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=86,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=3677287309,mdsmap=2043744197,mgr=544454686,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=3616038664}) 2024-08-19T10:40:41.818 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=88,mdsmap=3,mgr=8,mgr_command_descs=1} crc {logm=557443985,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062}) 2024-08-19T10:40:41.818 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.818 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=86,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=3677287309,mdsmap=2043744197,mgr=544454686,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=3616038664}) 2024-08-19T10:40:41.818 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=87,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=1} crc {logm=2431955928,mdsmap=2043744197,mgr=322451002,mgr_metadata=536852559,mgrstat=918027738}) 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=86,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=3677287309,mdsmap=2043744197,mgr=544454686,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=3616038664}) 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=87,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=1} crc {logm=1723481958,mdsmap=2043744197,mgr=4122621231,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=918027738}) 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=86,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=3677287309,mdsmap=2043744197,mgr=544454686,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=3616038664}) 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=86,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=3} crc {logm=2702771085,mdsmap=3121089713,mgr=3114150001,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=103479993}) 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=86,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=3677287309,mdsmap=2043744197,mgr=544454686,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=3616038664}) 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=87,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=1} crc {logm=3541175568,mdsmap=2043744197,mgr=3997098071,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=871144001}) 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=86,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=3677287309,mdsmap=2043744197,mgr=544454686,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=3616038664}) 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=87,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1} crc {logm=1977950796,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559}) 2024-08-19T10:40:41.819 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=86,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=2} crc {logm=3677287309,mdsmap=2043744197,mgr=544454686,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=3616038664}) 2024-08-19T10:40:41.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=88,mdsmap=3,mgr=8,mgr_command_descs=1} crc {logm=1138504311,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062}) 2024-08-19T10:40:41.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mgrstat=2639195657,mon_config_key=2487703808,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=768801051}) 2024-08-19T10:40:41.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgrstat=9,mon_config_key=17,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=62} crc {mgrstat=2639195657,mon_config_key=2552794901,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=3027648509}) 2024-08-19T10:40:41.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mgrstat=2639195657,mon_config_key=2487703808,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=768801051}) 2024-08-19T10:40:41.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgrstat=8,mon_config_key=17,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mgrstat=429885861,mon_config_key=2552794901,monmap=317198000,osd_metadata=2976096237,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=4227282211}) 2024-08-19T10:40:41.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mgrstat=2639195657,mon_config_key=2487703808,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=768801051}) 2024-08-19T10:40:41.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mgrstat=9,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=64} crc {mgrstat=2639195657,mon_config_key=1137862404,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=3058936622,osdmap=2052617262}) 2024-08-19T10:40:41.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mgrstat=2639195657,mon_config_key=2487703808,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=768801051}) 2024-08-19T10:40:41.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mgrstat=9,mon_config_key=17,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=62} crc {mgrstat=2639195657,mon_config_key=2552794901,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=841190636,osdmap=1769336220}) 2024-08-19T10:40:41.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mgrstat=2639195657,mon_config_key=2487703808,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=768801051}) 2024-08-19T10:40:41.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgrstat=8,mon_config_key=17,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=65} crc {mgrstat=3971154266,mon_config_key=2620370730,monmap=3140514009,osd_metadata=1074877036,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=3177984510}) 2024-08-19T10:40:41.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mgrstat=2639195657,mon_config_key=2487703808,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=768801051}) 2024-08-19T10:40:41.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mgrstat=8,mon_config_key=17,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mgrstat=2428289451,mon_config_key=2552794901,monmap=317198000,osd_metadata=2976096237,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=249223955}) 2024-08-19T10:40:41.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mgrstat=2639195657,mon_config_key=2487703808,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=768801051}) 2024-08-19T10:40:41.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mgrstat=9,mon_config_key=17,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=63} crc {mgrstat=2639195657,mon_config_key=2552794901,monmap=1845284827,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=2088896345}) 2024-08-19T10:40:41.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mgrstat=2639195657,mon_config_key=2487703808,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=768801051}) 2024-08-19T10:40:41.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.910+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgrstat=9,mon_config_key=16,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=64} crc {mgrstat=2639195657,mon_config_key=3239049122,monmap=317198000,osd_metadata=1074877036,osd_pg_creating=2516224954,osd_snap=841190635,osdmap=2231624490}) 2024-08-19T10:40:41.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient: get_monmap_and_config 2024-08-19T10:40:41.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient: build_initial_monmap 2024-08-19T10:40:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient: monmap: 2024-08-19T10:40:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:41.925338+0000 2024-08-19T10:40:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:41.925338+0000 2024-08-19T10:40:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:41.925 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:41.926 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:41.926 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 1 Processor -- start 2024-08-19T10:40:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 1 -- start start 2024-08-19T10:40:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient: init 2024-08-19T10:40:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient: _add_conns ranks=[4,1,0,5,7,3,9,6,10,2,8] 2024-08-19T10:40:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a3d00 0x7f72340a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient(hunting): picked mon.noname-c con 0x7f72340a3d00 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f72340a1a00 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f723413c910 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f723406db50 con 0x7f723413c910 2024-08-19T10:40:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f723409c9c0 con 0x7f72340a1a00 2024-08-19T10:40:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f723409d3d0 con 0x7f72340a3d00 2024-08-19T10:40:41.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f723ba54640 10 monclient(hunting): authenticate will time out at 1091.743115s 2024-08-19T10:40:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a3d00 0x7f72340a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a3d00 0x7f72340a40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54082/0 (socket says 172.21.15.89:54082) 2024-08-19T10:40:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 1 -- 172.21.15.89:0/1625149616 learned_addr learned my addr 172.21.15.89:0/1625149616 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 10 monclient(hunting): get_auth_request con 0x7f72340a3d00 auth_method 0 2024-08-19T10:40:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:41.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 10 monclient(hunting): handle_auth_done global_id 4999 payload 274 2024-08-19T10:40:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 1 -- 172.21.15.89:0/1625149616 >> v1:172.21.15.190:6789/0 conn(0x7f72340a1a00 legacy=0x7f72340a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 1 -- 172.21.15.89:0/1625149616 >> v1:172.21.15.89:6789/0 conn(0x7f723413c910 legacy=0x7f723413ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 10 monclient: _finish_hunting 0 2024-08-19T10:40:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 1 monclient: found mon.noname-c 2024-08-19T10:40:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:41.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.922+0000 7f72397f0640 1 -- 172.21.15.89:0/1625149616 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72340691b0 con 0x7f72340a3d00 2024-08-19T10:40:41.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f72397f0640 1 -- 172.21.15.89:0/1625149616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a3d00 msgr2=0x7f72340a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:41.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f72397f0640 1 --2- 172.21.15.89:0/1625149616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a3d00 0x7f72340a40e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f72300098a0 tx=0x7f723001c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1d0d9750da5c8fde server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:41.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 1 -- 172.21.15.89:0/1625149616 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f72300b0070 con 0x7f72340a3d00 2024-08-19T10:40:41.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:41.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 10 monclient: dump: 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:41.936 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 10 monclient: _finish_auth 0 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 1 -- 172.21.15.89:0/1625149616 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72300aa810 con 0x7f72340a3d00 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f723ba54640 5 monclient: authenticate success, global_id 4999 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f723ba54640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:41.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 1 -- 172.21.15.89:0/1625149616 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f72300aaa40 con 0x7f72340a3d00 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f723ba54640 4 set_mon_vals no callback set 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.926+0000 7f722bfff640 10 monclient: dump: 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:41.938 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient: shutdown 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 -- 172.21.15.89:0/1625149616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a3d00 msgr2=0x7f72340a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 --2- 172.21.15.89:0/1625149616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a3d00 0x7f72340a40e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f72300098a0 tx=0x7f723001c440 comp rx=0 tx=0).stop 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 -- 172.21.15.89:0/1625149616 shutdown_connections 2024-08-19T10:40:41.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 --2- 172.21.15.89:0/1625149616 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a3d00 0x7f72340a40e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 -- 172.21.15.89:0/1625149616 >> 172.21.15.89:0/1625149616 conn(0x7f72340a4df0 msgr2=0x7f72340a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 -- 172.21.15.89:0/1625149616 shutdown_connections 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 -- 172.21.15.89:0/1625149616 wait complete. 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient: build_initial_monmap 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient: monmap: 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:41.940 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 Processor -- start 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 -- start start 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient: init 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:41.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient: _add_conns ranks=[4,5,1,7,0,2,8,6,3] 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a1a00 0x7f7234137760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient(hunting): picked mon.noname-c con 0x7f72340a1a00 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f72340a3d00 0x7f7234139700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient(hunting): picked mon.noname-h con 0x7f72340a3d00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f723413c910 0x7f7234139c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient(hunting): picked mon.noname-f con 0x7f723413c910 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7234142e50 con 0x7f723413c910 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f723406fd00 con 0x7f72340a1a00 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7234142fd0 con 0x7f72340a3d00 2024-08-19T10:40:41.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f723ba54640 10 monclient(hunting): authenticate will time out at 1091.750264s 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a1a00 0x7f7234137760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a1a00 0x7f7234137760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54096/0 (socket says 172.21.15.89:54096) 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 1 -- 172.21.15.89:0/1506682866 learned_addr learned my addr 172.21.15.89:0/1506682866 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 10 monclient(hunting): get_auth_request con 0x7f72340a1a00 auth_method 0 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 10 monclient(hunting): handle_auth_done global_id 5008 payload 824 2024-08-19T10:40:41.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 1 -- 172.21.15.89:0/1506682866 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f72340a3d00 msgr2=0x7f7234139700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f72340a3d00 0x7f7234139700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f7239ff1640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f723413c910 0x7f7234139c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 1 -- 172.21.15.89:0/1506682866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f723413c910 msgr2=0x7f7234139c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f723413c910 0x7f7234139c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 10 monclient: _finish_hunting 0 2024-08-19T10:40:41.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 1 monclient: found mon.noname-c 2024-08-19T10:40:41.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:41.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:41.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.930+0000 7f72397f0640 1 -- 172.21.15.89:0/1506682866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723412df50 con 0x7f72340a1a00 2024-08-19T10:40:41.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f72397f0640 1 -- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a1a00 msgr2=0x7f7234137760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:41.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f72397f0640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a1a00 0x7f7234137760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7230000c10 tx=0x7f72300191e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:41.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f72300b9070 con 0x7f72340a1a00 2024-08-19T10:40:41.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:41.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:41.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 10 monclient: dump: 2024-08-19T10:40:41.948 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:41.948 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.948 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.948 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.949 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:41.949 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.949 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:41.949 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:41.949 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:41.949 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:41.949 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:41.949 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:41.949 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:41.949 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:41.949 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:41.950 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:41.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:41.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:41.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f723ba54640 5 monclient: authenticate success, global_id 5008 2024-08-19T10:40:41.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72300044b0 con 0x7f72340a1a00 2024-08-19T10:40:41.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f723ba54640 10 monclient: _renew_subs 2024-08-19T10:40:41.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f723ba54640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:41.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f723ba54640 1 -- 172.21.15.89:0/1506682866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72340a7d80 con 0x7f72340a1a00 2024-08-19T10:40:41.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:41.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f72300b0040 con 0x7f72340a1a00 2024-08-19T10:40:41.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:41.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:41.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f722affd640 4 set_mon_vals no callback set 2024-08-19T10:40:41.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 10 monclient: dump: 2024-08-19T10:40:41.951 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:41.951 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.951 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.951 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.952 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:41.952 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.952 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:41.952 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:41.952 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:41.952 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:41.952 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:41.952 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:41.952 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:41.952 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:41.953 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:41.953 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f723ba54640 10 monclient: _renew_subs 2024-08-19T10:40:41.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f723ba54640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:41.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f723ba54640 1 -- 172.21.15.89:0/1506682866 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f723413b100 con 0x7f72340a1a00 2024-08-19T10:40:41.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f723ba54640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:41.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f723ba54640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723413a180 0x7f723413a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f723ba54640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:41.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f723ba54640 1 -- 172.21.15.89:0/1506682866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f72341c8370 con 0x7f723413a180 2024-08-19T10:40:41.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7238fef640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723413a180 0x7f723413a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7238fef640 10 monclient: get_auth_request con 0x7f723413a180 auth_method 0 2024-08-19T10:40:41.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7238fef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:41.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7238fef640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:41.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7238fef640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:41.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7238fef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:41.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7238fef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:41.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7238fef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:41.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7230004040 con 0x7f72340a1a00 2024-08-19T10:40:41.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7238fef640 10 monclient(hunting): handle_auth_done global_id 5067 payload 824 2024-08-19T10:40:41.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7238fef640 1 -- 172.21.15.89:0/1506682866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f72340a40e0 con 0x7f723413a180 2024-08-19T10:40:41.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f721803be00 0x7f721803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(72..72 src has 1..72) ==== 5249+0+0 (secure 0 0 0) 0x7f72300e7cc0 con 0x7f72340a1a00 2024-08-19T10:40:41.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7239ff1640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f721803be00 0x7f721803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.934+0000 7f7239ff1640 10 monclient: get_auth_request con 0x7f721803be00 auth_method 0 2024-08-19T10:40:41.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7238fef640 1 -- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723413a180 msgr2=0x7f723413a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:41.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7238fef640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723413a180 0x7f723413a560 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7234074cb0 tx=0x7f722400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:41.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7224005480 con 0x7f723413a180 2024-08-19T10:40:41.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7229ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:41.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7229ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:41.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7229ffb640 10 monclient: dump: 2024-08-19T10:40:41.956 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:41.956 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.957 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.957 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.957 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:41.957 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.957 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:41.957 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:41.957 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:41.957 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:41.957 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:41.958 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:41.958 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:41.958 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:41.958 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:41.958 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7224003810 con 0x7f723413a180 2024-08-19T10:40:41.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7229ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:41.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7229ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:41.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723413a180 msgr2=0x7f723413a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7229ffb640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723413a180 0x7f723413a560 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7234074cb0 tx=0x7f722400c7a0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723413a180 msgr2=0x7f723413a560 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7229ffb640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f723413a180 0x7f723413a560 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.938+0000 7f7239ff1640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f721803be00 0x7f721803e2c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f72340695c0 tx=0x7f722c0071c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f723ba54640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f723ba54640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7234071cd0 0x7f72340720e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f723ba54640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f723ba54640 1 -- 172.21.15.89:0/1506682866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7234064590 con 0x7f7234071cd0 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f7238fef640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7234071cd0 0x7f72340720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f7238fef640 10 monclient: get_auth_request con 0x7f7234071cd0 auth_method 0 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f7238fef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f7238fef640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:41.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f7238fef640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:41.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f7238fef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:41.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f7238fef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:41.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f7238fef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:41.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f7238fef640 10 monclient(hunting): handle_auth_done global_id 5076 payload 824 2024-08-19T10:40:41.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.942+0000 7f7238fef640 1 -- 172.21.15.89:0/1506682866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f723413fdb0 con 0x7f7234071cd0 2024-08-19T10:40:41.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7238fef640 1 -- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7234071cd0 msgr2=0x7f72340720e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:41.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7238fef640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7234071cd0 0x7f72340720e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f723413b770 tx=0x7f7224001030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:41.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f72240074a0 con 0x7f7234071cd0 2024-08-19T10:40:41.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7229ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:41.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7229ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:41.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7229ffb640 10 monclient: dump: 2024-08-19T10:40:41.961 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:41.961 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:41.962 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:41.963 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:41.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9269 (secure 0 0 0) 0x7f7224007030 con 0x7f7234071cd0 2024-08-19T10:40:41.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7229ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:41.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7229ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:41.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7234071cd0 msgr2=0x7f72340720e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7229ffb640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7234071cd0 0x7f72340720e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f723413b770 tx=0x7f7224001030 comp rx=0 tx=0).stop 2024-08-19T10:40:41.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7229ffb640 1 -- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7234071cd0 msgr2=0x7f72340720e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:41.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7229ffb640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7234071cd0 0x7f72340720e0 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f723413b770 tx=0x7f7224001030 comp rx=0 tx=0).stop 2024-08-19T10:40:41.963 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:41.963 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 171759, 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:41.964 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:41.965 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:41.966 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:41.966 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:41.966 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:41.966 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:41.966 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:41.966 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:41.966 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:41.966 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:41.966 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:41.966 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:41.967 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:41.967 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:41.967 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:41.967 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:41.967 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:41.967 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:41.967 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:41.967 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:41.968 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:41.968 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:41.968 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.968 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:41.968 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:41.968 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:41.968 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:41.968 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:41.969 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:41.969 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:41.969 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:41.969 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:41.969 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:41.969 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:41.969 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:41.969 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:41.970 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:41.970 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:41.970 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:41.970 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:41.970 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:41.970 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:41.970 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:41.970 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:41.971 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:41.971 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:41.971 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:41.971 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:41.971 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:41.971 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:41.971 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:41.972 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:41.972 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:41.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.972 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:41.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.972 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:41.972 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:41.972 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:41.973 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:41.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.973 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:41.973 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:41.973 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.973 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:41.974 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:41.974 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.974 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.974 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.974 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:41.974 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:41.974 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:41.975 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:41.975 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:41.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.975 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:41.975 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:41.975 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:41.975 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:41.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.976 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:41.976 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:41.976 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.976 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:41.976 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:41.976 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.976 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.976 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:41.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:41.978 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:41.979 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:41.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:41.981 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:41.982 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.983 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.984 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:41.985 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:41.986 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:41.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:41.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:41.989 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:41.990 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:41.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:41.991 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:41.991 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:41.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f723ba54640 1 -- 172.21.15.89:0/1506682866 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f721803be00 msgr2=0x7f721803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f723ba54640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f721803be00 0x7f721803e2c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f72340695c0 tx=0x7f722c0071c0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f723ba54640 10 monclient: shutdown 2024-08-19T10:40:41.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f723ba54640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:41.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f723ba54640 1 -- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a1a00 msgr2=0x7f7234137760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:41.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f723ba54640 1 --2- 172.21.15.89:0/1506682866 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72340a1a00 0x7f7234137760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7230000c10 tx=0x7f72300191e0 comp rx=0 tx=0).stop 2024-08-19T10:40:41.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7239ff1640 1 -- 172.21.15.89:0/1506682866 reap_dead start 2024-08-19T10:40:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f7239ff1640 1 -- 172.21.15.89:0/1506682866 reap_dead start 2024-08-19T10:40:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f723ba54640 1 -- 172.21.15.89:0/1506682866 shutdown_connections 2024-08-19T10:40:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f723ba54640 1 -- 172.21.15.89:0/1506682866 >> 172.21.15.89:0/1506682866 conn(0x7f72340a4df0 msgr2=0x7f72340a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f723ba54640 1 -- 172.21.15.89:0/1506682866 shutdown_connections 2024-08-19T10:40:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:41.946+0000 7f723ba54640 1 -- 172.21.15.89:0/1506682866 wait complete. 2024-08-19T10:40:41.993 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:41.993 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:40:41.994 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:40:41.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=53} crc {osdmap=3584284665}) 2024-08-19T10:40:41.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=56} crc {osdmap=3960050588}) 2024-08-19T10:40:41.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=53} crc {osdmap=3584284665}) 2024-08-19T10:40:41.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=55} crc {osdmap=2870110111}) 2024-08-19T10:40:41.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=53} crc {osdmap=3584284665}) 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=55} crc {osdmap=4050319749}) 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=53} crc {osdmap=3584284665}) 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=51} crc {osdmap=1671272510}) 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=53} crc {osdmap=3584284665}) 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=53} crc {osdmap=887334383}) 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=53} crc {osdmap=3584284665}) 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=57} crc {osdmap=3562452419}) 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=53} crc {osdmap=3584284665}) 2024-08-19T10:40:41.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=56} crc {osdmap=2447270008}) 2024-08-19T10:40:41.998 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:40:41.998 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=53} crc {osdmap=3584284665}) 2024-08-19T10:40:41.998 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:40:41.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=53} crc {osdmap=3441763298}) 2024-08-19T10:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.106+0000 7f9e5a468640 10 monclient: get_monmap_and_config 2024-08-19T10:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.106+0000 7f9e5a468640 10 monclient: build_initial_monmap 2024-08-19T10:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient: monmap: 2024-08-19T10:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:42.112925+0000 2024-08-19T10:40:42.112 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:42.112925+0000 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:42.113 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 1 Processor -- start 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 1 -- start start 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient: init 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient: _add_conns ranks=[8,2,4,9,10,7,5,1,0,3,6] 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e540a67d0 0x7f9e540a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient(hunting): picked mon.noname-e con 0x7f9e540a67d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e540a6bb0 0x7f9e540a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient(hunting): picked mon.noname-b con 0x7f9e540a6bb0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e54139590 0x7f9e541399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient(hunting): picked mon.noname-c con 0x7f9e54139590 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9e540a8740 con 0x7f9e540a6bb0 2024-08-19T10:40:42.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9e540a3380 con 0x7f9e54139590 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9e540a8250 con 0x7f9e540a67d0 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e5a468640 10 monclient(hunting): authenticate will time out at 1091.929391s 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e540a67d0 0x7f9e540a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e58a05640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e54139590 0x7f9e541399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e537fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e540a6bb0 0x7f9e540a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e540a67d0 0x7f9e540a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40490/0 (socket says 172.21.15.89:40490) 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 1 -- 172.21.15.89:0/1554737416 learned_addr learned my addr 172.21.15.89:0/1554737416 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e58a05640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e54139590 0x7f9e541399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54100/0 (socket says 172.21.15.89:54100) 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 10 monclient(hunting): get_auth_request con 0x7f9e540a67d0 auth_method 0 2024-08-19T10:40:42.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e58a05640 10 monclient(hunting): get_auth_request con 0x7f9e54139590 auth_method 0 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e58a05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e58a05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e58a05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e537fe640 10 monclient(hunting): get_auth_request con 0x7f9e540a6bb0 auth_method 0 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e537fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e537fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e58a05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e58a05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e58a05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 10 monclient(hunting): handle_auth_done global_id 4841 payload 274 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 1 -- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e54139590 msgr2=0x7f9e541399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 1 --2- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e54139590 0x7f9e541399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 1 -- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e540a6bb0 msgr2=0x7f9e540a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 1 --2- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e540a6bb0 0x7f9e540a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 1 monclient: found mon.noname-e 2024-08-19T10:40:42.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:42.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e53fff640 1 -- 172.21.15.89:0/1554737416 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e540a4cc0 con 0x7f9e540a67d0 2024-08-19T10:40:42.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.110+0000 7f9e58a05640 1 --2- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e54139590 0x7f9e541399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:42.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e53fff640 1 -- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e540a67d0 msgr2=0x7f9e540a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e53fff640 1 --2- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e540a67d0 0x7f9e540a5500 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9e440098d0 tx=0x7f9e4401c4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2ffdd20241d6b0c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 1 -- 172.21.15.89:0/1554737416 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e440b0070 con 0x7f9e540a67d0 2024-08-19T10:40:42.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:42.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 10 monclient: dump: 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.126 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 10 monclient: _finish_auth 0 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 5 monclient: authenticate success, global_id 4841 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 1 -- 172.21.15.89:0/1554737416 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e440aa810 con 0x7f9e540a67d0 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:42.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 1 -- 172.21.15.89:0/1554737416 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e440a9300 con 0x7f9e540a67d0 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 4 set_mon_vals no callback set 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e52ffd640 10 monclient: dump: 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 10 monclient: shutdown 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 1 -- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e540a67d0 msgr2=0x7f9e540a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 1 --2- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e540a67d0 0x7f9e540a5500 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9e440098d0 tx=0x7f9e4401c4d0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 1 -- 172.21.15.89:0/1554737416 shutdown_connections 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 1 --2- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e54139590 0x7f9e541399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 1 --2- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e540a6bb0 0x7f9e540a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 1 --2- 172.21.15.89:0/1554737416 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e540a67d0 0x7f9e540a5500 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 1 -- 172.21.15.89:0/1554737416 >> 172.21.15.89:0/1554737416 conn(0x7f9e54069ae0 msgr2=0x7f9e540abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 1 -- 172.21.15.89:0/1554737416 shutdown_connections 2024-08-19T10:40:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 1 -- 172.21.15.89:0/1554737416 wait complete. 2024-08-19T10:40:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 10 monclient: build_initial_monmap 2024-08-19T10:40:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 10 monclient: monmap: 2024-08-19T10:40:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:42.130 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 1 Processor -- start 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 1 -- start start 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.114+0000 7f9e5a468640 10 monclient: init 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient: _add_conns ranks=[1,0,7,3,5,6,2,8,4] 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e540a67d0 0x7f9e54134b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient(hunting): picked mon.noname-f con 0x7f9e540a67d0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e540a6bb0 0x7f9e5409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient(hunting): picked mon.noname-a con 0x7f9e540a6bb0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e54139590 0x7f9e5409cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient(hunting): picked mon.noname-i con 0x7f9e54139590 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54143390 con 0x7f9e540a6bb0 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54143210 con 0x7f9e540a67d0 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9e54143510 con 0x7f9e54139590 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient(hunting): authenticate will time out at 1091.936308s 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e540a6bb0 0x7f9e5409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e540a6bb0 0x7f9e5409c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42670/0 (socket says 172.21.15.89:42670) 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 1 -- 172.21.15.89:0/1494610238 learned_addr learned my addr 172.21.15.89:0/1494610238 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 10 monclient(hunting): get_auth_request con 0x7f9e540a6bb0 auth_method 0 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 10 monclient(hunting): handle_auth_done global_id 5085 payload 824 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 1 -- 172.21.15.89:0/1494610238 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e54139590 msgr2=0x7f9e5409cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e54139590 0x7f9e5409cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 1 -- 172.21.15.89:0/1494610238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e540a67d0 msgr2=0x7f9e54134b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e540a67d0 0x7f9e54134b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 1 monclient: found mon.noname-a 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 1 -- 172.21.15.89:0/1494610238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e541404c0 con 0x7f9e540a6bb0 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 1 -- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e540a6bb0 msgr2=0x7f9e5409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e537fe640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e540a6bb0 0x7f9e5409c9e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800e960 tx=0x7f9e4800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e48017070 con 0x7f9e540a6bb0 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 10 monclient: dump: 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.135 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 10 monclient: _finish_auth 0 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e48013470 con 0x7f9e540a6bb0 2024-08-19T10:40:42.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 5 monclient: authenticate success, global_id 5085 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient: _renew_subs 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 1 -- 172.21.15.89:0/1494610238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e540a7020 con 0x7f9e540a6bb0 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e48013a90 con 0x7f9e540a6bb0 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e50ff9640 10 monclient: dump: 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient: _renew_subs 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e527fc640 4 set_mon_vals no callback set 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 1 -- 172.21.15.89:0/1494610238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e540a0760 con 0x7f9e540a6bb0 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5409ef60 0x7f9e5409d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e5a468640 1 -- 172.21.15.89:0/1494610238 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9e540a5500 con 0x7f9e5409ef60 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.118+0000 7f9e53fff640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5409ef60 0x7f9e5409d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e53fff640 10 monclient: get_auth_request con 0x7f9e5409ef60 auth_method 0 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e53fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e53fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e53fff640 10 monclient(hunting): handle_auth_done global_id 4934 payload 824 2024-08-19T10:40:42.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e53fff640 1 -- 172.21.15.89:0/1494610238 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e54064590 con 0x7f9e5409ef60 2024-08-19T10:40:42.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e537fe640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e540a6bb0 0x7f9e5409c9e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800e960 tx=0x7f9e4800ee20 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f9e4800f040 osd_map(72..72 src has 1..72) 2024-08-19T10:40:42.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9e4801a070 con 0x7f9e540a6bb0 2024-08-19T10:40:42.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e4003be00 0x7f9e4003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e58a05640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e4003be00 0x7f9e4003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e58a05640 10 monclient: get_auth_request con 0x7f9e4003be00 auth_method 0 2024-08-19T10:40:42.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e53fff640 1 -- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5409ef60 msgr2=0x7f9e5409d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e53fff640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5409ef60 0x7f9e5409d460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e440ad000 tx=0x7f9e440045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e440b0040 con 0x7f9e5409ef60 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 10 monclient: dump: 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.143 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9e440093d0 con 0x7f9e5409ef60 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5409ef60 msgr2=0x7f9e5409d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5409ef60 0x7f9e5409d460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e440ad000 tx=0x7f9e440045c0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5409ef60 msgr2=0x7f9e5409d460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:42.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.122+0000 7f9e50ff9640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e5409ef60 0x7f9e5409d460 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e440ad000 tx=0x7f9e440045c0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.126+0000 7f9e58a05640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e4003be00 0x7f9e4003e2c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e540acce0 tx=0x7f9e4c00d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e5a468640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e5a468640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e541350a0 0x7f9e5409d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e5a468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e5a468640 1 -- 172.21.15.89:0/1494610238 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9e54139370 con 0x7f9e541350a0 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e541350a0 0x7f9e5409d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 10 monclient: get_auth_request con 0x7f9e541350a0 auth_method 0 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 10 monclient(hunting): handle_auth_done global_id 4943 payload 824 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 1 -- 172.21.15.89:0/1494610238 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9e5406ca20 con 0x7f9e541350a0 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 1 -- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e541350a0 msgr2=0x7f9e5409d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e53fff640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e541350a0 0x7f9e5409d9a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e44000c10 tx=0x7f9e440a8000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e440b0040 con 0x7f9e541350a0 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e50ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e50ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e50ff9640 10 monclient: dump: 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.146 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9601 (secure 0 0 0) 0x7f9e440044e0 con 0x7f9e541350a0 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e50ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e50ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:42.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e541350a0 msgr2=0x7f9e5409d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e50ff9640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e541350a0 0x7f9e5409d9a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e44000c10 tx=0x7f9e440a8000 comp rx=0 tx=0).stop 2024-08-19T10:40:42.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e541350a0 msgr2=0x7f9e5409d9a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:42.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.130+0000 7f9e50ff9640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e541350a0 0x7f9e5409d9a0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e44000c10 tx=0x7f9e440a8000 comp rx=0 tx=0).stop 2024-08-19T10:40:42.148 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:42.148 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:42.148 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:42.148 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:42.148 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:42.148 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 171959, 2024-08-19T10:40:42.148 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:42.149 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:42.150 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.151 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:42.152 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.153 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.155 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.156 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:42.158 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:42.161 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.162 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:42.163 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:42.166 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:42.168 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.169 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.170 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:40:42.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.171 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.171 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.171 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:42.171 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.171 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.171 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:42.171 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e5a468640 1 -- 172.21.15.89:0/1494610238 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e4003be00 msgr2=0x7f9e4003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e5a468640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e4003be00 0x7f9e4003e2c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e540acce0 tx=0x7f9e4c00d040 comp rx=0 tx=0).stop 2024-08-19T10:40:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e5a468640 10 monclient: shutdown 2024-08-19T10:40:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e5a468640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e5a468640 1 -- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e540a6bb0 msgr2=0x7f9e5409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e58a05640 1 -- 172.21.15.89:0/1494610238 reap_dead start 2024-08-19T10:40:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e5a468640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e540a6bb0 0x7f9e5409c9e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800e960 tx=0x7f9e4800ee20 comp rx=0 tx=0).stop 2024-08-19T10:40:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e50ff9640 1 -- 172.21.15.89:0/1494610238 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5249+0+0 (unknown 0 0 0) 0x7f9e4800f040 con 0x7f9e540a6bb0 2024-08-19T10:40:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e5a468640 1 -- 172.21.15.89:0/1494610238 shutdown_connections 2024-08-19T10:40:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e5a468640 1 --2- 172.21.15.89:0/1494610238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e540a6bb0 0x7f9e5409c9e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e5a468640 1 -- 172.21.15.89:0/1494610238 >> 172.21.15.89:0/1494610238 conn(0x7f9e54069ae0 msgr2=0x7f9e540ab820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e5a468640 1 -- 172.21.15.89:0/1494610238 shutdown_connections 2024-08-19T10:40:42.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.134+0000 7f9e5a468640 1 -- 172.21.15.89:0/1494610238 wait complete. 2024-08-19T10:40:42.174 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:42.174 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:40:42.174 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:40:42.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient: get_monmap_and_config 2024-08-19T10:40:42.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient: build_initial_monmap 2024-08-19T10:40:42.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient: monmap: 2024-08-19T10:40:42.280 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:42.280 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:42.281015+0000 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:42.281015+0000 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:42.281 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:42.282 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:42.282 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 1 Processor -- start 2024-08-19T10:40:42.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 1 -- start start 2024-08-19T10:40:42.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient: init 2024-08-19T10:40:42.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient: _add_conns ranks=[5,1,10,9,7,0,6,4,2,3,8] 2024-08-19T10:40:42.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbdc409eae0 0x7fbdc409eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient(hunting): picked mon.noname-h con 0x7fbdc409eae0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:42.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbdc4069070 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbdc409dcc0 0x7fbdc40a4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient(hunting): picked mon.noname-f con 0x7fbdc409dcc0 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc409e170 con 0x7fbdc4069070 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbdc409c840 con 0x7fbdc409dcc0 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fbdc409d8a0 con 0x7fbdc409eae0 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdcbc29640 10 monclient(hunting): authenticate will time out at 1092.097667s 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbdc409eae0 0x7fbdc409eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbdc409eae0 0x7fbdc409eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:51872/0 (socket says 172.21.15.89:51872) 2024-08-19T10:40:42.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 1 -- 172.21.15.89:0/4126509517 learned_addr learned my addr 172.21.15.89:0/4126509517 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 10 monclient(hunting): get_auth_request con 0x7fbdc409eae0 auth_method 0 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdca1c6640 1 --2- 172.21.15.89:0/4126509517 >> v2:172.21.15.190:3300/0 conn(0x7fbdc409dcc0 0x7fbdc40a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdca1c6640 10 monclient(hunting): get_auth_request con 0x7fbdc409dcc0 auth_method 0 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdca1c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdca1c6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdca1c6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdca1c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdca1c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdca1c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 10 monclient(hunting): handle_auth_done global_id 4964 payload 274 2024-08-19T10:40:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 1 -- 172.21.15.89:0/4126509517 >> v2:172.21.15.190:3300/0 conn(0x7fbdc409dcc0 msgr2=0x7fbdc40a4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 1 --2- 172.21.15.89:0/4126509517 >> v2:172.21.15.190:3300/0 conn(0x7fbdc409dcc0 0x7fbdc40a4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 1 -- 172.21.15.89:0/4126509517 >> v1:172.21.15.190:6789/0 conn(0x7fbdc4069070 legacy=0x7fbdc40a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 1 monclient: found mon.noname-h 2024-08-19T10:40:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:42.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.278+0000 7fbdc99c5640 1 -- 172.21.15.89:0/4126509517 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc40694f0 con 0x7fbdc409eae0 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc99c5640 1 -- 172.21.15.89:0/4126509517 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbdc409eae0 msgr2=0x7fbdc409eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc99c5640 1 --2- 172.21.15.89:0/4126509517 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbdc409eae0 0x7fbdc409eec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009e50 tx=0x7fbdbc01c8f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=c6270ee9c3ef27bd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 1 -- 172.21.15.89:0/4126509517 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbdbc0090d0 con 0x7fbdc409eae0 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 10 monclient: dump: 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.289 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 10 monclient: _finish_auth 0 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 10 monclient: _check_auth_tickets 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 1 -- 172.21.15.89:0/4126509517 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdbc0a5ab0 con 0x7fbdc409eae0 2024-08-19T10:40:42.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 1 -- 172.21.15.89:0/4126509517 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbdbc0a5ca0 con 0x7fbdc409eae0 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdcbc29640 5 monclient: authenticate success, global_id 4964 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdcbc29640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdcbc29640 4 set_mon_vals no callback set 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.282+0000 7fbdc89c3640 10 monclient: dump: 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.291 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient: shutdown 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 -- 172.21.15.89:0/4126509517 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbdc409eae0 msgr2=0x7fbdc409eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 --2- 172.21.15.89:0/4126509517 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbdc409eae0 0x7fbdc409eec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009e50 tx=0x7fbdbc01c8f0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 -- 172.21.15.89:0/4126509517 shutdown_connections 2024-08-19T10:40:42.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 --2- 172.21.15.89:0/4126509517 >> v2:172.21.15.190:3300/0 conn(0x7fbdc409dcc0 0x7fbdc40a4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 --2- 172.21.15.89:0/4126509517 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbdc409eae0 0x7fbdc409eec0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 -- 172.21.15.89:0/4126509517 >> 172.21.15.89:0/4126509517 conn(0x7fbdc40a4e60 msgr2=0x7fbdc40a5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:42.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 -- 172.21.15.89:0/4126509517 shutdown_connections 2024-08-19T10:40:42.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 -- 172.21.15.89:0/4126509517 wait complete. 2024-08-19T10:40:42.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient: build_initial_monmap 2024-08-19T10:40:42.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient: monmap: 2024-08-19T10:40:42.298 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:42.298 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:42.299 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 Processor -- start 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 -- start start 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient: init 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient: _add_conns ranks=[6,3,7,0,5,2,1,8,4] 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbdc409dcc0 0x7fbdc409a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient(hunting): picked mon.noname-d con 0x7fbdc409dcc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbdc409eae0 0x7fbdc413f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient(hunting): picked mon.noname-g con 0x7fbdc409eae0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbdc409a9f0 0x7fbdc413f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient(hunting): picked mon.noname-i con 0x7fbdc409a9f0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc4143d50 con 0x7fbdc409eae0 2024-08-19T10:40:42.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fbdc40a4bd0 con 0x7fbdc409dcc0 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fbdc4143ed0 con 0x7fbdc409a9f0 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdcbc29640 10 monclient(hunting): authenticate will time out at 1092.106596s 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbdc409dcc0 0x7fbdc409a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbdc409dcc0 0x7fbdc409a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54356/0 (socket says 172.21.15.89:54356) 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 1 -- 172.21.15.89:0/3297689174 learned_addr learned my addr 172.21.15.89:0/3297689174 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 10 monclient(hunting): get_auth_request con 0x7fbdc409dcc0 auth_method 0 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc91c4640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbdc409eae0 0x7fbdc413f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdca1c6640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbdc409a9f0 0x7fbdc413f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc91c4640 10 monclient(hunting): get_auth_request con 0x7fbdc409eae0 auth_method 0 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc91c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc91c4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc91c4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdca1c6640 10 monclient(hunting): get_auth_request con 0x7fbdc409a9f0 auth_method 0 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdca1c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdca1c6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdca1c6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 10 monclient(hunting): handle_auth_done global_id 4956 payload 824 2024-08-19T10:40:42.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 1 -- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbdc409a9f0 msgr2=0x7fbdc413f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbdc409a9f0 0x7fbdc413f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 1 -- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbdc409eae0 msgr2=0x7fbdc413f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbdc409eae0 0x7fbdc413f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 1 monclient: found mon.noname-d 2024-08-19T10:40:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:42.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc99c5640 1 -- 172.21.15.89:0/3297689174 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc4140d90 con 0x7fbdc409dcc0 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.286+0000 7fbdc91c4640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbdc409eae0 0x7fbdc413f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc99c5640 1 -- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbdc409dcc0 msgr2=0x7fbdc409a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc99c5640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbdc409dcc0 0x7fbdc409a4b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc01c720 tx=0x7fbdbc019690 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbdbc0bb020 con 0x7fbdc409dcc0 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 10 monclient: dump: 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.306 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:42.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdcbc29640 5 monclient: authenticate success, global_id 4956 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdbc003dc0 con 0x7fbdc409dcc0 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdcbc29640 10 monclient: _renew_subs 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdcbc29640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdcbc29640 1 -- 172.21.15.89:0/3297689174 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc41cf380 con 0x7fbdc409dcc0 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbdbc0b6440 con 0x7fbdc409dcc0 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc37fe640 4 set_mon_vals no callback set 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 10 monclient: dump: 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.308 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdcbc29640 10 monclient: _renew_subs 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdcbc29640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdcbc29640 1 -- 172.21.15.89:0/3297689174 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc409f550 con 0x7fbdc409dcc0 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdcbc29640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdcbc29640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc4069070 0x7fbdc413feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdcbc29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdcbc29640 1 -- 172.21.15.89:0/3297689174 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbdc409eec0 con 0x7fbdc4069070 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbdbc0be020 con 0x7fbdc409dcc0 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc91c4640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc4069070 0x7fbdc413feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc91c4640 10 monclient: get_auth_request con 0x7fbdc4069070 auth_method 0 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc91c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc91c4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc91c4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc91c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc91c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc91c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbdb003be30 0x7fbdb003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(72..72 src has 1..72) ==== 5249+0+0 (secure 0 0 0) 0x7fbdbc0e9aa0 con 0x7fbdc409dcc0 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc91c4640 10 monclient(hunting): handle_auth_done global_id 5026 payload 824 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdc91c4640 1 -- 172.21.15.89:0/3297689174 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbdc4064590 con 0x7fbdc4069070 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdca1c6640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbdb003be30 0x7fbdb003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.290+0000 7fbdca1c6640 10 monclient: get_auth_request con 0x7fbdb003be30 auth_method 0 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc91c4640 1 -- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc4069070 msgr2=0x7fbdc413feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc91c4640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc4069070 0x7fbdc413feb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbdc406fec0 tx=0x7fbdb800e980 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbdb8013c70 con 0x7fbdc4069070 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:42.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc27fc640 10 monclient: dump: 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.312 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.313 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.313 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbdb8013800 con 0x7fbdc4069070 2024-08-19T10:40:42.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:42.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc27fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:42.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc4069070 msgr2=0x7fbdc413feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc27fc640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc4069070 0x7fbdc413feb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbdc406fec0 tx=0x7fbdb800e980 comp rx=0 tx=0).stop 2024-08-19T10:40:42.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc4069070 msgr2=0x7fbdc413feb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:42.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdc27fc640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc4069070 0x7fbdc413feb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.294+0000 7fbdca1c6640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbdb003be30 0x7fbdb003e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4141400 tx=0x7fbdb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdcbc29640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:42.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdcbc29640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc41403f0 0x7fbdc413c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdcbc29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdcbc29640 1 -- 172.21.15.89:0/3297689174 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbdc4140800 con 0x7fbdc41403f0 2024-08-19T10:40:42.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdc91c4640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc41403f0 0x7fbdc413c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdc91c4640 10 monclient: get_auth_request con 0x7fbdc41403f0 auth_method 0 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdc91c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdc91c4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdc91c4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdc91c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdc91c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.298+0000 7fbdc91c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc91c4640 10 monclient(hunting): handle_auth_done global_id 5035 payload 824 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc91c4640 1 -- 172.21.15.89:0/3297689174 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbdc40a2700 con 0x7fbdc41403f0 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc91c4640 1 -- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc41403f0 msgr2=0x7fbdc413c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc91c4640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc41403f0 0x7fbdc413c720 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbdc41d1f50 tx=0x7fbdb8003770 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbdb8006ce0 con 0x7fbdc41403f0 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc27fc640 10 monclient: dump: 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.317 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.318 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.318 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.318 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7fbdb8006ce0 con 0x7fbdc41403f0 2024-08-19T10:40:42.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:42.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc27fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:42.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc41403f0 msgr2=0x7fbdc413c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc27fc640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc41403f0 0x7fbdc413c720 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbdc41d1f50 tx=0x7fbdb8003770 comp rx=0 tx=0).stop 2024-08-19T10:40:42.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc27fc640 1 -- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc41403f0 msgr2=0x7fbdc413c720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:42.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdc27fc640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbdc41403f0 0x7fbdc413c720 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.318 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 172119, 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:42.319 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.320 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.321 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:42.322 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.323 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:42.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.325 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:42.327 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:42.330 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.331 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:42.332 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.335 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.337 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.337 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:42.337 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.337 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.337 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:42.337 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.337 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.338 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.339 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.340 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.341 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.341 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:42.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.341 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.341 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:42.341 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:42.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdcbc29640 1 -- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbdb003be30 msgr2=0x7fbdb003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdcbc29640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbdb003be30 0x7fbdb003e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4141400 tx=0x7fbdb4008040 comp rx=0 tx=0).stop 2024-08-19T10:40:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdcbc29640 10 monclient: shutdown 2024-08-19T10:40:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdcbc29640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdcbc29640 1 -- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbdc409dcc0 msgr2=0x7fbdc409a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdcbc29640 1 --2- 172.21.15.89:0/3297689174 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbdc409dcc0 0x7fbdc409a4b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc01c720 tx=0x7fbdbc019690 comp rx=0 tx=0).stop 2024-08-19T10:40:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdca1c6640 1 -- 172.21.15.89:0/3297689174 reap_dead start 2024-08-19T10:40:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdca1c6640 1 -- 172.21.15.89:0/3297689174 reap_dead start 2024-08-19T10:40:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdcbc29640 1 -- 172.21.15.89:0/3297689174 shutdown_connections 2024-08-19T10:40:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdcbc29640 1 -- 172.21.15.89:0/3297689174 >> 172.21.15.89:0/3297689174 conn(0x7fbdc40a4e60 msgr2=0x7fbdc40a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdcbc29640 1 -- 172.21.15.89:0/3297689174 shutdown_connections 2024-08-19T10:40:42.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.302+0000 7fbdcbc29640 1 -- 172.21.15.89:0/3297689174 wait complete. 2024-08-19T10:40:42.344 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:42.344 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:40:42.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:40:42.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient: get_monmap_and_config 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient: build_initial_monmap 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient: monmap: 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:42.457412+0000 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:42.457412+0000 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:42.457 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:42.458 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:42.458 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:42.458 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:42.458 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:42.458 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:42.458 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:42.458 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:42.458 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 1 Processor -- start 2024-08-19T10:40:42.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 1 -- start start 2024-08-19T10:40:42.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient: init 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient: _add_conns ranks=[6,0,3,8,1,5,10,2,9,7,4] 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 0x7f45240a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient(hunting): picked mon.noname-d con 0x7f45240a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f45240a7180 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45240a7560 0x7f4524139d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient(hunting): picked mon.noname-g con 0x7f45240a7560 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45240a33d0 con 0x7f45240a7180 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f45240a8190 con 0x7f45240a7560 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f45240a79f0 con 0x7f45240a6860 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452b6e2640 10 monclient(hunting): authenticate will time out at 1092.273991s 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 0x7f45240a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f4528c7d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f45240a7180 0x7f4524139620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59294/0 (socket says 172.21.15.89:59294) 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f4528c7d640 1 -- 172.21.15.89:0/1116286758 learned_addr learned my addr 172.21.15.89:0/1116286758 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 0x7f45240a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54358/0 (socket says 172.21.15.89:54358) 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 10 monclient(hunting): get_auth_request con 0x7f45240a6860 auth_method 0 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f4529c7f640 1 --2- 172.21.15.89:0/1116286758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45240a7560 0x7f4524139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f4529c7f640 10 monclient(hunting): get_auth_request con 0x7f45240a7560 auth_method 0 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f4529c7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f4529c7f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f4529c7f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 10 monclient(hunting): handle_auth_done global_id 4965 payload 274 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 1 -- 172.21.15.89:0/1116286758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45240a7560 msgr2=0x7f4524139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 1 --2- 172.21.15.89:0/1116286758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45240a7560 0x7f4524139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 1 -- 172.21.15.89:0/1116286758 >> v1:172.21.15.89:6789/0 conn(0x7f45240a7180 legacy=0x7f4524139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 1 monclient: found mon.noname-d 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:42.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.454+0000 7f452947e640 1 -- 172.21.15.89:0/1116286758 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45240a2ec0 con 0x7f45240a6860 2024-08-19T10:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452947e640 1 -- 172.21.15.89:0/1116286758 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 msgr2=0x7f45240a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452947e640 1 --2- 172.21.15.89:0/1116286758 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 0x7f45240a6c40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f45200097f0 tx=0x7f452001c3a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=40baaff9be04d39d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 1 -- 172.21.15.89:0/1116286758 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f45200b0070 con 0x7f45240a6860 2024-08-19T10:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 10 monclient: dump: 2024-08-19T10:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.470 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 10 monclient: _finish_auth 0 2024-08-19T10:40:42.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 1 -- 172.21.15.89:0/1116286758 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45200aa810 con 0x7f45240a6860 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 5 monclient: authenticate success, global_id 4965 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 4 set_mon_vals no callback set 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 1 -- 172.21.15.89:0/1116286758 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f45200a9300 con 0x7f45240a6860 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f4517fff640 10 monclient: dump: 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.472 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.473 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 10 monclient: shutdown 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 1 -- 172.21.15.89:0/1116286758 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 msgr2=0x7f45240a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 1 --2- 172.21.15.89:0/1116286758 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 0x7f45240a6c40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f45200097f0 tx=0x7f452001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 1 -- 172.21.15.89:0/1116286758 shutdown_connections 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 1 --2- 172.21.15.89:0/1116286758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45240a7560 0x7f4524139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 1 --2- 172.21.15.89:0/1116286758 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 0x7f45240a6c40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 1 -- 172.21.15.89:0/1116286758 >> 172.21.15.89:0/1116286758 conn(0x7f4524074b40 msgr2=0x7f45240abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 1 -- 172.21.15.89:0/1116286758 shutdown_connections 2024-08-19T10:40:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.458+0000 7f452b6e2640 1 -- 172.21.15.89:0/1116286758 wait complete. 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient: build_initial_monmap 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient: monmap: 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 1 Processor -- start 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 1 -- start start 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient: init 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient: _add_conns ranks=[6,7,1,4,2,5,3,8,0] 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 0x7f45241d1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient(hunting): picked mon.noname-d con 0x7f45240a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f45240a7180 0x7f4524139440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient(hunting): picked mon.noname-i con 0x7f45240a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45240a7560 0x7f452413e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient(hunting): picked mon.noname-f con 0x7f45240a7560 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f452413a3f0 con 0x7f45240a7560 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f452413a270 con 0x7f45240a6860 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f45241440a0 con 0x7f45240a7180 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452b6e2640 10 monclient(hunting): authenticate will time out at 1092.281245s 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 0x7f45241d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 0x7f45241d1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54362/0 (socket says 172.21.15.89:54362) 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 1 -- 172.21.15.89:0/3618582465 learned_addr learned my addr 172.21.15.89:0/3618582465 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 10 monclient(hunting): get_auth_request con 0x7f45240a6860 auth_method 0 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f4529c7f640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45240a7560 0x7f452413e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f4528c7d640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f45240a7180 0x7f4524139440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f4529c7f640 10 monclient(hunting): get_auth_request con 0x7f45240a7560 auth_method 0 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f4529c7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f4529c7f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f4529c7f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f4528c7d640 10 monclient(hunting): get_auth_request con 0x7f45240a7180 auth_method 0 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f4528c7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f4528c7d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f4528c7d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 10 monclient(hunting): handle_auth_done global_id 4974 payload 824 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 1 -- 172.21.15.89:0/3618582465 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f45240a7180 msgr2=0x7f4524139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f45240a7180 0x7f4524139440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 1 -- 172.21.15.89:0/3618582465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45240a7560 msgr2=0x7f452413e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45240a7560 0x7f452413e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 1 monclient: found mon.noname-d 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.462+0000 7f452947e640 1 -- 172.21.15.89:0/3618582465 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45240a2190 con 0x7f45240a6860 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452947e640 1 -- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 msgr2=0x7f45241d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452947e640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 0x7f45241d1190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4520009710 tx=0x7f4520019140 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f45200b0040 con 0x7f45240a6860 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: dump: 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45200b9070 con 0x7f45240a6860 2024-08-19T10:40:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452b6e2640 5 monclient: authenticate success, global_id 4974 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452b6e2640 10 monclient: _renew_subs 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452b6e2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452b6e2640 1 -- 172.21.15.89:0/3618582465 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4524123af0 con 0x7f45240a6860 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f45200044b0 con 0x7f45240a6860 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: dump: 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452b6e2640 10 monclient: _renew_subs 2024-08-19T10:40:42.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f45177fe640 4 set_mon_vals no callback set 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452b6e2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452b6e2640 1 -- 172.21.15.89:0/3618582465 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f45241225d0 con 0x7f45240a6860 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452b6e2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452b6e2640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4524139980 0x7f452413e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452b6e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f452b6e2640 1 -- 172.21.15.89:0/3618582465 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f45240ac060 con 0x7f4524139980 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4524139980 0x7f452413e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 10 monclient: get_auth_request con 0x7f4524139980 auth_method 0 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4520004040 con 0x7f45240a6860 2024-08-19T10:40:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 10 monclient(hunting): handle_auth_done global_id 4983 payload 824 2024-08-19T10:40:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 1 -- 172.21.15.89:0/3618582465 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45240a23c0 con 0x7f4524139980 2024-08-19T10:40:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f450803be00 0x7f450803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(72..72 src has 1..72) ==== 5249+0+0 (secure 0 0 0) 0x7f45200a4e00 con 0x7f45240a6860 2024-08-19T10:40:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4529c7f640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f450803be00 0x7f450803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4529c7f640 10 monclient: get_auth_request con 0x7f450803be00 auth_method 0 2024-08-19T10:40:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 1 -- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4524139980 msgr2=0x7f452413e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4528c7d640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4524139980 0x7f452413e940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4524140800 tx=0x7f451800c4b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4518017020 con 0x7f4524139980 2024-08-19T10:40:42.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.466+0000 7f4515ffb640 10 monclient: dump: 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.488 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.470+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f451800cdf0 con 0x7f4524139980 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.470+0000 7f4515ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.470+0000 7f4515ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.470+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4524139980 msgr2=0x7f452413e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.470+0000 7f4515ffb640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4524139980 0x7f452413e940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4524140800 tx=0x7f451800c4b0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.470+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4524139980 msgr2=0x7f452413e940 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.470+0000 7f4515ffb640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4524139980 0x7f452413e940 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.470+0000 7f4529c7f640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f450803be00 0x7f450803e2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f451c00d8d0 tx=0x7f451c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f452b6e2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f452b6e2640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f452413ee80 0x7f452413a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f452b6e2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f452b6e2640 1 -- 172.21.15.89:0/3618582465 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f452413f290 con 0x7f452413ee80 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f4528c7d640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f452413ee80 0x7f452413a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f4528c7d640 10 monclient: get_auth_request con 0x7f452413ee80 auth_method 0 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f4528c7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f4528c7d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f4528c7d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f4528c7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f4528c7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f4528c7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f4528c7d640 10 monclient(hunting): handle_auth_done global_id 4992 payload 824 2024-08-19T10:40:42.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.474+0000 7f4528c7d640 1 -- 172.21.15.89:0/3618582465 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f45240ac390 con 0x7f452413ee80 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4528c7d640 1 -- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f452413ee80 msgr2=0x7f452413a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4528c7d640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f452413ee80 0x7f452413a630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4518018000 tx=0x7f4518004cd0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4518017040 con 0x7f452413ee80 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4515ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4515ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4515ffb640 10 monclient: dump: 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.491 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f451801e020 con 0x7f452413ee80 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4515ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4515ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f452413ee80 msgr2=0x7f452413a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4515ffb640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f452413ee80 0x7f452413a630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4518018000 tx=0x7f4518004cd0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4515ffb640 1 -- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f452413ee80 msgr2=0x7f452413a630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:42.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4515ffb640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f452413ee80 0x7f452413a630 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4518018000 tx=0x7f4518004cd0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.493 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:42.493 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:42.493 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:42.493 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:42.493 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:42.493 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 172295, 2024-08-19T10:40:42.493 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:42.494 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.495 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.496 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:42.497 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.498 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.502 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.504 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.505 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.506 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.508 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:42.509 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.509 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:42.509 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.509 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.509 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.510 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.511 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.512 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:42.513 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.514 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:42.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:42.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.517 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.517 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.517 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:40:42.517 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.517 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.517 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:42.517 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:42.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f452b6e2640 1 -- 172.21.15.89:0/3618582465 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f450803be00 msgr2=0x7f450803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f452b6e2640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f450803be00 0x7f450803e2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f451c00d8d0 tx=0x7f451c008040 comp rx=0 tx=0).stop 2024-08-19T10:40:42.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f452b6e2640 10 monclient: shutdown 2024-08-19T10:40:42.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f452b6e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:42.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f452b6e2640 1 -- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 msgr2=0x7f45241d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f452b6e2640 1 --2- 172.21.15.89:0/3618582465 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f45240a6860 0x7f45241d1190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4520009710 tx=0x7f4520019140 comp rx=0 tx=0).stop 2024-08-19T10:40:42.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4529c7f640 1 -- 172.21.15.89:0/3618582465 reap_dead start 2024-08-19T10:40:42.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f4529c7f640 1 -- 172.21.15.89:0/3618582465 reap_dead start 2024-08-19T10:40:42.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f452b6e2640 1 -- 172.21.15.89:0/3618582465 shutdown_connections 2024-08-19T10:40:42.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f452b6e2640 1 -- 172.21.15.89:0/3618582465 >> 172.21.15.89:0/3618582465 conn(0x7f4524074b40 msgr2=0x7f452409f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:42.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f452b6e2640 1 -- 172.21.15.89:0/3618582465 shutdown_connections 2024-08-19T10:40:42.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.478+0000 7f452b6e2640 1 -- 172.21.15.89:0/3618582465 wait complete. 2024-08-19T10:40:42.519 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:42.519 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:40:42.520 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:40:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.626+0000 7fa964a7f640 10 monclient: get_monmap_and_config 2024-08-19T10:40:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.626+0000 7fa964a7f640 10 monclient: build_initial_monmap 2024-08-19T10:40:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.626+0000 7fa964a7f640 10 monclient: monmap: 2024-08-19T10:40:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:42.632776+0000 2024-08-19T10:40:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:42.632776+0000 2024-08-19T10:40:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.632 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:42.633 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:42.634 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 1 Processor -- start 2024-08-19T10:40:42.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 1 -- start start 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 10 monclient: init 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 10 monclient: _add_conns ranks=[8,3,1,6,5,0,4,7,2,10,9] 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa9601378c0 0x7fa960137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 10 monclient(hunting): picked mon.noname-e con 0x7fa9601378c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9601381e0 0x7fa9600a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 10 monclient(hunting): picked mon.noname-g con 0x7fa9601381e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa960072250 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa960074010 con 0x7fa960072250 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa9600a37b0 con 0x7fa9601381e0 2024-08-19T10:40:42.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa9600726d0 con 0x7fa9601378c0 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa964a7f640 10 monclient(hunting): authenticate will time out at 1092.449316s 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa9601378c0 0x7fa960137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa9601378c0 0x7fa960137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40492/0 (socket says 172.21.15.89:40492) 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 1 -- 172.21.15.89:0/3682753553 learned_addr learned my addr 172.21.15.89:0/3682753553 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 10 monclient(hunting): get_auth_request con 0x7fa9601378c0 auth_method 0 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95dd9b640 1 --2- 172.21.15.89:0/3682753553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9601381e0 0x7fa9600a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95dd9b640 10 monclient(hunting): get_auth_request con 0x7fa9601381e0 auth_method 0 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 10 monclient(hunting): handle_auth_done global_id 4850 payload 274 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 1 -- 172.21.15.89:0/3682753553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9601381e0 msgr2=0x7fa9600a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 1 --2- 172.21.15.89:0/3682753553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9601381e0 0x7fa9600a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 1 -- 172.21.15.89:0/3682753553 >> v1:172.21.15.190:6789/0 conn(0x7fa960072250 legacy=0x7fa9600a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 1 monclient: found mon.noname-e 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:42.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.630+0000 7fa95e59c640 1 -- 172.21.15.89:0/3682753553 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9600a4300 con 0x7fa9601378c0 2024-08-19T10:40:42.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95e59c640 1 -- 172.21.15.89:0/3682753553 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa9601378c0 msgr2=0x7fa960137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95e59c640 1 --2- 172.21.15.89:0/3682753553 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa9601378c0 0x7fa960137ca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa9540098a0 tx=0x7fa95401c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6840e2585696289e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 1 -- 172.21.15.89:0/3682753553 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa9540b0070 con 0x7fa9601378c0 2024-08-19T10:40:42.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 10 monclient: dump: 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.643 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 10 monclient: _finish_auth 0 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 5 monclient: authenticate success, global_id 4850 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 1 -- 172.21.15.89:0/3682753553 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9540aa810 con 0x7fa9601378c0 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:42.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 4 set_mon_vals no callback set 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 1 -- 172.21.15.89:0/3682753553 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa9540a9300 con 0x7fa9601378c0 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa95d59a640 10 monclient: dump: 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.645 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.646 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.646 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.646 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.646 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.646 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.646 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.646 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 10 monclient: shutdown 2024-08-19T10:40:42.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:42.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 1 -- 172.21.15.89:0/3682753553 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa9601378c0 msgr2=0x7fa960137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 1 --2- 172.21.15.89:0/3682753553 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa9601378c0 0x7fa960137ca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa9540098a0 tx=0x7fa95401c440 comp rx=0 tx=0).stop 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 1 -- 172.21.15.89:0/3682753553 shutdown_connections 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 1 --2- 172.21.15.89:0/3682753553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9601381e0 0x7fa9600a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 1 --2- 172.21.15.89:0/3682753553 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa9601378c0 0x7fa960137ca0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 1 -- 172.21.15.89:0/3682753553 >> 172.21.15.89:0/3682753553 conn(0x7fa9600aa240 msgr2=0x7fa9600aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 1 -- 172.21.15.89:0/3682753553 shutdown_connections 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 1 -- 172.21.15.89:0/3682753553 wait complete. 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 10 monclient: build_initial_monmap 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 10 monclient: monmap: 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:42.647 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 1 Processor -- start 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 1 -- start start 2024-08-19T10:40:42.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.634+0000 7fa964a7f640 10 monclient: init 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient: _add_conns ranks=[8,1,6,0,7,5,2,3,4] 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960072250 0x7fa9601cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient(hunting): picked mon.noname-e con 0x7fa960072250 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9601378c0 0x7fa960136780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient(hunting): picked mon.noname-f con 0x7fa9601378c0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa9601381e0 0x7fa96013b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient(hunting): picked mon.noname-d con 0x7fa9601381e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa960141c00 con 0x7fa9601378c0 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa960141a80 con 0x7fa9601381e0 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa960141d80 con 0x7fa960072250 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient(hunting): authenticate will time out at 1092.456373s 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960072250 0x7fa9601cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960072250 0x7fa9601cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40506/0 (socket says 172.21.15.89:40506) 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 1 -- 172.21.15.89:0/2037714596 learned_addr learned my addr 172.21.15.89:0/2037714596 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 10 monclient(hunting): get_auth_request con 0x7fa960072250 auth_method 0 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95ed9d640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa9601381e0 0x7fa96013b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95ed9d640 10 monclient(hunting): get_auth_request con 0x7fa9601381e0 auth_method 0 2024-08-19T10:40:42.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95dd9b640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9601378c0 0x7fa960136780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 10 monclient(hunting): handle_auth_done global_id 4859 payload 824 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 1 -- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa9601381e0 msgr2=0x7fa96013b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa9601381e0 0x7fa96013b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 1 -- 172.21.15.89:0/2037714596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9601378c0 msgr2=0x7fa960136780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9601378c0 0x7fa960136780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 1 monclient: found mon.noname-e 2024-08-19T10:40:42.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:42.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 1 -- 172.21.15.89:0/2037714596 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9601cfd90 con 0x7fa960072250 2024-08-19T10:40:42.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95ed9d640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa9601381e0 0x7fa96013b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:42.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95dd9b640 10 monclient: get_auth_request con 0x7fa9601378c0 auth_method 0 2024-08-19T10:40:42.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95dd9b640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9601378c0 0x7fa960136780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:42.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 1 -- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960072250 msgr2=0x7fa9601cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95e59c640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960072250 0x7fa9601cee70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9540ae040 tx=0x7fa9540191e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa9540b9070 con 0x7fa960072250 2024-08-19T10:40:42.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:42.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 10 monclient: dump: 2024-08-19T10:40:42.657 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.658 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 10 monclient: _finish_auth 0 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 5 monclient: authenticate success, global_id 4859 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9540044b0 con 0x7fa960072250 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa9540b0040 con 0x7fa960072250 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:42.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa95cd99640 4 set_mon_vals no callback set 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa94effd640 10 monclient: dump: 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.660 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient: _renew_subs 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 1 -- 172.21.15.89:0/2037714596 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa960120590 con 0x7fa960072250 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient: _renew_subs 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.638+0000 7fa964a7f640 1 -- 172.21.15.89:0/2037714596 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fa960120870 con 0x7fa960072250 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa964a7f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa964a7f640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960136cc0 0x7fa96002d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa964a7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa964a7f640 1 -- 172.21.15.89:0/2037714596 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa960137ca0 con 0x7fa960136cc0 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960136cc0 0x7fa96002d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 10 monclient: get_auth_request con 0x7fa960136cc0 auth_method 0 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa954004040 con 0x7fa960072250 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 10 monclient(hunting): handle_auth_done global_id 4868 payload 824 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 1 -- 172.21.15.89:0/2037714596 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa960064590 con 0x7fa960136cc0 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa94003be00 0x7fa94003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(72..72 src has 1..72) ==== 5249+0+0 (secure 0 0 0) 0x7fa9540e8190 con 0x7fa960072250 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95ed9d640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa94003be00 0x7fa94003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95ed9d640 10 monclient: get_auth_request con 0x7fa94003be00 auth_method 0 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 1 -- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960136cc0 msgr2=0x7fa96002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa95dd9b640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960136cc0 0x7fa96002d1d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e710 tx=0x7fa95000ebd0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa950017020 con 0x7fa960136cc0 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 10 monclient: dump: 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.663 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.664 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa950004030 con 0x7fa960136cc0 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960136cc0 msgr2=0x7fa96002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960136cc0 0x7fa96002d1d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e710 tx=0x7fa95000ebd0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960136cc0 msgr2=0x7fa96002d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.642+0000 7fa94effd640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960136cc0 0x7fa96002d1d0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e710 tx=0x7fa95000ebd0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.646+0000 7fa95ed9d640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa94003be00 0x7fa94003e2c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa960072660 tx=0x7fa958003d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa964a7f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa964a7f640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa96002d710 0x7fa96002db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa964a7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa964a7f640 1 -- 172.21.15.89:0/2037714596 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa9600a9920 con 0x7fa96002d710 2024-08-19T10:40:42.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa95dd9b640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa96002d710 0x7fa96002db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa95dd9b640 10 monclient: get_auth_request con 0x7fa96002d710 auth_method 0 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa95dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa95dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa95dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa95dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa95dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa95dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa95dd9b640 10 monclient(hunting): handle_auth_done global_id 4877 payload 824 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.650+0000 7fa95dd9b640 1 -- 172.21.15.89:0/2037714596 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa9600a0100 con 0x7fa96002d710 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa95dd9b640 1 -- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa96002d710 msgr2=0x7fa96002db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa95dd9b640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa96002d710 0x7fa96002db20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e710 tx=0x7fa9500046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa950017040 con 0x7fa96002d710 2024-08-19T10:40:42.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa94effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa94effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa94effd640 10 monclient: dump: 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.667 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7fa95001e020 con 0x7fa96002d710 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa94effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa94effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa96002d710 msgr2=0x7fa96002db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa94effd640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa96002d710 0x7fa96002db20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e710 tx=0x7fa9500046d0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa94effd640 1 -- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa96002d710 msgr2=0x7fa96002db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:42.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa94effd640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa96002d710 0x7fa96002db20 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e710 tx=0x7fa9500046d0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 172447, 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:42.669 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.670 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.671 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:42.672 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:42.673 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.674 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:42.677 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:42.678 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.679 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:42.681 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:42.682 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.683 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.685 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.686 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:42.688 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.689 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.690 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:42.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.692 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.692 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.692 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:40:42.692 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.692 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.692 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:42.692 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:42.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa964a7f640 1 -- 172.21.15.89:0/2037714596 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa94003be00 msgr2=0x7fa94003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa964a7f640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa94003be00 0x7fa94003e2c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa960072660 tx=0x7fa958003d50 comp rx=0 tx=0).stop 2024-08-19T10:40:42.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa964a7f640 10 monclient: shutdown 2024-08-19T10:40:42.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa964a7f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:42.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa964a7f640 1 -- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960072250 msgr2=0x7fa9601cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa964a7f640 1 --2- 172.21.15.89:0/2037714596 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa960072250 0x7fa9601cee70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9540ae040 tx=0x7fa9540191e0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa95ed9d640 1 -- 172.21.15.89:0/2037714596 reap_dead start 2024-08-19T10:40:42.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa95ed9d640 1 -- 172.21.15.89:0/2037714596 reap_dead start 2024-08-19T10:40:42.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa964a7f640 1 -- 172.21.15.89:0/2037714596 shutdown_connections 2024-08-19T10:40:42.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa964a7f640 1 -- 172.21.15.89:0/2037714596 >> 172.21.15.89:0/2037714596 conn(0x7fa9600aa240 msgr2=0x7fa9600a1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:42.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa964a7f640 1 -- 172.21.15.89:0/2037714596 shutdown_connections 2024-08-19T10:40:42.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.654+0000 7fa964a7f640 1 -- 172.21.15.89:0/2037714596 wait complete. 2024-08-19T10:40:42.695 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:42.695 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:40:42.695 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:40:42.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.802+0000 7fba612b4640 10 monclient: get_monmap_and_config 2024-08-19T10:40:42.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.802+0000 7fba612b4640 10 monclient: build_initial_monmap 2024-08-19T10:40:42.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.802+0000 7fba612b4640 10 monclient: monmap: 2024-08-19T10:40:42.808 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:42.808 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.808 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:42.808888+0000 2024-08-19T10:40:42.808 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:42.808888+0000 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:42.809 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 1 Processor -- start 2024-08-19T10:40:42.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 1 -- start start 2024-08-19T10:40:42.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 10 monclient: init 2024-08-19T10:40:42.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 10 monclient: _add_conns ranks=[5,1,0,7,4,9,6,8,2,3,10] 2024-08-19T10:40:42.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fba5c0a5270 0x7fba5c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 10 monclient(hunting): picked mon.noname-h con 0x7fba5c0a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:42.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fba5c0a6d30 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:42.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fba5c0a7110 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba5c0a8710 con 0x7fba5c0a7110 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba5c13a3d0 con 0x7fba5c0a6d30 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fba5c0a5a40 con 0x7fba5c0a5270 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba612b4640 10 monclient(hunting): authenticate will time out at 1092.625447s 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5b7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fba5c0a7110 0x7fba5c139cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59308/0 (socket says 172.21.15.89:59308) 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5b7fe640 1 -- 172.21.15.89:0/3859872602 learned_addr learned my addr 172.21.15.89:0/3859872602 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 1 --2- 172.21.15.89:0/3859872602 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fba5c0a5270 0x7fba5c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 10 monclient(hunting): get_auth_request con 0x7fba5c0a5270 auth_method 0 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 10 monclient(hunting): handle_auth_done global_id 4973 payload 274 2024-08-19T10:40:42.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 1 -- 172.21.15.89:0/3859872602 >> v1:172.21.15.190:6789/0 conn(0x7fba5c0a6d30 legacy=0x7fba5c1395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 1 -- 172.21.15.89:0/3859872602 >> v1:172.21.15.89:6789/0 conn(0x7fba5c0a7110 legacy=0x7fba5c139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 1 monclient: found mon.noname-h 2024-08-19T10:40:42.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:42.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.806+0000 7fba5affd640 1 -- 172.21.15.89:0/3859872602 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5c0a81d0 con 0x7fba5c0a5270 2024-08-19T10:40:42.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.810+0000 7fba5affd640 1 -- 172.21.15.89:0/3859872602 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fba5c0a5270 msgr2=0x7fba5c0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.810+0000 7fba5affd640 1 --2- 172.21.15.89:0/3859872602 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fba5c0a5270 0x7fba5c0a67f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0097f0 tx=0x7fba4c01c1c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=7c2e36830b42927d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 1 -- 172.21.15.89:0/3859872602 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fba4c0b0070 con 0x7fba5c0a5270 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 10 monclient: dump: 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.817 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 10 monclient: _finish_auth 0 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 1 -- 172.21.15.89:0/3859872602 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba4c0aa810 con 0x7fba5c0a5270 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:42.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 1 -- 172.21.15.89:0/3859872602 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fba4c0aae30 con 0x7fba5c0a5270 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba59ffb640 10 monclient: dump: 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.819 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.820 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.820 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.820 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.820 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.820 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.820 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 5 monclient: authenticate success, global_id 4973 2024-08-19T10:40:42.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:42.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 4 set_mon_vals no callback set 2024-08-19T10:40:42.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient: shutdown 2024-08-19T10:40:42.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:42.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 -- 172.21.15.89:0/3859872602 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fba5c0a5270 msgr2=0x7fba5c0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 --2- 172.21.15.89:0/3859872602 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fba5c0a5270 0x7fba5c0a67f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0097f0 tx=0x7fba4c01c1c0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 -- 172.21.15.89:0/3859872602 shutdown_connections 2024-08-19T10:40:42.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 --2- 172.21.15.89:0/3859872602 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fba5c0a5270 0x7fba5c0a67f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 -- 172.21.15.89:0/3859872602 >> 172.21.15.89:0/3859872602 conn(0x7fba5c069a80 msgr2=0x7fba5c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:42.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 -- 172.21.15.89:0/3859872602 shutdown_connections 2024-08-19T10:40:42.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 -- 172.21.15.89:0/3859872602 wait complete. 2024-08-19T10:40:42.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient: build_initial_monmap 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient: monmap: 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:42.828 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 Processor -- start 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 -- start start 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient: init 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient: _add_conns ranks=[5,3,2,0,7,4,8,6,1] 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fba5c0a5270 0x7fba5c138b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient(hunting): picked mon.noname-h con 0x7fba5c0a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:42.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba5c0a6d30 0x7fba5c1390b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient(hunting): picked mon.noname-g con 0x7fba5c0a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5c0a7110 0x7fba5c13f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient(hunting): picked mon.noname-b con 0x7fba5c0a7110 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fba5c0a5650 con 0x7fba5c0a7110 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fba5c0a7570 con 0x7fba5c0a6d30 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fba5c0a57d0 con 0x7fba5c0a5270 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba612b4640 10 monclient(hunting): authenticate will time out at 1092.634825s 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5c0a7110 0x7fba5c13f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5c0a7110 0x7fba5c13f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43068/0 (socket says 172.21.15.89:43068) 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 1 -- 172.21.15.89:0/3767400817 learned_addr learned my addr 172.21.15.89:0/3767400817 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 10 monclient(hunting): get_auth_request con 0x7fba5c0a7110 auth_method 0 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5affd640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fba5c0a5270 0x7fba5c138b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5a7fc640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba5c0a6d30 0x7fba5c1390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5a7fc640 10 monclient(hunting): get_auth_request con 0x7fba5c0a6d30 auth_method 0 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5affd640 10 monclient(hunting): get_auth_request con 0x7fba5c0a5270 auth_method 0 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 10 monclient(hunting): handle_auth_done global_id 4952 payload 824 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fba5c0a5270 msgr2=0x7fba5c138b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fba5c0a5270 0x7fba5c138b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba5c0a6d30 msgr2=0x7fba5c1390b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba5c0a6d30 0x7fba5c1390b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 1 monclient: found mon.noname-b 2024-08-19T10:40:42.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.814+0000 7fba5b7fe640 1 -- 172.21.15.89:0/3767400817 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5c141d30 con 0x7fba5c0a7110 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5c0a7110 msgr2=0x7fba5c13f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5c0a7110 0x7fba5c13f370 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fba5400caf0 tx=0x7fba5400a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5c0a7110 msgr2=0x7fba5c13f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5c0a7110 msgr2=0x7fba5c13f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5c0a7110 0x7fba5c13f370 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fba5400caf0 tx=0x7fba5400a060 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5c0a7110 0x7fba5c13f370 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fba5400caf0 tx=0x7fba5400a060 comp rx=0 tx=0).stop 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 10 monclient: ms_handle_reset current mon [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5c0a7110 msgr2=0x7fba5c13f370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fba5c0a7110 0x7fba5c13f370 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 10 monclient: _add_conns ranks=[4,6,0,8,7,1] 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba480035e0 0x7fba48005a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 10 monclient(hunting): picked mon.noname-c con 0x7fba480035e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fba48006580 0x7fba48008a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 10 monclient(hunting): picked mon.noname-d con 0x7fba48006580 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba480095b0 0x7fba4800ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 10 monclient(hunting): picked mon.noname-a con 0x7fba480095b0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fba4800c660 con 0x7fba480095b0 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fba4800c7e0 con 0x7fba480035e0 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fba4800c9e0 con 0x7fba48006580 2024-08-19T10:40:42.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba437fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba480035e0 0x7fba48005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5a7fc640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fba48006580 0x7fba48008a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 10 monclient(hunting): get_auth_request con 0x7fba480035e0 auth_method 0 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba480095b0 0x7fba4800ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5a7fc640 10 monclient(hunting): get_auth_request con 0x7fba48006580 auth_method 0 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 10 monclient(hunting): get_auth_request con 0x7fba480095b0 auth_method 0 2024-08-19T10:40:42.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 10 monclient(hunting): handle_auth_done global_id 4952 payload 293 2024-08-19T10:40:42.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fba48006580 msgr2=0x7fba48008a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fba48006580 0x7fba48008a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba480095b0 msgr2=0x7fba4800ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba480095b0 0x7fba4800ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 1 monclient: found mon.noname-c 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5affd640 1 -- 172.21.15.89:0/3767400817 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4800cca0 con 0x7fba480035e0 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.818+0000 7fba5b7fe640 1 -- 172.21.15.89:0/3767400817 reap_dead start 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5affd640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba480035e0 msgr2=0x7fba48005a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5affd640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba480035e0 0x7fba48005a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0ad000 tx=0x7fba54005410 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fba4c0b0040 con 0x7fba480035e0 2024-08-19T10:40:42.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 10 monclient: dump: 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.838 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba4c0044a0 con 0x7fba480035e0 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba612b4640 5 monclient: authenticate success, global_id 4952 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba612b4640 10 monclient: _renew_subs 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba612b4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba612b4640 1 -- 172.21.15.89:0/3767400817 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5c122b30 con 0x7fba480035e0 2024-08-19T10:40:42.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fba4c0aae30 con 0x7fba480035e0 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 10 monclient: dump: 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.840 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba597fa640 4 set_mon_vals no callback set 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba612b4640 10 monclient: _renew_subs 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba612b4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba612b4640 1 -- 172.21.15.89:0/3767400817 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5c122850 con 0x7fba480035e0 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba612b4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba612b4640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c138b70 0x7fba5c138f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba612b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba612b4640 1 -- 172.21.15.89:0/3767400817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c0a26c0 con 0x7fba5c138b70 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fba4c004030 con 0x7fba480035e0 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5a7fc640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c138b70 0x7fba5c138f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fba480406a0 0x7fba48042b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5a7fc640 10 monclient: get_auth_request con 0x7fba5c138b70 auth_method 0 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(72..72 src has 1..72) ==== 5249+0+0 (secure 0 0 0) 0x7fba4c0e7de0 con 0x7fba480035e0 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5b7fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fba480406a0 0x7fba48042b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5b7fe640 10 monclient: get_auth_request con 0x7fba480406a0 auth_method 0 2024-08-19T10:40:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5a7fc640 10 monclient(hunting): handle_auth_done global_id 14194 payload 824 2024-08-19T10:40:42.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.822+0000 7fba5a7fc640 1 -- 172.21.15.89:0/3767400817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba5c140d90 con 0x7fba5c138b70 2024-08-19T10:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba5a7fc640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c138b70 msgr2=0x7fba5c138f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba5a7fc640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c138b70 0x7fba5c138f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba48003500 tx=0x7fba50011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba5b7fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fba480406a0 0x7fba48042b60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba5c138b40 tx=0x7fba54010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fba50019020 con 0x7fba5c138b70 2024-08-19T10:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba437fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba437fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba437fe640 10 monclient: dump: 2024-08-19T10:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.845 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.846 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fba50007470 con 0x7fba5c138b70 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba437fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba437fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c138b70 msgr2=0x7fba5c138f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba437fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c138b70 0x7fba5c138f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba48003500 tx=0x7fba50011a30 comp rx=0 tx=0).stop 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c138b70 msgr2=0x7fba5c138f50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.826+0000 7fba437fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c138b70 0x7fba5c138f50 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba48003500 tx=0x7fba50011a30 comp rx=0 tx=0).stop 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba612b4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba612b4640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c139490 0x7fba5c13a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba612b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba612b4640 1 -- 172.21.15.89:0/3767400817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c0a67f0 con 0x7fba5c139490 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c139490 0x7fba5c13a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 10 monclient: get_auth_request con 0x7fba5c139490 auth_method 0 2024-08-19T10:40:42.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 10 monclient(hunting): handle_auth_done global_id 14203 payload 824 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 1 -- 172.21.15.89:0/3767400817 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fba5c064590 con 0x7fba5c139490 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c139490 msgr2=0x7fba5c13a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba5a7fc640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c139490 0x7fba5c13a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fba48003500 tx=0x7fba50003700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fba50019040 con 0x7fba5c139490 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba437fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba437fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:40:42.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.834+0000 7fba437fe640 10 monclient: dump: 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:42.849 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7fba50006cf0 con 0x7fba5c139490 2024-08-19T10:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba437fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba437fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c139490 msgr2=0x7fba5c13a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba437fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c139490 0x7fba5c13a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fba48003500 tx=0x7fba50003700 comp rx=0 tx=0).stop 2024-08-19T10:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba437fe640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c139490 msgr2=0x7fba5c13a7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:42.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba437fe640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c139490 0x7fba5c13a7a0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fba48003500 tx=0x7fba50003700 comp rx=0 tx=0).stop 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 20427, 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:42.851 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.852 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.853 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.854 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:42.855 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:42.856 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.857 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.858 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.859 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.860 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.861 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:42.863 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:42.864 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.865 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.867 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:42.870 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:42.871 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:42.872 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:42.873 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:42.874 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:42.874 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:40:42.874 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:42.874 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:42.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:42.874 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:42.874 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:42.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fba480406a0 msgr2=0x7fba48042b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fba480406a0 0x7fba48042b60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba5c138b40 tx=0x7fba54010000 comp rx=0 tx=0).stop 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 10 monclient: shutdown 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 -- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba480035e0 msgr2=0x7fba48005a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba480035e0 0x7fba48005a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0ad000 tx=0x7fba54005410 comp rx=0 tx=0).stop 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 -- 172.21.15.89:0/3767400817 shutdown_connections 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fba480406a0 0x7fba48042b60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fba480035e0 0x7fba48005a30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c138b70 0x7fba5c138f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 --2- 172.21.15.89:0/3767400817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba5c139490 0x7fba5c13a7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 -- 172.21.15.89:0/3767400817 >> 172.21.15.89:0/3767400817 conn(0x7fba5c069a80 msgr2=0x7fba5c09cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 -- 172.21.15.89:0/3767400817 shutdown_connections 2024-08-19T10:40:42.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.838+0000 7fba612b4640 1 -- 172.21.15.89:0/3767400817 wait complete. 2024-08-19T10:40:42.877 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:42.877 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:40:42.877 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:40:42.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff272ffd640 1 -- 172.21.15.190:0/1712644454 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v73) ==== 58+0+2 (secure 0 0 0) 0x7ff268010d10 con 0x7ff2740a3fc0 2024-08-19T10:40:42.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff272ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-08-19T10:40:42.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff272ffd640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:40:42.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff27c377640 1 -- 172.21.15.190:0/1712644454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff26003be00 msgr2=0x7ff26003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff27c377640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff26003be00 0x7ff26003e2c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2740a5510 tx=0x7ff26c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:40:42.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff27c377640 10 monclient: shutdown 2024-08-19T10:40:42.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff27c377640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:42.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff27c377640 1 -- 172.21.15.190:0/1712644454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2740a3fc0 msgr2=0x7ff2741cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:42.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff27c377640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2740a3fc0 0x7ff2741cfb30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c7d0 tx=0x7ff26800cc90 comp rx=0 tx=0).stop 2024-08-19T10:40:42.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff27c377640 1 -- 172.21.15.190:0/1712644454 shutdown_connections 2024-08-19T10:40:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff27c377640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff26003be00 0x7ff26003e2c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff27c377640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff274137b80 0x7ff274138e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff27c377640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2740a3fc0 0x7ff2741cfb30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.907+0000 7ff27c377640 1 --2- 172.21.15.190:0/1712644454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2740a36a0 0x7ff2741cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.911+0000 7ff27c377640 1 -- 172.21.15.190:0/1712644454 >> 172.21.15.190:0/1712644454 conn(0x7ff2740a4790 msgr2=0x7ff274132490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.911+0000 7ff27c377640 1 -- 172.21.15.190:0/1712644454 shutdown_connections 2024-08-19T10:40:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:42.911+0000 7ff27c377640 1 -- 172.21.15.190:0/1712644454 wait complete. 2024-08-19T10:40:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:+ o1=7 2024-08-19T10:40:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:++ ceph osd create 2024-08-19T10:40:42.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient: get_monmap_and_config 2024-08-19T10:40:42.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient: build_initial_monmap 2024-08-19T10:40:42.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient: monmap: 2024-08-19T10:40:42.990 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:42.990 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:42.990 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:42.989220+0000 2024-08-19T10:40:42.990 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:42.989220+0000 2024-08-19T10:40:42.990 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:42.990 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:42.990 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:42.990 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:42.991 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:42.992 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:42.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 1 Processor -- start 2024-08-19T10:40:42.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 1 -- start start 2024-08-19T10:40:42.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient: init 2024-08-19T10:40:42.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:42.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient: _add_conns ranks=[2,1,4,6,9,7,8,0,5,10,3] 2024-08-19T10:40:42.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 0x7f083c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient(hunting): picked mon.noname-b con 0x7f083c0a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:42.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f083c0a7180 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:42.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f083c0a7560 0x7f083c139d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:42.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient(hunting): picked mon.noname-c con 0x7f083c0a7560 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:42.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f083c0a33d0 con 0x7f083c0a7180 2024-08-19T10:40:42.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0a8190 con 0x7f083c0a6860 2024-08-19T10:40:42.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:42.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f083c0a79f0 con 0x7f083c0a7560 2024-08-19T10:40:42.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:42.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f0841ced640 10 monclient(hunting): authenticate will time out at 1092.805902s 2024-08-19T10:40:42.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f083c0a7560 0x7f083c139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f083c0a7560 0x7f083c139d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54150/0 (socket says 172.21.15.89:54150) 2024-08-19T10:40:42.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 1 -- 172.21.15.89:0/1765368326 learned_addr learned my addr 172.21.15.89:0/1765368326 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:42.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 10 monclient(hunting): get_auth_request con 0x7f083c0a7560 auth_method 0 2024-08-19T10:40:42.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083b7fe640 1 --2- 172.21.15.89:0/1765368326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 0x7f083c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083b7fe640 10 monclient(hunting): get_auth_request con 0x7f083c0a6860 auth_method 0 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 10 monclient(hunting): handle_auth_done global_id 5044 payload 274 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 1 -- 172.21.15.89:0/1765368326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 msgr2=0x7f083c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 1 --2- 172.21.15.89:0/1765368326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 0x7f083c0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:42.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 1 -- 172.21.15.89:0/1765368326 >> v1:172.21.15.190:6789/0 conn(0x7f083c0a7180 legacy=0x7f083c139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:42.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:42.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 1 monclient: found mon.noname-c 2024-08-19T10:40:42.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:42.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:42.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.986+0000 7f083bfff640 1 -- 172.21.15.89:0/1765368326 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c0a2ec0 con 0x7f083c0a7560 2024-08-19T10:40:43.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083bfff640 1 -- 172.21.15.89:0/1765368326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f083c0a7560 msgr2=0x7f083c139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083bfff640 1 --2- 172.21.15.89:0/1765368326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f083c0a7560 0x7f083c139d30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0834009e80 tx=0x7f083401c910 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1b182f29b5207d96 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 1 -- 172.21.15.89:0/1765368326 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08340090d0 con 0x7f083c0a7560 2024-08-19T10:40:43.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:43.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 10 monclient: dump: 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.005 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 5 monclient: authenticate success, global_id 5044 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 1 -- 172.21.15.89:0/1765368326 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08340a5a10 con 0x7f083c0a7560 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:43.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 4 set_mon_vals no callback set 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 1 -- 172.21.15.89:0/1765368326 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08340a5bc0 con 0x7f083c0a7560 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f083a7fc640 10 monclient: dump: 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.007 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 10 monclient: shutdown 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 1 -- 172.21.15.89:0/1765368326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f083c0a7560 msgr2=0x7f083c139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 1 --2- 172.21.15.89:0/1765368326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f083c0a7560 0x7f083c139d30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0834009e80 tx=0x7f083401c910 comp rx=0 tx=0).stop 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 1 -- 172.21.15.89:0/1765368326 shutdown_connections 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 1 --2- 172.21.15.89:0/1765368326 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f083c0a7560 0x7f083c139d30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 1 --2- 172.21.15.89:0/1765368326 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 0x7f083c0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 1 -- 172.21.15.89:0/1765368326 >> 172.21.15.89:0/1765368326 conn(0x7f083c074b40 msgr2=0x7f083c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 1 -- 172.21.15.89:0/1765368326 shutdown_connections 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.990+0000 7f0841ced640 1 -- 172.21.15.89:0/1765368326 wait complete. 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient: build_initial_monmap 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient: monmap: 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:43.009 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 1 Processor -- start 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 1 -- start start 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient: init 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient: _add_conns ranks=[2,8,3,5,7,6,1,0,4] 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 0x7f083c1d1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient(hunting): picked mon.noname-b con 0x7f083c0a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f083c0a7180 0x7f083c1d1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient(hunting): picked mon.noname-e con 0x7f083c0a7180 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c0a7560 0x7f083c13dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient(hunting): picked mon.noname-g con 0x7f083c0a7560 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f083c13a3f0 con 0x7f083c0a6860 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f083c13a270 con 0x7f083c0a7560 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f083c1440a0 con 0x7f083c0a7180 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f0841ced640 10 monclient(hunting): authenticate will time out at 1092.813180s 2024-08-19T10:40:43.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 0x7f083c1d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083affd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f083c0a7180 0x7f083c1d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 0x7f083c1d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43082/0 (socket says 172.21.15.89:43082) 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 1 -- 172.21.15.89:0/1011248339 learned_addr learned my addr 172.21.15.89:0/1011248339 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 10 monclient(hunting): get_auth_request con 0x7f083c0a6860 auth_method 0 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083affd640 10 monclient(hunting): get_auth_request con 0x7f083c0a7180 auth_method 0 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 10 monclient(hunting): handle_auth_done global_id 4970 payload 824 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 1 -- 172.21.15.89:0/1011248339 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f083c0a7180 msgr2=0x7f083c1d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f083c0a7180 0x7f083c1d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 1 -- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c0a7560 msgr2=0x7f083c13dfb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c0a7560 0x7f083c13dfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 1 monclient: found mon.noname-b 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:43.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 1 -- 172.21.15.89:0/1011248339 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c0a25b0 con 0x7f083c0a6860 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083affd640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f083c0a7180 0x7f083c1d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 1 -- 172.21.15.89:0/1011248339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 msgr2=0x7f083c1d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.994+0000 7f083b7fe640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 0x7f083c1d1320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f082c00e930 tx=0x7f082c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f082c017070 con 0x7f083c0a6860 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 10 monclient: dump: 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.014 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f082c013470 con 0x7f083c0a6860 2024-08-19T10:40:43.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0841ced640 5 monclient: authenticate success, global_id 4970 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0841ced640 10 monclient: _renew_subs 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0841ced640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0841ced640 1 -- 172.21.15.89:0/1011248339 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c123560 con 0x7f083c0a6860 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f082c013a90 con 0x7f083c0a6860 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 10 monclient: dump: 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.016 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0839ffb640 4 set_mon_vals no callback set 2024-08-19T10:40:43.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0841ced640 10 monclient: _renew_subs 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0841ced640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0841ced640 1 -- 172.21.15.89:0/1011248339 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c0a2060 con 0x7f083c0a6860 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0841ced640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0841ced640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13e4f0 0x7f083c13e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0841ced640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0841ced640 1 -- 172.21.15.89:0/1011248339 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0ac000 con 0x7f083c13e4f0 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f082c01a070 con 0x7f083c0a6860 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f082803be00 0x7f082803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f082c00f040 con 0x7f083c0a6860 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083affd640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13e4f0 0x7f083c13e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083affd640 10 monclient: get_auth_request con 0x7f083c13e4f0 auth_method 0 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083bfff640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f082803be00 0x7f082803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083bfff640 10 monclient: get_auth_request con 0x7f082803be00 auth_method 0 2024-08-19T10:40:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083affd640 10 monclient(hunting): handle_auth_done global_id 4989 payload 824 2024-08-19T10:40:43.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:42.998+0000 7f083affd640 1 -- 172.21.15.89:0/1011248339 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f083c0a6c40 con 0x7f083c13e4f0 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.002+0000 7f083affd640 1 -- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13e4f0 msgr2=0x7f083c13e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.002+0000 7f083bfff640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f082803be00 0x7f082803e2c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f083c0a5460 tx=0x7f08340a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.002+0000 7f083affd640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13e4f0 0x7f083c13e8d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f083000fae0 tx=0x7f0830004510 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.006+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08300054a0 con 0x7f083c13e4f0 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.006+0000 7f0823fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.006+0000 7f0823fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.006+0000 7f0823fff640 10 monclient: dump: 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.022 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.006+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f083000ddc0 con 0x7f083c13e4f0 2024-08-19T10:40:43.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.006+0000 7f0823fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.006+0000 7f0823fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.006+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13e4f0 msgr2=0x7f083c13e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.006+0000 7f0823fff640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13e4f0 0x7f083c13e8d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f083000fae0 tx=0x7f0830004510 comp rx=0 tx=0).stop 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.006+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13e4f0 msgr2=0x7f083c13e8d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.006+0000 7f0823fff640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13e4f0 0x7f083c13e8d0 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f083000fae0 tx=0x7f0830004510 comp rx=0 tx=0).stop 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f0841ced640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f0841ced640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13ee10 0x7f083c13a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f0841ced640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f0841ced640 1 -- 172.21.15.89:0/1011248339 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f083c064590 con 0x7f083c13ee10 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f083affd640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13ee10 0x7f083c13a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f083affd640 10 monclient: get_auth_request con 0x7f083c13ee10 auth_method 0 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f083affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f083affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f083affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f083affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f083affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.010+0000 7f083affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f083affd640 10 monclient(hunting): handle_auth_done global_id 4998 payload 824 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f083affd640 1 -- 172.21.15.89:0/1011248339 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f083c06cab0 con 0x7f083c13ee10 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f083affd640 1 -- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13ee10 msgr2=0x7f083c13a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f083affd640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13ee10 0x7f083c13a630 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0830018000 tx=0x7f0830003770 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0830004c60 con 0x7f083c13ee10 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0823fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0823fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0823fff640 10 monclient: dump: 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.025 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f083001e030 con 0x7f083c13ee10 2024-08-19T10:40:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0823fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0823fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13ee10 msgr2=0x7f083c13a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0823fff640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13ee10 0x7f083c13a630 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0830018000 tx=0x7f0830003770 comp rx=0 tx=0).stop 2024-08-19T10:40:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0823fff640 1 -- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13ee10 msgr2=0x7f083c13a630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:43.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0823fff640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f083c13ee10 0x7f083c13a630 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0830018000 tx=0x7f0830003770 comp rx=0 tx=0).stop 2024-08-19T10:40:43.027 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:43.027 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:43.027 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:43.027 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 172785, 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:43.028 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:43.029 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:43.030 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:43.031 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:43.032 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:43.033 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.034 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.038 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:43.040 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:43.041 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.041 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:43.041 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.041 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:43.042 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.042 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.042 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:43.042 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:43.042 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.042 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.042 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:43.043 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:43.044 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.045 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.046 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:43.046 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:43.046 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.046 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.046 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.046 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.046 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:43.046 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.048 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.049 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.050 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.051 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.052 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:43.052 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:43.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0841ced640 1 -- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f082803be00 msgr2=0x7f082803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0841ced640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f082803be00 0x7f082803e2c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f083c0a5460 tx=0x7f08340a7070 comp rx=0 tx=0).stop 2024-08-19T10:40:43.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0841ced640 10 monclient: shutdown 2024-08-19T10:40:43.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0841ced640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:43.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0841ced640 1 -- 172.21.15.89:0/1011248339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 msgr2=0x7f083c1d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0841ced640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 0x7f083c1d1320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f082c00e930 tx=0x7f082c00edf0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f083bfff640 1 -- 172.21.15.89:0/1011248339 reap_dead start 2024-08-19T10:40:43.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0841ced640 1 -- 172.21.15.89:0/1011248339 shutdown_connections 2024-08-19T10:40:43.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0841ced640 1 --2- 172.21.15.89:0/1011248339 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f083c0a6860 0x7f083c1d1320 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0841ced640 1 -- 172.21.15.89:0/1011248339 >> 172.21.15.89:0/1011248339 conn(0x7f083c074b40 msgr2=0x7f083c140f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:43.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0841ced640 1 -- 172.21.15.89:0/1011248339 shutdown_connections 2024-08-19T10:40:43.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.014+0000 7f0841ced640 1 -- 172.21.15.89:0/1011248339 wait complete. 2024-08-19T10:40:43.054 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:43.054 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:40:43.054 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:40:43.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.039+0000 7f3fbad27640 10 monclient: get_monmap_and_config 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.039+0000 7f3fbad27640 10 monclient: build_initial_monmap 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.039+0000 7f3fbad27640 10 monclient: monmap: 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:40:43.041432+0000 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:40:43.041432+0000 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:43.063 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.039+0000 7f3fbad27640 1 Processor -- start 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.039+0000 7f3fbad27640 1 -- start start 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.039+0000 7f3fbad27640 10 monclient: init 2024-08-19T10:40:43.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.039+0000 7f3fbad27640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 10 monclient: _add_conns ranks=[1,10,4,7,0,5,2,9,8,6,3] 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3fb40a6950 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3fb40a6d30 0x7f3fb40a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 10 monclient(hunting): picked mon.noname-f con 0x7f3fb40a6d30 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fb413eb60 0x7f3fb4140f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 10 monclient(hunting): picked mon.noname-c con 0x7f3fb413eb60 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fb40a63d0 con 0x7f3fb40a6950 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f3fb40a4640 con 0x7f3fb40a6d30 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3fb40a4920 con 0x7f3fb413eb60 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:43.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fbad27640 10 monclient(hunting): authenticate will time out at 1088.612138s 2024-08-19T10:40:43.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3fb40a6d30 0x7f3fb40a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb8ac3640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f3fb40a6950 0x7f3fb40a5750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:49346/0 (socket says 172.21.15.190:49346) 2024-08-19T10:40:43.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb8ac3640 1 -- 172.21.15.190:0/64067719 learned_addr learned my addr 172.21.15.190:0/64067719 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:40:43.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 10 monclient(hunting): get_auth_request con 0x7f3fb40a6d30 auth_method 0 2024-08-19T10:40:43.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 10 monclient(hunting): handle_auth_done global_id 14212 payload 274 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb92c4640 1 --2- 172.21.15.190:0/64067719 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fb413eb60 0x7f3fb4140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 1 -- 172.21.15.190:0/64067719 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fb413eb60 msgr2=0x7f3fb4140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 1 --2- 172.21.15.190:0/64067719 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fb413eb60 0x7f3fb4140f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 1 -- 172.21.15.190:0/64067719 >> v1:172.21.15.190:6789/0 conn(0x7f3fb40a6950 legacy=0x7f3fb40a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 1 monclient: found mon.noname-f 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.043+0000 7f3fb3fff640 1 -- 172.21.15.190:0/64067719 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb40a4c00 con 0x7f3fb40a6d30 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 1 -- 172.21.15.190:0/64067719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb40a6d30 msgr2=0x7f3fb40a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 1 --2- 172.21.15.190:0/64067719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb40a6d30 0x7f3fb40a7110 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80097c0 tx=0x7f3fa801c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f6a356aae09f647 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 1 -- 172.21.15.190:0/64067719 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3fa80b0070 con 0x7f3fb40a6d30 2024-08-19T10:40:43.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 10 monclient: dump: 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.068 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 1 -- 172.21.15.190:0/64067719 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa80aa810 con 0x7f3fb40a6d30 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 5 monclient: authenticate success, global_id 14212 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 1 -- 172.21.15.190:0/64067719 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3fa80a93e0 con 0x7f3fb40a6d30 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb37fe640 10 monclient: dump: 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.070 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 4 set_mon_vals no callback set 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient: shutdown 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 -- 172.21.15.190:0/64067719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb40a6d30 msgr2=0x7f3fb40a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 --2- 172.21.15.190:0/64067719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb40a6d30 0x7f3fb40a7110 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80097c0 tx=0x7f3fa801c360 comp rx=0 tx=0).stop 2024-08-19T10:40:43.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 -- 172.21.15.190:0/64067719 shutdown_connections 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 --2- 172.21.15.190:0/64067719 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fb413eb60 0x7f3fb4140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 --2- 172.21.15.190:0/64067719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb40a6d30 0x7f3fb40a7110 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 -- 172.21.15.190:0/64067719 >> 172.21.15.190:0/64067719 conn(0x7f3fb40abf50 msgr2=0x7f3fb40ac360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 -- 172.21.15.190:0/64067719 shutdown_connections 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 -- 172.21.15.190:0/64067719 wait complete. 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient: build_initial_monmap 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient: monmap: 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:43.072 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 Processor -- start 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 -- start start 2024-08-19T10:40:43.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient: init 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient: _add_conns ranks=[8,5,1,0,4,2,6,3,7] 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3fb40a6950 0x7f3fb41d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient(hunting): picked mon.noname-e con 0x7f3fb40a6950 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3fb40a6d30 0x7f3fb41399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient(hunting): picked mon.noname-h con 0x7f3fb40a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb413eb60 0x7f3fb40a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient(hunting): picked mon.noname-f con 0x7f3fb413eb60 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb4143e30 con 0x7f3fb413eb60 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f3fb4143cb0 con 0x7f3fb40a6d30 2024-08-19T10:40:43.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3fb4143fb0 con 0x7f3fb40a6950 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fbad27640 10 monclient(hunting): authenticate will time out at 1088.619491s 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3fb40a6d30 0x7f3fb41399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb92c4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb413eb60 0x7f3fb40a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3fb40a6d30 0x7f3fb41399b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:43422/0 (socket says 172.21.15.190:43422) 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 1 -- 172.21.15.190:0/4179186166 learned_addr learned my addr 172.21.15.190:0/4179186166 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 10 monclient(hunting): get_auth_request con 0x7f3fb40a6d30 auth_method 0 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb92c4640 10 monclient(hunting): get_auth_request con 0x7f3fb413eb60 auth_method 0 2024-08-19T10:40:43.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb92c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb92c4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb92c4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb8ac3640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3fb40a6950 0x7f3fb41d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb8ac3640 10 monclient(hunting): get_auth_request con 0x7f3fb40a6950 auth_method 0 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb8ac3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb8ac3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb8ac3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb92c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb92c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb92c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb8ac3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb8ac3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb8ac3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 10 monclient(hunting): handle_auth_done global_id 4991 payload 824 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 1 -- 172.21.15.190:0/4179186166 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3fb40a6950 msgr2=0x7f3fb41d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3fb40a6950 0x7f3fb41d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 1 -- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb413eb60 msgr2=0x7f3fb40a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb413eb60 0x7f3fb40a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 1 monclient: found mon.noname-h 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:40:43.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.047+0000 7f3fb3fff640 1 -- 172.21.15.190:0/4179186166 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb41226c0 con 0x7f3fb40a6d30 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb3fff640 1 -- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3fb40a6d30 msgr2=0x7f3fb41399b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb3fff640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3fb40a6d30 0x7f3fb41399b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80ae040 tx=0x7f3fa8019100 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 1 -- 172.21.15.190:0/4179186166 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3fa80b0040 con 0x7f3fb40a6d30 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 10 monclient: dump: 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.078 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 1 -- 172.21.15.190:0/4179186166 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa80b9070 con 0x7f3fb40a6d30 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fbad27640 5 monclient: authenticate success, global_id 4991 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 1 -- 172.21.15.190:0/4179186166 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3fa80044b0 con 0x7f3fb40a6d30 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 10 monclient: dump: 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.080 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fbad27640 10 monclient: _renew_subs 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fbad27640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:43.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb2ffd640 4 set_mon_vals no callback set 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fbad27640 1 -- 172.21.15.190:0/4179186166 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb4124060 con 0x7f3fb40a6d30 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fbad27640 10 monclient: _renew_subs 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fbad27640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fbad27640 1 -- 172.21.15.190:0/4179186166 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb40a92d0 con 0x7f3fb40a6d30 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fbad27640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fbad27640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fbad27640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fbad27640 1 -- 172.21.15.190:0/4179186166 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb413dd90 con 0x7f3fb40a6d30 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 1 -- 172.21.15.190:0/4179186166 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3fa8004040 con 0x7f3fb40a6d30 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f9803be00 0x7f3f9803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.051+0000 7f3fb17fa640 1 -- 172.21.15.190:0/4179186166 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f3fa80e8ab0 con 0x7f3fb40a6d30 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.055+0000 7f3fb8ac3640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f9803be00 0x7f3f9803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.055+0000 7f3fb8ac3640 10 monclient: get_auth_request con 0x7f3f9803be00 auth_method 0 2024-08-19T10:40:43.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.055+0000 7f3fb8ac3640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f9803be00 0x7f3f9803e2c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0007ca0 tx=0x7f3fa0007bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.059+0000 7f3fb17fa640 1 -- 172.21.15.190:0/4179186166 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3fa8016070 con 0x7f3fb40a6d30 2024-08-19T10:40:43.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.059+0000 7f3fb17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.059+0000 7f3fb17fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:43.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient: get_monmap_and_config 2024-08-19T10:40:43.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient: build_initial_monmap 2024-08-19T10:40:43.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient: monmap: 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:43.177053+0000 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:43.177053+0000 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:43.177 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:43.178 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:43.178 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:43.178 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:43.178 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:43.178 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 1 Processor -- start 2024-08-19T10:40:43.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 1 -- start start 2024-08-19T10:40:43.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient: init 2024-08-19T10:40:43.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient: _add_conns ranks=[2,8,7,0,4,10,6,9,1,3,5] 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee840a5270 0x7fee840a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient(hunting): picked mon.noname-b con 0x7fee840a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee840a6d30 0x7fee840a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient(hunting): picked mon.noname-e con 0x7fee840a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee841395b0 0x7fee841399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient(hunting): picked mon.noname-i con 0x7fee841395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fee840a7650 con 0x7fee840a5270 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fee840a8710 con 0x7fee841395b0 2024-08-19T10:40:43.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fee840a5a40 con 0x7fee840a6d30 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee8acf3640 10 monclient(hunting): authenticate will time out at 1092.993828s 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee83fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee840a6d30 0x7fee840a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee840a5270 0x7fee840a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee840a5270 0x7fee840a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43084/0 (socket says 172.21.15.89:43084) 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 1 -- 172.21.15.89:0/445323776 learned_addr learned my addr 172.21.15.89:0/445323776 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 10 monclient(hunting): get_auth_request con 0x7fee840a5270 auth_method 0 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee83fff640 10 monclient(hunting): get_auth_request con 0x7fee840a6d30 auth_method 0 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee83fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee83fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee89290640 1 --2- 172.21.15.89:0/445323776 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee841395b0 0x7fee841399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee89290640 10 monclient(hunting): get_auth_request con 0x7fee841395b0 auth_method 0 2024-08-19T10:40:43.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee89290640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee89290640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee89290640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 10 monclient(hunting): handle_auth_done global_id 4979 payload 274 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 1 -- 172.21.15.89:0/445323776 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee840a6d30 msgr2=0x7fee840a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 1 --2- 172.21.15.89:0/445323776 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee840a6d30 0x7fee840a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 1 -- 172.21.15.89:0/445323776 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee841395b0 msgr2=0x7fee841399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 1 --2- 172.21.15.89:0/445323776 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee841395b0 0x7fee841399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 10 monclient: _finish_hunting 0 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 1 monclient: found mon.noname-b 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:43.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.174+0000 7fee88a8f640 1 -- 172.21.15.89:0/445323776 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee840a81d0 con 0x7fee840a5270 2024-08-19T10:40:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee88a8f640 1 -- 172.21.15.89:0/445323776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee840a5270 msgr2=0x7fee840a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee88a8f640 1 --2- 172.21.15.89:0/445323776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee840a5270 0x7fee840a67f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009e80 tx=0x7fee7c01c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe2f3c6d25c0a07 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 1 -- 172.21.15.89:0/445323776 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee7c0090d0 con 0x7fee840a5270 2024-08-19T10:40:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 10 monclient: dump: 2024-08-19T10:40:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.191 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.192 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 1 -- 172.21.15.89:0/445323776 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7c0a5a50 con 0x7fee840a5270 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 5 monclient: authenticate success, global_id 4979 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 1 -- 172.21.15.89:0/445323776 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee7c0a5c00 con 0x7fee840a5270 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 4 set_mon_vals no callback set 2024-08-19T10:40:43.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee837fe640 10 monclient: dump: 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.194 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 10 monclient: shutdown 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 1 -- 172.21.15.89:0/445323776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee840a5270 msgr2=0x7fee840a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 1 --2- 172.21.15.89:0/445323776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee840a5270 0x7fee840a67f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009e80 tx=0x7fee7c01c910 comp rx=0 tx=0).stop 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 1 -- 172.21.15.89:0/445323776 shutdown_connections 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 1 --2- 172.21.15.89:0/445323776 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee840a6d30 0x7fee840a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 1 --2- 172.21.15.89:0/445323776 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee841395b0 0x7fee841399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 1 --2- 172.21.15.89:0/445323776 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee840a5270 0x7fee840a67f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 1 -- 172.21.15.89:0/445323776 >> 172.21.15.89:0/445323776 conn(0x7fee84069a80 msgr2=0x7fee840abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 1 -- 172.21.15.89:0/445323776 shutdown_connections 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.178+0000 7fee8acf3640 1 -- 172.21.15.89:0/445323776 wait complete. 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient: build_initial_monmap 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient: monmap: 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.196 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 1 Processor -- start 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 1 -- start start 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient: init 2024-08-19T10:40:43.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient: _add_conns ranks=[0,6,7,1,3,2,4,5,8] 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee840a5270 0x7fee84134b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient(hunting): picked mon.noname-a con 0x7fee840a5270 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fee840a6d30 0x7fee8409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient(hunting): picked mon.noname-d con 0x7fee840a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee841395b0 0x7fee8409cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient(hunting): picked mon.noname-i con 0x7fee841395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee840a57d0 con 0x7fee840a5270 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fee840a5650 con 0x7fee840a6d30 2024-08-19T10:40:43.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fee84143230 con 0x7fee841395b0 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee8acf3640 10 monclient(hunting): authenticate will time out at 1093.001182s 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee840a5270 0x7fee84134b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee83fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fee840a6d30 0x7fee8409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee840a5270 0x7fee84134b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42698/0 (socket says 172.21.15.89:42698) 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 1 -- 172.21.15.89:0/955932817 learned_addr learned my addr 172.21.15.89:0/955932817 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 10 monclient(hunting): get_auth_request con 0x7fee840a5270 auth_method 0 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee83fff640 10 monclient(hunting): get_auth_request con 0x7fee840a6d30 auth_method 0 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee83fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee83fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee89290640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee841395b0 0x7fee8409cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee89290640 10 monclient(hunting): get_auth_request con 0x7fee841395b0 auth_method 0 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee89290640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee89290640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee89290640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee89290640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee89290640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee89290640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 10 monclient(hunting): handle_auth_done global_id 5094 payload 824 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 1 -- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee841395b0 msgr2=0x7fee8409cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee841395b0 0x7fee8409cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 1 -- 172.21.15.89:0/955932817 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fee840a6d30 msgr2=0x7fee8409c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fee840a6d30 0x7fee8409c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 10 monclient: _finish_hunting 0 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 1 monclient: found mon.noname-a 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:43.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee88a8f640 1 -- 172.21.15.89:0/955932817 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8409e2e0 con 0x7fee840a5270 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.182+0000 7fee83fff640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fee840a6d30 0x7fee8409c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee88a8f640 1 -- 172.21.15.89:0/955932817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee840a5270 msgr2=0x7fee84134b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee88a8f640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee840a5270 0x7fee84134b10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009e20 tx=0x7fee7c0196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee7c0bc020 con 0x7fee840a5270 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 10 monclient: dump: 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.202 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7c003e10 con 0x7fee840a5270 2024-08-19T10:40:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee8acf3640 5 monclient: authenticate success, global_id 5094 2024-08-19T10:40:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee8acf3640 10 monclient: _renew_subs 2024-08-19T10:40:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee8acf3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee8acf3640 1 -- 172.21.15.89:0/955932817 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee841405f0 con 0x7fee840a5270 2024-08-19T10:40:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee7c0044b0 con 0x7fee840a5270 2024-08-19T10:40:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 10 monclient: dump: 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.205 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee82ffd640 4 set_mon_vals no callback set 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee8acf3640 10 monclient: _renew_subs 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee8acf3640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee8acf3640 1 -- 172.21.15.89:0/955932817 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee840a1020 con 0x7fee840a5270 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee8acf3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee8acf3640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee8409dc20 0x7fee8402d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee8acf3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee8acf3640 1 -- 172.21.15.89:0/955932817 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fee840a67f0 con 0x7fee8409dc20 2024-08-19T10:40:43.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fee7c0bf020 con 0x7fee840a5270 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee83fff640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee8409dc20 0x7fee8402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fee6403be00 0x7fee6403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee83fff640 10 monclient: get_auth_request con 0x7fee8409dc20 auth_method 0 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee83fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee83fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7fee7c0e9af0 con 0x7fee840a5270 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee89290640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fee6403be00 0x7fee6403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee89290640 10 monclient: get_auth_request con 0x7fee6403be00 auth_method 0 2024-08-19T10:40:43.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee83fff640 10 monclient(hunting): handle_auth_done global_id 5000 payload 824 2024-08-19T10:40:43.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.186+0000 7fee83fff640 1 -- 172.21.15.89:0/955932817 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee84064590 con 0x7fee8409dc20 2024-08-19T10:40:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee83fff640 1 -- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee8409dc20 msgr2=0x7fee8402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee83fff640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee8409dc20 0x7fee8402d1d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fee8409e950 tx=0x7fee6c00f610 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee89290640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fee6403be00 0x7fee6403e2c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee8409ed80 tx=0x7fee74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee6c00fe40 con 0x7fee8409dc20 2024-08-19T10:40:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee817fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:43.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee817fa640 10 monclient: dump: 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.211 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fee6c00fe40 con 0x7fee8409dc20 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee817fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee817fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee8409dc20 msgr2=0x7fee8402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee817fa640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee8409dc20 0x7fee8402d1d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fee8409e950 tx=0x7fee6c00f610 comp rx=0 tx=0).stop 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee8409dc20 msgr2=0x7fee8402d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.190+0000 7fee817fa640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee8409dc20 0x7fee8402d1d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee8acf3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:43.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee8acf3640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee84135050 0x7fee8402d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee8acf3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee8acf3640 1 -- 172.21.15.89:0/955932817 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fee8406c9e0 con 0x7fee84135050 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee84135050 0x7fee8402d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 10 monclient: get_auth_request con 0x7fee84135050 auth_method 0 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 10 monclient(hunting): handle_auth_done global_id 5009 payload 824 2024-08-19T10:40:43.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 1 -- 172.21.15.89:0/955932817 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fee8413c2d0 con 0x7fee84135050 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 1 -- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee84135050 msgr2=0x7fee8402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.198+0000 7fee83fff640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee84135050 0x7fee8402d710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fee6c01b000 tx=0x7fee6c0046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee6c02b020 con 0x7fee84135050 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee817fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee817fa640 10 monclient: dump: 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.214 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7fee6c0133a0 con 0x7fee84135050 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee817fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:43.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee817fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee84135050 msgr2=0x7fee8402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee817fa640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee84135050 0x7fee8402d710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fee6c01b000 tx=0x7fee6c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee817fa640 1 -- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee84135050 msgr2=0x7fee8402d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee817fa640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee84135050 0x7fee8402d710 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fee6c01b000 tx=0x7fee6c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:43.216 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 172973, 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:43.217 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:43.218 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:43.219 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:43.220 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:43.221 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:43.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:43.226 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.227 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:43.231 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.235 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:43.236 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:43.237 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.238 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:43.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.240 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:43.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.241 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:43.241 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:43.241 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 4 2024-08-19T10:40:43.241 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.241 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.241 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:43.241 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:43.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.235+0000 7f3fbad27640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-08-19T10:40:43.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.235+0000 7f3fbad27640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-08-19T10:40:43.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.235+0000 7f3fbad27640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:40:43.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:43.235+0000 7f3fbad27640 1 -- 172.21.15.190:0/4179186166 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f3fb40a7110 con 0x7f3fb40a6d30 2024-08-19T10:40:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee8acf3640 1 -- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fee6403be00 msgr2=0x7fee6403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee8acf3640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fee6403be00 0x7fee6403e2c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee8409ed80 tx=0x7fee74008040 comp rx=0 tx=0).stop 2024-08-19T10:40:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee8acf3640 10 monclient: shutdown 2024-08-19T10:40:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee8acf3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee8acf3640 1 -- 172.21.15.89:0/955932817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee840a5270 msgr2=0x7fee84134b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee8acf3640 1 --2- 172.21.15.89:0/955932817 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee840a5270 0x7fee84134b10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009e20 tx=0x7fee7c0196b0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee89290640 1 -- 172.21.15.89:0/955932817 reap_dead start 2024-08-19T10:40:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee89290640 1 -- 172.21.15.89:0/955932817 reap_dead start 2024-08-19T10:40:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee8acf3640 1 -- 172.21.15.89:0/955932817 shutdown_connections 2024-08-19T10:40:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee8acf3640 1 -- 172.21.15.89:0/955932817 >> 172.21.15.89:0/955932817 conn(0x7fee84069a80 msgr2=0x7fee84138950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee8acf3640 1 -- 172.21.15.89:0/955932817 shutdown_connections 2024-08-19T10:40:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.202+0000 7fee8acf3640 1 -- 172.21.15.89:0/955932817 wait complete. 2024-08-19T10:40:43.245 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:43.245 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:40:43.245 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:40:43.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient: get_monmap_and_config 2024-08-19T10:40:43.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient: build_initial_monmap 2024-08-19T10:40:43.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient: monmap: 2024-08-19T10:40:43.361 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:43.361 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.361 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:43.361523+0000 2024-08-19T10:40:43.361 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:43.361523+0000 2024-08-19T10:40:43.361 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:43.361 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.361 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:43.362 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:43.362 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:43.362 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:43.362 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:43.362 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:43.362 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:43.362 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:43.362 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:43.362 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:43.362 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:43.362 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 1 Processor -- start 2024-08-19T10:40:43.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 1 -- start start 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient: init 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient: _add_conns ranks=[7,8,3,0,6,2,9,4,5,10,1] 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da00a5270 0x7f5da00a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient(hunting): picked mon.noname-i con 0x7f5da00a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5da00a6d30 0x7f5da00a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient(hunting): picked mon.noname-e con 0x7f5da00a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5da01395b0 0x7f5da01399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient(hunting): picked mon.noname-g con 0x7f5da01395b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5da00a7650 con 0x7f5da01395b0 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5da00a8710 con 0x7f5da00a5270 2024-08-19T10:40:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f5da00a5a40 con 0x7f5da00a6d30 2024-08-19T10:40:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da8713640 10 monclient(hunting): authenticate will time out at 1093.178123s 2024-08-19T10:40:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5da00a6d30 0x7f5da00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5da00a6d30 0x7f5da00a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40554/0 (socket says 172.21.15.89:40554) 2024-08-19T10:40:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 1 -- 172.21.15.89:0/1393197731 learned_addr learned my addr 172.21.15.89:0/1393197731 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 10 monclient(hunting): get_auth_request con 0x7f5da00a6d30 auth_method 0 2024-08-19T10:40:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da64af640 1 --2- 172.21.15.89:0/1393197731 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da00a5270 0x7f5da00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da6cb0640 1 --2- 172.21.15.89:0/1393197731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5da01395b0 0x7f5da01399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da6cb0640 10 monclient(hunting): get_auth_request con 0x7f5da01395b0 auth_method 0 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da6cb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da6cb0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da6cb0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da64af640 10 monclient(hunting): get_auth_request con 0x7f5da00a5270 auth_method 0 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da64af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da64af640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da64af640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 10 monclient(hunting): handle_auth_done global_id 4913 payload 274 2024-08-19T10:40:43.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 1 -- 172.21.15.89:0/1393197731 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da00a5270 msgr2=0x7f5da00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:43.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 1 --2- 172.21.15.89:0/1393197731 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da00a5270 0x7f5da00a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 1 -- 172.21.15.89:0/1393197731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5da01395b0 msgr2=0x7f5da01399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:43.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 1 --2- 172.21.15.89:0/1393197731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5da01395b0 0x7f5da01399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 10 monclient: _finish_hunting 0 2024-08-19T10:40:43.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 1 monclient: found mon.noname-e 2024-08-19T10:40:43.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:43.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:43.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.358+0000 7f5da5cae640 1 -- 172.21.15.89:0/1393197731 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da00a81d0 con 0x7f5da00a6d30 2024-08-19T10:40:43.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da5cae640 1 -- 172.21.15.89:0/1393197731 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5da00a6d30 msgr2=0x7f5da00a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da5cae640 1 --2- 172.21.15.89:0/1393197731 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5da00a6d30 0x7f5da00a7110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0098a0 tx=0x7f5d9c01c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=850dde700ed7f066 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 1 -- 172.21.15.89:0/1393197731 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d9c0b0070 con 0x7f5da00a6d30 2024-08-19T10:40:43.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:43.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 10 monclient: dump: 2024-08-19T10:40:43.375 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.375 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.375 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.375 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.375 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 1 -- 172.21.15.89:0/1393197731 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9c0aa810 con 0x7f5da00a6d30 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 5 monclient: authenticate success, global_id 4913 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 1 -- 172.21.15.89:0/1393197731 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d9c0a9300 con 0x7f5da00a6d30 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 4 set_mon_vals no callback set 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da54ad640 10 monclient: dump: 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.377 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.378 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 10 monclient: shutdown 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 1 -- 172.21.15.89:0/1393197731 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5da00a6d30 msgr2=0x7f5da00a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 1 --2- 172.21.15.89:0/1393197731 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5da00a6d30 0x7f5da00a7110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0098a0 tx=0x7f5d9c01c440 comp rx=0 tx=0).stop 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 1 -- 172.21.15.89:0/1393197731 shutdown_connections 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 1 --2- 172.21.15.89:0/1393197731 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5da00a6d30 0x7f5da00a7110 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 1 --2- 172.21.15.89:0/1393197731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5da01395b0 0x7f5da01399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 1 --2- 172.21.15.89:0/1393197731 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da00a5270 0x7f5da00a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 1 -- 172.21.15.89:0/1393197731 >> 172.21.15.89:0/1393197731 conn(0x7f5da0069a80 msgr2=0x7f5da00abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 1 -- 172.21.15.89:0/1393197731 shutdown_connections 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.362+0000 7f5da8713640 1 -- 172.21.15.89:0/1393197731 wait complete. 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient: build_initial_monmap 2024-08-19T10:40:43.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient: monmap: 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:43.380 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 1 Processor -- start 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 1 -- start start 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient: init 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient: _add_conns ranks=[1,6,5,3,2,8,7,0,4] 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da00a5270 0x7f5da0131de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient(hunting): picked mon.noname-f con 0x7f5da00a5270 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5da00a6d30 0x7f5da013cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient(hunting): picked mon.noname-d con 0x7f5da00a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5da01395b0 0x7f5da013d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient(hunting): picked mon.noname-h con 0x7f5da01395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:43.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5da00a57d0 con 0x7f5da00a5270 2024-08-19T10:40:43.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f5da00a5650 con 0x7f5da01395b0 2024-08-19T10:40:43.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f5da0143230 con 0x7f5da00a6d30 2024-08-19T10:40:43.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:43.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da8713640 10 monclient(hunting): authenticate will time out at 1093.185376s 2024-08-19T10:40:43.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5da00a6d30 0x7f5da013cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5da00a6d30 0x7f5da013cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54418/0 (socket says 172.21.15.89:54418) 2024-08-19T10:40:43.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 1 -- 172.21.15.89:0/1268679746 learned_addr learned my addr 172.21.15.89:0/1268679746 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:43.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 10 monclient(hunting): get_auth_request con 0x7f5da00a6d30 auth_method 0 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da6cb0640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5da01395b0 0x7f5da013d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da64af640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da00a5270 0x7f5da0131de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 10 monclient(hunting): handle_auth_done global_id 5019 payload 824 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 1 -- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5da01395b0 msgr2=0x7f5da013d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5da01395b0 0x7f5da013d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 1 -- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da00a5270 msgr2=0x7f5da0131de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da00a5270 0x7f5da0131de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 10 monclient: _finish_hunting 0 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 1 monclient: found mon.noname-d 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da5cae640 1 -- 172.21.15.89:0/1268679746 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da009e290 con 0x7f5da00a6d30 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da64af640 10 monclient: get_auth_request con 0x7f5da00a5270 auth_method 0 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da6cb0640 10 monclient: get_auth_request con 0x7f5da01395b0 auth_method 0 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da64af640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5da00a5270 0x7f5da0131de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.366+0000 7f5da6cb0640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5da01395b0 0x7f5da013d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da5cae640 1 -- 172.21.15.89:0/1268679746 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5da00a6d30 msgr2=0x7f5da013cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da5cae640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5da00a6d30 0x7f5da013cef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0ae040 tx=0x7f5d9c0191e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d9c0b9070 con 0x7f5da00a6d30 2024-08-19T10:40:43.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 10 monclient: dump: 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.385 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da8713640 5 monclient: authenticate success, global_id 5019 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d9c0044b0 con 0x7f5da00a6d30 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d9c0b0040 con 0x7f5da00a6d30 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da4cac640 4 set_mon_vals no callback set 2024-08-19T10:40:43.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 10 monclient: dump: 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.387 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da8713640 10 monclient: _renew_subs 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da8713640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da8713640 1 -- 172.21.15.89:0/1268679746 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da009d8c0 con 0x7f5da00a6d30 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da8713640 10 monclient: _renew_subs 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da8713640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da8713640 1 -- 172.21.15.89:0/1268679746 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da00a7310 con 0x7f5da00a6d30 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da8713640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da8713640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da009d1f0 0x7f5da013d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da8713640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da8713640 1 -- 172.21.15.89:0/1268679746 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5da0140680 con 0x7f5da009d1f0 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5d9c004040 con 0x7f5da00a6d30 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da64af640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da009d1f0 0x7f5da013d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da64af640 10 monclient: get_auth_request con 0x7f5da009d1f0 auth_method 0 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da64af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da64af640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da64af640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d8403bdf0 0x7f5d8403e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da64af640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da64af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da64af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f5d9c0e8100 con 0x7f5da00a6d30 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da64af640 10 monclient(hunting): handle_auth_done global_id 14227 payload 824 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da6cb0640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d8403bdf0 0x7f5d8403e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da64af640 1 -- 172.21.15.89:0/1268679746 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5da00a67f0 con 0x7f5da009d1f0 2024-08-19T10:40:43.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.370+0000 7f5da6cb0640 10 monclient: get_auth_request con 0x7f5d8403bdf0 auth_method 0 2024-08-19T10:40:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5da64af640 1 -- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da009d1f0 msgr2=0x7f5da013d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5da64af640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da009d1f0 0x7f5da013d970 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e740 tx=0x7f5d9000ec00 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d90017020 con 0x7f5da009d1f0 2024-08-19T10:40:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5d96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5d96ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:40:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5d96ffd640 10 monclient: dump: 2024-08-19T10:40:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.392 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5da6cb0640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d8403bdf0 0x7f5d8403e2b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5da00acc30 tx=0x7f5d9800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5d90004030 con 0x7f5da009d1f0 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5d96ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5d96ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da009d1f0 msgr2=0x7f5da013d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5d96ffd640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da009d1f0 0x7f5da013d970 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e740 tx=0x7f5d9000ec00 comp rx=0 tx=0).stop 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da009d1f0 msgr2=0x7f5da013d970 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.374+0000 7f5d96ffd640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da009d1f0 0x7f5da013d970 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e740 tx=0x7f5d9000ec00 comp rx=0 tx=0).stop 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.378+0000 7f5da8713640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.378+0000 7f5da8713640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da013deb0 0x7f5da0137a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.378+0000 7f5da8713640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.378+0000 7f5da8713640 1 -- 172.21.15.89:0/1268679746 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5da0064590 con 0x7f5da013deb0 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da013deb0 0x7f5da0137a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 10 monclient: get_auth_request con 0x7f5da013deb0 auth_method 0 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 10 monclient(hunting): handle_auth_done global_id 14236 payload 824 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 1 -- 172.21.15.89:0/1268679746 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5da013e2c0 con 0x7f5da013deb0 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 1 -- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da013deb0 msgr2=0x7f5da0137a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5da64af640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da013deb0 0x7f5da0137a20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e740 tx=0x7f5d900046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d90017040 con 0x7f5da013deb0 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5d96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5d96ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:40:43.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.382+0000 7f5d96ffd640 10 monclient: dump: 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.396 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.397 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7f5d9001e020 con 0x7f5da013deb0 2024-08-19T10:40:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5d96ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:40:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5d96ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da013deb0 msgr2=0x7f5da0137a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5d96ffd640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da013deb0 0x7f5da0137a20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e740 tx=0x7f5d900046d0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5d96ffd640 1 -- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da013deb0 msgr2=0x7f5da0137a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:43.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5d96ffd640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5da013deb0 0x7f5da0137a20 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000e740 tx=0x7f5d900046d0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.398 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:43.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:43.398 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:43.398 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:40:43.398 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 26, 2024-08-19T10:40:43.398 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 20975, 2024-08-19T10:40:43.398 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:40:43.398 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:40:43.398 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:40:43.398 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:40:43.398 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:43.399 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:43.400 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:40:43.401 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:40:43.402 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:40:43.403 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:40:43.406 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.409 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:40:43.410 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:40:43.411 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.414 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.415 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:40:43.417 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:40:43.418 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:40:43.419 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:40:43.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:40:43.421 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:40:43.421 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:40:43.421 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:40:43.421 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:40:43.421 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:40:43.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:40:43.421 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:40:43.421 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:43.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da8713640 1 -- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d8403bdf0 msgr2=0x7f5d8403e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da8713640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d8403bdf0 0x7f5d8403e2b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5da00acc30 tx=0x7f5d9800b040 comp rx=0 tx=0).stop 2024-08-19T10:40:43.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da8713640 10 monclient: shutdown 2024-08-19T10:40:43.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da8713640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:43.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da8713640 1 -- 172.21.15.89:0/1268679746 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5da00a6d30 msgr2=0x7f5da013cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da8713640 1 --2- 172.21.15.89:0/1268679746 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5da00a6d30 0x7f5da013cef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0ae040 tx=0x7f5d9c0191e0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da6cb0640 1 -- 172.21.15.89:0/1268679746 reap_dead start 2024-08-19T10:40:43.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da6cb0640 1 -- 172.21.15.89:0/1268679746 reap_dead start 2024-08-19T10:40:43.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da8713640 1 -- 172.21.15.89:0/1268679746 shutdown_connections 2024-08-19T10:40:43.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da8713640 1 -- 172.21.15.89:0/1268679746 >> 172.21.15.89:0/1268679746 conn(0x7f5da0069a80 msgr2=0x7f5da00ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:43.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da8713640 1 -- 172.21.15.89:0/1268679746 shutdown_connections 2024-08-19T10:40:43.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.386+0000 7f5da8713640 1 -- 172.21.15.89:0/1268679746 wait complete. 2024-08-19T10:40:43.423 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:40:43.424 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['g'] 2024-08-19T10:40:43.424 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-08-19T10:40:43.424 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-08-19T10:40:43.424 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g store 2024-08-19T10:40:43.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g sync_force --yes-i-really-mean-it 2024-08-19T10:40:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient: get_monmap_and_config 2024-08-19T10:40:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient: build_initial_monmap 2024-08-19T10:40:43.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient: monmap: 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:43.534024+0000 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:43.534024+0000 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:43.534 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:43.535 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:43.535 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 1 Processor -- start 2024-08-19T10:40:43.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 1 -- start start 2024-08-19T10:40:43.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient: init 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient: _add_conns ranks=[8,7,3,2,0,1,10,6,5,9,4] 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2b780a6860 0x7f2b780a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient(hunting): picked mon.noname-e con 0x7f2b780a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2b780a7180 0x7f2b780a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient(hunting): picked mon.noname-i con 0x7f2b780a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b78139620 0x7f2b78139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient(hunting): picked mon.noname-g con 0x7f2b78139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2b780a8190 con 0x7f2b78139620 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2b780a33d0 con 0x7f2b780a7180 2024-08-19T10:40:43.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2b780a2ec0 con 0x7f2b780a6860 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7fd2b640 10 monclient(hunting): authenticate will time out at 1093.350761s 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2b780a6860 0x7f2b780a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2b780a6860 0x7f2b780a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40570/0 (socket says 172.21.15.89:40570) 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 1 -- 172.21.15.89:0/1542871884 learned_addr learned my addr 172.21.15.89:0/1542871884 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 10 monclient(hunting): get_auth_request con 0x7f2b780a6860 auth_method 0 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7e2c8640 1 --2- 172.21.15.89:0/1542871884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b78139620 0x7f2b78139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7e2c8640 10 monclient(hunting): get_auth_request con 0x7f2b78139620 auth_method 0 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7e2c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7e2c8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7e2c8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7d2c6640 1 --2- 172.21.15.89:0/1542871884 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2b780a7180 0x7f2b780a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7d2c6640 10 monclient(hunting): get_auth_request con 0x7f2b780a7180 auth_method 0 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7d2c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7d2c6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7d2c6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_done global_id 4922 payload 274 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 1 -- 172.21.15.89:0/1542871884 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2b780a7180 msgr2=0x7f2b780a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:43.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 1 --2- 172.21.15.89:0/1542871884 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2b780a7180 0x7f2b780a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 1 -- 172.21.15.89:0/1542871884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b78139620 msgr2=0x7f2b78139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 1 --2- 172.21.15.89:0/1542871884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b78139620 0x7f2b78139a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 10 monclient: _finish_hunting 0 2024-08-19T10:40:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 1 monclient: found mon.noname-e 2024-08-19T10:40:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:43.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.530+0000 7f2b7dac7640 1 -- 172.21.15.89:0/1542871884 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b780a4cd0 con 0x7f2b780a6860 2024-08-19T10:40:43.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7dac7640 1 -- 172.21.15.89:0/1542871884 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2b780a6860 msgr2=0x7f2b780a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7dac7640 1 --2- 172.21.15.89:0/1542871884 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2b780a6860 0x7f2b780a6c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009e80 tx=0x7f2b7001c910 comp rx=0 tx=0).ready entity=mon.8 client_cookie=751edafda4fcc628 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 1 -- 172.21.15.89:0/1542871884 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b700090d0 con 0x7f2b780a6860 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 10 monclient: dump: 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.545 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 1 -- 172.21.15.89:0/1542871884 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b700a6970 con 0x7f2b780a6860 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 1 -- 172.21.15.89:0/1542871884 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b700af480 con 0x7f2b780a6860 2024-08-19T10:40:43.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7fd2b640 5 monclient: authenticate success, global_id 4922 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7fd2b640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7fd2b640 4 set_mon_vals no callback set 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.534+0000 7f2b7cac5640 10 monclient: dump: 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.547 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient: shutdown 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/1542871884 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2b780a6860 msgr2=0x7f2b780a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 --2- 172.21.15.89:0/1542871884 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2b780a6860 0x7f2b780a6c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009e80 tx=0x7f2b7001c910 comp rx=0 tx=0).stop 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/1542871884 shutdown_connections 2024-08-19T10:40:43.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 --2- 172.21.15.89:0/1542871884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b78139620 0x7f2b78139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 --2- 172.21.15.89:0/1542871884 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2b780a7180 0x7f2b780a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 --2- 172.21.15.89:0/1542871884 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2b780a6860 0x7f2b780a6c40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/1542871884 >> 172.21.15.89:0/1542871884 conn(0x7f2b78074b40 msgr2=0x7f2b780abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/1542871884 shutdown_connections 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/1542871884 wait complete. 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient: build_initial_monmap 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient: monmap: 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.549 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 Processor -- start 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 -- start start 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient: init 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:43.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient: _add_conns ranks=[1,7,2,6,8,4,0,5,3] 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b780a6860 0x7f2b780a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient(hunting): picked mon.noname-f con 0x7f2b780a6860 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2b780a7180 0x7f2b780a2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient(hunting): picked mon.noname-i con 0x7f2b780a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b78139620 0x7f2b7809c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient(hunting): picked mon.noname-b con 0x7f2b78139620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2b78143210 con 0x7f2b780a6860 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f2b78143090 con 0x7f2b78139620 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2b78143390 con 0x7f2b780a7180 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:43.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7fd2b640 10 monclient(hunting): authenticate will time out at 1093.358446s 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b78139620 0x7f2b7809c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b78139620 0x7f2b7809c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43096/0 (socket says 172.21.15.89:43096) 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 1 -- 172.21.15.89:0/2693061185 learned_addr learned my addr 172.21.15.89:0/2693061185 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 10 monclient(hunting): get_auth_request con 0x7f2b78139620 auth_method 0 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7d2c6640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2b780a7180 0x7f2b780a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7dac7640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b780a6860 0x7f2b780a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7d2c6640 10 monclient(hunting): get_auth_request con 0x7f2b780a7180 auth_method 0 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7d2c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7d2c6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7d2c6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 10 monclient(hunting): handle_auth_done global_id 4988 payload 824 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 1 -- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2b780a7180 msgr2=0x7f2b780a2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2b780a7180 0x7f2b780a2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 1 -- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b780a6860 msgr2=0x7f2b780a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b780a6860 0x7f2b780a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 10 monclient: _finish_hunting 0 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 1 monclient: found mon.noname-b 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:43.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7e2c8640 1 -- 172.21.15.89:0/2693061185 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b780a2570 con 0x7f2b78139620 2024-08-19T10:40:43.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7dac7640 10 monclient: get_auth_request con 0x7f2b780a6860 auth_method 0 2024-08-19T10:40:43.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.538+0000 7f2b7dac7640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b780a6860 0x7f2b780a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7e2c8640 1 -- 172.21.15.89:0/2693061185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b78139620 msgr2=0x7f2b7809c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7e2c8640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b78139620 0x7f2b7809c9e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b68006e90 tx=0x7f2b6800d9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b68012c70 con 0x7f2b78139620 2024-08-19T10:40:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 10 monclient: dump: 2024-08-19T10:40:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.557 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7fd2b640 5 monclient: authenticate success, global_id 4988 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7fd2b640 10 monclient: _renew_subs 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b68019070 con 0x7f2b78139620 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7fd2b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/2693061185 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b780ac170 con 0x7f2b78139620 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b68012e30 con 0x7f2b78139620 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b77fff640 4 set_mon_vals no callback set 2024-08-19T10:40:43.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 10 monclient: dump: 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.560 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7fd2b640 10 monclient: _renew_subs 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7fd2b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/2693061185 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b780abc40 con 0x7f2b78139620 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7fd2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7fd2b640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7809f330 0x7f2b7802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7fd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/2693061185 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2b781406c0 con 0x7f2b7809f330 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2b68012800 con 0x7f2b78139620 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2b6403be00 0x7f2b6403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7dac7640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7809f330 0x7f2b7802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f2b6800f040 con 0x7f2b78139620 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7dac7640 10 monclient: get_auth_request con 0x7f2b7809f330 auth_method 0 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7dac7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7dac7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7dac7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7d2c6640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2b6403be00 0x7f2b6403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7d2c6640 10 monclient: get_auth_request con 0x7f2b6403be00 auth_method 0 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_done global_id 5025 payload 824 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.542+0000 7f2b7dac7640 1 -- 172.21.15.89:0/2693061185 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b780a7560 con 0x7f2b7809f330 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b7dac7640 1 -- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7809f330 msgr2=0x7f2b7802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b7dac7640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7809f330 0x7f2b7802d1d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2b700ae000 tx=0x7f2b700196b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b7d2c6640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2b6403be00 0x7f2b6403e2c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2b7813ec60 tx=0x7f2b6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b70003dc0 con 0x7f2b7809f330 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b767fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b767fc640 10 monclient: dump: 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.563 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2b70003dc0 con 0x7f2b7809f330 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b767fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b767fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7809f330 msgr2=0x7f2b7802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b767fc640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7809f330 0x7f2b7802d1d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2b700ae000 tx=0x7f2b700196b0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7809f330 msgr2=0x7f2b7802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:43.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.546+0000 7f2b767fc640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7809f330 0x7f2b7802d1d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7fd2b640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:40:43.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7fd2b640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7802d710 0x7f2b7802daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7fd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/2693061185 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7809ee90 con 0x7f2b7802d710 2024-08-19T10:40:43.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7dac7640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7802d710 0x7f2b7802daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7dac7640 10 monclient: get_auth_request con 0x7f2b7802d710 auth_method 0 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7dac7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7dac7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7dac7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7dac7640 10 monclient(hunting): handle_auth_done global_id 5034 payload 824 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.554+0000 7f2b7dac7640 1 -- 172.21.15.89:0/2693061185 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7f2b78064590 con 0x7f2b7802d710 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7dac7640 1 -- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7802d710 msgr2=0x7f2b7802daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7dac7640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7802d710 0x7f2b7802daf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2b7001c740 tx=0x7f2b700a9000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2b700cd080 con 0x7f2b7802d710 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b767fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b767fc640 10 monclient: dump: 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.568 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.569 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7f2b70004390 con 0x7f2b7802d710 2024-08-19T10:40:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b767fc640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:40:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b767fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7802d710 msgr2=0x7f2b7802daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b767fc640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7802d710 0x7f2b7802daf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2b7001c740 tx=0x7f2b700a9000 comp rx=0 tx=0).stop 2024-08-19T10:40:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b767fc640 1 -- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7802d710 msgr2=0x7f2b7802daf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:40:43.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b767fc640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b7802d710 0x7f2b7802daf0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2b7001c740 tx=0x7f2b700a9000 comp rx=0 tx=0).stop 2024-08-19T10:40:43.569 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:40:43.570 INFO:teuthology.orchestra.run.smithi089.stdout: "ret": 0, 2024-08-19T10:40:43.570 INFO:teuthology.orchestra.run.smithi089.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-08-19T10:40:43.570 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:40:43.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2b6403be00 msgr2=0x7f2b6403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7fd2b640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2b6403be00 0x7f2b6403e2c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2b7813ec60 tx=0x7f2b6c008040 comp rx=0 tx=0).stop 2024-08-19T10:40:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7fd2b640 10 monclient: shutdown 2024-08-19T10:40:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7fd2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/2693061185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b78139620 msgr2=0x7f2b7809c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7fd2b640 1 --2- 172.21.15.89:0/2693061185 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2b78139620 0x7f2b7809c9e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b68006e90 tx=0x7f2b6800d9e0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7e2c8640 1 -- 172.21.15.89:0/2693061185 reap_dead start 2024-08-19T10:40:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7e2c8640 1 -- 172.21.15.89:0/2693061185 reap_dead start 2024-08-19T10:40:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/2693061185 shutdown_connections 2024-08-19T10:40:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/2693061185 >> 172.21.15.89:0/2693061185 conn(0x7f2b78074b40 msgr2=0x7f2b780a8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/2693061185 shutdown_connections 2024-08-19T10:40:43.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.558+0000 7f2b7fd2b640 1 -- 172.21.15.89:0/2693061185 wait complete. 2024-08-19T10:40:43.577 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-08-19T10:40:43.578 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-08-19T10:40:43.578 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:40:43.600 INFO:tasks.ceph.mon.g:Stopped 2024-08-19T10:40:43.600 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-08-19T10:40:43.600 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:40:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient: get_monmap_and_config 2024-08-19T10:40:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient: build_initial_monmap 2024-08-19T10:40:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient: monmap: 2024-08-19T10:40:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:43.714607+0000 2024-08-19T10:40:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:43.714607+0000 2024-08-19T10:40:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:43.714 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:43.715 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:43.715 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:43.715 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:43.715 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:43.715 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:43.715 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:43.715 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:43.715 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:43.715 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:43.715 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 1 Processor -- start 2024-08-19T10:40:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 1 -- start start 2024-08-19T10:40:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient: init 2024-08-19T10:40:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient: _add_conns ranks=[2,6,0,8,5,10,7,3,4,1,9] 2024-08-19T10:40:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e5806c930 0x7f5e5806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient(hunting): picked mon.noname-b con 0x7f5e5806c930 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5e580742b0 0x7f5e5806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient(hunting): picked mon.noname-d con 0x7f5e580742b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5e58137500 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e5806d790 con 0x7f5e58137500 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f5e580a2df0 con 0x7f5e5806c930 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f5e5809f610 con 0x7f5e580742b0 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5d143640 10 monclient(hunting): authenticate will time out at 1093.531078s 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5659c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5e580742b0 0x7f5e5806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5759e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5e58137500 0x7f5e58137940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:59316/0 (socket says 172.21.15.89:59316) 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5759e640 1 -- 172.21.15.89:0/3558571538 learned_addr learned my addr 172.21.15.89:0/3558571538 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:40:43.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 1 --2- 172.21.15.89:0/3558571538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e5806c930 0x7f5e5806cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5659c640 10 monclient(hunting): get_auth_request con 0x7f5e580742b0 auth_method 0 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 10 monclient(hunting): get_auth_request con 0x7f5e5806c930 auth_method 0 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 10 monclient(hunting): handle_auth_done global_id 4997 payload 274 2024-08-19T10:40:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 1 -- 172.21.15.89:0/3558571538 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5e580742b0 msgr2=0x7f5e5806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 1 --2- 172.21.15.89:0/3558571538 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5e580742b0 0x7f5e5806d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 1 -- 172.21.15.89:0/3558571538 >> v1:172.21.15.89:6789/0 conn(0x7f5e58137500 legacy=0x7f5e58137940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:40:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 1 monclient: found mon.noname-b 2024-08-19T10:40:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e56d9d640 1 -- 172.21.15.89:0/3558571538 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e580a5150 con 0x7f5e5806c930 2024-08-19T10:40:43.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.710+0000 7f5e5659c640 1 --2- 172.21.15.89:0/3558571538 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5e580742b0 0x7f5e5806d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:43.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.714+0000 7f5e56d9d640 1 -- 172.21.15.89:0/3558571538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e5806c930 msgr2=0x7f5e5806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.714+0000 7f5e56d9d640 1 --2- 172.21.15.89:0/3558571538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e5806c930 0x7f5e5806cd10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e480097f0 tx=0x7f5e4801c1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ee9d0d1157b6f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 1 -- 172.21.15.89:0/3558571538 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5e480b1070 con 0x7f5e5806c930 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 10 monclient: dump: 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.725 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 5 monclient: authenticate success, global_id 4997 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 1 -- 172.21.15.89:0/3558571538 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e480a4c90 con 0x7f5e5806c930 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 4 set_mon_vals no callback set 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 1 -- 172.21.15.89:0/3558571538 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5e480a4e40 con 0x7f5e5806c930 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e55d9b640 10 monclient: dump: 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.727 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient: shutdown 2024-08-19T10:40:43.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 -- 172.21.15.89:0/3558571538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e5806c930 msgr2=0x7f5e5806cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 --2- 172.21.15.89:0/3558571538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e5806c930 0x7f5e5806cd10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e480097f0 tx=0x7f5e4801c1c0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 -- 172.21.15.89:0/3558571538 shutdown_connections 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 --2- 172.21.15.89:0/3558571538 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5e580742b0 0x7f5e5806d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 --2- 172.21.15.89:0/3558571538 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5e5806c930 0x7f5e5806cd10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 -- 172.21.15.89:0/3558571538 >> 172.21.15.89:0/3558571538 conn(0x7f5e580a8f60 msgr2=0x7f5e580a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 -- 172.21.15.89:0/3558571538 shutdown_connections 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 -- 172.21.15.89:0/3558571538 wait complete. 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient: build_initial_monmap 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient: monmap: 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:43.729 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:43.730 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:43.730 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:43.730 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.730 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:43.730 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:43.730 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:43.730 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:43.730 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:43.730 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:43.730 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:43.730 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:43.731 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:43.731 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 Processor -- start 2024-08-19T10:40:43.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 -- start start 2024-08-19T10:40:43.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient: init 2024-08-19T10:40:43.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:43.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient: _add_conns ranks=[1,3,7,2,4,0,8,5,6] 2024-08-19T10:40:43.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e5806c930 0x7f5e5811f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient(hunting): picked mon.noname-f con 0x7f5e5806c930 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:40:43.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e580742b0 0x7f5e58136800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient(hunting): picked mon.noname-g con 0x7f5e580742b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:43.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5e58137500 0x7f5e581cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient(hunting): picked mon.noname-i con 0x7f5e58137500 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:43.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58142100 con 0x7f5e5806c930 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5e58141f80 con 0x7f5e580742b0 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5e58142280 con 0x7f5e58137500 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5d143640 10 monclient(hunting): authenticate will time out at 1093.538411s 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5659c640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e580742b0 msgr2=0x7f5e58136800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5659c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e580742b0 0x7f5e58136800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5e58137500 0x7f5e581cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e56d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e5806c930 0x7f5e5811f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5e58137500 0x7f5e581cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:53898/0 (socket says 172.21.15.89:53898) 2024-08-19T10:40:43.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 1 -- 172.21.15.89:0/409063631 learned_addr learned my addr 172.21.15.89:0/409063631 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e56d9d640 10 monclient(hunting): get_auth_request con 0x7f5e5806c930 auth_method 0 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e56d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e56d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e56d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 10 monclient(hunting): get_auth_request con 0x7f5e58137500 auth_method 0 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e56d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e56d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e56d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 10 monclient(hunting): handle_auth_done global_id 14263 payload 824 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 1 -- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e580742b0 msgr2=0x7f5e58136800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e580742b0 0x7f5e58136800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 1 -- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e5806c930 msgr2=0x7f5e5811f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e5806c930 0x7f5e5811f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 10 monclient: _finish_hunting 0 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 1 monclient: found mon.noname-i 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:40:43.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e5759e640 1 -- 172.21.15.89:0/409063631 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e58120eb0 con 0x7f5e58137500 2024-08-19T10:40:43.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.718+0000 7f5e56d9d640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e5806c930 0x7f5e5811f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:43.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5759e640 1 -- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5e58137500 msgr2=0x7f5e581cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5759e640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5e58137500 0x7f5e581cf510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000b970 tx=0x7f5e5000be30 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 1 -- 172.21.15.89:0/409063631 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5e50019070 con 0x7f5e58137500 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 10 monclient: dump: 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.739 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:43.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 1 -- 172.21.15.89:0/409063631 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e50012340 con 0x7f5e58137500 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 1 -- 172.21.15.89:0/409063631 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5e500129a0 con 0x7f5e58137500 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5d143640 5 monclient: authenticate success, global_id 14263 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5d143640 10 monclient: _renew_subs 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5d143640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5d143640 1 -- 172.21.15.89:0/409063631 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e580a4440 con 0x7f5e58137500 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5559a640 4 set_mon_vals no callback set 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e3f7fe640 10 monclient: dump: 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.741 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:43.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5d143640 10 monclient: _renew_subs 2024-08-19T10:40:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5d143640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:40:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5d143640 1 -- 172.21.15.89:0/409063631 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5813e200 con 0x7f5e58137500 2024-08-19T10:40:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5d143640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5d143640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5d143640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:40:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.722+0000 7f5e5d143640 1 -- 172.21.15.89:0/409063631 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e58137940 con 0x7f5e58137500 2024-08-19T10:40:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.726+0000 7f5e3f7fe640 1 -- 172.21.15.89:0/409063631 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5e50012530 con 0x7f5e58137500 2024-08-19T10:40:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.726+0000 7f5e3f7fe640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5e4403be00 0x7f5e4403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.726+0000 7f5e3f7fe640 1 -- 172.21.15.89:0/409063631 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f5e5004ecf0 con 0x7f5e58137500 2024-08-19T10:40:43.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.726+0000 7f5e56d9d640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5e4403be00 0x7f5e4403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:43.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.726+0000 7f5e56d9d640 10 monclient: get_auth_request con 0x7f5e4403be00 auth_method 0 2024-08-19T10:40:43.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.726+0000 7f5e56d9d640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5e4403be00 0x7f5e4403e2c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5e480ab000 tx=0x7f5e480a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:43.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.738+0000 7f5e3f7fe640 1 -- 172.21.15.89:0/409063631 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f5e5004c4f0 con 0x7f5e58137500 2024-08-19T10:40:43.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.738+0000 7f5e3f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:43.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.738+0000 7f5e3f7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.962+0000 7f5e5d143640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:40:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.962+0000 7f5e5d143640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:40:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.962+0000 7f5e5d143640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:40:43.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.962+0000 7f5e5d143640 1 -- 172.21.15.89:0/409063631 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5e58076c60 con 0x7f5e58137500 2024-08-19T10:40:43.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e3f7fe640 1 -- 172.21.15.89:0/409063631 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7f5e5004c870 con 0x7f5e58137500 2024-08-19T10:40:43.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e3f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:40:43.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e3f7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:43.968 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":26,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:40:43.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 -- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5e4403be00 msgr2=0x7f5e4403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5e4403be00 0x7f5e4403e2c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5e480ab000 tx=0x7f5e480a6070 comp rx=0 tx=0).stop 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 10 monclient: shutdown 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 -- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5e58137500 msgr2=0x7f5e581cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5e58137500 0x7f5e581cf510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000b970 tx=0x7f5e5000be30 comp rx=0 tx=0).stop 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 -- 172.21.15.89:0/409063631 shutdown_connections 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5e4403be00 0x7f5e4403e2c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5e58137500 0x7f5e581cf510 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e580742b0 0x7f5e58136800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 --2- 172.21.15.89:0/409063631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e5806c930 0x7f5e5811f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 -- 172.21.15.89:0/409063631 >> 172.21.15.89:0/409063631 conn(0x7f5e580a8f60 msgr2=0x7f5e580a7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:43.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 -- 172.21.15.89:0/409063631 shutdown_connections 2024-08-19T10:40:43.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:43.966+0000 7f5e5d143640 1 -- 172.21.15.89:0/409063631 wait complete. 2024-08-19T10:40:46.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:46.047+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:46.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:46.047+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:46.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:46.047+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:46.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:46.047+0000 7f3f9ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:46.987 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:40:47.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:47.047+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:47.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:47.047+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:47.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:47.047+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:47.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient: get_monmap_and_config 2024-08-19T10:40:47.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient: build_initial_monmap 2024-08-19T10:40:47.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient: monmap: 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:47.118102+0000 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:47.118102+0000 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:47.119 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:47.120 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 1 Processor -- start 2024-08-19T10:40:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 1 -- start start 2024-08-19T10:40:47.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient: init 2024-08-19T10:40:47.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:47.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient: _add_conns ranks=[10,4,5,1,8,0,7,2,6,3,9] 2024-08-19T10:40:47.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f42dc06dfe0 0x7f42dc073fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:47.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient(hunting): picked mon.noname-f con 0x7f42dc06dfe0 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:47.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42dc1362c0 0x7f42dc139290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient(hunting): picked mon.noname-c con 0x7f42dc1362c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42dc1397d0 0x7f42dc139c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient(hunting): picked mon.noname-h con 0x7f42dc1397d0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f42dc136870 con 0x7f42dc06dfe0 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f42dc06ee40 con 0x7f42dc1362c0 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f42dc06f120 con 0x7f42dc1397d0 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42e1b64640 10 monclient(hunting): authenticate will time out at 1096.934946s 2024-08-19T10:40:47.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42dc1362c0 0x7f42dc139290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42dc1362c0 0x7f42dc139290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54164/0 (socket says 172.21.15.89:54164) 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 1 -- 172.21.15.89:0/669925433 learned_addr learned my addr 172.21.15.89:0/669925433 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 10 monclient(hunting): get_auth_request con 0x7f42dc1362c0 auth_method 0 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42db7fe640 1 --2- 172.21.15.89:0/669925433 >> v2:172.21.15.190:3300/0 conn(0x7f42dc06dfe0 0x7f42dc073fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42dbfff640 1 --2- 172.21.15.89:0/669925433 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42dc1397d0 0x7f42dc139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42dbfff640 10 monclient(hunting): get_auth_request con 0x7f42dc1397d0 auth_method 0 2024-08-19T10:40:47.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42dbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 10 monclient(hunting): handle_auth_done global_id 5053 payload 274 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 1 -- 172.21.15.89:0/669925433 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42dc1397d0 msgr2=0x7f42dc139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 1 --2- 172.21.15.89:0/669925433 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42dc1397d0 0x7f42dc139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 1 -- 172.21.15.89:0/669925433 >> v2:172.21.15.190:3300/0 conn(0x7f42dc06dfe0 msgr2=0x7f42dc073fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 1 --2- 172.21.15.89:0/669925433 >> v2:172.21.15.190:3300/0 conn(0x7f42dc06dfe0 0x7f42dc073fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 10 monclient: _finish_hunting 0 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 1 monclient: found mon.noname-c 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:40:47.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42daffd640 1 -- 172.21.15.89:0/669925433 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc06f400 con 0x7f42dc1362c0 2024-08-19T10:40:47.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42db7fe640 10 monclient: get_auth_request con 0x7f42dc06dfe0 auth_method 0 2024-08-19T10:40:47.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.114+0000 7f42db7fe640 1 --2- 172.21.15.89:0/669925433 >> v2:172.21.15.190:3300/0 conn(0x7f42dc06dfe0 0x7f42dc073fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:40:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.118+0000 7f42daffd640 1 -- 172.21.15.89:0/669925433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42dc1362c0 msgr2=0x7f42dc139290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.118+0000 7f42daffd640 1 --2- 172.21.15.89:0/669925433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42dc1362c0 0x7f42dc139290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009870 tx=0x7f42c401c420 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d121d11541c7d7b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.118+0000 7f42da7fc640 1 -- 172.21.15.89:0/669925433 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42c40b0070 con 0x7f42dc1362c0 2024-08-19T10:40:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.118+0000 7f42da7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.118+0000 7f42da7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:40:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.118+0000 7f42da7fc640 10 monclient: dump: 2024-08-19T10:40:47.131 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:47.132 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.118+0000 7f42da7fc640 10 monclient: _finish_auth 0 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.118+0000 7f42da7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.118+0000 7f42da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42da7fc640 1 -- 172.21.15.89:0/669925433 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c40aa810 con 0x7f42dc1362c0 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 5 monclient: authenticate success, global_id 5053 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42da7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42da7fc640 1 -- 172.21.15.89:0/669925433 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42c40a9300 con 0x7f42dc1362c0 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:47.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 4 set_mon_vals no callback set 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42da7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42da7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42da7fc640 10 monclient: dump: 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:47.134 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient: shutdown 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 -- 172.21.15.89:0/669925433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42dc1362c0 msgr2=0x7f42dc139290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 --2- 172.21.15.89:0/669925433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42dc1362c0 0x7f42dc139290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009870 tx=0x7f42c401c420 comp rx=0 tx=0).stop 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 -- 172.21.15.89:0/669925433 shutdown_connections 2024-08-19T10:40:47.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 --2- 172.21.15.89:0/669925433 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f42dc1397d0 0x7f42dc139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 --2- 172.21.15.89:0/669925433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42dc1362c0 0x7f42dc139290 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 --2- 172.21.15.89:0/669925433 >> v2:172.21.15.190:3300/0 conn(0x7f42dc06dfe0 0x7f42dc073fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 -- 172.21.15.89:0/669925433 >> 172.21.15.89:0/669925433 conn(0x7f42dc0aa080 msgr2=0x7f42dc0aa490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 -- 172.21.15.89:0/669925433 shutdown_connections 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 -- 172.21.15.89:0/669925433 wait complete. 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient: build_initial_monmap 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient: monmap: 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:47.136 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 Processor -- start 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 -- start start 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient: init 2024-08-19T10:40:47.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:47.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient: _add_conns ranks=[7,8,2,0,6,5,1,4,3] 2024-08-19T10:40:47.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42dc06dfe0 0x7f42dc13f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:47.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient(hunting): picked mon.noname-i con 0x7f42dc06dfe0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:47.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42dc1362c0 0x7f42dc13f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:47.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient(hunting): picked mon.noname-e con 0x7f42dc1362c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:47.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42dc1397d0 0x7f42dc13b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:47.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient(hunting): picked mon.noname-b con 0x7f42dc1397d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:47.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:47.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f42dc142e40 con 0x7f42dc1397d0 2024-08-19T10:40:47.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f42dc142fc0 con 0x7f42dc06dfe0 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f42dc143140 con 0x7f42dc1362c0 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42e1b64640 10 monclient(hunting): authenticate will time out at 1096.942302s 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42daffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42dc1362c0 0x7f42dc13f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42dc1397d0 0x7f42dc13b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42daffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42dc1362c0 0x7f42dc13f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40584/0 (socket says 172.21.15.89:40584) 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 10 monclient(hunting): get_auth_request con 0x7f42dc1397d0 auth_method 0 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42daffd640 1 -- 172.21.15.89:0/295067485 learned_addr learned my addr 172.21.15.89:0/295067485 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:47.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42daffd640 10 monclient(hunting): get_auth_request con 0x7f42dc1362c0 auth_method 0 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42daffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42daffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42db7fe640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42dc06dfe0 0x7f42dc13f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42db7fe640 10 monclient(hunting): get_auth_request con 0x7f42dc06dfe0 auth_method 0 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42db7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 10 monclient(hunting): handle_auth_done global_id 5006 payload 824 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 1 -- 172.21.15.89:0/295067485 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42dc1362c0 msgr2=0x7f42dc13f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42dc1362c0 0x7f42dc13f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 1 -- 172.21.15.89:0/295067485 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42dc06dfe0 msgr2=0x7f42dc13f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42dc06dfe0 0x7f42dc13f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 10 monclient: _finish_hunting 0 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 1 monclient: found mon.noname-b 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:40:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42dbfff640 1 -- 172.21.15.89:0/295067485 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc13ab20 con 0x7f42dc1397d0 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.122+0000 7f42daffd640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42dc1362c0 0x7f42dc13f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42dbfff640 1 -- 172.21.15.89:0/295067485 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42dc1397d0 msgr2=0x7f42dc13b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42dbfff640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42dc1397d0 0x7f42dc13b290 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f42cc007dc0 tx=0x7f42cc00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 1 -- 172.21.15.89:0/295067485 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42cc014890 con 0x7f42dc1397d0 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 10 monclient: dump: 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:47.142 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 10 monclient: _finish_auth 0 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42e1b64640 5 monclient: authenticate success, global_id 5006 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 1 -- 172.21.15.89:0/295067485 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42cc014a70 con 0x7f42dc1397d0 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42e1b64640 10 monclient: _renew_subs 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42e1b64640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42e1b64640 1 -- 172.21.15.89:0/295067485 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42dc131f90 con 0x7f42dc1397d0 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 1 -- 172.21.15.89:0/295067485 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42cc014c20 con 0x7f42dc1397d0 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d9ffb640 4 set_mon_vals no callback set 2024-08-19T10:40:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 10 monclient: dump: 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42e1b64640 10 monclient: _renew_subs 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42e1b64640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42e1b64640 1 -- 172.21.15.89:0/295067485 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f42dc13a840 con 0x7f42dc1397d0 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42e1b64640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42e1b64640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42e1b64640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42e1b64640 1 -- 172.21.15.89:0/295067485 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42dc139290 con 0x7f42dc1397d0 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 1 -- 172.21.15.89:0/295067485 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f42cc01b030 con 0x7f42dc1397d0 2024-08-19T10:40:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42bc03be00 0x7f42bc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:47.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42d3fff640 1 -- 172.21.15.89:0/295067485 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f42cc019020 con 0x7f42dc1397d0 2024-08-19T10:40:47.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42db7fe640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42bc03be00 0x7f42bc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:47.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.126+0000 7f42db7fe640 10 monclient: get_auth_request con 0x7f42bc03be00 auth_method 0 2024-08-19T10:40:47.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.130+0000 7f42db7fe640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42bc03be00 0x7f42bc03e2c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f42d4006fd0 tx=0x7f42d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:47.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.130+0000 7f42d3fff640 1 -- 172.21.15.89:0/295067485 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f42cc022060 con 0x7f42dc1397d0 2024-08-19T10:40:47.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.130+0000 7f42d3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:47.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.130+0000 7f42d3fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.370+0000 7f42e1b64640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:40:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.370+0000 7f42e1b64640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:40:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.370+0000 7f42e1b64640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:40:47.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.370+0000 7f42e1b64640 1 -- 172.21.15.89:0/295067485 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f42dc064590 con 0x7f42dc1397d0 2024-08-19T10:40:47.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.370+0000 7f42d3fff640 1 -- 172.21.15.89:0/295067485 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7f42cc01b320 con 0x7f42dc1397d0 2024-08-19T10:40:47.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.370+0000 7f42d3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:40:47.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.370+0000 7f42d3fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:47.374 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":26,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:40:47.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 -- 172.21.15.89:0/295067485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42bc03be00 msgr2=0x7f42bc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42bc03be00 0x7f42bc03e2c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f42d4006fd0 tx=0x7f42d4008040 comp rx=0 tx=0).stop 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 10 monclient: shutdown 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 -- 172.21.15.89:0/295067485 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42dc1397d0 msgr2=0x7f42dc13b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42dc1397d0 0x7f42dc13b290 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f42cc007dc0 tx=0x7f42cc00e4b0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 -- 172.21.15.89:0/295067485 shutdown_connections 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42bc03be00 0x7f42bc03e2c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42dc1362c0 0x7f42dc13f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42dc1397d0 0x7f42dc13b290 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 --2- 172.21.15.89:0/295067485 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42dc06dfe0 0x7f42dc13f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 -- 172.21.15.89:0/295067485 >> 172.21.15.89:0/295067485 conn(0x7f42dc0aa080 msgr2=0x7f42dc134360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:47.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 -- 172.21.15.89:0/295067485 shutdown_connections 2024-08-19T10:40:47.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:47.374+0000 7f42e1b64640 1 -- 172.21.15.89:0/295067485 wait complete. 2024-08-19T10:40:48.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:48.047+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:48.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:48.047+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:48.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:48.047+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:49.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:49.051+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:49.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:49.051+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:49.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:49.051+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:50.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:50.051+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:50.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:50.051+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:50.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:50.051+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:50.394 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:40:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient: get_monmap_and_config 2024-08-19T10:40:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient: build_initial_monmap 2024-08-19T10:40:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient: monmap: 2024-08-19T10:40:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:50.518087+0000 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:50.518087+0000 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:50.519 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 1 Processor -- start 2024-08-19T10:40:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 1 -- start start 2024-08-19T10:40:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient: init 2024-08-19T10:40:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient: _add_conns ranks=[7,8,10,6,4,3,1,2,9,5,0] 2024-08-19T10:40:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbf5c0a5270 0x7fbf5c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient(hunting): picked mon.noname-i con 0x7fbf5c0a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a6d30 0x7fbf5c0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient(hunting): picked mon.noname-e con 0x7fbf5c0a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbf5c1395b0 0x7fbf5c139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient(hunting): picked mon.noname-f con 0x7fbf5c1395b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbf5c0a7650 con 0x7fbf5c1395b0 2024-08-19T10:40:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fbf5c0a8710 con 0x7fbf5c0a5270 2024-08-19T10:40:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fbf5c0a5a40 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf625a5640 10 monclient(hunting): authenticate will time out at 1100.334563s 2024-08-19T10:40:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a6d30 0x7fbf5c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a6d30 0x7fbf5c0a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40598/0 (socket says 172.21.15.89:40598) 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 1 -- 172.21.15.89:0/45748946 learned_addr learned my addr 172.21.15.89:0/45748946 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 10 monclient(hunting): get_auth_request con 0x7fbf5c0a6d30 auth_method 0 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5bfff640 1 --2- 172.21.15.89:0/45748946 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbf5c0a5270 0x7fbf5c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5bfff640 10 monclient(hunting): get_auth_request con 0x7fbf5c0a5270 auth_method 0 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf60b42640 1 --2- 172.21.15.89:0/45748946 >> v2:172.21.15.190:3300/0 conn(0x7fbf5c1395b0 0x7fbf5c139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 10 monclient(hunting): handle_auth_done global_id 4940 payload 274 2024-08-19T10:40:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 1 -- 172.21.15.89:0/45748946 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbf5c0a5270 msgr2=0x7fbf5c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 1 --2- 172.21.15.89:0/45748946 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbf5c0a5270 0x7fbf5c0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 1 -- 172.21.15.89:0/45748946 >> v2:172.21.15.190:3300/0 conn(0x7fbf5c1395b0 msgr2=0x7fbf5c139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:40:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 1 --2- 172.21.15.89:0/45748946 >> v2:172.21.15.190:3300/0 conn(0x7fbf5c1395b0 0x7fbf5c139a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 1 monclient: found mon.noname-e 2024-08-19T10:40:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:40:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.514+0000 7fbf5b7fe640 1 -- 172.21.15.89:0/45748946 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c0a81d0 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5b7fe640 1 -- 172.21.15.89:0/45748946 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a6d30 msgr2=0x7fbf5c0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5b7fe640 1 --2- 172.21.15.89:0/45748946 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a6d30 0x7fbf5c0a7110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009870 tx=0x7fbf4c01c420 comp rx=0 tx=0).ready entity=mon.8 client_cookie=aaebed6c22b3a1bb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 1 -- 172.21.15.89:0/45748946 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbf4c0b1070 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:40:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 10 monclient: dump: 2024-08-19T10:40:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:50.535 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:50.536 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 10 monclient: _finish_auth 0 2024-08-19T10:40:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 10 monclient: _check_auth_tickets 2024-08-19T10:40:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 1 -- 172.21.15.89:0/45748946 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4c0ab810 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf625a5640 5 monclient: authenticate success, global_id 4940 2024-08-19T10:40:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf625a5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:40:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 1 -- 172.21.15.89:0/45748946 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbf4c0aa300 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf625a5640 10 monclient: get_monmap_and_config success 2024-08-19T10:40:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf625a5640 4 set_mon_vals no callback set 2024-08-19T10:40:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:40:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.518+0000 7fbf5affd640 10 monclient: dump: 2024-08-19T10:40:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:50.538 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient: shutdown 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 -- 172.21.15.89:0/45748946 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a6d30 msgr2=0x7fbf5c0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 --2- 172.21.15.89:0/45748946 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a6d30 0x7fbf5c0a7110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009870 tx=0x7fbf4c01c420 comp rx=0 tx=0).stop 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 -- 172.21.15.89:0/45748946 shutdown_connections 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 --2- 172.21.15.89:0/45748946 >> v2:172.21.15.190:3300/0 conn(0x7fbf5c1395b0 0x7fbf5c139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 --2- 172.21.15.89:0/45748946 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a6d30 0x7fbf5c0a7110 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 --2- 172.21.15.89:0/45748946 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbf5c0a5270 0x7fbf5c0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 -- 172.21.15.89:0/45748946 >> 172.21.15.89:0/45748946 conn(0x7fbf5c069a80 msgr2=0x7fbf5c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:50.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 -- 172.21.15.89:0/45748946 shutdown_connections 2024-08-19T10:40:50.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 -- 172.21.15.89:0/45748946 wait complete. 2024-08-19T10:40:50.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient: build_initial_monmap 2024-08-19T10:40:50.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient: monmap: 2024-08-19T10:40:50.540 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:50.540 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:40:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:40:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:40:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:40:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:40:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:50.541 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:50.542 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:50.542 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:50.542 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:50.542 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:50.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 Processor -- start 2024-08-19T10:40:50.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 -- start start 2024-08-19T10:40:50.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient: init 2024-08-19T10:40:50.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:50.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient: _add_conns ranks=[8,0,7,5,2,6,3,1,4] 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a5270 0x7fbf5c13efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient(hunting): picked mon.noname-e con 0x7fbf5c0a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf5c0a6d30 0x7fbf5c0a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient(hunting): picked mon.noname-a con 0x7fbf5c0a6d30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbf5c1395b0 0x7fbf5c1d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient(hunting): picked mon.noname-i con 0x7fbf5c1395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c0a5650 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fbf5c13a350 con 0x7fbf5c1395b0 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:50.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fbf5c0a57d0 con 0x7fbf5c0a5270 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf625a5640 10 monclient(hunting): authenticate will time out at 1100.342068s 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a5270 0x7fbf5c13efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf5c0a6d30 0x7fbf5c0a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a5270 0x7fbf5c13efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40600/0 (socket says 172.21.15.89:40600) 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5bfff640 1 -- 172.21.15.89:0/701814536 learned_addr learned my addr 172.21.15.89:0/701814536 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5bfff640 10 monclient(hunting): get_auth_request con 0x7fbf5c0a5270 auth_method 0 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 10 monclient(hunting): get_auth_request con 0x7fbf5c0a6d30 auth_method 0 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:50.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:40:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:40:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:40:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 10 monclient(hunting): handle_auth_done global_id 5103 payload 824 2024-08-19T10:40:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 1 -- 172.21.15.89:0/701814536 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a5270 msgr2=0x7fbf5c13efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:50.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a5270 0x7fbf5c13efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 1 -- 172.21.15.89:0/701814536 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbf5c1395b0 msgr2=0x7fbf5c1d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbf5c1395b0 0x7fbf5c1d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 1 monclient: found mon.noname-a 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5b7fe640 1 -- 172.21.15.89:0/701814536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c13f910 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.522+0000 7fbf5bfff640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a5270 0x7fbf5c13efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf5b7fe640 1 -- 172.21.15.89:0/701814536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf5c0a6d30 msgr2=0x7fbf5c0a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf5b7fe640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf5c0a6d30 0x7fbf5c0a1d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0ac000 tx=0x7fbf4c0191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 1 -- 172.21.15.89:0/701814536 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbf4c0b9020 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 10 monclient: dump: 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:50.547 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:50.548 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:50.548 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:50.548 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:50.548 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:50.548 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:50.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 10 monclient: _finish_auth 0 2024-08-19T10:40:50.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:40:50.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:50.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 1 -- 172.21.15.89:0/701814536 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4c004250 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf625a5640 5 monclient: authenticate success, global_id 5103 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf625a5640 10 monclient: _renew_subs 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf625a5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf625a5640 1 -- 172.21.15.89:0/701814536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5c09e050 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 1 -- 172.21.15.89:0/701814536 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbf4c0b1040 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 10 monclient: dump: 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:50.549 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:50.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf625a5640 10 monclient: _renew_subs 2024-08-19T10:40:50.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf5a7fc640 4 set_mon_vals no callback set 2024-08-19T10:40:50.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf625a5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:50.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf625a5640 1 -- 172.21.15.89:0/701814536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5c09f290 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf625a5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:50.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf625a5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:50.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf625a5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:50.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 1 -- 172.21.15.89:0/701814536 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbf4c003620 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf625a5640 1 -- 172.21.15.89:0/701814536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf5c09f6c0 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf4803be00 0x7fbf4803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:50.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf58ff9640 1 -- 172.21.15.89:0/701814536 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7fbf4c0e8230 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf5bfff640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf4803be00 0x7fbf4803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:50.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.526+0000 7fbf5bfff640 10 monclient: get_auth_request con 0x7fbf4803be00 auth_method 0 2024-08-19T10:40:50.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.530+0000 7fbf5bfff640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf4803be00 0x7fbf4803e2c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbf50007e70 tx=0x7fbf50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:40:50.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.530+0000 7fbf58ff9640 1 -- 172.21.15.89:0/701814536 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fbf4c016130 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.530+0000 7fbf58ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:40:50.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.530+0000 7fbf58ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:40:50.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.766+0000 7fbf625a5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:40:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.766+0000 7fbf625a5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:40:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.766+0000 7fbf625a5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:40:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.766+0000 7fbf625a5640 1 -- 172.21.15.89:0/701814536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbf5c0a7110 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.766+0000 7fbf58ff9640 1 -- 172.21.15.89:0/701814536 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7fbf4c0e69e0 con 0x7fbf5c0a6d30 2024-08-19T10:40:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.766+0000 7fbf58ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:40:50.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.766+0000 7fbf58ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:40:50.770 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":26,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:40:50.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 -- 172.21.15.89:0/701814536 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf4803be00 msgr2=0x7fbf4803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:50.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf4803be00 0x7fbf4803e2c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbf50007e70 tx=0x7fbf50008040 comp rx=0 tx=0).stop 2024-08-19T10:40:50.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 10 monclient: shutdown 2024-08-19T10:40:50.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:40:50.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 -- 172.21.15.89:0/701814536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf5c0a6d30 msgr2=0x7fbf5c0a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:50.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf5c0a6d30 0x7fbf5c0a1d90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0ac000 tx=0x7fbf4c0191c0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 -- 172.21.15.89:0/701814536 shutdown_connections 2024-08-19T10:40:50.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbf5c1395b0 0x7fbf5c1d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf5c0a6d30 0x7fbf5c0a1d90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf4803be00 0x7fbf4803e2c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 --2- 172.21.15.89:0/701814536 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf5c0a5270 0x7fbf5c13efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:50.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 -- 172.21.15.89:0/701814536 >> 172.21.15.89:0/701814536 conn(0x7fbf5c069a80 msgr2=0x7fbf5c0ab130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:40:50.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 -- 172.21.15.89:0/701814536 shutdown_connections 2024-08-19T10:40:50.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:50.770+0000 7fbf625a5640 1 -- 172.21.15.89:0/701814536 wait complete. 2024-08-19T10:40:51.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:51.051+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:51.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:51.051+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:51.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:51.051+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:52.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:52.051+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:52.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:52.051+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:52.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:52.051+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:53.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:53.051+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:53.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:53.051+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:53.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:53.051+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:53.790 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:40:53.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient: get_monmap_and_config 2024-08-19T10:40:53.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient: build_initial_monmap 2024-08-19T10:40:53.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient: monmap: 2024-08-19T10:40:53.918 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:40:53.918 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:40:53.918883+0000 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:40:53.918883+0000 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:40:53.919 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:40:53.920 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:40:53.920 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:40:53.920 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:40:53.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 1 Processor -- start 2024-08-19T10:40:53.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 1 -- start start 2024-08-19T10:40:53.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient: init 2024-08-19T10:40:53.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient: _reopen_session rank -1 2024-08-19T10:40:53.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient: _add_conns ranks=[5,2,0,10,8,7,6,4,3,1,9] 2024-08-19T10:40:53.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:53.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient(hunting): picked mon.noname-h con 0x7f568c0a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:40:53.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:53.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient(hunting): picked mon.noname-b con 0x7f568c0a7180 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f568c139620 addr v1:172.21.15.89:6789/0 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f568c0a33d0 con 0x7f568c139620 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f568c0a8190 con 0x7f568c0a7180 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f568c0a2ec0 con 0x7f568c0a6860 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f56912bb640 10 monclient(hunting): authenticate will time out at 1103.735538s 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568b7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f568c139620 0x7f568c139a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:38904/0 (socket says 172.21.15.89:38904) 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568b7fe640 1 -- 172.21.15.89:0/735514531 learned_addr learned my addr 172.21.15.89:0/735514531 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f568c0a7180 auth_method 0 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:53.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 msgr2=0x7f568c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 msgr2=0x7f568c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f568c0a6860 auth_method 0 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.914+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.918+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 msgr2=0x7f568c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.918+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 msgr2=0x7f568c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.918+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:53.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.918+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:53.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:53.918+0000 7f568b7fe640 1 -- 172.21.15.89:0/735514531 >> v1:172.21.15.89:6789/0 conn(0x7f568c139620 legacy=0x7f568c139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:54.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:54.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:54.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:54.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:54.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.118+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.118+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f568c0a7180 auth_method 0 2024-08-19T10:40:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.118+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.118+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.118+0000 7f568a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.118+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 msgr2=0x7f568c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.118+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 msgr2=0x7f568c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:54.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.118+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:54.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.118+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:40:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.122+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.122+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f568c0a6860 auth_method 0 2024-08-19T10:40:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.122+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.122+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:54.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.122+0000 7f568affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.122+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 msgr2=0x7f568c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.122+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 msgr2=0x7f568c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.122+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:54.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.122+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:40:54.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.522+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:54.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.522+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f568c0a7180 auth_method 0 2024-08-19T10:40:54.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.522+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:54.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.522+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:54.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.522+0000 7f568a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:54.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.522+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 msgr2=0x7f568c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:54.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.522+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 msgr2=0x7f568c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:54.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.522+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:54.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.522+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:40:54.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.526+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:54.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.526+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f568c0a6860 auth_method 0 2024-08-19T10:40:54.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.526+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:54.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.526+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:54.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.526+0000 7f568affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:54.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.526+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 msgr2=0x7f568c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:54.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.526+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 msgr2=0x7f568c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:54.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.526+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:54.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:54.526+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:40:55.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:55.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:55.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:55.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:55.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:55.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:55.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.326+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:55.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.326+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f568c0a7180 auth_method 0 2024-08-19T10:40:55.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.326+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:55.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.326+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:55.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.326+0000 7f568a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:55.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.326+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 msgr2=0x7f568c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:55.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.326+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 msgr2=0x7f568c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:55.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.326+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:55.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.326+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:40:55.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.330+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:55.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.330+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f568c0a6860 auth_method 0 2024-08-19T10:40:55.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.330+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:55.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.330+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:55.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.330+0000 7f568affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:55.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.330+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 msgr2=0x7f568c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:55.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.330+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 msgr2=0x7f568c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:55.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.330+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:55.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:55.330+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:40:56.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:56.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:56.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:56.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:56.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:56.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:56.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:56.050+0000 7f3f9ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:40:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 10 monclient(hunting): tick 2024-08-19T10:40:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:40:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 1 monclient(hunting): continuing hunt 2024-08-19T10:40:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:40:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 msgr2=0x7f568c0a6c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c0a6860 0x7f568c0a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 msgr2=0x7f568c0a7560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c0a7180 0x7f568c0a7560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 >> v1:172.21.15.89:6789/0 conn(0x7f568c139620 legacy=0x7f568c139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 10 monclient: _add_conns ranks=[10,6,1,9,7,8,4,3] 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f566c0038b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f566c006300 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f566c008e30 addr v1:172.21.15.190:6789/0 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f568c0a4cd0 con 0x7f566c008e30 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f568c0a8190 con 0x7f566c0038b0 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:56.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f568c0a2ec0 con 0x7f566c006300 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.914+0000 7f5688ff9640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f566c006300 auth_method 0 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f566c0038b0 auth_method 0 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 msgr2=0x7f566c008770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:40:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 msgr2=0x7f566c008770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 msgr2=0x7f566c005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 msgr2=0x7f566c005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:56.918+0000 7f568b7fe640 1 -- 172.21.15.89:0/735514531 >> v1:172.21.15.190:6789/0 conn(0x7f566c008e30 legacy=0x7f566c00b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:40:57.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:57.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:57.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:57.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:57.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:57.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:57.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:57.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f566c006300 auth_method 0 2024-08-19T10:40:57.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:57.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 msgr2=0x7f566c008770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 msgr2=0x7f566c008770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f566c0038b0 auth_method 0 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 msgr2=0x7f566c005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 msgr2=0x7f566c005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:57.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.121+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:40:57.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:57.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f566c006300 auth_method 0 2024-08-19T10:40:57.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 msgr2=0x7f566c008770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 msgr2=0x7f566c008770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f566c0038b0 auth_method 0 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:57.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 msgr2=0x7f566c005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:40:57.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 msgr2=0x7f566c005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:57.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:57.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:57.525+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:40:58.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:58.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:58.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:58.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:58.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:58.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:58.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.049+0000 7f5689ffb640 1 -- 172.21.15.89:0/735514531 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1260896724 0 0) 0x7f568c0a4cd0 con 0x7f566c008e30 2024-08-19T10:40:58.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.049+0000 7f5689ffb640 10 monclient(hunting): _negotiate 2024-08-19T10:40:58.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.049+0000 7f5689ffb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:58.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.049+0000 7f5689ffb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:58.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.049+0000 7f5689ffb640 10 monclient(hunting): my global_id is 14239 2024-08-19T10:40:58.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.049+0000 7f5689ffb640 1 -- 172.21.15.89:0/735514531 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5678003530 con 0x7f566c008e30 2024-08-19T10:40:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f566c006300 auth_method 0 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 msgr2=0x7f566c008770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 msgr2=0x7f566c008770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f566c0038b0 auth_method 0 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:40:58.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 msgr2=0x7f566c005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:40:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 msgr2=0x7f566c005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:58.329+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:40:59.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:59.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:40:59.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:59.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:40:59.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:40:59.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:40:59.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f5688ff9640 10 monclient(hunting): tick 2024-08-19T10:40:59.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f5688ff9640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:40:59.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f5688ff9640 1 monclient(hunting): continuing hunt 2024-08-19T10:40:59.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f5688ff9640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:40:59.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 msgr2=0x7f566c008770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:59.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f566c006300 0x7f566c008770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:59.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 msgr2=0x7f566c005d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:40:59.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.190:3300/0 conn(0x7f566c0038b0 0x7f566c005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:40:59.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 >> v1:172.21.15.190:6789/0 conn(0x7f566c008e30 legacy=0x7f566c00b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:40:59.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f5688ff9640 10 monclient: _add_conns ranks=[7,9,3,4,8] 2024-08-19T10:40:59.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f568b7fe640 1 -- 172.21.15.89:0/735514531 reap_dead start 2024-08-19T10:40:59.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.913+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:59.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f566c00ced0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:40:59.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568b7fe640 1 -- 172.21.15.89:0/735514531 reap_dead start 2024-08-19T10:40:59.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:59.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f566c003cf0 addr v2:172.21.15.89:3300/0 2024-08-19T10:40:59.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f566c006ca0 0x7f566c007120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:40:59.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f566c006ca0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:40:59.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:59.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f568c0a8190 con 0x7f566c003cf0 2024-08-19T10:40:59.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f568c0a2ec0 con 0x7f566c006ca0 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f566c007660 con 0x7f566c00ced0 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f5688ff9640 10 monclient(hunting): _renew_subs 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568b7fe640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f566c006ca0 msgr2=0x7f566c007120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568b7fe640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f566c006ca0 0x7f566c007120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f566c003cf0 auth_method 0 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:40:59.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 msgr2=0x7f566c006760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:40:59.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 msgr2=0x7f566c006760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:59.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:59.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f566c00ced0 auth_method 0 2024-08-19T10:40:59.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:40:59.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:40:59.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:40:59.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:40:59.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 msgr2=0x7f566c00d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:40:59.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 msgr2=0x7f566c00d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:40:59.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:40:59.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:40:59.917+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:00.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:41:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:00.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:00.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:00.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568b7fe640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f566c006ca0 msgr2=0x7f566c007120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:00.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568b7fe640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f566c006ca0 0x7f566c007120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:41:00.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f566c003cf0 auth_method 0 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 msgr2=0x7f566c006760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 msgr2=0x7f566c006760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f566c00ced0 auth_method 0 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:00.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:41:00.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 msgr2=0x7f566c00d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:41:00.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 msgr2=0x7f566c00d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:41:00.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:41:00.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.121+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:41:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568b7fe640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f566c006ca0 msgr2=0x7f566c007120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568b7fe640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f566c006ca0 0x7f566c007120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:41:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:00.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f566c00ced0 auth_method 0 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 msgr2=0x7f566c00d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 msgr2=0x7f566c00d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f566c003cf0 auth_method 0 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:41:00.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 msgr2=0x7f566c006760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:41:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 msgr2=0x7f566c006760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:41:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:41:00.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:00.525+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:41:01.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:01.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:41:01.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:01.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:01.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:01.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:01.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568b7fe640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f566c006ca0 msgr2=0x7f566c007120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:01.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568b7fe640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f566c006ca0 0x7f566c007120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:41:01.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:01.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:01.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f566c00ced0 auth_method 0 2024-08-19T10:41:01.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:01.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:01.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:41:01.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f566c003cf0 auth_method 0 2024-08-19T10:41:01.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:41:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 msgr2=0x7f566c006760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:41:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 msgr2=0x7f566c006760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:41:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:41:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 msgr2=0x7f566c00d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:41:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:41:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 msgr2=0x7f566c00d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:41:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:41:01.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:01.329+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:41:02.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:02.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:41:02.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:02.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:02.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:02.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:02.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 10 monclient(hunting): tick 2024-08-19T10:41:02.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 1 monclient(hunting): continuing hunt 2024-08-19T10:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 msgr2=0x7f566c00d2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f566c00ced0 0x7f566c00d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f566c006ca0 msgr2=0x7f566c007120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f566c006ca0 0x7f566c007120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 msgr2=0x7f566c006760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> v2:172.21.15.89:3300/0 conn(0x7f566c003cf0 0x7f566c006760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 10 monclient: _add_conns ranks=[4,8] 2024-08-19T10:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f566c00bb40 0x7f566c00b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:02.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f566c00bb40 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f566c00bf20 0x7f566c00c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f566c00bf20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f568c0a8190 con 0x7f566c00bb40 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 1 -- 172.21.15.89:0/735514531 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f568c0a2ec0 con 0x7f566c00bf20 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f5688ff9640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f566c00bf20 0x7f566c00c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f566c00bb40 0x7f566c00b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f566c00bf20 auth_method 0 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:02.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f566c00bb40 auth_method 0 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f566c00bb40 msgr2=0x7f566c00b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f566c00bb40 msgr2=0x7f566c00b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f566c00bf20 msgr2=0x7f566c00c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f566c00bb40 0x7f566c00b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568a7fc640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f566c00bf20 msgr2=0x7f566c00c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f566c00bf20 0x7f566c00c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f566c00bf20 0x7f566c00c300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:02.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:02.917+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f566c00bb40 0x7f566c00b2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:03.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:03.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:41:03.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:03.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:03.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:03.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f566c00bb40 0x7f566c00b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568a7fc640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f566c00bf20 0x7f566c00c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f566c00bb40 auth_method 0 2024-08-19T10:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f566c00bf20 auth_method 0 2024-08-19T10:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 10 monclient(hunting): handle_auth_done global_id 5062 payload 274 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f566c00bf20 msgr2=0x7f566c00c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f566c00bf20 0x7f566c00c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 1 monclient: found mon.noname-c 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.121+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c00ddb0 con 0x7f566c00bb40 2024-08-19T10:41:03.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f568affd640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f566c00bb40 msgr2=0x7f566c00b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f568affd640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f566c00bb40 0x7f566c00b2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f567c009eb0 tx=0x7f567c00bf20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d451c490b73c6f31 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 1 -- 172.21.15.89:0/735514531 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f567c0b4070 con 0x7f566c00bb40 2024-08-19T10:41:03.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:03.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 10 monclient: dump: 2024-08-19T10:41:03.135 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.135 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.135 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f56912bb640 5 monclient: authenticate success, global_id 5062 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 1 -- 172.21.15.89:0/735514531 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567c0a7c60 con 0x7f566c00bb40 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f56912bb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f56912bb640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f56912bb640 4 set_mon_vals no callback set 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 1 -- 172.21.15.89:0/735514531 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f567c0a7e10 con 0x7f566c00bb40 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f5689ffb640 10 monclient: dump: 2024-08-19T10:41:03.137 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.138 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.139 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f56912bb640 10 monclient: shutdown 2024-08-19T10:41:03.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f56912bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:03.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f56912bb640 1 -- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f566c00bb40 msgr2=0x7f566c00b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f56912bb640 1 --2- 172.21.15.89:0/735514531 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f566c00bb40 0x7f566c00b2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f567c009eb0 tx=0x7f567c00bf20 comp rx=0 tx=0).stop 2024-08-19T10:41:03.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f568b7fe640 1 -- 172.21.15.89:0/735514531 reap_dead start 2024-08-19T10:41:03.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f56912bb640 1 -- 172.21.15.89:0/735514531 shutdown_connections 2024-08-19T10:41:03.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f56912bb640 1 -- 172.21.15.89:0/735514531 >> 172.21.15.89:0/735514531 conn(0x7f568c074b40 msgr2=0x7f568c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:03.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.129+0000 7f56912bb640 1 -- 172.21.15.89:0/735514531 shutdown_connections 2024-08-19T10:41:03.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 1 -- 172.21.15.89:0/735514531 wait complete. 2024-08-19T10:41:03.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient: build_initial_monmap 2024-08-19T10:41:03.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient: monmap: 2024-08-19T10:41:03.142 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:03.142 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:03.142 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:41:03.142 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 1 Processor -- start 2024-08-19T10:41:03.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 1 -- start start 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient: init 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient: _add_conns ranks=[5,2,6,1,0,4,3,7,8] 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c1d1d90 0x7f568c1d2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient(hunting): picked mon.noname-h con 0x7f568c1d1d90 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c125230 0x7f568c125650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient(hunting): picked mon.noname-b con 0x7f568c125230 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f568c125b90 0x7f568c125ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient(hunting): picked mon.noname-d con 0x7f568c125b90 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f568c13a420 con 0x7f568c125230 2024-08-19T10:41:03.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f568c13c850 con 0x7f568c1d1d90 2024-08-19T10:41:03.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f568c0abe20 con 0x7f568c125b90 2024-08-19T10:41:03.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:03.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f56912bb640 10 monclient(hunting): authenticate will time out at 1112.954479s 2024-08-19T10:41:03.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c125230 0x7f568c125650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568b7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f568c125b90 0x7f568c125ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c125230 0x7f568c125650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55744/0 (socket says 172.21.15.89:55744) 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 1 -- 172.21.15.89:0/1281852853 learned_addr learned my addr 172.21.15.89:0/1281852853 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 10 monclient(hunting): get_auth_request con 0x7f568c125230 auth_method 0 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568b7fe640 10 monclient(hunting): get_auth_request con 0x7f568c125b90 auth_method 0 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568affd640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c1d1d90 0x7f568c1d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568affd640 10 monclient(hunting): get_auth_request con 0x7f568c1d1d90 auth_method 0 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 10 monclient(hunting): handle_auth_done global_id 5015 payload 824 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 1 -- 172.21.15.89:0/1281852853 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f568c125b90 msgr2=0x7f568c125ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f568c125b90 0x7f568c125ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 1 -- 172.21.15.89:0/1281852853 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c1d1d90 msgr2=0x7f568c1d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c1d1d90 0x7f568c1d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:41:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 1 monclient: found mon.noname-b 2024-08-19T10:41:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.133+0000 7f568a7fc640 1 -- 172.21.15.89:0/1281852853 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c128560 con 0x7f568c125230 2024-08-19T10:41:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f568a7fc640 1 -- 172.21.15.89:0/1281852853 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c125230 msgr2=0x7f568c125650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f568a7fc640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c125230 0x7f568c125650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5674017ad0 tx=0x7f5674017f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 1 -- 172.21.15.89:0/1281852853 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5674029070 con 0x7f568c125230 2024-08-19T10:41:03.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:03.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 10 monclient: dump: 2024-08-19T10:41:03.150 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.150 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.150 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.151 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 1 -- 172.21.15.89:0/1281852853 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56740258a0 con 0x7f568c125230 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 1 -- 172.21.15.89:0/1281852853 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f567400f040 con 0x7f568c125230 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 10 monclient: dump: 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.152 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56897fa640 4 set_mon_vals no callback set 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56912bb640 5 monclient: authenticate success, global_id 5015 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56912bb640 10 monclient: _renew_subs 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56912bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56912bb640 1 -- 172.21.15.89:0/1281852853 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c128840 con 0x7f568c125230 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56912bb640 10 monclient: _renew_subs 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56912bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56912bb640 1 -- 172.21.15.89:0/1281852853 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c1373b0 con 0x7f568c125230 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56912bb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56912bb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56912bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56912bb640 1 -- 172.21.15.89:0/1281852853 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568c09cd00 con 0x7f568c125230 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 1 -- 172.21.15.89:0/1281852853 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f567402c070 con 0x7f568c125230 2024-08-19T10:41:03.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f566c03df60 0x7f566c040420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.137+0000 7f56837fe640 1 -- 172.21.15.89:0/1281852853 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f5674050ec0 con 0x7f568c125230 2024-08-19T10:41:03.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.141+0000 7f568affd640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f566c03df60 0x7f566c040420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.141+0000 7f568affd640 10 monclient: get_auth_request con 0x7f566c03df60 auth_method 0 2024-08-19T10:41:03.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.141+0000 7f568affd640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f566c03df60 0x7f566c040420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f567c009eb0 tx=0x7f567c01b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.145+0000 7f56837fe640 1 -- 172.21.15.89:0/1281852853 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f567401f040 con 0x7f568c125230 2024-08-19T10:41:03.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.145+0000 7f56837fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:03.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.145+0000 7f56837fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:03.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.385+0000 7f56912bb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:41:03.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.385+0000 7f56912bb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:41:03.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.385+0000 7f56912bb640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:41:03.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.385+0000 7f56912bb640 1 -- 172.21.15.89:0/1281852853 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f568c0a4cd0 con 0x7f568c125230 2024-08-19T10:41:03.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.385+0000 7f56837fe640 1 -- 172.21.15.89:0/1281852853 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3186 (secure 0 0 0) 0x7f5674014d60 con 0x7f568c125230 2024-08-19T10:41:03.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.385+0000 7f56837fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:41:03.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.385+0000 7f56837fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:03.391 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":40,"quorum":[0,1,2,4,5,6,7,8],"quorum_names":["a","f","b","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:41:03.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 -- 172.21.15.89:0/1281852853 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f566c03df60 msgr2=0x7f566c040420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f566c03df60 0x7f566c040420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f567c009eb0 tx=0x7f567c01b040 comp rx=0 tx=0).stop 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 10 monclient: shutdown 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 -- 172.21.15.89:0/1281852853 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c125230 msgr2=0x7f568c125650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c125230 0x7f568c125650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5674017ad0 tx=0x7f5674017f90 comp rx=0 tx=0).stop 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 -- 172.21.15.89:0/1281852853 shutdown_connections 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f566c03df60 0x7f566c040420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f568c125b90 0x7f568c125ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f568c125230 0x7f568c125650 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 --2- 172.21.15.89:0/1281852853 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f568c1d1d90 0x7f568c1d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 -- 172.21.15.89:0/1281852853 >> 172.21.15.89:0/1281852853 conn(0x7f568c074b40 msgr2=0x7f568c137700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 -- 172.21.15.89:0/1281852853 shutdown_connections 2024-08-19T10:41:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.389+0000 7f56912bb640 1 -- 172.21.15.89:0/1281852853 wait complete. 2024-08-19T10:41:03.409 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-08-19T10:41:03.409 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-08-19T10:41:03.409 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:41:03.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.513+0000 7f88b0703640 10 monclient: get_monmap_and_config 2024-08-19T10:41:03.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.513+0000 7f88b0703640 10 monclient: build_initial_monmap 2024-08-19T10:41:03.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.513+0000 7f88b0703640 10 monclient: monmap: 2024-08-19T10:41:03.520 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:03.520 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.520 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:41:03.520742+0000 2024-08-19T10:41:03.520 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:41:03.520742+0000 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:03.521 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 1 Processor -- start 2024-08-19T10:41:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 1 -- start start 2024-08-19T10:41:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 10 monclient: init 2024-08-19T10:41:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 10 monclient: _add_conns ranks=[10,2,7,3,4,1,6,5,0,9,8] 2024-08-19T10:41:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f88a80a3620 0x7f88a80a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 10 monclient(hunting): picked mon.noname-f con 0x7f88a80a3620 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f88a80a3f40 0x7f88a81363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 10 monclient(hunting): picked mon.noname-b con 0x7f88a80a3f40 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:03.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88a8136920 0x7f88a8136d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 10 monclient(hunting): picked mon.noname-i con 0x7f88a8136920 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f88a809f390 con 0x7f88a80a3620 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f88a80a0640 con 0x7f88a80a3f40 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f88a809f670 con 0x7f88a8136920 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88b0703640 10 monclient(hunting): authenticate will time out at 1113.337054s 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f88a80a3f40 0x7f88a81363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f88a80a3f40 0x7f88a81363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55752/0 (socket says 172.21.15.89:55752) 2024-08-19T10:41:03.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 1 -- 172.21.15.89:0/1335742941 learned_addr learned my addr 172.21.15.89:0/1335742941 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:03.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 10 monclient(hunting): get_auth_request con 0x7f88a80a3f40 auth_method 0 2024-08-19T10:41:03.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88ae49f640 1 --2- 172.21.15.89:0/1335742941 >> v2:172.21.15.190:3300/0 conn(0x7f88a80a3620 0x7f88a80a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88ae49f640 10 monclient(hunting): get_auth_request con 0x7f88a80a3620 auth_method 0 2024-08-19T10:41:03.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88ae49f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88ae49f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88ae49f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88aeca0640 1 --2- 172.21.15.89:0/1335742941 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88a8136920 0x7f88a8136d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 10 monclient(hunting): handle_auth_done global_id 5024 payload 274 2024-08-19T10:41:03.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 1 -- 172.21.15.89:0/1335742941 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88a8136920 msgr2=0x7f88a8136d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:03.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 1 --2- 172.21.15.89:0/1335742941 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88a8136920 0x7f88a8136d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 1 -- 172.21.15.89:0/1335742941 >> v2:172.21.15.190:3300/0 conn(0x7f88a80a3620 msgr2=0x7f88a80a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:03.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 1 --2- 172.21.15.89:0/1335742941 >> v2:172.21.15.190:3300/0 conn(0x7f88a80a3620 0x7f88a80a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 10 monclient: _finish_hunting 0 2024-08-19T10:41:03.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 1 monclient: found mon.noname-b 2024-08-19T10:41:03.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:03.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:03.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88adc9e640 1 -- 172.21.15.89:0/1335742941 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a809e170 con 0x7f88a80a3f40 2024-08-19T10:41:03.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88aeca0640 10 monclient: get_auth_request con 0x7f88a8136920 auth_method 0 2024-08-19T10:41:03.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.517+0000 7f88aeca0640 1 --2- 172.21.15.89:0/1335742941 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88a8136920 0x7f88a8136d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:03.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88adc9e640 1 -- 172.21.15.89:0/1335742941 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f88a80a3f40 msgr2=0x7f88a81363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88adc9e640 1 --2- 172.21.15.89:0/1335742941 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f88a80a3f40 0x7f88a81363e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f889c0098a0 tx=0x7f889c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e2445dbc650ab server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 1 -- 172.21.15.89:0/1335742941 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f889c0b0070 con 0x7f88a80a3f40 2024-08-19T10:41:03.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 10 monclient: dump: 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 10 monclient: _finish_auth 0 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 10 monclient: _check_auth_tickets 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 5 monclient: authenticate success, global_id 5024 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 1 -- 172.21.15.89:0/1335742941 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889c0aa810 con 0x7f88a80a3f40 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:03.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 4 set_mon_vals no callback set 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 1 -- 172.21.15.89:0/1335742941 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f889c0a9300 con 0x7f88a80a3f40 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88ad49d640 10 monclient: dump: 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.536 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 10 monclient: shutdown 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 1 -- 172.21.15.89:0/1335742941 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f88a80a3f40 msgr2=0x7f88a81363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 1 --2- 172.21.15.89:0/1335742941 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f88a80a3f40 0x7f88a81363e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f889c0098a0 tx=0x7f889c01c490 comp rx=0 tx=0).stop 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 1 -- 172.21.15.89:0/1335742941 shutdown_connections 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 1 --2- 172.21.15.89:0/1335742941 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88a8136920 0x7f88a8136d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 1 --2- 172.21.15.89:0/1335742941 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f88a80a3f40 0x7f88a81363e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 1 --2- 172.21.15.89:0/1335742941 >> v2:172.21.15.190:3300/0 conn(0x7f88a80a3620 0x7f88a80a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 1 -- 172.21.15.89:0/1335742941 >> 172.21.15.89:0/1335742941 conn(0x7f88a80a8210 msgr2=0x7f88a80a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 1 -- 172.21.15.89:0/1335742941 shutdown_connections 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 1 -- 172.21.15.89:0/1335742941 wait complete. 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 10 monclient: build_initial_monmap 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 10 monclient: monmap: 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:03.538 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 1 Processor -- start 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 1 -- start start 2024-08-19T10:41:03.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 10 monclient: init 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.521+0000 7f88b0703640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient: _add_conns ranks=[8,7,1,4,6,3,2,0,5] 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88a80a3620 0x7f88a8135410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient(hunting): picked mon.noname-e con 0x7f88a80a3620 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88a80a3f40 0x7f88a8135950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient(hunting): picked mon.noname-i con 0x7f88a80a3f40 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88a8136920 0x7f88a813a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient(hunting): picked mon.noname-f con 0x7f88a8136920 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f88a80a2080 con 0x7f88a8136920 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f88a80a1f00 con 0x7f88a80a3f40 2024-08-19T10:41:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f88a809c9e0 con 0x7f88a80a3620 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient(hunting): authenticate will time out at 1113.344066s 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88a80a3620 0x7f88a8135410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88a80a3620 0x7f88a8135410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47876/0 (socket says 172.21.15.89:47876) 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 1 -- 172.21.15.89:0/79951456 learned_addr learned my addr 172.21.15.89:0/79951456 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 10 monclient(hunting): get_auth_request con 0x7f88a80a3620 auth_method 0 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 10 monclient(hunting): handle_auth_done global_id 4967 payload 824 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88aeca0640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88a8136920 0x7f88a813a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 1 -- 172.21.15.89:0/79951456 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88a80a3f40 msgr2=0x7f88a8135950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88a80a3f40 0x7f88a8135950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 1 -- 172.21.15.89:0/79951456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88a8136920 msgr2=0x7f88a813a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88a8136920 0x7f88a813a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 10 monclient: _finish_hunting 0 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 1 monclient: found mon.noname-e 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 1 -- 172.21.15.89:0/79951456 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a813c650 con 0x7f88a80a3620 2024-08-19T10:41:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 1 -- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88a80a3620 msgr2=0x7f88a8135410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88ae49f640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88a80a3620 0x7f88a8135410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8898007ef0 tx=0x7f889800e4b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8898013a00 con 0x7f88a80a3620 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 10 monclient: dump: 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 5 monclient: authenticate success, global_id 4967 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8898019070 con 0x7f88a80a3620 2024-08-19T10:41:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient: _renew_subs 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 1 -- 172.21.15.89:0/79951456 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a8136d90 con 0x7f88a80a3620 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8898013be0 con 0x7f88a80a3620 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a7fff640 4 set_mon_vals no callback set 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 10 monclient: dump: 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient: _renew_subs 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 1 -- 172.21.15.89:0/79951456 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a8123ee0 con 0x7f88a80a3620 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a8063b30 0x7f88a8063f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88b0703640 1 -- 172.21.15.89:0/79951456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f88a80a3a00 con 0x7f88a8063b30 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88adc9e640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a8063b30 0x7f88a8063f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8898013590 con 0x7f88a80a3620 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88adc9e640 10 monclient: get_auth_request con 0x7f88a8063b30 auth_method 0 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88adc9e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88adc9e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88adc9e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88adc9e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88adc9e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88adc9e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88adc9e640 10 monclient(hunting): handle_auth_done global_id 5112 payload 824 2024-08-19T10:41:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88a6ffd640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f889403be00 0x7f889403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.525+0000 7f88adc9e640 1 -- 172.21.15.89:0/79951456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88a81363e0 con 0x7f88a8063b30 2024-08-19T10:41:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f889804f720 con 0x7f88a80a3620 2024-08-19T10:41:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88aeca0640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f889403be00 0x7f889403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88aeca0640 10 monclient: get_auth_request con 0x7f889403be00 auth_method 0 2024-08-19T10:41:03.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88adc9e640 1 -- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a8063b30 msgr2=0x7f88a8063f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88adc9e640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a8063b30 0x7f88a8063f10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f889c0ad000 tx=0x7f889c006ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f889c0b0040 con 0x7f88a8063b30 2024-08-19T10:41:03.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 10 monclient: dump: 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.551 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f889c003830 con 0x7f88a8063b30 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a8063b30 msgr2=0x7f88a8063f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a8063b30 0x7f88a8063f10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f889c0ad000 tx=0x7f889c006ee0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a8063b30 msgr2=0x7f88a8063f10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88a6ffd640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a8063b30 0x7f88a8063f10 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f889c0ad000 tx=0x7f889c006ee0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.529+0000 7f88aeca0640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f889403be00 0x7f889403e2c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f88a000b760 tx=0x7f88a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88b0703640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:41:03.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88b0703640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a802d1d0 0x7f88a802d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88b0703640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88b0703640 1 -- 172.21.15.89:0/79951456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8064590 con 0x7f88a802d1d0 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a802d1d0 0x7f88a802d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 10 monclient: get_auth_request con 0x7f88a802d1d0 auth_method 0 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 10 monclient(hunting): handle_auth_done global_id 5121 payload 824 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 1 -- 172.21.15.89:0/79951456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f88a8122670 con 0x7f88a802d1d0 2024-08-19T10:41:03.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 1 -- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a802d1d0 msgr2=0x7f88a802d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88adc9e640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a802d1d0 0x7f88a802d5b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f889c000c10 tx=0x7f889c0a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f889c0b0040 con 0x7f88a802d1d0 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88a6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88a6ffd640 10 monclient: dump: 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.554 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9258 (secure 0 0 0) 0x7f889c0bcc20 con 0x7f88a802d1d0 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88a6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88a6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a802d1d0 msgr2=0x7f88a802d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88a6ffd640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a802d1d0 0x7f88a802d5b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f889c000c10 tx=0x7f889c0a8000 comp rx=0 tx=0).stop 2024-08-19T10:41:03.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88a6ffd640 1 -- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a802d1d0 msgr2=0x7f88a802d5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:03.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88a6ffd640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88a802d1d0 0x7f88a802d5b0 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f889c000c10 tx=0x7f889c0a8000 comp rx=0 tx=0).stop 2024-08-19T10:41:03.556 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:41:03.556 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:41:03.556 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:41:03.556 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:41:03.556 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 40, 2024-08-19T10:41:03.556 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 193350, 2024-08-19T10:41:03.556 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:41:03.556 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:41:03.556 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:41:03.556 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:03.557 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:41:03.558 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.559 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:41:03.560 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:03.561 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.562 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.563 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.565 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:41:03.566 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.567 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.569 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:41:03.570 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.573 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.574 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:41:03.575 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:41:03.576 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.577 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.578 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:03.579 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:03.579 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:03.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.579 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.579 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:41:03.579 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:41:03.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88b0703640 1 -- 172.21.15.89:0/79951456 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f889403be00 msgr2=0x7f889403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88b0703640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f889403be00 0x7f889403e2c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f88a000b760 tx=0x7f88a0008040 comp rx=0 tx=0).stop 2024-08-19T10:41:03.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88b0703640 10 monclient: shutdown 2024-08-19T10:41:03.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88b0703640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:03.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88b0703640 1 -- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88a80a3620 msgr2=0x7f88a8135410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88b0703640 1 --2- 172.21.15.89:0/79951456 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88a80a3620 0x7f88a8135410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8898007ef0 tx=0x7f889800e4b0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.537+0000 7f88aeca0640 1 -- 172.21.15.89:0/79951456 reap_dead start 2024-08-19T10:41:03.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.541+0000 7f88aeca0640 1 -- 172.21.15.89:0/79951456 reap_dead start 2024-08-19T10:41:03.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.541+0000 7f88b0703640 1 -- 172.21.15.89:0/79951456 shutdown_connections 2024-08-19T10:41:03.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.541+0000 7f88b0703640 1 -- 172.21.15.89:0/79951456 >> 172.21.15.89:0/79951456 conn(0x7f88a80a8210 msgr2=0x7f88a80a6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:03.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.541+0000 7f88b0703640 1 -- 172.21.15.89:0/79951456 shutdown_connections 2024-08-19T10:41:03.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.541+0000 7f88b0703640 1 -- 172.21.15.89:0/79951456 wait complete. 2024-08-19T10:41:03.582 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:41:03.582 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-08-19T10:41:03.582 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:41:03.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.689+0000 7f3b73c58640 10 monclient: get_monmap_and_config 2024-08-19T10:41:03.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.689+0000 7f3b73c58640 10 monclient: build_initial_monmap 2024-08-19T10:41:03.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient: monmap: 2024-08-19T10:41:03.696 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:03.696 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:41:03.696922+0000 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:41:03.696922+0000 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:03.697 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:03.698 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:03.698 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:03.698 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 1 Processor -- start 2024-08-19T10:41:03.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 1 -- start start 2024-08-19T10:41:03.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient: init 2024-08-19T10:41:03.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient: _add_conns ranks=[5,2,8,6,3,10,1,7,0,4,9] 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3b6c0a5270 0x7f3b6c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient(hunting): picked mon.noname-h con 0x7f3b6c0a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c0a6d30 0x7f3b6c0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient(hunting): picked mon.noname-b con 0x7f3b6c0a6d30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3b6c1395b0 0x7f3b6c1399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient(hunting): picked mon.noname-e con 0x7f3b6c1395b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3b6c0a7650 con 0x7f3b6c0a6d30 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f3b6c0a8710 con 0x7f3b6c0a5270 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3b6c0a5a40 con 0x7f3b6c1395b0 2024-08-19T10:41:03.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:03.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b73c58640 10 monclient(hunting): authenticate will time out at 1113.513583s 2024-08-19T10:41:03.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3b6c1395b0 0x7f3b6c1399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b711f3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c0a6d30 0x7f3b6c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3b6c1395b0 0x7f3b6c1399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47892/0 (socket says 172.21.15.89:47892) 2024-08-19T10:41:03.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 1 -- 172.21.15.89:0/2232433547 learned_addr learned my addr 172.21.15.89:0/2232433547 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:03.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b711f3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c0a6d30 0x7f3b6c0a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55766/0 (socket says 172.21.15.89:55766) 2024-08-19T10:41:03.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 10 monclient(hunting): get_auth_request con 0x7f3b6c1395b0 auth_method 0 2024-08-19T10:41:03.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b711f3640 10 monclient(hunting): get_auth_request con 0x7f3b6c0a6d30 auth_method 0 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b711f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b711f3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b711f3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b711f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b711f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b711f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b719f4640 1 --2- 172.21.15.89:0/2232433547 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3b6c0a5270 0x7f3b6c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b719f4640 10 monclient(hunting): get_auth_request con 0x7f3b6c0a5270 auth_method 0 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b719f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b719f4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b719f4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 10 monclient(hunting): handle_auth_done global_id 4976 payload 274 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 1 -- 172.21.15.89:0/2232433547 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3b6c0a5270 msgr2=0x7f3b6c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 1 --2- 172.21.15.89:0/2232433547 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3b6c0a5270 0x7f3b6c0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 1 -- 172.21.15.89:0/2232433547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c0a6d30 msgr2=0x7f3b6c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 1 --2- 172.21.15.89:0/2232433547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c0a6d30 0x7f3b6c0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 10 monclient: _finish_hunting 0 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 1 monclient: found mon.noname-e 2024-08-19T10:41:03.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:03.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:03.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b721f5640 1 -- 172.21.15.89:0/2232433547 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6c0a81d0 con 0x7f3b6c1395b0 2024-08-19T10:41:03.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.693+0000 7f3b711f3640 1 --2- 172.21.15.89:0/2232433547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c0a6d30 0x7f3b6c0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:03.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b721f5640 1 -- 172.21.15.89:0/2232433547 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3b6c1395b0 msgr2=0x7f3b6c1399d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b721f5640 1 --2- 172.21.15.89:0/2232433547 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3b6c1395b0 0x7f3b6c1399d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3b640098a0 tx=0x7f3b6401c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=feae2c0dbee7bd6c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 1 -- 172.21.15.89:0/2232433547 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3b640b0070 con 0x7f3b6c1395b0 2024-08-19T10:41:03.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:03.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 10 monclient: dump: 2024-08-19T10:41:03.711 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.711 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.711 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.711 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.711 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.711 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 10 monclient: _finish_auth 0 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 10 monclient: _check_auth_tickets 2024-08-19T10:41:03.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 5 monclient: authenticate success, global_id 4976 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 1 -- 172.21.15.89:0/2232433547 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b640aa810 con 0x7f3b6c1395b0 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 4 set_mon_vals no callback set 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 1 -- 172.21.15.89:0/2232433547 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3b640aae30 con 0x7f3b6c1395b0 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b709f2640 10 monclient: dump: 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.713 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 10 monclient: shutdown 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 1 -- 172.21.15.89:0/2232433547 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3b6c1395b0 msgr2=0x7f3b6c1399d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 1 --2- 172.21.15.89:0/2232433547 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3b6c1395b0 0x7f3b6c1399d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3b640098a0 tx=0x7f3b6401c440 comp rx=0 tx=0).stop 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 1 -- 172.21.15.89:0/2232433547 shutdown_connections 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 1 --2- 172.21.15.89:0/2232433547 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3b6c1395b0 0x7f3b6c1399d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 1 --2- 172.21.15.89:0/2232433547 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c0a6d30 0x7f3b6c0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 1 --2- 172.21.15.89:0/2232433547 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3b6c0a5270 0x7f3b6c0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 1 -- 172.21.15.89:0/2232433547 >> 172.21.15.89:0/2232433547 conn(0x7f3b6c069a80 msgr2=0x7f3b6c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 1 -- 172.21.15.89:0/2232433547 shutdown_connections 2024-08-19T10:41:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 1 -- 172.21.15.89:0/2232433547 wait complete. 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 10 monclient: build_initial_monmap 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.697+0000 7f3b73c58640 10 monclient: monmap: 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 1 Processor -- start 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 1 -- start start 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient: init 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient: _add_conns ranks=[3,4,5,7,2,8,0,6,1] 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b6c0a5270 0x7f3b6c13cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient(hunting): picked mon.noname-g con 0x7f3b6c0a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b6c0a6d30 0x7f3b6c13d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient(hunting): picked mon.noname-c con 0x7f3b6c0a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3b6c1395b0 0x7f3b6c13d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient(hunting): picked mon.noname-h con 0x7f3b6c1395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3b6c0a57d0 con 0x7f3b6c0a5270 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3b6c0a5650 con 0x7f3b6c0a6d30 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f3b6c143230 con 0x7f3b6c1395b0 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient(hunting): authenticate will time out at 1113.520796s 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b6c0a6d30 0x7f3b6c13d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b719f4640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b6c0a5270 msgr2=0x7f3b6c13cda0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b6c0a6d30 0x7f3b6c13d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48366/0 (socket says 172.21.15.89:48366) 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b719f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b6c0a5270 0x7f3b6c13cda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 1 -- 172.21.15.89:0/3587159232 learned_addr learned my addr 172.21.15.89:0/3587159232 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 10 monclient(hunting): get_auth_request con 0x7f3b6c0a6d30 auth_method 0 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b721f5640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3b6c1395b0 0x7f3b6c13d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b721f5640 10 monclient(hunting): get_auth_request con 0x7f3b6c1395b0 auth_method 0 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b721f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b721f5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b721f5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 10 monclient(hunting): handle_auth_done global_id 5071 payload 824 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 1 -- 172.21.15.89:0/3587159232 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3b6c1395b0 msgr2=0x7f3b6c13d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3b6c1395b0 0x7f3b6c13d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 1 -- 172.21.15.89:0/3587159232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b6c0a5270 msgr2=0x7f3b6c13cda0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b6c0a5270 0x7f3b6c13cda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 10 monclient: _finish_hunting 0 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 1 monclient: found mon.noname-c 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 1 -- 172.21.15.89:0/3587159232 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6c13e700 con 0x7f3b6c0a6d30 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 1 -- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b6c0a6d30 msgr2=0x7f3b6c13d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b711f3640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b6c0a6d30 0x7f3b6c13d2e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00ebe0 tx=0x7f3b5c00c520 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3b5c017070 con 0x7f3b6c0a6d30 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 10 monclient: dump: 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b5c00ce40 con 0x7f3b6c0a6d30 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 5 monclient: authenticate success, global_id 5071 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient: _renew_subs 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 1 -- 172.21.15.89:0/3587159232 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6c140640 con 0x7f3b6c0a6d30 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3b5c013d50 con 0x7f3b6c0a6d30 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b63fff640 4 set_mon_vals no callback set 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b627fc640 10 monclient: dump: 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient: _renew_subs 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.701+0000 7f3b73c58640 1 -- 172.21.15.89:0/3587159232 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6c09cea0 con 0x7f3b6c0a6d30 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b73c58640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b73c58640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1230 0x7f3b6c1d1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b73c58640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b73c58640 1 -- 172.21.15.89:0/3587159232 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3b6c0a7110 con 0x7f3b6c1d1230 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3b5c01a070 con 0x7f3b6c0a6d30 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1230 0x7f3b6c1d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 10 monclient: get_auth_request con 0x7f3b6c1d1230 auth_method 0 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3b5003be00 0x7f3b5003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 10 monclient(hunting): handle_auth_done global_id 5042 payload 824 2024-08-19T10:41:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 1 -- 172.21.15.89:0/3587159232 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b6c064590 con 0x7f3b6c1d1230 2024-08-19T10:41:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f3b5c00f040 con 0x7f3b6c0a6d30 2024-08-19T10:41:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b721f5640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3b5003be00 0x7f3b5003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b721f5640 10 monclient: get_auth_request con 0x7f3b5003be00 auth_method 0 2024-08-19T10:41:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 1 -- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1230 msgr2=0x7f3b6c1d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b719f4640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1230 0x7f3b6c1d1610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3b68007cb0 tx=0x7f3b6800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3b68019070 con 0x7f3b6c1d1230 2024-08-19T10:41:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 10 monclient: dump: 2024-08-19T10:41:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3b680039a0 con 0x7f3b6c1d1230 2024-08-19T10:41:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1230 msgr2=0x7f3b6c1d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1230 0x7f3b6c1d1610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3b68007cb0 tx=0x7f3b6800e4b0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1230 msgr2=0x7f3b6c1d1610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.705+0000 7f3b627fc640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1230 0x7f3b6c1d1610 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3b68007cb0 tx=0x7f3b6800e4b0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.709+0000 7f3b721f5640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3b5003be00 0x7f3b5003e2c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3b640ab000 tx=0x7f3b640a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b73c58640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b73c58640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1b50 0x7f3b6c132750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b73c58640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b73c58640 1 -- 172.21.15.89:0/3587159232 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3b6c09d490 con 0x7f3b6c1d1b50 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b719f4640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1b50 0x7f3b6c132750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b719f4640 10 monclient: get_auth_request con 0x7f3b6c1d1b50 auth_method 0 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b719f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b719f4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b719f4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b719f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b719f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b719f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b719f4640 10 monclient(hunting): handle_auth_done global_id 5051 payload 824 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.713+0000 7f3b719f4640 1 -- 172.21.15.89:0/3587159232 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3b6c135790 con 0x7f3b6c1d1b50 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b719f4640 1 -- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1b50 msgr2=0x7f3b6c132750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b719f4640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1b50 0x7f3b6c132750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3b68007cb0 tx=0x7f3b68004ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3b68019050 con 0x7f3b6c1d1b50 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b627fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b627fc640 10 monclient: dump: 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.733 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.734 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.734 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.734 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9590 (secure 0 0 0) 0x7f3b6801a040 con 0x7f3b6c1d1b50 2024-08-19T10:41:03.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b627fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:41:03.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b627fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:03.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1b50 msgr2=0x7f3b6c132750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b627fc640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1b50 0x7f3b6c132750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3b68007cb0 tx=0x7f3b68004ac0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b627fc640 1 -- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1b50 msgr2=0x7f3b6c132750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:03.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b627fc640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3b6c1d1b50 0x7f3b6c132750 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3b68007cb0 tx=0x7f3b68004ac0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 40, 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 193546, 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:41:03.735 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:03.736 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:03.737 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:41:03.738 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:41:03.739 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:03.740 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:41:03.741 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.742 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.747 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.748 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.749 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:41:03.751 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.754 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:41:03.755 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.757 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.758 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.760 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.761 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:03.762 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:03.762 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:41:03.762 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.762 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.762 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:41:03.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.762 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:03.762 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:03.762 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:41:03.763 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:41:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b73c58640 1 -- 172.21.15.89:0/3587159232 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3b5003be00 msgr2=0x7f3b5003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b73c58640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3b5003be00 0x7f3b5003e2c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3b640ab000 tx=0x7f3b640a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b73c58640 10 monclient: shutdown 2024-08-19T10:41:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b73c58640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b73c58640 1 -- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b6c0a6d30 msgr2=0x7f3b6c13d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b73c58640 1 --2- 172.21.15.89:0/3587159232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3b6c0a6d30 0x7f3b6c13d2e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00ebe0 tx=0x7f3b5c00c520 comp rx=0 tx=0).stop 2024-08-19T10:41:03.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b721f5640 1 -- 172.21.15.89:0/3587159232 reap_dead start 2024-08-19T10:41:03.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b721f5640 1 -- 172.21.15.89:0/3587159232 reap_dead start 2024-08-19T10:41:03.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b73c58640 1 -- 172.21.15.89:0/3587159232 shutdown_connections 2024-08-19T10:41:03.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b73c58640 1 -- 172.21.15.89:0/3587159232 >> 172.21.15.89:0/3587159232 conn(0x7f3b6c069a80 msgr2=0x7f3b6c0aab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:03.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b73c58640 1 -- 172.21.15.89:0/3587159232 shutdown_connections 2024-08-19T10:41:03.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.717+0000 7f3b73c58640 1 -- 172.21.15.89:0/3587159232 wait complete. 2024-08-19T10:41:03.766 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:41:03.766 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-08-19T10:41:03.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:41:03.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient: get_monmap_and_config 2024-08-19T10:41:03.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient: build_initial_monmap 2024-08-19T10:41:03.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient: monmap: 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:41:03.886003+0000 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:41:03.886003+0000 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:03.886 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:03.887 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:03.887 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:03.887 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:03.887 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:03.887 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:03.887 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 1 Processor -- start 2024-08-19T10:41:03.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 1 -- start start 2024-08-19T10:41:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient: init 2024-08-19T10:41:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient: _add_conns ranks=[9,10,4,6,0,8,7,1,2,3,5] 2024-08-19T10:41:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f74d80a5270 0x7f74d80a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient(hunting): picked mon.noname-a con 0x7f74d80a5270 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f74d80a6d30 0x7f74d80a7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient(hunting): picked mon.noname-f con 0x7f74d80a6d30 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 0x7f74d8139a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient(hunting): picked mon.noname-c con 0x7f74d81395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:03.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f74d80a7680 con 0x7f74d80a5270 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f74d80a8710 con 0x7f74d80a6d30 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f74d80a5a40 con 0x7f74d81395b0 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74dd614640 10 monclient(hunting): authenticate will time out at 1113.702582s 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f74d80a5270 0x7f74d80a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d77fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 0x7f74d8139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f74d80a5270 0x7f74d80a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33608/0 (socket says 172.21.15.89:33608) 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 1 -- 172.21.15.89:0/3968259774 learned_addr learned my addr 172.21.15.89:0/3968259774 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 10 monclient(hunting): get_auth_request con 0x7f74d80a5270 auth_method 0 2024-08-19T10:41:03.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d77fe640 10 monclient(hunting): get_auth_request con 0x7f74d81395b0 auth_method 0 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d67fc640 1 --2- 172.21.15.89:0/3968259774 >> v2:172.21.15.190:3300/0 conn(0x7f74d80a6d30 0x7f74d80a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d67fc640 10 monclient(hunting): get_auth_request con 0x7f74d80a6d30 auth_method 0 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d67fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d67fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_done global_id 5130 payload 274 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 1 -- 172.21.15.89:0/3968259774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 msgr2=0x7f74d8139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 1 --2- 172.21.15.89:0/3968259774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 0x7f74d8139a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 1 -- 172.21.15.89:0/3968259774 >> v2:172.21.15.190:3300/0 conn(0x7f74d80a6d30 msgr2=0x7f74d80a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 1 --2- 172.21.15.89:0/3968259774 >> v2:172.21.15.190:3300/0 conn(0x7f74d80a6d30 0x7f74d80a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 1 monclient: found mon.noname-a 2024-08-19T10:41:03.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:03.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:03.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d6ffd640 1 -- 172.21.15.89:0/3968259774 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d80a81d0 con 0x7f74d80a5270 2024-08-19T10:41:03.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.881+0000 7f74d77fe640 1 --2- 172.21.15.89:0/3968259774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 0x7f74d8139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d6ffd640 1 -- 172.21.15.89:0/3968259774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74d80a5270 msgr2=0x7f74d80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d6ffd640 1 --2- 172.21.15.89:0/3968259774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74d80a5270 0x7f74d80a67f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f74d80acc30 tx=0x7f74d001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603e366727d419bd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 1 -- 172.21.15.89:0/3968259774 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74d00090d0 con 0x7f74d80a5270 2024-08-19T10:41:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 10 monclient: dump: 2024-08-19T10:41:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74dd614640 5 monclient: authenticate success, global_id 5130 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 1 -- 172.21.15.89:0/3968259774 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d00a5ba0 con 0x7f74d80a5270 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74dd614640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 1 -- 172.21.15.89:0/3968259774 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74d00a5d50 con 0x7f74d80a5270 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74dd614640 4 set_mon_vals no callback set 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.885+0000 7f74d5ffb640 10 monclient: dump: 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.904 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.905 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.905 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.905 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.905 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient: shutdown 2024-08-19T10:41:03.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:03.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 -- 172.21.15.89:0/3968259774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74d80a5270 msgr2=0x7f74d80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 --2- 172.21.15.89:0/3968259774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74d80a5270 0x7f74d80a67f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f74d80acc30 tx=0x7f74d001c8b0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 -- 172.21.15.89:0/3968259774 shutdown_connections 2024-08-19T10:41:03.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 --2- 172.21.15.89:0/3968259774 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 0x7f74d8139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 --2- 172.21.15.89:0/3968259774 >> v2:172.21.15.190:3300/0 conn(0x7f74d80a6d30 0x7f74d80a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 --2- 172.21.15.89:0/3968259774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74d80a5270 0x7f74d80a67f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 -- 172.21.15.89:0/3968259774 >> 172.21.15.89:0/3968259774 conn(0x7f74d8069a80 msgr2=0x7f74d80abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:03.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 -- 172.21.15.89:0/3968259774 shutdown_connections 2024-08-19T10:41:03.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 -- 172.21.15.89:0/3968259774 wait complete. 2024-08-19T10:41:03.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient: build_initial_monmap 2024-08-19T10:41:03.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient: monmap: 2024-08-19T10:41:03.906 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:03.906 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:03.906 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 Processor -- start 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 -- start start 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient: init 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient: _add_conns ranks=[5,7,4,0,1,3,8,2,6] 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74d80a5270 0x7f74d81d1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient(hunting): picked mon.noname-h con 0x7f74d80a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74d80a6d30 0x7f74d80a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient(hunting): picked mon.noname-i con 0x7f74d80a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 0x7f74d80a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient(hunting): picked mon.noname-c con 0x7f74d81395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f74d80a5650 con 0x7f74d81395b0 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f74d813a360 con 0x7f74d80a5270 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f74d80a57d0 con 0x7f74d80a6d30 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74dd614640 10 monclient(hunting): authenticate will time out at 1113.710009s 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 0x7f74d80a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 0x7f74d80a2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48394/0 (socket says 172.21.15.89:48394) 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 1 -- 172.21.15.89:0/133754954 learned_addr learned my addr 172.21.15.89:0/133754954 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 10 monclient(hunting): get_auth_request con 0x7f74d81395b0 auth_method 0 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d6ffd640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74d80a5270 0x7f74d81d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d6ffd640 10 monclient(hunting): get_auth_request con 0x7f74d80a5270 auth_method 0 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d67fc640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74d80a6d30 0x7f74d80a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 10 monclient(hunting): handle_auth_done global_id 5089 payload 824 2024-08-19T10:41:03.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 1 -- 172.21.15.89:0/133754954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74d80a6d30 msgr2=0x7f74d80a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74d80a6d30 0x7f74d80a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 1 -- 172.21.15.89:0/133754954 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74d80a5270 msgr2=0x7f74d81d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74d80a5270 0x7f74d81d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:03.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 1 monclient: found mon.noname-c 2024-08-19T10:41:03.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:03.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:03.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.889+0000 7f74d77fe640 1 -- 172.21.15.89:0/133754954 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d8139c80 con 0x7f74d81395b0 2024-08-19T10:41:03.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d77fe640 1 -- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 msgr2=0x7f74d80a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d77fe640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 0x7f74d80a2510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e980 tx=0x7f74c800ee40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74c8017070 con 0x7f74d81395b0 2024-08-19T10:41:03.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:03.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 10 monclient: dump: 2024-08-19T10:41:03.912 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.912 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.912 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.912 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.912 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.912 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.912 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.913 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.913 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.913 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.913 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.913 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.913 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.913 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.913 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c800cc10 con 0x7f74d81395b0 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74dd614640 5 monclient: authenticate success, global_id 5089 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74dd614640 10 monclient: _renew_subs 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74dd614640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74dd614640 1 -- 172.21.15.89:0/133754954 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d80a69f0 con 0x7f74d81395b0 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74c800cdc0 con 0x7f74d81395b0 2024-08-19T10:41:03.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:03.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 10 monclient: dump: 2024-08-19T10:41:03.915 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.915 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.915 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.915 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.915 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.915 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74dd614640 10 monclient: _renew_subs 2024-08-19T10:41:03.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d57fa640 4 set_mon_vals no callback set 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74dd614640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74dd614640 1 -- 172.21.15.89:0/133754954 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d809da90 con 0x7f74d81395b0 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74dd614640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74dd614640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d070 0x7f74d813d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74dd614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74dd614640 1 -- 172.21.15.89:0/133754954 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f74d8135820 con 0x7f74d813d070 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d6ffd640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d070 0x7f74d813d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d6ffd640 10 monclient: get_auth_request con 0x7f74d813d070 auth_method 0 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f74c8013470 con 0x7f74d81395b0 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_done global_id 5098 payload 824 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d6ffd640 1 -- 172.21.15.89:0/133754954 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74d80a7140 con 0x7f74d813d070 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f74b803be00 0x7f74b803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f74c800f040 con 0x7f74d81395b0 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d67fc640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f74b803be00 0x7f74b803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.893+0000 7f74d67fc640 10 monclient: get_auth_request con 0x7f74b803be00 auth_method 0 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74d6ffd640 1 -- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d070 msgr2=0x7f74d813d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74d6ffd640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d070 0x7f74d813d450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f74d00ae000 tx=0x7f74d0019650 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74d00bd030 con 0x7f74d813d070 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74cf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74cf7fe640 10 monclient: dump: 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.919 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.920 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.920 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.920 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.920 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f74d0004040 con 0x7f74d813d070 2024-08-19T10:41:03.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74cf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:03.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74cf7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:03.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d070 msgr2=0x7f74d813d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74cf7fe640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d070 0x7f74d813d450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f74d00ae000 tx=0x7f74d0019650 comp rx=0 tx=0).stop 2024-08-19T10:41:03.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d070 msgr2=0x7f74d813d450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:03.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74cf7fe640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d070 0x7f74d813d450 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f74d00ae000 tx=0x7f74d0019650 comp rx=0 tx=0).stop 2024-08-19T10:41:03.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.897+0000 7f74d67fc640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f74b803be00 0x7f74b803e2c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f74d813e7d0 tx=0x7f74c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74dd614640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:41:03.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74dd614640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d990 0x7f74d8137a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:03.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74dd614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:03.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74dd614640 1 -- 172.21.15.89:0/133754954 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f74d8064590 con 0x7f74d813d990 2024-08-19T10:41:03.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74d6ffd640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d990 0x7f74d8137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:03.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74d6ffd640 10 monclient: get_auth_request con 0x7f74d813d990 auth_method 0 2024-08-19T10:41:03.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:03.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74d6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:03.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:03.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:03.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:03.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:03.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74d6ffd640 10 monclient(hunting): handle_auth_done global_id 5107 payload 824 2024-08-19T10:41:03.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.901+0000 7f74d6ffd640 1 -- 172.21.15.89:0/133754954 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f74d80a3fb0 con 0x7f74d813d990 2024-08-19T10:41:03.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74d6ffd640 1 -- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d990 msgr2=0x7f74d8137a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:03.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74d6ffd640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d990 0x7f74d8137a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f74d0000f90 tx=0x7f74d00a7000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:03.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74d00bd040 con 0x7f74d813d990 2024-08-19T10:41:03.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:03.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74cf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:03.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74cf7fe640 10 monclient: dump: 2024-08-19T10:41:03.925 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:03.925 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:03.926 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.926 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:03.926 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:03.926 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:03.926 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:03.927 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:03.927 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:03.927 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:03.927 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:03.927 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:03.927 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:03.927 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:03.928 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:03.928 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:03.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9256 (secure 0 0 0) 0x7f74d00c3810 con 0x7f74d813d990 2024-08-19T10:41:03.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74cf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:41:03.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74cf7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:03.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d990 msgr2=0x7f74d8137a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74cf7fe640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d990 0x7f74d8137a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f74d0000f90 tx=0x7f74d00a7000 comp rx=0 tx=0).stop 2024-08-19T10:41:03.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74cf7fe640 1 -- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d990 msgr2=0x7f74d8137a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:03.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74cf7fe640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d813d990 0x7f74d8137a40 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f74d0000f90 tx=0x7f74d00a7000 comp rx=0 tx=0).stop 2024-08-19T10:41:03.929 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:41:03.930 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:41:03.930 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:41:03.930 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:41:03.930 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 40, 2024-08-19T10:41:03.930 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 193722, 2024-08-19T10:41:03.931 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:41:03.931 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:41:03.931 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:41:03.931 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:41:03.931 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:41:03.931 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:41:03.932 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:41:03.932 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:41:03.932 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:41:03.932 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.933 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:41:03.933 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:03.933 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:41:03.933 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:41:03.933 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:03.933 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:03.934 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:03.934 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:03.934 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:03.934 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:03.934 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:03.935 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:03.935 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:03.935 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:03.935 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:03.935 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.935 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:41:03.936 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:41:03.936 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:03.936 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:03.936 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:03.936 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:03.936 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:03.937 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:03.937 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:03.937 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:03.937 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:03.937 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:03.937 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:03.938 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.938 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:41:03.938 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:41:03.938 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:41:03.938 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:41:03.939 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:41:03.939 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:41:03.939 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:03.939 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:03.939 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:41:03.940 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:41:03.940 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:41:03.940 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:41:03.940 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:41:03.940 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:41:03.940 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:41:03.941 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:03.941 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:41:03.941 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:03.941 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:03.941 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:03.941 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:03.942 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:03.942 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:03.942 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:03.942 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:03.942 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:03.942 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:03.943 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:03.943 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.943 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:41:03.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.943 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:41:03.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.944 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:41:03.944 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:41:03.944 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.945 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.945 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:41:03.945 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:41:03.946 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.946 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.946 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.947 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:41:03.948 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:41:03.951 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:41:03.952 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.954 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:41:03.955 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:41:03.956 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.958 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:41:03.959 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.960 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.961 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:41:03.962 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:41:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74dd614640 1 -- 172.21.15.89:0/133754954 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f74b803be00 msgr2=0x7f74b803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74dd614640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f74b803be00 0x7f74b803e2c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f74d813e7d0 tx=0x7f74c0008040 comp rx=0 tx=0).stop 2024-08-19T10:41:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74dd614640 10 monclient: shutdown 2024-08-19T10:41:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74dd614640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74dd614640 1 -- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 msgr2=0x7f74d80a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74d77fe640 1 -- 172.21.15.89:0/133754954 reap_dead start 2024-08-19T10:41:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74dd614640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 0x7f74d80a2510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e980 tx=0x7f74c800ee40 comp rx=0 tx=0).stop 2024-08-19T10:41:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74dd614640 1 -- 172.21.15.89:0/133754954 shutdown_connections 2024-08-19T10:41:03.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74dd614640 1 --2- 172.21.15.89:0/133754954 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74d81395b0 0x7f74d80a2510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:03.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74dd614640 1 -- 172.21.15.89:0/133754954 >> 172.21.15.89:0/133754954 conn(0x7f74d8069a80 msgr2=0x7f74d81346d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:03.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74dd614640 1 -- 172.21.15.89:0/133754954 shutdown_connections 2024-08-19T10:41:03.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:03.905+0000 7f74dd614640 1 -- 172.21.15.89:0/133754954 wait complete. 2024-08-19T10:41:03.965 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:41:03.965 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-08-19T10:41:03.965 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:41:04.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:04.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:41:04.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:04.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:04.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:04.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient: get_monmap_and_config 2024-08-19T10:41:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient: build_initial_monmap 2024-08-19T10:41:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient: monmap: 2024-08-19T10:41:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.073 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:41:04.073895+0000 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:41:04.073895+0000 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:04.074 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:04.075 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:04.075 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 1 Processor -- start 2024-08-19T10:41:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 1 -- start start 2024-08-19T10:41:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient: init 2024-08-19T10:41:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient: _add_conns ranks=[1,7,10,5,9,6,8,2,0,3,4] 2024-08-19T10:41:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f81cc0a3d70 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81cc069070 0x7f81cc0a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient(hunting): picked mon.noname-i con 0x7f81cc069070 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:04.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81cc0a4860 0x7f81cc13d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient(hunting): picked mon.noname-f con 0x7f81cc0a4860 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81cc0a1eb0 con 0x7f81cc0a3d70 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f81cc0694f0 con 0x7f81cc0a4860 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f81cc0a1610 con 0x7f81cc069070 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81d10cd640 10 monclient(hunting): authenticate will time out at 1113.890753s 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81cc069070 0x7f81cc0a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81cc069070 0x7f81cc0a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:53000/0 (socket says 172.21.15.89:53000) 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 1 -- 172.21.15.89:0/1601647966 learned_addr learned my addr 172.21.15.89:0/1601647966 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:04.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 10 monclient(hunting): get_auth_request con 0x7f81cc069070 auth_method 0 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81cb59e640 1 --2- 172.21.15.89:0/1601647966 >> v2:172.21.15.190:3300/0 conn(0x7f81cc0a4860 0x7f81cc13d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81cb59e640 10 monclient(hunting): get_auth_request con 0x7f81cc0a4860 auth_method 0 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81cb59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81cb59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81cb59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81cb59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81cb59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81cb59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 10 monclient(hunting): handle_auth_done global_id 14290 payload 274 2024-08-19T10:41:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 1 -- 172.21.15.89:0/1601647966 >> v2:172.21.15.190:3300/0 conn(0x7f81cc0a4860 msgr2=0x7f81cc13d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 1 --2- 172.21.15.89:0/1601647966 >> v2:172.21.15.190:3300/0 conn(0x7f81cc0a4860 0x7f81cc13d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 1 -- 172.21.15.89:0/1601647966 >> v1:172.21.15.190:6789/0 conn(0x7f81cc0a3d70 legacy=0x7f81cc0a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 10 monclient: _finish_hunting 0 2024-08-19T10:41:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 1 monclient: found mon.noname-i 2024-08-19T10:41:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:04.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.069+0000 7f81ca59c640 1 -- 172.21.15.89:0/1601647966 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc0a5ca0 con 0x7f81cc069070 2024-08-19T10:41:04.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 1 -- 172.21.15.89:0/1601647966 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81cc069070 msgr2=0x7f81cc0a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 1 --2- 172.21.15.89:0/1601647966 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81cc069070 0x7f81cc0a0970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81c00098a0 tx=0x7f81c001c400 comp rx=0 tx=0).ready entity=mon.7 client_cookie=1045a8e5d990d6d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 1 -- 172.21.15.89:0/1601647966 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81c00b0070 con 0x7f81cc069070 2024-08-19T10:41:04.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 10 monclient: dump: 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.085 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 10 monclient: _finish_auth 0 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 1 -- 172.21.15.89:0/1601647966 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c00a9980 con 0x7f81cc069070 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 5 monclient: authenticate success, global_id 14290 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 1 -- 172.21.15.89:0/1601647966 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81c00aa3a0 con 0x7f81cc069070 2024-08-19T10:41:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 4 set_mon_vals no callback set 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81c9d9b640 10 monclient: dump: 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.087 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient: shutdown 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 -- 172.21.15.89:0/1601647966 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81cc069070 msgr2=0x7f81cc0a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 --2- 172.21.15.89:0/1601647966 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81cc069070 0x7f81cc0a0970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81c00098a0 tx=0x7f81c001c400 comp rx=0 tx=0).stop 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 -- 172.21.15.89:0/1601647966 shutdown_connections 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 --2- 172.21.15.89:0/1601647966 >> v2:172.21.15.190:3300/0 conn(0x7f81cc0a4860 0x7f81cc13d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 --2- 172.21.15.89:0/1601647966 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f81cc069070 0x7f81cc0a0970 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 -- 172.21.15.89:0/1601647966 >> 172.21.15.89:0/1601647966 conn(0x7f81cc0a4e60 msgr2=0x7f81cc0a5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 -- 172.21.15.89:0/1601647966 shutdown_connections 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 -- 172.21.15.89:0/1601647966 wait complete. 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient: build_initial_monmap 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient: monmap: 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:04.089 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 Processor -- start 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 -- start start 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient: init 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient: _add_conns ranks=[5,1,3,0,8,2,6,4,7] 2024-08-19T10:41:04.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81cc069070 0x7f81cc1cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient(hunting): picked mon.noname-h con 0x7f81cc069070 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81cc0a3d70 0x7f81cc13cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient(hunting): picked mon.noname-f con 0x7f81cc0a3d70 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81cc0a4860 0x7f81cc1cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient(hunting): picked mon.noname-g con 0x7f81cc0a4860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc09fbf0 con 0x7f81cc0a3d70 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f81cc09fa70 con 0x7f81cc0a4860 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f81cc09fd70 con 0x7f81cc069070 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81d10cd640 10 monclient(hunting): authenticate will time out at 1113.898579s 2024-08-19T10:41:04.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cb59e640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81cc0a4860 msgr2=0x7f81cc1cb510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cb59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81cc0a4860 0x7f81cc1cb510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81cc0a3d70 0x7f81cc13cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cad9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81cc069070 0x7f81cc1cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cad9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81cc069070 0x7f81cc1cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:44036/0 (socket says 172.21.15.89:44036) 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81cc0a3d70 0x7f81cc13cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:46116/0 (socket says 172.21.15.89:46116) 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cad9d640 1 -- 172.21.15.89:0/187057563 learned_addr learned my addr 172.21.15.89:0/187057563 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cad9d640 10 monclient(hunting): get_auth_request con 0x7f81cc069070 auth_method 0 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 10 monclient(hunting): get_auth_request con 0x7f81cc0a3d70 auth_method 0 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 10 monclient(hunting): handle_auth_done global_id 14275 payload 824 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 1 -- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81cc069070 msgr2=0x7f81cc1cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81cc069070 0x7f81cc1cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 1 -- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81cc0a4860 msgr2=0x7f81cc1cb510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81cc0a4860 0x7f81cc1cb510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 10 monclient: _finish_hunting 0 2024-08-19T10:41:04.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 1 monclient: found mon.noname-f 2024-08-19T10:41:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81ca59c640 1 -- 172.21.15.89:0/187057563 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc0a43e0 con 0x7f81cc0a3d70 2024-08-19T10:41:04.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.077+0000 7f81cad9d640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81cc069070 0x7f81cc1cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:04.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.081+0000 7f81ca59c640 1 -- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81cc0a3d70 msgr2=0x7f81cc13cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.081+0000 7f81ca59c640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81cc0a3d70 0x7f81cc13cf20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f81c00ad000 tx=0x7f81c00191a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81c00b0070 con 0x7f81cc0a3d70 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 10 monclient: dump: 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.097 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:04.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c00b1040 con 0x7f81cc0a3d70 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81d10cd640 5 monclient: authenticate success, global_id 14275 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81d10cd640 10 monclient: _renew_subs 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81d10cd640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81d10cd640 1 -- 172.21.15.89:0/187057563 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81cc0a7860 con 0x7f81cc0a3d70 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81c0004450 con 0x7f81cc0a3d70 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 10 monclient: dump: 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.099 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81d10cd640 10 monclient: _renew_subs 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81d10cd640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:04.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81d10cd640 1 -- 172.21.15.89:0/187057563 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81cc0748d0 con 0x7f81cc0a3d70 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81c959a640 4 set_mon_vals no callback set 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81d10cd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81d10cd640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc13d460 0x7f81cc1cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81d10cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81d10cd640 1 -- 172.21.15.89:0/187057563 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f81cc072b70 con 0x7f81cc13d460 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc13d460 0x7f81cc1cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 10 monclient: get_auth_request con 0x7f81cc13d460 auth_method 0 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 10 monclient(hunting): handle_auth_done global_id 5046 payload 824 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 1 -- 172.21.15.89:0/187057563 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81cc0a0130 con 0x7f81cc13d460 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f81c00bc060 con 0x7f81cc0a3d70 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f81b803bdf0 0x7f81b803e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f81c00e81c0 con 0x7f81cc0a3d70 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cb59e640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f81b803bdf0 0x7f81b803e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cb59e640 10 monclient: get_auth_request con 0x7f81b803bdf0 auth_method 0 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 1 -- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc13d460 msgr2=0x7f81cc1cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.085+0000 7f81cad9d640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc13d460 0x7f81cc1cba50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00fb10 tx=0x7f81bc0045a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81bc018030 con 0x7f81cc13d460 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81b37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:41:04.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81b37fe640 10 monclient: dump: 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.103 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.104 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.104 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f81bc00de50 con 0x7f81cc13d460 2024-08-19T10:41:04.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81b37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:04.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81b37fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:04.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc13d460 msgr2=0x7f81cc1cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81b37fe640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc13d460 0x7f81cc1cba50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00fb10 tx=0x7f81bc0045a0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc13d460 msgr2=0x7f81cc1cba50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:04.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81b37fe640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc13d460 0x7f81cc1cba50 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00fb10 tx=0x7f81bc0045a0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.089+0000 7f81cb59e640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f81b803bdf0 0x7f81b803e2b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009a70 tx=0x7f81c400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81d10cd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:41:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81d10cd640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc1cbf90 0x7f81cc098540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81d10cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81d10cd640 1 -- 172.21.15.89:0/187057563 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f81cc1cc3a0 con 0x7f81cc1cbf90 2024-08-19T10:41:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81cad9d640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc1cbf90 0x7f81cc098540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81cad9d640 10 monclient: get_auth_request con 0x7f81cc1cbf90 auth_method 0 2024-08-19T10:41:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81cad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81cad9d640 10 monclient(hunting): handle_auth_done global_id 5055 payload 824 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.093+0000 7f81cad9d640 1 -- 172.21.15.89:0/187057563 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f81cc064590 con 0x7f81cc1cbf90 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81cad9d640 1 -- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc1cbf90 msgr2=0x7f81cc098540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81cad9d640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc1cbf90 0x7f81cc098540 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f81bc018000 tx=0x7f81bc007f90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81bc007490 con 0x7f81cc1cbf90 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81b37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81b37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81b37fe640 10 monclient: dump: 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.107 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.108 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.109 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9423 (secure 0 0 0) 0x7f81bc0045a0 con 0x7f81cc1cbf90 2024-08-19T10:41:04.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81b37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:41:04.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81b37fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:04.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc1cbf90 msgr2=0x7f81cc098540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81b37fe640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc1cbf90 0x7f81cc098540 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f81bc018000 tx=0x7f81bc007f90 comp rx=0 tx=0).stop 2024-08-19T10:41:04.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81b37fe640 1 -- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc1cbf90 msgr2=0x7f81cc098540 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:04.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81b37fe640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81cc1cbf90 0x7f81cc098540 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f81bc018000 tx=0x7f81bc007f90 comp rx=0 tx=0).stop 2024-08-19T10:41:04.109 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:41:04.109 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 40, 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 193914, 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:41:04.110 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.111 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.112 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:04.113 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.114 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:41:04.115 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.116 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.116 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.116 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:41:04.116 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.116 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.116 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:41:04.117 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.118 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.119 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:41:04.120 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:41:04.123 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.124 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.125 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.126 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.127 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:41:04.129 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.130 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.131 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.132 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.133 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.133 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:41:04.133 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.133 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.133 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:41:04.133 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:41:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81d10cd640 1 -- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f81b803bdf0 msgr2=0x7f81b803e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81d10cd640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f81b803bdf0 0x7f81b803e2b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009a70 tx=0x7f81c400b040 comp rx=0 tx=0).stop 2024-08-19T10:41:04.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81d10cd640 10 monclient: shutdown 2024-08-19T10:41:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81d10cd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81d10cd640 1 -- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81cc0a3d70 msgr2=0x7f81cc13cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81d10cd640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81cc0a3d70 0x7f81cc13cf20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f81c00ad000 tx=0x7f81c00191a0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81cb59e640 1 -- 172.21.15.89:0/187057563 reap_dead start 2024-08-19T10:41:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81d10cd640 1 -- 172.21.15.89:0/187057563 shutdown_connections 2024-08-19T10:41:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81d10cd640 1 --2- 172.21.15.89:0/187057563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81cc0a3d70 0x7f81cc13cf20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81d10cd640 1 -- 172.21.15.89:0/187057563 >> 172.21.15.89:0/187057563 conn(0x7f81cc0a4e60 msgr2=0x7f81cc09ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81d10cd640 1 -- 172.21.15.89:0/187057563 shutdown_connections 2024-08-19T10:41:04.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.097+0000 7f81d10cd640 1 -- 172.21.15.89:0/187057563 wait complete. 2024-08-19T10:41:04.135 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:41:04.136 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-08-19T10:41:04.136 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:41:04.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient: get_monmap_and_config 2024-08-19T10:41:04.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient: build_initial_monmap 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient: monmap: 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:41:04.253097+0000 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:41:04.253097+0000 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:04.253 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:04.254 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:04.254 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:04.254 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:04.254 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:04.254 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:04.254 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 1 Processor -- start 2024-08-19T10:41:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 1 -- start start 2024-08-19T10:41:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient: init 2024-08-19T10:41:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient: _add_conns ranks=[3,2,4,7,0,6,10,9,5,8,1] 2024-08-19T10:41:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8440a6860 0x7fd8440a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient(hunting): picked mon.noname-g con 0x7fd8440a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:04.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8440a7180 0x7fd8440a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient(hunting): picked mon.noname-b con 0x7fd8440a7180 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd844139620 0x7fd844139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient(hunting): picked mon.noname-c con 0x7fd844139620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd8440a8190 con 0x7fd8440a7180 2024-08-19T10:41:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd8440a33d0 con 0x7fd8440a6860 2024-08-19T10:41:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd8440a2ec0 con 0x7fd844139620 2024-08-19T10:41:04.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84c593640 10 monclient(hunting): authenticate will time out at 1114.069593s 2024-08-19T10:41:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd844139620 0x7fd844139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84a32f640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8440a6860 msgr2=0x7fd8440a6c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd849b2e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8440a7180 0x7fd8440a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84a32f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8440a6860 0x7fd8440a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd844139620 0x7fd844139a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48412/0 (socket says 172.21.15.89:48412) 2024-08-19T10:41:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 1 -- 172.21.15.89:0/3363167714 learned_addr learned my addr 172.21.15.89:0/3363167714 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 10 monclient(hunting): get_auth_request con 0x7fd844139620 auth_method 0 2024-08-19T10:41:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd849b2e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8440a7180 0x7fd8440a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55790/0 (socket says 172.21.15.89:55790) 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd849b2e640 10 monclient(hunting): get_auth_request con 0x7fd8440a7180 auth_method 0 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd849b2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd849b2e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd849b2e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd849b2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd849b2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd849b2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 10 monclient(hunting): handle_auth_done global_id 5116 payload 274 2024-08-19T10:41:04.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 1 -- 172.21.15.89:0/3363167714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8440a6860 msgr2=0x7fd8440a6c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 1 --2- 172.21.15.89:0/3363167714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8440a6860 0x7fd8440a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 1 -- 172.21.15.89:0/3363167714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8440a7180 msgr2=0x7fd8440a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 1 --2- 172.21.15.89:0/3363167714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8440a7180 0x7fd8440a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 10 monclient: _finish_hunting 0 2024-08-19T10:41:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 1 monclient: found mon.noname-c 2024-08-19T10:41:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd84ab30640 1 -- 172.21.15.89:0/3363167714 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8440a4cd0 con 0x7fd844139620 2024-08-19T10:41:04.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.249+0000 7fd849b2e640 1 --2- 172.21.15.89:0/3363167714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8440a7180 0x7fd8440a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:04.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84ab30640 1 -- 172.21.15.89:0/3363167714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd844139620 msgr2=0x7fd844139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84ab30640 1 --2- 172.21.15.89:0/3363167714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd844139620 0x7fd844139a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009e80 tx=0x7fd83c01c930 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c422f66c96381a8a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 1 -- 172.21.15.89:0/3363167714 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd83c0090d0 con 0x7fd844139620 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 10 monclient: dump: 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.269 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 10 monclient: _finish_auth 0 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 10 monclient: _check_auth_tickets 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:04.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 5 monclient: authenticate success, global_id 5116 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 1 -- 172.21.15.89:0/3363167714 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd83c0a5980 con 0x7fd844139620 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 1 -- 172.21.15.89:0/3363167714 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd83c0ad3f0 con 0x7fd844139620 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 4 set_mon_vals no callback set 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84932d640 10 monclient: dump: 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.271 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.272 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 10 monclient: shutdown 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 1 -- 172.21.15.89:0/3363167714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd844139620 msgr2=0x7fd844139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 1 --2- 172.21.15.89:0/3363167714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd844139620 0x7fd844139a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009e80 tx=0x7fd83c01c930 comp rx=0 tx=0).stop 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 1 -- 172.21.15.89:0/3363167714 shutdown_connections 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 1 --2- 172.21.15.89:0/3363167714 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd844139620 0x7fd844139a60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 1 --2- 172.21.15.89:0/3363167714 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd8440a7180 0x7fd8440a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 1 --2- 172.21.15.89:0/3363167714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8440a6860 0x7fd8440a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 1 -- 172.21.15.89:0/3363167714 >> 172.21.15.89:0/3363167714 conn(0x7fd844074b40 msgr2=0x7fd8440abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 1 -- 172.21.15.89:0/3363167714 shutdown_connections 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.253+0000 7fd84c593640 1 -- 172.21.15.89:0/3363167714 wait complete. 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient: build_initial_monmap 2024-08-19T10:41:04.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient: monmap: 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:04.274 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 1 Processor -- start 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 1 -- start start 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient: init 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient: _add_conns ranks=[7,6,0,1,4,2,8,5,3] 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8440a6860 0x7fd8441d1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient(hunting): picked mon.noname-i con 0x7fd8440a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8440a7180 0x7fd8440a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient(hunting): picked mon.noname-d con 0x7fd8440a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd844139620 0x7fd8440a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient(hunting): picked mon.noname-a con 0x7fd844139620 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd844143210 con 0x7fd844139620 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd844143090 con 0x7fd8440a7180 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd844143390 con 0x7fd8440a6860 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient(hunting): authenticate will time out at 1114.076895s 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84ab30640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd844139620 0x7fd8440a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8440a7180 0x7fd8440a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84ab30640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd844139620 0x7fd8440a20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33612/0 (socket says 172.21.15.89:33612) 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84ab30640 1 -- 172.21.15.89:0/455717016 learned_addr learned my addr 172.21.15.89:0/455717016 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:04.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84ab30640 10 monclient(hunting): get_auth_request con 0x7fd844139620 auth_method 0 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84ab30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84ab30640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84ab30640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 10 monclient(hunting): get_auth_request con 0x7fd8440a7180 auth_method 0 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84ab30640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84ab30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84ab30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84a32f640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8440a6860 0x7fd8441d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 10 monclient(hunting): handle_auth_done global_id 5064 payload 824 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 1 -- 172.21.15.89:0/455717016 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8440a6860 msgr2=0x7fd8441d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8440a6860 0x7fd8441d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 1 -- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd844139620 msgr2=0x7fd8440a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd844139620 0x7fd8440a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 10 monclient: _finish_hunting 0 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 1 monclient: found mon.noname-d 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 1 -- 172.21.15.89:0/455717016 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84409e2f0 con 0x7fd8440a7180 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84a32f640 10 monclient: get_auth_request con 0x7fd8440a6860 auth_method 0 2024-08-19T10:41:04.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84a32f640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8440a6860 0x7fd8441d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84ab30640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd844139620 0x7fd8440a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 1 -- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8440a7180 msgr2=0x7fd8440a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd849b2e640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8440a7180 0x7fd8440a1b80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd84000e9b0 tx=0x7fd84000ee70 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd840017070 con 0x7fd8440a7180 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 10 monclient: dump: 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.279 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 10 monclient: _finish_auth 0 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:04.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84000cbb0 con 0x7fd8440a7180 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 5 monclient: authenticate success, global_id 5064 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient: _renew_subs 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd84c593640 1 -- 172.21.15.89:0/455717016 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8441403e0 con 0x7fd8440a7180 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd84000cd60 con 0x7fd8440a7180 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd848b2c640 4 set_mon_vals no callback set 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.257+0000 7fd836ffd640 10 monclient: dump: 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.281 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.282 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84c593640 10 monclient: _renew_subs 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84c593640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84c593640 1 -- 172.21.15.89:0/455717016 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84409fdb0 con 0x7fd8440a7180 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84c593640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84c593640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409ee50 0x7fd84409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84c593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84c593640 1 -- 172.21.15.89:0/455717016 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd8440a3b60 con 0x7fd84409ee50 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409ee50 0x7fd84409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 10 monclient: get_auth_request con 0x7fd84409ee50 auth_method 0 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd84001a070 con 0x7fd8440a7180 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd82803be00 0x7fd82803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 10 monclient(hunting): handle_auth_done global_id 4985 payload 824 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 1 -- 172.21.15.89:0/455717016 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8440a96f0 con 0x7fd84409ee50 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7fd84000f040 con 0x7fd8440a7180 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84ab30640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd82803be00 0x7fd82803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84ab30640 10 monclient: get_auth_request con 0x7fd82803be00 auth_method 0 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 1 -- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409ee50 msgr2=0x7fd84409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd84a32f640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409ee50 0x7fd84409c9e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c6f0 tx=0x7fd83800cbb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd838003bd0 con 0x7fd84409ee50 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 10 monclient: dump: 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.285 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd838016070 con 0x7fd84409ee50 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409ee50 msgr2=0x7fd84409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409ee50 0x7fd84409c9e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c6f0 tx=0x7fd83800cbb0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409ee50 msgr2=0x7fd84409c9e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:04.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.261+0000 7fd836ffd640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409ee50 0x7fd84409c9e0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c6f0 tx=0x7fd83800cbb0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.265+0000 7fd84ab30640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd82803be00 0x7fd82803e2c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd83c01c760 tx=0x7fd83c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84c593640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84c593640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409cf20 0x7fd84409d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84c593640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84c593640 1 -- 172.21.15.89:0/455717016 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd84409d870 con 0x7fd84409cf20 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84a32f640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409cf20 0x7fd84409d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84a32f640 10 monclient: get_auth_request con 0x7fd84409cf20 auth_method 0 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84a32f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84a32f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84a32f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84a32f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84a32f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84a32f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84a32f640 10 monclient(hunting): handle_auth_done global_id 4994 payload 824 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.269+0000 7fd84a32f640 1 -- 172.21.15.89:0/455717016 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd8440abe70 con 0x7fd84409cf20 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84a32f640 1 -- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409cf20 msgr2=0x7fd84409d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84a32f640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409cf20 0x7fd84409d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd838015000 tx=0x7fd8380059e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd838007690 con 0x7fd84409cf20 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd836ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd836ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd836ffd640 10 monclient: dump: 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.290 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9256 (secure 0 0 0) 0x7fd8380054f0 con 0x7fd84409cf20 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd836ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:41:04.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd836ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:04.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409cf20 msgr2=0x7fd84409d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd836ffd640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409cf20 0x7fd84409d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd838015000 tx=0x7fd8380059e0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd836ffd640 1 -- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409cf20 msgr2=0x7fd84409d330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:04.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd836ffd640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd84409cf20 0x7fd84409d330 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd838015000 tx=0x7fd8380059e0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.292 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:41:04.292 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:41:04.292 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:41:04.292 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:41:04.292 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 40, 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 194066, 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:41:04.293 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.294 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.295 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:04.296 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.297 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.298 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.299 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.300 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:41:04.301 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:41:04.303 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.304 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.305 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.307 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.309 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.310 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.310 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:41:04.310 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:41:04.310 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.310 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:41:04.311 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.314 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:04.315 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.315 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.315 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:41:04.315 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84c593640 1 -- 172.21.15.89:0/455717016 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd82803be00 msgr2=0x7fd82803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84c593640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd82803be00 0x7fd82803e2c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd83c01c760 tx=0x7fd83c0a7070 comp rx=0 tx=0).stop 2024-08-19T10:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84c593640 10 monclient: shutdown 2024-08-19T10:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84c593640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84c593640 1 -- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8440a7180 msgr2=0x7fd8440a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84c593640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8440a7180 0x7fd8440a1b80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd84000e9b0 tx=0x7fd84000ee70 comp rx=0 tx=0).stop 2024-08-19T10:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84ab30640 1 -- 172.21.15.89:0/455717016 reap_dead start 2024-08-19T10:41:04.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84c593640 1 -- 172.21.15.89:0/455717016 shutdown_connections 2024-08-19T10:41:04.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84c593640 1 --2- 172.21.15.89:0/455717016 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8440a7180 0x7fd8440a1b80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84c593640 1 -- 172.21.15.89:0/455717016 >> 172.21.15.89:0/455717016 conn(0x7fd844074b40 msgr2=0x7fd8440ab3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:04.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84c593640 1 -- 172.21.15.89:0/455717016 shutdown_connections 2024-08-19T10:41:04.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.273+0000 7fd84c593640 1 -- 172.21.15.89:0/455717016 wait complete. 2024-08-19T10:41:04.317 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:41:04.317 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-08-19T10:41:04.317 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:41:04.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient: get_monmap_and_config 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient: build_initial_monmap 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient: monmap: 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:41:04.433242+0000 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:41:04.433242+0000 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:04.433 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:04.434 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:04.434 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 1 Processor -- start 2024-08-19T10:41:04.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 1 -- start start 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient: init 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient: _add_conns ranks=[9,0,1,7,2,3,5,8,4,6,10] 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb140a6860 0x7fbb140a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient(hunting): picked mon.noname-a con 0x7fbb140a6860 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbb140a7180 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbb140a7560 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb140a79d0 con 0x7fbb140a7180 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb140a33d0 con 0x7fbb140a7560 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fbb140a8190 con 0x7fbb140a6860 2024-08-19T10:41:04.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb1a83b640 10 monclient(hunting): authenticate will time out at 1114.249827s 2024-08-19T10:41:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb13fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb140a6860 0x7fbb140a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb13fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbb140a6860 0x7fbb140a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33628/0 (socket says 172.21.15.89:33628) 2024-08-19T10:41:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb13fff640 1 -- 172.21.15.89:0/3487394343 learned_addr learned my addr 172.21.15.89:0/3487394343 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb13fff640 10 monclient(hunting): get_auth_request con 0x7fbb140a6860 auth_method 0 2024-08-19T10:41:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb13fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb13fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.429+0000 7fbb13fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb137fe640 1 -- 172.21.15.89:0/3487394343 >> v1:172.21.15.89:6789/0 conn(0x7fbb140a7180 legacy=0x7fbb14139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb13fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb13fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb13fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb13fff640 10 monclient(hunting): handle_auth_done global_id 5148 payload 274 2024-08-19T10:41:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb13fff640 1 -- 172.21.15.89:0/3487394343 >> v1:172.21.15.190:6789/0 conn(0x7fbb140a7560 legacy=0x7fbb14139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:04.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb13fff640 1 -- 172.21.15.89:0/3487394343 >> v1:172.21.15.89:6789/0 conn(0x7fbb140a7180 legacy=0x7fbb14139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb13fff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb13fff640 1 monclient: found mon.noname-a 2024-08-19T10:41:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb13fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb13fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.433+0000 7fbb13fff640 1 -- 172.21.15.89:0/3487394343 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1413a440 con 0x7fbb140a6860 2024-08-19T10:41:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 1 -- 172.21.15.89:0/3487394343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb140a6860 msgr2=0x7fbb140a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 1 --2- 172.21.15.89:0/3487394343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb140a6860 0x7fbb140a6c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009ed0 tx=0x7fbb0401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e1a4bdc0899fa7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 1 -- 172.21.15.89:0/3487394343 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb040091b0 con 0x7fbb140a6860 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 10 monclient: dump: 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 10 monclient: _finish_auth 0 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 5 monclient: authenticate success, global_id 5148 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 1 -- 172.21.15.89:0/3487394343 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb040a5a80 con 0x7fbb140a6860 2024-08-19T10:41:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 1 -- 172.21.15.89:0/3487394343 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb040a5c30 con 0x7fbb140a6860 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 4 set_mon_vals no callback set 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb12ffd640 10 monclient: dump: 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient: shutdown 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3487394343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb140a6860 msgr2=0x7fbb140a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 --2- 172.21.15.89:0/3487394343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb140a6860 0x7fbb140a6c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009ed0 tx=0x7fbb0401c970 comp rx=0 tx=0).stop 2024-08-19T10:41:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3487394343 shutdown_connections 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 --2- 172.21.15.89:0/3487394343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb140a6860 0x7fbb140a6c40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3487394343 >> 172.21.15.89:0/3487394343 conn(0x7fbb14074b40 msgr2=0x7fbb140abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3487394343 shutdown_connections 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3487394343 wait complete. 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient: build_initial_monmap 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient: monmap: 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 Processor -- start 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 -- start start 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient: init 2024-08-19T10:41:04.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient: _add_conns ranks=[4,3,2,5,7,1,0,6,8] 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb140a6860 0x7fbb141d1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient(hunting): picked mon.noname-c con 0x7fbb140a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb140a7180 0x7fbb141d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient(hunting): picked mon.noname-g con 0x7fbb140a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb140a7560 0x7fbb1413e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient(hunting): picked mon.noname-b con 0x7fbb140a7560 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fbb14145640 con 0x7fbb140a7560 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbb141454c0 con 0x7fbb140a7180 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbb141457c0 con 0x7fbb140a6860 2024-08-19T10:41:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb1a83b640 10 monclient(hunting): authenticate will time out at 1114.258892s 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb140a6860 0x7fbb141d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb137fe640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb140a7180 msgr2=0x7fbb141d1e90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb140a6860 0x7fbb141d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48418/0 (socket says 172.21.15.89:48418) 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 1 -- 172.21.15.89:0/3928974644 learned_addr learned my addr 172.21.15.89:0/3928974644 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 10 monclient(hunting): get_auth_request con 0x7fbb140a6860 auth_method 0 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb137fe640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb140a7180 0x7fbb141d1e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb18dd8640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb140a7560 0x7fbb1413e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb18dd8640 10 monclient(hunting): get_auth_request con 0x7fbb140a7560 auth_method 0 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb18dd8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb18dd8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb18dd8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb18dd8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb18dd8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb18dd8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 10 monclient(hunting): handle_auth_done global_id 5125 payload 824 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 1 -- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb140a7180 msgr2=0x7fbb141d1e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb140a7180 0x7fbb141d1e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 1 -- 172.21.15.89:0/3928974644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb140a7560 msgr2=0x7fbb1413e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb140a7560 0x7fbb1413e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 1 monclient: found mon.noname-c 2024-08-19T10:41:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:04.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.437+0000 7fbb13fff640 1 -- 172.21.15.89:0/3928974644 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb140a2730 con 0x7fbb140a6860 2024-08-19T10:41:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb13fff640 1 -- 172.21.15.89:0/3928974644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb140a6860 msgr2=0x7fbb141d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb13fff640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb140a6860 0x7fbb141d1950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009e20 tx=0x7fbb04019710 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb040bc020 con 0x7fbb140a6860 2024-08-19T10:41:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 10 monclient: dump: 2024-08-19T10:41:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.457 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 10 monclient: _finish_auth 0 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb04003bb0 con 0x7fbb140a6860 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb1a83b640 5 monclient: authenticate success, global_id 5125 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb1a83b640 10 monclient: _renew_subs 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb1a83b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3928974644 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb14122b10 con 0x7fbb140a6860 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb04003d60 con 0x7fbb140a6860 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 10 monclient: dump: 2024-08-19T10:41:04.459 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.460 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb127fc640 4 set_mon_vals no callback set 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb1a83b640 10 monclient: _renew_subs 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb1a83b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3928974644 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb14122830 con 0x7fbb140a6860 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb1a83b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb1a83b640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1413ee50 0x7fbb1413f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb1a83b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3928974644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb140a21e0 con 0x7fbb1413ee50 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbb040bf020 con 0x7fbb140a6860 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbaf803be30 0x7fbaf803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb137fe640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1413ee50 0x7fbb1413f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb137fe640 10 monclient: get_auth_request con 0x7fbb1413ee50 auth_method 0 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb137fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7fbb040e9c90 con 0x7fbb140a6860 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb137fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb18dd8640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbaf803be30 0x7fbaf803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb18dd8640 10 monclient: get_auth_request con 0x7fbaf803be30 auth_method 0 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb137fe640 10 monclient(hunting): handle_auth_done global_id 14284 payload 824 2024-08-19T10:41:04.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.441+0000 7fbb137fe640 1 -- 172.21.15.89:0/3928974644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb140a6c40 con 0x7fbb1413ee50 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb137fe640 1 -- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1413ee50 msgr2=0x7fbb1413f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb137fe640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1413ee50 0x7fbb1413f230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0014c0 tx=0x7fbafc00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb18dd8640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbaf803be30 0x7fbaf803e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006fd0 tx=0x7fbb0c006f60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbafc017070 con 0x7fbb1413ee50 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb10ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb10ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb10ff9640 10 monclient: dump: 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.463 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbafc0075d0 con 0x7fbb1413ee50 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb10ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb10ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:04.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1413ee50 msgr2=0x7fbb1413f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb10ff9640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1413ee50 0x7fbb1413f230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0014c0 tx=0x7fbafc00c810 comp rx=0 tx=0).stop 2024-08-19T10:41:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1413ee50 msgr2=0x7fbb1413f230 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:04.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.445+0000 7fbb10ff9640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1413ee50 0x7fbb1413f230 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0014c0 tx=0x7fbafc00c810 comp rx=0 tx=0).stop 2024-08-19T10:41:04.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb1a83b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:41:04.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb1a83b640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1402d1d0 0x7fbb1402d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb1a83b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3928974644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14064590 con 0x7fbb1402d1d0 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb137fe640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1402d1d0 0x7fbb1402d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb137fe640 10 monclient: get_auth_request con 0x7fbb1402d1d0 auth_method 0 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb137fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb137fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb137fe640 10 monclient(hunting): handle_auth_done global_id 14293 payload 824 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.453+0000 7fbb137fe640 1 -- 172.21.15.89:0/3928974644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbb140a1eb0 con 0x7fbb1402d1d0 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb137fe640 1 -- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1402d1d0 msgr2=0x7fbb1402d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb137fe640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1402d1d0 0x7fbb1402d5e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0014c0 tx=0x7fbafc000bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbafc017040 con 0x7fbb1402d1d0 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb10ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb10ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb10ff9640 10 monclient: dump: 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.468 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9255 (secure 0 0 0) 0x7fbafc012400 con 0x7fbb1402d1d0 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb10ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb10ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1402d1d0 msgr2=0x7fbb1402d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb10ff9640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1402d1d0 0x7fbb1402d5e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0014c0 tx=0x7fbafc000bd0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb10ff9640 1 -- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1402d1d0 msgr2=0x7fbb1402d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:04.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb10ff9640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb1402d1d0 0x7fbb1402d5e0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0014c0 tx=0x7fbafc000bd0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.470 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:41:04.470 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:41:04.470 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:41:04.470 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:41:04.470 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 40, 2024-08-19T10:41:04.470 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 42046, 2024-08-19T10:41:04.470 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:41:04.470 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:41:04.471 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:41:04.472 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:41:04.474 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.475 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:41:04.477 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:41:04.480 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:41:04.481 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.482 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.483 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:41:04.484 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:41:04.485 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.485 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.485 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.485 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:41:04.485 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:41:04.485 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.485 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.485 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.486 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:41:04.486 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.486 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.486 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:41:04.486 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.486 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.486 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.486 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:41:04.487 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.488 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.488 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:41:04.488 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.488 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.488 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.488 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:41:04.488 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:41:04.488 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.490 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.490 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:41:04.490 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.490 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.491 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.491 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:41:04.491 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:41:04.491 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.491 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.491 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.491 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:41:04.492 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:41:04.492 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.492 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.492 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.492 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:41:04.492 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.493 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.494 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.494 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.494 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.494 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:41:04.494 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:41:04.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.494 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.494 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.494 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:41:04.495 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:41:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbaf803be30 msgr2=0x7fbaf803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb1a83b640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbaf803be30 0x7fbaf803e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006fd0 tx=0x7fbb0c006f60 comp rx=0 tx=0).stop 2024-08-19T10:41:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb1a83b640 10 monclient: shutdown 2024-08-19T10:41:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb1a83b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3928974644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb140a6860 msgr2=0x7fbb141d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb1a83b640 1 --2- 172.21.15.89:0/3928974644 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb140a6860 0x7fbb141d1950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009e20 tx=0x7fbb04019710 comp rx=0 tx=0).stop 2024-08-19T10:41:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb18dd8640 1 -- 172.21.15.89:0/3928974644 reap_dead start 2024-08-19T10:41:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb18dd8640 1 -- 172.21.15.89:0/3928974644 reap_dead start 2024-08-19T10:41:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3928974644 shutdown_connections 2024-08-19T10:41:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3928974644 >> 172.21.15.89:0/3928974644 conn(0x7fbb14074b40 msgr2=0x7fbb140a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:04.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3928974644 shutdown_connections 2024-08-19T10:41:04.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.457+0000 7fbb1a83b640 1 -- 172.21.15.89:0/3928974644 wait complete. 2024-08-19T10:41:04.498 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:41:04.498 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-08-19T10:41:04.498 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:41:04.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient: get_monmap_and_config 2024-08-19T10:41:04.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient: build_initial_monmap 2024-08-19T10:41:04.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient: monmap: 2024-08-19T10:41:04.613 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:04.613 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.613 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:41:04.613286+0000 2024-08-19T10:41:04.613 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:41:04.613286+0000 2024-08-19T10:41:04.613 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:04.613 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.613 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:04.613 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:04.613 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:04.614 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:04.614 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:04.614 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:04.614 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:04.614 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:04.614 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:04.614 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:04.614 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:04.614 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 1 Processor -- start 2024-08-19T10:41:04.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 1 -- start start 2024-08-19T10:41:04.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient: init 2024-08-19T10:41:04.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient: _add_conns ranks=[9,2,4,0,7,10,1,5,3,6,8] 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f67c80a5270 0x7f67c80a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient(hunting): picked mon.noname-a con 0x7f67c80a5270 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67c80a6d30 0x7f67c80a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient(hunting): picked mon.noname-b con 0x7f67c80a6d30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c81395b0 0x7f67c8139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient(hunting): picked mon.noname-c con 0x7f67c81395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f67c80a7650 con 0x7f67c80a5270 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f67c80a8710 con 0x7f67c80a6d30 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f67c80a5a40 con 0x7f67c81395b0 2024-08-19T10:41:04.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:04.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67ce0f2640 10 monclient(hunting): authenticate will time out at 1114.429934s 2024-08-19T10:41:04.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c6ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67c80a6d30 0x7f67c80a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f67c80a5270 0x7f67c80a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c6ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67c80a6d30 0x7f67c80a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:55808/0 (socket says 172.21.15.89:55808) 2024-08-19T10:41:04.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c7fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c81395b0 0x7f67c8139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c6ffd640 1 -- 172.21.15.89:0/433334782 learned_addr learned my addr 172.21.15.89:0/433334782 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:04.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c6ffd640 10 monclient(hunting): get_auth_request con 0x7f67c80a6d30 auth_method 0 2024-08-19T10:41:04.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 10 monclient(hunting): get_auth_request con 0x7f67c80a5270 auth_method 0 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c7fff640 10 monclient(hunting): get_auth_request con 0x7f67c81395b0 auth_method 0 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 10 monclient(hunting): handle_auth_done global_id 5166 payload 274 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 1 -- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c81395b0 msgr2=0x7f67c8139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 1 --2- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c81395b0 0x7f67c8139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 1 -- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67c80a6d30 msgr2=0x7f67c80a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 1 --2- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67c80a6d30 0x7f67c80a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:04.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 1 monclient: found mon.noname-a 2024-08-19T10:41:04.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:04.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:04.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c77fe640 1 -- 172.21.15.89:0/433334782 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c80a81d0 con 0x7f67c80a5270 2024-08-19T10:41:04.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.609+0000 7f67c6ffd640 1 --2- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67c80a6d30 0x7f67c80a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c77fe640 1 -- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67c80a5270 msgr2=0x7f67c80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c77fe640 1 --2- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67c80a5270 0x7f67c80a67f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009d90 tx=0x7f67c001c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23258217da6a48a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 1 -- 172.21.15.89:0/433334782 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f67c00b2070 con 0x7f67c80a5270 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 10 monclient: dump: 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.624 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:04.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 5 monclient: authenticate success, global_id 5166 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 1 -- 172.21.15.89:0/433334782 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c00a5ae0 con 0x7f67c80a5270 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 4 set_mon_vals no callback set 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 1 -- 172.21.15.89:0/433334782 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f67c00a5cd0 con 0x7f67c80a5270 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67c67fc640 10 monclient: dump: 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.626 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.627 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 10 monclient: shutdown 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 1 -- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67c80a5270 msgr2=0x7f67c80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 1 --2- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67c80a5270 0x7f67c80a67f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009d90 tx=0x7f67c001c830 comp rx=0 tx=0).stop 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 1 -- 172.21.15.89:0/433334782 shutdown_connections 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 1 --2- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c81395b0 0x7f67c8139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 1 --2- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f67c80a6d30 0x7f67c80a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 1 --2- 172.21.15.89:0/433334782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67c80a5270 0x7f67c80a67f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 1 -- 172.21.15.89:0/433334782 >> 172.21.15.89:0/433334782 conn(0x7f67c8069a80 msgr2=0x7f67c80abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 1 -- 172.21.15.89:0/433334782 shutdown_connections 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.613+0000 7f67ce0f2640 1 -- 172.21.15.89:0/433334782 wait complete. 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient: build_initial_monmap 2024-08-19T10:41:04.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient: monmap: 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:04.629 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 1 Processor -- start 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 1 -- start start 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient: init 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient: _add_conns ranks=[5,4,7,2,8,1,0,3,6] 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c80a5270 0x7f67c80a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient(hunting): picked mon.noname-h con 0x7f67c80a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c80a6d30 0x7f67c80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient(hunting): picked mon.noname-c con 0x7f67c80a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f67c81395b0 0x7f67c809c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient(hunting): picked mon.noname-i con 0x7f67c81395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f67c80a5650 con 0x7f67c80a6d30 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f67c80a4a70 con 0x7f67c80a5270 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f67c80a57d0 con 0x7f67c81395b0 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67ce0f2640 10 monclient(hunting): authenticate will time out at 1114.437250s 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c80a6d30 0x7f67c80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c80a6d30 0x7f67c80a2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48428/0 (socket says 172.21.15.89:48428) 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 1 -- 172.21.15.89:0/2905700479 learned_addr learned my addr 172.21.15.89:0/2905700479 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 10 monclient(hunting): get_auth_request con 0x7f67c80a6d30 auth_method 0 2024-08-19T10:41:04.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 10 monclient(hunting): handle_auth_done global_id 5143 payload 824 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c77fe640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c80a5270 0x7f67c80a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 1 -- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f67c81395b0 msgr2=0x7f67c809c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f67c81395b0 0x7f67c809c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 1 -- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c80a5270 msgr2=0x7f67c80a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c80a5270 0x7f67c80a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:04.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 1 monclient: found mon.noname-c 2024-08-19T10:41:04.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:04.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:04.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 1 -- 172.21.15.89:0/2905700479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c813fb30 con 0x7f67c80a6d30 2024-08-19T10:41:04.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 1 -- 172.21.15.89:0/2905700479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c80a6d30 msgr2=0x7f67c80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.617+0000 7f67c6ffd640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c80a6d30 0x7f67c80a2410 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f67b000e9d0 tx=0x7f67b000ee90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f67b0017070 con 0x7f67c80a6d30 2024-08-19T10:41:04.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 10 monclient: dump: 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.636 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 10 monclient: _finish_auth 0 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b000cc10 con 0x7f67c80a6d30 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67ce0f2640 5 monclient: authenticate success, global_id 5143 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67ce0f2640 10 monclient: _renew_subs 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67ce0f2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:04.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67ce0f2640 1 -- 172.21.15.89:0/2905700479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c80a6980 con 0x7f67c80a6d30 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f67b000cdc0 con 0x7f67c80a6d30 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c57fa640 4 set_mon_vals no callback set 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 10 monclient: dump: 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.638 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67ce0f2640 10 monclient: _renew_subs 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67ce0f2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67ce0f2640 1 -- 172.21.15.89:0/2905700479 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c809f0f0 con 0x7f67c80a6d30 2024-08-19T10:41:04.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67ce0f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:04.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67ce0f2640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802d1d0 0x7f67c802d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67ce0f2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67ce0f2640 1 -- 172.21.15.89:0/2905700479 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f67c80a7110 con 0x7f67c802d1d0 2024-08-19T10:41:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c77fe640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802d1d0 0x7f67c802d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f67b001a070 con 0x7f67c80a6d30 2024-08-19T10:41:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c77fe640 10 monclient: get_auth_request con 0x7f67c802d1d0 auth_method 0 2024-08-19T10:41:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f67a803be00 0x7f67a803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f67b000f040 con 0x7f67c80a6d30 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c77fe640 10 monclient(hunting): handle_auth_done global_id 5072 payload 824 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c77fe640 1 -- 172.21.15.89:0/2905700479 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67c8064590 con 0x7f67c802d1d0 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c7fff640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f67a803be00 0x7f67a803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.621+0000 7f67c7fff640 10 monclient: get_auth_request con 0x7f67a803be00 auth_method 0 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67c77fe640 1 -- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802d1d0 msgr2=0x7f67c802d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67c77fe640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802d1d0 0x7f67c802d5b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67c00af000 tx=0x7f67c00195d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67c7fff640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f67a803be00 0x7f67a803e2c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009a60 tx=0x7f67b8009520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f67c00bc020 con 0x7f67c802d1d0 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67bffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:04.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67bffff640 10 monclient: dump: 2024-08-19T10:41:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.646 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f67c0004040 con 0x7f67c802d1d0 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67bffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67bffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802d1d0 msgr2=0x7f67c802d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67bffff640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802d1d0 0x7f67c802d5b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67c00af000 tx=0x7f67c00195d0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802d1d0 msgr2=0x7f67c802d5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:04.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.625+0000 7f67bffff640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802d1d0 0x7f67c802d5b0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67c00af000 tx=0x7f67c00195d0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.629+0000 7f67ce0f2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:41:04.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.629+0000 7f67ce0f2640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802daf0 0x7f67c802df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.629+0000 7f67ce0f2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.629+0000 7f67ce0f2640 1 -- 172.21.15.89:0/2905700479 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f67c8139220 con 0x7f67c802daf0 2024-08-19T10:41:04.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.629+0000 7f67c77fe640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802daf0 0x7f67c802df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.629+0000 7f67c77fe640 10 monclient: get_auth_request con 0x7f67c802daf0 auth_method 0 2024-08-19T10:41:04.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.629+0000 7f67c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.629+0000 7f67c77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.629+0000 7f67c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.633+0000 7f67c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.633+0000 7f67c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.633+0000 7f67c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.633+0000 7f67c77fe640 10 monclient(hunting): handle_auth_done global_id 5081 payload 824 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.633+0000 7f67c77fe640 1 -- 172.21.15.89:0/2905700479 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f67c809f7b0 con 0x7f67c802daf0 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.633+0000 7f67c77fe640 1 -- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802daf0 msgr2=0x7f67c802df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.633+0000 7f67c77fe640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802daf0 0x7f67c802df00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67c00b0040 tx=0x7f67c00a7000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.633+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f67c00bc050 con 0x7f67c802daf0 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.633+0000 7f67bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.633+0000 7f67bffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.633+0000 7f67bffff640 10 monclient: dump: 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.650 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9423 (secure 0 0 0) 0x7f67c00c3af0 con 0x7f67c802daf0 2024-08-19T10:41:04.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67bffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:41:04.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67bffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:04.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802daf0 msgr2=0x7f67c802df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67bffff640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802daf0 0x7f67c802df00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67c00b0040 tx=0x7f67c00a7000 comp rx=0 tx=0).stop 2024-08-19T10:41:04.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67bffff640 1 -- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802daf0 msgr2=0x7f67c802df00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:04.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67bffff640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f67c802daf0 0x7f67c802df00 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67c00b0040 tx=0x7f67c00a7000 comp rx=0 tx=0).stop 2024-08-19T10:41:04.652 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:41:04.652 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:41:04.652 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 40, 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 194408, 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:41:04.653 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.654 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.655 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:41:04.656 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:04.657 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:04.657 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:41:04.657 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:41:04.657 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:41:04.657 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:41:04.657 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.658 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.659 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:41:04.660 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:41:04.663 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:41:04.664 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.666 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.667 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.668 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.669 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.670 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:41:04.672 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.674 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.674 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:41:04.674 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.674 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.674 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.674 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:41:04.674 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:41:04.674 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.674 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.675 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.675 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.675 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.675 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:41:04.675 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:41:04.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.676 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:41:04.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.677 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.677 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.677 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 4 2024-08-19T10:41:04.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.677 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.677 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:41:04.677 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:41:04.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67ce0f2640 1 -- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f67a803be00 msgr2=0x7f67a803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67ce0f2640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f67a803be00 0x7f67a803e2c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009a60 tx=0x7f67b8009520 comp rx=0 tx=0).stop 2024-08-19T10:41:04.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67ce0f2640 10 monclient: shutdown 2024-08-19T10:41:04.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67ce0f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:04.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67ce0f2640 1 -- 172.21.15.89:0/2905700479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c80a6d30 msgr2=0x7f67c80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67ce0f2640 1 --2- 172.21.15.89:0/2905700479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f67c80a6d30 0x7f67c80a2410 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f67b000e9d0 tx=0x7f67b000ee90 comp rx=0 tx=0).stop 2024-08-19T10:41:04.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67c7fff640 1 -- 172.21.15.89:0/2905700479 reap_dead start 2024-08-19T10:41:04.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67c7fff640 1 -- 172.21.15.89:0/2905700479 reap_dead start 2024-08-19T10:41:04.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67ce0f2640 1 -- 172.21.15.89:0/2905700479 shutdown_connections 2024-08-19T10:41:04.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67ce0f2640 1 -- 172.21.15.89:0/2905700479 >> 172.21.15.89:0/2905700479 conn(0x7f67c8069a80 msgr2=0x7f67c80ab1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:04.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67ce0f2640 1 -- 172.21.15.89:0/2905700479 shutdown_connections 2024-08-19T10:41:04.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.637+0000 7f67ce0f2640 1 -- 172.21.15.89:0/2905700479 wait complete. 2024-08-19T10:41:04.680 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:41:04.680 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-08-19T10:41:04.680 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:41:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.793+0000 7f94a0449640 10 monclient: get_monmap_and_config 2024-08-19T10:41:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.793+0000 7f94a0449640 10 monclient: build_initial_monmap 2024-08-19T10:41:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.793+0000 7f94a0449640 10 monclient: monmap: 2024-08-19T10:41:04.800 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:41:04.800794+0000 2024-08-19T10:41:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:41:04.800794+0000 2024-08-19T10:41:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:04.801 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:04.802 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:04.802 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 1 Processor -- start 2024-08-19T10:41:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 1 -- start start 2024-08-19T10:41:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 10 monclient: init 2024-08-19T10:41:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:04.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 10 monclient: _add_conns ranks=[5,8,6,1,4,3,9,10,2,7,0] 2024-08-19T10:41:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94980a5270 0x7f94980a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 10 monclient(hunting): picked mon.noname-h con 0x7f94980a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a6d30 0x7f94980a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 10 monclient(hunting): picked mon.noname-e con 0x7f94980a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94981395b0 0x7f94981399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 10 monclient(hunting): picked mon.noname-d con 0x7f94981395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f94980a7650 con 0x7f94980a5270 2024-08-19T10:41:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f94980a8710 con 0x7f94981395b0 2024-08-19T10:41:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f94980a5a40 con 0x7f94980a6d30 2024-08-19T10:41:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f94a0449640 10 monclient(hunting): authenticate will time out at 1114.617457s 2024-08-19T10:41:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a6d30 0x7f94980a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e9e6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94981395b0 0x7f94981399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e9e6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94981395b0 0x7f94981399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:33034/0 (socket says 172.21.15.89:33034) 2024-08-19T10:41:04.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a6d30 0x7f94980a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47920/0 (socket says 172.21.15.89:47920) 2024-08-19T10:41:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e9e6640 1 -- 172.21.15.89:0/2699881922 learned_addr learned my addr 172.21.15.89:0/2699881922 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e9e6640 10 monclient(hunting): get_auth_request con 0x7f94981395b0 auth_method 0 2024-08-19T10:41:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e9e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e9e6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e9e6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 10 monclient(hunting): get_auth_request con 0x7f94980a6d30 auth_method 0 2024-08-19T10:41:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e9e6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e9e6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e9e6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e1e5640 1 --2- 172.21.15.89:0/2699881922 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94980a5270 0x7f94980a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e1e5640 10 monclient(hunting): get_auth_request con 0x7f94980a5270 auth_method 0 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e1e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e1e5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e1e5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 10 monclient(hunting): handle_auth_done global_id 5003 payload 274 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 1 -- 172.21.15.89:0/2699881922 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94981395b0 msgr2=0x7f94981399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:04.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 1 --2- 172.21.15.89:0/2699881922 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94981395b0 0x7f94981399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 1 -- 172.21.15.89:0/2699881922 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94980a5270 msgr2=0x7f94980a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 1 --2- 172.21.15.89:0/2699881922 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94980a5270 0x7f94980a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 10 monclient: _finish_hunting 0 2024-08-19T10:41:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 1 monclient: found mon.noname-e 2024-08-19T10:41:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949d9e4640 1 -- 172.21.15.89:0/2699881922 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94980a81d0 con 0x7f94980a6d30 2024-08-19T10:41:04.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.797+0000 7f949e9e6640 1 --2- 172.21.15.89:0/2699881922 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94981395b0 0x7f94981399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d9e4640 1 -- 172.21.15.89:0/2699881922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a6d30 msgr2=0x7f94980a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d9e4640 1 --2- 172.21.15.89:0/2699881922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a6d30 0x7f94980a7110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94880097f0 tx=0x7f948801c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c30c909c4fa4f9a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 1 -- 172.21.15.89:0/2699881922 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94880b0070 con 0x7f94980a6d30 2024-08-19T10:41:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 10 monclient: dump: 2024-08-19T10:41:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.819 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.820 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 10 monclient: _finish_auth 0 2024-08-19T10:41:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 10 monclient: _check_auth_tickets 2024-08-19T10:41:04.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 5 monclient: authenticate success, global_id 5003 2024-08-19T10:41:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 1 -- 172.21.15.89:0/2699881922 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94880aa810 con 0x7f94980a6d30 2024-08-19T10:41:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 4 set_mon_vals no callback set 2024-08-19T10:41:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 1 -- 172.21.15.89:0/2699881922 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94880aae30 con 0x7f94980a6d30 2024-08-19T10:41:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f949d1e3640 10 monclient: dump: 2024-08-19T10:41:04.821 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.822 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.823 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 10 monclient: shutdown 2024-08-19T10:41:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 1 -- 172.21.15.89:0/2699881922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a6d30 msgr2=0x7f94980a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 1 --2- 172.21.15.89:0/2699881922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a6d30 0x7f94980a7110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94880097f0 tx=0x7f948801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 1 -- 172.21.15.89:0/2699881922 shutdown_connections 2024-08-19T10:41:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 1 --2- 172.21.15.89:0/2699881922 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94981395b0 0x7f94981399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 1 --2- 172.21.15.89:0/2699881922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a6d30 0x7f94980a7110 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 1 --2- 172.21.15.89:0/2699881922 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94980a5270 0x7f94980a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 1 -- 172.21.15.89:0/2699881922 >> 172.21.15.89:0/2699881922 conn(0x7f9498069a80 msgr2=0x7f94980abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 1 -- 172.21.15.89:0/2699881922 shutdown_connections 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 1 -- 172.21.15.89:0/2699881922 wait complete. 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 10 monclient: build_initial_monmap 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.801+0000 7f94a0449640 10 monclient: monmap: 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:04.824 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:04.825 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 1 Processor -- start 2024-08-19T10:41:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 1 -- start start 2024-08-19T10:41:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient: init 2024-08-19T10:41:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient: _add_conns ranks=[8,5,1,2,7,4,0,6,3] 2024-08-19T10:41:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a5270 0x7f9498131ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient(hunting): picked mon.noname-e con 0x7f94980a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94980a6d30 0x7f949813ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient(hunting): picked mon.noname-h con 0x7f94980a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:04.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94981395b0 0x7f949813d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient(hunting): picked mon.noname-f con 0x7f94981395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f94980a57d0 con 0x7f94981395b0 2024-08-19T10:41:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f94980a5650 con 0x7f94980a6d30 2024-08-19T10:41:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9498143230 con 0x7f94980a5270 2024-08-19T10:41:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient(hunting): authenticate will time out at 1114.624631s 2024-08-19T10:41:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a5270 0x7f9498131ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a5270 0x7f9498131ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47932/0 (socket says 172.21.15.89:47932) 2024-08-19T10:41:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 1 -- 172.21.15.89:0/107685414 learned_addr learned my addr 172.21.15.89:0/107685414 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:41:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 10 monclient(hunting): get_auth_request con 0x7f94980a5270 auth_method 0 2024-08-19T10:41:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 10 monclient(hunting): handle_auth_done global_id 5012 payload 824 2024-08-19T10:41:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 1 -- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94980a6d30 msgr2=0x7f949813ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94980a6d30 0x7f949813ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 1 -- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94981395b0 msgr2=0x7f949813d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e9e6640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94981395b0 0x7f949813d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94981395b0 0x7f949813d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 10 monclient: _finish_hunting 0 2024-08-19T10:41:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 1 monclient: found mon.noname-e 2024-08-19T10:41:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 1 -- 172.21.15.89:0/107685414 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949813e6b0 con 0x7f94980a5270 2024-08-19T10:41:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 1 -- 172.21.15.89:0/107685414 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a5270 msgr2=0x7f9498131ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949e1e5640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a5270 0x7f9498131ce0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f949400d570 tx=0x7f949400da30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9494012c70 con 0x7f94980a5270 2024-08-19T10:41:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 10 monclient: dump: 2024-08-19T10:41:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.830 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.831 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 10 monclient: _finish_auth 0 2024-08-19T10:41:04.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9494019070 con 0x7f94980a5270 2024-08-19T10:41:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 5 monclient: authenticate success, global_id 5012 2024-08-19T10:41:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient: _renew_subs 2024-08-19T10:41:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 1 -- 172.21.15.89:0/107685414 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94980a73c0 con 0x7f94980a5270 2024-08-19T10:41:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9494012e30 con 0x7f94980a5270 2024-08-19T10:41:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:04.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f949c9e2640 4 set_mon_vals no callback set 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f948effd640 10 monclient: dump: 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.833 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.834 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient: _renew_subs 2024-08-19T10:41:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.805+0000 7f94a0449640 1 -- 172.21.15.89:0/107685414 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f949809d440 con 0x7f94980a5270 2024-08-19T10:41:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f94a0449640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f94a0449640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9498132220 0x7f949813d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f94a0449640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f94a0449640 1 -- 172.21.15.89:0/107685414 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f949809d010 con 0x7f9498132220 2024-08-19T10:41:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9494012800 con 0x7f94980a5270 2024-08-19T10:41:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9498132220 0x7f949813d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 10 monclient: get_auth_request con 0x7f9498132220 auth_method 0 2024-08-19T10:41:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f948effd640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f947c03be00 0x7f947c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(73..73 src has 1..73) ==== 5364+0+0 (secure 0 0 0) 0x7f949400f040 con 0x7f94980a5270 2024-08-19T10:41:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949e9e6640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f947c03be00 0x7f947c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949e9e6640 10 monclient: get_auth_request con 0x7f947c03be00 auth_method 0 2024-08-19T10:41:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 10 monclient(hunting): handle_auth_done global_id 14299 payload 824 2024-08-19T10:41:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 1 -- 172.21.15.89:0/107685414 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94980a67f0 con 0x7f9498132220 2024-08-19T10:41:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 1 -- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9498132220 msgr2=0x7f949813d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.809+0000 7f949d9e4640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9498132220 0x7f949813d730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f94880097c0 tx=0x7f94880a4d60 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f949e9e6640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f947c03be00 0x7f947c03e2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9490009c90 tx=0x7f949000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94880b8020 con 0x7f9498132220 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f948effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f948effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f948effd640 10 monclient: dump: 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.837 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.838 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9488003830 con 0x7f9498132220 2024-08-19T10:41:04.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f948effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:04.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f948effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:04.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9498132220 msgr2=0x7f949813d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f948effd640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9498132220 0x7f949813d730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f94880097c0 tx=0x7f94880a4d60 comp rx=0 tx=0).stop 2024-08-19T10:41:04.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9498132220 msgr2=0x7f949813d730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:04.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.813+0000 7f948effd640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9498132220 0x7f949813d730 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f94880097c0 tx=0x7f94880a4d60 comp rx=0 tx=0).stop 2024-08-19T10:41:04.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f94a0449640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:41:04.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f94a0449640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f949813dc70 0x7f9498137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:04.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f94a0449640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:04.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f94a0449640 1 -- 172.21.15.89:0/107685414 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9498064590 con 0x7f949813dc70 2024-08-19T10:41:04.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f949d9e4640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f949813dc70 0x7f9498137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:04.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f949d9e4640 10 monclient: get_auth_request con 0x7f949813dc70 auth_method 0 2024-08-19T10:41:04.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f949d9e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:04.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f949d9e4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:04.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f949d9e4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:04.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f949d9e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:04.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f949d9e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:04.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f949d9e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:04.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f949d9e4640 10 monclient(hunting): handle_auth_done global_id 14308 payload 824 2024-08-19T10:41:04.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.817+0000 7f949d9e4640 1 -- 172.21.15.89:0/107685414 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f949813e050 con 0x7f949813dc70 2024-08-19T10:41:04.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f949d9e4640 1 -- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f949813dc70 msgr2=0x7f9498137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:04.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f949d9e4640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f949813dc70 0x7f9498137ab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f948801c3e0 tx=0x7f94880a8000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:04.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94880b8040 con 0x7f949813dc70 2024-08-19T10:41:04.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f948effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:04.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f948effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:04.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f948effd640 10 monclient: dump: 2024-08-19T10:41:04.843 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:41:04.843 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:04.843 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.843 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:04.843 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:04.843 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:41:04.843 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:04.843 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:04.844 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:04.844 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:04.844 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:04.844 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:04.844 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:04.844 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:04.844 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:04.844 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:41:04.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9255 (secure 0 0 0) 0x7f94880bd410 con 0x7f949813dc70 2024-08-19T10:41:04.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f948effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:41:04.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f948effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:04.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f949813dc70 msgr2=0x7f9498137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f948effd640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f949813dc70 0x7f9498137ab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f948801c3e0 tx=0x7f94880a8000 comp rx=0 tx=0).stop 2024-08-19T10:41:04.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f948effd640 1 -- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f949813dc70 msgr2=0x7f9498137ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:41:04.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.821+0000 7f948effd640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f949813dc70 0x7f9498137ab0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f948801c3e0 tx=0x7f94880a8000 comp rx=0 tx=0).stop 2024-08-19T10:41:04.845 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:41:04.845 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:41:04.845 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:41:04.845 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:41:04.846 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 40, 2024-08-19T10:41:04.846 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 42414, 2024-08-19T10:41:04.846 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:41:04.846 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:41:04.846 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:41:04.846 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:41:04.846 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:41:04.846 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:41:04.846 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:41:04.846 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:41:04.847 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:41:04.847 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.847 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:41:04.847 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:04.847 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:41:04.847 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:41:04.847 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.847 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.847 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.847 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.847 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.848 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.848 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.848 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.848 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.848 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.848 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.848 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.848 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:41:04.848 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:41:04.848 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.848 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.849 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.849 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.849 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.849 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.849 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.849 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.849 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.849 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.849 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.849 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.850 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:41:04.850 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:41:04.850 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:41:04.850 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:41:04.850 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:41:04.850 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:41:04.850 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:04.850 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:41:04.850 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:41:04.851 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:41:04.851 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:41:04.851 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:41:04.851 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:41:04.853 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:41:04.853 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:41:04.854 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:41:04.854 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:41:04.854 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:41:04.854 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:41:04.854 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:41:04.854 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:41:04.854 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:41:04.854 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:41:04.854 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:41:04.854 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:41:04.855 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:41:04.855 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:41:04.855 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:41:04.855 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.855 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:41:04.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.855 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:41:04.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.855 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:41:04.855 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:41:04.855 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.856 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.856 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.856 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:41:04.856 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.856 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.856 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:41:04.856 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.856 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:41:04.857 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.858 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.858 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.858 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:41:04.858 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.858 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.858 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:41:04.858 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.859 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.859 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.859 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:41:04.859 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:41:04.859 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.859 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.859 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.859 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:41:04.860 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.861 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.861 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.861 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:41:04.861 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:41:04.861 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.861 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.861 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.861 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:41:04.862 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:41:04.862 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.862 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:41:04.862 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.862 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:41:04.863 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.863 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.863 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:41:04.863 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:41:04.863 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.863 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.863 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.865 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:41:04.866 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:41:04.866 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:41:04.866 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.867 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:41:04.868 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:41:04.868 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:41:04.868 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:41:04.869 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.869 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.869 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:41:04.869 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:41:04.869 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.869 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.869 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.870 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:41:04.870 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:41:04.870 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:41:04.870 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.872 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:41:04.872 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:41:04.872 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:41:04.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:41:04.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.872 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:41:04.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:41:04.872 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.872 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.873 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.874 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:41:04.874 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:41:04.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.874 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.874 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.874 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:04.874 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.874 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:41:04.874 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:41:04.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:41:04.875 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:41:04.875 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:41:04.875 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:41:04.875 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:41:04.875 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:41:04.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:41:04.875 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:41:04.875 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:41:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f94a0449640 1 -- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f947c03be00 msgr2=0x7f947c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f94a0449640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f947c03be00 0x7f947c03e2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9490009c90 tx=0x7f949000b040 comp rx=0 tx=0).stop 2024-08-19T10:41:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f94a0449640 10 monclient: shutdown 2024-08-19T10:41:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f94a0449640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f94a0449640 1 -- 172.21.15.89:0/107685414 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a5270 msgr2=0x7f9498131ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f94a0449640 1 --2- 172.21.15.89:0/107685414 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f94980a5270 0x7f9498131ce0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f949400d570 tx=0x7f949400da30 comp rx=0 tx=0).stop 2024-08-19T10:41:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f949e9e6640 1 -- 172.21.15.89:0/107685414 reap_dead start 2024-08-19T10:41:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f949e9e6640 1 -- 172.21.15.89:0/107685414 reap_dead start 2024-08-19T10:41:04.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f94a0449640 1 -- 172.21.15.89:0/107685414 shutdown_connections 2024-08-19T10:41:04.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f94a0449640 1 -- 172.21.15.89:0/107685414 >> 172.21.15.89:0/107685414 conn(0x7f9498069a80 msgr2=0x7f9498134760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:04.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f94a0449640 1 -- 172.21.15.89:0/107685414 shutdown_connections 2024-08-19T10:41:04.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:41:04.825+0000 7f94a0449640 1 -- 172.21.15.89:0/107685414 wait complete. 2024-08-19T10:41:04.878 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:41:04.878 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-08-19T10:41:04.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:04.962+0000 7f3fb17fa640 1 -- 172.21.15.190:0/4179186166 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v74) ==== 58+0+2 (secure 0 0 0) 0x7f3fa80b3da0 con 0x7f3fb40a6d30 2024-08-19T10:41:04.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:04.962+0000 7f3fb17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-08-19T10:41:04.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:04.962+0000 7f3fb17fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:05.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.050+0000 7f3f9ffff640 10 monclient: tick 2024-08-19T10:41:05.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.050+0000 7f3f9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:05.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.050+0000 7f3f9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:05.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.082+0000 7f3fbad27640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-08-19T10:41:05.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.082+0000 7f3fbad27640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-08-19T10:41:05.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.082+0000 7f3fbad27640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:05.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.082+0000 7f3fbad27640 1 -- 172.21.15.190:0/4179186166 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f3fb41d1e20 con 0x7f3fb40a6d30 2024-08-19T10:41:05.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.378+0000 7f3fb17fa640 1 -- 172.21.15.190:0/4179186166 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v75) ==== 58+0+2 (secure 0 0 0) 0x7f3fa80bdcc0 con 0x7f3fb40a6d30 2024-08-19T10:41:05.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.378+0000 7f3fb17fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-08-19T10:41:05.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.378+0000 7f3fb17fa640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:05.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 -- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f9803be00 msgr2=0x7f3f9803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:05.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f9803be00 0x7f3f9803e2c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0007ca0 tx=0x7f3fa0007bf0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 10 monclient: shutdown 2024-08-19T10:41:05.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:05.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 -- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3fb40a6d30 msgr2=0x7f3fb41399b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:05.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3fb40a6d30 0x7f3fb41399b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80ae040 tx=0x7f3fa8019100 comp rx=0 tx=0).stop 2024-08-19T10:41:05.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 -- 172.21.15.190:0/4179186166 shutdown_connections 2024-08-19T10:41:05.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb413eb60 0x7f3fb40a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3fb40a6d30 0x7f3fb41399b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f9803be00 0x7f3f9803e2c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 --2- 172.21.15.190:0/4179186166 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3fb40a6950 0x7f3fb41d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 -- 172.21.15.190:0/4179186166 >> 172.21.15.190:0/4179186166 conn(0x7f3fb40abf50 msgr2=0x7f3fb413ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:05.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 -- 172.21.15.190:0/4179186166 shutdown_connections 2024-08-19T10:41:05.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.382+0000 7f3fbad27640 1 -- 172.21.15.190:0/4179186166 wait complete. 2024-08-19T10:41:05.411 INFO:tasks.workunit.client.0.smithi190.stderr:+ o2=9 2024-08-19T10:41:05.411 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add 7 1 host=host1 root=foo 2024-08-19T10:41:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient: get_monmap_and_config 2024-08-19T10:41:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient: build_initial_monmap 2024-08-19T10:41:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient: monmap: 2024-08-19T10:41:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:05.533437+0000 2024-08-19T10:41:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:05.533437+0000 2024-08-19T10:41:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:05.536 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 1 Processor -- start 2024-08-19T10:41:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 1 -- start start 2024-08-19T10:41:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient: init 2024-08-19T10:41:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient: _add_conns ranks=[8,7,10,0,2,6,1,5,9,3,4] 2024-08-19T10:41:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f32780a69c0 0x7f32780a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient(hunting): picked mon.noname-e con 0x7f32780a69c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32780a6da0 0x7f32780a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient(hunting): picked mon.noname-i con 0x7f32780a6da0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3278139780 0x7f3278139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient(hunting): picked mon.noname-f con 0x7f3278139780 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f32780a5c30 con 0x7f3278139780 2024-08-19T10:41:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f32780a3570 con 0x7f32780a6da0 2024-08-19T10:41:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f32780a4eb0 con 0x7f32780a69c0 2024-08-19T10:41:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.530+0000 7f327ebcd640 10 monclient(hunting): authenticate will time out at 1111.103971s 2024-08-19T10:41:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32780a6da0 0x7f32780a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327d16a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3278139780 0x7f3278139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32780a6da0 0x7f32780a7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44208/0 (socket says 172.21.15.190:44208) 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327d16a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3278139780 0x7f3278139bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39122/0 (socket says 172.21.15.190:39122) 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 1 -- 172.21.15.190:0/1836985664 learned_addr learned my addr 172.21.15.190:0/1836985664 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 10 monclient(hunting): get_auth_request con 0x7f32780a6da0 auth_method 0 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327d16a640 10 monclient(hunting): get_auth_request con 0x7f3278139780 auth_method 0 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327d16a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327d16a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327d16a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327d16a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327d16a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327d16a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327c969640 1 --2- 172.21.15.190:0/1836985664 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f32780a69c0 0x7f32780a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327c969640 10 monclient(hunting): get_auth_request con 0x7f32780a69c0 auth_method 0 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327c969640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327c969640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327c969640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 10 monclient(hunting): handle_auth_done global_id 14317 payload 274 2024-08-19T10:41:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 1 -- 172.21.15.190:0/1836985664 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f32780a69c0 msgr2=0x7f32780a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 1 --2- 172.21.15.190:0/1836985664 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f32780a69c0 0x7f32780a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 1 -- 172.21.15.190:0/1836985664 >> v2:172.21.15.190:3300/0 conn(0x7f3278139780 msgr2=0x7f3278139bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 1 --2- 172.21.15.190:0/1836985664 >> v2:172.21.15.190:3300/0 conn(0x7f3278139780 0x7f3278139bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 1 monclient: found mon.noname-i 2024-08-19T10:41:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f3277fff640 1 -- 172.21.15.190:0/1836985664 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32780a6330 con 0x7f32780a6da0 2024-08-19T10:41:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.534+0000 7f327d16a640 1 --2- 172.21.15.190:0/1836985664 >> v2:172.21.15.190:3300/0 conn(0x7f3278139780 0x7f3278139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f3277fff640 1 -- 172.21.15.190:0/1836985664 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32780a6da0 msgr2=0x7f32780a7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f3277fff640 1 --2- 172.21.15.190:0/1836985664 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32780a6da0 0x7f32780a7180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f326c0098a0 tx=0x7f326c01c400 comp rx=0 tx=0).ready entity=mon.7 client_cookie=68cafebe2b0a4f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 1 -- 172.21.15.190:0/1836985664 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f326c0b0070 con 0x7f32780a6da0 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 10 monclient: dump: 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 1 -- 172.21.15.190:0/1836985664 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326c0aa810 con 0x7f32780a6da0 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 1 -- 172.21.15.190:0/1836985664 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f326c0aae30 con 0x7f32780a6da0 2024-08-19T10:41:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f32777fe640 10 monclient: dump: 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:05.548 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 5 monclient: authenticate success, global_id 14317 2024-08-19T10:41:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 4 set_mon_vals no callback set 2024-08-19T10:41:05.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient: shutdown 2024-08-19T10:41:05.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:05.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 -- 172.21.15.190:0/1836985664 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32780a6da0 msgr2=0x7f32780a7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:05.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 --2- 172.21.15.190:0/1836985664 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32780a6da0 0x7f32780a7180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f326c0098a0 tx=0x7f326c01c400 comp rx=0 tx=0).stop 2024-08-19T10:41:05.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 -- 172.21.15.190:0/1836985664 shutdown_connections 2024-08-19T10:41:05.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 --2- 172.21.15.190:0/1836985664 >> v2:172.21.15.190:3300/0 conn(0x7f3278139780 0x7f3278139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 --2- 172.21.15.190:0/1836985664 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32780a6da0 0x7f32780a7180 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 --2- 172.21.15.190:0/1836985664 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f32780a69c0 0x7f32780a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 -- 172.21.15.190:0/1836985664 >> 172.21.15.190:0/1836985664 conn(0x7f32780abfc0 msgr2=0x7f32780ac3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:05.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 -- 172.21.15.190:0/1836985664 shutdown_connections 2024-08-19T10:41:05.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 -- 172.21.15.190:0/1836985664 wait complete. 2024-08-19T10:41:05.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient: build_initial_monmap 2024-08-19T10:41:05.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient: monmap: 2024-08-19T10:41:05.557 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:05.557 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:05.557 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:05.557 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:05.557 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:05.557 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:05.558 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:05.558 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:05.558 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:05.558 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:05.558 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:05.558 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:05.559 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:05.559 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:05.559 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:05.559 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:05.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 Processor -- start 2024-08-19T10:41:05.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 -- start start 2024-08-19T10:41:05.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient: init 2024-08-19T10:41:05.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:05.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient: _add_conns ranks=[3,8,5,6,2,1,0,4,7] 2024-08-19T10:41:05.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32780a69c0 0x7f327813cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:05.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient(hunting): picked mon.noname-g con 0x7f32780a69c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:05.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f32780a6da0 0x7f327813d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:05.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient(hunting): picked mon.noname-e con 0x7f32780a6da0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:05.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3278139780 0x7f327813da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:05.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient(hunting): picked mon.noname-h con 0x7f3278139780 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:05.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:05.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f32780a76c0 con 0x7f32780a69c0 2024-08-19T10:41:05.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f327813a510 con 0x7f3278139780 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f327813fcd0 con 0x7f32780a6da0 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327ebcd640 10 monclient(hunting): authenticate will time out at 1111.111330s 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327c969640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32780a69c0 msgr2=0x7f327813cfa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327c969640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32780a69c0 0x7f327813cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3278139780 0x7f327813da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3278139780 0x7f327813da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:46948/0 (socket says 172.21.15.190:46948) 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 1 -- 172.21.15.190:0/4205468409 learned_addr learned my addr 172.21.15.190:0/4205468409 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 10 monclient(hunting): get_auth_request con 0x7f3278139780 auth_method 0 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:05.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:05.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:05.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:05.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:05.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:05.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f3277fff640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f32780a6da0 0x7f327813d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:05.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f3277fff640 10 monclient(hunting): get_auth_request con 0x7f32780a6da0 auth_method 0 2024-08-19T10:41:05.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f3277fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:05.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f3277fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:05.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f3277fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:05.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 10 monclient(hunting): handle_auth_done global_id 5099 payload 824 2024-08-19T10:41:05.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 1 -- 172.21.15.190:0/4205468409 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f32780a6da0 msgr2=0x7f327813d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f32780a6da0 0x7f327813d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 1 -- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32780a69c0 msgr2=0x7f327813cfa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32780a69c0 0x7f327813cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 10 monclient: _finish_hunting 0 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 1 monclient: found mon.noname-h 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.538+0000 7f327d16a640 1 -- 172.21.15.190:0/4205468409 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327809d8b0 con 0x7f3278139780 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327d16a640 1 -- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3278139780 msgr2=0x7f327813da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327d16a640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3278139780 0x7f327813da20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3270009e60 tx=0x7f327000c490 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 1 -- 172.21.15.190:0/4205468409 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3270018030 con 0x7f3278139780 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:05.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 10 monclient: dump: 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:05.564 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 10 monclient: _finish_auth 0 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 10 monclient: _check_auth_tickets 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 1 -- 172.21.15.190:0/4205468409 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32700148f0 con 0x7f3278139780 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327ebcd640 5 monclient: authenticate success, global_id 5099 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327ebcd640 10 monclient: _renew_subs 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327ebcd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327ebcd640 1 -- 172.21.15.190:0/4205468409 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327813e900 con 0x7f3278139780 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:05.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 1 -- 172.21.15.190:0/4205468409 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32700133e0 con 0x7f3278139780 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 10 monclient: dump: 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:05.566 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:05.567 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:05.567 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:05.567 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:05.567 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:05.567 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:05.567 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327ebcd640 10 monclient: _renew_subs 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327ebcd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327ebcd640 1 -- 172.21.15.190:0/4205468409 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f327809d2f0 con 0x7f3278139780 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f3276ffd640 4 set_mon_vals no callback set 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 1 -- 172.21.15.190:0/4205468409 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f327001b070 con 0x7f3278139780 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327ebcd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327ebcd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327ebcd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327ebcd640 1 -- 172.21.15.190:0/4205468409 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32780a7180 con 0x7f3278139780 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326403be00 0x7f326403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f32757fa640 1 -- 172.21.15.190:0/4205468409 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(75..75 src has 1..75) ==== 5594+0+0 (secure 0 0 0) 0x7f3270019070 con 0x7f3278139780 2024-08-19T10:41:05.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327c969640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326403be00 0x7f326403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:05.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.542+0000 7f327c969640 10 monclient: get_auth_request con 0x7f326403be00 auth_method 0 2024-08-19T10:41:05.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.546+0000 7f327c969640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326403be00 0x7f326403e2c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3268000f00 tx=0x7f3268006e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:05.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.550+0000 7f32757fa640 1 -- 172.21.15.190:0/4205468409 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3270010040 con 0x7f3278139780 2024-08-19T10:41:05.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.550+0000 7f32757fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:05.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.550+0000 7f32757fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:05.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.718+0000 7f327ebcd640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:05.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.718+0000 7f327ebcd640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:05.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.718+0000 7f327ebcd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:05.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:05.718+0000 7f327ebcd640 1 -- 172.21.15.190:0/4205468409 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) -- 0x7f327809ca60 con 0x7f3278139780 2024-08-19T10:41:06.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.790+0000 7f32757fa640 1 -- 172.21.15.190:0/4205468409 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 add item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo} to crush map v76) ==== 203+0+0 (secure 0 0 0) 0x7f3270014710 con 0x7f3278139780 2024-08-19T10:41:06.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.790+0000 7f32757fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:06.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.790+0000 7f32757fa640 10 monclient: _finish_command 2 = system:0 add item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo} to crush map 2024-08-19T10:41:06.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.894+0000 7f327ebcd640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:06.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.894+0000 7f327ebcd640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:06.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.894+0000 7f327ebcd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:06.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.894+0000 7f327ebcd640 1 -- 172.21.15.190:0/4205468409 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) -- 0x7f32780008e0 con 0x7f3278139780 2024-08-19T10:41:06.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f32757fa640 1 -- 172.21.15.190:0/4205468409 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v76) ==== 201+0+0 (secure 0 0 0) 0x7f327003f020 con 0x7f3278139780 2024-08-19T10:41:06.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f32757fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:06.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f32757fa640 10 monclient: _finish_command 3 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-08-19T10:41:06.901 INFO:tasks.workunit.client.0.smithi190.stderr:set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-08-19T10:41:06.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 1 -- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326403be00 msgr2=0x7f326403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:06.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326403be00 0x7f326403e2c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3268000f00 tx=0x7f3268006e20 comp rx=0 tx=0).stop 2024-08-19T10:41:06.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 10 monclient: shutdown 2024-08-19T10:41:06.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:06.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 1 -- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3278139780 msgr2=0x7f327813da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:06.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3278139780 0x7f327813da20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3270009e60 tx=0x7f327000c490 comp rx=0 tx=0).stop 2024-08-19T10:41:06.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 1 -- 172.21.15.190:0/4205468409 shutdown_connections 2024-08-19T10:41:06.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3278139780 0x7f327813da20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:06.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326403be00 0x7f326403e2c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:06.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f32780a6da0 0x7f327813d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:06.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 1 --2- 172.21.15.190:0/4205468409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32780a69c0 0x7f327813cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:06.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.898+0000 7f327ebcd640 1 -- 172.21.15.190:0/4205468409 >> 172.21.15.190:0/4205468409 conn(0x7f32780abfc0 msgr2=0x7f32780ab570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:06.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.902+0000 7f327ebcd640 1 -- 172.21.15.190:0/4205468409 shutdown_connections 2024-08-19T10:41:06.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:06.902+0000 7f327ebcd640 1 -- 172.21.15.190:0/4205468409 wait complete. 2024-08-19T10:41:06.918 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add 7 1 host=host1 root=foo 2024-08-19T10:41:07.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.038+0000 7f2fd848c640 10 monclient: get_monmap_and_config 2024-08-19T10:41:07.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.038+0000 7f2fd848c640 10 monclient: build_initial_monmap 2024-08-19T10:41:07.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.038+0000 7f2fd848c640 10 monclient: monmap: 2024-08-19T10:41:07.041 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:07.041 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:07.042 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:07.041641+0000 2024-08-19T10:41:07.042 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:07.041641+0000 2024-08-19T10:41:07.042 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:07.042 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.042 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:07.042 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:07.042 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:07.042 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:07.042 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:07.042 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:07.042 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:07.043 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:07.043 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:07.043 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:07.043 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:07.043 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.038+0000 7f2fd848c640 1 Processor -- start 2024-08-19T10:41:07.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.038+0000 7f2fd848c640 1 -- start start 2024-08-19T10:41:07.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.038+0000 7f2fd848c640 10 monclient: init 2024-08-19T10:41:07.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:07.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 10 monclient: _add_conns ranks=[2,6,8,4,7,0,10,3,1,5,9] 2024-08-19T10:41:07.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2fd00a69c0 0x7f2fd0139780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 10 monclient(hunting): picked mon.noname-b con 0x7f2fd00a69c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:07.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 0x7f2fd013a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 10 monclient(hunting): picked mon.noname-d con 0x7f2fd0139cc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:07.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 0x7f2fd00a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 10 monclient(hunting): picked mon.noname-e con 0x7f2fd00a6da0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd013a5e0 con 0x7f2fd00a69c0 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f2fd00a46b0 con 0x7f2fd0139cc0 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2fd00a4990 con 0x7f2fd00a6da0 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd848c640 10 monclient(hunting): authenticate will time out at 1112.612439s 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd5a27640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 0x7f2fd013a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 0x7f2fd00a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd5a27640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 0x7f2fd013a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:34958/0 (socket says 172.21.15.190:34958) 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd5a27640 1 -- 172.21.15.190:0/2236416024 learned_addr learned my addr 172.21.15.190:0/2236416024 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd5a27640 10 monclient(hunting): get_auth_request con 0x7f2fd0139cc0 auth_method 0 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd5a27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd5a27640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd5a27640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 10 monclient(hunting): get_auth_request con 0x7f2fd00a6da0 auth_method 0 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6228640 1 --2- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2fd00a69c0 0x7f2fd0139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6228640 10 monclient(hunting): get_auth_request con 0x7f2fd00a69c0 auth_method 0 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6228640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:07.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6228640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6228640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd5a27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd5a27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd5a27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6228640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6228640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6228640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 10 monclient(hunting): handle_auth_done global_id 5039 payload 274 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 1 -- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 msgr2=0x7f2fd013a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:07.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 1 --2- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 0x7f2fd013a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 1 -- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2fd00a69c0 msgr2=0x7f2fd0139780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:07.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 1 --2- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2fd00a69c0 0x7f2fd0139780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 10 monclient: _finish_hunting 0 2024-08-19T10:41:07.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 1 monclient: found mon.noname-e 2024-08-19T10:41:07.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:07.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:07.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd6a29640 1 -- 172.21.15.190:0/2236416024 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd00a4c70 con 0x7f2fd00a6da0 2024-08-19T10:41:07.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.042+0000 7f2fd5a27640 1 --2- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 0x7f2fd013a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:07.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.046+0000 7f2fd6a29640 1 -- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 msgr2=0x7f2fd00a71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:07.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.046+0000 7f2fd6a29640 1 --2- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 0x7f2fd00a71a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009dc0 tx=0x7f2fc801c850 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2835d9942de84d53 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:07.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 1 -- 172.21.15.190:0/2236416024 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2fc80b2070 con 0x7f2fd00a6da0 2024-08-19T10:41:07.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:07.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:07.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 10 monclient: dump: 2024-08-19T10:41:07.051 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:07.052 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 10 monclient: _finish_auth 0 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 10 monclient: _check_auth_tickets 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 1 -- 172.21.15.190:0/2236416024 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc80a5ac0 con 0x7f2fd00a6da0 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 1 -- 172.21.15.190:0/2236416024 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2fc80a5cb0 con 0x7f2fd00a6da0 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 5 monclient: authenticate success, global_id 5039 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 4 set_mon_vals no callback set 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:07.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5226640 10 monclient: dump: 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:07.054 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:07.055 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:07.055 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:07.055 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:07.055 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:07.055 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient: shutdown 2024-08-19T10:41:07.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:07.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 -- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 msgr2=0x7f2fd00a71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:07.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 --2- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 0x7f2fd00a71a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009dc0 tx=0x7f2fc801c850 comp rx=0 tx=0).stop 2024-08-19T10:41:07.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 -- 172.21.15.190:0/2236416024 shutdown_connections 2024-08-19T10:41:07.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 --2- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 0x7f2fd00a71a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 --2- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 0x7f2fd013a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 --2- 172.21.15.190:0/2236416024 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2fd00a69c0 0x7f2fd0139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 -- 172.21.15.190:0/2236416024 >> 172.21.15.190:0/2236416024 conn(0x7f2fd00abfc0 msgr2=0x7f2fd00ac3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 -- 172.21.15.190:0/2236416024 shutdown_connections 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 -- 172.21.15.190:0/2236416024 wait complete. 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient: build_initial_monmap 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient: monmap: 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:07.061 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 Processor -- start 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 -- start start 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient: init 2024-08-19T10:41:07.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient: _add_conns ranks=[1,8,6,3,4,2,5,7,0] 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fd00a69c0 0x7f2fd00a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient(hunting): picked mon.noname-f con 0x7f2fd00a69c0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 0x7f2fd00a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient(hunting): picked mon.noname-e con 0x7f2fd00a6da0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 0x7f2fd009cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient(hunting): picked mon.noname-d con 0x7f2fd0139cc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd00a7c70 con 0x7f2fd00a69c0 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f2fd00a7af0 con 0x7f2fd0139cc0 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2fd01431f0 con 0x7f2fd00a6da0 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd848c640 10 monclient(hunting): authenticate will time out at 1112.622603s 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fd00a69c0 0x7f2fd00a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fd00a69c0 0x7f2fd00a1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39128/0 (socket says 172.21.15.190:39128) 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 1 -- 172.21.15.190:0/1318950345 learned_addr learned my addr 172.21.15.190:0/1318950345 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 10 monclient(hunting): get_auth_request con 0x7f2fd00a69c0 auth_method 0 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:07.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6a29640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 0x7f2fd009cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5a27640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 0x7f2fd00a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6a29640 10 monclient(hunting): get_auth_request con 0x7f2fd0139cc0 auth_method 0 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6a29640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6a29640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6a29640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5a27640 10 monclient(hunting): get_auth_request con 0x7f2fd00a6da0 auth_method 0 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5a27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5a27640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd5a27640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 10 monclient(hunting): handle_auth_done global_id 14311 payload 824 2024-08-19T10:41:07.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 1 -- 172.21.15.190:0/1318950345 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 msgr2=0x7f2fd00a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:07.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 0x7f2fd00a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 1 -- 172.21.15.190:0/1318950345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 msgr2=0x7f2fd009cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:07.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 0x7f2fd009cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 10 monclient: _finish_hunting 0 2024-08-19T10:41:07.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 1 monclient: found mon.noname-f 2024-08-19T10:41:07.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:07.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:07.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6228640 1 -- 172.21.15.190:0/1318950345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd013fc50 con 0x7f2fd00a69c0 2024-08-19T10:41:07.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.050+0000 7f2fd6a29640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 0x7f2fd009cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:41:07.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd6228640 1 -- 172.21.15.190:0/1318950345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fd00a69c0 msgr2=0x7f2fd00a1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:07.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd6228640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fd00a69c0 0x7f2fd00a1dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00e960 tx=0x7f2fcc00ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:07.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 1 -- 172.21.15.190:0/1318950345 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2fcc017070 con 0x7f2fd00a69c0 2024-08-19T10:41:07.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:07.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:07.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 10 monclient: dump: 2024-08-19T10:41:07.067 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:07.067 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:07.067 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.067 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.067 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:41:07.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 1 -- 172.21.15.190:0/1318950345 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fcc00cc10 con 0x7f2fd00a69c0 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd848c640 5 monclient: authenticate success, global_id 14311 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd848c640 10 monclient: _renew_subs 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd848c640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd848c640 1 -- 172.21.15.190:0/1318950345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd00ac670 con 0x7f2fd00a69c0 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 1 -- 172.21.15.190:0/1318950345 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2fcc00cdc0 con 0x7f2fd00a69c0 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 10 monclient: dump: 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:07.069 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:07.070 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc7fff640 4 set_mon_vals no callback set 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd848c640 10 monclient: _renew_subs 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd848c640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd848c640 1 -- 172.21.15.190:0/1318950345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd013ee80 con 0x7f2fd00a69c0 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd848c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd848c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd848c640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd848c640 1 -- 172.21.15.190:0/1318950345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd009e090 con 0x7f2fd00a69c0 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 1 -- 172.21.15.190:0/1318950345 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2fcc01a070 con 0x7f2fd00a69c0 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2fb403be00 0x7f2fb403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fc6ffd640 1 -- 172.21.15.190:0/1318950345 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5876+0+0 (secure 0 0 0) 0x7f2fcc00f040 con 0x7f2fd00a69c0 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd5a27640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2fb403be00 0x7f2fb403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.054+0000 7f2fd5a27640 10 monclient: get_auth_request con 0x7f2fb403be00 auth_method 0 2024-08-19T10:41:07.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.058+0000 7f2fd5a27640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2fb403be00 0x7f2fb403e2c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2fd013f4f0 tx=0x7f2fc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:07.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.058+0000 7f2fc6ffd640 1 -- 172.21.15.190:0/1318950345 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f2fcc021070 con 0x7f2fd00a69c0 2024-08-19T10:41:07.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.058+0000 7f2fc6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:07.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.058+0000 7f2fc6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:07.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.234+0000 7f2fd848c640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:07.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.234+0000 7f2fd848c640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:07.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.234+0000 7f2fd848c640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:07.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.234+0000 7f2fd848c640 1 -- 172.21.15.190:0/1318950345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) -- 0x7f2fd009f460 con 0x7f2fd00a69c0 2024-08-19T10:41:07.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.238+0000 7f2fc6ffd640 1 -- 172.21.15.190:0/1318950345 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v76) ==== 201+0+0 (secure 0 0 0) 0x7f2fcc04d740 con 0x7f2fd00a69c0 2024-08-19T10:41:07.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.238+0000 7f2fc6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:07.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.238+0000 7f2fc6ffd640 10 monclient: _finish_command 2 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-08-19T10:41:07.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.334+0000 7f2fd848c640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:07.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.334+0000 7f2fd848c640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:07.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.334+0000 7f2fd848c640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:07.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.334+0000 7f2fd848c640 1 -- 172.21.15.190:0/1318950345 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]} v 0) -- 0x7f2fd00a7180 con 0x7f2fd00a69c0 2024-08-19T10:41:07.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fc6ffd640 1 -- 172.21.15.190:0/1318950345 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}]=0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change v76) ==== 201+0+0 (secure 0 0 0) 0x7f2fcc01ac80 con 0x7f2fd00a69c0 2024-08-19T10:41:07.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fc6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 7, "weight": 1.0, "args": ["host=host1", "root=foo"]}] 2024-08-19T10:41:07.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fc6ffd640 10 monclient: _finish_command 3 = system:0 set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-08-19T10:41:07.341 INFO:tasks.workunit.client.0.smithi190.stderr:set item id 7 name 'osd.7' weight 1 at location {host=host1,root=foo}: no change 2024-08-19T10:41:07.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 -- 172.21.15.190:0/1318950345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2fb403be00 msgr2=0x7f2fb403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:07.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2fb403be00 0x7f2fb403e2c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2fd013f4f0 tx=0x7f2fc0008040 comp rx=0 tx=0).stop 2024-08-19T10:41:07.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 10 monclient: shutdown 2024-08-19T10:41:07.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:07.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 -- 172.21.15.190:0/1318950345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fd00a69c0 msgr2=0x7f2fd00a1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:07.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fd00a69c0 0x7f2fd00a1dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00e960 tx=0x7f2fcc00ee20 comp rx=0 tx=0).stop 2024-08-19T10:41:07.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 -- 172.21.15.190:0/1318950345 shutdown_connections 2024-08-19T10:41:07.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2fd0139cc0 0x7f2fd009cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2fb403be00 0x7f2fb403e2c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2fd00a6da0 0x7f2fd00a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 --2- 172.21.15.190:0/1318950345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fd00a69c0 0x7f2fd00a1dd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 -- 172.21.15.190:0/1318950345 >> 172.21.15.190:0/1318950345 conn(0x7f2fd00abfc0 msgr2=0x7f2fd00a3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:07.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 -- 172.21.15.190:0/1318950345 shutdown_connections 2024-08-19T10:41:07.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.338+0000 7f2fd848c640 1 -- 172.21.15.190:0/1318950345 wait complete. 2024-08-19T10:41:07.356 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add 9 1 host=host2 root=foo 2024-08-19T10:41:07.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient: get_monmap_and_config 2024-08-19T10:41:07.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient: build_initial_monmap 2024-08-19T10:41:07.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient: monmap: 2024-08-19T10:41:07.481 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:07.481 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:07.481 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:07.481412+0000 2024-08-19T10:41:07.481 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:07.481412+0000 2024-08-19T10:41:07.482 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:07.482 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.482 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:07.482 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:07.482 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:07.482 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:07.482 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:07.482 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:07.482 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:07.482 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:07.483 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:07.483 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:07.483 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:07.483 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 1 Processor -- start 2024-08-19T10:41:07.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 1 -- start start 2024-08-19T10:41:07.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient: init 2024-08-19T10:41:07.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:07.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient: _add_conns ranks=[8,9,6,3,4,0,2,1,7,5,10] 2024-08-19T10:41:07.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e501396a0 0x7f1e50139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient(hunting): picked mon.noname-e con 0x7f1e501396a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:07.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e50139fc0 0x7f1e500a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient(hunting): picked mon.noname-a con 0x7f1e50139fc0 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e500a6e20 0x7f1e500a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient(hunting): picked mon.noname-d con 0x7f1e500a6e20 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f1e500a4f20 con 0x7f1e50139fc0 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1e500a83e0 con 0x7f1e500a6e20 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f1e500a2f30 con 0x7f1e501396a0 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.478+0000 7f1e56c5c640 10 monclient(hunting): authenticate will time out at 1113.051928s 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e551f9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e500a6e20 0x7f1e500a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e501396a0 0x7f1e50139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e4ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1e50139fc0 0x7f1e500a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e551f9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e500a6e20 0x7f1e500a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:34976/0 (socket says 172.21.15.190:34976) 2024-08-19T10:41:07.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e551f9640 1 -- 172.21.15.190:0/3997109380 learned_addr learned my addr 172.21.15.190:0/3997109380 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:07.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e551f9640 10 monclient(hunting): get_auth_request con 0x7f1e500a6e20 auth_method 0 2024-08-19T10:41:07.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e551f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:07.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e551f9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:07.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e551f9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:07.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 10 monclient(hunting): get_auth_request con 0x7f1e501396a0 auth_method 0 2024-08-19T10:41:07.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:07.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:07.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:07.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e4ffff640 10 monclient(hunting): get_auth_request con 0x7f1e50139fc0 auth_method 0 2024-08-19T10:41:07.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:07.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e4ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:07.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:07.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e551f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:07.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e551f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:07.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e551f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:07.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:07.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:07.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:07.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:07.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:07.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:07.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 10 monclient(hunting): handle_auth_done global_id 5057 payload 274 2024-08-19T10:41:07.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 1 -- 172.21.15.190:0/3997109380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e500a6e20 msgr2=0x7f1e500a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:07.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 1 --2- 172.21.15.190:0/3997109380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e500a6e20 0x7f1e500a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 1 -- 172.21.15.190:0/3997109380 >> v2:172.21.15.89:3300/0 conn(0x7f1e50139fc0 msgr2=0x7f1e500a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:07.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 1 --2- 172.21.15.190:0/3997109380 >> v2:172.21.15.89:3300/0 conn(0x7f1e50139fc0 0x7f1e500a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 10 monclient: _finish_hunting 0 2024-08-19T10:41:07.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 1 monclient: found mon.noname-e 2024-08-19T10:41:07.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:07.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:07.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e549f8640 1 -- 172.21.15.190:0/3997109380 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e500a35e0 con 0x7f1e501396a0 2024-08-19T10:41:07.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e4ffff640 1 --2- 172.21.15.190:0/3997109380 >> v2:172.21.15.89:3300/0 conn(0x7f1e50139fc0 0x7f1e500a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:07.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.482+0000 7f1e551f9640 1 --2- 172.21.15.190:0/3997109380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e500a6e20 0x7f1e500a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:07.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e549f8640 1 -- 172.21.15.190:0/3997109380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e501396a0 msgr2=0x7f1e50139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:07.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e549f8640 1 --2- 172.21.15.190:0/3997109380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e501396a0 0x7f1e50139a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009dc0 tx=0x7f1e4001c810 comp rx=0 tx=0).ready entity=mon.8 client_cookie=17074c93e44c380e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:07.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 1 -- 172.21.15.190:0/3997109380 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e400ac070 con 0x7f1e501396a0 2024-08-19T10:41:07.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:07.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:07.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 10 monclient: dump: 2024-08-19T10:41:07.497 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:07.497 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:07.497 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.497 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.497 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:07.497 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.497 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:07.497 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:07.497 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:07.497 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:07.497 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:07.498 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:07.498 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:07.498 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:07.498 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:07.498 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:07.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:07.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:07.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 5 monclient: authenticate success, global_id 5057 2024-08-19T10:41:07.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:07.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 1 -- 172.21.15.190:0/3997109380 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e400a6460 con 0x7f1e501396a0 2024-08-19T10:41:07.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:07.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:07.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 4 set_mon_vals no callback set 2024-08-19T10:41:07.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 1 -- 172.21.15.190:0/3997109380 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e400a6b00 con 0x7f1e501396a0 2024-08-19T10:41:07.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:07.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:07.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e4f7fe640 10 monclient: dump: 2024-08-19T10:41:07.499 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:07.499 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:07.500 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.500 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.500 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:07.500 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.500 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:07.500 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:07.500 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:07.500 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:07.500 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:07.500 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:07.500 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:07.501 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:07.501 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:07.501 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 10 monclient: shutdown 2024-08-19T10:41:07.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:07.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 1 -- 172.21.15.190:0/3997109380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e501396a0 msgr2=0x7f1e50139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:07.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 1 --2- 172.21.15.190:0/3997109380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e501396a0 0x7f1e50139a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009dc0 tx=0x7f1e4001c810 comp rx=0 tx=0).stop 2024-08-19T10:41:07.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 1 -- 172.21.15.190:0/3997109380 shutdown_connections 2024-08-19T10:41:07.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 1 --2- 172.21.15.190:0/3997109380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e500a6e20 0x7f1e500a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 1 --2- 172.21.15.190:0/3997109380 >> v2:172.21.15.89:3300/0 conn(0x7f1e50139fc0 0x7f1e500a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 1 --2- 172.21.15.190:0/3997109380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e501396a0 0x7f1e50139a80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 1 -- 172.21.15.190:0/3997109380 >> 172.21.15.190:0/3997109380 conn(0x7f1e500abee0 msgr2=0x7f1e500ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:07.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 1 -- 172.21.15.190:0/3997109380 shutdown_connections 2024-08-19T10:41:07.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 1 -- 172.21.15.190:0/3997109380 wait complete. 2024-08-19T10:41:07.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 10 monclient: build_initial_monmap 2024-08-19T10:41:07.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 10 monclient: monmap: 2024-08-19T10:41:07.502 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:07.502 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:07.502 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:07.502 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:07.502 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:07.503 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.503 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:07.503 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:07.503 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:07.503 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:07.503 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:07.503 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:07.503 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:07.503 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:07.503 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:07.503 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 1 Processor -- start 2024-08-19T10:41:07.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 1 -- start start 2024-08-19T10:41:07.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.486+0000 7f1e56c5c640 10 monclient: init 2024-08-19T10:41:07.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:07.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient: _add_conns ranks=[3,5,4,2,0,8,6,7,1] 2024-08-19T10:41:07.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e500a6e20 0x7f1e500a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient(hunting): picked mon.noname-g con 0x7f1e500a6e20 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:07.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e501396a0 0x7f1e500a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient(hunting): picked mon.noname-h con 0x7f1e501396a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:07.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e50139fc0 0x7f1e5009ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient(hunting): picked mon.noname-c con 0x7f1e50139fc0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:07.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5013a3a0 con 0x7f1e500a6e20 2024-08-19T10:41:07.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f1e50143390 con 0x7f1e50139fc0 2024-08-19T10:41:07.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e549f8640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e500a6e20 msgr2=0x7f1e500a1d30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:07.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:07.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1e50140800 con 0x7f1e501396a0 2024-08-19T10:41:07.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:07.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient(hunting): authenticate will time out at 1113.060447s 2024-08-19T10:41:07.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e549f8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e500a6e20 0x7f1e500a1d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:07.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e501396a0 0x7f1e500a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e501396a0 0x7f1e500a2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:46952/0 (socket says 172.21.15.190:46952) 2024-08-19T10:41:07.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 1 -- 172.21.15.190:0/1839160057 learned_addr learned my addr 172.21.15.190:0/1839160057 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:07.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 10 monclient(hunting): get_auth_request con 0x7f1e501396a0 auth_method 0 2024-08-19T10:41:07.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:07.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:07.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:07.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:07.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:07.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:07.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 10 monclient(hunting): handle_auth_done global_id 5108 payload 824 2024-08-19T10:41:07.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e551f9640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e50139fc0 0x7f1e5009ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 1 -- 172.21.15.190:0/1839160057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e50139fc0 msgr2=0x7f1e5009ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:07.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e50139fc0 0x7f1e5009ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 1 -- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e500a6e20 msgr2=0x7f1e500a1d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:07.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e500a6e20 0x7f1e500a1d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:07.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 1 monclient: found mon.noname-h 2024-08-19T10:41:07.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:07.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:07.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 1 -- 172.21.15.190:0/1839160057 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5009e4a0 con 0x7f1e501396a0 2024-08-19T10:41:07.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 1 -- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e501396a0 msgr2=0x7f1e500a2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:07.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4ffff640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e501396a0 0x7f1e500a2270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400e980 tx=0x7f1e4400ee40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 1 -- 172.21.15.190:0/1839160057 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e44017070 con 0x7f1e501396a0 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 10 monclient: dump: 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:07.511 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 1 -- 172.21.15.190:0/1839160057 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e4400ccf0 con 0x7f1e501396a0 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:07.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 1 -- 172.21.15.190:0/1839160057 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e4400cea0 con 0x7f1e501396a0 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4d7fa640 10 monclient: dump: 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:07.513 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:07.514 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:07.514 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:07.514 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:07.514 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:07.514 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:07.514 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:07.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 5 monclient: authenticate success, global_id 5108 2024-08-19T10:41:07.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient: _renew_subs 2024-08-19T10:41:07.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e4effd640 4 set_mon_vals no callback set 2024-08-19T10:41:07.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:07.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 1 -- 172.21.15.190:0/1839160057 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e500a0800 con 0x7f1e501396a0 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient: _renew_subs 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.490+0000 7f1e56c5c640 1 -- 172.21.15.190:0/1839160057 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e501d1580 con 0x7f1e501396a0 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.494+0000 7f1e56c5c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.494+0000 7f1e56c5c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.494+0000 7f1e56c5c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.494+0000 7f1e56c5c640 1 -- 172.21.15.190:0/1839160057 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5013a520 con 0x7f1e501396a0 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.494+0000 7f1e4d7fa640 1 -- 172.21.15.190:0/1839160057 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1e4401a070 con 0x7f1e501396a0 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.494+0000 7f1e4d7fa640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e3c03be00 0x7f1e3c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.494+0000 7f1e4d7fa640 1 -- 172.21.15.190:0/1839160057 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(76..76 src has 1..76) ==== 5876+0+0 (secure 0 0 0) 0x7f1e4400f040 con 0x7f1e501396a0 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.494+0000 7f1e549f8640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e3c03be00 0x7f1e3c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:07.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.494+0000 7f1e549f8640 10 monclient: get_auth_request con 0x7f1e3c03be00 auth_method 0 2024-08-19T10:41:07.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.494+0000 7f1e549f8640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e3c03be00 0x7f1e3c03e2c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1e400ae000 tx=0x7f1e400a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:07.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.498+0000 7f1e4d7fa640 1 -- 172.21.15.190:0/1839160057 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1e5013a520 con 0x7f1e501396a0 2024-08-19T10:41:07.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.498+0000 7f1e4d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:07.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.498+0000 7f1e4d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:07.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.670+0000 7f1e56c5c640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:07.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.670+0000 7f1e56c5c640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:07.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.670+0000 7f1e56c5c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:07.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.670+0000 7f1e56c5c640 1 -- 172.21.15.190:0/1839160057 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) -- 0x7f1e50139a80 con 0x7f1e501396a0 2024-08-19T10:41:07.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.802+0000 7f1e4d7fa640 1 -- 172.21.15.190:0/1839160057 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 add item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo} to crush map v77) ==== 203+0+0 (secure 0 0 0) 0x7f1e44021070 con 0x7f1e501396a0 2024-08-19T10:41:07.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.802+0000 7f1e4d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:07.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.802+0000 7f1e4d7fa640 10 monclient: _finish_command 2 = system:0 add item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo} to crush map 2024-08-19T10:41:07.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.906+0000 7f1e56c5c640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:07.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.906+0000 7f1e56c5c640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:07.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.906+0000 7f1e56c5c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:07.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.906+0000 7f1e56c5c640 1 -- 172.21.15.190:0/1839160057 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) -- 0x7f1e50139480 con 0x7f1e501396a0 2024-08-19T10:41:07.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.910+0000 7f1e4d7fa640 1 -- 172.21.15.190:0/1839160057 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v77) ==== 201+0+0 (secure 0 0 0) 0x7f1e4401ac60 con 0x7f1e501396a0 2024-08-19T10:41:07.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.910+0000 7f1e4d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:07.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.910+0000 7f1e4d7fa640 10 monclient: _finish_command 3 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-08-19T10:41:07.913 INFO:tasks.workunit.client.0.smithi190.stderr:set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-08-19T10:41:07.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 -- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e3c03be00 msgr2=0x7f1e3c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:07.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e3c03be00 0x7f1e3c03e2c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1e400ae000 tx=0x7f1e400a7000 comp rx=0 tx=0).stop 2024-08-19T10:41:07.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 10 monclient: shutdown 2024-08-19T10:41:07.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:07.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 -- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e501396a0 msgr2=0x7f1e500a2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:07.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e501396a0 0x7f1e500a2270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400e980 tx=0x7f1e4400ee40 comp rx=0 tx=0).stop 2024-08-19T10:41:07.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 -- 172.21.15.190:0/1839160057 shutdown_connections 2024-08-19T10:41:07.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e3c03be00 0x7f1e3c03e2c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e50139fc0 0x7f1e5009ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e501396a0 0x7f1e500a2270 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 --2- 172.21.15.190:0/1839160057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e500a6e20 0x7f1e500a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:07.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 -- 172.21.15.190:0/1839160057 >> 172.21.15.190:0/1839160057 conn(0x7f1e500abee0 msgr2=0x7f1e500ab490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:07.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 -- 172.21.15.190:0/1839160057 shutdown_connections 2024-08-19T10:41:07.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:07.914+0000 7f1e56c5c640 1 -- 172.21.15.190:0/1839160057 wait complete. 2024-08-19T10:41:07.930 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add 9 1 host=host2 root=foo 2024-08-19T10:41:08.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient: get_monmap_and_config 2024-08-19T10:41:08.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient: build_initial_monmap 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient: monmap: 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:08.048971+0000 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:08.048971+0000 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:08.050 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:08.050 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:08.050 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:08.050 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:08.050 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:08.050 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:08.050 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 1 Processor -- start 2024-08-19T10:41:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 1 -- start start 2024-08-19T10:41:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient: init 2024-08-19T10:41:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient: _add_conns ranks=[1,9,7,6,10,5,0,3,4,2,8] 2024-08-19T10:41:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fea081396a0 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fea0813a190 0x7fea0813eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient(hunting): picked mon.noname-a con 0x7fea0813a190 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea0813f0a0 0x7fea08141490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient(hunting): picked mon.noname-i con 0x7fea0813f0a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea080a83e0 con 0x7fea081396a0 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fea080a4f20 con 0x7fea0813a190 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fea080a2f30 con 0x7fea0813f0a0 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea104c2640 10 monclient(hunting): authenticate will time out at 1113.618894s 2024-08-19T10:41:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea0813f0a0 0x7fea08141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0e25e640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fea081396a0 0x7fea08139a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:59836/0 (socket says 172.21.15.190:59836) 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0e25e640 1 -- 172.21.15.190:0/2658023272 learned_addr learned my addr 172.21.15.190:0/2658023272 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 10 monclient(hunting): get_auth_request con 0x7fea0813f0a0 auth_method 0 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0da5d640 1 --2- 172.21.15.190:0/2658023272 >> v2:172.21.15.89:3300/0 conn(0x7fea0813a190 0x7fea0813eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0da5d640 10 monclient(hunting): get_auth_request con 0x7fea0813a190 auth_method 0 2024-08-19T10:41:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0da5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0da5d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0da5d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 10 monclient(hunting): handle_auth_done global_id 14326 payload 274 2024-08-19T10:41:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 1 -- 172.21.15.190:0/2658023272 >> v2:172.21.15.89:3300/0 conn(0x7fea0813a190 msgr2=0x7fea0813eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 1 --2- 172.21.15.190:0/2658023272 >> v2:172.21.15.89:3300/0 conn(0x7fea0813a190 0x7fea0813eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 1 -- 172.21.15.190:0/2658023272 >> v1:172.21.15.190:6789/0 conn(0x7fea081396a0 legacy=0x7fea08139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 10 monclient: _finish_hunting 0 2024-08-19T10:41:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 1 monclient: found mon.noname-i 2024-08-19T10:41:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:08.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.046+0000 7fea0ea5f640 1 -- 172.21.15.190:0/2658023272 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea080a35e0 con 0x7fea0813f0a0 2024-08-19T10:41:08.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0ea5f640 1 -- 172.21.15.190:0/2658023272 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea0813f0a0 msgr2=0x7fea08141490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:08.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0ea5f640 1 --2- 172.21.15.190:0/2658023272 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea0813f0a0 0x7fea08141490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea00009d90 tx=0x7fea0001c830 comp rx=0 tx=0).ready entity=mon.7 client_cookie=fd9864b7c99b1b73 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:08.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 1 -- 172.21.15.190:0/2658023272 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fea000ac070 con 0x7fea0813f0a0 2024-08-19T10:41:08.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:08.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:08.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 10 monclient: dump: 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:08.062 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 10 monclient: _finish_auth 0 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 10 monclient: _check_auth_tickets 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 1 -- 172.21.15.190:0/2658023272 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea000a68f0 con 0x7fea0813f0a0 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea104c2640 5 monclient: authenticate success, global_id 14326 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea104c2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea104c2640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:08.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 1 -- 172.21.15.190:0/2658023272 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fea000a7460 con 0x7fea0813f0a0 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea104c2640 4 set_mon_vals no callback set 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.050+0000 7fea0d25c640 10 monclient: dump: 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient: shutdown 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 -- 172.21.15.190:0/2658023272 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea0813f0a0 msgr2=0x7fea08141490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 --2- 172.21.15.190:0/2658023272 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea0813f0a0 0x7fea08141490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea00009d90 tx=0x7fea0001c830 comp rx=0 tx=0).stop 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 -- 172.21.15.190:0/2658023272 shutdown_connections 2024-08-19T10:41:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 --2- 172.21.15.190:0/2658023272 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea0813f0a0 0x7fea08141490 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 --2- 172.21.15.190:0/2658023272 >> v2:172.21.15.89:3300/0 conn(0x7fea0813a190 0x7fea0813eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 -- 172.21.15.190:0/2658023272 >> 172.21.15.190:0/2658023272 conn(0x7fea080abee0 msgr2=0x7fea080ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 -- 172.21.15.190:0/2658023272 shutdown_connections 2024-08-19T10:41:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 -- 172.21.15.190:0/2658023272 wait complete. 2024-08-19T10:41:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient: build_initial_monmap 2024-08-19T10:41:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient: monmap: 2024-08-19T10:41:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:08.068 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 Processor -- start 2024-08-19T10:41:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 -- start start 2024-08-19T10:41:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient: init 2024-08-19T10:41:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient: _add_conns ranks=[8,1,4,7,0,6,5,2,3] 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea081396a0 0x7fea081d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient(hunting): picked mon.noname-e con 0x7fea081396a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea0813a190 0x7fea081d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient(hunting): picked mon.noname-f con 0x7fea0813a190 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea0813f0a0 0x7fea0813c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient(hunting): picked mon.noname-c con 0x7fea0813f0a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea081441f0 con 0x7fea0813a190 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fea080a78f0 con 0x7fea0813f0a0 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fea08144370 con 0x7fea081396a0 2024-08-19T10:41:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea104c2640 10 monclient(hunting): authenticate will time out at 1113.625865s 2024-08-19T10:41:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea0813a190 0x7fea081d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea0813a190 0x7fea081d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39130/0 (socket says 172.21.15.190:39130) 2024-08-19T10:41:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 1 -- 172.21.15.190:0/661048097 learned_addr learned my addr 172.21.15.190:0/661048097 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 10 monclient(hunting): get_auth_request con 0x7fea0813a190 auth_method 0 2024-08-19T10:41:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0e25e640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea081396a0 0x7fea081d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 10 monclient(hunting): handle_auth_done global_id 14320 payload 824 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0ea5f640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea0813f0a0 0x7fea0813c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 1 -- 172.21.15.190:0/661048097 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea081396a0 msgr2=0x7fea081d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea081396a0 0x7fea081d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 1 -- 172.21.15.190:0/661048097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea0813f0a0 msgr2=0x7fea0813c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea0813f0a0 0x7fea0813c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 10 monclient: _finish_hunting 0 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 1 monclient: found mon.noname-f 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.054+0000 7fea0da5d640 1 -- 172.21.15.190:0/661048097 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0809d4d0 con 0x7fea0813a190 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea0da5d640 1 -- 172.21.15.190:0/661048097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea0813a190 msgr2=0x7fea081d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea0da5d640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea0813a190 0x7fea081d1a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800d800 tx=0x7fe9f800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 1 -- 172.21.15.190:0/661048097 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe9f800bd40 con 0x7fea0813a190 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 10 monclient: dump: 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 10 monclient: _finish_auth 0 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 1 -- 172.21.15.190:0/661048097 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f8019070 con 0x7fea0813a190 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 1 -- 172.21.15.190:0/661048097 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe9f80148a0 con 0x7fea0813a190 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 10 monclient: dump: 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea104c2640 5 monclient: authenticate success, global_id 14320 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9fffff640 4 set_mon_vals no callback set 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea104c2640 10 monclient: _renew_subs 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea104c2640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea104c2640 1 -- 172.21.15.190:0/661048097 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea080ac4c0 con 0x7fea0813a190 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea104c2640 10 monclient: _renew_subs 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea104c2640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea104c2640 1 -- 172.21.15.190:0/661048097 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0809daa0 con 0x7fea0813a190 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 1 -- 172.21.15.190:0/661048097 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe9f8014430 con 0x7fea0813a190 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea104c2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea104c2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea104c2640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea104c2640 1 -- 172.21.15.190:0/661048097 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea08139a80 con 0x7fea0813a190 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe9ec03be00 0x7fe9ec03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fe9feffd640 1 -- 172.21.15.190:0/661048097 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6120+0+0 (secure 0 0 0) 0x7fe9f800f040 con 0x7fea0813a190 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea0e25e640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe9ec03be00 0x7fe9ec03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.058+0000 7fea0e25e640 10 monclient: get_auth_request con 0x7fe9ec03be00 auth_method 0 2024-08-19T10:41:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.062+0000 7fea0e25e640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe9ec03be00 0x7fe9ec03e2c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fea04005060 tx=0x7fea04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.062+0000 7fe9feffd640 1 -- 172.21.15.190:0/661048097 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe9f8022020 con 0x7fea0813a190 2024-08-19T10:41:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.062+0000 7fe9feffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.062+0000 7fe9feffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:08.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.222+0000 7fea104c2640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:08.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.222+0000 7fea104c2640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:08.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.222+0000 7fea104c2640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:08.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.222+0000 7fea104c2640 1 -- 172.21.15.190:0/661048097 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) -- 0x7fea0809e3a0 con 0x7fea0813a190 2024-08-19T10:41:08.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.226+0000 7fe9feffd640 1 -- 172.21.15.190:0/661048097 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v77) ==== 201+0+0 (secure 0 0 0) 0x7fea080ac4c0 con 0x7fea0813a190 2024-08-19T10:41:08.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.226+0000 7fe9feffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:08.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.226+0000 7fe9feffd640 10 monclient: _finish_command 2 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-08-19T10:41:08.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.322+0000 7fea104c2640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:08.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.322+0000 7fea104c2640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:08.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.322+0000 7fea104c2640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:08.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.322+0000 7fea104c2640 1 -- 172.21.15.190:0/661048097 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]} v 0) -- 0x7fea080008e0 con 0x7fea0813a190 2024-08-19T10:41:08.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.322+0000 7fe9feffd640 1 -- 172.21.15.190:0/661048097 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}]=0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change v77) ==== 201+0+0 (secure 0 0 0) 0x7fe9f804d580 con 0x7fea0813a190 2024-08-19T10:41:08.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.322+0000 7fe9feffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 9, "weight": 1.0, "args": ["host=host2", "root=foo"]}] 2024-08-19T10:41:08.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.322+0000 7fe9feffd640 10 monclient: _finish_command 3 = system:0 set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-08-19T10:41:08.326 INFO:tasks.workunit.client.0.smithi190.stderr:set item id 9 name 'osd.9' weight 1 at location {host=host2,root=foo}: no change 2024-08-19T10:41:08.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 -- 172.21.15.190:0/661048097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe9ec03be00 msgr2=0x7fe9ec03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:08.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe9ec03be00 0x7fe9ec03e2c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fea04005060 tx=0x7fea04009040 comp rx=0 tx=0).stop 2024-08-19T10:41:08.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 10 monclient: shutdown 2024-08-19T10:41:08.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:08.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 -- 172.21.15.190:0/661048097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea0813a190 msgr2=0x7fea081d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:08.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea0813a190 0x7fea081d1a80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800d800 tx=0x7fe9f800dcc0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 -- 172.21.15.190:0/661048097 shutdown_connections 2024-08-19T10:41:08.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea0813f0a0 0x7fea0813c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea0813a190 0x7fea081d1a80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe9ec03be00 0x7fe9ec03e2c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 --2- 172.21.15.190:0/661048097 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea081396a0 0x7fea081d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 -- 172.21.15.190:0/661048097 >> 172.21.15.190:0/661048097 conn(0x7fea080abee0 msgr2=0x7fea08134860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:08.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 -- 172.21.15.190:0/661048097 shutdown_connections 2024-08-19T10:41:08.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.326+0000 7fea104c2640 1 -- 172.21.15.190:0/661048097 wait complete. 2024-08-19T10:41:08.342 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket bar root 2024-08-19T10:41:08.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.454+0000 7f6debd92640 10 monclient: get_monmap_and_config 2024-08-19T10:41:08.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.454+0000 7f6debd92640 10 monclient: build_initial_monmap 2024-08-19T10:41:08.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.454+0000 7f6debd92640 10 monclient: monmap: 2024-08-19T10:41:08.458 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:08.458 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:08.458 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:08.458484+0000 2024-08-19T10:41:08.458 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:08.458484+0000 2024-08-19T10:41:08.458 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:08.458 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:08.459 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 1 Processor -- start 2024-08-19T10:41:08.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 1 -- start start 2024-08-19T10:41:08.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 10 monclient: init 2024-08-19T10:41:08.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:08.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 10 monclient: _add_conns ranks=[4,2,9,7,8,0,3,10,6,5,1] 2024-08-19T10:41:08.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6de4139980 0x7f6de4139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 10 monclient(hunting): picked mon.noname-c con 0x7f6de4139980 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:08.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6de413a2a0 0x7f6de40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 10 monclient(hunting): picked mon.noname-b con 0x7f6de413a2a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:08.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6de40a5c70 0x7f6de40a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 10 monclient(hunting): picked mon.noname-a con 0x7f6de40a5c70 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f6de409ca60 con 0x7f6de40a5c70 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6de40a3640 con 0x7f6de413a2a0 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6de40a8470 con 0x7f6de4139980 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6debd92640 10 monclient(hunting): authenticate will time out at 1114.028858s 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6de413a2a0 0x7f6de40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6dea32f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6de40a5c70 0x7f6de40a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de9b2e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6de4139980 0x7f6de4139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6de413a2a0 0x7f6de40a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:37312/0 (socket says 172.21.15.190:37312) 2024-08-19T10:41:08.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 1 -- 172.21.15.190:0/3890175063 learned_addr learned my addr 172.21.15.190:0/3890175063 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 10 monclient(hunting): get_auth_request con 0x7f6de413a2a0 auth_method 0 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de9b2e640 10 monclient(hunting): get_auth_request con 0x7f6de4139980 auth_method 0 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de9b2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de9b2e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de9b2e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6dea32f640 10 monclient(hunting): get_auth_request con 0x7f6de40a5c70 auth_method 0 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6dea32f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6dea32f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6dea32f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de9b2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:08.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de9b2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de9b2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6dea32f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6dea32f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6dea32f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 10 monclient(hunting): handle_auth_done global_id 5096 payload 274 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 1 -- 172.21.15.190:0/3890175063 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6de4139980 msgr2=0x7f6de4139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 1 --2- 172.21.15.190:0/3890175063 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6de4139980 0x7f6de4139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 1 -- 172.21.15.190:0/3890175063 >> v2:172.21.15.89:3300/0 conn(0x7f6de40a5c70 msgr2=0x7f6de40a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:08.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 1 --2- 172.21.15.190:0/3890175063 >> v2:172.21.15.89:3300/0 conn(0x7f6de40a5c70 0x7f6de40a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 10 monclient: _finish_hunting 0 2024-08-19T10:41:08.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 1 monclient: found mon.noname-b 2024-08-19T10:41:08.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:08.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:08.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.458+0000 7f6de932d640 1 -- 172.21.15.190:0/3890175063 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de40a3850 con 0x7f6de413a2a0 2024-08-19T10:41:08.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de932d640 1 -- 172.21.15.190:0/3890175063 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6de413a2a0 msgr2=0x7f6de40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:08.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de932d640 1 --2- 172.21.15.190:0/3890175063 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6de413a2a0 0x7f6de40a5730 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009870 tx=0x7f6dd401c160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a54ad96414e3b9a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:08.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 1 -- 172.21.15.190:0/3890175063 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dd40b0070 con 0x7f6de413a2a0 2024-08-19T10:41:08.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:08.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:08.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 10 monclient: dump: 2024-08-19T10:41:08.470 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:08.470 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:08.470 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.470 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 10 monclient: _finish_auth 0 2024-08-19T10:41:08.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 10 monclient: _check_auth_tickets 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 1 -- 172.21.15.190:0/3890175063 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd40aa810 con 0x7f6de413a2a0 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 1 -- 172.21.15.190:0/3890175063 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dd40a9300 con 0x7f6de413a2a0 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 5 monclient: authenticate success, global_id 5096 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 4 set_mon_vals no callback set 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6de8b2c640 10 monclient: dump: 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:08.472 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:08.473 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 10 monclient: shutdown 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 1 -- 172.21.15.190:0/3890175063 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6de413a2a0 msgr2=0x7f6de40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 1 --2- 172.21.15.190:0/3890175063 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6de413a2a0 0x7f6de40a5730 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009870 tx=0x7f6dd401c160 comp rx=0 tx=0).stop 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 1 -- 172.21.15.190:0/3890175063 shutdown_connections 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 1 --2- 172.21.15.190:0/3890175063 >> v2:172.21.15.89:3300/0 conn(0x7f6de40a5c70 0x7f6de40a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 1 --2- 172.21.15.190:0/3890175063 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6de413a2a0 0x7f6de40a5730 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 1 --2- 172.21.15.190:0/3890175063 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6de4139980 0x7f6de4139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 1 -- 172.21.15.190:0/3890175063 >> 172.21.15.190:0/3890175063 conn(0x7f6de40abde0 msgr2=0x7f6de40ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 1 -- 172.21.15.190:0/3890175063 shutdown_connections 2024-08-19T10:41:08.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.462+0000 7f6debd92640 1 -- 172.21.15.190:0/3890175063 wait complete. 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient: build_initial_monmap 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient: monmap: 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:08.475 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 1 Processor -- start 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 1 -- start start 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient: init 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient: _add_conns ranks=[4,0,3,7,1,2,5,8,6] 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6de40a5c70 0x7f6de413fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient(hunting): picked mon.noname-c con 0x7f6de40a5c70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:08.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de4139980 0x7f6de413ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient(hunting): picked mon.noname-a con 0x7f6de4139980 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6de41d1580 0x7f6de41d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient(hunting): picked mon.noname-g con 0x7f6de41d1580 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6de4143270 con 0x7f6de4139980 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6de41430f0 con 0x7f6de41d1580 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6de41433f0 con 0x7f6de40a5c70 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6debd92640 10 monclient(hunting): authenticate will time out at 1114.037118s 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6dea32f640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6de41d1580 msgr2=0x7f6de41d19c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6dea32f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6de41d1580 0x7f6de41d19c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:08.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de9b2e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6de40a5c70 0x7f6de413fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de4139980 0x7f6de413ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de9b2e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6de40a5c70 0x7f6de413fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:53260/0 (socket says 172.21.15.190:53260) 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de9b2e640 1 -- 172.21.15.190:0/1778881197 learned_addr learned my addr 172.21.15.190:0/1778881197 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de9b2e640 10 monclient(hunting): get_auth_request con 0x7f6de40a5c70 auth_method 0 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de9b2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de9b2e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de9b2e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 10 monclient(hunting): get_auth_request con 0x7f6de4139980 auth_method 0 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:08.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de9b2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de9b2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de9b2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 10 monclient(hunting): handle_auth_done global_id 5202 payload 824 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 1 -- 172.21.15.190:0/1778881197 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6de40a5c70 msgr2=0x7f6de413fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6de40a5c70 0x7f6de413fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 1 -- 172.21.15.190:0/1778881197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6de41d1580 msgr2=0x7f6de41d19c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6de41d1580 0x7f6de41d19c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 10 monclient: _finish_hunting 0 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 1 monclient: found mon.noname-a 2024-08-19T10:41:08.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:08.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:08.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.466+0000 7f6de932d640 1 -- 172.21.15.190:0/1778881197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de40a58e0 con 0x7f6de4139980 2024-08-19T10:41:08.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6de932d640 1 -- 172.21.15.190:0/1778881197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de4139980 msgr2=0x7f6de413ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:08.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6de932d640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de4139980 0x7f6de413ffc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6de40a7de0 tx=0x7f6dd4018f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:08.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 1 -- 172.21.15.190:0/1778881197 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dd40b9030 con 0x7f6de4139980 2024-08-19T10:41:08.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:08.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:08.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 10 monclient: dump: 2024-08-19T10:41:08.482 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:08.482 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:08.482 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.482 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.482 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:08.482 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:08.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 1 -- 172.21.15.190:0/1778881197 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd40b1040 con 0x7f6de4139980 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6debd92640 5 monclient: authenticate success, global_id 5202 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6debd92640 10 monclient: _renew_subs 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 1 -- 172.21.15.190:0/1778881197 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dd40b0070 con 0x7f6de4139980 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6debd92640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6debd92640 1 -- 172.21.15.190:0/1778881197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de40a8fc0 con 0x7f6de4139980 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6ddbfff640 4 set_mon_vals no callback set 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 10 monclient: dump: 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.484 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6debd92640 10 monclient: _renew_subs 2024-08-19T10:41:08.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6debd92640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:08.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6debd92640 1 -- 172.21.15.190:0/1778881197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de413d540 con 0x7f6de4139980 2024-08-19T10:41:08.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6debd92640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:08.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6debd92640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:08.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6debd92640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:08.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6debd92640 1 -- 172.21.15.190:0/1778881197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de4139d60 con 0x7f6de4139980 2024-08-19T10:41:08.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 1 -- 172.21.15.190:0/1778881197 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6dd4004250 con 0x7f6de4139980 2024-08-19T10:41:08.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc803bdf0 0x7f6dc803e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:08.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6dda7fc640 1 -- 172.21.15.190:0/1778881197 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6120+0+0 (secure 0 0 0) 0x7f6dd40ba070 con 0x7f6de4139980 2024-08-19T10:41:08.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6de9b2e640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc803bdf0 0x7f6dc803e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:08.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.470+0000 7f6de9b2e640 10 monclient: get_auth_request con 0x7f6dc803bdf0 auth_method 0 2024-08-19T10:41:08.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.474+0000 7f6de9b2e640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc803bdf0 0x7f6dc803e2b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009ba0 tx=0x7f6de0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:08.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.482+0000 7f6dda7fc640 1 -- 172.21.15.190:0/1778881197 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6dd4016130 con 0x7f6de4139980 2024-08-19T10:41:08.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.482+0000 7f6dda7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:08.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.482+0000 7f6dda7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:08.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.662+0000 7f6debd92640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:08.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.662+0000 7f6debd92640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:08.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.662+0000 7f6debd92640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:08.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.662+0000 7f6debd92640 1 -- 172.21.15.190:0/1778881197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) -- 0x7f6de413bd40 con 0x7f6de4139980 2024-08-19T10:41:08.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.946+0000 7f6dda7fc640 1 -- 172.21.15.190:0/1778881197 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 added bucket bar type root to crush map v78) ==== 138+0+0 (secure 0 0 0) 0x7f6dd40aa7e0 con 0x7f6de4139980 2024-08-19T10:41:08.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.946+0000 7f6dda7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:08.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:08.946+0000 7f6dda7fc640 10 monclient: _finish_command 2 = system:0 added bucket bar type root to crush map 2024-08-19T10:41:09.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.054+0000 7f6debd92640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:09.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.054+0000 7f6debd92640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:09.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.054+0000 7f6debd92640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:09.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.054+0000 7f6debd92640 1 -- 172.21.15.190:0/1778881197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) -- 0x7f6de40008e0 con 0x7f6de4139980 2024-08-19T10:41:09.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.054+0000 7f6dda7fc640 1 -- 172.21.15.190:0/1778881197 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v78) ==== 126+0+0 (secure 0 0 0) 0x7f6dd40e9080 con 0x7f6de4139980 2024-08-19T10:41:09.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.054+0000 7f6dda7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:09.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.054+0000 7f6dda7fc640 10 monclient: _finish_command 3 = system:0 bucket 'bar' already exists 2024-08-19T10:41:09.057 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'bar' already exists 2024-08-19T10:41:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 -- 172.21.15.190:0/1778881197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc803bdf0 msgr2=0x7f6dc803e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc803bdf0 0x7f6dc803e2b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009ba0 tx=0x7f6de0009190 comp rx=0 tx=0).stop 2024-08-19T10:41:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 10 monclient: shutdown 2024-08-19T10:41:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 -- 172.21.15.190:0/1778881197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de4139980 msgr2=0x7f6de413ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de4139980 0x7f6de413ffc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6de40a7de0 tx=0x7f6dd4018f00 comp rx=0 tx=0).stop 2024-08-19T10:41:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 -- 172.21.15.190:0/1778881197 shutdown_connections 2024-08-19T10:41:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc803bdf0 0x7f6dc803e2b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6de41d1580 0x7f6de41d19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de4139980 0x7f6de413ffc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 --2- 172.21.15.190:0/1778881197 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6de40a5c70 0x7f6de413fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 -- 172.21.15.190:0/1778881197 >> 172.21.15.190:0/1778881197 conn(0x7f6de40abde0 msgr2=0x7f6de40ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:09.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 -- 172.21.15.190:0/1778881197 shutdown_connections 2024-08-19T10:41:09.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.058+0000 7f6debd92640 1 -- 172.21.15.190:0/1778881197 wait complete. 2024-08-19T10:41:09.075 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket bar root 2024-08-19T10:41:09.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.186+0000 7f005367b640 10 monclient: get_monmap_and_config 2024-08-19T10:41:09.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.186+0000 7f005367b640 10 monclient: build_initial_monmap 2024-08-19T10:41:09.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.186+0000 7f005367b640 10 monclient: monmap: 2024-08-19T10:41:09.190 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:09.191 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:09.191 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:09.190586+0000 2024-08-19T10:41:09.191 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:09.190586+0000 2024-08-19T10:41:09.191 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:09.191 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.191 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:09.191 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:09.191 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:09.191 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:09.191 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:09.191 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:09.192 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:09.192 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:09.192 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:09.192 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:09.192 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:09.192 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 1 Processor -- start 2024-08-19T10:41:09.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 1 -- start start 2024-08-19T10:41:09.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 10 monclient: init 2024-08-19T10:41:09.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:09.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 10 monclient: _add_conns ranks=[0,3,2,1,9,8,7,10,5,6,4] 2024-08-19T10:41:09.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f004c069430 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:09.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c0a05d0 0x7f004c0a4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 10 monclient(hunting): picked mon.noname-g con 0x7f004c0a05d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:09.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f004c0a48d0 0x7f004c13da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 10 monclient(hunting): picked mon.noname-b con 0x7f004c0a48d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:09.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f004c09e520 con 0x7f004c069430 2024-08-19T10:41:09.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:09.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f004c09fe60 con 0x7f004c0a48d0 2024-08-19T10:41:09.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:09.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f004c069810 con 0x7f004c0a05d0 2024-08-19T10:41:09.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:09.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f005367b640 10 monclient(hunting): authenticate will time out at 1114.761318s 2024-08-19T10:41:09.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0050c16640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c0a05d0 msgr2=0x7f004c0a4390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:09.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0050c16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c0a05d0 0x7f004c0a4390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:09.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051417640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f004c069430 0x7f004c0a3c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:51600/0 (socket says 172.21.15.190:51600) 2024-08-19T10:41:09.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f004c0a48d0 0x7f004c13da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:09.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051417640 1 -- 172.21.15.190:0/666460980 learned_addr learned my addr 172.21.15.190:0/666460980 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:09.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 10 monclient(hunting): get_auth_request con 0x7f004c0a48d0 auth_method 0 2024-08-19T10:41:09.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:09.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:09.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:09.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:09.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:09.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:09.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 10 monclient(hunting): handle_auth_done global_id 5105 payload 274 2024-08-19T10:41:09.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 1 -- 172.21.15.190:0/666460980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c0a05d0 msgr2=0x7f004c0a4390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:09.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 1 --2- 172.21.15.190:0/666460980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c0a05d0 0x7f004c0a4390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 1 -- 172.21.15.190:0/666460980 >> v1:172.21.15.89:6789/0 conn(0x7f004c069430 legacy=0x7f004c0a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:09.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 10 monclient: _finish_hunting 0 2024-08-19T10:41:09.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 1 monclient: found mon.noname-b 2024-08-19T10:41:09.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:09.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:09.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.190+0000 7f0051c18640 1 -- 172.21.15.190:0/666460980 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c0a09d0 con 0x7f004c0a48d0 2024-08-19T10:41:09.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0051c18640 1 -- 172.21.15.190:0/666460980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f004c0a48d0 msgr2=0x7f004c13da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:09.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0051c18640 1 --2- 172.21.15.190:0/666460980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f004c0a48d0 0x7f004c13da80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0044009df0 tx=0x7f004401c630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e4c9a33351fe5b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:09.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 1 -- 172.21.15.190:0/666460980 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00440ad070 con 0x7f004c0a48d0 2024-08-19T10:41:09.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:09.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:09.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 10 monclient: dump: 2024-08-19T10:41:09.205 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:09.205 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:09.205 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.205 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.205 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:09.205 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.206 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:09.206 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:09.206 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:09.206 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:09.206 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:09.206 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:09.206 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:09.206 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:09.206 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:09.206 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:09.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:09.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:09.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 1 -- 172.21.15.190:0/666460980 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00440a9950 con 0x7f004c0a48d0 2024-08-19T10:41:09.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:09.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 1 -- 172.21.15.190:0/666460980 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00440b0360 con 0x7f004c0a48d0 2024-08-19T10:41:09.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 5 monclient: authenticate success, global_id 5105 2024-08-19T10:41:09.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:09.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 4 set_mon_vals no callback set 2024-08-19T10:41:09.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:09.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:09.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f0043fff640 10 monclient: dump: 2024-08-19T10:41:09.208 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:09.208 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:09.208 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.209 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.209 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:09.209 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.209 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:09.209 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:09.209 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:09.209 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:09.209 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:09.209 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:09.209 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:09.210 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:09.210 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:09.210 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 10 monclient: shutdown 2024-08-19T10:41:09.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:09.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 1 -- 172.21.15.190:0/666460980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f004c0a48d0 msgr2=0x7f004c13da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:09.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 1 --2- 172.21.15.190:0/666460980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f004c0a48d0 0x7f004c13da80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0044009df0 tx=0x7f004401c630 comp rx=0 tx=0).stop 2024-08-19T10:41:09.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 1 -- 172.21.15.190:0/666460980 shutdown_connections 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 1 --2- 172.21.15.190:0/666460980 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f004c0a48d0 0x7f004c13da80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 1 --2- 172.21.15.190:0/666460980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f004c0a05d0 0x7f004c0a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 1 -- 172.21.15.190:0/666460980 >> 172.21.15.190:0/666460980 conn(0x7f004c0a4d70 msgr2=0x7f004c0a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 1 -- 172.21.15.190:0/666460980 shutdown_connections 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.194+0000 7f005367b640 1 -- 172.21.15.190:0/666460980 wait complete. 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient: build_initial_monmap 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient: monmap: 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.211 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 1 Processor -- start 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 1 -- start start 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient: init 2024-08-19T10:41:09.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient: _add_conns ranks=[1,5,6,7,8,0,3,4,2] 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c069430 0x7f004c13aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient(hunting): picked mon.noname-f con 0x7f004c069430 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f004c0a05d0 0x7f004c122590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient(hunting): picked mon.noname-h con 0x7f004c0a05d0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f004c124be0 0x7f004c122ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient(hunting): picked mon.noname-d con 0x7f004c124be0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f004c140960 con 0x7f004c069430 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f004c1407e0 con 0x7f004c0a05d0 2024-08-19T10:41:09.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f004c140ae0 con 0x7f004c124be0 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f005367b640 10 monclient(hunting): authenticate will time out at 1114.769187s 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c069430 0x7f004c13aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0050c16640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f004c0a05d0 0x7f004c122590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c069430 0x7f004c13aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39144/0 (socket says 172.21.15.190:39144) 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0050c16640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f004c0a05d0 0x7f004c122590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:46968/0 (socket says 172.21.15.190:46968) 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 1 -- 172.21.15.190:0/406385010 learned_addr learned my addr 172.21.15.190:0/406385010 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 10 monclient(hunting): get_auth_request con 0x7f004c069430 auth_method 0 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:09.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0050c16640 10 monclient(hunting): get_auth_request con 0x7f004c0a05d0 auth_method 0 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0050c16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0050c16640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0050c16640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051c18640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f004c124be0 0x7f004c122ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0050c16640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0050c16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0050c16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:09.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051c18640 10 monclient(hunting): get_auth_request con 0x7f004c124be0 auth_method 0 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051c18640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051c18640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051c18640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051c18640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051c18640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051c18640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 10 monclient(hunting): handle_auth_done global_id 14329 payload 824 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 1 -- 172.21.15.190:0/406385010 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f004c124be0 msgr2=0x7f004c122ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f004c124be0 0x7f004c122ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 1 -- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f004c0a05d0 msgr2=0x7f004c122590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f004c0a05d0 0x7f004c122590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 10 monclient: _finish_hunting 0 2024-08-19T10:41:09.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 1 monclient: found mon.noname-f 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 1 -- 172.21.15.190:0/406385010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c125b40 con 0x7f004c069430 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0050c16640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f004c0a05d0 0x7f004c122590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 1 -- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c069430 msgr2=0x7f004c13aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.198+0000 7f0051417640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c069430 0x7f004c13aa40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f004800c4a0 tx=0x7f004800c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 1 -- 172.21.15.190:0/406385010 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0048017070 con 0x7f004c069430 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 10 monclient: dump: 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:09.217 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.218 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.218 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:09.218 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.218 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:09.218 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:09.218 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:09.218 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:09.218 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:09.218 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 1 -- 172.21.15.190:0/406385010 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0048005d00 con 0x7f004c069430 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f005367b640 5 monclient: authenticate success, global_id 14329 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f005367b640 10 monclient: _renew_subs 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f005367b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:09.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f005367b640 1 -- 172.21.15.190:0/406385010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004c13b7e0 con 0x7f004c069430 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 1 -- 172.21.15.190:0/406385010 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00480074e0 con 0x7f004c069430 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f00437fe640 4 set_mon_vals no callback set 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 10 monclient: dump: 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.220 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f005367b640 10 monclient: _renew_subs 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f005367b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f005367b640 1 -- 172.21.15.190:0/406385010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004c124080 con 0x7f004c069430 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f005367b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:09.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f005367b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:09.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f005367b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:09.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f005367b640 1 -- 172.21.15.190:0/406385010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004c0a3c80 con 0x7f004c069430 2024-08-19T10:41:09.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 1 -- 172.21.15.190:0/406385010 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0048007070 con 0x7f004c069430 2024-08-19T10:41:09.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f003003be00 0x7f003003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0041ffb640 1 -- 172.21.15.190:0/406385010 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6147+0+0 (secure 0 0 0) 0x7f004804e380 con 0x7f004c069430 2024-08-19T10:41:09.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0050c16640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f003003be00 0x7f003003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:09.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0050c16640 10 monclient: get_auth_request con 0x7f003003be00 auth_method 0 2024-08-19T10:41:09.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.202+0000 7f0050c16640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f003003be00 0x7f003003e2c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f004c123cf0 tx=0x7f003c0090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:09.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.206+0000 7f0041ffb640 1 -- 172.21.15.190:0/406385010 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f004c125b40 con 0x7f004c069430 2024-08-19T10:41:09.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.206+0000 7f0041ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:09.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.206+0000 7f0041ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:09.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.374+0000 7f005367b640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:09.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.374+0000 7f005367b640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:09.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.374+0000 7f005367b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:09.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.374+0000 7f005367b640 1 -- 172.21.15.190:0/406385010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) -- 0x7f004c06e4d0 con 0x7f004c069430 2024-08-19T10:41:09.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.378+0000 7f0041ffb640 1 -- 172.21.15.190:0/406385010 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v78) ==== 126+0+0 (secure 0 0 0) 0x7f004804e0d0 con 0x7f004c069430 2024-08-19T10:41:09.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.378+0000 7f0041ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:09.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.378+0000 7f0041ffb640 10 monclient: _finish_command 2 = system:0 bucket 'bar' already exists 2024-08-19T10:41:09.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.482+0000 7f005367b640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:09.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.482+0000 7f005367b640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:09.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.482+0000 7f005367b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:09.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.482+0000 7f005367b640 1 -- 172.21.15.190:0/406385010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "bar", "type": "root"} v 0) -- 0x7f004c0a48d0 con 0x7f004c069430 2024-08-19T10:41:09.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.482+0000 7f0041ffb640 1 -- 172.21.15.190:0/406385010 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}]=0 bucket 'bar' already exists v78) ==== 126+0+0 (secure 0 0 0) 0x7f0048010300 con 0x7f004c069430 2024-08-19T10:41:09.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.482+0000 7f0041ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "bar", "type": "root"}] 2024-08-19T10:41:09.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.482+0000 7f0041ffb640 10 monclient: _finish_command 3 = system:0 bucket 'bar' already exists 2024-08-19T10:41:09.487 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'bar' already exists 2024-08-19T10:41:09.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 -- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f003003be00 msgr2=0x7f003003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:09.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f003003be00 0x7f003003e2c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f004c123cf0 tx=0x7f003c0090e0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 10 monclient: shutdown 2024-08-19T10:41:09.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:09.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 -- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c069430 msgr2=0x7f004c13aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:09.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c069430 0x7f004c13aa40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f004800c4a0 tx=0x7f004800c960 comp rx=0 tx=0).stop 2024-08-19T10:41:09.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 -- 172.21.15.190:0/406385010 shutdown_connections 2024-08-19T10:41:09.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f003003be00 0x7f003003e2c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f004c124be0 0x7f004c122ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f004c0a05d0 0x7f004c122590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 --2- 172.21.15.190:0/406385010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f004c069430 0x7f004c13aa40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 -- 172.21.15.190:0/406385010 >> 172.21.15.190:0/406385010 conn(0x7f004c0a4d70 msgr2=0x7f004c0a0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:09.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 -- 172.21.15.190:0/406385010 shutdown_connections 2024-08-19T10:41:09.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.486+0000 7f005367b640 1 -- 172.21.15.190:0/406385010 wait complete. 2024-08-19T10:41:09.504 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush link host1 root=bar 2024-08-19T10:41:09.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.614+0000 7fafde5b3640 10 monclient: get_monmap_and_config 2024-08-19T10:41:09.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.614+0000 7fafde5b3640 10 monclient: build_initial_monmap 2024-08-19T10:41:09.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.614+0000 7fafde5b3640 10 monclient: monmap: 2024-08-19T10:41:09.618 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:09.618 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:09.618 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:09.618472+0000 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:09.618472+0000 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:09.619 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:09.620 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.614+0000 7fafde5b3640 1 Processor -- start 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.614+0000 7fafde5b3640 1 -- start start 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.614+0000 7fafde5b3640 10 monclient: init 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 10 monclient: _add_conns ranks=[4,2,0,6,5,10,7,9,3,8,1] 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafd8139980 0x7fafd8139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 10 monclient(hunting): picked mon.noname-c con 0x7fafd8139980 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafd813a2a0 0x7fafd80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 10 monclient(hunting): picked mon.noname-b con 0x7fafd813a2a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fafd80a5c70 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd80a8470 con 0x7fafd80a5c70 2024-08-19T10:41:09.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fafd809ca60 con 0x7fafd813a2a0 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fafd80a3640 con 0x7fafd8139980 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafde5b3640 10 monclient(hunting): authenticate will time out at 1115.188377s 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafd8139980 0x7fafd8139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd77fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafd813a2a0 0x7fafd80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafd8139980 0x7fafd8139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:53276/0 (socket says 172.21.15.190:53276) 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 1 -- 172.21.15.190:0/1812059786 learned_addr learned my addr 172.21.15.190:0/1812059786 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 10 monclient(hunting): get_auth_request con 0x7fafd8139980 auth_method 0 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:09.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafdcb50640 1 --1- 172.21.15.190:0/1812059786 >> v1:172.21.15.89:6789/0 conn(0x7fafd80a5c70 0x7fafd80a67e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:51616/0 (socket says 172.21.15.190:51616) 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd77fe640 10 monclient(hunting): get_auth_request con 0x7fafd813a2a0 auth_method 0 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 10 monclient(hunting): handle_auth_done global_id 5170 payload 274 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 1 -- 172.21.15.190:0/1812059786 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafd813a2a0 msgr2=0x7fafd80a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 1 --2- 172.21.15.190:0/1812059786 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafd813a2a0 0x7fafd80a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 1 -- 172.21.15.190:0/1812059786 >> v1:172.21.15.89:6789/0 conn(0x7fafd80a5c70 legacy=0x7fafd80a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:09.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:09.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 1 monclient: found mon.noname-c 2024-08-19T10:41:09.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:09.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:09.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.618+0000 7fafd7fff640 1 -- 172.21.15.190:0/1812059786 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd80a3850 con 0x7fafd8139980 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd7fff640 1 -- 172.21.15.190:0/1812059786 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafd8139980 msgr2=0x7fafd8139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd7fff640 1 --2- 172.21.15.190:0/1812059786 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafd8139980 0x7fafd8139d60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0098a0 tx=0x7fafcc01c410 comp rx=0 tx=0).ready entity=mon.4 client_cookie=72741a47ff042e75 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 1 -- 172.21.15.190:0/1812059786 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fafcc0b0070 con 0x7fafd8139980 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 10 monclient: dump: 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.627 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:09.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 1 -- 172.21.15.190:0/1812059786 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafcc0aa810 con 0x7fafd8139980 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 1 -- 172.21.15.190:0/1812059786 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fafcc0aae30 con 0x7fafd8139980 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 5 monclient: authenticate success, global_id 5170 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 4 set_mon_vals no callback set 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafd6ffd640 10 monclient: dump: 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:09.629 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 10 monclient: shutdown 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:09.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 1 -- 172.21.15.190:0/1812059786 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafd8139980 msgr2=0x7fafd8139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 1 --2- 172.21.15.190:0/1812059786 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafd8139980 0x7fafd8139d60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0098a0 tx=0x7fafcc01c410 comp rx=0 tx=0).stop 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 1 -- 172.21.15.190:0/1812059786 shutdown_connections 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 1 --2- 172.21.15.190:0/1812059786 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafd813a2a0 0x7fafd80a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 1 --2- 172.21.15.190:0/1812059786 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fafd8139980 0x7fafd8139d60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 1 -- 172.21.15.190:0/1812059786 >> 172.21.15.190:0/1812059786 conn(0x7fafd80abde0 msgr2=0x7fafd80ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 1 -- 172.21.15.190:0/1812059786 shutdown_connections 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 1 -- 172.21.15.190:0/1812059786 wait complete. 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 10 monclient: build_initial_monmap 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.622+0000 7fafde5b3640 10 monclient: monmap: 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:09.631 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 1 Processor -- start 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 1 -- start start 2024-08-19T10:41:09.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient: init 2024-08-19T10:41:09.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:09.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient: _add_conns ranks=[2,0,7,6,3,4,8,1,5] 2024-08-19T10:41:09.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafd80a5c70 0x7fafd8140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient(hunting): picked mon.noname-b con 0x7fafd80a5c70 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafd8139980 0x7fafd8140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient(hunting): picked mon.noname-a con 0x7fafd8139980 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fafd81d1510 0x7fafd81d1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient(hunting): picked mon.noname-i con 0x7fafd81d1510 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8144280 con 0x7fafd8139980 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fafd8144100 con 0x7fafd80a5c70 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fafd8144400 con 0x7fafd81d1510 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:09.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient(hunting): authenticate will time out at 1115.196752s 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fafd81d1510 0x7fafd81d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fafd81d1510 0x7fafd81d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44232/0 (socket says 172.21.15.190:44232) 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 1 -- 172.21.15.190:0/1460588062 learned_addr learned my addr 172.21.15.190:0/1460588062 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 10 monclient(hunting): get_auth_request con 0x7fafd81d1510 auth_method 0 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd7fff640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafd80a5c70 0x7fafd8140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:09.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd77fe640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafd8139980 0x7fafd8140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd7fff640 10 monclient(hunting): get_auth_request con 0x7fafd80a5c70 auth_method 0 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd77fe640 10 monclient(hunting): get_auth_request con 0x7fafd8139980 auth_method 0 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 10 monclient(hunting): handle_auth_done global_id 14335 payload 824 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 1 -- 172.21.15.190:0/1460588062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafd80a5c70 msgr2=0x7fafd8140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafd80a5c70 0x7fafd8140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 1 -- 172.21.15.190:0/1460588062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafd8139980 msgr2=0x7fafd8140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafd8139980 0x7fafd8140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:09.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 10 monclient: _finish_hunting 0 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 1 monclient: found mon.noname-i 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 1 -- 172.21.15.190:0/1460588062 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd8077810 con 0x7fafd81d1510 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd7fff640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafd80a5c70 0x7fafd8140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 1 -- 172.21.15.190:0/1460588062 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fafd81d1510 msgr2=0x7fafd81d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafdcb50640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fafd81d1510 0x7fafd81d1950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fafd000c730 tx=0x7fafd000cbf0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 1 -- 172.21.15.190:0/1460588062 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fafd0017070 con 0x7fafd81d1510 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 10 monclient: dump: 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:09.640 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:09.641 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 10 monclient: _finish_auth 0 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 1 -- 172.21.15.190:0/1460588062 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd000ce40 con 0x7fafd81d1510 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 5 monclient: authenticate success, global_id 14335 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient: _renew_subs 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 1 -- 172.21.15.190:0/1460588062 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd80a1a60 con 0x7fafd81d1510 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 1 -- 172.21.15.190:0/1460588062 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fafd0007070 con 0x7fafd81d1510 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:09.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd4ff9640 10 monclient: dump: 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:09.643 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafd67fc640 4 set_mon_vals no callback set 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient: _renew_subs 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.626+0000 7fafde5b3640 1 -- 172.21.15.190:0/1460588062 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd813ca80 con 0x7fafd81d1510 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.630+0000 7fafd4ff9640 1 -- 172.21.15.190:0/1460588062 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fafd00072d0 con 0x7fafd81d1510 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.630+0000 7fafde5b3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.630+0000 7fafde5b3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.630+0000 7fafde5b3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.630+0000 7fafde5b3640 1 -- 172.21.15.190:0/1460588062 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd80ac430 con 0x7fafd81d1510 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.630+0000 7fafd4ff9640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fafc403be00 0x7fafc403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:09.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.630+0000 7fafd4ff9640 1 -- 172.21.15.190:0/1460588062 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(78..78 src has 1..78) ==== 6147+0+0 (secure 0 0 0) 0x7fafd004e610 con 0x7fafd81d1510 2024-08-19T10:41:09.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.630+0000 7fafd7fff640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fafc403be00 0x7fafc403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:09.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.630+0000 7fafd7fff640 10 monclient: get_auth_request con 0x7fafc403be00 auth_method 0 2024-08-19T10:41:09.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.630+0000 7fafd7fff640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fafc403be00 0x7fafc403e2c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0098a0 tx=0x7fafcc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:09.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.634+0000 7fafd4ff9640 1 -- 172.21.15.190:0/1460588062 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fafd004cab0 con 0x7fafd81d1510 2024-08-19T10:41:09.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.634+0000 7fafd4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:09.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.634+0000 7fafd4ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:09.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.802+0000 7fafde5b3640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:09.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.802+0000 7fafde5b3640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:09.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.802+0000 7fafde5b3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:09.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.802+0000 7fafde5b3640 1 -- 172.21.15.190:0/1460588062 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) -- 0x7fafd80ac680 con 0x7fafd81d1510 2024-08-19T10:41:09.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.954+0000 7fafd4ff9640 1 -- 172.21.15.190:0/1460588062 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 linked item id -11 name 'host1' to location {root=bar} in crush map v79) ==== 168+0+0 (secure 0 0 0) 0x7fafd80ac680 con 0x7fafd81d1510 2024-08-19T10:41:09.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.954+0000 7fafd4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:09.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:09.954+0000 7fafd4ff9640 10 monclient: _finish_command 2 = system:0 linked item id -11 name 'host1' to location {root=bar} in crush map 2024-08-19T10:41:10.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.058+0000 7fafde5b3640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:10.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.058+0000 7fafde5b3640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:10.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.058+0000 7fafde5b3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.058+0000 7fafde5b3640 1 -- 172.21.15.190:0/1460588062 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) -- 0x7fafd8139d60 con 0x7fafd81d1510 2024-08-19T10:41:10.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.058+0000 7fafd4ff9640 1 -- 172.21.15.190:0/1460588062 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v79) ==== 177+0+0 (secure 0 0 0) 0x7fafd000a640 con 0x7fafd81d1510 2024-08-19T10:41:10.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.058+0000 7fafd4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:10.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.058+0000 7fafd4ff9640 10 monclient: _finish_command 3 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-08-19T10:41:10.061 INFO:tasks.workunit.client.0.smithi190.stderr:no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-08-19T10:41:10.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 -- 172.21.15.190:0/1460588062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fafc403be00 msgr2=0x7fafc403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:10.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fafc403be00 0x7fafc403e2c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0098a0 tx=0x7fafcc0a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:10.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 10 monclient: shutdown 2024-08-19T10:41:10.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:10.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 -- 172.21.15.190:0/1460588062 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fafd81d1510 msgr2=0x7fafd81d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:10.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fafd81d1510 0x7fafd81d1950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fafd000c730 tx=0x7fafd000cbf0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 -- 172.21.15.190:0/1460588062 shutdown_connections 2024-08-19T10:41:10.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fafc403be00 0x7fafc403e2c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fafd8139980 0x7fafd8140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fafd81d1510 0x7fafd81d1950 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 --2- 172.21.15.190:0/1460588062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fafd80a5c70 0x7fafd8140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 -- 172.21.15.190:0/1460588062 >> 172.21.15.190:0/1460588062 conn(0x7fafd80abde0 msgr2=0x7fafd80a90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:10.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 -- 172.21.15.190:0/1460588062 shutdown_connections 2024-08-19T10:41:10.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.062+0000 7fafde5b3640 1 -- 172.21.15.190:0/1460588062 wait complete. 2024-08-19T10:41:10.077 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush link host1 root=bar 2024-08-19T10:41:10.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient: get_monmap_and_config 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient: build_initial_monmap 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient: monmap: 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:10.195891+0000 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:10.195891+0000 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:10.196 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:10.197 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:10.197 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:10.197 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:10.197 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:10.197 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:10.197 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:10.197 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:10.197 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 1 Processor -- start 2024-08-19T10:41:10.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 1 -- start start 2024-08-19T10:41:10.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient: init 2024-08-19T10:41:10.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:10.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient: _add_conns ranks=[8,4,5,9,2,1,6,3,10,0,7] 2024-08-19T10:41:10.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea700a6740 0x7fea700a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient(hunting): picked mon.noname-e con 0x7fea700a6740 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a5bc0 0x7fea700a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient(hunting): picked mon.noname-c con 0x7fea700a5bc0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fea700a7060 0x7fea70139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient(hunting): picked mon.noname-h con 0x7fea700a7060 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fea700a4c20 con 0x7fea700a5bc0 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fea700a3530 con 0x7fea700a7060 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fea700a3740 con 0x7fea700a6740 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:10.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea7506b640 10 monclient(hunting): authenticate will time out at 1115.765956s 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fea700a7060 0x7fea70139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fea700a7060 0x7fea70139590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:46972/0 (socket says 172.21.15.190:46972) 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 1 -- 172.21.15.190:0/225636534 learned_addr learned my addr 172.21.15.190:0/225636534 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 10 monclient(hunting): get_auth_request con 0x7fea700a7060 auth_method 0 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6ed9d640 1 --2- 172.21.15.190:0/225636534 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea700a6740 0x7fea700a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:10.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6e59c640 1 --2- 172.21.15.190:0/225636534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a5bc0 0x7fea700a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6ed9d640 10 monclient(hunting): get_auth_request con 0x7fea700a6740 auth_method 0 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6e59c640 10 monclient(hunting): get_auth_request con 0x7fea700a5bc0 auth_method 0 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 10 monclient(hunting): handle_auth_done global_id 5126 payload 274 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 1 -- 172.21.15.190:0/225636534 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea700a6740 msgr2=0x7fea700a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 1 --2- 172.21.15.190:0/225636534 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea700a6740 0x7fea700a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 1 -- 172.21.15.190:0/225636534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a5bc0 msgr2=0x7fea700a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:10.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 1 --2- 172.21.15.190:0/225636534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a5bc0 0x7fea700a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 10 monclient: _finish_hunting 0 2024-08-19T10:41:10.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 1 monclient: found mon.noname-h 2024-08-19T10:41:10.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:10.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:10.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6f59e640 1 -- 172.21.15.190:0/225636534 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea700a3240 con 0x7fea700a7060 2024-08-19T10:41:10.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6ed9d640 1 --2- 172.21.15.190:0/225636534 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea700a6740 0x7fea700a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:41:10.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.194+0000 7fea6e59c640 1 --2- 172.21.15.190:0/225636534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a5bc0 0x7fea700a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:41:10.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6f59e640 1 -- 172.21.15.190:0/225636534 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fea700a7060 msgr2=0x7fea70139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:10.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6f59e640 1 --2- 172.21.15.190:0/225636534 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fea700a7060 0x7fea70139590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fea680098a0 tx=0x7fea6801c440 comp rx=0 tx=0).ready entity=mon.5 client_cookie=5540e0eb57dae49f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:10.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 1 -- 172.21.15.190:0/225636534 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fea680b0070 con 0x7fea700a7060 2024-08-19T10:41:10.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:10.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:10.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 10 monclient: dump: 2024-08-19T10:41:10.210 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:10.211 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 10 monclient: _finish_auth 0 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 10 monclient: _check_auth_tickets 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 1 -- 172.21.15.190:0/225636534 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea680aa810 con 0x7fea700a7060 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 1 -- 172.21.15.190:0/225636534 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fea680aae30 con 0x7fea700a7060 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea7506b640 5 monclient: authenticate success, global_id 5126 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea7506b640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea7506b640 4 set_mon_vals no callback set 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:10.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.198+0000 7fea6dd9b640 10 monclient: dump: 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:10.213 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:10.214 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:10.214 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:10.214 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient: shutdown 2024-08-19T10:41:10.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:10.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 -- 172.21.15.190:0/225636534 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fea700a7060 msgr2=0x7fea70139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:10.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 --2- 172.21.15.190:0/225636534 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fea700a7060 0x7fea70139590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fea680098a0 tx=0x7fea6801c440 comp rx=0 tx=0).stop 2024-08-19T10:41:10.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 -- 172.21.15.190:0/225636534 shutdown_connections 2024-08-19T10:41:10.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 --2- 172.21.15.190:0/225636534 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fea700a7060 0x7fea70139590 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 --2- 172.21.15.190:0/225636534 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a5bc0 0x7fea700a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 --2- 172.21.15.190:0/225636534 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fea700a6740 0x7fea700a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 -- 172.21.15.190:0/225636534 >> 172.21.15.190:0/225636534 conn(0x7fea700abd40 msgr2=0x7fea700ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 -- 172.21.15.190:0/225636534 shutdown_connections 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 -- 172.21.15.190:0/225636534 wait complete. 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient: build_initial_monmap 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient: monmap: 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:10.215 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 Processor -- start 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 -- start start 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient: init 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient: _add_conns ranks=[4,3,7,1,6,0,8,5,2] 2024-08-19T10:41:10.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a6740 0x7fea70135810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient(hunting): picked mon.noname-c con 0x7fea700a6740 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700a7060 0x7fea70135d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient(hunting): picked mon.noname-g con 0x7fea700a7060 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea700a2170 0x7fea701d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient(hunting): picked mon.noname-i con 0x7fea700a2170 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fea70143450 con 0x7fea700a7060 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fea701432d0 con 0x7fea700a6740 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fea701435d0 con 0x7fea700a2170 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:10.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea7506b640 10 monclient(hunting): authenticate will time out at 1115.773952s 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6e59c640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700a7060 msgr2=0x7fea70135d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6e59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700a7060 0x7fea70135d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea700a2170 0x7fea701d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea700a2170 0x7fea701d17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44248/0 (socket says 172.21.15.190:44248) 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 1 -- 172.21.15.190:0/3040809680 learned_addr learned my addr 172.21.15.190:0/3040809680 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 10 monclient(hunting): get_auth_request con 0x7fea700a2170 auth_method 0 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:10.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6ed9d640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a6740 0x7fea70135810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6ed9d640 10 monclient(hunting): get_auth_request con 0x7fea700a6740 auth_method 0 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 10 monclient(hunting): handle_auth_done global_id 14344 payload 824 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 1 -- 172.21.15.190:0/3040809680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a6740 msgr2=0x7fea70135810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a6740 0x7fea70135810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 1 -- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700a7060 msgr2=0x7fea70135d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700a7060 0x7fea70135d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 10 monclient: _finish_hunting 0 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 1 monclient: found mon.noname-i 2024-08-19T10:41:10.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6f59e640 1 -- 172.21.15.190:0/3040809680 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea700ab270 con 0x7fea700a2170 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.202+0000 7fea6ed9d640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a6740 0x7fea70135810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea6f59e640 1 -- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea700a2170 msgr2=0x7fea701d17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea6f59e640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea700a2170 0x7fea701d17e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fea6801c270 tx=0x7fea680191e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 1 -- 172.21.15.190:0/3040809680 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fea680b0070 con 0x7fea700a2170 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 10 monclient: dump: 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:10.220 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea7506b640 5 monclient: authenticate success, global_id 14344 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 1 -- 172.21.15.190:0/3040809680 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea680ba070 con 0x7fea700a2170 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea7506b640 10 monclient: _renew_subs 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 1 -- 172.21.15.190:0/3040809680 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fea68004460 con 0x7fea700a2170 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea7506b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea7506b640 1 -- 172.21.15.190:0/3040809680 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea701400d0 con 0x7fea700a2170 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea6cd99640 4 set_mon_vals no callback set 2024-08-19T10:41:10.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 10 monclient: dump: 2024-08-19T10:41:10.223 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:10.223 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:10.223 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.223 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.223 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:10.223 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.223 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:10.223 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:10.223 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:10.223 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea7506b640 10 monclient: _renew_subs 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea7506b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea7506b640 1 -- 172.21.15.190:0/3040809680 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fea700a1710 con 0x7fea700a2170 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea7506b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea7506b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea7506b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea7506b640 1 -- 172.21.15.190:0/3040809680 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea700a5680 con 0x7fea700a2170 2024-08-19T10:41:10.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 1 -- 172.21.15.190:0/3040809680 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fea68003620 con 0x7fea700a2170 2024-08-19T10:41:10.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fea5c03be30 0x7fea5c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea577fe640 1 -- 172.21.15.190:0/3040809680 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(79..79 src has 1..79) ==== 6273+0+0 (secure 0 0 0) 0x7fea680e9230 con 0x7fea700a2170 2024-08-19T10:41:10.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea6ed9d640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fea5c03be30 0x7fea5c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:10.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.206+0000 7fea6ed9d640 10 monclient: get_auth_request con 0x7fea5c03be30 auth_method 0 2024-08-19T10:41:10.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.210+0000 7fea6ed9d640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fea5c03be30 0x7fea5c03e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea640098f0 tx=0x7fea64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:10.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.210+0000 7fea577fe640 1 -- 172.21.15.190:0/3040809680 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fea68016150 con 0x7fea700a2170 2024-08-19T10:41:10.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.210+0000 7fea577fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:10.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.210+0000 7fea577fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:10.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.386+0000 7fea7506b640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:10.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.386+0000 7fea7506b640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:10.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.386+0000 7fea7506b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.386+0000 7fea7506b640 1 -- 172.21.15.190:0/3040809680 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) -- 0x7fea70139890 con 0x7fea700a2170 2024-08-19T10:41:10.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.386+0000 7fea577fe640 1 -- 172.21.15.190:0/3040809680 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v79) ==== 177+0+0 (secure 0 0 0) 0x7fea680b4db0 con 0x7fea700a2170 2024-08-19T10:41:10.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.386+0000 7fea577fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:10.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.386+0000 7fea577fe640 10 monclient: _finish_command 2 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-08-19T10:41:10.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.490+0000 7fea7506b640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:10.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.490+0000 7fea7506b640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:10.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.490+0000 7fea7506b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.490+0000 7fea7506b640 1 -- 172.21.15.190:0/3040809680 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]} v 0) -- 0x7fea70137410 con 0x7fea700a2170 2024-08-19T10:41:10.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea577fe640 1 -- 172.21.15.190:0/3040809680 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}]=0 no need to move item id -11 name 'host1' to location {root=bar} in crush map v79) ==== 177+0+0 (secure 0 0 0) 0x7fea680bf460 con 0x7fea700a2170 2024-08-19T10:41:10.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea577fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host1", "args": ["root=bar"]}] 2024-08-19T10:41:10.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea577fe640 10 monclient: _finish_command 3 = system:0 no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-08-19T10:41:10.496 INFO:tasks.workunit.client.0.smithi190.stderr:no need to move item id -11 name 'host1' to location {root=bar} in crush map 2024-08-19T10:41:10.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 -- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fea5c03be30 msgr2=0x7fea5c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:10.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fea5c03be30 0x7fea5c03e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea640098f0 tx=0x7fea64008040 comp rx=0 tx=0).stop 2024-08-19T10:41:10.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 10 monclient: shutdown 2024-08-19T10:41:10.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:10.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 -- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea700a2170 msgr2=0x7fea701d17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:10.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea700a2170 0x7fea701d17e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fea6801c270 tx=0x7fea680191e0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 -- 172.21.15.190:0/3040809680 shutdown_connections 2024-08-19T10:41:10.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fea5c03be30 0x7fea5c03e2f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fea700a2170 0x7fea701d17e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea700a7060 0x7fea70135d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 --2- 172.21.15.190:0/3040809680 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fea700a6740 0x7fea70135810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 -- 172.21.15.190:0/3040809680 >> 172.21.15.190:0/3040809680 conn(0x7fea700abd40 msgr2=0x7fea70136540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:10.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 -- 172.21.15.190:0/3040809680 shutdown_connections 2024-08-19T10:41:10.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.494+0000 7fea7506b640 1 -- 172.21.15.190:0/3040809680 wait complete. 2024-08-19T10:41:10.513 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush link host2 root=bar 2024-08-19T10:41:10.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.622+0000 7fd6bead9640 10 monclient: get_monmap_and_config 2024-08-19T10:41:10.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.622+0000 7fd6bead9640 10 monclient: build_initial_monmap 2024-08-19T10:41:10.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.622+0000 7fd6bead9640 10 monclient: monmap: 2024-08-19T10:41:10.626 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:10.626 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:10.626 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:10.626552+0000 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:10.626552+0000 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:10.627 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:10.628 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:10.628 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.622+0000 7fd6bead9640 1 Processor -- start 2024-08-19T10:41:10.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.622+0000 7fd6bead9640 1 -- start start 2024-08-19T10:41:10.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.622+0000 7fd6bead9640 10 monclient: init 2024-08-19T10:41:10.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:10.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 10 monclient: _add_conns ranks=[1,9,6,0,2,10,8,3,4,5,7] 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd6b8132d50 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd6b8133130 0x7fd6b813e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 10 monclient(hunting): picked mon.noname-a con 0x7fd6b8133130 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6b8133590 0x7fd6b8140c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 10 monclient(hunting): picked mon.noname-d con 0x7fd6b8133590 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b809f120 con 0x7fd6b8132d50 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b809cad0 con 0x7fd6b8133130 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd6b809e130 con 0x7fd6b8133590 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:10.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bead9640 10 monclient(hunting): authenticate will time out at 1116.196400s 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bc875640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd6b8132d50 0x7fd6b809ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:59848/0 (socket says 172.21.15.190:59848) 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bc875640 1 -- 172.21.15.190:0/4106877689 learned_addr learned my addr 172.21.15.190:0/4106877689 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 1 --2- 172.21.15.190:0/4106877689 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6b8133590 0x7fd6b8140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 10 monclient(hunting): get_auth_request con 0x7fd6b8133590 auth_method 0 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6b7fff640 1 --2- 172.21.15.190:0/4106877689 >> v2:172.21.15.89:3300/0 conn(0x7fd6b8133130 0x7fd6b813e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6b7fff640 10 monclient(hunting): get_auth_request con 0x7fd6b8133130 auth_method 0 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6b7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:10.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 10 monclient(hunting): handle_auth_done global_id 5118 payload 274 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 1 -- 172.21.15.190:0/4106877689 >> v2:172.21.15.89:3300/0 conn(0x7fd6b8133130 msgr2=0x7fd6b813e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 1 --2- 172.21.15.190:0/4106877689 >> v2:172.21.15.89:3300/0 conn(0x7fd6b8133130 0x7fd6b813e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 1 -- 172.21.15.190:0/4106877689 >> v1:172.21.15.190:6789/0 conn(0x7fd6b8132d50 legacy=0x7fd6b809ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 10 monclient: _finish_hunting 0 2024-08-19T10:41:10.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 1 monclient: found mon.noname-d 2024-08-19T10:41:10.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:10.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:41:10.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.626+0000 7fd6bd076640 1 -- 172.21.15.190:0/4106877689 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b809db70 con 0x7fd6b8133590 2024-08-19T10:41:10.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bd076640 1 -- 172.21.15.190:0/4106877689 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6b8133590 msgr2=0x7fd6b8140c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:10.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bd076640 1 --2- 172.21.15.190:0/4106877689 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6b8133590 0x7fd6b8140c30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009d90 tx=0x7fd6a801d2f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c6ccc057bd9fbc33 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:10.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 1 -- 172.21.15.190:0/4106877689 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6a80b2070 con 0x7fd6b8133590 2024-08-19T10:41:10.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 10 monclient: dump: 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:10.638 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 1 -- 172.21.15.190:0/4106877689 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a80a5b60 con 0x7fd6b8133590 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 1 -- 172.21.15.190:0/4106877689 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6a80a5d10 con 0x7fd6b8133590 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 5 monclient: authenticate success, global_id 5118 2024-08-19T10:41:10.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 4 set_mon_vals no callback set 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6b77fe640 10 monclient: dump: 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:10.640 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 10 monclient: shutdown 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 1 -- 172.21.15.190:0/4106877689 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6b8133590 msgr2=0x7fd6b8140c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 1 --2- 172.21.15.190:0/4106877689 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6b8133590 0x7fd6b8140c30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009d90 tx=0x7fd6a801d2f0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 1 -- 172.21.15.190:0/4106877689 shutdown_connections 2024-08-19T10:41:10.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 1 --2- 172.21.15.190:0/4106877689 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6b8133590 0x7fd6b8140c30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 1 --2- 172.21.15.190:0/4106877689 >> v2:172.21.15.89:3300/0 conn(0x7fd6b8133130 0x7fd6b813e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 1 -- 172.21.15.190:0/4106877689 >> 172.21.15.190:0/4106877689 conn(0x7fd6b80a4d70 msgr2=0x7fd6b80a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 1 -- 172.21.15.190:0/4106877689 shutdown_connections 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 1 -- 172.21.15.190:0/4106877689 wait complete. 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 10 monclient: build_initial_monmap 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 10 monclient: monmap: 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:10.642 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.630+0000 7fd6bead9640 1 Processor -- start 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 1 -- start start 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient: init 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:10.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient: _add_conns ranks=[7,4,3,2,1,5,8,6,0] 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b8132d50 0x7fd6b813a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient(hunting): picked mon.noname-i con 0x7fd6b8132d50 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b8133130 0x7fd6b813abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient(hunting): picked mon.noname-c con 0x7fd6b8133130 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b8133590 0x7fd6b813b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient(hunting): picked mon.noname-g con 0x7fd6b8133590 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b8143b10 con 0x7fd6b8133590 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd6b8143990 con 0x7fd6b8133130 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd6b8143c90 con 0x7fd6b8132d50 2024-08-19T10:41:10.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient(hunting): authenticate will time out at 1116.204444s 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bd076640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b8133590 msgr2=0x7fd6b813b130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bd076640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b8133590 0x7fd6b813b130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b8132d50 0x7fd6b813a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b8132d50 0x7fd6b813a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44252/0 (socket says 172.21.15.190:44252) 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 1 -- 172.21.15.190:0/3212777207 learned_addr learned my addr 172.21.15.190:0/3212777207 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 10 monclient(hunting): get_auth_request con 0x7fd6b8132d50 auth_method 0 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:10.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b7fff640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b8133130 0x7fd6b813abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 10 monclient(hunting): handle_auth_done global_id 14353 payload 824 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 1 -- 172.21.15.190:0/3212777207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b8133130 msgr2=0x7fd6b813abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b8133130 0x7fd6b813abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 1 -- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b8133590 msgr2=0x7fd6b813b130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b8133590 0x7fd6b813b130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 10 monclient: _finish_hunting 0 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 1 monclient: found mon.noname-i 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 1 -- 172.21.15.190:0/3212777207 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b813be50 con 0x7fd6b8132d50 2024-08-19T10:41:10.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b7fff640 10 monclient: get_auth_request con 0x7fd6b8133130 auth_method 0 2024-08-19T10:41:10.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b7fff640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b8133130 0x7fd6b813abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:10.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 1 -- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b8132d50 msgr2=0x7fd6b813a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:10.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bc875640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b8132d50 0x7fd6b813a6b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c470 tx=0x7fd6b000c930 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:10.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 1 -- 172.21.15.190:0/3212777207 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6b0005d00 con 0x7fd6b8132d50 2024-08-19T10:41:10.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:10.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:10.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 10 monclient: dump: 2024-08-19T10:41:10.649 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:10.650 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 10 monclient: _finish_auth 0 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 5 monclient: authenticate success, global_id 14353 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient: _renew_subs 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 1 -- 172.21.15.190:0/3212777207 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b00074e0 con 0x7fd6b8132d50 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 1 -- 172.21.15.190:0/3212777207 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b80a72f0 con 0x7fd6b8132d50 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:10.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 1 -- 172.21.15.190:0/3212777207 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6b0007690 con 0x7fd6b8132d50 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b6ffd640 4 set_mon_vals no callback set 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6b57fa640 10 monclient: dump: 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:10.652 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient: _renew_subs 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.634+0000 7fd6bead9640 1 -- 172.21.15.190:0/3212777207 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b813d910 con 0x7fd6b8132d50 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.638+0000 7fd6bead9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.638+0000 7fd6bead9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.638+0000 7fd6bead9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.638+0000 7fd6b57fa640 1 -- 172.21.15.190:0/3212777207 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd6b0007070 con 0x7fd6b8132d50 2024-08-19T10:41:10.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.638+0000 7fd6bead9640 1 -- 172.21.15.190:0/3212777207 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b809ea10 con 0x7fd6b8132d50 2024-08-19T10:41:10.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.638+0000 7fd6b57fa640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6a403be00 0x7fd6a403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:10.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.638+0000 7fd6b57fa640 1 -- 172.21.15.190:0/3212777207 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(79..79 src has 1..79) ==== 6273+0+0 (secure 0 0 0) 0x7fd6b004eeb0 con 0x7fd6b8132d50 2024-08-19T10:41:10.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.638+0000 7fd6b7fff640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6a403be00 0x7fd6a403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:10.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.638+0000 7fd6b7fff640 10 monclient: get_auth_request con 0x7fd6a403be00 auth_method 0 2024-08-19T10:41:10.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.638+0000 7fd6b7fff640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6a403be00 0x7fd6a403e2c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd6b813a070 tx=0x7fd6ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:10.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.642+0000 7fd6b57fa640 1 -- 172.21.15.190:0/3212777207 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd6b004d2d0 con 0x7fd6b8132d50 2024-08-19T10:41:10.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.642+0000 7fd6b57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:10.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.642+0000 7fd6b57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:10.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.818+0000 7fd6bead9640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:10.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.818+0000 7fd6bead9640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:10.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.818+0000 7fd6bead9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:10.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.818+0000 7fd6bead9640 1 -- 172.21.15.190:0/3212777207 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) -- 0x7fd6b8140480 con 0x7fd6b8132d50 2024-08-19T10:41:10.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.982+0000 7fd6b57fa640 1 -- 172.21.15.190:0/3212777207 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 linked item id -16 name 'host2' to location {root=bar} in crush map v80) ==== 168+0+0 (secure 0 0 0) 0x7fd6b0010d10 con 0x7fd6b8132d50 2024-08-19T10:41:10.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.982+0000 7fd6b57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:10.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:10.982+0000 7fd6b57fa640 10 monclient: _finish_command 2 = system:0 linked item id -16 name 'host2' to location {root=bar} in crush map 2024-08-19T10:41:11.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.082+0000 7fd6bead9640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:11.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.082+0000 7fd6bead9640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:11.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.082+0000 7fd6bead9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:11.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.082+0000 7fd6bead9640 1 -- 172.21.15.190:0/3212777207 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) -- 0x7fd6b80008e0 con 0x7fd6b8132d50 2024-08-19T10:41:11.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6b57fa640 1 -- 172.21.15.190:0/3212777207 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v80) ==== 177+0+0 (secure 0 0 0) 0x7fd6b0019c20 con 0x7fd6b8132d50 2024-08-19T10:41:11.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6b57fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:11.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6b57fa640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-08-19T10:41:11.087 INFO:tasks.workunit.client.0.smithi190.stderr:no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-08-19T10:41:11.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 -- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6a403be00 msgr2=0x7fd6a403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6a403be00 0x7fd6a403e2c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd6b813a070 tx=0x7fd6ac008040 comp rx=0 tx=0).stop 2024-08-19T10:41:11.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 10 monclient: shutdown 2024-08-19T10:41:11.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:11.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 -- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b8132d50 msgr2=0x7fd6b813a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b8132d50 0x7fd6b813a6b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c470 tx=0x7fd6b000c930 comp rx=0 tx=0).stop 2024-08-19T10:41:11.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 -- 172.21.15.190:0/3212777207 shutdown_connections 2024-08-19T10:41:11.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6a403be00 0x7fd6a403e2c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b8133130 0x7fd6b813abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b8133590 0x7fd6b813b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 --2- 172.21.15.190:0/3212777207 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b8132d50 0x7fd6b813a6b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 -- 172.21.15.190:0/3212777207 >> 172.21.15.190:0/3212777207 conn(0x7fd6b80a4d70 msgr2=0x7fd6b809fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:11.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 -- 172.21.15.190:0/3212777207 shutdown_connections 2024-08-19T10:41:11.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.086+0000 7fd6bead9640 1 -- 172.21.15.190:0/3212777207 wait complete. 2024-08-19T10:41:11.104 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush link host2 root=bar 2024-08-19T10:41:11.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient: get_monmap_and_config 2024-08-19T10:41:11.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient: build_initial_monmap 2024-08-19T10:41:11.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient: monmap: 2024-08-19T10:41:11.220 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:11.220 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:11.220 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:11.219987+0000 2024-08-19T10:41:11.220 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:11.219987+0000 2024-08-19T10:41:11.220 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:11.220 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.220 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:11.220 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:11.220 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:11.221 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:11.221 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:11.221 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:11.221 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:11.221 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:11.221 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:11.221 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:11.221 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:11.221 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 1 Processor -- start 2024-08-19T10:41:11.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 1 -- start start 2024-08-19T10:41:11.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient: init 2024-08-19T10:41:11.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:11.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient: _add_conns ranks=[3,8,7,9,0,4,1,6,5,2,10] 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b2c0a6740 0x7f6b2c0a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient(hunting): picked mon.noname-g con 0x7f6b2c0a6740 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6b2c0a5bc0 0x7f6b2c0a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient(hunting): picked mon.noname-e con 0x7f6b2c0a5bc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b2c0a7060 0x7f6b2c139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient(hunting): picked mon.noname-i con 0x7f6b2c0a7060 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2c0a4c20 con 0x7f6b2c0a6740 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6b2c0a3530 con 0x7f6b2c0a7060 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:11.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6b2c0a3740 con 0x7f6b2c0a5bc0 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b30ebd640 10 monclient(hunting): authenticate will time out at 1116.789874s 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2a59c640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b2c0a6740 msgr2=0x7f6b2c0a5680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2a59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b2c0a6740 0x7f6b2c0a5680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b2c0a7060 0x7f6b2c139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b2c0a7060 0x7f6b2c139590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44264/0 (socket says 172.21.15.190:44264) 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 1 -- 172.21.15.190:0/4278584152 learned_addr learned my addr 172.21.15.190:0/4278584152 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 10 monclient(hunting): get_auth_request con 0x7f6b2c0a7060 auth_method 0 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:11.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:11.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:11.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 10 monclient(hunting): handle_auth_done global_id 14362 payload 274 2024-08-19T10:41:11.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 1 -- 172.21.15.190:0/4278584152 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6b2c0a5bc0 msgr2=0x7f6b2c0a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:11.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 1 --2- 172.21.15.190:0/4278584152 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6b2c0a5bc0 0x7f6b2c0a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 1 -- 172.21.15.190:0/4278584152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b2c0a6740 msgr2=0x7f6b2c0a5680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:11.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 1 --2- 172.21.15.190:0/4278584152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b2c0a6740 0x7f6b2c0a5680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 10 monclient: _finish_hunting 0 2024-08-19T10:41:11.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 1 monclient: found mon.noname-i 2024-08-19T10:41:11.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:11.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:11.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.218+0000 7f6b2ad9d640 1 -- 172.21.15.190:0/4278584152 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c0a3240 con 0x7f6b2c0a7060 2024-08-19T10:41:11.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2ad9d640 1 -- 172.21.15.190:0/4278584152 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b2c0a7060 msgr2=0x7f6b2c139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:11.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2ad9d640 1 --2- 172.21.15.190:0/4278584152 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b2c0a7060 0x7f6b2c139590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009e80 tx=0x7f6b2401c930 comp rx=0 tx=0).ready entity=mon.7 client_cookie=f2fea9758e57e7da server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:11.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 1 -- 172.21.15.190:0/4278584152 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b240091b0 con 0x7f6b2c0a7060 2024-08-19T10:41:11.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:11.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:11.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 10 monclient: dump: 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:11.231 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 10 monclient: _finish_auth 0 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 10 monclient: _check_auth_tickets 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 1 -- 172.21.15.190:0/4278584152 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b240a5aa0 con 0x7f6b2c0a7060 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 1 -- 172.21.15.190:0/4278584152 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b240a5c50 con 0x7f6b2c0a7060 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 5 monclient: authenticate success, global_id 14362 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 4 set_mon_vals no callback set 2024-08-19T10:41:11.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b2959a640 10 monclient: dump: 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:11.233 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 10 monclient: shutdown 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 1 -- 172.21.15.190:0/4278584152 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b2c0a7060 msgr2=0x7f6b2c139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 1 --2- 172.21.15.190:0/4278584152 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b2c0a7060 0x7f6b2c139590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009e80 tx=0x7f6b2401c930 comp rx=0 tx=0).stop 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 1 -- 172.21.15.190:0/4278584152 shutdown_connections 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 1 --2- 172.21.15.190:0/4278584152 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6b2c0a5bc0 0x7f6b2c0a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 1 --2- 172.21.15.190:0/4278584152 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b2c0a7060 0x7f6b2c139590 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 1 --2- 172.21.15.190:0/4278584152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b2c0a6740 0x7f6b2c0a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 1 -- 172.21.15.190:0/4278584152 >> 172.21.15.190:0/4278584152 conn(0x7f6b2c0abd40 msgr2=0x7f6b2c0ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 1 -- 172.21.15.190:0/4278584152 shutdown_connections 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.222+0000 7f6b30ebd640 1 -- 172.21.15.190:0/4278584152 wait complete. 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient: build_initial_monmap 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient: monmap: 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.235 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 1 Processor -- start 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 1 -- start start 2024-08-19T10:41:11.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient: init 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient: _add_conns ranks=[5,1,2,6,0,8,4,7,3] 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b2c0a6740 0x7f6b2c0a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient(hunting): picked mon.noname-h con 0x7f6b2c0a6740 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b2c0a7060 0x7f6b2c1d1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient(hunting): picked mon.noname-f con 0x7f6b2c0a7060 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b2c1d1aa0 0x7f6b2c1d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient(hunting): picked mon.noname-b con 0x7f6b2c1d1aa0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c143450 con 0x7f6b2c0a7060 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:11.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2c1432d0 con 0x7f6b2c1d1aa0 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6b2c1435d0 con 0x7f6b2c0a6740 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b30ebd640 10 monclient(hunting): authenticate will time out at 1116.797096s 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b2a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b2c0a6740 0x7f6b2c0a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b2c0a7060 0x7f6b2c1d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b2c0a7060 0x7f6b2c1d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39146/0 (socket says 172.21.15.190:39146) 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b2a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b2c0a6740 0x7f6b2c0a1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:46974/0 (socket says 172.21.15.190:46974) 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 1 -- 172.21.15.190:0/904704301 learned_addr learned my addr 172.21.15.190:0/904704301 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 10 monclient(hunting): get_auth_request con 0x7f6b2c0a7060 auth_method 0 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:11.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b2a59c640 10 monclient(hunting): get_auth_request con 0x7f6b2c0a6740 auth_method 0 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b2a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b2a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b2a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b2a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b2a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b2ad9d640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b2c1d1aa0 0x7f6b2c1d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:11.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 10 monclient(hunting): handle_auth_done global_id 14338 payload 824 2024-08-19T10:41:11.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 1 -- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b2c0a6740 msgr2=0x7f6b2c0a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b2c0a6740 0x7f6b2c0a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 1 -- 172.21.15.190:0/904704301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b2c1d1aa0 msgr2=0x7f6b2c1d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b2c1d1aa0 0x7f6b2c1d1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:41:11.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 1 monclient: found mon.noname-f 2024-08-19T10:41:11.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:11.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:11.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 1 -- 172.21.15.190:0/904704301 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c13fc30 con 0x7f6b2c0a7060 2024-08-19T10:41:11.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b2a59c640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b2c0a6740 0x7f6b2c0a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:11.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 1 -- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b2c0a7060 msgr2=0x7f6b2c1d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:11.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b29d9b640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b2c0a7060 0x7f6b2c1d1560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000b550 tx=0x7f6b2000ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:11.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b1affd640 1 -- 172.21.15.190:0/904704301 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b20014070 con 0x7f6b2c0a7060 2024-08-19T10:41:11.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.226+0000 7f6b1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 10 monclient: dump: 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:11.243 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 10 monclient: _finish_auth 0 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 1 -- 172.21.15.190:0/904704301 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b20010c90 con 0x7f6b2c0a7060 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b30ebd640 5 monclient: authenticate success, global_id 14338 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b30ebd640 10 monclient: _renew_subs 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b30ebd640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:11.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b30ebd640 1 -- 172.21.15.190:0/904704301 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c0a17e0 con 0x7f6b2c0a7060 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 1 -- 172.21.15.190:0/904704301 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b20010e40 con 0x7f6b2c0a7060 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 10 monclient: dump: 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.245 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b30ebd640 10 monclient: _renew_subs 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b28d99640 4 set_mon_vals no callback set 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b30ebd640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b30ebd640 1 -- 172.21.15.190:0/904704301 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c09f480 con 0x7f6b2c0a7060 2024-08-19T10:41:11.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b30ebd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b30ebd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b30ebd640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b30ebd640 1 -- 172.21.15.190:0/904704301 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2c0a5680 con 0x7f6b2c0a7060 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 1 -- 172.21.15.190:0/904704301 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6b20017070 con 0x7f6b2c0a7060 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b0c03be00 0x7f6b0c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b1affd640 1 -- 172.21.15.190:0/904704301 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6297+0+0 (secure 0 0 0) 0x7f6b2000d040 con 0x7f6b2c0a7060 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b2a59c640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b0c03be00 0x7f6b0c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b2a59c640 10 monclient: get_auth_request con 0x7f6b0c03be00 auth_method 0 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.230+0000 7f6b2a59c640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b0c03be00 0x7f6b0c03e2c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c003990 tx=0x7f6b1c009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.234+0000 7f6b1affd640 1 -- 172.21.15.190:0/904704301 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6b20050e20 con 0x7f6b2c0a7060 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.234+0000 7f6b1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:11.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.234+0000 7f6b1affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:11.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.394+0000 7f6b30ebd640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:11.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.394+0000 7f6b30ebd640 10 monclient: _send_command 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:11.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.394+0000 7f6b30ebd640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:11.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.394+0000 7f6b30ebd640 1 -- 172.21.15.190:0/904704301 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) -- 0x7f6b2c06c8d0 con 0x7f6b2c0a7060 2024-08-19T10:41:11.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.398+0000 7f6b1affd640 1 -- 172.21.15.190:0/904704301 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v80) ==== 177+0+0 (secure 0 0 0) 0x7f6b2c13fc30 con 0x7f6b2c0a7060 2024-08-19T10:41:11.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.398+0000 7f6b1affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:11.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.398+0000 7f6b1affd640 10 monclient: _finish_command 2 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-08-19T10:41:11.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.494+0000 7f6b30ebd640 10 start_mon_command cmd=[{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:11.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.494+0000 7f6b30ebd640 10 monclient: _send_command 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:11.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.494+0000 7f6b30ebd640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:11.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.494+0000 7f6b30ebd640 1 -- 172.21.15.190:0/904704301 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]} v 0) -- 0x7f6b2c0a1460 con 0x7f6b2c0a7060 2024-08-19T10:41:11.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.494+0000 7f6b1affd640 1 -- 172.21.15.190:0/904704301 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}]=0 no need to move item id -16 name 'host2' to location {root=bar} in crush map v80) ==== 177+0+0 (secure 0 0 0) 0x7f6b20021da0 con 0x7f6b2c0a7060 2024-08-19T10:41:11.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.494+0000 7f6b1affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush link", "name": "host2", "args": ["root=bar"]}] 2024-08-19T10:41:11.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.494+0000 7f6b1affd640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-08-19T10:41:11.499 INFO:tasks.workunit.client.0.smithi190.stderr:no need to move item id -16 name 'host2' to location {root=bar} in crush map 2024-08-19T10:41:11.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 -- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b0c03be00 msgr2=0x7f6b0c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b0c03be00 0x7f6b0c03e2c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c003990 tx=0x7f6b1c009110 comp rx=0 tx=0).stop 2024-08-19T10:41:11.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 10 monclient: shutdown 2024-08-19T10:41:11.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:11.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 -- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b2c0a7060 msgr2=0x7f6b2c1d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b2c0a7060 0x7f6b2c1d1560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000b550 tx=0x7f6b2000ba10 comp rx=0 tx=0).stop 2024-08-19T10:41:11.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 -- 172.21.15.190:0/904704301 shutdown_connections 2024-08-19T10:41:11.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b0c03be00 0x7f6b0c03e2c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b2c1d1aa0 0x7f6b2c1d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b2c0a7060 0x7f6b2c1d1560 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 --2- 172.21.15.190:0/904704301 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b2c0a6740 0x7f6b2c0a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 -- 172.21.15.190:0/904704301 >> 172.21.15.190:0/904704301 conn(0x7f6b2c0abd40 msgr2=0x7f6b2c140300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:11.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 -- 172.21.15.190:0/904704301 shutdown_connections 2024-08-19T10:41:11.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.498+0000 7f6b30ebd640 1 -- 172.21.15.190:0/904704301 wait complete. 2024-08-19T10:41:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -c osd.7 2024-08-19T10:41:11.515 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -q 2 2024-08-19T10:41:11.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.622+0000 7f1578218640 10 monclient: get_monmap_and_config 2024-08-19T10:41:11.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.622+0000 7f1578218640 10 monclient: build_initial_monmap 2024-08-19T10:41:11.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.622+0000 7f1578218640 10 monclient: monmap: 2024-08-19T10:41:11.627 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:11.627 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:11.627 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:11.627368+0000 2024-08-19T10:41:11.627 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:11.627368+0000 2024-08-19T10:41:11.627 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:11.628 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 1 Processor -- start 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 1 -- start start 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 10 monclient: init 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 10 monclient: _add_conns ranks=[2,3,1,9,5,10,7,6,8,0,4] 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15701399f0 0x7f15700a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 10 monclient(hunting): picked mon.noname-b con 0x7f15701399f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1570139dd0 0x7f15700a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 10 monclient(hunting): picked mon.noname-g con 0x7f1570139dd0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f157013a1b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15700a8340 con 0x7f157013a1b0 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:11.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f15700a4eb0 con 0x7f15701399f0 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f15700a5b80 con 0x7f1570139dd0 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1578218640 10 monclient(hunting): authenticate will time out at 1117.197292s 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f15757b3640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1570139dd0 msgr2=0x7f15700a68e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f15757b3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1570139dd0 0x7f15700a68e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f15767b5640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f157013a1b0 0x7f15700a6e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:59858/0 (socket says 172.21.15.190:59858) 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f15767b5640 1 -- 172.21.15.190:0/2724554209 learned_addr learned my addr 172.21.15.190:0/2724554209 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 1 --2- 172.21.15.190:0/2724554209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15701399f0 0x7f15700a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 10 monclient(hunting): get_auth_request con 0x7f15701399f0 auth_method 0 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:11.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:11.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:11.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:11.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 10 monclient(hunting): handle_auth_done global_id 5132 payload 274 2024-08-19T10:41:11.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 1 -- 172.21.15.190:0/2724554209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1570139dd0 msgr2=0x7f15700a68e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:11.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 1 --2- 172.21.15.190:0/2724554209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1570139dd0 0x7f15700a68e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 1 -- 172.21.15.190:0/2724554209 >> v1:172.21.15.190:6789/0 conn(0x7f157013a1b0 legacy=0x7f15700a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:11.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 10 monclient: _finish_hunting 0 2024-08-19T10:41:11.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 1 monclient: found mon.noname-b 2024-08-19T10:41:11.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:11.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:11.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.626+0000 7f1575fb4640 1 -- 172.21.15.190:0/2724554209 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15700a4580 con 0x7f15701399f0 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1575fb4640 1 -- 172.21.15.190:0/2724554209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15701399f0 msgr2=0x7f15700a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1575fb4640 1 --2- 172.21.15.190:0/2724554209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15701399f0 0x7f15700a5640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f15600097f0 tx=0x7f156001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cc6e9f33b37aa78 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 1 -- 172.21.15.190:0/2724554209 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f15600b0070 con 0x7f15701399f0 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 10 monclient: dump: 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:11.635 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 10 monclient: _finish_auth 0 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 10 monclient: _check_auth_tickets 2024-08-19T10:41:11.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 5 monclient: authenticate success, global_id 5132 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 1 -- 172.21.15.190:0/2724554209 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15600aa810 con 0x7f15701399f0 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 4 set_mon_vals no callback set 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 1 -- 172.21.15.190:0/2724554209 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f15600a93e0 con 0x7f15701399f0 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1574fb2640 10 monclient: dump: 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.637 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 10 monclient: shutdown 2024-08-19T10:41:11.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:11.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 1 -- 172.21.15.190:0/2724554209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15701399f0 msgr2=0x7f15700a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 1 --2- 172.21.15.190:0/2724554209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15701399f0 0x7f15700a5640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f15600097f0 tx=0x7f156001c3c0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 1 -- 172.21.15.190:0/2724554209 shutdown_connections 2024-08-19T10:41:11.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 1 --2- 172.21.15.190:0/2724554209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1570139dd0 0x7f15700a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 1 --2- 172.21.15.190:0/2724554209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f15701399f0 0x7f15700a5640 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 1 -- 172.21.15.190:0/2724554209 >> 172.21.15.190:0/2724554209 conn(0x7f1570074c10 msgr2=0x7f15700abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:11.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 1 -- 172.21.15.190:0/2724554209 shutdown_connections 2024-08-19T10:41:11.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.630+0000 7f1578218640 1 -- 172.21.15.190:0/2724554209 wait complete. 2024-08-19T10:41:11.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient: build_initial_monmap 2024-08-19T10:41:11.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient: monmap: 2024-08-19T10:41:11.643 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:11.644 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 1 Processor -- start 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 1 -- start start 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient: init 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient: _add_conns ranks=[1,0,5,8,7,4,3,6,2] 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15701399f0 0x7f15700a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient(hunting): picked mon.noname-f con 0x7f15701399f0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1570139dd0 0x7f157013ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient(hunting): picked mon.noname-a con 0x7f1570139dd0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:11.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f157013a1b0 0x7f157013b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient(hunting): picked mon.noname-h con 0x7f157013a1b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:11.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:11.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f15700a76b0 con 0x7f1570139dd0 2024-08-19T10:41:11.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:11.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f15700a7530 con 0x7f15701399f0 2024-08-19T10:41:11.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:11.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1570144120 con 0x7f157013a1b0 2024-08-19T10:41:11.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:11.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1578218640 10 monclient(hunting): authenticate will time out at 1117.205311s 2024-08-19T10:41:11.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f157013a1b0 0x7f157013b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1575fb4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15701399f0 0x7f15700a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f157013a1b0 0x7f157013b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:46976/0 (socket says 172.21.15.190:46976) 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 1 -- 172.21.15.190:0/3425663917 learned_addr learned my addr 172.21.15.190:0/3425663917 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 10 monclient(hunting): get_auth_request con 0x7f157013a1b0 auth_method 0 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1575fb4640 10 monclient(hunting): get_auth_request con 0x7f15701399f0 auth_method 0 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1575fb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1575fb4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1575fb4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15757b3640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1570139dd0 0x7f157013ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:11.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1575fb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1575fb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1575fb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15757b3640 10 monclient(hunting): get_auth_request con 0x7f1570139dd0 auth_method 0 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15757b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15757b3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15757b3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15757b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15757b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15757b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:11.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 10 monclient(hunting): handle_auth_done global_id 5144 payload 824 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 1 -- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15701399f0 msgr2=0x7f15700a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15701399f0 0x7f15700a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 1 -- 172.21.15.190:0/3425663917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1570139dd0 msgr2=0x7f157013ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1570139dd0 0x7f157013ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 10 monclient: _finish_hunting 0 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 1 monclient: found mon.noname-h 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f15767b5640 1 -- 172.21.15.190:0/3425663917 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1570141210 con 0x7f157013a1b0 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.634+0000 7f1575fb4640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15701399f0 0x7f15700a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15767b5640 1 -- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f157013a1b0 msgr2=0x7f157013b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15767b5640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f157013a1b0 0x7f157013b0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1568009fd0 tx=0x7f156800b870 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:11.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 1 -- 172.21.15.190:0/3425663917 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1568015020 con 0x7f157013a1b0 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 10 monclient: dump: 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:11.650 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 1 -- 172.21.15.190:0/3425663917 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1568011cb0 con 0x7f157013a1b0 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1578218640 5 monclient: authenticate success, global_id 5144 2024-08-19T10:41:11.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 1 -- 172.21.15.190:0/3425663917 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1568011e60 con 0x7f157013a1b0 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1567fff640 4 set_mon_vals no callback set 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 10 monclient: dump: 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:11.652 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1578218640 10 monclient: _renew_subs 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1578218640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1578218640 1 -- 172.21.15.190:0/3425663917 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157013d250 con 0x7f157013a1b0 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1578218640 10 monclient: _renew_subs 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1578218640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:11.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1578218640 1 -- 172.21.15.190:0/3425663917 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f157013b600 con 0x7f157013a1b0 2024-08-19T10:41:11.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1578218640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:11.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1578218640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:11.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1578218640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:11.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1578218640 1 -- 172.21.15.190:0/3425663917 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15700a5640 con 0x7f157013a1b0 2024-08-19T10:41:11.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 1 -- 172.21.15.190:0/3425663917 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1568018020 con 0x7f157013a1b0 2024-08-19T10:41:11.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f155403be00 0x7f155403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:11.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f15667fc640 1 -- 172.21.15.190:0/3425663917 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(80..80 src has 1..80) ==== 6297+0+0 (secure 0 0 0) 0x7f156804ee60 con 0x7f157013a1b0 2024-08-19T10:41:11.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1575fb4640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f155403be00 0x7f155403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:11.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.638+0000 7f1575fb4640 10 monclient: get_auth_request con 0x7f155403be00 auth_method 0 2024-08-19T10:41:11.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.642+0000 7f1575fb4640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f155403be00 0x7f155403e2c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f15600ad000 tx=0x7f15600a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:11.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.642+0000 7f15667fc640 1 -- 172.21.15.190:0/3425663917 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1568052030 con 0x7f157013a1b0 2024-08-19T10:41:11.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.642+0000 7f15667fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:11.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.642+0000 7f15667fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:11.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.802+0000 7f1578218640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.802+0000 7f1578218640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.802+0000 7f1578218640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.802+0000 7f1578218640 1 -- 172.21.15.190:0/3425663917 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f15700a3500 con 0x7f157013a1b0 2024-08-19T10:41:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.802+0000 7f15667fc640 1 -- 172.21.15.190:0/3425663917 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v80) ==== 56+0+1474 (secure 0 0 0) 0x7f156801f020 con 0x7f157013a1b0 2024-08-19T10:41:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.802+0000 7f15667fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.802+0000 7f15667fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:11.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.894+0000 7f1578218640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:11.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.894+0000 7f1578218640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:11.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.894+0000 7f1578218640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:11.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.894+0000 7f1578218640 1 -- 172.21.15.190:0/3425663917 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f15701d1ec0 con 0x7f157013a1b0 2024-08-19T10:41:11.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.894+0000 7f15667fc640 1 -- 172.21.15.190:0/3425663917 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v80) ==== 56+0+1474 (secure 0 0 0) 0x7f156801a400 con 0x7f157013a1b0 2024-08-19T10:41:11.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.894+0000 7f15667fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:11.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.894+0000 7f15667fc640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:11.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 -- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f155403be00 msgr2=0x7f155403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f155403be00 0x7f155403e2c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f15600ad000 tx=0x7f15600a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 10 monclient: shutdown 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 -- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f157013a1b0 msgr2=0x7f157013b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f157013a1b0 0x7f157013b0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1568009fd0 tx=0x7f156800b870 comp rx=0 tx=0).stop 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 -- 172.21.15.190:0/3425663917 shutdown_connections 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f155403be00 0x7f155403e2c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f157013a1b0 0x7f157013b0c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1570139dd0 0x7f157013ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 --2- 172.21.15.190:0/3425663917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15701399f0 0x7f15700a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 -- 172.21.15.190:0/3425663917 >> 172.21.15.190:0/3425663917 conn(0x7f1570074c10 msgr2=0x7f15700aab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 -- 172.21.15.190:0/3425663917 shutdown_connections 2024-08-19T10:41:11.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:11.898+0000 7f1578218640 1 -- 172.21.15.190:0/3425663917 wait complete. 2024-08-19T10:41:11.914 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:11.914 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -c host1 2024-08-19T10:41:11.914 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -q 2 2024-08-19T10:41:12.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.022+0000 7ff4a35a7640 10 monclient: get_monmap_and_config 2024-08-19T10:41:12.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.022+0000 7ff4a35a7640 10 monclient: build_initial_monmap 2024-08-19T10:41:12.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient: monmap: 2024-08-19T10:41:12.027 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:12.027 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.027 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:12.027570+0000 2024-08-19T10:41:12.027 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:12.027570+0000 2024-08-19T10:41:12.027 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:12.028 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 1 Processor -- start 2024-08-19T10:41:12.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 1 -- start start 2024-08-19T10:41:12.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient: init 2024-08-19T10:41:12.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:12.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient: _add_conns ranks=[9,5,2,0,10,8,4,3,7,6,1] 2024-08-19T10:41:12.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff49c0a5340 0x7ff49c1396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient(hunting): picked mon.noname-a con 0x7ff49c0a5340 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:12.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff49c139be0 0x7ff49c139fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient(hunting): picked mon.noname-h con 0x7ff49c139be0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:12.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff49c0a68e0 0x7ff49c0a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient(hunting): picked mon.noname-b con 0x7ff49c0a68e0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff49c0a2f30 con 0x7ff49c0a5340 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff49c0a3500 con 0x7ff49c0a68e0 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff49c13a500 con 0x7ff49c139be0 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a35a7640 10 monclient(hunting): authenticate will time out at 1117.597328s 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff49c139be0 0x7ff49c139fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff49c139be0 0x7ff49c139fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:46988/0 (socket says 172.21.15.190:46988) 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 1 -- 172.21.15.190:0/2695388915 learned_addr learned my addr 172.21.15.190:0/2695388915 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:12.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 10 monclient(hunting): get_auth_request con 0x7ff49c139be0 auth_method 0 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a1343640 1 --2- 172.21.15.190:0/2695388915 >> v2:172.21.15.89:3300/0 conn(0x7ff49c0a5340 0x7ff49c1396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a1b44640 1 --2- 172.21.15.190:0/2695388915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff49c0a68e0 0x7ff49c0a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a1b44640 10 monclient(hunting): get_auth_request con 0x7ff49c0a68e0 auth_method 0 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a1b44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a1b44640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a1b44640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a1343640 10 monclient(hunting): get_auth_request con 0x7ff49c0a5340 auth_method 0 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a1343640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a1343640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a1343640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 10 monclient(hunting): handle_auth_done global_id 5153 payload 274 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 1 -- 172.21.15.190:0/2695388915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff49c0a68e0 msgr2=0x7ff49c0a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 1 --2- 172.21.15.190:0/2695388915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff49c0a68e0 0x7ff49c0a6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 1 -- 172.21.15.190:0/2695388915 >> v2:172.21.15.89:3300/0 conn(0x7ff49c0a5340 msgr2=0x7ff49c1396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 1 --2- 172.21.15.190:0/2695388915 >> v2:172.21.15.89:3300/0 conn(0x7ff49c0a5340 0x7ff49c1396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 10 monclient: _finish_hunting 0 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 1 monclient: found mon.noname-h 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:12.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.026+0000 7ff4a0b42640 1 -- 172.21.15.190:0/2695388915 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c0a8610 con 0x7ff49c139be0 2024-08-19T10:41:12.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a0b42640 1 -- 172.21.15.190:0/2695388915 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff49c139be0 msgr2=0x7ff49c139fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:12.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a0b42640 1 --2- 172.21.15.190:0/2695388915 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff49c139be0 0x7ff49c139fc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff490009870 tx=0x7ff49001c420 comp rx=0 tx=0).ready entity=mon.5 client_cookie=1699bac133015945 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:12.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 1 -- 172.21.15.190:0/2695388915 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4900b0070 con 0x7ff49c139be0 2024-08-19T10:41:12.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:12.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 10 monclient: dump: 2024-08-19T10:41:12.038 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.038 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.039 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.039 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.039 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.039 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.039 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.039 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.039 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.039 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.039 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.039 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.039 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 10 monclient: _finish_auth 0 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 1 -- 172.21.15.190:0/2695388915 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4900aa810 con 0x7ff49c139be0 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 1 -- 172.21.15.190:0/2695388915 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4900a9300 con 0x7ff49c139be0 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 5 monclient: authenticate success, global_id 5153 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 4 set_mon_vals no callback set 2024-08-19T10:41:12.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff49bfff640 10 monclient: dump: 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.041 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 10 monclient: shutdown 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2695388915 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff49c139be0 msgr2=0x7ff49c139fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 1 --2- 172.21.15.190:0/2695388915 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff49c139be0 0x7ff49c139fc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff490009870 tx=0x7ff49001c420 comp rx=0 tx=0).stop 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2695388915 shutdown_connections 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 1 --2- 172.21.15.190:0/2695388915 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff49c0a68e0 0x7ff49c0a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 1 --2- 172.21.15.190:0/2695388915 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff49c139be0 0x7ff49c139fc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 1 --2- 172.21.15.190:0/2695388915 >> v2:172.21.15.89:3300/0 conn(0x7ff49c0a5340 0x7ff49c1396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2695388915 >> 172.21.15.190:0/2695388915 conn(0x7ff49c074bd0 msgr2=0x7ff49c0abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2695388915 shutdown_connections 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2695388915 wait complete. 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 10 monclient: build_initial_monmap 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.030+0000 7ff4a35a7640 10 monclient: monmap: 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.043 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 1 Processor -- start 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 1 -- start start 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 10 monclient: init 2024-08-19T10:41:12.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 10 monclient: _add_conns ranks=[0,6,8,3,7,2,1,5,4] 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c0a5340 0x7ff49c135a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 10 monclient(hunting): picked mon.noname-a con 0x7ff49c0a5340 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff49c0a68e0 0x7ff49c135f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 10 monclient(hunting): picked mon.noname-d con 0x7ff49c0a68e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff49c139be0 0x7ff49c1d1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 10 monclient(hunting): picked mon.noname-e con 0x7ff49c139be0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c143320 con 0x7ff49c0a5340 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff49c0a8360 con 0x7ff49c0a68e0 2024-08-19T10:41:12.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff49c1434a0 con 0x7ff49c139be0 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a35a7640 10 monclient(hunting): authenticate will time out at 1117.604629s 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a0b42640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff49c0a68e0 0x7ff49c135f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c0a5340 0x7ff49c135a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c0a5340 0x7ff49c135a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:44106/0 (socket says 172.21.15.190:44106) 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1b44640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff49c139be0 0x7ff49c1d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a0b42640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff49c0a68e0 0x7ff49c135f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:35006/0 (socket says 172.21.15.190:35006) 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 1 -- 172.21.15.190:0/2106727681 learned_addr learned my addr 172.21.15.190:0/2106727681 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 10 monclient(hunting): get_auth_request con 0x7ff49c0a5340 auth_method 0 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a0b42640 10 monclient(hunting): get_auth_request con 0x7ff49c0a68e0 auth_method 0 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a0b42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a0b42640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a0b42640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1b44640 10 monclient(hunting): get_auth_request con 0x7ff49c139be0 auth_method 0 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1b44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1b44640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1b44640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:12.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a0b42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a0b42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a0b42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1b44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1b44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1b44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 10 monclient(hunting): handle_auth_done global_id 5247 payload 824 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 1 -- 172.21.15.190:0/2106727681 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff49c139be0 msgr2=0x7ff49c1d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff49c139be0 0x7ff49c1d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 1 -- 172.21.15.190:0/2106727681 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff49c0a68e0 msgr2=0x7ff49c135f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff49c0a68e0 0x7ff49c135f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 10 monclient: _finish_hunting 0 2024-08-19T10:41:12.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 1 monclient: found mon.noname-a 2024-08-19T10:41:12.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:12.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:12.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 1 -- 172.21.15.190:0/2106727681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c0a1790 con 0x7ff49c0a5340 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 1 -- 172.21.15.190:0/2106727681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c0a5340 msgr2=0x7ff49c135a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.034+0000 7ff4a1343640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c0a5340 0x7ff49c135a40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007e40 tx=0x7ff48c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 1 -- 172.21.15.190:0/2106727681 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff48c00ee30 con 0x7ff49c0a5340 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 10 monclient: dump: 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.051 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:12.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a35a7640 5 monclient: authenticate success, global_id 5247 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a35a7640 10 monclient: _renew_subs 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 1 -- 172.21.15.190:0/2106727681 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c019070 con 0x7ff49c0a5340 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a35a7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2106727681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c140070 con 0x7ff49c0a5340 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 1 -- 172.21.15.190:0/2106727681 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff48c013a00 con 0x7ff49c0a5340 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 10 monclient: dump: 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.053 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.054 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a35a7640 10 monclient: _renew_subs 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a35a7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2106727681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c0ab2b0 con 0x7ff49c0a5340 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff49b7fe640 4 set_mon_vals no callback set 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a35a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a35a7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a35a7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2106727681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c0a6d50 con 0x7ff49c0a5340 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 1 -- 172.21.15.190:0/2106727681 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff48c013c20 con 0x7ff49c0a5340 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff48803be00 0x7ff48803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff499ffb640 1 -- 172.21.15.190:0/2106727681 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6297+0+0 (secure 0 0 0) 0x7ff48c04fdd0 con 0x7ff49c0a5340 2024-08-19T10:41:12.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a0b42640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff48803be00 0x7ff48803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.038+0000 7ff4a0b42640 10 monclient: get_auth_request con 0x7ff48803be00 auth_method 0 2024-08-19T10:41:12.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.042+0000 7ff4a0b42640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff48803be00 0x7ff48803e2c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0acf10 tx=0x7ff490004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:12.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.050+0000 7ff499ffb640 1 -- 172.21.15.190:0/2106727681 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff48c023020 con 0x7ff49c0a5340 2024-08-19T10:41:12.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.050+0000 7ff499ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:12.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.050+0000 7ff499ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:12.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.218+0000 7ff4a35a7640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:12.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.218+0000 7ff4a35a7640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:12.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.218+0000 7ff4a35a7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:12.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.218+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2106727681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7ff49c09f430 con 0x7ff49c0a5340 2024-08-19T10:41:12.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.222+0000 7ff499ffb640 1 -- 172.21.15.190:0/2106727681 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v80) ==== 56+0+1474 (secure 0 0 0) 0x7ff48c013770 con 0x7ff49c0a5340 2024-08-19T10:41:12.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.222+0000 7ff499ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:12.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.222+0000 7ff499ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:12.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.314+0000 7ff4a35a7640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:12.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.314+0000 7ff4a35a7640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:12.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.314+0000 7ff4a35a7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:12.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.314+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2106727681 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7ff49c1364c0 con 0x7ff49c0a5340 2024-08-19T10:41:12.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff499ffb640 1 -- 172.21.15.190:0/2106727681 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v80) ==== 56+0+1474 (secure 0 0 0) 0x7ff48c004020 con 0x7ff49c0a5340 2024-08-19T10:41:12.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff499ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:12.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff499ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:12.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2106727681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff48803be00 msgr2=0x7ff48803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff48803be00 0x7ff48803e2c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0acf10 tx=0x7ff490004740 comp rx=0 tx=0).stop 2024-08-19T10:41:12.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 10 monclient: shutdown 2024-08-19T10:41:12.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:12.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2106727681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c0a5340 msgr2=0x7ff49c135a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c0a5340 0x7ff49c135a40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007e40 tx=0x7ff48c00e4b0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2106727681 shutdown_connections 2024-08-19T10:41:12.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff48803be00 0x7ff48803e2c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff49c139be0 0x7ff49c1d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff49c0a68e0 0x7ff49c135f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 --2- 172.21.15.190:0/2106727681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff49c0a5340 0x7ff49c135a40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2106727681 >> 172.21.15.190:0/2106727681 conn(0x7ff49c074bd0 msgr2=0x7ff49c0a9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:12.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2106727681 shutdown_connections 2024-08-19T10:41:12.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.318+0000 7ff4a35a7640 1 -- 172.21.15.190:0/2106727681 wait complete. 2024-08-19T10:41:12.336 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:12.336 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -c osd.9 2024-08-19T10:41:12.336 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -q 2 2024-08-19T10:41:12.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.442+0000 7f400d1b4640 10 monclient: get_monmap_and_config 2024-08-19T10:41:12.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.442+0000 7f400d1b4640 10 monclient: build_initial_monmap 2024-08-19T10:41:12.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.442+0000 7f400d1b4640 10 monclient: monmap: 2024-08-19T10:41:12.447 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:12.447 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.447 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:12.447461+0000 2024-08-19T10:41:12.447 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:12.447461+0000 2024-08-19T10:41:12.447 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:12.448 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.448 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:12.448 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:12.448 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:12.448 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:12.448 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:12.448 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:12.448 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:12.448 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:12.448 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:12.449 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:12.449 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:12.449 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 1 Processor -- start 2024-08-19T10:41:12.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 1 -- start start 2024-08-19T10:41:12.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 10 monclient: init 2024-08-19T10:41:12.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:12.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 10 monclient: _add_conns ranks=[9,4,3,6,7,5,1,0,2,10,8] 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40081399f0 0x7f40080a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 10 monclient(hunting): picked mon.noname-a con 0x7f40081399f0 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4008139dd0 0x7f40080a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 10 monclient(hunting): picked mon.noname-c con 0x7f4008139dd0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f400813a1b0 0x7f40080a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 10 monclient(hunting): picked mon.noname-g con 0x7f400813a1b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f40080a4eb0 con 0x7f40081399f0 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f40080a5b80 con 0x7f400813a1b0 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f40080a8340 con 0x7f4008139dd0 2024-08-19T10:41:12.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:12.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400d1b4640 10 monclient(hunting): authenticate will time out at 1118.017402s 2024-08-19T10:41:12.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400759e640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f400813a1b0 msgr2=0x7f40080a6e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:12.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f400813a1b0 0x7f40080a6e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:12.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40081399f0 0x7f40080a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40081399f0 0x7f40080a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:44108/0 (socket says 172.21.15.190:44108) 2024-08-19T10:41:12.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 1 -- 172.21.15.190:0/746155267 learned_addr learned my addr 172.21.15.190:0/746155267 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:12.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 10 monclient(hunting): get_auth_request con 0x7f40081399f0 auth_method 0 2024-08-19T10:41:12.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400659c640 1 --2- 172.21.15.190:0/746155267 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4008139dd0 0x7f40080a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400659c640 10 monclient(hunting): get_auth_request con 0x7f4008139dd0 auth_method 0 2024-08-19T10:41:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:12.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f400659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:12.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 10 monclient(hunting): handle_auth_done global_id 5256 payload 274 2024-08-19T10:41:12.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 1 -- 172.21.15.190:0/746155267 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4008139dd0 msgr2=0x7f40080a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:12.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 1 --2- 172.21.15.190:0/746155267 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4008139dd0 0x7f40080a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 1 -- 172.21.15.190:0/746155267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f400813a1b0 msgr2=0x7f40080a6e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:12.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 1 --2- 172.21.15.190:0/746155267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f400813a1b0 0x7f40080a6e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:41:12.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 1 monclient: found mon.noname-a 2024-08-19T10:41:12.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:12.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:12.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.446+0000 7f4006d9d640 1 -- 172.21.15.190:0/746155267 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40080a4580 con 0x7f40081399f0 2024-08-19T10:41:12.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4006d9d640 1 -- 172.21.15.190:0/746155267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40081399f0 msgr2=0x7f40080a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:12.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4006d9d640 1 --2- 172.21.15.190:0/746155267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40081399f0 0x7f40080a5640 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00098a0 tx=0x7f3ff001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c5a0d7d470cb97 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:12.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 1 -- 172.21.15.190:0/746155267 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3ff00b0070 con 0x7f40081399f0 2024-08-19T10:41:12.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:12.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 10 monclient: dump: 2024-08-19T10:41:12.465 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.465 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.466 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.466 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.466 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.466 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.466 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.466 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.467 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.467 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.467 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.467 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.468 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.468 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.468 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.468 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 10 monclient: _finish_auth 0 2024-08-19T10:41:12.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:41:12.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 5 monclient: authenticate success, global_id 5256 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 1 -- 172.21.15.190:0/746155267 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff00aa810 con 0x7f40081399f0 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 4 set_mon_vals no callback set 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 1 -- 172.21.15.190:0/746155267 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3ff00a9300 con 0x7f40081399f0 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f4005d9b640 10 monclient: dump: 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.469 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.470 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.471 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 10 monclient: shutdown 2024-08-19T10:41:12.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:12.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 1 -- 172.21.15.190:0/746155267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40081399f0 msgr2=0x7f40080a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 1 --2- 172.21.15.190:0/746155267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40081399f0 0x7f40080a5640 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00098a0 tx=0x7f3ff001c3f0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 1 -- 172.21.15.190:0/746155267 shutdown_connections 2024-08-19T10:41:12.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 1 --2- 172.21.15.190:0/746155267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f400813a1b0 0x7f40080a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 1 --2- 172.21.15.190:0/746155267 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4008139dd0 0x7f40080a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 1 --2- 172.21.15.190:0/746155267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40081399f0 0x7f40080a5640 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 1 -- 172.21.15.190:0/746155267 >> 172.21.15.190:0/746155267 conn(0x7f4008074c10 msgr2=0x7f40080abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 1 -- 172.21.15.190:0/746155267 shutdown_connections 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.450+0000 7f400d1b4640 1 -- 172.21.15.190:0/746155267 wait complete. 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient: build_initial_monmap 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient: monmap: 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:12.472 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 1 Processor -- start 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 1 -- start start 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient: init 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient: _add_conns ranks=[8,1,6,5,0,3,2,4,7] 2024-08-19T10:41:12.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40081399f0 0x7f400813fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient(hunting): picked mon.noname-e con 0x7f40081399f0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4008139dd0 0x7f4008140110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient(hunting): picked mon.noname-f con 0x7f4008139dd0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f400813a1b0 0x7f400809f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient(hunting): picked mon.noname-d con 0x7f400813a1b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4008143110 con 0x7f4008139dd0 2024-08-19T10:41:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f40080a3fd0 con 0x7f400813a1b0 2024-08-19T10:41:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4008143290 con 0x7f40081399f0 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400d1b4640 10 monclient(hunting): authenticate will time out at 1118.025199s 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4008139dd0 0x7f4008140110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4008139dd0 0x7f4008140110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39166/0 (socket says 172.21.15.190:39166) 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 1 -- 172.21.15.190:0/2948501110 learned_addr learned my addr 172.21.15.190:0/2948501110 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 10 monclient(hunting): get_auth_request con 0x7f4008139dd0 auth_method 0 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:12.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400759e640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f400813a1b0 0x7f400809f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400759e640 10 monclient(hunting): get_auth_request con 0x7f400813a1b0 auth_method 0 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f4006d9d640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40081399f0 0x7f400813fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400759e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400759e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f4006d9d640 10 monclient(hunting): get_auth_request con 0x7f40081399f0 auth_method 0 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f4006d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f4006d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f4006d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 10 monclient(hunting): handle_auth_done global_id 14356 payload 824 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 1 -- 172.21.15.190:0/2948501110 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40081399f0 msgr2=0x7f400813fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40081399f0 0x7f400813fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 1 -- 172.21.15.190:0/2948501110 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f400813a1b0 msgr2=0x7f400809f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f400813a1b0 0x7f400809f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 10 monclient: _finish_hunting 0 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 1 monclient: found mon.noname-f 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 1 -- 172.21.15.190:0/2948501110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40080abc30 con 0x7f4008139dd0 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400759e640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f400813a1b0 0x7f400809f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 1 -- 172.21.15.190:0/2948501110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4008139dd0 msgr2=0x7f4008140110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.454+0000 7f400659c640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4008139dd0 0x7f4008140110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800e930 tx=0x7f3ff800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 1 -- 172.21.15.190:0/2948501110 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3ff8017070 con 0x7f4008139dd0 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 10 monclient: dump: 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.478 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400d1b4640 5 monclient: authenticate success, global_id 14356 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 1 -- 172.21.15.190:0/2948501110 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff800cc10 con 0x7f4008139dd0 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 1 -- 172.21.15.190:0/2948501110 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3ff800cdc0 con 0x7f4008139dd0 2024-08-19T10:41:12.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400d1b4640 10 monclient: _renew_subs 2024-08-19T10:41:12.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400d1b4640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:12.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400d1b4640 1 -- 172.21.15.190:0/2948501110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400813b670 con 0x7f4008139dd0 2024-08-19T10:41:12.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:12.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400559a640 4 set_mon_vals no callback set 2024-08-19T10:41:12.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 10 monclient: dump: 2024-08-19T10:41:12.480 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.480 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.480 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.480 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400d1b4640 10 monclient: _renew_subs 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400d1b4640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400d1b4640 1 -- 172.21.15.190:0/2948501110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4008140650 con 0x7f4008139dd0 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400d1b4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400d1b4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400d1b4640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f400d1b4640 1 -- 172.21.15.190:0/2948501110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f400813b010 con 0x7f4008139dd0 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 1 -- 172.21.15.190:0/2948501110 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3ff801a070 con 0x7f4008139dd0 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3fe803be00 0x7f3fe803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f3fff7fe640 1 -- 172.21.15.190:0/2948501110 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6297+0+0 (secure 0 0 0) 0x7f3ff800f040 con 0x7f4008139dd0 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f4006d9d640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3fe803be00 0x7f3fe803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.458+0000 7f4006d9d640 10 monclient: get_auth_request con 0x7f3fe803be00 auth_method 0 2024-08-19T10:41:12.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.462+0000 7f4006d9d640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3fe803be00 0x7f3fe803e2c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f400806ce50 tx=0x7f3ff00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:12.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.462+0000 7f3fff7fe640 1 -- 172.21.15.190:0/2948501110 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3ff8021070 con 0x7f4008139dd0 2024-08-19T10:41:12.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.462+0000 7f3fff7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:12.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.462+0000 7f3fff7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.618+0000 7f400d1b4640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.618+0000 7f400d1b4640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.618+0000 7f400d1b4640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.618+0000 7f400d1b4640 1 -- 172.21.15.190:0/2948501110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f40080a5640 con 0x7f4008139dd0 2024-08-19T10:41:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.622+0000 7f3fff7fe640 1 -- 172.21.15.190:0/2948501110 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v80) ==== 56+0+1474 (secure 0 0 0) 0x7f40080abc30 con 0x7f4008139dd0 2024-08-19T10:41:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.622+0000 7f3fff7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.622+0000 7f3fff7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:12.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.714+0000 7f400d1b4640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:12.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.714+0000 7f400d1b4640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:12.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.714+0000 7f400d1b4640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:12.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.714+0000 7f400d1b4640 1 -- 172.21.15.190:0/2948501110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f40080a0d00 con 0x7f4008139dd0 2024-08-19T10:41:12.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.714+0000 7f3fff7fe640 1 -- 172.21.15.190:0/2948501110 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v80) ==== 56+0+1474 (secure 0 0 0) 0x7f3ff8005dd0 con 0x7f4008139dd0 2024-08-19T10:41:12.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.714+0000 7f3fff7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:12.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.714+0000 7f3fff7fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 -- 172.21.15.190:0/2948501110 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3fe803be00 msgr2=0x7f3fe803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3fe803be00 0x7f3fe803e2c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f400806ce50 tx=0x7f3ff00a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 10 monclient: shutdown 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 -- 172.21.15.190:0/2948501110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4008139dd0 msgr2=0x7f4008140110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4008139dd0 0x7f4008140110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800e930 tx=0x7f3ff800edf0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 -- 172.21.15.190:0/2948501110 shutdown_connections 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f400813a1b0 0x7f400809f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4008139dd0 0x7f4008140110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3fe803be00 0x7f3fe803e2c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 --2- 172.21.15.190:0/2948501110 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40081399f0 0x7f400813fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 -- 172.21.15.190:0/2948501110 >> 172.21.15.190:0/2948501110 conn(0x7f4008074c10 msgr2=0x7f4008135050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:12.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 -- 172.21.15.190:0/2948501110 shutdown_connections 2024-08-19T10:41:12.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.718+0000 7f400d1b4640 1 -- 172.21.15.190:0/2948501110 wait complete. 2024-08-19T10:41:12.735 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:12.735 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -q 2 2024-08-19T10:41:12.735 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -c host2 2024-08-19T10:41:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.838+0000 7f62b1788640 10 monclient: get_monmap_and_config 2024-08-19T10:41:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.838+0000 7f62b1788640 10 monclient: build_initial_monmap 2024-08-19T10:41:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.838+0000 7f62b1788640 10 monclient: monmap: 2024-08-19T10:41:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:12.843505+0000 2024-08-19T10:41:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:12.843505+0000 2024-08-19T10:41:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 1 Processor -- start 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 1 -- start start 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 10 monclient: init 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 10 monclient: _add_conns ranks=[2,4,5,3,10,7,6,0,8,1,9] 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62ac0a36a0 0x7f62ac0a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 10 monclient(hunting): picked mon.noname-b con 0x7f62ac0a36a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62ac0a3fc0 0x7f62ac137470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 10 monclient(hunting): picked mon.noname-c con 0x7f62ac0a3fc0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62ac1379b0 0x7f62ac137e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 10 monclient(hunting): picked mon.noname-h con 0x7f62ac1379b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f62ac09d570 con 0x7f62ac0a36a0 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f62ac09d850 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f62ac09db30 con 0x7f62ac1379b0 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62b1788640 10 monclient(hunting): authenticate will time out at 1118.413316s 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62ac1379b0 0x7f62ac137e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62ac1379b0 0x7f62ac137e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:46992/0 (socket says 172.21.15.190:46992) 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 1 -- 172.21.15.190:0/3245097863 learned_addr learned my addr 172.21.15.190:0/3245097863 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 10 monclient(hunting): get_auth_request con 0x7f62ac1379b0 auth_method 0 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62aa7fc640 1 --2- 172.21.15.190:0/3245097863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62ac0a3fc0 0x7f62ac137470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62aaffd640 1 --2- 172.21.15.190:0/3245097863 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62ac0a36a0 0x7f62ac0a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62aaffd640 10 monclient(hunting): get_auth_request con 0x7f62ac0a36a0 auth_method 0 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62aaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 10 monclient(hunting): handle_auth_done global_id 5162 payload 274 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 1 -- 172.21.15.190:0/3245097863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62ac0a3fc0 msgr2=0x7f62ac137470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 1 --2- 172.21.15.190:0/3245097863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62ac0a3fc0 0x7f62ac137470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 1 -- 172.21.15.190:0/3245097863 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62ac0a36a0 msgr2=0x7f62ac0a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 1 --2- 172.21.15.190:0/3245097863 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62ac0a36a0 0x7f62ac0a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 1 monclient: found mon.noname-h 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62ab7fe640 1 -- 172.21.15.190:0/3245097863 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac0a55a0 con 0x7f62ac1379b0 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62aa7fc640 10 monclient: get_auth_request con 0x7f62ac0a3fc0 auth_method 0 2024-08-19T10:41:12.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.842+0000 7f62aa7fc640 1 --2- 172.21.15.190:0/3245097863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62ac0a3fc0 0x7f62ac137470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62ab7fe640 1 -- 172.21.15.190:0/3245097863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62ac1379b0 msgr2=0x7f62ac137e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62ab7fe640 1 --2- 172.21.15.190:0/3245097863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62ac1379b0 0x7f62ac137e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009e80 tx=0x7f62a401c910 comp rx=0 tx=0).ready entity=mon.5 client_cookie=8f3332ea17dcd412 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 1 -- 172.21.15.190:0/3245097863 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62a40090d0 con 0x7f62ac1379b0 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 10 monclient: dump: 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 1 -- 172.21.15.190:0/3245097863 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a40a5a70 con 0x7f62ac1379b0 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 5 monclient: authenticate success, global_id 5162 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 1 -- 172.21.15.190:0/3245097863 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62a40a5c20 con 0x7f62ac1379b0 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 4 set_mon_vals no callback set 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62a9ffb640 10 monclient: dump: 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 10 monclient: shutdown 2024-08-19T10:41:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 1 -- 172.21.15.190:0/3245097863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62ac1379b0 msgr2=0x7f62ac137e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 1 --2- 172.21.15.190:0/3245097863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62ac1379b0 0x7f62ac137e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009e80 tx=0x7f62a401c910 comp rx=0 tx=0).stop 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 1 -- 172.21.15.190:0/3245097863 shutdown_connections 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 1 --2- 172.21.15.190:0/3245097863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62ac1379b0 0x7f62ac137e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 1 --2- 172.21.15.190:0/3245097863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62ac0a3fc0 0x7f62ac137470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 1 --2- 172.21.15.190:0/3245097863 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62ac0a36a0 0x7f62ac0a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 1 -- 172.21.15.190:0/3245097863 >> 172.21.15.190:0/3245097863 conn(0x7f62ac0a4790 msgr2=0x7f62ac0a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 1 -- 172.21.15.190:0/3245097863 shutdown_connections 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 1 -- 172.21.15.190:0/3245097863 wait complete. 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 10 monclient: build_initial_monmap 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.846+0000 7f62b1788640 10 monclient: monmap: 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:12.861 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:12.862 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 1 Processor -- start 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 1 -- start start 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient: init 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient: _add_conns ranks=[2,1,6,4,5,0,3,8,7] 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62ac0a36a0 0x7f62ac131f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient(hunting): picked mon.noname-b con 0x7f62ac0a36a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62ac0a3fc0 0x7f62ac132470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient(hunting): picked mon.noname-f con 0x7f62ac0a3fc0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f62ac1379b0 0x7f62ac09f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient(hunting): picked mon.noname-d con 0x7f62ac1379b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:12.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac0a4520 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f62ac0a43a0 con 0x7f62ac0a36a0 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f62ac0a1df0 con 0x7f62ac1379b0 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient(hunting): authenticate will time out at 1118.420737s 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62ac0a3fc0 0x7f62ac132470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62ac0a3fc0 0x7f62ac132470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39178/0 (socket says 172.21.15.190:39178) 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 1 -- 172.21.15.190:0/4278751579 learned_addr learned my addr 172.21.15.190:0/4278751579 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 10 monclient(hunting): get_auth_request con 0x7f62ac0a3fc0 auth_method 0 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62ab7fe640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f62ac1379b0 0x7f62ac09f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62ab7fe640 10 monclient(hunting): get_auth_request con 0x7f62ac1379b0 auth_method 0 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62ab7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 10 monclient(hunting): handle_auth_done global_id 14365 payload 824 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 1 -- 172.21.15.190:0/4278751579 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f62ac1379b0 msgr2=0x7f62ac09f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f62ac1379b0 0x7f62ac09f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 1 -- 172.21.15.190:0/4278751579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62ac0a36a0 msgr2=0x7f62ac131f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:12.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62ac0a36a0 0x7f62ac131f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 1 monclient: found mon.noname-f 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 1 -- 172.21.15.190:0/4278751579 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac11fca0 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 1 -- 172.21.15.190:0/4278751579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62ac0a3fc0 msgr2=0x7f62ac132470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62aa7fc640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62ac0a3fc0 0x7f62ac132470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c7f0 tx=0x7f62a000ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 1 -- 172.21.15.190:0/4278751579 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62a0015070 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 10 monclient: dump: 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.866 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.867 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 1 -- 172.21.15.190:0/4278751579 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a00113f0 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 1 -- 172.21.15.190:0/4278751579 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62a0011a10 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 5 monclient: authenticate success, global_id 14365 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62a97fa640 4 set_mon_vals no callback set 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62937fe640 10 monclient: dump: 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:12.868 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:12.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient: _renew_subs 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 1 -- 172.21.15.190:0/4278751579 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac133400 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient: _renew_subs 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.850+0000 7f62b1788640 1 -- 172.21.15.190:0/4278751579 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac0a0a70 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.854+0000 7f62b1788640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.854+0000 7f62b1788640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.854+0000 7f62b1788640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.854+0000 7f62b1788640 1 -- 172.21.15.190:0/4278751579 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ac0008e0 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.854+0000 7f62937fe640 1 -- 172.21.15.190:0/4278751579 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f62a0018070 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.854+0000 7f62937fe640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f629803be00 0x7f629803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.854+0000 7f62937fe640 1 -- 172.21.15.190:0/4278751579 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6297+0+0 (secure 0 0 0) 0x7f62a004ebe0 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.854+0000 7f62aaffd640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f629803be00 0x7f629803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:12.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.854+0000 7f62aaffd640 10 monclient: get_auth_request con 0x7f629803be00 auth_method 0 2024-08-19T10:41:12.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.854+0000 7f62aaffd640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f629803be00 0x7f629803e2c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f629c009870 tx=0x7f629c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:12.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.858+0000 7f62937fe640 1 -- 172.21.15.190:0/4278751579 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f62a00193b0 con 0x7f62ac0a3fc0 2024-08-19T10:41:12.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.858+0000 7f62937fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:12.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:12.858+0000 7f62937fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:13.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.014+0000 7f62b1788640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:13.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.014+0000 7f62b1788640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:13.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.014+0000 7f62b1788640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:13.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.014+0000 7f62b1788640 1 -- 172.21.15.190:0/4278751579 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f62ac1329b0 con 0x7f62ac0a3fc0 2024-08-19T10:41:13.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.202+0000 7f62937fe640 1 -- 172.21.15.190:0/4278751579 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v80) ==== 56+0+1474 (secure 0 0 0) 0x7f62a0011c70 con 0x7f62ac0a3fc0 2024-08-19T10:41:13.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.206+0000 7f62937fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:13.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.206+0000 7f62937fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:13.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 -- 172.21.15.190:0/4278751579 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f62ac0a04f0 con 0x7f62ac0a3fc0 2024-08-19T10:41:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62937fe640 1 -- 172.21.15.190:0/4278751579 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v80) ==== 56+0+1474 (secure 0 0 0) 0x7f62a0007a40 con 0x7f62ac0a3fc0 2024-08-19T10:41:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62937fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:13.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62937fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:13.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 -- 172.21.15.190:0/4278751579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f629803be00 msgr2=0x7f629803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:13.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f629803be00 0x7f629803e2c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f629c009870 tx=0x7f629c008040 comp rx=0 tx=0).stop 2024-08-19T10:41:13.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 10 monclient: shutdown 2024-08-19T10:41:13.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:13.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 -- 172.21.15.190:0/4278751579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62ac0a3fc0 msgr2=0x7f62ac132470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:13.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62ac0a3fc0 0x7f62ac132470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c7f0 tx=0x7f62a000ccb0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 -- 172.21.15.190:0/4278751579 shutdown_connections 2024-08-19T10:41:13.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f629803be00 0x7f629803e2c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f62ac1379b0 0x7f62ac09f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62ac0a3fc0 0x7f62ac132470 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 --2- 172.21.15.190:0/4278751579 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62ac0a36a0 0x7f62ac131f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 -- 172.21.15.190:0/4278751579 >> 172.21.15.190:0/4278751579 conn(0x7f62ac0a4790 msgr2=0x7f62ac072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:13.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.302+0000 7f62b1788640 1 -- 172.21.15.190:0/4278751579 shutdown_connections 2024-08-19T10:41:13.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.306+0000 7f62b1788640 1 -- 172.21.15.190:0/4278751579 wait complete. 2024-08-19T10:41:13.321 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush rm host1 foo 2024-08-19T10:41:13.322 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:41:13.322 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm host1 foo 2024-08-19T10:41:13.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient: get_monmap_and_config 2024-08-19T10:41:13.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient: build_initial_monmap 2024-08-19T10:41:13.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient: monmap: 2024-08-19T10:41:13.439 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:13.439689+0000 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:13.439689+0000 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:13.440 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:13.441 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:13.441 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:13.441 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 1 Processor -- start 2024-08-19T10:41:13.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 1 -- start start 2024-08-19T10:41:13.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient: init 2024-08-19T10:41:13.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:13.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient: _add_conns ranks=[1,10,2,3,7,4,5,0,9,8,6] 2024-08-19T10:41:13.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5d780a5840 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:13.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5d7806e660 0x7f5d7813ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient(hunting): picked mon.noname-f con 0x7f5d7806e660 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:13.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d78072000 0x7f5d78140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient(hunting): picked mon.noname-b con 0x7f5d78072000 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d7806f370 con 0x7f5d780a5840 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f5d78073a70 con 0x7f5d7806e660 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f5d7806f090 con 0x7f5d78072000 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7f1c5640 10 monclient(hunting): authenticate will time out at 1119.009703s 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d77fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5d7806e660 0x7f5d7813ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d77fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5d7806e660 0x7f5d7813ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39192/0 (socket says 172.21.15.190:39192) 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d77fff640 1 -- 172.21.15.190:0/1839342846 learned_addr learned my addr 172.21.15.190:0/1839342846 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d77fff640 10 monclient(hunting): get_auth_request con 0x7f5d7806e660 auth_method 0 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:13.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d77fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d77fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 1 --2- 172.21.15.190:0/1839342846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d78072000 0x7f5d78140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 10 monclient(hunting): get_auth_request con 0x7f5d78072000 auth_method 0 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 10 monclient(hunting): handle_auth_done global_id 5159 payload 274 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 1 -- 172.21.15.190:0/1839342846 >> v2:172.21.15.190:3300/0 conn(0x7f5d7806e660 msgr2=0x7f5d7813ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 1 --2- 172.21.15.190:0/1839342846 >> v2:172.21.15.190:3300/0 conn(0x7f5d7806e660 0x7f5d7813ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 1 -- 172.21.15.190:0/1839342846 >> v1:172.21.15.190:6789/0 conn(0x7f5d780a5840 legacy=0x7f5d780a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:13.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 10 monclient: _finish_hunting 0 2024-08-19T10:41:13.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 1 monclient: found mon.noname-b 2024-08-19T10:41:13.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:13.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:13.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.438+0000 7f5d7d762640 1 -- 172.21.15.190:0/1839342846 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7806e310 con 0x7f5d78072000 2024-08-19T10:41:13.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d7d762640 1 -- 172.21.15.190:0/1839342846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d78072000 msgr2=0x7f5d78140fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:13.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d7d762640 1 --2- 172.21.15.190:0/1839342846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d78072000 0x7f5d78140fb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009d90 tx=0x7f5d7001c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b13ca21d42820d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:13.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 1 -- 172.21.15.190:0/1839342846 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d700ac070 con 0x7f5d78072000 2024-08-19T10:41:13.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:13.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:13.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 10 monclient: dump: 2024-08-19T10:41:13.448 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:13.448 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:13.448 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.448 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:13.449 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d7f1c5640 5 monclient: authenticate success, global_id 5159 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d7f1c5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 1 -- 172.21.15.190:0/1839342846 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d700a6440 con 0x7f5d78072000 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d7f1c5640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d7f1c5640 4 set_mon_vals no callback set 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 1 -- 172.21.15.190:0/1839342846 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d700a6ae0 con 0x7f5d78072000 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:13.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:13.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.442+0000 7f5d777fe640 10 monclient: dump: 2024-08-19T10:41:13.451 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:13.451 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:13.451 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.451 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.451 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:13.451 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.451 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:13.451 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:13.451 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient: shutdown 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 -- 172.21.15.190:0/1839342846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d78072000 msgr2=0x7f5d78140fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 --2- 172.21.15.190:0/1839342846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d78072000 0x7f5d78140fb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009d90 tx=0x7f5d7001c6e0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 -- 172.21.15.190:0/1839342846 shutdown_connections 2024-08-19T10:41:13.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 --2- 172.21.15.190:0/1839342846 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d78072000 0x7f5d78140fb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 --2- 172.21.15.190:0/1839342846 >> v2:172.21.15.190:3300/0 conn(0x7f5d7806e660 0x7f5d7813ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 -- 172.21.15.190:0/1839342846 >> 172.21.15.190:0/1839342846 conn(0x7f5d780aa430 msgr2=0x7f5d780a6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 -- 172.21.15.190:0/1839342846 shutdown_connections 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 -- 172.21.15.190:0/1839342846 wait complete. 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient: build_initial_monmap 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient: monmap: 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:13.453 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:13.454 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:13.454 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:13.454 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:13.454 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:13.454 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:13.454 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:13.454 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:13.454 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 Processor -- start 2024-08-19T10:41:13.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 -- start start 2024-08-19T10:41:13.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient: init 2024-08-19T10:41:13.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:13.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient: _add_conns ranks=[0,4,3,6,5,1,2,7,8] 2024-08-19T10:41:13.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d7806e660 0x7f5d78128b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient(hunting): picked mon.noname-a con 0x7f5d7806e660 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d78072000 0x7f5d7813f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient(hunting): picked mon.noname-c con 0x7f5d78072000 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d780a5840 0x7f5d781264f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient(hunting): picked mon.noname-g con 0x7f5d780a5840 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5d78143e90 con 0x7f5d7806e660 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5d78143d10 con 0x7f5d780a5840 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f5d78144010 con 0x7f5d78072000 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7f1c5640 10 monclient(hunting): authenticate will time out at 1119.018224s 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7d762640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d780a5840 msgr2=0x7f5d781264f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7d762640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d780a5840 0x7f5d781264f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d7806e660 0x7f5d78128b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d77fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d78072000 0x7f5d7813f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d7806e660 0x7f5d78128b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:44116/0 (socket says 172.21.15.190:44116) 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 1 -- 172.21.15.190:0/3102855220 learned_addr learned my addr 172.21.15.190:0/3102855220 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 10 monclient(hunting): get_auth_request con 0x7f5d7806e660 auth_method 0 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d77fff640 10 monclient(hunting): get_auth_request con 0x7f5d78072000 auth_method 0 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d77fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d77fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 10 monclient(hunting): handle_auth_done global_id 5265 payload 824 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 1 -- 172.21.15.190:0/3102855220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d78072000 msgr2=0x7f5d7813f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 1 --2- 172.21.15.190:0/3102855220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d78072000 0x7f5d7813f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 1 -- 172.21.15.190:0/3102855220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d780a5840 msgr2=0x7f5d781264f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 1 --2- 172.21.15.190:0/3102855220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d780a5840 0x7f5d781264f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 10 monclient: _finish_hunting 0 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 1 monclient: found mon.noname-a 2024-08-19T10:41:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d7cf61640 1 -- 172.21.15.190:0/3102855220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d780730f0 con 0x7f5d7806e660 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.446+0000 7f5d77fff640 1 --2- 172.21.15.190:0/3102855220 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d78072000 0x7f5d7813f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7cf61640 1 -- 172.21.15.190:0/3102855220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d7806e660 msgr2=0x7f5d78128b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7cf61640 1 --2- 172.21.15.190:0/3102855220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d7806e660 0x7f5d78128b10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000b410 tx=0x7f5d6000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 1 -- 172.21.15.190:0/3102855220 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d60015030 con 0x7f5d7806e660 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 10 monclient: dump: 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:13.462 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:13.463 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 10 monclient: _finish_auth 0 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 10 monclient: _check_auth_tickets 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 1 -- 172.21.15.190:0/3102855220 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d60011cb0 con 0x7f5d7806e660 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7f1c5640 5 monclient: authenticate success, global_id 5265 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7f1c5640 10 monclient: _renew_subs 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7f1c5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7f1c5640 1 -- 172.21.15.190:0/3102855220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d78129d50 con 0x7f5d7806e660 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 1 -- 172.21.15.190:0/3102855220 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d600107f0 con 0x7f5d7806e660 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 10 monclient: dump: 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d76ffd640 4 set_mon_vals no callback set 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7f1c5640 10 monclient: _renew_subs 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7f1c5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7f1c5640 1 -- 172.21.15.190:0/3102855220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d781c8f30 con 0x7f5d7806e660 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7f1c5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7f1c5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7f1c5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d7f1c5640 1 -- 172.21.15.190:0/3102855220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d781d1db0 con 0x7f5d7806e660 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 1 -- 172.21.15.190:0/3102855220 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5d60018070 con 0x7f5d7806e660 2024-08-19T10:41:13.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 1 --2- 172.21.15.190:0/3102855220 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d5803be00 0x7f5d5803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.450+0000 7f5d757fa640 1 -- 172.21.15.190:0/3102855220 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6297+0+0 (secure 0 0 0) 0x7f5d60016070 con 0x7f5d7806e660 2024-08-19T10:41:13.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.454+0000 7f5d77fff640 1 --2- 172.21.15.190:0/3102855220 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d5803be00 0x7f5d5803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:13.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.454+0000 7f5d77fff640 10 monclient: get_auth_request con 0x7f5d5803be00 auth_method 0 2024-08-19T10:41:13.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.454+0000 7f5d77fff640 1 --2- 172.21.15.190:0/3102855220 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d5803be00 0x7f5d5803e2c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009d40 tx=0x7f5d68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:13.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.466+0000 7f5d757fa640 1 -- 172.21.15.190:0/3102855220 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f5d6001f020 con 0x7f5d7806e660 2024-08-19T10:41:13.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.466+0000 7f5d757fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:13.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.466+0000 7f5d757fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.638+0000 7f5d7f1c5640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.638+0000 7f5d7f1c5640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.638+0000 7f5d7f1c5640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.638+0000 7f5d7f1c5640 1 -- 172.21.15.190:0/3102855220 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"} v 0) -- 0x7f5d780a5c20 con 0x7f5d7806e660 2024-08-19T10:41:13.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.638+0000 7f5d757fa640 1 -- 172.21.15.190:0/3102855220 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}]=-39 (39) Directory not empty v80) ==== 120+0+0 (secure 0 0 0) 0x7f5d780730f0 con 0x7f5d7806e660 2024-08-19T10:41:13.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.638+0000 7f5d757fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:13.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.638+0000 7f5d757fa640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-08-19T10:41:13.641 INFO:tasks.workunit.client.0.smithi190.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-08-19T10:41:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:41:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush unlink host1 foo 2024-08-19T10:41:13.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.762+0000 7fc9bff46640 10 monclient: get_monmap_and_config 2024-08-19T10:41:13.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.762+0000 7fc9bff46640 10 monclient: build_initial_monmap 2024-08-19T10:41:13.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.762+0000 7fc9bff46640 10 monclient: monmap: 2024-08-19T10:41:13.766 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:13.766 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:13.766 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:13.766088+0000 2024-08-19T10:41:13.766 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:13.766088+0000 2024-08-19T10:41:13.766 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:13.766 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.766 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:13.766 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:13.767 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:13.767 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:13.767 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:13.767 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:13.767 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:13.767 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:13.767 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:13.767 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:13.767 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:13.767 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.762+0000 7fc9bff46640 1 Processor -- start 2024-08-19T10:41:13.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.762+0000 7fc9bff46640 1 -- start start 2024-08-19T10:41:13.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.762+0000 7fc9bff46640 10 monclient: init 2024-08-19T10:41:13.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.762+0000 7fc9bff46640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:13.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 10 monclient: _add_conns ranks=[2,5,8,7,9,3,0,1,10,4,6] 2024-08-19T10:41:13.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9b8139980 0x7fc9b8139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 10 monclient(hunting): picked mon.noname-b con 0x7fc9b8139980 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:13.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9b813a2a0 0x7fc9b80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 10 monclient(hunting): picked mon.noname-h con 0x7fc9b813a2a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:13.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9b80a5c70 0x7fc9b80a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 10 monclient(hunting): picked mon.noname-e con 0x7fc9b80a5c70 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:13.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:13.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b809ca60 con 0x7fc9b8139980 2024-08-19T10:41:13.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:13.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc9b80a3640 con 0x7fc9b813a2a0 2024-08-19T10:41:13.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:13.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc9b80a8470 con 0x7fc9b80a5c70 2024-08-19T10:41:13.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:13.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bff46640 10 monclient(hunting): authenticate will time out at 1119.336160s 2024-08-19T10:41:13.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9b813a2a0 0x7fc9b80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:13.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9b813a2a0 0x7fc9b80a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:47002/0 (socket says 172.21.15.190:47002) 2024-08-19T10:41:13.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 1 -- 172.21.15.190:0/652330654 learned_addr learned my addr 172.21.15.190:0/652330654 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:13.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 10 monclient(hunting): get_auth_request con 0x7fc9b813a2a0 auth_method 0 2024-08-19T10:41:13.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:13.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:13.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:13.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:13.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:13.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:13.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bdce2640 1 --2- 172.21.15.190:0/652330654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9b8139980 0x7fc9b8139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:13.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9be4e3640 1 --2- 172.21.15.190:0/652330654 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9b80a5c70 0x7fc9b80a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:13.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bdce2640 10 monclient(hunting): get_auth_request con 0x7fc9b8139980 auth_method 0 2024-08-19T10:41:13.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bdce2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:13.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bdce2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:13.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bdce2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:13.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 10 monclient(hunting): handle_auth_done global_id 5171 payload 274 2024-08-19T10:41:13.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 1 -- 172.21.15.190:0/652330654 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9b80a5c70 msgr2=0x7fc9b80a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:13.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 1 --2- 172.21.15.190:0/652330654 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9b80a5c70 0x7fc9b80a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 1 -- 172.21.15.190:0/652330654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9b8139980 msgr2=0x7fc9b8139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:13.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 1 --2- 172.21.15.190:0/652330654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9b8139980 0x7fc9b8139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 10 monclient: _finish_hunting 0 2024-08-19T10:41:13.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 1 monclient: found mon.noname-h 2024-08-19T10:41:13.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:13.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:13.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9bd4e1640 1 -- 172.21.15.190:0/652330654 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b80a3850 con 0x7fc9b813a2a0 2024-08-19T10:41:13.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9be4e3640 10 monclient: get_auth_request con 0x7fc9b80a5c70 auth_method 0 2024-08-19T10:41:13.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.766+0000 7fc9be4e3640 1 --2- 172.21.15.190:0/652330654 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9b80a5c70 0x7fc9b80a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:13.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 1 -- 172.21.15.190:0/652330654 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9b813a2a0 msgr2=0x7fc9b80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:13.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 1 --2- 172.21.15.190:0/652330654 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9b813a2a0 0x7fc9b80a5730 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80098d0 tx=0x7fc9a801c150 comp rx=0 tx=0).ready entity=mon.5 client_cookie=60452f8bbb55cbd1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:13.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 1 -- 172.21.15.190:0/652330654 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9a80b0070 con 0x7fc9b813a2a0 2024-08-19T10:41:13.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:13.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:13.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 10 monclient: dump: 2024-08-19T10:41:13.781 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:13.781 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:13.781 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.781 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.781 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:13.782 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.782 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:13.782 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:13.782 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:13.782 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:13.782 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:13.782 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:13.782 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:13.782 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:13.782 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:13.782 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 10 monclient: _finish_auth 0 2024-08-19T10:41:13.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 10 monclient: _check_auth_tickets 2024-08-19T10:41:13.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:13.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 1 -- 172.21.15.190:0/652330654 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a80aa810 con 0x7fc9b813a2a0 2024-08-19T10:41:13.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:13.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 1 -- 172.21.15.190:0/652330654 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9a80aae30 con 0x7fc9b813a2a0 2024-08-19T10:41:13.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 5 monclient: authenticate success, global_id 5171 2024-08-19T10:41:13.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:13.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 4 set_mon_vals no callback set 2024-08-19T10:41:13.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:13.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:13.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bcce0640 10 monclient: dump: 2024-08-19T10:41:13.784 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:13.784 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:13.784 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.784 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.784 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:13.784 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.785 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:13.785 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:13.785 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:13.785 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:13.785 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:13.785 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:13.785 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:13.785 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:13.785 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:13.785 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient: shutdown 2024-08-19T10:41:13.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:13.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 -- 172.21.15.190:0/652330654 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9b813a2a0 msgr2=0x7fc9b80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:13.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 --2- 172.21.15.190:0/652330654 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9b813a2a0 0x7fc9b80a5730 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80098d0 tx=0x7fc9a801c150 comp rx=0 tx=0).stop 2024-08-19T10:41:13.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 -- 172.21.15.190:0/652330654 shutdown_connections 2024-08-19T10:41:13.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 --2- 172.21.15.190:0/652330654 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9b80a5c70 0x7fc9b80a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 --2- 172.21.15.190:0/652330654 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9b813a2a0 0x7fc9b80a5730 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 --2- 172.21.15.190:0/652330654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9b8139980 0x7fc9b8139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 -- 172.21.15.190:0/652330654 >> 172.21.15.190:0/652330654 conn(0x7fc9b80abde0 msgr2=0x7fc9b80ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:13.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 -- 172.21.15.190:0/652330654 shutdown_connections 2024-08-19T10:41:13.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 -- 172.21.15.190:0/652330654 wait complete. 2024-08-19T10:41:13.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient: build_initial_monmap 2024-08-19T10:41:13.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient: monmap: 2024-08-19T10:41:13.787 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:13.787 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:13.787 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:13.787 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:13.787 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:13.787 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.787 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:13.787 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:13.787 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:13.788 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:13.788 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:13.788 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:13.788 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:13.788 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:13.788 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:13.788 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 Processor -- start 2024-08-19T10:41:13.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 -- start start 2024-08-19T10:41:13.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient: init 2024-08-19T10:41:13.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:13.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient: _add_conns ranks=[3,1,8,2,7,6,4,5,0] 2024-08-19T10:41:13.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9b80a5c70 0x7fc9b81317e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient(hunting): picked mon.noname-g con 0x7fc9b80a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:13.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b8139980 0x7fc9b8131d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient(hunting): picked mon.noname-f con 0x7fc9b8139980 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:13.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9b809ef50 0x7fc9b809f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bdce2640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9b80a5c70 msgr2=0x7fc9b81317e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:13.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient(hunting): picked mon.noname-e con 0x7fc9b809ef50 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:13.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:13.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8143270 con 0x7fc9b8139980 2024-08-19T10:41:13.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:13.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bdce2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9b80a5c70 0x7fc9b81317e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:13.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b81430f0 con 0x7fc9b80a5c70 2024-08-19T10:41:13.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:13.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc9b81433f0 con 0x7fc9b809ef50 2024-08-19T10:41:13.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:13.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bff46640 10 monclient(hunting): authenticate will time out at 1119.343446s 2024-08-19T10:41:13.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b8139980 0x7fc9b8131d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:13.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b8139980 0x7fc9b8131d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39206/0 (socket says 172.21.15.190:39206) 2024-08-19T10:41:13.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 1 -- 172.21.15.190:0/3086607289 learned_addr learned my addr 172.21.15.190:0/3086607289 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:13.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 10 monclient(hunting): get_auth_request con 0x7fc9b8139980 auth_method 0 2024-08-19T10:41:13.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:13.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:13.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:13.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:13.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:13.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:13.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 10 monclient(hunting): handle_auth_done global_id 14383 payload 824 2024-08-19T10:41:13.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 1 -- 172.21.15.190:0/3086607289 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9b809ef50 msgr2=0x7fc9b809f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:13.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9be4e3640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9b809ef50 0x7fc9b809f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:13.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9b809ef50 0x7fc9b809f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 1 -- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9b80a5c70 msgr2=0x7fc9b81317e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9b80a5c70 0x7fc9b81317e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 10 monclient: _finish_hunting 0 2024-08-19T10:41:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 1 monclient: found mon.noname-f 2024-08-19T10:41:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.770+0000 7fc9bd4e1640 1 -- 172.21.15.190:0/3086607289 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b8135810 con 0x7fc9b8139980 2024-08-19T10:41:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bd4e1640 1 -- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b8139980 msgr2=0x7fc9b8131d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bd4e1640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b8139980 0x7fc9b8131d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80097c0 tx=0x7fc9a8018ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 1 -- 172.21.15.190:0/3086607289 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9a80b9020 con 0x7fc9b8139980 2024-08-19T10:41:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 10 monclient: dump: 2024-08-19T10:41:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:13.794 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:13.794 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:13.794 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:13.794 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:13.794 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:13.794 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:13.794 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:13.794 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:13.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:13.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:13.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 1 -- 172.21.15.190:0/3086607289 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a8004250 con 0x7fc9b8139980 2024-08-19T10:41:13.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bff46640 5 monclient: authenticate success, global_id 14383 2024-08-19T10:41:13.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bff46640 10 monclient: _renew_subs 2024-08-19T10:41:13.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bff46640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:13.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bff46640 1 -- 172.21.15.190:0/3086607289 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b813ae80 con 0x7fc9b8139980 2024-08-19T10:41:13.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:13.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 1 -- 172.21.15.190:0/3086607289 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9a80b0040 con 0x7fc9b8139980 2024-08-19T10:41:13.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:13.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:13.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9affff640 4 set_mon_vals no callback set 2024-08-19T10:41:13.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 10 monclient: dump: 2024-08-19T10:41:13.796 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:13.796 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:13.796 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.796 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:13.796 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:13.796 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:13.796 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:13.796 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:13.796 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:13.796 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:13.797 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:13.797 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:13.797 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:13.797 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:13.797 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:13.797 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:13.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bff46640 10 monclient: _renew_subs 2024-08-19T10:41:13.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bff46640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:13.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bff46640 1 -- 172.21.15.190:0/3086607289 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b80a5990 con 0x7fc9b8139980 2024-08-19T10:41:13.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bff46640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:13.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bff46640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:13.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bff46640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:13.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bff46640 1 -- 172.21.15.190:0/3086607289 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b8139d60 con 0x7fc9b8139980 2024-08-19T10:41:13.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 1 -- 172.21.15.190:0/3086607289 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc9a80bc020 con 0x7fc9b8139980 2024-08-19T10:41:13.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc99c03be20 0x7fc99c03e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:13.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9ae7fc640 1 -- 172.21.15.190:0/3086607289 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6297+0+0 (secure 0 0 0) 0x7fc9a80e9110 con 0x7fc9b8139980 2024-08-19T10:41:13.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bdce2640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc99c03be20 0x7fc99c03e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:13.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.774+0000 7fc9bdce2640 10 monclient: get_auth_request con 0x7fc99c03be20 auth_method 0 2024-08-19T10:41:13.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.778+0000 7fc9bdce2640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc99c03be20 0x7fc99c03e2e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4000f00 tx=0x7fc9b4006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:13.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.778+0000 7fc9ae7fc640 1 -- 172.21.15.190:0/3086607289 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc9a80c3030 con 0x7fc9b8139980 2024-08-19T10:41:13.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.778+0000 7fc9ae7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:13.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.778+0000 7fc9ae7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:13.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.958+0000 7fc9bff46640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:13.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.958+0000 7fc9bff46640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:13.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.958+0000 7fc9bff46640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:13.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:13.958+0000 7fc9bff46640 1 -- 172.21.15.190:0/3086607289 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) -- 0x7fc9b80a6a00 con 0x7fc9b8139980 2024-08-19T10:41:14.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.218+0000 7fc9ae7fc640 1 -- 172.21.15.190:0/3086607289 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 removed item id -11 name 'host1' from crush map v81) ==== 147+0+0 (secure 0 0 0) 0x7fc9b8135810 con 0x7fc9b8139980 2024-08-19T10:41:14.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.218+0000 7fc9ae7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:14.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.218+0000 7fc9ae7fc640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-08-19T10:41:14.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.330+0000 7fc9bff46640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:14.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.330+0000 7fc9bff46640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:14.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.330+0000 7fc9bff46640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:14.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.330+0000 7fc9bff46640 1 -- 172.21.15.190:0/3086607289 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) -- 0x7fc9b80008e0 con 0x7fc9b8139980 2024-08-19T10:41:14.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.330+0000 7fc9ae7fc640 1 -- 172.21.15.190:0/3086607289 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v81) ==== 141+0+0 (secure 0 0 0) 0x7fc9a8016150 con 0x7fc9b8139980 2024-08-19T10:41:14.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.330+0000 7fc9ae7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:14.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.330+0000 7fc9ae7fc640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-08-19T10:41:14.333 INFO:tasks.workunit.client.0.smithi190.stderr:item -11 does not appear in that position 2024-08-19T10:41:14.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 -- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc99c03be20 msgr2=0x7fc99c03e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:14.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc99c03be20 0x7fc99c03e2e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4000f00 tx=0x7fc9b4006a30 comp rx=0 tx=0).stop 2024-08-19T10:41:14.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 10 monclient: shutdown 2024-08-19T10:41:14.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:14.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 -- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b8139980 msgr2=0x7fc9b8131d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:14.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b8139980 0x7fc9b8131d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80097c0 tx=0x7fc9a8018ef0 comp rx=0 tx=0).stop 2024-08-19T10:41:14.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 -- 172.21.15.190:0/3086607289 shutdown_connections 2024-08-19T10:41:14.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc99c03be20 0x7fc99c03e2e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:14.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9b809ef50 0x7fc9b809f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:14.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b8139980 0x7fc9b8131d20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:14.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 --2- 172.21.15.190:0/3086607289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9b80a5c70 0x7fc9b81317e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:14.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 -- 172.21.15.190:0/3086607289 >> 172.21.15.190:0/3086607289 conn(0x7fc9b80abde0 msgr2=0x7fc9b80ab280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:14.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 -- 172.21.15.190:0/3086607289 shutdown_connections 2024-08-19T10:41:14.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.334+0000 7fc9bff46640 1 -- 172.21.15.190:0/3086607289 wait complete. 2024-08-19T10:41:14.351 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush unlink host1 foo 2024-08-19T10:41:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient: get_monmap_and_config 2024-08-19T10:41:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient: build_initial_monmap 2024-08-19T10:41:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient: monmap: 2024-08-19T10:41:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:14.471865+0000 2024-08-19T10:41:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:14.471865+0000 2024-08-19T10:41:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:14.474 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:14.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 1 Processor -- start 2024-08-19T10:41:14.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 1 -- start start 2024-08-19T10:41:14.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient: init 2024-08-19T10:41:14.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:14.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient: _add_conns ranks=[8,1,5,10,3,0,9,2,4,7,6] 2024-08-19T10:41:14.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7b40a6740 0x7fd7b40a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:14.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient(hunting): picked mon.noname-e con 0x7fd7b40a6740 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:14.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd7b40a5bc0 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7b40a7230 0x7fd7b4140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient(hunting): picked mon.noname-h con 0x7fd7b40a7230 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7b40a3740 con 0x7fd7b40a5bc0 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd7b40a4c20 con 0x7fd7b40a7230 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd7b40a3530 con 0x7fd7b40a6740 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7bbd06640 10 monclient(hunting): authenticate will time out at 1120.041688s 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7b92a1640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd7b40a5bc0 0x7fd7b40a6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:59872/0 (socket says 172.21.15.190:59872) 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7b92a1640 1 -- 172.21.15.190:0/2038213433 learned_addr learned my addr 172.21.15.190:0/2038213433 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 1 --2- 172.21.15.190:0/2038213433 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7b40a7230 0x7fd7b4140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:14.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 10 monclient(hunting): get_auth_request con 0x7fd7b40a7230 auth_method 0 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7b9aa2640 1 --2- 172.21.15.190:0/2038213433 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7b40a6740 0x7fd7b40a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7b9aa2640 10 monclient(hunting): get_auth_request con 0x7fd7b40a6740 auth_method 0 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7b9aa2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7b9aa2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7b9aa2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7b9aa2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:14.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7b9aa2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:14.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7b9aa2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:14.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 10 monclient(hunting): handle_auth_done global_id 5180 payload 274 2024-08-19T10:41:14.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 1 -- 172.21.15.190:0/2038213433 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7b40a6740 msgr2=0x7fd7b40a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:14.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 1 --2- 172.21.15.190:0/2038213433 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7b40a6740 0x7fd7b40a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:14.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 1 -- 172.21.15.190:0/2038213433 >> v1:172.21.15.190:6789/0 conn(0x7fd7b40a5bc0 legacy=0x7fd7b40a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:14.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 10 monclient: _finish_hunting 0 2024-08-19T10:41:14.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 1 monclient: found mon.noname-h 2024-08-19T10:41:14.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:14.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:14.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.470+0000 7fd7ba2a3640 1 -- 172.21.15.190:0/2038213433 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b40a3240 con 0x7fd7b40a7230 2024-08-19T10:41:14.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7ba2a3640 1 -- 172.21.15.190:0/2038213433 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7b40a7230 msgr2=0x7fd7b4140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:14.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7ba2a3640 1 --2- 172.21.15.190:0/2038213433 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7b40a7230 0x7fd7b4140ab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009dc0 tx=0x7fd7a401c850 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a5c2035ab1cefee0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:14.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 1 -- 172.21.15.190:0/2038213433 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7a40b2070 con 0x7fd7b40a7230 2024-08-19T10:41:14.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:14.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:14.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 10 monclient: dump: 2024-08-19T10:41:14.485 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:14.485 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:14.485 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:14.486 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 10 monclient: _finish_auth 0 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 10 monclient: _check_auth_tickets 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 5 monclient: authenticate success, global_id 5180 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 1 -- 172.21.15.190:0/2038213433 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a40a5cd0 con 0x7fd7b40a7230 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 4 set_mon_vals no callback set 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 1 -- 172.21.15.190:0/2038213433 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7a40ab7b0 con 0x7fd7b40a7230 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7b8aa0640 10 monclient: dump: 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:14.488 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:14.489 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:14.489 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:14.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 10 monclient: shutdown 2024-08-19T10:41:14.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:14.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 1 -- 172.21.15.190:0/2038213433 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7b40a7230 msgr2=0x7fd7b4140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:14.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 1 --2- 172.21.15.190:0/2038213433 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7b40a7230 0x7fd7b4140ab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009dc0 tx=0x7fd7a401c850 comp rx=0 tx=0).stop 2024-08-19T10:41:14.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 1 -- 172.21.15.190:0/2038213433 shutdown_connections 2024-08-19T10:41:14.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 1 --2- 172.21.15.190:0/2038213433 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd7b40a7230 0x7fd7b4140ab0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:14.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 1 --2- 172.21.15.190:0/2038213433 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7b40a6740 0x7fd7b40a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:14.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 1 -- 172.21.15.190:0/2038213433 >> 172.21.15.190:0/2038213433 conn(0x7fd7b40abd40 msgr2=0x7fd7b40ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:14.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 1 -- 172.21.15.190:0/2038213433 shutdown_connections 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.474+0000 7fd7bbd06640 1 -- 172.21.15.190:0/2038213433 wait complete. 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient: build_initial_monmap 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient: monmap: 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:14.490 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 1 Processor -- start 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 1 -- start start 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient: init 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient: _add_conns ranks=[1,2,0,7,4,8,6,3,5] 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7b40a6740 0x7fd7b4138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:14.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient(hunting): picked mon.noname-f con 0x7fd7b40a6740 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b40a7230 0x7fd7b4139210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient(hunting): picked mon.noname-b con 0x7fd7b40a7230 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7b413e8d0 0x7fd7b413ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient(hunting): picked mon.noname-a con 0x7fd7b413e8d0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b4143990 con 0x7fd7b413e8d0 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b4143810 con 0x7fd7b40a6740 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b4143b10 con 0x7fd7b40a7230 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7bbd06640 10 monclient(hunting): authenticate will time out at 1120.049430s 2024-08-19T10:41:14.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7b40a6740 0x7fd7b4138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:14.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7b40a6740 0x7fd7b4138cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39220/0 (socket says 172.21.15.190:39220) 2024-08-19T10:41:14.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 1 -- 172.21.15.190:0/3124529687 learned_addr learned my addr 172.21.15.190:0/3124529687 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:14.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 10 monclient(hunting): get_auth_request con 0x7fd7b40a6740 auth_method 0 2024-08-19T10:41:14.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:14.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:14.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:14.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:14.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:14.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:14.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 10 monclient(hunting): handle_auth_done global_id 14392 payload 824 2024-08-19T10:41:14.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 1 -- 172.21.15.190:0/3124529687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b40a7230 msgr2=0x7fd7b4139210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:14.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b40a7230 0x7fd7b4139210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:14.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 1 -- 172.21.15.190:0/3124529687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7b413e8d0 msgr2=0x7fd7b413ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:14.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7b413e8d0 0x7fd7b413ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:14.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 10 monclient: _finish_hunting 0 2024-08-19T10:41:14.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 1 monclient: found mon.noname-f 2024-08-19T10:41:14.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:14.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.478+0000 7fd7b9aa2640 1 -- 172.21.15.190:0/3124529687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b4123d20 con 0x7fd7b40a6740 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b9aa2640 1 -- 172.21.15.190:0/3124529687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7b40a6740 msgr2=0x7fd7b4138cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b9aa2640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7b40a6740 0x7fd7b4138cd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00e9b0 tx=0x7fd7ac00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 1 -- 172.21.15.190:0/3124529687 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7ac017070 con 0x7fd7b40a6740 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 10 monclient: dump: 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:14.495 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:14.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 1 -- 172.21.15.190:0/3124529687 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ac013470 con 0x7fd7b40a6740 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7bbd06640 5 monclient: authenticate success, global_id 14392 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7bbd06640 10 monclient: _renew_subs 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7bbd06640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7bbd06640 1 -- 172.21.15.190:0/3124529687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b40a8f20 con 0x7fd7b40a6740 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 1 -- 172.21.15.190:0/3124529687 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7ac013a90 con 0x7fd7b40a6740 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 10 monclient: dump: 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:14.497 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:14.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b3fff640 4 set_mon_vals no callback set 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7bbd06640 10 monclient: _renew_subs 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7bbd06640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7bbd06640 1 -- 172.21.15.190:0/3124529687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b4123a40 con 0x7fd7b40a6740 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7bbd06640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7bbd06640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7bbd06640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7bbd06640 1 -- 172.21.15.190:0/3124529687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b40a5680 con 0x7fd7b40a6740 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 1 -- 172.21.15.190:0/3124529687 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd7ac01a070 con 0x7fd7b40a6740 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7a003be00 0x7fd7a003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b27fc640 1 -- 172.21.15.190:0/3124529687 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6289+0+0 (secure 0 0 0) 0x7fd7ac00f040 con 0x7fd7b40a6740 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b92a1640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7a003be00 0x7fd7a003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:14.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.482+0000 7fd7b92a1640 10 monclient: get_auth_request con 0x7fd7a003be00 auth_method 0 2024-08-19T10:41:14.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.486+0000 7fd7b92a1640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7a003be00 0x7fd7a003e2c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd7b41224b0 tx=0x7fd7a8007ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:14.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.486+0000 7fd7b27fc640 1 -- 172.21.15.190:0/3124529687 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd7ac011350 con 0x7fd7b40a6740 2024-08-19T10:41:14.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.486+0000 7fd7b27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:14.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.486+0000 7fd7b27fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:14.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.662+0000 7fd7bbd06640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:14.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.662+0000 7fd7bbd06640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:14.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.662+0000 7fd7bbd06640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:14.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.662+0000 7fd7bbd06640 1 -- 172.21.15.190:0/3124529687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) -- 0x7fd7b41227f0 con 0x7fd7b40a6740 2024-08-19T10:41:14.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.666+0000 7fd7b27fc640 1 -- 172.21.15.190:0/3124529687 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v81) ==== 141+0+0 (secure 0 0 0) 0x7fd7ac013d20 con 0x7fd7b40a6740 2024-08-19T10:41:14.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.666+0000 7fd7b27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:14.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.666+0000 7fd7b27fc640 10 monclient: _finish_command 2 = system:0 item -11 does not appear in that position 2024-08-19T10:41:14.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.766+0000 7fd7bbd06640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:14.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.766+0000 7fd7bbd06640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:14.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.766+0000 7fd7bbd06640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:14.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:14.766+0000 7fd7bbd06640 1 -- 172.21.15.190:0/3124529687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"} v 0) -- 0x7fd7b40a92f0 con 0x7fd7b40a6740 2024-08-19T10:41:15.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.198+0000 7fd7b27fc640 1 -- 172.21.15.190:0/3124529687 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}]=0 item -11 does not appear in that position v81) ==== 141+0+0 (secure 0 0 0) 0x7fd7ac01be20 con 0x7fd7b40a6740 2024-08-19T10:41:15.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.198+0000 7fd7b27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "foo"}] 2024-08-19T10:41:15.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.198+0000 7fd7b27fc640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-08-19T10:41:15.203 INFO:tasks.workunit.client.0.smithi190.stderr:item -11 does not appear in that position 2024-08-19T10:41:15.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 -- 172.21.15.190:0/3124529687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7a003be00 msgr2=0x7fd7a003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:15.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7a003be00 0x7fd7a003e2c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd7b41224b0 tx=0x7fd7a8007ab0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 10 monclient: shutdown 2024-08-19T10:41:15.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:15.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 -- 172.21.15.190:0/3124529687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7b40a6740 msgr2=0x7fd7b4138cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:15.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7b40a6740 0x7fd7b4138cd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00e9b0 tx=0x7fd7ac00ee70 comp rx=0 tx=0).stop 2024-08-19T10:41:15.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 -- 172.21.15.190:0/3124529687 shutdown_connections 2024-08-19T10:41:15.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7a003be00 0x7fd7a003e2c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7b413e8d0 0x7fd7b413ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd7b40a7230 0x7fd7b4139210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 --2- 172.21.15.190:0/3124529687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7b40a6740 0x7fd7b4138cd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 -- 172.21.15.190:0/3124529687 >> 172.21.15.190:0/3124529687 conn(0x7fd7b40abd40 msgr2=0x7fd7b40ac120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:15.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 -- 172.21.15.190:0/3124529687 shutdown_connections 2024-08-19T10:41:15.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.202+0000 7fd7bbd06640 1 -- 172.21.15.190:0/3124529687 wait complete. 2024-08-19T10:41:15.228 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:15.228 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -c host1 2024-08-19T10:41:15.229 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -q 1 2024-08-19T10:41:15.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.342+0000 7ff58a19b640 10 monclient: get_monmap_and_config 2024-08-19T10:41:15.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.342+0000 7ff58a19b640 10 monclient: build_initial_monmap 2024-08-19T10:41:15.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.342+0000 7ff58a19b640 10 monclient: monmap: 2024-08-19T10:41:15.346 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:15.346 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:15.346 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:15.346042+0000 2024-08-19T10:41:15.346 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:15.346042+0000 2024-08-19T10:41:15.346 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:15.346 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:15.347 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.342+0000 7ff58a19b640 1 Processor -- start 2024-08-19T10:41:15.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.342+0000 7ff58a19b640 1 -- start start 2024-08-19T10:41:15.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.342+0000 7ff58a19b640 10 monclient: init 2024-08-19T10:41:15.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:15.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 10 monclient: _add_conns ranks=[5,10,9,7,4,3,8,1,2,6,0] 2024-08-19T10:41:15.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58409b8d0 0x7ff58409bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 10 monclient(hunting): picked mon.noname-h con 0x7ff58409b8d0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:15.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff58409c7a0 0x7ff58406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 10 monclient(hunting): picked mon.noname-f con 0x7ff58409c7a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:15.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff58406ce10 0x7ff58406d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 10 monclient(hunting): picked mon.noname-a con 0x7ff58406ce10 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff58409d1d0 con 0x7ff58406ce10 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff58409d4b0 con 0x7ff58409c7a0 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff58409d790 con 0x7ff58409b8d0 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff58a19b640 10 monclient(hunting): authenticate will time out at 1120.916525s 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff58409c7a0 0x7ff58406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff5837fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58409b8d0 0x7ff58409bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff5837fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58409b8d0 0x7ff58409bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:47020/0 (socket says 172.21.15.190:47020) 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff58409c7a0 0x7ff58406c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39230/0 (socket says 172.21.15.190:39230) 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff5837fe640 1 -- 172.21.15.190:0/1181953943 learned_addr learned my addr 172.21.15.190:0/1181953943 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:15.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff5837fe640 10 monclient(hunting): get_auth_request con 0x7ff58409b8d0 auth_method 0 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff5837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff5837fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff5837fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 10 monclient(hunting): get_auth_request con 0x7ff58409c7a0 auth_method 0 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff5837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff5837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff5837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:15.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff583fff640 1 --2- 172.21.15.190:0/1181953943 >> v2:172.21.15.89:3300/0 conn(0x7ff58406ce10 0x7ff58406d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 10 monclient(hunting): handle_auth_done global_id 14401 payload 274 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 1 -- 172.21.15.190:0/1181953943 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58409b8d0 msgr2=0x7ff58409bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 1 --2- 172.21.15.190:0/1181953943 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58409b8d0 0x7ff58409bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 1 -- 172.21.15.190:0/1181953943 >> v2:172.21.15.89:3300/0 conn(0x7ff58406ce10 msgr2=0x7ff58406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 1 --2- 172.21.15.190:0/1181953943 >> v2:172.21.15.89:3300/0 conn(0x7ff58406ce10 0x7ff58406d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 1 monclient: found mon.noname-f 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff582ffd640 1 -- 172.21.15.190:0/1181953943 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff584069a80 con 0x7ff58409c7a0 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff583fff640 10 monclient: get_auth_request con 0x7ff58406ce10 auth_method 0 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff583fff640 1 --2- 172.21.15.190:0/1181953943 >> v2:172.21.15.89:3300/0 conn(0x7ff58406ce10 0x7ff58406d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:15.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.346+0000 7ff5837fe640 1 --2- 172.21.15.190:0/1181953943 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58409b8d0 0x7ff58409bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:15.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff582ffd640 1 -- 172.21.15.190:0/1181953943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58409c7a0 msgr2=0x7ff58406c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff582ffd640 1 --2- 172.21.15.190:0/1181953943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58409c7a0 0x7ff58406c8d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0097c0 tx=0x7ff56c01c320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f940319965f3b4ad server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 1 -- 172.21.15.190:0/1181953943 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff56c0b0070 con 0x7ff58409c7a0 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 10 monclient: dump: 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:15.358 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 1 -- 172.21.15.190:0/1181953943 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff56c0aa810 con 0x7ff58409c7a0 2024-08-19T10:41:15.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 5 monclient: authenticate success, global_id 14401 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 1 -- 172.21.15.190:0/1181953943 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff56c0a9300 con 0x7ff58409c7a0 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 4 set_mon_vals no callback set 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5827fc640 10 monclient: dump: 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.360 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient: shutdown 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:15.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 -- 172.21.15.190:0/1181953943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58409c7a0 msgr2=0x7ff58406c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 --2- 172.21.15.190:0/1181953943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58409c7a0 0x7ff58406c8d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0097c0 tx=0x7ff56c01c320 comp rx=0 tx=0).stop 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 -- 172.21.15.190:0/1181953943 shutdown_connections 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 --2- 172.21.15.190:0/1181953943 >> v2:172.21.15.89:3300/0 conn(0x7ff58406ce10 0x7ff58406d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 --2- 172.21.15.190:0/1181953943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58409c7a0 0x7ff58406c8d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 --2- 172.21.15.190:0/1181953943 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58409b8d0 0x7ff58409bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 -- 172.21.15.190:0/1181953943 >> 172.21.15.190:0/1181953943 conn(0x7ff5840a8370 msgr2=0x7ff5840a8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 -- 172.21.15.190:0/1181953943 shutdown_connections 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 -- 172.21.15.190:0/1181953943 wait complete. 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient: build_initial_monmap 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient: monmap: 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:15.362 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 Processor -- start 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 -- start start 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient: init 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient: _add_conns ranks=[5,8,2,6,4,1,3,7,0] 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58406ce10 0x7ff584092400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient(hunting): picked mon.noname-h con 0x7ff58406ce10 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff58409b8d0 0x7ff584092940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient(hunting): picked mon.noname-e con 0x7ff58409b8d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff58409c7a0 0x7ff584092e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient(hunting): picked mon.noname-b con 0x7ff58409c7a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff584142d50 con 0x7ff58409c7a0 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:15.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff584142ed0 con 0x7ff58406ce10 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff584143050 con 0x7ff58409b8d0 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff58a19b640 10 monclient(hunting): authenticate will time out at 1120.923487s 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58406ce10 0x7ff584092400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58406ce10 0x7ff584092400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:47024/0 (socket says 172.21.15.190:47024) 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 1 -- 172.21.15.190:0/2929493505 learned_addr learned my addr 172.21.15.190:0/2929493505 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 10 monclient(hunting): get_auth_request con 0x7ff58406ce10 auth_method 0 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:15.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 10 monclient(hunting): handle_auth_done global_id 5198 payload 824 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 1 -- 172.21.15.190:0/2929493505 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff58409b8d0 msgr2=0x7ff584092940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff58409b8d0 0x7ff584092940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 1 -- 172.21.15.190:0/2929493505 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff58409c7a0 msgr2=0x7ff584092e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff58409c7a0 0x7ff584092e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 1 monclient: found mon.noname-h 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.350+0000 7ff5837fe640 1 -- 172.21.15.190:0/2929493505 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58406c8d0 con 0x7ff58406ce10 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff5837fe640 1 -- 172.21.15.190:0/2929493505 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58406ce10 msgr2=0x7ff584092400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff5837fe640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58406ce10 0x7ff584092400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff57400e9d0 tx=0x7ff57400ee90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:15.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 1 -- 172.21.15.190:0/2929493505 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff574017070 con 0x7ff58406ce10 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 10 monclient: dump: 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:15.367 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 10 monclient: _finish_auth 0 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 1 -- 172.21.15.190:0/2929493505 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57400cc70 con 0x7ff58406ce10 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff58a19b640 5 monclient: authenticate success, global_id 5198 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff58a19b640 10 monclient: _renew_subs 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff58a19b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:15.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff58a19b640 1 -- 172.21.15.190:0/2929493505 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff584094850 con 0x7ff58406ce10 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 1 -- 172.21.15.190:0/2929493505 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff57400ce20 con 0x7ff58406ce10 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 10 monclient: dump: 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:15.369 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:15.370 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:15.370 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:15.370 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:15.370 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:15.370 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:15.370 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:15.370 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff581ffb640 4 set_mon_vals no callback set 2024-08-19T10:41:15.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff58a19b640 10 monclient: _renew_subs 2024-08-19T10:41:15.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff58a19b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:15.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff58a19b640 1 -- 172.21.15.190:0/2929493505 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ff584094e10 con 0x7ff58406ce10 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff58a19b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff58a19b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff58a19b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff58a19b640 1 -- 172.21.15.190:0/2929493505 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58409ad70 con 0x7ff58406ce10 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 1 -- 172.21.15.190:0/2929493505 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff57401a070 con 0x7ff58406ce10 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff56403be00 0x7ff56403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff57bfff640 1 -- 172.21.15.190:0/2929493505 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(81..81 src has 1..81) ==== 6289+0+0 (secure 0 0 0) 0x7ff57400f040 con 0x7ff58406ce10 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff582ffd640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff56403be00 0x7ff56403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.354+0000 7ff582ffd640 10 monclient: get_auth_request con 0x7ff56403be00 auth_method 0 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.358+0000 7ff582ffd640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff56403be00 0x7ff56403e2c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff5840a9410 tx=0x7ff56c004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.358+0000 7ff57bfff640 1 -- 172.21.15.190:0/2929493505 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff58409ad70 con 0x7ff58406ce10 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.358+0000 7ff57bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:15.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.358+0000 7ff57bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:15.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.518+0000 7ff58a19b640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:15.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.518+0000 7ff58a19b640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:15.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.518+0000 7ff58a19b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:15.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.518+0000 7ff58a19b640 1 -- 172.21.15.190:0/2929493505 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7ff58406d2c0 con 0x7ff58406ce10 2024-08-19T10:41:15.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.518+0000 7ff57bfff640 1 -- 172.21.15.190:0/2929493505 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v81) ==== 56+0+1340 (secure 0 0 0) 0x7ff574021070 con 0x7ff58406ce10 2024-08-19T10:41:15.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.518+0000 7ff57bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:15.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.518+0000 7ff57bfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:15.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.614+0000 7ff58a19b640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:15.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.614+0000 7ff58a19b640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:15.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.614+0000 7ff58a19b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:15.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.614+0000 7ff58a19b640 1 -- 172.21.15.190:0/2929493505 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7ff584094290 con 0x7ff58406ce10 2024-08-19T10:41:15.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.614+0000 7ff57bfff640 1 -- 172.21.15.190:0/2929493505 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v81) ==== 56+0+1340 (secure 0 0 0) 0x7ff58406c8d0 con 0x7ff58406ce10 2024-08-19T10:41:15.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.614+0000 7ff57bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:15.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.614+0000 7ff57bfff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:15.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 -- 172.21.15.190:0/2929493505 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff56403be00 msgr2=0x7ff56403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:15.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff56403be00 0x7ff56403e2c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff5840a9410 tx=0x7ff56c004740 comp rx=0 tx=0).stop 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 10 monclient: shutdown 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 -- 172.21.15.190:0/2929493505 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58406ce10 msgr2=0x7ff584092400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58406ce10 0x7ff584092400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff57400e9d0 tx=0x7ff57400ee90 comp rx=0 tx=0).stop 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 -- 172.21.15.190:0/2929493505 shutdown_connections 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff58409c7a0 0x7ff584092e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff56403be00 0x7ff56403e2c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff58409b8d0 0x7ff584092940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 --2- 172.21.15.190:0/2929493505 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff58406ce10 0x7ff584092400 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 -- 172.21.15.190:0/2929493505 >> 172.21.15.190:0/2929493505 conn(0x7ff5840a8370 msgr2=0x7ff5840a06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 -- 172.21.15.190:0/2929493505 shutdown_connections 2024-08-19T10:41:15.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.618+0000 7ff58a19b640 1 -- 172.21.15.190:0/2929493505 wait complete. 2024-08-19T10:41:15.635 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush rm foo 2024-08-19T10:41:15.635 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:41:15.635 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm foo 2024-08-19T10:41:15.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.746+0000 7f546fa0d640 10 monclient: get_monmap_and_config 2024-08-19T10:41:15.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.746+0000 7f546fa0d640 10 monclient: build_initial_monmap 2024-08-19T10:41:15.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.746+0000 7f546fa0d640 10 monclient: monmap: 2024-08-19T10:41:15.750 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:15.750 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:15.750 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:15.750501+0000 2024-08-19T10:41:15.750 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:15.750501+0000 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:15.751 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.746+0000 7f546fa0d640 1 Processor -- start 2024-08-19T10:41:15.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.746+0000 7f546fa0d640 1 -- start start 2024-08-19T10:41:15.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.746+0000 7f546fa0d640 10 monclient: init 2024-08-19T10:41:15.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 10 monclient: _add_conns ranks=[1,3,6,10,7,4,2,9,0,5,8] 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f54681377c0 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54680703c0 0x7f546813c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 10 monclient(hunting): picked mon.noname-g con 0x7f54680703c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f546809ca60 0x7f546813eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 10 monclient(hunting): picked mon.noname-d con 0x7f546809ca60 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54680a14d0 con 0x7f54681377c0 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f54680743f0 con 0x7f54680703c0 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f5468070840 con 0x7f546809ca60 2024-08-19T10:41:15.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546fa0d640 10 monclient(hunting): authenticate will time out at 1121.320324s 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546cfa8640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54680703c0 msgr2=0x7f546813c620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546d7a9640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f54681377c0 0x7f5468137ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:53328/0 (socket says 172.21.15.190:53328) 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546d7a9640 1 -- 172.21.15.190:0/3114243480 learned_addr learned my addr 172.21.15.190:0/3114243480 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546cfa8640 1 --2- 172.21.15.190:0/3114243480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54680703c0 0x7f546813c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 1 --2- 172.21.15.190:0/3114243480 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f546809ca60 0x7f546813eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 10 monclient(hunting): get_auth_request con 0x7f546809ca60 auth_method 0 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:15.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 10 monclient(hunting): handle_auth_done global_id 5154 payload 274 2024-08-19T10:41:15.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 1 -- 172.21.15.190:0/3114243480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54680703c0 msgr2=0x7f546813c620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:15.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 1 --2- 172.21.15.190:0/3114243480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54680703c0 0x7f546813c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 1 -- 172.21.15.190:0/3114243480 >> v1:172.21.15.190:6789/0 conn(0x7f54681377c0 legacy=0x7f5468137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:15.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 10 monclient: _finish_hunting 0 2024-08-19T10:41:15.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 1 monclient: found mon.noname-d 2024-08-19T10:41:15.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:15.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:41:15.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.750+0000 7f546dfaa640 1 -- 172.21.15.190:0/3114243480 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5468074110 con 0x7f546809ca60 2024-08-19T10:41:15.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546dfaa640 1 -- 172.21.15.190:0/3114243480 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f546809ca60 msgr2=0x7f546813eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:15.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546dfaa640 1 --2- 172.21.15.190:0/3114243480 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f546809ca60 0x7f546813eb70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5460009dc0 tx=0x7f546001c690 comp rx=0 tx=0).ready entity=mon.6 client_cookie=db12adfce2723356 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:15.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 1 -- 172.21.15.190:0/3114243480 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54600aa970 con 0x7f546809ca60 2024-08-19T10:41:15.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:15.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:41:15.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 10 monclient: dump: 2024-08-19T10:41:15.758 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:15.758 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:15.758 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.758 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.758 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:15.758 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 10 monclient: _finish_auth 0 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:15.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 1 -- 172.21.15.190:0/3114243480 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54600af070 con 0x7f546809ca60 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 1 -- 172.21.15.190:0/3114243480 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54600a9380 con 0x7f546809ca60 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f545ffff640 10 monclient: dump: 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:15.760 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 5 monclient: authenticate success, global_id 5154 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 4 set_mon_vals no callback set 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 10 monclient: shutdown 2024-08-19T10:41:15.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:15.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 1 -- 172.21.15.190:0/3114243480 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f546809ca60 msgr2=0x7f546813eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:15.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 1 --2- 172.21.15.190:0/3114243480 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f546809ca60 0x7f546813eb70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5460009dc0 tx=0x7f546001c690 comp rx=0 tx=0).stop 2024-08-19T10:41:15.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 1 -- 172.21.15.190:0/3114243480 shutdown_connections 2024-08-19T10:41:15.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 1 --2- 172.21.15.190:0/3114243480 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f546809ca60 0x7f546813eb70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 1 --2- 172.21.15.190:0/3114243480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54680703c0 0x7f546813c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 1 -- 172.21.15.190:0/3114243480 >> 172.21.15.190:0/3114243480 conn(0x7f54680a4700 msgr2=0x7f54680a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:15.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 1 -- 172.21.15.190:0/3114243480 shutdown_connections 2024-08-19T10:41:15.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 1 -- 172.21.15.190:0/3114243480 wait complete. 2024-08-19T10:41:15.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 10 monclient: build_initial_monmap 2024-08-19T10:41:15.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.754+0000 7f546fa0d640 10 monclient: monmap: 2024-08-19T10:41:15.766 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:15.766 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:15.766 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:15.767 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 1 Processor -- start 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 1 -- start start 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient: init 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient: _add_conns ranks=[7,0,2,5,1,8,6,3,4] 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54680703c0 0x7f5468136680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient(hunting): picked mon.noname-i con 0x7f54680703c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546809ca60 0x7f5468136bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient(hunting): picked mon.noname-a con 0x7f546809ca60 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54681377c0 0x7f5468070d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient(hunting): picked mon.noname-b con 0x7f54681377c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5468141a50 con 0x7f546809ca60 2024-08-19T10:41:15.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f54681418d0 con 0x7f54681377c0 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5468141bd0 con 0x7f54680703c0 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient(hunting): authenticate will time out at 1121.328540s 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54680703c0 0x7f5468136680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54680703c0 0x7f5468136680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44094/0 (socket says 172.21.15.190:44094) 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 1 -- 172.21.15.190:0/3790527015 learned_addr learned my addr 172.21.15.190:0/3790527015 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 10 monclient(hunting): get_auth_request con 0x7f54680703c0 auth_method 0 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:15.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 10 monclient(hunting): handle_auth_done global_id 14371 payload 824 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546cfa8640 1 --2- 172.21.15.190:0/3790527015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546809ca60 0x7f5468136bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 1 -- 172.21.15.190:0/3790527015 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54681377c0 msgr2=0x7f5468070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 1 --2- 172.21.15.190:0/3790527015 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54681377c0 0x7f5468070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 1 -- 172.21.15.190:0/3790527015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546809ca60 msgr2=0x7f5468136bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 1 --2- 172.21.15.190:0/3790527015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f546809ca60 0x7f5468136bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 10 monclient: _finish_hunting 0 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 1 monclient: found mon.noname-i 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:15.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 1 -- 172.21.15.190:0/3790527015 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546813dc70 con 0x7f54680703c0 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 1 -- 172.21.15.190:0/3790527015 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54680703c0 msgr2=0x7f5468136680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546d7a9640 1 --2- 172.21.15.190:0/3790527015 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54680703c0 0x7f5468136680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f546400b3b0 tx=0x7f546400b870 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 1 -- 172.21.15.190:0/3790527015 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5464015020 con 0x7f54680703c0 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 10 monclient: dump: 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:15.771 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 1 -- 172.21.15.190:0/3790527015 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5464011cb0 con 0x7f54680703c0 2024-08-19T10:41:15.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 1 -- 172.21.15.190:0/3790527015 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5464011e60 con 0x7f54680703c0 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 5 monclient: authenticate success, global_id 14371 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient: _renew_subs 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 1 -- 172.21.15.190:0/3790527015 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5468123030 con 0x7f54680703c0 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545f7fe640 4 set_mon_vals no callback set 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f545dffb640 10 monclient: dump: 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.773 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient: _renew_subs 2024-08-19T10:41:15.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:15.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.758+0000 7f546fa0d640 1 -- 172.21.15.190:0/3790527015 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f5468122790 con 0x7f54680703c0 2024-08-19T10:41:15.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.762+0000 7f546fa0d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:15.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.762+0000 7f546fa0d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:15.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.762+0000 7f546fa0d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:15.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.762+0000 7f546fa0d640 1 -- 172.21.15.190:0/3790527015 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5468137ba0 con 0x7f54680703c0 2024-08-19T10:41:15.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.762+0000 7f545dffb640 1 -- 172.21.15.190:0/3790527015 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5464018020 con 0x7f54680703c0 2024-08-19T10:41:15.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.762+0000 7f545dffb640 1 --2- 172.21.15.190:0/3790527015 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f544c03be00 0x7f544c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:15.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.762+0000 7f545dffb640 1 -- 172.21.15.190:0/3790527015 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(81..81 src has 1..81) ==== 6289+0+0 (secure 0 0 0) 0x7f546404f0d0 con 0x7f54680703c0 2024-08-19T10:41:15.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.762+0000 7f546cfa8640 1 --2- 172.21.15.190:0/3790527015 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f544c03be00 0x7f544c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:15.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.762+0000 7f546cfa8640 10 monclient: get_auth_request con 0x7f544c03be00 auth_method 0 2024-08-19T10:41:15.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.762+0000 7f546cfa8640 1 --2- 172.21.15.190:0/3790527015 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f544c03be00 0x7f544c03e2c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f546813d930 tx=0x7f5458008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:15.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.766+0000 7f545dffb640 1 -- 172.21.15.190:0/3790527015 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f546401f020 con 0x7f54680703c0 2024-08-19T10:41:15.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.766+0000 7f545dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:15.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.766+0000 7f545dffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:15.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.922+0000 7f546fa0d640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:15.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.922+0000 7f546fa0d640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:15.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.922+0000 7f546fa0d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:15.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.922+0000 7f546fa0d640 1 -- 172.21.15.190:0/3790527015 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7f5468072cc0 con 0x7f54680703c0 2024-08-19T10:41:15.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.926+0000 7f545dffb640 1 -- 172.21.15.190:0/3790527015 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=-39 (39) Directory not empty v81) ==== 99+0+0 (secure 0 0 0) 0x7f5464022350 con 0x7f54680703c0 2024-08-19T10:41:15.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.926+0000 7f545dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:15.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:15.926+0000 7f545dffb640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-08-19T10:41:15.929 INFO:tasks.workunit.client.0.smithi190.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-08-19T10:41:15.934 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:41:15.934 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush rm bar 2024-08-19T10:41:15.934 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:41:15.934 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm bar 2024-08-19T10:41:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.046+0000 7ff6e17a1640 10 monclient: get_monmap_and_config 2024-08-19T10:41:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.046+0000 7ff6e17a1640 10 monclient: build_initial_monmap 2024-08-19T10:41:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.046+0000 7ff6e17a1640 10 monclient: monmap: 2024-08-19T10:41:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:16.050470+0000 2024-08-19T10:41:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:16.050470+0000 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:16.052 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.046+0000 7ff6e17a1640 1 Processor -- start 2024-08-19T10:41:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.046+0000 7ff6e17a1640 1 -- start start 2024-08-19T10:41:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.046+0000 7ff6e17a1640 10 monclient: init 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 10 monclient: _add_conns ranks=[8,9,2,4,3,5,7,6,10,0,1] 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6dc0a5eb0 0x7ff6dc0a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 10 monclient(hunting): picked mon.noname-e con 0x7ff6dc0a5eb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff6dc0a4bf0 0x7ff6dc0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 10 monclient(hunting): picked mon.noname-a con 0x7ff6dc0a4bf0 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6dc139860 0x7ff6dc139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 10 monclient(hunting): picked mon.noname-b con 0x7ff6dc139860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff6dc06cad0 con 0x7ff6dc0a4bf0 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff6dc0a5510 con 0x7ff6dc139860 2024-08-19T10:41:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff6dc0a6290 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6e17a1640 10 monclient(hunting): authenticate will time out at 1121.620299s 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6dc139860 0x7ff6dc139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6daffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6dc0a5eb0 0x7ff6dc0a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6d27fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff6dc0a4bf0 0x7ff6dc0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6dc139860 0x7ff6dc139ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:39862/0 (socket says 172.21.15.190:39862) 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6daffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6dc0a5eb0 0x7ff6dc0a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:34824/0 (socket says 172.21.15.190:34824) 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 1 -- 172.21.15.190:0/4053832670 learned_addr learned my addr 172.21.15.190:0/4053832670 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 10 monclient(hunting): get_auth_request con 0x7ff6dc139860 auth_method 0 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6daffd640 10 monclient(hunting): get_auth_request con 0x7ff6dc0a5eb0 auth_method 0 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6daffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6daffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6d27fc640 10 monclient(hunting): get_auth_request con 0x7ff6dc0a4bf0 auth_method 0 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6d27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6d27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6d27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6d27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6d27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6d27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 10 monclient(hunting): handle_auth_done global_id 5177 payload 274 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 1 -- 172.21.15.190:0/4053832670 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6dc0a5eb0 msgr2=0x7ff6dc0a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 1 --2- 172.21.15.190:0/4053832670 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6dc0a5eb0 0x7ff6dc0a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 1 -- 172.21.15.190:0/4053832670 >> v2:172.21.15.89:3300/0 conn(0x7ff6dc0a4bf0 msgr2=0x7ff6dc0a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 1 --2- 172.21.15.190:0/4053832670 >> v2:172.21.15.89:3300/0 conn(0x7ff6dc0a4bf0 0x7ff6dc0a4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 1 monclient: found mon.noname-b 2024-08-19T10:41:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.050+0000 7ff6db7fe640 1 -- 172.21.15.190:0/4053832670 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc06cd10 con 0x7ff6dc139860 2024-08-19T10:41:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6db7fe640 1 -- 172.21.15.190:0/4053832670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6dc139860 msgr2=0x7ff6dc139ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6db7fe640 1 --2- 172.21.15.190:0/4053832670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6dc139860 0x7ff6dc139ce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009e80 tx=0x7ff6d401c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce07d43d9003cf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 1 -- 172.21.15.190:0/4053832670 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6d40090d0 con 0x7ff6dc139860 2024-08-19T10:41:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 10 monclient: dump: 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 1 -- 172.21.15.190:0/4053832670 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d40a5a70 con 0x7ff6dc139860 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 1 -- 172.21.15.190:0/4053832670 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6d40a5c20 con 0x7ff6dc139860 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6da7fc640 10 monclient: dump: 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 5 monclient: authenticate success, global_id 5177 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 4 set_mon_vals no callback set 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 10 monclient: shutdown 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 1 -- 172.21.15.190:0/4053832670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6dc139860 msgr2=0x7ff6dc139ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 1 --2- 172.21.15.190:0/4053832670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6dc139860 0x7ff6dc139ce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009e80 tx=0x7ff6d401c8d0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 1 -- 172.21.15.190:0/4053832670 shutdown_connections 2024-08-19T10:41:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 1 --2- 172.21.15.190:0/4053832670 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff6dc139860 0x7ff6dc139ce0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 1 --2- 172.21.15.190:0/4053832670 >> v2:172.21.15.89:3300/0 conn(0x7ff6dc0a4bf0 0x7ff6dc0a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 1 --2- 172.21.15.190:0/4053832670 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6dc0a5eb0 0x7ff6dc0a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 1 -- 172.21.15.190:0/4053832670 >> 172.21.15.190:0/4053832670 conn(0x7ff6dc0a92a0 msgr2=0x7ff6dc0a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 1 -- 172.21.15.190:0/4053832670 shutdown_connections 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.054+0000 7ff6e17a1640 1 -- 172.21.15.190:0/4053832670 wait complete. 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient: build_initial_monmap 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient: monmap: 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 1 Processor -- start 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 1 -- start start 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient: init 2024-08-19T10:41:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient: _add_conns ranks=[4,7,3,5,8,1,6,0,2] 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6dc0a4bf0 0x7ff6dc1d1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient(hunting): picked mon.noname-c con 0x7ff6dc0a4bf0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff6dc0a5eb0 0x7ff6dc1d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient(hunting): picked mon.noname-i con 0x7ff6dc0a5eb0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6dc139860 0x7ff6dc1d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient(hunting): picked mon.noname-g con 0x7ff6dc139860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff6dc0a1770 con 0x7ff6dc139860 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff6dc13a630 con 0x7ff6dc0a4bf0 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:16.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff6dc0a18f0 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:16.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6e17a1640 10 monclient(hunting): authenticate will time out at 1121.629078s 2024-08-19T10:41:16.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6db7fe640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6dc139860 msgr2=0x7ff6dc1d1af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:16.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6db7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6dc139860 0x7ff6dc1d1af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:16.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff6dc0a5eb0 0x7ff6dc1d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff6dc0a5eb0 0x7ff6dc1d15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44100/0 (socket says 172.21.15.190:44100) 2024-08-19T10:41:16.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 1 -- 172.21.15.190:0/3163857255 learned_addr learned my addr 172.21.15.190:0/3163857255 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:16.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 10 monclient(hunting): get_auth_request con 0x7ff6dc0a5eb0 auth_method 0 2024-08-19T10:41:16.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6daffd640 1 --2- 172.21.15.190:0/3163857255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6dc0a4bf0 0x7ff6dc1d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 10 monclient(hunting): handle_auth_done global_id 14380 payload 824 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 1 -- 172.21.15.190:0/3163857255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6dc0a4bf0 msgr2=0x7ff6dc1d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 1 --2- 172.21.15.190:0/3163857255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6dc0a4bf0 0x7ff6dc1d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 1 -- 172.21.15.190:0/3163857255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6dc139860 msgr2=0x7ff6dc1d1af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 1 --2- 172.21.15.190:0/3163857255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6dc139860 0x7ff6dc1d1af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 10 monclient: _finish_hunting 0 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 1 monclient: found mon.noname-i 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 1 -- 172.21.15.190:0/3163857255 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc13deb0 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6daffd640 10 monclient: get_auth_request con 0x7ff6dc0a4bf0 auth_method 0 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6daffd640 1 --2- 172.21.15.190:0/3163857255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff6dc0a4bf0 0x7ff6dc1d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 1 -- 172.21.15.190:0/3163857255 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff6dc0a5eb0 msgr2=0x7ff6dc1d15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.058+0000 7ff6d27fc640 1 --2- 172.21.15.190:0/3163857255 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff6dc0a5eb0 0x7ff6dc1d15b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400e870 tx=0x7ff6c400ec70 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 1 -- 172.21.15.190:0/3163857255 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6c4017070 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 10 monclient: dump: 2024-08-19T10:41:16.073 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:16.074 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 1 -- 172.21.15.190:0/3163857255 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6c4013470 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 1 -- 172.21.15.190:0/3163857255 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6c4013a90 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 10 monclient: dump: 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:16.075 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.076 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.076 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:16.076 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.076 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:16.076 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:16.076 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:16.076 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:16.076 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d9ffb640 4 set_mon_vals no callback set 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6e17a1640 5 monclient: authenticate success, global_id 14380 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6e17a1640 10 monclient: _renew_subs 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6e17a1640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6e17a1640 1 -- 172.21.15.190:0/3163857255 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6dc13f3a0 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6e17a1640 10 monclient: _renew_subs 2024-08-19T10:41:16.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6e17a1640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:16.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6e17a1640 1 -- 172.21.15.190:0/3163857255 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6dc124340 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6e17a1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:16.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6e17a1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:16.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6e17a1640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:16.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6e17a1640 1 -- 172.21.15.190:0/3163857255 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6dc06d6a0 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 1 -- 172.21.15.190:0/3163857255 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff6c401a070 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 1 --2- 172.21.15.190:0/3163857255 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff6bc03be00 0x7ff6bc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6d3fff640 1 -- 172.21.15.190:0/3163857255 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(81..81 src has 1..81) ==== 6289+0+0 (secure 0 0 0) 0x7ff6c400f040 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6daffd640 1 --2- 172.21.15.190:0/3163857255 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff6bc03be00 0x7ff6bc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.062+0000 7ff6daffd640 10 monclient: get_auth_request con 0x7ff6bc03be00 auth_method 0 2024-08-19T10:41:16.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.066+0000 7ff6daffd640 1 --2- 172.21.15.190:0/3163857255 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff6bc03be00 0x7ff6bc03e2c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009a60 tx=0x7ff6cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:16.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.066+0000 7ff6d3fff640 1 -- 172.21.15.190:0/3163857255 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff6c4021070 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.066+0000 7ff6d3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:16.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.066+0000 7ff6d3fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:16.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.242+0000 7ff6e17a1640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:16.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.242+0000 7ff6e17a1640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:16.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.242+0000 7ff6e17a1640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:16.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.242+0000 7ff6e17a1640 1 -- 172.21.15.190:0/3163857255 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) -- 0x7ff6dc06ec50 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.250+0000 7ff6d3fff640 1 -- 172.21.15.190:0/3163857255 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=-39 (39) Directory not empty v81) ==== 99+0+0 (secure 0 0 0) 0x7ff6c4013620 con 0x7ff6dc0a5eb0 2024-08-19T10:41:16.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.250+0000 7ff6d3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:16.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.250+0000 7ff6d3fff640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-08-19T10:41:16.253 INFO:tasks.workunit.client.0.smithi190.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-08-19T10:41:16.257 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:41:16.258 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush unlink host1 bar 2024-08-19T10:41:16.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient: get_monmap_and_config 2024-08-19T10:41:16.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient: build_initial_monmap 2024-08-19T10:41:16.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient: monmap: 2024-08-19T10:41:16.372 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:16.372 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:16.372 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:16.372404+0000 2024-08-19T10:41:16.372 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:16.372404+0000 2024-08-19T10:41:16.372 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:16.373 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 1 Processor -- start 2024-08-19T10:41:16.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 1 -- start start 2024-08-19T10:41:16.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient: init 2024-08-19T10:41:16.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:16.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient: _add_conns ranks=[4,0,5,7,9,2,10,3,6,8,1] 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080a56c0 0x7ffa080a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient(hunting): picked mon.noname-c con 0x7ffa080a56c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffa080722a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a61b0 0x7ffa0813e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient(hunting): picked mon.noname-h con 0x7ffa080a61b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa08072720 con 0x7ffa080722a0 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ffa080737c0 con 0x7ffa080a56c0 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ffa080a71e0 con 0x7ffa080a61b0 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0f91f640 10 monclient(hunting): authenticate will time out at 1121.942373s 2024-08-19T10:41:16.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a61b0 0x7ffa0813e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a61b0 0x7ffa0813e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:39592/0 (socket says 172.21.15.190:39592) 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 1 -- 172.21.15.190:0/3868082814 learned_addr learned my addr 172.21.15.190:0/3868082814 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 10 monclient(hunting): get_auth_request con 0x7ffa080a61b0 auth_method 0 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 10 monclient(hunting): handle_auth_done global_id 5207 payload 274 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 1 -- 172.21.15.190:0/3868082814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080a56c0 msgr2=0x7ffa080a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0d6bb640 1 --2- 172.21.15.190:0/3868082814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080a56c0 0x7ffa080a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 1 --2- 172.21.15.190:0/3868082814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080a56c0 0x7ffa080a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 1 -- 172.21.15.190:0/3868082814 >> v1:172.21.15.89:6789/0 conn(0x7ffa080722a0 legacy=0x7ffa080a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:16.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 10 monclient: _finish_hunting 0 2024-08-19T10:41:16.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 1 monclient: found mon.noname-h 2024-08-19T10:41:16.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:16.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:16.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0debc640 1 -- 172.21.15.190:0/3868082814 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0813fcf0 con 0x7ffa080a61b0 2024-08-19T10:41:16.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0d6bb640 10 monclient: get_auth_request con 0x7ffa080a56c0 auth_method 0 2024-08-19T10:41:16.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.370+0000 7ffa0d6bb640 1 --2- 172.21.15.190:0/3868082814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080a56c0 0x7ffa080a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:16.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ffa0debc640 1 -- 172.21.15.190:0/3868082814 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a61b0 msgr2=0x7ffa0813e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ffa0debc640 1 --2- 172.21.15.190:0/3868082814 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a61b0 0x7ffa0813e7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009e50 tx=0x7ffa0001c910 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a74610a134c7744 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 1 -- 172.21.15.190:0/3868082814 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa000a6970 con 0x7ffa080a61b0 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 10 monclient: dump: 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:16.380 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 10 monclient: _finish_auth 0 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 1 -- 172.21.15.190:0/3868082814 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa000a6b30 con 0x7ffa080a61b0 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:16.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 1 -- 172.21.15.190:0/3868082814 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa000a6ce0 con 0x7ffa080a61b0 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ffa0f91f640 5 monclient: authenticate success, global_id 5207 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ffa0f91f640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ffa0f91f640 4 set_mon_vals no callback set 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.374+0000 7ff9fffff640 10 monclient: dump: 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.382 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient: shutdown 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3868082814 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a61b0 msgr2=0x7ffa0813e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 --2- 172.21.15.190:0/3868082814 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a61b0 0x7ffa0813e7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009e50 tx=0x7ffa0001c910 comp rx=0 tx=0).stop 2024-08-19T10:41:16.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3868082814 shutdown_connections 2024-08-19T10:41:16.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 --2- 172.21.15.190:0/3868082814 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffa080a61b0 0x7ffa0813e7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 --2- 172.21.15.190:0/3868082814 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080a56c0 0x7ffa080a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3868082814 >> 172.21.15.190:0/3868082814 conn(0x7ffa080aa2b0 msgr2=0x7ffa080a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:16.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3868082814 shutdown_connections 2024-08-19T10:41:16.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3868082814 wait complete. 2024-08-19T10:41:16.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient: build_initial_monmap 2024-08-19T10:41:16.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient: monmap: 2024-08-19T10:41:16.388 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:16.388 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:16.388 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:16.388 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:16.388 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:16.388 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.388 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 Processor -- start 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 -- start start 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient: init 2024-08-19T10:41:16.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:16.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient: _add_conns ranks=[4,6,2,3,1,0,7,5,8] 2024-08-19T10:41:16.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080722a0 0x7ffa0809fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient(hunting): picked mon.noname-c con 0x7ffa080722a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:16.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffa080a56c0 0x7ffa081d1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient(hunting): picked mon.noname-d con 0x7ffa080a56c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:16.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a61b0 0x7ffa081d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient(hunting): picked mon.noname-b con 0x7ffa080a61b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:16.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:16.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0813f670 con 0x7ffa080a61b0 2024-08-19T10:41:16.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ffa0813f4f0 con 0x7ffa080722a0 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ffa0813f7f0 con 0x7ffa080a56c0 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0f91f640 10 monclient(hunting): authenticate will time out at 1121.950253s 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080722a0 0x7ffa0809fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0ceba640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffa080a56c0 0x7ffa081d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0debc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a61b0 0x7ffa081d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080722a0 0x7ffa0809fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:40860/0 (socket says 172.21.15.190:40860) 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 1 -- 172.21.15.190:0/3613624028 learned_addr learned my addr 172.21.15.190:0/3613624028 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 10 monclient(hunting): get_auth_request con 0x7ffa080722a0 auth_method 0 2024-08-19T10:41:16.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0ceba640 10 monclient(hunting): get_auth_request con 0x7ffa080a56c0 auth_method 0 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0ceba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0ceba640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0ceba640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0debc640 10 monclient(hunting): get_auth_request con 0x7ffa080a61b0 auth_method 0 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0debc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0debc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0debc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:16.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0debc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0debc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0debc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0ceba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0ceba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0ceba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 10 monclient(hunting): handle_auth_done global_id 5215 payload 824 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 1 -- 172.21.15.190:0/3613624028 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffa080a56c0 msgr2=0x7ffa081d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffa080a56c0 0x7ffa081d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 1 -- 172.21.15.190:0/3613624028 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a61b0 msgr2=0x7ffa081d16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a61b0 0x7ffa081d16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 10 monclient: _finish_hunting 0 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 1 monclient: found mon.noname-c 2024-08-19T10:41:16.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.378+0000 7ffa0d6bb640 1 -- 172.21.15.190:0/3613624028 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa080a0670 con 0x7ffa080722a0 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0d6bb640 1 -- 172.21.15.190:0/3613624028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080722a0 msgr2=0x7ffa0809fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0d6bb640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080722a0 0x7ffa0809fce0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400c7f0 tx=0x7ffa0400ccb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 1 -- 172.21.15.190:0/3613624028 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa04007e30 con 0x7ffa080722a0 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 10 monclient: dump: 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.394 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:16.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 1 -- 172.21.15.190:0/3613624028 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa040113f0 con 0x7ffa080722a0 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 1 -- 172.21.15.190:0/3613624028 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa040115a0 con 0x7ffa080722a0 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9ff7fe640 4 set_mon_vals no callback set 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ff9fdffb640 10 monclient: dump: 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:16.396 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0f91f640 5 monclient: authenticate success, global_id 5215 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0f91f640 10 monclient: _renew_subs 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0f91f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3613624028 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa081c6be0 con 0x7ffa080722a0 2024-08-19T10:41:16.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0f91f640 10 monclient: _renew_subs 2024-08-19T10:41:16.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0f91f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:16.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3613624028 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa08120b30 con 0x7ffa080722a0 2024-08-19T10:41:16.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0f91f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:16.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0f91f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:16.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0f91f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:16.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.382+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3613624028 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa080a1670 con 0x7ffa080722a0 2024-08-19T10:41:16.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.386+0000 7ff9fdffb640 1 -- 172.21.15.190:0/3613624028 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ffa040079c0 con 0x7ffa080722a0 2024-08-19T10:41:16.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.386+0000 7ff9fdffb640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff9ec03be00 0x7ff9ec03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:16.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.386+0000 7ff9fdffb640 1 -- 172.21.15.190:0/3613624028 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(81..81 src has 1..81) ==== 6289+0+0 (secure 0 0 0) 0x7ffa0404f680 con 0x7ffa080722a0 2024-08-19T10:41:16.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.386+0000 7ffa0ceba640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff9ec03be00 0x7ff9ec03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:16.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.386+0000 7ffa0ceba640 10 monclient: get_auth_request con 0x7ff9ec03be00 auth_method 0 2024-08-19T10:41:16.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.386+0000 7ffa0ceba640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff9ec03be00 0x7ff9ec03e2c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffa0809ee80 tx=0x7ff9f8003700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:16.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.398+0000 7ff9fdffb640 1 -- 172.21.15.190:0/3613624028 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ffa081c6be0 con 0x7ffa080722a0 2024-08-19T10:41:16.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.398+0000 7ff9fdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:16.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.398+0000 7ff9fdffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:16.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.574+0000 7ffa0f91f640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-08-19T10:41:16.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.574+0000 7ffa0f91f640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-08-19T10:41:16.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.574+0000 7ffa0f91f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:16.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:16.574+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3613624028 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"} v 0) -- 0x7ffa0806fd10 con 0x7ffa080722a0 2024-08-19T10:41:17.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.266+0000 7ff9fdffb640 1 -- 172.21.15.190:0/3613624028 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}]=0 removed item id -11 name 'host1' from crush map v82) ==== 147+0+0 (secure 0 0 0) 0x7ffa04015020 con 0x7ffa080722a0 2024-08-19T10:41:17.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.266+0000 7ff9fdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-08-19T10:41:17.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.266+0000 7ff9fdffb640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-08-19T10:41:17.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.374+0000 7ffa0f91f640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-08-19T10:41:17.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.374+0000 7ffa0f91f640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-08-19T10:41:17.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.374+0000 7ffa0f91f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:17.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.374+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3613624028 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"} v 0) -- 0x7ffa08134730 con 0x7ffa080722a0 2024-08-19T10:41:17.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.374+0000 7ff9fdffb640 1 -- 172.21.15.190:0/3613624028 <== mon.4 v2:172.21.15.89:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}]=0 item -11 does not appear in that position v82) ==== 141+0+0 (secure 0 0 0) 0x7ffa04020d10 con 0x7ffa080722a0 2024-08-19T10:41:17.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.374+0000 7ff9fdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host1", "ancestor": "bar"}] 2024-08-19T10:41:17.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.374+0000 7ff9fdffb640 10 monclient: _finish_command 3 = system:0 item -11 does not appear in that position 2024-08-19T10:41:17.379 INFO:tasks.workunit.client.0.smithi190.stderr:item -11 does not appear in that position 2024-08-19T10:41:17.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3613624028 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff9ec03be00 msgr2=0x7ff9ec03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:17.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff9ec03be00 0x7ff9ec03e2c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffa0809ee80 tx=0x7ff9f8003700 comp rx=0 tx=0).stop 2024-08-19T10:41:17.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 10 monclient: shutdown 2024-08-19T10:41:17.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:17.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3613624028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080722a0 msgr2=0x7ffa0809fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:17.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080722a0 0x7ffa0809fce0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400c7f0 tx=0x7ffa0400ccb0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3613624028 shutdown_connections 2024-08-19T10:41:17.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff9ec03be00 0x7ff9ec03e2c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa080a61b0 0x7ffa081d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffa080a56c0 0x7ffa081d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 --2- 172.21.15.190:0/3613624028 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ffa080722a0 0x7ffa0809fce0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3613624028 >> 172.21.15.190:0/3613624028 conn(0x7ffa080aa2b0 msgr2=0x7ffa080a83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:17.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3613624028 shutdown_connections 2024-08-19T10:41:17.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.378+0000 7ffa0f91f640 1 -- 172.21.15.190:0/3613624028 wait complete. 2024-08-19T10:41:17.398 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -c host1 2024-08-19T10:41:17.398 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:17.398 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -q 1 2024-08-19T10:41:17.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient: get_monmap_and_config 2024-08-19T10:41:17.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient: build_initial_monmap 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient: monmap: 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:17.513942+0000 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:17.513942+0000 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:17.514 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:17.515 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:17.515 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:17.515 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:17.515 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:17.515 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:17.515 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 1 Processor -- start 2024-08-19T10:41:17.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 1 -- start start 2024-08-19T10:41:17.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient: init 2024-08-19T10:41:17.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:17.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient: _add_conns ranks=[1,0,6,10,3,4,9,2,5,8,7] 2024-08-19T10:41:17.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fda3c1399f0 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:17.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fda3c0a5640 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:17.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c140000 0x7fda3c1423f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient(hunting): picked mon.noname-d con 0x7fda3c140000 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda3c13a4e0 con 0x7fda3c0a5640 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda3c0a8340 con 0x7fda3c1399f0 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fda3c0a4580 con 0x7fda3c140000 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.510+0000 7fda41218640 10 monclient(hunting): authenticate will time out at 1123.083979s 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3ad9d640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fda3c1399f0 0x7fda3c139dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:53330/0 (socket says 172.21.15.190:53330) 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3ad9d640 1 -- 172.21.15.190:0/2734971625 learned_addr learned my addr 172.21.15.190:0/2734971625 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 1 --2- 172.21.15.190:0/2734971625 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c140000 0x7fda3c1423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 10 monclient(hunting): get_auth_request con 0x7fda3c140000 auth_method 0 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:17.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 10 monclient(hunting): handle_auth_done global_id 5172 payload 274 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 1 -- 172.21.15.190:0/2734971625 >> v1:172.21.15.190:6789/0 conn(0x7fda3c1399f0 legacy=0x7fda3c139dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 1 -- 172.21.15.190:0/2734971625 >> v1:172.21.15.89:6789/0 conn(0x7fda3c0a5640 legacy=0x7fda3c13e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 10 monclient: _finish_hunting 0 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 1 monclient: found mon.noname-d 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:41:17.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.514+0000 7fda3b59e640 1 -- 172.21.15.190:0/2734971625 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c0a4860 con 0x7fda3c140000 2024-08-19T10:41:17.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda3b59e640 1 -- 172.21.15.190:0/2734971625 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c140000 msgr2=0x7fda3c1423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:17.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda3b59e640 1 --2- 172.21.15.190:0/2734971625 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c140000 0x7fda3c1423f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009dc0 tx=0x7fda2c01c850 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c6a732a7fb43f8d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:17.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 1 -- 172.21.15.190:0/2734971625 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda2c0b2070 con 0x7fda3c140000 2024-08-19T10:41:17.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:17.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:41:17.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 10 monclient: dump: 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:17.522 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 10 monclient: _finish_auth 0 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 1 -- 172.21.15.190:0/2734971625 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c0a5a60 con 0x7fda3c140000 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 5 monclient: authenticate success, global_id 5172 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 4 set_mon_vals no callback set 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 1 -- 172.21.15.190:0/2734971625 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda2c0a5c10 con 0x7fda3c140000 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:17.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:41:17.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda39d9b640 10 monclient: dump: 2024-08-19T10:41:17.524 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:17.524 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:17.524 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.524 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.524 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:17.524 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.524 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:17.524 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:17.525 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:17.525 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:17.525 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:17.525 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:17.525 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:17.525 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:17.525 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:17.525 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 10 monclient: shutdown 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 1 -- 172.21.15.190:0/2734971625 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c140000 msgr2=0x7fda3c1423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 1 --2- 172.21.15.190:0/2734971625 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c140000 0x7fda3c1423f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009dc0 tx=0x7fda2c01c850 comp rx=0 tx=0).stop 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 1 -- 172.21.15.190:0/2734971625 shutdown_connections 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 1 --2- 172.21.15.190:0/2734971625 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c140000 0x7fda3c1423f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 1 -- 172.21.15.190:0/2734971625 >> 172.21.15.190:0/2734971625 conn(0x7fda3c074c10 msgr2=0x7fda3c0abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 1 -- 172.21.15.190:0/2734971625 shutdown_connections 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 1 -- 172.21.15.190:0/2734971625 wait complete. 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 10 monclient: build_initial_monmap 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.518+0000 7fda41218640 10 monclient: monmap: 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:17.532 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 1 Processor -- start 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 1 -- start start 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 10 monclient: init 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 10 monclient: _add_conns ranks=[6,0,4,8,7,2,5,1,3] 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c0a5640 0x7fda3c1d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 10 monclient(hunting): picked mon.noname-d con 0x7fda3c0a5640 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda3c1399f0 0x7fda3c1d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 10 monclient(hunting): picked mon.noname-a con 0x7fda3c1399f0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda3c140000 0x7fda3c09e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 10 monclient(hunting): picked mon.noname-c con 0x7fda3c140000 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:17.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c145560 con 0x7fda3c1399f0 2024-08-19T10:41:17.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:17.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fda3c0a5a20 con 0x7fda3c140000 2024-08-19T10:41:17.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:17.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fda3c1456e0 con 0x7fda3c0a5640 2024-08-19T10:41:17.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:17.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda41218640 10 monclient(hunting): authenticate will time out at 1123.092673s 2024-08-19T10:41:17.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda3c1399f0 0x7fda3c1d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:17.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda3c1399f0 0x7fda3c1d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:40646/0 (socket says 172.21.15.190:40646) 2024-08-19T10:41:17.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3ad9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c0a5640 0x7fda3c1d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:17.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 1 -- 172.21.15.190:0/1841587520 learned_addr learned my addr 172.21.15.190:0/1841587520 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:17.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 10 monclient(hunting): get_auth_request con 0x7fda3c1399f0 auth_method 0 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3b59e640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda3c140000 0x7fda3c09e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3ad9d640 10 monclient(hunting): get_auth_request con 0x7fda3c0a5640 auth_method 0 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3b59e640 10 monclient(hunting): get_auth_request con 0x7fda3c140000 auth_method 0 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3b59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3b59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:17.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 10 monclient(hunting): handle_auth_done global_id 5283 payload 824 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 1 -- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c0a5640 msgr2=0x7fda3c1d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c0a5640 0x7fda3c1d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 1 -- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda3c140000 msgr2=0x7fda3c09e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda3c140000 0x7fda3c09e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 1 monclient: found mon.noname-a 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3a59c640 1 -- 172.21.15.190:0/1841587520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c0a2010 con 0x7fda3c1399f0 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.522+0000 7fda3ad9d640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c0a5640 0x7fda3c1d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda3a59c640 1 -- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda3c1399f0 msgr2=0x7fda3c1d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda3a59c640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda3c1399f0 0x7fda3c1d1de0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c450 tx=0x7fda2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 1 -- 172.21.15.190:0/1841587520 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda24012cb0 con 0x7fda3c1399f0 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 10 monclient: dump: 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:17.537 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:17.538 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 1 -- 172.21.15.190:0/1841587520 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda24017070 con 0x7fda3c1399f0 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 1 -- 172.21.15.190:0/1841587520 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda240117f0 con 0x7fda3c1399f0 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda3959a640 4 set_mon_vals no callback set 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 10 monclient: dump: 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:17.539 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.540 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.540 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:17.540 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.540 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:17.540 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:17.540 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:17.541 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:17.541 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:17.541 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda41218640 5 monclient: authenticate success, global_id 5283 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda41218640 10 monclient: _renew_subs 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda41218640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda41218640 1 -- 172.21.15.190:0/1841587520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c0a0b50 con 0x7fda3c1399f0 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda41218640 10 monclient: _renew_subs 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda41218640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda41218640 1 -- 172.21.15.190:0/1841587520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c13fdc0 con 0x7fda3c1399f0 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda41218640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:17.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda41218640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:17.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda41218640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:17.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda41218640 1 -- 172.21.15.190:0/1841587520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c13f820 con 0x7fda3c1399f0 2024-08-19T10:41:17.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 1 -- 172.21.15.190:0/1841587520 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fda24011380 con 0x7fda3c1399f0 2024-08-19T10:41:17.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda1c03be00 0x7fda1c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:17.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda337fe640 1 -- 172.21.15.190:0/1841587520 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6249+0+0 (secure 0 0 0) 0x7fda2404f810 con 0x7fda3c1399f0 2024-08-19T10:41:17.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda3ad9d640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda1c03be00 0x7fda1c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:17.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.526+0000 7fda3ad9d640 10 monclient: get_auth_request con 0x7fda1c03be00 auth_method 0 2024-08-19T10:41:17.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.530+0000 7fda3ad9d640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda1c03be00 0x7fda1c03e2c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fda34007ca0 tx=0x7fda34007bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:17.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.538+0000 7fda337fe640 1 -- 172.21.15.190:0/1841587520 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fda2401ba20 con 0x7fda3c1399f0 2024-08-19T10:41:17.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.538+0000 7fda337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:17.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.538+0000 7fda337fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:17.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.706+0000 7fda41218640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:17.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.706+0000 7fda41218640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:17.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.706+0000 7fda41218640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:17.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.710+0000 7fda41218640 1 -- 172.21.15.190:0/1841587520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fda3c0a3500 con 0x7fda3c1399f0 2024-08-19T10:41:17.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.710+0000 7fda337fe640 1 -- 172.21.15.190:0/1841587520 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v82) ==== 56+0+1340 (secure 0 0 0) 0x7fda2401bce0 con 0x7fda3c1399f0 2024-08-19T10:41:17.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.710+0000 7fda337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:17.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.710+0000 7fda337fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:17.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.810+0000 7fda41218640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:17.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.810+0000 7fda41218640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:17.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.810+0000 7fda41218640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:17.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.810+0000 7fda41218640 1 -- 172.21.15.190:0/1841587520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fda3c139dd0 con 0x7fda3c1399f0 2024-08-19T10:41:17.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.810+0000 7fda337fe640 1 -- 172.21.15.190:0/1841587520 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v82) ==== 56+0+1340 (secure 0 0 0) 0x7fda2401c460 con 0x7fda3c1399f0 2024-08-19T10:41:17.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.810+0000 7fda337fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:17.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.810+0000 7fda337fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:17.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 -- 172.21.15.190:0/1841587520 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda1c03be00 msgr2=0x7fda1c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:17.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda1c03be00 0x7fda1c03e2c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fda34007ca0 tx=0x7fda34007bf0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 10 monclient: shutdown 2024-08-19T10:41:17.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:17.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 -- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda3c1399f0 msgr2=0x7fda3c1d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:17.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda3c1399f0 0x7fda3c1d1de0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c450 tx=0x7fda2400c910 comp rx=0 tx=0).stop 2024-08-19T10:41:17.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 -- 172.21.15.190:0/1841587520 shutdown_connections 2024-08-19T10:41:17.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda1c03be00 0x7fda1c03e2c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fda3c140000 0x7fda3c09e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda3c1399f0 0x7fda3c1d1de0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 --2- 172.21.15.190:0/1841587520 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fda3c0a5640 0x7fda3c1d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 -- 172.21.15.190:0/1841587520 >> 172.21.15.190:0/1841587520 conn(0x7fda3c074c10 msgr2=0x7fda3c140c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:17.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 -- 172.21.15.190:0/1841587520 shutdown_connections 2024-08-19T10:41:17.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.814+0000 7fda41218640 1 -- 172.21.15.190:0/1841587520 wait complete. 2024-08-19T10:41:17.831 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm osd.7 host1 2024-08-19T10:41:17.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.946+0000 7ff963a48640 10 monclient: get_monmap_and_config 2024-08-19T10:41:17.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.946+0000 7ff963a48640 10 monclient: build_initial_monmap 2024-08-19T10:41:17.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.946+0000 7ff963a48640 10 monclient: monmap: 2024-08-19T10:41:17.951 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:17.951 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:17.951 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:17.950958+0000 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:17.950958+0000 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:17.952 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:17.953 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 1 Processor -- start 2024-08-19T10:41:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 1 -- start start 2024-08-19T10:41:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 10 monclient: init 2024-08-19T10:41:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 10 monclient: _add_conns ranks=[0,2,8,1,3,4,5,6,7,9,10] 2024-08-19T10:41:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff95c139980 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff95c0a5730 0x7ff95c0a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 10 monclient(hunting): picked mon.noname-b con 0x7ff95c0a5730 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff95c13e8c0 0x7ff95c140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 10 monclient(hunting): picked mon.noname-e con 0x7ff95c13e8c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff95c0a8470 con 0x7ff95c139980 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff95c09ca60 con 0x7ff95c0a5730 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff95c0a3640 con 0x7ff95c13e8c0 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff963a48640 10 monclient(hunting): authenticate will time out at 1123.521123s 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff9617e4640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff95c139980 0x7ff95c139d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:60512/0 (socket says 172.21.15.190:60512) 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff9617e4640 1 -- 172.21.15.190:0/4072121149 learned_addr learned my addr 172.21.15.190:0/4072121149 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff95c0a5730 0x7ff95c0a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 10 monclient(hunting): get_auth_request con 0x7ff95c0a5730 auth_method 0 2024-08-19T10:41:17.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff961fe5640 1 --2- 172.21.15.190:0/4072121149 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff95c13e8c0 0x7ff95c140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff961fe5640 10 monclient(hunting): get_auth_request con 0x7ff95c13e8c0 auth_method 0 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff961fe5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff961fe5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff961fe5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff961fe5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff961fe5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:17.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff961fe5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:17.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 10 monclient(hunting): handle_auth_done global_id 5195 payload 274 2024-08-19T10:41:17.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 1 -- 172.21.15.190:0/4072121149 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff95c13e8c0 msgr2=0x7ff95c140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:17.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 1 --2- 172.21.15.190:0/4072121149 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff95c13e8c0 0x7ff95c140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 1 -- 172.21.15.190:0/4072121149 >> v1:172.21.15.89:6789/0 conn(0x7ff95c139980 legacy=0x7ff95c139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:17.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 10 monclient: _finish_hunting 0 2024-08-19T10:41:17.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 1 monclient: found mon.noname-b 2024-08-19T10:41:17.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:17.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:17.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.950+0000 7ff960fe3640 1 -- 172.21.15.190:0/4072121149 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c0a3850 con 0x7ff95c0a5730 2024-08-19T10:41:17.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.954+0000 7ff960fe3640 1 -- 172.21.15.190:0/4072121149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff95c0a5730 msgr2=0x7ff95c0a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:17.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.954+0000 7ff960fe3640 1 --2- 172.21.15.190:0/4072121149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff95c0a5730 0x7ff95c0a5b10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009820 tx=0x7ff94c01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c39340e6e6333d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:17.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 1 -- 172.21.15.190:0/4072121149 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff94c0b0070 con 0x7ff95c0a5730 2024-08-19T10:41:17.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:17.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:17.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 10 monclient: dump: 2024-08-19T10:41:17.960 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:17.960 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:17.960 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:17.961 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 1 -- 172.21.15.190:0/4072121149 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff94c0aa810 con 0x7ff95c0a5730 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 5 monclient: authenticate success, global_id 5195 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 1 -- 172.21.15.190:0/4072121149 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff94c0a9300 con 0x7ff95c0a5730 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 4 set_mon_vals no callback set 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:17.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff953fff640 10 monclient: dump: 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:17.963 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient: shutdown 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 -- 172.21.15.190:0/4072121149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff95c0a5730 msgr2=0x7ff95c0a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 --2- 172.21.15.190:0/4072121149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff95c0a5730 0x7ff95c0a5b10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009820 tx=0x7ff94c01c380 comp rx=0 tx=0).stop 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 -- 172.21.15.190:0/4072121149 shutdown_connections 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 --2- 172.21.15.190:0/4072121149 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff95c13e8c0 0x7ff95c140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 --2- 172.21.15.190:0/4072121149 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff95c0a5730 0x7ff95c0a5b10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 -- 172.21.15.190:0/4072121149 >> 172.21.15.190:0/4072121149 conn(0x7ff95c0abde0 msgr2=0x7ff95c0ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 -- 172.21.15.190:0/4072121149 shutdown_connections 2024-08-19T10:41:17.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 -- 172.21.15.190:0/4072121149 wait complete. 2024-08-19T10:41:17.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient: build_initial_monmap 2024-08-19T10:41:17.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient: monmap: 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:17.969 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 Processor -- start 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 -- start start 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient: init 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient: _add_conns ranks=[1,4,6,7,0,3,5,8,2] 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0a5730 0x7ff95c132790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient(hunting): picked mon.noname-f con 0x7ff95c0a5730 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff95c139980 0x7ff95c132cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient(hunting): picked mon.noname-c con 0x7ff95c139980 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff95c13e8c0 0x7ff95c13b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:17.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient(hunting): picked mon.noname-d con 0x7ff95c13e8c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:17.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:17.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c143b90 con 0x7ff95c0a5730 2024-08-19T10:41:17.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:17.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff95c143a10 con 0x7ff95c139980 2024-08-19T10:41:17.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:17.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff95c143d10 con 0x7ff95c13e8c0 2024-08-19T10:41:17.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:17.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff963a48640 10 monclient(hunting): authenticate will time out at 1123.530892s 2024-08-19T10:41:17.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0a5730 0x7ff95c132790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0a5730 0x7ff95c132790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56772/0 (socket says 172.21.15.190:56772) 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 1 -- 172.21.15.190:0/1064497696 learned_addr learned my addr 172.21.15.190:0/1064497696 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 10 monclient(hunting): get_auth_request con 0x7ff95c0a5730 auth_method 0 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff961fe5640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff95c13e8c0 0x7ff95c13b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff960fe3640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff95c139980 0x7ff95c132cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 10 monclient(hunting): handle_auth_done global_id 14410 payload 824 2024-08-19T10:41:17.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 1 -- 172.21.15.190:0/1064497696 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff95c13e8c0 msgr2=0x7ff95c13b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff95c13e8c0 0x7ff95c13b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 1 -- 172.21.15.190:0/1064497696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff95c139980 msgr2=0x7ff95c132cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff95c139980 0x7ff95c132cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 10 monclient: _finish_hunting 0 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 1 monclient: found mon.noname-f 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff9617e4640 1 -- 172.21.15.190:0/1064497696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c09cbf0 con 0x7ff95c0a5730 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff960fe3640 10 monclient: get_auth_request con 0x7ff95c139980 auth_method 0 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.958+0000 7ff960fe3640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff95c139980 0x7ff95c132cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff9617e4640 1 -- 172.21.15.190:0/1064497696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0a5730 msgr2=0x7ff95c132790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff9617e4640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0a5730 0x7ff95c132790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c4a0 tx=0x7ff95800c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:17.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 1 -- 172.21.15.190:0/1064497696 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff958017070 con 0x7ff95c0a5730 2024-08-19T10:41:17.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:17.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:17.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 10 monclient: dump: 2024-08-19T10:41:17.974 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:17.974 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:17.974 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.974 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.974 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:17.974 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.974 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:17.974 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff963a48640 5 monclient: authenticate success, global_id 14410 2024-08-19T10:41:17.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff963a48640 10 monclient: _renew_subs 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff963a48640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff963a48640 1 -- 172.21.15.190:0/1064497696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95c134700 con 0x7ff95c0a5730 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff963a48640 10 monclient: _renew_subs 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff963a48640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff963a48640 1 -- 172.21.15.190:0/1064497696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95c09d750 con 0x7ff95c0a5730 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 1 -- 172.21.15.190:0/1064497696 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff958005d60 con 0x7ff95c0a5730 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 1 -- 172.21.15.190:0/1064497696 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9580074e0 con 0x7ff95c0a5730 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 10 monclient: dump: 2024-08-19T10:41:17.976 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:17.977 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:17.978 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:17.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff9537fe640 4 set_mon_vals no callback set 2024-08-19T10:41:17.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff963a48640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:17.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff963a48640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:17.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff963a48640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:17.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff963a48640 1 -- 172.21.15.190:0/1064497696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff93c005090 con 0x7ff95c0a5730 2024-08-19T10:41:17.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 1 -- 172.21.15.190:0/1064497696 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff958007070 con 0x7ff95c0a5730 2024-08-19T10:41:17.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff94003be00 0x7ff94003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:17.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff951ffb640 1 -- 172.21.15.190:0/1064497696 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6249+0+0 (secure 0 0 0) 0x7ff95804e460 con 0x7ff95c0a5730 2024-08-19T10:41:17.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff960fe3640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff94003be00 0x7ff94003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:17.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.962+0000 7ff960fe3640 10 monclient: get_auth_request con 0x7ff94003be00 auth_method 0 2024-08-19T10:41:17.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.966+0000 7ff960fe3640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff94003be00 0x7ff94003e2c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0a3110 tx=0x7ff94c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:17.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.970+0000 7ff951ffb640 1 -- 172.21.15.190:0/1064497696 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff95c134700 con 0x7ff95c0a5730 2024-08-19T10:41:17.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.970+0000 7ff951ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:17.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:17.970+0000 7ff951ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:18.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.150+0000 7ff963a48640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-08-19T10:41:18.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.150+0000 7ff963a48640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-08-19T10:41:18.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.150+0000 7ff963a48640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:18.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.150+0000 7ff963a48640 1 -- 172.21.15.190:0/1064497696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"} v 0) -- 0x7ff93c005b90 con 0x7ff95c0a5730 2024-08-19T10:41:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.394+0000 7ff951ffb640 1 -- 172.21.15.190:0/1064497696 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}]=0 removed item id 7 name 'osd.7' from crush map v83) ==== 143+0+0 (secure 0 0 0) 0x7ff958007680 con 0x7ff95c0a5730 2024-08-19T10:41:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.394+0000 7ff951ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-08-19T10:41:18.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.394+0000 7ff951ffb640 10 monclient: _finish_command 2 = system:0 removed item id 7 name 'osd.7' from crush map 2024-08-19T10:41:18.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.502+0000 7ff963a48640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-08-19T10:41:18.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.502+0000 7ff963a48640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-08-19T10:41:18.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.502+0000 7ff963a48640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:18.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.502+0000 7ff963a48640 1 -- 172.21.15.190:0/1064497696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"} v 0) -- 0x7ff93c002290 con 0x7ff95c0a5730 2024-08-19T10:41:18.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.506+0000 7ff951ffb640 1 -- 172.21.15.190:0/1064497696 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}]=0 device 'osd.7' does not appear in the crush map v83) ==== 145+0+0 (secure 0 0 0) 0x7ff95804e080 con 0x7ff95c0a5730 2024-08-19T10:41:18.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.506+0000 7ff951ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.7", "ancestor": "host1"}] 2024-08-19T10:41:18.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.506+0000 7ff951ffb640 10 monclient: _finish_command 3 = system:0 device 'osd.7' does not appear in the crush map 2024-08-19T10:41:18.509 INFO:tasks.workunit.client.0.smithi190.stderr:device 'osd.7' does not appear in the crush map 2024-08-19T10:41:18.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 -- 172.21.15.190:0/1064497696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff94003be00 msgr2=0x7ff94003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:18.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff94003be00 0x7ff94003e2c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0a3110 tx=0x7ff94c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:18.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 10 monclient: shutdown 2024-08-19T10:41:18.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:18.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 -- 172.21.15.190:0/1064497696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0a5730 msgr2=0x7ff95c132790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:18.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0a5730 0x7ff95c132790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c4a0 tx=0x7ff95800c960 comp rx=0 tx=0).stop 2024-08-19T10:41:18.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 -- 172.21.15.190:0/1064497696 shutdown_connections 2024-08-19T10:41:18.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff94003be00 0x7ff94003e2c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:18.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff95c13e8c0 0x7ff95c13b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:18.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff95c139980 0x7ff95c132cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:18.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 --2- 172.21.15.190:0/1064497696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff95c0a5730 0x7ff95c132790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:18.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 -- 172.21.15.190:0/1064497696 >> 172.21.15.190:0/1064497696 conn(0x7ff95c0abde0 msgr2=0x7ff95c0a9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:18.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 -- 172.21.15.190:0/1064497696 shutdown_connections 2024-08-19T10:41:18.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.510+0000 7ff963a48640 1 -- 172.21.15.190:0/1064497696 wait complete. 2024-08-19T10:41:18.527 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm host1 2024-08-19T10:41:18.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.642+0000 7f87ac700640 10 monclient: get_monmap_and_config 2024-08-19T10:41:18.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.642+0000 7f87ac700640 10 monclient: build_initial_monmap 2024-08-19T10:41:18.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.642+0000 7f87ac700640 10 monclient: monmap: 2024-08-19T10:41:18.647 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:18.648 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:18.648 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:18.647167+0000 2024-08-19T10:41:18.648 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:18.647167+0000 2024-08-19T10:41:18.648 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:18.648 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:18.648 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:18.648 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:18.648 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:18.648 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:18.649 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:18.649 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:18.649 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:18.649 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:18.649 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:18.649 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:18.649 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:18.649 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 1 Processor -- start 2024-08-19T10:41:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 1 -- start start 2024-08-19T10:41:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 10 monclient: init 2024-08-19T10:41:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 10 monclient: _add_conns ranks=[2,0,4,3,9,7,6,8,1,10,5] 2024-08-19T10:41:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87a40a6740 0x7f87a40a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 10 monclient(hunting): picked mon.noname-b con 0x7f87a40a6740 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f87a40a5bc0 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 0x7f87a4140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 10 monclient(hunting): picked mon.noname-c con 0x7f87a40a7230 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87a40a3740 con 0x7f87a40a5bc0 2024-08-19T10:41:18.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:18.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f87a40a4c20 con 0x7f87a40a6740 2024-08-19T10:41:18.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:18.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f87a40a3530 con 0x7f87a40a7230 2024-08-19T10:41:18.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:18.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87ac700640 10 monclient(hunting): authenticate will time out at 1124.217070s 2024-08-19T10:41:18.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 0x7f87a4140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:18.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aa49c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87a40a6740 0x7f87a40a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:18.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87a9c9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f87a40a5bc0 0x7f87a40a6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:60524/0 (socket says 172.21.15.190:60524) 2024-08-19T10:41:18.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87a9c9b640 1 -- 172.21.15.190:0/72570323 learned_addr learned my addr 172.21.15.190:0/72570323 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:18.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aa49c640 10 monclient(hunting): get_auth_request con 0x7f87a40a6740 auth_method 0 2024-08-19T10:41:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aa49c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aa49c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aa49c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 10 monclient(hunting): get_auth_request con 0x7f87a40a7230 auth_method 0 2024-08-19T10:41:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aa49c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aa49c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aa49c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 10 monclient(hunting): handle_auth_done global_id 5233 payload 274 2024-08-19T10:41:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 1 -- 172.21.15.190:0/72570323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87a40a6740 msgr2=0x7f87a40a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 1 --2- 172.21.15.190:0/72570323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87a40a6740 0x7f87a40a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 1 -- 172.21.15.190:0/72570323 >> v1:172.21.15.89:6789/0 conn(0x7f87a40a5bc0 legacy=0x7f87a40a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 10 monclient: _finish_hunting 0 2024-08-19T10:41:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 1 monclient: found mon.noname-c 2024-08-19T10:41:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.646+0000 7f87aac9d640 1 -- 172.21.15.190:0/72570323 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a40a3240 con 0x7f87a40a7230 2024-08-19T10:41:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87aac9d640 1 -- 172.21.15.190:0/72570323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 msgr2=0x7f87a4140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87aac9d640 1 --2- 172.21.15.190:0/72570323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 0x7f87a4140ab0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f879c009df0 tx=0x7f879c01c7f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=7db88339c280319 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 1 -- 172.21.15.190:0/72570323 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f879c0b2070 con 0x7f87a40a7230 2024-08-19T10:41:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 10 monclient: dump: 2024-08-19T10:41:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:18.674 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:18.674 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:18.674 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:18.674 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:18.674 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:18.675 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:18.675 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:18.675 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:18.675 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:18.675 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:18.675 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:18.676 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:18.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 10 monclient: _finish_auth 0 2024-08-19T10:41:18.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 10 monclient: _check_auth_tickets 2024-08-19T10:41:18.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:18.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 1 -- 172.21.15.190:0/72570323 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879c0a5cd0 con 0x7f87a40a7230 2024-08-19T10:41:18.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 5 monclient: authenticate success, global_id 5233 2024-08-19T10:41:18.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:18.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:18.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:18.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 1 -- 172.21.15.190:0/72570323 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f879c0ab7b0 con 0x7f87a40a7230 2024-08-19T10:41:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 4 set_mon_vals no callback set 2024-08-19T10:41:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87a949a640 10 monclient: dump: 2024-08-19T10:41:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:18.680 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 10 monclient: shutdown 2024-08-19T10:41:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 1 -- 172.21.15.190:0/72570323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 msgr2=0x7f87a4140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 1 --2- 172.21.15.190:0/72570323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 0x7f87a4140ab0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f879c009df0 tx=0x7f879c01c7f0 comp rx=0 tx=0).stop 2024-08-19T10:41:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 1 -- 172.21.15.190:0/72570323 shutdown_connections 2024-08-19T10:41:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 1 --2- 172.21.15.190:0/72570323 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 0x7f87a4140ab0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 1 --2- 172.21.15.190:0/72570323 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87a40a6740 0x7f87a40a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 1 -- 172.21.15.190:0/72570323 >> 172.21.15.190:0/72570323 conn(0x7f87a40abd40 msgr2=0x7f87a40ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 1 -- 172.21.15.190:0/72570323 shutdown_connections 2024-08-19T10:41:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.650+0000 7f87ac700640 1 -- 172.21.15.190:0/72570323 wait complete. 2024-08-19T10:41:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient: build_initial_monmap 2024-08-19T10:41:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient: monmap: 2024-08-19T10:41:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 1 Processor -- start 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 1 -- start start 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient: init 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient: _add_conns ranks=[6,4,1,3,8,0,7,2,5] 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f87a40a6740 0x7f87a41364d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient(hunting): picked mon.noname-d con 0x7f87a40a6740 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 0x7f87a4136a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient(hunting): picked mon.noname-c con 0x7f87a40a7230 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a413de30 0x7f87a4136f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient(hunting): picked mon.noname-f con 0x7f87a413de30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f87a4143990 con 0x7f87a413de30 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f87a4143810 con 0x7f87a40a7230 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f87a4143b10 con 0x7f87a40a6740 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87ac700640 10 monclient(hunting): authenticate will time out at 1124.225173s 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a413de30 0x7f87a4136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a413de30 0x7f87a4136f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56784/0 (socket says 172.21.15.190:56784) 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 1 -- 172.21.15.190:0/1007907368 learned_addr learned my addr 172.21.15.190:0/1007907368 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 10 monclient(hunting): get_auth_request con 0x7f87a413de30 auth_method 0 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87a9c9b640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 0x7f87a4136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aa49c640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f87a40a6740 0x7f87a41364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87a9c9b640 10 monclient(hunting): get_auth_request con 0x7f87a40a7230 auth_method 0 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87a9c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87a9c9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87a9c9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 10 monclient(hunting): handle_auth_done global_id 14419 payload 824 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 1 -- 172.21.15.190:0/1007907368 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f87a40a6740 msgr2=0x7f87a41364d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f87a40a6740 0x7f87a41364d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 1 -- 172.21.15.190:0/1007907368 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 msgr2=0x7f87a4136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 0x7f87a4136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 10 monclient: _finish_hunting 0 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 1 monclient: found mon.noname-f 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aac9d640 1 -- 172.21.15.190:0/1007907368 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a413d2d0 con 0x7f87a413de30 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aa49c640 10 monclient: get_auth_request con 0x7f87a40a6740 auth_method 0 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.654+0000 7f87aa49c640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f87a40a6740 0x7f87a41364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87aac9d640 1 -- 172.21.15.190:0/1007907368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a413de30 msgr2=0x7f87a4136f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87aac9d640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a413de30 0x7f87a4136f50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f879c0b0040 tx=0x7f879c0195f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 1 -- 172.21.15.190:0/1007907368 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f879c0bb020 con 0x7f87a413de30 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 10 monclient: dump: 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:18.688 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 10 monclient: _finish_auth 0 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 1 -- 172.21.15.190:0/1007907368 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879c0b3040 con 0x7f87a413de30 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87ac700640 5 monclient: authenticate success, global_id 14419 2024-08-19T10:41:18.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 1 -- 172.21.15.190:0/1007907368 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f879c0b2070 con 0x7f87a413de30 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87a8c99640 4 set_mon_vals no callback set 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 10 monclient: dump: 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:18.690 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87ac700640 10 monclient: _renew_subs 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87ac700640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87ac700640 1 -- 172.21.15.190:0/1007907368 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a40a9470 con 0x7f87a413de30 2024-08-19T10:41:18.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87ac700640 10 monclient: _renew_subs 2024-08-19T10:41:18.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87ac700640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:18.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87ac700640 1 -- 172.21.15.190:0/1007907368 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a413ed90 con 0x7f87a413de30 2024-08-19T10:41:18.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87ac700640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:18.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87ac700640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:18.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87ac700640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:18.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87ac700640 1 -- 172.21.15.190:0/1007907368 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a40a5680 con 0x7f87a413de30 2024-08-19T10:41:18.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 1 -- 172.21.15.190:0/1007907368 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f879c0be020 con 0x7f87a413de30 2024-08-19T10:41:18.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878803be00 0x7f878803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:18.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f879affd640 1 -- 172.21.15.190:0/1007907368 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6228+0+0 (secure 0 0 0) 0x7f879c0e9b00 con 0x7f87a413de30 2024-08-19T10:41:18.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87aa49c640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878803be00 0x7f878803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:18.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.658+0000 7f87aa49c640 10 monclient: get_auth_request con 0x7f878803be00 auth_method 0 2024-08-19T10:41:18.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.662+0000 7f87aa49c640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878803be00 0x7f878803e2c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009ae0 tx=0x7f87a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:18.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.662+0000 7f879affd640 1 -- 172.21.15.190:0/1007907368 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f879c0c5030 con 0x7f87a413de30 2024-08-19T10:41:18.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.662+0000 7f879affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:18.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.662+0000 7f879affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:18.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.838+0000 7f87ac700640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1"}] 2024-08-19T10:41:18.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.838+0000 7f87ac700640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host1"}] 2024-08-19T10:41:18.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.838+0000 7f87ac700640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:18.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:18.838+0000 7f87ac700640 1 -- 172.21.15.190:0/1007907368 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host1"} v 0) -- 0x7f87a406c8d0 con 0x7f87a413de30 2024-08-19T10:41:19.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.862+0000 7f879affd640 1 -- 172.21.15.190:0/1007907368 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1"}]=0 removed item id -11 name 'host1' from crush map v84) ==== 124+0+0 (secure 0 0 0) 0x7f879c016560 con 0x7f87a413de30 2024-08-19T10:41:19.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.862+0000 7f879affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host1"}] 2024-08-19T10:41:19.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.862+0000 7f879affd640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host1' from crush map 2024-08-19T10:41:19.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.974+0000 7f87ac700640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host1"}] 2024-08-19T10:41:19.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.974+0000 7f87ac700640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host1"}] 2024-08-19T10:41:19.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.974+0000 7f87ac700640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:19.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.974+0000 7f87ac700640 1 -- 172.21.15.190:0/1007907368 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host1"} v 0) -- 0x7f87a40a5bc0 con 0x7f87a413de30 2024-08-19T10:41:19.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.974+0000 7f879affd640 1 -- 172.21.15.190:0/1007907368 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host1"}]=0 device 'host1' does not appear in the crush map v84) ==== 124+0+0 (secure 0 0 0) 0x7f879c0abab0 con 0x7f87a413de30 2024-08-19T10:41:19.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.974+0000 7f879affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host1"}] 2024-08-19T10:41:19.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.974+0000 7f879affd640 10 monclient: _finish_command 3 = system:0 device 'host1' does not appear in the crush map 2024-08-19T10:41:19.979 INFO:tasks.workunit.client.0.smithi190.stderr:device 'host1' does not appear in the crush map 2024-08-19T10:41:19.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 -- 172.21.15.190:0/1007907368 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878803be00 msgr2=0x7f878803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:19.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878803be00 0x7f878803e2c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009ae0 tx=0x7f87a0008040 comp rx=0 tx=0).stop 2024-08-19T10:41:19.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 10 monclient: shutdown 2024-08-19T10:41:19.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:19.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 -- 172.21.15.190:0/1007907368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a413de30 msgr2=0x7f87a4136f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:19.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a413de30 0x7f87a4136f50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f879c0b0040 tx=0x7f879c0195f0 comp rx=0 tx=0).stop 2024-08-19T10:41:19.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 -- 172.21.15.190:0/1007907368 shutdown_connections 2024-08-19T10:41:19.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878803be00 0x7f878803e2c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:19.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87a413de30 0x7f87a4136f50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:19.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f87a40a7230 0x7f87a4136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:19.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 --2- 172.21.15.190:0/1007907368 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f87a40a6740 0x7f87a41364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:19.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 -- 172.21.15.190:0/1007907368 >> 172.21.15.190:0/1007907368 conn(0x7f87a40abd40 msgr2=0x7f87a4139700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:19.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 -- 172.21.15.190:0/1007907368 shutdown_connections 2024-08-19T10:41:19.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:19.978+0000 7f87ac700640 1 -- 172.21.15.190:0/1007907368 wait complete. 2024-08-19T10:41:19.997 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -c host1 2024-08-19T10:41:19.997 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:19.997 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -q 0 2024-08-19T10:41:20.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.106+0000 7f69d9af2640 10 monclient: get_monmap_and_config 2024-08-19T10:41:20.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.106+0000 7f69d9af2640 10 monclient: build_initial_monmap 2024-08-19T10:41:20.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.106+0000 7f69d9af2640 10 monclient: monmap: 2024-08-19T10:41:20.111 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:20.111 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.111 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:20.111387+0000 2024-08-19T10:41:20.111 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:20.111387+0000 2024-08-19T10:41:20.111 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:20.112 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:20.113 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 1 Processor -- start 2024-08-19T10:41:20.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 1 -- start start 2024-08-19T10:41:20.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 10 monclient: init 2024-08-19T10:41:20.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:20.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 10 monclient: _add_conns ranks=[10,9,6,8,4,1,2,5,3,0,7] 2024-08-19T10:41:20.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f69d40a36a0 0x7f69d40a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 10 monclient(hunting): picked mon.noname-f con 0x7f69d40a36a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:20.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f69d40a3fc0 0x7f69d4136540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 10 monclient(hunting): picked mon.noname-a con 0x7f69d40a3fc0 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:20.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 0x7f69d4136f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 10 monclient(hunting): picked mon.noname-d con 0x7f69d4136a80 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:20.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f69d40a09d0 con 0x7f69d40a3fc0 2024-08-19T10:41:20.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f69d409e550 con 0x7f69d40a36a0 2024-08-19T10:41:20.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f69d409ffc0 con 0x7f69d4136a80 2024-08-19T10:41:20.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:20.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d9af2640 10 monclient(hunting): authenticate will time out at 1125.681395s 2024-08-19T10:41:20.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f69d40a36a0 0x7f69d40a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f69d40a36a0 0x7f69d40a3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56790/0 (socket says 172.21.15.190:56790) 2024-08-19T10:41:20.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 1 -- 172.21.15.190:0/3579075686 learned_addr learned my addr 172.21.15.190:0/3579075686 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:20.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 10 monclient(hunting): get_auth_request con 0x7f69d40a36a0 auth_method 0 2024-08-19T10:41:20.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d2ffd640 1 --2- 172.21.15.190:0/3579075686 >> v2:172.21.15.89:3300/0 conn(0x7f69d40a3fc0 0x7f69d4136540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d3fff640 1 --2- 172.21.15.190:0/3579075686 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 0x7f69d4136f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d2ffd640 10 monclient(hunting): get_auth_request con 0x7f69d40a3fc0 auth_method 0 2024-08-19T10:41:20.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d3fff640 10 monclient(hunting): get_auth_request con 0x7f69d4136a80 auth_method 0 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 10 monclient(hunting): handle_auth_done global_id 14428 payload 274 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 1 -- 172.21.15.190:0/3579075686 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 msgr2=0x7f69d4136f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 1 --2- 172.21.15.190:0/3579075686 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 0x7f69d4136f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 1 -- 172.21.15.190:0/3579075686 >> v2:172.21.15.89:3300/0 conn(0x7f69d40a3fc0 msgr2=0x7f69d4136540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 1 --2- 172.21.15.190:0/3579075686 >> v2:172.21.15.89:3300/0 conn(0x7f69d40a3fc0 0x7f69d4136540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 1 monclient: found mon.noname-f 2024-08-19T10:41:20.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:20.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:20.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d37fe640 1 -- 172.21.15.190:0/3579075686 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d409db70 con 0x7f69d40a36a0 2024-08-19T10:41:20.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.110+0000 7f69d2ffd640 1 --2- 172.21.15.190:0/3579075686 >> v2:172.21.15.89:3300/0 conn(0x7f69d40a3fc0 0x7f69d4136540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:41:20.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d37fe640 1 -- 172.21.15.190:0/3579075686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d40a36a0 msgr2=0x7f69d40a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:20.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d37fe640 1 --2- 172.21.15.190:0/3579075686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d40a36a0 0x7f69d40a3a80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f69d409e920 tx=0x7f69bc01c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3792f14541f10ae5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:20.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 1 -- 172.21.15.190:0/3579075686 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69bc0b1070 con 0x7f69d40a36a0 2024-08-19T10:41:20.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:20.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 10 monclient: dump: 2024-08-19T10:41:20.125 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.125 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.125 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.125 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.126 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.126 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.126 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.126 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.126 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.126 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.126 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.126 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.126 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.126 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.126 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 1 -- 172.21.15.190:0/3579075686 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69bc0aa9b0 con 0x7f69d40a36a0 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 1 -- 172.21.15.190:0/3579075686 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69bc0a8050 con 0x7f69d40a36a0 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 5 monclient: authenticate success, global_id 14428 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 4 set_mon_vals no callback set 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:20.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d27fc640 10 monclient: dump: 2024-08-19T10:41:20.128 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.128 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.128 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.128 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.128 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.128 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.128 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.128 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.128 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.128 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.129 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.129 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.129 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.129 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.129 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.129 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 10 monclient: shutdown 2024-08-19T10:41:20.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:20.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 1 -- 172.21.15.190:0/3579075686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d40a36a0 msgr2=0x7f69d40a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:20.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 1 --2- 172.21.15.190:0/3579075686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d40a36a0 0x7f69d40a3a80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f69d409e920 tx=0x7f69bc01c500 comp rx=0 tx=0).stop 2024-08-19T10:41:20.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 1 -- 172.21.15.190:0/3579075686 shutdown_connections 2024-08-19T10:41:20.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 1 --2- 172.21.15.190:0/3579075686 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 0x7f69d4136f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 1 --2- 172.21.15.190:0/3579075686 >> v2:172.21.15.89:3300/0 conn(0x7f69d40a3fc0 0x7f69d4136540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 1 --2- 172.21.15.190:0/3579075686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69d40a36a0 0x7f69d40a3a80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 1 -- 172.21.15.190:0/3579075686 >> 172.21.15.190:0/3579075686 conn(0x7f69d40a4790 msgr2=0x7f69d40a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:20.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 1 -- 172.21.15.190:0/3579075686 shutdown_connections 2024-08-19T10:41:20.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 1 -- 172.21.15.190:0/3579075686 wait complete. 2024-08-19T10:41:20.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 10 monclient: build_initial_monmap 2024-08-19T10:41:20.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.114+0000 7f69d9af2640 10 monclient: monmap: 2024-08-19T10:41:20.130 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:20.130 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:20.131 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:20.131 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:20.131 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:20.131 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.131 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:20.131 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:20.131 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:20.131 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:20.131 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:20.131 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:20.131 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 1 Processor -- start 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 1 -- start start 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 10 monclient: init 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 10 monclient: _add_conns ranks=[3,2,6,1,8,0,5,4,7] 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d40a36a0 0x7f69d4132cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 10 monclient(hunting): picked mon.noname-g con 0x7f69d40a36a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69d40a3fc0 0x7f69d41331f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 10 monclient(hunting): picked mon.noname-b con 0x7f69d40a3fc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:20.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 0x7f69d4133730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 10 monclient(hunting): picked mon.noname-d con 0x7f69d4136a80 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:20.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f69d40a5510 con 0x7f69d40a3fc0 2024-08-19T10:41:20.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f69d409fce0 con 0x7f69d40a36a0 2024-08-19T10:41:20.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f69d40a7910 con 0x7f69d4136a80 2024-08-19T10:41:20.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:20.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d9af2640 10 monclient(hunting): authenticate will time out at 1125.689001s 2024-08-19T10:41:20.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d37fe640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d40a36a0 msgr2=0x7f69d4132cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:20.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d37fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d40a36a0 0x7f69d4132cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:20.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 0x7f69d4133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 0x7f69d4133730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:56168/0 (socket says 172.21.15.190:56168) 2024-08-19T10:41:20.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 1 -- 172.21.15.190:0/1317746667 learned_addr learned my addr 172.21.15.190:0/1317746667 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:20.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 10 monclient(hunting): get_auth_request con 0x7f69d4136a80 auth_method 0 2024-08-19T10:41:20.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d2ffd640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69d40a3fc0 0x7f69d41331f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d2ffd640 10 monclient(hunting): get_auth_request con 0x7f69d40a3fc0 auth_method 0 2024-08-19T10:41:20.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 10 monclient(hunting): handle_auth_done global_id 5199 payload 824 2024-08-19T10:41:20.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 1 -- 172.21.15.190:0/1317746667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d40a36a0 msgr2=0x7f69d4132cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d40a36a0 0x7f69d4132cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 1 -- 172.21.15.190:0/1317746667 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69d40a3fc0 msgr2=0x7f69d41331f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69d40a3fc0 0x7f69d41331f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 1 monclient: found mon.noname-d 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.118+0000 7f69d3fff640 1 -- 172.21.15.190:0/1317746667 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d413bd20 con 0x7f69d4136a80 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d3fff640 1 -- 172.21.15.190:0/1317746667 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 msgr2=0x7f69d4133730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d3fff640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 0x7f69d4133730 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69cc007e60 tx=0x7f69cc00e4b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 1 -- 172.21.15.190:0/1317746667 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69cc014890 con 0x7f69d4136a80 2024-08-19T10:41:20.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 10 monclient: dump: 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.137 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.138 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.138 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.138 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.138 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.138 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.138 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 10 monclient: _finish_auth 0 2024-08-19T10:41:20.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:20.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:20.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 1 -- 172.21.15.190:0/1317746667 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69cc019070 con 0x7f69d4136a80 2024-08-19T10:41:20.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 1 -- 172.21.15.190:0/1317746667 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f69cc013380 con 0x7f69d4136a80 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d9af2640 5 monclient: authenticate success, global_id 5199 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d9af2640 10 monclient: _renew_subs 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d9af2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d9af2640 1 -- 172.21.15.190:0/1317746667 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d40a4d90 con 0x7f69d4136a80 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 10 monclient: dump: 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.139 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.140 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d1ffb640 4 set_mon_vals no callback set 2024-08-19T10:41:20.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d9af2640 10 monclient: _renew_subs 2024-08-19T10:41:20.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d9af2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:20.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d9af2640 1 -- 172.21.15.190:0/1317746667 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d413ba40 con 0x7f69d4136a80 2024-08-19T10:41:20.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d9af2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:20.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d9af2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:20.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d9af2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:20.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d9af2640 1 -- 172.21.15.190:0/1317746667 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d40a43a0 con 0x7f69d4136a80 2024-08-19T10:41:20.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 1 -- 172.21.15.190:0/1317746667 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f69cc014420 con 0x7f69d4136a80 2024-08-19T10:41:20.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f69b403be00 0x7f69b403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69cbfff640 1 -- 172.21.15.190:0/1317746667 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(84..84 src has 1..84) ==== 6093+0+0 (secure 0 0 0) 0x7f69cc050200 con 0x7f69d4136a80 2024-08-19T10:41:20.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d37fe640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f69b403be00 0x7f69b403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.122+0000 7f69d37fe640 10 monclient: get_auth_request con 0x7f69b403be00 auth_method 0 2024-08-19T10:41:20.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.126+0000 7f69d37fe640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f69b403be00 0x7f69b403e2c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f69d409e920 tx=0x7f69bc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:20.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.134+0000 7f69cbfff640 1 -- 172.21.15.190:0/1317746667 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f69cc04d730 con 0x7f69d4136a80 2024-08-19T10:41:20.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.134+0000 7f69cbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:20.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.134+0000 7f69cbfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.310+0000 7f69d9af2640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.310+0000 7f69d9af2640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.310+0000 7f69d9af2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.310+0000 7f69d9af2640 1 -- 172.21.15.190:0/1317746667 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f69d409eee0 con 0x7f69d4136a80 2024-08-19T10:41:20.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.310+0000 7f69cbfff640 1 -- 172.21.15.190:0/1317746667 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v85) ==== 56+0+1273 (secure 0 0 0) 0x7f69cc04dab0 con 0x7f69d4136a80 2024-08-19T10:41:20.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.310+0000 7f69cbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:20.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.310+0000 7f69cbfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:20.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.410+0000 7f69d9af2640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:20.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.410+0000 7f69d9af2640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:20.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.410+0000 7f69d9af2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:20.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.410+0000 7f69d9af2640 1 -- 172.21.15.190:0/1317746667 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f69d40008e0 con 0x7f69d4136a80 2024-08-19T10:41:20.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69cbfff640 1 -- 172.21.15.190:0/1317746667 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v85) ==== 56+0+1273 (secure 0 0 0) 0x7f69cc013a40 con 0x7f69d4136a80 2024-08-19T10:41:20.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69cbfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:20.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69cbfff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 -- 172.21.15.190:0/1317746667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f69b403be00 msgr2=0x7f69b403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f69b403be00 0x7f69b403e2c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f69d409e920 tx=0x7f69bc0a7000 comp rx=0 tx=0).stop 2024-08-19T10:41:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 10 monclient: shutdown 2024-08-19T10:41:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 -- 172.21.15.190:0/1317746667 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 msgr2=0x7f69d4133730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 0x7f69d4133730 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69cc007e60 tx=0x7f69cc00e4b0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 -- 172.21.15.190:0/1317746667 shutdown_connections 2024-08-19T10:41:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f69b403be00 0x7f69b403e2c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69d4136a80 0x7f69d4133730 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69d40a3fc0 0x7f69d41331f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 --2- 172.21.15.190:0/1317746667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d40a36a0 0x7f69d4132cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 -- 172.21.15.190:0/1317746667 >> 172.21.15.190:0/1317746667 conn(0x7f69d40a4790 msgr2=0x7f69d413cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:20.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 -- 172.21.15.190:0/1317746667 shutdown_connections 2024-08-19T10:41:20.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.414+0000 7f69d9af2640 1 -- 172.21.15.190:0/1317746667 wait complete. 2024-08-19T10:41:20.435 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd tree-from host1 2024-08-19T10:41:20.435 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:41:20.435 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree-from host1 2024-08-19T10:41:20.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.538+0000 7f9a92024640 10 monclient: get_monmap_and_config 2024-08-19T10:41:20.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient: build_initial_monmap 2024-08-19T10:41:20.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient: monmap: 2024-08-19T10:41:20.543 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:20.543 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.543 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:20.543701+0000 2024-08-19T10:41:20.543 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:20.543701+0000 2024-08-19T10:41:20.543 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:20.544 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 1 Processor -- start 2024-08-19T10:41:20.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 1 -- start start 2024-08-19T10:41:20.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient: init 2024-08-19T10:41:20.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:20.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient: _add_conns ranks=[2,8,4,9,6,5,10,3,7,0,1] 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a8c139850 0x7f9a8c139c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient(hunting): picked mon.noname-b con 0x7f9a8c139850 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9a8c06e600 0x7f9a8c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient(hunting): picked mon.noname-e con 0x7f9a8c06e600 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a8c06e0e0 0x7f9a8c0a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient(hunting): picked mon.noname-c con 0x7f9a8c06e0e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9a8c06da20 con 0x7f9a8c139850 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9a8c06f180 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9a8c06f460 con 0x7f9a8c06e600 2024-08-19T10:41:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a92024640 10 monclient(hunting): authenticate will time out at 1126.113777s 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9a8c06e600 0x7f9a8c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a8c139850 0x7f9a8c139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a8c06e0e0 0x7f9a8c0a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9a8c06e600 0x7f9a8c0a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:34846/0 (socket says 172.21.15.190:34846) 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 1 -- 172.21.15.190:0/733905611 learned_addr learned my addr 172.21.15.190:0/733905611 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 10 monclient(hunting): get_auth_request con 0x7f9a8c06e600 auth_method 0 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8b7fe640 10 monclient(hunting): get_auth_request con 0x7f9a8c139850 auth_method 0 2024-08-19T10:41:20.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8bfff640 10 monclient(hunting): get_auth_request con 0x7f9a8c06e0e0 auth_method 0 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 10 monclient(hunting): handle_auth_done global_id 5120 payload 274 2024-08-19T10:41:20.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 1 -- 172.21.15.190:0/733905611 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a8c06e0e0 msgr2=0x7f9a8c0a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:20.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 1 --2- 172.21.15.190:0/733905611 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a8c06e0e0 0x7f9a8c0a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 1 -- 172.21.15.190:0/733905611 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a8c139850 msgr2=0x7f9a8c139c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:20.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 1 --2- 172.21.15.190:0/733905611 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a8c139850 0x7f9a8c139c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:20.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 1 monclient: found mon.noname-e 2024-08-19T10:41:20.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:20.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:20.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.542+0000 7f9a8affd640 1 -- 172.21.15.190:0/733905611 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c0a7220 con 0x7f9a8c06e600 2024-08-19T10:41:20.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.546+0000 7f9a8affd640 1 -- 172.21.15.190:0/733905611 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9a8c06e600 msgr2=0x7f9a8c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:20.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.546+0000 7f9a8affd640 1 --2- 172.21.15.190:0/733905611 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9a8c06e600 0x7f9a8c0a5730 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a800098a0 tx=0x7f9a8001c380 comp rx=0 tx=0).ready entity=mon.8 client_cookie=b392da080e09f0f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:20.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 1 -- 172.21.15.190:0/733905611 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a800b0070 con 0x7f9a8c06e600 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 10 monclient: dump: 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.552 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 1 -- 172.21.15.190:0/733905611 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a800aa810 con 0x7f9a8c06e600 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 1 -- 172.21.15.190:0/733905611 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a800aae30 con 0x7f9a8c06e600 2024-08-19T10:41:20.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8a7fc640 10 monclient: dump: 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.555 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 5 monclient: authenticate success, global_id 5120 2024-08-19T10:41:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 4 set_mon_vals no callback set 2024-08-19T10:41:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient: shutdown 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 -- 172.21.15.190:0/733905611 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9a8c06e600 msgr2=0x7f9a8c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 --2- 172.21.15.190:0/733905611 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9a8c06e600 0x7f9a8c0a5730 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a800098a0 tx=0x7f9a8001c380 comp rx=0 tx=0).stop 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 -- 172.21.15.190:0/733905611 shutdown_connections 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 --2- 172.21.15.190:0/733905611 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9a8c06e0e0 0x7f9a8c0a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 --2- 172.21.15.190:0/733905611 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9a8c06e600 0x7f9a8c0a5730 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 --2- 172.21.15.190:0/733905611 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9a8c139850 0x7f9a8c139c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 -- 172.21.15.190:0/733905611 >> 172.21.15.190:0/733905611 conn(0x7f9a8c0aa320 msgr2=0x7f9a8c0a6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 -- 172.21.15.190:0/733905611 shutdown_connections 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 -- 172.21.15.190:0/733905611 wait complete. 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient: build_initial_monmap 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient: monmap: 2024-08-19T10:41:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 Processor -- start 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 -- start start 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient: init 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient: _add_conns ranks=[1,3,7,2,8,0,4,5,6] 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a8c06e0e0 0x7f9a8c1334a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient(hunting): picked mon.noname-f con 0x7f9a8c06e0e0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a8c06e600 0x7f9a8c1339e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient(hunting): picked mon.noname-g con 0x7f9a8c06e600 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a8c139850 0x7f9a8c133f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient(hunting): picked mon.noname-i con 0x7f9a8c139850 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c142f70 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9a8c1430f0 con 0x7f9a8c06e600 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9a8c143270 con 0x7f9a8c139850 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a92024640 10 monclient(hunting): authenticate will time out at 1126.123365s 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8affd640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a8c06e600 msgr2=0x7f9a8c1339e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8affd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a8c06e600 0x7f9a8c1339e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a8c06e0e0 0x7f9a8c1334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8bfff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a8c139850 0x7f9a8c133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a8c06e0e0 0x7f9a8c1334a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56800/0 (socket says 172.21.15.190:56800) 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 1 -- 172.21.15.190:0/175928762 learned_addr learned my addr 172.21.15.190:0/175928762 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 10 monclient(hunting): get_auth_request con 0x7f9a8c06e0e0 auth_method 0 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8bfff640 10 monclient(hunting): get_auth_request con 0x7f9a8c139850 auth_method 0 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 10 monclient(hunting): handle_auth_done global_id 14437 payload 824 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 1 -- 172.21.15.190:0/175928762 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a8c139850 msgr2=0x7f9a8c133f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 1 --2- 172.21.15.190:0/175928762 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9a8c139850 0x7f9a8c133f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 1 -- 172.21.15.190:0/175928762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a8c06e600 msgr2=0x7f9a8c1339e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 1 --2- 172.21.15.190:0/175928762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a8c06e600 0x7f9a8c1339e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 1 monclient: found mon.noname-f 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.550+0000 7f9a8b7fe640 1 -- 172.21.15.190:0/175928762 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c13ec70 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a8b7fe640 1 -- 172.21.15.190:0/175928762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a8c06e0e0 msgr2=0x7f9a8c1334a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a8b7fe640 1 --2- 172.21.15.190:0/175928762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a8c06e0e0 0x7f9a8c1334a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00e870 tx=0x7f9a7c00ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 1 -- 172.21.15.190:0/175928762 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a7c017070 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 10 monclient: dump: 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 1 -- 172.21.15.190:0/175928762 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7c013470 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 1 -- 172.21.15.190:0/175928762 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9a7c013a90 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 10 monclient: dump: 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a897fa640 4 set_mon_vals no callback set 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a92024640 5 monclient: authenticate success, global_id 14437 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a92024640 10 monclient: _renew_subs 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a92024640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a92024640 1 -- 172.21.15.190:0/175928762 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c13ef50 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a92024640 10 monclient: _renew_subs 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a92024640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a92024640 1 -- 172.21.15.190:0/175928762 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c1403c0 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a92024640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a92024640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a92024640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a92024640 1 -- 172.21.15.190:0/175928762 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c139c30 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 1 -- 172.21.15.190:0/175928762 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9a7c01a070 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 1 --2- 172.21.15.190:0/175928762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9a7803be30 0x7f9a7803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a73fff640 1 -- 172.21.15.190:0/175928762 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6093+0+0 (secure 0 0 0) 0x7f9a7c00f040 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a8affd640 1 --2- 172.21.15.190:0/175928762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9a7803be30 0x7f9a7803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.554+0000 7f9a8affd640 10 monclient: get_auth_request con 0x7f9a7803be30 auth_method 0 2024-08-19T10:41:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.558+0000 7f9a8affd640 1 --2- 172.21.15.190:0/175928762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9a7803be30 0x7f9a7803e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9a800ab000 tx=0x7f9a800a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.562+0000 7f9a73fff640 1 -- 172.21.15.190:0/175928762 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9a7c021070 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.562+0000 7f9a73fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.562+0000 7f9a73fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:20.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.730+0000 7f9a92024640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host1"}] 2024-08-19T10:41:20.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.730+0000 7f9a92024640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "host1"}] 2024-08-19T10:41:20.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.730+0000 7f9a92024640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:20.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.730+0000 7f9a92024640 1 -- 172.21.15.190:0/175928762 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd tree-from", "bucket": "host1"} v 0) -- 0x7f9a8c13cbf0 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.730+0000 7f9a73fff640 1 -- 172.21.15.190:0/175928762 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host1"}]=-2 bucket 'host1' does not exist v85) ==== 109+0+0 (secure 0 0 0) 0x7f9a7c013620 con 0x7f9a8c06e0e0 2024-08-19T10:41:20.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.730+0000 7f9a73fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "host1"}] 2024-08-19T10:41:20.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.730+0000 7f9a73fff640 10 monclient: _finish_command 2 = mon:2 bucket 'host1' does not exist 2024-08-19T10:41:20.733 INFO:tasks.workunit.client.0.smithi190.stderr:Error ENOENT: bucket 'host1' does not exist 2024-08-19T10:41:20.738 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:41:20.738 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree-from host2 2024-08-19T10:41:20.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient: get_monmap_and_config 2024-08-19T10:41:20.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient: build_initial_monmap 2024-08-19T10:41:20.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient: monmap: 2024-08-19T10:41:20.839 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:20.839 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.839 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:20.839742+0000 2024-08-19T10:41:20.839 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:20.839742+0000 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:20.840 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:20.841 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 1 Processor -- start 2024-08-19T10:41:20.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 1 -- start start 2024-08-19T10:41:20.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient: init 2024-08-19T10:41:20.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:20.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient: _add_conns ranks=[2,3,9,5,0,7,1,10,8,6,4] 2024-08-19T10:41:20.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60c00a46b0 0x7f60c00a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient(hunting): picked mon.noname-b con 0x7f60c00a46b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:20.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 0x7f60c0139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient(hunting): picked mon.noname-g con 0x7f60c00a4fd0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f60c0139da0 0x7f60c013a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient(hunting): picked mon.noname-a con 0x7f60c0139da0 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f60c00a0210 con 0x7f60c0139da0 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f60c00a1cc0 con 0x7f60c00a46b0 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f60c006cea0 con 0x7f60c00a4fd0 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c6f83640 10 monclient(hunting): authenticate will time out at 1126.409694s 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60bffff640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 msgr2=0x7f60c0139860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60bffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 0x7f60c0139860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:20.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f60c0139da0 0x7f60c013a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c4d1f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60c00a46b0 0x7f60c00a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f60c0139da0 0x7f60c013a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:40650/0 (socket says 172.21.15.190:40650) 2024-08-19T10:41:20.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 1 -- 172.21.15.190:0/2778774493 learned_addr learned my addr 172.21.15.190:0/2778774493 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:20.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 10 monclient(hunting): get_auth_request con 0x7f60c0139da0 auth_method 0 2024-08-19T10:41:20.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c4d1f640 10 monclient(hunting): get_auth_request con 0x7f60c00a46b0 auth_method 0 2024-08-19T10:41:20.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c4d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c4d1f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c4d1f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c4d1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c4d1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c4d1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 10 monclient(hunting): handle_auth_done global_id 5301 payload 274 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 1 -- 172.21.15.190:0/2778774493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 msgr2=0x7f60c0139860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 1 --2- 172.21.15.190:0/2778774493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 0x7f60c0139860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 1 -- 172.21.15.190:0/2778774493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60c00a46b0 msgr2=0x7f60c00a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 1 --2- 172.21.15.190:0/2778774493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60c00a46b0 0x7f60c00a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 10 monclient: _finish_hunting 0 2024-08-19T10:41:20.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 1 monclient: found mon.noname-a 2024-08-19T10:41:20.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:20.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:20.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c5520640 1 -- 172.21.15.190:0/2778774493 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c006ca10 con 0x7f60c0139da0 2024-08-19T10:41:20.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.838+0000 7f60c4d1f640 1 --2- 172.21.15.190:0/2778774493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60c00a46b0 0x7f60c00a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:20.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60c5520640 1 -- 172.21.15.190:0/2778774493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c0139da0 msgr2=0x7f60c013a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:20.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60c5520640 1 --2- 172.21.15.190:0/2778774493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c0139da0 0x7f60c013a200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009e50 tx=0x7f60b001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f22015ba688433f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:20.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 1 -- 172.21.15.190:0/2778774493 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60b00090d0 con 0x7f60c0139da0 2024-08-19T10:41:20.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:20.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 10 monclient: dump: 2024-08-19T10:41:20.850 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.851 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.851 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.851 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.851 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.851 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.851 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.851 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.851 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.851 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.851 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.851 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 1 -- 172.21.15.190:0/2778774493 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b00a59c0 con 0x7f60c0139da0 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 1 -- 172.21.15.190:0/2778774493 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60b00ad3f0 con 0x7f60c0139da0 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60c6f83640 5 monclient: authenticate success, global_id 5301 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60c6f83640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60c6f83640 4 set_mon_vals no callback set 2024-08-19T10:41:20.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.842+0000 7f60bf7fe640 10 monclient: dump: 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.853 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.854 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.854 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.854 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.854 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.854 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient: shutdown 2024-08-19T10:41:20.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 -- 172.21.15.190:0/2778774493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c0139da0 msgr2=0x7f60c013a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 --2- 172.21.15.190:0/2778774493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c0139da0 0x7f60c013a200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009e50 tx=0x7f60b001c910 comp rx=0 tx=0).stop 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 -- 172.21.15.190:0/2778774493 shutdown_connections 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 --2- 172.21.15.190:0/2778774493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60c0139da0 0x7f60c013a200 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 --2- 172.21.15.190:0/2778774493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 0x7f60c0139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 --2- 172.21.15.190:0/2778774493 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60c00a46b0 0x7f60c00a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 -- 172.21.15.190:0/2778774493 >> 172.21.15.190:0/2778774493 conn(0x7f60c00a92a0 msgr2=0x7f60c00a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 -- 172.21.15.190:0/2778774493 shutdown_connections 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 -- 172.21.15.190:0/2778774493 wait complete. 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient: build_initial_monmap 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient: monmap: 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:20.855 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:20.856 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 Processor -- start 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 -- start start 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient: init 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient: _add_conns ranks=[5,3,7,1,2,4,0,6,8] 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f60c00a46b0 0x7f60c01d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient(hunting): picked mon.noname-h con 0x7f60c00a46b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 0x7f60c01d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient(hunting): picked mon.noname-g con 0x7f60c00a4fd0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60c0139da0 0x7f60c01d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient(hunting): picked mon.noname-i con 0x7f60c0139da0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f60c00a6170 con 0x7f60c00a4fd0 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f60c006d120 con 0x7f60c00a46b0 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f60c006d2a0 con 0x7f60c0139da0 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c6f83640 10 monclient(hunting): authenticate will time out at 1126.417846s 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60bffff640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 msgr2=0x7f60c01d1a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60bffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 0x7f60c01d1a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f60c00a46b0 0x7f60c01d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c5520640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60c0139da0 0x7f60c01d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f60c00a46b0 0x7f60c01d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:39598/0 (socket says 172.21.15.190:39598) 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c5520640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60c0139da0 0x7f60c01d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44120/0 (socket says 172.21.15.190:44120) 2024-08-19T10:41:20.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 1 -- 172.21.15.190:0/3790036477 learned_addr learned my addr 172.21.15.190:0/3790036477 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:20.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 10 monclient(hunting): get_auth_request con 0x7f60c00a46b0 auth_method 0 2024-08-19T10:41:20.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c5520640 10 monclient(hunting): get_auth_request con 0x7f60c0139da0 auth_method 0 2024-08-19T10:41:20.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c5520640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:20.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c5520640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:20.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c5520640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:20.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c5520640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c5520640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c5520640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 10 monclient(hunting): handle_auth_done global_id 5216 payload 824 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 1 -- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60c0139da0 msgr2=0x7f60c01d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60c0139da0 0x7f60c01d1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 1 -- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 msgr2=0x7f60c01d1a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 0x7f60c01d1a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 10 monclient: _finish_hunting 0 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 1 monclient: found mon.noname-h 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:20.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:20.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c4d1f640 1 -- 172.21.15.190:0/3790036477 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c01239e0 con 0x7f60c00a46b0 2024-08-19T10:41:20.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.846+0000 7f60c5520640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60c0139da0 0x7f60c01d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:20.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c4d1f640 1 -- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f60c00a46b0 msgr2=0x7f60c01d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:20.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c4d1f640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f60c00a46b0 0x7f60c01d14f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f60b800e9e0 tx=0x7f60b800eea0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:20.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 1 -- 172.21.15.190:0/3790036477 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60b8017070 con 0x7f60c00a46b0 2024-08-19T10:41:20.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:20.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 10 monclient: dump: 2024-08-19T10:41:20.863 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.863 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.864 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 10 monclient: _finish_auth 0 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 1 -- 172.21.15.190:0/3790036477 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b800cc40 con 0x7f60c00a46b0 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c6f83640 5 monclient: authenticate success, global_id 5216 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c6f83640 10 monclient: _renew_subs 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c6f83640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c6f83640 1 -- 172.21.15.190:0/3790036477 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c0123cc0 con 0x7f60c00a46b0 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 1 -- 172.21.15.190:0/3790036477 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60b800cdf0 con 0x7f60c00a46b0 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:20.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 10 monclient: dump: 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:20.866 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60beffd640 4 set_mon_vals no callback set 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c6f83640 10 monclient: _renew_subs 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c6f83640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c6f83640 1 -- 172.21.15.190:0/3790036477 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c0140260 con 0x7f60c00a46b0 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c6f83640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c6f83640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c6f83640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60c6f83640 1 -- 172.21.15.190:0/3790036477 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c0077860 con 0x7f60c00a46b0 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 1 -- 172.21.15.190:0/3790036477 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f60b801a070 con 0x7f60c00a46b0 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f60a003be00 0x7f60a003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:20.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bd7fa640 1 -- 172.21.15.190:0/3790036477 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(85..85 src has 1..85) ==== 6093+0+0 (secure 0 0 0) 0x7f60b800f040 con 0x7f60c00a46b0 2024-08-19T10:41:20.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bffff640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f60a003be00 0x7f60a003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:20.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.850+0000 7f60bffff640 10 monclient: get_auth_request con 0x7f60a003be00 auth_method 0 2024-08-19T10:41:20.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.854+0000 7f60bffff640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f60a003be00 0x7f60a003e2c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f60c013ead0 tx=0x7f60a8006cc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:20.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.854+0000 7f60bd7fa640 1 -- 172.21.15.190:0/3790036477 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f60b8021070 con 0x7f60c00a46b0 2024-08-19T10:41:20.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.854+0000 7f60bd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:20.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:20.854+0000 7f60bd7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:21.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.014+0000 7f60c6f83640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host2"}] 2024-08-19T10:41:21.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.014+0000 7f60c6f83640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-08-19T10:41:21.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.014+0000 7f60c6f83640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:21.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.014+0000 7f60c6f83640 1 -- 172.21.15.190:0/3790036477 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd tree-from", "bucket": "host2"} v 0) -- 0x7f60c006ec50 con 0x7f60c00a46b0 2024-08-19T10:41:21.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.014+0000 7f60bd7fa640 1 -- 172.21.15.190:0/3790036477 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host2"}]=0 v85) ==== 80+0+177 (secure 0 0 0) 0x7f60b801bdc0 con 0x7f60c00a46b0 2024-08-19T10:41:21.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.014+0000 7f60bd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-08-19T10:41:21.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.014+0000 7f60bd7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:21.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.106+0000 7f60c6f83640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "host2"}] 2024-08-19T10:41:21.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.106+0000 7f60c6f83640 10 monclient: _send_command 3 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-08-19T10:41:21.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.106+0000 7f60c6f83640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:21.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.106+0000 7f60c6f83640 1 -- 172.21.15.190:0/3790036477 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd tree-from", "bucket": "host2"} v 0) -- 0x7f60c01397a0 con 0x7f60c00a46b0 2024-08-19T10:41:21.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60bd7fa640 1 -- 172.21.15.190:0/3790036477 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "host2"}]=0 v85) ==== 80+0+177 (secure 0 0 0) 0x7f60b801c450 con 0x7f60c00a46b0 2024-08-19T10:41:21.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60bd7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree-from", "bucket": "host2"}] 2024-08-19T10:41:21.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60bd7fa640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:21.112 INFO:tasks.workunit.client.0.smithi190.stdout:ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF 2024-08-19T10:41:21.112 INFO:tasks.workunit.client.0.smithi190.stdout:-16 1.00000 host host2 2024-08-19T10:41:21.112 INFO:tasks.workunit.client.0.smithi190.stdout: 9 1.00000 osd.9 down 1.00000 1.00000 2024-08-19T10:41:21.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 -- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f60a003be00 msgr2=0x7f60a003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:21.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f60a003be00 0x7f60a003e2c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f60c013ead0 tx=0x7f60a8006cc0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 10 monclient: shutdown 2024-08-19T10:41:21.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:21.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 -- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f60c00a46b0 msgr2=0x7f60c01d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:21.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f60c00a46b0 0x7f60c01d14f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f60b800e9e0 tx=0x7f60b800eea0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 -- 172.21.15.190:0/3790036477 shutdown_connections 2024-08-19T10:41:21.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f60a003be00 0x7f60a003e2c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60c0139da0 0x7f60c01d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60c00a4fd0 0x7f60c01d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 --2- 172.21.15.190:0/3790036477 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f60c00a46b0 0x7f60c01d14f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 -- 172.21.15.190:0/3790036477 >> 172.21.15.190:0/3790036477 conn(0x7f60c00a92a0 msgr2=0x7f60c0073830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:21.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 -- 172.21.15.190:0/3790036477 shutdown_connections 2024-08-19T10:41:21.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.110+0000 7f60c6f83640 1 -- 172.21.15.190:0/3790036477 wait complete. 2024-08-19T10:41:21.128 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd tree-from osd.9 2024-08-19T10:41:21.128 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:41:21.129 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree-from osd.9 2024-08-19T10:41:21.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient: get_monmap_and_config 2024-08-19T10:41:21.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient: build_initial_monmap 2024-08-19T10:41:21.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient: monmap: 2024-08-19T10:41:21.245 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:21.245 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.245 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:21.245473+0000 2024-08-19T10:41:21.245 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:21.245473+0000 2024-08-19T10:41:21.245 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:21.245 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:21.246 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 1 Processor -- start 2024-08-19T10:41:21.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 1 -- start start 2024-08-19T10:41:21.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient: init 2024-08-19T10:41:21.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:21.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient: _add_conns ranks=[7,2,3,10,0,1,8,6,9,5,4] 2024-08-19T10:41:21.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f37581397e0 0x7f3758071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient(hunting): picked mon.noname-i con 0x7f37581397e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3758139bc0 0x7f37580a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient(hunting): picked mon.noname-b con 0x7f3758139bc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3758139fa0 0x7f37580a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient(hunting): picked mon.noname-g con 0x7f3758139fa0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3758072520 con 0x7f3758139bc0 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f375806de00 con 0x7f3758139fa0 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f375806f120 con 0x7f37581397e0 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:21.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375ec4a640 10 monclient(hunting): authenticate will time out at 1126.815369s 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375d1e7640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3758139fa0 msgr2=0x7f37580a5c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375d1e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3758139fa0 0x7f37580a5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f37581397e0 0x7f3758071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f37581397e0 0x7f3758071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44122/0 (socket says 172.21.15.190:44122) 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 1 -- 172.21.15.190:0/3664014012 learned_addr learned my addr 172.21.15.190:0/3664014012 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 10 monclient(hunting): get_auth_request con 0x7f37581397e0 auth_method 0 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:21.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:21.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f374ffff640 1 --2- 172.21.15.190:0/3664014012 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3758139bc0 0x7f37580a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 10 monclient(hunting): handle_auth_done global_id 14407 payload 274 2024-08-19T10:41:21.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 1 -- 172.21.15.190:0/3664014012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3758139fa0 msgr2=0x7f37580a5c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:21.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 1 --2- 172.21.15.190:0/3664014012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3758139fa0 0x7f37580a5c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 1 -- 172.21.15.190:0/3664014012 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3758139bc0 msgr2=0x7f37580a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:21.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 1 --2- 172.21.15.190:0/3664014012 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3758139bc0 0x7f37580a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 10 monclient: _finish_hunting 0 2024-08-19T10:41:21.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 1 monclient: found mon.noname-i 2024-08-19T10:41:21.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:21.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:21.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.242+0000 7f375c9e6640 1 -- 172.21.15.190:0/3664014012 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375806f400 con 0x7f37581397e0 2024-08-19T10:41:21.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375c9e6640 1 -- 172.21.15.190:0/3664014012 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f37581397e0 msgr2=0x7f3758071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:21.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375c9e6640 1 --2- 172.21.15.190:0/3664014012 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f37581397e0 0x7f3758071fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37540098a0 tx=0x7f375401c460 comp rx=0 tx=0).ready entity=mon.7 client_cookie=855876b2d053021 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:21.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 1 -- 172.21.15.190:0/3664014012 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f37540b0070 con 0x7f37581397e0 2024-08-19T10:41:21.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:21.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 10 monclient: dump: 2024-08-19T10:41:21.253 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.253 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.253 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.254 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:21.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:21.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:21.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 1 -- 172.21.15.190:0/3664014012 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37540aa810 con 0x7f37581397e0 2024-08-19T10:41:21.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 5 monclient: authenticate success, global_id 14407 2024-08-19T10:41:21.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:21.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:21.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:21.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 1 -- 172.21.15.190:0/3664014012 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f37540a93e0 con 0x7f37581397e0 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 4 set_mon_vals no callback set 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374f7fe640 10 monclient: dump: 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.256 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient: shutdown 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 -- 172.21.15.190:0/3664014012 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f37581397e0 msgr2=0x7f3758071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 --2- 172.21.15.190:0/3664014012 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f37581397e0 0x7f3758071fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37540098a0 tx=0x7f375401c460 comp rx=0 tx=0).stop 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 -- 172.21.15.190:0/3664014012 shutdown_connections 2024-08-19T10:41:21.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 --2- 172.21.15.190:0/3664014012 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3758139fa0 0x7f37580a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 --2- 172.21.15.190:0/3664014012 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3758139bc0 0x7f37580a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 --2- 172.21.15.190:0/3664014012 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f37581397e0 0x7f3758071fe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 -- 172.21.15.190:0/3664014012 >> 172.21.15.190:0/3664014012 conn(0x7f37580aa2b0 msgr2=0x7f37580a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 -- 172.21.15.190:0/3664014012 shutdown_connections 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 -- 172.21.15.190:0/3664014012 wait complete. 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient: build_initial_monmap 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient: monmap: 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:21.258 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 Processor -- start 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 -- start start 2024-08-19T10:41:21.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient: init 2024-08-19T10:41:21.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient: _add_conns ranks=[4,7,8,3,1,0,2,5,6] 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37581397e0 0x7f375813a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient(hunting): picked mon.noname-c con 0x7f37581397e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3758139bc0 0x7f375813a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient(hunting): picked mon.noname-i con 0x7f3758139bc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3758139fa0 0x7f375813af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient(hunting): picked mon.noname-e con 0x7f3758139fa0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3758143080 con 0x7f37581397e0 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f3758142f00 con 0x7f3758139bc0 2024-08-19T10:41:21.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3758143200 con 0x7f3758139fa0 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375ec4a640 10 monclient(hunting): authenticate will time out at 1126.822505s 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3758139bc0 0x7f375813a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3758139bc0 0x7f375813a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44138/0 (socket says 172.21.15.190:44138) 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 1 -- 172.21.15.190:0/1568299811 learned_addr learned my addr 172.21.15.190:0/1568299811 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 10 monclient(hunting): get_auth_request con 0x7f3758139bc0 auth_method 0 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:21.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375d1e7640 1 --2- 172.21.15.190:0/1568299811 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3758139fa0 0x7f375813af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375d1e7640 10 monclient(hunting): get_auth_request con 0x7f3758139fa0 auth_method 0 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375d1e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375d1e7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f375d1e7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 10 monclient(hunting): handle_auth_done global_id 14416 payload 824 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 1 -- 172.21.15.190:0/1568299811 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3758139fa0 msgr2=0x7f375813af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 1 --2- 172.21.15.190:0/1568299811 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3758139fa0 0x7f375813af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 1 -- 172.21.15.190:0/1568299811 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37581397e0 msgr2=0x7f375813a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 1 --2- 172.21.15.190:0/1568299811 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f37581397e0 0x7f375813a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:21.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 1 monclient: found mon.noname-i 2024-08-19T10:41:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.250+0000 7f374ffff640 1 -- 172.21.15.190:0/1568299811 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375813d3d0 con 0x7f3758139bc0 2024-08-19T10:41:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374ffff640 1 -- 172.21.15.190:0/1568299811 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3758139bc0 msgr2=0x7f375813a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374ffff640 1 --2- 172.21.15.190:0/1568299811 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3758139bc0 0x7f375813a9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f374800c7f0 tx=0x7f374800ccb0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 1 -- 172.21.15.190:0/1568299811 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3748015070 con 0x7f3758139bc0 2024-08-19T10:41:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 10 monclient: dump: 2024-08-19T10:41:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.267 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.268 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 1 -- 172.21.15.190:0/1568299811 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3748007ca0 con 0x7f3758139bc0 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 1 -- 172.21.15.190:0/1568299811 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3748007e50 con 0x7f3758139bc0 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 10 monclient: dump: 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.269 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.270 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374effd640 4 set_mon_vals no callback set 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375ec4a640 5 monclient: authenticate success, global_id 14416 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375ec4a640 10 monclient: _renew_subs 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375ec4a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375ec4a640 1 -- 172.21.15.190:0/1568299811 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375813d6b0 con 0x7f3758139bc0 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375ec4a640 10 monclient: _renew_subs 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375ec4a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375ec4a640 1 -- 172.21.15.190:0/1568299811 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f375813cb00 con 0x7f3758139bc0 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 1 -- 172.21.15.190:0/1568299811 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3748007830 con 0x7f3758139bc0 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375ec4a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375ec4a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375ec4a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:21.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375ec4a640 1 -- 172.21.15.190:0/1568299811 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3758071fe0 con 0x7f3758139bc0 2024-08-19T10:41:21.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 1 --2- 172.21.15.190:0/1568299811 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f374403bfa0 0x7f374403e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f374d7fa640 1 -- 172.21.15.190:0/1568299811 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(85..85 src has 1..85) ==== 6093+0+0 (secure 0 0 0) 0x7f374804eb40 con 0x7f3758139bc0 2024-08-19T10:41:21.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375c9e6640 1 --2- 172.21.15.190:0/1568299811 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f374403bfa0 0x7f374403e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.254+0000 7f375c9e6640 10 monclient: get_auth_request con 0x7f374403bfa0 auth_method 0 2024-08-19T10:41:21.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.258+0000 7f375c9e6640 1 --2- 172.21.15.190:0/1568299811 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f374403bfa0 0x7f374403e460 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3754009870 tx=0x7f3754004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:21.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.258+0000 7f374d7fa640 1 -- 172.21.15.190:0/1568299811 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f374801e070 con 0x7f3758139bc0 2024-08-19T10:41:21.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.258+0000 7f374d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:21.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.258+0000 7f374d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:21.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.418+0000 7f375ec4a640 10 start_mon_command cmd=[{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-08-19T10:41:21.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.418+0000 7f375ec4a640 10 monclient: _send_command 2 [{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-08-19T10:41:21.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.418+0000 7f375ec4a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:21.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.418+0000 7f375ec4a640 1 -- 172.21.15.190:0/1568299811 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd tree-from", "bucket": "osd.9"} v 0) -- 0x7f375806fd10 con 0x7f3758139bc0 2024-08-19T10:41:21.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.418+0000 7f374d7fa640 1 -- 172.21.15.190:0/1568299811 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd tree-from", "bucket": "osd.9"}]=-22 "osd.9" is not a bucket v85) ==== 103+0+0 (secure 0 0 0) 0x7f374804d020 con 0x7f3758139bc0 2024-08-19T10:41:21.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.418+0000 7f374d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree-from", "bucket": "osd.9"}] 2024-08-19T10:41:21.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.418+0000 7f374d7fa640 10 monclient: _finish_command 2 = mon:22 "osd.9" is not a bucket 2024-08-19T10:41:21.423 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: "osd.9" is not a bucket 2024-08-19T10:41:21.427 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:41:21.427 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush rm bar 2024-08-19T10:41:21.427 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:41:21.427 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm bar 2024-08-19T10:41:21.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.538+0000 7f008c543640 10 monclient: get_monmap_and_config 2024-08-19T10:41:21.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.538+0000 7f008c543640 10 monclient: build_initial_monmap 2024-08-19T10:41:21.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.538+0000 7f008c543640 10 monclient: monmap: 2024-08-19T10:41:21.542 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:21.542 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.542 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:21.542346+0000 2024-08-19T10:41:21.542 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:21.542346+0000 2024-08-19T10:41:21.542 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:21.543 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.538+0000 7f008c543640 1 Processor -- start 2024-08-19T10:41:21.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.538+0000 7f008c543640 1 -- start start 2024-08-19T10:41:21.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.538+0000 7f008c543640 10 monclient: init 2024-08-19T10:41:21.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:21.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 10 monclient: _add_conns ranks=[6,4,8,3,0,2,9,1,5,7,10] 2024-08-19T10:41:21.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00840a3610 0x7f00840a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 10 monclient(hunting): picked mon.noname-d con 0x7f00840a3610 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:21.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00840a3f30 0x7f00841363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 10 monclient(hunting): picked mon.noname-c con 0x7f00840a3f30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0084136910 0x7f0084136d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 10 monclient(hunting): picked mon.noname-e con 0x7f0084136910 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f0084073290 con 0x7f00840a3f30 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f00840729f0 con 0x7f00840a3610 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f0084072cd0 con 0x7f0084136910 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008c543640 10 monclient(hunting): authenticate will time out at 1127.112236s 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008a2df640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00840a3610 0x7f00840a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008aae0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0084136910 0x7f0084136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00840a3f30 0x7f00841363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008a2df640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00840a3610 0x7f00840a39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:56180/0 (socket says 172.21.15.190:56180) 2024-08-19T10:41:21.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008a2df640 1 -- 172.21.15.190:0/1781635046 learned_addr learned my addr 172.21.15.190:0/1781635046 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:21.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008a2df640 10 monclient(hunting): get_auth_request con 0x7f00840a3610 auth_method 0 2024-08-19T10:41:21.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008a2df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008a2df640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008a2df640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 10 monclient(hunting): get_auth_request con 0x7f00840a3f30 auth_method 0 2024-08-19T10:41:21.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008aae0640 10 monclient(hunting): get_auth_request con 0x7f0084136910 auth_method 0 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008aae0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008aae0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008aae0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008aae0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008aae0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008aae0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008a2df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:21.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008a2df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:21.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f008a2df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:21.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 10 monclient(hunting): handle_auth_done global_id 5260 payload 274 2024-08-19T10:41:21.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 1 -- 172.21.15.190:0/1781635046 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0084136910 msgr2=0x7f0084136d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:21.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 1 --2- 172.21.15.190:0/1781635046 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0084136910 0x7f0084136d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 1 -- 172.21.15.190:0/1781635046 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00840a3610 msgr2=0x7f00840a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:21.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 1 --2- 172.21.15.190:0/1781635046 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00840a3610 0x7f00840a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 10 monclient: _finish_hunting 0 2024-08-19T10:41:21.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 1 monclient: found mon.noname-c 2024-08-19T10:41:21.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:21.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:21.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.542+0000 7f0089ade640 1 -- 172.21.15.190:0/1781635046 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00840a5570 con 0x7f00840a3f30 2024-08-19T10:41:21.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f0089ade640 1 -- 172.21.15.190:0/1781635046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00840a3f30 msgr2=0x7f00841363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:21.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f0089ade640 1 --2- 172.21.15.190:0/1781635046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00840a3f30 0x7f00841363d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f00800097f0 tx=0x7f008001c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=b7d06ce2c9fa6c5b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:21.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 1 -- 172.21.15.190:0/1781635046 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00800b0070 con 0x7f00840a3f30 2024-08-19T10:41:21.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:21.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 10 monclient: dump: 2024-08-19T10:41:21.554 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.554 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.554 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.554 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.554 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 10 monclient: _finish_auth 0 2024-08-19T10:41:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 1 -- 172.21.15.190:0/1781635046 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00800aa810 con 0x7f00840a3f30 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 1 -- 172.21.15.190:0/1781635046 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00800aae30 con 0x7f00840a3f30 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 5 monclient: authenticate success, global_id 5260 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 4 set_mon_vals no callback set 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f00892dd640 10 monclient: dump: 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.557 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 10 monclient: shutdown 2024-08-19T10:41:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 1 -- 172.21.15.190:0/1781635046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00840a3f30 msgr2=0x7f00841363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 1 --2- 172.21.15.190:0/1781635046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00840a3f30 0x7f00841363d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f00800097f0 tx=0x7f008001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 1 -- 172.21.15.190:0/1781635046 shutdown_connections 2024-08-19T10:41:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 1 --2- 172.21.15.190:0/1781635046 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0084136910 0x7f0084136d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 1 --2- 172.21.15.190:0/1781635046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00840a3f30 0x7f00841363d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 1 --2- 172.21.15.190:0/1781635046 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00840a3610 0x7f00840a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 1 -- 172.21.15.190:0/1781635046 >> 172.21.15.190:0/1781635046 conn(0x7f00840a4700 msgr2=0x7f00840a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 1 -- 172.21.15.190:0/1781635046 shutdown_connections 2024-08-19T10:41:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.546+0000 7f008c543640 1 -- 172.21.15.190:0/1781635046 wait complete. 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient: build_initial_monmap 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient: monmap: 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:21.559 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 1 Processor -- start 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 1 -- start start 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient: init 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient: _add_conns ranks=[3,6,1,7,2,4,8,5,0] 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00840a3610 0x7f00841ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient(hunting): picked mon.noname-g con 0x7f00840a3610 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00840a3f30 0x7f00841353d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient(hunting): picked mon.noname-d con 0x7f00840a3f30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0084136910 0x7f0084135910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient(hunting): picked mon.noname-f con 0x7f0084136910 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0084073e20 con 0x7f0084136910 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0084073ca0 con 0x7f00840a3610 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f0084071540 con 0x7f00840a3f30 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008c543640 10 monclient(hunting): authenticate will time out at 1127.121158s 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008a2df640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00840a3610 msgr2=0x7f00841ceca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008a2df640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00840a3610 0x7f00841ceca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0084136910 0x7f0084135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0084136910 0x7f0084135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56806/0 (socket says 172.21.15.190:56806) 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 1 -- 172.21.15.190:0/1728192557 learned_addr learned my addr 172.21.15.190:0/1728192557 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 10 monclient(hunting): get_auth_request con 0x7f0084136910 auth_method 0 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f0089ade640 1 --2- 172.21.15.190:0/1728192557 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00840a3f30 0x7f00841353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f0089ade640 10 monclient(hunting): get_auth_request con 0x7f00840a3f30 auth_method 0 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f0089ade640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f0089ade640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f0089ade640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 10 monclient(hunting): handle_auth_done global_id 14446 payload 824 2024-08-19T10:41:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 1 -- 172.21.15.190:0/1728192557 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00840a3f30 msgr2=0x7f00841353d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 1 --2- 172.21.15.190:0/1728192557 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00840a3f30 0x7f00841353d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 1 -- 172.21.15.190:0/1728192557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00840a3610 msgr2=0x7f00841ceca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 1 --2- 172.21.15.190:0/1728192557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00840a3610 0x7f00841ceca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 10 monclient: _finish_hunting 0 2024-08-19T10:41:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 1 monclient: found mon.noname-f 2024-08-19T10:41:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 1 -- 172.21.15.190:0/1728192557 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008411fa20 con 0x7f0084136910 2024-08-19T10:41:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 1 -- 172.21.15.190:0/1728192557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0084136910 msgr2=0x7f0084135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.550+0000 7f008aae0640 1 --2- 172.21.15.190:0/1728192557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0084136910 0x7f0084135910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f007c00efc0 tx=0x7f007c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 1 -- 172.21.15.190:0/1728192557 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f007c018020 con 0x7f0084136910 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 10 monclient: dump: 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 10 monclient: _finish_auth 0 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008c543640 5 monclient: authenticate success, global_id 14446 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 1 -- 172.21.15.190:0/1728192557 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f007c0148f0 con 0x7f0084136910 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 1 -- 172.21.15.190:0/1728192557 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f007c009e60 con 0x7f0084136910 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008c543640 10 monclient: _renew_subs 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008c543640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008c543640 1 -- 172.21.15.190:0/1728192557 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00840a0c50 con 0x7f0084136910 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f0088adc640 4 set_mon_vals no callback set 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 10 monclient: dump: 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.569 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008c543640 10 monclient: _renew_subs 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008c543640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008c543640 1 -- 172.21.15.190:0/1728192557 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00841214e0 con 0x7f0084136910 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008c543640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008c543640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008c543640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008c543640 1 -- 172.21.15.190:0/1728192557 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00841363d0 con 0x7f0084136910 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 1 -- 172.21.15.190:0/1728192557 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f007c01b020 con 0x7f0084136910 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 1 --2- 172.21.15.190:0/1728192557 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f006803be00 0x7f006803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f007affd640 1 -- 172.21.15.190:0/1728192557 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6093+0+0 (secure 0 0 0) 0x7f007c04fc20 con 0x7f0084136910 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008a2df640 1 --2- 172.21.15.190:0/1728192557 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f006803be00 0x7f006803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008a2df640 10 monclient: get_auth_request con 0x7f006803be00 auth_method 0 2024-08-19T10:41:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.554+0000 7f008a2df640 1 --2- 172.21.15.190:0/1728192557 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f006803be00 0x7f006803e2c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0074006c30 tx=0x7f0074009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.558+0000 7f007affd640 1 -- 172.21.15.190:0/1728192557 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f007c022030 con 0x7f0084136910 2024-08-19T10:41:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.558+0000 7f007affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.558+0000 7f007affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:21.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.734+0000 7f008c543640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:21.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.734+0000 7f008c543640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:21.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.734+0000 7f008c543640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:21.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.734+0000 7f008c543640 1 -- 172.21.15.190:0/1728192557 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) -- 0x7f0084121720 con 0x7f0084136910 2024-08-19T10:41:21.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.734+0000 7f007affd640 1 -- 172.21.15.190:0/1728192557 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=-39 (39) Directory not empty v85) ==== 99+0+0 (secure 0 0 0) 0x7f007c010040 con 0x7f0084136910 2024-08-19T10:41:21.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.734+0000 7f007affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:21.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.734+0000 7f007affd640 10 monclient: _finish_command 2 = mon:39 (39) Directory not empty 2024-08-19T10:41:21.738 INFO:tasks.workunit.client.0.smithi190.stderr:Error ENOTEMPTY: (39) Directory not empty 2024-08-19T10:41:21.742 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:41:21.742 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush unlink host2 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient: get_monmap_and_config 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient: build_initial_monmap 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient: monmap: 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:21.857242+0000 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:21.857242+0000 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:21.857 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:21.858 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:21.858 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:21.858 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:21.858 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:21.858 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:21.858 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:21.858 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:21.858 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:21.858 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 1 Processor -- start 2024-08-19T10:41:21.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 1 -- start start 2024-08-19T10:41:21.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient: init 2024-08-19T10:41:21.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:21.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient: _add_conns ranks=[3,10,5,7,0,6,1,2,8,9,4] 2024-08-19T10:41:21.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0430139980 0x7f0430139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient(hunting): picked mon.noname-g con 0x7f0430139980 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:21.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f043013a2a0 0x7f04300a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient(hunting): picked mon.noname-f con 0x7f043013a2a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04300a5c70 0x7f04300a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient(hunting): picked mon.noname-h con 0x7f04300a5c70 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f043009ca60 con 0x7f043013a2a0 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f04300a3640 con 0x7f0430139980 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f04300a8470 con 0x7f04300a5c70 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f0434a0e640 10 monclient(hunting): authenticate will time out at 1127.427164s 2024-08-19T10:41:21.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042e59c640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0430139980 msgr2=0x7f0430139d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:21.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042e59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0430139980 0x7f0430139d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:21.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04300a5c70 0x7f04300a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042dd9b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f043013a2a0 0x7f04300a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042dd9b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f043013a2a0 0x7f04300a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56820/0 (socket says 172.21.15.190:56820) 2024-08-19T10:41:21.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04300a5c70 0x7f04300a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:39604/0 (socket says 172.21.15.190:39604) 2024-08-19T10:41:21.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042dd9b640 1 -- 172.21.15.190:0/4214610595 learned_addr learned my addr 172.21.15.190:0/4214610595 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:21.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042dd9b640 10 monclient(hunting): get_auth_request con 0x7f043013a2a0 auth_method 0 2024-08-19T10:41:21.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 10 monclient(hunting): get_auth_request con 0x7f04300a5c70 auth_method 0 2024-08-19T10:41:21.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:21.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:21.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:21.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:21.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 10 monclient(hunting): handle_auth_done global_id 5225 payload 274 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 1 -- 172.21.15.190:0/4214610595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0430139980 msgr2=0x7f0430139d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 1 --2- 172.21.15.190:0/4214610595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0430139980 0x7f0430139d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 1 -- 172.21.15.190:0/4214610595 >> v2:172.21.15.190:3300/0 conn(0x7f043013a2a0 msgr2=0x7f04300a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 1 --2- 172.21.15.190:0/4214610595 >> v2:172.21.15.190:3300/0 conn(0x7f043013a2a0 0x7f04300a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 1 monclient: found mon.noname-h 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042ed9d640 1 -- 172.21.15.190:0/4214610595 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04300a3850 con 0x7f04300a5c70 2024-08-19T10:41:21.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.854+0000 7f042dd9b640 1 --2- 172.21.15.190:0/4214610595 >> v2:172.21.15.190:3300/0 conn(0x7f043013a2a0 0x7f04300a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:21.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042ed9d640 1 -- 172.21.15.190:0/4214610595 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04300a5c70 msgr2=0x7f04300a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:21.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042ed9d640 1 --2- 172.21.15.190:0/4214610595 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04300a5c70 0x7f04300a67e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04200097f0 tx=0x7f042001c0c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=9df58010c982257c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:21.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 1 -- 172.21.15.190:0/4214610595 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04200b1070 con 0x7f04300a5c70 2024-08-19T10:41:21.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:21.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 10 monclient: dump: 2024-08-19T10:41:21.867 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.867 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.868 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 10 monclient: _finish_auth 0 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 1 -- 172.21.15.190:0/4214610595 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04200a4d00 con 0x7f04300a5c70 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 1 -- 172.21.15.190:0/4214610595 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04200aac20 con 0x7f04300a5c70 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 5 monclient: authenticate success, global_id 5225 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 4 set_mon_vals no callback set 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:21.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f042d59a640 10 monclient: dump: 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.870 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 10 monclient: shutdown 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 1 -- 172.21.15.190:0/4214610595 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04300a5c70 msgr2=0x7f04300a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 1 --2- 172.21.15.190:0/4214610595 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04300a5c70 0x7f04300a67e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04200097f0 tx=0x7f042001c0c0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 1 -- 172.21.15.190:0/4214610595 shutdown_connections 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 1 --2- 172.21.15.190:0/4214610595 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04300a5c70 0x7f04300a67e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 1 --2- 172.21.15.190:0/4214610595 >> v2:172.21.15.190:3300/0 conn(0x7f043013a2a0 0x7f04300a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 1 --2- 172.21.15.190:0/4214610595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0430139980 0x7f0430139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 1 -- 172.21.15.190:0/4214610595 >> 172.21.15.190:0/4214610595 conn(0x7f04300abde0 msgr2=0x7f04300ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 1 -- 172.21.15.190:0/4214610595 shutdown_connections 2024-08-19T10:41:21.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.862+0000 7f0434a0e640 1 -- 172.21.15.190:0/4214610595 wait complete. 2024-08-19T10:41:21.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient: build_initial_monmap 2024-08-19T10:41:21.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient: monmap: 2024-08-19T10:41:21.876 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:21.876 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:21.876 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:21.876 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:21.876 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:21.876 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.876 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 1 Processor -- start 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 1 -- start start 2024-08-19T10:41:21.877 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient: init 2024-08-19T10:41:21.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:21.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient: _add_conns ranks=[4,0,6,2,8,5,3,1,7] 2024-08-19T10:41:21.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04300a5c70 0x7f043013fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient(hunting): picked mon.noname-c con 0x7f04300a5c70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:21.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0430139980 0x7f04300a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient(hunting): picked mon.noname-a con 0x7f0430139980 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:21.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f04300a0d70 0x7f04300a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient(hunting): picked mon.noname-d con 0x7f04300a0d70 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:21.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0430143270 con 0x7f0430139980 2024-08-19T10:41:21.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f04301430f0 con 0x7f04300a5c70 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f04301433f0 con 0x7f04300a0d70 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f0434a0e640 10 monclient(hunting): authenticate will time out at 1127.437823s 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0430139980 0x7f04300a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0430139980 0x7f04300a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:40660/0 (socket says 172.21.15.190:40660) 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042ed9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f04300a0d70 0x7f04300a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042dd9b640 1 -- 172.21.15.190:0/2780475355 learned_addr learned my addr 172.21.15.190:0/2780475355 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042dd9b640 10 monclient(hunting): get_auth_request con 0x7f0430139980 auth_method 0 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04300a5c70 0x7f043013fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 10 monclient(hunting): get_auth_request con 0x7f04300a5c70 auth_method 0 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042ed9d640 10 monclient(hunting): get_auth_request con 0x7f04300a0d70 auth_method 0 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:21.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 10 monclient(hunting): handle_auth_done global_id 5269 payload 824 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 1 -- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f04300a0d70 msgr2=0x7f04300a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f04300a0d70 0x7f04300a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 1 -- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0430139980 msgr2=0x7f04300a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0430139980 0x7f04300a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 1 monclient: found mon.noname-c 2024-08-19T10:41:21.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:21.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:21.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042e59c640 1 -- 172.21.15.190:0/2780475355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043013a720 con 0x7f04300a5c70 2024-08-19T10:41:21.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.866+0000 7f042dd9b640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0430139980 0x7f04300a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:21.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f042e59c640 1 -- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04300a5c70 msgr2=0x7f043013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:21.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f042e59c640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04300a5c70 0x7f043013fe70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f042800be00 tx=0x7f0428009f70 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:21.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 1 -- 172.21.15.190:0/2780475355 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04280158e0 con 0x7f04300a5c70 2024-08-19T10:41:21.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:21.882 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 10 monclient: dump: 2024-08-19T10:41:21.882 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.882 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.883 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.883 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.883 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.883 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.883 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.883 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.883 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.883 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.883 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.883 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 10 monclient: _finish_auth 0 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f0434a0e640 5 monclient: authenticate success, global_id 5269 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 1 -- 172.21.15.190:0/2780475355 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0428010070 con 0x7f04300a5c70 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f0434a0e640 10 monclient: _renew_subs 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f0434a0e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:21.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f0434a0e640 1 -- 172.21.15.190:0/2780475355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043013c7a0 con 0x7f04300a5c70 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 1 -- 172.21.15.190:0/2780475355 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0428013370 con 0x7f04300a5c70 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f042cd99640 4 set_mon_vals no callback set 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 10 monclient: dump: 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:21.885 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:21.886 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:21.886 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:21.886 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:21.886 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:21.886 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:21.886 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:21.886 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:21.886 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f0434a0e640 10 monclient: _renew_subs 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f0434a0e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f0434a0e640 1 -- 172.21.15.190:0/2780475355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f04300a5950 con 0x7f04300a5c70 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f0434a0e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f0434a0e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f0434a0e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f0434a0e640 1 -- 172.21.15.190:0/2780475355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f043013ae00 con 0x7f04300a5c70 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 1 -- 172.21.15.190:0/2780475355 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0428015b00 con 0x7f04300a5c70 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f041003be00 0x7f041003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f041effd640 1 -- 172.21.15.190:0/2780475355 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(85..85 src has 1..85) ==== 6093+0+0 (secure 0 0 0) 0x7f042804f290 con 0x7f04300a5c70 2024-08-19T10:41:21.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f042dd9b640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f041003be00 0x7f041003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:21.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.870+0000 7f042dd9b640 10 monclient: get_auth_request con 0x7f041003be00 auth_method 0 2024-08-19T10:41:21.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.874+0000 7f042dd9b640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f041003be00 0x7f041003e2c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0424009650 tx=0x7f0424009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:21.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.886+0000 7f041effd640 1 -- 172.21.15.190:0/2780475355 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0428005070 con 0x7f04300a5c70 2024-08-19T10:41:21.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.886+0000 7f041effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:21.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:21.886+0000 7f041effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:22.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:22.046+0000 7f0434a0e640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host2"}] 2024-08-19T10:41:22.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:22.046+0000 7f0434a0e640 10 monclient: _send_command 2 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-08-19T10:41:22.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:22.046+0000 7f0434a0e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:22.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:22.046+0000 7f0434a0e640 1 -- 172.21.15.190:0/2780475355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd crush unlink", "name": "host2"} v 0) -- 0x7f04300a8fc0 con 0x7f04300a5c70 2024-08-19T10:41:22.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:22.882+0000 7f041effd640 1 -- 172.21.15.190:0/2780475355 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v86) ==== 128+0+0 (secure 0 0 0) 0x7f0428013510 con 0x7f04300a5c70 2024-08-19T10:41:22.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:22.882+0000 7f041effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-08-19T10:41:22.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:22.882+0000 7f041effd640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'host2' from crush map 2024-08-19T10:41:22.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:22.982+0000 7f0434a0e640 10 start_mon_command cmd=[{"prefix": "osd crush unlink", "name": "host2"}] 2024-08-19T10:41:22.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:22.982+0000 7f0434a0e640 10 monclient: _send_command 3 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-08-19T10:41:22.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:22.982+0000 7f0434a0e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:22.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:22.982+0000 7f0434a0e640 1 -- 172.21.15.190:0/2780475355 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd crush unlink", "name": "host2"} v 0) -- 0x7f0430139d60 con 0x7f04300a5c70 2024-08-19T10:41:23.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.886+0000 7f041effd640 1 -- 172.21.15.190:0/2780475355 <== mon.4 v2:172.21.15.89:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush unlink", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v87) ==== 128+0+0 (secure 0 0 0) 0x7f042801d510 con 0x7f04300a5c70 2024-08-19T10:41:23.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.886+0000 7f041effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush unlink", "name": "host2"}] 2024-08-19T10:41:23.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.886+0000 7f041effd640 10 monclient: _finish_command 3 = system:0 removed item id -16 name 'host2' from crush map 2024-08-19T10:41:23.890 INFO:tasks.workunit.client.0.smithi190.stderr:removed item id -16 name 'host2' from crush map 2024-08-19T10:41:23.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 -- 172.21.15.190:0/2780475355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f041003be00 msgr2=0x7f041003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:23.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f041003be00 0x7f041003e2c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0424009650 tx=0x7f0424009240 comp rx=0 tx=0).stop 2024-08-19T10:41:23.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 10 monclient: shutdown 2024-08-19T10:41:23.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:23.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 -- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04300a5c70 msgr2=0x7f043013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:23.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04300a5c70 0x7f043013fe70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f042800be00 tx=0x7f0428009f70 comp rx=0 tx=0).stop 2024-08-19T10:41:23.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 -- 172.21.15.190:0/2780475355 shutdown_connections 2024-08-19T10:41:23.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f041003be00 0x7f041003e2c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:23.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f04300a0d70 0x7f04300a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:23.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0430139980 0x7f04300a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:23.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 --2- 172.21.15.190:0/2780475355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04300a5c70 0x7f043013fe70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:23.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 -- 172.21.15.190:0/2780475355 >> 172.21.15.190:0/2780475355 conn(0x7f04300abde0 msgr2=0x7f04300ab6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:23.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 -- 172.21.15.190:0/2780475355 shutdown_connections 2024-08-19T10:41:23.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:23.890+0000 7f0434a0e640 1 -- 172.21.15.190:0/2780475355 wait complete. 2024-08-19T10:41:23.914 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket host-for-test host root=root-for-test rack=rack-for-test 2024-08-19T10:41:24.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.034+0000 7f11479c5640 10 monclient: get_monmap_and_config 2024-08-19T10:41:24.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.034+0000 7f11479c5640 10 monclient: build_initial_monmap 2024-08-19T10:41:24.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.034+0000 7f11479c5640 10 monclient: monmap: 2024-08-19T10:41:24.037 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:24.037 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:24.037454+0000 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:24.037454+0000 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:24.038 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:24.039 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:24.039 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:24.039 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:24.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.034+0000 7f11479c5640 1 Processor -- start 2024-08-19T10:41:24.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.034+0000 7f11479c5640 1 -- start start 2024-08-19T10:41:24.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.034+0000 7f11479c5640 10 monclient: init 2024-08-19T10:41:24.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:24.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 10 monclient: _add_conns ranks=[1,0,2,5,10,7,9,6,3,8,4] 2024-08-19T10:41:24.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f11400a36a0 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:24.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f11400a19d0 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:24.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11400a4190 0x7f114013fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:24.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 10 monclient(hunting): picked mon.noname-b con 0x7f11400a4190 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f114009d050 con 0x7f11400a19d0 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f114006f250 con 0x7f11400a36a0 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f114009dec0 con 0x7f11400a4190 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f11479c5640 10 monclient(hunting): authenticate will time out at 1129.608226s 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145761640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f11400a36a0 0x7f11400a3a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:53332/0 (socket says 172.21.15.190:53332) 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145761640 1 -- 172.21.15.190:0/572861272 learned_addr learned my addr 172.21.15.190:0/572861272 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 1 --2- 172.21.15.190:0/572861272 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11400a4190 0x7f114013fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 10 monclient(hunting): get_auth_request con 0x7f11400a4190 auth_method 0 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:24.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:24.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:24.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:24.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:24.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 10 monclient(hunting): handle_auth_done global_id 5240 payload 274 2024-08-19T10:41:24.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 1 -- 172.21.15.190:0/572861272 >> v1:172.21.15.190:6789/0 conn(0x7f11400a36a0 legacy=0x7f11400a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:24.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 1 -- 172.21.15.190:0/572861272 >> v1:172.21.15.89:6789/0 conn(0x7f11400a19d0 legacy=0x7f114013c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:24.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 10 monclient: _finish_hunting 0 2024-08-19T10:41:24.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 1 monclient: found mon.noname-b 2024-08-19T10:41:24.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:24.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:24.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.038+0000 7f1145f62640 1 -- 172.21.15.190:0/572861272 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114009d620 con 0x7f11400a4190 2024-08-19T10:41:24.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1145f62640 1 -- 172.21.15.190:0/572861272 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11400a4190 msgr2=0x7f114013fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:24.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1145f62640 1 --2- 172.21.15.190:0/572861272 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11400a4190 0x7f114013fff0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1138009dc0 tx=0x7f113801c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d0187cd0af18470 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:24.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 1 -- 172.21.15.190:0/572861272 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f11380b2070 con 0x7f11400a4190 2024-08-19T10:41:24.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:24.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 10 monclient: dump: 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:24.046 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 1 -- 172.21.15.190:0/572861272 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11380a5a90 con 0x7f11400a4190 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 1 -- 172.21.15.190:0/572861272 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f11380a5c80 con 0x7f11400a4190 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 5 monclient: authenticate success, global_id 5240 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 4 set_mon_vals no callback set 2024-08-19T10:41:24.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f1137fff640 10 monclient: dump: 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:24.048 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:24.049 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:24.049 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:24.049 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:24.049 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:24.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 10 monclient: shutdown 2024-08-19T10:41:24.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:24.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 1 -- 172.21.15.190:0/572861272 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11400a4190 msgr2=0x7f114013fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:24.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 1 --2- 172.21.15.190:0/572861272 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11400a4190 0x7f114013fff0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1138009dc0 tx=0x7f113801c7f0 comp rx=0 tx=0).stop 2024-08-19T10:41:24.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 1 -- 172.21.15.190:0/572861272 shutdown_connections 2024-08-19T10:41:24.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 1 --2- 172.21.15.190:0/572861272 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11400a4190 0x7f114013fff0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:24.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 1 -- 172.21.15.190:0/572861272 >> 172.21.15.190:0/572861272 conn(0x7f11400a4790 msgr2=0x7f11400a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:24.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 1 -- 172.21.15.190:0/572861272 shutdown_connections 2024-08-19T10:41:24.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.042+0000 7f11479c5640 1 -- 172.21.15.190:0/572861272 wait complete. 2024-08-19T10:41:24.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient: build_initial_monmap 2024-08-19T10:41:24.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient: monmap: 2024-08-19T10:41:24.054 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:24.054 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:24.054 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:24.054 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 1 Processor -- start 2024-08-19T10:41:24.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 1 -- start start 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient: init 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient: _add_conns ranks=[3,1,4,0,5,6,7,8,2] 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11400a19d0 0x7f1140126a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient(hunting): picked mon.noname-g con 0x7f11400a19d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11400a36a0 0x7f114013e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient(hunting): picked mon.noname-f con 0x7f11400a36a0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11400a4190 0x7f1140124450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient(hunting): picked mon.noname-c con 0x7f11400a4190 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1140050c20 con 0x7f11400a36a0 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:24.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1140142d50 con 0x7f11400a19d0 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f114006a540 con 0x7f11400a4190 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f11479c5640 10 monclient(hunting): authenticate will time out at 1129.617139s 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1145761640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11400a19d0 msgr2=0x7f1140126a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1145761640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11400a19d0 0x7f1140126a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11400a36a0 0x7f114013e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11400a36a0 0x7f114013e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56836/0 (socket says 172.21.15.190:56836) 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 1 -- 172.21.15.190:0/3154620480 learned_addr learned my addr 172.21.15.190:0/3154620480 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 10 monclient(hunting): get_auth_request con 0x7f11400a36a0 auth_method 0 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:24.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 10 monclient(hunting): handle_auth_done global_id 14464 payload 824 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 1 -- 172.21.15.190:0/3154620480 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11400a4190 msgr2=0x7f1140124450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11400a4190 0x7f1140124450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 1 -- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11400a19d0 msgr2=0x7f1140126a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11400a19d0 0x7f1140126a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 10 monclient: _finish_hunting 0 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 1 monclient: found mon.noname-f 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:24.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 1 -- 172.21.15.190:0/3154620480 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1140073970 con 0x7f11400a36a0 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 1 -- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11400a36a0 msgr2=0x7f114013e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1144f60640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11400a36a0 0x7f114013e8d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f113000c450 tx=0x7f113000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.046+0000 7f1135ffb640 1 -- 172.21.15.190:0/3154620480 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1130011c10 con 0x7f11400a36a0 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 10 monclient: dump: 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:24.059 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:24.060 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:24.060 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:24.060 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:24.060 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:24.060 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:24.060 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:24.060 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 1 -- 172.21.15.190:0/3154620480 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1130017070 con 0x7f11400a36a0 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11479c5640 5 monclient: authenticate success, global_id 14464 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11479c5640 10 monclient: _renew_subs 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11479c5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11479c5640 1 -- 172.21.15.190:0/3154620480 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1140127f90 con 0x7f11400a36a0 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:24.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 1 -- 172.21.15.190:0/3154620480 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1130011df0 con 0x7f11400a36a0 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11377fe640 4 set_mon_vals no callback set 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 10 monclient: dump: 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:24.062 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11479c5640 10 monclient: _renew_subs 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11479c5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11479c5640 1 -- 172.21.15.190:0/3154620480 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1140127cb0 con 0x7f11400a36a0 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11479c5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:24.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11479c5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:24.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11479c5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:24.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 1 -- 172.21.15.190:0/3154620480 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f11300117a0 con 0x7f11400a36a0 2024-08-19T10:41:24.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f11479c5640 1 -- 172.21.15.190:0/3154620480 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11400a3000 con 0x7f11400a36a0 2024-08-19T10:41:24.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f112403be00 0x7f112403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:24.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1135ffb640 1 -- 172.21.15.190:0/3154620480 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6045+0+0 (secure 0 0 0) 0x7f113004ee70 con 0x7f11400a36a0 2024-08-19T10:41:24.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1145761640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f112403be00 0x7f112403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:24.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1145761640 10 monclient: get_auth_request con 0x7f112403be00 auth_method 0 2024-08-19T10:41:24.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.050+0000 7f1145761640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f112403be00 0x7f112403e2c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f113c004820 tx=0x7f113c007b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:24.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.054+0000 7f1135ffb640 1 -- 172.21.15.190:0/3154620480 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1140073970 con 0x7f11400a36a0 2024-08-19T10:41:24.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.054+0000 7f1135ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:24.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.054+0000 7f1135ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:24.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.222+0000 7f11479c5640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-08-19T10:41:24.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.222+0000 7f11479c5640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-08-19T10:41:24.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.222+0000 7f11479c5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:24.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.222+0000 7f11479c5640 1 -- 172.21.15.190:0/3154620480 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]} v 0) -- 0x7f11400a3a80 con 0x7f11400a36a0 2024-08-19T10:41:24.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.914+0000 7f1135ffb640 1 -- 172.21.15.190:0/3154620480 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}]=0 added bucket host-for-test type host to location {rack=rack-for-test,root=root-for-test} v88) ==== 251+0+0 (secure 0 0 0) 0x7f113001ac60 con 0x7f11400a36a0 2024-08-19T10:41:24.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.914+0000 7f1135ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-08-19T10:41:24.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:24.914+0000 7f1135ffb640 10 monclient: _finish_command 2 = system:0 added bucket host-for-test type host to location {rack=rack-for-test,root=root-for-test} 2024-08-19T10:41:25.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.014+0000 7f11479c5640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-08-19T10:41:25.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.014+0000 7f11479c5640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-08-19T10:41:25.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.014+0000 7f11479c5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:25.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.014+0000 7f11479c5640 1 -- 172.21.15.190:0/3154620480 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]} v 0) -- 0x7f11400008e0 con 0x7f11400a36a0 2024-08-19T10:41:25.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f1135ffb640 1 -- 172.21.15.190:0/3154620480 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}]=0 bucket 'host-for-test' already exists v88) ==== 200+0+0 (secure 0 0 0) 0x7f113004d380 con 0x7f11400a36a0 2024-08-19T10:41:25.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f1135ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "host-for-test", "type": "host", "args": ["root=root-for-test", "rack=rack-for-test"]}] 2024-08-19T10:41:25.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f1135ffb640 10 monclient: _finish_command 3 = system:0 bucket 'host-for-test' already exists 2024-08-19T10:41:25.020 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'host-for-test' already exists 2024-08-19T10:41:25.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 -- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f112403be00 msgr2=0x7f112403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:25.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f112403be00 0x7f112403e2c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f113c004820 tx=0x7f113c007b70 comp rx=0 tx=0).stop 2024-08-19T10:41:25.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 10 monclient: shutdown 2024-08-19T10:41:25.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:25.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 -- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11400a36a0 msgr2=0x7f114013e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:25.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11400a36a0 0x7f114013e8d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f113000c450 tx=0x7f113000c910 comp rx=0 tx=0).stop 2024-08-19T10:41:25.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 -- 172.21.15.190:0/3154620480 shutdown_connections 2024-08-19T10:41:25.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f112403be00 0x7f112403e2c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11400a4190 0x7f1140124450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11400a36a0 0x7f114013e8d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 --2- 172.21.15.190:0/3154620480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11400a19d0 0x7f1140126a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 -- 172.21.15.190:0/3154620480 >> 172.21.15.190:0/3154620480 conn(0x7f11400a4790 msgr2=0x7f11400a6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:25.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 -- 172.21.15.190:0/3154620480 shutdown_connections 2024-08-19T10:41:25.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.018+0000 7f11479c5640 1 -- 172.21.15.190:0/3154620480 wait complete. 2024-08-19T10:41:25.037 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:25.037 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep host-for-test 2024-08-19T10:41:25.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.146+0000 7f3e657d1640 10 monclient: get_monmap_and_config 2024-08-19T10:41:25.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.146+0000 7f3e657d1640 10 monclient: build_initial_monmap 2024-08-19T10:41:25.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.146+0000 7f3e657d1640 10 monclient: monmap: 2024-08-19T10:41:25.151 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:25.151 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:25.151 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:25.151163+0000 2024-08-19T10:41:25.151 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:25.151163+0000 2024-08-19T10:41:25.151 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:25.151 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.151 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:25.151 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:25.152 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:25.152 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:25.152 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:25.152 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:25.152 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:25.152 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:25.152 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:25.152 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:25.152 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:25.152 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 1 Processor -- start 2024-08-19T10:41:25.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 1 -- start start 2024-08-19T10:41:25.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 10 monclient: init 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 10 monclient: _add_conns ranks=[4,7,5,8,3,0,10,2,6,9,1] 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e600a36a0 0x7f3e600a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 10 monclient(hunting): picked mon.noname-c con 0x7f3e600a36a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3e600a3fc0 0x7f3e60136540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 10 monclient(hunting): picked mon.noname-i con 0x7f3e600a3fc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3e60136a80 0x7f3e60136ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 10 monclient(hunting): picked mon.noname-h con 0x7f3e60136a80 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3e600a09d0 con 0x7f3e600a36a0 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:25.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f3e6009e550 con 0x7f3e60136a80 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f3e6009ffc0 con 0x7f3e600a3fc0 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e657d1640 10 monclient(hunting): authenticate will time out at 1130.720983s 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e5f7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3e60136a80 0x7f3e60136ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3e600a3fc0 0x7f3e60136540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3e600a3fc0 0x7f3e60136540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:44146/0 (socket says 172.21.15.190:44146) 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e5f7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3e60136a80 0x7f3e60136ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:39612/0 (socket says 172.21.15.190:39612) 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 1 -- 172.21.15.190:0/2073315761 learned_addr learned my addr 172.21.15.190:0/2073315761 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 10 monclient(hunting): get_auth_request con 0x7f3e600a3fc0 auth_method 0 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:25.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e5f7fe640 10 monclient(hunting): get_auth_request con 0x7f3e60136a80 auth_method 0 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e5f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 10 monclient(hunting): handle_auth_done global_id 14425 payload 274 2024-08-19T10:41:25.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 1 -- 172.21.15.190:0/2073315761 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3e60136a80 msgr2=0x7f3e60136ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:25.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 1 --2- 172.21.15.190:0/2073315761 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3e60136a80 0x7f3e60136ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 1 -- 172.21.15.190:0/2073315761 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e600a36a0 msgr2=0x7f3e600a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:25.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 1 --2- 172.21.15.190:0/2073315761 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e600a36a0 0x7f3e600a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 10 monclient: _finish_hunting 0 2024-08-19T10:41:25.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 1 monclient: found mon.noname-i 2024-08-19T10:41:25.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:25.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:25.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e567fc640 1 -- 172.21.15.190:0/2073315761 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6009ecc0 con 0x7f3e600a3fc0 2024-08-19T10:41:25.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.150+0000 7f3e5f7fe640 1 --2- 172.21.15.190:0/2073315761 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3e60136a80 0x7f3e60136ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:25.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e567fc640 1 -- 172.21.15.190:0/2073315761 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3e600a3fc0 msgr2=0x7f3e60136540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e567fc640 1 --2- 172.21.15.190:0/2073315761 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3e600a3fc0 0x7f3e60136540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009900 tx=0x7f3e4801c4b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=886f04463f4965b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 1 -- 172.21.15.190:0/2073315761 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e480b0070 con 0x7f3e600a3fc0 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 10 monclient: dump: 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:25.165 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:25.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 1 -- 172.21.15.190:0/2073315761 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e480aa810 con 0x7f3e600a3fc0 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 5 monclient: authenticate success, global_id 14425 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 1 -- 172.21.15.190:0/2073315761 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e480aae30 con 0x7f3e600a3fc0 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 4 set_mon_vals no callback set 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e5e7fc640 10 monclient: dump: 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.167 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 10 monclient: shutdown 2024-08-19T10:41:25.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 1 -- 172.21.15.190:0/2073315761 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3e600a3fc0 msgr2=0x7f3e60136540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 1 --2- 172.21.15.190:0/2073315761 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3e600a3fc0 0x7f3e60136540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009900 tx=0x7f3e4801c4b0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 1 -- 172.21.15.190:0/2073315761 shutdown_connections 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 1 --2- 172.21.15.190:0/2073315761 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3e60136a80 0x7f3e60136ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 1 --2- 172.21.15.190:0/2073315761 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3e600a3fc0 0x7f3e60136540 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 1 --2- 172.21.15.190:0/2073315761 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3e600a36a0 0x7f3e600a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 1 -- 172.21.15.190:0/2073315761 >> 172.21.15.190:0/2073315761 conn(0x7f3e600a4790 msgr2=0x7f3e600a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 1 -- 172.21.15.190:0/2073315761 shutdown_connections 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 1 -- 172.21.15.190:0/2073315761 wait complete. 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 10 monclient: build_initial_monmap 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 10 monclient: monmap: 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:25.169 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 1 Processor -- start 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 1 -- start start 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.154+0000 7f3e657d1640 10 monclient: init 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 10 monclient: _add_conns ranks=[3,2,1,4,6,8,7,5,0] 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e600a36a0 0x7f3e601d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 10 monclient(hunting): picked mon.noname-g con 0x7f3e600a36a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e600a3fc0 0x7f3e601d2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 10 monclient(hunting): picked mon.noname-b con 0x7f3e600a3fc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60136a80 0x7f3e6013b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 10 monclient(hunting): picked mon.noname-f con 0x7f3e60136a80 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3e600a5510 con 0x7f3e60136a80 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3e600a43a0 con 0x7f3e600a3fc0 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3e600a70a0 con 0x7f3e600a36a0 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e657d1640 10 monclient(hunting): authenticate will time out at 1130.728227s 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5effd640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e600a36a0 msgr2=0x7f3e601d1b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e600a36a0 0x7f3e601d1b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60136a80 0x7f3e6013b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60136a80 0x7f3e6013b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56842/0 (socket says 172.21.15.190:56842) 2024-08-19T10:41:25.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 1 -- 172.21.15.190:0/2771943894 learned_addr learned my addr 172.21.15.190:0/2771943894 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 10 monclient(hunting): get_auth_request con 0x7f3e60136a80 auth_method 0 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e567fc640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e600a3fc0 0x7f3e601d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 10 monclient(hunting): handle_auth_done global_id 14473 payload 824 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 1 -- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e600a36a0 msgr2=0x7f3e601d1b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e600a36a0 0x7f3e601d1b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 1 -- 172.21.15.190:0/2771943894 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e600a3fc0 msgr2=0x7f3e601d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:25.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e600a3fc0 0x7f3e601d2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:25.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 1 monclient: found mon.noname-f 2024-08-19T10:41:25.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:25.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:25.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 1 -- 172.21.15.190:0/2771943894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e600a75f0 con 0x7f3e60136a80 2024-08-19T10:41:25.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e567fc640 10 monclient: get_auth_request con 0x7f3e600a3fc0 auth_method 0 2024-08-19T10:41:25.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e567fc640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e600a3fc0 0x7f3e601d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:25.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 1 -- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60136a80 msgr2=0x7f3e6013b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:25.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e5f7fe640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60136a80 0x7f3e6013b260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800ef90 tx=0x7f3e5800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:25.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.158+0000 7f3e57fff640 1 -- 172.21.15.190:0/2771943894 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:41:25.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 1 -- 172.21.15.190:0/2771943894 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e58018020 con 0x7f3e60136a80 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 10 monclient: dump: 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:25.175 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 1 -- 172.21.15.190:0/2771943894 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e580148f0 con 0x7f3e60136a80 2024-08-19T10:41:25.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 1 -- 172.21.15.190:0/2771943894 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3e580133e0 con 0x7f3e60136a80 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 10 monclient: dump: 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:25.177 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e657d1640 5 monclient: authenticate success, global_id 14473 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e657d1640 10 monclient: _renew_subs 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e5dffb640 4 set_mon_vals no callback set 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e657d1640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:25.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e657d1640 1 -- 172.21.15.190:0/2771943894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e60122890 con 0x7f3e60136a80 2024-08-19T10:41:25.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e657d1640 10 monclient: _renew_subs 2024-08-19T10:41:25.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e657d1640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:25.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e657d1640 1 -- 172.21.15.190:0/2771943894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e60136f50 con 0x7f3e60136a80 2024-08-19T10:41:25.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e657d1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:25.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e657d1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:25.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e657d1640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:25.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e657d1640 1 -- 172.21.15.190:0/2771943894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e6009eee0 con 0x7f3e60136a80 2024-08-19T10:41:25.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 1 -- 172.21.15.190:0/2771943894 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3e5801b020 con 0x7f3e60136a80 2024-08-19T10:41:25.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3e4003be00 0x7f3e4003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e57fff640 1 -- 172.21.15.190:0/2771943894 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6570+0+0 (secure 0 0 0) 0x7f3e5804fff0 con 0x7f3e60136a80 2024-08-19T10:41:25.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e5effd640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3e4003be00 0x7f3e4003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:25.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.162+0000 7f3e5effd640 10 monclient: get_auth_request con 0x7f3e4003be00 auth_method 0 2024-08-19T10:41:25.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.166+0000 7f3e5effd640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3e4003be00 0x7f3e4003e2c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e50000f10 tx=0x7f3e50007bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:25.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.166+0000 7f3e57fff640 1 -- 172.21.15.190:0/2771943894 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3e58022020 con 0x7f3e60136a80 2024-08-19T10:41:25.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.166+0000 7f3e57fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:25.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.166+0000 7f3e57fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:25.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.342+0000 7f3e657d1640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:25.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.342+0000 7f3e657d1640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:25.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.342+0000 7f3e657d1640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:25.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.342+0000 7f3e657d1640 1 -- 172.21.15.190:0/2771943894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f3e60136540 con 0x7f3e60136a80 2024-08-19T10:41:25.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.342+0000 7f3e57fff640 1 -- 172.21.15.190:0/2771943894 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v89) ==== 56+0+1500 (secure 0 0 0) 0x7f3e600a75f0 con 0x7f3e60136a80 2024-08-19T10:41:25.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.342+0000 7f3e57fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:25.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.342+0000 7f3e57fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:25.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.442+0000 7f3e657d1640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:25.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.442+0000 7f3e657d1640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:25.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.442+0000 7f3e657d1640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:25.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.442+0000 7f3e657d1640 1 -- 172.21.15.190:0/2771943894 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f3e6013f640 con 0x7f3e60136a80 2024-08-19T10:41:25.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.442+0000 7f3e57fff640 1 -- 172.21.15.190:0/2771943894 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v89) ==== 56+0+1500 (secure 0 0 0) 0x7f3e58010040 con 0x7f3e60136a80 2024-08-19T10:41:25.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.442+0000 7f3e57fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:25.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.442+0000 7f3e57fff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:25.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 -- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3e4003be00 msgr2=0x7f3e4003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:25.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3e4003be00 0x7f3e4003e2c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e50000f10 tx=0x7f3e50007bd0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 10 monclient: shutdown 2024-08-19T10:41:25.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:25.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 -- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60136a80 msgr2=0x7f3e6013b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:25.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60136a80 0x7f3e6013b260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800ef90 tx=0x7f3e5800c490 comp rx=0 tx=0).stop 2024-08-19T10:41:25.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 -- 172.21.15.190:0/2771943894 shutdown_connections 2024-08-19T10:41:25.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3e4003be00 0x7f3e4003e2c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60136a80 0x7f3e6013b260 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3e600a3fc0 0x7f3e601d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 --2- 172.21.15.190:0/2771943894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e600a36a0 0x7f3e601d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 -- 172.21.15.190:0/2771943894 >> 172.21.15.190:0/2771943894 conn(0x7f3e600a4790 msgr2=0x7f3e6013cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:25.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 -- 172.21.15.190:0/2771943894 shutdown_connections 2024-08-19T10:41:25.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.446+0000 7f3e657d1640 1 -- 172.21.15.190:0/2771943894 wait complete. 2024-08-19T10:41:25.463 INFO:tasks.workunit.client.0.smithi190.stdout:-11 0 host host-for-test 2024-08-19T10:41:25.464 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:25.464 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep rack-for-test 2024-08-19T10:41:25.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.574+0000 7f1656fe6640 10 monclient: get_monmap_and_config 2024-08-19T10:41:25.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.574+0000 7f1656fe6640 10 monclient: build_initial_monmap 2024-08-19T10:41:25.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.574+0000 7f1656fe6640 10 monclient: monmap: 2024-08-19T10:41:25.579 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:25.579 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:25.579 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:25.579235+0000 2024-08-19T10:41:25.579 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:25.579235+0000 2024-08-19T10:41:25.579 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:25.579 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.579 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:25.580 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:25.580 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:25.580 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:25.580 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:25.580 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:25.580 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:25.580 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:25.580 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:25.580 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:25.580 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:25.580 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 1 Processor -- start 2024-08-19T10:41:25.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 1 -- start start 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 10 monclient: init 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 10 monclient: _add_conns ranks=[4,1,0,10,6,3,7,2,8,5,9] 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16501399f0 0x7f16500a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 10 monclient(hunting): picked mon.noname-c con 0x7f16501399f0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1650139dd0 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f165013a1b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16500a7700 con 0x7f165013a1b0 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16500a8340 con 0x7f1650139dd0 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f16500a4eb0 con 0x7f16501399f0 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:25.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1656fe6640 10 monclient(hunting): authenticate will time out at 1131.149263s 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f164ffff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f1650139dd0 0x7f16500a68e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:34044/0 (socket says 172.21.15.190:34044) 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f164ffff640 1 -- 172.21.15.190:0/1616290959 learned_addr learned my addr 172.21.15.190:0/1616290959 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 1 --2- 172.21.15.190:0/1616290959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16501399f0 0x7f16500a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 10 monclient(hunting): get_auth_request con 0x7f16501399f0 auth_method 0 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 10 monclient(hunting): handle_auth_done global_id 5278 payload 274 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 1 -- 172.21.15.190:0/1616290959 >> v1:172.21.15.190:6789/0 conn(0x7f1650139dd0 legacy=0x7f16500a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:25.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 1 -- 172.21.15.190:0/1616290959 >> v1:172.21.15.89:6789/0 conn(0x7f165013a1b0 legacy=0x7f16500a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:25.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 10 monclient: _finish_hunting 0 2024-08-19T10:41:25.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 1 monclient: found mon.noname-c 2024-08-19T10:41:25.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:25.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:25.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.578+0000 7f1654d82640 1 -- 172.21.15.190:0/1616290959 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16500a4580 con 0x7f16501399f0 2024-08-19T10:41:25.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f1654d82640 1 -- 172.21.15.190:0/1616290959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16501399f0 msgr2=0x7f16500a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:25.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f1654d82640 1 --2- 172.21.15.190:0/1616290959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16501399f0 0x7f16500a5640 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1640009dc0 tx=0x7f164001c850 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f210b9a107d6d5f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:25.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 1 -- 172.21.15.190:0/1616290959 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16400a6a20 con 0x7f16501399f0 2024-08-19T10:41:25.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:25.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:25.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 10 monclient: dump: 2024-08-19T10:41:25.586 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:25.586 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:25.586 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.586 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:25.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 1 -- 172.21.15.190:0/1616290959 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16400a6be0 con 0x7f16501399f0 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 1 -- 172.21.15.190:0/1616290959 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16400a6dd0 con 0x7f16501399f0 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f164f7fe640 10 monclient: dump: 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.588 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f1656fe6640 5 monclient: authenticate success, global_id 5278 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f1656fe6640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:25.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f1656fe6640 4 set_mon_vals no callback set 2024-08-19T10:41:25.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f1656fe6640 10 monclient: shutdown 2024-08-19T10:41:25.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f1656fe6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:25.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f1656fe6640 1 -- 172.21.15.190:0/1616290959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16501399f0 msgr2=0x7f16500a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:25.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.582+0000 7f1656fe6640 1 --2- 172.21.15.190:0/1616290959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16501399f0 0x7f16500a5640 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1640009dc0 tx=0x7f164001c850 comp rx=0 tx=0).stop 2024-08-19T10:41:25.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 -- 172.21.15.190:0/1616290959 shutdown_connections 2024-08-19T10:41:25.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 --2- 172.21.15.190:0/1616290959 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f16501399f0 0x7f16500a5640 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 -- 172.21.15.190:0/1616290959 >> 172.21.15.190:0/1616290959 conn(0x7f1650074c10 msgr2=0x7f16500abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 -- 172.21.15.190:0/1616290959 shutdown_connections 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 -- 172.21.15.190:0/1616290959 wait complete. 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient: build_initial_monmap 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient: monmap: 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:25.595 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 Processor -- start 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 -- start start 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient: init 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient: _add_conns ranks=[7,4,0,8,6,1,5,3,2] 2024-08-19T10:41:25.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f16501399f0 0x7f1650132a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient(hunting): picked mon.noname-i con 0x7f16501399f0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1650139dd0 0x7f1650132f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient(hunting): picked mon.noname-c con 0x7f1650139dd0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f165013a1b0 0x7f16501374d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient(hunting): picked mon.noname-a con 0x7f165013a1b0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1650145540 con 0x7f165013a1b0 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f16500a3fd0 con 0x7f1650139dd0 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f16501456c0 con 0x7f16501399f0 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:25.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1656fe6640 10 monclient(hunting): authenticate will time out at 1131.157967s 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f16501399f0 0x7f1650132a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f16501399f0 0x7f1650132a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:46186/0 (socket says 172.21.15.190:46186) 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 1 -- 172.21.15.190:0/2320375237 learned_addr learned my addr 172.21.15.190:0/2320375237 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 10 monclient(hunting): get_auth_request con 0x7f16501399f0 auth_method 0 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1655583640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f165013a1b0 0x7f16501374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:25.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1655583640 10 monclient(hunting): get_auth_request con 0x7f165013a1b0 auth_method 0 2024-08-19T10:41:25.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f164ffff640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1650139dd0 0x7f1650132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:25.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1655583640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:25.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1655583640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:25.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1655583640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:25.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f164ffff640 10 monclient(hunting): get_auth_request con 0x7f1650139dd0 auth_method 0 2024-08-19T10:41:25.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f164ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:25.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f164ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:25.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f164ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:25.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 10 monclient(hunting): handle_auth_done global_id 14434 payload 824 2024-08-19T10:41:25.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 1 -- 172.21.15.190:0/2320375237 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1650139dd0 msgr2=0x7f1650132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:25.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1650139dd0 0x7f1650132f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 1 -- 172.21.15.190:0/2320375237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f165013a1b0 msgr2=0x7f16501374d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f165013a1b0 0x7f16501374d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 10 monclient: _finish_hunting 0 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 1 monclient: found mon.noname-i 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.586+0000 7f1654d82640 1 -- 172.21.15.190:0/2320375237 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16500ab540 con 0x7f16501399f0 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1654d82640 1 -- 172.21.15.190:0/2320375237 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f16501399f0 msgr2=0x7f1650132a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1654d82640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f16501399f0 0x7f1650132a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16400b0040 tx=0x7f16400195f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 1 -- 172.21.15.190:0/2320375237 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16400bd070 con 0x7f16501399f0 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:25.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 10 monclient: dump: 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:25.601 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1656fe6640 5 monclient: authenticate success, global_id 14434 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1656fe6640 10 monclient: _renew_subs 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 1 -- 172.21.15.190:0/2320375237 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1640003dc0 con 0x7f16501399f0 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1656fe6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1656fe6640 1 -- 172.21.15.190:0/2320375237 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165013bdf0 con 0x7f16501399f0 2024-08-19T10:41:25.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 1 -- 172.21.15.190:0/2320375237 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f16400b9430 con 0x7f16501399f0 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164effd640 4 set_mon_vals no callback set 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 10 monclient: dump: 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:25.603 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1656fe6640 10 monclient: _renew_subs 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1656fe6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1656fe6640 1 -- 172.21.15.190:0/2320375237 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f165013a8b0 con 0x7f16501399f0 2024-08-19T10:41:25.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1656fe6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:25.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1656fe6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:25.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1656fe6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:25.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 1 -- 172.21.15.190:0/2320375237 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f16400c0070 con 0x7f16501399f0 2024-08-19T10:41:25.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f1656fe6640 1 -- 172.21.15.190:0/2320375237 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16500a5640 con 0x7f16501399f0 2024-08-19T10:41:25.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f163003be00 0x7f163003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:25.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164d7fa640 1 -- 172.21.15.190:0/2320375237 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(89..89 src has 1..89) ==== 6570+0+0 (secure 0 0 0) 0x7f16400b4040 con 0x7f16501399f0 2024-08-19T10:41:25.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164ffff640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f163003be00 0x7f163003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:25.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.590+0000 7f164ffff640 10 monclient: get_auth_request con 0x7f163003be00 auth_method 0 2024-08-19T10:41:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.594+0000 7f164ffff640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f163003be00 0x7f163003e2c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1638007fd0 tx=0x7f1638009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.594+0000 7f164d7fa640 1 -- 172.21.15.190:0/2320375237 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f16500ab540 con 0x7f16501399f0 2024-08-19T10:41:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.594+0000 7f164d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:25.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.594+0000 7f164d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:25.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.766+0000 7f1656fe6640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:25.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.766+0000 7f1656fe6640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:25.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.766+0000 7f1656fe6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:25.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.766+0000 7f1656fe6640 1 -- 172.21.15.190:0/2320375237 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f16500ab010 con 0x7f16501399f0 2024-08-19T10:41:25.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.766+0000 7f164d7fa640 1 -- 172.21.15.190:0/2320375237 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v89) ==== 56+0+1500 (secure 0 0 0) 0x7f1640016560 con 0x7f16501399f0 2024-08-19T10:41:25.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.766+0000 7f164d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:25.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.766+0000 7f164d7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:25.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.866+0000 7f1656fe6640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:25.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.866+0000 7f1656fe6640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:25.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.866+0000 7f1656fe6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:25.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.866+0000 7f1656fe6640 1 -- 172.21.15.190:0/2320375237 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f16500a3500 con 0x7f16501399f0 2024-08-19T10:41:25.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f164d7fa640 1 -- 172.21.15.190:0/2320375237 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v89) ==== 56+0+1500 (secure 0 0 0) 0x7f16400e8a00 con 0x7f16501399f0 2024-08-19T10:41:25.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f164d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:25.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f164d7fa640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:25.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 -- 172.21.15.190:0/2320375237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f163003be00 msgr2=0x7f163003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:25.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f163003be00 0x7f163003e2c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1638007fd0 tx=0x7f1638009040 comp rx=0 tx=0).stop 2024-08-19T10:41:25.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 10 monclient: shutdown 2024-08-19T10:41:25.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:25.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 -- 172.21.15.190:0/2320375237 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f16501399f0 msgr2=0x7f1650132a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:25.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f16501399f0 0x7f1650132a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16400b0040 tx=0x7f16400195f0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 -- 172.21.15.190:0/2320375237 shutdown_connections 2024-08-19T10:41:25.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f163003be00 0x7f163003e2c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f165013a1b0 0x7f16501374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1650139dd0 0x7f1650132f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 --2- 172.21.15.190:0/2320375237 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f16501399f0 0x7f1650132a20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:25.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 -- 172.21.15.190:0/2320375237 >> 172.21.15.190:0/2320375237 conn(0x7f1650074c10 msgr2=0x7f165009d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:25.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 -- 172.21.15.190:0/2320375237 shutdown_connections 2024-08-19T10:41:25.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:25.870+0000 7f1656fe6640 1 -- 172.21.15.190:0/2320375237 wait complete. 2024-08-19T10:41:25.888 INFO:tasks.workunit.client.0.smithi190.stdout:-12 0 rack rack-for-test 2024-08-19T10:41:25.889 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:25.889 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep root-for-test 2024-08-19T10:41:26.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient: get_monmap_and_config 2024-08-19T10:41:26.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient: build_initial_monmap 2024-08-19T10:41:26.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient: monmap: 2024-08-19T10:41:26.003 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:26.003781+0000 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:26.003781+0000 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:26.004 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:26.005 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:26.005 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:26.005 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 1 Processor -- start 2024-08-19T10:41:26.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 1 -- start start 2024-08-19T10:41:26.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient: init 2024-08-19T10:41:26.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:26.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient: _add_conns ranks=[2,9,7,0,5,4,1,8,10,6,3] 2024-08-19T10:41:26.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84c41352b0 0x7f84c409eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient(hunting): picked mon.noname-b con 0x7f84c41352b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:26.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f84c4135690 0x7f84c40a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient(hunting): picked mon.noname-a con 0x7f84c4135690 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:26.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c40a3470 0x7f84c40a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient(hunting): picked mon.noname-i con 0x7f84c40a3470 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f84c409d880 con 0x7f84c4135690 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f84c409db60 con 0x7f84c41352b0 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f84c409de40 con 0x7f84c40a3470 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84cb617640 10 monclient(hunting): authenticate will time out at 1131.573906s 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c40a3470 0x7f84c40a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c40a3470 0x7f84c40a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:46194/0 (socket says 172.21.15.190:46194) 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 1 -- 172.21.15.190:0/1519891386 learned_addr learned my addr 172.21.15.190:0/1519891386 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:26.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 10 monclient(hunting): get_auth_request con 0x7f84c40a3470 auth_method 0 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c93b3640 1 --2- 172.21.15.190:0/1519891386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84c41352b0 0x7f84c409eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c8bb2640 1 --2- 172.21.15.190:0/1519891386 >> v2:172.21.15.89:3300/0 conn(0x7f84c4135690 0x7f84c40a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c8bb2640 10 monclient(hunting): get_auth_request con 0x7f84c4135690 auth_method 0 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c8bb2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c8bb2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c8bb2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:26.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c93b3640 10 monclient(hunting): get_auth_request con 0x7f84c41352b0 auth_method 0 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c93b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c93b3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c93b3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 10 monclient(hunting): handle_auth_done global_id 14443 payload 274 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 1 -- 172.21.15.190:0/1519891386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84c41352b0 msgr2=0x7f84c409eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 1 --2- 172.21.15.190:0/1519891386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84c41352b0 0x7f84c409eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 1 -- 172.21.15.190:0/1519891386 >> v2:172.21.15.89:3300/0 conn(0x7f84c4135690 msgr2=0x7f84c40a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 1 --2- 172.21.15.190:0/1519891386 >> v2:172.21.15.89:3300/0 conn(0x7f84c4135690 0x7f84c40a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 10 monclient: _finish_hunting 0 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 1 monclient: found mon.noname-i 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c9bb4640 1 -- 172.21.15.190:0/1519891386 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c40a4af0 con 0x7f84c40a3470 2024-08-19T10:41:26.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.002+0000 7f84c8bb2640 1 --2- 172.21.15.190:0/1519891386 >> v2:172.21.15.89:3300/0 conn(0x7f84c4135690 0x7f84c40a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:41:26.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84c9bb4640 1 -- 172.21.15.190:0/1519891386 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c40a3470 msgr2=0x7f84c40a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:26.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84c9bb4640 1 --2- 172.21.15.190:0/1519891386 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c40a3470 0x7f84c40a38f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009870 tx=0x7f84b401c4b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=b60fb22528b57748 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:26.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 1 -- 172.21.15.190:0/1519891386 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84b40b0070 con 0x7f84c40a3470 2024-08-19T10:41:26.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:26.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 10 monclient: dump: 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:26.015 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 10 monclient: _finish_auth 0 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 1 -- 172.21.15.190:0/1519891386 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b40aa810 con 0x7f84c40a3470 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84cb617640 5 monclient: authenticate success, global_id 14443 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84cb617640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:26.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84cb617640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 1 -- 172.21.15.190:0/1519891386 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84b40aae30 con 0x7f84c40a3470 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84cb617640 4 set_mon_vals no callback set 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.006+0000 7f84bbfff640 10 monclient: dump: 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.017 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient: shutdown 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 -- 172.21.15.190:0/1519891386 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c40a3470 msgr2=0x7f84c40a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:26.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 --2- 172.21.15.190:0/1519891386 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c40a3470 0x7f84c40a38f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009870 tx=0x7f84b401c4b0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 -- 172.21.15.190:0/1519891386 shutdown_connections 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 --2- 172.21.15.190:0/1519891386 >> v2:172.21.15.89:3300/0 conn(0x7f84c4135690 0x7f84c40a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 --2- 172.21.15.190:0/1519891386 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c40a3470 0x7f84c40a38f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 --2- 172.21.15.190:0/1519891386 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84c41352b0 0x7f84c409eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 -- 172.21.15.190:0/1519891386 >> 172.21.15.190:0/1519891386 conn(0x7f84c40a84b0 msgr2=0x7f84c40a88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 -- 172.21.15.190:0/1519891386 shutdown_connections 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 -- 172.21.15.190:0/1519891386 wait complete. 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient: build_initial_monmap 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient: monmap: 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:26.019 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:26.020 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 Processor -- start 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 -- start start 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient: init 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient: _add_conns ranks=[2,7,0,1,8,3,4,6,5] 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84c40a3470 0x7f84c4134500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient(hunting): picked mon.noname-b con 0x7f84c40a3470 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c41352b0 0x7f84c4139370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient(hunting): picked mon.noname-i con 0x7f84c41352b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c4135690 0x7f84c41398b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient(hunting): picked mon.noname-a con 0x7f84c4135690 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f84c4142f30 con 0x7f84c4135690 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f84c4135c80 con 0x7f84c40a3470 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f84c41430b0 con 0x7f84c41352b0 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84cb617640 10 monclient(hunting): authenticate will time out at 1131.582004s 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c41352b0 0x7f84c4139370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c41352b0 0x7f84c4139370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:46208/0 (socket says 172.21.15.190:46208) 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 1 -- 172.21.15.190:0/1951579400 learned_addr learned my addr 172.21.15.190:0/1951579400 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 10 monclient(hunting): get_auth_request con 0x7f84c41352b0 auth_method 0 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:26.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c93b3640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84c40a3470 0x7f84c4134500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c9bb4640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c4135690 0x7f84c41398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 10 monclient(hunting): handle_auth_done global_id 14452 payload 824 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 1 -- 172.21.15.190:0/1951579400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84c40a3470 msgr2=0x7f84c4134500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84c40a3470 0x7f84c4134500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 1 -- 172.21.15.190:0/1951579400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c4135690 msgr2=0x7f84c41398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c4135690 0x7f84c41398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 10 monclient: _finish_hunting 0 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 1 monclient: found mon.noname-i 2024-08-19T10:41:26.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:26.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c8bb2640 1 -- 172.21.15.190:0/1951579400 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c413ac30 con 0x7f84c41352b0 2024-08-19T10:41:26.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c93b3640 10 monclient: get_auth_request con 0x7f84c40a3470 auth_method 0 2024-08-19T10:41:26.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c93b3640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84c40a3470 0x7f84c4134500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:26.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c9bb4640 10 monclient: get_auth_request con 0x7f84c4135690 auth_method 0 2024-08-19T10:41:26.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.010+0000 7f84c9bb4640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c4135690 0x7f84c41398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:26.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84c8bb2640 1 -- 172.21.15.190:0/1951579400 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c41352b0 msgr2=0x7f84c4139370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:26.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84c8bb2640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c41352b0 0x7f84c4139370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84c0007ef0 tx=0x7f84c000e4b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 1 -- 172.21.15.190:0/1951579400 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84c0013a00 con 0x7f84c41352b0 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 10 monclient: dump: 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:26.027 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 1 -- 172.21.15.190:0/1951579400 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c0019070 con 0x7f84c41352b0 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:26.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 1 -- 172.21.15.190:0/1951579400 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84c0013be0 con 0x7f84c41352b0 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84bb7fe640 4 set_mon_vals no callback set 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 10 monclient: dump: 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:26.029 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84cb617640 5 monclient: authenticate success, global_id 14452 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84cb617640 10 monclient: _renew_subs 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84cb617640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84cb617640 1 -- 172.21.15.190:0/1951579400 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c413c6f0 con 0x7f84c41352b0 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84cb617640 10 monclient: _renew_subs 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84cb617640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84cb617640 1 -- 172.21.15.190:0/1951579400 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c413b960 con 0x7f84c41352b0 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84cb617640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84cb617640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84cb617640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84cb617640 1 -- 172.21.15.190:0/1951579400 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c409eb20 con 0x7f84c41352b0 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 1 -- 172.21.15.190:0/1951579400 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f84c0013590 con 0x7f84c41352b0 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f84a803be00 0x7f84a803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84b9ffb640 1 -- 172.21.15.190:0/1951579400 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(89..89 src has 1..89) ==== 6570+0+0 (secure 0 0 0) 0x7f84c004fa60 con 0x7f84c41352b0 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84c93b3640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f84a803be00 0x7f84a803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.014+0000 7f84c93b3640 10 monclient: get_auth_request con 0x7f84a803be00 auth_method 0 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.018+0000 7f84c93b3640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f84a803be00 0x7f84a803e2c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0097c0 tx=0x7f84bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.018+0000 7f84b9ffb640 1 -- 172.21.15.190:0/1951579400 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f84c0013d80 con 0x7f84c41352b0 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.018+0000 7f84b9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:26.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.018+0000 7f84b9ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:26.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.186+0000 7f84cb617640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:26.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.186+0000 7f84cb617640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:26.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.186+0000 7f84cb617640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.186+0000 7f84cb617640 1 -- 172.21.15.190:0/1951579400 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f84c406e4d0 con 0x7f84c41352b0 2024-08-19T10:41:26.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.186+0000 7f84b9ffb640 1 -- 172.21.15.190:0/1951579400 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v89) ==== 56+0+1500 (secure 0 0 0) 0x7f84c001ec60 con 0x7f84c41352b0 2024-08-19T10:41:26.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.186+0000 7f84b9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:26.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.186+0000 7f84b9ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.278+0000 7f84cb617640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.278+0000 7f84cb617640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.278+0000 7f84cb617640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.278+0000 7f84cb617640 1 -- 172.21.15.190:0/1951579400 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f84c413d970 con 0x7f84c41352b0 2024-08-19T10:41:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.278+0000 7f84b9ffb640 1 -- 172.21.15.190:0/1951579400 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v89) ==== 56+0+1500 (secure 0 0 0) 0x7f84c001c360 con 0x7f84c41352b0 2024-08-19T10:41:26.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.278+0000 7f84b9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:26.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.278+0000 7f84b9ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:26.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 -- 172.21.15.190:0/1951579400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f84a803be00 msgr2=0x7f84a803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:26.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f84a803be00 0x7f84a803e2c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0097c0 tx=0x7f84bc008040 comp rx=0 tx=0).stop 2024-08-19T10:41:26.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 10 monclient: shutdown 2024-08-19T10:41:26.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 -- 172.21.15.190:0/1951579400 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c41352b0 msgr2=0x7f84c4139370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c41352b0 0x7f84c4139370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84c0007ef0 tx=0x7f84c000e4b0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 -- 172.21.15.190:0/1951579400 shutdown_connections 2024-08-19T10:41:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f84a803be00 0x7f84a803e2c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84c4135690 0x7f84c41398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84c41352b0 0x7f84c4139370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 --2- 172.21.15.190:0/1951579400 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84c40a3470 0x7f84c4134500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 -- 172.21.15.190:0/1951579400 >> 172.21.15.190:0/1951579400 conn(0x7f84c40a84b0 msgr2=0x7f84c40a16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 -- 172.21.15.190:0/1951579400 shutdown_connections 2024-08-19T10:41:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.282+0000 7f84cb617640 1 -- 172.21.15.190:0/1951579400 wait complete. 2024-08-19T10:41:26.299 INFO:tasks.workunit.client.0.smithi190.stdout:-14 0 root root-for-test 2024-08-19T10:41:26.299 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm host-for-test 2024-08-19T10:41:26.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.410+0000 7fd6b580b640 10 monclient: get_monmap_and_config 2024-08-19T10:41:26.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.410+0000 7fd6b580b640 10 monclient: build_initial_monmap 2024-08-19T10:41:26.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.410+0000 7fd6b580b640 10 monclient: monmap: 2024-08-19T10:41:26.414 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:26.414 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:26.414724+0000 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:26.414724+0000 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:26.415 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:26.416 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:26.416 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:26.416 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.410+0000 7fd6b580b640 1 Processor -- start 2024-08-19T10:41:26.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.410+0000 7fd6b580b640 1 -- start start 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.410+0000 7fd6b580b640 10 monclient: init 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 10 monclient: _add_conns ranks=[4,5,7,0,2,8,9,3,10,6,1] 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b00a5940 0x7fd6b00720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 10 monclient(hunting): picked mon.noname-c con 0x7fd6b00a5940 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6b00a5d20 0x7fd6b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 10 monclient(hunting): picked mon.noname-h con 0x7fd6b00a5d20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 0x7fd6b0139b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 10 monclient(hunting): picked mon.noname-i con 0x7fd6b0139710 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd6b006df10 con 0x7fd6b00a5940 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd6b006efb0 con 0x7fd6b00a5d20 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd6b006f290 con 0x7fd6b0139710 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6b580b640 10 monclient(hunting): authenticate will time out at 1131.984609s 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6b00a5d20 0x7fd6b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6af7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 0x7fd6b0139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6b00a5d20 0x7fd6b00a6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48224/0 (socket says 172.21.15.190:48224) 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 1 -- 172.21.15.190:0/776809294 learned_addr learned my addr 172.21.15.190:0/776809294 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6af7fe640 10 monclient(hunting): get_auth_request con 0x7fd6b0139710 auth_method 0 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6af7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:26.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 10 monclient(hunting): get_auth_request con 0x7fd6b00a5d20 auth_method 0 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 10 monclient(hunting): handle_auth_done global_id 5243 payload 274 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6aeffd640 1 --2- 172.21.15.190:0/776809294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b00a5940 0x7fd6b00720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 1 -- 172.21.15.190:0/776809294 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 msgr2=0x7fd6b0139b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:26.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 1 --2- 172.21.15.190:0/776809294 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 0x7fd6b0139b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 1 -- 172.21.15.190:0/776809294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b00a5940 msgr2=0x7fd6b00720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:26.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 1 --2- 172.21.15.190:0/776809294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b00a5940 0x7fd6b00720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:41:26.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 1 monclient: found mon.noname-h 2024-08-19T10:41:26.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:26.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:26.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6ae7fc640 1 -- 172.21.15.190:0/776809294 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b00a73d0 con 0x7fd6b00a5d20 2024-08-19T10:41:26.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6aeffd640 10 monclient: get_auth_request con 0x7fd6b00a5940 auth_method 0 2024-08-19T10:41:26.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.414+0000 7fd6aeffd640 1 --2- 172.21.15.190:0/776809294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b00a5940 0x7fd6b00720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:26.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6ae7fc640 1 -- 172.21.15.190:0/776809294 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6b00a5d20 msgr2=0x7fd6b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:26.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6ae7fc640 1 --2- 172.21.15.190:0/776809294 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6b00a5d20 0x7fd6b00a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00098d0 tx=0x7fd6a001c440 comp rx=0 tx=0).ready entity=mon.5 client_cookie=12625f082710e4d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:26.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 1 -- 172.21.15.190:0/776809294 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6a00b0070 con 0x7fd6b00a5d20 2024-08-19T10:41:26.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:26.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:26.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 10 monclient: dump: 2024-08-19T10:41:26.425 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:26.425 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:26.426 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 5 monclient: authenticate success, global_id 5243 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 1 -- 172.21.15.190:0/776809294 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a00aa810 con 0x7fd6b00a5d20 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 4 set_mon_vals no callback set 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 1 -- 172.21.15.190:0/776809294 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6a00a9300 con 0x7fd6b00a5d20 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:26.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6adffb640 10 monclient: dump: 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:26.428 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 10 monclient: shutdown 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 1 -- 172.21.15.190:0/776809294 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6b00a5d20 msgr2=0x7fd6b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 1 --2- 172.21.15.190:0/776809294 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6b00a5d20 0x7fd6b00a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00098d0 tx=0x7fd6a001c440 comp rx=0 tx=0).stop 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 1 -- 172.21.15.190:0/776809294 shutdown_connections 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 1 --2- 172.21.15.190:0/776809294 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 0x7fd6b0139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 1 --2- 172.21.15.190:0/776809294 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6b00a5d20 0x7fd6b00a6100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 1 --2- 172.21.15.190:0/776809294 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd6b00a5940 0x7fd6b00720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 1 -- 172.21.15.190:0/776809294 >> 172.21.15.190:0/776809294 conn(0x7fd6b00aa530 msgr2=0x7fd6b00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:26.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 1 -- 172.21.15.190:0/776809294 shutdown_connections 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 1 -- 172.21.15.190:0/776809294 wait complete. 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 10 monclient: build_initial_monmap 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 10 monclient: monmap: 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:26.430 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 1 Processor -- start 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 1 -- start start 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.418+0000 7fd6b580b640 10 monclient: init 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient: _add_conns ranks=[1,3,7,0,8,6,5,2,4] 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b00a5940 0x7fd6b0120a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient(hunting): picked mon.noname-f con 0x7fd6b00a5940 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b00a5d20 0x7fd6b0138960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient(hunting): picked mon.noname-g con 0x7fd6b00a5d20 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 0x7fd6b01d1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient(hunting): picked mon.noname-i con 0x7fd6b0139710 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b006e7e0 con 0x7fd6b00a5940 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b006e660 con 0x7fd6b00a5d20 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd6b0143390 con 0x7fd6b0139710 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:26.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient(hunting): authenticate will time out at 1131.992139s 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6ae7fc640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b00a5d20 msgr2=0x7fd6b0138960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6ae7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b00a5d20 0x7fd6b0138960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6aeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b00a5940 0x7fd6b0120a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 0x7fd6b01d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6aeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b00a5940 0x7fd6b0120a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34448/0 (socket says 172.21.15.190:34448) 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6aeffd640 1 -- 172.21.15.190:0/785454277 learned_addr learned my addr 172.21.15.190:0/785454277 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6aeffd640 10 monclient(hunting): get_auth_request con 0x7fd6b00a5940 auth_method 0 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 0x7fd6b01d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:46224/0 (socket says 172.21.15.190:46224) 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6aeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 10 monclient(hunting): get_auth_request con 0x7fd6b0139710 auth_method 0 2024-08-19T10:41:26.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 10 monclient(hunting): handle_auth_done global_id 14470 payload 824 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 1 -- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b00a5d20 msgr2=0x7fd6b0138960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b00a5d20 0x7fd6b0138960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 1 -- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b00a5940 msgr2=0x7fd6b0120a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:26.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b00a5940 0x7fd6b0120a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:26.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:26.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 1 monclient: found mon.noname-i 2024-08-19T10:41:26.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:26.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 1 -- 172.21.15.190:0/785454277 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b0121fa0 con 0x7fd6b0139710 2024-08-19T10:41:26.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 1 -- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 msgr2=0x7fd6b01d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:26.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6af7fe640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 0x7fd6b01d1530 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800eba0 tx=0x7fd6a800c490 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:26.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 1 -- 172.21.15.190:0/785454277 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6a8018020 con 0x7fd6b0139710 2024-08-19T10:41:26.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:26.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:26.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 10 monclient: dump: 2024-08-19T10:41:26.437 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:26.437 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:26.438 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 1 -- 172.21.15.190:0/785454277 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a800ccd0 con 0x7fd6b0139710 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 5 monclient: authenticate success, global_id 14470 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient: _renew_subs 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 1 -- 172.21.15.190:0/785454277 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b00a9ac0 con 0x7fd6b0139710 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 1 -- 172.21.15.190:0/785454277 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6a800ce80 con 0x7fd6b0139710 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:26.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6ad7fa640 4 set_mon_vals no callback set 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6a7fff640 10 monclient: dump: 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:26.440 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient: _renew_subs 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 1 -- 172.21.15.190:0/785454277 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b0123d40 con 0x7fd6b0139710 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.422+0000 7fd6b580b640 1 -- 172.21.15.190:0/785454277 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b0139e70 con 0x7fd6b0139710 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.426+0000 7fd6a7fff640 1 -- 172.21.15.190:0/785454277 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd6a801b020 con 0x7fd6b0139710 2024-08-19T10:41:26.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.426+0000 7fd6a7fff640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd69003be00 0x7fd69003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:26.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.426+0000 7fd6a7fff640 1 -- 172.21.15.190:0/785454277 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(89..89 src has 1..89) ==== 6570+0+0 (secure 0 0 0) 0x7fd6a804fcd0 con 0x7fd6b0139710 2024-08-19T10:41:26.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.426+0000 7fd6aeffd640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd69003be00 0x7fd69003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:26.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.426+0000 7fd6aeffd640 10 monclient: get_auth_request con 0x7fd69003be00 auth_method 0 2024-08-19T10:41:26.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.426+0000 7fd6aeffd640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd69003be00 0x7fd69003e2c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd698006fd0 tx=0x7fd698008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:26.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.430+0000 7fd6a7fff640 1 -- 172.21.15.190:0/785454277 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd6a8022030 con 0x7fd6b0139710 2024-08-19T10:41:26.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.430+0000 7fd6a7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:26.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.430+0000 7fd6a7fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:26.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.594+0000 7fd6b580b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-08-19T10:41:26.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.594+0000 7fd6b580b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-08-19T10:41:26.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.594+0000 7fd6b580b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:26.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.594+0000 7fd6b580b640 1 -- 172.21.15.190:0/785454277 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "host-for-test"} v 0) -- 0x7fd6b006ff20 con 0x7fd6b0139710 2024-08-19T10:41:26.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.926+0000 7fd6a7fff640 1 -- 172.21.15.190:0/785454277 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host-for-test"}]=0 removed item id -11 name 'host-for-test' from crush map v90) ==== 140+0+0 (secure 0 0 0) 0x7fd6a8010040 con 0x7fd6b0139710 2024-08-19T10:41:26.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.926+0000 7fd6a7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-08-19T10:41:26.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:26.926+0000 7fd6a7fff640 10 monclient: _finish_command 2 = system:0 removed item id -11 name 'host-for-test' from crush map 2024-08-19T10:41:27.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.030+0000 7fd6b580b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-08-19T10:41:27.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.030+0000 7fd6b580b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-08-19T10:41:27.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.030+0000 7fd6b580b640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:27.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.030+0000 7fd6b580b640 1 -- 172.21.15.190:0/785454277 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "host-for-test"} v 0) -- 0x7fd6b0122dc0 con 0x7fd6b0139710 2024-08-19T10:41:27.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.030+0000 7fd6a7fff640 1 -- 172.21.15.190:0/785454277 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host-for-test"}]=0 device 'host-for-test' does not appear in the crush map v90) ==== 140+0+0 (secure 0 0 0) 0x7fd6a8007aa0 con 0x7fd6b0139710 2024-08-19T10:41:27.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.030+0000 7fd6a7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host-for-test"}] 2024-08-19T10:41:27.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.030+0000 7fd6a7fff640 10 monclient: _finish_command 3 = system:0 device 'host-for-test' does not appear in the crush map 2024-08-19T10:41:27.034 INFO:tasks.workunit.client.0.smithi190.stderr:device 'host-for-test' does not appear in the crush map 2024-08-19T10:41:27.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 -- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd69003be00 msgr2=0x7fd69003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:27.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd69003be00 0x7fd69003e2c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd698006fd0 tx=0x7fd698008040 comp rx=0 tx=0).stop 2024-08-19T10:41:27.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 10 monclient: shutdown 2024-08-19T10:41:27.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:27.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 -- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 msgr2=0x7fd6b01d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:27.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 0x7fd6b01d1530 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800eba0 tx=0x7fd6a800c490 comp rx=0 tx=0).stop 2024-08-19T10:41:27.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 -- 172.21.15.190:0/785454277 shutdown_connections 2024-08-19T10:41:27.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd69003be00 0x7fd69003e2c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:27.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6b0139710 0x7fd6b01d1530 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:27.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6b00a5d20 0x7fd6b0138960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:27.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 --2- 172.21.15.190:0/785454277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6b00a5940 0x7fd6b0120a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:27.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 -- 172.21.15.190:0/785454277 >> 172.21.15.190:0/785454277 conn(0x7fd6b00aa530 msgr2=0x7fd6b00a4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:27.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 -- 172.21.15.190:0/785454277 shutdown_connections 2024-08-19T10:41:27.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.034+0000 7fd6b580b640 1 -- 172.21.15.190:0/785454277 wait complete. 2024-08-19T10:41:27.050 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm rack-for-test 2024-08-19T10:41:27.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient: get_monmap_and_config 2024-08-19T10:41:27.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient: build_initial_monmap 2024-08-19T10:41:27.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient: monmap: 2024-08-19T10:41:27.168 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:27.169 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:27.169 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:27.168797+0000 2024-08-19T10:41:27.169 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:27.168797+0000 2024-08-19T10:41:27.169 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:27.169 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:27.169 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:27.169 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:27.169 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:27.169 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:27.170 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:27.170 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:27.170 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:27.170 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:27.170 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:27.170 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:27.170 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:27.170 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:27.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 1 Processor -- start 2024-08-19T10:41:27.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 1 -- start start 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient: init 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient: _add_conns ranks=[2,5,0,7,6,1,8,9,10,4,3] 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95800a3b90 0x7f95800a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient(hunting): picked mon.noname-b con 0x7f95800a3b90 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f95800a44b0 0x7f95800a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient(hunting): picked mon.noname-h con 0x7f95800a44b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9580136570 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95800a1ba0 con 0x7f9580136570 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f95800a04b0 con 0x7f95800a3b90 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f95800a1e80 con 0x7f95800a44b0 2024-08-19T10:41:27.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:27.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f9584e29640 10 monclient(hunting): authenticate will time out at 1132.739339s 2024-08-19T10:41:27.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f95800a44b0 0x7f95800a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:27.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f95800a44b0 0x7f95800a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48232/0 (socket says 172.21.15.190:48232) 2024-08-19T10:41:27.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 1 -- 172.21.15.190:0/94734693 learned_addr learned my addr 172.21.15.190:0/94734693 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:27.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 10 monclient(hunting): get_auth_request con 0x7f95800a44b0 auth_method 0 2024-08-19T10:41:27.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:27.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:27.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:27.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:27.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:27.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:27.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957e59c640 1 --2- 172.21.15.190:0/94734693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95800a3b90 0x7f95800a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:27.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957e59c640 10 monclient(hunting): get_auth_request con 0x7f95800a3b90 auth_method 0 2024-08-19T10:41:27.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:27.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:27.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:27.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 10 monclient(hunting): handle_auth_done global_id 5252 payload 274 2024-08-19T10:41:27.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 1 -- 172.21.15.190:0/94734693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95800a3b90 msgr2=0x7f95800a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.166+0000 7f957dd9b640 1 --2- 172.21.15.190:0/94734693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95800a3b90 0x7f95800a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.170+0000 7f957dd9b640 1 -- 172.21.15.190:0/94734693 >> v1:172.21.15.89:6789/0 conn(0x7f9580136570 legacy=0x7f9580136a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.170+0000 7f957dd9b640 10 monclient: _finish_hunting 0 2024-08-19T10:41:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.170+0000 7f957dd9b640 1 monclient: found mon.noname-h 2024-08-19T10:41:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.170+0000 7f957dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.170+0000 7f957dd9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:27.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.170+0000 7f957dd9b640 1 -- 172.21.15.190:0/94734693 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958009fe50 con 0x7f95800a44b0 2024-08-19T10:41:27.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957dd9b640 1 -- 172.21.15.190:0/94734693 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f95800a44b0 msgr2=0x7f95800a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:27.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957dd9b640 1 --2- 172.21.15.190:0/94734693 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f95800a44b0 0x7f95800a48b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f95740098a0 tx=0x7f957401cfc0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=ff3370a99789670d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:27.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 1 -- 172.21.15.190:0/94734693 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f95740b2070 con 0x7f95800a44b0 2024-08-19T10:41:27.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:27.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:27.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 10 monclient: dump: 2024-08-19T10:41:27.181 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:27.182 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:27.182 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:27.182 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:27.182 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:27.182 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:27.182 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:27.182 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:27.182 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:27.182 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:27.182 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:27.182 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:27.183 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:27.183 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:27.183 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:27.183 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:27.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 10 monclient: _finish_auth 0 2024-08-19T10:41:27.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:41:27.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:27.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 5 monclient: authenticate success, global_id 5252 2024-08-19T10:41:27.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:27.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 1 -- 172.21.15.190:0/94734693 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95740ab7b0 con 0x7f95800a44b0 2024-08-19T10:41:27.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:27.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 1 -- 172.21.15.190:0/94734693 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f95740ab960 con 0x7f95800a44b0 2024-08-19T10:41:27.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:27.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:27.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f957d59a640 10 monclient: dump: 2024-08-19T10:41:27.184 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:27.184 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:27.184 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:27.185 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 4 set_mon_vals no callback set 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 10 monclient: shutdown 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 1 -- 172.21.15.190:0/94734693 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f95800a44b0 msgr2=0x7f95800a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 1 --2- 172.21.15.190:0/94734693 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f95800a44b0 0x7f95800a48b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f95740098a0 tx=0x7f957401cfc0 comp rx=0 tx=0).stop 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 1 -- 172.21.15.190:0/94734693 shutdown_connections 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 1 --2- 172.21.15.190:0/94734693 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f95800a44b0 0x7f95800a48b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 1 --2- 172.21.15.190:0/94734693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95800a3b90 0x7f95800a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 1 -- 172.21.15.190:0/94734693 >> 172.21.15.190:0/94734693 conn(0x7f958006dde0 msgr2=0x7f958006e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 1 -- 172.21.15.190:0/94734693 shutdown_connections 2024-08-19T10:41:27.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 1 -- 172.21.15.190:0/94734693 wait complete. 2024-08-19T10:41:27.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 10 monclient: build_initial_monmap 2024-08-19T10:41:27.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 10 monclient: monmap: 2024-08-19T10:41:27.187 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:27.187 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:27.187 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:27.187 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:27.187 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:27.187 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:27.187 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:27.187 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:27.187 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:27.188 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:27.188 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:27.188 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:27.188 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:27.188 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:27.188 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:27.188 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:27.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 1 Processor -- start 2024-08-19T10:41:27.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 1 -- start start 2024-08-19T10:41:27.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.174+0000 7f9584e29640 10 monclient: init 2024-08-19T10:41:27.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:27.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 10 monclient: _add_conns ranks=[1,6,0,4,3,8,7,5,2] 2024-08-19T10:41:27.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95800a3b90 0x7f95801cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:27.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 10 monclient(hunting): picked mon.noname-f con 0x7f95800a3b90 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:27.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f95800a44b0 0x7f9580135550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:27.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 10 monclient(hunting): picked mon.noname-d con 0x7f95800a44b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:27.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9580136570 0x7f958013b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:27.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 10 monclient(hunting): picked mon.noname-a con 0x7f9580136570 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:27.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9580140ff0 con 0x7f9580136570 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f95800a0b00 con 0x7f95800a3b90 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9580141170 con 0x7f95800a44b0 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f9584e29640 10 monclient(hunting): authenticate will time out at 1132.748385s 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95800a3b90 0x7f95801cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95800a3b90 0x7f95801cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34464/0 (socket says 172.21.15.190:34464) 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 1 -- 172.21.15.190:0/3178483132 learned_addr learned my addr 172.21.15.190:0/3178483132 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 10 monclient(hunting): get_auth_request con 0x7f95800a3b90 auth_method 0 2024-08-19T10:41:27.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 10 monclient(hunting): handle_auth_done global_id 14491 payload 824 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957dd9b640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f95800a44b0 0x7f9580135550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957ed9d640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9580136570 0x7f958013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 1 -- 172.21.15.190:0/3178483132 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f95800a44b0 msgr2=0x7f9580135550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f95800a44b0 0x7f9580135550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 1 -- 172.21.15.190:0/3178483132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9580136570 msgr2=0x7f958013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9580136570 0x7f958013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:27.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:41:27.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 1 monclient: found mon.noname-f 2024-08-19T10:41:27.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:27.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:27.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 1 -- 172.21.15.190:0/3178483132 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95800a4900 con 0x7f95800a3b90 2024-08-19T10:41:27.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 1 -- 172.21.15.190:0/3178483132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95800a3b90 msgr2=0x7f95801cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:27.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f957e59c640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95800a3b90 0x7f95801cf100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f957000d800 tx=0x7f957000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:27.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.178+0000 7f956effd640 1 -- 172.21.15.190:0/3178483132 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:41:27.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 1 -- 172.21.15.190:0/3178483132 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f957000bd40 con 0x7f95800a3b90 2024-08-19T10:41:27.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:27.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:27.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 10 monclient: dump: 2024-08-19T10:41:27.195 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:27.195 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:27.195 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:27.195 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:27.196 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:27.196 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:27.196 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:27.197 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 10 monclient: _finish_auth 0 2024-08-19T10:41:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f9584e29640 5 monclient: authenticate success, global_id 14491 2024-08-19T10:41:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 1 -- 172.21.15.190:0/3178483132 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9570019070 con 0x7f95800a3b90 2024-08-19T10:41:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f9584e29640 10 monclient: _renew_subs 2024-08-19T10:41:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 1 -- 172.21.15.190:0/3178483132 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f95700148a0 con 0x7f95800a3b90 2024-08-19T10:41:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f9584e29640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f9584e29640 1 -- 172.21.15.190:0/3178483132 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9580120ef0 con 0x7f95800a3b90 2024-08-19T10:41:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f957cd99640 4 set_mon_vals no callback set 2024-08-19T10:41:27.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f9584e29640 10 monclient: _renew_subs 2024-08-19T10:41:27.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f9584e29640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:27.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f9584e29640 1 -- 172.21.15.190:0/3178483132 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95801362f0 con 0x7f95800a3b90 2024-08-19T10:41:27.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:27.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:27.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 10 monclient: dump: 2024-08-19T10:41:27.200 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:27.200 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:27.200 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:27.200 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:27.201 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:27.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f9584e29640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:27.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f9584e29640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:27.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f9584e29640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:27.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f9584e29640 1 -- 172.21.15.190:0/3178483132 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95800a8aa0 con 0x7f95800a3b90 2024-08-19T10:41:27.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 1 -- 172.21.15.190:0/3178483132 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9570014430 con 0x7f95800a3b90 2024-08-19T10:41:27.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f956003be00 0x7f956003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:27.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f956effd640 1 -- 172.21.15.190:0/3178483132 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6387+0+0 (secure 0 0 0) 0x7f957000f040 con 0x7f95800a3b90 2024-08-19T10:41:27.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f957dd9b640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f956003be00 0x7f956003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:27.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.182+0000 7f957dd9b640 10 monclient: get_auth_request con 0x7f956003be00 auth_method 0 2024-08-19T10:41:27.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.186+0000 7f957dd9b640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f956003be00 0x7f956003e2c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f95800a53f0 tx=0x7f9574004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:27.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.186+0000 7f956effd640 1 -- 172.21.15.190:0/3178483132 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f95800a8aa0 con 0x7f95800a3b90 2024-08-19T10:41:27.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.186+0000 7f956effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:27.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.186+0000 7f956effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.354+0000 7f9584e29640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-08-19T10:41:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.354+0000 7f9584e29640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-08-19T10:41:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.354+0000 7f9584e29640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.354+0000 7f9584e29640 1 -- 172.21.15.190:0/3178483132 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "rack-for-test"} v 0) -- 0x7f95800a3f70 con 0x7f95800a3b90 2024-08-19T10:41:27.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.930+0000 7f956effd640 1 -- 172.21.15.190:0/3178483132 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "rack-for-test"}]=0 removed item id -12 name 'rack-for-test' from crush map v91) ==== 140+0+0 (secure 0 0 0) 0x7f9570022020 con 0x7f95800a3b90 2024-08-19T10:41:27.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.930+0000 7f956effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-08-19T10:41:27.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:27.930+0000 7f956effd640 10 monclient: _finish_command 2 = system:0 removed item id -12 name 'rack-for-test' from crush map 2024-08-19T10:41:28.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.034+0000 7f9584e29640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-08-19T10:41:28.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.034+0000 7f9584e29640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-08-19T10:41:28.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.034+0000 7f9584e29640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:28.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.034+0000 7f9584e29640 1 -- 172.21.15.190:0/3178483132 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "rack-for-test"} v 0) -- 0x7f95800008e0 con 0x7f95800a3b90 2024-08-19T10:41:28.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.034+0000 7f956effd640 1 -- 172.21.15.190:0/3178483132 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "rack-for-test"}]=0 device 'rack-for-test' does not appear in the crush map v91) ==== 140+0+0 (secure 0 0 0) 0x7f95800a4900 con 0x7f95800a3b90 2024-08-19T10:41:28.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.034+0000 7f956effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "rack-for-test"}] 2024-08-19T10:41:28.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.034+0000 7f956effd640 10 monclient: _finish_command 3 = system:0 device 'rack-for-test' does not appear in the crush map 2024-08-19T10:41:28.039 INFO:tasks.workunit.client.0.smithi190.stderr:device 'rack-for-test' does not appear in the crush map 2024-08-19T10:41:28.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 -- 172.21.15.190:0/3178483132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f956003be00 msgr2=0x7f956003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:28.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f956003be00 0x7f956003e2c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f95800a53f0 tx=0x7f9574004740 comp rx=0 tx=0).stop 2024-08-19T10:41:28.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 10 monclient: shutdown 2024-08-19T10:41:28.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:28.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 -- 172.21.15.190:0/3178483132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95800a3b90 msgr2=0x7f95801cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:28.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95800a3b90 0x7f95801cf100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f957000d800 tx=0x7f957000dcc0 comp rx=0 tx=0).stop 2024-08-19T10:41:28.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 -- 172.21.15.190:0/3178483132 shutdown_connections 2024-08-19T10:41:28.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f956003be00 0x7f956003e2c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:28.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9580136570 0x7f958013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:28.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f95800a44b0 0x7f9580135550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:28.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 --2- 172.21.15.190:0/3178483132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95800a3b90 0x7f95801cf100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:28.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 -- 172.21.15.190:0/3178483132 >> 172.21.15.190:0/3178483132 conn(0x7f958006dde0 msgr2=0x7f958009ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:28.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 -- 172.21.15.190:0/3178483132 shutdown_connections 2024-08-19T10:41:28.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.038+0000 7f9584e29640 1 -- 172.21.15.190:0/3178483132 wait complete. 2024-08-19T10:41:28.056 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm root-for-test 2024-08-19T10:41:28.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient: get_monmap_and_config 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient: build_initial_monmap 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient: monmap: 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:28.169111+0000 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:28.169111+0000 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:28.170 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:28.170 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:28.170 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:28.170 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:28.170 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:28.170 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:28.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 1 Processor -- start 2024-08-19T10:41:28.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 1 -- start start 2024-08-19T10:41:28.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient: init 2024-08-19T10:41:28.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient: _add_conns ranks=[3,10,0,9,1,4,5,8,2,7,6] 2024-08-19T10:41:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff44139a60 0x7eff44139e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient(hunting): picked mon.noname-g con 0x7eff44139a60 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7eff440a5730 0x7eff440a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient(hunting): picked mon.noname-f con 0x7eff440a5730 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7eff440a6850 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff440a33c0 con 0x7eff440a6850 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7eff4409ca60 con 0x7eff440a5730 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7eff440a3640 con 0x7eff44139a60 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff4a7f8640 10 monclient(hunting): authenticate will time out at 1133.738958s 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff43fff640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff44139a60 msgr2=0x7eff44139e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff43fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff44139a60 0x7eff44139e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7eff440a5730 0x7eff440a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7eff440a5730 0x7eff440a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34468/0 (socket says 172.21.15.190:34468) 2024-08-19T10:41:28.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 1 -- 172.21.15.190:0/2389403192 learned_addr learned my addr 172.21.15.190:0/2389403192 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 10 monclient(hunting): get_auth_request con 0x7eff440a5730 auth_method 0 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 10 monclient(hunting): handle_auth_done global_id 14500 payload 274 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 1 -- 172.21.15.190:0/2389403192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff44139a60 msgr2=0x7eff44139e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 1 --2- 172.21.15.190:0/2389403192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff44139a60 0x7eff44139e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 1 -- 172.21.15.190:0/2389403192 >> v1:172.21.15.89:6789/0 conn(0x7eff440a6850 legacy=0x7eff440a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 1 monclient: found mon.noname-f 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:28.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:28.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.166+0000 7eff437fe640 1 -- 172.21.15.190:0/2389403192 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff440a46e0 con 0x7eff440a5730 2024-08-19T10:41:28.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff437fe640 1 -- 172.21.15.190:0/2389403192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff440a5730 msgr2=0x7eff440a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:28.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff437fe640 1 --2- 172.21.15.190:0/2389403192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff440a5730 0x7eff440a5b10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7eff34009870 tx=0x7eff3401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5f12b4380ddda6c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:28.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 1 -- 172.21.15.190:0/2389403192 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff340b0070 con 0x7eff440a5730 2024-08-19T10:41:28.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 10 monclient: dump: 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:28.182 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 10 monclient: _finish_auth 0 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 1 -- 172.21.15.190:0/2389403192 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff340aa810 con 0x7eff440a5730 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 1 -- 172.21.15.190:0/2389403192 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff340a93e0 con 0x7eff440a5730 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 5 monclient: authenticate success, global_id 14500 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:28.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 4 set_mon_vals no callback set 2024-08-19T10:41:28.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:28.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:28.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff42ffd640 10 monclient: dump: 2024-08-19T10:41:28.186 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:28.186 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:28.186 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:28.186 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:28.186 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:28.186 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:28.187 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:28.187 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:28.187 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:28.187 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:28.187 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:28.187 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:28.187 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:28.187 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:28.187 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:28.187 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:28.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient: shutdown 2024-08-19T10:41:28.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:28.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 -- 172.21.15.190:0/2389403192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff440a5730 msgr2=0x7eff440a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:28.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 --2- 172.21.15.190:0/2389403192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff440a5730 0x7eff440a5b10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7eff34009870 tx=0x7eff3401c3e0 comp rx=0 tx=0).stop 2024-08-19T10:41:28.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 -- 172.21.15.190:0/2389403192 shutdown_connections 2024-08-19T10:41:28.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 --2- 172.21.15.190:0/2389403192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff440a5730 0x7eff440a5b10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:28.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 --2- 172.21.15.190:0/2389403192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff44139a60 0x7eff44139e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:28.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 -- 172.21.15.190:0/2389403192 >> 172.21.15.190:0/2389403192 conn(0x7eff440abe50 msgr2=0x7eff440ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:28.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 -- 172.21.15.190:0/2389403192 shutdown_connections 2024-08-19T10:41:28.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 -- 172.21.15.190:0/2389403192 wait complete. 2024-08-19T10:41:28.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient: build_initial_monmap 2024-08-19T10:41:28.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient: monmap: 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:28.190 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:28.190 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:28.190 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:28.190 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:28.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 Processor -- start 2024-08-19T10:41:28.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 -- start start 2024-08-19T10:41:28.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient: init 2024-08-19T10:41:28.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:28.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient: _add_conns ranks=[3,4,5,2,8,6,1,7,0] 2024-08-19T10:41:28.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff440a5730 0x7eff4413a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:28.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient(hunting): picked mon.noname-g con 0x7eff440a5730 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:28.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff440a6850 0x7eff4413ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:28.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient(hunting): picked mon.noname-c con 0x7eff440a6850 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:28.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7eff44139a60 0x7eff4413b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:28.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient(hunting): picked mon.noname-h con 0x7eff44139a60 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:28.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:28.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7eff44144280 con 0x7eff440a5730 2024-08-19T10:41:28.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:28.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7eff44144100 con 0x7eff440a6850 2024-08-19T10:41:28.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:28.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7eff44144400 con 0x7eff44139a60 2024-08-19T10:41:28.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:28.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff4a7f8640 10 monclient(hunting): authenticate will time out at 1133.746754s 2024-08-19T10:41:28.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff43fff640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff440a5730 msgr2=0x7eff4413a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:28.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff43fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff440a5730 0x7eff4413a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:28.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7eff44139a60 0x7eff4413b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:28.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7eff44139a60 0x7eff4413b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48238/0 (socket says 172.21.15.190:48238) 2024-08-19T10:41:28.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 1 -- 172.21.15.190:0/1026894202 learned_addr learned my addr 172.21.15.190:0/1026894202 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:28.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 10 monclient(hunting): get_auth_request con 0x7eff44139a60 auth_method 0 2024-08-19T10:41:28.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:28.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:28.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff437fe640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff440a6850 0x7eff4413ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 10 monclient(hunting): handle_auth_done global_id 5261 payload 824 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 1 -- 172.21.15.190:0/1026894202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff440a6850 msgr2=0x7eff4413ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff440a6850 0x7eff4413ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 1 -- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff440a5730 msgr2=0x7eff4413a620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff440a5730 0x7eff4413a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 10 monclient: _finish_hunting 0 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 1 monclient: found mon.noname-h 2024-08-19T10:41:28.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:28.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:28.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff48d95640 1 -- 172.21.15.190:0/1026894202 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4413c990 con 0x7eff44139a60 2024-08-19T10:41:28.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff437fe640 10 monclient: get_auth_request con 0x7eff440a6850 auth_method 0 2024-08-19T10:41:28.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.174+0000 7eff437fe640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff440a6850 0x7eff4413ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:28.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff48d95640 1 -- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7eff44139a60 msgr2=0x7eff4413b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:28.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff48d95640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7eff44139a60 0x7eff4413b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00c780 tx=0x7eff3c00cc40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:28.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 1 -- 172.21.15.190:0/1026894202 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff3c004970 con 0x7eff44139a60 2024-08-19T10:41:28.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:28.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:28.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 10 monclient: dump: 2024-08-19T10:41:28.194 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:28.195 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:28.196 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:28.196 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:28.196 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:28.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 10 monclient: _finish_auth 0 2024-08-19T10:41:28.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:41:28.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:28.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 1 -- 172.21.15.190:0/1026894202 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3c004b30 con 0x7eff44139a60 2024-08-19T10:41:28.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:28.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 1 -- 172.21.15.190:0/1026894202 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff3c004ce0 con 0x7eff44139a60 2024-08-19T10:41:28.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:28.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 10 monclient: dump: 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:28.197 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:28.198 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff427fc640 4 set_mon_vals no callback set 2024-08-19T10:41:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff4a7f8640 5 monclient: authenticate success, global_id 5261 2024-08-19T10:41:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff4a7f8640 10 monclient: _renew_subs 2024-08-19T10:41:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff4a7f8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff4a7f8640 1 -- 172.21.15.190:0/1026894202 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4413cc70 con 0x7eff44139a60 2024-08-19T10:41:28.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff4a7f8640 10 monclient: _renew_subs 2024-08-19T10:41:28.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff4a7f8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:28.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff4a7f8640 1 -- 172.21.15.190:0/1026894202 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4413cf50 con 0x7eff44139a60 2024-08-19T10:41:28.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff4a7f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:28.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff4a7f8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:28.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff4a7f8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:28.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff4a7f8640 1 -- 172.21.15.190:0/1026894202 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff440a5b10 con 0x7eff44139a60 2024-08-19T10:41:28.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 1 -- 172.21.15.190:0/1026894202 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7eff3c00aa60 con 0x7eff44139a60 2024-08-19T10:41:28.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7eff3003be00 0x7eff3003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:28.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff40ff9640 1 -- 172.21.15.190:0/1026894202 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(91..91 src has 1..91) ==== 6204+0+0 (secure 0 0 0) 0x7eff3c04f3a0 con 0x7eff44139a60 2024-08-19T10:41:28.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff43fff640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7eff3003be00 0x7eff3003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:28.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.178+0000 7eff43fff640 10 monclient: get_auth_request con 0x7eff3003be00 auth_method 0 2024-08-19T10:41:28.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.182+0000 7eff43fff640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7eff3003be00 0x7eff3003e2c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7eff38003990 tx=0x7eff38006cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:28.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.182+0000 7eff40ff9640 1 -- 172.21.15.190:0/1026894202 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7eff3c020070 con 0x7eff44139a60 2024-08-19T10:41:28.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.182+0000 7eff40ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:28.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.182+0000 7eff40ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:28.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.342+0000 7eff4a7f8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-08-19T10:41:28.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.342+0000 7eff4a7f8640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-08-19T10:41:28.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.342+0000 7eff4a7f8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:28.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.342+0000 7eff4a7f8640 1 -- 172.21.15.190:0/1026894202 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "root-for-test"} v 0) -- 0x7eff44140e30 con 0x7eff44139a60 2024-08-19T10:41:28.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.958+0000 7eff40ff9640 1 -- 172.21.15.190:0/1026894202 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "root-for-test"}]=0 removed item id -14 name 'root-for-test' from crush map v92) ==== 140+0+0 (secure 0 0 0) 0x7eff3c04f0d0 con 0x7eff44139a60 2024-08-19T10:41:28.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.958+0000 7eff40ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-08-19T10:41:28.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:28.958+0000 7eff40ff9640 10 monclient: _finish_command 2 = system:0 removed item id -14 name 'root-for-test' from crush map 2024-08-19T10:41:29.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.066+0000 7eff4a7f8640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-08-19T10:41:29.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.066+0000 7eff4a7f8640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-08-19T10:41:29.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.066+0000 7eff4a7f8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:29.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.066+0000 7eff4a7f8640 1 -- 172.21.15.190:0/1026894202 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "root-for-test"} v 0) -- 0x7eff4413b750 con 0x7eff44139a60 2024-08-19T10:41:29.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff40ff9640 1 -- 172.21.15.190:0/1026894202 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "root-for-test"}]=0 device 'root-for-test' does not appear in the crush map v92) ==== 140+0+0 (secure 0 0 0) 0x7eff3c019a40 con 0x7eff44139a60 2024-08-19T10:41:29.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff40ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "root-for-test"}] 2024-08-19T10:41:29.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff40ff9640 10 monclient: _finish_command 3 = system:0 device 'root-for-test' does not appear in the crush map 2024-08-19T10:41:29.072 INFO:tasks.workunit.client.0.smithi190.stderr:device 'root-for-test' does not appear in the crush map 2024-08-19T10:41:29.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 -- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7eff3003be00 msgr2=0x7eff3003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7eff3003be00 0x7eff3003e2c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7eff38003990 tx=0x7eff38006cf0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 10 monclient: shutdown 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 -- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7eff44139a60 msgr2=0x7eff4413b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7eff44139a60 0x7eff4413b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00c780 tx=0x7eff3c00cc40 comp rx=0 tx=0).stop 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 -- 172.21.15.190:0/1026894202 shutdown_connections 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7eff3003be00 0x7eff3003e2c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7eff44139a60 0x7eff4413b0a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff440a6850 0x7eff4413ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 --2- 172.21.15.190:0/1026894202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff440a5730 0x7eff4413a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 -- 172.21.15.190:0/1026894202 >> 172.21.15.190:0/1026894202 conn(0x7eff440abe50 msgr2=0x7eff44134ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 -- 172.21.15.190:0/1026894202 shutdown_connections 2024-08-19T10:41:29.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.070+0000 7eff4a7f8640 1 -- 172.21.15.190:0/1026894202 wait complete. 2024-08-19T10:41:29.090 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule create-simple foo-rule foo host firstn 2024-08-19T10:41:29.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.210+0000 7f73fe443640 10 monclient: get_monmap_and_config 2024-08-19T10:41:29.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.210+0000 7f73fe443640 10 monclient: build_initial_monmap 2024-08-19T10:41:29.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.210+0000 7f73fe443640 10 monclient: monmap: 2024-08-19T10:41:29.214 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:29.214 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:29.214642+0000 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:29.214642+0000 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:29.215 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:29.216 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:29.216 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:29.216 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:29.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 1 Processor -- start 2024-08-19T10:41:29.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 1 -- start start 2024-08-19T10:41:29.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 10 monclient: init 2024-08-19T10:41:29.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:29.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 10 monclient: _add_conns ranks=[10,8,2,7,9,5,6,4,1,3,0] 2024-08-19T10:41:29.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73f81399f0 0x7f73f80a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:29.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 10 monclient(hunting): picked mon.noname-f con 0x7f73f81399f0 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:29.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73f8139dd0 0x7f73f80a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:29.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 10 monclient(hunting): picked mon.noname-e con 0x7f73f8139dd0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:29.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f813a1b0 0x7f73f80a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:29.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 10 monclient(hunting): picked mon.noname-b con 0x7f73f813a1b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:29.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:29.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f73f80a4eb0 con 0x7f73f81399f0 2024-08-19T10:41:29.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:29.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f73f80a5b80 con 0x7f73f813a1b0 2024-08-19T10:41:29.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f73f80a8340 con 0x7f73f8139dd0 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fe443640 10 monclient(hunting): authenticate will time out at 1134.785104s 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73f81399f0 0x7f73f80a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73f81399f0 0x7f73f80a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34478/0 (socket says 172.21.15.190:34478) 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 1 -- 172.21.15.190:0/640286950 learned_addr learned my addr 172.21.15.190:0/640286950 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 10 monclient(hunting): get_auth_request con 0x7f73f81399f0 auth_method 0 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:29.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73ef7fe640 1 --2- 172.21.15.190:0/640286950 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73f8139dd0 0x7f73f80a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 10 monclient(hunting): handle_auth_done global_id 14509 payload 274 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fc9e0640 1 --2- 172.21.15.190:0/640286950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f813a1b0 0x7f73f80a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 1 -- 172.21.15.190:0/640286950 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73f8139dd0 msgr2=0x7f73f80a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 1 --2- 172.21.15.190:0/640286950 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73f8139dd0 0x7f73f80a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 1 -- 172.21.15.190:0/640286950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f813a1b0 msgr2=0x7f73f80a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 1 --2- 172.21.15.190:0/640286950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f813a1b0 0x7f73f80a6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 1 monclient: found mon.noname-f 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:29.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:29.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73f7fff640 1 -- 172.21.15.190:0/640286950 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f80a4580 con 0x7f73f81399f0 2024-08-19T10:41:29.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73ef7fe640 10 monclient: get_auth_request con 0x7f73f8139dd0 auth_method 0 2024-08-19T10:41:29.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73ef7fe640 1 --2- 172.21.15.190:0/640286950 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73f8139dd0 0x7f73f80a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:29.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fc9e0640 10 monclient: get_auth_request con 0x7f73f813a1b0 auth_method 0 2024-08-19T10:41:29.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.214+0000 7f73fc9e0640 1 --2- 172.21.15.190:0/640286950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f813a1b0 0x7f73f80a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:29.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f7fff640 1 -- 172.21.15.190:0/640286950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73f81399f0 msgr2=0x7f73f80a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f7fff640 1 --2- 172.21.15.190:0/640286950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73f81399f0 0x7f73f80a5640 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73e80098a0 tx=0x7f73e801c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcd00035043d79f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 1 -- 172.21.15.190:0/640286950 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73e80b0070 con 0x7f73f81399f0 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 10 monclient: dump: 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:29.227 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 5 monclient: authenticate success, global_id 14509 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:29.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 1 -- 172.21.15.190:0/640286950 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e80aa810 con 0x7f73f81399f0 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 4 set_mon_vals no callback set 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 1 -- 172.21.15.190:0/640286950 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73e80a93e0 con 0x7f73f81399f0 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73f77fe640 10 monclient: dump: 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:29.229 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:29.230 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:29.230 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:29.230 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:29.230 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:29.230 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:29.230 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:29.230 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:29.230 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:29.230 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:29.230 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 10 monclient: shutdown 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 1 -- 172.21.15.190:0/640286950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73f81399f0 msgr2=0x7f73f80a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 1 --2- 172.21.15.190:0/640286950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73f81399f0 0x7f73f80a5640 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73e80098a0 tx=0x7f73e801c3d0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 1 -- 172.21.15.190:0/640286950 shutdown_connections 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 1 --2- 172.21.15.190:0/640286950 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f813a1b0 0x7f73f80a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 1 --2- 172.21.15.190:0/640286950 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73f8139dd0 0x7f73f80a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 1 --2- 172.21.15.190:0/640286950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73f81399f0 0x7f73f80a5640 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 1 -- 172.21.15.190:0/640286950 >> 172.21.15.190:0/640286950 conn(0x7f73f8074c10 msgr2=0x7f73f80abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 1 -- 172.21.15.190:0/640286950 shutdown_connections 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 1 -- 172.21.15.190:0/640286950 wait complete. 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 10 monclient: build_initial_monmap 2024-08-19T10:41:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 10 monclient: monmap: 2024-08-19T10:41:29.232 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:29.232 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:29.232 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:29.232 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:29.232 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:29.232 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:29.232 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:29.232 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:29.232 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:29.232 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:29.232 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 1 Processor -- start 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 1 -- start start 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 10 monclient: init 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.218+0000 7f73fe443640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient: _add_conns ranks=[5,7,2,4,0,8,3,1,6] 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f73f81399f0 0x7f73f813a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient(hunting): picked mon.noname-h con 0x7f73f81399f0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73f8139dd0 0x7f73f813abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:29.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient(hunting): picked mon.noname-i con 0x7f73f8139dd0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f813a1b0 0x7f73f813b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient(hunting): picked mon.noname-b con 0x7f73f813a1b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f73f8143110 con 0x7f73f813a1b0 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f73f80a3fd0 con 0x7f73f81399f0 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f73f8143290 con 0x7f73f8139dd0 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient(hunting): authenticate will time out at 1134.792074s 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f7fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f73f81399f0 0x7f73f813a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73f8139dd0 0x7f73f813abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:29.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73f8139dd0 0x7f73f813abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:46226/0 (socket says 172.21.15.190:46226) 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 1 -- 172.21.15.190:0/1852822862 learned_addr learned my addr 172.21.15.190:0/1852822862 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f7fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f73f81399f0 0x7f73f813a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48246/0 (socket says 172.21.15.190:48246) 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 10 monclient(hunting): get_auth_request con 0x7f73f8139dd0 auth_method 0 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f7fff640 10 monclient(hunting): get_auth_request con 0x7f73f81399f0 auth_method 0 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:29.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fc9e0640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f813a1b0 0x7f73f813b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fc9e0640 10 monclient(hunting): get_auth_request con 0x7f73f813a1b0 auth_method 0 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fc9e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fc9e0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fc9e0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 10 monclient(hunting): handle_auth_done global_id 14479 payload 824 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 1 -- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f73f81399f0 msgr2=0x7f73f813a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f73f81399f0 0x7f73f813a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 1 -- 172.21.15.190:0/1852822862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f813a1b0 msgr2=0x7f73f813b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f813a1b0 0x7f73f813b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:29.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:29.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 1 monclient: found mon.noname-i 2024-08-19T10:41:29.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:29.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:29.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 1 -- 172.21.15.190:0/1852822862 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f80aba40 con 0x7f73f8139dd0 2024-08-19T10:41:29.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 1 -- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73f8139dd0 msgr2=0x7f73f813abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:29.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73ef7fe640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73f8139dd0 0x7f73f813abb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f73e000c7c0 tx=0x7f73e000cc80 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:29.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 1 -- 172.21.15.190:0/1852822862 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73e00113f0 con 0x7f73f8139dd0 2024-08-19T10:41:29.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:29.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 10 monclient: dump: 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:29.238 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:29.239 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:29.239 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:29.239 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:29.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 10 monclient: _finish_auth 0 2024-08-19T10:41:29.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:41:29.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:29.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 5 monclient: authenticate success, global_id 14479 2024-08-19T10:41:29.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 1 -- 172.21.15.190:0/1852822862 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e00115d0 con 0x7f73f8139dd0 2024-08-19T10:41:29.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:29.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 1 -- 172.21.15.190:0/1852822862 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73e0011bf0 con 0x7f73f8139dd0 2024-08-19T10:41:29.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:29.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f6ffd640 4 set_mon_vals no callback set 2024-08-19T10:41:29.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:29.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73f57fa640 10 monclient: dump: 2024-08-19T10:41:29.240 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:29.240 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:29.240 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:29.240 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:29.240 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:29.240 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:29.240 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:29.240 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient: _renew_subs 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 1 -- 172.21.15.190:0/1852822862 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f813ca00 con 0x7f73f8139dd0 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient: _renew_subs 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:29.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.222+0000 7f73fe443640 1 -- 172.21.15.190:0/1852822862 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f8135880 con 0x7f73f8139dd0 2024-08-19T10:41:29.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.226+0000 7f73fe443640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:29.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.226+0000 7f73fe443640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:29.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.226+0000 7f73fe443640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:29.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.226+0000 7f73fe443640 1 -- 172.21.15.190:0/1852822862 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f80a5640 con 0x7f73f8139dd0 2024-08-19T10:41:29.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.226+0000 7f73f57fa640 1 -- 172.21.15.190:0/1852822862 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f73e0011780 con 0x7f73f8139dd0 2024-08-19T10:41:29.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.226+0000 7f73f57fa640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f73d803be00 0x7f73d803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:29.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.226+0000 7f73f57fa640 1 -- 172.21.15.190:0/1852822862 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(92..92 src has 1..92) ==== 6045+0+0 (secure 0 0 0) 0x7f73e004f1c0 con 0x7f73f8139dd0 2024-08-19T10:41:29.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.226+0000 7f73f7fff640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f73d803be00 0x7f73d803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:29.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.226+0000 7f73f7fff640 10 monclient: get_auth_request con 0x7f73d803be00 auth_method 0 2024-08-19T10:41:29.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.226+0000 7f73f7fff640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f73d803be00 0x7f73d803e2c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f73f806ce50 tx=0x7f73e80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:29.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.230+0000 7f73f57fa640 1 -- 172.21.15.190:0/1852822862 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f73f80aba40 con 0x7f73f8139dd0 2024-08-19T10:41:29.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.230+0000 7f73f57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:29.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.230+0000 7f73f57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:29.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.402+0000 7f73fe443640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-08-19T10:41:29.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.402+0000 7f73fe443640 10 monclient: _send_command 2 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-08-19T10:41:29.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.402+0000 7f73fe443640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:29.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.402+0000 7f73fe443640 1 -- 172.21.15.190:0/1852822862 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"} v 0) -- 0x7f73f813b840 con 0x7f73f8139dd0 2024-08-19T10:41:29.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.965+0000 7f73f57fa640 1 -- 172.21.15.190:0/1852822862 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}]=0 v93) ==== 145+0+0 (secure 0 0 0) 0x7f73e0015020 con 0x7f73f8139dd0 2024-08-19T10:41:29.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.965+0000 7f73f57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-08-19T10:41:29.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:29.965+0000 7f73f57fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:30.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.073+0000 7f73fe443640 10 start_mon_command cmd=[{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-08-19T10:41:30.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.073+0000 7f73fe443640 10 monclient: _send_command 3 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-08-19T10:41:30.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.073+0000 7f73fe443640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:30.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.073+0000 7f73fe443640 1 -- 172.21.15.190:0/1852822862 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"} v 0) -- 0x7f73f80a3500 con 0x7f73f8139dd0 2024-08-19T10:41:30.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73f57fa640 1 -- 172.21.15.190:0/1852822862 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}]=0 rule foo-rule already exists v93) ==== 173+0+0 (secure 0 0 0) 0x7f73e0018d60 con 0x7f73f8139dd0 2024-08-19T10:41:30.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73f57fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule create-simple", "name": "foo-rule", "root": "foo", "type": "host", "mode": "firstn"}] 2024-08-19T10:41:30.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73f57fa640 10 monclient: _finish_command 3 = system:0 rule foo-rule already exists 2024-08-19T10:41:30.080 INFO:tasks.workunit.client.0.smithi190.stderr:rule foo-rule already exists 2024-08-19T10:41:30.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 -- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f73d803be00 msgr2=0x7f73d803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:30.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f73d803be00 0x7f73d803e2c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f73f806ce50 tx=0x7f73e80a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:30.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 10 monclient: shutdown 2024-08-19T10:41:30.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:30.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 -- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73f8139dd0 msgr2=0x7f73f813abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:30.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73f8139dd0 0x7f73f813abb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f73e000c7c0 tx=0x7f73e000cc80 comp rx=0 tx=0).stop 2024-08-19T10:41:30.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 -- 172.21.15.190:0/1852822862 shutdown_connections 2024-08-19T10:41:30.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f73d803be00 0x7f73d803e2c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f73f813a1b0 0x7f73f813b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f73f8139dd0 0x7f73f813abb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 --2- 172.21.15.190:0/1852822862 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f73f81399f0 0x7f73f813a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 -- 172.21.15.190:0/1852822862 >> 172.21.15.190:0/1852822862 conn(0x7f73f8074c10 msgr2=0x7f73f80abc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:30.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 -- 172.21.15.190:0/1852822862 shutdown_connections 2024-08-19T10:41:30.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.077+0000 7f73fe443640 1 -- 172.21.15.190:0/1852822862 wait complete. 2024-08-19T10:41:30.098 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush rm foo 2024-08-19T10:41:30.098 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:41:30.098 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm foo 2024-08-19T10:41:30.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient: get_monmap_and_config 2024-08-19T10:41:30.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient: build_initial_monmap 2024-08-19T10:41:30.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient: monmap: 2024-08-19T10:41:30.209 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:30.209 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:30.209 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:30.209444+0000 2024-08-19T10:41:30.209 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:30.209444+0000 2024-08-19T10:41:30.209 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:30.209 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:30.211 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:30.211 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:30.211 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 1 Processor -- start 2024-08-19T10:41:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 1 -- start start 2024-08-19T10:41:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient: init 2024-08-19T10:41:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient: _add_conns ranks=[10,6,5,2,4,7,8,0,1,3,9] 2024-08-19T10:41:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efc4c0a5220 0x7efc4c136000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient(hunting): picked mon.noname-f con 0x7efc4c0a5220 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efc4c073f40 0x7efc4c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient(hunting): picked mon.noname-d con 0x7efc4c073f40 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efc4c074360 0x7efc4c070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient(hunting): picked mon.noname-h con 0x7efc4c074360 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7efc4c0710d0 con 0x7efc4c0a5220 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7efc4c0a5600 con 0x7efc4c074360 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7efc4c072170 con 0x7efc4c073f40 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.205+0000 7efc52467640 10 monclient(hunting): authenticate will time out at 1135.779933s 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efc4c074360 0x7efc4c070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4bfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efc4c0a5220 0x7efc4c136000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efc4c074360 0x7efc4c070140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48248/0 (socket says 172.21.15.190:48248) 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 1 -- 172.21.15.190:0/2565632388 learned_addr learned my addr 172.21.15.190:0/2565632388 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 10 monclient(hunting): get_auth_request con 0x7efc4c074360 auth_method 0 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4bfff640 10 monclient(hunting): get_auth_request con 0x7efc4c0a5220 auth_method 0 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4b7fe640 1 --2- 172.21.15.190:0/2565632388 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efc4c073f40 0x7efc4c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 10 monclient(hunting): handle_auth_done global_id 5279 payload 274 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 1 -- 172.21.15.190:0/2565632388 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efc4c073f40 msgr2=0x7efc4c06fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 1 --2- 172.21.15.190:0/2565632388 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efc4c073f40 0x7efc4c06fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 1 -- 172.21.15.190:0/2565632388 >> v2:172.21.15.190:3300/0 conn(0x7efc4c0a5220 msgr2=0x7efc4c136000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 1 --2- 172.21.15.190:0/2565632388 >> v2:172.21.15.190:3300/0 conn(0x7efc4c0a5220 0x7efc4c136000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 10 monclient: _finish_hunting 0 2024-08-19T10:41:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 1 monclient: found mon.noname-h 2024-08-19T10:41:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc50a04640 1 -- 172.21.15.190:0/2565632388 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c072450 con 0x7efc4c074360 2024-08-19T10:41:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4b7fe640 10 monclient: get_auth_request con 0x7efc4c073f40 auth_method 0 2024-08-19T10:41:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4b7fe640 1 --2- 172.21.15.190:0/2565632388 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efc4c073f40 0x7efc4c06fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.209+0000 7efc4bfff640 1 --2- 172.21.15.190:0/2565632388 >> v2:172.21.15.190:3300/0 conn(0x7efc4c0a5220 0x7efc4c136000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc50a04640 1 -- 172.21.15.190:0/2565632388 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efc4c074360 msgr2=0x7efc4c070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc50a04640 1 --2- 172.21.15.190:0/2565632388 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efc4c074360 0x7efc4c070140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc44009e50 tx=0x7efc4401c8f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=8bd2f4eb5e51a98b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 1 -- 172.21.15.190:0/2565632388 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efc440090d0 con 0x7efc4c074360 2024-08-19T10:41:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 10 monclient: dump: 2024-08-19T10:41:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:30.226 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 10 monclient: _finish_auth 0 2024-08-19T10:41:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:30.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:30.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 5 monclient: authenticate success, global_id 5279 2024-08-19T10:41:30.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:30.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 1 -- 172.21.15.190:0/2565632388 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc440a6930 con 0x7efc4c074360 2024-08-19T10:41:30.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:30.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:30.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 4 set_mon_vals no callback set 2024-08-19T10:41:30.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 1 -- 172.21.15.190:0/2565632388 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efc440af480 con 0x7efc4c074360 2024-08-19T10:41:30.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:30.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:30.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4affd640 10 monclient: dump: 2024-08-19T10:41:30.228 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:30.228 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:30.228 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.228 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.228 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:30.228 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.228 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:30.228 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:30.228 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient: shutdown 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 -- 172.21.15.190:0/2565632388 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efc4c074360 msgr2=0x7efc4c070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 --2- 172.21.15.190:0/2565632388 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efc4c074360 0x7efc4c070140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc44009e50 tx=0x7efc4401c8f0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 -- 172.21.15.190:0/2565632388 shutdown_connections 2024-08-19T10:41:30.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 --2- 172.21.15.190:0/2565632388 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efc4c074360 0x7efc4c070140 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 --2- 172.21.15.190:0/2565632388 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efc4c073f40 0x7efc4c06fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 --2- 172.21.15.190:0/2565632388 >> v2:172.21.15.190:3300/0 conn(0x7efc4c0a5220 0x7efc4c136000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 -- 172.21.15.190:0/2565632388 >> 172.21.15.190:0/2565632388 conn(0x7efc4c0a4700 msgr2=0x7efc4c0a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:30.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 -- 172.21.15.190:0/2565632388 shutdown_connections 2024-08-19T10:41:30.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 -- 172.21.15.190:0/2565632388 wait complete. 2024-08-19T10:41:30.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient: build_initial_monmap 2024-08-19T10:41:30.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient: monmap: 2024-08-19T10:41:30.230 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:30.230 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:30.230 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:30.231 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:30.231 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:30.231 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.231 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:30.231 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:30.231 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:30.231 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:30.231 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:30.231 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:30.231 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:30.231 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:30.232 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:30.232 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 Processor -- start 2024-08-19T10:41:30.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 -- start start 2024-08-19T10:41:30.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient: init 2024-08-19T10:41:30.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:30.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient: _add_conns ranks=[1,7,2,6,5,4,3,0,8] 2024-08-19T10:41:30.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4c073f40 0x7efc4c1cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient(hunting): picked mon.noname-f con 0x7efc4c073f40 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:30.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efc4c074360 0x7efc4c1cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient(hunting): picked mon.noname-i con 0x7efc4c074360 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:30.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc4c0a5220 0x7efc4c1cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient(hunting): picked mon.noname-b con 0x7efc4c0a5220 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:30.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c13ec50 con 0x7efc4c073f40 2024-08-19T10:41:30.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7efc4c13edd0 con 0x7efc4c0a5220 2024-08-19T10:41:30.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7efc4c13ef50 con 0x7efc4c074360 2024-08-19T10:41:30.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:30.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc52467640 10 monclient(hunting): authenticate will time out at 1135.787269s 2024-08-19T10:41:30.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4c073f40 0x7efc4c1cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4c073f40 0x7efc4c1cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34496/0 (socket says 172.21.15.190:34496) 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 1 -- 172.21.15.190:0/1792445919 learned_addr learned my addr 172.21.15.190:0/1792445919 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 10 monclient(hunting): get_auth_request con 0x7efc4c073f40 auth_method 0 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4b7fe640 1 --2- 172.21.15.190:0/1792445919 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efc4c074360 0x7efc4c1cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4b7fe640 10 monclient(hunting): get_auth_request con 0x7efc4c074360 auth_method 0 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:30.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:30.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:30.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:30.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:30.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:30.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:30.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 10 monclient(hunting): handle_auth_done global_id 14527 payload 824 2024-08-19T10:41:30.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 1 -- 172.21.15.190:0/1792445919 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efc4c074360 msgr2=0x7efc4c1cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:30.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 1 --2- 172.21.15.190:0/1792445919 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efc4c074360 0x7efc4c1cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 1 -- 172.21.15.190:0/1792445919 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc4c0a5220 msgr2=0x7efc4c1cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:30.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 1 --2- 172.21.15.190:0/1792445919 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efc4c0a5220 0x7efc4c1cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:30.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 1 monclient: found mon.noname-f 2024-08-19T10:41:30.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:30.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:30.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.213+0000 7efc4bfff640 1 -- 172.21.15.190:0/1792445919 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c131510 con 0x7efc4c073f40 2024-08-19T10:41:30.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc4bfff640 1 -- 172.21.15.190:0/1792445919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4c073f40 msgr2=0x7efc4c1cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:30.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc4bfff640 1 --2- 172.21.15.190:0/1792445919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4c073f40 0x7efc4c1cf000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00e940 tx=0x7efc3c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:30.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 1 -- 172.21.15.190:0/1792445919 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efc3c016070 con 0x7efc4c073f40 2024-08-19T10:41:30.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:30.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:30.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 10 monclient: dump: 2024-08-19T10:41:30.236 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:30.237 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:30.238 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:30.238 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:30.238 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 10 monclient: _finish_auth 0 2024-08-19T10:41:30.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:41:30.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:30.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc52467640 5 monclient: authenticate success, global_id 14527 2024-08-19T10:41:30.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 1 -- 172.21.15.190:0/1792445919 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc3c00cc30 con 0x7efc4c073f40 2024-08-19T10:41:30.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc52467640 10 monclient: _renew_subs 2024-08-19T10:41:30.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc52467640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:30.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc52467640 1 -- 172.21.15.190:0/1792445919 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c13b670 con 0x7efc4c073f40 2024-08-19T10:41:30.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:30.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 1 -- 172.21.15.190:0/1792445919 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efc3c00b040 con 0x7efc4c073f40 2024-08-19T10:41:30.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:30.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:30.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc4a7fc640 4 set_mon_vals no callback set 2024-08-19T10:41:30.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 10 monclient: dump: 2024-08-19T10:41:30.239 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:30.239 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:30.239 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.239 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.239 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:30.240 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.240 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:30.240 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:30.240 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:30.240 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:30.240 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:30.240 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:30.240 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:30.240 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:30.240 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:30.240 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc52467640 10 monclient: _renew_subs 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc52467640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc52467640 1 -- 172.21.15.190:0/1792445919 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c0a5c00 con 0x7efc4c073f40 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc52467640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc52467640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc52467640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc52467640 1 -- 172.21.15.190:0/1792445919 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c136000 con 0x7efc4c073f40 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 1 -- 172.21.15.190:0/1792445919 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efc3c019070 con 0x7efc4c073f40 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 1 --2- 172.21.15.190:0/1792445919 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efc3803be00 0x7efc3803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc48ff9640 1 -- 172.21.15.190:0/1792445919 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6105+0+0 (secure 0 0 0) 0x7efc3c04e9d0 con 0x7efc4c073f40 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc4b7fe640 1 --2- 172.21.15.190:0/1792445919 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efc3803be00 0x7efc3803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:30.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.217+0000 7efc4b7fe640 10 monclient: get_auth_request con 0x7efc3803be00 auth_method 0 2024-08-19T10:41:30.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.221+0000 7efc4b7fe640 1 --2- 172.21.15.190:0/1792445919 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efc3803be00 0x7efc3803e2c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc4c139980 tx=0x7efc40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:30.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.221+0000 7efc48ff9640 1 -- 172.21.15.190:0/1792445919 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7efc4c131510 con 0x7efc4c073f40 2024-08-19T10:41:30.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.221+0000 7efc48ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:30.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.221+0000 7efc48ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:30.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.397+0000 7efc52467640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:30.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.397+0000 7efc52467640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:30.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.397+0000 7efc52467640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:30.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.397+0000 7efc52467640 1 -- 172.21.15.190:0/1792445919 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7efc4c13af40 con 0x7efc4c073f40 2024-08-19T10:41:30.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.397+0000 7efc48ff9640 1 -- 172.21.15.190:0/1792445919 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=-16 (16) Device or resource busy v93) ==== 103+0+0 (secure 0 0 0) 0x7efc4c13af40 con 0x7efc4c073f40 2024-08-19T10:41:30.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.397+0000 7efc48ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:30.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.397+0000 7efc48ff9640 10 monclient: _finish_command 2 = mon:16 (16) Device or resource busy 2024-08-19T10:41:30.402 INFO:tasks.workunit.client.0.smithi190.stderr:Error EBUSY: (16) Device or resource busy 2024-08-19T10:41:30.407 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:41:30.407 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rule rm foo-rule 2024-08-19T10:41:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient: get_monmap_and_config 2024-08-19T10:41:30.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient: build_initial_monmap 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient: monmap: 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:30.521988+0000 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:30.521988+0000 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:30.522 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:30.523 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:30.523 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:30.523 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:30.523 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:30.523 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:30.523 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 1 Processor -- start 2024-08-19T10:41:30.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 1 -- start start 2024-08-19T10:41:30.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient: init 2024-08-19T10:41:30.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:30.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient: _add_conns ranks=[1,3,0,2,7,6,8,10,4,5,9] 2024-08-19T10:41:30.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff034139a60 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:30.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0340a5730 0x7ff0340a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient(hunting): picked mon.noname-g con 0x7ff0340a5730 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff03413e8c0 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0340a33c0 con 0x7ff03413e8c0 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0340a3640 con 0x7ff034139a60 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff03409ca60 con 0x7ff0340a5730 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff03a76f640 10 monclient(hunting): authenticate will time out at 1136.091797s 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff0337fe640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0340a5730 msgr2=0x7ff0340a5b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff0337fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0340a5730 0x7ff0340a5b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff033fff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff034139a60 0x7ff034139e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:34054/0 (socket says 172.21.15.190:34054) 2024-08-19T10:41:30.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.517+0000 7ff033fff640 1 -- 172.21.15.190:0/1323483887 learned_addr learned my addr 172.21.15.190:0/1323483887 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:30.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 1 -- 172.21.15.190:0/1323483887 >> v1:172.21.15.190:6789/0 conn(0x7ff034139a60 legacy=0x7ff034139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1620763107 0 0) 0x7ff024001460 con 0x7ff034139a60 2024-08-19T10:41:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): _negotiate 2024-08-19T10:41:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): my global_id is 14536 2024-08-19T10:41:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff01c003530 con 0x7ff034139a60 2024-08-19T10:41:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff038d0c640 1 -- 172.21.15.190:0/1323483887 >> v1:172.21.15.89:6789/0 conn(0x7ff03413e8c0 legacy=0x7ff034140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3664248860 0 0) 0x7ff01c003530 con 0x7ff034139a60 2024-08-19T10:41:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0340a5730 msgr2=0x7ff0340a5b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 --2- 172.21.15.190:0/1323483887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0340a5730 0x7ff0340a5b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 >> v1:172.21.15.89:6789/0 conn(0x7ff03413e8c0 legacy=0x7ff034140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 monclient: found mon.noname-f-legacy 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.190:6789/0 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0340a46e0 con 0x7ff034139a60 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient: _finish_auth 0 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff03a76f640 5 monclient: authenticate success, global_id 14536 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 3927028378 0 0) 0x7ff0240048b0 con 0x7ff034139a60 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-08-19T10:41:30.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient: dump: 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:30.530 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] but i'm connected to v1:172.21.15.190:6789/0, reconnecting 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 >> v1:172.21.15.190:6789/0 conn(0x7ff034139a60 legacy=0x7ff034139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient: _add_conns ranks=[7,2,8,4,1,0,5,3,6] 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 --2- 172.21.15.190:0/1323483887 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff01c004210 0x7ff01c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): picked mon.i con 0x7ff01c004210 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 --2- 172.21.15.190:0/1323483887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff01c004f00 0x7ff01c0099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): picked mon.b con 0x7ff01c004f00 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 --2- 172.21.15.190:0/1323483887 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff01c009f00 0x7ff01c00c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): picked mon.e con 0x7ff01c009f00 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff01c00cb00 con 0x7ff01c004f00 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff01c00ccb0 con 0x7ff01c004210 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff01c00ce60 con 0x7ff01c009f00 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff032ffd640 10 monclient(hunting): _finish_auth 0 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff03a76f640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 1 --2- 172.21.15.190:0/1323483887 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff01c004210 0x7ff01c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient(hunting): get_auth_request con 0x7ff01c004210 auth_method 0 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:30.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient(hunting): handle_auth_done global_id 14536 payload 293 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 1 -- 172.21.15.190:0/1323483887 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff01c009f00 msgr2=0x7ff01c00c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 1 --2- 172.21.15.190:0/1323483887 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff01c009f00 0x7ff01c00c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 1 -- 172.21.15.190:0/1323483887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff01c004f00 msgr2=0x7ff01c0099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 1 --2- 172.21.15.190:0/1323483887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff01c004f00 0x7ff01c0099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 1 monclient: found mon.i 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:30.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 1 -- 172.21.15.190:0/1323483887 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c003530 con 0x7ff01c004210 2024-08-19T10:41:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff033fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:30.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.521+0000 7ff038d0c640 1 -- 172.21.15.190:0/1323483887 reap_dead start 2024-08-19T10:41:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff033fff640 1 -- 172.21.15.190:0/1323483887 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff01c004210 msgr2=0x7ff01c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff033fff640 1 --2- 172.21.15.190:0/1323483887 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff01c004210 0x7ff01c004810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff0240016b0 tx=0x7ff02401ca40 comp rx=0 tx=0).ready entity=mon.7 client_cookie=5408eb50daffd536 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff024008c20 con 0x7ff01c004210 2024-08-19T10:41:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff032ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:30.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff032ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff032ffd640 10 monclient: dump: 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:30.540 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff03a76f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff033fff640 1 --2- 172.21.15.190:0/1323483887 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff01c004210 0x7ff01c004810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff0240016b0 tx=0x7ff02401ca40 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7ff0240081b0 mon_map magic: 0 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0240058e0 con 0x7ff01c004210 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff032ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff03a76f640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff03a76f640 4 set_mon_vals no callback set 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff03a76f640 10 monclient: shutdown 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff03a76f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff03a76f640 1 -- 172.21.15.190:0/1323483887 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff01c004210 msgr2=0x7ff01c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:30.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff03a76f640 1 --2- 172.21.15.190:0/1323483887 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff01c004210 0x7ff01c004810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff0240016b0 tx=0x7ff02401ca40 comp rx=0 tx=0).stop 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff032ffd640 1 -- 172.21.15.190:0/1323483887 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 0 0 0) 0x7ff0240081b0 con 0x7ff01c004210 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff032ffd640 10 monclient: discarding stray monitor message mon_map magic: 0 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.525+0000 7ff03a76f640 1 -- 172.21.15.190:0/1323483887 shutdown_connections 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 --2- 172.21.15.190:0/1323483887 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff01c004210 0x7ff01c004810 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 -- 172.21.15.190:0/1323483887 >> 172.21.15.190:0/1323483887 conn(0x7ff0340abe50 msgr2=0x7ff0340ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 -- 172.21.15.190:0/1323483887 shutdown_connections 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 -- 172.21.15.190:0/1323483887 wait complete. 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient: build_initial_monmap 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient: monmap: 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:30.542 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 Processor -- start 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 -- start start 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient: init 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient: _add_conns ranks=[2,4,7,5,3,6,8,1,0] 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff01c004210 0x7ff034123af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient(hunting): picked mon.noname-b con 0x7ff01c004210 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff034124030 0x7ff034124490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient(hunting): picked mon.noname-c con 0x7ff034124030 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0341249d0 0x7ff0341d7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient(hunting): picked mon.noname-i con 0x7ff0341249d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff0340a69d0 con 0x7ff01c004210 2024-08-19T10:41:30.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff0340a6850 con 0x7ff034124030 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff0340a6b50 con 0x7ff0341249d0 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff03a76f640 10 monclient(hunting): authenticate will time out at 1136.101658s 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0341249d0 0x7ff0341d7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0341249d0 0x7ff0341d7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:46254/0 (socket says 172.21.15.190:46254) 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 1 -- 172.21.15.190:0/3413970681 learned_addr learned my addr 172.21.15.190:0/3413970681 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 10 monclient(hunting): get_auth_request con 0x7ff0341249d0 auth_method 0 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:30.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff033fff640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff01c004210 0x7ff034123af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff033fff640 10 monclient(hunting): get_auth_request con 0x7ff01c004210 auth_method 0 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff033fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff033fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff033fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 10 monclient(hunting): handle_auth_done global_id 14497 payload 824 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff0337fe640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff034124030 0x7ff034124490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 1 -- 172.21.15.190:0/3413970681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff034124030 msgr2=0x7ff034124490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff034124030 0x7ff034124490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 1 -- 172.21.15.190:0/3413970681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff01c004210 msgr2=0x7ff034123af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:30.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff01c004210 0x7ff034123af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 10 monclient: _finish_hunting 0 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 1 monclient: found mon.noname-i 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.529+0000 7ff038d0c640 1 -- 172.21.15.190:0/3413970681 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff034126a90 con 0x7ff0341249d0 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff038d0c640 1 -- 172.21.15.190:0/3413970681 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0341249d0 msgr2=0x7ff0341d7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff038d0c640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0341249d0 0x7ff0341d7ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff0340a4bd0 tx=0x7ff02c00cad0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 1 -- 172.21.15.190:0/3413970681 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff02c017070 con 0x7ff0341249d0 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 10 monclient: dump: 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:30.547 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:30.548 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 10 monclient: _finish_auth 0 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 1 -- 172.21.15.190:0/3413970681 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02c007540 con 0x7ff0341249d0 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff03a76f640 5 monclient: authenticate success, global_id 14497 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff03a76f640 10 monclient: _renew_subs 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff03a76f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff03a76f640 1 -- 172.21.15.190:0/3413970681 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0341255a0 con 0x7ff0341249d0 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 1 -- 172.21.15.190:0/3413970681 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff02c011360 con 0x7ff0341249d0 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff031ffb640 4 set_mon_vals no callback set 2024-08-19T10:41:30.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 10 monclient: dump: 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:30.550 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff03a76f640 10 monclient: _renew_subs 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff03a76f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff03a76f640 1 -- 172.21.15.190:0/3413970681 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff034124e50 con 0x7ff0341249d0 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff03a76f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff03a76f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff03a76f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff03a76f640 1 -- 172.21.15.190:0/3413970681 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0340a6ee0 con 0x7ff0341249d0 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 1 -- 172.21.15.190:0/3413970681 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff02c0076f0 con 0x7ff0341249d0 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff02003be30 0x7ff02003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:30.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff030ff9640 1 -- 172.21.15.190:0/3413970681 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(93..93 src has 1..93) ==== 6105+0+0 (secure 0 0 0) 0x7ff02c04e2f0 con 0x7ff0341249d0 2024-08-19T10:41:30.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff033fff640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff02003be30 0x7ff02003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:30.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.533+0000 7ff033fff640 10 monclient: get_auth_request con 0x7ff02003be30 auth_method 0 2024-08-19T10:41:30.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.537+0000 7ff033fff640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff02003be30 0x7ff02003e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0240a9000 tx=0x7ff03406a2d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:30.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.537+0000 7ff030ff9640 1 -- 172.21.15.190:0/3413970681 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff02c019310 con 0x7ff0341249d0 2024-08-19T10:41:30.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.537+0000 7ff030ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:30.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.537+0000 7ff030ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:30.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.705+0000 7ff03a76f640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-08-19T10:41:30.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.705+0000 7ff03a76f640 10 monclient: _send_command 2 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-08-19T10:41:30.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.705+0000 7ff03a76f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:30.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:30.705+0000 7ff03a76f640 1 -- 172.21.15.190:0/3413970681 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo-rule"} v 0) -- 0x7ff0340a3110 con 0x7ff0341249d0 2024-08-19T10:41:31.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.013+0000 7ff030ff9640 1 -- 172.21.15.190:0/3413970681 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo-rule"}]=0 v94) ==== 85+0+0 (secure 0 0 0) 0x7ff0340a3110 con 0x7ff0341249d0 2024-08-19T10:41:31.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.013+0000 7ff030ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-08-19T10:41:31.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.013+0000 7ff030ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:31.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.113+0000 7ff03a76f640 10 start_mon_command cmd=[{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-08-19T10:41:31.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.113+0000 7ff03a76f640 10 monclient: _send_command 3 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-08-19T10:41:31.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.113+0000 7ff03a76f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:31.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.113+0000 7ff03a76f640 1 -- 172.21.15.190:0/3413970681 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rule rm", "name": "foo-rule"} v 0) -- 0x7ff03413e560 con 0x7ff0341249d0 2024-08-19T10:41:31.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff030ff9640 1 -- 172.21.15.190:0/3413970681 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rule rm", "name": "foo-rule"}]=0 rule foo-rule does not exist v94) ==== 113+0+0 (secure 0 0 0) 0x7ff02c007300 con 0x7ff0341249d0 2024-08-19T10:41:31.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff030ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rule rm", "name": "foo-rule"}] 2024-08-19T10:41:31.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff030ff9640 10 monclient: _finish_command 3 = system:0 rule foo-rule does not exist 2024-08-19T10:41:31.119 INFO:tasks.workunit.client.0.smithi190.stderr:rule foo-rule does not exist 2024-08-19T10:41:31.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 -- 172.21.15.190:0/3413970681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff02003be30 msgr2=0x7ff02003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:31.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff02003be30 0x7ff02003e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0240a9000 tx=0x7ff03406a2d0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 10 monclient: shutdown 2024-08-19T10:41:31.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:31.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 -- 172.21.15.190:0/3413970681 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0341249d0 msgr2=0x7ff0341d7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:31.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0341249d0 0x7ff0341d7ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff0340a4bd0 tx=0x7ff02c00cad0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 -- 172.21.15.190:0/3413970681 shutdown_connections 2024-08-19T10:41:31.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff02003be30 0x7ff02003e2f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff034124030 0x7ff034124490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0341249d0 0x7ff0341d7ac0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 --2- 172.21.15.190:0/3413970681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff01c004210 0x7ff034123af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 -- 172.21.15.190:0/3413970681 >> 172.21.15.190:0/3413970681 conn(0x7ff0340abe50 msgr2=0x7ff034138aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:31.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 -- 172.21.15.190:0/3413970681 shutdown_connections 2024-08-19T10:41:31.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.117+0000 7ff03a76f640 1 -- 172.21.15.190:0/3413970681 wait complete. 2024-08-19T10:41:31.136 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm bar 2024-08-19T10:41:31.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.245+0000 7f095e1e6640 10 monclient: get_monmap_and_config 2024-08-19T10:41:31.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.245+0000 7f095e1e6640 10 monclient: build_initial_monmap 2024-08-19T10:41:31.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.245+0000 7f095e1e6640 10 monclient: monmap: 2024-08-19T10:41:31.250 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:31.250 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:31.250 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:31.250560+0000 2024-08-19T10:41:31.250 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:31.250560+0000 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:31.251 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:31.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.245+0000 7f095e1e6640 1 Processor -- start 2024-08-19T10:41:31.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.245+0000 7f095e1e6640 1 -- start start 2024-08-19T10:41:31.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.245+0000 7f095e1e6640 10 monclient: init 2024-08-19T10:41:31.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:31.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 10 monclient: _add_conns ranks=[4,8,9,5,10,2,3,6,0,1,7] 2024-08-19T10:41:31.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09580a56c0 0x7f0958071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:31.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 10 monclient(hunting): picked mon.noname-c con 0x7f09580a56c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:31.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f09580a5aa0 0x7f09580a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:31.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 10 monclient(hunting): picked mon.noname-e con 0x7f09580a5aa0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:31.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0958139490 0x7f0958139910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:31.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 10 monclient(hunting): picked mon.noname-a con 0x7f0958139490 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f095806de60 con 0x7f0958139490 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f095806e0e0 con 0x7f09580a56c0 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f095806f180 con 0x7f09580a5aa0 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f095e1e6640 10 monclient(hunting): authenticate will time out at 1136.820376s 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f09577fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09580a56c0 0x7f0958071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f09580a5aa0 0x7f09580a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0957fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0958139490 0x7f0958139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:31.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f09580a5aa0 0x7f09580a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:50888/0 (socket says 172.21.15.190:50888) 2024-08-19T10:41:31.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f09577fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09580a56c0 0x7f0958071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:50306/0 (socket says 172.21.15.190:50306) 2024-08-19T10:41:31.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 1 -- 172.21.15.190:0/2817782438 learned_addr learned my addr 172.21.15.190:0/2817782438 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:31.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 10 monclient(hunting): get_auth_request con 0x7f09580a5aa0 auth_method 0 2024-08-19T10:41:31.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:31.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:31.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:31.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f09577fe640 10 monclient(hunting): get_auth_request con 0x7f09580a56c0 auth_method 0 2024-08-19T10:41:31.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f09577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:31.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f09577fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f09577fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0957fff640 10 monclient(hunting): get_auth_request con 0x7f0958139490 auth_method 0 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0957fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0957fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0957fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f09577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f09577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f09577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0957fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0957fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:31.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0957fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:31.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 10 monclient(hunting): handle_auth_done global_id 5147 payload 274 2024-08-19T10:41:31.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 1 -- 172.21.15.190:0/2817782438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09580a56c0 msgr2=0x7f0958071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:31.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 1 --2- 172.21.15.190:0/2817782438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09580a56c0 0x7f0958071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 1 -- 172.21.15.190:0/2817782438 >> v2:172.21.15.89:3300/0 conn(0x7f0958139490 msgr2=0x7f0958139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:31.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 1 --2- 172.21.15.190:0/2817782438 >> v2:172.21.15.89:3300/0 conn(0x7f0958139490 0x7f0958139910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:31.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 1 monclient: found mon.noname-e 2024-08-19T10:41:31.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:31.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:31.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.249+0000 7f0956ffd640 1 -- 172.21.15.190:0/2817782438 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095806f460 con 0x7f09580a5aa0 2024-08-19T10:41:31.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f0956ffd640 1 -- 172.21.15.190:0/2817782438 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f09580a5aa0 msgr2=0x7f09580a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:31.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f0956ffd640 1 --2- 172.21.15.190:0/2817782438 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f09580a5aa0 0x7f09580a5ec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0940009870 tx=0x7f094001c4b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f71461b759e8f51f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:31.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 1 -- 172.21.15.190:0/2817782438 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09400b0070 con 0x7f09580a5aa0 2024-08-19T10:41:31.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:31.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:31.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 10 monclient: dump: 2024-08-19T10:41:31.266 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:31.266 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:31.266 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:31.266 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:31.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 1 -- 172.21.15.190:0/2817782438 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09400aa810 con 0x7f09580a5aa0 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 1 -- 172.21.15.190:0/2817782438 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09400a9300 con 0x7f09580a5aa0 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f09567fc640 10 monclient: dump: 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:31.268 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f095e1e6640 5 monclient: authenticate success, global_id 5147 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f095e1e6640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:31.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f095e1e6640 4 set_mon_vals no callback set 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f095e1e6640 10 monclient: shutdown 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f095e1e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f095e1e6640 1 -- 172.21.15.190:0/2817782438 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f09580a5aa0 msgr2=0x7f09580a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f095e1e6640 1 --2- 172.21.15.190:0/2817782438 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f09580a5aa0 0x7f09580a5ec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0940009870 tx=0x7f094001c4b0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f095e1e6640 1 -- 172.21.15.190:0/2817782438 shutdown_connections 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f095e1e6640 1 --2- 172.21.15.190:0/2817782438 >> v2:172.21.15.89:3300/0 conn(0x7f0958139490 0x7f0958139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f095e1e6640 1 --2- 172.21.15.190:0/2817782438 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f09580a5aa0 0x7f09580a5ec0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.253+0000 7f095e1e6640 1 --2- 172.21.15.190:0/2817782438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09580a56c0 0x7f0958071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 1 -- 172.21.15.190:0/2817782438 >> 172.21.15.190:0/2817782438 conn(0x7f09580aa2b0 msgr2=0x7f09580a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 1 -- 172.21.15.190:0/2817782438 shutdown_connections 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 1 -- 172.21.15.190:0/2817782438 wait complete. 2024-08-19T10:41:31.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient: build_initial_monmap 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient: monmap: 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:31.271 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 1 Processor -- start 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 1 -- start start 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient: init 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient: _add_conns ranks=[3,1,0,2,5,8,7,6,4] 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09580a56c0 0x7f09581223b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient(hunting): picked mon.noname-g con 0x7f09580a56c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09580a5aa0 0x7f09581d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient(hunting): picked mon.noname-f con 0x7f09580a5aa0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:31.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0958139490 0x7f0958120300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient(hunting): picked mon.noname-a con 0x7f0958139490 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f09580a6400 con 0x7f0958139490 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0958072520 con 0x7f09580a5aa0 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0958143180 con 0x7f09580a56c0 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f095e1e6640 10 monclient(hunting): authenticate will time out at 1136.829638s 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f09577fe640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09580a56c0 msgr2=0x7f09581223b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f09577fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09580a56c0 0x7f09581223b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09580a5aa0 0x7f09581d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:31.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09580a5aa0 0x7f09581d16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34508/0 (socket says 172.21.15.190:34508) 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 1 -- 172.21.15.190:0/1302269329 learned_addr learned my addr 172.21.15.190:0/1302269329 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 10 monclient(hunting): get_auth_request con 0x7f09580a5aa0 auth_method 0 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0957fff640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0958139490 0x7f0958120300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0957fff640 10 monclient(hunting): get_auth_request con 0x7f0958139490 auth_method 0 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0957fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0957fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:31.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0957fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 10 monclient(hunting): handle_auth_done global_id 14545 payload 824 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 1 -- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09580a56c0 msgr2=0x7f09581223b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09580a56c0 0x7f09581223b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 1 -- 172.21.15.190:0/1302269329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0958139490 msgr2=0x7f0958120300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0958139490 0x7f0958120300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 1 monclient: found mon.noname-f 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.257+0000 7f0956ffd640 1 -- 172.21.15.190:0/1302269329 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0958123bb0 con 0x7f09580a5aa0 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f0956ffd640 1 -- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09580a5aa0 msgr2=0x7f09581d16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f0956ffd640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09580a5aa0 0x7f09581d16b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f094001c2e0 tx=0x7f0940019250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 1 -- 172.21.15.190:0/1302269329 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09400ba070 con 0x7f09580a5aa0 2024-08-19T10:41:31.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 10 monclient: dump: 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:31.276 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 10 monclient: _finish_auth 0 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 1 -- 172.21.15.190:0/1302269329 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09400b1040 con 0x7f09580a5aa0 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f095e1e6640 5 monclient: authenticate success, global_id 14545 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f095e1e6640 10 monclient: _renew_subs 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f095e1e6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f095e1e6640 1 -- 172.21.15.190:0/1302269329 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09581217d0 con 0x7f09580a5aa0 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:31.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 1 -- 172.21.15.190:0/1302269329 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09400b0070 con 0x7f09580a5aa0 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 10 monclient: dump: 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:31.278 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f09557fa640 4 set_mon_vals no callback set 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f095e1e6640 10 monclient: _renew_subs 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f095e1e6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f095e1e6640 1 -- 172.21.15.190:0/1302269329 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09581235f0 con 0x7f09580a5aa0 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f095e1e6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f095e1e6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:31.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f095e1e6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:31.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f095e1e6640 1 -- 172.21.15.190:0/1302269329 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09580a5ec0 con 0x7f09580a5aa0 2024-08-19T10:41:31.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 1 -- 172.21.15.190:0/1302269329 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0940004040 con 0x7f09580a5aa0 2024-08-19T10:41:31.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f093803be00 0x7f093803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:31.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f094ffff640 1 -- 172.21.15.190:0/1302269329 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6045+0+0 (secure 0 0 0) 0x7f09400e8160 con 0x7f09580a5aa0 2024-08-19T10:41:31.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f09577fe640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f093803be00 0x7f093803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:31.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.261+0000 7f09577fe640 10 monclient: get_auth_request con 0x7f093803be00 auth_method 0 2024-08-19T10:41:31.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.265+0000 7f09577fe640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f093803be00 0x7f093803e2c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0948009b30 tx=0x7f094800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:31.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.265+0000 7f094ffff640 1 -- 172.21.15.190:0/1302269329 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0940016130 con 0x7f09580a5aa0 2024-08-19T10:41:31.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.265+0000 7f094ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:31.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.265+0000 7f094ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:31.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.429+0000 7f095e1e6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:31.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.429+0000 7f095e1e6640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:31.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.429+0000 7f095e1e6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:31.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:31.429+0000 7f095e1e6640 1 -- 172.21.15.190:0/1302269329 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) -- 0x7f09581237c0 con 0x7f09580a5aa0 2024-08-19T10:41:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.017+0000 7f094ffff640 1 -- 172.21.15.190:0/1302269329 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=0 removed item id -19 name 'bar' from crush map v95) ==== 120+0+0 (secure 0 0 0) 0x7f09400bcdf0 con 0x7f09580a5aa0 2024-08-19T10:41:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.017+0000 7f094ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.017+0000 7f094ffff640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'bar' from crush map 2024-08-19T10:41:32.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.125+0000 7f095e1e6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:32.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.125+0000 7f095e1e6640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:32.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.125+0000 7f095e1e6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:32.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.125+0000 7f095e1e6640 1 -- 172.21.15.190:0/1302269329 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "bar"} v 0) -- 0x7f09581c8010 con 0x7f09580a5aa0 2024-08-19T10:41:32.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f094ffff640 1 -- 172.21.15.190:0/1302269329 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "bar"}]=0 device 'bar' does not appear in the crush map v95) ==== 120+0+0 (secure 0 0 0) 0x7f09400b4da0 con 0x7f09580a5aa0 2024-08-19T10:41:32.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f094ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "bar"}] 2024-08-19T10:41:32.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f094ffff640 10 monclient: _finish_command 3 = system:0 device 'bar' does not appear in the crush map 2024-08-19T10:41:32.131 INFO:tasks.workunit.client.0.smithi190.stderr:device 'bar' does not appear in the crush map 2024-08-19T10:41:32.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 -- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f093803be00 msgr2=0x7f093803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:32.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f093803be00 0x7f093803e2c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0948009b30 tx=0x7f094800b040 comp rx=0 tx=0).stop 2024-08-19T10:41:32.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 10 monclient: shutdown 2024-08-19T10:41:32.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:32.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 -- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09580a5aa0 msgr2=0x7f09581d16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:32.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09580a5aa0 0x7f09581d16b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f094001c2e0 tx=0x7f0940019250 comp rx=0 tx=0).stop 2024-08-19T10:41:32.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 -- 172.21.15.190:0/1302269329 shutdown_connections 2024-08-19T10:41:32.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f093803be00 0x7f093803e2c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:32.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0958139490 0x7f0958120300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:32.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09580a5aa0 0x7f09581d16b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:32.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 --2- 172.21.15.190:0/1302269329 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09580a56c0 0x7f09581223b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:32.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 -- 172.21.15.190:0/1302269329 >> 172.21.15.190:0/1302269329 conn(0x7f09580aa2b0 msgr2=0x7f0958072730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:32.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 -- 172.21.15.190:0/1302269329 shutdown_connections 2024-08-19T10:41:32.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.129+0000 7f095e1e6640 1 -- 172.21.15.190:0/1302269329 wait complete. 2024-08-19T10:41:32.149 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm foo 2024-08-19T10:41:32.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.261+0000 7efe5982f640 10 monclient: get_monmap_and_config 2024-08-19T10:41:32.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.261+0000 7efe5982f640 10 monclient: build_initial_monmap 2024-08-19T10:41:32.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.261+0000 7efe5982f640 10 monclient: monmap: 2024-08-19T10:41:32.265 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:32.265 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:32.265 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:32.265848+0000 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:32.265848+0000 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:32.266 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:32.267 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:32.267 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:32.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.261+0000 7efe5982f640 1 Processor -- start 2024-08-19T10:41:32.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.261+0000 7efe5982f640 1 -- start start 2024-08-19T10:41:32.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.261+0000 7efe5982f640 10 monclient: init 2024-08-19T10:41:32.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:32.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 10 monclient: _add_conns ranks=[5,6,10,9,0,2,1,8,3,4,7] 2024-08-19T10:41:32.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe540a3b90 0x7efe540a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:32.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 10 monclient(hunting): picked mon.noname-h con 0x7efe540a3b90 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efe540a44b0 0x7efe540a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 10 monclient(hunting): picked mon.noname-d con 0x7efe540a44b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efe54136570 0x7efe54136a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 10 monclient(hunting): picked mon.noname-f con 0x7efe54136570 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7efe540a04b0 con 0x7efe54136570 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7efe540a1ba0 con 0x7efe540a3b90 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7efe540a1e80 con 0x7efe540a44b0 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:32.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe5982f640 10 monclient(hunting): authenticate will time out at 1137.836527s 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe540a3b90 0x7efe540a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe537fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efe54136570 0x7efe54136a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe540a3b90 0x7efe540a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48264/0 (socket says 172.21.15.190:48264) 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 1 -- 172.21.15.190:0/2782120473 learned_addr learned my addr 172.21.15.190:0/2782120473 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 10 monclient(hunting): get_auth_request con 0x7efe540a3b90 auth_method 0 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe537fe640 10 monclient(hunting): get_auth_request con 0x7efe54136570 auth_method 0 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe537fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe537fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:32.271 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe4a7fc640 1 --2- 172.21.15.190:0/2782120473 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efe540a44b0 0x7efe540a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe4a7fc640 10 monclient(hunting): get_auth_request con 0x7efe540a44b0 auth_method 0 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe4a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe4a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe4a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 10 monclient(hunting): handle_auth_done global_id 5288 payload 274 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 1 -- 172.21.15.190:0/2782120473 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efe540a44b0 msgr2=0x7efe540a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:32.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 1 --2- 172.21.15.190:0/2782120473 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efe540a44b0 0x7efe540a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:32.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 1 -- 172.21.15.190:0/2782120473 >> v2:172.21.15.190:3300/0 conn(0x7efe54136570 msgr2=0x7efe54136a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:32.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 1 --2- 172.21.15.190:0/2782120473 >> v2:172.21.15.190:3300/0 conn(0x7efe54136570 0x7efe54136a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:32.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:32.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 1 monclient: found mon.noname-h 2024-08-19T10:41:32.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:32.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:32.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.265+0000 7efe52ffd640 1 -- 172.21.15.190:0/2782120473 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5409fe50 con 0x7efe540a3b90 2024-08-19T10:41:32.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 1 -- 172.21.15.190:0/2782120473 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe540a3b90 msgr2=0x7efe540a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:32.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 1 --2- 172.21.15.190:0/2782120473 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe540a3b90 0x7efe540a3f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efe44009870 tx=0x7efe4401c540 comp rx=0 tx=0).ready entity=mon.5 client_cookie=184e4b6497369322 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 1 -- 172.21.15.190:0/2782120473 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe440b1070 con 0x7efe540a3b90 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 10 monclient: dump: 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:32.277 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:32.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 5 monclient: authenticate success, global_id 5288 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 1 -- 172.21.15.190:0/2782120473 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe440ab840 con 0x7efe540a3b90 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 4 set_mon_vals no callback set 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 1 -- 172.21.15.190:0/2782120473 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe440a8050 con 0x7efe540a3b90 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe527fc640 10 monclient: dump: 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:32.279 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient: shutdown 2024-08-19T10:41:32.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 -- 172.21.15.190:0/2782120473 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe540a3b90 msgr2=0x7efe540a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 --2- 172.21.15.190:0/2782120473 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe540a3b90 0x7efe540a3f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efe44009870 tx=0x7efe4401c540 comp rx=0 tx=0).stop 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 -- 172.21.15.190:0/2782120473 shutdown_connections 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 --2- 172.21.15.190:0/2782120473 >> v2:172.21.15.190:3300/0 conn(0x7efe54136570 0x7efe54136a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 --2- 172.21.15.190:0/2782120473 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efe540a44b0 0x7efe540a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 --2- 172.21.15.190:0/2782120473 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efe540a3b90 0x7efe540a3f70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 -- 172.21.15.190:0/2782120473 >> 172.21.15.190:0/2782120473 conn(0x7efe5406dde0 msgr2=0x7efe5406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 -- 172.21.15.190:0/2782120473 shutdown_connections 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 -- 172.21.15.190:0/2782120473 wait complete. 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient: build_initial_monmap 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient: monmap: 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:32.281 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:32.282 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 Processor -- start 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 -- start start 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient: init 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient: _add_conns ranks=[1,2,4,0,6,5,7,3,8] 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe540a3b90 0x7efe5411f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient(hunting): picked mon.noname-f con 0x7efe540a3b90 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe540a44b0 0x7efe5411f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient(hunting): picked mon.noname-b con 0x7efe540a44b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe54136570 0x7efe5411fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient(hunting): picked mon.noname-c con 0x7efe54136570 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:32.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efe540a0b00 con 0x7efe540a3b90 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7efe540a2c80 con 0x7efe540a44b0 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7efe5413ffe0 con 0x7efe54136570 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe5982f640 10 monclient(hunting): authenticate will time out at 1137.843660s 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe540a3b90 0x7efe5411f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe540a3b90 0x7efe5411f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34522/0 (socket says 172.21.15.190:34522) 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 1 -- 172.21.15.190:0/2620735696 learned_addr learned my addr 172.21.15.190:0/2620735696 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 10 monclient(hunting): get_auth_request con 0x7efe540a3b90 auth_method 0 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:32.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.269+0000 7efe52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 10 monclient(hunting): handle_auth_done global_id 14563 payload 824 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe537fe640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe54136570 0x7efe5411fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 1 -- 172.21.15.190:0/2620735696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe54136570 msgr2=0x7efe5411fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe4a7fc640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe540a44b0 0x7efe5411f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe54136570 0x7efe5411fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 1 -- 172.21.15.190:0/2620735696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe540a44b0 msgr2=0x7efe5411f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe540a44b0 0x7efe5411f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 1 monclient: found mon.noname-f 2024-08-19T10:41:32.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:32.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:32.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 1 -- 172.21.15.190:0/2620735696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5406e3a0 con 0x7efe540a3b90 2024-08-19T10:41:32.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 1 -- 172.21.15.190:0/2620735696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe540a3b90 msgr2=0x7efe5411f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:32.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe52ffd640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe540a3b90 0x7efe5411f400 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efe440097c0 tx=0x7efe440192e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:32.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 1 -- 172.21.15.190:0/2620735696 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe440ba070 con 0x7efe540a3b90 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 10 monclient: dump: 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:32.288 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 10 monclient: _finish_auth 0 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe5982f640 5 monclient: authenticate success, global_id 14563 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 1 -- 172.21.15.190:0/2620735696 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe440044b0 con 0x7efe540a3b90 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe5982f640 10 monclient: _renew_subs 2024-08-19T10:41:32.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe5982f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe5982f640 1 -- 172.21.15.190:0/2620735696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe54121b30 con 0x7efe540a3b90 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 1 -- 172.21.15.190:0/2620735696 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efe440b1040 con 0x7efe540a3b90 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe51ffb640 4 set_mon_vals no callback set 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 10 monclient: dump: 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:32.290 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe5982f640 10 monclient: _renew_subs 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe5982f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:32.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe5982f640 1 -- 172.21.15.190:0/2620735696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5413be80 con 0x7efe540a3b90 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe5982f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe5982f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe5982f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe5982f640 1 -- 172.21.15.190:0/2620735696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe540a3f70 con 0x7efe540a3b90 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.273+0000 7efe50ff9640 1 -- 172.21.15.190:0/2620735696 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efe44004040 con 0x7efe540a3b90 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.277+0000 7efe50ff9640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe3403be30 0x7efe3403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.277+0000 7efe50ff9640 1 -- 172.21.15.190:0/2620735696 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5916+0+0 (secure 0 0 0) 0x7efe440e7ff0 con 0x7efe540a3b90 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.277+0000 7efe4a7fc640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe3403be30 0x7efe3403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.277+0000 7efe4a7fc640 10 monclient: get_auth_request con 0x7efe3403be30 auth_method 0 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.277+0000 7efe4a7fc640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe3403be30 0x7efe3403e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efe5413c4f0 tx=0x7efe3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.281+0000 7efe50ff9640 1 -- 172.21.15.190:0/2620735696 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7efe5406e3a0 con 0x7efe540a3b90 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.281+0000 7efe50ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:32.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.281+0000 7efe50ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:32.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.445+0000 7efe5982f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:32.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.445+0000 7efe5982f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:32.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.445+0000 7efe5982f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:32.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:32.445+0000 7efe5982f640 1 -- 172.21.15.190:0/2620735696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7efe5406ecb0 con 0x7efe540a3b90 2024-08-19T10:41:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.381+0000 7efe50ff9640 1 -- 172.21.15.190:0/2620735696 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 removed item id -10 name 'foo' from crush map v96) ==== 120+0+0 (secure 0 0 0) 0x7efe44016130 con 0x7efe540a3b90 2024-08-19T10:41:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.381+0000 7efe50ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.381+0000 7efe50ff9640 10 monclient: _finish_command 2 = system:0 removed item id -10 name 'foo' from crush map 2024-08-19T10:41:33.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.493+0000 7efe5982f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:33.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.493+0000 7efe5982f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:33.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.493+0000 7efe5982f640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:33.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.493+0000 7efe5982f640 1 -- 172.21.15.190:0/2620735696 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7efe54120610 con 0x7efe540a3b90 2024-08-19T10:41:33.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.493+0000 7efe50ff9640 1 -- 172.21.15.190:0/2620735696 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 device 'foo' does not appear in the crush map v96) ==== 120+0+0 (secure 0 0 0) 0x7efe440b4380 con 0x7efe540a3b90 2024-08-19T10:41:33.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.493+0000 7efe50ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:33.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.493+0000 7efe50ff9640 10 monclient: _finish_command 3 = system:0 device 'foo' does not appear in the crush map 2024-08-19T10:41:33.498 INFO:tasks.workunit.client.0.smithi190.stderr:device 'foo' does not appear in the crush map 2024-08-19T10:41:33.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 -- 172.21.15.190:0/2620735696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe3403be30 msgr2=0x7efe3403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:33.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe3403be30 0x7efe3403e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efe5413c4f0 tx=0x7efe3c008040 comp rx=0 tx=0).stop 2024-08-19T10:41:33.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 10 monclient: shutdown 2024-08-19T10:41:33.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:33.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 -- 172.21.15.190:0/2620735696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe540a3b90 msgr2=0x7efe5411f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:33.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe540a3b90 0x7efe5411f400 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efe440097c0 tx=0x7efe440192e0 comp rx=0 tx=0).stop 2024-08-19T10:41:33.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 -- 172.21.15.190:0/2620735696 shutdown_connections 2024-08-19T10:41:33.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efe3403be30 0x7efe3403e2f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:33.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efe540a44b0 0x7efe5411f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:33.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efe54136570 0x7efe5411fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:33.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 --2- 172.21.15.190:0/2620735696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe540a3b90 0x7efe5411f400 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:33.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 -- 172.21.15.190:0/2620735696 >> 172.21.15.190:0/2620735696 conn(0x7efe5406dde0 msgr2=0x7efe5406c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:33.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 -- 172.21.15.190:0/2620735696 shutdown_connections 2024-08-19T10:41:33.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.497+0000 7efe5982f640 1 -- 172.21.15.190:0/2620735696 wait complete. 2024-08-19T10:41:33.515 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm osd.9 host2 2024-08-19T10:41:33.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.629+0000 7f36618d7640 10 monclient: get_monmap_and_config 2024-08-19T10:41:33.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.629+0000 7f36618d7640 10 monclient: build_initial_monmap 2024-08-19T10:41:33.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.629+0000 7f36618d7640 10 monclient: monmap: 2024-08-19T10:41:33.634 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:33.634 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:33.634 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:33.634351+0000 2024-08-19T10:41:33.634 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:33.634351+0000 2024-08-19T10:41:33.634 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:33.635 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:33.636 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:33.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.629+0000 7f36618d7640 1 Processor -- start 2024-08-19T10:41:33.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.629+0000 7f36618d7640 1 -- start start 2024-08-19T10:41:33.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.629+0000 7f36618d7640 10 monclient: init 2024-08-19T10:41:33.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:33.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 10 monclient: _add_conns ranks=[8,1,10,5,7,3,9,0,4,6,2] 2024-08-19T10:41:33.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f365c0a3b90 0x7f365c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:33.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 10 monclient(hunting): picked mon.noname-e con 0x7f365c0a3b90 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:33.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f365c0a44b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:33.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f365c0a48b0 0x7f365c136c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:33.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 10 monclient(hunting): picked mon.noname-f con 0x7f365c0a48b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f365c0a1ba0 con 0x7f365c0a44b0 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f365c0a04b0 con 0x7f365c0a48b0 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f365c0a4d10 con 0x7f365c0a3b90 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f36618d7640 10 monclient(hunting): authenticate will time out at 1139.204622s 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f3653fff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f365c0a44b0 0x7f365c136570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:34066/0 (socket says 172.21.15.190:34066) 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f3653fff640 1 -- 172.21.15.190:0/2663772850 learned_addr learned my addr 172.21.15.190:0/2663772850 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365b7fe640 1 --2- 172.21.15.190:0/2663772850 >> v2:172.21.15.190:3300/0 conn(0x7f365c0a48b0 0x7f365c136c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365b7fe640 10 monclient(hunting): get_auth_request con 0x7f365c0a48b0 auth_method 0 2024-08-19T10:41:33.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 1 --2- 172.21.15.190:0/2663772850 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f365c0a3b90 0x7f365c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 10 monclient(hunting): get_auth_request con 0x7f365c0a3b90 auth_method 0 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 10 monclient(hunting): handle_auth_done global_id 5156 payload 274 2024-08-19T10:41:33.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 1 -- 172.21.15.190:0/2663772850 >> v2:172.21.15.190:3300/0 conn(0x7f365c0a48b0 msgr2=0x7f365c136c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:33.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 1 --2- 172.21.15.190:0/2663772850 >> v2:172.21.15.190:3300/0 conn(0x7f365c0a48b0 0x7f365c136c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:33.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 1 -- 172.21.15.190:0/2663772850 >> v1:172.21.15.190:6789/0 conn(0x7f365c0a44b0 legacy=0x7f365c136570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:33.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:33.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 1 monclient: found mon.noname-e 2024-08-19T10:41:33.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:33.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:33.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.633+0000 7f365affd640 1 -- 172.21.15.190:0/2663772850 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c0a1e80 con 0x7f365c0a3b90 2024-08-19T10:41:33.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365affd640 1 -- 172.21.15.190:0/2663772850 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f365c0a3b90 msgr2=0x7f365c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:33.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365affd640 1 --2- 172.21.15.190:0/2663772850 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f365c0a3b90 0x7f365c0a3f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f364c0097c0 tx=0x7f364c01c3f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9c7c6ca0e792013 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:33.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 1 -- 172.21.15.190:0/2663772850 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f364c0b0070 con 0x7f365c0a3b90 2024-08-19T10:41:33.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:33.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:33.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 10 monclient: dump: 2024-08-19T10:41:33.645 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:33.645 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:33.645 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:33.645 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:33.645 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:33.646 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:33.646 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:33.646 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:33.646 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:33.646 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:33.646 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:33.646 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:33.646 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:33.646 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:33.646 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:33.646 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 1 -- 172.21.15.190:0/2663772850 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f364c0aa810 con 0x7f365c0a3b90 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 5 monclient: authenticate success, global_id 5156 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 1 -- 172.21.15.190:0/2663772850 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f364c0a9300 con 0x7f365c0a3b90 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 4 set_mon_vals no callback set 2024-08-19T10:41:33.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:33.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f365a7fc640 10 monclient: dump: 2024-08-19T10:41:33.648 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:33.648 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:33.648 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:33.648 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:33.648 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:33.648 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:33.648 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:33.648 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:33.648 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:33.648 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 10 monclient: shutdown 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 1 -- 172.21.15.190:0/2663772850 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f365c0a3b90 msgr2=0x7f365c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 1 --2- 172.21.15.190:0/2663772850 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f365c0a3b90 0x7f365c0a3f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f364c0097c0 tx=0x7f364c01c3f0 comp rx=0 tx=0).stop 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 1 -- 172.21.15.190:0/2663772850 shutdown_connections 2024-08-19T10:41:33.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 1 --2- 172.21.15.190:0/2663772850 >> v2:172.21.15.190:3300/0 conn(0x7f365c0a48b0 0x7f365c136c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:33.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 1 --2- 172.21.15.190:0/2663772850 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f365c0a3b90 0x7f365c0a3f70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:33.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 1 -- 172.21.15.190:0/2663772850 >> 172.21.15.190:0/2663772850 conn(0x7f365c06dde0 msgr2=0x7f365c06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:33.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 1 -- 172.21.15.190:0/2663772850 shutdown_connections 2024-08-19T10:41:33.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 1 -- 172.21.15.190:0/2663772850 wait complete. 2024-08-19T10:41:33.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 10 monclient: build_initial_monmap 2024-08-19T10:41:33.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.637+0000 7f36618d7640 10 monclient: monmap: 2024-08-19T10:41:33.650 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:33.650 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:33.650 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:33.650 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:33.650 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:33.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 1 Processor -- start 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 1 -- start start 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient: init 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient: _add_conns ranks=[7,6,4,8,0,5,2,3,1] 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f365c0a3b90 0x7f365c1ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient(hunting): picked mon.noname-i con 0x7f365c0a3b90 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f365c0a44b0 0x7f365c1cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient(hunting): picked mon.noname-d con 0x7f365c0a44b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f365c0a48b0 0x7f365c1cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient(hunting): picked mon.noname-c con 0x7f365c0a48b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:33.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f365c0a0b00 con 0x7f365c0a48b0 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f365c0a2c80 con 0x7f365c0a44b0 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f365c1371c0 con 0x7f365c0a3b90 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient(hunting): authenticate will time out at 1139.212792s 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f365c0a3b90 0x7f365c1ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f365c0a3b90 0x7f365c1ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:46268/0 (socket says 172.21.15.190:46268) 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 1 -- 172.21.15.190:0/2999029790 learned_addr learned my addr 172.21.15.190:0/2999029790 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 10 monclient(hunting): get_auth_request con 0x7f365c0a3b90 auth_method 0 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:33.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:33.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:33.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:33.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:33.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:33.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365b7fe640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f365c0a48b0 0x7f365c1cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:33.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3653fff640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f365c0a44b0 0x7f365c1cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:33.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365b7fe640 10 monclient(hunting): get_auth_request con 0x7f365c0a48b0 auth_method 0 2024-08-19T10:41:33.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:33.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:33.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:33.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 10 monclient(hunting): handle_auth_done global_id 14506 payload 824 2024-08-19T10:41:33.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 1 -- 172.21.15.190:0/2999029790 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f365c0a44b0 msgr2=0x7f365c1cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:33.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f365c0a44b0 0x7f365c1cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:33.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 1 -- 172.21.15.190:0/2999029790 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f365c0a48b0 msgr2=0x7f365c1cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:33.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f365c0a48b0 0x7f365c1cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:33.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:33.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 1 monclient: found mon.noname-i 2024-08-19T10:41:33.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:33.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:33.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 1 -- 172.21.15.190:0/2999029790 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c06d220 con 0x7f365c0a3b90 2024-08-19T10:41:33.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3653fff640 10 monclient: get_auth_request con 0x7f365c0a44b0 auth_method 0 2024-08-19T10:41:33.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3653fff640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f365c0a44b0 0x7f365c1cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:33.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 1 -- 172.21.15.190:0/2999029790 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f365c0a3b90 msgr2=0x7f365c1ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:33.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f365affd640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f365c0a3b90 0x7f365c1ccd70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f364c01aaa0 tx=0x7f364c019190 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:33.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 1 -- 172.21.15.190:0/2999029790 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f364c0b0040 con 0x7f365c0a3b90 2024-08-19T10:41:33.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:33.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:33.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 10 monclient: dump: 2024-08-19T10:41:33.658 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:33.658 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:33.658 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:33.658 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:33.659 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:33.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 10 monclient: _finish_auth 0 2024-08-19T10:41:33.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:41:33.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:33.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 1 -- 172.21.15.190:0/2999029790 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f364c0044b0 con 0x7f365c0a3b90 2024-08-19T10:41:33.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:33.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 1 -- 172.21.15.190:0/2999029790 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f364c003a90 con 0x7f365c0a3b90 2024-08-19T10:41:33.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 5 monclient: authenticate success, global_id 14506 2024-08-19T10:41:33.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient: _renew_subs 2024-08-19T10:41:33.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:33.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 1 -- 172.21.15.190:0/2999029790 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365c09f230 con 0x7f365c0a3b90 2024-08-19T10:41:33.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3659ffb640 4 set_mon_vals no callback set 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f3658ff9640 10 monclient: dump: 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:33.661 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient: _renew_subs 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.641+0000 7f36618d7640 1 -- 172.21.15.190:0/2999029790 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f365c11f4c0 con 0x7f365c0a3b90 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.645+0000 7f3658ff9640 1 -- 172.21.15.190:0/2999029790 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f364c0bb030 con 0x7f365c0a3b90 2024-08-19T10:41:33.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.645+0000 7f36618d7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.645+0000 7f36618d7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.645+0000 7f36618d7640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.645+0000 7f36618d7640 1 -- 172.21.15.190:0/2999029790 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f365c0a3f70 con 0x7f365c0a3b90 2024-08-19T10:41:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.645+0000 7f3658ff9640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f363c03be00 0x7f363c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.645+0000 7f3658ff9640 1 -- 172.21.15.190:0/2999029790 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(96..96 src has 1..96) ==== 5787+0+0 (secure 0 0 0) 0x7f364c0b9020 con 0x7f365c0a3b90 2024-08-19T10:41:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.645+0000 7f3653fff640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f363c03be00 0x7f363c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.645+0000 7f3653fff640 10 monclient: get_auth_request con 0x7f363c03be00 auth_method 0 2024-08-19T10:41:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.645+0000 7f3653fff640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f363c03be00 0x7f363c03e2c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3644007fd0 tx=0x7f3644009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.649+0000 7f3658ff9640 1 -- 172.21.15.190:0/2999029790 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f364c0c2060 con 0x7f365c0a3b90 2024-08-19T10:41:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.649+0000 7f3658ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:33.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.649+0000 7f3658ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:33.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.821+0000 7f36618d7640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-08-19T10:41:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.821+0000 7f36618d7640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-08-19T10:41:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.821+0000 7f36618d7640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:33.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:33.821+0000 7f36618d7640 1 -- 172.21.15.190:0/2999029790 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"} v 0) -- 0x7f365c06ecb0 con 0x7f365c0a3b90 2024-08-19T10:41:34.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.385+0000 7f3658ff9640 1 -- 172.21.15.190:0/2999029790 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}]=0 removed item id 9 name 'osd.9' from crush map v97) ==== 143+0+0 (secure 0 0 0) 0x7f364c0e6e20 con 0x7f365c0a3b90 2024-08-19T10:41:34.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.385+0000 7f3658ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-08-19T10:41:34.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.385+0000 7f3658ff9640 10 monclient: _finish_command 2 = system:0 removed item id 9 name 'osd.9' from crush map 2024-08-19T10:41:34.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.497+0000 7f36618d7640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-08-19T10:41:34.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.497+0000 7f36618d7640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-08-19T10:41:34.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.497+0000 7f36618d7640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:34.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.497+0000 7f36618d7640 1 -- 172.21.15.190:0/2999029790 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"} v 0) -- 0x7f365c13c330 con 0x7f365c0a3b90 2024-08-19T10:41:34.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.497+0000 7f3658ff9640 1 -- 172.21.15.190:0/2999029790 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}]=0 device 'osd.9' does not appear in the crush map v97) ==== 145+0+0 (secure 0 0 0) 0x7f365c06d220 con 0x7f365c0a3b90 2024-08-19T10:41:34.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.497+0000 7f3658ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.9", "ancestor": "host2"}] 2024-08-19T10:41:34.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.497+0000 7f3658ff9640 10 monclient: _finish_command 3 = system:0 device 'osd.9' does not appear in the crush map 2024-08-19T10:41:34.501 INFO:tasks.workunit.client.0.smithi190.stderr:device 'osd.9' does not appear in the crush map 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 -- 172.21.15.190:0/2999029790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f363c03be00 msgr2=0x7f363c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f363c03be00 0x7f363c03e2c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3644007fd0 tx=0x7f3644009040 comp rx=0 tx=0).stop 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 10 monclient: shutdown 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 -- 172.21.15.190:0/2999029790 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f365c0a3b90 msgr2=0x7f365c1ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f365c0a3b90 0x7f365c1ccd70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f364c01aaa0 tx=0x7f364c019190 comp rx=0 tx=0).stop 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 -- 172.21.15.190:0/2999029790 shutdown_connections 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f363c03be00 0x7f363c03e2c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f365c0a48b0 0x7f365c1cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f365c0a44b0 0x7f365c1cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 --2- 172.21.15.190:0/2999029790 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f365c0a3b90 0x7f365c1ccd70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:34.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 -- 172.21.15.190:0/2999029790 >> 172.21.15.190:0/2999029790 conn(0x7f365c06dde0 msgr2=0x7f365c0a7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:34.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 -- 172.21.15.190:0/2999029790 shutdown_connections 2024-08-19T10:41:34.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.501+0000 7f36618d7640 1 -- 172.21.15.190:0/2999029790 wait complete. 2024-08-19T10:41:34.519 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm host2 2024-08-19T10:41:34.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient: get_monmap_and_config 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient: build_initial_monmap 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient: monmap: 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:34.637130+0000 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:34.637130+0000 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:34.638 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:34.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 1 Processor -- start 2024-08-19T10:41:34.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 1 -- start start 2024-08-19T10:41:34.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient: init 2024-08-19T10:41:34.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:34.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient: _add_conns ranks=[9,6,2,5,4,7,0,1,8,3,10] 2024-08-19T10:41:34.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc2f00a6850 0x7fc2f00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:34.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient(hunting): picked mon.noname-a con 0x7fc2f00a6850 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:34.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc2f00a6c30 0x7fc2f00a7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:34.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient(hunting): picked mon.noname-d con 0x7fc2f00a6c30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:34.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2f0139610 0x7fc2f0139ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:34.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient(hunting): picked mon.noname-b con 0x7fc2f0139610 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:34.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f00a3350 con 0x7fc2f00a6850 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f00a46a0 con 0x7fc2f0139610 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fc2f00a4980 con 0x7fc2f00a6c30 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2f527b640 10 monclient(hunting): authenticate will time out at 1140.207167s 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ee7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc2f00a6c30 0x7fc2f00a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2eeffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc2f00a6850 0x7fc2f00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ef7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2f0139610 0x7fc2f0139ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2eeffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc2f00a6850 0x7fc2f00a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:54986/0 (socket says 172.21.15.190:54986) 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2eeffd640 1 -- 172.21.15.190:0/1727333037 learned_addr learned my addr 172.21.15.190:0/1727333037 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:34.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2eeffd640 10 monclient(hunting): get_auth_request con 0x7fc2f00a6850 auth_method 0 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2eeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2eeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ef7fe640 10 monclient(hunting): get_auth_request con 0x7fc2f0139610 auth_method 0 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ef7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ee7fc640 10 monclient(hunting): get_auth_request con 0x7fc2f00a6c30 auth_method 0 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ee7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:34.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.633+0000 7fc2ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.637+0000 7fc2ef7fe640 10 monclient(hunting): handle_auth_done global_id 5285 payload 274 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.637+0000 7fc2ef7fe640 1 -- 172.21.15.190:0/1727333037 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc2f00a6c30 msgr2=0x7fc2f00a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.637+0000 7fc2ef7fe640 1 --2- 172.21.15.190:0/1727333037 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc2f00a6c30 0x7fc2f00a7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.637+0000 7fc2ef7fe640 1 -- 172.21.15.190:0/1727333037 >> v2:172.21.15.89:3300/0 conn(0x7fc2f00a6850 msgr2=0x7fc2f00a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.637+0000 7fc2ef7fe640 1 --2- 172.21.15.190:0/1727333037 >> v2:172.21.15.89:3300/0 conn(0x7fc2f00a6850 0x7fc2f00a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:34.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.637+0000 7fc2ef7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:34.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.637+0000 7fc2ef7fe640 1 monclient: found mon.noname-b 2024-08-19T10:41:34.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.637+0000 7fc2ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:34.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.637+0000 7fc2ef7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:34.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.637+0000 7fc2ef7fe640 1 -- 172.21.15.190:0/1727333037 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f00a5c70 con 0x7fc2f0139610 2024-08-19T10:41:34.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.637+0000 7fc2eeffd640 1 --2- 172.21.15.190:0/1727333037 >> v2:172.21.15.89:3300/0 conn(0x7fc2f00a6850 0x7fc2f00a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:34.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2ef7fe640 1 -- 172.21.15.190:0/1727333037 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2f0139610 msgr2=0x7fc2f0139ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2ef7fe640 1 --2- 172.21.15.190:0/1727333037 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2f0139610 0x7fc2f0139ac0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009d90 tx=0x7fc2e801c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=582d670438d875d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 1 -- 172.21.15.190:0/1727333037 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc2e80b2070 con 0x7fc2f0139610 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 10 monclient: dump: 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:34.649 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:34.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 1 -- 172.21.15.190:0/1727333037 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e80a5a00 con 0x7fc2f0139610 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 1 -- 172.21.15.190:0/1727333037 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc2e80a5c30 con 0x7fc2f0139610 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 5 monclient: authenticate success, global_id 5285 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 4 set_mon_vals no callback set 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2edffb640 10 monclient: dump: 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:34.651 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 10 monclient: shutdown 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:34.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 -- 172.21.15.190:0/1727333037 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2f0139610 msgr2=0x7fc2f0139ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 --2- 172.21.15.190:0/1727333037 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2f0139610 0x7fc2f0139ac0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009d90 tx=0x7fc2e801c830 comp rx=0 tx=0).stop 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 -- 172.21.15.190:0/1727333037 shutdown_connections 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 --2- 172.21.15.190:0/1727333037 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc2f0139610 0x7fc2f0139ac0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 --2- 172.21.15.190:0/1727333037 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc2f00a6c30 0x7fc2f00a7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 --2- 172.21.15.190:0/1727333037 >> v2:172.21.15.89:3300/0 conn(0x7fc2f00a6850 0x7fc2f00a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 -- 172.21.15.190:0/1727333037 >> 172.21.15.190:0/1727333037 conn(0x7fc2f00abe50 msgr2=0x7fc2f00ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 -- 172.21.15.190:0/1727333037 shutdown_connections 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 -- 172.21.15.190:0/1727333037 wait complete. 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 10 monclient: build_initial_monmap 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 10 monclient: monmap: 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:34.654 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 Processor -- start 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 -- start start 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 10 monclient: init 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 10 monclient: _add_conns ranks=[1,5,8,7,0,4,6,2,3] 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f00a6850 0x7fc2f01349e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 10 monclient(hunting): picked mon.noname-f con 0x7fc2f00a6850 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc2f00a6c30 0x7fc2f0134f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 10 monclient(hunting): picked mon.noname-h con 0x7fc2f00a6c30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.641+0000 7fc2f527b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc2f0139610 0x7fc2f009ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 10 monclient(hunting): picked mon.noname-e con 0x7fc2f0139610 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f00a3780 con 0x7fc2f00a6850 2024-08-19T10:41:34.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc2f00a3600 con 0x7fc2f00a6c30 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc2f0143300 con 0x7fc2f0139610 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 10 monclient(hunting): authenticate will time out at 1140.216039s 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f00a6850 0x7fc2f01349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ee7fc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc2f00a6c30 0x7fc2f0134f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f00a6850 0x7fc2f01349e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34538/0 (socket says 172.21.15.190:34538) 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 1 -- 172.21.15.190:0/1441547789 learned_addr learned my addr 172.21.15.190:0/1441547789 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 10 monclient(hunting): get_auth_request con 0x7fc2f00a6850 auth_method 0 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:34.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ee7fc640 10 monclient(hunting): get_auth_request con 0x7fc2f00a6c30 auth_method 0 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ee7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ee7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ee7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ee7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ee7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ee7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 10 monclient(hunting): handle_auth_done global_id 14581 payload 824 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ef7fe640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc2f0139610 0x7fc2f009ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:34.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 1 -- 172.21.15.190:0/1441547789 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc2f0139610 msgr2=0x7fc2f009ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:34.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc2f0139610 0x7fc2f009ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:34.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 1 -- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc2f00a6c30 msgr2=0x7fc2f0134f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:34.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc2f00a6c30 0x7fc2f0134f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:34.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:34.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 1 monclient: found mon.noname-f 2024-08-19T10:41:34.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:34.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:34.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 1 -- 172.21.15.190:0/1441547789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f009e140 con 0x7fc2f00a6850 2024-08-19T10:41:34.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ef7fe640 10 monclient: get_auth_request con 0x7fc2f0139610 auth_method 0 2024-08-19T10:41:34.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ef7fe640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc2f0139610 0x7fc2f009ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:34.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ee7fc640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc2f00a6c30 0x7fc2f0134f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:34.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 1 -- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f00a6850 msgr2=0x7fc2f01349e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:34.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2eeffd640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f00a6850 0x7fc2f01349e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800e9d0 tx=0x7fc2d800ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:34.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 1 -- 172.21.15.190:0/1441547789 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc2d8017070 con 0x7fc2f00a6850 2024-08-19T10:41:34.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:34.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:34.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 10 monclient: dump: 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:34.661 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 5 monclient: authenticate success, global_id 14581 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 10 monclient: _renew_subs 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 1 -- 172.21.15.190:0/1441547789 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d800cc70 con 0x7fc2f00a6850 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 1 -- 172.21.15.190:0/1441547789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f00a5820 con 0x7fc2f00a6850 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:34.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 1 -- 172.21.15.190:0/1441547789 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc2d800ce20 con 0x7fc2f00a6850 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2ed7fa640 4 set_mon_vals no callback set 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2e77fe640 10 monclient: dump: 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:34.663 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 10 monclient: _renew_subs 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.645+0000 7fc2f527b640 1 -- 172.21.15.190:0/1441547789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f00a0eb0 con 0x7fc2f00a6850 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.649+0000 7fc2f527b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.649+0000 7fc2f527b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.649+0000 7fc2f527b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.649+0000 7fc2f527b640 1 -- 172.21.15.190:0/1441547789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f00a7010 con 0x7fc2f00a6850 2024-08-19T10:41:34.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.649+0000 7fc2eeffd640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f00a6850 0x7fc2f01349e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800e9d0 tx=0x7fc2d800ee90 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fc2d801a070 mgrmap(e 5) 2024-08-19T10:41:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.649+0000 7fc2e77fe640 1 -- 172.21.15.190:0/1441547789 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc2d801a070 con 0x7fc2f00a6850 2024-08-19T10:41:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.649+0000 7fc2e77fe640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc2d003be30 0x7fc2d003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.649+0000 7fc2ee7fc640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc2d003be30 0x7fc2d003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.649+0000 7fc2ee7fc640 10 monclient: get_auth_request con 0x7fc2d003be30 auth_method 0 2024-08-19T10:41:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.649+0000 7fc2ee7fc640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc2d003be30 0x7fc2d003e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc2f009e790 tx=0x7fc2e0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:34.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:34.653+0000 7fc2e77fe640 1 -- 172.21.15.190:0/1441547789 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5766+0+0 (secure 0 0 0) 0x7fc2d800f040 con 0x7fc2f00a6850 2024-08-19T10:41:35.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:35.477+0000 7fc2e77fe640 1 -- 172.21.15.190:0/1441547789 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc2d8021070 con 0x7fc2f00a6850 2024-08-19T10:41:35.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:35.477+0000 7fc2e77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:35.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:35.477+0000 7fc2e77fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:35.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:35.661+0000 7fc2f527b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host2"}] 2024-08-19T10:41:35.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:35.661+0000 7fc2f527b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "host2"}] 2024-08-19T10:41:35.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:35.661+0000 7fc2f527b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:35.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:35.661+0000 7fc2f527b640 1 -- 172.21.15.190:0/1441547789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host2"} v 0) -- 0x7fc2f006c8d0 con 0x7fc2f00a6850 2024-08-19T10:41:36.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.409+0000 7fc2e77fe640 1 -- 172.21.15.190:0/1441547789 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host2"}]=0 removed item id -16 name 'host2' from crush map v98) ==== 124+0+0 (secure 0 0 0) 0x7fc2d801bdc0 con 0x7fc2f00a6850 2024-08-19T10:41:36.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.409+0000 7fc2e77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "host2"}] 2024-08-19T10:41:36.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.409+0000 7fc2e77fe640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'host2' from crush map 2024-08-19T10:41:36.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.517+0000 7fc2f527b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "host2"}] 2024-08-19T10:41:36.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.517+0000 7fc2f527b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "host2"}] 2024-08-19T10:41:36.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.517+0000 7fc2f527b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:36.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.517+0000 7fc2f527b640 1 -- 172.21.15.190:0/1441547789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "host2"} v 0) -- 0x7fc2f009ec80 con 0x7fc2f00a6850 2024-08-19T10:41:36.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2e77fe640 1 -- 172.21.15.190:0/1441547789 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "host2"}]=0 device 'host2' does not appear in the crush map v98) ==== 124+0+0 (secure 0 0 0) 0x7fc2d801bb20 con 0x7fc2f00a6850 2024-08-19T10:41:36.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2e77fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "host2"}] 2024-08-19T10:41:36.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2e77fe640 10 monclient: _finish_command 3 = system:0 device 'host2' does not appear in the crush map 2024-08-19T10:41:36.525 INFO:tasks.workunit.client.0.smithi190.stderr:device 'host2' does not appear in the crush map 2024-08-19T10:41:36.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 1 -- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc2d003be30 msgr2=0x7fc2d003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:36.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc2d003be30 0x7fc2d003e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc2f009e790 tx=0x7fc2e0009190 comp rx=0 tx=0).stop 2024-08-19T10:41:36.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 10 monclient: shutdown 2024-08-19T10:41:36.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:36.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 1 -- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f00a6850 msgr2=0x7fc2f01349e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:36.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f00a6850 0x7fc2f01349e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800e9d0 tx=0x7fc2d800ee90 comp rx=0 tx=0).stop 2024-08-19T10:41:36.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 1 -- 172.21.15.190:0/1441547789 shutdown_connections 2024-08-19T10:41:36.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc2d003be30 0x7fc2d003e2f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc2f0139610 0x7fc2f009ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc2f00a6c30 0x7fc2f0134f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 1 --2- 172.21.15.190:0/1441547789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f00a6850 0x7fc2f01349e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.521+0000 7fc2f527b640 1 -- 172.21.15.190:0/1441547789 >> 172.21.15.190:0/1441547789 conn(0x7fc2f00abe50 msgr2=0x7fc2f01377f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:36.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.525+0000 7fc2f527b640 1 -- 172.21.15.190:0/1441547789 shutdown_connections 2024-08-19T10:41:36.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.525+0000 7fc2f527b640 1 -- 172.21.15.190:0/1441547789 wait complete. 2024-08-19T10:41:36.542 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket foo host 2024-08-19T10:41:36.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.657+0000 7f7e2ba65640 10 monclient: get_monmap_and_config 2024-08-19T10:41:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.657+0000 7f7e2ba65640 10 monclient: build_initial_monmap 2024-08-19T10:41:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.657+0000 7f7e2ba65640 10 monclient: monmap: 2024-08-19T10:41:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:36.662271+0000 2024-08-19T10:41:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:36.662271+0000 2024-08-19T10:41:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:36.663 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:36.663 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:36.663 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:36.663 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:36.663 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:36.663 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:36.663 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:36.663 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:36.663 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:36.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.657+0000 7f7e2ba65640 1 Processor -- start 2024-08-19T10:41:36.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.657+0000 7f7e2ba65640 1 -- start start 2024-08-19T10:41:36.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.657+0000 7f7e2ba65640 10 monclient: init 2024-08-19T10:41:36.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.657+0000 7f7e2ba65640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:36.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.657+0000 7f7e2ba65640 10 monclient: _add_conns ranks=[6,7,10,2,4,1,0,3,8,9,5] 2024-08-19T10:41:36.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.657+0000 7f7e2ba65640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7e240a46b0 0x7f7e240a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:36.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.657+0000 7f7e2ba65640 10 monclient(hunting): picked mon.noname-d con 0x7f7e240a46b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:36.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e240a27d0 0x7f7e240a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:36.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 10 monclient(hunting): picked mon.noname-i con 0x7f7e240a27d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:36.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e240a4fd0 0x7f7e241374e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:36.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 10 monclient(hunting): picked mon.noname-f con 0x7f7e240a4fd0 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7e2406d290 con 0x7f7e240a4fd0 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7e240a61d0 con 0x7f7e240a46b0 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7e240a53d0 con 0x7f7e240a27d0 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2ba65640 10 monclient(hunting): authenticate will time out at 1142.232085s 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e240a4fd0 0x7f7e241374e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29000640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e240a27d0 0x7f7e240a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e240a4fd0 0x7f7e241374e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51228/0 (socket says 172.21.15.190:51228) 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29000640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e240a27d0 0x7f7e240a4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:55154/0 (socket says 172.21.15.190:55154) 2024-08-19T10:41:36.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 1 -- 172.21.15.190:0/3134006516 learned_addr learned my addr 172.21.15.190:0/3134006516 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 10 monclient(hunting): get_auth_request con 0x7f7e240a4fd0 auth_method 0 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29000640 10 monclient(hunting): get_auth_request con 0x7f7e240a27d0 auth_method 0 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29000640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29000640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29000640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29000640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:36.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29000640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:36.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29000640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:36.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29801640 1 --2- 172.21.15.190:0/3134006516 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7e240a46b0 0x7f7e240a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:36.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 10 monclient(hunting): handle_auth_done global_id 14590 payload 274 2024-08-19T10:41:36.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 1 -- 172.21.15.190:0/3134006516 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e240a27d0 msgr2=0x7f7e240a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:36.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 1 --2- 172.21.15.190:0/3134006516 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e240a27d0 0x7f7e240a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 1 -- 172.21.15.190:0/3134006516 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7e240a46b0 msgr2=0x7f7e240a2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:36.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 1 --2- 172.21.15.190:0/3134006516 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7e240a46b0 0x7f7e240a2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 10 monclient: _finish_hunting 0 2024-08-19T10:41:36.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 1 monclient: found mon.noname-f 2024-08-19T10:41:36.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:36.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:36.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e2a002640 1 -- 172.21.15.190:0/3134006516 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2409d5c0 con 0x7f7e240a4fd0 2024-08-19T10:41:36.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29000640 1 --2- 172.21.15.190:0/3134006516 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e240a27d0 0x7f7e240a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:36.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29801640 10 monclient: get_auth_request con 0x7f7e240a46b0 auth_method 0 2024-08-19T10:41:36.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.661+0000 7f7e29801640 1 --2- 172.21.15.190:0/3134006516 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7e240a46b0 0x7f7e240a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:36.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2a002640 1 -- 172.21.15.190:0/3134006516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e240a4fd0 msgr2=0x7f7e241374e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:36.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2a002640 1 --2- 172.21.15.190:0/3134006516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e240a4fd0 0x7f7e241374e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009e50 tx=0x7f7e1c01c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a57cda72b9f9c99 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:36.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 1 -- 172.21.15.190:0/3134006516 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e1c0aa8c0 con 0x7f7e240a4fd0 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 10 monclient: dump: 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:36.677 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 10 monclient: _finish_auth 0 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 1 -- 172.21.15.190:0/3134006516 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1c0af070 con 0x7f7e240a4fd0 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 1 -- 172.21.15.190:0/3134006516 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e1c0073f0 con 0x7f7e240a4fd0 2024-08-19T10:41:36.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 5 monclient: authenticate success, global_id 14590 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 4 set_mon_vals no callback set 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e1bfff640 10 monclient: dump: 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:36.679 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient: shutdown 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 -- 172.21.15.190:0/3134006516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e240a4fd0 msgr2=0x7f7e241374e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 --2- 172.21.15.190:0/3134006516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e240a4fd0 0x7f7e241374e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009e50 tx=0x7f7e1c01c6c0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 -- 172.21.15.190:0/3134006516 shutdown_connections 2024-08-19T10:41:36.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 --2- 172.21.15.190:0/3134006516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e240a4fd0 0x7f7e241374e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 --2- 172.21.15.190:0/3134006516 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e240a27d0 0x7f7e240a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 --2- 172.21.15.190:0/3134006516 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7e240a46b0 0x7f7e240a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 -- 172.21.15.190:0/3134006516 >> 172.21.15.190:0/3134006516 conn(0x7f7e240a92a0 msgr2=0x7f7e240a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 -- 172.21.15.190:0/3134006516 shutdown_connections 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 -- 172.21.15.190:0/3134006516 wait complete. 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient: build_initial_monmap 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient: monmap: 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 Processor -- start 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 -- start start 2024-08-19T10:41:36.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient: init 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient: _add_conns ranks=[2,4,0,6,7,1,3,8,5] 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e240a27d0 0x7f7e24132d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient(hunting): picked mon.noname-b con 0x7f7e240a27d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e240a46b0 0x7f7e241332b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient(hunting): picked mon.noname-c con 0x7f7e240a46b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e240a4fd0 0x7f7e240a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient(hunting): picked mon.noname-a con 0x7f7e240a4fd0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24141140 con 0x7f7e240a4fd0 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7e24140fc0 con 0x7f7e240a27d0 2024-08-19T10:41:36.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f7e241412c0 con 0x7f7e240a46b0 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2ba65640 10 monclient(hunting): authenticate will time out at 1142.239175s 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29000640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e240a46b0 0x7f7e241332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2a002640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e240a4fd0 0x7f7e240a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29801640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e240a27d0 0x7f7e24132d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29000640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e240a46b0 0x7f7e241332b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:44886/0 (socket says 172.21.15.190:44886) 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29000640 1 -- 172.21.15.190:0/2135192101 learned_addr learned my addr 172.21.15.190:0/2135192101 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29801640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e240a27d0 0x7f7e24132d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:46792/0 (socket says 172.21.15.190:46792) 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29000640 10 monclient(hunting): get_auth_request con 0x7f7e240a46b0 auth_method 0 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29000640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29000640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:36.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29000640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29801640 10 monclient(hunting): get_auth_request con 0x7f7e240a27d0 auth_method 0 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29801640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29801640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29801640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2a002640 10 monclient(hunting): get_auth_request con 0x7f7e240a4fd0 auth_method 0 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2a002640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2a002640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2a002640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29000640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29000640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29000640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29801640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29801640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:36.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e29801640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2a002640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2a002640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.665+0000 7f7e2a002640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 10 monclient(hunting): handle_auth_done global_id 5314 payload 824 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 1 -- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e240a27d0 msgr2=0x7f7e24132d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e240a27d0 0x7f7e24132d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 1 -- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e240a4fd0 msgr2=0x7f7e240a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e240a4fd0 0x7f7e240a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 10 monclient: _finish_hunting 0 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 1 monclient: found mon.noname-c 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:36.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 1 -- 172.21.15.190:0/2135192101 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e24120a50 con 0x7f7e240a46b0 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29801640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e240a27d0 0x7f7e24132d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 1 -- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e240a46b0 msgr2=0x7f7e241332b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e29000640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e240a46b0 0x7f7e241332b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400e9b0 tx=0x7f7e1400ee70 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 1 -- 172.21.15.190:0/2135192101 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e14017070 con 0x7f7e240a46b0 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 10 monclient: dump: 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:36.687 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:36.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e2ba65640 5 monclient: authenticate success, global_id 5314 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 1 -- 172.21.15.190:0/2135192101 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1400cc10 con 0x7f7e240a46b0 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e2ba65640 10 monclient: _renew_subs 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e2ba65640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e2ba65640 1 -- 172.21.15.190:0/2135192101 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e240a1d60 con 0x7f7e240a46b0 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 1 -- 172.21.15.190:0/2135192101 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e1400cdc0 con 0x7f7e240a46b0 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e1b7fe640 4 set_mon_vals no callback set 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e19ffb640 10 monclient: dump: 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:36.689 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:36.690 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e2ba65640 10 monclient: _renew_subs 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e2ba65640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e2ba65640 1 -- 172.21.15.190:0/2135192101 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e240a8160 con 0x7f7e240a46b0 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e2ba65640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e2ba65640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e2ba65640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.669+0000 7f7e2ba65640 1 -- 172.21.15.190:0/2135192101 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e240a2c50 con 0x7f7e240a46b0 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.673+0000 7f7e19ffb640 1 -- 172.21.15.190:0/2135192101 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7e1401a070 con 0x7f7e240a46b0 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.673+0000 7f7e19ffb640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e0803be00 0x7f7e0803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:36.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.673+0000 7f7e19ffb640 1 -- 172.21.15.190:0/2135192101 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(98..98 src has 1..98) ==== 5631+0+0 (secure 0 0 0) 0x7f7e1400f040 con 0x7f7e240a46b0 2024-08-19T10:41:36.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.673+0000 7f7e29801640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e0803be00 0x7f7e0803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:36.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.673+0000 7f7e29801640 10 monclient: get_auth_request con 0x7f7e0803be00 auth_method 0 2024-08-19T10:41:36.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.673+0000 7f7e29801640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e0803be00 0x7f7e0803e2c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000aeb0 tx=0x7f7e20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:36.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.685+0000 7f7e19ffb640 1 -- 172.21.15.190:0/2135192101 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7e14021070 con 0x7f7e240a46b0 2024-08-19T10:41:36.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.685+0000 7f7e19ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:36.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.685+0000 7f7e19ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:36.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.845+0000 7f7e2ba65640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-08-19T10:41:36.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.845+0000 7f7e2ba65640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-08-19T10:41:36.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.845+0000 7f7e2ba65640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:36.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:36.845+0000 7f7e2ba65640 1 -- 172.21.15.190:0/2135192101 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "host"} v 0) -- 0x7f7e2406ec50 con 0x7f7e240a46b0 2024-08-19T10:41:37.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.433+0000 7f7e19ffb640 1 -- 172.21.15.190:0/2135192101 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}]=0 added bucket foo type host to crush map v99) ==== 138+0+0 (secure 0 0 0) 0x7f7e14012cc0 con 0x7f7e240a46b0 2024-08-19T10:41:37.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.433+0000 7f7e19ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-08-19T10:41:37.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.433+0000 7f7e19ffb640 10 monclient: _finish_command 2 = system:0 added bucket foo type host to crush map 2024-08-19T10:41:37.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.537+0000 7f7e2ba65640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-08-19T10:41:37.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.537+0000 7f7e2ba65640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-08-19T10:41:37.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.537+0000 7f7e2ba65640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:41:37.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.537+0000 7f7e2ba65640 1 -- 172.21.15.190:0/2135192101 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "foo", "type": "host"} v 0) -- 0x7f7e24133c70 con 0x7f7e240a46b0 2024-08-19T10:41:37.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.541+0000 7f7e19ffb640 1 -- 172.21.15.190:0/2135192101 <== mon.4 v2:172.21.15.89:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}]=0 bucket 'foo' already exists v99) ==== 126+0+0 (secure 0 0 0) 0x7f7e14011d60 con 0x7f7e240a46b0 2024-08-19T10:41:37.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.541+0000 7f7e19ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "foo", "type": "host"}] 2024-08-19T10:41:37.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.541+0000 7f7e19ffb640 10 monclient: _finish_command 3 = system:0 bucket 'foo' already exists 2024-08-19T10:41:37.545 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'foo' already exists 2024-08-19T10:41:37.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 -- 172.21.15.190:0/2135192101 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e0803be00 msgr2=0x7f7e0803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:37.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e0803be00 0x7f7e0803e2c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000aeb0 tx=0x7f7e20008040 comp rx=0 tx=0).stop 2024-08-19T10:41:37.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 10 monclient: shutdown 2024-08-19T10:41:37.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:37.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 -- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e240a46b0 msgr2=0x7f7e241332b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:37.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e240a46b0 0x7f7e241332b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400e9b0 tx=0x7f7e1400ee70 comp rx=0 tx=0).stop 2024-08-19T10:41:37.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 -- 172.21.15.190:0/2135192101 shutdown_connections 2024-08-19T10:41:37.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e0803be00 0x7f7e0803e2c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:37.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e240a4fd0 0x7f7e240a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:37.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e240a46b0 0x7f7e241332b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:37.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 --2- 172.21.15.190:0/2135192101 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e240a27d0 0x7f7e24132d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:37.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 -- 172.21.15.190:0/2135192101 >> 172.21.15.190:0/2135192101 conn(0x7f7e240a92a0 msgr2=0x7f7e240a55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:37.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 -- 172.21.15.190:0/2135192101 shutdown_connections 2024-08-19T10:41:37.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.545+0000 7f7e2ba65640 1 -- 172.21.15.190:0/2135192101 wait complete. 2024-08-19T10:41:37.562 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush move foo root=default rack=localrack 2024-08-19T10:41:37.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient: get_monmap_and_config 2024-08-19T10:41:37.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient: build_initial_monmap 2024-08-19T10:41:37.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient: monmap: 2024-08-19T10:41:37.677 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:37.677 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:37.677 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:37.677454+0000 2024-08-19T10:41:37.677 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:37.677454+0000 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:37.678 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:37.679 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:37.679 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:37.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 1 Processor -- start 2024-08-19T10:41:37.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 1 -- start start 2024-08-19T10:41:37.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient: init 2024-08-19T10:41:37.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:37.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient: _add_conns ranks=[5,2,1,7,10,0,8,3,4,9,6] 2024-08-19T10:41:37.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e80a69c0 0x7f63e80a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:37.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient(hunting): picked mon.noname-h con 0x7f63e80a69c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63e80a6da0 0x7f63e80a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient(hunting): picked mon.noname-b con 0x7f63e80a6da0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f63e8139780 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e80a4eb0 con 0x7f63e8139780 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f63e80a5c30 con 0x7f63e80a6da0 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f63e80a3570 con 0x7f63e80a69c0 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63ee04a640 10 monclient(hunting): authenticate will time out at 1143.247319s 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e80a69c0 0x7f63e80a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e7fff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f63e8139780 0x7f63e8139bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:40124/0 (socket says 172.21.15.190:40124) 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e7fff640 1 -- 172.21.15.190:0/1130633865 learned_addr learned my addr 172.21.15.190:0/1130633865 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 10 monclient(hunting): get_auth_request con 0x7f63e80a69c0 auth_method 0 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e6ffd640 1 --2- 172.21.15.190:0/1130633865 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63e80a6da0 0x7f63e80a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 10 monclient(hunting): handle_auth_done global_id 5306 payload 274 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 1 -- 172.21.15.190:0/1130633865 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63e80a6da0 msgr2=0x7f63e80a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:37.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 1 --2- 172.21.15.190:0/1130633865 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63e80a6da0 0x7f63e80a7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 1 -- 172.21.15.190:0/1130633865 >> v1:172.21.15.190:6789/0 conn(0x7f63e8139780 legacy=0x7f63e8139bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 1 monclient: found mon.noname-h 2024-08-19T10:41:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.673+0000 7f63e77fe640 1 -- 172.21.15.190:0/1130633865 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e80a6330 con 0x7f63e80a69c0 2024-08-19T10:41:37.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 1 -- 172.21.15.190:0/1130633865 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e80a69c0 msgr2=0x7f63e80a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:37.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 1 --2- 172.21.15.190:0/1130633865 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e80a69c0 0x7f63e80a56f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63d80097f0 tx=0x7f63d801c340 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d88c780e7a0d8e4e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:37.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 1 -- 172.21.15.190:0/1130633865 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63d80b0070 con 0x7f63e80a69c0 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 10 monclient: dump: 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:37.690 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 1 -- 172.21.15.190:0/1130633865 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d80aa810 con 0x7f63e80a69c0 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 5 monclient: authenticate success, global_id 5306 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:37.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 1 -- 172.21.15.190:0/1130633865 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63d80aae30 con 0x7f63e80a69c0 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 4 set_mon_vals no callback set 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e67fc640 10 monclient: dump: 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:37.692 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient: shutdown 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 -- 172.21.15.190:0/1130633865 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e80a69c0 msgr2=0x7f63e80a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 --2- 172.21.15.190:0/1130633865 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e80a69c0 0x7f63e80a56f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63d80097f0 tx=0x7f63d801c340 comp rx=0 tx=0).stop 2024-08-19T10:41:37.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 -- 172.21.15.190:0/1130633865 shutdown_connections 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 --2- 172.21.15.190:0/1130633865 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63e80a6da0 0x7f63e80a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 --2- 172.21.15.190:0/1130633865 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e80a69c0 0x7f63e80a56f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 -- 172.21.15.190:0/1130633865 >> 172.21.15.190:0/1130633865 conn(0x7f63e80abfc0 msgr2=0x7f63e80ac3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 -- 172.21.15.190:0/1130633865 shutdown_connections 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 -- 172.21.15.190:0/1130633865 wait complete. 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient: build_initial_monmap 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient: monmap: 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:37.694 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 Processor -- start 2024-08-19T10:41:37.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 -- start start 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient: init 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient: _add_conns ranks=[1,7,5,8,0,4,6,3,2] 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e80a69c0 0x7f63e8138490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient(hunting): picked mon.noname-f con 0x7f63e80a69c0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e80a6da0 0x7f63e81389d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient(hunting): picked mon.noname-i con 0x7f63e80a6da0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e8139780 0x7f63e813dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient(hunting): picked mon.noname-h con 0x7f63e8139780 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f63e80a76c0 con 0x7f63e80a69c0 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:37.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f63e8144410 con 0x7f63e8139780 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f63e81400f0 con 0x7f63e80a6da0 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63ee04a640 10 monclient(hunting): authenticate will time out at 1143.254731s 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e80a69c0 0x7f63e8138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e80a69c0 0x7f63e8138490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51240/0 (socket says 172.21.15.190:51240) 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 1 -- 172.21.15.190:0/2822456602 learned_addr learned my addr 172.21.15.190:0/2822456602 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 10 monclient(hunting): get_auth_request con 0x7f63e80a69c0 auth_method 0 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:37.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e7fff640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e8139780 0x7f63e813dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e7fff640 10 monclient(hunting): get_auth_request con 0x7f63e8139780 auth_method 0 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e6ffd640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e80a6da0 0x7f63e81389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e6ffd640 10 monclient(hunting): get_auth_request con 0x7f63e80a6da0 auth_method 0 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:37.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 10 monclient(hunting): handle_auth_done global_id 14599 payload 824 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 1 -- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e80a6da0 msgr2=0x7f63e81389d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e80a6da0 0x7f63e81389d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 1 -- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e8139780 msgr2=0x7f63e813dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e8139780 0x7f63e813dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 1 monclient: found mon.noname-f 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e77fe640 1 -- 172.21.15.190:0/2822456602 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e80aaf60 con 0x7f63e80a69c0 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.681+0000 7f63e7fff640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e8139780 0x7f63e813dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63e77fe640 1 -- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e80a69c0 msgr2=0x7f63e8138490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63e77fe640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e80a69c0 0x7f63e8138490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f63d801aa50 tx=0x7f63d8019140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 1 -- 172.21.15.190:0/2822456602 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63d80b0040 con 0x7f63e80a69c0 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 10 monclient: dump: 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:37.700 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:37.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 10 monclient: _finish_auth 0 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 1 -- 172.21.15.190:0/2822456602 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d80044b0 con 0x7f63e80a69c0 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 1 -- 172.21.15.190:0/2822456602 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63d8003a90 con 0x7f63e80a69c0 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 10 monclient: dump: 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:37.702 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:37.703 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63ee04a640 5 monclient: authenticate success, global_id 14599 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63ee04a640 10 monclient: _renew_subs 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63ee04a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63e57fa640 4 set_mon_vals no callback set 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63ee04a640 1 -- 172.21.15.190:0/2822456602 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e813f590 con 0x7f63e80a69c0 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63ee04a640 10 monclient: _renew_subs 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63ee04a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63ee04a640 1 -- 172.21.15.190:0/2822456602 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e8138f10 con 0x7f63e80a69c0 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63ee04a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63ee04a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63ee04a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63ee04a640 1 -- 172.21.15.190:0/2822456602 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e8122ed0 con 0x7f63e80a69c0 2024-08-19T10:41:37.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 1 -- 172.21.15.190:0/2822456602 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f63d80bb030 con 0x7f63e80a69c0 2024-08-19T10:41:37.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63d403bdf0 0x7f63d403e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:37.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63cffff640 1 -- 172.21.15.190:0/2822456602 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5658+0+0 (secure 0 0 0) 0x7f63d80b9020 con 0x7f63e80a69c0 2024-08-19T10:41:37.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63e6ffd640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63d403bdf0 0x7f63d403e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:37.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.685+0000 7f63e6ffd640 10 monclient: get_auth_request con 0x7f63d403bdf0 auth_method 0 2024-08-19T10:41:37.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.689+0000 7f63e6ffd640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63d403bdf0 0x7f63d403e2b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f63e813f200 tx=0x7f63dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:37.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.689+0000 7f63cffff640 1 -- 172.21.15.190:0/2822456602 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f63d80c2020 con 0x7f63e80a69c0 2024-08-19T10:41:37.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.689+0000 7f63cffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:37.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.689+0000 7f63cffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:37.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.861+0000 7f63ee04a640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-08-19T10:41:37.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.861+0000 7f63ee04a640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-08-19T10:41:37.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.861+0000 7f63ee04a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:37.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:37.861+0000 7f63ee04a640 1 -- 172.21.15.190:0/2822456602 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]} v 0) -- 0x7f63e806c9e0 con 0x7f63e80a69c0 2024-08-19T10:41:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.433+0000 7f63cffff640 1 -- 172.21.15.190:0/2822456602 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}]=0 moved item id -10 name 'foo' to location {rack=localrack,root=default} in crush map v100) ==== 204+0+0 (secure 0 0 0) 0x7f63d80160b0 con 0x7f63e80a69c0 2024-08-19T10:41:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.433+0000 7f63cffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-08-19T10:41:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.433+0000 7f63cffff640 10 monclient: _finish_command 2 = system:0 moved item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-08-19T10:41:38.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.541+0000 7f63ee04a640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-08-19T10:41:38.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.541+0000 7f63ee04a640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-08-19T10:41:38.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.541+0000 7f63ee04a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:38.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.541+0000 7f63ee04a640 1 -- 172.21.15.190:0/2822456602 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]} v 0) -- 0x7f63e80a56f0 con 0x7f63e80a69c0 2024-08-19T10:41:38.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.545+0000 7f63cffff640 1 -- 172.21.15.190:0/2822456602 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}]=0 no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map v100) ==== 214+0+0 (secure 0 0 0) 0x7f63d80aa7e0 con 0x7f63e80a69c0 2024-08-19T10:41:38.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.545+0000 7f63cffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "foo", "args": ["root=default", "rack=localrack"]}] 2024-08-19T10:41:38.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.545+0000 7f63cffff640 10 monclient: _finish_command 3 = system:0 no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-08-19T10:41:38.549 INFO:tasks.workunit.client.0.smithi190.stderr:no need to move item id -10 name 'foo' to location {rack=localrack,root=default} in crush map 2024-08-19T10:41:38.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 -- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63d403bdf0 msgr2=0x7f63d403e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:38.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63d403bdf0 0x7f63d403e2b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f63e813f200 tx=0x7f63dc008040 comp rx=0 tx=0).stop 2024-08-19T10:41:38.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 10 monclient: shutdown 2024-08-19T10:41:38.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:38.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 -- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e80a69c0 msgr2=0x7f63e8138490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:38.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e80a69c0 0x7f63e8138490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f63d801aa50 tx=0x7f63d8019140 comp rx=0 tx=0).stop 2024-08-19T10:41:38.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 -- 172.21.15.190:0/2822456602 shutdown_connections 2024-08-19T10:41:38.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63d403bdf0 0x7f63d403e2b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e8139780 0x7f63e813dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e80a6da0 0x7f63e81389d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 --2- 172.21.15.190:0/2822456602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63e80a69c0 0x7f63e8138490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 -- 172.21.15.190:0/2822456602 >> 172.21.15.190:0/2822456602 conn(0x7f63e80abfc0 msgr2=0x7f63e80ab3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:38.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 -- 172.21.15.190:0/2822456602 shutdown_connections 2024-08-19T10:41:38.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.549+0000 7f63ee04a640 1 -- 172.21.15.190:0/2822456602 wait complete. 2024-08-19T10:41:38.566 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush create-or-move osd.7 1.0 root=default 2024-08-19T10:41:38.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient: get_monmap_and_config 2024-08-19T10:41:38.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient: build_initial_monmap 2024-08-19T10:41:38.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient: monmap: 2024-08-19T10:41:38.688 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:38.688 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:38.688 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:38.687972+0000 2024-08-19T10:41:38.688 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:38.687972+0000 2024-08-19T10:41:38.688 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:38.688 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:38.689 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:38.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 1 Processor -- start 2024-08-19T10:41:38.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 1 -- start start 2024-08-19T10:41:38.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient: init 2024-08-19T10:41:38.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient: _add_conns ranks=[5,9,7,4,0,3,2,10,6,8,1] 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29480a3d80 0x7f294809cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient(hunting): picked mon.noname-h con 0x7f29480a3d80 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2948069750 0x7f29480a4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient(hunting): picked mon.noname-a con 0x7f2948069750 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f29480a46a0 0x7f294809b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient(hunting): picked mon.noname-i con 0x7f29480a46a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f294809c580 con 0x7f2948069750 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f29480a5c80 con 0x7f29480a3d80 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:38.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f29480a1fc0 con 0x7f29480a46a0 2024-08-19T10:41:38.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:38.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f2950452640 10 monclient(hunting): authenticate will time out at 1144.258722s 2024-08-19T10:41:38.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29480a3d80 0x7f294809cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:38.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29480a3d80 0x7f294809cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:33712/0 (socket says 172.21.15.190:33712) 2024-08-19T10:41:38.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 1 -- 172.21.15.190:0/3893200832 learned_addr learned my addr 172.21.15.190:0/3893200832 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:38.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e9ef640 1 --2- 172.21.15.190:0/3893200832 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f29480a46a0 0x7f294809b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:38.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 10 monclient(hunting): get_auth_request con 0x7f29480a3d80 auth_method 0 2024-08-19T10:41:38.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e9ef640 10 monclient(hunting): get_auth_request con 0x7f29480a46a0 auth_method 0 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e9ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e9ef640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e9ef640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e9ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e9ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e9ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294d9ed640 1 --2- 172.21.15.190:0/3893200832 >> v2:172.21.15.89:3300/0 conn(0x7f2948069750 0x7f29480a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 10 monclient(hunting): handle_auth_done global_id 5324 payload 274 2024-08-19T10:41:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 1 -- 172.21.15.190:0/3893200832 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f29480a46a0 msgr2=0x7f294809b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 1 --2- 172.21.15.190:0/3893200832 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f29480a46a0 0x7f294809b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 1 -- 172.21.15.190:0/3893200832 >> v2:172.21.15.89:3300/0 conn(0x7f2948069750 msgr2=0x7f29480a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 1 --2- 172.21.15.190:0/3893200832 >> v2:172.21.15.89:3300/0 conn(0x7f2948069750 0x7f29480a4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 10 monclient: _finish_hunting 0 2024-08-19T10:41:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 1 monclient: found mon.noname-h 2024-08-19T10:41:38.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:38.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:38.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294e1ee640 1 -- 172.21.15.190:0/3893200832 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29480a4b00 con 0x7f29480a3d80 2024-08-19T10:41:38.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294d9ed640 10 monclient: get_auth_request con 0x7f2948069750 auth_method 0 2024-08-19T10:41:38.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.685+0000 7f294d9ed640 1 --2- 172.21.15.190:0/3893200832 >> v2:172.21.15.89:3300/0 conn(0x7f2948069750 0x7f29480a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:38.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.689+0000 7f294e1ee640 1 -- 172.21.15.190:0/3893200832 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29480a3d80 msgr2=0x7f294809cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:38.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.689+0000 7f294e1ee640 1 --2- 172.21.15.190:0/3893200832 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29480a3d80 0x7f294809cfd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2944009870 tx=0x7f294401c4b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=fe5d424abfe561bb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:38.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 1 -- 172.21.15.190:0/3893200832 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f29440b0070 con 0x7f29480a3d80 2024-08-19T10:41:38.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:38.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:38.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 10 monclient: dump: 2024-08-19T10:41:38.701 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:38.701 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:38.701 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:38.701 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:38.701 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:38.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 10 monclient: _finish_auth 0 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 10 monclient: _check_auth_tickets 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 1 -- 172.21.15.190:0/3893200832 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29440a9980 con 0x7f29480a3d80 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 5 monclient: authenticate success, global_id 5324 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 1 -- 172.21.15.190:0/3893200832 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f29440aa3a0 con 0x7f29480a3d80 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 4 set_mon_vals no callback set 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294d1ec640 10 monclient: dump: 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:38.703 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:38.704 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:38.704 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:38.704 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:38.704 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:38.704 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:38.704 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:38.704 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:38.704 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:38.704 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:38.704 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:38.705 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:38.705 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:38.705 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:38.705 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:38.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient: shutdown 2024-08-19T10:41:38.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:38.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 -- 172.21.15.190:0/3893200832 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29480a3d80 msgr2=0x7f294809cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:38.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 --2- 172.21.15.190:0/3893200832 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29480a3d80 0x7f294809cfd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2944009870 tx=0x7f294401c4b0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 -- 172.21.15.190:0/3893200832 shutdown_connections 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 --2- 172.21.15.190:0/3893200832 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f29480a46a0 0x7f294809b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 --2- 172.21.15.190:0/3893200832 >> v2:172.21.15.89:3300/0 conn(0x7f2948069750 0x7f29480a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 --2- 172.21.15.190:0/3893200832 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29480a3d80 0x7f294809cfd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 -- 172.21.15.190:0/3893200832 >> 172.21.15.190:0/3893200832 conn(0x7f29480a4e70 msgr2=0x7f29480a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 -- 172.21.15.190:0/3893200832 shutdown_connections 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 -- 172.21.15.190:0/3893200832 wait complete. 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient: build_initial_monmap 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient: monmap: 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:38.706 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 Processor -- start 2024-08-19T10:41:38.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 -- start start 2024-08-19T10:41:38.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient: init 2024-08-19T10:41:38.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:38.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient: _add_conns ranks=[3,0,6,7,2,5,4,1,8] 2024-08-19T10:41:38.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2948069750 0x7f29481d1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:38.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient(hunting): picked mon.noname-g con 0x7f2948069750 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:38.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29480a3d80 0x7f29481d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:38.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient(hunting): picked mon.noname-a con 0x7f29480a3d80 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:38.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f29480a46a0 0x7f29481d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:38.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient(hunting): picked mon.noname-d con 0x7f29480a46a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:38.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2948141340 con 0x7f29480a3d80 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f29481411c0 con 0x7f2948069750 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f29481414c0 con 0x7f29480a46a0 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f2950452640 10 monclient(hunting): authenticate will time out at 1144.267900s 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294e1ee640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2948069750 msgr2=0x7f29481d1570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.693+0000 7f294e1ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2948069750 0x7f29481d1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29480a3d80 0x7f29481d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294e9ef640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f29480a46a0 0x7f29481d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:38.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294e9ef640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f29480a46a0 0x7f29481d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:56908/0 (socket says 172.21.15.190:56908) 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294e9ef640 1 -- 172.21.15.190:0/3917718400 learned_addr learned my addr 172.21.15.190:0/3917718400 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294e9ef640 10 monclient(hunting): get_auth_request con 0x7f29480a46a0 auth_method 0 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294e9ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294e9ef640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294e9ef640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 10 monclient(hunting): get_auth_request con 0x7f29480a3d80 auth_method 0 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294e9ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:38.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294e9ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:38.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294e9ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:38.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 10 monclient(hunting): handle_auth_done global_id 5382 payload 824 2024-08-19T10:41:38.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 1 -- 172.21.15.190:0/3917718400 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f29480a46a0 msgr2=0x7f29481d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:38.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f29480a46a0 0x7f29481d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 1 -- 172.21.15.190:0/3917718400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2948069750 msgr2=0x7f29481d1570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:38.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2948069750 0x7f29481d1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:38.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 10 monclient: _finish_hunting 0 2024-08-19T10:41:38.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 1 monclient: found mon.noname-a 2024-08-19T10:41:38.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:38.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:38.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 1 -- 172.21.15.190:0/3917718400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294813dc50 con 0x7f29480a3d80 2024-08-19T10:41:38.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 1 -- 172.21.15.190:0/3917718400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29480a3d80 msgr2=0x7f29481d1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:38.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f294d9ed640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29480a3d80 0x7f29481d1ab0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f293800c820 tx=0x7f293800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:38.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.697+0000 7f293effd640 1 -- 172.21.15.190:0/3917718400 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2938015070 con 0x7f29480a3d80 2024-08-19T10:41:38.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:38.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:38.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 10 monclient: dump: 2024-08-19T10:41:38.714 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:38.715 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:38.715 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:38.715 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:38.715 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:38.715 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:38.715 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:38.716 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:38.716 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:38.716 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:38.716 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:38.716 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:38.716 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:38.717 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:38.717 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:38.717 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:38.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 10 monclient: _finish_auth 0 2024-08-19T10:41:38.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:38.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:38.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 1 -- 172.21.15.190:0/3917718400 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293800ce10 con 0x7f29480a3d80 2024-08-19T10:41:38.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:38.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 1 -- 172.21.15.190:0/3917718400 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2938011860 con 0x7f29480a3d80 2024-08-19T10:41:38.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:38.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:38.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 10 monclient: dump: 2024-08-19T10:41:38.718 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:38.719 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:38.719 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:38.719 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:38.719 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:38.719 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:38.719 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:38.719 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:38.720 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:38.720 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:38.720 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:38.720 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:38.720 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:38.720 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:38.721 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:38.721 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:38.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f294c9eb640 4 set_mon_vals no callback set 2024-08-19T10:41:38.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f2950452640 5 monclient: authenticate success, global_id 5382 2024-08-19T10:41:38.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f2950452640 10 monclient: _renew_subs 2024-08-19T10:41:38.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f2950452640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:38.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f2950452640 1 -- 172.21.15.190:0/3917718400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29480a56c0 con 0x7f29480a3d80 2024-08-19T10:41:38.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f2950452640 10 monclient: _renew_subs 2024-08-19T10:41:38.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f2950452640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:38.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f2950452640 1 -- 172.21.15.190:0/3917718400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29480a1500 con 0x7f29480a3d80 2024-08-19T10:41:38.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f2950452640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:38.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f2950452640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:38.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f2950452640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:38.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f2950452640 1 -- 172.21.15.190:0/3917718400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294809cfd0 con 0x7f29480a3d80 2024-08-19T10:41:38.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 1 -- 172.21.15.190:0/3917718400 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f29380113f0 con 0x7f29480a3d80 2024-08-19T10:41:38.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f292c03be00 0x7f292c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:38.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f293effd640 1 -- 172.21.15.190:0/3917718400 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5955+0+0 (secure 0 0 0) 0x7f293804ec90 con 0x7f29480a3d80 2024-08-19T10:41:38.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f294e1ee640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f292c03be00 0x7f292c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:38.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.701+0000 7f294e1ee640 10 monclient: get_auth_request con 0x7f292c03be00 auth_method 0 2024-08-19T10:41:38.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.705+0000 7f294e1ee640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f292c03be00 0x7f292c03e2c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f29480a1f70 tx=0x7f29440a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:38.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.713+0000 7f293effd640 1 -- 172.21.15.190:0/3917718400 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f293801e030 con 0x7f29480a3d80 2024-08-19T10:41:38.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.713+0000 7f293effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:38.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.713+0000 7f293effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:38.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.889+0000 7f2950452640 10 start_mon_command cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-08-19T10:41:38.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.889+0000 7f2950452640 10 monclient: _send_command 2 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-08-19T10:41:38.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.889+0000 7f2950452640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:38.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:38.889+0000 7f2950452640 1 -- 172.21.15.190:0/3917718400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]} v 0) -- 0x7f294809ffe0 con 0x7f29480a3d80 2024-08-19T10:41:39.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.457+0000 7f293effd640 1 -- 172.21.15.190:0/3917718400 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}]=0 create-or-move updating item name 'osd.7' weight 1 at location {root=default} to crush map v101) ==== 212+0+0 (secure 0 0 0) 0x7f29480a56c0 con 0x7f29480a3d80 2024-08-19T10:41:39.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.457+0000 7f293effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-08-19T10:41:39.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.457+0000 7f293effd640 10 monclient: _finish_command 2 = system:0 create-or-move updating item name 'osd.7' weight 1 at location {root=default} to crush map 2024-08-19T10:41:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.569+0000 7f2950452640 10 start_mon_command cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-08-19T10:41:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.569+0000 7f2950452640 10 monclient: _send_command 3 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-08-19T10:41:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.569+0000 7f2950452640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.569+0000 7f2950452640 1 -- 172.21.15.190:0/3917718400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]} v 0) -- 0x7f29480008e0 con 0x7f29480a3d80 2024-08-19T10:41:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f293effd640 1 -- 172.21.15.190:0/3917718400 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}]=0 create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map v101) ==== 211+0+0 (secure 0 0 0) 0x7f294813dc50 con 0x7f29480a3d80 2024-08-19T10:41:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f293effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush create-or-move", "id": 7, "weight": 1.0, "args": ["root=default"]}] 2024-08-19T10:41:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f293effd640 10 monclient: _finish_command 3 = system:0 create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map 2024-08-19T10:41:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:create-or-move updated item name 'osd.7' weight 1 at location {root=default} to crush map 2024-08-19T10:41:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 -- 172.21.15.190:0/3917718400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f292c03be00 msgr2=0x7f292c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f292c03be00 0x7f292c03e2c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f29480a1f70 tx=0x7f29440a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 10 monclient: shutdown 2024-08-19T10:41:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 -- 172.21.15.190:0/3917718400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29480a3d80 msgr2=0x7f29481d1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29480a3d80 0x7f29481d1ab0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f293800c820 tx=0x7f293800cce0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 -- 172.21.15.190:0/3917718400 shutdown_connections 2024-08-19T10:41:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f292c03be00 0x7f292c03e2c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f29480a46a0 0x7f29481d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29480a3d80 0x7f29481d1ab0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 --2- 172.21.15.190:0/3917718400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2948069750 0x7f29481d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 -- 172.21.15.190:0/3917718400 >> 172.21.15.190:0/3917718400 conn(0x7f29480a4e70 msgr2=0x7f2948098330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 -- 172.21.15.190:0/3917718400 shutdown_connections 2024-08-19T10:41:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.573+0000 7f2950452640 1 -- 172.21.15.190:0/3917718400 wait complete. 2024-08-19T10:41:39.593 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush move osd.7 host=foo 2024-08-19T10:41:39.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.709+0000 7fe33f1e7640 10 monclient: get_monmap_and_config 2024-08-19T10:41:39.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.709+0000 7fe33f1e7640 10 monclient: build_initial_monmap 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.709+0000 7fe33f1e7640 10 monclient: monmap: 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:39.714033+0000 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:39.714033+0000 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:39.714 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:39.715 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:39.715 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:39.715 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:39.715 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:39.715 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:39.715 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:39.715 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:39.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.709+0000 7fe33f1e7640 1 Processor -- start 2024-08-19T10:41:39.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.709+0000 7fe33f1e7640 1 -- start start 2024-08-19T10:41:39.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.709+0000 7fe33f1e7640 10 monclient: init 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 10 monclient: _add_conns ranks=[5,8,9,6,10,1,2,0,7,3,4] 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe3380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 10 monclient(hunting): picked mon.noname-h con 0x7fe3381399d0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe338139db0 0x7fe33813a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 10 monclient(hunting): picked mon.noname-e con 0x7fe338139db0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe3380a6850 0x7fe3380a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 10 monclient(hunting): picked mon.noname-a con 0x7fe3380a6850 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fe3380a3470 con 0x7fe3380a6850 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:39.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe3380a49c0 con 0x7fe3381399d0 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fe3380a46e0 con 0x7fe338139db0 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33f1e7640 10 monclient(hunting): authenticate will time out at 1145.284558s 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe3380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe3380a5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:33722/0 (socket says 172.21.15.190:33722) 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 1 -- 172.21.15.190:0/3212332491 learned_addr learned my addr 172.21.15.190:0/3212332491 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 10 monclient(hunting): get_auth_request con 0x7fe3381399d0 auth_method 0 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:39.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33d784640 1 --2- 172.21.15.190:0/3212332491 >> v2:172.21.15.89:3300/0 conn(0x7fe3380a6850 0x7fe3380a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 10 monclient(hunting): handle_auth_done global_id 5333 payload 274 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe337fff640 1 --2- 172.21.15.190:0/3212332491 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe338139db0 0x7fe33813a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 1 -- 172.21.15.190:0/3212332491 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe338139db0 msgr2=0x7fe33813a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 1 --2- 172.21.15.190:0/3212332491 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe338139db0 0x7fe33813a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 1 -- 172.21.15.190:0/3212332491 >> v2:172.21.15.89:3300/0 conn(0x7fe3380a6850 msgr2=0x7fe3380a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 1 --2- 172.21.15.190:0/3212332491 >> v2:172.21.15.89:3300/0 conn(0x7fe3380a6850 0x7fe3380a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 10 monclient: _finish_hunting 0 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 1 monclient: found mon.noname-h 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:39.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33cf83640 1 -- 172.21.15.190:0/3212332491 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3380a4ca0 con 0x7fe3381399d0 2024-08-19T10:41:39.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33d784640 10 monclient: get_auth_request con 0x7fe3380a6850 auth_method 0 2024-08-19T10:41:39.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.713+0000 7fe33d784640 1 --2- 172.21.15.190:0/3212332491 >> v2:172.21.15.89:3300/0 conn(0x7fe3380a6850 0x7fe3380a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:39.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33cf83640 1 -- 172.21.15.190:0/3212332491 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 msgr2=0x7fe3380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:39.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33cf83640 1 --2- 172.21.15.190:0/3212332491 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe3380a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009870 tx=0x7fe32c01c420 comp rx=0 tx=0).ready entity=mon.5 client_cookie=f5b8eee629b8d051 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:39.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 1 -- 172.21.15.190:0/3212332491 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe32c0b0070 con 0x7fe3381399d0 2024-08-19T10:41:39.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 10 monclient: dump: 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:39.725 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 5 monclient: authenticate success, global_id 5333 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 1 -- 172.21.15.190:0/3212332491 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32c0aa810 con 0x7fe3381399d0 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:39.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 4 set_mon_vals no callback set 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 1 -- 172.21.15.190:0/3212332491 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe32c0aae30 con 0x7fe3381399d0 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe3377fe640 10 monclient: dump: 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:39.727 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:39.728 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:39.728 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:39.728 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:39.728 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:39.728 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:39.728 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:39.728 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:39.728 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:39.728 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:39.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient: shutdown 2024-08-19T10:41:39.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:39.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3212332491 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 msgr2=0x7fe3380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:39.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 --2- 172.21.15.190:0/3212332491 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe3380a5790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009870 tx=0x7fe32c01c420 comp rx=0 tx=0).stop 2024-08-19T10:41:39.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3212332491 shutdown_connections 2024-08-19T10:41:39.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 --2- 172.21.15.190:0/3212332491 >> v2:172.21.15.89:3300/0 conn(0x7fe3380a6850 0x7fe3380a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 --2- 172.21.15.190:0/3212332491 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe338139db0 0x7fe33813a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 --2- 172.21.15.190:0/3212332491 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe3380a5790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3212332491 >> 172.21.15.190:0/3212332491 conn(0x7fe3380abe50 msgr2=0x7fe3380ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:39.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3212332491 shutdown_connections 2024-08-19T10:41:39.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3212332491 wait complete. 2024-08-19T10:41:39.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient: build_initial_monmap 2024-08-19T10:41:39.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient: monmap: 2024-08-19T10:41:39.730 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:39.730 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:39.730 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:39.730 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:39.730 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:39.730 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:39.730 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:39.730 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 Processor -- start 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 -- start start 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient: init 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient: _add_conns ranks=[2,5,8,6,3,4,7,0,1] 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3380a6850 0x7fe338137ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient(hunting): picked mon.noname-b con 0x7fe3380a6850 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe338138220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient(hunting): picked mon.noname-h con 0x7fe3381399d0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe338139db0 0x7fe33813de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient(hunting): picked mon.noname-e con 0x7fe338139db0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fe3380a5e50 con 0x7fe3380a6850 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe3380a5cd0 con 0x7fe3381399d0 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fe338143160 con 0x7fe338139db0 2024-08-19T10:41:39.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.717+0000 7fe33f1e7640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33f1e7640 10 monclient(hunting): authenticate will time out at 1145.292011s 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe338138220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe338138220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:33730/0 (socket says 172.21.15.190:33730) 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 1 -- 172.21.15.190:0/3849074126 learned_addr learned my addr 172.21.15.190:0/3849074126 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 10 monclient(hunting): get_auth_request con 0x7fe3381399d0 auth_method 0 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:39.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33d784640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe338139db0 0x7fe33813de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:39.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33d784640 10 monclient(hunting): get_auth_request con 0x7fe338139db0 auth_method 0 2024-08-19T10:41:39.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33d784640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:39.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33d784640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:39.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33d784640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:39.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33cf83640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3380a6850 0x7fe338137ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:39.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 10 monclient(hunting): handle_auth_done global_id 5342 payload 824 2024-08-19T10:41:39.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 1 -- 172.21.15.190:0/3849074126 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe338139db0 msgr2=0x7fe33813de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:39.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe338139db0 0x7fe33813de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 1 -- 172.21.15.190:0/3849074126 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3380a6850 msgr2=0x7fe338137ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:39.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3380a6850 0x7fe338137ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:39.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:39.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 1 monclient: found mon.noname-h 2024-08-19T10:41:39.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:39.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:39.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 1 -- 172.21.15.190:0/3849074126 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33813fef0 con 0x7fe3381399d0 2024-08-19T10:41:39.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 1 -- 172.21.15.190:0/3849074126 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 msgr2=0x7fe338138220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:39.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe337fff640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe338138220 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe32400c820 tx=0x7fe32400cce0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:39.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 1 -- 172.21.15.190:0/3849074126 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe324015070 con 0x7fe3381399d0 2024-08-19T10:41:39.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:39.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:39.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 10 monclient: dump: 2024-08-19T10:41:39.740 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:39.740 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:39.740 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:39.740 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:39.740 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:39.740 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:39.740 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:39.740 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:39.741 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:39.741 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:39.741 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:39.741 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:39.741 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:39.741 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:39.741 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:39.742 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:39.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 10 monclient: _finish_auth 0 2024-08-19T10:41:39.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 10 monclient: _check_auth_tickets 2024-08-19T10:41:39.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:39.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 1 -- 172.21.15.190:0/3849074126 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32400ce10 con 0x7fe3381399d0 2024-08-19T10:41:39.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33f1e7640 5 monclient: authenticate success, global_id 5342 2024-08-19T10:41:39.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:39.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 1 -- 172.21.15.190:0/3849074126 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe324011860 con 0x7fe3381399d0 2024-08-19T10:41:39.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:39.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:39.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3357fa640 10 monclient: dump: 2024-08-19T10:41:39.743 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:39.744 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:39.744 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:39.744 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:39.744 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:39.744 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:39.744 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:39.744 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:39.744 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe3367fc640 4 set_mon_vals no callback set 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33f1e7640 10 monclient: _renew_subs 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33f1e7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3849074126 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe338123ae0 con 0x7fe3381399d0 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33f1e7640 10 monclient: _renew_subs 2024-08-19T10:41:39.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33f1e7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:39.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3849074126 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fe338140280 con 0x7fe3381399d0 2024-08-19T10:41:39.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33f1e7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:39.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.721+0000 7fe33f1e7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:39.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.725+0000 7fe33f1e7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:39.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.725+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3849074126 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3380a92a0 con 0x7fe3381399d0 2024-08-19T10:41:39.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.725+0000 7fe3357fa640 1 -- 172.21.15.190:0/3849074126 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe324018070 con 0x7fe3381399d0 2024-08-19T10:41:39.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.725+0000 7fe3357fa640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe31c03be30 0x7fe31c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:39.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.725+0000 7fe3357fa640 1 -- 172.21.15.190:0/3849074126 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(101..101 src has 1..101) ==== 5976+0+0 (secure 0 0 0) 0x7fe32404ecb0 con 0x7fe3381399d0 2024-08-19T10:41:39.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.725+0000 7fe33cf83640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe31c03be30 0x7fe31c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:39.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.725+0000 7fe33cf83640 10 monclient: get_auth_request con 0x7fe31c03be30 auth_method 0 2024-08-19T10:41:39.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.725+0000 7fe33cf83640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe31c03be30 0x7fe31c03e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009870 tx=0x7fe32c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:39.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.729+0000 7fe3357fa640 1 -- 172.21.15.190:0/3849074126 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe32401f030 con 0x7fe3381399d0 2024-08-19T10:41:39.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.729+0000 7fe3357fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:39.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.729+0000 7fe3357fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:39.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.901+0000 7fe33f1e7640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-08-19T10:41:39.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.901+0000 7fe33f1e7640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-08-19T10:41:39.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.901+0000 7fe33f1e7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:39.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:39.901+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3849074126 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]} v 0) -- 0x7fe33806c8d0 con 0x7fe3381399d0 2024-08-19T10:41:40.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.249+0000 7fe3357fa640 1 -- 172.21.15.190:0/3849074126 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}]=0 moved item id 7 name 'osd.7' to location {host=foo} in crush map v102) ==== 165+0+0 (secure 0 0 0) 0x7fe324018360 con 0x7fe3381399d0 2024-08-19T10:41:40.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.249+0000 7fe3357fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-08-19T10:41:40.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.249+0000 7fe3357fa640 10 monclient: _finish_command 2 = system:0 moved item id 7 name 'osd.7' to location {host=foo} in crush map 2024-08-19T10:41:40.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.365+0000 7fe33f1e7640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-08-19T10:41:40.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.365+0000 7fe33f1e7640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-08-19T10:41:40.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.365+0000 7fe33f1e7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:40.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.365+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3849074126 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]} v 0) -- 0x7fe3380a5790 con 0x7fe3381399d0 2024-08-19T10:41:40.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.365+0000 7fe3357fa640 1 -- 172.21.15.190:0/3849074126 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}]=0 no need to move item id 7 name 'osd.7' to location {host=foo} in crush map v102) ==== 175+0+0 (secure 0 0 0) 0x7fe324011680 con 0x7fe3381399d0 2024-08-19T10:41:40.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.365+0000 7fe3357fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "osd.7", "args": ["host=foo"]}] 2024-08-19T10:41:40.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.365+0000 7fe3357fa640 10 monclient: _finish_command 3 = system:0 no need to move item id 7 name 'osd.7' to location {host=foo} in crush map 2024-08-19T10:41:40.369 INFO:tasks.workunit.client.0.smithi190.stderr:no need to move item id 7 name 'osd.7' to location {host=foo} in crush map 2024-08-19T10:41:40.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3849074126 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe31c03be30 msgr2=0x7fe31c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:40.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe31c03be30 0x7fe31c03e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009870 tx=0x7fe32c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:40.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 10 monclient: shutdown 2024-08-19T10:41:40.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:40.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3849074126 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 msgr2=0x7fe338138220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:40.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe338138220 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe32400c820 tx=0x7fe32400cce0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3849074126 shutdown_connections 2024-08-19T10:41:40.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe31c03be30 0x7fe31c03e2f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe338139db0 0x7fe33813de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3381399d0 0x7fe338138220 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 --2- 172.21.15.190:0/3849074126 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3380a6850 0x7fe338137ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3849074126 >> 172.21.15.190:0/3849074126 conn(0x7fe3380abe50 msgr2=0x7fe3380aaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:40.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3849074126 shutdown_connections 2024-08-19T10:41:40.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.369+0000 7fe33f1e7640 1 -- 172.21.15.190:0/3849074126 wait complete. 2024-08-19T10:41:40.388 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd find osd.7 2024-08-19T10:41:40.388 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep foo 2024-08-19T10:41:40.388 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep host 2024-08-19T10:41:40.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.493+0000 7f991b568640 10 monclient: get_monmap_and_config 2024-08-19T10:41:40.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.493+0000 7f991b568640 10 monclient: build_initial_monmap 2024-08-19T10:41:40.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 10 monclient: monmap: 2024-08-19T10:41:40.505 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:40.505 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:40.505 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:40.499603+0000 2024-08-19T10:41:40.505 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:40.499603+0000 2024-08-19T10:41:40.506 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:40.506 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.506 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:40.506 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:40.507 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:40.507 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:40.507 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:40.507 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:40.508 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:40.508 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:40.508 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:40.508 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:40.509 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:40.509 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 1 Processor -- start 2024-08-19T10:41:40.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 1 -- start start 2024-08-19T10:41:40.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 10 monclient: init 2024-08-19T10:41:40.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:40.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 10 monclient: _add_conns ranks=[3,1,4,6,8,9,2,10,5,7,0] 2024-08-19T10:41:40.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99140a56c0 0x7f9914071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 10 monclient(hunting): picked mon.noname-g con 0x7f99140a56c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:40.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f99140a5aa0 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:40.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991413e950 0x7f9914140d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 10 monclient(hunting): picked mon.noname-c con 0x7f991413e950 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:40.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991406e930 con 0x7f99140a5aa0 2024-08-19T10:41:40.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:40.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f991406de60 con 0x7f99140a56c0 2024-08-19T10:41:40.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:40.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f991406e0e0 con 0x7f991413e950 2024-08-19T10:41:40.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:40.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f991b568640 10 monclient(hunting): authenticate will time out at 1146.069558s 2024-08-19T10:41:40.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919304640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99140a56c0 msgr2=0x7f9914071fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:40.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919304640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99140a56c0 0x7f9914071fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:40.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9918b03640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f99140a5aa0 0x7f99140a5ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:40134/0 (socket says 172.21.15.190:40134) 2024-08-19T10:41:40.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9918b03640 1 -- 172.21.15.190:0/3549708123 learned_addr learned my addr 172.21.15.190:0/3549708123 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:40.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 1 --2- 172.21.15.190:0/3549708123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991413e950 0x7f9914140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 10 monclient(hunting): get_auth_request con 0x7f991413e950 auth_method 0 2024-08-19T10:41:40.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:40.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:40.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:40.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:40.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:40.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:40.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 10 monclient(hunting): handle_auth_done global_id 5323 payload 274 2024-08-19T10:41:40.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 1 -- 172.21.15.190:0/3549708123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99140a56c0 msgr2=0x7f9914071fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:40.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 1 --2- 172.21.15.190:0/3549708123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99140a56c0 0x7f9914071fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 1 -- 172.21.15.190:0/3549708123 >> v1:172.21.15.190:6789/0 conn(0x7f99140a5aa0 legacy=0x7f99140a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:40.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 10 monclient: _finish_hunting 0 2024-08-19T10:41:40.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 1 monclient: found mon.noname-c 2024-08-19T10:41:40.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:40.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:40.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.497+0000 7f9919b05640 1 -- 172.21.15.190:0/3549708123 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991406f180 con 0x7f991413e950 2024-08-19T10:41:40.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f9919b05640 1 -- 172.21.15.190:0/3549708123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991413e950 msgr2=0x7f9914140d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:40.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f9919b05640 1 --2- 172.21.15.190:0/3549708123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991413e950 0x7f9914140d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f990c009dc0 tx=0x7f990c01c870 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1587d780bc8ba607 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:40.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 1 -- 172.21.15.190:0/3549708123 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f990c0a6a60 con 0x7f991413e950 2024-08-19T10:41:40.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:40.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:40.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 10 monclient: dump: 2024-08-19T10:41:40.519 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:40.519 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:40.519 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:40.520 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 10 monclient: _finish_auth 0 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 1 -- 172.21.15.190:0/3549708123 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c0a6c20 con 0x7f991413e950 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 1 -- 172.21.15.190:0/3549708123 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f990c0a6dd0 con 0x7f991413e950 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f990bfff640 10 monclient: dump: 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:40.521 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f991b568640 5 monclient: authenticate success, global_id 5323 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f991b568640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.501+0000 7f991b568640 4 set_mon_vals no callback set 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient: shutdown 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 -- 172.21.15.190:0/3549708123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991413e950 msgr2=0x7f9914140d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 --2- 172.21.15.190:0/3549708123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991413e950 0x7f9914140d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f990c009dc0 tx=0x7f990c01c870 comp rx=0 tx=0).stop 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 -- 172.21.15.190:0/3549708123 shutdown_connections 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 --2- 172.21.15.190:0/3549708123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f991413e950 0x7f9914140d40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 --2- 172.21.15.190:0/3549708123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99140a56c0 0x7f9914071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 -- 172.21.15.190:0/3549708123 >> 172.21.15.190:0/3549708123 conn(0x7f99140aa2b0 msgr2=0x7f99140a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 -- 172.21.15.190:0/3549708123 shutdown_connections 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 -- 172.21.15.190:0/3549708123 wait complete. 2024-08-19T10:41:40.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient: build_initial_monmap 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient: monmap: 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:40.524 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 Processor -- start 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 -- start start 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient: init 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient: _add_conns ranks=[2,8,0,3,6,4,5,7,1] 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99140a56c0 0x7f99141258e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient(hunting): picked mon.noname-b con 0x7f99140a56c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f99140a5aa0 0x7f99141d1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient(hunting): picked mon.noname-e con 0x7f99140a5aa0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:40.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991413e950 0x7f9914123380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient(hunting): picked mon.noname-a con 0x7f991413e950 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9914143c20 con 0x7f991413e950 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9914143aa0 con 0x7f99140a56c0 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9914143da0 con 0x7f99140a5aa0 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f991b568640 10 monclient(hunting): authenticate will time out at 1146.077681s 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9918b03640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f99140a5aa0 0x7f99141d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99140a56c0 0x7f99141258e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919b05640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991413e950 0x7f9914123380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9918b03640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f99140a5aa0 0x7f99141d1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:44026/0 (socket says 172.21.15.190:44026) 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9918b03640 1 -- 172.21.15.190:0/1001867683 learned_addr learned my addr 172.21.15.190:0/1001867683 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9918b03640 10 monclient(hunting): get_auth_request con 0x7f99140a5aa0 auth_method 0 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9918b03640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9918b03640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9918b03640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 10 monclient(hunting): get_auth_request con 0x7f99140a56c0 auth_method 0 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919b05640 10 monclient(hunting): get_auth_request con 0x7f991413e950 auth_method 0 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919b05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919b05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:40.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919b05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9918b03640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9918b03640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9918b03640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919b05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919b05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919b05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 10 monclient(hunting): handle_auth_done global_id 5303 payload 824 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 1 -- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f99140a5aa0 msgr2=0x7f99141d1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f99140a5aa0 0x7f99141d1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 1 -- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991413e950 msgr2=0x7f9914123380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991413e950 0x7f9914123380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 10 monclient: _finish_hunting 0 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 1 monclient: found mon.noname-b 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919304640 1 -- 172.21.15.190:0/1001867683 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99140a8650 con 0x7f99140a56c0 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9918b03640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f99140a5aa0 0x7f99141d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.505+0000 7f9919b05640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991413e950 0x7f9914123380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9919304640 1 -- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99140a56c0 msgr2=0x7f99141258e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9919304640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99140a56c0 0x7f99141258e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f991000b430 tx=0x7f991000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 1 -- 172.21.15.190:0/1001867683 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9910016070 con 0x7f99140a56c0 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:40.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 10 monclient: dump: 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:40.530 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f991b568640 5 monclient: authenticate success, global_id 5303 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 1 -- 172.21.15.190:0/1001867683 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9910003de0 con 0x7f99140a56c0 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 1 -- 172.21.15.190:0/1001867683 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f99100103a0 con 0x7f99140a56c0 2024-08-19T10:41:40.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f990b7fe640 4 set_mon_vals no callback set 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 10 monclient: dump: 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:40.532 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:40.533 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:40.533 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:40.533 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:40.533 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:40.533 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f991b568640 10 monclient: _renew_subs 2024-08-19T10:41:40.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f991b568640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:41:40.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f991b568640 1 -- 172.21.15.190:0/1001867683 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9914126e00 con 0x7f99140a56c0 2024-08-19T10:41:40.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f991b568640 10 monclient: _renew_subs 2024-08-19T10:41:40.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f991b568640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:41:40.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f991b568640 1 -- 172.21.15.190:0/1001867683 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f99140a8100 con 0x7f99140a56c0 2024-08-19T10:41:40.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f991b568640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:40.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f991b568640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:40.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f991b568640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:41:40.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f991b568640 1 -- 172.21.15.190:0/1001867683 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9914071fe0 con 0x7f99140a56c0 2024-08-19T10:41:40.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 1 -- 172.21.15.190:0/1001867683 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9910005070 con 0x7f99140a56c0 2024-08-19T10:41:40.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98f803be00 0x7f98f803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.509+0000 7f9909ffb640 1 -- 172.21.15.190:0/1001867683 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 6150+0+0 (secure 0 0 0) 0x7f991004e670 con 0x7f99140a56c0 2024-08-19T10:41:40.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.513+0000 7f9918b03640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98f803be00 0x7f98f803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.513+0000 7f9918b03640 10 monclient: get_auth_request con 0x7f98f803be00 auth_method 0 2024-08-19T10:41:40.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.513+0000 7f9918b03640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98f803be00 0x7f98f803e2c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9904007d30 tx=0x7f9904007c80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:40.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.525+0000 7f9909ffb640 1 -- 172.21.15.190:0/1001867683 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f991001f020 con 0x7f99140a56c0 2024-08-19T10:41:40.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.525+0000 7f9909ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:40.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.525+0000 7f9909ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:40.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.693+0000 7f991b568640 10 start_mon_command cmd=[{"prefix": "osd find", "id": 7}] 2024-08-19T10:41:40.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.693+0000 7f991b568640 10 monclient: _send_command 2 [{"prefix": "osd find", "id": 7}] 2024-08-19T10:41:40.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.693+0000 7f991b568640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:41:40.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.693+0000 7f991b568640 1 -- 172.21.15.190:0/1001867683 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd find", "id": 7} v 0) -- 0x7f99141c8cc0 con 0x7f99140a56c0 2024-08-19T10:41:40.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.709+0000 7f9909ffb640 1 -- 172.21.15.190:0/1001867683 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd find", "id": 7}]=0 v102) ==== 65+0+226 (secure 0 0 0) 0x7f99140a8650 con 0x7f99140a56c0 2024-08-19T10:41:40.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.709+0000 7f9909ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd find", "id": 7}] 2024-08-19T10:41:40.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.709+0000 7f9909ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:40.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 10 start_mon_command cmd=[{"prefix": "osd find", "id": 7}] 2024-08-19T10:41:40.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 10 monclient: _send_command 3 [{"prefix": "osd find", "id": 7}] 2024-08-19T10:41:40.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:41:40.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 -- 172.21.15.190:0/1001867683 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd find", "id": 7} v 0) -- 0x7f9914126270 con 0x7f99140a56c0 2024-08-19T10:41:40.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f9909ffb640 1 -- 172.21.15.190:0/1001867683 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd find", "id": 7}]=0 v102) ==== 65+0+226 (secure 0 0 0) 0x7f991000f300 con 0x7f99140a56c0 2024-08-19T10:41:40.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f9909ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd find", "id": 7}] 2024-08-19T10:41:40.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f9909ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:40.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 -- 172.21.15.190:0/1001867683 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98f803be00 msgr2=0x7f98f803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:40.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98f803be00 0x7f98f803e2c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9904007d30 tx=0x7f9904007c80 comp rx=0 tx=0).stop 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 10 monclient: shutdown 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 -- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99140a56c0 msgr2=0x7f99141258e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99140a56c0 0x7f99141258e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f991000b430 tx=0x7f991000b8f0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 -- 172.21.15.190:0/1001867683 shutdown_connections 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f991413e950 0x7f9914123380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98f803be00 0x7f98f803e2c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f99140a5aa0 0x7f99141d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 --2- 172.21.15.190:0/1001867683 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99140a56c0 0x7f99141258e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 -- 172.21.15.190:0/1001867683 >> 172.21.15.190:0/1001867683 conn(0x7f99140aa2b0 msgr2=0x7f99140a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 -- 172.21.15.190:0/1001867683 shutdown_connections 2024-08-19T10:41:40.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.813+0000 7f991b568640 1 -- 172.21.15.190:0/1001867683 wait complete. 2024-08-19T10:41:40.833 INFO:tasks.workunit.client.0.smithi190.stdout: "host": "foo", 2024-08-19T10:41:40.833 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm osd.7 2024-08-19T10:41:40.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.945+0000 7f04418da640 10 monclient: get_monmap_and_config 2024-08-19T10:41:40.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.945+0000 7f04418da640 10 monclient: build_initial_monmap 2024-08-19T10:41:40.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.945+0000 7f04418da640 10 monclient: monmap: 2024-08-19T10:41:40.950 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:40.950 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:40.950 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:40.950299+0000 2024-08-19T10:41:40.950 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:40.950299+0000 2024-08-19T10:41:40.950 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:40.951 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.951 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:40.951 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:40.951 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:40.951 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:40.951 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:40.951 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:40.951 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:40.951 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:40.952 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:40.952 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:40.952 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:40.952 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.945+0000 7f04418da640 1 Processor -- start 2024-08-19T10:41:40.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.945+0000 7f04418da640 1 -- start start 2024-08-19T10:41:40.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.945+0000 7f04418da640 10 monclient: init 2024-08-19T10:41:40.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:40.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 10 monclient: _add_conns ranks=[2,8,4,1,0,3,5,6,7,10,9] 2024-08-19T10:41:40.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 0x7f043c0a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 10 monclient(hunting): picked mon.noname-b con 0x7f043c0a6850 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:40.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f043c0a6c30 0x7f043c0a7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 10 monclient(hunting): picked mon.noname-e con 0x7f043c0a6c30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:40.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f043c1396a0 0x7f043c139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 10 monclient(hunting): picked mon.noname-c con 0x7f043c1396a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:40.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:40.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f043c0a3640 con 0x7f043c0a6850 2024-08-19T10:41:40.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:40.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f043c0a3850 con 0x7f043c1396a0 2024-08-19T10:41:40.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:40.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f043c0a46e0 con 0x7f043c0a6c30 2024-08-19T10:41:40.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:40.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f04418da640 10 monclient(hunting): authenticate will time out at 1146.520157s 2024-08-19T10:41:40.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043affd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 0x7f043c0a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f043c0a6c30 0x7f043c0a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043b7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f043c1396a0 0x7f043c139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043affd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 0x7f043c0a57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:46842/0 (socket says 172.21.15.190:46842) 2024-08-19T10:41:40.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f043c0a6c30 0x7f043c0a7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:44038/0 (socket says 172.21.15.190:44038) 2024-08-19T10:41:40.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043affd640 1 -- 172.21.15.190:0/4126341940 learned_addr learned my addr 172.21.15.190:0/4126341940 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:40.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043affd640 10 monclient(hunting): get_auth_request con 0x7f043c0a6850 auth_method 0 2024-08-19T10:41:40.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:40.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:40.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:40.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043b7fe640 10 monclient(hunting): get_auth_request con 0x7f043c1396a0 auth_method 0 2024-08-19T10:41:40.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:40.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:40.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:40.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 10 monclient(hunting): get_auth_request con 0x7f043c0a6c30 auth_method 0 2024-08-19T10:41:40.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:40.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:40.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:40.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:40.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:40.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:40.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:40.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:40.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:40.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:40.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:40.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:40.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 10 monclient(hunting): handle_auth_done global_id 5183 payload 274 2024-08-19T10:41:40.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 1 -- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f043c1396a0 msgr2=0x7f043c139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:40.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 1 --2- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f043c1396a0 0x7f043c139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 1 -- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 msgr2=0x7f043c0a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:40.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 1 --2- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 0x7f043c0a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:41:40.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 1 monclient: found mon.noname-e 2024-08-19T10:41:40.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:40.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:40.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043a7fc640 1 -- 172.21.15.190:0/4126341940 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043c0a3350 con 0x7f043c0a6c30 2024-08-19T10:41:40.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.949+0000 7f043affd640 1 --2- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 0x7f043c0a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f043a7fc640 1 -- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f043c0a6c30 msgr2=0x7f043c0a7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f043a7fc640 1 --2- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f043c0a6c30 0x7f043c0a7010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f042c0097f0 tx=0x7f042c01c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=bbcbd3f896667ecd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 1 -- 172.21.15.190:0/4126341940 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f042c0b0070 con 0x7f043c0a6c30 2024-08-19T10:41:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 10 monclient: dump: 2024-08-19T10:41:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:40.966 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:40.967 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 1 -- 172.21.15.190:0/4126341940 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f042c0aa810 con 0x7f043c0a6c30 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 1 -- 172.21.15.190:0/4126341940 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f042c0aaa40 con 0x7f043c0a6c30 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f0439ffb640 10 monclient: dump: 2024-08-19T10:41:40.968 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:40.969 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 5 monclient: authenticate success, global_id 5183 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 4 set_mon_vals no callback set 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 10 monclient: shutdown 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 1 -- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f043c0a6c30 msgr2=0x7f043c0a7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 1 --2- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f043c0a6c30 0x7f043c0a7010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f042c0097f0 tx=0x7f042c01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 1 -- 172.21.15.190:0/4126341940 shutdown_connections 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 1 --2- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f043c1396a0 0x7f043c139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 1 --2- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f043c0a6c30 0x7f043c0a7010 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 1 --2- 172.21.15.190:0/4126341940 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 0x7f043c0a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 1 -- 172.21.15.190:0/4126341940 >> 172.21.15.190:0/4126341940 conn(0x7f043c0abe50 msgr2=0x7f043c0ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:40.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 1 -- 172.21.15.190:0/4126341940 shutdown_connections 2024-08-19T10:41:40.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 1 -- 172.21.15.190:0/4126341940 wait complete. 2024-08-19T10:41:40.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 10 monclient: build_initial_monmap 2024-08-19T10:41:40.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 10 monclient: monmap: 2024-08-19T10:41:40.971 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:40.971 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:40.971 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:40.971 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:40.971 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:40.972 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.972 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:40.972 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:40.972 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:40.972 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:40.972 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:40.972 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:40.972 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:40.972 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:40.973 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:40.973 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 1 Processor -- start 2024-08-19T10:41:40.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 1 -- start start 2024-08-19T10:41:40.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.953+0000 7f04418da640 10 monclient: init 2024-08-19T10:41:40.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:40.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient: _add_conns ranks=[2,0,7,6,4,3,8,5,1] 2024-08-19T10:41:40.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 0x7f043c09cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient(hunting): picked mon.noname-b con 0x7f043c0a6850 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:40.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043c0a6c30 0x7f043c09d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient(hunting): picked mon.noname-a con 0x7f043c0a6c30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:40.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f043c1396a0 0x7f043c09d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient(hunting): picked mon.noname-i con 0x7f043c1396a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:40.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:40.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f043c0a5eb0 con 0x7f043c0a6c30 2024-08-19T10:41:40.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:40.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f043c0a5d30 con 0x7f043c0a6850 2024-08-19T10:41:40.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:40.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f043c1433e0 con 0x7f043c1396a0 2024-08-19T10:41:40.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:40.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient(hunting): authenticate will time out at 1146.528468s 2024-08-19T10:41:40.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f043c1396a0 0x7f043c09d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f043c1396a0 0x7f043c09d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:55178/0 (socket says 172.21.15.190:55178) 2024-08-19T10:41:40.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 1 -- 172.21.15.190:0/844968698 learned_addr learned my addr 172.21.15.190:0/844968698 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:40.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 10 monclient(hunting): get_auth_request con 0x7f043c1396a0 auth_method 0 2024-08-19T10:41:40.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:40.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:40.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:40.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:40.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:40.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:40.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043affd640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 0x7f043c09cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043affd640 10 monclient(hunting): get_auth_request con 0x7f043c0a6850 auth_method 0 2024-08-19T10:41:40.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:40.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:40.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:40.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043a7fc640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043c0a6c30 0x7f043c09d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 10 monclient(hunting): handle_auth_done global_id 14542 payload 824 2024-08-19T10:41:40.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 1 -- 172.21.15.190:0/844968698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 msgr2=0x7f043c09cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:40.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 0x7f043c09cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 1 -- 172.21.15.190:0/844968698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043c0a6c30 msgr2=0x7f043c09d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:40.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043c0a6c30 0x7f043c09d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:40.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:40.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 1 monclient: found mon.noname-i 2024-08-19T10:41:40.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:40.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:40.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 1 -- 172.21.15.190:0/844968698 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043c135760 con 0x7f043c1396a0 2024-08-19T10:41:40.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 1 -- 172.21.15.190:0/844968698 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f043c1396a0 msgr2=0x7f043c09d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:40.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f043b7fe640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f043c1396a0 0x7f043c09d5c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0434009cb0 tx=0x7f043400c490 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:40.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 1 -- 172.21.15.190:0/844968698 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0434018020 con 0x7f043c1396a0 2024-08-19T10:41:40.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:40.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:40.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 10 monclient: dump: 2024-08-19T10:41:40.978 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:40.978 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:40.978 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.978 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.978 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:40.978 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.978 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:40.978 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:40.978 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 5 monclient: authenticate success, global_id 14542 2024-08-19T10:41:40.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient: _renew_subs 2024-08-19T10:41:40.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 1 -- 172.21.15.190:0/844968698 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04340148f0 con 0x7f043c1396a0 2024-08-19T10:41:40.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:40.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 1 -- 172.21.15.190:0/844968698 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043c140790 con 0x7f043c1396a0 2024-08-19T10:41:40.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient: _renew_subs 2024-08-19T10:41:40.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:40.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04418da640 1 -- 172.21.15.190:0/844968698 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f043c09f360 con 0x7f043c1396a0 2024-08-19T10:41:40.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:40.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 1 -- 172.21.15.190:0/844968698 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04340133e0 con 0x7f043c1396a0 2024-08-19T10:41:40.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:40.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f04397fa640 4 set_mon_vals no callback set 2024-08-19T10:41:40.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:40.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.957+0000 7f0423fff640 10 monclient: dump: 2024-08-19T10:41:40.981 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:40.981 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:40.981 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.981 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:40.981 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:40.981 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:40.982 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:40.982 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:40.982 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:40.982 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:40.982 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:40.982 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:40.982 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:40.982 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:40.982 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:40.983 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:40.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.961+0000 7f04418da640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:40.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.961+0000 7f04418da640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:40.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.961+0000 7f04418da640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:40.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.961+0000 7f04418da640 1 -- 172.21.15.190:0/844968698 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041c005090 con 0x7f043c1396a0 2024-08-19T10:41:40.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.961+0000 7f0423fff640 1 -- 172.21.15.190:0/844968698 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f043401b020 con 0x7f043c1396a0 2024-08-19T10:41:40.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.961+0000 7f0423fff640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f042803be00 0x7f042803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:40.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.961+0000 7f0423fff640 1 -- 172.21.15.190:0/844968698 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(102..102 src has 1..102) ==== 6150+0+0 (secure 0 0 0) 0x7f043404fc60 con 0x7f043c1396a0 2024-08-19T10:41:40.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.961+0000 7f043affd640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f042803be00 0x7f042803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:40.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.961+0000 7f043affd640 10 monclient: get_auth_request con 0x7f042803be00 auth_method 0 2024-08-19T10:41:40.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.961+0000 7f043affd640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f042803be00 0x7f042803e2c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0430009af0 tx=0x7f0430008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:40.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.965+0000 7f0423fff640 1 -- 172.21.15.190:0/844968698 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0434022030 con 0x7f043c1396a0 2024-08-19T10:41:40.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.965+0000 7f0423fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:40.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:40.965+0000 7f0423fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:41.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.125+0000 7f04418da640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7"}] 2024-08-19T10:41:41.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.125+0000 7f04418da640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-08-19T10:41:41.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.125+0000 7f04418da640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:41.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.125+0000 7f04418da640 1 -- 172.21.15.190:0/844968698 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7"} v 0) -- 0x7f041c005380 con 0x7f043c1396a0 2024-08-19T10:41:41.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.273+0000 7f0423fff640 1 -- 172.21.15.190:0/844968698 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7"}]=0 removed item id 7 name 'osd.7' from crush map v103) ==== 122+0+0 (secure 0 0 0) 0x7f0434010040 con 0x7f043c1396a0 2024-08-19T10:41:41.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.273+0000 7f0423fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-08-19T10:41:41.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.273+0000 7f0423fff640 10 monclient: _finish_command 2 = system:0 removed item id 7 name 'osd.7' from crush map 2024-08-19T10:41:41.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.373+0000 7f04418da640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.7"}] 2024-08-19T10:41:41.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.373+0000 7f04418da640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-08-19T10:41:41.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.373+0000 7f04418da640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:41.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.373+0000 7f04418da640 1 -- 172.21.15.190:0/844968698 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.7"} v 0) -- 0x7f041c002250 con 0x7f043c1396a0 2024-08-19T10:41:41.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f0423fff640 1 -- 172.21.15.190:0/844968698 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.7"}]=0 device 'osd.7' does not appear in the crush map v103) ==== 124+0+0 (secure 0 0 0) 0x7f043404e100 con 0x7f043c1396a0 2024-08-19T10:41:41.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f0423fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.7"}] 2024-08-19T10:41:41.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f0423fff640 10 monclient: _finish_command 3 = system:0 device 'osd.7' does not appear in the crush map 2024-08-19T10:41:41.380 INFO:tasks.workunit.client.0.smithi190.stderr:device 'osd.7' does not appear in the crush map 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 -- 172.21.15.190:0/844968698 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f042803be00 msgr2=0x7f042803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f042803be00 0x7f042803e2c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0430009af0 tx=0x7f0430008040 comp rx=0 tx=0).stop 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 10 monclient: shutdown 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 -- 172.21.15.190:0/844968698 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f043c1396a0 msgr2=0x7f043c09d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f043c1396a0 0x7f043c09d5c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0434009cb0 tx=0x7f043400c490 comp rx=0 tx=0).stop 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 -- 172.21.15.190:0/844968698 shutdown_connections 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f042803be00 0x7f042803e2c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043c0a6c30 0x7f043c09d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f043c1396a0 0x7f043c09d5c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 --2- 172.21.15.190:0/844968698 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f043c0a6850 0x7f043c09cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 -- 172.21.15.190:0/844968698 >> 172.21.15.190:0/844968698 conn(0x7f043c0abe50 msgr2=0x7f043c0ab300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:41.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 -- 172.21.15.190:0/844968698 shutdown_connections 2024-08-19T10:41:41.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.377+0000 7f04418da640 1 -- 172.21.15.190:0/844968698 wait complete. 2024-08-19T10:41:41.397 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm osd.9 2024-08-19T10:41:41.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient: get_monmap_and_config 2024-08-19T10:41:41.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient: build_initial_monmap 2024-08-19T10:41:41.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient: monmap: 2024-08-19T10:41:41.512 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:41.513 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:41.513 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:41.513065+0000 2024-08-19T10:41:41.513 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:41.513065+0000 2024-08-19T10:41:41.513 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:41.513 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.513 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:41.513 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:41.513 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:41.513 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:41.513 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:41.514 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:41.514 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:41.514 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:41.514 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:41.514 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:41.514 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:41.514 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 1 Processor -- start 2024-08-19T10:41:41.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 1 -- start start 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient: init 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient: _add_conns ranks=[6,8,5,2,9,0,1,10,3,7,4] 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3300139980 0x7f33000a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient(hunting): picked mon.noname-d con 0x7f3300139980 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3300139d60 0x7f33000a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient(hunting): picked mon.noname-e con 0x7f3300139d60 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330013a140 0x7f33000a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient(hunting): picked mon.noname-h con 0x7f330013a140 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f33000a8620 con 0x7f330013a140 2024-08-19T10:41:41.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f33000a5b50 con 0x7f3300139980 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f33000a45b0 con 0x7f3300139d60 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f33053f9640 10 monclient(hunting): authenticate will time out at 1147.083218s 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330013a140 0x7f33000a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330013a140 0x7f33000a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:33738/0 (socket says 172.21.15.190:33738) 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 1 -- 172.21.15.190:0/2347275441 learned_addr learned my addr 172.21.15.190:0/2347275441 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 10 monclient(hunting): get_auth_request con 0x7f330013a140 auth_method 0 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:41.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:41.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:41.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:41.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:41.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32fe7fc640 1 --2- 172.21.15.190:0/2347275441 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3300139d60 0x7f33000a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:41.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32feffd640 1 --2- 172.21.15.190:0/2347275441 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3300139980 0x7f33000a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:41.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32fe7fc640 10 monclient(hunting): get_auth_request con 0x7f3300139d60 auth_method 0 2024-08-19T10:41:41.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32fe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:41.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32fe7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:41.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32fe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:41.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32feffd640 10 monclient(hunting): get_auth_request con 0x7f3300139980 auth_method 0 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32feffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32feffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 10 monclient(hunting): handle_auth_done global_id 5351 payload 274 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 1 -- 172.21.15.190:0/2347275441 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3300139d60 msgr2=0x7f33000a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 1 --2- 172.21.15.190:0/2347275441 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3300139d60 0x7f33000a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 1 -- 172.21.15.190:0/2347275441 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3300139980 msgr2=0x7f33000a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 1 --2- 172.21.15.190:0/2347275441 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3300139980 0x7f33000a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 1 monclient: found mon.noname-h 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:41.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:41.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.509+0000 7f32ff7fe640 1 -- 172.21.15.190:0/2347275441 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33000a4890 con 0x7f330013a140 2024-08-19T10:41:41.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.513+0000 7f32ff7fe640 1 -- 172.21.15.190:0/2347275441 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330013a140 msgr2=0x7f33000a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:41.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.513+0000 7f32ff7fe640 1 --2- 172.21.15.190:0/2347275441 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330013a140 0x7f33000a6d20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009e80 tx=0x7f32f801c9a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=ca5824569816248c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:41.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 1 -- 172.21.15.190:0/2347275441 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32f80090d0 con 0x7f330013a140 2024-08-19T10:41:41.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:41.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:41.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 10 monclient: dump: 2024-08-19T10:41:41.526 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:41.526 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:41.526 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.526 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.526 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:41.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 1 -- 172.21.15.190:0/2347275441 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f80a59f0 con 0x7f330013a140 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 5 monclient: authenticate success, global_id 5351 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 1 -- 172.21.15.190:0/2347275441 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32f80a5ba0 con 0x7f330013a140 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 4 set_mon_vals no callback set 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fdffb640 10 monclient: dump: 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:41.528 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:41.529 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient: shutdown 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 -- 172.21.15.190:0/2347275441 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330013a140 msgr2=0x7f33000a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 --2- 172.21.15.190:0/2347275441 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330013a140 0x7f33000a6d20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009e80 tx=0x7f32f801c9a0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 -- 172.21.15.190:0/2347275441 shutdown_connections 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 --2- 172.21.15.190:0/2347275441 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330013a140 0x7f33000a6d20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 --2- 172.21.15.190:0/2347275441 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3300139d60 0x7f33000a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 --2- 172.21.15.190:0/2347275441 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3300139980 0x7f33000a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 -- 172.21.15.190:0/2347275441 >> 172.21.15.190:0/2347275441 conn(0x7f33000abde0 msgr2=0x7f33000ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 -- 172.21.15.190:0/2347275441 shutdown_connections 2024-08-19T10:41:41.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 -- 172.21.15.190:0/2347275441 wait complete. 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient: build_initial_monmap 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient: monmap: 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:41.531 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 Processor -- start 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 -- start start 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient: init 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient: _add_conns ranks=[3,1,0,2,5,6,4,7,8] 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300139980 0x7f33001347e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient(hunting): picked mon.noname-g con 0x7f3300139980 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:41.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300139d60 0x7f3300134d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient(hunting): picked mon.noname-f con 0x7f3300139d60 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f330013a140 0x7f330013a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient(hunting): picked mon.noname-a con 0x7f330013a140 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3300143190 con 0x7f330013a140 2024-08-19T10:41:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3300143010 con 0x7f3300139d60 2024-08-19T10:41:41.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3300143310 con 0x7f3300139980 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f33053f9640 10 monclient(hunting): authenticate will time out at 1147.090307s 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32feffd640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300139980 msgr2=0x7f33001347e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32feffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300139980 0x7f33001347e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300139d60 0x7f3300134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300139d60 0x7f3300134d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51246/0 (socket says 172.21.15.190:51246) 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 1 -- 172.21.15.190:0/2885491687 learned_addr learned my addr 172.21.15.190:0/2885491687 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 10 monclient(hunting): get_auth_request con 0x7f3300139d60 auth_method 0 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:41.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32ff7fe640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f330013a140 0x7f330013a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32ff7fe640 10 monclient(hunting): get_auth_request con 0x7f330013a140 auth_method 0 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32ff7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 10 monclient(hunting): handle_auth_done global_id 14608 payload 824 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 1 -- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300139980 msgr2=0x7f33001347e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300139980 0x7f33001347e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 1 -- 172.21.15.190:0/2885491687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f330013a140 msgr2=0x7f330013a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f330013a140 0x7f330013a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 1 monclient: found mon.noname-f 2024-08-19T10:41:41.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.517+0000 7f32fe7fc640 1 -- 172.21.15.190:0/2885491687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330013dbe0 con 0x7f3300139d60 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32fe7fc640 1 -- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300139d60 msgr2=0x7f3300134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32fe7fc640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300139d60 0x7f3300134d20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d830 tx=0x7f32f000dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 1 -- 172.21.15.190:0/2885491687 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32f000bd40 con 0x7f3300139d60 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 10 monclient: dump: 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.536 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 1 -- 172.21.15.190:0/2885491687 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32f0019070 con 0x7f3300139d60 2024-08-19T10:41:41.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f33053f9640 5 monclient: authenticate success, global_id 14608 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 1 -- 172.21.15.190:0/2885491687 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32f00148a0 con 0x7f3300139d60 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f33053f9640 10 monclient: _renew_subs 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f33053f9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f33053f9640 1 -- 172.21.15.190:0/2885491687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330013bdc0 con 0x7f3300139d60 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32fd7fa640 4 set_mon_vals no callback set 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 10 monclient: dump: 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.538 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:41.539 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f33053f9640 10 monclient: _renew_subs 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f33053f9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f33053f9640 1 -- 172.21.15.190:0/2885491687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33000a2540 con 0x7f3300139d60 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f33053f9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f33053f9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f33053f9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 1 -- 172.21.15.190:0/2885491687 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f32f0014430 con 0x7f3300139d60 2024-08-19T10:41:41.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f33053f9640 1 -- 172.21.15.190:0/2885491687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33000aacf0 con 0x7f3300139d60 2024-08-19T10:41:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32e003be00 0x7f32e003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32f77fe640 1 -- 172.21.15.190:0/2885491687 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6100+0+0 (secure 0 0 0) 0x7f32f000f040 con 0x7f3300139d60 2024-08-19T10:41:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32feffd640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32e003be00 0x7f32e003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.521+0000 7f32feffd640 10 monclient: get_auth_request con 0x7f32e003be00 auth_method 0 2024-08-19T10:41:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.525+0000 7f32feffd640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32e003be00 0x7f32e003e2c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000f00 tx=0x7f32e8006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:41.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.525+0000 7f32f77fe640 1 -- 172.21.15.190:0/2885491687 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f32f004db30 con 0x7f3300139d60 2024-08-19T10:41:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.525+0000 7f32f77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:41.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.525+0000 7f32f77fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:41.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.701+0000 7f33053f9640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9"}] 2024-08-19T10:41:41.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.701+0000 7f33053f9640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-08-19T10:41:41.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.701+0000 7f33053f9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:41.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.701+0000 7f33053f9640 1 -- 172.21.15.190:0/2885491687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9"} v 0) -- 0x7f330006c8d0 con 0x7f3300139d60 2024-08-19T10:41:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.713+0000 7f32f77fe640 1 -- 172.21.15.190:0/2885491687 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9"}]=0 device 'osd.9' does not appear in the crush map v103) ==== 124+0+0 (secure 0 0 0) 0x7f32f001dce0 con 0x7f3300139d60 2024-08-19T10:41:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.713+0000 7f32f77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-08-19T10:41:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.713+0000 7f32f77fe640 10 monclient: _finish_command 2 = system:0 device 'osd.9' does not appear in the crush map 2024-08-19T10:41:41.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.817+0000 7f33053f9640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.9"}] 2024-08-19T10:41:41.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.817+0000 7f33053f9640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-08-19T10:41:41.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.817+0000 7f33053f9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:41.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.817+0000 7f33053f9640 1 -- 172.21.15.190:0/2885491687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.9"} v 0) -- 0x7f33000a5610 con 0x7f3300139d60 2024-08-19T10:41:41.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.817+0000 7f32f77fe640 1 -- 172.21.15.190:0/2885491687 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.9"}]=0 device 'osd.9' does not appear in the crush map v103) ==== 124+0+0 (secure 0 0 0) 0x7f32f0014a40 con 0x7f3300139d60 2024-08-19T10:41:41.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.817+0000 7f32f77fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.9"}] 2024-08-19T10:41:41.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.817+0000 7f32f77fe640 10 monclient: _finish_command 3 = system:0 device 'osd.9' does not appear in the crush map 2024-08-19T10:41:41.823 INFO:tasks.workunit.client.0.smithi190.stderr:device 'osd.9' does not appear in the crush map 2024-08-19T10:41:41.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 -- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32e003be00 msgr2=0x7f32e003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:41.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32e003be00 0x7f32e003e2c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000f00 tx=0x7f32e8006a30 comp rx=0 tx=0).stop 2024-08-19T10:41:41.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 10 monclient: shutdown 2024-08-19T10:41:41.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:41.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 -- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300139d60 msgr2=0x7f3300134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:41.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300139d60 0x7f3300134d20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d830 tx=0x7f32f000dcf0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 -- 172.21.15.190:0/2885491687 shutdown_connections 2024-08-19T10:41:41.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32e003be00 0x7f32e003e2c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f330013a140 0x7f330013a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3300139d60 0x7f3300134d20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 --2- 172.21.15.190:0/2885491687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3300139980 0x7f33001347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 -- 172.21.15.190:0/2885491687 >> 172.21.15.190:0/2885491687 conn(0x7f33000abde0 msgr2=0x7f3300135c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:41.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 -- 172.21.15.190:0/2885491687 shutdown_connections 2024-08-19T10:41:41.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.821+0000 7f33053f9640 1 -- 172.21.15.190:0/2885491687 wait complete. 2024-08-19T10:41:41.840 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm foo 2024-08-19T10:41:41.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient: get_monmap_and_config 2024-08-19T10:41:41.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient: build_initial_monmap 2024-08-19T10:41:41.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient: monmap: 2024-08-19T10:41:41.949 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:41.949 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:41.949 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:41.949077+0000 2024-08-19T10:41:41.949 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:41.949077+0000 2024-08-19T10:41:41.949 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:41.949 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.949 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:41.949 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:41.949 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:41.950 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:41.950 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:41.950 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:41.950 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:41.950 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:41.950 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:41.950 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:41.950 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:41.950 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 1 Processor -- start 2024-08-19T10:41:41.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 1 -- start start 2024-08-19T10:41:41.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient: init 2024-08-19T10:41:41.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:41.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient: _add_conns ranks=[0,8,3,4,5,9,1,2,10,7,6] 2024-08-19T10:41:41.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc9e0139980 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:41.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9e00a5730 0x7fc9e00a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient(hunting): picked mon.noname-e con 0x7fc9e00a5730 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:41.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e013e8c0 0x7fc9e0140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient(hunting): picked mon.noname-g con 0x7fc9e013e8c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:41.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e00a8470 con 0x7fc9e0139980 2024-08-19T10:41:41.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:41.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e009ca60 con 0x7fc9e013e8c0 2024-08-19T10:41:41.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:41.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc9e00a3640 con 0x7fc9e00a5730 2024-08-19T10:41:41.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:41.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9e5c4e640 10 monclient(hunting): authenticate will time out at 1147.519016s 2024-08-19T10:41:41.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9dffff640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e013e8c0 msgr2=0x7fc9e0140cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:41.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9dffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e013e8c0 0x7fc9e0140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:41.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9e00a5730 0x7fc9e00a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:41.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9df7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc9e0139980 0x7fc9e0139d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:54846/0 (socket says 172.21.15.190:54846) 2024-08-19T10:41:41.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9df7fe640 1 -- 172.21.15.190:0/4258338276 learned_addr learned my addr 172.21.15.190:0/4258338276 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:41.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 10 monclient(hunting): get_auth_request con 0x7fc9e00a5730 auth_method 0 2024-08-19T10:41:41.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:41.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:41.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:41.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:41.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:41.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:41.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 10 monclient(hunting): handle_auth_done global_id 5201 payload 274 2024-08-19T10:41:41.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 1 -- 172.21.15.190:0/4258338276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e013e8c0 msgr2=0x7fc9e0140cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:41.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 1 --2- 172.21.15.190:0/4258338276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e013e8c0 0x7fc9e0140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 1 -- 172.21.15.190:0/4258338276 >> v1:172.21.15.89:6789/0 conn(0x7fc9e0139980 legacy=0x7fc9e0139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:41.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:41.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 1 monclient: found mon.noname-e 2024-08-19T10:41:41.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:41.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:41.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.945+0000 7fc9deffd640 1 -- 172.21.15.190:0/4258338276 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e00a3850 con 0x7fc9e00a5730 2024-08-19T10:41:41.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9deffd640 1 -- 172.21.15.190:0/4258338276 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9e00a5730 msgr2=0x7fc9e00a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:41.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9deffd640 1 --2- 172.21.15.190:0/4258338276 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9e00a5730 0x7fc9e00a5b10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00097f0 tx=0x7fc9d001c3c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c9b2c280c7375a06 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:41.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 1 -- 172.21.15.190:0/4258338276 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9d00b0070 con 0x7fc9e00a5730 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 10 monclient: dump: 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:41.964 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 1 -- 172.21.15.190:0/4258338276 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d00aa810 con 0x7fc9e00a5730 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 5 monclient: authenticate success, global_id 5201 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:41.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:41.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:41.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 4 set_mon_vals no callback set 2024-08-19T10:41:41.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 1 -- 172.21.15.190:0/4258338276 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9d00aae30 con 0x7fc9e00a5730 2024-08-19T10:41:41.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:41.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:41.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9de7fc640 10 monclient: dump: 2024-08-19T10:41:41.966 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:41.966 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:41.966 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.967 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.967 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:41.967 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.967 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:41.967 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:41.967 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:41.967 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:41.967 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:41.967 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient: shutdown 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/4258338276 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9e00a5730 msgr2=0x7fc9e00a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 --2- 172.21.15.190:0/4258338276 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9e00a5730 0x7fc9e00a5b10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00097f0 tx=0x7fc9d001c3c0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/4258338276 shutdown_connections 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 --2- 172.21.15.190:0/4258338276 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9e00a5730 0x7fc9e00a5b10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 --2- 172.21.15.190:0/4258338276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e013e8c0 0x7fc9e0140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/4258338276 >> 172.21.15.190:0/4258338276 conn(0x7fc9e00abde0 msgr2=0x7fc9e00ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:41.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/4258338276 shutdown_connections 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/4258338276 wait complete. 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient: build_initial_monmap 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient: monmap: 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:41.969 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:41.970 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:41.970 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:41.970 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:41.970 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:41.970 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:41.970 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 Processor -- start 2024-08-19T10:41:41.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 -- start start 2024-08-19T10:41:41.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient: init 2024-08-19T10:41:41.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:41.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient: _add_conns ranks=[5,3,2,4,1,8,6,0,7] 2024-08-19T10:41:41.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9e00a5730 0x7fc9e0133b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient(hunting): picked mon.noname-h con 0x7fc9e00a5730 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:41.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e0139980 0x7fc9e0134060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient(hunting): picked mon.noname-g con 0x7fc9e0139980 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:41.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9e013e8c0 0x7fc9e01367f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient(hunting): picked mon.noname-b con 0x7fc9e013e8c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:41.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:41.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e0143b90 con 0x7fc9e013e8c0 2024-08-19T10:41:41.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:41.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e0143a10 con 0x7fc9e0139980 2024-08-19T10:41:41.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:41.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc9e0143d10 con 0x7fc9e00a5730 2024-08-19T10:41:41.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:41.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9e5c4e640 10 monclient(hunting): authenticate will time out at 1147.527631s 2024-08-19T10:41:41.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9deffd640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e0139980 msgr2=0x7fc9e0134060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:41.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e0139980 0x7fc9e0134060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:41.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9e00a5730 0x7fc9e0133b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:41.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9e00a5730 0x7fc9e0133b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:33744/0 (socket says 172.21.15.190:33744) 2024-08-19T10:41:41.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9df7fe640 1 -- 172.21.15.190:0/600244927 learned_addr learned my addr 172.21.15.190:0/600244927 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9df7fe640 10 monclient(hunting): get_auth_request con 0x7fc9e00a5730 auth_method 0 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9df7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.953+0000 7fc9df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 10 monclient(hunting): handle_auth_done global_id 5360 payload 824 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 1 -- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e0139980 msgr2=0x7fc9e0134060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e0139980 0x7fc9e0134060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 1 -- 172.21.15.190:0/600244927 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9e013e8c0 msgr2=0x7fc9e01367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9e013e8c0 0x7fc9e01367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:41.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 1 monclient: found mon.noname-h 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 1 -- 172.21.15.190:0/600244927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e0138220 con 0x7fc9e00a5730 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 1 -- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9e00a5730 msgr2=0x7fc9e0133b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9e00a5730 0x7fc9e0133b20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800c4a0 tx=0x7fc9c800c960 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 1 -- 172.21.15.190:0/600244927 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9c8017070 con 0x7fc9e00a5730 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9df7fe640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9e00a5730 0x7fc9e0133b20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800c4a0 tx=0x7fc9c800c960 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fc9c80074e0 mon_map magic: 0 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 10 monclient: dump: 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:41.974 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:41.975 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 1 -- 172.21.15.190:0/600244927 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c8005d60 con 0x7fc9e00a5730 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9e5c4e640 5 monclient: authenticate success, global_id 5360 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9e5c4e640 10 monclient: _renew_subs 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9e5c4e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/600244927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e009e8b0 con 0x7fc9e00a5730 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9dd7fa640 4 set_mon_vals no callback set 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9e5c4e640 10 monclient: _renew_subs 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9e5c4e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/600244927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e009d280 con 0x7fc9e00a5730 2024-08-19T10:41:41.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9e5c4e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:41.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9e5c4e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:41.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9e5c4e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:41.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/600244927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e00a5b90 con 0x7fc9e00a5730 2024-08-19T10:41:41.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 1 -- 172.21.15.190:0/600244927 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9c80074e0 con 0x7fc9e00a5730 2024-08-19T10:41:41.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:41.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 10 monclient: dump: 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:41.978 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:41.979 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:41.979 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:41.979 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:41.979 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:41.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 1 -- 172.21.15.190:0/600244927 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc9c8007070 con 0x7fc9e00a5730 2024-08-19T10:41:41.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.957+0000 7fc9d7fff640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc9c003be50 0x7fc9c003e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:41.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.961+0000 7fc9deffd640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc9c003be50 0x7fc9c003e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:41.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.961+0000 7fc9deffd640 10 monclient: get_auth_request con 0x7fc9c003be50 auth_method 0 2024-08-19T10:41:41.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.961+0000 7fc9deffd640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc9c003be50 0x7fc9c003e310 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00097f0 tx=0x7fc9d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:41.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:41.965+0000 7fc9d7fff640 1 -- 172.21.15.190:0/600244927 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(103..103 src has 1..103) ==== 6100+0+0 (secure 0 0 0) 0x7fc9c804e990 con 0x7fc9e00a5730 2024-08-19T10:41:42.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:42.581+0000 7fc9d7fff640 1 -- 172.21.15.190:0/600244927 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc9c8018c60 con 0x7fc9e00a5730 2024-08-19T10:41:42.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:42.581+0000 7fc9d7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:42.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:42.581+0000 7fc9d7fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:42.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:42.757+0000 7fc9e5c4e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:42.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:42.757+0000 7fc9e5c4e640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:42.757+0000 7fc9e5c4e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:42.757+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/600244927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7fc9e009f220 con 0x7fc9e00a5730 2024-08-19T10:41:43.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.289+0000 7fc9d7fff640 1 -- 172.21.15.190:0/600244927 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 removed item id -10 name 'foo' from crush map v105) ==== 120+0+0 (secure 0 0 0) 0x7fc9c804ce60 con 0x7fc9e00a5730 2024-08-19T10:41:43.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.289+0000 7fc9d7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:43.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.289+0000 7fc9d7fff640 10 monclient: _finish_command 2 = system:0 removed item id -10 name 'foo' from crush map 2024-08-19T10:41:43.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.393+0000 7fc9e5c4e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:43.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.393+0000 7fc9e5c4e640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:43.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.393+0000 7fc9e5c4e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:43.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.393+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/600244927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "foo"} v 0) -- 0x7fc9e00ac5c0 con 0x7fc9e00a5730 2024-08-19T10:41:43.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9d7fff640 1 -- 172.21.15.190:0/600244927 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "foo"}]=0 device 'foo' does not appear in the crush map v105) ==== 120+0+0 (secure 0 0 0) 0x7fc9c80189c0 con 0x7fc9e00a5730 2024-08-19T10:41:43.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9d7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "foo"}] 2024-08-19T10:41:43.400 INFO:tasks.workunit.client.0.smithi190.stderr:device 'foo' does not appear in the crush map 2024-08-19T10:41:43.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9d7fff640 10 monclient: _finish_command 3 = system:0 device 'foo' does not appear in the crush map 2024-08-19T10:41:43.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc9c003be50 msgr2=0x7fc9c003e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:43.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc9c003be50 0x7fc9c003e310 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00097f0 tx=0x7fc9d0008040 comp rx=0 tx=0).stop 2024-08-19T10:41:43.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 10 monclient: shutdown 2024-08-19T10:41:43.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:43.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9e00a5730 msgr2=0x7fc9e0133b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:43.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9e00a5730 0x7fc9e0133b20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800c4a0 tx=0x7fc9c800c960 comp rx=0 tx=0).stop 2024-08-19T10:41:43.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/600244927 shutdown_connections 2024-08-19T10:41:43.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc9c003be50 0x7fc9c003e310 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:43.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc9e013e8c0 0x7fc9e01367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:43.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e0139980 0x7fc9e0134060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:43.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 --2- 172.21.15.190:0/600244927 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc9e00a5730 0x7fc9e0133b20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:43.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/600244927 >> 172.21.15.190:0/600244927 conn(0x7fc9e00abde0 msgr2=0x7fc9e00aa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:43.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/600244927 shutdown_connections 2024-08-19T10:41:43.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.397+0000 7fc9e5c4e640 1 -- 172.21.15.190:0/600244927 wait complete. 2024-08-19T10:41:43.417 INFO:tasks.workunit.client.0.smithi190.stderr:++ ceph osd create 2024-08-19T10:41:43.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient: get_monmap_and_config 2024-08-19T10:41:43.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient: build_initial_monmap 2024-08-19T10:41:43.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient: monmap: 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:43.533974+0000 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:43.533974+0000 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:43.534 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:43.535 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:43.535 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:43.535 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:43.535 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:43.535 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:43.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 1 Processor -- start 2024-08-19T10:41:43.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 1 -- start start 2024-08-19T10:41:43.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient: init 2024-08-19T10:41:43.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:43.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient: _add_conns ranks=[2,4,5,3,7,6,9,1,8,0,10] 2024-08-19T10:41:43.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8c1399f0 0x7f9b8c0a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient(hunting): picked mon.noname-b con 0x7f9b8c1399f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8c139dd0 0x7f9b8c0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient(hunting): picked mon.noname-c con 0x7f9b8c139dd0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c13a1b0 0x7f9b8c0a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient(hunting): picked mon.noname-h con 0x7f9b8c13a1b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c0a4eb0 con 0x7f9b8c1399f0 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9b8c0a5b80 con 0x7f9b8c139dd0 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f9b8c0a8340 con 0x7f9b8c13a1b0 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:43.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.529+0000 7f9b93457640 10 monclient(hunting): authenticate will time out at 1149.103858s 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c13a1b0 0x7f9b8c0a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c13a1b0 0x7f9b8c0a6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:33754/0 (socket says 172.21.15.190:33754) 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 1 -- 172.21.15.190:0/2022778596 learned_addr learned my addr 172.21.15.190:0/2022778596 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 10 monclient(hunting): get_auth_request con 0x7f9b8c13a1b0 auth_method 0 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b909f2640 1 --2- 172.21.15.190:0/2022778596 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8c139dd0 0x7f9b8c0a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b911f3640 1 --2- 172.21.15.190:0/2022778596 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8c1399f0 0x7f9b8c0a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:43.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 10 monclient(hunting): handle_auth_done global_id 5369 payload 274 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 1 -- 172.21.15.190:0/2022778596 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8c139dd0 msgr2=0x7f9b8c0a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 1 --2- 172.21.15.190:0/2022778596 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8c139dd0 0x7f9b8c0a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 1 -- 172.21.15.190:0/2022778596 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8c1399f0 msgr2=0x7f9b8c0a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 1 --2- 172.21.15.190:0/2022778596 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8c1399f0 0x7f9b8c0a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 10 monclient: _finish_hunting 0 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 1 monclient: found mon.noname-h 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b919f4640 1 -- 172.21.15.190:0/2022778596 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c0a4580 con 0x7f9b8c13a1b0 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b909f2640 10 monclient: get_auth_request con 0x7f9b8c139dd0 auth_method 0 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b909f2640 1 --2- 172.21.15.190:0/2022778596 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8c139dd0 0x7f9b8c0a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b911f3640 10 monclient: get_auth_request con 0x7f9b8c1399f0 auth_method 0 2024-08-19T10:41:43.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.533+0000 7f9b911f3640 1 --2- 172.21.15.190:0/2022778596 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8c1399f0 0x7f9b8c0a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b919f4640 1 -- 172.21.15.190:0/2022778596 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c13a1b0 msgr2=0x7f9b8c0a6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b919f4640 1 --2- 172.21.15.190:0/2022778596 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c13a1b0 0x7f9b8c0a6e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9b880098a0 tx=0x7f9b8801c440 comp rx=0 tx=0).ready entity=mon.5 client_cookie=1bb18385579db326 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 1 -- 172.21.15.190:0/2022778596 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b880b0070 con 0x7f9b8c13a1b0 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 10 monclient: dump: 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:43.544 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:43.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 1 -- 172.21.15.190:0/2022778596 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b880aa810 con 0x7f9b8c13a1b0 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 1 -- 172.21.15.190:0/2022778596 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b880aae30 con 0x7f9b8c13a1b0 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 5 monclient: authenticate success, global_id 5369 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 4 set_mon_vals no callback set 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b83fff640 10 monclient: dump: 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:43.546 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:43.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient: shutdown 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 -- 172.21.15.190:0/2022778596 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c13a1b0 msgr2=0x7f9b8c0a6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 --2- 172.21.15.190:0/2022778596 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c13a1b0 0x7f9b8c0a6e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9b880098a0 tx=0x7f9b8801c440 comp rx=0 tx=0).stop 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 -- 172.21.15.190:0/2022778596 shutdown_connections 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 --2- 172.21.15.190:0/2022778596 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c13a1b0 0x7f9b8c0a6e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 --2- 172.21.15.190:0/2022778596 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b8c139dd0 0x7f9b8c0a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 --2- 172.21.15.190:0/2022778596 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b8c1399f0 0x7f9b8c0a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 -- 172.21.15.190:0/2022778596 >> 172.21.15.190:0/2022778596 conn(0x7f9b8c074c10 msgr2=0x7f9b8c0abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 -- 172.21.15.190:0/2022778596 shutdown_connections 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 -- 172.21.15.190:0/2022778596 wait complete. 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient: build_initial_monmap 2024-08-19T10:41:43.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient: monmap: 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:43.549 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 Processor -- start 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 -- start start 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient: init 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient: _add_conns ranks=[1,5,3,7,8,2,0,6,4] 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b8c1399f0 0x7f9b8c133640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient(hunting): picked mon.noname-f con 0x7f9b8c1399f0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c139dd0 0x7f9b8c133b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient(hunting): picked mon.noname-h con 0x7f9b8c139dd0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c13a1b0 0x7f9b8c1340c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient(hunting): picked mon.noname-g con 0x7f9b8c13a1b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:43.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c143290 con 0x7f9b8c1399f0 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c143110 con 0x7f9b8c13a1b0 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f9b8c143410 con 0x7f9b8c139dd0 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b93457640 10 monclient(hunting): authenticate will time out at 1149.111102s 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b919f4640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c13a1b0 msgr2=0x7f9b8c1340c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b919f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c13a1b0 0x7f9b8c1340c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c139dd0 0x7f9b8c133b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c139dd0 0x7f9b8c133b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:33758/0 (socket says 172.21.15.190:33758) 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 1 -- 172.21.15.190:0/437536324 learned_addr learned my addr 172.21.15.190:0/437536324 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 10 monclient(hunting): get_auth_request con 0x7f9b8c139dd0 auth_method 0 2024-08-19T10:41:43.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:43.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:43.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:43.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b911f3640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b8c1399f0 0x7f9b8c133640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:43.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:43.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:43.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:43.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b911f3640 10 monclient(hunting): get_auth_request con 0x7f9b8c1399f0 auth_method 0 2024-08-19T10:41:43.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b911f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b911f3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b911f3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b911f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b911f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b911f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 10 monclient(hunting): handle_auth_done global_id 5378 payload 824 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 1 -- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c13a1b0 msgr2=0x7f9b8c1340c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c13a1b0 0x7f9b8c1340c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 1 -- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b8c1399f0 msgr2=0x7f9b8c133640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b8c1399f0 0x7f9b8c133640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 10 monclient: _finish_hunting 0 2024-08-19T10:41:43.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 1 monclient: found mon.noname-h 2024-08-19T10:41:43.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:43.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:43.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.537+0000 7f9b909f2640 1 -- 172.21.15.190:0/437536324 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c0ab460 con 0x7f9b8c139dd0 2024-08-19T10:41:43.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b909f2640 1 -- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c139dd0 msgr2=0x7f9b8c133b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:43.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b909f2640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c139dd0 0x7f9b8c133b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c820 tx=0x7f9b7c00cce0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:43.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 1 -- 172.21.15.190:0/437536324 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b7c0113f0 con 0x7f9b8c139dd0 2024-08-19T10:41:43.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:43.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:43.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 10 monclient: dump: 2024-08-19T10:41:43.557 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:43.557 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:43.558 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 1 -- 172.21.15.190:0/437536324 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c0115d0 con 0x7f9b8c139dd0 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b93457640 5 monclient: authenticate success, global_id 5378 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b93457640 10 monclient: _renew_subs 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 1 -- 172.21.15.190:0/437536324 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b7c011bf0 con 0x7f9b8c139dd0 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b93457640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b93457640 1 -- 172.21.15.190:0/437536324 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c0a1740 con 0x7f9b8c139dd0 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b82ffd640 4 set_mon_vals no callback set 2024-08-19T10:41:43.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 10 monclient: dump: 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:43.560 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b93457640 10 monclient: _renew_subs 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b93457640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b93457640 1 -- 172.21.15.190:0/437536324 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c13c7f0 con 0x7f9b8c139dd0 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b93457640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b93457640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b93457640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b93457640 1 -- 172.21.15.190:0/437536324 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8c0a5640 con 0x7f9b8c139dd0 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 1 -- 172.21.15.190:0/437536324 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9b7c011780 con 0x7f9b8c139dd0 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9b7003be00 0x7f9b7003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:43.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b81ffb640 1 -- 172.21.15.190:0/437536324 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(105..105 src has 1..105) ==== 5802+0+0 (secure 0 0 0) 0x7f9b7c04f080 con 0x7f9b8c139dd0 2024-08-19T10:41:43.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b911f3640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9b7003be00 0x7f9b7003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:43.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.541+0000 7f9b911f3640 10 monclient: get_auth_request con 0x7f9b7003be00 auth_method 0 2024-08-19T10:41:43.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.545+0000 7f9b911f3640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9b7003be00 0x7f9b7003e2c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9b84007130 tx=0x7f9b840077c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:43.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.545+0000 7f9b81ffb640 1 -- 172.21.15.190:0/437536324 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9b7c015020 con 0x7f9b8c139dd0 2024-08-19T10:41:43.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.545+0000 7f9b81ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:43.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.545+0000 7f9b81ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.725+0000 7f9b93457640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-08-19T10:41:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.725+0000 7f9b93457640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-08-19T10:41:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.725+0000 7f9b93457640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:43.725+0000 7f9b93457640 1 -- 172.21.15.190:0/437536324 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f9b8c0a3500 con 0x7f9b8c139dd0 2024-08-19T10:41:44.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:44.293+0000 7f9b81ffb640 1 -- 172.21.15.190:0/437536324 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v106) ==== 58+0+3 (secure 0 0 0) 0x7f9b7c011e10 con 0x7f9b8c139dd0 2024-08-19T10:41:44.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:44.293+0000 7f9b81ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-08-19T10:41:44.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:44.293+0000 7f9b81ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:44.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:44.401+0000 7f9b93457640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-08-19T10:41:44.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:44.401+0000 7f9b93457640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-08-19T10:41:44.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:44.401+0000 7f9b93457640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:44.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:44.401+0000 7f9b93457640 1 -- 172.21.15.190:0/437536324 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f9b8c0008e0 con 0x7f9b8c139dd0 2024-08-19T10:41:45.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b81ffb640 1 -- 172.21.15.190:0/437536324 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v107) ==== 58+0+3 (secure 0 0 0) 0x7f9b7c018d60 con 0x7f9b8c139dd0 2024-08-19T10:41:45.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b81ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-08-19T10:41:45.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b81ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:45.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 -- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9b7003be00 msgr2=0x7f9b7003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:45.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9b7003be00 0x7f9b7003e2c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9b84007130 tx=0x7f9b840077c0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 10 monclient: shutdown 2024-08-19T10:41:45.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:45.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 -- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c139dd0 msgr2=0x7f9b8c133b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:45.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c139dd0 0x7f9b8c133b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c820 tx=0x7f9b7c00cce0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 -- 172.21.15.190:0/437536324 shutdown_connections 2024-08-19T10:41:45.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9b7003be00 0x7f9b7003e2c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c13a1b0 0x7f9b8c1340c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b8c139dd0 0x7f9b8c133b80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 --2- 172.21.15.190:0/437536324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b8c1399f0 0x7f9b8c133640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 -- 172.21.15.190:0/437536324 >> 172.21.15.190:0/437536324 conn(0x7f9b8c074c10 msgr2=0x7f9b8c0abbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:45.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 -- 172.21.15.190:0/437536324 shutdown_connections 2024-08-19T10:41:45.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.253+0000 7f9b93457640 1 -- 172.21.15.190:0/437536324 wait complete. 2024-08-19T10:41:45.276 INFO:tasks.workunit.client.0.smithi190.stderr:+ o3=11 2024-08-19T10:41:45.276 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add 11 123 root=default 2024-08-19T10:41:45.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient: get_monmap_and_config 2024-08-19T10:41:45.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient: build_initial_monmap 2024-08-19T10:41:45.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient: monmap: 2024-08-19T10:41:45.400 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:45.400 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:45.400 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:45.400315+0000 2024-08-19T10:41:45.400 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:45.400315+0000 2024-08-19T10:41:45.400 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:45.400 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:45.400 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:45.401 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:45.401 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:45.401 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:45.401 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:45.401 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:45.401 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:45.401 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:45.401 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:45.401 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:45.401 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:45.401 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:45.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 1 Processor -- start 2024-08-19T10:41:45.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 1 -- start start 2024-08-19T10:41:45.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient: init 2024-08-19T10:41:45.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:45.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient: _add_conns ranks=[10,6,8,5,3,7,1,9,0,4,2] 2024-08-19T10:41:45.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd07809b510 0x7fd07809b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:45.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient(hunting): picked mon.noname-f con 0x7fd07809b510 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:45.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd07809ea30 0x7fd0780a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:45.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient(hunting): picked mon.noname-d con 0x7fd07809ea30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:45.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ee30 0x7fd0780a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:45.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient(hunting): picked mon.noname-e con 0x7fd07809ee30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:45.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:45.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd07809db00 con 0x7fd07809b510 2024-08-19T10:41:45.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:45.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd07809dde0 con 0x7fd07809ea30 2024-08-19T10:41:45.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:45.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd07809e0c0 con 0x7fd07809ee30 2024-08-19T10:41:45.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:45.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07ede3640 10 monclient(hunting): authenticate will time out at 1150.971111s 2024-08-19T10:41:45.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd07809b510 0x7fd07809b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:45.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd07809b510 0x7fd07809b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51256/0 (socket says 172.21.15.190:51256) 2024-08-19T10:41:45.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 1 -- 172.21.15.190:0/3119883913 learned_addr learned my addr 172.21.15.190:0/3119883913 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:45.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 10 monclient(hunting): get_auth_request con 0x7fd07809b510 auth_method 0 2024-08-19T10:41:45.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:45.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:45.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:45.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:45.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:45.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:45.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd077fff640 1 --2- 172.21.15.190:0/3119883913 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd07809ea30 0x7fd0780a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:45.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd077fff640 10 monclient(hunting): get_auth_request con 0x7fd07809ea30 auth_method 0 2024-08-19T10:41:45.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd077fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:45.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd077fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:45.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd077fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:45.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07d380640 1 --2- 172.21.15.190:0/3119883913 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ee30 0x7fd0780a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:45.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 10 monclient(hunting): handle_auth_done global_id 14626 payload 274 2024-08-19T10:41:45.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 1 -- 172.21.15.190:0/3119883913 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ee30 msgr2=0x7fd0780a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:45.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 1 --2- 172.21.15.190:0/3119883913 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ee30 0x7fd0780a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 1 -- 172.21.15.190:0/3119883913 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd07809ea30 msgr2=0x7fd0780a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:45.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 1 --2- 172.21.15.190:0/3119883913 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd07809ea30 0x7fd0780a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 10 monclient: _finish_hunting 0 2024-08-19T10:41:45.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 1 monclient: found mon.noname-f 2024-08-19T10:41:45.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:45.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:45.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.397+0000 7fd07cb7f640 1 -- 172.21.15.190:0/3119883913 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0780a5c50 con 0x7fd07809b510 2024-08-19T10:41:45.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.401+0000 7fd07cb7f640 1 -- 172.21.15.190:0/3119883913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd07809b510 msgr2=0x7fd07809b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:45.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.401+0000 7fd07cb7f640 1 --2- 172.21.15.190:0/3119883913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd07809b510 0x7fd07809b8f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009870 tx=0x7fd06c01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b60437674594c250 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:45.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 1 -- 172.21.15.190:0/3119883913 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd06c0b0070 con 0x7fd07809b510 2024-08-19T10:41:45.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 10 monclient: dump: 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:45.413 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:45.414 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:45.414 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:45.414 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:45.414 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:45.414 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:45.414 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:45.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:45.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:45.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:45.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 5 monclient: authenticate success, global_id 14626 2024-08-19T10:41:45.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 1 -- 172.21.15.190:0/3119883913 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06c0aa810 con 0x7fd07809b510 2024-08-19T10:41:45.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:45.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:45.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 4 set_mon_vals no callback set 2024-08-19T10:41:45.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 1 -- 172.21.15.190:0/3119883913 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd06c0aae30 con 0x7fd07809b510 2024-08-19T10:41:45.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:45.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:45.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd0777fe640 10 monclient: dump: 2024-08-19T10:41:45.415 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:45.415 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:45.415 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:45.415 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:45.416 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:45.416 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:45.416 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:45.416 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:45.416 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:45.416 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:45.416 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:45.416 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:45.416 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:45.416 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:45.416 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:45.417 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:45.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 10 monclient: shutdown 2024-08-19T10:41:45.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:45.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 1 -- 172.21.15.190:0/3119883913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd07809b510 msgr2=0x7fd07809b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:45.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 1 --2- 172.21.15.190:0/3119883913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd07809b510 0x7fd07809b8f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009870 tx=0x7fd06c01c420 comp rx=0 tx=0).stop 2024-08-19T10:41:45.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 1 -- 172.21.15.190:0/3119883913 shutdown_connections 2024-08-19T10:41:45.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 1 --2- 172.21.15.190:0/3119883913 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ee30 0x7fd0780a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 1 --2- 172.21.15.190:0/3119883913 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd07809ea30 0x7fd0780a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 1 --2- 172.21.15.190:0/3119883913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd07809b510 0x7fd07809b8f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 1 -- 172.21.15.190:0/3119883913 >> 172.21.15.190:0/3119883913 conn(0x7fd0780a4e70 msgr2=0x7fd0780a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 1 -- 172.21.15.190:0/3119883913 shutdown_connections 2024-08-19T10:41:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 1 -- 172.21.15.190:0/3119883913 wait complete. 2024-08-19T10:41:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 10 monclient: build_initial_monmap 2024-08-19T10:41:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 10 monclient: monmap: 2024-08-19T10:41:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:45.418 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:45.419 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:45.419 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:45.419 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:45.419 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:45.419 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:45.419 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:45.419 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:45.419 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:45.419 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:45.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 1 Processor -- start 2024-08-19T10:41:45.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 1 -- start start 2024-08-19T10:41:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.405+0000 7fd07ede3640 10 monclient: init 2024-08-19T10:41:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 10 monclient: _add_conns ranks=[2,8,3,4,1,5,0,7,6] 2024-08-19T10:41:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd07809b510 0x7fd0781d1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 10 monclient(hunting): picked mon.noname-b con 0x7fd07809b510 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ea30 0x7fd0781d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 10 monclient(hunting): picked mon.noname-e con 0x7fd07809ea30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07809ee30 0x7fd078090e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 10 monclient(hunting): picked mon.noname-g con 0x7fd07809ee30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:45.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd078050c20 con 0x7fd07809b510 2024-08-19T10:41:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd07806a540 con 0x7fd07809ee30 2024-08-19T10:41:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd0780a4c10 con 0x7fd07809ea30 2024-08-19T10:41:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07ede3640 10 monclient(hunting): authenticate will time out at 1150.980341s 2024-08-19T10:41:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07d380640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07809ee30 msgr2=0x7fd078090e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07d380640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07809ee30 0x7fd078090e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ea30 0x7fd0781d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:45.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07cb7f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd07809b510 0x7fd0781d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:45.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07cb7f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd07809b510 0x7fd0781d1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:46880/0 (socket says 172.21.15.190:46880) 2024-08-19T10:41:45.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07cb7f640 1 -- 172.21.15.190:0/2108415492 learned_addr learned my addr 172.21.15.190:0/2108415492 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:45.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ea30 0x7fd0781d1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:44070/0 (socket says 172.21.15.190:44070) 2024-08-19T10:41:45.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07cb7f640 10 monclient(hunting): get_auth_request con 0x7fd07809b510 auth_method 0 2024-08-19T10:41:45.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07cb7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:45.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07cb7f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:45.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07cb7f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:45.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 10 monclient(hunting): get_auth_request con 0x7fd07809ea30 auth_method 0 2024-08-19T10:41:45.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:45.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:45.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:45.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:45.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:45.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:45.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07cb7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:45.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07cb7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:45.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07cb7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:45.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 10 monclient(hunting): handle_auth_done global_id 5210 payload 824 2024-08-19T10:41:45.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 1 -- 172.21.15.190:0/2108415492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07809ee30 msgr2=0x7fd078090e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:45.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07809ee30 0x7fd078090e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 1 -- 172.21.15.190:0/2108415492 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd07809b510 msgr2=0x7fd0781d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:45.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd07809b510 0x7fd0781d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:45.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:45.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 1 monclient: found mon.noname-e 2024-08-19T10:41:45.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:45.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:45.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd077fff640 1 -- 172.21.15.190:0/2108415492 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd078094c40 con 0x7fd07809ea30 2024-08-19T10:41:45.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.409+0000 7fd07cb7f640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd07809b510 0x7fd0781d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:45.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd077fff640 1 -- 172.21.15.190:0/2108415492 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ea30 msgr2=0x7fd0781d1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:45.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd077fff640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ea30 0x7fd0781d1e70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd06400d5a0 tx=0x7fd06400da60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:45.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 1 -- 172.21.15.190:0/2108415492 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd064019070 con 0x7fd07809ea30 2024-08-19T10:41:45.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 10 monclient: dump: 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:45.427 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:45.428 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:45.428 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:45.428 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:45.428 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:45.428 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:45.428 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:45.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 10 monclient: _finish_auth 0 2024-08-19T10:41:45.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 10 monclient: _check_auth_tickets 2024-08-19T10:41:45.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:45.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07ede3640 5 monclient: authenticate success, global_id 5210 2024-08-19T10:41:45.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07ede3640 10 monclient: _renew_subs 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07ede3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07ede3640 1 -- 172.21.15.190:0/2108415492 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0780a7410 con 0x7fd07809ea30 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07ede3640 10 monclient: _renew_subs 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07ede3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07ede3640 1 -- 172.21.15.190:0/2108415492 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0780928e0 con 0x7fd07809ea30 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 1 -- 172.21.15.190:0/2108415492 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd064004450 con 0x7fd07809ea30 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 1 -- 172.21.15.190:0/2108415492 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd064003a90 con 0x7fd07809ea30 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 10 monclient: dump: 2024-08-19T10:41:45.429 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:45.430 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:45.431 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:45.431 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:45.431 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:45.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd076ffd640 4 set_mon_vals no callback set 2024-08-19T10:41:45.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07ede3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:45.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07ede3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:45.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07ede3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:45.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07ede3640 1 -- 172.21.15.190:0/2108415492 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07809b8f0 con 0x7fd07809ea30 2024-08-19T10:41:45.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 1 -- 172.21.15.190:0/2108415492 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd064003c70 con 0x7fd07809ea30 2024-08-19T10:41:45.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd05c03be00 0x7fd05c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:45.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd0757fa640 1 -- 172.21.15.190:0/2108415492 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(107..107 src has 1..107) ==== 6032+0+0 (secure 0 0 0) 0x7fd06400f040 con 0x7fd07809ea30 2024-08-19T10:41:45.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07cb7f640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd05c03be00 0x7fd05c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:45.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.413+0000 7fd07cb7f640 10 monclient: get_auth_request con 0x7fd05c03be00 auth_method 0 2024-08-19T10:41:45.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.417+0000 7fd07cb7f640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd05c03be00 0x7fd05c03e2c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd06c01ccc0 tx=0x7fd06c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:45.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.425+0000 7fd0757fa640 1 -- 172.21.15.190:0/2108415492 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd06404c340 con 0x7fd07809ea30 2024-08-19T10:41:45.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.425+0000 7fd0757fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:45.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.425+0000 7fd0757fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:45.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.601+0000 7fd07ede3640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-08-19T10:41:45.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.601+0000 7fd07ede3640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-08-19T10:41:45.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.601+0000 7fd07ede3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:45.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:45.601+0000 7fd07ede3640 1 -- 172.21.15.190:0/2108415492 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]} v 0) -- 0x7fd078093440 con 0x7fd07809ea30 2024-08-19T10:41:46.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.317+0000 7fd0757fa640 1 -- 172.21.15.190:0/2108415492 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}]=0 add item id 11 name 'osd.11' weight 123 at location {root=default} to crush map v108) ==== 193+0+0 (secure 0 0 0) 0x7fd0640220b0 con 0x7fd07809ea30 2024-08-19T10:41:46.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.317+0000 7fd0757fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-08-19T10:41:46.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.317+0000 7fd0757fa640 10 monclient: _finish_command 2 = system:0 add item id 11 name 'osd.11' weight 123 at location {root=default} to crush map 2024-08-19T10:41:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.429+0000 7fd07ede3640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-08-19T10:41:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.429+0000 7fd07ede3640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-08-19T10:41:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.429+0000 7fd07ede3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.429+0000 7fd07ede3640 1 -- 172.21.15.190:0/2108415492 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]} v 0) -- 0x7fd07813c8c0 con 0x7fd07809ea30 2024-08-19T10:41:46.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.429+0000 7fd0757fa640 1 -- 172.21.15.190:0/2108415492 <== mon.8 v2:172.21.15.89:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}]=0 set item id 11 name 'osd.11' weight 123 at location {root=default}: no change v108) ==== 191+0+0 (secure 0 0 0) 0x7fd06404c6c0 con 0x7fd07809ea30 2024-08-19T10:41:46.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.429+0000 7fd0757fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 11, "weight": 123.0, "args": ["root=default"]}] 2024-08-19T10:41:46.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.429+0000 7fd0757fa640 10 monclient: _finish_command 3 = system:0 set item id 11 name 'osd.11' weight 123 at location {root=default}: no change 2024-08-19T10:41:46.434 INFO:tasks.workunit.client.0.smithi190.stderr:set item id 11 name 'osd.11' weight 123 at location {root=default}: no change 2024-08-19T10:41:46.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 -- 172.21.15.190:0/2108415492 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd05c03be00 msgr2=0x7fd05c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:46.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd05c03be00 0x7fd05c03e2c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd06c01ccc0 tx=0x7fd06c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:46.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 10 monclient: shutdown 2024-08-19T10:41:46.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:46.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 -- 172.21.15.190:0/2108415492 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ea30 msgr2=0x7fd0781d1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:46.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ea30 0x7fd0781d1e70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd06400d5a0 tx=0x7fd06400da60 comp rx=0 tx=0).stop 2024-08-19T10:41:46.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 -- 172.21.15.190:0/2108415492 shutdown_connections 2024-08-19T10:41:46.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd05c03be00 0x7fd05c03e2c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd07809ea30 0x7fd0781d1e70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07809ee30 0x7fd078090e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 --2- 172.21.15.190:0/2108415492 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd07809b510 0x7fd0781d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 -- 172.21.15.190:0/2108415492 >> 172.21.15.190:0/2108415492 conn(0x7fd0780a4e70 msgr2=0x7fd0780a0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:46.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 -- 172.21.15.190:0/2108415492 shutdown_connections 2024-08-19T10:41:46.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.433+0000 7fd07ede3640 1 -- 172.21.15.190:0/2108415492 wait complete. 2024-08-19T10:41:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep osd.11 2024-08-19T10:41:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 123 2024-08-19T10:41:46.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.561+0000 7fe7692ca640 10 monclient: get_monmap_and_config 2024-08-19T10:41:46.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.561+0000 7fe7692ca640 10 monclient: build_initial_monmap 2024-08-19T10:41:46.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.561+0000 7fe7692ca640 10 monclient: monmap: 2024-08-19T10:41:46.567 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:46.567 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:46.567 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:46.567485+0000 2024-08-19T10:41:46.567 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:46.567485+0000 2024-08-19T10:41:46.567 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:46.568 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:46.568 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:46.568 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:46.568 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:46.568 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:46.568 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:46.568 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:46.568 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:46.568 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:46.569 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:46.569 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:46.569 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:46.569 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:46.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 1 Processor -- start 2024-08-19T10:41:46.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 1 -- start start 2024-08-19T10:41:46.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 10 monclient: init 2024-08-19T10:41:46.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:46.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 10 monclient: _add_conns ranks=[1,7,0,3,10,4,8,2,5,9,6] 2024-08-19T10:41:46.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe7641399f0 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:46.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe7640a5640 0x7fe76413e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:46.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 10 monclient(hunting): picked mon.noname-i con 0x7fe7640a5640 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:46.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe76413ee20 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:46.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe76413a4e0 con 0x7fe76413ee20 2024-08-19T10:41:46.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7640a8340 con 0x7fe7641399f0 2024-08-19T10:41:46.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:46.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fe7640a4580 con 0x7fe7640a5640 2024-08-19T10:41:46.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:46.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7692ca640 10 monclient(hunting): authenticate will time out at 1152.137479s 2024-08-19T10:41:46.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe762ffd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe7641399f0 0x7fe764139dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:41422/0 (socket says 172.21.15.190:41422) 2024-08-19T10:41:46.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe762ffd640 1 -- 172.21.15.190:0/2999834433 learned_addr learned my addr 172.21.15.190:0/2999834433 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:46.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 1 --2- 172.21.15.190:0/2999834433 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe7640a5640 0x7fe76413e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:46.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 10 monclient(hunting): get_auth_request con 0x7fe7640a5640 auth_method 0 2024-08-19T10:41:46.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:46.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:46.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:46.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:46.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:46.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:46.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 10 monclient(hunting): handle_auth_done global_id 14551 payload 274 2024-08-19T10:41:46.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 1 -- 172.21.15.190:0/2999834433 >> v1:172.21.15.190:6789/0 conn(0x7fe7641399f0 legacy=0x7fe764139dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:46.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 1 -- 172.21.15.190:0/2999834433 >> v1:172.21.15.89:6789/0 conn(0x7fe76413ee20 legacy=0x7fe764141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:46.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 10 monclient: _finish_hunting 0 2024-08-19T10:41:46.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 1 monclient: found mon.noname-i 2024-08-19T10:41:46.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:46.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:46.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.565+0000 7fe7627fc640 1 -- 172.21.15.190:0/2999834433 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7640a4860 con 0x7fe7640a5640 2024-08-19T10:41:46.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7627fc640 1 -- 172.21.15.190:0/2999834433 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe7640a5640 msgr2=0x7fe76413e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:46.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7627fc640 1 --2- 172.21.15.190:0/2999834433 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe7640a5640 0x7fe76413e8e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe7580097f0 tx=0x7fe75801c3a0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=187280cb2901b80e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:46.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 1 -- 172.21.15.190:0/2999834433 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7580b0070 con 0x7fe7640a5640 2024-08-19T10:41:46.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:46.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:46.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 10 monclient: dump: 2024-08-19T10:41:46.590 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:46.590 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:46.590 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.590 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.590 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:46.590 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:46.590 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:46.591 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:46.591 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:46.591 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:46.591 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:46.591 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:46.591 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:46.591 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:46.591 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:46.592 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:46.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:46.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:46.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:46.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 1 -- 172.21.15.190:0/2999834433 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7580aa810 con 0x7fe7640a5640 2024-08-19T10:41:46.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:46.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 1 -- 172.21.15.190:0/2999834433 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7580aae30 con 0x7fe7640a5640 2024-08-19T10:41:46.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:46.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:46.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe761ffb640 10 monclient: dump: 2024-08-19T10:41:46.593 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:46.593 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:46.593 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.593 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.594 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:46.594 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:46.594 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:46.594 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:46.594 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:46.595 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:46.595 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:46.595 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:46.595 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:46.595 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:46.595 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:46.596 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:46.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 5 monclient: authenticate success, global_id 14551 2024-08-19T10:41:46.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:46.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 4 set_mon_vals no callback set 2024-08-19T10:41:46.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 10 monclient: shutdown 2024-08-19T10:41:46.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:46.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 1 -- 172.21.15.190:0/2999834433 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe7640a5640 msgr2=0x7fe76413e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:46.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 1 --2- 172.21.15.190:0/2999834433 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe7640a5640 0x7fe76413e8e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe7580097f0 tx=0x7fe75801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 1 -- 172.21.15.190:0/2999834433 shutdown_connections 2024-08-19T10:41:46.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 1 --2- 172.21.15.190:0/2999834433 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe7640a5640 0x7fe76413e8e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 1 -- 172.21.15.190:0/2999834433 >> 172.21.15.190:0/2999834433 conn(0x7fe764074c10 msgr2=0x7fe7640abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:46.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 1 -- 172.21.15.190:0/2999834433 shutdown_connections 2024-08-19T10:41:46.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 1 -- 172.21.15.190:0/2999834433 wait complete. 2024-08-19T10:41:46.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 10 monclient: build_initial_monmap 2024-08-19T10:41:46.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.569+0000 7fe7692ca640 10 monclient: monmap: 2024-08-19T10:41:46.598 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:46.598 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:46.598 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:46.598 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:46.598 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:46.598 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:46.599 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:46.599 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:46.599 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:46.599 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:46.599 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:46.599 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:46.599 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:46.599 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:46.600 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:46.600 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:46.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 1 Processor -- start 2024-08-19T10:41:46.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 1 -- start start 2024-08-19T10:41:46.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient: init 2024-08-19T10:41:46.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:46.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient: _add_conns ranks=[5,8,2,3,7,6,1,0,4] 2024-08-19T10:41:46.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7640a5640 0x7fe7640a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:46.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient(hunting): picked mon.noname-h con 0x7fe7640a5640 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:46.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7641399f0 0x7fe7641d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:46.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient(hunting): picked mon.noname-e con 0x7fe7641399f0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:46.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe76413ee20 0x7fe76409e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:46.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient(hunting): picked mon.noname-b con 0x7fe76413ee20 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:46.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:46.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fe764145560 con 0x7fe76413ee20 2024-08-19T10:41:46.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:46.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe7640a5a20 con 0x7fe7640a5640 2024-08-19T10:41:46.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:46.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fe7641456e0 con 0x7fe7641399f0 2024-08-19T10:41:46.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:46.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient(hunting): authenticate will time out at 1152.144822s 2024-08-19T10:41:46.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7640a5640 0x7fe7640a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:46.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7640a5640 0x7fe7640a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:59618/0 (socket says 172.21.15.190:59618) 2024-08-19T10:41:46.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 1 -- 172.21.15.190:0/4272381269 learned_addr learned my addr 172.21.15.190:0/4272381269 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:46.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 10 monclient(hunting): get_auth_request con 0x7fe7640a5640 auth_method 0 2024-08-19T10:41:46.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:46.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:46.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:46.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:46.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:46.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:46.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7627fc640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7641399f0 0x7fe7641d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:46.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7637fe640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe76413ee20 0x7fe76409e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:46.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 10 monclient(hunting): handle_auth_done global_id 5387 payload 824 2024-08-19T10:41:46.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 1 -- 172.21.15.190:0/4272381269 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7641399f0 msgr2=0x7fe7641d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:46.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7641399f0 0x7fe7641d1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 1 -- 172.21.15.190:0/4272381269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe76413ee20 msgr2=0x7fe76409e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:46.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe76413ee20 0x7fe76409e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:46.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 1 monclient: found mon.noname-h 2024-08-19T10:41:46.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:46.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:46.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 1 -- 172.21.15.190:0/4272381269 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe764141940 con 0x7fe7640a5640 2024-08-19T10:41:46.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7627fc640 10 monclient: get_auth_request con 0x7fe7641399f0 auth_method 0 2024-08-19T10:41:46.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7627fc640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7641399f0 0x7fe7641d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:46.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7637fe640 10 monclient: get_auth_request con 0x7fe76413ee20 auth_method 0 2024-08-19T10:41:46.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7637fe640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe76413ee20 0x7fe76409e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:46.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 1 -- 172.21.15.190:0/4272381269 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7640a5640 msgr2=0x7fe7640a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:46.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe762ffd640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7640a5640 0x7fe7640a0e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe75400cc40 tx=0x7fe75400a430 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:46.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 1 -- 172.21.15.190:0/4272381269 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe75400ad50 con 0x7fe7640a5640 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 10 monclient: dump: 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:46.607 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 1 -- 172.21.15.190:0/4272381269 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe754012870 con 0x7fe7640a5640 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 5 monclient: authenticate success, global_id 5387 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient: _renew_subs 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7692ca640 1 -- 172.21.15.190:0/4272381269 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7640a2340 con 0x7fe7640a5640 2024-08-19T10:41:46.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 1 -- 172.21.15.190:0/4272381269 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe754007070 con 0x7fe7640a5640 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe74b7fe640 10 monclient: dump: 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:46.609 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe7692ca640 10 monclient: _renew_subs 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.573+0000 7fe7617fa640 4 set_mon_vals no callback set 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe7692ca640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe7692ca640 1 -- 172.21.15.190:0/4272381269 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7640ab010 con 0x7fe7640a5640 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe7692ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:46.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe7692ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:46.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe7692ca640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:46.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe7692ca640 1 -- 172.21.15.190:0/4272381269 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe764139dd0 con 0x7fe7640a5640 2024-08-19T10:41:46.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe74b7fe640 1 -- 172.21.15.190:0/4272381269 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe754012400 con 0x7fe7640a5640 2024-08-19T10:41:46.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe74b7fe640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe75003bd30 0x7fe75003e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:46.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe74b7fe640 1 -- 172.21.15.190:0/4272381269 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(108..108 src has 1..108) ==== 6054+0+0 (secure 0 0 0) 0x7fe75404f7e0 con 0x7fe7640a5640 2024-08-19T10:41:46.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe7627fc640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe75003bd30 0x7fe75003e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:46.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe7627fc640 10 monclient: get_auth_request con 0x7fe75003bd30 auth_method 0 2024-08-19T10:41:46.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.577+0000 7fe7627fc640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe75003bd30 0x7fe75003e1f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe7580ab000 tx=0x7fe7580a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:46.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.581+0000 7fe74b7fe640 1 -- 172.21.15.190:0/4272381269 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe764141940 con 0x7fe7640a5640 2024-08-19T10:41:46.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.581+0000 7fe74b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:46.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.581+0000 7fe74b7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:46.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.745+0000 7fe7692ca640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:46.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.745+0000 7fe7692ca640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:46.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.745+0000 7fe7692ca640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:46.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.745+0000 7fe7692ca640 1 -- 172.21.15.190:0/4272381269 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fe7640a17b0 con 0x7fe7640a5640 2024-08-19T10:41:46.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.745+0000 7fe74b7fe640 1 -- 172.21.15.190:0/4272381269 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v108) ==== 56+0+1173 (secure 0 0 0) 0x7fe754007290 con 0x7fe7640a5640 2024-08-19T10:41:46.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.745+0000 7fe74b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:46.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.745+0000 7fe74b7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:46.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.837+0000 7fe7692ca640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:46.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.837+0000 7fe7692ca640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:46.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.837+0000 7fe7692ca640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:46.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.837+0000 7fe7692ca640 1 -- 172.21.15.190:0/4272381269 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fe7640a3500 con 0x7fe7640a5640 2024-08-19T10:41:46.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe74b7fe640 1 -- 172.21.15.190:0/4272381269 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v108) ==== 56+0+1173 (secure 0 0 0) 0x7fe754051c50 con 0x7fe7640a5640 2024-08-19T10:41:46.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe74b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:46.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe74b7fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:46.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 -- 172.21.15.190:0/4272381269 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe75003bd30 msgr2=0x7fe75003e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:46.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe75003bd30 0x7fe75003e1f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe7580ab000 tx=0x7fe7580a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:46.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 10 monclient: shutdown 2024-08-19T10:41:46.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:46.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 -- 172.21.15.190:0/4272381269 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7640a5640 msgr2=0x7fe7640a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:46.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7640a5640 0x7fe7640a0e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe75400cc40 tx=0x7fe75400a430 comp rx=0 tx=0).stop 2024-08-19T10:41:46.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 -- 172.21.15.190:0/4272381269 shutdown_connections 2024-08-19T10:41:46.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe76413ee20 0x7fe76409e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe75003bd30 0x7fe75003e1f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7641399f0 0x7fe7641d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 --2- 172.21.15.190:0/4272381269 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7640a5640 0x7fe7640a0e20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 -- 172.21.15.190:0/4272381269 >> 172.21.15.190:0/4272381269 conn(0x7fe764074c10 msgr2=0x7fe7640a9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:46.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 -- 172.21.15.190:0/4272381269 shutdown_connections 2024-08-19T10:41:46.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.841+0000 7fe7692ca640 1 -- 172.21.15.190:0/4272381269 wait complete. 2024-08-19T10:41:46.859 INFO:tasks.workunit.client.0.smithi190.stdout: 11 123.00000 osd.11 down 1.00000 1.00000 2024-08-19T10:41:46.859 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush reweight osd.11 113 2024-08-19T10:41:46.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.973+0000 7fa007612640 10 monclient: get_monmap_and_config 2024-08-19T10:41:46.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.973+0000 7fa007612640 10 monclient: build_initial_monmap 2024-08-19T10:41:46.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.973+0000 7fa007612640 10 monclient: monmap: 2024-08-19T10:41:46.977 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:46.978038+0000 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:46.978038+0000 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:46.978 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:46.979 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:46.979 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:46.979 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:46.979 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:46.979 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:46.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 1 Processor -- start 2024-08-19T10:41:46.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 1 -- start start 2024-08-19T10:41:46.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 10 monclient: init 2024-08-19T10:41:46.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:46.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 10 monclient: _add_conns ranks=[0,4,2,7,5,1,9,3,8,6,10] 2024-08-19T10:41:46.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa000069430 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:46.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa0000a4390 0x7fa0000a4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:46.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 10 monclient(hunting): picked mon.noname-c con 0x7fa0000a4390 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:46.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa00013ba70 0x7fa00013de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:46.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 10 monclient(hunting): picked mon.noname-b con 0x7fa00013ba70 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:46.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa00009de60 con 0x7fa000069430 2024-08-19T10:41:46.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:46.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa000069810 con 0x7fa00013ba70 2024-08-19T10:41:46.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:46.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa0000a0140 con 0x7fa0000a4390 2024-08-19T10:41:46.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:46.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa007612640 10 monclient(hunting): authenticate will time out at 1152.549132s 2024-08-19T10:41:46.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa005baf640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa00013ba70 0x7fa00013de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:46.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa0000a4390 0x7fa0000a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:46.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa0053ae640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa000069430 0x7fa0000a3c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:56314/0 (socket says 172.21.15.190:56314) 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa0053ae640 1 -- 172.21.15.190:0/2992643615 learned_addr learned my addr 172.21.15.190:0/2992643615 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 10 monclient(hunting): get_auth_request con 0x7fa0000a4390 auth_method 0 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa005baf640 10 monclient(hunting): get_auth_request con 0x7fa00013ba70 auth_method 0 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa005baf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa005baf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa005baf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:46.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:46.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa005baf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:46.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa005baf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:46.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa005baf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:46.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 10 monclient(hunting): handle_auth_done global_id 5341 payload 274 2024-08-19T10:41:46.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 1 -- 172.21.15.190:0/2992643615 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa00013ba70 msgr2=0x7fa00013de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:46.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 1 --2- 172.21.15.190:0/2992643615 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa00013ba70 0x7fa00013de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 1 -- 172.21.15.190:0/2992643615 >> v1:172.21.15.89:6789/0 conn(0x7fa000069430 legacy=0x7fa0000a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:46.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 10 monclient: _finish_hunting 0 2024-08-19T10:41:46.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 1 monclient: found mon.noname-c 2024-08-19T10:41:46.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:46.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:41:46.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.977+0000 7fa004bad640 1 -- 172.21.15.190:0/2992643615 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00009e520 con 0x7fa0000a4390 2024-08-19T10:41:46.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa004bad640 1 -- 172.21.15.190:0/2992643615 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa0000a4390 msgr2=0x7fa0000a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:46.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa004bad640 1 --2- 172.21.15.190:0/2992643615 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa0000a4390 0x7fa0000a4770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00097f0 tx=0x7f9ff001c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a959ce04e864d759 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:46.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 1 -- 172.21.15.190:0/2992643615 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ff00b0070 con 0x7fa0000a4390 2024-08-19T10:41:46.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:46.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:41:46.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 10 monclient: dump: 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:46.991 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 1 -- 172.21.15.190:0/2992643615 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff00aa810 con 0x7fa0000a4390 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 5 monclient: authenticate success, global_id 5341 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 1 -- 172.21.15.190:0/2992643615 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ff00a9300 con 0x7fa0000a4390 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:46.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7f9ff7fff640 10 monclient: dump: 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:46.993 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:46.994 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:46.994 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:46.994 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:46.994 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:46.994 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:46.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:46.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 4 set_mon_vals no callback set 2024-08-19T10:41:46.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 10 monclient: shutdown 2024-08-19T10:41:46.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:46.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 1 -- 172.21.15.190:0/2992643615 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa0000a4390 msgr2=0x7fa0000a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:46.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 1 --2- 172.21.15.190:0/2992643615 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa0000a4390 0x7fa0000a4770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00097f0 tx=0x7f9ff001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 1 -- 172.21.15.190:0/2992643615 shutdown_connections 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 1 --2- 172.21.15.190:0/2992643615 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa0000a4390 0x7fa0000a4770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 1 --2- 172.21.15.190:0/2992643615 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa00013ba70 0x7fa00013de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 1 -- 172.21.15.190:0/2992643615 >> 172.21.15.190:0/2992643615 conn(0x7fa0000a4d70 msgr2=0x7fa0000a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 1 -- 172.21.15.190:0/2992643615 shutdown_connections 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.981+0000 7fa007612640 1 -- 172.21.15.190:0/2992643615 wait complete. 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient: build_initial_monmap 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient: monmap: 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:46.995 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:46.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 1 Processor -- start 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 1 -- start start 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient: init 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient: _add_conns ranks=[0,2,1,4,7,6,8,3,5] 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa000069430 0x7fa0001cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient(hunting): picked mon.noname-a con 0x7fa000069430 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0000a4390 0x7fa000136b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient(hunting): picked mon.noname-b con 0x7fa0000a4390 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa00013ba70 0x7fa00011e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient(hunting): picked mon.noname-f con 0x7fa00013ba70 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa0000a24d0 con 0x7fa000069430 2024-08-19T10:41:46.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:46.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0000a2350 con 0x7fa00013ba70 2024-08-19T10:41:46.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:46.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa000140bc0 con 0x7fa0000a4390 2024-08-19T10:41:46.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:46.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa007612640 10 monclient(hunting): authenticate will time out at 1152.557597s 2024-08-19T10:41:46.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa00013ba70 0x7fa00011e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:46.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa00013ba70 0x7fa00011e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:58134/0 (socket says 172.21.15.190:58134) 2024-08-19T10:41:46.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 1 -- 172.21.15.190:0/552296006 learned_addr learned my addr 172.21.15.190:0/552296006 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:46.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 10 monclient(hunting): get_auth_request con 0x7fa00013ba70 auth_method 0 2024-08-19T10:41:46.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:46.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa0053ae640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa000069430 0x7fa0001cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa004bad640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0000a4390 0x7fa000136b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa0053ae640 10 monclient(hunting): get_auth_request con 0x7fa000069430 auth_method 0 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa0053ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa0053ae640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa0053ae640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa004bad640 10 monclient(hunting): get_auth_request con 0x7fa0000a4390 auth_method 0 2024-08-19T10:41:46.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa004bad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa004bad640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa004bad640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa0053ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa0053ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa0053ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa004bad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa004bad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa004bad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 10 monclient(hunting): handle_auth_done global_id 14635 payload 824 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 1 -- 172.21.15.190:0/552296006 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0000a4390 msgr2=0x7fa000136b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0000a4390 0x7fa000136b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 1 -- 172.21.15.190:0/552296006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa000069430 msgr2=0x7fa0001cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:47.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa000069430 0x7fa0001cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:47.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 10 monclient: _finish_hunting 0 2024-08-19T10:41:47.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 1 monclient: found mon.noname-f 2024-08-19T10:41:47.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:47.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:47.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.985+0000 7fa005baf640 1 -- 172.21.15.190:0/552296006 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa000137b00 con 0x7fa00013ba70 2024-08-19T10:41:47.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa005baf640 1 -- 172.21.15.190:0/552296006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa00013ba70 msgr2=0x7fa00011e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:47.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa005baf640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa00013ba70 0x7fa00011e8c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009cb0 tx=0x7f9ff800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:47.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 1 -- 172.21.15.190:0/552296006 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ff8018020 con 0x7fa00013ba70 2024-08-19T10:41:47.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:47.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:47.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 10 monclient: dump: 2024-08-19T10:41:47.003 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:47.004 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa007612640 5 monclient: authenticate success, global_id 14635 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 1 -- 172.21.15.190:0/552296006 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff80148f0 con 0x7fa00013ba70 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa007612640 10 monclient: _renew_subs 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa007612640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa007612640 1 -- 172.21.15.190:0/552296006 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0000a3ee0 con 0x7fa00013ba70 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa007612640 10 monclient: _renew_subs 2024-08-19T10:41:47.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa007612640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa007612640 1 -- 172.21.15.190:0/552296006 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa000120d30 con 0x7fa00013ba70 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 1 -- 172.21.15.190:0/552296006 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ff80133e0 con 0x7fa00013ba70 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff77fe640 4 set_mon_vals no callback set 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 10 monclient: dump: 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:47.006 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:47.007 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:47.007 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:47.007 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:47.007 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:47.007 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:47.007 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:47.007 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:47.007 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:47.007 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:47.007 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:47.007 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa007612640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa007612640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa007612640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa007612640 1 -- 172.21.15.190:0/552296006 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe0005090 con 0x7fa00013ba70 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 1 -- 172.21.15.190:0/552296006 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9ff801b020 con 0x7fa00013ba70 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9fe403be00 0x7f9fe403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7f9ff5ffb640 1 -- 172.21.15.190:0/552296006 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6054+0+0 (secure 0 0 0) 0x7f9ff804fb40 con 0x7fa00013ba70 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa0053ae640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9fe403be00 0x7f9fe403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.989+0000 7fa0053ae640 10 monclient: get_auth_request con 0x7f9fe403be00 auth_method 0 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.993+0000 7fa0053ae640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9fe403be00 0x7f9fe403e2c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc007ca0 tx=0x7f9ffc007bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.993+0000 7f9ff5ffb640 1 -- 172.21.15.190:0/552296006 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9ff8022030 con 0x7fa00013ba70 2024-08-19T10:41:47.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.993+0000 7f9ff5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:47.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:46.993+0000 7f9ff5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:47.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:47.157+0000 7fa007612640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-08-19T10:41:47.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:47.157+0000 7fa007612640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-08-19T10:41:47.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:47.157+0000 7fa007612640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:47.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:47.157+0000 7fa007612640 1 -- 172.21.15.190:0/552296006 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0} v 0) -- 0x7f9fe0005b90 con 0x7fa00013ba70 2024-08-19T10:41:47.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:47.329+0000 7f9ff5ffb640 1 -- 172.21.15.190:0/552296006 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}]=0 reweighted item id 11 name 'osd.11' to 113 in crush map v109) ==== 156+0+0 (secure 0 0 0) 0x7fa0000a3ee0 con 0x7fa00013ba70 2024-08-19T10:41:47.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:47.329+0000 7f9ff5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-08-19T10:41:47.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:47.329+0000 7f9ff5ffb640 10 monclient: _finish_command 2 = system:0 reweighted item id 11 name 'osd.11' to 113 in crush map 2024-08-19T10:41:47.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:47.437+0000 7fa007612640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-08-19T10:41:47.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:47.437+0000 7fa007612640 10 monclient: _send_command 3 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-08-19T10:41:47.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:47.437+0000 7fa007612640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:47.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:47.437+0000 7fa007612640 1 -- 172.21.15.190:0/552296006 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0} v 0) -- 0x7f9fe0002290 con 0x7fa00013ba70 2024-08-19T10:41:48.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.333+0000 7f9ff5ffb640 1 -- 172.21.15.190:0/552296006 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}]=0 reweighted item id 11 name 'osd.11' to 113 in crush map v110) ==== 156+0+0 (secure 0 0 0) 0x7fa000137b00 con 0x7fa00013ba70 2024-08-19T10:41:48.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.333+0000 7f9ff5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 113.0}] 2024-08-19T10:41:48.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.333+0000 7f9ff5ffb640 10 monclient: _finish_command 3 = system:0 reweighted item id 11 name 'osd.11' to 113 in crush map 2024-08-19T10:41:48.337 INFO:tasks.workunit.client.0.smithi190.stderr:reweighted item id 11 name 'osd.11' to 113 in crush map 2024-08-19T10:41:48.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 -- 172.21.15.190:0/552296006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9fe403be00 msgr2=0x7f9fe403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:48.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9fe403be00 0x7f9fe403e2c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc007ca0 tx=0x7f9ffc007bf0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 10 monclient: shutdown 2024-08-19T10:41:48.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:48.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 -- 172.21.15.190:0/552296006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa00013ba70 msgr2=0x7fa00011e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:48.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa00013ba70 0x7fa00011e8c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009cb0 tx=0x7f9ff800c490 comp rx=0 tx=0).stop 2024-08-19T10:41:48.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 -- 172.21.15.190:0/552296006 shutdown_connections 2024-08-19T10:41:48.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9fe403be00 0x7f9fe403e2c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa00013ba70 0x7fa00011e8c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa0000a4390 0x7fa000136b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 --2- 172.21.15.190:0/552296006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa000069430 0x7fa0001cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 -- 172.21.15.190:0/552296006 >> 172.21.15.190:0/552296006 conn(0x7fa0000a4d70 msgr2=0x7fa0000a1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:48.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 -- 172.21.15.190:0/552296006 shutdown_connections 2024-08-19T10:41:48.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.337+0000 7fa007612640 1 -- 172.21.15.190:0/552296006 wait complete. 2024-08-19T10:41:48.367 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush reweight osd.11 123456 2024-08-19T10:41:48.367 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:41:48.367 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush reweight osd.11 123456 2024-08-19T10:41:48.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.485+0000 7fa44a082640 10 monclient: get_monmap_and_config 2024-08-19T10:41:48.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.485+0000 7fa44a082640 10 monclient: build_initial_monmap 2024-08-19T10:41:48.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.485+0000 7fa44a082640 10 monclient: monmap: 2024-08-19T10:41:48.490 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:48.490 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:48.490 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:48.490493+0000 2024-08-19T10:41:48.490 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:48.490493+0000 2024-08-19T10:41:48.490 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:48.490 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.490 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:48.490 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:48.491 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:48.491 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:48.491 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:48.491 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:48.491 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:48.491 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:48.491 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:48.491 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:48.491 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:48.491 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 1 Processor -- start 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 1 -- start start 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 10 monclient: init 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 10 monclient: _add_conns ranks=[0,2,7,1,4,5,6,10,9,8,3] 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa444138770 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa44409ec00 0x7fa44413e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 10 monclient(hunting): picked mon.noname-b con 0x7fa44409ec00 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa44409f020 0x7fa444140c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 10 monclient(hunting): picked mon.noname-i con 0x7fa44409f020 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44409db90 con 0x7fa444138770 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:48.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa44409de70 con 0x7fa44409ec00 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa44409e150 con 0x7fa44409f020 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa44a082640 10 monclient(hunting): authenticate will time out at 1154.061081s 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa44409f020 0x7fa444140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa44409f020 0x7fa444140c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:60792/0 (socket says 172.21.15.190:60792) 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 1 -- 172.21.15.190:0/4102159815 learned_addr learned my addr 172.21.15.190:0/4102159815 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 10 monclient(hunting): get_auth_request con 0x7fa44409f020 auth_method 0 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:48.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa43bfff640 1 --2- 172.21.15.190:0/4102159815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa44409ec00 0x7fa44413e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 10 monclient(hunting): handle_auth_done global_id 14560 payload 274 2024-08-19T10:41:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 1 -- 172.21.15.190:0/4102159815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa44409ec00 msgr2=0x7fa44413e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 1 --2- 172.21.15.190:0/4102159815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa44409ec00 0x7fa44413e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 1 -- 172.21.15.190:0/4102159815 >> v1:172.21.15.89:6789/0 conn(0x7fa444138770 legacy=0x7fa444138b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 1 monclient: found mon.noname-i 2024-08-19T10:41:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.489+0000 7fa443fff640 1 -- 172.21.15.190:0/4102159815 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44409e430 con 0x7fa44409f020 2024-08-19T10:41:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa443fff640 1 -- 172.21.15.190:0/4102159815 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa44409f020 msgr2=0x7fa444140c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa443fff640 1 --2- 172.21.15.190:0/4102159815 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa44409f020 0x7fa444140c00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009e80 tx=0x7fa43c01c910 comp rx=0 tx=0).ready entity=mon.7 client_cookie=40e3e2b55ee3707a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 1 -- 172.21.15.190:0/4102159815 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa43c0091b0 con 0x7fa44409f020 2024-08-19T10:41:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 10 monclient: dump: 2024-08-19T10:41:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:48.501 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 10 monclient: _finish_auth 0 2024-08-19T10:41:48.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:48.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:48.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 1 -- 172.21.15.190:0/4102159815 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43c0a5b80 con 0x7fa44409f020 2024-08-19T10:41:48.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 5 monclient: authenticate success, global_id 14560 2024-08-19T10:41:48.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:48.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:48.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:48.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 1 -- 172.21.15.190:0/4102159815 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa43c0a5d30 con 0x7fa44409f020 2024-08-19T10:41:48.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:48.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 4 set_mon_vals no callback set 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa442ffd640 10 monclient: dump: 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:48.503 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 10 monclient: shutdown 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 1 -- 172.21.15.190:0/4102159815 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa44409f020 msgr2=0x7fa444140c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 1 --2- 172.21.15.190:0/4102159815 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa44409f020 0x7fa444140c00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009e80 tx=0x7fa43c01c910 comp rx=0 tx=0).stop 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 1 -- 172.21.15.190:0/4102159815 shutdown_connections 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 1 --2- 172.21.15.190:0/4102159815 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa44409f020 0x7fa444140c00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 1 --2- 172.21.15.190:0/4102159815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa44409ec00 0x7fa44413e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 1 -- 172.21.15.190:0/4102159815 >> 172.21.15.190:0/4102159815 conn(0x7fa4440a4d70 msgr2=0x7fa4440a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 1 -- 172.21.15.190:0/4102159815 shutdown_connections 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 1 -- 172.21.15.190:0/4102159815 wait complete. 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 10 monclient: build_initial_monmap 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 10 monclient: monmap: 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:48.505 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 1 Processor -- start 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 1 -- start start 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.493+0000 7fa44a082640 10 monclient: init 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient: _add_conns ranks=[6,1,4,3,5,8,7,0,2] 2024-08-19T10:41:48.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa44409ec00 0x7fa444132de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient(hunting): picked mon.noname-d con 0x7fa44409ec00 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa44409f020 0x7fa444133320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient(hunting): picked mon.noname-f con 0x7fa44409f020 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa444138770 0x7fa44413bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient(hunting): picked mon.noname-c con 0x7fa444138770 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa444143ae0 con 0x7fa44409f020 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa444143960 con 0x7fa444138770 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa444143c60 con 0x7fa44409ec00 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:48.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient(hunting): authenticate will time out at 1154.068183s 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa44409f020 0x7fa444133320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa44409f020 0x7fa444133320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:58140/0 (socket says 172.21.15.190:58140) 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 1 -- 172.21.15.190:0/2951744339 learned_addr learned my addr 172.21.15.190:0/2951744339 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 10 monclient(hunting): get_auth_request con 0x7fa44409f020 auth_method 0 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa4437fe640 1 --2- 172.21.15.190:0/2951744339 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa44409ec00 0x7fa444132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa443fff640 1 --2- 172.21.15.190:0/2951744339 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa444138770 0x7fa44413bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:48.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa4437fe640 10 monclient(hunting): get_auth_request con 0x7fa44409ec00 auth_method 0 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa4437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa4437fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa4437fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 10 monclient(hunting): handle_auth_done global_id 14644 payload 824 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 1 -- 172.21.15.190:0/2951744339 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa44409ec00 msgr2=0x7fa444132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 1 --2- 172.21.15.190:0/2951744339 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa44409ec00 0x7fa444132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 1 -- 172.21.15.190:0/2951744339 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa444138770 msgr2=0x7fa44413bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 1 --2- 172.21.15.190:0/2951744339 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa444138770 0x7fa44413bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 1 monclient: found mon.noname-f 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:48.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:48.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 1 -- 172.21.15.190:0/2951744339 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4440a7ae0 con 0x7fa44409f020 2024-08-19T10:41:48.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa443fff640 10 monclient: get_auth_request con 0x7fa444138770 auth_method 0 2024-08-19T10:41:48.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa443fff640 1 --2- 172.21.15.190:0/2951744339 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa444138770 0x7fa44413bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:48.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 1 -- 172.21.15.190:0/2951744339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa44409f020 msgr2=0x7fa444133320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:48.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa43bfff640 1 --2- 172.21.15.190:0/2951744339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa44409f020 0x7fa444133320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00c7f0 tx=0x7fa42c00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:48.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 1 -- 172.21.15.190:0/2951744339 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa42c0044a0 con 0x7fa44409f020 2024-08-19T10:41:48.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:48.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:48.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 10 monclient: dump: 2024-08-19T10:41:48.512 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:48.512 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:48.512 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.512 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 10 monclient: _finish_auth 0 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 1 -- 172.21.15.190:0/2951744339 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa42c0077d0 con 0x7fa44409f020 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 1 -- 172.21.15.190:0/2951744339 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa42c007df0 con 0x7fa44409f020 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 5 monclient: authenticate success, global_id 14644 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient: _renew_subs 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 1 -- 172.21.15.190:0/2951744339 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44413d9b0 con 0x7fa44409f020 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa4427fc640 4 set_mon_vals no callback set 2024-08-19T10:41:48.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa440ff9640 10 monclient: dump: 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:48.515 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient: _renew_subs 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.497+0000 7fa44a082640 1 -- 172.21.15.190:0/2951744339 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44413fd10 con 0x7fa44409f020 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.501+0000 7fa44a082640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.501+0000 7fa44a082640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.501+0000 7fa44a082640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.501+0000 7fa44a082640 1 -- 172.21.15.190:0/2951744339 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa444138b50 con 0x7fa44409f020 2024-08-19T10:41:48.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.501+0000 7fa440ff9640 1 -- 172.21.15.190:0/2951744339 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa42c007980 con 0x7fa44409f020 2024-08-19T10:41:48.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.501+0000 7fa440ff9640 1 --2- 172.21.15.190:0/2951744339 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa42403be00 0x7fa42403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:48.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.501+0000 7fa440ff9640 1 -- 172.21.15.190:0/2951744339 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6083+0+0 (secure 0 0 0) 0x7fa42c04efa0 con 0x7fa44409f020 2024-08-19T10:41:48.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.501+0000 7fa4437fe640 1 --2- 172.21.15.190:0/2951744339 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa42403be00 0x7fa42403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:48.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.501+0000 7fa4437fe640 10 monclient: get_auth_request con 0x7fa42403be00 auth_method 0 2024-08-19T10:41:48.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.501+0000 7fa4437fe640 1 --2- 172.21.15.190:0/2951744339 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa42403be00 0x7fa42403e2c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa43400b810 tx=0x7fa434008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:48.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.505+0000 7fa440ff9640 1 -- 172.21.15.190:0/2951744339 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fa4440a7ae0 con 0x7fa44409f020 2024-08-19T10:41:48.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.505+0000 7fa440ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:48.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.505+0000 7fa440ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:48.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.681+0000 7fa44a082640 10 start_mon_command cmd=[{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-08-19T10:41:48.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.681+0000 7fa44a082640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-08-19T10:41:48.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.681+0000 7fa44a082640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:48.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.681+0000 7fa44a082640 1 -- 172.21.15.190:0/2951744339 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0} v 0) -- 0x7fa444133860 con 0x7fa44409f020 2024-08-19T10:41:48.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.681+0000 7fa440ff9640 1 -- 172.21.15.190:0/2951744339 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}]=-75 (75) Value too large for defined data type v110) ==== 146+0+0 (secure 0 0 0) 0x7fa42c015020 con 0x7fa44409f020 2024-08-19T10:41:48.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.681+0000 7fa440ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight", "name": "osd.11", "weight": 123456.0}] 2024-08-19T10:41:48.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.681+0000 7fa440ff9640 10 monclient: _finish_command 2 = mon:75 (75) Value too large for defined data type 2024-08-19T10:41:48.686 INFO:tasks.workunit.client.0.smithi190.stderr:Error EOVERFLOW: (75) Value too large for defined data type 2024-08-19T10:41:48.691 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:41:48.691 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:48.691 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep osd.11 2024-08-19T10:41:48.691 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 113 2024-08-19T10:41:48.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.793+0000 7f32833eb640 10 monclient: get_monmap_and_config 2024-08-19T10:41:48.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.793+0000 7f32833eb640 10 monclient: build_initial_monmap 2024-08-19T10:41:48.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.793+0000 7f32833eb640 10 monclient: monmap: 2024-08-19T10:41:48.799 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:48.799 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:48.799 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:48.799279+0000 2024-08-19T10:41:48.799 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:48.799279+0000 2024-08-19T10:41:48.799 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:48.799 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.799 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:48.799 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:48.799 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:48.800 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:48.800 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:48.800 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:48.800 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:48.800 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:48.800 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:48.800 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:48.800 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:48.800 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 1 Processor -- start 2024-08-19T10:41:48.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 1 -- start start 2024-08-19T10:41:48.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 10 monclient: init 2024-08-19T10:41:48.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:48.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 10 monclient: _add_conns ranks=[9,0,1,4,7,6,5,3,2,8,10] 2024-08-19T10:41:48.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f327c0a36a0 0x7f327c0a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:48.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 10 monclient(hunting): picked mon.noname-a con 0x7f327c0a36a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:48.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f327c09cd50 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:48.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f327c13e970 addr v1:172.21.15.190:6789/0 2024-08-19T10:41:48.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f327c09ddf0 con 0x7f327c09cd50 2024-08-19T10:41:48.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f327c09e0d0 con 0x7f327c13e970 2024-08-19T10:41:48.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:48.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f327c09e3b0 con 0x7f327c0a36a0 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f32833eb640 10 monclient(hunting): authenticate will time out at 1154.369099s 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281988640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f327c13e970 0x7f327c140d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:41434/0 (socket says 172.21.15.190:41434) 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281988640 1 -- 172.21.15.190:0/961499966 learned_addr learned my addr 172.21.15.190:0/961499966 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 1 --2- 172.21.15.190:0/961499966 >> v2:172.21.15.89:3300/0 conn(0x7f327c0a36a0 0x7f327c0a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 10 monclient(hunting): get_auth_request con 0x7f327c0a36a0 auth_method 0 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:48.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 10 monclient(hunting): handle_auth_done global_id 5418 payload 274 2024-08-19T10:41:48.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 1 -- 172.21.15.190:0/961499966 >> v1:172.21.15.190:6789/0 conn(0x7f327c13e970 legacy=0x7f327c140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:48.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 1 -- 172.21.15.190:0/961499966 >> v1:172.21.15.89:6789/0 conn(0x7f327c09cd50 legacy=0x7f327c0a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:48.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 10 monclient: _finish_hunting 0 2024-08-19T10:41:48.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 1 monclient: found mon.noname-a 2024-08-19T10:41:48.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:48.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:48.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.797+0000 7f3281187640 1 -- 172.21.15.190:0/961499966 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c0a55a0 con 0x7f327c0a36a0 2024-08-19T10:41:48.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3281187640 1 -- 172.21.15.190:0/961499966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f327c0a36a0 msgr2=0x7f327c0a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:48.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3281187640 1 --2- 172.21.15.190:0/961499966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f327c0a36a0 0x7f327c0a3a80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f32780097c0 tx=0x7f327801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e48e6bfef284a66 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:48.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 1 -- 172.21.15.190:0/961499966 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32780b0070 con 0x7f327c0a36a0 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 10 monclient: dump: 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:48.808 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 1 -- 172.21.15.190:0/961499966 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32780aa810 con 0x7f327c0a36a0 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:48.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 1 -- 172.21.15.190:0/961499966 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32780aae30 con 0x7f327c0a36a0 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f3273fff640 10 monclient: dump: 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:48.810 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 5 monclient: authenticate success, global_id 5418 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 4 set_mon_vals no callback set 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 10 monclient: shutdown 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 1 -- 172.21.15.190:0/961499966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f327c0a36a0 msgr2=0x7f327c0a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:48.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 1 --2- 172.21.15.190:0/961499966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f327c0a36a0 0x7f327c0a3a80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f32780097c0 tx=0x7f327801c360 comp rx=0 tx=0).stop 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 1 -- 172.21.15.190:0/961499966 shutdown_connections 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 1 --2- 172.21.15.190:0/961499966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f327c0a36a0 0x7f327c0a3a80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 1 -- 172.21.15.190:0/961499966 >> 172.21.15.190:0/961499966 conn(0x7f327c0a4790 msgr2=0x7f327c0a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 1 -- 172.21.15.190:0/961499966 shutdown_connections 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.801+0000 7f32833eb640 1 -- 172.21.15.190:0/961499966 wait complete. 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient: build_initial_monmap 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient: monmap: 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.812 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 1 Processor -- start 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 1 -- start start 2024-08-19T10:41:48.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient: init 2024-08-19T10:41:48.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient: _add_conns ranks=[4,5,7,6,1,8,2,3,0] 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f327c09cd50 0x7f327c125b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient(hunting): picked mon.noname-c con 0x7f327c09cd50 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f327c0a36a0 0x7f327c1242b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient(hunting): picked mon.noname-h con 0x7f327c0a36a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f327c13e970 0x7f327c1d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient(hunting): picked mon.noname-i con 0x7f327c13e970 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f327c144e20 con 0x7f327c09cd50 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f327c144ca0 con 0x7f327c0a36a0 2024-08-19T10:41:48.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f327c06b6c0 con 0x7f327c13e970 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f32833eb640 10 monclient(hunting): authenticate will time out at 1154.377224s 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f327c0a36a0 0x7f327c1242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281988640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f327c13e970 0x7f327c1d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f327c0a36a0 0x7f327c1242b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:59630/0 (socket says 172.21.15.190:59630) 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 1 -- 172.21.15.190:0/142403004 learned_addr learned my addr 172.21.15.190:0/142403004 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281988640 10 monclient(hunting): get_auth_request con 0x7f327c13e970 auth_method 0 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281988640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281988640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281988640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 10 monclient(hunting): get_auth_request con 0x7f327c0a36a0 auth_method 0 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:48.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281988640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281988640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281988640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281187640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f327c09cd50 0x7f327c125b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281187640 10 monclient(hunting): get_auth_request con 0x7f327c09cd50 auth_method 0 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281187640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281187640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281187640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:48.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 10 monclient(hunting): handle_auth_done global_id 5396 payload 824 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 1 -- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f327c13e970 msgr2=0x7f327c1d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f327c13e970 0x7f327c1d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 1 -- 172.21.15.190:0/142403004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f327c09cd50 msgr2=0x7f327c125b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f327c09cd50 0x7f327c125b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 10 monclient: _finish_hunting 0 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 1 monclient: found mon.noname-h 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3280986640 1 -- 172.21.15.190:0/142403004 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c127320 con 0x7f327c0a36a0 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.805+0000 7f3281988640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f327c13e970 0x7f327c1d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3280986640 1 -- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f327c0a36a0 msgr2=0x7f327c1242b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3280986640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f327c0a36a0 0x7f327c1242b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c480 tx=0x7f326c00c940 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:48.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 1 -- 172.21.15.190:0/142403004 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f326c012cb0 con 0x7f327c0a36a0 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 10 monclient: dump: 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:48.822 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 1 -- 172.21.15.190:0/142403004 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326c017070 con 0x7f327c0a36a0 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 1 -- 172.21.15.190:0/142403004 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f326c0117f0 con 0x7f327c0a36a0 2024-08-19T10:41:48.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 10 monclient: dump: 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:48.824 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32737fe640 4 set_mon_vals no callback set 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32833eb640 5 monclient: authenticate success, global_id 5396 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32833eb640 10 monclient: _renew_subs 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32833eb640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32833eb640 1 -- 172.21.15.190:0/142403004 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327c1264b0 con 0x7f327c0a36a0 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32833eb640 10 monclient: _renew_subs 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32833eb640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32833eb640 1 -- 172.21.15.190:0/142403004 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f327c127040 con 0x7f327c0a36a0 2024-08-19T10:41:48.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32833eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32833eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32833eb640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f32833eb640 1 -- 172.21.15.190:0/142403004 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327c141e80 con 0x7f327c0a36a0 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 1 -- 172.21.15.190:0/142403004 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f326c011380 con 0x7f327c0a36a0 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326003be00 0x7f326003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3271ffb640 1 -- 172.21.15.190:0/142403004 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(110..110 src has 1..110) ==== 6083+0+0 (secure 0 0 0) 0x7f326c04ed90 con 0x7f327c0a36a0 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3281187640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326003be00 0x7f326003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.809+0000 7f3281187640 10 monclient: get_auth_request con 0x7f326003be00 auth_method 0 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.813+0000 7f3281187640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326003be00 0x7f326003e2c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f32780ab000 tx=0x7f32780a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.813+0000 7f3271ffb640 1 -- 172.21.15.190:0/142403004 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f326c012ad0 con 0x7f327c0a36a0 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.813+0000 7f3271ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:48.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.813+0000 7f3271ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:48.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.981+0000 7f32833eb640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:48.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.981+0000 7f32833eb640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:48.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.981+0000 7f32833eb640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:48.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.981+0000 7f32833eb640 1 -- 172.21.15.190:0/142403004 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f327c0a3a80 con 0x7f327c0a36a0 2024-08-19T10:41:48.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.981+0000 7f3271ffb640 1 -- 172.21.15.190:0/142403004 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v110) ==== 56+0+1173 (secure 0 0 0) 0x7f326c04d270 con 0x7f327c0a36a0 2024-08-19T10:41:48.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.981+0000 7f3271ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:48.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:48.981+0000 7f3271ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:49.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:49.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:49.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:49.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 -- 172.21.15.190:0/142403004 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f327c0008e0 con 0x7f327c0a36a0 2024-08-19T10:41:49.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f3271ffb640 1 -- 172.21.15.190:0/142403004 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v110) ==== 56+0+1173 (secure 0 0 0) 0x7f326c03f1c0 con 0x7f327c0a36a0 2024-08-19T10:41:49.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f3271ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:49.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f3271ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:49.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 -- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326003be00 msgr2=0x7f326003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:49.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326003be00 0x7f326003e2c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f32780ab000 tx=0x7f32780a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:49.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 10 monclient: shutdown 2024-08-19T10:41:49.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:49.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 -- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f327c0a36a0 msgr2=0x7f327c1242b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:49.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f327c0a36a0 0x7f327c1242b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c480 tx=0x7f326c00c940 comp rx=0 tx=0).stop 2024-08-19T10:41:49.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 -- 172.21.15.190:0/142403004 shutdown_connections 2024-08-19T10:41:49.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f326003be00 0x7f326003e2c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:49.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f327c13e970 0x7f327c1d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:49.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f327c0a36a0 0x7f327c1242b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:49.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 --2- 172.21.15.190:0/142403004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f327c09cd50 0x7f327c125b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:49.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 -- 172.21.15.190:0/142403004 >> 172.21.15.190:0/142403004 conn(0x7f327c0a4790 msgr2=0x7f327c0a5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:49.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 -- 172.21.15.190:0/142403004 shutdown_connections 2024-08-19T10:41:49.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.077+0000 7f32833eb640 1 -- 172.21.15.190:0/142403004 wait complete. 2024-08-19T10:41:49.096 INFO:tasks.workunit.client.0.smithi190.stdout: 11 113.00000 osd.11 down 1.00000 1.00000 2024-08-19T10:41:49.096 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm osd.11 2024-08-19T10:41:49.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient: get_monmap_and_config 2024-08-19T10:41:49.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient: build_initial_monmap 2024-08-19T10:41:49.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient: monmap: 2024-08-19T10:41:49.212 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:49.212 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:49.212 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:49.212731+0000 2024-08-19T10:41:49.212 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:49.212731+0000 2024-08-19T10:41:49.212 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:49.213 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:49.213 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:49.213 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:49.213 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:49.213 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:49.213 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:49.213 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:49.213 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:49.213 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:49.214 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:49.214 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:49.214 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:49.214 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:49.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 1 Processor -- start 2024-08-19T10:41:49.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 1 -- start start 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient: init 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient: _add_conns ranks=[0,10,5,2,7,3,8,6,1,9,4] 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe7e8139980 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7e80a5730 0x7fe7e80a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient(hunting): picked mon.noname-f con 0x7fe7e80a5730 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7e813e8c0 0x7fe7e8140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient(hunting): picked mon.noname-h con 0x7fe7e813e8c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e80a8470 con 0x7fe7e8139980 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e809ca60 con 0x7fe7e80a5730 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:49.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe7e80a3640 con 0x7fe7e813e8c0 2024-08-19T10:41:49.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:49.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ef327640 10 monclient(hunting): authenticate will time out at 1154.782983s 2024-08-19T10:41:49.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ec8c2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7e80a5730 0x7fe7e80a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:49.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7e813e8c0 0x7fe7e8140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:49.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7e813e8c0 0x7fe7e8140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:59640/0 (socket says 172.21.15.190:59640) 2024-08-19T10:41:49.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 1 -- 172.21.15.190:0/1265084336 learned_addr learned my addr 172.21.15.190:0/1265084336 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:49.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 10 monclient(hunting): get_auth_request con 0x7fe7e813e8c0 auth_method 0 2024-08-19T10:41:49.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ec8c2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7e80a5730 0x7fe7e80a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:58154/0 (socket says 172.21.15.190:58154) 2024-08-19T10:41:49.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:49.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:49.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:49.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ec8c2640 10 monclient(hunting): get_auth_request con 0x7fe7e80a5730 auth_method 0 2024-08-19T10:41:49.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ec8c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:49.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ec8c2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:49.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ec8c2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:49.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:49.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:49.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:49.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ec8c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:49.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ec8c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:49.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ec8c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:49.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 10 monclient(hunting): handle_auth_done global_id 5405 payload 274 2024-08-19T10:41:49.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 1 -- 172.21.15.190:0/1265084336 >> v2:172.21.15.190:3300/0 conn(0x7fe7e80a5730 msgr2=0x7fe7e80a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:49.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 1 --2- 172.21.15.190:0/1265084336 >> v2:172.21.15.190:3300/0 conn(0x7fe7e80a5730 0x7fe7e80a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:49.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 1 -- 172.21.15.190:0/1265084336 >> v1:172.21.15.89:6789/0 conn(0x7fe7e8139980 legacy=0x7fe7e8139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:49.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 10 monclient: _finish_hunting 0 2024-08-19T10:41:49.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 1 monclient: found mon.noname-h 2024-08-19T10:41:49.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:49.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:49.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ed8c4640 1 -- 172.21.15.190:0/1265084336 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e80a3850 con 0x7fe7e813e8c0 2024-08-19T10:41:49.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.209+0000 7fe7ec8c2640 1 --2- 172.21.15.190:0/1265084336 >> v2:172.21.15.190:3300/0 conn(0x7fe7e80a5730 0x7fe7e80a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:49.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.213+0000 7fe7ed8c4640 1 -- 172.21.15.190:0/1265084336 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7e813e8c0 msgr2=0x7fe7e8140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:49.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.213+0000 7fe7ed8c4640 1 --2- 172.21.15.190:0/1265084336 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7e813e8c0 0x7fe7e8140cb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0009e50 tx=0x7fe7e001c8f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=6b702b38748c1e48 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:49.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.213+0000 7fe7dffff640 1 -- 172.21.15.190:0/1265084336 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7e00090d0 con 0x7fe7e813e8c0 2024-08-19T10:41:49.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.213+0000 7fe7dffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:49.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.213+0000 7fe7dffff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:49.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.213+0000 7fe7dffff640 10 monclient: dump: 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:49.230 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.213+0000 7fe7dffff640 10 monclient: _finish_auth 0 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.213+0000 7fe7dffff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.213+0000 7fe7dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 5 monclient: authenticate success, global_id 5405 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7dffff640 1 -- 172.21.15.190:0/1265084336 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e00a5a90 con 0x7fe7e813e8c0 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7dffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7dffff640 1 -- 172.21.15.190:0/1265084336 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7e00a5c40 con 0x7fe7e813e8c0 2024-08-19T10:41:49.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7dffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 4 set_mon_vals no callback set 2024-08-19T10:41:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7dffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7dffff640 10 monclient: dump: 2024-08-19T10:41:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:49.232 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient: shutdown 2024-08-19T10:41:49.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 -- 172.21.15.190:0/1265084336 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7e813e8c0 msgr2=0x7fe7e8140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 --2- 172.21.15.190:0/1265084336 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7e813e8c0 0x7fe7e8140cb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0009e50 tx=0x7fe7e001c8f0 comp rx=0 tx=0).stop 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 -- 172.21.15.190:0/1265084336 shutdown_connections 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 --2- 172.21.15.190:0/1265084336 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe7e813e8c0 0x7fe7e8140cb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 --2- 172.21.15.190:0/1265084336 >> v2:172.21.15.190:3300/0 conn(0x7fe7e80a5730 0x7fe7e80a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 -- 172.21.15.190:0/1265084336 >> 172.21.15.190:0/1265084336 conn(0x7fe7e80abde0 msgr2=0x7fe7e80ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 -- 172.21.15.190:0/1265084336 shutdown_connections 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 -- 172.21.15.190:0/1265084336 wait complete. 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient: build_initial_monmap 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient: monmap: 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:49.234 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:49.235 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 Processor -- start 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 -- start start 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient: init 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient: _add_conns ranks=[2,3,8,4,0,1,5,6,7] 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe7e80a5730 0x7fe7e8132790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient(hunting): picked mon.noname-b con 0x7fe7e80a5730 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e8139980 0x7fe7e8132cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient(hunting): picked mon.noname-g con 0x7fe7e8139980 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7e813e8c0 0x7fe7e8135630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient(hunting): picked mon.noname-e con 0x7fe7e813e8c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:49.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e8143b90 con 0x7fe7e80a5730 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e8143a10 con 0x7fe7e8139980 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fe7e8143d10 con 0x7fe7e813e8c0 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ef327640 10 monclient(hunting): authenticate will time out at 1154.790053s 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ec8c2640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e8139980 msgr2=0x7fe7e8132cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ec8c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e8139980 0x7fe7e8132cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7e813e8c0 0x7fe7e8135630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7e813e8c0 0x7fe7e8135630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:40742/0 (socket says 172.21.15.190:40742) 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 1 -- 172.21.15.190:0/3760493953 learned_addr learned my addr 172.21.15.190:0/3760493953 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:49.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 10 monclient(hunting): get_auth_request con 0x7fe7e813e8c0 auth_method 0 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed0c3640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe7e80a5730 0x7fe7e8132790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed0c3640 10 monclient(hunting): get_auth_request con 0x7fe7e80a5730 auth_method 0 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed0c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed0c3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed0c3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:49.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed0c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:49.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed0c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:49.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed0c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:49.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 10 monclient(hunting): handle_auth_done global_id 5219 payload 824 2024-08-19T10:41:49.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 1 -- 172.21.15.190:0/3760493953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e8139980 msgr2=0x7fe7e8132cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:41:49.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e8139980 0x7fe7e8132cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:49.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 1 -- 172.21.15.190:0/3760493953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe7e80a5730 msgr2=0x7fe7e8132790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:49.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe7e80a5730 0x7fe7e8132790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 10 monclient: _finish_hunting 0 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 1 monclient: found mon.noname-e 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.217+0000 7fe7ed8c4640 1 -- 172.21.15.190:0/3760493953 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e80a2a30 con 0x7fe7e813e8c0 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ed8c4640 1 -- 172.21.15.190:0/3760493953 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7e813e8c0 msgr2=0x7fe7e8135630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ed8c4640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7e813e8c0 0x7fe7e8135630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00af040 tx=0x7fe7e0019690 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 1 -- 172.21.15.190:0/3760493953 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7e00bc020 con 0x7fe7e813e8c0 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 10 monclient: dump: 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:49.240 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:49.241 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:49.241 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:49.241 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:49.241 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:49.241 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:49.241 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:49.241 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:49.241 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:49.241 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 1 -- 172.21.15.190:0/3760493953 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e0003bb0 con 0x7fe7e813e8c0 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ef327640 5 monclient: authenticate success, global_id 5219 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ef327640 10 monclient: _renew_subs 2024-08-19T10:41:49.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ef327640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ef327640 1 -- 172.21.15.190:0/3760493953 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e80a7150 con 0x7fe7e813e8c0 2024-08-19T10:41:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 1 -- 172.21.15.190:0/3760493953 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe7e0003d60 con 0x7fe7e813e8c0 2024-08-19T10:41:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7df7fe640 4 set_mon_vals no callback set 2024-08-19T10:41:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:41:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 10 monclient: dump: 2024-08-19T10:41:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:49.243 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:49.244 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:49.244 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ef327640 10 monclient: _renew_subs 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ef327640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ef327640 1 -- 172.21.15.190:0/3760493953 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e809d510 con 0x7fe7e813e8c0 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ef327640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ef327640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ef327640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ef327640 1 -- 172.21.15.190:0/3760493953 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e80a5b10 con 0x7fe7e813e8c0 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 1 -- 172.21.15.190:0/3760493953 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe7e00bf020 con 0x7fe7e813e8c0 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe7cc03bd30 0x7fe7cc03e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ddffb640 1 -- 172.21.15.190:0/3760493953 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(110..110 src has 1..110) ==== 6083+0+0 (secure 0 0 0) 0x7fe7e00e9ff0 con 0x7fe7e813e8c0 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ed0c3640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe7cc03bd30 0x7fe7cc03e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.221+0000 7fe7ed0c3640 10 monclient: get_auth_request con 0x7fe7cc03bd30 auth_method 0 2024-08-19T10:41:49.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.225+0000 7fe7ed0c3640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe7cc03bd30 0x7fe7cc03e1f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4007fd0 tx=0x7fe7e4007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:49.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.237+0000 7fe7ddffb640 1 -- 172.21.15.190:0/3760493953 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe7e00c6030 con 0x7fe7e813e8c0 2024-08-19T10:41:49.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.237+0000 7fe7ddffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:49.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.237+0000 7fe7ddffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.409+0000 7fe7ef327640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.11"}] 2024-08-19T10:41:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.409+0000 7fe7ef327640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-08-19T10:41:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.409+0000 7fe7ef327640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:49.409+0000 7fe7ef327640 1 -- 172.21.15.190:0/3760493953 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.11"} v 0) -- 0x7fe7e8137ae0 con 0x7fe7e813e8c0 2024-08-19T10:41:50.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.589+0000 7fe7ddffb640 1 -- 172.21.15.190:0/3760493953 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.11"}]=0 removed item id 11 name 'osd.11' from crush map v111) ==== 125+0+0 (secure 0 0 0) 0x7fe7e0016600 con 0x7fe7e813e8c0 2024-08-19T10:41:50.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.589+0000 7fe7ddffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-08-19T10:41:50.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.589+0000 7fe7ddffb640 10 monclient: _finish_command 2 = system:0 removed item id 11 name 'osd.11' from crush map 2024-08-19T10:41:50.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.705+0000 7fe7ef327640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.11"}] 2024-08-19T10:41:50.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.705+0000 7fe7ef327640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-08-19T10:41:50.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.705+0000 7fe7ef327640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:41:50.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.705+0000 7fe7ef327640 1 -- 172.21.15.190:0/3760493953 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.11"} v 0) -- 0x7fe7e80008e0 con 0x7fe7e813e8c0 2024-08-19T10:41:50.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ddffb640 1 -- 172.21.15.190:0/3760493953 <== mon.8 v2:172.21.15.89:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.11"}]=0 device 'osd.11' does not appear in the crush map v111) ==== 126+0+0 (secure 0 0 0) 0x7fe7e00042d0 con 0x7fe7e813e8c0 2024-08-19T10:41:50.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ddffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.11"}] 2024-08-19T10:41:50.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ddffb640 10 monclient: _finish_command 3 = system:0 device 'osd.11' does not appear in the crush map 2024-08-19T10:41:50.712 INFO:tasks.workunit.client.0.smithi190.stderr:device 'osd.11' does not appear in the crush map 2024-08-19T10:41:50.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 -- 172.21.15.190:0/3760493953 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe7cc03bd30 msgr2=0x7fe7cc03e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:50.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe7cc03bd30 0x7fe7cc03e1f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4007fd0 tx=0x7fe7e4007e40 comp rx=0 tx=0).stop 2024-08-19T10:41:50.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 10 monclient: shutdown 2024-08-19T10:41:50.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:50.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 -- 172.21.15.190:0/3760493953 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7e813e8c0 msgr2=0x7fe7e8135630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7e813e8c0 0x7fe7e8135630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00af040 tx=0x7fe7e0019690 comp rx=0 tx=0).stop 2024-08-19T10:41:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 -- 172.21.15.190:0/3760493953 shutdown_connections 2024-08-19T10:41:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe7cc03bd30 0x7fe7cc03e1f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe7e813e8c0 0x7fe7e8135630 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7e8139980 0x7fe7e8132cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 --2- 172.21.15.190:0/3760493953 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe7e80a5730 0x7fe7e8132790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 -- 172.21.15.190:0/3760493953 >> 172.21.15.190:0/3760493953 conn(0x7fe7e80abde0 msgr2=0x7fe7e80a9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 -- 172.21.15.190:0/3760493953 shutdown_connections 2024-08-19T10:41:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.709+0000 7fe7ef327640 1 -- 172.21.15.190:0/3760493953 wait complete. 2024-08-19T10:41:50.729 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd rm osd.11 2024-08-19T10:41:50.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient: get_monmap_and_config 2024-08-19T10:41:50.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient: build_initial_monmap 2024-08-19T10:41:50.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient: monmap: 2024-08-19T10:41:50.844 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:50.844 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:50.844 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:50.843694+0000 2024-08-19T10:41:50.844 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:50.843694+0000 2024-08-19T10:41:50.844 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:50.844 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:50.844 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:50.845 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:50.845 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:50.845 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:50.845 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:50.845 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:50.845 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:50.846 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:50.846 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:50.846 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:50.846 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:50.846 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:50.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 1 Processor -- start 2024-08-19T10:41:50.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 1 -- start start 2024-08-19T10:41:50.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient: init 2024-08-19T10:41:50.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:50.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient: _add_conns ranks=[2,3,9,4,1,6,10,5,7,0,8] 2024-08-19T10:41:50.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff03c0a3cf0 0x7ff03c0a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:50.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient(hunting): picked mon.noname-b con 0x7ff03c0a3cf0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:50.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c09d7e0 0x7ff03c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:50.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient(hunting): picked mon.noname-g con 0x7ff03c09d7e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:50.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff03c135aa0 0x7ff03c135f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:50.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient(hunting): picked mon.noname-a con 0x7ff03c135aa0 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:50.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:50.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff03c09e180 con 0x7ff03c135aa0 2024-08-19T10:41:50.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:50.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff03c0a4b50 con 0x7ff03c0a3cf0 2024-08-19T10:41:50.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:50.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff03c09e460 con 0x7ff03c09d7e0 2024-08-19T10:41:50.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:50.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff0415ea640 10 monclient(hunting): authenticate will time out at 1156.413581s 2024-08-19T10:41:50.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03a7fc640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c09d7e0 msgr2=0x7ff03c0a4610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:50.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03a7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c09d7e0 0x7ff03c0a4610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff03c0a3cf0 0x7ff03c0a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03b7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff03c135aa0 0x7ff03c135f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff03c0a3cf0 0x7ff03c0a40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:39874/0 (socket says 172.21.15.190:39874) 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 1 -- 172.21.15.190:0/3372050255 learned_addr learned my addr 172.21.15.190:0/3372050255 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 10 monclient(hunting): get_auth_request con 0x7ff03c0a3cf0 auth_method 0 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03b7fe640 10 monclient(hunting): get_auth_request con 0x7ff03c135aa0 auth_method 0 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:50.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 10 monclient(hunting): handle_auth_done global_id 5366 payload 274 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 1 -- 172.21.15.190:0/3372050255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c09d7e0 msgr2=0x7ff03c0a4610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 1 --2- 172.21.15.190:0/3372050255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c09d7e0 0x7ff03c0a4610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 1 -- 172.21.15.190:0/3372050255 >> v2:172.21.15.89:3300/0 conn(0x7ff03c135aa0 msgr2=0x7ff03c135f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 1 --2- 172.21.15.190:0/3372050255 >> v2:172.21.15.89:3300/0 conn(0x7ff03c135aa0 0x7ff03c135f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:50.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 1 monclient: found mon.noname-b 2024-08-19T10:41:50.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:50.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:41:50.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.841+0000 7ff03affd640 1 -- 172.21.15.190:0/3372050255 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c0a5c20 con 0x7ff03c0a3cf0 2024-08-19T10:41:50.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff03affd640 1 -- 172.21.15.190:0/3372050255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff03c0a3cf0 msgr2=0x7ff03c0a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:50.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff03affd640 1 --2- 172.21.15.190:0/3372050255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff03c0a3cf0 0x7ff03c0a40d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff034009e50 tx=0x7ff03401c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8311b38df201a163 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:50.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 1 -- 172.21.15.190:0/3372050255 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0340090d0 con 0x7ff03c0a3cf0 2024-08-19T10:41:50.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:50.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:41:50.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 10 monclient: dump: 2024-08-19T10:41:50.861 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:50.861 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:50.862 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:50.862 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:50.862 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:50.862 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:50.862 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:50.862 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:50.862 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:50.862 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:50.862 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:50.862 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:50.862 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 1 -- 172.21.15.190:0/3372050255 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0340a6950 con 0x7ff03c0a3cf0 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff0415ea640 5 monclient: authenticate success, global_id 5366 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff0415ea640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff0415ea640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff0415ea640 4 set_mon_vals no callback set 2024-08-19T10:41:50.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 1 -- 172.21.15.190:0/3372050255 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0340af480 con 0x7ff03c0a3cf0 2024-08-19T10:41:50.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:50.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:41:50.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.845+0000 7ff039ffb640 10 monclient: dump: 2024-08-19T10:41:50.864 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:50.864 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:50.864 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:50.864 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:50.865 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:50.865 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:50.865 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:50.865 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:50.865 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:50.865 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:50.865 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:50.865 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:50.865 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:50.865 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:50.865 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:50.866 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:50.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient: shutdown 2024-08-19T10:41:50.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:50.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 -- 172.21.15.190:0/3372050255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff03c0a3cf0 msgr2=0x7ff03c0a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:50.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 --2- 172.21.15.190:0/3372050255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff03c0a3cf0 0x7ff03c0a40d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff034009e50 tx=0x7ff03401c910 comp rx=0 tx=0).stop 2024-08-19T10:41:50.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 -- 172.21.15.190:0/3372050255 shutdown_connections 2024-08-19T10:41:50.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 --2- 172.21.15.190:0/3372050255 >> v2:172.21.15.89:3300/0 conn(0x7ff03c135aa0 0x7ff03c135f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:50.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 --2- 172.21.15.190:0/3372050255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff03c09d7e0 0x7ff03c0a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:50.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 --2- 172.21.15.190:0/3372050255 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff03c0a3cf0 0x7ff03c0a40d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:50.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 -- 172.21.15.190:0/3372050255 >> 172.21.15.190:0/3372050255 conn(0x7ff03c0a4de0 msgr2=0x7ff03c0a51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:50.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 -- 172.21.15.190:0/3372050255 shutdown_connections 2024-08-19T10:41:50.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 -- 172.21.15.190:0/3372050255 wait complete. 2024-08-19T10:41:50.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient: build_initial_monmap 2024-08-19T10:41:50.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient: monmap: 2024-08-19T10:41:50.867 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:50.867 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:50.867 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:50.867 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:50.867 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:50.867 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:50.868 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:50.868 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:50.868 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:50.868 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:50.868 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:50.868 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:50.868 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:50.868 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:50.868 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:50.868 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:50.868 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 Processor -- start 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 -- start start 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient: init 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient: _add_conns ranks=[8,1,7,0,6,3,2,4,5] 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff03c09d7e0 0x7ff03c1cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient(hunting): picked mon.noname-e con 0x7ff03c09d7e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff03c0a3cf0 0x7ff03c13d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient(hunting): picked mon.noname-f con 0x7ff03c0a3cf0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff03c135aa0 0x7ff03c13d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient(hunting): picked mon.noname-i con 0x7ff03c135aa0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:50.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c09ce50 con 0x7ff03c0a3cf0 2024-08-19T10:41:50.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:50.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff03c09ccd0 con 0x7ff03c135aa0 2024-08-19T10:41:50.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:50.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff03c140020 con 0x7ff03c09d7e0 2024-08-19T10:41:50.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:50.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff0415ea640 10 monclient(hunting): authenticate will time out at 1156.422151s 2024-08-19T10:41:50.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff03c135aa0 0x7ff03c13d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:50.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff03c0a3cf0 0x7ff03c13d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:50.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff03c135aa0 0x7ff03c13d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:60816/0 (socket says 172.21.15.190:60816) 2024-08-19T10:41:50.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 1 -- 172.21.15.190:0/2405832350 learned_addr learned my addr 172.21.15.190:0/2405832350 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03a7fc640 10 monclient(hunting): get_auth_request con 0x7ff03c0a3cf0 auth_method 0 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 10 monclient(hunting): get_auth_request con 0x7ff03c135aa0 auth_method 0 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:50.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:50.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:50.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03affd640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff03c09d7e0 0x7ff03c1cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:50.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03affd640 10 monclient(hunting): get_auth_request con 0x7ff03c09d7e0 auth_method 0 2024-08-19T10:41:50.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:50.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:50.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:50.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 10 monclient(hunting): handle_auth_done global_id 14578 payload 824 2024-08-19T10:41:50.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 1 -- 172.21.15.190:0/2405832350 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff03c09d7e0 msgr2=0x7ff03c1cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:50.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff03c09d7e0 0x7ff03c1cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:50.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 1 -- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff03c0a3cf0 msgr2=0x7ff03c13d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:50.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff03c0a3cf0 0x7ff03c13d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:50.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:50.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 1 monclient: found mon.noname-i 2024-08-19T10:41:50.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:50.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:50.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03b7fe640 1 -- 172.21.15.190:0/2405832350 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c0a74a0 con 0x7ff03c135aa0 2024-08-19T10:41:50.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.849+0000 7ff03a7fc640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff03c0a3cf0 0x7ff03c13d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:50.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff03b7fe640 1 -- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff03c135aa0 msgr2=0x7ff03c13d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:50.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff03b7fe640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff03c135aa0 0x7ff03c13d620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00e7f0 tx=0x7ff02c00ecb0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 1 -- 172.21.15.190:0/2405832350 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff02c017070 con 0x7ff03c135aa0 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 10 monclient: dump: 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:50.876 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0415ea640 5 monclient: authenticate success, global_id 14578 2024-08-19T10:41:50.877 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0415ea640 10 monclient: _renew_subs 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 1 -- 172.21.15.190:0/2405832350 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02c013470 con 0x7ff03c135aa0 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0415ea640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0415ea640 1 -- 172.21.15.190:0/2405832350 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c13db60 con 0x7ff03c135aa0 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 1 -- 172.21.15.190:0/2405832350 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff02c00eea0 con 0x7ff03c135aa0 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 10 monclient: dump: 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:50.878 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:50.879 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff038ff9640 4 set_mon_vals no callback set 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0415ea640 10 monclient: _renew_subs 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0415ea640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0415ea640 1 -- 172.21.15.190:0/2405832350 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c13aba0 con 0x7ff03c135aa0 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0415ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0415ea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0415ea640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0415ea640 1 -- 172.21.15.190:0/2405832350 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03c1393a0 con 0x7ff03c135aa0 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 1 -- 172.21.15.190:0/2405832350 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff02c01a070 con 0x7ff03c135aa0 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff01c03be00 0x7ff01c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff0337fe640 1 -- 172.21.15.190:0/2405832350 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(111..111 src has 1..111) ==== 6061+0+0 (secure 0 0 0) 0x7ff02c00f040 con 0x7ff03c135aa0 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff03affd640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff01c03be00 0x7ff01c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:50.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.853+0000 7ff03affd640 10 monclient: get_auth_request con 0x7ff01c03be00 auth_method 0 2024-08-19T10:41:50.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.857+0000 7ff03affd640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff01c03be00 0x7ff01c03e2c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff0340ae000 tx=0x7ff0340a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:50.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.857+0000 7ff0337fe640 1 -- 172.21.15.190:0/2405832350 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff02c01a360 con 0x7ff03c135aa0 2024-08-19T10:41:50.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.857+0000 7ff0337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:50.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:50.857+0000 7ff0337fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:51.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:51.041+0000 7ff0415ea640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-08-19T10:41:51.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:51.041+0000 7ff0415ea640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-08-19T10:41:51.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:51.041+0000 7ff0415ea640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:51.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:51.041+0000 7ff0415ea640 1 -- 172.21.15.190:0/2405832350 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.11"]} v 0) -- 0x7ff03c06e4d0 con 0x7ff03c135aa0 2024-08-19T10:41:51.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:41:51.044+0000 7f16cc065640 -1 mon.a@0(leader).osd e111 definitely_dead 0 2024-08-19T10:41:52.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.197+0000 7ff0337fe640 1 -- 172.21.15.190:0/2405832350 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.11"]}]=0 removed osd.11 v112) ==== 87+0+0 (secure 0 0 0) 0x7ff02c036090 con 0x7ff03c135aa0 2024-08-19T10:41:52.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.197+0000 7ff0337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-08-19T10:41:52.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.197+0000 7ff0337fe640 10 monclient: _finish_command 2 = system:0 removed osd.11 2024-08-19T10:41:52.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.317+0000 7ff0415ea640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-08-19T10:41:52.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.317+0000 7ff0415ea640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-08-19T10:41:52.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.317+0000 7ff0415ea640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:41:52.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.317+0000 7ff0415ea640 1 -- 172.21.15.190:0/2405832350 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.11"]} v 0) -- 0x7ff03c13a540 con 0x7ff03c135aa0 2024-08-19T10:41:52.322 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:41:52.320+0000 7f16cc065640 -1 mon.a@0(leader).osd e112 definitely_dead 0 2024-08-19T10:41:52.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0337fe640 1 -- 172.21.15.190:0/2405832350 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.11"]}]=0 osd.11 does not exist. v112) ==== 96+0+0 (secure 0 0 0) 0x7ff02c04d580 con 0x7ff03c135aa0 2024-08-19T10:41:52.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0337fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.11"]}] 2024-08-19T10:41:52.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0337fe640 10 monclient: _finish_command 3 = system:0 osd.11 does not exist. 2024-08-19T10:41:52.323 INFO:tasks.workunit.client.0.smithi190.stderr:osd.11 does not exist. 2024-08-19T10:41:52.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 -- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff01c03be00 msgr2=0x7ff01c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:52.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff01c03be00 0x7ff01c03e2c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff0340ae000 tx=0x7ff0340a7000 comp rx=0 tx=0).stop 2024-08-19T10:41:52.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 10 monclient: shutdown 2024-08-19T10:41:52.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:52.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 -- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff03c135aa0 msgr2=0x7ff03c13d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:52.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff03c135aa0 0x7ff03c13d620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00e7f0 tx=0x7ff02c00ecb0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 -- 172.21.15.190:0/2405832350 shutdown_connections 2024-08-19T10:41:52.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff03c135aa0 0x7ff03c13d620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff03c0a3cf0 0x7ff03c13d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff01c03be00 0x7ff01c03e2c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 --2- 172.21.15.190:0/2405832350 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff03c09d7e0 0x7ff03c1cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 -- 172.21.15.190:0/2405832350 >> 172.21.15.190:0/2405832350 conn(0x7ff03c0a4de0 msgr2=0x7ff03c131d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:52.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 -- 172.21.15.190:0/2405832350 shutdown_connections 2024-08-19T10:41:52.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.321+0000 7ff0415ea640 1 -- 172.21.15.190:0/2405832350 wait complete. 2024-08-19T10:41:52.342 INFO:tasks.workunit.client.0.smithi190.stderr:++ ceph osd create 2024-08-19T10:41:52.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient: get_monmap_and_config 2024-08-19T10:41:52.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient: build_initial_monmap 2024-08-19T10:41:52.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient: monmap: 2024-08-19T10:41:52.457 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:52.457 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:52.457848+0000 2024-08-19T10:41:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:52.457848+0000 2024-08-19T10:41:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:52.458 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:52.459 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:52.459 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 1 Processor -- start 2024-08-19T10:41:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 1 -- start start 2024-08-19T10:41:52.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient: init 2024-08-19T10:41:52.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:52.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient: _add_conns ranks=[3,8,7,1,2,6,9,5,0,4,10] 2024-08-19T10:41:52.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b440a5340 0x7f8b441396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:52.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient(hunting): picked mon.noname-g con 0x7f8b440a5340 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:52.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b44139be0 0x7f8b44139fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:52.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient(hunting): picked mon.noname-e con 0x7f8b44139be0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:41:52.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b440a68e0 0x7f8b440a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:52.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient(hunting): picked mon.noname-i con 0x7f8b440a68e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:52.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:52.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8b440a2f30 con 0x7f8b440a5340 2024-08-19T10:41:52.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:52.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8b440a3500 con 0x7f8b440a68e0 2024-08-19T10:41:52.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:52.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f8b4413a500 con 0x7f8b44139be0 2024-08-19T10:41:52.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:52.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4b87f640 10 monclient(hunting): authenticate will time out at 1158.027786s 2024-08-19T10:41:52.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4961b640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b440a5340 msgr2=0x7f8b441396a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:52.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.453+0000 7f8b4961b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b440a5340 0x7f8b441396a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:52.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b440a68e0 0x7f8b440a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b440a68e0 0x7f8b440a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:60832/0 (socket says 172.21.15.190:60832) 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 1 -- 172.21.15.190:0/2018696224 learned_addr learned my addr 172.21.15.190:0/2018696224 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 10 monclient(hunting): get_auth_request con 0x7f8b440a68e0 auth_method 0 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b48e1a640 1 --2- 172.21.15.190:0/2018696224 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b44139be0 0x7f8b44139fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b48e1a640 10 monclient(hunting): get_auth_request con 0x7f8b44139be0 auth_method 0 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b48e1a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b48e1a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b48e1a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:52.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b48e1a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b48e1a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b48e1a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 10 monclient(hunting): handle_auth_done global_id 14587 payload 274 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 1 -- 172.21.15.190:0/2018696224 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b44139be0 msgr2=0x7f8b44139fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 1 --2- 172.21.15.190:0/2018696224 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b44139be0 0x7f8b44139fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 1 -- 172.21.15.190:0/2018696224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b440a5340 msgr2=0x7f8b441396a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 1 --2- 172.21.15.190:0/2018696224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b440a5340 0x7f8b441396a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 10 monclient: _finish_hunting 0 2024-08-19T10:41:52.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 1 monclient: found mon.noname-i 2024-08-19T10:41:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:41:52.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.457+0000 7f8b49e1c640 1 -- 172.21.15.190:0/2018696224 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b440a8610 con 0x7f8b440a68e0 2024-08-19T10:41:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 1 -- 172.21.15.190:0/2018696224 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b440a68e0 msgr2=0x7f8b440a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 1 --2- 172.21.15.190:0/2018696224 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b440a68e0 0x7f8b440a6d20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0098a0 tx=0x7f8b3c01c460 comp rx=0 tx=0).ready entity=mon.7 client_cookie=433b755adc4052d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 1 -- 172.21.15.190:0/2018696224 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b3c0b0070 con 0x7f8b440a68e0 2024-08-19T10:41:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:41:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 10 monclient: dump: 2024-08-19T10:41:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:52.473 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:52.474 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 10 monclient: _finish_auth 0 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 1 -- 172.21.15.190:0/2018696224 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3c0aa810 con 0x7f8b440a68e0 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 1 -- 172.21.15.190:0/2018696224 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b3c0aae30 con 0x7f8b440a68e0 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 5 monclient: authenticate success, global_id 14587 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 4 set_mon_vals no callback set 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:52.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b3bfff640 10 monclient: dump: 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:52.476 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient: shutdown 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 -- 172.21.15.190:0/2018696224 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b440a68e0 msgr2=0x7f8b440a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 --2- 172.21.15.190:0/2018696224 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b440a68e0 0x7f8b440a6d20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0098a0 tx=0x7f8b3c01c460 comp rx=0 tx=0).stop 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 -- 172.21.15.190:0/2018696224 shutdown_connections 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 --2- 172.21.15.190:0/2018696224 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b44139be0 0x7f8b44139fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 --2- 172.21.15.190:0/2018696224 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b440a68e0 0x7f8b440a6d20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 --2- 172.21.15.190:0/2018696224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b440a5340 0x7f8b441396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 -- 172.21.15.190:0/2018696224 >> 172.21.15.190:0/2018696224 conn(0x7f8b44074bd0 msgr2=0x7f8b440abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:52.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 -- 172.21.15.190:0/2018696224 shutdown_connections 2024-08-19T10:41:52.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 -- 172.21.15.190:0/2018696224 wait complete. 2024-08-19T10:41:52.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient: build_initial_monmap 2024-08-19T10:41:52.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient: monmap: 2024-08-19T10:41:52.478 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:52.478 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:52.479 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:52.480 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 Processor -- start 2024-08-19T10:41:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 -- start start 2024-08-19T10:41:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient: init 2024-08-19T10:41:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient: _add_conns ranks=[2,0,4,3,6,1,5,7,8] 2024-08-19T10:41:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b440a5340 0x7f8b44131dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient(hunting): picked mon.noname-b con 0x7f8b440a5340 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440a68e0 0x7f8b4413cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:52.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient(hunting): picked mon.noname-a con 0x7f8b440a68e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b44139be0 0x7f8b4413d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient(hunting): picked mon.noname-c con 0x7f8b44139be0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8b44143320 con 0x7f8b440a68e0 2024-08-19T10:41:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8b440a8360 con 0x7f8b440a5340 2024-08-19T10:41:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:52.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f8b441434a0 con 0x7f8b44139be0 2024-08-19T10:41:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4b87f640 10 monclient(hunting): authenticate will time out at 1158.035051s 2024-08-19T10:41:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b48e1a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440a68e0 0x7f8b4413cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b44139be0 0x7f8b4413d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4961b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b440a5340 0x7f8b44131dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b44139be0 0x7f8b4413d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:44740/0 (socket says 172.21.15.190:44740) 2024-08-19T10:41:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 1 -- 172.21.15.190:0/367096135 learned_addr learned my addr 172.21.15.190:0/367096135 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 10 monclient(hunting): get_auth_request con 0x7f8b44139be0 auth_method 0 2024-08-19T10:41:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b48e1a640 10 monclient(hunting): get_auth_request con 0x7f8b440a68e0 auth_method 0 2024-08-19T10:41:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b48e1a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b48e1a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b48e1a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4961b640 10 monclient(hunting): get_auth_request con 0x7f8b440a5340 auth_method 0 2024-08-19T10:41:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4961b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4961b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4961b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:52.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b48e1a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:52.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b48e1a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:52.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b48e1a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:52.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4961b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:52.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4961b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:52.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b4961b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:52.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:52.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:52.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.461+0000 7f8b49e1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:52.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 10 monclient(hunting): handle_auth_done global_id 5436 payload 824 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 1 -- 172.21.15.190:0/367096135 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b44139be0 msgr2=0x7f8b4413d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b44139be0 0x7f8b4413d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 1 -- 172.21.15.190:0/367096135 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b440a5340 msgr2=0x7f8b44131dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b440a5340 0x7f8b44131dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 10 monclient: _finish_hunting 0 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 1 monclient: found mon.noname-a 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 1 -- 172.21.15.190:0/367096135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4409d530 con 0x7f8b440a68e0 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 1 -- 172.21.15.190:0/367096135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440a68e0 msgr2=0x7f8b4413cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b48e1a640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440a68e0 0x7f8b4413cf60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400e7f0 tx=0x7f8b3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:52.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 1 -- 172.21.15.190:0/367096135 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b34017070 con 0x7f8b440a68e0 2024-08-19T10:41:52.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:52.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:41:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 10 monclient: dump: 2024-08-19T10:41:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b4b87f640 5 monclient: authenticate success, global_id 5436 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b4b87f640 10 monclient: _renew_subs 2024-08-19T10:41:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b4b87f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b4b87f640 1 -- 172.21.15.190:0/367096135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b440ab390 con 0x7f8b440a68e0 2024-08-19T10:41:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 1 -- 172.21.15.190:0/367096135 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3400cb70 con 0x7f8b440a68e0 2024-08-19T10:41:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b3b7fe640 4 set_mon_vals no callback set 2024-08-19T10:41:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b4b87f640 10 monclient: _renew_subs 2024-08-19T10:41:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 1 -- 172.21.15.190:0/367096135 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b3400cd20 con 0x7f8b440a68e0 2024-08-19T10:41:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b4b87f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b4b87f640 1 -- 172.21.15.190:0/367096135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4409d250 con 0x7f8b440a68e0 2024-08-19T10:41:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:41:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b39ffb640 10 monclient: dump: 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:52.492 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b4b87f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b4b87f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b4b87f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.465+0000 7f8b4b87f640 1 -- 172.21.15.190:0/367096135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b440a6d20 con 0x7f8b440a68e0 2024-08-19T10:41:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.469+0000 7f8b39ffb640 1 -- 172.21.15.190:0/367096135 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8b3401a070 con 0x7f8b440a68e0 2024-08-19T10:41:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.469+0000 7f8b39ffb640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b2803be00 0x7f8b2803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.469+0000 7f8b39ffb640 1 -- 172.21.15.190:0/367096135 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6080+0+0 (secure 0 0 0) 0x7f8b3400f040 con 0x7f8b440a68e0 2024-08-19T10:41:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.469+0000 7f8b4961b640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b2803be00 0x7f8b2803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.469+0000 7f8b4961b640 10 monclient: get_auth_request con 0x7f8b2803be00 auth_method 0 2024-08-19T10:41:52.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.469+0000 7f8b4961b640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b2803be00 0x7f8b2803e2c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8b40003d10 tx=0x7f8b4000d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:52.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.481+0000 7f8b39ffb640 1 -- 172.21.15.190:0/367096135 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8b34021070 con 0x7f8b440a68e0 2024-08-19T10:41:52.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.481+0000 7f8b39ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:52.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.481+0000 7f8b39ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:52.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.661+0000 7f8b4b87f640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-08-19T10:41:52.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.661+0000 7f8b4b87f640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-08-19T10:41:52.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.661+0000 7f8b4b87f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:52.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:52.661+0000 7f8b4b87f640 1 -- 172.21.15.190:0/367096135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f8b440a2390 con 0x7f8b440a68e0 2024-08-19T10:41:53.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:53.201+0000 7f8b39ffb640 1 -- 172.21.15.190:0/367096135 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v113) ==== 58+0+3 (secure 0 0 0) 0x7f8b3404ccf0 con 0x7f8b440a68e0 2024-08-19T10:41:53.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:53.201+0000 7f8b39ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-08-19T10:41:53.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:53.201+0000 7f8b39ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:53.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:53.317+0000 7f8b4b87f640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-08-19T10:41:53.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:53.317+0000 7f8b4b87f640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-08-19T10:41:53.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:53.317+0000 7f8b4b87f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:41:53.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:53.317+0000 7f8b4b87f640 1 -- 172.21.15.190:0/367096135 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f8b4413f5b0 con 0x7f8b440a68e0 2024-08-19T10:41:54.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.213+0000 7f8b39ffb640 1 -- 172.21.15.190:0/367096135 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v114) ==== 58+0+3 (secure 0 0 0) 0x7f8b3404d070 con 0x7f8b440a68e0 2024-08-19T10:41:54.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.213+0000 7f8b39ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-08-19T10:41:54.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.213+0000 7f8b39ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:54.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 -- 172.21.15.190:0/367096135 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b2803be00 msgr2=0x7f8b2803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:54.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b2803be00 0x7f8b2803e2c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8b40003d10 tx=0x7f8b4000d040 comp rx=0 tx=0).stop 2024-08-19T10:41:54.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 10 monclient: shutdown 2024-08-19T10:41:54.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:54.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 -- 172.21.15.190:0/367096135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440a68e0 msgr2=0x7f8b4413cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:54.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440a68e0 0x7f8b4413cf60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400e7f0 tx=0x7f8b3400ecb0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 -- 172.21.15.190:0/367096135 shutdown_connections 2024-08-19T10:41:54.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b2803be00 0x7f8b2803e2c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8b44139be0 0x7f8b4413d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b440a68e0 0x7f8b4413cf60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 --2- 172.21.15.190:0/367096135 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8b440a5340 0x7f8b44131dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 -- 172.21.15.190:0/367096135 >> 172.21.15.190:0/367096135 conn(0x7f8b44074bd0 msgr2=0x7f8b440a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:54.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 -- 172.21.15.190:0/367096135 shutdown_connections 2024-08-19T10:41:54.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.217+0000 7f8b4b87f640 1 -- 172.21.15.190:0/367096135 wait complete. 2024-08-19T10:41:54.241 INFO:tasks.workunit.client.0.smithi190.stderr:+ o4=12 2024-08-19T10:41:54.241 INFO:tasks.workunit.client.0.smithi190.stderr:++ ceph osd create 2024-08-19T10:41:54.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient: get_monmap_and_config 2024-08-19T10:41:54.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient: build_initial_monmap 2024-08-19T10:41:54.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient: monmap: 2024-08-19T10:41:54.349 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:54.349 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:54.349 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:54.349773+0000 2024-08-19T10:41:54.349 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:54.349773+0000 2024-08-19T10:41:54.349 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:54.350 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:54.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 1 Processor -- start 2024-08-19T10:41:54.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 1 -- start start 2024-08-19T10:41:54.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient: init 2024-08-19T10:41:54.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:54.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient: _add_conns ranks=[10,4,6,8,2,5,3,7,1,0,9] 2024-08-19T10:41:54.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62280a3b90 0x7f62280a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:54.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient(hunting): picked mon.noname-f con 0x7f62280a3b90 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:54.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62280a44b0 0x7f62280a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:54.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient(hunting): picked mon.noname-c con 0x7f62280a44b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:54.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6228136570 0x7f6228136a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:54.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient(hunting): picked mon.noname-d con 0x7f6228136570 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:54.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f62280a04b0 con 0x7f62280a3b90 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f62280a1ba0 con 0x7f62280a44b0 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f62280a1e80 con 0x7f6228136570 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f6230524640 10 monclient(hunting): authenticate will time out at 1159.919592s 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f622e2c0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62280a3b90 0x7f62280a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f622e2c0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62280a3b90 0x7f62280a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:58180/0 (socket says 172.21.15.190:58180) 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f622e2c0640 1 -- 172.21.15.190:0/2175463474 learned_addr learned my addr 172.21.15.190:0/2175463474 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f622e2c0640 10 monclient(hunting): get_auth_request con 0x7f62280a3b90 auth_method 0 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f622e2c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:54.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f622e2c0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:54.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f622e2c0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:54.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f622e2c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:54.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f622e2c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:54.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.345+0000 7f622e2c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:54.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622e2c0640 10 monclient(hunting): handle_auth_done global_id 14671 payload 274 2024-08-19T10:41:54.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622e2c0640 1 -- 172.21.15.190:0/2175463474 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6228136570 msgr2=0x7f6228136a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:54.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622eac1640 1 --2- 172.21.15.190:0/2175463474 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6228136570 0x7f6228136a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:54.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622dabf640 1 --2- 172.21.15.190:0/2175463474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62280a44b0 0x7f62280a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:54.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622e2c0640 1 --2- 172.21.15.190:0/2175463474 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6228136570 0x7f6228136a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622e2c0640 1 -- 172.21.15.190:0/2175463474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62280a44b0 msgr2=0x7f62280a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:54.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622e2c0640 1 --2- 172.21.15.190:0/2175463474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62280a44b0 0x7f62280a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622e2c0640 10 monclient: _finish_hunting 0 2024-08-19T10:41:54.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622e2c0640 1 monclient: found mon.noname-f 2024-08-19T10:41:54.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622e2c0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:54.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622e2c0640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:54.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622e2c0640 1 -- 172.21.15.190:0/2175463474 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622809fe50 con 0x7f62280a3b90 2024-08-19T10:41:54.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622dabf640 10 monclient: get_auth_request con 0x7f62280a44b0 auth_method 0 2024-08-19T10:41:54.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.349+0000 7f622dabf640 1 --2- 172.21.15.190:0/2175463474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62280a44b0 0x7f62280a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622e2c0640 1 -- 172.21.15.190:0/2175463474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62280a3b90 msgr2=0x7f62280a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622e2c0640 1 --2- 172.21.15.190:0/2175463474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62280a3b90 0x7f62280a3f70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6224009870 tx=0x7f622401c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74e396f8c00ccf1f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 1 -- 172.21.15.190:0/2175463474 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62240b0070 con 0x7f62280a3b90 2024-08-19T10:41:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 10 monclient: dump: 2024-08-19T10:41:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:54.362 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 10 monclient: _finish_auth 0 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 10 monclient: _check_auth_tickets 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 5 monclient: authenticate success, global_id 14671 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 1 -- 172.21.15.190:0/2175463474 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62240aa810 con 0x7f62280a3b90 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 4 set_mon_vals no callback set 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 1 -- 172.21.15.190:0/2175463474 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62240a93e0 con 0x7f62280a3b90 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f622d2be640 10 monclient: dump: 2024-08-19T10:41:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 10 monclient: shutdown 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 1 -- 172.21.15.190:0/2175463474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62280a3b90 msgr2=0x7f62280a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 1 --2- 172.21.15.190:0/2175463474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62280a3b90 0x7f62280a3f70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6224009870 tx=0x7f622401c4b0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 1 -- 172.21.15.190:0/2175463474 shutdown_connections 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 1 --2- 172.21.15.190:0/2175463474 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6228136570 0x7f6228136a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 1 --2- 172.21.15.190:0/2175463474 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f62280a44b0 0x7f62280a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 1 --2- 172.21.15.190:0/2175463474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62280a3b90 0x7f62280a3f70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 1 -- 172.21.15.190:0/2175463474 >> 172.21.15.190:0/2175463474 conn(0x7f622806dde0 msgr2=0x7f622806e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 1 -- 172.21.15.190:0/2175463474 shutdown_connections 2024-08-19T10:41:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 1 -- 172.21.15.190:0/2175463474 wait complete. 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 10 monclient: build_initial_monmap 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 10 monclient: monmap: 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 1 Processor -- start 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 1 -- start start 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.353+0000 7f6230524640 10 monclient: init 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient: _add_conns ranks=[2,5,6,4,1,3,7,8,0] 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62280a3b90 0x7f622813b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient(hunting): picked mon.noname-b con 0x7f62280a3b90 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62280a44b0 0x7f6228135700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient(hunting): picked mon.noname-h con 0x7f62280a44b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6228136570 0x7f6228135c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient(hunting): picked mon.noname-d con 0x7f6228136570 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f62280a0b00 con 0x7f62280a3b90 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f62280a2c80 con 0x7f62280a44b0 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f622813ffe0 con 0x7f6228136570 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient(hunting): authenticate will time out at 1159.928345s 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62280a44b0 0x7f6228135700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62280a44b0 0x7f6228135700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:59662/0 (socket says 172.21.15.190:59662) 2024-08-19T10:41:54.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 1 -- 172.21.15.190:0/1150533078 learned_addr learned my addr 172.21.15.190:0/1150533078 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:54.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 10 monclient(hunting): get_auth_request con 0x7f62280a44b0 auth_method 0 2024-08-19T10:41:54.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:54.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:54.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:54.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:54.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:54.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622e2c0640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62280a3b90 0x7f622813b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622eac1640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6228136570 0x7f6228135c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 10 monclient(hunting): handle_auth_done global_id 5414 payload 824 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 1 -- 172.21.15.190:0/1150533078 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6228136570 msgr2=0x7f6228135c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6228136570 0x7f6228135c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 1 -- 172.21.15.190:0/1150533078 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62280a3b90 msgr2=0x7f622813b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62280a3b90 0x7f622813b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 10 monclient: _finish_hunting 0 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 1 monclient: found mon.noname-h 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 1 -- 172.21.15.190:0/1150533078 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62280a4bf0 con 0x7f62280a44b0 2024-08-19T10:41:54.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622e2c0640 10 monclient: get_auth_request con 0x7f62280a3b90 auth_method 0 2024-08-19T10:41:54.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622eac1640 10 monclient: get_auth_request con 0x7f6228136570 auth_method 0 2024-08-19T10:41:54.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622e2c0640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62280a3b90 0x7f622813b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:54.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622eac1640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6228136570 0x7f6228135c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:54.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 1 -- 172.21.15.190:0/1150533078 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62280a44b0 msgr2=0x7f6228135700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:54.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622dabf640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62280a44b0 0x7f6228135700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f621800c7f0 tx=0x7f621800ccb0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:54.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 1 -- 172.21.15.190:0/1150533078 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6218015070 con 0x7f62280a44b0 2024-08-19T10:41:54.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:54.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:54.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 10 monclient: dump: 2024-08-19T10:41:54.373 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 10 monclient: _finish_auth 0 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 1 -- 172.21.15.190:0/1150533078 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62180113f0 con 0x7f62280a44b0 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 5 monclient: authenticate success, global_id 5414 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient: _renew_subs 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 1 -- 172.21.15.190:0/1150533078 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622813c120 con 0x7f62280a44b0 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 1 -- 172.21.15.190:0/1150533078 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6218011a10 con 0x7f62280a44b0 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f621effd640 10 monclient: dump: 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f622cabd640 4 set_mon_vals no callback set 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient: _renew_subs 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.357+0000 7f6230524640 1 -- 172.21.15.190:0/1150533078 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f62281cf0f0 con 0x7f62280a44b0 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.361+0000 7f6230524640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.361+0000 7f6230524640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.361+0000 7f6230524640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.361+0000 7f6230524640 1 -- 172.21.15.190:0/1150533078 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f622806e5d0 con 0x7f62280a44b0 2024-08-19T10:41:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.361+0000 7f621effd640 1 -- 172.21.15.190:0/1150533078 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6218018070 con 0x7f62280a44b0 2024-08-19T10:41:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.361+0000 7f621effd640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f620c03be30 0x7f620c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.361+0000 7f621effd640 1 -- 172.21.15.190:0/1150533078 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(114..114 src has 1..114) ==== 6199+0+0 (secure 0 0 0) 0x7f621804ec50 con 0x7f62280a44b0 2024-08-19T10:41:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.361+0000 7f622e2c0640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f620c03be30 0x7f620c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.361+0000 7f622e2c0640 10 monclient: get_auth_request con 0x7f620c03be30 auth_method 0 2024-08-19T10:41:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.361+0000 7f622e2c0640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f620c03be30 0x7f620c03e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f62240ab000 tx=0x7f62240a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.365+0000 7f621effd640 1 -- 172.21.15.190:0/1150533078 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f622806e5d0 con 0x7f62280a44b0 2024-08-19T10:41:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.365+0000 7f621effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.365+0000 7f621effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:54.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.545+0000 7f6230524640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-08-19T10:41:54.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.545+0000 7f6230524640 10 monclient: _send_command 2 [{"prefix": "osd create"}] 2024-08-19T10:41:54.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.545+0000 7f6230524640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:54.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:54.545+0000 7f6230524640 1 -- 172.21.15.190:0/1150533078 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f62280a3f70 con 0x7f62280a44b0 2024-08-19T10:41:55.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:55.245+0000 7f621effd640 1 -- 172.21.15.190:0/1150533078 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd create"}]=0 v115) ==== 58+0+3 (secure 0 0 0) 0x7f62180193b0 con 0x7f62280a44b0 2024-08-19T10:41:55.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:55.245+0000 7f621effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create"}] 2024-08-19T10:41:55.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:55.245+0000 7f621effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:55.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:55.361+0000 7f6230524640 10 start_mon_command cmd=[{"prefix": "osd create"}] 2024-08-19T10:41:55.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:55.361+0000 7f6230524640 10 monclient: _send_command 3 [{"prefix": "osd create"}] 2024-08-19T10:41:55.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:55.361+0000 7f6230524640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:55.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:55.361+0000 7f6230524640 1 -- 172.21.15.190:0/1150533078 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd create"} v 0) -- 0x7f6228076ed0 con 0x7f62280a44b0 2024-08-19T10:41:56.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.249+0000 7f621effd640 1 -- 172.21.15.190:0/1150533078 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd create"}]=0 v116) ==== 58+0+3 (secure 0 0 0) 0x7f6218011d20 con 0x7f62280a44b0 2024-08-19T10:41:56.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.249+0000 7f621effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create"}] 2024-08-19T10:41:56.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.249+0000 7f621effd640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:56.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 -- 172.21.15.190:0/1150533078 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f620c03be30 msgr2=0x7f620c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:56.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f620c03be30 0x7f620c03e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f62240ab000 tx=0x7f62240a6000 comp rx=0 tx=0).stop 2024-08-19T10:41:56.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 10 monclient: shutdown 2024-08-19T10:41:56.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:56.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 -- 172.21.15.190:0/1150533078 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62280a44b0 msgr2=0x7f6228135700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:56.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62280a44b0 0x7f6228135700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f621800c7f0 tx=0x7f621800ccb0 comp rx=0 tx=0).stop 2024-08-19T10:41:56.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 -- 172.21.15.190:0/1150533078 shutdown_connections 2024-08-19T10:41:56.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f620c03be30 0x7f620c03e2f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:56.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6228136570 0x7f6228135c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:56.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f62280a44b0 0x7f6228135700 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:56.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 --2- 172.21.15.190:0/1150533078 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62280a3b90 0x7f622813b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:56.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 -- 172.21.15.190:0/1150533078 >> 172.21.15.190:0/1150533078 conn(0x7f622806dde0 msgr2=0x7f62280a7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:56.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 -- 172.21.15.190:0/1150533078 shutdown_connections 2024-08-19T10:41:56.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.253+0000 7f6230524640 1 -- 172.21.15.190:0/1150533078 wait complete. 2024-08-19T10:41:56.281 INFO:tasks.workunit.client.0.smithi190.stderr:+ o5=14 2024-08-19T10:41:56.282 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add 12 123 root=default host=foobaz 2024-08-19T10:41:56.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.401+0000 7f44650a8640 10 monclient: get_monmap_and_config 2024-08-19T10:41:56.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.401+0000 7f44650a8640 10 monclient: build_initial_monmap 2024-08-19T10:41:56.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.401+0000 7f44650a8640 10 monclient: monmap: 2024-08-19T10:41:56.405 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:56.406067+0000 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:56.406067+0000 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:56.406 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:56.407 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:56.407 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:56.407 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:56.407 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:56.407 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:56.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.401+0000 7f44650a8640 1 Processor -- start 2024-08-19T10:41:56.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 1 -- start start 2024-08-19T10:41:56.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 10 monclient: init 2024-08-19T10:41:56.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:56.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 10 monclient: _add_conns ranks=[0,10,4,3,5,7,8,2,9,1,6] 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f44600a2280 addr v1:172.21.15.89:6789/0 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4460069910 0x7f44600a2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 10 monclient(hunting): picked mon.noname-f con 0x7f4460069910 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44600a1810 0x7f446006cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 10 monclient(hunting): picked mon.noname-c con 0x7f44600a1810 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44600a0580 con 0x7f44600a2280 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f446006dd90 con 0x7f4460069910 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f44600a0860 con 0x7f44600a1810 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:56.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f44650a8640 10 monclient(hunting): authenticate will time out at 1161.976716s 2024-08-19T10:41:56.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4460069910 0x7f44600a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:56.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4460069910 0x7f44600a2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:53804/0 (socket says 172.21.15.190:53804) 2024-08-19T10:41:56.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 1 -- 172.21.15.190:0/1606802536 learned_addr learned my addr 172.21.15.190:0/1606802536 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:56.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 10 monclient(hunting): get_auth_request con 0x7f4460069910 auth_method 0 2024-08-19T10:41:56.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:56.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:56.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:56.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:56.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:56.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:56.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445f59e640 1 --2- 172.21.15.190:0/1606802536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44600a1810 0x7f446006cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445f59e640 10 monclient(hunting): get_auth_request con 0x7f44600a1810 auth_method 0 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445f59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445f59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 10 monclient(hunting): handle_auth_done global_id 14680 payload 274 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 1 -- 172.21.15.190:0/1606802536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44600a1810 msgr2=0x7f446006cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 1 --2- 172.21.15.190:0/1606802536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44600a1810 0x7f446006cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 1 -- 172.21.15.190:0/1606802536 >> v1:172.21.15.89:6789/0 conn(0x7f44600a2280 legacy=0x7f446006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 1 monclient: found mon.noname-f 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:56.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.405+0000 7f445e59c640 1 -- 172.21.15.190:0/1606802536 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44600a0b40 con 0x7f4460069910 2024-08-19T10:41:56.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.409+0000 7f445e59c640 1 -- 172.21.15.190:0/1606802536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4460069910 msgr2=0x7f44600a2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:56.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.409+0000 7f445e59c640 1 --2- 172.21.15.190:0/1606802536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4460069910 0x7f44600a2660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4454009870 tx=0x7f445401c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71660024f990b642 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.409+0000 7f445dd9b640 1 -- 172.21.15.190:0/1606802536 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44540b1070 con 0x7f4460069910 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.409+0000 7f445dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445dd9b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445dd9b640 10 monclient: dump: 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:56.415 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445dd9b640 10 monclient: _finish_auth 0 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445dd9b640 10 monclient: _check_auth_tickets 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 5 monclient: authenticate success, global_id 14680 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445dd9b640 1 -- 172.21.15.190:0/1606802536 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446006d730 con 0x7f4460069910 2024-08-19T10:41:56.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445dd9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445dd9b640 1 -- 172.21.15.190:0/1606802536 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44540ab830 con 0x7f4460069910 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 4 set_mon_vals no callback set 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445dd9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445dd9b640 10 monclient: dump: 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:56.417 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:56.418 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:56.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient: shutdown 2024-08-19T10:41:56.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:56.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 -- 172.21.15.190:0/1606802536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4460069910 msgr2=0x7f44600a2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:56.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 --2- 172.21.15.190:0/1606802536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4460069910 0x7f44600a2660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4454009870 tx=0x7f445401c420 comp rx=0 tx=0).stop 2024-08-19T10:41:56.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 -- 172.21.15.190:0/1606802536 shutdown_connections 2024-08-19T10:41:56.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 --2- 172.21.15.190:0/1606802536 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44600a1810 0x7f446006cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:56.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 --2- 172.21.15.190:0/1606802536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4460069910 0x7f44600a2660 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:56.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 -- 172.21.15.190:0/1606802536 >> 172.21.15.190:0/1606802536 conn(0x7f44600a84c0 msgr2=0x7f44600a88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 -- 172.21.15.190:0/1606802536 shutdown_connections 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 -- 172.21.15.190:0/1606802536 wait complete. 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient: build_initial_monmap 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient: monmap: 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:56.424 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 Processor -- start 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 -- start start 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient: init 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:56.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient: _add_conns ranks=[5,6,7,0,1,2,8,3,4] 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44600a1810 0x7f44600730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient(hunting): picked mon.noname-h con 0x7f44600a1810 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44600a2280 0x7f4460073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient(hunting): picked mon.noname-d con 0x7f44600a2280 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4460073b60 0x7f44601d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient(hunting): picked mon.noname-i con 0x7f4460073b60 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4460144030 con 0x7f44600a1810 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4460143eb0 con 0x7f44600a2280 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f44601441b0 con 0x7f4460073b60 2024-08-19T10:41:56.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f44650a8640 10 monclient(hunting): authenticate will time out at 1161.986366s 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445f59e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4460073b60 0x7f44601d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44600a1810 0x7f44600730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445f59e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4460073b60 0x7f44601d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:51636/0 (socket says 172.21.15.190:51636) 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44600a1810 0x7f44600730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:38590/0 (socket says 172.21.15.190:38590) 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445f59e640 1 -- 172.21.15.190:0/300754801 learned_addr learned my addr 172.21.15.190:0/300754801 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445f59e640 10 monclient(hunting): get_auth_request con 0x7f4460073b60 auth_method 0 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445f59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445f59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 10 monclient(hunting): get_auth_request con 0x7f44600a1810 auth_method 0 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:56.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445e59c640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44600a2280 0x7f4460073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 10 monclient(hunting): handle_auth_done global_id 5423 payload 824 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 1 -- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4460073b60 msgr2=0x7f44601d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4460073b60 0x7f44601d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 1 -- 172.21.15.190:0/300754801 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44600a2280 msgr2=0x7f4460073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:56.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44600a2280 0x7f4460073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 1 monclient: found mon.noname-h 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445ed9d640 1 -- 172.21.15.190:0/300754801 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44600a3ed0 con 0x7f44600a1810 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445e59c640 10 monclient: get_auth_request con 0x7f44600a2280 auth_method 0 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445e59c640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44600a2280 0x7f4460073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.413+0000 7f445f59e640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4460073b60 0x7f44601d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f445ed9d640 1 -- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44600a1810 msgr2=0x7f44600730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f445ed9d640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44600a1810 0x7f44600730e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f445000b440 tx=0x7f445000b900 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 1 -- 172.21.15.190:0/300754801 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4450011cb0 con 0x7f44600a1810 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:56.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 10 monclient: dump: 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:56.430 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 10 monclient: _finish_auth 0 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 10 monclient: _check_auth_tickets 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 1 -- 172.21.15.190:0/300754801 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4450016070 con 0x7f44600a1810 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 1 -- 172.21.15.190:0/300754801 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44500107f0 con 0x7f44600a1810 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:56.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 10 monclient: dump: 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:56.432 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f445cd99640 4 set_mon_vals no callback set 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44650a8640 5 monclient: authenticate success, global_id 5423 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44650a8640 10 monclient: _renew_subs 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44650a8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44650a8640 1 -- 172.21.15.190:0/300754801 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4460093fa0 con 0x7f44600a1810 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44650a8640 10 monclient: _renew_subs 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44650a8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44650a8640 1 -- 172.21.15.190:0/300754801 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f446013c770 con 0x7f44600a1810 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44650a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44650a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:56.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44650a8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:56.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44650a8640 1 -- 172.21.15.190:0/300754801 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44600a4330 con 0x7f44600a1810 2024-08-19T10:41:56.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 1 -- 172.21.15.190:0/300754801 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4450010380 con 0x7f44600a1810 2024-08-19T10:41:56.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f444c03be30 0x7f444c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:56.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f44477fe640 1 -- 172.21.15.190:0/300754801 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(116..116 src has 1..116) ==== 6437+0+0 (secure 0 0 0) 0x7f445004ea80 con 0x7f44600a1810 2024-08-19T10:41:56.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f445e59c640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f444c03be30 0x7f444c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:56.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.417+0000 7f445e59c640 10 monclient: get_auth_request con 0x7f444c03be30 auth_method 0 2024-08-19T10:41:56.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.421+0000 7f445e59c640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f444c03be30 0x7f444c03e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f44540ae000 tx=0x7f44540a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:56.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.421+0000 7f44477fe640 1 -- 172.21.15.190:0/300754801 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4450020020 con 0x7f44600a1810 2024-08-19T10:41:56.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.421+0000 7f44477fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:56.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.421+0000 7f44477fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:56.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.605+0000 7f44650a8640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:56.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.605+0000 7f44650a8640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:56.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.605+0000 7f44650a8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:56.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:56.605+0000 7f44650a8640 1 -- 172.21.15.190:0/300754801 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) -- 0x7f4460073f40 con 0x7f44600a1810 2024-08-19T10:41:57.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.265+0000 7f44477fe640 1 -- 172.21.15.190:0/300754801 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 add item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default} to crush map v117) ==== 220+0+0 (secure 0 0 0) 0x7f4460073f40 con 0x7f44600a1810 2024-08-19T10:41:57.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.265+0000 7f44477fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:57.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.265+0000 7f44477fe640 10 monclient: _finish_command 2 = system:0 add item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default} to crush map 2024-08-19T10:41:57.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.377+0000 7f44650a8640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:57.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.377+0000 7f44650a8640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:57.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.377+0000 7f44650a8640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:57.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.377+0000 7f44650a8640 1 -- 172.21.15.190:0/300754801 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) -- 0x7f44600a2660 con 0x7f44600a1810 2024-08-19T10:41:57.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44477fe640 1 -- 172.21.15.190:0/300754801 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change v117) ==== 218+0+0 (secure 0 0 0) 0x7f44600a3ed0 con 0x7f44600a1810 2024-08-19T10:41:57.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44477fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 12, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:57.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44477fe640 10 monclient: _finish_command 3 = system:0 set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change 2024-08-19T10:41:57.383 INFO:tasks.workunit.client.0.smithi190.stderr:set item id 12 name 'osd.12' weight 123 at location {host=foobaz,root=default}: no change 2024-08-19T10:41:57.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 -- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f444c03be30 msgr2=0x7f444c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:57.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f444c03be30 0x7f444c03e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f44540ae000 tx=0x7f44540a7000 comp rx=0 tx=0).stop 2024-08-19T10:41:57.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 10 monclient: shutdown 2024-08-19T10:41:57.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:57.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 -- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44600a1810 msgr2=0x7f44600730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:57.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44600a1810 0x7f44600730e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f445000b440 tx=0x7f445000b900 comp rx=0 tx=0).stop 2024-08-19T10:41:57.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 -- 172.21.15.190:0/300754801 shutdown_connections 2024-08-19T10:41:57.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f444c03be30 0x7f444c03e2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:57.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4460073b60 0x7f44601d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:57.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44600a2280 0x7f4460073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:57.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 --2- 172.21.15.190:0/300754801 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44600a1810 0x7f44600730e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:57.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 -- 172.21.15.190:0/300754801 >> 172.21.15.190:0/300754801 conn(0x7f44600a84c0 msgr2=0x7f446009e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:57.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 -- 172.21.15.190:0/300754801 shutdown_connections 2024-08-19T10:41:57.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.381+0000 7f44650a8640 1 -- 172.21.15.190:0/300754801 wait complete. 2024-08-19T10:41:57.402 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add 14 123 root=default host=foobaz 2024-08-19T10:41:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.521+0000 7f10ba8f6640 10 monclient: get_monmap_and_config 2024-08-19T10:41:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.521+0000 7f10ba8f6640 10 monclient: build_initial_monmap 2024-08-19T10:41:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.521+0000 7f10ba8f6640 10 monclient: monmap: 2024-08-19T10:41:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:57.526234+0000 2024-08-19T10:41:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:57.526234+0000 2024-08-19T10:41:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:57.527 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:57.527 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:57.527 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:57.527 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:57.527 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:57.527 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:57.527 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:57.527 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:57.527 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:57.527 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:57.527 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:57.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.521+0000 7f10ba8f6640 1 Processor -- start 2024-08-19T10:41:57.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.521+0000 7f10ba8f6640 1 -- start start 2024-08-19T10:41:57.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.521+0000 7f10ba8f6640 10 monclient: init 2024-08-19T10:41:57.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:57.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 10 monclient: _add_conns ranks=[2,5,3,0,9,6,10,4,1,8,7] 2024-08-19T10:41:57.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10b40a3c90 0x7f10b40a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:57.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 10 monclient(hunting): picked mon.noname-b con 0x7f10b40a3c90 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:41:57.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b40a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:57.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 10 monclient(hunting): picked mon.noname-h con 0x7f10b40a4070 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:57.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b40a49d0 0x7f10b4139970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:57.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 10 monclient(hunting): picked mon.noname-g con 0x7f10b40a49d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:57.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f10b40a0e30 con 0x7f10b40a3c90 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f10b40a1110 con 0x7f10b40a49d0 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f10b40a13f0 con 0x7f10b40a4070 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10ba8f6640 10 monclient(hunting): authenticate will time out at 1163.096690s 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b8e93640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b40a49d0 msgr2=0x7f10b4139970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b40a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b8e93640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b40a49d0 0x7f10b4139970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b40a4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:38594/0 (socket says 172.21.15.190:38594) 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 1 -- 172.21.15.190:0/789842922 learned_addr learned my addr 172.21.15.190:0/789842922 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 10 monclient(hunting): get_auth_request con 0x7f10b40a4070 auth_method 0 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b3fff640 1 --2- 172.21.15.190:0/789842922 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10b40a3c90 0x7f10b40a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b3fff640 10 monclient(hunting): get_auth_request con 0x7f10b40a3c90 auth_method 0 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 10 monclient(hunting): handle_auth_done global_id 5432 payload 274 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 1 -- 172.21.15.190:0/789842922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b40a49d0 msgr2=0x7f10b4139970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 1 --2- 172.21.15.190:0/789842922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b40a49d0 0x7f10b4139970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:57.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 1 -- 172.21.15.190:0/789842922 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10b40a3c90 msgr2=0x7f10b40a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:57.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 1 --2- 172.21.15.190:0/789842922 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10b40a3c90 0x7f10b40a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:57.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:57.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 1 monclient: found mon.noname-h 2024-08-19T10:41:57.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:57.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:57.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.525+0000 7f10b37fe640 1 -- 172.21.15.190:0/789842922 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b406cbc0 con 0x7f10b40a4070 2024-08-19T10:41:57.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b37fe640 1 -- 172.21.15.190:0/789842922 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 msgr2=0x7f10b40a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:57.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b37fe640 1 --2- 172.21.15.190:0/789842922 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b40a4490 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f10a80098a0 tx=0x7f10a801c230 comp rx=0 tx=0).ready entity=mon.5 client_cookie=8e85723e7c0b5f4d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:57.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 1 -- 172.21.15.190:0/789842922 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10a80b1070 con 0x7f10b40a4070 2024-08-19T10:41:57.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:57.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:57.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 10 monclient: dump: 2024-08-19T10:41:57.537 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:57.537 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:57.537 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:57.537 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 5 monclient: authenticate success, global_id 5432 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 1 -- 172.21.15.190:0/789842922 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a80a4d00 con 0x7f10b40a4070 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 4 set_mon_vals no callback set 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 1 -- 172.21.15.190:0/789842922 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10a80aac20 con 0x7f10b40a4070 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10b2ffd640 10 monclient: dump: 2024-08-19T10:41:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:57.540 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient: shutdown 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 -- 172.21.15.190:0/789842922 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 msgr2=0x7f10b40a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 --2- 172.21.15.190:0/789842922 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b40a4490 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f10a80098a0 tx=0x7f10a801c230 comp rx=0 tx=0).stop 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 -- 172.21.15.190:0/789842922 shutdown_connections 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 --2- 172.21.15.190:0/789842922 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b40a4490 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 --2- 172.21.15.190:0/789842922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b40a49d0 0x7f10b4139970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 --2- 172.21.15.190:0/789842922 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10b40a3c90 0x7f10b40a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 -- 172.21.15.190:0/789842922 >> 172.21.15.190:0/789842922 conn(0x7f10b406de10 msgr2=0x7f10b406e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 -- 172.21.15.190:0/789842922 shutdown_connections 2024-08-19T10:41:57.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 -- 172.21.15.190:0/789842922 wait complete. 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient: build_initial_monmap 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient: monmap: 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:57.542 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 Processor -- start 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 -- start start 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient: init 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient: _add_conns ranks=[0,5,4,8,7,3,6,2,1] 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10b40a3c90 0x7f10b41d1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:57.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient(hunting): picked mon.noname-a con 0x7f10b40a3c90 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b4139170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient(hunting): picked mon.noname-h con 0x7f10b40a4070 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10b40a49d0 0x7f10b41396b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient(hunting): picked mon.noname-c con 0x7f10b40a49d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f10b4143560 con 0x7f10b40a3c90 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f10b41433e0 con 0x7f10b40a49d0 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f10b41436e0 con 0x7f10b40a4070 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.529+0000 7f10ba8f6640 10 monclient(hunting): authenticate will time out at 1163.103971s 2024-08-19T10:41:57.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b4139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b4139170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:38608/0 (socket says 172.21.15.190:38608) 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 1 -- 172.21.15.190:0/1889053896 learned_addr learned my addr 172.21.15.190:0/1889053896 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 10 monclient(hunting): get_auth_request con 0x7f10b40a4070 auth_method 0 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 10 monclient(hunting): handle_auth_done global_id 5441 payload 824 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 1 -- 172.21.15.190:0/1889053896 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10b40a49d0 msgr2=0x7f10b41396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10b40a49d0 0x7f10b41396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:57.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 1 -- 172.21.15.190:0/1889053896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10b40a3c90 msgr2=0x7f10b41d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:57.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10b40a3c90 0x7f10b41d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:57.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:41:57.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 1 monclient: found mon.noname-h 2024-08-19T10:41:57.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:57.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:41:57.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 1 -- 172.21.15.190:0/1889053896 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b413f2a0 con 0x7f10b40a4070 2024-08-19T10:41:57.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 1 -- 172.21.15.190:0/1889053896 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 msgr2=0x7f10b4139170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:57.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b37fe640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b4139170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f10a80af040 tx=0x7f10a8018fd0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:57.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 1 -- 172.21.15.190:0/1889053896 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10a80b1040 con 0x7f10b40a4070 2024-08-19T10:41:57.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:57.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:41:57.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 10 monclient: dump: 2024-08-19T10:41:57.548 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:57.548 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:57.548 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:57.548 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:57.549 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:57.549 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:57.549 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:57.549 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:57.549 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:57.549 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:57.549 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:57.549 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:57.549 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:57.549 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 10 monclient: _finish_auth 0 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 1 -- 172.21.15.190:0/1889053896 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a8003c40 con 0x7f10b40a4070 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10ba8f6640 5 monclient: authenticate success, global_id 5441 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10ba8f6640 10 monclient: _renew_subs 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10ba8f6640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10ba8f6640 1 -- 172.21.15.190:0/1889053896 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b40a29c0 con 0x7f10b40a4070 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:57.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 1 -- 172.21.15.190:0/1889053896 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10a8003df0 con 0x7f10b40a4070 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 10 monclient: dump: 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:57.551 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b27fc640 4 set_mon_vals no callback set 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10ba8f6640 10 monclient: _renew_subs 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10ba8f6640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10ba8f6640 1 -- 172.21.15.190:0/1889053896 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b409edb0 con 0x7f10b40a4070 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10b0ff9640 1 -- 172.21.15.190:0/1889053896 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f10a80ac440 con 0x7f10b40a4070 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10ba8f6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10ba8f6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:57.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10ba8f6640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:57.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.533+0000 7f10ba8f6640 1 -- 172.21.15.190:0/1889053896 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b409f290 con 0x7f10b40a4070 2024-08-19T10:41:57.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.537+0000 7f10b0ff9640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f10a003be00 0x7f10a003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:57.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.537+0000 7f10b0ff9640 1 -- 172.21.15.190:0/1889053896 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(117..117 src has 1..117) ==== 6621+0+0 (secure 0 0 0) 0x7f10a80e8eb0 con 0x7f10b40a4070 2024-08-19T10:41:57.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.537+0000 7f10b3fff640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f10a003be00 0x7f10a003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:57.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.537+0000 7f10b3fff640 10 monclient: get_auth_request con 0x7f10a003be00 auth_method 0 2024-08-19T10:41:57.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.537+0000 7f10b3fff640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f10a003be00 0x7f10a003e2c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f10a40098f0 tx=0x7f10a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:57.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.541+0000 7f10b0ff9640 1 -- 172.21.15.190:0/1889053896 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f10b40a29c0 con 0x7f10b40a4070 2024-08-19T10:41:57.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.541+0000 7f10b0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:57.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.541+0000 7f10b0ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:57.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.713+0000 7f10ba8f6640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:57.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.713+0000 7f10ba8f6640 10 monclient: _send_command 2 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:57.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.713+0000 7f10ba8f6640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:57.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:57.713+0000 7f10ba8f6640 1 -- 172.21.15.190:0/1889053896 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) -- 0x7f10b40a4490 con 0x7f10b40a4070 2024-08-19T10:41:58.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:58.977+0000 7f10b0ff9640 1 -- 172.21.15.190:0/1889053896 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 add item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default} to crush map v118) ==== 220+0+0 (secure 0 0 0) 0x7f10a8016170 con 0x7f10b40a4070 2024-08-19T10:41:58.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:58.977+0000 7f10b0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:58.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:58.977+0000 7f10b0ff9640 10 monclient: _finish_command 2 = system:0 add item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default} to crush map 2024-08-19T10:41:59.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.089+0000 7f10ba8f6640 10 start_mon_command cmd=[{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:59.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.089+0000 7f10ba8f6640 10 monclient: _send_command 3 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:59.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.089+0000 7f10ba8f6640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:41:59.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.089+0000 7f10ba8f6640 1 -- 172.21.15.190:0/1889053896 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]} v 0) -- 0x7f10b4124c20 con 0x7f10b40a4070 2024-08-19T10:41:59.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.089+0000 7f10b0ff9640 1 -- 172.21.15.190:0/1889053896 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}]=0 set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change v118) ==== 218+0+0 (secure 0 0 0) 0x7f10a80e7180 con 0x7f10b40a4070 2024-08-19T10:41:59.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.089+0000 7f10b0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add", "id": 14, "weight": 123.0, "args": ["root=default", "host=foobaz"]}] 2024-08-19T10:41:59.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.089+0000 7f10b0ff9640 10 monclient: _finish_command 3 = system:0 set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change 2024-08-19T10:41:59.094 INFO:tasks.workunit.client.0.smithi190.stderr:set item id 14 name 'osd.14' weight 123 at location {host=foobaz,root=default}: no change 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 -- 172.21.15.190:0/1889053896 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f10a003be00 msgr2=0x7f10a003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f10a003be00 0x7f10a003e2c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f10a40098f0 tx=0x7f10a4008040 comp rx=0 tx=0).stop 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 10 monclient: shutdown 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 -- 172.21.15.190:0/1889053896 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 msgr2=0x7f10b4139170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b4139170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f10a80af040 tx=0x7f10a8018fd0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 -- 172.21.15.190:0/1889053896 shutdown_connections 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f10a003be00 0x7f10a003e2c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10b40a49d0 0x7f10b41396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f10b40a4070 0x7f10b4139170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 --2- 172.21.15.190:0/1889053896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10b40a3c90 0x7f10b41d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 -- 172.21.15.190:0/1889053896 >> 172.21.15.190:0/1889053896 conn(0x7f10b406de10 msgr2=0x7f10b40a7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:59.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 -- 172.21.15.190:0/1889053896 shutdown_connections 2024-08-19T10:41:59.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.093+0000 7f10ba8f6640 1 -- 172.21.15.190:0/1889053896 wait complete. 2024-08-19T10:41:59.114 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:59.114 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 123 2024-08-19T10:41:59.114 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep osd.12 2024-08-19T10:41:59.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient: get_monmap_and_config 2024-08-19T10:41:59.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient: build_initial_monmap 2024-08-19T10:41:59.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient: monmap: 2024-08-19T10:41:59.229 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:59.229 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:59.229 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:59.229637+0000 2024-08-19T10:41:59.229 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:59.229637+0000 2024-08-19T10:41:59.230 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:59.230 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.230 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:59.230 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:59.230 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:59.230 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:59.230 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:59.230 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:59.230 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:59.230 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:59.230 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:59.231 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:59.231 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:59.231 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 1 Processor -- start 2024-08-19T10:41:59.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 1 -- start start 2024-08-19T10:41:59.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient: init 2024-08-19T10:41:59.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:59.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient: _add_conns ranks=[10,9,4,6,8,1,2,7,3,0,5] 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe3240a57d0 0x7fe324071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient(hunting): picked mon.noname-f con 0x7fe3240a57d0 addr v2:172.21.15.190:3300/0 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe3240a5bb0 0x7fe3240a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient(hunting): picked mon.noname-a con 0x7fe3240a5bb0 addr v2:172.21.15.89:3300/0 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 0x7fe324139a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient(hunting): picked mon.noname-c con 0x7fe3241395a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fe32406ddc0 con 0x7fe3240a5bb0 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe32406f020 con 0x7fe3240a57d0 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fe32406f300 con 0x7fe3241395a0 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe32b6c4640 10 monclient(hunting): authenticate will time out at 1164.799477s 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe329460640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe3240a57d0 0x7fe324071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe329460640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe3240a57d0 0x7fe324071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:53808/0 (socket says 172.21.15.190:53808) 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe329460640 1 -- 172.21.15.190:0/2081190694 learned_addr learned my addr 172.21.15.190:0/2081190694 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe329460640 10 monclient(hunting): get_auth_request con 0x7fe3240a57d0 auth_method 0 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe329460640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe329460640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe329460640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe329460640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe329460640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:59.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.225+0000 7fe329460640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe328c5f640 1 --2- 172.21.15.190:0/2081190694 >> v2:172.21.15.89:3300/0 conn(0x7fe3240a5bb0 0x7fe3240a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe329c61640 1 --2- 172.21.15.190:0/2081190694 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 0x7fe324139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe329460640 10 monclient(hunting): handle_auth_done global_id 14689 payload 274 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe329460640 1 -- 172.21.15.190:0/2081190694 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 msgr2=0x7fe324139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe329460640 1 --2- 172.21.15.190:0/2081190694 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 0x7fe324139a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe329460640 1 -- 172.21.15.190:0/2081190694 >> v2:172.21.15.89:3300/0 conn(0x7fe3240a5bb0 msgr2=0x7fe3240a5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe329460640 1 --2- 172.21.15.190:0/2081190694 >> v2:172.21.15.89:3300/0 conn(0x7fe3240a5bb0 0x7fe3240a5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe329460640 10 monclient: _finish_hunting 0 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe329460640 1 monclient: found mon.noname-f 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe329460640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe329460640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:59.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe329460640 1 -- 172.21.15.190:0/2081190694 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32406f5e0 con 0x7fe3240a57d0 2024-08-19T10:41:59.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe328c5f640 10 monclient: get_auth_request con 0x7fe3240a5bb0 auth_method 0 2024-08-19T10:41:59.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.229+0000 7fe328c5f640 1 --2- 172.21.15.190:0/2081190694 >> v2:172.21.15.89:3300/0 conn(0x7fe3240a5bb0 0x7fe3240a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:59.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329460640 1 -- 172.21.15.190:0/2081190694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3240a57d0 msgr2=0x7fe324071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:59.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329460640 1 --2- 172.21.15.190:0/2081190694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3240a57d0 0x7fe324071f60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe3240746f0 tx=0x7fe32001c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=917eb97c678eb446 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:59.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 1 -- 172.21.15.190:0/2081190694 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe3200091b0 con 0x7fe3240a57d0 2024-08-19T10:41:59.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:59.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:59.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 10 monclient: dump: 2024-08-19T10:41:59.241 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:59.241 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:59.241 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.241 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.241 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:59.241 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.241 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:59.241 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:59.241 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 1 -- 172.21.15.190:0/2081190694 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3200a5b80 con 0x7fe3240a57d0 2024-08-19T10:41:59.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 5 monclient: authenticate success, global_id 14689 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 1 -- 172.21.15.190:0/2081190694 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe3200a5d30 con 0x7fe3240a57d0 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 4 set_mon_vals no callback set 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe317fff640 10 monclient: dump: 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.243 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient: shutdown 2024-08-19T10:41:59.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2081190694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3240a57d0 msgr2=0x7fe324071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 --2- 172.21.15.190:0/2081190694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3240a57d0 0x7fe324071f60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe3240746f0 tx=0x7fe32001c870 comp rx=0 tx=0).stop 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2081190694 shutdown_connections 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 --2- 172.21.15.190:0/2081190694 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 0x7fe324139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 --2- 172.21.15.190:0/2081190694 >> v2:172.21.15.89:3300/0 conn(0x7fe3240a5bb0 0x7fe3240a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 --2- 172.21.15.190:0/2081190694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3240a57d0 0x7fe324071f60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2081190694 >> 172.21.15.190:0/2081190694 conn(0x7fe3240aa3c0 msgr2=0x7fe3240a68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2081190694 shutdown_connections 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2081190694 wait complete. 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient: build_initial_monmap 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient: monmap: 2024-08-19T10:41:59.245 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:59.246 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 Processor -- start 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 -- start start 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient: init 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient: _add_conns ranks=[0,6,4,7,2,5,1,8,3] 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3240a57d0 0x7fe324122840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient(hunting): picked mon.noname-a con 0x7fe3240a57d0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe3240a5bb0 0x7fe3241386a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient(hunting): picked mon.noname-d con 0x7fe3240a5bb0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 0x7fe3241d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient(hunting): picked mon.noname-c con 0x7fe3241395a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:59.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe32406e660 con 0x7fe3240a57d0 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fe3240a64f0 con 0x7fe3241395a0 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fe324139aa0 con 0x7fe3240a5bb0 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe32b6c4640 10 monclient(hunting): authenticate will time out at 1164.806909s 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329c61640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 0x7fe3241d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329c61640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 0x7fe3241d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:46632/0 (socket says 172.21.15.190:46632) 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe328c5f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe3240a5bb0 0x7fe3241386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329c61640 1 -- 172.21.15.190:0/2814707463 learned_addr learned my addr 172.21.15.190:0/2814707463 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329c61640 10 monclient(hunting): get_auth_request con 0x7fe3241395a0 auth_method 0 2024-08-19T10:41:59.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329c61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:59.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329c61640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:59.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329c61640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:59.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329460640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3240a57d0 0x7fe324122840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe328c5f640 10 monclient(hunting): get_auth_request con 0x7fe3240a5bb0 auth_method 0 2024-08-19T10:41:59.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe328c5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:59.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe328c5f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:59.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe328c5f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:59.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329460640 10 monclient(hunting): get_auth_request con 0x7fe3240a57d0 auth_method 0 2024-08-19T10:41:59.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329460640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:59.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329460640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:59.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329460640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329c61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329c61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329c61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe328c5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe328c5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe328c5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329460640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329460640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.233+0000 7fe329460640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 10 monclient(hunting): handle_auth_done global_id 5289 payload 824 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 1 -- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 msgr2=0x7fe3241d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 0x7fe3241d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 1 -- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3240a57d0 msgr2=0x7fe324122840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3240a57d0 0x7fe324122840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 10 monclient: _finish_hunting 0 2024-08-19T10:41:59.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 1 monclient: found mon.noname-d 2024-08-19T10:41:59.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:59.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:41:59.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 1 -- 172.21.15.190:0/2814707463 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe324071f60 con 0x7fe3240a5bb0 2024-08-19T10:41:59.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe329460640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3240a57d0 0x7fe324122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:59.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe329c61640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 0x7fe3241d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:41:59.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 1 -- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe3240a5bb0 msgr2=0x7fe3241386a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:59.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe328c5f640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe3240a5bb0 0x7fe3241386a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00c7f0 tx=0x7fe31c00ccb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:59.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 1 -- 172.21.15.190:0/2814707463 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe31c015070 con 0x7fe3240a5bb0 2024-08-19T10:41:59.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:59.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:41:59.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 10 monclient: dump: 2024-08-19T10:41:59.254 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:59.254 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:59.254 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.254 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.254 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:59.254 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.254 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:59.254 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 10 monclient: _finish_auth 0 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:59.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe32b6c4640 5 monclient: authenticate success, global_id 5289 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe32b6c4640 10 monclient: _renew_subs 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe32b6c4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2814707463 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe324120c90 con 0x7fe3240a5bb0 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe32b6c4640 10 monclient: _renew_subs 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe32b6c4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2814707463 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fe324124040 con 0x7fe3240a5bb0 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 1 -- 172.21.15.190:0/2814707463 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe31c007870 con 0x7fe3240a5bb0 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 1 -- 172.21.15.190:0/2814707463 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe31c019400 con 0x7fe3240a5bb0 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe3177fe640 4 set_mon_vals no callback set 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe315ffb640 10 monclient: dump: 2024-08-19T10:41:59.256 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:59.257 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe32b6c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe32b6c4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe32b6c4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.237+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2814707463 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe304005090 con 0x7fe3240a5bb0 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.241+0000 7fe315ffb640 1 -- 172.21.15.190:0/2814707463 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe31c007a20 con 0x7fe3240a5bb0 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.241+0000 7fe315ffb640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30803be00 0x7fe30803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.241+0000 7fe315ffb640 1 -- 172.21.15.190:0/2814707463 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(118..118 src has 1..118) ==== 6643+0+0 (secure 0 0 0) 0x7fe31c04f280 con 0x7fe3240a5bb0 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.241+0000 7fe329460640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30803be00 0x7fe30803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.241+0000 7fe329460640 10 monclient: get_auth_request con 0x7fe30803be00 auth_method 0 2024-08-19T10:41:59.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.241+0000 7fe329460640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30803be00 0x7fe30803e2c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe3200ab000 tx=0x7fe3200a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:59.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.253+0000 7fe315ffb640 1 -- 172.21.15.190:0/2814707463 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe31c01e070 con 0x7fe3240a5bb0 2024-08-19T10:41:59.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.253+0000 7fe315ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:59.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.253+0000 7fe315ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:59.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.421+0000 7fe32b6c4640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:59.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.421+0000 7fe32b6c4640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:59.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.425+0000 7fe32b6c4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:59.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.425+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2814707463 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fe304005650 con 0x7fe3240a5bb0 2024-08-19T10:41:59.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.425+0000 7fe315ffb640 1 -- 172.21.15.190:0/2814707463 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v118) ==== 56+0+1449 (secure 0 0 0) 0x7fe31c007d10 con 0x7fe3240a5bb0 2024-08-19T10:41:59.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.425+0000 7fe315ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:59.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.425+0000 7fe315ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:59.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.517+0000 7fe32b6c4640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:59.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.517+0000 7fe32b6c4640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:59.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.517+0000 7fe32b6c4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:41:59.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.517+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2814707463 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fe304005820 con 0x7fe3240a5bb0 2024-08-19T10:41:59.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe315ffb640 1 -- 172.21.15.190:0/2814707463 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v118) ==== 56+0+1449 (secure 0 0 0) 0x7fe31c04c900 con 0x7fe3240a5bb0 2024-08-19T10:41:59.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe315ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:59.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe315ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:59.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2814707463 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30803be00 msgr2=0x7fe30803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30803be00 0x7fe30803e2c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe3200ab000 tx=0x7fe3200a7070 comp rx=0 tx=0).stop 2024-08-19T10:41:59.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 10 monclient: shutdown 2024-08-19T10:41:59.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:59.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe3240a5bb0 msgr2=0x7fe3241386a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe3240a5bb0 0x7fe3241386a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00c7f0 tx=0x7fe31c00ccb0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2814707463 shutdown_connections 2024-08-19T10:41:59.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30803be00 0x7fe30803e2c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3241395a0 0x7fe3241d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe3240a5bb0 0x7fe3241386a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 --2- 172.21.15.190:0/2814707463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3240a57d0 0x7fe324122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2814707463 >> 172.21.15.190:0/2814707463 conn(0x7fe3240aa3c0 msgr2=0x7fe3240a7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:59.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2814707463 shutdown_connections 2024-08-19T10:41:59.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.521+0000 7fe32b6c4640 1 -- 172.21.15.190:0/2814707463 wait complete. 2024-08-19T10:41:59.539 INFO:tasks.workunit.client.0.smithi190.stdout: 12 123.00000 osd.12 down 1.00000 1.00000 2024-08-19T10:41:59.540 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:41:59.540 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep osd.14 2024-08-19T10:41:59.540 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 123 2024-08-19T10:41:59.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient: get_monmap_and_config 2024-08-19T10:41:59.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient: build_initial_monmap 2024-08-19T10:41:59.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient: monmap: 2024-08-19T10:41:59.641 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:59.641 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:59.641 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:41:59.641536+0000 2024-08-19T10:41:59.641 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:41:59.641536+0000 2024-08-19T10:41:59.641 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:59.641 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.641 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:41:59.641 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:41:59.641 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:59.642 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:59.642 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:59.642 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:59.642 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:59.642 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:59.642 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:59.642 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:41:59.642 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:41:59.642 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 1 Processor -- start 2024-08-19T10:41:59.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 1 -- start start 2024-08-19T10:41:59.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient: init 2024-08-19T10:41:59.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:59.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient: _add_conns ranks=[3,4,6,10,7,0,1,9,5,8,2] 2024-08-19T10:41:59.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f620c0a5340 0x7f620c1396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient(hunting): picked mon.noname-g con 0x7f620c0a5340 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:41:59.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f620c139be0 0x7f620c139fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient(hunting): picked mon.noname-c con 0x7f620c139be0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:41:59.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c0a68e0 0x7f620c0a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient(hunting): picked mon.noname-d con 0x7f620c0a68e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:59.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f620c0a2f30 con 0x7f620c0a5340 2024-08-19T10:41:59.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f620c0a3500 con 0x7f620c139be0 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f620c13a500 con 0x7f620c0a68e0 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f6211e27640 10 monclient(hunting): authenticate will time out at 1165.211549s 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f620b7fe640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f620c0a5340 msgr2=0x7f620c1396a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f620b7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f620c0a5340 0x7f620c1396a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f620affd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f620c139be0 0x7f620c139fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.637+0000 7f620bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c0a68e0 0x7f620c0a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620affd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f620c139be0 0x7f620c139fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:46642/0 (socket says 172.21.15.190:46642) 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c0a68e0 0x7f620c0a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:46440/0 (socket says 172.21.15.190:46440) 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620affd640 1 -- 172.21.15.190:0/3994607454 learned_addr learned my addr 172.21.15.190:0/3994607454 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620affd640 10 monclient(hunting): get_auth_request con 0x7f620c139be0 auth_method 0 2024-08-19T10:41:59.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 10 monclient(hunting): get_auth_request con 0x7f620c0a68e0 auth_method 0 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:59.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 10 monclient(hunting): handle_auth_done global_id 5298 payload 274 2024-08-19T10:41:59.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 1 -- 172.21.15.190:0/3994607454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f620c139be0 msgr2=0x7f620c139fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:41:59.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 1 --2- 172.21.15.190:0/3994607454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f620c139be0 0x7f620c139fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 1 -- 172.21.15.190:0/3994607454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f620c0a5340 msgr2=0x7f620c1396a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:41:59.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 1 --2- 172.21.15.190:0/3994607454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f620c0a5340 0x7f620c1396a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:41:59.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 1 monclient: found mon.noname-d 2024-08-19T10:41:59.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:59.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:41:59.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.641+0000 7f620bfff640 1 -- 172.21.15.190:0/3994607454 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c0a8610 con 0x7f620c0a68e0 2024-08-19T10:41:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620bfff640 1 -- 172.21.15.190:0/3994607454 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c0a68e0 msgr2=0x7f620c0a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620bfff640 1 --2- 172.21.15.190:0/3994607454 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c0a68e0 0x7f620c0a6d20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6204009dc0 tx=0x7f620401c870 comp rx=0 tx=0).ready entity=mon.6 client_cookie=4bb317a186ddcd6e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 1 -- 172.21.15.190:0/3994607454 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62040ac070 con 0x7f620c0a68e0 2024-08-19T10:41:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:41:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 10 monclient: dump: 2024-08-19T10:41:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 1 -- 172.21.15.190:0/3994607454 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62040a68d0 con 0x7f620c0a68e0 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 1 -- 172.21.15.190:0/3994607454 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62040a7440 con 0x7f620c0a68e0 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f620a7fc640 10 monclient: dump: 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.654 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 5 monclient: authenticate success, global_id 5298 2024-08-19T10:41:59.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 10 monclient: get_monmap_and_config success 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 4 set_mon_vals no callback set 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 10 monclient: shutdown 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 -- 172.21.15.190:0/3994607454 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c0a68e0 msgr2=0x7f620c0a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 --2- 172.21.15.190:0/3994607454 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c0a68e0 0x7f620c0a6d20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6204009dc0 tx=0x7f620401c870 comp rx=0 tx=0).stop 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 -- 172.21.15.190:0/3994607454 shutdown_connections 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 --2- 172.21.15.190:0/3994607454 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c0a68e0 0x7f620c0a6d20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 --2- 172.21.15.190:0/3994607454 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f620c139be0 0x7f620c139fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 --2- 172.21.15.190:0/3994607454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f620c0a5340 0x7f620c1396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 -- 172.21.15.190:0/3994607454 >> 172.21.15.190:0/3994607454 conn(0x7f620c074bd0 msgr2=0x7f620c0abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 -- 172.21.15.190:0/3994607454 shutdown_connections 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 -- 172.21.15.190:0/3994607454 wait complete. 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 10 monclient: build_initial_monmap 2024-08-19T10:41:59.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 10 monclient: monmap: 2024-08-19T10:41:59.657 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:41:59.657 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:41:59.657 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:41:59.657 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:41:59.657 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:41:59.657 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.657 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:41:59.657 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 Processor -- start 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 -- start start 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 10 monclient: init 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 10 monclient: _reopen_session rank -1 2024-08-19T10:41:59.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 10 monclient: _add_conns ranks=[0,1,6,8,5,3,7,2,4] 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c0a5340 0x7f620c131dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 10 monclient(hunting): picked mon.noname-a con 0x7f620c0a5340 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0a68e0 0x7f620c13cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 10 monclient(hunting): picked mon.noname-f con 0x7f620c0a68e0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.645+0000 7f6211e27640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c139be0 0x7f620c13d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient(hunting): picked mon.noname-d con 0x7f620c139be0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f620c143320 con 0x7f620c0a5340 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f620c0a8360 con 0x7f620c0a68e0 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f620c1434a0 con 0x7f620c139be0 2024-08-19T10:41:59.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient(hunting): _renew_subs 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient(hunting): authenticate will time out at 1165.220027s 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0a68e0 0x7f620c13cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0a68e0 0x7f620c13cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:53810/0 (socket says 172.21.15.190:53810) 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 1 -- 172.21.15.190:0/644094952 learned_addr learned my addr 172.21.15.190:0/644094952 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 10 monclient(hunting): get_auth_request con 0x7f620c0a68e0 auth_method 0 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620bfff640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c139be0 0x7f620c13d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620b7fe640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c0a5340 0x7f620c131dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 10 monclient(hunting): handle_auth_done global_id 14698 payload 824 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 1 -- 172.21.15.190:0/644094952 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c139be0 msgr2=0x7f620c13d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c139be0 0x7f620c13d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 1 -- 172.21.15.190:0/644094952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c0a5340 msgr2=0x7f620c131dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c0a5340 0x7f620c131dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 10 monclient: _finish_hunting 0 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 1 monclient: found mon.noname-f 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 1 -- 172.21.15.190:0/644094952 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c09d530 con 0x7f620c0a68e0 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620b7fe640 10 monclient: get_auth_request con 0x7f620c0a5340 auth_method 0 2024-08-19T10:41:59.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620b7fe640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c0a5340 0x7f620c131dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:41:59.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 1 -- 172.21.15.190:0/644094952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0a68e0 msgr2=0x7f620c13cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:41:59.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f620affd640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0a68e0 0x7f620c13cf60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00e910 tx=0x7f61fc00edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 1 -- 172.21.15.190:0/644094952 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61fc017070 con 0x7f620c0a68e0 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 10 monclient: dump: 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 10 monclient: _finish_auth 0 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:41:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 5 monclient: authenticate success, global_id 14698 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient: _renew_subs 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 1 -- 172.21.15.190:0/644094952 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61fc00cc10 con 0x7f620c0a68e0 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 1 -- 172.21.15.190:0/644094952 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620c0ab390 con 0x7f620c0a68e0 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 1 -- 172.21.15.190:0/644094952 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61fc00cdc0 con 0x7f620c0a68e0 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6209ffb640 4 set_mon_vals no callback set 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 10 monclient: dump: 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:41:59.666 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:41:59.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient: _renew_subs 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 1 -- 172.21.15.190:0/644094952 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620c09d250 con 0x7f620c0a68e0 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f6211e27640 1 -- 172.21.15.190:0/644094952 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f620c0a6d20 con 0x7f620c0a68e0 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.649+0000 7f61f3fff640 1 -- 172.21.15.190:0/644094952 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f61fc01a070 con 0x7f620c0a68e0 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.653+0000 7f61f3fff640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f61f803be00 0x7f61f803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.653+0000 7f61f3fff640 1 -- 172.21.15.190:0/644094952 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6643+0+0 (secure 0 0 0) 0x7f61fc00f040 con 0x7f620c0a68e0 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.653+0000 7f620b7fe640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f61f803be00 0x7f61f803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.653+0000 7f620b7fe640 10 monclient: get_auth_request con 0x7f61f803be00 auth_method 0 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.653+0000 7f620b7fe640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f61f803be00 0x7f61f803e2c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6200003990 tx=0x7f6200008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:41:59.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.657+0000 7f61f3fff640 1 -- 172.21.15.190:0/644094952 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f61fc021070 con 0x7f620c0a68e0 2024-08-19T10:41:59.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.657+0000 7f61f3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:41:59.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.657+0000 7f61f3fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:41:59.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.809+0000 7f6211e27640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:59.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.809+0000 7f6211e27640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:59.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.809+0000 7f6211e27640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:59.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.809+0000 7f6211e27640 1 -- 172.21.15.190:0/644094952 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f620c0a2390 con 0x7f620c0a68e0 2024-08-19T10:41:59.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.813+0000 7f61f3fff640 1 -- 172.21.15.190:0/644094952 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v118) ==== 56+0+1449 (secure 0 0 0) 0x7f61fc04f110 con 0x7f620c0a68e0 2024-08-19T10:41:59.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.813+0000 7f61f3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:41:59.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.813+0000 7f61f3fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:41:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:41:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:41:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 -- 172.21.15.190:0/644094952 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f620c13f5b0 con 0x7f620c0a68e0 2024-08-19T10:41:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f61f3fff640 1 -- 172.21.15.190:0/644094952 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v118) ==== 56+0+1449 (secure 0 0 0) 0x7f61fc01ac80 con 0x7f620c0a68e0 2024-08-19T10:41:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f61f3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:41:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f61f3fff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:41:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 -- 172.21.15.190:0/644094952 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f61f803be00 msgr2=0x7f61f803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f61f803be00 0x7f61f803e2c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6200003990 tx=0x7f6200008040 comp rx=0 tx=0).stop 2024-08-19T10:41:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 10 monclient: shutdown 2024-08-19T10:41:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:41:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 -- 172.21.15.190:0/644094952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0a68e0 msgr2=0x7f620c13cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:41:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0a68e0 0x7f620c13cf60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00e910 tx=0x7f61fc00edd0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 -- 172.21.15.190:0/644094952 shutdown_connections 2024-08-19T10:41:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f61f803be00 0x7f61f803e2c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f620c139be0 0x7f620c13d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f620c0a68e0 0x7f620c13cf60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 --2- 172.21.15.190:0/644094952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c0a5340 0x7f620c131dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:41:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 -- 172.21.15.190:0/644094952 >> 172.21.15.190:0/644094952 conn(0x7f620c074bd0 msgr2=0x7f620c0a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:41:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 -- 172.21.15.190:0/644094952 shutdown_connections 2024-08-19T10:41:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:41:59.905+0000 7f6211e27640 1 -- 172.21.15.190:0/644094952 wait complete. 2024-08-19T10:41:59.925 INFO:tasks.workunit.client.0.smithi190.stdout: 14 123.00000 osd.14 down 1.00000 1.00000 2024-08-19T10:41:59.925 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush reweight-subtree foobaz 155 2024-08-19T10:42:00.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.033+0000 7fd62c0de640 10 monclient: get_monmap_and_config 2024-08-19T10:42:00.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.033+0000 7fd62c0de640 10 monclient: build_initial_monmap 2024-08-19T10:42:00.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.033+0000 7fd62c0de640 10 monclient: monmap: 2024-08-19T10:42:00.038 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:00.038 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:00.038 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:00.039169+0000 2024-08-19T10:42:00.039 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:00.039169+0000 2024-08-19T10:42:00.039 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:00.039 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:00.039 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:00.039 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:00.039 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:00.039 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:00.039 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:00.039 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:00.039 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:00.039 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:00.040 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:00.040 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:00.040 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:00.040 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:00.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 1 Processor -- start 2024-08-19T10:42:00.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 1 -- start start 2024-08-19T10:42:00.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 10 monclient: init 2024-08-19T10:42:00.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 10 monclient: _add_conns ranks=[8,2,7,4,10,5,3,6,9,1,0] 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd6240a1cb0 0x7fd624133010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 10 monclient(hunting): picked mon.noname-e con 0x7fd6240a1cb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6240a2090 0x7fd6240a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 10 monclient(hunting): picked mon.noname-b con 0x7fd6240a2090 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6240a3cf0 0x7fd6240a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 10 monclient(hunting): picked mon.noname-i con 0x7fd6240a3cf0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd624069900 con 0x7fd6240a2090 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd62409ef70 con 0x7fd6240a3cf0 2024-08-19T10:42:00.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd62409e9b0 con 0x7fd6240a1cb0 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62c0de640 10 monclient(hunting): authenticate will time out at 1165.609471s 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62a67b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6240a3cf0 0x7fd6240a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6240a2090 0x7fd6240a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6240a2090 0x7fd6240a2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:51870/0 (socket says 172.21.15.190:51870) 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 1 -- 172.21.15.190:0/424642623 learned_addr learned my addr 172.21.15.190:0/424642623 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 10 monclient(hunting): get_auth_request con 0x7fd6240a2090 auth_method 0 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629e7a640 1 --2- 172.21.15.190:0/424642623 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd6240a1cb0 0x7fd624133010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:00.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62a67b640 10 monclient(hunting): get_auth_request con 0x7fd6240a3cf0 auth_method 0 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62a67b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62a67b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62a67b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629e7a640 10 monclient(hunting): get_auth_request con 0x7fd6240a1cb0 auth_method 0 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629e7a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629e7a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629e7a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62a67b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62a67b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd62a67b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:00.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629e7a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629e7a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629e7a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 10 monclient(hunting): handle_auth_done global_id 5393 payload 274 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 1 -- 172.21.15.190:0/424642623 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd6240a1cb0 msgr2=0x7fd624133010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 1 --2- 172.21.15.190:0/424642623 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd6240a1cb0 0x7fd624133010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 1 -- 172.21.15.190:0/424642623 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6240a3cf0 msgr2=0x7fd6240a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 1 --2- 172.21.15.190:0/424642623 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6240a3cf0 0x7fd6240a4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 10 monclient: _finish_hunting 0 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 1 monclient: found mon.noname-b 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:42:00.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.037+0000 7fd629679640 1 -- 172.21.15.190:0/424642623 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62409ec90 con 0x7fd6240a2090 2024-08-19T10:42:00.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd629679640 1 -- 172.21.15.190:0/424642623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6240a2090 msgr2=0x7fd6240a2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:00.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd629679640 1 --2- 172.21.15.190:0/424642623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6240a2090 0x7fd6240a2470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd6180098d0 tx=0x7fd61801c120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928222af7160370e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:00.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 1 -- 172.21.15.190:0/424642623 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6180b1070 con 0x7fd6240a2090 2024-08-19T10:42:00.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:00.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:42:00.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 10 monclient: dump: 2024-08-19T10:42:00.050 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:00.050 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:00.051 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 10 monclient: _finish_auth 0 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 10 monclient: _check_auth_tickets 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 1 -- 172.21.15.190:0/424642623 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6180a49f0 con 0x7fd6240a2090 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 5 monclient: authenticate success, global_id 5393 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 1 -- 172.21.15.190:0/424642623 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6180a4be0 con 0x7fd6240a2090 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 4 set_mon_vals no callback set 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:00.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd628e78640 10 monclient: dump: 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:00.053 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 10 monclient: shutdown 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 1 -- 172.21.15.190:0/424642623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6240a2090 msgr2=0x7fd6240a2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 1 --2- 172.21.15.190:0/424642623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6240a2090 0x7fd6240a2470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd6180098d0 tx=0x7fd61801c120 comp rx=0 tx=0).stop 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 1 -- 172.21.15.190:0/424642623 shutdown_connections 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 1 --2- 172.21.15.190:0/424642623 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd6240a3cf0 0x7fd6240a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 1 --2- 172.21.15.190:0/424642623 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd6240a2090 0x7fd6240a2470 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 1 --2- 172.21.15.190:0/424642623 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd6240a1cb0 0x7fd624133010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 1 -- 172.21.15.190:0/424642623 >> 172.21.15.190:0/424642623 conn(0x7fd6240a4de0 msgr2=0x7fd6240a51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:00.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 1 -- 172.21.15.190:0/424642623 shutdown_connections 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.041+0000 7fd62c0de640 1 -- 172.21.15.190:0/424642623 wait complete. 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient: build_initial_monmap 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient: monmap: 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:00.055 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 1 Processor -- start 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 1 -- start start 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient: init 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient: _add_conns ranks=[0,6,1,4,5,8,3,7,2] 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6240a1cb0 0x7fd6241cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient(hunting): picked mon.noname-a con 0x7fd6240a1cb0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6240a2090 0x7fd62413bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:00.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient(hunting): picked mon.noname-d con 0x7fd6240a2090 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6240a3cf0 0x7fd62413c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient(hunting): picked mon.noname-f con 0x7fd6240a3cf0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd62413fde0 con 0x7fd6240a1cb0 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd62413fc60 con 0x7fd6240a3cf0 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd62406b6c0 con 0x7fd6240a2090 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62c0de640 10 monclient(hunting): authenticate will time out at 1165.617310s 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6240a3cf0 0x7fd62413c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6240a3cf0 0x7fd62413c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:53822/0 (socket says 172.21.15.190:53822) 2024-08-19T10:42:00.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 1 -- 172.21.15.190:0/936361067 learned_addr learned my addr 172.21.15.190:0/936361067 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 10 monclient(hunting): get_auth_request con 0x7fd6240a3cf0 auth_method 0 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd629e7a640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6240a1cb0 0x7fd6241cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd629e7a640 10 monclient(hunting): get_auth_request con 0x7fd6240a1cb0 auth_method 0 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd629e7a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd629e7a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd629e7a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:00.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd629e7a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd629e7a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd629679640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6240a2090 0x7fd62413bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd629e7a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 10 monclient(hunting): handle_auth_done global_id 14707 payload 824 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 1 -- 172.21.15.190:0/936361067 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6240a2090 msgr2=0x7fd62413bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6240a2090 0x7fd62413bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 1 -- 172.21.15.190:0/936361067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6240a1cb0 msgr2=0x7fd6241cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6240a1cb0 0x7fd6241cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 1 monclient: found mon.noname-f 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:00.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd62a67b640 1 -- 172.21.15.190:0/936361067 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd624139a90 con 0x7fd6240a3cf0 2024-08-19T10:42:00.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd629679640 10 monclient: get_auth_request con 0x7fd6240a2090 auth_method 0 2024-08-19T10:42:00.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.045+0000 7fd629679640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6240a2090 0x7fd62413bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:00.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62a67b640 1 -- 172.21.15.190:0/936361067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6240a3cf0 msgr2=0x7fd62413c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:00.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62a67b640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6240a3cf0 0x7fd62413c510 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd614009e30 tx=0x7fd61400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:00.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 1 -- 172.21.15.190:0/936361067 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd614018020 con 0x7fd6240a3cf0 2024-08-19T10:42:00.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:00.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:00.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 10 monclient: dump: 2024-08-19T10:42:00.062 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:00.062 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:00.062 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:00.063 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:00.063 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:00.063 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:00.063 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:00.063 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:00.063 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:00.063 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:00.063 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:00.063 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62c0de640 5 monclient: authenticate success, global_id 14707 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62c0de640 10 monclient: _renew_subs 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62c0de640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62c0de640 1 -- 172.21.15.190:0/936361067 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd624138750 con 0x7fd6240a3cf0 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 1 -- 172.21.15.190:0/936361067 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd614014480 con 0x7fd6240a3cf0 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:00.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 1 -- 172.21.15.190:0/936361067 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd614014aa0 con 0x7fd6240a3cf0 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 10 monclient: dump: 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:00.065 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6237fe640 4 set_mon_vals no callback set 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62c0de640 10 monclient: _renew_subs 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62c0de640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62c0de640 1 -- 172.21.15.190:0/936361067 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6241391f0 con 0x7fd6240a3cf0 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62c0de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62c0de640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62c0de640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:00.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd62c0de640 1 -- 172.21.15.190:0/936361067 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6240a2470 con 0x7fd6240a3cf0 2024-08-19T10:42:00.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 1 -- 172.21.15.190:0/936361067 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd61401b020 con 0x7fd6240a3cf0 2024-08-19T10:42:00.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd61003be30 0x7fd61003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:00.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd6227fc640 1 -- 172.21.15.190:0/936361067 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6643+0+0 (secure 0 0 0) 0x7fd61404ff00 con 0x7fd6240a3cf0 2024-08-19T10:42:00.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd629e7a640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd61003be30 0x7fd61003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:00.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.049+0000 7fd629e7a640 10 monclient: get_auth_request con 0x7fd61003be30 auth_method 0 2024-08-19T10:42:00.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.053+0000 7fd629e7a640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd61003be30 0x7fd61003e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd61c013340 tx=0x7fd61c00d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:00.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.053+0000 7fd6227fc640 1 -- 172.21.15.190:0/936361067 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd614022030 con 0x7fd6240a3cf0 2024-08-19T10:42:00.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.053+0000 7fd6227fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:00.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.053+0000 7fd6227fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:00.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.229+0000 7fd62c0de640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-08-19T10:42:00.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.229+0000 7fd62c0de640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-08-19T10:42:00.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.229+0000 7fd62c0de640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:00.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.229+0000 7fd62c0de640 1 -- 172.21.15.190:0/936361067 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0} v 0) -- 0x7fd6240a7a10 con 0x7fd6240a3cf0 2024-08-19T10:42:00.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.257+0000 7fd6227fc640 1 -- 172.21.15.190:0/936361067 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}]=0 reweighted subtree id -10 name 'foobaz' to 155 in crush map v119) ==== 168+0+0 (secure 0 0 0) 0x7fd61404f020 con 0x7fd6240a3cf0 2024-08-19T10:42:00.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.257+0000 7fd6227fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-08-19T10:42:00.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.257+0000 7fd6227fc640 10 monclient: _finish_command 2 = system:0 reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-08-19T10:42:00.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.373+0000 7fd62c0de640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-08-19T10:42:00.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.373+0000 7fd62c0de640 10 monclient: _send_command 3 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-08-19T10:42:00.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.373+0000 7fd62c0de640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:00.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:00.373+0000 7fd62c0de640 1 -- 172.21.15.190:0/936361067 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0} v 0) -- 0x7fd6240008e0 con 0x7fd6240a3cf0 2024-08-19T10:42:01.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.277+0000 7fd6227fc640 1 -- 172.21.15.190:0/936361067 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}]=0 reweighted subtree id -10 name 'foobaz' to 155 in crush map v120) ==== 168+0+0 (secure 0 0 0) 0x7fd614010040 con 0x7fd6240a3cf0 2024-08-19T10:42:01.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.277+0000 7fd6227fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 155.0}] 2024-08-19T10:42:01.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.277+0000 7fd6227fc640 10 monclient: _finish_command 3 = system:0 reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-08-19T10:42:01.280 INFO:tasks.workunit.client.0.smithi190.stderr:reweighted subtree id -10 name 'foobaz' to 155 in crush map 2024-08-19T10:42:01.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.277+0000 7fd62c0de640 1 -- 172.21.15.190:0/936361067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd61003be30 msgr2=0x7fd61003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:01.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.277+0000 7fd62c0de640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd61003be30 0x7fd61003e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd61c013340 tx=0x7fd61c00d040 comp rx=0 tx=0).stop 2024-08-19T10:42:01.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.277+0000 7fd62c0de640 10 monclient: shutdown 2024-08-19T10:42:01.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.277+0000 7fd62c0de640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:01.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.277+0000 7fd62c0de640 1 -- 172.21.15.190:0/936361067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6240a3cf0 msgr2=0x7fd62413c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:01.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.277+0000 7fd62c0de640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6240a3cf0 0x7fd62413c510 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd614009e30 tx=0x7fd61400c490 comp rx=0 tx=0).stop 2024-08-19T10:42:01.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.281+0000 7fd62c0de640 1 -- 172.21.15.190:0/936361067 shutdown_connections 2024-08-19T10:42:01.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.281+0000 7fd62c0de640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd61003be30 0x7fd61003e2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.281+0000 7fd62c0de640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6240a3cf0 0x7fd62413c510 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.281+0000 7fd62c0de640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6240a2090 0x7fd62413bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.281+0000 7fd62c0de640 1 --2- 172.21.15.190:0/936361067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6240a1cb0 0x7fd6241cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.281+0000 7fd62c0de640 1 -- 172.21.15.190:0/936361067 >> 172.21.15.190:0/936361067 conn(0x7fd6240a4de0 msgr2=0x7fd6240a7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:01.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.281+0000 7fd62c0de640 1 -- 172.21.15.190:0/936361067 shutdown_connections 2024-08-19T10:42:01.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.281+0000 7fd62c0de640 1 -- 172.21.15.190:0/936361067 wait complete. 2024-08-19T10:42:01.303 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush reweight-subtree foobaz 123456 2024-08-19T10:42:01.304 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:42:01.304 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush reweight-subtree foobaz 123456 2024-08-19T10:42:01.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.421+0000 7fed838be640 10 monclient: get_monmap_and_config 2024-08-19T10:42:01.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.421+0000 7fed838be640 10 monclient: build_initial_monmap 2024-08-19T10:42:01.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.421+0000 7fed838be640 10 monclient: monmap: 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:01.427010+0000 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:01.427010+0000 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:01.427 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:01.428 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:01.428 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:01.428 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:01.428 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:01.428 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 1 Processor -- start 2024-08-19T10:42:01.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 1 -- start start 2024-08-19T10:42:01.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 10 monclient: init 2024-08-19T10:42:01.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:01.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 10 monclient: _add_conns ranks=[6,5,4,1,8,10,7,0,2,9,3] 2024-08-19T10:42:01.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fed7c0a6740 0x7fed7c0a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 10 monclient(hunting): picked mon.noname-d con 0x7fed7c0a6740 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a5bc0 0x7fed7c0a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 10 monclient(hunting): picked mon.noname-h con 0x7fed7c0a5bc0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed7c0a7060 0x7fed7c139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 10 monclient(hunting): picked mon.noname-c con 0x7fed7c0a7060 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fed7c0a4c20 con 0x7fed7c0a7060 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fed7c0a3530 con 0x7fed7c0a5bc0 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fed7c0a3740 con 0x7fed7c0a6740 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed838be640 10 monclient(hunting): authenticate will time out at 1166.997170s 2024-08-19T10:42:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a5bc0 0x7fed7c0a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a5bc0 0x7fed7c0a6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:38612/0 (socket says 172.21.15.190:38612) 2024-08-19T10:42:01.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 1 -- 172.21.15.190:0/3345793908 learned_addr learned my addr 172.21.15.190:0/3345793908 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:01.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 10 monclient(hunting): get_auth_request con 0x7fed7c0a5bc0 auth_method 0 2024-08-19T10:42:01.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:01.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:01.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:01.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:01.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:01.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:01.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed81e5b640 1 --2- 172.21.15.190:0/3345793908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed7c0a7060 0x7fed7c139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 10 monclient(hunting): handle_auth_done global_id 5450 payload 274 2024-08-19T10:42:01.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed8165a640 1 --2- 172.21.15.190:0/3345793908 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fed7c0a6740 0x7fed7c0a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 1 -- 172.21.15.190:0/3345793908 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fed7c0a6740 msgr2=0x7fed7c0a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:01.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 1 --2- 172.21.15.190:0/3345793908 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fed7c0a6740 0x7fed7c0a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 1 -- 172.21.15.190:0/3345793908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed7c0a7060 msgr2=0x7fed7c139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:01.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 1 --2- 172.21.15.190:0/3345793908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed7c0a7060 0x7fed7c139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 10 monclient: _finish_hunting 0 2024-08-19T10:42:01.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 1 monclient: found mon.noname-h 2024-08-19T10:42:01.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:01.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:01.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed80e59640 1 -- 172.21.15.190:0/3345793908 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c0a3240 con 0x7fed7c0a5bc0 2024-08-19T10:42:01.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed81e5b640 10 monclient: get_auth_request con 0x7fed7c0a7060 auth_method 0 2024-08-19T10:42:01.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.425+0000 7fed81e5b640 1 --2- 172.21.15.190:0/3345793908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed7c0a7060 0x7fed7c139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:01.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed80e59640 1 -- 172.21.15.190:0/3345793908 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a5bc0 msgr2=0x7fed7c0a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed80e59640 1 --2- 172.21.15.190:0/3345793908 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a5bc0 0x7fed7c0a6b20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0098a0 tx=0x7fed6c01c440 comp rx=0 tx=0).ready entity=mon.5 client_cookie=3c1be7baa7a66e03 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 1 -- 172.21.15.190:0/3345793908 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fed6c0b0070 con 0x7fed7c0a5bc0 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 10 monclient: dump: 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:01.438 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 1 -- 172.21.15.190:0/3345793908 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6c0aa810 con 0x7fed7c0a5bc0 2024-08-19T10:42:01.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 1 -- 172.21.15.190:0/3345793908 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fed6c0aae30 con 0x7fed7c0a5bc0 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 5 monclient: authenticate success, global_id 5450 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 4 set_mon_vals no callback set 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed73fff640 10 monclient: dump: 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:01.440 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 10 monclient: shutdown 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:01.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 1 -- 172.21.15.190:0/3345793908 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a5bc0 msgr2=0x7fed7c0a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 1 --2- 172.21.15.190:0/3345793908 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a5bc0 0x7fed7c0a6b20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0098a0 tx=0x7fed6c01c440 comp rx=0 tx=0).stop 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 1 -- 172.21.15.190:0/3345793908 shutdown_connections 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 1 --2- 172.21.15.190:0/3345793908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed7c0a7060 0x7fed7c139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 1 --2- 172.21.15.190:0/3345793908 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a5bc0 0x7fed7c0a6b20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 1 --2- 172.21.15.190:0/3345793908 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fed7c0a6740 0x7fed7c0a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 1 -- 172.21.15.190:0/3345793908 >> 172.21.15.190:0/3345793908 conn(0x7fed7c0abd40 msgr2=0x7fed7c0ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 1 -- 172.21.15.190:0/3345793908 shutdown_connections 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 1 -- 172.21.15.190:0/3345793908 wait complete. 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 10 monclient: build_initial_monmap 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 10 monclient: monmap: 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:01.442 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:01.443 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:01.444 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.429+0000 7fed838be640 1 Processor -- start 2024-08-19T10:42:01.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 1 -- start start 2024-08-19T10:42:01.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 10 monclient: init 2024-08-19T10:42:01.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:01.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 10 monclient: _add_conns ranks=[7,5,4,3,8,6,0,1,2] 2024-08-19T10:42:01.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed7c0a6740 0x7fed7c135880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 10 monclient(hunting): picked mon.noname-i con 0x7fed7c0a6740 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:01.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a7060 0x7fed7c135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 10 monclient(hunting): picked mon.noname-h con 0x7fed7c0a7060 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed7c09d890 0x7fed7c09dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 10 monclient(hunting): picked mon.noname-c con 0x7fed7c09d890 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fed7c143450 con 0x7fed7c09d890 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fed7c1432d0 con 0x7fed7c0a7060 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fed7c1435d0 con 0x7fed7c0a6740 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 10 monclient(hunting): authenticate will time out at 1167.004641s 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed7c0a6740 0x7fed7c135880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed80e59640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a7060 0x7fed7c135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed80e59640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a7060 0x7fed7c135dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:38624/0 (socket says 172.21.15.190:38624) 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed80e59640 1 -- 172.21.15.190:0/3260824055 learned_addr learned my addr 172.21.15.190:0/3260824055 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed80e59640 10 monclient(hunting): get_auth_request con 0x7fed7c0a7060 auth_method 0 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed80e59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed80e59640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed80e59640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 10 monclient(hunting): get_auth_request con 0x7fed7c0a6740 auth_method 0 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:01.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed80e59640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed80e59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed80e59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed81e5b640 1 --2- 172.21.15.190:0/3260824055 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed7c09d890 0x7fed7c09dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed81e5b640 10 monclient(hunting): get_auth_request con 0x7fed7c09d890 auth_method 0 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed81e5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed81e5b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed81e5b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 10 monclient(hunting): handle_auth_done global_id 14614 payload 824 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 1 -- 172.21.15.190:0/3260824055 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a7060 msgr2=0x7fed7c135dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 1 --2- 172.21.15.190:0/3260824055 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a7060 0x7fed7c135dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 1 -- 172.21.15.190:0/3260824055 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed7c09d890 msgr2=0x7fed7c09dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:01.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 1 --2- 172.21.15.190:0/3260824055 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed7c09d890 0x7fed7c09dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 10 monclient: _finish_hunting 0 2024-08-19T10:42:01.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 1 monclient: found mon.noname-i 2024-08-19T10:42:01.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:01.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:01.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 1 -- 172.21.15.190:0/3260824055 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c0a1310 con 0x7fed7c0a6740 2024-08-19T10:42:01.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed80e59640 1 --2- 172.21.15.190:0/3260824055 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed7c0a7060 0x7fed7c135dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:01.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 1 -- 172.21.15.190:0/3260824055 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed7c0a6740 msgr2=0x7fed7c135880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:01.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed8165a640 1 --2- 172.21.15.190:0/3260824055 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed7c0a6740 0x7fed7c135880 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fed7800cdc0 tx=0x7fed7800a660 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:01.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 1 -- 172.21.15.190:0/3260824055 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fed7800ea70 con 0x7fed7c0a6740 2024-08-19T10:42:01.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:01.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:01.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 10 monclient: dump: 2024-08-19T10:42:01.450 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:01.450 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:01.450 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:01.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:01.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:01.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 1 -- 172.21.15.190:0/3260824055 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7800ec70 con 0x7fed7c0a6740 2024-08-19T10:42:01.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed838be640 5 monclient: authenticate success, global_id 14614 2024-08-19T10:42:01.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:01.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 1 -- 172.21.15.190:0/3260824055 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fed7800ee20 con 0x7fed7c0a6740 2024-08-19T10:42:01.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:01.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:01.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed71ffb640 10 monclient: dump: 2024-08-19T10:42:01.452 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:01.452 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:01.453 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:01.454 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.433+0000 7fed737fe640 4 set_mon_vals no callback set 2024-08-19T10:42:01.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed838be640 10 monclient: _renew_subs 2024-08-19T10:42:01.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed838be640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:01.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed838be640 1 -- 172.21.15.190:0/3260824055 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7c09f4f0 con 0x7fed7c0a6740 2024-08-19T10:42:01.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed838be640 10 monclient: _renew_subs 2024-08-19T10:42:01.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed838be640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:01.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed838be640 1 -- 172.21.15.190:0/3260824055 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7c13fd10 con 0x7fed7c0a6740 2024-08-19T10:42:01.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed838be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:01.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed838be640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:01.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed838be640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:01.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed838be640 1 -- 172.21.15.190:0/3260824055 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed7c0ab340 con 0x7fed7c0a6740 2024-08-19T10:42:01.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed71ffb640 1 -- 172.21.15.190:0/3260824055 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fed7800e600 con 0x7fed7c0a6740 2024-08-19T10:42:01.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed71ffb640 1 --2- 172.21.15.190:0/3260824055 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fed6003be00 0x7fed6003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed71ffb640 1 -- 172.21.15.190:0/3260824055 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(120..120 src has 1..120) ==== 6643+0+0 (secure 0 0 0) 0x7fed7804fce0 con 0x7fed7c0a6740 2024-08-19T10:42:01.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed80e59640 1 --2- 172.21.15.190:0/3260824055 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fed6003be00 0x7fed6003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed80e59640 10 monclient: get_auth_request con 0x7fed6003be00 auth_method 0 2024-08-19T10:42:01.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.437+0000 7fed80e59640 1 --2- 172.21.15.190:0/3260824055 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fed6003be00 0x7fed6003e2c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fed6c01c480 tx=0x7fed6c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:01.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.441+0000 7fed71ffb640 1 -- 172.21.15.190:0/3260824055 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fed78013ce0 con 0x7fed7c0a6740 2024-08-19T10:42:01.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.441+0000 7fed71ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:01.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.441+0000 7fed71ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.617+0000 7fed838be640 10 start_mon_command cmd=[{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-08-19T10:42:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.617+0000 7fed838be640 10 monclient: _send_command 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-08-19T10:42:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.617+0000 7fed838be640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:01.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.617+0000 7fed838be640 1 -- 172.21.15.190:0/3260824055 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0} v 0) -- 0x7fed7c0a5680 con 0x7fed7c0a6740 2024-08-19T10:42:01.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.805+0000 7fed71ffb640 1 -- 172.21.15.190:0/3260824055 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}]=-75 (75) Value too large for defined data type v120) ==== 154+0+0 (secure 0 0 0) 0x7fed780153d0 con 0x7fed7c0a6740 2024-08-19T10:42:01.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.805+0000 7fed71ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush reweight-subtree", "name": "foobaz", "weight": 123456.0}] 2024-08-19T10:42:01.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.805+0000 7fed71ffb640 10 monclient: _finish_command 2 = mon:75 (75) Value too large for defined data type 2024-08-19T10:42:01.810 INFO:tasks.workunit.client.0.smithi190.stderr:Error EOVERFLOW: (75) Value too large for defined data type 2024-08-19T10:42:01.816 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:42:01.816 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:42:01.816 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep osd.12 2024-08-19T10:42:01.817 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 155 2024-08-19T10:42:01.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient: get_monmap_and_config 2024-08-19T10:42:01.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient: build_initial_monmap 2024-08-19T10:42:01.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient: monmap: 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:01.929970+0000 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:01.929970+0000 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:01.930 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:01.931 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:01.931 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:01.931 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:01.931 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:01.931 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:01.931 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 1 Processor -- start 2024-08-19T10:42:01.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 1 -- start start 2024-08-19T10:42:01.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient: init 2024-08-19T10:42:01.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:01.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient: _add_conns ranks=[4,2,0,1,5,3,6,8,9,7,10] 2024-08-19T10:42:01.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a5340 0x7f5cc41396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient(hunting): picked mon.noname-c con 0x7f5cc40a5340 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:01.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cc4139be0 0x7f5cc4139fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient(hunting): picked mon.noname-b con 0x7f5cc4139be0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5cc40a68e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc40a8610 con 0x7f5cc40a68e0 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc40a2f30 con 0x7f5cc4139be0 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f5cc40a3500 con 0x7f5cc40a5340 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.925+0000 7f5cc93d6640 10 monclient(hunting): authenticate will time out at 1167.499980s 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a5340 0x7f5cc41396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc27fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cc4139be0 0x7f5cc4139fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc37fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5cc40a68e0 0x7f5cc40a6d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:34104/0 (socket says 172.21.15.190:34104) 2024-08-19T10:42:01.935 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc37fe640 1 -- 172.21.15.190:0/1806126407 learned_addr learned my addr 172.21.15.190:0/1806126407 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc27fc640 10 monclient(hunting): get_auth_request con 0x7f5cc4139be0 auth_method 0 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 10 monclient(hunting): get_auth_request con 0x7f5cc40a5340 auth_method 0 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:01.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:01.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:01.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 10 monclient(hunting): handle_auth_done global_id 5404 payload 274 2024-08-19T10:42:01.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 1 -- 172.21.15.190:0/1806126407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cc4139be0 msgr2=0x7f5cc4139fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:01.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 1 --2- 172.21.15.190:0/1806126407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cc4139be0 0x7f5cc4139fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 1 -- 172.21.15.190:0/1806126407 >> v1:172.21.15.89:6789/0 conn(0x7f5cc40a68e0 legacy=0x7f5cc40a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:01.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:42:01.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 1 monclient: found mon.noname-c 2024-08-19T10:42:01.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:01.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:01.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.929+0000 7f5cc2ffd640 1 -- 172.21.15.190:0/1806126407 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc40a3ff0 con 0x7f5cc40a5340 2024-08-19T10:42:01.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc2ffd640 1 -- 172.21.15.190:0/1806126407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a5340 msgr2=0x7f5cc41396a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:01.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc2ffd640 1 --2- 172.21.15.190:0/1806126407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a5340 0x7f5cc41396a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097f0 tx=0x7f5cb401c430 comp rx=0 tx=0).ready entity=mon.4 client_cookie=150cf14a35373648 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:01.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 1 -- 172.21.15.190:0/1806126407 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5cb40b0070 con 0x7f5cc40a5340 2024-08-19T10:42:01.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:01.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:01.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 10 monclient: dump: 2024-08-19T10:42:01.943 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:01.943 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:01.943 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.944 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 1 -- 172.21.15.190:0/1806126407 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb40aa810 con 0x7f5cc40a5340 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 1 -- 172.21.15.190:0/1806126407 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5cb40a9300 con 0x7f5cc40a5340 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc1ffb640 10 monclient: dump: 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.945 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 5 monclient: authenticate success, global_id 5404 2024-08-19T10:42:01.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 4 set_mon_vals no callback set 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 10 monclient: shutdown 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1806126407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a5340 msgr2=0x7f5cc41396a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 1 --2- 172.21.15.190:0/1806126407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a5340 0x7f5cc41396a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097f0 tx=0x7f5cb401c430 comp rx=0 tx=0).stop 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1806126407 shutdown_connections 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 1 --2- 172.21.15.190:0/1806126407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cc4139be0 0x7f5cc4139fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 1 --2- 172.21.15.190:0/1806126407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a5340 0x7f5cc41396a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1806126407 >> 172.21.15.190:0/1806126407 conn(0x7f5cc4074bd0 msgr2=0x7f5cc40abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1806126407 shutdown_connections 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1806126407 wait complete. 2024-08-19T10:42:01.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 10 monclient: build_initial_monmap 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 10 monclient: monmap: 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:01.948 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 1 Processor -- start 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 1 -- start start 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.933+0000 7f5cc93d6640 10 monclient: init 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient: _add_conns ranks=[7,4,2,3,1,0,5,6,8] 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5cc40a5340 0x7f5cc413fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient(hunting): picked mon.noname-i con 0x7f5cc40a5340 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a68e0 0x7f5cc41384e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient(hunting): picked mon.noname-c con 0x7f5cc40a68e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:01.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cc4139be0 0x7f5cc413d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient(hunting): picked mon.noname-b con 0x7f5cc4139be0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc40a8360 con 0x7f5cc4139be0 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f5cc40a5720 con 0x7f5cc40a68e0 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5cc4144330 con 0x7f5cc40a5340 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient(hunting): authenticate will time out at 1167.508530s 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5cc40a5340 0x7f5cc413fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5cc40a5340 0x7f5cc413fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:51654/0 (socket says 172.21.15.190:51654) 2024-08-19T10:42:01.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 1 -- 172.21.15.190:0/1452247552 learned_addr learned my addr 172.21.15.190:0/1452247552 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:01.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 10 monclient(hunting): get_auth_request con 0x7f5cc40a5340 auth_method 0 2024-08-19T10:42:01.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:01.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:01.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:01.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:01.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:01.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:01.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc27fc640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a68e0 0x7f5cc41384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc37fe640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cc4139be0 0x7f5cc413d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc27fc640 10 monclient(hunting): get_auth_request con 0x7f5cc40a68e0 auth_method 0 2024-08-19T10:42:01.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:01.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:01.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:01.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc37fe640 10 monclient(hunting): get_auth_request con 0x7f5cc4139be0 auth_method 0 2024-08-19T10:42:01.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:01.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:01.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:01.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 10 monclient(hunting): handle_auth_done global_id 14623 payload 824 2024-08-19T10:42:01.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 1 -- 172.21.15.190:0/1452247552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a68e0 msgr2=0x7f5cc41384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:01.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a68e0 0x7f5cc41384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 1 -- 172.21.15.190:0/1452247552 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cc4139be0 msgr2=0x7f5cc413d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cc4139be0 0x7f5cc413d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 1 monclient: found mon.noname-i 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 1 -- 172.21.15.190:0/1452247552 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc4140700 con 0x7f5cc40a5340 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 1 -- 172.21.15.190:0/1452247552 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5cc40a5340 msgr2=0x7f5cc413fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc2ffd640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5cc40a5340 0x7f5cc413fd70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009710 tx=0x7f5cb40191d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 1 -- 172.21.15.190:0/1452247552 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5cb40b0040 con 0x7f5cc40a5340 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:01.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 10 monclient: dump: 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:01.954 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 5 monclient: authenticate success, global_id 14623 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 1 -- 172.21.15.190:0/1452247552 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb40b9070 con 0x7f5cc40a5340 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient: _renew_subs 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 1 -- 172.21.15.190:0/1452247552 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5cb40044b0 con 0x7f5cc40a5340 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:01.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1452247552 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc4122e70 con 0x7f5cc40a5340 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc17fa640 4 set_mon_vals no callback set 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cab7fe640 10 monclient: dump: 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:01.956 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient: _renew_subs 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.937+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1452247552 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc40aafa0 con 0x7f5cc40a5340 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.941+0000 7f5cc93d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.941+0000 7f5cc93d6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:01.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.941+0000 7f5cc93d6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:01.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.941+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1452247552 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc413f210 con 0x7f5cc40a5340 2024-08-19T10:42:01.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.941+0000 7f5cab7fe640 1 -- 172.21.15.190:0/1452247552 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5cb4004040 con 0x7f5cc40a5340 2024-08-19T10:42:01.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.941+0000 7f5cab7fe640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5cb003be00 0x7f5cb003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:01.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.941+0000 7f5cab7fe640 1 -- 172.21.15.190:0/1452247552 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(120..120 src has 1..120) ==== 6643+0+0 (secure 0 0 0) 0x7f5cb40e86a0 con 0x7f5cc40a5340 2024-08-19T10:42:01.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.941+0000 7f5cc27fc640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5cb003be00 0x7f5cb003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:01.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.941+0000 7f5cc27fc640 10 monclient: get_auth_request con 0x7f5cb003be00 auth_method 0 2024-08-19T10:42:01.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.941+0000 7f5cc27fc640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5cb003be00 0x7f5cb003e2c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5cc413ee80 tx=0x7f5cb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:01.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.945+0000 7f5cab7fe640 1 -- 172.21.15.190:0/1452247552 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f5cb40c2060 con 0x7f5cc40a5340 2024-08-19T10:42:01.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.945+0000 7f5cab7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:01.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:01.945+0000 7f5cab7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:02.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.101+0000 7f5cc93d6640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:42:02.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.101+0000 7f5cc93d6640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:42:02.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.101+0000 7f5cc93d6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:02.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.101+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1452247552 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f5cc41396a0 con 0x7f5cc40a5340 2024-08-19T10:42:02.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.105+0000 7f5cab7fe640 1 -- 172.21.15.190:0/1452247552 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v120) ==== 56+0+1449 (secure 0 0 0) 0x7f5cb40160b0 con 0x7f5cc40a5340 2024-08-19T10:42:02.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.105+0000 7f5cab7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:42:02.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.105+0000 7f5cab7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:02.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.197+0000 7f5cc93d6640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:42:02.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.197+0000 7f5cc93d6640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:42:02.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.197+0000 7f5cc93d6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:02.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.197+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1452247552 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7f5cc40008e0 con 0x7f5cc40a5340 2024-08-19T10:42:02.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cab7fe640 1 -- 172.21.15.190:0/1452247552 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v120) ==== 56+0+1449 (secure 0 0 0) 0x7f5cb40bcc90 con 0x7f5cc40a5340 2024-08-19T10:42:02.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cab7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:42:02.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cab7fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:02.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1452247552 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5cb003be00 msgr2=0x7f5cb003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:02.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5cb003be00 0x7f5cb003e2c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5cc413ee80 tx=0x7f5cb8008040 comp rx=0 tx=0).stop 2024-08-19T10:42:02.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 10 monclient: shutdown 2024-08-19T10:42:02.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:02.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1452247552 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5cc40a5340 msgr2=0x7f5cc413fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:02.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5cc40a5340 0x7f5cc413fd70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009710 tx=0x7f5cb40191d0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1452247552 shutdown_connections 2024-08-19T10:42:02.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5cb003be00 0x7f5cb003e2c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5cc40a68e0 0x7f5cc41384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5cc4139be0 0x7f5cc413d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 --2- 172.21.15.190:0/1452247552 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5cc40a5340 0x7f5cc413fd70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1452247552 >> 172.21.15.190:0/1452247552 conn(0x7f5cc4074bd0 msgr2=0x7f5cc4141320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:02.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1452247552 shutdown_connections 2024-08-19T10:42:02.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.201+0000 7f5cc93d6640 1 -- 172.21.15.190:0/1452247552 wait complete. 2024-08-19T10:42:02.219 INFO:tasks.workunit.client.0.smithi190.stdout: 12 155.00000 osd.12 down 1.00000 1.00000 2024-08-19T10:42:02.219 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd tree 2024-08-19T10:42:02.220 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep osd.14 2024-08-19T10:42:02.220 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 155 2024-08-19T10:42:02.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient: get_monmap_and_config 2024-08-19T10:42:02.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient: build_initial_monmap 2024-08-19T10:42:02.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient: monmap: 2024-08-19T10:42:02.333 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:02.333 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:02.333 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:02.333954+0000 2024-08-19T10:42:02.333 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:02.333954+0000 2024-08-19T10:42:02.333 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:02.334 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:02.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 1 Processor -- start 2024-08-19T10:42:02.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 1 -- start start 2024-08-19T10:42:02.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient: init 2024-08-19T10:42:02.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:02.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient: _add_conns ranks=[5,3,10,0,6,2,4,7,1,8,9] 2024-08-19T10:42:02.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc93c139a60 0x7fc93c0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:02.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient(hunting): picked mon.noname-h con 0x7fc93c139a60 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc93c139e40 0x7fc93c0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient(hunting): picked mon.noname-g con 0x7fc93c139e40 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc93c13a220 0x7fc93c0a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient(hunting): picked mon.noname-f con 0x7fc93c13a220 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc93c0a5bb0 con 0x7fc93c13a220 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc93c0a8340 con 0x7fc93c139e40 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc93c0a4580 con 0x7fc93c139a60 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc940cb7640 10 monclient(hunting): authenticate will time out at 1167.903878s 2024-08-19T10:42:02.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc939d9b640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc93c139e40 msgr2=0x7fc93c0a68e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.329+0000 7fc939d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc93c139e40 0x7fc93c0a68e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc93c139a60 0x7fc93c0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93ad9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc93c13a220 0x7fc93c0a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc93c139a60 0x7fc93c0a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:38626/0 (socket says 172.21.15.190:38626) 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 1 -- 172.21.15.190:0/1452799231 learned_addr learned my addr 172.21.15.190:0/1452799231 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 10 monclient(hunting): get_auth_request con 0x7fc93c139a60 auth_method 0 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93ad9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc93c13a220 0x7fc93c0a6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:53838/0 (socket says 172.21.15.190:53838) 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93ad9d640 10 monclient(hunting): get_auth_request con 0x7fc93c13a220 auth_method 0 2024-08-19T10:42:02.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 10 monclient(hunting): handle_auth_done global_id 5468 payload 274 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 1 -- 172.21.15.190:0/1452799231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc93c139e40 msgr2=0x7fc93c0a68e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 1 --2- 172.21.15.190:0/1452799231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc93c139e40 0x7fc93c0a68e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 1 -- 172.21.15.190:0/1452799231 >> v2:172.21.15.190:3300/0 conn(0x7fc93c13a220 msgr2=0x7fc93c0a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 1 --2- 172.21.15.190:0/1452799231 >> v2:172.21.15.190:3300/0 conn(0x7fc93c13a220 0x7fc93c0a6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:42:02.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 1 monclient: found mon.noname-h 2024-08-19T10:42:02.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:02.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:02.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93a59c640 1 -- 172.21.15.190:0/1452799231 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c0a4860 con 0x7fc93c139a60 2024-08-19T10:42:02.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.333+0000 7fc93ad9d640 1 --2- 172.21.15.190:0/1452799231 >> v2:172.21.15.190:3300/0 conn(0x7fc93c13a220 0x7fc93c0a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:02.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93a59c640 1 -- 172.21.15.190:0/1452799231 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc93c139a60 msgr2=0x7fc93c0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:02.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93a59c640 1 --2- 172.21.15.190:0/1452799231 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc93c139a60 0x7fc93c0a5670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009e50 tx=0x7fc92c01c910 comp rx=0 tx=0).ready entity=mon.5 client_cookie=edc963704a1d1611 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:02.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 1 -- 172.21.15.190:0/1452799231 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc92c0090d0 con 0x7fc93c139a60 2024-08-19T10:42:02.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:02.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:02.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 10 monclient: dump: 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:02.346 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 10 monclient: _finish_auth 0 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 10 monclient: _check_auth_tickets 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 1 -- 172.21.15.190:0/1452799231 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92c0a5a50 con 0x7fc93c139a60 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 5 monclient: authenticate success, global_id 5468 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 1 -- 172.21.15.190:0/1452799231 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc92c0a5c00 con 0x7fc93c139a60 2024-08-19T10:42:02.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 4 set_mon_vals no callback set 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc93959a640 10 monclient: dump: 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:02.348 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 10 monclient: shutdown 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 1 -- 172.21.15.190:0/1452799231 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc93c139a60 msgr2=0x7fc93c0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 1 --2- 172.21.15.190:0/1452799231 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc93c139a60 0x7fc93c0a5670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009e50 tx=0x7fc92c01c910 comp rx=0 tx=0).stop 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 1 -- 172.21.15.190:0/1452799231 shutdown_connections 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 1 --2- 172.21.15.190:0/1452799231 >> v2:172.21.15.190:3300/0 conn(0x7fc93c13a220 0x7fc93c0a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 1 --2- 172.21.15.190:0/1452799231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc93c139e40 0x7fc93c0a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 1 --2- 172.21.15.190:0/1452799231 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc93c139a60 0x7fc93c0a5670 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 1 -- 172.21.15.190:0/1452799231 >> 172.21.15.190:0/1452799231 conn(0x7fc93c0abee0 msgr2=0x7fc93c0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 1 -- 172.21.15.190:0/1452799231 shutdown_connections 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.337+0000 7fc940cb7640 1 -- 172.21.15.190:0/1452799231 wait complete. 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient: build_initial_monmap 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient: monmap: 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:02.350 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 1 Processor -- start 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 1 -- start start 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient: init 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:02.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient: _add_conns ranks=[6,4,2,1,8,5,3,0,7] 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc93c139a60 0x7fc93c1348e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient(hunting): picked mon.noname-d con 0x7fc93c139a60 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc93c139e40 0x7fc93c134e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient(hunting): picked mon.noname-c con 0x7fc93c139e40 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc93c13a220 0x7fc93c13a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient(hunting): picked mon.noname-b con 0x7fc93c13a220 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc93c143290 con 0x7fc93c13a220 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc93c143110 con 0x7fc93c139e40 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fc93c143410 con 0x7fc93c139a60 2024-08-19T10:42:02.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc940cb7640 10 monclient(hunting): authenticate will time out at 1167.912964s 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93a59c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc93c139a60 0x7fc93c1348e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc93c13a220 0x7fc93c13a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc939d9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc93c139e40 0x7fc93c134e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc93c13a220 0x7fc93c13a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:51902/0 (socket says 172.21.15.190:51902) 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 1 -- 172.21.15.190:0/1530711228 learned_addr learned my addr 172.21.15.190:0/1530711228 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 10 monclient(hunting): get_auth_request con 0x7fc93c13a220 auth_method 0 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93a59c640 10 monclient(hunting): get_auth_request con 0x7fc93c139a60 auth_method 0 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:02.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc939d9b640 10 monclient(hunting): get_auth_request con 0x7fc93c139e40 auth_method 0 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc939d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc939d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc939d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc939d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc939d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:02.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc939d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:02.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 10 monclient(hunting): handle_auth_done global_id 5420 payload 824 2024-08-19T10:42:02.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 1 -- 172.21.15.190:0/1530711228 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc93c139a60 msgr2=0x7fc93c1348e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:02.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc93c139a60 0x7fc93c1348e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 1 -- 172.21.15.190:0/1530711228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc93c139e40 msgr2=0x7fc93c134e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:02.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc93c139e40 0x7fc93c134e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:02.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 10 monclient: _finish_hunting 0 2024-08-19T10:42:02.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 1 monclient: found mon.noname-b 2024-08-19T10:42:02.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:02.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:42:02.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.341+0000 7fc93ad9d640 1 -- 172.21.15.190:0/1530711228 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c0ab0d0 con 0x7fc93c13a220 2024-08-19T10:42:02.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc93ad9d640 1 -- 172.21.15.190:0/1530711228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc93c13a220 msgr2=0x7fc93c13a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:02.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc93ad9d640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc93c13a220 0x7fc93c13a6b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc93400e900 tx=0x7fc93400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:02.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 1 -- 172.21.15.190:0/1530711228 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc934017070 con 0x7fc93c13a220 2024-08-19T10:42:02.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:02.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:42:02.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 10 monclient: dump: 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:02.358 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 10 monclient: _finish_auth 0 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 1 -- 172.21.15.190:0/1530711228 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc934013470 con 0x7fc93c13a220 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc940cb7640 5 monclient: authenticate success, global_id 5420 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc940cb7640 10 monclient: _renew_subs 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc940cb7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc940cb7640 1 -- 172.21.15.190:0/1530711228 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93c0aad30 con 0x7fc93c13a220 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:02.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 1 -- 172.21.15.190:0/1530711228 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc934013a90 con 0x7fc93c13a220 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 10 monclient: dump: 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:02.360 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc938d99640 4 set_mon_vals no callback set 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc940cb7640 10 monclient: _renew_subs 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc940cb7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc940cb7640 1 -- 172.21.15.190:0/1530711228 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93c13e250 con 0x7fc93c13a220 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc940cb7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc940cb7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:02.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc940cb7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:02.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc940cb7640 1 -- 172.21.15.190:0/1530711228 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc93c0a3500 con 0x7fc93c13a220 2024-08-19T10:42:02.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 1 -- 172.21.15.190:0/1530711228 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc93401a070 con 0x7fc93c13a220 2024-08-19T10:42:02.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc91c03be00 0x7fc91c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:02.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc92affd640 1 -- 172.21.15.190:0/1530711228 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(120..120 src has 1..120) ==== 6643+0+0 (secure 0 0 0) 0x7fc93400f040 con 0x7fc93c13a220 2024-08-19T10:42:02.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc93a59c640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc91c03be00 0x7fc91c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:02.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.345+0000 7fc93a59c640 10 monclient: get_auth_request con 0x7fc91c03be00 auth_method 0 2024-08-19T10:42:02.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.349+0000 7fc93a59c640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc91c03be00 0x7fc91c03e2c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0acf00 tx=0x7fc92c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:02.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.357+0000 7fc92affd640 1 -- 172.21.15.190:0/1530711228 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc934013c90 con 0x7fc93c13a220 2024-08-19T10:42:02.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.357+0000 7fc92affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:02.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.357+0000 7fc92affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:02.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.529+0000 7fc940cb7640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:42:02.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.533+0000 7fc940cb7640 10 monclient: _send_command 2 [{"prefix": "osd tree"}] 2024-08-19T10:42:02.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.533+0000 7fc940cb7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:02.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:02.533+0000 7fc940cb7640 1 -- 172.21.15.190:0/1530711228 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fc93c06c9c0 con 0x7fc93c13a220 2024-08-19T10:42:03.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.257+0000 7fc92affd640 1 -- 172.21.15.190:0/1530711228 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v120) ==== 56+0+1449 (secure 0 0 0) 0x7fc93404cd80 con 0x7fc93c13a220 2024-08-19T10:42:03.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.257+0000 7fc92affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd tree"}] 2024-08-19T10:42:03.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.257+0000 7fc92affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:03.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.361+0000 7fc940cb7640 10 start_mon_command cmd=[{"prefix": "osd tree"}] 2024-08-19T10:42:03.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.361+0000 7fc940cb7640 10 monclient: _send_command 3 [{"prefix": "osd tree"}] 2024-08-19T10:42:03.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.361+0000 7fc940cb7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:03.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.361+0000 7fc940cb7640 1 -- 172.21.15.190:0/1530711228 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd tree"} v 0) -- 0x7fc93c135360 con 0x7fc93c13a220 2024-08-19T10:42:03.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.361+0000 7fc92affd640 1 -- 172.21.15.190:0/1530711228 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd tree"}]=0 v120) ==== 56+0+1449 (secure 0 0 0) 0x7fc93404d100 con 0x7fc93c13a220 2024-08-19T10:42:03.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.361+0000 7fc92affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd tree"}] 2024-08-19T10:42:03.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.361+0000 7fc92affd640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:03.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 -- 172.21.15.190:0/1530711228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc91c03be00 msgr2=0x7fc91c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:03.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc91c03be00 0x7fc91c03e2c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0acf00 tx=0x7fc92c0a7070 comp rx=0 tx=0).stop 2024-08-19T10:42:03.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 10 monclient: shutdown 2024-08-19T10:42:03.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:03.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 -- 172.21.15.190:0/1530711228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc93c13a220 msgr2=0x7fc93c13a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:03.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc93c13a220 0x7fc93c13a6b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc93400e900 tx=0x7fc93400edc0 comp rx=0 tx=0).stop 2024-08-19T10:42:03.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 -- 172.21.15.190:0/1530711228 shutdown_connections 2024-08-19T10:42:03.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc91c03be00 0x7fc91c03e2c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:03.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc93c13a220 0x7fc93c13a6b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:03.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc93c139e40 0x7fc93c134e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:03.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 --2- 172.21.15.190:0/1530711228 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc93c139a60 0x7fc93c1348e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:03.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 -- 172.21.15.190:0/1530711228 >> 172.21.15.190:0/1530711228 conn(0x7fc93c0abee0 msgr2=0x7fc93c135dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:03.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 -- 172.21.15.190:0/1530711228 shutdown_connections 2024-08-19T10:42:03.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.365+0000 7fc940cb7640 1 -- 172.21.15.190:0/1530711228 wait complete. 2024-08-19T10:42:03.383 INFO:tasks.workunit.client.0.smithi190.stdout: 14 155.00000 osd.14 down 1.00000 1.00000 2024-08-19T10:42:03.384 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm osd.12 2024-08-19T10:42:03.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.497+0000 7fb6f750e640 10 monclient: get_monmap_and_config 2024-08-19T10:42:03.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.497+0000 7fb6f750e640 10 monclient: build_initial_monmap 2024-08-19T10:42:03.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.497+0000 7fb6f750e640 10 monclient: monmap: 2024-08-19T10:42:03.502 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:03.502 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:03.502 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:03.502425+0000 2024-08-19T10:42:03.502 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:03.502425+0000 2024-08-19T10:42:03.502 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:03.502 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:03.502 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:03.502 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:03.502 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:03.503 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:03.503 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:03.503 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:03.503 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:03.503 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:03.503 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:03.503 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:03.503 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:03.503 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:03.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.497+0000 7fb6f750e640 1 Processor -- start 2024-08-19T10:42:03.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.497+0000 7fb6f750e640 1 -- start start 2024-08-19T10:42:03.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.497+0000 7fb6f750e640 10 monclient: init 2024-08-19T10:42:03.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:03.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 10 monclient: _add_conns ranks=[2,10,1,9,6,4,0,7,8,3,5] 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6f0139980 0x7fb6f0139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 10 monclient(hunting): picked mon.noname-b con 0x7fb6f0139980 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb6f013a2a0 0x7fb6f00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 10 monclient(hunting): picked mon.noname-f con 0x7fb6f013a2a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb6f00a5c70 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f00a8470 con 0x7fb6f00a5c70 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f009ca60 con 0x7fb6f013a2a0 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f00a3640 con 0x7fb6f0139980 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f750e640 10 monclient(hunting): authenticate will time out at 1169.072297s 2024-08-19T10:42:03.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb6f013a2a0 0x7fb6f00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:03.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f5aab640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fb6f00a5c70 0x7fb6f00a67e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:38898/0 (socket says 172.21.15.190:38898) 2024-08-19T10:42:03.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f5aab640 1 -- 172.21.15.190:0/2487470456 learned_addr learned my addr 172.21.15.190:0/2487470456 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:03.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 10 monclient(hunting): get_auth_request con 0x7fb6f013a2a0 auth_method 0 2024-08-19T10:42:03.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:03.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:03.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:03.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f52aa640 1 --2- 172.21.15.190:0/2487470456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6f0139980 0x7fb6f0139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:03.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:03.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:03.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:03.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f52aa640 10 monclient(hunting): get_auth_request con 0x7fb6f0139980 auth_method 0 2024-08-19T10:42:03.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f52aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:03.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f52aa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:03.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f52aa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:03.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 10 monclient(hunting): handle_auth_done global_id 14725 payload 274 2024-08-19T10:42:03.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 1 -- 172.21.15.190:0/2487470456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6f0139980 msgr2=0x7fb6f0139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:03.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 1 --2- 172.21.15.190:0/2487470456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6f0139980 0x7fb6f0139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:03.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 1 -- 172.21.15.190:0/2487470456 >> v1:172.21.15.190:6789/0 conn(0x7fb6f00a5c70 legacy=0x7fb6f00a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:03.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 10 monclient: _finish_hunting 0 2024-08-19T10:42:03.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 1 monclient: found mon.noname-f 2024-08-19T10:42:03.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:03.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:03.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f4aa9640 1 -- 172.21.15.190:0/2487470456 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f00a3850 con 0x7fb6f013a2a0 2024-08-19T10:42:03.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.501+0000 7fb6f52aa640 1 --2- 172.21.15.190:0/2487470456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6f0139980 0x7fb6f0139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:42:03.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f4aa9640 1 -- 172.21.15.190:0/2487470456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6f013a2a0 msgr2=0x7fb6f00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:03.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f4aa9640 1 --2- 172.21.15.190:0/2487470456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6f013a2a0 0x7fb6f00a5730 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00097f0 tx=0x7fb6e001c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e2616ee94c00333 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:03.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 1 -- 172.21.15.190:0/2487470456 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6e00b0070 con 0x7fb6f013a2a0 2024-08-19T10:42:03.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:03.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:03.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 10 monclient: dump: 2024-08-19T10:42:03.518 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:03.518 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:03.519 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:03.519 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:03.519 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:03.519 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:03.519 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:03.519 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:03.519 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:03.519 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:03.519 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:03.519 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:03.519 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:03.520 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:03.520 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:03.520 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:03.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:03.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:03.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:03.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 1 -- 172.21.15.190:0/2487470456 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e00aa810 con 0x7fb6f013a2a0 2024-08-19T10:42:03.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:03.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 1 -- 172.21.15.190:0/2487470456 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6e00a9300 con 0x7fb6f013a2a0 2024-08-19T10:42:03.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 5 monclient: authenticate success, global_id 14725 2024-08-19T10:42:03.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:03.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 4 set_mon_vals no callback set 2024-08-19T10:42:03.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:03.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:03.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6e7fff640 10 monclient: dump: 2024-08-19T10:42:03.522 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:03.522 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:03.522 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:03.522 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:03.522 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:03.522 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:03.522 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:03.522 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:03.523 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:03.523 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:03.523 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:03.523 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:03.523 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:03.523 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:03.523 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:03.524 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:03.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 10 monclient: shutdown 2024-08-19T10:42:03.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:03.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 1 -- 172.21.15.190:0/2487470456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6f013a2a0 msgr2=0x7fb6f00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:03.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 1 --2- 172.21.15.190:0/2487470456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6f013a2a0 0x7fb6f00a5730 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00097f0 tx=0x7fb6e001c360 comp rx=0 tx=0).stop 2024-08-19T10:42:03.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 1 -- 172.21.15.190:0/2487470456 shutdown_connections 2024-08-19T10:42:03.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 1 --2- 172.21.15.190:0/2487470456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6f013a2a0 0x7fb6f00a5730 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:03.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 1 --2- 172.21.15.190:0/2487470456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6f0139980 0x7fb6f0139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:03.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 1 -- 172.21.15.190:0/2487470456 >> 172.21.15.190:0/2487470456 conn(0x7fb6f00abde0 msgr2=0x7fb6f00ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:03.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 1 -- 172.21.15.190:0/2487470456 shutdown_connections 2024-08-19T10:42:03.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 1 -- 172.21.15.190:0/2487470456 wait complete. 2024-08-19T10:42:03.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 10 monclient: build_initial_monmap 2024-08-19T10:42:03.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 10 monclient: monmap: 2024-08-19T10:42:03.526 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:03.526 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:03.526 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:03.526 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:03.526 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:03.526 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:03.526 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 1 Processor -- start 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 1 -- start start 2024-08-19T10:42:03.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.505+0000 7fb6f750e640 10 monclient: init 2024-08-19T10:42:03.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:03.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient: _add_conns ranks=[3,6,5,8,2,0,7,4,1] 2024-08-19T10:42:03.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6f00a5c70 0x7fb6f01d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:03.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient(hunting): picked mon.noname-g con 0x7fb6f00a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:03.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6f0139980 0x7fb6f0140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:03.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient(hunting): picked mon.noname-d con 0x7fb6f0139980 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:03.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6f0140dd0 0x7fb6f00a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:03.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient(hunting): picked mon.noname-h con 0x7fb6f0140dd0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:03.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:03.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f0144280 con 0x7fb6f00a5c70 2024-08-19T10:42:03.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:03.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb6f0144100 con 0x7fb6f0140dd0 2024-08-19T10:42:03.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:03.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb6f0144400 con 0x7fb6f0139980 2024-08-19T10:42:03.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:03.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient(hunting): authenticate will time out at 1169.080384s 2024-08-19T10:42:03.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f52aa640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6f00a5c70 msgr2=0x7fb6f01d14a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:03.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f52aa640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6f00a5c70 0x7fb6f01d14a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:03.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6f0140dd0 0x7fb6f00a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:03.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6f0140dd0 0x7fb6f00a00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:38640/0 (socket says 172.21.15.190:38640) 2024-08-19T10:42:03.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 1 -- 172.21.15.190:0/2243793306 learned_addr learned my addr 172.21.15.190:0/2243793306 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:03.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 10 monclient(hunting): get_auth_request con 0x7fb6f0140dd0 auth_method 0 2024-08-19T10:42:03.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:03.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:03.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:03.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:03.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:03.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:03.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 10 monclient(hunting): handle_auth_done global_id 5477 payload 824 2024-08-19T10:42:03.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 1 -- 172.21.15.190:0/2243793306 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6f0139980 msgr2=0x7fb6f0140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6f0139980 0x7fb6f0140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 1 -- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6f00a5c70 msgr2=0x7fb6f01d14a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6f00a5c70 0x7fb6f01d14a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 10 monclient: _finish_hunting 0 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 1 monclient: found mon.noname-h 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 1 -- 172.21.15.190:0/2243793306 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f0134770 con 0x7fb6f0140dd0 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 1 -- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6f0140dd0 msgr2=0x7fb6f00a00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f5aab640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6f0140dd0 0x7fb6f00a00d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800caf0 tx=0x7fb6e800a060 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 1 -- 172.21.15.190:0/2243793306 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6e800acb0 con 0x7fb6f0140dd0 2024-08-19T10:42:03.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:03.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:03.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 10 monclient: dump: 2024-08-19T10:42:03.532 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:03.532 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:03.532 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:03.532 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:03.532 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:03.532 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:03.532 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:03.532 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:03.532 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 1 -- 172.21.15.190:0/2243793306 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e8012870 con 0x7fb6f0140dd0 2024-08-19T10:42:03.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 5 monclient: authenticate success, global_id 5477 2024-08-19T10:42:03.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient: _renew_subs 2024-08-19T10:42:03.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:03.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 1 -- 172.21.15.190:0/2243793306 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f013c350 con 0x7fb6f0140dd0 2024-08-19T10:42:03.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:03.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 1 -- 172.21.15.190:0/2243793306 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6e8007070 con 0x7fb6f0140dd0 2024-08-19T10:42:03.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:03.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:03.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e77fe640 4 set_mon_vals no callback set 2024-08-19T10:42:03.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6e5ffb640 10 monclient: dump: 2024-08-19T10:42:03.534 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:03.534 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:03.535 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:03.535 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:03.535 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:03.535 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:03.535 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:03.535 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:03.535 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:03.535 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:03.535 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:03.535 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:03.536 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:03.536 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:03.536 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:03.536 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:03.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient: _renew_subs 2024-08-19T10:42:03.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:03.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.509+0000 7fb6f750e640 1 -- 172.21.15.190:0/2243793306 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f00ac1c0 con 0x7fb6f0140dd0 2024-08-19T10:42:03.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.513+0000 7fb6f750e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:03.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.513+0000 7fb6f750e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:03.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.513+0000 7fb6f750e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:03.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.513+0000 7fb6f750e640 1 -- 172.21.15.190:0/2243793306 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f0139d60 con 0x7fb6f0140dd0 2024-08-19T10:42:03.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.513+0000 7fb6e5ffb640 1 -- 172.21.15.190:0/2243793306 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb6e8012400 con 0x7fb6f0140dd0 2024-08-19T10:42:03.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.513+0000 7fb6e5ffb640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6d403be00 0x7fb6d403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:03.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.513+0000 7fb6e5ffb640 1 -- 172.21.15.190:0/2243793306 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(120..120 src has 1..120) ==== 6643+0+0 (secure 0 0 0) 0x7fb6e804f8f0 con 0x7fb6f0140dd0 2024-08-19T10:42:03.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.513+0000 7fb6f52aa640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6d403be00 0x7fb6d403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:03.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.513+0000 7fb6f52aa640 10 monclient: get_auth_request con 0x7fb6d403be00 auth_method 0 2024-08-19T10:42:03.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.513+0000 7fb6f52aa640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6d403be00 0x7fb6d403e2c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009120 tx=0x7fb6ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:03.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.517+0000 7fb6e5ffb640 1 -- 172.21.15.190:0/2243793306 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb6f0134770 con 0x7fb6f0140dd0 2024-08-19T10:42:03.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.517+0000 7fb6e5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:03.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.517+0000 7fb6e5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:03.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.681+0000 7fb6f750e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.12"}] 2024-08-19T10:42:03.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.681+0000 7fb6f750e640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-08-19T10:42:03.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.681+0000 7fb6f750e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:03.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:03.681+0000 7fb6f750e640 1 -- 172.21.15.190:0/2243793306 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.12"} v 0) -- 0x7fb6f00a1bc0 con 0x7fb6f0140dd0 2024-08-19T10:42:04.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.409+0000 7fb6e5ffb640 1 -- 172.21.15.190:0/2243793306 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.12"}]=0 removed item id 12 name 'osd.12' from crush map v121) ==== 125+0+0 (secure 0 0 0) 0x7fb6e801a360 con 0x7fb6f0140dd0 2024-08-19T10:42:04.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.409+0000 7fb6e5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-08-19T10:42:04.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.409+0000 7fb6e5ffb640 10 monclient: _finish_command 2 = system:0 removed item id 12 name 'osd.12' from crush map 2024-08-19T10:42:04.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.517+0000 7fb6f750e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.12"}] 2024-08-19T10:42:04.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.517+0000 7fb6f750e640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-08-19T10:42:04.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.517+0000 7fb6f750e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:04.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.517+0000 7fb6f750e640 1 -- 172.21.15.190:0/2243793306 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.12"} v 0) -- 0x7fb6f00008e0 con 0x7fb6f0140dd0 2024-08-19T10:42:04.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.665+0000 7fb6e5ffb640 1 -- 172.21.15.190:0/2243793306 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.12"}]=0 device 'osd.12' does not appear in the crush map v121) ==== 126+0+0 (secure 0 0 0) 0x7fb6e8007250 con 0x7fb6f0140dd0 2024-08-19T10:42:04.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.665+0000 7fb6e5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.12"}] 2024-08-19T10:42:04.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.665+0000 7fb6e5ffb640 10 monclient: _finish_command 3 = system:0 device 'osd.12' does not appear in the crush map 2024-08-19T10:42:04.671 INFO:tasks.workunit.client.0.smithi190.stderr:device 'osd.12' does not appear in the crush map 2024-08-19T10:42:04.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 -- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6d403be00 msgr2=0x7fb6d403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:04.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6d403be00 0x7fb6d403e2c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009120 tx=0x7fb6ec008040 comp rx=0 tx=0).stop 2024-08-19T10:42:04.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 10 monclient: shutdown 2024-08-19T10:42:04.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:04.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 -- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6f0140dd0 msgr2=0x7fb6f00a00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:04.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6f0140dd0 0x7fb6f00a00d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800caf0 tx=0x7fb6e800a060 comp rx=0 tx=0).stop 2024-08-19T10:42:04.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 -- 172.21.15.190:0/2243793306 shutdown_connections 2024-08-19T10:42:04.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6d403be00 0x7fb6d403e2c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:04.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6f0140dd0 0x7fb6f00a00d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:04.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6f0139980 0x7fb6f0140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:04.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 --2- 172.21.15.190:0/2243793306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6f00a5c70 0x7fb6f01d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:04.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 -- 172.21.15.190:0/2243793306 >> 172.21.15.190:0/2243793306 conn(0x7fb6f00abde0 msgr2=0x7fb6f0133640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:04.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 -- 172.21.15.190:0/2243793306 shutdown_connections 2024-08-19T10:42:04.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.673+0000 7fb6f750e640 1 -- 172.21.15.190:0/2243793306 wait complete. 2024-08-19T10:42:04.695 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm osd.14 2024-08-19T10:42:04.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient: get_monmap_and_config 2024-08-19T10:42:04.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient: build_initial_monmap 2024-08-19T10:42:04.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient: monmap: 2024-08-19T10:42:04.819 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:04.819 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:04.819 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:04.819919+0000 2024-08-19T10:42:04.819 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:04.819919+0000 2024-08-19T10:42:04.819 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:04.820 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:04.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 1 Processor -- start 2024-08-19T10:42:04.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 1 -- start start 2024-08-19T10:42:04.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient: init 2024-08-19T10:42:04.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:04.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient: _add_conns ranks=[0,2,1,7,3,6,9,5,4,8,10] 2024-08-19T10:42:04.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc43c0a6740 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:04.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c0a6b20 0x7fc43c0a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:04.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient(hunting): picked mon.noname-b con 0x7fc43c0a6b20 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:04.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc43c13eaa0 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:04.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc43c0a5d90 con 0x7fc43c0a6740 2024-08-19T10:42:04.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc43c0a3740 con 0x7fc43c13eaa0 2024-08-19T10:42:04.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:04.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc43c0a4c20 con 0x7fc43c0a6b20 2024-08-19T10:42:04.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:04.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc44458e640 10 monclient(hunting): authenticate will time out at 1170.389819s 2024-08-19T10:42:04.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc442b2b640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fc43c13eaa0 0x7fc43c140e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:38906/0 (socket says 172.21.15.190:38906) 2024-08-19T10:42:04.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc442b2b640 1 -- 172.21.15.190:0/1447041699 learned_addr learned my addr 172.21.15.190:0/1447041699 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:04.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 1 --2- 172.21.15.190:0/1447041699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c0a6b20 0x7fc43c0a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:04.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 10 monclient(hunting): get_auth_request con 0x7fc43c0a6b20 auth_method 0 2024-08-19T10:42:04.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:04.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:04.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:04.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:04.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:04.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:04.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 10 monclient(hunting): handle_auth_done global_id 5438 payload 274 2024-08-19T10:42:04.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 1 -- 172.21.15.190:0/1447041699 >> v1:172.21.15.190:6789/0 conn(0x7fc43c13eaa0 legacy=0x7fc43c140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:04.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 1 -- 172.21.15.190:0/1447041699 >> v1:172.21.15.89:6789/0 conn(0x7fc43c0a6740 legacy=0x7fc43c0a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:04.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 10 monclient: _finish_hunting 0 2024-08-19T10:42:04.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 1 monclient: found mon.noname-b 2024-08-19T10:42:04.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:04.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:42:04.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.817+0000 7fc441b29640 1 -- 172.21.15.190:0/1447041699 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c0a3240 con 0x7fc43c0a6b20 2024-08-19T10:42:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.821+0000 7fc441b29640 1 -- 172.21.15.190:0/1447041699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c0a6b20 msgr2=0x7fc43c0a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.821+0000 7fc441b29640 1 --2- 172.21.15.190:0/1447041699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c0a6b20 0x7fc43c0a6f00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009820 tx=0x7fc42c01c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8dc6785f8b2edab server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 1 -- 172.21.15.190:0/1447041699 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc42c0b0070 con 0x7fc43c0a6b20 2024-08-19T10:42:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:42:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 10 monclient: dump: 2024-08-19T10:42:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:04.830 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 10 monclient: _finish_auth 0 2024-08-19T10:42:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 10 monclient: _check_auth_tickets 2024-08-19T10:42:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 1 -- 172.21.15.190:0/1447041699 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c0aa810 con 0x7fc43c0a6b20 2024-08-19T10:42:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 5 monclient: authenticate success, global_id 5438 2024-08-19T10:42:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 4 set_mon_vals no callback set 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 1 -- 172.21.15.190:0/1447041699 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc42c0aae30 con 0x7fc43c0a6b20 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441328640 10 monclient: dump: 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:04.831 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:04.832 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:04.832 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:04.832 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:04.832 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:04.832 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:04.832 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:04.833 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient: shutdown 2024-08-19T10:42:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 -- 172.21.15.190:0/1447041699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c0a6b20 msgr2=0x7fc43c0a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 --2- 172.21.15.190:0/1447041699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c0a6b20 0x7fc43c0a6f00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009820 tx=0x7fc42c01c320 comp rx=0 tx=0).stop 2024-08-19T10:42:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 -- 172.21.15.190:0/1447041699 shutdown_connections 2024-08-19T10:42:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 --2- 172.21.15.190:0/1447041699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c0a6b20 0x7fc43c0a6f00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 -- 172.21.15.190:0/1447041699 >> 172.21.15.190:0/1447041699 conn(0x7fc43c0abd40 msgr2=0x7fc43c0ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 -- 172.21.15.190:0/1447041699 shutdown_connections 2024-08-19T10:42:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 -- 172.21.15.190:0/1447041699 wait complete. 2024-08-19T10:42:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient: build_initial_monmap 2024-08-19T10:42:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient: monmap: 2024-08-19T10:42:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:04.836 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:04.836 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:04.836 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:04.836 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:04.836 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:04.836 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:04.836 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:04.836 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:04.836 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:04.836 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:04.836 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:04.837 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:04.837 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 Processor -- start 2024-08-19T10:42:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 -- start start 2024-08-19T10:42:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient: init 2024-08-19T10:42:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient: _add_conns ranks=[0,6,2,8,5,7,3,1,4] 2024-08-19T10:42:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc43c0a6740 0x7fc43c1d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient(hunting): picked mon.noname-a con 0x7fc43c0a6740 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc43c0a6b20 0x7fc43c139620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient(hunting): picked mon.noname-d con 0x7fc43c0a6b20 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c13eaa0 0x7fc43c09f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient(hunting): picked mon.noname-b con 0x7fc43c13eaa0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c06b6c0 con 0x7fc43c0a6740 2024-08-19T10:42:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc43c144dd0 con 0x7fc43c13eaa0 2024-08-19T10:42:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fc43c144f50 con 0x7fc43c0a6b20 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44458e640 10 monclient(hunting): authenticate will time out at 1170.398851s 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc442b2b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c13eaa0 0x7fc43c09f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc43c0a6740 0x7fc43c1d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc442b2b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c13eaa0 0x7fc43c09f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:51916/0 (socket says 172.21.15.190:51916) 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc442b2b640 1 -- 172.21.15.190:0/3535810512 learned_addr learned my addr 172.21.15.190:0/3535810512 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc442b2b640 10 monclient(hunting): get_auth_request con 0x7fc43c13eaa0 auth_method 0 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc442b2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc442b2b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc442b2b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 10 monclient(hunting): get_auth_request con 0x7fc43c0a6740 auth_method 0 2024-08-19T10:42:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441b29640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc43c0a6b20 0x7fc43c139620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441b29640 10 monclient(hunting): get_auth_request con 0x7fc43c0a6b20 auth_method 0 2024-08-19T10:42:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441b29640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441b29640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441b29640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc442b2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc442b2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc442b2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441b29640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441b29640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc441b29640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 10 monclient(hunting): handle_auth_done global_id 5463 payload 824 2024-08-19T10:42:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 1 -- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc43c0a6b20 msgr2=0x7fc43c139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc43c0a6b20 0x7fc43c139620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 1 -- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c13eaa0 msgr2=0x7fc43c09f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:04.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c13eaa0 0x7fc43c09f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:04.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 10 monclient: _finish_hunting 0 2024-08-19T10:42:04.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 1 monclient: found mon.noname-a 2024-08-19T10:42:04.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.825+0000 7fc44232a640 1 -- 172.21.15.190:0/3535810512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c122cc0 con 0x7fc43c0a6740 2024-08-19T10:42:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44232a640 1 -- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc43c0a6740 msgr2=0x7fc43c1d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44232a640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc43c0a6740 0x7fc43c1d0f30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc43800c410 tx=0x7fc43800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 1 -- 172.21.15.190:0/3535810512 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc438017070 con 0x7fc43c0a6740 2024-08-19T10:42:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 10 monclient: dump: 2024-08-19T10:42:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:04.851 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 10 monclient: _finish_auth 0 2024-08-19T10:42:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 1 -- 172.21.15.190:0/3535810512 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc438006c50 con 0x7fc43c0a6740 2024-08-19T10:42:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44458e640 5 monclient: authenticate success, global_id 5463 2024-08-19T10:42:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 1 -- 172.21.15.190:0/3535810512 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc438006e40 con 0x7fc43c0a6740 2024-08-19T10:42:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc432ffd640 10 monclient: dump: 2024-08-19T10:42:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:04.853 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:04.853 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:04.853 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:04.853 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:04.853 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:04.853 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:04.853 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:04.853 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:04.853 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:04.853 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:04.854 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc440b27640 4 set_mon_vals no callback set 2024-08-19T10:42:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44458e640 10 monclient: _renew_subs 2024-08-19T10:42:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44458e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44458e640 1 -- 172.21.15.190:0/3535810512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43c13dc40 con 0x7fc43c0a6740 2024-08-19T10:42:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44458e640 10 monclient: _renew_subs 2024-08-19T10:42:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44458e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44458e640 1 -- 172.21.15.190:0/3535810512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc43c0ab150 con 0x7fc43c0a6740 2024-08-19T10:42:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44458e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44458e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44458e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.829+0000 7fc44458e640 1 -- 172.21.15.190:0/3535810512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc43c0a6f00 con 0x7fc43c0a6740 2024-08-19T10:42:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.833+0000 7fc432ffd640 1 -- 172.21.15.190:0/3535810512 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc4380067e0 con 0x7fc43c0a6740 2024-08-19T10:42:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.833+0000 7fc432ffd640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc42003be00 0x7fc42003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.833+0000 7fc432ffd640 1 -- 172.21.15.190:0/3535810512 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6621+0+0 (secure 0 0 0) 0x7fc43804e940 con 0x7fc43c0a6740 2024-08-19T10:42:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.833+0000 7fc441b29640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc42003be00 0x7fc42003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.833+0000 7fc441b29640 10 monclient: get_auth_request con 0x7fc42003be00 auth_method 0 2024-08-19T10:42:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.833+0000 7fc441b29640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc42003be00 0x7fc42003e2c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0ab000 tx=0x7fc42c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.845+0000 7fc432ffd640 1 -- 172.21.15.190:0/3535810512 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc438020020 con 0x7fc43c0a6740 2024-08-19T10:42:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.845+0000 7fc432ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:04.845+0000 7fc432ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:05.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.017+0000 7fc44458e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.14"}] 2024-08-19T10:42:05.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.021+0000 7fc44458e640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-08-19T10:42:05.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.021+0000 7fc44458e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:05.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.021+0000 7fc44458e640 1 -- 172.21.15.190:0/3535810512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.14"} v 0) -- 0x7fc43c0a1dc0 con 0x7fc43c0a6740 2024-08-19T10:42:05.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.425+0000 7fc432ffd640 1 -- 172.21.15.190:0/3535810512 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.14"}]=0 removed item id 14 name 'osd.14' from crush map v122) ==== 125+0+0 (secure 0 0 0) 0x7fc4380102e0 con 0x7fc43c0a6740 2024-08-19T10:42:05.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.425+0000 7fc432ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-08-19T10:42:05.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.425+0000 7fc432ffd640 10 monclient: _finish_command 2 = system:0 removed item id 14 name 'osd.14' from crush map 2024-08-19T10:42:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.537+0000 7fc44458e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "osd.14"}] 2024-08-19T10:42:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.537+0000 7fc44458e640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-08-19T10:42:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.537+0000 7fc44458e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.537+0000 7fc44458e640 1 -- 172.21.15.190:0/3535810512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "osd.14"} v 0) -- 0x7fc43c0008e0 con 0x7fc43c0a6740 2024-08-19T10:42:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc432ffd640 1 -- 172.21.15.190:0/3535810512 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "osd.14"}]=0 device 'osd.14' does not appear in the crush map v122) ==== 126+0+0 (secure 0 0 0) 0x7fc43803ae20 con 0x7fc43c0a6740 2024-08-19T10:42:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc432ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "osd.14"}] 2024-08-19T10:42:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc432ffd640 10 monclient: _finish_command 3 = system:0 device 'osd.14' does not appear in the crush map 2024-08-19T10:42:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:device 'osd.14' does not appear in the crush map 2024-08-19T10:42:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 -- 172.21.15.190:0/3535810512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc42003be00 msgr2=0x7fc42003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc42003be00 0x7fc42003e2c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0ab000 tx=0x7fc42c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 10 monclient: shutdown 2024-08-19T10:42:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 -- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc43c0a6740 msgr2=0x7fc43c1d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc43c0a6740 0x7fc43c1d0f30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc43800c410 tx=0x7fc43800c8d0 comp rx=0 tx=0).stop 2024-08-19T10:42:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 -- 172.21.15.190:0/3535810512 shutdown_connections 2024-08-19T10:42:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc42003be00 0x7fc42003e2c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc43c13eaa0 0x7fc43c09f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc43c0a6b20 0x7fc43c139620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 --2- 172.21.15.190:0/3535810512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc43c0a6740 0x7fc43c1d0f30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 -- 172.21.15.190:0/3535810512 >> 172.21.15.190:0/3535810512 conn(0x7fc43c0abd40 msgr2=0x7fc43c0a9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 -- 172.21.15.190:0/3535810512 shutdown_connections 2024-08-19T10:42:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.541+0000 7fc44458e640 1 -- 172.21.15.190:0/3535810512 wait complete. 2024-08-19T10:42:05.560 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd rm osd.12 2024-08-19T10:42:05.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient: get_monmap_and_config 2024-08-19T10:42:05.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient: build_initial_monmap 2024-08-19T10:42:05.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient: monmap: 2024-08-19T10:42:05.675 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:05.675 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:05.676 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:05.676089+0000 2024-08-19T10:42:05.676 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:05.676089+0000 2024-08-19T10:42:05.676 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:05.676 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:05.676 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:05.676 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:05.676 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:05.676 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:05.676 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:05.676 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:05.677 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:05.677 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:05.677 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:05.677 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:05.677 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:05.677 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:05.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 1 Processor -- start 2024-08-19T10:42:05.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 1 -- start start 2024-08-19T10:42:05.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient: init 2024-08-19T10:42:05.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:05.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient: _add_conns ranks=[2,0,5,7,9,4,10,3,8,1,6] 2024-08-19T10:42:05.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed9c1399d0 0x7fed9c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:05.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient(hunting): picked mon.noname-b con 0x7fed9c1399d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:05.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fed9c139db0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:05.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed9c13a190 0x7fed9c0a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:05.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient(hunting): picked mon.noname-h con 0x7fed9c13a190 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:05.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed9c0a46e0 con 0x7fed9c139db0 2024-08-19T10:42:05.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:05.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fed9c0a3470 con 0x7fed9c1399d0 2024-08-19T10:42:05.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:05.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fed9c0a49c0 con 0x7fed9c13a190 2024-08-19T10:42:05.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:05.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda400d640 10 monclient(hunting): authenticate will time out at 1171.246007s 2024-08-19T10:42:05.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed9c13a190 0x7fed9c0a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:05.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed9c13a190 0x7fed9c0a6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:43480/0 (socket says 172.21.15.190:43480) 2024-08-19T10:42:05.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 1 -- 172.21.15.190:0/3830421051 learned_addr learned my addr 172.21.15.190:0/3830421051 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:05.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 10 monclient(hunting): get_auth_request con 0x7fed9c13a190 auth_method 0 2024-08-19T10:42:05.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:05.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:05.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:05.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:05.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:05.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:05.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda1da9640 1 --2- 172.21.15.190:0/3830421051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed9c1399d0 0x7fed9c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:05.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda1da9640 10 monclient(hunting): get_auth_request con 0x7fed9c1399d0 auth_method 0 2024-08-19T10:42:05.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda1da9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:05.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda1da9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:05.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda1da9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:05.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 10 monclient(hunting): handle_auth_done global_id 5486 payload 274 2024-08-19T10:42:05.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 1 -- 172.21.15.190:0/3830421051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed9c1399d0 msgr2=0x7fed9c0a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:05.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 1 --2- 172.21.15.190:0/3830421051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed9c1399d0 0x7fed9c0a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:05.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 1 -- 172.21.15.190:0/3830421051 >> v1:172.21.15.89:6789/0 conn(0x7fed9c139db0 legacy=0x7fed9c0a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:05.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 10 monclient: _finish_hunting 0 2024-08-19T10:42:05.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 1 monclient: found mon.noname-h 2024-08-19T10:42:05.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:05.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:05.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.673+0000 7feda25aa640 1 -- 172.21.15.190:0/3830421051 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c0a4ca0 con 0x7fed9c13a190 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda25aa640 1 -- 172.21.15.190:0/3830421051 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed9c13a190 msgr2=0x7fed9c0a6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda25aa640 1 --2- 172.21.15.190:0/3830421051 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed9c13a190 0x7fed9c0a6f60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed94009e80 tx=0x7fed9401c910 comp rx=0 tx=0).ready entity=mon.5 client_cookie=99a65bb30b0583e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 1 -- 172.21.15.190:0/3830421051 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fed940090d0 con 0x7fed9c13a190 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 10 monclient: dump: 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:05.690 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 10 monclient: _finish_auth 0 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 10 monclient: _check_auth_tickets 2024-08-19T10:42:05.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 1 -- 172.21.15.190:0/3830421051 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed940a5b80 con 0x7fed9c13a190 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 5 monclient: authenticate success, global_id 5486 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 1 -- 172.21.15.190:0/3830421051 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fed940a5d30 con 0x7fed9c13a190 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 4 set_mon_vals no callback set 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda0da7640 10 monclient: dump: 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:05.692 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:05.693 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:05.694 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:05.694 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:05.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 10 monclient: shutdown 2024-08-19T10:42:05.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:05.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 1 -- 172.21.15.190:0/3830421051 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed9c13a190 msgr2=0x7fed9c0a6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:05.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 1 --2- 172.21.15.190:0/3830421051 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed9c13a190 0x7fed9c0a6f60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed94009e80 tx=0x7fed9401c910 comp rx=0 tx=0).stop 2024-08-19T10:42:05.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 1 -- 172.21.15.190:0/3830421051 shutdown_connections 2024-08-19T10:42:05.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 1 --2- 172.21.15.190:0/3830421051 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fed9c13a190 0x7fed9c0a6f60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:05.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 1 --2- 172.21.15.190:0/3830421051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fed9c1399d0 0x7fed9c0a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:05.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 1 -- 172.21.15.190:0/3830421051 >> 172.21.15.190:0/3830421051 conn(0x7fed9c0abe50 msgr2=0x7fed9c0ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:05.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 1 -- 172.21.15.190:0/3830421051 shutdown_connections 2024-08-19T10:42:05.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.677+0000 7feda400d640 1 -- 172.21.15.190:0/3830421051 wait complete. 2024-08-19T10:42:05.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient: build_initial_monmap 2024-08-19T10:42:05.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient: monmap: 2024-08-19T10:42:05.695 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:05.696 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 1 Processor -- start 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 1 -- start start 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient: init 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient: _add_conns ranks=[4,3,7,6,0,8,1,2,5] 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed9c1399d0 0x7fed9c13a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient(hunting): picked mon.noname-c con 0x7fed9c1399d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed9c139db0 0x7fed9c13ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient(hunting): picked mon.noname-g con 0x7fed9c139db0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed9c13a190 0x7fed9c13b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:05.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient(hunting): picked mon.noname-i con 0x7fed9c13a190 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:05.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:05.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fed9c0a7620 con 0x7fed9c139db0 2024-08-19T10:42:05.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:05.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fed9c0a74a0 con 0x7fed9c1399d0 2024-08-19T10:42:05.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:05.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fed9c144170 con 0x7fed9c13a190 2024-08-19T10:42:05.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:05.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda400d640 10 monclient(hunting): authenticate will time out at 1171.253263s 2024-08-19T10:42:05.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda15a8640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed9c139db0 msgr2=0x7fed9c13ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:05.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda15a8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed9c139db0 0x7fed9c13ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:05.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed9c13a190 0x7fed9c13b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:05.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed9c13a190 0x7fed9c13b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42196/0 (socket says 172.21.15.190:42196) 2024-08-19T10:42:05.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 1 -- 172.21.15.190:0/2253597202 learned_addr learned my addr 172.21.15.190:0/2253597202 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:05.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 10 monclient(hunting): get_auth_request con 0x7fed9c13a190 auth_method 0 2024-08-19T10:42:05.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:05.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:05.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:05.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:05.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:05.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:05.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 10 monclient(hunting): handle_auth_done global_id 14632 payload 824 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda1da9640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed9c1399d0 0x7fed9c13a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 1 -- 172.21.15.190:0/2253597202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed9c1399d0 msgr2=0x7fed9c13a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed9c1399d0 0x7fed9c13a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 1 -- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed9c139db0 msgr2=0x7fed9c13ac40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed9c139db0 0x7fed9c13ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 10 monclient: _finish_hunting 0 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 1 monclient: found mon.noname-i 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.681+0000 7feda25aa640 1 -- 172.21.15.190:0/2253597202 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c13d130 con 0x7fed9c13a190 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda25aa640 1 -- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed9c13a190 msgr2=0x7fed9c13b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:05.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda25aa640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed9c13a190 0x7fed9c13b180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed940af040 tx=0x7fed940196b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 1 -- 172.21.15.190:0/2253597202 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fed940bc020 con 0x7fed9c13a190 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 10 monclient: dump: 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:05.701 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 1 -- 172.21.15.190:0/2253597202 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed940044b0 con 0x7fed9c13a190 2024-08-19T10:42:05.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda400d640 5 monclient: authenticate success, global_id 14632 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda400d640 10 monclient: _renew_subs 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda400d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda400d640 1 -- 172.21.15.190:0/2253597202 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9c0a2790 con 0x7fed9c13a190 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 1 -- 172.21.15.190:0/2253597202 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fed94004660 con 0x7fed9c13a190 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8ffff640 4 set_mon_vals no callback set 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 10 monclient: dump: 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:05.703 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda400d640 10 monclient: _renew_subs 2024-08-19T10:42:05.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda400d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:05.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda400d640 1 -- 172.21.15.190:0/2253597202 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fed9c13ce50 con 0x7fed9c13a190 2024-08-19T10:42:05.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda400d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:05.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda400d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:05.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda400d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:05.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda400d640 1 -- 172.21.15.190:0/2253597202 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed9c0a5790 con 0x7fed9c13a190 2024-08-19T10:42:05.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 1 -- 172.21.15.190:0/2253597202 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fed940bf020 con 0x7fed9c13a190 2024-08-19T10:42:05.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fed8003be00 0x7fed8003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:05.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7fed8e7fc640 1 -- 172.21.15.190:0/2253597202 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(122..122 src has 1..122) ==== 6599+0+0 (secure 0 0 0) 0x7fed940b3040 con 0x7fed9c13a190 2024-08-19T10:42:05.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda1da9640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fed8003be00 0x7fed8003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:05.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.685+0000 7feda1da9640 10 monclient: get_auth_request con 0x7fed8003be00 auth_method 0 2024-08-19T10:42:05.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.689+0000 7feda1da9640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fed8003be00 0x7fed8003e2c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fed98009790 tx=0x7fed98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:05.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.689+0000 7fed8e7fc640 1 -- 172.21.15.190:0/2253597202 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fed940c6030 con 0x7fed9c13a190 2024-08-19T10:42:05.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.689+0000 7fed8e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:05.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.689+0000 7fed8e7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:05.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.865+0000 7feda400d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-08-19T10:42:05.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.865+0000 7feda400d640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-08-19T10:42:05.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.865+0000 7feda400d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:05.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:05.865+0000 7feda400d640 1 -- 172.21.15.190:0/2253597202 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.12"]} v 0) -- 0x7fed9c141010 con 0x7fed9c13a190 2024-08-19T10:42:05.872 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:05.868+0000 7f16cc065640 -1 mon.a@0(leader).osd e122 definitely_dead 0 2024-08-19T10:42:06.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.481+0000 7fed8e7fc640 1 -- 172.21.15.190:0/2253597202 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.12"]}]=0 removed osd.12 v123) ==== 87+0+0 (secure 0 0 0) 0x7fed94016620 con 0x7fed9c13a190 2024-08-19T10:42:06.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.481+0000 7fed8e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-08-19T10:42:06.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.481+0000 7fed8e7fc640 10 monclient: _finish_command 2 = system:0 removed osd.12 2024-08-19T10:42:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.601+0000 7feda400d640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-08-19T10:42:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.601+0000 7feda400d640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-08-19T10:42:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.601+0000 7feda400d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.601+0000 7feda400d640 1 -- 172.21.15.190:0/2253597202 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.12"]} v 0) -- 0x7fed9c0008e0 con 0x7fed9c13a190 2024-08-19T10:42:06.606 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:06.604+0000 7f16cc065640 -1 mon.a@0(leader).osd e123 definitely_dead 0 2024-08-19T10:42:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7fed8e7fc640 1 -- 172.21.15.190:0/2253597202 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.12"]}]=0 osd.12 does not exist. v123) ==== 96+0+0 (secure 0 0 0) 0x7fed940b7970 con 0x7fed9c13a190 2024-08-19T10:42:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7fed8e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.12"]}] 2024-08-19T10:42:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7fed8e7fc640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-08-19T10:42:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:osd.12 does not exist. 2024-08-19T10:42:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 -- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fed8003be00 msgr2=0x7fed8003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fed8003be00 0x7fed8003e2c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fed98009790 tx=0x7fed98008040 comp rx=0 tx=0).stop 2024-08-19T10:42:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 10 monclient: shutdown 2024-08-19T10:42:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 -- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed9c13a190 msgr2=0x7fed9c13b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed9c13a190 0x7fed9c13b180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed940af040 tx=0x7fed940196b0 comp rx=0 tx=0).stop 2024-08-19T10:42:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 -- 172.21.15.190:0/2253597202 shutdown_connections 2024-08-19T10:42:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fed8003be00 0x7fed8003e2c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fed9c13a190 0x7fed9c13b180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed9c139db0 0x7fed9c13ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 --2- 172.21.15.190:0/2253597202 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fed9c1399d0 0x7fed9c13a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 -- 172.21.15.190:0/2253597202 >> 172.21.15.190:0/2253597202 conn(0x7fed9c0abe50 msgr2=0x7fed9c134e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 -- 172.21.15.190:0/2253597202 shutdown_connections 2024-08-19T10:42:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.605+0000 7feda400d640 1 -- 172.21.15.190:0/2253597202 wait complete. 2024-08-19T10:42:06.625 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd rm osd.14 2024-08-19T10:42:06.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.737+0000 7ff8e7f15640 10 monclient: get_monmap_and_config 2024-08-19T10:42:06.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.737+0000 7ff8e7f15640 10 monclient: build_initial_monmap 2024-08-19T10:42:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.737+0000 7ff8e7f15640 10 monclient: monmap: 2024-08-19T10:42:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:06.743376+0000 2024-08-19T10:42:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:06.743376+0000 2024-08-19T10:42:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:06.743 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:06.744 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:06.744 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 1 Processor -- start 2024-08-19T10:42:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 1 -- start start 2024-08-19T10:42:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 10 monclient: init 2024-08-19T10:42:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 10 monclient: _add_conns ranks=[1,4,3,5,8,10,2,9,0,7,6] 2024-08-19T10:42:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff8e0137750 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e009ca60 0x7ff8e013c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:06.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 10 monclient(hunting): picked mon.noname-c con 0x7ff8e009ca60 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:06.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e006d090 0x7ff8e013eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:06.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 10 monclient(hunting): picked mon.noname-g con 0x7ff8e006d090 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:06.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8e006d510 con 0x7ff8e0137750 2024-08-19T10:42:06.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:06.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e00a2520 con 0x7ff8e006d090 2024-08-19T10:42:06.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:06.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff8e0074350 con 0x7ff8e009ca60 2024-08-19T10:42:06.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:06.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e7f15640 10 monclient(hunting): authenticate will time out at 1172.313414s 2024-08-19T10:42:06.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e64b2640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e006d090 msgr2=0x7ff8e013eb00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:06.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e64b2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e006d090 0x7ff8e013eb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:06.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e5cb1640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff8e0137750 0x7ff8e0137b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:42978/0 (socket says 172.21.15.190:42978) 2024-08-19T10:42:06.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e5cb1640 1 -- 172.21.15.190:0/2545081870 learned_addr learned my addr 172.21.15.190:0/2545081870 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:06.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 1 --2- 172.21.15.190:0/2545081870 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e009ca60 0x7ff8e013c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:06.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 10 monclient(hunting): get_auth_request con 0x7ff8e009ca60 auth_method 0 2024-08-19T10:42:06.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:06.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:06.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:06.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:06.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:06.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:06.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 10 monclient(hunting): handle_auth_done global_id 5431 payload 274 2024-08-19T10:42:06.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 1 -- 172.21.15.190:0/2545081870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e006d090 msgr2=0x7ff8e013eb00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:06.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 1 --2- 172.21.15.190:0/2545081870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e006d090 0x7ff8e013eb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:06.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 1 -- 172.21.15.190:0/2545081870 >> v1:172.21.15.190:6789/0 conn(0x7ff8e0137750 legacy=0x7ff8e0137b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 10 monclient: _finish_hunting 0 2024-08-19T10:42:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 1 monclient: found mon.noname-c 2024-08-19T10:42:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:06.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:06.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.741+0000 7ff8e54b0640 1 -- 172.21.15.190:0/2545081870 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e0073200 con 0x7ff8e009ca60 2024-08-19T10:42:06.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e54b0640 1 -- 172.21.15.190:0/2545081870 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e009ca60 msgr2=0x7ff8e013c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e54b0640 1 --2- 172.21.15.190:0/2545081870 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e009ca60 0x7ff8e013c5b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009820 tx=0x7ff8d001c3d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=87d1d4a7a2a71f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 1 -- 172.21.15.190:0/2545081870 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8d00b0070 con 0x7ff8e009ca60 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 10 monclient: dump: 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:06.757 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 10 monclient: _finish_auth 0 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 10 monclient: _check_auth_tickets 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:06.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 1 -- 172.21.15.190:0/2545081870 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8d00aa810 con 0x7ff8e009ca60 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e7f15640 5 monclient: authenticate success, global_id 5431 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 1 -- 172.21.15.190:0/2545081870 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8d00a93e0 con 0x7ff8e009ca60 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e7f15640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e7f15640 4 set_mon_vals no callback set 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e4caf640 10 monclient: dump: 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:06.759 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:06.760 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:06.761 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:06.761 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:06.761 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:06.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e7f15640 10 monclient: shutdown 2024-08-19T10:42:06.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e7f15640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:06.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e7f15640 1 -- 172.21.15.190:0/2545081870 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e009ca60 msgr2=0x7ff8e013c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:06.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e7f15640 1 --2- 172.21.15.190:0/2545081870 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e009ca60 0x7ff8e013c5b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009820 tx=0x7ff8d001c3d0 comp rx=0 tx=0).stop 2024-08-19T10:42:06.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e7f15640 1 -- 172.21.15.190:0/2545081870 shutdown_connections 2024-08-19T10:42:06.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e7f15640 1 --2- 172.21.15.190:0/2545081870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e006d090 0x7ff8e013eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:06.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e7f15640 1 --2- 172.21.15.190:0/2545081870 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e009ca60 0x7ff8e013c5b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:06.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.745+0000 7ff8e7f15640 1 -- 172.21.15.190:0/2545081870 >> 172.21.15.190:0/2545081870 conn(0x7ff8e00a93d0 msgr2=0x7ff8e00a56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:06.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 1 -- 172.21.15.190:0/2545081870 shutdown_connections 2024-08-19T10:42:06.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 1 -- 172.21.15.190:0/2545081870 wait complete. 2024-08-19T10:42:06.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient: build_initial_monmap 2024-08-19T10:42:06.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient: monmap: 2024-08-19T10:42:06.762 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:06.762 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:06.762 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:06.762 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 1 Processor -- start 2024-08-19T10:42:06.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 1 -- start start 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient: init 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient: _add_conns ranks=[4,8,3,2,6,5,1,7,0] 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e006d090 0x7ff8e01d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient(hunting): picked mon.noname-c con 0x7ff8e006d090 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8e009ca60 0x7ff8e0136900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient(hunting): picked mon.noname-e con 0x7ff8e009ca60 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e0137750 0x7ff8e0136e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient(hunting): picked mon.noname-g con 0x7ff8e0137750 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e01419e0 con 0x7ff8e0137750 2024-08-19T10:42:06.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff8e0141860 con 0x7ff8e006d090 2024-08-19T10:42:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff8e0141b60 con 0x7ff8e009ca60 2024-08-19T10:42:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e7f15640 10 monclient(hunting): authenticate will time out at 1172.321498s 2024-08-19T10:42:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e64b2640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e0137750 msgr2=0x7ff8e0136e40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e64b2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e0137750 0x7ff8e0136e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e006d090 0x7ff8e01d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e54b0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8e009ca60 0x7ff8e0136900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e006d090 0x7ff8e01d10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:37542/0 (socket says 172.21.15.190:37542) 2024-08-19T10:42:06.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e54b0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8e009ca60 0x7ff8e0136900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:34350/0 (socket says 172.21.15.190:34350) 2024-08-19T10:42:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 1 -- 172.21.15.190:0/273313918 learned_addr learned my addr 172.21.15.190:0/273313918 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 10 monclient(hunting): get_auth_request con 0x7ff8e006d090 auth_method 0 2024-08-19T10:42:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e54b0640 10 monclient(hunting): get_auth_request con 0x7ff8e009ca60 auth_method 0 2024-08-19T10:42:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e54b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:06.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e54b0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e54b0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e54b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e54b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e54b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 10 monclient(hunting): handle_auth_done global_id 5440 payload 824 2024-08-19T10:42:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 1 -- 172.21.15.190:0/273313918 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8e009ca60 msgr2=0x7ff8e0136900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:06.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8e009ca60 0x7ff8e0136900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:06.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 1 -- 172.21.15.190:0/273313918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e0137750 msgr2=0x7ff8e0136e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:06.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e0137750 0x7ff8e0136e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:06.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 10 monclient: _finish_hunting 0 2024-08-19T10:42:06.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 1 monclient: found mon.noname-c 2024-08-19T10:42:06.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:06.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:06.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.749+0000 7ff8e5cb1640 1 -- 172.21.15.190:0/273313918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e013e3a0 con 0x7ff8e006d090 2024-08-19T10:42:06.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8e5cb1640 1 -- 172.21.15.190:0/273313918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e006d090 msgr2=0x7ff8e01d10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:06.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8e5cb1640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e006d090 0x7ff8e01d10b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00b430 tx=0x7ff8dc00b8f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:06.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 1 -- 172.21.15.190:0/273313918 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:42:06.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 1 -- 172.21.15.190:0/273313918 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8dc015020 con 0x7ff8e006d090 2024-08-19T10:42:06.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:06.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 10 monclient: dump: 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:06.772 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:06.773 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:06.773 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:06.773 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:06.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:06.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:06.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:06.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 1 -- 172.21.15.190:0/273313918 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8dc011cb0 con 0x7ff8e006d090 2024-08-19T10:42:06.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:06.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 1 -- 172.21.15.190:0/273313918 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8dc0107f0 con 0x7ff8e006d090 2024-08-19T10:42:06.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d67fc640 10 monclient: dump: 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:06.774 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8d77fe640 4 set_mon_vals no callback set 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8e7f15640 5 monclient: authenticate success, global_id 5440 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8e7f15640 10 monclient: _renew_subs 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8e7f15640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8e7f15640 1 -- 172.21.15.190:0/273313918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e0120f50 con 0x7ff8e006d090 2024-08-19T10:42:06.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8e7f15640 10 monclient: _renew_subs 2024-08-19T10:42:06.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8e7f15640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:06.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.753+0000 7ff8e7f15640 1 -- 172.21.15.190:0/273313918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e01d15f0 con 0x7ff8e006d090 2024-08-19T10:42:06.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.757+0000 7ff8e7f15640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:06.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.757+0000 7ff8e7f15640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:06.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.757+0000 7ff8e7f15640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:06.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.757+0000 7ff8e7f15640 1 -- 172.21.15.190:0/273313918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e0137bb0 con 0x7ff8e006d090 2024-08-19T10:42:06.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.757+0000 7ff8d67fc640 1 -- 172.21.15.190:0/273313918 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff8dc018020 con 0x7ff8e006d090 2024-08-19T10:42:06.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.757+0000 7ff8d67fc640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff8c403be00 0x7ff8c403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:06.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.757+0000 7ff8d67fc640 1 -- 172.21.15.190:0/273313918 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(123..123 src has 1..123) ==== 6599+0+0 (secure 0 0 0) 0x7ff8dc04f210 con 0x7ff8e006d090 2024-08-19T10:42:06.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.757+0000 7ff8e54b0640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff8c403be00 0x7ff8c403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:06.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.757+0000 7ff8e54b0640 10 monclient: get_auth_request con 0x7ff8c403be00 auth_method 0 2024-08-19T10:42:06.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.757+0000 7ff8e54b0640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff8c403be00 0x7ff8c403e2c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00a6640 tx=0x7ff8d00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:06.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.769+0000 7ff8d67fc640 1 -- 172.21.15.190:0/273313918 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff8dc01f070 con 0x7ff8e006d090 2024-08-19T10:42:06.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.769+0000 7ff8d67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:06.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.769+0000 7ff8d67fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:06.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.949+0000 7ff8e7f15640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-08-19T10:42:06.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.949+0000 7ff8e7f15640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-08-19T10:42:06.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.949+0000 7ff8e7f15640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:06.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:06.949+0000 7ff8e7f15640 1 -- 172.21.15.190:0/273313918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.14"]} v 0) -- 0x7ff8e006d7c0 con 0x7ff8e006d090 2024-08-19T10:42:06.952 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:06.952+0000 7f16cc065640 -1 mon.a@0(leader).osd e123 definitely_dead 0 2024-08-19T10:42:07.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.517+0000 7ff8d67fc640 1 -- 172.21.15.190:0/273313918 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.14"]}]=0 removed osd.14 v124) ==== 87+0+0 (secure 0 0 0) 0x7ff8dc01a410 con 0x7ff8e006d090 2024-08-19T10:42:07.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.517+0000 7ff8d67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-08-19T10:42:07.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.517+0000 7ff8d67fc640 10 monclient: _finish_command 2 = system:0 removed osd.14 2024-08-19T10:42:07.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.629+0000 7ff8e7f15640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-08-19T10:42:07.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.629+0000 7ff8e7f15640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-08-19T10:42:07.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.629+0000 7ff8e7f15640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:07.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.629+0000 7ff8e7f15640 1 -- 172.21.15.190:0/273313918 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd rm", "ids": ["osd.14"]} v 0) -- 0x7ff8e00008e0 con 0x7ff8e006d090 2024-08-19T10:42:07.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:07.632+0000 7f16cc065640 -1 mon.a@0(leader).osd e124 definitely_dead 0 2024-08-19T10:42:07.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8d67fc640 1 -- 172.21.15.190:0/273313918 <== mon.4 v2:172.21.15.89:3302/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["osd.14"]}]=0 osd.14 does not exist. v124) ==== 96+0+0 (secure 0 0 0) 0x7ff8dc005590 con 0x7ff8e006d090 2024-08-19T10:42:07.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8d67fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["osd.14"]}] 2024-08-19T10:42:07.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8d67fc640 10 monclient: _finish_command 3 = system:0 osd.14 does not exist. 2024-08-19T10:42:07.635 INFO:tasks.workunit.client.0.smithi190.stderr:osd.14 does not exist. 2024-08-19T10:42:07.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 -- 172.21.15.190:0/273313918 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff8c403be00 msgr2=0x7ff8c403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:07.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff8c403be00 0x7ff8c403e2c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00a6640 tx=0x7ff8d00a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:07.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 10 monclient: shutdown 2024-08-19T10:42:07.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:07.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 -- 172.21.15.190:0/273313918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e006d090 msgr2=0x7ff8e01d10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:07.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e006d090 0x7ff8e01d10b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00b430 tx=0x7ff8dc00b8f0 comp rx=0 tx=0).stop 2024-08-19T10:42:07.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 -- 172.21.15.190:0/273313918 shutdown_connections 2024-08-19T10:42:07.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8e0137750 0x7ff8e0136e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:07.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff8c403be00 0x7ff8c403e2c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:07.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8e009ca60 0x7ff8e0136900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:07.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 --2- 172.21.15.190:0/273313918 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8e006d090 0x7ff8e01d10b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:07.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 -- 172.21.15.190:0/273313918 >> 172.21.15.190:0/273313918 conn(0x7ff8e00a93d0 msgr2=0x7ff8e00a34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:07.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 -- 172.21.15.190:0/273313918 shutdown_connections 2024-08-19T10:42:07.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.633+0000 7ff8e7f15640 1 -- 172.21.15.190:0/273313918 wait complete. 2024-08-19T10:42:07.654 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd set-require-min-compat-client luminous 2024-08-19T10:42:07.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.765+0000 7fd3ece45640 10 monclient: get_monmap_and_config 2024-08-19T10:42:07.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.765+0000 7fd3ece45640 10 monclient: build_initial_monmap 2024-08-19T10:42:07.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.765+0000 7fd3ece45640 10 monclient: monmap: 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:07.771388+0000 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:07.771388+0000 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:07.771 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:07.772 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:07.772 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:07.772 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:07.772 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:07.772 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:07.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 1 Processor -- start 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 1 -- start start 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 10 monclient: init 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 10 monclient: _add_conns ranks=[4,10,0,7,9,6,1,5,8,3,2] 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3e8139980 0x7fd3e8139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 10 monclient(hunting): picked mon.noname-c con 0x7fd3e8139980 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd3e813a2a0 0x7fd3e80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 10 monclient(hunting): picked mon.noname-f con 0x7fd3e813a2a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd3e80a5c70 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3e80a8470 con 0x7fd3e80a5c70 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:07.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd3e809ca60 con 0x7fd3e813a2a0 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd3e80a3640 con 0x7fd3e8139980 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3ece45640 10 monclient(hunting): authenticate will time out at 1173.341279s 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd3e813a2a0 0x7fd3e80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd3e813a2a0 0x7fd3e80a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49936/0 (socket says 172.21.15.190:49936) 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 1 -- 172.21.15.190:0/417507931 learned_addr learned my addr 172.21.15.190:0/417507931 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 10 monclient(hunting): get_auth_request con 0x7fd3e813a2a0 auth_method 0 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:07.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e659c640 1 --2- 172.21.15.190:0/417507931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3e8139980 0x7fd3e8139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 10 monclient(hunting): handle_auth_done global_id 14734 payload 274 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 1 -- 172.21.15.190:0/417507931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3e8139980 msgr2=0x7fd3e8139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 1 --2- 172.21.15.190:0/417507931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3e8139980 0x7fd3e8139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 1 -- 172.21.15.190:0/417507931 >> v1:172.21.15.89:6789/0 conn(0x7fd3e80a5c70 legacy=0x7fd3e80a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 1 monclient: found mon.noname-f 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e5d9b640 1 -- 172.21.15.190:0/417507931 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e80a3850 con 0x7fd3e813a2a0 2024-08-19T10:42:07.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e659c640 10 monclient: get_auth_request con 0x7fd3e8139980 auth_method 0 2024-08-19T10:42:07.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.769+0000 7fd3e659c640 1 --2- 172.21.15.190:0/417507931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3e8139980 0x7fd3e8139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:07.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e5d9b640 1 -- 172.21.15.190:0/417507931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e813a2a0 msgr2=0x7fd3e80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:07.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e5d9b640 1 --2- 172.21.15.190:0/417507931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e813a2a0 0x7fd3e80a5730 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0098a0 tx=0x7fd3dc01c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ba88d509f0a5ef9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:07.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 1 -- 172.21.15.190:0/417507931 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd3dc0b0070 con 0x7fd3e813a2a0 2024-08-19T10:42:07.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:07.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:07.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 10 monclient: dump: 2024-08-19T10:42:07.781 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:07.781 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:07.782 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 10 monclient: _finish_auth 0 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 10 monclient: _check_auth_tickets 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 1 -- 172.21.15.190:0/417507931 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3dc0aa810 con 0x7fd3e813a2a0 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 1 -- 172.21.15.190:0/417507931 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd3dc0aae30 con 0x7fd3e813a2a0 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3e559a640 10 monclient: dump: 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:07.783 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:07.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 5 monclient: authenticate success, global_id 14734 2024-08-19T10:42:07.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:07.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 4 set_mon_vals no callback set 2024-08-19T10:42:07.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 10 monclient: shutdown 2024-08-19T10:42:07.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:07.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 1 -- 172.21.15.190:0/417507931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e813a2a0 msgr2=0x7fd3e80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:07.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 1 --2- 172.21.15.190:0/417507931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e813a2a0 0x7fd3e80a5730 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0098a0 tx=0x7fd3dc01c400 comp rx=0 tx=0).stop 2024-08-19T10:42:07.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 1 -- 172.21.15.190:0/417507931 shutdown_connections 2024-08-19T10:42:07.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 1 --2- 172.21.15.190:0/417507931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e813a2a0 0x7fd3e80a5730 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:07.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 1 --2- 172.21.15.190:0/417507931 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3e8139980 0x7fd3e8139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:07.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 1 -- 172.21.15.190:0/417507931 >> 172.21.15.190:0/417507931 conn(0x7fd3e80abde0 msgr2=0x7fd3e80ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 1 -- 172.21.15.190:0/417507931 shutdown_connections 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 1 -- 172.21.15.190:0/417507931 wait complete. 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 10 monclient: build_initial_monmap 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.773+0000 7fd3ece45640 10 monclient: monmap: 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:07.786 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 1 Processor -- start 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 1 -- start start 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 10 monclient: init 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 10 monclient: _add_conns ranks=[2,1,3,7,4,6,5,0,8] 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3e80a5c70 0x7fd3e813a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:07.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 10 monclient(hunting): picked mon.noname-b con 0x7fd3e80a5c70 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:07.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8139980 0x7fd3e813abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:07.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 10 monclient(hunting): picked mon.noname-f con 0x7fd3e8139980 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:07.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e813b110 0x7fd3e8133540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:07.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 10 monclient(hunting): picked mon.noname-g con 0x7fd3e813b110 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:07.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:07.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8144280 con 0x7fd3e8139980 2024-08-19T10:42:07.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:07.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e8144100 con 0x7fd3e80a5c70 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e8144400 con 0x7fd3e813b110 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3ece45640 10 monclient(hunting): authenticate will time out at 1173.348993s 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e6d9d640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e813b110 msgr2=0x7fd3e8133540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e6d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e813b110 0x7fd3e8133540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8139980 0x7fd3e813abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8139980 0x7fd3e813abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49940/0 (socket says 172.21.15.190:49940) 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 1 -- 172.21.15.190:0/2299024013 learned_addr learned my addr 172.21.15.190:0/2299024013 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 10 monclient(hunting): get_auth_request con 0x7fd3e8139980 auth_method 0 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:07.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e659c640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3e80a5c70 0x7fd3e813a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e659c640 10 monclient(hunting): get_auth_request con 0x7fd3e80a5c70 auth_method 0 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 10 monclient(hunting): handle_auth_done global_id 14743 payload 824 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 1 -- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e813b110 msgr2=0x7fd3e8133540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e813b110 0x7fd3e8133540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 1 -- 172.21.15.190:0/2299024013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3e80a5c70 msgr2=0x7fd3e813a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:07.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3e80a5c70 0x7fd3e813a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:07.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:07.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 1 monclient: found mon.noname-f 2024-08-19T10:42:07.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:07.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:07.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 1 -- 172.21.15.190:0/2299024013 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e80a2a00 con 0x7fd3e8139980 2024-08-19T10:42:07.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 1 -- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8139980 msgr2=0x7fd3e813abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:07.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3e5d9b640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8139980 0x7fd3e813abd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc01c230 tx=0x7fd3dc0191a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:07.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3d6ffd640 1 -- 172.21.15.190:0/2299024013 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd3dc0ba070 con 0x7fd3e8139980 2024-08-19T10:42:07.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:07.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3d6ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:07.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.777+0000 7fd3d6ffd640 10 monclient: dump: 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:07.794 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3ece45640 5 monclient: authenticate success, global_id 14743 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3ece45640 10 monclient: _renew_subs 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 1 -- 172.21.15.190:0/2299024013 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3dc0b1040 con 0x7fd3e8139980 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3ece45640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3ece45640 1 -- 172.21.15.190:0/2299024013 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e813cae0 con 0x7fd3e8139980 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:07.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 1 -- 172.21.15.190:0/2299024013 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd3dc0b0070 con 0x7fd3e8139980 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3e4d99640 4 set_mon_vals no callback set 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 10 monclient: dump: 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:07.796 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3ece45640 10 monclient: _renew_subs 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3ece45640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3ece45640 1 -- 172.21.15.190:0/2299024013 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e813d0a0 con 0x7fd3e8139980 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3ece45640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3ece45640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3ece45640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:07.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3ece45640 1 -- 172.21.15.190:0/2299024013 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e8139d60 con 0x7fd3e8139980 2024-08-19T10:42:07.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 1 -- 172.21.15.190:0/2299024013 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd3dc003620 con 0x7fd3e8139980 2024-08-19T10:42:07.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd3c803be30 0x7fd3c803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:07.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3d6ffd640 1 -- 172.21.15.190:0/2299024013 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6599+0+0 (secure 0 0 0) 0x7fd3dc0e8660 con 0x7fd3e8139980 2024-08-19T10:42:07.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3e659c640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd3c803be30 0x7fd3c803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:07.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3e659c640 10 monclient: get_auth_request con 0x7fd3c803be30 auth_method 0 2024-08-19T10:42:07.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.781+0000 7fd3e659c640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd3c803be30 0x7fd3c803e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800a9a0 tx=0x7fd3d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:07.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.785+0000 7fd3d6ffd640 1 -- 172.21.15.190:0/2299024013 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd3e80a2a00 con 0x7fd3e8139980 2024-08-19T10:42:07.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.785+0000 7fd3d6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:07.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.785+0000 7fd3d6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:07.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.957+0000 7fd3ece45640 10 start_mon_command cmd=[{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-08-19T10:42:07.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.957+0000 7fd3ece45640 10 monclient: _send_command 2 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-08-19T10:42:07.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.957+0000 7fd3ece45640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:07.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:07.957+0000 7fd3ece45640 1 -- 172.21.15.190:0/2299024013 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "luminous"} v 0) -- 0x7fd3e806c8d0 con 0x7fd3e8139980 2024-08-19T10:42:08.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:08.529+0000 7fd3d6ffd640 1 -- 172.21.15.190:0/2299024013 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "luminous"}]=0 set require_min_compat_client to luminous v125) ==== 145+0+0 (secure 0 0 0) 0x7fd3dc016110 con 0x7fd3e8139980 2024-08-19T10:42:08.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:08.529+0000 7fd3d6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-08-19T10:42:08.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:08.529+0000 7fd3d6ffd640 10 monclient: _finish_command 2 = system:0 set require_min_compat_client to luminous 2024-08-19T10:42:08.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:08.641+0000 7fd3ece45640 10 start_mon_command cmd=[{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-08-19T10:42:08.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:08.641+0000 7fd3ece45640 10 monclient: _send_command 3 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-08-19T10:42:08.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:08.641+0000 7fd3ece45640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:08.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:08.641+0000 7fd3ece45640 1 -- 172.21.15.190:0/2299024013 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "luminous"} v 0) -- 0x7fd3e8134240 con 0x7fd3e8139980 2024-08-19T10:42:09.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.541+0000 7fd3d6ffd640 1 -- 172.21.15.190:0/2299024013 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "luminous"}]=0 set require_min_compat_client to luminous v126) ==== 145+0+0 (secure 0 0 0) 0x7fd3dc0aa7e0 con 0x7fd3e8139980 2024-08-19T10:42:09.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.541+0000 7fd3d6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd set-require-min-compat-client", "version": "luminous"}] 2024-08-19T10:42:09.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.541+0000 7fd3d6ffd640 10 monclient: _finish_command 3 = system:0 set require_min_compat_client to luminous 2024-08-19T10:42:09.544 INFO:tasks.workunit.client.0.smithi190.stderr:set require_min_compat_client to luminous 2024-08-19T10:42:09.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 -- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd3c803be30 msgr2=0x7fd3c803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:09.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd3c803be30 0x7fd3c803e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800a9a0 tx=0x7fd3d8008040 comp rx=0 tx=0).stop 2024-08-19T10:42:09.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 10 monclient: shutdown 2024-08-19T10:42:09.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:09.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 -- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8139980 msgr2=0x7fd3e813abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:09.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8139980 0x7fd3e813abd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc01c230 tx=0x7fd3dc0191a0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 -- 172.21.15.190:0/2299024013 shutdown_connections 2024-08-19T10:42:09.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd3c803be30 0x7fd3c803e2f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3e8139980 0x7fd3e813abd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3e813b110 0x7fd3e8133540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 --2- 172.21.15.190:0/2299024013 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd3e80a5c70 0x7fd3e813a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 -- 172.21.15.190:0/2299024013 >> 172.21.15.190:0/2299024013 conn(0x7fd3e80abde0 msgr2=0x7fd3e8134d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:09.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 -- 172.21.15.190:0/2299024013 shutdown_connections 2024-08-19T10:42:09.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.545+0000 7fd3ece45640 1 -- 172.21.15.190:0/2299024013 wait complete. 2024-08-19T10:42:09.565 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set dump 2024-08-19T10:42:09.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient: get_monmap_and_config 2024-08-19T10:42:09.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient: build_initial_monmap 2024-08-19T10:42:09.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient: monmap: 2024-08-19T10:42:09.684 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:09.684 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:09.684 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:09.684926+0000 2024-08-19T10:42:09.684 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:09.684926+0000 2024-08-19T10:42:09.684 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:09.685 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:09.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 1 Processor -- start 2024-08-19T10:42:09.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 1 -- start start 2024-08-19T10:42:09.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient: init 2024-08-19T10:42:09.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:09.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient: _add_conns ranks=[4,1,5,2,7,0,9,6,8,10,3] 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c240a6740 0x7f3c240a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient(hunting): picked mon.noname-c con 0x7f3c240a6740 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3c240a5bc0 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3c240a7230 0x7f3c24140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient(hunting): picked mon.noname-h con 0x7f3c240a7230 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c240a3740 con 0x7f3c240a5bc0 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3c240a4c20 con 0x7f3c240a6740 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f3c240a3530 con 0x7f3c240a7230 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c28e13640 10 monclient(hunting): authenticate will time out at 1175.255043s 2024-08-19T10:42:09.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c21d9b640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f3c240a5bc0 0x7f3c240a6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:42992/0 (socket says 172.21.15.190:42992) 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c21d9b640 1 -- 172.21.15.190:0/350506704 learned_addr learned my addr 172.21.15.190:0/350506704 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 1 --2- 172.21.15.190:0/350506704 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3c240a7230 0x7f3c24140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 10 monclient(hunting): get_auth_request con 0x7f3c240a7230 auth_method 0 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c2259c640 1 --2- 172.21.15.190:0/350506704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c240a6740 0x7f3c240a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c2259c640 10 monclient(hunting): get_auth_request con 0x7f3c240a6740 auth_method 0 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c2259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c2259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c2259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:09.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c2259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c2259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c2259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 10 monclient(hunting): handle_auth_done global_id 5495 payload 274 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 1 -- 172.21.15.190:0/350506704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c240a6740 msgr2=0x7f3c240a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 1 --2- 172.21.15.190:0/350506704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c240a6740 0x7f3c240a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 1 -- 172.21.15.190:0/350506704 >> v1:172.21.15.190:6789/0 conn(0x7f3c240a5bc0 legacy=0x7f3c240a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 1 monclient: found mon.noname-h 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:09.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.681+0000 7f3c22d9d640 1 -- 172.21.15.190:0/350506704 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c240a3240 con 0x7f3c240a7230 2024-08-19T10:42:09.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 1 -- 172.21.15.190:0/350506704 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3c240a7230 msgr2=0x7f3c24140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:09.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 1 --2- 172.21.15.190:0/350506704 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3c240a7230 0x7f3c24140ab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009820 tx=0x7f3c0c01c320 comp rx=0 tx=0).ready entity=mon.5 client_cookie=799e4bb1a87fbc73 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 1 -- 172.21.15.190:0/350506704 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c0c0b0070 con 0x7f3c240a7230 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 10 monclient: dump: 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:09.697 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 10 monclient: _finish_auth 0 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 10 monclient: _check_auth_tickets 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 1 -- 172.21.15.190:0/350506704 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c0a9980 con 0x7f3c240a7230 2024-08-19T10:42:09.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 1 -- 172.21.15.190:0/350506704 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c0c0aa3a0 con 0x7f3c240a7230 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 5 monclient: authenticate success, global_id 5495 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 4 set_mon_vals no callback set 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2159a640 10 monclient: dump: 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:09.699 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient: shutdown 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 -- 172.21.15.190:0/350506704 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3c240a7230 msgr2=0x7f3c24140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:09.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 --2- 172.21.15.190:0/350506704 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3c240a7230 0x7f3c24140ab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009820 tx=0x7f3c0c01c320 comp rx=0 tx=0).stop 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 -- 172.21.15.190:0/350506704 shutdown_connections 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 --2- 172.21.15.190:0/350506704 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3c240a7230 0x7f3c24140ab0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 --2- 172.21.15.190:0/350506704 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c240a6740 0x7f3c240a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 -- 172.21.15.190:0/350506704 >> 172.21.15.190:0/350506704 conn(0x7f3c240abd40 msgr2=0x7f3c240ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 -- 172.21.15.190:0/350506704 shutdown_connections 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 -- 172.21.15.190:0/350506704 wait complete. 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient: build_initial_monmap 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient: monmap: 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:09.701 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 Processor -- start 2024-08-19T10:42:09.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 -- start start 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient: init 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient: _add_conns ranks=[2,0,7,5,3,6,1,8,4] 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c240a6740 0x7f3c241364d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient(hunting): picked mon.noname-b con 0x7f3c240a6740 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c240a7230 0x7f3c24136a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient(hunting): picked mon.noname-a con 0x7f3c240a7230 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c2413de30 0x7f3c24136f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient(hunting): picked mon.noname-i con 0x7f3c2413de30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24143990 con 0x7f3c240a7230 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:09.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3c24143810 con 0x7f3c240a6740 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f3c24143b10 con 0x7f3c2413de30 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c28e13640 10 monclient(hunting): authenticate will time out at 1175.262391s 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c2413de30 0x7f3c24136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c2413de30 0x7f3c24136f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42200/0 (socket says 172.21.15.190:42200) 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 1 -- 172.21.15.190:0/921194535 learned_addr learned my addr 172.21.15.190:0/921194535 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 10 monclient(hunting): get_auth_request con 0x7f3c2413de30 auth_method 0 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:09.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2259c640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c240a6740 0x7f3c241364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2259c640 10 monclient(hunting): get_auth_request con 0x7f3c240a6740 auth_method 0 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c2259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c21d9b640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c240a7230 0x7f3c24136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c21d9b640 10 monclient(hunting): get_auth_request con 0x7f3c240a7230 auth_method 0 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c21d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c21d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c21d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 10 monclient(hunting): handle_auth_done global_id 14641 payload 824 2024-08-19T10:42:09.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 1 -- 172.21.15.190:0/921194535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c240a6740 msgr2=0x7f3c241364d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c240a6740 0x7f3c241364d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 1 -- 172.21.15.190:0/921194535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c240a7230 msgr2=0x7f3c24136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c240a7230 0x7f3c24136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 1 monclient: found mon.noname-i 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.689+0000 7f3c22d9d640 1 -- 172.21.15.190:0/921194535 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2413d2d0 con 0x7f3c2413de30 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c22d9d640 1 -- 172.21.15.190:0/921194535 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c2413de30 msgr2=0x7f3c24136f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c22d9d640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c2413de30 0x7f3c24136f50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c01cbc0 tx=0x7f3c0c019120 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 1 -- 172.21.15.190:0/921194535 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c0c0b9020 con 0x7f3c2413de30 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:09.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 10 monclient: dump: 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:09.707 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 10 monclient: _finish_auth 0 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 1 -- 172.21.15.190:0/921194535 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c004250 con 0x7f3c2413de30 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c28e13640 5 monclient: authenticate success, global_id 14641 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c28e13640 10 monclient: _renew_subs 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c28e13640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c28e13640 1 -- 172.21.15.190:0/921194535 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c240a9470 con 0x7f3c2413de30 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1bfff640 4 set_mon_vals no callback set 2024-08-19T10:42:09.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c28e13640 10 monclient: _renew_subs 2024-08-19T10:42:09.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c28e13640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:09.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c28e13640 1 -- 172.21.15.190:0/921194535 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2413ed90 con 0x7f3c2413de30 2024-08-19T10:42:09.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 1 -- 172.21.15.190:0/921194535 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c0c0b0040 con 0x7f3c2413de30 2024-08-19T10:42:09.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:09.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 10 monclient: dump: 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:09.711 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c28e13640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c28e13640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c28e13640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c28e13640 1 -- 172.21.15.190:0/921194535 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c240a5680 con 0x7f3c2413de30 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 1 -- 172.21.15.190:0/921194535 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3c0c0bc020 con 0x7f3c2413de30 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c0803be00 0x7f3c0803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c1affd640 1 -- 172.21.15.190:0/921194535 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(126..126 src has 1..126) ==== 6599+0+0 (secure 0 0 0) 0x7f3c0c0e90d0 con 0x7f3c2413de30 2024-08-19T10:42:09.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c2259c640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c0803be00 0x7f3c0803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:09.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.693+0000 7f3c2259c640 10 monclient: get_auth_request con 0x7f3c0803be00 auth_method 0 2024-08-19T10:42:09.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.697+0000 7f3c2259c640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c0803be00 0x7f3c0803e2c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0074a0 tx=0x7f3c1c00d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:09.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.697+0000 7f3c1affd640 1 -- 172.21.15.190:0/921194535 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3c0c0c3030 con 0x7f3c2413de30 2024-08-19T10:42:09.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.697+0000 7f3c1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:09.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.697+0000 7f3c1affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:09.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.869+0000 7f3c28e13640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:09.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.873+0000 7f3c28e13640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:09.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.873+0000 7f3c28e13640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:09.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.873+0000 7f3c28e13640 1 -- 172.21.15.190:0/921194535 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) -- 0x7f3c2406c8d0 con 0x7f3c2413de30 2024-08-19T10:42:09.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.873+0000 7f3c1affd640 1 -- 172.21.15.190:0/921194535 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v126) ==== 73+0+3 (secure 0 0 0) 0x7f3c0c016090 con 0x7f3c2413de30 2024-08-19T10:42:09.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.873+0000 7f3c1affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:09.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.873+0000 7f3c1affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:09.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.969+0000 7f3c28e13640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:09.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.969+0000 7f3c28e13640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:09.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.969+0000 7f3c28e13640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:09.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.969+0000 7f3c28e13640 1 -- 172.21.15.190:0/921194535 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) -- 0x7f3c240a5bc0 con 0x7f3c2413de30 2024-08-19T10:42:09.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.969+0000 7f3c1affd640 1 -- 172.21.15.190:0/921194535 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v126) ==== 73+0+3 (secure 0 0 0) 0x7f3c0c0044b0 con 0x7f3c2413de30 2024-08-19T10:42:09.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.969+0000 7f3c1affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:09.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.969+0000 7f3c1affd640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:09.974 INFO:tasks.workunit.client.0.smithi190.stdout:{} 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 -- 172.21.15.190:0/921194535 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c0803be00 msgr2=0x7f3c0803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c0803be00 0x7f3c0803e2c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0074a0 tx=0x7f3c1c00d040 comp rx=0 tx=0).stop 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 10 monclient: shutdown 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 -- 172.21.15.190:0/921194535 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c2413de30 msgr2=0x7f3c24136f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c2413de30 0x7f3c24136f50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c01cbc0 tx=0x7f3c0c019120 comp rx=0 tx=0).stop 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 -- 172.21.15.190:0/921194535 shutdown_connections 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c0803be00 0x7f3c0803e2c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c240a7230 0x7f3c24136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3c2413de30 0x7f3c24136f50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 --2- 172.21.15.190:0/921194535 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3c240a6740 0x7f3c241364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:09.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 -- 172.21.15.190:0/921194535 >> 172.21.15.190:0/921194535 conn(0x7f3c240abd40 msgr2=0x7f3c24139700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:09.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 -- 172.21.15.190:0/921194535 shutdown_connections 2024-08-19T10:42:09.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:09.973+0000 7f3c28e13640 1 -- 172.21.15.190:0/921194535 wait complete. 2024-08-19T10:42:09.990 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set ls 2024-08-19T10:42:10.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient: get_monmap_and_config 2024-08-19T10:42:10.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient: build_initial_monmap 2024-08-19T10:42:10.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient: monmap: 2024-08-19T10:42:10.096 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:10.096 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:10.096 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:10.096821+0000 2024-08-19T10:42:10.096 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:10.096821+0000 2024-08-19T10:42:10.096 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:10.096 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.096 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:10.096 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:10.097 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:10.097 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:10.097 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:10.097 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:10.097 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:10.097 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:10.097 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:10.097 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:10.097 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:10.097 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 1 Processor -- start 2024-08-19T10:42:10.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 1 -- start start 2024-08-19T10:42:10.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient: init 2024-08-19T10:42:10.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:10.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient: _add_conns ranks=[8,4,3,9,1,2,6,0,10,5,7] 2024-08-19T10:42:10.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df8139a60 0x7f7df8139e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient(hunting): picked mon.noname-e con 0x7f7df8139a60 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:10.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df80a5730 0x7f7df80a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient(hunting): picked mon.noname-c con 0x7f7df80a5730 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:10.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7df80a6850 0x7f7df80a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient(hunting): picked mon.noname-g con 0x7f7df80a6850 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:10.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7df809ca60 con 0x7f7df80a6850 2024-08-19T10:42:10.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f7df80a33c0 con 0x7f7df80a5730 2024-08-19T10:42:10.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7df80a3640 con 0x7f7df8139a60 2024-08-19T10:42:10.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:10.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7dfccd8640 10 monclient(hunting): authenticate will time out at 1175.667235s 2024-08-19T10:42:10.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df6d9d640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7df80a6850 msgr2=0x7f7df80a6cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df6d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7df80a6850 0x7f7df80a6cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df659c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df8139a60 0x7f7df8139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df659c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df8139a60 0x7f7df8139e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:34356/0 (socket says 172.21.15.190:34356) 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df659c640 1 -- 172.21.15.190:0/2816352856 learned_addr learned my addr 172.21.15.190:0/2816352856 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df659c640 10 monclient(hunting): get_auth_request con 0x7f7df8139a60 auth_method 0 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df5d9b640 1 --2- 172.21.15.190:0/2816352856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df80a5730 0x7f7df80a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df5d9b640 10 monclient(hunting): get_auth_request con 0x7f7df80a5730 auth_method 0 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:10.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.093+0000 7f7df5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.097+0000 7f7df659c640 10 monclient(hunting): handle_auth_done global_id 5264 payload 274 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.097+0000 7f7df659c640 1 -- 172.21.15.190:0/2816352856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df80a5730 msgr2=0x7f7df80a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.097+0000 7f7df659c640 1 --2- 172.21.15.190:0/2816352856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df80a5730 0x7f7df80a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.097+0000 7f7df659c640 1 -- 172.21.15.190:0/2816352856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7df80a6850 msgr2=0x7f7df80a6cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.097+0000 7f7df659c640 1 --2- 172.21.15.190:0/2816352856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7df80a6850 0x7f7df80a6cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.097+0000 7f7df659c640 10 monclient: _finish_hunting 0 2024-08-19T10:42:10.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.097+0000 7f7df659c640 1 monclient: found mon.noname-e 2024-08-19T10:42:10.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.097+0000 7f7df659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:10.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.097+0000 7f7df659c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:10.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.097+0000 7f7df659c640 1 -- 172.21.15.190:0/2816352856 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df80a46e0 con 0x7f7df8139a60 2024-08-19T10:42:10.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df659c640 1 -- 172.21.15.190:0/2816352856 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df8139a60 msgr2=0x7f7df8139e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:10.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df659c640 1 --2- 172.21.15.190:0/2816352856 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df8139a60 0x7f7df8139e40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7de80098a0 tx=0x7f7de801c400 comp rx=0 tx=0).ready entity=mon.8 client_cookie=49cd48bee1921683 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:10.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 1 -- 172.21.15.190:0/2816352856 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7de80b0070 con 0x7f7df8139a60 2024-08-19T10:42:10.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:10.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:10.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 10 monclient: dump: 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:10.113 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 10 monclient: _finish_auth 0 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 10 monclient: _check_auth_tickets 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 1 -- 172.21.15.190:0/2816352856 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de80aa810 con 0x7f7df8139a60 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 5 monclient: authenticate success, global_id 5264 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 4 set_mon_vals no callback set 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 1 -- 172.21.15.190:0/2816352856 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7de80aaa40 con 0x7f7df8139a60 2024-08-19T10:42:10.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df559a640 10 monclient: dump: 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:10.115 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient: shutdown 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 -- 172.21.15.190:0/2816352856 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df8139a60 msgr2=0x7f7df8139e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 --2- 172.21.15.190:0/2816352856 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df8139a60 0x7f7df8139e40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7de80098a0 tx=0x7f7de801c400 comp rx=0 tx=0).stop 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 -- 172.21.15.190:0/2816352856 shutdown_connections 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 --2- 172.21.15.190:0/2816352856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7df80a6850 0x7f7df80a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 --2- 172.21.15.190:0/2816352856 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7df80a5730 0x7f7df80a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 --2- 172.21.15.190:0/2816352856 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df8139a60 0x7f7df8139e40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 -- 172.21.15.190:0/2816352856 >> 172.21.15.190:0/2816352856 conn(0x7f7df80abe50 msgr2=0x7f7df80ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 -- 172.21.15.190:0/2816352856 shutdown_connections 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 -- 172.21.15.190:0/2816352856 wait complete. 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient: build_initial_monmap 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient: monmap: 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:10.117 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 Processor -- start 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 -- start start 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient: init 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient: _add_conns ranks=[8,6,7,0,4,3,1,2,5] 2024-08-19T10:42:10.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df80a5730 0x7f7df81335b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient(hunting): picked mon.noname-e con 0x7f7df80a5730 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7df80a6850 0x7f7df8133af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient(hunting): picked mon.noname-d con 0x7f7df80a6850 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7df8139a60 0x7f7df8134030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient(hunting): picked mon.noname-i con 0x7f7df8139a60 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7df8143270 con 0x7f7df80a6850 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7df81430f0 con 0x7f7df8139a60 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7df81433f0 con 0x7f7df80a5730 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:10.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7dfccd8640 10 monclient(hunting): authenticate will time out at 1175.675718s 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df6d9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7df8139a60 0x7f7df8134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df6d9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7df8139a60 0x7f7df8134030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42216/0 (socket says 172.21.15.190:42216) 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df6d9d640 1 -- 172.21.15.190:0/3948582886 learned_addr learned my addr 172.21.15.190:0/3948582886 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df6d9d640 10 monclient(hunting): get_auth_request con 0x7f7df8139a60 auth_method 0 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.101+0000 7f7df6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df659c640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df80a5730 0x7f7df81335b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df5d9b640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7df80a6850 0x7f7df8133af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df659c640 10 monclient(hunting): get_auth_request con 0x7f7df80a5730 auth_method 0 2024-08-19T10:42:10.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df5d9b640 10 monclient(hunting): get_auth_request con 0x7f7df80a6850 auth_method 0 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 10 monclient(hunting): handle_auth_done global_id 14650 payload 824 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 1 -- 172.21.15.190:0/3948582886 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df80a5730 msgr2=0x7f7df81335b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df80a5730 0x7f7df81335b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 1 -- 172.21.15.190:0/3948582886 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7df80a6850 msgr2=0x7f7df8133af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7df80a6850 0x7f7df8133af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:42:10.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 1 monclient: found mon.noname-i 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 1 -- 172.21.15.190:0/3948582886 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df8135760 con 0x7f7df8139a60 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 1 -- 172.21.15.190:0/3948582886 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7df8139a60 msgr2=0x7f7df8134030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df6d9d640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7df8139a60 0x7f7df8134030 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7df000be60 tx=0x7f7df000bf50 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 1 -- 172.21.15.190:0/3948582886 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7df0015850 con 0x7f7df8139a60 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 10 monclient: dump: 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.122 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:42:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 1 -- 172.21.15.190:0/3948582886 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df0010030 con 0x7f7df8139a60 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7dfccd8640 5 monclient: authenticate success, global_id 14650 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 1 -- 172.21.15.190:0/3948582886 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7df00060d0 con 0x7f7df8139a60 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 10 monclient: dump: 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.124 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7df4d99640 4 set_mon_vals no callback set 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7dfccd8640 10 monclient: _renew_subs 2024-08-19T10:42:10.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7dfccd8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7dfccd8640 1 -- 172.21.15.190:0/3948582886 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df813d1c0 con 0x7f7df8139a60 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7dfccd8640 10 monclient: _renew_subs 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7dfccd8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7dfccd8640 1 -- 172.21.15.190:0/3948582886 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df80a1b50 con 0x7f7df8139a60 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7dfccd8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7dfccd8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7dfccd8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7dfccd8640 1 -- 172.21.15.190:0/3948582886 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df80a6cb0 con 0x7f7df8139a60 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.105+0000 7f7de6ffd640 1 -- 172.21.15.190:0/3948582886 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7df0003d10 con 0x7f7df8139a60 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.109+0000 7f7de6ffd640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd803be30 0x7f7dd803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.109+0000 7f7de6ffd640 1 -- 172.21.15.190:0/3948582886 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(126..126 src has 1..126) ==== 6599+0+0 (secure 0 0 0) 0x7f7df004f3e0 con 0x7f7df8139a60 2024-08-19T10:42:10.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.109+0000 7f7df659c640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd803be30 0x7f7dd803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:10.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.109+0000 7f7df659c640 10 monclient: get_auth_request con 0x7f7dd803be30 auth_method 0 2024-08-19T10:42:10.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.109+0000 7f7df659c640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd803be30 0x7f7dd803e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7de80ad000 tx=0x7f7de80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:10.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.113+0000 7f7de6ffd640 1 -- 172.21.15.190:0/3948582886 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7df8135760 con 0x7f7df8139a60 2024-08-19T10:42:10.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.113+0000 7f7de6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:10.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.113+0000 7f7de6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:10.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.289+0000 7f7dfccd8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:10.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.293+0000 7f7dfccd8640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:10.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.293+0000 7f7dfccd8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.293+0000 7f7dfccd8640 1 -- 172.21.15.190:0/3948582886 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f7df813b9c0 con 0x7f7df8139a60 2024-08-19T10:42:10.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.293+0000 7f7de6ffd640 1 -- 172.21.15.190:0/3948582886 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v127) ==== 71+0+0 (secure 0 0 0) 0x7f7df004f080 con 0x7f7df8139a60 2024-08-19T10:42:10.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.293+0000 7f7de6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:10.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.293+0000 7f7de6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:10.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.393+0000 7f7dfccd8640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:10.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.393+0000 7f7dfccd8640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:10.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.393+0000 7f7dfccd8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.393+0000 7f7dfccd8640 1 -- 172.21.15.190:0/3948582886 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f7df80008e0 con 0x7f7df8139a60 2024-08-19T10:42:10.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.393+0000 7f7de6ffd640 1 -- 172.21.15.190:0/3948582886 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v127) ==== 71+0+0 (secure 0 0 0) 0x7f7df0013cf0 con 0x7f7df8139a60 2024-08-19T10:42:10.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.393+0000 7f7de6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:10.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.393+0000 7f7de6ffd640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:10.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 -- 172.21.15.190:0/3948582886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd803be30 msgr2=0x7f7dd803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:10.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd803be30 0x7f7dd803e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7de80ad000 tx=0x7f7de80a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:10.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 10 monclient: shutdown 2024-08-19T10:42:10.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:10.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 -- 172.21.15.190:0/3948582886 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7df8139a60 msgr2=0x7f7df8134030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:10.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7df8139a60 0x7f7df8134030 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7df000be60 tx=0x7f7df000bf50 comp rx=0 tx=0).stop 2024-08-19T10:42:10.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 -- 172.21.15.190:0/3948582886 shutdown_connections 2024-08-19T10:42:10.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7df8139a60 0x7f7df8134030 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7df80a6850 0x7f7df8133af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd803be30 0x7f7dd803e2f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 --2- 172.21.15.190:0/3948582886 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df80a5730 0x7f7df81335b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 -- 172.21.15.190:0/3948582886 >> 172.21.15.190:0/3948582886 conn(0x7f7df80abe50 msgr2=0x7f7df8134e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:10.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 -- 172.21.15.190:0/3948582886 shutdown_connections 2024-08-19T10:42:10.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.397+0000 7f7dfccd8640 1 -- 172.21.15.190:0/3948582886 wait complete. 2024-08-19T10:42:10.415 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 2024-08-19T10:42:10.415 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:42:10.415 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-08-19T10:42:10.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.529+0000 7f8f3a99c640 10 monclient: get_monmap_and_config 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.529+0000 7f8f3a99c640 10 monclient: build_initial_monmap 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.529+0000 7f8f3a99c640 10 monclient: monmap: 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:10.533527+0000 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:10.533527+0000 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:10.533 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:10.534 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:10.534 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:10.534 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:10.534 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:10.534 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:10.534 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:10.534 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:10.534 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.529+0000 7f8f3a99c640 1 Processor -- start 2024-08-19T10:42:10.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.529+0000 7f8f3a99c640 1 -- start start 2024-08-19T10:42:10.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.529+0000 7f8f3a99c640 10 monclient: init 2024-08-19T10:42:10.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:10.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 10 monclient: _add_conns ranks=[7,9,6,4,2,8,5,10,0,1,3] 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a69c0 0x7f8f34139780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 10 monclient(hunting): picked mon.noname-i con 0x7f8f340a69c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8f34139cc0 0x7f8f3413a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 10 monclient(hunting): picked mon.noname-a con 0x7f8f34139cc0 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8f340a6da0 0x7f8f340a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 10 monclient(hunting): picked mon.noname-d con 0x7f8f340a6da0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f8f3413a5e0 con 0x7f8f34139cc0 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8f340a46b0 con 0x7f8f340a6da0 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8f340a4990 con 0x7f8f340a69c0 2024-08-19T10:42:10.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:10.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f3a99c640 10 monclient(hunting): authenticate will time out at 1176.104247s 2024-08-19T10:42:10.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a69c0 0x7f8f34139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:10.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a69c0 0x7f8f34139780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42228/0 (socket says 172.21.15.190:42228) 2024-08-19T10:42:10.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 1 -- 172.21.15.190:0/3572712037 learned_addr learned my addr 172.21.15.190:0/3572712037 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:10.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 10 monclient(hunting): get_auth_request con 0x7f8f340a69c0 auth_method 0 2024-08-19T10:42:10.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:10.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:10.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f38f39640 1 --2- 172.21.15.190:0/3572712037 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8f340a6da0 0x7f8f340a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 10 monclient(hunting): handle_auth_done global_id 14659 payload 274 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 1 -- 172.21.15.190:0/3572712037 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8f340a6da0 msgr2=0x7f8f340a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 1 --2- 172.21.15.190:0/3572712037 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8f340a6da0 0x7f8f340a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 1 -- 172.21.15.190:0/3572712037 >> v2:172.21.15.89:3300/0 conn(0x7f8f34139cc0 msgr2=0x7f8f3413a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 1 --2- 172.21.15.190:0/3572712037 >> v2:172.21.15.89:3300/0 conn(0x7f8f34139cc0 0x7f8f3413a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 1 monclient: found mon.noname-i 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:10.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.533+0000 7f8f33fff640 1 -- 172.21.15.190:0/3572712037 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f340a4c70 con 0x7f8f340a69c0 2024-08-19T10:42:10.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f33fff640 1 -- 172.21.15.190:0/3572712037 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a69c0 msgr2=0x7f8f34139780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:10.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f33fff640 1 --2- 172.21.15.190:0/3572712037 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a69c0 0x7f8f34139780 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009870 tx=0x7f8f2401c420 comp rx=0 tx=0).ready entity=mon.7 client_cookie=1dc370791d0d0fce server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:10.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 1 -- 172.21.15.190:0/3572712037 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8f240b1070 con 0x7f8f340a69c0 2024-08-19T10:42:10.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:10.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:10.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 10 monclient: dump: 2024-08-19T10:42:10.547 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:10.547 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:10.547 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:10.548 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 1 -- 172.21.15.190:0/3572712037 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f240ab830 con 0x7f8f340a69c0 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 1 -- 172.21.15.190:0/3572712037 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8f240aa400 con 0x7f8f340a69c0 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 5 monclient: authenticate success, global_id 14659 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 4 set_mon_vals no callback set 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f337fe640 10 monclient: dump: 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:10.549 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:10.550 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient: shutdown 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 -- 172.21.15.190:0/3572712037 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a69c0 msgr2=0x7f8f34139780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 --2- 172.21.15.190:0/3572712037 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a69c0 0x7f8f34139780 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009870 tx=0x7f8f2401c420 comp rx=0 tx=0).stop 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 -- 172.21.15.190:0/3572712037 shutdown_connections 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 --2- 172.21.15.190:0/3572712037 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8f340a6da0 0x7f8f340a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 --2- 172.21.15.190:0/3572712037 >> v2:172.21.15.89:3300/0 conn(0x7f8f34139cc0 0x7f8f3413a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 --2- 172.21.15.190:0/3572712037 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a69c0 0x7f8f34139780 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 -- 172.21.15.190:0/3572712037 >> 172.21.15.190:0/3572712037 conn(0x7f8f340abfc0 msgr2=0x7f8f340ac3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 -- 172.21.15.190:0/3572712037 shutdown_connections 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 -- 172.21.15.190:0/3572712037 wait complete. 2024-08-19T10:42:10.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient: build_initial_monmap 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient: monmap: 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:10.552 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 Processor -- start 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 -- start start 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient: init 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient: _add_conns ranks=[3,7,2,8,0,4,6,1,5] 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f340a69c0 0x7f8f340a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient(hunting): picked mon.noname-g con 0x7f8f340a69c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a6da0 0x7f8f34135c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient(hunting): picked mon.noname-i con 0x7f8f340a6da0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:10.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f34139cc0 0x7f8f341361d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient(hunting): picked mon.noname-b con 0x7f8f34139cc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8f340a7cd0 con 0x7f8f34139cc0 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8f340a7b50 con 0x7f8f340a69c0 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8f341431f0 con 0x7f8f340a6da0 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f3a99c640 10 monclient(hunting): authenticate will time out at 1176.111259s 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f33fff640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f340a69c0 msgr2=0x7f8f340a2100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f33fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f340a69c0 0x7f8f340a2100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:10.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a6da0 0x7f8f34135c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a6da0 0x7f8f34135c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42234/0 (socket says 172.21.15.190:42234) 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 1 -- 172.21.15.190:0/3351454271 learned_addr learned my addr 172.21.15.190:0/3351454271 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 10 monclient(hunting): get_auth_request con 0x7f8f340a6da0 auth_method 0 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f38f39640 1 --2- 172.21.15.190:0/3351454271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f34139cc0 0x7f8f341361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f38f39640 10 monclient(hunting): get_auth_request con 0x7f8f34139cc0 auth_method 0 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f38f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:10.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f38f39640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f38f39640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 10 monclient(hunting): handle_auth_done global_id 14668 payload 824 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 1 -- 172.21.15.190:0/3351454271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f340a69c0 msgr2=0x7f8f340a2100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 1 --2- 172.21.15.190:0/3351454271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f340a69c0 0x7f8f340a2100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 1 -- 172.21.15.190:0/3351454271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f34139cc0 msgr2=0x7f8f341361d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 1 --2- 172.21.15.190:0/3351454271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f34139cc0 0x7f8f341361d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 1 monclient: found mon.noname-i 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f2b7fe640 1 -- 172.21.15.190:0/3351454271 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3409f690 con 0x7f8f340a6da0 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.537+0000 7f8f38f39640 1 --2- 172.21.15.190:0/3351454271 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8f34139cc0 0x7f8f341361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f2b7fe640 1 -- 172.21.15.190:0/3351454271 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a6da0 msgr2=0x7f8f34135c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:10.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f2b7fe640 1 --2- 172.21.15.190:0/3351454271 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8f340a6da0 0x7f8f34135c90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00b440 tx=0x7f8f1c00b900 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 1 -- 172.21.15.190:0/3351454271 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8f1c015020 con 0x7f8f340a6da0 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 10 monclient: dump: 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 10 monclient: _finish_auth 0 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 10 monclient: _check_auth_tickets 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f3a99c640 5 monclient: authenticate success, global_id 14668 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 1 -- 172.21.15.190:0/3351454271 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1c011d10 con 0x7f8f340a6da0 2024-08-19T10:42:10.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f3a99c640 10 monclient: _renew_subs 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f3a99c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f3a99c640 1 -- 172.21.15.190:0/3351454271 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f340ac5e0 con 0x7f8f340a6da0 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 1 -- 172.21.15.190:0/3351454271 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8f1c010850 con 0x7f8f340a6da0 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f32ffd640 4 set_mon_vals no callback set 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 10 monclient: dump: 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:10.559 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f3a99c640 10 monclient: _renew_subs 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f3a99c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f3a99c640 1 -- 172.21.15.190:0/3351454271 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f340a19f0 con 0x7f8f340a6da0 2024-08-19T10:42:10.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f3a99c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:10.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f3a99c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:10.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f3a99c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f3a99c640 1 -- 172.21.15.190:0/3351454271 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f340a7200 con 0x7f8f340a6da0 2024-08-19T10:42:10.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 1 -- 172.21.15.190:0/3351454271 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8f1c018020 con 0x7f8f340a6da0 2024-08-19T10:42:10.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 1 --2- 172.21.15.190:0/3351454271 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8f1403be00 0x7f8f1403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:10.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f317fa640 1 -- 172.21.15.190:0/3351454271 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(127..127 src has 1..127) ==== 6599+0+0 (secure 0 0 0) 0x7f8f1c04f230 con 0x7f8f340a6da0 2024-08-19T10:42:10.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f33fff640 1 --2- 172.21.15.190:0/3351454271 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8f1403be00 0x7f8f1403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:10.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.541+0000 7f8f33fff640 10 monclient: get_auth_request con 0x7f8f1403be00 auth_method 0 2024-08-19T10:42:10.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.545+0000 7f8f33fff640 1 --2- 172.21.15.190:0/3351454271 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8f1403be00 0x7f8f1403e2c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8f340a8600 tx=0x7f8f24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:10.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.549+0000 7f8f317fa640 1 -- 172.21.15.190:0/3351454271 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8f1c01f020 con 0x7f8f340a6da0 2024-08-19T10:42:10.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.549+0000 7f8f317fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:10.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.549+0000 7f8f317fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:10.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.725+0000 7f8f3a99c640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:10.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.725+0000 7f8f3a99c640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:10.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.725+0000 7f8f3a99c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:10.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.725+0000 7f8f3a99c640 1 -- 172.21.15.190:0/3351454271 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) -- 0x7f8f340a01f0 con 0x7f8f340a6da0 2024-08-19T10:42:10.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.905+0000 7f8f317fa640 1 -- 172.21.15.190:0/3351454271 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=-2 pool 'fooset' not found v127) ==== 152+0+0 (secure 0 0 0) 0x7f8f1c00d040 con 0x7f8f340a6da0 2024-08-19T10:42:10.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.905+0000 7f8f317fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:10.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:10.905+0000 7f8f317fa640 10 monclient: _finish_command 2 = mon:2 pool 'fooset' not found 2024-08-19T10:42:10.908 INFO:tasks.workunit.client.0.smithi190.stderr:Error ENOENT: pool 'fooset' not found 2024-08-19T10:42:10.913 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:42:10.913 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create fooset 8 2024-08-19T10:42:11.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.025+0000 7f9019e62640 10 monclient: get_monmap_and_config 2024-08-19T10:42:11.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.025+0000 7f9019e62640 10 monclient: build_initial_monmap 2024-08-19T10:42:11.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.025+0000 7f9019e62640 10 monclient: monmap: 2024-08-19T10:42:11.030 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:11.030 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:11.030 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:11.030437+0000 2024-08-19T10:42:11.030 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:11.030437+0000 2024-08-19T10:42:11.030 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:11.030 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.031 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:11.031 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:11.031 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:11.031 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:11.031 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:11.031 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:11.031 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:11.031 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:11.031 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:11.031 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:11.031 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:11.032 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.025+0000 7f9019e62640 1 Processor -- start 2024-08-19T10:42:11.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.025+0000 7f9019e62640 1 -- start start 2024-08-19T10:42:11.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.025+0000 7f9019e62640 10 monclient: init 2024-08-19T10:42:11.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:11.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 10 monclient: _add_conns ranks=[7,2,1,5,9,10,6,8,4,0,3] 2024-08-19T10:42:11.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 0x7f9014139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 10 monclient(hunting): picked mon.noname-i con 0x7f9014139980 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:11.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f901413a2a0 0x7f90140a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 10 monclient(hunting): picked mon.noname-b con 0x7f901413a2a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:11.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f90140a5c70 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:11.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90140a8470 con 0x7f90140a5c70 2024-08-19T10:42:11.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:11.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f901409ca60 con 0x7f901413a2a0 2024-08-19T10:42:11.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:11.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f90140a3640 con 0x7f9014139980 2024-08-19T10:42:11.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:11.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9019e62640 10 monclient(hunting): authenticate will time out at 1176.600557s 2024-08-19T10:42:11.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9013fff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f90140a5c70 0x7f90140a67e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:43004/0 (socket says 172.21.15.190:43004) 2024-08-19T10:42:11.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9013fff640 1 -- 172.21.15.190:0/290963471 learned_addr learned my addr 172.21.15.190:0/290963471 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:11.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 1 --2- 172.21.15.190:0/290963471 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 0x7f9014139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 10 monclient(hunting): get_auth_request con 0x7f9014139980 auth_method 0 2024-08-19T10:42:11.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:11.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:11.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:11.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:11.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:11.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:11.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9012ffd640 1 --2- 172.21.15.190:0/290963471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f901413a2a0 0x7f90140a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9012ffd640 10 monclient(hunting): get_auth_request con 0x7f901413a2a0 auth_method 0 2024-08-19T10:42:11.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9012ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:11.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9012ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:11.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f9012ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:11.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 10 monclient(hunting): handle_auth_done global_id 14677 payload 274 2024-08-19T10:42:11.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 1 -- 172.21.15.190:0/290963471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f901413a2a0 msgr2=0x7f90140a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:11.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 1 --2- 172.21.15.190:0/290963471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f901413a2a0 0x7f90140a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 1 -- 172.21.15.190:0/290963471 >> v1:172.21.15.190:6789/0 conn(0x7f90140a5c70 legacy=0x7f90140a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:11.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:11.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 1 monclient: found mon.noname-i 2024-08-19T10:42:11.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:11.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:11.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.029+0000 7f90137fe640 1 -- 172.21.15.190:0/290963471 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90140a3850 con 0x7f9014139980 2024-08-19T10:42:11.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90137fe640 1 -- 172.21.15.190:0/290963471 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 msgr2=0x7f9014139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:11.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90137fe640 1 --2- 172.21.15.190:0/290963471 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 0x7f9014139d60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90040097f0 tx=0x7f900401c370 comp rx=0 tx=0).ready entity=mon.7 client_cookie=c4e56c42d0baad2d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:11.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 1 -- 172.21.15.190:0/290963471 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90040b0070 con 0x7f9014139980 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 10 monclient: dump: 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:11.051 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 5 monclient: authenticate success, global_id 14677 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 1 -- 172.21.15.190:0/290963471 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90040aa810 con 0x7f9014139980 2024-08-19T10:42:11.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 4 set_mon_vals no callback set 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 1 -- 172.21.15.190:0/290963471 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90040a4e60 con 0x7f9014139980 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f90127fc640 10 monclient: dump: 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.053 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:11.054 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.054 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:11.054 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:11.054 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:11.054 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:11.054 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:11.054 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:11.054 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:11.054 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:11.054 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:11.054 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 10 monclient: shutdown 2024-08-19T10:42:11.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:11.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 1 -- 172.21.15.190:0/290963471 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 msgr2=0x7f9014139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:11.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 1 --2- 172.21.15.190:0/290963471 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 0x7f9014139d60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90040097f0 tx=0x7f900401c370 comp rx=0 tx=0).stop 2024-08-19T10:42:11.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 1 -- 172.21.15.190:0/290963471 shutdown_connections 2024-08-19T10:42:11.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 1 --2- 172.21.15.190:0/290963471 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f901413a2a0 0x7f90140a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 1 --2- 172.21.15.190:0/290963471 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 0x7f9014139d60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 1 -- 172.21.15.190:0/290963471 >> 172.21.15.190:0/290963471 conn(0x7f90140abde0 msgr2=0x7f90140ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:11.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 1 -- 172.21.15.190:0/290963471 shutdown_connections 2024-08-19T10:42:11.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 1 -- 172.21.15.190:0/290963471 wait complete. 2024-08-19T10:42:11.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 10 monclient: build_initial_monmap 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 10 monclient: monmap: 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:11.056 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 1 Processor -- start 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 1 -- start start 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.033+0000 7f9019e62640 10 monclient: init 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient: _add_conns ranks=[8,7,6,4,2,0,3,5,1] 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90140a5c70 0x7f9014140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient(hunting): picked mon.noname-e con 0x7f90140a5c70 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 0x7f9014140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient(hunting): picked mon.noname-i con 0x7f9014139980 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:11.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90141d1510 0x7f90141d1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient(hunting): picked mon.noname-d con 0x7f90141d1510 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:11.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:11.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9014144280 con 0x7f90141d1510 2024-08-19T10:42:11.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:11.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9014144100 con 0x7f9014139980 2024-08-19T10:42:11.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:11.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9014144400 con 0x7f90140a5c70 2024-08-19T10:42:11.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:11.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient(hunting): authenticate will time out at 1176.608143s 2024-08-19T10:42:11.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 0x7f9014140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 0x7f9014140db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42258/0 (socket says 172.21.15.190:42258) 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 1 -- 172.21.15.190:0/3690038933 learned_addr learned my addr 172.21.15.190:0/3690038933 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 10 monclient(hunting): get_auth_request con 0x7f9014139980 auth_method 0 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f90137fe640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90140a5c70 0x7f9014140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9013fff640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90141d1510 0x7f90141d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 10 monclient(hunting): handle_auth_done global_id 14686 payload 824 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 1 -- 172.21.15.190:0/3690038933 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90140a5c70 msgr2=0x7f9014140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:11.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90140a5c70 0x7f9014140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 1 -- 172.21.15.190:0/3690038933 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90141d1510 msgr2=0x7f90141d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90141d1510 0x7f90141d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 1 monclient: found mon.noname-i 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 1 -- 172.21.15.190:0/3690038933 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9014077810 con 0x7f9014139980 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f90137fe640 10 monclient: get_auth_request con 0x7f90140a5c70 auth_method 0 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f90137fe640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90140a5c70 0x7f9014140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9013fff640 10 monclient: get_auth_request con 0x7f90141d1510 auth_method 0 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9013fff640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90141d1510 0x7f90141d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 1 -- 172.21.15.190:0/3690038933 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 msgr2=0x7f9014140db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:11.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9012ffd640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 0x7f9014140db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f900800d5f0 tx=0x7f900800dab0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:11.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 1 -- 172.21.15.190:0/3690038933 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9008007380 con 0x7f9014139980 2024-08-19T10:42:11.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 10 monclient: dump: 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:11.062 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 10 monclient: _finish_auth 0 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 1 -- 172.21.15.190:0/3690038933 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9008019070 con 0x7f9014139980 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 5 monclient: authenticate success, global_id 14686 2024-08-19T10:42:11.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient: _renew_subs 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 1 -- 172.21.15.190:0/3690038933 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90140a1a60 con 0x7f9014139980 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 1 -- 172.21.15.190:0/3690038933 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9008012cb0 con 0x7f9014139980 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9011ffb640 4 set_mon_vals no callback set 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f8ffbfff640 10 monclient: dump: 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.064 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient: _renew_subs 2024-08-19T10:42:11.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:11.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 1 -- 172.21.15.190:0/3690038933 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f901413ca80 con 0x7f9014139980 2024-08-19T10:42:11.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:11.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:11.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:11.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.037+0000 7f9019e62640 1 -- 172.21.15.190:0/3690038933 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90140ac430 con 0x7f9014139980 2024-08-19T10:42:11.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.041+0000 7f8ffbfff640 1 -- 172.21.15.190:0/3690038933 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9008012800 con 0x7f9014139980 2024-08-19T10:42:11.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.041+0000 7f8ffbfff640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f900003be00 0x7f900003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.041+0000 7f8ffbfff640 1 -- 172.21.15.190:0/3690038933 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(127..127 src has 1..127) ==== 6599+0+0 (secure 0 0 0) 0x7f900800f040 con 0x7f9014139980 2024-08-19T10:42:11.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.041+0000 7f90137fe640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f900003be00 0x7f900003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.041+0000 7f90137fe640 10 monclient: get_auth_request con 0x7f900003be00 auth_method 0 2024-08-19T10:42:11.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.041+0000 7f90137fe640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f900003be00 0x7f900003e2c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f90040097f0 tx=0x7f90040a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:11.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.045+0000 7f8ffbfff640 1 -- 172.21.15.190:0/3690038933 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9008023020 con 0x7f9014139980 2024-08-19T10:42:11.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.045+0000 7f8ffbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:11.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.045+0000 7f8ffbfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:11.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.201+0000 7f9019e62640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-08-19T10:42:11.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.201+0000 7f9019e62640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-08-19T10:42:11.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.201+0000 7f9019e62640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:11.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.201+0000 7f9019e62640 1 -- 172.21.15.190:0/3690038933 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd pool create", "pool": "fooset", "pg_num": 8} v 0) -- 0x7f90140ac680 con 0x7f9014139980 2024-08-19T10:42:11.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.569+0000 7f8ffbfff640 1 -- 172.21.15.190:0/3690038933 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}]=0 pool 'fooset' created v128) ==== 115+0+0 (secure 0 0 0) 0x7f90140ac680 con 0x7f9014139980 2024-08-19T10:42:11.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.569+0000 7f8ffbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-08-19T10:42:11.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.569+0000 7f8ffbfff640 10 monclient: _finish_command 2 = system:0 pool 'fooset' created 2024-08-19T10:42:11.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.669+0000 7f9019e62640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-08-19T10:42:11.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.669+0000 7f9019e62640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-08-19T10:42:11.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.669+0000 7f9019e62640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:11.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.669+0000 7f9019e62640 1 -- 172.21.15.190:0/3690038933 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd pool create", "pool": "fooset", "pg_num": 8} v 0) -- 0x7f9014139d60 con 0x7f9014139980 2024-08-19T10:42:11.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.669+0000 7f8ffbfff640 1 -- 172.21.15.190:0/3690038933 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}]=0 pool 'fooset' already exists v128) ==== 122+0+0 (secure 0 0 0) 0x7f9008011340 con 0x7f9014139980 2024-08-19T10:42:11.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.669+0000 7f8ffbfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "fooset", "pg_num": 8}] 2024-08-19T10:42:11.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.669+0000 7f8ffbfff640 10 monclient: _finish_command 3 = system:0 pool 'fooset' already exists 2024-08-19T10:42:11.673 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'fooset' already exists 2024-08-19T10:42:11.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 -- 172.21.15.190:0/3690038933 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f900003be00 msgr2=0x7f900003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:11.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f900003be00 0x7f900003e2c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f90040097f0 tx=0x7f90040a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 10 monclient: shutdown 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 -- 172.21.15.190:0/3690038933 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 msgr2=0x7f9014140db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 0x7f9014140db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f900800d5f0 tx=0x7f900800dab0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 -- 172.21.15.190:0/3690038933 shutdown_connections 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90141d1510 0x7f90141d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9014139980 0x7f9014140db0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f900003be00 0x7f900003e2c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 --2- 172.21.15.190:0/3690038933 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90140a5c70 0x7f9014140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 -- 172.21.15.190:0/3690038933 >> 172.21.15.190:0/3690038933 conn(0x7f90140abde0 msgr2=0x7f90140a90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 -- 172.21.15.190:0/3690038933 shutdown_connections 2024-08-19T10:42:11.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.673+0000 7f9019e62640 1 -- 172.21.15.190:0/3690038933 wait complete. 2024-08-19T10:42:11.691 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create barset 8 2024-08-19T10:42:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.793+0000 7fcd9bb9a640 10 monclient: get_monmap_and_config 2024-08-19T10:42:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.793+0000 7fcd9bb9a640 10 monclient: build_initial_monmap 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.793+0000 7fcd9bb9a640 10 monclient: monmap: 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:11.798424+0000 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:11.798424+0000 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:11.799 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.793+0000 7fcd9bb9a640 1 Processor -- start 2024-08-19T10:42:11.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.793+0000 7fcd9bb9a640 1 -- start start 2024-08-19T10:42:11.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.793+0000 7fcd9bb9a640 10 monclient: init 2024-08-19T10:42:11.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:11.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 10 monclient: _add_conns ranks=[6,9,2,0,7,4,3,1,10,5,8] 2024-08-19T10:42:11.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcd94139a60 0x7fcd94139e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 10 monclient(hunting): picked mon.noname-d con 0x7fcd94139a60 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:11.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcd940a5730 0x7fcd940a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 10 monclient(hunting): picked mon.noname-a con 0x7fcd940a5730 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd940a6850 0x7fcd940a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 10 monclient(hunting): picked mon.noname-b con 0x7fcd940a6850 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9409ca60 con 0x7fcd940a5730 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fcd940a33c0 con 0x7fcd940a6850 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fcd940a3640 con 0x7fcd94139a60 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9bb9a640 10 monclient(hunting): authenticate will time out at 1177.368242s 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd940a6850 0x7fcd940a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99936640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcd94139a60 0x7fcd94139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd940a6850 0x7fcd940a6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:36208/0 (socket says 172.21.15.190:36208) 2024-08-19T10:42:11.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 1 -- 172.21.15.190:0/1675156045 learned_addr learned my addr 172.21.15.190:0/1675156045 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:11.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 10 monclient(hunting): get_auth_request con 0x7fcd940a6850 auth_method 0 2024-08-19T10:42:11.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:11.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:11.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:11.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99135640 1 --2- 172.21.15.190:0/1675156045 >> v2:172.21.15.89:3300/0 conn(0x7fcd940a5730 0x7fcd940a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99936640 10 monclient(hunting): get_auth_request con 0x7fcd94139a60 auth_method 0 2024-08-19T10:42:11.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99936640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:11.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99936640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99936640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99135640 10 monclient(hunting): get_auth_request con 0x7fcd940a5730 auth_method 0 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99135640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99135640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99135640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99936640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99936640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99936640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99135640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:11.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99135640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:11.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd99135640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:11.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 10 monclient(hunting): handle_auth_done global_id 5501 payload 274 2024-08-19T10:42:11.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 1 -- 172.21.15.190:0/1675156045 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcd94139a60 msgr2=0x7fcd94139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:11.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 1 --2- 172.21.15.190:0/1675156045 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcd94139a60 0x7fcd94139e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 1 -- 172.21.15.190:0/1675156045 >> v2:172.21.15.89:3300/0 conn(0x7fcd940a5730 msgr2=0x7fcd940a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:11.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 1 --2- 172.21.15.190:0/1675156045 >> v2:172.21.15.89:3300/0 conn(0x7fcd940a5730 0x7fcd940a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 10 monclient: _finish_hunting 0 2024-08-19T10:42:11.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 1 monclient: found mon.noname-b 2024-08-19T10:42:11.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:11.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:42:11.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.797+0000 7fcd9a137640 1 -- 172.21.15.190:0/1675156045 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd940a46e0 con 0x7fcd940a6850 2024-08-19T10:42:11.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9a137640 1 -- 172.21.15.190:0/1675156045 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd940a6850 msgr2=0x7fcd940a6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:11.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9a137640 1 --2- 172.21.15.190:0/1675156045 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd940a6850 0x7fcd940a6cb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009e50 tx=0x7fcd8c01c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b400abffe0387dd4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:11.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 1 -- 172.21.15.190:0/1675156045 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcd8c0090d0 con 0x7fcd940a6850 2024-08-19T10:42:11.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 10 monclient: dump: 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:11.814 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 10 monclient: _finish_auth 0 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 10 monclient: _check_auth_tickets 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 1 -- 172.21.15.190:0/1675156045 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8c0a59a0 con 0x7fcd940a6850 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 5 monclient: authenticate success, global_id 5501 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 1 -- 172.21.15.190:0/1675156045 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcd8c0ad3f0 con 0x7fcd940a6850 2024-08-19T10:42:11.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd98934640 10 monclient: dump: 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:11.816 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:11.817 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:11.817 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:11.817 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:11.817 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:11.817 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:11.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 4 set_mon_vals no callback set 2024-08-19T10:42:11.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 10 monclient: shutdown 2024-08-19T10:42:11.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/1675156045 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd940a6850 msgr2=0x7fcd940a6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 1 --2- 172.21.15.190:0/1675156045 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd940a6850 0x7fcd940a6cb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009e50 tx=0x7fcd8c01c8f0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/1675156045 shutdown_connections 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 1 --2- 172.21.15.190:0/1675156045 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcd940a6850 0x7fcd940a6cb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 1 --2- 172.21.15.190:0/1675156045 >> v2:172.21.15.89:3300/0 conn(0x7fcd940a5730 0x7fcd940a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 1 --2- 172.21.15.190:0/1675156045 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcd94139a60 0x7fcd94139e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/1675156045 >> 172.21.15.190:0/1675156045 conn(0x7fcd940abe50 msgr2=0x7fcd940ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/1675156045 shutdown_connections 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/1675156045 wait complete. 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 10 monclient: build_initial_monmap 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.801+0000 7fcd9bb9a640 10 monclient: monmap: 2024-08-19T10:42:11.818 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:11.819 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 1 Processor -- start 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 1 -- start start 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient: init 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient: _add_conns ranks=[8,0,1,3,7,5,4,2,6] 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd940a5730 0x7fcd941349b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient(hunting): picked mon.noname-e con 0x7fcd940a5730 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd940a6850 0x7fcd94134ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient(hunting): picked mon.noname-a con 0x7fcd940a6850 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd94139a60 0x7fcd9413a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient(hunting): picked mon.noname-f con 0x7fcd94139a60 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:11.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcd94143270 con 0x7fcd940a6850 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcd941430f0 con 0x7fcd94139a60 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fcd941433f0 con 0x7fcd940a5730 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient(hunting): authenticate will time out at 1177.376749s 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd94139a60 0x7fcd9413a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd94139a60 0x7fcd9413a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49944/0 (socket says 172.21.15.190:49944) 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 1 -- 172.21.15.190:0/3908302837 learned_addr learned my addr 172.21.15.190:0/3908302837 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 10 monclient(hunting): get_auth_request con 0x7fcd94139a60 auth_method 0 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:11.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:11.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:11.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:11.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:11.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:11.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd99135640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd940a6850 0x7fcd94134ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd99135640 10 monclient(hunting): get_auth_request con 0x7fcd940a6850 auth_method 0 2024-08-19T10:42:11.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd99135640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd99135640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd99135640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd99936640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd940a5730 0x7fcd941349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 10 monclient(hunting): handle_auth_done global_id 14752 payload 824 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 1 -- 172.21.15.190:0/3908302837 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd940a5730 msgr2=0x7fcd941349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd940a5730 0x7fcd941349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 1 -- 172.21.15.190:0/3908302837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd940a6850 msgr2=0x7fcd94134ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd940a6850 0x7fcd94134ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 10 monclient: _finish_hunting 0 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 1 monclient: found mon.noname-f 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:11.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 1 -- 172.21.15.190:0/3908302837 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd94139e40 con 0x7fcd94139a60 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd99936640 10 monclient: get_auth_request con 0x7fcd940a5730 auth_method 0 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd99936640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd940a5730 0x7fcd941349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 1 -- 172.21.15.190:0/3908302837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd94139a60 msgr2=0x7fcd9413a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9a137640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd94139a60 0x7fcd9413a590 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0af040 tx=0x7fcd8c019690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 1 -- 172.21.15.190:0/3908302837 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcd8c003e10 con 0x7fcd94139a60 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 10 monclient: dump: 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.824 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:11.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 1 -- 172.21.15.190:0/3908302837 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8c0bc070 con 0x7fcd94139a60 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 1 -- 172.21.15.190:0/3908302837 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcd8c0a5d60 con 0x7fcd94139a60 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8a7fc640 10 monclient: dump: 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:11.826 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd8b7fe640 4 set_mon_vals no callback set 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 5 monclient: authenticate success, global_id 14752 2024-08-19T10:42:11.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient: _renew_subs 2024-08-19T10:42:11.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:11.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/3908302837 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9413e070 con 0x7fcd94139a60 2024-08-19T10:42:11.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient: _renew_subs 2024-08-19T10:42:11.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:11.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.805+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/3908302837 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9413c870 con 0x7fcd94139a60 2024-08-19T10:42:11.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.809+0000 7fcd9bb9a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:11.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.809+0000 7fcd9bb9a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:11.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.809+0000 7fcd9bb9a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:11.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.809+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/3908302837 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd940a6cb0 con 0x7fcd94139a60 2024-08-19T10:42:11.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.809+0000 7fcd8a7fc640 1 -- 172.21.15.190:0/3908302837 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fcd8c0039a0 con 0x7fcd94139a60 2024-08-19T10:42:11.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.809+0000 7fcd8a7fc640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd7803be00 0x7fcd7803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:11.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.809+0000 7fcd8a7fc640 1 -- 172.21.15.190:0/3908302837 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6944+0+0 (secure 0 0 0) 0x7fcd8c0e9630 con 0x7fcd94139a60 2024-08-19T10:42:11.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.809+0000 7fcd99936640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd7803be00 0x7fcd7803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:11.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.809+0000 7fcd99936640 10 monclient: get_auth_request con 0x7fcd7803be00 auth_method 0 2024-08-19T10:42:11.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.809+0000 7fcd99936640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd7803be00 0x7fcd7803e2c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcd900099c0 tx=0x7fcd90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:11.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.813+0000 7fcd8a7fc640 1 -- 172.21.15.190:0/3908302837 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fcd8c0c5020 con 0x7fcd94139a60 2024-08-19T10:42:11.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.813+0000 7fcd8a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:11.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.813+0000 7fcd8a7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:11.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.985+0000 7fcd9bb9a640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-08-19T10:42:11.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.985+0000 7fcd9bb9a640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-08-19T10:42:11.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.985+0000 7fcd9bb9a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:11.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:11.985+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/3908302837 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "barset", "pg_num": 8} v 0) -- 0x7fcd940a5b10 con 0x7fcd94139a60 2024-08-19T10:42:12.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.581+0000 7fcd8a7fc640 1 -- 172.21.15.190:0/3908302837 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}]=0 pool 'barset' created v129) ==== 115+0+0 (secure 0 0 0) 0x7fcd94139e40 con 0x7fcd94139a60 2024-08-19T10:42:12.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.581+0000 7fcd8a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-08-19T10:42:12.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.581+0000 7fcd8a7fc640 10 monclient: _finish_command 2 = system:0 pool 'barset' created 2024-08-19T10:42:12.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.693+0000 7fcd9bb9a640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-08-19T10:42:12.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.693+0000 7fcd9bb9a640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-08-19T10:42:12.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.693+0000 7fcd9bb9a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:12.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.693+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/3908302837 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "barset", "pg_num": 8} v 0) -- 0x7fcd940008e0 con 0x7fcd94139a60 2024-08-19T10:42:12.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.693+0000 7fcd8a7fc640 1 -- 172.21.15.190:0/3908302837 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}]=0 pool 'barset' already exists v129) ==== 122+0+0 (secure 0 0 0) 0x7fcd8c016600 con 0x7fcd94139a60 2024-08-19T10:42:12.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.693+0000 7fcd8a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "barset", "pg_num": 8}] 2024-08-19T10:42:12.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.693+0000 7fcd8a7fc640 10 monclient: _finish_command 3 = system:0 pool 'barset' already exists 2024-08-19T10:42:12.697 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'barset' already exists 2024-08-19T10:42:12.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/3908302837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd7803be00 msgr2=0x7fcd7803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:12.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd7803be00 0x7fcd7803e2c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcd900099c0 tx=0x7fcd90008040 comp rx=0 tx=0).stop 2024-08-19T10:42:12.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 10 monclient: shutdown 2024-08-19T10:42:12.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:12.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/3908302837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd94139a60 msgr2=0x7fcd9413a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:12.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd94139a60 0x7fcd9413a590 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0af040 tx=0x7fcd8c019690 comp rx=0 tx=0).stop 2024-08-19T10:42:12.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/3908302837 shutdown_connections 2024-08-19T10:42:12.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd94139a60 0x7fcd9413a590 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:12.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcd940a6850 0x7fcd94134ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:12.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd7803be00 0x7fcd7803e2c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:12.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 --2- 172.21.15.190:0/3908302837 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd940a5730 0x7fcd941349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:12.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/3908302837 >> 172.21.15.190:0/3908302837 conn(0x7fcd940abe50 msgr2=0x7fcd94138d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:12.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/3908302837 shutdown_connections 2024-08-19T10:42:12.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.697+0000 7fcd9bb9a640 1 -- 172.21.15.190:0/3908302837 wait complete. 2024-08-19T10:42:12.715 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set barset size 3 2024-08-19T10:42:12.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.829+0000 7f5b54681640 10 monclient: get_monmap_and_config 2024-08-19T10:42:12.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.829+0000 7f5b54681640 10 monclient: build_initial_monmap 2024-08-19T10:42:12.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.829+0000 7f5b54681640 10 monclient: monmap: 2024-08-19T10:42:12.834 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:12.834 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:12.834 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:12.834695+0000 2024-08-19T10:42:12.834 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:12.834695+0000 2024-08-19T10:42:12.834 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:12.835 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:12.836 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:12.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 1 Processor -- start 2024-08-19T10:42:12.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 1 -- start start 2024-08-19T10:42:12.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 10 monclient: init 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 10 monclient: _add_conns ranks=[0,10,3,6,1,9,7,2,5,4,8] 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5b4c0a46b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5b4c0a51a0 0x7f5b4c13c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 10 monclient(hunting): picked mon.noname-f con 0x7f5b4c0a51a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b4c13ce00 0x7f5b4c13f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 10 monclient(hunting): picked mon.noname-g con 0x7f5b4c13ce00 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b4c09cc70 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f5b4c0a2d60 con 0x7f5b4c0a51a0 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:12.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5b4c0a2460 con 0x7f5b4c13ce00 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b54681640 10 monclient(hunting): authenticate will time out at 1178.405570s 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b52c1e640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b4c13ce00 msgr2=0x7f5b4c13f1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b52c1e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b4c13ce00 0x7f5b4c13f1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5b4c0a51a0 0x7f5b4c13c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5b4c0a51a0 0x7f5b4c13c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49950/0 (socket says 172.21.15.190:49950) 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 1 -- 172.21.15.190:0/1003334691 learned_addr learned my addr 172.21.15.190:0/1003334691 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 10 monclient(hunting): get_auth_request con 0x7f5b4c0a51a0 auth_method 0 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:12.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:12.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:12.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:12.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 10 monclient(hunting): handle_auth_done global_id 14761 payload 274 2024-08-19T10:42:12.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 1 -- 172.21.15.190:0/1003334691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b4c13ce00 msgr2=0x7f5b4c13f1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:12.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 1 --2- 172.21.15.190:0/1003334691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b4c13ce00 0x7f5b4c13f1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:12.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 1 -- 172.21.15.190:0/1003334691 >> v1:172.21.15.89:6789/0 conn(0x7f5b4c0a46b0 legacy=0x7f5b4c0a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:12.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 10 monclient: _finish_hunting 0 2024-08-19T10:42:12.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 1 monclient: found mon.noname-f 2024-08-19T10:42:12.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:12.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:12.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.833+0000 7f5b51c1c640 1 -- 172.21.15.190:0/1003334691 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c0a2ae0 con 0x7f5b4c0a51a0 2024-08-19T10:42:12.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b51c1c640 1 -- 172.21.15.190:0/1003334691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b4c0a51a0 msgr2=0x7f5b4c13c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:12.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b51c1c640 1 --2- 172.21.15.190:0/1003334691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b4c0a51a0 0x7f5b4c13c8c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009870 tx=0x7f5b3c01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f323279945714dc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:12.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 1 -- 172.21.15.190:0/1003334691 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5b3c0b0070 con 0x7f5b4c0a51a0 2024-08-19T10:42:12.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:12.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 10 monclient: dump: 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:12.843 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 10 monclient: _finish_auth 0 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 10 monclient: _check_auth_tickets 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 5 monclient: authenticate success, global_id 14761 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 1 -- 172.21.15.190:0/1003334691 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c0aa810 con 0x7f5b4c0a51a0 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:12.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 1 -- 172.21.15.190:0/1003334691 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5b3c0a93e0 con 0x7f5b4c0a51a0 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 4 set_mon_vals no callback set 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b5141b640 10 monclient: dump: 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:12.845 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 10 monclient: shutdown 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 1 -- 172.21.15.190:0/1003334691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b4c0a51a0 msgr2=0x7f5b4c13c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 1 --2- 172.21.15.190:0/1003334691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b4c0a51a0 0x7f5b4c13c8c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009870 tx=0x7f5b3c01c440 comp rx=0 tx=0).stop 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 1 -- 172.21.15.190:0/1003334691 shutdown_connections 2024-08-19T10:42:12.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 1 --2- 172.21.15.190:0/1003334691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b4c0a51a0 0x7f5b4c13c8c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 1 --2- 172.21.15.190:0/1003334691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b4c13ce00 0x7f5b4c13f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 1 -- 172.21.15.190:0/1003334691 >> 172.21.15.190:0/1003334691 conn(0x7f5b4c0a92a0 msgr2=0x7f5b4c0a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 1 -- 172.21.15.190:0/1003334691 shutdown_connections 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 1 -- 172.21.15.190:0/1003334691 wait complete. 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 10 monclient: build_initial_monmap 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 10 monclient: monmap: 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:12.847 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:12.848 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.837+0000 7f5b54681640 1 Processor -- start 2024-08-19T10:42:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 1 -- start start 2024-08-19T10:42:12.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient: init 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient: _add_conns ranks=[7,0,8,2,5,6,4,3,1] 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4c0a46b0 0x7f5b4c134a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient(hunting): picked mon.noname-i con 0x7f5b4c0a46b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4c0a51a0 0x7f5b4c13b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient(hunting): picked mon.noname-a con 0x7f5b4c0a51a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4c13ce00 0x7f5b4c0a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient(hunting): picked mon.noname-e con 0x7f5b4c13ce00 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c0a30f0 con 0x7f5b4c0a51a0 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:12.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5b4c0a2f70 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f5b4c09fd50 con 0x7f5b4c13ce00 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient(hunting): authenticate will time out at 1178.412504s 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4c0a46b0 0x7f5b4c134a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4c0a46b0 0x7f5b4c134a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42264/0 (socket says 172.21.15.190:42264) 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 1 -- 172.21.15.190:0/275799767 learned_addr learned my addr 172.21.15.190:0/275799767 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 10 monclient(hunting): get_auth_request con 0x7f5b4c0a46b0 auth_method 0 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:12.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b52c1e640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4c13ce00 0x7f5b4c0a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b51c1c640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4c0a51a0 0x7f5b4c13b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 10 monclient(hunting): handle_auth_done global_id 14695 payload 824 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 1 -- 172.21.15.190:0/275799767 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4c13ce00 msgr2=0x7f5b4c0a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4c13ce00 0x7f5b4c0a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 1 -- 172.21.15.190:0/275799767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4c0a51a0 msgr2=0x7f5b4c13b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4c0a51a0 0x7f5b4c13b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 10 monclient: _finish_hunting 0 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 1 monclient: found mon.noname-i 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:12.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 1 -- 172.21.15.190:0/275799767 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c139fb0 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b52c1e640 10 monclient: get_auth_request con 0x7f5b4c13ce00 auth_method 0 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b51c1c640 10 monclient: get_auth_request con 0x7f5b4c0a51a0 auth_method 0 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b52c1e640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4c13ce00 0x7f5b4c0a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b51c1c640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4c0a51a0 0x7f5b4c13b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 1 -- 172.21.15.190:0/275799767 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4c0a46b0 msgr2=0x7f5b4c134a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b5241d640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4c0a46b0 0x7f5b4c134a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c4c0 tx=0x7f5b48001e80 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 1 -- 172.21.15.190:0/275799767 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5b48017070 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 10 monclient: dump: 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:12.855 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:12.856 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 10 monclient: _finish_auth 0 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 1 -- 172.21.15.190:0/275799767 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b48003a70 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 5 monclient: authenticate success, global_id 14695 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient: _renew_subs 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 1 -- 172.21.15.190:0/275799767 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4c09ed40 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 1 -- 172.21.15.190:0/275799767 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5b48003c20 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:12.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b42ffd640 10 monclient: dump: 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:12.858 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient: _renew_subs 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b50c1a640 4 set_mon_vals no callback set 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.841+0000 7f5b54681640 1 -- 172.21.15.190:0/275799767 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4c13c0c0 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.845+0000 7f5b54681640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.845+0000 7f5b54681640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.845+0000 7f5b54681640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.845+0000 7f5b54681640 1 -- 172.21.15.190:0/275799767 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4c0a4a90 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.845+0000 7f5b42ffd640 1 -- 172.21.15.190:0/275799767 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5b48003600 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.845+0000 7f5b42ffd640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5b3003be00 0x7f5b3003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.845+0000 7f5b42ffd640 1 -- 172.21.15.190:0/275799767 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(129..129 src has 1..129) ==== 7289+0+0 (secure 0 0 0) 0x7f5b4804e6b0 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.845+0000 7f5b51c1c640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5b3003be00 0x7f5b3003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.845+0000 7f5b51c1c640 10 monclient: get_auth_request con 0x7f5b3003be00 auth_method 0 2024-08-19T10:42:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.845+0000 7f5b51c1c640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5b3003be00 0x7f5b3003e2c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c09cb80 tx=0x7f5b3c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.849+0000 7f5b42ffd640 1 -- 172.21.15.190:0/275799767 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f5b48010300 con 0x7f5b4c0a46b0 2024-08-19T10:42:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.849+0000 7f5b42ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:12.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:12.849+0000 7f5b42ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:13.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:13.021+0000 7f5b54681640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-08-19T10:42:13.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:13.021+0000 7f5b54681640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-08-19T10:42:13.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:13.021+0000 7f5b54681640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:13.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:13.021+0000 7f5b54681640 1 -- 172.21.15.190:0/275799767 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"} v 0) -- 0x7f5b4c13f200 con 0x7f5b4c0a46b0 2024-08-19T10:42:13.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:13.581+0000 7f5b42ffd640 1 -- 172.21.15.190:0/275799767 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}]=0 set pool 10 size to 3 v130) ==== 126+0+0 (secure 0 0 0) 0x7f5b480104e0 con 0x7f5b4c0a46b0 2024-08-19T10:42:13.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:13.581+0000 7f5b42ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-08-19T10:42:13.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:13.581+0000 7f5b42ffd640 10 monclient: _finish_command 2 = system:0 set pool 10 size to 3 2024-08-19T10:42:13.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:13.685+0000 7f5b54681640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-08-19T10:42:13.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:13.685+0000 7f5b54681640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-08-19T10:42:13.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:13.685+0000 7f5b54681640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:13.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:13.685+0000 7f5b54681640 1 -- 172.21.15.190:0/275799767 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"} v 0) -- 0x7f5b4c0008e0 con 0x7f5b4c0a46b0 2024-08-19T10:42:14.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.589+0000 7f5b42ffd640 1 -- 172.21.15.190:0/275799767 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}]=0 set pool 10 size to 3 v131) ==== 126+0+0 (secure 0 0 0) 0x7f5b48011bf0 con 0x7f5b4c0a46b0 2024-08-19T10:42:14.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.589+0000 7f5b42ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "barset", "var": "size", "val": "3"}] 2024-08-19T10:42:14.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.589+0000 7f5b42ffd640 10 monclient: _finish_command 3 = system:0 set pool 10 size to 3 2024-08-19T10:42:14.592 INFO:tasks.workunit.client.0.smithi190.stderr:set pool 10 size to 3 2024-08-19T10:42:14.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 -- 172.21.15.190:0/275799767 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5b3003be00 msgr2=0x7f5b3003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:14.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5b3003be00 0x7f5b3003e2c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c09cb80 tx=0x7f5b3c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:14.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 10 monclient: shutdown 2024-08-19T10:42:14.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:14.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 -- 172.21.15.190:0/275799767 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4c0a46b0 msgr2=0x7f5b4c134a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:14.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4c0a46b0 0x7f5b4c134a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c4c0 tx=0x7f5b48001e80 comp rx=0 tx=0).stop 2024-08-19T10:42:14.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 -- 172.21.15.190:0/275799767 shutdown_connections 2024-08-19T10:42:14.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5b3003be00 0x7f5b3003e2c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:14.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4c13ce00 0x7f5b4c0a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:14.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b4c0a51a0 0x7f5b4c13b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:14.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 --2- 172.21.15.190:0/275799767 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4c0a46b0 0x7f5b4c134a20 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c4c0 tx=0x7f5b48001e80 comp rx=0 tx=0).stop 2024-08-19T10:42:14.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 -- 172.21.15.190:0/275799767 >> 172.21.15.190:0/275799767 conn(0x7f5b4c0a92a0 msgr2=0x7f5b4c09d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:14.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 -- 172.21.15.190:0/275799767 shutdown_connections 2024-08-19T10:42:14.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.593+0000 7f5b54681640 1 -- 172.21.15.190:0/275799767 wait complete. 2024-08-19T10:42:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 2024-08-19T10:42:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:42:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-08-19T10:42:14.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient: get_monmap_and_config 2024-08-19T10:42:14.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient: build_initial_monmap 2024-08-19T10:42:14.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient: monmap: 2024-08-19T10:42:14.748 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:14.748 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:14.748 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:14.748816+0000 2024-08-19T10:42:14.748 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:14.748816+0000 2024-08-19T10:42:14.748 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:14.749 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:14.750 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:14.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 1 Processor -- start 2024-08-19T10:42:14.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 1 -- start start 2024-08-19T10:42:14.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient: init 2024-08-19T10:42:14.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:14.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient: _add_conns ranks=[9,0,1,3,7,10,2,5,4,8,6] 2024-08-19T10:42:14.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f36f40a58d0 0x7f36f40a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:14.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient(hunting): picked mon.noname-a con 0x7f36f40a58d0 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f36f40a61f0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f36f4137d90 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f4072180 con 0x7f36f40a61f0 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f4073ea0 con 0x7f36f4137d90 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f36f4073bc0 con 0x7f36f40a58d0 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36fae2a640 10 monclient(hunting): authenticate will time out at 1180.319490s 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36f93c7640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f36f4137d90 0x7f36f413f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:43016/0 (socket says 172.21.15.190:43016) 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36f93c7640 1 -- 172.21.15.190:0/3949034773 learned_addr learned my addr 172.21.15.190:0/3949034773 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36f8bc6640 1 --2- 172.21.15.190:0/3949034773 >> v2:172.21.15.89:3300/0 conn(0x7f36f40a58d0 0x7f36f40a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:14.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36f8bc6640 10 monclient(hunting): get_auth_request con 0x7f36f40a58d0 auth_method 0 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36f8bc6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.745+0000 7f36f8bc6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 10 monclient(hunting): handle_auth_done global_id 5499 payload 274 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 1 -- 172.21.15.190:0/3949034773 >> v1:172.21.15.190:6789/0 conn(0x7f36f4137d90 legacy=0x7f36f413f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 1 -- 172.21.15.190:0/3949034773 >> v1:172.21.15.89:6789/0 conn(0x7f36f40a61f0 legacy=0x7f36f4137680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 10 monclient: _finish_hunting 0 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 1 monclient: found mon.noname-a 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:14.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:14.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.749+0000 7f36f8bc6640 1 -- 172.21.15.190:0/3949034773 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f4074180 con 0x7f36f40a58d0 2024-08-19T10:42:14.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f8bc6640 1 -- 172.21.15.190:0/3949034773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36f40a58d0 msgr2=0x7f36f40a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:14.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f8bc6640 1 --2- 172.21.15.190:0/3949034773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36f40a58d0 0x7f36f40a5cb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f36e80097f0 tx=0x7f36e801c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fc14fefb9dea6e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:14.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 1 -- 172.21.15.190:0/3949034773 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36e80b0070 con 0x7f36f40a58d0 2024-08-19T10:42:14.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:14.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:14.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 10 monclient: dump: 2024-08-19T10:42:14.758 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:14.758 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:14.759 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:14.759 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:14.759 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:14.759 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:14.759 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:14.759 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:14.759 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:14.759 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:14.759 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:14.759 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:14.759 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 1 -- 172.21.15.190:0/3949034773 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e80aa810 con 0x7f36f40a58d0 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 1 -- 172.21.15.190:0/3949034773 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36e80a93e0 con 0x7f36f40a58d0 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36f37fe640 10 monclient: dump: 2024-08-19T10:42:14.760 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:14.761 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 5 monclient: authenticate success, global_id 5499 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 4 set_mon_vals no callback set 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 10 monclient: shutdown 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 1 -- 172.21.15.190:0/3949034773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36f40a58d0 msgr2=0x7f36f40a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 1 --2- 172.21.15.190:0/3949034773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36f40a58d0 0x7f36f40a5cb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f36e80097f0 tx=0x7f36e801c320 comp rx=0 tx=0).stop 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 1 -- 172.21.15.190:0/3949034773 shutdown_connections 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 1 --2- 172.21.15.190:0/3949034773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36f40a58d0 0x7f36f40a5cb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:14.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 1 -- 172.21.15.190:0/3949034773 >> 172.21.15.190:0/3949034773 conn(0x7f36f40aa4c0 msgr2=0x7f36f40a69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 1 -- 172.21.15.190:0/3949034773 shutdown_connections 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 1 -- 172.21.15.190:0/3949034773 wait complete. 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 10 monclient: build_initial_monmap 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 10 monclient: monmap: 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:14.763 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:14.764 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:14.764 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:14.764 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:14.764 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:14.764 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:14.764 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:14.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 1 Processor -- start 2024-08-19T10:42:14.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 1 -- start start 2024-08-19T10:42:14.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.753+0000 7f36fae2a640 10 monclient: init 2024-08-19T10:42:14.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:14.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient: _add_conns ranks=[8,1,7,5,4,2,3,0,6] 2024-08-19T10:42:14.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f36f40a58d0 0x7f36f406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:14.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient(hunting): picked mon.noname-e con 0x7f36f40a58d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:14.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40a61f0 0x7f36f406e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:14.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient(hunting): picked mon.noname-f con 0x7f36f40a61f0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:14.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f36f4137d90 0x7f36f406e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:14.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient(hunting): picked mon.noname-i con 0x7f36f4137d90 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:14.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:14.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36f40a65d0 con 0x7f36f40a61f0 2024-08-19T10:42:14.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:14.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f36f40a7420 con 0x7f36f4137d90 2024-08-19T10:42:14.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:14.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f36f40a5610 con 0x7f36f40a58d0 2024-08-19T10:42:14.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:14.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient(hunting): authenticate will time out at 1180.328594s 2024-08-19T10:42:14.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40a61f0 0x7f36f406e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:14.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f93c7640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f36f4137d90 0x7f36f406e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:14.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40a61f0 0x7f36f406e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49966/0 (socket says 172.21.15.190:49966) 2024-08-19T10:42:14.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 1 -- 172.21.15.190:0/3651573384 learned_addr learned my addr 172.21.15.190:0/3651573384 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:14.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 10 monclient(hunting): get_auth_request con 0x7f36f40a61f0 auth_method 0 2024-08-19T10:42:14.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:14.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f93c7640 10 monclient(hunting): get_auth_request con 0x7f36f4137d90 auth_method 0 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f93c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f93c7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f93c7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f93c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f93c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f93c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:14.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f8bc6640 1 --2- 172.21.15.190:0/3651573384 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f36f40a58d0 0x7f36f406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f8bc6640 10 monclient(hunting): get_auth_request con 0x7f36f40a58d0 auth_method 0 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f8bc6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f8bc6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f8bc6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 10 monclient(hunting): handle_auth_done global_id 14770 payload 824 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 1 -- 172.21.15.190:0/3651573384 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f36f40a58d0 msgr2=0x7f36f406dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 1 --2- 172.21.15.190:0/3651573384 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f36f40a58d0 0x7f36f406dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 1 -- 172.21.15.190:0/3651573384 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f36f4137d90 msgr2=0x7f36f406e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 1 --2- 172.21.15.190:0/3651573384 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f36f4137d90 0x7f36f406e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 1 monclient: found mon.noname-f 2024-08-19T10:42:14.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 1 -- 172.21.15.190:0/3651573384 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f40a50c0 con 0x7f36f40a61f0 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f93c7640 1 --2- 172.21.15.190:0/3651573384 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f36f4137d90 0x7f36f406e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 1 -- 172.21.15.190:0/3651573384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40a61f0 msgr2=0x7f36f406e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f3fff640 1 --2- 172.21.15.190:0/3651573384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f40a61f0 0x7f36f406e240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36e000c450 tx=0x7f36e000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 1 -- 172.21.15.190:0/3651573384 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36e0012cb0 con 0x7f36f40a61f0 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 10 monclient: dump: 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:14.773 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:14.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 10 monclient: _finish_auth 0 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 10 monclient: _check_auth_tickets 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 1 -- 172.21.15.190:0/3651573384 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e0017070 con 0x7f36f40a61f0 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 5 monclient: authenticate success, global_id 14770 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient: _renew_subs 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 1 -- 172.21.15.190:0/3651573384 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f406f440 con 0x7f36f40a61f0 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 1 -- 172.21.15.190:0/3651573384 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36e00117f0 con 0x7f36f40a61f0 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f17fa640 10 monclient: dump: 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:14.775 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:14.776 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:14.776 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:14.776 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:14.776 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:14.776 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:14.776 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:14.776 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:14.776 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:14.776 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:14.776 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:14.776 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:14.777 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:14.777 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36f2ffd640 4 set_mon_vals no callback set 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient: _renew_subs 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.757+0000 7f36fae2a640 1 -- 172.21.15.190:0/3651573384 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f413e270 con 0x7f36f40a61f0 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.761+0000 7f36fae2a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.761+0000 7f36fae2a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.761+0000 7f36fae2a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.761+0000 7f36f17fa640 1 -- 172.21.15.190:0/3651573384 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f36e0011380 con 0x7f36f40a61f0 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.761+0000 7f36fae2a640 1 -- 172.21.15.190:0/3651573384 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f40a52f0 con 0x7f36f40a61f0 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.761+0000 7f36f17fa640 1 --2- 172.21.15.190:0/3651573384 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f36d803be60 0x7f36d803e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:14.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.761+0000 7f36f17fa640 1 -- 172.21.15.190:0/3651573384 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 7289+0+0 (secure 0 0 0) 0x7f36e004f120 con 0x7f36f40a61f0 2024-08-19T10:42:14.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.761+0000 7f36f8bc6640 1 --2- 172.21.15.190:0/3651573384 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f36d803be60 0x7f36d803e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:14.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.761+0000 7f36f8bc6640 10 monclient: get_auth_request con 0x7f36d803be60 auth_method 0 2024-08-19T10:42:14.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.761+0000 7f36f8bc6640 1 --2- 172.21.15.190:0/3651573384 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f36d803be60 0x7f36d803e320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f36e80ad000 tx=0x7f36e80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:14.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.765+0000 7f36f17fa640 1 -- 172.21.15.190:0/3651573384 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f36f40a52f0 con 0x7f36f40a61f0 2024-08-19T10:42:14.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.765+0000 7f36f17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:14.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.765+0000 7f36f17fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:14.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.945+0000 7f36fae2a640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:14.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.945+0000 7f36fae2a640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:14.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.945+0000 7f36fae2a640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:14.949 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.945+0000 7f36fae2a640 1 -- 172.21.15.190:0/3651573384 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) -- 0x7f36f40723c0 con 0x7f36f40a61f0 2024-08-19T10:42:14.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.949+0000 7f36f17fa640 1 -- 172.21.15.190:0/3651573384 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=-2 no weight-set for pool 'fooset' v131) ==== 160+0+0 (secure 0 0 0) 0x7f36e004d150 con 0x7f36f40a61f0 2024-08-19T10:42:14.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.949+0000 7f36f17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:14.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:14.949+0000 7f36f17fa640 10 monclient: _finish_command 2 = mon:2 no weight-set for pool 'fooset' 2024-08-19T10:42:14.951 INFO:tasks.workunit.client.0.smithi190.stderr:Error ENOENT: no weight-set for pool 'fooset' 2024-08-19T10:42:14.955 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:42:14.955 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set create fooset flat 2024-08-19T10:42:15.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.073+0000 7f60b1bd4640 10 monclient: get_monmap_and_config 2024-08-19T10:42:15.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.073+0000 7f60b1bd4640 10 monclient: build_initial_monmap 2024-08-19T10:42:15.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.073+0000 7f60b1bd4640 10 monclient: monmap: 2024-08-19T10:42:15.078 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:15.077735+0000 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:15.077735+0000 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:15.079 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:15.080 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:15.080 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:15.080 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:15.080 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:15.080 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.073+0000 7f60b1bd4640 1 Processor -- start 2024-08-19T10:42:15.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.073+0000 7f60b1bd4640 1 -- start start 2024-08-19T10:42:15.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.073+0000 7f60b1bd4640 10 monclient: init 2024-08-19T10:42:15.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:15.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 10 monclient: _add_conns ranks=[7,2,8,0,10,4,1,9,3,6,5] 2024-08-19T10:42:15.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60ac069410 0x7f60ac09dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 10 monclient(hunting): picked mon.noname-i con 0x7f60ac069410 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:15.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60ac0a3c80 0x7f60ac0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 10 monclient(hunting): picked mon.noname-b con 0x7f60ac0a3c80 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f60ac0a45a0 0x7f60ac13ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 10 monclient(hunting): picked mon.noname-e con 0x7f60ac0a45a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f60ac0697f0 con 0x7f60ac0a3c80 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f60ac0a0740 con 0x7f60ac069410 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f60ac09f8a0 con 0x7f60ac0a45a0 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60b1bd4640 10 monclient(hunting): authenticate will time out at 1180.648624s 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60ac069410 0x7f60ac09dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60ac069410 0x7f60ac09dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42282/0 (socket says 172.21.15.190:42282) 2024-08-19T10:42:15.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 1 -- 172.21.15.190:0/2556853529 learned_addr learned my addr 172.21.15.190:0/2556853529 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:15.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 10 monclient(hunting): get_auth_request con 0x7f60ac069410 auth_method 0 2024-08-19T10:42:15.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:15.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:15.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:15.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:15.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:15.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:15.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 10 monclient(hunting): handle_auth_done global_id 14713 payload 274 2024-08-19T10:42:15.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60aaffd640 1 --2- 172.21.15.190:0/2556853529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60ac0a3c80 0x7f60ac0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60abfff640 1 --2- 172.21.15.190:0/2556853529 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f60ac0a45a0 0x7f60ac13ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 1 -- 172.21.15.190:0/2556853529 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f60ac0a45a0 msgr2=0x7f60ac13ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:15.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 1 --2- 172.21.15.190:0/2556853529 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f60ac0a45a0 0x7f60ac13ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 1 -- 172.21.15.190:0/2556853529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60ac0a3c80 msgr2=0x7f60ac0a4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:15.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 1 --2- 172.21.15.190:0/2556853529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60ac0a3c80 0x7f60ac0a4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:15.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 1 monclient: found mon.noname-i 2024-08-19T10:42:15.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:15.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:15.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60ab7fe640 1 -- 172.21.15.190:0/2556853529 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac09fb80 con 0x7f60ac069410 2024-08-19T10:42:15.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60abfff640 10 monclient: get_auth_request con 0x7f60ac0a45a0 auth_method 0 2024-08-19T10:42:15.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60abfff640 1 --2- 172.21.15.190:0/2556853529 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f60ac0a45a0 0x7f60ac13ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:15.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60aaffd640 10 monclient: get_auth_request con 0x7f60ac0a3c80 auth_method 0 2024-08-19T10:42:15.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.077+0000 7f60aaffd640 1 --2- 172.21.15.190:0/2556853529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60ac0a3c80 0x7f60ac0a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:15.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60ab7fe640 1 -- 172.21.15.190:0/2556853529 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60ac069410 msgr2=0x7f60ac09dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:15.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60ab7fe640 1 --2- 172.21.15.190:0/2556853529 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60ac069410 0x7f60ac09dc00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098a0 tx=0x7f609c01c120 comp rx=0 tx=0).ready entity=mon.7 client_cookie=6e4ee34d411fd4a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:15.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 1 -- 172.21.15.190:0/2556853529 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f609c0b1070 con 0x7f60ac069410 2024-08-19T10:42:15.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:15.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:15.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 10 monclient: dump: 2024-08-19T10:42:15.094 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:15.094 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:15.094 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.094 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.094 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:15.094 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.094 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:15.094 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 1 -- 172.21.15.190:0/2556853529 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609c0a4ad0 con 0x7f60ac069410 2024-08-19T10:42:15.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 1 -- 172.21.15.190:0/2556853529 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f609c0a4cc0 con 0x7f60ac069410 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60aa7fc640 10 monclient: dump: 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:15.096 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:15.097 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:15.097 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:15.097 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:15.097 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:15.097 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:15.097 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:15.097 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 5 monclient: authenticate success, global_id 14713 2024-08-19T10:42:15.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:15.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 4 set_mon_vals no callback set 2024-08-19T10:42:15.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 10 monclient: shutdown 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 1 -- 172.21.15.190:0/2556853529 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60ac069410 msgr2=0x7f60ac09dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 1 --2- 172.21.15.190:0/2556853529 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60ac069410 0x7f60ac09dc00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f609c0098a0 tx=0x7f609c01c120 comp rx=0 tx=0).stop 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 1 -- 172.21.15.190:0/2556853529 shutdown_connections 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 1 --2- 172.21.15.190:0/2556853529 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f60ac0a45a0 0x7f60ac13ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 1 --2- 172.21.15.190:0/2556853529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f60ac0a3c80 0x7f60ac0a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 1 --2- 172.21.15.190:0/2556853529 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f60ac069410 0x7f60ac09dc00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 1 -- 172.21.15.190:0/2556853529 >> 172.21.15.190:0/2556853529 conn(0x7f60ac0a8ca0 msgr2=0x7f60ac0a4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 1 -- 172.21.15.190:0/2556853529 shutdown_connections 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 1 -- 172.21.15.190:0/2556853529 wait complete. 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 10 monclient: build_initial_monmap 2024-08-19T10:42:15.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 10 monclient: monmap: 2024-08-19T10:42:15.099 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:15.099 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:15.099 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:15.099 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:15.099 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:15.099 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.099 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:15.099 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:15.099 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 1 Processor -- start 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 1 -- start start 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.081+0000 7f60b1bd4640 10 monclient: init 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient: _add_conns ranks=[4,8,1,6,0,5,3,2,7] 2024-08-19T10:42:15.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60ac069410 0x7f60ac09ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient(hunting): picked mon.noname-c con 0x7f60ac069410 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f60ac0a3c80 0x7f60ac11ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient(hunting): picked mon.noname-e con 0x7f60ac0a3c80 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ac0a45a0 0x7f60ac11f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient(hunting): picked mon.noname-f con 0x7f60ac0a45a0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac13fff0 con 0x7f60ac0a45a0 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f60ac06b6c0 con 0x7f60ac069410 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f60ac140170 con 0x7f60ac0a3c80 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:15.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient(hunting): authenticate will time out at 1180.656327s 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ac0a45a0 0x7f60ac11f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ac0a45a0 0x7f60ac11f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:49970/0 (socket says 172.21.15.190:49970) 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 1 -- 172.21.15.190:0/1404764355 learned_addr learned my addr 172.21.15.190:0/1404764355 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 10 monclient(hunting): get_auth_request con 0x7f60ac0a45a0 auth_method 0 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60ab7fe640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60ac069410 0x7f60ac09ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60aaffd640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f60ac0a3c80 0x7f60ac11ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60ab7fe640 10 monclient(hunting): get_auth_request con 0x7f60ac069410 auth_method 0 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60ab7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60aaffd640 10 monclient(hunting): get_auth_request con 0x7f60ac0a3c80 auth_method 0 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60aaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60aaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 10 monclient(hunting): handle_auth_done global_id 14779 payload 824 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 1 -- 172.21.15.190:0/1404764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f60ac0a3c80 msgr2=0x7f60ac11ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f60ac0a3c80 0x7f60ac11ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 1 -- 172.21.15.190:0/1404764355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60ac069410 msgr2=0x7f60ac09ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:15.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60ac069410 0x7f60ac09ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 1 monclient: found mon.noname-f 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 1 -- 172.21.15.190:0/1404764355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac09d160 con 0x7f60ac0a45a0 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 1 -- 172.21.15.190:0/1404764355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ac0a45a0 msgr2=0x7f60ac11f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60abfff640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ac0a45a0 0x7f60ac11f230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60a400cd80 tx=0x7f60a400a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 1 -- 172.21.15.190:0/1404764355 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60a400fb00 con 0x7f60ac0a45a0 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 10 monclient: dump: 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:15.104 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:15.105 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.105 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.105 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:15.105 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.105 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:15.105 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:15.105 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:15.105 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:15.105 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:15.105 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:15.105 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 1 -- 172.21.15.190:0/1404764355 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a400fd00 con 0x7f60ac0a45a0 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 1 -- 172.21.15.190:0/1404764355 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f60a400e850 con 0x7f60ac0a45a0 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 5 monclient: authenticate success, global_id 14779 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:15.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f6093fff640 10 monclient: dump: 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:15.107 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60a9ffb640 4 set_mon_vals no callback set 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient: _renew_subs 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 1 -- 172.21.15.190:0/1404764355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac1368b0 con 0x7f60ac0a45a0 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient: _renew_subs 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.085+0000 7f60b1bd4640 1 -- 172.21.15.190:0/1404764355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac125bb0 con 0x7f60ac0a45a0 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.089+0000 7f60b1bd4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:15.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.089+0000 7f60b1bd4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:15.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.089+0000 7f60b1bd4640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:15.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.089+0000 7f6093fff640 1 -- 172.21.15.190:0/1404764355 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f60a400e3e0 con 0x7f60ac0a45a0 2024-08-19T10:42:15.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.089+0000 7f60b1bd4640 1 -- 172.21.15.190:0/1404764355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ac0739e0 con 0x7f60ac0a45a0 2024-08-19T10:42:15.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.089+0000 7f6093fff640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f609803be30 0x7f609803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.089+0000 7f6093fff640 1 -- 172.21.15.190:0/1404764355 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 7289+0+0 (secure 0 0 0) 0x7f60a404fab0 con 0x7f60ac0a45a0 2024-08-19T10:42:15.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.089+0000 7f60ab7fe640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f609803be30 0x7f609803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.089+0000 7f60ab7fe640 10 monclient: get_auth_request con 0x7f609803be30 auth_method 0 2024-08-19T10:42:15.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.089+0000 7f60ab7fe640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f609803be30 0x7f609803e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f609c009870 tx=0x7f609c0a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:15.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.093+0000 7f6093fff640 1 -- 172.21.15.190:0/1404764355 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f60ac0739e0 con 0x7f60ac0a45a0 2024-08-19T10:42:15.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.093+0000 7f6093fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:15.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.093+0000 7f6093fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:15.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.277+0000 7f60b1bd4640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-08-19T10:42:15.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.277+0000 7f60b1bd4640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-08-19T10:42:15.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.277+0000 7f60b1bd4640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:15.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.277+0000 7f60b1bd4640 1 -- 172.21.15.190:0/1404764355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"} v 0) -- 0x7f60ac09dc00 con 0x7f60ac0a45a0 2024-08-19T10:42:15.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.605+0000 7f6093fff640 1 -- 172.21.15.190:0/1404764355 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}]=0 v132) ==== 109+0+0 (secure 0 0 0) 0x7f60a401d070 con 0x7f60ac0a45a0 2024-08-19T10:42:15.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.605+0000 7f6093fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-08-19T10:42:15.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.605+0000 7f6093fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:15.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.721+0000 7f60b1bd4640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-08-19T10:42:15.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.721+0000 7f60b1bd4640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-08-19T10:42:15.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.721+0000 7f60b1bd4640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:15.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.721+0000 7f60b1bd4640 1 -- 172.21.15.190:0/1404764355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"} v 0) -- 0x7f60ac0a11f0 con 0x7f60ac0a45a0 2024-08-19T10:42:15.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f6093fff640 1 -- 172.21.15.190:0/1404764355 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}]=0 weight-set for pool 'fooset' already created v132) ==== 153+0+0 (secure 0 0 0) 0x7f60ac0a11f0 con 0x7f60ac0a45a0 2024-08-19T10:42:15.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f6093fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "fooset", "mode": "flat"}] 2024-08-19T10:42:15.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f6093fff640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'fooset' already created 2024-08-19T10:42:15.728 INFO:tasks.workunit.client.0.smithi190.stderr:weight-set for pool 'fooset' already created 2024-08-19T10:42:15.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 -- 172.21.15.190:0/1404764355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f609803be30 msgr2=0x7f609803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:15.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f609803be30 0x7f609803e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f609c009870 tx=0x7f609c0a6070 comp rx=0 tx=0).stop 2024-08-19T10:42:15.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 10 monclient: shutdown 2024-08-19T10:42:15.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:15.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 -- 172.21.15.190:0/1404764355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ac0a45a0 msgr2=0x7f60ac11f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:15.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ac0a45a0 0x7f60ac11f230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60a400cd80 tx=0x7f60a400a650 comp rx=0 tx=0).stop 2024-08-19T10:42:15.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 -- 172.21.15.190:0/1404764355 shutdown_connections 2024-08-19T10:42:15.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f609803be30 0x7f609803e2f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f60ac0a3c80 0x7f60ac11ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60ac0a45a0 0x7f60ac11f230 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 --2- 172.21.15.190:0/1404764355 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f60ac069410 0x7f60ac09ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 -- 172.21.15.190:0/1404764355 >> 172.21.15.190:0/1404764355 conn(0x7f60ac0a8ca0 msgr2=0x7f60ac0a6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:15.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 -- 172.21.15.190:0/1404764355 shutdown_connections 2024-08-19T10:42:15.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.725+0000 7f60b1bd4640 1 -- 172.21.15.190:0/1404764355 wait complete. 2024-08-19T10:42:15.745 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set create barset positional 2024-08-19T10:42:15.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient: get_monmap_and_config 2024-08-19T10:42:15.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient: build_initial_monmap 2024-08-19T10:42:15.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient: monmap: 2024-08-19T10:42:15.867 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:15.867 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:15.867952+0000 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:15.867952+0000 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:15.868 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:15.869 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:15.869 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:15.869 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 1 Processor -- start 2024-08-19T10:42:15.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 1 -- start start 2024-08-19T10:42:15.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient: init 2024-08-19T10:42:15.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:15.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient: _add_conns ranks=[4,7,2,1,6,10,3,9,8,5,0] 2024-08-19T10:42:15.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff51c0a46b0 0x7ff51c0a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient(hunting): picked mon.noname-c con 0x7ff51c0a46b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:15.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a28a0 0x7ff51c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient(hunting): picked mon.noname-i con 0x7ff51c0a28a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:15.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff51c137470 0x7ff51c1378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.871 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient(hunting): picked mon.noname-b con 0x7ff51c137470 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff51c06cae0 con 0x7ff51c137470 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff51c0a5510 con 0x7ff51c0a46b0 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff51c09cc20 con 0x7ff51c0a28a0 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff520c38640 10 monclient(hunting): authenticate will time out at 1181.438033s 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a28a0 0x7ff51c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a28a0 0x7ff51c0a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34422/0 (socket says 172.21.15.190:34422) 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 1 -- 172.21.15.190:0/2406999050 learned_addr learned my addr 172.21.15.190:0/2406999050 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:15.872 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 10 monclient(hunting): get_auth_request con 0x7ff51c0a28a0 auth_method 0 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff51a59c640 1 --2- 172.21.15.190:0/2406999050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff51c0a46b0 0x7ff51c0a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff51ad9d640 1 --2- 172.21.15.190:0/2406999050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff51c137470 0x7ff51c1378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff51a59c640 10 monclient(hunting): get_auth_request con 0x7ff51c0a46b0 auth_method 0 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff51a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff51a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff51a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:15.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff51ad9d640 10 monclient(hunting): get_auth_request con 0x7ff51c137470 auth_method 0 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff51ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff51ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff51ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 10 monclient(hunting): handle_auth_done global_id 14722 payload 274 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 1 -- 172.21.15.190:0/2406999050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff51c0a46b0 msgr2=0x7ff51c0a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 1 --2- 172.21.15.190:0/2406999050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff51c0a46b0 0x7ff51c0a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 1 -- 172.21.15.190:0/2406999050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff51c137470 msgr2=0x7ff51c1378f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 1 --2- 172.21.15.190:0/2406999050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff51c137470 0x7ff51c1378f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 1 monclient: found mon.noname-i 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:15.874 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:15.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.865+0000 7ff519d9b640 1 -- 172.21.15.190:0/2406999050 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c0a2f30 con 0x7ff51c0a28a0 2024-08-19T10:42:15.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff519d9b640 1 -- 172.21.15.190:0/2406999050 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a28a0 msgr2=0x7ff51c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:15.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff519d9b640 1 --2- 172.21.15.190:0/2406999050 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a28a0 0x7ff51c0a4fd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff5100098a0 tx=0x7ff51001c440 comp rx=0 tx=0).ready entity=mon.7 client_cookie=a96e31615d21e5ab server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:15.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 1 -- 172.21.15.190:0/2406999050 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5100b0070 con 0x7ff51c0a28a0 2024-08-19T10:42:15.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:15.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:15.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 10 monclient: dump: 2024-08-19T10:42:15.885 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:15.886 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 10 monclient: _finish_auth 0 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 10 monclient: _check_auth_tickets 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 1 -- 172.21.15.190:0/2406999050 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5100aa810 con 0x7ff51c0a28a0 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 5 monclient: authenticate success, global_id 14722 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 1 -- 172.21.15.190:0/2406999050 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5100a9300 con 0x7ff51c0a28a0 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 4 set_mon_vals no callback set 2024-08-19T10:42:15.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff51959a640 10 monclient: dump: 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:15.888 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 10 monclient: shutdown 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 1 -- 172.21.15.190:0/2406999050 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a28a0 msgr2=0x7ff51c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 1 --2- 172.21.15.190:0/2406999050 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a28a0 0x7ff51c0a4fd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff5100098a0 tx=0x7ff51001c440 comp rx=0 tx=0).stop 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 1 -- 172.21.15.190:0/2406999050 shutdown_connections 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 1 --2- 172.21.15.190:0/2406999050 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff51c137470 0x7ff51c1378f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 1 --2- 172.21.15.190:0/2406999050 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a28a0 0x7ff51c0a4fd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 1 --2- 172.21.15.190:0/2406999050 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff51c0a46b0 0x7ff51c0a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 1 -- 172.21.15.190:0/2406999050 >> 172.21.15.190:0/2406999050 conn(0x7ff51c0a92a0 msgr2=0x7ff51c0a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 1 -- 172.21.15.190:0/2406999050 shutdown_connections 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.869+0000 7ff520c38640 1 -- 172.21.15.190:0/2406999050 wait complete. 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient: build_initial_monmap 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient: monmap: 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:15.890 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 1 Processor -- start 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 1 -- start start 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient: init 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient: _add_conns ranks=[1,7,8,3,5,0,4,6,2] 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff51c0a28a0 0x7ff51c1d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient(hunting): picked mon.noname-f con 0x7ff51c0a28a0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:15.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a46b0 0x7ff51c1d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient(hunting): picked mon.noname-i con 0x7ff51c0a46b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:15.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff51c137470 0x7ff51c1d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient(hunting): picked mon.noname-e con 0x7ff51c137470 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:15.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c09fce0 con 0x7ff51c0a28a0 2024-08-19T10:42:15.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff51c09fb60 con 0x7ff51c0a46b0 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff51c1410f0 con 0x7ff51c137470 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff520c38640 10 monclient(hunting): authenticate will time out at 1181.445249s 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a46b0 0x7ff51c1d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a46b0 0x7ff51c1d15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34434/0 (socket says 172.21.15.190:34434) 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 1 -- 172.21.15.190:0/257514188 learned_addr learned my addr 172.21.15.190:0/257514188 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 10 monclient(hunting): get_auth_request con 0x7ff51c0a46b0 auth_method 0 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:15.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51a59c640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff51c0a28a0 0x7ff51c1d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51a59c640 10 monclient(hunting): get_auth_request con 0x7ff51c0a28a0 auth_method 0 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51ad9d640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff51c137470 0x7ff51c1d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51ad9d640 10 monclient(hunting): get_auth_request con 0x7ff51c137470 auth_method 0 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:15.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff51ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 10 monclient(hunting): handle_auth_done global_id 14731 payload 824 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 1 -- 172.21.15.190:0/257514188 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff51c137470 msgr2=0x7ff51c1d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff51c137470 0x7ff51c1d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 1 -- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff51c0a28a0 msgr2=0x7ff51c1d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff51c0a28a0 0x7ff51c1d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 1 monclient: found mon.noname-i 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.873+0000 7ff519d9b640 1 -- 172.21.15.190:0/257514188 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c123ea0 con 0x7ff51c0a46b0 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff519d9b640 1 -- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a46b0 msgr2=0x7ff51c1d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff519d9b640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a46b0 0x7ff51c1d15e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff5100097c0 tx=0x7ff5100191e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:15.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 1 -- 172.21.15.190:0/257514188 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5100b9070 con 0x7ff51c0a46b0 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 10 monclient: dump: 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:15.897 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 10 monclient: _finish_auth 0 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 1 -- 172.21.15.190:0/257514188 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5100044b0 con 0x7ff51c0a46b0 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff520c38640 5 monclient: authenticate success, global_id 14731 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff520c38640 10 monclient: _renew_subs 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff520c38640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:15.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff520c38640 1 -- 172.21.15.190:0/257514188 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c13d910 con 0x7ff51c0a46b0 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 1 -- 172.21.15.190:0/257514188 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5100b0040 con 0x7ff51c0a46b0 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 10 monclient: dump: 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:15.899 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff518d99640 4 set_mon_vals no callback set 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff520c38640 10 monclient: _renew_subs 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff520c38640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff520c38640 1 -- 172.21.15.190:0/257514188 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c13c8d0 con 0x7ff51c0a46b0 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 1 -- 172.21.15.190:0/257514188 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff5100aac90 con 0x7ff51c0a46b0 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff520c38640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:15.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff520c38640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:15.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff520c38640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:15.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff520c38640 1 -- 172.21.15.190:0/257514188 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51c09e330 con 0x7ff51c0a46b0 2024-08-19T10:42:15.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4fc03be00 0x7ff4fc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:15.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff50affd640 1 -- 172.21.15.190:0/257514188 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(132..132 src has 1..132) ==== 7621+0+0 (secure 0 0 0) 0x7ff5100e87d0 con 0x7ff51c0a46b0 2024-08-19T10:42:15.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff51a59c640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4fc03be00 0x7ff4fc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:15.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.877+0000 7ff51a59c640 10 monclient: get_auth_request con 0x7ff4fc03be00 auth_method 0 2024-08-19T10:42:15.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.881+0000 7ff51a59c640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4fc03be00 0x7ff4fc03e2c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff50c007a60 tx=0x7ff50c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:15.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.881+0000 7ff50affd640 1 -- 172.21.15.190:0/257514188 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff510016150 con 0x7ff51c0a46b0 2024-08-19T10:42:15.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.881+0000 7ff50affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:15.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:15.881+0000 7ff50affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:16.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.057+0000 7ff520c38640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-08-19T10:42:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.057+0000 7ff520c38640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-08-19T10:42:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.061+0000 7ff520c38640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.061+0000 7ff520c38640 1 -- 172.21.15.190:0/257514188 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"} v 0) -- 0x7ff51c1378f0 con 0x7ff51c0a46b0 2024-08-19T10:42:16.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.636+0000 7ff50affd640 1 -- 172.21.15.190:0/257514188 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}]=0 v133) ==== 115+0+0 (secure 0 0 0) 0x7ff5100aa7e0 con 0x7ff51c0a46b0 2024-08-19T10:42:16.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.636+0000 7ff50affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-08-19T10:42:16.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.636+0000 7ff50affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:16.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.752+0000 7ff520c38640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-08-19T10:42:16.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.752+0000 7ff520c38640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-08-19T10:42:16.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.752+0000 7ff520c38640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:16.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.752+0000 7ff520c38640 1 -- 172.21.15.190:0/257514188 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"} v 0) -- 0x7ff51c13cf20 con 0x7ff51c0a46b0 2024-08-19T10:42:16.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.752+0000 7ff50affd640 1 -- 172.21.15.190:0/257514188 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}]=0 weight-set for pool 'barset' already created v133) ==== 159+0+0 (secure 0 0 0) 0x7ff5100b33c0 con 0x7ff51c0a46b0 2024-08-19T10:42:16.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.752+0000 7ff50affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "barset", "mode": "positional"}] 2024-08-19T10:42:16.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.752+0000 7ff50affd640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'barset' already created 2024-08-19T10:42:16.758 INFO:tasks.workunit.client.0.smithi190.stderr:weight-set for pool 'barset' already created 2024-08-19T10:42:16.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 -- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4fc03be00 msgr2=0x7ff4fc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:16.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4fc03be00 0x7ff4fc03e2c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff50c007a60 tx=0x7ff50c008040 comp rx=0 tx=0).stop 2024-08-19T10:42:16.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 10 monclient: shutdown 2024-08-19T10:42:16.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:16.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 -- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a46b0 msgr2=0x7ff51c1d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:16.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a46b0 0x7ff51c1d15e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff5100097c0 tx=0x7ff5100191e0 comp rx=0 tx=0).stop 2024-08-19T10:42:16.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 -- 172.21.15.190:0/257514188 shutdown_connections 2024-08-19T10:42:16.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4fc03be00 0x7ff4fc03e2c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:16.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff51c137470 0x7ff51c1d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:16.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff51c0a46b0 0x7ff51c1d15e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:16.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 --2- 172.21.15.190:0/257514188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff51c0a28a0 0x7ff51c1d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:16.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 -- 172.21.15.190:0/257514188 >> 172.21.15.190:0/257514188 conn(0x7ff51c0a92a0 msgr2=0x7ff51c0a4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:16.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 -- 172.21.15.190:0/257514188 shutdown_connections 2024-08-19T10:42:16.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.756+0000 7ff520c38640 1 -- 172.21.15.190:0/257514188 wait complete. 2024-08-19T10:42:16.776 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set ls 2024-08-19T10:42:16.776 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep fooset 2024-08-19T10:42:16.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.888+0000 7f802a178640 10 monclient: get_monmap_and_config 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.888+0000 7f802a178640 10 monclient: build_initial_monmap 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.888+0000 7f802a178640 10 monclient: monmap: 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:16.894498+0000 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:16.894498+0000 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:16.894 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:16.895 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:16.895 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:16.895 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:16.895 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:16.895 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:16.895 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:16.895 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:16.895 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:16.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.888+0000 7f802a178640 1 Processor -- start 2024-08-19T10:42:16.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.888+0000 7f802a178640 1 -- start start 2024-08-19T10:42:16.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.888+0000 7f802a178640 10 monclient: init 2024-08-19T10:42:16.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 10 monclient: _add_conns ranks=[0,1,5,6,10,2,8,7,3,9,4] 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f80240a56c0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f80240722c0 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f80240a61b0 0x7f80240a1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 10 monclient(hunting): picked mon.noname-h con 0x7f80240a61b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8024072740 con 0x7f80240a56c0 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80240737e0 con 0x7f80240722c0 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f8024074080 con 0x7f80240a61b0 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f802a178640 10 monclient(hunting): authenticate will time out at 1182.464624s 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8022ffd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f80240722c0 0x7f802413c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:53482/0 (socket says 172.21.15.190:53482) 2024-08-19T10:42:16.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8022ffd640 1 -- 172.21.15.190:0/538808377 learned_addr learned my addr 172.21.15.190:0/538808377 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 1 --2- 172.21.15.190:0/538808377 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f80240a61b0 0x7f80240a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 10 monclient(hunting): get_auth_request con 0x7f80240a61b0 auth_method 0 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 10 monclient(hunting): handle_auth_done global_id 5504 payload 274 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 1 -- 172.21.15.190:0/538808377 >> v1:172.21.15.190:6789/0 conn(0x7f80240722c0 legacy=0x7f802413c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 1 -- 172.21.15.190:0/538808377 >> v1:172.21.15.89:6789/0 conn(0x7f80240a56c0 legacy=0x7f80240a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 1 monclient: found mon.noname-h 2024-08-19T10:42:16.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:16.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:16.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.892+0000 7f8023fff640 1 -- 172.21.15.190:0/538808377 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80240a71e0 con 0x7f80240a61b0 2024-08-19T10:42:16.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f8023fff640 1 -- 172.21.15.190:0/538808377 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f80240a61b0 msgr2=0x7f80240a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:16.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f8023fff640 1 --2- 172.21.15.190:0/538808377 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f80240a61b0 0x7f80240a1730 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f800c0097f0 tx=0x7f800c01c3a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=3ce052f4e34c1b90 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:16.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 1 -- 172.21.15.190:0/538808377 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f800c0b0070 con 0x7f80240a61b0 2024-08-19T10:42:16.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:16.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 10 monclient: dump: 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:16.902 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 1 -- 172.21.15.190:0/538808377 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c0a9a60 con 0x7f80240a61b0 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 1 -- 172.21.15.190:0/538808377 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f800c0a9c50 con 0x7f80240a61b0 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 5 monclient: authenticate success, global_id 5504 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 4 set_mon_vals no callback set 2024-08-19T10:42:16.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80227fc640 10 monclient: dump: 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:16.904 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient: shutdown 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 -- 172.21.15.190:0/538808377 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f80240a61b0 msgr2=0x7f80240a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 --2- 172.21.15.190:0/538808377 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f80240a61b0 0x7f80240a1730 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f800c0097f0 tx=0x7f800c01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 -- 172.21.15.190:0/538808377 shutdown_connections 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 --2- 172.21.15.190:0/538808377 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f80240a61b0 0x7f80240a1730 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 -- 172.21.15.190:0/538808377 >> 172.21.15.190:0/538808377 conn(0x7f80240aa2b0 msgr2=0x7f80240a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 -- 172.21.15.190:0/538808377 shutdown_connections 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 -- 172.21.15.190:0/538808377 wait complete. 2024-08-19T10:42:16.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient: build_initial_monmap 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient: monmap: 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:16.906 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:16.907 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:16.907 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:16.907 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:16.907 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:16.907 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:16.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 Processor -- start 2024-08-19T10:42:16.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 -- start start 2024-08-19T10:42:16.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient: init 2024-08-19T10:42:16.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:16.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient: _add_conns ranks=[3,8,0,6,5,1,4,2,7] 2024-08-19T10:42:16.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80240a56c0 0x7f80241d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:16.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient(hunting): picked mon.noname-g con 0x7f80240a56c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:16.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f80240a61b0 0x7f80241d1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:16.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient(hunting): picked mon.noname-e con 0x7f80240a61b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:16.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802413ee20 0x7f80241d1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:16.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient(hunting): picked mon.noname-a con 0x7f802413ee20 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:16.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:16.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8024142990 con 0x7f802413ee20 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8024142810 con 0x7f80240a56c0 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f8024142b10 con 0x7f80240a61b0 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f802a178640 10 monclient(hunting): authenticate will time out at 1182.471857s 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80237fe640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80240a56c0 msgr2=0x7f80241d0cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.896+0000 7f80237fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80240a56c0 0x7f80241d0cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802413ee20 0x7f80241d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8022ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f80240a61b0 0x7f80241d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802413ee20 0x7f80241d1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:39076/0 (socket says 172.21.15.190:39076) 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 1 -- 172.21.15.190:0/670489826 learned_addr learned my addr 172.21.15.190:0/670489826 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:16.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 10 monclient(hunting): get_auth_request con 0x7f802413ee20 auth_method 0 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8022ffd640 10 monclient(hunting): get_auth_request con 0x7f80240a61b0 auth_method 0 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8022ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8022ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8022ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8022ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8022ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:16.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8022ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 10 monclient(hunting): handle_auth_done global_id 5508 payload 824 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 1 -- 172.21.15.190:0/670489826 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f80240a61b0 msgr2=0x7f80241d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f80240a61b0 0x7f80241d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 1 -- 172.21.15.190:0/670489826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80240a56c0 msgr2=0x7f80241d0cc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80240a56c0 0x7f80241d0cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 1 monclient: found mon.noname-a 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 1 -- 172.21.15.190:0/670489826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802413fb30 con 0x7f802413ee20 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 1 -- 172.21.15.190:0/670489826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802413ee20 msgr2=0x7f80241d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8023fff640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802413ee20 0x7f80241d1740 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f800c01c880 tx=0x7f800c019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:16.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 1 -- 172.21.15.190:0/670489826 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f800c0b0040 con 0x7f802413ee20 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 10 monclient: dump: 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:16.917 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 10 monclient: _finish_auth 0 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 1 -- 172.21.15.190:0/670489826 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c004390 con 0x7f802413ee20 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f802a178640 5 monclient: authenticate success, global_id 5508 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f802a178640 10 monclient: _renew_subs 2024-08-19T10:42:16.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f802a178640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f802a178640 1 -- 172.21.15.190:0/670489826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802413e6c0 con 0x7f802413ee20 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 1 -- 172.21.15.190:0/670489826 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f800c003a90 con 0x7f802413ee20 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f8021ffb640 4 set_mon_vals no callback set 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f801bfff640 10 monclient: dump: 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:16.919 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f802a178640 10 monclient: _renew_subs 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f802a178640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:16.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.900+0000 7f802a178640 1 -- 172.21.15.190:0/670489826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802413e3e0 con 0x7f802413ee20 2024-08-19T10:42:16.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.904+0000 7f802a178640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:16.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.904+0000 7f802a178640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:16.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.904+0000 7f802a178640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:16.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.904+0000 7f802a178640 1 -- 172.21.15.190:0/670489826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80240a17b0 con 0x7f802413ee20 2024-08-19T10:42:16.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.904+0000 7f801bfff640 1 -- 172.21.15.190:0/670489826 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f800c003c30 con 0x7f802413ee20 2024-08-19T10:42:16.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.904+0000 7f801bfff640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f800403be00 0x7f800403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:16.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.904+0000 7f801bfff640 1 -- 172.21.15.190:0/670489826 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 8233+0+0 (secure 0 0 0) 0x7f800c0ea3b0 con 0x7f802413ee20 2024-08-19T10:42:16.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.904+0000 7f80237fe640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f800403be00 0x7f800403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:16.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.904+0000 7f80237fe640 10 monclient: get_auth_request con 0x7f800403be00 auth_method 0 2024-08-19T10:42:16.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.908+0000 7f80237fe640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f800403be00 0x7f800403e2c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8014007fd0 tx=0x7f8014007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:16.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.916+0000 7f801bfff640 1 -- 172.21.15.190:0/670489826 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f802413fb30 con 0x7f802413ee20 2024-08-19T10:42:16.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.916+0000 7f801bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:16.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:16.916+0000 7f801bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:17.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.092+0000 7f802a178640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.092+0000 7f802a178640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.092+0000 7f802a178640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:17.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.092+0000 7f802a178640 1 -- 172.21.15.190:0/670489826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f802413cec0 con 0x7f802413ee20 2024-08-19T10:42:17.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.092+0000 7f801bfff640 1 -- 172.21.15.190:0/670489826 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v133) ==== 71+0+14 (secure 0 0 0) 0x7f800c0160b0 con 0x7f802413ee20 2024-08-19T10:42:17.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.092+0000 7f801bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.092+0000 7f801bfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:17.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.200+0000 7f802a178640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.200+0000 7f802a178640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.200+0000 7f802a178640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:17.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.200+0000 7f802a178640 1 -- 172.21.15.190:0/670489826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f802406fca0 con 0x7f802413ee20 2024-08-19T10:42:17.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.200+0000 7f801bfff640 1 -- 172.21.15.190:0/670489826 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v133) ==== 71+0+14 (secure 0 0 0) 0x7f800c0aad90 con 0x7f802413ee20 2024-08-19T10:42:17.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.200+0000 7f801bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.200+0000 7f801bfff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:17.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 -- 172.21.15.190:0/670489826 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f800403be00 msgr2=0x7f800403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:17.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f800403be00 0x7f800403e2c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8014007fd0 tx=0x7f8014007de0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 10 monclient: shutdown 2024-08-19T10:42:17.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:17.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 -- 172.21.15.190:0/670489826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802413ee20 msgr2=0x7f80241d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:17.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802413ee20 0x7f80241d1740 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f800c01c880 tx=0x7f800c019140 comp rx=0 tx=0).stop 2024-08-19T10:42:17.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 -- 172.21.15.190:0/670489826 shutdown_connections 2024-08-19T10:42:17.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802413ee20 0x7f80241d1740 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f800403be00 0x7f800403e2c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f80240a61b0 0x7f80241d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 --2- 172.21.15.190:0/670489826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80240a56c0 0x7f80241d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 -- 172.21.15.190:0/670489826 >> 172.21.15.190:0/670489826 conn(0x7f80240aa2b0 msgr2=0x7f80240a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:17.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 -- 172.21.15.190:0/670489826 shutdown_connections 2024-08-19T10:42:17.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.204+0000 7f802a178640 1 -- 172.21.15.190:0/670489826 wait complete. 2024-08-19T10:42:17.222 INFO:tasks.workunit.client.0.smithi190.stdout:fooset 2024-08-19T10:42:17.223 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep barset 2024-08-19T10:42:17.223 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set ls 2024-08-19T10:42:17.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.336+0000 7fefdc1dc640 10 monclient: get_monmap_and_config 2024-08-19T10:42:17.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.336+0000 7fefdc1dc640 10 monclient: build_initial_monmap 2024-08-19T10:42:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.336+0000 7fefdc1dc640 10 monclient: monmap: 2024-08-19T10:42:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:17.341385+0000 2024-08-19T10:42:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:17.341385+0000 2024-08-19T10:42:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:17.342 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:17.342 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:17.342 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:17.342 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:17.342 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:17.342 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:17.342 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:17.342 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:17.342 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:17.342 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:17.343 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.336+0000 7fefdc1dc640 1 Processor -- start 2024-08-19T10:42:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.336+0000 7fefdc1dc640 1 -- start start 2024-08-19T10:42:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.336+0000 7fefdc1dc640 10 monclient: init 2024-08-19T10:42:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.336+0000 7fefdc1dc640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.336+0000 7fefdc1dc640 10 monclient: _add_conns ranks=[10,3,5,6,1,8,7,0,4,9,2] 2024-08-19T10:42:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fefd406c8d0 0x7fefd409d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 10 monclient(hunting): picked mon.noname-f con 0x7fefd406c8d0 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd406ccb0 0x7fefd406d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 10 monclient(hunting): picked mon.noname-g con 0x7fefd406ccb0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fefd4137580 0x7fefd41379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 10 monclient(hunting): picked mon.noname-h con 0x7fefd4137580 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fefd40a56c0 con 0x7fefd406c8d0 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fefd40a5070 con 0x7fefd406ccb0 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fefd40a5350 con 0x7fefd4137580 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefdc1dc640 10 monclient(hunting): authenticate will time out at 1182.912097s 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9777640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd406ccb0 msgr2=0x7fefd406d090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9777640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd406ccb0 0x7fefd406d090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fefd406c8d0 0x7fefd409d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefda779640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fefd4137580 0x7fefd41379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fefd406c8d0 0x7fefd409d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39000/0 (socket says 172.21.15.190:39000) 2024-08-19T10:42:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 1 -- 172.21.15.190:0/162154192 learned_addr learned my addr 172.21.15.190:0/162154192 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefda779640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fefd4137580 0x7fefd41379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36910/0 (socket says 172.21.15.190:36910) 2024-08-19T10:42:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 10 monclient(hunting): get_auth_request con 0x7fefd406c8d0 auth_method 0 2024-08-19T10:42:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefda779640 10 monclient(hunting): get_auth_request con 0x7fefd4137580 auth_method 0 2024-08-19T10:42:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefda779640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefda779640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefda779640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefda779640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefda779640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefda779640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 10 monclient(hunting): handle_auth_done global_id 14797 payload 274 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 1 -- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fefd4137580 msgr2=0x7fefd41379f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 1 --2- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fefd4137580 0x7fefd41379f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 1 -- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd406ccb0 msgr2=0x7fefd406d090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 1 --2- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd406ccb0 0x7fefd406d090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 10 monclient: _finish_hunting 0 2024-08-19T10:42:17.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 1 monclient: found mon.noname-f 2024-08-19T10:42:17.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:17.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:17.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefd9f78640 1 -- 172.21.15.190:0/162154192 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd40a28b0 con 0x7fefd406c8d0 2024-08-19T10:42:17.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.340+0000 7fefda779640 1 --2- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fefd4137580 0x7fefd41379f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:17.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd9f78640 1 -- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd406c8d0 msgr2=0x7fefd409d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:17.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd9f78640 1 --2- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd406c8d0 0x7fefd409d470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009870 tx=0x7fefd001c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5261772a30284b3d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:17.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 1 -- 172.21.15.190:0/162154192 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fefd00b0070 con 0x7fefd406c8d0 2024-08-19T10:42:17.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:17.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 10 monclient: dump: 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:17.354 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 10 monclient: _finish_auth 0 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 10 monclient: _check_auth_tickets 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 1 -- 172.21.15.190:0/162154192 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd00aa810 con 0x7fefd406c8d0 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 5 monclient: authenticate success, global_id 14797 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:17.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 4 set_mon_vals no callback set 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 1 -- 172.21.15.190:0/162154192 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fefd00a9300 con 0x7fefd406c8d0 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd8f76640 10 monclient: dump: 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient: shutdown 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 -- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd406c8d0 msgr2=0x7fefd409d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 --2- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd406c8d0 0x7fefd409d470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009870 tx=0x7fefd001c440 comp rx=0 tx=0).stop 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 -- 172.21.15.190:0/162154192 shutdown_connections 2024-08-19T10:42:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 --2- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fefd4137580 0x7fefd41379f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 --2- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefd406ccb0 0x7fefd406d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 --2- 172.21.15.190:0/162154192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd406c8d0 0x7fefd409d470 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 -- 172.21.15.190:0/162154192 >> 172.21.15.190:0/162154192 conn(0x7fefd406dde0 msgr2=0x7fefd406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 -- 172.21.15.190:0/162154192 shutdown_connections 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 -- 172.21.15.190:0/162154192 wait complete. 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient: build_initial_monmap 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient: monmap: 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 Processor -- start 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 -- start start 2024-08-19T10:42:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient: init 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient: _add_conns ranks=[2,4,1,0,5,7,6,8,3] 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd406c8d0 0x7fefd413c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient(hunting): picked mon.noname-b con 0x7fefd406c8d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd406ccb0 0x7fefd4136820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient(hunting): picked mon.noname-c con 0x7fefd406ccb0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd4137580 0x7fefd4136d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient(hunting): picked mon.noname-f con 0x7fefd4137580 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fefd406d5d0 con 0x7fefd4137580 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fefd409d9b0 con 0x7fefd406c8d0 2024-08-19T10:42:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fefd406e5f0 con 0x7fefd406ccb0 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefdc1dc640 10 monclient(hunting): authenticate will time out at 1182.919426s 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd4137580 0x7fefd4136d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd4137580 0x7fefd4136d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39002/0 (socket says 172.21.15.190:39002) 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 1 -- 172.21.15.190:0/2668906181 learned_addr learned my addr 172.21.15.190:0/2668906181 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 10 monclient(hunting): get_auth_request con 0x7fefd4137580 auth_method 0 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 10 monclient(hunting): handle_auth_done global_id 14806 payload 824 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd9f78640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd406c8d0 0x7fefd413c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 1 -- 172.21.15.190:0/2668906181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd406ccb0 msgr2=0x7fefd4136820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd9777640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd406ccb0 0x7fefd4136820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd406ccb0 0x7fefd4136820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 1 -- 172.21.15.190:0/2668906181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd406c8d0 msgr2=0x7fefd413c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd406c8d0 0x7fefd413c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 10 monclient: _finish_hunting 0 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 1 monclient: found mon.noname-f 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefda779640 1 -- 172.21.15.190:0/2668906181 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd406e1c0 con 0x7fefd4137580 2024-08-19T10:42:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd9f78640 10 monclient: get_auth_request con 0x7fefd406c8d0 auth_method 0 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.344+0000 7fefd9f78640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd406c8d0 0x7fefd413c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefda779640 1 -- 172.21.15.190:0/2668906181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd4137580 msgr2=0x7fefd4136d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefda779640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd4137580 0x7fefd4136d60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00efc0 tx=0x7fefcc00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 1 -- 172.21.15.190:0/2668906181 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fefcc018030 con 0x7fefd4137580 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 10 monclient: dump: 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 1 -- 172.21.15.190:0/2668906181 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefcc0148f0 con 0x7fefd4137580 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 1 -- 172.21.15.190:0/2668906181 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fefcc0133e0 con 0x7fefd4137580 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefdc1dc640 5 monclient: authenticate success, global_id 14806 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefcbfff640 4 set_mon_vals no callback set 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 10 monclient: dump: 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefdc1dc640 10 monclient: _renew_subs 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefdc1dc640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefdc1dc640 1 -- 172.21.15.190:0/2668906181 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd413d130 con 0x7fefd4137580 2024-08-19T10:42:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefdc1dc640 10 monclient: _renew_subs 2024-08-19T10:42:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefdc1dc640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefdc1dc640 1 -- 172.21.15.190:0/2668906181 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd41241e0 con 0x7fefd4137580 2024-08-19T10:42:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefdc1dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefdc1dc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefdc1dc640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefdc1dc640 1 -- 172.21.15.190:0/2668906181 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd406d090 con 0x7fefd4137580 2024-08-19T10:42:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 1 -- 172.21.15.190:0/2668906181 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fefcc01b070 con 0x7fefd4137580 2024-08-19T10:42:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefb803be00 0x7fefb803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefca7fc640 1 -- 172.21.15.190:0/2668906181 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 8233+0+0 (secure 0 0 0) 0x7fefcc019070 con 0x7fefd4137580 2024-08-19T10:42:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefd9f78640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefb803be00 0x7fefb803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.348+0000 7fefd9f78640 10 monclient: get_auth_request con 0x7fefb803be00 auth_method 0 2024-08-19T10:42:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.352+0000 7fefd9f78640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefb803be00 0x7fefb803e2c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fefd40a00e0 tx=0x7fefd00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.356+0000 7fefca7fc640 1 -- 172.21.15.190:0/2668906181 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fefcc010040 con 0x7fefd4137580 2024-08-19T10:42:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.356+0000 7fefca7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.356+0000 7fefca7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:17.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.524+0000 7fefdc1dc640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.524+0000 7fefdc1dc640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.524+0000 7fefdc1dc640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.524+0000 7fefdc1dc640 1 -- 172.21.15.190:0/2668906181 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7fefd41239a0 con 0x7fefd4137580 2024-08-19T10:42:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.528+0000 7fefca7fc640 1 -- 172.21.15.190:0/2668906181 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v133) ==== 71+0+14 (secure 0 0 0) 0x7fefcc04daa0 con 0x7fefd4137580 2024-08-19T10:42:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.528+0000 7fefca7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.528+0000 7fefca7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:17.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.624+0000 7fefdc1dc640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.624+0000 7fefdc1dc640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.624+0000 7fefdc1dc640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:17.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.624+0000 7fefdc1dc640 1 -- 172.21.15.190:0/2668906181 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7fefd40008e0 con 0x7fefd4137580 2024-08-19T10:42:17.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.624+0000 7fefca7fc640 1 -- 172.21.15.190:0/2668906181 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v133) ==== 71+0+14 (secure 0 0 0) 0x7fefcc014aa0 con 0x7fefd4137580 2024-08-19T10:42:17.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.624+0000 7fefca7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:17.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.624+0000 7fefca7fc640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:17.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 -- 172.21.15.190:0/2668906181 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefb803be00 msgr2=0x7fefb803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:17.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefb803be00 0x7fefb803e2c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fefd40a00e0 tx=0x7fefd00a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:17.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 10 monclient: shutdown 2024-08-19T10:42:17.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:17.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 -- 172.21.15.190:0/2668906181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd4137580 msgr2=0x7fefd4136d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:17.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd4137580 0x7fefd4136d60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00efc0 tx=0x7fefcc00c490 comp rx=0 tx=0).stop 2024-08-19T10:42:17.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 -- 172.21.15.190:0/2668906181 shutdown_connections 2024-08-19T10:42:17.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefb803be00 0x7fefb803e2c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefd4137580 0x7fefd4136d60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fefd406ccb0 0x7fefd4136820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 --2- 172.21.15.190:0/2668906181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fefd406c8d0 0x7fefd413c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 -- 172.21.15.190:0/2668906181 >> 172.21.15.190:0/2668906181 conn(0x7fefd406dde0 msgr2=0x7fefd409e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:17.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 -- 172.21.15.190:0/2668906181 shutdown_connections 2024-08-19T10:42:17.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.628+0000 7fefdc1dc640 1 -- 172.21.15.190:0/2668906181 wait complete. 2024-08-19T10:42:17.646 INFO:tasks.workunit.client.0.smithi190.stdout:barset 2024-08-19T10:42:17.646 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set dump 2024-08-19T10:42:17.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.756+0000 7f79531a5640 10 monclient: get_monmap_and_config 2024-08-19T10:42:17.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.756+0000 7f79531a5640 10 monclient: build_initial_monmap 2024-08-19T10:42:17.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.756+0000 7f79531a5640 10 monclient: monmap: 2024-08-19T10:42:17.762 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:17.762 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:17.762 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:17.762603+0000 2024-08-19T10:42:17.762 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:17.762603+0000 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:17.763 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 1 Processor -- start 2024-08-19T10:42:17.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 1 -- start start 2024-08-19T10:42:17.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 10 monclient: init 2024-08-19T10:42:17.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:17.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 10 monclient: _add_conns ranks=[9,1,3,7,6,0,5,10,2,4,8] 2024-08-19T10:42:17.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f794c069410 0x7f794c0a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 10 monclient(hunting): picked mon.noname-a con 0x7f794c069410 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:17.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f794c0a41c0 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:17.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c0a45a0 0x7f794c13e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 10 monclient(hunting): picked mon.noname-g con 0x7f794c0a45a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:17.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f794c074a20 con 0x7f794c0a41c0 2024-08-19T10:42:17.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:17.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f794c09e520 con 0x7f794c069410 2024-08-19T10:42:17.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:17.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f794c0697f0 con 0x7f794c0a45a0 2024-08-19T10:42:17.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:17.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f79531a5640 10 monclient(hunting): authenticate will time out at 1183.332849s 2024-08-19T10:42:17.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7951742640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c0a45a0 msgr2=0x7f794c13e190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:17.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7951742640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c0a45a0 0x7f794c13e190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:17.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f794bfff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f794c0a41c0 0x7f794c13aa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:53486/0 (socket says 172.21.15.190:53486) 2024-08-19T10:42:17.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f794bfff640 1 -- 172.21.15.190:0/292540232 learned_addr learned my addr 172.21.15.190:0/292540232 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:17.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 1 --2- 172.21.15.190:0/292540232 >> v2:172.21.15.89:3300/0 conn(0x7f794c069410 0x7f794c0a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:17.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 10 monclient(hunting): get_auth_request con 0x7f794c069410 auth_method 0 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 10 monclient(hunting): handle_auth_done global_id 5517 payload 274 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 1 -- 172.21.15.190:0/292540232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c0a45a0 msgr2=0x7f794c13e190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 1 --2- 172.21.15.190:0/292540232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c0a45a0 0x7f794c13e190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 1 -- 172.21.15.190:0/292540232 >> v1:172.21.15.190:6789/0 conn(0x7f794c0a41c0 legacy=0x7f794c13aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 10 monclient: _finish_hunting 0 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 1 monclient: found mon.noname-a 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:17.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:17.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.760+0000 7f7950f41640 1 -- 172.21.15.190:0/292540232 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c0a0120 con 0x7f794c069410 2024-08-19T10:42:17.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f7950f41640 1 -- 172.21.15.190:0/292540232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c069410 msgr2=0x7f794c0a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:17.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f7950f41640 1 --2- 172.21.15.190:0/292540232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c069410 0x7f794c0a3c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f79400097f0 tx=0x7f794001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79818bb2e9f0bf9f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:17.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 1 -- 172.21.15.190:0/292540232 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79400b2070 con 0x7f794c069410 2024-08-19T10:42:17.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:17.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:17.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 10 monclient: dump: 2024-08-19T10:42:17.771 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:17.772 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:17.773 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:17.773 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:17.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:17.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:17.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 1 -- 172.21.15.190:0/292540232 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79400ab7f0 con 0x7f794c069410 2024-08-19T10:42:17.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:17.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 1 -- 172.21.15.190:0/292540232 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79400abe10 con 0x7f794c069410 2024-08-19T10:42:17.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f79531a5640 5 monclient: authenticate success, global_id 5517 2024-08-19T10:42:17.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f79531a5640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:17.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f79531a5640 4 set_mon_vals no callback set 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.764+0000 7f794b7fe640 10 monclient: dump: 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:17.774 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient: shutdown 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 -- 172.21.15.190:0/292540232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c069410 msgr2=0x7f794c0a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 --2- 172.21.15.190:0/292540232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c069410 0x7f794c0a3c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f79400097f0 tx=0x7f794001c3e0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 -- 172.21.15.190:0/292540232 shutdown_connections 2024-08-19T10:42:17.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 --2- 172.21.15.190:0/292540232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c0a45a0 0x7f794c13e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 --2- 172.21.15.190:0/292540232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c069410 0x7f794c0a3c80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 -- 172.21.15.190:0/292540232 >> 172.21.15.190:0/292540232 conn(0x7f794c0a4d70 msgr2=0x7f794c0a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:17.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 -- 172.21.15.190:0/292540232 shutdown_connections 2024-08-19T10:42:17.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 -- 172.21.15.190:0/292540232 wait complete. 2024-08-19T10:42:17.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient: build_initial_monmap 2024-08-19T10:42:17.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient: monmap: 2024-08-19T10:42:17.780 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:17.780 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:17.780 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:17.780 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:17.780 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:17.780 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.780 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:17.780 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:17.780 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 Processor -- start 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 -- start start 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient: init 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient: _add_conns ranks=[8,6,5,0,4,1,7,3,2] 2024-08-19T10:42:17.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f794c069410 0x7f794c135250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient(hunting): picked mon.noname-e con 0x7f794c069410 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f794c0a41c0 0x7f794c135790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient(hunting): picked mon.noname-d con 0x7f794c0a41c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f794c0a45a0 0x7f794c13b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient(hunting): picked mon.noname-h con 0x7f794c0a45a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f794c0a2590 con 0x7f794c0a45a0 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f794c0a2410 con 0x7f794c0a41c0 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f794c09e110 con 0x7f794c069410 2024-08-19T10:42:17.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f79531a5640 10 monclient(hunting): authenticate will time out at 1183.342349s 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f794c0a45a0 0x7f794c13b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f794c0a45a0 0x7f794c13b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36914/0 (socket says 172.21.15.190:36914) 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 1 -- 172.21.15.190:0/928787927 learned_addr learned my addr 172.21.15.190:0/928787927 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 10 monclient(hunting): get_auth_request con 0x7f794c0a45a0 auth_method 0 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:17.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f794bfff640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f794c0a41c0 0x7f794c135790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7950f41640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f794c069410 0x7f794c135250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7950f41640 10 monclient(hunting): get_auth_request con 0x7f794c069410 auth_method 0 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7950f41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7950f41640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7950f41640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f794bfff640 10 monclient(hunting): get_auth_request con 0x7f794c0a41c0 auth_method 0 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f794bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f794bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f794bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 10 monclient(hunting): handle_auth_done global_id 5522 payload 824 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 1 -- 172.21.15.190:0/928787927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f794c069410 msgr2=0x7f794c135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f794c069410 0x7f794c135250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 1 -- 172.21.15.190:0/928787927 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f794c0a41c0 msgr2=0x7f794c135790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f794c0a41c0 0x7f794c135790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 10 monclient: _finish_hunting 0 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 1 monclient: found mon.noname-h 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.768+0000 7f7951742640 1 -- 172.21.15.190:0/928787927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c13d340 con 0x7f794c0a45a0 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f7951742640 1 -- 172.21.15.190:0/928787927 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f794c0a45a0 msgr2=0x7f794c13b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f7951742640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f794c0a45a0 0x7f794c13b300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f794400be60 tx=0x7f794400bf50 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 1 -- 172.21.15.190:0/928787927 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 1 -- 172.21.15.190:0/928787927 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7944015850 con 0x7f794c0a45a0 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:17.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 10 monclient: dump: 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:17.786 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 10 monclient: _finish_auth 0 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 10 monclient: _check_auth_tickets 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 1 -- 172.21.15.190:0/928787927 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7944010030 con 0x7f794c0a45a0 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79531a5640 5 monclient: authenticate success, global_id 5522 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79531a5640 10 monclient: _renew_subs 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79531a5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79531a5640 1 -- 172.21.15.190:0/928787927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794c11ec40 con 0x7f794c0a45a0 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:17.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 1 -- 172.21.15.190:0/928787927 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79440060d0 con 0x7f794c0a45a0 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f794affd640 4 set_mon_vals no callback set 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79497fa640 10 monclient: dump: 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:17.788 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79531a5640 10 monclient: _renew_subs 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79531a5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.772+0000 7f79531a5640 1 -- 172.21.15.190:0/928787927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f794c0a7b00 con 0x7f794c0a45a0 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.776+0000 7f79531a5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.776+0000 7f79531a5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.776+0000 7f79531a5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.776+0000 7f79531a5640 1 -- 172.21.15.190:0/928787927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f794c0a75b0 con 0x7f794c0a45a0 2024-08-19T10:42:17.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.776+0000 7f79497fa640 1 -- 172.21.15.190:0/928787927 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f79440153e0 con 0x7f794c0a45a0 2024-08-19T10:42:17.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.776+0000 7f79497fa640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f793003be00 0x7f793003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:17.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.776+0000 7f79497fa640 1 -- 172.21.15.190:0/928787927 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(133..133 src has 1..133) ==== 8233+0+0 (secure 0 0 0) 0x7f794404fcc0 con 0x7f794c0a45a0 2024-08-19T10:42:17.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.776+0000 7f7950f41640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f793003be00 0x7f793003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:17.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.776+0000 7f7950f41640 10 monclient: get_auth_request con 0x7f793003be00 auth_method 0 2024-08-19T10:42:17.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.776+0000 7f7950f41640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f793003be00 0x7f793003e2c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f79400ac000 tx=0x7f79400a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:17.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.780+0000 7f79497fa640 1 -- 172.21.15.190:0/928787927 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f794404d930 con 0x7f794c0a45a0 2024-08-19T10:42:17.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.780+0000 7f79497fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:17.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.780+0000 7f79497fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:17.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.948+0000 7f79531a5640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:17.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.948+0000 7f79531a5640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:17.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.948+0000 7f79531a5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.948+0000 7f79531a5640 1 -- 172.21.15.190:0/928787927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) -- 0x7f794c0a3c80 con 0x7f794c0a45a0 2024-08-19T10:42:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.952+0000 7f79497fa640 1 -- 172.21.15.190:0/928787927 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v133) ==== 73+0+6748 (secure 0 0 0) 0x7f7944013cf0 con 0x7f794c0a45a0 2024-08-19T10:42:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.952+0000 7f79497fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:17.955 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:17.952+0000 7f79497fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:18.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.048+0000 7f79531a5640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:18.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.048+0000 7f79531a5640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:18.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.048+0000 7f79531a5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:18.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.048+0000 7f79531a5640 1 -- 172.21.15.190:0/928787927 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set dump"} v 0) -- 0x7f794c0a3a30 con 0x7f794c0a45a0 2024-08-19T10:42:18.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.048+0000 7f79497fa640 1 -- 172.21.15.190:0/928787927 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set dump"}]=0 v133) ==== 73+0+6748 (secure 0 0 0) 0x7f7944015c50 con 0x7f794c0a45a0 2024-08-19T10:42:18.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.048+0000 7f79497fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set dump"}] 2024-08-19T10:42:18.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.048+0000 7f79497fa640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:18.054 INFO:tasks.workunit.client.0.smithi190.stdout:{ 2024-08-19T10:42:18.054 INFO:tasks.workunit.client.0.smithi190.stdout: "9": [ 2024-08-19T10:42:18.054 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.054 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -1, 2024-08-19T10:42:18.054 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.054 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.054 INFO:tasks.workunit.client.0.smithi190.stdout: 0.263671875, 2024-08-19T10:42:18.054 INFO:tasks.workunit.client.0.smithi190.stdout: 0.263671875, 2024-08-19T10:42:18.054 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -2, 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -3, 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.055 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -4, 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.056 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -5, 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.057 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -6, 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -7, 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.058 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.059 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.059 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.059 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.059 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -8, 2024-08-19T10:42:18.059 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.059 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.059 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.059 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.059 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.060 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.060 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.060 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -9, 2024-08-19T10:42:18.060 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.060 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.060 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.060 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.060 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.060 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.060 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -10, 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -11, 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.061 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.062 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.062 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.062 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.062 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -12, 2024-08-19T10:42:18.062 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.062 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.062 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.062 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.062 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.062 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -13, 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -14, 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -15, 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.063 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: "10": [ 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -1, 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: 0.263671875, 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: 0.263671875, 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.064 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: 0.263671875, 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: 0.263671875, 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: 0.263671875, 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: 0.263671875, 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.065 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -2, 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -3, 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.066 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.067 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.067 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.067 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.067 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.067 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.067 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.067 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.067 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.067 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.068 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.068 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.068 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.068 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.068 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.068 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.068 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -4, 2024-08-19T10:42:18.068 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.068 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -5, 2024-08-19T10:42:18.069 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.070 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.070 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.070 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.070 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.070 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.070 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.070 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.070 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.070 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.070 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.070 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.071 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.071 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.071 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.071 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.071 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.071 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.071 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.071 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -6, 2024-08-19T10:42:18.071 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.071 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.071 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.072 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.072 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.072 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.072 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.072 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.072 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.072 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.072 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.072 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.073 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.073 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.073 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.073 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.073 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.073 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.073 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.073 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.073 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.073 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -7, 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -8, 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.074 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -9, 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.075 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: ], 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: [ 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: 0.087890625, 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: 0, 2024-08-19T10:42:18.076 INFO:tasks.workunit.client.0.smithi190.stdout: 0 2024-08-19T10:42:18.077 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.077 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.077 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.077 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.077 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -10, 2024-08-19T10:42:18.077 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.077 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.077 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.077 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.077 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -11, 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -12, 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.078 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -13, 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.079 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -14, 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: }, 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: { 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: "bucket_id": -15, 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: "weight_set": [ 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: [], 2024-08-19T10:42:18.080 INFO:tasks.workunit.client.0.smithi190.stdout: [] 2024-08-19T10:42:18.081 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.081 INFO:tasks.workunit.client.0.smithi190.stdout: } 2024-08-19T10:42:18.081 INFO:tasks.workunit.client.0.smithi190.stdout: ] 2024-08-19T10:42:18.081 INFO:tasks.workunit.client.0.smithi190.stdout:} 2024-08-19T10:42:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 -- 172.21.15.190:0/928787927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f793003be00 msgr2=0x7f793003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f793003be00 0x7f793003e2c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f79400ac000 tx=0x7f79400a7000 comp rx=0 tx=0).stop 2024-08-19T10:42:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 10 monclient: shutdown 2024-08-19T10:42:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 -- 172.21.15.190:0/928787927 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f794c0a45a0 msgr2=0x7f794c13b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:18.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f794c0a45a0 0x7f794c13b300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f794400be60 tx=0x7f794400bf50 comp rx=0 tx=0).stop 2024-08-19T10:42:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 -- 172.21.15.190:0/928787927 shutdown_connections 2024-08-19T10:42:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f794c0a45a0 0x7f794c13b300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f794c0a41c0 0x7f794c135790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f793003be00 0x7f793003e2c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 --2- 172.21.15.190:0/928787927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f794c069410 0x7f794c135250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 -- 172.21.15.190:0/928787927 >> 172.21.15.190:0/928787927 conn(0x7f794c0a4d70 msgr2=0x7f794c138e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 -- 172.21.15.190:0/928787927 shutdown_connections 2024-08-19T10:42:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.052+0000 7f79531a5640 1 -- 172.21.15.190:0/928787927 wait complete. 2024-08-19T10:42:18.083 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 2024-08-19T10:42:18.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient: get_monmap_and_config 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient: build_initial_monmap 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient: monmap: 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:18.189527+0000 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:18.189527+0000 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:18.189 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:18.190 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:18.190 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:18.190 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:18.190 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:18.190 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:18.190 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:18.190 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:18.190 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:18.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 1 Processor -- start 2024-08-19T10:42:18.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 1 -- start start 2024-08-19T10:42:18.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient: init 2024-08-19T10:42:18.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient: _add_conns ranks=[1,0,9,8,5,7,3,4,6,10,2] 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb410139850 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb41009d4a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb41013fdf0 0x7fb4101421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient(hunting): picked mon.noname-a con 0x7fb41013fdf0 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41009e700 con 0x7fb41009d4a0 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb410074400 con 0x7fb410139850 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fb41009cdc0 con 0x7fb41013fdf0 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb416418640 10 monclient(hunting): authenticate will time out at 1183.759610s 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb40ffff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fb410139850 0x7fb410139c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:53498/0 (socket says 172.21.15.190:53498) 2024-08-19T10:42:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb40ffff640 1 -- 172.21.15.190:0/2251245061 learned_addr learned my addr 172.21.15.190:0/2251245061 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.184+0000 7fb4149b5640 1 --2- 172.21.15.190:0/2251245061 >> v2:172.21.15.89:3300/0 conn(0x7fb41013fdf0 0x7fb4101421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 10 monclient(hunting): get_auth_request con 0x7fb41013fdf0 auth_method 0 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 10 monclient(hunting): handle_auth_done global_id 5526 payload 274 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 1 -- 172.21.15.190:0/2251245061 >> v1:172.21.15.190:6789/0 conn(0x7fb410139850 legacy=0x7fb410139c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 1 -- 172.21.15.190:0/2251245061 >> v1:172.21.15.89:6789/0 conn(0x7fb41009d4a0 legacy=0x7fb41013e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 10 monclient: _finish_hunting 0 2024-08-19T10:42:18.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 1 monclient: found mon.noname-a 2024-08-19T10:42:18.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:18.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:18.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.188+0000 7fb4149b5640 1 -- 172.21.15.190:0/2251245061 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41009de60 con 0x7fb41013fdf0 2024-08-19T10:42:18.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb4149b5640 1 -- 172.21.15.190:0/2251245061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013fdf0 msgr2=0x7fb4101421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:18.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb4149b5640 1 --2- 172.21.15.190:0/2251245061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013fdf0 0x7fb4101421e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d90 tx=0x7fb40801c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95efc03a86ade5c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:18.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 1 -- 172.21.15.190:0/2251245061 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4080b2070 con 0x7fb41013fdf0 2024-08-19T10:42:18.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:18.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 10 monclient: dump: 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:18.197 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 10 monclient: _finish_auth 0 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 1 -- 172.21.15.190:0/2251245061 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4080a5ad0 con 0x7fb41013fdf0 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 1 -- 172.21.15.190:0/2251245061 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4080a5c80 con 0x7fb41013fdf0 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 5 monclient: authenticate success, global_id 5526 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 4 set_mon_vals no callback set 2024-08-19T10:42:18.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb40effd640 10 monclient: dump: 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:18.199 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:18.200 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:18.200 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:18.201 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:18.201 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:18.201 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:18.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 10 monclient: shutdown 2024-08-19T10:42:18.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:18.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 1 -- 172.21.15.190:0/2251245061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013fdf0 msgr2=0x7fb4101421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:18.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 1 --2- 172.21.15.190:0/2251245061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013fdf0 0x7fb4101421e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d90 tx=0x7fb40801c830 comp rx=0 tx=0).stop 2024-08-19T10:42:18.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 1 -- 172.21.15.190:0/2251245061 shutdown_connections 2024-08-19T10:42:18.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 1 --2- 172.21.15.190:0/2251245061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013fdf0 0x7fb4101421e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:18.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 1 -- 172.21.15.190:0/2251245061 >> 172.21.15.190:0/2251245061 conn(0x7fb4100a4790 msgr2=0x7fb4100a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:18.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 1 -- 172.21.15.190:0/2251245061 shutdown_connections 2024-08-19T10:42:18.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 1 -- 172.21.15.190:0/2251245061 wait complete. 2024-08-19T10:42:18.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 10 monclient: build_initial_monmap 2024-08-19T10:42:18.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 10 monclient: monmap: 2024-08-19T10:42:18.209 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:18.209 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:18.210 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:18.210 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:18.210 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:18.210 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:18.210 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:18.210 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:18.210 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:18.210 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:18.211 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:18.211 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:18.211 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:18.211 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:18.211 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:18.211 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:18.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 1 Processor -- start 2024-08-19T10:42:18.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 1 -- start start 2024-08-19T10:42:18.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.192+0000 7fb416418640 10 monclient: init 2024-08-19T10:42:18.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:18.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 10 monclient: _add_conns ranks=[8,3,0,5,7,4,1,6,2] 2024-08-19T10:42:18.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb410139850 0x7fb410133e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:18.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 10 monclient(hunting): picked mon.noname-e con 0x7fb410139850 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:18.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb41013fdf0 0x7fb410134380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:18.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 10 monclient(hunting): picked mon.noname-g con 0x7fb41013fdf0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:18.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013b2b0 0x7fb41013b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:18.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 10 monclient(hunting): picked mon.noname-a con 0x7fb41013b2b0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:18.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:18.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb410050c20 con 0x7fb41013b2b0 2024-08-19T10:42:18.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:18.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb41009d000 con 0x7fb41013fdf0 2024-08-19T10:42:18.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:18.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb41006a540 con 0x7fb410139850 2024-08-19T10:42:18.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:18.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb416418640 10 monclient(hunting): authenticate will time out at 1183.768368s 2024-08-19T10:42:18.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb40f7fe640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb41013fdf0 msgr2=0x7fb410134380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:18.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb40f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb41013fdf0 0x7fb410134380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:18.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013b2b0 0x7fb41013b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:18.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb40ffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb410139850 0x7fb410133e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:18.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013b2b0 0x7fb41013b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:39102/0 (socket says 172.21.15.190:39102) 2024-08-19T10:42:18.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 1 -- 172.21.15.190:0/2936995228 learned_addr learned my addr 172.21.15.190:0/2936995228 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:18.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 10 monclient(hunting): get_auth_request con 0x7fb41013b2b0 auth_method 0 2024-08-19T10:42:18.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:18.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:18.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:18.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb40ffff640 10 monclient(hunting): get_auth_request con 0x7fb410139850 auth_method 0 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb40ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb40ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb40ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb40ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb40ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb40ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 10 monclient(hunting): handle_auth_done global_id 5535 payload 824 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 1 -- 172.21.15.190:0/2936995228 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb410139850 msgr2=0x7fb410133e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:18.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb410139850 0x7fb410133e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:18.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 1 -- 172.21.15.190:0/2936995228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb41013fdf0 msgr2=0x7fb410134380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:18.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb41013fdf0 0x7fb410134380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:18.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 10 monclient: _finish_hunting 0 2024-08-19T10:42:18.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 1 monclient: found mon.noname-a 2024-08-19T10:42:18.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:18.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:18.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 1 -- 172.21.15.190:0/2936995228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4100a76f0 con 0x7fb41013b2b0 2024-08-19T10:42:18.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 1 -- 172.21.15.190:0/2936995228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013b2b0 msgr2=0x7fb41013b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:18.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.196+0000 7fb4149b5640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013b2b0 0x7fb41013b6f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb40801d0d0 tx=0x7fb4080195d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:18.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 1 -- 172.21.15.190:0/2936995228 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4080bb030 con 0x7fb41013b2b0 2024-08-19T10:42:18.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:18.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:18.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 10 monclient: dump: 2024-08-19T10:42:18.217 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:18.217 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:18.217 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:18.217 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:18.217 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:18.217 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:18.218 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:18.218 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:18.218 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:18.218 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:18.218 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:18.218 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:18.218 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:18.218 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:18.218 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:18.218 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:18.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 10 monclient: _finish_auth 0 2024-08-19T10:42:18.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 10 monclient: _check_auth_tickets 2024-08-19T10:42:18.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:18.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 1 -- 172.21.15.190:0/2936995228 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb408003c70 con 0x7fb41013b2b0 2024-08-19T10:42:18.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb416418640 5 monclient: authenticate success, global_id 5535 2024-08-19T10:42:18.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb416418640 10 monclient: _renew_subs 2024-08-19T10:42:18.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb416418640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:18.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb416418640 1 -- 172.21.15.190:0/2936995228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41013cbd0 con 0x7fb41013b2b0 2024-08-19T10:42:18.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:18.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 1 -- 172.21.15.190:0/2936995228 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4080b2040 con 0x7fb41013b2b0 2024-08-19T10:42:18.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:18.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:18.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40dffb640 4 set_mon_vals no callback set 2024-08-19T10:42:18.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 10 monclient: dump: 2024-08-19T10:42:18.220 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:18.220 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:18.220 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:18.220 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:18.221 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:18.221 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:18.221 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:18.221 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:18.221 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:18.221 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:18.221 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:18.221 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:18.221 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:18.221 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:18.222 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:18.222 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:18.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb416418640 10 monclient: _renew_subs 2024-08-19T10:42:18.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb416418640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:18.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb416418640 1 -- 172.21.15.190:0/2936995228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41013ef30 con 0x7fb41013b2b0 2024-08-19T10:42:18.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb416418640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:18.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb416418640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:18.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb416418640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:18.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb416418640 1 -- 172.21.15.190:0/2936995228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41013d4b0 con 0x7fb41013b2b0 2024-08-19T10:42:18.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 1 -- 172.21.15.190:0/2936995228 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb4080be070 con 0x7fb41013b2b0 2024-08-19T10:42:18.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3f003bd60 0x7fb3f003e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:18.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40cff9640 1 -- 172.21.15.190:0/2936995228 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 8233+0+0 (secure 0 0 0) 0x7fb4080bc070 con 0x7fb41013b2b0 2024-08-19T10:42:18.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40ffff640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3f003bd60 0x7fb3f003e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:18.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.200+0000 7fb40ffff640 10 monclient: get_auth_request con 0x7fb3f003bd60 auth_method 0 2024-08-19T10:42:18.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.204+0000 7fb40ffff640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3f003bd60 0x7fb3f003e220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb400007ca0 tx=0x7fb400007bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:18.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.212+0000 7fb40cff9640 1 -- 172.21.15.190:0/2936995228 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb4080c5020 con 0x7fb41013b2b0 2024-08-19T10:42:18.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.212+0000 7fb40cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:18.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.212+0000 7fb40cff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.392+0000 7fb416418640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.392+0000 7fb416418640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.392+0000 7fb416418640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.392+0000 7fb416418640 1 -- 172.21.15.190:0/2936995228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) -- 0x7fb4100a0c00 con 0x7fb41013b2b0 2024-08-19T10:42:18.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.720+0000 7fb40cff9640 1 -- 172.21.15.190:0/2936995228 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=0 v134) ==== 129+0+0 (secure 0 0 0) 0x7fb4100a76f0 con 0x7fb41013b2b0 2024-08-19T10:42:18.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.720+0000 7fb40cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:18.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.720+0000 7fb40cff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:18.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.836+0000 7fb416418640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:18.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.836+0000 7fb416418640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:18.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.836+0000 7fb416418640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:18.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:18.836+0000 7fb416418640 1 -- 172.21.15.190:0/2936995228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]} v 0) -- 0x7fb410139c30 con 0x7fb41013b2b0 2024-08-19T10:42:19.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.724+0000 7fb40cff9640 1 -- 172.21.15.190:0/2936995228 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}]=0 v135) ==== 129+0+0 (secure 0 0 0) 0x7fb408016540 con 0x7fb41013b2b0 2024-08-19T10:42:19.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.724+0000 7fb40cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:19.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.724+0000 7fb40cff9640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:19.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 -- 172.21.15.190:0/2936995228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3f003bd60 msgr2=0x7fb3f003e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:19.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3f003bd60 0x7fb3f003e220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb400007ca0 tx=0x7fb400007bf0 comp rx=0 tx=0).stop 2024-08-19T10:42:19.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 10 monclient: shutdown 2024-08-19T10:42:19.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:19.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 -- 172.21.15.190:0/2936995228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013b2b0 msgr2=0x7fb41013b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:19.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013b2b0 0x7fb41013b6f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb40801d0d0 tx=0x7fb4080195d0 comp rx=0 tx=0).stop 2024-08-19T10:42:19.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 -- 172.21.15.190:0/2936995228 shutdown_connections 2024-08-19T10:42:19.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb41013b2b0 0x7fb41013b6f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:19.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb41013fdf0 0x7fb410134380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:19.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3f003bd60 0x7fb3f003e220 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:19.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 --2- 172.21.15.190:0/2936995228 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb410139850 0x7fb410133e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:19.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 -- 172.21.15.190:0/2936995228 >> 172.21.15.190:0/2936995228 conn(0x7fb4100a4790 msgr2=0x7fb4100a2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:19.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 -- 172.21.15.190:0/2936995228 shutdown_connections 2024-08-19T10:42:19.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.728+0000 7fb416418640 1 -- 172.21.15.190:0/2936995228 wait complete. 2024-08-19T10:42:19.755 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush weight-set reweight fooset osd.0 .9 .9 2024-08-19T10:42:19.755 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:42:19.755 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight fooset osd.0 .9 .9 2024-08-19T10:42:19.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient: get_monmap_and_config 2024-08-19T10:42:19.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient: build_initial_monmap 2024-08-19T10:42:19.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient: monmap: 2024-08-19T10:42:19.879 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:19.880 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:19.880 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:19.879894+0000 2024-08-19T10:42:19.880 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:19.879894+0000 2024-08-19T10:42:19.880 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:19.880 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:19.880 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:19.880 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:19.880 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:19.880 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:19.880 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:19.881 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:19.881 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:19.881 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:19.881 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:19.881 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:19.881 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:19.881 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:19.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 1 Processor -- start 2024-08-19T10:42:19.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 1 -- start start 2024-08-19T10:42:19.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient: init 2024-08-19T10:42:19.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:19.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient: _add_conns ranks=[8,5,0,7,10,6,2,4,1,9,3] 2024-08-19T10:42:19.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0440a57d0 0x7fc044071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:19.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient(hunting): picked mon.noname-e con 0x7fc0440a57d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:19.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc0440a5bb0 0x7fc0440a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:19.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient(hunting): picked mon.noname-h con 0x7fc0440a5bb0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:19.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc0441395a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:19.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04406f020 con 0x7fc0441395a0 2024-08-19T10:42:19.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:19.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc04406dd90 con 0x7fc0440a5bb0 2024-08-19T10:42:19.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:19.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc04406f300 con 0x7fc0440a57d0 2024-08-19T10:42:19.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:19.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc04ae94640 10 monclient(hunting): authenticate will time out at 1185.449740s 2024-08-19T10:42:19.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc0440a5bb0 0x7fc0440a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:19.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc0440a5bb0 0x7fc0440a5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36930/0 (socket says 172.21.15.190:36930) 2024-08-19T10:42:19.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 1 -- 172.21.15.190:0/1933034666 learned_addr learned my addr 172.21.15.190:0/1933034666 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:19.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 10 monclient(hunting): get_auth_request con 0x7fc0440a5bb0 auth_method 0 2024-08-19T10:42:19.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:19.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:19.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:19.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:19.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:19.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:19.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc048c30640 1 --2- 172.21.15.190:0/1933034666 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0440a57d0 0x7fc044071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:19.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc048c30640 10 monclient(hunting): get_auth_request con 0x7fc0440a57d0 auth_method 0 2024-08-19T10:42:19.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc048c30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:19.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc048c30640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:19.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc048c30640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:19.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 10 monclient(hunting): handle_auth_done global_id 5531 payload 274 2024-08-19T10:42:19.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 1 -- 172.21.15.190:0/1933034666 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0440a57d0 msgr2=0x7fc044071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:19.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 1 --2- 172.21.15.190:0/1933034666 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0440a57d0 0x7fc044071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:19.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 1 -- 172.21.15.190:0/1933034666 >> v1:172.21.15.89:6789/0 conn(0x7fc0441395a0 legacy=0x7fc044139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:19.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:19.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 1 monclient: found mon.noname-h 2024-08-19T10:42:19.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:19.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:19.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.876+0000 7fc043fff640 1 -- 172.21.15.190:0/1933034666 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04406f5e0 con 0x7fc0440a5bb0 2024-08-19T10:42:19.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc043fff640 1 -- 172.21.15.190:0/1933034666 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc0440a5bb0 msgr2=0x7fc0440a5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:19.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc043fff640 1 --2- 172.21.15.190:0/1933034666 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc0440a5bb0 0x7fc0440a5fb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc0300098a0 tx=0x7fc03001c400 comp rx=0 tx=0).ready entity=mon.5 client_cookie=85662a78d74644b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:19.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 1 -- 172.21.15.190:0/1933034666 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0300b2070 con 0x7fc0440a5bb0 2024-08-19T10:42:19.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:19.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:19.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 10 monclient: dump: 2024-08-19T10:42:19.898 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:19.899 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:19.899 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:19.899 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:19.899 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:19.899 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:19.899 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:19.899 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:19.899 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:19.899 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:19.899 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:19.900 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:19.900 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:19.900 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:19.900 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:19.900 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:19.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:19.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:19.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:19.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 5 monclient: authenticate success, global_id 5531 2024-08-19T10:42:19.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 1 -- 172.21.15.190:0/1933034666 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0300ab7f0 con 0x7fc0440a5bb0 2024-08-19T10:42:19.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:19.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:19.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 4 set_mon_vals no callback set 2024-08-19T10:42:19.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 1 -- 172.21.15.190:0/1933034666 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0300abe50 con 0x7fc0440a5bb0 2024-08-19T10:42:19.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:19.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:19.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc0437fe640 10 monclient: dump: 2024-08-19T10:42:19.902 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:19.902 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:19.902 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:19.902 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:19.902 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:19.902 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:19.902 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:19.903 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:19.903 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:19.903 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:19.903 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:19.903 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:19.904 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:19.904 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:19.904 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:19.904 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:19.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 10 monclient: shutdown 2024-08-19T10:42:19.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:19.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 1 -- 172.21.15.190:0/1933034666 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc0440a5bb0 msgr2=0x7fc0440a5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 1 --2- 172.21.15.190:0/1933034666 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc0440a5bb0 0x7fc0440a5fb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc0300098a0 tx=0x7fc03001c400 comp rx=0 tx=0).stop 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 1 -- 172.21.15.190:0/1933034666 shutdown_connections 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 1 --2- 172.21.15.190:0/1933034666 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc0440a5bb0 0x7fc0440a5fb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 1 --2- 172.21.15.190:0/1933034666 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0440a57d0 0x7fc044071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 1 -- 172.21.15.190:0/1933034666 >> 172.21.15.190:0/1933034666 conn(0x7fc0440aa3c0 msgr2=0x7fc0440a68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 1 -- 172.21.15.190:0/1933034666 shutdown_connections 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 1 -- 172.21.15.190:0/1933034666 wait complete. 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 10 monclient: build_initial_monmap 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.880+0000 7fc04ae94640 10 monclient: monmap: 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:19.906 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:19.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 1 Processor -- start 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 1 -- start start 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 10 monclient: init 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 10 monclient: _add_conns ranks=[4,3,6,7,0,5,2,1,8] 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc0440a57d0 0x7fc044120410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 10 monclient(hunting): picked mon.noname-c con 0x7fc0440a57d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440a5bb0 0x7fc044120950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 10 monclient(hunting): picked mon.noname-g con 0x7fc0440a5bb0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0441395a0 0x7fc044120e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 10 monclient(hunting): picked mon.noname-d con 0x7fc0441395a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc04406e660 con 0x7fc0440a5bb0 2024-08-19T10:42:19.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:19.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc0441442a0 con 0x7fc0440a57d0 2024-08-19T10:42:19.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:19.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fc0441214d0 con 0x7fc0441395a0 2024-08-19T10:42:19.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:19.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc04ae94640 10 monclient(hunting): authenticate will time out at 1185.456842s 2024-08-19T10:42:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc043fff640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440a5bb0 msgr2=0x7fc044120950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc043fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440a5bb0 0x7fc044120950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc0440a57d0 0x7fc044120410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc0440a57d0 0x7fc044120410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:49738/0 (socket says 172.21.15.190:49738) 2024-08-19T10:42:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 1 -- 172.21.15.190:0/2278577264 learned_addr learned my addr 172.21.15.190:0/2278577264 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 10 monclient(hunting): get_auth_request con 0x7fc0440a57d0 auth_method 0 2024-08-19T10:42:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:19.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc049431640 1 --2- 172.21.15.190:0/2278577264 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0441395a0 0x7fc044120e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc049431640 10 monclient(hunting): get_auth_request con 0x7fc0441395a0 auth_method 0 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc049431640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc049431640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc049431640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc049431640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc049431640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc049431640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:19.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 10 monclient(hunting): handle_auth_done global_id 5485 payload 824 2024-08-19T10:42:19.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 1 -- 172.21.15.190:0/2278577264 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0441395a0 msgr2=0x7fc044120e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:19.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 1 --2- 172.21.15.190:0/2278577264 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0441395a0 0x7fc044120e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:19.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 1 -- 172.21.15.190:0/2278577264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440a5bb0 msgr2=0x7fc044120950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:19.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 1 --2- 172.21.15.190:0/2278577264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0440a5bb0 0x7fc044120950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:19.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 10 monclient: _finish_hunting 0 2024-08-19T10:42:19.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 1 monclient: found mon.noname-c 2024-08-19T10:42:19.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:19.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:19.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.884+0000 7fc048c30640 1 -- 172.21.15.190:0/2278577264 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0441406c0 con 0x7fc0440a57d0 2024-08-19T10:42:19.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc048c30640 1 -- 172.21.15.190:0/2278577264 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc0440a57d0 msgr2=0x7fc044120410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc048c30640 1 --2- 172.21.15.190:0/2278577264 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc0440a57d0 0x7fc044120410 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc03800d800 tx=0x7fc03800dcc0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 1 -- 172.21.15.190:0/2278577264 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc03800bd40 con 0x7fc0440a57d0 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 10 monclient: dump: 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:19.913 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 10 monclient: _finish_auth 0 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 10 monclient: _check_auth_tickets 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:19.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 1 -- 172.21.15.190:0/2278577264 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc038019070 con 0x7fc0440a57d0 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 1 -- 172.21.15.190:0/2278577264 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0380148a0 con 0x7fc0440a57d0 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc04ae94640 5 monclient: authenticate success, global_id 5485 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc04ae94640 10 monclient: _renew_subs 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc04ae94640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc04ae94640 1 -- 172.21.15.190:0/2278577264 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0441229f0 con 0x7fc0440a57d0 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc042ffd640 4 set_mon_vals no callback set 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 10 monclient: dump: 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:19.915 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:19.916 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc04ae94640 10 monclient: _renew_subs 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc04ae94640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc04ae94640 1 -- 172.21.15.190:0/2278577264 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0440a7cc0 con 0x7fc0440a57d0 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc04ae94640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc04ae94640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc04ae94640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc04ae94640 1 -- 172.21.15.190:0/2278577264 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0440a5f90 con 0x7fc0440a57d0 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 1 -- 172.21.15.190:0/2278577264 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc038014430 con 0x7fc0440a57d0 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 1 --2- 172.21.15.190:0/2278577264 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc02803be00 0x7fc02803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc0417fa640 1 -- 172.21.15.190:0/2278577264 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(135..135 src has 1..135) ==== 8233+0+0 (secure 0 0 0) 0x7fc03800f040 con 0x7fc0440a57d0 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc043fff640 1 --2- 172.21.15.190:0/2278577264 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc02803be00 0x7fc02803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.888+0000 7fc043fff640 10 monclient: get_auth_request con 0x7fc02803be00 auth_method 0 2024-08-19T10:42:19.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.892+0000 7fc043fff640 1 --2- 172.21.15.190:0/2278577264 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc02803be00 0x7fc02803e2c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc0300af000 tx=0x7fc030008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:19.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.900+0000 7fc0417fa640 1 -- 172.21.15.190:0/2278577264 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc038022020 con 0x7fc0440a57d0 2024-08-19T10:42:19.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.900+0000 7fc0417fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:19.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:19.900+0000 7fc0417fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:20.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.084+0000 7fc04ae94640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-08-19T10:42:20.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.084+0000 7fc04ae94640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-08-19T10:42:20.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.084+0000 7fc04ae94640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:20.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.084+0000 7fc04ae94640 1 -- 172.21.15.190:0/2278577264 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]} v 0) -- 0x7fc044071f60 con 0x7fc0440a57d0 2024-08-19T10:42:20.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.088+0000 7fc0417fa640 1 -- 172.21.15.190:0/2278577264 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}]=-22 must specify exact 1 weight values v135) ==== 168+0+0 (secure 0 0 0) 0x7fc03804c900 con 0x7fc0440a57d0 2024-08-19T10:42:20.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.088+0000 7fc0417fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "fooset", "item": "osd.0", "weight": [0.9, 0.9]}] 2024-08-19T10:42:20.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.088+0000 7fc0417fa640 10 monclient: _finish_command 2 = mon:22 must specify exact 1 weight values 2024-08-19T10:42:20.091 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: must specify exact 1 weight values 2024-08-19T10:42:20.096 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:42:20.096 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect_false ceph osd crush weight-set reweight barset osd.0 .9 2024-08-19T10:42:20.097 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -x 2024-08-19T10:42:20.097 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight barset osd.0 .9 2024-08-19T10:42:20.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.212+0000 7ff599182640 10 monclient: get_monmap_and_config 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.212+0000 7ff599182640 10 monclient: build_initial_monmap 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.212+0000 7ff599182640 10 monclient: monmap: 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:20.218335+0000 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:20.218335+0000 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:20.218 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:20.219 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:20.219 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:20.219 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:20.219 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:20.219 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:20.219 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:20.219 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:20.219 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.212+0000 7ff599182640 1 Processor -- start 2024-08-19T10:42:20.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.212+0000 7ff599182640 1 -- start start 2024-08-19T10:42:20.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.212+0000 7ff599182640 10 monclient: init 2024-08-19T10:42:20.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:20.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 10 monclient: _add_conns ranks=[4,5,7,6,1,8,3,9,2,10,0] 2024-08-19T10:42:20.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5940a69c0 0x7ff594139780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 10 monclient(hunting): picked mon.noname-c con 0x7ff5940a69c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:20.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff594139cc0 0x7ff59413a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 10 monclient(hunting): picked mon.noname-h con 0x7ff594139cc0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5940a6da0 0x7ff5940a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 10 monclient(hunting): picked mon.noname-i con 0x7ff5940a6da0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff59413a5e0 con 0x7ff5940a69c0 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff5940a46b0 con 0x7ff594139cc0 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff5940a4990 con 0x7ff5940a6da0 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff599182640 10 monclient(hunting): authenticate will time out at 1185.788371s 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5940a6da0 0x7ff5940a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59259c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff594139cc0 0x7ff59413a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5940a6da0 0x7ff5940a71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34450/0 (socket says 172.21.15.190:34450) 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 1 -- 172.21.15.190:0/4271999049 learned_addr learned my addr 172.21.15.190:0/4271999049 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 10 monclient(hunting): get_auth_request con 0x7ff5940a6da0 auth_method 0 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59259c640 10 monclient(hunting): get_auth_request con 0x7ff594139cc0 auth_method 0 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:20.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff592d9d640 1 --2- 172.21.15.190:0/4271999049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5940a69c0 0x7ff594139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff592d9d640 10 monclient(hunting): get_auth_request con 0x7ff5940a69c0 auth_method 0 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff592d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff592d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff592d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 10 monclient(hunting): handle_auth_done global_id 14740 payload 274 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 1 -- 172.21.15.190:0/4271999049 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff594139cc0 msgr2=0x7ff59413a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 1 --2- 172.21.15.190:0/4271999049 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff594139cc0 0x7ff59413a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 1 -- 172.21.15.190:0/4271999049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5940a69c0 msgr2=0x7ff594139780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 1 --2- 172.21.15.190:0/4271999049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5940a69c0 0x7ff594139780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 10 monclient: _finish_hunting 0 2024-08-19T10:42:20.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 1 monclient: found mon.noname-i 2024-08-19T10:42:20.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:20.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:20.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59359e640 1 -- 172.21.15.190:0/4271999049 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5940a4c70 con 0x7ff5940a6da0 2024-08-19T10:42:20.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.216+0000 7ff59259c640 1 --2- 172.21.15.190:0/4271999049 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff594139cc0 0x7ff59413a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:20.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff59359e640 1 -- 172.21.15.190:0/4271999049 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5940a6da0 msgr2=0x7ff5940a71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:20.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff59359e640 1 --2- 172.21.15.190:0/4271999049 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5940a6da0 0x7ff5940a71a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009e80 tx=0x7ff58c01c910 comp rx=0 tx=0).ready entity=mon.7 client_cookie=553662f1f65bcdf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:20.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 1 -- 172.21.15.190:0/4271999049 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff58c0090d0 con 0x7ff5940a6da0 2024-08-19T10:42:20.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:20.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:20.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 10 monclient: dump: 2024-08-19T10:42:20.231 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:20.232 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 10 monclient: _finish_auth 0 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 1 -- 172.21.15.190:0/4271999049 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff58c0a5b80 con 0x7ff5940a6da0 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 5 monclient: authenticate success, global_id 14740 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 1 -- 172.21.15.190:0/4271999049 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff58c0a5d30 con 0x7ff5940a6da0 2024-08-19T10:42:20.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 4 set_mon_vals no callback set 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff591d9b640 10 monclient: dump: 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:20.234 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient: shutdown 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 -- 172.21.15.190:0/4271999049 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5940a6da0 msgr2=0x7ff5940a71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 --2- 172.21.15.190:0/4271999049 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5940a6da0 0x7ff5940a71a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009e80 tx=0x7ff58c01c910 comp rx=0 tx=0).stop 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 -- 172.21.15.190:0/4271999049 shutdown_connections 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 --2- 172.21.15.190:0/4271999049 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5940a6da0 0x7ff5940a71a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 --2- 172.21.15.190:0/4271999049 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff594139cc0 0x7ff59413a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 --2- 172.21.15.190:0/4271999049 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5940a69c0 0x7ff594139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 -- 172.21.15.190:0/4271999049 >> 172.21.15.190:0/4271999049 conn(0x7ff5940abfc0 msgr2=0x7ff5940ac3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 -- 172.21.15.190:0/4271999049 shutdown_connections 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 -- 172.21.15.190:0/4271999049 wait complete. 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient: build_initial_monmap 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient: monmap: 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:20.236 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 Processor -- start 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 -- start start 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient: init 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:20.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient: _add_conns ranks=[1,0,8,7,4,2,3,6,5] 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5940a69c0 0x7ff59409cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient(hunting): picked mon.noname-f con 0x7ff5940a69c0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5940a6da0 0x7ff59409d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient(hunting): picked mon.noname-a con 0x7ff5940a6da0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff594139cc0 0x7ff59409d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient(hunting): picked mon.noname-e con 0x7ff594139cc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940a7c70 con 0x7ff5940a6da0 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940a7af0 con 0x7ff5940a69c0 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff5941431f0 con 0x7ff594139cc0 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff599182640 10 monclient(hunting): authenticate will time out at 1185.795851s 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.220+0000 7ff592d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5940a69c0 0x7ff59409cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5940a69c0 0x7ff59409cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39010/0 (socket says 172.21.15.190:39010) 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 1 -- 172.21.15.190:0/3799449775 learned_addr learned my addr 172.21.15.190:0/3799449775 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 10 monclient(hunting): get_auth_request con 0x7ff5940a69c0 auth_method 0 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff59259c640 1 --2- 172.21.15.190:0/3799449775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5940a6da0 0x7ff59409d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff59359e640 1 --2- 172.21.15.190:0/3799449775 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff594139cc0 0x7ff59409d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff59259c640 10 monclient(hunting): get_auth_request con 0x7ff5940a6da0 auth_method 0 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff59259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff59259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff59259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 10 monclient(hunting): handle_auth_done global_id 14815 payload 824 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 1 -- 172.21.15.190:0/3799449775 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff594139cc0 msgr2=0x7ff59409d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 1 --2- 172.21.15.190:0/3799449775 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff594139cc0 0x7ff59409d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 1 -- 172.21.15.190:0/3799449775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5940a6da0 msgr2=0x7ff59409d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 1 --2- 172.21.15.190:0/3799449775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5940a6da0 0x7ff59409d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 1 monclient: found mon.noname-f 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:20.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 1 -- 172.21.15.190:0/3799449775 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5940a2660 con 0x7ff5940a69c0 2024-08-19T10:42:20.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff59359e640 10 monclient: get_auth_request con 0x7ff594139cc0 auth_method 0 2024-08-19T10:42:20.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff59359e640 1 --2- 172.21.15.190:0/3799449775 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff594139cc0 0x7ff59409d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:20.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 1 -- 172.21.15.190:0/3799449775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5940a69c0 msgr2=0x7ff59409cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:20.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff592d9d640 1 --2- 172.21.15.190:0/3799449775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5940a69c0 0x7ff59409cd20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00d800 tx=0x7ff57c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:20.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 1 -- 172.21.15.190:0/3799449775 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff57c00bd40 con 0x7ff5940a69c0 2024-08-19T10:42:20.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:20.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:20.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 10 monclient: dump: 2024-08-19T10:42:20.241 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:20.241 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:20.242 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 1 -- 172.21.15.190:0/3799449775 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c019070 con 0x7ff5940a69c0 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff599182640 5 monclient: authenticate success, global_id 14815 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff599182640 10 monclient: _renew_subs 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff599182640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff599182640 1 -- 172.21.15.190:0/3799449775 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59409f820 con 0x7ff5940a69c0 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 1 -- 172.21.15.190:0/3799449775 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff57c0148a0 con 0x7ff5940a69c0 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:20.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff58b7fe640 10 monclient: dump: 2024-08-19T10:42:20.244 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:20.244 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:20.244 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.244 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.244 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:20.244 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.244 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:20.244 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:20.244 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:20.244 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff599182640 10 monclient: _renew_subs 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff599182640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff59159a640 4 set_mon_vals no callback set 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.224+0000 7ff599182640 1 -- 172.21.15.190:0/3799449775 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff59413e6f0 con 0x7ff5940a69c0 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.228+0000 7ff599182640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.228+0000 7ff599182640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.228+0000 7ff599182640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:20.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.228+0000 7ff599182640 1 -- 172.21.15.190:0/3799449775 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5940a71a0 con 0x7ff5940a69c0 2024-08-19T10:42:20.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.228+0000 7ff58b7fe640 1 -- 172.21.15.190:0/3799449775 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff57c014430 con 0x7ff5940a69c0 2024-08-19T10:42:20.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.228+0000 7ff58b7fe640 1 --2- 172.21.15.190:0/3799449775 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff57403be30 0x7ff57403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.228+0000 7ff58b7fe640 1 -- 172.21.15.190:0/3799449775 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 8233+0+0 (secure 0 0 0) 0x7ff57c00f040 con 0x7ff5940a69c0 2024-08-19T10:42:20.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.228+0000 7ff59259c640 1 --2- 172.21.15.190:0/3799449775 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff57403be30 0x7ff57403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.228+0000 7ff59259c640 10 monclient: get_auth_request con 0x7ff57403be30 auth_method 0 2024-08-19T10:42:20.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.228+0000 7ff59259c640 1 --2- 172.21.15.190:0/3799449775 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff57403be30 0x7ff57403e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff59413e430 tx=0x7ff584008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:20.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.232+0000 7ff58b7fe640 1 -- 172.21.15.190:0/3799449775 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff57c022020 con 0x7ff5940a69c0 2024-08-19T10:42:20.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.232+0000 7ff58b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:20.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.232+0000 7ff58b7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:20.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.408+0000 7ff599182640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:20.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.408+0000 7ff599182640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:20.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.408+0000 7ff599182640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:20.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.408+0000 7ff599182640 1 -- 172.21.15.190:0/3799449775 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]} v 0) -- 0x7ff59406c9c0 con 0x7ff5940a69c0 2024-08-19T10:42:20.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.424+0000 7ff58b7fe640 1 -- 172.21.15.190:0/3799449775 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}]=-22 must specify exact 3 weight values v135) ==== 163+0+0 (secure 0 0 0) 0x7ff57c04d540 con 0x7ff5940a69c0 2024-08-19T10:42:20.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.424+0000 7ff58b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9]}] 2024-08-19T10:42:20.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.424+0000 7ff58b7fe640 10 monclient: _finish_command 2 = mon:22 must specify exact 3 weight values 2024-08-19T10:42:20.433 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: must specify exact 3 weight values 2024-08-19T10:42:20.434 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:42:20.434 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight barset osd.0 .9 .9 .9 2024-08-19T10:42:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient: get_monmap_and_config 2024-08-19T10:42:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient: build_initial_monmap 2024-08-19T10:42:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient: monmap: 2024-08-19T10:42:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:20.561339+0000 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:20.561339+0000 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:20.562 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:20.563 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 1 Processor -- start 2024-08-19T10:42:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 1 -- start start 2024-08-19T10:42:20.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient: init 2024-08-19T10:42:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient: _add_conns ranks=[10,2,3,0,5,1,8,9,4,6,7] 2024-08-19T10:42:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f18b8139710 0x7f18b8139af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient(hunting): picked mon.noname-f con 0x7f18b8139710 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b813a030 0x7f18b80a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient(hunting): picked mon.noname-b con 0x7f18b813a030 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 0x7f18b80a5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient(hunting): picked mon.noname-g con 0x7f18b80a51e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f18b806f850 con 0x7f18b8139710 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f18b806de80 con 0x7f18b813a030 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f18b806e090 con 0x7f18b80a51e0 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18bd668640 10 monclient(hunting): authenticate will time out at 1186.131227s 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b77fe640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 msgr2=0x7f18b80a5670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 0x7f18b80a5670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f18b8139710 0x7f18b8139af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f18b8139710 0x7f18b8139af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39014/0 (socket says 172.21.15.190:39014) 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 1 -- 172.21.15.190:0/2356653853 learned_addr learned my addr 172.21.15.190:0/2356653853 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 10 monclient(hunting): get_auth_request con 0x7f18b8139710 auth_method 0 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b67fc640 1 --2- 172.21.15.190:0/2356653853 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b813a030 0x7f18b80a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b67fc640 10 monclient(hunting): get_auth_request con 0x7f18b813a030 auth_method 0 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b67fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 10 monclient(hunting): handle_auth_done global_id 14824 payload 274 2024-08-19T10:42:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 1 -- 172.21.15.190:0/2356653853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 msgr2=0x7f18b80a5670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 1 --2- 172.21.15.190:0/2356653853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 0x7f18b80a5670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 1 -- 172.21.15.190:0/2356653853 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b813a030 msgr2=0x7f18b80a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 1 --2- 172.21.15.190:0/2356653853 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b813a030 0x7f18b80a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:42:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 1 monclient: found mon.noname-f 2024-08-19T10:42:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.556+0000 7f18b6ffd640 1 -- 172.21.15.190:0/2356653853 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b806ef20 con 0x7f18b8139710 2024-08-19T10:42:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b6ffd640 1 -- 172.21.15.190:0/2356653853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8139710 msgr2=0x7f18b8139af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b6ffd640 1 --2- 172.21.15.190:0/2356653853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8139710 0x7f18b8139af0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009870 tx=0x7f18a801c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=760dd4a7b3d331e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 1 -- 172.21.15.190:0/2356653853 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18a80b0070 con 0x7f18b8139710 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 10 monclient: dump: 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 1 -- 172.21.15.190:0/2356653853 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a80aa810 con 0x7f18b8139710 2024-08-19T10:42:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 1 -- 172.21.15.190:0/2356653853 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18a80aae30 con 0x7f18b8139710 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 5 monclient: authenticate success, global_id 14824 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 4 set_mon_vals no callback set 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b5ffb640 10 monclient: dump: 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.577 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:20.577 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:20.577 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:20.577 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:20.577 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:20.577 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:20.577 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:20.577 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:20.577 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:20.577 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient: shutdown 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 -- 172.21.15.190:0/2356653853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8139710 msgr2=0x7f18b8139af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 --2- 172.21.15.190:0/2356653853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8139710 0x7f18b8139af0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009870 tx=0x7f18a801c440 comp rx=0 tx=0).stop 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 -- 172.21.15.190:0/2356653853 shutdown_connections 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 --2- 172.21.15.190:0/2356653853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 0x7f18b80a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 --2- 172.21.15.190:0/2356653853 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b813a030 0x7f18b80a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 --2- 172.21.15.190:0/2356653853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18b8139710 0x7f18b8139af0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 -- 172.21.15.190:0/2356653853 >> 172.21.15.190:0/2356653853 conn(0x7f18b80aa330 msgr2=0x7f18b80aa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 -- 172.21.15.190:0/2356653853 shutdown_connections 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 -- 172.21.15.190:0/2356653853 wait complete. 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient: build_initial_monmap 2024-08-19T10:42:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient: monmap: 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:20.580 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 Processor -- start 2024-08-19T10:42:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 -- start start 2024-08-19T10:42:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient: init 2024-08-19T10:42:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient: _add_conns ranks=[3,2,7,0,6,1,8,5,4] 2024-08-19T10:42:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 0x7f18b8122a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient(hunting): picked mon.noname-g con 0x7f18b80a51e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b8139710 0x7f18b8138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient(hunting): picked mon.noname-b con 0x7f18b8139710 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18b813a030 0x7f18b81d1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient(hunting): picked mon.noname-i con 0x7f18b813a030 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f18b80a5fc0 con 0x7f18b8139710 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f18b806dbb0 con 0x7f18b80a51e0 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f18b813a410 con 0x7f18b813a030 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18bd668640 10 monclient(hunting): authenticate will time out at 1186.138730s 2024-08-19T10:42:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b6ffd640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 msgr2=0x7f18b8122a00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b6ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 0x7f18b8122a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18b813a030 0x7f18b81d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18b813a030 0x7f18b81d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34452/0 (socket says 172.21.15.190:34452) 2024-08-19T10:42:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 1 -- 172.21.15.190:0/2978604558 learned_addr learned my addr 172.21.15.190:0/2978604558 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 10 monclient(hunting): get_auth_request con 0x7f18b813a030 auth_method 0 2024-08-19T10:42:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b67fc640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b8139710 0x7f18b8138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b67fc640 10 monclient(hunting): get_auth_request con 0x7f18b8139710 auth_method 0 2024-08-19T10:42:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b67fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b67fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 10 monclient(hunting): handle_auth_done global_id 14749 payload 824 2024-08-19T10:42:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 1 -- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 msgr2=0x7f18b8122a00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 0x7f18b8122a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 1 -- 172.21.15.190:0/2978604558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b8139710 msgr2=0x7f18b8138860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b8139710 0x7f18b8138860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 1 monclient: found mon.noname-i 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.564+0000 7f18b77fe640 1 -- 172.21.15.190:0/2978604558 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b813fb70 con 0x7f18b813a030 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18b77fe640 1 -- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18b813a030 msgr2=0x7f18b81d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18b77fe640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18b813a030 0x7f18b81d1310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f18b000be90 tx=0x7f18b000bf80 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 1 -- 172.21.15.190:0/2978604558 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18b0015850 con 0x7f18b813a030 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 10 monclient: dump: 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:20.584 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:20.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:20.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:20.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 1 -- 172.21.15.190:0/2978604558 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b0010030 con 0x7f18b813a030 2024-08-19T10:42:20.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18bd668640 5 monclient: authenticate success, global_id 14749 2024-08-19T10:42:20.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18bd668640 10 monclient: _renew_subs 2024-08-19T10:42:20.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18bd668640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:20.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18bd668640 1 -- 172.21.15.190:0/2978604558 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b80aa8a0 con 0x7f18b813a030 2024-08-19T10:42:20.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:20.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 1 -- 172.21.15.190:0/2978604558 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18b00060d0 con 0x7f18b813a030 2024-08-19T10:42:20.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:20.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 10 monclient: dump: 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:20.587 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:20.588 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:20.588 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:20.588 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:20.588 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:20.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18b4ff9640 4 set_mon_vals no callback set 2024-08-19T10:42:20.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18bd668640 10 monclient: _renew_subs 2024-08-19T10:42:20.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18bd668640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:20.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18bd668640 1 -- 172.21.15.190:0/2978604558 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b8120e00 con 0x7f18b813a030 2024-08-19T10:42:20.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18bd668640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:20.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18bd668640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:20.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18bd668640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:20.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18bd668640 1 -- 172.21.15.190:0/2978604558 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b8139af0 con 0x7f18b813a030 2024-08-19T10:42:20.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 1 -- 172.21.15.190:0/2978604558 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f18b00153e0 con 0x7f18b813a030 2024-08-19T10:42:20.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f18a403be00 0x7f18a403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:20.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f189f7fe640 1 -- 172.21.15.190:0/2978604558 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(135..135 src has 1..135) ==== 8233+0+0 (secure 0 0 0) 0x7f18b004ff10 con 0x7f18b813a030 2024-08-19T10:42:20.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18b6ffd640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f18a403be00 0x7f18a403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:20.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.568+0000 7f18b6ffd640 10 monclient: get_auth_request con 0x7f18a403be00 auth_method 0 2024-08-19T10:42:20.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.572+0000 7f18b6ffd640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f18a403be00 0x7f18a403e2c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009870 tx=0x7f18a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:20.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.572+0000 7f189f7fe640 1 -- 172.21.15.190:0/2978604558 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f18b001b020 con 0x7f18b813a030 2024-08-19T10:42:20.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.572+0000 7f189f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:20.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.572+0000 7f189f7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:20.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.756+0000 7f18bd668640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-08-19T10:42:20.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.756+0000 7f18bd668640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-08-19T10:42:20.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.756+0000 7f18bd668640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:20.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:20.756+0000 7f18bd668640 1 -- 172.21.15.190:0/2978604558 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]} v 0) -- 0x7f18b8123d20 con 0x7f18b813a030 2024-08-19T10:42:21.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:21.756+0000 7f189f7fe640 1 -- 172.21.15.190:0/2978604558 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}]=0 v136) ==== 139+0+0 (secure 0 0 0) 0x7f18b001ddf0 con 0x7f18b813a030 2024-08-19T10:42:21.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:21.756+0000 7f189f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-08-19T10:42:21.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:21.756+0000 7f189f7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:21.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:21.872+0000 7f18bd668640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-08-19T10:42:21.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:21.872+0000 7f18bd668640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-08-19T10:42:21.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:21.872+0000 7f18bd668640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:21.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:21.872+0000 7f18bd668640 1 -- 172.21.15.190:0/2978604558 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]} v 0) -- 0x7f18b8124260 con 0x7f18b813a030 2024-08-19T10:42:22.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.796+0000 7f189f7fe640 1 -- 172.21.15.190:0/2978604558 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}]=0 v137) ==== 139+0+0 (secure 0 0 0) 0x7f18b004db80 con 0x7f18b813a030 2024-08-19T10:42:22.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.796+0000 7f189f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "barset", "item": "osd.0", "weight": [0.9, 0.9, 0.9]}] 2024-08-19T10:42:22.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.796+0000 7f189f7fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:22.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 -- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f18a403be00 msgr2=0x7f18a403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f18a403be00 0x7f18a403e2c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009870 tx=0x7f18a8008040 comp rx=0 tx=0).stop 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 10 monclient: shutdown 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 -- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18b813a030 msgr2=0x7f18b81d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18b813a030 0x7f18b81d1310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f18b000be90 tx=0x7f18b000bf80 comp rx=0 tx=0).stop 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 -- 172.21.15.190:0/2978604558 shutdown_connections 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f18a403be00 0x7f18a403e2c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18b813a030 0x7f18b81d1310 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f18b8139710 0x7f18b8138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 --2- 172.21.15.190:0/2978604558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18b80a51e0 0x7f18b8122a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 -- 172.21.15.190:0/2978604558 >> 172.21.15.190:0/2978604558 conn(0x7f18b80aa330 msgr2=0x7f18b80a7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:22.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 -- 172.21.15.190:0/2978604558 shutdown_connections 2024-08-19T10:42:22.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.800+0000 7f18bd668640 1 -- 172.21.15.190:0/2978604558 wait complete. 2024-08-19T10:42:22.825 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set ls 2024-08-19T10:42:22.825 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -c fooset 2024-08-19T10:42:22.826 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -q 1 2024-08-19T10:42:22.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.936+0000 7f05bb58a640 10 monclient: get_monmap_and_config 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.936+0000 7f05bb58a640 10 monclient: build_initial_monmap 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.936+0000 7f05bb58a640 10 monclient: monmap: 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:22.942286+0000 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:22.942286+0000 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:22.942 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:22.943 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:22.943 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:22.943 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:22.943 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:22.943 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:22.943 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:22.943 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:22.943 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:22.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.936+0000 7f05bb58a640 1 Processor -- start 2024-08-19T10:42:22.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.936+0000 7f05bb58a640 1 -- start start 2024-08-19T10:42:22.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.936+0000 7f05bb58a640 10 monclient: init 2024-08-19T10:42:22.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:22.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 10 monclient: _add_conns ranks=[9,0,10,6,5,3,2,8,7,1,4] 2024-08-19T10:42:22.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f05b41398c0 0x7f05b4139ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:22.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 10 monclient(hunting): picked mon.noname-a con 0x7f05b41398c0 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:22.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f05b4071200 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:22.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05b40a3d20 0x7f05b4141740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 10 monclient(hunting): picked mon.noname-f con 0x7f05b40a3d20 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b40a1830 con 0x7f05b4071200 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f05b40a1ab0 con 0x7f05b41398c0 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f05b4070460 con 0x7f05b40a3d20 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05bb58a640 10 monclient(hunting): authenticate will time out at 1188.512166s 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05b40a3d20 0x7f05b4141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05b40a3d20 0x7f05b4141740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39018/0 (socket says 172.21.15.190:39018) 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 1 -- 172.21.15.190:0/3817209503 learned_addr learned my addr 172.21.15.190:0/3817209503 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 10 monclient(hunting): get_auth_request con 0x7f05b40a3d20 auth_method 0 2024-08-19T10:42:22.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9326640 1 --2- 172.21.15.190:0/3817209503 >> v2:172.21.15.89:3300/0 conn(0x7f05b41398c0 0x7f05b4139ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9326640 10 monclient(hunting): get_auth_request con 0x7f05b41398c0 auth_method 0 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9326640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9326640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9326640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 10 monclient(hunting): handle_auth_done global_id 14833 payload 274 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 1 -- 172.21.15.190:0/3817209503 >> v2:172.21.15.89:3300/0 conn(0x7f05b41398c0 msgr2=0x7f05b4139ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 1 --2- 172.21.15.190:0/3817209503 >> v2:172.21.15.89:3300/0 conn(0x7f05b41398c0 0x7f05b4139ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:22.947 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 1 -- 172.21.15.190:0/3817209503 >> v1:172.21.15.89:6789/0 conn(0x7f05b4071200 legacy=0x7f05b40a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:22.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 10 monclient: _finish_hunting 0 2024-08-19T10:42:22.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 1 monclient: found mon.noname-f 2024-08-19T10:42:22.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:22.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:22.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.940+0000 7f05b9b27640 1 -- 172.21.15.190:0/3817209503 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b4070740 con 0x7f05b40a3d20 2024-08-19T10:42:22.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05b9b27640 1 -- 172.21.15.190:0/3817209503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05b40a3d20 msgr2=0x7f05b4141740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:22.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05b9b27640 1 --2- 172.21.15.190:0/3817209503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05b40a3d20 0x7f05b4141740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f05b40a5a00 tx=0x7f05ac01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8d56267ff0d6b07 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:22.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 1 -- 172.21.15.190:0/3817209503 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f05ac0090d0 con 0x7f05b40a3d20 2024-08-19T10:42:22.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:22.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:22.950 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 10 monclient: dump: 2024-08-19T10:42:22.950 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:22.950 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:22.950 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:22.950 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 10 monclient: _finish_auth 0 2024-08-19T10:42:22.951 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 1 -- 172.21.15.190:0/3817209503 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05ac0a5ac0 con 0x7f05b40a3d20 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 1 -- 172.21.15.190:0/3817209503 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f05ac0a5c70 con 0x7f05b40a3d20 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05bb58a640 5 monclient: authenticate success, global_id 14833 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05bb58a640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05bb58a640 4 set_mon_vals no callback set 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.944+0000 7f05abfff640 10 monclient: dump: 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:22.952 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:22.953 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:22.954 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:22.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient: shutdown 2024-08-19T10:42:22.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:22.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 -- 172.21.15.190:0/3817209503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05b40a3d20 msgr2=0x7f05b4141740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:22.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 --2- 172.21.15.190:0/3817209503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05b40a3d20 0x7f05b4141740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f05b40a5a00 tx=0x7f05ac01c8b0 comp rx=0 tx=0).stop 2024-08-19T10:42:22.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 -- 172.21.15.190:0/3817209503 shutdown_connections 2024-08-19T10:42:22.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 --2- 172.21.15.190:0/3817209503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05b40a3d20 0x7f05b4141740 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 --2- 172.21.15.190:0/3817209503 >> v2:172.21.15.89:3300/0 conn(0x7f05b41398c0 0x7f05b4139ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 -- 172.21.15.190:0/3817209503 >> 172.21.15.190:0/3817209503 conn(0x7f05b40a4700 msgr2=0x7f05b40a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 -- 172.21.15.190:0/3817209503 shutdown_connections 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 -- 172.21.15.190:0/3817209503 wait complete. 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient: build_initial_monmap 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient: monmap: 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:22.959 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 Processor -- start 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 -- start start 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient: init 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient: _add_conns ranks=[8,2,7,0,3,4,6,1,5] 2024-08-19T10:42:22.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f05b4071200 0x7f05b411e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient(hunting): picked mon.noname-e con 0x7f05b4071200 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b40a3d20 0x7f05b411e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient(hunting): picked mon.noname-b con 0x7f05b40a3d20 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05b41398c0 0x7f05b411ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient(hunting): picked mon.noname-i con 0x7f05b41398c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f05b406fd80 con 0x7f05b40a3d20 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f05b406fc00 con 0x7f05b41398c0 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f05b41444a0 con 0x7f05b4071200 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:22.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05bb58a640 10 monclient(hunting): authenticate will time out at 1188.521858s 2024-08-19T10:42:22.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05b41398c0 0x7f05b411ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:22.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05b41398c0 0x7f05b411ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34464/0 (socket says 172.21.15.190:34464) 2024-08-19T10:42:22.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 1 -- 172.21.15.190:0/3018761855 learned_addr learned my addr 172.21.15.190:0/3018761855 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:22.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 10 monclient(hunting): get_auth_request con 0x7f05b41398c0 auth_method 0 2024-08-19T10:42:22.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:22.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:22.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:22.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:22.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:22.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:22.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b8b25640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b40a3d20 0x7f05b411e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:22.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b8b25640 10 monclient(hunting): get_auth_request con 0x7f05b40a3d20 auth_method 0 2024-08-19T10:42:22.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b8b25640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:22.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b8b25640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:22.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b8b25640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 10 monclient(hunting): handle_auth_done global_id 14758 payload 824 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 1 -- 172.21.15.190:0/3018761855 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f05b4071200 msgr2=0x7f05b411e250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f05b4071200 0x7f05b411e250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 1 -- 172.21.15.190:0/3018761855 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b40a3d20 msgr2=0x7f05b411e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b40a3d20 0x7f05b411e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 10 monclient: _finish_hunting 0 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 1 monclient: found mon.noname-i 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.948+0000 7f05b9b27640 1 -- 172.21.15.190:0/3018761855 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b4121860 con 0x7f05b41398c0 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05b9b27640 1 -- 172.21.15.190:0/3018761855 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05b41398c0 msgr2=0x7f05b411ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05b9b27640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05b41398c0 0x7f05b411ecd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009e20 tx=0x7f05ac019650 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:22.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 1 -- 172.21.15.190:0/3018761855 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f05ac0bb020 con 0x7f05b41398c0 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 10 monclient: dump: 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:22.965 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 1 -- 172.21.15.190:0/3018761855 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05ac0044b0 con 0x7f05b41398c0 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05bb58a640 5 monclient: authenticate success, global_id 14758 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05bb58a640 10 monclient: _renew_subs 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05bb58a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:22.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05bb58a640 1 -- 172.21.15.190:0/3018761855 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b4121b40 con 0x7f05b41398c0 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 1 -- 172.21.15.190:0/3018761855 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f05ac004660 con 0x7f05b41398c0 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05ab7fe640 4 set_mon_vals no callback set 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 10 monclient: dump: 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:22.967 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05bb58a640 10 monclient: _renew_subs 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05bb58a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05bb58a640 1 -- 172.21.15.190:0/3018761855 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b4120cd0 con 0x7f05b41398c0 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05bb58a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05bb58a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:22.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05bb58a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:22.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05bb58a640 1 -- 172.21.15.190:0/3018761855 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b41213f0 con 0x7f05b41398c0 2024-08-19T10:42:22.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 1 -- 172.21.15.190:0/3018761855 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f05ac0be020 con 0x7f05b41398c0 2024-08-19T10:42:22.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f059803be00 0x7f059803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:22.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05a9ffb640 1 -- 172.21.15.190:0/3018761855 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(137..137 src has 1..137) ==== 8233+0+0 (secure 0 0 0) 0x7f05ac0ea7a0 con 0x7f05b41398c0 2024-08-19T10:42:22.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05b9326640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f059803be00 0x7f059803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:22.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.952+0000 7f05b9326640 10 monclient: get_auth_request con 0x7f059803be00 auth_method 0 2024-08-19T10:42:22.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.956+0000 7f05b9326640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f059803be00 0x7f059803e2c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f05a400b770 tx=0x7f05a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:22.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.956+0000 7f05a9ffb640 1 -- 172.21.15.190:0/3018761855 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f05ac0c5030 con 0x7f05b41398c0 2024-08-19T10:42:22.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.956+0000 7f05a9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:22.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:22.956+0000 7f05a9ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:23.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.136+0000 7f05bb58a640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:23.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.136+0000 7f05bb58a640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:23.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.140+0000 7f05bb58a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:23.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.140+0000 7f05bb58a640 1 -- 172.21.15.190:0/3018761855 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f05b406db30 con 0x7f05b41398c0 2024-08-19T10:42:23.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.140+0000 7f05a9ffb640 1 -- 172.21.15.190:0/3018761855 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v137) ==== 71+0+14 (secure 0 0 0) 0x7f05ac0165c0 con 0x7f05b41398c0 2024-08-19T10:42:23.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.140+0000 7f05a9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:23.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.140+0000 7f05a9ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:23.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.244+0000 7f05bb58a640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:23.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.244+0000 7f05bb58a640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:23.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.244+0000 7f05bb58a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:23.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.244+0000 7f05bb58a640 1 -- 172.21.15.190:0/3018761855 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f05b4139ca0 con 0x7f05b41398c0 2024-08-19T10:42:23.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05a9ffb640 1 -- 172.21.15.190:0/3018761855 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v137) ==== 71+0+14 (secure 0 0 0) 0x7f05ac0bfe20 con 0x7f05b41398c0 2024-08-19T10:42:23.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05a9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:23.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05a9ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:23.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 -- 172.21.15.190:0/3018761855 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f059803be00 msgr2=0x7f059803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:23.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f059803be00 0x7f059803e2c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f05a400b770 tx=0x7f05a4008040 comp rx=0 tx=0).stop 2024-08-19T10:42:23.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 10 monclient: shutdown 2024-08-19T10:42:23.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:23.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 -- 172.21.15.190:0/3018761855 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05b41398c0 msgr2=0x7f05b411ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:23.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05b41398c0 0x7f05b411ecd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009e20 tx=0x7f05ac019650 comp rx=0 tx=0).stop 2024-08-19T10:42:23.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 -- 172.21.15.190:0/3018761855 shutdown_connections 2024-08-19T10:42:23.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05b41398c0 0x7f05b411ecd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:23.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f05b40a3d20 0x7f05b411e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:23.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f059803be00 0x7f059803e2c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:23.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 --2- 172.21.15.190:0/3018761855 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f05b4071200 0x7f05b411e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:23.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 -- 172.21.15.190:0/3018761855 >> 172.21.15.190:0/3018761855 conn(0x7f05b40a4700 msgr2=0x7f05b40a6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:23.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 -- 172.21.15.190:0/3018761855 shutdown_connections 2024-08-19T10:42:23.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.248+0000 7f05bb58a640 1 -- 172.21.15.190:0/3018761855 wait complete. 2024-08-19T10:42:23.267 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set rm fooset 2024-08-19T10:42:23.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient: get_monmap_and_config 2024-08-19T10:42:23.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient: build_initial_monmap 2024-08-19T10:42:23.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient: monmap: 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:23.388270+0000 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:23.388270+0000 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:23.388 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:23.389 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:23.389 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:23.389 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:23.389 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:23.389 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:23.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 1 Processor -- start 2024-08-19T10:42:23.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 1 -- start start 2024-08-19T10:42:23.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient: init 2024-08-19T10:42:23.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:23.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient: _add_conns ranks=[5,0,10,7,3,8,2,6,1,9,4] 2024-08-19T10:42:23.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c0139980 0x7f82c0139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:23.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient(hunting): picked mon.noname-h con 0x7f82c0139980 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:23.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f82c013a2a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:23.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f82c013e8c0 0x7f82c0140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:23.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient(hunting): picked mon.noname-f con 0x7f82c013e8c0 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:23.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82c00a8470 con 0x7f82c013a2a0 2024-08-19T10:42:23.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f82c009ca60 con 0x7f82c013e8c0 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f82c00a3640 con 0x7f82c0139980 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c7531640 10 monclient(hunting): authenticate will time out at 1188.959085s 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c0139980 0x7f82c0139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c5ace640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f82c013e8c0 0x7f82c0140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c0139980 0x7f82c0139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36954/0 (socket says 172.21.15.190:36954) 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 1 -- 172.21.15.190:0/1523360325 learned_addr learned my addr 172.21.15.190:0/1523360325 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 10 monclient(hunting): get_auth_request con 0x7f82c0139980 auth_method 0 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:23.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c5ace640 10 monclient(hunting): get_auth_request con 0x7f82c013e8c0 auth_method 0 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c5ace640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c5ace640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c5ace640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c5ace640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c5ace640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c5ace640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 10 monclient(hunting): handle_auth_done global_id 5549 payload 274 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 1 -- 172.21.15.190:0/1523360325 >> v2:172.21.15.190:3300/0 conn(0x7f82c013e8c0 msgr2=0x7f82c0140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:23.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 1 --2- 172.21.15.190:0/1523360325 >> v2:172.21.15.190:3300/0 conn(0x7f82c013e8c0 0x7f82c0140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:23.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 1 -- 172.21.15.190:0/1523360325 >> v1:172.21.15.89:6789/0 conn(0x7f82c013a2a0 legacy=0x7f82c00a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:23.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 10 monclient: _finish_hunting 0 2024-08-19T10:42:23.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 1 monclient: found mon.noname-h 2024-08-19T10:42:23.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:23.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:23.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c52cd640 1 -- 172.21.15.190:0/1523360325 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c00a3850 con 0x7f82c0139980 2024-08-19T10:42:23.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.384+0000 7f82c5ace640 1 --2- 172.21.15.190:0/1523360325 >> v2:172.21.15.190:3300/0 conn(0x7f82c013e8c0 0x7f82c0140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:23.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.388+0000 7f82c52cd640 1 -- 172.21.15.190:0/1523360325 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c0139980 msgr2=0x7f82c0139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:23.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.388+0000 7f82c52cd640 1 --2- 172.21.15.190:0/1523360325 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c0139980 0x7f82c0139d60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009870 tx=0x7f82b001c3f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=e6b6f5e950bd0f24 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:23.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 1 -- 172.21.15.190:0/1523360325 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f82b00b0070 con 0x7f82c0139980 2024-08-19T10:42:23.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:23.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:23.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 10 monclient: dump: 2024-08-19T10:42:23.399 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:23.399 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:23.400 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:23.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:23.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:23.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:23.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 5 monclient: authenticate success, global_id 5549 2024-08-19T10:42:23.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 1 -- 172.21.15.190:0/1523360325 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82b00aa810 con 0x7f82c0139980 2024-08-19T10:42:23.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:23.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:23.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:23.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 4 set_mon_vals no callback set 2024-08-19T10:42:23.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 1 -- 172.21.15.190:0/1523360325 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f82b00aae30 con 0x7f82c0139980 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82b7fff640 10 monclient: dump: 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:23.402 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient: shutdown 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 -- 172.21.15.190:0/1523360325 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c0139980 msgr2=0x7f82c0139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 --2- 172.21.15.190:0/1523360325 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c0139980 0x7f82c0139d60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009870 tx=0x7f82b001c3f0 comp rx=0 tx=0).stop 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 -- 172.21.15.190:0/1523360325 shutdown_connections 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 --2- 172.21.15.190:0/1523360325 >> v2:172.21.15.190:3300/0 conn(0x7f82c013e8c0 0x7f82c0140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 --2- 172.21.15.190:0/1523360325 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c0139980 0x7f82c0139d60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:23.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 -- 172.21.15.190:0/1523360325 >> 172.21.15.190:0/1523360325 conn(0x7f82c00abde0 msgr2=0x7f82c00ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 -- 172.21.15.190:0/1523360325 shutdown_connections 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 -- 172.21.15.190:0/1523360325 wait complete. 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient: build_initial_monmap 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient: monmap: 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:23.404 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 Processor -- start 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 -- start start 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient: init 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient: _add_conns ranks=[2,5,6,0,4,7,1,8,3] 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c0139980 0x7f82c0133b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient(hunting): picked mon.noname-b con 0x7f82c0139980 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:23.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c013e8c0 0x7f82c0137660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient(hunting): picked mon.noname-h con 0x7f82c013e8c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f82c009d1c0 0x7f82c0137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient(hunting): picked mon.noname-d con 0x7f82c009d1c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f82c0143b90 con 0x7f82c0139980 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f82c0143a10 con 0x7f82c013e8c0 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f82c0143d10 con 0x7f82c009d1c0 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c7531640 10 monclient(hunting): authenticate will time out at 1188.966584s 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c013e8c0 0x7f82c0137660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:23.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c013e8c0 0x7f82c0137660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36962/0 (socket says 172.21.15.190:36962) 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 1 -- 172.21.15.190:0/3558645838 learned_addr learned my addr 172.21.15.190:0/3558645838 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 10 monclient(hunting): get_auth_request con 0x7f82c013e8c0 auth_method 0 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c52cd640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c0139980 0x7f82c0133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c5ace640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f82c009d1c0 0x7f82c0137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c52cd640 10 monclient(hunting): get_auth_request con 0x7f82c0139980 auth_method 0 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c52cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:23.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c52cd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c52cd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c5ace640 10 monclient(hunting): get_auth_request con 0x7f82c009d1c0 auth_method 0 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c5ace640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c5ace640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c5ace640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 10 monclient(hunting): handle_auth_done global_id 5558 payload 824 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 1 -- 172.21.15.190:0/3558645838 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f82c009d1c0 msgr2=0x7f82c0137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f82c009d1c0 0x7f82c0137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 1 -- 172.21.15.190:0/3558645838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c0139980 msgr2=0x7f82c0133b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c0139980 0x7f82c0133b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 10 monclient: _finish_hunting 0 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 1 monclient: found mon.noname-h 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:23.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:23.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c4acc640 1 -- 172.21.15.190:0/3558645838 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c0139080 con 0x7f82c013e8c0 2024-08-19T10:42:23.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.392+0000 7f82c52cd640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c0139980 0x7f82c0133b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:42:23.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c4acc640 1 -- 172.21.15.190:0/3558645838 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c013e8c0 msgr2=0x7f82c0137660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:23.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c4acc640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c013e8c0 0x7f82c0137660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c480 tx=0x7f82bc00c940 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:23.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 1 -- 172.21.15.190:0/3558645838 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f82bc012cb0 con 0x7f82c013e8c0 2024-08-19T10:42:23.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:23.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:23.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 10 monclient: dump: 2024-08-19T10:42:23.411 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:23.411 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:23.411 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:23.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 1 -- 172.21.15.190:0/3558645838 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc017070 con 0x7f82c013e8c0 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 1 -- 172.21.15.190:0/3558645838 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f82bc0117f0 con 0x7f82c013e8c0 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 10 monclient: dump: 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:23.413 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c7531640 5 monclient: authenticate success, global_id 5558 2024-08-19T10:42:23.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b77fe640 4 set_mon_vals no callback set 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c7531640 10 monclient: _renew_subs 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c7531640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c7531640 1 -- 172.21.15.190:0/3558645838 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c00ac550 con 0x7f82c013e8c0 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c7531640 10 monclient: _renew_subs 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c7531640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c7531640 1 -- 172.21.15.190:0/3558645838 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c009f3d0 con 0x7f82c013e8c0 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c7531640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c7531640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c7531640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c7531640 1 -- 172.21.15.190:0/3558645838 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c0139d60 con 0x7f82c013e8c0 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 1 -- 172.21.15.190:0/3558645838 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f82bc011380 con 0x7f82c013e8c0 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82a403be00 0x7f82a403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82b5ffb640 1 -- 172.21.15.190:0/3558645838 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(137..137 src has 1..137) ==== 8233+0+0 (secure 0 0 0) 0x7f82bc04f660 con 0x7f82c013e8c0 2024-08-19T10:42:23.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c52cd640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82a403be00 0x7f82a403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:23.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.396+0000 7f82c52cd640 10 monclient: get_auth_request con 0x7f82a403be00 auth_method 0 2024-08-19T10:42:23.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.400+0000 7f82c52cd640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82a403be00 0x7f82a403e2c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009870 tx=0x7f82b00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:23.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.400+0000 7f82b5ffb640 1 -- 172.21.15.190:0/3558645838 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f82bc012ad0 con 0x7f82c013e8c0 2024-08-19T10:42:23.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.400+0000 7f82b5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:23.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.400+0000 7f82b5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.580+0000 7f82c7531640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-08-19T10:42:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.580+0000 7f82c7531640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-08-19T10:42:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.580+0000 7f82c7531640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.580+0000 7f82c7531640 1 -- 172.21.15.190:0/3558645838 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "fooset"} v 0) -- 0x7f82c006c8d0 con 0x7f82c013e8c0 2024-08-19T10:42:23.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.952+0000 7f82b5ffb640 1 -- 172.21.15.190:0/3558645838 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "fooset"}]=0 v138) ==== 89+0+0 (secure 0 0 0) 0x7f82bc04d2d0 con 0x7f82c013e8c0 2024-08-19T10:42:23.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.952+0000 7f82b5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-08-19T10:42:23.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:23.952+0000 7f82b5ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:24.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:24.060+0000 7f82c7531640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-08-19T10:42:24.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:24.060+0000 7f82c7531640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-08-19T10:42:24.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:24.060+0000 7f82c7531640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:24.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:24.060+0000 7f82c7531640 1 -- 172.21.15.190:0/3558645838 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "fooset"} v 0) -- 0x7f82c009e2f0 con 0x7f82c013e8c0 2024-08-19T10:42:25.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.072+0000 7f82b5ffb640 1 -- 172.21.15.190:0/3558645838 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "fooset"}]=0 v139) ==== 89+0+0 (secure 0 0 0) 0x7f82bc011a50 con 0x7f82c013e8c0 2024-08-19T10:42:25.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.072+0000 7f82b5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm", "pool": "fooset"}] 2024-08-19T10:42:25.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.072+0000 7f82b5ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:25.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 1 -- 172.21.15.190:0/3558645838 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82a403be00 msgr2=0x7f82a403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:25.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82a403be00 0x7f82a403e2c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009870 tx=0x7f82b00a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:25.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 10 monclient: shutdown 2024-08-19T10:42:25.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:25.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 1 -- 172.21.15.190:0/3558645838 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c013e8c0 msgr2=0x7f82c0137660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:25.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c013e8c0 0x7f82c0137660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c480 tx=0x7f82bc00c940 comp rx=0 tx=0).stop 2024-08-19T10:42:25.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 1 -- 172.21.15.190:0/3558645838 shutdown_connections 2024-08-19T10:42:25.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f82a403be00 0x7f82a403e2c0 secure :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009870 tx=0x7f82b00a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:25.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f82c009d1c0 0x7f82c0137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82c013e8c0 0x7f82c0137660 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 1 --2- 172.21.15.190:0/3558645838 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82c0139980 0x7f82c0133b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.076+0000 7f82c7531640 1 -- 172.21.15.190:0/3558645838 >> 172.21.15.190:0/3558645838 conn(0x7f82c00abde0 msgr2=0x7f82c013fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:25.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.080+0000 7f82c7531640 1 -- 172.21.15.190:0/3558645838 shutdown_connections 2024-08-19T10:42:25.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.080+0000 7f82c7531640 1 -- 172.21.15.190:0/3558645838 wait complete. 2024-08-19T10:42:25.106 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set ls 2024-08-19T10:42:25.106 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -c fooset 2024-08-19T10:42:25.106 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -q 0 2024-08-19T10:42:25.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient: get_monmap_and_config 2024-08-19T10:42:25.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient: build_initial_monmap 2024-08-19T10:42:25.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient: monmap: 2024-08-19T10:42:25.224 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:25.224 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:25.224986+0000 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:25.224986+0000 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:25.225 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:25.226 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:25.226 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:25.226 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 1 Processor -- start 2024-08-19T10:42:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 1 -- start start 2024-08-19T10:42:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient: init 2024-08-19T10:42:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient: _add_conns ranks=[3,5,6,8,10,7,4,1,0,9,2] 2024-08-19T10:42:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0f4069430 0x7fb0f40a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient(hunting): picked mon.noname-g con 0x7fb0f4069430 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f40a41c0 0x7fb0f40a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient(hunting): picked mon.noname-h con 0x7fb0f40a41c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb0f413aa60 0x7fb0f413ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient(hunting): picked mon.noname-d con 0x7fb0f413aa60 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f409e4e0 con 0x7fb0f4069430 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb0f409db70 con 0x7fb0f40a41c0 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb0f4069810 con 0x7fb0f413aa60 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fc00f640 10 monclient(hunting): authenticate will time out at 1190.794904s 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f9dab640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0f4069430 msgr2=0x7fb0f40a3c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f9dab640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0f4069430 0x7fb0f40a3c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f40a41c0 0x7fb0f40a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f40a41c0 0x7fb0f40a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36978/0 (socket says 172.21.15.190:36978) 2024-08-19T10:42:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 1 -- 172.21.15.190:0/3676593614 learned_addr learned my addr 172.21.15.190:0/3676593614 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 10 monclient(hunting): get_auth_request con 0x7fb0f40a41c0 auth_method 0 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fa5ac640 1 --2- 172.21.15.190:0/3676593614 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb0f413aa60 0x7fb0f413ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fa5ac640 10 monclient(hunting): get_auth_request con 0x7fb0f413aa60 auth_method 0 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fa5ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fa5ac640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0fa5ac640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:25.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 10 monclient(hunting): handle_auth_done global_id 5567 payload 274 2024-08-19T10:42:25.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 1 -- 172.21.15.190:0/3676593614 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb0f413aa60 msgr2=0x7fb0f413ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:25.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 1 --2- 172.21.15.190:0/3676593614 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb0f413aa60 0x7fb0f413ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 1 -- 172.21.15.190:0/3676593614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0f4069430 msgr2=0x7fb0f40a3c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:25.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 1 --2- 172.21.15.190:0/3676593614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0f4069430 0x7fb0f40a3c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 10 monclient: _finish_hunting 0 2024-08-19T10:42:25.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 1 monclient: found mon.noname-h 2024-08-19T10:42:25.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:25.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:25.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.220+0000 7fb0f95aa640 1 -- 172.21.15.190:0/3676593614 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f409e260 con 0x7fb0f40a41c0 2024-08-19T10:42:25.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.224+0000 7fb0f95aa640 1 -- 172.21.15.190:0/3676593614 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f40a41c0 msgr2=0x7fb0f40a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:25.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.224+0000 7fb0f95aa640 1 --2- 172.21.15.190:0/3676593614 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f40a41c0 0x7fb0f40a45e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40098d0 tx=0x7fb0e401c460 comp rx=0 tx=0).ready entity=mon.5 client_cookie=e6b6da28c28187ff server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:25.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 1 -- 172.21.15.190:0/3676593614 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0e40b0070 con 0x7fb0f40a41c0 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 10 monclient: dump: 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:25.236 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 10 monclient: _finish_auth 0 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 10 monclient: _check_auth_tickets 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 1 -- 172.21.15.190:0/3676593614 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e40aa810 con 0x7fb0f40a41c0 2024-08-19T10:42:25.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 5 monclient: authenticate success, global_id 5567 2024-08-19T10:42:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 1 -- 172.21.15.190:0/3676593614 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0e40a9300 con 0x7fb0f40a41c0 2024-08-19T10:42:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 4 set_mon_vals no callback set 2024-08-19T10:42:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f8da9640 10 monclient: dump: 2024-08-19T10:42:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient: shutdown 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 -- 172.21.15.190:0/3676593614 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f40a41c0 msgr2=0x7fb0f40a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 --2- 172.21.15.190:0/3676593614 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f40a41c0 0x7fb0f40a45e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40098d0 tx=0x7fb0e401c460 comp rx=0 tx=0).stop 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 -- 172.21.15.190:0/3676593614 shutdown_connections 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 --2- 172.21.15.190:0/3676593614 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb0f413aa60 0x7fb0f413ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 --2- 172.21.15.190:0/3676593614 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f40a41c0 0x7fb0f40a45e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 --2- 172.21.15.190:0/3676593614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0f4069430 0x7fb0f40a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 -- 172.21.15.190:0/3676593614 >> 172.21.15.190:0/3676593614 conn(0x7fb0f40a4f40 msgr2=0x7fb0f40a5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 -- 172.21.15.190:0/3676593614 shutdown_connections 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 -- 172.21.15.190:0/3676593614 wait complete. 2024-08-19T10:42:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient: build_initial_monmap 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient: monmap: 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 Processor -- start 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 -- start start 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient: init 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient: _add_conns ranks=[0,1,5,3,2,8,6,4,7] 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0f4069430 0x7fb0f41d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient(hunting): picked mon.noname-a con 0x7fb0f4069430 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f40a41c0 0x7fb0f4139e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient(hunting): picked mon.noname-f con 0x7fb0f40a41c0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f413aa60 0x7fb0f413a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient(hunting): picked mon.noname-h con 0x7fb0f413aa60 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40a2650 con 0x7fb0f4069430 2024-08-19T10:42:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40a24d0 con 0x7fb0f40a41c0 2024-08-19T10:42:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb0f409ca60 con 0x7fb0f413aa60 2024-08-19T10:42:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fc00f640 10 monclient(hunting): authenticate will time out at 1190.802361s 2024-08-19T10:42:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f413aa60 0x7fb0f413a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f95aa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f40a41c0 0x7fb0f4139e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f413aa60 0x7fb0f413a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:36986/0 (socket says 172.21.15.190:36986) 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f95aa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f40a41c0 0x7fb0f4139e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39036/0 (socket says 172.21.15.190:39036) 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 1 -- 172.21.15.190:0/423025237 learned_addr learned my addr 172.21.15.190:0/423025237 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 10 monclient(hunting): get_auth_request con 0x7fb0f413aa60 auth_method 0 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f95aa640 10 monclient(hunting): get_auth_request con 0x7fb0f40a41c0 auth_method 0 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f95aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f95aa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f95aa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f9dab640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0f4069430 0x7fb0f41d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f9dab640 10 monclient(hunting): get_auth_request con 0x7fb0f4069430 auth_method 0 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f9dab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f9dab640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f9dab640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f95aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f95aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f95aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f9dab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f9dab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f9dab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 10 monclient(hunting): handle_auth_done global_id 5576 payload 824 2024-08-19T10:42:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 1 -- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f40a41c0 msgr2=0x7fb0f4139e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f40a41c0 0x7fb0f4139e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 1 -- 172.21.15.190:0/423025237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0f4069430 msgr2=0x7fb0f41d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0f4069430 0x7fb0f41d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 10 monclient: _finish_hunting 0 2024-08-19T10:42:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 1 monclient: found mon.noname-h 2024-08-19T10:42:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0fa5ac640 1 -- 172.21.15.190:0/423025237 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f41276a0 con 0x7fb0f413aa60 2024-08-19T10:42:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.228+0000 7fb0f95aa640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f40a41c0 0x7fb0f4139e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fa5ac640 1 -- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f413aa60 msgr2=0x7fb0f413a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fa5ac640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f413aa60 0x7fb0f413a3a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec007d90 tx=0x7fb0ec00e4b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 1 -- 172.21.15.190:0/423025237 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0ec014890 con 0x7fb0f413aa60 2024-08-19T10:42:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 10 monclient: dump: 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 1 -- 172.21.15.190:0/423025237 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ec014a50 con 0x7fb0f413aa60 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fc00f640 5 monclient: authenticate success, global_id 5576 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fc00f640 10 monclient: _renew_subs 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fc00f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fc00f640 1 -- 172.21.15.190:0/423025237 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f409ce20 con 0x7fb0f413aa60 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 1 -- 172.21.15.190:0/423025237 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0ec014c00 con 0x7fb0f413aa60 2024-08-19T10:42:25.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:25.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:25.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 10 monclient: dump: 2024-08-19T10:42:25.251 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:25.251 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:25.251 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.251 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.251 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ebfff640 4 set_mon_vals no callback set 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fc00f640 10 monclient: _renew_subs 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fc00f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fc00f640 1 -- 172.21.15.190:0/423025237 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f4128830 con 0x7fb0f413aa60 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fc00f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fc00f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fc00f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 1 -- 172.21.15.190:0/423025237 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb0ec01b030 con 0x7fb0f413aa60 2024-08-19T10:42:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0fc00f640 1 -- 172.21.15.190:0/423025237 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f40a3c80 con 0x7fb0f413aa60 2024-08-19T10:42:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0d803be00 0x7fb0d803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0ea7fc640 1 -- 172.21.15.190:0/423025237 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(139..139 src has 1..139) ==== 7901+0+0 (secure 0 0 0) 0x7fb0ec019020 con 0x7fb0f413aa60 2024-08-19T10:42:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0f9dab640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0d803be00 0x7fb0d803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.232+0000 7fb0f9dab640 10 monclient: get_auth_request con 0x7fb0d803be00 auth_method 0 2024-08-19T10:42:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.236+0000 7fb0f9dab640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0d803be00 0x7fb0d803e2c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0003990 tx=0x7fb0f0009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.236+0000 7fb0ea7fc640 1 -- 172.21.15.190:0/423025237 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb0ec022060 con 0x7fb0f413aa60 2024-08-19T10:42:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.236+0000 7fb0ea7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.236+0000 7fb0ea7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.416+0000 7fb0fc00f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.416+0000 7fb0fc00f640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.416+0000 7fb0fc00f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.416+0000 7fb0fc00f640 1 -- 172.21.15.190:0/423025237 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7fb0f406e4d0 con 0x7fb0f413aa60 2024-08-19T10:42:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.416+0000 7fb0ea7fc640 1 -- 172.21.15.190:0/423025237 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v139) ==== 71+0+7 (secure 0 0 0) 0x7fb0ec01b320 con 0x7fb0f413aa60 2024-08-19T10:42:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.416+0000 7fb0ea7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.416+0000 7fb0ea7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:25.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:25.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 -- 172.21.15.190:0/423025237 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7fb0f4127030 con 0x7fb0f413aa60 2024-08-19T10:42:25.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0ea7fc640 1 -- 172.21.15.190:0/423025237 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v139) ==== 71+0+7 (secure 0 0 0) 0x7fb0ec014e20 con 0x7fb0f413aa60 2024-08-19T10:42:25.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0ea7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0ea7fc640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:25.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 -- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0d803be00 msgr2=0x7fb0d803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:25.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0d803be00 0x7fb0d803e2c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0003990 tx=0x7fb0f0009110 comp rx=0 tx=0).stop 2024-08-19T10:42:25.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 10 monclient: shutdown 2024-08-19T10:42:25.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:25.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 -- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f413aa60 msgr2=0x7fb0f413a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:25.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f413aa60 0x7fb0f413a3a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec007d90 tx=0x7fb0ec00e4b0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 -- 172.21.15.190:0/423025237 shutdown_connections 2024-08-19T10:42:25.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0d803be00 0x7fb0d803e2c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0f413aa60 0x7fb0f413a3a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0f40a41c0 0x7fb0f4139e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 --2- 172.21.15.190:0/423025237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0f4069430 0x7fb0f41d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 -- 172.21.15.190:0/423025237 >> 172.21.15.190:0/423025237 conn(0x7fb0f40a4f40 msgr2=0x7fb0f40a0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:25.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 -- 172.21.15.190:0/423025237 shutdown_connections 2024-08-19T10:42:25.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.524+0000 7fb0fc00f640 1 -- 172.21.15.190:0/423025237 wait complete. 2024-08-19T10:42:25.547 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set ls 2024-08-19T10:42:25.547 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep barset 2024-08-19T10:42:25.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.656+0000 7fb47122f640 10 monclient: get_monmap_and_config 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.656+0000 7fb47122f640 10 monclient: build_initial_monmap 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.656+0000 7fb47122f640 10 monclient: monmap: 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:25.662768+0000 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:25.662768+0000 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:25.667 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.656+0000 7fb47122f640 1 Processor -- start 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 1 -- start start 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 10 monclient: init 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 10 monclient: _add_conns ranks=[10,1,5,3,6,9,4,7,8,0,2] 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb46c0a6740 0x7fb46c0a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 10 monclient(hunting): picked mon.noname-f con 0x7fb46c0a6740 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb46c0a5bc0 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb46c0a7230 0x7fb46c140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 10 monclient(hunting): picked mon.noname-h con 0x7fb46c0a7230 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb46c0a3740 con 0x7fb46c0a5bc0 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb46c0a4c20 con 0x7fb46c0a6740 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb46c0a3530 con 0x7fb46c0a7230 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb47122f640 10 monclient(hunting): authenticate will time out at 1191.232614s 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46ad9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb46c0a6740 0x7fb46c0a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb46c0a7230 0x7fb46c140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:25.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46ad9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb46c0a6740 0x7fb46c0a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51050/0 (socket says 172.21.15.190:51050) 2024-08-19T10:42:25.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46ad9d640 1 -- 172.21.15.190:0/3855597799 learned_addr learned my addr 172.21.15.190:0/3855597799 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:25.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46ad9d640 10 monclient(hunting): get_auth_request con 0x7fb46c0a6740 auth_method 0 2024-08-19T10:42:25.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:25.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:25.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:25.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 10 monclient(hunting): get_auth_request con 0x7fb46c0a7230 auth_method 0 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 10 monclient(hunting): handle_auth_done global_id 5585 payload 274 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 1 -- 172.21.15.190:0/3855597799 >> v2:172.21.15.190:3300/0 conn(0x7fb46c0a6740 msgr2=0x7fb46c0a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 1 --2- 172.21.15.190:0/3855597799 >> v2:172.21.15.190:3300/0 conn(0x7fb46c0a6740 0x7fb46c0a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 1 -- 172.21.15.190:0/3855597799 >> v1:172.21.15.190:6789/0 conn(0x7fb46c0a5bc0 legacy=0x7fb46c0a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:25.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 10 monclient: _finish_hunting 0 2024-08-19T10:42:25.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 1 monclient: found mon.noname-h 2024-08-19T10:42:25.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:25.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:25.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.660+0000 7fb46b59e640 1 -- 172.21.15.190:0/3855597799 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c0a3240 con 0x7fb46c0a7230 2024-08-19T10:42:25.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb46b59e640 1 -- 172.21.15.190:0/3855597799 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb46c0a7230 msgr2=0x7fb46c140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:25.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb46b59e640 1 --2- 172.21.15.190:0/3855597799 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb46c0a7230 0x7fb46c140ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb464009d90 tx=0x7fb46401c830 comp rx=0 tx=0).ready entity=mon.5 client_cookie=99f199d1218f1483 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:25.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 1 -- 172.21.15.190:0/3855597799 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4640b2070 con 0x7fb46c0a7230 2024-08-19T10:42:25.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:25.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:25.677 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 10 monclient: dump: 2024-08-19T10:42:25.677 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:25.677 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:25.677 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:25.678 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 10 monclient: _finish_auth 0 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 1 -- 172.21.15.190:0/3855597799 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4640a5d10 con 0x7fb46c0a7230 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 5 monclient: authenticate success, global_id 5585 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 1 -- 172.21.15.190:0/3855597799 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4640ab7b0 con 0x7fb46c0a7230 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 4 set_mon_vals no callback set 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb469d9b640 10 monclient: dump: 2024-08-19T10:42:25.679 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:25.680 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 10 monclient: shutdown 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 1 -- 172.21.15.190:0/3855597799 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb46c0a7230 msgr2=0x7fb46c140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 1 --2- 172.21.15.190:0/3855597799 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb46c0a7230 0x7fb46c140ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb464009d90 tx=0x7fb46401c830 comp rx=0 tx=0).stop 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 1 -- 172.21.15.190:0/3855597799 shutdown_connections 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 1 --2- 172.21.15.190:0/3855597799 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb46c0a7230 0x7fb46c140ab0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 1 --2- 172.21.15.190:0/3855597799 >> v2:172.21.15.190:3300/0 conn(0x7fb46c0a6740 0x7fb46c0a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 1 -- 172.21.15.190:0/3855597799 >> 172.21.15.190:0/3855597799 conn(0x7fb46c0abd40 msgr2=0x7fb46c0ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 1 -- 172.21.15.190:0/3855597799 shutdown_connections 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 1 -- 172.21.15.190:0/3855597799 wait complete. 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 10 monclient: build_initial_monmap 2024-08-19T10:42:25.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 10 monclient: monmap: 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:25.682 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.664+0000 7fb47122f640 1 Processor -- start 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 1 -- start start 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient: init 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient: _add_conns ranks=[3,7,4,6,1,2,8,5,0] 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb46c0a6740 0x7fb46c1d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient(hunting): picked mon.noname-g con 0x7fb46c0a6740 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb46c0a7230 0x7fb46c1d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient(hunting): picked mon.noname-i con 0x7fb46c0a7230 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:25.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c0a1aa0 0x7fb46c1d1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient(hunting): picked mon.noname-c con 0x7fb46c0a1aa0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb46c143990 con 0x7fb46c0a6740 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb46c143810 con 0x7fb46c0a1aa0 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb46c143b10 con 0x7fb46c0a7230 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46ad9d640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb46c0a6740 msgr2=0x7fb46c1d0ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient(hunting): authenticate will time out at 1191.240533s 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46ad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb46c0a6740 0x7fb46c1d0ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb46c0a7230 0x7fb46c1d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:25.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb46c0a7230 0x7fb46c1d0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:35858/0 (socket says 172.21.15.190:35858) 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 1 -- 172.21.15.190:0/2459802235 learned_addr learned my addr 172.21.15.190:0/2459802235 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 10 monclient(hunting): get_auth_request con 0x7fb46c0a7230 auth_method 0 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 10 monclient(hunting): handle_auth_done global_id 14767 payload 824 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 1 -- 172.21.15.190:0/2459802235 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c0a1aa0 msgr2=0x7fb46c1d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c0a1aa0 0x7fb46c1d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 1 -- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb46c0a6740 msgr2=0x7fb46c1d0ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb46c0a6740 0x7fb46c1d0ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 1 monclient: found mon.noname-i 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 1 -- 172.21.15.190:0/2459802235 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c140d10 con 0x7fb46c0a7230 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 1 -- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb46c0a7230 msgr2=0x7fb46c1d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46a59c640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb46c0a7230 0x7fb46c1d0ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c3d0 tx=0x7fb46000c890 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 1 -- 172.21.15.190:0/2459802235 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb460011c10 con 0x7fb46c0a7230 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 10 monclient: dump: 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:25.686 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:25.687 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 5 monclient: authenticate success, global_id 14767 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 1 -- 172.21.15.190:0/2459802235 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb460017070 con 0x7fb46c0a7230 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 1 -- 172.21.15.190:0/2459802235 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb460011df0 con 0x7fb46c0a7230 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb46959a640 4 set_mon_vals no callback set 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb4537fe640 10 monclient: dump: 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:25.688 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient: _renew_subs 2024-08-19T10:42:25.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 1 -- 172.21.15.190:0/2459802235 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c13cec0 con 0x7fb46c0a7230 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient: _renew_subs 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.668+0000 7fb47122f640 1 -- 172.21.15.190:0/2459802235 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c0a1490 con 0x7fb46c0a7230 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.672+0000 7fb47122f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.672+0000 7fb47122f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.672+0000 7fb47122f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.672+0000 7fb47122f640 1 -- 172.21.15.190:0/2459802235 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46c0a5680 con 0x7fb46c0a7230 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.672+0000 7fb4537fe640 1 -- 172.21.15.190:0/2459802235 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb4600117a0 con 0x7fb46c0a7230 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.672+0000 7fb4537fe640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb45803be00 0x7fb45803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.672+0000 7fb4537fe640 1 -- 172.21.15.190:0/2459802235 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(139..139 src has 1..139) ==== 7901+0+0 (secure 0 0 0) 0x7fb46004f5b0 con 0x7fb46c0a7230 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.672+0000 7fb46ad9d640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb45803be00 0x7fb45803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:25.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.672+0000 7fb46ad9d640 10 monclient: get_auth_request con 0x7fb45803be00 auth_method 0 2024-08-19T10:42:25.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.672+0000 7fb4537fe640 1 -- 172.21.15.190:0/2459802235 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:42:25.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.672+0000 7fb46ad9d640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb45803be00 0x7fb45803e2c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb45c006c30 tx=0x7fb45c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:25.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.676+0000 7fb4537fe640 1 -- 172.21.15.190:0/2459802235 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb46c140d10 con 0x7fb46c0a7230 2024-08-19T10:42:25.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.676+0000 7fb4537fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:25.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.676+0000 7fb4537fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:25.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.852+0000 7fb47122f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.852+0000 7fb47122f640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.852+0000 7fb47122f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:25.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.852+0000 7fb47122f640 1 -- 172.21.15.190:0/2459802235 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7fb46c13cca0 con 0x7fb46c0a7230 2024-08-19T10:42:25.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.852+0000 7fb4537fe640 1 -- 172.21.15.190:0/2459802235 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v139) ==== 71+0+7 (secure 0 0 0) 0x7fb46001ac60 con 0x7fb46c0a7230 2024-08-19T10:42:25.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.852+0000 7fb4537fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.852+0000 7fb4537fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:25.953 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.948+0000 7fb47122f640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.948+0000 7fb47122f640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.948+0000 7fb47122f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:25.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.948+0000 7fb47122f640 1 -- 172.21.15.190:0/2459802235 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7fb46c0008e0 con 0x7fb46c0a7230 2024-08-19T10:42:25.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.948+0000 7fb4537fe640 1 -- 172.21.15.190:0/2459802235 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v139) ==== 71+0+7 (secure 0 0 0) 0x7fb460010cf0 con 0x7fb46c0a7230 2024-08-19T10:42:25.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.948+0000 7fb4537fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:25.954 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.948+0000 7fb4537fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 -- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb45803be00 msgr2=0x7fb45803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb45803be00 0x7fb45803e2c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb45c006c30 tx=0x7fb45c009240 comp rx=0 tx=0).stop 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 10 monclient: shutdown 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 -- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb46c0a7230 msgr2=0x7fb46c1d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb46c0a7230 0x7fb46c1d0ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c3d0 tx=0x7fb46000c890 comp rx=0 tx=0).stop 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 -- 172.21.15.190:0/2459802235 shutdown_connections 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb45803be00 0x7fb45803e2c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb46c0a1aa0 0x7fb46c1d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb46c0a7230 0x7fb46c1d0ff0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 --2- 172.21.15.190:0/2459802235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb46c0a6740 0x7fb46c1d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:25.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 -- 172.21.15.190:0/2459802235 >> 172.21.15.190:0/2459802235 conn(0x7fb46c0abd40 msgr2=0x7fb46c0ab2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 -- 172.21.15.190:0/2459802235 shutdown_connections 2024-08-19T10:42:25.958 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:25.952+0000 7fb47122f640 1 -- 172.21.15.190:0/2459802235 wait complete. 2024-08-19T10:42:25.971 INFO:tasks.workunit.client.0.smithi190.stdout:barset 2024-08-19T10:42:25.971 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set rm barset 2024-08-19T10:42:26.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.084+0000 7f826c445640 10 monclient: get_monmap_and_config 2024-08-19T10:42:26.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.084+0000 7f826c445640 10 monclient: build_initial_monmap 2024-08-19T10:42:26.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.084+0000 7f826c445640 10 monclient: monmap: 2024-08-19T10:42:26.090 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:26.091102+0000 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:26.091102+0000 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:26.091 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:26.092 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:26.092 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:26.092 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:26.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 1 Processor -- start 2024-08-19T10:42:26.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 1 -- start start 2024-08-19T10:42:26.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 10 monclient: init 2024-08-19T10:42:26.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:26.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 10 monclient: _add_conns ranks=[8,6,3,0,10,1,4,2,9,5,7] 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8264139980 0x7f8264139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 10 monclient(hunting): picked mon.noname-e con 0x7f8264139980 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f826413a2a0 0x7f82640a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 10 monclient(hunting): picked mon.noname-d con 0x7f826413a2a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82640a5c70 0x7f82640a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 10 monclient(hunting): picked mon.noname-g con 0x7f82640a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f826409ca60 con 0x7f82640a5c70 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f82640a3640 con 0x7f826413a2a0 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:26.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f82640a8470 con 0x7f8264139980 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826c445640 10 monclient(hunting): authenticate will time out at 1191.660758s 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826a9e2640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82640a5c70 msgr2=0x7f82640a67e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826a9e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82640a5c70 0x7f82640a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f826413a2a0 0x7f82640a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826a1e1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8264139980 0x7f8264139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f826413a2a0 0x7f82640a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:52462/0 (socket says 172.21.15.190:52462) 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 1 -- 172.21.15.190:0/2649609439 learned_addr learned my addr 172.21.15.190:0/2649609439 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 10 monclient(hunting): get_auth_request con 0x7f826413a2a0 auth_method 0 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:26.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826a1e1640 10 monclient(hunting): get_auth_request con 0x7f8264139980 auth_method 0 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826a1e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826a1e1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826a1e1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826a1e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826a1e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f826a1e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 10 monclient(hunting): handle_auth_done global_id 5379 payload 274 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 1 -- 172.21.15.190:0/2649609439 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8264139980 msgr2=0x7f8264139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 1 --2- 172.21.15.190:0/2649609439 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8264139980 0x7f8264139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 1 -- 172.21.15.190:0/2649609439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82640a5c70 msgr2=0x7f82640a67e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:26.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 1 --2- 172.21.15.190:0/2649609439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82640a5c70 0x7f82640a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:26.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 10 monclient: _finish_hunting 0 2024-08-19T10:42:26.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 1 monclient: found mon.noname-d 2024-08-19T10:42:26.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:26.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:26.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.088+0000 7f82699e0640 1 -- 172.21.15.190:0/2649609439 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82640a3850 con 0x7f826413a2a0 2024-08-19T10:42:26.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82699e0640 1 -- 172.21.15.190:0/2649609439 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f826413a2a0 msgr2=0x7f82640a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:26.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82699e0640 1 --2- 172.21.15.190:0/2649609439 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f826413a2a0 0x7f82640a5730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f82540098d0 tx=0x7f825401c170 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c2dd4cef1259e275 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:26.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 1 -- 172.21.15.190:0/2649609439 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f82540b0070 con 0x7f826413a2a0 2024-08-19T10:42:26.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:26.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 10 monclient: dump: 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:26.102 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:26.103 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:26.103 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:26.103 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:26.103 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:26.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 10 monclient: _finish_auth 0 2024-08-19T10:42:26.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 10 monclient: _check_auth_tickets 2024-08-19T10:42:26.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:26.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 1 -- 172.21.15.190:0/2649609439 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82540aa810 con 0x7f826413a2a0 2024-08-19T10:42:26.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 5 monclient: authenticate success, global_id 5379 2024-08-19T10:42:26.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:26.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:26.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:26.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 4 set_mon_vals no callback set 2024-08-19T10:42:26.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 1 -- 172.21.15.190:0/2649609439 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f82540aae30 con 0x7f826413a2a0 2024-08-19T10:42:26.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:26.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f82691df640 10 monclient: dump: 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:26.105 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:26.106 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:26.106 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:26.106 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:26.106 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:26.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 10 monclient: shutdown 2024-08-19T10:42:26.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:26.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 1 -- 172.21.15.190:0/2649609439 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f826413a2a0 msgr2=0x7f82640a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:26.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 1 --2- 172.21.15.190:0/2649609439 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f826413a2a0 0x7f82640a5730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f82540098d0 tx=0x7f825401c170 comp rx=0 tx=0).stop 2024-08-19T10:42:26.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 1 -- 172.21.15.190:0/2649609439 shutdown_connections 2024-08-19T10:42:26.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 1 --2- 172.21.15.190:0/2649609439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82640a5c70 0x7f82640a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:26.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 1 --2- 172.21.15.190:0/2649609439 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f826413a2a0 0x7f82640a5730 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 1 --2- 172.21.15.190:0/2649609439 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8264139980 0x7f8264139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.092+0000 7f826c445640 1 -- 172.21.15.190:0/2649609439 >> 172.21.15.190:0/2649609439 conn(0x7f82640abde0 msgr2=0x7f82640ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 1 -- 172.21.15.190:0/2649609439 shutdown_connections 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 1 -- 172.21.15.190:0/2649609439 wait complete. 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient: build_initial_monmap 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient: monmap: 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:26.107 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 1 Processor -- start 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 1 -- start start 2024-08-19T10:42:26.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient: init 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient: _add_conns ranks=[0,3,1,7,5,8,2,6,4] 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82640a5c70 0x7f82640a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient(hunting): picked mon.noname-a con 0x7f82640a5c70 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8264139980 0x7f82641d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient(hunting): picked mon.noname-g con 0x7f8264139980 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82641d1f60 0x7f826413a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient(hunting): picked mon.noname-f con 0x7f82641d1f60 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8264143270 con 0x7f82640a5c70 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f82641430f0 con 0x7f82641d1f60 2024-08-19T10:42:26.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f82641433f0 con 0x7f8264139980 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826c445640 10 monclient(hunting): authenticate will time out at 1191.669529s 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f82699e0640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8264139980 msgr2=0x7f82641d1a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f82699e0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8264139980 0x7f82641d1a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82641d1f60 0x7f826413a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82641d1f60 0x7f826413a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51062/0 (socket says 172.21.15.190:51062) 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 1 -- 172.21.15.190:0/2723270686 learned_addr learned my addr 172.21.15.190:0/2723270686 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 10 monclient(hunting): get_auth_request con 0x7f82641d1f60 auth_method 0 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:26.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a1e1640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82640a5c70 0x7f82640a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a1e1640 10 monclient(hunting): get_auth_request con 0x7f82640a5c70 auth_method 0 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a1e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a1e1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a1e1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 10 monclient(hunting): handle_auth_done global_id 14869 payload 824 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 1 -- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8264139980 msgr2=0x7f82641d1a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8264139980 0x7f82641d1a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 1 -- 172.21.15.190:0/2723270686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82640a5c70 msgr2=0x7f82640a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82640a5c70 0x7f82640a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:26.111 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 10 monclient: _finish_hunting 0 2024-08-19T10:42:26.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 1 monclient: found mon.noname-f 2024-08-19T10:42:26.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:26.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:26.112 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.096+0000 7f826a9e2640 1 -- 172.21.15.190:0/2723270686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82640a1d10 con 0x7f82641d1f60 2024-08-19T10:42:26.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826a9e2640 1 -- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82641d1f60 msgr2=0x7f826413a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:26.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826a9e2640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82641d1f60 0x7f826413a690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f825c00bdd0 tx=0x7f825c009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:26.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 1 -- 172.21.15.190:0/2723270686 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:42:26.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 1 -- 172.21.15.190:0/2723270686 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f825c00ce00 con 0x7f82641d1f60 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 10 monclient: dump: 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:26.116 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 10 monclient: _finish_auth 0 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 1 -- 172.21.15.190:0/2723270686 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825c010030 con 0x7f82641d1f60 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 1 -- 172.21.15.190:0/2723270686 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f825c015cc0 con 0x7f82641d1f60 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:26.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 10 monclient: dump: 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:26.118 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826c445640 5 monclient: authenticate success, global_id 14869 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826c445640 10 monclient: _renew_subs 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825bfff640 4 set_mon_vals no callback set 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826c445640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826c445640 1 -- 172.21.15.190:0/2723270686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8264135780 con 0x7f82641d1f60 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826c445640 10 monclient: _renew_subs 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826c445640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826c445640 1 -- 172.21.15.190:0/2723270686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826413d160 con 0x7f82641d1f60 2024-08-19T10:42:26.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826c445640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826c445640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826c445640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826c445640 1 -- 172.21.15.190:0/2723270686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8264139d60 con 0x7f82641d1f60 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 1 -- 172.21.15.190:0/2723270686 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f825c0153e0 con 0x7f82641d1f60 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f824803be00 0x7f824803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f825affd640 1 -- 172.21.15.190:0/2723270686 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 7901+0+0 (secure 0 0 0) 0x7f825c04f9f0 con 0x7f82641d1f60 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826a1e1640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f824803be00 0x7f824803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.100+0000 7f826a1e1640 10 monclient: get_auth_request con 0x7f824803be00 auth_method 0 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.104+0000 7f826a1e1640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f824803be00 0x7f824803e2c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8260009af0 tx=0x7f8260008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.108+0000 7f825affd640 1 -- 172.21.15.190:0/2723270686 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8264135780 con 0x7f82641d1f60 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.108+0000 7f825affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:26.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.108+0000 7f825affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:26.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.268+0000 7f826c445640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-08-19T10:42:26.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.268+0000 7f826c445640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-08-19T10:42:26.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.268+0000 7f826c445640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:26.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:26.268+0000 7f826c445640 1 -- 172.21.15.190:0/2723270686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "barset"} v 0) -- 0x7f82641d23a0 con 0x7f82641d1f60 2024-08-19T10:42:27.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:27.092+0000 7f825affd640 1 -- 172.21.15.190:0/2723270686 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "barset"}]=0 v140) ==== 89+0+0 (secure 0 0 0) 0x7f825c01ec20 con 0x7f82641d1f60 2024-08-19T10:42:27.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:27.092+0000 7f825affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-08-19T10:42:27.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:27.092+0000 7f825affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:27.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:27.196+0000 7f826c445640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-08-19T10:42:27.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:27.196+0000 7f826c445640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-08-19T10:42:27.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:27.196+0000 7f826c445640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:27.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:27.196+0000 7f826c445640 1 -- 172.21.15.190:0/2723270686 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set rm", "pool": "barset"} v 0) -- 0x7f826413bc40 con 0x7f82641d1f60 2024-08-19T10:42:28.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.092+0000 7f825affd640 1 -- 172.21.15.190:0/2723270686 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm", "pool": "barset"}]=0 v141) ==== 89+0+0 (secure 0 0 0) 0x7f825c01ee00 con 0x7f82641d1f60 2024-08-19T10:42:28.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f825affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm", "pool": "barset"}] 2024-08-19T10:42:28.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f825affd640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:28.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 1 -- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f824803be00 msgr2=0x7f824803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:28.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f824803be00 0x7f824803e2c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8260009af0 tx=0x7f8260008040 comp rx=0 tx=0).stop 2024-08-19T10:42:28.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 10 monclient: shutdown 2024-08-19T10:42:28.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:28.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 1 -- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82641d1f60 msgr2=0x7f826413a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:28.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82641d1f60 0x7f826413a690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f825c00bdd0 tx=0x7f825c009ed0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 1 -- 172.21.15.190:0/2723270686 shutdown_connections 2024-08-19T10:42:28.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f824803be00 0x7f824803e2c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82641d1f60 0x7f826413a690 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8264139980 0x7f82641d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 1 --2- 172.21.15.190:0/2723270686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82640a5c70 0x7f82640a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.096+0000 7f826c445640 1 -- 172.21.15.190:0/2723270686 >> 172.21.15.190:0/2723270686 conn(0x7f82640abde0 msgr2=0x7f82640a91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:28.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.100+0000 7f826c445640 1 -- 172.21.15.190:0/2723270686 shutdown_connections 2024-08-19T10:42:28.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.100+0000 7f826c445640 1 -- 172.21.15.190:0/2723270686 wait complete. 2024-08-19T10:42:28.128 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set ls 2024-08-19T10:42:28.128 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -c barset 2024-08-19T10:42:28.128 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep -q 0 2024-08-19T10:42:28.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient: get_monmap_and_config 2024-08-19T10:42:28.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient: build_initial_monmap 2024-08-19T10:42:28.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient: monmap: 2024-08-19T10:42:28.242 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:28.242 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:28.242 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:28.242056+0000 2024-08-19T10:42:28.242 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:28.242056+0000 2024-08-19T10:42:28.242 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:28.242 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.242 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:28.242 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:28.242 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:28.242 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:28.243 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:28.243 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:28.243 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:28.243 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:28.243 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:28.243 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:28.243 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:28.243 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 1 Processor -- start 2024-08-19T10:42:28.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 1 -- start start 2024-08-19T10:42:28.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient: init 2024-08-19T10:42:28.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:28.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient: _add_conns ranks=[9,4,7,2,3,0,6,8,1,10,5] 2024-08-19T10:42:28.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f85d0139980 0x7f85d0139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient(hunting): picked mon.noname-a con 0x7f85d0139980 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:28.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85d013a2a0 0x7f85d00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient(hunting): picked mon.noname-c con 0x7f85d013a2a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:28.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a5c70 0x7f85d00a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient(hunting): picked mon.noname-i con 0x7f85d00a5c70 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:28.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:28.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f85d009ca60 con 0x7f85d0139980 2024-08-19T10:42:28.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:28.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f85d00a3640 con 0x7f85d013a2a0 2024-08-19T10:42:28.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:28.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f85d00a8470 con 0x7f85d00a5c70 2024-08-19T10:42:28.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:28.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.236+0000 7f85d6242640 10 monclient(hunting): authenticate will time out at 1193.811866s 2024-08-19T10:42:28.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a5c70 0x7f85d00a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a5c70 0x7f85d00a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:35870/0 (socket says 172.21.15.190:35870) 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 1 -- 172.21.15.190:0/2845395742 learned_addr learned my addr 172.21.15.190:0/2845395742 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 10 monclient(hunting): get_auth_request con 0x7f85d00a5c70 auth_method 0 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85ceffd640 1 --2- 172.21.15.190:0/2845395742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85d013a2a0 0x7f85d00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cf7fe640 1 --2- 172.21.15.190:0/2845395742 >> v2:172.21.15.89:3300/0 conn(0x7f85d0139980 0x7f85d0139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:28.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85ceffd640 10 monclient(hunting): get_auth_request con 0x7f85d013a2a0 auth_method 0 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85ceffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cf7fe640 10 monclient(hunting): get_auth_request con 0x7f85d0139980 auth_method 0 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 10 monclient(hunting): handle_auth_done global_id 14776 payload 274 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 1 -- 172.21.15.190:0/2845395742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85d013a2a0 msgr2=0x7f85d00a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 1 --2- 172.21.15.190:0/2845395742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85d013a2a0 0x7f85d00a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 1 -- 172.21.15.190:0/2845395742 >> v2:172.21.15.89:3300/0 conn(0x7f85d0139980 msgr2=0x7f85d0139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 1 --2- 172.21.15.190:0/2845395742 >> v2:172.21.15.89:3300/0 conn(0x7f85d0139980 0x7f85d0139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:28.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 1 monclient: found mon.noname-i 2024-08-19T10:42:28.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:28.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:28.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.240+0000 7f85cffff640 1 -- 172.21.15.190:0/2845395742 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d00a3850 con 0x7f85d00a5c70 2024-08-19T10:42:28.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cffff640 1 -- 172.21.15.190:0/2845395742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a5c70 msgr2=0x7f85d00a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:28.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cffff640 1 --2- 172.21.15.190:0/2845395742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a5c70 0x7f85d00a67e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009e80 tx=0x7f85c801c610 comp rx=0 tx=0).ready entity=mon.7 client_cookie=dacf13c036faa814 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:28.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 1 -- 172.21.15.190:0/2845395742 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f85c80090d0 con 0x7f85d00a5c70 2024-08-19T10:42:28.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:28.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:28.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 10 monclient: dump: 2024-08-19T10:42:28.259 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:28.259 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:28.259 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.259 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.259 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:28.259 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.259 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:28.259 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 5 monclient: authenticate success, global_id 14776 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:28.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 1 -- 172.21.15.190:0/2845395742 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c80ae070 con 0x7f85d00a5c70 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 4 set_mon_vals no callback set 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 1 -- 172.21.15.190:0/2845395742 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f85c80a9d50 con 0x7f85d00a5c70 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ce7fc640 10 monclient: dump: 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:28.261 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient: shutdown 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:28.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 -- 172.21.15.190:0/2845395742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a5c70 msgr2=0x7f85d00a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 --2- 172.21.15.190:0/2845395742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a5c70 0x7f85d00a67e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009e80 tx=0x7f85c801c610 comp rx=0 tx=0).stop 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 -- 172.21.15.190:0/2845395742 shutdown_connections 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 --2- 172.21.15.190:0/2845395742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f85d00a5c70 0x7f85d00a67e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 --2- 172.21.15.190:0/2845395742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85d013a2a0 0x7f85d00a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 --2- 172.21.15.190:0/2845395742 >> v2:172.21.15.89:3300/0 conn(0x7f85d0139980 0x7f85d0139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 -- 172.21.15.190:0/2845395742 >> 172.21.15.190:0/2845395742 conn(0x7f85d00abde0 msgr2=0x7f85d00ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 -- 172.21.15.190:0/2845395742 shutdown_connections 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 -- 172.21.15.190:0/2845395742 wait complete. 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient: build_initial_monmap 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient: monmap: 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:28.263 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:28.264 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:28.264 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.264 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:28.264 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:28.264 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:28.264 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:28.264 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:28.264 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 Processor -- start 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 -- start start 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient: init 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient: _add_conns ranks=[1,4,6,0,5,3,7,2,8] 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d00a5c70 0x7f85d013a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient(hunting): picked mon.noname-f con 0x7f85d00a5c70 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85d0139980 0x7f85d013abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient(hunting): picked mon.noname-c con 0x7f85d0139980 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:28.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f85d013bb60 0x7f85d013b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient(hunting): picked mon.noname-d con 0x7f85d013bb60 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:28.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:28.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85d0143270 con 0x7f85d00a5c70 2024-08-19T10:42:28.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:28.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f85d01430f0 con 0x7f85d0139980 2024-08-19T10:42:28.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:28.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f85d01433f0 con 0x7f85d013bb60 2024-08-19T10:42:28.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:28.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85d6242640 10 monclient(hunting): authenticate will time out at 1193.819270s 2024-08-19T10:42:28.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d00a5c70 0x7f85d013a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:28.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d00a5c70 0x7f85d013a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51074/0 (socket says 172.21.15.190:51074) 2024-08-19T10:42:28.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 1 -- 172.21.15.190:0/2398843146 learned_addr learned my addr 172.21.15.190:0/2398843146 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:28.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 10 monclient(hunting): get_auth_request con 0x7f85d00a5c70 auth_method 0 2024-08-19T10:42:28.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:28.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:28.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:28.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:28.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:28.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:28.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ceffd640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85d0139980 0x7f85d013abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:28.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cffff640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f85d013bb60 0x7f85d013b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:28.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ceffd640 10 monclient(hunting): get_auth_request con 0x7f85d0139980 auth_method 0 2024-08-19T10:42:28.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:28.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ceffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cffff640 10 monclient(hunting): get_auth_request con 0x7f85d013bb60 auth_method 0 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 10 monclient(hunting): handle_auth_done global_id 14878 payload 824 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 1 -- 172.21.15.190:0/2398843146 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f85d013bb60 msgr2=0x7f85d013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f85d013bb60 0x7f85d013b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 1 -- 172.21.15.190:0/2398843146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85d0139980 msgr2=0x7f85d013abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85d0139980 0x7f85d013abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 1 monclient: found mon.noname-f 2024-08-19T10:42:28.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:28.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:28.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.244+0000 7f85cf7fe640 1 -- 172.21.15.190:0/2398843146 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d00776f0 con 0x7f85d00a5c70 2024-08-19T10:42:28.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85cf7fe640 1 -- 172.21.15.190:0/2398843146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d00a5c70 msgr2=0x7f85d013a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:28.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85cf7fe640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d00a5c70 0x7f85d013a690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f85c000d800 tx=0x7f85c000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:28.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 1 -- 172.21.15.190:0/2398843146 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f85c000bd40 con 0x7f85d00a5c70 2024-08-19T10:42:28.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:28.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:28.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 10 monclient: dump: 2024-08-19T10:42:28.270 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:28.271 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 1 -- 172.21.15.190:0/2398843146 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c0019070 con 0x7f85d00a5c70 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85d6242640 5 monclient: authenticate success, global_id 14878 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85d6242640 10 monclient: _renew_subs 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85d6242640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85d6242640 1 -- 172.21.15.190:0/2398843146 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d00a1bf0 con 0x7f85d00a5c70 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:28.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 1 -- 172.21.15.190:0/2398843146 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f85c00148a0 con 0x7f85d00a5c70 2024-08-19T10:42:28.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:28.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:28.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 10 monclient: dump: 2024-08-19T10:42:28.273 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:28.273 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:28.273 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.273 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.273 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:28.273 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.274 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:28.274 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:28.274 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:28.274 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:28.274 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:28.274 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:28.274 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:28.275 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:28.275 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:28.275 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85cdffb640 4 set_mon_vals no callback set 2024-08-19T10:42:28.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85d6242640 10 monclient: _renew_subs 2024-08-19T10:42:28.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85d6242640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:28.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85d6242640 1 -- 172.21.15.190:0/2398843146 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d013cac0 con 0x7f85d00a5c70 2024-08-19T10:42:28.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85d6242640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:28.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85d6242640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:28.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85d6242640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:28.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 1 -- 172.21.15.190:0/2398843146 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f85c0014430 con 0x7f85d00a5c70 2024-08-19T10:42:28.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85d6242640 1 -- 172.21.15.190:0/2398843146 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d0076e60 con 0x7f85d00a5c70 2024-08-19T10:42:28.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85bc03be00 0x7f85bc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85b7fff640 1 -- 172.21.15.190:0/2398843146 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 7289+0+0 (secure 0 0 0) 0x7f85c000f040 con 0x7f85d00a5c70 2024-08-19T10:42:28.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85ceffd640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85bc03be00 0x7f85bc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:28.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.248+0000 7f85ceffd640 10 monclient: get_auth_request con 0x7f85bc03be00 auth_method 0 2024-08-19T10:42:28.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.252+0000 7f85ceffd640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85bc03be00 0x7f85bc03e2c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f85d00a2260 tx=0x7f85c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:28.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.256+0000 7f85b7fff640 1 -- 172.21.15.190:0/2398843146 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f85c0022020 con 0x7f85d00a5c70 2024-08-19T10:42:28.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.256+0000 7f85b7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:28.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.256+0000 7f85b7fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:28.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.428+0000 7f85d6242640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:28.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.428+0000 7f85d6242640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:28.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.428+0000 7f85d6242640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:28.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.428+0000 7f85d6242640 1 -- 172.21.15.190:0/2398843146 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f85d0139d60 con 0x7f85d00a5c70 2024-08-19T10:42:28.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.428+0000 7f85b7fff640 1 -- 172.21.15.190:0/2398843146 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v141) ==== 71+0+0 (secure 0 0 0) 0x7f85c004d520 con 0x7f85d00a5c70 2024-08-19T10:42:28.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.428+0000 7f85b7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:28.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.428+0000 7f85b7fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:28.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.524+0000 7f85d6242640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:28.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.524+0000 7f85d6242640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:28.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.524+0000 7f85d6242640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:28.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.524+0000 7f85d6242640 1 -- 172.21.15.190:0/2398843146 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f85d00008e0 con 0x7f85d00a5c70 2024-08-19T10:42:28.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85b7fff640 1 -- 172.21.15.190:0/2398843146 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v141) ==== 71+0+0 (secure 0 0 0) 0x7f85c0014a40 con 0x7f85d00a5c70 2024-08-19T10:42:28.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85b7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:28.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85b7fff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:28.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 -- 172.21.15.190:0/2398843146 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85bc03be00 msgr2=0x7f85bc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:28.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85bc03be00 0x7f85bc03e2c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f85d00a2260 tx=0x7f85c4008040 comp rx=0 tx=0).stop 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 10 monclient: shutdown 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 -- 172.21.15.190:0/2398843146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d00a5c70 msgr2=0x7f85d013a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d00a5c70 0x7f85d013a690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f85c000d800 tx=0x7f85c000dcc0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 -- 172.21.15.190:0/2398843146 shutdown_connections 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f85bc03be00 0x7f85bc03e2c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f85d013bb60 0x7f85d013b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f85d0139980 0x7f85d013abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 --2- 172.21.15.190:0/2398843146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85d00a5c70 0x7f85d013a690 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 -- 172.21.15.190:0/2398843146 >> 172.21.15.190:0/2398843146 conn(0x7f85d00abde0 msgr2=0x7f85d00ab4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 -- 172.21.15.190:0/2398843146 shutdown_connections 2024-08-19T10:42:28.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.528+0000 7f85d6242640 1 -- 172.21.15.190:0/2398843146 wait complete. 2024-08-19T10:42:28.547 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set create-compat 2024-08-19T10:42:28.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient: get_monmap_and_config 2024-08-19T10:42:28.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient: build_initial_monmap 2024-08-19T10:42:28.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient: monmap: 2024-08-19T10:42:28.664 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:28.664 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:28.664 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:28.664782+0000 2024-08-19T10:42:28.664 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:28.664782+0000 2024-08-19T10:42:28.664 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:28.665 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 1 Processor -- start 2024-08-19T10:42:28.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 1 -- start start 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient: init 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient: _add_conns ranks=[1,9,3,7,5,8,4,6,10,2,0] 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4b1c139980 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4b1c0a5730 0x7f4b1c0a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient(hunting): picked mon.noname-a con 0x7f4b1c0a5730 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b1c13e8c0 0x7f4b1c140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient(hunting): picked mon.noname-g con 0x7f4b1c13e8c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b1c0a8470 con 0x7f4b1c139980 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f4b1c09ca60 con 0x7f4b1c0a5730 2024-08-19T10:42:28.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1c0a3640 con 0x7f4b1c13e8c0 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b2328b640 10 monclient(hunting): authenticate will time out at 1194.235509s 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b21828640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b1c13e8c0 msgr2=0x7f4b1c140cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b21027640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f4b1c139980 0x7f4b1c139d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:48572/0 (socket says 172.21.15.190:48572) 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b21828640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b1c13e8c0 0x7f4b1c140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b21027640 1 -- 172.21.15.190:0/2059064574 learned_addr learned my addr 172.21.15.190:0/2059064574 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.660+0000 7f4b20826640 1 --2- 172.21.15.190:0/2059064574 >> v2:172.21.15.89:3300/0 conn(0x7f4b1c0a5730 0x7f4b1c0a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 10 monclient(hunting): get_auth_request con 0x7f4b1c0a5730 auth_method 0 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:28.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:28.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:28.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:28.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 10 monclient(hunting): handle_auth_done global_id 5589 payload 274 2024-08-19T10:42:28.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 1 -- 172.21.15.190:0/2059064574 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b1c13e8c0 msgr2=0x7f4b1c140cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:28.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 1 --2- 172.21.15.190:0/2059064574 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b1c13e8c0 0x7f4b1c140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 1 -- 172.21.15.190:0/2059064574 >> v1:172.21.15.190:6789/0 conn(0x7f4b1c139980 legacy=0x7f4b1c139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:28.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 10 monclient: _finish_hunting 0 2024-08-19T10:42:28.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 1 monclient: found mon.noname-a 2024-08-19T10:42:28.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:28.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:28.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.664+0000 7f4b20826640 1 -- 172.21.15.190:0/2059064574 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c0a3850 con 0x7f4b1c0a5730 2024-08-19T10:42:28.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b20826640 1 -- 172.21.15.190:0/2059064574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1c0a5730 msgr2=0x7f4b1c0a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:28.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b20826640 1 --2- 172.21.15.190:0/2059064574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1c0a5730 0x7f4b1c0a5b10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0097f0 tx=0x7f4b0c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a91eb55d99c6c5c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 1 -- 172.21.15.190:0/2059064574 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b0c0b0070 con 0x7f4b1c0a5730 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 10 monclient: dump: 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:28.673 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:28.674 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:28.674 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:28.674 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:28.674 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:28.674 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:28.674 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:28.674 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:28.674 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 1 -- 172.21.15.190:0/2059064574 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0c0aa810 con 0x7f4b1c0a5730 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 1 -- 172.21.15.190:0/2059064574 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b0c0aae30 con 0x7f4b1c0a5730 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b13fff640 10 monclient: dump: 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:28.675 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 5 monclient: authenticate success, global_id 5589 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:28.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 4 set_mon_vals no callback set 2024-08-19T10:42:28.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 10 monclient: shutdown 2024-08-19T10:42:28.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 1 -- 172.21.15.190:0/2059064574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1c0a5730 msgr2=0x7f4b1c0a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 1 --2- 172.21.15.190:0/2059064574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1c0a5730 0x7f4b1c0a5b10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0097f0 tx=0x7f4b0c01c380 comp rx=0 tx=0).stop 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 1 -- 172.21.15.190:0/2059064574 shutdown_connections 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 1 --2- 172.21.15.190:0/2059064574 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b1c13e8c0 0x7f4b1c140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 1 --2- 172.21.15.190:0/2059064574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4b1c0a5730 0x7f4b1c0a5b10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 1 -- 172.21.15.190:0/2059064574 >> 172.21.15.190:0/2059064574 conn(0x7f4b1c0abde0 msgr2=0x7f4b1c0ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 1 -- 172.21.15.190:0/2059064574 shutdown_connections 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 1 -- 172.21.15.190:0/2059064574 wait complete. 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 10 monclient: build_initial_monmap 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.668+0000 7f4b2328b640 10 monclient: monmap: 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:28.682 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:28.683 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:28.683 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:28.683 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.683 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:28.683 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:28.683 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:28.683 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:28.683 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:28.683 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:28.683 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 1 Processor -- start 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 1 -- start start 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient: init 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient: _add_conns ranks=[7,1,6,3,5,2,0,8,4] 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b1c0a5730 0x7f4b1c13da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient(hunting): picked mon.noname-i con 0x7f4b1c0a5730 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b1c139980 0x7f4b1c1d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient(hunting): picked mon.noname-f con 0x7f4b1c139980 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:28.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4b1c13e8c0 0x7f4b1c1d1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient(hunting): picked mon.noname-d con 0x7f4b1c13e8c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c143b90 con 0x7f4b1c139980 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4b1c143a10 con 0x7f4b1c13e8c0 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4b1c143d10 con 0x7f4b1c0a5730 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient(hunting): authenticate will time out at 1194.244666s 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b1c139980 0x7f4b1c1d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b21027640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b1c0a5730 0x7f4b1c13da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b1c139980 0x7f4b1c1d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51078/0 (socket says 172.21.15.190:51078) 2024-08-19T10:42:28.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 1 -- 172.21.15.190:0/1268803929 learned_addr learned my addr 172.21.15.190:0/1268803929 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b21027640 10 monclient(hunting): get_auth_request con 0x7f4b1c0a5730 auth_method 0 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b21027640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b21027640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b21027640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 10 monclient(hunting): get_auth_request con 0x7f4b1c139980 auth_method 0 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b21027640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b21027640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b21027640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:28.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 10 monclient(hunting): handle_auth_done global_id 14887 payload 824 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 1 -- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b1c0a5730 msgr2=0x7f4b1c13da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b1c0a5730 0x7f4b1c13da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 1 -- 172.21.15.190:0/1268803929 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4b1c13e8c0 msgr2=0x7f4b1c1d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4b1c13e8c0 0x7f4b1c1d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 10 monclient: _finish_hunting 0 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 1 monclient: found mon.noname-f 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 1 -- 172.21.15.190:0/1268803929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c0a73f0 con 0x7f4b1c139980 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b21027640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b1c0a5730 0x7f4b1c13da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 1 -- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b1c139980 msgr2=0x7f4b1c1d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:28.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b20826640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b1c139980 0x7f4b1c1d0f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0097f0 tx=0x7f4b0c019120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:28.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 1 -- 172.21.15.190:0/1268803929 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b0c0b0040 con 0x7f4b1c139980 2024-08-19T10:42:28.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:28.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:28.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 10 monclient: dump: 2024-08-19T10:42:28.688 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:28.688 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:28.688 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.688 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.688 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:28.688 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:28.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 5 monclient: authenticate success, global_id 14887 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient: _renew_subs 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 1 -- 172.21.15.190:0/1268803929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1c1379a0 con 0x7f4b1c139980 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 1 -- 172.21.15.190:0/1268803929 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0c004240 con 0x7f4b1c139980 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 1 -- 172.21.15.190:0/1268803929 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b0c0043f0 con 0x7f4b1c139980 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b11ffb640 10 monclient: dump: 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:28.690 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:28.691 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b137fe640 4 set_mon_vals no callback set 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient: _renew_subs 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.672+0000 7f4b2328b640 1 -- 172.21.15.190:0/1268803929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b1c135d20 con 0x7f4b1c139980 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.676+0000 7f4b2328b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.676+0000 7f4b2328b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.676+0000 7f4b2328b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.676+0000 7f4b2328b640 1 -- 172.21.15.190:0/1268803929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b1c0a5b90 con 0x7f4b1c139980 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.676+0000 7f4b11ffb640 1 -- 172.21.15.190:0/1268803929 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4b0c0b3070 con 0x7f4b1c139980 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.676+0000 7f4b11ffb640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b0003be00 0x7f4b0003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.676+0000 7f4b11ffb640 1 -- 172.21.15.190:0/1268803929 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 7289+0+0 (secure 0 0 0) 0x7f4b0c0e99c0 con 0x7f4b1c139980 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.676+0000 7f4b21027640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b0003be00 0x7f4b0003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:28.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.676+0000 7f4b21027640 10 monclient: get_auth_request con 0x7f4b0003be00 auth_method 0 2024-08-19T10:42:28.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.680+0000 7f4b21027640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b0003be00 0x7f4b0003e2c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4b18007ca0 tx=0x7f4b18007bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:28.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.680+0000 7f4b11ffb640 1 -- 172.21.15.190:0/1268803929 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4b0c016090 con 0x7f4b1c139980 2024-08-19T10:42:28.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.680+0000 7f4b11ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:28.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.680+0000 7f4b11ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:28.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.856+0000 7f4b2328b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:28.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.856+0000 7f4b2328b640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:28.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.856+0000 7f4b2328b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:28.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:28.856+0000 7f4b2328b640 1 -- 172.21.15.190:0/1268803929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7f4b1c06c8d0 con 0x7f4b1c139980 2024-08-19T10:42:29.944 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:29.940+0000 7f4b11ffb640 1 -- 172.21.15.190:0/1268803929 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v142) ==== 82+0+0 (secure 0 0 0) 0x7f4b0c0c6d90 con 0x7f4b1c139980 2024-08-19T10:42:29.944 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:29.940+0000 7f4b11ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:29.944 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:29.940+0000 7f4b11ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:30.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.052+0000 7f4b2328b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:30.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.052+0000 7f4b2328b640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:30.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.052+0000 7f4b2328b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:30.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.052+0000 7f4b2328b640 1 -- 172.21.15.190:0/1268803929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7f4b1c137100 con 0x7f4b1c139980 2024-08-19T10:42:30.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b11ffb640 1 -- 172.21.15.190:0/1268803929 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v142) ==== 115+0+0 (secure 0 0 0) 0x7f4b0c0e6a40 con 0x7f4b1c139980 2024-08-19T10:42:30.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b11ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:30.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b11ffb640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-08-19T10:42:30.059 INFO:tasks.workunit.client.0.smithi190.stderr:compat weight-set already created 2024-08-19T10:42:30.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 -- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b0003be00 msgr2=0x7f4b0003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b0003be00 0x7f4b0003e2c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4b18007ca0 tx=0x7f4b18007bf0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 10 monclient: shutdown 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 -- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b1c139980 msgr2=0x7f4b1c1d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b1c139980 0x7f4b1c1d0f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0097f0 tx=0x7f4b0c019120 comp rx=0 tx=0).stop 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 -- 172.21.15.190:0/1268803929 shutdown_connections 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b0003be00 0x7f4b0003e2c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4b1c13e8c0 0x7f4b1c1d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b1c139980 0x7f4b1c1d0f20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 --2- 172.21.15.190:0/1268803929 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b1c0a5730 0x7f4b1c13da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 -- 172.21.15.190:0/1268803929 >> 172.21.15.190:0/1268803929 conn(0x7f4b1c0abde0 msgr2=0x7f4b1c0a9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:30.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 -- 172.21.15.190:0/1268803929 shutdown_connections 2024-08-19T10:42:30.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.056+0000 7f4b2328b640 1 -- 172.21.15.190:0/1268803929 wait complete. 2024-08-19T10:42:30.077 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set ls 2024-08-19T10:42:30.077 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep '(compat)' 2024-08-19T10:42:30.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient: get_monmap_and_config 2024-08-19T10:42:30.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient: build_initial_monmap 2024-08-19T10:42:30.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient: monmap: 2024-08-19T10:42:30.196 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:30.196 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:30.196 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:30.196634+0000 2024-08-19T10:42:30.196 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:30.196634+0000 2024-08-19T10:42:30.196 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:30.196 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:30.197 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 1 Processor -- start 2024-08-19T10:42:30.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 1 -- start start 2024-08-19T10:42:30.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient: init 2024-08-19T10:42:30.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:30.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient: _add_conns ranks=[6,0,8,1,9,7,4,2,3,5,10] 2024-08-19T10:42:30.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ad00a5940 0x7f6ad00720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient(hunting): picked mon.noname-d con 0x7f6ad00a5940 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:30.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6ad00a5d20 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 0x7f6ad0140f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient(hunting): picked mon.noname-e con 0x7f6ad013eb60 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ad006efb0 con 0x7f6ad00a5d20 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6ad006df10 con 0x7f6ad00a5940 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6ad006f290 con 0x7f6ad013eb60 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad6bd5640 10 monclient(hunting): authenticate will time out at 1195.766510s 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad4971640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ad00a5940 0x7f6ad00720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 0x7f6ad0140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6acffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6ad00a5d20 0x7f6ad00a6100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:60128/0 (socket says 172.21.15.190:60128) 2024-08-19T10:42:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6acffff640 1 -- 172.21.15.190:0/3115169763 learned_addr learned my addr 172.21.15.190:0/3115169763 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:30.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 10 monclient(hunting): get_auth_request con 0x7f6ad013eb60 auth_method 0 2024-08-19T10:42:30.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:30.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:30.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:30.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad4971640 10 monclient(hunting): get_auth_request con 0x7f6ad00a5940 auth_method 0 2024-08-19T10:42:30.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad4971640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:30.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad4971640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:30.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad4971640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:30.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:30.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:30.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad4971640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad4971640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad4971640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 10 monclient(hunting): handle_auth_done global_id 5354 payload 274 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 1 -- 172.21.15.190:0/3115169763 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ad00a5940 msgr2=0x7f6ad00720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 1 --2- 172.21.15.190:0/3115169763 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ad00a5940 0x7f6ad00720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 1 -- 172.21.15.190:0/3115169763 >> v1:172.21.15.89:6789/0 conn(0x7f6ad00a5d20 legacy=0x7f6ad00a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 10 monclient: _finish_hunting 0 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 1 monclient: found mon.noname-e 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad5172640 1 -- 172.21.15.190:0/3115169763 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad00a73d0 con 0x7f6ad013eb60 2024-08-19T10:42:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.192+0000 7f6ad4971640 1 --2- 172.21.15.190:0/3115169763 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ad00a5940 0x7f6ad00720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad5172640 1 -- 172.21.15.190:0/3115169763 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 msgr2=0x7f6ad0140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad5172640 1 --2- 172.21.15.190:0/3115169763 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 0x7f6ad0140f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009dc0 tx=0x7f6ac801c8e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=46978bc57455c699 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 1 -- 172.21.15.190:0/3115169763 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ac80a6940 con 0x7f6ad013eb60 2024-08-19T10:42:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 10 monclient: dump: 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:30.205 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 1 -- 172.21.15.190:0/3115169763 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ac80a6b00 con 0x7f6ad013eb60 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 1 -- 172.21.15.190:0/3115169763 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ac80a6cf0 con 0x7f6ad013eb60 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6acf7fe640 10 monclient: dump: 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:30.206 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:30.207 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.207 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.207 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:30.207 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.207 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:30.207 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:30.207 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:30.207 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:30.207 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:30.207 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:30.207 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:30.209 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:30.210 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 5 monclient: authenticate success, global_id 5354 2024-08-19T10:42:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:30.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 4 set_mon_vals no callback set 2024-08-19T10:42:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient: shutdown 2024-08-19T10:42:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/3115169763 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 msgr2=0x7f6ad0140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 --2- 172.21.15.190:0/3115169763 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 0x7f6ad0140f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009dc0 tx=0x7f6ac801c8e0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/3115169763 shutdown_connections 2024-08-19T10:42:30.214 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 --2- 172.21.15.190:0/3115169763 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 0x7f6ad0140f50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 --2- 172.21.15.190:0/3115169763 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6ad00a5940 0x7f6ad00720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/3115169763 >> 172.21.15.190:0/3115169763 conn(0x7f6ad00aa530 msgr2=0x7f6ad00a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/3115169763 shutdown_connections 2024-08-19T10:42:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/3115169763 wait complete. 2024-08-19T10:42:30.215 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient: build_initial_monmap 2024-08-19T10:42:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient: monmap: 2024-08-19T10:42:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:30.216 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 Processor -- start 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 -- start start 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient: init 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient: _add_conns ranks=[0,5,8,4,1,6,7,2,3] 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ad00a5940 0x7f6ad0136450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient(hunting): picked mon.noname-a con 0x7f6ad00a5940 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ad00a5d20 0x7f6ad0136990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient(hunting): picked mon.noname-h con 0x7f6ad00a5d20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 0x7f6ad013a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient(hunting): picked mon.noname-e con 0x7f6ad013eb60 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0143e30 con 0x7f6ad00a5940 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6ad0143cb0 con 0x7f6ad00a5d20 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6ad0143fb0 con 0x7f6ad013eb60 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.200+0000 7f6ad6bd5640 10 monclient(hunting): authenticate will time out at 1195.775945s 2024-08-19T10:42:30.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ad00a5d20 0x7f6ad0136990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ad00a5d20 0x7f6ad0136990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:56946/0 (socket says 172.21.15.190:56946) 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 1 -- 172.21.15.190:0/4193240266 learned_addr learned my addr 172.21.15.190:0/4193240266 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 10 monclient(hunting): get_auth_request con 0x7f6ad00a5d20 auth_method 0 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad5172640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 0x7f6ad013a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad4971640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ad00a5940 0x7f6ad0136450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad5172640 10 monclient(hunting): get_auth_request con 0x7f6ad013eb60 auth_method 0 2024-08-19T10:42:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad5172640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad5172640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad5172640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad4971640 10 monclient(hunting): get_auth_request con 0x7f6ad00a5940 auth_method 0 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad4971640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad4971640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad4971640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 10 monclient(hunting): handle_auth_done global_id 5594 payload 824 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 1 -- 172.21.15.190:0/4193240266 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 msgr2=0x7f6ad013a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 0x7f6ad013a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 1 -- 172.21.15.190:0/4193240266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ad00a5940 msgr2=0x7f6ad0136450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ad00a5940 0x7f6ad0136450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 1 monclient: found mon.noname-h 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 1 -- 172.21.15.190:0/4193240266 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad0120f50 con 0x7f6ad00a5d20 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 1 -- 172.21.15.190:0/4193240266 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ad00a5d20 msgr2=0x7f6ad0136990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acffff640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ad00a5d20 0x7f6ad0136990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c4a0 tx=0x7f6abc00c960 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 1 -- 172.21.15.190:0/4193240266 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6abc017070 con 0x7f6ad00a5d20 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 10 monclient: dump: 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:30.221 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:30.222 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 10 monclient: _finish_auth 0 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad6bd5640 5 monclient: authenticate success, global_id 5594 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 1 -- 172.21.15.190:0/4193240266 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6abc005d00 con 0x7f6ad00a5d20 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad6bd5640 10 monclient: _renew_subs 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 1 -- 172.21.15.190:0/4193240266 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6abc0074e0 con 0x7f6ad00a5d20 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad6bd5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/4193240266 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad00a84f0 con 0x7f6ad00a5d20 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ace7fc640 4 set_mon_vals no callback set 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:30.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6acd7fa640 10 monclient: dump: 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:30.224 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad6bd5640 10 monclient: _renew_subs 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad6bd5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/4193240266 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad013dc00 con 0x7f6ad00a5d20 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad6bd5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad6bd5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad6bd5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.204+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/4193240266 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad013c8e0 con 0x7f6ad00a5d20 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.208+0000 7f6acd7fa640 1 -- 172.21.15.190:0/4193240266 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6abc007070 con 0x7f6ad00a5d20 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.208+0000 7f6acd7fa640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ab403be00 0x7f6ab403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.208+0000 7f6acd7fa640 1 -- 172.21.15.190:0/4193240266 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(142..142 src has 1..142) ==== 7621+0+0 (secure 0 0 0) 0x7f6abc04e930 con 0x7f6ad00a5d20 2024-08-19T10:42:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.208+0000 7f6ad4971640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ab403be00 0x7f6ab403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.208+0000 7f6ad4971640 10 monclient: get_auth_request con 0x7f6ab403be00 auth_method 0 2024-08-19T10:42:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.208+0000 7f6ad4971640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ab403be00 0x7f6ab403e2c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009580 tx=0x7f6ac4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.212+0000 7f6acd7fa640 1 -- 172.21.15.190:0/4193240266 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6abc007680 con 0x7f6ad00a5d20 2024-08-19T10:42:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.212+0000 7f6acd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:30.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.212+0000 7f6acd7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:30.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.388+0000 7f6ad6bd5640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:30.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.388+0000 7f6ad6bd5640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:30.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.388+0000 7f6ad6bd5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:30.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.388+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/4193240266 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f6ad00720a0 con 0x7f6ad00a5d20 2024-08-19T10:42:30.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.392+0000 7f6acd7fa640 1 -- 172.21.15.190:0/4193240266 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v143) ==== 71+0+9 (secure 0 0 0) 0x7f6abc01bc90 con 0x7f6ad00a5d20 2024-08-19T10:42:30.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.392+0000 7f6acd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:30.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.392+0000 7f6acd7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:30.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.492+0000 7f6ad6bd5640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:30.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.492+0000 7f6ad6bd5640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:30.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.492+0000 7f6ad6bd5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:30.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.492+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/4193240266 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set ls"} v 0) -- 0x7f6ad00008e0 con 0x7f6ad00a5d20 2024-08-19T10:42:30.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.492+0000 7f6acd7fa640 1 -- 172.21.15.190:0/4193240266 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set ls"}]=0 v143) ==== 71+0+9 (secure 0 0 0) 0x7f6abc019310 con 0x7f6ad00a5d20 2024-08-19T10:42:30.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.492+0000 7f6acd7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set ls"}] 2024-08-19T10:42:30.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.492+0000 7f6acd7fa640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:30.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/4193240266 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ab403be00 msgr2=0x7f6ab403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:30.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ab403be00 0x7f6ab403e2c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009580 tx=0x7f6ac4008040 comp rx=0 tx=0).stop 2024-08-19T10:42:30.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 10 monclient: shutdown 2024-08-19T10:42:30.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:30.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/4193240266 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ad00a5d20 msgr2=0x7f6ad0136990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:30.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ad00a5d20 0x7f6ad0136990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c4a0 tx=0x7f6abc00c960 comp rx=0 tx=0).stop 2024-08-19T10:42:30.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/4193240266 shutdown_connections 2024-08-19T10:42:30.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ab403be00 0x7f6ab403e2c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ad013eb60 0x7f6ad013a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ad00a5d20 0x7f6ad0136990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 --2- 172.21.15.190:0/4193240266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ad00a5940 0x7f6ad0136450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/4193240266 >> 172.21.15.190:0/4193240266 conn(0x7f6ad00aa530 msgr2=0x7f6ad013de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:30.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/4193240266 shutdown_connections 2024-08-19T10:42:30.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.496+0000 7f6ad6bd5640 1 -- 172.21.15.190:0/4193240266 wait complete. 2024-08-19T10:42:30.513 INFO:tasks.workunit.client.0.smithi190.stdout:(compat) 2024-08-19T10:42:30.513 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set rm-compat 2024-08-19T10:42:30.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient: get_monmap_and_config 2024-08-19T10:42:30.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient: build_initial_monmap 2024-08-19T10:42:30.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient: monmap: 2024-08-19T10:42:30.628 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:30.628 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:30.628976+0000 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:30.628976+0000 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:30.629 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:30.630 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:30.630 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:30.630 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 1 Processor -- start 2024-08-19T10:42:30.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 1 -- start start 2024-08-19T10:42:30.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient: init 2024-08-19T10:42:30.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:30.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient: _add_conns ranks=[10,9,5,0,8,7,6,4,3,1,2] 2024-08-19T10:42:30.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f74bc0a5eb0 0x7f74bc0a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient(hunting): picked mon.noname-f con 0x7f74bc0a5eb0 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f74bc0a4bf0 0x7f74bc0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient(hunting): picked mon.noname-a con 0x7f74bc0a4bf0 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74bc139860 0x7f74bc139d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient(hunting): picked mon.noname-h con 0x7f74bc139860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f74bc06cad0 con 0x7f74bc0a4bf0 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f74bc0a5510 con 0x7f74bc0a5eb0 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f74bc0a6290 con 0x7f74bc139860 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c3cc3640 10 monclient(hunting): authenticate will time out at 1196.199003s 2024-08-19T10:42:30.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74bc139860 0x7f74bc139d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c1a5f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f74bc0a5eb0 0x7f74bc0a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74bc139860 0x7f74bc139d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:56956/0 (socket says 172.21.15.190:56956) 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 1 -- 172.21.15.190:0/234317925 learned_addr learned my addr 172.21.15.190:0/234317925 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c1a5f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f74bc0a5eb0 0x7f74bc0a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51090/0 (socket says 172.21.15.190:51090) 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 10 monclient(hunting): get_auth_request con 0x7f74bc139860 auth_method 0 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c1a5f640 10 monclient(hunting): get_auth_request con 0x7f74bc0a5eb0 auth_method 0 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c1a5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c1a5f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c1a5f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:30.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c1a5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:30.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c1a5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:30.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c1a5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:30.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:30.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:30.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:30.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c125e640 1 --2- 172.21.15.190:0/234317925 >> v2:172.21.15.89:3300/0 conn(0x7f74bc0a4bf0 0x7f74bc0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c125e640 10 monclient(hunting): get_auth_request con 0x7f74bc0a4bf0 auth_method 0 2024-08-19T10:42:30.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c125e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:30.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c125e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:30.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c125e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:30.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 10 monclient(hunting): handle_auth_done global_id 5603 payload 274 2024-08-19T10:42:30.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 1 -- 172.21.15.190:0/234317925 >> v2:172.21.15.190:3300/0 conn(0x7f74bc0a5eb0 msgr2=0x7f74bc0a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:30.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 1 --2- 172.21.15.190:0/234317925 >> v2:172.21.15.190:3300/0 conn(0x7f74bc0a5eb0 0x7f74bc0a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 1 -- 172.21.15.190:0/234317925 >> v2:172.21.15.89:3300/0 conn(0x7f74bc0a4bf0 msgr2=0x7f74bc0a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:30.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 1 --2- 172.21.15.190:0/234317925 >> v2:172.21.15.89:3300/0 conn(0x7f74bc0a4bf0 0x7f74bc0a4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 10 monclient: _finish_hunting 0 2024-08-19T10:42:30.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 1 monclient: found mon.noname-h 2024-08-19T10:42:30.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:30.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:30.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c2260640 1 -- 172.21.15.190:0/234317925 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc06cd10 con 0x7f74bc139860 2024-08-19T10:42:30.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.624+0000 7f74c1a5f640 1 --2- 172.21.15.190:0/234317925 >> v2:172.21.15.190:3300/0 conn(0x7f74bc0a5eb0 0x7f74bc0a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:30.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.628+0000 7f74c2260640 1 -- 172.21.15.190:0/234317925 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74bc139860 msgr2=0x7f74bc139d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:30.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.628+0000 7f74c2260640 1 --2- 172.21.15.190:0/234317925 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74bc139860 0x7f74bc139d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0a5770 tx=0x7f74b401c3e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=c9f5d66f2dce28bc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.628+0000 7f74c0a5d640 1 -- 172.21.15.190:0/234317925 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74b40b0070 con 0x7f74bc139860 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.628+0000 7f74c0a5d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c0a5d640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c0a5d640 10 monclient: dump: 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:30.643 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c0a5d640 10 monclient: _finish_auth 0 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c0a5d640 10 monclient: _check_auth_tickets 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c0a5d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c0a5d640 1 -- 172.21.15.190:0/234317925 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b40aa810 con 0x7f74bc139860 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c0a5d640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c0a5d640 1 -- 172.21.15.190:0/234317925 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74b40aae30 con 0x7f74bc139860 2024-08-19T10:42:30.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 5 monclient: authenticate success, global_id 5603 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 4 set_mon_vals no callback set 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c0a5d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c0a5d640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c0a5d640 10 monclient: dump: 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:30.645 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:30.646 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:30.646 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:30.646 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:30.646 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:30.646 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:30.646 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:30.646 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient: shutdown 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 -- 172.21.15.190:0/234317925 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74bc139860 msgr2=0x7f74bc139d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 --2- 172.21.15.190:0/234317925 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74bc139860 0x7f74bc139d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0a5770 tx=0x7f74b401c3e0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 -- 172.21.15.190:0/234317925 shutdown_connections 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 --2- 172.21.15.190:0/234317925 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f74bc139860 0x7f74bc139d10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 --2- 172.21.15.190:0/234317925 >> v2:172.21.15.89:3300/0 conn(0x7f74bc0a4bf0 0x7f74bc0a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 --2- 172.21.15.190:0/234317925 >> v2:172.21.15.190:3300/0 conn(0x7f74bc0a5eb0 0x7f74bc0a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 -- 172.21.15.190:0/234317925 >> 172.21.15.190:0/234317925 conn(0x7f74bc0a92a0 msgr2=0x7f74bc0a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 -- 172.21.15.190:0/234317925 shutdown_connections 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 -- 172.21.15.190:0/234317925 wait complete. 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient: build_initial_monmap 2024-08-19T10:42:30.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient: monmap: 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:30.648 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 Processor -- start 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 -- start start 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient: init 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient: _add_conns ranks=[8,4,7,5,2,3,6,0,1] 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74bc0a4bf0 0x7f74bc1383b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient(hunting): picked mon.noname-e con 0x7f74bc0a4bf0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74bc0a5eb0 0x7f74bc1388f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient(hunting): picked mon.noname-c con 0x7f74bc0a5eb0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:30.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74bc139860 0x7f74bc13c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient(hunting): picked mon.noname-i con 0x7f74bc139860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f74bc0a1770 con 0x7f74bc0a5eb0 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f74bc06cf10 con 0x7f74bc139860 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f74bc0a18f0 con 0x7f74bc0a4bf0 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c3cc3640 10 monclient(hunting): authenticate will time out at 1196.206330s 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74bc139860 0x7f74bc13c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74bc139860 0x7f74bc13c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:35886/0 (socket says 172.21.15.190:35886) 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 1 -- 172.21.15.190:0/3039223522 learned_addr learned my addr 172.21.15.190:0/3039223522 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:30.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 10 monclient(hunting): get_auth_request con 0x7f74bc139860 auth_method 0 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c1a5f640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74bc0a4bf0 0x7f74bc1383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c125e640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74bc0a5eb0 0x7f74bc1388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c125e640 10 monclient(hunting): get_auth_request con 0x7f74bc0a5eb0 auth_method 0 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c125e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c125e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c125e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:30.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c1a5f640 10 monclient(hunting): get_auth_request con 0x7f74bc0a4bf0 auth_method 0 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c1a5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c1a5f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c1a5f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 10 monclient(hunting): handle_auth_done global_id 14794 payload 824 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 1 -- 172.21.15.190:0/3039223522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74bc0a4bf0 msgr2=0x7f74bc1383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74bc0a4bf0 0x7f74bc1383b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 1 -- 172.21.15.190:0/3039223522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74bc0a5eb0 msgr2=0x7f74bc1388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74bc0a5eb0 0x7f74bc1388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 10 monclient: _finish_hunting 0 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 1 monclient: found mon.noname-i 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.632+0000 7f74c2260640 1 -- 172.21.15.190:0/3039223522 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc123ea0 con 0x7f74bc139860 2024-08-19T10:42:30.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c2260640 1 -- 172.21.15.190:0/3039223522 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74bc139860 msgr2=0x7f74bc13c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c2260640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74bc139860 0x7f74bc13c790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f74b401aa90 tx=0x7f74b4019180 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 1 -- 172.21.15.190:0/3039223522 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74b40b0050 con 0x7f74bc139860 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 10 monclient: dump: 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:30.653 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:30.654 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:30.654 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:30.654 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:30.654 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:30.654 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:30.654 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:30.654 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:30.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:30.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:30.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 1 -- 172.21.15.190:0/3039223522 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b40bc070 con 0x7f74bc139860 2024-08-19T10:42:30.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:30.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 1 -- 172.21.15.190:0/3039223522 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74b4004250 con 0x7f74bc139860 2024-08-19T10:42:30.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:30.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:30.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 10 monclient: dump: 2024-08-19T10:42:30.656 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:30.656 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:30.656 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.656 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:30.656 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:30.656 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:30.656 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:30.656 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:30.656 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:30.657 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:30.657 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:30.657 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:30.657 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:30.657 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:30.657 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:30.657 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:30.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74bb7fe640 4 set_mon_vals no callback set 2024-08-19T10:42:30.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c3cc3640 5 monclient: authenticate success, global_id 14794 2024-08-19T10:42:30.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c3cc3640 10 monclient: _renew_subs 2024-08-19T10:42:30.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c3cc3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c3cc3640 1 -- 172.21.15.190:0/3039223522 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc124180 con 0x7f74bc139860 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c3cc3640 10 monclient: _renew_subs 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c3cc3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c3cc3640 1 -- 172.21.15.190:0/3039223522 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc0a88c0 con 0x7f74bc139860 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c3cc3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c3cc3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c3cc3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c3cc3640 1 -- 172.21.15.190:0/3039223522 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc06d720 con 0x7f74bc139860 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 1 -- 172.21.15.190:0/3039223522 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f74b4003620 con 0x7f74bc139860 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f74a803be00 0x7f74a803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74ba7fc640 1 -- 172.21.15.190:0/3039223522 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(143..143 src has 1..143) ==== 7621+0+0 (secure 0 0 0) 0x7f74b40e9dc0 con 0x7f74bc139860 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c1a5f640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f74a803be00 0x7f74a803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:30.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.636+0000 7f74c1a5f640 10 monclient: get_auth_request con 0x7f74a803be00 auth_method 0 2024-08-19T10:42:30.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.640+0000 7f74c1a5f640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f74a803be00 0x7f74a803e2c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009bd0 tx=0x7f74ac00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:30.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.640+0000 7f74ba7fc640 1 -- 172.21.15.190:0/3039223522 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f74b40160f0 con 0x7f74bc139860 2024-08-19T10:42:30.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.640+0000 7f74ba7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:30.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.640+0000 7f74ba7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:30.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.816+0000 7f74c3cc3640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:42:30.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.816+0000 7f74c3cc3640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:42:30.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.816+0000 7f74c3cc3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:30.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:30.816+0000 7f74c3cc3640 1 -- 172.21.15.190:0/3039223522 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f74bc13ea70 con 0x7f74bc139860 2024-08-19T10:42:31.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:31.300+0000 7f74ba7fc640 1 -- 172.21.15.190:0/3039223522 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v144) ==== 78+0+0 (secure 0 0 0) 0x7f74b40c0dd0 con 0x7f74bc139860 2024-08-19T10:42:31.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:31.300+0000 7f74ba7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:42:31.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:31.300+0000 7f74ba7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:31.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:31.412+0000 7f74c3cc3640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:42:31.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:31.412+0000 7f74c3cc3640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:42:31.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:31.412+0000 7f74c3cc3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:31.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:31.412+0000 7f74c3cc3640 1 -- 172.21.15.190:0/3039223522 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f74bc0008e0 con 0x7f74bc139860 2024-08-19T10:42:32.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.300+0000 7f74ba7fc640 1 -- 172.21.15.190:0/3039223522 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v145) ==== 78+0+0 (secure 0 0 0) 0x7f74b40043f0 con 0x7f74bc139860 2024-08-19T10:42:32.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.300+0000 7f74ba7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:42:32.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.300+0000 7f74ba7fc640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:42:32.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 -- 172.21.15.190:0/3039223522 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f74a803be00 msgr2=0x7f74a803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:32.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f74a803be00 0x7f74a803e2c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009bd0 tx=0x7f74ac00b040 comp rx=0 tx=0).stop 2024-08-19T10:42:32.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 10 monclient: shutdown 2024-08-19T10:42:32.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:32.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 -- 172.21.15.190:0/3039223522 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74bc139860 msgr2=0x7f74bc13c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:32.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74bc139860 0x7f74bc13c790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f74b401aa90 tx=0x7f74b4019180 comp rx=0 tx=0).stop 2024-08-19T10:42:32.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 -- 172.21.15.190:0/3039223522 shutdown_connections 2024-08-19T10:42:32.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f74bc139860 0x7f74bc13c790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:32.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74bc0a5eb0 0x7f74bc1388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:32.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f74a803be00 0x7f74a803e2c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:32.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 --2- 172.21.15.190:0/3039223522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74bc0a4bf0 0x7f74bc1383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:32.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 -- 172.21.15.190:0/3039223522 >> 172.21.15.190:0/3039223522 conn(0x7f74bc0a92a0 msgr2=0x7f74bc072d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:32.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 -- 172.21.15.190:0/3039223522 shutdown_connections 2024-08-19T10:42:32.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.304+0000 7f74c3cc3640 1 -- 172.21.15.190:0/3039223522 wait complete. 2024-08-19T10:42:32.332 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create cool 2 2024-08-19T10:42:32.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.448+0000 7f8900f55640 10 monclient: get_monmap_and_config 2024-08-19T10:42:32.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.448+0000 7f8900f55640 10 monclient: build_initial_monmap 2024-08-19T10:42:32.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.448+0000 7f8900f55640 10 monclient: monmap: 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:32.454302+0000 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:32.454302+0000 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:32.457 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.448+0000 7f8900f55640 1 Processor -- start 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.448+0000 7f8900f55640 1 -- start start 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.448+0000 7f8900f55640 10 monclient: init 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 10 monclient: _add_conns ranks=[6,10,4,8,7,2,5,3,0,9,1] 2024-08-19T10:42:32.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88fc139980 0x7f88fc139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:32.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 10 monclient(hunting): picked mon.noname-d con 0x7f88fc139980 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:32.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f88fc13a2a0 0x7f88fc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:32.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 10 monclient(hunting): picked mon.noname-f con 0x7f88fc13a2a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:32.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f88fc0a5c70 0x7f88fc0a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:32.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 10 monclient(hunting): picked mon.noname-c con 0x7f88fc0a5c70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:32.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:32.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f88fc09ca60 con 0x7f88fc13a2a0 2024-08-19T10:42:32.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:32.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f88fc0a3640 con 0x7f88fc0a5c70 2024-08-19T10:42:32.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f88fc0a8470 con 0x7f88fc139980 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f8900f55640 10 monclient(hunting): authenticate will time out at 1198.024376s 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f88fc13a2a0 0x7f88fc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f88fc13a2a0 0x7f88fc0a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51094/0 (socket says 172.21.15.190:51094) 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 1 -- 172.21.15.190:0/3451584860 learned_addr learned my addr 172.21.15.190:0/3451584860 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 10 monclient(hunting): get_auth_request con 0x7f88fc13a2a0 auth_method 0 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:32.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88fa59c640 1 --2- 172.21.15.190:0/3451584860 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88fc139980 0x7f88fc139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88fa59c640 10 monclient(hunting): get_auth_request con 0x7f88fc139980 auth_method 0 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88fa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88fad9d640 1 --2- 172.21.15.190:0/3451584860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f88fc0a5c70 0x7f88fc0a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 10 monclient(hunting): handle_auth_done global_id 14905 payload 274 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 1 -- 172.21.15.190:0/3451584860 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88fc139980 msgr2=0x7f88fc139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 1 --2- 172.21.15.190:0/3451584860 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88fc139980 0x7f88fc139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 1 -- 172.21.15.190:0/3451584860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f88fc0a5c70 msgr2=0x7f88fc0a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 1 --2- 172.21.15.190:0/3451584860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f88fc0a5c70 0x7f88fc0a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:32.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 1 monclient: found mon.noname-f 2024-08-19T10:42:32.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:32.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:32.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.452+0000 7f88f1d9b640 1 -- 172.21.15.190:0/3451584860 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc0a3850 con 0x7f88fc13a2a0 2024-08-19T10:42:32.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f1d9b640 1 -- 172.21.15.190:0/3451584860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc13a2a0 msgr2=0x7f88fc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:32.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f1d9b640 1 --2- 172.21.15.190:0/3451584860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc13a2a0 0x7f88fc0a5730 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0098a0 tx=0x7f88ec01c140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24165a045e86b978 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:32.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 1 -- 172.21.15.190:0/3451584860 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88ec0b0070 con 0x7f88fc13a2a0 2024-08-19T10:42:32.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:32.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:32.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 10 monclient: dump: 2024-08-19T10:42:32.468 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:32.468 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:32.468 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:32.468 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:32.468 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:32.468 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 10 monclient: _finish_auth 0 2024-08-19T10:42:32.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:42:32.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:32.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 1 -- 172.21.15.190:0/3451584860 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ec0aa810 con 0x7f88fc13a2a0 2024-08-19T10:42:32.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:32.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 1 -- 172.21.15.190:0/3451584860 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88ec0aae30 con 0x7f88fc13a2a0 2024-08-19T10:42:32.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 5 monclient: authenticate success, global_id 14905 2024-08-19T10:42:32.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:32.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 4 set_mon_vals no callback set 2024-08-19T10:42:32.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:32.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:32.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f88f9d9b640 10 monclient: dump: 2024-08-19T10:42:32.471 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:32.471 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:32.471 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:32.471 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:32.471 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:32.471 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:32.471 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:32.471 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:32.471 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:32.471 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:32.472 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:32.472 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:32.472 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:32.472 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:32.472 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:32.472 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:32.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 10 monclient: shutdown 2024-08-19T10:42:32.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:32.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 1 -- 172.21.15.190:0/3451584860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc13a2a0 msgr2=0x7f88fc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:32.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 1 --2- 172.21.15.190:0/3451584860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc13a2a0 0x7f88fc0a5730 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0098a0 tx=0x7f88ec01c140 comp rx=0 tx=0).stop 2024-08-19T10:42:32.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 1 -- 172.21.15.190:0/3451584860 shutdown_connections 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 1 --2- 172.21.15.190:0/3451584860 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f88fc0a5c70 0x7f88fc0a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 1 --2- 172.21.15.190:0/3451584860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88fc13a2a0 0x7f88fc0a5730 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 1 --2- 172.21.15.190:0/3451584860 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88fc139980 0x7f88fc139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 1 -- 172.21.15.190:0/3451584860 >> 172.21.15.190:0/3451584860 conn(0x7f88fc0abde0 msgr2=0x7f88fc0ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 1 -- 172.21.15.190:0/3451584860 shutdown_connections 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 1 -- 172.21.15.190:0/3451584860 wait complete. 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 10 monclient: build_initial_monmap 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 10 monclient: monmap: 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:32.473 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 1 Processor -- start 2024-08-19T10:42:32.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 1 -- start start 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.456+0000 7f8900f55640 10 monclient: init 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient: _add_conns ranks=[0,5,4,3,7,2,8,1,6] 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88fc0a5c70 0x7f88fc133890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient(hunting): picked mon.noname-a con 0x7f88fc0a5c70 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88fc139980 0x7f88fc133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient(hunting): picked mon.noname-h con 0x7f88fc139980 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f88fc13a690 0x7f88fc13aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient(hunting): picked mon.noname-c con 0x7f88fc13a690 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc143270 con 0x7f88fc0a5c70 2024-08-19T10:42:32.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f88fc1430f0 con 0x7f88fc13a690 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f88fc1433f0 con 0x7f88fc139980 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient(hunting): authenticate will time out at 1198.032145s 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88fc139980 0x7f88fc133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88fc139980 0x7f88fc133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:56966/0 (socket says 172.21.15.190:56966) 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 1 -- 172.21.15.190:0/1530271104 learned_addr learned my addr 172.21.15.190:0/1530271104 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 10 monclient(hunting): get_auth_request con 0x7f88fc139980 auth_method 0 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:32.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88fad9d640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f88fc13a690 0x7f88fc13aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88fa59c640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88fc0a5c70 0x7f88fc133890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88fad9d640 10 monclient(hunting): get_auth_request con 0x7f88fc13a690 auth_method 0 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88fad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88fad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88fad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88fa59c640 10 monclient(hunting): get_auth_request con 0x7f88fc0a5c70 auth_method 0 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88fa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:32.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 10 monclient(hunting): handle_auth_done global_id 5612 payload 824 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 1 -- 172.21.15.190:0/1530271104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f88fc13a690 msgr2=0x7f88fc13aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f88fc13a690 0x7f88fc13aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 1 -- 172.21.15.190:0/1530271104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88fc0a5c70 msgr2=0x7f88fc133890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88fc0a5c70 0x7f88fc133890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 1 monclient: found mon.noname-h 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 1 -- 172.21.15.190:0/1530271104 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc13d160 con 0x7f88fc139980 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 1 -- 172.21.15.190:0/1530271104 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88fc139980 msgr2=0x7f88fc133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:32.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f1d9b640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88fc139980 0x7f88fc133dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f88ec01a7f0 tx=0x7f88ec018ee0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 1 -- 172.21.15.190:0/1530271104 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88ec0b0070 con 0x7f88fc139980 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 10 monclient: dump: 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:32.479 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 5 monclient: authenticate success, global_id 5612 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 1 -- 172.21.15.190:0/1530271104 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ec0b1040 con 0x7f88fc139980 2024-08-19T10:42:32.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient: _renew_subs 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 1 -- 172.21.15.190:0/1530271104 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88fc13b9c0 con 0x7f88fc139980 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 1 -- 172.21.15.190:0/1530271104 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88ec0aae20 con 0x7f88fc139980 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f959a640 4 set_mon_vals no callback set 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f88f37fe640 10 monclient: dump: 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:32.481 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient: _renew_subs 2024-08-19T10:42:32.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:32.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 1 -- 172.21.15.190:0/1530271104 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f88fc135760 con 0x7f88fc139980 2024-08-19T10:42:32.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:32.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.460+0000 7f8900f55640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:32.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.464+0000 7f8900f55640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:32.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.464+0000 7f8900f55640 1 -- 172.21.15.190:0/1530271104 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88fc139d60 con 0x7f88fc139980 2024-08-19T10:42:32.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.464+0000 7f88f37fe640 1 -- 172.21.15.190:0/1530271104 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f88ec003620 con 0x7f88fc139980 2024-08-19T10:42:32.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.464+0000 7f88f37fe640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88dc03be00 0x7f88dc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:32.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.464+0000 7f88f37fe640 1 -- 172.21.15.190:0/1530271104 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(145..145 src has 1..145) ==== 7289+0+0 (secure 0 0 0) 0x7f88ec0e8cc0 con 0x7f88fc139980 2024-08-19T10:42:32.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.464+0000 7f88fa59c640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88dc03be00 0x7f88dc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:32.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.464+0000 7f88fa59c640 10 monclient: get_auth_request con 0x7f88dc03be00 auth_method 0 2024-08-19T10:42:32.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.464+0000 7f88fa59c640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88dc03be00 0x7f88dc03e2c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f88e400b950 tx=0x7f88e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:32.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.468+0000 7f88f37fe640 1 -- 172.21.15.190:0/1530271104 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f88ec016110 con 0x7f88fc139980 2024-08-19T10:42:32.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.468+0000 7f88f37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:32.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.468+0000 7f88f37fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:32.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.636+0000 7f8900f55640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-08-19T10:42:32.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.636+0000 7f8900f55640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-08-19T10:42:32.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.636+0000 7f8900f55640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:32.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:32.636+0000 7f8900f55640 1 -- 172.21.15.190:0/1530271104 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "cool", "pg_num": 2} v 0) -- 0x7f88fc06c8d0 con 0x7f88fc139980 2024-08-19T10:42:33.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.328+0000 7f88f37fe640 1 -- 172.21.15.190:0/1530271104 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}]=0 pool 'cool' created v146) ==== 111+0+0 (secure 0 0 0) 0x7f88ec004460 con 0x7f88fc139980 2024-08-19T10:42:33.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.328+0000 7f88f37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-08-19T10:42:33.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.328+0000 7f88f37fe640 10 monclient: _finish_command 2 = system:0 pool 'cool' created 2024-08-19T10:42:33.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.428+0000 7f8900f55640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-08-19T10:42:33.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.428+0000 7f8900f55640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-08-19T10:42:33.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.428+0000 7f8900f55640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:33.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.428+0000 7f8900f55640 1 -- 172.21.15.190:0/1530271104 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "cool", "pg_num": 2} v 0) -- 0x7f88fc0a1560 con 0x7f88fc139980 2024-08-19T10:42:33.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f88f37fe640 1 -- 172.21.15.190:0/1530271104 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}]=0 pool 'cool' already exists v146) ==== 118+0+0 (secure 0 0 0) 0x7f88ec0bd3d0 con 0x7f88fc139980 2024-08-19T10:42:33.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f88f37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "cool", "pg_num": 2}] 2024-08-19T10:42:33.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f88f37fe640 10 monclient: _finish_command 3 = system:0 pool 'cool' already exists 2024-08-19T10:42:33.436 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'cool' already exists 2024-08-19T10:42:33.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 1 -- 172.21.15.190:0/1530271104 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88dc03be00 msgr2=0x7f88dc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:33.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88dc03be00 0x7f88dc03e2c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f88e400b950 tx=0x7f88e4008040 comp rx=0 tx=0).stop 2024-08-19T10:42:33.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 10 monclient: shutdown 2024-08-19T10:42:33.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:33.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 1 -- 172.21.15.190:0/1530271104 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88fc139980 msgr2=0x7f88fc133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:33.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88fc139980 0x7f88fc133dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f88ec01a7f0 tx=0x7f88ec018ee0 comp rx=0 tx=0).stop 2024-08-19T10:42:33.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 1 -- 172.21.15.190:0/1530271104 shutdown_connections 2024-08-19T10:42:33.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88dc03be00 0x7f88dc03e2c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:33.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f88fc13a690 0x7f88fc13aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:33.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88fc139980 0x7f88fc133dd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:33.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 1 --2- 172.21.15.190:0/1530271104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88fc0a5c70 0x7f88fc133890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:33.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.432+0000 7f8900f55640 1 -- 172.21.15.190:0/1530271104 >> 172.21.15.190:0/1530271104 conn(0x7f88fc0abde0 msgr2=0x7f88fc0ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:33.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.436+0000 7f8900f55640 1 -- 172.21.15.190:0/1530271104 shutdown_connections 2024-08-19T10:42:33.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.436+0000 7f8900f55640 1 -- 172.21.15.190:0/1530271104 wait complete. 2024-08-19T10:42:33.454 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool create cold 2 2024-08-19T10:42:33.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.568+0000 7f72148c2640 10 monclient: get_monmap_and_config 2024-08-19T10:42:33.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.568+0000 7f72148c2640 10 monclient: build_initial_monmap 2024-08-19T10:42:33.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.568+0000 7f72148c2640 10 monclient: monmap: 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:33.574146+0000 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:33.574146+0000 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:33.574 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:33.575 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:33.575 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:33.575 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:33.575 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:33.575 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:33.575 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:33.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.568+0000 7f72148c2640 1 Processor -- start 2024-08-19T10:42:33.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.568+0000 7f72148c2640 1 -- start start 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.568+0000 7f72148c2640 10 monclient: init 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 10 monclient: _add_conns ranks=[5,3,0,2,9,6,7,8,4,10,1] 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f7210139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 10 monclient(hunting): picked mon.noname-h con 0x7f7210139980 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f721013a2a0 0x7f72100a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 10 monclient(hunting): picked mon.noname-g con 0x7f721013a2a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f72100a5c70 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72100a8470 con 0x7f72100a5c70 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f721009ca60 con 0x7f721013a2a0 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:33.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f72100a3640 con 0x7f7210139980 2024-08-19T10:42:33.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:33.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720dd9b640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f721013a2a0 msgr2=0x7f72100a5730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:33.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f72148c2640 10 monclient(hunting): authenticate will time out at 1199.144263s 2024-08-19T10:42:33.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720dd9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f721013a2a0 0x7f72100a5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:33.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f7210139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:33.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f7210139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:56974/0 (socket says 172.21.15.190:56974) 2024-08-19T10:42:33.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 1 -- 172.21.15.190:0/685452273 learned_addr learned my addr 172.21.15.190:0/685452273 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:33.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 10 monclient(hunting): get_auth_request con 0x7f7210139980 auth_method 0 2024-08-19T10:42:33.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:33.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:33.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 10 monclient(hunting): handle_auth_done global_id 5621 payload 274 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 1 -- 172.21.15.190:0/685452273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f721013a2a0 msgr2=0x7f72100a5730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 1 --2- 172.21.15.190:0/685452273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f721013a2a0 0x7f72100a5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 1 -- 172.21.15.190:0/685452273 >> v1:172.21.15.89:6789/0 conn(0x7f72100a5c70 legacy=0x7f72100a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 1 monclient: found mon.noname-h 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:33.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.572+0000 7f720e59c640 1 -- 172.21.15.190:0/685452273 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72100a3850 con 0x7f7210139980 2024-08-19T10:42:33.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720e59c640 1 -- 172.21.15.190:0/685452273 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 msgr2=0x7f7210139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720e59c640 1 --2- 172.21.15.190:0/685452273 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f7210139d60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71f80098a0 tx=0x7f71f801c430 comp rx=0 tx=0).ready entity=mon.5 client_cookie=3b5b28be4e3402c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 1 -- 172.21.15.190:0/685452273 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f71f80b0070 con 0x7f7210139980 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 10 monclient: dump: 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:33.585 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 10 monclient: _finish_auth 0 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:33.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 5 monclient: authenticate success, global_id 5621 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 1 -- 172.21.15.190:0/685452273 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f80aa810 con 0x7f7210139980 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 1 -- 172.21.15.190:0/685452273 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f71f80aae30 con 0x7f7210139980 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720d59a640 10 monclient: dump: 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:33.587 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:33.588 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:33.588 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:33.588 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:33.588 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:33.588 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:33.588 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:33.588 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:33.588 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 4 set_mon_vals no callback set 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient: shutdown 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 -- 172.21.15.190:0/685452273 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 msgr2=0x7f7210139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 --2- 172.21.15.190:0/685452273 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f7210139d60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71f80098a0 tx=0x7f71f801c430 comp rx=0 tx=0).stop 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 -- 172.21.15.190:0/685452273 shutdown_connections 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 --2- 172.21.15.190:0/685452273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f721013a2a0 0x7f72100a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 --2- 172.21.15.190:0/685452273 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f7210139d60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 -- 172.21.15.190:0/685452273 >> 172.21.15.190:0/685452273 conn(0x7f72100abde0 msgr2=0x7f72100ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 -- 172.21.15.190:0/685452273 shutdown_connections 2024-08-19T10:42:33.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 -- 172.21.15.190:0/685452273 wait complete. 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient: build_initial_monmap 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient: monmap: 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:33.590 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 Processor -- start 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 -- start start 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient: init 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient: _add_conns ranks=[4,5,2,6,7,8,1,0,3] 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72100a5c70 0x7f721013a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient(hunting): picked mon.noname-c con 0x7f72100a5c70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:33.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f721013abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient(hunting): picked mon.noname-h con 0x7f7210139980 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f721013b110 0x7f7210133540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient(hunting): picked mon.noname-b con 0x7f721013b110 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7210144280 con 0x7f721013b110 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f7210144100 con 0x7f72100a5c70 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7210144400 con 0x7f7210139980 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f72148c2640 10 monclient(hunting): authenticate will time out at 1199.151842s 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720dd9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f721013abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:33.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.576+0000 7f720dd9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f721013abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:56988/0 (socket says 172.21.15.190:56988) 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 1 -- 172.21.15.190:0/2219273283 learned_addr learned my addr 172.21.15.190:0/2219273283 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 10 monclient(hunting): get_auth_request con 0x7f7210139980 auth_method 0 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720e59c640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72100a5c70 0x7f721013a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 10 monclient(hunting): handle_auth_done global_id 5630 payload 824 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 1 -- 172.21.15.190:0/2219273283 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72100a5c70 msgr2=0x7f721013a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720ed9d640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f721013b110 0x7f7210133540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:33.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72100a5c70 0x7f721013a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:33.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 1 -- 172.21.15.190:0/2219273283 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f721013b110 msgr2=0x7f7210133540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:33.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f721013b110 0x7f7210133540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:33.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:33.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 1 monclient: found mon.noname-h 2024-08-19T10:42:33.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:33.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:33.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 1 -- 172.21.15.190:0/2219273283 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721013ceb0 con 0x7f7210139980 2024-08-19T10:42:33.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720e59c640 10 monclient: get_auth_request con 0x7f72100a5c70 auth_method 0 2024-08-19T10:42:33.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720ed9d640 10 monclient: get_auth_request con 0x7f721013b110 auth_method 0 2024-08-19T10:42:33.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720e59c640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72100a5c70 0x7f721013a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:33.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720ed9d640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f721013b110 0x7f7210133540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:33.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 1 -- 172.21.15.190:0/2219273283 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 msgr2=0x7f721013abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:33.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720dd9b640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f721013abd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f720800b550 tx=0x7f720800ba10 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:33.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 1 -- 172.21.15.190:0/2219273283 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7208003de0 con 0x7f7210139980 2024-08-19T10:42:33.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:33.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:33.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 10 monclient: dump: 2024-08-19T10:42:33.598 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:33.598 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:33.598 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:33.599 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 10 monclient: _finish_auth 0 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 1 -- 172.21.15.190:0/2219273283 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7208015070 con 0x7f7210139980 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 1 -- 172.21.15.190:0/2219273283 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7208010480 con 0x7f7210139980 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f72148c2640 5 monclient: authenticate success, global_id 5630 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 10 monclient: dump: 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:33.600 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:33.601 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:33.601 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:33.602 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:33.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f72148c2640 10 monclient: _renew_subs 2024-08-19T10:42:33.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f720cd99640 4 set_mon_vals no callback set 2024-08-19T10:42:33.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f72148c2640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:33.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f72148c2640 1 -- 172.21.15.190:0/2219273283 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721013d190 con 0x7f7210139980 2024-08-19T10:42:33.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f72148c2640 10 monclient: _renew_subs 2024-08-19T10:42:33.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f72148c2640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:33.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f72148c2640 1 -- 172.21.15.190:0/2219273283 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f72100a1180 con 0x7f7210139980 2024-08-19T10:42:33.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f72148c2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:33.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f72148c2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:33.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f72148c2640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:33.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f72148c2640 1 -- 172.21.15.190:0/2219273283 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7210139de0 con 0x7f7210139980 2024-08-19T10:42:33.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.580+0000 7f7206ffd640 1 -- 172.21.15.190:0/2219273283 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f72080050f0 con 0x7f7210139980 2024-08-19T10:42:33.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.584+0000 7f7206ffd640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f71f403be30 0x7f71f403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:33.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.584+0000 7f7206ffd640 1 -- 172.21.15.190:0/2219273283 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(146..146 src has 1..146) ==== 7632+0+0 (secure 0 0 0) 0x7f720804f490 con 0x7f7210139980 2024-08-19T10:42:33.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.584+0000 7f720e59c640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f71f403be30 0x7f71f403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:33.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.584+0000 7f720e59c640 10 monclient: get_auth_request con 0x7f71f403be30 auth_method 0 2024-08-19T10:42:33.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.584+0000 7f720e59c640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f71f403be30 0x7f71f403e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f71f80098a0 tx=0x7f71f80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:33.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.584+0000 7f7206ffd640 1 -- 172.21.15.190:0/2219273283 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:42:33.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.588+0000 7f7206ffd640 1 -- 172.21.15.190:0/2219273283 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7208010620 con 0x7f7210139980 2024-08-19T10:42:33.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.588+0000 7f7206ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:33.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.588+0000 7f7206ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:33.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.748+0000 7f72148c2640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-08-19T10:42:33.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.748+0000 7f72148c2640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-08-19T10:42:33.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.748+0000 7f72148c2640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:33.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:33.748+0000 7f72148c2640 1 -- 172.21.15.190:0/2219273283 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "cold", "pg_num": 2} v 0) -- 0x7f7210134670 con 0x7f7210139980 2024-08-19T10:42:34.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.336+0000 7f7206ffd640 1 -- 172.21.15.190:0/2219273283 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}]=0 pool 'cold' created v147) ==== 111+0+0 (secure 0 0 0) 0x7f720800e340 con 0x7f7210139980 2024-08-19T10:42:34.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.336+0000 7f7206ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-08-19T10:42:34.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.336+0000 7f7206ffd640 10 monclient: _finish_command 2 = system:0 pool 'cold' created 2024-08-19T10:42:34.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.444+0000 7f72148c2640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-08-19T10:42:34.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.444+0000 7f72148c2640 10 monclient: _send_command 3 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-08-19T10:42:34.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.444+0000 7f72148c2640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:34.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.444+0000 7f72148c2640 1 -- 172.21.15.190:0/2219273283 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "cold", "pg_num": 2} v 0) -- 0x7f72100008e0 con 0x7f7210139980 2024-08-19T10:42:34.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f7206ffd640 1 -- 172.21.15.190:0/2219273283 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}]=0 pool 'cold' already exists v147) ==== 118+0+0 (secure 0 0 0) 0x7f720800db90 con 0x7f7210139980 2024-08-19T10:42:34.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f7206ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool create", "pool": "cold", "pg_num": 2}] 2024-08-19T10:42:34.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f7206ffd640 10 monclient: _finish_command 3 = system:0 pool 'cold' already exists 2024-08-19T10:42:34.452 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'cold' already exists 2024-08-19T10:42:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 -- 172.21.15.190:0/2219273283 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f71f403be30 msgr2=0x7f71f403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f71f403be30 0x7f71f403e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f71f80098a0 tx=0x7f71f80a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 10 monclient: shutdown 2024-08-19T10:42:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 -- 172.21.15.190:0/2219273283 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 msgr2=0x7f721013abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f721013abd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f720800b550 tx=0x7f720800ba10 comp rx=0 tx=0).stop 2024-08-19T10:42:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 -- 172.21.15.190:0/2219273283 shutdown_connections 2024-08-19T10:42:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f71f403be30 0x7f71f403e2f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f721013b110 0x7f7210133540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7210139980 0x7f721013abd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:34.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 --2- 172.21.15.190:0/2219273283 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f72100a5c70 0x7f721013a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:34.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 -- 172.21.15.190:0/2219273283 >> 172.21.15.190:0/2219273283 conn(0x7f72100abde0 msgr2=0x7f7210135120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:34.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 -- 172.21.15.190:0/2219273283 shutdown_connections 2024-08-19T10:42:34.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.448+0000 7f72148c2640 1 -- 172.21.15.190:0/2219273283 wait complete. 2024-08-19T10:42:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool set cold size 2 2024-08-19T10:42:34.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient: get_monmap_and_config 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient: build_initial_monmap 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient: monmap: 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:34.588365+0000 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:34.588365+0000 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:34.588 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:34.589 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:34.589 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:34.589 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:34.589 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:34.589 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:34.589 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:34.589 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:34.589 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:34.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 1 Processor -- start 2024-08-19T10:42:34.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 1 -- start start 2024-08-19T10:42:34.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient: init 2024-08-19T10:42:34.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:34.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient: _add_conns ranks=[6,9,0,7,3,8,5,10,4,2,1] 2024-08-19T10:42:34.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90d009b8d0 0x7f90d009bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:34.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient(hunting): picked mon.noname-d con 0x7f90d009b8d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:34.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90d009dfb0 0x7f90d00a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:34.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient(hunting): picked mon.noname-a con 0x7f90d009dfb0 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:34.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f90d00a42c0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:34.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90d009d2f0 con 0x7f90d00a42c0 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f90d009d8b0 con 0x7f90d009dfb0 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f90d009d5d0 con 0x7f90d009b8d0 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90d5e0e640 10 monclient(hunting): authenticate will time out at 1200.159224s 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90cf7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90d009b8d0 0x7f90d009bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90ceffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90d009dfb0 0x7f90d00a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90cf7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90d009b8d0 0x7f90d009bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:52510/0 (socket says 172.21.15.190:52510) 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90cf7fe640 1 -- 172.21.15.190:0/1564619697 learned_addr learned my addr 172.21.15.190:0/1564619697 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90cf7fe640 10 monclient(hunting): get_auth_request con 0x7f90d009b8d0 auth_method 0 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:34.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90cf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90ceffd640 10 monclient(hunting): get_auth_request con 0x7f90d009dfb0 auth_method 0 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90ceffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.584+0000 7f90ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.588+0000 7f90cf7fe640 10 monclient(hunting): handle_auth_done global_id 5415 payload 274 2024-08-19T10:42:34.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.588+0000 7f90cf7fe640 1 -- 172.21.15.190:0/1564619697 >> v2:172.21.15.89:3300/0 conn(0x7f90d009dfb0 msgr2=0x7f90d00a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:34.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.588+0000 7f90cf7fe640 1 --2- 172.21.15.190:0/1564619697 >> v2:172.21.15.89:3300/0 conn(0x7f90d009dfb0 0x7f90d00a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:34.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.588+0000 7f90cf7fe640 1 -- 172.21.15.190:0/1564619697 >> v1:172.21.15.89:6789/0 conn(0x7f90d00a42c0 legacy=0x7f90d0140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:34.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.588+0000 7f90cf7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:34.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.588+0000 7f90cf7fe640 1 monclient: found mon.noname-d 2024-08-19T10:42:34.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.588+0000 7f90cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:34.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.588+0000 7f90cf7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:34.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.588+0000 7f90cf7fe640 1 -- 172.21.15.190:0/1564619697 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d009c950 con 0x7f90d009b8d0 2024-08-19T10:42:34.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90cf7fe640 1 -- 172.21.15.190:0/1564619697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90d009b8d0 msgr2=0x7f90d009bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:34.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90cf7fe640 1 --2- 172.21.15.190:0/1564619697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90d009b8d0 0x7f90d009bcb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009e50 tx=0x7f90c001c980 comp rx=0 tx=0).ready entity=mon.6 client_cookie=74c847733e7681ec server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:34.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 1 -- 172.21.15.190:0/1564619697 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90c00090d0 con 0x7f90d009b8d0 2024-08-19T10:42:34.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:34.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:34.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 10 monclient: dump: 2024-08-19T10:42:34.597 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:34.597 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:34.597 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:34.597 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:34.597 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:34.597 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:34.597 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:34.598 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:34.598 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:34.598 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:34.598 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:34.598 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:34.598 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:34.598 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:34.598 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:34.598 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:34.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:34.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 1 -- 172.21.15.190:0/1564619697 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c00a5c50 con 0x7f90d009b8d0 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 1 -- 172.21.15.190:0/1564619697 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90c00a5e00 con 0x7f90d009b8d0 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90ce7fc640 10 monclient: dump: 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:34.599 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 5 monclient: authenticate success, global_id 5415 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:34.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 4 set_mon_vals no callback set 2024-08-19T10:42:34.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient: shutdown 2024-08-19T10:42:34.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:34.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 -- 172.21.15.190:0/1564619697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90d009b8d0 msgr2=0x7f90d009bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:34.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 --2- 172.21.15.190:0/1564619697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90d009b8d0 0x7f90d009bcb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009e50 tx=0x7f90c001c980 comp rx=0 tx=0).stop 2024-08-19T10:42:34.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 -- 172.21.15.190:0/1564619697 shutdown_connections 2024-08-19T10:42:34.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 --2- 172.21.15.190:0/1564619697 >> v2:172.21.15.89:3300/0 conn(0x7f90d009dfb0 0x7f90d00a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:34.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 --2- 172.21.15.190:0/1564619697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f90d009b8d0 0x7f90d009bcb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:34.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 -- 172.21.15.190:0/1564619697 >> 172.21.15.190:0/1564619697 conn(0x7f90d00a4e70 msgr2=0x7f90d00a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:34.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 -- 172.21.15.190:0/1564619697 shutdown_connections 2024-08-19T10:42:34.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 -- 172.21.15.190:0/1564619697 wait complete. 2024-08-19T10:42:34.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient: build_initial_monmap 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient: monmap: 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:34.606 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 Processor -- start 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 -- start start 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient: init 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient: _add_conns ranks=[3,8,0,4,7,2,5,6,1] 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90d009b8d0 0x7f90d013c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient(hunting): picked mon.noname-g con 0x7f90d009b8d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:34.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90d009dfb0 0x7f90d013ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient(hunting): picked mon.noname-e con 0x7f90d009dfb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90d00a42c0 0x7f90d0092400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient(hunting): picked mon.noname-a con 0x7f90d00a42c0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f90d009e390 con 0x7f90d00a42c0 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f90d00a5c80 con 0x7f90d009b8d0 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f90d009e510 con 0x7f90d009dfb0 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90cf7fe640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90d009b8d0 msgr2=0x7f90d013c900 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:42:34.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90d5e0e640 10 monclient(hunting): authenticate will time out at 1200.167888s 2024-08-19T10:42:34.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.592+0000 7f90cf7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90d009b8d0 0x7f90d013c900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:34.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90d009dfb0 0x7f90d013ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:34.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90cffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90d00a42c0 0x7f90d0092400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:34.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90d009dfb0 0x7f90d013ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:43142/0 (socket says 172.21.15.190:43142) 2024-08-19T10:42:34.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 1 -- 172.21.15.190:0/3668604399 learned_addr learned my addr 172.21.15.190:0/3668604399 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:34.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 10 monclient(hunting): get_auth_request con 0x7f90d009dfb0 auth_method 0 2024-08-19T10:42:34.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:34.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:34.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:34.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90cffff640 10 monclient(hunting): get_auth_request con 0x7f90d00a42c0 auth_method 0 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90cffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90cffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 10 monclient(hunting): handle_auth_done global_id 5381 payload 824 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 1 -- 172.21.15.190:0/3668604399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90d009b8d0 msgr2=0x7f90d013c900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:34.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90d009b8d0 0x7f90d013c900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:34.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 1 -- 172.21.15.190:0/3668604399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90d00a42c0 msgr2=0x7f90d0092400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:34.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90d00a42c0 0x7f90d0092400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:34.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 10 monclient: _finish_hunting 0 2024-08-19T10:42:34.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 1 monclient: found mon.noname-e 2024-08-19T10:42:34.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:34.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:34.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 1 -- 172.21.15.190:0/3668604399 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d00a3b30 con 0x7f90d009dfb0 2024-08-19T10:42:34.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90cffff640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90d00a42c0 0x7f90d0092400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:34.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 1 -- 172.21.15.190:0/3668604399 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90d009dfb0 msgr2=0x7f90d013ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:34.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90ceffd640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90d009dfb0 0x7f90d013ce40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90c400e930 tx=0x7f90c400edf0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 1 -- 172.21.15.190:0/3668604399 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90c4017070 con 0x7f90d009dfb0 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 10 monclient: dump: 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:34.612 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:34.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90d5e0e640 5 monclient: authenticate success, global_id 5381 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90d5e0e640 10 monclient: _renew_subs 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90d5e0e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90d5e0e640 1 -- 172.21.15.190:0/3668604399 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d00744e0 con 0x7f90d009dfb0 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90d5e0e640 10 monclient: _renew_subs 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90d5e0e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90d5e0e640 1 -- 172.21.15.190:0/3668604399 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d00946d0 con 0x7f90d009dfb0 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 1 -- 172.21.15.190:0/3668604399 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c400cc10 con 0x7f90d009dfb0 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 1 -- 172.21.15.190:0/3668604399 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90c400cdc0 con 0x7f90d009dfb0 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90cdffb640 4 set_mon_vals no callback set 2024-08-19T10:42:34.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.596+0000 7f90b7fff640 10 monclient: dump: 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:34.615 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.600+0000 7f90d5e0e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.600+0000 7f90d5e0e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.600+0000 7f90d5e0e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.600+0000 7f90d5e0e640 1 -- 172.21.15.190:0/3668604399 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b0005090 con 0x7f90d009dfb0 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.600+0000 7f90b7fff640 1 -- 172.21.15.190:0/3668604399 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f90c401a070 con 0x7f90d009dfb0 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.600+0000 7f90b7fff640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f90bc03be00 0x7f90bc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.600+0000 7f90b7fff640 1 -- 172.21.15.190:0/3668604399 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(147..147 src has 1..147) ==== 7975+0+0 (secure 0 0 0) 0x7f90c400f040 con 0x7f90d009dfb0 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.600+0000 7f90cf7fe640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f90bc03be00 0x7f90bc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:34.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.600+0000 7f90cf7fe640 10 monclient: get_auth_request con 0x7f90bc03be00 auth_method 0 2024-08-19T10:42:34.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.604+0000 7f90cf7fe640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f90bc03be00 0x7f90bc03e2c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f90c00ab000 tx=0x7f90c00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:34.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.612+0000 7f90b7fff640 1 -- 172.21.15.190:0/3668604399 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f90d00a3b30 con 0x7f90d009dfb0 2024-08-19T10:42:34.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.612+0000 7f90b7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:34.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.612+0000 7f90b7fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:34.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.788+0000 7f90d5e0e640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-08-19T10:42:34.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.788+0000 7f90d5e0e640 10 monclient: _send_command 2 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-08-19T10:42:34.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.788+0000 7f90d5e0e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:34.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:34.788+0000 7f90d5e0e640 1 -- 172.21.15.190:0/3668604399 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"} v 0) -- 0x7f90b0005650 con 0x7f90d009dfb0 2024-08-19T10:42:34.879 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-08-19T10:42:34.879 INFO:tasks.ceph.mon.g:Restarting daemon 2024-08-19T10:42:34.879 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-08-19T10:42:34.882 INFO:tasks.ceph.mon.g:Started 2024-08-19T10:42:34.882 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:42:34.882 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:42:34.968 INFO:tasks.ceph.mon.g.smithi190.stderr:2024-08-19T10:42:34.964+0000 7eff3a123dc0 -1 mon.g@-1(???).log v0 log_external_backlog rewinding external_log_to from 116 -> 0 (sync_force? mon rebuild?) 2024-08-19T10:42:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient: get_monmap_and_config 2024-08-19T10:42:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient: build_initial_monmap 2024-08-19T10:42:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient: monmap: 2024-08-19T10:42:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:35.014289+0000 2024-08-19T10:42:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:35.014289+0000 2024-08-19T10:42:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:35.014 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:35.015 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:35.015 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:35.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 1 Processor -- start 2024-08-19T10:42:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 1 -- start start 2024-08-19T10:42:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient: init 2024-08-19T10:42:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient: _add_conns ranks=[8,3,7,6,4,0,9,5,10,1,2] 2024-08-19T10:42:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48f80a5270 0x7f48f80a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient(hunting): picked mon.noname-e con 0x7f48f80a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a6d30 0x7f48f80a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient(hunting): picked mon.noname-g con 0x7f48f80a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48f81395b0 0x7f48f81399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient(hunting): picked mon.noname-i con 0x7f48f81395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:35.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f48f80a7650 con 0x7f48f80a6d30 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f48f80a8710 con 0x7f48f81395b0 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f48f80a5a40 con 0x7f48f80a5270 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fee8e640 10 monclient(hunting): authenticate will time out at 1204.831038s 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48f80a5270 0x7f48f80a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48f80a5270 0x7f48f80a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:39388/0 (socket says 172.21.15.89:39388) 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 1 -- 172.21.15.89:0/3048577159 learned_addr learned my addr 172.21.15.89:0/3048577159 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 10 monclient(hunting): get_auth_request con 0x7f48f80a5270 auth_method 0 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:35.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fd42b640 1 --2- 172.21.15.89:0/3048577159 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48f81395b0 0x7f48f81399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48f7fff640 1 --2- 172.21.15.89:0/3048577159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a6d30 0x7f48f80a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fd42b640 10 monclient(hunting): get_auth_request con 0x7f48f81395b0 auth_method 0 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fd42b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fd42b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fd42b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48f7fff640 10 monclient(hunting): get_auth_request con 0x7f48f80a6d30 auth_method 0 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:35.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48f7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 10 monclient(hunting): handle_auth_done global_id 5390 payload 274 2024-08-19T10:42:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 1 -- 172.21.15.89:0/3048577159 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48f81395b0 msgr2=0x7f48f81399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 1 --2- 172.21.15.89:0/3048577159 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48f81395b0 0x7f48f81399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 1 -- 172.21.15.89:0/3048577159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a6d30 msgr2=0x7f48f80a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 1 --2- 172.21.15.89:0/3048577159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a6d30 0x7f48f80a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 10 monclient: _finish_hunting 0 2024-08-19T10:42:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 1 monclient: found mon.noname-e 2024-08-19T10:42:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:35.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.011+0000 7f48fcc2a640 1 -- 172.21.15.89:0/3048577159 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f80a81d0 con 0x7f48f80a5270 2024-08-19T10:42:35.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.015+0000 7f48fcc2a640 1 -- 172.21.15.89:0/3048577159 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48f80a5270 msgr2=0x7f48f80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:35.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.015+0000 7f48fcc2a640 1 --2- 172.21.15.89:0/3048577159 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48f80a5270 0x7f48f80a67f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f48e80098a0 tx=0x7f48e801c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f5685ede58bb15ae server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:35.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 1 -- 172.21.15.89:0/3048577159 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48e80b0070 con 0x7f48f80a5270 2024-08-19T10:42:35.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:35.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:35.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 10 monclient: dump: 2024-08-19T10:42:35.024 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:35.024 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:35.024 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:35.024 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:35.024 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 1 -- 172.21.15.89:0/3048577159 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e80aa810 con 0x7f48f80a5270 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 1 -- 172.21.15.89:0/3048577159 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48e80aae30 con 0x7f48f80a5270 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48fee8e640 5 monclient: authenticate success, global_id 5390 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48fee8e640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48fee8e640 4 set_mon_vals no callback set 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.019+0000 7f48f77fe640 10 monclient: dump: 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:35.027 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:35.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient: shutdown 2024-08-19T10:42:35.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:35.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 -- 172.21.15.89:0/3048577159 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48f80a5270 msgr2=0x7f48f80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:35.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 --2- 172.21.15.89:0/3048577159 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48f80a5270 0x7f48f80a67f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f48e80098a0 tx=0x7f48e801c440 comp rx=0 tx=0).stop 2024-08-19T10:42:35.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 -- 172.21.15.89:0/3048577159 shutdown_connections 2024-08-19T10:42:35.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 --2- 172.21.15.89:0/3048577159 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48f81395b0 0x7f48f81399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:35.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 --2- 172.21.15.89:0/3048577159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a6d30 0x7f48f80a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:35.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 --2- 172.21.15.89:0/3048577159 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48f80a5270 0x7f48f80a67f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:35.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 -- 172.21.15.89:0/3048577159 >> 172.21.15.89:0/3048577159 conn(0x7f48f8069a80 msgr2=0x7f48f80abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:35.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 -- 172.21.15.89:0/3048577159 shutdown_connections 2024-08-19T10:42:35.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 -- 172.21.15.89:0/3048577159 wait complete. 2024-08-19T10:42:35.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient: build_initial_monmap 2024-08-19T10:42:35.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient: monmap: 2024-08-19T10:42:35.034 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:35.035 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 Processor -- start 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 -- start start 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient: init 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient: _add_conns ranks=[3,1,4,6,8,0,7,2,5] 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a5270 0x7f48f80a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient(hunting): picked mon.noname-g con 0x7f48f80a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f80a6d30 0x7f48f80a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient(hunting): picked mon.noname-f con 0x7f48f80a6d30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48f81395b0 0x7f48f813cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:35.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient(hunting): picked mon.noname-c con 0x7f48f81395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48f80a57d0 con 0x7f48f80a6d30 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f48f80a5650 con 0x7f48f80a5270 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f48f8143230 con 0x7f48f81395b0 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fee8e640 10 monclient(hunting): authenticate will time out at 1204.842423s 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48f81395b0 0x7f48f813cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48f81395b0 0x7f48f813cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:57606/0 (socket says 172.21.15.89:57606) 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 1 -- 172.21.15.89:0/2471658119 learned_addr learned my addr 172.21.15.89:0/2471658119 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 10 monclient(hunting): get_auth_request con 0x7f48f81395b0 auth_method 0 2024-08-19T10:42:35.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fcc2a640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a5270 0x7f48f80a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48f7fff640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f80a6d30 0x7f48f80a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fcc2a640 10 monclient(hunting): get_auth_request con 0x7f48f80a5270 auth_method 0 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fcc2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fcc2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fcc2a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48f7fff640 10 monclient(hunting): get_auth_request con 0x7f48f80a6d30 auth_method 0 2024-08-19T10:42:35.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48f7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 10 monclient(hunting): handle_auth_done global_id 5539 payload 824 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fcc2a640 1 -- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a5270 msgr2=0x7f48f80a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fcc2a640 1 -- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a5270 msgr2=0x7f48f80a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fcc2a640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a5270 0x7f48f80a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 1 -- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a5270 msgr2=0x7f48f80a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fcc2a640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a5270 0x7f48f80a1f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a5270 0x7f48f80a1f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 1 -- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f80a6d30 msgr2=0x7f48f80a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f80a6d30 0x7f48f80a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:35.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 1 monclient: found mon.noname-c 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48fd42b640 1 -- 172.21.15.89:0/2471658119 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f8135890 con 0x7f48f81395b0 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.023+0000 7f48f7fff640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f80a6d30 0x7f48f80a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fd42b640 1 -- 172.21.15.89:0/2471658119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48f81395b0 msgr2=0x7f48f813cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fd42b640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48f81395b0 0x7f48f813cf60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48f0007e30 tx=0x7f48f000e4b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 1 -- 172.21.15.89:0/2471658119 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48f0013a00 con 0x7f48f81395b0 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 10 monclient: dump: 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:35.040 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:35.041 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:35.041 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:35.041 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:35.041 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:35.041 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:35.041 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:35.041 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:35.041 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:35.041 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 10 monclient: _finish_auth 0 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 1 -- 172.21.15.89:0/2471658119 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48f0019070 con 0x7f48f81395b0 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 1 -- 172.21.15.89:0/2471658119 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48f0013bc0 con 0x7f48f81395b0 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:35.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 10 monclient: dump: 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:35.043 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f6ffd640 4 set_mon_vals no callback set 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fee8e640 5 monclient: authenticate success, global_id 5539 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fee8e640 10 monclient: _renew_subs 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fee8e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fee8e640 1 -- 172.21.15.89:0/2471658119 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f8140760 con 0x7f48f81395b0 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fee8e640 10 monclient: _renew_subs 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fee8e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fee8e640 1 -- 172.21.15.89:0/2471658119 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f809d6c0 con 0x7f48f81395b0 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fee8e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fee8e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:35.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fee8e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:35.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 1 -- 172.21.15.89:0/2471658119 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f48f0013590 con 0x7f48f81395b0 2024-08-19T10:42:35.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48fee8e640 1 -- 172.21.15.89:0/2471658119 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f80a2980 con 0x7f48f81395b0 2024-08-19T10:42:35.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f48d803be00 0x7f48d803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:35.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.027+0000 7f48f57fa640 1 -- 172.21.15.89:0/2471658119 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(147..147 src has 1..147) ==== 7975+0+0 (secure 0 0 0) 0x7f48f004ffc0 con 0x7f48f81395b0 2024-08-19T10:42:35.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.031+0000 7f48fcc2a640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f48d803be00 0x7f48d803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:35.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.031+0000 7f48fcc2a640 10 monclient: get_auth_request con 0x7f48d803be00 auth_method 0 2024-08-19T10:42:35.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.031+0000 7f48fcc2a640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f48d803be00 0x7f48d803e2c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f48e80ab000 tx=0x7f48e80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:35.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.035+0000 7f48f57fa640 1 -- 172.21.15.89:0/2471658119 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f48f8140760 con 0x7f48f81395b0 2024-08-19T10:42:35.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.035+0000 7f48f57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:35.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.035+0000 7f48f57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.271+0000 7f48fee8e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:42:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.271+0000 7f48fee8e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:42:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.271+0000 7f48fee8e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:35.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:35.271+0000 7f48fee8e640 1 -- 172.21.15.89:0/2471658119 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f48f80a73a0 con 0x7f48f81395b0 2024-08-19T10:42:37.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:37.592+0000 7f90b6ffd640 10 monclient: tick 2024-08-19T10:42:37.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:37.592+0000 7f90b6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:37.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:37.596+0000 7f90b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:37.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:37.596+0000 7f90b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:38.023+0000 7f48effff640 10 monclient: tick 2024-08-19T10:42:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:38.023+0000 7f48effff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:38.023+0000 7f48effff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:38.023+0000 7f48effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:38.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:38.596+0000 7f90b6ffd640 10 monclient: tick 2024-08-19T10:42:38.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:38.596+0000 7f90b6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:38.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:38.596+0000 7f90b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:39.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:39.023+0000 7f48effff640 10 monclient: tick 2024-08-19T10:42:39.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:39.023+0000 7f48effff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:39.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:39.023+0000 7f48effff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:39.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:39.596+0000 7f90b6ffd640 10 monclient: tick 2024-08-19T10:42:39.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:39.596+0000 7f90b6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:39.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:39.596+0000 7f90b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:40.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.023+0000 7f48effff640 10 monclient: tick 2024-08-19T10:42:40.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.023+0000 7f48effff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:40.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.023+0000 7f48effff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:40.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.067+0000 7f48f57fa640 1 -- 172.21.15.89:0/2471658119 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3186 (secure 0 0 0) 0x7f48f8135890 con 0x7f48f81395b0 2024-08-19T10:42:40.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.067+0000 7f48f57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:42:40.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.067+0000 7f48f57fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:40.072 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":44,"quorum":[0,1,2,4,5,6,7,8],"quorum_names":["a","f","b","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:42:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 -- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f48d803be00 msgr2=0x7f48d803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f48d803be00 0x7f48d803e2c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f48e80ab000 tx=0x7f48e80a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 10 monclient: shutdown 2024-08-19T10:42:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 -- 172.21.15.89:0/2471658119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48f81395b0 msgr2=0x7f48f813cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48f81395b0 0x7f48f813cf60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48f0007e30 tx=0x7f48f000e4b0 comp rx=0 tx=0).stop 2024-08-19T10:42:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 -- 172.21.15.89:0/2471658119 shutdown_connections 2024-08-19T10:42:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f48d803be00 0x7f48d803e2c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f48f81395b0 0x7f48f813cf60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48f80a6d30 0x7f48f80a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 --2- 172.21.15.89:0/2471658119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48f80a5270 0x7f48f80a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 -- 172.21.15.89:0/2471658119 >> 172.21.15.89:0/2471658119 conn(0x7f48f8069a80 msgr2=0x7f48f80ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 -- 172.21.15.89:0/2471658119 shutdown_connections 2024-08-19T10:42:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:40.071+0000 7f48fee8e640 1 -- 172.21.15.89:0/2471658119 wait complete. 2024-08-19T10:42:40.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:40.596+0000 7f90b6ffd640 10 monclient: tick 2024-08-19T10:42:40.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:40.596+0000 7f90b6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:40.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:40.596+0000 7f90b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:41.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:41.596+0000 7f90b6ffd640 10 monclient: tick 2024-08-19T10:42:41.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:41.596+0000 7f90b6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:41.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:41.596+0000 7f90b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:41.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:41.600+0000 7f90b7fff640 1 -- 172.21.15.190:0/3668604399 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}]=0 set pool 12 size to 2 v148) ==== 124+0+0 (secure 0 0 0) 0x7f90c404cca0 con 0x7f90d009dfb0 2024-08-19T10:42:41.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:41.600+0000 7f90b7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-08-19T10:42:41.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:41.600+0000 7f90b7fff640 10 monclient: _finish_command 2 = system:0 set pool 12 size to 2 2024-08-19T10:42:41.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:41.708+0000 7f90d5e0e640 10 start_mon_command cmd=[{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-08-19T10:42:41.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:41.708+0000 7f90d5e0e640 10 monclient: _send_command 3 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-08-19T10:42:41.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:41.708+0000 7f90d5e0e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:41.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:41.708+0000 7f90d5e0e640 1 -- 172.21.15.190:0/3668604399 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"} v 0) -- 0x7f90b0005b40 con 0x7f90d009dfb0 2024-08-19T10:42:42.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.596+0000 7f90b6ffd640 10 monclient: tick 2024-08-19T10:42:42.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.596+0000 7f90b6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:42.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.596+0000 7f90b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:42.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.608+0000 7f90b7fff640 1 -- 172.21.15.190:0/3668604399 <== mon.8 v2:172.21.15.89:3304/0 8 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}]=0 set pool 12 size to 2 v149) ==== 124+0+0 (secure 0 0 0) 0x7f90c4013c70 con 0x7f90d009dfb0 2024-08-19T10:42:42.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.608+0000 7f90b7fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool set", "pool": "cold", "var": "size", "val": "2"}] 2024-08-19T10:42:42.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.608+0000 7f90b7fff640 10 monclient: _finish_command 3 = system:0 set pool 12 size to 2 2024-08-19T10:42:42.615 INFO:tasks.workunit.client.0.smithi190.stderr:set pool 12 size to 2 2024-08-19T10:42:42.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 -- 172.21.15.190:0/3668604399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f90bc03be00 msgr2=0x7f90bc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:42.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f90bc03be00 0x7f90bc03e2c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f90c00ab000 tx=0x7f90c00a7070 comp rx=0 tx=0).stop 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 10 monclient: shutdown 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 -- 172.21.15.190:0/3668604399 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90d009dfb0 msgr2=0x7f90d013ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90d009dfb0 0x7f90d013ce40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90c400e930 tx=0x7f90c400edf0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 -- 172.21.15.190:0/3668604399 shutdown_connections 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90d00a42c0 0x7f90d0092400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f90bc03be00 0x7f90bc03e2c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f90d009dfb0 0x7f90d013ce40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 --2- 172.21.15.190:0/3668604399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90d009b8d0 0x7f90d013c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 -- 172.21.15.190:0/3668604399 >> 172.21.15.190:0/3668604399 conn(0x7f90d00a4e70 msgr2=0x7f90d00a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 -- 172.21.15.190:0/3668604399 shutdown_connections 2024-08-19T10:42:42.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.612+0000 7f90d5e0e640 1 -- 172.21.15.190:0/3668604399 wait complete. 2024-08-19T10:42:42.635 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set create-compat 2024-08-19T10:42:42.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.748+0000 7f7ee976d640 10 monclient: get_monmap_and_config 2024-08-19T10:42:42.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.748+0000 7f7ee976d640 10 monclient: build_initial_monmap 2024-08-19T10:42:42.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.748+0000 7f7ee976d640 10 monclient: monmap: 2024-08-19T10:42:42.754 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:42.754 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:42.754 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:42.754842+0000 2024-08-19T10:42:42.754 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:42.754842+0000 2024-08-19T10:42:42.754 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:42.755 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:42.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 1 Processor -- start 2024-08-19T10:42:42.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 1 -- start start 2024-08-19T10:42:42.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 10 monclient: init 2024-08-19T10:42:42.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:42.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 10 monclient: _add_conns ranks=[4,2,5,8,0,6,3,1,10,9,7] 2024-08-19T10:42:42.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ee40a3b90 0x7f7ee40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:42.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 10 monclient(hunting): picked mon.noname-c con 0x7f7ee40a3b90 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:42.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ee40a44b0 0x7f7ee40a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:42.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 10 monclient(hunting): picked mon.noname-b con 0x7f7ee40a44b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:42.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7ee4136570 0x7f7ee41369d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:42.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 10 monclient(hunting): picked mon.noname-h con 0x7f7ee4136570 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee40a04b0 con 0x7f7ee40a44b0 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f7ee40a1ba0 con 0x7f7ee40a3b90 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7ee40a1e80 con 0x7f7ee4136570 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee976d640 10 monclient(hunting): authenticate will time out at 1208.325628s 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7ee4136570 0x7f7ee41369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7ee4136570 0x7f7ee41369d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54596/0 (socket says 172.21.15.190:54596) 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 1 -- 172.21.15.190:0/2074689029 learned_addr learned my addr 172.21.15.190:0/2074689029 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 10 monclient(hunting): get_auth_request con 0x7f7ee4136570 auth_method 0 2024-08-19T10:42:42.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee2ffd640 1 --2- 172.21.15.190:0/2074689029 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ee40a3b90 0x7f7ee40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee2ffd640 10 monclient(hunting): get_auth_request con 0x7f7ee40a3b90 auth_method 0 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee27fc640 1 --2- 172.21.15.190:0/2074689029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ee40a44b0 0x7f7ee40a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 10 monclient(hunting): handle_auth_done global_id 5639 payload 274 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 1 -- 172.21.15.190:0/2074689029 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ee40a3b90 msgr2=0x7f7ee40a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 1 --2- 172.21.15.190:0/2074689029 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ee40a3b90 0x7f7ee40a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 1 -- 172.21.15.190:0/2074689029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ee40a44b0 msgr2=0x7f7ee40a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:42.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 1 --2- 172.21.15.190:0/2074689029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ee40a44b0 0x7f7ee40a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:42.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 1 monclient: found mon.noname-h 2024-08-19T10:42:42.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:42.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:42.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.752+0000 7f7ee37fe640 1 -- 172.21.15.190:0/2074689029 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee409fe50 con 0x7f7ee4136570 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee37fe640 1 -- 172.21.15.190:0/2074689029 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7ee4136570 msgr2=0x7f7ee41369d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee37fe640 1 --2- 172.21.15.190:0/2074689029 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7ee4136570 0x7f7ee41369d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009e80 tx=0x7f7edc01c9a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=5585da4f376927ce server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 1 -- 172.21.15.190:0/2074689029 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7edc0090d0 con 0x7f7ee4136570 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 10 monclient: dump: 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:42.763 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:42.764 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:42.764 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:42.764 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:42.764 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:42.764 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:42.764 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:42.764 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:42.764 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:42.764 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 1 -- 172.21.15.190:0/2074689029 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7edc0a5b00 con 0x7f7ee4136570 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 1 -- 172.21.15.190:0/2074689029 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7edc0a5cb0 con 0x7f7ee4136570 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee1ffb640 10 monclient: dump: 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:42.767 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:42.767 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:42.767 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:42.767 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:42.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 5 monclient: authenticate success, global_id 5639 2024-08-19T10:42:42.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:42.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 4 set_mon_vals no callback set 2024-08-19T10:42:42.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 10 monclient: shutdown 2024-08-19T10:42:42.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:42.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 1 -- 172.21.15.190:0/2074689029 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7ee4136570 msgr2=0x7f7ee41369d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 1 --2- 172.21.15.190:0/2074689029 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7ee4136570 0x7f7ee41369d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009e80 tx=0x7f7edc01c9a0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 1 -- 172.21.15.190:0/2074689029 shutdown_connections 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 1 --2- 172.21.15.190:0/2074689029 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7ee4136570 0x7f7ee41369d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 1 --2- 172.21.15.190:0/2074689029 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ee40a44b0 0x7f7ee40a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 1 --2- 172.21.15.190:0/2074689029 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ee40a3b90 0x7f7ee40a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 1 -- 172.21.15.190:0/2074689029 >> 172.21.15.190:0/2074689029 conn(0x7f7ee406dde0 msgr2=0x7f7ee406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 1 -- 172.21.15.190:0/2074689029 shutdown_connections 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 1 -- 172.21.15.190:0/2074689029 wait complete. 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.756+0000 7f7ee976d640 10 monclient: build_initial_monmap 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient: monmap: 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:42.768 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 1 Processor -- start 2024-08-19T10:42:42.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 1 -- start start 2024-08-19T10:42:42.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient: init 2024-08-19T10:42:42.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:42.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient: _add_conns ranks=[6,3,7,5,8,4,1,2,0] 2024-08-19T10:42:42.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7ee40a3b90 0x7f7ee41cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:42.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient(hunting): picked mon.noname-d con 0x7f7ee40a3b90 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:42.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ee40a44b0 0x7f7ee41355f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:42.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient(hunting): picked mon.noname-g con 0x7f7ee40a44b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:42.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ee4136570 0x7f7ee4139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:42.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient(hunting): picked mon.noname-i con 0x7f7ee4136570 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee40a0b00 con 0x7f7ee40a44b0 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7ee40a2c80 con 0x7f7ee40a3b90 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7ee406c8d0 con 0x7f7ee4136570 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee976d640 10 monclient(hunting): authenticate will time out at 1208.332988s 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ee4136570 0x7f7ee4139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee27fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ee40a44b0 0x7f7ee41355f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ee4136570 0x7f7ee4139590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:38256/0 (socket says 172.21.15.190:38256) 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 1 -- 172.21.15.190:0/1360583642 learned_addr learned my addr 172.21.15.190:0/1360583642 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:42.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 10 monclient(hunting): get_auth_request con 0x7f7ee4136570 auth_method 0 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee27fc640 10 monclient(hunting): get_auth_request con 0x7f7ee40a44b0 auth_method 0 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee27fc640 1 -- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ee40a44b0 msgr2=0x7f7ee41355f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee27fc640 1 -- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ee40a44b0 msgr2=0x7f7ee41355f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee27fc640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ee40a44b0 0x7f7ee41355f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:42:42.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee27fc640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ee40a44b0 0x7f7ee41355f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee2ffd640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7ee40a3b90 0x7f7ee41cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee2ffd640 10 monclient(hunting): get_auth_request con 0x7f7ee40a3b90 auth_method 0 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 10 monclient(hunting): handle_auth_done global_id 14812 payload 824 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 1 -- 172.21.15.190:0/1360583642 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7ee40a3b90 msgr2=0x7f7ee41cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7ee40a3b90 0x7f7ee41cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 1 -- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ee40a44b0 msgr2=0x7f7ee41355f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ee40a44b0 0x7f7ee41355f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:42.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 1 monclient: found mon.noname-i 2024-08-19T10:42:42.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:42.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:42.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 1 -- 172.21.15.190:0/1360583642 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee411f9c0 con 0x7f7ee4136570 2024-08-19T10:42:42.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 1 -- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ee4136570 msgr2=0x7f7ee4139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:42.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ee37fe640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ee4136570 0x7f7ee4139590 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0af040 tx=0x7f7edc019740 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:42.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ecb7fe640 1 -- 172.21.15.190:0/1360583642 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7edc0bc020 con 0x7f7ee4136570 2024-08-19T10:42:42.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ecb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:42.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ecb7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:42.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.760+0000 7f7ecb7fe640 10 monclient: dump: 2024-08-19T10:42:42.778 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:42.779 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee976d640 5 monclient: authenticate success, global_id 14812 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 1 -- 172.21.15.190:0/1360583642 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7edc0044b0 con 0x7f7ee4136570 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 1 -- 172.21.15.190:0/1360583642 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7edc004660 con 0x7f7ee4136570 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee976d640 10 monclient: _renew_subs 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee976d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee976d640 1 -- 172.21.15.190:0/1360583642 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee411fca0 con 0x7f7ee4136570 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:42.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee17fa640 4 set_mon_vals no callback set 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 10 monclient: dump: 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:42.781 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee976d640 10 monclient: _renew_subs 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee976d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee976d640 1 -- 172.21.15.190:0/1360583642 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee406d460 con 0x7f7ee4136570 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 1 -- 172.21.15.190:0/1360583642 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7edc0bf020 con 0x7f7ee4136570 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee976d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee976d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee976d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee976d640 1 -- 172.21.15.190:0/1360583642 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee40a8980 con 0x7f7ee4136570 2024-08-19T10:42:42.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7ed003be00 0x7f7ed003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:42.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ecb7fe640 1 -- 172.21.15.190:0/1360583642 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(149..149 src has 1..149) ==== 7975+0+0 (secure 0 0 0) 0x7f7edc0b3040 con 0x7f7ee4136570 2024-08-19T10:42:42.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee2ffd640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7ed003be00 0x7f7ed003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:42.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee2ffd640 10 monclient: get_auth_request con 0x7f7ed003be00 auth_method 0 2024-08-19T10:42:42.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.764+0000 7f7ee2ffd640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7ed003be00 0x7f7ed003e2c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400ba30 tx=0x7f7ed4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:42.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.768+0000 7f7ecb7fe640 1 -- 172.21.15.190:0/1360583642 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7ee40a8980 con 0x7f7ee4136570 2024-08-19T10:42:42.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.768+0000 7f7ecb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:42.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.768+0000 7f7ecb7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:42.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.944+0000 7f7ee976d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:42.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.944+0000 7f7ee976d640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:42.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.944+0000 7f7ee976d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:42.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:42.944+0000 7f7ee976d640 1 -- 172.21.15.190:0/1360583642 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7f7ee40a3f70 con 0x7f7ee4136570 2024-08-19T10:42:43.098 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:42:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient: get_monmap_and_config 2024-08-19T10:42:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient: build_initial_monmap 2024-08-19T10:42:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient: monmap: 2024-08-19T10:42:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:43.226310+0000 2024-08-19T10:42:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:43.226310+0000 2024-08-19T10:42:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:43.226 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:43.227 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:43.227 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 1 Processor -- start 2024-08-19T10:42:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 1 -- start start 2024-08-19T10:42:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient: init 2024-08-19T10:42:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient: _add_conns ranks=[6,3,10,1,5,7,9,2,4,8,0] 2024-08-19T10:42:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a5270 0x7f08900a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient(hunting): picked mon.noname-d con 0x7f08900a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08900a6d30 0x7f08900a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient(hunting): picked mon.noname-g con 0x7f08900a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:43.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f08901395b0 0x7f0890139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient(hunting): picked mon.noname-f con 0x7f08901395b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f08900a7650 con 0x7f08901395b0 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f08900a8710 con 0x7f08900a6d30 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f08900a5a40 con 0x7f08900a5270 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f0895f0e640 10 monclient(hunting): authenticate will time out at 1213.042915s 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a5270 0x7f08900a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a5270 0x7f08900a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54404/0 (socket says 172.21.15.89:54404) 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 1 -- 172.21.15.89:0/1377072185 learned_addr learned my addr 172.21.15.89:0/1377072185 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:43.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 10 monclient(hunting): get_auth_request con 0x7f08900a5270 auth_method 0 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088effd640 1 --2- 172.21.15.89:0/1377072185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08900a6d30 0x7f08900a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088ffff640 1 --2- 172.21.15.89:0/1377072185 >> v2:172.21.15.190:3300/0 conn(0x7f08901395b0 0x7f0890139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088effd640 10 monclient(hunting): get_auth_request con 0x7f08900a6d30 auth_method 0 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088ffff640 10 monclient(hunting): get_auth_request con 0x7f08901395b0 auth_method 0 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 10 monclient(hunting): handle_auth_done global_id 5433 payload 274 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 1 -- 172.21.15.89:0/1377072185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08900a6d30 msgr2=0x7f08900a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 1 --2- 172.21.15.89:0/1377072185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08900a6d30 0x7f08900a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 1 -- 172.21.15.89:0/1377072185 >> v2:172.21.15.190:3300/0 conn(0x7f08901395b0 msgr2=0x7f0890139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 1 --2- 172.21.15.89:0/1377072185 >> v2:172.21.15.190:3300/0 conn(0x7f08901395b0 0x7f0890139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 1 monclient: found mon.noname-d 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:43.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.223+0000 7f088f7fe640 1 -- 172.21.15.89:0/1377072185 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08900a81d0 con 0x7f08900a5270 2024-08-19T10:42:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088f7fe640 1 -- 172.21.15.89:0/1377072185 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a5270 msgr2=0x7f08900a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088f7fe640 1 --2- 172.21.15.89:0/1377072185 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a5270 0x7f08900a67f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0880009870 tx=0x7f088001c420 comp rx=0 tx=0).ready entity=mon.6 client_cookie=4e0db1ca11f1190d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088e7fc640 1 -- 172.21.15.89:0/1377072185 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08800b1070 con 0x7f08900a5270 2024-08-19T10:42:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088e7fc640 10 monclient: dump: 2024-08-19T10:42:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.236 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.237 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088e7fc640 1 -- 172.21.15.89:0/1377072185 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08800ab810 con 0x7f08900a5270 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f0895f0e640 5 monclient: authenticate success, global_id 5433 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f0895f0e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f0895f0e640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088e7fc640 1 -- 172.21.15.89:0/1377072185 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08800abe30 con 0x7f08900a5270 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f088e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.227+0000 7f0895f0e640 4 set_mon_vals no callback set 2024-08-19T10:42:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088e7fc640 10 monclient: dump: 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient: shutdown 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 -- 172.21.15.89:0/1377072185 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a5270 msgr2=0x7f08900a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 --2- 172.21.15.89:0/1377072185 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a5270 0x7f08900a67f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0880009870 tx=0x7f088001c420 comp rx=0 tx=0).stop 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 -- 172.21.15.89:0/1377072185 shutdown_connections 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 --2- 172.21.15.89:0/1377072185 >> v2:172.21.15.190:3300/0 conn(0x7f08901395b0 0x7f0890139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 --2- 172.21.15.89:0/1377072185 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08900a6d30 0x7f08900a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 --2- 172.21.15.89:0/1377072185 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a5270 0x7f08900a67f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 -- 172.21.15.89:0/1377072185 >> 172.21.15.89:0/1377072185 conn(0x7f0890069a80 msgr2=0x7f08900abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 -- 172.21.15.89:0/1377072185 shutdown_connections 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 -- 172.21.15.89:0/1377072185 wait complete. 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient: build_initial_monmap 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient: monmap: 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 Processor -- start 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 -- start start 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient: init 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient: _add_conns ranks=[3,6,1,8,7,4,0,5,2] 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08900a5270 0x7f089013d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient(hunting): picked mon.noname-g con 0x7f08900a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a6d30 0x7f089013d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient(hunting): picked mon.noname-d con 0x7f08900a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08901395b0 0x7f089013db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient(hunting): picked mon.noname-f con 0x7f08901395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f08900a5650 con 0x7f08901395b0 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f089013a350 con 0x7f08900a5270 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f08900a57d0 con 0x7f08900a6d30 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f0895f0e640 10 monclient(hunting): authenticate will time out at 1213.050069s 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a6d30 0x7f089013d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a6d30 0x7f089013d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54408/0 (socket says 172.21.15.89:54408) 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 1 -- 172.21.15.89:0/3267466728 learned_addr learned my addr 172.21.15.89:0/3267466728 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 10 monclient(hunting): get_auth_request con 0x7f08900a6d30 auth_method 0 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088ffff640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08901395b0 0x7f089013db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088f7fe640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08900a5270 0x7f089013d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088ffff640 10 monclient(hunting): get_auth_request con 0x7f08901395b0 auth_method 0 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088f7fe640 10 monclient(hunting): get_auth_request con 0x7f08900a5270 auth_method 0 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 10 monclient(hunting): handle_auth_done global_id 5442 payload 824 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 1 -- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08900a5270 msgr2=0x7f089013d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08900a5270 0x7f089013d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 1 -- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08901395b0 msgr2=0x7f089013db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08901395b0 0x7f089013db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 10 monclient: _finish_hunting 0 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 1 monclient: found mon.noname-d 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:43.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.231+0000 7f088effd640 1 -- 172.21.15.89:0/3267466728 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0890139c70 con 0x7f08900a6d30 2024-08-19T10:42:43.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f088effd640 1 -- 172.21.15.89:0/3267466728 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a6d30 msgr2=0x7f089013d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f088effd640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a6d30 0x7f089013d5c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0878007ef0 tx=0x7f087800e4b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 1 -- 172.21.15.89:0/3267466728 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0878013a00 con 0x7f08900a6d30 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 10 monclient: dump: 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.248 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 1 -- 172.21.15.89:0/3267466728 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0878019070 con 0x7f08900a6d30 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 1 -- 172.21.15.89:0/3267466728 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0878013c00 con 0x7f08900a6d30 2024-08-19T10:42:43.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 10 monclient: dump: 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.250 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f088dffb640 4 set_mon_vals no callback set 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0895f0e640 5 monclient: authenticate success, global_id 5442 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0895f0e640 10 monclient: _renew_subs 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0895f0e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0895f0e640 1 -- 172.21.15.89:0/3267466728 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08900a6a80 con 0x7f08900a6d30 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0895f0e640 10 monclient: _renew_subs 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0895f0e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:43.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0895f0e640 1 -- 172.21.15.89:0/3267466728 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f089009d140 con 0x7f08900a6d30 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0895f0e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0895f0e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0895f0e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0895f0e640 1 -- 172.21.15.89:0/3267466728 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08900a7110 con 0x7f08900a6d30 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 1 -- 172.21.15.89:0/3267466728 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0878013590 con 0x7f08900a6d30 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f087403be00 0x7f087403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f0887fff640 1 -- 172.21.15.89:0/3267466728 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(149..149 src has 1..149) ==== 7975+0+0 (secure 0 0 0) 0x7f08780500c0 con 0x7f08900a6d30 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f088f7fe640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f087403be00 0x7f087403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.235+0000 7f088f7fe640 10 monclient: get_auth_request con 0x7f087403be00 auth_method 0 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.239+0000 7f088f7fe640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f087403be00 0x7f087403e2c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f08800ac000 tx=0x7f08800a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.239+0000 7f0887fff640 1 -- 172.21.15.89:0/3267466728 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0878013de0 con 0x7f08900a6d30 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.239+0000 7f0887fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:43.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.239+0000 7f0887fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:43.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:42:43.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:42:43.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:43.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 -- 172.21.15.89:0/3267466728 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0890064590 con 0x7f08900a6d30 2024-08-19T10:42:43.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0887fff640 1 -- 172.21.15.89:0/3267466728 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7f087804fd40 con 0x7f08900a6d30 2024-08-19T10:42:43.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0887fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:42:43.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0887fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:43.478 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":52,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:42:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 -- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f087403be00 msgr2=0x7f087403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f087403be00 0x7f087403e2c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f08800ac000 tx=0x7f08800a7000 comp rx=0 tx=0).stop 2024-08-19T10:42:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 10 monclient: shutdown 2024-08-19T10:42:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 -- 172.21.15.89:0/3267466728 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a6d30 msgr2=0x7f089013d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a6d30 0x7f089013d5c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0878007ef0 tx=0x7f087800e4b0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 -- 172.21.15.89:0/3267466728 shutdown_connections 2024-08-19T10:42:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f087403be00 0x7f087403e2c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08901395b0 0x7f089013db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08900a6d30 0x7f089013d5c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 --2- 172.21.15.89:0/3267466728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08900a5270 0x7f089013d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 -- 172.21.15.89:0/3267466728 >> 172.21.15.89:0/3267466728 conn(0x7f0890069a80 msgr2=0x7f08900ab1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 -- 172.21.15.89:0/3267466728 shutdown_connections 2024-08-19T10:42:43.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.475+0000 7f0895f0e640 1 -- 172.21.15.89:0/3267466728 wait complete. 2024-08-19T10:42:43.496 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:42:43.496 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:42:43.496 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:42:43.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient: get_monmap_and_config 2024-08-19T10:42:43.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient: build_initial_monmap 2024-08-19T10:42:43.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient: monmap: 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:43.601960+0000 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:43.601960+0000 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:43.602 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:43.603 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:43.603 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 1 Processor -- start 2024-08-19T10:42:43.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 1 -- start start 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient: init 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient: _add_conns ranks=[7,6,4,1,2,9,0,5,8,10,3] 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdff40a6860 0x7fdff40a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient(hunting): picked mon.noname-i con 0x7fdff40a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 0x7fdff40a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient(hunting): picked mon.noname-d con 0x7fdff40a8480 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff4139620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient(hunting): picked mon.noname-c con 0x7fdff40a76c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fdff40a8240 con 0x7fdff40a76c0 2024-08-19T10:42:43.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fdff40a4d50 con 0x7fdff40a8480 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fdff40a2ec0 con 0x7fdff40a6860 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff89bc640 10 monclient(hunting): authenticate will time out at 1213.418512s 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff2d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff4139620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 0x7fdff40a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 0x7fdff40a7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54424/0 (socket says 172.21.15.89:54424) 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff2d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff4139620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38336/0 (socket says 172.21.15.89:38336) 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 1 -- 172.21.15.89:0/733028047 learned_addr learned my addr 172.21.15.89:0/733028047 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 10 monclient(hunting): get_auth_request con 0x7fdff40a8480 auth_method 0 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff2d9d640 10 monclient(hunting): get_auth_request con 0x7fdff40a76c0 auth_method 0 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff2d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff259c640 1 --2- 172.21.15.89:0/733028047 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdff40a6860 0x7fdff40a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff259c640 10 monclient(hunting): get_auth_request con 0x7fdff40a6860 auth_method 0 2024-08-19T10:42:43.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 10 monclient(hunting): handle_auth_done global_id 5451 payload 274 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 1 -- 172.21.15.89:0/733028047 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdff40a6860 msgr2=0x7fdff40a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 1 --2- 172.21.15.89:0/733028047 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdff40a6860 0x7fdff40a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 1 -- 172.21.15.89:0/733028047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 msgr2=0x7fdff4139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 1 --2- 172.21.15.89:0/733028047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff4139620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 1 monclient: found mon.noname-d 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff1d9b640 1 -- 172.21.15.89:0/733028047 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff40a4a70 con 0x7fdff40a8480 2024-08-19T10:42:43.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.599+0000 7fdff2d9d640 1 --2- 172.21.15.89:0/733028047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff4139620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 1 -- 172.21.15.89:0/733028047 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 msgr2=0x7fdff40a7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 1 --2- 172.21.15.89:0/733028047 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 0x7fdff40a7180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80098a0 tx=0x7fdfe801c180 comp rx=0 tx=0).ready entity=mon.6 client_cookie=74686e6196ad8c55 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 1 -- 172.21.15.89:0/733028047 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe80b1070 con 0x7fdff40a8480 2024-08-19T10:42:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 10 monclient: dump: 2024-08-19T10:42:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.614 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 10 monclient: _finish_auth 0 2024-08-19T10:42:43.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 10 monclient: _check_auth_tickets 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 1 -- 172.21.15.89:0/733028047 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe80a4c50 con 0x7fdff40a8480 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 1 -- 172.21.15.89:0/733028047 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe80a4e40 con 0x7fdff40a8480 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 5 monclient: authenticate success, global_id 5451 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 4 set_mon_vals no callback set 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff159a640 10 monclient: dump: 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.616 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.617 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient: shutdown 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 -- 172.21.15.89:0/733028047 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 msgr2=0x7fdff40a7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 --2- 172.21.15.89:0/733028047 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 0x7fdff40a7180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80098a0 tx=0x7fdfe801c180 comp rx=0 tx=0).stop 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 -- 172.21.15.89:0/733028047 shutdown_connections 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 --2- 172.21.15.89:0/733028047 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff4139620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 --2- 172.21.15.89:0/733028047 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 0x7fdff40a7180 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 --2- 172.21.15.89:0/733028047 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdff40a6860 0x7fdff40a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 -- 172.21.15.89:0/733028047 >> 172.21.15.89:0/733028047 conn(0x7fdff40699d0 msgr2=0x7fdff40abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 -- 172.21.15.89:0/733028047 shutdown_connections 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 -- 172.21.15.89:0/733028047 wait complete. 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient: build_initial_monmap 2024-08-19T10:42:43.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient: monmap: 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:43.619 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 Processor -- start 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 -- start start 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient: init 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient: _add_conns ranks=[2,4,6,1,0,3,8,7,5] 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff40a6860 0x7fdff40a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient(hunting): picked mon.noname-b con 0x7fdff40a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff40a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient(hunting): picked mon.noname-c con 0x7fdff40a76c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 0x7fdff413dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient(hunting): picked mon.noname-d con 0x7fdff40a8480 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:43.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fdff4143420 con 0x7fdff40a6860 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fdff41432a0 con 0x7fdff40a76c0 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fdff41435a0 con 0x7fdff40a8480 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff89bc640 10 monclient(hunting): authenticate will time out at 1213.426487s 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff259c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff40a6860 0x7fdff40a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff40a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff259c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff40a6860 0x7fdff40a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40070/0 (socket says 172.21.15.89:40070) 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff2d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 0x7fdff413dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff40a2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38340/0 (socket says 172.21.15.89:38340) 2024-08-19T10:42:43.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff259c640 1 -- 172.21.15.89:0/113723109 learned_addr learned my addr 172.21.15.89:0/113723109 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 10 monclient(hunting): get_auth_request con 0x7fdff40a76c0 auth_method 0 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff259c640 10 monclient(hunting): get_auth_request con 0x7fdff40a6860 auth_method 0 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff2d9d640 10 monclient(hunting): get_auth_request con 0x7fdff40a8480 auth_method 0 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff2d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 10 monclient(hunting): handle_auth_done global_id 5566 payload 824 2024-08-19T10:42:43.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 1 -- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 msgr2=0x7fdff413dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 0x7fdff413dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 1 -- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff40a6860 msgr2=0x7fdff40a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff40a6860 0x7fdff40a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 1 monclient: found mon.noname-c 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff1d9b640 1 -- 172.21.15.89:0/113723109 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff40ab0a0 con 0x7fdff40a76c0 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff259c640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdff40a6860 0x7fdff40a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.607+0000 7fdff2d9d640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdff40a8480 0x7fdff413dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff1d9b640 1 -- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 msgr2=0x7fdff40a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff1d9b640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff40a2530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80ab000 tx=0x7fdfe8018f20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe80b9030 con 0x7fdff40a76c0 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 10 monclient: dump: 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.625 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe8003be0 con 0x7fdff40a76c0 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:43.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe80b1040 con 0x7fdff40a76c0 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff89bc640 5 monclient: authenticate success, global_id 5566 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 10 monclient: dump: 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.627 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff89bc640 10 monclient: _renew_subs 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff0d99640 4 set_mon_vals no callback set 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff89bc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff89bc640 1 -- 172.21.15.89:0/113723109 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff40aad70 con 0x7fdff40a76c0 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff89bc640 10 monclient: _renew_subs 2024-08-19T10:42:43.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff89bc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:43.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff89bc640 1 -- 172.21.15.89:0/113723109 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff409da90 con 0x7fdff40a76c0 2024-08-19T10:42:43.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff89bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:43.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fdfe80bc070 con 0x7fdff40a76c0 2024-08-19T10:42:43.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff89bc640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff409d1f0 0x7fdff413e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff89bc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff89bc640 1 -- 172.21.15.89:0/113723109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdff41d2590 con 0x7fdff409d1f0 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff259c640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff409d1f0 0x7fdff413e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff259c640 10 monclient: get_auth_request con 0x7fdff409d1f0 auth_method 0 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdfd403be00 0x7fdfd403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(149..149 src has 1..149) ==== 7975+0+0 (secure 0 0 0) 0x7fdfe80ba070 con 0x7fdff40a76c0 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff259c640 10 monclient(hunting): handle_auth_done global_id 5643 payload 824 2024-08-19T10:42:43.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff259c640 1 -- 172.21.15.89:0/113723109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdff413f570 con 0x7fdff409d1f0 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff2d9d640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdfd403be00 0x7fdfd403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.611+0000 7fdff2d9d640 10 monclient: get_auth_request con 0x7fdfd403be00 auth_method 0 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdff259c640 1 -- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff409d1f0 msgr2=0x7fdff413e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdff259c640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff409d1f0 0x7fdff413e130 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4010730 tx=0x7fdfe4010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe400ed90 con 0x7fdff409d1f0 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdfe2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdfe2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdfe2ffd640 10 monclient: dump: 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.634 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdfe401c070 con 0x7fdff409d1f0 2024-08-19T10:42:43.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdfe2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdfe2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff409d1f0 msgr2=0x7fdff413e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdfe2ffd640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff409d1f0 0x7fdff413e130 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4010730 tx=0x7fdfe4010bf0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff409d1f0 msgr2=0x7fdff413e130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdfe2ffd640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff409d1f0 0x7fdff413e130 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4010730 tx=0x7fdfe4010bf0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.615+0000 7fdff2d9d640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdfd403be00 0x7fdfd403e2c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009580 tx=0x7fdfec009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff89bc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff89bc640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff41d1890 0x7fdff413e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff89bc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff89bc640 1 -- 172.21.15.89:0/113723109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4139620 con 0x7fdff41d1890 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff259c640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff41d1890 0x7fdff413e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff259c640 10 monclient: get_auth_request con 0x7fdff41d1890 auth_method 0 2024-08-19T10:42:43.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff259c640 10 monclient(hunting): handle_auth_done global_id 5652 payload 824 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.619+0000 7fdff259c640 1 -- 172.21.15.89:0/113723109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdff4064590 con 0x7fdff41d1890 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff259c640 1 -- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff41d1890 msgr2=0x7fdff413e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff259c640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff41d1890 0x7fdff413e670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401b000 tx=0x7fdfe4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe4021e40 con 0x7fdff41d1890 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdfe2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdfe2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdfe2ffd640 10 monclient: dump: 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.638 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.639 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9269 (secure 0 0 0) 0x7fdfe40176b0 con 0x7fdff41d1890 2024-08-19T10:42:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdfe2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdfe2ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff41d1890 msgr2=0x7fdff413e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdfe2ffd640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff41d1890 0x7fdff413e670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401b000 tx=0x7fdfe4004a50 comp rx=0 tx=0).stop 2024-08-19T10:42:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdfe2ffd640 1 -- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff41d1890 msgr2=0x7fdff413e670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:43.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdfe2ffd640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdff41d1890 0x7fdff413e670 secure :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401b000 tx=0x7fdfe4004a50 comp rx=0 tx=0).stop 2024-08-19T10:42:43.640 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:43.640 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:43.640 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:43.640 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:42:43.640 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:43.640 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 293436, 2024-08-19T10:42:43.640 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:43.640 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:43.640 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:43.640 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:43.640 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:43.641 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:43.642 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.643 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:43.644 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:43.645 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.646 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:43.647 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:43.648 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.649 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.652 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:43.653 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.654 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.655 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:43.656 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.658 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.659 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:43.660 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.661 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:43.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.664 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:43.664 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:43.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff89bc640 1 -- 172.21.15.89:0/113723109 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdfd403be00 msgr2=0x7fdfd403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff89bc640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdfd403be00 0x7fdfd403e2c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009580 tx=0x7fdfec009190 comp rx=0 tx=0).stop 2024-08-19T10:42:43.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff89bc640 10 monclient: shutdown 2024-08-19T10:42:43.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff89bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:43.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff89bc640 1 -- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 msgr2=0x7fdff40a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff89bc640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff40a2530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80ab000 tx=0x7fdfe8018f20 comp rx=0 tx=0).stop 2024-08-19T10:42:43.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff2d9d640 1 -- 172.21.15.89:0/113723109 reap_dead start 2024-08-19T10:42:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff89bc640 1 -- 172.21.15.89:0/113723109 shutdown_connections 2024-08-19T10:42:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff89bc640 1 --2- 172.21.15.89:0/113723109 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdff40a76c0 0x7fdff40a2530 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff89bc640 1 -- 172.21.15.89:0/113723109 >> 172.21.15.89:0/113723109 conn(0x7fdff40699d0 msgr2=0x7fdff4134f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff89bc640 1 -- 172.21.15.89:0/113723109 shutdown_connections 2024-08-19T10:42:43.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.623+0000 7fdff89bc640 1 -- 172.21.15.89:0/113723109 wait complete. 2024-08-19T10:42:43.666 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:43.666 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:42:43.666 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:42:43.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.712+0000 7f7ecb7fe640 1 -- 172.21.15.190:0/1360583642 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v150) ==== 82+0+0 (secure 0 0 0) 0x7f7edc0c6030 con 0x7f7ee4136570 2024-08-19T10:42:43.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.712+0000 7f7ecb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:43.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.712+0000 7f7ecb7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:43.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient: get_monmap_and_config 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient: build_initial_monmap 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient: monmap: 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:43.777260+0000 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:43.777260+0000 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:43.777 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:43.778 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:43.778 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 1 Processor -- start 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 1 -- start start 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient: init 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient: _add_conns ranks=[6,9,5,4,1,10,2,8,3,0,7] 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8340a67d0 0x7fa8340a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient(hunting): picked mon.noname-d con 0x7fa8340a67d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8340a6bb0 0x7fa8340a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient(hunting): picked mon.noname-a con 0x7fa8340a6bb0 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa834139590 0x7fa834139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient(hunting): picked mon.noname-h con 0x7fa834139590 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fa8340a8740 con 0x7fa8340a6bb0 2024-08-19T10:42:43.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa8340a3380 con 0x7fa834139590 2024-08-19T10:42:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa8340a8250 con 0x7fa8340a67d0 2024-08-19T10:42:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa83b523640 10 monclient(hunting): authenticate will time out at 1213.593784s 2024-08-19T10:42:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8340a6bb0 0x7fa8340a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa8392bf640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8340a67d0 0x7fa8340a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa8340a6bb0 0x7fa8340a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37570/0 (socket says 172.21.15.89:37570) 2024-08-19T10:42:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa8392bf640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8340a67d0 0x7fa8340a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54444/0 (socket says 172.21.15.89:54444) 2024-08-19T10:42:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 1 -- 172.21.15.89:0/297737560 learned_addr learned my addr 172.21.15.89:0/297737560 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:43.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 10 monclient(hunting): get_auth_request con 0x7fa8340a6bb0 auth_method 0 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa8392bf640 10 monclient(hunting): get_auth_request con 0x7fa8340a67d0 auth_method 0 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa8392bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa8392bf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa8392bf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa8392bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa8392bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa8392bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa839ac0640 1 --2- 172.21.15.89:0/297737560 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa834139590 0x7fa834139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa839ac0640 10 monclient(hunting): get_auth_request con 0x7fa834139590 auth_method 0 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa839ac0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa839ac0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa839ac0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 10 monclient(hunting): handle_auth_done global_id 5661 payload 274 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 1 -- 172.21.15.89:0/297737560 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8340a67d0 msgr2=0x7fa8340a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 1 --2- 172.21.15.89:0/297737560 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8340a67d0 0x7fa8340a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 1 -- 172.21.15.89:0/297737560 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa834139590 msgr2=0x7fa834139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 1 --2- 172.21.15.89:0/297737560 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa834139590 0x7fa834139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 1 monclient: found mon.noname-a 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:43.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:43.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa838abe640 1 -- 172.21.15.89:0/297737560 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8340a4cc0 con 0x7fa8340a6bb0 2024-08-19T10:42:43.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.775+0000 7fa8392bf640 1 --2- 172.21.15.89:0/297737560 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8340a67d0 0x7fa8340a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:43.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa838abe640 1 -- 172.21.15.89:0/297737560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a6bb0 msgr2=0x7fa8340a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa838abe640 1 --2- 172.21.15.89:0/297737560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a6bb0 0x7fa8340a6f90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009e50 tx=0x7fa82c01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2120f9ef73a48ee server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 1 -- 172.21.15.89:0/297737560 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa82c0090d0 con 0x7fa8340a6bb0 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 10 monclient: dump: 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.791 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 1 -- 172.21.15.89:0/297737560 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa82c0a5a30 con 0x7fa8340a6bb0 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 5 monclient: authenticate success, global_id 5661 2024-08-19T10:42:43.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 1 -- 172.21.15.89:0/297737560 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa82c0a5be0 con 0x7fa8340a6bb0 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 4 set_mon_vals no callback set 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa833fff640 10 monclient: dump: 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.793 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 10 monclient: shutdown 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 1 -- 172.21.15.89:0/297737560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a6bb0 msgr2=0x7fa8340a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 1 --2- 172.21.15.89:0/297737560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a6bb0 0x7fa8340a6f90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009e50 tx=0x7fa82c01c980 comp rx=0 tx=0).stop 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 1 -- 172.21.15.89:0/297737560 shutdown_connections 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 1 --2- 172.21.15.89:0/297737560 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa834139590 0x7fa834139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 1 --2- 172.21.15.89:0/297737560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8340a6bb0 0x7fa8340a6f90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 1 --2- 172.21.15.89:0/297737560 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8340a67d0 0x7fa8340a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 1 -- 172.21.15.89:0/297737560 >> 172.21.15.89:0/297737560 conn(0x7fa834069ae0 msgr2=0x7fa8340abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 1 -- 172.21.15.89:0/297737560 shutdown_connections 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 1 -- 172.21.15.89:0/297737560 wait complete. 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 10 monclient: build_initial_monmap 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.779+0000 7fa83b523640 10 monclient: monmap: 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:43.795 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 1 Processor -- start 2024-08-19T10:42:43.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 1 -- start start 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient: init 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient: _add_conns ranks=[3,4,7,8,2,1,5,6,0] 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8340a67d0 0x7fa83409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient(hunting): picked mon.noname-g con 0x7fa8340a67d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a6bb0 0x7fa83409cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient(hunting): picked mon.noname-c con 0x7fa8340a6bb0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa834139590 0x7fa83409d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient(hunting): picked mon.noname-i con 0x7fa834139590 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa834143390 con 0x7fa8340a67d0 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa834143210 con 0x7fa8340a6bb0 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa834143510 con 0x7fa834139590 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient(hunting): authenticate will time out at 1213.600624s 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a6bb0 0x7fa83409cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a6bb0 0x7fa83409cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38348/0 (socket says 172.21.15.89:38348) 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 1 -- 172.21.15.89:0/3048849332 learned_addr learned my addr 172.21.15.89:0/3048849332 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 10 monclient(hunting): get_auth_request con 0x7fa8340a6bb0 auth_method 0 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa839ac0640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa834139590 0x7fa83409d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa8392bf640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8340a67d0 0x7fa83409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa839ac0640 10 monclient(hunting): get_auth_request con 0x7fa834139590 auth_method 0 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa839ac0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa839ac0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa839ac0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa8392bf640 10 monclient(hunting): get_auth_request con 0x7fa8340a67d0 auth_method 0 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa8392bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa8392bf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa8392bf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 10 monclient(hunting): handle_auth_done global_id 5575 payload 824 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 1 -- 172.21.15.89:0/3048849332 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa834139590 msgr2=0x7fa83409d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa834139590 0x7fa83409d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 1 -- 172.21.15.89:0/3048849332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8340a67d0 msgr2=0x7fa83409c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8340a67d0 0x7fa83409c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 1 monclient: found mon.noname-c 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 1 -- 172.21.15.89:0/3048849332 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83413f970 con 0x7fa8340a6bb0 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 1 -- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a6bb0 msgr2=0x7fa83409cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa838abe640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a6bb0 0x7fa83409cf20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa82c01d220 tx=0x7fa82c019720 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa82c0bc020 con 0x7fa8340a6bb0 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:43.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 10 monclient: dump: 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.801 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa82c003e10 con 0x7fa8340a6bb0 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 5 monclient: authenticate success, global_id 5575 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient: _renew_subs 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 1 -- 172.21.15.89:0/3048849332 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83409eeb0 con 0x7fa8340a6bb0 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa82c0044b0 con 0x7fa8340a6bb0 2024-08-19T10:42:43.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa831ffb640 10 monclient: dump: 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.803 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa8337fe640 4 set_mon_vals no callback set 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient: _renew_subs 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.783+0000 7fa83b523640 1 -- 172.21.15.89:0/3048849332 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fa834140650 con 0x7fa8340a6bb0 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa83b523640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa83b523640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133530 0x7fa834133910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa83b523640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa83b523640 1 -- 172.21.15.89:0/3048849332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa834139bb0 con 0x7fa834133530 2024-08-19T10:42:43.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133530 0x7fa834133910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 10 monclient: get_auth_request con 0x7fa834133530 auth_method 0 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa82c0bf020 con 0x7fa8340a6bb0 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 10 monclient(hunting): handle_auth_done global_id 5564 payload 824 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 1 -- 172.21.15.89:0/3048849332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa834135700 con 0x7fa834133530 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa82003bd30 0x7fa82003e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(150..150 src has 1..150) ==== 8307+0+0 (secure 0 0 0) 0x7fa82c0ea780 con 0x7fa8340a6bb0 2024-08-19T10:42:43.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa839ac0640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa82003bd30 0x7fa82003e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa839ac0640 10 monclient: get_auth_request con 0x7fa82003bd30 auth_method 0 2024-08-19T10:42:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 1 -- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133530 msgr2=0x7fa834133910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa8392bf640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133530 0x7fa834133910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa828011600 tx=0x7fa828011ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa82800fea0 con 0x7fa834133530 2024-08-19T10:42:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 10 monclient: dump: 2024-08-19T10:42:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.808 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa82800fa30 con 0x7fa834133530 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133530 msgr2=0x7fa834133910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133530 0x7fa834133910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa828011600 tx=0x7fa828011ac0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133530 msgr2=0x7fa834133910 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa831ffb640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133530 0x7fa834133910 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa828011600 tx=0x7fa828011ac0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.787+0000 7fa839ac0640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa82003bd30 0x7fa82003e1f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa83413e7b0 tx=0x7fa824008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa83b523640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa83b523640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133e50 0x7fa83413ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa83b523640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa83b523640 1 -- 172.21.15.89:0/3048849332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa8340a6f90 con 0x7fa834133e50 2024-08-19T10:42:43.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa8392bf640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133e50 0x7fa83413ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa8392bf640 10 monclient: get_auth_request con 0x7fa834133e50 auth_method 0 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa8392bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa8392bf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa8392bf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa8392bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa8392bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa8392bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa8392bf640 10 monclient(hunting): handle_auth_done global_id 5573 payload 824 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.795+0000 7fa8392bf640 1 -- 172.21.15.89:0/3048849332 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa834064590 con 0x7fa834133e50 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa8392bf640 1 -- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133e50 msgr2=0x7fa83413ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa8392bf640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133e50 0x7fa83413ab70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa828011600 tx=0x7fa8280046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8280064b0 con 0x7fa834133e50 2024-08-19T10:42:43.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa831ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa831ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa831ffb640 10 monclient: dump: 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.812 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7fa828005ed0 con 0x7fa834133e50 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa831ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa831ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133e50 msgr2=0x7fa83413ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa831ffb640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133e50 0x7fa83413ab70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa828011600 tx=0x7fa8280046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa831ffb640 1 -- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133e50 msgr2=0x7fa83413ab70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:43.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa831ffb640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa834133e50 0x7fa83413ab70 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa828011600 tx=0x7fa8280046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.814 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:43.814 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:43.814 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:43.814 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:43.814 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:43.814 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 293628, 2024-08-19T10:42:43.814 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:43.814 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:43.814 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:43.815 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:43.816 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:43.817 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:43.818 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:43.819 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.820 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:43.821 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:43.822 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.823 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.825 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:43.826 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:43.827 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.828 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:43.830 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:43.831 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.832 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.833 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.834 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:43.835 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:43.836 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.837 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:42:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:43.838 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:43.838 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:43.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa83b523640 1 -- 172.21.15.89:0/3048849332 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa82003bd30 msgr2=0x7fa82003e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa83b523640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa82003bd30 0x7fa82003e1f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa83413e7b0 tx=0x7fa824008040 comp rx=0 tx=0).stop 2024-08-19T10:42:43.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa83b523640 10 monclient: shutdown 2024-08-19T10:42:43.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa83b523640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:43.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa83b523640 1 -- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a6bb0 msgr2=0x7fa83409cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa83b523640 1 --2- 172.21.15.89:0/3048849332 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8340a6bb0 0x7fa83409cf20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa82c01d220 tx=0x7fa82c019720 comp rx=0 tx=0).stop 2024-08-19T10:42:43.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa839ac0640 1 -- 172.21.15.89:0/3048849332 reap_dead start 2024-08-19T10:42:43.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa839ac0640 1 -- 172.21.15.89:0/3048849332 reap_dead start 2024-08-19T10:42:43.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa83b523640 1 -- 172.21.15.89:0/3048849332 shutdown_connections 2024-08-19T10:42:43.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa83b523640 1 -- 172.21.15.89:0/3048849332 >> 172.21.15.89:0/3048849332 conn(0x7fa834069ae0 msgr2=0x7fa8341345b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:43.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa83b523640 1 -- 172.21.15.89:0/3048849332 shutdown_connections 2024-08-19T10:42:43.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.799+0000 7fa83b523640 1 -- 172.21.15.89:0/3048849332 wait complete. 2024-08-19T10:42:43.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.824+0000 7f7ee976d640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:43.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.824+0000 7f7ee976d640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.824+0000 7f7ee976d640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.824+0000 7f7ee976d640 1 -- 172.21.15.190:0/1360583642 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7f7ee413bd80 con 0x7f7ee4136570 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.824+0000 7f7ecb7fe640 1 -- 172.21.15.190:0/1360583642 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v150) ==== 115+0+0 (secure 0 0 0) 0x7f7edc016620 con 0x7f7ee4136570 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.824+0000 7f7ecb7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.824+0000 7f7ecb7fe640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:compat weight-set already created 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 -- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7ed003be00 msgr2=0x7f7ed003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7ed003be00 0x7f7ed003e2c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400ba30 tx=0x7f7ed4008040 comp rx=0 tx=0).stop 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 10 monclient: shutdown 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 -- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ee4136570 msgr2=0x7f7ee4139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ee4136570 0x7f7ee4139590 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0af040 tx=0x7f7edc019740 comp rx=0 tx=0).stop 2024-08-19T10:42:43.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 -- 172.21.15.190:0/1360583642 shutdown_connections 2024-08-19T10:42:43.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7ed003be00 0x7f7ed003e2c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ee4136570 0x7f7ee4139590 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ee40a44b0 0x7f7ee41355f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 --2- 172.21.15.190:0/1360583642 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7ee40a3b90 0x7f7ee41cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 -- 172.21.15.190:0/1360583642 >> 172.21.15.190:0/1360583642 conn(0x7f7ee406dde0 msgr2=0x7f7ee413d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:43.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 -- 172.21.15.190:0/1360583642 shutdown_connections 2024-08-19T10:42:43.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.828+0000 7f7ee976d640 1 -- 172.21.15.190:0/1360583642 wait complete. 2024-08-19T10:42:43.843 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:43.843 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:42:43.844 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:42:43.847 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set create cool flat 2024-08-19T10:42:43.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient: get_monmap_and_config 2024-08-19T10:42:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient: build_initial_monmap 2024-08-19T10:42:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient: monmap: 2024-08-19T10:42:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:43.955237+0000 2024-08-19T10:42:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:43.955237+0000 2024-08-19T10:42:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:43.955 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:43.956 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:43.956 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 1 Processor -- start 2024-08-19T10:42:43.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 1 -- start start 2024-08-19T10:42:43.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient: init 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient: _add_conns ranks=[0,1,5,7,10,6,9,4,8,2,3] 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7febdc0a3620 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7febdc09c9e0 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7febdc0a4110 0x7febdc13fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient(hunting): picked mon.noname-h con 0x7febdc0a4110 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febdc0a1790 con 0x7febdc0a3620 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febdc09d1a0 con 0x7febdc09c9e0 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7febdc09dab0 con 0x7febdc0a4110 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.951+0000 7febe1b3a640 10 monclient(hunting): authenticate will time out at 1213.771884s 2024-08-19T10:42:43.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdb7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7febdc0a3620 0x7febdc0a3a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52896/0 (socket says 172.21.15.89:52896) 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdb7fe640 1 -- 172.21.15.89:0/2248112625 learned_addr learned my addr 172.21.15.89:0/2248112625 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 1 --2- 172.21.15.89:0/2248112625 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7febdc0a4110 0x7febdc13fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 10 monclient(hunting): get_auth_request con 0x7febdc0a4110 auth_method 0 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 10 monclient(hunting): handle_auth_done global_id 5657 payload 274 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 1 -- 172.21.15.89:0/2248112625 >> v1:172.21.15.190:6789/0 conn(0x7febdc09c9e0 legacy=0x7febdc13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 1 -- 172.21.15.89:0/2248112625 >> v1:172.21.15.89:6789/0 conn(0x7febdc0a3620 legacy=0x7febdc0a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:43.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:43.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 1 monclient: found mon.noname-h 2024-08-19T10:42:43.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:43.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:43.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.955+0000 7febdbfff640 1 -- 172.21.15.89:0/2248112625 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc09d480 con 0x7febdc0a4110 2024-08-19T10:42:43.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdbfff640 1 -- 172.21.15.89:0/2248112625 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7febdc0a4110 msgr2=0x7febdc13fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdbfff640 1 --2- 172.21.15.89:0/2248112625 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7febdc0a4110 0x7febdc13fd60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7febd4009dc0 tx=0x7febd401c850 comp rx=0 tx=0).ready entity=mon.5 client_cookie=72acd8b518dd0131 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 1 -- 172.21.15.89:0/2248112625 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7febd40b2070 con 0x7febdc0a4110 2024-08-19T10:42:43.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:43.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 10 monclient: dump: 2024-08-19T10:42:43.976 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.977 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 1 -- 172.21.15.89:0/2248112625 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd40a5a90 con 0x7febdc0a4110 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 5 monclient: authenticate success, global_id 5657 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 1 -- 172.21.15.89:0/2248112625 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7febd40a5cc0 con 0x7febdc0a4110 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 4 set_mon_vals no callback set 2024-08-19T10:42:43.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febda7fc640 10 monclient: dump: 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.979 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient: shutdown 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 -- 172.21.15.89:0/2248112625 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7febdc0a4110 msgr2=0x7febdc13fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 --2- 172.21.15.89:0/2248112625 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7febdc0a4110 0x7febdc13fd60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7febd4009dc0 tx=0x7febd401c850 comp rx=0 tx=0).stop 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 -- 172.21.15.89:0/2248112625 shutdown_connections 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 --2- 172.21.15.89:0/2248112625 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7febdc0a4110 0x7febdc13fd60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 -- 172.21.15.89:0/2248112625 >> 172.21.15.89:0/2248112625 conn(0x7febdc0a8210 msgr2=0x7febdc0a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:43.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 -- 172.21.15.89:0/2248112625 shutdown_connections 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 -- 172.21.15.89:0/2248112625 wait complete. 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient: build_initial_monmap 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient: monmap: 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:43.981 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 Processor -- start 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 -- start start 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient: init 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient: _add_conns ranks=[8,5,3,1,6,7,0,2,4] 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7febdc09c9e0 0x7febdc121570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient(hunting): picked mon.noname-e con 0x7febdc09c9e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:43.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7febdc0a3620 0x7febdc1d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient(hunting): picked mon.noname-h con 0x7febdc0a3620 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febdc0a4110 0x7febdc11f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient(hunting): picked mon.noname-g con 0x7febdc0a4110 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7febdc142e50 con 0x7febdc0a4110 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7febdc142cd0 con 0x7febdc0a3620 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7febdc142fd0 con 0x7febdc09c9e0 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febe1b3a640 10 monclient(hunting): authenticate will time out at 1213.779609s 2024-08-19T10:42:43.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdb7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7febdc09c9e0 0x7febdc121570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdb7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7febdc09c9e0 0x7febdc121570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40376/0 (socket says 172.21.15.89:40376) 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdb7fe640 1 -- 172.21.15.89:0/330559228 learned_addr learned my addr 172.21.15.89:0/330559228 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdb7fe640 10 monclient(hunting): get_auth_request con 0x7febdc09c9e0 auth_method 0 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdbfff640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febdc0a4110 0x7febdc11f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdbfff640 10 monclient(hunting): get_auth_request con 0x7febdc0a4110 auth_method 0 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdaffd640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7febdc0a3620 0x7febdc1d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdaffd640 10 monclient(hunting): get_auth_request con 0x7febdc0a3620 auth_method 0 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdbfff640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febdc0a4110 msgr2=0x7febdc11f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdbfff640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febdc0a4110 msgr2=0x7febdc11f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdbfff640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febdc0a4110 0x7febdc11f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdbfff640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febdc0a4110 0x7febdc11f0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.959+0000 7febdaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 10 monclient(hunting): handle_auth_done global_id 5399 payload 824 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7febdc0a3620 msgr2=0x7febdc1d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7febdc0a3620 0x7febdc1d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febdc0a4110 msgr2=0x7febdc11f0e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febdc0a4110 0x7febdc11f0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 1 monclient: found mon.noname-e 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 1 -- 172.21.15.89:0/330559228 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc0a2f60 con 0x7febdc09c9e0 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7febdc09c9e0 msgr2=0x7febdc121570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdb7fe640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7febdc09c9e0 0x7febdc121570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7febd000c4a0 tx=0x7febd000c960 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7febd0017070 con 0x7febdc09c9e0 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 10 monclient: dump: 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.987 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd0005d00 con 0x7febdc09c9e0 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7febd00074e0 con 0x7febdc09c9e0 2024-08-19T10:42:43.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 10 monclient: dump: 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.989 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febd9ffb640 4 set_mon_vals no callback set 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febe1b3a640 5 monclient: authenticate success, global_id 5399 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febe1b3a640 10 monclient: _renew_subs 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febe1b3a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febe1b3a640 1 -- 172.21.15.89:0/330559228 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febdc1227b0 con 0x7febdc09c9e0 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febe1b3a640 10 monclient: _renew_subs 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febe1b3a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:43.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febe1b3a640 1 -- 172.21.15.89:0/330559228 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7febdc1c7e60 con 0x7febdc09c9e0 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febe1b3a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febe1b3a640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc1d1b10 0x7febdc11f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febe1b3a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febe1b3a640 1 -- 172.21.15.89:0/330559228 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7febdc13fa50 con 0x7febdc1d1b10 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdaffd640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc1d1b10 0x7febdc11f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdaffd640 10 monclient: get_auth_request con 0x7febdc1d1b10 auth_method 0 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdaffd640 10 monclient(hunting): handle_auth_done global_id 5584 payload 824 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febdaffd640 1 -- 172.21.15.89:0/330559228 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febdc121f00 con 0x7febdc1d1b10 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7febd0007070 con 0x7febdc09c9e0 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7febc803be00 0x7febc803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.963+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(150..150 src has 1..150) ==== 8307+0+0 (secure 0 0 0) 0x7febd004ec60 con 0x7febdc09c9e0 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febdbfff640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7febc803be00 0x7febc803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febdbfff640 10 monclient: get_auth_request con 0x7febc803be00 auth_method 0 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febdaffd640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc1d1b10 msgr2=0x7febdc11f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febdaffd640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc1d1b10 0x7febdc11f620 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7febcc007fd0 tx=0x7febcc00fc00 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7febcc00d7a0 con 0x7febdc1d1b10 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febc3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febc3fff640 10 monclient: dump: 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:43.992 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:43.993 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:43.994 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7febcc019070 con 0x7febdc1d1b10 2024-08-19T10:42:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febc3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febc3fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc1d1b10 msgr2=0x7febdc11f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febc3fff640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc1d1b10 0x7febdc11f620 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7febcc007fd0 tx=0x7febcc00fc00 comp rx=0 tx=0).stop 2024-08-19T10:42:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc1d1b10 msgr2=0x7febdc11f620 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febc3fff640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc1d1b10 0x7febdc11f620 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:43.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.967+0000 7febdbfff640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7febc803be00 0x7febc803e2c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7febdc0a5080 tx=0x7febd40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:43.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient: get_monmap_and_config 2024-08-19T10:42:43.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient: build_initial_monmap 2024-08-19T10:42:43.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient: monmap: 2024-08-19T10:42:43.995 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:43.995 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:43.995 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:43.960564+0000 2024-08-19T10:42:43.995 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:43.960564+0000 2024-08-19T10:42:43.995 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:43.996 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 1 Processor -- start 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 1 -- start start 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient: init 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient: _add_conns ranks=[0,3,7,2,8,5,6,1,4,10,9] 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4a4809b510 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4809ea30 0x7f4a4813ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient(hunting): picked mon.noname-g con 0x7f4a4809ea30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a4809ee30 0x7f4a4813d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient(hunting): picked mon.noname-i con 0x7f4a4809ee30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a4809db00 con 0x7f4a4809b510 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4809dde0 con 0x7f4a4809ea30 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4a4809e0c0 con 0x7f4a4809ee30 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4f57b640 10 monclient(hunting): authenticate will time out at 1209.531453s 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4cb16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4809ea30 0x7f4a4813ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a4809ee30 0x7f4a4813d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4cb16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4809ea30 0x7f4a4813ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:56072/0 (socket says 172.21.15.190:56072) 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4cb16640 1 -- 172.21.15.190:0/3361112448 learned_addr learned my addr 172.21.15.190:0/3361112448 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4cb16640 10 monclient(hunting): get_auth_request con 0x7f4a4809ea30 auth_method 0 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4cb16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4cb16640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4cb16640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4d317640 1 --1- 172.21.15.190:0/3361112448 >> v1:172.21.15.89:6789/0 conn(0x7f4a4809b510 0x7f4a4809b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:57900/0 (socket says 172.21.15.190:57900) 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 10 monclient(hunting): get_auth_request con 0x7f4a4809ee30 auth_method 0 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4cb16640 1 -- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4809ea30 msgr2=0x7f4a4813ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4cb16640 1 -- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4809ea30 msgr2=0x7f4a4813ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4cb16640 1 --2- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4809ea30 0x7f4a4813ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4cb16640 1 --2- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4809ea30 0x7f4a4813ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:43.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 10 monclient(hunting): handle_auth_done global_id 14839 payload 274 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 1 -- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4809ea30 msgr2=0x7f4a4813ac90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 1 --2- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4809ea30 0x7f4a4813ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 1 -- 172.21.15.190:0/3361112448 >> v1:172.21.15.89:6789/0 conn(0x7f4a4809b510 legacy=0x7f4a4809b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.956+0000 7f4a4db18640 10 monclient: _finish_hunting 0 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.960+0000 7f4a4db18640 1 monclient: found mon.noname-i 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.960+0000 7f4a4db18640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.960+0000 7f4a4db18640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.960+0000 7f4a4db18640 1 -- 172.21.15.190:0/3361112448 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a480a5c50 con 0x7f4a4809ee30 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4db18640 1 -- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a4809ee30 msgr2=0x7f4a4813d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4db18640 1 --2- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a4809ee30 0x7f4a4813d1e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009dc0 tx=0x7f4a4001c850 comp rx=0 tx=0).ready entity=mon.7 client_cookie=578fbdbadda4e978 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 1 -- 172.21.15.190:0/3361112448 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a400a8aa0 con 0x7f4a4809ee30 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 10 monclient: dump: 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.001 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 10 monclient: _finish_auth 0 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 1 -- 172.21.15.190:0/3361112448 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a400a8c60 con 0x7f4a4809ee30 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 5 monclient: authenticate success, global_id 14839 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:44.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 1 -- 172.21.15.190:0/3361112448 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a400a8e10 con 0x7f4a4809ee30 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 4 set_mon_vals no callback set 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a3ffff640 10 monclient: dump: 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.003 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient: shutdown 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 -- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a4809ee30 msgr2=0x7f4a4813d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 --2- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a4809ee30 0x7f4a4813d1e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009dc0 tx=0x7f4a4001c850 comp rx=0 tx=0).stop 2024-08-19T10:42:44.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 -- 172.21.15.190:0/3361112448 shutdown_connections 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 --2- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a4809ee30 0x7f4a4813d1e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 --2- 172.21.15.190:0/3361112448 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a4809ea30 0x7f4a4813ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 -- 172.21.15.190:0/3361112448 >> 172.21.15.190:0/3361112448 conn(0x7f4a480a4e70 msgr2=0x7f4a480a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 -- 172.21.15.190:0/3361112448 shutdown_connections 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 -- 172.21.15.190:0/3361112448 wait complete. 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient: build_initial_monmap 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient: monmap: 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:44.005 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 Processor -- start 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 -- start start 2024-08-19T10:42:44.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient: init 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient: _add_conns ranks=[8,5,6,2,7,3,4,1,0] 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4a4809b510 0x7f4a48092f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient(hunting): picked mon.noname-e con 0x7f4a4809b510 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a4809ea30 0x7f4a48093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient(hunting): picked mon.noname-h con 0x7f4a4809ea30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a4809ee30 0x7f4a480939a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient(hunting): picked mon.noname-d con 0x7f4a4809ee30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4a4813e0b0 con 0x7f4a4809ea30 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4a4813df30 con 0x7f4a4809ee30 2024-08-19T10:42:44.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4a4813e230 con 0x7f4a4809b510 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4f57b640 10 monclient(hunting): authenticate will time out at 1209.538527s 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a4809ea30 0x7f4a48093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a4809ea30 0x7f4a48093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54604/0 (socket says 172.21.15.190:54604) 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 1 -- 172.21.15.190:0/1767900062 learned_addr learned my addr 172.21.15.190:0/1767900062 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 10 monclient(hunting): get_auth_request con 0x7f4a4809ea30 auth_method 0 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4db18640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a4809ee30 0x7f4a480939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4d317640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4a4809b510 0x7f4a48092f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4db18640 10 monclient(hunting): get_auth_request con 0x7f4a4809ee30 auth_method 0 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4db18640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4db18640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4db18640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4d317640 10 monclient(hunting): get_auth_request con 0x7f4a4809b510 auth_method 0 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4d317640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4d317640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4d317640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4db18640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4db18640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4db18640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4d317640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4d317640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4d317640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 10 monclient(hunting): handle_auth_done global_id 5675 payload 824 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 1 -- 172.21.15.190:0/1767900062 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4a4809b510 msgr2=0x7f4a48092f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4a4809b510 0x7f4a48092f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 1 -- 172.21.15.190:0/1767900062 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a4809ee30 msgr2=0x7f4a480939a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a4809ee30 0x7f4a480939a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 10 monclient: _finish_hunting 0 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 1 monclient: found mon.noname-h 2024-08-19T10:42:44.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.964+0000 7f4a4cb16640 1 -- 172.21.15.190:0/1767900062 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a480a48f0 con 0x7f4a4809ea30 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4cb16640 1 -- 172.21.15.190:0/1767900062 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a4809ea30 msgr2=0x7f4a48093460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4cb16640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a4809ea30 0x7f4a48093460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800e840 tx=0x7f4a3800ed00 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 1 -- 172.21.15.190:0/1767900062 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a38017070 con 0x7f4a4809ea30 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 10 monclient: dump: 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.011 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:44.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 1 -- 172.21.15.190:0/1767900062 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a3800cac0 con 0x7f4a4809ea30 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4f57b640 5 monclient: authenticate success, global_id 5675 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4f57b640 10 monclient: _renew_subs 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4f57b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4f57b640 1 -- 172.21.15.190:0/1767900062 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a48095b70 con 0x7f4a4809ea30 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 1 -- 172.21.15.190:0/1767900062 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a3800f040 con 0x7f4a4809ea30 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3f7fe640 4 set_mon_vals no callback set 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 10 monclient: dump: 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:44.013 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.014 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4f57b640 10 monclient: _renew_subs 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4f57b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4f57b640 1 -- 172.21.15.190:0/1767900062 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a48095890 con 0x7f4a4809ea30 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4f57b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4f57b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4f57b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4f57b640 1 -- 172.21.15.190:0/1767900062 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4809b8f0 con 0x7f4a4809ea30 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 1 -- 172.21.15.190:0/1767900062 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4a3800cc70 con 0x7f4a4809ea30 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a2c03be60 0x7f4a2c03e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a3dffb640 1 -- 172.21.15.190:0/1767900062 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(150..150 src has 1..150) ==== 8307+0+0 (secure 0 0 0) 0x7f4a3804f010 con 0x7f4a4809ea30 2024-08-19T10:42:44.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4d317640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a2c03be60 0x7f4a2c03e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.968+0000 7f4a4d317640 10 monclient: get_auth_request con 0x7f4a2c03be60 auth_method 0 2024-08-19T10:42:44.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.972+0000 7f4a4d317640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a2c03be60 0x7f4a2c03e320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4a44007240 tx=0x7f4a44007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.972+0000 7f4a3dffb640 1 -- 172.21.15.190:0/1767900062 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4a38020070 con 0x7f4a4809ea30 2024-08-19T10:42:44.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.972+0000 7f4a3dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:44.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:43.972+0000 7f4a3dffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:44.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:44.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc11fb60 0x7febdc070d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 1 -- 172.21.15.89:0/330559228 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7febdc11ff70 con 0x7febdc11fb60 2024-08-19T10:42:44.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc11fb60 0x7febdc070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 10 monclient: get_auth_request con 0x7febdc11fb60 auth_method 0 2024-08-19T10:42:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 10 monclient(hunting): handle_auth_done global_id 5593 payload 824 2024-08-19T10:42:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 1 -- 172.21.15.89:0/330559228 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7febdc064590 con 0x7febdc11fb60 2024-08-19T10:42:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc11fb60 msgr2=0x7febdc070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdaffd640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc11fb60 0x7febdc070d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7febcc007fd0 tx=0x7febcc003770 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7febcc00dd50 con 0x7febdc11fb60 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febc3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febc3fff640 10 monclient: dump: 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.019 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7febcc00dd50 con 0x7febdc11fb60 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febc3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febc3fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc11fb60 msgr2=0x7febdc070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febc3fff640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc11fb60 0x7febdc070d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7febcc007fd0 tx=0x7febcc003770 comp rx=0 tx=0).stop 2024-08-19T10:42:44.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febc3fff640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc11fb60 msgr2=0x7febdc070d10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:44.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febc3fff640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7febdc11fb60 0x7febdc070d10 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7febcc007fd0 tx=0x7febcc003770 comp rx=0 tx=0).stop 2024-08-19T10:42:44.021 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:44.021 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:44.021 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:44.021 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:44.021 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:44.021 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 293792, 2024-08-19T10:42:44.021 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:44.021 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:44.022 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.023 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.024 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:44.025 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.026 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:44.027 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:44.029 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:44.029 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.029 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.029 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.029 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:44.029 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:44.029 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.030 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:44.034 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.036 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:44.038 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:44.041 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:44.042 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.043 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.044 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:44.045 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:44.045 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:44.045 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.045 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.045 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:44.045 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:44.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7febc803be00 msgr2=0x7febc803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7febc803be00 0x7febc803e2c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7febdc0a5080 tx=0x7febd40a7070 comp rx=0 tx=0).stop 2024-08-19T10:42:44.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 10 monclient: shutdown 2024-08-19T10:42:44.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:44.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 1 -- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7febdc09c9e0 msgr2=0x7febdc121570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 1 --2- 172.21.15.89:0/330559228 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7febdc09c9e0 0x7febdc121570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7febd000c4a0 tx=0x7febd000c960 comp rx=0 tx=0).stop 2024-08-19T10:42:44.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdbfff640 1 -- 172.21.15.89:0/330559228 reap_dead start 2024-08-19T10:42:44.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febdbfff640 1 -- 172.21.15.89:0/330559228 reap_dead start 2024-08-19T10:42:44.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 1 -- 172.21.15.89:0/330559228 shutdown_connections 2024-08-19T10:42:44.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 1 -- 172.21.15.89:0/330559228 >> 172.21.15.89:0/330559228 conn(0x7febdc0a8210 msgr2=0x7febdc09fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:44.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 1 -- 172.21.15.89:0/330559228 shutdown_connections 2024-08-19T10:42:44.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:43.975+0000 7febe1b3a640 1 -- 172.21.15.89:0/330559228 wait complete. 2024-08-19T10:42:44.048 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:44.048 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:42:44.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:42:44.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.152+0000 7f4a4f57b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-08-19T10:42:44.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.152+0000 7f4a4f57b640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-08-19T10:42:44.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.152+0000 7f4a4f57b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:44.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.152+0000 7f4a4f57b640 1 -- 172.21.15.190:0/1767900062 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"} v 0) -- 0x7f4a4806e490 con 0x7f4a4809ea30 2024-08-19T10:42:44.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient: get_monmap_and_config 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient: build_initial_monmap 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient: monmap: 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:44.173233+0000 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:44.173233+0000 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:44.173 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:44.174 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:44.174 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:44.174 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:44.174 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:44.174 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:44.174 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:44.174 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 1 Processor -- start 2024-08-19T10:42:44.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 1 -- start start 2024-08-19T10:42:44.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient: init 2024-08-19T10:42:44.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:44.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient: _add_conns ranks=[8,1,0,5,2,10,9,6,4,3,7] 2024-08-19T10:42:44.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f10600a6860 0x7f10600a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient(hunting): picked mon.noname-e con 0x7f10600a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:44.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f10600a7180 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:44.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f10600a7560 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:44.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10600a33d0 con 0x7f10600a7560 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f106013a440 con 0x7f10600a7180 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f10600a8190 con 0x7f10600a6860 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1067925640 10 monclient(hunting): authenticate will time out at 1213.989951s 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f10600a6860 0x7f10600a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1065ec2640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f10600a7560 0x7f1060139d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52912/0 (socket says 172.21.15.89:52912) 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f1065ec2640 1 -- 172.21.15.89:0/876998655 learned_addr learned my addr 172.21.15.89:0/876998655 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 10 monclient(hunting): get_auth_request con 0x7f10600a6860 auth_method 0 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 10 monclient(hunting): handle_auth_done global_id 5417 payload 274 2024-08-19T10:42:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 1 -- 172.21.15.89:0/876998655 >> v1:172.21.15.190:6789/0 conn(0x7f10600a7180 legacy=0x7f1060139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 1 -- 172.21.15.89:0/876998655 >> v1:172.21.15.89:6789/0 conn(0x7f10600a7560 legacy=0x7f1060139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 10 monclient: _finish_hunting 0 2024-08-19T10:42:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 1 monclient: found mon.noname-e 2024-08-19T10:42:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:44.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.171+0000 7f10656c1640 1 -- 172.21.15.89:0/876998655 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10600a2ec0 con 0x7f10600a6860 2024-08-19T10:42:44.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f10656c1640 1 -- 172.21.15.89:0/876998655 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f10600a6860 msgr2=0x7f10600a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f10656c1640 1 --2- 172.21.15.89:0/876998655 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f10600a6860 0x7f10600a6c40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f105c0097f0 tx=0x7f105c01c430 comp rx=0 tx=0).ready entity=mon.8 client_cookie=cd545155000164ad server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 1 -- 172.21.15.89:0/876998655 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f105c0b0070 con 0x7f10600a6860 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 10 monclient: dump: 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.181 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.182 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:44.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 1 -- 172.21.15.89:0/876998655 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105c0aa810 con 0x7f10600a6860 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 1 -- 172.21.15.89:0/876998655 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f105c0a93e0 con 0x7f10600a6860 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1057fff640 10 monclient: dump: 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.183 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 5 monclient: authenticate success, global_id 5417 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 4 set_mon_vals no callback set 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 10 monclient: shutdown 2024-08-19T10:42:44.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 1 -- 172.21.15.89:0/876998655 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f10600a6860 msgr2=0x7f10600a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 1 --2- 172.21.15.89:0/876998655 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f10600a6860 0x7f10600a6c40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f105c0097f0 tx=0x7f105c01c430 comp rx=0 tx=0).stop 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 1 -- 172.21.15.89:0/876998655 shutdown_connections 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 1 --2- 172.21.15.89:0/876998655 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f10600a6860 0x7f10600a6c40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 1 -- 172.21.15.89:0/876998655 >> 172.21.15.89:0/876998655 conn(0x7f1060074b40 msgr2=0x7f10600abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 1 -- 172.21.15.89:0/876998655 shutdown_connections 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.175+0000 7f1067925640 1 -- 172.21.15.89:0/876998655 wait complete. 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient: build_initial_monmap 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient: monmap: 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:44.185 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:44.186 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 1 Processor -- start 2024-08-19T10:42:44.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 1 -- start start 2024-08-19T10:42:44.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient: init 2024-08-19T10:42:44.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:44.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient: _add_conns ranks=[6,4,3,7,5,8,0,2,1] 2024-08-19T10:42:44.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10600a6860 0x7f1060138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient(hunting): picked mon.noname-d con 0x7f10600a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10600a7180 0x7f1060139220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient(hunting): picked mon.noname-c con 0x7f10600a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10600a7560 0x7f106013e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient(hunting): picked mon.noname-g con 0x7f10600a7560 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f10601454c0 con 0x7f10600a7560 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f10600a79f0 con 0x7f10600a7180 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1060145640 con 0x7f10600a6860 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:44.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1067925640 10 monclient(hunting): authenticate will time out at 1213.997442s 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10600a6860 0x7f1060138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1064ec0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10600a7180 0x7f1060139220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10600a6860 0x7f1060138ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54450/0 (socket says 172.21.15.89:54450) 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 1 -- 172.21.15.89:0/386110618 learned_addr learned my addr 172.21.15.89:0/386110618 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 10 monclient(hunting): get_auth_request con 0x7f10600a6860 auth_method 0 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1064ec0640 10 monclient(hunting): get_auth_request con 0x7f10600a7180 auth_method 0 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1064ec0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1064ec0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1064ec0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1064ec0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1064ec0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1064ec0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1065ec2640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10600a7560 0x7f106013e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1065ec2640 10 monclient(hunting): get_auth_request con 0x7f10600a7560 auth_method 0 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1065ec2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1065ec2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f1065ec2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 10 monclient(hunting): handle_auth_done global_id 5487 payload 824 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 1 -- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10600a7180 msgr2=0x7f1060139220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10600a7180 0x7f1060139220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 1 -- 172.21.15.89:0/386110618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10600a7560 msgr2=0x7f106013e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10600a7560 0x7f106013e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 10 monclient: _finish_hunting 0 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 1 monclient: found mon.noname-d 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.179+0000 7f10656c1640 1 -- 172.21.15.89:0/386110618 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1060122830 con 0x7f10600a6860 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f10656c1640 1 -- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10600a6860 msgr2=0x7f1060138ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f10656c1640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10600a6860 0x7f1060138ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f105c0ae040 tx=0x7f105c0191d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f105c0b0040 con 0x7f10600a6860 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:44.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 10 monclient: dump: 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.196 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105c0b9070 con 0x7f10600a6860 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f105c0044b0 con 0x7f10600a6860 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 10 monclient: dump: 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.197 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.198 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f10577fe640 4 set_mon_vals no callback set 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1067925640 5 monclient: authenticate success, global_id 5487 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1067925640 10 monclient: _renew_subs 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1067925640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1067925640 1 -- 172.21.15.89:0/386110618 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1060140090 con 0x7f10600a6860 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1067925640 10 monclient: _renew_subs 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1067925640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1067925640 1 -- 172.21.15.89:0/386110618 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f10600a9060 con 0x7f10600a6860 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1067925640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1067925640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013e7d0 0x7f106013ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1067925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1067925640 1 -- 172.21.15.89:0/386110618 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1060140750 con 0x7f106013e7d0 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1064ec0640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013e7d0 0x7f106013ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1064ec0640 10 monclient: get_auth_request con 0x7f106013e7d0 auth_method 0 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1064ec0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1064ec0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1064ec0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1064ec0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1064ec0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1064ec0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f105c003620 con 0x7f10600a6860 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1064ec0640 10 monclient(hunting): handle_auth_done global_id 5496 payload 824 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1064ec0640 1 -- 172.21.15.89:0/386110618 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10600a2670 con 0x7f106013e7d0 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f104403be30 0x7f104403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(150..150 src has 1..150) ==== 8307+0+0 (secure 0 0 0) 0x7f105c0e97f0 con 0x7f10600a6860 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1065ec2640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f104403be30 0x7f104403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.183+0000 7f1065ec2640 10 monclient: get_auth_request con 0x7f104403be30 auth_method 0 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1064ec0640 1 -- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013e7d0 msgr2=0x7f106013ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1064ec0640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013e7d0 0x7f106013ebb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1050007fd0 tx=0x7f105000fc00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f105000d7a0 con 0x7f106013e7d0 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1055ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1055ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1055ffb640 10 monclient: dump: 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.201 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1050019070 con 0x7f106013e7d0 2024-08-19T10:42:44.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1055ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:44.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1055ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:44.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013e7d0 msgr2=0x7f106013ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1055ffb640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013e7d0 0x7f106013ebb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1050007fd0 tx=0x7f105000fc00 comp rx=0 tx=0).stop 2024-08-19T10:42:44.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013e7d0 msgr2=0x7f106013ebb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:44.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1055ffb640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013e7d0 0x7f106013ebb0 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1050007fd0 tx=0x7f105000fc00 comp rx=0 tx=0).stop 2024-08-19T10:42:44.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.187+0000 7f1065ec2640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f104403be30 0x7f104403e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10601412d0 tx=0x7f1058009480 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.191+0000 7f1067925640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:44.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.191+0000 7f1067925640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013f0f0 0x7f10600a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.191+0000 7f1067925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.191+0000 7f1067925640 1 -- 172.21.15.89:0/386110618 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f10600a6c40 con 0x7f106013f0f0 2024-08-19T10:42:44.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013f0f0 0x7f10600a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 10 monclient: get_auth_request con 0x7f106013f0f0 auth_method 0 2024-08-19T10:42:44.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 10 monclient(hunting): handle_auth_done global_id 5505 payload 824 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 1 -- 172.21.15.89:0/386110618 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1060064590 con 0x7f106013f0f0 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 1 -- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013f0f0 msgr2=0x7f10600a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1064ec0640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013f0f0 0x7f10600a09a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1050016040 tx=0x7f1050003770 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f105000dbc0 con 0x7f106013f0f0 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1055ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1055ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1055ffb640 10 monclient: dump: 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.206 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.207 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9601 (secure 0 0 0) 0x7f105000dbc0 con 0x7f106013f0f0 2024-08-19T10:42:44.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1055ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:44.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1055ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:44.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013f0f0 msgr2=0x7f10600a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1055ffb640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013f0f0 0x7f10600a09a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1050016040 tx=0x7f1050003770 comp rx=0 tx=0).stop 2024-08-19T10:42:44.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1055ffb640 1 -- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013f0f0 msgr2=0x7f10600a09a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:44.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1055ffb640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f106013f0f0 0x7f10600a09a0 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1050016040 tx=0x7f1050003770 comp rx=0 tx=0).stop 2024-08-19T10:42:44.208 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 294012, 2024-08-19T10:42:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.211 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:44.212 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:44.213 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.214 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:44.218 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.219 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:44.221 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:44.223 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:44.223 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.223 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.223 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:44.225 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.227 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.229 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:44.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:44.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.232 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.233 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.234 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:44.234 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1067925640 1 -- 172.21.15.89:0/386110618 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f104403be30 msgr2=0x7f104403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1067925640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f104403be30 0x7f104403e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10601412d0 tx=0x7f1058009480 comp rx=0 tx=0).stop 2024-08-19T10:42:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1067925640 10 monclient: shutdown 2024-08-19T10:42:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1067925640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1067925640 1 -- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10600a6860 msgr2=0x7f1060138ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1067925640 1 --2- 172.21.15.89:0/386110618 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10600a6860 0x7f1060138ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f105c0ae040 tx=0x7f105c0191d0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1065ec2640 1 -- 172.21.15.89:0/386110618 reap_dead start 2024-08-19T10:42:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1065ec2640 1 -- 172.21.15.89:0/386110618 reap_dead start 2024-08-19T10:42:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1067925640 1 -- 172.21.15.89:0/386110618 shutdown_connections 2024-08-19T10:42:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1067925640 1 -- 172.21.15.89:0/386110618 >> 172.21.15.89:0/386110618 conn(0x7f1060074b40 msgr2=0x7f106009cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1067925640 1 -- 172.21.15.89:0/386110618 shutdown_connections 2024-08-19T10:42:44.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.195+0000 7f1067925640 1 -- 172.21.15.89:0/386110618 wait complete. 2024-08-19T10:42:44.237 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:44.237 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:42:44.237 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:42:44.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient: get_monmap_and_config 2024-08-19T10:42:44.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient: build_initial_monmap 2024-08-19T10:42:44.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient: monmap: 2024-08-19T10:42:44.353 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:44.353 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:44.353719+0000 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:44.353719+0000 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:44.354 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:44.355 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:44.355 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 1 Processor -- start 2024-08-19T10:42:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 1 -- start start 2024-08-19T10:42:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient: init 2024-08-19T10:42:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient: _add_conns ranks=[2,8,10,3,5,0,6,7,1,9,4] 2024-08-19T10:42:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f39280a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient(hunting): picked mon.noname-b con 0x7f39280a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39280a6d30 0x7f39280a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient(hunting): picked mon.noname-e con 0x7f39280a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f39281395b0 0x7f3928139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient(hunting): picked mon.noname-f con 0x7f39281395b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f39280a7650 con 0x7f39281395b0 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f39280a8710 con 0x7f39280a5270 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f39280a5a40 con 0x7f39280a6d30 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f392dba6640 10 monclient(hunting): authenticate will time out at 1214.170407s 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f39280a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3926ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39280a6d30 0x7f39280a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f39280a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40090/0 (socket says 172.21.15.89:40090) 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3926ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39280a6d30 0x7f39280a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40390/0 (socket says 172.21.15.89:40390) 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 1 -- 172.21.15.89:0/3951521403 learned_addr learned my addr 172.21.15.89:0/3951521403 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 10 monclient(hunting): get_auth_request con 0x7f39280a5270 auth_method 0 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3926ffd640 10 monclient(hunting): get_auth_request con 0x7f39280a6d30 auth_method 0 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3926ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3926ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3926ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3926ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3926ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3926ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3927fff640 1 --2- 172.21.15.89:0/3951521403 >> v2:172.21.15.190:3300/0 conn(0x7f39281395b0 0x7f3928139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 10 monclient(hunting): handle_auth_done global_id 5582 payload 274 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 1 -- 172.21.15.89:0/3951521403 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39280a6d30 msgr2=0x7f39280a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 1 --2- 172.21.15.89:0/3951521403 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39280a6d30 0x7f39280a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 1 -- 172.21.15.89:0/3951521403 >> v2:172.21.15.190:3300/0 conn(0x7f39281395b0 msgr2=0x7f3928139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 1 --2- 172.21.15.89:0/3951521403 >> v2:172.21.15.190:3300/0 conn(0x7f39281395b0 0x7f3928139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 1 monclient: found mon.noname-b 2024-08-19T10:42:44.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:44.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:42:44.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f39277fe640 1 -- 172.21.15.89:0/3951521403 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39280a81d0 con 0x7f39280a5270 2024-08-19T10:42:44.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3927fff640 10 monclient: get_auth_request con 0x7f39281395b0 auth_method 0 2024-08-19T10:42:44.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.351+0000 7f3927fff640 1 --2- 172.21.15.89:0/3951521403 >> v2:172.21.15.190:3300/0 conn(0x7f39281395b0 0x7f3928139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:44.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39277fe640 1 -- 172.21.15.89:0/3951521403 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 msgr2=0x7f39280a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39277fe640 1 --2- 172.21.15.89:0/3951521403 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f39280a67f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3918009870 tx=0x7f391801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fafdf35100ee9c01 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 1 -- 172.21.15.89:0/3951521403 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39180b1070 con 0x7f39280a5270 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 10 monclient: dump: 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.367 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:44.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 1 -- 172.21.15.89:0/3951521403 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39180ab810 con 0x7f39280a5270 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 1 -- 172.21.15.89:0/3951521403 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39180aa300 con 0x7f39280a5270 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f392dba6640 5 monclient: authenticate success, global_id 5582 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f392dba6640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f392dba6640 4 set_mon_vals no callback set 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f39267fc640 10 monclient: dump: 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.369 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f392dba6640 10 monclient: shutdown 2024-08-19T10:42:44.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f392dba6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f392dba6640 1 -- 172.21.15.89:0/3951521403 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 msgr2=0x7f39280a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f392dba6640 1 --2- 172.21.15.89:0/3951521403 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f39280a67f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3918009870 tx=0x7f391801c420 comp rx=0 tx=0).stop 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f392dba6640 1 -- 172.21.15.89:0/3951521403 shutdown_connections 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f392dba6640 1 --2- 172.21.15.89:0/3951521403 >> v2:172.21.15.190:3300/0 conn(0x7f39281395b0 0x7f3928139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f392dba6640 1 --2- 172.21.15.89:0/3951521403 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39280a6d30 0x7f39280a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.355+0000 7f392dba6640 1 --2- 172.21.15.89:0/3951521403 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f39280a67f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 1 -- 172.21.15.89:0/3951521403 >> 172.21.15.89:0/3951521403 conn(0x7f3928069a80 msgr2=0x7f39280abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 1 -- 172.21.15.89:0/3951521403 shutdown_connections 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 1 -- 172.21.15.89:0/3951521403 wait complete. 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient: build_initial_monmap 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient: monmap: 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:44.371 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:44.372 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 1 Processor -- start 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 1 -- start start 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient: init 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient: _add_conns ranks=[2,5,0,6,4,3,7,8,1] 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f392813ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient(hunting): picked mon.noname-b con 0x7f39280a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39280a6d30 0x7f392813d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient(hunting): picked mon.noname-h con 0x7f39280a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39281395b0 0x7f392813d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient(hunting): picked mon.noname-a con 0x7f39281395b0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:44.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39280a5650 con 0x7f39281395b0 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f392813a350 con 0x7f39280a5270 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f39280a57d0 con 0x7f39280a6d30 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f392dba6640 10 monclient(hunting): authenticate will time out at 1214.177508s 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f392813ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f392813ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40098/0 (socket says 172.21.15.89:40098) 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f3927fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39281395b0 0x7f392813d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 1 -- 172.21.15.89:0/2831095385 learned_addr learned my addr 172.21.15.89:0/2831095385 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 10 monclient(hunting): get_auth_request con 0x7f39280a5270 auth_method 0 2024-08-19T10:42:44.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f3927fff640 10 monclient(hunting): get_auth_request con 0x7f39281395b0 auth_method 0 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f3927fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f3927fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f3927fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f3927fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f3927fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f3927fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 10 monclient(hunting): handle_auth_done global_id 5591 payload 824 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 1 -- 172.21.15.89:0/2831095385 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39280a6d30 msgr2=0x7f392813d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f3926ffd640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39280a6d30 0x7f392813d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f39280a6d30 0x7f392813d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 1 -- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39281395b0 msgr2=0x7f392813d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39281395b0 0x7f392813d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 1 monclient: found mon.noname-b 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f39277fe640 1 -- 172.21.15.89:0/2831095385 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392809d7b0 con 0x7f39280a5270 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.359+0000 7f3927fff640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39281395b0 0x7f392813d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f39277fe640 1 -- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 msgr2=0x7f392813ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f39277fe640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f392813ce80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f39180af040 tx=0x7f39180191c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39180b1040 con 0x7f39280a5270 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 10 monclient: dump: 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.377 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 10 monclient: _finish_auth 0 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f392dba6640 5 monclient: authenticate success, global_id 5591 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39180ba070 con 0x7f39280a5270 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f392dba6640 10 monclient: _renew_subs 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f392dba6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:44.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f392dba6640 1 -- 172.21.15.89:0/2831095385 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3928139be0 con 0x7f39280a5270 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f39180abe30 con 0x7f39280a5270 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 10 monclient: dump: 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.379 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3925ffb640 4 set_mon_vals no callback set 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f392dba6640 10 monclient: _renew_subs 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f392dba6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f392dba6640 1 -- 172.21.15.89:0/2831095385 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f392813e7e0 con 0x7f39280a5270 2024-08-19T10:42:44.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f392dba6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f392dba6640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39281d1800 0x7f39281d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f392dba6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f392dba6640 1 -- 172.21.15.89:0/2831095385 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3928135790 con 0x7f39281d1800 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3918004250 con 0x7f39280a5270 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3926ffd640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39281d1800 0x7f39281d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3926ffd640 10 monclient: get_auth_request con 0x7f39281d1800 auth_method 0 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3926ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3926ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3926ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3926ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3926ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3926ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f391403be00 0x7f391403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(150..150 src has 1..150) ==== 8307+0+0 (secure 0 0 0) 0x7f39180e9560 con 0x7f39280a5270 2024-08-19T10:42:44.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3926ffd640 10 monclient(hunting): handle_auth_done global_id 5435 payload 824 2024-08-19T10:42:44.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3926ffd640 1 -- 172.21.15.89:0/2831095385 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f392809d0e0 con 0x7f39281d1800 2024-08-19T10:42:44.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3927fff640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f391403be00 0x7f391403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.363+0000 7f3927fff640 10 monclient: get_auth_request con 0x7f391403be00 auth_method 0 2024-08-19T10:42:44.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f3926ffd640 1 -- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39281d1800 msgr2=0x7f39281d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f3926ffd640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39281d1800 0x7f39281d1be0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f391c010790 tx=0x7f391c010c50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f391c00f040 con 0x7f39281d1800 2024-08-19T10:42:44.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f390ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f390ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:44.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f390ffff640 10 monclient: dump: 2024-08-19T10:42:44.384 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.385 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f391c01c400 con 0x7f39281d1800 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f390ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f390ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39281d1800 msgr2=0x7f39281d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f390ffff640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39281d1800 0x7f39281d1be0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f391c010790 tx=0x7f391c010c50 comp rx=0 tx=0).stop 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39281d1800 msgr2=0x7f39281d1be0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f390ffff640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f39281d1800 0x7f39281d1be0 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f391c010790 tx=0x7f391c010c50 comp rx=0 tx=0).stop 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.367+0000 7f3927fff640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f391403be00 0x7f391403e2c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3920007fd0 tx=0x7f3920007780 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f392dba6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f392dba6640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f392802d1d0 0x7f392802d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f392dba6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f392dba6640 1 -- 172.21.15.89:0/2831095385 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f39280a7110 con 0x7f392802d1d0 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f3926ffd640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f392802d1d0 0x7f392802d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f3926ffd640 10 monclient: get_auth_request con 0x7f392802d1d0 auth_method 0 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f3926ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f3926ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f3926ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f3926ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f3926ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f3926ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f3926ffd640 10 monclient(hunting): handle_auth_done global_id 5444 payload 824 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.371+0000 7f3926ffd640 1 -- 172.21.15.89:0/2831095385 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3928064590 con 0x7f392802d1d0 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f3926ffd640 1 -- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f392802d1d0 msgr2=0x7f392802d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f3926ffd640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f392802d1d0 0x7f392802d5e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f391c010790 tx=0x7f391c0046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f391c0064a0 con 0x7f392802d1d0 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f390ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f390ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f390ffff640 10 monclient: dump: 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.388 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f391c000e50 con 0x7f392802d1d0 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f390ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f390ffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f392802d1d0 msgr2=0x7f392802d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f390ffff640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f392802d1d0 0x7f392802d5e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f391c010790 tx=0x7f391c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f390ffff640 1 -- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f392802d1d0 msgr2=0x7f392802d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:44.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f390ffff640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f392802d1d0 0x7f392802d5e0 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f391c010790 tx=0x7f391c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.390 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:44.390 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:44.390 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:44.390 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:44.390 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:44.390 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 294168, 2024-08-19T10:42:44.390 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:44.390 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:44.390 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.391 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.392 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:44.393 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:44.394 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.395 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.397 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:44.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.402 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:44.405 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.406 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.408 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.409 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:44.411 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.412 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.413 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:44.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.413 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:44.413 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:44.413 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:44.413 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.413 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.413 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:44.413 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f392dba6640 1 -- 172.21.15.89:0/2831095385 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f391403be00 msgr2=0x7f391403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f392dba6640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f391403be00 0x7f391403e2c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3920007fd0 tx=0x7f3920007780 comp rx=0 tx=0).stop 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f392dba6640 10 monclient: shutdown 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f392dba6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f392dba6640 1 -- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 msgr2=0x7f392813ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f392dba6640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f392813ce80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f39180af040 tx=0x7f39180191c0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f3927fff640 1 -- 172.21.15.89:0/2831095385 reap_dead start 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f392dba6640 1 -- 172.21.15.89:0/2831095385 shutdown_connections 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f392dba6640 1 --2- 172.21.15.89:0/2831095385 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f39280a5270 0x7f392813ce80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f392dba6640 1 -- 172.21.15.89:0/2831095385 >> 172.21.15.89:0/2831095385 conn(0x7f3928069a80 msgr2=0x7f39280ab130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f392dba6640 1 -- 172.21.15.89:0/2831095385 shutdown_connections 2024-08-19T10:42:44.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.375+0000 7f392dba6640 1 -- 172.21.15.89:0/2831095385 wait complete. 2024-08-19T10:42:44.416 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:44.416 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:42:44.416 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:42:44.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient: get_monmap_and_config 2024-08-19T10:42:44.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient: build_initial_monmap 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient: monmap: 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:44.533088+0000 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:44.533088+0000 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:44.533 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:44.534 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:44.534 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:44.534 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:44.534 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:44.534 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:44.534 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 1 Processor -- start 2024-08-19T10:42:44.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 1 -- start start 2024-08-19T10:42:44.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient: init 2024-08-19T10:42:44.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:44.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient: _add_conns ranks=[2,6,0,5,10,3,9,7,1,4,8] 2024-08-19T10:42:44.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc0a6860 0x7fb1fc0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient(hunting): picked mon.noname-b con 0x7fb1fc0a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:44.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1fc0a7180 0x7fb1fc0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient(hunting): picked mon.noname-d con 0x7fb1fc0a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:44.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb1fc139620 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:44.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1fc0a33d0 con 0x7fb1fc139620 2024-08-19T10:42:44.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc0a8190 con 0x7fb1fc0a6860 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb1fc0a2ec0 con 0x7fb1fc0a7180 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb200eba640 10 monclient(hunting): authenticate will time out at 1214.349583s 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fa59c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc0a6860 0x7fb1fc0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1fc0a7180 0x7fb1fc0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fa59c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc0a6860 0x7fb1fc0a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40112/0 (socket says 172.21.15.89:40112) 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1fc0a7180 0x7fb1fc0a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54476/0 (socket says 172.21.15.89:54476) 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fad9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb1fc139620 0x7fb1fc139a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52914/0 (socket says 172.21.15.89:52914) 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fa59c640 1 -- 172.21.15.89:0/951491428 learned_addr learned my addr 172.21.15.89:0/951491428 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fa59c640 10 monclient(hunting): get_auth_request con 0x7fb1fc0a6860 auth_method 0 2024-08-19T10:42:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 10 monclient(hunting): get_auth_request con 0x7fb1fc0a7180 auth_method 0 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 10 monclient(hunting): handle_auth_done global_id 5514 payload 274 2024-08-19T10:42:44.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 1 -- 172.21.15.89:0/951491428 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc0a6860 msgr2=0x7fb1fc0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:44.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 1 --2- 172.21.15.89:0/951491428 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc0a6860 0x7fb1fc0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 1 -- 172.21.15.89:0/951491428 >> v1:172.21.15.89:6789/0 conn(0x7fb1fc139620 legacy=0x7fb1fc139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:44.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:44.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 1 monclient: found mon.noname-d 2024-08-19T10:42:44.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:44.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:44.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1f9d9b640 1 -- 172.21.15.89:0/951491428 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc0a4cd0 con 0x7fb1fc0a7180 2024-08-19T10:42:44.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.531+0000 7fb1fa59c640 1 --2- 172.21.15.89:0/951491428 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc0a6860 0x7fb1fc0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:44.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f9d9b640 1 -- 172.21.15.89:0/951491428 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1fc0a7180 msgr2=0x7fb1fc0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f9d9b640 1 --2- 172.21.15.89:0/951491428 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1fc0a7180 0x7fb1fc0a7560 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40097f0 tx=0x7fb1e401c3a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=2c6913d19b849fe9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 1 -- 172.21.15.89:0/951491428 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1e40b0070 con 0x7fb1fc0a7180 2024-08-19T10:42:44.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 10 monclient: dump: 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 10 monclient: _finish_auth 0 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 10 monclient: _check_auth_tickets 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 1 -- 172.21.15.89:0/951491428 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e40aa810 con 0x7fb1fc0a7180 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 5 monclient: authenticate success, global_id 5514 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 4 set_mon_vals no callback set 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 1 -- 172.21.15.89:0/951491428 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1e40aae30 con 0x7fb1fc0a7180 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb1f959a640 10 monclient: dump: 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 10 monclient: shutdown 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 1 -- 172.21.15.89:0/951491428 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1fc0a7180 msgr2=0x7fb1fc0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 1 --2- 172.21.15.89:0/951491428 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1fc0a7180 0x7fb1fc0a7560 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40097f0 tx=0x7fb1e401c3a0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 1 -- 172.21.15.89:0/951491428 shutdown_connections 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 1 --2- 172.21.15.89:0/951491428 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1fc0a7180 0x7fb1fc0a7560 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 1 --2- 172.21.15.89:0/951491428 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc0a6860 0x7fb1fc0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.535+0000 7fb200eba640 1 -- 172.21.15.89:0/951491428 >> 172.21.15.89:0/951491428 conn(0x7fb1fc074b40 msgr2=0x7fb1fc0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 1 -- 172.21.15.89:0/951491428 shutdown_connections 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 1 -- 172.21.15.89:0/951491428 wait complete. 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient: build_initial_monmap 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient: monmap: 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 1 Processor -- start 2024-08-19T10:42:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 1 -- start start 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient: init 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient: _add_conns ranks=[3,4,2,7,8,1,5,0,6] 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1fc0a6860 0x7fb1fc13f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient(hunting): picked mon.noname-g con 0x7fb1fc0a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1fc0a7180 0x7fb1fc1d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient(hunting): picked mon.noname-c con 0x7fb1fc0a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc139620 0x7fb1fc13d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient(hunting): picked mon.noname-b con 0x7fb1fc139620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc144430 con 0x7fb1fc139620 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc1442b0 con 0x7fb1fc0a6860 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb1fc1445b0 con 0x7fb1fc0a7180 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb200eba640 10 monclient(hunting): authenticate will time out at 1214.357456s 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fad9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc139620 0x7fb1fc13d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1fc0a7180 0x7fb1fc1d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fad9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc139620 0x7fb1fc13d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40122/0 (socket says 172.21.15.89:40122) 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fad9d640 1 -- 172.21.15.89:0/470952537 learned_addr learned my addr 172.21.15.89:0/470952537 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 10 monclient(hunting): get_auth_request con 0x7fb1fc0a7180 auth_method 0 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fad9d640 10 monclient(hunting): get_auth_request con 0x7fb1fc139620 auth_method 0 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fa59c640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1fc0a6860 0x7fb1fc13f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fa59c640 10 monclient(hunting): get_auth_request con 0x7fb1fc0a6860 auth_method 0 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 10 monclient(hunting): handle_auth_done global_id 5611 payload 824 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 1 -- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1fc0a6860 msgr2=0x7fb1fc13f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1fc0a6860 0x7fb1fc13f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 1 -- 172.21.15.89:0/470952537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc139620 msgr2=0x7fb1fc13d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc139620 0x7fb1fc13d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 1 monclient: found mon.noname-c 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1f9d9b640 1 -- 172.21.15.89:0/470952537 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc0aad90 con 0x7fb1fc0a7180 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.539+0000 7fb1fad9d640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1fc139620 0x7fb1fc13d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f9d9b640 1 -- 172.21.15.89:0/470952537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1fc0a7180 msgr2=0x7fb1fc1d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f9d9b640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1fc0a7180 0x7fb1fc1d1a80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40ad000 tx=0x7fb1e4019140 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1e40b8020 con 0x7fb1fc0a7180 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 10 monclient: dump: 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e40044b0 con 0x7fb1fc0a7180 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb200eba640 5 monclient: authenticate success, global_id 5611 2024-08-19T10:42:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb200eba640 10 monclient: _renew_subs 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb200eba640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb200eba640 1 -- 172.21.15.89:0/470952537 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1fc122ca0 con 0x7fb1fc0a7180 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1e40b0040 con 0x7fb1fc0a7180 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 10 monclient: dump: 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb200eba640 10 monclient: _renew_subs 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f8d99640 4 set_mon_vals no callback set 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb200eba640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb200eba640 1 -- 172.21.15.89:0/470952537 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1fc140220 con 0x7fb1fc0a7180 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb200eba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb200eba640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc1d1fc0 0x7fb1fc02d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb200eba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb200eba640 1 -- 172.21.15.89:0/470952537 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc139440 con 0x7fb1fc1d1fc0 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f9d9b640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1fc0a7180 0x7fb1fc1d1a80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40ad000 tx=0x7fb1e4019140 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fb1e40e8a10 osd_map(150..150 src has 1..150) 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb1e40bb070 con 0x7fb1fc0a7180 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fa59c640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc1d1fc0 0x7fb1fc02d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fa59c640 10 monclient: get_auth_request con 0x7fb1fc1d1fc0 auth_method 0 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1f2ffd640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb1dc03be00 0x7fb1dc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fad9d640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb1dc03be00 0x7fb1dc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fad9d640 10 monclient: get_auth_request con 0x7fb1dc03be00 auth_method 0 2024-08-19T10:42:44.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fa59c640 10 monclient(hunting): handle_auth_done global_id 14941 payload 824 2024-08-19T10:42:44.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.543+0000 7fb1fa59c640 1 -- 172.21.15.89:0/470952537 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1fc13ed30 con 0x7fb1fc1d1fc0 2024-08-19T10:42:44.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1fa59c640 1 -- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc1d1fc0 msgr2=0x7fb1fc02d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1fa59c640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc1d1fc0 0x7fb1fc02d1d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4011600 tx=0x7fb1f4011ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1f400fe90 con 0x7fb1fc1d1fc0 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1fad9d640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb1dc03be00 0x7fb1dc03e2c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00c4d0 tx=0x7fb1ec00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1f2ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1f2ffd640 10 monclient: dump: 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.567 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb1f400fa20 con 0x7fb1fc1d1fc0 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1f2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1f2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc1d1fc0 msgr2=0x7fb1fc02d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1f2ffd640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc1d1fc0 0x7fb1fc02d1d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4011600 tx=0x7fb1f4011ac0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc1d1fc0 msgr2=0x7fb1fc02d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.547+0000 7fb1f2ffd640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc1d1fc0 0x7fb1fc02d1d0 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4011600 tx=0x7fb1f4011ac0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.551+0000 7fb200eba640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.551+0000 7fb200eba640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc02d710 0x7fb1fc02db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.551+0000 7fb200eba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.551+0000 7fb200eba640 1 -- 172.21.15.89:0/470952537 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc0a6c40 con 0x7fb1fc02d710 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc02d710 0x7fb1fc02db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 10 monclient: get_auth_request con 0x7fb1fc02d710 auth_method 0 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 10 monclient(hunting): handle_auth_done global_id 14950 payload 824 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 1 -- 172.21.15.89:0/470952537 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb1fc064590 con 0x7fb1fc02d710 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 1 -- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc02d710 msgr2=0x7fb1fc02db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1fa59c640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc02d710 0x7fb1fc02db20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4000c10 tx=0x7fb1f40046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1f4022570 con 0x7fb1fc02d710 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.555+0000 7fb1f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1f2ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1f2ffd640 10 monclient: dump: 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.571 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7fb1f4000e50 con 0x7fb1fc02d710 2024-08-19T10:42:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1f2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1f2ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc02d710 msgr2=0x7fb1fc02db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1f2ffd640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc02d710 0x7fb1fc02db20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4000c10 tx=0x7fb1f40046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc02d710 msgr2=0x7fb1fc02db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:44.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1f2ffd640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1fc02d710 0x7fb1fc02db20 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4000c10 tx=0x7fb1f40046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.572 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:44.572 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:44.572 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 142144, 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:44.573 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.574 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.575 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:44.576 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:44.577 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:44.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.579 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:44.580 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.581 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:44.583 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:44.584 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.585 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.586 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.587 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.588 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.591 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.592 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.594 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:44.595 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:44.596 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb200eba640 1 -- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb1dc03be00 msgr2=0x7fb1dc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb200eba640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb1dc03be00 0x7fb1dc03e2c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00c4d0 tx=0x7fb1ec00b040 comp rx=0 tx=0).stop 2024-08-19T10:42:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb200eba640 10 monclient: shutdown 2024-08-19T10:42:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb200eba640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb200eba640 1 -- 172.21.15.89:0/470952537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1fc0a7180 msgr2=0x7fb1fc1d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb200eba640 1 --2- 172.21.15.89:0/470952537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb1fc0a7180 0x7fb1fc1d1a80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40ad000 tx=0x7fb1e4019140 comp rx=0 tx=0).stop 2024-08-19T10:42:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1fad9d640 1 -- 172.21.15.89:0/470952537 reap_dead start 2024-08-19T10:42:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1f2ffd640 1 -- 172.21.15.89:0/470952537 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(150..150 src has 1..150) ==== 8307+0+0 (unknown 0 0 0) 0x7fb1e40e8a10 con 0x7fb1fc0a7180 2024-08-19T10:42:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb1fad9d640 1 -- 172.21.15.89:0/470952537 reap_dead start 2024-08-19T10:42:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb200eba640 1 -- 172.21.15.89:0/470952537 shutdown_connections 2024-08-19T10:42:44.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb200eba640 1 -- 172.21.15.89:0/470952537 >> 172.21.15.89:0/470952537 conn(0x7fb1fc074b40 msgr2=0x7fb1fc0a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:44.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb200eba640 1 -- 172.21.15.89:0/470952537 shutdown_connections 2024-08-19T10:42:44.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.559+0000 7fb200eba640 1 -- 172.21.15.89:0/470952537 wait complete. 2024-08-19T10:42:44.599 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:44.599 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:42:44.599 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:42:44.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient: get_monmap_and_config 2024-08-19T10:42:44.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient: build_initial_monmap 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient: monmap: 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:44.717082+0000 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:44.717082+0000 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:44.717 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:44.718 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:44.718 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:44.718 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:44.718 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:44.718 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:44.718 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 1 Processor -- start 2024-08-19T10:42:44.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 1 -- start start 2024-08-19T10:42:44.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient: init 2024-08-19T10:42:44.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:44.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient: _add_conns ranks=[4,2,0,8,1,7,5,9,10,6,3] 2024-08-19T10:42:44.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42c00a3c70 0x7f42c00a4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient(hunting): picked mon.noname-c con 0x7f42c00a3c70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:44.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42c00a4590 0x7f42c0139960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient(hunting): picked mon.noname-b con 0x7f42c00a4590 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f42c0139ea0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42c00a5640 con 0x7f42c0139ea0 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f42c006cfb0 con 0x7f42c00a4590 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f42c00a0b40 con 0x7f42c00a3c70 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42c5681640 10 monclient(hunting): authenticate will time out at 1214.534336s 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42c00a3c70 0x7f42c00a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42bf7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f42c0139ea0 0x7f42c0140c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52920/0 (socket says 172.21.15.89:52920) 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42be7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42c00a4590 0x7f42c0139960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42bf7fe640 1 -- 172.21.15.89:0/2744038390 learned_addr learned my addr 172.21.15.89:0/2744038390 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 10 monclient(hunting): get_auth_request con 0x7f42c00a3c70 auth_method 0 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42be7fc640 10 monclient(hunting): get_auth_request con 0x7f42c00a4590 auth_method 0 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42be7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42be7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42be7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42be7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 10 monclient(hunting): handle_auth_done global_id 5620 payload 274 2024-08-19T10:42:44.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 1 -- 172.21.15.89:0/2744038390 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42c00a4590 msgr2=0x7f42c0139960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:44.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 1 --2- 172.21.15.89:0/2744038390 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42c00a4590 0x7f42c0139960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 1 -- 172.21.15.89:0/2744038390 >> v1:172.21.15.89:6789/0 conn(0x7f42c0139ea0 legacy=0x7f42c0140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:44.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 10 monclient: _finish_hunting 0 2024-08-19T10:42:44.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 1 monclient: found mon.noname-c 2024-08-19T10:42:44.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:44.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:44.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42beffd640 1 -- 172.21.15.89:0/2744038390 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c00a0e20 con 0x7f42c00a3c70 2024-08-19T10:42:44.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.715+0000 7f42be7fc640 1 --2- 172.21.15.89:0/2744038390 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42c00a4590 0x7f42c0139960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:44.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.719+0000 7f42beffd640 1 -- 172.21.15.89:0/2744038390 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42c00a3c70 msgr2=0x7f42c00a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.719+0000 7f42beffd640 1 --2- 172.21.15.89:0/2744038390 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42c00a3c70 0x7f42c00a4050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f42b00097f0 tx=0x7f42b001c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=622942617ce6c19e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 1 -- 172.21.15.89:0/2744038390 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42b00b0070 con 0x7f42c00a3c70 2024-08-19T10:42:44.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:44.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 10 monclient: dump: 2024-08-19T10:42:44.727 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.727 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.727 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.727 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.727 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:44.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 1 -- 172.21.15.89:0/2744038390 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b00aa810 con 0x7f42c00a3c70 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 5 monclient: authenticate success, global_id 5620 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 1 -- 172.21.15.89:0/2744038390 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42b00aae30 con 0x7f42c00a3c70 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 4 set_mon_vals no callback set 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42bdffb640 10 monclient: dump: 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.729 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.730 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 10 monclient: shutdown 2024-08-19T10:42:44.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:44.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 1 -- 172.21.15.89:0/2744038390 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42c00a3c70 msgr2=0x7f42c00a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 1 --2- 172.21.15.89:0/2744038390 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42c00a3c70 0x7f42c00a4050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f42b00097f0 tx=0x7f42b001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 1 -- 172.21.15.89:0/2744038390 shutdown_connections 2024-08-19T10:42:44.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 1 --2- 172.21.15.89:0/2744038390 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42c00a4590 0x7f42c0139960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 1 --2- 172.21.15.89:0/2744038390 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f42c00a3c70 0x7f42c00a4050 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 1 -- 172.21.15.89:0/2744038390 >> 172.21.15.89:0/2744038390 conn(0x7f42c006de40 msgr2=0x7f42c006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:44.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 1 -- 172.21.15.89:0/2744038390 shutdown_connections 2024-08-19T10:42:44.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.723+0000 7f42c5681640 1 -- 172.21.15.89:0/2744038390 wait complete. 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient: build_initial_monmap 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient: monmap: 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:44.735 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 1 Processor -- start 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 1 -- start start 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient: init 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient: _add_conns ranks=[7,3,1,0,2,4,6,8,5] 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42c00a3c70 0x7f42c0140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient(hunting): picked mon.noname-i con 0x7f42c00a3c70 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:44.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c00a4590 0x7f42c01d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient(hunting): picked mon.noname-g con 0x7f42c00a4590 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42c0139ea0 0x7f42c01d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient(hunting): picked mon.noname-f con 0x7f42c0139ea0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42c00a4af0 con 0x7f42c0139ea0 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f42c00a4970 con 0x7f42c00a4590 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f42c006d230 con 0x7f42c00a3c70 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42c5681640 10 monclient(hunting): authenticate will time out at 1214.545122s 2024-08-19T10:42:44.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42beffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42c00a3c70 0x7f42c0140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42be7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c00a4590 0x7f42c01d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42c0139ea0 0x7f42c01d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42beffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42c00a3c70 0x7f42c0140200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:50364/0 (socket says 172.21.15.89:50364) 2024-08-19T10:42:44.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42be7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c00a4590 0x7f42c01d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:59556/0 (socket says 172.21.15.89:59556) 2024-08-19T10:42:44.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42beffd640 1 -- 172.21.15.89:0/2604931459 learned_addr learned my addr 172.21.15.89:0/2604931459 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:44.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42beffd640 10 monclient(hunting): get_auth_request con 0x7f42c00a3c70 auth_method 0 2024-08-19T10:42:44.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42beffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42beffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42be7fc640 10 monclient(hunting): get_auth_request con 0x7f42c00a4590 auth_method 0 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42be7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 10 monclient(hunting): get_auth_request con 0x7f42c0139ea0 auth_method 0 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:44.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42be7fc640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c00a4590 msgr2=0x7f42c01d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42be7fc640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c00a4590 msgr2=0x7f42c01d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42be7fc640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c00a4590 0x7f42c01d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42be7fc640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c00a4590 0x7f42c01d1db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 10 monclient(hunting): handle_auth_done global_id 14959 payload 824 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42c00a3c70 msgr2=0x7f42c0140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42c00a3c70 0x7f42c0140200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c00a4590 msgr2=0x7f42c01d1db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c00a4590 0x7f42c01d1db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 1 monclient: found mon.noname-f 2024-08-19T10:42:44.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:44.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:42:44.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42bf7fe640 1 -- 172.21.15.89:0/2604931459 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c0141720 con 0x7f42c0139ea0 2024-08-19T10:42:44.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.727+0000 7f42beffd640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42c00a3c70 0x7f42c0140200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:44.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.731+0000 7f42bf7fe640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42c0139ea0 msgr2=0x7f42c01d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:44.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.731+0000 7f42bf7fe640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42c0139ea0 0x7f42c01d22f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f42b800be10 tx=0x7f42b800bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.732+0000 7f4a3dffb640 1 -- 172.21.15.190:0/1767900062 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}]=0 v151) ==== 107+0+0 (secure 0 0 0) 0x7f4a480a48f0 con 0x7f4a4809ea30 2024-08-19T10:42:44.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.732+0000 7f4a3dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-08-19T10:42:44.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.732+0000 7f4a3dffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42b8011b90 con 0x7f42c0139ea0 2024-08-19T10:42:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:42:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 10 monclient: dump: 2024-08-19T10:42:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.745 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.746 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b8011d70 con 0x7f42c0139ea0 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42b8013870 con 0x7f42c0139ea0 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42a77fe640 10 monclient: dump: 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.747 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:44.748 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42bd7fa640 4 set_mon_vals no callback set 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42c5681640 5 monclient: authenticate success, global_id 14959 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42c5681640 10 monclient: _renew_subs 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42c5681640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42c5681640 1 -- 172.21.15.89:0/2604931459 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c0141a00 con 0x7f42c0139ea0 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42c5681640 10 monclient: _renew_subs 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42c5681640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42c5681640 1 -- 172.21.15.89:0/2604931459 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c00a6620 con 0x7f42c0139ea0 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42c5681640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42c5681640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 0x7f42c013dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42c5681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.735+0000 7f42c5681640 1 -- 172.21.15.89:0/2604931459 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f42c0069f50 con 0x7f42c013dbe0 2024-08-19T10:42:44.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42beffd640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 0x7f42c013dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42beffd640 10 monclient: get_auth_request con 0x7f42c013dbe0 auth_method 0 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42beffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42beffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f42b8013400 con 0x7f42c0139ea0 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42beffd640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 msgr2=0x7f42c013dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42beffd640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 msgr2=0x7f42c013dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42beffd640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 0x7f42c013dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42beffd640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 0x7f42c013dfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42a77fe640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42ac03be30 0x7f42ac03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 8639+0+0 (secure 0 0 0) 0x7f42b80500c0 con 0x7f42c0139ea0 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42be7fc640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42ac03be30 0x7f42ac03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42be7fc640 10 monclient: get_auth_request con 0x7f42ac03be30 auth_method 0 2024-08-19T10:42:44.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.739+0000 7f42be7fc640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42ac03be30 0x7f42ac03e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f42c013fd10 tx=0x7f42b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:44.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.848+0000 7f4a4f57b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-08-19T10:42:44.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.848+0000 7f4a4f57b640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-08-19T10:42:44.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.848+0000 7f4a4f57b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:42:44.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.848+0000 7f4a4f57b640 1 -- 172.21.15.190:0/1767900062 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"} v 0) -- 0x7f4a4813e9c0 con 0x7f4a4809ea30 2024-08-19T10:42:44.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.848+0000 7f4a3dffb640 1 -- 172.21.15.190:0/1767900062 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}]=0 weight-set for pool 'cool' already created v151) ==== 149+0+0 (secure 0 0 0) 0x7f4a3804cc30 con 0x7f4a4809ea30 2024-08-19T10:42:44.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.848+0000 7f4a3dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "cool", "mode": "flat"}] 2024-08-19T10:42:44.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.848+0000 7f4a3dffb640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'cool' already created 2024-08-19T10:42:44.854 INFO:tasks.workunit.client.0.smithi190.stderr:weight-set for pool 'cool' already created 2024-08-19T10:42:44.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 -- 172.21.15.190:0/1767900062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a2c03be60 msgr2=0x7f4a2c03e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a2c03be60 0x7f4a2c03e320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4a44007240 tx=0x7f4a44007e70 comp rx=0 tx=0).stop 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 10 monclient: shutdown 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 -- 172.21.15.190:0/1767900062 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a4809ea30 msgr2=0x7f4a48093460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a4809ea30 0x7f4a48093460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800e840 tx=0x7f4a3800ed00 comp rx=0 tx=0).stop 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 -- 172.21.15.190:0/1767900062 shutdown_connections 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a4809ee30 0x7f4a480939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a4809ea30 0x7f4a48093460 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a2c03be60 0x7f4a2c03e320 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 --2- 172.21.15.190:0/1767900062 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4a4809b510 0x7f4a48092f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 -- 172.21.15.190:0/1767900062 >> 172.21.15.190:0/1767900062 conn(0x7f4a480a4e70 msgr2=0x7f4a480a00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 -- 172.21.15.190:0/1767900062 shutdown_connections 2024-08-19T10:42:44.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.852+0000 7f4a4f57b640 1 -- 172.21.15.190:0/1767900062 wait complete. 2024-08-19T10:42:44.872 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set create cold positional 2024-08-19T10:42:44.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.943+0000 7f42beffd640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 0x7f42c013dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:44.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.943+0000 7f42beffd640 10 monclient: get_auth_request con 0x7f42c013dbe0 auth_method 0 2024-08-19T10:42:44.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.943+0000 7f42beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.943+0000 7f42beffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.943+0000 7f42beffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:42:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.943+0000 7f42beffd640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 msgr2=0x7f42c013dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:42:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.943+0000 7f42beffd640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 msgr2=0x7f42c013dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:42:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.943+0000 7f42beffd640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 0x7f42c013dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:42:44.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:44.943+0000 7f42beffd640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 0x7f42c013dfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient: get_monmap_and_config 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient: build_initial_monmap 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient: monmap: 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:44.996611+0000 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:44.996611+0000 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:44.996 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:44.997 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:44.997 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:44.997 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:44.997 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:44.997 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:44.997 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:44.997 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:44.997 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:44.997 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:44.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 1 Processor -- start 2024-08-19T10:42:44.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 1 -- start start 2024-08-19T10:42:44.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient: init 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient: _add_conns ranks=[6,0,1,10,8,2,3,5,7,9,4] 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a00a3c90 0x7f69a00a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient(hunting): picked mon.noname-d con 0x7f69a00a3c90 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f69a00a4070 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f69a013ebb0 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a00a0e30 con 0x7f69a00a4070 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a00a1110 con 0x7f69a013ebb0 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f69a00a13f0 con 0x7f69a00a3c90 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a8022640 10 monclient(hunting): authenticate will time out at 1210.566825s 2024-08-19T10:42:45.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a65bf640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f69a013ebb0 0x7f69a0140fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:42616/0 (socket says 172.21.15.190:42616) 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a65bf640 1 -- 172.21.15.190:0/3564566249 learned_addr learned my addr 172.21.15.190:0/3564566249 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 1 --2- 172.21.15.190:0/3564566249 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a00a3c90 0x7f69a00a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 10 monclient(hunting): get_auth_request con 0x7f69a00a3c90 auth_method 0 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 10 monclient(hunting): handle_auth_done global_id 5523 payload 274 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 1 -- 172.21.15.190:0/3564566249 >> v1:172.21.15.190:6789/0 conn(0x7f69a013ebb0 legacy=0x7f69a0140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 1 -- 172.21.15.190:0/3564566249 >> v1:172.21.15.89:6789/0 conn(0x7f69a00a4070 legacy=0x7f69a00a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:45.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:45.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 1 monclient: found mon.noname-d 2024-08-19T10:42:45.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:45.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:45.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:44.992+0000 7f69a5dbe640 1 -- 172.21.15.190:0/3564566249 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a006cbc0 con 0x7f69a00a3c90 2024-08-19T10:42:45.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a5dbe640 1 -- 172.21.15.190:0/3564566249 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a00a3c90 msgr2=0x7f69a00a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a5dbe640 1 --2- 172.21.15.190:0/3564566249 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a00a3c90 0x7f69a00a1810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f699c0097f0 tx=0x7f699c01c3a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=452f4a57b2b7915b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 1 -- 172.21.15.190:0/3564566249 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f699c0b0070 con 0x7f69a00a3c90 2024-08-19T10:42:45.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:45.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 10 monclient: dump: 2024-08-19T10:42:45.004 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:45.004 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.004 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.004 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.004 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.004 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:45.004 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 10 monclient: _finish_auth 0 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:45.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 1 -- 172.21.15.190:0/3564566249 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699c0aa810 con 0x7f69a00a3c90 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 1 -- 172.21.15.190:0/3564566249 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f699c0a4e60 con 0x7f69a00a3c90 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a4dbc640 10 monclient: dump: 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.006 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.007 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.007 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.007 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.007 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.007 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.007 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.007 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.007 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:45.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 5 monclient: authenticate success, global_id 5523 2024-08-19T10:42:45.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:45.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 4 set_mon_vals no callback set 2024-08-19T10:42:45.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient: shutdown 2024-08-19T10:42:45.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:45.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 -- 172.21.15.190:0/3564566249 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a00a3c90 msgr2=0x7f69a00a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 --2- 172.21.15.190:0/3564566249 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a00a3c90 0x7f69a00a1810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f699c0097f0 tx=0x7f699c01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 -- 172.21.15.190:0/3564566249 shutdown_connections 2024-08-19T10:42:45.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 --2- 172.21.15.190:0/3564566249 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a00a3c90 0x7f69a00a1810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 -- 172.21.15.190:0/3564566249 >> 172.21.15.190:0/3564566249 conn(0x7f69a006de10 msgr2=0x7f69a006e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 -- 172.21.15.190:0/3564566249 shutdown_connections 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 -- 172.21.15.190:0/3564566249 wait complete. 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient: build_initial_monmap 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient: monmap: 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:45.013 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 Processor -- start 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 -- start start 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient: init 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:45.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient: _add_conns ranks=[2,3,6,1,0,4,8,7,5] 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69a00a3c90 0x7f69a0072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient(hunting): picked mon.noname-b con 0x7f69a00a3c90 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69a00a4070 0x7f69a013cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient(hunting): picked mon.noname-g con 0x7f69a00a4070 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a013ebb0 0x7f69a0123810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient(hunting): picked mon.noname-d con 0x7f69a013ebb0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f69a006a540 con 0x7f69a00a3c90 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f69a0050c20 con 0x7f69a00a4070 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f69a0144ee0 con 0x7f69a013ebb0 2024-08-19T10:42:45.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a8022640 10 monclient(hunting): authenticate will time out at 1210.575565s 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a55bd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69a00a4070 0x7f69a013cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a55bd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69a00a4070 0x7f69a013cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:56074/0 (socket says 172.21.15.190:56074) 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a55bd640 1 -- 172.21.15.190:0/83246669 learned_addr learned my addr 172.21.15.190:0/83246669 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a55bd640 10 monclient(hunting): get_auth_request con 0x7f69a00a4070 auth_method 0 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a55bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a55bd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a55bd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a55bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a55bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a55bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a5dbe640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69a00a3c90 0x7f69a0072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a65bf640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a013ebb0 0x7f69a0123810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a5dbe640 10 monclient(hunting): get_auth_request con 0x7f69a00a3c90 auth_method 0 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a5dbe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a5dbe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a5dbe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.000+0000 7f69a55bd640 10 monclient(hunting): handle_auth_done global_id 34104 payload 824 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a55bd640 1 -- 172.21.15.190:0/83246669 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a013ebb0 msgr2=0x7f69a0123810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a55bd640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a013ebb0 0x7f69a0123810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a55bd640 1 -- 172.21.15.190:0/83246669 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69a00a3c90 msgr2=0x7f69a0072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a55bd640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69a00a3c90 0x7f69a0072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a55bd640 10 monclient: _finish_hunting 0 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a55bd640 1 monclient: found mon.noname-g 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a55bd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:45.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a55bd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a55bd640 1 -- 172.21.15.190:0/83246669 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a0126750 con 0x7f69a00a4070 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a65bf640 10 monclient: get_auth_request con 0x7f69a013ebb0 auth_method 0 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a65bf640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a013ebb0 0x7f69a0123810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a55bd640 1 -- 172.21.15.190:0/83246669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69a00a4070 msgr2=0x7f69a013cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a55bd640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69a00a4070 0x7f69a013cdb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f699000c440 tx=0x7f699000c900 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 1 -- 172.21.15.190:0/83246669 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6990016020 con 0x7f69a00a4070 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 10 monclient: dump: 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.018 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:45.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 1 -- 172.21.15.190:0/83246669 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6990012cb0 con 0x7f69a00a4070 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 1 -- 172.21.15.190:0/83246669 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6990012e60 con 0x7f69a00a4070 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 10 monclient: dump: 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.020 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f6997fff640 4 set_mon_vals no callback set 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a8022640 5 monclient: authenticate success, global_id 34104 2024-08-19T10:42:45.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a8022640 10 monclient: _renew_subs 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a8022640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a8022640 1 -- 172.21.15.190:0/83246669 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a01258e0 con 0x7f69a00a4070 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a8022640 10 monclient: _renew_subs 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a8022640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a8022640 1 -- 172.21.15.190:0/83246669 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a0126470 con 0x7f69a00a4070 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 1 -- 172.21.15.190:0/83246669 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6990019020 con 0x7f69a00a4070 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a8022640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a8022640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a8022640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a8022640 1 -- 172.21.15.190:0/83246669 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a00a1810 con 0x7f69a00a4070 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698403be30 0x7f698403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69967fc640 1 -- 172.21.15.190:0/83246669 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(151..151 src has 1..151) ==== 8639+0+0 (secure 0 0 0) 0x7f699004f720 con 0x7f69a00a4070 2024-08-19T10:42:45.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.004+0000 7f69a5dbe640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698403be30 0x7f698403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.008+0000 7f69a5dbe640 10 monclient: get_auth_request con 0x7f698403be30 auth_method 0 2024-08-19T10:42:45.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.008+0000 7f69a5dbe640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698403be30 0x7f698403e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f69a006cb30 tx=0x7f699c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.012+0000 7f69967fc640 1 -- 172.21.15.190:0/83246669 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6990020030 con 0x7f69a00a4070 2024-08-19T10:42:45.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.012+0000 7f69967fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:45.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.012+0000 7f69967fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:45.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.188+0000 7f69a8022640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-08-19T10:42:45.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.188+0000 7f69a8022640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-08-19T10:42:45.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.188+0000 7f69a8022640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:42:45.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:45.188+0000 7f69a8022640 1 -- 172.21.15.190:0/83246669 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"} v 0) -- 0x7f69a0142250 con 0x7f69a00a4070 2024-08-19T10:42:45.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.347+0000 7f42beffd640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 0x7f42c013dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.347+0000 7f42beffd640 10 monclient: get_auth_request con 0x7f42c013dbe0 auth_method 0 2024-08-19T10:42:45.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.347+0000 7f42beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.347+0000 7f42beffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.347+0000 7f42beffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:42:45.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.347+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.347+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.347+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.347+0000 7f42beffd640 10 monclient(hunting): handle_auth_done global_id 34113 payload 824 2024-08-19T10:42:45.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.347+0000 7f42beffd640 1 -- 172.21.15.89:0/2604931459 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42c00a6320 con 0x7f42c013dbe0 2024-08-19T10:42:45.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42beffd640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 msgr2=0x7f42c013dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42beffd640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 0x7f42c013dfc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009710 tx=0x7f42b0019140 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42b00b0040 con 0x7f42c013dbe0 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42a77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42a77fe640 10 monclient: dump: 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.354 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.355 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f42b00ba070 con 0x7f42c013dbe0 2024-08-19T10:42:45.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42a77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:45.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42a77fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:45.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 msgr2=0x7f42c013dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42a77fe640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 0x7f42c013dfc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009710 tx=0x7f42b0019140 comp rx=0 tx=0).stop 2024-08-19T10:42:45.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 msgr2=0x7f42c013dfc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:45.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.351+0000 7f42a77fe640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013dbe0 0x7f42c013dfc0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009710 tx=0x7f42b0019140 comp rx=0 tx=0).stop 2024-08-19T10:42:45.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42c5681640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:45.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42c5681640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013e500 0x7f42c009d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42c5681640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42c5681640 1 -- 172.21.15.89:0/2604931459 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f42c0141f60 con 0x7f42c013e500 2024-08-19T10:42:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42beffd640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013e500 0x7f42c009d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42beffd640 10 monclient: get_auth_request con 0x7f42c013e500 auth_method 0 2024-08-19T10:42:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42beffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42beffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42beffd640 10 monclient(hunting): handle_auth_done global_id 34122 payload 824 2024-08-19T10:42:45.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.359+0000 7f42beffd640 1 -- 172.21.15.89:0/2604931459 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f42c00a4050 con 0x7f42c013e500 2024-08-19T10:42:45.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42beffd640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013e500 msgr2=0x7f42c009d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42beffd640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013e500 0x7f42c009d9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f42b001c3e0 tx=0x7f42b00a8000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42b00b0040 con 0x7f42c013e500 2024-08-19T10:42:45.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42a77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:42:45.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42a77fe640 10 monclient: dump: 2024-08-19T10:42:45.367 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.367 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.367 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.367 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.368 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.368 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.368 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.368 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.368 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.368 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.368 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.368 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.368 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.369 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.369 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.369 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7f42b00c8030 con 0x7f42c013e500 2024-08-19T10:42:45.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42a77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:45.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42a77fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:45.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013e500 msgr2=0x7f42c009d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42a77fe640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013e500 0x7f42c009d9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f42b001c3e0 tx=0x7f42b00a8000 comp rx=0 tx=0).stop 2024-08-19T10:42:45.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42a77fe640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013e500 msgr2=0x7f42c009d9f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:45.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42a77fe640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42c013e500 0x7f42c009d9f0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f42b001c3e0 tx=0x7f42b00a8000 comp rx=0 tx=0).stop 2024-08-19T10:42:45.371 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:45.371 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:45.371 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:45.371 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 10403, 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:45.372 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:45.373 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:45.374 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:45.375 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:45.376 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:45.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:45.378 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:45.379 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.380 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.383 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.384 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.385 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.386 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.387 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:45.389 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:45.390 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.391 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.393 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:45.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.395 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:45.395 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:45.395 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:42:45.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.395 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.395 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:45.395 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42c5681640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42ac03be30 msgr2=0x7f42ac03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42c5681640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42ac03be30 0x7f42ac03e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f42c013fd10 tx=0x7f42b4008040 comp rx=0 tx=0).stop 2024-08-19T10:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42c5681640 10 monclient: shutdown 2024-08-19T10:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42c5681640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42c5681640 1 -- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42c0139ea0 msgr2=0x7f42c01d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42c5681640 1 --2- 172.21.15.89:0/2604931459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42c0139ea0 0x7f42c01d22f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f42b800be10 tx=0x7f42b800bf00 comp rx=0 tx=0).stop 2024-08-19T10:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42bf7fe640 1 -- 172.21.15.89:0/2604931459 reap_dead start 2024-08-19T10:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.363+0000 7f42bf7fe640 1 -- 172.21.15.89:0/2604931459 reap_dead start 2024-08-19T10:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.367+0000 7f42c5681640 1 -- 172.21.15.89:0/2604931459 shutdown_connections 2024-08-19T10:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.367+0000 7f42c5681640 1 -- 172.21.15.89:0/2604931459 >> 172.21.15.89:0/2604931459 conn(0x7f42c006de40 msgr2=0x7f42c0072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.367+0000 7f42c5681640 1 -- 172.21.15.89:0/2604931459 shutdown_connections 2024-08-19T10:42:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.367+0000 7f42c5681640 1 -- 172.21.15.89:0/2604931459 wait complete. 2024-08-19T10:42:45.398 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:45.398 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:42:45.398 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:42:45.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient: get_monmap_and_config 2024-08-19T10:42:45.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient: build_initial_monmap 2024-08-19T10:42:45.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient: monmap: 2024-08-19T10:42:45.521 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:45.521 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.522 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:45.521543+0000 2024-08-19T10:42:45.522 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:45.521543+0000 2024-08-19T10:42:45.522 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:45.522 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.522 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:45.522 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:45.522 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:45.522 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:45.523 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:45.523 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:45.523 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:45.523 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:45.523 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:45.523 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:45.523 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:45.523 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 1 Processor -- start 2024-08-19T10:42:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 1 -- start start 2024-08-19T10:42:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient: init 2024-08-19T10:42:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient: _add_conns ranks=[6,1,2,0,5,3,10,7,8,9,4] 2024-08-19T10:42:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a5270 0x7f1d040a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient(hunting): picked mon.noname-d con 0x7f1d040a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1d040a6d30 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d040a7110 0x7f1d04139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient(hunting): picked mon.noname-b con 0x7f1d040a7110 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d040a8710 con 0x7f1d040a6d30 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1d040a5a40 con 0x7f1d040a7110 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1d040a81d0 con 0x7f1d040a5270 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d091cb640 10 monclient(hunting): authenticate will time out at 1215.338110s 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a5270 0x7f1d040a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a5270 0x7f1d040a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54490/0 (socket says 172.21.15.89:54490) 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d0359e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d040a7110 0x7f1d04139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 1 -- 172.21.15.89:0/2161142768 learned_addr learned my addr 172.21.15.89:0/2161142768 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 10 monclient(hunting): get_auth_request con 0x7f1d040a5270 auth_method 0 2024-08-19T10:42:45.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d0359e640 10 monclient(hunting): get_auth_request con 0x7f1d040a7110 auth_method 0 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d0359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d0359e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d0359e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d0359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d0359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d0359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_done global_id 5532 payload 274 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 1 -- 172.21.15.89:0/2161142768 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d040a7110 msgr2=0x7f1d04139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 1 --2- 172.21.15.89:0/2161142768 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d040a7110 0x7f1d04139cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 1 -- 172.21.15.89:0/2161142768 >> v1:172.21.15.190:6789/0 conn(0x7f1d040a6d30 legacy=0x7f1d041395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 1 monclient: found mon.noname-d 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.519+0000 7f1d02d9d640 1 -- 172.21.15.89:0/2161142768 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d040a4790 con 0x7f1d040a5270 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d02d9d640 1 -- 172.21.15.89:0/2161142768 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a5270 msgr2=0x7f1d040a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d02d9d640 1 --2- 172.21.15.89:0/2161142768 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a5270 0x7f1d040a67f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40098a0 tx=0x7f1cf401c230 comp rx=0 tx=0).ready entity=mon.6 client_cookie=abe3dcae49cd91cd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 1 -- 172.21.15.89:0/2161142768 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cf40b0070 con 0x7f1d040a5270 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:45.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 10 monclient: dump: 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.532 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 10 monclient: _finish_auth 0 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 1 -- 172.21.15.89:0/2161142768 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf40aa810 con 0x7f1d040a5270 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 5 monclient: authenticate success, global_id 5532 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 1 -- 172.21.15.89:0/2161142768 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cf40a9300 con 0x7f1d040a5270 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 4 set_mon_vals no callback set 2024-08-19T10:42:45.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d01d9b640 10 monclient: dump: 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.534 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 10 monclient: shutdown 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 1 -- 172.21.15.89:0/2161142768 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a5270 msgr2=0x7f1d040a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 1 --2- 172.21.15.89:0/2161142768 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a5270 0x7f1d040a67f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40098a0 tx=0x7f1cf401c230 comp rx=0 tx=0).stop 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 1 -- 172.21.15.89:0/2161142768 shutdown_connections 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 1 --2- 172.21.15.89:0/2161142768 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1d040a7110 0x7f1d04139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 1 --2- 172.21.15.89:0/2161142768 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a5270 0x7f1d040a67f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 1 -- 172.21.15.89:0/2161142768 >> 172.21.15.89:0/2161142768 conn(0x7f1d04069a80 msgr2=0x7f1d040abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 1 -- 172.21.15.89:0/2161142768 shutdown_connections 2024-08-19T10:42:45.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.523+0000 7f1d091cb640 1 -- 172.21.15.89:0/2161142768 wait complete. 2024-08-19T10:42:45.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient: build_initial_monmap 2024-08-19T10:42:45.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient: monmap: 2024-08-19T10:42:45.541 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:45.541 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:45.541 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:45.541 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:45.542 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:45.542 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.542 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:45.542 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:45.542 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:45.542 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:45.542 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:45.542 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:45.542 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:45.542 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:45.543 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:45.543 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 1 Processor -- start 2024-08-19T10:42:45.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 1 -- start start 2024-08-19T10:42:45.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient: init 2024-08-19T10:42:45.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:45.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient: _add_conns ranks=[5,6,1,7,3,8,2,4,0] 2024-08-19T10:42:45.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d040a5270 0x7f1d04138bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient(hunting): picked mon.noname-h con 0x7f1d040a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:45.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a6d30 0x7f1d04139100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient(hunting): picked mon.noname-d con 0x7f1d040a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:45.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d040a7110 0x7f1d0413e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient(hunting): picked mon.noname-f con 0x7f1d040a7110 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:45.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d040a57d0 con 0x7f1d040a7110 2024-08-19T10:42:45.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1d040a5650 con 0x7f1d040a5270 2024-08-19T10:42:45.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1d0413a200 con 0x7f1d040a6d30 2024-08-19T10:42:45.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:45.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d091cb640 10 monclient(hunting): authenticate will time out at 1215.345929s 2024-08-19T10:42:45.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a6d30 0x7f1d04139100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a6d30 0x7f1d04139100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54502/0 (socket says 172.21.15.89:54502) 2024-08-19T10:42:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 1 -- 172.21.15.89:0/2351739571 learned_addr learned my addr 172.21.15.89:0/2351739571 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 10 monclient(hunting): get_auth_request con 0x7f1d040a6d30 auth_method 0 2024-08-19T10:42:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 10 monclient(hunting): handle_auth_done global_id 5541 payload 824 2024-08-19T10:42:45.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 1 -- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d040a5270 msgr2=0x7f1d04138bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d040a5270 0x7f1d04138bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 1 -- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d040a7110 msgr2=0x7f1d0413e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d040a7110 0x7f1d0413e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 10 monclient: _finish_hunting 0 2024-08-19T10:42:45.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 1 monclient: found mon.noname-d 2024-08-19T10:42:45.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:45.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:45.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.527+0000 7f1d0259c640 1 -- 172.21.15.89:0/2351739571 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d04123a00 con 0x7f1d040a6d30 2024-08-19T10:42:45.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d0259c640 1 -- 172.21.15.89:0/2351739571 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a6d30 msgr2=0x7f1d04139100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d0259c640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a6d30 0x7f1d04139100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800c450 tx=0x7f1cf800c910 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cf8011c10 con 0x7f1d040a6d30 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 10 monclient: dump: 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.547 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.548 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.548 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.548 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.548 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.548 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.548 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.548 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.548 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:45.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:45.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:45.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf8017070 con 0x7f1d040a6d30 2024-08-19T10:42:45.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d091cb640 5 monclient: authenticate success, global_id 5541 2024-08-19T10:42:45.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d091cb640 10 monclient: _renew_subs 2024-08-19T10:42:45.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d091cb640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:45.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d091cb640 1 -- 172.21.15.89:0/2351739571 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0413fcc0 con 0x7f1d040a6d30 2024-08-19T10:42:45.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:45.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cf8011df0 con 0x7f1d040a6d30 2024-08-19T10:42:45.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:45.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 10 monclient: dump: 2024-08-19T10:42:45.549 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.550 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.551 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.551 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.551 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.551 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d091cb640 10 monclient: _renew_subs 2024-08-19T10:42:45.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d091cb640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:45.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d091cb640 1 -- 172.21.15.89:0/2351739571 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d040a1ff0 con 0x7f1d040a6d30 2024-08-19T10:42:45.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d0159a640 4 set_mon_vals no callback set 2024-08-19T10:42:45.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d091cb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:45.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d091cb640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413e870 0x7f1d0413ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d091cb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d091cb640 1 -- 172.21.15.89:0/2351739571 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1d04122bf0 con 0x7f1d0413e870 2024-08-19T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1cf80117a0 con 0x7f1d040a6d30 2024-08-19T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1cf003be30 0x7f1cf003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d02d9d640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413e870 0x7f1d0413ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(151..151 src has 1..151) ==== 8639+0+0 (secure 0 0 0) 0x7f1cf804f830 con 0x7f1d040a6d30 2024-08-19T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d02d9d640 10 monclient: get_auth_request con 0x7f1d0413e870 auth_method 0 2024-08-19T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d02d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d02d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d02d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d0359e640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1cf003be30 0x7f1cf003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d0359e640 10 monclient: get_auth_request con 0x7f1cf003be30 auth_method 0 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_done global_id 5684 payload 824 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.531+0000 7f1d02d9d640 1 -- 172.21.15.89:0/2351739571 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d040a67f0 con 0x7f1d0413e870 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1d02d9d640 1 -- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413e870 msgr2=0x7f1d0413ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1d02d9d640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413e870 0x7f1d0413ec50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40ad000 tx=0x7f1cf401c4a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1d0359e640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1cf003be30 0x7f1cf003e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00e190 tx=0x7f1cfc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cf40b9070 con 0x7f1d0413e870 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1ceb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1ceb7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:45.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1ceb7fe640 10 monclient: dump: 2024-08-19T10:42:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.554 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.555 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.558 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.558 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.558 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.558 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1cf40093d0 con 0x7f1d0413e870 2024-08-19T10:42:45.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1ceb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:45.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1ceb7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:45.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413e870 msgr2=0x7f1d0413ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1ceb7fe640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413e870 0x7f1d0413ec50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40ad000 tx=0x7f1cf401c4a0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413e870 msgr2=0x7f1d0413ec50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:45.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.535+0000 7f1ceb7fe640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413e870 0x7f1d0413ec50 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40ad000 tx=0x7f1cf401c4a0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d091cb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d091cb640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413f190 0x7f1d0413a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d091cb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d091cb640 1 -- 172.21.15.89:0/2351739571 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1d04064590 con 0x7f1d0413f190 2024-08-19T10:42:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413f190 0x7f1d0413a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 10 monclient: get_auth_request con 0x7f1d0413f190 auth_method 0 2024-08-19T10:42:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 10 monclient(hunting): handle_auth_done global_id 5693 payload 824 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 1 -- 172.21.15.89:0/2351739571 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1d0406c9e0 con 0x7f1d0413f190 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 1 -- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413f190 msgr2=0x7f1d0413a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.543+0000 7f1d02d9d640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413f190 0x7f1d0413a7d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4000c10 tx=0x7f1cf40a8000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1cf40b9070 con 0x7f1d0413f190 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1ceb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1ceb7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1ceb7fe640 10 monclient: dump: 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.562 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f1cf40ba040 con 0x7f1d0413f190 2024-08-19T10:42:45.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1ceb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:45.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1ceb7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:45.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413f190 msgr2=0x7f1d0413a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1ceb7fe640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413f190 0x7f1d0413a7d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4000c10 tx=0x7f1cf40a8000 comp rx=0 tx=0).stop 2024-08-19T10:42:45.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1ceb7fe640 1 -- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413f190 msgr2=0x7f1d0413a7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:45.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1ceb7fe640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1d0413f190 0x7f1d0413a7d0 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4000c10 tx=0x7f1cf40a8000 comp rx=0 tx=0).stop 2024-08-19T10:42:45.564 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:45.564 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:45.564 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:45.564 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:45.564 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 295314, 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:42:45.565 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:45.566 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:45.567 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:45.568 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:45.569 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:45.570 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:45.570 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:45.570 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:45.570 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:45.570 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:45.570 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:45.570 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.570 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:45.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.570 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:45.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.571 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:45.571 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:45.571 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.571 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.571 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.571 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:45.571 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.571 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.573 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:45.573 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:45.573 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:45.573 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.574 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:45.574 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.574 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.574 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.574 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:45.574 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:45.574 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.574 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.574 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.575 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:45.575 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:45.575 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:45.575 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.576 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.577 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:45.577 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:45.577 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.577 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.577 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.577 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:45.577 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.578 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.578 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:45.578 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.578 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.578 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.578 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:45.578 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:45.578 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.578 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.579 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.579 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:45.579 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:45.579 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.579 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.579 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.579 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:45.579 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:45.580 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.580 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.580 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:45.580 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:45.580 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.581 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.581 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.581 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:45.581 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:45.581 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:45.582 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.582 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:45.582 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.582 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.582 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:45.582 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:45.584 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:45.584 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.584 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.584 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:45.584 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:45.585 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.586 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.586 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:45.586 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.586 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.586 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:45.586 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:45.586 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.587 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.587 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.587 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:45.587 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:45.587 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.587 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.587 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.587 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.588 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.589 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:45.590 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:45.590 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:42:45.590 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.590 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.590 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:45.590 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:45.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d091cb640 1 -- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1cf003be30 msgr2=0x7f1cf003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d091cb640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1cf003be30 0x7f1cf003e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00e190 tx=0x7f1cfc00b040 comp rx=0 tx=0).stop 2024-08-19T10:42:45.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d091cb640 10 monclient: shutdown 2024-08-19T10:42:45.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d091cb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:45.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d091cb640 1 -- 172.21.15.89:0/2351739571 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a6d30 msgr2=0x7f1d04139100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d091cb640 1 --2- 172.21.15.89:0/2351739571 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1d040a6d30 0x7f1d04139100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800c450 tx=0x7f1cf800c910 comp rx=0 tx=0).stop 2024-08-19T10:42:45.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d0359e640 1 -- 172.21.15.89:0/2351739571 reap_dead start 2024-08-19T10:42:45.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d0359e640 1 -- 172.21.15.89:0/2351739571 reap_dead start 2024-08-19T10:42:45.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d091cb640 1 -- 172.21.15.89:0/2351739571 shutdown_connections 2024-08-19T10:42:45.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d091cb640 1 -- 172.21.15.89:0/2351739571 >> 172.21.15.89:0/2351739571 conn(0x7f1d04069a80 msgr2=0x7f1d0409f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:45.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d091cb640 1 -- 172.21.15.89:0/2351739571 shutdown_connections 2024-08-19T10:42:45.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.547+0000 7f1d091cb640 1 -- 172.21.15.89:0/2351739571 wait complete. 2024-08-19T10:42:45.593 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:45.593 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:42:45.593 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:42:45.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.715+0000 7fbd915a9640 10 monclient: get_monmap_and_config 2024-08-19T10:42:45.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.715+0000 7fbd915a9640 10 monclient: build_initial_monmap 2024-08-19T10:42:45.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.715+0000 7fbd915a9640 10 monclient: monmap: 2024-08-19T10:42:45.720 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:45.720 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.720 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:45.720338+0000 2024-08-19T10:42:45.720 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:45.720338+0000 2024-08-19T10:42:45.720 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:45.720 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.720 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:45.721 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:45.721 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:45.721 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:45.721 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:45.721 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:45.721 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:45.721 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:45.721 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:45.721 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:45.721 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:45.721 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.715+0000 7fbd915a9640 1 Processor -- start 2024-08-19T10:42:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 1 -- start start 2024-08-19T10:42:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 10 monclient: init 2024-08-19T10:42:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 10 monclient: _add_conns ranks=[9,3,7,5,6,4,2,10,0,1,8] 2024-08-19T10:42:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbd8c0a56b0 0x7fbd8c0a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 10 monclient(hunting): picked mon.noname-a con 0x7fbd8c0a56b0 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd8c0a1660 0x7fbd8c0a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 10 monclient(hunting): picked mon.noname-g con 0x7fbd8c0a1660 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 0x7fbd8c137460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 10 monclient(hunting): picked mon.noname-i con 0x7fbd8c072250 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:45.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fbd8c0a6510 con 0x7fbd8c0a56b0 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8c074010 con 0x7fbd8c0a1660 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fbd8c072010 con 0x7fbd8c072250 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd915a9640 10 monclient(hunting): authenticate will time out at 1215.536583s 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbd8c0a56b0 0x7fbd8c0a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbd8c0a56b0 0x7fbd8c0a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37600/0 (socket says 172.21.15.89:37600) 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 1 -- 172.21.15.89:0/739709770 learned_addr learned my addr 172.21.15.89:0/739709770 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 10 monclient(hunting): get_auth_request con 0x7fbd8c0a56b0 auth_method 0 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 10 monclient(hunting): handle_auth_done global_id 5679 payload 274 2024-08-19T10:42:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 1 -- 172.21.15.89:0/739709770 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 msgr2=0x7fbd8c137460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8b7fe640 1 --2- 172.21.15.89:0/739709770 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 0x7fbd8c137460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 1 --2- 172.21.15.89:0/739709770 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 0x7fbd8c137460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8a7fc640 1 --2- 172.21.15.89:0/739709770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd8c0a1660 0x7fbd8c0a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 1 -- 172.21.15.89:0/739709770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd8c0a1660 msgr2=0x7fbd8c0a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 1 --2- 172.21.15.89:0/739709770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd8c0a1660 0x7fbd8c0a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 10 monclient: _finish_hunting 0 2024-08-19T10:42:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 1 monclient: found mon.noname-a 2024-08-19T10:42:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8affd640 1 -- 172.21.15.89:0/739709770 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c073d30 con 0x7fbd8c0a56b0 2024-08-19T10:42:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8a7fc640 10 monclient: get_auth_request con 0x7fbd8c0a1660 auth_method 0 2024-08-19T10:42:45.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.719+0000 7fbd8a7fc640 1 --2- 172.21.15.89:0/739709770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd8c0a1660 0x7fbd8c0a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:45.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8affd640 1 -- 172.21.15.89:0/739709770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c0a56b0 msgr2=0x7fbd8c0a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8affd640 1 --2- 172.21.15.89:0/739709770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c0a56b0 0x7fbd8c0a5a90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009870 tx=0x7fbd7c01c160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247f8096cba8d7bd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 1 -- 172.21.15.89:0/739709770 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbd7c0b0070 con 0x7fbd8c0a56b0 2024-08-19T10:42:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 10 monclient: dump: 2024-08-19T10:42:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.734 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:45.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 1 -- 172.21.15.89:0/739709770 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7c0aa810 con 0x7fbd8c0a56b0 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 1 -- 172.21.15.89:0/739709770 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbd7c0a93e0 con 0x7fbd8c0a56b0 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 5 monclient: authenticate success, global_id 5679 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 4 set_mon_vals no callback set 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd89ffb640 10 monclient: dump: 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.736 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient: shutdown 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 -- 172.21.15.89:0/739709770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c0a56b0 msgr2=0x7fbd8c0a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 --2- 172.21.15.89:0/739709770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c0a56b0 0x7fbd8c0a5a90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009870 tx=0x7fbd7c01c160 comp rx=0 tx=0).stop 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 -- 172.21.15.89:0/739709770 shutdown_connections 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 --2- 172.21.15.89:0/739709770 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 0x7fbd8c137460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 --2- 172.21.15.89:0/739709770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd8c0a1660 0x7fbd8c0a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 --2- 172.21.15.89:0/739709770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c0a56b0 0x7fbd8c0a5a90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 -- 172.21.15.89:0/739709770 >> 172.21.15.89:0/739709770 conn(0x7fbd8c0aa2a0 msgr2=0x7fbd8c0a67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 -- 172.21.15.89:0/739709770 shutdown_connections 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 -- 172.21.15.89:0/739709770 wait complete. 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient: build_initial_monmap 2024-08-19T10:42:45.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient: monmap: 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:45.739 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 Processor -- start 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 -- start start 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient: init 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient: _add_conns ranks=[5,0,1,3,7,4,8,2,6] 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbd8c0a56b0 0x7fbd8c136140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient(hunting): picked mon.noname-h con 0x7fbd8c0a56b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c136680 0x7fbd8c06da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient(hunting): picked mon.noname-a con 0x7fbd8c136680 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd8c06df80 0x7fbd8c06e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient(hunting): picked mon.noname-f con 0x7fbd8c06df80 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:45.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c141260 con 0x7fbd8c136680 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c1410e0 con 0x7fbd8c06df80 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fbd8c1413e0 con 0x7fbd8c0a56b0 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd915a9640 10 monclient(hunting): authenticate will time out at 1215.543614s 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c136680 0x7fbd8c06da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8a7fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c136680 0x7fbd8c06da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37612/0 (socket says 172.21.15.89:37612) 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8a7fc640 1 -- 172.21.15.89:0/3701654088 learned_addr learned my addr 172.21.15.89:0/3701654088 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8a7fc640 10 monclient(hunting): get_auth_request con 0x7fbd8c136680 auth_method 0 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.723+0000 7fbd8a7fc640 10 monclient(hunting): handle_auth_done global_id 5688 payload 824 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8a7fc640 1 -- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbd8c0a56b0 msgr2=0x7fbd8c136140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8a7fc640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbd8c0a56b0 0x7fbd8c136140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8a7fc640 1 -- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd8c06df80 msgr2=0x7fbd8c06e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8b7fe640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd8c06df80 0x7fbd8c06e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8a7fc640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd8c06df80 0x7fbd8c06e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8a7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8a7fc640 1 monclient: found mon.noname-a 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:45.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8a7fc640 1 -- 172.21.15.89:0/3701654088 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c06f400 con 0x7fbd8c136680 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8a7fc640 1 -- 172.21.15.89:0/3701654088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c136680 msgr2=0x7fbd8c06da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8a7fc640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c136680 0x7fbd8c06da40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c7f0 tx=0x7fbd8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbd80015070 con 0x7fbd8c136680 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 10 monclient: dump: 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.743 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:45.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd80007e50 con 0x7fbd8c136680 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbd80011860 con 0x7fbd8c136680 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd915a9640 5 monclient: authenticate success, global_id 5688 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd915a9640 10 monclient: _renew_subs 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd915a9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd915a9640 1 -- 172.21.15.89:0/3701654088 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8c13cd00 con 0x7fbd8c136680 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd897fa640 4 set_mon_vals no callback set 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd915a9640 10 monclient: _renew_subs 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd915a9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd915a9640 1 -- 172.21.15.89:0/3701654088 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8c137180 con 0x7fbd8c136680 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd737fe640 10 monclient: dump: 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.745 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.746 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd915a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd915a9640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 0x7fbd8c09e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd915a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd915a9640 1 -- 172.21.15.89:0/3701654088 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fbd8c0a2cb0 con 0x7fbd8c072250 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.727+0000 7fbd8affd640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 0x7fbd8c09e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8affd640 10 monclient: get_auth_request con 0x7fbd8c072250 auth_method 0 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbd80018070 con 0x7fbd8c136680 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd737fe640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbd7803be00 0x7fbd7803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 8639+0+0 (secure 0 0 0) 0x7fbd8004f500 con 0x7fbd8c136680 2024-08-19T10:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8b7fe640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbd7803be00 0x7fbd7803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8b7fe640 10 monclient: get_auth_request con 0x7fbd7803be00 auth_method 0 2024-08-19T10:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8affd640 10 monclient(hunting): handle_auth_done global_id 14857 payload 824 2024-08-19T10:42:45.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8affd640 1 -- 172.21.15.89:0/3701654088 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd8c137460 con 0x7fbd8c072250 2024-08-19T10:42:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8affd640 1 -- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 msgr2=0x7fbd8c09e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8affd640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 0x7fbd8c09e9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0ad000 tx=0x7fbd7c003850 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd8b7fe640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbd7803be00 0x7fbd7803e2c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c06f160 tx=0x7fbd840083e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbd7c0b0040 con 0x7fbd8c072250 2024-08-19T10:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd737fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.731+0000 7fbd737fe640 10 monclient: dump: 2024-08-19T10:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.752 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.735+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbd7c0b9070 con 0x7fbd8c072250 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.735+0000 7fbd737fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.735+0000 7fbd737fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:45.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.735+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 msgr2=0x7fbd8c09e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.735+0000 7fbd737fe640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 0x7fbd8c09e9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0ad000 tx=0x7fbd7c003850 comp rx=0 tx=0).stop 2024-08-19T10:42:45.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.735+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 msgr2=0x7fbd8c09e9f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:45.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.735+0000 7fbd737fe640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c072250 0x7fbd8c09e9f0 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0ad000 tx=0x7fbd7c003850 comp rx=0 tx=0).stop 2024-08-19T10:42:45.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd915a9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:45.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd915a9640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c0a15f0 0x7fbd8c09ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd915a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd915a9640 1 -- 172.21.15.89:0/3701654088 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fbd8c064590 con 0x7fbd8c0a15f0 2024-08-19T10:42:45.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd8affd640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c0a15f0 0x7fbd8c09ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd8affd640 10 monclient: get_auth_request con 0x7fbd8c0a15f0 auth_method 0 2024-08-19T10:42:45.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd8affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd8affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd8affd640 10 monclient(hunting): handle_auth_done global_id 14866 payload 824 2024-08-19T10:42:45.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.739+0000 7fbd8affd640 1 -- 172.21.15.89:0/3701654088 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbd8c0a9970 con 0x7fbd8c0a15f0 2024-08-19T10:42:45.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd8affd640 1 -- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c0a15f0 msgr2=0x7fbd8c09ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd8affd640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c0a15f0 0x7fbd8c09ef30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0ae040 tx=0x7fbd7c0a8000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbd7c0c2070 con 0x7fbd8c0a15f0 2024-08-19T10:42:45.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd737fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd737fe640 10 monclient: dump: 2024-08-19T10:42:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.756 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.757 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7fbd7c0be7c0 con 0x7fbd8c0a15f0 2024-08-19T10:42:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd737fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:45.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd737fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:45.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c0a15f0 msgr2=0x7fbd8c09ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd737fe640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c0a15f0 0x7fbd8c09ef30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0ae040 tx=0x7fbd7c0a8000 comp rx=0 tx=0).stop 2024-08-19T10:42:45.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd737fe640 1 -- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c0a15f0 msgr2=0x7fbd8c09ef30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:45.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd737fe640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbd8c0a15f0 0x7fbd8c09ef30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.758 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:45.758 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:45.758 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:45.759 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:45.759 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:45.759 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 143332, 2024-08-19T10:42:45.759 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:45.759 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:45.759 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:45.759 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:45.759 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:45.759 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:45.759 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:45.762 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:45.762 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:45.762 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:45.762 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.762 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:42:45.763 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:45.763 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:45.763 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:45.763 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:45.763 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:45.763 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:45.763 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:45.763 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:45.763 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:45.763 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:45.763 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:45.764 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:45.765 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:45.766 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:45.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.768 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.769 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:45.770 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.771 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.772 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:45.773 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.775 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:45.776 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.778 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.782 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:45.784 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.785 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:45.786 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:45.786 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:45.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd915a9640 1 -- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbd7803be00 msgr2=0x7fbd7803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd915a9640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbd7803be00 0x7fbd7803e2c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c06f160 tx=0x7fbd840083e0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd915a9640 10 monclient: shutdown 2024-08-19T10:42:45.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd915a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:45.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd915a9640 1 -- 172.21.15.89:0/3701654088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c136680 msgr2=0x7fbd8c06da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd915a9640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c136680 0x7fbd8c06da40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c7f0 tx=0x7fbd8000ccb0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd8b7fe640 1 -- 172.21.15.89:0/3701654088 reap_dead start 2024-08-19T10:42:45.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd915a9640 1 -- 172.21.15.89:0/3701654088 shutdown_connections 2024-08-19T10:42:45.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd915a9640 1 --2- 172.21.15.89:0/3701654088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8c136680 0x7fbd8c06da40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd915a9640 1 -- 172.21.15.89:0/3701654088 >> 172.21.15.89:0/3701654088 conn(0x7fbd8c0aa2a0 msgr2=0x7fbd8c13de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:45.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.743+0000 7fbd915a9640 1 -- 172.21.15.89:0/3701654088 shutdown_connections 2024-08-19T10:42:45.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.747+0000 7fbd915a9640 1 -- 172.21.15.89:0/3701654088 wait complete. 2024-08-19T10:42:45.789 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:45.789 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-08-19T10:42:45.789 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-08-19T10:42:45.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.899+0000 7f2007ec8640 10 monclient: get_monmap_and_config 2024-08-19T10:42:45.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.899+0000 7f2007ec8640 10 monclient: build_initial_monmap 2024-08-19T10:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.899+0000 7f2007ec8640 10 monclient: monmap: 2024-08-19T10:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:45.904001+0000 2024-08-19T10:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:45.904001+0000 2024-08-19T10:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:45.904 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:45.905 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.899+0000 7f2007ec8640 1 Processor -- start 2024-08-19T10:42:45.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 1 -- start start 2024-08-19T10:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 10 monclient: init 2024-08-19T10:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 10 monclient: _add_conns ranks=[9,3,8,5,2,1,4,10,7,0,6] 2024-08-19T10:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20000a5270 0x7f20000a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 10 monclient(hunting): picked mon.noname-a con 0x7f20000a5270 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20000a6d30 0x7f20000a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 10 monclient(hunting): picked mon.noname-g con 0x7f20000a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20001395b0 0x7f2000139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 10 monclient(hunting): picked mon.noname-e con 0x7f20001395b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:45.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f20000a7650 con 0x7f20000a5270 2024-08-19T10:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f20000a8710 con 0x7f20000a6d30 2024-08-19T10:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f20000a5a40 con 0x7f20001395b0 2024-08-19T10:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2007ec8640 10 monclient(hunting): authenticate will time out at 1215.720792s 2024-08-19T10:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20001395b0 0x7f2000139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005c64640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20000a5270 0x7f20000a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20001395b0 0x7f2000139a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40414/0 (socket says 172.21.15.89:40414) 2024-08-19T10:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 1 -- 172.21.15.89:0/1572851609 learned_addr learned my addr 172.21.15.89:0/1572851609 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 10 monclient(hunting): get_auth_request con 0x7f20001395b0 auth_method 0 2024-08-19T10:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005c64640 10 monclient(hunting): get_auth_request con 0x7f20000a5270 auth_method 0 2024-08-19T10:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005c64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005c64640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005c64640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005c64640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005c64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005c64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 10 monclient(hunting): handle_auth_done global_id 5453 payload 274 2024-08-19T10:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 1 -- 172.21.15.89:0/1572851609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20000a6d30 msgr2=0x7f20000a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005463640 1 --2- 172.21.15.89:0/1572851609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20000a6d30 0x7f20000a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 1 --2- 172.21.15.89:0/1572851609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20000a6d30 0x7f20000a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 1 -- 172.21.15.89:0/1572851609 >> v2:172.21.15.89:3300/0 conn(0x7f20000a5270 msgr2=0x7f20000a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:45.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 1 --2- 172.21.15.89:0/1572851609 >> v2:172.21.15.89:3300/0 conn(0x7f20000a5270 0x7f20000a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 10 monclient: _finish_hunting 0 2024-08-19T10:42:45.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 1 monclient: found mon.noname-e 2024-08-19T10:42:45.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:45.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:45.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2006465640 1 -- 172.21.15.89:0/1572851609 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20000a81d0 con 0x7f20001395b0 2024-08-19T10:42:45.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005463640 10 monclient: get_auth_request con 0x7f20000a6d30 auth_method 0 2024-08-19T10:42:45.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005c64640 1 --2- 172.21.15.89:0/1572851609 >> v2:172.21.15.89:3300/0 conn(0x7f20000a5270 0x7f20000a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:45.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.903+0000 7f2005463640 1 --2- 172.21.15.89:0/1572851609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20000a6d30 0x7f20000a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:45.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2006465640 1 -- 172.21.15.89:0/1572851609 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20001395b0 msgr2=0x7f2000139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2006465640 1 --2- 172.21.15.89:0/1572851609 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20001395b0 0x7f2000139a40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009e50 tx=0x7f1ff801c8f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=72f3c1ccf6a3aa6a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 1 -- 172.21.15.89:0/1572851609 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ff80090d0 con 0x7f20001395b0 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 10 monclient: dump: 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.920 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 10 monclient: _finish_auth 0 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 10 monclient: _check_auth_tickets 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 5 monclient: authenticate success, global_id 5453 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:45.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 1 -- 172.21.15.89:0/1572851609 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff80a5730 con 0x7f20001395b0 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 1 -- 172.21.15.89:0/1572851609 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ff8009270 con 0x7f20001395b0 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2004c62640 10 monclient: dump: 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.922 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 4 set_mon_vals no callback set 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient: shutdown 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 -- 172.21.15.89:0/1572851609 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20001395b0 msgr2=0x7f2000139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 --2- 172.21.15.89:0/1572851609 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20001395b0 0x7f2000139a40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009e50 tx=0x7f1ff801c8f0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 -- 172.21.15.89:0/1572851609 shutdown_connections 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 --2- 172.21.15.89:0/1572851609 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20001395b0 0x7f2000139a40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 --2- 172.21.15.89:0/1572851609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20000a6d30 0x7f20000a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 --2- 172.21.15.89:0/1572851609 >> v2:172.21.15.89:3300/0 conn(0x7f20000a5270 0x7f20000a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 -- 172.21.15.89:0/1572851609 >> 172.21.15.89:0/1572851609 conn(0x7f2000069a80 msgr2=0x7f20000abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 -- 172.21.15.89:0/1572851609 shutdown_connections 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 -- 172.21.15.89:0/1572851609 wait complete. 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient: build_initial_monmap 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient: monmap: 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.924 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 Processor -- start 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 -- start start 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient: init 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:45.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient: _add_conns ranks=[0,8,4,7,2,1,3,5,6] 2024-08-19T10:42:45.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20000a5270 0x7f20001347c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient(hunting): picked mon.noname-a con 0x7f20000a5270 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:45.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20000a6d30 0x7f2000134d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient(hunting): picked mon.noname-e con 0x7f20000a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:45.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20001395b0 0x7f200009c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient(hunting): picked mon.noname-c con 0x7f20001395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:45.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f20000a5650 con 0x7f20000a5270 2024-08-19T10:42:45.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f20000a4a70 con 0x7f20001395b0 2024-08-19T10:42:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f20000a57d0 con 0x7f20000a6d30 2024-08-19T10:42:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.907+0000 7f2007ec8640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2007ec8640 10 monclient(hunting): authenticate will time out at 1215.728008s 2024-08-19T10:42:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20000a5270 0x7f20001347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20000a5270 0x7f20001347c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37638/0 (socket says 172.21.15.89:37638) 2024-08-19T10:42:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 1 -- 172.21.15.89:0/4258403225 learned_addr learned my addr 172.21.15.89:0/4258403225 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 10 monclient(hunting): get_auth_request con 0x7f20000a5270 auth_method 0 2024-08-19T10:42:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2006465640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20001395b0 0x7f200009c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2006465640 10 monclient(hunting): get_auth_request con 0x7f20001395b0 auth_method 0 2024-08-19T10:42:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2006465640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2006465640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2006465640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005463640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20000a6d30 0x7f2000134d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005463640 10 monclient(hunting): get_auth_request con 0x7f20000a6d30 auth_method 0 2024-08-19T10:42:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005463640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005463640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005463640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2006465640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2006465640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2006465640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005463640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005463640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005463640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 10 monclient(hunting): handle_auth_done global_id 5706 payload 824 2024-08-19T10:42:45.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 1 -- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20000a6d30 msgr2=0x7f2000134d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20000a6d30 0x7f2000134d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 1 -- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20001395b0 msgr2=0x7f200009c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20001395b0 0x7f200009c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 10 monclient: _finish_hunting 0 2024-08-19T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 1 monclient: found mon.noname-a 2024-08-19T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 1 -- 172.21.15.89:0/4258403225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20000a0200 con 0x7f20000a5270 2024-08-19T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2006465640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20001395b0 0x7f200009c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 1 -- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20000a5270 msgr2=0x7f20001347c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:45.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2005c64640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20000a5270 0x7f20001347c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000e9b0 tx=0x7f1ff000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 1 -- 172.21.15.89:0/4258403225 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ff0017070 con 0x7f20000a5270 2024-08-19T10:42:45.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:45.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 10 monclient: dump: 2024-08-19T10:42:45.931 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.931 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.931 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.931 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.931 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.931 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.931 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.932 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.932 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.932 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.932 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.932 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.932 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.932 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.932 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.932 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 1 -- 172.21.15.89:0/4258403225 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff0013470 con 0x7f20000a5270 2024-08-19T10:42:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 1 -- 172.21.15.89:0/4258403225 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ff0013a90 con 0x7f20000a5270 2024-08-19T10:42:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff67fc640 10 monclient: dump: 2024-08-19T10:42:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:45.933 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.934 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:45.934 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:45.934 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:45.934 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:45.934 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:45.934 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:45.934 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:45.934 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:45.934 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:45.934 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:45.935 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:45.935 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:45.935 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:45.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2007ec8640 5 monclient: authenticate success, global_id 5706 2024-08-19T10:42:45.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2007ec8640 10 monclient: _renew_subs 2024-08-19T10:42:45.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f1ff7fff640 4 set_mon_vals no callback set 2024-08-19T10:42:45.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2007ec8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:45.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2007ec8640 1 -- 172.21.15.89:0/4258403225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20000a28d0 con 0x7f20000a5270 2024-08-19T10:42:45.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2007ec8640 10 monclient: _renew_subs 2024-08-19T10:42:45.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2007ec8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:45.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2007ec8640 1 -- 172.21.15.89:0/4258403225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200009fc40 con 0x7f20000a5270 2024-08-19T10:42:45.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.911+0000 7f2007ec8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:45.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.915+0000 7f2007ec8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:45.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.915+0000 7f2007ec8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:45.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.915+0000 7f1ff67fc640 1 -- 172.21.15.89:0/4258403225 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1ff001a070 con 0x7f20000a5270 2024-08-19T10:42:45.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.915+0000 7f2007ec8640 1 -- 172.21.15.89:0/4258403225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20000a7110 con 0x7f20000a5270 2024-08-19T10:42:45.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.915+0000 7f1ff67fc640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fe403be00 0x7f1fe403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:45.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.915+0000 7f1ff67fc640 1 -- 172.21.15.89:0/4258403225 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 8639+0+0 (secure 0 0 0) 0x7f1ff000f040 con 0x7f20000a5270 2024-08-19T10:42:45.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.915+0000 7f2005463640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fe403be00 0x7f1fe403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:45.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.915+0000 7f2005463640 10 monclient: get_auth_request con 0x7f1fe403be00 auth_method 0 2024-08-19T10:42:45.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.915+0000 7f2005463640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fe403be00 0x7f1fe403e2c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f200013f9d0 tx=0x7f1ffc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:45.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.919+0000 7f1ff67fc640 1 -- 172.21.15.89:0/4258403225 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1ff0021070 con 0x7f20000a5270 2024-08-19T10:42:45.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.919+0000 7f1ff67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:45.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:45.919+0000 7f1ff67fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:46.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.159+0000 7f2007ec8640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-08-19T10:42:46.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.159+0000 7f2007ec8640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-08-19T10:42:46.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.159+0000 7f2007ec8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:46.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.159+0000 7f2007ec8640 1 -- 172.21.15.89:0/4258403225 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f2000064590 con 0x7f20000a5270 2024-08-19T10:42:46.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.159+0000 7f1ff67fc640 1 -- 172.21.15.89:0/4258403225 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f1ff004d4b0 con 0x7f20000a5270 2024-08-19T10:42:46.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.159+0000 7f1ff67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-08-19T10:42:46.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.159+0000 7f1ff67fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:46.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 -- 172.21.15.89:0/4258403225 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fe403be00 msgr2=0x7f1fe403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:46.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fe403be00 0x7f1fe403e2c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f200013f9d0 tx=0x7f1ffc008040 comp rx=0 tx=0).stop 2024-08-19T10:42:46.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 10 monclient: shutdown 2024-08-19T10:42:46.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:46.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 -- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20000a5270 msgr2=0x7f20001347c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:46.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20000a5270 0x7f20001347c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000e9b0 tx=0x7f1ff000ee70 comp rx=0 tx=0).stop 2024-08-19T10:42:46.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 -- 172.21.15.89:0/4258403225 shutdown_connections 2024-08-19T10:42:46.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20001395b0 0x7f200009c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fe403be00 0x7f1fe403e2c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20000a6d30 0x7f2000134d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 --2- 172.21.15.89:0/4258403225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20000a5270 0x7f20001347c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 -- 172.21.15.89:0/4258403225 >> 172.21.15.89:0/4258403225 conn(0x7f2000069a80 msgr2=0x7f20001378b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:46.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 -- 172.21.15.89:0/4258403225 shutdown_connections 2024-08-19T10:42:46.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:46.163+0000 7f2007ec8640 1 -- 172.21.15.89:0/4258403225 wait complete. 2024-08-19T10:42:46.174 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.174 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=1574944564,config=2586702516,health=3475484141,kv=1101891974,logm=2403713131}) 2024-08-19T10:42:46.174 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=12,logm=64} crc {auth=1574944564,config=2586702516,health=3780704006,kv=1101891974,logm=4279509856}) 2024-08-19T10:42:46.174 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.174 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=1574944564,config=2586702516,health=3475484141,kv=1101891974,logm=2403713131}) 2024-08-19T10:42:46.174 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=1574944564,config=2586702516,health=3780704006,kv=4035213304,logm=4146054123}) 2024-08-19T10:42:46.174 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.174 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=1574944564,config=2586702516,health=3475484141,kv=1101891974,logm=2403713131}) 2024-08-19T10:42:46.174 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=2,health=12,kv=11,logm=66} crc {auth=992517093,config=2586702516,health=3780704006,kv=846547809,logm=2094009544}) 2024-08-19T10:42:46.174 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=1574944564,config=2586702516,health=3475484141,kv=1101891974,logm=2403713131}) 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=2532719078,config=2586702516,health=3780704006,kv=1101891974,logm=632849748}) 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=1574944564,config=2586702516,health=3475484141,kv=1101891974,logm=2403713131}) 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=212908386,config=2586702517,health=3780704006,kv=1101891974,logm=6183750}) 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=1574944564,config=2586702516,health=3475484141,kv=1101891974,logm=2403713131}) 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=10,config=2,health=12,kv=12,logm=64} crc {auth=3202902650,config=2586702516,health=3780704006,kv=1101891974,logm=2820212566}) 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=1574944564,config=2586702516,health=3475484141,kv=1101891974,logm=2403713131}) 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=10,config=2,health=11,kv=11,logm=66} crc {auth=2284024620,config=2586702516,health=3796471680,kv=1905375608,logm=423511307}) 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.175 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=1574944564,config=2586702516,health=3475484141,kv=1101891974,logm=2403713131}) 2024-08-19T10:42:46.176 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.167+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=2728305508,config=2586702516,health=3780704006,kv=1101891974,logm=2209037946}) 2024-08-19T10:42:46.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.168+0000 7f69967fc640 1 -- 172.21.15.190:0/83246669 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}]=0 v152) ==== 113+0+0 (secure 0 0 0) 0x7f6990022300 con 0x7f69a00a4070 2024-08-19T10:42:46.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.168+0000 7f69967fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-08-19T10:42:46.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.168+0000 7f69967fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:46.188 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-08-19T10:42:46.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.280+0000 7f69a8022640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-08-19T10:42:46.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.280+0000 7f69a8022640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-08-19T10:42:46.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.280+0000 7f69a8022640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:42:46.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.280+0000 7f69a8022640 1 -- 172.21.15.190:0/83246669 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"} v 0) -- 0x7f69a0125fb0 con 0x7f69a00a4070 2024-08-19T10:42:46.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.284+0000 7f69967fc640 1 -- 172.21.15.190:0/83246669 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}]=0 weight-set for pool 'cold' already created v152) ==== 155+0+0 (secure 0 0 0) 0x7f6990012ad0 con 0x7f69a00a4070 2024-08-19T10:42:46.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.284+0000 7f69967fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create", "pool": "cold", "mode": "positional"}] 2024-08-19T10:42:46.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.284+0000 7f69967fc640 10 monclient: _finish_command 3 = system:0 weight-set for pool 'cold' already created 2024-08-19T10:42:46.288 INFO:tasks.workunit.client.0.smithi190.stderr:weight-set for pool 'cold' already created 2024-08-19T10:42:46.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.284+0000 7f69a8022640 1 -- 172.21.15.190:0/83246669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698403be30 msgr2=0x7f698403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:46.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.284+0000 7f69a8022640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698403be30 0x7f698403e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f69a006cb30 tx=0x7f699c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:46.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.284+0000 7f69a8022640 10 monclient: shutdown 2024-08-19T10:42:46.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.284+0000 7f69a8022640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:46.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.284+0000 7f69a8022640 1 -- 172.21.15.190:0/83246669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69a00a4070 msgr2=0x7f69a013cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:46.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.284+0000 7f69a8022640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69a00a4070 0x7f69a013cdb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f699000c440 tx=0x7f699000c900 comp rx=0 tx=0).stop 2024-08-19T10:42:46.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.288+0000 7f69a8022640 1 -- 172.21.15.190:0/83246669 shutdown_connections 2024-08-19T10:42:46.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.288+0000 7f69a8022640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f698403be30 0x7f698403e2f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.288+0000 7f69a8022640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f69a013ebb0 0x7f69a0123810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.288+0000 7f69a8022640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69a00a4070 0x7f69a013cdb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.288+0000 7f69a8022640 1 --2- 172.21.15.190:0/83246669 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f69a00a3c90 0x7f69a0072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.288+0000 7f69a8022640 1 -- 172.21.15.190:0/83246669 >> 172.21.15.190:0/83246669 conn(0x7f69a006de10 msgr2=0x7f69a013ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:46.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.288+0000 7f69a8022640 1 -- 172.21.15.190:0/83246669 shutdown_connections 2024-08-19T10:42:46.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.288+0000 7f69a8022640 1 -- 172.21.15.190:0/83246669 wait complete. 2024-08-19T10:42:46.306 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm-device-class osd.0 2024-08-19T10:42:46.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient: get_monmap_and_config 2024-08-19T10:42:46.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient: build_initial_monmap 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient: monmap: 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:46.425439+0000 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:46.425439+0000 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:46.425 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:46.426 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:46.426 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:46.426 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:46.426 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:46.426 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:46.426 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:46.426 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:46.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 1 Processor -- start 2024-08-19T10:42:46.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 1 -- start start 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient: init 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient: _add_conns ranks=[8,5,7,2,6,9,3,1,4,10,0] 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6de00a3610 0x7f6de00a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient(hunting): picked mon.noname-e con 0x7f6de00a3610 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6de00a3f30 0x7f6de01363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient(hunting): picked mon.noname-h con 0x7f6de00a3f30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de0136910 0x7f6de0136d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient(hunting): picked mon.noname-i con 0x7f6de0136910 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6de0073290 con 0x7f6de00a3f30 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:46.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6de00729f0 con 0x7f6de0136910 2024-08-19T10:42:46.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:46.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6de0072cd0 con 0x7f6de00a3610 2024-08-19T10:42:46.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:46.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de6ef5640 10 monclient(hunting): authenticate will time out at 1211.995453s 2024-08-19T10:42:46.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de0136910 0x7f6de0136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:46.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6ddffff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6de00a3f30 0x7f6de01363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:46.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de0136910 0x7f6de0136d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:43830/0 (socket says 172.21.15.190:43830) 2024-08-19T10:42:46.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 1 -- 172.21.15.190:0/2953542496 learned_addr learned my addr 172.21.15.190:0/2953542496 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:46.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 10 monclient(hunting): get_auth_request con 0x7f6de0136910 auth_method 0 2024-08-19T10:42:46.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:46.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6ddffff640 10 monclient(hunting): get_auth_request con 0x7f6de00a3f30 auth_method 0 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6ddffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6ddffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6ddffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6ddffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6ddffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6ddffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de4c91640 1 --2- 172.21.15.190:0/2953542496 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6de00a3610 0x7f6de00a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:46.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de4c91640 10 monclient(hunting): get_auth_request con 0x7f6de00a3610 auth_method 0 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de4c91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de4c91640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de4c91640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 10 monclient(hunting): handle_auth_done global_id 14875 payload 274 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 1 -- 172.21.15.190:0/2953542496 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6de00a3610 msgr2=0x7f6de00a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 1 --2- 172.21.15.190:0/2953542496 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6de00a3610 0x7f6de00a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 1 -- 172.21.15.190:0/2953542496 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6de00a3f30 msgr2=0x7f6de01363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.420+0000 7f6de5492640 1 --2- 172.21.15.190:0/2953542496 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6de00a3f30 0x7f6de01363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.424+0000 7f6de5492640 10 monclient: _finish_hunting 0 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.424+0000 7f6de5492640 1 monclient: found mon.noname-i 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.424+0000 7f6de5492640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.424+0000 7f6de5492640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:46.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.424+0000 7f6de5492640 1 -- 172.21.15.190:0/2953542496 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de00a5570 con 0x7f6de0136910 2024-08-19T10:42:46.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.424+0000 7f6ddffff640 1 --2- 172.21.15.190:0/2953542496 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6de00a3f30 0x7f6de01363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:46.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de5492640 1 -- 172.21.15.190:0/2953542496 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de0136910 msgr2=0x7f6de0136d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:46.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de5492640 1 --2- 172.21.15.190:0/2953542496 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de0136910 0x7f6de0136d30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80098a0 tx=0x7f6dd801c440 comp rx=0 tx=0).ready entity=mon.7 client_cookie=9ca989338a396fa0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:46.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 1 -- 172.21.15.190:0/2953542496 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dd80b0070 con 0x7f6de0136910 2024-08-19T10:42:46.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:46.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:46.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 10 monclient: dump: 2024-08-19T10:42:46.438 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:46.439 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 1 -- 172.21.15.190:0/2953542496 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd80aa810 con 0x7f6de0136910 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 5 monclient: authenticate success, global_id 14875 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 1 -- 172.21.15.190:0/2953542496 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dd80a9300 con 0x7f6de0136910 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 4 set_mon_vals no callback set 2024-08-19T10:42:46.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddf7fe640 10 monclient: dump: 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:46.441 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient: shutdown 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 -- 172.21.15.190:0/2953542496 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de0136910 msgr2=0x7f6de0136d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 --2- 172.21.15.190:0/2953542496 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de0136910 0x7f6de0136d30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80098a0 tx=0x7f6dd801c440 comp rx=0 tx=0).stop 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 -- 172.21.15.190:0/2953542496 shutdown_connections 2024-08-19T10:42:46.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 --2- 172.21.15.190:0/2953542496 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de0136910 0x7f6de0136d30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 --2- 172.21.15.190:0/2953542496 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6de00a3f30 0x7f6de01363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 --2- 172.21.15.190:0/2953542496 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6de00a3610 0x7f6de00a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 -- 172.21.15.190:0/2953542496 >> 172.21.15.190:0/2953542496 conn(0x7f6de00a4700 msgr2=0x7f6de00a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 -- 172.21.15.190:0/2953542496 shutdown_connections 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 -- 172.21.15.190:0/2953542496 wait complete. 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient: build_initial_monmap 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient: monmap: 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:46.443 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 Processor -- start 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 -- start start 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient: init 2024-08-19T10:42:46.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient: _add_conns ranks=[7,0,1,6,2,3,8,4,5] 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de00a3610 0x7f6de0134df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient(hunting): picked mon.noname-i con 0x7f6de00a3610 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de00a3f30 0x7f6de0135330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient(hunting): picked mon.noname-a con 0x7f6de00a3f30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de0136910 0x7f6de009ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient(hunting): picked mon.noname-f con 0x7f6de0136910 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0073e20 con 0x7f6de00a3f30 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0073ca0 con 0x7f6de0136910 2024-08-19T10:42:46.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6de0071540 con 0x7f6de00a3610 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de6ef5640 10 monclient(hunting): authenticate will time out at 1212.002580s 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de00a3610 0x7f6de0134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de5492640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de0136910 0x7f6de009ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de00a3610 0x7f6de0134df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:43840/0 (socket says 172.21.15.190:43840) 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 1 -- 172.21.15.190:0/3856004454 learned_addr learned my addr 172.21.15.190:0/3856004454 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 10 monclient(hunting): get_auth_request con 0x7f6de00a3610 auth_method 0 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de5492640 10 monclient(hunting): get_auth_request con 0x7f6de0136910 auth_method 0 2024-08-19T10:42:46.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de5492640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de5492640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de5492640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de5492640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de5492640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de5492640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddffff640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de00a3f30 0x7f6de0135330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 10 monclient(hunting): handle_auth_done global_id 14884 payload 824 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 1 -- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de0136910 msgr2=0x7f6de009ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:46.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de0136910 0x7f6de009ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 1 -- 172.21.15.190:0/3856004454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de00a3f30 msgr2=0x7f6de0135330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:46.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de00a3f30 0x7f6de0135330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:46.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 10 monclient: _finish_hunting 0 2024-08-19T10:42:46.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 1 monclient: found mon.noname-i 2024-08-19T10:42:46.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:46.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:42:46.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de4c91640 1 -- 172.21.15.190:0/3856004454 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de0123460 con 0x7f6de00a3610 2024-08-19T10:42:46.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddffff640 10 monclient: get_auth_request con 0x7f6de00a3f30 auth_method 0 2024-08-19T10:42:46.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6ddffff640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de00a3f30 0x7f6de0135330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:46.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.428+0000 7f6de5492640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de0136910 0x7f6de009ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:46.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de4c91640 1 -- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de00a3610 msgr2=0x7f6de0134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de4c91640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de00a3610 0x7f6de0134df0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000d830 tx=0x7f6dd000dcf0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 1 -- 172.21.15.190:0/3856004454 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dd000bd40 con 0x7f6de00a3610 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 10 monclient: dump: 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:46.449 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 10 monclient: _finish_auth 0 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:46.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 1 -- 172.21.15.190:0/3856004454 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd0019070 con 0x7f6de00a3610 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de6ef5640 5 monclient: authenticate success, global_id 14884 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de6ef5640 10 monclient: _renew_subs 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de6ef5640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de6ef5640 1 -- 172.21.15.190:0/3856004454 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de0123a20 con 0x7f6de00a3610 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 1 -- 172.21.15.190:0/3856004454 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dd00148a0 con 0x7f6de00a3610 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddeffd640 4 set_mon_vals no callback set 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 10 monclient: dump: 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:46.451 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:46.452 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de6ef5640 10 monclient: _renew_subs 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de6ef5640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de6ef5640 1 -- 172.21.15.190:0/3856004454 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de0135870 con 0x7f6de00a3610 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de6ef5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de6ef5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de6ef5640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6de6ef5640 1 -- 172.21.15.190:0/3856004454 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de01363d0 con 0x7f6de00a3610 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 1 -- 172.21.15.190:0/3856004454 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6dd0014430 con 0x7f6de00a3610 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc003be00 0x7f6dc003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddd7fa640 1 -- 172.21.15.190:0/3856004454 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(152..152 src has 1..152) ==== 9111+0+0 (secure 0 0 0) 0x7f6dd000f040 con 0x7f6de00a3610 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddffff640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc003be00 0x7f6dc003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:46.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.432+0000 7f6ddffff640 10 monclient: get_auth_request con 0x7f6dc003be00 auth_method 0 2024-08-19T10:42:46.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.436+0000 7f6ddffff640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc003be00 0x7f6dc003e2c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009bd0 tx=0x7f6dc800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:46.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.436+0000 7f6ddd7fa640 1 -- 172.21.15.190:0/3856004454 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6dd0022020 con 0x7f6de00a3610 2024-08-19T10:42:46.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.436+0000 7f6ddd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:46.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.436+0000 7f6ddd7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:46.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.616+0000 7f6de6ef5640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:42:46.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.616+0000 7f6de6ef5640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:42:46.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.616+0000 7f6de6ef5640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:46.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:46.616+0000 7f6de6ef5640 1 -- 172.21.15.190:0/3856004454 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7f6de006db30 con 0x7f6de00a3610 2024-08-19T10:42:46.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758109126}) 2024-08-19T10:42:46.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=637928967}) 2024-08-19T10:42:46.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758109126}) 2024-08-19T10:42:46.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908727489}) 2024-08-19T10:42:46.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758109126}) 2024-08-19T10:42:46.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1909583471}) 2024-08-19T10:42:46.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758109126}) 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2016290361}) 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758109126}) 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3378310894}) 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758109126}) 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3738045594}) 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758109126}) 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3894610654}) 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758109126}) 2024-08-19T10:42:46.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.983+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2626256888}) 2024-08-19T10:42:46.995 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.995 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=480363898}) 2024-08-19T10:42:46.995 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4243613873}) 2024-08-19T10:42:46.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=480363898}) 2024-08-19T10:42:46.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119693123}) 2024-08-19T10:42:46.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=480363898}) 2024-08-19T10:42:46.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2558733722}) 2024-08-19T10:42:46.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=480363898}) 2024-08-19T10:42:46.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3290658235}) 2024-08-19T10:42:46.996 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=480363898}) 2024-08-19T10:42:46.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1492911319}) 2024-08-19T10:42:46.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=480363898}) 2024-08-19T10:42:46.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=656328426}) 2024-08-19T10:42:46.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=480363898}) 2024-08-19T10:42:46.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2852936716}) 2024-08-19T10:42:46.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=480363898}) 2024-08-19T10:42:46.997 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.991+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3865451180}) 2024-08-19T10:42:46.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:46.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3757541361}) 2024-08-19T10:42:46.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3186975879}) 2024-08-19T10:42:46.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3757541361}) 2024-08-19T10:42:47.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3093734755}) 2024-08-19T10:42:47.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3757541361}) 2024-08-19T10:42:47.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2586063352}) 2024-08-19T10:42:47.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3757541361}) 2024-08-19T10:42:47.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2530677869}) 2024-08-19T10:42:47.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3757541361}) 2024-08-19T10:42:47.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3240685516}) 2024-08-19T10:42:47.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3757541361}) 2024-08-19T10:42:47.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1662060664}) 2024-08-19T10:42:47.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3757541361}) 2024-08-19T10:42:47.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=219924939}) 2024-08-19T10:42:47.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3757541361}) 2024-08-19T10:42:47.001 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.995+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1513958886}) 2024-08-19T10:42:47.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1535452993}) 2024-08-19T10:42:47.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1679525306}) 2024-08-19T10:42:47.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1535452993}) 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2279064204}) 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1535452993}) 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4028273351}) 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1535452993}) 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4071538255}) 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1535452993}) 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2150785609}) 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1535452993}) 2024-08-19T10:42:47.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3761563531}) 2024-08-19T10:42:47.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1535452993}) 2024-08-19T10:42:47.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1999198761}) 2024-08-19T10:42:47.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1535452993}) 2024-08-19T10:42:47.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:46.999+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2950280780}) 2024-08-19T10:42:47.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624502795}) 2024-08-19T10:42:47.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4245896299}) 2024-08-19T10:42:47.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624502795}) 2024-08-19T10:42:47.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3729057619}) 2024-08-19T10:42:47.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624502795}) 2024-08-19T10:42:47.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1767110378}) 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624502795}) 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3637301473}) 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624502795}) 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=578613350}) 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624502795}) 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3220006532}) 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624502795}) 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=494930688}) 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624502795}) 2024-08-19T10:42:47.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.003+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3704356397}) 2024-08-19T10:42:47.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1672997732}) 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1999328900}) 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1672997732}) 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2018847599}) 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1672997732}) 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3942572709}) 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1672997732}) 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4231406104}) 2024-08-19T10:42:47.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1672997732}) 2024-08-19T10:42:47.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2955913210}) 2024-08-19T10:42:47.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1672997732}) 2024-08-19T10:42:47.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=785214001}) 2024-08-19T10:42:47.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1672997732}) 2024-08-19T10:42:47.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3143667449}) 2024-08-19T10:42:47.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1672997732}) 2024-08-19T10:42:47.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.007+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1420185108}) 2024-08-19T10:42:47.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934983663}) 2024-08-19T10:42:47.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2872700150}) 2024-08-19T10:42:47.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934983663}) 2024-08-19T10:42:47.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3152833545}) 2024-08-19T10:42:47.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934983663}) 2024-08-19T10:42:47.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3036438121}) 2024-08-19T10:42:47.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934983663}) 2024-08-19T10:42:47.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=928515056}) 2024-08-19T10:42:47.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934983663}) 2024-08-19T10:42:47.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4242382991}) 2024-08-19T10:42:47.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934983663}) 2024-08-19T10:42:47.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=144807528}) 2024-08-19T10:42:47.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934983663}) 2024-08-19T10:42:47.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2310538418}) 2024-08-19T10:42:47.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934983663}) 2024-08-19T10:42:47.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.011+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3061864615}) 2024-08-19T10:42:47.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035431208}) 2024-08-19T10:42:47.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2082109304}) 2024-08-19T10:42:47.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035431208}) 2024-08-19T10:42:47.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=545365515}) 2024-08-19T10:42:47.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035431208}) 2024-08-19T10:42:47.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3003003540}) 2024-08-19T10:42:47.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035431208}) 2024-08-19T10:42:47.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=602263727}) 2024-08-19T10:42:47.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035431208}) 2024-08-19T10:42:47.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=61602200}) 2024-08-19T10:42:47.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035431208}) 2024-08-19T10:42:47.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1806724998}) 2024-08-19T10:42:47.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035431208}) 2024-08-19T10:42:47.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2409671741}) 2024-08-19T10:42:47.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035431208}) 2024-08-19T10:42:47.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.015+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=608957590}) 2024-08-19T10:42:47.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.019+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.019+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1202440569}) 2024-08-19T10:42:47.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.019+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3391825036}) 2024-08-19T10:42:47.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.019+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.019+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1202440569}) 2024-08-19T10:42:47.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.019+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=699212677}) 2024-08-19T10:42:47.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.019+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.019+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1202440569}) 2024-08-19T10:42:47.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.019+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3050631538}) 2024-08-19T10:42:47.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1202440569}) 2024-08-19T10:42:47.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3709082408}) 2024-08-19T10:42:47.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1202440569}) 2024-08-19T10:42:47.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=523049753}) 2024-08-19T10:42:47.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1202440569}) 2024-08-19T10:42:47.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1673114073}) 2024-08-19T10:42:47.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1202440569}) 2024-08-19T10:42:47.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=280763002}) 2024-08-19T10:42:47.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1202440569}) 2024-08-19T10:42:47.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.023+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=769317517}) 2024-08-19T10:42:47.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3664388777}) 2024-08-19T10:42:47.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4259725396}) 2024-08-19T10:42:47.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3664388777}) 2024-08-19T10:42:47.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2421110087}) 2024-08-19T10:42:47.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3664388777}) 2024-08-19T10:42:47.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4030848938}) 2024-08-19T10:42:47.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3664388777}) 2024-08-19T10:42:47.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1606810066}) 2024-08-19T10:42:47.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3664388777}) 2024-08-19T10:42:47.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=591282755}) 2024-08-19T10:42:47.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3664388777}) 2024-08-19T10:42:47.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1799837983}) 2024-08-19T10:42:47.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3664388777}) 2024-08-19T10:42:47.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2881827454}) 2024-08-19T10:42:47.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3664388777}) 2024-08-19T10:42:47.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.027+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2812979951}) 2024-08-19T10:42:47.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212760669}) 2024-08-19T10:42:47.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4239685437}) 2024-08-19T10:42:47.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212760669}) 2024-08-19T10:42:47.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1757459938}) 2024-08-19T10:42:47.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212760669}) 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2946303178}) 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212760669}) 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2326202488}) 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212760669}) 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=660052511}) 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212760669}) 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3297844342}) 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212760669}) 2024-08-19T10:42:47.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3923731980}) 2024-08-19T10:42:47.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=212760669}) 2024-08-19T10:42:47.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.039+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=104895774}) 2024-08-19T10:42:47.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3013803497}) 2024-08-19T10:42:47.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3282211940}) 2024-08-19T10:42:47.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3013803497}) 2024-08-19T10:42:47.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4147619862}) 2024-08-19T10:42:47.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3013803497}) 2024-08-19T10:42:47.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1639118221}) 2024-08-19T10:42:47.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3013803497}) 2024-08-19T10:42:47.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=819891130}) 2024-08-19T10:42:47.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3013803497}) 2024-08-19T10:42:47.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4012179755}) 2024-08-19T10:42:47.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3013803497}) 2024-08-19T10:42:47.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2543584016}) 2024-08-19T10:42:47.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3013803497}) 2024-08-19T10:42:47.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=387267064}) 2024-08-19T10:42:47.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3013803497}) 2024-08-19T10:42:47.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.043+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1617042284}) 2024-08-19T10:42:47.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545572227}) 2024-08-19T10:42:47.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1969831470}) 2024-08-19T10:42:47.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545572227}) 2024-08-19T10:42:47.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2185521088}) 2024-08-19T10:42:47.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545572227}) 2024-08-19T10:42:47.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3454832856}) 2024-08-19T10:42:47.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545572227}) 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1490247453}) 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545572227}) 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2194501586}) 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545572227}) 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=847935636}) 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545572227}) 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2909540642}) 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545572227}) 2024-08-19T10:42:47.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4165029923}) 2024-08-19T10:42:47.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503418038}) 2024-08-19T10:42:47.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3875997939}) 2024-08-19T10:42:47.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503418038}) 2024-08-19T10:42:47.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1605806604}) 2024-08-19T10:42:47.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503418038}) 2024-08-19T10:42:47.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4231574034}) 2024-08-19T10:42:47.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503418038}) 2024-08-19T10:42:47.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2736609408}) 2024-08-19T10:42:47.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503418038}) 2024-08-19T10:42:47.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2880181288}) 2024-08-19T10:42:47.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503418038}) 2024-08-19T10:42:47.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2849118160}) 2024-08-19T10:42:47.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503418038}) 2024-08-19T10:42:47.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3391658659}) 2024-08-19T10:42:47.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503418038}) 2024-08-19T10:42:47.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.087+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=279454929}) 2024-08-19T10:42:47.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958220281}) 2024-08-19T10:42:47.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3017929406}) 2024-08-19T10:42:47.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958220281}) 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2881302747}) 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958220281}) 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=327478777}) 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958220281}) 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2478104325}) 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958220281}) 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2521233178}) 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958220281}) 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2265803063}) 2024-08-19T10:42:47.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958220281}) 2024-08-19T10:42:47.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1803664050}) 2024-08-19T10:42:47.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=958220281}) 2024-08-19T10:42:47.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.091+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4117089409}) 2024-08-19T10:42:47.167 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.167 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397987835}) 2024-08-19T10:42:47.167 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2286709636}) 2024-08-19T10:42:47.167 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397987835}) 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1094628431}) 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397987835}) 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2104766221}) 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397987835}) 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1668791260}) 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397987835}) 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2743222011}) 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.168 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397987835}) 2024-08-19T10:42:47.169 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1829149755}) 2024-08-19T10:42:47.169 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.169 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397987835}) 2024-08-19T10:42:47.169 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1448480096}) 2024-08-19T10:42:47.169 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.169 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397987835}) 2024-08-19T10:42:47.169 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.163+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1426791204}) 2024-08-19T10:42:47.178 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.178 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3466332244}) 2024-08-19T10:42:47.178 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2831006808}) 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3466332244}) 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1459551088}) 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3466332244}) 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2236471447}) 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3466332244}) 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=296132546}) 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3466332244}) 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2808335904}) 2024-08-19T10:42:47.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3466332244}) 2024-08-19T10:42:47.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3380453566}) 2024-08-19T10:42:47.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3466332244}) 2024-08-19T10:42:47.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2215706914}) 2024-08-19T10:42:47.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3466332244}) 2024-08-19T10:42:47.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.175+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1817743134}) 2024-08-19T10:42:47.189 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:42:47.190 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:42:47.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1206312379}) 2024-08-19T10:42:47.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=776987476}) 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1206312379}) 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1791781587}) 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1206312379}) 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2537863453}) 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1206312379}) 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3769945700}) 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1206312379}) 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2167242498}) 2024-08-19T10:42:47.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1206312379}) 2024-08-19T10:42:47.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3757308929}) 2024-08-19T10:42:47.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1206312379}) 2024-08-19T10:42:47.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2166882065}) 2024-08-19T10:42:47.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1206312379}) 2024-08-19T10:42:47.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.187+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2312055509}) 2024-08-19T10:42:47.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:47.188+0000 7f6ddd7fa640 1 -- 172.21.15.190:0/3856004454 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v153) ==== 123+0+0 (secure 0 0 0) 0x7f6dd004d4e0 con 0x7f6de00a3610 2024-08-19T10:42:47.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:47.188+0000 7f6ddd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:42:47.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:47.188+0000 7f6ddd7fa640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-08-19T10:42:47.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:47.296+0000 7f6de6ef5640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:42:47.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:47.296+0000 7f6de6ef5640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:42:47.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:47.296+0000 7f6de6ef5640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:42:47.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:47.296+0000 7f6de6ef5640 1 -- 172.21.15.190:0/3856004454 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7f6de009e690 con 0x7f6de00a3610 2024-08-19T10:42:47.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.311+0000 7f6d27b7b640 10 monclient: get_monmap_and_config 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.311+0000 7f6d27b7b640 10 monclient: build_initial_monmap 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.311+0000 7f6d27b7b640 10 monclient: monmap: 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:47.316410+0000 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:47.316410+0000 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:47.316 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:47.317 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:47.317 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:47.317 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:47.317 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:47.317 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:47.317 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:47.317 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:47.317 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:47.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 1 Processor -- start 2024-08-19T10:42:47.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 1 -- start start 2024-08-19T10:42:47.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 10 monclient: init 2024-08-19T10:42:47.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 10 monclient: _add_conns ranks=[2,8,6,9,1,7,3,5,0,10,4] 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d200a3c70 0x7f6d200a4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 10 monclient(hunting): picked mon.noname-b con 0x7f6d200a3c70 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a4590 0x7f6d20139960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 10 monclient(hunting): picked mon.noname-e con 0x7f6d200a4590 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 0x7f6d2013a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 10 monclient(hunting): picked mon.noname-d con 0x7f6d20139ea0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6d200a5640 con 0x7f6d200a3c70 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6d2006cfb0 con 0x7f6d20139ea0 2024-08-19T10:42:47.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6d200a0b40 con 0x7f6d200a4590 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d27b7b640 10 monclient(hunting): authenticate will time out at 1217.133346s 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d200a3c70 0x7f6d200a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d26118640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 0x7f6d2013a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25116640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a4590 0x7f6d20139960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d26118640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 0x7f6d2013a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54510/0 (socket says 172.21.15.89:54510) 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d26118640 1 -- 172.21.15.89:0/3851371460 learned_addr learned my addr 172.21.15.89:0/3851371460 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d200a3c70 0x7f6d200a4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40152/0 (socket says 172.21.15.89:40152) 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25116640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a4590 0x7f6d20139960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40446/0 (socket says 172.21.15.89:40446) 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d26118640 10 monclient(hunting): get_auth_request con 0x7f6d20139ea0 auth_method 0 2024-08-19T10:42:47.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d26118640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d26118640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d26118640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 10 monclient(hunting): get_auth_request con 0x7f6d200a3c70 auth_method 0 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25116640 10 monclient(hunting): get_auth_request con 0x7f6d200a4590 auth_method 0 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25116640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25116640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25116640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:47.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d26118640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d26118640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d26118640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25116640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25116640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25116640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 10 monclient(hunting): handle_auth_done global_id 5645 payload 274 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 1 -- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a4590 msgr2=0x7f6d20139960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 1 --2- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a4590 0x7f6d20139960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 1 -- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 msgr2=0x7f6d2013a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 1 --2- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 0x7f6d2013a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 10 monclient: _finish_hunting 0 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 1 monclient: found mon.noname-b 2024-08-19T10:42:47.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:47.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:42:47.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d25917640 1 -- 172.21.15.89:0/3851371460 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d200a0e20 con 0x7f6d200a3c70 2024-08-19T10:42:47.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.315+0000 7f6d26118640 1 --2- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 0x7f6d2013a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d25917640 1 -- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d200a3c70 msgr2=0x7f6d200a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d25917640 1 --2- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d200a3c70 0x7f6d200a4050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0097f0 tx=0x7f6d1c01c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d00b3ac162536653 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 1 -- 172.21.15.89:0/3851371460 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d1c0b0070 con 0x7f6d200a3c70 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 10 monclient: dump: 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:47.329 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 10 monclient: _finish_auth 0 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 10 monclient: _check_auth_tickets 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:47.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 1 -- 172.21.15.89:0/3851371460 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c0aa810 con 0x7f6d200a3c70 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 5 monclient: authenticate success, global_id 5645 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 1 -- 172.21.15.89:0/3851371460 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d1c0a9300 con 0x7f6d200a3c70 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 4 set_mon_vals no callback set 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d24915640 10 monclient: dump: 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:47.331 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:47.332 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 10 monclient: shutdown 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 1 -- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d200a3c70 msgr2=0x7f6d200a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 1 --2- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d200a3c70 0x7f6d200a4050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0097f0 tx=0x7f6d1c01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 1 -- 172.21.15.89:0/3851371460 shutdown_connections 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 1 --2- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 0x7f6d2013a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 1 --2- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a4590 0x7f6d20139960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 1 --2- 172.21.15.89:0/3851371460 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d200a3c70 0x7f6d200a4050 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 1 -- 172.21.15.89:0/3851371460 >> 172.21.15.89:0/3851371460 conn(0x7f6d2006de40 msgr2=0x7f6d2006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 1 -- 172.21.15.89:0/3851371460 shutdown_connections 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 1 -- 172.21.15.89:0/3851371460 wait complete. 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 10 monclient: build_initial_monmap 2024-08-19T10:42:47.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 10 monclient: monmap: 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:47.334 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 1 Processor -- start 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 1 -- start start 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.319+0000 7f6d27b7b640 10 monclient: init 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient: _add_conns ranks=[8,4,6,1,7,3,0,2,5] 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a3c70 0x7f6d201d1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient(hunting): picked mon.noname-e con 0x7f6d200a3c70 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a4590 0x7f6d201d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient(hunting): picked mon.noname-c con 0x7f6d200a4590 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 0x7f6d201d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient(hunting): picked mon.noname-d con 0x7f6d20139ea0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:47.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6d2006d3b0 con 0x7f6d200a4590 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6d2006d230 con 0x7f6d20139ea0 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6d20143440 con 0x7f6d200a3c70 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient(hunting): authenticate will time out at 1217.140402s 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a3c70 0x7f6d201d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a3c70 0x7f6d201d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40450/0 (socket says 172.21.15.89:40450) 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 1 -- 172.21.15.89:0/4088023399 learned_addr learned my addr 172.21.15.89:0/4088023399 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 10 monclient(hunting): get_auth_request con 0x7f6d200a3c70 auth_method 0 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:47.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25116640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a4590 0x7f6d201d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d26118640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 0x7f6d201d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25116640 10 monclient(hunting): get_auth_request con 0x7f6d200a4590 auth_method 0 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25116640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25116640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25116640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25116640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25116640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:47.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25116640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 10 monclient(hunting): handle_auth_done global_id 5489 payload 824 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 1 -- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 msgr2=0x7f6d201d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 0x7f6d201d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 1 -- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a4590 msgr2=0x7f6d201d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a4590 0x7f6d201d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 10 monclient: _finish_hunting 0 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 1 monclient: found mon.noname-e 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 1 -- 172.21.15.89:0/4088023399 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2013efc0 con 0x7f6d200a3c70 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d26118640 10 monclient: get_auth_request con 0x7f6d20139ea0 auth_method 0 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d26118640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 0x7f6d201d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 1 -- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a3c70 msgr2=0x7f6d201d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:47.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d25917640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a3c70 0x7f6d201d1360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0ae040 tx=0x7f6d1c019140 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 1 -- 172.21.15.89:0/4088023399 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d1c0b0040 con 0x7f6d200a3c70 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 10 monclient: dump: 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:47.339 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 5 monclient: authenticate success, global_id 5489 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 1 -- 172.21.15.89:0/4088023399 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c0b9070 con 0x7f6d200a3c70 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient: _renew_subs 2024-08-19T10:42:47.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 1 -- 172.21.15.89:0/4088023399 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2013f6a0 con 0x7f6d200a3c70 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 1 -- 172.21.15.89:0/4088023399 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d1c0aae30 con 0x7f6d200a3c70 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d17fff640 4 set_mon_vals no callback set 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d167fc640 10 monclient: dump: 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:47.341 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient: _renew_subs 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:47.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.323+0000 7f6d27b7b640 1 -- 172.21.15.89:0/4088023399 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2013d790 con 0x7f6d200a3c70 2024-08-19T10:42:47.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.327+0000 7f6d27b7b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:47.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.327+0000 7f6d27b7b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:47.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.327+0000 7f6d27b7b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:47.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.327+0000 7f6d27b7b640 1 -- 172.21.15.89:0/4088023399 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d20139960 con 0x7f6d200a3c70 2024-08-19T10:42:47.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.327+0000 7f6d25917640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a3c70 0x7f6d201d1360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0ae040 tx=0x7f6d1c019140 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f6d1c0e96f0 osd_map(153..153 src has 1..153) 2024-08-19T10:42:47.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.327+0000 7f6d167fc640 1 -- 172.21.15.89:0/4088023399 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6d1c004040 con 0x7f6d200a3c70 2024-08-19T10:42:47.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.327+0000 7f6d167fc640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0403bdf0 0x7f6d0403e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:47.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.327+0000 7f6d25116640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0403bdf0 0x7f6d0403e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:47.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.327+0000 7f6d25116640 10 monclient: get_auth_request con 0x7f6d0403bdf0 auth_method 0 2024-08-19T10:42:47.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.331+0000 7f6d25116640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0403bdf0 0x7f6d0403e2b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6d2013d4d0 tx=0x7f6d10009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:47.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:47.331+0000 7f6d167fc640 1 -- 172.21.15.89:0/4088023399 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(153..153 src has 1..153) ==== 9079+0+0 (secure 0 0 0) 0x7f6d1c0e96f0 con 0x7f6d200a3c70 2024-08-19T10:42:47.383 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.383 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2160923531}) 2024-08-19T10:42:47.383 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3788802878}) 2024-08-19T10:42:47.383 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2160923531}) 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=749075603}) 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2160923531}) 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1449937013}) 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2160923531}) 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1034302070}) 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2160923531}) 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2507235062}) 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2160923531}) 2024-08-19T10:42:47.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2875128183}) 2024-08-19T10:42:47.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2160923531}) 2024-08-19T10:42:47.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1713511729}) 2024-08-19T10:42:47.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2160923531}) 2024-08-19T10:42:47.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.379+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3520841153}) 2024-08-19T10:42:47.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.627+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.627+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3364629018}) 2024-08-19T10:42:47.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.627+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=336207692}) 2024-08-19T10:42:47.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.627+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.627+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3364629018}) 2024-08-19T10:42:47.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.627+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=898684817}) 2024-08-19T10:42:47.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3364629018}) 2024-08-19T10:42:47.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3623040937}) 2024-08-19T10:42:47.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3364629018}) 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=782834685}) 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3364629018}) 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3656891901}) 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3364629018}) 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3568664139}) 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3364629018}) 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1382051723}) 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3364629018}) 2024-08-19T10:42:47.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.631+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2843902678}) 2024-08-19T10:42:47.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581657799}) 2024-08-19T10:42:47.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3137757998}) 2024-08-19T10:42:47.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581657799}) 2024-08-19T10:42:47.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395681968}) 2024-08-19T10:42:47.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581657799}) 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=743668277}) 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581657799}) 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=107414614}) 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581657799}) 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1824375742}) 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581657799}) 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1161795305}) 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581657799}) 2024-08-19T10:42:47.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2705442945}) 2024-08-19T10:42:47.641 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.641 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581657799}) 2024-08-19T10:42:47.641 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.635+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4181568814}) 2024-08-19T10:42:47.644 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.644 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159922342}) 2024-08-19T10:42:47.644 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2142480251}) 2024-08-19T10:42:47.644 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.644 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159922342}) 2024-08-19T10:42:47.644 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1857129910}) 2024-08-19T10:42:47.644 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.644 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159922342}) 2024-08-19T10:42:47.644 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2392173000}) 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159922342}) 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1024847706}) 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159922342}) 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=104309935}) 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159922342}) 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=712650197}) 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159922342}) 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=393310096}) 2024-08-19T10:42:47.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159922342}) 2024-08-19T10:42:47.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.639+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1406579895}) 2024-08-19T10:42:47.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125395397}) 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2156892002}) 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125395397}) 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2957322858}) 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125395397}) 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=194352218}) 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125395397}) 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1286115279}) 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125395397}) 2024-08-19T10:42:47.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1531634896}) 2024-08-19T10:42:47.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125395397}) 2024-08-19T10:42:47.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608392136}) 2024-08-19T10:42:47.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125395397}) 2024-08-19T10:42:47.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1999812797}) 2024-08-19T10:42:47.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125395397}) 2024-08-19T10:42:47.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.651+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=713834873}) 2024-08-19T10:42:47.661 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.655+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.655+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2400368237}) 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.655+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2616057866}) 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.655+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.655+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2400368237}) 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4197395360}) 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2400368237}) 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3375253975}) 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2400368237}) 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1090455537}) 2024-08-19T10:42:47.662 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.663 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2400368237}) 2024-08-19T10:42:47.663 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4161435012}) 2024-08-19T10:42:47.663 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.663 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2400368237}) 2024-08-19T10:42:47.663 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3746573245}) 2024-08-19T10:42:47.663 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.663 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2400368237}) 2024-08-19T10:42:47.663 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2588470497}) 2024-08-19T10:42:47.663 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:47.663 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2400368237}) 2024-08-19T10:42:47.663 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:47.659+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1771025552}) 2024-08-19T10:42:48.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4227059890}) 2024-08-19T10:42:48.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3292266750}) 2024-08-19T10:42:48.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4227059890}) 2024-08-19T10:42:48.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2936044769}) 2024-08-19T10:42:48.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4227059890}) 2024-08-19T10:42:48.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3606575022}) 2024-08-19T10:42:48.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4227059890}) 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1792014760}) 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4227059890}) 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3142435234}) 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4227059890}) 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=564150480}) 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4227059890}) 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2315857990}) 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4227059890}) 2024-08-19T10:42:48.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.051+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4103335815}) 2024-08-19T10:42:48.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3700311376}) 2024-08-19T10:42:48.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3306076330}) 2024-08-19T10:42:48.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3700311376}) 2024-08-19T10:42:48.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2350112818}) 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3700311376}) 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2802307105}) 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3700311376}) 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2401068412}) 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3700311376}) 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2448764841}) 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3700311376}) 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3709381586}) 2024-08-19T10:42:48.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3700311376}) 2024-08-19T10:42:48.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1084719358}) 2024-08-19T10:42:48.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3700311376}) 2024-08-19T10:42:48.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.055+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2212907808}) 2024-08-19T10:42:48.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=601481303}) 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4007818044}) 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=601481303}) 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1404793200}) 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=601481303}) 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3086119289}) 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=601481303}) 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3265178573}) 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=601481303}) 2024-08-19T10:42:48.065 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2025937542}) 2024-08-19T10:42:48.065 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.065 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=601481303}) 2024-08-19T10:42:48.065 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1985773227}) 2024-08-19T10:42:48.065 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.065 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=601481303}) 2024-08-19T10:42:48.065 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1527802813}) 2024-08-19T10:42:48.065 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.065 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=601481303}) 2024-08-19T10:42:48.065 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.059+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=94150844}) 2024-08-19T10:42:48.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.067+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.067+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=26,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=4,osd_snap=2,osdmap=28} crc {logm=3781861400,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=1677515204,monmap=317198000,osd_metadata=3410341935,osd_snap=841190635,osdmap=3795326583}) 2024-08-19T10:42:48.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.067+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=29,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=17,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=22} crc {logm=640381445,mdsmap=2043744197,mgr=3114150001,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=2723323424,mon_config_key=2552794901,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4213060480,osd_snap=841190635,osdmap=3290128035}) 2024-08-19T10:42:48.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.067+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.067+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=26,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=4,osd_snap=2,osdmap=28} crc {logm=3781861400,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=1677515204,monmap=317198000,osd_metadata=3410341935,osd_snap=841190635,osdmap=3795326583}) 2024-08-19T10:42:48.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.067+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=28,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=16,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=24} crc {logm=3196910220,mdsmap=2043744197,mgr=3970944889,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=2708232688,monmap=317198000,osd_metadata=3182429268,osd_pg_creating=4213060480,osd_snap=841190635,osdmap=2472355021}) 2024-08-19T10:42:48.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.067+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=26,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=4,osd_snap=2,osdmap=28} crc {logm=3781861400,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=1677515204,monmap=317198000,osd_metadata=3410341935,osd_snap=841190635,osdmap=3795326583}) 2024-08-19T10:42:48.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=27,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=16,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1,osdmap=25} crc {logm=2929330228,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=4290588900,monmap=1845284827,osd_metadata=1520236208,osd_pg_creating=4213060480,osd_snap=3823785925,osdmap=1403644020}) 2024-08-19T10:42:48.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=26,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=4,osd_snap=2,osdmap=28} crc {logm=3781861400,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=1677515204,monmap=317198000,osd_metadata=3410341935,osd_snap=841190635,osdmap=3795326583}) 2024-08-19T10:42:48.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=27,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=17,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=23} crc {logm=2308441620,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=599686559,mon_config_key=2552794901,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4213060480,osd_snap=841190635,osdmap=240686946}) 2024-08-19T10:42:48.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=26,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=4,osd_snap=2,osdmap=28} crc {logm=3781861400,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=1677515204,monmap=317198000,osd_metadata=3410341935,osd_snap=841190635,osdmap=3795326583}) 2024-08-19T10:42:48.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=27,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=25} crc {logm=1092450880,mdsmap=2043744197,mgr=3997098071,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=1419624712,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4213060480,osd_snap=841190635,osdmap=4015954556}) 2024-08-19T10:42:48.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=26,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=4,osd_snap=2,osdmap=28} crc {logm=3781861400,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=1677515204,monmap=317198000,osd_metadata=3410341935,osd_snap=841190635,osdmap=3795326583}) 2024-08-19T10:42:48.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=29,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=23} crc {logm=640381445,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=3819633564,monmap=317198000,osd_metadata=2385667796,osd_pg_creating=4213060480,osd_snap=841190635,osdmap=240686946}) 2024-08-19T10:42:48.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=26,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=4,osd_snap=2,osdmap=28} crc {logm=3781861400,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=1677515204,monmap=317198000,osd_metadata=3410341935,osd_snap=841190635,osdmap=3795326583}) 2024-08-19T10:42:48.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=28,mdsmap=3,mgr=5,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=14,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=28} crc {logm=2075854679,mdsmap=2043744197,mgr=342737233,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=263578996,monmap=317198000,osd_metadata=3182429268,osd_pg_creating=4213060480,osd_snap=841190635,osdmap=3731093016}) 2024-08-19T10:42:48.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=26,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=4,osd_snap=2,osdmap=28} crc {logm=3781861400,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=971705281,mon_config_key=1677515204,monmap=317198000,osd_metadata=3410341935,osd_snap=841190635,osdmap=3795326583}) 2024-08-19T10:42:48.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.071+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=24,mdsmap=3,mgr=7,mgr_metadata=1,mgrstat=6,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=31} crc {logm=3248175305,mdsmap=2043744197,mgr=3114150001,mgr_metadata=536852559,mgrstat=1957708150,mon_config_key=1397469448,monmap=317198000,osd_metadata=3409533347,osd_pg_creating=4213060480,osd_snap=841190635,osdmap=1600598947}) 2024-08-19T10:42:48.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=504788140}) 2024-08-19T10:42:48.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3725304758}) 2024-08-19T10:42:48.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=504788140}) 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=614626422}) 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=504788140}) 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3911096854}) 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=504788140}) 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3148046961}) 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=504788140}) 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3327024339}) 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.075+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.079+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=504788140}) 2024-08-19T10:42:48.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.079+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=86691101}) 2024-08-19T10:42:48.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.079+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.079+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=504788140}) 2024-08-19T10:42:48.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.079+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=4098961503}) 2024-08-19T10:42:48.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.079+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.079+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=504788140}) 2024-08-19T10:42:48.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.079+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3473182040}) 2024-08-19T10:42:48.123 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.123 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4032069917}) 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3666271824}) 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4032069917}) 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2437988967}) 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4032069917}) 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3916360017}) 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4032069917}) 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4100279243}) 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4032069917}) 2024-08-19T10:42:48.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2895635616}) 2024-08-19T10:42:48.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4032069917}) 2024-08-19T10:42:48.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4255563365}) 2024-08-19T10:42:48.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4032069917}) 2024-08-19T10:42:48.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1461075612}) 2024-08-19T10:42:48.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4032069917}) 2024-08-19T10:42:48.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.119+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1231778993}) 2024-08-19T10:42:48.137 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.131+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.138 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.131+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=5} crc {osdmap=806801420}) 2024-08-19T10:42:48.138 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.131+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=8} crc {osdmap=1407444038}) 2024-08-19T10:42:48.138 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.131+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.138 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.131+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=5} crc {osdmap=806801420}) 2024-08-19T10:42:48.138 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.131+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=7} crc {osdmap=1510790709}) 2024-08-19T10:42:48.138 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.138 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=5} crc {osdmap=806801420}) 2024-08-19T10:42:48.138 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=8} crc {osdmap=432820350}) 2024-08-19T10:42:48.138 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.138 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=5} crc {osdmap=806801420}) 2024-08-19T10:42:48.138 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=8} crc {osdmap=432820350}) 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=5} crc {osdmap=806801420}) 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=8} crc {osdmap=432820350}) 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=5} crc {osdmap=806801420}) 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=8} crc {osdmap=432820350}) 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=5} crc {osdmap=806801420}) 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=7} crc {osdmap=3190284181}) 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=5} crc {osdmap=806801420}) 2024-08-19T10:42:48.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:42:48.135+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=7} crc {osdmap=1613701478}) 2024-08-19T10:42:48.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.155+0000 7f6d167fc640 1 -- 172.21.15.89:0/4088023399 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6d1c0160b0 con 0x7f6d200a3c70 2024-08-19T10:42:48.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.155+0000 7f6d167fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:48.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.155+0000 7f6d167fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.391+0000 7f6d27b7b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:42:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.391+0000 7f6d27b7b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:42:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.391+0000 7f6d27b7b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.391+0000 7f6d27b7b640 1 -- 172.21.15.89:0/4088023399 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6d2013f8d0 con 0x7f6d200a3c70 2024-08-19T10:42:48.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.391+0000 7f6d167fc640 1 -- 172.21.15.89:0/4088023399 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7f6d1c0aa7e0 con 0x7f6d200a3c70 2024-08-19T10:42:48.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.391+0000 7f6d167fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:42:48.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.391+0000 7f6d167fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:48.396 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":52,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:42:48.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 -- 172.21.15.89:0/4088023399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0403bdf0 msgr2=0x7f6d0403e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0403bdf0 0x7f6d0403e2b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6d2013d4d0 tx=0x7f6d10009190 comp rx=0 tx=0).stop 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 10 monclient: shutdown 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 -- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a3c70 msgr2=0x7f6d201d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a3c70 0x7f6d201d1360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0ae040 tx=0x7f6d1c019140 comp rx=0 tx=0).stop 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 -- 172.21.15.89:0/4088023399 shutdown_connections 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6d20139ea0 0x7f6d201d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a4590 0x7f6d201d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0403bdf0 0x7f6d0403e2b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 --2- 172.21.15.89:0/4088023399 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d200a3c70 0x7f6d201d1360 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 -- 172.21.15.89:0/4088023399 >> 172.21.15.89:0/4088023399 conn(0x7f6d2006de40 msgr2=0x7f6d2009ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:48.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 -- 172.21.15.89:0/4088023399 shutdown_connections 2024-08-19T10:42:48.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.395+0000 7f6d27b7b640 1 -- 172.21.15.89:0/4088023399 wait complete. 2024-08-19T10:42:48.415 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:42:48.416 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-08-19T10:42:48.416 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:42:48.416 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:42:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.492+0000 7f6ddd7fa640 1 -- 172.21.15.190:0/3856004454 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v154) ==== 149+0+0 (secure 0 0 0) 0x7f6dd001d410 con 0x7f6de00a3610 2024-08-19T10:42:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.492+0000 7f6ddd7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:42:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.492+0000 7f6ddd7fa640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-08-19T10:42:48.496 INFO:tasks.workunit.client.0.smithi190.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-08-19T10:42:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 -- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc003be00 msgr2=0x7f6dc003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc003be00 0x7f6dc003e2c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009bd0 tx=0x7f6dc800b040 comp rx=0 tx=0).stop 2024-08-19T10:42:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 10 monclient: shutdown 2024-08-19T10:42:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 -- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de00a3610 msgr2=0x7f6de0134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de00a3610 0x7f6de0134df0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000d830 tx=0x7f6dd000dcf0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 -- 172.21.15.190:0/3856004454 shutdown_connections 2024-08-19T10:42:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6dc003be00 0x7f6dc003e2c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de0136910 0x7f6de009ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6de00a3f30 0x7f6de0135330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 --2- 172.21.15.190:0/3856004454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6de00a3610 0x7f6de0134df0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 -- 172.21.15.190:0/3856004454 >> 172.21.15.190:0/3856004454 conn(0x7f6de00a4700 msgr2=0x7f6de006fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 -- 172.21.15.190:0/3856004454 shutdown_connections 2024-08-19T10:42:48.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.496+0000 7f6de6ef5640 1 -- 172.21.15.190:0/3856004454 wait complete. 2024-08-19T10:42:48.518 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight-compat osd.0 10.5 2024-08-19T10:42:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.527+0000 7f283c2ce640 10 monclient: get_monmap_and_config 2024-08-19T10:42:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.527+0000 7f283c2ce640 10 monclient: build_initial_monmap 2024-08-19T10:42:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.527+0000 7f283c2ce640 10 monclient: monmap: 2024-08-19T10:42:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:48.532660+0000 2024-08-19T10:42:48.532 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:48.532660+0000 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:48.533 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 1 Processor -- start 2024-08-19T10:42:48.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 1 -- start start 2024-08-19T10:42:48.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 10 monclient: init 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 10 monclient: _add_conns ranks=[7,8,0,3,4,1,2,10,5,9,6] 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f28340a3d00 0x7f283409eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 10 monclient(hunting): picked mon.noname-i con 0x7f28340a3d00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f28340a40e0 0x7f28340a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 10 monclient(hunting): picked mon.noname-e con 0x7f28340a40e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2834136440 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f283406db50 con 0x7f2834136440 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f283409e870 con 0x7f28340a3d00 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f283409eac0 con 0x7f28340a40e0 2024-08-19T10:42:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283c2ce640 10 monclient(hunting): authenticate will time out at 1218.349512s 2024-08-19T10:42:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f28340a40e0 0x7f28340a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283a86b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2834136440 0x7f2834136880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52928/0 (socket says 172.21.15.89:52928) 2024-08-19T10:42:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283a86b640 1 -- 172.21.15.89:0/3666088869 learned_addr learned my addr 172.21.15.89:0/3666088869 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:42:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 10 monclient(hunting): get_auth_request con 0x7f28340a40e0 auth_method 0 2024-08-19T10:42:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283a06a640 1 --2- 172.21.15.89:0/3666088869 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f28340a3d00 0x7f283409eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283a06a640 10 monclient(hunting): get_auth_request con 0x7f28340a3d00 auth_method 0 2024-08-19T10:42:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283a06a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283a06a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f283a06a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 10 monclient(hunting): handle_auth_done global_id 5498 payload 274 2024-08-19T10:42:48.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 1 -- 172.21.15.89:0/3666088869 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f28340a3d00 msgr2=0x7f283409eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:48.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 1 --2- 172.21.15.89:0/3666088869 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f28340a3d00 0x7f283409eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 1 -- 172.21.15.89:0/3666088869 >> v1:172.21.15.89:6789/0 conn(0x7f2834136440 legacy=0x7f2834136880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:48.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 10 monclient: _finish_hunting 0 2024-08-19T10:42:48.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 1 monclient: found mon.noname-e 2024-08-19T10:42:48.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:48.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:48.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.531+0000 7f2839869640 1 -- 172.21.15.89:0/3666088869 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283406fd00 con 0x7f28340a40e0 2024-08-19T10:42:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839869640 1 -- 172.21.15.89:0/3666088869 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f28340a40e0 msgr2=0x7f28340a44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839869640 1 --2- 172.21.15.89:0/3666088869 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f28340a40e0 0x7f28340a44c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f28280097f0 tx=0x7f282801c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=8bfa58c191cc918 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 1 -- 172.21.15.89:0/3666088869 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f28280b0070 con 0x7f28340a40e0 2024-08-19T10:42:48.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 10 monclient: dump: 2024-08-19T10:42:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.551 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.552 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.553 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 10 monclient: _finish_auth 0 2024-08-19T10:42:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 10 monclient: _check_auth_tickets 2024-08-19T10:42:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 1 -- 172.21.15.89:0/3666088869 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28280aa810 con 0x7f28340a40e0 2024-08-19T10:42:48.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 5 monclient: authenticate success, global_id 5498 2024-08-19T10:42:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 1 -- 172.21.15.89:0/3666088869 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f28280a4e60 con 0x7f28340a40e0 2024-08-19T10:42:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 4 set_mon_vals no callback set 2024-08-19T10:42:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f2839068640 10 monclient: dump: 2024-08-19T10:42:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.554 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.555 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 10 monclient: shutdown 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 1 -- 172.21.15.89:0/3666088869 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f28340a40e0 msgr2=0x7f28340a44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 1 --2- 172.21.15.89:0/3666088869 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f28340a40e0 0x7f28340a44c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f28280097f0 tx=0x7f282801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 1 -- 172.21.15.89:0/3666088869 shutdown_connections 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 1 --2- 172.21.15.89:0/3666088869 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f28340a40e0 0x7f28340a44c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 1 --2- 172.21.15.89:0/3666088869 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f28340a3d00 0x7f283409eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 1 -- 172.21.15.89:0/3666088869 >> 172.21.15.89:0/3666088869 conn(0x7f28340a4df0 msgr2=0x7f28340a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 1 -- 172.21.15.89:0/3666088869 shutdown_connections 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.535+0000 7f283c2ce640 1 -- 172.21.15.89:0/3666088869 wait complete. 2024-08-19T10:42:48.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient: build_initial_monmap 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient: monmap: 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:48.557 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:48.558 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 1 Processor -- start 2024-08-19T10:42:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 1 -- start start 2024-08-19T10:42:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient: init 2024-08-19T10:42:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient: _add_conns ranks=[5,0,8,7,4,2,1,3,6] 2024-08-19T10:42:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f28340a3d00 0x7f2834135d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient(hunting): picked mon.noname-h con 0x7f28340a3d00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:48.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340a40e0 0x7f283413cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient(hunting): picked mon.noname-a con 0x7f28340a40e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:48.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2834136440 0x7f2834070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient(hunting): picked mon.noname-e con 0x7f2834136440 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:48.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2834141040 con 0x7f28340a40e0 2024-08-19T10:42:48.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2834140ec0 con 0x7f28340a3d00 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f28341411c0 con 0x7f2834136440 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283c2ce640 10 monclient(hunting): authenticate will time out at 1218.357284s 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340a40e0 0x7f283413cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a86b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2834136440 0x7f2834070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340a40e0 0x7f283413cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37642/0 (socket says 172.21.15.89:37642) 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 1 -- 172.21.15.89:0/4223423472 learned_addr learned my addr 172.21.15.89:0/4223423472 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 10 monclient(hunting): get_auth_request con 0x7f28340a40e0 auth_method 0 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a86b640 10 monclient(hunting): get_auth_request con 0x7f2834136440 auth_method 0 2024-08-19T10:42:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a86b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a86b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a86b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a86b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a86b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a86b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a06a640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f28340a3d00 0x7f2834135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a06a640 10 monclient(hunting): get_auth_request con 0x7f28340a3d00 auth_method 0 2024-08-19T10:42:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a06a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a06a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f283a06a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 10 monclient(hunting): handle_auth_done global_id 5715 payload 824 2024-08-19T10:42:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 1 -- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2834136440 msgr2=0x7f2834070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2834136440 0x7f2834070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 1 -- 172.21.15.89:0/4223423472 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f28340a3d00 msgr2=0x7f2834135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f28340a3d00 0x7f2834135d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 10 monclient: _finish_hunting 0 2024-08-19T10:42:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 1 monclient: found mon.noname-a 2024-08-19T10:42:48.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.539+0000 7f2839869640 1 -- 172.21.15.89:0/4223423472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2834072770 con 0x7f28340a40e0 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2839869640 1 -- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340a40e0 msgr2=0x7f283413cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2839869640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340a40e0 0x7f283413cb20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f282801c1d0 tx=0x7f2828019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f28280b0040 con 0x7f28340a40e0 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: dump: 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.563 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: _finish_auth 0 2024-08-19T10:42:48.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28280b9070 con 0x7f28340a40e0 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283c2ce640 5 monclient: authenticate success, global_id 5715 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283c2ce640 10 monclient: _renew_subs 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283c2ce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283c2ce640 1 -- 172.21.15.89:0/4223423472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28340a45e0 con 0x7f28340a40e0 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f28280044b0 con 0x7f28340a40e0 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: dump: 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.565 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.566 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283c2ce640 10 monclient: _renew_subs 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283c2ce640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2833fff640 4 set_mon_vals no callback set 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283c2ce640 1 -- 172.21.15.89:0/4223423472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283413ac00 con 0x7f28340a40e0 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283c2ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283c2ce640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340712b0 0x7f2834071690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283c2ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283c2ce640 1 -- 172.21.15.89:0/4223423472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2834071bd0 con 0x7f28340712b0 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340712b0 0x7f2834071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 10 monclient: get_auth_request con 0x7f28340712b0 auth_method 0 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2828004040 con 0x7f28340a40e0 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 10 monclient(hunting): handle_auth_done global_id 5724 payload 824 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 1 -- 172.21.15.89:0/4223423472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f283409eee0 con 0x7f28340712b0 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f282003be00 0x7f282003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9079+0+0 (secure 0 0 0) 0x7f28280e9680 con 0x7f28340a40e0 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a86b640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f282003be00 0x7f282003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a86b640 10 monclient: get_auth_request con 0x7f282003be00 auth_method 0 2024-08-19T10:42:48.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 1 -- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340712b0 msgr2=0x7f2834071690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f283a06a640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340712b0 0x7f2834071690 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f282c00f4b0 tx=0x7f282c00f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f282c019020 con 0x7f28340712b0 2024-08-19T10:42:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.543+0000 7f2832ffd640 10 monclient: dump: 2024-08-19T10:42:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.569 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.570 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.570 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.547+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f282c014880 con 0x7f28340712b0 2024-08-19T10:42:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.547+0000 7f2832ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.547+0000 7f2832ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.547+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340712b0 msgr2=0x7f2834071690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.547+0000 7f2832ffd640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340712b0 0x7f2834071690 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f282c00f4b0 tx=0x7f282c00f970 comp rx=0 tx=0).stop 2024-08-19T10:42:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.547+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340712b0 msgr2=0x7f2834071690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.547+0000 7f2832ffd640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340712b0 0x7f2834071690 secure :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f282c00f4b0 tx=0x7f282c00f970 comp rx=0 tx=0).stop 2024-08-19T10:42:48.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.547+0000 7f283a86b640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f282003be00 0x7f282003e2c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f283413b3a0 tx=0x7f2824009b10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283c2ce640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:48.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283c2ce640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409a240 0x7f283409a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283c2ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283c2ce640 1 -- 172.21.15.89:0/4223423472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2834064590 con 0x7f283409a240 2024-08-19T10:42:48.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283a06a640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409a240 0x7f283409a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283a06a640 10 monclient: get_auth_request con 0x7f283409a240 auth_method 0 2024-08-19T10:42:48.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283a06a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283a06a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283a06a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283a06a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283a06a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283a06a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283a06a640 10 monclient(hunting): handle_auth_done global_id 5733 payload 824 2024-08-19T10:42:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.551+0000 7f283a06a640 1 -- 172.21.15.89:0/4223423472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f283406e560 con 0x7f283409a240 2024-08-19T10:42:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283a06a640 1 -- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409a240 msgr2=0x7f283409a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283a06a640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409a240 0x7f283409a650 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f282c00f4b0 tx=0x7f282c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f282c019040 con 0x7f283409a240 2024-08-19T10:42:48.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f2832ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f2832ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f2832ffd640 10 monclient: dump: 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.576 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9269 (secure 0 0 0) 0x7f282c004030 con 0x7f283409a240 2024-08-19T10:42:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f2832ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f2832ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409a240 msgr2=0x7f283409a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f2832ffd640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409a240 0x7f283409a650 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f282c00f4b0 tx=0x7f282c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f2832ffd640 1 -- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409a240 msgr2=0x7f283409a650 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f2832ffd640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f283409a240 0x7f283409a650 secure :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f282c00f4b0 tx=0x7f282c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.577 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:48.577 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:48.577 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:48.577 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:42:48.577 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:48.577 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 298368, 2024-08-19T10:42:48.577 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:48.577 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:48.577 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:48.577 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:48.577 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:48.578 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:48.578 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:48.578 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:48.578 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:48.578 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:48.578 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.578 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 8, 2024-08-19T10:42:48.578 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:48.582 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:48.583 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:48.584 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:48.585 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:48.586 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:48.587 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:48.587 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:48.587 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:48.587 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:48.587 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:48.587 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:48.587 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:48.587 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.587 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:48.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.588 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:48.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.588 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:48.588 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:48.588 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:48.589 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:48.590 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.591 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.593 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:48.593 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.593 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.593 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.593 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:48.593 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.594 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.595 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.596 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.597 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.598 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:48.601 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:48.601 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.601 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.601 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.601 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:48.601 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:48.601 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:48.603 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:48.603 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.603 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.603 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.604 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.605 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:42:48.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.607 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.607 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:48.607 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:48.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283c2ce640 1 -- 172.21.15.89:0/4223423472 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f282003be00 msgr2=0x7f282003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283c2ce640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f282003be00 0x7f282003e2c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f283413b3a0 tx=0x7f2824009b10 comp rx=0 tx=0).stop 2024-08-19T10:42:48.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283c2ce640 10 monclient: shutdown 2024-08-19T10:42:48.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283c2ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:48.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283c2ce640 1 -- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340a40e0 msgr2=0x7f283413cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283a86b640 1 -- 172.21.15.89:0/4223423472 reap_dead start 2024-08-19T10:42:48.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283c2ce640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340a40e0 0x7f283413cb20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f282801c1d0 tx=0x7f2828019140 comp rx=0 tx=0).stop 2024-08-19T10:42:48.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283c2ce640 1 -- 172.21.15.89:0/4223423472 shutdown_connections 2024-08-19T10:42:48.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283c2ce640 1 --2- 172.21.15.89:0/4223423472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f28340a40e0 0x7f283413cb20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283c2ce640 1 -- 172.21.15.89:0/4223423472 >> 172.21.15.89:0/4223423472 conn(0x7f28340a4df0 msgr2=0x7f283409c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:48.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283c2ce640 1 -- 172.21.15.89:0/4223423472 shutdown_connections 2024-08-19T10:42:48.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.555+0000 7f283c2ce640 1 -- 172.21.15.89:0/4223423472 wait complete. 2024-08-19T10:42:48.609 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:48.609 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:42:48.610 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:42:48.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient: get_monmap_and_config 2024-08-19T10:42:48.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient: build_initial_monmap 2024-08-19T10:42:48.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient: monmap: 2024-08-19T10:42:48.640 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:48.640 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.640 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:42:48.640352+0000 2024-08-19T10:42:48.641 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:42:48.640352+0000 2024-08-19T10:42:48.641 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:48.641 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:48.641 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:48.641 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:48.641 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:48.642 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:48.642 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:48.642 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:48.642 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:48.642 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:48.642 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:48.642 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:48.642 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:48.642 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:48.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 1 Processor -- start 2024-08-19T10:42:48.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 1 -- start start 2024-08-19T10:42:48.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient: init 2024-08-19T10:42:48.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:48.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient: _add_conns ranks=[3,9,5,4,0,1,6,8,2,7,10] 2024-08-19T10:42:48.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf200a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient(hunting): picked mon.noname-g con 0x7faf200a4720 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:48.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faf200a5040 0x7faf201398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient(hunting): picked mon.noname-a con 0x7faf200a5040 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faf20139e10 0x7faf2013a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient(hunting): picked mon.noname-h con 0x7faf20139e10 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7faf200a1400 con 0x7faf200a5040 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faf2006d280 con 0x7faf200a4720 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7faf2006cbc0 con 0x7faf20139e10 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf25860640 10 monclient(hunting): authenticate will time out at 1214.211057s 2024-08-19T10:42:48.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf200a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faf20139e10 0x7faf2013a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf200a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:56980/0 (socket says 172.21.15.190:56980) 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faf20139e10 0x7faf2013a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:51844/0 (socket says 172.21.15.190:51844) 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1effd640 1 -- 172.21.15.190:0/349270379 learned_addr learned my addr 172.21.15.190:0/349270379 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1effd640 10 monclient(hunting): get_auth_request con 0x7faf200a4720 auth_method 0 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 10 monclient(hunting): get_auth_request con 0x7faf20139e10 auth_method 0 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1e7fc640 1 --2- 172.21.15.190:0/349270379 >> v2:172.21.15.89:3300/0 conn(0x7faf200a5040 0x7faf201398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1e7fc640 10 monclient(hunting): get_auth_request con 0x7faf200a5040 auth_method 0 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 10 monclient(hunting): handle_auth_done global_id 5720 payload 274 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 1 -- 172.21.15.190:0/349270379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 msgr2=0x7faf200a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:48.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 1 --2- 172.21.15.190:0/349270379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf200a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 1 -- 172.21.15.190:0/349270379 >> v2:172.21.15.89:3300/0 conn(0x7faf200a5040 msgr2=0x7faf201398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:48.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 1 --2- 172.21.15.190:0/349270379 >> v2:172.21.15.89:3300/0 conn(0x7faf200a5040 0x7faf201398d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:48.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 1 monclient: found mon.noname-h 2024-08-19T10:42:48.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:48.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:48.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1f7fe640 1 -- 172.21.15.190:0/349270379 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf200a0b60 con 0x7faf20139e10 2024-08-19T10:42:48.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1effd640 1 --2- 172.21.15.190:0/349270379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf200a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:48.649 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.636+0000 7faf1e7fc640 1 --2- 172.21.15.190:0/349270379 >> v2:172.21.15.89:3300/0 conn(0x7faf200a5040 0x7faf201398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:42:48.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.640+0000 7faf1f7fe640 1 -- 172.21.15.190:0/349270379 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faf20139e10 msgr2=0x7faf2013a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.640+0000 7faf1f7fe640 1 --2- 172.21.15.190:0/349270379 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faf20139e10 0x7faf2013a270 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faf18009d90 tx=0x7faf1801d290 comp rx=0 tx=0).ready entity=mon.5 client_cookie=978efeb07bac5786 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 1 -- 172.21.15.190:0/349270379 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faf180b2070 con 0x7faf20139e10 2024-08-19T10:42:48.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:48.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 10 monclient: dump: 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.654 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 1 -- 172.21.15.190:0/349270379 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf180a59f0 con 0x7faf20139e10 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 5 monclient: authenticate success, global_id 5720 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 1 -- 172.21.15.190:0/349270379 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faf180a5ba0 con 0x7faf20139e10 2024-08-19T10:42:48.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:48.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 4 set_mon_vals no callback set 2024-08-19T10:42:48.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:48.656 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf1dffb640 10 monclient: dump: 2024-08-19T10:42:48.656 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:48.656 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.656 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.656 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.656 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.656 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 10 monclient: shutdown 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:48.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 1 -- 172.21.15.190:0/349270379 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faf20139e10 msgr2=0x7faf2013a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 1 --2- 172.21.15.190:0/349270379 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faf20139e10 0x7faf2013a270 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faf18009d90 tx=0x7faf1801d290 comp rx=0 tx=0).stop 2024-08-19T10:42:48.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 1 -- 172.21.15.190:0/349270379 shutdown_connections 2024-08-19T10:42:48.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 1 --2- 172.21.15.190:0/349270379 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faf20139e10 0x7faf2013a270 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 1 --2- 172.21.15.190:0/349270379 >> v2:172.21.15.89:3300/0 conn(0x7faf200a5040 0x7faf201398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 1 --2- 172.21.15.190:0/349270379 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf200a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 1 -- 172.21.15.190:0/349270379 >> 172.21.15.190:0/349270379 conn(0x7faf200a9310 msgr2=0x7faf200a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:48.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 1 -- 172.21.15.190:0/349270379 shutdown_connections 2024-08-19T10:42:48.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 1 -- 172.21.15.190:0/349270379 wait complete. 2024-08-19T10:42:48.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 10 monclient: build_initial_monmap 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 10 monclient: monmap: 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:48.659 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 1 Processor -- start 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 1 -- start start 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.644+0000 7faf25860640 10 monclient: init 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient: _add_conns ranks=[3,1,4,0,5,8,2,7,6] 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf2013d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient(hunting): picked mon.noname-g con 0x7faf200a4720 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf200a5040 0x7faf2013d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient(hunting): picked mon.noname-f con 0x7faf200a5040 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:48.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf20139e10 0x7faf2013de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient(hunting): picked mon.noname-c con 0x7faf20139e10 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faf2006cf50 con 0x7faf200a5040 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faf200a1de0 con 0x7faf200a4720 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7faf200a5420 con 0x7faf20139e10 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient(hunting): authenticate will time out at 1214.220476s 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf2013d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf2013d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:56992/0 (socket says 172.21.15.190:56992) 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 1 -- 172.21.15.190:0/3653778388 learned_addr learned my addr 172.21.15.190:0/3653778388 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:42:48.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 10 monclient(hunting): get_auth_request con 0x7faf200a4720 auth_method 0 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1e7fc640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf200a5040 0x7faf2013d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1e7fc640 10 monclient(hunting): get_auth_request con 0x7faf200a5040 auth_method 0 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1f7fe640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf20139e10 0x7faf2013de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1f7fe640 10 monclient(hunting): get_auth_request con 0x7faf20139e10 auth_method 0 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 10 monclient(hunting): handle_auth_done global_id 34140 payload 824 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 1 -- 172.21.15.190:0/3653778388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf20139e10 msgr2=0x7faf2013de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf20139e10 0x7faf2013de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 1 -- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf200a5040 msgr2=0x7faf2013d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf200a5040 0x7faf2013d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 10 monclient: _finish_hunting 0 2024-08-19T10:42:48.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 1 monclient: found mon.noname-g 2024-08-19T10:42:48.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:48.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:42:48.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 1 -- 172.21.15.190:0/3653778388 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf20140950 con 0x7faf200a4720 2024-08-19T10:42:48.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 1 -- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 msgr2=0x7faf2013d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1effd640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf2013d3d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf10009fd0 tx=0x7faf1000ee10 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 1 -- 172.21.15.190:0/3653778388 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faf100044a0 con 0x7faf200a4720 2024-08-19T10:42:48.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:42:48.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 10 monclient: dump: 2024-08-19T10:42:48.667 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:48.667 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.667 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.668 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 1 -- 172.21.15.190:0/3653778388 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf1000cbb0 con 0x7faf200a4720 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 1 -- 172.21.15.190:0/3653778388 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faf1000cd60 con 0x7faf200a4720 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf07fff640 10 monclient: dump: 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.669 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.670 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf1d7fa640 4 set_mon_vals no callback set 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 5 monclient: authenticate success, global_id 34140 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient: _renew_subs 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.648+0000 7faf25860640 1 -- 172.21.15.190:0/3653778388 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf200a8930 con 0x7faf200a4720 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf25860640 10 monclient: _renew_subs 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf25860640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf25860640 1 -- 172.21.15.190:0/3653778388 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faf20123b60 con 0x7faf200a4720 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf25860640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf25860640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf25860640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf25860640 1 -- 172.21.15.190:0/3653778388 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf200008e0 con 0x7faf200a4720 2024-08-19T10:42:48.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf07fff640 1 -- 172.21.15.190:0/3653778388 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7faf10004030 con 0x7faf200a4720 2024-08-19T10:42:48.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf07fff640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7faf0c03be00 0x7faf0c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf07fff640 1 -- 172.21.15.190:0/3653778388 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(154..154 src has 1..154) ==== 9079+0+0 (secure 0 0 0) 0x7faf1000f040 con 0x7faf200a4720 2024-08-19T10:42:48.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf1e7fc640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7faf0c03be00 0x7faf0c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf1e7fc640 10 monclient: get_auth_request con 0x7faf0c03be00 auth_method 0 2024-08-19T10:42:48.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.652+0000 7faf1e7fc640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7faf0c03be00 0x7faf0c03e2c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faf14006e90 tx=0x7faf14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.656+0000 7faf07fff640 1 -- 172.21.15.190:0/3653778388 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7faf200a8930 con 0x7faf200a4720 2024-08-19T10:42:48.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.656+0000 7faf07fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:48.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.656+0000 7faf07fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:48.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient: get_monmap_and_config 2024-08-19T10:42:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient: build_initial_monmap 2024-08-19T10:42:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient: monmap: 2024-08-19T10:42:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.726 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:48.726008+0000 2024-08-19T10:42:48.727 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:48.726008+0000 2024-08-19T10:42:48.727 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:48.727 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.727 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:48.727 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:48.727 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:48.728 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:48.729 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:48.729 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:48.729 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 1 Processor -- start 2024-08-19T10:42:48.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 1 -- start start 2024-08-19T10:42:48.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient: init 2024-08-19T10:42:48.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:48.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient: _add_conns ranks=[4,8,1,10,3,7,2,5,9,0,6] 2024-08-19T10:42:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 0x7fd2f00a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient(hunting): picked mon.noname-c con 0x7fd2f00a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2f00a6d30 0x7fd2f00a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient(hunting): picked mon.noname-e con 0x7fd2f00a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd2f01395b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2f00a8710 con 0x7fd2f01395b0 2024-08-19T10:42:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd2f00a7650 con 0x7fd2f00a5270 2024-08-19T10:42:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd2f00a5a40 con 0x7fd2f00a6d30 2024-08-19T10:42:48.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f854d640 10 monclient(hunting): authenticate will time out at 1218.542567s 2024-08-19T10:42:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 0x7fd2f00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f5ae8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2f00a6d30 0x7fd2f00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 0x7fd2f00a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38410/0 (socket says 172.21.15.89:38410) 2024-08-19T10:42:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f5ae8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2f00a6d30 0x7fd2f00a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40474/0 (socket says 172.21.15.89:40474) 2024-08-19T10:42:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 1 -- 172.21.15.89:0/2466932525 learned_addr learned my addr 172.21.15.89:0/2466932525 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 10 monclient(hunting): get_auth_request con 0x7fd2f00a5270 auth_method 0 2024-08-19T10:42:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f5ae8640 10 monclient(hunting): get_auth_request con 0x7fd2f00a6d30 auth_method 0 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f5ae8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f5ae8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f5ae8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f5ae8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f5ae8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f5ae8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 10 monclient(hunting): handle_auth_done global_id 5656 payload 274 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 1 -- 172.21.15.89:0/2466932525 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2f00a6d30 msgr2=0x7fd2f00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 1 --2- 172.21.15.89:0/2466932525 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2f00a6d30 0x7fd2f00a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 1 -- 172.21.15.89:0/2466932525 >> v1:172.21.15.190:6789/0 conn(0x7fd2f01395b0 legacy=0x7fd2f0139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:48.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 10 monclient: _finish_hunting 0 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 1 monclient: found mon.noname-c 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f62e9640 1 -- 172.21.15.89:0/2466932525 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f00a81d0 con 0x7fd2f00a5270 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.723+0000 7fd2f5ae8640 1 --2- 172.21.15.89:0/2466932525 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2f00a6d30 0x7fd2f00a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.727+0000 7fd2f62e9640 1 -- 172.21.15.89:0/2466932525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 msgr2=0x7fd2f00a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.727+0000 7fd2f62e9640 1 --2- 172.21.15.89:0/2466932525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 0x7fd2f00a67f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009e80 tx=0x7fd2e801c730 comp rx=0 tx=0).ready entity=mon.4 client_cookie=6afd7d5ea472e5f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.727+0000 7fd2f52e7640 1 -- 172.21.15.89:0/2466932525 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2e80090d0 con 0x7fd2f00a5270 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.727+0000 7fd2f52e7640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.727+0000 7fd2f52e7640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.727+0000 7fd2f52e7640 10 monclient: dump: 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.744 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.745 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f52e7640 10 monclient: _finish_auth 0 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f52e7640 10 monclient: _check_auth_tickets 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f52e7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 5 monclient: authenticate success, global_id 5656 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f52e7640 1 -- 172.21.15.89:0/2466932525 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e80a5980 con 0x7fd2f00a5270 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f52e7640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 4 set_mon_vals no callback set 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f52e7640 1 -- 172.21.15.89:0/2466932525 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2e80ad3f0 con 0x7fd2f00a5270 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f52e7640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f52e7640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:48.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f52e7640 10 monclient: dump: 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.747 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient: shutdown 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 -- 172.21.15.89:0/2466932525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 msgr2=0x7fd2f00a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 --2- 172.21.15.89:0/2466932525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 0x7fd2f00a67f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009e80 tx=0x7fd2e801c730 comp rx=0 tx=0).stop 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 -- 172.21.15.89:0/2466932525 shutdown_connections 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 --2- 172.21.15.89:0/2466932525 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2f00a6d30 0x7fd2f00a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 --2- 172.21.15.89:0/2466932525 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 0x7fd2f00a67f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 -- 172.21.15.89:0/2466932525 >> 172.21.15.89:0/2466932525 conn(0x7fd2f0069a80 msgr2=0x7fd2f00abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 -- 172.21.15.89:0/2466932525 shutdown_connections 2024-08-19T10:42:48.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 -- 172.21.15.89:0/2466932525 wait complete. 2024-08-19T10:42:48.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient: build_initial_monmap 2024-08-19T10:42:48.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient: monmap: 2024-08-19T10:42:48.749 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:48.749 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:48.749 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:48.749 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:48.750 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:48.750 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.750 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:48.750 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:48.750 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:48.750 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:48.750 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 Processor -- start 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 -- start start 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient: init 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient: _add_conns ranks=[4,7,5,3,0,8,6,1,2] 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 0x7fd2f013fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient(hunting): picked mon.noname-c con 0x7fd2f00a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2f00a6d30 0x7fd2f01d1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient(hunting): picked mon.noname-i con 0x7fd2f00a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2f01395b0 0x7fd2f013d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient(hunting): picked mon.noname-h con 0x7fd2f01395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd2f00a57d0 con 0x7fd2f00a5270 2024-08-19T10:42:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd2f00a5650 con 0x7fd2f01395b0 2024-08-19T10:42:48.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd2f0144240 con 0x7fd2f00a6d30 2024-08-19T10:42:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f854d640 10 monclient(hunting): authenticate will time out at 1218.550298s 2024-08-19T10:42:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 0x7fd2f013fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 0x7fd2f013fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38424/0 (socket says 172.21.15.89:38424) 2024-08-19T10:42:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 1 -- 172.21.15.89:0/1782365559 learned_addr learned my addr 172.21.15.89:0/1782365559 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 10 monclient(hunting): get_auth_request con 0x7fd2f00a5270 auth_method 0 2024-08-19T10:42:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f5ae8640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2f00a6d30 0x7fd2f01d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 10 monclient(hunting): handle_auth_done global_id 5665 payload 824 2024-08-19T10:42:48.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f6aea640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2f01395b0 0x7fd2f013d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 1 -- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2f00a6d30 msgr2=0x7fd2f01d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2f00a6d30 0x7fd2f01d1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 1 -- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2f01395b0 msgr2=0x7fd2f013d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2f01395b0 0x7fd2f013d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 10 monclient: _finish_hunting 0 2024-08-19T10:42:48.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 1 monclient: found mon.noname-c 2024-08-19T10:42:48.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:48.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:48.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f62e9640 1 -- 172.21.15.89:0/1782365559 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f00aadc0 con 0x7fd2f00a5270 2024-08-19T10:42:48.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f5ae8640 10 monclient: get_auth_request con 0x7fd2f00a6d30 auth_method 0 2024-08-19T10:42:48.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.731+0000 7fd2f5ae8640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2f00a6d30 0x7fd2f01d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:48.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f62e9640 1 -- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 msgr2=0x7fd2f013fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f62e9640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 0x7fd2f013fba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009e20 tx=0x7fd2e80194d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2e8003bb0 con 0x7fd2f00a5270 2024-08-19T10:42:48.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:48.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 10 monclient: dump: 2024-08-19T10:42:48.757 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.757 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.757 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.757 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.757 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.757 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.758 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.758 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.758 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.758 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.758 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.758 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.758 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.758 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.759 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.759 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:42:48.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:48.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:48.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e8003dd0 con 0x7fd2f00a5270 2024-08-19T10:42:48.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:48.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2e80b6480 con 0x7fd2f00a5270 2024-08-19T10:42:48.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f854d640 5 monclient: authenticate success, global_id 5665 2024-08-19T10:42:48.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f854d640 10 monclient: _renew_subs 2024-08-19T10:42:48.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f854d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:48.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f854d640 1 -- 172.21.15.89:0/1782365559 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f013f040 con 0x7fd2f00a5270 2024-08-19T10:42:48.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:48.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 10 monclient: dump: 2024-08-19T10:42:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.761 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.762 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.762 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.762 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.762 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.762 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.762 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.762 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.762 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f4ae6640 4 set_mon_vals no callback set 2024-08-19T10:42:48.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f854d640 10 monclient: _renew_subs 2024-08-19T10:42:48.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f854d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:48.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f854d640 1 -- 172.21.15.89:0/1782365559 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f0122500 con 0x7fd2f00a5270 2024-08-19T10:42:48.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f854d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:48.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f854d640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f01d1d80 0x7fd2f013dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f854d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f854d640 1 -- 172.21.15.89:0/1782365559 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd2f0122970 con 0x7fd2f01d1d80 2024-08-19T10:42:48.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f5ae8640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f01d1d80 0x7fd2f013dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f5ae8640 10 monclient: get_auth_request con 0x7fd2f01d1d80 auth_method 0 2024-08-19T10:42:48.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f5ae8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f5ae8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f5ae8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd2e80be030 con 0x7fd2f00a5270 2024-08-19T10:42:48.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f5ae8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f5ae8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f5ae8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f5ae8640 10 monclient(hunting): handle_auth_done global_id 5654 payload 824 2024-08-19T10:42:48.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f5ae8640 1 -- 172.21.15.89:0/1782365559 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2f00a67f0 con 0x7fd2f01d1d80 2024-08-19T10:42:48.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd2dc03be00 0x7fd2dc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(154..154 src has 1..154) ==== 9079+0+0 (secure 0 0 0) 0x7fd2e80bc020 con 0x7fd2f00a5270 2024-08-19T10:42:48.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f6aea640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd2dc03be00 0x7fd2dc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.735+0000 7fd2f6aea640 10 monclient: get_auth_request con 0x7fd2dc03be00 auth_method 0 2024-08-19T10:42:48.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2f5ae8640 1 -- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f01d1d80 msgr2=0x7fd2f013dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2f5ae8640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f01d1d80 0x7fd2f013dad0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2f013ecb0 tx=0x7fd2ec010c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2ec00f040 con 0x7fd2f01d1d80 2024-08-19T10:42:48.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2e6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2e6ffd640 10 monclient: dump: 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.769 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd2ec01c400 con 0x7fd2f01d1d80 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2e6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2e6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f01d1d80 msgr2=0x7fd2f013dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2e6ffd640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f01d1d80 0x7fd2f013dad0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2f013ecb0 tx=0x7fd2ec010c20 comp rx=0 tx=0).stop 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f01d1d80 msgr2=0x7fd2f013dad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2e6ffd640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f01d1d80 0x7fd2f013dad0 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2f013ecb0 tx=0x7fd2ec010c20 comp rx=0 tx=0).stop 2024-08-19T10:42:48.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.739+0000 7fd2f6aea640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd2dc03be00 0x7fd2dc03e2c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0003ce0 tx=0x7fd2e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f854d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f854d640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f013e010 0x7fd2f009fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f854d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f854d640 1 -- 172.21.15.89:0/1782365559 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd2f013e420 con 0x7fd2f013e010 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f5ae8640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f013e010 0x7fd2f009fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f5ae8640 10 monclient: get_auth_request con 0x7fd2f013e010 auth_method 0 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f5ae8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f5ae8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f5ae8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f5ae8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f5ae8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f5ae8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f5ae8640 10 monclient(hunting): handle_auth_done global_id 5663 payload 824 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.743+0000 7fd2f5ae8640 1 -- 172.21.15.89:0/1782365559 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd2f0064590 con 0x7fd2f013e010 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f5ae8640 1 -- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f013e010 msgr2=0x7fd2f009fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f5ae8640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f013e010 0x7fd2f009fe20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec01b000 tx=0x7fd2ec0046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2ec004490 con 0x7fd2f013e010 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2e6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2e6ffd640 10 monclient: dump: 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.772 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7fd2ec025070 con 0x7fd2f013e010 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2e6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2e6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:48.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f013e010 msgr2=0x7fd2f009fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2e6ffd640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f013e010 0x7fd2f009fe20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec01b000 tx=0x7fd2ec0046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2e6ffd640 1 -- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f013e010 msgr2=0x7fd2f009fe20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:48.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2e6ffd640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2f013e010 0x7fd2f009fe20 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec01b000 tx=0x7fd2ec0046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.774 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:48.774 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:48.774 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:48.774 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:48.774 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:48.776 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 298576, 2024-08-19T10:42:48.776 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:48.776 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:48.776 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:48.776 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:48.776 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:48.776 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:48.776 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:48.776 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:48.776 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 8, 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:48.777 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:48.778 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:48.779 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:48.780 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.781 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.782 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:48.784 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:48.787 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.788 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:48.789 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.792 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:48.794 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:48.795 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.796 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:48.797 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:48.798 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:48.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f854d640 1 -- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd2dc03be00 msgr2=0x7fd2dc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f854d640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd2dc03be00 0x7fd2dc03e2c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0003ce0 tx=0x7fd2e0009040 comp rx=0 tx=0).stop 2024-08-19T10:42:48.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f854d640 10 monclient: shutdown 2024-08-19T10:42:48.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f854d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:48.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f854d640 1 -- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 msgr2=0x7fd2f013fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f854d640 1 --2- 172.21.15.89:0/1782365559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2f00a5270 0x7fd2f013fba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009e20 tx=0x7fd2e80194d0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f6aea640 1 -- 172.21.15.89:0/1782365559 reap_dead start 2024-08-19T10:42:48.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f6aea640 1 -- 172.21.15.89:0/1782365559 reap_dead start 2024-08-19T10:42:48.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f854d640 1 -- 172.21.15.89:0/1782365559 shutdown_connections 2024-08-19T10:42:48.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f854d640 1 -- 172.21.15.89:0/1782365559 >> 172.21.15.89:0/1782365559 conn(0x7fd2f0069a80 msgr2=0x7fd2f00a9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:48.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f854d640 1 -- 172.21.15.89:0/1782365559 shutdown_connections 2024-08-19T10:42:48.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.747+0000 7fd2f854d640 1 -- 172.21.15.89:0/1782365559 wait complete. 2024-08-19T10:42:48.801 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:48.801 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:42:48.801 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:42:48.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.836+0000 7faf25860640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-08-19T10:42:48.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.836+0000 7faf25860640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-08-19T10:42:48.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.836+0000 7faf25860640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:42:48.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:48.836+0000 7faf25860640 1 -- 172.21.15.190:0/3653778388 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]} v 0) -- 0x7faf201242d0 con 0x7faf200a4720 2024-08-19T10:42:48.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient: get_monmap_and_config 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient: build_initial_monmap 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient: monmap: 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:48.925091+0000 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:48.925091+0000 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:48.925 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:48.926 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:48.926 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 1 Processor -- start 2024-08-19T10:42:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 1 -- start start 2024-08-19T10:42:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient: init 2024-08-19T10:42:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient: _add_conns ranks=[8,0,4,2,9,1,10,3,7,5,6] 2024-08-19T10:42:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30081378c0 0x7f3008137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient(hunting): picked mon.noname-e con 0x7f30081378c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f30081381e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3008072250 0x7f30080a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient(hunting): picked mon.noname-c con 0x7f3008072250 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3008074010 con 0x7f30081381e0 2024-08-19T10:42:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f30080a37b0 con 0x7f3008072250 2024-08-19T10:42:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f30080726d0 con 0x7f30081378c0 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300f859640 10 monclient(hunting): authenticate will time out at 1218.741741s 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300d5f5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30081378c0 0x7f3008137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300cdf4640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f30081381e0 0x7f30080a4df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52930/0 (socket says 172.21.15.89:52930) 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3008072250 0x7f30080a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300cdf4640 1 -- 172.21.15.89:0/34711433 learned_addr learned my addr 172.21.15.89:0/34711433 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300d5f5640 10 monclient(hunting): get_auth_request con 0x7f30081378c0 auth_method 0 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300d5f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300d5f5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300d5f5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 10 monclient(hunting): get_auth_request con 0x7f3008072250 auth_method 0 2024-08-19T10:42:48.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300d5f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300d5f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300d5f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 10 monclient(hunting): handle_auth_done global_id 5674 payload 274 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 1 -- 172.21.15.89:0/34711433 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30081378c0 msgr2=0x7f3008137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 1 --2- 172.21.15.89:0/34711433 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30081378c0 0x7f3008137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 1 -- 172.21.15.89:0/34711433 >> v1:172.21.15.89:6789/0 conn(0x7f30081381e0 legacy=0x7f30080a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:48.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 10 monclient: _finish_hunting 0 2024-08-19T10:42:48.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 1 monclient: found mon.noname-c 2024-08-19T10:42:48.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:48.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:48.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.923+0000 7f300ddf6640 1 -- 172.21.15.89:0/34711433 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30080a4300 con 0x7f3008072250 2024-08-19T10:42:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300ddf6640 1 -- 172.21.15.89:0/34711433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3008072250 msgr2=0x7f30080a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300ddf6640 1 --2- 172.21.15.89:0/34711433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3008072250 0x7f30080a5500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80097f0 tx=0x7f2ff801c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2706812c9bfbc7b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 1 -- 172.21.15.89:0/34711433 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ff80b0070 con 0x7f3008072250 2024-08-19T10:42:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 10 monclient: dump: 2024-08-19T10:42:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.939 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.940 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 10 monclient: _finish_auth 0 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 5 monclient: authenticate success, global_id 5674 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 1 -- 172.21.15.89:0/34711433 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff80aa810 con 0x7f3008072250 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 4 set_mon_vals no callback set 2024-08-19T10:42:48.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 1 -- 172.21.15.89:0/34711433 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ff80a4e60 con 0x7f3008072250 2024-08-19T10:42:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f3007fff640 10 monclient: dump: 2024-08-19T10:42:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.942 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 10 monclient: shutdown 2024-08-19T10:42:48.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 1 -- 172.21.15.89:0/34711433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3008072250 msgr2=0x7f30080a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 1 --2- 172.21.15.89:0/34711433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3008072250 0x7f30080a5500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80097f0 tx=0x7f2ff801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 1 -- 172.21.15.89:0/34711433 shutdown_connections 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 1 --2- 172.21.15.89:0/34711433 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3008072250 0x7f30080a5500 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 1 --2- 172.21.15.89:0/34711433 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30081378c0 0x7f3008137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 1 -- 172.21.15.89:0/34711433 >> 172.21.15.89:0/34711433 conn(0x7f30080aa240 msgr2=0x7f30080aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 1 -- 172.21.15.89:0/34711433 shutdown_connections 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.927+0000 7f300f859640 1 -- 172.21.15.89:0/34711433 wait complete. 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient: build_initial_monmap 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient: monmap: 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:48.944 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:48.945 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:48.946 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 1 Processor -- start 2024-08-19T10:42:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 1 -- start start 2024-08-19T10:42:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient: init 2024-08-19T10:42:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient: _add_conns ranks=[3,0,8,4,1,2,6,5,7] 2024-08-19T10:42:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3008072250 0x7f3008136da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient(hunting): picked mon.noname-g con 0x7f3008072250 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30081378c0 0x7f300806da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient(hunting): picked mon.noname-a con 0x7f30081378c0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:48.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30081381e0 0x7f300806df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient(hunting): picked mon.noname-e con 0x7f30081381e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3008141c00 con 0x7f30081378c0 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3008141a80 con 0x7f3008072250 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3008141d80 con 0x7f30081381e0 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300f859640 10 monclient(hunting): authenticate will time out at 1218.749361s 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30081378c0 0x7f300806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300ddf6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30081381e0 0x7f300806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30081378c0 0x7f300806da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37678/0 (socket says 172.21.15.89:37678) 2024-08-19T10:42:48.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 1 -- 172.21.15.89:0/1536241969 learned_addr learned my addr 172.21.15.89:0/1536241969 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 10 monclient(hunting): get_auth_request con 0x7f30081378c0 auth_method 0 2024-08-19T10:42:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300ddf6640 10 monclient(hunting): get_auth_request con 0x7f30081381e0 auth_method 0 2024-08-19T10:42:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300ddf6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300ddf6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300ddf6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300ddf6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300ddf6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300ddf6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 10 monclient(hunting): handle_auth_done global_id 5751 payload 824 2024-08-19T10:42:48.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300d5f5640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3008072250 0x7f3008136da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 1 -- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30081381e0 msgr2=0x7f300806df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f30081381e0 0x7f300806df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 1 -- 172.21.15.89:0/1536241969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3008072250 msgr2=0x7f3008136da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3008072250 0x7f3008136da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 10 monclient: _finish_hunting 0 2024-08-19T10:42:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 1 monclient: found mon.noname-a 2024-08-19T10:42:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:48.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:48.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300cdf4640 1 -- 172.21.15.89:0/1536241969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300813cd50 con 0x7f30081378c0 2024-08-19T10:42:48.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300d5f5640 10 monclient: get_auth_request con 0x7f3008072250 auth_method 0 2024-08-19T10:42:48.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.931+0000 7f300d5f5640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3008072250 0x7f3008136da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:48.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300cdf4640 1 -- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30081378c0 msgr2=0x7f300806da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300cdf4640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30081378c0 0x7f300806da40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00b430 tx=0x7f2ffc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ffc015020 con 0x7f30081378c0 2024-08-19T10:42:48.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 10 monclient: dump: 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.952 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300f859640 5 monclient: authenticate success, global_id 5751 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc011cb0 con 0x7f30081378c0 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ffc011e60 con 0x7f30081378c0 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300f859640 10 monclient: _renew_subs 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300f859640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300f859640 1 -- 172.21.15.89:0/1536241969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300813d030 con 0x7f30081378c0 2024-08-19T10:42:48.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f30077fe640 4 set_mon_vals no callback set 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 10 monclient: dump: 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.954 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300f859640 10 monclient: _renew_subs 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300f859640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300f859640 1 -- 172.21.15.89:0/1536241969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300813e910 con 0x7f30081378c0 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300f859640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300f859640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300806e4c0 0x7f300809e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300f859640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300f859640 1 -- 172.21.15.89:0/1536241969 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f300806e8a0 con 0x7f300806e4c0 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300d5f5640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300806e4c0 0x7f300809e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300d5f5640 10 monclient: get_auth_request con 0x7f300806e4c0 auth_method 0 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300d5f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300d5f5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300d5f5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300d5f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300d5f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300d5f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2ffc018020 con 0x7f30081378c0 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300d5f5640 10 monclient(hunting): handle_auth_done global_id 5683 payload 824 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300d5f5640 1 -- 172.21.15.89:0/1536241969 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30080a2ff0 con 0x7f300806e4c0 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ff403be00 0x7f2ff403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9079+0+0 (secure 0 0 0) 0x7f2ffc04f9d0 con 0x7f30081378c0 2024-08-19T10:42:48.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300ddf6640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ff403be00 0x7f2ff403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.935+0000 7f300ddf6640 10 monclient: get_auth_request con 0x7f2ff403be00 auth_method 0 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f300d5f5640 1 -- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300806e4c0 msgr2=0x7f300809e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f300d5f5640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300806e4c0 0x7f300809e9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f300000f3b0 tx=0x7f300000f870 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3000009e50 con 0x7f300806e4c0 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f3005ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f3005ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f3005ffb640 10 monclient: dump: 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.960 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f30000147f0 con 0x7f300806e4c0 2024-08-19T10:42:48.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f3005ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f3005ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300806e4c0 msgr2=0x7f300809e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f3005ffb640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300806e4c0 0x7f300809e9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f300000f3b0 tx=0x7f300000f870 comp rx=0 tx=0).stop 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300806e4c0 msgr2=0x7f300809e9f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f3005ffb640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300806e4c0 0x7f300809e9f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.939+0000 7f300ddf6640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ff403be00 0x7f2ff403e2c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f30080730b0 tx=0x7f2ff80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300f859640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300f859640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300809ef30 0x7f300809f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300f859640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300f859640 1 -- 172.21.15.89:0/1536241969 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3008064590 con 0x7f300809ef30 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300d5f5640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300809ef30 0x7f300809f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:48.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300d5f5640 10 monclient: get_auth_request con 0x7f300809ef30 auth_method 0 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300d5f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300d5f5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300d5f5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300d5f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300d5f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300d5f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300d5f5640 10 monclient(hunting): handle_auth_done global_id 5692 payload 824 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.943+0000 7f300d5f5640 1 -- 172.21.15.89:0/1536241969 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f300813df50 con 0x7f300809ef30 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300d5f5640 1 -- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300809ef30 msgr2=0x7f300809f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300d5f5640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300809ef30 0x7f300809f340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f300000f3b0 tx=0x7f3000004a50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3000003e10 con 0x7f300809ef30 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f3005ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:48.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f3005ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f3005ffb640 10 monclient: dump: 2024-08-19T10:42:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:48.964 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:48.965 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:48.965 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:48.965 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:48.965 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:48.965 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:48.965 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:48.965 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:48.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f30000039a0 con 0x7f300809ef30 2024-08-19T10:42:48.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f3005ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:48.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f3005ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:48.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300809ef30 msgr2=0x7f300809f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f3005ffb640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300809ef30 0x7f300809f340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f300000f3b0 tx=0x7f3000004a50 comp rx=0 tx=0).stop 2024-08-19T10:42:48.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f3005ffb640 1 -- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300809ef30 msgr2=0x7f300809f340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:48.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f3005ffb640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f300809ef30 0x7f300809f340 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f300000f3b0 tx=0x7f3000004a50 comp rx=0 tx=0).stop 2024-08-19T10:42:48.966 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:48.966 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:48.966 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:48.967 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:48.967 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:48.967 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 298764, 2024-08-19T10:42:48.967 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:48.967 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:48.967 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:48.967 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:48.968 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:48.968 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:48.968 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:48.968 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:48.968 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:48.968 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:48.968 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.968 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 8, 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:48.969 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:48.970 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:48.971 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:48.972 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:48.973 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:48.973 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:48.973 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:48.973 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:48.973 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:48.973 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:48.973 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:48.973 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:48.973 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:48.973 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:48.973 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.974 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:48.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.974 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:48.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.974 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:48.975 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.976 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.979 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.979 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:48.979 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.979 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.979 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:48.979 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.980 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.980 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.980 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:48.980 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:48.980 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.980 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.980 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.981 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:48.981 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:48.981 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.981 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.981 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.981 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:48.982 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.982 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.982 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:48.982 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.982 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.982 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.982 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:48.984 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.984 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.984 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:48.984 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.984 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.984 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.984 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:48.984 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.986 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.986 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:48.986 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.986 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.986 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.986 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:48.986 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:48.986 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.986 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:48.988 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:48.989 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.989 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.989 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:48.989 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.989 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:48.990 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.990 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.990 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.990 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:48.990 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:48.990 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.990 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.990 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.991 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:48.991 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:48.991 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:48.991 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:48.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.991 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:48.991 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:48.991 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:48.992 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:48.993 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:48.994 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:48.994 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:48.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:48.994 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:48.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:48.994 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:48.994 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300f859640 1 -- 172.21.15.89:0/1536241969 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ff403be00 msgr2=0x7f2ff403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300f859640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ff403be00 0x7f2ff403e2c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f30080730b0 tx=0x7f2ff80a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300f859640 10 monclient: shutdown 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300f859640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300f859640 1 -- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30081378c0 msgr2=0x7f300806da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300f859640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30081378c0 0x7f300806da40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00b430 tx=0x7f2ffc00b8f0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300ddf6640 1 -- 172.21.15.89:0/1536241969 reap_dead start 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300f859640 1 -- 172.21.15.89:0/1536241969 shutdown_connections 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300f859640 1 --2- 172.21.15.89:0/1536241969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30081378c0 0x7f300806da40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300f859640 1 -- 172.21.15.89:0/1536241969 >> 172.21.15.89:0/1536241969 conn(0x7f30080aa240 msgr2=0x7f300813f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300f859640 1 -- 172.21.15.89:0/1536241969 shutdown_connections 2024-08-19T10:42:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:48.947+0000 7f300f859640 1 -- 172.21.15.89:0/1536241969 wait complete. 2024-08-19T10:42:48.997 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:48.997 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:42:48.997 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:42:49.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient: get_monmap_and_config 2024-08-19T10:42:49.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient: build_initial_monmap 2024-08-19T10:42:49.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient: monmap: 2024-08-19T10:42:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:49.117474+0000 2024-08-19T10:42:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:49.117474+0000 2024-08-19T10:42:49.118 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:49.119 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.119 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:49.119 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:49.119 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:49.119 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:49.120 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:49.120 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:49.120 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:49.120 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:49.120 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:49.121 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:49.121 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:49.121 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 1 Processor -- start 2024-08-19T10:42:49.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 1 -- start start 2024-08-19T10:42:49.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient: init 2024-08-19T10:42:49.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:49.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient: _add_conns ranks=[5,0,9,3,10,7,8,1,4,6,2] 2024-08-19T10:42:49.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa61406c930 0x7fa61409d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient(hunting): picked mon.noname-h con 0x7fa61406c930 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:49.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa61406cd10 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:49.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa61413caa0 0x7fa61413ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient(hunting): picked mon.noname-a con 0x7fa61413caa0 addr v2:172.21.15.89:3300/0 2024-08-19T10:42:49.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6140a3ea0 con 0x7fa61406cd10 2024-08-19T10:42:49.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fa614074320 con 0x7fa61413caa0 2024-08-19T10:42:49.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa61409ca80 con 0x7fa61406c930 2024-08-19T10:42:49.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:49.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa618b13640 10 monclient(hunting): authenticate will time out at 1218.934138s 2024-08-19T10:42:49.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa611d9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa61406cd10 0x7fa61406d0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52936/0 (socket says 172.21.15.89:52936) 2024-08-19T10:42:49.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa611d9b640 1 -- 172.21.15.89:0/1431360611 learned_addr learned my addr 172.21.15.89:0/1431360611 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:42:49.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 1 --2- 172.21.15.89:0/1431360611 >> v2:172.21.15.89:3300/0 conn(0x7fa61413caa0 0x7fa61413ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 10 monclient(hunting): get_auth_request con 0x7fa61413caa0 auth_method 0 2024-08-19T10:42:49.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa61259c640 1 --2- 172.21.15.89:0/1431360611 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa61406c930 0x7fa61409d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 10 monclient(hunting): handle_auth_done global_id 5760 payload 274 2024-08-19T10:42:49.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 1 -- 172.21.15.89:0/1431360611 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa61406c930 msgr2=0x7fa61409d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:49.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 1 --2- 172.21.15.89:0/1431360611 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa61406c930 0x7fa61409d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 1 -- 172.21.15.89:0/1431360611 >> v1:172.21.15.89:6789/0 conn(0x7fa61406cd10 legacy=0x7fa61406d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:49.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:42:49.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 1 monclient: found mon.noname-a 2024-08-19T10:42:49.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:49.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:42:49.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.115+0000 7fa612d9d640 1 -- 172.21.15.89:0/1431360611 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6140a4f40 con 0x7fa61413caa0 2024-08-19T10:42:49.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa612d9d640 1 -- 172.21.15.89:0/1431360611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa61413caa0 msgr2=0x7fa61413ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa612d9d640 1 --2- 172.21.15.89:0/1431360611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa61413caa0 0x7fa61413ee90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0097c0 tx=0x7fa60c01c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60356b2d25835a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 1 -- 172.21.15.89:0/1431360611 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa60c0b1070 con 0x7fa61413caa0 2024-08-19T10:42:49.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:42:49.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 10 monclient: dump: 2024-08-19T10:42:49.149 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.149 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.149 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.149 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.149 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.149 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.150 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.150 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.150 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.150 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.150 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.151 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.151 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.151 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.151 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.151 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 10 monclient: _finish_auth 0 2024-08-19T10:42:49.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 10 monclient: _check_auth_tickets 2024-08-19T10:42:49.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:49.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 5 monclient: authenticate success, global_id 5760 2024-08-19T10:42:49.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:49.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 1 -- 172.21.15.89:0/1431360611 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa60c0a4d90 con 0x7fa61413caa0 2024-08-19T10:42:49.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:49.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:49.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 4 set_mon_vals no callback set 2024-08-19T10:42:49.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 1 -- 172.21.15.89:0/1431360611 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa60c0aac20 con 0x7fa61413caa0 2024-08-19T10:42:49.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:42:49.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa61159a640 10 monclient: dump: 2024-08-19T10:42:49.154 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.154 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.154 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.154 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.155 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.155 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.155 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.155 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.155 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.156 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.156 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.156 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.156 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.156 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.156 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.157 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 10 monclient: shutdown 2024-08-19T10:42:49.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:49.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 1 -- 172.21.15.89:0/1431360611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa61413caa0 msgr2=0x7fa61413ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 1 --2- 172.21.15.89:0/1431360611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa61413caa0 0x7fa61413ee90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0097c0 tx=0x7fa60c01c180 comp rx=0 tx=0).stop 2024-08-19T10:42:49.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 1 -- 172.21.15.89:0/1431360611 shutdown_connections 2024-08-19T10:42:49.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 1 --2- 172.21.15.89:0/1431360611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa61413caa0 0x7fa61413ee90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 1 --2- 172.21.15.89:0/1431360611 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa61406c930 0x7fa61409d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 1 -- 172.21.15.89:0/1431360611 >> 172.21.15.89:0/1431360611 conn(0x7fa6140a9130 msgr2=0x7fa6140a9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:49.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 1 -- 172.21.15.89:0/1431360611 shutdown_connections 2024-08-19T10:42:49.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.119+0000 7fa618b13640 1 -- 172.21.15.89:0/1431360611 wait complete. 2024-08-19T10:42:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient: build_initial_monmap 2024-08-19T10:42:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient: monmap: 2024-08-19T10:42:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:49.159 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:49.160 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:49.161 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:49.162 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 1 Processor -- start 2024-08-19T10:42:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 1 -- start start 2024-08-19T10:42:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient: init 2024-08-19T10:42:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:49.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient: _add_conns ranks=[0,6,4,3,5,2,8,7,1] 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa61406c930 0x7fa61413aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient(hunting): picked mon.noname-a con 0x7fa61406c930 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61406cd10 0x7fa614138f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient(hunting): picked mon.noname-d con 0x7fa61406cd10 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa61413caa0 0x7fa614134740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient(hunting): picked mon.noname-c con 0x7fa61413caa0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa614141f80 con 0x7fa61406c930 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa614141e00 con 0x7fa61413caa0 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa614142100 con 0x7fa61406cd10 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:49.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa618b13640 10 monclient(hunting): authenticate will time out at 1218.941313s 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa612d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa61413caa0 0x7fa614134740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61406cd10 0x7fa614138f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa612d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa61413caa0 0x7fa614134740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38448/0 (socket says 172.21.15.89:38448) 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa612d9d640 1 -- 172.21.15.89:0/2353387395 learned_addr learned my addr 172.21.15.89:0/2353387395 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa612d9d640 10 monclient(hunting): get_auth_request con 0x7fa61413caa0 auth_method 0 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa61259c640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa61406c930 0x7fa61413aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa612d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa612d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa612d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 10 monclient(hunting): get_auth_request con 0x7fa61406cd10 auth_method 0 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa61259c640 10 monclient(hunting): get_auth_request con 0x7fa61406c930 auth_method 0 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa61259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa61259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa61259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa612d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa612d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa612d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa61259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa61259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa61259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 10 monclient(hunting): handle_auth_done global_id 5559 payload 824 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 1 -- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa61413caa0 msgr2=0x7fa614134740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa61413caa0 0x7fa614134740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 1 -- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa61406c930 msgr2=0x7fa61413aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa61406c930 0x7fa61413aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 1 monclient: found mon.noname-d 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa611d9b640 1 -- 172.21.15.89:0/2353387395 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61413c230 con 0x7fa61406cd10 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa61259c640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa61406c930 0x7fa61413aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:49.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.123+0000 7fa612d9d640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa61413caa0 0x7fa614134740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa611d9b640 1 -- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61406cd10 msgr2=0x7fa614138f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa611d9b640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61406cd10 0x7fa614138f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa60800c440 tx=0x7fa60800c900 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa608017070 con 0x7fa61406cd10 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 10 monclient: dump: 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.167 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 10 monclient: _finish_auth 0 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:49.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa608005d00 con 0x7fa61406cd10 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa618b13640 5 monclient: authenticate success, global_id 5559 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa618b13640 10 monclient: _renew_subs 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa618b13640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa618b13640 1 -- 172.21.15.89:0/2353387395 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6141353e0 con 0x7fa61406cd10 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa608007540 con 0x7fa61406cd10 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa610d99640 4 set_mon_vals no callback set 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 10 monclient: dump: 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.169 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.170 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa618b13640 10 monclient: _renew_subs 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa618b13640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa618b13640 1 -- 172.21.15.89:0/2353387395 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61413a490 con 0x7fa61406cd10 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa618b13640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa618b13640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa614134c80 0x7fa6141cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa618b13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa618b13640 1 -- 172.21.15.89:0/2353387395 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa61409d450 con 0x7fa614134c80 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa61259c640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa614134c80 0x7fa6141cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa61259c640 10 monclient: get_auth_request con 0x7fa614134c80 auth_method 0 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa61259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa61259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa61259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa608007070 con 0x7fa61406cd10 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa61259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa61259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa61259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa61259c640 10 monclient(hunting): handle_auth_done global_id 5568 payload 824 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa61259c640 1 -- 172.21.15.89:0/2353387395 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa614064590 con 0x7fa614134c80 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa5f403be00 0x7fa5f403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(154..154 src has 1..154) ==== 9079+0+0 (secure 0 0 0) 0x7fa60804eed0 con 0x7fa61406cd10 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa612d9d640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa5f403be00 0x7fa5f403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.127+0000 7fa612d9d640 10 monclient: get_auth_request con 0x7fa5f403be00 auth_method 0 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa61259c640 1 -- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa614134c80 msgr2=0x7fa6141cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa61259c640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa614134c80 0x7fa6141cf8b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa60400ffb0 tx=0x7fa60400d810 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa60400f320 con 0x7fa614134c80 2024-08-19T10:42:49.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa602ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa602ffd640 10 monclient: dump: 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.173 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa60400c640 con 0x7fa614134c80 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa602ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa602ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa614134c80 msgr2=0x7fa6141cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa602ffd640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa614134c80 0x7fa6141cf8b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa60400ffb0 tx=0x7fa60400d810 comp rx=0 tx=0).stop 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa614134c80 msgr2=0x7fa6141cf8b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa602ffd640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa614134c80 0x7fa6141cf8b0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa60400ffb0 tx=0x7fa60400d810 comp rx=0 tx=0).stop 2024-08-19T10:42:49.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.131+0000 7fa612d9d640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa5f403be00 0x7fa5f403e2c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa60c01c1c0 tx=0x7fa60c0a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa618b13640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa618b13640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61402d1d0 0x7fa61402d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa618b13640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa618b13640 1 -- 172.21.15.89:0/2353387395 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa61413bf50 con 0x7fa61402d1d0 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa61259c640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61402d1d0 0x7fa61402d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa61259c640 10 monclient: get_auth_request con 0x7fa61402d1d0 auth_method 0 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa61259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa61259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa61259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa61259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa61259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa61259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa61259c640 10 monclient(hunting): handle_auth_done global_id 5577 payload 824 2024-08-19T10:42:49.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.135+0000 7fa61259c640 1 -- 172.21.15.89:0/2353387395 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa614074520 con 0x7fa61402d1d0 2024-08-19T10:42:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa61259c640 1 -- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61402d1d0 msgr2=0x7fa61402d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa61259c640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61402d1d0 0x7fa61402d5e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa604018000 tx=0x7fa604004a50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa604018440 con 0x7fa61402d1d0 2024-08-19T10:42:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa602ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa602ffd640 10 monclient: dump: 2024-08-19T10:42:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.177 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9601 (secure 0 0 0) 0x7fa60401e040 con 0x7fa61402d1d0 2024-08-19T10:42:49.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa602ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:49.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa602ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:49.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61402d1d0 msgr2=0x7fa61402d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa602ffd640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61402d1d0 0x7fa61402d5e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa604018000 tx=0x7fa604004a50 comp rx=0 tx=0).stop 2024-08-19T10:42:49.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa602ffd640 1 -- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61402d1d0 msgr2=0x7fa61402d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:49.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa602ffd640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61402d1d0 0x7fa61402d5e0 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa604018000 tx=0x7fa604004a50 comp rx=0 tx=0).stop 2024-08-19T10:42:49.179 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:49.179 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 298956, 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.180 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 9, 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.181 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.182 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:49.183 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.184 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:49.185 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.187 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:49.188 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:49.189 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.190 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.191 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:49.192 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.193 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:49.195 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.198 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:49.200 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.201 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:42:49.202 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.203 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.203 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:42:49.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.203 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.203 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.203 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:49.203 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.203 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.203 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:49.203 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa618b13640 1 -- 172.21.15.89:0/2353387395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa5f403be00 msgr2=0x7fa5f403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa618b13640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa5f403be00 0x7fa5f403e2c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa60c01c1c0 tx=0x7fa60c0a6070 comp rx=0 tx=0).stop 2024-08-19T10:42:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa618b13640 10 monclient: shutdown 2024-08-19T10:42:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa618b13640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa618b13640 1 -- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61406cd10 msgr2=0x7fa614138f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa618b13640 1 --2- 172.21.15.89:0/2353387395 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa61406cd10 0x7fa614138f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa60800c440 tx=0x7fa60800c900 comp rx=0 tx=0).stop 2024-08-19T10:42:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa612d9d640 1 -- 172.21.15.89:0/2353387395 reap_dead start 2024-08-19T10:42:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa612d9d640 1 -- 172.21.15.89:0/2353387395 reap_dead start 2024-08-19T10:42:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa618b13640 1 -- 172.21.15.89:0/2353387395 shutdown_connections 2024-08-19T10:42:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa618b13640 1 -- 172.21.15.89:0/2353387395 >> 172.21.15.89:0/2353387395 conn(0x7fa6140a9130 msgr2=0x7fa614135910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa618b13640 1 -- 172.21.15.89:0/2353387395 shutdown_connections 2024-08-19T10:42:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.139+0000 7fa618b13640 1 -- 172.21.15.89:0/2353387395 wait complete. 2024-08-19T10:42:49.206 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:49.206 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:42:49.206 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:42:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient: get_monmap_and_config 2024-08-19T10:42:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient: build_initial_monmap 2024-08-19T10:42:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient: monmap: 2024-08-19T10:42:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:49.329197+0000 2024-08-19T10:42:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:49.329197+0000 2024-08-19T10:42:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:49.329 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:49.330 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:49.330 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 1 Processor -- start 2024-08-19T10:42:49.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 1 -- start start 2024-08-19T10:42:49.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient: init 2024-08-19T10:42:49.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:49.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient: _add_conns ranks=[8,5,4,7,3,6,2,9,10,1,0] 2024-08-19T10:42:49.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a3620 0x7f64f40a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient(hunting): picked mon.noname-e con 0x7f64f40a3620 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f64f409c9e0 0x7f64f40a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient(hunting): picked mon.noname-h con 0x7f64f409c9e0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 0x7f64f4137810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient(hunting): picked mon.noname-c con 0x7f64f41373f0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f64f409dab0 con 0x7f64f41373f0 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f64f40a1f20 con 0x7f64f409c9e0 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f64f40a4480 con 0x7f64f40a3620 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:49.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fc053640 10 monclient(hunting): authenticate will time out at 1219.145808s 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 0x7f64f4137810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 0x7f64f4137810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38450/0 (socket says 172.21.15.89:38450) 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 1 -- 172.21.15.89:0/4055776827 learned_addr learned my addr 172.21.15.89:0/4055776827 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 10 monclient(hunting): get_auth_request con 0x7f64f41373f0 auth_method 0 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64f9def640 1 --2- 172.21.15.89:0/4055776827 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a3620 0x7f64f40a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64f9def640 10 monclient(hunting): get_auth_request con 0x7f64f40a3620 auth_method 0 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64f9def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64f9def640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64f9def640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64f95ee640 1 --2- 172.21.15.89:0/4055776827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f64f409c9e0 0x7f64f40a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 10 monclient(hunting): handle_auth_done global_id 5710 payload 274 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 1 -- 172.21.15.89:0/4055776827 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a3620 msgr2=0x7f64f40a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 1 --2- 172.21.15.89:0/4055776827 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a3620 0x7f64f40a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 1 -- 172.21.15.89:0/4055776827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f64f409c9e0 msgr2=0x7f64f40a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 1 --2- 172.21.15.89:0/4055776827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f64f409c9e0 0x7f64f40a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 10 monclient: _finish_hunting 0 2024-08-19T10:42:49.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 1 monclient: found mon.noname-c 2024-08-19T10:42:49.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:49.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:49.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64fa5f0640 1 -- 172.21.15.89:0/4055776827 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f40a1790 con 0x7f64f41373f0 2024-08-19T10:42:49.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64f95ee640 10 monclient: get_auth_request con 0x7f64f409c9e0 auth_method 0 2024-08-19T10:42:49.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.327+0000 7f64f95ee640 1 --2- 172.21.15.89:0/4055776827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f64f409c9e0 0x7f64f40a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:49.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fa5f0640 1 -- 172.21.15.89:0/4055776827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 msgr2=0x7f64f4137810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fa5f0640 1 --2- 172.21.15.89:0/4055776827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 0x7f64f4137810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009eb0 tx=0x7f64ec01c8f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=bf06e97f417e8e7e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 1 -- 172.21.15.89:0/4055776827 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64ec0090d0 con 0x7f64f41373f0 2024-08-19T10:42:49.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:49.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 10 monclient: dump: 2024-08-19T10:42:49.339 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.340 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 10 monclient: _finish_auth 0 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 10 monclient: _check_auth_tickets 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 5 monclient: authenticate success, global_id 5710 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 1 -- 172.21.15.89:0/4055776827 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ec0a5a10 con 0x7f64f41373f0 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 1 -- 172.21.15.89:0/4055776827 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64ec0a5bc0 con 0x7f64f41373f0 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 4 set_mon_vals no callback set 2024-08-19T10:42:49.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64f8ded640 10 monclient: dump: 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 10 monclient: shutdown 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 1 -- 172.21.15.89:0/4055776827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 msgr2=0x7f64f4137810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 1 --2- 172.21.15.89:0/4055776827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 0x7f64f4137810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009eb0 tx=0x7f64ec01c8f0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 1 -- 172.21.15.89:0/4055776827 shutdown_connections 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 1 --2- 172.21.15.89:0/4055776827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 0x7f64f4137810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 1 --2- 172.21.15.89:0/4055776827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f64f409c9e0 0x7f64f40a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 1 --2- 172.21.15.89:0/4055776827 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a3620 0x7f64f40a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 1 -- 172.21.15.89:0/4055776827 >> 172.21.15.89:0/4055776827 conn(0x7f64f40a8210 msgr2=0x7f64f40a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 1 -- 172.21.15.89:0/4055776827 shutdown_connections 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.331+0000 7f64fc053640 1 -- 172.21.15.89:0/4055776827 wait complete. 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient: build_initial_monmap 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient: monmap: 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 1 Processor -- start 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 1 -- start start 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient: init 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient: _add_conns ranks=[0,8,4,6,5,2,1,7,3] 2024-08-19T10:42:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409c9e0 0x7f64f4136640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient(hunting): picked mon.noname-a con 0x7f64f409c9e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a3620 0x7f64f409f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient(hunting): picked mon.noname-e con 0x7f64f40a3620 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 0x7f64f409f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient(hunting): picked mon.noname-c con 0x7f64f41373f0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4140e60 con 0x7f64f409c9e0 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f64f409df40 con 0x7f64f41373f0 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f64f4140fe0 con 0x7f64f40a3620 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:49.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fc053640 10 monclient(hunting): authenticate will time out at 1219.153162s 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f9def640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409c9e0 0x7f64f4136640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f9def640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409c9e0 0x7f64f4136640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37704/0 (socket says 172.21.15.89:37704) 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f95ee640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a3620 0x7f64f409f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 0x7f64f409f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f9def640 1 -- 172.21.15.89:0/2301333595 learned_addr learned my addr 172.21.15.89:0/2301333595 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f9def640 10 monclient(hunting): get_auth_request con 0x7f64f409c9e0 auth_method 0 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f9def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f9def640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f9def640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 10 monclient(hunting): get_auth_request con 0x7f64f41373f0 auth_method 0 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f95ee640 10 monclient(hunting): get_auth_request con 0x7f64f40a3620 auth_method 0 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f95ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f95ee640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f95ee640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f95ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f95ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f95ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 10 monclient(hunting): handle_auth_done global_id 5719 payload 824 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 1 -- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a3620 msgr2=0x7f64f409f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a3620 0x7f64f409f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 1 -- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409c9e0 msgr2=0x7f64f4136640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409c9e0 0x7f64f4136640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 10 monclient: _finish_hunting 0 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 1 monclient: found mon.noname-c 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 1 -- 172.21.15.89:0/2301333595 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f40a6c70 con 0x7f64f41373f0 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f9def640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64f409c9e0 0x7f64f4136640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:49.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64f95ee640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a3620 0x7f64f409f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:49.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 1 -- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 msgr2=0x7f64f409f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.335+0000 7f64fa5f0640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 0x7f64f409f6c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0ab000 tx=0x7f64ec019690 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64ec0bc030 con 0x7f64f41373f0 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: dump: 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.354 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:49.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ec003e10 con 0x7f64f41373f0 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64ec0a57b0 con 0x7f64f41373f0 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: dump: 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.356 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64eb7fe640 4 set_mon_vals no callback set 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64fc053640 5 monclient: authenticate success, global_id 5719 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64fc053640 10 monclient: _renew_subs 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64fc053640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64fc053640 1 -- 172.21.15.89:0/2301333595 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f413d680 con 0x7f64f41373f0 2024-08-19T10:42:49.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64fc053640 10 monclient: _renew_subs 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64fc053640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64fc053640 1 -- 172.21.15.89:0/2301333595 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f4123520 con 0x7f64f41373f0 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64fc053640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64fc053640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f4136b80 0x7f64f409fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64fc053640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64fc053640 1 -- 172.21.15.89:0/2301333595 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f64f41d1e20 con 0x7f64f4136b80 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f4136b80 0x7f64f409fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 10 monclient: get_auth_request con 0x7f64f4136b80 auth_method 0 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f64ec0039a0 con 0x7f64f41373f0 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 10 monclient(hunting): handle_auth_done global_id 5561 payload 824 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 1 -- 172.21.15.89:0/2301333595 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64f4137810 con 0x7f64f4136b80 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f64d803be00 0x7f64d803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(154..154 src has 1..154) ==== 9079+0+0 (secure 0 0 0) 0x7f64ec0bd070 con 0x7f64f41373f0 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f95ee640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f64d803be00 0x7f64d803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f95ee640 10 monclient: get_auth_request con 0x7f64d803be00 auth_method 0 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 1 -- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f4136b80 msgr2=0x7f64f409fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64f9def640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f4136b80 0x7f64f409fc00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f64f0011d30 tx=0x7f64f000f610 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64f000fe70 con 0x7f64f4136b80 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.339+0000 7f64ea7fc640 10 monclient: dump: 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.360 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.343+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f64f000fe70 con 0x7f64f4136b80 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.343+0000 7f64ea7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.343+0000 7f64ea7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.343+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f4136b80 msgr2=0x7f64f409fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.343+0000 7f64ea7fc640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f4136b80 0x7f64f409fc00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f64f0011d30 tx=0x7f64f000f610 comp rx=0 tx=0).stop 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.343+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f4136b80 msgr2=0x7f64f409fc00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.343+0000 7f64ea7fc640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f4136b80 0x7f64f409fc00 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f64f0011d30 tx=0x7f64f000f610 comp rx=0 tx=0).stop 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.343+0000 7f64f95ee640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f64d803be00 0x7f64d803e2c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f64f413d340 tx=0x7f64e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64fc053640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64fc053640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a0140 0x7f64f4138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64fc053640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64fc053640 1 -- 172.21.15.89:0/2301333595 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f64f40a0550 con 0x7f64f40a0140 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64f9def640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a0140 0x7f64f4138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64f9def640 10 monclient: get_auth_request con 0x7f64f40a0140 auth_method 0 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64f9def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64f9def640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64f9def640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64f9def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64f9def640 10 monclient(hunting): handle_auth_done global_id 5570 payload 824 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.347+0000 7f64f9def640 1 -- 172.21.15.89:0/2301333595 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f64f4064590 con 0x7f64f40a0140 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64f9def640 1 -- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a0140 msgr2=0x7f64f4138400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64f9def640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a0140 0x7f64f4138400 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f64f0011d30 tx=0x7f64f00046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64f0004490 con 0x7f64f40a0140 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64ea7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64ea7fc640 10 monclient: dump: 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.363 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f64f0004020 con 0x7f64f40a0140 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64ea7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64ea7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a0140 msgr2=0x7f64f4138400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64ea7fc640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a0140 0x7f64f4138400 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f64f0011d30 tx=0x7f64f00046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64ea7fc640 1 -- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a0140 msgr2=0x7f64f4138400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:49.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64ea7fc640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64f40a0140 0x7f64f4138400 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f64f0011d30 tx=0x7f64f00046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.365 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:49.365 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:49.365 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:49.365 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:49.365 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:49.365 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 299144, 2024-08-19T10:42:49.365 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:49.365 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:49.365 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:49.366 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:49.366 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:49.366 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 9, 2024-08-19T10:42:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:49.367 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.368 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.368 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.368 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.368 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.368 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.368 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.368 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.368 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.368 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.368 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.368 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.369 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:49.370 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.371 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:49.372 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.373 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:49.374 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.376 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:49.376 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:49.376 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.376 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.376 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.376 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:49.376 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:49.376 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.377 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.379 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.381 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.382 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:49.384 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.387 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:49.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:49.387 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:49.388 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:49.388 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.388 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.389 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.390 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:49.391 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:49.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fc053640 1 -- 172.21.15.89:0/2301333595 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f64d803be00 msgr2=0x7f64d803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fc053640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f64d803be00 0x7f64d803e2c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f64f413d340 tx=0x7f64e4008040 comp rx=0 tx=0).stop 2024-08-19T10:42:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fc053640 10 monclient: shutdown 2024-08-19T10:42:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fc053640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fc053640 1 -- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 msgr2=0x7f64f409f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fa5f0640 1 -- 172.21.15.89:0/2301333595 reap_dead start 2024-08-19T10:42:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fc053640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 0x7f64f409f6c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0ab000 tx=0x7f64ec019690 comp rx=0 tx=0).stop 2024-08-19T10:42:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fc053640 1 -- 172.21.15.89:0/2301333595 shutdown_connections 2024-08-19T10:42:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fc053640 1 --2- 172.21.15.89:0/2301333595 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64f41373f0 0x7f64f409f6c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fc053640 1 -- 172.21.15.89:0/2301333595 >> 172.21.15.89:0/2301333595 conn(0x7f64f40a8210 msgr2=0x7f64f40a7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fc053640 1 -- 172.21.15.89:0/2301333595 shutdown_connections 2024-08-19T10:42:49.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.351+0000 7f64fc053640 1 -- 172.21.15.89:0/2301333595 wait complete. 2024-08-19T10:42:49.394 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:49.394 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:42:49.394 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:42:49.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient: get_monmap_and_config 2024-08-19T10:42:49.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient: build_initial_monmap 2024-08-19T10:42:49.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient: monmap: 2024-08-19T10:42:49.509 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:49.509 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.509 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:49.509490+0000 2024-08-19T10:42:49.509 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:49.509490+0000 2024-08-19T10:42:49.509 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:49.509 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:49.510 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 1 Processor -- start 2024-08-19T10:42:49.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 1 -- start start 2024-08-19T10:42:49.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient: init 2024-08-19T10:42:49.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:49.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient: _add_conns ranks=[3,8,1,7,9,2,10,4,6,5,0] 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4040a5270 0x7fb4040a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient(hunting): picked mon.noname-g con 0x7fb4040a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a6d30 0x7fb4040a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient(hunting): picked mon.noname-e con 0x7fb4040a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb4041395b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4040a8710 con 0x7fb4041395b0 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb4040a7650 con 0x7fb4040a5270 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb4040a5a40 con 0x7fb4040a6d30 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb40ac45640 10 monclient(hunting): authenticate will time out at 1219.326101s 2024-08-19T10:42:49.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a6d30 0x7fb4040a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a6d30 0x7fb4040a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40532/0 (socket says 172.21.15.89:40532) 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 1 -- 172.21.15.89:0/2834913821 learned_addr learned my addr 172.21.15.89:0/2834913821 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 10 monclient(hunting): get_auth_request con 0x7fb4040a6d30 auth_method 0 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb4089e1640 1 --2- 172.21.15.89:0/2834913821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4040a5270 0x7fb4040a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 10 monclient(hunting): handle_auth_done global_id 5579 payload 274 2024-08-19T10:42:49.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 1 -- 172.21.15.89:0/2834913821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4040a5270 msgr2=0x7fb4040a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:49.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 1 --2- 172.21.15.89:0/2834913821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4040a5270 0x7fb4040a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 1 -- 172.21.15.89:0/2834913821 >> v1:172.21.15.190:6789/0 conn(0x7fb4041395b0 legacy=0x7fb404139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:49.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:49.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 1 monclient: found mon.noname-e 2024-08-19T10:42:49.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:49.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:49.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb403fff640 1 -- 172.21.15.89:0/2834913821 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4040a81d0 con 0x7fb4040a6d30 2024-08-19T10:42:49.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb4089e1640 10 monclient: get_auth_request con 0x7fb4040a5270 auth_method 0 2024-08-19T10:42:49.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.507+0000 7fb4089e1640 1 --2- 172.21.15.89:0/2834913821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4040a5270 0x7fb4040a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:42:49.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb403fff640 1 -- 172.21.15.89:0/2834913821 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a6d30 msgr2=0x7fb4040a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb403fff640 1 --2- 172.21.15.89:0/2834913821 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a6d30 0x7fb4040a7110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009e80 tx=0x7fb3f801c730 comp rx=0 tx=0).ready entity=mon.8 client_cookie=3809233cdfae2a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 1 -- 172.21.15.89:0/2834913821 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3f80091b0 con 0x7fb4040a6d30 2024-08-19T10:42:49.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:49.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 10 monclient: dump: 2024-08-19T10:42:49.519 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.519 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.519 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.519 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:49.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 1 -- 172.21.15.89:0/2834913821 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f80a5980 con 0x7fb4040a6d30 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 1 -- 172.21.15.89:0/2834913821 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3f80ad3f0 con 0x7fb4040a6d30 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb4037fe640 10 monclient: dump: 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.521 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 5 monclient: authenticate success, global_id 5579 2024-08-19T10:42:49.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 4 set_mon_vals no callback set 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 10 monclient: shutdown 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 1 -- 172.21.15.89:0/2834913821 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a6d30 msgr2=0x7fb4040a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 1 --2- 172.21.15.89:0/2834913821 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a6d30 0x7fb4040a7110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009e80 tx=0x7fb3f801c730 comp rx=0 tx=0).stop 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 1 -- 172.21.15.89:0/2834913821 shutdown_connections 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 1 --2- 172.21.15.89:0/2834913821 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a6d30 0x7fb4040a7110 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 1 --2- 172.21.15.89:0/2834913821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4040a5270 0x7fb4040a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 1 -- 172.21.15.89:0/2834913821 >> 172.21.15.89:0/2834913821 conn(0x7fb404069a80 msgr2=0x7fb4040abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 1 -- 172.21.15.89:0/2834913821 shutdown_connections 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.511+0000 7fb40ac45640 1 -- 172.21.15.89:0/2834913821 wait complete. 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient: build_initial_monmap 2024-08-19T10:42:49.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient: monmap: 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:49.524 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:49.525 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:49.525 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:49.525 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 1 Processor -- start 2024-08-19T10:42:49.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 1 -- start start 2024-08-19T10:42:49.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient: init 2024-08-19T10:42:49.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:49.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient: _add_conns ranks=[8,3,6,4,2,5,1,0,7] 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a5270 0x7fb4041d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient(hunting): picked mon.noname-e con 0x7fb4040a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4040a6d30 0x7fb4041d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient(hunting): picked mon.noname-g con 0x7fb4040a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb4041395b0 0x7fb40413d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient(hunting): picked mon.noname-d con 0x7fb4041395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb4040a57d0 con 0x7fb4040a6d30 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb4040a5650 con 0x7fb4041395b0 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb404144240 con 0x7fb4040a5270 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb40ac45640 10 monclient(hunting): authenticate will time out at 1219.333571s 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a5270 0x7fb4041d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a5270 0x7fb4041d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40544/0 (socket says 172.21.15.89:40544) 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 1 -- 172.21.15.89:0/3437363502 learned_addr learned my addr 172.21.15.89:0/3437363502 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 10 monclient(hunting): get_auth_request con 0x7fb4040a5270 auth_method 0 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4091e2640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb4041395b0 0x7fb40413d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4091e2640 10 monclient(hunting): get_auth_request con 0x7fb4041395b0 auth_method 0 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4091e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4091e2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4091e2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4091e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4091e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4091e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 10 monclient(hunting): handle_auth_done global_id 5588 payload 824 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb403fff640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4040a6d30 0x7fb4041d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 1 -- 172.21.15.89:0/3437363502 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb4041395b0 msgr2=0x7fb40413d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb4041395b0 0x7fb40413d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 1 -- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4040a6d30 msgr2=0x7fb4041d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4040a6d30 0x7fb4041d18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 10 monclient: _finish_hunting 0 2024-08-19T10:42:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 1 monclient: found mon.noname-e 2024-08-19T10:42:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:49.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.515+0000 7fb4089e1640 1 -- 172.21.15.89:0/3437363502 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb404140b60 con 0x7fb4040a5270 2024-08-19T10:42:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4089e1640 1 -- 172.21.15.89:0/3437363502 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a5270 msgr2=0x7fb4041d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4089e1640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a5270 0x7fb4041d13b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00c820 tx=0x7fb3fc00cce0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3fc015070 con 0x7fb4040a5270 2024-08-19T10:42:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 10 monclient: dump: 2024-08-19T10:42:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.532 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.533 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 10 monclient: _finish_auth 0 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 10 monclient: _check_auth_tickets 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3fc00ce10 con 0x7fb4040a5270 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3fc011860 con 0x7fb4040a5270 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb40ac45640 5 monclient: authenticate success, global_id 5588 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb40ac45640 10 monclient: _renew_subs 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb40ac45640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb40ac45640 1 -- 172.21.15.89:0/3437363502 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb404122970 con 0x7fb4040a5270 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4027fc640 4 set_mon_vals no callback set 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 10 monclient: dump: 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.535 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb40ac45640 10 monclient: _renew_subs 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb40ac45640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb40ac45640 1 -- 172.21.15.89:0/3437363502 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fb404122690 con 0x7fb4040a5270 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb40ac45640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb40ac45640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413dc10 0x7fb40413dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb40ac45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb40ac45640 1 -- 172.21.15.89:0/3437363502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040aad90 con 0x7fb40413dc10 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb3fc018070 con 0x7fb4040a5270 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb403fff640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413dc10 0x7fb40413dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3f003be00 0x7fb3f003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb403fff640 10 monclient: get_auth_request con 0x7fb40413dc10 auth_method 0 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb403fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb403fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb403fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(154..154 src has 1..154) ==== 9079+0+0 (secure 0 0 0) 0x7fb3fc04fa70 con 0x7fb4040a5270 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb403fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb403fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb403fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4091e2640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3f003be00 0x7fb3f003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb4091e2640 10 monclient: get_auth_request con 0x7fb3f003be00 auth_method 0 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb403fff640 10 monclient(hunting): handle_auth_done global_id 14986 payload 824 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.519+0000 7fb403fff640 1 -- 172.21.15.89:0/3437363502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4040a67f0 con 0x7fb40413dc10 2024-08-19T10:42:49.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb403fff640 1 -- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413dc10 msgr2=0x7fb40413dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb403fff640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413dc10 0x7fb40413dff0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80ae000 tx=0x7fb3f80194d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4091e2640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3f003be00 0x7fb3f003e2c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4007fd0 tx=0x7fb3f4007dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3f80bb070 con 0x7fb40413dc10 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4017fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4017fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4017fa640 10 monclient: dump: 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.538 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb3f8004460 con 0x7fb40413dc10 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4017fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4017fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413dc10 msgr2=0x7fb40413dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4017fa640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413dc10 0x7fb40413dff0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80ae000 tx=0x7fb3f80194d0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413dc10 msgr2=0x7fb40413dff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:49.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.523+0000 7fb4017fa640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413dc10 0x7fb40413dff0 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80ae000 tx=0x7fb3f80194d0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb40ac45640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb40ac45640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413e530 0x7fb4040a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb40ac45640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb40ac45640 1 -- 172.21.15.89:0/3437363502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb404064590 con 0x7fb40413e530 2024-08-19T10:42:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb403fff640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413e530 0x7fb4040a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb403fff640 10 monclient: get_auth_request con 0x7fb40413e530 auth_method 0 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb403fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb403fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb403fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb403fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb403fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb403fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb403fff640 10 monclient(hunting): handle_auth_done global_id 14995 payload 824 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.531+0000 7fb403fff640 1 -- 172.21.15.89:0/3437363502 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb40406c9e0 con 0x7fb40413e530 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb403fff640 1 -- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413e530 msgr2=0x7fb4040a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb403fff640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413e530 0x7fb4040a0130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009e80 tx=0x7fb3f80a7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3f80c08c0 con 0x7fb40413e530 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4017fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4017fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4017fa640 10 monclient: dump: 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.543 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.544 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7fb3f80c0450 con 0x7fb40413e530 2024-08-19T10:42:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4017fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4017fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413e530 msgr2=0x7fb4040a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4017fa640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413e530 0x7fb4040a0130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009e80 tx=0x7fb3f80a7000 comp rx=0 tx=0).stop 2024-08-19T10:42:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4017fa640 1 -- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413e530 msgr2=0x7fb4040a0130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:49.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4017fa640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb40413e530 0x7fb4040a0130 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009e80 tx=0x7fb3f80a7000 comp rx=0 tx=0).stop 2024-08-19T10:42:49.545 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:49.545 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:49.545 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:49.545 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:49.545 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:49.545 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 147120, 2024-08-19T10:42:49.545 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:49.545 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:49.545 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:49.545 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:49.545 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 9, 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.546 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:49.547 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:49.548 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:49.549 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.550 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.551 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:49.552 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:49.553 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.554 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.555 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.556 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:49.557 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:49.558 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.559 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.560 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:49.561 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.563 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:49.564 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:49.565 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.567 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.568 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.569 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:49.569 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.569 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.569 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:49.569 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb40ac45640 1 -- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3f003be00 msgr2=0x7fb3f003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb40ac45640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3f003be00 0x7fb3f003e2c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4007fd0 tx=0x7fb3f4007dc0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb40ac45640 10 monclient: shutdown 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb40ac45640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb40ac45640 1 -- 172.21.15.89:0/3437363502 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a5270 msgr2=0x7fb4041d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb40ac45640 1 --2- 172.21.15.89:0/3437363502 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4040a5270 0x7fb4041d13b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00c820 tx=0x7fb3fc00cce0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4091e2640 1 -- 172.21.15.89:0/3437363502 reap_dead start 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb4091e2640 1 -- 172.21.15.89:0/3437363502 reap_dead start 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb40ac45640 1 -- 172.21.15.89:0/3437363502 shutdown_connections 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb40ac45640 1 -- 172.21.15.89:0/3437363502 >> 172.21.15.89:0/3437363502 conn(0x7fb404069a80 msgr2=0x7fb4040a9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb40ac45640 1 -- 172.21.15.89:0/3437363502 shutdown_connections 2024-08-19T10:42:49.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.535+0000 7fb40ac45640 1 -- 172.21.15.89:0/3437363502 wait complete. 2024-08-19T10:42:49.572 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:49.572 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:42:49.572 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:42:49.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient: get_monmap_and_config 2024-08-19T10:42:49.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient: build_initial_monmap 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient: monmap: 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:49.689095+0000 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:49.689095+0000 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:49.689 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:49.690 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:49.690 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:49.690 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:49.690 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:49.690 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:49.690 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:49.690 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 1 Processor -- start 2024-08-19T10:42:49.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 1 -- start start 2024-08-19T10:42:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient: init 2024-08-19T10:42:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient: _add_conns ranks=[1,2,7,4,3,0,5,9,8,10,6] 2024-08-19T10:42:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f91a806c930 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91a80a4220 0x7f91a813e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient(hunting): picked mon.noname-b con 0x7f91a80a4220 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f91a80a4600 0x7f91a8140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient(hunting): picked mon.noname-i con 0x7f91a80a4600 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a80a4df0 con 0x7f91a806c930 2024-08-19T10:42:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f91a80a1240 con 0x7f91a80a4220 2024-08-19T10:42:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f91a8074400 con 0x7f91a80a4600 2024-08-19T10:42:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91af596640 10 monclient(hunting): authenticate will time out at 1219.505742s 2024-08-19T10:42:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91a80a4220 0x7f91a813e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91ad332640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f91a806c930 0x7f91a80a3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.89:42290/0 (socket says 172.21.15.89:42290) 2024-08-19T10:42:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91a80a4220 0x7f91a813e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40186/0 (socket says 172.21.15.89:40186) 2024-08-19T10:42:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91ad332640 1 -- 172.21.15.89:0/1102272918 learned_addr learned my addr 172.21.15.89:0/1102272918 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:42:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 10 monclient(hunting): get_auth_request con 0x7f91a80a4220 auth_method 0 2024-08-19T10:42:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91adb33640 1 --2- 172.21.15.89:0/1102272918 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f91a80a4600 0x7f91a8140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91adb33640 10 monclient(hunting): get_auth_request con 0x7f91a80a4600 auth_method 0 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91adb33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91adb33640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91adb33640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 10 monclient(hunting): handle_auth_done global_id 5672 payload 274 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 1 -- 172.21.15.89:0/1102272918 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f91a80a4600 msgr2=0x7f91a8140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:49.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 1 --2- 172.21.15.89:0/1102272918 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f91a80a4600 0x7f91a8140ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 1 -- 172.21.15.89:0/1102272918 >> v1:172.21.15.190:6789/0 conn(0x7f91a806c930 legacy=0x7f91a80a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:49.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 10 monclient: _finish_hunting 0 2024-08-19T10:42:49.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 1 monclient: found mon.noname-b 2024-08-19T10:42:49.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:49.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:42:49.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.687+0000 7f91acb31640 1 -- 172.21.15.89:0/1102272918 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a80a0910 con 0x7f91a80a4220 2024-08-19T10:42:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91acb31640 1 -- 172.21.15.89:0/1102272918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91a80a4220 msgr2=0x7f91a813e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91acb31640 1 --2- 172.21.15.89:0/1102272918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91a80a4220 0x7f91a813e990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f91a00097f0 tx=0x7f91a001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a57b93e2127fb3f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 1 -- 172.21.15.89:0/1102272918 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91a00b0070 con 0x7f91a80a4220 2024-08-19T10:42:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:42:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 10 monclient: dump: 2024-08-19T10:42:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.700 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.701 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 10 monclient: _finish_auth 0 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 1 -- 172.21.15.89:0/1102272918 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a00aa810 con 0x7f91a80a4220 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 1 -- 172.21.15.89:0/1102272918 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91a00a4e60 con 0x7f91a80a4220 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 5 monclient: authenticate success, global_id 5672 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 4 set_mon_vals no callback set 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:49.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f919ffff640 10 monclient: dump: 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.703 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 10 monclient: shutdown 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 1 -- 172.21.15.89:0/1102272918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91a80a4220 msgr2=0x7f91a813e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 1 --2- 172.21.15.89:0/1102272918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91a80a4220 0x7f91a813e990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f91a00097f0 tx=0x7f91a001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 1 -- 172.21.15.89:0/1102272918 shutdown_connections 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 1 --2- 172.21.15.89:0/1102272918 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f91a80a4600 0x7f91a8140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 1 --2- 172.21.15.89:0/1102272918 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f91a80a4220 0x7f91a813e990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 1 -- 172.21.15.89:0/1102272918 >> 172.21.15.89:0/1102272918 conn(0x7f91a806de40 msgr2=0x7f91a806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 1 -- 172.21.15.89:0/1102272918 shutdown_connections 2024-08-19T10:42:49.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.691+0000 7f91af596640 1 -- 172.21.15.89:0/1102272918 wait complete. 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient: build_initial_monmap 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient: monmap: 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:49.705 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 1 Processor -- start 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 1 -- start start 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient: init 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient: _add_conns ranks=[3,8,4,2,6,5,1,0,7] 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a806c930 0x7f91a81405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient(hunting): picked mon.noname-g con 0x7f91a806c930 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:49.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f91a80a4220 0x7f91a8139800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient(hunting): picked mon.noname-e con 0x7f91a80a4220 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91a80a4600 0x7f91a813e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient(hunting): picked mon.noname-c con 0x7f91a80a4600 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f91a8143e50 con 0x7f91a806c930 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f91a806cd10 con 0x7f91a80a4600 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f91a8143fd0 con 0x7f91a80a4220 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91af596640 10 monclient(hunting): authenticate will time out at 1219.513616s 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91acb31640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f91a80a4220 0x7f91a8139800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91a80a4600 0x7f91a813e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91acb31640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f91a80a4220 0x7f91a8139800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40552/0 (socket says 172.21.15.89:40552) 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91acb31640 1 -- 172.21.15.89:0/135317647 learned_addr learned my addr 172.21.15.89:0/135317647 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91acb31640 10 monclient(hunting): get_auth_request con 0x7f91a80a4220 auth_method 0 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91acb31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91acb31640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91acb31640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 10 monclient(hunting): get_auth_request con 0x7f91a80a4600 auth_method 0 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91ad332640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a806c930 0x7f91a81405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91ad332640 10 monclient(hunting): get_auth_request con 0x7f91a806c930 auth_method 0 2024-08-19T10:42:49.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91ad332640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91ad332640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91ad332640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91acb31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91acb31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91acb31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91ad332640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91ad332640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91ad332640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 10 monclient(hunting): handle_auth_done global_id 5728 payload 824 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 1 -- 172.21.15.89:0/135317647 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f91a80a4220 msgr2=0x7f91a8139800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f91a80a4220 0x7f91a8139800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 1 -- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a806c930 msgr2=0x7f91a81405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a806c930 0x7f91a81405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 10 monclient: _finish_hunting 0 2024-08-19T10:42:49.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 1 monclient: found mon.noname-c 2024-08-19T10:42:49.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:49.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:49.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91adb33640 1 -- 172.21.15.89:0/135317647 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a809f5b0 con 0x7f91a80a4600 2024-08-19T10:42:49.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.695+0000 7f91acb31640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f91a80a4220 0x7f91a8139800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:49.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91adb33640 1 -- 172.21.15.89:0/135317647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91a80a4600 msgr2=0x7f91a813e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91adb33640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91a80a4600 0x7f91a813e100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f919800d560 tx=0x7f919800da20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9198012c70 con 0x7f91a80a4600 2024-08-19T10:42:49.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:49.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 10 monclient: dump: 2024-08-19T10:42:49.714 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.714 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.714 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.715 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 10 monclient: _finish_auth 0 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91af596640 5 monclient: authenticate success, global_id 5728 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9198019070 con 0x7f91a80a4600 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91af596640 10 monclient: _renew_subs 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91af596640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91af596640 1 -- 172.21.15.89:0/135317647 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a813fcb0 con 0x7f91a80a4600 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91980148a0 con 0x7f91a80a4600 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:49.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919f7fe640 4 set_mon_vals no callback set 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 10 monclient: dump: 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.717 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91af596640 10 monclient: _renew_subs 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91af596640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91af596640 1 -- 172.21.15.89:0/135317647 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a8123eb0 con 0x7f91a80a4600 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91af596640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91af596640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a8124080 0x7f91a813e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91af596640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91af596640 1 -- 172.21.15.89:0/135317647 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f91a80a2310 con 0x7f91a8124080 2024-08-19T10:42:49.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9198014430 con 0x7f91a80a4600 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91ad332640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a8124080 0x7f91a813e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91ad332640 10 monclient: get_auth_request con 0x7f91a8124080 auth_method 0 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91ad332640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91ad332640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91ad332640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f918c03be00 0x7f918c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(154..154 src has 1..154) ==== 9079+0+0 (secure 0 0 0) 0x7f919800f040 con 0x7f91a80a4600 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91ad332640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91ad332640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91ad332640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91acb31640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f918c03be00 0x7f918c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91ad332640 10 monclient(hunting): handle_auth_done global_id 34158 payload 824 2024-08-19T10:42:49.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91ad332640 1 -- 172.21.15.89:0/135317647 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91a80a3b10 con 0x7f91a8124080 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.699+0000 7f91acb31640 10 monclient: get_auth_request con 0x7f918c03be00 auth_method 0 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f91ad332640 1 -- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a8124080 msgr2=0x7f91a813e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f91ad332640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a8124080 0x7f91a813e640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009f80 tx=0x7f91a400b8c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91a401a070 con 0x7f91a8124080 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f919dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f919dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f91acb31640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f918c03be00 0x7f918c03e2c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f91a0000c10 tx=0x7f91a00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f919dffb640 10 monclient: dump: 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.720 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f91a400a980 con 0x7f91a8124080 2024-08-19T10:42:49.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f919dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f919dffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a8124080 msgr2=0x7f91a813e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f919dffb640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a8124080 0x7f91a813e640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009f80 tx=0x7f91a400b8c0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a8124080 msgr2=0x7f91a813e640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.703+0000 7f919dffb640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a8124080 0x7f91a813e640 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009f80 tx=0x7f91a400b8c0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91af596640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91af596640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a813eb80 0x7f91a809d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91af596640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91af596640 1 -- 172.21.15.89:0/135317647 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f91a813ef90 con 0x7f91a813eb80 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a813eb80 0x7f91a809d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 10 monclient: get_auth_request con 0x7f91a813eb80 auth_method 0 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 10 monclient(hunting): handle_auth_done global_id 34167 payload 824 2024-08-19T10:42:49.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 1 -- 172.21.15.89:0/135317647 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f91a806ed10 con 0x7f91a813eb80 2024-08-19T10:42:49.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 1 -- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a813eb80 msgr2=0x7f91a809d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f91ad332640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a813eb80 0x7f91a809d9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91a4017000 tx=0x7f91a4005c60 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91a401a030 con 0x7f91a813eb80 2024-08-19T10:42:49.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f919dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f919dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:42:49.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.711+0000 7f919dffb640 10 monclient: dump: 2024-08-19T10:42:49.724 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.724 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.725 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.726 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7f91a4025410 con 0x7f91a813eb80 2024-08-19T10:42:49.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f919dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:49.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f919dffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:49.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a813eb80 msgr2=0x7f91a809d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f919dffb640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a813eb80 0x7f91a809d9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91a4017000 tx=0x7f91a4005c60 comp rx=0 tx=0).stop 2024-08-19T10:42:49.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f919dffb640 1 -- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a813eb80 msgr2=0x7f91a809d9f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:49.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f919dffb640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a813eb80 0x7f91a809d9f0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91a4017000 tx=0x7f91a4005c60 comp rx=0 tx=0).stop 2024-08-19T10:42:49.726 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 14755, 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:49.727 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 9, 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.728 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.729 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:49.730 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:49.731 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.732 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:49.735 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:49.736 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.738 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.739 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.740 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:49.741 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.743 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.744 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.745 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:49.746 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.747 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.748 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:49.749 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91af596640 1 -- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f918c03be00 msgr2=0x7f918c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91af596640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f918c03be00 0x7f918c03e2c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f91a0000c10 tx=0x7f91a00a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91af596640 10 monclient: shutdown 2024-08-19T10:42:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91af596640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91af596640 1 -- 172.21.15.89:0/135317647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91a80a4600 msgr2=0x7f91a813e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91adb33640 1 -- 172.21.15.89:0/135317647 reap_dead start 2024-08-19T10:42:49.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91af596640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91a80a4600 0x7f91a813e100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f919800d560 tx=0x7f919800da20 comp rx=0 tx=0).stop 2024-08-19T10:42:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91af596640 1 -- 172.21.15.89:0/135317647 shutdown_connections 2024-08-19T10:42:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91af596640 1 --2- 172.21.15.89:0/135317647 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91a80a4600 0x7f91a813e100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91af596640 1 -- 172.21.15.89:0/135317647 >> 172.21.15.89:0/135317647 conn(0x7f91a806de40 msgr2=0x7f91a813cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91af596640 1 -- 172.21.15.89:0/135317647 shutdown_connections 2024-08-19T10:42:49.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.715+0000 7f91af596640 1 -- 172.21.15.89:0/135317647 wait complete. 2024-08-19T10:42:49.752 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:49.752 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:42:49.752 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:42:49.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:49.800+0000 7faf07fff640 1 -- 172.21.15.190:0/3653778388 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}]=0 v155) ==== 119+0+0 (secure 0 0 0) 0x7faf10017020 con 0x7faf200a4720 2024-08-19T10:42:49.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:49.800+0000 7faf07fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-08-19T10:42:49.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:49.800+0000 7faf07fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:49.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.879+0000 7f684c7a3640 10 monclient: get_monmap_and_config 2024-08-19T10:42:49.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.879+0000 7f684c7a3640 10 monclient: build_initial_monmap 2024-08-19T10:42:49.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.879+0000 7f684c7a3640 10 monclient: monmap: 2024-08-19T10:42:49.884 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:49.884 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.884 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:49.884633+0000 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:49.884633+0000 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:49.885 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:49.886 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:49.886 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 1 Processor -- start 2024-08-19T10:42:49.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 1 -- start start 2024-08-19T10:42:49.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 10 monclient: init 2024-08-19T10:42:49.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:49.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 10 monclient: _add_conns ranks=[2,7,10,9,4,1,5,3,0,6,8] 2024-08-19T10:42:49.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68440a5270 0x7f68440a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 10 monclient(hunting): picked mon.noname-b con 0x7f68440a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:49.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f68440a6d30 0x7f68440a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 10 monclient(hunting): picked mon.noname-i con 0x7f68440a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:49.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68441395b0 0x7f6844139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 10 monclient(hunting): picked mon.noname-f con 0x7f68441395b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:49.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f68440a7650 con 0x7f68441395b0 2024-08-19T10:42:49.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f68440a8710 con 0x7f68440a5270 2024-08-19T10:42:49.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f68440a5a40 con 0x7f68440a6d30 2024-08-19T10:42:49.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:49.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684c7a3640 10 monclient(hunting): authenticate will time out at 1219.701275s 2024-08-19T10:42:49.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68440a5270 0x7f68440a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68440a5270 0x7f68440a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40198/0 (socket says 172.21.15.89:40198) 2024-08-19T10:42:49.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 1 -- 172.21.15.89:0/3537230892 learned_addr learned my addr 172.21.15.89:0/3537230892 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:49.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 10 monclient(hunting): get_auth_request con 0x7f68440a5270 auth_method 0 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f6849d3e640 1 --2- 172.21.15.89:0/3537230892 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f68440a6d30 0x7f68440a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f6849d3e640 10 monclient(hunting): get_auth_request con 0x7f68440a6d30 auth_method 0 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f6849d3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f6849d3e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f6849d3e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684ad40640 1 --2- 172.21.15.89:0/3537230892 >> v2:172.21.15.190:3300/0 conn(0x7f68441395b0 0x7f6844139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684ad40640 10 monclient(hunting): get_auth_request con 0x7f68441395b0 auth_method 0 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684ad40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684ad40640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684ad40640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f6849d3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f6849d3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f6849d3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684ad40640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684ad40640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684ad40640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 10 monclient(hunting): handle_auth_done global_id 5681 payload 274 2024-08-19T10:42:49.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 1 -- 172.21.15.89:0/3537230892 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f68440a6d30 msgr2=0x7f68440a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 1 --2- 172.21.15.89:0/3537230892 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f68440a6d30 0x7f68440a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 1 -- 172.21.15.89:0/3537230892 >> v2:172.21.15.190:3300/0 conn(0x7f68441395b0 msgr2=0x7f6844139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 1 --2- 172.21.15.89:0/3537230892 >> v2:172.21.15.190:3300/0 conn(0x7f68441395b0 0x7f6844139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 10 monclient: _finish_hunting 0 2024-08-19T10:42:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 1 monclient: found mon.noname-b 2024-08-19T10:42:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:42:49.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.883+0000 7f684a53f640 1 -- 172.21.15.89:0/3537230892 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68440a81d0 con 0x7f68440a5270 2024-08-19T10:42:49.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684a53f640 1 -- 172.21.15.89:0/3537230892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68440a5270 msgr2=0x7f68440a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684a53f640 1 --2- 172.21.15.89:0/3537230892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68440a5270 0x7f68440a67f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6834009870 tx=0x7f683401c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c31a5daae474a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 1 -- 172.21.15.89:0/3537230892 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68340b1070 con 0x7f68440a5270 2024-08-19T10:42:49.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:42:49.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 10 monclient: dump: 2024-08-19T10:42:49.897 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.898 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.898 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.898 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.898 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.898 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.898 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.898 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.898 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.898 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.898 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.899 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.899 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.899 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.899 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.899 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 10 monclient: _finish_auth 0 2024-08-19T10:42:49.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 10 monclient: _check_auth_tickets 2024-08-19T10:42:49.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:49.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684c7a3640 5 monclient: authenticate success, global_id 5681 2024-08-19T10:42:49.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684c7a3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:49.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 1 -- 172.21.15.89:0/3537230892 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68340ab810 con 0x7f68440a5270 2024-08-19T10:42:49.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:49.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 1 -- 172.21.15.89:0/3537230892 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68340aa300 con 0x7f68440a5270 2024-08-19T10:42:49.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:49.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684953d640 10 monclient: dump: 2024-08-19T10:42:49.900 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.900 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.900 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.900 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.901 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.901 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.901 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.901 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.901 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.901 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.901 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.901 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.901 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.901 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.901 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.902 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684c7a3640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.887+0000 7f684c7a3640 4 set_mon_vals no callback set 2024-08-19T10:42:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient: shutdown 2024-08-19T10:42:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 -- 172.21.15.89:0/3537230892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68440a5270 msgr2=0x7f68440a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 --2- 172.21.15.89:0/3537230892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68440a5270 0x7f68440a67f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6834009870 tx=0x7f683401c420 comp rx=0 tx=0).stop 2024-08-19T10:42:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 -- 172.21.15.89:0/3537230892 shutdown_connections 2024-08-19T10:42:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 --2- 172.21.15.89:0/3537230892 >> v2:172.21.15.190:3300/0 conn(0x7f68441395b0 0x7f6844139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 --2- 172.21.15.89:0/3537230892 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f68440a6d30 0x7f68440a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 --2- 172.21.15.89:0/3537230892 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f68440a5270 0x7f68440a67f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 -- 172.21.15.89:0/3537230892 >> 172.21.15.89:0/3537230892 conn(0x7f6844069a80 msgr2=0x7f68440abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 -- 172.21.15.89:0/3537230892 shutdown_connections 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 -- 172.21.15.89:0/3537230892 wait complete. 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient: build_initial_monmap 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient: monmap: 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:49.903 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 Processor -- start 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 -- start start 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient: init 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:49.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient: _add_conns ranks=[8,7,1,0,4,3,2,6,5] 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f68440a5270 0x7f684413d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient(hunting): picked mon.noname-e con 0x7f68440a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f68440a6d30 0x7f684413d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient(hunting): picked mon.noname-i con 0x7f68440a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68441395b0 0x7f684413db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient(hunting): picked mon.noname-f con 0x7f68441395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f68440a5650 con 0x7f68441395b0 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f684413a350 con 0x7f68440a6d30 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f68440a57d0 con 0x7f68440a5270 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684c7a3640 10 monclient(hunting): authenticate will time out at 1219.710453s 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f68440a5270 0x7f684413d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f68440a5270 0x7f684413d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40556/0 (socket says 172.21.15.89:40556) 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 1 -- 172.21.15.89:0/919798485 learned_addr learned my addr 172.21.15.89:0/919798485 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 10 monclient(hunting): get_auth_request con 0x7f68440a5270 auth_method 0 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f6849d3e640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f68440a6d30 0x7f684413d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684ad40640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68441395b0 0x7f684413db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f6849d3e640 10 monclient(hunting): get_auth_request con 0x7f68440a6d30 auth_method 0 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f6849d3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f6849d3e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f6849d3e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684ad40640 10 monclient(hunting): get_auth_request con 0x7f68441395b0 auth_method 0 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684ad40640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684ad40640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684ad40640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 10 monclient(hunting): handle_auth_done global_id 5606 payload 824 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 1 -- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f68440a6d30 msgr2=0x7f684413d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f68440a6d30 0x7f684413d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 1 -- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68441395b0 msgr2=0x7f684413db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68441395b0 0x7f684413db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 10 monclient: _finish_hunting 0 2024-08-19T10:42:49.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 1 monclient: found mon.noname-e 2024-08-19T10:42:49.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:49.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:49.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.891+0000 7f684a53f640 1 -- 172.21.15.89:0/919798485 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6844139c70 con 0x7f68440a5270 2024-08-19T10:42:49.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684a53f640 1 -- 172.21.15.89:0/919798485 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f68440a5270 msgr2=0x7f684413d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684a53f640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f68440a5270 0x7f684413d080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68340af040 tx=0x7f68340191c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68340b1040 con 0x7f68440a5270 2024-08-19T10:42:49.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:49.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 10 monclient: dump: 2024-08-19T10:42:49.915 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.916 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.917 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.917 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.917 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 10 monclient: _finish_auth 0 2024-08-19T10:42:49.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:49.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:49.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68340ba070 con 0x7f68440a5270 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68340044b0 con 0x7f68440a5270 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 10 monclient: dump: 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.918 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683bfff640 4 set_mon_vals no callback set 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684c7a3640 5 monclient: authenticate success, global_id 5606 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684c7a3640 10 monclient: _renew_subs 2024-08-19T10:42:49.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684c7a3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684c7a3640 1 -- 172.21.15.89:0/919798485 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68440a6a80 con 0x7f68440a5270 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684c7a3640 10 monclient: _renew_subs 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684c7a3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684c7a3640 1 -- 172.21.15.89:0/919798485 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f684409d140 con 0x7f68440a5270 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684c7a3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684c7a3640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d11b0 0x7f68441d1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684c7a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684c7a3640 1 -- 172.21.15.89:0/919798485 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f68440a7110 con 0x7f68441d11b0 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6834004040 con 0x7f68440a5270 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f682803bdf0 0x7f682803e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(155..155 src has 1..155) ==== 9079+0+0 (secure 0 0 0) 0x7f68340e9860 con 0x7f68440a5270 2024-08-19T10:42:49.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f6849d3e640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d11b0 0x7f68441d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f6849d3e640 10 monclient: get_auth_request con 0x7f68441d11b0 auth_method 0 2024-08-19T10:42:49.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f6849d3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f6849d3e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f6849d3e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f6849d3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f6849d3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f6849d3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684ad40640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f682803bdf0 0x7f682803e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f684ad40640 10 monclient: get_auth_request con 0x7f682803bdf0 auth_method 0 2024-08-19T10:42:49.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f6849d3e640 10 monclient(hunting): handle_auth_done global_id 5729 payload 824 2024-08-19T10:42:49.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.895+0000 7f6849d3e640 1 -- 172.21.15.89:0/919798485 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6844064590 con 0x7f68441d11b0 2024-08-19T10:42:49.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f6849d3e640 1 -- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d11b0 msgr2=0x7f68441d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f6849d3e640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d11b0 0x7f68441d1590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f68400115d0 tx=0x7f6840011a90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f684000fe70 con 0x7f68441d11b0 2024-08-19T10:42:49.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f683affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f683affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:49.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f683affd640 10 monclient: dump: 2024-08-19T10:42:49.922 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.922 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.923 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.924 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.924 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f684ad40640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f682803bdf0 0x7f682803e2b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f684413f4d0 tx=0x7f683c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f684000fa00 con 0x7f68441d11b0 2024-08-19T10:42:49.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f683affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:49.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f683affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:49.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d11b0 msgr2=0x7f68441d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f683affd640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d11b0 0x7f68441d1590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f68400115d0 tx=0x7f6840011a90 comp rx=0 tx=0).stop 2024-08-19T10:42:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d11b0 msgr2=0x7f68441d1590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.899+0000 7f683affd640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d11b0 0x7f68441d1590 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f684c7a3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f684c7a3640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d1ad0 0x7f68441327b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f684c7a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:49.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f684c7a3640 1 -- 172.21.15.89:0/919798485 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f684413fcb0 con 0x7f68441d1ad0 2024-08-19T10:42:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d1ad0 0x7f68441327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 10 monclient: get_auth_request con 0x7f68441d1ad0 auth_method 0 2024-08-19T10:42:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 10 monclient(hunting): handle_auth_done global_id 5738 payload 824 2024-08-19T10:42:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 1 -- 172.21.15.89:0/919798485 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f684406c9e0 con 0x7f68441d1ad0 2024-08-19T10:42:49.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 1 -- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d1ad0 msgr2=0x7f68441327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.907+0000 7f6849d3e640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d1ad0 0x7f68441327b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f684001b000 tx=0x7f68400046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6840003e20 con 0x7f68441d1ad0 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f683affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f683affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f683affd640 10 monclient: dump: 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:49.927 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f6840003e20 con 0x7f68441d1ad0 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f683affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f683affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d1ad0 msgr2=0x7f68441327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f683affd640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d1ad0 0x7f68441327b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f684001b000 tx=0x7f68400046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f683affd640 1 -- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d1ad0 msgr2=0x7f68441327b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:49.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f683affd640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68441d1ad0 0x7f68441327b0 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f684001b000 tx=0x7f68400046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.929 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:49.929 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:49.929 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:49.929 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:49.929 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:49.929 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 299678, 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 9, 2024-08-19T10:42:49.930 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:49.931 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:49.931 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:49.931 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.931 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.931 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.931 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.931 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.931 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.931 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.932 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.932 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.932 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.932 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.932 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.932 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:49.932 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:49.932 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.934 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:49.934 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:49.934 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:49.934 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:49.934 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:49.934 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:49.934 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:49.934 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:49.934 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:49.934 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:49.935 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:49.936 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.937 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.937 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.937 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:49.937 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.937 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.938 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:49.938 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.938 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.938 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.938 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:49.938 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:49.938 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.938 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.938 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.940 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.940 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:49.940 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.940 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.940 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.940 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:49.940 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:49.940 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.941 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.941 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.941 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:49.941 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:49.941 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.941 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.941 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.942 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:49.942 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.942 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.943 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:49.943 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.943 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.943 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.943 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:49.943 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:49.943 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.943 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.943 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:49.944 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:49.945 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.948 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.949 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.950 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:49.950 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.950 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.950 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.950 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:49.950 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:49.950 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.950 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.952 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.952 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:49.952 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.952 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.952 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.952 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:49.952 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:49.952 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.953 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.953 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.953 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.953 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:49.953 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:49.953 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:49.953 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.955 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:49.955 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:49.955 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:49.955 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:49.955 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:49.955 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.955 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.955 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:49.955 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:49.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:49.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:49.957 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:49.957 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684c7a3640 1 -- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f682803bdf0 msgr2=0x7f682803e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684c7a3640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f682803bdf0 0x7f682803e2b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f684413f4d0 tx=0x7f683c00b040 comp rx=0 tx=0).stop 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684c7a3640 10 monclient: shutdown 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684c7a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684c7a3640 1 -- 172.21.15.89:0/919798485 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f68440a5270 msgr2=0x7f684413d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684c7a3640 1 --2- 172.21.15.89:0/919798485 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f68440a5270 0x7f684413d080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68340af040 tx=0x7f68340191c0 comp rx=0 tx=0).stop 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684ad40640 1 -- 172.21.15.89:0/919798485 reap_dead start 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684ad40640 1 -- 172.21.15.89:0/919798485 reap_dead start 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684c7a3640 1 -- 172.21.15.89:0/919798485 shutdown_connections 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684c7a3640 1 -- 172.21.15.89:0/919798485 >> 172.21.15.89:0/919798485 conn(0x7f6844069a80 msgr2=0x7f68440ab1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684c7a3640 1 -- 172.21.15.89:0/919798485 shutdown_connections 2024-08-19T10:42:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:49.911+0000 7f684c7a3640 1 -- 172.21.15.89:0/919798485 wait complete. 2024-08-19T10:42:49.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:49.920+0000 7faf25860640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-08-19T10:42:49.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:49.920+0000 7faf25860640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-08-19T10:42:49.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:49.920+0000 7faf25860640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:42:49.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:49.920+0000 7faf25860640 1 -- 172.21.15.190:0/3653778388 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]} v 0) -- 0x7faf2013fcb0 con 0x7faf200a4720 2024-08-19T10:42:49.960 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:49.960 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:42:49.961 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:42:50.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient: get_monmap_and_config 2024-08-19T10:42:50.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient: build_initial_monmap 2024-08-19T10:42:50.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient: monmap: 2024-08-19T10:42:50.084 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:50.084 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.084 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:50.081775+0000 2024-08-19T10:42:50.084 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:50.081775+0000 2024-08-19T10:42:50.084 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 1 Processor -- start 2024-08-19T10:42:50.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 1 -- start start 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient: init 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient: _add_conns ranks=[3,1,4,6,5,9,2,8,0,7,10] 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecf40a3d00 0x7fecf40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient(hunting): picked mon.noname-g con 0x7fecf40a3d00 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fecf40a4620 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecf409b750 0x7fecf413f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient(hunting): picked mon.noname-c con 0x7fecf409b750 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecf409ca30 con 0x7fecf40a4620 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fecf406db50 con 0x7fecf40a3d00 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fecf40a1e50 con 0x7fecf409b750 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfc169640 10 monclient(hunting): authenticate will time out at 1219.898654s 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecf409b750 0x7fecf413f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecf409b750 0x7fecf413f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38482/0 (socket says 172.21.15.89:38482) 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 1 -- 172.21.15.89:0/956912731 learned_addr learned my addr 172.21.15.89:0/956912731 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 10 monclient(hunting): get_auth_request con 0x7fecf409b750 auth_method 0 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecf9f05640 1 --2- 172.21.15.89:0/956912731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecf40a3d00 0x7fecf40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecf9f05640 10 monclient(hunting): get_auth_request con 0x7fecf40a3d00 auth_method 0 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecf9f05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecf9f05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecf9f05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 10 monclient(hunting): handle_auth_done global_id 5737 payload 274 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 1 -- 172.21.15.89:0/956912731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecf40a3d00 msgr2=0x7fecf40a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 1 --2- 172.21.15.89:0/956912731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecf40a3d00 0x7fecf40a40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 1 -- 172.21.15.89:0/956912731 >> v1:172.21.15.190:6789/0 conn(0x7fecf40a4620 legacy=0x7fecf409b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 10 monclient: _finish_hunting 0 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 1 monclient: found mon.noname-c 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:42:50.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.079+0000 7fecfa706640 1 -- 172.21.15.89:0/956912731 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf40a27d0 con 0x7fecf409b750 2024-08-19T10:42:50.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecfa706640 1 -- 172.21.15.89:0/956912731 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecf409b750 msgr2=0x7fecf413f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecfa706640 1 --2- 172.21.15.89:0/956912731 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecf409b750 0x7fecf413f8c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fecec009e80 tx=0x7fecec01c910 comp rx=0 tx=0).ready entity=mon.4 client_cookie=cb4f8b6b7959b09c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 1 -- 172.21.15.89:0/956912731 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fecec0090d0 con 0x7fecf409b750 2024-08-19T10:42:50.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:42:50.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 10 monclient: dump: 2024-08-19T10:42:50.092 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.093 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 10 monclient: _finish_auth 0 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 10 monclient: _check_auth_tickets 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 1 -- 172.21.15.89:0/956912731 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecec0a5b40 con 0x7fecf409b750 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecfc169640 5 monclient: authenticate success, global_id 5737 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecfc169640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecfc169640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 1 -- 172.21.15.89:0/956912731 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fecec0a5cf0 con 0x7fecf409b750 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecfc169640 4 set_mon_vals no callback set 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.083+0000 7fecf8f03640 10 monclient: dump: 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.095 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient: shutdown 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 -- 172.21.15.89:0/956912731 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecf409b750 msgr2=0x7fecf413f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 --2- 172.21.15.89:0/956912731 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecf409b750 0x7fecf413f8c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fecec009e80 tx=0x7fecec01c910 comp rx=0 tx=0).stop 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 -- 172.21.15.89:0/956912731 shutdown_connections 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 --2- 172.21.15.89:0/956912731 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fecf409b750 0x7fecf413f8c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 --2- 172.21.15.89:0/956912731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecf40a3d00 0x7fecf40a40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 -- 172.21.15.89:0/956912731 >> 172.21.15.89:0/956912731 conn(0x7fecf40a4df0 msgr2=0x7fecf40a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:50.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 -- 172.21.15.89:0/956912731 shutdown_connections 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 -- 172.21.15.89:0/956912731 wait complete. 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient: build_initial_monmap 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient: monmap: 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:50.097 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 Processor -- start 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 -- start start 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient: init 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient: _add_conns ranks=[8,1,3,7,4,6,5,2,0] 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecf409b750 0x7fecf413a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient(hunting): picked mon.noname-e con 0x7fecf409b750 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:50.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecf40a3d00 0x7fecf413aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient(hunting): picked mon.noname-f con 0x7fecf40a3d00 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecf40a4620 0x7fecf413af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient(hunting): picked mon.noname-g con 0x7fecf40a4620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fecf40a4b80 con 0x7fecf40a3d00 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fecf40a4a00 con 0x7fecf40a4620 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fecf40a1a00 con 0x7fecf409b750 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfc169640 10 monclient(hunting): authenticate will time out at 1219.906332s 2024-08-19T10:42:50.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecf409b750 0x7fecf413a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecf409b750 0x7fecf413a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:40572/0 (socket says 172.21.15.89:40572) 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 1 -- 172.21.15.89:0/1743180874 learned_addr learned my addr 172.21.15.89:0/1743180874 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 10 monclient(hunting): get_auth_request con 0x7fecf409b750 auth_method 0 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9704640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecf40a3d00 0x7fecf413aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfa706640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecf40a4620 0x7fecf413af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9704640 10 monclient(hunting): get_auth_request con 0x7fecf40a3d00 auth_method 0 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9704640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9704640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9704640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfa706640 10 monclient(hunting): get_auth_request con 0x7fecf40a4620 auth_method 0 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfa706640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfa706640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecfa706640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9704640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9704640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9704640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 10 monclient(hunting): handle_auth_done global_id 5615 payload 824 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 1 -- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecf40a4620 msgr2=0x7fecf413af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecf40a4620 0x7fecf413af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 1 -- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecf40a3d00 msgr2=0x7fecf413aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecf40a3d00 0x7fecf413aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 10 monclient: _finish_hunting 0 2024-08-19T10:42:50.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 1 monclient: found mon.noname-e 2024-08-19T10:42:50.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:50.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:50.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.087+0000 7fecf9f05640 1 -- 172.21.15.89:0/1743180874 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf413dca0 con 0x7fecf409b750 2024-08-19T10:42:50.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecf9f05640 1 -- 172.21.15.89:0/1743180874 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecf409b750 msgr2=0x7fecf413a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecf9f05640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecf409b750 0x7fecf413a4f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fece400d800 tx=0x7fece400dcc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fece400bd40 con 0x7fecf409b750 2024-08-19T10:42:50.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:50.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 10 monclient: dump: 2024-08-19T10:42:50.105 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.105 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.105 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.105 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.105 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece4019070 con 0x7fecf409b750 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fece40148a0 con 0x7fecf409b750 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecea7fc640 10 monclient: dump: 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.107 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecfc169640 5 monclient: authenticate success, global_id 5615 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecfc169640 10 monclient: _renew_subs 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecebfff640 4 set_mon_vals no callback set 2024-08-19T10:42:50.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecfc169640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecfc169640 1 -- 172.21.15.89:0/1743180874 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf413df80 con 0x7fecf409b750 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecfc169640 10 monclient: _renew_subs 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecfc169640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.091+0000 7fecfc169640 1 -- 172.21.15.89:0/1743180874 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf413d9c0 con 0x7fecf409b750 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecfc169640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecfc169640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf40992c0 0x7fecf40996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecfc169640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecfc169640 1 -- 172.21.15.89:0/1743180874 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fecf40a40e0 con 0x7fecf40992c0 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fece4014430 con 0x7fecf409b750 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecf9704640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf40992c0 0x7fecf40996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecea7fc640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fecd803be30 0x7fecd803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecf9704640 10 monclient: get_auth_request con 0x7fecf40992c0 auth_method 0 2024-08-19T10:42:50.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecf9704640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecf9704640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecf9704640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(155..155 src has 1..155) ==== 9079+0+0 (secure 0 0 0) 0x7fece400f040 con 0x7fecf409b750 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecf9704640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecf9704640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecf9704640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecfa706640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fecd803be30 0x7fecd803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecfa706640 10 monclient: get_auth_request con 0x7fecd803be30 auth_method 0 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecf9704640 10 monclient(hunting): handle_auth_done global_id 14929 payload 824 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.095+0000 7fecf9704640 1 -- 172.21.15.89:0/1743180874 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fecf4064590 con 0x7fecf40992c0 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecf9704640 1 -- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf40992c0 msgr2=0x7fecf40996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecf9704640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf40992c0 0x7fecf40996a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fecf40709b0 tx=0x7fecf000ec00 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecfa706640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fecd803be30 0x7fecd803e2f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fecf40a5ba0 tx=0x7fecec0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fecf0017070 con 0x7fecf40992c0 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecea7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecea7fc640 10 monclient: dump: 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.111 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fecf000c9b0 con 0x7fecf40992c0 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecea7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecea7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf40992c0 msgr2=0x7fecf40996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecea7fc640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf40992c0 0x7fecf40996a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fecf40709b0 tx=0x7fecf000ec00 comp rx=0 tx=0).stop 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf40992c0 msgr2=0x7fecf40996a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.099+0000 7fecea7fc640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf40992c0 0x7fecf40996a0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fecf40709b0 tx=0x7fecf000ec00 comp rx=0 tx=0).stop 2024-08-19T10:42:50.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.103+0000 7fecfc169640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:42:50.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.103+0000 7fecfc169640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf4099be0 0x7fecf41d9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.103+0000 7fecfc169640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.103+0000 7fecfc169640 1 -- 172.21.15.89:0/1743180874 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fecf413fbd0 con 0x7fecf4099be0 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf4099be0 0x7fecf41d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 10 monclient: get_auth_request con 0x7fecf4099be0 auth_method 0 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 10 monclient(hunting): handle_auth_done global_id 14938 payload 824 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 1 -- 172.21.15.89:0/1743180874 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fecf413c300 con 0x7fecf4099be0 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 1 -- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf4099be0 msgr2=0x7fecf41d9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecf9704640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf4099be0 0x7fecf41d9400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fecf40709b0 tx=0x7fecf0004700 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fecf0017030 con 0x7fecf4099be0 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecea7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.107+0000 7fecea7fc640 10 monclient: dump: 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.116 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7fecf00036c0 con 0x7fecf4099be0 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecea7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecea7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf4099be0 msgr2=0x7fecf41d9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecea7fc640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf4099be0 0x7fecf41d9400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fecf40709b0 tx=0x7fecf0004700 comp rx=0 tx=0).stop 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecea7fc640 1 -- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf4099be0 msgr2=0x7fecf41d9400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:50.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecea7fc640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fecf4099be0 0x7fecf41d9400 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fecf40709b0 tx=0x7fecf0004700 comp rx=0 tx=0).stop 2024-08-19T10:42:50.118 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:50.118 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:50.118 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:50.118 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:42:50.118 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 52, 2024-08-19T10:42:50.118 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 147696, 2024-08-19T10:42:50.118 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:42:50.118 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:42:50.118 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:42:50.118 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 9, 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:50.119 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:42:50.120 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:42:50.121 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:42:50.122 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:42:50.123 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:50.124 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:50.125 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.126 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.127 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.128 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:50.129 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.130 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:50.131 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:50.132 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:50.133 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:50.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.133 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:42:50.133 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:42:50.133 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:50.133 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:50.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.133 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:42:50.134 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.136 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.137 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:42:50.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:42:50.139 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:42:50.140 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:42:50.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:42:50.141 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:42:50.141 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:42:50.141 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:42:50.141 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:42:50.141 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:42:50.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:42:50.141 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:42:50.141 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfc169640 1 -- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fecd803be30 msgr2=0x7fecd803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfc169640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fecd803be30 0x7fecd803e2f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fecf40a5ba0 tx=0x7fecec0a7070 comp rx=0 tx=0).stop 2024-08-19T10:42:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfc169640 10 monclient: shutdown 2024-08-19T10:42:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfc169640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfc169640 1 -- 172.21.15.89:0/1743180874 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecf409b750 msgr2=0x7fecf413a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfc169640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecf409b750 0x7fecf413a4f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fece400d800 tx=0x7fece400dcc0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfa706640 1 -- 172.21.15.89:0/1743180874 reap_dead start 2024-08-19T10:42:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfc169640 1 -- 172.21.15.89:0/1743180874 shutdown_connections 2024-08-19T10:42:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfc169640 1 --2- 172.21.15.89:0/1743180874 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fecf409b750 0x7fecf413a4f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfc169640 1 -- 172.21.15.89:0/1743180874 >> 172.21.15.89:0/1743180874 conn(0x7fecf40a4df0 msgr2=0x7fecf40a61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:50.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfc169640 1 -- 172.21.15.89:0/1743180874 shutdown_connections 2024-08-19T10:42:50.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.111+0000 7fecfc169640 1 -- 172.21.15.89:0/1743180874 wait complete. 2024-08-19T10:42:50.144 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:42:50.144 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['f', 'c', 'd'] 2024-08-19T10:42:50.144 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-08-19T10:42:50.144 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-08-19T10:42:50.144 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f store 2024-08-19T10:42:50.144 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f sync_force --yes-i-really-mean-it 2024-08-19T10:42:50.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.259+0000 7f5fe9f4b640 10 monclient: get_monmap_and_config 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.259+0000 7f5fe9f4b640 10 monclient: build_initial_monmap 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.259+0000 7f5fe9f4b640 10 monclient: monmap: 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:50.263185+0000 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:50.263185+0000 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:50.264 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:50.264 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.259+0000 7f5fe9f4b640 1 Processor -- start 2024-08-19T10:42:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.259+0000 7f5fe9f4b640 1 -- start start 2024-08-19T10:42:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.259+0000 7f5fe9f4b640 10 monclient: init 2024-08-19T10:42:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 10 monclient: _add_conns ranks=[0,6,8,3,1,2,9,5,10,4,7] 2024-08-19T10:42:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5fe40a3a70 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 0x7f5fe413eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 10 monclient(hunting): picked mon.noname-d con 0x7f5fe40a0330 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5fe40a4560 0x7f5fe41410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 10 monclient(hunting): picked mon.noname-e con 0x7f5fe40a4560 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe40a11f0 con 0x7f5fe40a3a70 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f5fe4074320 con 0x7f5fe40a0330 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f5fe40a5490 con 0x7f5fe40a4560 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe9f4b640 10 monclient(hunting): authenticate will time out at 1220.080562s 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe37fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5fe40a3a70 0x7f5fe40a3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52938/0 (socket says 172.21.15.89:52938) 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe37fe640 1 -- 172.21.15.89:0/3306128182 learned_addr learned my addr 172.21.15.89:0/3306128182 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe2ffd640 1 --2- 172.21.15.89:0/3306128182 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 0x7f5fe413eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 1 --2- 172.21.15.89:0/3306128182 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5fe40a4560 0x7f5fe41410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe2ffd640 10 monclient(hunting): get_auth_request con 0x7f5fe40a0330 auth_method 0 2024-08-19T10:42:50.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 10 monclient(hunting): get_auth_request con 0x7f5fe40a4560 auth_method 0 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 10 monclient(hunting): handle_auth_done global_id 5624 payload 274 2024-08-19T10:42:50.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 1 -- 172.21.15.89:0/3306128182 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 msgr2=0x7f5fe413eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 1 --2- 172.21.15.89:0/3306128182 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 0x7f5fe413eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 1 -- 172.21.15.89:0/3306128182 >> v1:172.21.15.89:6789/0 conn(0x7f5fe40a3a70 legacy=0x7f5fe40a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 1 monclient: found mon.noname-e 2024-08-19T10:42:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe3fff640 1 -- 172.21.15.89:0/3306128182 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe40a0950 con 0x7f5fe40a4560 2024-08-19T10:42:50.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.263+0000 7f5fe2ffd640 1 --2- 172.21.15.89:0/3306128182 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 0x7f5fe413eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe3fff640 1 -- 172.21.15.89:0/3306128182 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5fe40a4560 msgr2=0x7f5fe41410f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe3fff640 1 --2- 172.21.15.89:0/3306128182 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5fe40a4560 0x7f5fe41410f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0097f0 tx=0x7f5fdc01c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=386376a5b277fb20 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 1 -- 172.21.15.89:0/3306128182 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5fdc0b0070 con 0x7f5fe40a4560 2024-08-19T10:42:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 10 monclient: dump: 2024-08-19T10:42:50.274 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.275 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 10 monclient: _finish_auth 0 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 1 -- 172.21.15.89:0/3306128182 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fdc0aa810 con 0x7f5fe40a4560 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 5 monclient: authenticate success, global_id 5624 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 1 -- 172.21.15.89:0/3306128182 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5fdc0aae30 con 0x7f5fe40a4560 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 4 set_mon_vals no callback set 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:50.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe27fc640 10 monclient: dump: 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.277 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 10 monclient: shutdown 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/3306128182 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5fe40a4560 msgr2=0x7f5fe41410f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 1 --2- 172.21.15.89:0/3306128182 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5fe40a4560 0x7f5fe41410f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0097f0 tx=0x7f5fdc01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/3306128182 shutdown_connections 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 1 --2- 172.21.15.89:0/3306128182 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5fe40a4560 0x7f5fe41410f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 1 --2- 172.21.15.89:0/3306128182 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 0x7f5fe413eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/3306128182 >> 172.21.15.89:0/3306128182 conn(0x7f5fe406de40 msgr2=0x7f5fe406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/3306128182 shutdown_connections 2024-08-19T10:42:50.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/3306128182 wait complete. 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 10 monclient: build_initial_monmap 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 10 monclient: monmap: 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:50.279 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 1 Processor -- start 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 1 -- start start 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.267+0000 7f5fe9f4b640 10 monclient: init 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient: _add_conns ranks=[6,7,2,8,4,1,3,0,5] 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 0x7f5fe41d1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient(hunting): picked mon.noname-d con 0x7f5fe40a0330 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5fe40a3a70 0x7f5fe41d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient(hunting): picked mon.noname-i con 0x7f5fe40a3a70 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fe40a4560 0x7f5fe4126320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient(hunting): picked mon.noname-b con 0x7f5fe40a4560 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe4144060 con 0x7f5fe40a4560 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f5fe40a32c0 con 0x7f5fe40a0330 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5fe41441e0 con 0x7f5fe40a3a70 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient(hunting): authenticate will time out at 1220.088238s 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 0x7f5fe41d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fe40a4560 0x7f5fe4126320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 0x7f5fe41d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54556/0 (socket says 172.21.15.89:54556) 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 1 -- 172.21.15.89:0/1909010318 learned_addr learned my addr 172.21.15.89:0/1909010318 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fe40a4560 0x7f5fe4126320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40200/0 (socket says 172.21.15.89:40200) 2024-08-19T10:42:50.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 10 monclient(hunting): get_auth_request con 0x7f5fe40a0330 auth_method 0 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe3fff640 10 monclient(hunting): get_auth_request con 0x7f5fe40a4560 auth_method 0 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe2ffd640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5fe40a3a70 0x7f5fe41d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe2ffd640 10 monclient(hunting): get_auth_request con 0x7f5fe40a3a70 auth_method 0 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 10 monclient(hunting): handle_auth_done global_id 5604 payload 824 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 1 -- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5fe40a3a70 msgr2=0x7f5fe41d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5fe40a3a70 0x7f5fe41d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 1 -- 172.21.15.89:0/1909010318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fe40a4560 msgr2=0x7f5fe4126320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fe40a4560 0x7f5fe4126320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 1 monclient: found mon.noname-d 2024-08-19T10:42:50.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:50.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:50.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 1 -- 172.21.15.89:0/1909010318 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe4129bb0 con 0x7f5fe40a0330 2024-08-19T10:42:50.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe3fff640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5fe40a4560 0x7f5fe4126320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 1 -- 172.21.15.89:0/1909010318 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 msgr2=0x7f5fe41d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe37fe640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 0x7f5fe41d1380 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400cc40 tx=0x7f5fd400a430 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5fd400ad50 con 0x7f5fe40a0330 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 10 monclient: dump: 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.287 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 10 monclient: _finish_auth 0 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:42:50.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 5 monclient: authenticate success, global_id 5604 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd4012870 con 0x7f5fe40a0330 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5fd4007070 con 0x7f5fe40a0330 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient: _renew_subs 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/1909010318 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe4129e90 con 0x7f5fe40a0330 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe1ffb640 4 set_mon_vals no callback set 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fcbfff640 10 monclient: dump: 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.289 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient: _renew_subs 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.271+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/1909010318 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe41298d0 con 0x7f5fe40a0330 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe9f4b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe9f4b640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127020 0x7f5fe402d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe9f4b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/1909010318 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe40a2270 con 0x7f5fe4127020 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5fd4012400 con 0x7f5fe40a0330 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127020 0x7f5fe402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 10 monclient: get_auth_request con 0x7f5fe4127020 auth_method 0 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fcbfff640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5fd003be00 0x7f5fd003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(155..155 src has 1..155) ==== 9079+0+0 (secure 0 0 0) 0x7f5fd4050270 con 0x7f5fe40a0330 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe3fff640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5fd003be00 0x7f5fd003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe3fff640 10 monclient: get_auth_request con 0x7f5fd003be00 auth_method 0 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 10 monclient(hunting): handle_auth_done global_id 15031 payload 824 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 1 -- 172.21.15.89:0/1909010318 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5fe40a3e50 con 0x7f5fe4127020 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 1 -- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127020 msgr2=0x7f5fe402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe2ffd640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127020 0x7f5fe402d1d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40a68b0 tx=0x7f5fd8011a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5fd8015dd0 con 0x7f5fe4127020 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fcbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fcbfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:50.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fcbfff640 10 monclient: dump: 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.293 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.294 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.275+0000 7f5fe3fff640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5fd003be00 0x7f5fd003e2c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0097f0 tx=0x7f5fdc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.279+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5fd8015960 con 0x7f5fe4127020 2024-08-19T10:42:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.279+0000 7f5fcbfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.279+0000 7f5fcbfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.279+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127020 msgr2=0x7f5fe402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.279+0000 7f5fcbfff640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127020 0x7f5fe402d1d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40a68b0 tx=0x7f5fd8011a60 comp rx=0 tx=0).stop 2024-08-19T10:42:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.279+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127020 msgr2=0x7f5fe402d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:50.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.279+0000 7f5fcbfff640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127020 0x7f5fe402d1d0 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40a68b0 tx=0x7f5fd8011a60 comp rx=0 tx=0).stop 2024-08-19T10:42:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.283+0000 7f5fe9f4b640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:42:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.283+0000 7f5fe9f4b640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127400 0x7f5fe402d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.283+0000 7f5fe9f4b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.283+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/1909010318 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe409f240 con 0x7f5fe4127400 2024-08-19T10:42:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.283+0000 7f5fe2ffd640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127400 0x7f5fe402d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.283+0000 7f5fe2ffd640 10 monclient: get_auth_request con 0x7f5fe4127400 auth_method 0 2024-08-19T10:42:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.283+0000 7f5fe2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.283+0000 7f5fe2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.283+0000 7f5fe2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.287+0000 7f5fe2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.287+0000 7f5fe2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.287+0000 7f5fe2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.287+0000 7f5fe2ffd640 10 monclient(hunting): handle_auth_done global_id 15040 payload 824 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.287+0000 7f5fe2ffd640 1 -- 172.21.15.89:0/1909010318 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7f5fe4064590 con 0x7f5fe4127400 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.287+0000 7f5fe2ffd640 1 -- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127400 msgr2=0x7f5fe402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.287+0000 7f5fe2ffd640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127400 0x7f5fe402d710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40a68b0 tx=0x7f5fd80046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.287+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5fd80066c0 con 0x7f5fe4127400 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.287+0000 7f5fcbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.287+0000 7f5fcbfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.287+0000 7f5fcbfff640 10 monclient: dump: 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.297 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7f5fd8026070 con 0x7f5fe4127400 2024-08-19T10:42:50.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fcbfff640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:42:50.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fcbfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:50.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127400 msgr2=0x7f5fe402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fcbfff640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127400 0x7f5fe402d710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40a68b0 tx=0x7f5fd80046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fcbfff640 1 -- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127400 msgr2=0x7f5fe402d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:50.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fcbfff640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4127400 0x7f5fe402d710 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40a68b0 tx=0x7f5fd80046d0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.299 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: "ret": 0, 2024-08-19T10:42:50.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-08-19T10:42:50.299 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:50.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5fd003be00 msgr2=0x7f5fd003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe9f4b640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5fd003be00 0x7f5fd003e2c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0097f0 tx=0x7f5fdc0a6000 comp rx=0 tx=0).stop 2024-08-19T10:42:50.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe9f4b640 10 monclient: shutdown 2024-08-19T10:42:50.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe9f4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:50.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/1909010318 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 msgr2=0x7f5fe41d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe9f4b640 1 --2- 172.21.15.89:0/1909010318 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5fe40a0330 0x7f5fe41d1380 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400cc40 tx=0x7f5fd400a430 comp rx=0 tx=0).stop 2024-08-19T10:42:50.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe3fff640 1 -- 172.21.15.89:0/1909010318 reap_dead start 2024-08-19T10:42:50.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe3fff640 1 -- 172.21.15.89:0/1909010318 reap_dead start 2024-08-19T10:42:50.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/1909010318 shutdown_connections 2024-08-19T10:42:50.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/1909010318 >> 172.21.15.89:0/1909010318 conn(0x7f5fe406de40 msgr2=0x7f5fe409f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:50.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/1909010318 shutdown_connections 2024-08-19T10:42:50.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.291+0000 7f5fe9f4b640 1 -- 172.21.15.89:0/1909010318 wait complete. 2024-08-19T10:42:50.307 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-08-19T10:42:50.308 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-08-19T10:42:50.308 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:42:50.333 INFO:tasks.ceph.mon.f:Stopped 2024-08-19T10:42:50.333 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-08-19T10:42:50.333 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-08-19T10:42:50.333 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-08-19T10:42:50.333 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:42:50.344 INFO:tasks.ceph.mon.c:Stopped 2024-08-19T10:42:50.344 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d 2024-08-19T10:42:50.344 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d store 2024-08-19T10:42:50.345 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d sync_force --yes-i-really-mean-it 2024-08-19T10:42:50.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient: get_monmap_and_config 2024-08-19T10:42:50.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient: build_initial_monmap 2024-08-19T10:42:50.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient: monmap: 2024-08-19T10:42:50.498 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:50.498 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:50.498889+0000 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:50.498889+0000 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:50.499 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:50.500 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:50.500 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 1 Processor -- start 2024-08-19T10:42:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 1 -- start start 2024-08-19T10:42:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient: init 2024-08-19T10:42:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient: _add_conns ranks=[3,10,4,2,0,8,7,6,1,9,5] 2024-08-19T10:42:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40a67d0 0x7fdab40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient(hunting): picked mon.noname-g con 0x7fdab40a67d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:42:50.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdab40a6bb0 0x7fdab40a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient(hunting): picked mon.noname-f con 0x7fdab40a6bb0 addr v2:172.21.15.190:3300/0 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdab4139590 0x7fdab4139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient(hunting): picked mon.noname-c con 0x7fdab4139590 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fdab40a8740 con 0x7fdab40a6bb0 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdab40a3380 con 0x7fdab40a67d0 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fdab40a8250 con 0x7fdab4139590 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdaba8f5640 10 monclient(hunting): authenticate will time out at 1220.315395s 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab8e92640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdab4139590 msgr2=0x7fdab4139a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:42:50.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab8e92640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdab4139590 0x7fdab4139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab37fe640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fdab40a6bb0 msgr2=0x7fdab40a6f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab37fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdab40a6bb0 0x7fdab40a6f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40a67d0 0x7fdab40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40a67d0 0x7fdab40a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:59698/0 (socket says 172.21.15.89:59698) 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab3fff640 1 -- 172.21.15.89:0/3745592861 learned_addr learned my addr 172.21.15.89:0/3745592861 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab3fff640 10 monclient(hunting): get_auth_request con 0x7fdab40a67d0 auth_method 0 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.495+0000 7fdab3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.499+0000 7fdab3fff640 10 monclient(hunting): handle_auth_done global_id 34194 payload 274 2024-08-19T10:42:50.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.499+0000 7fdab3fff640 1 -- 172.21.15.89:0/3745592861 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdab4139590 msgr2=0x7fdab4139a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:50.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.499+0000 7fdab3fff640 1 --2- 172.21.15.89:0/3745592861 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdab4139590 0x7fdab4139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.499+0000 7fdab3fff640 1 -- 172.21.15.89:0/3745592861 >> v2:172.21.15.190:3300/0 conn(0x7fdab40a6bb0 msgr2=0x7fdab40a6f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:50.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.499+0000 7fdab3fff640 1 --2- 172.21.15.89:0/3745592861 >> v2:172.21.15.190:3300/0 conn(0x7fdab40a6bb0 0x7fdab40a6f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.499+0000 7fdab3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:50.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.499+0000 7fdab3fff640 1 monclient: found mon.noname-g 2024-08-19T10:42:50.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.499+0000 7fdab3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:50.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.499+0000 7fdab3fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:42:50.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.499+0000 7fdab3fff640 1 -- 172.21.15.89:0/3745592861 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab40a4cc0 con 0x7fdab40a67d0 2024-08-19T10:42:50.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab3fff640 1 -- 172.21.15.89:0/3745592861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40a67d0 msgr2=0x7fdab40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab3fff640 1 --2- 172.21.15.89:0/3745592861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40a67d0 0x7fdab40a5500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80098a0 tx=0x7fdaa801c4d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=e008e8235160bb0e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 1 -- 172.21.15.89:0/3745592861 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaa80b0070 con 0x7fdab40a67d0 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 10 monclient: dump: 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.507 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:50.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 1 -- 172.21.15.89:0/3745592861 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa80aa810 con 0x7fdab40a67d0 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 1 -- 172.21.15.89:0/3745592861 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaa80aae30 con 0x7fdab40a67d0 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 5 monclient: authenticate success, global_id 34194 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 4 set_mon_vals no callback set 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab2ffd640 10 monclient: dump: 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.509 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient: shutdown 2024-08-19T10:42:50.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:50.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 -- 172.21.15.89:0/3745592861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40a67d0 msgr2=0x7fdab40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 --2- 172.21.15.89:0/3745592861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40a67d0 0x7fdab40a5500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80098a0 tx=0x7fdaa801c4d0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 -- 172.21.15.89:0/3745592861 shutdown_connections 2024-08-19T10:42:50.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 --2- 172.21.15.89:0/3745592861 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdab4139590 0x7fdab4139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 --2- 172.21.15.89:0/3745592861 >> v2:172.21.15.190:3300/0 conn(0x7fdab40a6bb0 0x7fdab40a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 --2- 172.21.15.89:0/3745592861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40a67d0 0x7fdab40a5500 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 -- 172.21.15.89:0/3745592861 >> 172.21.15.89:0/3745592861 conn(0x7fdab4069ae0 msgr2=0x7fdab40abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:50.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 -- 172.21.15.89:0/3745592861 shutdown_connections 2024-08-19T10:42:50.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 -- 172.21.15.89:0/3745592861 wait complete. 2024-08-19T10:42:50.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient: build_initial_monmap 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient: monmap: 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:50.517 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 Processor -- start 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 -- start start 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient: init 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient: _add_conns ranks=[6,1,7,5,0,4,8,3,2] 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab40a67d0 0x7fdab409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient(hunting): picked mon.noname-d con 0x7fdab40a67d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab40a6bb0 0x7fdab409cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient(hunting): picked mon.noname-f con 0x7fdab40a6bb0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdab4139590 0x7fdab409d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient(hunting): picked mon.noname-i con 0x7fdab4139590 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4143390 con 0x7fdab40a6bb0 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fdab4143210 con 0x7fdab40a67d0 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fdab4143510 con 0x7fdab4139590 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:50.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdaba8f5640 10 monclient(hunting): authenticate will time out at 1220.323863s 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab3fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab40a67d0 0x7fdab409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.503+0000 7fdab3fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab40a67d0 0x7fdab409c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54568/0 (socket says 172.21.15.89:54568) 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 1 -- 172.21.15.89:0/1127214216 learned_addr learned my addr 172.21.15.89:0/1127214216 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 10 monclient(hunting): get_auth_request con 0x7fdab40a67d0 auth_method 0 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab37fe640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab40a6bb0 msgr2=0x7fdab409cf20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab37fe640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab40a6bb0 0x7fdab409cf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 10 monclient(hunting): handle_auth_done global_id 5613 payload 824 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdab4139590 msgr2=0x7fdab409d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab8e92640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdab4139590 0x7fdab409d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdab4139590 0x7fdab409d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab40a6bb0 msgr2=0x7fdab409cf20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab40a6bb0 0x7fdab409cf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 1 monclient: found mon.noname-d 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 1 -- 172.21.15.89:0/1127214216 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab409f610 con 0x7fdab40a67d0 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab40a67d0 msgr2=0x7fdab409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab3fff640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab40a67d0 0x7fdab409c9e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdaa801c300 tx=0x7fdaa8019270 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaa80b0070 con 0x7fdab40a67d0 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 10 monclient: dump: 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.522 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 10 monclient: _finish_auth 0 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa80b1040 con 0x7fdab40a67d0 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaa80aae40 con 0x7fdab40a67d0 2024-08-19T10:42:50.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab0ff9640 10 monclient: dump: 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.524 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdab1ffb640 4 set_mon_vals no callback set 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdaba8f5640 5 monclient: authenticate success, global_id 5613 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdaba8f5640 10 monclient: _renew_subs 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdaba8f5640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdaba8f5640 1 -- 172.21.15.89:0/1127214216 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab40a9450 con 0x7fdab40a67d0 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdaba8f5640 10 monclient: _renew_subs 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdaba8f5640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdaba8f5640 1 -- 172.21.15.89:0/1127214216 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab413f5b0 con 0x7fdab40a67d0 2024-08-19T10:42:50.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdaba8f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdaba8f5640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab4133cf0 0x7fdab402d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdaba8f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.507+0000 7fdaba8f5640 1 -- 172.21.15.89:0/1127214216 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fdab40a6f90 con 0x7fdab4133cf0 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab4133cf0 0x7fdab402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 10 monclient: get_auth_request con 0x7fdab4133cf0 auth_method 0 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fdaa80bc060 con 0x7fdab40a67d0 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 10 monclient(hunting): handle_auth_done global_id 5622 payload 824 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 1 -- 172.21.15.89:0/1127214216 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdab4064590 con 0x7fdab4133cf0 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdaa003be30 0x7fdaa003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(155..155 src has 1..155) ==== 9079+0+0 (secure 0 0 0) 0x7fdaa80e8fa0 con 0x7fdab40a67d0 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab8e92640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdaa003be30 0x7fdaa003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab8e92640 10 monclient: get_auth_request con 0x7fdaa003be30 auth_method 0 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab4133cf0 msgr2=0x7fdab402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab37fe640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab4133cf0 0x7fdab402d1d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdab413fc20 tx=0x7fdaa400b8e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaa4016070 con 0x7fdab4133cf0 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 10 monclient: dump: 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.527 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.528 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdaa4005070 con 0x7fdab4133cf0 2024-08-19T10:42:50.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:50.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:50.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab4133cf0 msgr2=0x7fdab402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab4133cf0 0x7fdab402d1d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdab413fc20 tx=0x7fdaa400b8e0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab4133cf0 msgr2=0x7fdab402d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:50.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab0ff9640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab4133cf0 0x7fdab402d1d0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdab413fc20 tx=0x7fdaa400b8e0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.511+0000 7fdab8e92640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdaa003be30 0x7fdaa003e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdab41406a0 tx=0x7fdaac0071c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdaba8f5640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdaba8f5640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab41340d0 0x7fdab402d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdaba8f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdaba8f5640 1 -- 172.21.15.89:0/1127214216 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fdab4139110 con 0x7fdab41340d0 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdab37fe640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab41340d0 0x7fdab402d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdab37fe640 10 monclient: get_auth_request con 0x7fdab41340d0 auth_method 0 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdab37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdab37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdab37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdab37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdab37fe640 10 monclient(hunting): handle_auth_done global_id 5631 payload 824 2024-08-19T10:42:50.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.519+0000 7fdab37fe640 1 -- 172.21.15.89:0/1127214216 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7fdab40a24e0 con 0x7fdab41340d0 2024-08-19T10:42:50.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab37fe640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab41340d0 msgr2=0x7fdab402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab37fe640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab41340d0 0x7fdab402d710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4013000 tx=0x7fdaa40059e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaa4026080 con 0x7fdab41340d0 2024-08-19T10:42:50.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:42:50.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab0ff9640 10 monclient: dump: 2024-08-19T10:42:50.532 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.533 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7fdaa4018830 con 0x7fdab41340d0 2024-08-19T10:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab0ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab41340d0 msgr2=0x7fdab402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab0ff9640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab41340d0 0x7fdab402d710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4013000 tx=0x7fdaa40059e0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab0ff9640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab41340d0 msgr2=0x7fdab402d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:42:50.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab0ff9640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab41340d0 0x7fdab402d710 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4013000 tx=0x7fdaa40059e0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.535 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:42:50.535 INFO:teuthology.orchestra.run.smithi089.stdout: "ret": 0, 2024-08-19T10:42:50.535 INFO:teuthology.orchestra.run.smithi089.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-08-19T10:42:50.535 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdaba8f5640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdaa003be30 msgr2=0x7fdaa003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdaba8f5640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdaa003be30 0x7fdaa003e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdab41406a0 tx=0x7fdaac0071c0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdaba8f5640 10 monclient: shutdown 2024-08-19T10:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdaba8f5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdaba8f5640 1 -- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab40a67d0 msgr2=0x7fdab409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdaba8f5640 1 --2- 172.21.15.89:0/1127214216 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdab40a67d0 0x7fdab409c9e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdaa801c300 tx=0x7fdaa8019270 comp rx=0 tx=0).stop 2024-08-19T10:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab8e92640 1 -- 172.21.15.89:0/1127214216 reap_dead start 2024-08-19T10:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdab8e92640 1 -- 172.21.15.89:0/1127214216 reap_dead start 2024-08-19T10:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdaba8f5640 1 -- 172.21.15.89:0/1127214216 shutdown_connections 2024-08-19T10:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdaba8f5640 1 -- 172.21.15.89:0/1127214216 >> 172.21.15.89:0/1127214216 conn(0x7fdab4069ae0 msgr2=0x7fdab4134dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:50.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdaba8f5640 1 -- 172.21.15.89:0/1127214216 shutdown_connections 2024-08-19T10:42:50.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.523+0000 7fdaba8f5640 1 -- 172.21.15.89:0/1127214216 wait complete. 2024-08-19T10:42:50.540 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-08-19T10:42:50.540 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-08-19T10:42:50.540 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:42:50.552 INFO:tasks.ceph.mon.d:Stopped 2024-08-19T10:42:50.552 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-08-19T10:42:50.552 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:42:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient: get_monmap_and_config 2024-08-19T10:42:50.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient: build_initial_monmap 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient: monmap: 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:50.706152+0000 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:50.706152+0000 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:50.706 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:50.707 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:50.707 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 1 Processor -- start 2024-08-19T10:42:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 1 -- start start 2024-08-19T10:42:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient: init 2024-08-19T10:42:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient: _add_conns ranks=[5,0,1,6,9,4,3,2,8,7,10] 2024-08-19T10:42:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a5270 0x7fc01c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient(hunting): picked mon.noname-h con 0x7fc01c0a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:50.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc01c0a6d30 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:50.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc01c0a7110 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:50.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01c0a8710 con 0x7fc01c0a6d30 2024-08-19T10:42:50.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01c13a3d0 con 0x7fc01c0a7110 2024-08-19T10:42:50.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc01c0a5a40 con 0x7fc01c0a5270 2024-08-19T10:42:50.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:50.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc021167640 10 monclient(hunting): authenticate will time out at 1220.522752s 2024-08-19T10:42:50.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01a59c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc01c0a6d30 0x7fc01c1395b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:52940/0 (socket says 172.21.15.89:52940) 2024-08-19T10:42:50.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01a59c640 1 -- 172.21.15.89:0/36850266 learned_addr learned my addr 172.21.15.89:0/36850266 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:42:50.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01b59e640 1 -- 172.21.15.89:0/36850266 >> v1:172.21.15.190:6789/0 conn(0x7fc01c0a7110 legacy=0x7fc01c139cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:42:50.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 1 --2- 172.21.15.89:0/36850266 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a5270 0x7fc01c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 10 monclient(hunting): get_auth_request con 0x7fc01c0a5270 auth_method 0 2024-08-19T10:42:50.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 10 monclient(hunting): handle_auth_done global_id 5747 payload 274 2024-08-19T10:42:50.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 1 -- 172.21.15.89:0/36850266 >> v1:172.21.15.190:6789/0 conn(0x7fc01c0a7110 legacy=0x7fc01c139cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:50.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 1 -- 172.21.15.89:0/36850266 >> v1:172.21.15.89:6789/0 conn(0x7fc01c0a6d30 legacy=0x7fc01c1395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:50.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 10 monclient: _finish_hunting 0 2024-08-19T10:42:50.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 1 monclient: found mon.noname-h 2024-08-19T10:42:50.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:50.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:42:50.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.703+0000 7fc01ad9d640 1 -- 172.21.15.89:0/36850266 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0a81d0 con 0x7fc01c0a5270 2024-08-19T10:42:50.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01ad9d640 1 -- 172.21.15.89:0/36850266 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a5270 msgr2=0x7fc01c0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01ad9d640 1 --2- 172.21.15.89:0/36850266 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a5270 0x7fc01c0a67f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0097f0 tx=0x7fc00c01c1c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=f0217d710ce2de1d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 1 -- 172.21.15.89:0/36850266 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc00c0b0070 con 0x7fc01c0a5270 2024-08-19T10:42:50.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:42:50.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 10 monclient: dump: 2024-08-19T10:42:50.718 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.718 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.718 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.718 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.718 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 10 monclient: _finish_auth 0 2024-08-19T10:42:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:42:50.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:50.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 1 -- 172.21.15.89:0/36850266 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c0aa810 con 0x7fc01c0a5270 2024-08-19T10:42:50.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:50.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 1 -- 172.21.15.89:0/36850266 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc00c0a93e0 con 0x7fc01c0a5270 2024-08-19T10:42:50.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:42:50.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc019d9b640 10 monclient: dump: 2024-08-19T10:42:50.720 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.720 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.720 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.720 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 5 monclient: authenticate success, global_id 5747 2024-08-19T10:42:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 4 set_mon_vals no callback set 2024-08-19T10:42:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient: shutdown 2024-08-19T10:42:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 -- 172.21.15.89:0/36850266 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a5270 msgr2=0x7fc01c0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 --2- 172.21.15.89:0/36850266 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a5270 0x7fc01c0a67f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0097f0 tx=0x7fc00c01c1c0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 -- 172.21.15.89:0/36850266 shutdown_connections 2024-08-19T10:42:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 --2- 172.21.15.89:0/36850266 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a5270 0x7fc01c0a67f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 -- 172.21.15.89:0/36850266 >> 172.21.15.89:0/36850266 conn(0x7fc01c069a80 msgr2=0x7fc01c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 -- 172.21.15.89:0/36850266 shutdown_connections 2024-08-19T10:42:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 -- 172.21.15.89:0/36850266 wait complete. 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient: build_initial_monmap 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient: monmap: 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:50.724 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:50.724 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:50.724 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:50.724 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:50.724 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 Processor -- start 2024-08-19T10:42:50.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 -- start start 2024-08-19T10:42:50.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient: init 2024-08-19T10:42:50.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient: _add_conns ranks=[4,5,2,8,1,6,7,0,3] 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc01c0a5270 0x7fc01c138bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient(hunting): picked mon.noname-c con 0x7fc01c0a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a6d30 0x7fc01c1390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient(hunting): picked mon.noname-h con 0x7fc01c0a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc01c0a7110 0x7fc01c13f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient(hunting): picked mon.noname-b con 0x7fc01c0a7110 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c0a5650 con 0x7fc01c0a7110 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc01c0a7570 con 0x7fc01c0a5270 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:50.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc01c0a57d0 con 0x7fc01c0a6d30 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01ad9d640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc01c0a5270 msgr2=0x7fc01c138bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc021167640 10 monclient(hunting): authenticate will time out at 1220.531658s 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01ad9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc01c0a5270 0x7fc01c138bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01b59e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc01c0a7110 0x7fc01c13f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01b59e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc01c0a7110 0x7fc01c13f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:40212/0 (socket says 172.21.15.89:40212) 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01b59e640 1 -- 172.21.15.89:0/3297306888 learned_addr learned my addr 172.21.15.89:0/3297306888 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01b59e640 10 monclient(hunting): get_auth_request con 0x7fc01c0a7110 auth_method 0 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01b59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01b59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.711+0000 7fc01a59c640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a6d30 0x7fc01c1390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01a59c640 10 monclient(hunting): get_auth_request con 0x7fc01c0a6d30 auth_method 0 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 10 monclient(hunting): handle_auth_done global_id 5699 payload 824 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 1 -- 172.21.15.89:0/3297306888 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a6d30 msgr2=0x7fc01c1390f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a6d30 0x7fc01c1390f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 1 -- 172.21.15.89:0/3297306888 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc01c0a5270 msgr2=0x7fc01c138bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:42:50.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc01c0a5270 0x7fc01c138bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:50.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 10 monclient: _finish_hunting 0 2024-08-19T10:42:50.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 1 monclient: found mon.noname-b 2024-08-19T10:42:50.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:50.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:42:50.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 1 -- 172.21.15.89:0/3297306888 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0ab550 con 0x7fc01c0a7110 2024-08-19T10:42:50.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 1 -- 172.21.15.89:0/3297306888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc01c0a7110 msgr2=0x7fc01c13f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:50.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc01c0a7110 0x7fc01c13f4a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc01400b400 tx=0x7fc01400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01b59e640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc01c0a7110 0x7fc01c13f4a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc01400b400 tx=0x7fc01400b8c0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fc014016070 mon_map magic: 0 2024-08-19T10:42:50.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 1 -- 172.21.15.89:0/3297306888 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc014016070 con 0x7fc01c0a7110 2024-08-19T10:42:50.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 10 monclient: dump: 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.732 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 10 monclient: _finish_auth 0 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 10 monclient: _check_auth_tickets 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 1 -- 172.21.15.89:0/3297306888 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc014003d90 con 0x7fc01c0a7110 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc021167640 5 monclient: authenticate success, global_id 5699 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc021167640 10 monclient: _renew_subs 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc021167640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc021167640 1 -- 172.21.15.89:0/3297306888 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c122850 con 0x7fc01c0a7110 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:50.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc01959a640 4 set_mon_vals no callback set 2024-08-19T10:42:50.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc021167640 10 monclient: _renew_subs 2024-08-19T10:42:50.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc021167640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:50.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc021167640 1 -- 172.21.15.89:0/3297306888 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c142820 con 0x7fc01c0a7110 2024-08-19T10:42:50.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc021167640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:50.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc021167640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:50.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc021167640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:50.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc021167640 1 -- 172.21.15.89:0/3297306888 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c0a2690 con 0x7fc01c0a7110 2024-08-19T10:42:50.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 1 -- 172.21.15.89:0/3297306888 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc0140103a0 con 0x7fc01c0a7110 2024-08-19T10:42:50.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 10 monclient: dump: 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:50.735 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.715+0000 7fc0137fe640 1 -- 172.21.15.89:0/3297306888 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc014005070 con 0x7fc01c0a7110 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.719+0000 7fc0137fe640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc00003be50 0x7fc00003e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.719+0000 7fc01ad9d640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc00003be50 0x7fc00003e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.719+0000 7fc01ad9d640 10 monclient: get_auth_request con 0x7fc00003be50 auth_method 0 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.719+0000 7fc0137fe640 1 -- 172.21.15.89:0/3297306888 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.719+0000 7fc01ad9d640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc00003be50 0x7fc00003e310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0a34f0 tx=0x7fc00c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:50.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:50.723+0000 7fc0137fe640 1 -- 172.21.15.89:0/3297306888 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(155..155 src has 1..155) ==== 9079+0+0 (secure 0 0 0) 0x7fc01404f1d0 con 0x7fc01c0a7110 2024-08-19T10:42:51.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.339+0000 7fc0137fe640 1 -- 172.21.15.89:0/3297306888 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc01c0ab550 con 0x7fc01c0a7110 2024-08-19T10:42:51.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.339+0000 7fc0137fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:51.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.339+0000 7fc0137fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:51.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.571+0000 7fc021167640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:42:51.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.571+0000 7fc021167640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:42:51.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.571+0000 7fc021167640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:51.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.571+0000 7fc021167640 1 -- 172.21.15.89:0/3297306888 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc01c0abaa0 con 0x7fc01c0a7110 2024-08-19T10:42:51.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc0137fe640 1 -- 172.21.15.89:0/3297306888 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7fc01404caf0 con 0x7fc01c0a7110 2024-08-19T10:42:51.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc0137fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:42:51.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc0137fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:51.577 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":52,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:42:51.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 -- 172.21.15.89:0/3297306888 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc00003be50 msgr2=0x7fc00003e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc00003be50 0x7fc00003e310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0a34f0 tx=0x7fc00c008040 comp rx=0 tx=0).stop 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 10 monclient: shutdown 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 -- 172.21.15.89:0/3297306888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc01c0a7110 msgr2=0x7fc01c13f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc01c0a7110 0x7fc01c13f4a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc01400b400 tx=0x7fc01400b8c0 comp rx=0 tx=0).stop 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 -- 172.21.15.89:0/3297306888 shutdown_connections 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc00003be50 0x7fc00003e310 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc01c0a7110 0x7fc01c13f4a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc01c0a6d30 0x7fc01c1390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 --2- 172.21.15.89:0/3297306888 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc01c0a5270 0x7fc01c138bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 -- 172.21.15.89:0/3297306888 >> 172.21.15.89:0/3297306888 conn(0x7fc01c069a80 msgr2=0x7fc01c0a0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 -- 172.21.15.89:0/3297306888 shutdown_connections 2024-08-19T10:42:51.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:51.575+0000 7fc021167640 1 -- 172.21.15.89:0/3297306888 wait complete. 2024-08-19T10:42:51.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:51.648+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:42:51.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:51.648+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:51.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:51.648+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:51.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:51.648+0000 7faf06ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:52.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:52.652+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:42:52.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:52.652+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:52.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:52.652+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:53.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:53.648+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:42:53.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:53.648+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:53.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:53.648+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:54.595 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:42:54.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:54.648+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:42:54.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:54.648+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:54.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:54.648+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:54.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient: get_monmap_and_config 2024-08-19T10:42:54.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient: build_initial_monmap 2024-08-19T10:42:54.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient: monmap: 2024-08-19T10:42:54.722 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:54.722797+0000 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:54.722797+0000 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:54.723 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:54.724 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:54.724 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:54.724 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:54.724 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:54.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 1 Processor -- start 2024-08-19T10:42:54.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 1 -- start start 2024-08-19T10:42:54.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient: init 2024-08-19T10:42:54.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:54.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient: _add_conns ranks=[1,8,0,3,6,10,7,4,5,9,2] 2024-08-19T10:42:54.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f687c0a5270 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:54.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f687c0a6f00 0x7f687c13ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:54.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient(hunting): picked mon.noname-e con 0x7f687c0a6f00 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:42:54.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f687c0a72e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:42:54.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687c0a8710 con 0x7f687c0a72e0 2024-08-19T10:42:54.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687c0a5a40 con 0x7f687c0a5270 2024-08-19T10:42:54.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:54.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f687c0a81d0 con 0x7f687c0a6f00 2024-08-19T10:42:54.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:54.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6880b49640 10 monclient(hunting): authenticate will time out at 1224.539388s 2024-08-19T10:42:54.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f687a59c640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f687c0a5270 legacy=0x7f687c0a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:42:54.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6879d9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f687c0a6f00 0x7f687c13ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:54.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f687ad9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f687c0a72e0 0x7f687c140f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:53568/0 (socket says 172.21.15.89:53568) 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f687ad9d640 1 -- 172.21.15.89:0/2674694221 learned_addr learned my addr 172.21.15.89:0/2674694221 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6879d9b640 10 monclient(hunting): get_auth_request con 0x7f687c0a6f00 auth_method 0 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6879d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6879d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6879d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6879d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6879d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.719+0000 7f6879d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.723+0000 7f6879d9b640 10 monclient(hunting): handle_auth_done global_id 5633 payload 274 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.723+0000 7f6879d9b640 1 -- 172.21.15.89:0/2674694221 >> v1:172.21.15.190:6789/0 conn(0x7f687c0a5270 legacy=0x7f687c0a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.723+0000 7f6879d9b640 1 -- 172.21.15.89:0/2674694221 >> v1:172.21.15.89:6789/0 conn(0x7f687c0a72e0 legacy=0x7f687c140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.723+0000 7f6879d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:54.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.723+0000 7f6879d9b640 1 monclient: found mon.noname-e 2024-08-19T10:42:54.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.723+0000 7f6879d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:54.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.723+0000 7f6879d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:42:54.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.723+0000 7f6879d9b640 1 -- 172.21.15.89:0/2674694221 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c0a4790 con 0x7f687c0a6f00 2024-08-19T10:42:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 1 -- 172.21.15.89:0/2674694221 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f687c0a6f00 msgr2=0x7f687c13ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 1 --2- 172.21.15.89:0/2674694221 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f687c0a6f00 0x7f687c13ea00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f686c0097f0 tx=0x7f686c01c190 comp rx=0 tx=0).ready entity=mon.8 client_cookie=305ea2aa6898f9b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 1 -- 172.21.15.89:0/2674694221 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f686c0b0070 con 0x7f687c0a6f00 2024-08-19T10:42:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:42:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 10 monclient: dump: 2024-08-19T10:42:54.731 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:54.732 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 10 monclient: _finish_auth 0 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 10 monclient: _check_auth_tickets 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 1 -- 172.21.15.89:0/2674694221 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c0aa810 con 0x7f687c0a6f00 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 5 monclient: authenticate success, global_id 5633 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient: get_monmap_and_config success 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 1 -- 172.21.15.89:0/2674694221 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f686c0a93e0 con 0x7f687c0a6f00 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 4 set_mon_vals no callback set 2024-08-19T10:42:54.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f687959a640 10 monclient: dump: 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:54.734 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient: shutdown 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 -- 172.21.15.89:0/2674694221 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f687c0a6f00 msgr2=0x7f687c13ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 --2- 172.21.15.89:0/2674694221 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f687c0a6f00 0x7f687c13ea00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f686c0097f0 tx=0x7f686c01c190 comp rx=0 tx=0).stop 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 -- 172.21.15.89:0/2674694221 shutdown_connections 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 --2- 172.21.15.89:0/2674694221 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f687c0a6f00 0x7f687c13ea00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 -- 172.21.15.89:0/2674694221 >> 172.21.15.89:0/2674694221 conn(0x7f687c069a80 msgr2=0x7f687c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 -- 172.21.15.89:0/2674694221 shutdown_connections 2024-08-19T10:42:54.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 -- 172.21.15.89:0/2674694221 wait complete. 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient: build_initial_monmap 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient: monmap: 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:54.736 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 Processor -- start 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 -- start start 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient: init 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient: _add_conns ranks=[5,2,7,3,4,8,0,1,6] 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f687c0a5270 0x7f687c1d1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient(hunting): picked mon.noname-h con 0x7f687c0a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f687c0a6f00 0x7f687c1d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:54.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient(hunting): picked mon.noname-b con 0x7f687c0a6f00 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f687c0a72e0 0x7f687c1d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient(hunting): picked mon.noname-i con 0x7f687c0a72e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f687c145010 con 0x7f687c0a6f00 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f687c144e90 con 0x7f687c0a5270 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f687c145190 con 0x7f687c0a72e0 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6880b49640 10 monclient(hunting): authenticate will time out at 1224.546628s 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f687c0a6f00 0x7f687c1d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:54.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f687c0a6f00 0x7f687c1d15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:52642/0 (socket says 172.21.15.89:52642) 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 1 -- 172.21.15.89:0/687709872 learned_addr learned my addr 172.21.15.89:0/687709872 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 10 monclient(hunting): get_auth_request con 0x7f687c0a6f00 auth_method 0 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 10 monclient(hunting): handle_auth_done global_id 5708 payload 824 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 1 -- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f687c0a72e0 msgr2=0x7f687c1d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f687c0a72e0 0x7f687c1d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 1 -- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f687c0a5270 msgr2=0x7f687c1d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:54.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f687c0a5270 0x7f687c1d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:54.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:42:54.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 1 monclient: found mon.noname-b 2024-08-19T10:42:54.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:42:54.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:42:54.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.727+0000 7f6879d9b640 1 -- 172.21.15.89:0/687709872 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c0a25f0 con 0x7f687c0a6f00 2024-08-19T10:42:54.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6879d9b640 1 -- 172.21.15.89:0/687709872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f687c0a6f00 msgr2=0x7f687c1d15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:42:54.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6879d9b640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f687c0a6f00 0x7f687c1d15d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f686c0ae040 tx=0x7f686c018f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:54.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 1 -- 172.21.15.89:0/687709872 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f686c0b0040 con 0x7f687c0a6f00 2024-08-19T10:42:54.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:54.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:42:54.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 10 monclient: dump: 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:54.743 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6879d9b640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f687c0a6f00 0x7f687c1d15d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f686c0ae040 tx=0x7f686c018f30 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f686c004400 mon_map magic: 0 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 10 monclient: _finish_auth 0 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6880b49640 5 monclient: authenticate success, global_id 5708 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6880b49640 10 monclient: _renew_subs 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 1 -- 172.21.15.89:0/687709872 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c004250 con 0x7f687c0a6f00 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6880b49640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6880b49640 1 -- 172.21.15.89:0/687709872 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687c129a90 con 0x7f687c0a6f00 2024-08-19T10:42:54.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6878d99640 4 set_mon_vals no callback set 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6880b49640 10 monclient: _renew_subs 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6880b49640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6880b49640 1 -- 172.21.15.89:0/687709872 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f687c12a470 con 0x7f687c0a6f00 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6880b49640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6880b49640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6880b49640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f6880b49640 1 -- 172.21.15.89:0/687709872 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687c0a67f0 con 0x7f687c0a6f00 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 1 -- 172.21.15.89:0/687709872 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f686c004400 con 0x7f687c0a6f00 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 10 monclient: dump: 2024-08-19T10:42:54.745 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:42:54.746 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:42:54.747 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:42:54.747 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:54.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 1 -- 172.21.15.89:0/687709872 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f686c003620 con 0x7f687c0a6f00 2024-08-19T10:42:54.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f686affd640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f685c03be80 0x7f685c03e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:54.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f687a59c640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f685c03be80 0x7f685c03e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:42:54.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.731+0000 7f687a59c640 10 monclient: get_auth_request con 0x7f685c03be80 auth_method 0 2024-08-19T10:42:54.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.735+0000 7f687a59c640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f685c03be80 0x7f685c03e340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6870003940 tx=0x7f6870006cb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:42:54.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:54.735+0000 7f686affd640 1 -- 172.21.15.89:0/687709872 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(155..155 src has 1..155) ==== 9079+0+0 (secure 0 0 0) 0x7f686c0e9f10 con 0x7f687c0a6f00 2024-08-19T10:42:55.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.355+0000 7f686affd640 1 -- 172.21.15.89:0/687709872 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f687c0a25f0 con 0x7f687c0a6f00 2024-08-19T10:42:55.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.355+0000 7f686affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:42:55.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.355+0000 7f686affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:42:55.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f6880b49640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:42:55.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f6880b49640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:42:55.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f6880b49640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:42:55.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f6880b49640 1 -- 172.21.15.89:0/687709872 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f687c064590 con 0x7f687c0a6f00 2024-08-19T10:42:55.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f686affd640 1 -- 172.21.15.89:0/687709872 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7f687c0a8710 con 0x7f687c0a6f00 2024-08-19T10:42:55.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f686affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:42:55.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f686affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:42:55.586 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":52,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:42:55.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f6880b49640 1 -- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f685c03be80 msgr2=0x7f685c03e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:55.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f6880b49640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f685c03be80 0x7f685c03e340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6870003940 tx=0x7f6870006cb0 comp rx=0 tx=0).stop 2024-08-19T10:42:55.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f6880b49640 10 monclient: shutdown 2024-08-19T10:42:55.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f6880b49640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:42:55.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f6880b49640 1 -- 172.21.15.89:0/687709872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f687c0a6f00 msgr2=0x7f687c1d15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:42:55.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.583+0000 7f6880b49640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f687c0a6f00 0x7f687c1d15d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f686c0ae040 tx=0x7f686c018f30 comp rx=0 tx=0).stop 2024-08-19T10:42:55.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.587+0000 7f6880b49640 1 -- 172.21.15.89:0/687709872 shutdown_connections 2024-08-19T10:42:55.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.587+0000 7f6880b49640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f685c03be80 0x7f685c03e340 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:55.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.587+0000 7f6880b49640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f687c0a72e0 0x7f687c1d1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:55.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.587+0000 7f6880b49640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f687c0a6f00 0x7f687c1d15d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:55.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.587+0000 7f6880b49640 1 --2- 172.21.15.89:0/687709872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f687c0a5270 0x7f687c1d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:42:55.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.587+0000 7f6880b49640 1 -- 172.21.15.89:0/687709872 >> 172.21.15.89:0/687709872 conn(0x7f687c069a80 msgr2=0x7f687c0aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:42:55.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.587+0000 7f6880b49640 1 -- 172.21.15.89:0/687709872 shutdown_connections 2024-08-19T10:42:55.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:55.587+0000 7f6880b49640 1 -- 172.21.15.89:0/687709872 wait complete. 2024-08-19T10:42:55.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:55.648+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:42:55.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:55.648+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:55.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:55.648+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:56.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:56.647+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:42:56.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:56.647+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:56.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:56.647+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:57.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:57.651+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:42:57.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:57.651+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:57.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:57.651+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:58.605 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:42:58.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:58.651+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:42:58.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:58.651+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:58.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:58.651+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient: get_monmap_and_config 2024-08-19T10:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient: build_initial_monmap 2024-08-19T10:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient: monmap: 2024-08-19T10:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:42:58.730515+0000 2024-08-19T10:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:42:58.730515+0000 2024-08-19T10:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:42:58.730 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:42:58.731 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:42:58.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 1 Processor -- start 2024-08-19T10:42:58.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 1 -- start start 2024-08-19T10:42:58.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient: init 2024-08-19T10:42:58.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient: _reopen_session rank -1 2024-08-19T10:42:58.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient: _add_conns ranks=[6,4,1,8,2,5,0,7,3,10,9] 2024-08-19T10:42:58.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6fa00a39e0 0x7f6fa00a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:58.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient(hunting): picked mon.noname-d con 0x7f6fa00a39e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:42:58.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fa00a1550 0x7f6fa00a4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient(hunting): picked mon.noname-c con 0x7f6fa00a1550 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6fa00a4840 addr v1:172.21.15.190:6789/0 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa00a0900 con 0x7f6fa00a4840 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6fa00a1130 con 0x7f6fa00a1550 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient(hunting): start opening mon connection 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6fa00a4e10 con 0x7f6fa00a39e0 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient(hunting): _renew_subs 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6fa6040640 10 monclient(hunting): authenticate will time out at 1228.547093s 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6f9f7fe640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6fa00a39e0 msgr2=0x7f6fa00a3dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6f9effd640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fa00a1550 msgr2=0x7f6fa00a4300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6f9f7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6fa00a39e0 0x7f6fa00a3dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:58.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fa00a1550 0x7f6fa00a4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:42:58.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.727+0000 7f6f9ffff640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f6fa00a4840 legacy=0x7f6fa013f880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:42:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.931+0000 7f6f9effd640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fa00a1550 msgr2=0x7f6fa00a4300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:42:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.931+0000 7f6f9f7fe640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6fa00a39e0 msgr2=0x7f6fa00a3dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:42:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.931+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fa00a1550 0x7f6fa00a4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:42:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.931+0000 7f6f9f7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6fa00a39e0 0x7f6fa00a3dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:42:58.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:58.931+0000 7f6f9ffff640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f6fa00a4840 legacy=0x7f6fa013f880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:42:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:59.335+0000 7f6f9effd640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fa00a1550 msgr2=0x7f6fa00a4300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:42:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:59.335+0000 7f6f9f7fe640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6fa00a39e0 msgr2=0x7f6fa00a3dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:42:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:59.335+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fa00a1550 0x7f6fa00a4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:42:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:59.335+0000 7f6f9f7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6fa00a39e0 0x7f6fa00a3dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:42:59.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:42:59.335+0000 7f6f9ffff640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f6fa00a4840 legacy=0x7f6fa013f880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:42:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:59.647+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:42:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:59.647+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:42:59.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:42:59.647+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:00.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:00.139+0000 7f6f9f7fe640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6fa00a39e0 msgr2=0x7f6fa00a3dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:00.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:00.139+0000 7f6f9f7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6fa00a39e0 0x7f6fa00a3dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:43:00.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:00.139+0000 7f6f9effd640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fa00a1550 msgr2=0x7f6fa00a4300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:00.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:00.139+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fa00a1550 0x7f6fa00a4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:43:00.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:00.139+0000 7f6f9ffff640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f6fa00a4840 legacy=0x7f6fa013f880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:00.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:00.139+0000 7f6f9e7fc640 1 -- DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:43:00.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:00.647+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:43:00.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:00.647+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:00.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:00.647+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:01.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:01.651+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:43:01.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:01.651+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:01.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:01.651+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:01.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:01.651+0000 7faf06ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 10 monclient(hunting): tick 2024-08-19T10:43:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:43:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 monclient(hunting): continuing hunt 2024-08-19T10:43:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:43:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6fa00a39e0 msgr2=0x7f6fa00a3dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6fa00a39e0 0x7f6fa00a3dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fa00a1550 msgr2=0x7f6fa00a4300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6fa00a1550 0x7f6fa00a4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f6fa00a4840 legacy=0x7f6fa013f880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 10 monclient: _add_conns ranks=[8,2,7,0,10,3,5,9] 2024-08-19T10:43:01.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 10 monclient(hunting): picked mon.noname-e con 0x7f6f80003860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 10 monclient(hunting): picked mon.noname-b con 0x7f6f80006210 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 10 monclient(hunting): picked mon.noname-i con 0x7f6f80008d40 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa00a4e10 con 0x7f6f80006210 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6f8000b960 con 0x7f6f80008d40 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6f8000bb60 con 0x7f6f80003860 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9d7fa640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:01.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9f7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9f7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:46046/0 (socket says 172.21.15.89:46046) 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43778/0 (socket says 172.21.15.89:43778) 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 learned_addr learned my addr 172.21.15.89:0/2556998176 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9f7fe640 10 monclient(hunting): get_auth_request con 0x7f6f80003860 auth_method 0 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9effd640 10 monclient(hunting): get_auth_request con 0x7f6f80006210 auth_method 0 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:01.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request con 0x7f6f80008d40 auth_method 0 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 msgr2=0x7f6f80005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 msgr2=0x7f6f80005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9effd640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 msgr2=0x7f6f800086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9effd640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 msgr2=0x7f6f800086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 msgr2=0x7f6f8000b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:43:01.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:01.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 msgr2=0x7f6f8000b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:01.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:01.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.731+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:01.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:01.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:01.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9effd640 10 monclient(hunting): get_auth_request con 0x7f6f80006210 auth_method 0 2024-08-19T10:43:01.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:01.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:01.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:01.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9f7fe640 10 monclient(hunting): get_auth_request con 0x7f6f80003860 auth_method 0 2024-08-19T10:43:01.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:01.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:01.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:01.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:01.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request con 0x7f6f80008d40 auth_method 0 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 msgr2=0x7f6f80005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 msgr2=0x7f6f80005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9effd640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 msgr2=0x7f6f800086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9effd640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 msgr2=0x7f6f800086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 msgr2=0x7f6f8000b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:43:01.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 msgr2=0x7f6f8000b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:01.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:43:01.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:01.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:01.935+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:43:02.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:02.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:02.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9f7fe640 10 monclient(hunting): get_auth_request con 0x7f6f80003860 auth_method 0 2024-08-19T10:43:02.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:02.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:02.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:02.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9effd640 10 monclient(hunting): get_auth_request con 0x7f6f80006210 auth_method 0 2024-08-19T10:43:02.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request con 0x7f6f80008d40 auth_method 0 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 msgr2=0x7f6f80005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 msgr2=0x7f6f80005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:43:02.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9effd640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 msgr2=0x7f6f800086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:43:02.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9effd640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 msgr2=0x7f6f800086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:02.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:02.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:43:02.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 msgr2=0x7f6f8000b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:43:02.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 msgr2=0x7f6f8000b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:02.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:02.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:02.338+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:43:02.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:02.651+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:43:02.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:02.651+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:02.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:02.651+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9effd640 10 monclient(hunting): get_auth_request con 0x7f6f80006210 auth_method 0 2024-08-19T10:43:03.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9f7fe640 10 monclient(hunting): get_auth_request con 0x7f6f80003860 auth_method 0 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9effd640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 msgr2=0x7f6f800086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request con 0x7f6f80008d40 auth_method 0 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9effd640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 msgr2=0x7f6f800086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:03.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 msgr2=0x7f6f80005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:43:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 msgr2=0x7f6f80005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:43:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:43:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 msgr2=0x7f6f8000b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:43:03.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 msgr2=0x7f6f8000b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:03.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:03.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:03.142+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:43:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:03.651+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:43:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:03.651+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:03.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:03.651+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:04.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:04.651+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:43:04.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:04.651+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:04.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:04.651+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 10 monclient(hunting): tick 2024-08-19T10:43:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:43:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 monclient(hunting): continuing hunt 2024-08-19T10:43:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:43:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 msgr2=0x7f6f80005cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f80003860 0x7f6f80005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 msgr2=0x7f6f8000b1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6f80008d40 0x7f6f8000b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 msgr2=0x7f6f800086a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80006210 0x7f6f800086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:04.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 10 monclient: _add_conns ranks=[10,0,3,9,5] 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 --2- 172.21.15.89:0/2556998176 >> v2:172.21.15.190:3300/0 conn(0x7f6f8000c1c0 0x7f6f8000e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 reap_dead start 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 10 monclient(hunting): picked mon.noname-f con 0x7f6f8000c1c0 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6f8000ec20 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 0x7f6f80014d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 10 monclient(hunting): picked mon.noname-g con 0x7f6f80012890 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 -- 172.21.15.89:0/2556998176 --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa00a0b70 con 0x7f6f8000ec20 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 -- 172.21.15.89:0/2556998176 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa00a4e10 con 0x7f6f8000c1c0 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 1 -- 172.21.15.89:0/2556998176 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8000b960 con 0x7f6f80012890 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9d7fa640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:04.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.726+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 reap_dead start 2024-08-19T10:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 >> v2:172.21.15.190:3300/0 conn(0x7f6f8000c1c0 msgr2=0x7f6f8000e5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> v2:172.21.15.190:3300/0 conn(0x7f6f8000c1c0 0x7f6f8000e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 0x7f6f80014d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request con 0x7f6f80012890 auth_method 0 2024-08-19T10:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 msgr2=0x7f6f80014d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 msgr2=0x7f6f80014d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 0x7f6f80014d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:04.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 0x7f6f80014d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:04.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.730+0000 7f6f9effd640 1 -- 172.21.15.89:0/2556998176 >> v1:172.21.15.89:6789/0 conn(0x7f6f8000ec20 legacy=0x7f6f80011040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:04.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.934+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 0x7f6f80014d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:04.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.934+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request con 0x7f6f80012890 auth_method 0 2024-08-19T10:43:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.934+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.934+0000 7f6f9ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.934+0000 7f6f9ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.934+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 msgr2=0x7f6f80014d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:43:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.934+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 msgr2=0x7f6f80014d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:43:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.934+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 0x7f6f80014d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.934+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 0x7f6f80014d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:43:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.934+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 >> v2:172.21.15.190:3300/0 conn(0x7f6f8000c1c0 msgr2=0x7f6f8000e5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:04.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:04.934+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> v2:172.21.15.190:3300/0 conn(0x7f6f8000c1c0 0x7f6f8000e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:43:05.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9f7fe640 1 -- 172.21.15.89:0/2556998176 >> v2:172.21.15.190:3300/0 conn(0x7f6f8000c1c0 msgr2=0x7f6f8000e5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:05.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2556998176 >> v2:172.21.15.190:3300/0 conn(0x7f6f8000c1c0 0x7f6f8000e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:43:05.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 0x7f6f80014d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request con 0x7f6f80012890 auth_method 0 2024-08-19T10:43:05.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 10 monclient(hunting): handle_auth_done global_id 34203 payload 274 2024-08-19T10:43:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> v2:172.21.15.190:3300/0 conn(0x7f6f8000c1c0 msgr2=0x7f6f8000e5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> v2:172.21.15.190:3300/0 conn(0x7f6f8000c1c0 0x7f6f8000e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> v1:172.21.15.89:6789/0 conn(0x7f6f8000ec20 legacy=0x7f6f80011040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:43:05.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 1 monclient: found mon.noname-g 2024-08-19T10:43:05.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:05.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:43:05.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.338+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa00a0900 con 0x7f6f80012890 2024-08-19T10:43:05.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 msgr2=0x7f6f80014d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:05.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 0x7f6f80014d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00a4f90 tx=0x7f6f9801dae0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=e9be4661ef08403a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:05.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 1 -- 172.21.15.89:0/2556998176 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f980ac070 con 0x7f6f80012890 2024-08-19T10:43:05.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:43:05.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 10 monclient: dump: 2024-08-19T10:43:05.351 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.352 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.352 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.352 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.352 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.352 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.352 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.352 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.352 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.352 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.352 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.352 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.353 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.353 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.353 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.353 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:43:05.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:43:05.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:05.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 1 -- 172.21.15.89:0/2556998176 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f980a6b30 con 0x7f6f80012890 2024-08-19T10:43:05.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:05.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 1 -- 172.21.15.89:0/2556998176 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f9800b040 con 0x7f6f80012890 2024-08-19T10:43:05.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6f9e7fc640 10 monclient: dump: 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.354 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.355 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.355 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.355 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.355 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.355 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.355 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6fa6040640 5 monclient: authenticate success, global_id 34203 2024-08-19T10:43:05.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6fa6040640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:05.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6fa6040640 4 set_mon_vals no callback set 2024-08-19T10:43:05.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.346+0000 7f6fa6040640 10 monclient: shutdown 2024-08-19T10:43:05.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:05.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 -- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 msgr2=0x7f6f80014d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 0x7f6f80014d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00a4f90 tx=0x7f6f9801dae0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 -- 172.21.15.89:0/2556998176 shutdown_connections 2024-08-19T10:43:05.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 --2- 172.21.15.89:0/2556998176 >> v2:172.21.15.190:3300/0 conn(0x7f6f8000c1c0 0x7f6f8000e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 --2- 172.21.15.89:0/2556998176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f80012890 0x7f6f80014d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 -- 172.21.15.89:0/2556998176 >> 172.21.15.89:0/2556998176 conn(0x7f6fa00a9090 msgr2=0x7f6fa00a94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:05.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 -- 172.21.15.89:0/2556998176 shutdown_connections 2024-08-19T10:43:05.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 -- 172.21.15.89:0/2556998176 wait complete. 2024-08-19T10:43:05.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient: build_initial_monmap 2024-08-19T10:43:05.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient: monmap: 2024-08-19T10:43:05.363 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:05.363 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:05.364 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 Processor -- start 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 -- start start 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient: init 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient: _add_conns ranks=[6,8,2,0,3,5,4,1,7] 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6f8000c1c0 0x7f6fa01aa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient(hunting): picked mon.noname-d con 0x7f6f8000c1c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f8000ec20 0x7f6fa01be660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient(hunting): picked mon.noname-e con 0x7f6f8000ec20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80012890 0x7f6fa01beba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient(hunting): picked mon.noname-b con 0x7f6f80012890 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:05.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa00a3dc0 con 0x7f6f80012890 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6fa006cc50 con 0x7f6f8000c1c0 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6fa00a3f40 con 0x7f6f8000ec20 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6fa6040640 10 monclient(hunting): authenticate will time out at 1235.170857s 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9f7fe640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6f8000c1c0 msgr2=0x7f6fa01aa820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9f7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6f8000c1c0 0x7f6fa01aa820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f8000ec20 0x7f6fa01be660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9ffff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80012890 0x7f6fa01beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9ffff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80012890 0x7f6fa01beba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43804/0 (socket says 172.21.15.89:43804) 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f8000ec20 0x7f6fa01be660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:46080/0 (socket says 172.21.15.89:46080) 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9ffff640 1 -- 172.21.15.89:0/2828093673 learned_addr learned my addr 172.21.15.89:0/2828093673 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request con 0x7f6f80012890 auth_method 0 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 10 monclient(hunting): get_auth_request con 0x7f6f8000ec20 auth_method 0 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:05.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 10 monclient(hunting): handle_auth_done global_id 5642 payload 824 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 1 -- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6f8000c1c0 msgr2=0x7f6fa01aa820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6f8000c1c0 0x7f6fa01aa820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 1 -- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80012890 msgr2=0x7f6fa01beba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80012890 0x7f6fa01beba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 10 monclient: _finish_hunting 0 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 1 monclient: found mon.noname-e 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:43:05.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9effd640 1 -- 172.21.15.89:0/2828093673 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa009ecb0 con 0x7f6f8000ec20 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.350+0000 7f6f9ffff640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80012890 0x7f6fa01beba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f9effd640 1 -- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f8000ec20 msgr2=0x7f6fa01be660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f9effd640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f8000ec20 0x7f6fa01be660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f88003fd0 tx=0x7f6f8800cb00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 1 -- 172.21.15.89:0/2828093673 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f8801c070 con 0x7f6f8000ec20 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 10 monclient: dump: 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.369 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 10 monclient: _finish_auth 0 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:05.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 1 -- 172.21.15.89:0/2828093673 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f8800be20 con 0x7f6f8000ec20 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6fa6040640 5 monclient: authenticate success, global_id 5642 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6fa6040640 10 monclient: _renew_subs 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6fa6040640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6fa6040640 1 -- 172.21.15.89:0/2828093673 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa009f1e0 con 0x7f6f8000ec20 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 1 -- 172.21.15.89:0/2828093673 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6f88016810 con 0x7f6f8000ec20 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f9dffb640 4 set_mon_vals no callback set 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 10 monclient: dump: 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.371 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6fa6040640 10 monclient: _renew_subs 2024-08-19T10:43:05.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6fa6040640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:05.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6fa6040640 1 -- 172.21.15.89:0/2828093673 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa01ac020 con 0x7f6f8000ec20 2024-08-19T10:43:05.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6fa6040640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:05.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6fa6040640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:05.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6fa6040640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:05.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6fa6040640 1 -- 172.21.15.89:0/2828093673 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa01abbd0 con 0x7f6f8000ec20 2024-08-19T10:43:05.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.354+0000 7f6f97fff640 1 -- 172.21.15.89:0/2828093673 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6f880163a0 con 0x7f6f8000ec20 2024-08-19T10:43:05.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.358+0000 7f6f97fff640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6f80040b50 0x7f6f80043010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.358+0000 7f6f97fff640 1 -- 172.21.15.89:0/2828093673 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(155..155 src has 1..155) ==== 9079+0+0 (secure 0 0 0) 0x7f6f8804fa30 con 0x7f6f8000ec20 2024-08-19T10:43:05.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.358+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6f80040b50 0x7f6f80043010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.358+0000 7f6f9f7fe640 10 monclient: get_auth_request con 0x7f6f80040b50 auth_method 0 2024-08-19T10:43:05.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.358+0000 7f6f9f7fe640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6f80040b50 0x7f6f80043010 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6f90003b40 tx=0x7f6f90005040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:05.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.362+0000 7f6f97fff640 1 -- 172.21.15.89:0/2828093673 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6fa01abbd0 con 0x7f6f8000ec20 2024-08-19T10:43:05.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.362+0000 7f6f97fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:05.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.362+0000 7f6f97fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:05.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.626+0000 7f6fa6040640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:43:05.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.626+0000 7f6fa6040640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:43:05.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.626+0000 7f6fa6040640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:05.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.626+0000 7f6fa6040640 1 -- 172.21.15.89:0/2828093673 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6fa01abe20 con 0x7f6f8000ec20 2024-08-19T10:43:05.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.626+0000 7f6f97fff640 1 -- 172.21.15.89:0/2828093673 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3174 (secure 0 0 0) 0x7f6f8804d350 con 0x7f6f8000ec20 2024-08-19T10:43:05.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.626+0000 7f6f97fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:43:05.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.626+0000 7f6f97fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:05.630 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":60,"quorum":[0,2,3,5,7,8],"quorum_names":["a","b","g","h","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:43:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 -- 172.21.15.89:0/2828093673 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6f80040b50 msgr2=0x7f6f80043010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6f80040b50 0x7f6f80043010 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6f90003b40 tx=0x7f6f90005040 comp rx=0 tx=0).stop 2024-08-19T10:43:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 10 monclient: shutdown 2024-08-19T10:43:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 -- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f8000ec20 msgr2=0x7f6fa01be660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f8000ec20 0x7f6fa01be660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f88003fd0 tx=0x7f6f8800cb00 comp rx=0 tx=0).stop 2024-08-19T10:43:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 -- 172.21.15.89:0/2828093673 shutdown_connections 2024-08-19T10:43:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6f80012890 0x7f6fa01beba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6f80040b50 0x7f6f80043010 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6f8000ec20 0x7f6fa01be660 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 --2- 172.21.15.89:0/2828093673 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6f8000c1c0 0x7f6fa01aa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 -- 172.21.15.89:0/2828093673 >> 172.21.15.89:0/2828093673 conn(0x7f6fa00a9090 msgr2=0x7f6fa00a6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 -- 172.21.15.89:0/2828093673 shutdown_connections 2024-08-19T10:43:05.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.630+0000 7f6fa6040640 1 -- 172.21.15.89:0/2828093673 wait complete. 2024-08-19T10:43:05.650 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-08-19T10:43:05.650 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon a 2024-08-19T10:43:05.650 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:43:05.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:05.647+0000 7faf06ffd640 10 monclient: tick 2024-08-19T10:43:05.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:05.647+0000 7faf06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:05.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:05.647+0000 7faf06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient: get_monmap_and_config 2024-08-19T10:43:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient: build_initial_monmap 2024-08-19T10:43:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient: monmap: 2024-08-19T10:43:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:43:05.765627+0000 2024-08-19T10:43:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:43:05.765627+0000 2024-08-19T10:43:05.765 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:05.766 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.766 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:05.766 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:05.767 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:05.768 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:05.768 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:05.768 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:05.768 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:05.768 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 1 Processor -- start 2024-08-19T10:43:05.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 1 -- start start 2024-08-19T10:43:05.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient: init 2024-08-19T10:43:05.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:05.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient: _add_conns ranks=[7,8,0,4,3,6,2,10,5,9,1] 2024-08-19T10:43:05.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e6406c930 0x7f9e640a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient(hunting): picked mon.noname-i con 0x7f9e6406c930 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:05.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e640a3fa0 0x7f9e640a4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient(hunting): picked mon.noname-e con 0x7f9e640a3fa0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:05.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9e6413d910 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:05.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e640a4ae0 con 0x7f9e6413d910 2024-08-19T10:43:05.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9e640744f0 con 0x7f9e6406c930 2024-08-19T10:43:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9e640a48c0 con 0x7f9e640a3fa0 2024-08-19T10:43:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6925c640 10 monclient(hunting): authenticate will time out at 1235.582507s 2024-08-19T10:43:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6359e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9e6413d910 0x7f9e6413fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:33864/0 (socket says 172.21.15.89:33864) 2024-08-19T10:43:05.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6359e640 1 -- 172.21.15.89:0/335300541 learned_addr learned my addr 172.21.15.89:0/335300541 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:43:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 1 --2- 172.21.15.89:0/335300541 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e640a3fa0 0x7f9e640a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 10 monclient(hunting): get_auth_request con 0x7f9e640a3fa0 auth_method 0 2024-08-19T10:43:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:05.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:05.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:05.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:05.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e62d9d640 1 --2- 172.21.15.89:0/335300541 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e6406c930 0x7f9e640a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e62d9d640 10 monclient(hunting): get_auth_request con 0x7f9e6406c930 auth_method 0 2024-08-19T10:43:05.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e62d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e62d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e62d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:05.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 10 monclient(hunting): handle_auth_done global_id 5651 payload 274 2024-08-19T10:43:05.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 1 -- 172.21.15.89:0/335300541 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e6406c930 msgr2=0x7f9e640a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:05.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 1 --2- 172.21.15.89:0/335300541 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e6406c930 0x7f9e640a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 1 -- 172.21.15.89:0/335300541 >> v1:172.21.15.89:6789/0 conn(0x7f9e6413d910 legacy=0x7f9e6413fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:05.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 10 monclient: _finish_hunting 0 2024-08-19T10:43:05.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 1 monclient: found mon.noname-e 2024-08-19T10:43:05.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:05.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:43:05.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.762+0000 7f9e6259c640 1 -- 172.21.15.89:0/335300541 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e640a0910 con 0x7f9e640a3fa0 2024-08-19T10:43:05.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e6259c640 1 -- 172.21.15.89:0/335300541 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e640a3fa0 msgr2=0x7f9e640a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:05.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e6259c640 1 --2- 172.21.15.89:0/335300541 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e640a3fa0 0x7f9e640a4380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e580097f0 tx=0x7f9e5801c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=98d3cb068e1f6baf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:05.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 1 -- 172.21.15.89:0/335300541 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e580b0070 con 0x7f9e640a3fa0 2024-08-19T10:43:05.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:43:05.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 10 monclient: dump: 2024-08-19T10:43:05.799 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.799 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.799 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.800 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.800 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.800 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.800 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.800 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.800 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.800 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.801 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.801 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.801 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.801 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.801 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.801 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 10 monclient: _finish_auth 0 2024-08-19T10:43:05.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:43:05.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:05.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 1 -- 172.21.15.89:0/335300541 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e580aa810 con 0x7f9e640a3fa0 2024-08-19T10:43:05.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e6925c640 5 monclient: authenticate success, global_id 5651 2024-08-19T10:43:05.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:05.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e6925c640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:05.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 1 -- 172.21.15.89:0/335300541 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e580a9300 con 0x7f9e640a3fa0 2024-08-19T10:43:05.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e6925c640 4 set_mon_vals no callback set 2024-08-19T10:43:05.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:05.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.766+0000 7f9e61d9b640 10 monclient: dump: 2024-08-19T10:43:05.804 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.804 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.804 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.804 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.804 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.805 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.806 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.806 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.806 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.806 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient: shutdown 2024-08-19T10:43:05.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:05.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 -- 172.21.15.89:0/335300541 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e640a3fa0 msgr2=0x7f9e640a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 --2- 172.21.15.89:0/335300541 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e640a3fa0 0x7f9e640a4380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e580097f0 tx=0x7f9e5801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 -- 172.21.15.89:0/335300541 shutdown_connections 2024-08-19T10:43:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 --2- 172.21.15.89:0/335300541 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9e640a3fa0 0x7f9e640a4380 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 --2- 172.21.15.89:0/335300541 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e6406c930 0x7f9e640a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 -- 172.21.15.89:0/335300541 >> 172.21.15.89:0/335300541 conn(0x7f9e640a8f60 msgr2=0x7f9e640a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 -- 172.21.15.89:0/335300541 shutdown_connections 2024-08-19T10:43:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 -- 172.21.15.89:0/335300541 wait complete. 2024-08-19T10:43:05.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient: build_initial_monmap 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient: monmap: 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:05.808 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:05.809 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:05.809 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:05.809 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:05.809 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 Processor -- start 2024-08-19T10:43:05.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 -- start start 2024-08-19T10:43:05.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient: init 2024-08-19T10:43:05.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:05.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient: _add_conns ranks=[4,7,2,1,5,3,0,6,8] 2024-08-19T10:43:05.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e6406c930 0x7f9e6413c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient(hunting): picked mon.noname-c con 0x7f9e6406c930 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e640a3fa0 0x7f9e6413c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient(hunting): picked mon.noname-i con 0x7f9e640a3fa0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e6413d910 0x7f9e64123800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient(hunting): picked mon.noname-b con 0x7f9e6413d910 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9e64143e50 con 0x7f9e6413d910 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9e640a4e20 con 0x7f9e6406c930 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9e64143fd0 con 0x7f9e640a3fa0 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:05.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6925c640 10 monclient(hunting): authenticate will time out at 1235.589738s 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e62d9d640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e6406c930 msgr2=0x7f9e6413c400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e62d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e6406c930 0x7f9e6413c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e6413d910 0x7f9e64123800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e6413d910 0x7f9e64123800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43812/0 (socket says 172.21.15.89:43812) 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 1 -- 172.21.15.89:0/1580546762 learned_addr learned my addr 172.21.15.89:0/1580546762 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 10 monclient(hunting): get_auth_request con 0x7f9e6413d910 auth_method 0 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:05.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6259c640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e640a3fa0 0x7f9e6413c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6259c640 10 monclient(hunting): get_auth_request con 0x7f9e640a3fa0 auth_method 0 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 10 monclient(hunting): handle_auth_done global_id 5726 payload 824 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e640a3fa0 msgr2=0x7f9e6413c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9e640a3fa0 0x7f9e6413c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e6406c930 msgr2=0x7f9e6413c400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9e6406c930 0x7f9e6413c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 10 monclient: _finish_hunting 0 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 1 monclient: found mon.noname-b 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:05.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.770+0000 7f9e6359e640 1 -- 172.21.15.89:0/1580546762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e64140d20 con 0x7f9e6413d910 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6359e640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e6413d910 msgr2=0x7f9e64123800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6359e640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e6413d910 0x7f9e64123800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00caf0 tx=0x7f9e5c00a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e5c00acb0 con 0x7f9e6413d910 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 10 monclient: dump: 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.813 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5c012870 con 0x7f9e6413d910 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6925c640 5 monclient: authenticate success, global_id 5726 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6925c640 10 monclient: _renew_subs 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6925c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6925c640 1 -- 172.21.15.89:0/1580546762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e641272a0 con 0x7f9e6413d910 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e5c007070 con 0x7f9e6413d910 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6159a640 4 set_mon_vals no callback set 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 10 monclient: dump: 2024-08-19T10:43:05.815 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.816 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6925c640 10 monclient: _renew_subs 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6925c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6925c640 1 -- 172.21.15.89:0/1580546762 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e64126fc0 con 0x7f9e6413d910 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6925c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6925c640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 0x7f9e64123d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6925c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6925c640 1 -- 172.21.15.89:0/1580546762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9e64126c40 con 0x7f9e6413ce80 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e62d9d640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 0x7f9e64123d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e62d9d640 10 monclient: get_auth_request con 0x7f9e6413ce80 auth_method 0 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e62d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e62d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e62d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9e5c012400 con 0x7f9e6413d910 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_done global_id 5787 payload 824 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e62d9d640 1 -- 172.21.15.89:0/1580546762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e640a3a60 con 0x7f9e6413ce80 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e5003be30 0x7f9e5003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(156..156 src has 1..156) ==== 9265+0+0 (secure 0 0 0) 0x7f9e5c050260 con 0x7f9e6413d910 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6259c640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e5003be30 0x7f9e5003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.774+0000 7f9e6259c640 10 monclient: get_auth_request con 0x7f9e5003be30 auth_method 0 2024-08-19T10:43:05.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 msgr2=0x7f9e64123d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 0x7f9e64123d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9e54006f90 tx=0x7f9e5400d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 msgr2=0x7f9e64123d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 msgr2=0x7f9e64123d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 0x7f9e64123d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9e54006f90 tx=0x7f9e5400d960 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 0x7f9e64123d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9e54006f90 tx=0x7f9e5400d960 comp rx=0 tx=0).stop 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 msgr2=0x7f9e64123d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 0x7f9e64123d40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e500467f0 0x7f9e50048c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 msgr2=0x7f9e64123d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e6413ce80 0x7f9e64123d40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9e50049270 con 0x7f9e500467f0 2024-08-19T10:43:05.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e500467f0 0x7f9e50048c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 10 monclient: get_auth_request con 0x7f9e500467f0 auth_method 0 2024-08-19T10:43:05.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:05.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:05.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:05.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:05.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_done global_id 5796 payload 824 2024-08-19T10:43:05.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 1 -- 172.21.15.89:0/1580546762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e5c00ee50 con 0x7f9e500467f0 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e6259c640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e5003be30 0x7f9e5003e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9e6406da10 tx=0x7f9e580a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e500467f0 msgr2=0x7f9e50048c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e62d9d640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e500467f0 0x7f9e50048c10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9e54006f60 tx=0x7f9e54004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e5402b070 con 0x7f9e500467f0 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 10 monclient: dump: 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.821 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9e54019070 con 0x7f9e500467f0 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:05.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e500467f0 msgr2=0x7f9e50048c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e500467f0 0x7f9e50048c10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9e54006f60 tx=0x7f9e54004740 comp rx=0 tx=0).stop 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e500467f0 msgr2=0x7f9e50048c10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.778+0000 7f9e4b7fe640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e500467f0 0x7f9e50048c10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e6925c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e6925c640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e64124280 0x7f9e64124690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e6925c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e6925c640 1 -- 172.21.15.89:0/1580546762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641410e0 con 0x7f9e64124280 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e62d9d640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e64124280 0x7f9e64124690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e62d9d640 10 monclient: get_auth_request con 0x7f9e64124280 auth_method 0 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e62d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e62d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e62d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:05.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:05.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:05.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:05.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e62d9d640 10 monclient(hunting): handle_auth_done global_id 5805 payload 824 2024-08-19T10:43:05.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.786+0000 7f9e62d9d640 1 -- 172.21.15.89:0/1580546762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9e64064590 con 0x7f9e64124280 2024-08-19T10:43:05.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e62d9d640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e64124280 msgr2=0x7f9e64124690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:05.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e62d9d640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e64124280 0x7f9e64124690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e54018000 tx=0x7f9e54019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:05.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9e5402b040 con 0x7f9e64124280 2024-08-19T10:43:05.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e4b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e4b7fe640 10 monclient: dump: 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.826 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9236 (secure 0 0 0) 0x7f9e5402d2c0 con 0x7f9e64124280 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e4b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e4b7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e64124280 msgr2=0x7f9e64124690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e4b7fe640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e64124280 0x7f9e64124690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e54018000 tx=0x7f9e54019000 comp rx=0 tx=0).stop 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e4b7fe640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e64124280 msgr2=0x7f9e64124690 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e4b7fe640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e64124280 0x7f9e64124690 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6359e640 1 -- 172.21.15.89:0/1580546762 reap_dead start 2024-08-19T10:43:05.828 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:43:05.828 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:43:05.828 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:43:05.828 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:43:05.828 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 60, 2024-08-19T10:43:05.828 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 315604, 2024-08-19T10:43:05.828 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:43:05.828 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:43:05.828 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:43:05.828 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:05.829 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:05.830 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:43:05.831 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:05.832 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:05.833 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:05.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:05.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.837 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:43:05.838 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:43:05.838 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:05.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:05.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:05.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:43:05.838 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:05.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:43:05.839 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.841 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:43:05.842 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:05.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:43:05.845 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.845 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.845 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.846 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:05.847 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:43:05.850 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:43:05.851 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:05.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:05.853 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:43:05.853 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:43:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e5003be30 msgr2=0x7f9e5003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e5003be30 0x7f9e5003e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9e6406da10 tx=0x7f9e580a6000 comp rx=0 tx=0).stop 2024-08-19T10:43:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 10 monclient: shutdown 2024-08-19T10:43:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 1 -- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e6413d910 msgr2=0x7f9e64123800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e6413d910 0x7f9e64123800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00caf0 tx=0x7f9e5c00a060 comp rx=0 tx=0).stop 2024-08-19T10:43:05.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 1 -- 172.21.15.89:0/1580546762 shutdown_connections 2024-08-19T10:43:05.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e5003be30 0x7f9e5003e2f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 1 --2- 172.21.15.89:0/1580546762 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9e6413d910 0x7f9e64123800 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 1 -- 172.21.15.89:0/1580546762 >> 172.21.15.89:0/1580546762 conn(0x7f9e640a8f60 msgr2=0x7f9e64072e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:05.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 1 -- 172.21.15.89:0/1580546762 shutdown_connections 2024-08-19T10:43:05.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.790+0000 7f9e6925c640 1 -- 172.21.15.89:0/1580546762 wait complete. 2024-08-19T10:43:05.855 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:43:05.855 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-08-19T10:43:05.856 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:43:05.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.974+0000 7f62ad462640 10 monclient: get_monmap_and_config 2024-08-19T10:43:05.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.974+0000 7f62ad462640 10 monclient: build_initial_monmap 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.974+0000 7f62ad462640 10 monclient: monmap: 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:43:05.979259+0000 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:43:05.979259+0000 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:05.979 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:05.980 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:05.980 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:05.980 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:05.980 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:05.980 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:05.980 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:05.980 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.974+0000 7f62ad462640 1 Processor -- start 2024-08-19T10:43:05.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.974+0000 7f62ad462640 1 -- start start 2024-08-19T10:43:05.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.974+0000 7f62ad462640 10 monclient: init 2024-08-19T10:43:05.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:05.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 10 monclient: _add_conns ranks=[8,2,10,6,9,3,5,1,4,0,7] 2024-08-19T10:43:05.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f62a80a5270 0x7f62a80a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 10 monclient(hunting): picked mon.noname-e con 0x7f62a80a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:05.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a6d30 0x7f62a80a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 10 monclient(hunting): picked mon.noname-b con 0x7f62a80a6d30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62a81395b0 0x7f62a8139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 10 monclient(hunting): picked mon.noname-f con 0x7f62a81395b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f62a80a7650 con 0x7f62a81395b0 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f62a80a8710 con 0x7f62a80a6d30 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f62a80a5a40 con 0x7f62a80a5270 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62ad462640 10 monclient(hunting): authenticate will time out at 1235.796395s 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f62a80a5270 0x7f62a80a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f629ffff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a6d30 0x7f62a80a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f62a80a5270 0x7f62a80a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:46088/0 (socket says 172.21.15.89:46088) 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a77fe640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f62a81395b0 msgr2=0x7f62a8139a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 1 -- 172.21.15.89:0/2628514900 learned_addr learned my addr 172.21.15.89:0/2628514900 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f629ffff640 10 monclient(hunting): get_auth_request con 0x7f62a80a6d30 auth_method 0 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f629ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f629ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f629ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a77fe640 1 --2- 172.21.15.89:0/2628514900 >> v2:172.21.15.190:3300/0 conn(0x7f62a81395b0 0x7f62a8139a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 10 monclient(hunting): get_auth_request con 0x7f62a80a5270 auth_method 0 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:05.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:05.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:05.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:05.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:05.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:05.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:05.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:05.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 10 monclient(hunting): handle_auth_done global_id 5660 payload 274 2024-08-19T10:43:05.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 1 -- 172.21.15.89:0/2628514900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a6d30 msgr2=0x7f62a80a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:05.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 1 --2- 172.21.15.89:0/2628514900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a6d30 0x7f62a80a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 1 -- 172.21.15.89:0/2628514900 >> v2:172.21.15.190:3300/0 conn(0x7f62a81395b0 msgr2=0x7f62a8139a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:05.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 1 --2- 172.21.15.89:0/2628514900 >> v2:172.21.15.190:3300/0 conn(0x7f62a81395b0 0x7f62a8139a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:43:05.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 1 monclient: found mon.noname-e 2024-08-19T10:43:05.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:05.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:43:05.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f62a6ffd640 1 -- 172.21.15.89:0/2628514900 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a80a81d0 con 0x7f62a80a5270 2024-08-19T10:43:05.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.978+0000 7f629ffff640 1 --2- 172.21.15.89:0/2628514900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a6d30 0x7f62a80a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:43:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 1 -- 172.21.15.89:0/2628514900 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f62a80a5270 msgr2=0x7f62a80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 1 --2- 172.21.15.89:0/2628514900 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f62a80a5270 0x7f62a80a67f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6298009870 tx=0x7f629801c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=94427522dd2928bd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 1 -- 172.21.15.89:0/2628514900 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62980b1070 con 0x7f62a80a5270 2024-08-19T10:43:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:43:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 10 monclient: dump: 2024-08-19T10:43:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.995 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 10 monclient: _finish_auth 0 2024-08-19T10:43:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:43:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 1 -- 172.21.15.89:0/2628514900 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62980ab830 con 0x7f62a80a5270 2024-08-19T10:43:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 5 monclient: authenticate success, global_id 5660 2024-08-19T10:43:05.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:05.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:05.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:05.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 4 set_mon_vals no callback set 2024-08-19T10:43:05.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 1 -- 172.21.15.89:0/2628514900 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62980aa320 con 0x7f62a80a5270 2024-08-19T10:43:05.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:05.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a67fc640 10 monclient: dump: 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:05.998 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient: shutdown 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 -- 172.21.15.89:0/2628514900 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f62a80a5270 msgr2=0x7f62a80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 --2- 172.21.15.89:0/2628514900 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f62a80a5270 0x7f62a80a67f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6298009870 tx=0x7f629801c440 comp rx=0 tx=0).stop 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 -- 172.21.15.89:0/2628514900 shutdown_connections 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 --2- 172.21.15.89:0/2628514900 >> v2:172.21.15.190:3300/0 conn(0x7f62a81395b0 0x7f62a8139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:05.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 --2- 172.21.15.89:0/2628514900 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a6d30 0x7f62a80a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 --2- 172.21.15.89:0/2628514900 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f62a80a5270 0x7f62a80a67f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 -- 172.21.15.89:0/2628514900 >> 172.21.15.89:0/2628514900 conn(0x7f62a8069a80 msgr2=0x7f62a80abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 -- 172.21.15.89:0/2628514900 shutdown_connections 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 -- 172.21.15.89:0/2628514900 wait complete. 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient: build_initial_monmap 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient: monmap: 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:06.000 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 Processor -- start 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 -- start start 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient: init 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:06.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient: _add_conns ranks=[2,0,1,3,4,5,7,6,8] 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a5270 0x7f62a81d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient(hunting): picked mon.noname-b con 0x7f62a80a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a80a6d30 0x7f62a80a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient(hunting): picked mon.noname-a con 0x7f62a80a6d30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a81395b0 0x7f62a813d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient(hunting): picked mon.noname-f con 0x7f62a81395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f62a80a5650 con 0x7f62a80a6d30 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f62a813a350 con 0x7f62a81395b0 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f62a80a57d0 con 0x7f62a80a5270 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62ad462640 10 monclient(hunting): authenticate will time out at 1235.803649s 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a5270 0x7f62a81d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a5270 0x7f62a81d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43828/0 (socket says 172.21.15.89:43828) 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 1 -- 172.21.15.89:0/3526446420 learned_addr learned my addr 172.21.15.89:0/3526446420 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 10 monclient(hunting): get_auth_request con 0x7f62a80a5270 auth_method 0 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f629ffff640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a80a6d30 0x7f62a80a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f629ffff640 10 monclient(hunting): get_auth_request con 0x7f62a80a6d30 auth_method 0 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f629ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f629ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f629ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a77fe640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a81395b0 msgr2=0x7f62a813d950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a77fe640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a81395b0 0x7f62a813d950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f62a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.982+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 10 monclient(hunting): handle_auth_done global_id 5744 payload 824 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a81395b0 msgr2=0x7f62a813d950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a81395b0 0x7f62a813d950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a80a6d30 msgr2=0x7f62a80a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a80a6d30 0x7f62a80a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 1 monclient: found mon.noname-b 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 1 -- 172.21.15.89:0/3526446420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a809ca00 con 0x7f62a80a5270 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a5270 msgr2=0x7f62a81d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a6ffd640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a5270 0x7f62a81d1140 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f629801cce0 tx=0x7f62980191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62980ba070 con 0x7f62a80a5270 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 10 monclient: dump: 2024-08-19T10:43:06.005 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.006 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.007 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.007 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:43:06.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:43:06.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:06.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62980044b0 con 0x7f62a80a5270 2024-08-19T10:43:06.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62ad462640 5 monclient: authenticate success, global_id 5744 2024-08-19T10:43:06.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62ad462640 10 monclient: _renew_subs 2024-08-19T10:43:06.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62ad462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:06.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62ad462640 1 -- 172.21.15.89:0/3526446420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a809d020 con 0x7f62a80a5270 2024-08-19T10:43:06.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f62980b1040 con 0x7f62a80a5270 2024-08-19T10:43:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 10 monclient: dump: 2024-08-19T10:43:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.008 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62a5ffb640 4 set_mon_vals no callback set 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62ad462640 10 monclient: _renew_subs 2024-08-19T10:43:06.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62ad462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62ad462640 1 -- 172.21.15.89:0/3526446420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a813ff50 con 0x7f62a80a5270 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62ad462640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62ad462640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a813de90 0x7f62a813e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62ad462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f62ad462640 1 -- 172.21.15.89:0/3526446420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f62a81399e0 con 0x7f62a813de90 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629ffff640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a813de90 0x7f62a813e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629ffff640 10 monclient: get_auth_request con 0x7f62a813de90 auth_method 0 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6298004040 con 0x7f62a80a5270 2024-08-19T10:43:06.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629ffff640 10 monclient(hunting): handle_auth_done global_id 5753 payload 824 2024-08-19T10:43:06.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629ffff640 1 -- 172.21.15.89:0/3526446420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62a809d490 con 0x7f62a813de90 2024-08-19T10:43:06.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.986+0000 7f629f7fe640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f628803be00 0x7f628803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(156..156 src has 1..156) ==== 9265+0+0 (secure 0 0 0) 0x7f62980e8d30 con 0x7f62a80a5270 2024-08-19T10:43:06.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f62a77fe640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f628803be00 0x7f628803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f62a77fe640 10 monclient: get_auth_request con 0x7f628803be00 auth_method 0 2024-08-19T10:43:06.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629ffff640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a813de90 msgr2=0x7f62a813e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629ffff640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a813de90 0x7f62a813e270 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f62900119d0 tx=0x7f6290011e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6290019070 con 0x7f62a813de90 2024-08-19T10:43:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 10 monclient: dump: 2024-08-19T10:43:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.014 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.015 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.016 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f629000f8f0 con 0x7f62a813de90 2024-08-19T10:43:06.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:06.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a813de90 msgr2=0x7f62a813e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a813de90 0x7f62a813e270 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f62900119d0 tx=0x7f6290011e90 comp rx=0 tx=0).stop 2024-08-19T10:43:06.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a813de90 msgr2=0x7f62a813e270 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f629f7fe640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a813de90 0x7f62a813e270 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f62900119d0 tx=0x7f6290011e90 comp rx=0 tx=0).stop 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.990+0000 7f62a77fe640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f628803be00 0x7f628803e2c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f62a0003c70 tx=0x7f62a00091f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f62ad462640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f62ad462640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a8132770 0x7f62a8132b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f62ad462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f62ad462640 1 -- 172.21.15.89:0/3526446420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f62a813e7b0 con 0x7f62a8132770 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a8132770 0x7f62a8132b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 10 monclient: get_auth_request con 0x7f62a8132770 auth_method 0 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 10 monclient(hunting): handle_auth_done global_id 5762 payload 824 2024-08-19T10:43:06.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 1 -- 172.21.15.89:0/3526446420 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f62a80a7110 con 0x7f62a8132770 2024-08-19T10:43:06.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a8132770 msgr2=0x7f62a8132b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629ffff640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a8132770 0x7f62a8132b80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f629001b000 tx=0x7f62900046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6290019040 con 0x7f62a8132770 2024-08-19T10:43:06.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629f7fe640 10 monclient: dump: 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.019 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.020 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.020 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.020 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.020 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9401 (secure 0 0 0) 0x7f6290003e10 con 0x7f62a8132770 2024-08-19T10:43:06.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:43:06.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629f7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:06.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a8132770 msgr2=0x7f62a8132b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629f7fe640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a8132770 0x7f62a8132b80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f629001b000 tx=0x7f62900046d0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629f7fe640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a8132770 msgr2=0x7f62a8132b80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:06.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:05.998+0000 7f629f7fe640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a8132770 0x7f62a8132b80 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f629001b000 tx=0x7f62900046d0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.021 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:43:06.021 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:43:06.021 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:43:06.021 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:43:06.021 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 60, 2024-08-19T10:43:06.021 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 315828, 2024-08-19T10:43:06.021 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:43:06.021 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:43:06.021 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:43:06.021 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:43:06.021 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:43:06.022 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:43:06.022 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:43:06.022 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.022 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:43:06.022 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:06.022 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:43:06.022 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:43:06.022 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.022 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.022 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.022 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.023 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:43:06.024 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:43:06.025 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:06.026 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:43:06.026 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.026 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.026 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.026 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.026 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.026 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.026 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.026 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.026 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.026 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.027 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.027 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.027 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:43:06.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.027 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:43:06.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.027 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:43:06.027 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:43:06.027 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.028 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:43:06.028 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.028 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.029 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:43:06.030 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:43:06.031 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.034 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:43:06.034 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.035 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:43:06.039 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.040 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.043 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:43:06.044 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.045 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:43:06.046 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.046 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.046 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:43:06.046 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.046 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.046 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.046 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:43:06.046 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.046 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.046 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:43:06.046 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.047 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.047 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.047 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:43:06.047 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.047 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.047 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:43:06.047 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:43:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62ad462640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f628803be00 msgr2=0x7f628803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62ad462640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f628803be00 0x7f628803e2c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f62a0003c70 tx=0x7f62a00091f0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62ad462640 10 monclient: shutdown 2024-08-19T10:43:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62ad462640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62ad462640 1 -- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a5270 msgr2=0x7f62a81d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62ad462640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a5270 0x7f62a81d1140 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f629801cce0 tx=0x7f62980191e0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62a77fe640 1 -- 172.21.15.89:0/3526446420 reap_dead start 2024-08-19T10:43:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62ad462640 1 -- 172.21.15.89:0/3526446420 shutdown_connections 2024-08-19T10:43:06.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62ad462640 1 --2- 172.21.15.89:0/3526446420 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f62a80a5270 0x7f62a81d1140 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62ad462640 1 -- 172.21.15.89:0/3526446420 >> 172.21.15.89:0/3526446420 conn(0x7f62a8069a80 msgr2=0x7f62a8134990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62ad462640 1 -- 172.21.15.89:0/3526446420 shutdown_connections 2024-08-19T10:43:06.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.002+0000 7f62ad462640 1 -- 172.21.15.89:0/3526446420 wait complete. 2024-08-19T10:43:06.051 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:43:06.051 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-08-19T10:43:06.051 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:43:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient: get_monmap_and_config 2024-08-19T10:43:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient: build_initial_monmap 2024-08-19T10:43:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient: monmap: 2024-08-19T10:43:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:43:06.165877+0000 2024-08-19T10:43:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:43:06.165877+0000 2024-08-19T10:43:06.165 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:06.166 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:06.167 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 1 Processor -- start 2024-08-19T10:43:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 1 -- start start 2024-08-19T10:43:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient: init 2024-08-19T10:43:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient: _add_conns ranks=[7,1,5,6,2,10,0,8,4,9,3] 2024-08-19T10:43:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5340a56b0 0x7fa5340a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient(hunting): picked mon.noname-i con 0x7fa5340a56b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa5340a1660 addr v1:172.21.15.190:6789/0 2024-08-19T10:43:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa534072250 0x7fa53413e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient(hunting): picked mon.noname-h con 0x7fa534072250 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa534074010 con 0x7fa5340a1660 2024-08-19T10:43:06.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa534072010 con 0x7fa534072250 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa534073d30 con 0x7fa5340a56b0 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa538d5c640 10 monclient(hunting): authenticate will time out at 1235.983202s 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa531d9b640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fa5340a1660 legacy=0x7fa5340a5fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa53259c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5340a56b0 0x7fa5340a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa532d9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa534072250 0x7fa53413e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa53259c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5340a56b0 0x7fa5340a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:45974/0 (socket says 172.21.15.89:45974) 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa53259c640 1 -- 172.21.15.89:0/1037445913 learned_addr learned my addr 172.21.15.89:0/1037445913 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa53259c640 10 monclient(hunting): get_auth_request con 0x7fa5340a56b0 auth_method 0 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa53259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa53259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa53259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa532d9d640 10 monclient(hunting): get_auth_request con 0x7fa534072250 auth_method 0 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa532d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa532d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa532d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa53259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa53259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa53259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa532d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa532d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.162+0000 7fa532d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.166+0000 7fa532d9d640 10 monclient(hunting): handle_auth_done global_id 5765 payload 274 2024-08-19T10:43:06.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.166+0000 7fa532d9d640 1 -- 172.21.15.89:0/1037445913 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5340a56b0 msgr2=0x7fa5340a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.166+0000 7fa532d9d640 1 --2- 172.21.15.89:0/1037445913 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5340a56b0 0x7fa5340a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.166+0000 7fa532d9d640 1 -- 172.21.15.89:0/1037445913 >> v1:172.21.15.190:6789/0 conn(0x7fa5340a1660 legacy=0x7fa5340a5fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.166+0000 7fa532d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:43:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.166+0000 7fa532d9d640 1 monclient: found mon.noname-h 2024-08-19T10:43:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.166+0000 7fa532d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.166+0000 7fa532d9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.166+0000 7fa532d9d640 1 -- 172.21.15.89:0/1037445913 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5340a71a0 con 0x7fa534072250 2024-08-19T10:43:06.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.166+0000 7fa53259c640 1 --2- 172.21.15.89:0/1037445913 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5340a56b0 0x7fa5340a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:43:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 1 -- 172.21.15.89:0/1037445913 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa534072250 msgr2=0x7fa53413e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 1 --2- 172.21.15.89:0/1037445913 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa534072250 0x7fa53413e6b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009dc0 tx=0x7fa52c01c8e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=51b517c34983436c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 1 -- 172.21.15.89:0/1037445913 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa52c0b2070 con 0x7fa534072250 2024-08-19T10:43:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 10 monclient: dump: 2024-08-19T10:43:06.177 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.178 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 10 monclient: _finish_auth 0 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 10 monclient: _check_auth_tickets 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 1 -- 172.21.15.89:0/1037445913 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52c0a5c50 con 0x7fa534072250 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 1 -- 172.21.15.89:0/1037445913 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa52c0a5e00 con 0x7fa534072250 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 5 monclient: authenticate success, global_id 5765 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 4 set_mon_vals no callback set 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:06.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53159a640 10 monclient: dump: 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.180 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient: shutdown 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 -- 172.21.15.89:0/1037445913 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa534072250 msgr2=0x7fa53413e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 --2- 172.21.15.89:0/1037445913 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa534072250 0x7fa53413e6b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009dc0 tx=0x7fa52c01c8e0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 -- 172.21.15.89:0/1037445913 shutdown_connections 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 --2- 172.21.15.89:0/1037445913 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa534072250 0x7fa53413e6b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 --2- 172.21.15.89:0/1037445913 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5340a56b0 0x7fa5340a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 -- 172.21.15.89:0/1037445913 >> 172.21.15.89:0/1037445913 conn(0x7fa5340aa2a0 msgr2=0x7fa5340a67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 -- 172.21.15.89:0/1037445913 shutdown_connections 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 -- 172.21.15.89:0/1037445913 wait complete. 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient: build_initial_monmap 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient: monmap: 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:06.182 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 Processor -- start 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 -- start start 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient: init 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient: _add_conns ranks=[4,5,8,2,1,7,0,6,3] 2024-08-19T10:43:06.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5340a56b0 0x7fa53406da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient(hunting): picked mon.noname-c con 0x7fa5340a56b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53406df80 0x7fa53406e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient(hunting): picked mon.noname-h con 0x7fa53406df80 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa5341325a0 0x7fa534132a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient(hunting): picked mon.noname-e con 0x7fa5341325a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa5341417a0 con 0x7fa5340a56b0 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa534141620 con 0x7fa53406df80 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa534141920 con 0x7fa5341325a0 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:06.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa538d5c640 10 monclient(hunting): authenticate will time out at 1235.991359s 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53259c640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5340a56b0 msgr2=0x7fa53406da40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa53259c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5340a56b0 0x7fa53406da40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa5341325a0 0x7fa534132a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa5341325a0 0x7fa534132a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:46104/0 (socket says 172.21.15.89:46104) 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 1 -- 172.21.15.89:0/3404081733 learned_addr learned my addr 172.21.15.89:0/3404081733 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 10 monclient(hunting): get_auth_request con 0x7fa5341325a0 auth_method 0 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa531d9b640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53406df80 0x7fa53406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa531d9b640 10 monclient(hunting): get_auth_request con 0x7fa53406df80 auth_method 0 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa531d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa531d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa531d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 10 monclient(hunting): handle_auth_done global_id 5669 payload 824 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 1 -- 172.21.15.89:0/3404081733 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53406df80 msgr2=0x7fa53406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa53406df80 0x7fa53406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 1 -- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5340a56b0 msgr2=0x7fa53406da40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa5340a56b0 0x7fa53406da40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 1 monclient: found mon.noname-e 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:06.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:43:06.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.170+0000 7fa532d9d640 1 -- 172.21.15.89:0/3404081733 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53413cee0 con 0x7fa5341325a0 2024-08-19T10:43:06.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa532d9d640 1 -- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa5341325a0 msgr2=0x7fa534132a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa532d9d640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa5341325a0 0x7fa534132a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa52c01c710 tx=0x7fa52c019680 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa52c0bb030 con 0x7fa5341325a0 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 10 monclient: dump: 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.191 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 10 monclient: _finish_auth 0 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa538d5c640 5 monclient: authenticate success, global_id 5669 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa538d5c640 10 monclient: _renew_subs 2024-08-19T10:43:06.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa538d5c640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa538d5c640 1 -- 172.21.15.89:0/3404081733 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53413d1c0 con 0x7fa5341325a0 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa538d5c640 10 monclient: _renew_subs 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa538d5c640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa538d5c640 1 -- 172.21.15.89:0/3404081733 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53413cc00 con 0x7fa5341325a0 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52c004040 con 0x7fa5341325a0 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa52c0b2040 con 0x7fa5341325a0 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa530d99640 4 set_mon_vals no callback set 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 10 monclient: dump: 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.193 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.194 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa538d5c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa538d5c640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518002890 0x7fa518004d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa538d5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa538d5c640 1 -- 172.21.15.89:0/3404081733 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa518007cc0 con 0x7fa518002890 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa53259c640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518002890 0x7fa518004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa53259c640 10 monclient: get_auth_request con 0x7fa518002890 auth_method 0 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa53259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa53259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa53259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa53259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa53259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa53259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa52c0be070 con 0x7fa5341325a0 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa53259c640 10 monclient(hunting): handle_auth_done global_id 5678 payload 824 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa53259c640 1 -- 172.21.15.89:0/3404081733 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa518007ec0 con 0x7fa518002890 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa51403be00 0x7fa51403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.174+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(156..156 src has 1..156) ==== 9265+0+0 (secure 0 0 0) 0x7fa52c0bc070 con 0x7fa5341325a0 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa531d9b640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa51403be00 0x7fa51403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa531d9b640 10 monclient: get_auth_request con 0x7fa51403be00 auth_method 0 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa53259c640 1 -- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518002890 msgr2=0x7fa518004d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa53259c640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518002890 0x7fa518004d30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa524006c90 tx=0x7fa52400ec00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa524017070 con 0x7fa518002890 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa522ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa522ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa522ffd640 10 monclient: dump: 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.197 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa52400ee10 con 0x7fa518002890 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa522ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa522ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518002890 msgr2=0x7fa518004d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa522ffd640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518002890 0x7fa518004d30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa524006c90 tx=0x7fa52400ec00 comp rx=0 tx=0).stop 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518002890 msgr2=0x7fa518004d30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa522ffd640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518002890 0x7fa518004d30 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa524006c90 tx=0x7fa52400ec00 comp rx=0 tx=0).stop 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.178+0000 7fa531d9b640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa51403be00 0x7fa51403e2c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa528006fd0 tx=0x7fa528008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa538d5c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa538d5c640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518006ce0 0x7fa518007170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa538d5c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa538d5c640 1 -- 172.21.15.89:0/3404081733 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa51800a200 con 0x7fa518006ce0 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518006ce0 0x7fa518007170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 10 monclient: get_auth_request con 0x7fa518006ce0 auth_method 0 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 10 monclient(hunting): handle_auth_done global_id 5687 payload 824 2024-08-19T10:43:06.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 1 -- 172.21.15.89:0/3404081733 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa5180076b0 con 0x7fa518006ce0 2024-08-19T10:43:06.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 1 -- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518006ce0 msgr2=0x7fa518007170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa53259c640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518006ce0 0x7fa518007170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa524019000 tx=0x7fa524004740 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa524017070 con 0x7fa518006ce0 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa522ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa522ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa522ffd640 10 monclient: dump: 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.201 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9568 (secure 0 0 0) 0x7fa524006650 con 0x7fa518006ce0 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa522ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa522ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:06.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518006ce0 msgr2=0x7fa518007170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa522ffd640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518006ce0 0x7fa518007170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa524019000 tx=0x7fa524004740 comp rx=0 tx=0).stop 2024-08-19T10:43:06.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa522ffd640 1 -- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518006ce0 msgr2=0x7fa518007170 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:06.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.186+0000 7fa522ffd640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa518006ce0 0x7fa518007170 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa524019000 tx=0x7fa524004740 comp rx=0 tx=0).stop 2024-08-19T10:43:06.203 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:43:06.203 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:43:06.203 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:43:06.203 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:43:06.203 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 60, 2024-08-19T10:43:06.203 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 315980, 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:43:06.204 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:43:06.205 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:43:06.207 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.208 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.210 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.211 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.212 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.213 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:43:06.215 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:43:06.216 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.218 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.219 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.220 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:43:06.223 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:43:06.224 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.226 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.227 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.228 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:43:06.229 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:43:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa538d5c640 1 -- 172.21.15.89:0/3404081733 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa51403be00 msgr2=0x7fa51403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa538d5c640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa51403be00 0x7fa51403e2c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa528006fd0 tx=0x7fa528008040 comp rx=0 tx=0).stop 2024-08-19T10:43:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa538d5c640 10 monclient: shutdown 2024-08-19T10:43:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa538d5c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa538d5c640 1 -- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa5341325a0 msgr2=0x7fa534132a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa538d5c640 1 --2- 172.21.15.89:0/3404081733 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa5341325a0 0x7fa534132a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa52c01c710 tx=0x7fa52c019680 comp rx=0 tx=0).stop 2024-08-19T10:43:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa532d9d640 1 -- 172.21.15.89:0/3404081733 reap_dead start 2024-08-19T10:43:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa532d9d640 1 -- 172.21.15.89:0/3404081733 reap_dead start 2024-08-19T10:43:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa538d5c640 1 -- 172.21.15.89:0/3404081733 shutdown_connections 2024-08-19T10:43:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa538d5c640 1 -- 172.21.15.89:0/3404081733 >> 172.21.15.89:0/3404081733 conn(0x7fa5340aa2a0 msgr2=0x7fa5340a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa538d5c640 1 -- 172.21.15.89:0/3404081733 shutdown_connections 2024-08-19T10:43:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.190+0000 7fa538d5c640 1 -- 172.21.15.89:0/3404081733 wait complete. 2024-08-19T10:43:06.231 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:43:06.231 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-08-19T10:43:06.232 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:43:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient: get_monmap_and_config 2024-08-19T10:43:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient: build_initial_monmap 2024-08-19T10:43:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient: monmap: 2024-08-19T10:43:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:43:06.345594+0000 2024-08-19T10:43:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:43:06.345594+0000 2024-08-19T10:43:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:06.345 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:06.346 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:06.346 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:06.346 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:06.346 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:06.346 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:06.346 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:06.346 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:06.346 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:06.346 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:06.346 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 1 Processor -- start 2024-08-19T10:43:06.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 1 -- start start 2024-08-19T10:43:06.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient: init 2024-08-19T10:43:06.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:06.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient: _add_conns ranks=[9,3,0,8,6,7,5,2,4,1,10] 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f280a6860 0x7f3f280a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient(hunting): picked mon.noname-a con 0x7f3f280a6860 addr v2:172.21.15.89:3300/0 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280a7180 0x7f3f280a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient(hunting): picked mon.noname-g con 0x7f3f280a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3f28139620 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f280a33d0 con 0x7f3f28139620 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f3f280a8190 con 0x7f3f280a6860 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3f280a2ec0 con 0x7f3f280a7180 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:06.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f2de8a640 10 monclient(hunting): authenticate will time out at 1236.162958s 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3f280a6860 0x7f3f280a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f27fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3f28139620 0x7f3f28139ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:33868/0 (socket says 172.21.15.89:33868) 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f27fff640 1 -- 172.21.15.89:0/3653771361 learned_addr learned my addr 172.21.15.89:0/3653771361 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 10 monclient(hunting): get_auth_request con 0x7f3f280a6860 auth_method 0 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 10 monclient(hunting): handle_auth_done global_id 5823 payload 274 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 1 -- 172.21.15.89:0/3653771361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280a7180 msgr2=0x7f3f280a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:06.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 1 --2- 172.21.15.89:0/3653771361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280a7180 0x7f3f280a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 1 -- 172.21.15.89:0/3653771361 >> v1:172.21.15.89:6789/0 conn(0x7f3f28139620 legacy=0x7f3f28139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:06.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:06.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 1 monclient: found mon.noname-a 2024-08-19T10:43:06.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:06.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:43:06.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.342+0000 7f3f277fe640 1 -- 172.21.15.89:0/3653771361 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f280a4cd0 con 0x7f3f280a6860 2024-08-19T10:43:06.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f277fe640 1 -- 172.21.15.89:0/3653771361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f280a6860 msgr2=0x7f3f280a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f277fe640 1 --2- 172.21.15.89:0/3653771361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f280a6860 0x7f3f280a6c40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3f180097c0 tx=0x7f3f1801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8158e4cbe249c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 1 -- 172.21.15.89:0/3653771361 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f180b0070 con 0x7f3f280a6860 2024-08-19T10:43:06.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:43:06.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 10 monclient: dump: 2024-08-19T10:43:06.357 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.357 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.357 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.357 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.357 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.357 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.357 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.357 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.357 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.357 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 10 monclient: _finish_auth 0 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 10 monclient: _check_auth_tickets 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 1 -- 172.21.15.89:0/3653771361 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f180aa810 con 0x7f3f280a6860 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f2de8a640 5 monclient: authenticate success, global_id 5823 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f2de8a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:06.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f2de8a640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 1 -- 172.21.15.89:0/3653771361 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f180aae30 con 0x7f3f280a6860 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f2de8a640 4 set_mon_vals no callback set 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.350+0000 7f3f267fc640 10 monclient: dump: 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.359 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient: shutdown 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 -- 172.21.15.89:0/3653771361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f280a6860 msgr2=0x7f3f280a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 --2- 172.21.15.89:0/3653771361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f280a6860 0x7f3f280a6c40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3f180097c0 tx=0x7f3f1801c360 comp rx=0 tx=0).stop 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 -- 172.21.15.89:0/3653771361 shutdown_connections 2024-08-19T10:43:06.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 --2- 172.21.15.89:0/3653771361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280a7180 0x7f3f280a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 --2- 172.21.15.89:0/3653771361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f280a6860 0x7f3f280a6c40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 -- 172.21.15.89:0/3653771361 >> 172.21.15.89:0/3653771361 conn(0x7f3f28074b40 msgr2=0x7f3f280abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 -- 172.21.15.89:0/3653771361 shutdown_connections 2024-08-19T10:43:06.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 -- 172.21.15.89:0/3653771361 wait complete. 2024-08-19T10:43:06.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient: build_initial_monmap 2024-08-19T10:43:06.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient: monmap: 2024-08-19T10:43:06.366 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:06.366 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:06.367 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 Processor -- start 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 -- start start 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient: init 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient: _add_conns ranks=[2,1,4,3,6,5,0,7,8] 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f280a6860 0x7f3f2813f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient(hunting): picked mon.noname-b con 0x7f3f280a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f280a7180 0x7f3f281d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient(hunting): picked mon.noname-f con 0x7f3f280a7180 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f28139620 0x7f3f2813d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient(hunting): picked mon.noname-c con 0x7f3f28139620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3f28144430 con 0x7f3f280a7180 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3f281442b0 con 0x7f3f280a6860 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3f281445b0 con 0x7f3f28139620 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f2de8a640 10 monclient(hunting): authenticate will time out at 1236.173845s 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f27fff640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f28139620 msgr2=0x7f3f2813d2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f27fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f28139620 0x7f3f2813d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f280a6860 0x7f3f2813f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f280a6860 0x7f3f2813f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43854/0 (socket says 172.21.15.89:43854) 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 1 -- 172.21.15.89:0/1979460514 learned_addr learned my addr 172.21.15.89:0/1979460514 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 10 monclient(hunting): get_auth_request con 0x7f3f280a6860 auth_method 0 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f26ffd640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f280a7180 msgr2=0x7f3f281d1a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f26ffd640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f280a7180 0x7f3f281d1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 10 monclient(hunting): handle_auth_done global_id 5771 payload 824 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f28139620 msgr2=0x7f3f2813d2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f28139620 0x7f3f2813d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f280a7180 msgr2=0x7f3f281d1a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f280a7180 0x7f3f281d1a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:06.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 1 monclient: found mon.noname-b 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.354+0000 7f3f277fe640 1 -- 172.21.15.89:0/1979460514 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f28122c80 con 0x7f3f280a6860 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f277fe640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f280a6860 msgr2=0x7f3f2813f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f277fe640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f280a6860 0x7f3f2813f870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3f1801aa10 tx=0x7f3f18019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f180b0040 con 0x7f3f280a6860 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f277fe640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f280a6860 0x7f3f2813f870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3f1801aa10 tx=0x7f3f18019100 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f3f18004250 mon_map magic: 0 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 10 monclient: dump: 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.371 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 10 monclient: _finish_auth 0 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f2de8a640 5 monclient: authenticate success, global_id 5771 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f180aae40 con 0x7f3f280a6860 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f2de8a640 10 monclient: _renew_subs 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f2de8a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f2de8a640 1 -- 172.21.15.89:0/1979460514 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f28139420 con 0x7f3f280a6860 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f2de8a640 10 monclient: _renew_subs 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f257fa640 4 set_mon_vals no callback set 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f2de8a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f2de8a640 1 -- 172.21.15.89:0/1979460514 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f281229a0 con 0x7f3f280a6860 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f2de8a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f2de8a640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f281d1fc0 0x7f3f2802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f2de8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f2de8a640 1 -- 172.21.15.89:0/1979460514 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3f280ac0a0 con 0x7f3f281d1fc0 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f18004250 con 0x7f3f280a6860 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 10 monclient: dump: 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.374 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3f180bb030 con 0x7f3f280a6860 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f0ffff640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f1403be80 0x7f3f1403e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f26ffd640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f281d1fc0 0x7f3f2802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f26ffd640 10 monclient: get_auth_request con 0x7f3f281d1fc0 auth_method 0 2024-08-19T10:43:06.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f26ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f26ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f26ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f26ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f26ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f26ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f27fff640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f1403be80 0x7f3f1403e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f27fff640 10 monclient: get_auth_request con 0x7f3f1403be80 auth_method 0 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f26ffd640 10 monclient(hunting): handle_auth_done global_id 34212 payload 824 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.358+0000 7f3f26ffd640 1 -- 172.21.15.89:0/1979460514 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f2813ed40 con 0x7f3f281d1fc0 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f27fff640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f1403be80 0x7f3f1403e340 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3f20004ba0 tx=0x7f3f20004ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f26ffd640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f281d1fc0 msgr2=0x7f3f2802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f26ffd640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f281d1fc0 0x7f3f2802d1d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3f2813e9c0 tx=0x7f3f1c00c8e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f1c017070 con 0x7f3f281d1fc0 2024-08-19T10:43:06.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f0ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f0ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f0ffff640 10 monclient: dump: 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.377 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.378 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3f1c0070d0 con 0x7f3f281d1fc0 2024-08-19T10:43:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f0ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f0ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f281d1fc0 msgr2=0x7f3f2802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f0ffff640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f281d1fc0 0x7f3f2802d1d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3f2813e9c0 tx=0x7f3f1c00c8e0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f281d1fc0 msgr2=0x7f3f2802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:06.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.362+0000 7f3f0ffff640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f281d1fc0 0x7f3f2802d1d0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3f2813e9c0 tx=0x7f3f1c00c8e0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f2de8a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:43:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f2de8a640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2802d710 0x7f3f2802db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f2de8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f2de8a640 1 -- 172.21.15.89:0/1979460514 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3f280a6c40 con 0x7f3f2802d710 2024-08-19T10:43:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2802d710 0x7f3f2802db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 10 monclient: get_auth_request con 0x7f3f2802d710 auth_method 0 2024-08-19T10:43:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 10 monclient(hunting): handle_auth_done global_id 34221 payload 824 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 1 -- 172.21.15.89:0/1979460514 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3f28064590 con 0x7f3f2802d710 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2802d710 msgr2=0x7f3f2802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f26ffd640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2802d710 0x7f3f2802db20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c014000 tx=0x7f3f1c000f60 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f1c017070 con 0x7f3f2802d710 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f0ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f0ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.370+0000 7f3f0ffff640 10 monclient: dump: 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.381 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.382 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.383 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9233 (secure 0 0 0) 0x7f3f1c004450 con 0x7f3f2802d710 2024-08-19T10:43:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f0ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:43:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f0ffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2802d710 msgr2=0x7f3f2802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f0ffff640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2802d710 0x7f3f2802db20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c014000 tx=0x7f3f1c000f60 comp rx=0 tx=0).stop 2024-08-19T10:43:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2802d710 msgr2=0x7f3f2802db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:06.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f0ffff640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f2802d710 0x7f3f2802db20 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c014000 tx=0x7f3f1c000f60 comp rx=0 tx=0).stop 2024-08-19T10:43:06.384 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:43:06.384 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:43:06.384 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:43:06.384 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:43:06.384 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 60, 2024-08-19T10:43:06.384 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 31415, 2024-08-19T10:43:06.384 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:43:06.384 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:43:06.384 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:43:06.384 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.385 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.386 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.386 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.386 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.386 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.386 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.386 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.386 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.386 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:43:06.386 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:43:06.386 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.387 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:43:06.388 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.389 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:43:06.390 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.391 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:43:06.393 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.394 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:43:06.396 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:43:06.400 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.403 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.405 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.406 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.407 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:43:06.407 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:43:06.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f2de8a640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f1403be80 msgr2=0x7f3f1403e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f2de8a640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f1403be80 0x7f3f1403e340 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3f20004ba0 tx=0x7f3f20004ff0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f2de8a640 10 monclient: shutdown 2024-08-19T10:43:06.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f2de8a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:06.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f2de8a640 1 -- 172.21.15.89:0/1979460514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f280a6860 msgr2=0x7f3f2813f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f2de8a640 1 --2- 172.21.15.89:0/1979460514 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f280a6860 0x7f3f2813f870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3f1801aa10 tx=0x7f3f18019100 comp rx=0 tx=0).stop 2024-08-19T10:43:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f27fff640 1 -- 172.21.15.89:0/1979460514 reap_dead start 2024-08-19T10:43:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f0ffff640 1 -- 172.21.15.89:0/1979460514 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(156..156 src has 1..156) ==== 9265+0+0 (unknown 0 0 0) 0x7f3f180b9020 con 0x7f3f280a6860 2024-08-19T10:43:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f27fff640 1 -- 172.21.15.89:0/1979460514 reap_dead start 2024-08-19T10:43:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f2de8a640 1 -- 172.21.15.89:0/1979460514 shutdown_connections 2024-08-19T10:43:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f2de8a640 1 -- 172.21.15.89:0/1979460514 >> 172.21.15.89:0/1979460514 conn(0x7f3f28074b40 msgr2=0x7f3f280a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f2de8a640 1 -- 172.21.15.89:0/1979460514 shutdown_connections 2024-08-19T10:43:06.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.374+0000 7f3f2de8a640 1 -- 172.21.15.89:0/1979460514 wait complete. 2024-08-19T10:43:06.409 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:43:06.409 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-08-19T10:43:06.410 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:43:06.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.411+0000 7faf07fff640 1 -- 172.21.15.190:0/3653778388 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}]=0 v157) ==== 119+0+0 (secure 0 0 0) 0x7faf1004da70 con 0x7faf200a4720 2024-08-19T10:43:06.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.411+0000 7faf07fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [10.5]}] 2024-08-19T10:43:06.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.411+0000 7faf07fff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:06.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 -- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7faf0c03be00 msgr2=0x7faf0c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7faf0c03be00 0x7faf0c03e2c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faf14006e90 tx=0x7faf14008040 comp rx=0 tx=0).stop 2024-08-19T10:43:06.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 10 monclient: shutdown 2024-08-19T10:43:06.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:06.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 -- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 msgr2=0x7faf2013d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf2013d3d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf10009fd0 tx=0x7faf1000ee10 comp rx=0 tx=0).stop 2024-08-19T10:43:06.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 -- 172.21.15.190:0/3653778388 shutdown_connections 2024-08-19T10:43:06.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7faf0c03be00 0x7faf0c03e2c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faf20139e10 0x7faf2013de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf200a5040 0x7faf2013d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 --2- 172.21.15.190:0/3653778388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf200a4720 0x7faf2013d3d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 -- 172.21.15.190:0/3653778388 >> 172.21.15.190:0/3653778388 conn(0x7faf200a9310 msgr2=0x7faf200a1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 -- 172.21.15.190:0/3653778388 shutdown_connections 2024-08-19T10:43:06.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.415+0000 7faf25860640 1 -- 172.21.15.190:0/3653778388 wait complete. 2024-08-19T10:43:06.440 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight cool osd.0 11.5 2024-08-19T10:43:06.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient: get_monmap_and_config 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient: build_initial_monmap 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient: monmap: 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:43:06.523407+0000 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:43:06.523407+0000 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:06.523 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:06.524 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:06.524 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:06.524 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:06.524 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:06.524 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:06.524 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:06.524 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 1 Processor -- start 2024-08-19T10:43:06.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 1 -- start start 2024-08-19T10:43:06.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient: init 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient: _add_conns ranks=[2,10,7,4,8,5,6,9,1,0,3] 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a3d00 0x7f4e5809eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient(hunting): picked mon.noname-b con 0x7f4e580a3d00 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4e580a40e0 0x7f4e580a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient(hunting): picked mon.noname-f con 0x7f4e580a40e0 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e58136440 0x7f4e581368c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient(hunting): picked mon.noname-i con 0x7f4e58136440 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f4e5809e870 con 0x7f4e580a40e0 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4e5809eac0 con 0x7f4e580a3d00 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4e58069630 con 0x7f4e58136440 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e600d7640 10 monclient(hunting): authenticate will time out at 1236.339813s 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e5de73640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a3d00 0x7f4e5809eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e5de73640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a3d00 0x7f4e5809eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43862/0 (socket says 172.21.15.89:43862) 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e5de73640 1 -- 172.21.15.89:0/21160589 learned_addr learned my addr 172.21.15.89:0/21160589 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e5de73640 10 monclient(hunting): get_auth_request con 0x7f4e580a3d00 auth_method 0 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e5de73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e5de73640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e5de73640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.518+0000 7f4e5d672640 1 -- 172.21.15.89:0/21160589 >> v2:172.21.15.190:3300/0 conn(0x7f4e580a40e0 msgr2=0x7f4e580a44c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5d672640 1 --2- 172.21.15.89:0/21160589 >> v2:172.21.15.190:3300/0 conn(0x7f4e580a40e0 0x7f4e580a44c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:06.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 10 monclient(hunting): handle_auth_done global_id 5780 payload 274 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5e674640 1 --2- 172.21.15.89:0/21160589 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e58136440 0x7f4e581368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 1 -- 172.21.15.89:0/21160589 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e58136440 msgr2=0x7f4e581368c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 1 --2- 172.21.15.89:0/21160589 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e58136440 0x7f4e581368c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 1 -- 172.21.15.89:0/21160589 >> v2:172.21.15.190:3300/0 conn(0x7f4e580a40e0 msgr2=0x7f4e580a44c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 1 --2- 172.21.15.89:0/21160589 >> v2:172.21.15.190:3300/0 conn(0x7f4e580a40e0 0x7f4e580a44c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 10 monclient: _finish_hunting 0 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 1 monclient: found mon.noname-b 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5de73640 1 -- 172.21.15.89:0/21160589 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5806db50 con 0x7f4e580a3d00 2024-08-19T10:43:06.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5e674640 10 monclient: get_auth_request con 0x7f4e58136440 auth_method 0 2024-08-19T10:43:06.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.522+0000 7f4e5e674640 1 --2- 172.21.15.89:0/21160589 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e58136440 0x7f4e581368c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:43:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5de73640 1 -- 172.21.15.89:0/21160589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a3d00 msgr2=0x7f4e5809eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5de73640 1 --2- 172.21.15.89:0/21160589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a3d00 0x7f4e5809eee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009870 tx=0x7f4e5401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fe5f54544fccf25 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 1 -- 172.21.15.89:0/21160589 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e540b0070 con 0x7f4e580a3d00 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 10 monclient: dump: 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.537 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 10 monclient: _finish_auth 0 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 10 monclient: _check_auth_tickets 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 1 -- 172.21.15.89:0/21160589 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e540aa810 con 0x7f4e580a3d00 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 1 -- 172.21.15.89:0/21160589 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e540a93e0 con 0x7f4e580a3d00 2024-08-19T10:43:06.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5ce71640 10 monclient: dump: 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.539 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.540 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 5 monclient: authenticate success, global_id 5780 2024-08-19T10:43:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 4 set_mon_vals no callback set 2024-08-19T10:43:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient: shutdown 2024-08-19T10:43:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:06.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 -- 172.21.15.89:0/21160589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a3d00 msgr2=0x7f4e5809eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 --2- 172.21.15.89:0/21160589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a3d00 0x7f4e5809eee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009870 tx=0x7f4e5401c440 comp rx=0 tx=0).stop 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 -- 172.21.15.89:0/21160589 shutdown_connections 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 --2- 172.21.15.89:0/21160589 >> v2:172.21.15.190:3300/0 conn(0x7f4e580a40e0 0x7f4e580a44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 --2- 172.21.15.89:0/21160589 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e58136440 0x7f4e581368c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 --2- 172.21.15.89:0/21160589 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a3d00 0x7f4e5809eee0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 -- 172.21.15.89:0/21160589 >> 172.21.15.89:0/21160589 conn(0x7f4e580a4df0 msgr2=0x7f4e580a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 -- 172.21.15.89:0/21160589 shutdown_connections 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 -- 172.21.15.89:0/21160589 wait complete. 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient: build_initial_monmap 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient: monmap: 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:06.541 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:06.542 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:06.543 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 Processor -- start 2024-08-19T10:43:06.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 -- start start 2024-08-19T10:43:06.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient: init 2024-08-19T10:43:06.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:06.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient: _add_conns ranks=[1,2,8,0,5,4,7,3,6] 2024-08-19T10:43:06.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e580a3d00 0x7f4e5813a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient(hunting): picked mon.noname-f con 0x7f4e580a3d00 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:06.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a40e0 0x7f4e5813ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient(hunting): picked mon.noname-b con 0x7f4e580a40e0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e58136440 0x7f4e5813b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient(hunting): picked mon.noname-e con 0x7f4e58136440 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e580a4b80 con 0x7f4e580a3d00 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4e580a4a00 con 0x7f4e580a40e0 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4e5813feb0 con 0x7f4e58136440 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e600d7640 10 monclient(hunting): authenticate will time out at 1236.346999s 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5de73640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e580a3d00 msgr2=0x7f4e5813a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5de73640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e580a3d00 0x7f4e5813a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:06.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a40e0 0x7f4e5813ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5e674640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e58136440 0x7f4e5813b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a40e0 0x7f4e5813ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43874/0 (socket says 172.21.15.89:43874) 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 1 -- 172.21.15.89:0/2342275816 learned_addr learned my addr 172.21.15.89:0/2342275816 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 10 monclient(hunting): get_auth_request con 0x7f4e580a40e0 auth_method 0 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5e674640 10 monclient(hunting): get_auth_request con 0x7f4e58136440 auth_method 0 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5e674640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5e674640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5e674640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5e674640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5e674640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5e674640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 10 monclient(hunting): handle_auth_done global_id 5789 payload 824 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 1 -- 172.21.15.89:0/2342275816 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e58136440 msgr2=0x7f4e5813b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e58136440 0x7f4e5813b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 1 -- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e580a3d00 msgr2=0x7f4e5813a5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e580a3d00 0x7f4e5813a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 10 monclient: _finish_hunting 0 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 1 monclient: found mon.noname-b 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:06.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.526+0000 7f4e5d672640 1 -- 172.21.15.89:0/2342275816 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5812ed90 con 0x7f4e580a40e0 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5d672640 1 -- 172.21.15.89:0/2342275816 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a40e0 msgr2=0x7f4e5813ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5d672640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a40e0 0x7f4e5813ab10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800f9e0 tx=0x7f4e4800fea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e48003bd0 con 0x7f4e580a40e0 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 10 monclient: dump: 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.547 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:43:06.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e600d7640 5 monclient: authenticate success, global_id 5789 2024-08-19T10:43:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e48014030 con 0x7f4e580a40e0 2024-08-19T10:43:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e600d7640 10 monclient: _renew_subs 2024-08-19T10:43:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e600d7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e600d7640 1 -- 172.21.15.89:0/2342275816 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5812f070 con 0x7f4e580a40e0 2024-08-19T10:43:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e48003e10 con 0x7f4e580a40e0 2024-08-19T10:43:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4ffff640 4 set_mon_vals no callback set 2024-08-19T10:43:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 10 monclient: dump: 2024-08-19T10:43:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.550 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.551 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e600d7640 10 monclient: _renew_subs 2024-08-19T10:43:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e600d7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e600d7640 1 -- 172.21.15.89:0/2342275816 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5813c9e0 con 0x7f4e580a40e0 2024-08-19T10:43:06.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e600d7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e600d7640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cefe0 0x7f4e581cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e600d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e600d7640 1 -- 172.21.15.89:0/2342275816 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4e58072cd0 con 0x7f4e581cefe0 2024-08-19T10:43:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5de73640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cefe0 0x7f4e581cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4e48005070 con 0x7f4e580a40e0 2024-08-19T10:43:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5de73640 10 monclient: get_auth_request con 0x7f4e581cefe0 auth_method 0 2024-08-19T10:43:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5de73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5de73640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5de73640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5de73640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5de73640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5de73640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e3c03be00 0x7f4e3c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(157..157 src has 1..157) ==== 9265+0+0 (secure 0 0 0) 0x7f4e480532e0 con 0x7f4e580a40e0 2024-08-19T10:43:06.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5de73640 10 monclient(hunting): handle_auth_done global_id 5783 payload 824 2024-08-19T10:43:06.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5de73640 1 -- 172.21.15.89:0/2342275816 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e580a51d0 con 0x7f4e581cefe0 2024-08-19T10:43:06.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5e674640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e3c03be00 0x7f4e3c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.530+0000 7f4e5e674640 10 monclient: get_auth_request con 0x7f4e3c03be00 auth_method 0 2024-08-19T10:43:06.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e5de73640 1 -- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cefe0 msgr2=0x7f4e581cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e5de73640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cefe0 0x7f4e581cf3c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4e540ad000 tx=0x7f4e5401ac80 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e540b0040 con 0x7f4e581cefe0 2024-08-19T10:43:06.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e4e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:06.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e4e7fc640 10 monclient: dump: 2024-08-19T10:43:06.556 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.556 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.556 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.556 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e5e674640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e3c03be00 0x7f4e3c03e2c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000f3f0 tx=0x7f4e500076e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4e540093d0 con 0x7f4e581cefe0 2024-08-19T10:43:06.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e4e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e4e7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:06.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cefe0 msgr2=0x7f4e581cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e4e7fc640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cefe0 0x7f4e581cf3c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4e540ad000 tx=0x7f4e5401ac80 comp rx=0 tx=0).stop 2024-08-19T10:43:06.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cefe0 msgr2=0x7f4e581cf3c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:06.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.534+0000 7f4e4e7fc640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cefe0 0x7f4e581cf3c0 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4e540ad000 tx=0x7f4e5401ac80 comp rx=0 tx=0).stop 2024-08-19T10:43:06.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e600d7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:43:06.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e600d7640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cf900 0x7f4e58137450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e600d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e600d7640 1 -- 172.21.15.89:0/2342275816 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4e580a44c0 con 0x7f4e581cf900 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cf900 0x7f4e58137450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 10 monclient: get_auth_request con 0x7f4e581cf900 auth_method 0 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 10 monclient(hunting): handle_auth_done global_id 5792 payload 824 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 1 -- 172.21.15.89:0/2342275816 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4e58064590 con 0x7f4e581cf900 2024-08-19T10:43:06.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 1 -- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cf900 msgr2=0x7f4e58137450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e5de73640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cf900 0x7f4e58137450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009870 tx=0x7f4e540191e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e540b0040 con 0x7f4e581cf900 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e4e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.542+0000 7f4e4e7fc640 10 monclient: dump: 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.560 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9401 (secure 0 0 0) 0x7f4e540beb40 con 0x7f4e581cf900 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e4e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e4e7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cf900 msgr2=0x7f4e58137450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e4e7fc640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cf900 0x7f4e58137450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009870 tx=0x7f4e540191e0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e4e7fc640 1 -- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cf900 msgr2=0x7f4e58137450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:06.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e4e7fc640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e581cf900 0x7f4e58137450 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009870 tx=0x7f4e540191e0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.562 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:43:06.562 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:43:06.563 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:43:06.563 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:43:06.563 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 60, 2024-08-19T10:43:06.563 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 316314, 2024-08-19T10:43:06.563 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:43:06.563 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:43:06.563 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:43:06.563 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:43:06.563 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:43:06.563 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.564 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.565 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:43:06.566 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.567 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.569 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.570 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.571 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.574 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.576 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.577 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.583 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.586 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:43:06.587 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:43:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e600d7640 1 -- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e3c03be00 msgr2=0x7f4e3c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e600d7640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e3c03be00 0x7f4e3c03e2c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000f3f0 tx=0x7f4e500076e0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e600d7640 10 monclient: shutdown 2024-08-19T10:43:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e600d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e600d7640 1 -- 172.21.15.89:0/2342275816 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a40e0 msgr2=0x7f4e5813ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e600d7640 1 --2- 172.21.15.89:0/2342275816 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e580a40e0 0x7f4e5813ab10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800f9e0 tx=0x7f4e4800fea0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e5e674640 1 -- 172.21.15.89:0/2342275816 reap_dead start 2024-08-19T10:43:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e5e674640 1 -- 172.21.15.89:0/2342275816 reap_dead start 2024-08-19T10:43:06.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e600d7640 1 -- 172.21.15.89:0/2342275816 shutdown_connections 2024-08-19T10:43:06.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e600d7640 1 -- 172.21.15.89:0/2342275816 >> 172.21.15.89:0/2342275816 conn(0x7f4e580a4df0 msgr2=0x7f4e580a21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e600d7640 1 -- 172.21.15.89:0/2342275816 shutdown_connections 2024-08-19T10:43:06.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.546+0000 7f4e600d7640 1 -- 172.21.15.89:0/2342275816 wait complete. 2024-08-19T10:43:06.591 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:43:06.591 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-08-19T10:43:06.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:43:06.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.559+0000 7f4c821bf640 10 monclient: get_monmap_and_config 2024-08-19T10:43:06.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.559+0000 7f4c821bf640 10 monclient: build_initial_monmap 2024-08-19T10:43:06.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.559+0000 7f4c821bf640 10 monclient: monmap: 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:06.566528+0000 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:06.566528+0000 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:06.598 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.559+0000 7f4c821bf640 1 Processor -- start 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.559+0000 7f4c821bf640 1 -- start start 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.559+0000 7f4c821bf640 10 monclient: init 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 10 monclient: _add_conns ranks=[10,5,4,2,8,6,3,0,7,1,9] 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c7c0a69c0 0x7f4c7c0a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 10 monclient(hunting): picked mon.noname-f con 0x7f4c7c0a69c0 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:06.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c7c0a6da0 0x7f4c7c0a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 10 monclient(hunting): picked mon.noname-h con 0x7f4c7c0a6da0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c7c139780 0x7f4c7c139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 10 monclient(hunting): picked mon.noname-c con 0x7f4c7c139780 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f4c7c0a5c30 con 0x7f4c7c0a69c0 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f4c7c0a3570 con 0x7f4c7c139780 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4c7c0a4eb0 con 0x7f4c7c0a6da0 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c821bf640 10 monclient(hunting): authenticate will time out at 1232.136528s 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c7b7fe640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f4c7c0a69c0 msgr2=0x7f4c7c0a56f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:06.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c7b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c7c0a69c0 0x7f4c7c0a56f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c7bfff640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c7c139780 msgr2=0x7f4c7c139bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c7c139780 0x7f4c7c139bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c7c0a6da0 0x7f4c7c0a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c7c0a6da0 0x7f4c7c0a7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:57126/0 (socket says 172.21.15.190:57126) 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 1 -- 172.21.15.190:0/2149378275 learned_addr learned my addr 172.21.15.190:0/2149378275 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 10 monclient(hunting): get_auth_request con 0x7f4c7c0a6da0 auth_method 0 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 10 monclient(hunting): handle_auth_done global_id 5801 payload 274 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 1 -- 172.21.15.190:0/2149378275 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c7c139780 msgr2=0x7f4c7c139bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 1 --2- 172.21.15.190:0/2149378275 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c7c139780 0x7f4c7c139bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 1 -- 172.21.15.190:0/2149378275 >> v2:172.21.15.190:3300/0 conn(0x7f4c7c0a69c0 msgr2=0x7f4c7c0a56f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 1 --2- 172.21.15.190:0/2149378275 >> v2:172.21.15.190:3300/0 conn(0x7f4c7c0a69c0 0x7f4c7c0a56f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 1 monclient: found mon.noname-h 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.563+0000 7f4c72ffd640 1 -- 172.21.15.190:0/2149378275 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c0a6330 con 0x7f4c7c0a6da0 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c72ffd640 1 -- 172.21.15.190:0/2149378275 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c7c0a6da0 msgr2=0x7f4c7c0a7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c72ffd640 1 --2- 172.21.15.190:0/2149378275 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c7c0a6da0 0x7f4c7c0a7180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c640098a0 tx=0x7f4c6401c440 comp rx=0 tx=0).ready entity=mon.5 client_cookie=b9eb29d0946293aa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 1 -- 172.21.15.190:0/2149378275 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4c640b0070 con 0x7f4c7c0a6da0 2024-08-19T10:43:06.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 10 monclient: dump: 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.603 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 10 monclient: _finish_auth 0 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 1 -- 172.21.15.190:0/2149378275 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c640aa810 con 0x7f4c7c0a6da0 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 1 -- 172.21.15.190:0/2149378275 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4c640a93e0 con 0x7f4c7c0a6da0 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 5 monclient: authenticate success, global_id 5801 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:06.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 4 set_mon_vals no callback set 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c7affd640 10 monclient: dump: 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.605 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 10 monclient: shutdown 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 1 -- 172.21.15.190:0/2149378275 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c7c0a6da0 msgr2=0x7f4c7c0a7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 1 --2- 172.21.15.190:0/2149378275 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c7c0a6da0 0x7f4c7c0a7180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c640098a0 tx=0x7f4c6401c440 comp rx=0 tx=0).stop 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 1 -- 172.21.15.190:0/2149378275 shutdown_connections 2024-08-19T10:43:06.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 1 --2- 172.21.15.190:0/2149378275 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4c7c139780 0x7f4c7c139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 1 --2- 172.21.15.190:0/2149378275 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c7c0a6da0 0x7f4c7c0a7180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 1 --2- 172.21.15.190:0/2149378275 >> v2:172.21.15.190:3300/0 conn(0x7f4c7c0a69c0 0x7f4c7c0a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 1 -- 172.21.15.190:0/2149378275 >> 172.21.15.190:0/2149378275 conn(0x7f4c7c0abfc0 msgr2=0x7f4c7c0ac3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 1 -- 172.21.15.190:0/2149378275 shutdown_connections 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 1 -- 172.21.15.190:0/2149378275 wait complete. 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 10 monclient: build_initial_monmap 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 10 monclient: monmap: 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:06.607 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 1 Processor -- start 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 1 -- start start 2024-08-19T10:43:06.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 10 monclient: init 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.567+0000 7f4c821bf640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient: _add_conns ranks=[3,6,1,7,8,4,5,2,0] 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c0a69c0 0x7f4c7c1d1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient(hunting): picked mon.noname-g con 0x7f4c7c0a69c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c7c0a6da0 0x7f4c7c0a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient(hunting): picked mon.noname-d con 0x7f4c7c0a6da0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7c139780 0x7f4c7c0a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient(hunting): picked mon.noname-f con 0x7f4c7c139780 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c0a76c0 con 0x7f4c7c139780 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4c7c13a540 con 0x7f4c7c0a69c0 2024-08-19T10:43:06.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4c7c0a59a0 con 0x7f4c7c0a6da0 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient(hunting): authenticate will time out at 1232.144085s 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7bfff640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7c139780 msgr2=0x7f4c7c0a2500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7c139780 0x7f4c7c0a2500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c0a69c0 0x7f4c7c1d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c0a69c0 0x7f4c7c1d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:58046/0 (socket says 172.21.15.190:58046) 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 1 -- 172.21.15.190:0/3578351794 learned_addr learned my addr 172.21.15.190:0/3578351794 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 10 monclient(hunting): get_auth_request con 0x7f4c7c0a69c0 auth_method 0 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c72ffd640 1 -- 172.21.15.190:0/3578351794 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c7c0a6da0 msgr2=0x7f4c7c0a1fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c72ffd640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c7c0a6da0 0x7f4c7c0a1fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 10 monclient(hunting): handle_auth_done global_id 34230 payload 824 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 1 -- 172.21.15.190:0/3578351794 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c7c0a6da0 msgr2=0x7f4c7c0a1fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c7c0a6da0 0x7f4c7c0a1fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 1 -- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7c139780 msgr2=0x7f4c7c0a2500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7c139780 0x7f4c7c0a2500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 1 monclient: found mon.noname-g 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:06.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 1 -- 172.21.15.190:0/3578351794 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c13ecd0 con 0x7f4c7c0a69c0 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 1 -- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c0a69c0 msgr2=0x7f4c7c1d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7b7fe640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c0a69c0 0x7f4c7c1d1380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00b550 tx=0x7f4c6c00ba10 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 1 -- 172.21.15.190:0/3578351794 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4c6c003de0 con 0x7f4c7c0a69c0 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 10 monclient: dump: 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.612 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 10 monclient: _finish_auth 0 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:43:06.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:06.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 1 -- 172.21.15.190:0/3578351794 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c6c015070 con 0x7f4c7c0a69c0 2024-08-19T10:43:06.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:06.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 1 -- 172.21.15.190:0/3578351794 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4c6c010480 con 0x7f4c7c0a69c0 2024-08-19T10:43:06.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 5 monclient: authenticate success, global_id 34230 2024-08-19T10:43:06.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient: _renew_subs 2024-08-19T10:43:06.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:06.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c7a7fc640 4 set_mon_vals no callback set 2024-08-19T10:43:06.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 1 -- 172.21.15.190:0/3578351794 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7c09da60 con 0x7f4c7c0a69c0 2024-08-19T10:43:06.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:06.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c78ff9640 10 monclient: dump: 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.615 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient: _renew_subs 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.571+0000 7f4c821bf640 1 -- 172.21.15.190:0/3578351794 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7c1d18c0 con 0x7f4c7c0a69c0 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.575+0000 7f4c821bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.575+0000 7f4c821bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.575+0000 7f4c821bf640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.575+0000 7f4c821bf640 1 -- 172.21.15.190:0/3578351794 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c7c0abdd0 con 0x7f4c7c0a69c0 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.575+0000 7f4c7b7fe640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c0a69c0 0x7f4c7c1d1380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00b550 tx=0x7f4c6c00ba10 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f4c6c0050f0 mgrmap(e 5) 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.575+0000 7f4c78ff9640 1 -- 172.21.15.190:0/3578351794 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4c6c0050f0 con 0x7f4c7c0a69c0 2024-08-19T10:43:06.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.575+0000 7f4c78ff9640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c5c03be00 0x7f4c5c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.575+0000 7f4c72ffd640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c5c03be00 0x7f4c5c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.575+0000 7f4c72ffd640 10 monclient: get_auth_request con 0x7f4c5c03be00 auth_method 0 2024-08-19T10:43:06.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.575+0000 7f4c72ffd640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c5c03be00 0x7f4c5c03e2c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c06cd20 tx=0x7f4c64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:06.579+0000 7f4c78ff9640 1 -- 172.21.15.190:0/3578351794 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(157..157 src has 1..157) ==== 9265+0+0 (secure 0 0 0) 0x7f4c6c04fb30 con 0x7f4c7c0a69c0 2024-08-19T10:43:06.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient: get_monmap_and_config 2024-08-19T10:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient: build_initial_monmap 2024-08-19T10:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient: monmap: 2024-08-19T10:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:43:06.709178+0000 2024-08-19T10:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:43:06.709178+0000 2024-08-19T10:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.709 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:06.710 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 1 Processor -- start 2024-08-19T10:43:06.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 1 -- start start 2024-08-19T10:43:06.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient: init 2024-08-19T10:43:06.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:06.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient: _add_conns ranks=[8,2,0,9,4,10,5,1,3,6,7] 2024-08-19T10:43:06.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d1c0a6860 0x7f2d1c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient(hunting): picked mon.noname-e con 0x7f2d1c0a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:06.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d1c0a7180 0x7f2d1c0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient(hunting): picked mon.noname-b con 0x7f2d1c0a7180 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:06.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2d1c139620 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:06.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1c0a33d0 con 0x7f2d1c139620 2024-08-19T10:43:06.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1c0a8190 con 0x7f2d1c0a7180 2024-08-19T10:43:06.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2d1c0a2ec0 con 0x7f2d1c0a6860 2024-08-19T10:43:06.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:06.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d20d25640 10 monclient(hunting): authenticate will time out at 1236.525982s 2024-08-19T10:43:06.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d19d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d1c0a7180 0x7f2d1c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d1c0a6860 0x7f2d1c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d19d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d1c0a7180 0x7f2d1c0a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43876/0 (socket says 172.21.15.89:43876) 2024-08-19T10:43:06.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d19d9b640 1 -- 172.21.15.89:0/82425525 learned_addr learned my addr 172.21.15.89:0/82425525 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:06.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d19d9b640 10 monclient(hunting): get_auth_request con 0x7f2d1c0a7180 auth_method 0 2024-08-19T10:43:06.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d19d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d19d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d19d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 10 monclient(hunting): get_auth_request con 0x7f2d1c0a6860 auth_method 0 2024-08-19T10:43:06.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 10 monclient(hunting): handle_auth_done global_id 5705 payload 274 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 1 -- 172.21.15.89:0/82425525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d1c0a7180 msgr2=0x7f2d1c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 1 --2- 172.21.15.89:0/82425525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d1c0a7180 0x7f2d1c0a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 1 -- 172.21.15.89:0/82425525 >> v1:172.21.15.89:6789/0 conn(0x7f2d1c139620 legacy=0x7f2d1c139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:06.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:43:06.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 1 monclient: found mon.noname-e 2024-08-19T10:43:06.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:06.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:43:06.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.706+0000 7f2d1a59c640 1 -- 172.21.15.89:0/82425525 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c0a4cd0 con 0x7f2d1c0a6860 2024-08-19T10:43:06.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1a59c640 1 -- 172.21.15.89:0/82425525 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d1c0a6860 msgr2=0x7f2d1c0a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1a59c640 1 --2- 172.21.15.89:0/82425525 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d1c0a6860 0x7f2d1c0a6c40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0097f0 tx=0x7f2d0c01c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=44a47d8903cab19c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 1 -- 172.21.15.89:0/82425525 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d0c0b0070 con 0x7f2d1c0a6860 2024-08-19T10:43:06.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:43:06.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 10 monclient: dump: 2024-08-19T10:43:06.721 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.722 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.723 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 10 monclient: _finish_auth 0 2024-08-19T10:43:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 10 monclient: _check_auth_tickets 2024-08-19T10:43:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 1 -- 172.21.15.89:0/82425525 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c0aa810 con 0x7f2d1c0a6860 2024-08-19T10:43:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 1 -- 172.21.15.89:0/82425525 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d0c0a9300 con 0x7f2d1c0a6860 2024-08-19T10:43:06.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d1959a640 10 monclient: dump: 2024-08-19T10:43:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.724 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.725 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.725 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.725 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.725 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.725 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.725 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.725 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 5 monclient: authenticate success, global_id 5705 2024-08-19T10:43:06.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:06.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 4 set_mon_vals no callback set 2024-08-19T10:43:06.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 10 monclient: shutdown 2024-08-19T10:43:06.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:06.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 1 -- 172.21.15.89:0/82425525 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d1c0a6860 msgr2=0x7f2d1c0a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 1 --2- 172.21.15.89:0/82425525 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d1c0a6860 0x7f2d1c0a6c40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0097f0 tx=0x7f2d0c01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 1 -- 172.21.15.89:0/82425525 shutdown_connections 2024-08-19T10:43:06.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 1 --2- 172.21.15.89:0/82425525 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2d1c0a7180 0x7f2d1c0a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 1 --2- 172.21.15.89:0/82425525 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d1c0a6860 0x7f2d1c0a6c40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 1 -- 172.21.15.89:0/82425525 >> 172.21.15.89:0/82425525 conn(0x7f2d1c074b40 msgr2=0x7f2d1c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.710+0000 7f2d20d25640 1 -- 172.21.15.89:0/82425525 shutdown_connections 2024-08-19T10:43:06.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 1 -- 172.21.15.89:0/82425525 wait complete. 2024-08-19T10:43:06.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient: build_initial_monmap 2024-08-19T10:43:06.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient: monmap: 2024-08-19T10:43:06.727 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:43:06.727 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:06.727 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:43:06.727 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:43:06.727 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:06.727 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.727 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:06.727 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:06.727 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:06.727 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 1 Processor -- start 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 1 -- start start 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient: init 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient: _add_conns ranks=[3,7,6,1,0,8,2,4,5] 2024-08-19T10:43:06.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0a6860 0x7f2d1c138480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient(hunting): picked mon.noname-g con 0x7f2d1c0a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c0a7180 0x7f2d1c13d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient(hunting): picked mon.noname-i con 0x7f2d1c0a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2d1c139620 0x7f2d1c13db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient(hunting): picked mon.noname-d con 0x7f2d1c139620 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1c144430 con 0x7f2d1c0a6860 2024-08-19T10:43:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f2d1c1442b0 con 0x7f2d1c139620 2024-08-19T10:43:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2d1c1445b0 con 0x7f2d1c0a7180 2024-08-19T10:43:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d20d25640 10 monclient(hunting): authenticate will time out at 1236.533486s 2024-08-19T10:43:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1ad9d640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2d1c139620 msgr2=0x7f2d1c13db80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1ad9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2d1c139620 0x7f2d1c13db80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0a6860 0x7f2d1c138480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d19d9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c0a7180 0x7f2d1c13d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0a6860 0x7f2d1c138480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:42358/0 (socket says 172.21.15.89:42358) 2024-08-19T10:43:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d19d9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c0a7180 0x7f2d1c13d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:45996/0 (socket says 172.21.15.89:45996) 2024-08-19T10:43:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 1 -- 172.21.15.89:0/3905556666 learned_addr learned my addr 172.21.15.89:0/3905556666 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:43:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 10 monclient(hunting): get_auth_request con 0x7f2d1c0a6860 auth_method 0 2024-08-19T10:43:06.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d19d9b640 10 monclient(hunting): get_auth_request con 0x7f2d1c0a7180 auth_method 0 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d19d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d19d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d19d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 10 monclient(hunting): handle_auth_done global_id 34239 payload 824 2024-08-19T10:43:06.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 1 -- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c0a7180 msgr2=0x7f2d1c13d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c0a7180 0x7f2d1c13d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 1 -- 172.21.15.89:0/3905556666 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2d1c139620 msgr2=0x7f2d1c13db80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:06.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2d1c139620 0x7f2d1c13db80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:43:06.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 1 monclient: found mon.noname-g 2024-08-19T10:43:06.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:06.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:43:06.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.714+0000 7f2d1a59c640 1 -- 172.21.15.89:0/3905556666 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c13fde0 con 0x7f2d1c0a6860 2024-08-19T10:43:06.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d1a59c640 1 -- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0a6860 msgr2=0x7f2d1c138480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d1a59c640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0a6860 0x7f2d1c138480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c01c1d0 tx=0x7f2d0c019140 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d0c0b0040 con 0x7f2d1c0a6860 2024-08-19T10:43:06.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:43:06.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 10 monclient: dump: 2024-08-19T10:43:06.738 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.738 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.738 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.738 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.738 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.738 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.738 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.738 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.738 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.739 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.739 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.739 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.739 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.739 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.739 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.739 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 10 monclient: _finish_auth 0 2024-08-19T10:43:06.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:06.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c0b9070 con 0x7f2d1c0a6860 2024-08-19T10:43:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d0c0044b0 con 0x7f2d1c0a6860 2024-08-19T10:43:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d20d25640 5 monclient: authenticate success, global_id 34239 2024-08-19T10:43:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d20d25640 10 monclient: _renew_subs 2024-08-19T10:43:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d20d25640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d20d25640 1 -- 172.21.15.89:0/3905556666 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c0ab2b0 con 0x7f2d1c0a6860 2024-08-19T10:43:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d18d99640 4 set_mon_vals no callback set 2024-08-19T10:43:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 10 monclient: dump: 2024-08-19T10:43:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.742 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d20d25640 10 monclient: _renew_subs 2024-08-19T10:43:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d20d25640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d20d25640 1 -- 172.21.15.89:0/3905556666 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c122c50 con 0x7f2d1c0a6860 2024-08-19T10:43:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d20d25640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d20d25640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c1389c0 0x7f2d1c13e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d20d25640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d20d25640 1 -- 172.21.15.89:0/3905556666 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2d1c076ca0 con 0x7f2d1c1389c0 2024-08-19T10:43:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d19d9b640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c1389c0 0x7f2d1c13e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d19d9b640 10 monclient: get_auth_request con 0x7f2d1c1389c0 auth_method 0 2024-08-19T10:43:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d19d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d19d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d19d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2d0c004040 con 0x7f2d1c0a6860 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_done global_id 14992 payload 824 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2cfc03be00 0x7f2cfc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d19d9b640 1 -- 172.21.15.89:0/3905556666 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d1c0a6c40 con 0x7f2d1c1389c0 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(157..157 src has 1..157) ==== 9265+0+0 (secure 0 0 0) 0x7f2d0c0e9090 con 0x7f2d1c0a6860 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d1ad9d640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2cfc03be00 0x7f2cfc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.718+0000 7f2d1ad9d640 10 monclient: get_auth_request con 0x7f2cfc03be00 auth_method 0 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d19d9b640 1 -- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c1389c0 msgr2=0x7f2d1c13e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d19d9b640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c1389c0 0x7f2d1c13e0c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c1395f0 tx=0x7f2d10011d60 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d1000fe50 con 0x7f2d1c1389c0 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d0affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d0affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:06.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d0affd640 10 monclient: dump: 2024-08-19T10:43:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.746 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d1ad9d640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2cfc03be00 0x7f2cfc03e2c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d14007fd0 tx=0x7f2d14007d60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2d1000fe50 con 0x7f2d1c1389c0 2024-08-19T10:43:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d0affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d0affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c1389c0 msgr2=0x7f2d1c13e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d0affd640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c1389c0 0x7f2d1c13e0c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c1395f0 tx=0x7f2d10011d60 comp rx=0 tx=0).stop 2024-08-19T10:43:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c1389c0 msgr2=0x7f2d1c13e0c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.722+0000 7f2d0affd640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c1389c0 0x7f2d1c13e0c0 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c1395f0 tx=0x7f2d10011d60 comp rx=0 tx=0).stop 2024-08-19T10:43:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d20d25640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:43:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d20d25640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c13e600 0x7f2d1c0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d20d25640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d20d25640 1 -- 172.21.15.89:0/3905556666 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2d1c140910 con 0x7f2d1c13e600 2024-08-19T10:43:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d19d9b640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c13e600 0x7f2d1c0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d19d9b640 10 monclient: get_auth_request con 0x7f2d1c13e600 auth_method 0 2024-08-19T10:43:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d19d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d19d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d19d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d19d9b640 10 monclient(hunting): handle_auth_done global_id 15001 payload 824 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.730+0000 7f2d19d9b640 1 -- 172.21.15.89:0/3905556666 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2d1c1228d0 con 0x7f2d1c13e600 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d19d9b640 1 -- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c13e600 msgr2=0x7f2d1c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d19d9b640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c13e600 0x7f2d1c0a01f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c1395f0 tx=0x7f2d100046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d100148b0 con 0x7f2d1c13e600 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d0affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d0affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d0affd640 10 monclient: dump: 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:06.750 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:06.750 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:06.750 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:06.750 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:06.750 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:43:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9234 (secure 0 0 0) 0x7f2d100133a0 con 0x7f2d1c13e600 2024-08-19T10:43:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d0affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:43:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d0affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c13e600 msgr2=0x7f2d1c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d0affd640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c13e600 0x7f2d1c0a01f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c1395f0 tx=0x7f2d100046d0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d0affd640 1 -- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c13e600 msgr2=0x7f2d1c0a01f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d0affd640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d1c13e600 0x7f2d1c0a01f0 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c1395f0 tx=0x7f2d100046d0 comp rx=0 tx=0).stop 2024-08-19T10:43:06.751 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:43:06.752 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:43:06.752 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:43:06.752 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:43:06.752 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 60, 2024-08-19T10:43:06.752 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 164319, 2024-08-19T10:43:06.752 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:43:06.752 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:43:06.752 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:43:06.752 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:43:06.753 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:43:06.753 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:43:06.753 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:43:06.753 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.753 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:43:06.753 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:06.753 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:43:06.753 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:43:06.753 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.753 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.754 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.754 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.754 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.754 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.754 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.754 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.754 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.754 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.754 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.754 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.755 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:43:06.755 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:43:06.755 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.755 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.755 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.755 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.755 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.755 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.755 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.755 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.755 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.756 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.756 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.756 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.756 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:43:06.756 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:43:06.756 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:43:06.756 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:43:06.756 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:43:06.756 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:43:06.756 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:43:06.757 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:43:06.758 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:43:06.758 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:43:06.758 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:43:06.758 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:43:06.758 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:43:06.758 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:43:06.758 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:43:06.758 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:43:06.758 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:43:06.758 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:43:06.759 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.760 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.760 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:43:06.760 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.760 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.760 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.760 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:43:06.760 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:43:06.761 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.761 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.761 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.761 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:43:06.761 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:43:06.761 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.761 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.761 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.762 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:43:06.762 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.762 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.762 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:43:06.762 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.762 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.762 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.763 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:43:06.763 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:43:06.763 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.763 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.763 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.763 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:43:06.763 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:43:06.763 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.763 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.764 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.764 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:43:06.764 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.764 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.764 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:43:06.764 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.764 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.765 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.765 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:43:06.765 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:43:06.765 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.765 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.765 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.765 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:43:06.765 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:43:06.766 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.766 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.766 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.766 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:43:06.766 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.766 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.766 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:43:06.766 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.767 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.767 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.767 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:43:06.767 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:43:06.767 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.767 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.767 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.767 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:43:06.768 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:43:06.768 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.768 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:43:06.768 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.768 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.769 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:43:06.770 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:43:06.770 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:43:06.770 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.772 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:43:06.772 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:43:06.772 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.772 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.772 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.772 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:43:06.772 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.772 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.773 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:43:06.774 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:43:06.774 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.774 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.774 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.774 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:43:06.774 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.774 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.776 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:43:06.776 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:43:06.776 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:43:06.776 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:43:06.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.776 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:43:06.776 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:43:06.776 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.776 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:43:06.777 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:43:06.777 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:43:06.777 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.777 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.777 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:43:06.777 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:43:06.777 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:43:06.777 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:43:06.777 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:43:06.777 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.778 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.778 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:43:06.778 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:43:06.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.778 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.778 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.778 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:43:06.778 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.778 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:43:06.778 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:43:06.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:43:06.779 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:43:06.779 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:43:06.779 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:43:06.779 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:43:06.779 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:43:06.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:43:06.779 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:43:06.779 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:43:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d20d25640 1 -- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2cfc03be00 msgr2=0x7f2cfc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d20d25640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2cfc03be00 0x7f2cfc03e2c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d14007fd0 tx=0x7f2d14007d60 comp rx=0 tx=0).stop 2024-08-19T10:43:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d20d25640 10 monclient: shutdown 2024-08-19T10:43:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d20d25640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d20d25640 1 -- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0a6860 msgr2=0x7f2d1c138480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d20d25640 1 --2- 172.21.15.89:0/3905556666 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d1c0a6860 0x7f2d1c138480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c01c1d0 tx=0x7f2d0c019140 comp rx=0 tx=0).stop 2024-08-19T10:43:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d1ad9d640 1 -- 172.21.15.89:0/3905556666 reap_dead start 2024-08-19T10:43:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d1ad9d640 1 -- 172.21.15.89:0/3905556666 reap_dead start 2024-08-19T10:43:06.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d20d25640 1 -- 172.21.15.89:0/3905556666 shutdown_connections 2024-08-19T10:43:06.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d20d25640 1 -- 172.21.15.89:0/3905556666 >> 172.21.15.89:0/3905556666 conn(0x7f2d1c074b40 msgr2=0x7f2d1c0a9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:06.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d20d25640 1 -- 172.21.15.89:0/3905556666 shutdown_connections 2024-08-19T10:43:06.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:43:06.734+0000 7f2d20d25640 1 -- 172.21.15.89:0/3905556666 wait complete. 2024-08-19T10:43:06.782 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:43:06.782 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-08-19T10:43:07.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.199+0000 7f4c78ff9640 1 -- 172.21.15.190:0/3578351794 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4c6c04d390 con 0x7f4c7c0a69c0 2024-08-19T10:43:07.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.199+0000 7f4c78ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:07.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.199+0000 7f4c78ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:07.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.383+0000 7f4c821bf640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-08-19T10:43:07.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.383+0000 7f4c821bf640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-08-19T10:43:07.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.383+0000 7f4c821bf640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:07.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.383+0000 7f4c821bf640 1 -- 172.21.15.190:0/3578351794 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]} v 0) -- 0x7f4c7c0a7180 con 0x7f4c7c0a69c0 2024-08-19T10:43:07.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.515+0000 7f4c78ff9640 1 -- 172.21.15.190:0/3578351794 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}]=0 v158) ==== 128+0+0 (secure 0 0 0) 0x7f4c6c010620 con 0x7f4c7c0a69c0 2024-08-19T10:43:07.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.515+0000 7f4c78ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-08-19T10:43:07.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.515+0000 7f4c78ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:07.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.627+0000 7f4c821bf640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-08-19T10:43:07.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.627+0000 7f4c821bf640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-08-19T10:43:07.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.627+0000 7f4c821bf640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:07.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:07.627+0000 7f4c821bf640 1 -- 172.21.15.190:0/3578351794 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]} v 0) -- 0x7f4c7c0008e0 con 0x7f4c7c0a69c0 2024-08-19T10:43:08.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:08.519+0000 7f4c7b7fe640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c0a69c0 0x7f4c7c1d1380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00b550 tx=0x7f4c6c00ba10 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f4c6c03f030 mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}]=0 v159) 2024-08-19T10:43:09.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c78ff9640 1 -- 172.21.15.190:0/3578351794 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}]=0 v159) ==== 128+0+0 (secure 0 0 0) 0x7f4c6c03f030 con 0x7f4c7c0a69c0 2024-08-19T10:43:09.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c78ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [11.5]}] 2024-08-19T10:43:09.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c78ff9640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:09.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 -- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c5c03be00 msgr2=0x7f4c5c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:09.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c5c03be00 0x7f4c5c03e2c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c06cd20 tx=0x7f4c64008040 comp rx=0 tx=0).stop 2024-08-19T10:43:09.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 10 monclient: shutdown 2024-08-19T10:43:09.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:09.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 -- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c0a69c0 msgr2=0x7f4c7c1d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:09.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c0a69c0 0x7f4c7c1d1380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00b550 tx=0x7f4c6c00ba10 comp rx=0 tx=0).stop 2024-08-19T10:43:09.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 -- 172.21.15.190:0/3578351794 shutdown_connections 2024-08-19T10:43:09.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c5c03be00 0x7f4c5c03e2c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:09.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7c139780 0x7f4c7c0a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:09.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c7c0a6da0 0x7f4c7c0a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:09.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 --2- 172.21.15.190:0/3578351794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c0a69c0 0x7f4c7c1d1380 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:09.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 -- 172.21.15.190:0/3578351794 >> 172.21.15.190:0/3578351794 conn(0x7f4c7c0abfc0 msgr2=0x7f4c7c140260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:09.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 -- 172.21.15.190:0/3578351794 shutdown_connections 2024-08-19T10:43:09.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.351+0000 7f4c821bf640 1 -- 172.21.15.190:0/3578351794 wait complete. 2024-08-19T10:43:09.376 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight cold osd.0 12.5 12.4 2024-08-19T10:43:09.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.499+0000 7fd303ac0640 10 monclient: get_monmap_and_config 2024-08-19T10:43:09.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.499+0000 7fd303ac0640 10 monclient: build_initial_monmap 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.499+0000 7fd303ac0640 10 monclient: monmap: 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:09.506695+0000 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:09.506695+0000 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:09.506 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:09.507 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:09.507 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:09.507 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:09.507 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:09.507 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:09.507 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:09.507 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:09.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.499+0000 7fd303ac0640 1 Processor -- start 2024-08-19T10:43:09.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 1 -- start start 2024-08-19T10:43:09.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 10 monclient: init 2024-08-19T10:43:09.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:09.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 10 monclient: _add_conns ranks=[4,2,5,3,10,7,0,8,1,6,9] 2024-08-19T10:43:09.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc0a57d0 0x7fd2fc0a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:09.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 10 monclient(hunting): picked mon.noname-c con 0x7fd2fc0a57d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:09.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc06e610 0x7fd2fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:09.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 10 monclient(hunting): picked mon.noname-b con 0x7fd2fc06e610 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:09.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2fc06dae0 0x7fd2fc1395a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:09.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 10 monclient(hunting): picked mon.noname-h con 0x7fd2fc06dae0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:09.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:09.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd2fc06f8c0 con 0x7fd2fc06e610 2024-08-19T10:43:09.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:09.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd2fc06f020 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd2fc06f300 con 0x7fd2fc06dae0 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd303ac0640 10 monclient(hunting): authenticate will time out at 1235.076710s 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30185c640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc0a57d0 msgr2=0x7fd2fc0a5bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30185c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc0a57d0 0x7fd2fc0a5bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2fc06dae0 0x7fd2fc1395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2fc06dae0 0x7fd2fc1395a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:57140/0 (socket says 172.21.15.190:57140) 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 1 -- 172.21.15.190:0/4198574812 learned_addr learned my addr 172.21.15.190:0/4198574812 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 10 monclient(hunting): get_auth_request con 0x7fd2fc06dae0 auth_method 0 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:09.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30105b640 1 --2- 172.21.15.190:0/4198574812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc06e610 0x7fd2fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:09.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30105b640 10 monclient(hunting): get_auth_request con 0x7fd2fc06e610 auth_method 0 2024-08-19T10:43:09.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30105b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:09.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30105b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:09.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30105b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:09.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:09.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:09.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:09.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30105b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:09.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30105b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:09.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30105b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:09.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 10 monclient(hunting): handle_auth_done global_id 5810 payload 274 2024-08-19T10:43:09.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 1 -- 172.21.15.190:0/4198574812 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc0a57d0 msgr2=0x7fd2fc0a5bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:09.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 1 --2- 172.21.15.190:0/4198574812 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc0a57d0 0x7fd2fc0a5bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:09.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 1 -- 172.21.15.190:0/4198574812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc06e610 msgr2=0x7fd2fc0a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:09.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 1 --2- 172.21.15.190:0/4198574812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc06e610 0x7fd2fc0a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:09.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 10 monclient: _finish_hunting 0 2024-08-19T10:43:09.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 1 monclient: found mon.noname-h 2024-08-19T10:43:09.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:09.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:09.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.503+0000 7fd30205d640 1 -- 172.21.15.190:0/4198574812 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc0a7320 con 0x7fd2fc06dae0 2024-08-19T10:43:09.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30205d640 1 -- 172.21.15.190:0/4198574812 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2fc06dae0 msgr2=0x7fd2fc1395a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:09.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30205d640 1 --2- 172.21.15.190:0/4198574812 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2fc06dae0 0x7fd2fc1395a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009e80 tx=0x7fd2f401c670 comp rx=0 tx=0).ready entity=mon.5 client_cookie=244b049c610bd19e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:09.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 1 -- 172.21.15.190:0/4198574812 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2f40090d0 con 0x7fd2fc06dae0 2024-08-19T10:43:09.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:09.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:09.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 10 monclient: dump: 2024-08-19T10:43:09.520 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:09.521 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 10 monclient: _finish_auth 0 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 10 monclient: _check_auth_tickets 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 1 -- 172.21.15.190:0/4198574812 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2f40ae070 con 0x7fd2fc06dae0 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 1 -- 172.21.15.190:0/4198574812 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2f40a9d50 con 0x7fd2fc06dae0 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 5 monclient: authenticate success, global_id 5810 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 4 set_mon_vals no callback set 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:09.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd30085a640 10 monclient: dump: 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:09.523 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 10 monclient: shutdown 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 1 -- 172.21.15.190:0/4198574812 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2fc06dae0 msgr2=0x7fd2fc1395a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 1 --2- 172.21.15.190:0/4198574812 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2fc06dae0 0x7fd2fc1395a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009e80 tx=0x7fd2f401c670 comp rx=0 tx=0).stop 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 1 -- 172.21.15.190:0/4198574812 shutdown_connections 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 1 --2- 172.21.15.190:0/4198574812 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2fc06dae0 0x7fd2fc1395a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 1 --2- 172.21.15.190:0/4198574812 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc06e610 0x7fd2fc0a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 1 --2- 172.21.15.190:0/4198574812 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc0a57d0 0x7fd2fc0a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 1 -- 172.21.15.190:0/4198574812 >> 172.21.15.190:0/4198574812 conn(0x7fd2fc0aa3c0 msgr2=0x7fd2fc0a68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 1 -- 172.21.15.190:0/4198574812 shutdown_connections 2024-08-19T10:43:09.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 1 -- 172.21.15.190:0/4198574812 wait complete. 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.507+0000 7fd303ac0640 10 monclient: build_initial_monmap 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient: monmap: 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:09.525 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 1 Processor -- start 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 1 -- start start 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient: init 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient: _add_conns ranks=[0,4,2,5,8,6,1,7,3] 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2fc06dae0 0x7fd2fc1380a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient(hunting): picked mon.noname-a con 0x7fd2fc06dae0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc06e610 0x7fd2fc1385e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:09.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient(hunting): picked mon.noname-c con 0x7fd2fc06e610 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc0a57d0 0x7fd2fc120410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient(hunting): picked mon.noname-b con 0x7fd2fc0a57d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1433a0 con 0x7fd2fc06dae0 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd2fc143220 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd2fc143520 con 0x7fd2fc06e610 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd303ac0640 10 monclient(hunting): authenticate will time out at 1235.084874s 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30105b640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc06e610 msgr2=0x7fd2fc1385e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30105b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc06e610 0x7fd2fc1385e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc0a57d0 0x7fd2fc120410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:09.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30185c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2fc06dae0 0x7fd2fc1380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc0a57d0 0x7fd2fc120410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:54176/0 (socket says 172.21.15.190:54176) 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 1 -- 172.21.15.190:0/286220825 learned_addr learned my addr 172.21.15.190:0/286220825 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 10 monclient(hunting): get_auth_request con 0x7fd2fc0a57d0 auth_method 0 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30185c640 10 monclient(hunting): get_auth_request con 0x7fd2fc06dae0 auth_method 0 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30185c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30185c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30185c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:09.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:09.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30185c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:09.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30185c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:09.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30185c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:09.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 10 monclient(hunting): handle_auth_done global_id 5816 payload 824 2024-08-19T10:43:09.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 1 -- 172.21.15.190:0/286220825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc06e610 msgr2=0x7fd2fc1385e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:09.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc06e610 0x7fd2fc1385e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:09.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 1 -- 172.21.15.190:0/286220825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2fc06dae0 msgr2=0x7fd2fc1380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:09.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2fc06dae0 0x7fd2fc1380a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:09.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 10 monclient: _finish_hunting 0 2024-08-19T10:43:09.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 1 monclient: found mon.noname-b 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.511+0000 7fd30205d640 1 -- 172.21.15.190:0/286220825 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc123ad0 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd30205d640 1 -- 172.21.15.190:0/286220825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc0a57d0 msgr2=0x7fd2fc120410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd30205d640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc0a57d0 0x7fd2fc120410 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009e20 tx=0x7fd2f4019410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 1 -- 172.21.15.190:0/286220825 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2f40046c0 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 10 monclient: dump: 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:09.532 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 10 monclient: _finish_auth 0 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:43:09.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 1 -- 172.21.15.190:0/286220825 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2f4004860 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 1 -- 172.21.15.190:0/286220825 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2f40a98e0 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd303ac0640 5 monclient: authenticate success, global_id 5816 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd303ac0640 10 monclient: _renew_subs 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd303ac0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd303ac0640 1 -- 172.21.15.190:0/286220825 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc121ef0 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2effff640 4 set_mon_vals no callback set 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 10 monclient: dump: 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:09.534 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:09.535 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:09.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd303ac0640 10 monclient: _renew_subs 2024-08-19T10:43:09.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd303ac0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:09.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd303ac0640 1 -- 172.21.15.190:0/286220825 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc0a7ea0 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd303ac0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:09.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd303ac0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:09.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd303ac0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:09.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd303ac0640 1 -- 172.21.15.190:0/286220825 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2fc139580 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 1 -- 172.21.15.190:0/286220825 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd2f40a9470 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd2e003be00 0x7fd2e003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:09.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd2ee7fc640 1 -- 172.21.15.190:0/286220825 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(159..159 src has 1..159) ==== 9265+0+0 (secure 0 0 0) 0x7fd2f40ea860 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd30185c640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd2e003be00 0x7fd2e003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:09.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.515+0000 7fd30185c640 10 monclient: get_auth_request con 0x7fd2e003be00 auth_method 0 2024-08-19T10:43:09.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.519+0000 7fd30185c640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd2e003be00 0x7fd2e003e2c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00093d0 tx=0x7fd2f0009360 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:09.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.531+0000 7fd2ee7fc640 1 -- 172.21.15.190:0/286220825 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd2f4016640 con 0x7fd2fc0a57d0 2024-08-19T10:43:09.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.531+0000 7fd2ee7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:09.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.531+0000 7fd2ee7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:09.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.711+0000 7fd303ac0640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-08-19T10:43:09.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.711+0000 7fd303ac0640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-08-19T10:43:09.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.711+0000 7fd303ac0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:09.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:09.711+0000 7fd303ac0640 1 -- 172.21.15.190:0/286220825 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]} v 0) -- 0x7fd2fc06fd50 con 0x7fd2fc0a57d0 2024-08-19T10:43:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:10.551+0000 7fd2ee7fc640 1 -- 172.21.15.190:0/286220825 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}]=0 v160) ==== 134+0+0 (secure 0 0 0) 0x7fd2f40e7110 con 0x7fd2fc0a57d0 2024-08-19T10:43:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:10.551+0000 7fd2ee7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-08-19T10:43:10.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:10.551+0000 7fd2ee7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:10.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:10.663+0000 7fd303ac0640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-08-19T10:43:10.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:10.663+0000 7fd303ac0640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-08-19T10:43:10.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:10.663+0000 7fd303ac0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:10.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:10.663+0000 7fd303ac0640 1 -- 172.21.15.190:0/286220825 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]} v 0) -- 0x7fd2fc138b20 con 0x7fd2fc0a57d0 2024-08-19T10:43:11.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.607+0000 7fd2ee7fc640 1 -- 172.21.15.190:0/286220825 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}]=0 v161) ==== 134+0+0 (secure 0 0 0) 0x7fd2f40c8070 con 0x7fd2fc0a57d0 2024-08-19T10:43:11.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.611+0000 7fd2ee7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [12.5, 12.4]}] 2024-08-19T10:43:11.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.611+0000 7fd2ee7fc640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:11.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 -- 172.21.15.190:0/286220825 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd2e003be00 msgr2=0x7fd2e003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:11.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd2e003be00 0x7fd2e003e2c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00093d0 tx=0x7fd2f0009360 comp rx=0 tx=0).stop 2024-08-19T10:43:11.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 10 monclient: shutdown 2024-08-19T10:43:11.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:11.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 -- 172.21.15.190:0/286220825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc0a57d0 msgr2=0x7fd2fc120410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:11.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc0a57d0 0x7fd2fc120410 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009e20 tx=0x7fd2f4019410 comp rx=0 tx=0).stop 2024-08-19T10:43:11.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 -- 172.21.15.190:0/286220825 shutdown_connections 2024-08-19T10:43:11.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd2e003be00 0x7fd2e003e2c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:11.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2fc0a57d0 0x7fd2fc120410 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:11.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2fc06e610 0x7fd2fc1385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:11.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 --2- 172.21.15.190:0/286220825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2fc06dae0 0x7fd2fc1380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:11.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 -- 172.21.15.190:0/286220825 >> 172.21.15.190:0/286220825 conn(0x7fd2fc0aa3c0 msgr2=0x7fd2fc0a5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:11.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 -- 172.21.15.190:0/286220825 shutdown_connections 2024-08-19T10:43:11.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.615+0000 7fd303ac0640 1 -- 172.21.15.190:0/286220825 wait complete. 2024-08-19T10:43:11.646 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush set-device-class fish osd.0 2024-08-19T10:43:11.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.763+0000 7fce667a0640 10 monclient: get_monmap_and_config 2024-08-19T10:43:11.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.763+0000 7fce667a0640 10 monclient: build_initial_monmap 2024-08-19T10:43:11.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.763+0000 7fce667a0640 10 monclient: monmap: 2024-08-19T10:43:11.769 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:11.769 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:11.770 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:11.770081+0000 2024-08-19T10:43:11.770 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:11.770081+0000 2024-08-19T10:43:11.770 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:11.770 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:11.770 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:11.770 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:11.770 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:11.770 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:11.770 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:11.770 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:11.771 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:11.771 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:11.771 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:11.771 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:11.771 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:11.771 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:11.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.763+0000 7fce667a0640 1 Processor -- start 2024-08-19T10:43:11.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.763+0000 7fce667a0640 1 -- start start 2024-08-19T10:43:11.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.763+0000 7fce667a0640 10 monclient: init 2024-08-19T10:43:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 10 monclient: _add_conns ranks=[7,5,6,10,1,8,9,2,0,4,3] 2024-08-19T10:43:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fce600a3610 0x7fce600a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 10 monclient(hunting): picked mon.noname-i con 0x7fce600a3610 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fce600a3f30 0x7fce601373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 10 monclient(hunting): picked mon.noname-h con 0x7fce600a3f30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fce60137920 0x7fce60137d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 10 monclient(hunting): picked mon.noname-d con 0x7fce60137920 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fce60070830 con 0x7fce600a3f30 2024-08-19T10:43:11.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:11.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fce60074380 con 0x7fce60137920 2024-08-19T10:43:11.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:11.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fce60070170 con 0x7fce600a3610 2024-08-19T10:43:11.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:11.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce667a0640 10 monclient(hunting): authenticate will time out at 1237.340545s 2024-08-19T10:43:11.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fce600a3f30 0x7fce601373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:11.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce64d3d640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fce60137920 msgr2=0x7fce60137d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:11.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce64d3d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fce60137920 0x7fce60137d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:11.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fce600a3f30 0x7fce601373e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:57148/0 (socket says 172.21.15.190:57148) 2024-08-19T10:43:11.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5ffff640 1 --2- 172.21.15.190:0/2927385881 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fce600a3610 0x7fce600a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:11.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 1 -- 172.21.15.190:0/2927385881 learned_addr learned my addr 172.21.15.190:0/2927385881 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:11.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 10 monclient(hunting): get_auth_request con 0x7fce600a3f30 auth_method 0 2024-08-19T10:43:11.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:11.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:11.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:11.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5ffff640 10 monclient(hunting): get_auth_request con 0x7fce600a3610 auth_method 0 2024-08-19T10:43:11.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:11.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:11.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:11.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:11.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:11.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:11.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:11.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:11.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:11.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 10 monclient(hunting): handle_auth_done global_id 5819 payload 274 2024-08-19T10:43:11.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 1 -- 172.21.15.190:0/2927385881 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fce600a3610 msgr2=0x7fce600a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:11.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 1 --2- 172.21.15.190:0/2927385881 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fce600a3610 0x7fce600a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:11.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 1 -- 172.21.15.190:0/2927385881 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fce60137920 msgr2=0x7fce60137d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:11.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 1 --2- 172.21.15.190:0/2927385881 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fce60137920 0x7fce60137d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:11.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:11.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 1 monclient: found mon.noname-h 2024-08-19T10:43:11.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:11.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:11.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.767+0000 7fce5f7fe640 1 -- 172.21.15.190:0/2927385881 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce600740a0 con 0x7fce600a3f30 2024-08-19T10:43:11.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5f7fe640 1 -- 172.21.15.190:0/2927385881 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fce600a3f30 msgr2=0x7fce601373e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:11.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5f7fe640 1 --2- 172.21.15.190:0/2927385881 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fce600a3f30 0x7fce601373e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fce540098a0 tx=0x7fce5401c460 comp rx=0 tx=0).ready entity=mon.5 client_cookie=aa0c7c168b1e1d9f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:11.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 1 -- 172.21.15.190:0/2927385881 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fce540b0070 con 0x7fce600a3f30 2024-08-19T10:43:11.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:11.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:11.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 10 monclient: dump: 2024-08-19T10:43:11.784 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:11.784 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:11.784 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:11.785 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:11.785 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:11.785 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:11.785 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:11.785 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:11.785 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:11.785 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:11.785 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:11.785 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:11.785 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:11.785 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:11.786 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:11.786 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:11.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 10 monclient: _finish_auth 0 2024-08-19T10:43:11.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:11.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:11.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 1 -- 172.21.15.190:0/2927385881 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce540aa810 con 0x7fce600a3f30 2024-08-19T10:43:11.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 5 monclient: authenticate success, global_id 5819 2024-08-19T10:43:11.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:11.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:11.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 1 -- 172.21.15.190:0/2927385881 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fce540aae30 con 0x7fce600a3f30 2024-08-19T10:43:11.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:11.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 4 set_mon_vals no callback set 2024-08-19T10:43:11.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:11.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:11.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce5effd640 10 monclient: dump: 2024-08-19T10:43:11.787 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:11.787 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:11.787 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:11.787 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:11.787 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:11.787 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:11.787 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:11.788 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:11.788 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:11.788 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:11.788 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:11.788 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:11.788 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:11.788 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:11.788 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:11.788 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:11.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 10 monclient: shutdown 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 1 -- 172.21.15.190:0/2927385881 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fce600a3f30 msgr2=0x7fce601373e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 1 --2- 172.21.15.190:0/2927385881 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fce600a3f30 0x7fce601373e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fce540098a0 tx=0x7fce5401c460 comp rx=0 tx=0).stop 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 1 -- 172.21.15.190:0/2927385881 shutdown_connections 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 1 --2- 172.21.15.190:0/2927385881 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fce60137920 0x7fce60137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 1 --2- 172.21.15.190:0/2927385881 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fce600a3f30 0x7fce601373e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 1 --2- 172.21.15.190:0/2927385881 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fce600a3610 0x7fce600a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 1 -- 172.21.15.190:0/2927385881 >> 172.21.15.190:0/2927385881 conn(0x7fce600a4700 msgr2=0x7fce600a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 1 -- 172.21.15.190:0/2927385881 shutdown_connections 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.771+0000 7fce667a0640 1 -- 172.21.15.190:0/2927385881 wait complete. 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient: build_initial_monmap 2024-08-19T10:43:11.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient: monmap: 2024-08-19T10:43:11.790 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:11.790 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:11.790 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:11.790 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:11.790 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:11.790 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:11.790 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:11.790 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:11.790 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:11.790 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:11.790 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:11.791 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:11.791 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:11.791 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:11.791 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:11.791 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:11.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 1 Processor -- start 2024-08-19T10:43:11.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 1 -- start start 2024-08-19T10:43:11.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient: init 2024-08-19T10:43:11.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:11.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient: _add_conns ranks=[1,3,0,7,6,4,2,8,5] 2024-08-19T10:43:11.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce600a3610 0x7fce601d1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:11.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient(hunting): picked mon.noname-f con 0x7fce600a3610 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:11.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce600a3f30 0x7fce601d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:11.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient(hunting): picked mon.noname-g con 0x7fce600a3f30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:11.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce60137920 0x7fce601d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:11.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient(hunting): picked mon.noname-a con 0x7fce60137920 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:11.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:11.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fce600a4310 con 0x7fce60137920 2024-08-19T10:43:11.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:11.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fce60070a30 con 0x7fce600a3610 2024-08-19T10:43:11.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:11.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fce60137240 con 0x7fce600a3f30 2024-08-19T10:43:11.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:11.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5ffff640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce600a3610 msgr2=0x7fce601d1670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:11.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce667a0640 10 monclient(hunting): authenticate will time out at 1237.349248s 2024-08-19T10:43:11.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce600a3610 0x7fce601d1670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:11.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce600a3f30 0x7fce601d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:11.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce600a3f30 0x7fce601d1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:58078/0 (socket says 172.21.15.190:58078) 2024-08-19T10:43:11.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 1 -- 172.21.15.190:0/3200905244 learned_addr learned my addr 172.21.15.190:0/3200905244 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:11.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 10 monclient(hunting): get_auth_request con 0x7fce600a3f30 auth_method 0 2024-08-19T10:43:11.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:11.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:11.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:11.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:11.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:11.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:11.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce64d3d640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce60137920 0x7fce601d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:11.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 10 monclient(hunting): handle_auth_done global_id 34248 payload 824 2024-08-19T10:43:11.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 1 -- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce600a3610 msgr2=0x7fce601d1670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:11.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce600a3610 0x7fce601d1670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 1 -- 172.21.15.190:0/3200905244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce60137920 msgr2=0x7fce601d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce60137920 0x7fce601d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 1 monclient: found mon.noname-g 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 1 -- 172.21.15.190:0/3200905244 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce600726e0 con 0x7fce600a3f30 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce64d3d640 10 monclient: get_auth_request con 0x7fce60137920 auth_method 0 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce64d3d640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce60137920 0x7fce601d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.775+0000 7fce5f7fe640 1 -- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce600a3f30 msgr2=0x7fce601d1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5f7fe640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce600a3f30 0x7fce601d1bb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce5401c290 tx=0x7fce54019200 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:11.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 1 -- 172.21.15.190:0/3200905244 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fce540ba070 con 0x7fce600a3f30 2024-08-19T10:43:11.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:11.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:43:11.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 10 monclient: dump: 2024-08-19T10:43:11.796 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:11.796 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:11.796 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:11.796 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:11.796 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:11.796 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:11.796 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:11.796 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:11.797 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 10 monclient: _finish_auth 0 2024-08-19T10:43:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 10 monclient: _check_auth_tickets 2024-08-19T10:43:11.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 1 -- 172.21.15.190:0/3200905244 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce540b1040 con 0x7fce600a3f30 2024-08-19T10:43:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 1 -- 172.21.15.190:0/3200905244 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fce540b0070 con 0x7fce600a3f30 2024-08-19T10:43:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 10 monclient: dump: 2024-08-19T10:43:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:11.798 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:11.799 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:11.799 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:11.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce667a0640 5 monclient: authenticate success, global_id 34248 2024-08-19T10:43:11.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5e7fc640 4 set_mon_vals no callback set 2024-08-19T10:43:11.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce667a0640 10 monclient: _renew_subs 2024-08-19T10:43:11.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce667a0640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:11.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce667a0640 1 -- 172.21.15.190:0/3200905244 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6011f250 con 0x7fce600a3f30 2024-08-19T10:43:11.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce667a0640 10 monclient: _renew_subs 2024-08-19T10:43:11.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce667a0640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:11.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce667a0640 1 -- 172.21.15.190:0/3200905244 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6011e9a0 con 0x7fce600a3f30 2024-08-19T10:43:11.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce667a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:11.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce667a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:11.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce667a0640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:11.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce667a0640 1 -- 172.21.15.190:0/3200905244 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce6013cae0 con 0x7fce600a3f30 2024-08-19T10:43:11.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 1 -- 172.21.15.190:0/3200905244 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fce54004250 con 0x7fce600a3f30 2024-08-19T10:43:11.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fce4c03be00 0x7fce4c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:11.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5cff9640 1 -- 172.21.15.190:0/3200905244 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(161..161 src has 1..161) ==== 9265+0+0 (secure 0 0 0) 0x7fce540e8d60 con 0x7fce600a3f30 2024-08-19T10:43:11.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5ffff640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fce4c03be00 0x7fce4c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:11.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.779+0000 7fce5ffff640 10 monclient: get_auth_request con 0x7fce4c03be00 auth_method 0 2024-08-19T10:43:11.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.783+0000 7fce5ffff640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fce4c03be00 0x7fce4c03e2c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fce500094e0 tx=0x7fce50009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:11.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.783+0000 7fce5cff9640 1 -- 172.21.15.190:0/3200905244 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fce540ee020 con 0x7fce600a3f30 2024-08-19T10:43:11.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.783+0000 7fce5cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:11.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.783+0000 7fce5cff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.955+0000 7fce667a0640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-08-19T10:43:11.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.955+0000 7fce667a0640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-08-19T10:43:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.955+0000 7fce667a0640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:11.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:11.955+0000 7fce667a0640 1 -- 172.21.15.190:0/3200905244 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]} v 0) -- 0x7fce601373c0 con 0x7fce600a3f30 2024-08-19T10:43:13.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:13.387+0000 7fce5cff9640 1 -- 172.21.15.190:0/3200905244 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'fish' v162) ==== 137+0+0 (secure 0 0 0) 0x7fce600726e0 con 0x7fce600a3f30 2024-08-19T10:43:13.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:13.387+0000 7fce5cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-08-19T10:43:13.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:13.387+0000 7fce5cff9640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'fish' 2024-08-19T10:43:13.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:13.499+0000 7fce667a0640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-08-19T10:43:13.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:13.499+0000 7fce667a0640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-08-19T10:43:13.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:13.499+0000 7fce667a0640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:13.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:13.499+0000 7fce667a0640 1 -- 172.21.15.190:0/3200905244 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]} v 0) -- 0x7fce600008e0 con 0x7fce600a3f30 2024-08-19T10:43:14.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.391+0000 7fce5cff9640 1 -- 172.21.15.190:0/3200905244 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}]=0 osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' v163) ==== 241+0+0 (secure 0 0 0) 0x7fce54016170 con 0x7fce600a3f30 2024-08-19T10:43:14.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.391+0000 7fce5cff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "fish", "ids": ["osd.0"]}] 2024-08-19T10:43:14.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.391+0000 7fce5cff9640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' 2024-08-19T10:43:14.396 INFO:tasks.workunit.client.0.smithi190.stderr:osd.0 already set to class fish. set-device-class item id 0 name 'osd.0' device_class 'fish': no change. set osd(s) to class 'fish' 2024-08-19T10:43:14.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 -- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fce4c03be00 msgr2=0x7fce4c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:14.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fce4c03be00 0x7fce4c03e2c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fce500094e0 tx=0x7fce50009190 comp rx=0 tx=0).stop 2024-08-19T10:43:14.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 10 monclient: shutdown 2024-08-19T10:43:14.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:14.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 -- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce600a3f30 msgr2=0x7fce601d1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:14.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce600a3f30 0x7fce601d1bb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce5401c290 tx=0x7fce54019200 comp rx=0 tx=0).stop 2024-08-19T10:43:14.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 -- 172.21.15.190:0/3200905244 shutdown_connections 2024-08-19T10:43:14.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fce4c03be00 0x7fce4c03e2c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce60137920 0x7fce601d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce600a3f30 0x7fce601d1bb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 --2- 172.21.15.190:0/3200905244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce600a3610 0x7fce601d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 -- 172.21.15.190:0/3200905244 >> 172.21.15.190:0/3200905244 conn(0x7fce600a4700 msgr2=0x7fce60070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:14.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 -- 172.21.15.190:0/3200905244 shutdown_connections 2024-08-19T10:43:14.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.395+0000 7fce667a0640 1 -- 172.21.15.190:0/3200905244 wait complete. 2024-08-19T10:43:14.419 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush tree --show-shadow 2024-08-19T10:43:14.420 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 'osd\.0' 2024-08-19T10:43:14.420 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep fish 2024-08-19T10:43:14.420 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep '10\.' 2024-08-19T10:43:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.527+0000 7f732bbbc640 10 monclient: get_monmap_and_config 2024-08-19T10:43:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.527+0000 7f732bbbc640 10 monclient: build_initial_monmap 2024-08-19T10:43:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.527+0000 7f732bbbc640 10 monclient: monmap: 2024-08-19T10:43:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:14.535097+0000 2024-08-19T10:43:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:14.535097+0000 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:14.535 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 1 Processor -- start 2024-08-19T10:43:14.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 1 -- start start 2024-08-19T10:43:14.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 10 monclient: init 2024-08-19T10:43:14.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:14.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 10 monclient: _add_conns ranks=[10,2,4,6,8,1,0,5,3,7,9] 2024-08-19T10:43:14.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73240a46b0 0x7f73240a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 10 monclient(hunting): picked mon.noname-f con 0x7f73240a46b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:14.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f732409fd00 0x7f73240a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 10 monclient(hunting): picked mon.noname-b con 0x7f732409fd00 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:14.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7324136460 0x7f73241368c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 10 monclient(hunting): picked mon.noname-c con 0x7f7324136460 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f73240a0ad0 con 0x7f73240a46b0 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f73240a5510 con 0x7f732409fd00 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f73240a0410 con 0x7f7324136460 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329958640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f73240a46b0 msgr2=0x7f73240a4a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732bbbc640 10 monclient(hunting): authenticate will time out at 1240.105248s 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329958640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73240a46b0 0x7f73240a4a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732a159640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7324136460 msgr2=0x7f73241368c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f732a159640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7324136460 0x7f73241368c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f732409fd00 0x7f73240a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f732409fd00 0x7f73240a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:54206/0 (socket says 172.21.15.190:54206) 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 1 -- 172.21.15.190:0/181470209 learned_addr learned my addr 172.21.15.190:0/181470209 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 10 monclient(hunting): get_auth_request con 0x7f732409fd00 auth_method 0 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 10 monclient(hunting): handle_auth_done global_id 5825 payload 274 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 1 -- 172.21.15.190:0/181470209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7324136460 msgr2=0x7f73241368c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 1 --2- 172.21.15.190:0/181470209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7324136460 0x7f73241368c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 1 -- 172.21.15.190:0/181470209 >> v2:172.21.15.190:3300/0 conn(0x7f73240a46b0 msgr2=0x7f73240a4a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 1 --2- 172.21.15.190:0/181470209 >> v2:172.21.15.190:3300/0 conn(0x7f73240a46b0 0x7f73240a4a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 10 monclient: _finish_hunting 0 2024-08-19T10:43:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 1 monclient: found mon.noname-b 2024-08-19T10:43:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.531+0000 7f7329157640 1 -- 172.21.15.190:0/181470209 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73240a2dd0 con 0x7f732409fd00 2024-08-19T10:43:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7329157640 1 -- 172.21.15.190:0/181470209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f732409fd00 msgr2=0x7f73240a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7329157640 1 --2- 172.21.15.190:0/181470209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f732409fd00 0x7f73240a4fd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f732000bb70 tx=0x7f732001eb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=114a0970623b9f68 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 1 -- 172.21.15.190:0/181470209 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73200b4070 con 0x7f732409fd00 2024-08-19T10:43:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 10 monclient: dump: 2024-08-19T10:43:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 10 monclient: _finish_auth 0 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 10 monclient: _check_auth_tickets 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 1 -- 172.21.15.190:0/181470209 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73200a7cc0 con 0x7f732409fd00 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 1 -- 172.21.15.190:0/181470209 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73200ad7b0 con 0x7f732409fd00 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f732bbbc640 5 monclient: authenticate success, global_id 5825 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f732bbbc640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f732bbbc640 4 set_mon_vals no callback set 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.535+0000 7f7328956640 10 monclient: dump: 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:14.545 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient: shutdown 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 -- 172.21.15.190:0/181470209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f732409fd00 msgr2=0x7f73240a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 --2- 172.21.15.190:0/181470209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f732409fd00 0x7f73240a4fd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f732000bb70 tx=0x7f732001eb10 comp rx=0 tx=0).stop 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 -- 172.21.15.190:0/181470209 shutdown_connections 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 --2- 172.21.15.190:0/181470209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7324136460 0x7f73241368c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 --2- 172.21.15.190:0/181470209 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f732409fd00 0x7f73240a4fd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 --2- 172.21.15.190:0/181470209 >> v2:172.21.15.190:3300/0 conn(0x7f73240a46b0 0x7f73240a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 -- 172.21.15.190:0/181470209 >> 172.21.15.190:0/181470209 conn(0x7f73240a92a0 msgr2=0x7f73240a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 -- 172.21.15.190:0/181470209 shutdown_connections 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 -- 172.21.15.190:0/181470209 wait complete. 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient: build_initial_monmap 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient: monmap: 2024-08-19T10:43:14.551 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:14.552 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 Processor -- start 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 -- start start 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient: init 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient: _add_conns ranks=[4,0,1,2,8,3,7,5,6] 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f732409fd00 0x7f7324139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient(hunting): picked mon.noname-c con 0x7f732409fd00 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a46b0 0x7f732413a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient(hunting): picked mon.noname-a con 0x7f73240a46b0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7324136460 0x7f732413a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient(hunting): picked mon.noname-f con 0x7f7324136460 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:14.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f73240a61a0 con 0x7f73240a46b0 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f73240a07a0 con 0x7f7324136460 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f732406d0e0 con 0x7f732409fd00 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732bbbc640 10 monclient(hunting): authenticate will time out at 1240.113878s 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732a159640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7324136460 msgr2=0x7f732413a840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f732a159640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7324136460 0x7f732413a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329958640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f732409fd00 msgr2=0x7f7324139dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329958640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f732409fd00 0x7f7324139dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a46b0 0x7f732413a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:14.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a46b0 0x7f732413a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:39818/0 (socket says 172.21.15.190:39818) 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 1 -- 172.21.15.190:0/2066348309 learned_addr learned my addr 172.21.15.190:0/2066348309 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 10 monclient(hunting): get_auth_request con 0x7f73240a46b0 auth_method 0 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 10 monclient(hunting): handle_auth_done global_id 5841 payload 824 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 1 -- 172.21.15.190:0/2066348309 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f732409fd00 msgr2=0x7f7324139dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f732409fd00 0x7f7324139dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 1 -- 172.21.15.190:0/2066348309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7324136460 msgr2=0x7f732413a840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7324136460 0x7f732413a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 10 monclient: _finish_hunting 0 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 1 monclient: found mon.noname-a 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.539+0000 7f7329157640 1 -- 172.21.15.190:0/2066348309 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732409f520 con 0x7f73240a46b0 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f7329157640 1 -- 172.21.15.190:0/2066348309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a46b0 msgr2=0x7f732413a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f7329157640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a46b0 0x7f732413a300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f732000b2c0 tx=0x7f732001b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 1 -- 172.21.15.190:0/2066348309 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73200bd070 con 0x7f73240a46b0 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 10 monclient: dump: 2024-08-19T10:43:14.556 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:14.557 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 10 monclient: _finish_auth 0 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 10 monclient: _check_auth_tickets 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 1 -- 172.21.15.190:0/2066348309 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7320003a90 con 0x7f73240a46b0 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 1 -- 172.21.15.190:0/2066348309 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73200b4040 con 0x7f73240a46b0 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 10 monclient: dump: 2024-08-19T10:43:14.558 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:14.559 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:14.559 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.559 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.559 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:14.559 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.559 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:14.559 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:14.559 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f7317fff640 4 set_mon_vals no callback set 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f732bbbc640 5 monclient: authenticate success, global_id 5841 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f732bbbc640 10 monclient: _renew_subs 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f732bbbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:14.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f732bbbc640 1 -- 172.21.15.190:0/2066348309 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7324120410 con 0x7f73240a46b0 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f732bbbc640 10 monclient: _renew_subs 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f732bbbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f732bbbc640 1 -- 172.21.15.190:0/2066348309 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732413c0c0 con 0x7f73240a46b0 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f732bbbc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f732bbbc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f732bbbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f732bbbc640 1 -- 172.21.15.190:0/2066348309 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732406d560 con 0x7f73240a46b0 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.543+0000 7f73167fc640 1 -- 172.21.15.190:0/2066348309 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7320003c40 con 0x7f73240a46b0 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.547+0000 7f73167fc640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f730803be30 0x7f730803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.547+0000 7f73167fc640 1 -- 172.21.15.190:0/2066348309 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 9930+0+0 (secure 0 0 0) 0x7f73200e9e20 con 0x7f73240a46b0 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.547+0000 7f7329958640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f730803be30 0x7f730803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:14.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.547+0000 7f7329958640 10 monclient: get_auth_request con 0x7f730803be30 auth_method 0 2024-08-19T10:43:14.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.547+0000 7f7329958640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f730803be30 0x7f730803e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7318006fd0 tx=0x7f7318006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:14.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.559+0000 7f73167fc640 1 -- 172.21.15.190:0/2066348309 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f732409f520 con 0x7f73240a46b0 2024-08-19T10:43:14.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.559+0000 7f73167fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:14.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.559+0000 7f73167fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:14.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.727+0000 7f732bbbc640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:14.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.727+0000 7f732bbbc640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:14.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.727+0000 7f732bbbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:14.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.727+0000 7f732bbbc640 1 -- 172.21.15.190:0/2066348309 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f7324073460 con 0x7f73240a46b0 2024-08-19T10:43:14.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.727+0000 7f73167fc640 1 -- 172.21.15.190:0/2066348309 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v163) ==== 83+0+2175 (secure 0 0 0) 0x7f73240a2dd0 con 0x7f73240a46b0 2024-08-19T10:43:14.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.727+0000 7f73167fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:14.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.727+0000 7f73167fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.827+0000 7f732bbbc640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.827+0000 7f732bbbc640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.827+0000 7f732bbbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:14.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.827+0000 7f732bbbc640 1 -- 172.21.15.190:0/2066348309 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f73240008e0 con 0x7f73240a46b0 2024-08-19T10:43:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.827+0000 7f73167fc640 1 -- 172.21.15.190:0/2066348309 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v163) ==== 83+0+2175 (secure 0 0 0) 0x7f7320018820 con 0x7f73240a46b0 2024-08-19T10:43:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.827+0000 7f73167fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:14.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.827+0000 7f73167fc640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 -- 172.21.15.190:0/2066348309 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f730803be30 msgr2=0x7f730803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f730803be30 0x7f730803e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7318006fd0 tx=0x7f7318006d40 comp rx=0 tx=0).stop 2024-08-19T10:43:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 10 monclient: shutdown 2024-08-19T10:43:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 -- 172.21.15.190:0/2066348309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a46b0 msgr2=0x7f732413a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a46b0 0x7f732413a300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f732000b2c0 tx=0x7f732001b8b0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 -- 172.21.15.190:0/2066348309 shutdown_connections 2024-08-19T10:43:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f730803be30 0x7f730803e2f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73240a46b0 0x7f732413a300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7324136460 0x7f732413a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 --2- 172.21.15.190:0/2066348309 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f732409fd00 0x7f7324139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 -- 172.21.15.190:0/2066348309 >> 172.21.15.190:0/2066348309 conn(0x7f73240a92a0 msgr2=0x7f73240a1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:14.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 -- 172.21.15.190:0/2066348309 shutdown_connections 2024-08-19T10:43:14.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.831+0000 7f732bbbc640 1 -- 172.21.15.190:0/2066348309 wait complete. 2024-08-19T10:43:14.850 INFO:tasks.workunit.client.0.smithi190.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:14.850 INFO:tasks.workunit.client.0.smithi190.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:14.851 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 'osd\.0' 2024-08-19T10:43:14.851 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush tree --show-shadow 2024-08-19T10:43:14.851 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep fish 2024-08-19T10:43:14.851 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep '11\.' 2024-08-19T10:43:14.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient: get_monmap_and_config 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient: build_initial_monmap 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient: monmap: 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:14.969699+0000 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:14.969699+0000 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:14.969 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:14.970 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:14.970 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:14.970 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:14.970 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:14.970 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:14.970 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:14.970 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 1 Processor -- start 2024-08-19T10:43:14.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 1 -- start start 2024-08-19T10:43:14.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient: init 2024-08-19T10:43:14.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:14.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient: _add_conns ranks=[0,8,6,9,4,1,7,10,3,5,2] 2024-08-19T10:43:14.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc0a80a5940 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:14.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 0x7fc0a813c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient(hunting): picked mon.noname-e con 0x7fc0a80727d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:14.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0a80a6430 0x7fc0a813ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient(hunting): picked mon.noname-d con 0x7fc0a80a6430 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0a80a43f0 con 0x7fc0a80a5940 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fc0a8074110 con 0x7fc0a80a6430 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc0a8072550 con 0x7fc0a80727d0 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0ae9be640 10 monclient(hunting): authenticate will time out at 1240.539516s 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0acf5b640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0a80a6430 msgr2=0x7fc0a813ee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0acf5b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0a80a6430 0x7fc0a813ee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0a7fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc0a80a5940 0x7fc0a80a5d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:41410/0 (socket says 172.21.15.190:41410) 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0a77fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 0x7fc0a813c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:14.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0a7fff640 1 -- 172.21.15.190:0/546210347 learned_addr learned my addr 172.21.15.190:0/546210347 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0a77fe640 10 monclient(hunting): get_auth_request con 0x7fc0a80727d0 auth_method 0 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0a77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.963+0000 7fc0a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 10 monclient(hunting): handle_auth_done global_id 5714 payload 274 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 1 -- 172.21.15.190:0/546210347 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0a80a6430 msgr2=0x7fc0a813ee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 1 --2- 172.21.15.190:0/546210347 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0a80a6430 0x7fc0a813ee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 1 -- 172.21.15.190:0/546210347 >> v1:172.21.15.89:6789/0 conn(0x7fc0a80a5940 legacy=0x7fc0a80a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:14.974 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 1 monclient: found mon.noname-e 2024-08-19T10:43:14.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:14.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:43:14.975 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.967+0000 7fc0a77fe640 1 -- 172.21.15.190:0/546210347 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a8073870 con 0x7fc0a80727d0 2024-08-19T10:43:14.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a77fe640 1 -- 172.21.15.190:0/546210347 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 msgr2=0x7fc0a813c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:14.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a77fe640 1 --2- 172.21.15.190:0/546210347 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 0x7fc0a813c930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0097f0 tx=0x7fc09c01c450 comp rx=0 tx=0).ready entity=mon.8 client_cookie=8957c6929dfeb645 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:14.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 1 -- 172.21.15.190:0/546210347 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc09c0b0070 con 0x7fc0a80727d0 2024-08-19T10:43:14.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:14.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:43:14.978 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 10 monclient: dump: 2024-08-19T10:43:14.978 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:14.979 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:14.979 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.979 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.979 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:14.979 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.979 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:14.979 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:14.979 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:14.979 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:14.979 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:14.979 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 1 -- 172.21.15.190:0/546210347 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc09c0aa810 con 0x7fc0a80727d0 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 1 -- 172.21.15.190:0/546210347 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc09c0a93e0 con 0x7fc0a80727d0 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:14.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0a6ffd640 10 monclient: dump: 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:14.981 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:14.982 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:14.982 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:14.982 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:14.982 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 5 monclient: authenticate success, global_id 5714 2024-08-19T10:43:14.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:14.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 4 set_mon_vals no callback set 2024-08-19T10:43:14.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 10 monclient: shutdown 2024-08-19T10:43:14.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:14.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 1 -- 172.21.15.190:0/546210347 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 msgr2=0x7fc0a813c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:14.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 1 --2- 172.21.15.190:0/546210347 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 0x7fc0a813c930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0097f0 tx=0x7fc09c01c450 comp rx=0 tx=0).stop 2024-08-19T10:43:14.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 1 -- 172.21.15.190:0/546210347 shutdown_connections 2024-08-19T10:43:14.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 1 --2- 172.21.15.190:0/546210347 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc0a80a6430 0x7fc0a813ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 1 --2- 172.21.15.190:0/546210347 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 0x7fc0a813c930 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 1 -- 172.21.15.190:0/546210347 >> 172.21.15.190:0/546210347 conn(0x7fc0a80aa530 msgr2=0x7fc0a80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:14.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 1 -- 172.21.15.190:0/546210347 shutdown_connections 2024-08-19T10:43:14.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.971+0000 7fc0ae9be640 1 -- 172.21.15.190:0/546210347 wait complete. 2024-08-19T10:43:14.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient: build_initial_monmap 2024-08-19T10:43:14.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient: monmap: 2024-08-19T10:43:14.984 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:14.984 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:14.985 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:14.985 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:14.985 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:14.985 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.985 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:14.985 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:14.985 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:14.985 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:14.985 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:14.986 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:14.986 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:14.986 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:14.986 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:14.986 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 1 Processor -- start 2024-08-19T10:43:14.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 1 -- start start 2024-08-19T10:43:14.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient: init 2024-08-19T10:43:14.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:14.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient: _add_conns ranks=[8,7,4,3,5,1,0,2,6] 2024-08-19T10:43:14.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 0x7fc0a8126a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient(hunting): picked mon.noname-e con 0x7fc0a80727d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:14.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0a80a5940 0x7fc0a813d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient(hunting): picked mon.noname-i con 0x7fc0a80a5940 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:14.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc0a80a6430 0x7fc0a81243e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient(hunting): picked mon.noname-c con 0x7fc0a80a6430 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:14.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:14.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc0a8141d60 con 0x7fc0a80a6430 2024-08-19T10:43:14.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc0a8141be0 con 0x7fc0a80a5940 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc0a8141ee0 con 0x7fc0a80727d0 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0ae9be640 10 monclient(hunting): authenticate will time out at 1240.549109s 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0acf5b640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc0a80a6430 msgr2=0x7fc0a81243e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0a80a5940 0x7fc0a813d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0acf5b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc0a80a6430 0x7fc0a81243e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0a80a5940 0x7fc0a813d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:35600/0 (socket says 172.21.15.190:35600) 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 1 -- 172.21.15.190:0/1449857877 learned_addr learned my addr 172.21.15.190:0/1449857877 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 10 monclient(hunting): get_auth_request con 0x7fc0a80a5940 auth_method 0 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:14.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:14.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:14.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:14.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:14.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:14.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a7fff640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 0x7fc0a8126a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:14.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a7fff640 10 monclient(hunting): get_auth_request con 0x7fc0a80727d0 auth_method 0 2024-08-19T10:43:14.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:14.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:14.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:14.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 10 monclient(hunting): handle_auth_done global_id 15019 payload 824 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 1 -- 172.21.15.190:0/1449857877 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 msgr2=0x7fc0a8126a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 0x7fc0a8126a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 1 -- 172.21.15.190:0/1449857877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc0a80a6430 msgr2=0x7fc0a81243e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc0a80a6430 0x7fc0a81243e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 1 monclient: found mon.noname-i 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 1 -- 172.21.15.190:0/1449857877 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a81cfc30 con 0x7fc0a80a5940 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a7fff640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 0x7fc0a8126a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 1 -- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0a80a5940 msgr2=0x7fc0a813d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:14.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.975+0000 7fc0a77fe640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0a80a5940 0x7fc0a813d8a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0ae040 tx=0x7fc09c0191f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a77fe640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0a80a5940 0x7fc0a813d8a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0ae040 tx=0x7fc09c0191f0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fc09c0b0040 mon_map magic: 0 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 1 -- 172.21.15.190:0/1449857877 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc09c0b0040 con 0x7fc0a80a5940 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 10 monclient: dump: 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:14.995 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:14.996 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:14.996 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:14.996 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:14.996 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:14.996 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:14.996 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:14.996 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:14.996 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 10 monclient: _finish_auth 0 2024-08-19T10:43:14.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 1 -- 172.21.15.190:0/1449857877 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc09c0044b0 con 0x7fc0a80a5940 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0ae9be640 5 monclient: authenticate success, global_id 15019 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0ae9be640 10 monclient: _renew_subs 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0ae9be640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0ae9be640 1 -- 172.21.15.190:0/1449857877 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a81273c0 con 0x7fc0a80a5940 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0ae9be640 10 monclient: _renew_subs 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0ae9be640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a67fc640 4 set_mon_vals no callback set 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0ae9be640 1 -- 172.21.15.190:0/1449857877 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a80a95e0 con 0x7fc0a80a5940 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0ae9be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0ae9be640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:14.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0ae9be640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0ae9be640 1 -- 172.21.15.190:0/1449857877 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a80a5d20 con 0x7fc0a80a5940 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 1 -- 172.21.15.190:0/1449857877 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc09c003a90 con 0x7fc0a80a5940 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 10 monclient: dump: 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:14.998 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 1 -- 172.21.15.190:0/1449857877 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc09c003c50 con 0x7fc0a80a5940 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a4ff9640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc09403be50 0x7fc09403e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a7fff640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc09403be50 0x7fc09403e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.979+0000 7fc0a7fff640 10 monclient: get_auth_request con 0x7fc09403be50 auth_method 0 2024-08-19T10:43:14.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.983+0000 7fc0a7fff640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc09403be50 0x7fc09403e310 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc098005c00 tx=0x7fc098009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:15.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:14.983+0000 7fc0a4ff9640 1 -- 172.21.15.190:0/1449857877 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(163..163 src has 1..163) ==== 9930+0+0 (secure 0 0 0) 0x7fc09c0a4e00 con 0x7fc0a80a5940 2024-08-19T10:43:15.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.603+0000 7fc0a4ff9640 1 -- 172.21.15.190:0/1449857877 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc09c0160d0 con 0x7fc0a80a5940 2024-08-19T10:43:15.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.603+0000 7fc0a4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:15.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.603+0000 7fc0a4ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:15.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.791+0000 7fc0ae9be640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:15.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.791+0000 7fc0ae9be640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:15.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.791+0000 7fc0ae9be640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:15.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.791+0000 7fc0ae9be640 1 -- 172.21.15.190:0/1449857877 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7fc0a8125ed0 con 0x7fc0a80a5940 2024-08-19T10:43:15.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.791+0000 7fc0a4ff9640 1 -- 172.21.15.190:0/1449857877 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v163) ==== 83+0+2175 (secure 0 0 0) 0x7fc09c0b3390 con 0x7fc0a80a5940 2024-08-19T10:43:15.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.791+0000 7fc0a4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:15.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.791+0000 7fc0a4ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:15.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.899+0000 7fc0ae9be640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:15.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.899+0000 7fc0ae9be640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:15.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.899+0000 7fc0ae9be640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:15.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.899+0000 7fc0ae9be640 1 -- 172.21.15.190:0/1449857877 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7fc0a80008e0 con 0x7fc0a80a5940 2024-08-19T10:43:15.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.899+0000 7fc0a4ff9640 1 -- 172.21.15.190:0/1449857877 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v163) ==== 83+0+2175 (secure 0 0 0) 0x7fc09c0e75f0 con 0x7fc0a80a5940 2024-08-19T10:43:15.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.899+0000 7fc0a4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:15.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.899+0000 7fc0a4ff9640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:15.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 -- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc09403be50 msgr2=0x7fc09403e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:15.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc09403be50 0x7fc09403e310 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc098005c00 tx=0x7fc098009040 comp rx=0 tx=0).stop 2024-08-19T10:43:15.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 10 monclient: shutdown 2024-08-19T10:43:15.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:15.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 -- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0a80a5940 msgr2=0x7fc0a813d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:15.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0a80a5940 0x7fc0a813d8a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0ae040 tx=0x7fc09c0191f0 comp rx=0 tx=0).stop 2024-08-19T10:43:15.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 -- 172.21.15.190:0/1449857877 shutdown_connections 2024-08-19T10:43:15.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc0a80a6430 0x7fc0a81243e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:15.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc0a80a5940 0x7fc0a813d8a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:15.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc09403be50 0x7fc09403e310 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:15.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 --2- 172.21.15.190:0/1449857877 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc0a80727d0 0x7fc0a8126a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:15.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 -- 172.21.15.190:0/1449857877 >> 172.21.15.190:0/1449857877 conn(0x7fc0a80aa530 msgr2=0x7fc0a80a1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:15.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 -- 172.21.15.190:0/1449857877 shutdown_connections 2024-08-19T10:43:15.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:15.903+0000 7fc0ae9be640 1 -- 172.21.15.190:0/1449857877 wait complete. 2024-08-19T10:43:15.924 INFO:tasks.workunit.client.0.smithi190.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:15.924 INFO:tasks.workunit.client.0.smithi190.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:15.924 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush tree --show-shadow 2024-08-19T10:43:15.924 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 'osd\.0' 2024-08-19T10:43:15.925 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep fish 2024-08-19T10:43:15.925 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep '12\.' 2024-08-19T10:43:16.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient: get_monmap_and_config 2024-08-19T10:43:16.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient: build_initial_monmap 2024-08-19T10:43:16.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient: monmap: 2024-08-19T10:43:16.040 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:16.040 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:16.040 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:16.040683+0000 2024-08-19T10:43:16.040 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:16.040683+0000 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:16.041 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:16.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 1 Processor -- start 2024-08-19T10:43:16.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 1 -- start start 2024-08-19T10:43:16.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient: init 2024-08-19T10:43:16.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:16.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient: _add_conns ranks=[1,5,10,4,9,3,6,8,2,0,7] 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fdd380a3610 addr v1:172.21.15.190:6789/0 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd380703c0 0x7fdd38142c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient(hunting): picked mon.noname-h con 0x7fdd380703c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd380a4100 0x7fdd38145150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient(hunting): picked mon.noname-f con 0x7fdd380a4100 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd38070840 con 0x7fdd380a3610 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fdd38072bb0 con 0x7fdd380a4100 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fdd38071ea0 con 0x7fdd380703c0 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:16.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3f462640 10 monclient(hunting): authenticate will time out at 1241.610381s 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3d9ff640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fdd380a4100 msgr2=0x7fdd38145150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3d9ff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd380a4100 0x7fdd38145150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3d1fe640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fdd380a3610 legacy=0x7fdd380a39f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd380703c0 0x7fdd38142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd380703c0 0x7fdd38142c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:50300/0 (socket says 172.21.15.190:50300) 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 1 -- 172.21.15.190:0/571395873 learned_addr learned my addr 172.21.15.190:0/571395873 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 10 monclient(hunting): get_auth_request con 0x7fdd380703c0 auth_method 0 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:16.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:16.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 10 monclient(hunting): handle_auth_done global_id 5828 payload 274 2024-08-19T10:43:16.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 1 -- 172.21.15.190:0/571395873 >> v2:172.21.15.190:3300/0 conn(0x7fdd380a4100 msgr2=0x7fdd38145150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:16.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 1 --2- 172.21.15.190:0/571395873 >> v2:172.21.15.190:3300/0 conn(0x7fdd380a4100 0x7fdd38145150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:16.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 1 -- 172.21.15.190:0/571395873 >> v1:172.21.15.190:6789/0 conn(0x7fdd380a3610 legacy=0x7fdd380a39f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:16.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 10 monclient: _finish_hunting 0 2024-08-19T10:43:16.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 1 monclient: found mon.noname-h 2024-08-19T10:43:16.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:16.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:16.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.035+0000 7fdd3c9fd640 1 -- 172.21.15.190:0/571395873 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd38072180 con 0x7fdd380703c0 2024-08-19T10:43:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd3c9fd640 1 -- 172.21.15.190:0/571395873 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd380703c0 msgr2=0x7fdd38142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd3c9fd640 1 --2- 172.21.15.190:0/571395873 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd380703c0 0x7fdd38142c00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009870 tx=0x7fdd2801cc90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=c72861233b6459f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 1 -- 172.21.15.190:0/571395873 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd280b0070 con 0x7fdd380703c0 2024-08-19T10:43:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:16.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 10 monclient: dump: 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:16.051 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 10 monclient: _finish_auth 0 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd3f462640 5 monclient: authenticate success, global_id 5828 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd3f462640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 1 -- 172.21.15.190:0/571395873 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd280aa810 con 0x7fdd380703c0 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd3f462640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd3f462640 4 set_mon_vals no callback set 2024-08-19T10:43:16.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 1 -- 172.21.15.190:0/571395873 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd280a4e60 con 0x7fdd380703c0 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.039+0000 7fdd2ffff640 10 monclient: dump: 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:16.053 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient: shutdown 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 -- 172.21.15.190:0/571395873 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd380703c0 msgr2=0x7fdd38142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 --2- 172.21.15.190:0/571395873 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd380703c0 0x7fdd38142c00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009870 tx=0x7fdd2801cc90 comp rx=0 tx=0).stop 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 -- 172.21.15.190:0/571395873 shutdown_connections 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 --2- 172.21.15.190:0/571395873 >> v2:172.21.15.190:3300/0 conn(0x7fdd380a4100 0x7fdd38145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 --2- 172.21.15.190:0/571395873 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd380703c0 0x7fdd38142c00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:16.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 -- 172.21.15.190:0/571395873 >> 172.21.15.190:0/571395873 conn(0x7fdd380a4700 msgr2=0x7fdd380a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 -- 172.21.15.190:0/571395873 shutdown_connections 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 -- 172.21.15.190:0/571395873 wait complete. 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient: build_initial_monmap 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient: monmap: 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:16.055 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 Processor -- start 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 -- start start 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient: init 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient: _add_conns ranks=[2,4,1,7,5,8,6,3,0] 2024-08-19T10:43:16.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd380703c0 0x7fdd381d5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient(hunting): picked mon.noname-b con 0x7fdd380703c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd380a3610 0x7fdd3813cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient(hunting): picked mon.noname-c con 0x7fdd380a3610 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd380a4100 0x7fdd38142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient(hunting): picked mon.noname-f con 0x7fdd380a4100 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdd380a2110 con 0x7fdd380a4100 2024-08-19T10:43:16.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fdd380a1f90 con 0x7fdd380703c0 2024-08-19T10:43:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fdd38147eb0 con 0x7fdd380a3610 2024-08-19T10:43:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:16.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3f462640 10 monclient(hunting): authenticate will time out at 1241.618138s 2024-08-19T10:43:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d9ff640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd380a4100 msgr2=0x7fdd38142480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d9ff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd380a4100 0x7fdd38142480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3c9fd640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd380a3610 msgr2=0x7fdd3813cd10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3c9fd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd380a3610 0x7fdd3813cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd380703c0 0x7fdd381d5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd380703c0 0x7fdd381d5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:48846/0 (socket says 172.21.15.190:48846) 2024-08-19T10:43:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 1 -- 172.21.15.190:0/2020890949 learned_addr learned my addr 172.21.15.190:0/2020890949 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 10 monclient(hunting): get_auth_request con 0x7fdd380703c0 auth_method 0 2024-08-19T10:43:16.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 10 monclient(hunting): handle_auth_done global_id 5834 payload 824 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 1 -- 172.21.15.190:0/2020890949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd380a3610 msgr2=0x7fdd3813cd10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd380a3610 0x7fdd3813cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 1 -- 172.21.15.190:0/2020890949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd380a4100 msgr2=0x7fdd38142480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd380a4100 0x7fdd38142480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 1 monclient: found mon.noname-b 2024-08-19T10:43:16.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:16.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:16.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.043+0000 7fdd3d1fe640 1 -- 172.21.15.190:0/2020890949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd381270e0 con 0x7fdd380703c0 2024-08-19T10:43:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3d1fe640 1 -- 172.21.15.190:0/2020890949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd380703c0 msgr2=0x7fdd381d5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3d1fe640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd380703c0 0x7fdd381d5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400b580 tx=0x7fdd3400ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 1 -- 172.21.15.190:0/2020890949 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd34003de0 con 0x7fdd380703c0 2024-08-19T10:43:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 10 monclient: dump: 2024-08-19T10:43:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:16.063 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:16.064 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 10 monclient: _finish_auth 0 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3f462640 5 monclient: authenticate success, global_id 5834 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 1 -- 172.21.15.190:0/2020890949 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd34015070 con 0x7fdd380703c0 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3f462640 10 monclient: _renew_subs 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3f462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3f462640 1 -- 172.21.15.190:0/2020890949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd381273c0 con 0x7fdd380703c0 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 1 -- 172.21.15.190:0/2020890949 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd34010480 con 0x7fdd380703c0 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:16.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2f7fe640 4 set_mon_vals no callback set 2024-08-19T10:43:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 10 monclient: dump: 2024-08-19T10:43:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:16.066 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:16.067 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3f462640 10 monclient: _renew_subs 2024-08-19T10:43:16.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3f462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3f462640 1 -- 172.21.15.190:0/2020890949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd38126e00 con 0x7fdd380703c0 2024-08-19T10:43:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3f462640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3f462640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3f462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3f462640 1 -- 172.21.15.190:0/2020890949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3813d9b0 con 0x7fdd380703c0 2024-08-19T10:43:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 1 -- 172.21.15.190:0/2020890949 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fdd340050f0 con 0x7fdd380703c0 2024-08-19T10:43:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdd1c03be00 0x7fdd1c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd2dffb640 1 -- 172.21.15.190:0/2020890949 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(163..163 src has 1..163) ==== 9930+0+0 (secure 0 0 0) 0x7fdd340508e0 con 0x7fdd380703c0 2024-08-19T10:43:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3c9fd640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdd1c03be00 0x7fdd1c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3c9fd640 10 monclient: get_auth_request con 0x7fdd1c03be00 auth_method 0 2024-08-19T10:43:16.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.051+0000 7fdd3c9fd640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdd1c03be00 0x7fdd1c03e2c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009870 tx=0x7fdd28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.067+0000 7fdd2dffb640 1 -- 172.21.15.190:0/2020890949 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fdd3813d9b0 con 0x7fdd380703c0 2024-08-19T10:43:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.067+0000 7fdd2dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:16.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.067+0000 7fdd2dffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:16.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.259+0000 7fdd3f462640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:16.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.259+0000 7fdd3f462640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:16.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.259+0000 7fdd3f462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:16.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.259+0000 7fdd3f462640 1 -- 172.21.15.190:0/2020890949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7fdd380a3a70 con 0x7fdd380703c0 2024-08-19T10:43:16.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:16.259+0000 7fdd3d1fe640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd380703c0 0x7fdd381d5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400b580 tx=0x7fdd3400ba40 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fdd3403f090 mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v163) 2024-08-19T10:43:17.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.091+0000 7fdd2dffb640 1 -- 172.21.15.190:0/2020890949 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v163) ==== 83+0+2175 (secure 0 0 0) 0x7fdd3403f090 con 0x7fdd380703c0 2024-08-19T10:43:17.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.091+0000 7fdd2dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:17.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.091+0000 7fdd2dffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:17.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.199+0000 7fdd3f462640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:17.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.199+0000 7fdd3f462640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:17.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.199+0000 7fdd3f462640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:43:17.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.199+0000 7fdd3f462640 1 -- 172.21.15.190:0/2020890949 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7fdd380a1410 con 0x7fdd380703c0 2024-08-19T10:43:17.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd2dffb640 1 -- 172.21.15.190:0/2020890949 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v163) ==== 83+0+2175 (secure 0 0 0) 0x7fdd3404cf00 con 0x7fdd380703c0 2024-08-19T10:43:17.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd2dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:17.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd2dffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:17.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 -- 172.21.15.190:0/2020890949 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdd1c03be00 msgr2=0x7fdd1c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:17.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdd1c03be00 0x7fdd1c03e2c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdd28009870 tx=0x7fdd28008040 comp rx=0 tx=0).stop 2024-08-19T10:43:17.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 10 monclient: shutdown 2024-08-19T10:43:17.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:17.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 -- 172.21.15.190:0/2020890949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd380703c0 msgr2=0x7fdd381d5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:17.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd380703c0 0x7fdd381d5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400b580 tx=0x7fdd3400ba40 comp rx=0 tx=0).stop 2024-08-19T10:43:17.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 -- 172.21.15.190:0/2020890949 shutdown_connections 2024-08-19T10:43:17.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdd1c03be00 0x7fdd1c03e2c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:17.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd380a4100 0x7fdd38142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:17.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd380a3610 0x7fdd3813cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:17.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 --2- 172.21.15.190:0/2020890949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd380703c0 0x7fdd381d5050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:17.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 -- 172.21.15.190:0/2020890949 >> 172.21.15.190:0/2020890949 conn(0x7fdd380a4700 msgr2=0x7fdd380a6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:17.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 -- 172.21.15.190:0/2020890949 shutdown_connections 2024-08-19T10:43:17.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.203+0000 7fdd3f462640 1 -- 172.21.15.190:0/2020890949 wait complete. 2024-08-19T10:43:17.221 INFO:tasks.workunit.client.0.smithi190.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:17.221 INFO:tasks.workunit.client.0.smithi190.stdout: 0 fish 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:17.221 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm-device-class osd.0 2024-08-19T10:43:17.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient: get_monmap_and_config 2024-08-19T10:43:17.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient: build_initial_monmap 2024-08-19T10:43:17.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient: monmap: 2024-08-19T10:43:17.340 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:17.340 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:17.340 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:17.340788+0000 2024-08-19T10:43:17.340 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:17.340788+0000 2024-08-19T10:43:17.340 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:17.340 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:17.341 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:17.342 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:17.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 1 Processor -- start 2024-08-19T10:43:17.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 1 -- start start 2024-08-19T10:43:17.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient: init 2024-08-19T10:43:17.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:17.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient: _add_conns ranks=[10,7,2,9,1,8,3,5,4,6,0] 2024-08-19T10:43:17.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efce00a3c80 0x7efce00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:17.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient(hunting): picked mon.noname-f con 0x7efce00a3c80 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:17.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efce00a45a0 0x7efce013ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:17.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient(hunting): picked mon.noname-i con 0x7efce00a45a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:17.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efce013bfb0 0x7efce013e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient(hunting): picked mon.noname-b con 0x7efce013bfb0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7efce009e9b0 con 0x7efce00a3c80 2024-08-19T10:43:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7efce00a22f0 con 0x7efce013bfb0 2024-08-19T10:43:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7efce009dd30 con 0x7efce00a45a0 2024-08-19T10:43:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce8732640 10 monclient(hunting): authenticate will time out at 1242.910694s 2024-08-19T10:43:17.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce64ce640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7efce00a3c80 msgr2=0x7efce00a4060 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce64ce640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efce00a3c80 0x7efce00a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efce00a45a0 0x7efce013ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efce00a45a0 0x7efce013ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34410/0 (socket says 172.21.15.190:34410) 2024-08-19T10:43:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 1 -- 172.21.15.190:0/900644989 learned_addr learned my addr 172.21.15.190:0/900644989 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 10 monclient(hunting): get_auth_request con 0x7efce00a45a0 auth_method 0 2024-08-19T10:43:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:17.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce6ccf640 1 --2- 172.21.15.190:0/900644989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efce013bfb0 0x7efce013e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce6ccf640 10 monclient(hunting): get_auth_request con 0x7efce013bfb0 auth_method 0 2024-08-19T10:43:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce6ccf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce6ccf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce6ccf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:17.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 10 monclient(hunting): handle_auth_done global_id 15028 payload 274 2024-08-19T10:43:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 1 -- 172.21.15.190:0/900644989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efce013bfb0 msgr2=0x7efce013e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 1 --2- 172.21.15.190:0/900644989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efce013bfb0 0x7efce013e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 1 -- 172.21.15.190:0/900644989 >> v2:172.21.15.190:3300/0 conn(0x7efce00a3c80 msgr2=0x7efce00a4060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 1 --2- 172.21.15.190:0/900644989 >> v2:172.21.15.190:3300/0 conn(0x7efce00a3c80 0x7efce00a4060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 10 monclient: _finish_hunting 0 2024-08-19T10:43:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 1 monclient: found mon.noname-i 2024-08-19T10:43:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:17.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.339+0000 7efce5ccd640 1 -- 172.21.15.190:0/900644989 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce00a2070 con 0x7efce00a45a0 2024-08-19T10:43:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce5ccd640 1 -- 172.21.15.190:0/900644989 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efce00a45a0 msgr2=0x7efce013ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce5ccd640 1 --2- 172.21.15.190:0/900644989 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efce00a45a0 0x7efce013ba70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009870 tx=0x7efcdc01c440 comp rx=0 tx=0).ready entity=mon.7 client_cookie=7b412a0df95d18b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 1 -- 172.21.15.190:0/900644989 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efcdc0b0070 con 0x7efce00a45a0 2024-08-19T10:43:17.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 10 monclient: dump: 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:17.357 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:17.358 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 10 monclient: _finish_auth 0 2024-08-19T10:43:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 10 monclient: _check_auth_tickets 2024-08-19T10:43:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:17.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 1 -- 172.21.15.190:0/900644989 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcdc0aa810 con 0x7efce00a45a0 2024-08-19T10:43:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce8732640 5 monclient: authenticate success, global_id 15028 2024-08-19T10:43:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce8732640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce8732640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 1 -- 172.21.15.190:0/900644989 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efcdc0a9300 con 0x7efce00a45a0 2024-08-19T10:43:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce8732640 4 set_mon_vals no callback set 2024-08-19T10:43:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:17.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.343+0000 7efce54cc640 10 monclient: dump: 2024-08-19T10:43:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:17.360 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:17.361 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:17.362 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient: shutdown 2024-08-19T10:43:17.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 -- 172.21.15.190:0/900644989 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efce00a45a0 msgr2=0x7efce013ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 --2- 172.21.15.190:0/900644989 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efce00a45a0 0x7efce013ba70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009870 tx=0x7efcdc01c440 comp rx=0 tx=0).stop 2024-08-19T10:43:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 -- 172.21.15.190:0/900644989 shutdown_connections 2024-08-19T10:43:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 --2- 172.21.15.190:0/900644989 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efce013bfb0 0x7efce013e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 --2- 172.21.15.190:0/900644989 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efce00a45a0 0x7efce013ba70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 --2- 172.21.15.190:0/900644989 >> v2:172.21.15.190:3300/0 conn(0x7efce00a3c80 0x7efce00a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 -- 172.21.15.190:0/900644989 >> 172.21.15.190:0/900644989 conn(0x7efce00a4f40 msgr2=0x7efce00a5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 -- 172.21.15.190:0/900644989 shutdown_connections 2024-08-19T10:43:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 -- 172.21.15.190:0/900644989 wait complete. 2024-08-19T10:43:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient: build_initial_monmap 2024-08-19T10:43:17.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient: monmap: 2024-08-19T10:43:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:17.364 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 Processor -- start 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 -- start start 2024-08-19T10:43:17.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient: init 2024-08-19T10:43:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient: _add_conns ranks=[1,0,2,5,4,7,3,8,6] 2024-08-19T10:43:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce00a3c80 0x7efce0120340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient(hunting): picked mon.noname-f con 0x7efce00a3c80 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce00a45a0 0x7efce0120880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient(hunting): picked mon.noname-a con 0x7efce00a45a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efce013bfb0 0x7efce0120dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient(hunting): picked mon.noname-b con 0x7efce013bfb0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:17.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efce00a4980 con 0x7efce00a45a0 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efce0069720 con 0x7efce00a3c80 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7efce0124190 con 0x7efce013bfb0 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce8732640 10 monclient(hunting): authenticate will time out at 1242.917863s 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce64ce640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce00a3c80 msgr2=0x7efce0120340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce64ce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce00a3c80 0x7efce0120340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce00a45a0 0x7efce0120880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce6ccf640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efce013bfb0 0x7efce0120dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce00a45a0 0x7efce0120880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:55452/0 (socket says 172.21.15.190:55452) 2024-08-19T10:43:17.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 1 -- 172.21.15.190:0/3260336837 learned_addr learned my addr 172.21.15.190:0/3260336837 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 10 monclient(hunting): get_auth_request con 0x7efce00a45a0 auth_method 0 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce6ccf640 10 monclient(hunting): get_auth_request con 0x7efce013bfb0 auth_method 0 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce6ccf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce6ccf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce6ccf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce6ccf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce6ccf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce6ccf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:17.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 10 monclient(hunting): handle_auth_done global_id 5850 payload 824 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 1 -- 172.21.15.190:0/3260336837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efce013bfb0 msgr2=0x7efce0120dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efce013bfb0 0x7efce0120dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 1 -- 172.21.15.190:0/3260336837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce00a3c80 msgr2=0x7efce0120340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce00a3c80 0x7efce0120340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 10 monclient: _finish_hunting 0 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 1 monclient: found mon.noname-a 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce5ccd640 1 -- 172.21.15.190:0/3260336837 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce00a33c0 con 0x7efce00a45a0 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.347+0000 7efce6ccf640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efce013bfb0 0x7efce0120dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce5ccd640 1 -- 172.21.15.190:0/3260336837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce00a45a0 msgr2=0x7efce0120880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce5ccd640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce00a45a0 0x7efce0120880 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0ae040 tx=0x7efcdc0191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:17.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 1 -- 172.21.15.190:0/3260336837 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efcdc0b0040 con 0x7efce00a45a0 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 10 monclient: dump: 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:17.370 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 1 -- 172.21.15.190:0/3260336837 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcdc0044b0 con 0x7efce00a45a0 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce8732640 5 monclient: authenticate success, global_id 5850 2024-08-19T10:43:17.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce8732640 10 monclient: _renew_subs 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce8732640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 1 -- 172.21.15.190:0/3260336837 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efcdc003620 con 0x7efce00a45a0 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce8732640 1 -- 172.21.15.190:0/3260336837 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce00a4060 con 0x7efce00a45a0 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 10 monclient: dump: 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:17.372 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce4ccb640 4 set_mon_vals no callback set 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce8732640 10 monclient: _renew_subs 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce8732640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce8732640 1 -- 172.21.15.190:0/3260336837 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce0121e30 con 0x7efce00a45a0 2024-08-19T10:43:17.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce8732640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:17.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce8732640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:17.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce8732640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:17.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce8732640 1 -- 172.21.15.190:0/3260336837 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce009cb80 con 0x7efce00a45a0 2024-08-19T10:43:17.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 1 -- 172.21.15.190:0/3260336837 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efcdc004040 con 0x7efce00a45a0 2024-08-19T10:43:17.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcc403bd30 0x7efcc403e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:17.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efcd6ffd640 1 -- 172.21.15.190:0/3260336837 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 9930+0+0 (secure 0 0 0) 0x7efcdc0e93f0 con 0x7efce00a45a0 2024-08-19T10:43:17.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce64ce640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcc403bd30 0x7efcc403e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:17.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.351+0000 7efce64ce640 10 monclient: get_auth_request con 0x7efcc403bd30 auth_method 0 2024-08-19T10:43:17.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.355+0000 7efce64ce640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcc403bd30 0x7efcc403e1f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efcd0003990 tx=0x7efcd0006cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:17.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.363+0000 7efcd6ffd640 1 -- 172.21.15.190:0/3260336837 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7efcdc016150 con 0x7efce00a45a0 2024-08-19T10:43:17.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.363+0000 7efcd6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:17.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.363+0000 7efcd6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:17.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.555+0000 7efce8732640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:17.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.555+0000 7efce8732640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:17.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.555+0000 7efce8732640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:17.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:17.555+0000 7efce8732640 1 -- 172.21.15.190:0/3260336837 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7efce006e4d0 con 0x7efce00a45a0 2024-08-19T10:43:18.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:18.435+0000 7efcd6ffd640 1 -- 172.21.15.190:0/3260336837 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v164) ==== 123+0+0 (secure 0 0 0) 0x7efcdc0e5a10 con 0x7efce00a45a0 2024-08-19T10:43:18.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:18.435+0000 7efcd6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:18.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:18.435+0000 7efcd6ffd640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-08-19T10:43:18.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:18.547+0000 7efce8732640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:18.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:18.547+0000 7efce8732640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:18.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:18.547+0000 7efce8732640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:18.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:18.547+0000 7efce8732640 1 -- 172.21.15.190:0/3260336837 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7efce0122bf0 con 0x7efce00a45a0 2024-08-19T10:43:19.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.439+0000 7efcd6ffd640 1 -- 172.21.15.190:0/3260336837 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v165) ==== 149+0+0 (secure 0 0 0) 0x7efcdc0e5d90 con 0x7efce00a45a0 2024-08-19T10:43:19.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.439+0000 7efcd6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:19.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.439+0000 7efcd6ffd640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-08-19T10:43:19.442 INFO:tasks.workunit.client.0.smithi190.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-08-19T10:43:19.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 -- 172.21.15.190:0/3260336837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcc403bd30 msgr2=0x7efcc403e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:19.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcc403bd30 0x7efcc403e1f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efcd0003990 tx=0x7efcd0006cf0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 10 monclient: shutdown 2024-08-19T10:43:19.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:19.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 -- 172.21.15.190:0/3260336837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce00a45a0 msgr2=0x7efce0120880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:19.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce00a45a0 0x7efce0120880 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0ae040 tx=0x7efcdc0191e0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 -- 172.21.15.190:0/3260336837 shutdown_connections 2024-08-19T10:43:19.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcc403bd30 0x7efcc403e1f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efce013bfb0 0x7efce0120dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efce00a45a0 0x7efce0120880 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 --2- 172.21.15.190:0/3260336837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce00a3c80 0x7efce0120340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 -- 172.21.15.190:0/3260336837 >> 172.21.15.190:0/3260336837 conn(0x7efce00a4f40 msgr2=0x7efce00a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:19.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 -- 172.21.15.190:0/3260336837 shutdown_connections 2024-08-19T10:43:19.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.443+0000 7efce8732640 1 -- 172.21.15.190:0/3260336837 wait complete. 2024-08-19T10:43:19.466 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush set-device-class globster osd.0 2024-08-19T10:43:19.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient: get_monmap_and_config 2024-08-19T10:43:19.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient: build_initial_monmap 2024-08-19T10:43:19.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient: monmap: 2024-08-19T10:43:19.587 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:19.587 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:19.587 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:19.587980+0000 2024-08-19T10:43:19.587 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:19.587980+0000 2024-08-19T10:43:19.588 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:19.588 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:19.588 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:19.588 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:19.588 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:19.588 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:19.588 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:19.588 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:19.588 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:19.588 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:19.589 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:19.589 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:19.589 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:19.589 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:19.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 1 Processor -- start 2024-08-19T10:43:19.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 1 -- start start 2024-08-19T10:43:19.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient: init 2024-08-19T10:43:19.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:19.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient: _add_conns ranks=[2,7,5,4,0,9,10,6,1,3,8] 2024-08-19T10:43:19.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe6940a46b0 0x7fe6940a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:19.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient(hunting): picked mon.noname-b con 0x7fe6940a46b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:19.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6940a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:19.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient(hunting): picked mon.noname-i con 0x7fe6940a2e00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:19.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe694136640 0x7fe694136a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:19.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient(hunting): picked mon.noname-h con 0x7fe694136640 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fe6940a2180 con 0x7fe6940a46b0 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe6940a23d0 con 0x7fe694136640 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fe6940a0a90 con 0x7fe6940a2e00 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69b5e0640 10 monclient(hunting): authenticate will time out at 1245.158611s 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6940a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe699b7d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe694136640 0x7fe694136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6940a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34412/0 (socket says 172.21.15.190:34412) 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 1 -- 172.21.15.190:0/3573003607 learned_addr learned my addr 172.21.15.190:0/3573003607 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:19.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 10 monclient(hunting): get_auth_request con 0x7fe6940a2e00 auth_method 0 2024-08-19T10:43:19.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:19.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:19.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:19.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe699b7d640 10 monclient(hunting): get_auth_request con 0x7fe694136640 auth_method 0 2024-08-19T10:43:19.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe699b7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:19.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe699b7d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:19.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe699b7d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:19.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:19.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:19.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:19.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe699b7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:19.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe699b7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:19.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe699b7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:19.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 10 monclient(hunting): handle_auth_done global_id 15037 payload 274 2024-08-19T10:43:19.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe69937c640 1 --2- 172.21.15.190:0/3573003607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe6940a46b0 0x7fe6940a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:19.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 1 -- 172.21.15.190:0/3573003607 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe694136640 msgr2=0x7fe694136a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:19.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 1 --2- 172.21.15.190:0/3573003607 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe694136640 0x7fe694136a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 1 -- 172.21.15.190:0/3573003607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe6940a46b0 msgr2=0x7fe6940a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:19.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 1 --2- 172.21.15.190:0/3573003607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe6940a46b0 0x7fe6940a4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 10 monclient: _finish_hunting 0 2024-08-19T10:43:19.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 1 monclient: found mon.noname-i 2024-08-19T10:43:19.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:19.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:19.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.587+0000 7fe698b7b640 1 -- 172.21.15.190:0/3573003607 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6940a0810 con 0x7fe6940a2e00 2024-08-19T10:43:19.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.591+0000 7fe698b7b640 1 -- 172.21.15.190:0/3573003607 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 msgr2=0x7fe6940a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:19.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.591+0000 7fe698b7b640 1 --2- 172.21.15.190:0/3573003607 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6940a4fd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6840098d0 tx=0x7fe68401e650 comp rx=0 tx=0).ready entity=mon.7 client_cookie=fa21cecc9a9c80f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:19.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 1 -- 172.21.15.190:0/3573003607 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe6840ac870 con 0x7fe6940a2e00 2024-08-19T10:43:19.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:19.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:19.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 10 monclient: dump: 2024-08-19T10:43:19.600 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:19.600 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:19.600 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:19.601 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:19.601 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:19.601 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:19.601 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:19.601 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:19.601 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:19.601 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:19.601 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:19.601 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:19.601 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:19.601 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:19.602 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:19.602 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:19.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 10 monclient: _finish_auth 0 2024-08-19T10:43:19.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:19.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:19.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 1 -- 172.21.15.190:0/3573003607 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6840b1070 con 0x7fe6940a2e00 2024-08-19T10:43:19.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:19.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 1 -- 172.21.15.190:0/3573003607 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe6840ab360 con 0x7fe6940a2e00 2024-08-19T10:43:19.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 5 monclient: authenticate success, global_id 15037 2024-08-19T10:43:19.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:19.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 4 set_mon_vals no callback set 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe68bfff640 10 monclient: dump: 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:19.603 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient: shutdown 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 -- 172.21.15.190:0/3573003607 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 msgr2=0x7fe6940a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 --2- 172.21.15.190:0/3573003607 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6940a4fd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6840098d0 tx=0x7fe68401e650 comp rx=0 tx=0).stop 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 -- 172.21.15.190:0/3573003607 shutdown_connections 2024-08-19T10:43:19.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 --2- 172.21.15.190:0/3573003607 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe694136640 0x7fe694136a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 --2- 172.21.15.190:0/3573003607 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6940a4fd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 --2- 172.21.15.190:0/3573003607 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe6940a46b0 0x7fe6940a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 -- 172.21.15.190:0/3573003607 >> 172.21.15.190:0/3573003607 conn(0x7fe6940a92a0 msgr2=0x7fe6940a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 -- 172.21.15.190:0/3573003607 shutdown_connections 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 -- 172.21.15.190:0/3573003607 wait complete. 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient: build_initial_monmap 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient: monmap: 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:19.605 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 Processor -- start 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 -- start start 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient: init 2024-08-19T10:43:19.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient: _add_conns ranks=[7,4,2,6,3,8,1,0,5] 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6941d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient(hunting): picked mon.noname-i con 0x7fe6940a2e00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe6940a46b0 0x7fe6941d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient(hunting): picked mon.noname-c con 0x7fe6940a46b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe694136640 0x7fe694139660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient(hunting): picked mon.noname-b con 0x7fe694136640 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fe6940a61a0 con 0x7fe694136640 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fe6940a5510 con 0x7fe6940a46b0 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:19.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fe69406d0e0 con 0x7fe6940a2e00 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69b5e0640 10 monclient(hunting): authenticate will time out at 1245.166437s 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6941d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe698b7b640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe6940a46b0 msgr2=0x7fe6941d1f60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe698b7b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe6940a46b0 0x7fe6941d1f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6941d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34420/0 (socket says 172.21.15.190:34420) 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 1 -- 172.21.15.190:0/1119432373 learned_addr learned my addr 172.21.15.190:0/1119432373 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 10 monclient(hunting): get_auth_request con 0x7fe6940a2e00 auth_method 0 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:19.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe699b7d640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe694136640 0x7fe694139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 10 monclient(hunting): handle_auth_done global_id 15046 payload 824 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 1 -- 172.21.15.190:0/1119432373 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe6940a46b0 msgr2=0x7fe6941d1f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe6940a46b0 0x7fe6941d1f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 1 -- 172.21.15.190:0/1119432373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe694136640 msgr2=0x7fe694139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe694136640 0x7fe694139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 10 monclient: _finish_hunting 0 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 1 monclient: found mon.noname-i 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:19.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe69937c640 1 -- 172.21.15.190:0/1119432373 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69413b9e0 con 0x7fe6940a2e00 2024-08-19T10:43:19.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe699b7d640 10 monclient: get_auth_request con 0x7fe694136640 auth_method 0 2024-08-19T10:43:19.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.595+0000 7fe699b7d640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe694136640 0x7fe694139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:43:19.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe69937c640 1 -- 172.21.15.190:0/1119432373 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 msgr2=0x7fe6941d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:19.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe69937c640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6941d1a20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c7f0 tx=0x7fe69000ccb0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:19.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 1 -- 172.21.15.190:0/1119432373 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:43:19.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 1 -- 172.21.15.190:0/1119432373 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe690015070 con 0x7fe6940a2e00 2024-08-19T10:43:19.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:19.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:19.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 10 monclient: dump: 2024-08-19T10:43:19.612 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:19.612 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:19.612 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:19.612 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:19.612 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:19.613 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:19.613 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:19.613 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:19.613 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:19.613 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:19.613 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:19.613 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:19.613 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:19.613 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:19.613 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:19.614 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:19.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 10 monclient: _finish_auth 0 2024-08-19T10:43:19.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:43:19.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:19.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 1 -- 172.21.15.190:0/1119432373 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe690007ca0 con 0x7fe6940a2e00 2024-08-19T10:43:19.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:19.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 1 -- 172.21.15.190:0/1119432373 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe690007e50 con 0x7fe6940a2e00 2024-08-19T10:43:19.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:19.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:19.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe689ffb640 10 monclient: dump: 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:19.615 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe68affd640 4 set_mon_vals no callback set 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe69b5e0640 5 monclient: authenticate success, global_id 15046 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe69b5e0640 10 monclient: _renew_subs 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe69b5e0640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe69b5e0640 1 -- 172.21.15.190:0/1119432373 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe694123700 con 0x7fe6940a2e00 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe69b5e0640 10 monclient: _renew_subs 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe69b5e0640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.599+0000 7fe69b5e0640 1 -- 172.21.15.190:0/1119432373 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6940a8620 con 0x7fe6940a2e00 2024-08-19T10:43:19.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.603+0000 7fe69b5e0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.603+0000 7fe69b5e0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.603+0000 7fe69b5e0640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.603+0000 7fe69b5e0640 1 -- 172.21.15.190:0/1119432373 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69406d470 con 0x7fe6940a2e00 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.603+0000 7fe689ffb640 1 -- 172.21.15.190:0/1119432373 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe690007830 con 0x7fe6940a2e00 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.603+0000 7fe689ffb640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe67803bfa0 0x7fe67803e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.603+0000 7fe689ffb640 1 -- 172.21.15.190:0/1119432373 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(165..165 src has 1..165) ==== 9265+0+0 (secure 0 0 0) 0x7fe69004f6f0 con 0x7fe6940a2e00 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.603+0000 7fe698b7b640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe67803bfa0 0x7fe67803e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.603+0000 7fe698b7b640 10 monclient: get_auth_request con 0x7fe67803bfa0 auth_method 0 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.603+0000 7fe698b7b640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe67803bfa0 0x7fe67803e460 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe694070cd0 tx=0x7fe684008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.607+0000 7fe689ffb640 1 -- 172.21.15.190:0/1119432373 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe69001e070 con 0x7fe6940a2e00 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.607+0000 7fe689ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:19.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.607+0000 7fe689ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:19.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.779+0000 7fe69b5e0640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-08-19T10:43:19.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.779+0000 7fe69b5e0640 10 monclient: _send_command 2 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-08-19T10:43:19.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.779+0000 7fe69b5e0640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:19.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:19.779+0000 7fe69b5e0640 1 -- 172.21.15.190:0/1119432373 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]} v 0) -- 0x7fe694074410 con 0x7fe6940a2e00 2024-08-19T10:43:20.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:20.903+0000 7fe689ffb640 1 -- 172.21.15.190:0/1119432373 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}]=0 set osd(s) 0 to class 'globster' v166) ==== 145+0+0 (secure 0 0 0) 0x7fe690011e20 con 0x7fe6940a2e00 2024-08-19T10:43:20.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:20.903+0000 7fe689ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-08-19T10:43:20.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:20.903+0000 7fe689ffb640 10 monclient: _finish_command 2 = system:0 set osd(s) 0 to class 'globster' 2024-08-19T10:43:21.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:21.011+0000 7fe69b5e0640 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-08-19T10:43:21.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:21.011+0000 7fe69b5e0640 10 monclient: _send_command 3 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-08-19T10:43:21.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:21.011+0000 7fe69b5e0640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:21.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:21.011+0000 7fe69b5e0640 1 -- 172.21.15.190:0/1119432373 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]} v 0) -- 0x7fe6940008e0 con 0x7fe6940a2e00 2024-08-19T10:43:22.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:21.999+0000 7fe689ffb640 1 -- 172.21.15.190:0/1119432373 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}]=0 osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' v167) ==== 257+0+0 (secure 0 0 0) 0x7fe69004cf50 con 0x7fe6940a2e00 2024-08-19T10:43:22.000 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:21.999+0000 7fe689ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush set-device-class", "class": "globster", "ids": ["osd.0"]}] 2024-08-19T10:43:22.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:21.999+0000 7fe689ffb640 10 monclient: _finish_command 3 = system:0 osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' 2024-08-19T10:43:22.001 INFO:tasks.workunit.client.0.smithi190.stderr:osd.0 already set to class globster. set-device-class item id 0 name 'osd.0' device_class 'globster': no change. set osd(s) to class 'globster' 2024-08-19T10:43:22.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 -- 172.21.15.190:0/1119432373 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe67803bfa0 msgr2=0x7fe67803e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:22.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe67803bfa0 0x7fe67803e460 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe694070cd0 tx=0x7fe684008040 comp rx=0 tx=0).stop 2024-08-19T10:43:22.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 10 monclient: shutdown 2024-08-19T10:43:22.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:22.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 -- 172.21.15.190:0/1119432373 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 msgr2=0x7fe6941d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:22.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6941d1a20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c7f0 tx=0x7fe69000ccb0 comp rx=0 tx=0).stop 2024-08-19T10:43:22.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 -- 172.21.15.190:0/1119432373 shutdown_connections 2024-08-19T10:43:22.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe67803bfa0 0x7fe67803e460 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:22.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe6940a46b0 0x7fe6941d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:22.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe694136640 0x7fe694139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:22.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 --2- 172.21.15.190:0/1119432373 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe6940a2e00 0x7fe6941d1a20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:22.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 -- 172.21.15.190:0/1119432373 >> 172.21.15.190:0/1119432373 conn(0x7fe6940a92a0 msgr2=0x7fe6940a1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:22.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 -- 172.21.15.190:0/1119432373 shutdown_connections 2024-08-19T10:43:22.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.003+0000 7fe69b5e0640 1 -- 172.21.15.190:0/1119432373 wait complete. 2024-08-19T10:43:22.026 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush tree --show-shadow 2024-08-19T10:43:22.026 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 'osd\.0' 2024-08-19T10:43:22.026 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep globster 2024-08-19T10:43:22.027 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep '10\.' 2024-08-19T10:43:22.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient: get_monmap_and_config 2024-08-19T10:43:22.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient: build_initial_monmap 2024-08-19T10:43:22.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient: monmap: 2024-08-19T10:43:22.148 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:22.148 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:22.148 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:22.149009+0000 2024-08-19T10:43:22.148 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:22.149009+0000 2024-08-19T10:43:22.148 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:22.149 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:22.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 1 Processor -- start 2024-08-19T10:43:22.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 1 -- start start 2024-08-19T10:43:22.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient: init 2024-08-19T10:43:22.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:22.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient: _add_conns ranks=[0,8,4,7,9,3,5,1,2,6,10] 2024-08-19T10:43:22.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7c0c0a6740 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:22.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 0x7f7c0c0a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:22.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient(hunting): picked mon.noname-e con 0x7f7c0c0a6b20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:22.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c0c13eaa0 0x7f7c0c140e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:22.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient(hunting): picked mon.noname-c con 0x7f7c0c13eaa0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:22.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c0c0a5d90 con 0x7f7c0c0a6740 2024-08-19T10:43:22.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:22.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f7c0c0a4c20 con 0x7f7c0c13eaa0 2024-08-19T10:43:22.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:22.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7c0c0a3530 con 0x7f7c0c0a6b20 2024-08-19T10:43:22.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:22.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c11d5a640 10 monclient(hunting): authenticate will time out at 1247.718812s 2024-08-19T10:43:22.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0bfff640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c0c13eaa0 msgr2=0x7f7c0c140e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:22.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c0c13eaa0 0x7f7c0c140e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:22.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0b7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7c0c0a6740 0x7f7c0c0a5680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:36812/0 (socket says 172.21.15.190:36812) 2024-08-19T10:43:22.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0b7fe640 1 -- 172.21.15.190:0/689103724 learned_addr learned my addr 172.21.15.190:0/689103724 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:43:22.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 1 --2- 172.21.15.190:0/689103724 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 0x7f7c0c0a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:22.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 10 monclient(hunting): get_auth_request con 0x7f7c0c0a6b20 auth_method 0 2024-08-19T10:43:22.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:22.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:22.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:22.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:22.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:22.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:22.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 10 monclient(hunting): handle_auth_done global_id 5732 payload 274 2024-08-19T10:43:22.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 1 -- 172.21.15.190:0/689103724 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c0c13eaa0 msgr2=0x7f7c0c140e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:22.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 1 --2- 172.21.15.190:0/689103724 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c0c13eaa0 0x7f7c0c140e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:22.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 1 -- 172.21.15.190:0/689103724 >> v1:172.21.15.89:6789/0 conn(0x7f7c0c0a6740 legacy=0x7f7c0c0a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:22.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 10 monclient: _finish_hunting 0 2024-08-19T10:43:22.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 1 monclient: found mon.noname-e 2024-08-19T10:43:22.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:22.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:43:22.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.147+0000 7f7c0affd640 1 -- 172.21.15.190:0/689103724 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c0a3740 con 0x7f7c0c0a6b20 2024-08-19T10:43:22.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0affd640 1 -- 172.21.15.190:0/689103724 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 msgr2=0x7f7c0c0a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:22.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0affd640 1 --2- 172.21.15.190:0/689103724 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 0x7f7c0c0a6f00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009820 tx=0x7f7c0001c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=582cb389dccce4ef server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:22.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 1 -- 172.21.15.190:0/689103724 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7c000b0070 con 0x7f7c0c0a6b20 2024-08-19T10:43:22.168 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:22.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:43:22.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 10 monclient: dump: 2024-08-19T10:43:22.169 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:22.169 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:22.169 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:22.170 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:22.170 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:22.170 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:22.170 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:22.170 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:22.170 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:22.171 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:22.171 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:22.171 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:22.171 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:22.171 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:22.172 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:22.172 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:22.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:43:22.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:43:22.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:22.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 1 -- 172.21.15.190:0/689103724 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c000aa810 con 0x7f7c0c0a6b20 2024-08-19T10:43:22.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:22.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 1 -- 172.21.15.190:0/689103724 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7c000aae30 con 0x7f7c0c0a6b20 2024-08-19T10:43:22.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:22.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:22.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0a7fc640 10 monclient: dump: 2024-08-19T10:43:22.174 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:22.174 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:22.174 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:22.174 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:22.174 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:22.175 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:22.175 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:22.175 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:22.175 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:22.175 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:22.176 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:22.176 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:22.176 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:22.176 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:22.176 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:22.177 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:22.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 5 monclient: authenticate success, global_id 5732 2024-08-19T10:43:22.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:22.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 4 set_mon_vals no callback set 2024-08-19T10:43:22.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient: shutdown 2024-08-19T10:43:22.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:22.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 -- 172.21.15.190:0/689103724 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 msgr2=0x7f7c0c0a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:22.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 --2- 172.21.15.190:0/689103724 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 0x7f7c0c0a6f00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009820 tx=0x7f7c0001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:43:22.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 -- 172.21.15.190:0/689103724 shutdown_connections 2024-08-19T10:43:22.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 --2- 172.21.15.190:0/689103724 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7c0c13eaa0 0x7f7c0c140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:22.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 --2- 172.21.15.190:0/689103724 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 0x7f7c0c0a6f00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:22.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 -- 172.21.15.190:0/689103724 >> 172.21.15.190:0/689103724 conn(0x7f7c0c0abd40 msgr2=0x7f7c0c0ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:22.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 -- 172.21.15.190:0/689103724 shutdown_connections 2024-08-19T10:43:22.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 -- 172.21.15.190:0/689103724 wait complete. 2024-08-19T10:43:22.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient: build_initial_monmap 2024-08-19T10:43:22.179 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient: monmap: 2024-08-19T10:43:22.179 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:22.180 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:22.180 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:22.180 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:22.180 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:22.180 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:22.181 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:22.181 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:22.181 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:22.181 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:22.181 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:22.182 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:22.182 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:22.182 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:22.182 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:22.182 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:22.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 Processor -- start 2024-08-19T10:43:22.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 -- start start 2024-08-19T10:43:22.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient: init 2024-08-19T10:43:22.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:22.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient: _add_conns ranks=[5,8,0,7,4,3,6,2,1] 2024-08-19T10:43:22.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7c0c0a6740 0x7f7c0c1225d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:22.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient(hunting): picked mon.noname-h con 0x7f7c0c0a6740 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:22.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 0x7f7c0c1d1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:22.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient(hunting): picked mon.noname-e con 0x7f7c0c0a6b20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:22.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c0c13eaa0 0x7f7c0c0a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:22.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient(hunting): picked mon.noname-a con 0x7f7c0c13eaa0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:22.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:22.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c143d70 con 0x7f7c0c13eaa0 2024-08-19T10:43:22.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:22.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7c0c143bf0 con 0x7f7c0c0a6740 2024-08-19T10:43:22.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:22.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7c0c143ef0 con 0x7f7c0c0a6b20 2024-08-19T10:43:22.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:22.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c11d5a640 10 monclient(hunting): authenticate will time out at 1247.727687s 2024-08-19T10:43:22.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0b7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7c0c0a6740 0x7f7c0c1225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:22.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0b7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7c0c0a6740 0x7f7c0c1225d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:50320/0 (socket says 172.21.15.190:50320) 2024-08-19T10:43:22.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0b7fe640 1 -- 172.21.15.190:0/3369248400 learned_addr learned my addr 172.21.15.190:0/3369248400 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:22.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0b7fe640 10 monclient(hunting): get_auth_request con 0x7f7c0c0a6740 auth_method 0 2024-08-19T10:43:22.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:22.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:22.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:22.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:22.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:22.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.155+0000 7f7c0b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:22.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0bfff640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c0c13eaa0 0x7f7c0c0a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:22.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 10 monclient(hunting): handle_auth_done global_id 5846 payload 824 2024-08-19T10:43:22.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0affd640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 0x7f7c0c1d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:22.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 1 -- 172.21.15.190:0/3369248400 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 msgr2=0x7f7c0c1d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:22.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 0x7f7c0c1d1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:22.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 1 -- 172.21.15.190:0/3369248400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c0c13eaa0 msgr2=0x7f7c0c0a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:22.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c0c13eaa0 0x7f7c0c0a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:22.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:22.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 1 monclient: found mon.noname-h 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 1 -- 172.21.15.190:0/3369248400 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c13dc60 con 0x7f7c0c0a6740 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0bfff640 10 monclient: get_auth_request con 0x7f7c0c13eaa0 auth_method 0 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0bfff640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c0c13eaa0 0x7f7c0c0a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 1 -- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7c0c0a6740 msgr2=0x7f7c0c1225d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7c0c0a6740 0x7f7c0c1225d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00c470 tx=0x7f7bfc00c930 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c0b7fe640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7c0c0a6740 0x7f7c0c1225d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00c470 tx=0x7f7bfc00c930 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f7bfc005d00 mon_map magic: 0 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7bf3fff640 1 -- 172.21.15.190:0/3369248400 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7bfc005d00 con 0x7f7c0c0a6740 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7bf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7bf3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7bf3fff640 10 monclient: dump: 2024-08-19T10:43:22.190 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:22.191 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7bf3fff640 10 monclient: _finish_auth 0 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7bf3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7bf3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c11d5a640 5 monclient: authenticate success, global_id 5846 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7bf3fff640 1 -- 172.21.15.190:0/3369248400 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bfc0074e0 con 0x7f7c0c0a6740 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7bf3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c09ffb640 4 set_mon_vals no callback set 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c11d5a640 10 monclient: _renew_subs 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c11d5a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c11d5a640 1 -- 172.21.15.190:0/3369248400 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c123810 con 0x7f7c0c0a6740 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c11d5a640 10 monclient: _renew_subs 2024-08-19T10:43:22.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c11d5a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.159+0000 7f7c11d5a640 1 -- 172.21.15.190:0/3369248400 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c122390 con 0x7f7c0c0a6740 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7c11d5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7c11d5a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7c11d5a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7c11d5a640 1 -- 172.21.15.190:0/3369248400 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0c0a6f00 con 0x7f7c0c0a6740 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7bf3fff640 1 -- 172.21.15.190:0/3369248400 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7bfc007690 con 0x7f7c0c0a6740 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7bf3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7bf3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7bf3fff640 10 monclient: dump: 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:22.193 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7bf3fff640 1 -- 172.21.15.190:0/3369248400 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7bfc007070 con 0x7f7c0c0a6740 2024-08-19T10:43:22.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7bf3fff640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7bf803be50 0x7f7bf803e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:22.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7c0affd640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7bf803be50 0x7f7bf803e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:22.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7c0affd640 10 monclient: get_auth_request con 0x7f7bf803be50 auth_method 0 2024-08-19T10:43:22.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.163+0000 7f7c0affd640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7bf803be50 0x7f7bf803e310 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009820 tx=0x7f7c000a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:22.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.167+0000 7f7bf3fff640 1 -- 172.21.15.190:0/3369248400 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(167..167 src has 1..167) ==== 9954+0+0 (secure 0 0 0) 0x7f7bfc050200 con 0x7f7c0c0a6740 2024-08-19T10:43:22.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.783+0000 7f7bf3fff640 1 -- 172.21.15.190:0/3369248400 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7c0c13dc60 con 0x7f7c0c0a6740 2024-08-19T10:43:22.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.783+0000 7f7bf3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:22.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.783+0000 7f7bf3fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:22.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.967+0000 7f7c11d5a640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:22.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.967+0000 7f7c11d5a640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:22.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.967+0000 7f7c11d5a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:22.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.967+0000 7f7c11d5a640 1 -- 172.21.15.190:0/3369248400 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f7c0c1c8e10 con 0x7f7c0c0a6740 2024-08-19T10:43:22.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.971+0000 7f7bf3fff640 1 -- 172.21.15.190:0/3369248400 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2378 (secure 0 0 0) 0x7f7bfc011c70 con 0x7f7c0c0a6740 2024-08-19T10:43:22.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.971+0000 7f7bf3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:22.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:22.971+0000 7f7bf3fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:23.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.079+0000 7f7c11d5a640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:23.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.079+0000 7f7c11d5a640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:23.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.079+0000 7f7c11d5a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:23.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.079+0000 7f7c11d5a640 1 -- 172.21.15.190:0/3369248400 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f7c0c0008e0 con 0x7f7c0c0a6740 2024-08-19T10:43:23.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7bf3fff640 1 -- 172.21.15.190:0/3369248400 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2378 (secure 0 0 0) 0x7f7bfc04d7b0 con 0x7f7c0c0a6740 2024-08-19T10:43:23.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7bf3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:23.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7bf3fff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:23.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 -- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7bf803be50 msgr2=0x7f7bf803e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:23.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7bf803be50 0x7f7bf803e310 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009820 tx=0x7f7c000a6000 comp rx=0 tx=0).stop 2024-08-19T10:43:23.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 10 monclient: shutdown 2024-08-19T10:43:23.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:23.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 -- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7c0c0a6740 msgr2=0x7f7c0c1225d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:23.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7c0c0a6740 0x7f7c0c1225d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00c470 tx=0x7f7bfc00c930 comp rx=0 tx=0).stop 2024-08-19T10:43:23.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 -- 172.21.15.190:0/3369248400 shutdown_connections 2024-08-19T10:43:23.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c0c13eaa0 0x7f7c0c0a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:23.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7bf803be50 0x7f7bf803e310 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:23.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7c0c0a6b20 0x7f7c0c1d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:23.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 --2- 172.21.15.190:0/3369248400 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7c0c0a6740 0x7f7c0c1225d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:23.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 -- 172.21.15.190:0/3369248400 >> 172.21.15.190:0/3369248400 conn(0x7f7c0c0abd40 msgr2=0x7f7c0c13ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:23.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 -- 172.21.15.190:0/3369248400 shutdown_connections 2024-08-19T10:43:23.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.083+0000 7f7c11d5a640 1 -- 172.21.15.190:0/3369248400 wait complete. 2024-08-19T10:43:23.100 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:23.101 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush tree --show-shadow 2024-08-19T10:43:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 'osd\.0' 2024-08-19T10:43:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep globster 2024-08-19T10:43:23.101 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep '11\.' 2024-08-19T10:43:23.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.215+0000 7f9463774640 10 monclient: get_monmap_and_config 2024-08-19T10:43:23.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.215+0000 7f9463774640 10 monclient: build_initial_monmap 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.215+0000 7f9463774640 10 monclient: monmap: 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:23.218436+0000 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:23.218436+0000 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:23.218 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:23.219 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:23.219 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:23.219 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:23.219 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:23.219 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:23.219 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:23.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 1 Processor -- start 2024-08-19T10:43:23.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 1 -- start start 2024-08-19T10:43:23.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 10 monclient: init 2024-08-19T10:43:23.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:23.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 10 monclient: _add_conns ranks=[1,6,7,8,10,5,3,0,9,4,2] 2024-08-19T10:43:23.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f945c069430 addr v1:172.21.15.190:6789/0 2024-08-19T10:43:23.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c0a4390 0x7f945c13ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:23.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 10 monclient(hunting): picked mon.noname-d con 0x7f945c0a4390 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:23.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a47b0 0x7f945c13dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:23.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 10 monclient(hunting): picked mon.noname-i con 0x7f945c0a47b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:23.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f945c09e260 con 0x7f945c069430 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f945c09e4e0 con 0x7f945c0a4390 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f945c09db70 con 0x7f945c0a47b0 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9463774640 10 monclient(hunting): authenticate will time out at 1248.789071s 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461510640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f945c069430 legacy=0x7f945c0a3c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a47b0 0x7f945c13dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a47b0 0x7f945c13dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34428/0 (socket says 172.21.15.190:34428) 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 1 -- 172.21.15.190:0/3268130110 learned_addr learned my addr 172.21.15.190:0/3268130110 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 10 monclient(hunting): get_auth_request con 0x7f945c0a47b0 auth_method 0 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:23.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9460d0f640 1 -- 172.21.15.190:0/3268130110 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c0a4390 msgr2=0x7f945c13ba70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9453fff640 1 -- 172.21.15.190:0/3268130110 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9460d0f640 1 --2- 172.21.15.190:0/3268130110 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c0a4390 0x7f945c13ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 10 monclient(hunting): handle_auth_done global_id 15055 payload 274 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 1 -- 172.21.15.190:0/3268130110 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c0a4390 msgr2=0x7f945c13ba70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 1 --2- 172.21.15.190:0/3268130110 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c0a4390 0x7f945c13ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 1 -- 172.21.15.190:0/3268130110 >> v1:172.21.15.190:6789/0 conn(0x7f945c069430 legacy=0x7f945c0a3c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 10 monclient: _finish_hunting 0 2024-08-19T10:43:23.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 1 monclient: found mon.noname-i 2024-08-19T10:43:23.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:23.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:23.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.219+0000 7f9461d11640 1 -- 172.21.15.190:0/3268130110 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c0a0140 con 0x7f945c0a47b0 2024-08-19T10:43:23.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9461d11640 1 -- 172.21.15.190:0/3268130110 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a47b0 msgr2=0x7f945c13dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:23.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9461d11640 1 --2- 172.21.15.190:0/3268130110 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a47b0 0x7f945c13dfc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9454009e80 tx=0x7f945401c930 comp rx=0 tx=0).ready entity=mon.7 client_cookie=9b8419448d1fc65d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:23.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 1 -- 172.21.15.190:0/3268130110 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94540091b0 con 0x7f945c0a47b0 2024-08-19T10:43:23.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:23.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:23.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 10 monclient: dump: 2024-08-19T10:43:23.228 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:23.228 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:23.228 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:23.228 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 10 monclient: _finish_auth 0 2024-08-19T10:43:23.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 1 -- 172.21.15.190:0/3268130110 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94540a5980 con 0x7f945c0a47b0 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 5 monclient: authenticate success, global_id 15055 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 1 -- 172.21.15.190:0/3268130110 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94540ad3f0 con 0x7f945c0a47b0 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 4 set_mon_vals no callback set 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9453fff640 10 monclient: dump: 2024-08-19T10:43:23.230 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:23.231 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 10 monclient: shutdown 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 1 -- 172.21.15.190:0/3268130110 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a47b0 msgr2=0x7f945c13dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 1 --2- 172.21.15.190:0/3268130110 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a47b0 0x7f945c13dfc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9454009e80 tx=0x7f945401c930 comp rx=0 tx=0).stop 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 1 -- 172.21.15.190:0/3268130110 shutdown_connections 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 1 --2- 172.21.15.190:0/3268130110 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a47b0 0x7f945c13dfc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 1 --2- 172.21.15.190:0/3268130110 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c0a4390 0x7f945c13ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 1 -- 172.21.15.190:0/3268130110 >> 172.21.15.190:0/3268130110 conn(0x7f945c0a4f40 msgr2=0x7f945c0a5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 1 -- 172.21.15.190:0/3268130110 shutdown_connections 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 1 -- 172.21.15.190:0/3268130110 wait complete. 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 10 monclient: build_initial_monmap 2024-08-19T10:43:23.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 10 monclient: monmap: 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:23.233 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 1 Processor -- start 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 1 -- start start 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.223+0000 7f9463774640 10 monclient: init 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient: _add_conns ranks=[6,7,3,1,5,4,0,8,2] 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c069430 0x7f945c1d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient(hunting): picked mon.noname-d con 0x7f945c069430 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a4390 0x7f945c13af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient(hunting): picked mon.noname-i con 0x7f945c0a4390 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c0a47b0 0x7f945c122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:23.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient(hunting): picked mon.noname-g con 0x7f945c0a47b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f945c09ca60 con 0x7f945c0a47b0 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f945c069810 con 0x7f945c069430 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f945c09cbe0 con 0x7f945c0a4390 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient(hunting): authenticate will time out at 1248.796174s 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a4390 0x7f945c13af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9461d11640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c0a47b0 0x7f945c122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a4390 0x7f945c13af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34444/0 (socket says 172.21.15.190:34444) 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9461d11640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c0a47b0 0x7f945c122cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:51418/0 (socket says 172.21.15.190:51418) 2024-08-19T10:43:23.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 1 -- 172.21.15.190:0/1544938598 learned_addr learned my addr 172.21.15.190:0/1544938598 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 10 monclient(hunting): get_auth_request con 0x7f945c0a4390 auth_method 0 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9461510640 1 -- 172.21.15.190:0/1544938598 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c069430 msgr2=0x7f945c1d0d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9461d11640 10 monclient(hunting): get_auth_request con 0x7f945c0a47b0 auth_method 0 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9461d11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9461d11640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9461d11640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9461510640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c069430 0x7f945c1d0d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:23.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9461d11640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9461d11640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9461d11640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 10 monclient(hunting): handle_auth_done global_id 15064 payload 824 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 1 -- 172.21.15.190:0/1544938598 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c069430 msgr2=0x7f945c1d0d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c069430 0x7f945c1d0d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 1 -- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c0a47b0 msgr2=0x7f945c122cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c0a47b0 0x7f945c122cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 10 monclient: _finish_hunting 0 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 1 monclient: found mon.noname-i 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:23.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 1 -- 172.21.15.190:0/1544938598 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c0a6170 con 0x7f945c0a4390 2024-08-19T10:43:23.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 1 -- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a4390 msgr2=0x7f945c13af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9460d0f640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a4390 0x7f945c13af20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f944c00b550 tx=0x7f944c00ba10 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 1 -- 172.21.15.190:0/1544938598 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f944c013020 con 0x7f945c0a4390 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 10 monclient: dump: 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:23.240 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 10 monclient: _finish_auth 0 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 1 -- 172.21.15.190:0/1544938598 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944c003dd0 con 0x7f945c0a4390 2024-08-19T10:43:23.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 1 -- 172.21.15.190:0/1544938598 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f944c00f400 con 0x7f945c0a4390 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9451ffb640 10 monclient: dump: 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:23.242 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f94537fe640 4 set_mon_vals no callback set 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 5 monclient: authenticate success, global_id 15064 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient: _renew_subs 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 1 -- 172.21.15.190:0/1544938598 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945c1265e0 con 0x7f945c0a4390 2024-08-19T10:43:23.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient: _renew_subs 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.227+0000 7f9463774640 1 -- 172.21.15.190:0/1544938598 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f945c0a7520 con 0x7f945c0a4390 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.231+0000 7f9463774640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.231+0000 7f9463774640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.231+0000 7f9463774640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.231+0000 7f9451ffb640 1 -- 172.21.15.190:0/1544938598 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f944c016020 con 0x7f945c0a4390 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.231+0000 7f9463774640 1 -- 172.21.15.190:0/1544938598 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f945c0a3c80 con 0x7f945c0a4390 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.231+0000 7f9451ffb640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f944003be00 0x7f944003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.231+0000 7f9451ffb640 1 -- 172.21.15.190:0/1544938598 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(167..167 src has 1..167) ==== 9954+0+0 (secure 0 0 0) 0x7f944c04fac0 con 0x7f945c0a4390 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.231+0000 7f9461510640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f944003be00 0x7f944003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.231+0000 7f9461510640 10 monclient: get_auth_request con 0x7f944003be00 auth_method 0 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.231+0000 7f9461510640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f944003be00 0x7f944003e2c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9458006fd0 tx=0x7f9458006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:23.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:23.235+0000 7f9460d0f640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a4390 0x7f945c13af20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f944c00b550 tx=0x7f944c00ba10 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f944c01d020 mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) 2024-08-19T10:43:24.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.063+0000 7f9451ffb640 1 -- 172.21.15.190:0/1544938598 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f944c01d020 con 0x7f945c0a4390 2024-08-19T10:43:24.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.063+0000 7f9451ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:24.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.063+0000 7f9451ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:24.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.251+0000 7f9463774640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:24.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.251+0000 7f9463774640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:24.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.251+0000 7f9463774640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:24.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.251+0000 7f9463774640 1 -- 172.21.15.190:0/1544938598 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f945c124560 con 0x7f945c0a4390 2024-08-19T10:43:24.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.255+0000 7f9451ffb640 1 -- 172.21.15.190:0/1544938598 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2378 (secure 0 0 0) 0x7f945c0a6170 con 0x7f945c0a4390 2024-08-19T10:43:24.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.255+0000 7f9451ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:24.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.255+0000 7f9451ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:24.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:24.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:24.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:24.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 -- 172.21.15.190:0/1544938598 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f945c0008e0 con 0x7f945c0a4390 2024-08-19T10:43:24.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9451ffb640 1 -- 172.21.15.190:0/1544938598 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2378 (secure 0 0 0) 0x7f944c017ce0 con 0x7f945c0a4390 2024-08-19T10:43:24.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9451ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:24.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9451ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:24.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 -- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f944003be00 msgr2=0x7f944003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:24.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f944003be00 0x7f944003e2c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9458006fd0 tx=0x7f9458006d40 comp rx=0 tx=0).stop 2024-08-19T10:43:24.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 10 monclient: shutdown 2024-08-19T10:43:24.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:24.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 -- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a4390 msgr2=0x7f945c13af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:24.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a4390 0x7f945c13af20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f944c00b550 tx=0x7f944c00ba10 comp rx=0 tx=0).stop 2024-08-19T10:43:24.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 -- 172.21.15.190:0/1544938598 shutdown_connections 2024-08-19T10:43:24.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f944003be00 0x7f944003e2c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:24.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c0a47b0 0x7f945c122cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:24.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f945c0a4390 0x7f945c13af20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:24.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 --2- 172.21.15.190:0/1544938598 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f945c069430 0x7f945c1d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:24.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 -- 172.21.15.190:0/1544938598 >> 172.21.15.190:0/1544938598 conn(0x7f945c0a4f40 msgr2=0x7f945c0a2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:24.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 -- 172.21.15.190:0/1544938598 shutdown_connections 2024-08-19T10:43:24.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.367+0000 7f9463774640 1 -- 172.21.15.190:0/1544938598 wait complete. 2024-08-19T10:43:24.386 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:24.386 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:24.387 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush tree --show-shadow 2024-08-19T10:43:24.387 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 'osd\.0' 2024-08-19T10:43:24.387 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep globster 2024-08-19T10:43:24.388 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep '12\.' 2024-08-19T10:43:24.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient: get_monmap_and_config 2024-08-19T10:43:24.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient: build_initial_monmap 2024-08-19T10:43:24.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient: monmap: 2024-08-19T10:43:24.504 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:24.504 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:24.504 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:24.504424+0000 2024-08-19T10:43:24.504 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:24.504424+0000 2024-08-19T10:43:24.504 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:24.504 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:24.504 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:24.504 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:24.504 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:24.504 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:24.505 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:24.505 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:24.505 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:24.505 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:24.505 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:24.505 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:24.505 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:24.505 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:24.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 1 Processor -- start 2024-08-19T10:43:24.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 1 -- start start 2024-08-19T10:43:24.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient: init 2024-08-19T10:43:24.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:24.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient: _add_conns ranks=[6,2,10,8,0,5,9,1,4,3,7] 2024-08-19T10:43:24.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84a00a6740 0x7f84a00a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:24.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient(hunting): picked mon.noname-d con 0x7f84a00a6740 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:24.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84a00a5bc0 0x7f84a00a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:24.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient(hunting): picked mon.noname-b con 0x7f84a00a5bc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:24.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f84a00a7060 0x7f84a0139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:24.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient(hunting): picked mon.noname-f con 0x7f84a00a7060 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:24.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:24.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f84a00a4c20 con 0x7f84a00a7060 2024-08-19T10:43:24.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:24.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f84a00a3530 con 0x7f84a00a5bc0 2024-08-19T10:43:24.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:24.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f84a00a3740 con 0x7f84a00a6740 2024-08-19T10:43:24.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:24.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f84a53fc640 10 monclient(hunting): authenticate will time out at 1250.074202s 2024-08-19T10:43:24.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849f7fe640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f84a00a7060 msgr2=0x7f84a0139590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:24.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849f7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f84a00a7060 0x7f84a0139590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:24.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849effd640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84a00a6740 msgr2=0x7f84a00a5680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:24.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849effd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84a00a6740 0x7f84a00a5680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:24.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84a00a5bc0 0x7f84a00a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:24.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84a00a5bc0 0x7f84a00a6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:48928/0 (socket says 172.21.15.190:48928) 2024-08-19T10:43:24.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 1 -- 172.21.15.190:0/1331299181 learned_addr learned my addr 172.21.15.190:0/1331299181 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:24.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 10 monclient(hunting): get_auth_request con 0x7f84a00a5bc0 auth_method 0 2024-08-19T10:43:24.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:24.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:24.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:24.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:24.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:24.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:24.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 10 monclient(hunting): handle_auth_done global_id 5861 payload 274 2024-08-19T10:43:24.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 1 -- 172.21.15.190:0/1331299181 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84a00a6740 msgr2=0x7f84a00a5680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:24.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 1 --2- 172.21.15.190:0/1331299181 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84a00a6740 0x7f84a00a5680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:24.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 1 -- 172.21.15.190:0/1331299181 >> v2:172.21.15.190:3300/0 conn(0x7f84a00a7060 msgr2=0x7f84a0139590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:24.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 1 --2- 172.21.15.190:0/1331299181 >> v2:172.21.15.190:3300/0 conn(0x7f84a00a7060 0x7f84a0139590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:24.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:43:24.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 1 monclient: found mon.noname-b 2024-08-19T10:43:24.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:24.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:24.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.503+0000 7f849e7fc640 1 -- 172.21.15.190:0/1331299181 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a00a3240 con 0x7f84a00a5bc0 2024-08-19T10:43:24.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.507+0000 7f849e7fc640 1 -- 172.21.15.190:0/1331299181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84a00a5bc0 msgr2=0x7f84a00a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:24.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.507+0000 7f849e7fc640 1 --2- 172.21.15.190:0/1331299181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84a00a5bc0 0x7f84a00a6b20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8494009870 tx=0x7f849401c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a6672feae7627db server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:24.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 1 -- 172.21.15.190:0/1331299181 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84940ab870 con 0x7f84a00a5bc0 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 10 monclient: dump: 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:24.518 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:24.519 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:24.519 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:24.519 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:24.519 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:24.519 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:24.519 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:24.519 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:24.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 10 monclient: _finish_auth 0 2024-08-19T10:43:24.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:43:24.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:24.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 5 monclient: authenticate success, global_id 5861 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 1 -- 172.21.15.190:0/1331299181 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84940b0070 con 0x7f84a00a5bc0 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 4 set_mon_vals no callback set 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 1 -- 172.21.15.190:0/1331299181 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f84940aba10 con 0x7f84a00a5bc0 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849dffb640 10 monclient: dump: 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:24.520 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient: shutdown 2024-08-19T10:43:24.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 -- 172.21.15.190:0/1331299181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84a00a5bc0 msgr2=0x7f84a00a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 --2- 172.21.15.190:0/1331299181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84a00a5bc0 0x7f84a00a6b20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8494009870 tx=0x7f849401c460 comp rx=0 tx=0).stop 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 -- 172.21.15.190:0/1331299181 shutdown_connections 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 --2- 172.21.15.190:0/1331299181 >> v2:172.21.15.190:3300/0 conn(0x7f84a00a7060 0x7f84a0139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 --2- 172.21.15.190:0/1331299181 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f84a00a5bc0 0x7f84a00a6b20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 --2- 172.21.15.190:0/1331299181 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f84a00a6740 0x7f84a00a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 -- 172.21.15.190:0/1331299181 >> 172.21.15.190:0/1331299181 conn(0x7f84a00abd40 msgr2=0x7f84a00ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 -- 172.21.15.190:0/1331299181 shutdown_connections 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 -- 172.21.15.190:0/1331299181 wait complete. 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient: build_initial_monmap 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient: monmap: 2024-08-19T10:43:24.522 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:24.523 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 Processor -- start 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 -- start start 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient: init 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient: _add_conns ranks=[7,8,4,6,1,5,3,2,0] 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84a00a6740 0x7f84a013eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient(hunting): picked mon.noname-i con 0x7f84a00a6740 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f84a00a7060 0x7f84a00a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient(hunting): picked mon.noname-e con 0x7f84a00a7060 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84a00a2620 0x7f84a009ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:24.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient(hunting): picked mon.noname-c con 0x7f84a00a2620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f84a0143450 con 0x7f84a00a2620 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f84a01432d0 con 0x7f84a00a6740 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f84a01435d0 con 0x7f84a00a7060 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f84a53fc640 10 monclient(hunting): authenticate will time out at 1250.082502s 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84a00a6740 0x7f84a013eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849f7fe640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84a00a2620 msgr2=0x7f84a009ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84a00a6740 0x7f84a013eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34452/0 (socket says 172.21.15.190:34452) 2024-08-19T10:43:24.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 1 -- 172.21.15.190:0/507613014 learned_addr learned my addr 172.21.15.190:0/507613014 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 10 monclient(hunting): get_auth_request con 0x7f84a00a6740 auth_method 0 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849f7fe640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84a00a2620 0x7f84a009ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849e7fc640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f84a00a7060 0x7f84a00a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849e7fc640 10 monclient(hunting): get_auth_request con 0x7f84a00a7060 auth_method 0 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:24.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:24.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 10 monclient(hunting): handle_auth_done global_id 15073 payload 824 2024-08-19T10:43:24.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 1 -- 172.21.15.190:0/507613014 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f84a00a7060 msgr2=0x7f84a00a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:24.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f84a00a7060 0x7f84a00a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:24.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 1 -- 172.21.15.190:0/507613014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84a00a2620 msgr2=0x7f84a009ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:24.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84a00a2620 0x7f84a009ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:24.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 10 monclient: _finish_hunting 0 2024-08-19T10:43:24.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 1 monclient: found mon.noname-i 2024-08-19T10:43:24.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:24.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:24.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.511+0000 7f849effd640 1 -- 172.21.15.190:0/507613014 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a013f500 con 0x7f84a00a6740 2024-08-19T10:43:24.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f849effd640 1 -- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84a00a6740 msgr2=0x7f84a013eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f849effd640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84a00a6740 0x7f84a013eb90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f849000b560 tx=0x7f849000ba20 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 1 -- 172.21.15.190:0/507613014 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8490013020 con 0x7f84a00a6740 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f849effd640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84a00a6740 0x7f84a013eb90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f849000b560 tx=0x7f849000ba20 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f8490003de0 config(0 keys) 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 10 monclient: dump: 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:24.528 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 10 monclient: _finish_auth 0 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 10 monclient: _check_auth_tickets 2024-08-19T10:43:24.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:24.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 1 -- 172.21.15.190:0/507613014 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8490003de0 con 0x7f84a00a6740 2024-08-19T10:43:24.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84a53fc640 5 monclient: authenticate success, global_id 15073 2024-08-19T10:43:24.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84a53fc640 10 monclient: _renew_subs 2024-08-19T10:43:24.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84a53fc640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:24.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84a53fc640 1 -- 172.21.15.190:0/507613014 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a009e170 con 0x7f84a00a6740 2024-08-19T10:43:24.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:24.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f849d7fa640 4 set_mon_vals no callback set 2024-08-19T10:43:24.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84a53fc640 10 monclient: _renew_subs 2024-08-19T10:43:24.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84a53fc640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:24.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84a53fc640 1 -- 172.21.15.190:0/507613014 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a009f0d0 con 0x7f84a00a6740 2024-08-19T10:43:24.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 1 -- 172.21.15.190:0/507613014 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f849000f460 con 0x7f84a00a6740 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 10 monclient: dump: 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:24.531 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:24.532 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:24.532 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:24.532 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:24.532 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:24.532 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:24.532 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:24.532 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:24.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 1 -- 172.21.15.190:0/507613014 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8490016020 con 0x7f84a00a6740 2024-08-19T10:43:24.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84a53fc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:24.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84a53fc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:24.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84a53fc640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:24.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84a53fc640 1 -- 172.21.15.190:0/507613014 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a00a5680 con 0x7f84a00a6740 2024-08-19T10:43:24.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f84877fe640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848c03be50 0x7f848c03e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:24.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f849e7fc640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848c03be50 0x7f848c03e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:24.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.515+0000 7f849e7fc640 10 monclient: get_auth_request con 0x7f848c03be50 auth_method 0 2024-08-19T10:43:24.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.519+0000 7f849e7fc640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848c03be50 0x7f848c03e310 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8494009870 tx=0x7f8494009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:24.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:24.519+0000 7f849effd640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84a00a6740 0x7f84a013eb90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f849000b560 tx=0x7f849000ba20 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f849001d020 mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) 2024-08-19T10:43:25.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.139+0000 7f84877fe640 1 -- 172.21.15.190:0/507613014 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(167..167 src has 1..167) ==== 9954+0+0 (secure 0 0 0) 0x7f8490050180 con 0x7f84a00a6740 2024-08-19T10:43:25.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.347+0000 7f84877fe640 1 -- 172.21.15.190:0/507613014 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f849001d020 con 0x7f84a00a6740 2024-08-19T10:43:25.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.347+0000 7f84877fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:25.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.347+0000 7f84877fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:25.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.527+0000 7f84a53fc640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:25.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.527+0000 7f84a53fc640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:25.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.527+0000 7f84a53fc640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:25.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.527+0000 7f84a53fc640 1 -- 172.21.15.190:0/507613014 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f84a01375a0 con 0x7f84a00a6740 2024-08-19T10:43:25.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.531+0000 7f84877fe640 1 -- 172.21.15.190:0/507613014 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2378 (secure 0 0 0) 0x7f849004d730 con 0x7f84a00a6740 2024-08-19T10:43:25.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.531+0000 7f84877fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:25.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.531+0000 7f84877fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:25.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.635+0000 7f84a53fc640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:25.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.635+0000 7f84a53fc640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:25.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.635+0000 7f84a53fc640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:25.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.635+0000 7f84a53fc640 1 -- 172.21.15.190:0/507613014 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f84a00008e0 con 0x7f84a00a6740 2024-08-19T10:43:25.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.635+0000 7f84877fe640 1 -- 172.21.15.190:0/507613014 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v167) ==== 83+0+2378 (secure 0 0 0) 0x7f8490005250 con 0x7f84a00a6740 2024-08-19T10:43:25.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.635+0000 7f84877fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:25.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.635+0000 7f84877fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 -- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848c03be50 msgr2=0x7f848c03e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848c03be50 0x7f848c03e310 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8494009870 tx=0x7f8494009040 comp rx=0 tx=0).stop 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 10 monclient: shutdown 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 -- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84a00a6740 msgr2=0x7f84a013eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84a00a6740 0x7f84a013eb90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f849000b560 tx=0x7f849000ba20 comp rx=0 tx=0).stop 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 -- 172.21.15.190:0/507613014 shutdown_connections 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f84a00a2620 0x7f84a009ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f848c03be50 0x7f848c03e310 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f84a00a7060 0x7f84a00a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 --2- 172.21.15.190:0/507613014 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f84a00a6740 0x7f84a013eb90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:25.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 -- 172.21.15.190:0/507613014 >> 172.21.15.190:0/507613014 conn(0x7f84a00abd40 msgr2=0x7f84a01366d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:25.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 -- 172.21.15.190:0/507613014 shutdown_connections 2024-08-19T10:43:25.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.639+0000 7f84a53fc640 1 -- 172.21.15.190:0/507613014 wait complete. 2024-08-19T10:43:25.656 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:25.656 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 10.50000 11.50000 12.50000 osd.0 2024-08-19T10:43:25.657 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight-compat osd.0 7.5 2024-08-19T10:43:25.778 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.775+0000 7efec4c3b640 10 monclient: get_monmap_and_config 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.775+0000 7efec4c3b640 10 monclient: build_initial_monmap 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient: monmap: 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:25.779558+0000 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:25.779558+0000 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:25.779 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:25.780 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:25.780 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:25.780 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:25.780 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:25.780 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:25.780 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:25.780 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:25.780 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:25.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 1 Processor -- start 2024-08-19T10:43:25.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 1 -- start start 2024-08-19T10:43:25.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient: init 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient: _add_conns ranks=[8,9,7,2,4,5,3,10,6,0,1] 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 0x7efec0139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient(hunting): picked mon.noname-e con 0x7efec01396a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efec0139fc0 0x7efec00a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient(hunting): picked mon.noname-a con 0x7efec0139fc0 addr v2:172.21.15.89:3300/0 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efec00a6e20 0x7efec00a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient(hunting): picked mon.noname-i con 0x7efec00a6e20 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7efec00a4f20 con 0x7efec0139fc0 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:25.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7efec00a83e0 con 0x7efec00a6e20 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7efec00a2f30 con 0x7efec01396a0 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efec4c3b640 10 monclient(hunting): authenticate will time out at 1251.349422s 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efec00a6e20 0x7efec00a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efec00a6e20 0x7efec00a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:59316/0 (socket says 172.21.15.190:59316) 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 1 -- 172.21.15.190:0/1467143188 learned_addr learned my addr 172.21.15.190:0/1467143188 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 10 monclient(hunting): get_auth_request con 0x7efec00a6e20 auth_method 0 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:25.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebdd9b640 1 --2- 172.21.15.190:0/1467143188 >> v2:172.21.15.89:3300/0 conn(0x7efec0139fc0 0x7efec00a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebe59c640 1 --2- 172.21.15.190:0/1467143188 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 0x7efec0139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 10 monclient(hunting): handle_auth_done global_id 15082 payload 274 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 1 -- 172.21.15.190:0/1467143188 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 msgr2=0x7efec0139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 1 --2- 172.21.15.190:0/1467143188 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 0x7efec0139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 1 -- 172.21.15.190:0/1467143188 >> v2:172.21.15.89:3300/0 conn(0x7efec0139fc0 msgr2=0x7efec00a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 1 --2- 172.21.15.190:0/1467143188 >> v2:172.21.15.89:3300/0 conn(0x7efec0139fc0 0x7efec00a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 1 monclient: found mon.noname-i 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:25.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.779+0000 7efebed9d640 1 -- 172.21.15.190:0/1467143188 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec00a35e0 con 0x7efec00a6e20 2024-08-19T10:43:25.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebed9d640 1 -- 172.21.15.190:0/1467143188 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efec00a6e20 msgr2=0x7efec00a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:25.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebed9d640 1 --2- 172.21.15.190:0/1467143188 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efec00a6e20 0x7efec00a7220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efeb00098a0 tx=0x7efeb001c400 comp rx=0 tx=0).ready entity=mon.7 client_cookie=b9eeef51838aadf8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:25.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 1 -- 172.21.15.190:0/1467143188 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efeb00b0070 con 0x7efec00a6e20 2024-08-19T10:43:25.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:25.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:25.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 10 monclient: dump: 2024-08-19T10:43:25.789 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:25.789 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:25.789 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:25.789 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:25.789 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 10 monclient: _finish_auth 0 2024-08-19T10:43:25.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 10 monclient: _check_auth_tickets 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 1 -- 172.21.15.190:0/1467143188 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb00aa810 con 0x7efec00a6e20 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 1 -- 172.21.15.190:0/1467143188 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efeb00a93e0 con 0x7efec00a6e20 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 5 monclient: authenticate success, global_id 15082 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 4 set_mon_vals no callback set 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efebd59a640 10 monclient: dump: 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:25.791 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:25.792 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 10 monclient: shutdown 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 1 -- 172.21.15.190:0/1467143188 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efec00a6e20 msgr2=0x7efec00a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 1 --2- 172.21.15.190:0/1467143188 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efec00a6e20 0x7efec00a7220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efeb00098a0 tx=0x7efeb001c400 comp rx=0 tx=0).stop 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 1 -- 172.21.15.190:0/1467143188 shutdown_connections 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 1 --2- 172.21.15.190:0/1467143188 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efec00a6e20 0x7efec00a7220 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 1 --2- 172.21.15.190:0/1467143188 >> v2:172.21.15.89:3300/0 conn(0x7efec0139fc0 0x7efec00a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 1 --2- 172.21.15.190:0/1467143188 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 0x7efec0139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 1 -- 172.21.15.190:0/1467143188 >> 172.21.15.190:0/1467143188 conn(0x7efec00abee0 msgr2=0x7efec00ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 1 -- 172.21.15.190:0/1467143188 shutdown_connections 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 1 -- 172.21.15.190:0/1467143188 wait complete. 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 10 monclient: build_initial_monmap 2024-08-19T10:43:25.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.783+0000 7efec4c3b640 10 monclient: monmap: 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:25.794 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 1 Processor -- start 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 1 -- start start 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 10 monclient: init 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 10 monclient: _add_conns ranks=[0,8,2,3,7,4,6,1,5] 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec00a6e20 0x7efec013f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 10 monclient(hunting): picked mon.noname-a con 0x7efec00a6e20 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 0x7efec00a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 10 monclient(hunting): picked mon.noname-e con 0x7efec01396a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efec0139fc0 0x7efec00a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:25.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 10 monclient(hunting): picked mon.noname-b con 0x7efec0139fc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efec013a3a0 con 0x7efec00a6e20 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7efec0143390 con 0x7efec0139fc0 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7efec00a28f0 con 0x7efec01396a0 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efec4c3b640 10 monclient(hunting): authenticate will time out at 1251.356600s 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 0x7efec00a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebed9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efec0139fc0 0x7efec00a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebe59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec00a6e20 0x7efec013f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 0x7efec00a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:48992/0 (socket says 172.21.15.190:48992) 2024-08-19T10:43:25.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 1 -- 172.21.15.190:0/3787683484 learned_addr learned my addr 172.21.15.190:0/3787683484 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 10 monclient(hunting): get_auth_request con 0x7efec01396a0 auth_method 0 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebe59c640 10 monclient(hunting): get_auth_request con 0x7efec00a6e20 auth_method 0 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebe59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebe59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebed9d640 10 monclient(hunting): get_auth_request con 0x7efec0139fc0 auth_method 0 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:25.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebe59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebe59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebe59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 10 monclient(hunting): handle_auth_done global_id 5750 payload 824 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 1 -- 172.21.15.190:0/3787683484 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efec0139fc0 msgr2=0x7efec00a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efec0139fc0 0x7efec00a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 1 -- 172.21.15.190:0/3787683484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec00a6e20 msgr2=0x7efec013f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:25.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec00a6e20 0x7efec013f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:25.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 10 monclient: _finish_hunting 0 2024-08-19T10:43:25.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 1 monclient: found mon.noname-e 2024-08-19T10:43:25.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:25.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:43:25.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.787+0000 7efebdd9b640 1 -- 172.21.15.190:0/3787683484 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec009f090 con 0x7efec01396a0 2024-08-19T10:43:25.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efebdd9b640 1 -- 172.21.15.190:0/3787683484 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 msgr2=0x7efec00a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:25.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efebdd9b640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 0x7efec00a1e70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efea800c7f0 tx=0x7efea800ccb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:25.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 1 -- 172.21.15.190:0/3787683484 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efea8015070 con 0x7efec01396a0 2024-08-19T10:43:25.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:25.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 10 monclient: dump: 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:25.803 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 1 -- 172.21.15.190:0/3787683484 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea8007c20 con 0x7efec01396a0 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efec4c3b640 5 monclient: authenticate success, global_id 5750 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efec4c3b640 10 monclient: _renew_subs 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efec4c3b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efec4c3b640 1 -- 172.21.15.190:0/3787683484 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec013faa0 con 0x7efec01396a0 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 1 -- 172.21.15.190:0/3787683484 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efea8007dd0 con 0x7efec01396a0 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:25.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 10 monclient: dump: 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:25.805 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:25.806 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:25.806 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:25.806 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:25.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efebcd99640 4 set_mon_vals no callback set 2024-08-19T10:43:25.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efec4c3b640 10 monclient: _renew_subs 2024-08-19T10:43:25.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efec4c3b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:25.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efec4c3b640 1 -- 172.21.15.190:0/3787683484 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7efec009e130 con 0x7efec01396a0 2024-08-19T10:43:25.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efec4c3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:25.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efec4c3b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:25.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efec4c3b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:25.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efec4c3b640 1 -- 172.21.15.190:0/3787683484 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec0139a80 con 0x7efec01396a0 2024-08-19T10:43:25.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 1 -- 172.21.15.190:0/3787683484 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efea8018070 con 0x7efec01396a0 2024-08-19T10:43:25.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efea003be30 0x7efea003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:25.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efeb6ffd640 1 -- 172.21.15.190:0/3787683484 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(167..167 src has 1..167) ==== 9954+0+0 (secure 0 0 0) 0x7efea804ff10 con 0x7efec01396a0 2024-08-19T10:43:25.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efebe59c640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efea003be30 0x7efea003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:25.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.791+0000 7efebe59c640 10 monclient: get_auth_request con 0x7efea003be30 auth_method 0 2024-08-19T10:43:25.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.795+0000 7efebe59c640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efea003be30 0x7efea003e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efeb800f510 tx=0x7efeb8007790 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:25.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.803+0000 7efeb6ffd640 1 -- 172.21.15.190:0/3787683484 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7efea801f030 con 0x7efec01396a0 2024-08-19T10:43:25.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.803+0000 7efeb6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:25.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.803+0000 7efeb6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:25.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.979+0000 7efec4c3b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-08-19T10:43:25.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.979+0000 7efec4c3b640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-08-19T10:43:25.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.979+0000 7efec4c3b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:25.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:25.979+0000 7efec4c3b640 1 -- 172.21.15.190:0/3787683484 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]} v 0) -- 0x7efec00a7320 con 0x7efec01396a0 2024-08-19T10:43:26.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:26.071+0000 7efeb6ffd640 1 -- 172.21.15.190:0/3787683484 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}]=0 v168) ==== 118+0+0 (secure 0 0 0) 0x7efec009f090 con 0x7efec01396a0 2024-08-19T10:43:26.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:26.071+0000 7efeb6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-08-19T10:43:26.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:26.071+0000 7efeb6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:26.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:26.183+0000 7efec4c3b640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-08-19T10:43:26.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:26.183+0000 7efec4c3b640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-08-19T10:43:26.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:26.183+0000 7efec4c3b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:26.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:26.183+0000 7efec4c3b640 1 -- 172.21.15.190:0/3787683484 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]} v 0) -- 0x7efec009f2d0 con 0x7efec01396a0 2024-08-19T10:43:27.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.219+0000 7efeb6ffd640 1 -- 172.21.15.190:0/3787683484 <== mon.8 v2:172.21.15.89:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}]=0 v169) ==== 118+0+0 (secure 0 0 0) 0x7efea804c510 con 0x7efec01396a0 2024-08-19T10:43:27.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.219+0000 7efeb6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight-compat", "item": "osd.0", "weight": [7.5]}] 2024-08-19T10:43:27.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.219+0000 7efeb6ffd640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.223+0000 7efec4c3b640 1 -- 172.21.15.190:0/3787683484 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efea003be30 msgr2=0x7efea003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.223+0000 7efec4c3b640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efea003be30 0x7efea003e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efeb800f510 tx=0x7efeb8007790 comp rx=0 tx=0).stop 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.223+0000 7efec4c3b640 10 monclient: shutdown 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.223+0000 7efec4c3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.223+0000 7efec4c3b640 1 -- 172.21.15.190:0/3787683484 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 msgr2=0x7efec00a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.223+0000 7efec4c3b640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 0x7efec00a1e70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efea800c7f0 tx=0x7efea800ccb0 comp rx=0 tx=0).stop 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.227+0000 7efec4c3b640 1 -- 172.21.15.190:0/3787683484 shutdown_connections 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.227+0000 7efec4c3b640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efec0139fc0 0x7efec00a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.227+0000 7efec4c3b640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efea003be30 0x7efea003e2f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.227+0000 7efec4c3b640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efec01396a0 0x7efec00a1e70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.227+0000 7efec4c3b640 1 --2- 172.21.15.190:0/3787683484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efec00a6e20 0x7efec013f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:27.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.227+0000 7efec4c3b640 1 -- 172.21.15.190:0/3787683484 >> 172.21.15.190:0/3787683484 conn(0x7efec00abee0 msgr2=0x7efec0134f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:27.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.227+0000 7efec4c3b640 1 -- 172.21.15.190:0/3787683484 shutdown_connections 2024-08-19T10:43:27.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.227+0000 7efec4c3b640 1 -- 172.21.15.190:0/3787683484 wait complete. 2024-08-19T10:43:27.245 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight cool osd.0 8.5 2024-08-19T10:43:27.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient: get_monmap_and_config 2024-08-19T10:43:27.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient: build_initial_monmap 2024-08-19T10:43:27.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient: monmap: 2024-08-19T10:43:27.368 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:27.368 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:27.368 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:27.369080+0000 2024-08-19T10:43:27.368 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:27.369080+0000 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:27.369 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:27.370 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 1 Processor -- start 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 1 -- start start 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient: init 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient: _add_conns ranks=[8,7,0,9,4,1,10,2,5,3,6] 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee2c139990 0x7fee2c139d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient(hunting): picked mon.noname-e con 0x7fee2c139990 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee2c13a2b0 0x7fee2c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient(hunting): picked mon.noname-i con 0x7fee2c13a2b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fee2c09fae0 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee2c09b590 con 0x7fee2c09fae0 2024-08-19T10:43:27.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fee2c069d20 con 0x7fee2c13a2b0 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fee2c0a0460 con 0x7fee2c139990 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee33050640 10 monclient(hunting): authenticate will time out at 1252.938915s 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee2c13a2b0 0x7fee2c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee2c13a2b0 0x7fee2c06c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:59320/0 (socket says 172.21.15.190:59320) 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 1 -- 172.21.15.190:0/923681910 learned_addr learned my addr 172.21.15.190:0/923681910 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 10 monclient(hunting): get_auth_request con 0x7fee2c13a2b0 auth_method 0 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:27.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee30dec640 1 --2- 172.21.15.190:0/923681910 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee2c139990 0x7fee2c139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee30dec640 10 monclient(hunting): get_auth_request con 0x7fee2c139990 auth_method 0 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee30dec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee30dec640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee30dec640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 10 monclient(hunting): handle_auth_done global_id 15091 payload 274 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 1 -- 172.21.15.190:0/923681910 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee2c139990 msgr2=0x7fee2c139d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 1 --2- 172.21.15.190:0/923681910 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee2c139990 0x7fee2c139d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:27.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 1 -- 172.21.15.190:0/923681910 >> v1:172.21.15.89:6789/0 conn(0x7fee2c09fae0 legacy=0x7fee2c06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:27.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:43:27.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 1 monclient: found mon.noname-i 2024-08-19T10:43:27.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:27.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:27.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.367+0000 7fee2bfff640 1 -- 172.21.15.190:0/923681910 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c06daf0 con 0x7fee2c13a2b0 2024-08-19T10:43:27.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2bfff640 1 -- 172.21.15.190:0/923681910 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee2c13a2b0 msgr2=0x7fee2c06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:27.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2bfff640 1 --2- 172.21.15.190:0/923681910 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee2c13a2b0 0x7fee2c06c8d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fee200098a0 tx=0x7fee2001c4d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=aa82f0a9cb030996 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:27.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 1 -- 172.21.15.190:0/923681910 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee200b2070 con 0x7fee2c13a2b0 2024-08-19T10:43:27.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:27.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:27.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 10 monclient: dump: 2024-08-19T10:43:27.379 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:27.379 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:27.379 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:27.380 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 5 monclient: authenticate success, global_id 15091 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 1 -- 172.21.15.190:0/923681910 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee200ab7b0 con 0x7fee2c13a2b0 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 1 -- 172.21.15.190:0/923681910 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee200ab960 con 0x7fee2c13a2b0 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2b7fe640 10 monclient: dump: 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:27.381 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:27.382 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 4 set_mon_vals no callback set 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient: shutdown 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 -- 172.21.15.190:0/923681910 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee2c13a2b0 msgr2=0x7fee2c06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 --2- 172.21.15.190:0/923681910 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee2c13a2b0 0x7fee2c06c8d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fee200098a0 tx=0x7fee2001c4d0 comp rx=0 tx=0).stop 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 -- 172.21.15.190:0/923681910 shutdown_connections 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 --2- 172.21.15.190:0/923681910 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee2c13a2b0 0x7fee2c06c8d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 --2- 172.21.15.190:0/923681910 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee2c139990 0x7fee2c139d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 -- 172.21.15.190:0/923681910 >> 172.21.15.190:0/923681910 conn(0x7fee2c0a8450 msgr2=0x7fee2c0a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 -- 172.21.15.190:0/923681910 shutdown_connections 2024-08-19T10:43:27.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 -- 172.21.15.190:0/923681910 wait complete. 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient: build_initial_monmap 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient: monmap: 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:27.384 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 Processor -- start 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 -- start start 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient: init 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient: _add_conns ranks=[5,1,2,0,4,3,7,8,6] 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee2c09fae0 0x7fee2c1d1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient(hunting): picked mon.noname-h con 0x7fee2c09fae0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2c139990 0x7fee2c1363a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:27.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient(hunting): picked mon.noname-f con 0x7fee2c139990 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee2c13a2b0 0x7fee2c1368e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient(hunting): picked mon.noname-b con 0x7fee2c13a2b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c144130 con 0x7fee2c139990 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fee2c1442b0 con 0x7fee2c13a2b0 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fee2c144430 con 0x7fee2c09fae0 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee33050640 10 monclient(hunting): authenticate will time out at 1252.946748s 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2bfff640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2c139990 msgr2=0x7fee2c1363a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee2bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2c139990 0x7fee2c1363a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:27.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee2c09fae0 0x7fee2c1d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee2c09fae0 0x7fee2c1d1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:46960/0 (socket says 172.21.15.190:46960) 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 1 -- 172.21.15.190:0/2666514345 learned_addr learned my addr 172.21.15.190:0/2666514345 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 10 monclient(hunting): get_auth_request con 0x7fee2c09fae0 auth_method 0 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 10 monclient(hunting): handle_auth_done global_id 5855 payload 824 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee315ed640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee2c13a2b0 0x7fee2c1368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:27.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 1 -- 172.21.15.190:0/2666514345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee2c13a2b0 msgr2=0x7fee2c1368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:27.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee2c13a2b0 0x7fee2c1368e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:27.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 1 -- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2c139990 msgr2=0x7fee2c1363a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:27.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2c139990 0x7fee2c1363a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:27.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 10 monclient: _finish_hunting 0 2024-08-19T10:43:27.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 1 monclient: found mon.noname-h 2024-08-19T10:43:27.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:27.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:27.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.375+0000 7fee30dec640 1 -- 172.21.15.190:0/2666514345 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c13af10 con 0x7fee2c09fae0 2024-08-19T10:43:27.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee30dec640 1 -- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee2c09fae0 msgr2=0x7fee2c1d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:27.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee30dec640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee2c09fae0 0x7fee2c1d1960 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00d800 tx=0x7fee1c00dcc0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:27.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 1 -- 172.21.15.190:0/2666514345 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee1c00bd40 con 0x7fee2c09fae0 2024-08-19T10:43:27.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:27.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:27.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 10 monclient: dump: 2024-08-19T10:43:27.390 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:27.391 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 10 monclient: _finish_auth 0 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 10 monclient: _check_auth_tickets 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 1 -- 172.21.15.190:0/2666514345 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee1c019070 con 0x7fee2c09fae0 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee33050640 5 monclient: authenticate success, global_id 5855 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee33050640 10 monclient: _renew_subs 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee33050640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee33050640 1 -- 172.21.15.190:0/2666514345 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2c13b1f0 con 0x7fee2c09fae0 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 1 -- 172.21.15.190:0/2666514345 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee1c0148a0 con 0x7fee2c09fae0 2024-08-19T10:43:27.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 10 monclient: dump: 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:27.393 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:27.394 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:27.394 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:27.394 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:27.394 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:27.394 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:27.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee33050640 10 monclient: _renew_subs 2024-08-19T10:43:27.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee33050640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:27.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee33050640 1 -- 172.21.15.190:0/2666514345 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2c0a4080 con 0x7fee2c09fae0 2024-08-19T10:43:27.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee2a7fc640 4 set_mon_vals no callback set 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee33050640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee33050640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee33050640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee33050640 1 -- 172.21.15.190:0/2666514345 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2c139d70 con 0x7fee2c09fae0 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 1 -- 172.21.15.190:0/2666514345 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fee1c014430 con 0x7fee2c09fae0 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fee1803be00 0x7fee1803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee297fa640 1 -- 172.21.15.190:0/2666514345 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(169..169 src has 1..169) ==== 9892+0+0 (secure 0 0 0) 0x7fee1c00f040 con 0x7fee2c09fae0 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee2bfff640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fee1803be00 0x7fee1803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.379+0000 7fee2bfff640 10 monclient: get_auth_request con 0x7fee1803be00 auth_method 0 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.383+0000 7fee297fa640 1 -- 172.21.15.190:0/2666514345 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.383+0000 7fee2bfff640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fee1803be00 0x7fee1803e2c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0a94b0 tx=0x7fee200a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.383+0000 7fee297fa640 1 -- 172.21.15.190:0/2666514345 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fee1c022020 con 0x7fee2c09fae0 2024-08-19T10:43:27.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.383+0000 7fee297fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:27.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.383+0000 7fee297fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:27.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.563+0000 7fee33050640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-08-19T10:43:27.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.563+0000 7fee33050640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-08-19T10:43:27.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.563+0000 7fee33050640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:27.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:27.563+0000 7fee33050640 1 -- 172.21.15.190:0/2666514345 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]} v 0) -- 0x7fee2c137e10 con 0x7fee2c09fae0 2024-08-19T10:43:28.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:28.223+0000 7fee297fa640 1 -- 172.21.15.190:0/2666514345 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}]=0 v170) ==== 127+0+0 (secure 0 0 0) 0x7fee1c04d600 con 0x7fee2c09fae0 2024-08-19T10:43:28.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:28.223+0000 7fee297fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-08-19T10:43:28.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:28.223+0000 7fee297fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:28.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:28.339+0000 7fee33050640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-08-19T10:43:28.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:28.339+0000 7fee33050640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-08-19T10:43:28.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:28.339+0000 7fee33050640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:28.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:28.339+0000 7fee33050640 1 -- 172.21.15.190:0/2666514345 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]} v 0) -- 0x7fee2c0008e0 con 0x7fee2c09fae0 2024-08-19T10:43:29.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.406+0000 7fee297fa640 1 -- 172.21.15.190:0/2666514345 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}]=0 v171) ==== 127+0+0 (secure 0 0 0) 0x7fee1c014a80 con 0x7fee2c09fae0 2024-08-19T10:43:29.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.406+0000 7fee297fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cool", "item": "osd.0", "weight": [8.5]}] 2024-08-19T10:43:29.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.406+0000 7fee297fa640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:29.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 -- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fee1803be00 msgr2=0x7fee1803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:29.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fee1803be00 0x7fee1803e2c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0a94b0 tx=0x7fee200a7070 comp rx=0 tx=0).stop 2024-08-19T10:43:29.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 10 monclient: shutdown 2024-08-19T10:43:29.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:29.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 -- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee2c09fae0 msgr2=0x7fee2c1d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:29.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee2c09fae0 0x7fee2c1d1960 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00d800 tx=0x7fee1c00dcc0 comp rx=0 tx=0).stop 2024-08-19T10:43:29.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 -- 172.21.15.190:0/2666514345 shutdown_connections 2024-08-19T10:43:29.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fee1803be00 0x7fee1803e2c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:29.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fee2c13a2b0 0x7fee2c1368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:29.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee2c139990 0x7fee2c1363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:29.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 --2- 172.21.15.190:0/2666514345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee2c09fae0 0x7fee2c1d1960 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:29.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 -- 172.21.15.190:0/2666514345 >> 172.21.15.190:0/2666514345 conn(0x7fee2c0a8450 msgr2=0x7fee2c09c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:29.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 -- 172.21.15.190:0/2666514345 shutdown_connections 2024-08-19T10:43:29.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.410+0000 7fee33050640 1 -- 172.21.15.190:0/2666514345 wait complete. 2024-08-19T10:43:29.431 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set reweight cold osd.0 6.5 6.6 2024-08-19T10:43:29.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient: get_monmap_and_config 2024-08-19T10:43:29.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient: build_initial_monmap 2024-08-19T10:43:29.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient: monmap: 2024-08-19T10:43:29.551 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:29.551 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:29.552187+0000 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:29.552187+0000 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:29.552 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:29.553 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:29.553 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:29.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 1 Processor -- start 2024-08-19T10:43:29.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 1 -- start start 2024-08-19T10:43:29.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient: init 2024-08-19T10:43:29.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:29.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient: _add_conns ranks=[7,9,1,5,6,3,4,0,8,10,2] 2024-08-19T10:43:29.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac073f30 0x7f56ac09ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:29.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient(hunting): picked mon.noname-i con 0x7f56ac073f30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:29.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f56ac0a1f30 0x7f56ac09cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:29.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient(hunting): picked mon.noname-a con 0x7f56ac0a1f30 addr v2:172.21.15.89:3300/0 2024-08-19T10:43:29.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f56ac09d4e0 addr v1:172.21.15.190:6789/0 2024-08-19T10:43:29.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56ac0a55a0 con 0x7f56ac09d4e0 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f56ac09f7c0 con 0x7f56ac0a1f30 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f56ac09dcf0 con 0x7f56ac073f30 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b3be3640 10 monclient(hunting): authenticate will time out at 1255.122705s 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b2180640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f56ac09d4e0 legacy=0x7f56ac13c490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac073f30 0x7f56ac09ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac073f30 0x7f56ac09ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:59326/0 (socket says 172.21.15.190:59326) 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 1 -- 172.21.15.190:0/2503736479 learned_addr learned my addr 172.21.15.190:0/2503736479 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 10 monclient(hunting): get_auth_request con 0x7f56ac073f30 auth_method 0 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:29.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 10 monclient(hunting): handle_auth_done global_id 15100 payload 274 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b117e640 1 --2- 172.21.15.190:0/2503736479 >> v2:172.21.15.89:3300/0 conn(0x7f56ac0a1f30 0x7f56ac09cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 1 -- 172.21.15.190:0/2503736479 >> v2:172.21.15.89:3300/0 conn(0x7f56ac0a1f30 msgr2=0x7f56ac09cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 1 --2- 172.21.15.190:0/2503736479 >> v2:172.21.15.89:3300/0 conn(0x7f56ac0a1f30 0x7f56ac09cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 1 -- 172.21.15.190:0/2503736479 >> v1:172.21.15.190:6789/0 conn(0x7f56ac09d4e0 legacy=0x7f56ac13c490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 10 monclient: _finish_hunting 0 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 1 monclient: found mon.noname-i 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b197f640 1 -- 172.21.15.190:0/2503736479 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac074310 con 0x7f56ac073f30 2024-08-19T10:43:29.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b117e640 10 monclient: get_auth_request con 0x7f56ac0a1f30 auth_method 0 2024-08-19T10:43:29.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.550+0000 7f56b117e640 1 --2- 172.21.15.190:0/2503736479 >> v2:172.21.15.89:3300/0 conn(0x7f56ac0a1f30 0x7f56ac09cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:43:29.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b197f640 1 -- 172.21.15.190:0/2503736479 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac073f30 msgr2=0x7f56ac09ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:29.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b197f640 1 --2- 172.21.15.190:0/2503736479 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac073f30 0x7f56ac09ca60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006fd0 tx=0x7f56a801e8b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=6ef0d24d4d3c2d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 1 -- 172.21.15.190:0/2503736479 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f56a80ad070 con 0x7f56ac073f30 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 10 monclient: dump: 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:29.560 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 10 monclient: _finish_auth 0 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 10 monclient: _check_auth_tickets 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 1 -- 172.21.15.190:0/2503736479 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a80044a0 con 0x7f56ac073f30 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b3be3640 5 monclient: authenticate success, global_id 15100 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b3be3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:29.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b3be3640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 1 -- 172.21.15.190:0/2503736479 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f56a80a7ae0 con 0x7f56ac073f30 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b3be3640 4 set_mon_vals no callback set 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.554+0000 7f56b097d640 10 monclient: dump: 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:29.562 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient: shutdown 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 -- 172.21.15.190:0/2503736479 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac073f30 msgr2=0x7f56ac09ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 --2- 172.21.15.190:0/2503736479 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac073f30 0x7f56ac09ca60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006fd0 tx=0x7f56a801e8b0 comp rx=0 tx=0).stop 2024-08-19T10:43:29.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 -- 172.21.15.190:0/2503736479 shutdown_connections 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 --2- 172.21.15.190:0/2503736479 >> v2:172.21.15.89:3300/0 conn(0x7f56ac0a1f30 0x7f56ac09cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 --2- 172.21.15.190:0/2503736479 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac073f30 0x7f56ac09ca60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 -- 172.21.15.190:0/2503736479 >> 172.21.15.190:0/2503736479 conn(0x7f56ac0a4790 msgr2=0x7f56ac0a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 -- 172.21.15.190:0/2503736479 shutdown_connections 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 -- 172.21.15.190:0/2503736479 wait complete. 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient: build_initial_monmap 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient: monmap: 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:29.564 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 Processor -- start 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 -- start start 2024-08-19T10:43:29.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient: init 2024-08-19T10:43:29.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:29.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient: _add_conns ranks=[0,7,8,1,3,6,2,4,5] 2024-08-19T10:43:29.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ac073f30 0x7f56ac1341b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:29.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient(hunting): picked mon.noname-a con 0x7f56ac073f30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:29.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac09d4e0 0x7f56ac13a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:29.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient(hunting): picked mon.noname-i con 0x7f56ac09d4e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:29.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f56ac0a1f30 0x7f56ac13add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:29.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient(hunting): picked mon.noname-e con 0x7f56ac0a1f30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:29.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac13fbc0 con 0x7f56ac073f30 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f56ac09fdd0 con 0x7f56ac09d4e0 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f56ac13fd40 con 0x7f56ac0a1f30 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b3be3640 10 monclient(hunting): authenticate will time out at 1255.129872s 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac09d4e0 0x7f56ac13a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac09d4e0 0x7f56ac13a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:59336/0 (socket says 172.21.15.190:59336) 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 1 -- 172.21.15.190:0/3520664328 learned_addr learned my addr 172.21.15.190:0/3520664328 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 10 monclient(hunting): get_auth_request con 0x7f56ac09d4e0 auth_method 0 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:29.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b197f640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ac073f30 0x7f56ac1341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b2180640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f56ac0a1f30 0x7f56ac13add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b197f640 10 monclient(hunting): get_auth_request con 0x7f56ac073f30 auth_method 0 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b197f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b197f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b197f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b2180640 10 monclient(hunting): get_auth_request con 0x7f56ac0a1f30 auth_method 0 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b2180640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b2180640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:29.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b2180640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 10 monclient(hunting): handle_auth_done global_id 15109 payload 824 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 1 -- 172.21.15.190:0/3520664328 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f56ac0a1f30 msgr2=0x7f56ac13add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f56ac0a1f30 0x7f56ac13add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 1 -- 172.21.15.190:0/3520664328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ac073f30 msgr2=0x7f56ac1341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ac073f30 0x7f56ac1341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 10 monclient: _finish_hunting 0 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 1 monclient: found mon.noname-i 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.558+0000 7f56b117e640 1 -- 172.21.15.190:0/3520664328 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac13d190 con 0x7f56ac09d4e0 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b117e640 1 -- 172.21.15.190:0/3520664328 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac09d4e0 msgr2=0x7f56ac13a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b117e640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac09d4e0 0x7f56ac13a890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f569c00c7f0 tx=0x7f569c00ccb0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 1 -- 172.21.15.190:0/3520664328 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f569c015070 con 0x7f56ac09d4e0 2024-08-19T10:43:29.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 10 monclient: dump: 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:29.574 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 10 monclient: _finish_auth 0 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b3be3640 5 monclient: authenticate success, global_id 15109 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 1 -- 172.21.15.190:0/3520664328 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f569c007ca0 con 0x7f56ac09d4e0 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b3be3640 10 monclient: _renew_subs 2024-08-19T10:43:29.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 1 -- 172.21.15.190:0/3520664328 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f569c007e50 con 0x7f56ac09d4e0 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b3be3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b3be3640 1 -- 172.21.15.190:0/3520664328 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac0a6120 con 0x7f56ac09d4e0 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a3fff640 4 set_mon_vals no callback set 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 10 monclient: dump: 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:29.576 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b3be3640 10 monclient: _renew_subs 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b3be3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b3be3640 1 -- 172.21.15.190:0/3520664328 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac11ecb0 con 0x7f56ac09d4e0 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b3be3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b3be3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:29.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b3be3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:29.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b3be3640 1 -- 172.21.15.190:0/3520664328 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ac09ca60 con 0x7f56ac09d4e0 2024-08-19T10:43:29.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 1 -- 172.21.15.190:0/3520664328 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f569c018070 con 0x7f56ac09d4e0 2024-08-19T10:43:29.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f569003bfd0 0x7f569003e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:29.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56a27fc640 1 -- 172.21.15.190:0/3520664328 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(171..171 src has 1..171) ==== 9892+0+0 (secure 0 0 0) 0x7f569c04fdf0 con 0x7f56ac09d4e0 2024-08-19T10:43:29.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b197f640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f569003bfd0 0x7f569003e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:29.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.562+0000 7f56b197f640 10 monclient: get_auth_request con 0x7f569003bfd0 auth_method 0 2024-08-19T10:43:29.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.566+0000 7f56b197f640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f569003bfd0 0x7f569003e490 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f56ac06f9f0 tx=0x7f56a80a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:29.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.566+0000 7f56a27fc640 1 -- 172.21.15.190:0/3520664328 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f569c018360 con 0x7f56ac09d4e0 2024-08-19T10:43:29.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.566+0000 7f56a27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:29.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.566+0000 7f56a27fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:29.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.746+0000 7f56b3be3640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-08-19T10:43:29.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.746+0000 7f56b3be3640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-08-19T10:43:29.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.746+0000 7f56b3be3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:29.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:29.746+0000 7f56b3be3640 1 -- 172.21.15.190:0/3520664328 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]} v 0) -- 0x7f56ac13d920 con 0x7f56ac09d4e0 2024-08-19T10:43:32.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:32.058+0000 7f56a27fc640 1 -- 172.21.15.190:0/3520664328 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}]=0 v172) ==== 132+0+0 (secure 0 0 0) 0x7f569c04d3e0 con 0x7f56ac09d4e0 2024-08-19T10:43:32.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:32.058+0000 7f56a27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-08-19T10:43:32.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:32.058+0000 7f56a27fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:32.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:32.170+0000 7f56b3be3640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-08-19T10:43:32.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:32.170+0000 7f56b3be3640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-08-19T10:43:32.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:32.170+0000 7f56b3be3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:32.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:32.170+0000 7f56b3be3640 1 -- 172.21.15.190:0/3520664328 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]} v 0) -- 0x7f56ac13cd50 con 0x7f56ac09d4e0 2024-08-19T10:43:32.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:32.558+0000 7f56a17fa640 10 monclient: tick 2024-08-19T10:43:32.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:32.558+0000 7f56a17fa640 10 monclient: _check_auth_tickets 2024-08-19T10:43:32.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:32.558+0000 7f56a17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:32.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:32.558+0000 7f56a17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:33.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.230+0000 7f56a27fc640 1 -- 172.21.15.190:0/3520664328 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}]=0 v173) ==== 132+0+0 (secure 0 0 0) 0x7f569c03f020 con 0x7f56ac09d4e0 2024-08-19T10:43:33.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.230+0000 7f56a27fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set reweight", "pool": "cold", "item": "osd.0", "weight": [6.5, 6.6]}] 2024-08-19T10:43:33.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.230+0000 7f56a27fc640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:33.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 -- 172.21.15.190:0/3520664328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f569003bfd0 msgr2=0x7f569003e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:33.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f569003bfd0 0x7f569003e490 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f56ac06f9f0 tx=0x7f56a80a9070 comp rx=0 tx=0).stop 2024-08-19T10:43:33.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 10 monclient: shutdown 2024-08-19T10:43:33.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:33.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 -- 172.21.15.190:0/3520664328 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac09d4e0 msgr2=0x7f56ac13a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:33.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac09d4e0 0x7f56ac13a890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f569c00c7f0 tx=0x7f569c00ccb0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 -- 172.21.15.190:0/3520664328 shutdown_connections 2024-08-19T10:43:33.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f569003bfd0 0x7f569003e490 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f56ac0a1f30 0x7f56ac13add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f56ac09d4e0 0x7f56ac13a890 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 --2- 172.21.15.190:0/3520664328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ac073f30 0x7f56ac1341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 -- 172.21.15.190:0/3520664328 >> 172.21.15.190:0/3520664328 conn(0x7f56ac0a4790 msgr2=0x7f56ac1378a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:33.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 -- 172.21.15.190:0/3520664328 shutdown_connections 2024-08-19T10:43:33.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.234+0000 7f56b3be3640 1 -- 172.21.15.190:0/3520664328 wait complete. 2024-08-19T10:43:33.257 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush tree --show-shadow 2024-08-19T10:43:33.257 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 'osd\.0' 2024-08-19T10:43:33.257 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep globster 2024-08-19T10:43:33.257 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep '7\.' 2024-08-19T10:43:33.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient: get_monmap_and_config 2024-08-19T10:43:33.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient: build_initial_monmap 2024-08-19T10:43:33.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient: monmap: 2024-08-19T10:43:33.376 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:33.376 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:33.376 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:33.376769+0000 2024-08-19T10:43:33.376 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:33.376769+0000 2024-08-19T10:43:33.376 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:33.377 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:33.378 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:33.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 1 Processor -- start 2024-08-19T10:43:33.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 1 -- start start 2024-08-19T10:43:33.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient: init 2024-08-19T10:43:33.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:33.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient: _add_conns ranks=[10,6,2,1,0,8,9,4,5,7,3] 2024-08-19T10:43:33.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f94800a6740 0x7f94800a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:33.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient(hunting): picked mon.noname-f con 0x7f94800a6740 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a5bc0 0x7f94800a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient(hunting): picked mon.noname-d con 0x7f94800a5bc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94800a7060 0x7f9480139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient(hunting): picked mon.noname-b con 0x7f94800a7060 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f94800a4c20 con 0x7f94800a6740 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f94800a3530 con 0x7f94800a7060 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f94800a3740 con 0x7f94800a5bc0 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f94854ce640 10 monclient(hunting): authenticate will time out at 1258.947513s 2024-08-19T10:43:33.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f947effd640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f94800a6740 msgr2=0x7f94800a5680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f947effd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f94800a6740 0x7f94800a5680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f947e7fc640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a5bc0 msgr2=0x7f94800a6b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f947e7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a5bc0 0x7f94800a6b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.374+0000 7f947f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94800a7060 0x7f9480139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94800a7060 0x7f9480139590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:33068/0 (socket says 172.21.15.190:33068) 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 1 -- 172.21.15.190:0/2986354182 learned_addr learned my addr 172.21.15.190:0/2986354182 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 10 monclient(hunting): get_auth_request con 0x7f94800a7060 auth_method 0 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:33.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:33.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 10 monclient(hunting): handle_auth_done global_id 5879 payload 274 2024-08-19T10:43:33.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 1 -- 172.21.15.190:0/2986354182 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a5bc0 msgr2=0x7f94800a6b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:33.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 1 --2- 172.21.15.190:0/2986354182 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a5bc0 0x7f94800a6b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 1 -- 172.21.15.190:0/2986354182 >> v2:172.21.15.190:3300/0 conn(0x7f94800a6740 msgr2=0x7f94800a5680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:33.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 1 --2- 172.21.15.190:0/2986354182 >> v2:172.21.15.190:3300/0 conn(0x7f94800a6740 0x7f94800a5680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:33.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 1 monclient: found mon.noname-b 2024-08-19T10:43:33.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:33.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:33.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.378+0000 7f947f7fe640 1 -- 172.21.15.190:0/2986354182 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94800a74d0 con 0x7f94800a7060 2024-08-19T10:43:33.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947f7fe640 1 -- 172.21.15.190:0/2986354182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94800a7060 msgr2=0x7f9480139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:33.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947f7fe640 1 --2- 172.21.15.190:0/2986354182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94800a7060 0x7f9480139590 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9478009e80 tx=0x7f947801c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80a273f6504eb699 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:33.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 1 -- 172.21.15.190:0/2986354182 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94780091b0 con 0x7f94800a7060 2024-08-19T10:43:33.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 10 monclient: dump: 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:33.386 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 10 monclient: _finish_auth 0 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 1 -- 172.21.15.190:0/2986354182 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94780a5be0 con 0x7f94800a7060 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 1 -- 172.21.15.190:0/2986354182 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f94780a5d90 con 0x7f94800a7060 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:33.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f947dffb640 10 monclient: dump: 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:33.388 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 5 monclient: authenticate success, global_id 5879 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 4 set_mon_vals no callback set 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 10 monclient: shutdown 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 1 -- 172.21.15.190:0/2986354182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94800a7060 msgr2=0x7f9480139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 1 --2- 172.21.15.190:0/2986354182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94800a7060 0x7f9480139590 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9478009e80 tx=0x7f947801c910 comp rx=0 tx=0).stop 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 1 -- 172.21.15.190:0/2986354182 shutdown_connections 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 1 --2- 172.21.15.190:0/2986354182 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f94800a7060 0x7f9480139590 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 1 --2- 172.21.15.190:0/2986354182 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a5bc0 0x7f94800a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 1 --2- 172.21.15.190:0/2986354182 >> v2:172.21.15.190:3300/0 conn(0x7f94800a6740 0x7f94800a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 1 -- 172.21.15.190:0/2986354182 >> 172.21.15.190:0/2986354182 conn(0x7f94800abd40 msgr2=0x7f94800ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 1 -- 172.21.15.190:0/2986354182 shutdown_connections 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 1 -- 172.21.15.190:0/2986354182 wait complete. 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 10 monclient: build_initial_monmap 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 10 monclient: monmap: 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:33.390 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:33.391 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:33.391 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:33.391 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:33.391 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:33.391 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:33.391 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:33.391 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:33.391 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:33.391 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:33.391 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:33.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 1 Processor -- start 2024-08-19T10:43:33.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 1 -- start start 2024-08-19T10:43:33.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.382+0000 7f94854ce640 10 monclient: init 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient: _add_conns ranks=[6,5,4,7,3,8,2,1,0] 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a6740 0x7f9480133880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient(hunting): picked mon.noname-d con 0x7f94800a6740 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94800a7060 0x7f9480133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient(hunting): picked mon.noname-h con 0x7f94800a7060 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948009ca60 0x7f948009cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient(hunting): picked mon.noname-c con 0x7f948009ca60 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9480143450 con 0x7f948009ca60 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:33.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f94801432d0 con 0x7f94800a7060 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f94801435d0 con 0x7f94800a6740 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient(hunting): authenticate will time out at 1258.956275s 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947f7fe640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948009ca60 msgr2=0x7f948009cec0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947effd640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a6740 msgr2=0x7f9480133880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94800a7060 0x7f9480133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947f7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948009ca60 0x7f948009cec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947effd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a6740 0x7f9480133880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94800a7060 0x7f9480133dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:46966/0 (socket says 172.21.15.190:46966) 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 1 -- 172.21.15.190:0/1217486558 learned_addr learned my addr 172.21.15.190:0/1217486558 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 10 monclient(hunting): get_auth_request con 0x7f94800a7060 auth_method 0 2024-08-19T10:43:33.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 10 monclient(hunting): handle_auth_done global_id 5864 payload 824 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 1 -- 172.21.15.190:0/1217486558 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a6740 msgr2=0x7f9480133880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a6740 0x7f9480133880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 1 -- 172.21.15.190:0/1217486558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948009ca60 msgr2=0x7f948009cec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948009ca60 0x7f948009cec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:43:33.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 1 monclient: found mon.noname-h 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 1 -- 172.21.15.190:0/1217486558 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948009f700 con 0x7f94800a7060 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 1 -- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94800a7060 msgr2=0x7f9480133dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947e7fc640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94800a7060 0x7f9480133dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f947400b530 tx=0x7f947400b9f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 1 -- 172.21.15.190:0/1217486558 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9474013020 con 0x7f94800a7060 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 10 monclient: dump: 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:33.399 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 10 monclient: _finish_auth 0 2024-08-19T10:43:33.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 10 monclient: _check_auth_tickets 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 1 -- 172.21.15.190:0/1217486558 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9474003de0 con 0x7f94800a7060 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 1 -- 172.21.15.190:0/1217486558 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f947400f400 con 0x7f94800a7060 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 5 monclient: authenticate success, global_id 5864 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient: _renew_subs 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 1 -- 172.21.15.190:0/1217486558 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948009f9e0 con 0x7f94800a7060 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f947d7fa640 4 set_mon_vals no callback set 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94677fe640 10 monclient: dump: 2024-08-19T10:43:33.401 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:33.402 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient: _renew_subs 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.386+0000 7f94854ce640 1 -- 172.21.15.190:0/1217486558 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f948009e1e0 con 0x7f94800a7060 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.390+0000 7f94854ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.390+0000 7f94854ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.390+0000 7f94854ce640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.390+0000 7f94854ce640 1 -- 172.21.15.190:0/1217486558 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94800a5680 con 0x7f94800a7060 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.390+0000 7f947e7fc640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94800a7060 0x7f9480133dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f947400b530 tx=0x7f947400b9f0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f9474016020 mgrmap(e 5) 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.390+0000 7f94677fe640 1 -- 172.21.15.190:0/1217486558 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9474016020 con 0x7f94800a7060 2024-08-19T10:43:33.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.390+0000 7f94677fe640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f946c03be00 0x7f946c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:33.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.390+0000 7f947effd640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f946c03be00 0x7f946c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:33.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.390+0000 7f947effd640 10 monclient: get_auth_request con 0x7f946c03be00 auth_method 0 2024-08-19T10:43:33.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.390+0000 7f947effd640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f946c03be00 0x7f946c03e2c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9470003990 tx=0x7f9470006c70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:33.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:33.394+0000 7f94677fe640 1 -- 172.21.15.190:0/1217486558 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(173..173 src has 1..173) ==== 9892+0+0 (secure 0 0 0) 0x7f947404fbb0 con 0x7f94800a7060 2024-08-19T10:43:34.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.014+0000 7f94677fe640 1 -- 172.21.15.190:0/1217486558 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f947401d020 con 0x7f94800a7060 2024-08-19T10:43:34.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.014+0000 7f94677fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:34.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.014+0000 7f94677fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:34.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.202+0000 7f94854ce640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:34.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.202+0000 7f94854ce640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:34.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.202+0000 7f94854ce640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:34.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.202+0000 7f94854ce640 1 -- 172.21.15.190:0/1217486558 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f948013fac0 con 0x7f94800a7060 2024-08-19T10:43:34.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.206+0000 7f94677fe640 1 -- 172.21.15.190:0/1217486558 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v173) ==== 83+0+2320 (secure 0 0 0) 0x7f947404d1a0 con 0x7f94800a7060 2024-08-19T10:43:34.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.206+0000 7f94677fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:34.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.206+0000 7f94677fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:34.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.314+0000 7f94854ce640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:34.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.314+0000 7f94854ce640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:34.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.314+0000 7f94854ce640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:34.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.314+0000 7f94854ce640 1 -- 172.21.15.190:0/1217486558 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f94800a2820 con 0x7f94800a7060 2024-08-19T10:43:34.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.314+0000 7f94677fe640 1 -- 172.21.15.190:0/1217486558 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v173) ==== 83+0+2320 (secure 0 0 0) 0x7f947400f620 con 0x7f94800a7060 2024-08-19T10:43:34.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.314+0000 7f94677fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:34.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.314+0000 7f94677fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:34.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 -- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f946c03be00 msgr2=0x7f946c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:34.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f946c03be00 0x7f946c03e2c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9470003990 tx=0x7f9470006c70 comp rx=0 tx=0).stop 2024-08-19T10:43:34.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 10 monclient: shutdown 2024-08-19T10:43:34.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:34.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 -- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94800a7060 msgr2=0x7f9480133dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:34.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94800a7060 0x7f9480133dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f947400b530 tx=0x7f947400b9f0 comp rx=0 tx=0).stop 2024-08-19T10:43:34.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 -- 172.21.15.190:0/1217486558 shutdown_connections 2024-08-19T10:43:34.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f946c03be00 0x7f946c03e2c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:34.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f948009ca60 0x7f948009cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:34.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f94800a7060 0x7f9480133dc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:34.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 --2- 172.21.15.190:0/1217486558 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f94800a6740 0x7f9480133880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:34.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 -- 172.21.15.190:0/1217486558 >> 172.21.15.190:0/1217486558 conn(0x7f94800abd40 msgr2=0x7f94801366b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:34.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 -- 172.21.15.190:0/1217486558 shutdown_connections 2024-08-19T10:43:34.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.318+0000 7f94854ce640 1 -- 172.21.15.190:0/1217486558 wait complete. 2024-08-19T10:43:34.334 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-08-19T10:43:34.334 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-08-19T10:43:34.335 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush tree --show-shadow 2024-08-19T10:43:34.335 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 'osd\.0' 2024-08-19T10:43:34.335 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep globster 2024-08-19T10:43:34.335 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep '8\.' 2024-08-19T10:43:34.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient: get_monmap_and_config 2024-08-19T10:43:34.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient: build_initial_monmap 2024-08-19T10:43:34.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient: monmap: 2024-08-19T10:43:34.452 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:34.453127+0000 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:34.453127+0000 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:34.453 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:34.454 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:34.454 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:34.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 1 Processor -- start 2024-08-19T10:43:34.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 1 -- start start 2024-08-19T10:43:34.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient: init 2024-08-19T10:43:34.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:34.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient: _add_conns ranks=[2,7,1,3,9,5,0,10,8,6,4] 2024-08-19T10:43:34.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8bc0a5940 0x7fa8bc0720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:34.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient(hunting): picked mon.noname-b con 0x7fa8bc0a5940 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:34.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8bc0a5d20 0x7fa8bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient(hunting): picked mon.noname-i con 0x7fa8bc0a5d20 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa8bc139710 addr v1:172.21.15.190:6789/0 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8bc06efb0 con 0x7fa8bc139710 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc06df10 con 0x7fa8bc0a5940 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa8bc06f290 con 0x7fa8bc0a5d20 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c34c8640 10 monclient(hunting): authenticate will time out at 1260.023214s 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c1a65640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fa8bc139710 legacy=0x7fa8bc139b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8bc0a5d20 0x7fa8bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8bc0a5d20 0x7fa8bc0a6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:59344/0 (socket says 172.21.15.190:59344) 2024-08-19T10:43:34.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 1 -- 172.21.15.190:0/307764628 learned_addr learned my addr 172.21.15.190:0/307764628 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 10 monclient(hunting): get_auth_request con 0x7fa8bc0a5d20 auth_method 0 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c1264640 1 --2- 172.21.15.190:0/307764628 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8bc0a5940 0x7fa8bc0720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c1264640 10 monclient(hunting): get_auth_request con 0x7fa8bc0a5940 auth_method 0 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c1264640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c1264640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c1264640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 10 monclient(hunting): handle_auth_done global_id 15118 payload 274 2024-08-19T10:43:34.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 1 -- 172.21.15.190:0/307764628 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8bc0a5940 msgr2=0x7fa8bc0720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:34.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 1 --2- 172.21.15.190:0/307764628 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8bc0a5940 0x7fa8bc0720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:34.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 1 -- 172.21.15.190:0/307764628 >> v1:172.21.15.190:6789/0 conn(0x7fa8bc139710 legacy=0x7fa8bc139b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:34.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 10 monclient: _finish_hunting 0 2024-08-19T10:43:34.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 1 monclient: found mon.noname-i 2024-08-19T10:43:34.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:34.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:34.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.450+0000 7fa8c0a63640 1 -- 172.21.15.190:0/307764628 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc0a73d0 con 0x7fa8bc0a5d20 2024-08-19T10:43:34.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c0a63640 1 -- 172.21.15.190:0/307764628 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8bc0a5d20 msgr2=0x7fa8bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:34.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c0a63640 1 --2- 172.21.15.190:0/307764628 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8bc0a5d20 0x7fa8bc0a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0098a0 tx=0x7fa8ac01c440 comp rx=0 tx=0).ready entity=mon.7 client_cookie=cc440e34def49c56 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:34.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 1 -- 172.21.15.190:0/307764628 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8ac0b0070 con 0x7fa8bc0a5d20 2024-08-19T10:43:34.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 10 monclient: dump: 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:34.465 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 10 monclient: _finish_auth 0 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 1 -- 172.21.15.190:0/307764628 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8ac0aa810 con 0x7fa8bc0a5d20 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 5 monclient: authenticate success, global_id 15118 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:34.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 1 -- 172.21.15.190:0/307764628 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8ac0aae30 con 0x7fa8bc0a5d20 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8b3fff640 10 monclient: dump: 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:34.467 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 4 set_mon_vals no callback set 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient: shutdown 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 -- 172.21.15.190:0/307764628 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8bc0a5d20 msgr2=0x7fa8bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 --2- 172.21.15.190:0/307764628 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8bc0a5d20 0x7fa8bc0a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0098a0 tx=0x7fa8ac01c440 comp rx=0 tx=0).stop 2024-08-19T10:43:34.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 -- 172.21.15.190:0/307764628 shutdown_connections 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 --2- 172.21.15.190:0/307764628 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa8bc0a5d20 0x7fa8bc0a6100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 --2- 172.21.15.190:0/307764628 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8bc0a5940 0x7fa8bc0720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 -- 172.21.15.190:0/307764628 >> 172.21.15.190:0/307764628 conn(0x7fa8bc0aa530 msgr2=0x7fa8bc0a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 -- 172.21.15.190:0/307764628 shutdown_connections 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 -- 172.21.15.190:0/307764628 wait complete. 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient: build_initial_monmap 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient: monmap: 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:34.469 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 Processor -- start 2024-08-19T10:43:34.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 -- start start 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient: init 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient: _add_conns ranks=[6,8,3,4,1,5,0,2,7] 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8bc0a5940 0x7fa8bc120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient(hunting): picked mon.noname-d con 0x7fa8bc0a5940 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8bc0a5d20 0x7fa8bc120ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient(hunting): picked mon.noname-e con 0x7fa8bc0a5d20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8bc139710 0x7fa8bc121000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient(hunting): picked mon.noname-g con 0x7fa8bc139710 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc144520 con 0x7fa8bc139710 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:34.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa8bc1443a0 con 0x7fa8bc0a5940 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa8bc1446a0 con 0x7fa8bc0a5d20 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c34c8640 10 monclient(hunting): authenticate will time out at 1260.030669s 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8bc139710 0x7fa8bc121000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1264640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8bc0a5940 msgr2=0x7fa8bc120580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8bc139710 0x7fa8bc121000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:58120/0 (socket says 172.21.15.190:58120) 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1264640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8bc0a5940 0x7fa8bc120580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 1 -- 172.21.15.190:0/166011998 learned_addr learned my addr 172.21.15.190:0/166011998 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 10 monclient(hunting): get_auth_request con 0x7fa8bc139710 auth_method 0 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:34.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c0a63640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8bc0a5d20 0x7fa8bc120ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c0a63640 10 monclient(hunting): get_auth_request con 0x7fa8bc0a5d20 auth_method 0 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c0a63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c0a63640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c0a63640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 10 monclient(hunting): handle_auth_done global_id 34266 payload 824 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 1 -- 172.21.15.190:0/166011998 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8bc0a5d20 msgr2=0x7fa8bc120ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:34.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8bc0a5d20 0x7fa8bc120ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 1 -- 172.21.15.190:0/166011998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8bc0a5940 msgr2=0x7fa8bc120580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8bc0a5940 0x7fa8bc120580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 10 monclient: _finish_hunting 0 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 1 monclient: found mon.noname-g 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.458+0000 7fa8c1a65640 1 -- 172.21.15.190:0/166011998 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc0a98a0 con 0x7fa8bc139710 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c1a65640 1 -- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8bc139710 msgr2=0x7fa8bc121000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c1a65640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8bc139710 0x7fa8bc121000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400be90 tx=0x7fa8b400bf80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c1a65640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8bc139710 0x7fa8bc121000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400be90 tx=0x7fa8b400bf80 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fa8b4015850 mon_map magic: 0 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 1 -- 172.21.15.190:0/166011998 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8b4015850 con 0x7fa8bc139710 2024-08-19T10:43:34.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 10 monclient: dump: 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:34.475 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c34c8640 5 monclient: authenticate success, global_id 34266 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 1 -- 172.21.15.190:0/166011998 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b4010030 con 0x7fa8bc139710 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c34c8640 10 monclient: _renew_subs 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c34c8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c34c8640 1 -- 172.21.15.190:0/166011998 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc139da0 con 0x7fa8bc139710 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:34.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b37fe640 4 set_mon_vals no callback set 2024-08-19T10:43:34.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c34c8640 10 monclient: _renew_subs 2024-08-19T10:43:34.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c34c8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:34.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c34c8640 1 -- 172.21.15.190:0/166011998 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc139480 con 0x7fa8bc139710 2024-08-19T10:43:34.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c34c8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:34.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c34c8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:34.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c34c8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:34.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c34c8640 1 -- 172.21.15.190:0/166011998 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8bc0720a0 con 0x7fa8bc139710 2024-08-19T10:43:34.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 1 -- 172.21.15.190:0/166011998 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8b40060d0 con 0x7fa8bc139710 2024-08-19T10:43:34.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:34.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:34.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 10 monclient: dump: 2024-08-19T10:43:34.480 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:34.480 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:34.480 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:34.480 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:34.480 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:34.480 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:34.480 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:34.480 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:34.480 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:34.480 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:34.481 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:34.481 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:34.481 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:34.481 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:34.481 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:34.482 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:34.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 1 -- 172.21.15.190:0/166011998 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa8b40153e0 con 0x7fa8bc139710 2024-08-19T10:43:34.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8b1ffb640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa8a003be80 0x7fa8a003e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:34.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c1264640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa8a003be80 0x7fa8a003e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:34.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.462+0000 7fa8c1264640 10 monclient: get_auth_request con 0x7fa8a003be80 auth_method 0 2024-08-19T10:43:34.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.466+0000 7fa8c1264640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa8a003be80 0x7fa8a003e340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8006e90 tx=0x7fa8b8006e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:34.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:34.466+0000 7fa8b1ffb640 1 -- 172.21.15.190:0/166011998 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(173..173 src has 1..173) ==== 9892+0+0 (secure 0 0 0) 0x7fa8b4050800 con 0x7fa8bc139710 2024-08-19T10:43:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.086+0000 7fa8b1ffb640 1 -- 172.21.15.190:0/166011998 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fa8b40072f0 con 0x7fa8bc139710 2024-08-19T10:43:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.086+0000 7fa8b1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.086+0000 7fa8b1ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:35.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.278+0000 7fa8c34c8640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:35.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.278+0000 7fa8c34c8640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:35.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.278+0000 7fa8c34c8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:35.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.278+0000 7fa8c34c8640 1 -- 172.21.15.190:0/166011998 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7fa8bc06ff20 con 0x7fa8bc139710 2024-08-19T10:43:35.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.282+0000 7fa8b1ffb640 1 -- 172.21.15.190:0/166011998 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v173) ==== 83+0+2320 (secure 0 0 0) 0x7fa8b4007e10 con 0x7fa8bc139710 2024-08-19T10:43:35.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.282+0000 7fa8b1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:35.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.282+0000 7fa8b1ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:35.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.386+0000 7fa8c34c8640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:35.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.386+0000 7fa8c34c8640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:35.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.386+0000 7fa8c34c8640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:35.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.386+0000 7fa8c34c8640 1 -- 172.21.15.190:0/166011998 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7fa8bc121860 con 0x7fa8bc139710 2024-08-19T10:43:35.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.386+0000 7fa8b1ffb640 1 -- 172.21.15.190:0/166011998 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v173) ==== 83+0+2320 (secure 0 0 0) 0x7fa8b404ddf0 con 0x7fa8bc139710 2024-08-19T10:43:35.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.386+0000 7fa8b1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:35.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.386+0000 7fa8b1ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:35.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 -- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa8a003be80 msgr2=0x7fa8a003e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:35.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa8a003be80 0x7fa8a003e340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8006e90 tx=0x7fa8b8006e20 comp rx=0 tx=0).stop 2024-08-19T10:43:35.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 10 monclient: shutdown 2024-08-19T10:43:35.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:35.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 -- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8bc139710 msgr2=0x7fa8bc121000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:35.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8bc139710 0x7fa8bc121000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400be90 tx=0x7fa8b400bf80 comp rx=0 tx=0).stop 2024-08-19T10:43:35.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 -- 172.21.15.190:0/166011998 shutdown_connections 2024-08-19T10:43:35.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8bc139710 0x7fa8bc121000 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa8a003be80 0x7fa8a003e340 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8bc0a5d20 0x7fa8bc120ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 --2- 172.21.15.190:0/166011998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa8bc0a5940 0x7fa8bc120580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 -- 172.21.15.190:0/166011998 >> 172.21.15.190:0/166011998 conn(0x7fa8bc0aa530 msgr2=0x7fa8bc141470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:35.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 -- 172.21.15.190:0/166011998 shutdown_connections 2024-08-19T10:43:35.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.390+0000 7fa8c34c8640 1 -- 172.21.15.190:0/166011998 wait complete. 2024-08-19T10:43:35.408 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-08-19T10:43:35.408 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-08-19T10:43:35.408 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush tree --show-shadow 2024-08-19T10:43:35.408 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep 'osd\.0' 2024-08-19T10:43:35.409 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep globster 2024-08-19T10:43:35.409 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep '6\.' 2024-08-19T10:43:35.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient: get_monmap_and_config 2024-08-19T10:43:35.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient: build_initial_monmap 2024-08-19T10:43:35.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient: monmap: 2024-08-19T10:43:35.528 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:35.528 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:35.528 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:35.528883+0000 2024-08-19T10:43:35.528 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:35.528883+0000 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:35.529 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:35.530 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:35.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 1 Processor -- start 2024-08-19T10:43:35.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 1 -- start start 2024-08-19T10:43:35.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient: init 2024-08-19T10:43:35.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:35.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient: _add_conns ranks=[4,9,3,7,2,0,5,6,8,1,10] 2024-08-19T10:43:35.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a540a6740 0x7f2a540a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:35.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient(hunting): picked mon.noname-c con 0x7f2a540a6740 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:35.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2a540a5bc0 0x7f2a540a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:35.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient(hunting): picked mon.noname-a con 0x7f2a540a5bc0 addr v2:172.21.15.89:3300/0 2024-08-19T10:43:35.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a540a7060 0x7f2a54139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:35.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient(hunting): picked mon.noname-g con 0x7f2a540a7060 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:35.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:35.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f2a540a4c20 con 0x7f2a540a5bc0 2024-08-19T10:43:35.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:35.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2a540a3530 con 0x7f2a540a7060 2024-08-19T10:43:35.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:35.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f2a540a3740 con 0x7f2a540a6740 2024-08-19T10:43:35.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:35.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a59851640 10 monclient(hunting): authenticate will time out at 1261.098708s 2024-08-19T10:43:35.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a540a7060 0x7f2a54139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:35.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a52ffd640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a540a6740 msgr2=0x7f2a540a5680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:35.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a540a7060 0x7f2a54139590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:43346/0 (socket says 172.21.15.190:43346) 2024-08-19T10:43:35.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a52ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a540a6740 0x7f2a540a5680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:35.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 1 -- 172.21.15.190:0/3966376677 learned_addr learned my addr 172.21.15.190:0/3966376677 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:35.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 10 monclient(hunting): get_auth_request con 0x7f2a540a7060 auth_method 0 2024-08-19T10:43:35.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:35.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:35.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:35.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:35.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:35.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:35.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a527fc640 1 --2- 172.21.15.190:0/3966376677 >> v2:172.21.15.89:3300/0 conn(0x7f2a540a5bc0 0x7f2a540a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:35.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a527fc640 10 monclient(hunting): get_auth_request con 0x7f2a540a5bc0 auth_method 0 2024-08-19T10:43:35.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:35.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a527fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:35.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a527fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:35.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 10 monclient(hunting): handle_auth_done global_id 34275 payload 274 2024-08-19T10:43:35.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 1 -- 172.21.15.190:0/3966376677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a540a6740 msgr2=0x7f2a540a5680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:35.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 1 --2- 172.21.15.190:0/3966376677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a540a6740 0x7f2a540a5680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 1 -- 172.21.15.190:0/3966376677 >> v2:172.21.15.89:3300/0 conn(0x7f2a540a5bc0 msgr2=0x7f2a540a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:35.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 1 --2- 172.21.15.190:0/3966376677 >> v2:172.21.15.89:3300/0 conn(0x7f2a540a5bc0 0x7f2a540a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:35.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 1 monclient: found mon.noname-g 2024-08-19T10:43:35.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:35.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:43:35.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.526+0000 7f2a537fe640 1 -- 172.21.15.190:0/3966376677 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a540a74d0 con 0x7f2a540a7060 2024-08-19T10:43:35.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a537fe640 1 -- 172.21.15.190:0/3966376677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a540a7060 msgr2=0x7f2a54139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:35.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a537fe640 1 --2- 172.21.15.190:0/3966376677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a540a7060 0x7f2a54139590 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009e80 tx=0x7f2a4c01c8f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c5224b5f087ec129 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:35.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 1 -- 172.21.15.190:0/3966376677 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a4c0090d0 con 0x7f2a540a7060 2024-08-19T10:43:35.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:35.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 10 monclient: dump: 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:35.544 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 10 monclient: _finish_auth 0 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 1 -- 172.21.15.190:0/3966376677 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a4c0a5bc0 con 0x7f2a540a7060 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 1 -- 172.21.15.190:0/3966376677 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a4c0a5d70 con 0x7f2a540a7060 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a59851640 5 monclient: authenticate success, global_id 34275 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a59851640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:35.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a59851640 4 set_mon_vals no callback set 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.530+0000 7f2a51ffb640 10 monclient: dump: 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient: shutdown 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 -- 172.21.15.190:0/3966376677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a540a7060 msgr2=0x7f2a54139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 --2- 172.21.15.190:0/3966376677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a540a7060 0x7f2a54139590 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009e80 tx=0x7f2a4c01c8f0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 -- 172.21.15.190:0/3966376677 shutdown_connections 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 --2- 172.21.15.190:0/3966376677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a540a7060 0x7f2a54139590 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 --2- 172.21.15.190:0/3966376677 >> v2:172.21.15.89:3300/0 conn(0x7f2a540a5bc0 0x7f2a540a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 --2- 172.21.15.190:0/3966376677 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a540a6740 0x7f2a540a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 -- 172.21.15.190:0/3966376677 >> 172.21.15.190:0/3966376677 conn(0x7f2a540abd40 msgr2=0x7f2a540ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 -- 172.21.15.190:0/3966376677 shutdown_connections 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 -- 172.21.15.190:0/3966376677 wait complete. 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient: build_initial_monmap 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient: monmap: 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:35.548 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 Processor -- start 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 -- start start 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient: init 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:35.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient: _add_conns ranks=[5,2,8,7,3,6,0,4,1] 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 0x7f2a541d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient(hunting): picked mon.noname-h con 0x7f2a540a6740 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a7060 0x7f2a541d2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient(hunting): picked mon.noname-b con 0x7f2a540a7060 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2a5409ca60 0x7f2a5409ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient(hunting): picked mon.noname-e con 0x7f2a5409ca60 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f2a54143450 con 0x7f2a540a7060 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2a541432d0 con 0x7f2a540a6740 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:35.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2a541435d0 con 0x7f2a5409ca60 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a59851640 10 monclient(hunting): authenticate will time out at 1261.105841s 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 0x7f2a541d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 0x7f2a541d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:58060/0 (socket says 172.21.15.190:58060) 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 -- 172.21.15.190:0/1368742532 learned_addr learned my addr 172.21.15.190:0/1368742532 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 10 monclient(hunting): get_auth_request con 0x7f2a540a6740 auth_method 0 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:35.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a537fe640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2a5409ca60 0x7f2a5409ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a527fc640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a7060 0x7f2a541d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a537fe640 10 monclient(hunting): get_auth_request con 0x7f2a5409ca60 auth_method 0 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a537fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a537fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a527fc640 10 monclient(hunting): get_auth_request con 0x7f2a540a7060 auth_method 0 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a527fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a527fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a527fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 10 monclient(hunting): handle_auth_done global_id 5873 payload 824 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2a5409ca60 msgr2=0x7f2a5409ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2a5409ca60 0x7f2a5409ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a7060 msgr2=0x7f2a541d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:35.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f2a540a7060 0x7f2a541d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:43:35.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 monclient: found mon.noname-h 2024-08-19T10:43:35.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:35.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:35.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 -- 172.21.15.190:0/1368742532 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a54140160 con 0x7f2a540a6740 2024-08-19T10:43:35.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a537fe640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2a5409ca60 0x7f2a5409ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:43:35.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 msgr2=0x7f2a541d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:43:35.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 msgr2=0x7f2a541d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:43:35.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 0x7f2a541d1c50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800b530 tx=0x7f2a4800b9f0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:35.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.534+0000 7f2a52ffd640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 0x7f2a541d1c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:35.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a52ffd640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 0x7f2a541d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:35.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a52ffd640 10 monclient: get_auth_request con 0x7f2a540a6740 auth_method 0 2024-08-19T10:43:35.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a52ffd640 0 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 0x7f2a541d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-08-19T10:43:35.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a52ffd640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 0x7f2a541d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 10 monclient: ms_handle_reset current mon [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:35.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:35.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 msgr2=0x7f2a541d1c50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:35.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2a540a6740 0x7f2a541d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 10 monclient: _add_conns ranks=[4,1,3,6,0,7] 2024-08-19T10:43:35.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a400035e0 0x7f2a40005a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:35.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 10 monclient(hunting): picked mon.noname-c con 0x7f2a400035e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:35.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a400065d0 0x7f2a40008a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 10 monclient(hunting): picked mon.noname-f con 0x7f2a400065d0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a40009640 0x7f2a4000bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 10 monclient(hunting): picked mon.noname-g con 0x7f2a40009640 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4000c6f0 con 0x7f2a400065d0 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4000c870 con 0x7f2a40009640 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f2a4000ca70 con 0x7f2a400035e0 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a3bfff640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a527fc640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a400065d0 msgr2=0x7f2a40008a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:35.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a52ffd640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a400035e0 msgr2=0x7f2a40005a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a527fc640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a400065d0 0x7f2a40008a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a52ffd640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a400035e0 0x7f2a40005a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a40009640 0x7f2a4000bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 10 monclient(hunting): get_auth_request con 0x7f2a40009640 auth_method 0 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 10 monclient(hunting): handle_auth_done global_id 5873 payload 293 2024-08-19T10:43:35.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a400035e0 msgr2=0x7f2a40005a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2a400035e0 0x7f2a40005a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a400065d0 msgr2=0x7f2a40008a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a400065d0 0x7f2a40008a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 1 monclient: found mon.noname-g 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 1 -- 172.21.15.190:0/1368742532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4000cd30 con 0x7f2a40009640 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.738+0000 7f2a537fe640 1 -- 172.21.15.190:0/1368742532 reap_dead start 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a537fe640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a40009640 msgr2=0x7f2a4000bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a537fe640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a40009640 0x7f2a4000bb00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0ae000 tx=0x7f2a48009210 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a4c0bb070 con 0x7f2a40009640 2024-08-19T10:43:35.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 10 monclient: dump: 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:35.753 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 10 monclient: _finish_auth 0 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a4c009260 con 0x7f2a40009640 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2a4c003e10 con 0x7f2a40009640 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:35.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a517fa640 4 set_mon_vals no callback set 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 10 monclient: dump: 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:35.755 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a59851640 5 monclient: authenticate success, global_id 5873 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a59851640 10 monclient: _renew_subs 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a59851640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a59851640 1 -- 172.21.15.190:0/1368742532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5409f970 con 0x7f2a40009640 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a59851640 10 monclient: _renew_subs 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a59851640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a59851640 1 -- 172.21.15.190:0/1368742532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a540ab2d0 con 0x7f2a40009640 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2a4c0039a0 con 0x7f2a40009640 2024-08-19T10:43:35.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a59851640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:35.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a59851640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:35.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a59851640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:35.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a59851640 1 -- 172.21.15.190:0/1368742532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a540a5680 con 0x7f2a40009640 2024-08-19T10:43:35.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2a40040600 0x7f2a40042ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:35.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(173..173 src has 1..173) ==== 9892+0+0 (secure 0 0 0) 0x7f2a4c0e93c0 con 0x7f2a40009640 2024-08-19T10:43:35.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a52ffd640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2a40040600 0x7f2a40042ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:35.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.742+0000 7f2a52ffd640 10 monclient: get_auth_request con 0x7f2a40040600 auth_method 0 2024-08-19T10:43:35.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.746+0000 7f2a52ffd640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2a40040600 0x7f2a40042ac0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800b160 tx=0x7f2a4800f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:35.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.750+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f2a4c016600 con 0x7f2a40009640 2024-08-19T10:43:35.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.750+0000 7f2a3bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:35.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.750+0000 7f2a3bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:35.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.922+0000 7f2a59851640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:35.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.922+0000 7f2a59851640 10 monclient: _send_command 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:35.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.922+0000 7f2a59851640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:35.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.922+0000 7f2a59851640 1 -- 172.21.15.190:0/1368742532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f2a5413f340 con 0x7f2a40009640 2024-08-19T10:43:35.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.922+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v173) ==== 83+0+2320 (secure 0 0 0) 0x7f2a4c0e69b0 con 0x7f2a40009640 2024-08-19T10:43:35.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.922+0000 7f2a3bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:35.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:35.922+0000 7f2a3bfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:36.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a59851640 10 start_mon_command cmd=[{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:36.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a59851640 10 monclient: _send_command 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:36.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a59851640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:36.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a59851640 1 -- 172.21.15.190:0/1368742532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush tree", "show_shadow": true} v 0) -- 0x7f2a54137580 con 0x7f2a40009640 2024-08-19T10:43:36.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a3bfff640 1 -- 172.21.15.190:0/1368742532 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush tree", "show_shadow": true}]=0 v173) ==== 83+0+2320 (secure 0 0 0) 0x7f2a4c004040 con 0x7f2a40009640 2024-08-19T10:43:36.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a3bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush tree", "show_shadow": true}] 2024-08-19T10:43:36.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a3bfff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:36.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a59851640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2a40040600 msgr2=0x7f2a40042ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:36.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a59851640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2a40040600 0x7f2a40042ac0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800b160 tx=0x7f2a4800f000 comp rx=0 tx=0).stop 2024-08-19T10:43:36.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a59851640 10 monclient: shutdown 2024-08-19T10:43:36.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a59851640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:36.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a59851640 1 -- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a40009640 msgr2=0x7f2a4000bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:36.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.022+0000 7f2a59851640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a40009640 0x7f2a4000bb00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0ae000 tx=0x7f2a48009210 comp rx=0 tx=0).stop 2024-08-19T10:43:36.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.026+0000 7f2a59851640 1 -- 172.21.15.190:0/1368742532 shutdown_connections 2024-08-19T10:43:36.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.026+0000 7f2a59851640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2a40040600 0x7f2a40042ac0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:36.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.026+0000 7f2a59851640 1 --2- 172.21.15.190:0/1368742532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a40009640 0x7f2a4000bb00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:36.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.026+0000 7f2a59851640 1 -- 172.21.15.190:0/1368742532 >> 172.21.15.190:0/1368742532 conn(0x7f2a540abd40 msgr2=0x7f2a541366b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:36.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.026+0000 7f2a59851640 1 -- 172.21.15.190:0/1368742532 shutdown_connections 2024-08-19T10:43:36.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.026+0000 7f2a59851640 1 -- 172.21.15.190:0/1368742532 wait complete. 2024-08-19T10:43:36.041 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-08-19T10:43:36.041 INFO:tasks.workunit.client.0.smithi190.stdout: 0 globster 0.08789 7.50000 8.50000 6.50000 osd.0 2024-08-19T10:43:36.042 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm-device-class osd.0 2024-08-19T10:43:36.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.154+0000 7efdef01b640 10 monclient: get_monmap_and_config 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.154+0000 7efdef01b640 10 monclient: build_initial_monmap 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.154+0000 7efdef01b640 10 monclient: monmap: 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:36.158511+0000 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:36.158511+0000 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:36.158 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:36.159 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:36.159 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:36.159 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:36.159 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:36.159 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:36.159 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:36.159 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:36.159 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:36.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.154+0000 7efdef01b640 1 Processor -- start 2024-08-19T10:43:36.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.154+0000 7efdef01b640 1 -- start start 2024-08-19T10:43:36.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.154+0000 7efdef01b640 10 monclient: init 2024-08-19T10:43:36.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:36.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 10 monclient: _add_conns ranks=[0,4,10,9,5,2,7,6,1,3,8] 2024-08-19T10:43:36.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efde80a6850 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:36.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efde80a6c30 0x7efde80a7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:36.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 10 monclient(hunting): picked mon.noname-c con 0x7efde80a6c30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:36.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efde813ebb0 0x7efde8140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:36.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 10 monclient(hunting): picked mon.noname-f con 0x7efde813ebb0 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde80a5f00 con 0x7efde80a6850 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7efde80a3640 con 0x7efde813ebb0 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7efde80a3850 con 0x7efde80a6c30 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 10 monclient(hunting): authenticate will time out at 1261.728360s 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efded5b8640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7efde813ebb0 msgr2=0x7efde8140fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efded5b8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efde813ebb0 0x7efde8140fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde7fff640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efde80a6c30 msgr2=0x7efde80a7010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde7fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efde80a6c30 0x7efde80a7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:36.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdecdb7640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efde80a6850 0x7efde80a57f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:60232/0 (socket says 172.21.15.190:60232) 2024-08-19T10:43:36.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdecdb7640 1 -- 172.21.15.190:0/3065774654 learned_addr learned my addr 172.21.15.190:0/3065774654 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:43:36.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdecdb7640 1 -- 172.21.15.190:0/3065774654 >> v1:172.21.15.89:6789/0 conn(0x7efde80a6850 legacy=0x7efde80a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:36.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 989610656 0 0) 0x7efde80a5f00 con 0x7efde80a6850 2024-08-19T10:43:36.165 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): _negotiate 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): my global_id is 5886 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdd4003530 con 0x7efde80a6850 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1936124821 0 0) 0x7efdd4003530 con 0x7efde80a6850 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efde80a6c30 msgr2=0x7efde80a7010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efde80a6c30 0x7efde80a7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 >> v2:172.21.15.190:3300/0 conn(0x7efde813ebb0 msgr2=0x7efde8140fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 --2- 172.21.15.190:0/3065774654 >> v2:172.21.15.190:3300/0 conn(0x7efde813ebb0 0x7efde8140fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 monclient: found mon.noname-a-legacy 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:36.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.89:6789/0 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde80a46e0 con 0x7efde80a6850 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient: _finish_auth 0 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 3927028378 0 0) 0x7efdd8001c20 con 0x7efde80a6850 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 5 monclient: authenticate success, global_id 5886 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient: dump: 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:36.167 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:36.168 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:36.168 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:36.168 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:36.168 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:36.168 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:36.168 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:36.168 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] but i'm connected to v1:172.21.15.89:6789/0, reconnecting 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 >> v1:172.21.15.89:6789/0 conn(0x7efde80a6850 legacy=0x7efde80a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient: _add_conns ranks=[2,3,1,0,8,5,7,6,4] 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efdd4004210 0x7efdd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): picked mon.b con 0x7efdd4004210 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:36.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdd4004f00 0x7efdd40099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): picked mon.g con 0x7efdd4004f00 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd4009f00 0x7efdd400c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): picked mon.f con 0x7efdd4009f00 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efde80a3640 con 0x7efdd4009f00 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7efde80a3850 con 0x7efdd4004210 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7efdd400cb60 con 0x7efdd4004f00 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efde77fe640 10 monclient(hunting): _finish_auth 0 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efdef01b640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:36.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.158+0000 7efded5b8640 1 -- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd4009f00 msgr2=0x7efdd400c2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efded5b8640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd4009f00 0x7efdd400c2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdd4004f00 0x7efdd40099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient(hunting): get_auth_request con 0x7efdd4004f00 auth_method 0 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient(hunting): handle_auth_done global_id 5886 payload 293 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efdecdb7640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efdd4004210 0x7efdd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 1 -- 172.21.15.190:0/3065774654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efdd4004210 msgr2=0x7efdd4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:36.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efdd4004210 0x7efdd4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:36.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 1 -- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd4009f00 msgr2=0x7efdd400c2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:36.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd4009f00 0x7efdd400c2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:36.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:43:36.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 1 monclient: found mon.g 2024-08-19T10:43:36.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efded5b8640 1 -- 172.21.15.190:0/3065774654 reap_dead start 2024-08-19T10:43:36.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:36.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:36.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 1 -- 172.21.15.190:0/3065774654 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd4003530 con 0x7efdd4004f00 2024-08-19T10:43:36.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient: _finish_auth 0 2024-08-19T10:43:36.172 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:36.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efde7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:36.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efdecdb7640 10 monclient: get_auth_request con 0x7efdd4004210 auth_method 0 2024-08-19T10:43:36.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.162+0000 7efdecdb7640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efdd4004210 0x7efdd4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:43:36.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde7fff640 1 -- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdd4004f00 msgr2=0x7efdd40099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:36.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde7fff640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdd4004f00 0x7efdd40099c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efdd0003940 tx=0x7efdd001c420 comp rx=0 tx=0).ready entity=mon.3 client_cookie=f905b04364db4f95 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:36.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdd0009520 con 0x7efdd4004f00 2024-08-19T10:43:36.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:36.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:36.178 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde77fe640 10 monclient: dump: 2024-08-19T10:43:36.178 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:36.178 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:36.179 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:36.179 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:36.179 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:36.179 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:36.179 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:36.179 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:36.179 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:36.179 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:36.179 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:36.179 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd00039c0 con 0x7efdd4004f00 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 4 set_mon_vals no callback set 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde77fe640 1 -- 172.21.15.190:0/3065774654 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdd0005070 con 0x7efdd4004f00 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde77fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde77fe640 10 monclient: dump: 2024-08-19T10:43:36.180 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:36.181 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:36.181 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:36.181 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:36.181 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:36.181 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:36.181 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:36.181 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:36.181 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:36.181 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:36.181 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:36.181 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:36.182 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:36.182 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:36.182 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:36.182 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:36.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient: shutdown 2024-08-19T10:43:36.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:36.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 -- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdd4004f00 msgr2=0x7efdd40099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdd4004f00 0x7efdd40099c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efdd0003940 tx=0x7efdd001c420 comp rx=0 tx=0).stop 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 -- 172.21.15.190:0/3065774654 shutdown_connections 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 --2- 172.21.15.190:0/3065774654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdd4004f00 0x7efdd40099c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 -- 172.21.15.190:0/3065774654 >> 172.21.15.190:0/3065774654 conn(0x7efde80abe50 msgr2=0x7efde80ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 -- 172.21.15.190:0/3065774654 shutdown_connections 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 -- 172.21.15.190:0/3065774654 wait complete. 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient: build_initial_monmap 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient: monmap: 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:36.183 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:36.184 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:36.184 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:36.184 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:36.184 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:36.184 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:36.184 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:36.184 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:36.184 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 Processor -- start 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 -- start start 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient: init 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient: _add_conns ranks=[5,2,1,4,6,0,8,3,7] 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efdd4004f00 0x7efde81d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient(hunting): picked mon.noname-h con 0x7efdd4004f00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efde8138690 0x7efde8138ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:36.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient(hunting): picked mon.noname-b con 0x7efde8138690 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efde81224a0 0x7efde81228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient(hunting): picked mon.noname-f con 0x7efde81224a0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efde8139820 con 0x7efde81224a0 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7efde81396a0 con 0x7efde8138690 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7efde81399a0 con 0x7efdd4004f00 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdef01b640 10 monclient(hunting): authenticate will time out at 1261.739162s 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efded5b8640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efde81224a0 msgr2=0x7efde81228c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:36.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efded5b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efde81224a0 0x7efde81228c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efdd4004f00 0x7efde81d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efdd4004f00 0x7efde81d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:58074/0 (socket says 172.21.15.190:58074) 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 1 -- 172.21.15.190:0/3654643236 learned_addr learned my addr 172.21.15.190:0/3654643236 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 10 monclient(hunting): get_auth_request con 0x7efdd4004f00 auth_method 0 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efde7fff640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efde8138690 0x7efde8138ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 10 monclient(hunting): handle_auth_done global_id 5882 payload 824 2024-08-19T10:43:36.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 1 -- 172.21.15.190:0/3654643236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efde8138690 msgr2=0x7efde8138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:36.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efde8138690 0x7efde8138ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:36.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 1 -- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efde81224a0 msgr2=0x7efde81228c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:36.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efde81224a0 0x7efde81228c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:36.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 10 monclient: _finish_hunting 0 2024-08-19T10:43:36.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 1 monclient: found mon.noname-h 2024-08-19T10:43:36.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:36.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.166+0000 7efdecdb7640 1 -- 172.21.15.190:0/3654643236 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde8139010 con 0x7efdd4004f00 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdecdb7640 1 -- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efdd4004f00 msgr2=0x7efde81d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdecdb7640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efdd4004f00 0x7efde81d1140 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efdd800d750 tx=0x7efdd800dc10 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 1 -- 172.21.15.190:0/3654643236 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdd8007110 con 0x7efdd4004f00 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 10 monclient: dump: 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:36.190 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 10 monclient: _finish_auth 0 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:43:36.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 1 -- 172.21.15.190:0/3654643236 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd80072b0 con 0x7efdd4004f00 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 1 -- 172.21.15.190:0/3654643236 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efdd800b640 con 0x7efdd4004f00 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdef01b640 5 monclient: authenticate success, global_id 5882 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdef01b640 10 monclient: _renew_subs 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdef01b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdef01b640 1 -- 172.21.15.190:0/3654643236 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde813e2a0 con 0x7efdd4004f00 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde6ffd640 4 set_mon_vals no callback set 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 10 monclient: dump: 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:36.192 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:36.193 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:36.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdef01b640 10 monclient: _renew_subs 2024-08-19T10:43:36.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdef01b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:36.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdef01b640 1 -- 172.21.15.190:0/3654643236 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7efde81d1680 con 0x7efdd4004f00 2024-08-19T10:43:36.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdef01b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:36.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdef01b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:36.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdef01b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:36.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdef01b640 1 -- 172.21.15.190:0/3654643236 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde80a5e10 con 0x7efdd4004f00 2024-08-19T10:43:36.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efdecdb7640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efdd4004f00 0x7efde81d1140 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efdd800d750 tx=0x7efdd800dc10 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7efdd8012070 mgrmap(e 5) 2024-08-19T10:43:36.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 1 -- 172.21.15.190:0/3654643236 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efdd8012070 con 0x7efdd4004f00 2024-08-19T10:43:36.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde57fa640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efdc803be00 0x7efdc803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:36.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde7fff640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efdc803be00 0x7efdc803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:36.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.170+0000 7efde7fff640 10 monclient: get_auth_request con 0x7efdc803be00 auth_method 0 2024-08-19T10:43:36.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.174+0000 7efde7fff640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efdc803be00 0x7efdc803e2c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efdd00aa000 tx=0x7efdd0005040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:36.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.174+0000 7efde57fa640 1 -- 172.21.15.190:0/3654643236 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(173..173 src has 1..173) ==== 9892+0+0 (secure 0 0 0) 0x7efdd804fa60 con 0x7efdd4004f00 2024-08-19T10:43:37.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.998+0000 7efde57fa640 1 -- 172.21.15.190:0/3654643236 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7efdd801ee20 con 0x7efdd4004f00 2024-08-19T10:43:37.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.998+0000 7efde57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:37.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:36.998+0000 7efde57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:37.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:37.178+0000 7efdef01b640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:37.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:37.178+0000 7efdef01b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:37.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:37.178+0000 7efdef01b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:37.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:37.178+0000 7efdef01b640 1 -- 172.21.15.190:0/3654643236 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7efde806c8d0 con 0x7efdd4004f00 2024-08-19T10:43:37.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:37.250+0000 7efde57fa640 1 -- 172.21.15.190:0/3654643236 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 done removing class of osd(s): 0 v174) ==== 123+0+0 (secure 0 0 0) 0x7efdd80133b0 con 0x7efdd4004f00 2024-08-19T10:43:37.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:37.250+0000 7efde57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:37.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:37.250+0000 7efde57fa640 10 monclient: _finish_command 2 = system:0 done removing class of osd(s): 0 2024-08-19T10:43:37.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:37.350+0000 7efdef01b640 10 start_mon_command cmd=[{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:37.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:37.350+0000 7efdef01b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:37.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:37.350+0000 7efdef01b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:37.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:37.350+0000 7efdef01b640 1 -- 172.21.15.190:0/3654643236 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm-device-class", "ids": ["osd.0"]} v 0) -- 0x7efde8122e00 con 0x7efdd4004f00 2024-08-19T10:43:38.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.270+0000 7efde57fa640 1 -- 172.21.15.190:0/3654643236 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}]=0 osd.0 belongs to no class, done removing class of osd(s): v175) ==== 149+0+0 (secure 0 0 0) 0x7efdd8004df0 con 0x7efdd4004f00 2024-08-19T10:43:38.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.270+0000 7efde57fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm-device-class", "ids": ["osd.0"]}] 2024-08-19T10:43:38.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.270+0000 7efde57fa640 10 monclient: _finish_command 3 = system:0 osd.0 belongs to no class, done removing class of osd(s): 2024-08-19T10:43:38.272 INFO:tasks.workunit.client.0.smithi190.stderr:osd.0 belongs to no class, done removing class of osd(s): 2024-08-19T10:43:38.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 1 -- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efdc803be00 msgr2=0x7efdc803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:38.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efdc803be00 0x7efdc803e2c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efdd00aa000 tx=0x7efdd0005040 comp rx=0 tx=0).stop 2024-08-19T10:43:38.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 10 monclient: shutdown 2024-08-19T10:43:38.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:38.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 1 -- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efdd4004f00 msgr2=0x7efde81d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:38.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efdd4004f00 0x7efde81d1140 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efdd800d750 tx=0x7efdd800dc10 comp rx=0 tx=0).stop 2024-08-19T10:43:38.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 1 -- 172.21.15.190:0/3654643236 shutdown_connections 2024-08-19T10:43:38.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efdc803be00 0x7efdc803e2c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:38.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efde81224a0 0x7efde81228c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:38.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efde8138690 0x7efde8138ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:38.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 1 --2- 172.21.15.190:0/3654643236 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efdd4004f00 0x7efde81d1140 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efdd800d750 tx=0x7efdd800dc10 comp rx=0 tx=0).stop 2024-08-19T10:43:38.277 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.274+0000 7efdef01b640 1 -- 172.21.15.190:0/3654643236 >> 172.21.15.190:0/3654643236 conn(0x7efde80abe50 msgr2=0x7efde813cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:38.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.278+0000 7efdef01b640 1 -- 172.21.15.190:0/3654643236 shutdown_connections 2024-08-19T10:43:38.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.278+0000 7efdef01b640 1 -- 172.21.15.190:0/3654643236 wait complete. 2024-08-19T10:43:38.299 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool rm cool cool --yes-i-really-really-mean-it 2024-08-19T10:43:38.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient: get_monmap_and_config 2024-08-19T10:43:38.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient: build_initial_monmap 2024-08-19T10:43:38.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient: monmap: 2024-08-19T10:43:38.421 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:38.421 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:38.422 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:38.421315+0000 2024-08-19T10:43:38.422 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:38.421315+0000 2024-08-19T10:43:38.422 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:38.422 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:38.422 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:38.422 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:38.422 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:38.422 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:38.423 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:38.423 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:38.423 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:38.423 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:38.423 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:38.423 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:38.423 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:38.423 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:38.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 1 Processor -- start 2024-08-19T10:43:38.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 1 -- start start 2024-08-19T10:43:38.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient: init 2024-08-19T10:43:38.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:38.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient: _add_conns ranks=[8,4,1,10,7,9,6,5,3,2,0] 2024-08-19T10:43:38.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4ed00a69c0 0x7f4ed00a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:38.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient(hunting): picked mon.noname-e con 0x7f4ed00a69c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:38.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed00a6da0 0x7f4ed00a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient(hunting): picked mon.noname-c con 0x7f4ed00a6da0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4ed0139780 addr v1:172.21.15.190:6789/0 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed00a4eb0 con 0x7f4ed0139780 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f4ed00a5c30 con 0x7f4ed00a6da0 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4ed00a3570 con 0x7f4ed00a69c0 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed666c640 10 monclient(hunting): authenticate will time out at 1263.991172s 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ed4c09640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f4ed0139780 legacy=0x7f4ed0139bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecf7fe640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed00a6da0 msgr2=0x7f4ed00a7180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecf7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed00a6da0 0x7f4ed00a7180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:38.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4ed00a69c0 0x7f4ed00a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4ed00a69c0 0x7f4ed00a56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:42140/0 (socket says 172.21.15.190:42140) 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecffff640 1 -- 172.21.15.190:0/688610519 learned_addr learned my addr 172.21.15.190:0/688610519 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecffff640 10 monclient(hunting): get_auth_request con 0x7f4ed00a69c0 auth_method 0 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.418+0000 7f4ecffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.422+0000 7f4ecffff640 10 monclient(hunting): handle_auth_done global_id 5795 payload 274 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.422+0000 7f4ecffff640 1 -- 172.21.15.190:0/688610519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed00a6da0 msgr2=0x7f4ed00a7180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.422+0000 7f4ecffff640 1 --2- 172.21.15.190:0/688610519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed00a6da0 0x7f4ed00a7180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:38.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.422+0000 7f4ecffff640 1 -- 172.21.15.190:0/688610519 >> v1:172.21.15.190:6789/0 conn(0x7f4ed0139780 legacy=0x7f4ed0139bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:38.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.422+0000 7f4ecffff640 10 monclient: _finish_hunting 0 2024-08-19T10:43:38.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.422+0000 7f4ecffff640 1 monclient: found mon.noname-e 2024-08-19T10:43:38.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.422+0000 7f4ecffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:38.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.422+0000 7f4ecffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:43:38.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.422+0000 7f4ecffff640 1 -- 172.21.15.190:0/688610519 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed00a6330 con 0x7f4ed00a69c0 2024-08-19T10:43:38.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecffff640 1 -- 172.21.15.190:0/688610519 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4ed00a69c0 msgr2=0x7f4ed00a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:38.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecffff640 1 --2- 172.21.15.190:0/688610519 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4ed00a69c0 0x7f4ed00a56f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00098a0 tx=0x7f4ec001c400 comp rx=0 tx=0).ready entity=mon.8 client_cookie=af5eb55ca8318350 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:38.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 1 -- 172.21.15.190:0/688610519 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ec00b0070 con 0x7f4ed00a69c0 2024-08-19T10:43:38.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:38.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:43:38.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 10 monclient: dump: 2024-08-19T10:43:38.432 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:38.433 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:38.433 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:38.433 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:38.433 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:38.433 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:38.433 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:38.433 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:38.433 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:38.433 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:38.434 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:38.434 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:38.434 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:38.434 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:38.434 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:38.434 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:38.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 10 monclient: _finish_auth 0 2024-08-19T10:43:38.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:38.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:38.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 5 monclient: authenticate success, global_id 5795 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 1 -- 172.21.15.190:0/688610519 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec00aa810 con 0x7f4ed00a69c0 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 1 -- 172.21.15.190:0/688610519 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ec00a93e0 con 0x7f4ed00a69c0 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 4 set_mon_vals no callback set 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4eceffd640 10 monclient: dump: 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:38.435 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:38.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient: shutdown 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 -- 172.21.15.190:0/688610519 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4ed00a69c0 msgr2=0x7f4ed00a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 --2- 172.21.15.190:0/688610519 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4ed00a69c0 0x7f4ed00a56f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00098a0 tx=0x7f4ec001c400 comp rx=0 tx=0).stop 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 -- 172.21.15.190:0/688610519 shutdown_connections 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 --2- 172.21.15.190:0/688610519 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed00a6da0 0x7f4ed00a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 --2- 172.21.15.190:0/688610519 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4ed00a69c0 0x7f4ed00a56f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 -- 172.21.15.190:0/688610519 >> 172.21.15.190:0/688610519 conn(0x7f4ed00abfc0 msgr2=0x7f4ed00ac3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 -- 172.21.15.190:0/688610519 shutdown_connections 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 -- 172.21.15.190:0/688610519 wait complete. 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient: build_initial_monmap 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient: monmap: 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:38.437 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:38.438 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 Processor -- start 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 -- start start 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient: init 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient: _add_conns ranks=[0,2,4,3,1,7,5,6,8] 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed00a69c0 0x7f4ed013fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient(hunting): picked mon.noname-a con 0x7f4ed00a69c0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ed00a6da0 0x7f4ed01385c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient(hunting): picked mon.noname-b con 0x7f4ed00a6da0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed0139780 0x7f4ed013d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:38.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient(hunting): picked mon.noname-c con 0x7f4ed0139780 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed00a76c0 con 0x7f4ed00a69c0 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed0144410 con 0x7f4ed00a6da0 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f4ed01d2190 con 0x7f4ed0139780 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed666c640 10 monclient(hunting): authenticate will time out at 1263.999305s 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed4c09640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed0139780 msgr2=0x7f4ed013d780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ed4c09640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed0139780 0x7f4ed013d780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed00a69c0 0x7f4ed013fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:38.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecf7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ed00a6da0 0x7f4ed01385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecf7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ed00a6da0 0x7f4ed01385c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:33404/0 (socket says 172.21.15.190:33404) 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecf7fe640 1 -- 172.21.15.190:0/2293157041 learned_addr learned my addr 172.21.15.190:0/2293157041 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecf7fe640 10 monclient(hunting): get_auth_request con 0x7f4ed00a6da0 auth_method 0 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecffff640 10 monclient(hunting): get_auth_request con 0x7f4ed00a69c0 auth_method 0 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.426+0000 7f4ecffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:38.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 10 monclient(hunting): handle_auth_done global_id 5895 payload 824 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 1 -- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed0139780 msgr2=0x7f4ed013d780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed0139780 0x7f4ed013d780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 1 -- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ed00a6da0 msgr2=0x7f4ed01385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ed00a6da0 0x7f4ed01385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 10 monclient: _finish_hunting 0 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 1 monclient: found mon.noname-a 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 1 -- 172.21.15.190:0/2293157041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed01406e0 con 0x7f4ed00a69c0 2024-08-19T10:43:38.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecf7fe640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ed00a6da0 0x7f4ed01385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:43:38.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 1 -- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed00a69c0 msgr2=0x7f4ed013fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:38.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ecffff640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed00a69c0 0x7f4ed013fd50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00ab000 tx=0x7f4ec0019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:38.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 1 -- 172.21.15.190:0/2293157041 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ec00b0040 con 0x7f4ed00a69c0 2024-08-19T10:43:38.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:38.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:43:38.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 10 monclient: dump: 2024-08-19T10:43:38.448 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:38.448 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:38.449 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 10 monclient: _finish_auth 0 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 10 monclient: _check_auth_tickets 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ed666c640 5 monclient: authenticate success, global_id 5895 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ed666c640 10 monclient: _renew_subs 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ed666c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ed666c640 1 -- 172.21.15.190:0/2293157041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed0122f50 con 0x7f4ed00a69c0 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 1 -- 172.21.15.190:0/2293157041 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec0004040 con 0x7f4ed00a69c0 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:38.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ece7fc640 4 set_mon_vals no callback set 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ed666c640 10 monclient: _renew_subs 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ed666c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4ed666c640 1 -- 172.21.15.190:0/2293157041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed00a2840 con 0x7f4ed00a69c0 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 1 -- 172.21.15.190:0/2293157041 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ec0003620 con 0x7f4ed00a69c0 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.430+0000 7f4eccff9640 10 monclient: dump: 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:38.451 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:38.452 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:38.452 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:38.452 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:38.452 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:38.452 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:38.452 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:38.452 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:38.452 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:38.452 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:38.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.434+0000 7f4ed666c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:38.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.434+0000 7f4ed666c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:38.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.434+0000 7f4ed666c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:38.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.434+0000 7f4ed666c640 1 -- 172.21.15.190:0/2293157041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed00a56f0 con 0x7f4ed00a69c0 2024-08-19T10:43:38.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.434+0000 7f4eccff9640 1 -- 172.21.15.190:0/2293157041 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4ec00041f0 con 0x7f4ed00a69c0 2024-08-19T10:43:38.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.434+0000 7f4eccff9640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ebc03be00 0x7f4ebc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:38.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.434+0000 7f4eccff9640 1 -- 172.21.15.190:0/2293157041 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 9203+0+0 (secure 0 0 0) 0x7f4ec00e9170 con 0x7f4ed00a69c0 2024-08-19T10:43:38.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.434+0000 7f4ecf7fe640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ebc03be00 0x7f4ebc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:38.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.434+0000 7f4ecf7fe640 10 monclient: get_auth_request con 0x7f4ebc03be00 auth_method 0 2024-08-19T10:43:38.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.434+0000 7f4ecf7fe640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ebc03be00 0x7f4ebc03e2c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4ed013f860 tx=0x7f4ec4003ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:38.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.446+0000 7f4eccff9640 1 -- 172.21.15.190:0/2293157041 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4ec0016170 con 0x7f4ed00a69c0 2024-08-19T10:43:38.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.446+0000 7f4eccff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:38.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.446+0000 7f4eccff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:38.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.622+0000 7f4ed666c640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:38.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.622+0000 7f4ed666c640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:38.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.622+0000 7f4ed666c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:38.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:38.622+0000 7f4ed666c640 1 -- 172.21.15.190:0/2293157041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4ed013f1f0 con 0x7f4ed00a69c0 2024-08-19T10:43:39.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.290+0000 7f4eccff9640 1 -- 172.21.15.190:0/2293157041 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}]=0 pool 'cool' removed v176) ==== 148+0+0 (secure 0 0 0) 0x7f4ec0003d90 con 0x7f4ed00a69c0 2024-08-19T10:43:39.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.290+0000 7f4eccff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:39.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.290+0000 7f4eccff9640 10 monclient: _finish_command 2 = system:0 pool 'cool' removed 2024-08-19T10:43:39.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.398+0000 7f4ed666c640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:39.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.398+0000 7f4ed666c640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:39.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.398+0000 7f4ed666c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:39.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.398+0000 7f4ed666c640 1 -- 172.21.15.190:0/2293157041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4ed00008e0 con 0x7f4ed00a69c0 2024-08-19T10:43:39.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4eccff9640 1 -- 172.21.15.190:0/2293157041 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}]=0 pool 'cool' does not exist v176) ==== 155+0+0 (secure 0 0 0) 0x7f4ec00a9df0 con 0x7f4ed00a69c0 2024-08-19T10:43:39.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4eccff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "cool", "pool2": "cool", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:39.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4eccff9640 10 monclient: _finish_command 3 = system:0 pool 'cool' does not exist 2024-08-19T10:43:39.403 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'cool' does not exist 2024-08-19T10:43:39.405 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 -- 172.21.15.190:0/2293157041 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ebc03be00 msgr2=0x7f4ebc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ebc03be00 0x7f4ebc03e2c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4ed013f860 tx=0x7f4ec4003ba0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 10 monclient: shutdown 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 -- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed00a69c0 msgr2=0x7f4ed013fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed00a69c0 0x7f4ed013fd50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00ab000 tx=0x7f4ec0019200 comp rx=0 tx=0).stop 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 -- 172.21.15.190:0/2293157041 shutdown_connections 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ebc03be00 0x7f4ebc03e2c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ed0139780 0x7f4ed013d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ed00a6da0 0x7f4ed01385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 --2- 172.21.15.190:0/2293157041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ed00a69c0 0x7f4ed013fd50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 -- 172.21.15.190:0/2293157041 >> 172.21.15.190:0/2293157041 conn(0x7f4ed00abfc0 msgr2=0x7f4ed00aad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:39.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 -- 172.21.15.190:0/2293157041 shutdown_connections 2024-08-19T10:43:39.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.402+0000 7f4ed666c640 1 -- 172.21.15.190:0/2293157041 wait complete. 2024-08-19T10:43:39.422 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd pool rm cold cold --yes-i-really-really-mean-it 2024-08-19T10:43:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient: get_monmap_and_config 2024-08-19T10:43:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient: build_initial_monmap 2024-08-19T10:43:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient: monmap: 2024-08-19T10:43:39.543 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:39.544165+0000 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:39.544165+0000 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:39.544 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:39.545 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:39.545 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:39.545 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:39.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 1 Processor -- start 2024-08-19T10:43:39.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 1 -- start start 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient: init 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient: _add_conns ranks=[0,10,6,8,3,7,5,9,2,4,1] 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f44c00a36a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f44c00a19d0 0x7f44c013c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient(hunting): picked mon.noname-f con 0x7f44c00a19d0 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44c00a4190 0x7f44c013ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient(hunting): picked mon.noname-d con 0x7f44c00a4190 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44c009d050 con 0x7f44c00a36a0 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f44c006f250 con 0x7f44c00a19d0 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:39.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f44c009dec0 con 0x7f44c00a4190 2024-08-19T10:43:39.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:39.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c6b39640 10 monclient(hunting): authenticate will time out at 1265.114160s 2024-08-19T10:43:39.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44bffff640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f44c00a19d0 msgr2=0x7f44c013c8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:39.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44bffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f44c00a19d0 0x7f44c013c8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:39.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c50d6640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44c00a4190 msgr2=0x7f44c013ee10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:39.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c50d6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44c00a4190 0x7f44c013ee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:39.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c48d5640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f44c00a36a0 0x7f44c00a3a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:60236/0 (socket says 172.21.15.190:60236) 2024-08-19T10:43:39.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.542+0000 7f44c48d5640 1 -- 172.21.15.190:0/2044086708 learned_addr learned my addr 172.21.15.190:0/2044086708 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:43:39.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c48d5640 1 -- 172.21.15.190:0/2044086708 >> v1:172.21.15.89:6789/0 conn(0x7f44c00a36a0 legacy=0x7f44c00a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:39.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3590966552 0 0) 0x7f44c009d050 con 0x7f44c00a36a0 2024-08-19T10:43:39.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): _negotiate 2024-08-19T10:43:39.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:39.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:39.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): my global_id is 5904 2024-08-19T10:43:39.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44ac003530 con 0x7f44c00a36a0 2024-08-19T10:43:39.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2127236325 0 0) 0x7f44ac003530 con 0x7f44c00a36a0 2024-08-19T10:43:39.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44c00a4190 msgr2=0x7f44c013ee10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44c00a4190 0x7f44c013ee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 >> v2:172.21.15.190:3300/0 conn(0x7f44c00a19d0 msgr2=0x7f44c013c8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 --2- 172.21.15.190:0/2044086708 >> v2:172.21.15.190:3300/0 conn(0x7f44c00a19d0 0x7f44c013c8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 monclient: found mon.noname-a-legacy 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.89:6789/0 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c009d620 con 0x7f44c00a36a0 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:39.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 3927028378 0 0) 0x7f44b0001ce0 con 0x7f44c00a36a0 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient: dump: 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:39.553 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] but i'm connected to v1:172.21.15.89:6789/0, reconnecting 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 >> v1:172.21.15.89:6789/0 conn(0x7f44c00a36a0 legacy=0x7f44c00a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient: _add_conns ranks=[0,6,5,7,4,2,1,8,3] 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44ac004210 0x7f44ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): picked mon.a con 0x7f44ac004210 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:39.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44c00a4190 0x7f44ac004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): picked mon.d con 0x7f44c00a4190 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44ac0099c0 0x7f44ac00bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): picked mon.h con 0x7f44ac0099c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f44c006f250 con 0x7f44ac004210 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f44c009dec0 con 0x7f44ac0099c0 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f44ac00c5c0 con 0x7f44c00a4190 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bf7fe640 10 monclient(hunting): _finish_auth 0 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bffff640 1 -- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44c00a4190 msgr2=0x7f44ac004f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:39.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44bffff640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44c00a4190 0x7f44ac004f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44ac0099c0 0x7f44ac00bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient(hunting): get_auth_request con 0x7f44ac0099c0 auth_method 0 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c48d5640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44ac004210 0x7f44ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c48d5640 10 monclient(hunting): get_auth_request con 0x7f44ac004210 auth_method 0 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c48d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:39.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c48d5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c48d5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient(hunting): handle_auth_done global_id 5904 payload 293 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 1 -- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44c00a4190 msgr2=0x7f44ac004f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f44c00a4190 0x7f44ac004f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 1 -- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44ac004210 msgr2=0x7f44ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44ac004210 0x7f44ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient: _finish_hunting 0 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 1 monclient: found mon.h 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 1 -- 172.21.15.190:0/2044086708 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac003530 con 0x7f44ac0099c0 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient: _finish_auth 0 2024-08-19T10:43:39.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 10 monclient: _check_auth_tickets 2024-08-19T10:43:39.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c50d6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:39.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c6b39640 5 monclient: authenticate success, global_id 5904 2024-08-19T10:43:39.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c6b39640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:39.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.546+0000 7f44c48d5640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44ac004210 0x7f44ac004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:43:39.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c50d6640 1 -- 172.21.15.190:0/2044086708 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44ac0099c0 msgr2=0x7f44ac00bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:39.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c50d6640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44ac0099c0 0x7f44ac00bdb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009e30 tx=0x7f44b801caf0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=aa72d1078629feb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44b8009a30 con 0x7f44ac0099c0 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44bf7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44bf7fe640 10 monclient: dump: 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:39.565 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b80a9070 con 0x7f44ac0099c0 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44bf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44bf7fe640 1 -- 172.21.15.190:0/2044086708 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44b80ae470 con 0x7f44ac0099c0 2024-08-19T10:43:39.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44bf7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44bf7fe640 10 monclient: dump: 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:39.567 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c6b39640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c6b39640 4 set_mon_vals no callback set 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c6b39640 10 monclient: shutdown 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c6b39640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c6b39640 1 -- 172.21.15.190:0/2044086708 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44ac0099c0 msgr2=0x7f44ac00bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c6b39640 1 --2- 172.21.15.190:0/2044086708 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44ac0099c0 0x7f44ac00bdb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009e30 tx=0x7f44b801caf0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c50d6640 1 -- 172.21.15.190:0/2044086708 reap_dead start 2024-08-19T10:43:39.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c6b39640 1 -- 172.21.15.190:0/2044086708 shutdown_connections 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c6b39640 1 -- 172.21.15.190:0/2044086708 >> 172.21.15.190:0/2044086708 conn(0x7f44c00a4790 msgr2=0x7f44c00a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c6b39640 1 -- 172.21.15.190:0/2044086708 shutdown_connections 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.550+0000 7f44c6b39640 1 -- 172.21.15.190:0/2044086708 wait complete. 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient: build_initial_monmap 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient: monmap: 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:39.569 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 1 Processor -- start 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 1 -- start start 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient: init 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:39.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient: _add_conns ranks=[4,7,2,3,6,0,5,1,8] 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44c01ce8d0 0x7f44c0135780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient(hunting): picked mon.noname-c con 0x7f44c01ce8d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44c0135cc0 0x7f44c011e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient(hunting): picked mon.noname-i con 0x7f44c0135cc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44c011ea90 0x7f44c011eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient(hunting): picked mon.noname-b con 0x7f44c011ea90 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f44c0050c20 con 0x7f44c011ea90 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f44c006a540 con 0x7f44c01ce8d0 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f44c013f620 con 0x7f44c0135cc0 2024-08-19T10:43:39.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c6b39640 10 monclient(hunting): authenticate will time out at 1265.125145s 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44c0135cc0 0x7f44c011e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44c0135cc0 0x7f44c011e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:56246/0 (socket says 172.21.15.190:56246) 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c48d5640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44c01ce8d0 msgr2=0x7f44c0135780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 learned_addr learned my addr 172.21.15.190:0/79181132 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 10 monclient(hunting): get_auth_request con 0x7f44c0135cc0 auth_method 0 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c48d5640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44c01ce8d0 0x7f44c0135780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:39.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c50d6640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44c011ea90 0x7f44c011eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 10 monclient(hunting): handle_auth_done global_id 15127 payload 824 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44c01ce8d0 msgr2=0x7f44c0135780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44c01ce8d0 0x7f44c0135780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44c011ea90 msgr2=0x7f44c011eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44c011ea90 0x7f44c011eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 10 monclient: _finish_hunting 0 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 1 monclient: found mon.noname-i 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c0074ea0 con 0x7f44c0135cc0 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c50d6640 10 monclient: get_auth_request con 0x7f44c011ea90 auth_method 0 2024-08-19T10:43:39.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44c50d6640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f44c011ea90 0x7f44c011eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44c0135cc0 msgr2=0x7f44c011e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.554+0000 7f44bffff640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44c0135cc0 0x7f44c011e550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44a800b530 tx=0x7f44a800b9f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44a8013020 con 0x7f44c0135cc0 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: dump: 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:39.574 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: _finish_auth 0 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a8003de0 con 0x7f44c0135cc0 2024-08-19T10:43:39.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44a800f400 con 0x7f44c0135cc0 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c6b39640 5 monclient: authenticate success, global_id 15127 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: dump: 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:39.576 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44beffd640 4 set_mon_vals no callback set 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c6b39640 10 monclient: _renew_subs 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c6b39640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c6b39640 1 -- 172.21.15.190:0/79181132 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c0121c70 con 0x7f44c0135cc0 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c6b39640 10 monclient: _renew_subs 2024-08-19T10:43:39.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c6b39640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c6b39640 1 -- 172.21.15.190:0/79181132 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c0120a30 con 0x7f44c0135cc0 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44c0135cc0 msgr2=0x7f44c011e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44c0135cc0 msgr2=0x7f44c011e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44c0135cc0 0x7f44c011e550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44a800b530 tx=0x7f44a800b9f0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44c0135cc0 0x7f44c011e550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44a800b530 tx=0x7f44a800b9f0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: ms_handle_reset current mon [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44c0135cc0 msgr2=0x7f44c011e550 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f44c0135cc0 0x7f44c011e550 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: _add_conns ranks=[4,5,0,7,2,1,6,3,8] 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44c01ce8d0 0x7f44a0006da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient(hunting): picked mon.c con 0x7f44c01ce8d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44a00074f0 0x7f44a00098e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient(hunting): picked mon.h con 0x7f44a00074f0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44a000a290 0x7f44a000c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient(hunting): picked mon.a con 0x7f44a000a290 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f44a000d2d0 con 0x7f44a000a290 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f44a000d480 con 0x7f44c01ce8d0 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f44a000d630 con 0x7f44a00074f0 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44a00074f0 0x7f44a00098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:39.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c48d5640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44c01ce8d0 msgr2=0x7f44a0006da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient(hunting): get_auth_request con 0x7f44a00074f0 auth_method 0 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c48d5640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44c01ce8d0 0x7f44a0006da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c6b39640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c6b39640 10 monclient(hunting): _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient(hunting): handle_auth_done global_id 15127 payload 293 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44c50d6640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44a000a290 0x7f44a000c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44c01ce8d0 msgr2=0x7f44a0006da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:39.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f44c01ce8d0 0x7f44a0006da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44a000a290 msgr2=0x7f44a000c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44a000a290 0x7f44a000c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient: _finish_hunting 0 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 monclient: found mon.h 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) -- 0x7f44a000d9b0 con 0x7f44a00074f0 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c01216c0 con 0x7f44a00074f0 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c0120a30 con 0x7f44a00074f0 2024-08-19T10:43:39.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient: _finish_auth 0 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44a00074f0 msgr2=0x7f44a00098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bffff640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44a00074f0 0x7f44a00098e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f44a80059b0 tx=0x7f44a8009210 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f44a8031020 con 0x7f44a00074f0 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: dump: 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:39.582 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a800fa90 con 0x7f44a00074f0 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44bd7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:39.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.558+0000 7f44be7fc640 4 set_mon_vals no callback set 2024-08-19T10:43:39.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.562+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f44a8016020 con 0x7f44a00074f0 2024-08-19T10:43:39.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.562+0000 7f44bd7fa640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f44a0045100 0x7f44a00475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:39.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.562+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 <== mon.5 v2:172.21.15.190:3302/0 4 ==== osd_map(176..176 src has 1..176) ==== 8532+0+0 (secure 0 0 0) 0x7f44a80518b0 con 0x7f44a00074f0 2024-08-19T10:43:39.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.562+0000 7f44c48d5640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f44a0045100 0x7f44a00475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:39.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.562+0000 7f44c48d5640 10 monclient: get_auth_request con 0x7f44a0045100 auth_method 0 2024-08-19T10:43:39.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.562+0000 7f44c48d5640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f44a0045100 0x7f44a00475c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f44b0001e50 tx=0x7f44b00047f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:39.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.566+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 <== mon.5 v2:172.21.15.190:3302/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f44a801d020 con 0x7f44a00074f0 2024-08-19T10:43:39.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.566+0000 7f44bd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:39.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.566+0000 7f44bd7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:39.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.570+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f44c0074ea0 con 0x7f44a00074f0 2024-08-19T10:43:39.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.570+0000 7f44bd7fa640 10 monclient: handle_mon_command_ack 1 not found 2024-08-19T10:43:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.726+0000 7f44c6b39640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.730+0000 7f44c6b39640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.730+0000 7f44c6b39640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:39.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:39.730+0000 7f44c6b39640 1 -- 172.21.15.190:0/79181132 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true} v 0) -- 0x7f44c0073320 con 0x7f44a00074f0 2024-08-19T10:43:40.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.294+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}]=0 pool 'cold' removed v177) ==== 148+0+0 (secure 0 0 0) 0x7f44a800ed30 con 0x7f44a00074f0 2024-08-19T10:43:40.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.294+0000 7f44bd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:40.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.294+0000 7f44bd7fa640 10 monclient: _finish_command 2 = system:0 pool 'cold' removed 2024-08-19T10:43:40.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.414+0000 7f44c6b39640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:40.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.414+0000 7f44c6b39640 10 monclient: _send_command 3 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:40.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.414+0000 7f44c6b39640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:40.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.414+0000 7f44c6b39640 1 -- 172.21.15.190:0/79181132 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true} v 0) -- 0x7f44c00a30e0 con 0x7f44a00074f0 2024-08-19T10:43:40.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.414+0000 7f44bd7fa640 1 -- 172.21.15.190:0/79181132 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}]=0 pool 'cold' does not exist v177) ==== 155+0+0 (secure 0 0 0) 0x7f44a804c020 con 0x7f44a00074f0 2024-08-19T10:43:40.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.414+0000 7f44bd7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd pool rm", "pool": "cold", "pool2": "cold", "yes_i_really_really_mean_it": true}] 2024-08-19T10:43:40.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.414+0000 7f44bd7fa640 10 monclient: _finish_command 3 = system:0 pool 'cold' does not exist 2024-08-19T10:43:40.417 INFO:tasks.workunit.client.0.smithi190.stderr:pool 'cold' does not exist 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c6b39640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f44a0045100 msgr2=0x7f44a00475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c6b39640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f44a0045100 0x7f44a00475c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f44b0001e50 tx=0x7f44b00047f0 comp rx=0 tx=0).stop 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c6b39640 10 monclient: shutdown 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c6b39640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c6b39640 1 -- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44a00074f0 msgr2=0x7f44a00098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c50d6640 1 -- 172.21.15.190:0/79181132 reap_dead start 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c6b39640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44a00074f0 0x7f44a00098e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f44a80059b0 tx=0x7f44a8009210 comp rx=0 tx=0).stop 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c6b39640 1 -- 172.21.15.190:0/79181132 shutdown_connections 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c6b39640 1 --2- 172.21.15.190:0/79181132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f44a00074f0 0x7f44a00098e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c6b39640 1 -- 172.21.15.190:0/79181132 >> 172.21.15.190:0/79181132 conn(0x7f44c00a4790 msgr2=0x7f44c00a6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c6b39640 1 -- 172.21.15.190:0/79181132 shutdown_connections 2024-08-19T10:43:40.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.418+0000 7f44c6b39640 1 -- 172.21.15.190:0/79181132 wait complete. 2024-08-19T10:43:40.435 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set rm-compat 2024-08-19T10:43:40.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient: get_monmap_and_config 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient: build_initial_monmap 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient: monmap: 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:40.553274+0000 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:40.553274+0000 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:40.553 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:40.554 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:40.554 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:40.554 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:40.554 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:40.554 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:40.554 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:40.554 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:40.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 1 Processor -- start 2024-08-19T10:43:40.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 1 -- start start 2024-08-19T10:43:40.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient: init 2024-08-19T10:43:40.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:40.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient: _add_conns ranks=[7,4,5,2,8,1,9,3,10,0,6] 2024-08-19T10:43:40.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3610 0x7f1e100a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient(hunting): picked mon.noname-i con 0x7f1e100a3610 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e100a3f30 0x7f1e101363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient(hunting): picked mon.noname-c con 0x7f1e100a3f30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e10136910 0x7f1e10136d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient(hunting): picked mon.noname-h con 0x7f1e10136910 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f1e10073290 con 0x7f1e100a3f30 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1e100729f0 con 0x7f1e10136910 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f1e10072cd0 con 0x7f1e100a3610 2024-08-19T10:43:40.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e14bc9640 10 monclient(hunting): authenticate will time out at 1266.123792s 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e0e59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3610 0x7f1e100a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e0e59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3610 0x7f1e100a39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:56250/0 (socket says 172.21.15.190:56250) 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e0e59c640 1 -- 172.21.15.190:0/2293584284 learned_addr learned my addr 172.21.15.190:0/2293584284 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e0e59c640 10 monclient(hunting): get_auth_request con 0x7f1e100a3610 auth_method 0 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e0e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e0e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e0e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e0ed9d640 1 --2- 172.21.15.190:0/2293584284 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e10136910 0x7f1e10136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.550+0000 7f1e0dd9b640 1 -- 172.21.15.190:0/2293584284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e100a3f30 msgr2=0x7f1e101363d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0ed9d640 10 monclient(hunting): get_auth_request con 0x7f1e10136910 auth_method 0 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:40.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0dd9b640 1 --2- 172.21.15.190:0/2293584284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e100a3f30 0x7f1e101363d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 10 monclient(hunting): handle_auth_done global_id 15136 payload 274 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 1 -- 172.21.15.190:0/2293584284 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e10136910 msgr2=0x7f1e10136d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 1 --2- 172.21.15.190:0/2293584284 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e10136910 0x7f1e10136d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 1 -- 172.21.15.190:0/2293584284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e100a3f30 msgr2=0x7f1e101363d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 1 --2- 172.21.15.190:0/2293584284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e100a3f30 0x7f1e101363d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:40.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:43:40.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 1 monclient: found mon.noname-i 2024-08-19T10:43:40.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:40.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:40.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.554+0000 7f1e0e59c640 1 -- 172.21.15.190:0/2293584284 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e100a5570 con 0x7f1e100a3610 2024-08-19T10:43:40.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 1 -- 172.21.15.190:0/2293584284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3610 msgr2=0x7f1e100a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:40.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 1 --2- 172.21.15.190:0/2293584284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3610 0x7f1e100a39f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009e80 tx=0x7f1e0001c930 comp rx=0 tx=0).ready entity=mon.7 client_cookie=3ced4eff0c061481 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:40.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 1 -- 172.21.15.190:0/2293584284 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e000ac020 con 0x7f1e100a3610 2024-08-19T10:43:40.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:40.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:40.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 10 monclient: dump: 2024-08-19T10:43:40.564 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:40.564 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:40.564 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:40.564 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:40.564 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:40.564 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:40.564 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:40.564 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 10 monclient: _finish_auth 0 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 1 -- 172.21.15.190:0/2293584284 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e00009110 con 0x7f1e100a3610 2024-08-19T10:43:40.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 5 monclient: authenticate success, global_id 15136 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 4 set_mon_vals no callback set 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 1 -- 172.21.15.190:0/2293584284 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e000a6d10 con 0x7f1e100a3610 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0d59a640 10 monclient: dump: 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:40.566 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient: shutdown 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:40.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 -- 172.21.15.190:0/2293584284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3610 msgr2=0x7f1e100a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:40.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 --2- 172.21.15.190:0/2293584284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3610 0x7f1e100a39f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009e80 tx=0x7f1e0001c930 comp rx=0 tx=0).stop 2024-08-19T10:43:40.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 -- 172.21.15.190:0/2293584284 shutdown_connections 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 --2- 172.21.15.190:0/2293584284 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e10136910 0x7f1e10136d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 --2- 172.21.15.190:0/2293584284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e100a3f30 0x7f1e101363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 --2- 172.21.15.190:0/2293584284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3610 0x7f1e100a39f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 -- 172.21.15.190:0/2293584284 >> 172.21.15.190:0/2293584284 conn(0x7f1e100a4700 msgr2=0x7f1e100a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 -- 172.21.15.190:0/2293584284 shutdown_connections 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 -- 172.21.15.190:0/2293584284 wait complete. 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient: build_initial_monmap 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient: monmap: 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:40.569 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 Processor -- start 2024-08-19T10:43:40.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 -- start start 2024-08-19T10:43:40.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient: init 2024-08-19T10:43:40.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:40.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient: _add_conns ranks=[3,7,5,1,6,2,0,4,8] 2024-08-19T10:43:40.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e100a3610 0x7f1e101378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:40.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient(hunting): picked mon.noname-g con 0x7f1e100a3610 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:40.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3f30 0x7f1e10137e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:40.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient(hunting): picked mon.noname-i con 0x7f1e100a3f30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:40.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e10136910 0x7f1e101d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:40.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient(hunting): picked mon.noname-h con 0x7f1e10136910 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:40.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:40.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1e10073e20 con 0x7f1e100a3610 2024-08-19T10:43:40.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:40.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1e10073ca0 con 0x7f1e10136910 2024-08-19T10:43:40.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f1e10071540 con 0x7f1e100a3f30 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e14bc9640 10 monclient(hunting): authenticate will time out at 1266.130870s 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e100a3610 0x7f1e101378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3f30 0x7f1e10137e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0ed9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e10136910 0x7f1e101d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e100a3610 0x7f1e101378c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:43360/0 (socket says 172.21.15.190:43360) 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 1 -- 172.21.15.190:0/1297181782 learned_addr learned my addr 172.21.15.190:0/1297181782 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0ed9d640 10 monclient(hunting): get_auth_request con 0x7f1e10136910 auth_method 0 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:40.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 10 monclient(hunting): get_auth_request con 0x7f1e100a3f30 auth_method 0 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 10 monclient(hunting): get_auth_request con 0x7f1e100a3610 auth_method 0 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:40.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 10 monclient(hunting): handle_auth_done global_id 15145 payload 824 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 1 -- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e10136910 msgr2=0x7f1e101d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e10136910 0x7f1e101d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 1 -- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e100a3610 msgr2=0x7f1e101378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e100a3610 0x7f1e101378c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 10 monclient: _finish_hunting 0 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 1 monclient: found mon.noname-i 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:40.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.558+0000 7f1e0dd9b640 1 -- 172.21.15.190:0/1297181782 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e10136cf0 con 0x7f1e100a3f30 2024-08-19T10:43:40.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e0dd9b640 1 -- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3f30 msgr2=0x7f1e10137e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:40.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e0dd9b640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3f30 0x7f1e10137e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400b470 tx=0x7f1e0400b930 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:40.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 1 -- 172.21.15.190:0/1297181782 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e040092a0 con 0x7f1e100a3f30 2024-08-19T10:43:40.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:40.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:40.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 10 monclient: dump: 2024-08-19T10:43:40.578 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:40.578 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:40.578 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:40.578 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:40.578 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:40.578 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 10 monclient: _finish_auth 0 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:40.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 1 -- 172.21.15.190:0/1297181782 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e04015070 con 0x7f1e100a3f30 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e14bc9640 5 monclient: authenticate success, global_id 15145 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e14bc9640 10 monclient: _renew_subs 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e14bc9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e14bc9640 1 -- 172.21.15.190:0/1297181782 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e100a10f0 con 0x7f1e100a3f30 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 1 -- 172.21.15.190:0/1297181782 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e04003de0 con 0x7f1e100a3f30 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 10 monclient: dump: 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:40.580 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e0cd99640 4 set_mon_vals no callback set 2024-08-19T10:43:40.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e14bc9640 10 monclient: _renew_subs 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e14bc9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e14bc9640 1 -- 172.21.15.190:0/1297181782 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e10138d10 con 0x7f1e100a3f30 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e14bc9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e14bc9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e14bc9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e14bc9640 1 -- 172.21.15.190:0/1297181782 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e101363d0 con 0x7f1e100a3f30 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 1 -- 172.21.15.190:0/1297181782 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1e0400f070 con 0x7f1e100a3f30 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1df003be30 0x7f1df003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1dfeffd640 1 -- 172.21.15.190:0/1297181782 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(177..177 src has 1..177) ==== 7725+0+0 (secure 0 0 0) 0x7f1e0404edb0 con 0x7f1e100a3f30 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e0e59c640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1df003be30 0x7f1df003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.562+0000 7f1e0e59c640 10 monclient: get_auth_request con 0x7f1df003be30 auth_method 0 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.566+0000 7f1e0e59c640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1df003be30 0x7f1df003e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1e00000f90 tx=0x7f1e000a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:40.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.566+0000 7f1dfeffd640 1 -- 172.21.15.190:0/1297181782 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1e0400d040 con 0x7f1e100a3f30 2024-08-19T10:43:40.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.566+0000 7f1dfeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:40.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.566+0000 7f1dfeffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:40.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.742+0000 7f1e14bc9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:40.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.742+0000 7f1e14bc9640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:40.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.742+0000 7f1e14bc9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:40.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:40.742+0000 7f1e14bc9640 1 -- 172.21.15.190:0/1297181782 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f1e10139e40 con 0x7f1e100a3f30 2024-08-19T10:43:41.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:41.342+0000 7f1dfeffd640 1 -- 172.21.15.190:0/1297181782 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v178) ==== 78+0+0 (secure 0 0 0) 0x7f1e0404cc30 con 0x7f1e100a3f30 2024-08-19T10:43:41.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:41.342+0000 7f1dfeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:41.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:41.342+0000 7f1dfeffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:41.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:41.454+0000 7f1e14bc9640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:41.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:41.454+0000 7f1e14bc9640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:41.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:41.454+0000 7f1e14bc9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:41.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:41.454+0000 7f1e14bc9640 1 -- 172.21.15.190:0/1297181782 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f1e100008e0 con 0x7f1e100a3f30 2024-08-19T10:43:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.754+0000 7f1dfeffd640 1 -- 172.21.15.190:0/1297181782 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v179) ==== 78+0+0 (secure 0 0 0) 0x7f1e04019cc0 con 0x7f1e100a3f30 2024-08-19T10:43:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.754+0000 7f1dfeffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:42.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.754+0000 7f1dfeffd640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 1 -- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1df003be30 msgr2=0x7f1df003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1df003be30 0x7f1df003e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1e00000f90 tx=0x7f1e000a7000 comp rx=0 tx=0).stop 2024-08-19T10:43:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 10 monclient: shutdown 2024-08-19T10:43:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 1 -- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3f30 msgr2=0x7f1e10137e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:42.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3f30 0x7f1e10137e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400b470 tx=0x7f1e0400b930 comp rx=0 tx=0).stop 2024-08-19T10:43:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 1 -- 172.21.15.190:0/1297181782 shutdown_connections 2024-08-19T10:43:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1df003be30 0x7f1df003e2f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e10136910 0x7f1e101d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1e100a3f30 0x7f1e10137e00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:42.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 1 --2- 172.21.15.190:0/1297181782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e100a3610 0x7f1e101378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:42.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.762+0000 7f1e14bc9640 1 -- 172.21.15.190:0/1297181782 >> 172.21.15.190:0/1297181782 conn(0x7f1e100a4700 msgr2=0x7f1e10135750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.766+0000 7f1e14bc9640 1 -- 172.21.15.190:0/1297181782 shutdown_connections 2024-08-19T10:43:42.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.766+0000 7f1e14bc9640 1 -- 172.21.15.190:0/1297181782 wait complete. 2024-08-19T10:43:42.793 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set create-compat 2024-08-19T10:43:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient: get_monmap_and_config 2024-08-19T10:43:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient: build_initial_monmap 2024-08-19T10:43:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient: monmap: 2024-08-19T10:43:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:42.912616+0000 2024-08-19T10:43:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:42.912616+0000 2024-08-19T10:43:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:42.912 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:42.913 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:42.913 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:42.913 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:42.913 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:42.913 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:42.913 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:42.913 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:42.913 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:42.913 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:42.913 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:42.913 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:42.914 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:42.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 1 Processor -- start 2024-08-19T10:43:42.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 1 -- start start 2024-08-19T10:43:42.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient: init 2024-08-19T10:43:42.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:42.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient: _add_conns ranks=[6,0,3,7,9,5,8,1,2,10,4] 2024-08-19T10:43:42.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ad80a6850 0x7f8ad80a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:42.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient(hunting): picked mon.noname-d con 0x7f8ad80a6850 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:42.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8ad80a6c30 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:42.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ad813ebb0 0x7f8ad8140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:42.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient(hunting): picked mon.noname-g con 0x7f8ad813ebb0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:42.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ad80a3850 con 0x7f8ad80a6c30 2024-08-19T10:43:42.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:42.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad80a3640 con 0x7f8ad813ebb0 2024-08-19T10:43:42.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8ad80a46e0 con 0x7f8ad80a6850 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8adddcc640 10 monclient(hunting): authenticate will time out at 1268.482624s 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad77fe640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ad80a6850 msgr2=0x7f8ad80a57f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad77fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ad80a6850 0x7f8ad80a57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ad813ebb0 0x7f8ad8140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ad813ebb0 0x7f8ad8140fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:43376/0 (socket says 172.21.15.190:43376) 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 1 -- 172.21.15.190:0/1536993279 learned_addr learned my addr 172.21.15.190:0/1536993279 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 10 monclient(hunting): get_auth_request con 0x7f8ad813ebb0 auth_method 0 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:42.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 10 monclient(hunting): handle_auth_done global_id 34293 payload 274 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 1 -- 172.21.15.190:0/1536993279 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ad80a6850 msgr2=0x7f8ad80a57f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 1 --2- 172.21.15.190:0/1536993279 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ad80a6850 0x7f8ad80a57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 1 -- 172.21.15.190:0/1536993279 >> v1:172.21.15.89:6789/0 conn(0x7f8ad80a6c30 legacy=0x7f8ad80a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 1 monclient: found mon.noname-g 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:43:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.910+0000 7f8ad7fff640 1 -- 172.21.15.190:0/1536993279 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad80a3350 con 0x7f8ad813ebb0 2024-08-19T10:43:42.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad7fff640 1 -- 172.21.15.190:0/1536993279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ad813ebb0 msgr2=0x7f8ad8140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:42.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad7fff640 1 --2- 172.21.15.190:0/1536993279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ad813ebb0 0x7f8ad8140fa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009e80 tx=0x7f8ad001c930 comp rx=0 tx=0).ready entity=mon.3 client_cookie=b35184958c6b98d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:42.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 1 -- 172.21.15.190:0/1536993279 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ad00091b0 con 0x7f8ad813ebb0 2024-08-19T10:43:42.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:42.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:43:42.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 10 monclient: dump: 2024-08-19T10:43:42.924 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:42.924 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:42.924 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:42.924 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:42.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 10 monclient: _finish_auth 0 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 1 -- 172.21.15.190:0/1536993279 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad00a5980 con 0x7f8ad813ebb0 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8adddcc640 5 monclient: authenticate success, global_id 34293 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8adddcc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8adddcc640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 1 -- 172.21.15.190:0/1536993279 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ad00ad3f0 con 0x7f8ad813ebb0 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8adddcc640 4 set_mon_vals no callback set 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.914+0000 7f8ad67fc640 10 monclient: dump: 2024-08-19T10:43:42.926 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:42.927 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient: shutdown 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 -- 172.21.15.190:0/1536993279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ad813ebb0 msgr2=0x7f8ad8140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 --2- 172.21.15.190:0/1536993279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ad813ebb0 0x7f8ad8140fa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009e80 tx=0x7f8ad001c930 comp rx=0 tx=0).stop 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 -- 172.21.15.190:0/1536993279 shutdown_connections 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 --2- 172.21.15.190:0/1536993279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ad813ebb0 0x7f8ad8140fa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 --2- 172.21.15.190:0/1536993279 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8ad80a6850 0x7f8ad80a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 -- 172.21.15.190:0/1536993279 >> 172.21.15.190:0/1536993279 conn(0x7f8ad80abe50 msgr2=0x7f8ad80ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 -- 172.21.15.190:0/1536993279 shutdown_connections 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 -- 172.21.15.190:0/1536993279 wait complete. 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient: build_initial_monmap 2024-08-19T10:43:42.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient: monmap: 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:42.929 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 Processor -- start 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 -- start start 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient: init 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient: _add_conns ranks=[0,8,2,5,1,7,3,6,4] 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad80a6850 0x7f8ad8123360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient(hunting): picked mon.noname-a con 0x7f8ad80a6850 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8ad80a6c30 0x7f8ad81238a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient(hunting): picked mon.noname-e con 0x7f8ad80a6c30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad813ebb0 0x7f8ad8123de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:42.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient(hunting): picked mon.noname-b con 0x7f8ad813ebb0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8143d00 con 0x7f8ad80a6850 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad80a5f40 con 0x7f8ad813ebb0 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f8ad8143e80 con 0x7f8ad80a6c30 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8adddcc640 10 monclient(hunting): authenticate will time out at 1268.490015s 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8ad80a6c30 0x7f8ad81238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad813ebb0 0x7f8ad8123de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8ad80a6c30 0x7f8ad81238a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:42142/0 (socket says 172.21.15.190:42142) 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 1 -- 172.21.15.190:0/2650730081 learned_addr learned my addr 172.21.15.190:0/2650730081 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:42.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad80a6850 0x7f8ad8123360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 10 monclient(hunting): get_auth_request con 0x7f8ad80a6c30 auth_method 0 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad7fff640 10 monclient(hunting): get_auth_request con 0x7f8ad813ebb0 auth_method 0 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad77fe640 10 monclient(hunting): get_auth_request con 0x7f8ad80a6850 auth_method 0 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:42.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 10 monclient(hunting): handle_auth_done global_id 5804 payload 824 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 1 -- 172.21.15.190:0/2650730081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad813ebb0 msgr2=0x7f8ad8123de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad813ebb0 0x7f8ad8123de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 1 -- 172.21.15.190:0/2650730081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad80a6850 msgr2=0x7f8ad8123360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad80a6850 0x7f8ad8123360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:42.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:43:42.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 1 monclient: found mon.noname-e 2024-08-19T10:43:42.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:42.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:43:42.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.918+0000 7f8ad6ffd640 1 -- 172.21.15.190:0/2650730081 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad8126ea0 con 0x7f8ad80a6c30 2024-08-19T10:43:42.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8ad6ffd640 1 -- 172.21.15.190:0/2650730081 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8ad80a6c30 msgr2=0x7f8ad81238a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:42.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8ad6ffd640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8ad80a6c30 0x7f8ad81238a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00c450 tx=0x7f8acc00c910 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:42.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 1 -- 172.21.15.190:0/2650730081 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8acc012cb0 con 0x7f8ad80a6c30 2024-08-19T10:43:42.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:42.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:43:42.936 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 10 monclient: dump: 2024-08-19T10:43:42.936 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:42.936 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:42.936 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:42.936 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:42.936 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 10 monclient: _finish_auth 0 2024-08-19T10:43:42.937 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 1 -- 172.21.15.190:0/2650730081 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8acc017070 con 0x7f8ad80a6c30 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8adddcc640 5 monclient: authenticate success, global_id 5804 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8adddcc640 10 monclient: _renew_subs 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8adddcc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8adddcc640 1 -- 172.21.15.190:0/2650730081 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad8127180 con 0x7f8ad80a6c30 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 1 -- 172.21.15.190:0/2650730081 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8acc012840 con 0x7f8ad80a6c30 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 10 monclient: dump: 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:42.938 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:42.939 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8ad57fa640 4 set_mon_vals no callback set 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8adddcc640 10 monclient: _renew_subs 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8adddcc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8adddcc640 1 -- 172.21.15.190:0/2650730081 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad8126bc0 con 0x7f8ad80a6c30 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8adddcc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8adddcc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8adddcc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8adddcc640 1 -- 172.21.15.190:0/2650730081 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad80a57f0 con 0x7f8ad80a6c30 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 1 -- 172.21.15.190:0/2650730081 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8acc01c460 con 0x7f8ad80a6c30 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac403be00 0x7f8ac403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8abffff640 1 -- 172.21.15.190:0/2650730081 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(179..179 src has 1..179) ==== 7273+0+0 (secure 0 0 0) 0x7f8acc04fcd0 con 0x7f8ad80a6c30 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8ad77fe640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac403be00 0x7f8ac403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:42.940 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.922+0000 7f8ad77fe640 10 monclient: get_auth_request con 0x7f8ac403be00 auth_method 0 2024-08-19T10:43:42.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.926+0000 7f8ad77fe640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac403be00 0x7f8ac403e2c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80093d0 tx=0x7f8ac8009360 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:42.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.934+0000 7f8abffff640 1 -- 172.21.15.190:0/2650730081 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8acc04cd60 con 0x7f8ad80a6c30 2024-08-19T10:43:42.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.934+0000 7f8abffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:42.941 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:42.934+0000 7f8abffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:43.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.110+0000 7f8adddcc640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:43:43.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.110+0000 7f8adddcc640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:43:43.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.110+0000 7f8adddcc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:43.114 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.110+0000 7f8adddcc640 1 -- 172.21.15.190:0/2650730081 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7f8ad80a92c0 con 0x7f8ad80a6c30 2024-08-19T10:43:43.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.758+0000 7f8abffff640 1 -- 172.21.15.190:0/2650730081 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v180) ==== 82+0+0 (secure 0 0 0) 0x7f8acc04d0e0 con 0x7f8ad80a6c30 2024-08-19T10:43:43.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.758+0000 7f8abffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:43:43.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.758+0000 7f8abffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:43.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.874+0000 7f8adddcc640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:43:43.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.874+0000 7f8adddcc640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:43:43.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.874+0000 7f8adddcc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:43:43.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.874+0000 7f8adddcc640 1 -- 172.21.15.190:0/2650730081 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7f8ad80ac5e0 con 0x7f8ad80a6c30 2024-08-19T10:43:43.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.878+0000 7f8abffff640 1 -- 172.21.15.190:0/2650730081 <== mon.8 v2:172.21.15.89:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v180) ==== 115+0+0 (secure 0 0 0) 0x7f8acc01aa50 con 0x7f8ad80a6c30 2024-08-19T10:43:43.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.878+0000 7f8abffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:43:43.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.878+0000 7f8abffff640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-08-19T10:43:43.881 INFO:tasks.workunit.client.0.smithi190.stderr:compat weight-set already created 2024-08-19T10:43:43.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 -- 172.21.15.190:0/2650730081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac403be00 msgr2=0x7f8ac403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:43.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac403be00 0x7f8ac403e2c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80093d0 tx=0x7f8ac8009360 comp rx=0 tx=0).stop 2024-08-19T10:43:43.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 10 monclient: shutdown 2024-08-19T10:43:43.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:43.883 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 -- 172.21.15.190:0/2650730081 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8ad80a6c30 msgr2=0x7f8ad81238a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:43.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8ad80a6c30 0x7f8ad81238a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00c450 tx=0x7f8acc00c910 comp rx=0 tx=0).stop 2024-08-19T10:43:43.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 -- 172.21.15.190:0/2650730081 shutdown_connections 2024-08-19T10:43:43.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ad813ebb0 0x7f8ad8123de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:43.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ac403be00 0x7f8ac403e2c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:43.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8ad80a6c30 0x7f8ad81238a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:43.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 --2- 172.21.15.190:0/2650730081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ad80a6850 0x7f8ad8123360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:43.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 -- 172.21.15.190:0/2650730081 >> 172.21.15.190:0/2650730081 conn(0x7f8ad80abe50 msgr2=0x7f8ad813ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:43.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 -- 172.21.15.190:0/2650730081 shutdown_connections 2024-08-19T10:43:43.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:43.882+0000 7f8adddcc640 1 -- 172.21.15.190:0/2650730081 wait complete. 2024-08-19T10:43:43.898 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket fooo host 2024-08-19T10:43:44.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.014+0000 7fe4d83a5640 10 monclient: get_monmap_and_config 2024-08-19T10:43:44.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.014+0000 7fe4d83a5640 10 monclient: build_initial_monmap 2024-08-19T10:43:44.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.014+0000 7fe4d83a5640 10 monclient: monmap: 2024-08-19T10:43:44.018 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:44.018 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:44.018 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:44.018457+0000 2024-08-19T10:43:44.018 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:44.018457+0000 2024-08-19T10:43:44.018 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:44.018 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:44.018 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:44.018 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:44.018 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:44.019 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:44.019 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:44.019 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:44.019 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:44.019 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:44.019 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:44.019 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:44.019 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:44.019 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:44.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.014+0000 7fe4d83a5640 1 Processor -- start 2024-08-19T10:43:44.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.014+0000 7fe4d83a5640 1 -- start start 2024-08-19T10:43:44.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.014+0000 7fe4d83a5640 10 monclient: init 2024-08-19T10:43:44.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:44.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 10 monclient: _add_conns ranks=[8,2,4,0,1,5,10,3,9,6,7] 2024-08-19T10:43:44.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe4d0139980 0x7fe4d0139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:44.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 10 monclient(hunting): picked mon.noname-e con 0x7fe4d0139980 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:44.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4d013a2a0 0x7fe4d00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:44.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 10 monclient(hunting): picked mon.noname-b con 0x7fe4d013a2a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:44.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4d00a5c70 0x7fe4d00a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 10 monclient(hunting): picked mon.noname-c con 0x7fe4d00a5c70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d009ca60 con 0x7fe4d013a2a0 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fe4d00a3640 con 0x7fe4d00a5c70 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fe4d00a8470 con 0x7fe4d0139980 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d83a5640 10 monclient(hunting): authenticate will time out at 1269.588311s 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6942640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4d00a5c70 msgr2=0x7fe4d00a67e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6942640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4d00a5c70 0x7fe4d00a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4d013a2a0 0x7fe4d00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:44.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6141640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe4d0139980 0x7fe4d0139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4d013a2a0 0x7fe4d00a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:33436/0 (socket says 172.21.15.190:33436) 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6141640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe4d0139980 0x7fe4d0139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:42152/0 (socket says 172.21.15.190:42152) 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 1 -- 172.21.15.190:0/3836437533 learned_addr learned my addr 172.21.15.190:0/3836437533 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 10 monclient(hunting): get_auth_request con 0x7fe4d013a2a0 auth_method 0 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6141640 10 monclient(hunting): get_auth_request con 0x7fe4d0139980 auth_method 0 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6141640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6141640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6141640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:44.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6141640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6141640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6141640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 10 monclient(hunting): handle_auth_done global_id 5924 payload 274 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 1 -- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe4d0139980 msgr2=0x7fe4d0139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 1 --2- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe4d0139980 0x7fe4d0139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 1 -- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4d00a5c70 msgr2=0x7fe4d00a67e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 1 --2- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4d00a5c70 0x7fe4d00a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 10 monclient: _finish_hunting 0 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 1 monclient: found mon.noname-b 2024-08-19T10:43:44.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:44.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:44.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d5940640 1 -- 172.21.15.190:0/3836437533 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d00a3850 con 0x7fe4d013a2a0 2024-08-19T10:43:44.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.018+0000 7fe4d6141640 1 --2- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe4d0139980 0x7fe4d0139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:43:44.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d5940640 1 -- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4d013a2a0 msgr2=0x7fe4d00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:44.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d5940640 1 --2- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4d013a2a0 0x7fe4d00a5730 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00098d0 tx=0x7fe4c001c170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8135728c3313108c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:44.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 1 -- 172.21.15.190:0/3836437533 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe4c00b0070 con 0x7fe4d013a2a0 2024-08-19T10:43:44.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:44.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:44.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 10 monclient: dump: 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:44.030 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 10 monclient: _finish_auth 0 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 10 monclient: _check_auth_tickets 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 1 -- 172.21.15.190:0/3836437533 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c00aa810 con 0x7fe4d013a2a0 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 1 -- 172.21.15.190:0/3836437533 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe4c00aae30 con 0x7fe4d013a2a0 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:44.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d513f640 10 monclient: dump: 2024-08-19T10:43:44.032 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:44.032 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:44.032 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:44.032 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:44.032 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:44.032 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:44.032 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:44.032 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:44.032 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 5 monclient: authenticate success, global_id 5924 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 4 set_mon_vals no callback set 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 10 monclient: shutdown 2024-08-19T10:43:44.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4d013a2a0 msgr2=0x7fe4d00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 1 --2- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4d013a2a0 0x7fe4d00a5730 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00098d0 tx=0x7fe4c001c170 comp rx=0 tx=0).stop 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3836437533 shutdown_connections 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 1 --2- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe4d00a5c70 0x7fe4d00a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 1 --2- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe4d013a2a0 0x7fe4d00a5730 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 1 --2- 172.21.15.190:0/3836437533 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe4d0139980 0x7fe4d0139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3836437533 >> 172.21.15.190:0/3836437533 conn(0x7fe4d00abde0 msgr2=0x7fe4d00ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3836437533 shutdown_connections 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.022+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3836437533 wait complete. 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient: build_initial_monmap 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient: monmap: 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:44.034 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:44.035 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 1 Processor -- start 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 1 -- start start 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient: init 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient: _add_conns ranks=[7,6,1,0,3,8,5,4,2] 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe4d00a5c70 0x7fe4d013a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient(hunting): picked mon.noname-i con 0x7fe4d00a5c70 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe4d0139980 0x7fe4d013abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient(hunting): picked mon.noname-d con 0x7fe4d0139980 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4d013b110 0x7fe4d0133540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient(hunting): picked mon.noname-f con 0x7fe4d013b110 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:44.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d0143270 con 0x7fe4d013b110 2024-08-19T10:43:44.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:44.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fe4d01430f0 con 0x7fe4d0139980 2024-08-19T10:43:44.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:44.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fe4d01433f0 con 0x7fe4d00a5c70 2024-08-19T10:43:44.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:44.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d83a5640 10 monclient(hunting): authenticate will time out at 1269.597053s 2024-08-19T10:43:44.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6942640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4d013b110 msgr2=0x7fe4d0133540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:44.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6942640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4d013b110 0x7fe4d0133540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:44.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe4d00a5c70 0x7fe4d013a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:44.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe4d00a5c70 0x7fe4d013a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:56268/0 (socket says 172.21.15.190:56268) 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d5940640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe4d0139980 msgr2=0x7fe4d013abd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 -- 172.21.15.190:0/3366311487 learned_addr learned my addr 172.21.15.190:0/3366311487 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 10 monclient(hunting): get_auth_request con 0x7fe4d00a5c70 auth_method 0 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d5940640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe4d0139980 0x7fe4d013abd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 10 monclient(hunting): handle_auth_done global_id 15154 payload 824 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 -- 172.21.15.190:0/3366311487 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe4d0139980 msgr2=0x7fe4d013abd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:44.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe4d0139980 0x7fe4d013abd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:44.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 -- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4d013b110 msgr2=0x7fe4d0133540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:44.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4d013b110 0x7fe4d0133540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:44.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 10 monclient: _finish_hunting 0 2024-08-19T10:43:44.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 monclient: found mon.noname-i 2024-08-19T10:43:44.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:44.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:44.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 -- 172.21.15.190:0/3366311487 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d013ce10 con 0x7fe4d00a5c70 2024-08-19T10:43:44.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 -- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe4d00a5c70 msgr2=0x7fe4d013a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe4d00a5c70 0x7fe4d013a690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00ebe0 tx=0x7fe4cc00c520 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4d6141640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe4d00a5c70 0x7fe4d013a690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00ebe0 tx=0x7fe4cc00c520 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fe4cc017070 mon_map magic: 0 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4c6ffd640 1 -- 172.21.15.190:0/3366311487 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe4cc017070 con 0x7fe4d00a5c70 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.026+0000 7fe4c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 10 monclient: dump: 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:44.042 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:44.043 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:44.043 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:44.043 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:44.043 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:44.043 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:44.043 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:44.043 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:44.043 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:44.043 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:44.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:43:44.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:44.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:44.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d83a5640 5 monclient: authenticate success, global_id 15154 2024-08-19T10:43:44.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 1 -- 172.21.15.190:0/3366311487 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc00ce40 con 0x7fe4d00a5c70 2024-08-19T10:43:44.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d83a5640 10 monclient: _renew_subs 2024-08-19T10:43:44.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d83a5640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:44.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3366311487 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d013d0f0 con 0x7fe4d00a5c70 2024-08-19T10:43:44.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:44.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d83a5640 10 monclient: _renew_subs 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d83a5640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3366311487 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d013bbd0 con 0x7fe4d00a5c70 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d493e640 4 set_mon_vals no callback set 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d83a5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d83a5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d83a5640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3366311487 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d0139d60 con 0x7fe4d00a5c70 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 1 -- 172.21.15.190:0/3366311487 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe4cc013d50 con 0x7fe4d00a5c70 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 10 monclient: dump: 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:44.046 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:44.047 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:44.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 1 -- 172.21.15.190:0/3366311487 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe4cc01a070 con 0x7fe4d00a5c70 2024-08-19T10:43:44.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4c6ffd640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe4b403be50 0x7fe4b403e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:44.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d5940640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe4b403be50 0x7fe4b403e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:44.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d5940640 10 monclient: get_auth_request con 0x7fe4b403be50 auth_method 0 2024-08-19T10:43:44.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.030+0000 7fe4d5940640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe4b403be50 0x7fe4b403e310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00098d0 tx=0x7fe4c00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:44.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.034+0000 7fe4c6ffd640 1 -- 172.21.15.190:0/3366311487 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(180..180 src has 1..180) ==== 7601+0+0 (secure 0 0 0) 0x7fe4cc00f040 con 0x7fe4d00a5c70 2024-08-19T10:43:44.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.650+0000 7fe4c6ffd640 1 -- 172.21.15.190:0/3366311487 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe4cc051e20 con 0x7fe4d00a5c70 2024-08-19T10:43:44.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.650+0000 7fe4c6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:44.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.650+0000 7fe4c6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:44.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.834+0000 7fe4d83a5640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-08-19T10:43:44.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.834+0000 7fe4d83a5640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-08-19T10:43:44.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.834+0000 7fe4d83a5640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:44.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:44.834+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3366311487 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"} v 0) -- 0x7fe4d0077780 con 0x7fe4d00a5c70 2024-08-19T10:43:45.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.610+0000 7fe4c6ffd640 1 -- 172.21.15.190:0/3366311487 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}]=0 added bucket fooo type host to crush map v181) ==== 140+0+0 (secure 0 0 0) 0x7fe4cc021070 con 0x7fe4d00a5c70 2024-08-19T10:43:45.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.610+0000 7fe4c6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-08-19T10:43:45.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.610+0000 7fe4c6ffd640 10 monclient: _finish_command 2 = system:0 added bucket fooo type host to crush map 2024-08-19T10:43:45.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.718+0000 7fe4d83a5640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-08-19T10:43:45.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.718+0000 7fe4d83a5640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-08-19T10:43:45.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.718+0000 7fe4d83a5640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:45.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.718+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3366311487 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"} v 0) -- 0x7fe4d00a5900 con 0x7fe4d00a5c70 2024-08-19T10:43:45.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.718+0000 7fe4c6ffd640 1 -- 172.21.15.190:0/3366311487 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}]=0 bucket 'fooo' already exists v181) ==== 128+0+0 (secure 0 0 0) 0x7fe4cc012df0 con 0x7fe4d00a5c70 2024-08-19T10:43:45.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.718+0000 7fe4c6ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "fooo", "type": "host"}] 2024-08-19T10:43:45.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.718+0000 7fe4c6ffd640 10 monclient: _finish_command 3 = system:0 bucket 'fooo' already exists 2024-08-19T10:43:45.721 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'fooo' already exists 2024-08-19T10:43:45.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe4b403be50 msgr2=0x7fe4b403e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:45.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe4b403be50 0x7fe4b403e310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00098d0 tx=0x7fe4c00a6000 comp rx=0 tx=0).stop 2024-08-19T10:43:45.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 10 monclient: shutdown 2024-08-19T10:43:45.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:45.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe4d00a5c70 msgr2=0x7fe4d013a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:45.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe4d00a5c70 0x7fe4d013a690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00ebe0 tx=0x7fe4cc00c520 comp rx=0 tx=0).stop 2024-08-19T10:43:45.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3366311487 shutdown_connections 2024-08-19T10:43:45.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe4b403be50 0x7fe4b403e310 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:45.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4d013b110 0x7fe4d0133540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:45.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fe4d0139980 0x7fe4d013abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:45.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 --2- 172.21.15.190:0/3366311487 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe4d00a5c70 0x7fe4d013a690 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:45.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3366311487 >> 172.21.15.190:0/3366311487 conn(0x7fe4d00abde0 msgr2=0x7fe4d0134d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:45.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3366311487 shutdown_connections 2024-08-19T10:43:45.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.722+0000 7fe4d83a5640 1 -- 172.21.15.190:0/3366311487 wait complete. 2024-08-19T10:43:45.739 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush move fooo root=default 2024-08-19T10:43:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.858+0000 7fb11c60a640 10 monclient: get_monmap_and_config 2024-08-19T10:43:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.858+0000 7fb11c60a640 10 monclient: build_initial_monmap 2024-08-19T10:43:45.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.858+0000 7fb11c60a640 10 monclient: monmap: 2024-08-19T10:43:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:45.862328+0000 2024-08-19T10:43:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:45.862328+0000 2024-08-19T10:43:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:45.862 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:45.863 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:45.863 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.858+0000 7fb11c60a640 1 Processor -- start 2024-08-19T10:43:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.858+0000 7fb11c60a640 1 -- start start 2024-08-19T10:43:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.858+0000 7fb11c60a640 10 monclient: init 2024-08-19T10:43:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 10 monclient: _add_conns ranks=[6,10,1,3,9,7,0,2,8,4,5] 2024-08-19T10:43:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1140a3610 0x7fb1140a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 10 monclient(hunting): picked mon.noname-d con 0x7fb1140a3610 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1140a3f30 0x7fb1141373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 10 monclient(hunting): picked mon.noname-f con 0x7fb1140a3f30 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:45.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb114137920 addr v1:172.21.15.190:6789/0 2024-08-19T10:43:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb114074380 con 0x7fb114137920 2024-08-19T10:43:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb114070830 con 0x7fb1140a3f30 2024-08-19T10:43:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb114070170 con 0x7fb1140a3610 2024-08-19T10:43:45.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11c60a640 10 monclient(hunting): authenticate will time out at 1271.432206s 2024-08-19T10:43:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11aba7640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fb114137920 legacy=0x7fb11413e8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb119ba5640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fb1140a3f30 msgr2=0x7fb1141373e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb119ba5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1140a3f30 0x7fb1141373e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11a3a6640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1140a3610 msgr2=0x7fb1140a39f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:45.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:45.862+0000 7fb11a3a6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1140a3610 0x7fb1140a39f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:46.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:46.066+0000 7fb11aba7640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fb114137920 legacy=0x7fb11413e8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:46.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:46.066+0000 7fb11a3a6640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1140a3610 msgr2=0x7fb1140a39f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:46.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:46.066+0000 7fb11a3a6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1140a3610 0x7fb1140a39f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:43:46.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:46.066+0000 7fb119ba5640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fb1140a3f30 msgr2=0x7fb1141373e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:46.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:46.066+0000 7fb119ba5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1140a3f30 0x7fb1141373e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:43:46.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:46.470+0000 7fb11aba7640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fb114137920 legacy=0x7fb11413e8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:46.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:46.470+0000 7fb11a3a6640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1140a3610 msgr2=0x7fb1140a39f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:46.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:46.470+0000 7fb11a3a6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1140a3610 0x7fb1140a39f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:43:46.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:46.470+0000 7fb119ba5640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fb1140a3f30 msgr2=0x7fb1141373e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:46.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:46.470+0000 7fb119ba5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1140a3f30 0x7fb1141373e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:43:47.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:47.274+0000 7fb119ba5640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fb1140a3f30 msgr2=0x7fb1141373e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:47.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:47.274+0000 7fb119ba5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1140a3f30 0x7fb1141373e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:43:47.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:47.274+0000 7fb11aba7640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fb114137920 legacy=0x7fb11413e8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:47.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:47.274+0000 7fb11a3a6640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1140a3610 msgr2=0x7fb1140a39f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:47.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:47.274+0000 7fb11a3a6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1140a3610 0x7fb1140a39f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:43:47.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:47.274+0000 7fb1193a4640 1 -- DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:43:48.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 10 monclient(hunting): tick 2024-08-19T10:43:48.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:43:48.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 1 monclient(hunting): continuing hunt 2024-08-19T10:43:48.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:43:48.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1140a3610 msgr2=0x7fb1140a39f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:48.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb1140a3610 0x7fb1140a39f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:48.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fb1140a3f30 msgr2=0x7fb1141373e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:48.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1140a3f30 0x7fb1141373e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:48.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fb114137920 legacy=0x7fb11413e8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:48.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 10 monclient: _add_conns ranks=[8,0,4,7,9,3,5,2] 2024-08-19T10:43:48.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0f8003830 0x7fb0f8005c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:48.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 10 monclient(hunting): picked mon.noname-e con 0x7fb0f8003830 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:48.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb0f8006280 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:48.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0f8009f50 0x7fb0f800c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:48.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 10 monclient(hunting): picked mon.noname-c con 0x7fb0f8009f50 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:48.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1140740a0 con 0x7fb0f8006280 2024-08-19T10:43:48.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:48.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb114070170 con 0x7fb0f8009f50 2024-08-19T10:43:48.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:48.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb0f800dbe0 con 0x7fb0f8003830 2024-08-19T10:43:48.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb10bfff640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:48.865 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11aba7640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0f8009f50 msgr2=0x7fb0f800c410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:48.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11aba7640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0f8009f50 0x7fb0f800c410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:48.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb119ba5640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fb0f8006280 0x7fb0f8008710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:37568/0 (socket says 172.21.15.190:37568) 2024-08-19T10:43:48.866 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb119ba5640 1 -- 172.21.15.190:0/3401868521 learned_addr learned my addr 172.21.15.190:0/3401868521 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:43:48.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 1 --2- 172.21.15.190:0/3401868521 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0f8003830 0x7fb0f8005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:48.867 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 10 monclient(hunting): get_auth_request con 0x7fb0f8003830 auth_method 0 2024-08-19T10:43:48.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:48.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:48.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:48.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:48.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:48.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:48.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb119ba5640 1 -- 172.21.15.190:0/3401868521 >> v1:172.21.15.89:6789/0 conn(0x7fb0f8006280 legacy=0x7fb0f8008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:48.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 10 monclient(hunting): handle_auth_done global_id 5822 payload 274 2024-08-19T10:43:48.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 1 -- 172.21.15.190:0/3401868521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0f8009f50 msgr2=0x7fb0f800c410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:48.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 1 --2- 172.21.15.190:0/3401868521 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0f8009f50 0x7fb0f800c410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:48.869 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 1 -- 172.21.15.190:0/3401868521 >> v1:172.21.15.89:6789/0 conn(0x7fb0f8006280 legacy=0x7fb0f8008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:48.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 10 monclient: _finish_hunting 0 2024-08-19T10:43:48.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 1 monclient: found mon.noname-e 2024-08-19T10:43:48.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11aba7640 1 -- 172.21.15.190:0/3401868521 reap_dead start 2024-08-19T10:43:48.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:48.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:43:48.870 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.862+0000 7fb11a3a6640 1 -- 172.21.15.190:0/3401868521 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb114074380 con 0x7fb0f8003830 2024-08-19T10:43:48.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11a3a6640 1 -- 172.21.15.190:0/3401868521 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0f8003830 msgr2=0x7fb0f8005c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:48.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11a3a6640 1 --2- 172.21.15.190:0/3401868521 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0f8003830 0x7fb0f8005c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb110003cc0 tx=0x7fb11001ca90 comp rx=0 tx=0).ready entity=mon.8 client_cookie=87dad81e41dc2991 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:48.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 1 -- 172.21.15.190:0/3401868521 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1100a5ba0 con 0x7fb0f8003830 2024-08-19T10:43:48.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:48.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:43:48.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 10 monclient: dump: 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:48.877 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 10 monclient: _finish_auth 0 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 10 monclient: _check_auth_tickets 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 1 -- 172.21.15.190:0/3401868521 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1100a9070 con 0x7fb0f8003830 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 1 -- 172.21.15.190:0/3401868521 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb1100a5d80 con 0x7fb0f8003830 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb1193a4640 10 monclient: dump: 2024-08-19T10:43:48.878 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:48.879 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:48.880 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:48.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 5 monclient: authenticate success, global_id 5822 2024-08-19T10:43:48.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:48.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 4 set_mon_vals no callback set 2024-08-19T10:43:48.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 10 monclient: shutdown 2024-08-19T10:43:48.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:48.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 1 -- 172.21.15.190:0/3401868521 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0f8003830 msgr2=0x7fb0f8005c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:48.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 1 --2- 172.21.15.190:0/3401868521 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0f8003830 0x7fb0f8005c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb110003cc0 tx=0x7fb11001ca90 comp rx=0 tx=0).stop 2024-08-19T10:43:48.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 1 -- 172.21.15.190:0/3401868521 shutdown_connections 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 1 --2- 172.21.15.190:0/3401868521 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0f8003830 0x7fb0f8005c80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 1 -- 172.21.15.190:0/3401868521 >> 172.21.15.190:0/3401868521 conn(0x7fb1140a4700 msgr2=0x7fb1140a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 1 -- 172.21.15.190:0/3401868521 shutdown_connections 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 1 -- 172.21.15.190:0/3401868521 wait complete. 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 10 monclient: build_initial_monmap 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.874+0000 7fb11c60a640 10 monclient: monmap: 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:48.893 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 1 Processor -- start 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 1 -- start start 2024-08-19T10:43:48.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 10 monclient: init 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 10 monclient: _add_conns ranks=[2,0,6,8,7,1,5,4,3] 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1141357e0 0x7fb114135bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 10 monclient(hunting): picked mon.noname-b con 0x7fb1141357e0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb11409da50 0x7fb11409de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 10 monclient(hunting): picked mon.noname-a con 0x7fb11409da50 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb11409e370 0x7fb11409e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 10 monclient(hunting): picked mon.noname-d con 0x7fb11409e370 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb1140a4310 con 0x7fb11409da50 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb114070a30 con 0x7fb1141357e0 2024-08-19T10:43:48.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb1140a4ce0 con 0x7fb11409e370 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11c60a640 10 monclient(hunting): authenticate will time out at 1274.449052s 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11aba7640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb11409e370 msgr2=0x7fb11409e7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11aba7640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb11409e370 0x7fb11409e7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb11409da50 0x7fb11409de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb11409da50 0x7fb11409de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:52216/0 (socket says 172.21.15.190:52216) 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 1 -- 172.21.15.190:0/3454292293 learned_addr learned my addr 172.21.15.190:0/3454292293 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 10 monclient(hunting): get_auth_request con 0x7fb11409da50 auth_method 0 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:48.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11a3a6640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1141357e0 0x7fb114135bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11a3a6640 10 monclient(hunting): get_auth_request con 0x7fb1141357e0 auth_method 0 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11a3a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11a3a6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11a3a6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11a3a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11a3a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb11a3a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 10 monclient(hunting): handle_auth_done global_id 5931 payload 824 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 1 -- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb11409e370 msgr2=0x7fb11409e7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:48.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb11409e370 0x7fb11409e7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 1 -- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1141357e0 msgr2=0x7fb114135bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1141357e0 0x7fb114135bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 10 monclient: _finish_hunting 0 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 1 monclient: found mon.noname-a 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.878+0000 7fb119ba5640 1 -- 172.21.15.190:0/3454292293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb114122660 con 0x7fb11409da50 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb119ba5640 1 -- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb11409da50 msgr2=0x7fb11409de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb119ba5640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb11409da50 0x7fb11409de30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb10400b4c0 tx=0x7fb10400b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 1 -- 172.21.15.190:0/3454292293 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb10400dc70 con 0x7fb11409da50 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:43:48.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 10 monclient: dump: 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:48.899 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 10 monclient: _finish_auth 0 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 1 -- 172.21.15.190:0/3454292293 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb10400de30 con 0x7fb11409da50 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 1 -- 172.21.15.190:0/3454292293 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb10400f8d0 con 0x7fb11409da50 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb118ba3640 4 set_mon_vals no callback set 2024-08-19T10:43:48.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 10 monclient: dump: 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:48.901 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11c60a640 5 monclient: authenticate success, global_id 5931 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11c60a640 10 monclient: _renew_subs 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11c60a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11c60a640 1 -- 172.21.15.190:0/3454292293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb114136cf0 con 0x7fb11409da50 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11c60a640 10 monclient: _renew_subs 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11c60a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11c60a640 1 -- 172.21.15.190:0/3454292293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb114122ee0 con 0x7fb11409da50 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11c60a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11c60a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:48.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11c60a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:48.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11c60a640 1 -- 172.21.15.190:0/3454292293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb114137db0 con 0x7fb11409da50 2024-08-19T10:43:48.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 1 -- 172.21.15.190:0/3454292293 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb10400f460 con 0x7fb11409da50 2024-08-19T10:43:48.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0f803d230 0x7fb0f803f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:48.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb10affd640 1 -- 172.21.15.190:0/3454292293 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 7629+0+0 (secure 0 0 0) 0x7fb10404f970 con 0x7fb11409da50 2024-08-19T10:43:48.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.882+0000 7fb11a3a6640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0f803d230 0x7fb0f803f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:48.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.886+0000 7fb11a3a6640 10 monclient: get_auth_request con 0x7fb0f803d230 auth_method 0 2024-08-19T10:43:48.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.886+0000 7fb11a3a6640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0f803d230 0x7fb0f803f6f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8003750 tx=0x7fb1100a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:48.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.898+0000 7fb10affd640 1 -- 172.21.15.190:0/3454292293 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb114136cf0 con 0x7fb11409da50 2024-08-19T10:43:48.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.898+0000 7fb10affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:48.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:48.898+0000 7fb10affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:49.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:49.078+0000 7fb11c60a640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-08-19T10:43:49.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:49.078+0000 7fb11c60a640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-08-19T10:43:49.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:49.078+0000 7fb11c60a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:49.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:49.078+0000 7fb11c60a640 1 -- 172.21.15.190:0/3454292293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]} v 0) -- 0x7fb11406db30 con 0x7fb11409da50 2024-08-19T10:43:50.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.050+0000 7fb10affd640 1 -- 172.21.15.190:0/3454292293 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}]=0 moved item id -16 name 'fooo' to location {root=default} in crush map v182) ==== 173+0+0 (secure 0 0 0) 0x7fb10404c8a0 con 0x7fb11409da50 2024-08-19T10:43:50.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.050+0000 7fb10affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-08-19T10:43:50.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.050+0000 7fb10affd640 10 monclient: _finish_command 2 = system:0 moved item id -16 name 'fooo' to location {root=default} in crush map 2024-08-19T10:43:50.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-08-19T10:43:50.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-08-19T10:43:50.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:50.163 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 -- 172.21.15.190:0/3454292293 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]} v 0) -- 0x7fb11409fbf0 con 0x7fb11409da50 2024-08-19T10:43:50.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb10affd640 1 -- 172.21.15.190:0/3454292293 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}]=0 no need to move item id -16 name 'fooo' to location {root=default} in crush map v182) ==== 183+0+0 (secure 0 0 0) 0x7fb10404cc20 con 0x7fb11409da50 2024-08-19T10:43:50.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb10affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["root=default"]}] 2024-08-19T10:43:50.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb10affd640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'fooo' to location {root=default} in crush map 2024-08-19T10:43:50.164 INFO:tasks.workunit.client.0.smithi190.stderr:no need to move item id -16 name 'fooo' to location {root=default} in crush map 2024-08-19T10:43:50.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 -- 172.21.15.190:0/3454292293 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0f803d230 msgr2=0x7fb0f803f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:50.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0f803d230 0x7fb0f803f6f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8003750 tx=0x7fb1100a7070 comp rx=0 tx=0).stop 2024-08-19T10:43:50.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 10 monclient: shutdown 2024-08-19T10:43:50.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:50.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 -- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb11409da50 msgr2=0x7fb11409de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:50.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb11409da50 0x7fb11409de30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb10400b4c0 tx=0x7fb10400b980 comp rx=0 tx=0).stop 2024-08-19T10:43:50.166 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 -- 172.21.15.190:0/3454292293 shutdown_connections 2024-08-19T10:43:50.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0f803d230 0x7fb0f803f6f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:50.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb11409e370 0x7fb11409e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:50.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb11409da50 0x7fb11409de30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:50.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 --2- 172.21.15.190:0/3454292293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb1141357e0 0x7fb114135bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:50.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 -- 172.21.15.190:0/3454292293 >> 172.21.15.190:0/3454292293 conn(0x7fb1140a4700 msgr2=0x7fb1141c5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:50.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 -- 172.21.15.190:0/3454292293 shutdown_connections 2024-08-19T10:43:50.167 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.162+0000 7fb11c60a640 1 -- 172.21.15.190:0/3454292293 wait complete. 2024-08-19T10:43:50.182 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket barr rack 2024-08-19T10:43:50.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.298+0000 7f4a78f3d640 10 monclient: get_monmap_and_config 2024-08-19T10:43:50.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.298+0000 7f4a78f3d640 10 monclient: build_initial_monmap 2024-08-19T10:43:50.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.298+0000 7f4a78f3d640 10 monclient: monmap: 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:50.302422+0000 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:50.302422+0000 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:50.302 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:50.303 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:50.303 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:50.303 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:50.303 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:50.303 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:50.303 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:50.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.298+0000 7f4a78f3d640 1 Processor -- start 2024-08-19T10:43:50.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.298+0000 7f4a78f3d640 1 -- start start 2024-08-19T10:43:50.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.298+0000 7f4a78f3d640 10 monclient: init 2024-08-19T10:43:50.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:50.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 10 monclient: _add_conns ranks=[7,10,5,4,9,0,1,6,8,3,2] 2024-08-19T10:43:50.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a740a5940 0x7f4a740720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:50.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 10 monclient(hunting): picked mon.noname-i con 0x7f4a740a5940 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:50.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4a740a5d20 0x7f4a740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:50.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 10 monclient(hunting): picked mon.noname-f con 0x7f4a740a5d20 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:50.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a74139710 0x7f4a74139b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:50.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 10 monclient(hunting): picked mon.noname-h con 0x7f4a74139710 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:50.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:50.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f4a7406df10 con 0x7f4a740a5d20 2024-08-19T10:43:50.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:50.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4a7406efb0 con 0x7f4a74139710 2024-08-19T10:43:50.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:50.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4a7406f290 con 0x7f4a740a5940 2024-08-19T10:43:50.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:50.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a78f3d640 10 monclient(hunting): authenticate will time out at 1275.872311s 2024-08-19T10:43:50.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a69d9b640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f4a740a5d20 msgr2=0x7f4a740a6100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:50.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a69d9b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4a740a5d20 0x7f4a740a6100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:50.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a74139710 0x7f4a74139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:50.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a740a5940 0x7f4a740720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:50.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a74139710 0x7f4a74139b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54328/0 (socket says 172.21.15.190:54328) 2024-08-19T10:43:50.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 1 -- 172.21.15.190:0/2874237793 learned_addr learned my addr 172.21.15.190:0/2874237793 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:50.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 10 monclient(hunting): get_auth_request con 0x7f4a74139710 auth_method 0 2024-08-19T10:43:50.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:50.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:50.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:50.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a7259c640 10 monclient(hunting): get_auth_request con 0x7f4a740a5940 auth_method 0 2024-08-19T10:43:50.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a7259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:50.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a7259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:50.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a7259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:50.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:50.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:50.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:50.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a7259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:50.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a7259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:50.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a7259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:50.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 10 monclient(hunting): handle_auth_done global_id 5909 payload 274 2024-08-19T10:43:50.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 1 -- 172.21.15.190:0/2874237793 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a740a5940 msgr2=0x7f4a740720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:50.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 1 --2- 172.21.15.190:0/2874237793 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a740a5940 0x7f4a740720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:50.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 1 -- 172.21.15.190:0/2874237793 >> v2:172.21.15.190:3300/0 conn(0x7f4a740a5d20 msgr2=0x7f4a740a6100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:50.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 1 --2- 172.21.15.190:0/2874237793 >> v2:172.21.15.190:3300/0 conn(0x7f4a740a5d20 0x7f4a740a6100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:50.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:43:50.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 1 monclient: found mon.noname-h 2024-08-19T10:43:50.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:50.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:50.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a72d9d640 1 -- 172.21.15.190:0/2874237793 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a740a73d0 con 0x7f4a74139710 2024-08-19T10:43:50.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.302+0000 7f4a7259c640 1 --2- 172.21.15.190:0/2874237793 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a740a5940 0x7f4a740720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:43:50.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a72d9d640 1 -- 172.21.15.190:0/2874237793 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a74139710 msgr2=0x7f4a74139b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:50.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a72d9d640 1 --2- 172.21.15.190:0/2874237793 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a74139710 0x7f4a74139b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009e80 tx=0x7f4a6c01ca10 comp rx=0 tx=0).ready entity=mon.5 client_cookie=b55cdabaf0c835a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:50.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 1 -- 172.21.15.190:0/2874237793 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a6c0090d0 con 0x7f4a74139710 2024-08-19T10:43:50.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:50.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:50.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 10 monclient: dump: 2024-08-19T10:43:50.318 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:50.319 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:50.319 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:50.319 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:50.319 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:50.319 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:50.319 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:50.319 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:50.319 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:50.319 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:50.319 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:50.319 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 10 monclient: _finish_auth 0 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 1 -- 172.21.15.190:0/2874237793 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a6c0a5ac0 con 0x7f4a74139710 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 5 monclient: authenticate success, global_id 5909 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:50.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 1 -- 172.21.15.190:0/2874237793 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a6c0a5c70 con 0x7f4a74139710 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 4 set_mon_vals no callback set 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a71d9b640 10 monclient: dump: 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:50.321 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:50.322 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:50.322 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:50.322 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:50.322 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:50.322 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:50.322 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:50.322 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:50.322 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:50.322 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:50.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient: shutdown 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2874237793 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a74139710 msgr2=0x7f4a74139b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 --2- 172.21.15.190:0/2874237793 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a74139710 0x7f4a74139b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009e80 tx=0x7f4a6c01ca10 comp rx=0 tx=0).stop 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2874237793 shutdown_connections 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 --2- 172.21.15.190:0/2874237793 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a74139710 0x7f4a74139b90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 --2- 172.21.15.190:0/2874237793 >> v2:172.21.15.190:3300/0 conn(0x7f4a740a5d20 0x7f4a740a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 --2- 172.21.15.190:0/2874237793 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a740a5940 0x7f4a740720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2874237793 >> 172.21.15.190:0/2874237793 conn(0x7f4a740aa530 msgr2=0x7f4a740a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2874237793 shutdown_connections 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2874237793 wait complete. 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient: build_initial_monmap 2024-08-19T10:43:50.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient: monmap: 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:50.324 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 Processor -- start 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 -- start start 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient: init 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient: _add_conns ranks=[0,6,1,4,2,8,5,7,3] 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a740a5940 0x7f4a741d1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient(hunting): picked mon.noname-a con 0x7f4a740a5940 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:50.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a740a5d20 0x7f4a74136770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient(hunting): picked mon.noname-d con 0x7f4a740a5d20 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a74139710 0x7f4a7413da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient(hunting): picked mon.noname-f con 0x7f4a74139710 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7406e7e0 con 0x7f4a740a5940 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7406e660 con 0x7f4a74139710 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4a74143390 con 0x7f4a740a5d20 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a78f3d640 10 monclient(hunting): authenticate will time out at 1275.879770s 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a72d9d640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a74139710 msgr2=0x7f4a7413da00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:50.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.306+0000 7f4a72d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a74139710 0x7f4a7413da00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:50.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a69d9b640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a740a5d20 msgr2=0x7f4a74136770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:50.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a69d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a740a5d20 0x7f4a74136770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:50.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a740a5940 0x7f4a741d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:50.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a740a5940 0x7f4a741d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:52230/0 (socket says 172.21.15.190:52230) 2024-08-19T10:43:50.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 1 -- 172.21.15.190:0/2290144667 learned_addr learned my addr 172.21.15.190:0/2290144667 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:50.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 10 monclient(hunting): get_auth_request con 0x7f4a740a5940 auth_method 0 2024-08-19T10:43:50.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 10 monclient(hunting): handle_auth_done global_id 5940 payload 824 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 1 -- 172.21.15.190:0/2290144667 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a740a5d20 msgr2=0x7f4a74136770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a740a5d20 0x7f4a74136770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 1 -- 172.21.15.190:0/2290144667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a74139710 msgr2=0x7f4a7413da00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a74139710 0x7f4a7413da00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 10 monclient: _finish_hunting 0 2024-08-19T10:43:50.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 1 monclient: found mon.noname-a 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 1 -- 172.21.15.190:0/2290144667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a740a80c0 con 0x7f4a740a5940 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 1 -- 172.21.15.190:0/2290144667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a740a5940 msgr2=0x7f4a741d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a7259c640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a740a5940 0x7f4a741d1320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400e9b0 tx=0x7f4a6400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a6b7fe640 1 -- 172.21.15.190:0/2290144667 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a64017070 con 0x7f4a740a5940 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a6b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a6b7fe640 10 monclient: dump: 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:50.329 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:50.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a6b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a6b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.310+0000 7f4a6b7fe640 1 -- 172.21.15.190:0/2290144667 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a6400cc10 con 0x7f4a740a5940 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a78f3d640 5 monclient: authenticate success, global_id 5940 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a6b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a6b7fe640 1 -- 172.21.15.190:0/2290144667 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a6400cdc0 con 0x7f4a740a5940 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a6b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a7159a640 4 set_mon_vals no callback set 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a6b7fe640 10 monclient: dump: 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:50.331 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:50.332 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:50.332 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:50.332 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:50.332 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:50.332 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:50.332 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:50.332 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a78f3d640 10 monclient: _renew_subs 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a78f3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2290144667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a740733d0 con 0x7f4a740a5940 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a78f3d640 10 monclient: _renew_subs 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a78f3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2290144667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a74122110 con 0x7f4a740a5940 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a78f3d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:50.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a78f3d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:50.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a78f3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:50.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2290144667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a740a6100 con 0x7f4a740a5940 2024-08-19T10:43:50.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a6b7fe640 1 -- 172.21.15.190:0/2290144667 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4a6401a070 con 0x7f4a740a5940 2024-08-19T10:43:50.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a6b7fe640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a5403be30 0x7f4a5403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:50.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a6b7fe640 1 -- 172.21.15.190:0/2290144667 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 7817+0+0 (secure 0 0 0) 0x7f4a6400f040 con 0x7f4a740a5940 2024-08-19T10:43:50.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a69d9b640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a5403be30 0x7f4a5403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:50.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.314+0000 7f4a69d9b640 10 monclient: get_auth_request con 0x7f4a5403be30 auth_method 0 2024-08-19T10:43:50.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.318+0000 7f4a69d9b640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a5403be30 0x7f4a5403e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c006fd0 tx=0x7f4a5c006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:50.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.326+0000 7f4a6b7fe640 1 -- 172.21.15.190:0/2290144667 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4a64021070 con 0x7f4a740a5940 2024-08-19T10:43:50.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.326+0000 7f4a6b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:50.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.326+0000 7f4a6b7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:50.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.502+0000 7f4a78f3d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-08-19T10:43:50.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.502+0000 7f4a78f3d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-08-19T10:43:50.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.502+0000 7f4a78f3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:50.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:50.502+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2290144667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"} v 0) -- 0x7f4a74121ce0 con 0x7f4a740a5940 2024-08-19T10:43:52.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.086+0000 7f4a6b7fe640 1 -- 172.21.15.190:0/2290144667 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}]=0 added bucket barr type rack to crush map v184) ==== 140+0+0 (secure 0 0 0) 0x7f4a64011350 con 0x7f4a740a5940 2024-08-19T10:43:52.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.086+0000 7f4a6b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-08-19T10:43:52.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.086+0000 7f4a6b7fe640 10 monclient: _finish_command 2 = system:0 added bucket barr type rack to crush map 2024-08-19T10:43:52.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.194+0000 7f4a78f3d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-08-19T10:43:52.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.194+0000 7f4a78f3d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-08-19T10:43:52.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.198+0000 7f4a78f3d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:43:52.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.198+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2290144667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"} v 0) -- 0x7f4a740008e0 con 0x7f4a740a5940 2024-08-19T10:43:52.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.198+0000 7f4a6b7fe640 1 -- 172.21.15.190:0/2290144667 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}]=0 bucket 'barr' already exists v184) ==== 128+0+0 (secure 0 0 0) 0x7f4a64005dd0 con 0x7f4a740a5940 2024-08-19T10:43:52.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.198+0000 7f4a6b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "barr", "type": "rack"}] 2024-08-19T10:43:52.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.198+0000 7f4a6b7fe640 10 monclient: _finish_command 3 = system:0 bucket 'barr' already exists 2024-08-19T10:43:52.200 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'barr' already exists 2024-08-19T10:43:52.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2290144667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a5403be30 msgr2=0x7f4a5403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:52.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a5403be30 0x7f4a5403e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c006fd0 tx=0x7f4a5c006d40 comp rx=0 tx=0).stop 2024-08-19T10:43:52.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 10 monclient: shutdown 2024-08-19T10:43:52.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:52.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2290144667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a740a5940 msgr2=0x7f4a741d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:52.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a740a5940 0x7f4a741d1320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400e9b0 tx=0x7f4a6400ee70 comp rx=0 tx=0).stop 2024-08-19T10:43:52.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2290144667 shutdown_connections 2024-08-19T10:43:52.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4a5403be30 0x7f4a5403e2f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:52.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a74139710 0x7f4a7413da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:52.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a740a5d20 0x7f4a74136770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:52.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 --2- 172.21.15.190:0/2290144667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a740a5940 0x7f4a741d1320 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:52.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2290144667 >> 172.21.15.190:0/2290144667 conn(0x7f4a740aa530 msgr2=0x7f4a7413ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:52.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2290144667 shutdown_connections 2024-08-19T10:43:52.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.202+0000 7f4a78f3d640 1 -- 172.21.15.190:0/2290144667 wait complete. 2024-08-19T10:43:52.218 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush move barr root=default 2024-08-19T10:43:52.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient: get_monmap_and_config 2024-08-19T10:43:52.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient: build_initial_monmap 2024-08-19T10:43:52.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient: monmap: 2024-08-19T10:43:52.339 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:52.339 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:52.339 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:52.339877+0000 2024-08-19T10:43:52.339 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:52.339877+0000 2024-08-19T10:43:52.339 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:52.340 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:52.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 1 Processor -- start 2024-08-19T10:43:52.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 1 -- start start 2024-08-19T10:43:52.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient: init 2024-08-19T10:43:52.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:52.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient: _add_conns ranks=[10,0,7,9,2,4,8,6,5,1,3] 2024-08-19T10:43:52.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa80c0a3610 0x7fa80c0a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:52.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient(hunting): picked mon.noname-f con 0x7fa80c0a3610 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:52.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa80c0a3f30 addr v1:172.21.15.89:6789/0 2024-08-19T10:43:52.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c13e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient(hunting): picked mon.noname-i con 0x7fa80c136ae0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811544640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fa80c0a3610 msgr2=0x7fa80c0a39f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa80c071940 con 0x7fa80c0a3f30 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811544640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa80c0a3610 0x7fa80c0a39f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa80c073280 con 0x7fa80c0a3610 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa80c0729e0 con 0x7fa80c136ae0 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa8137a8640 10 monclient(hunting): authenticate will time out at 1277.910053s 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c13e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c13e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:50416/0 (socket says 172.21.15.190:50416) 2024-08-19T10:43:52.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 1 -- 172.21.15.190:0/250433997 learned_addr learned my addr 172.21.15.190:0/250433997 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 10 monclient(hunting): get_auth_request con 0x7fa80c136ae0 auth_method 0 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 10 monclient(hunting): handle_auth_done global_id 15172 payload 274 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 1 -- 172.21.15.190:0/250433997 >> v2:172.21.15.190:3300/0 conn(0x7fa80c0a3610 msgr2=0x7fa80c0a39f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 1 --2- 172.21.15.190:0/250433997 >> v2:172.21.15.190:3300/0 conn(0x7fa80c0a3610 0x7fa80c0a39f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 1 -- 172.21.15.190:0/250433997 >> v1:172.21.15.89:6789/0 conn(0x7fa80c0a3f30 legacy=0x7fa80c1363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 10 monclient: _finish_hunting 0 2024-08-19T10:43:52.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 1 monclient: found mon.noname-i 2024-08-19T10:43:52.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:52.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:52.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.338+0000 7fa811d45640 1 -- 172.21.15.190:0/250433997 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c0a5540 con 0x7fa80c136ae0 2024-08-19T10:43:52.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa811d45640 1 -- 172.21.15.190:0/250433997 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 msgr2=0x7fa80c13e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:52.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa811d45640 1 --2- 172.21.15.190:0/250433997 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c13e630 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa808009870 tx=0x7fa80801c260 comp rx=0 tx=0).ready entity=mon.7 client_cookie=bbb8bc8ecbe67c15 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:52.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 1 -- 172.21.15.190:0/250433997 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8080b0070 con 0x7fa80c136ae0 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 10 monclient: dump: 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:52.350 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 10 monclient: _finish_auth 0 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 10 monclient: _check_auth_tickets 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 1 -- 172.21.15.190:0/250433997 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8080aa810 con 0x7fa80c136ae0 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 5 monclient: authenticate success, global_id 15172 2024-08-19T10:43:52.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:52.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:52.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:52.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 4 set_mon_vals no callback set 2024-08-19T10:43:52.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 1 -- 172.21.15.190:0/250433997 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8080a9300 con 0x7fa80c136ae0 2024-08-19T10:43:52.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:52.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:52.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa7fffff640 10 monclient: dump: 2024-08-19T10:43:52.352 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:52.352 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:52.352 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:52.352 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:52.353 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:52.353 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:52.353 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:52.353 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:52.353 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:52.353 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:52.353 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:52.353 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:52.353 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:52.353 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:52.353 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 10 monclient: shutdown 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 1 -- 172.21.15.190:0/250433997 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 msgr2=0x7fa80c13e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 1 --2- 172.21.15.190:0/250433997 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c13e630 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa808009870 tx=0x7fa80801c260 comp rx=0 tx=0).stop 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 1 -- 172.21.15.190:0/250433997 shutdown_connections 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 1 --2- 172.21.15.190:0/250433997 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c13e630 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 1 --2- 172.21.15.190:0/250433997 >> v2:172.21.15.190:3300/0 conn(0x7fa80c0a3610 0x7fa80c0a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 1 -- 172.21.15.190:0/250433997 >> 172.21.15.190:0/250433997 conn(0x7fa80c0a4700 msgr2=0x7fa80c0a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 1 -- 172.21.15.190:0/250433997 shutdown_connections 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.342+0000 7fa8137a8640 1 -- 172.21.15.190:0/250433997 wait complete. 2024-08-19T10:43:52.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient: build_initial_monmap 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient: monmap: 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:52.355 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 1 Processor -- start 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 1 -- start start 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient: init 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient: _add_conns ranks=[1,8,7,4,5,0,6,3,2] 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa80c0a3610 0x7fa80c139660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient(hunting): picked mon.noname-f con 0x7fa80c0a3610 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa80c0a3f30 0x7fa80c1373e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:52.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient(hunting): picked mon.noname-e con 0x7fa80c0a3f30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient(hunting): picked mon.noname-i con 0x7fa80c136ae0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c0a2110 con 0x7fa80c0a3610 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa80c0a1f90 con 0x7fa80c136ae0 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa80c071530 con 0x7fa80c0a3f30 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa8137a8640 10 monclient(hunting): authenticate will time out at 1277.917319s 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811544640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa80c0a3610 msgr2=0x7fa80c139660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811544640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa80c0a3610 0x7fa80c139660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:52.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:52.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c137920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:50428/0 (socket says 172.21.15.190:50428) 2024-08-19T10:43:52.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 1 -- 172.21.15.190:0/2510206097 learned_addr learned my addr 172.21.15.190:0/2510206097 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:52.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 10 monclient(hunting): get_auth_request con 0x7fa80c136ae0 auth_method 0 2024-08-19T10:43:52.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:52.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:52.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:52.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:52.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:52.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:52.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 10 monclient(hunting): handle_auth_done global_id 15181 payload 824 2024-08-19T10:43:52.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 1 -- 172.21.15.190:0/2510206097 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa80c0a3f30 msgr2=0x7fa80c1373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:52.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa80c0a3f30 0x7fa80c1373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:52.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 1 -- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa80c0a3610 msgr2=0x7fa80c139660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:52.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa80c0a3610 0x7fa80c139660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:52.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 10 monclient: _finish_hunting 0 2024-08-19T10:43:52.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 1 monclient: found mon.noname-i 2024-08-19T10:43:52.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:52.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:52.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 1 -- 172.21.15.190:0/2510206097 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c13ab80 con 0x7fa80c136ae0 2024-08-19T10:43:52.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 1 -- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 msgr2=0x7fa80c137920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:52.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.346+0000 7fa811d45640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c137920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa80801cb00 tx=0x7fa808019000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 1 -- 172.21.15.190:0/2510206097 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8080b9070 con 0x7fa80c136ae0 2024-08-19T10:43:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 10 monclient: dump: 2024-08-19T10:43:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 10 monclient: _finish_auth 0 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:52.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa8137a8640 5 monclient: authenticate success, global_id 15181 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 1 -- 172.21.15.190:0/2510206097 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8080044b0 con 0x7fa80c136ae0 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa8137a8640 10 monclient: _renew_subs 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa8137a8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa8137a8640 1 -- 172.21.15.190:0/2510206097 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c13ae60 con 0x7fa80c136ae0 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 1 -- 172.21.15.190:0/2510206097 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8080b0040 con 0x7fa80c136ae0 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 10 monclient: dump: 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:52.364 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:52.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7ff7fe640 4 set_mon_vals no callback set 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa8137a8640 10 monclient: _renew_subs 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa8137a8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa8137a8640 1 -- 172.21.15.190:0/2510206097 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80c138b00 con 0x7fa80c136ae0 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa8137a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa8137a8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa8137a8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa8137a8640 1 -- 172.21.15.190:0/2510206097 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa80c0a39f0 con 0x7fa80c136ae0 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 1 -- 172.21.15.190:0/2510206097 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa808004040 con 0x7fa80c136ae0 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa7f003be30 0x7fa7f003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa7fdffb640 1 -- 172.21.15.190:0/2510206097 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(184..184 src has 1..184) ==== 7845+0+0 (secure 0 0 0) 0x7fa8080e87d0 con 0x7fa80c136ae0 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa811544640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa7f003be30 0x7fa7f003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.350+0000 7fa811544640 10 monclient: get_auth_request con 0x7fa7f003be30 auth_method 0 2024-08-19T10:43:52.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.354+0000 7fa811544640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa7f003be30 0x7fa7f003e2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa804003d10 tx=0x7fa804009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:52.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.354+0000 7fa7fdffb640 1 -- 172.21.15.190:0/2510206097 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fa80c071940 con 0x7fa80c136ae0 2024-08-19T10:43:52.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.354+0000 7fa7fdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:52.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.354+0000 7fa7fdffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:52.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.522+0000 7fa8137a8640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-08-19T10:43:52.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.522+0000 7fa8137a8640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-08-19T10:43:52.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.522+0000 7fa8137a8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:52.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:52.522+0000 7fa8137a8640 1 -- 172.21.15.190:0/2510206097 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush move", "name": "barr", "args": ["root=default"]} v 0) -- 0x7fa80c06db30 con 0x7fa80c136ae0 2024-08-19T10:43:53.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.114+0000 7fa7fdffb640 1 -- 172.21.15.190:0/2510206097 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}]=0 moved item id -19 name 'barr' to location {root=default} in crush map v185) ==== 173+0+0 (secure 0 0 0) 0x7fa8080aa7e0 con 0x7fa80c136ae0 2024-08-19T10:43:53.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.114+0000 7fa7fdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-08-19T10:43:53.115 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.114+0000 7fa7fdffb640 10 monclient: _finish_command 2 = system:0 moved item id -19 name 'barr' to location {root=default} in crush map 2024-08-19T10:43:53.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.222+0000 7fa8137a8640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-08-19T10:43:53.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.222+0000 7fa8137a8640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-08-19T10:43:53.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.222+0000 7fa8137a8640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:53.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.222+0000 7fa8137a8640 1 -- 172.21.15.190:0/2510206097 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush move", "name": "barr", "args": ["root=default"]} v 0) -- 0x7fa80c13a5c0 con 0x7fa80c136ae0 2024-08-19T10:43:53.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa7fdffb640 1 -- 172.21.15.190:0/2510206097 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}]=0 no need to move item id -19 name 'barr' to location {root=default} in crush map v185) ==== 183+0+0 (secure 0 0 0) 0x7fa808003b60 con 0x7fa80c136ae0 2024-08-19T10:43:53.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa7fdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "barr", "args": ["root=default"]}] 2024-08-19T10:43:53.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa7fdffb640 10 monclient: _finish_command 3 = system:0 no need to move item id -19 name 'barr' to location {root=default} in crush map 2024-08-19T10:43:53.235 INFO:tasks.workunit.client.0.smithi190.stderr:no need to move item id -19 name 'barr' to location {root=default} in crush map 2024-08-19T10:43:53.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 -- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa7f003be30 msgr2=0x7fa7f003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:53.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa7f003be30 0x7fa7f003e2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa804003d10 tx=0x7fa804009290 comp rx=0 tx=0).stop 2024-08-19T10:43:53.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 10 monclient: shutdown 2024-08-19T10:43:53.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:53.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 -- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 msgr2=0x7fa80c137920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:53.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c137920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa80801cb00 tx=0x7fa808019000 comp rx=0 tx=0).stop 2024-08-19T10:43:53.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 -- 172.21.15.190:0/2510206097 shutdown_connections 2024-08-19T10:43:53.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa80c136ae0 0x7fa80c137920 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:53.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa7f003be30 0x7fa7f003e2f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:53.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa80c0a3f30 0x7fa80c1373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:53.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 --2- 172.21.15.190:0/2510206097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa80c0a3610 0x7fa80c139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:53.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 -- 172.21.15.190:0/2510206097 >> 172.21.15.190:0/2510206097 conn(0x7fa80c0a4700 msgr2=0x7fa80c0a5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:53.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 -- 172.21.15.190:0/2510206097 shutdown_connections 2024-08-19T10:43:53.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.234+0000 7fa8137a8640 1 -- 172.21.15.190:0/2510206097 wait complete. 2024-08-19T10:43:53.253 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush move fooo rack=barr 2024-08-19T10:43:53.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient: get_monmap_and_config 2024-08-19T10:43:53.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient: build_initial_monmap 2024-08-19T10:43:53.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient: monmap: 2024-08-19T10:43:53.363 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:53.364 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:53.364 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:53.364311+0000 2024-08-19T10:43:53.364 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:53.364311+0000 2024-08-19T10:43:53.364 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:53.364 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:53.364 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:53.364 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:53.364 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:53.364 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:53.364 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:53.364 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:53.365 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:53.365 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:53.365 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:53.365 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:53.365 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:53.365 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:53.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 1 Processor -- start 2024-08-19T10:43:53.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 1 -- start start 2024-08-19T10:43:53.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient: init 2024-08-19T10:43:53.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:53.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient: _add_conns ranks=[5,2,1,9,4,3,6,0,7,8,10] 2024-08-19T10:43:53.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3ac069410 0x7fe3ac0a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient(hunting): picked mon.noname-h con 0x7fe3ac069410 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3ac0a41c0 0x7fe3ac0a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient(hunting): picked mon.noname-b con 0x7fe3ac0a41c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe3ac13aa60 addr v1:172.21.15.190:6789/0 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3ac074a20 con 0x7fe3ac13aa60 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ac09e520 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe3ac0a4ae0 con 0x7fe3ac069410 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b2298640 10 monclient(hunting): authenticate will time out at 1278.934929s 2024-08-19T10:43:53.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3b0835640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fe3ac13aa60 legacy=0x7fe3ac13ce50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:53.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3ac069410 0x7fe3ac0a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:53.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3ac069410 0x7fe3ac0a3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54330/0 (socket says 172.21.15.190:54330) 2024-08-19T10:43:53.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 1 -- 172.21.15.190:0/391549351 learned_addr learned my addr 172.21.15.190:0/391549351 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:53.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 10 monclient(hunting): get_auth_request con 0x7fe3ac069410 auth_method 0 2024-08-19T10:43:53.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:53.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:53.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:53.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:53.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:53.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:53.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3ab7fe640 1 --2- 172.21.15.190:0/391549351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3ac0a41c0 0x7fe3ac0a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3ab7fe640 10 monclient(hunting): get_auth_request con 0x7fe3ac0a41c0 auth_method 0 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3ab7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 10 monclient(hunting): handle_auth_done global_id 5918 payload 274 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 1 -- 172.21.15.190:0/391549351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3ac0a41c0 msgr2=0x7fe3ac0a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 1 --2- 172.21.15.190:0/391549351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3ac0a41c0 0x7fe3ac0a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 1 -- 172.21.15.190:0/391549351 >> v1:172.21.15.190:6789/0 conn(0x7fe3ac13aa60 legacy=0x7fe3ac13ce50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 10 monclient: _finish_hunting 0 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 1 monclient: found mon.noname-h 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:53.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:53.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.362+0000 7fe3abfff640 1 -- 172.21.15.190:0/391549351 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ac0a0120 con 0x7fe3ac069410 2024-08-19T10:43:53.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3abfff640 1 -- 172.21.15.190:0/391549351 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3ac069410 msgr2=0x7fe3ac0a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:53.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3abfff640 1 --2- 172.21.15.190:0/391549351 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3ac069410 0x7fe3ac0a3c80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0098a0 tx=0x7fe39c01c4d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=31764925d3a0d06 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 1 -- 172.21.15.190:0/391549351 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe39c0b0070 con 0x7fe3ac069410 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 10 monclient: dump: 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:53.378 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:53.379 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:53.379 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:53.379 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:53.379 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:53.379 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:53.379 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:53.379 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 10 monclient: _finish_auth 0 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 1 -- 172.21.15.190:0/391549351 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe39c0aa810 con 0x7fe3ac069410 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3b2298640 5 monclient: authenticate success, global_id 5918 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3b2298640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3b2298640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 1 -- 172.21.15.190:0/391549351 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe39c0aae30 con 0x7fe3ac069410 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3b2298640 4 set_mon_vals no callback set 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.366+0000 7fe3aaffd640 10 monclient: dump: 2024-08-19T10:43:53.380 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:53.381 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient: shutdown 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 -- 172.21.15.190:0/391549351 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3ac069410 msgr2=0x7fe3ac0a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 --2- 172.21.15.190:0/391549351 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3ac069410 0x7fe3ac0a3c80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0098a0 tx=0x7fe39c01c4d0 comp rx=0 tx=0).stop 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 -- 172.21.15.190:0/391549351 shutdown_connections 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 --2- 172.21.15.190:0/391549351 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3ac0a41c0 0x7fe3ac0a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 --2- 172.21.15.190:0/391549351 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3ac069410 0x7fe3ac0a3c80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 -- 172.21.15.190:0/391549351 >> 172.21.15.190:0/391549351 conn(0x7fe3ac0a4d70 msgr2=0x7fe3ac0a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 -- 172.21.15.190:0/391549351 shutdown_connections 2024-08-19T10:43:53.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 -- 172.21.15.190:0/391549351 wait complete. 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient: build_initial_monmap 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient: monmap: 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:53.383 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 Processor -- start 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 -- start start 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient: init 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient: _add_conns ranks=[2,7,0,8,3,5,6,4,1] 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3ac069410 0x7fe3ac1d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient(hunting): picked mon.noname-b con 0x7fe3ac069410 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3ac0a41c0 0x7fe3ac1d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient(hunting): picked mon.noname-i con 0x7fe3ac0a41c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:53.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3ac13aa60 0x7fe3ac13ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient(hunting): picked mon.noname-a con 0x7fe3ac13aa60 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac09e290 con 0x7fe3ac13aa60 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ac09e110 con 0x7fe3ac069410 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fe3ac140d90 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b2298640 10 monclient(hunting): authenticate will time out at 1278.941989s 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3ac0a41c0 0x7fe3ac1d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3ac0a41c0 0x7fe3ac1d15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:50434/0 (socket says 172.21.15.190:50434) 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 1 -- 172.21.15.190:0/1636395073 learned_addr learned my addr 172.21.15.190:0/1636395073 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:53.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 10 monclient(hunting): get_auth_request con 0x7fe3ac0a41c0 auth_method 0 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3abfff640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3ac069410 0x7fe3ac1d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 10 monclient(hunting): handle_auth_done global_id 15190 payload 824 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3b0835640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3ac13aa60 0x7fe3ac13ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 1 -- 172.21.15.190:0/1636395073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3ac069410 msgr2=0x7fe3ac1d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3ac069410 0x7fe3ac1d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 1 -- 172.21.15.190:0/1636395073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3ac13aa60 msgr2=0x7fe3ac13ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:53.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3ac13aa60 0x7fe3ac13ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 1 monclient: found mon.noname-i 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3ab7fe640 1 -- 172.21.15.190:0/1636395073 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ac073ac0 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3abfff640 10 monclient: get_auth_request con 0x7fe3ac069410 auth_method 0 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.370+0000 7fe3abfff640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3ac069410 0x7fe3ac1d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3ab7fe640 1 -- 172.21.15.190:0/1636395073 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3ac0a41c0 msgr2=0x7fe3ac1d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3ab7fe640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3ac0a41c0 0x7fe3ac1d15e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000d830 tx=0x7fe3a000dcf0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 1 -- 172.21.15.190:0/1636395073 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe3a000bce0 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:53.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 10 monclient: dump: 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:53.388 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:53.389 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:53.389 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:53.389 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:53.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 10 monclient: _finish_auth 0 2024-08-19T10:43:53.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:43:53.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:53.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3b2298640 5 monclient: authenticate success, global_id 15190 2024-08-19T10:43:53.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 1 -- 172.21.15.190:0/1636395073 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a0019070 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3b2298640 10 monclient: _renew_subs 2024-08-19T10:43:53.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3b2298640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:53.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3b2298640 1 -- 172.21.15.190:0/1636395073 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ac073da0 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 1 -- 172.21.15.190:0/1636395073 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe3a00148a0 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3aa7fc640 4 set_mon_vals no callback set 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 10 monclient: dump: 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:53.391 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3b2298640 10 monclient: _renew_subs 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3b2298640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3b2298640 1 -- 172.21.15.190:0/1636395073 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ac0a33c0 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3b2298640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:53.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3b2298640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:53.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3b2298640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:53.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3b2298640 1 -- 172.21.15.190:0/1636395073 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ac0a3c80 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 1 -- 172.21.15.190:0/1636395073 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe3a0014430 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39803be00 0x7fe39803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:53.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3a8ff9640 1 -- 172.21.15.190:0/1636395073 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(185..185 src has 1..185) ==== 8033+0+0 (secure 0 0 0) 0x7fe3a000f040 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3abfff640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39803be00 0x7fe39803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:53.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.374+0000 7fe3abfff640 10 monclient: get_auth_request con 0x7fe39803be00 auth_method 0 2024-08-19T10:43:53.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.378+0000 7fe3a8ff9640 1 -- 172.21.15.190:0/1636395073 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:43:53.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.378+0000 7fe3abfff640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39803be00 0x7fe39803e2c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0ab000 tx=0x7fe39c0a4f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:53.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.378+0000 7fe3a8ff9640 1 -- 172.21.15.190:0/1636395073 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe3a004db50 con 0x7fe3ac0a41c0 2024-08-19T10:43:53.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.378+0000 7fe3a8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:53.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.378+0000 7fe3a8ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:53.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.542+0000 7fe3b2298640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-08-19T10:43:53.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.542+0000 7fe3b2298640 10 monclient: _send_command 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-08-19T10:43:53.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.542+0000 7fe3b2298640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:53.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:53.542+0000 7fe3b2298640 1 -- 172.21.15.190:0/1636395073 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]} v 0) -- 0x7fe3ac135410 con 0x7fe3ac0a41c0 2024-08-19T10:43:54.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.118+0000 7fe3a8ff9640 1 -- 172.21.15.190:0/1636395073 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}]=0 moved item id -16 name 'fooo' to location {rack=barr} in crush map v186) ==== 167+0+0 (secure 0 0 0) 0x7fe3a0014a40 con 0x7fe3ac0a41c0 2024-08-19T10:43:54.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.118+0000 7fe3a8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-08-19T10:43:54.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.118+0000 7fe3a8ff9640 10 monclient: _finish_command 2 = system:0 moved item id -16 name 'fooo' to location {rack=barr} in crush map 2024-08-19T10:43:54.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.222+0000 7fe3b2298640 10 start_mon_command cmd=[{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-08-19T10:43:54.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.222+0000 7fe3b2298640 10 monclient: _send_command 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-08-19T10:43:54.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.222+0000 7fe3b2298640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:43:54.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.222+0000 7fe3b2298640 1 -- 172.21.15.190:0/1636395073 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]} v 0) -- 0x7fe3ac0008e0 con 0x7fe3ac0a41c0 2024-08-19T10:43:54.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.222+0000 7fe3a8ff9640 1 -- 172.21.15.190:0/1636395073 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}]=0 no need to move item id -16 name 'fooo' to location {rack=barr} in crush map v186) ==== 177+0+0 (secure 0 0 0) 0x7fe3a0013aa0 con 0x7fe3ac0a41c0 2024-08-19T10:43:54.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.222+0000 7fe3a8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush move", "name": "fooo", "args": ["rack=barr"]}] 2024-08-19T10:43:54.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.222+0000 7fe3a8ff9640 10 monclient: _finish_command 3 = system:0 no need to move item id -16 name 'fooo' to location {rack=barr} in crush map 2024-08-19T10:43:54.226 INFO:tasks.workunit.client.0.smithi190.stderr:no need to move item id -16 name 'fooo' to location {rack=barr} in crush map 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 -- 172.21.15.190:0/1636395073 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39803be00 msgr2=0x7fe39803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39803be00 0x7fe39803e2c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0ab000 tx=0x7fe39c0a4f90 comp rx=0 tx=0).stop 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 10 monclient: shutdown 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 -- 172.21.15.190:0/1636395073 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3ac0a41c0 msgr2=0x7fe3ac1d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3ac0a41c0 0x7fe3ac1d15e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000d830 tx=0x7fe3a000dcf0 comp rx=0 tx=0).stop 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 -- 172.21.15.190:0/1636395073 shutdown_connections 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39803be00 0x7fe39803e2c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3ac13aa60 0x7fe3ac13ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3ac0a41c0 0x7fe3ac1d15e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 --2- 172.21.15.190:0/1636395073 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3ac069410 0x7fe3ac1d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:54.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 -- 172.21.15.190:0/1636395073 >> 172.21.15.190:0/1636395073 conn(0x7fe3ac0a4d70 msgr2=0x7fe3ac136430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:54.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 -- 172.21.15.190:0/1636395073 shutdown_connections 2024-08-19T10:43:54.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.226+0000 7fe3b2298640 1 -- 172.21.15.190:0/1636395073 wait complete. 2024-08-19T10:43:54.244 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm fooo 2024-08-19T10:43:54.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient: get_monmap_and_config 2024-08-19T10:43:54.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient: build_initial_monmap 2024-08-19T10:43:54.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient: monmap: 2024-08-19T10:43:54.360 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:54.360 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:54.360 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:54.360810+0000 2024-08-19T10:43:54.360 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:54.360810+0000 2024-08-19T10:43:54.360 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:54.360 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:54.360 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:54.360 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:54.361 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:54.361 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 1 Processor -- start 2024-08-19T10:43:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 1 -- start start 2024-08-19T10:43:54.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient: init 2024-08-19T10:43:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:54.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient: _add_conns ranks=[6,9,1,2,10,8,5,4,7,3,0] 2024-08-19T10:43:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2c30139980 0x7f2c30139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient(hunting): picked mon.noname-d con 0x7f2c30139980 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:43:54.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2c3013a2a0 0x7f2c300a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient(hunting): picked mon.noname-a con 0x7f2c3013a2a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2c300a5c70 addr v1:172.21.15.190:6789/0 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c300a8470 con 0x7f2c300a5c70 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f2c3009ca60 con 0x7f2c3013a2a0 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f2c300a3640 con 0x7f2c30139980 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c385ce640 10 monclient(hunting): authenticate will time out at 1279.930632s 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c36b6b640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f2c300a5c70 legacy=0x7f2c300a67e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c3636a640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2c30139980 msgr2=0x7f2c30139d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:43:54.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c3636a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2c30139980 0x7f2c30139d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2c3013a2a0 0x7f2c300a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2c3013a2a0 0x7f2c300a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:52268/0 (socket says 172.21.15.190:52268) 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 1 -- 172.21.15.190:0/318014005 learned_addr learned my addr 172.21.15.190:0/318014005 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 10 monclient(hunting): get_auth_request con 0x7f2c3013a2a0 auth_method 0 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 10 monclient(hunting): handle_auth_done global_id 5949 payload 274 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 1 -- 172.21.15.190:0/318014005 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2c30139980 msgr2=0x7f2c30139d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 1 --2- 172.21.15.190:0/318014005 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2c30139980 0x7f2c30139d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:54.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 1 -- 172.21.15.190:0/318014005 >> v1:172.21.15.190:6789/0 conn(0x7f2c300a5c70 legacy=0x7f2c300a67e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 10 monclient: _finish_hunting 0 2024-08-19T10:43:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 1 monclient: found mon.noname-a 2024-08-19T10:43:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:43:54.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.358+0000 7f2c35b69640 1 -- 172.21.15.190:0/318014005 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c300a3850 con 0x7f2c3013a2a0 2024-08-19T10:43:54.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35b69640 1 -- 172.21.15.190:0/318014005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3013a2a0 msgr2=0x7f2c300a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:54.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35b69640 1 --2- 172.21.15.190:0/318014005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3013a2a0 0x7f2c300a5730 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c200098a0 tx=0x7f2c2001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35641fd0ab39df7b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:54.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 1 -- 172.21.15.190:0/318014005 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2c200b0070 con 0x7f2c3013a2a0 2024-08-19T10:43:54.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:54.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:43:54.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 10 monclient: dump: 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:54.372 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:54.373 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:54.373 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:54.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 10 monclient: _finish_auth 0 2024-08-19T10:43:54.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 10 monclient: _check_auth_tickets 2024-08-19T10:43:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 5 monclient: authenticate success, global_id 5949 2024-08-19T10:43:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 1 -- 172.21.15.190:0/318014005 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c200aa3a0 con 0x7f2c3013a2a0 2024-08-19T10:43:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:54.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 4 set_mon_vals no callback set 2024-08-19T10:43:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 1 -- 172.21.15.190:0/318014005 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2c200a4e60 con 0x7f2c3013a2a0 2024-08-19T10:43:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:43:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35368640 10 monclient: dump: 2024-08-19T10:43:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:54.375 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:54.376 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient: shutdown 2024-08-19T10:43:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 -- 172.21.15.190:0/318014005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3013a2a0 msgr2=0x7f2c300a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 --2- 172.21.15.190:0/318014005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3013a2a0 0x7f2c300a5730 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c200098a0 tx=0x7f2c2001c420 comp rx=0 tx=0).stop 2024-08-19T10:43:54.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 -- 172.21.15.190:0/318014005 shutdown_connections 2024-08-19T10:43:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 --2- 172.21.15.190:0/318014005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3013a2a0 0x7f2c300a5730 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 --2- 172.21.15.190:0/318014005 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2c30139980 0x7f2c30139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 -- 172.21.15.190:0/318014005 >> 172.21.15.190:0/318014005 conn(0x7f2c300abde0 msgr2=0x7f2c300ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:54.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 -- 172.21.15.190:0/318014005 shutdown_connections 2024-08-19T10:43:54.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 -- 172.21.15.190:0/318014005 wait complete. 2024-08-19T10:43:54.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient: build_initial_monmap 2024-08-19T10:43:54.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient: monmap: 2024-08-19T10:43:54.380 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:54.380 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:54.380 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:54.380 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:54.380 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 Processor -- start 2024-08-19T10:43:54.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 -- start start 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient: init 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient: _add_conns ranks=[7,1,5,8,4,0,3,6,2] 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2c300a5c70 0x7f2c301d1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient(hunting): picked mon.noname-i con 0x7f2c300a5c70 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c30139980 0x7f2c30140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient(hunting): picked mon.noname-f con 0x7f2c30139980 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2c30141240 0x7f2c3009ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient(hunting): picked mon.noname-h con 0x7f2c30141240 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c30144280 con 0x7f2c30139980 2024-08-19T10:43:54.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2c30144100 con 0x7f2c30141240 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2c30144400 con 0x7f2c300a5c70 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c385ce640 10 monclient(hunting): authenticate will time out at 1279.938694s 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35b69640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c30139980 msgr2=0x7f2c30140d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c35b69640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c30139980 0x7f2c30140d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c3636a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2c300a5c70 0x7f2c301d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2c30141240 0x7f2c3009ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c3636a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2c300a5c70 0x7f2c301d1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:50446/0 (socket says 172.21.15.190:50446) 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c3636a640 1 -- 172.21.15.190:0/2527429295 learned_addr learned my addr 172.21.15.190:0/2527429295 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:54.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c3636a640 10 monclient(hunting): get_auth_request con 0x7f2c300a5c70 auth_method 0 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c3636a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c3636a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c3636a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 10 monclient(hunting): get_auth_request con 0x7f2c30141240 auth_method 0 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c3636a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c3636a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:54.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c3636a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:54.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 10 monclient(hunting): handle_auth_done global_id 5927 payload 824 2024-08-19T10:43:54.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 1 -- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2c300a5c70 msgr2=0x7f2c301d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:54.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2c300a5c70 0x7f2c301d1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:54.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 1 -- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c30139980 msgr2=0x7f2c30140d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:43:54.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c30139980 0x7f2c30140d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:54.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 10 monclient: _finish_hunting 0 2024-08-19T10:43:54.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 1 monclient: found mon.noname-h 2024-08-19T10:43:54.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:54.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:54.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.366+0000 7f2c36b6b640 1 -- 172.21.15.190:0/2527429295 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c300a23f0 con 0x7f2c30141240 2024-08-19T10:43:54.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c36b6b640 1 -- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2c30141240 msgr2=0x7f2c3009ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:54.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c36b6b640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2c30141240 0x7f2c3009ec90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800bdd0 tx=0x7f2c28009ed0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 1 -- 172.21.15.190:0/2527429295 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2c2800ce00 con 0x7f2c30141240 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 10 monclient: dump: 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:54.388 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 10 monclient: _finish_auth 0 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c385ce640 5 monclient: authenticate success, global_id 5927 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 1 -- 172.21.15.190:0/2527429295 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c28010030 con 0x7f2c30141240 2024-08-19T10:43:54.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c385ce640 10 monclient: _renew_subs 2024-08-19T10:43:54.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c385ce640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:54.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c385ce640 1 -- 172.21.15.190:0/2527429295 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c301d1d80 con 0x7f2c30141240 2024-08-19T10:43:54.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:54.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 1 -- 172.21.15.190:0/2527429295 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2c28015cc0 con 0x7f2c30141240 2024-08-19T10:43:54.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:54.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:54.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c34b67640 4 set_mon_vals no callback set 2024-08-19T10:43:54.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 10 monclient: dump: 2024-08-19T10:43:54.390 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:54.390 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:54.391 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:54.391 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:54.391 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:54.391 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:54.391 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:54.392 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:54.392 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:54.392 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:54.392 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:54.392 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:54.392 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:54.392 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:54.392 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:54.392 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:54.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c385ce640 10 monclient: _renew_subs 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c385ce640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c385ce640 1 -- 172.21.15.190:0/2527429295 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c300a13d0 con 0x7f2c30141240 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c385ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c385ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c385ce640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c385ce640 1 -- 172.21.15.190:0/2527429295 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c300a5960 con 0x7f2c30141240 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 1 -- 172.21.15.190:0/2527429295 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2c280153e0 con 0x7f2c30141240 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2c1403be00 0x7f2c1403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c26ffd640 1 -- 172.21.15.190:0/2527429295 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(186..186 src has 1..186) ==== 8033+0+0 (secure 0 0 0) 0x7f2c2804fab0 con 0x7f2c30141240 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c3636a640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2c1403be00 0x7f2c1403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.370+0000 7f2c3636a640 10 monclient: get_auth_request con 0x7f2c1403be00 auth_method 0 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.374+0000 7f2c3636a640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2c1403be00 0x7f2c1403e2c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009470 tx=0x7f2c2c009400 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:54.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:54.374+0000 7f2c36b6b640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2c30141240 0x7f2c3009ec90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800bdd0 tx=0x7f2c28009ed0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f2c300a5960 mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) 2024-08-19T10:43:55.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:55.202+0000 7f2c26ffd640 1 -- 172.21.15.190:0/2527429295 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f2c300a5960 con 0x7f2c30141240 2024-08-19T10:43:55.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:55.202+0000 7f2c26ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:55.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:55.202+0000 7f2c26ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:55.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:55.394+0000 7f2c385ce640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "fooo"}] 2024-08-19T10:43:55.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:55.394+0000 7f2c385ce640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-08-19T10:43:55.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:55.394+0000 7f2c385ce640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:55.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:55.394+0000 7f2c385ce640 1 -- 172.21.15.190:0/2527429295 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "fooo"} v 0) -- 0x7f2c30139de0 con 0x7f2c30141240 2024-08-19T10:43:56.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.302+0000 7f2c26ffd640 1 -- 172.21.15.190:0/2527429295 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "fooo"}]=0 removed item id -16 name 'fooo' from crush map v188) ==== 122+0+0 (secure 0 0 0) 0x7f2c2801ec00 con 0x7f2c30141240 2024-08-19T10:43:56.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.302+0000 7f2c26ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-08-19T10:43:56.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.302+0000 7f2c26ffd640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'fooo' from crush map 2024-08-19T10:43:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.410+0000 7f2c385ce640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "fooo"}] 2024-08-19T10:43:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.410+0000 7f2c385ce640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-08-19T10:43:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.410+0000 7f2c385ce640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:56.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.410+0000 7f2c385ce640 1 -- 172.21.15.190:0/2527429295 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "fooo"} v 0) -- 0x7f2c300008e0 con 0x7f2c30141240 2024-08-19T10:43:56.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.758+0000 7f2c26ffd640 1 -- 172.21.15.190:0/2527429295 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "fooo"}]=0 device 'fooo' does not appear in the crush map v188) ==== 122+0+0 (secure 0 0 0) 0x7f2c28014e10 con 0x7f2c30141240 2024-08-19T10:43:56.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.758+0000 7f2c26ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "fooo"}] 2024-08-19T10:43:56.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.758+0000 7f2c26ffd640 10 monclient: _finish_command 3 = system:0 device 'fooo' does not appear in the crush map 2024-08-19T10:43:56.762 INFO:tasks.workunit.client.0.smithi190.stderr:device 'fooo' does not appear in the crush map 2024-08-19T10:43:56.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 -- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2c1403be00 msgr2=0x7f2c1403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:56.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2c1403be00 0x7f2c1403e2c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009470 tx=0x7f2c2c009400 comp rx=0 tx=0).stop 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 10 monclient: shutdown 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 -- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2c30141240 msgr2=0x7f2c3009ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2c30141240 0x7f2c3009ec90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800bdd0 tx=0x7f2c28009ed0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 -- 172.21.15.190:0/2527429295 shutdown_connections 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2c1403be00 0x7f2c1403e2c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2c30141240 0x7f2c3009ec90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c30139980 0x7f2c30140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 --2- 172.21.15.190:0/2527429295 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2c300a5c70 0x7f2c301d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 -- 172.21.15.190:0/2527429295 >> 172.21.15.190:0/2527429295 conn(0x7f2c300abde0 msgr2=0x7f2c300aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 -- 172.21.15.190:0/2527429295 shutdown_connections 2024-08-19T10:43:56.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.762+0000 7f2c385ce640 1 -- 172.21.15.190:0/2527429295 wait complete. 2024-08-19T10:43:56.785 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm barr 2024-08-19T10:43:56.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.902+0000 7f6c6e4c7640 10 monclient: get_monmap_and_config 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.902+0000 7f6c6e4c7640 10 monclient: build_initial_monmap 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.902+0000 7f6c6e4c7640 10 monclient: monmap: 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:56.906688+0000 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:56.906688+0000 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:56.906 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:56.907 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:56.907 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:56.907 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:56.907 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:56.907 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:56.907 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:56.907 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:56.907 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:56.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 1 Processor -- start 2024-08-19T10:43:56.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 1 -- start start 2024-08-19T10:43:56.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 10 monclient: init 2024-08-19T10:43:56.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:56.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 10 monclient: _add_conns ranks=[2,10,4,7,3,1,0,8,5,6,9] 2024-08-19T10:43:56.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c68139980 0x7f6c68139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 10 monclient(hunting): picked mon.noname-b con 0x7f6c68139980 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6c6813a2a0 0x7f6c680a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 10 monclient(hunting): picked mon.noname-f con 0x7f6c6813a2a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c680a5c70 0x7f6c680a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 10 monclient(hunting): picked mon.noname-c con 0x7f6c680a5c70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6c6809ca60 con 0x7f6c6813a2a0 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6c680a3640 con 0x7f6c68139980 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6c680a8470 con 0x7f6c680a5c70 2024-08-19T10:43:56.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6e4c7640 10 monclient(hunting): authenticate will time out at 1282.477162s 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c677fe640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f6c6813a2a0 msgr2=0x7f6c680a5730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c677fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6c6813a2a0 0x7f6c680a5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6ca64640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c680a5c70 msgr2=0x7f6c680a67e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c6ca64640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c680a5c70 0x7f6c680a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c68139980 0x7f6c68139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c68139980 0x7f6c68139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:38088/0 (socket says 172.21.15.190:38088) 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 1 -- 172.21.15.190:0/193558383 learned_addr learned my addr 172.21.15.190:0/193558383 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 10 monclient(hunting): get_auth_request con 0x7f6c68139980 auth_method 0 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:56.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 10 monclient(hunting): handle_auth_done global_id 5951 payload 274 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 1 -- 172.21.15.190:0/193558383 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c680a5c70 msgr2=0x7f6c680a67e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 1 --2- 172.21.15.190:0/193558383 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c680a5c70 0x7f6c680a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 1 -- 172.21.15.190:0/193558383 >> v2:172.21.15.190:3300/0 conn(0x7f6c6813a2a0 msgr2=0x7f6c680a5730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 1 --2- 172.21.15.190:0/193558383 >> v2:172.21.15.190:3300/0 conn(0x7f6c6813a2a0 0x7f6c680a5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 10 monclient: _finish_hunting 0 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 1 monclient: found mon.noname-b 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:43:56.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.906+0000 7f6c67fff640 1 -- 172.21.15.190:0/193558383 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c680a3850 con 0x7f6c68139980 2024-08-19T10:43:56.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c67fff640 1 -- 172.21.15.190:0/193558383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c68139980 msgr2=0x7f6c68139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:56.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c67fff640 1 --2- 172.21.15.190:0/193558383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c68139980 0x7f6c68139d60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009870 tx=0x7f6c5801c1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e19476c82dd882a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:56.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 1 -- 172.21.15.190:0/193558383 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6c580b0070 con 0x7f6c68139980 2024-08-19T10:43:56.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:56.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:43:56.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 10 monclient: dump: 2024-08-19T10:43:56.914 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:56.914 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:56.914 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:56.914 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:56.914 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:56.914 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 10 monclient: _finish_auth 0 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:56.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 1 -- 172.21.15.190:0/193558383 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c580aa810 con 0x7f6c68139980 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 1 -- 172.21.15.190:0/193558383 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6c580a93e0 con 0x7f6c68139980 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c6e4c7640 5 monclient: authenticate success, global_id 5951 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c6e4c7640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c6e4c7640 4 set_mon_vals no callback set 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.910+0000 7f6c66ffd640 10 monclient: dump: 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:56.916 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:56.917 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:56.917 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:56.917 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:56.917 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:56.917 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:56.917 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:56.917 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:56.917 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:56.917 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:56.917 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:56.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient: shutdown 2024-08-19T10:43:56.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:56.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/193558383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c68139980 msgr2=0x7f6c68139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:56.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 --2- 172.21.15.190:0/193558383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c68139980 0x7f6c68139d60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009870 tx=0x7f6c5801c1a0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/193558383 shutdown_connections 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 --2- 172.21.15.190:0/193558383 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6c680a5c70 0x7f6c680a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 --2- 172.21.15.190:0/193558383 >> v2:172.21.15.190:3300/0 conn(0x7f6c6813a2a0 0x7f6c680a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 --2- 172.21.15.190:0/193558383 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6c68139980 0x7f6c68139d60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/193558383 >> 172.21.15.190:0/193558383 conn(0x7f6c680abde0 msgr2=0x7f6c680ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/193558383 shutdown_connections 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/193558383 wait complete. 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient: build_initial_monmap 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient: monmap: 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:56.923 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 Processor -- start 2024-08-19T10:43:56.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 -- start start 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient: init 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient: _add_conns ranks=[5,8,0,2,3,4,6,1,7] 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6c680a5c70 0x7f6c6809f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient(hunting): picked mon.noname-h con 0x7f6c680a5c70 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6c68139980 0x7f6c6809fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient(hunting): picked mon.noname-e con 0x7f6c68139980 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c6809ffa0 0x7f6c681d1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient(hunting): picked mon.noname-a con 0x7f6c6809ffa0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68143270 con 0x7f6c6809ffa0 2024-08-19T10:43:56.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6c681430f0 con 0x7f6c680a5c70 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6c681433f0 con 0x7f6c68139980 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6e4c7640 10 monclient(hunting): authenticate will time out at 1282.485826s 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6c680a5c70 0x7f6c6809f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6c680a5c70 0x7f6c6809f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:40772/0 (socket says 172.21.15.190:40772) 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 1 -- 172.21.15.190:0/1657406700 learned_addr learned my addr 172.21.15.190:0/1657406700 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 10 monclient(hunting): get_auth_request con 0x7f6c680a5c70 auth_method 0 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:56.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6ca64640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c6809ffa0 0x7f6c681d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c677fe640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6c68139980 0x7f6c6809fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6ca64640 10 monclient(hunting): get_auth_request con 0x7f6c6809ffa0 auth_method 0 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6ca64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6ca64640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c6ca64640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c677fe640 10 monclient(hunting): get_auth_request con 0x7f6c68139980 auth_method 0 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c677fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c677fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:56.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 10 monclient(hunting): handle_auth_done global_id 5936 payload 824 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 1 -- 172.21.15.190:0/1657406700 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6c68139980 msgr2=0x7f6c6809fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6c68139980 0x7f6c6809fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 1 -- 172.21.15.190:0/1657406700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c6809ffa0 msgr2=0x7f6c681d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c6809ffa0 0x7f6c681d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 10 monclient: _finish_hunting 0 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 1 monclient: found mon.noname-h 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.914+0000 7f6c67fff640 1 -- 172.21.15.190:0/1657406700 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c681404b0 con 0x7f6c680a5c70 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c67fff640 1 -- 172.21.15.190:0/1657406700 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6c680a5c70 msgr2=0x7f6c6809f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c67fff640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6c680a5c70 0x7f6c6809f520 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6c5801bfd0 tx=0x7f6c58018f40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 1 -- 172.21.15.190:0/1657406700 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6c580b0070 con 0x7f6c680a5c70 2024-08-19T10:43:56.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 10 monclient: dump: 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:56.929 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 10 monclient: _finish_auth 0 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 1 -- 172.21.15.190:0/1657406700 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c580b1040 con 0x7f6c680a5c70 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c6e4c7640 5 monclient: authenticate success, global_id 5936 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c6e4c7640 10 monclient: _renew_subs 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c6e4c7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:56.930 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/1657406700 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6813b570 con 0x7f6c680a5c70 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 1 -- 172.21.15.190:0/1657406700 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6c58004450 con 0x7f6c680a5c70 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 10 monclient: dump: 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:56.931 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c6e4c7640 10 monclient: _renew_subs 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c6e4c7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c667fc640 4 set_mon_vals no callback set 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/1657406700 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c680a0f70 con 0x7f6c680a5c70 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c6e4c7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:56.932 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c6e4c7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:56.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c6e4c7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:56.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/1657406700 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c680a0380 con 0x7f6c680a5c70 2024-08-19T10:43:56.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 1 -- 172.21.15.190:0/1657406700 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6c580bc060 con 0x7f6c680a5c70 2024-08-19T10:43:56.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6c5403bdf0 0x7f6c5403e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:56.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c64ff9640 1 -- 172.21.15.190:0/1657406700 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(188..188 src has 1..188) ==== 7817+0+0 (secure 0 0 0) 0x7f6c580e8ce0 con 0x7f6c680a5c70 2024-08-19T10:43:56.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c677fe640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6c5403bdf0 0x7f6c5403e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:56.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.918+0000 7f6c677fe640 10 monclient: get_auth_request con 0x7f6c5403bdf0 auth_method 0 2024-08-19T10:43:56.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.922+0000 7f6c677fe640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6c5403bdf0 0x7f6c5403e2b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c006fd0 tx=0x7f6c5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:56.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.922+0000 7f6c64ff9640 1 -- 172.21.15.190:0/1657406700 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6c680a0380 con 0x7f6c680a5c70 2024-08-19T10:43:56.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.922+0000 7f6c64ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:56.934 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:56.922+0000 7f6c64ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:57.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.102+0000 7f6c6e4c7640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "barr"}] 2024-08-19T10:43:57.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.102+0000 7f6c6e4c7640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "barr"}] 2024-08-19T10:43:57.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.102+0000 7f6c6e4c7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:57.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.102+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/1657406700 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "barr"} v 0) -- 0x7f6c68139de0 con 0x7f6c680a5c70 2024-08-19T10:43:57.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.414+0000 7f6c64ff9640 1 -- 172.21.15.190:0/1657406700 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "barr"}]=0 removed item id -19 name 'barr' from crush map v189) ==== 122+0+0 (secure 0 0 0) 0x7f6c58016170 con 0x7f6c680a5c70 2024-08-19T10:43:57.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.414+0000 7f6c64ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "barr"}] 2024-08-19T10:43:57.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.414+0000 7f6c64ff9640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'barr' from crush map 2024-08-19T10:43:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.522+0000 7f6c6e4c7640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "barr"}] 2024-08-19T10:43:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.522+0000 7f6c6e4c7640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "barr"}] 2024-08-19T10:43:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.522+0000 7f6c6e4c7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:57.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.522+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/1657406700 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "barr"} v 0) -- 0x7f6c680a1ad0 con 0x7f6c680a5c70 2024-08-19T10:43:57.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.526+0000 7f6c64ff9640 1 -- 172.21.15.190:0/1657406700 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "barr"}]=0 device 'barr' does not appear in the crush map v189) ==== 122+0+0 (secure 0 0 0) 0x7f6c580bdae0 con 0x7f6c680a5c70 2024-08-19T10:43:57.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.526+0000 7f6c64ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "barr"}] 2024-08-19T10:43:57.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.526+0000 7f6c64ff9640 10 monclient: _finish_command 3 = system:0 device 'barr' does not appear in the crush map 2024-08-19T10:43:57.528 INFO:tasks.workunit.client.0.smithi190.stderr:device 'barr' does not appear in the crush map 2024-08-19T10:43:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/1657406700 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6c5403bdf0 msgr2=0x7f6c5403e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6c5403bdf0 0x7f6c5403e2b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c006fd0 tx=0x7f6c5c008040 comp rx=0 tx=0).stop 2024-08-19T10:43:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 10 monclient: shutdown 2024-08-19T10:43:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/1657406700 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6c680a5c70 msgr2=0x7f6c6809f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6c680a5c70 0x7f6c6809f520 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6c5801bfd0 tx=0x7f6c58018f40 comp rx=0 tx=0).stop 2024-08-19T10:43:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/1657406700 shutdown_connections 2024-08-19T10:43:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c6809ffa0 0x7f6c681d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6c5403bdf0 0x7f6c5403e2b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6c68139980 0x7f6c6809fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 --2- 172.21.15.190:0/1657406700 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6c680a5c70 0x7f6c6809f520 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/1657406700 >> 172.21.15.190:0/1657406700 conn(0x7f6c680abde0 msgr2=0x7f6c68132740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/1657406700 shutdown_connections 2024-08-19T10:43:57.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.530+0000 7f6c6e4c7640 1 -- 172.21.15.190:0/1657406700 wait complete. 2024-08-19T10:43:57.546 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set rm-compat 2024-08-19T10:43:57.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.662+0000 7f59afafc640 10 monclient: get_monmap_and_config 2024-08-19T10:43:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.662+0000 7f59afafc640 10 monclient: build_initial_monmap 2024-08-19T10:43:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.662+0000 7f59afafc640 10 monclient: monmap: 2024-08-19T10:43:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:57.666606+0000 2024-08-19T10:43:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:57.666606+0000 2024-08-19T10:43:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:57.666 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:57.667 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:57.667 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:57.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.662+0000 7f59afafc640 1 Processor -- start 2024-08-19T10:43:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.662+0000 7f59afafc640 1 -- start start 2024-08-19T10:43:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.662+0000 7f59afafc640 10 monclient: init 2024-08-19T10:43:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 10 monclient: _add_conns ranks=[9,5,7,10,1,0,6,3,4,2,8] 2024-08-19T10:43:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f59a80a5940 0x7f59a80720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 10 monclient(hunting): picked mon.noname-a con 0x7f59a80a5940 addr v2:172.21.15.89:3300/0 2024-08-19T10:43:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 0x7f59a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 10 monclient(hunting): picked mon.noname-h con 0x7f59a80a5d20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:57.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59a8139710 0x7f59a8139b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 10 monclient(hunting): picked mon.noname-i con 0x7f59a8139710 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f59a806df10 con 0x7f59a80a5940 2024-08-19T10:43:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f59a806efb0 con 0x7f59a80a5d20 2024-08-19T10:43:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f59a806f290 con 0x7f59a8139710 2024-08-19T10:43:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59afafc640 10 monclient(hunting): authenticate will time out at 1283.236631s 2024-08-19T10:43:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59a8139710 0x7f59a8139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:57.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad097640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 0x7f59a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59a8139710 0x7f59a8139b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:40514/0 (socket says 172.21.15.190:40514) 2024-08-19T10:43:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 1 -- 172.21.15.190:0/2345695019 learned_addr learned my addr 172.21.15.190:0/2345695019 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 10 monclient(hunting): get_auth_request con 0x7f59a8139710 auth_method 0 2024-08-19T10:43:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad097640 10 monclient(hunting): get_auth_request con 0x7f59a80a5d20 auth_method 0 2024-08-19T10:43:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad097640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad097640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad097640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad898640 1 --2- 172.21.15.190:0/2345695019 >> v2:172.21.15.89:3300/0 conn(0x7f59a80a5940 0x7f59a80720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad097640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad097640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad097640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad898640 10 monclient(hunting): get_auth_request con 0x7f59a80a5940 auth_method 0 2024-08-19T10:43:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad898640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:57.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad898640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad898640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 10 monclient(hunting): handle_auth_done global_id 15208 payload 274 2024-08-19T10:43:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 1 -- 172.21.15.190:0/2345695019 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 msgr2=0x7f59a80a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 1 --2- 172.21.15.190:0/2345695019 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 0x7f59a80a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 1 -- 172.21.15.190:0/2345695019 >> v2:172.21.15.89:3300/0 conn(0x7f59a80a5940 msgr2=0x7f59a80720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 1 --2- 172.21.15.190:0/2345695019 >> v2:172.21.15.89:3300/0 conn(0x7f59a80a5940 0x7f59a80720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 10 monclient: _finish_hunting 0 2024-08-19T10:43:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 1 monclient: found mon.noname-i 2024-08-19T10:43:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:57.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ae099640 1 -- 172.21.15.190:0/2345695019 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a80a73d0 con 0x7f59a8139710 2024-08-19T10:43:57.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.666+0000 7f59ad898640 1 --2- 172.21.15.190:0/2345695019 >> v2:172.21.15.89:3300/0 conn(0x7f59a80a5940 0x7f59a80720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:43:57.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ae099640 1 -- 172.21.15.190:0/2345695019 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59a8139710 msgr2=0x7f59a8139b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:57.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ae099640 1 --2- 172.21.15.190:0/2345695019 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59a8139710 0x7f59a8139b90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009e80 tx=0x7f59a001c9f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=17f8b542b2a6089c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:57.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 1 -- 172.21.15.190:0/2345695019 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59a00090d0 con 0x7f59a8139710 2024-08-19T10:43:57.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:57.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:57.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 10 monclient: dump: 2024-08-19T10:43:57.681 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:57.681 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:57.681 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:57.681 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:57.682 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:57.682 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:57.682 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:57.682 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:57.682 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:57.682 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:57.682 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:57.682 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:57.682 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:57.683 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:57.683 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:57.683 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:57.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 10 monclient: _finish_auth 0 2024-08-19T10:43:57.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 10 monclient: _check_auth_tickets 2024-08-19T10:43:57.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:57.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 1 -- 172.21.15.190:0/2345695019 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a00a5aa0 con 0x7f59a8139710 2024-08-19T10:43:57.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 5 monclient: authenticate success, global_id 15208 2024-08-19T10:43:57.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:43:57.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:57.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:57.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 1 -- 172.21.15.190:0/2345695019 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59a00a5c50 con 0x7f59a8139710 2024-08-19T10:43:57.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 4 set_mon_vals no callback set 2024-08-19T10:43:57.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:57.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:57.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59ac896640 10 monclient: dump: 2024-08-19T10:43:57.684 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:57.684 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:57.684 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:57.684 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:57.685 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient: shutdown 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 -- 172.21.15.190:0/2345695019 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59a8139710 msgr2=0x7f59a8139b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 --2- 172.21.15.190:0/2345695019 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59a8139710 0x7f59a8139b90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009e80 tx=0x7f59a001c9f0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 -- 172.21.15.190:0/2345695019 shutdown_connections 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 --2- 172.21.15.190:0/2345695019 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59a8139710 0x7f59a8139b90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 --2- 172.21.15.190:0/2345695019 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 0x7f59a80a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 --2- 172.21.15.190:0/2345695019 >> v2:172.21.15.89:3300/0 conn(0x7f59a80a5940 0x7f59a80720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 -- 172.21.15.190:0/2345695019 >> 172.21.15.190:0/2345695019 conn(0x7f59a80aa530 msgr2=0x7f59a80a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 -- 172.21.15.190:0/2345695019 shutdown_connections 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 -- 172.21.15.190:0/2345695019 wait complete. 2024-08-19T10:43:57.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient: build_initial_monmap 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient: monmap: 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:57.687 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 Processor -- start 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 -- start start 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient: init 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient: _add_conns ranks=[2,5,0,3,6,7,4,1,8] 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59a80a5940 0x7f59a8138210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient(hunting): picked mon.noname-b con 0x7f59a80a5940 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 0x7f59a8138750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:57.688 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient(hunting): picked mon.noname-h con 0x7f59a80a5d20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a8139710 0x7f59a8120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient(hunting): picked mon.noname-a con 0x7f59a8139710 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f59a806e7e0 con 0x7f59a8139710 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f59a806e660 con 0x7f59a80a5940 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f59a8143390 con 0x7f59a80a5d20 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.670+0000 7f59afafc640 10 monclient(hunting): authenticate will time out at 1283.243903s 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 0x7f59a8138750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:57.689 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 0x7f59a8138750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:40780/0 (socket says 172.21.15.190:40780) 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 1 -- 172.21.15.190:0/625177681 learned_addr learned my addr 172.21.15.190:0/625177681 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 10 monclient(hunting): get_auth_request con 0x7f59a80a5d20 auth_method 0 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ae099640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a8139710 0x7f59a8120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad898640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59a80a5940 0x7f59a8138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ae099640 10 monclient(hunting): get_auth_request con 0x7f59a8139710 auth_method 0 2024-08-19T10:43:57.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ae099640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ae099640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ae099640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad898640 10 monclient(hunting): get_auth_request con 0x7f59a80a5940 auth_method 0 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad898640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad898640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad898640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 10 monclient(hunting): handle_auth_done global_id 5954 payload 824 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 1 -- 172.21.15.190:0/625177681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59a80a5940 msgr2=0x7f59a8138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59a80a5940 0x7f59a8138210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 1 -- 172.21.15.190:0/625177681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a8139710 msgr2=0x7f59a8120580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a8139710 0x7f59a8120580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:57.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 10 monclient: _finish_hunting 0 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 1 monclient: found mon.noname-h 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 1 -- 172.21.15.190:0/625177681 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a80745c0 con 0x7f59a80a5d20 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 1 -- 172.21.15.190:0/625177681 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 msgr2=0x7f59a8138750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59ad097640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 0x7f59a8138750 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f59a400e930 tx=0x7f59a400edf0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 1 -- 172.21.15.190:0/625177681 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59a4017070 con 0x7f59a80a5d20 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 10 monclient: dump: 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:57.692 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:57.693 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 1 -- 172.21.15.190:0/625177681 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a400cc10 con 0x7f59a80a5d20 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59afafc640 5 monclient: authenticate success, global_id 5954 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59afafc640 10 monclient: _renew_subs 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59afafc640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59afafc640 1 -- 172.21.15.190:0/625177681 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a8139c10 con 0x7f59a80a5d20 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 1 -- 172.21.15.190:0/625177681 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59a400cdc0 con 0x7f59a80a5d20 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:57.694 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 10 monclient: dump: 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:57.695 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599f7fe640 4 set_mon_vals no callback set 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59afafc640 10 monclient: _renew_subs 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59afafc640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59afafc640 1 -- 172.21.15.190:0/625177681 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a8123c40 con 0x7f59a80a5d20 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f599e7fc640 1 -- 172.21.15.190:0/625177681 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f59a401a070 con 0x7f59a80a5d20 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59afafc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59afafc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59afafc640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:57.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.674+0000 7f59afafc640 1 -- 172.21.15.190:0/625177681 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a80a6100 con 0x7f59a80a5d20 2024-08-19T10:43:57.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.678+0000 7f599e7fc640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f598c03be30 0x7f598c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:57.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.678+0000 7f599e7fc640 1 -- 172.21.15.190:0/625177681 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(189..189 src has 1..189) ==== 7601+0+0 (secure 0 0 0) 0x7f59a400f040 con 0x7f59a80a5d20 2024-08-19T10:43:57.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.678+0000 7f59ad898640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f598c03be30 0x7f598c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:57.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.678+0000 7f59ad898640 10 monclient: get_auth_request con 0x7f598c03be30 auth_method 0 2024-08-19T10:43:57.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.678+0000 7f59ad898640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f598c03be30 0x7f598c03e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f599800c880 tx=0x7f5998008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:57.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.682+0000 7f599e7fc640 1 -- 172.21.15.190:0/625177681 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f59a4021070 con 0x7f59a80a5d20 2024-08-19T10:43:57.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.682+0000 7f599e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:57.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.682+0000 7f599e7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:57.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.858+0000 7f59afafc640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:57.861 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.858+0000 7f59afafc640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:57.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.858+0000 7f59afafc640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:57.862 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:57.858+0000 7f59afafc640 1 -- 172.21.15.190:0/625177681 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f59a806ff20 con 0x7f59a80a5d20 2024-08-19T10:43:58.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:58.414+0000 7f599e7fc640 1 -- 172.21.15.190:0/625177681 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v190) ==== 78+0+0 (secure 0 0 0) 0x7f59a404d720 con 0x7f59a80a5d20 2024-08-19T10:43:58.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:58.414+0000 7f599e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:58.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:58.414+0000 7f599e7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:43:58.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:58.526+0000 7f59afafc640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:58.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:58.526+0000 7f59afafc640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:58.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:58.526+0000 7f59afafc640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:43:58.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:58.526+0000 7f59afafc640 1 -- 172.21.15.190:0/625177681 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7f59a8138c90 con 0x7f59a80a5d20 2024-08-19T10:43:59.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.434+0000 7f599e7fc640 1 -- 172.21.15.190:0/625177681 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v191) ==== 78+0+0 (secure 0 0 0) 0x7f59a401ac80 con 0x7f59a80a5d20 2024-08-19T10:43:59.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.434+0000 7f599e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:43:59.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.434+0000 7f599e7fc640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:43:59.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 -- 172.21.15.190:0/625177681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f598c03be30 msgr2=0x7f598c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:59.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f598c03be30 0x7f598c03e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f599800c880 tx=0x7f5998008040 comp rx=0 tx=0).stop 2024-08-19T10:43:59.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 10 monclient: shutdown 2024-08-19T10:43:59.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:59.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 -- 172.21.15.190:0/625177681 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 msgr2=0x7f59a8138750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:59.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 0x7f59a8138750 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f59a400e930 tx=0x7f59a400edf0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 -- 172.21.15.190:0/625177681 shutdown_connections 2024-08-19T10:43:59.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f598c03be30 0x7f598c03e2f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59a8139710 0x7f59a8120580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59a80a5d20 0x7f59a8138750 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 --2- 172.21.15.190:0/625177681 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59a80a5940 0x7f59a8138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 -- 172.21.15.190:0/625177681 >> 172.21.15.190:0/625177681 conn(0x7f59a80aa530 msgr2=0x7f59a80a4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:59.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 -- 172.21.15.190:0/625177681 shutdown_connections 2024-08-19T10:43:59.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.438+0000 7f59afafc640 1 -- 172.21.15.190:0/625177681 wait complete. 2024-08-19T10:43:59.459 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set create-compat 2024-08-19T10:43:59.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient: get_monmap_and_config 2024-08-19T10:43:59.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient: build_initial_monmap 2024-08-19T10:43:59.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient: monmap: 2024-08-19T10:43:59.576 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:59.576 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:43:59.577328+0000 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:43:59.577328+0000 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:59.577 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:43:59.578 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:43:59.578 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:59.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 1 Processor -- start 2024-08-19T10:43:59.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 1 -- start start 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient: init 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient: _add_conns ranks=[7,10,2,0,4,5,3,1,9,8,6] 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139980 0x7f17540a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient(hunting): picked mon.noname-i con 0x7f1754139980 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1754139d60 0x7f17540a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient(hunting): picked mon.noname-f con 0x7f1754139d60 addr v2:172.21.15.190:3300/0 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f175413a140 0x7f17540a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient(hunting): picked mon.noname-b con 0x7f175413a140 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f17540a8620 con 0x7f1754139d60 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:59.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f17540a5b50 con 0x7f175413a140 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f17540a45b0 con 0x7f1754139980 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f175adc3640 10 monclient(hunting): authenticate will time out at 1285.147200s 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1753fff640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f1754139d60 msgr2=0x7f17540a67e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1753fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1754139d60 0x7f17540a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139980 0x7f17540a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139980 0x7f17540a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:40530/0 (socket says 172.21.15.190:40530) 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 1 -- 172.21.15.190:0/4232949592 learned_addr learned my addr 172.21.15.190:0/4232949592 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 10 monclient(hunting): get_auth_request con 0x7f1754139980 auth_method 0 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:59.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1759360640 1 --2- 172.21.15.190:0/4232949592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f175413a140 0x7f17540a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 10 monclient(hunting): handle_auth_done global_id 15217 payload 274 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 1 -- 172.21.15.190:0/4232949592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f175413a140 msgr2=0x7f17540a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 1 --2- 172.21.15.190:0/4232949592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f175413a140 0x7f17540a6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 1 -- 172.21.15.190:0/4232949592 >> v2:172.21.15.190:3300/0 conn(0x7f1754139d60 msgr2=0x7f17540a67e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 1 --2- 172.21.15.190:0/4232949592 >> v2:172.21.15.190:3300/0 conn(0x7f1754139d60 0x7f17540a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 10 monclient: _finish_hunting 0 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 1 monclient: found mon.noname-i 2024-08-19T10:43:59.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:43:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1758b5f640 1 -- 172.21.15.190:0/4232949592 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17540a4890 con 0x7f1754139980 2024-08-19T10:43:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1759360640 10 monclient: get_auth_request con 0x7f175413a140 auth_method 0 2024-08-19T10:43:59.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.574+0000 7f1759360640 1 --2- 172.21.15.190:0/4232949592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f175413a140 0x7f17540a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:43:59.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 1 -- 172.21.15.190:0/4232949592 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139980 msgr2=0x7f17540a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:59.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 1 --2- 172.21.15.190:0/4232949592 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139980 0x7f17540a5610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17440098a0 tx=0x7f174401c420 comp rx=0 tx=0).ready entity=mon.7 client_cookie=6bf188284c7f8a87 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:59.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 1 -- 172.21.15.190:0/4232949592 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17440b0070 con 0x7f1754139980 2024-08-19T10:43:59.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:59.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:43:59.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 10 monclient: dump: 2024-08-19T10:43:59.766 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:59.766 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:59.766 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:59.767 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 10 monclient: _finish_auth 0 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 10 monclient: _check_auth_tickets 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 1 -- 172.21.15.190:0/4232949592 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17440aa810 con 0x7f1754139980 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 1 -- 172.21.15.190:0/4232949592 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17440a93e0 con 0x7f1754139980 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 5 monclient: authenticate success, global_id 15217 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient: get_monmap_and_config success 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 4 set_mon_vals no callback set 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f17537fe640 10 monclient: dump: 2024-08-19T10:43:59.768 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:59.769 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:59.770 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:59.770 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:59.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient: shutdown 2024-08-19T10:43:59.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:43:59.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 -- 172.21.15.190:0/4232949592 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139980 msgr2=0x7f17540a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:59.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 --2- 172.21.15.190:0/4232949592 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139980 0x7f17540a5610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17440098a0 tx=0x7f174401c420 comp rx=0 tx=0).stop 2024-08-19T10:43:59.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 -- 172.21.15.190:0/4232949592 shutdown_connections 2024-08-19T10:43:59.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 --2- 172.21.15.190:0/4232949592 >> v2:172.21.15.190:3300/0 conn(0x7f1754139d60 0x7f17540a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 --2- 172.21.15.190:0/4232949592 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f175413a140 0x7f17540a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 --2- 172.21.15.190:0/4232949592 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139980 0x7f17540a5610 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 -- 172.21.15.190:0/4232949592 >> 172.21.15.190:0/4232949592 conn(0x7f17540abde0 msgr2=0x7f17540ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:43:59.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 -- 172.21.15.190:0/4232949592 shutdown_connections 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 -- 172.21.15.190:0/4232949592 wait complete. 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient: build_initial_monmap 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient: monmap: 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:43:59.772 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 Processor -- start 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 -- start start 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient: init 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient: _reopen_session rank -1 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient: _add_conns ranks=[3,7,8,5,0,6,4,1,2] 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1754139980 0x7f17541d1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:59.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient(hunting): picked mon.noname-g con 0x7f1754139980 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:43:59.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139d60 0x7f17541d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:59.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient(hunting): picked mon.noname-i con 0x7f1754139d60 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:43:59.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f175413a140 0x7f175413a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:59.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient(hunting): picked mon.noname-e con 0x7f175413a140 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:43:59.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:59.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1754143190 con 0x7f1754139980 2024-08-19T10:43:59.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:59.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f1754143010 con 0x7f1754139d60 2024-08-19T10:43:59.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:43:59.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f1754143310 con 0x7f175413a140 2024-08-19T10:43:59.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient(hunting): _renew_subs 2024-08-19T10:43:59.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f175adc3640 10 monclient(hunting): authenticate will time out at 1285.155429s 2024-08-19T10:43:59.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1753fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139d60 0x7f17541d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:59.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1753fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139d60 0x7f17541d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:40540/0 (socket says 172.21.15.190:40540) 2024-08-19T10:43:59.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1753fff640 1 -- 172.21.15.190:0/2595898187 learned_addr learned my addr 172.21.15.190:0/2595898187 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:43:59.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1754139980 0x7f17541d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:59.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1753fff640 10 monclient(hunting): get_auth_request con 0x7f1754139d60 auth_method 0 2024-08-19T10:43:59.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1753fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:59.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1753fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:59.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1753fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:59.775 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 10 monclient(hunting): get_auth_request con 0x7f1754139980 auth_method 0 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1753fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1753fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1753fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 10 monclient(hunting): handle_auth_done global_id 34302 payload 824 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1759360640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f175413a140 0x7f175413a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 1 -- 172.21.15.190:0/2595898187 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f175413a140 msgr2=0x7f175413a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:59.776 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f175413a140 0x7f175413a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 1 -- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139d60 msgr2=0x7f17541d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:43:59.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139d60 0x7f17541d19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:43:59.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 10 monclient: _finish_hunting 0 2024-08-19T10:43:59.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 1 monclient: found mon.noname-g 2024-08-19T10:43:59.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:43:59.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:43:59.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1758b5f640 1 -- 172.21.15.190:0/2595898187 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175413d670 con 0x7f1754139980 2024-08-19T10:43:59.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1759360640 10 monclient: get_auth_request con 0x7f175413a140 auth_method 0 2024-08-19T10:43:59.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1753fff640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139d60 0x7f17541d19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:43:59.777 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.582+0000 7f1759360640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f175413a140 0x7f175413a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:43:59.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f1758b5f640 1 -- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1754139980 msgr2=0x7f17541d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:43:59.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f1758b5f640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1754139980 0x7f17541d1490 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f17440ab000 tx=0x7f17440191c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:59.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 1 -- 172.21.15.190:0/2595898187 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17440b0040 con 0x7f1754139980 2024-08-19T10:43:59.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:59.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:43:59.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 10 monclient: dump: 2024-08-19T10:43:59.780 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:59.781 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:59.782 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:59.782 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:59.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 10 monclient: _finish_auth 0 2024-08-19T10:43:59.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 10 monclient: _check_auth_tickets 2024-08-19T10:43:59.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:43:59.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f175adc3640 5 monclient: authenticate success, global_id 34302 2024-08-19T10:43:59.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 1 -- 172.21.15.190:0/2595898187 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17440044a0 con 0x7f1754139980 2024-08-19T10:43:59.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f175adc3640 10 monclient: _renew_subs 2024-08-19T10:43:59.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f175adc3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:59.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f175adc3640 1 -- 172.21.15.190:0/2595898187 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17541355f0 con 0x7f1754139980 2024-08-19T10:43:59.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 1 -- 172.21.15.190:0/2595898187 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1744003a90 con 0x7f1754139980 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 10 monclient: dump: 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:43:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:43:59.784 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:43:59.784 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:43:59.784 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:43:59.784 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:43:59.784 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:43:59.784 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:43:59.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f1752ffd640 4 set_mon_vals no callback set 2024-08-19T10:43:59.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f175adc3640 10 monclient: _renew_subs 2024-08-19T10:43:59.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f175adc3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:59.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f175adc3640 1 -- 172.21.15.190:0/2595898187 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f175413c710 con 0x7f1754139980 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 1 -- 172.21.15.190:0/2595898187 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1744003c70 con 0x7f1754139980 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f175adc3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f175adc3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f175adc3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f175adc3640 1 -- 172.21.15.190:0/2595898187 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17540a5610 con 0x7f1754139980 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f174003be00 0x7f174003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f17517fa640 1 -- 172.21.15.190:0/2595898187 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(191..191 src has 1..191) ==== 7273+0+0 (secure 0 0 0) 0x7f17440e8790 con 0x7f1754139980 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f1753fff640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f174003be00 0x7f174003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.586+0000 7f1753fff640 10 monclient: get_auth_request con 0x7f174003be00 auth_method 0 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.590+0000 7f1753fff640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f174003be00 0x7f174003e2c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f17540a1b60 tx=0x7f17480091c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.594+0000 7f17517fa640 1 -- 172.21.15.190:0/2595898187 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1744016130 con 0x7f1754139980 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.594+0000 7f17517fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:43:59.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.594+0000 7f17517fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:43:59.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.770+0000 7f175adc3640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:43:59.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.770+0000 7f175adc3640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:43:59.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.770+0000 7f175adc3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:43:59.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:43:59.770+0000 7f175adc3640 1 -- 172.21.15.190:0/2595898187 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7f175406c8d0 con 0x7f1754139980 2024-08-19T10:44:00.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.438+0000 7f17517fa640 1 -- 172.21.15.190:0/2595898187 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 v192) ==== 82+0+0 (secure 0 0 0) 0x7f17440e67d0 con 0x7f1754139980 2024-08-19T10:44:00.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.438+0000 7f17517fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:44:00.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.438+0000 7f17517fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:00.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.550+0000 7f175adc3640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:44:00.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.550+0000 7f175adc3640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:44:00.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.550+0000 7f175adc3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:00.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.550+0000 7f175adc3640 1 -- 172.21.15.190:0/2595898187 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush weight-set create-compat"} v 0) -- 0x7f17541d1f10 con 0x7f1754139980 2024-08-19T10:44:00.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.902+0000 7f17517fa640 1 -- 172.21.15.190:0/2595898187 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set create-compat"}]=0 compat weight-set already created v192) ==== 115+0+0 (secure 0 0 0) 0x7f17440bca90 con 0x7f1754139980 2024-08-19T10:44:00.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.902+0000 7f17517fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set create-compat"}] 2024-08-19T10:44:00.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.902+0000 7f17517fa640 10 monclient: _finish_command 3 = system:0 compat weight-set already created 2024-08-19T10:44:00.906 INFO:tasks.workunit.client.0.smithi190.stderr:compat weight-set already created 2024-08-19T10:44:00.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.906+0000 7f175adc3640 1 -- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f174003be00 msgr2=0x7f174003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:00.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.906+0000 7f175adc3640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f174003be00 0x7f174003e2c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f17540a1b60 tx=0x7f17480091c0 comp rx=0 tx=0).stop 2024-08-19T10:44:00.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.906+0000 7f175adc3640 10 monclient: shutdown 2024-08-19T10:44:00.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.906+0000 7f175adc3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:00.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.906+0000 7f175adc3640 1 -- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1754139980 msgr2=0x7f17541d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:00.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.906+0000 7f175adc3640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1754139980 0x7f17541d1490 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f17440ab000 tx=0x7f17440191c0 comp rx=0 tx=0).stop 2024-08-19T10:44:00.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.910+0000 7f175adc3640 1 -- 172.21.15.190:0/2595898187 shutdown_connections 2024-08-19T10:44:00.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.910+0000 7f175adc3640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f174003be00 0x7f174003e2c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:00.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.910+0000 7f175adc3640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f175413a140 0x7f175413a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:00.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.910+0000 7f175adc3640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1754139d60 0x7f17541d19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:00.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.910+0000 7f175adc3640 1 --2- 172.21.15.190:0/2595898187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1754139980 0x7f17541d1490 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:00.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.910+0000 7f175adc3640 1 -- 172.21.15.190:0/2595898187 >> 172.21.15.190:0/2595898187 conn(0x7f17540abde0 msgr2=0x7f17541388d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:00.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.910+0000 7f175adc3640 1 -- 172.21.15.190:0/2595898187 shutdown_connections 2024-08-19T10:44:00.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:00.910+0000 7f175adc3640 1 -- 172.21.15.190:0/2595898187 wait complete. 2024-08-19T10:44:00.934 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket r1 rack root=default 2024-08-19T10:44:01.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient: get_monmap_and_config 2024-08-19T10:44:01.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient: build_initial_monmap 2024-08-19T10:44:01.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient: monmap: 2024-08-19T10:44:01.055 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:01.055 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:01.055 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:01.055908+0000 2024-08-19T10:44:01.055 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:01.055908+0000 2024-08-19T10:44:01.055 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:01.055 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:01.055 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:01.056 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:01.056 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:01.056 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:01.056 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:01.056 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:01.056 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:01.056 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:01.056 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:01.056 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:01.056 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:01.056 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:01.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 1 Processor -- start 2024-08-19T10:44:01.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 1 -- start start 2024-08-19T10:44:01.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient: init 2024-08-19T10:44:01.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:01.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient: _add_conns ranks=[7,10,5,6,4,9,0,1,8,2,3] 2024-08-19T10:44:01.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05100a36a0 0x7f05100a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:01.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient(hunting): picked mon.noname-i con 0x7f05100a36a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:01.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05100a19d0 0x7f05100a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:01.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient(hunting): picked mon.noname-f con 0x7f05100a19d0 addr v2:172.21.15.190:3300/0 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0510137470 0x7f05101378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient(hunting): picked mon.noname-h con 0x7f0510137470 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f05100a4500 con 0x7f05100a19d0 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f051009d050 con 0x7f0510137470 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f051006f250 con 0x7f05100a36a0 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f0516b61640 10 monclient(hunting): authenticate will time out at 1286.625703s 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f050ffff640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7f05100a19d0 msgr2=0x7f05100a3fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:01.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f050ffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05100a19d0 0x7f05100a3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05150fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0510137470 0x7f05101378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05100a36a0 0x7f05100a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05150fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0510137470 0x7f05101378d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:40794/0 (socket says 172.21.15.190:40794) 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05150fe640 1 -- 172.21.15.190:0/491545135 learned_addr learned my addr 172.21.15.190:0/491545135 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05150fe640 10 monclient(hunting): get_auth_request con 0x7f0510137470 auth_method 0 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05150fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05150fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05150fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 10 monclient(hunting): get_auth_request con 0x7f05100a36a0 auth_method 0 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:01.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05150fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05150fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05150fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 10 monclient(hunting): handle_auth_done global_id 15235 payload 274 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 1 -- 172.21.15.190:0/491545135 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0510137470 msgr2=0x7f05101378d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 1 --2- 172.21.15.190:0/491545135 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0510137470 0x7f05101378d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 1 -- 172.21.15.190:0/491545135 >> v2:172.21.15.190:3300/0 conn(0x7f05100a19d0 msgr2=0x7f05100a3fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 1 --2- 172.21.15.190:0/491545135 >> v2:172.21.15.190:3300/0 conn(0x7f05100a19d0 0x7f05100a3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:01.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 1 monclient: found mon.noname-i 2024-08-19T10:44:01.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:01.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:01.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05148fd640 1 -- 172.21.15.190:0/491545135 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051009dec0 con 0x7f05100a36a0 2024-08-19T10:44:01.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.054+0000 7f05150fe640 1 --2- 172.21.15.190:0/491545135 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0510137470 0x7f05101378d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:01.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f05148fd640 1 -- 172.21.15.190:0/491545135 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05100a36a0 msgr2=0x7f05100a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:01.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f05148fd640 1 --2- 172.21.15.190:0/491545135 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05100a36a0 0x7f05100a3a80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f05000097f0 tx=0x7f050001c380 comp rx=0 tx=0).ready entity=mon.7 client_cookie=190c9c363195669a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:01.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 1 -- 172.21.15.190:0/491545135 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f05000b0070 con 0x7f05100a36a0 2024-08-19T10:44:01.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:01.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:01.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 10 monclient: dump: 2024-08-19T10:44:01.069 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:01.069 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:01.069 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:01.069 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:01.070 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:01.070 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:01.070 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:01.070 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:01.071 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:01.071 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:01.071 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:01.071 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:01.071 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:01.071 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:01.071 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:01.071 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:01.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:44:01.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:44:01.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 1 -- 172.21.15.190:0/491545135 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05000aa810 con 0x7f05100a36a0 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 5 monclient: authenticate success, global_id 15235 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 1 -- 172.21.15.190:0/491545135 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f05000a9300 con 0x7f05100a36a0 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f050f7fe640 10 monclient: dump: 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:01.072 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 4 set_mon_vals no callback set 2024-08-19T10:44:01.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 10 monclient: shutdown 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 1 -- 172.21.15.190:0/491545135 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05100a36a0 msgr2=0x7f05100a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 1 --2- 172.21.15.190:0/491545135 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05100a36a0 0x7f05100a3a80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f05000097f0 tx=0x7f050001c380 comp rx=0 tx=0).stop 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 1 -- 172.21.15.190:0/491545135 shutdown_connections 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 1 --2- 172.21.15.190:0/491545135 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0510137470 0x7f05101378d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 1 --2- 172.21.15.190:0/491545135 >> v2:172.21.15.190:3300/0 conn(0x7f05100a19d0 0x7f05100a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 1 --2- 172.21.15.190:0/491545135 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f05100a36a0 0x7f05100a3a80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 1 -- 172.21.15.190:0/491545135 >> 172.21.15.190:0/491545135 conn(0x7f05100a4790 msgr2=0x7f05100a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 1 -- 172.21.15.190:0/491545135 shutdown_connections 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.058+0000 7f0516b61640 1 -- 172.21.15.190:0/491545135 wait complete. 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient: build_initial_monmap 2024-08-19T10:44:01.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient: monmap: 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:01.075 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 1 Processor -- start 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 1 -- start start 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient: init 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient: _add_conns ranks=[1,3,0,4,7,2,8,5,6] 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05100a19d0 0x7f0510134470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient(hunting): picked mon.noname-f con 0x7f05100a19d0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05100a36a0 0x7f05101349b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient(hunting): picked mon.noname-g con 0x7f05100a36a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0510137470 0x7f051011e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:01.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient(hunting): picked mon.noname-a con 0x7f0510137470 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f051009ca60 con 0x7f0510137470 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f05100a2050 con 0x7f05100a19d0 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f05101410f0 con 0x7f05100a36a0 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f05148fd640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05100a19d0 msgr2=0x7f0510134470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f0516b61640 10 monclient(hunting): authenticate will time out at 1286.633144s 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f05148fd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05100a19d0 0x7f0510134470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05100a36a0 0x7f05101349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:01.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05100a36a0 0x7f05101349b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:33788/0 (socket says 172.21.15.190:33788) 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 1 -- 172.21.15.190:0/2753518395 learned_addr learned my addr 172.21.15.190:0/2753518395 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 10 monclient(hunting): get_auth_request con 0x7f05100a36a0 auth_method 0 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 10 monclient(hunting): handle_auth_done global_id 34311 payload 824 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 1 -- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05100a19d0 msgr2=0x7f0510134470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05100a19d0 0x7f0510134470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:01.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 1 -- 172.21.15.190:0/2753518395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0510137470 msgr2=0x7f051011e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0510137470 0x7f051011e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 1 monclient: found mon.noname-g 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 1 -- 172.21.15.190:0/2753518395 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0510121e30 con 0x7f05100a36a0 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 1 -- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05100a36a0 msgr2=0x7f05101349b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050ffff640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05100a36a0 0x7f05101349b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f050400b550 tx=0x7f050400ba10 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050d7fa640 1 -- 172.21.15.190:0/2753518395 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0504003db0 con 0x7f05100a36a0 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050d7fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.062+0000 7f050d7fa640 10 monclient: dump: 2024-08-19T10:44:01.079 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:01.080 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f0516b61640 5 monclient: authenticate success, global_id 34311 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 1 -- 172.21.15.190:0/2753518395 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05040054e0 con 0x7f05100a36a0 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 1 -- 172.21.15.190:0/2753518395 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0504005690 con 0x7f05100a36a0 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050effd640 4 set_mon_vals no callback set 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 10 monclient: dump: 2024-08-19T10:44:01.081 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:01.082 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f0516b61640 10 monclient: _renew_subs 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f0516b61640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f0516b61640 1 -- 172.21.15.190:0/2753518395 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0510077880 con 0x7f05100a36a0 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f0516b61640 10 monclient: _renew_subs 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f0516b61640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f0516b61640 1 -- 172.21.15.190:0/2753518395 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0510121590 con 0x7f05100a36a0 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f0516b61640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f0516b61640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f0516b61640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f0516b61640 1 -- 172.21.15.190:0/2753518395 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05101378d0 con 0x7f05100a36a0 2024-08-19T10:44:01.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 1 -- 172.21.15.190:0/2753518395 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0504017030 con 0x7f05100a36a0 2024-08-19T10:44:01.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04fc03be00 0x7f04fc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:01.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f050d7fa640 1 -- 172.21.15.190:0/2753518395 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(192..192 src has 1..192) ==== 7601+0+0 (secure 0 0 0) 0x7f0504015020 con 0x7f05100a36a0 2024-08-19T10:44:01.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f05148fd640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04fc03be00 0x7f04fc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:01.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.066+0000 7f05148fd640 10 monclient: get_auth_request con 0x7f04fc03be00 auth_method 0 2024-08-19T10:44:01.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.070+0000 7f05148fd640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04fc03be00 0x7f04fc03e2c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f051006f1c0 tx=0x7f0500008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:01.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.070+0000 7f050d7fa640 1 -- 172.21.15.190:0/2753518395 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f050401e060 con 0x7f05100a36a0 2024-08-19T10:44:01.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.070+0000 7f050d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:01.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.070+0000 7f050d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:01.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.246+0000 7f0516b61640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-08-19T10:44:01.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.246+0000 7f0516b61640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-08-19T10:44:01.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.246+0000 7f0516b61640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:01.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.246+0000 7f0516b61640 1 -- 172.21.15.190:0/2753518395 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]} v 0) -- 0x7f051006dba0 con 0x7f05100a36a0 2024-08-19T10:44:01.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.914+0000 7f050d7fa640 1 -- 172.21.15.190:0/2753518395 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}]=0 added bucket r1 type rack to location {root=default} v193) ==== 176+0+0 (secure 0 0 0) 0x7f050400d040 con 0x7f05100a36a0 2024-08-19T10:44:01.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.914+0000 7f050d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-08-19T10:44:01.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:01.914+0000 7f050d7fa640 10 monclient: _finish_command 2 = system:0 added bucket r1 type rack to location {root=default} 2024-08-19T10:44:02.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.022+0000 7f0516b61640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-08-19T10:44:02.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.022+0000 7f0516b61640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-08-19T10:44:02.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.022+0000 7f0516b61640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:02.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.022+0000 7f0516b61640 1 -- 172.21.15.190:0/2753518395 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]} v 0) -- 0x7f0510120630 con 0x7f05100a36a0 2024-08-19T10:44:02.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f050d7fa640 1 -- 172.21.15.190:0/2753518395 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}]=0 bucket 'r1' already exists v193) ==== 150+0+0 (secure 0 0 0) 0x7f0504010e20 con 0x7f05100a36a0 2024-08-19T10:44:02.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f050d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "r1", "type": "rack", "args": ["root=default"]}] 2024-08-19T10:44:02.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f050d7fa640 10 monclient: _finish_command 3 = system:0 bucket 'r1' already exists 2024-08-19T10:44:02.028 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'r1' already exists 2024-08-19T10:44:02.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 -- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04fc03be00 msgr2=0x7f04fc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:02.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04fc03be00 0x7f04fc03e2c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f051006f1c0 tx=0x7f0500008040 comp rx=0 tx=0).stop 2024-08-19T10:44:02.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 10 monclient: shutdown 2024-08-19T10:44:02.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:02.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 -- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05100a36a0 msgr2=0x7f05101349b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:02.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05100a36a0 0x7f05101349b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f050400b550 tx=0x7f050400ba10 comp rx=0 tx=0).stop 2024-08-19T10:44:02.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 -- 172.21.15.190:0/2753518395 shutdown_connections 2024-08-19T10:44:02.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04fc03be00 0x7f04fc03e2c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:02.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0510137470 0x7f051011e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:02.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05100a36a0 0x7f05101349b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:02.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 --2- 172.21.15.190:0/2753518395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05100a19d0 0x7f0510134470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:02.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 -- 172.21.15.190:0/2753518395 >> 172.21.15.190:0/2753518395 conn(0x7f05100a4790 msgr2=0x7f05100a5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:02.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 -- 172.21.15.190:0/2753518395 shutdown_connections 2024-08-19T10:44:02.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.026+0000 7f0516b61640 1 -- 172.21.15.190:0/2753518395 wait complete. 2024-08-19T10:44:02.046 INFO:tasks.workunit.client.0.smithi190.stderr:++ seq 1 32 2024-08-19T10:44:02.329 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:02.329 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h1 host rack=r1 2024-08-19T10:44:02.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient: get_monmap_and_config 2024-08-19T10:44:02.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient: build_initial_monmap 2024-08-19T10:44:02.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient: monmap: 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:02.457460+0000 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:02.457460+0000 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:02.457 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:02.458 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:02.458 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:02.458 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:02.458 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:02.458 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:02.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 1 Processor -- start 2024-08-19T10:44:02.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 1 -- start start 2024-08-19T10:44:02.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient: init 2024-08-19T10:44:02.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:02.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient: _add_conns ranks=[7,4,10,6,9,1,0,5,3,2,8] 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0a4069710 0x7ff0a409ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient(hunting): picked mon.noname-i con 0x7ff0a4069710 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0a41375c0 0x7ff0a41379a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient(hunting): picked mon.noname-c con 0x7ff0a41375c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0a412d510 0x7ff0a412d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient(hunting): picked mon.noname-f con 0x7ff0a412d510 addr v2:172.21.15.190:3300/0 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff0a40a20f0 con 0x7ff0a412d510 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff0a40a2950 con 0x7ff0a41375c0 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff0a40a1cd0 con 0x7ff0a4069710 2024-08-19T10:44:02.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0aa458640 10 monclient(hunting): authenticate will time out at 1288.027896s 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0a89f5640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7ff0a412d510 msgr2=0x7ff0a412d990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.454+0000 7ff0a89f5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0a412d510 0x7ff0a412d990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a37fe640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0a41375c0 msgr2=0x7ff0a41379a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0a4069710 0x7ff0a409ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a37fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0a41375c0 0x7ff0a41379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0a4069710 0x7ff0a409ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:40554/0 (socket says 172.21.15.190:40554) 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 1 -- 172.21.15.190:0/173421289 learned_addr learned my addr 172.21.15.190:0/173421289 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 10 monclient(hunting): get_auth_request con 0x7ff0a4069710 auth_method 0 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:02.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 10 monclient(hunting): handle_auth_done global_id 15244 payload 274 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 1 -- 172.21.15.190:0/173421289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0a41375c0 msgr2=0x7ff0a41379a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 1 --2- 172.21.15.190:0/173421289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0a41375c0 0x7ff0a41379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 1 -- 172.21.15.190:0/173421289 >> v2:172.21.15.190:3300/0 conn(0x7ff0a412d510 msgr2=0x7ff0a412d990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 1 --2- 172.21.15.190:0/173421289 >> v2:172.21.15.190:3300/0 conn(0x7ff0a412d510 0x7ff0a412d990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 1 monclient: found mon.noname-i 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:02.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.458+0000 7ff0a3fff640 1 -- 172.21.15.190:0/173421289 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a40a2670 con 0x7ff0a4069710 2024-08-19T10:44:02.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a3fff640 1 -- 172.21.15.190:0/173421289 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0a4069710 msgr2=0x7ff0a409ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:02.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a3fff640 1 --2- 172.21.15.190:0/173421289 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0a4069710 0x7ff0a409ed60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff0940098a0 tx=0x7ff09401c440 comp rx=0 tx=0).ready entity=mon.7 client_cookie=921193f7b7cf0f10 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:02.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 1 -- 172.21.15.190:0/173421289 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0940b0070 con 0x7ff0a4069710 2024-08-19T10:44:02.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:02.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:02.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 10 monclient: dump: 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:02.468 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 1 -- 172.21.15.190:0/173421289 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0940a4e50 con 0x7ff0a4069710 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 1 -- 172.21.15.190:0/173421289 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0940aac80 con 0x7ff0a4069710 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 5 monclient: authenticate success, global_id 15244 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 4 set_mon_vals no callback set 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:02.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a2ffd640 10 monclient: dump: 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:02.470 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient: shutdown 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 -- 172.21.15.190:0/173421289 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0a4069710 msgr2=0x7ff0a409ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 --2- 172.21.15.190:0/173421289 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0a4069710 0x7ff0a409ed60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff0940098a0 tx=0x7ff09401c440 comp rx=0 tx=0).stop 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 -- 172.21.15.190:0/173421289 shutdown_connections 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 --2- 172.21.15.190:0/173421289 >> v2:172.21.15.190:3300/0 conn(0x7ff0a412d510 0x7ff0a412d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 --2- 172.21.15.190:0/173421289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0a41375c0 0x7ff0a41379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 --2- 172.21.15.190:0/173421289 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff0a4069710 0x7ff0a409ed60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:02.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 -- 172.21.15.190:0/173421289 >> 172.21.15.190:0/173421289 conn(0x7ff0a40a8450 msgr2=0x7ff0a40a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 -- 172.21.15.190:0/173421289 shutdown_connections 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 -- 172.21.15.190:0/173421289 wait complete. 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient: build_initial_monmap 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient: monmap: 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:02.472 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 Processor -- start 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 -- start start 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient: init 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient: _add_conns ranks=[6,0,8,3,5,4,7,2,1] 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff0a4069710 0x7ff0a412eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient(hunting): picked mon.noname-d con 0x7ff0a4069710 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:02.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0a412d510 0x7ff0a412efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient(hunting): picked mon.noname-a con 0x7ff0a412d510 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0a41375c0 0x7ff0a412f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient(hunting): picked mon.noname-e con 0x7ff0a41375c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4140e80 con 0x7ff0a412d510 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff0a409c580 con 0x7ff0a4069710 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff0a4141000 con 0x7ff0a41375c0 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0aa458640 10 monclient(hunting): authenticate will time out at 1288.035167s 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a3fff640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff0a4069710 msgr2=0x7ff0a412eaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:02.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff0a4069710 0x7ff0a412eaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0a412d510 0x7ff0a412efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a89f5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0a41375c0 0x7ff0a412f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0a412d510 0x7ff0a412efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:34514/0 (socket says 172.21.15.190:34514) 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a37fe640 1 -- 172.21.15.190:0/418152973 learned_addr learned my addr 172.21.15.190:0/418152973 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a89f5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0a41375c0 0x7ff0a412f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:52566/0 (socket says 172.21.15.190:52566) 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a37fe640 10 monclient(hunting): get_auth_request con 0x7ff0a412d510 auth_method 0 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a89f5640 10 monclient(hunting): get_auth_request con 0x7ff0a41375c0 auth_method 0 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a89f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a89f5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a89f5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:02.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a89f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a89f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a89f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.462+0000 7ff0a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 10 monclient(hunting): handle_auth_done global_id 5840 payload 824 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 1 -- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff0a4069710 msgr2=0x7ff0a412eaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff0a4069710 0x7ff0a412eaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 1 -- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0a412d510 msgr2=0x7ff0a412efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0a412d510 0x7ff0a412efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 10 monclient: _finish_hunting 0 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 1 monclient: found mon.noname-e 2024-08-19T10:44:02.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:02.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:02.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 1 -- 172.21.15.190:0/418152973 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a409ee80 con 0x7ff0a41375c0 2024-08-19T10:44:02.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a37fe640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0a412d510 0x7ff0a412efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 1 -- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0a41375c0 msgr2=0x7ff0a412f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a89f5640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0a41375c0 0x7ff0a412f520 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff09c003d10 tx=0x7ff09c009ed0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 1 -- 172.21.15.190:0/418152973 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff09c00ce00 con 0x7ff0a41375c0 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 10 monclient: dump: 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:02.479 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:02.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 1 -- 172.21.15.190:0/418152973 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09c010030 con 0x7ff0a41375c0 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 1 -- 172.21.15.190:0/418152973 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff09c015850 con 0x7ff0a41375c0 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a0ff9640 10 monclient: dump: 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:02.481 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:02.482 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:02.482 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:02.482 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:02.482 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:02.482 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:02.482 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:02.482 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:02.482 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:02.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0aa458640 5 monclient: authenticate success, global_id 5840 2024-08-19T10:44:02.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0aa458640 10 monclient: _renew_subs 2024-08-19T10:44:02.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0a1ffb640 4 set_mon_vals no callback set 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0aa458640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0aa458640 1 -- 172.21.15.190:0/418152973 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a406d340 con 0x7ff0a41375c0 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0aa458640 10 monclient: _renew_subs 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0aa458640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0aa458640 1 -- 172.21.15.190:0/418152973 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a4071e60 con 0x7ff0a41375c0 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0aa458640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0aa458640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0aa458640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.466+0000 7ff0aa458640 1 -- 172.21.15.190:0/418152973 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a412d990 con 0x7ff0a41375c0 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.470+0000 7ff0a0ff9640 1 -- 172.21.15.190:0/418152973 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff09c0153e0 con 0x7ff0a41375c0 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.470+0000 7ff0a0ff9640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff09003be00 0x7ff09003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.470+0000 7ff0a0ff9640 1 -- 172.21.15.190:0/418152973 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(193..193 src has 1..193) ==== 7811+0+0 (secure 0 0 0) 0x7ff09c04fde0 con 0x7ff0a41375c0 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.470+0000 7ff0a3fff640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff09003be00 0x7ff09003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:02.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.470+0000 7ff0a3fff640 10 monclient: get_auth_request con 0x7ff09003be00 auth_method 0 2024-08-19T10:44:02.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.470+0000 7ff0a3fff640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff09003be00 0x7ff09003e2c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff0940098a0 tx=0x7ff094008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:02.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.482+0000 7ff0a0ff9640 1 -- 172.21.15.190:0/418152973 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff0a409ee80 con 0x7ff0a41375c0 2024-08-19T10:44:02.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.482+0000 7ff0a0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:02.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.482+0000 7ff0a0ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:02.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.654+0000 7ff0aa458640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:02.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.654+0000 7ff0aa458640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:02.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.654+0000 7ff0aa458640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:02.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.654+0000 7ff0aa458640 1 -- 172.21.15.190:0/418152973 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7ff0a406e470 con 0x7ff0a41375c0 2024-08-19T10:44:02.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.918+0000 7ff0a0ff9640 1 -- 172.21.15.190:0/418152973 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}]=0 added bucket h1 type host to location {rack=r1} v194) ==== 166+0+0 (secure 0 0 0) 0x7ff09c04cc50 con 0x7ff0a41375c0 2024-08-19T10:44:02.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.918+0000 7ff0a0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:02.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:02.918+0000 7ff0a0ff9640 10 monclient: _finish_command 2 = system:0 added bucket h1 type host to location {rack=r1} 2024-08-19T10:44:03.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.026+0000 7ff0aa458640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:03.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.026+0000 7ff0aa458640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:03.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.026+0000 7ff0aa458640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:03.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.026+0000 7ff0aa458640 1 -- 172.21.15.190:0/418152973 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7ff0a4130300 con 0x7ff0a41375c0 2024-08-19T10:44:03.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.034+0000 7ff0a0ff9640 1 -- 172.21.15.190:0/418152973 <== mon.8 v2:172.21.15.89:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h1' already exists v194) ==== 145+0+0 (secure 0 0 0) 0x7ff09c024020 con 0x7ff0a41375c0 2024-08-19T10:44:03.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.034+0000 7ff0a0ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h1", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:03.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.034+0000 7ff0a0ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h1' already exists 2024-08-19T10:44:03.037 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h1' already exists 2024-08-19T10:44:03.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 -- 172.21.15.190:0/418152973 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff09003be00 msgr2=0x7ff09003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:03.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff09003be00 0x7ff09003e2c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff0940098a0 tx=0x7ff094008040 comp rx=0 tx=0).stop 2024-08-19T10:44:03.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 10 monclient: shutdown 2024-08-19T10:44:03.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:03.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 -- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0a41375c0 msgr2=0x7ff0a412f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:03.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0a41375c0 0x7ff0a412f520 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff09c003d10 tx=0x7ff09c009ed0 comp rx=0 tx=0).stop 2024-08-19T10:44:03.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 -- 172.21.15.190:0/418152973 shutdown_connections 2024-08-19T10:44:03.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff09003be00 0x7ff09003e2c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:03.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0a41375c0 0x7ff0a412f520 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:03.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0a412d510 0x7ff0a412efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:03.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 --2- 172.21.15.190:0/418152973 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff0a4069710 0x7ff0a412eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:03.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 -- 172.21.15.190:0/418152973 >> 172.21.15.190:0/418152973 conn(0x7ff0a40a8450 msgr2=0x7ff0a4135920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:03.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 -- 172.21.15.190:0/418152973 shutdown_connections 2024-08-19T10:44:03.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.038+0000 7ff0aa458640 1 -- 172.21.15.190:0/418152973 wait complete. 2024-08-19T10:44:03.055 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:03.056 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h2 host rack=r1 2024-08-19T10:44:03.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.174+0000 7fb01ad32640 10 monclient: get_monmap_and_config 2024-08-19T10:44:03.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.174+0000 7fb01ad32640 10 monclient: build_initial_monmap 2024-08-19T10:44:03.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.174+0000 7fb01ad32640 10 monclient: monmap: 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:03.177486+0000 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:03.177486+0000 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:03.177 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:03.178 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:03.178 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:03.178 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:03.178 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:03.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.174+0000 7fb01ad32640 1 Processor -- start 2024-08-19T10:44:03.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.174+0000 7fb01ad32640 1 -- start start 2024-08-19T10:44:03.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.174+0000 7fb01ad32640 10 monclient: init 2024-08-19T10:44:03.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:03.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 10 monclient: _add_conns ranks=[10,1,9,8,3,4,2,0,7,6,5] 2024-08-19T10:44:03.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb0140a57d0 0x7fb0140a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:03.180 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 10 monclient(hunting): picked mon.noname-f con 0x7fb0140a57d0 addr v2:172.21.15.190:3300/0 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb014072360 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb0140a60f0 0x7fb014137c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 10 monclient(hunting): picked mon.noname-a con 0x7fb0140a60f0 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb014074120 con 0x7fb014072360 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fb014073bf0 con 0x7fb0140a60f0 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb014073e40 con 0x7fb0140a57d0 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb01ad32640 10 monclient(hunting): authenticate will time out at 1288.748163s 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb013fff640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fb014072360 legacy=0x7fb014137580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb018ace640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7fb0140a57d0 msgr2=0x7fb0140a5bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb018ace640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb0140a57d0 0x7fb0140a5bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:03.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb0140a60f0 0x7fb014137c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:03.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fb0140a60f0 0x7fb014137c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:34526/0 (socket says 172.21.15.190:34526) 2024-08-19T10:44:03.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 1 -- 172.21.15.190:0/3179922639 learned_addr learned my addr 172.21.15.190:0/3179922639 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:03.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 10 monclient(hunting): get_auth_request con 0x7fb0140a60f0 auth_method 0 2024-08-19T10:44:03.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:03.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 10 monclient(hunting): handle_auth_done global_id 5994 payload 274 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 1 -- 172.21.15.190:0/3179922639 >> v2:172.21.15.190:3300/0 conn(0x7fb0140a57d0 msgr2=0x7fb0140a5bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 1 --2- 172.21.15.190:0/3179922639 >> v2:172.21.15.190:3300/0 conn(0x7fb0140a57d0 0x7fb0140a5bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 1 -- 172.21.15.190:0/3179922639 >> v1:172.21.15.190:6789/0 conn(0x7fb014072360 legacy=0x7fb014137580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 10 monclient: _finish_hunting 0 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 1 monclient: found mon.noname-a 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:03.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.178+0000 7fb0192cf640 1 -- 172.21.15.190:0/3179922639 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0140745b0 con 0x7fb0140a60f0 2024-08-19T10:44:03.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0192cf640 1 -- 172.21.15.190:0/3179922639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0140a60f0 msgr2=0x7fb014137c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:03.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0192cf640 1 --2- 172.21.15.190:0/3179922639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0140a60f0 0x7fb014137c90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb014072da0 tx=0x7fb00c01d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b9cbcd52ff7626 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:03.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 1 -- 172.21.15.190:0/3179922639 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb00c0091b0 con 0x7fb0140a60f0 2024-08-19T10:44:03.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:03.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:03.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 10 monclient: dump: 2024-08-19T10:44:03.186 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:03.187 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:03.187 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:03.187 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:03.187 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:03.187 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:03.187 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:03.187 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:03.187 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:03.187 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:03.187 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:03.187 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:03.188 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:03.188 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:03.188 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:03.188 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 10 monclient: _finish_auth 0 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 10 monclient: _check_auth_tickets 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 1 -- 172.21.15.190:0/3179922639 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb00c0a5ad0 con 0x7fb0140a60f0 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 1 -- 172.21.15.190:0/3179922639 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb00c0a5c80 con 0x7fb0140a60f0 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb0137fe640 10 monclient: dump: 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:03.189 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:03.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 5 monclient: authenticate success, global_id 5994 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 4 set_mon_vals no callback set 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 10 monclient: shutdown 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 1 -- 172.21.15.190:0/3179922639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0140a60f0 msgr2=0x7fb014137c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 1 --2- 172.21.15.190:0/3179922639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0140a60f0 0x7fb014137c90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb014072da0 tx=0x7fb00c01d330 comp rx=0 tx=0).stop 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 1 -- 172.21.15.190:0/3179922639 shutdown_connections 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 1 --2- 172.21.15.190:0/3179922639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0140a60f0 0x7fb014137c90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 1 --2- 172.21.15.190:0/3179922639 >> v2:172.21.15.190:3300/0 conn(0x7fb0140a57d0 0x7fb0140a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 1 -- 172.21.15.190:0/3179922639 >> 172.21.15.190:0/3179922639 conn(0x7fb0140aa3c0 msgr2=0x7fb0140a68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 1 -- 172.21.15.190:0/3179922639 shutdown_connections 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 1 -- 172.21.15.190:0/3179922639 wait complete. 2024-08-19T10:44:03.191 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 10 monclient: build_initial_monmap 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.182+0000 7fb01ad32640 10 monclient: monmap: 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:03.192 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:03.193 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:03.193 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:03.193 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 1 Processor -- start 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 1 -- start start 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient: init 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient: _add_conns ranks=[8,0,3,2,4,6,7,5,1] 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0140a57d0 0x7fb014136710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient(hunting): picked mon.noname-e con 0x7fb0140a57d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0140a60f0 0x7fb014136c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient(hunting): picked mon.noname-a con 0x7fb0140a60f0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb01413bdd0 0x7fb01413c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient(hunting): picked mon.noname-g con 0x7fb01413bdd0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:03.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb0140a1970 con 0x7fb0140a60f0 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb0140a17f0 con 0x7fb01413bdd0 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb0141381d0 con 0x7fb0140a57d0 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient(hunting): authenticate will time out at 1288.756818s 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb01413bdd0 0x7fb01413c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb01413bdd0 0x7fb01413c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:33798/0 (socket says 172.21.15.190:33798) 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 1 -- 172.21.15.190:0/1949137938 learned_addr learned my addr 172.21.15.190:0/1949137938 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 10 monclient(hunting): get_auth_request con 0x7fb01413bdd0 auth_method 0 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:03.199 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb018ace640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0140a57d0 0x7fb014136710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb013fff640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0140a60f0 0x7fb014136c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb018ace640 10 monclient(hunting): get_auth_request con 0x7fb0140a57d0 auth_method 0 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb018ace640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb018ace640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb018ace640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb013fff640 10 monclient(hunting): get_auth_request con 0x7fb0140a60f0 auth_method 0 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb013fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb013fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:03.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb013fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 10 monclient(hunting): handle_auth_done global_id 34320 payload 824 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 1 -- 172.21.15.190:0/1949137938 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0140a57d0 msgr2=0x7fb014136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0140a57d0 0x7fb014136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 1 -- 172.21.15.190:0/1949137938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0140a60f0 msgr2=0x7fb014136c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0140a60f0 0x7fb014136c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 10 monclient: _finish_hunting 0 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 1 monclient: found mon.noname-g 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 1 -- 172.21.15.190:0/1949137938 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01413d8c0 con 0x7fb01413bdd0 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 1 -- 172.21.15.190:0/1949137938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb01413bdd0 msgr2=0x7fb01413c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0192cf640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb01413bdd0 0x7fb01413c210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb00c01d170 tx=0x7fb00c019670 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:03.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 1 -- 172.21.15.190:0/1949137938 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb00c0bc020 con 0x7fb01413bdd0 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 10 monclient: dump: 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:03.202 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:03.203 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:03.203 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:03.203 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:03.203 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:03.203 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:03.203 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:03.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 10 monclient: _finish_auth 0 2024-08-19T10:44:03.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 5 monclient: authenticate success, global_id 34320 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 1 -- 172.21.15.190:0/1949137938 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb00c003bb0 con 0x7fb01413bdd0 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient: _renew_subs 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 1 -- 172.21.15.190:0/1949137938 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb00c003d60 con 0x7fb01413bdd0 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 1 -- 172.21.15.190:0/1949137938 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb014120f70 con 0x7fb01413bdd0 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb0117fa640 10 monclient: dump: 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:03.204 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:03.205 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb012ffd640 4 set_mon_vals no callback set 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient: _renew_subs 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.186+0000 7fb01ad32640 1 -- 172.21.15.190:0/1949137938 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01413e420 con 0x7fb01413bdd0 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.190+0000 7fb01ad32640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.190+0000 7fb01ad32640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.190+0000 7fb01ad32640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.190+0000 7fb01ad32640 1 -- 172.21.15.190:0/1949137938 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0140a5bb0 con 0x7fb01413bdd0 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.190+0000 7fb0117fa640 1 -- 172.21.15.190:0/1949137938 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb00c0bf020 con 0x7fb01413bdd0 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.190+0000 7fb0117fa640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb00003be00 0x7fb00003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.190+0000 7fb0117fa640 1 -- 172.21.15.190:0/1949137938 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(194..194 src has 1..194) ==== 8021+0+0 (secure 0 0 0) 0x7fb00c0ea860 con 0x7fb01413bdd0 2024-08-19T10:44:03.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.190+0000 7fb018ace640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb00003be00 0x7fb00003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:03.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.190+0000 7fb018ace640 10 monclient: get_auth_request con 0x7fb00003be00 auth_method 0 2024-08-19T10:44:03.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.190+0000 7fb018ace640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb00003be00 0x7fb00003e2c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb004006fd0 tx=0x7fb004008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:03.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.194+0000 7fb0117fa640 1 -- 172.21.15.190:0/1949137938 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb00c0c6030 con 0x7fb01413bdd0 2024-08-19T10:44:03.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.194+0000 7fb0117fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:03.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.194+0000 7fb0117fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:03.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.370+0000 7fb01ad32640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:03.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.370+0000 7fb01ad32640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:03.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.370+0000 7fb01ad32640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:03.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.370+0000 7fb01ad32640 1 -- 172.21.15.190:0/1949137938 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fb0141211b0 con 0x7fb01413bdd0 2024-08-19T10:44:03.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.946+0000 7fb0117fa640 1 -- 172.21.15.190:0/1949137938 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}]=0 added bucket h2 type host to location {rack=r1} v195) ==== 166+0+0 (secure 0 0 0) 0x7fb00c0165e0 con 0x7fb01413bdd0 2024-08-19T10:44:03.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.946+0000 7fb0117fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:03.948 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:03.946+0000 7fb0117fa640 10 monclient: _finish_command 2 = system:0 added bucket h2 type host to location {rack=r1} 2024-08-19T10:44:04.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.054+0000 7fb01ad32640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:04.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.054+0000 7fb01ad32640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:04.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.054+0000 7fb01ad32640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:04.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.054+0000 7fb01ad32640 1 -- 172.21.15.190:0/1949137938 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fb0140008e0 con 0x7fb01413bdd0 2024-08-19T10:44:04.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb0117fa640 1 -- 172.21.15.190:0/1949137938 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h2' already exists v195) ==== 145+0+0 (secure 0 0 0) 0x7fb00c0042d0 con 0x7fb01413bdd0 2024-08-19T10:44:04.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb0117fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h2", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:04.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb0117fa640 10 monclient: _finish_command 3 = system:0 bucket 'h2' already exists 2024-08-19T10:44:04.058 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h2' already exists 2024-08-19T10:44:04.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 -- 172.21.15.190:0/1949137938 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb00003be00 msgr2=0x7fb00003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:04.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb00003be00 0x7fb00003e2c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb004006fd0 tx=0x7fb004008040 comp rx=0 tx=0).stop 2024-08-19T10:44:04.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 10 monclient: shutdown 2024-08-19T10:44:04.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:04.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 -- 172.21.15.190:0/1949137938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb01413bdd0 msgr2=0x7fb01413c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:04.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb01413bdd0 0x7fb01413c210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb00c01d170 tx=0x7fb00c019670 comp rx=0 tx=0).stop 2024-08-19T10:44:04.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 -- 172.21.15.190:0/1949137938 shutdown_connections 2024-08-19T10:44:04.061 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb01413bdd0 0x7fb01413c210 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:04.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0140a60f0 0x7fb014136c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:04.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb00003be00 0x7fb00003e2c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:04.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 --2- 172.21.15.190:0/1949137938 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0140a57d0 0x7fb014136710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:04.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 -- 172.21.15.190:0/1949137938 >> 172.21.15.190:0/1949137938 conn(0x7fb0140aa3c0 msgr2=0x7fb01413ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:04.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 -- 172.21.15.190:0/1949137938 shutdown_connections 2024-08-19T10:44:04.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.058+0000 7fb01ad32640 1 -- 172.21.15.190:0/1949137938 wait complete. 2024-08-19T10:44:04.076 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:04.077 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h3 host rack=r1 2024-08-19T10:44:04.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient: get_monmap_and_config 2024-08-19T10:44:04.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient: build_initial_monmap 2024-08-19T10:44:04.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient: monmap: 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:04.199695+0000 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:04.199695+0000 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:04.199 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:04.200 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:04.200 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:04.200 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:04.200 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:04.200 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:04.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 1 Processor -- start 2024-08-19T10:44:04.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 1 -- start start 2024-08-19T10:44:04.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient: init 2024-08-19T10:44:04.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:04.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient: _add_conns ranks=[3,0,1,4,5,9,10,7,6,2,8] 2024-08-19T10:44:04.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2780a4720 0x7fb2780a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient(hunting): picked mon.noname-g con 0x7fb2780a4720 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb2780a5040 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb278139fe0 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2780a1d80 con 0x7fb2780a5040 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2780a13a0 con 0x7fb278139fe0 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb2780a0e70 con 0x7fb2780a4720 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ef22640 10 monclient(hunting): authenticate will time out at 1289.769638s 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27d4bf640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fb278139fe0 legacy=0x7fb278141b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2780a4720 0x7fb2780a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2780a4720 0x7fb2780a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:33802/0 (socket says 172.21.15.190:33802) 2024-08-19T10:44:04.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 1 -- 172.21.15.190:0/2379592506 learned_addr learned my addr 172.21.15.190:0/2379592506 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 10 monclient(hunting): get_auth_request con 0x7fb2780a4720 auth_method 0 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 10 monclient(hunting): handle_auth_done global_id 34329 payload 274 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 1 -- 172.21.15.190:0/2379592506 >> v1:172.21.15.190:6789/0 conn(0x7fb278139fe0 legacy=0x7fb278141b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 1 -- 172.21.15.190:0/2379592506 >> v1:172.21.15.89:6789/0 conn(0x7fb2780a5040 legacy=0x7fb2781398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:04.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 1 monclient: found mon.noname-g 2024-08-19T10:44:04.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:04.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:04.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.198+0000 7fb27ccbe640 1 -- 172.21.15.190:0/2379592506 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27806d000 con 0x7fb2780a4720 2024-08-19T10:44:04.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ccbe640 1 -- 172.21.15.190:0/2379592506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2780a4720 msgr2=0x7fb2780a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:04.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ccbe640 1 --2- 172.21.15.190:0/2379592506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2780a4720 0x7fb2780a4b00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb270009e80 tx=0x7fb27001c730 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3bb2f24453b3d5d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:04.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 1 -- 172.21.15.190:0/2379592506 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2700ac020 con 0x7fb2780a4720 2024-08-19T10:44:04.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:04.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:04.207 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 10 monclient: dump: 2024-08-19T10:44:04.207 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:04.208 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 10 monclient: _finish_auth 0 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 10 monclient: _check_auth_tickets 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 1 -- 172.21.15.190:0/2379592506 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2700091f0 con 0x7fb2780a4720 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 5 monclient: authenticate success, global_id 34329 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 1 -- 172.21.15.190:0/2379592506 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2700a6d10 con 0x7fb2780a4720 2024-08-19T10:44:04.209 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 4 set_mon_vals no callback set 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb2777fe640 10 monclient: dump: 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:04.210 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 10 monclient: shutdown 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 1 -- 172.21.15.190:0/2379592506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2780a4720 msgr2=0x7fb2780a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 1 --2- 172.21.15.190:0/2379592506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2780a4720 0x7fb2780a4b00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb270009e80 tx=0x7fb27001c730 comp rx=0 tx=0).stop 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 1 -- 172.21.15.190:0/2379592506 shutdown_connections 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 1 --2- 172.21.15.190:0/2379592506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2780a4720 0x7fb2780a4b00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:04.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 1 -- 172.21.15.190:0/2379592506 >> 172.21.15.190:0/2379592506 conn(0x7fb2780a9310 msgr2=0x7fb2780a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 1 -- 172.21.15.190:0/2379592506 shutdown_connections 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.202+0000 7fb27ef22640 1 -- 172.21.15.190:0/2379592506 wait complete. 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient: build_initial_monmap 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient: monmap: 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:04.212 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:04.213 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:04.213 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:04.213 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:04.213 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:04.213 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:04.213 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:04.213 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:04.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 1 Processor -- start 2024-08-19T10:44:04.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 1 -- start start 2024-08-19T10:44:04.213 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient: init 2024-08-19T10:44:04.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:04.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient: _add_conns ranks=[5,7,0,4,6,8,1,3,2] 2024-08-19T10:44:04.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2780a4720 0x7fb27813d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:04.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient(hunting): picked mon.noname-h con 0x7fb2780a4720 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:04.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb2780a5040 0x7fb27813def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:04.218 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient(hunting): picked mon.noname-i con 0x7fb2780a5040 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb278139fe0 0x7fb27813e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient(hunting): picked mon.noname-a con 0x7fb278139fe0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb27806d4c0 con 0x7fb278139fe0 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb2780a6240 con 0x7fb2780a4720 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb2780a5420 con 0x7fb2780a5040 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ef22640 10 monclient(hunting): authenticate will time out at 1289.777641s 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2780a4720 0x7fb27813d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb277fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb2780a5040 0x7fb27813def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:04.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2780a4720 0x7fb27813d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:40796/0 (socket says 172.21.15.190:40796) 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 1 -- 172.21.15.190:0/261461885 learned_addr learned my addr 172.21.15.190:0/261461885 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 10 monclient(hunting): get_auth_request con 0x7fb2780a4720 auth_method 0 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb277fff640 10 monclient(hunting): get_auth_request con 0x7fb2780a5040 auth_method 0 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb277fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb277fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb277fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:04.220 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb277fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb277fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb277fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27d4bf640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb278139fe0 0x7fb27813e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27d4bf640 10 monclient(hunting): get_auth_request con 0x7fb278139fe0 auth_method 0 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27d4bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27d4bf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27d4bf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 10 monclient(hunting): handle_auth_done global_id 5972 payload 824 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 1 -- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb2780a5040 msgr2=0x7fb27813def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb2780a5040 0x7fb27813def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 1 -- 172.21.15.190:0/261461885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb278139fe0 msgr2=0x7fb27813e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:04.221 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb278139fe0 0x7fb27813e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:04.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:04.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 1 monclient: found mon.noname-h 2024-08-19T10:44:04.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:04.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:04.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27ccbe640 1 -- 172.21.15.190:0/261461885 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27813f3e0 con 0x7fb2780a4720 2024-08-19T10:44:04.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb277fff640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb2780a5040 0x7fb27813def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:04.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.206+0000 7fb27d4bf640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb278139fe0 0x7fb27813e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:44:04.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ccbe640 1 -- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2780a4720 msgr2=0x7fb27813d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:04.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ccbe640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2780a4720 0x7fb27813d9b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb270009e20 tx=0x7fb2700194d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:04.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 1 -- 172.21.15.190:0/261461885 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2700ac040 con 0x7fb2780a4720 2024-08-19T10:44:04.222 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 10 monclient: dump: 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:04.223 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 10 monclient: _finish_auth 0 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 1 -- 172.21.15.190:0/261461885 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2700b6030 con 0x7fb2780a4720 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ef22640 5 monclient: authenticate success, global_id 5972 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ef22640 10 monclient: _renew_subs 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ef22640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:04.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ef22640 1 -- 172.21.15.190:0/261461885 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb278141740 con 0x7fb2780a4720 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 1 -- 172.21.15.190:0/261461885 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2700b18c0 con 0x7fb2780a4720 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb276ffd640 4 set_mon_vals no callback set 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 10 monclient: dump: 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:04.225 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ef22640 10 monclient: _renew_subs 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ef22640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ef22640 1 -- 172.21.15.190:0/261461885 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27813ff40 con 0x7fb2780a4720 2024-08-19T10:44:04.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ef22640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ef22640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ef22640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb27ef22640 1 -- 172.21.15.190:0/261461885 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2780a4b00 con 0x7fb2780a4720 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 1 -- 172.21.15.190:0/261461885 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb2700af020 con 0x7fb2780a4720 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb26403be00 0x7fb26403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb2757fa640 1 -- 172.21.15.190:0/261461885 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(195..195 src has 1..195) ==== 8231+0+0 (secure 0 0 0) 0x7fb2700e99a0 con 0x7fb2780a4720 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb277fff640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb26403be00 0x7fb26403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.210+0000 7fb277fff640 10 monclient: get_auth_request con 0x7fb26403be00 auth_method 0 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.214+0000 7fb277fff640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb26403be00 0x7fb26403e2c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb278142c40 tx=0x7fb26c007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.214+0000 7fb2757fa640 1 -- 172.21.15.190:0/261461885 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb2700c0070 con 0x7fb2780a4720 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.214+0000 7fb2757fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:04.227 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.214+0000 7fb2757fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:04.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.390+0000 7fb27ef22640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:04.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.390+0000 7fb27ef22640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:04.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.390+0000 7fb27ef22640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:04.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.390+0000 7fb27ef22640 1 -- 172.21.15.190:0/261461885 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fb27806ed90 con 0x7fb2780a4720 2024-08-19T10:44:04.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.958+0000 7fb2757fa640 1 -- 172.21.15.190:0/261461885 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}]=0 added bucket h3 type host to location {rack=r1} v196) ==== 166+0+0 (secure 0 0 0) 0x7fb2780a1d80 con 0x7fb2780a4720 2024-08-19T10:44:04.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.958+0000 7fb2757fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:04.962 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:04.958+0000 7fb2757fa640 10 monclient: _finish_command 2 = system:0 added bucket h3 type host to location {rack=r1} 2024-08-19T10:44:05.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.066+0000 7fb27ef22640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:05.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.066+0000 7fb27ef22640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:05.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.066+0000 7fb27ef22640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:05.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.066+0000 7fb27ef22640 1 -- 172.21.15.190:0/261461885 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fb278140ae0 con 0x7fb2780a4720 2024-08-19T10:44:05.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.070+0000 7fb2757fa640 1 -- 172.21.15.190:0/261461885 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h3' already exists v196) ==== 145+0+0 (secure 0 0 0) 0x7fb2700d9020 con 0x7fb2780a4720 2024-08-19T10:44:05.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.070+0000 7fb2757fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h3", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:05.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.070+0000 7fb2757fa640 10 monclient: _finish_command 3 = system:0 bucket 'h3' already exists 2024-08-19T10:44:05.074 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h3' already exists 2024-08-19T10:44:05.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 -- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb26403be00 msgr2=0x7fb26403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb26403be00 0x7fb26403e2c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb278142c40 tx=0x7fb26c007e40 comp rx=0 tx=0).stop 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 10 monclient: shutdown 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 -- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2780a4720 msgr2=0x7fb27813d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2780a4720 0x7fb27813d9b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb270009e20 tx=0x7fb2700194d0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 -- 172.21.15.190:0/261461885 shutdown_connections 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb26403be00 0x7fb26403e2c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb278139fe0 0x7fb27813e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb2780a5040 0x7fb27813def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 --2- 172.21.15.190:0/261461885 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2780a4720 0x7fb27813d9b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 -- 172.21.15.190:0/261461885 >> 172.21.15.190:0/261461885 conn(0x7fb2780a9310 msgr2=0x7fb278070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:05.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 -- 172.21.15.190:0/261461885 shutdown_connections 2024-08-19T10:44:05.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.074+0000 7fb27ef22640 1 -- 172.21.15.190:0/261461885 wait complete. 2024-08-19T10:44:05.090 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:05.516 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h4 host rack=r1 2024-08-19T10:44:05.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient: get_monmap_and_config 2024-08-19T10:44:05.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient: build_initial_monmap 2024-08-19T10:44:05.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient: monmap: 2024-08-19T10:44:05.526 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:05.526 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:05.526 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:05.204173+0000 2024-08-19T10:44:05.526 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:05.204173+0000 2024-08-19T10:44:05.526 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:05.526 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:05.526 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:05.526 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:05.527 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:05.527 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:05.527 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:05.527 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:05.527 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:05.527 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:05.527 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:05.527 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:05.527 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:05.528 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:05.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 1 Processor -- start 2024-08-19T10:44:05.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 1 -- start start 2024-08-19T10:44:05.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient: init 2024-08-19T10:44:05.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:05.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient: _add_conns ranks=[6,4,9,0,2,8,1,3,10,7,5] 2024-08-19T10:44:05.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f243c1396a0 0x7f243c139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:05.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient(hunting): picked mon.noname-d con 0x7f243c1396a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:05.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 0x7f243c0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:05.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient(hunting): picked mon.noname-c con 0x7f243c139fc0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:05.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f243c0a6e20 0x7f243c0a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:05.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient(hunting): picked mon.noname-a con 0x7f243c0a6e20 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:05.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:05.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f243c0a4f20 con 0x7f243c0a6e20 2024-08-19T10:44:05.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:05.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f243c0a83e0 con 0x7f243c139fc0 2024-08-19T10:44:05.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:05.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f243c0a2f30 con 0x7f243c1396a0 2024-08-19T10:44:05.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:05.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f2443b22640 10 monclient(hunting): authenticate will time out at 1290.774005s 2024-08-19T10:44:05.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24410bd640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 msgr2=0x7f243c0a68e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:05.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24418be640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f243c1396a0 msgr2=0x7f243c139a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:05.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24418be640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f243c1396a0 0x7f243c139a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:05.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24410bd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 0x7f243c0a68e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:05.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f243c0a6e20 0x7f243c0a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:05.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f243c0a6e20 0x7f243c0a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:34552/0 (socket says 172.21.15.190:34552) 2024-08-19T10:44:05.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 1 -- 172.21.15.190:0/2516428336 learned_addr learned my addr 172.21.15.190:0/2516428336 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:05.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 10 monclient(hunting): get_auth_request con 0x7f243c0a6e20 auth_method 0 2024-08-19T10:44:05.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:05.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:05.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:05.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:05.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:05.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:05.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 10 monclient(hunting): handle_auth_done global_id 6021 payload 274 2024-08-19T10:44:05.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 1 -- 172.21.15.190:0/2516428336 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f243c1396a0 msgr2=0x7f243c139a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:05.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 1 --2- 172.21.15.190:0/2516428336 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f243c1396a0 0x7f243c139a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 1 -- 172.21.15.190:0/2516428336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 msgr2=0x7f243c0a68e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:05.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 1 --2- 172.21.15.190:0/2516428336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 0x7f243c0a68e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 10 monclient: _finish_hunting 0 2024-08-19T10:44:05.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 1 monclient: found mon.noname-a 2024-08-19T10:44:05.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:05.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:05.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.202+0000 7f24420bf640 1 -- 172.21.15.190:0/2516428336 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c0a35e0 con 0x7f243c0a6e20 2024-08-19T10:44:05.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.206+0000 7f24420bf640 1 -- 172.21.15.190:0/2516428336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f243c0a6e20 msgr2=0x7f243c0a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:05.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.206+0000 7f24420bf640 1 --2- 172.21.15.190:0/2516428336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f243c0a6e20 0x7f243c0a7220 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2434009e50 tx=0x7f243401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9680a2225428eb65 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:05.532 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 1 -- 172.21.15.190:0/2516428336 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24340091b0 con 0x7f243c0a6e20 2024-08-19T10:44:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 10 monclient: dump: 2024-08-19T10:44:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:05.533 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:05.534 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:05.534 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 10 monclient: _finish_auth 0 2024-08-19T10:44:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 1 -- 172.21.15.190:0/2516428336 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24340a59e0 con 0x7f243c0a6e20 2024-08-19T10:44:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 1 -- 172.21.15.190:0/2516428336 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24340ad3f0 con 0x7f243c0a6e20 2024-08-19T10:44:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:05.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24408bc640 10 monclient: dump: 2024-08-19T10:44:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:05.536 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:05.537 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:05.537 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:05.537 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:05.537 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:05.537 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:05.537 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:05.537 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:05.537 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:05.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 5 monclient: authenticate success, global_id 6021 2024-08-19T10:44:05.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:05.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 4 set_mon_vals no callback set 2024-08-19T10:44:05.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient: shutdown 2024-08-19T10:44:05.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:05.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 -- 172.21.15.190:0/2516428336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f243c0a6e20 msgr2=0x7f243c0a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:05.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 --2- 172.21.15.190:0/2516428336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f243c0a6e20 0x7f243c0a7220 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2434009e50 tx=0x7f243401c930 comp rx=0 tx=0).stop 2024-08-19T10:44:05.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 -- 172.21.15.190:0/2516428336 shutdown_connections 2024-08-19T10:44:05.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 --2- 172.21.15.190:0/2516428336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f243c0a6e20 0x7f243c0a7220 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 --2- 172.21.15.190:0/2516428336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 0x7f243c0a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 --2- 172.21.15.190:0/2516428336 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f243c1396a0 0x7f243c139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 -- 172.21.15.190:0/2516428336 >> 172.21.15.190:0/2516428336 conn(0x7f243c0abee0 msgr2=0x7f243c0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:05.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 -- 172.21.15.190:0/2516428336 shutdown_connections 2024-08-19T10:44:05.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 -- 172.21.15.190:0/2516428336 wait complete. 2024-08-19T10:44:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient: build_initial_monmap 2024-08-19T10:44:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient: monmap: 2024-08-19T10:44:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:05.540 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:05.541 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:05.542 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 Processor -- start 2024-08-19T10:44:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 -- start start 2024-08-19T10:44:05.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient: init 2024-08-19T10:44:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient: _add_conns ranks=[5,7,4,2,8,6,3,1,0] 2024-08-19T10:44:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f243c0a6e20 0x7f243c134a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient(hunting): picked mon.noname-h con 0x7f243c0a6e20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f243c1396a0 0x7f243c134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient(hunting): picked mon.noname-i con 0x7f243c1396a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 0x7f243c09ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient(hunting): picked mon.noname-c con 0x7f243c139fc0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:05.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f243c13a3a0 con 0x7f243c139fc0 2024-08-19T10:44:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f243c143390 con 0x7f243c0a6e20 2024-08-19T10:44:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f243c140600 con 0x7f243c1396a0 2024-08-19T10:44:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f2443b22640 10 monclient(hunting): authenticate will time out at 1290.782412s 2024-08-19T10:44:05.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24420bf640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 msgr2=0x7f243c09ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f243c0a6e20 0x7f243c134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24420bf640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 0x7f243c09ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24410bd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f243c1396a0 0x7f243c134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f243c0a6e20 0x7f243c134a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:40804/0 (socket says 172.21.15.190:40804) 2024-08-19T10:44:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 1 -- 172.21.15.190:0/2202280618 learned_addr learned my addr 172.21.15.190:0/2202280618 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 10 monclient(hunting): get_auth_request con 0x7f243c0a6e20 auth_method 0 2024-08-19T10:44:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:05.545 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24410bd640 10 monclient(hunting): get_auth_request con 0x7f243c1396a0 auth_method 0 2024-08-19T10:44:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24410bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24410bd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24410bd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24410bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24410bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24410bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:05.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 10 monclient(hunting): handle_auth_done global_id 5981 payload 824 2024-08-19T10:44:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 1 -- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f243c1396a0 msgr2=0x7f243c134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f243c1396a0 0x7f243c134fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 1 -- 172.21.15.190:0/2202280618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 msgr2=0x7f243c09ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 0x7f243c09ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 10 monclient: _finish_hunting 0 2024-08-19T10:44:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 1 monclient: found mon.noname-h 2024-08-19T10:44:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:05.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24418be640 1 -- 172.21.15.190:0/2202280618 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c09e350 con 0x7f243c0a6e20 2024-08-19T10:44:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.210+0000 7f24410bd640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f243c1396a0 0x7f243c134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24418be640 1 -- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f243c0a6e20 msgr2=0x7f243c134a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24418be640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f243c0a6e20 0x7f243c134a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f243800b550 tx=0x7f243800ba10 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24418be640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f243c0a6e20 0x7f243c134a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f243800b550 tx=0x7f243800ba10 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f24380092d0 mon_map magic: 0 2024-08-19T10:44:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 1 -- 172.21.15.190:0/2202280618 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24380092d0 con 0x7f243c0a6e20 2024-08-19T10:44:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 10 monclient: dump: 2024-08-19T10:44:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:05.548 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:05.549 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:05.549 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:05.549 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:05.549 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:05.549 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:05.549 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:05.549 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:05.549 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:05.549 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:05.550 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:05.550 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:05.550 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:05.550 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:05.550 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:05.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:05.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:05.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:05.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 1 -- 172.21.15.190:0/2202280618 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2438015070 con 0x7f243c0a6e20 2024-08-19T10:44:05.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2443b22640 5 monclient: authenticate success, global_id 5981 2024-08-19T10:44:05.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:05.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2433fff640 4 set_mon_vals no callback set 2024-08-19T10:44:05.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2443b22640 10 monclient: _renew_subs 2024-08-19T10:44:05.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2443b22640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:05.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2443b22640 1 -- 172.21.15.190:0/2202280618 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243c0a7270 con 0x7f243c0a6e20 2024-08-19T10:44:05.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2443b22640 10 monclient: _renew_subs 2024-08-19T10:44:05.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2443b22640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:05.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2443b22640 1 -- 172.21.15.190:0/2202280618 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f243c0a10c0 con 0x7f243c0a6e20 2024-08-19T10:44:05.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2443b22640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:05.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2443b22640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:05.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2443b22640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:05.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f2443b22640 1 -- 172.21.15.190:0/2202280618 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243c139a80 con 0x7f243c0a6e20 2024-08-19T10:44:05.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 1 -- 172.21.15.190:0/2202280618 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2438003d10 con 0x7f243c0a6e20 2024-08-19T10:44:05.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:05.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:05.552 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 10 monclient: dump: 2024-08-19T10:44:05.552 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:05.553 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:05.553 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:05.553 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:05.553 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:05.553 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:05.553 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:05.553 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:05.553 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:05.553 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:05.553 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:05.554 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:05.554 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:05.554 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:05.554 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:05.554 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:05.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 1 -- 172.21.15.190:0/2202280618 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f24380108f0 con 0x7f243c0a6e20 2024-08-19T10:44:05.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24327fc640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f242003be80 0x7f242003e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:05.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24410bd640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f242003be80 0x7f242003e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:05.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.214+0000 7f24410bd640 10 monclient: get_auth_request con 0x7f242003be80 auth_method 0 2024-08-19T10:44:05.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.218+0000 7f24410bd640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f242003be80 0x7f242003e340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f243c09e9a0 tx=0x7f242c003ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:05.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.218+0000 7f24327fc640 1 -- 172.21.15.190:0/2202280618 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(196..196 src has 1..196) ==== 8441+0+0 (secure 0 0 0) 0x7f243804faf0 con 0x7f243c0a6e20 2024-08-19T10:44:05.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.838+0000 7f24327fc640 1 -- 172.21.15.190:0/2202280618 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f243804d690 con 0x7f243c0a6e20 2024-08-19T10:44:05.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.838+0000 7f24327fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:05.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:05.838+0000 7f24327fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:06.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:06.014+0000 7f2443b22640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:06.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:06.014+0000 7f2443b22640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:06.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:06.014+0000 7f2443b22640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:06.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:06.014+0000 7f2443b22640 1 -- 172.21.15.190:0/2202280618 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f243c139480 con 0x7f243c0a6e20 2024-08-19T10:44:07.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.790+0000 7f24327fc640 1 -- 172.21.15.190:0/2202280618 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}]=0 added bucket h4 type host to location {rack=r1} v198) ==== 166+0+0 (secure 0 0 0) 0x7f2438010dc0 con 0x7f243c0a6e20 2024-08-19T10:44:07.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.790+0000 7f24327fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:07.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.790+0000 7f24327fc640 10 monclient: _finish_command 2 = system:0 added bucket h4 type host to location {rack=r1} 2024-08-19T10:44:07.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.902+0000 7f2443b22640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:07.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.902+0000 7f2443b22640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:07.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.902+0000 7f2443b22640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:07.903 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.902+0000 7f2443b22640 1 -- 172.21.15.190:0/2202280618 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f243c0008e0 con 0x7f243c0a6e20 2024-08-19T10:44:07.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.902+0000 7f24327fc640 1 -- 172.21.15.190:0/2202280618 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h4' already exists v198) ==== 145+0+0 (secure 0 0 0) 0x7f243800dd70 con 0x7f243c0a6e20 2024-08-19T10:44:07.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.902+0000 7f24327fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h4", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:07.905 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.902+0000 7f24327fc640 10 monclient: _finish_command 3 = system:0 bucket 'h4' already exists 2024-08-19T10:44:07.905 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h4' already exists 2024-08-19T10:44:07.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 -- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f242003be80 msgr2=0x7f242003e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:07.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f242003be80 0x7f242003e340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f243c09e9a0 tx=0x7f242c003ba0 comp rx=0 tx=0).stop 2024-08-19T10:44:07.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 10 monclient: shutdown 2024-08-19T10:44:07.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:07.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 -- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f243c0a6e20 msgr2=0x7f243c134a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:07.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f243c0a6e20 0x7f243c134a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f243800b550 tx=0x7f243800ba10 comp rx=0 tx=0).stop 2024-08-19T10:44:07.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 -- 172.21.15.190:0/2202280618 shutdown_connections 2024-08-19T10:44:07.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f242003be80 0x7f242003e340 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:07.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f243c139fc0 0x7f243c09ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:07.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f243c1396a0 0x7f243c134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:07.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 --2- 172.21.15.190:0/2202280618 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f243c0a6e20 0x7f243c134a70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:07.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 -- 172.21.15.190:0/2202280618 >> 172.21.15.190:0/2202280618 conn(0x7f243c0abee0 msgr2=0x7f243c137880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:07.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 -- 172.21.15.190:0/2202280618 shutdown_connections 2024-08-19T10:44:07.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:07.906+0000 7f2443b22640 1 -- 172.21.15.190:0/2202280618 wait complete. 2024-08-19T10:44:07.923 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:07.923 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h5 host rack=r1 2024-08-19T10:44:08.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient: get_monmap_and_config 2024-08-19T10:44:08.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient: build_initial_monmap 2024-08-19T10:44:08.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient: monmap: 2024-08-19T10:44:08.047 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:08.048 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:08.048 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:08.047749+0000 2024-08-19T10:44:08.048 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:08.047749+0000 2024-08-19T10:44:08.048 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:08.048 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:08.048 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:08.048 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:08.048 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 1 Processor -- start 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 1 -- start start 2024-08-19T10:44:08.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient: init 2024-08-19T10:44:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient: _add_conns ranks=[9,5,2,3,8,7,10,4,6,1,0] 2024-08-19T10:44:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c980a4720 0x7f4c980a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient(hunting): picked mon.noname-a con 0x7f4c980a4720 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c980a5040 0x7f4c981398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient(hunting): picked mon.noname-h con 0x7f4c980a5040 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c98139e10 0x7f4c9813a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:08.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient(hunting): picked mon.noname-b con 0x7f4c98139e10 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f4c980a1400 con 0x7f4c980a4720 2024-08-19T10:44:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9806d280 con 0x7f4c98139e10 2024-08-19T10:44:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4c9806cbc0 con 0x7f4c980a5040 2024-08-19T10:44:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4ca05a4640 10 monclient(hunting): authenticate will time out at 1293.617606s 2024-08-19T10:44:08.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c980a5040 0x7f4c981398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c980a5040 0x7f4c981398d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48226/0 (socket says 172.21.15.190:48226) 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 1 -- 172.21.15.190:0/2026733780 learned_addr learned my addr 172.21.15.190:0/2026733780 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 10 monclient(hunting): get_auth_request con 0x7f4c980a5040 auth_method 0 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9e340640 1 --2- 172.21.15.190:0/2026733780 >> v2:172.21.15.89:3300/0 conn(0x7f4c980a4720 0x7f4c980a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9e340640 10 monclient(hunting): get_auth_request con 0x7f4c980a4720 auth_method 0 2024-08-19T10:44:08.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9e340640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9eb41640 1 --2- 172.21.15.190:0/2026733780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c98139e10 0x7f4c9813a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9e340640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9e340640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9eb41640 10 monclient(hunting): get_auth_request con 0x7f4c98139e10 auth_method 0 2024-08-19T10:44:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9eb41640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9eb41640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:08.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9eb41640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:08.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 10 monclient(hunting): handle_auth_done global_id 5990 payload 274 2024-08-19T10:44:08.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 1 -- 172.21.15.190:0/2026733780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c98139e10 msgr2=0x7f4c9813a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:08.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 1 --2- 172.21.15.190:0/2026733780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c98139e10 0x7f4c9813a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:08.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 1 -- 172.21.15.190:0/2026733780 >> v2:172.21.15.89:3300/0 conn(0x7f4c980a4720 msgr2=0x7f4c980a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:08.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 1 --2- 172.21.15.190:0/2026733780 >> v2:172.21.15.89:3300/0 conn(0x7f4c980a4720 0x7f4c980a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:08.056 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 10 monclient: _finish_hunting 0 2024-08-19T10:44:08.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 1 monclient: found mon.noname-h 2024-08-19T10:44:08.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:08.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:08.057 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.046+0000 7f4c9db3f640 1 -- 172.21.15.190:0/2026733780 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c980a0b60 con 0x7f4c980a5040 2024-08-19T10:44:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9db3f640 1 -- 172.21.15.190:0/2026733780 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c980a5040 msgr2=0x7f4c981398d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9db3f640 1 --2- 172.21.15.190:0/2026733780 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c980a5040 0x7f4c981398d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009e50 tx=0x7f4c9001d350 comp rx=0 tx=0).ready entity=mon.5 client_cookie=9d6ec4603084926d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:08.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 1 -- 172.21.15.190:0/2026733780 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4c900090d0 con 0x7f4c980a5040 2024-08-19T10:44:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 10 monclient: dump: 2024-08-19T10:44:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:08.065 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 10 monclient: _finish_auth 0 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 10 monclient: _check_auth_tickets 2024-08-19T10:44:08.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 1 -- 172.21.15.190:0/2026733780 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c900a5ab0 con 0x7f4c980a5040 2024-08-19T10:44:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 5 monclient: authenticate success, global_id 5990 2024-08-19T10:44:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 1 -- 172.21.15.190:0/2026733780 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4c900a5c60 con 0x7f4c980a5040 2024-08-19T10:44:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 4 set_mon_vals no callback set 2024-08-19T10:44:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4c9d33e640 10 monclient: dump: 2024-08-19T10:44:08.067 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:08.068 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:08.069 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:08.070 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 10 monclient: shutdown 2024-08-19T10:44:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 1 -- 172.21.15.190:0/2026733780 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c980a5040 msgr2=0x7f4c981398d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:08.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 1 --2- 172.21.15.190:0/2026733780 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c980a5040 0x7f4c981398d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009e50 tx=0x7f4c9001d350 comp rx=0 tx=0).stop 2024-08-19T10:44:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 1 -- 172.21.15.190:0/2026733780 shutdown_connections 2024-08-19T10:44:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 1 --2- 172.21.15.190:0/2026733780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c98139e10 0x7f4c9813a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 1 --2- 172.21.15.190:0/2026733780 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4c980a5040 0x7f4c981398d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 1 --2- 172.21.15.190:0/2026733780 >> v2:172.21.15.89:3300/0 conn(0x7f4c980a4720 0x7f4c980a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:08.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 1 -- 172.21.15.190:0/2026733780 >> 172.21.15.190:0/2026733780 conn(0x7f4c980a9310 msgr2=0x7f4c980a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 1 -- 172.21.15.190:0/2026733780 shutdown_connections 2024-08-19T10:44:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.050+0000 7f4ca05a4640 1 -- 172.21.15.190:0/2026733780 wait complete. 2024-08-19T10:44:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient: build_initial_monmap 2024-08-19T10:44:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient: monmap: 2024-08-19T10:44:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:08.072 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:08.073 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:08.074 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 1 Processor -- start 2024-08-19T10:44:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 1 -- start start 2024-08-19T10:44:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient: init 2024-08-19T10:44:08.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient: _add_conns ranks=[0,2,6,7,1,5,4,8,3] 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c980a4720 0x7f4c981d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient(hunting): picked mon.noname-a con 0x7f4c980a4720 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c980a5040 0x7f4c981d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient(hunting): picked mon.noname-b con 0x7f4c980a5040 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c98139e10 0x7f4c981d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient(hunting): picked mon.noname-d con 0x7f4c98139e10 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9806cf50 con 0x7f4c980a4720 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4c980a1de0 con 0x7f4c980a5040 2024-08-19T10:44:08.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4c980a5420 con 0x7f4c98139e10 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4ca05a4640 10 monclient(hunting): authenticate will time out at 1293.625088s 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9eb41640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c98139e10 msgr2=0x7f4c981d1f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9eb41640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c98139e10 0x7f4c981d1f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9e340640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c980a4720 0x7f4c981d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c980a5040 0x7f4c981d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9e340640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c980a4720 0x7f4c981d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:51458/0 (socket says 172.21.15.190:51458) 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9e340640 1 -- 172.21.15.190:0/658903086 learned_addr learned my addr 172.21.15.190:0/658903086 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9e340640 10 monclient(hunting): get_auth_request con 0x7f4c980a4720 auth_method 0 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9e340640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9e340640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:08.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9e340640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 10 monclient(hunting): get_auth_request con 0x7f4c980a5040 auth_method 0 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9e340640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9e340640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9e340640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 10 monclient(hunting): handle_auth_done global_id 5978 payload 824 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 1 -- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c98139e10 msgr2=0x7f4c981d1f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:08.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c98139e10 0x7f4c981d1f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:08.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 1 -- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c980a4720 msgr2=0x7f4c981d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:08.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c980a4720 0x7f4c981d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:08.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 10 monclient: _finish_hunting 0 2024-08-19T10:44:08.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 1 monclient: found mon.noname-b 2024-08-19T10:44:08.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:08.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:44:08.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9db3f640 1 -- 172.21.15.190:0/658903086 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c980a8a50 con 0x7f4c980a5040 2024-08-19T10:44:08.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.054+0000 7f4c9e340640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c980a4720 0x7f4c981d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:08.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c9db3f640 1 -- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c980a5040 msgr2=0x7f4c981d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:08.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c9db3f640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c980a5040 0x7f4c981d1a30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c900ab000 tx=0x7f4c90019690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:08.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 1 -- 172.21.15.190:0/658903086 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4c900bc070 con 0x7f4c980a5040 2024-08-19T10:44:08.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:08.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:44:08.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 10 monclient: dump: 2024-08-19T10:44:08.079 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:08.079 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:08.079 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:08.080 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:08.080 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:08.080 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:08.080 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:08.080 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:08.080 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:08.080 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:08.080 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:08.080 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:08.080 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:08.080 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:08.081 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:08.081 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:08.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 10 monclient: _finish_auth 0 2024-08-19T10:44:08.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:08.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:08.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 1 -- 172.21.15.190:0/658903086 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c900a58d0 con 0x7f4c980a5040 2024-08-19T10:44:08.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:08.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4ca05a4640 5 monclient: authenticate success, global_id 5978 2024-08-19T10:44:08.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4ca05a4640 10 monclient: _renew_subs 2024-08-19T10:44:08.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4ca05a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:08.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4ca05a4640 1 -- 172.21.15.190:0/658903086 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c98123eb0 con 0x7f4c980a5040 2024-08-19T10:44:08.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 1 -- 172.21.15.190:0/658903086 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4c90003bb0 con 0x7f4c980a5040 2024-08-19T10:44:08.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:08.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c9cb3d640 4 set_mon_vals no callback set 2024-08-19T10:44:08.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:08.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 10 monclient: dump: 2024-08-19T10:44:08.082 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:08.082 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:08.083 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4ca05a4640 10 monclient: _renew_subs 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4ca05a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4ca05a4640 1 -- 172.21.15.190:0/658903086 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c980a4b00 con 0x7f4c980a5040 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4ca05a4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4ca05a4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4ca05a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4ca05a4640 1 -- 172.21.15.190:0/658903086 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c980008e0 con 0x7f4c980a5040 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 1 -- 172.21.15.190:0/658903086 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4c90004040 con 0x7f4c980a5040 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c7c03bdf0 0x7f4c7c03e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c8affd640 1 -- 172.21.15.190:0/658903086 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(198..198 src has 1..198) ==== 8651+0+0 (secure 0 0 0) 0x7f4c900ea1f0 con 0x7f4c980a5040 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c9e340640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c7c03bdf0 0x7f4c7c03e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:08.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.058+0000 7f4c9e340640 10 monclient: get_auth_request con 0x7f4c7c03bdf0 auth_method 0 2024-08-19T10:44:08.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.062+0000 7f4c9e340640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c7c03bdf0 0x7f4c7c03e2b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c007f00 tx=0x7f4c8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:08.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.070+0000 7f4c8affd640 1 -- 172.21.15.190:0/658903086 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4c980a8a50 con 0x7f4c980a5040 2024-08-19T10:44:08.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.070+0000 7f4c8affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:08.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.070+0000 7f4c8affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.246+0000 7f4ca05a4640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.246+0000 7f4ca05a4640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.246+0000 7f4ca05a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:08.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.246+0000 7f4ca05a4640 1 -- 172.21.15.190:0/658903086 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f4c981398d0 con 0x7f4c980a5040 2024-08-19T10:44:08.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.798+0000 7f4c8affd640 1 -- 172.21.15.190:0/658903086 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}]=0 added bucket h5 type host to location {rack=r1} v199) ==== 166+0+0 (secure 0 0 0) 0x7f4c90016600 con 0x7f4c980a5040 2024-08-19T10:44:08.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.798+0000 7f4c8affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:08.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.798+0000 7f4c8affd640 10 monclient: _finish_command 2 = system:0 added bucket h5 type host to location {rack=r1} 2024-08-19T10:44:08.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.906+0000 7f4ca05a4640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:08.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.906+0000 7f4ca05a4640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:08.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.910+0000 7f4ca05a4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:08.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.910+0000 7f4ca05a4640 1 -- 172.21.15.190:0/658903086 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f4c9813fcc0 con 0x7f4c980a5040 2024-08-19T10:44:08.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.910+0000 7f4c8affd640 1 -- 172.21.15.190:0/658903086 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h5' already exists v199) ==== 145+0+0 (secure 0 0 0) 0x7f4c900b6d60 con 0x7f4c980a5040 2024-08-19T10:44:08.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.910+0000 7f4c8affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h5", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:08.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.910+0000 7f4c8affd640 10 monclient: _finish_command 3 = system:0 bucket 'h5' already exists 2024-08-19T10:44:08.917 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h5' already exists 2024-08-19T10:44:08.917 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 -- 172.21.15.190:0/658903086 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c7c03bdf0 msgr2=0x7f4c7c03e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:08.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c7c03bdf0 0x7f4c7c03e2b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c007f00 tx=0x7f4c8c008040 comp rx=0 tx=0).stop 2024-08-19T10:44:08.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 10 monclient: shutdown 2024-08-19T10:44:08.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:08.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 -- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c980a5040 msgr2=0x7f4c981d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:08.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c980a5040 0x7f4c981d1a30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c900ab000 tx=0x7f4c90019690 comp rx=0 tx=0).stop 2024-08-19T10:44:08.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 -- 172.21.15.190:0/658903086 shutdown_connections 2024-08-19T10:44:08.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4c7c03bdf0 0x7f4c7c03e2b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:08.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4c98139e10 0x7f4c981d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:08.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4c980a5040 0x7f4c981d1a30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:08.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 --2- 172.21.15.190:0/658903086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c980a4720 0x7f4c981d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:08.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 -- 172.21.15.190:0/658903086 >> 172.21.15.190:0/658903086 conn(0x7f4c980a9310 msgr2=0x7f4c980a6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:08.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 -- 172.21.15.190:0/658903086 shutdown_connections 2024-08-19T10:44:08.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:08.914+0000 7f4ca05a4640 1 -- 172.21.15.190:0/658903086 wait complete. 2024-08-19T10:44:08.930 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:08.930 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h6 host rack=r1 2024-08-19T10:44:09.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient: get_monmap_and_config 2024-08-19T10:44:09.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient: build_initial_monmap 2024-08-19T10:44:09.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient: monmap: 2024-08-19T10:44:09.047 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:09.047 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:09.047 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:09.047788+0000 2024-08-19T10:44:09.047 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:09.047788+0000 2024-08-19T10:44:09.047 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:09.047 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:09.047 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:09.047 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:09.047 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:09.048 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:09.048 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:09.048 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:09.048 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:09.048 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:09.048 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:09.048 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:09.048 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:09.048 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:09.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 1 Processor -- start 2024-08-19T10:44:09.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 1 -- start start 2024-08-19T10:44:09.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient: init 2024-08-19T10:44:09.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:09.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient: _add_conns ranks=[7,3,0,1,10,8,6,5,4,2,9] 2024-08-19T10:44:09.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c040a68e0 0x7f5c040a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient(hunting): picked mon.noname-i con 0x7f5c040a68e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c040a6cc0 0x7f5c040a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient(hunting): picked mon.noname-g con 0x7f5c040a6cc0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5c041396a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c040a86c0 con 0x7f5c041396a0 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5c040a8410 con 0x7f5c040a6cc0 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5c040a63a0 con 0x7f5c040a68e0 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c0999e640 10 monclient(hunting): authenticate will time out at 1294.617802s 2024-08-19T10:44:09.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5bfa7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c040a6cc0 0x7f5c040a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c040a68e0 0x7f5c040a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5bfa7fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c040a6cc0 0x7f5c040a70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:59984/0 (socket says 172.21.15.190:59984) 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5bfa7fc640 1 -- 172.21.15.190:0/3758500040 learned_addr learned my addr 172.21.15.190:0/3758500040 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5bfa7fc640 10 monclient(hunting): get_auth_request con 0x7f5c040a6cc0 auth_method 0 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5bfa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5bfa7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5bfa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 10 monclient(hunting): get_auth_request con 0x7f5c040a68e0 auth_method 0 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5bfa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:09.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5bfa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:09.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5bfa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:09.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:09.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:09.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:09.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 10 monclient(hunting): handle_auth_done global_id 15271 payload 274 2024-08-19T10:44:09.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 1 -- 172.21.15.190:0/3758500040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c040a6cc0 msgr2=0x7f5c040a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:09.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 1 --2- 172.21.15.190:0/3758500040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c040a6cc0 0x7f5c040a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:09.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 1 -- 172.21.15.190:0/3758500040 >> v1:172.21.15.89:6789/0 conn(0x7f5c041396a0 legacy=0x7f5c04139af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:09.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:09.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 1 monclient: found mon.noname-i 2024-08-19T10:44:09.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:09.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:09.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5c02ffd640 1 -- 172.21.15.190:0/3758500040 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c040a4680 con 0x7f5c040a68e0 2024-08-19T10:44:09.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.046+0000 7f5bfa7fc640 1 --2- 172.21.15.190:0/3758500040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c040a6cc0 0x7f5c040a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c02ffd640 1 -- 172.21.15.190:0/3758500040 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c040a68e0 msgr2=0x7f5c040a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c02ffd640 1 --2- 172.21.15.190:0/3758500040 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c040a68e0 0x7f5c040a5760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40097f0 tx=0x7f5bf401c080 comp rx=0 tx=0).ready entity=mon.7 client_cookie=3502e3384d1c5182 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 1 -- 172.21.15.190:0/3758500040 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5bf40a78c0 con 0x7f5c040a68e0 2024-08-19T10:44:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 10 monclient: dump: 2024-08-19T10:44:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:09.060 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:09.061 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:09.061 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:09.061 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:09.061 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:09.061 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:09.061 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:09.061 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:09.061 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 1 -- 172.21.15.190:0/3758500040 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf40a7a80 con 0x7f5c040a68e0 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 5 monclient: authenticate success, global_id 15271 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:09.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 1 -- 172.21.15.190:0/3758500040 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5bf40a7c30 con 0x7f5c040a68e0 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 4 set_mon_vals no callback set 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c027fc640 10 monclient: dump: 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:09.063 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 10 monclient: shutdown 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 1 -- 172.21.15.190:0/3758500040 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c040a68e0 msgr2=0x7f5c040a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:09.064 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 1 --2- 172.21.15.190:0/3758500040 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c040a68e0 0x7f5c040a5760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40097f0 tx=0x7f5bf401c080 comp rx=0 tx=0).stop 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 1 -- 172.21.15.190:0/3758500040 shutdown_connections 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 1 --2- 172.21.15.190:0/3758500040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c040a6cc0 0x7f5c040a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 1 --2- 172.21.15.190:0/3758500040 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c040a68e0 0x7f5c040a5760 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 1 -- 172.21.15.190:0/3758500040 >> 172.21.15.190:0/3758500040 conn(0x7f5c040abee0 msgr2=0x7f5c040ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 1 -- 172.21.15.190:0/3758500040 shutdown_connections 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.050+0000 7f5c0999e640 1 -- 172.21.15.190:0/3758500040 wait complete. 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient: build_initial_monmap 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient: monmap: 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:09.065 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:09.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 1 Processor -- start 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 1 -- start start 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient: init 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient: _add_conns ranks=[8,1,3,5,2,0,7,4,6] 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c040a68e0 0x7f5c041380f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient(hunting): picked mon.noname-e con 0x7f5c040a68e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c040a6cc0 0x7f5c04138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient(hunting): picked mon.noname-f con 0x7f5c040a6cc0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c041396a0 0x7f5c0413dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient(hunting): picked mon.noname-g con 0x7f5c041396a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04144cd0 con 0x7f5c040a6cc0 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:09.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5c04144b50 con 0x7f5c041396a0 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f5c04144e50 con 0x7f5c040a68e0 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c0999e640 10 monclient(hunting): authenticate will time out at 1294.625367s 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5bfa7fc640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c040a6cc0 msgr2=0x7f5c04138630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5bfa7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c040a6cc0 0x7f5c04138630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c041396a0 0x7f5c0413dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c041396a0 0x7f5c0413dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:60000/0 (socket says 172.21.15.190:60000) 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 1 -- 172.21.15.190:0/773449392 learned_addr learned my addr 172.21.15.190:0/773449392 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 10 monclient(hunting): get_auth_request con 0x7f5c041396a0 auth_method 0 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:09.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 10 monclient(hunting): handle_auth_done global_id 34347 payload 824 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 1 -- 172.21.15.190:0/773449392 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c040a68e0 msgr2=0x7f5c041380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c040a68e0 0x7f5c041380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 1 -- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c040a6cc0 msgr2=0x7f5c04138630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c040a6cc0 0x7f5c04138630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 1 monclient: found mon.noname-g 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:09.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.054+0000 7f5c037fe640 1 -- 172.21.15.190:0/773449392 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c040aadf0 con 0x7f5c041396a0 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c037fe640 1 -- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c041396a0 msgr2=0x7f5c0413dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c037fe640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c041396a0 0x7f5c0413dad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b970 tx=0x7f5bfc00be30 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 1 -- 172.21.15.190:0/773449392 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5bfc019070 con 0x7f5c041396a0 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 10 monclient: dump: 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:09.070 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:09.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 1 -- 172.21.15.190:0/773449392 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bfc00cbb0 con 0x7f5c041396a0 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 1 -- 172.21.15.190:0/773449392 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5bfc00cdc0 con 0x7f5c041396a0 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 10 monclient: dump: 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:09.072 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c017fa640 4 set_mon_vals no callback set 2024-08-19T10:44:09.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c0999e640 5 monclient: authenticate success, global_id 34347 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c0999e640 10 monclient: _renew_subs 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c0999e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c0999e640 1 -- 172.21.15.190:0/773449392 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c04122ad0 con 0x7f5c041396a0 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c0999e640 10 monclient: _renew_subs 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c0999e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c0999e640 1 -- 172.21.15.190:0/773449392 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c04140dc0 con 0x7f5c041396a0 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c0999e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c0999e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c0999e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c0999e640 1 -- 172.21.15.190:0/773449392 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c040a5760 con 0x7f5c041396a0 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 1 -- 172.21.15.190:0/773449392 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5bfc006d10 con 0x7f5c041396a0 2024-08-19T10:44:09.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5be403be00 0x7f5be403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:09.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c00ff9640 1 -- 172.21.15.190:0/773449392 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(199..199 src has 1..199) ==== 8989+0+0 (secure 0 0 0) 0x7f5bfc04f480 con 0x7f5c041396a0 2024-08-19T10:44:09.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c02ffd640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5be403be00 0x7f5be403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:09.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.058+0000 7f5c02ffd640 10 monclient: get_auth_request con 0x7f5be403be00 auth_method 0 2024-08-19T10:44:09.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.062+0000 7f5c02ffd640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5be403be00 0x7f5be403e2c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5c040a7f30 tx=0x7f5bf4004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:09.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.062+0000 7f5c00ff9640 1 -- 172.21.15.190:0/773449392 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f5c040aadf0 con 0x7f5c041396a0 2024-08-19T10:44:09.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.062+0000 7f5c00ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:09.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.062+0000 7f5c00ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:09.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.234+0000 7f5c0999e640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:09.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.234+0000 7f5c0999e640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:09.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.234+0000 7f5c0999e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:09.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:09.234+0000 7f5c0999e640 1 -- 172.21.15.190:0/773449392 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f5c0406c9c0 con 0x7f5c041396a0 2024-08-19T10:44:10.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.002+0000 7f5c00ff9640 1 -- 172.21.15.190:0/773449392 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}]=0 added bucket h6 type host to location {rack=r1} v200) ==== 166+0+0 (secure 0 0 0) 0x7f5bfc013560 con 0x7f5c041396a0 2024-08-19T10:44:10.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.002+0000 7f5c00ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:10.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.002+0000 7f5c00ff9640 10 monclient: _finish_command 2 = system:0 added bucket h6 type host to location {rack=r1} 2024-08-19T10:44:10.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.102+0000 7f5c0999e640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:10.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.102+0000 7f5c0999e640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:10.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.102+0000 7f5c0999e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:10.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.102+0000 7f5c0999e640 1 -- 172.21.15.190:0/773449392 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f5c041227f0 con 0x7f5c041396a0 2024-08-19T10:44:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.102+0000 7f5c00ff9640 1 -- 172.21.15.190:0/773449392 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h6' already exists v200) ==== 145+0+0 (secure 0 0 0) 0x7f5bfc04f1f0 con 0x7f5c041396a0 2024-08-19T10:44:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.102+0000 7f5c00ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h6", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.102+0000 7f5c00ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h6' already exists 2024-08-19T10:44:10.106 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h6' already exists 2024-08-19T10:44:10.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 -- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5be403be00 msgr2=0x7f5be403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:10.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5be403be00 0x7f5be403e2c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5c040a7f30 tx=0x7f5bf4004740 comp rx=0 tx=0).stop 2024-08-19T10:44:10.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 10 monclient: shutdown 2024-08-19T10:44:10.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:10.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 -- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c041396a0 msgr2=0x7f5c0413dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:10.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c041396a0 0x7f5c0413dad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b970 tx=0x7f5bfc00be30 comp rx=0 tx=0).stop 2024-08-19T10:44:10.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 -- 172.21.15.190:0/773449392 shutdown_connections 2024-08-19T10:44:10.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c041396a0 0x7f5c0413dad0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c040a6cc0 0x7f5c04138630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5be403be00 0x7f5be403e2c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.109 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 --2- 172.21.15.190:0/773449392 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c040a68e0 0x7f5c041380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 -- 172.21.15.190:0/773449392 >> 172.21.15.190:0/773449392 conn(0x7f5c040abee0 msgr2=0x7f5c040a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:10.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 -- 172.21.15.190:0/773449392 shutdown_connections 2024-08-19T10:44:10.110 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.106+0000 7f5c0999e640 1 -- 172.21.15.190:0/773449392 wait complete. 2024-08-19T10:44:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:10.123 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h7 host rack=r1 2024-08-19T10:44:10.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.242+0000 7f767e8d7640 10 monclient: get_monmap_and_config 2024-08-19T10:44:10.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.242+0000 7f767e8d7640 10 monclient: build_initial_monmap 2024-08-19T10:44:10.244 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.242+0000 7f767e8d7640 10 monclient: monmap: 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:10.245712+0000 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:10.245712+0000 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:10.245 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:10.246 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:10.246 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:10.246 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:10.246 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:10.246 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:10.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.242+0000 7f767e8d7640 1 Processor -- start 2024-08-19T10:44:10.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.242+0000 7f767e8d7640 1 -- start start 2024-08-19T10:44:10.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.242+0000 7f767e8d7640 10 monclient: init 2024-08-19T10:44:10.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:10.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 10 monclient: _add_conns ranks=[9,6,3,2,0,7,1,4,10,8,5] 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f767809f950 0x7f767809fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 10 monclient(hunting): picked mon.noname-a con 0x7f767809f950 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76780a0ed0 0x7f76780a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 10 monclient(hunting): picked mon.noname-d con 0x7f76780a0ed0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767809b0c0 0x7f767809b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 10 monclient(hunting): picked mon.noname-g con 0x7f767809b0c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f767809e940 con 0x7f767809f950 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f76780a07b0 con 0x7f767809b0c0 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f76780a1a30 con 0x7f76780a0ed0 2024-08-19T10:44:10.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767e8d7640 10 monclient(hunting): authenticate will time out at 1295.816406s 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f76777fe640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76780a0ed0 msgr2=0x7f76780a0270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f76777fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76780a0ed0 0x7f76780a0270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767809b0c0 0x7f767809b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767809b0c0 0x7f767809b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:60016/0 (socket says 172.21.15.190:60016) 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 1 -- 172.21.15.190:0/3590997700 learned_addr learned my addr 172.21.15.190:0/3590997700 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 10 monclient(hunting): get_auth_request con 0x7f767809b0c0 auth_method 0 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:10.250 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f7677fff640 1 --2- 172.21.15.190:0/3590997700 >> v2:172.21.15.89:3300/0 conn(0x7f767809f950 0x7f767809fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f7677fff640 10 monclient(hunting): get_auth_request con 0x7f767809f950 auth_method 0 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f7677fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f7677fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f7677fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 10 monclient(hunting): handle_auth_done global_id 34356 payload 274 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 1 -- 172.21.15.190:0/3590997700 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76780a0ed0 msgr2=0x7f76780a0270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 1 --2- 172.21.15.190:0/3590997700 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76780a0ed0 0x7f76780a0270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 1 -- 172.21.15.190:0/3590997700 >> v2:172.21.15.89:3300/0 conn(0x7f767809f950 msgr2=0x7f767809fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 1 --2- 172.21.15.190:0/3590997700 >> v2:172.21.15.89:3300/0 conn(0x7f767809f950 0x7f767809fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 10 monclient: _finish_hunting 0 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 1 monclient: found mon.noname-g 2024-08-19T10:44:10.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:10.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:10.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f767ce74640 1 -- 172.21.15.190:0/3590997700 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767806dc70 con 0x7f767809b0c0 2024-08-19T10:44:10.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.246+0000 7f7677fff640 1 --2- 172.21.15.190:0/3590997700 >> v2:172.21.15.89:3300/0 conn(0x7f767809f950 0x7f767809fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:44:10.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767ce74640 1 -- 172.21.15.190:0/3590997700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767809b0c0 msgr2=0x7f767809b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:10.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767ce74640 1 --2- 172.21.15.190:0/3590997700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767809b0c0 0x7f767809b540 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7670009e50 tx=0x7f767001c8b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=d11b9ab19226f2ca server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:10.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 1 -- 172.21.15.190:0/3590997700 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76700090d0 con 0x7f767809b0c0 2024-08-19T10:44:10.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:10.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 10 monclient: dump: 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:10.256 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 1 -- 172.21.15.190:0/3590997700 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76700a5a90 con 0x7f767809b0c0 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 5 monclient: authenticate success, global_id 34356 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:10.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 1 -- 172.21.15.190:0/3590997700 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76700a5c40 con 0x7f767809b0c0 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 4 set_mon_vals no callback set 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7676ffd640 10 monclient: dump: 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:10.258 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient: shutdown 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 -- 172.21.15.190:0/3590997700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767809b0c0 msgr2=0x7f767809b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 --2- 172.21.15.190:0/3590997700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767809b0c0 0x7f767809b540 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7670009e50 tx=0x7f767001c8b0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 -- 172.21.15.190:0/3590997700 shutdown_connections 2024-08-19T10:44:10.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 --2- 172.21.15.190:0/3590997700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f767809b0c0 0x7f767809b540 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 --2- 172.21.15.190:0/3590997700 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76780a0ed0 0x7f76780a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 --2- 172.21.15.190:0/3590997700 >> v2:172.21.15.89:3300/0 conn(0x7f767809f950 0x7f767809fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 -- 172.21.15.190:0/3590997700 >> 172.21.15.190:0/3590997700 conn(0x7f76780a8450 msgr2=0x7f76780a8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 -- 172.21.15.190:0/3590997700 shutdown_connections 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 -- 172.21.15.190:0/3590997700 wait complete. 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient: build_initial_monmap 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient: monmap: 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:10.260 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 Processor -- start 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 -- start start 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient: init 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:10.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient: _add_conns ranks=[5,0,2,4,8,7,6,1,3] 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f767809b0c0 0x7f7678099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient(hunting): picked mon.noname-h con 0x7f767809b0c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f767809f950 0x7f7678070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient(hunting): picked mon.noname-a con 0x7f767809f950 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76780a0ed0 0x7f76780712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient(hunting): picked mon.noname-b con 0x7f76780a0ed0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f76780695b0 con 0x7f767809f950 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f76780a2800 con 0x7f76780a0ed0 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7678140010 con 0x7f767809b0c0 2024-08-19T10:44:10.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f767e8d7640 10 monclient(hunting): authenticate will time out at 1295.823490s 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f767809b0c0 0x7f7678099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f767809b0c0 0x7f7678099ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48232/0 (socket says 172.21.15.190:48232) 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 1 -- 172.21.15.190:0/1238005725 learned_addr learned my addr 172.21.15.190:0/1238005725 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 10 monclient(hunting): get_auth_request con 0x7f767809b0c0 auth_method 0 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 10 monclient(hunting): handle_auth_done global_id 5999 payload 824 2024-08-19T10:44:10.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f76777fe640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f767809f950 0x7f7678070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 1 -- 172.21.15.190:0/1238005725 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76780a0ed0 msgr2=0x7f76780712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76780a0ed0 0x7f76780712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 1 -- 172.21.15.190:0/1238005725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f767809f950 msgr2=0x7f7678070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f767809f950 0x7f7678070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 1 monclient: found mon.noname-h 2024-08-19T10:44:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f7677fff640 1 -- 172.21.15.190:0/1238005725 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7678095fd0 con 0x7f767809b0c0 2024-08-19T10:44:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f76777fe640 10 monclient: get_auth_request con 0x7f767809f950 auth_method 0 2024-08-19T10:44:10.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.250+0000 7f76777fe640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f767809f950 0x7f7678070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:10.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7677fff640 1 -- 172.21.15.190:0/1238005725 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f767809b0c0 msgr2=0x7f7678099ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:10.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7677fff640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f767809b0c0 0x7f7678099ac0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f766800e7c0 tx=0x7f766800ec80 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:10.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 1 -- 172.21.15.190:0/1238005725 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7668017070 con 0x7f767809b0c0 2024-08-19T10:44:10.267 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 10 monclient: dump: 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:10.268 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f767e8d7640 5 monclient: authenticate success, global_id 5999 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 1 -- 172.21.15.190:0/1238005725 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766800cd50 con 0x7f767809b0c0 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:10.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 1 -- 172.21.15.190:0/1238005725 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f766800f040 con 0x7f767809b0c0 2024-08-19T10:44:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f767e8d7640 10 monclient: _renew_subs 2024-08-19T10:44:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f767e8d7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f767e8d7640 1 -- 172.21.15.190:0/1238005725 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7678095120 con 0x7f767809b0c0 2024-08-19T10:44:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 10 monclient: dump: 2024-08-19T10:44:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:10.270 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:10.271 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f76767fc640 4 set_mon_vals no callback set 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f767e8d7640 10 monclient: _renew_subs 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f767e8d7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f767e8d7640 1 -- 172.21.15.190:0/1238005725 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f7678095cf0 con 0x7f767809b0c0 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f767e8d7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f767e8d7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f767e8d7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f767e8d7640 1 -- 172.21.15.190:0/1238005725 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f767809b540 con 0x7f767809b0c0 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 1 -- 172.21.15.190:0/1238005725 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f766801a070 con 0x7f767809b0c0 2024-08-19T10:44:10.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f766403be00 0x7f766403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:10.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f7674ff9640 1 -- 172.21.15.190:0/1238005725 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(200..200 src has 1..200) ==== 9199+0+0 (secure 0 0 0) 0x7f766804f830 con 0x7f767809b0c0 2024-08-19T10:44:10.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f76777fe640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f766403be00 0x7f766403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:10.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.254+0000 7f76777fe640 10 monclient: get_auth_request con 0x7f766403be00 auth_method 0 2024-08-19T10:44:10.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.258+0000 7f76777fe640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f766403be00 0x7f766403e2c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f76780724c0 tx=0x7f766c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:10.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.258+0000 7f7674ff9640 1 -- 172.21.15.190:0/1238005725 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:44:10.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.262+0000 7f7674ff9640 1 -- 172.21.15.190:0/1238005725 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7668012dc0 con 0x7f767809b0c0 2024-08-19T10:44:10.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.262+0000 7f7674ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:10.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.262+0000 7f7674ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:10.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.454+0000 7f767e8d7640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:10.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.454+0000 7f767e8d7640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:10.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.454+0000 7f767e8d7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:10.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:10.454+0000 7f767e8d7640 1 -- 172.21.15.190:0/1238005725 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f76780a0c90 con 0x7f767809b0c0 2024-08-19T10:44:11.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.330+0000 7f7674ff9640 1 -- 172.21.15.190:0/1238005725 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}]=0 added bucket h7 type host to location {rack=r1} v202) ==== 166+0+0 (secure 0 0 0) 0x7f766804d0e0 con 0x7f767809b0c0 2024-08-19T10:44:11.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.330+0000 7f7674ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:11.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.330+0000 7f7674ff9640 10 monclient: _finish_command 2 = system:0 added bucket h7 type host to location {rack=r1} 2024-08-19T10:44:11.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.438+0000 7f767e8d7640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:11.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.438+0000 7f767e8d7640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:11.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.438+0000 7f767e8d7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:11.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.438+0000 7f767e8d7640 1 -- 172.21.15.190:0/1238005725 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f7678072ee0 con 0x7f767809b0c0 2024-08-19T10:44:11.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f7674ff9640 1 -- 172.21.15.190:0/1238005725 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h7' already exists v202) ==== 145+0+0 (secure 0 0 0) 0x7f7668013410 con 0x7f767809b0c0 2024-08-19T10:44:11.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f7674ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h7", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:11.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f7674ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h7' already exists 2024-08-19T10:44:11.443 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h7' already exists 2024-08-19T10:44:11.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 1 -- 172.21.15.190:0/1238005725 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f766403be00 msgr2=0x7f766403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:11.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f766403be00 0x7f766403e2c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f76780724c0 tx=0x7f766c008040 comp rx=0 tx=0).stop 2024-08-19T10:44:11.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 10 monclient: shutdown 2024-08-19T10:44:11.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:11.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 1 -- 172.21.15.190:0/1238005725 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f767809b0c0 msgr2=0x7f7678099ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:11.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f767809b0c0 0x7f7678099ac0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f766800e7c0 tx=0x7f766800ec80 comp rx=0 tx=0).stop 2024-08-19T10:44:11.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 1 -- 172.21.15.190:0/1238005725 shutdown_connections 2024-08-19T10:44:11.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f766403be00 0x7f766403e2c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:11.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f76780a0ed0 0x7f76780712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:11.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f767809f950 0x7f7678070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:11.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 1 --2- 172.21.15.190:0/1238005725 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f767809b0c0 0x7f7678099ac0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:11.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.442+0000 7f767e8d7640 1 -- 172.21.15.190:0/1238005725 >> 172.21.15.190:0/1238005725 conn(0x7f76780a8450 msgr2=0x7f767813ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:11.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.446+0000 7f767e8d7640 1 -- 172.21.15.190:0/1238005725 shutdown_connections 2024-08-19T10:44:11.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.446+0000 7f767e8d7640 1 -- 172.21.15.190:0/1238005725 wait complete. 2024-08-19T10:44:11.461 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:11.461 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h8 host rack=r1 2024-08-19T10:44:11.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient: get_monmap_and_config 2024-08-19T10:44:11.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient: build_initial_monmap 2024-08-19T10:44:11.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient: monmap: 2024-08-19T10:44:11.580 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:11.580 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:11.580 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:11.581386+0000 2024-08-19T10:44:11.580 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:11.581386+0000 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:11.581 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:11.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 1 Processor -- start 2024-08-19T10:44:11.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 1 -- start start 2024-08-19T10:44:11.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient: init 2024-08-19T10:44:11.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:11.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient: _add_conns ranks=[4,1,3,7,0,10,5,9,2,8,6] 2024-08-19T10:44:11.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54dc0a36a0 0x7f54dc0a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:11.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient(hunting): picked mon.noname-c con 0x7f54dc0a36a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:11.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f54dc0a3fc0 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54dc136b70 0x7f54dc13e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient(hunting): picked mon.noname-g con 0x7f54dc136b70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54dc06f220 con 0x7f54dc0a3fc0 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f54dc09ecb0 con 0x7f54dc136b70 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f54dc09f750 con 0x7f54dc0a36a0 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e429e640 10 monclient(hunting): authenticate will time out at 1297.151253s 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e1839640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f54dc0a3fc0 legacy=0x7f54dc136460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54dc136b70 0x7f54dc13e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:11.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e203a640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54dc0a36a0 msgr2=0x7f54dc0a3a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e203a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54dc0a36a0 0x7f54dc0a3a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54dc136b70 0x7f54dc13e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:60032/0 (socket says 172.21.15.190:60032) 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 1 -- 172.21.15.190:0/2569886123 learned_addr learned my addr 172.21.15.190:0/2569886123 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 10 monclient(hunting): get_auth_request con 0x7f54dc136b70 auth_method 0 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 10 monclient(hunting): handle_auth_done global_id 34365 payload 274 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 1 -- 172.21.15.190:0/2569886123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54dc0a36a0 msgr2=0x7f54dc0a3a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 1 --2- 172.21.15.190:0/2569886123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54dc0a36a0 0x7f54dc0a3a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:11.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 1 -- 172.21.15.190:0/2569886123 >> v1:172.21.15.190:6789/0 conn(0x7f54dc0a3fc0 legacy=0x7f54dc136460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:11.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 10 monclient: _finish_hunting 0 2024-08-19T10:44:11.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 1 monclient: found mon.noname-g 2024-08-19T10:44:11.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:11.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:11.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.578+0000 7f54e283b640 1 -- 172.21.15.190:0/2569886123 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54dc09fa30 con 0x7f54dc136b70 2024-08-19T10:44:11.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e283b640 1 -- 172.21.15.190:0/2569886123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54dc136b70 msgr2=0x7f54dc13e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:11.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e283b640 1 --2- 172.21.15.190:0/2569886123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54dc136b70 0x7f54dc13e6c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009eb0 tx=0x7f54d401c880 comp rx=0 tx=0).ready entity=mon.3 client_cookie=eb21db094ae2aeec server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:11.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 1 -- 172.21.15.190:0/2569886123 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54d40091b0 con 0x7f54dc136b70 2024-08-19T10:44:11.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:11.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 10 monclient: dump: 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:11.591 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 10 monclient: _finish_auth 0 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 10 monclient: _check_auth_tickets 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 1 -- 172.21.15.190:0/2569886123 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d40a5ac0 con 0x7f54dc136b70 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 1 -- 172.21.15.190:0/2569886123 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54d40a5c70 con 0x7f54dc136b70 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 5 monclient: authenticate success, global_id 34365 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 4 set_mon_vals no callback set 2024-08-19T10:44:11.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1038640 10 monclient: dump: 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:11.593 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient: shutdown 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 -- 172.21.15.190:0/2569886123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54dc136b70 msgr2=0x7f54dc13e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 --2- 172.21.15.190:0/2569886123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54dc136b70 0x7f54dc13e6c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009eb0 tx=0x7f54d401c880 comp rx=0 tx=0).stop 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 -- 172.21.15.190:0/2569886123 shutdown_connections 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 --2- 172.21.15.190:0/2569886123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54dc136b70 0x7f54dc13e6c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 --2- 172.21.15.190:0/2569886123 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54dc0a36a0 0x7f54dc0a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:11.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 -- 172.21.15.190:0/2569886123 >> 172.21.15.190:0/2569886123 conn(0x7f54dc0a4790 msgr2=0x7f54dc0a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 -- 172.21.15.190:0/2569886123 shutdown_connections 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 -- 172.21.15.190:0/2569886123 wait complete. 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient: build_initial_monmap 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient: monmap: 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:11.595 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:11.596 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:11.596 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:11.596 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:11.596 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:11.596 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:11.596 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:11.596 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:11.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 Processor -- start 2024-08-19T10:44:11.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 -- start start 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient: init 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient: _add_conns ranks=[5,8,0,1,2,3,6,4,7] 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f54dc0a36a0 0x7f54dc12c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient(hunting): picked mon.noname-h con 0x7f54dc0a36a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54dc0a3fc0 0x7f54dc12cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient(hunting): picked mon.noname-e con 0x7f54dc0a3fc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54dc136b70 0x7f54dc133eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient(hunting): picked mon.noname-a con 0x7f54dc136b70 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc0a43a0 con 0x7f54dc136b70 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f54dc0a09d0 con 0x7f54dc0a36a0 2024-08-19T10:44:11.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f54dc0a4520 con 0x7f54dc0a3fc0 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e429e640 10 monclient(hunting): authenticate will time out at 1297.158781s 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f54dc0a36a0 0x7f54dc12c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f54dc0a36a0 0x7f54dc12c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48256/0 (socket says 172.21.15.190:48256) 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 1 -- 172.21.15.190:0/3521615405 learned_addr learned my addr 172.21.15.190:0/3521615405 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 10 monclient(hunting): get_auth_request con 0x7f54dc0a36a0 auth_method 0 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:11.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e1839640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54dc0a3fc0 0x7f54dc12cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e283b640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54dc136b70 0x7f54dc133eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 10 monclient(hunting): handle_auth_done global_id 6008 payload 824 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 1 -- 172.21.15.190:0/3521615405 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54dc0a3fc0 msgr2=0x7f54dc12cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54dc0a3fc0 0x7f54dc12cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 1 -- 172.21.15.190:0/3521615405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54dc136b70 msgr2=0x7f54dc133eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54dc136b70 0x7f54dc133eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 10 monclient: _finish_hunting 0 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 1 monclient: found mon.noname-h 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e203a640 1 -- 172.21.15.190:0/3521615405 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54dc135800 con 0x7f54dc0a36a0 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e283b640 10 monclient: get_auth_request con 0x7f54dc136b70 auth_method 0 2024-08-19T10:44:11.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.586+0000 7f54e283b640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54dc136b70 0x7f54dc133eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:11.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e203a640 1 -- 172.21.15.190:0/3521615405 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f54dc0a36a0 msgr2=0x7f54dc12c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:11.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e203a640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f54dc0a36a0 0x7f54dc12c7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b530 tx=0x7f54d800b9f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:11.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 1 -- 172.21.15.190:0/3521615405 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54d8013020 con 0x7f54dc0a36a0 2024-08-19T10:44:11.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:11.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:11.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 10 monclient: dump: 2024-08-19T10:44:11.602 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:11.602 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:11.602 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:11.603 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 1 -- 172.21.15.190:0/3521615405 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d8003de0 con 0x7f54dc0a36a0 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e429e640 5 monclient: authenticate success, global_id 6008 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 1 -- 172.21.15.190:0/3521615405 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54d800f400 con 0x7f54dc0a36a0 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 10 monclient: dump: 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:11.604 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:11.605 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e0837640 4 set_mon_vals no callback set 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e429e640 10 monclient: _renew_subs 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e429e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e429e640 1 -- 172.21.15.190:0/3521615405 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54dc09cd40 con 0x7f54dc0a36a0 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e429e640 10 monclient: _renew_subs 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e429e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e429e640 1 -- 172.21.15.190:0/3521615405 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f54dc136740 con 0x7f54dc0a36a0 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e429e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e429e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e429e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e429e640 1 -- 172.21.15.190:0/3521615405 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54dc0a7710 con 0x7f54dc0a36a0 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 1 -- 172.21.15.190:0/3521615405 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f54d8016020 con 0x7f54dc0a36a0 2024-08-19T10:44:11.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f54c003be00 0x7f54c003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:11.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54d2ffd640 1 -- 172.21.15.190:0/3521615405 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(202..202 src has 1..202) ==== 9409+0+0 (secure 0 0 0) 0x7f54d804fa40 con 0x7f54dc0a36a0 2024-08-19T10:44:11.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e1839640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f54c003be00 0x7f54c003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:11.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.590+0000 7f54e1839640 10 monclient: get_auth_request con 0x7f54c003be00 auth_method 0 2024-08-19T10:44:11.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.594+0000 7f54e1839640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f54c003be00 0x7f54c003e2c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f54dc141b20 tx=0x7f54cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:11.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.594+0000 7f54d2ffd640 1 -- 172.21.15.190:0/3521615405 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f54d801d020 con 0x7f54dc0a36a0 2024-08-19T10:44:11.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.594+0000 7f54d2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:11.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.594+0000 7f54d2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:11.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.770+0000 7f54e429e640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.770+0000 7f54e429e640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.770+0000 7f54e429e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:11.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:11.770+0000 7f54e429e640 1 -- 172.21.15.190:0/3521615405 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f54dc09e5b0 con 0x7f54dc0a36a0 2024-08-19T10:44:12.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.342+0000 7f54d2ffd640 1 -- 172.21.15.190:0/3521615405 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}]=0 added bucket h8 type host to location {rack=r1} v203) ==== 166+0+0 (secure 0 0 0) 0x7f54dc09cd40 con 0x7f54dc0a36a0 2024-08-19T10:44:12.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.342+0000 7f54d2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:12.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.342+0000 7f54d2ffd640 10 monclient: _finish_command 2 = system:0 added bucket h8 type host to location {rack=r1} 2024-08-19T10:44:12.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.450+0000 7f54e429e640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:12.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.450+0000 7f54e429e640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:12.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.450+0000 7f54e429e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:12.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.450+0000 7f54e429e640 1 -- 172.21.15.190:0/3521615405 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f54dc135220 con 0x7f54dc0a36a0 2024-08-19T10:44:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.454+0000 7f54d2ffd640 1 -- 172.21.15.190:0/3521615405 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h8' already exists v203) ==== 145+0+0 (secure 0 0 0) 0x7f54d8005300 con 0x7f54dc0a36a0 2024-08-19T10:44:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.454+0000 7f54d2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h8", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.454+0000 7f54d2ffd640 10 monclient: _finish_command 3 = system:0 bucket 'h8' already exists 2024-08-19T10:44:12.455 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h8' already exists 2024-08-19T10:44:12.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.454+0000 7f54e429e640 1 -- 172.21.15.190:0/3521615405 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f54c003be00 msgr2=0x7f54c003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:12.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.454+0000 7f54e429e640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f54c003be00 0x7f54c003e2c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f54dc141b20 tx=0x7f54cc008040 comp rx=0 tx=0).stop 2024-08-19T10:44:12.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.454+0000 7f54e429e640 10 monclient: shutdown 2024-08-19T10:44:12.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.454+0000 7f54e429e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:12.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.454+0000 7f54e429e640 1 -- 172.21.15.190:0/3521615405 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f54dc0a36a0 msgr2=0x7f54dc12c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:12.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.454+0000 7f54e429e640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f54dc0a36a0 0x7f54dc12c7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b530 tx=0x7f54d800b9f0 comp rx=0 tx=0).stop 2024-08-19T10:44:12.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.458+0000 7f54e429e640 1 -- 172.21.15.190:0/3521615405 shutdown_connections 2024-08-19T10:44:12.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.458+0000 7f54e429e640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54dc136b70 0x7f54dc133eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:12.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.458+0000 7f54e429e640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f54c003be00 0x7f54c003e2c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:12.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.458+0000 7f54e429e640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54dc0a3fc0 0x7f54dc12cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:12.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.458+0000 7f54e429e640 1 --2- 172.21.15.190:0/3521615405 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f54dc0a36a0 0x7f54dc12c7a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:12.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.458+0000 7f54e429e640 1 -- 172.21.15.190:0/3521615405 >> 172.21.15.190:0/3521615405 conn(0x7f54dc0a4790 msgr2=0x7f54dc132b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:12.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.458+0000 7f54e429e640 1 -- 172.21.15.190:0/3521615405 shutdown_connections 2024-08-19T10:44:12.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.458+0000 7f54e429e640 1 -- 172.21.15.190:0/3521615405 wait complete. 2024-08-19T10:44:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:12.474 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h9 host rack=r1 2024-08-19T10:44:12.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient: get_monmap_and_config 2024-08-19T10:44:12.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient: build_initial_monmap 2024-08-19T10:44:12.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient: monmap: 2024-08-19T10:44:12.595 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:12.595 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:12.595 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:12.595818+0000 2024-08-19T10:44:12.595 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:12.595818+0000 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:12.598 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:12.599 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:12.599 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:12.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 1 Processor -- start 2024-08-19T10:44:12.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 1 -- start start 2024-08-19T10:44:12.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient: init 2024-08-19T10:44:12.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:12.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient: _add_conns ranks=[0,2,7,3,4,1,5,6,9,10,8] 2024-08-19T10:44:12.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc32c1396a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:12.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc32c13a190 0x7fc32c13eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:12.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient(hunting): picked mon.noname-b con 0x7fc32c13a190 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c13f0a0 0x7fc32c141490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient(hunting): picked mon.noname-i con 0x7fc32c13f0a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc32c0a83e0 con 0x7fc32c1396a0 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c0a4f20 con 0x7fc32c13a190 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc32c0a2f30 con 0x7fc32c13f0a0 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33236e640 10 monclient(hunting): authenticate will time out at 1298.165899s 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c13f0a0 0x7fc32c141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc32bfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc32c1396a0 0x7fc32c139a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:51654/0 (socket says 172.21.15.190:51654) 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc32bfff640 1 -- 172.21.15.190:0/805657442 learned_addr learned my addr 172.21.15.190:0/805657442 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 10 monclient(hunting): get_auth_request con 0x7fc32c13f0a0 auth_method 0 2024-08-19T10:44:12.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc32b7fe640 1 --2- 172.21.15.190:0/805657442 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc32c13a190 0x7fc32c13eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc32b7fe640 10 monclient(hunting): get_auth_request con 0x7fc32c13a190 auth_method 0 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc32b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc32b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc32b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc32b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:12.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc32b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc32b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 10 monclient(hunting): handle_auth_done global_id 15280 payload 274 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 1 -- 172.21.15.190:0/805657442 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc32c13a190 msgr2=0x7fc32c13eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 1 --2- 172.21.15.190:0/805657442 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc32c13a190 0x7fc32c13eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 1 -- 172.21.15.190:0/805657442 >> v1:172.21.15.89:6789/0 conn(0x7fc32c1396a0 legacy=0x7fc32c139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 10 monclient: _finish_hunting 0 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 1 monclient: found mon.noname-i 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.594+0000 7fc33090b640 1 -- 172.21.15.190:0/805657442 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c0a35e0 con 0x7fc32c13f0a0 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33090b640 1 -- 172.21.15.190:0/805657442 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c13f0a0 msgr2=0x7fc32c141490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:12.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33090b640 1 --2- 172.21.15.190:0/805657442 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c13f0a0 0x7fc32c141490 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc324009dc0 tx=0x7fc32401c850 comp rx=0 tx=0).ready entity=mon.7 client_cookie=b1b667dbb5e21b6d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 1 -- 172.21.15.190:0/805657442 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc3240ac070 con 0x7fc32c13f0a0 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 10 monclient: dump: 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:12.606 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 10 monclient: _finish_auth 0 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 1 -- 172.21.15.190:0/805657442 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3240a68b0 con 0x7fc32c13f0a0 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33236e640 5 monclient: authenticate success, global_id 15280 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33236e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:12.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33236e640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 1 -- 172.21.15.190:0/805657442 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc3240a7420 con 0x7fc32c13f0a0 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33236e640 4 set_mon_vals no callback set 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc32affd640 10 monclient: dump: 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:12.608 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33236e640 10 monclient: shutdown 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33236e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33236e640 1 -- 172.21.15.190:0/805657442 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c13f0a0 msgr2=0x7fc32c141490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33236e640 1 --2- 172.21.15.190:0/805657442 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c13f0a0 0x7fc32c141490 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc324009dc0 tx=0x7fc32401c850 comp rx=0 tx=0).stop 2024-08-19T10:44:12.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33236e640 1 -- 172.21.15.190:0/805657442 shutdown_connections 2024-08-19T10:44:12.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.598+0000 7fc33236e640 1 --2- 172.21.15.190:0/805657442 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c13f0a0 0x7fc32c141490 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:12.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 --2- 172.21.15.190:0/805657442 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc32c13a190 0x7fc32c13eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:12.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 -- 172.21.15.190:0/805657442 >> 172.21.15.190:0/805657442 conn(0x7fc32c0abee0 msgr2=0x7fc32c0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:12.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 -- 172.21.15.190:0/805657442 shutdown_connections 2024-08-19T10:44:12.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 -- 172.21.15.190:0/805657442 wait complete. 2024-08-19T10:44:12.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient: build_initial_monmap 2024-08-19T10:44:12.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient: monmap: 2024-08-19T10:44:12.614 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:12.614 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:12.614 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:12.614 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:12.615 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 Processor -- start 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 -- start start 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient: init 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient: _add_conns ranks=[7,3,1,6,5,8,4,2,0] 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c1396a0 0x7fc32c1d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient(hunting): picked mon.noname-i con 0x7fc32c1396a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc32c13a190 0x7fc32c1d1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient(hunting): picked mon.noname-g con 0x7fc32c13a190 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c13f0a0 0x7fc32c1d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient(hunting): picked mon.noname-f con 0x7fc32c13f0a0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:12.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c1441f0 con 0x7fc32c13f0a0 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c0a78f0 con 0x7fc32c13a190 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc32c144370 con 0x7fc32c1396a0 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33236e640 10 monclient(hunting): authenticate will time out at 1298.173555s 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33090b640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c13f0a0 msgr2=0x7fc32c1d1d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc33090b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c13f0a0 0x7fc32c1d1d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c1396a0 0x7fc32c1d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c1396a0 0x7fc32c1d12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:46946/0 (socket says 172.21.15.190:46946) 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 1 -- 172.21.15.190:0/3121507874 learned_addr learned my addr 172.21.15.190:0/3121507874 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 10 monclient(hunting): get_auth_request con 0x7fc32c1396a0 auth_method 0 2024-08-19T10:44:12.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32b7fe640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc32c13a190 0x7fc32c1d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32b7fe640 10 monclient(hunting): get_auth_request con 0x7fc32c13a190 auth_method 0 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:12.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 10 monclient(hunting): handle_auth_done global_id 15289 payload 824 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 1 -- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc32c13a190 msgr2=0x7fc32c1d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc32c13a190 0x7fc32c1d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 1 -- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c13f0a0 msgr2=0x7fc32c1d1d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c13f0a0 0x7fc32c1d1d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 1 monclient: found mon.noname-i 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.602+0000 7fc32bfff640 1 -- 172.21.15.190:0/3121507874 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c13e830 con 0x7fc32c1396a0 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc32bfff640 1 -- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c1396a0 msgr2=0x7fc32c1d12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:12.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc32bfff640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c1396a0 0x7fc32c1d12f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc32000c470 tx=0x7fc32000c930 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 1 -- 172.21.15.190:0/3121507874 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc320005d00 con 0x7fc32c1396a0 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 10 monclient: dump: 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:12.620 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 1 -- 172.21.15.190:0/3121507874 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3200074e0 con 0x7fc32c1396a0 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 1 -- 172.21.15.190:0/3121507874 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc320007690 con 0x7fc32c1396a0 2024-08-19T10:44:12.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc33236e640 5 monclient: authenticate success, global_id 15289 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc32a7fc640 4 set_mon_vals no callback set 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 10 monclient: dump: 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:12.622 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc33236e640 10 monclient: _renew_subs 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc33236e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc33236e640 1 -- 172.21.15.190:0/3121507874 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c09e2a0 con 0x7fc32c1396a0 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc33236e640 10 monclient: _renew_subs 2024-08-19T10:44:12.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc33236e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc33236e640 1 -- 172.21.15.190:0/3121507874 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c13e150 con 0x7fc32c1396a0 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 1 -- 172.21.15.190:0/3121507874 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc320007070 con 0x7fc32c1396a0 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc33236e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc33236e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc33236e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc33236e640 1 -- 172.21.15.190:0/3121507874 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32c139a80 con 0x7fc32c1396a0 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc31803be30 0x7fc31803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc328ff9640 1 -- 172.21.15.190:0/3121507874 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(203..203 src has 1..203) ==== 9619+0+0 (secure 0 0 0) 0x7fc32004fa00 con 0x7fc32c1396a0 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc32b7fe640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc31803be30 0x7fc31803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.606+0000 7fc32b7fe640 10 monclient: get_auth_request con 0x7fc31803be30 auth_method 0 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.610+0000 7fc32b7fe640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc31803be30 0x7fc31803e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc32c13de60 tx=0x7fc31c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.610+0000 7fc328ff9640 1 -- 172.21.15.190:0/3121507874 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc32004d100 con 0x7fc32c1396a0 2024-08-19T10:44:12.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.610+0000 7fc328ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:12.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.610+0000 7fc328ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:12.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.782+0000 7fc33236e640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:12.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.782+0000 7fc33236e640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:12.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.782+0000 7fc33236e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:12.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:12.782+0000 7fc33236e640 1 -- 172.21.15.190:0/3121507874 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fc32c09d670 con 0x7fc32c1396a0 2024-08-19T10:44:13.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.354+0000 7fc328ff9640 1 -- 172.21.15.190:0/3121507874 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}]=0 added bucket h9 type host to location {rack=r1} v204) ==== 166+0+0 (secure 0 0 0) 0x7fc320010d10 con 0x7fc32c1396a0 2024-08-19T10:44:13.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.354+0000 7fc328ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:13.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.354+0000 7fc328ff9640 10 monclient: _finish_command 2 = system:0 added bucket h9 type host to location {rack=r1} 2024-08-19T10:44:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.458+0000 7fc33236e640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.458+0000 7fc33236e640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:13.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.458+0000 7fc33236e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:13.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.458+0000 7fc33236e640 1 -- 172.21.15.190:0/3121507874 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fc32c0008e0 con 0x7fc32c1396a0 2024-08-19T10:44:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.458+0000 7fc328ff9640 1 -- 172.21.15.190:0/3121507874 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h9' already exists v204) ==== 145+0+0 (secure 0 0 0) 0x7fc320012400 con 0x7fc32c1396a0 2024-08-19T10:44:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.458+0000 7fc328ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h9", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.458+0000 7fc328ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h9' already exists 2024-08-19T10:44:13.461 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h9' already exists 2024-08-19T10:44:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 -- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc31803be30 msgr2=0x7fc31803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc31803be30 0x7fc31803e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc32c13de60 tx=0x7fc31c009190 comp rx=0 tx=0).stop 2024-08-19T10:44:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 10 monclient: shutdown 2024-08-19T10:44:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 -- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c1396a0 msgr2=0x7fc32c1d12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c1396a0 0x7fc32c1d12f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc32000c470 tx=0x7fc32000c930 comp rx=0 tx=0).stop 2024-08-19T10:44:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 -- 172.21.15.190:0/3121507874 shutdown_connections 2024-08-19T10:44:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc31803be30 0x7fc31803e2f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc32c13f0a0 0x7fc32c1d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:13.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc32c13a190 0x7fc32c1d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 --2- 172.21.15.190:0/3121507874 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc32c1396a0 0x7fc32c1d12f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 -- 172.21.15.190:0/3121507874 >> 172.21.15.190:0/3121507874 conn(0x7fc32c0abee0 msgr2=0x7fc32c134860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 -- 172.21.15.190:0/3121507874 shutdown_connections 2024-08-19T10:44:13.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.462+0000 7fc33236e640 1 -- 172.21.15.190:0/3121507874 wait complete. 2024-08-19T10:44:13.478 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:13.479 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h10 host rack=r1 2024-08-19T10:44:13.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient: get_monmap_and_config 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient: build_initial_monmap 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient: monmap: 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:13.600499+0000 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:13.600499+0000 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:13.600 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:13.601 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:13.601 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:13.601 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:13.601 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:13.601 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:13.601 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:13.601 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:13.601 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:13.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 1 Processor -- start 2024-08-19T10:44:13.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 1 -- start start 2024-08-19T10:44:13.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient: init 2024-08-19T10:44:13.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:13.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient: _add_conns ranks=[2,0,3,1,8,9,10,4,7,6,5] 2024-08-19T10:44:13.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61440a3d80 0x7f614409cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:13.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient(hunting): picked mon.noname-b con 0x7f61440a3d80 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:13.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f61440a4160 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:13.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f614413c920 0x7f614413ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:13.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient(hunting): picked mon.noname-g con 0x7f614413c920 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:13.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6144069c10 con 0x7f61440a4160 2024-08-19T10:44:13.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:13.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f614406d760 con 0x7f61440a3d80 2024-08-19T10:44:13.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:13.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6144069770 con 0x7f614413c920 2024-08-19T10:44:13.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:13.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f614939d640 10 monclient(hunting): authenticate will time out at 1299.171158s 2024-08-19T10:44:13.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f61437fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f614413c920 0x7f614413ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:13.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f61437fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f614413c920 0x7f614413ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:60066/0 (socket says 172.21.15.190:60066) 2024-08-19T10:44:13.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f61437fe640 1 -- 172.21.15.190:0/1341349333 learned_addr learned my addr 172.21.15.190:0/1341349333 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:13.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f61437fe640 10 monclient(hunting): get_auth_request con 0x7f614413c920 auth_method 0 2024-08-19T10:44:13.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f61437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:13.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f61437fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f61437fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f61437fe640 1 -- 172.21.15.190:0/1341349333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f614413c920 msgr2=0x7f614413ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f61437fe640 1 -- 172.21.15.190:0/1341349333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f614413c920 msgr2=0x7f614413ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f61437fe640 1 --2- 172.21.15.190:0/1341349333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f614413c920 0x7f614413ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f61437fe640 1 --2- 172.21.15.190:0/1341349333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f614413c920 0x7f614413ed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f6142ffd640 1 --2- 172.21.15.190:0/1341349333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61440a3d80 0x7f614409cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f6142ffd640 10 monclient(hunting): get_auth_request con 0x7f61440a3d80 auth_method 0 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f6142ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f6142ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f6142ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f6142ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f6142ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:13.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.598+0000 7f6142ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:13.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.602+0000 7f6142ffd640 10 monclient(hunting): handle_auth_done global_id 5996 payload 274 2024-08-19T10:44:13.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.602+0000 7f6142ffd640 1 -- 172.21.15.190:0/1341349333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f614413c920 msgr2=0x7f614413ed10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:13.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.602+0000 7f6142ffd640 1 --2- 172.21.15.190:0/1341349333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f614413c920 0x7f614413ed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:13.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.602+0000 7f6142ffd640 1 -- 172.21.15.190:0/1341349333 >> v1:172.21.15.89:6789/0 conn(0x7f61440a4160 legacy=0x7f61440a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:13.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.602+0000 7f6142ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:13.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.602+0000 7f6142ffd640 1 monclient: found mon.noname-b 2024-08-19T10:44:13.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.602+0000 7f6142ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:13.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.602+0000 7f6142ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:44:13.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.602+0000 7f6142ffd640 1 -- 172.21.15.190:0/1341349333 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614409c5f0 con 0x7f61440a3d80 2024-08-19T10:44:13.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6142ffd640 1 -- 172.21.15.190:0/1341349333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61440a3d80 msgr2=0x7f614409cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:13.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6142ffd640 1 --2- 172.21.15.190:0/1341349333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61440a3d80 0x7f614409cfd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f61340097f0 tx=0x7f613401c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36ac05d054b41b26 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:13.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 1 -- 172.21.15.190:0/1341349333 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61340b0070 con 0x7f61440a3d80 2024-08-19T10:44:13.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:13.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:44:13.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 10 monclient: dump: 2024-08-19T10:44:13.611 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:13.611 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:13.611 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:13.611 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:13.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 1 -- 172.21.15.190:0/1341349333 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61340aa810 con 0x7f61440a3d80 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 1 -- 172.21.15.190:0/1341349333 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61340a9300 con 0x7f61440a3d80 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6141ffb640 10 monclient: dump: 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:13.613 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 5 monclient: authenticate success, global_id 5996 2024-08-19T10:44:13.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 4 set_mon_vals no callback set 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient: shutdown 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 -- 172.21.15.190:0/1341349333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61440a3d80 msgr2=0x7f614409cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 --2- 172.21.15.190:0/1341349333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61440a3d80 0x7f614409cfd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f61340097f0 tx=0x7f613401c360 comp rx=0 tx=0).stop 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 -- 172.21.15.190:0/1341349333 shutdown_connections 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 --2- 172.21.15.190:0/1341349333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f614413c920 0x7f614413ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 --2- 172.21.15.190:0/1341349333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f61440a3d80 0x7f614409cfd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 -- 172.21.15.190:0/1341349333 >> 172.21.15.190:0/1341349333 conn(0x7f61440a4e70 msgr2=0x7f61440a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 -- 172.21.15.190:0/1341349333 shutdown_connections 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 -- 172.21.15.190:0/1341349333 wait complete. 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient: build_initial_monmap 2024-08-19T10:44:13.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient: monmap: 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:13.616 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 Processor -- start 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 -- start start 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient: init 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient: _add_conns ranks=[3,7,5,4,0,6,1,8,2] 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61440a3d80 0x7f6144137950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient(hunting): picked mon.noname-g con 0x7f61440a3d80 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f61440a4160 0x7f6144137e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient(hunting): picked mon.noname-i con 0x7f61440a4160 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f614413c920 0x7f6144072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:13.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient(hunting): picked mon.noname-h con 0x7f614413c920 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6144141a70 con 0x7f61440a3d80 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f61440a1fd0 con 0x7f614413c920 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6144141bf0 con 0x7f61440a4160 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f614939d640 10 monclient(hunting): authenticate will time out at 1299.179716s 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6142ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61440a3d80 0x7f6144137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f61437fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f614413c920 0x7f6144072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6142ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61440a3d80 0x7f6144137950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:60078/0 (socket says 172.21.15.190:60078) 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6142ffd640 1 -- 172.21.15.190:0/3063057446 learned_addr learned my addr 172.21.15.190:0/3063057446 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:13.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6142ffd640 10 monclient(hunting): get_auth_request con 0x7f61440a3d80 auth_method 0 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6142ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6142ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f6142ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f61437fe640 10 monclient(hunting): get_auth_request con 0x7f614413c920 auth_method 0 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f61437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f61437fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f61437fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f61427fc640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f61440a4160 0x7f6144137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f61427fc640 10 monclient(hunting): get_auth_request con 0x7f61440a4160 auth_method 0 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f61427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f61427fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.606+0000 7f61427fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f61437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f61437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f61437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f61427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f61427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f61427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 10 monclient(hunting): handle_auth_done global_id 34383 payload 824 2024-08-19T10:44:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 1 -- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f61440a4160 msgr2=0x7f6144137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:13.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f61440a4160 0x7f6144137e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:13.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 1 -- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f614413c920 msgr2=0x7f6144072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:13.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f614413c920 0x7f6144072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:13.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:13.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 1 monclient: found mon.noname-g 2024-08-19T10:44:13.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:13.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:13.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 1 -- 172.21.15.190:0/3063057446 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614413ef40 con 0x7f61440a3d80 2024-08-19T10:44:13.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f61437fe640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f614413c920 0x7f6144072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 1 -- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61440a3d80 msgr2=0x7f6144137950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f6142ffd640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61440a3d80 0x7f6144137950 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f61340ae040 tx=0x7f6134019100 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 1 -- 172.21.15.190:0/3063057446 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f61340b0040 con 0x7f61440a3d80 2024-08-19T10:44:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 10 monclient: dump: 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 1 -- 172.21.15.190:0/3063057446 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61340044b0 con 0x7f61440a3d80 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 1 -- 172.21.15.190:0/3063057446 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6134003a90 con 0x7f61440a3d80 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f612b7fe640 10 monclient: dump: 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f61417fa640 4 set_mon_vals no callback set 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f614939d640 5 monclient: authenticate success, global_id 34383 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f614939d640 10 monclient: _renew_subs 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f614939d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f614939d640 1 -- 172.21.15.190:0/3063057446 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6144074dc0 con 0x7f61440a3d80 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f614939d640 10 monclient: _renew_subs 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f614939d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.610+0000 7f614939d640 1 -- 172.21.15.190:0/3063057446 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f61440a1130 con 0x7f61440a3d80 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.614+0000 7f612b7fe640 1 -- 172.21.15.190:0/3063057446 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6134003620 con 0x7f61440a3d80 2024-08-19T10:44:13.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.614+0000 7f614939d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:13.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.614+0000 7f614939d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:13.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.614+0000 7f614939d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:13.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.614+0000 7f614939d640 1 -- 172.21.15.190:0/3063057446 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614409cfd0 con 0x7f61440a3d80 2024-08-19T10:44:13.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.614+0000 7f612b7fe640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f613003be30 0x7f613003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:13.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.614+0000 7f612b7fe640 1 -- 172.21.15.190:0/3063057446 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(204..204 src has 1..204) ==== 9829+0+0 (secure 0 0 0) 0x7f61340e9c40 con 0x7f61440a3d80 2024-08-19T10:44:13.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.614+0000 7f61427fc640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f613003be30 0x7f613003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:13.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.614+0000 7f61427fc640 10 monclient: get_auth_request con 0x7f613003be30 auth_method 0 2024-08-19T10:44:13.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.614+0000 7f61427fc640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f613003be30 0x7f613003e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f614409fe80 tx=0x7f6138007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:13.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.618+0000 7f612b7fe640 1 -- 172.21.15.190:0/3063057446 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6134016070 con 0x7f61440a3d80 2024-08-19T10:44:13.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.618+0000 7f612b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:13.628 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.618+0000 7f612b7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.790+0000 7f614939d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:13.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.790+0000 7f614939d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.790+0000 7f614939d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:13.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:13.790+0000 7f614939d640 1 -- 172.21.15.190:0/3063057446 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f614406e490 con 0x7f61440a3d80 2024-08-19T10:44:14.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.358+0000 7f612b7fe640 1 -- 172.21.15.190:0/3063057446 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}]=0 added bucket h10 type host to location {rack=r1} v205) ==== 168+0+0 (secure 0 0 0) 0x7f61340aa7e0 con 0x7f61440a3d80 2024-08-19T10:44:14.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.358+0000 7f612b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:14.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.358+0000 7f612b7fe640 10 monclient: _finish_command 2 = system:0 added bucket h10 type host to location {rack=r1} 2024-08-19T10:44:14.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.466+0000 7f614939d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:14.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.466+0000 7f614939d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:14.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.466+0000 7f614939d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:14.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.466+0000 7f614939d640 1 -- 172.21.15.190:0/3063057446 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f614409fef0 con 0x7f61440a3d80 2024-08-19T10:44:14.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.466+0000 7f612b7fe640 1 -- 172.21.15.190:0/3063057446 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h10' already exists v205) ==== 147+0+0 (secure 0 0 0) 0x7f61340e7270 con 0x7f61440a3d80 2024-08-19T10:44:14.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.466+0000 7f612b7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h10", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:14.469 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.466+0000 7f612b7fe640 10 monclient: _finish_command 3 = system:0 bucket 'h10' already exists 2024-08-19T10:44:14.469 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h10' already exists 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 -- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f613003be30 msgr2=0x7f613003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f613003be30 0x7f613003e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f614409fe80 tx=0x7f6138007e40 comp rx=0 tx=0).stop 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 10 monclient: shutdown 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 -- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61440a3d80 msgr2=0x7f6144137950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61440a3d80 0x7f6144137950 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f61340ae040 tx=0x7f6134019100 comp rx=0 tx=0).stop 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 -- 172.21.15.190:0/3063057446 shutdown_connections 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f613003be30 0x7f613003e2f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f614413c920 0x7f6144072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f61440a4160 0x7f6144137e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 --2- 172.21.15.190:0/3063057446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61440a3d80 0x7f6144137950 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:14.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 -- 172.21.15.190:0/3063057446 >> 172.21.15.190:0/3063057446 conn(0x7f61440a4e70 msgr2=0x7f614413bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 -- 172.21.15.190:0/3063057446 shutdown_connections 2024-08-19T10:44:14.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.470+0000 7f614939d640 1 -- 172.21.15.190:0/3063057446 wait complete. 2024-08-19T10:44:14.486 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:14.487 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h11 host rack=r1 2024-08-19T10:44:14.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.606+0000 7fc4edd2b640 10 monclient: get_monmap_and_config 2024-08-19T10:44:14.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.606+0000 7fc4edd2b640 10 monclient: build_initial_monmap 2024-08-19T10:44:14.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.606+0000 7fc4edd2b640 10 monclient: monmap: 2024-08-19T10:44:14.609 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:14.609 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:14.609 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:14.610131+0000 2024-08-19T10:44:14.609 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:14.610131+0000 2024-08-19T10:44:14.609 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:14.609 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:14.609 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:14.609 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:14.609 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:14.610 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:14.610 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:14.610 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:14.610 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:14.610 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:14.610 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:14.610 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:14.610 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:14.610 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:14.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.606+0000 7fc4edd2b640 1 Processor -- start 2024-08-19T10:44:14.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.606+0000 7fc4edd2b640 1 -- start start 2024-08-19T10:44:14.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.606+0000 7fc4edd2b640 10 monclient: init 2024-08-19T10:44:14.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:14.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 10 monclient: _add_conns ranks=[3,4,8,10,0,2,1,7,6,5,9] 2024-08-19T10:44:14.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4e806c8d0 0x7fc4e806ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:14.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 10 monclient(hunting): picked mon.noname-g con 0x7fc4e806c8d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:14.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4e809fae0 0x7fc4e806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:14.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 10 monclient(hunting): picked mon.noname-c con 0x7fc4e809fae0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:14.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4e8139660 0x7fc4e8139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:14.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 10 monclient(hunting): picked mon.noname-e con 0x7fc4e8139660 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:14.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:14.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e809b590 con 0x7fc4e806c8d0 2024-08-19T10:44:14.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:14.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc4e8069d90 con 0x7fc4e809fae0 2024-08-19T10:44:14.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:14.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc4e8069a60 con 0x7fc4e8139660 2024-08-19T10:44:14.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:14.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4edd2b640 10 monclient(hunting): authenticate will time out at 1300.180522s 2024-08-19T10:44:14.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4e806c8d0 0x7fc4e806ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4e806c8d0 0x7fc4e806ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:60084/0 (socket says 172.21.15.190:60084) 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4dffff640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4e809fae0 msgr2=0x7fc4e806d1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 1 -- 172.21.15.190:0/2556376595 learned_addr learned my addr 172.21.15.190:0/2556376595 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 10 monclient(hunting): get_auth_request con 0x7fc4e806c8d0 auth_method 0 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4dffff640 1 --2- 172.21.15.190:0/2556376595 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4e809fae0 0x7fc4e806d1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e6ffd640 1 -- 172.21.15.190:0/2556376595 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e7fff640 1 --2- 172.21.15.190:0/2556376595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4e8139660 0x7fc4e8139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:14.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e7fff640 10 monclient(hunting): get_auth_request con 0x7fc4e8139660 auth_method 0 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 10 monclient(hunting): handle_auth_done global_id 34392 payload 274 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 1 -- 172.21.15.190:0/2556376595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4e8139660 msgr2=0x7fc4e8139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 1 --2- 172.21.15.190:0/2556376595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4e8139660 0x7fc4e8139a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 1 -- 172.21.15.190:0/2556376595 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4e809fae0 msgr2=0x7fc4e806d1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 1 --2- 172.21.15.190:0/2556376595 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4e809fae0 0x7fc4e806d1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 1 monclient: found mon.noname-g 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:14.616 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.610+0000 7fc4e77fe640 1 -- 172.21.15.190:0/2556376595 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e806daf0 con 0x7fc4e806c8d0 2024-08-19T10:44:14.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e77fe640 1 -- 172.21.15.190:0/2556376595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4e806c8d0 msgr2=0x7fc4e806ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:14.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e77fe640 1 --2- 172.21.15.190:0/2556376595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4e806c8d0 0x7fc4e806ccb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80098a0 tx=0x7fc4d801c460 comp rx=0 tx=0).ready entity=mon.3 client_cookie=4b5d8dc328971745 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:14.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 1 -- 172.21.15.190:0/2556376595 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4d80b0070 con 0x7fc4e806c8d0 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 10 monclient: dump: 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:14.623 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:14.624 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:14.624 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:14.624 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:14.624 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:14.624 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:14.624 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:14.624 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:14.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:14.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:14.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 1 -- 172.21.15.190:0/2556376595 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d80aa810 con 0x7fc4e806c8d0 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 1 -- 172.21.15.190:0/2556376595 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4d80a9300 con 0x7fc4e806c8d0 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e6ffd640 10 monclient: dump: 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:14.625 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 5 monclient: authenticate success, global_id 34392 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 4 set_mon_vals no callback set 2024-08-19T10:44:14.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient: shutdown 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 -- 172.21.15.190:0/2556376595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4e806c8d0 msgr2=0x7fc4e806ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 --2- 172.21.15.190:0/2556376595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4e806c8d0 0x7fc4e806ccb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80098a0 tx=0x7fc4d801c460 comp rx=0 tx=0).stop 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 -- 172.21.15.190:0/2556376595 shutdown_connections 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 --2- 172.21.15.190:0/2556376595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc4e8139660 0x7fc4e8139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 --2- 172.21.15.190:0/2556376595 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4e809fae0 0x7fc4e806d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 --2- 172.21.15.190:0/2556376595 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4e806c8d0 0x7fc4e806ccb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 -- 172.21.15.190:0/2556376595 >> 172.21.15.190:0/2556376595 conn(0x7fc4e80a8450 msgr2=0x7fc4e80a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 -- 172.21.15.190:0/2556376595 shutdown_connections 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 -- 172.21.15.190:0/2556376595 wait complete. 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient: build_initial_monmap 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient: monmap: 2024-08-19T10:44:14.627 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:14.628 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:14.628 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:14.628 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:14.628 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:14.628 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:14.628 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:14.628 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:14.628 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:14.628 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:14.628 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 Processor -- start 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 -- start start 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient: init 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient: _add_conns ranks=[2,0,5,8,7,3,6,4,1] 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4e806c8d0 0x7fc4e81d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient(hunting): picked mon.noname-b con 0x7fc4e806c8d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:14.629 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e809fae0 0x7fc4e809a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient(hunting): picked mon.noname-a con 0x7fc4e809fae0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc4e8139660 0x7fc4e809ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient(hunting): picked mon.noname-h con 0x7fc4e8139660 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e8143520 con 0x7fc4e809fae0 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e81433a0 con 0x7fc4e806c8d0 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc4e81436a0 con 0x7fc4e8139660 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4edd2b640 10 monclient(hunting): authenticate will time out at 1300.187760s 2024-08-19T10:44:14.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e7fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc4e8139660 0x7fc4e809ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e7fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc4e8139660 0x7fc4e809ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:48264/0 (socket says 172.21.15.190:48264) 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e7fff640 1 -- 172.21.15.190:0/3113853448 learned_addr learned my addr 172.21.15.190:0/3113853448 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e7fff640 10 monclient(hunting): get_auth_request con 0x7fc4e8139660 auth_method 0 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.614+0000 7fc4e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4dffff640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e809fae0 0x7fc4e809a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e77fe640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4e806c8d0 0x7fc4e81d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e77fe640 10 monclient(hunting): get_auth_request con 0x7fc4e806c8d0 auth_method 0 2024-08-19T10:44:14.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4dffff640 10 monclient(hunting): get_auth_request con 0x7fc4e809fae0 auth_method 0 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4dffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4dffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 10 monclient(hunting): handle_auth_done global_id 6026 payload 824 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 1 -- 172.21.15.190:0/3113853448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4e806c8d0 msgr2=0x7fc4e81d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4e806c8d0 0x7fc4e81d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 1 -- 172.21.15.190:0/3113853448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e809fae0 msgr2=0x7fc4e809a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e809fae0 0x7fc4e809a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:14.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 1 monclient: found mon.noname-h 2024-08-19T10:44:14.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:14.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:14.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 1 -- 172.21.15.190:0/3113853448 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e8077940 con 0x7fc4e8139660 2024-08-19T10:44:14.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 1 -- 172.21.15.190:0/3113853448 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc4e8139660 msgr2=0x7fc4e809ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:14.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e7fff640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc4e8139660 0x7fc4e809ac30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009fd0 tx=0x7fc4e000e4b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:14.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 1 -- 172.21.15.190:0/3113853448 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4e0014890 con 0x7fc4e8139660 2024-08-19T10:44:14.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:14.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:14.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 10 monclient: dump: 2024-08-19T10:44:14.634 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:14.634 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:14.634 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:14.634 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:14.634 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:14.634 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:14.634 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:14.634 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:14.634 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:14.634 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:14.635 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:14.635 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:14.635 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:14.635 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:14.635 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:14.635 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:14.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:14.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:14.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:14.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 1 -- 172.21.15.190:0/3113853448 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e0019070 con 0x7fc4e8139660 2024-08-19T10:44:14.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:14.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 1 -- 172.21.15.190:0/3113853448 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4e0013380 con 0x7fc4e8139660 2024-08-19T10:44:14.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:14.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:14.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e4ff9640 10 monclient: dump: 2024-08-19T10:44:14.636 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:14.636 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:14.637 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4e67fc640 4 set_mon_vals no callback set 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4edd2b640 5 monclient: authenticate success, global_id 6026 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4edd2b640 10 monclient: _renew_subs 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4edd2b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4edd2b640 1 -- 172.21.15.190:0/3113853448 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e813f050 con 0x7fc4e8139660 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4edd2b640 10 monclient: _renew_subs 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4edd2b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.618+0000 7fc4edd2b640 1 -- 172.21.15.190:0/3113853448 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e813d5d0 con 0x7fc4e8139660 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.622+0000 7fc4edd2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.622+0000 7fc4edd2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.622+0000 7fc4edd2b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:14.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.622+0000 7fc4edd2b640 1 -- 172.21.15.190:0/3113853448 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e806de00 con 0x7fc4e8139660 2024-08-19T10:44:14.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.622+0000 7fc4e4ff9640 1 -- 172.21.15.190:0/3113853448 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc4e0014420 con 0x7fc4e8139660 2024-08-19T10:44:14.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.622+0000 7fc4e4ff9640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4c803be30 0x7fc4c803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:14.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.622+0000 7fc4e4ff9640 1 -- 172.21.15.190:0/3113853448 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(205..205 src has 1..205) ==== 10042+0+0 (secure 0 0 0) 0x7fc4e00507a0 con 0x7fc4e8139660 2024-08-19T10:44:14.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.622+0000 7fc4e77fe640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4c803be30 0x7fc4c803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:14.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.622+0000 7fc4e77fe640 10 monclient: get_auth_request con 0x7fc4c803be30 auth_method 0 2024-08-19T10:44:14.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.622+0000 7fc4e77fe640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4c803be30 0x7fc4c803e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009870 tx=0x7fc4d80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:14.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.626+0000 7fc4e4ff9640 1 -- 172.21.15.190:0/3113853448 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc4e0014d20 con 0x7fc4e8139660 2024-08-19T10:44:14.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.626+0000 7fc4e4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:14.641 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.626+0000 7fc4e4ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:14.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.798+0000 7fc4edd2b640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:14.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.798+0000 7fc4edd2b640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:14.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.798+0000 7fc4edd2b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:14.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:14.798+0000 7fc4edd2b640 1 -- 172.21.15.190:0/3113853448 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fc4e8139a80 con 0x7fc4e8139660 2024-08-19T10:44:15.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:15.914+0000 7fc4e4ff9640 1 -- 172.21.15.190:0/3113853448 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}]=0 added bucket h11 type host to location {rack=r1} v206) ==== 168+0+0 (secure 0 0 0) 0x7fc4e004dd00 con 0x7fc4e8139660 2024-08-19T10:44:15.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:15.914+0000 7fc4e4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:15.918 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:15.914+0000 7fc4e4ff9640 10 monclient: _finish_command 2 = system:0 added bucket h11 type host to location {rack=r1} 2024-08-19T10:44:16.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.026+0000 7fc4edd2b640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:16.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.026+0000 7fc4edd2b640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:16.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.026+0000 7fc4edd2b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:16.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.026+0000 7fc4edd2b640 1 -- 172.21.15.190:0/3113853448 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fc4e806e470 con 0x7fc4e8139660 2024-08-19T10:44:16.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.026+0000 7fc4e4ff9640 1 -- 172.21.15.190:0/3113853448 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h11' already exists v206) ==== 147+0+0 (secure 0 0 0) 0x7fc4e0013520 con 0x7fc4e8139660 2024-08-19T10:44:16.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.026+0000 7fc4e4ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h11", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:16.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.026+0000 7fc4e4ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h11' already exists 2024-08-19T10:44:16.030 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h11' already exists 2024-08-19T10:44:16.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 -- 172.21.15.190:0/3113853448 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4c803be30 msgr2=0x7fc4c803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:16.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4c803be30 0x7fc4c803e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009870 tx=0x7fc4d80a6000 comp rx=0 tx=0).stop 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 10 monclient: shutdown 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 -- 172.21.15.190:0/3113853448 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc4e8139660 msgr2=0x7fc4e809ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc4e8139660 0x7fc4e809ac30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009fd0 tx=0x7fc4e000e4b0 comp rx=0 tx=0).stop 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 -- 172.21.15.190:0/3113853448 shutdown_connections 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc4c803be30 0x7fc4c803e2f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc4e8139660 0x7fc4e809ac30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e809fae0 0x7fc4e809a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 --2- 172.21.15.190:0/3113853448 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4e806c8d0 0x7fc4e81d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 -- 172.21.15.190:0/3113853448 >> 172.21.15.190:0/3113853448 conn(0x7fc4e80a8450 msgr2=0x7fc4e80a2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 -- 172.21.15.190:0/3113853448 shutdown_connections 2024-08-19T10:44:16.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.030+0000 7fc4edd2b640 1 -- 172.21.15.190:0/3113853448 wait complete. 2024-08-19T10:44:16.048 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:16.048 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h12 host rack=r1 2024-08-19T10:44:16.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.166+0000 7fd7efbf0640 10 monclient: get_monmap_and_config 2024-08-19T10:44:16.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.166+0000 7fd7efbf0640 10 monclient: build_initial_monmap 2024-08-19T10:44:16.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 10 monclient: monmap: 2024-08-19T10:44:16.170 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:16.170 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:16.171591+0000 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:16.171591+0000 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:16.171 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:16.172 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:16.172 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:16.172 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:16.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 1 Processor -- start 2024-08-19T10:44:16.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 1 -- start start 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 10 monclient: init 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 10 monclient: _add_conns ranks=[8,1,6,2,0,7,3,4,9,5,10] 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7e80a69c0 0x7fd7e80a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 10 monclient(hunting): picked mon.noname-e con 0x7fd7e80a69c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd7e80a6da0 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7e813ebd0 0x7fd7e8140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 10 monclient(hunting): picked mon.noname-d con 0x7fd7e813ebd0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7e80a4eb0 con 0x7fd7e80a6da0 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd7e80a5c30 con 0x7fd7e813ebd0 2024-08-19T10:44:16.175 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd7e80a3570 con 0x7fd7e80a69c0 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7efbf0640 10 monclient(hunting): authenticate will time out at 1301.741588s 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed18b640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fd7e80a6da0 legacy=0x7fd7e80a7180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ee18d640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7e813ebd0 msgr2=0x7fd7e8140fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ee18d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7e813ebd0 0x7fd7e8140fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7e80a69c0 0x7fd7e80a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7e80a69c0 0x7fd7e80a56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:38376/0 (socket says 172.21.15.190:38376) 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 1 -- 172.21.15.190:0/3349973277 learned_addr learned my addr 172.21.15.190:0/3349973277 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 10 monclient(hunting): get_auth_request con 0x7fd7e80a69c0 auth_method 0 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:16.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 10 monclient(hunting): handle_auth_done global_id 5867 payload 274 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 1 -- 172.21.15.190:0/3349973277 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7e813ebd0 msgr2=0x7fd7e8140fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 1 --2- 172.21.15.190:0/3349973277 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7e813ebd0 0x7fd7e8140fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 1 -- 172.21.15.190:0/3349973277 >> v1:172.21.15.190:6789/0 conn(0x7fd7e80a6da0 legacy=0x7fd7e80a7180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 10 monclient: _finish_hunting 0 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 1 monclient: found mon.noname-e 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:16.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.170+0000 7fd7ed98c640 1 -- 172.21.15.190:0/3349973277 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e80a6330 con 0x7fd7e80a69c0 2024-08-19T10:44:16.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ed98c640 1 -- 172.21.15.190:0/3349973277 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7e80a69c0 msgr2=0x7fd7e80a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:16.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ed98c640 1 --2- 172.21.15.190:0/3349973277 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7e80a69c0 0x7fd7e80a56f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40098a0 tx=0x7fd7e401c400 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c27ea26a05ba88a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:16.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 1 -- 172.21.15.190:0/3349973277 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7e40b0070 con 0x7fd7e80a69c0 2024-08-19T10:44:16.181 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 10 monclient: dump: 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:16.182 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 10 monclient: _finish_auth 0 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 10 monclient: _check_auth_tickets 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7efbf0640 5 monclient: authenticate success, global_id 5867 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7efbf0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 1 -- 172.21.15.190:0/3349973277 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e40aa810 con 0x7fd7e80a69c0 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:16.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7efbf0640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7efbf0640 4 set_mon_vals no callback set 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 1 -- 172.21.15.190:0/3349973277 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7e40a93e0 con 0x7fd7e80a69c0 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.174+0000 7fd7ec98a640 10 monclient: dump: 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:16.184 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient: shutdown 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 -- 172.21.15.190:0/3349973277 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7e80a69c0 msgr2=0x7fd7e80a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 --2- 172.21.15.190:0/3349973277 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7e80a69c0 0x7fd7e80a56f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40098a0 tx=0x7fd7e401c400 comp rx=0 tx=0).stop 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 -- 172.21.15.190:0/3349973277 shutdown_connections 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 --2- 172.21.15.190:0/3349973277 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7e813ebd0 0x7fd7e8140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:16.185 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 --2- 172.21.15.190:0/3349973277 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd7e80a69c0 0x7fd7e80a56f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 -- 172.21.15.190:0/3349973277 >> 172.21.15.190:0/3349973277 conn(0x7fd7e80abfc0 msgr2=0x7fd7e80ac3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 -- 172.21.15.190:0/3349973277 shutdown_connections 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 -- 172.21.15.190:0/3349973277 wait complete. 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient: build_initial_monmap 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient: monmap: 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:16.186 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 Processor -- start 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 -- start start 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient: init 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient: _add_conns ranks=[7,1,4,3,8,0,5,6,2] 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7e80a69c0 0x7fd7e8124040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:16.187 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient(hunting): picked mon.noname-i con 0x7fd7e80a69c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7e80a6da0 0x7fd7e8126030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient(hunting): picked mon.noname-f con 0x7fd7e80a6da0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7e813ebd0 0x7fd7e813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient(hunting): picked mon.noname-c con 0x7fd7e813ebd0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e8143ea0 con 0x7fd7e80a6da0 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd7e8143d20 con 0x7fd7e813ebd0 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd7e8144020 con 0x7fd7e80a69c0 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7efbf0640 10 monclient(hunting): authenticate will time out at 1301.750068s 2024-08-19T10:44:16.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed18b640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7e80a6da0 msgr2=0x7fd7e8126030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed18b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7e80a6da0 0x7fd7e8126030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7e80a69c0 0x7fd7e8124040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7e80a69c0 0x7fd7e8124040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:51150/0 (socket says 172.21.15.190:51150) 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 1 -- 172.21.15.190:0/2197088455 learned_addr learned my addr 172.21.15.190:0/2197088455 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 10 monclient(hunting): get_auth_request con 0x7fd7e80a69c0 auth_method 0 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ee18d640 1 -- 172.21.15.190:0/2197088455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7e813ebd0 msgr2=0x7fd7e813cea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ee18d640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7e813ebd0 0x7fd7e813cea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:16.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 10 monclient(hunting): handle_auth_done global_id 15307 payload 824 2024-08-19T10:44:16.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 1 -- 172.21.15.190:0/2197088455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7e813ebd0 msgr2=0x7fd7e813cea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:16.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7e813ebd0 0x7fd7e813cea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:16.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 1 -- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7e80a6da0 msgr2=0x7fd7e8126030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:16.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7e80a6da0 0x7fd7e8126030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:16.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 10 monclient: _finish_hunting 0 2024-08-19T10:44:16.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 1 monclient: found mon.noname-i 2024-08-19T10:44:16.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:16.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:16.190 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.178+0000 7fd7ed98c640 1 -- 172.21.15.190:0/2197088455 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e81254d0 con 0x7fd7e80a69c0 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7ed98c640 1 -- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7e80a69c0 msgr2=0x7fd7e8124040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7ed98c640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7e80a69c0 0x7fd7e8124040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40ae040 tx=0x7fd7e4019200 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 1 -- 172.21.15.190:0/2197088455 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7e40b0040 con 0x7fd7e80a69c0 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 10 monclient: dump: 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:16.193 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:16.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 1 -- 172.21.15.190:0/2197088455 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e40044a0 con 0x7fd7e80a69c0 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 1 -- 172.21.15.190:0/2197088455 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7e4003a90 con 0x7fd7e80a69c0 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 10 monclient: dump: 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:16.195 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7dffff640 4 set_mon_vals no callback set 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7efbf0640 5 monclient: authenticate success, global_id 15307 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7efbf0640 10 monclient: _renew_subs 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7efbf0640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7efbf0640 1 -- 172.21.15.190:0/2197088455 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e80ac4c0 con 0x7fd7e80a69c0 2024-08-19T10:44:16.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7efbf0640 10 monclient: _renew_subs 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7efbf0640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7efbf0640 1 -- 172.21.15.190:0/2197088455 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e8126f90 con 0x7fd7e80a69c0 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7efbf0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7efbf0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7efbf0640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7efbf0640 1 -- 172.21.15.190:0/2197088455 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e80a56f0 con 0x7fd7e80a69c0 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7ed98c640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7e80a69c0 0x7fd7e8124040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40ae040 tx=0x7fd7e4019200 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fd7e40bb060 mgrmap(e 5) 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 1 -- 172.21.15.190:0/2197088455 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd7e40bb060 con 0x7fd7e80a69c0 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7de7fc640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7cc03be00 0x7fd7cc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7ed18b640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7cc03be00 0x7fd7cc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.182+0000 7fd7ed18b640 10 monclient: get_auth_request con 0x7fd7cc03be00 auth_method 0 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.186+0000 7fd7ed18b640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7cc03be00 0x7fd7cc03e2c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd7e81251e0 tx=0x7fd7d8006cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:16.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.186+0000 7fd7de7fc640 1 -- 172.21.15.190:0/2197088455 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(206..206 src has 1..206) ==== 10255+0+0 (secure 0 0 0) 0x7fd7e40e9300 con 0x7fd7e80a69c0 2024-08-19T10:44:16.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.806+0000 7fd7de7fc640 1 -- 172.21.15.190:0/2197088455 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd7e4016170 con 0x7fd7e80a69c0 2024-08-19T10:44:16.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.806+0000 7fd7de7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:16.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.806+0000 7fd7de7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:16.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.994+0000 7fd7efbf0640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:16.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.994+0000 7fd7efbf0640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:16.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.994+0000 7fd7efbf0640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:16.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:16.994+0000 7fd7efbf0640 1 -- 172.21.15.190:0/2197088455 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fd7e81271d0 con 0x7fd7e80a69c0 2024-08-19T10:44:18.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.406+0000 7fd7de7fc640 1 -- 172.21.15.190:0/2197088455 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}]=0 added bucket h12 type host to location {rack=r1} v207) ==== 168+0+0 (secure 0 0 0) 0x7fd7e40aa7e0 con 0x7fd7e80a69c0 2024-08-19T10:44:18.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.406+0000 7fd7de7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:18.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.406+0000 7fd7de7fc640 10 monclient: _finish_command 2 = system:0 added bucket h12 type host to location {rack=r1} 2024-08-19T10:44:18.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.514+0000 7fd7efbf0640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:18.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.514+0000 7fd7efbf0640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:18.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.514+0000 7fd7efbf0640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:18.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.514+0000 7fd7efbf0640 1 -- 172.21.15.190:0/2197088455 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fd7e80008e0 con 0x7fd7e80a69c0 2024-08-19T10:44:18.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7de7fc640 1 -- 172.21.15.190:0/2197088455 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h12' already exists v207) ==== 147+0+0 (secure 0 0 0) 0x7fd7e4003c30 con 0x7fd7e80a69c0 2024-08-19T10:44:18.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7de7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h12", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:18.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7de7fc640 10 monclient: _finish_command 3 = system:0 bucket 'h12' already exists 2024-08-19T10:44:18.519 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h12' already exists 2024-08-19T10:44:18.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 1 -- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7cc03be00 msgr2=0x7fd7cc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:18.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7cc03be00 0x7fd7cc03e2c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd7e81251e0 tx=0x7fd7d8006cf0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 10 monclient: shutdown 2024-08-19T10:44:18.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:18.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 1 -- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7e80a69c0 msgr2=0x7fd7e8124040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:18.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7e80a69c0 0x7fd7e8124040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40ae040 tx=0x7fd7e4019200 comp rx=0 tx=0).stop 2024-08-19T10:44:18.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 1 -- 172.21.15.190:0/2197088455 shutdown_connections 2024-08-19T10:44:18.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7cc03be00 0x7fd7cc03e2c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd7e813ebd0 0x7fd7e813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7e80a6da0 0x7fd7e8126030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 1 --2- 172.21.15.190:0/2197088455 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd7e80a69c0 0x7fd7e8124040 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.518+0000 7fd7efbf0640 1 -- 172.21.15.190:0/2197088455 >> 172.21.15.190:0/2197088455 conn(0x7fd7e80abfc0 msgr2=0x7fd7e8140120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:18.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.522+0000 7fd7efbf0640 1 -- 172.21.15.190:0/2197088455 shutdown_connections 2024-08-19T10:44:18.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.522+0000 7fd7efbf0640 1 -- 172.21.15.190:0/2197088455 wait complete. 2024-08-19T10:44:18.538 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:18.539 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h13 host rack=r1 2024-08-19T10:44:18.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient: get_monmap_and_config 2024-08-19T10:44:18.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient: build_initial_monmap 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient: monmap: 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:18.660013+0000 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:18.660013+0000 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:18.659 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:18.660 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:18.660 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:18.660 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:18.660 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:18.660 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:18.660 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:18.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 1 Processor -- start 2024-08-19T10:44:18.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 1 -- start start 2024-08-19T10:44:18.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient: init 2024-08-19T10:44:18.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient: _add_conns ranks=[4,5,2,8,3,7,1,0,6,9,10] 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8acc1396a0 0x7f8acc139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient(hunting): picked mon.noname-c con 0x7f8acc1396a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient(hunting): picked mon.noname-h con 0x7f8acc139fc0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8acc0a6e20 0x7f8acc0a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient(hunting): picked mon.noname-b con 0x7f8acc0a6e20 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc0a4f20 con 0x7f8acc0a6e20 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f8acc0a83e0 con 0x7f8acc1396a0 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:18.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f8acc0a2f30 con 0x7f8acc139fc0 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8ad1ab3640 10 monclient(hunting): authenticate will time out at 1304.229912s 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acb7fe640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8acc1396a0 msgr2=0x7f8acc139a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:41466/0 (socket says 172.21.15.190:41466) 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 1 -- 172.21.15.190:0/3726346279 learned_addr learned my addr 172.21.15.190:0/3726346279 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 10 monclient(hunting): get_auth_request con 0x7f8acc139fc0 auth_method 0 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acb7fe640 1 --2- 172.21.15.190:0/3726346279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8acc1396a0 0x7f8acc139a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:18.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acbfff640 1 --2- 172.21.15.190:0/3726346279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8acc0a6e20 0x7f8acc0a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acbfff640 10 monclient(hunting): get_auth_request con 0x7f8acc0a6e20 auth_method 0 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 10 monclient(hunting): handle_auth_done global_id 6035 payload 274 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 1 -- 172.21.15.190:0/3726346279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8acc1396a0 msgr2=0x7f8acc139a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 1 --2- 172.21.15.190:0/3726346279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8acc1396a0 0x7f8acc139a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 1 -- 172.21.15.190:0/3726346279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8acc0a6e20 msgr2=0x7f8acc0a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 1 --2- 172.21.15.190:0/3726346279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8acc0a6e20 0x7f8acc0a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:18.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 1 monclient: found mon.noname-h 2024-08-19T10:44:18.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:18.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:18.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acaffd640 1 -- 172.21.15.190:0/3726346279 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc0a35e0 con 0x7f8acc139fc0 2024-08-19T10:44:18.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.658+0000 7f8acbfff640 1 --2- 172.21.15.190:0/3726346279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8acc0a6e20 0x7f8acc0a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:44:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acaffd640 1 -- 172.21.15.190:0/3726346279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 msgr2=0x7f8acc0a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acaffd640 1 --2- 172.21.15.190:0/3726346279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a68e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ac00098d0 tx=0x7f8ac001c460 comp rx=0 tx=0).ready entity=mon.5 client_cookie=6527a15f0509f277 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 1 -- 172.21.15.190:0/3726346279 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ac00b0070 con 0x7f8acc139fc0 2024-08-19T10:44:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 10 monclient: dump: 2024-08-19T10:44:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:18.668 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:18.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 1 -- 172.21.15.190:0/3726346279 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac00aa810 con 0x7f8acc139fc0 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 5 monclient: authenticate success, global_id 6035 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 1 -- 172.21.15.190:0/3726346279 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ac00a9300 con 0x7f8acc139fc0 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 4 set_mon_vals no callback set 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8aca7fc640 10 monclient: dump: 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:18.670 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:18.671 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient: shutdown 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3726346279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 msgr2=0x7f8acc0a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 --2- 172.21.15.190:0/3726346279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a68e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ac00098d0 tx=0x7f8ac001c460 comp rx=0 tx=0).stop 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3726346279 shutdown_connections 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 --2- 172.21.15.190:0/3726346279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8acc0a6e20 0x7f8acc0a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 --2- 172.21.15.190:0/3726346279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a68e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 --2- 172.21.15.190:0/3726346279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8acc1396a0 0x7f8acc139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3726346279 >> 172.21.15.190:0/3726346279 conn(0x7f8acc0abee0 msgr2=0x7f8acc0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3726346279 shutdown_connections 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3726346279 wait complete. 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient: build_initial_monmap 2024-08-19T10:44:18.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient: monmap: 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:18.673 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:18.674 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:18.674 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 Processor -- start 2024-08-19T10:44:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 -- start start 2024-08-19T10:44:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient: init 2024-08-19T10:44:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient: _add_conns ranks=[1,2,5,7,4,0,3,8,6] 2024-08-19T10:44:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8acc0a6e20 0x7f8acc1d1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient(hunting): picked mon.noname-f con 0x7f8acc0a6e20 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8acc1396a0 0x7f8acc0a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient(hunting): picked mon.noname-b con 0x7f8acc1396a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:18.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient(hunting): picked mon.noname-h con 0x7f8acc139fc0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc13a3a0 con 0x7f8acc0a6e20 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc143390 con 0x7f8acc1396a0 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acb7fe640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8acc0a6e20 msgr2=0x7f8acc1d1890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f8acc0a2a90 con 0x7f8acc139fc0 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8ad1ab3640 10 monclient(hunting): authenticate will time out at 1304.239261s 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8acc0a6e20 0x7f8acc1d1890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:41480/0 (socket says 172.21.15.190:41480) 2024-08-19T10:44:18.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 1 -- 172.21.15.190:0/3049380279 learned_addr learned my addr 172.21.15.190:0/3049380279 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 10 monclient(hunting): get_auth_request con 0x7f8acc139fc0 auth_method 0 2024-08-19T10:44:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acaffd640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8acc1396a0 0x7f8acc0a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acaffd640 10 monclient(hunting): get_auth_request con 0x7f8acc1396a0 auth_method 0 2024-08-19T10:44:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:18.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 10 monclient(hunting): handle_auth_done global_id 6044 payload 824 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 1 -- 172.21.15.190:0/3049380279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8acc1396a0 msgr2=0x7f8acc0a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8acc1396a0 0x7f8acc0a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 1 -- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8acc0a6e20 msgr2=0x7f8acc1d1890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8acc0a6e20 0x7f8acc1d1890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 1 monclient: found mon.noname-h 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.666+0000 7f8acbfff640 1 -- 172.21.15.190:0/3049380279 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc135730 con 0x7f8acc139fc0 2024-08-19T10:44:18.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8acbfff640 1 -- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 msgr2=0x7f8acc0a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8acbfff640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a2550 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4007f20 tx=0x7f8ac400d560 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 1 -- 172.21.15.190:0/3049380279 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ac401a070 con 0x7f8acc139fc0 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 10 monclient: dump: 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:18.682 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 1 -- 172.21.15.190:0/3049380279 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac40129d0 con 0x7f8acc139fc0 2024-08-19T10:44:18.683 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 1 -- 172.21.15.190:0/3049380279 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ac40143a0 con 0x7f8acc139fc0 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ad1ab3640 5 monclient: authenticate success, global_id 6044 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ad1ab3640 10 monclient: _renew_subs 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ad1ab3640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3049380279 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc0ac570 con 0x7f8acc139fc0 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ac9ffb640 4 set_mon_vals no callback set 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 10 monclient: dump: 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:18.684 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ad1ab3640 10 monclient: _renew_subs 2024-08-19T10:44:18.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ad1ab3640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3049380279 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc0a0800 con 0x7f8acc139fc0 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 1 -- 172.21.15.190:0/3049380279 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8ac4012560 con 0x7f8acc139fc0 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ad1ab3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ad1ab3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ad1ab3640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3049380279 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8acc139a80 con 0x7f8acc139fc0 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ab803bd30 0x7f8ab803e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8ab3fff640 1 -- 172.21.15.190:0/3049380279 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(207..207 src has 1..207) ==== 10468+0+0 (secure 0 0 0) 0x7f8ac4050370 con 0x7f8acc139fc0 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8acb7fe640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ab803bd30 0x7f8ab803e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.670+0000 7f8acb7fe640 10 monclient: get_auth_request con 0x7f8ab803bd30 auth_method 0 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.674+0000 7f8acb7fe640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ab803bd30 0x7f8ab803e1f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8abc006fd0 tx=0x7f8abc006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.674+0000 7f8ab3fff640 1 -- 172.21.15.190:0/3049380279 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:44:18.686 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.678+0000 7f8ab3fff640 1 -- 172.21.15.190:0/3049380279 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8acc135730 con 0x7f8acc139fc0 2024-08-19T10:44:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.678+0000 7f8ab3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:18.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.678+0000 7f8ab3fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:18.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.846+0000 7f8ad1ab3640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:18.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.846+0000 7f8ad1ab3640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:18.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.846+0000 7f8ad1ab3640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:18.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:18.846+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3049380279 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f8acc09f2e0 con 0x7f8acc139fc0 2024-08-19T10:44:19.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.566+0000 7f8ab3fff640 1 -- 172.21.15.190:0/3049380279 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}]=0 added bucket h13 type host to location {rack=r1} v208) ==== 168+0+0 (secure 0 0 0) 0x7f8ac4010070 con 0x7f8acc139fc0 2024-08-19T10:44:19.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.566+0000 7f8ab3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:19.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.566+0000 7f8ab3fff640 10 monclient: _finish_command 2 = system:0 added bucket h13 type host to location {rack=r1} 2024-08-19T10:44:19.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.674+0000 7f8ad1ab3640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:19.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.674+0000 7f8ad1ab3640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:19.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.674+0000 7f8ad1ab3640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:19.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.674+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3049380279 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f8acc0ac7a0 con 0x7f8acc139fc0 2024-08-19T10:44:19.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ab3fff640 1 -- 172.21.15.190:0/3049380279 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h13' already exists v208) ==== 147+0+0 (secure 0 0 0) 0x7f8ac4013300 con 0x7f8acc139fc0 2024-08-19T10:44:19.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ab3fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h13", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:19.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ab3fff640 10 monclient: _finish_command 3 = system:0 bucket 'h13' already exists 2024-08-19T10:44:19.678 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h13' already exists 2024-08-19T10:44:19.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ab803bd30 msgr2=0x7f8ab803e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ab803bd30 0x7f8ab803e1f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8abc006fd0 tx=0x7f8abc006d40 comp rx=0 tx=0).stop 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 10 monclient: shutdown 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 msgr2=0x7f8acc0a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a2550 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4007f20 tx=0x7f8ac400d560 comp rx=0 tx=0).stop 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3049380279 shutdown_connections 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8ab803bd30 0x7f8ab803e1f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8acc139fc0 0x7f8acc0a2550 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8acc1396a0 0x7f8acc0a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 --2- 172.21.15.190:0/3049380279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8acc0a6e20 0x7f8acc1d1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3049380279 >> 172.21.15.190:0/3049380279 conn(0x7f8acc0abee0 msgr2=0x7f8acc0ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:19.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3049380279 shutdown_connections 2024-08-19T10:44:19.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.678+0000 7f8ad1ab3640 1 -- 172.21.15.190:0/3049380279 wait complete. 2024-08-19T10:44:19.696 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:19.696 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h14 host rack=r1 2024-08-19T10:44:19.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient: get_monmap_and_config 2024-08-19T10:44:19.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient: build_initial_monmap 2024-08-19T10:44:19.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient: monmap: 2024-08-19T10:44:19.818 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:19.818 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:19.818 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:19.819672+0000 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:19.819672+0000 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:19.819 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:19.820 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:19.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 1 Processor -- start 2024-08-19T10:44:19.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 1 -- start start 2024-08-19T10:44:19.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient: init 2024-08-19T10:44:19.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:19.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient: _add_conns ranks=[5,0,2,9,10,8,3,4,6,7,1] 2024-08-19T10:44:19.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6fcc0a47c0 0x7f6fcc0a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:19.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient(hunting): picked mon.noname-h con 0x7f6fcc0a47c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:19.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6fcc0a2f90 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:19.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fcc0a50e0 0x7f6fcc136cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:19.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient(hunting): picked mon.noname-b con 0x7f6fcc0a50e0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:19.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fcc0a09a0 con 0x7f6fcc0a2f90 2024-08-19T10:44:19.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:19.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc0a25f0 con 0x7f6fcc0a50e0 2024-08-19T10:44:19.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:19.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6fcc0a0c20 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:19.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd331d640 10 monclient(hunting): authenticate will time out at 1305.389486s 2024-08-19T10:44:19.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6fcc0a47c0 0x7f6fcc0a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:19.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6fcc0a47c0 0x7f6fcc0a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:41490/0 (socket says 172.21.15.190:41490) 2024-08-19T10:44:19.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 1 -- 172.21.15.190:0/1455415802 learned_addr learned my addr 172.21.15.190:0/1455415802 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:19.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 10 monclient(hunting): get_auth_request con 0x7f6fcc0a47c0 auth_method 0 2024-08-19T10:44:19.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:19.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:19.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:19.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:19.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:19.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:19.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd18ba640 1 --2- 172.21.15.190:0/1455415802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fcc0a50e0 0x7f6fcc136cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:19.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd18ba640 10 monclient(hunting): get_auth_request con 0x7f6fcc0a50e0 auth_method 0 2024-08-19T10:44:19.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd18ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:19.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd18ba640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:19.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd18ba640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:19.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 10 monclient(hunting): handle_auth_done global_id 6053 payload 274 2024-08-19T10:44:19.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 1 -- 172.21.15.190:0/1455415802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fcc0a50e0 msgr2=0x7f6fcc136cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:19.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 1 --2- 172.21.15.190:0/1455415802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fcc0a50e0 0x7f6fcc136cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:19.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 1 -- 172.21.15.190:0/1455415802 >> v1:172.21.15.89:6789/0 conn(0x7f6fcc0a2f90 legacy=0x7f6fcc1365e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:19.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 10 monclient: _finish_hunting 0 2024-08-19T10:44:19.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 1 monclient: found mon.noname-h 2024-08-19T10:44:19.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:19.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:19.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.818+0000 7f6fd10b9640 1 -- 172.21.15.190:0/1455415802 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc0a1cc0 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd10b9640 1 -- 172.21.15.190:0/1455415802 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6fcc0a47c0 msgr2=0x7f6fcc0a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:19.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd10b9640 1 --2- 172.21.15.190:0/1455415802 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6fcc0a47c0 0x7f6fcc0a4ba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009e80 tx=0x7f6fc401c910 comp rx=0 tx=0).ready entity=mon.5 client_cookie=6388ed4ad7036560 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:19.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 1 -- 172.21.15.190:0/1455415802 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6fc40090d0 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:19.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:19.830 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 10 monclient: dump: 2024-08-19T10:44:19.830 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:19.830 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:19.830 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:19.831 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 5 monclient: authenticate success, global_id 6053 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 1 -- 172.21.15.190:0/1455415802 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc40a5b40 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 4 set_mon_vals no callback set 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 1 -- 172.21.15.190:0/1455415802 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6fc40a5cf0 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:19.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fc3fff640 10 monclient: dump: 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:19.833 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 10 monclient: shutdown 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 1 -- 172.21.15.190:0/1455415802 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6fcc0a47c0 msgr2=0x7f6fcc0a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 1 --2- 172.21.15.190:0/1455415802 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6fcc0a47c0 0x7f6fcc0a4ba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009e80 tx=0x7f6fc401c910 comp rx=0 tx=0).stop 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 1 -- 172.21.15.190:0/1455415802 shutdown_connections 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 1 --2- 172.21.15.190:0/1455415802 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fcc0a50e0 0x7f6fcc136cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 1 --2- 172.21.15.190:0/1455415802 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6fcc0a47c0 0x7f6fcc0a4ba0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 1 -- 172.21.15.190:0/1455415802 >> 172.21.15.190:0/1455415802 conn(0x7f6fcc06dfe0 msgr2=0x7f6fcc0a58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:19.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 1 -- 172.21.15.190:0/1455415802 shutdown_connections 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 1 -- 172.21.15.190:0/1455415802 wait complete. 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 10 monclient: build_initial_monmap 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.822+0000 7f6fd331d640 10 monclient: monmap: 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:19.835 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 1 Processor -- start 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 1 -- start start 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient: init 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient: _add_conns ranks=[2,3,1,8,6,0,4,5,7] 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fcc0a2f90 0x7f6fcc13a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient(hunting): picked mon.noname-b con 0x7f6fcc0a2f90 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:19.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc0a47c0 0x7f6fcc13f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient(hunting): picked mon.noname-g con 0x7f6fcc0a47c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc0a50e0 0x7f6fcc13fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient(hunting): picked mon.noname-f con 0x7f6fcc0a50e0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc137230 con 0x7f6fcc0a50e0 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc0a62b0 con 0x7f6fcc0a2f90 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc1373b0 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient(hunting): authenticate will time out at 1305.396738s 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd18ba640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc0a50e0 msgr2=0x7f6fcc13fe00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:19.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd18ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc0a50e0 0x7f6fcc13fe00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc0a47c0 0x7f6fcc13f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc0a47c0 0x7f6fcc13f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42190/0 (socket says 172.21.15.190:42190) 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 1 -- 172.21.15.190:0/3096455529 learned_addr learned my addr 172.21.15.190:0/3096455529 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 10 monclient(hunting): get_auth_request con 0x7f6fcc0a47c0 auth_method 0 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd10b9640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fcc0a2f90 0x7f6fcc13a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd10b9640 10 monclient(hunting): get_auth_request con 0x7f6fcc0a2f90 auth_method 0 2024-08-19T10:44:19.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd10b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd10b9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd10b9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 10 monclient(hunting): handle_auth_done global_id 34401 payload 824 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 1 -- 172.21.15.190:0/3096455529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fcc0a2f90 msgr2=0x7f6fcc13a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fcc0a2f90 0x7f6fcc13a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 1 -- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc0a50e0 msgr2=0x7f6fcc13fe00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc0a50e0 0x7f6fcc13fe00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 10 monclient: _finish_hunting 0 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 1 monclient: found mon.noname-g 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 1 -- 172.21.15.190:0/3096455529 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc13b1c0 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 1 -- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc0a47c0 msgr2=0x7f6fcc13f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd08b8640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc0a47c0 0x7f6fcc13f8c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00c410 tx=0x7f6fbc00c8d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 1 -- 172.21.15.190:0/3096455529 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6fbc011c10 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 10 monclient: dump: 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:19.840 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:19.841 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:19.841 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:19.841 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:19.841 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:19.841 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:19.841 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:19.841 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:19.841 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:19.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:19.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 1 -- 172.21.15.190:0/3096455529 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fbc017070 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 5 monclient: authenticate success, global_id 34401 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient: _renew_subs 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 1 -- 172.21.15.190:0/3096455529 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fcc13ced0 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 1 -- 172.21.15.190:0/3096455529 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6fbc011e30 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc37fe640 4 set_mon_vals no callback set 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fc1ffb640 10 monclient: dump: 2024-08-19T10:44:19.842 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:19.843 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:19.844 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:19.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient: _renew_subs 2024-08-19T10:44:19.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:19.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.826+0000 7f6fd331d640 1 -- 172.21.15.190:0/3096455529 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fcc0a6df0 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.830+0000 7f6fc1ffb640 1 -- 172.21.15.190:0/3096455529 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6fbc0117a0 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.830+0000 7f6fd331d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:19.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.830+0000 7f6fd331d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:19.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.830+0000 7f6fd331d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:19.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.830+0000 7f6fd331d640 1 -- 172.21.15.190:0/3096455529 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fcc06a0b0 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.830+0000 7f6fc1ffb640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6fb003be00 0x7f6fb003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:19.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.830+0000 7f6fc1ffb640 1 -- 172.21.15.190:0/3096455529 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(208..208 src has 1..208) ==== 10681+0+0 (secure 0 0 0) 0x7f6fbc04fe00 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.830+0000 7f6fd10b9640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6fb003be00 0x7f6fb003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:19.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.830+0000 7f6fd10b9640 10 monclient: get_auth_request con 0x7f6fb003be00 auth_method 0 2024-08-19T10:44:19.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.830+0000 7f6fd10b9640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6fb003be00 0x7f6fb003e2c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0a6220 tx=0x7f6fc40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:19.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.834+0000 7f6fc1ffb640 1 -- 172.21.15.190:0/3096455529 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6fbc021020 con 0x7f6fcc0a47c0 2024-08-19T10:44:19.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.834+0000 7f6fc1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:19.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:19.834+0000 7f6fc1ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:20.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.010+0000 7f6fd331d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:20.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.010+0000 7f6fd331d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:20.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.010+0000 7f6fd331d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:20.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.010+0000 7f6fd331d640 1 -- 172.21.15.190:0/3096455529 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f6fcc0a8a80 con 0x7f6fcc0a47c0 2024-08-19T10:44:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.306+0000 7f6fc1ffb640 1 -- 172.21.15.190:0/3096455529 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}]=0 added bucket h14 type host to location {rack=r1} v209) ==== 168+0+0 (secure 0 0 0) 0x7f6fbc04d0e0 con 0x7f6fcc0a47c0 2024-08-19T10:44:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.306+0000 7f6fc1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.306+0000 7f6fc1ffb640 10 monclient: _finish_command 2 = system:0 added bucket h14 type host to location {rack=r1} 2024-08-19T10:44:20.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.418+0000 7f6fd331d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.418+0000 7f6fd331d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.418+0000 7f6fd331d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:20.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.418+0000 7f6fd331d640 1 -- 172.21.15.190:0/3096455529 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f6fcc0a4ba0 con 0x7f6fcc0a47c0 2024-08-19T10:44:20.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.418+0000 7f6fc1ffb640 1 -- 172.21.15.190:0/3096455529 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h14' already exists v209) ==== 147+0+0 (secure 0 0 0) 0x7f6fbc010cf0 con 0x7f6fcc0a47c0 2024-08-19T10:44:20.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.418+0000 7f6fc1ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h14", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:20.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.418+0000 7f6fc1ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h14' already exists 2024-08-19T10:44:20.420 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h14' already exists 2024-08-19T10:44:20.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 -- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6fb003be00 msgr2=0x7f6fb003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:20.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6fb003be00 0x7f6fb003e2c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0a6220 tx=0x7f6fc40a7070 comp rx=0 tx=0).stop 2024-08-19T10:44:20.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 10 monclient: shutdown 2024-08-19T10:44:20.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:20.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 -- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc0a47c0 msgr2=0x7f6fcc13f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:20.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc0a47c0 0x7f6fcc13f8c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00c410 tx=0x7f6fbc00c8d0 comp rx=0 tx=0).stop 2024-08-19T10:44:20.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 -- 172.21.15.190:0/3096455529 shutdown_connections 2024-08-19T10:44:20.423 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6fb003be00 0x7f6fb003e2c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:20.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6fcc0a50e0 0x7f6fcc13fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:20.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6fcc0a47c0 0x7f6fcc13f8c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:20.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 --2- 172.21.15.190:0/3096455529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6fcc0a2f90 0x7f6fcc13a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:20.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 -- 172.21.15.190:0/3096455529 >> 172.21.15.190:0/3096455529 conn(0x7f6fcc06dfe0 msgr2=0x7f6fcc06c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:20.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 -- 172.21.15.190:0/3096455529 shutdown_connections 2024-08-19T10:44:20.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.422+0000 7f6fd331d640 1 -- 172.21.15.190:0/3096455529 wait complete. 2024-08-19T10:44:20.439 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:20.439 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h15 host rack=r1 2024-08-19T10:44:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient: get_monmap_and_config 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient: build_initial_monmap 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient: monmap: 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:20.556162+0000 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:20.556162+0000 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:20.555 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:20.556 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:20.556 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:20.556 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:20.556 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:20.556 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:20.556 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:20.556 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:20.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 1 Processor -- start 2024-08-19T10:44:20.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 1 -- start start 2024-08-19T10:44:20.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient: init 2024-08-19T10:44:20.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient: _add_conns ranks=[6,0,7,5,1,2,8,3,10,9,4] 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a006c8d0 0x7f33a009d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient(hunting): picked mon.noname-d con 0x7f33a006c8d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f33a006ccb0 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33a013c8a0 0x7f33a013ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient(hunting): picked mon.noname-i con 0x7f33a013c8a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a00a28b0 con 0x7f33a006ccb0 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f33a00a56a0 con 0x7f33a006c8d0 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f33a009ff00 con 0x7f33a013c8a0 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:20.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33a013c8a0 0x7f33a013ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f33a5f4d640 10 monclient(hunting): authenticate will time out at 1306.126338s 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33a013c8a0 0x7f33a013ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:51152/0 (socket says 172.21.15.190:51152) 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339f7fe640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a006c8d0 msgr2=0x7f33a009d5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 1 -- 172.21.15.190:0/3711237563 learned_addr learned my addr 172.21.15.190:0/3711237563 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 10 monclient(hunting): get_auth_request con 0x7f33a013c8a0 auth_method 0 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339f7fe640 1 --2- 172.21.15.190:0/3711237563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a006c8d0 0x7f33a009d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339e7fc640 1 -- 172.21.15.190:0/3711237563 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:20.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 10 monclient(hunting): handle_auth_done global_id 15316 payload 274 2024-08-19T10:44:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 1 -- 172.21.15.190:0/3711237563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a006c8d0 msgr2=0x7f33a009d5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 1 --2- 172.21.15.190:0/3711237563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a006c8d0 0x7f33a009d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 1 -- 172.21.15.190:0/3711237563 >> v1:172.21.15.89:6789/0 conn(0x7f33a006ccb0 legacy=0x7f33a006d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 1 monclient: found mon.noname-i 2024-08-19T10:44:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:20.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.554+0000 7f339ffff640 1 -- 172.21.15.190:0/3711237563 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a00a3da0 con 0x7f33a013c8a0 2024-08-19T10:44:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339ffff640 1 -- 172.21.15.190:0/3711237563 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33a013c8a0 msgr2=0x7f33a013ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339ffff640 1 --2- 172.21.15.190:0/3711237563 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33a013c8a0 0x7f33a013ec90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3398009e80 tx=0x7f339801c930 comp rx=0 tx=0).ready entity=mon.7 client_cookie=39a2463bb2d9bfc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 1 -- 172.21.15.190:0/3711237563 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33980091b0 con 0x7f33a013c8a0 2024-08-19T10:44:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 10 monclient: dump: 2024-08-19T10:44:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:20.566 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:20.567 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 1 -- 172.21.15.190:0/3711237563 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33980a5a90 con 0x7f33a013c8a0 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f33a5f4d640 5 monclient: authenticate success, global_id 15316 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f33a5f4d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f33a5f4d640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f33a5f4d640 4 set_mon_vals no callback set 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 1 -- 172.21.15.190:0/3711237563 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33980a5c40 con 0x7f33a013c8a0 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:20.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f339e7fc640 10 monclient: dump: 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:20.569 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f33a5f4d640 10 monclient: shutdown 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f33a5f4d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f33a5f4d640 1 -- 172.21.15.190:0/3711237563 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33a013c8a0 msgr2=0x7f33a013ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f33a5f4d640 1 --2- 172.21.15.190:0/3711237563 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33a013c8a0 0x7f33a013ec90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3398009e80 tx=0x7f339801c930 comp rx=0 tx=0).stop 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f33a5f4d640 1 -- 172.21.15.190:0/3711237563 shutdown_connections 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f33a5f4d640 1 --2- 172.21.15.190:0/3711237563 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33a013c8a0 0x7f33a013ec90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.558+0000 7f33a5f4d640 1 --2- 172.21.15.190:0/3711237563 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a006c8d0 0x7f33a009d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 1 -- 172.21.15.190:0/3711237563 >> 172.21.15.190:0/3711237563 conn(0x7f33a006de10 msgr2=0x7f33a006e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 1 -- 172.21.15.190:0/3711237563 shutdown_connections 2024-08-19T10:44:20.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 1 -- 172.21.15.190:0/3711237563 wait complete. 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient: build_initial_monmap 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient: monmap: 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:20.571 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 1 Processor -- start 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 1 -- start start 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient: init 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient: _add_conns ranks=[0,3,6,1,2,8,5,4,7] 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33a006c8d0 0x7f33a01d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient(hunting): picked mon.noname-a con 0x7f33a006c8d0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a006ccb0 0x7f33a013b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:20.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient(hunting): picked mon.noname-g con 0x7f33a006ccb0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a013c8a0 0x7f33a013beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient(hunting): picked mon.noname-d con 0x7f33a013c8a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01419f0 con 0x7f33a006c8d0 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f33a009cc00 con 0x7f33a006ccb0 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f33a0141b70 con 0x7f33a013c8a0 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f33a5f4d640 10 monclient(hunting): authenticate will time out at 1306.133816s 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339ffff640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a013c8a0 msgr2=0x7f33a013beb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339ffff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a013c8a0 0x7f33a013beb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:20.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a006ccb0 0x7f33a013b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a006ccb0 0x7f33a013b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42194/0 (socket says 172.21.15.190:42194) 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 1 -- 172.21.15.190:0/2049666659 learned_addr learned my addr 172.21.15.190:0/2049666659 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 10 monclient(hunting): get_auth_request con 0x7f33a006ccb0 auth_method 0 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339f7fe640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33a006c8d0 0x7f33a01d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339f7fe640 10 monclient(hunting): get_auth_request con 0x7f33a006c8d0 auth_method 0 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:20.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 10 monclient(hunting): handle_auth_done global_id 34410 payload 824 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 1 -- 172.21.15.190:0/2049666659 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a013c8a0 msgr2=0x7f33a013beb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a013c8a0 0x7f33a013beb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 1 -- 172.21.15.190:0/2049666659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33a006c8d0 msgr2=0x7f33a01d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33a006c8d0 0x7f33a01d0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 1 monclient: found mon.noname-g 2024-08-19T10:44:20.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:20.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.562+0000 7f339effd640 1 -- 172.21.15.190:0/2049666659 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a0128320 con 0x7f33a006ccb0 2024-08-19T10:44:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f339effd640 1 -- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a006ccb0 msgr2=0x7f33a013b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f339effd640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a006ccb0 0x7f33a013b970 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f339000c410 tx=0x7f339000c8d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f339effd640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a006ccb0 0x7f33a013b970 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f339000c410 tx=0x7f339000c8d0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f3390011c10 mon_map magic: 0 2024-08-19T10:44:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f3387fff640 1 -- 172.21.15.190:0/2049666659 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3390011c10 con 0x7f33a006ccb0 2024-08-19T10:44:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f3387fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f3387fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f3387fff640 10 monclient: dump: 2024-08-19T10:44:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:20.578 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:20.579 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f3387fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f3387fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f3387fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f3387fff640 1 -- 172.21.15.190:0/2049666659 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3390017070 con 0x7f33a006ccb0 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f33a5f4d640 5 monclient: authenticate success, global_id 34410 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f33a5f4d640 10 monclient: _renew_subs 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f33a5f4d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f33a5f4d640 1 -- 172.21.15.190:0/2049666659 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a0125fc0 con 0x7f33a006ccb0 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f3387fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f339dffb640 4 set_mon_vals no callback set 2024-08-19T10:44:20.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f33a5f4d640 10 monclient: _renew_subs 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f33a5f4d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f33a5f4d640 1 -- 172.21.15.190:0/2049666659 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a0127d60 con 0x7f33a006ccb0 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f33a5f4d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f3387fff640 1 -- 172.21.15.190:0/2049666659 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3390011df0 con 0x7f33a006ccb0 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f33a5f4d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f33a5f4d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.566+0000 7f33a5f4d640 1 -- 172.21.15.190:0/2049666659 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a009d640 con 0x7f33a006ccb0 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.570+0000 7f3387fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.570+0000 7f3387fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.570+0000 7f3387fff640 10 monclient: dump: 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:20.581 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:20.582 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:20.583 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.570+0000 7f3387fff640 1 -- 172.21.15.190:0/2049666659 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f33900117a0 con 0x7f33a006ccb0 2024-08-19T10:44:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.570+0000 7f3387fff640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f338c03be50 0x7f338c03e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.570+0000 7f339f7fe640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f338c03be50 0x7f338c03e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.570+0000 7f339f7fe640 10 monclient: get_auth_request con 0x7f338c03be50 auth_method 0 2024-08-19T10:44:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.570+0000 7f339f7fe640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f338c03be50 0x7f338c03e310 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f33940093d0 tx=0x7f3394009360 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:20.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:20.574+0000 7f3387fff640 1 -- 172.21.15.190:0/2049666659 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(209..209 src has 1..209) ==== 10894+0+0 (secure 0 0 0) 0x7f33900502c0 con 0x7f33a006ccb0 2024-08-19T10:44:21.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:21.190+0000 7f3387fff640 1 -- 172.21.15.190:0/2049666659 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3390021020 con 0x7f33a006ccb0 2024-08-19T10:44:21.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:21.190+0000 7f3387fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:21.193 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:21.190+0000 7f3387fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:21.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:21.374+0000 7f33a5f4d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:21.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:21.374+0000 7f33a5f4d640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:21.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:21.374+0000 7f33a5f4d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:21.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:21.374+0000 7f33a5f4d640 1 -- 172.21.15.190:0/2049666659 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f33a0127f60 con 0x7f33a006ccb0 2024-08-19T10:44:22.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.326+0000 7f3387fff640 1 -- 172.21.15.190:0/2049666659 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}]=0 added bucket h15 type host to location {rack=r1} v210) ==== 168+0+0 (secure 0 0 0) 0x7f339004d4d0 con 0x7f33a006ccb0 2024-08-19T10:44:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.326+0000 7f3387fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:22.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.326+0000 7f3387fff640 10 monclient: _finish_command 2 = system:0 added bucket h15 type host to location {rack=r1} 2024-08-19T10:44:22.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.438+0000 7f33a5f4d640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:22.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.438+0000 7f33a5f4d640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:22.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.438+0000 7f33a5f4d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:22.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.438+0000 7f33a5f4d640 1 -- 172.21.15.190:0/2049666659 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f33a00008e0 con 0x7f33a006ccb0 2024-08-19T10:44:22.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.438+0000 7f3387fff640 1 -- 172.21.15.190:0/2049666659 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h15' already exists v210) ==== 147+0+0 (secure 0 0 0) 0x7f339001bc30 con 0x7f33a006ccb0 2024-08-19T10:44:22.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.438+0000 7f3387fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h15", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:22.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.438+0000 7f3387fff640 10 monclient: _finish_command 3 = system:0 bucket 'h15' already exists 2024-08-19T10:44:22.440 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h15' already exists 2024-08-19T10:44:22.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 -- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f338c03be50 msgr2=0x7f338c03e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:22.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f338c03be50 0x7f338c03e310 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f33940093d0 tx=0x7f3394009360 comp rx=0 tx=0).stop 2024-08-19T10:44:22.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 10 monclient: shutdown 2024-08-19T10:44:22.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:22.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 -- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a006ccb0 msgr2=0x7f33a013b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:22.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a006ccb0 0x7f33a013b970 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f339000c410 tx=0x7f339000c8d0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 -- 172.21.15.190:0/2049666659 shutdown_connections 2024-08-19T10:44:22.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f338c03be50 0x7f338c03e310 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f33a013c8a0 0x7f33a013beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33a006ccb0 0x7f33a013b970 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 --2- 172.21.15.190:0/2049666659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33a006c8d0 0x7f33a01d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 -- 172.21.15.190:0/2049666659 >> 172.21.15.190:0/2049666659 conn(0x7f33a006de10 msgr2=0x7f33a00a4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:22.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 -- 172.21.15.190:0/2049666659 shutdown_connections 2024-08-19T10:44:22.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.442+0000 7f33a5f4d640 1 -- 172.21.15.190:0/2049666659 wait complete. 2024-08-19T10:44:22.459 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:22.459 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h16 host rack=r1 2024-08-19T10:44:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient: get_monmap_and_config 2024-08-19T10:44:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient: build_initial_monmap 2024-08-19T10:44:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient: monmap: 2024-08-19T10:44:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:22.576308+0000 2024-08-19T10:44:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:22.576308+0000 2024-08-19T10:44:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:22.576 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:22.576 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:22.576 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:22.576 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:22.576 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:22.576 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:22.576 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:22.576 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:22.576 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:22.576 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 1 Processor -- start 2024-08-19T10:44:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 1 -- start start 2024-08-19T10:44:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient: init 2024-08-19T10:44:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient: _add_conns ranks=[6,9,5,7,4,8,0,1,10,2,3] 2024-08-19T10:44:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a46b0 0x7fac980a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient(hunting): picked mon.noname-d con 0x7fac980a46b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fac980a1d60 0x7fac980a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient(hunting): picked mon.noname-a con 0x7fac980a1d60 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac981399b0 0x7fac98139e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient(hunting): picked mon.noname-h con 0x7fac981399b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fac9806cb10 con 0x7fac980a1d60 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fac980a5510 con 0x7fac981399b0 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fac980a0ab0 con 0x7fac980a46b0 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac9dc1e640 10 monclient(hunting): authenticate will time out at 1308.146988s 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac977fe640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a46b0 msgr2=0x7fac980a4a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac977fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a46b0 0x7fac980a4a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac981399b0 0x7fac98139e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac981399b0 0x7fac98139e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:41498/0 (socket says 172.21.15.190:41498) 2024-08-19T10:44:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 1 -- 172.21.15.190:0/1793058974 learned_addr learned my addr 172.21.15.190:0/1793058974 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 10 monclient(hunting): get_auth_request con 0x7fac981399b0 auth_method 0 2024-08-19T10:44:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac96ffd640 1 --2- 172.21.15.190:0/1793058974 >> v2:172.21.15.89:3300/0 conn(0x7fac980a1d60 0x7fac980a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac96ffd640 10 monclient(hunting): get_auth_request con 0x7fac980a1d60 auth_method 0 2024-08-19T10:44:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac96ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac96ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac96ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 10 monclient(hunting): handle_auth_done global_id 6062 payload 274 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 1 -- 172.21.15.190:0/1793058974 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a46b0 msgr2=0x7fac980a4a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 1 --2- 172.21.15.190:0/1793058974 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a46b0 0x7fac980a4a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 1 -- 172.21.15.190:0/1793058974 >> v2:172.21.15.89:3300/0 conn(0x7fac980a1d60 msgr2=0x7fac980a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 1 --2- 172.21.15.190:0/1793058974 >> v2:172.21.15.89:3300/0 conn(0x7fac980a1d60 0x7fac980a4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 1 monclient: found mon.noname-h 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.574+0000 7fac97fff640 1 -- 172.21.15.190:0/1793058974 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac980a0d90 con 0x7fac981399b0 2024-08-19T10:44:22.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.578+0000 7fac97fff640 1 -- 172.21.15.190:0/1793058974 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac981399b0 msgr2=0x7fac98139e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:22.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.578+0000 7fac97fff640 1 --2- 172.21.15.190:0/1793058974 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac981399b0 0x7fac98139e60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fac90009e50 tx=0x7fac9001c8d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=e7e391e8fb762859 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:22.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 1 -- 172.21.15.190:0/1793058974 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac900090d0 con 0x7fac981399b0 2024-08-19T10:44:22.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:22.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:22.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 10 monclient: dump: 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 1 -- 172.21.15.190:0/1793058974 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac900a5be0 con 0x7fac981399b0 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 5 monclient: authenticate success, global_id 6062 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 1 -- 172.21.15.190:0/1793058974 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac900a5d90 con 0x7fac981399b0 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 4 set_mon_vals no callback set 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:22.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac967fc640 10 monclient: dump: 2024-08-19T10:44:22.589 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:22.589 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:22.589 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:22.589 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:22.589 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:22.589 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:22.589 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:22.589 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:22.589 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient: shutdown 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 -- 172.21.15.190:0/1793058974 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac981399b0 msgr2=0x7fac98139e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 --2- 172.21.15.190:0/1793058974 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac981399b0 0x7fac98139e60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fac90009e50 tx=0x7fac9001c8d0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 -- 172.21.15.190:0/1793058974 shutdown_connections 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 --2- 172.21.15.190:0/1793058974 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fac981399b0 0x7fac98139e60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 --2- 172.21.15.190:0/1793058974 >> v2:172.21.15.89:3300/0 conn(0x7fac980a1d60 0x7fac980a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 --2- 172.21.15.190:0/1793058974 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a46b0 0x7fac980a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 -- 172.21.15.190:0/1793058974 >> 172.21.15.190:0/1793058974 conn(0x7fac980a92a0 msgr2=0x7fac980a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 -- 172.21.15.190:0/1793058974 shutdown_connections 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 -- 172.21.15.190:0/1793058974 wait complete. 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient: build_initial_monmap 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient: monmap: 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 Processor -- start 2024-08-19T10:44:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 -- start start 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient: init 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient: _add_conns ranks=[6,0,7,8,2,4,5,3,1] 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a1d60 0x7fac98138070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient(hunting): picked mon.noname-d con 0x7fac980a1d60 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac980a46b0 0x7fac981385b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient(hunting): picked mon.noname-a con 0x7fac980a46b0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac981399b0 0x7fac9813c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient(hunting): picked mon.noname-i con 0x7fac981399b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fac9806d3d0 con 0x7fac980a46b0 2024-08-19T10:44:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fac9806ce30 con 0x7fac980a1d60 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fac98124040 con 0x7fac981399b0 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac9dc1e640 10 monclient(hunting): authenticate will time out at 1308.154485s 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac981399b0 0x7fac9813c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac981399b0 0x7fac9813c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:51160/0 (socket says 172.21.15.190:51160) 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac977fe640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a1d60 msgr2=0x7fac98138070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 1 -- 172.21.15.190:0/971322248 learned_addr learned my addr 172.21.15.190:0/971322248 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 10 monclient(hunting): get_auth_request con 0x7fac981399b0 auth_method 0 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac977fe640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a1d60 0x7fac98138070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac96ffd640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac980a46b0 0x7fac981385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 10 monclient(hunting): handle_auth_done global_id 15325 payload 824 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 1 -- 172.21.15.190:0/971322248 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a1d60 msgr2=0x7fac98138070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a1d60 0x7fac98138070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 1 -- 172.21.15.190:0/971322248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac980a46b0 msgr2=0x7fac981385b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac980a46b0 0x7fac981385b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 1 monclient: found mon.noname-i 2024-08-19T10:44:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac97fff640 1 -- 172.21.15.190:0/971322248 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac98123a80 con 0x7fac981399b0 2024-08-19T10:44:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac96ffd640 10 monclient: get_auth_request con 0x7fac980a46b0 auth_method 0 2024-08-19T10:44:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.582+0000 7fac96ffd640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac980a46b0 0x7fac981385b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac97fff640 1 -- 172.21.15.190:0/971322248 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac981399b0 msgr2=0x7fac9813c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac97fff640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac981399b0 0x7fac9813c7d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fac9001c700 tx=0x7fac90019670 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 1 -- 172.21.15.190:0/971322248 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac900bc030 con 0x7fac981399b0 2024-08-19T10:44:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 10 monclient: dump: 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 10 monclient: _finish_auth 0 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac9dc1e640 5 monclient: authenticate success, global_id 15325 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac9dc1e640 10 monclient: _renew_subs 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac9dc1e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac9dc1e640 1 -- 172.21.15.190:0/971322248 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9813fc90 con 0x7fac981399b0 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac9dc1e640 10 monclient: _renew_subs 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac9dc1e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:22.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac9dc1e640 1 -- 172.21.15.190:0/971322248 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fac980a8870 con 0x7fac981399b0 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 1 -- 172.21.15.190:0/971322248 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac90004040 con 0x7fac981399b0 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 1 -- 172.21.15.190:0/971322248 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fac900046a0 con 0x7fac981399b0 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 10 monclient: dump: 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:22.601 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac95ffb640 4 set_mon_vals no callback set 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac9dc1e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac9dc1e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac9dc1e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:22.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac9dc1e640 1 -- 172.21.15.190:0/971322248 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac98139e60 con 0x7fac981399b0 2024-08-19T10:44:22.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 1 -- 172.21.15.190:0/971322248 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fac900bf070 con 0x7fac981399b0 2024-08-19T10:44:22.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8403be00 0x7fac8403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:22.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac7ffff640 1 -- 172.21.15.190:0/971322248 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(210..210 src has 1..210) ==== 11107+0+0 (secure 0 0 0) 0x7fac900bd070 con 0x7fac981399b0 2024-08-19T10:44:22.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac977fe640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8403be00 0x7fac8403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:22.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.586+0000 7fac977fe640 10 monclient: get_auth_request con 0x7fac8403be00 auth_method 0 2024-08-19T10:44:22.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.590+0000 7fac977fe640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8403be00 0x7fac8403e2c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac88006fd0 tx=0x7fac88006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:22.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.590+0000 7fac7ffff640 1 -- 172.21.15.190:0/971322248 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fac900c6020 con 0x7fac981399b0 2024-08-19T10:44:22.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.590+0000 7fac7ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:22.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.590+0000 7fac7ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:22.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.762+0000 7fac9dc1e640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:22.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.762+0000 7fac9dc1e640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:22.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.762+0000 7fac9dc1e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:22.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:22.762+0000 7fac9dc1e640 1 -- 172.21.15.190:0/971322248 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fac9806d740 con 0x7fac981399b0 2024-08-19T10:44:24.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.158+0000 7fac7ffff640 1 -- 172.21.15.190:0/971322248 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}]=0 added bucket h16 type host to location {rack=r1} v211) ==== 168+0+0 (secure 0 0 0) 0x7fac900165e0 con 0x7fac981399b0 2024-08-19T10:44:24.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.158+0000 7fac7ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:24.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.158+0000 7fac7ffff640 10 monclient: _finish_command 2 = system:0 added bucket h16 type host to location {rack=r1} 2024-08-19T10:44:24.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.266+0000 7fac9dc1e640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:24.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.266+0000 7fac9dc1e640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:24.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.266+0000 7fac9dc1e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:24.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.266+0000 7fac9dc1e640 1 -- 172.21.15.190:0/971322248 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fac9813e900 con 0x7fac981399b0 2024-08-19T10:44:24.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.266+0000 7fac7ffff640 1 -- 172.21.15.190:0/971322248 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h16' already exists v211) ==== 147+0+0 (secure 0 0 0) 0x7fac900e8010 con 0x7fac981399b0 2024-08-19T10:44:24.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.266+0000 7fac7ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h16", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:24.270 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.266+0000 7fac7ffff640 10 monclient: _finish_command 3 = system:0 bucket 'h16' already exists 2024-08-19T10:44:24.270 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h16' already exists 2024-08-19T10:44:24.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 -- 172.21.15.190:0/971322248 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8403be00 msgr2=0x7fac8403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:24.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8403be00 0x7fac8403e2c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac88006fd0 tx=0x7fac88006d40 comp rx=0 tx=0).stop 2024-08-19T10:44:24.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 10 monclient: shutdown 2024-08-19T10:44:24.272 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:24.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 -- 172.21.15.190:0/971322248 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac981399b0 msgr2=0x7fac9813c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:24.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac981399b0 0x7fac9813c7d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fac9001c700 tx=0x7fac90019670 comp rx=0 tx=0).stop 2024-08-19T10:44:24.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 -- 172.21.15.190:0/971322248 shutdown_connections 2024-08-19T10:44:24.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fac8403be00 0x7fac8403e2c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:24.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fac981399b0 0x7fac9813c7d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:24.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac980a46b0 0x7fac981385b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:24.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 --2- 172.21.15.190:0/971322248 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fac980a1d60 0x7fac98138070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:24.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 -- 172.21.15.190:0/971322248 >> 172.21.15.190:0/971322248 conn(0x7fac980a92a0 msgr2=0x7fac9809edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:24.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 -- 172.21.15.190:0/971322248 shutdown_connections 2024-08-19T10:44:24.273 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.270+0000 7fac9dc1e640 1 -- 172.21.15.190:0/971322248 wait complete. 2024-08-19T10:44:24.287 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:24.287 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h17 host rack=r1 2024-08-19T10:44:24.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.406+0000 7f2675347640 10 monclient: get_monmap_and_config 2024-08-19T10:44:24.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.406+0000 7f2675347640 10 monclient: build_initial_monmap 2024-08-19T10:44:24.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.406+0000 7f2675347640 10 monclient: monmap: 2024-08-19T10:44:24.408 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:24.408 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:24.408 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:24.409408+0000 2024-08-19T10:44:24.408 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:24.409408+0000 2024-08-19T10:44:24.408 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:24.408 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:24.409 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:24.409 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:24.409 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:24.409 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:24.409 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:24.409 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:24.409 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:24.409 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:24.409 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:24.409 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:24.409 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:24.410 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:24.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.406+0000 7f2675347640 1 Processor -- start 2024-08-19T10:44:24.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.406+0000 7f2675347640 1 -- start start 2024-08-19T10:44:24.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.406+0000 7f2675347640 10 monclient: init 2024-08-19T10:44:24.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.406+0000 7f2675347640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:24.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.406+0000 7f2675347640 10 monclient: _add_conns ranks=[1,6,7,9,10,5,2,0,4,8,3] 2024-08-19T10:44:24.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f26700a4720 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:24.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f26700a2f30 0x7f267013b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:24.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 10 monclient(hunting): picked mon.noname-d con 0x7f26700a2f30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f26700a5210 0x7f267013df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 10 monclient(hunting): picked mon.noname-i con 0x7f26700a5210 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26700a2500 con 0x7f26700a4720 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f26700a0bc0 con 0x7f26700a2f30 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2670074350 con 0x7f26700a5210 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f2675347640 10 monclient(hunting): authenticate will time out at 1309.980096s 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266effd640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f26700a4720 legacy=0x7f26700a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266e7fc640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f26700a2f30 msgr2=0x7f267013b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:24.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f26700a5210 0x7f267013df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:24.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266e7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f26700a2f30 0x7f267013b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:24.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f26700a5210 0x7f267013df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:51162/0 (socket says 172.21.15.190:51162) 2024-08-19T10:44:24.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 1 -- 172.21.15.190:0/999710954 learned_addr learned my addr 172.21.15.190:0/999710954 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:24.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 10 monclient(hunting): get_auth_request con 0x7f26700a5210 auth_method 0 2024-08-19T10:44:24.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:24.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:24.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:24.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:24.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:24.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:24.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 10 monclient(hunting): handle_auth_done global_id 15334 payload 274 2024-08-19T10:44:24.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 1 -- 172.21.15.190:0/999710954 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f26700a2f30 msgr2=0x7f267013b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:24.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 1 --2- 172.21.15.190:0/999710954 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f26700a2f30 0x7f267013b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:24.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 1 -- 172.21.15.190:0/999710954 >> v1:172.21.15.190:6789/0 conn(0x7f26700a4720 legacy=0x7f26700a4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:24.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:24.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 1 monclient: found mon.noname-i 2024-08-19T10:44:24.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:24.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:24.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.410+0000 7f266f7fe640 1 -- 172.21.15.190:0/999710954 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26700a0940 con 0x7f26700a5210 2024-08-19T10:44:24.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266f7fe640 1 -- 172.21.15.190:0/999710954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f26700a5210 msgr2=0x7f267013df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:24.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266f7fe640 1 --2- 172.21.15.190:0/999710954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f26700a5210 0x7f267013df20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2668009eb0 tx=0x7f266801d370 comp rx=0 tx=0).ready entity=mon.7 client_cookie=a19324ed0070a66f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:24.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 1 -- 172.21.15.190:0/999710954 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26680091b0 con 0x7f26700a5210 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 10 monclient: dump: 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:24.420 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:24.421 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:24.421 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:24.421 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:24.421 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:24.421 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:24.421 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:24.421 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:24.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:24.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:24.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:24.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 1 -- 172.21.15.190:0/999710954 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26680a5a70 con 0x7f26700a5210 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 5 monclient: authenticate success, global_id 15334 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 4 set_mon_vals no callback set 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 1 -- 172.21.15.190:0/999710954 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26680a5c20 con 0x7f26700a5210 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266dffb640 10 monclient: dump: 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:24.422 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:24.423 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:24.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient: shutdown 2024-08-19T10:44:24.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:24.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 -- 172.21.15.190:0/999710954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f26700a5210 msgr2=0x7f267013df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:24.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 --2- 172.21.15.190:0/999710954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f26700a5210 0x7f267013df20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2668009eb0 tx=0x7f266801d370 comp rx=0 tx=0).stop 2024-08-19T10:44:24.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 -- 172.21.15.190:0/999710954 shutdown_connections 2024-08-19T10:44:24.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 --2- 172.21.15.190:0/999710954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f26700a5210 0x7f267013df20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:24.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 --2- 172.21.15.190:0/999710954 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f26700a2f30 0x7f267013b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:24.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 -- 172.21.15.190:0/999710954 >> 172.21.15.190:0/999710954 conn(0x7f26700a9310 msgr2=0x7f26700a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:24.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 -- 172.21.15.190:0/999710954 shutdown_connections 2024-08-19T10:44:24.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 -- 172.21.15.190:0/999710954 wait complete. 2024-08-19T10:44:24.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient: build_initial_monmap 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient: monmap: 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:24.425 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 Processor -- start 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 -- start start 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient: init 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient: _add_conns ranks=[8,1,5,6,2,3,0,7,4] 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f26700a2f30 0x7f26701cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient(hunting): picked mon.noname-e con 0x7f26700a2f30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:24.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26700a4720 0x7f26701cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient(hunting): picked mon.noname-f con 0x7f26700a4720 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f26700a5210 0x7f267006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient(hunting): picked mon.noname-h con 0x7f26700a5210 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2670140e00 con 0x7f26700a4720 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2670140c80 con 0x7f26700a5210 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2670140f80 con 0x7f26700a2f30 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f2675347640 10 monclient(hunting): authenticate will time out at 1309.987891s 2024-08-19T10:44:24.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266e7fc640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26700a4720 msgr2=0x7f26701cf9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.414+0000 7f266e7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26700a4720 0x7f26701cf9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f26700a5210 0x7f267006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f26700a5210 0x7f267006c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:41512/0 (socket says 172.21.15.190:41512) 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 1 -- 172.21.15.190:0/1976972444 learned_addr learned my addr 172.21.15.190:0/1976972444 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 10 monclient(hunting): get_auth_request con 0x7f26700a5210 auth_method 0 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:24.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266effd640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f26700a2f30 0x7f26701cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:24.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266effd640 10 monclient(hunting): get_auth_request con 0x7f26700a2f30 auth_method 0 2024-08-19T10:44:24.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:24.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:24.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:24.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 10 monclient(hunting): handle_auth_done global_id 6071 payload 824 2024-08-19T10:44:24.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 1 -- 172.21.15.190:0/1976972444 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f26700a2f30 msgr2=0x7f26701cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:24.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f26700a2f30 0x7f26701cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:24.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 1 -- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26700a4720 msgr2=0x7f26701cf9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:24.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26700a4720 0x7f26701cf9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:24.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:24.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 1 monclient: found mon.noname-h 2024-08-19T10:44:24.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:24.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:24.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 1 -- 172.21.15.190:0/1976972444 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26700a8300 con 0x7f26700a5210 2024-08-19T10:44:24.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 1 -- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f26700a5210 msgr2=0x7f267006c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:24.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266f7fe640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f26700a5210 0x7f267006c8d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2668009040 tx=0x7f26680196b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 1 -- 172.21.15.190:0/1976972444 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26680bc020 con 0x7f26700a5210 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 10 monclient: dump: 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:24.432 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 10 monclient: _finish_auth 0 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 10 monclient: _check_auth_tickets 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:24.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 1 -- 172.21.15.190:0/1976972444 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2668003dc0 con 0x7f26700a5210 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f2675347640 5 monclient: authenticate success, global_id 6071 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f2675347640 10 monclient: _renew_subs 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f2675347640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f2675347640 1 -- 172.21.15.190:0/1976972444 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2670073df0 con 0x7f26700a5210 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 1 -- 172.21.15.190:0/1976972444 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f26680b8470 con 0x7f26700a5210 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 10 monclient: dump: 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:24.434 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:24.435 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f266d7fa640 4 set_mon_vals no callback set 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f2675347640 10 monclient: _renew_subs 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f2675347640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f2675347640 1 -- 172.21.15.190:0/1976972444 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f26701377c0 con 0x7f26700a5210 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f2675347640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f2675347640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f2675347640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f2675347640 1 -- 172.21.15.190:0/1976972444 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26700a4b00 con 0x7f26700a5210 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.418+0000 7f26577fe640 1 -- 172.21.15.190:0/1976972444 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f26680bf020 con 0x7f26700a5210 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.422+0000 7f26577fe640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f265c03be00 0x7f265c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.422+0000 7f26577fe640 1 -- 172.21.15.190:0/1976972444 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(211..211 src has 1..211) ==== 11576+0+0 (secure 0 0 0) 0x7f26680eb7e0 con 0x7f26700a5210 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.422+0000 7f266effd640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f265c03be00 0x7f265c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.422+0000 7f266effd640 10 monclient: get_auth_request con 0x7f265c03be00 auth_method 0 2024-08-19T10:44:24.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.422+0000 7f266effd640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f265c03be00 0x7f265c03e2c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2660006fd0 tx=0x7f2660008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:24.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.426+0000 7f26577fe640 1 -- 172.21.15.190:0/1976972444 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f26680c6030 con 0x7f26700a5210 2024-08-19T10:44:24.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.426+0000 7f26577fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:24.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.426+0000 7f26577fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:24.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.594+0000 7f2675347640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:24.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.594+0000 7f2675347640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:24.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.594+0000 7f2675347640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:24.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:24.594+0000 7f2675347640 1 -- 172.21.15.190:0/1976972444 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f2670137a90 con 0x7f26700a5210 2024-08-19T10:44:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.170+0000 7f26577fe640 1 -- 172.21.15.190:0/1976972444 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}]=0 added bucket h17 type host to location {rack=r1} v212) ==== 168+0+0 (secure 0 0 0) 0x7f2668016620 con 0x7f26700a5210 2024-08-19T10:44:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.170+0000 7f26577fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:25.170 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.170+0000 7f26577fe640 10 monclient: _finish_command 2 = system:0 added bucket h17 type host to location {rack=r1} 2024-08-19T10:44:25.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.274+0000 7f2675347640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:25.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.274+0000 7f2675347640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:25.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.274+0000 7f2675347640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:25.274 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.274+0000 7f2675347640 1 -- 172.21.15.190:0/1976972444 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f2670074550 con 0x7f26700a5210 2024-08-19T10:44:25.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.274+0000 7f26577fe640 1 -- 172.21.15.190:0/1976972444 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h17' already exists v212) ==== 147+0+0 (secure 0 0 0) 0x7f26680b5d80 con 0x7f26700a5210 2024-08-19T10:44:25.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.274+0000 7f26577fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h17", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:25.276 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.274+0000 7f26577fe640 10 monclient: _finish_command 3 = system:0 bucket 'h17' already exists 2024-08-19T10:44:25.276 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h17' already exists 2024-08-19T10:44:25.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 -- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f265c03be00 msgr2=0x7f265c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:25.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f265c03be00 0x7f265c03e2c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2660006fd0 tx=0x7f2660008040 comp rx=0 tx=0).stop 2024-08-19T10:44:25.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 10 monclient: shutdown 2024-08-19T10:44:25.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:25.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 -- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f26700a5210 msgr2=0x7f267006c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:25.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f26700a5210 0x7f267006c8d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2668009040 tx=0x7f26680196b0 comp rx=0 tx=0).stop 2024-08-19T10:44:25.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 -- 172.21.15.190:0/1976972444 shutdown_connections 2024-08-19T10:44:25.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f26700a5210 0x7f267006c8d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:25.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26700a4720 0x7f26701cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:25.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f265c03be00 0x7f265c03e2c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:25.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 --2- 172.21.15.190:0/1976972444 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f26700a2f30 0x7f26701cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:25.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 -- 172.21.15.190:0/1976972444 >> 172.21.15.190:0/1976972444 conn(0x7f26700a9310 msgr2=0x7f26700a7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:25.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 -- 172.21.15.190:0/1976972444 shutdown_connections 2024-08-19T10:44:25.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.278+0000 7f2675347640 1 -- 172.21.15.190:0/1976972444 wait complete. 2024-08-19T10:44:25.293 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:25.293 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h18 host rack=r1 2024-08-19T10:44:25.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient: get_monmap_and_config 2024-08-19T10:44:25.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient: build_initial_monmap 2024-08-19T10:44:25.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient: monmap: 2024-08-19T10:44:25.416 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:25.416 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:25.416 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:25.417485+0000 2024-08-19T10:44:25.416 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:25.417485+0000 2024-08-19T10:44:25.416 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:25.417 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:25.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 1 Processor -- start 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 1 -- start start 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient: init 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient: _add_conns ranks=[10,3,0,4,6,8,9,7,1,2,5] 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4900a4720 0x7ff4900a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient(hunting): picked mon.noname-f con 0x7ff4900a4720 addr v2:172.21.15.190:3300/0 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900a5040 0x7ff4901398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient(hunting): picked mon.noname-g con 0x7ff4900a5040 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff490139e10 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff49006cbc0 con 0x7ff490139e10 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff4900a1400 con 0x7ff4900a4720 2024-08-19T10:44:25.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff49006d280 con 0x7ff4900a5040 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff496046640 10 monclient(hunting): authenticate will time out at 1310.987340s 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48f7fe640 1 -- >> v2:172.21.15.190:3300/0 conn(0x7ff4900a4720 msgr2=0x7ff4900a4b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48f7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4900a4720 0x7ff4900a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900a5040 0x7ff4901398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900a5040 0x7ff4901398d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:42222/0 (socket says 172.21.15.190:42222) 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 1 -- 172.21.15.190:0/2559601099 learned_addr learned my addr 172.21.15.190:0/2559601099 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 10 monclient(hunting): get_auth_request con 0x7ff4900a5040 auth_method 0 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:25.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 10 monclient(hunting): handle_auth_done global_id 34419 payload 274 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 1 -- 172.21.15.190:0/2559601099 >> v2:172.21.15.190:3300/0 conn(0x7ff4900a4720 msgr2=0x7ff4900a4b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 1 --2- 172.21.15.190:0/2559601099 >> v2:172.21.15.190:3300/0 conn(0x7ff4900a4720 0x7ff4900a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 1 -- 172.21.15.190:0/2559601099 >> v1:172.21.15.89:6789/0 conn(0x7ff490139e10 legacy=0x7ff490140b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 1 monclient: found mon.noname-g 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:25.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.414+0000 7ff48effd640 1 -- 172.21.15.190:0/2559601099 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900a0b60 con 0x7ff4900a5040 2024-08-19T10:44:25.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48effd640 1 -- 172.21.15.190:0/2559601099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900a5040 msgr2=0x7ff4901398d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:25.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48effd640 1 --2- 172.21.15.190:0/2559601099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900a5040 0x7ff4901398d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff484009870 tx=0x7ff48401c3e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=536589b790d411af server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:25.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 1 -- 172.21.15.190:0/2559601099 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4840b2070 con 0x7ff4900a5040 2024-08-19T10:44:25.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:25.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:25.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 10 monclient: dump: 2024-08-19T10:44:25.424 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:25.424 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:25.425 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 1 -- 172.21.15.190:0/2559601099 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4840a9070 con 0x7ff4900a5040 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 5 monclient: authenticate success, global_id 34419 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 1 -- 172.21.15.190:0/2559601099 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4840abc40 con 0x7ff4900a5040 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 4 set_mon_vals no callback set 2024-08-19T10:44:25.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48e7fc640 10 monclient: dump: 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:25.427 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient: shutdown 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 -- 172.21.15.190:0/2559601099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900a5040 msgr2=0x7ff4901398d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 --2- 172.21.15.190:0/2559601099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900a5040 0x7ff4901398d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff484009870 tx=0x7ff48401c3e0 comp rx=0 tx=0).stop 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 -- 172.21.15.190:0/2559601099 shutdown_connections 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 --2- 172.21.15.190:0/2559601099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900a5040 0x7ff4901398d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 --2- 172.21.15.190:0/2559601099 >> v2:172.21.15.190:3300/0 conn(0x7ff4900a4720 0x7ff4900a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 -- 172.21.15.190:0/2559601099 >> 172.21.15.190:0/2559601099 conn(0x7ff4900a9310 msgr2=0x7ff4900a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 -- 172.21.15.190:0/2559601099 shutdown_connections 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 -- 172.21.15.190:0/2559601099 wait complete. 2024-08-19T10:44:25.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient: build_initial_monmap 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient: monmap: 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:25.429 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:25.430 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:25.430 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:25.430 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:25.430 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:25.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 Processor -- start 2024-08-19T10:44:25.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 -- start start 2024-08-19T10:44:25.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient: init 2024-08-19T10:44:25.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:25.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient: _add_conns ranks=[6,1,7,2,8,4,3,0,5] 2024-08-19T10:44:25.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4900a4720 0x7ff49013d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:25.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient(hunting): picked mon.noname-d con 0x7ff4900a4720 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:25.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900a5040 0x7ff49013dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:25.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient(hunting): picked mon.noname-f con 0x7ff4900a5040 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:25.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff490139e10 0x7ff49013e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:25.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient(hunting): picked mon.noname-i con 0x7ff490139e10 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:25.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:25.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900a1de0 con 0x7ff4900a5040 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff4900a6240 con 0x7ff4900a4720 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff49006cf50 con 0x7ff490139e10 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff496046640 10 monclient(hunting): authenticate will time out at 1310.995397s 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48effd640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900a5040 msgr2=0x7ff49013dd40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900a5040 0x7ff49013dd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff490139e10 0x7ff49013e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48f7fe640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4900a4720 msgr2=0x7ff49013d800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48f7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4900a4720 0x7ff49013d800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff490139e10 0x7ff49013e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:51174/0 (socket says 172.21.15.190:51174) 2024-08-19T10:44:25.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 1 -- 172.21.15.190:0/1169525876 learned_addr learned my addr 172.21.15.190:0/1169525876 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 10 monclient(hunting): get_auth_request con 0x7ff490139e10 auth_method 0 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 10 monclient(hunting): handle_auth_done global_id 15343 payload 824 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 1 -- 172.21.15.190:0/1169525876 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4900a4720 msgr2=0x7ff49013d800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4900a4720 0x7ff49013d800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 1 -- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900a5040 msgr2=0x7ff49013dd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900a5040 0x7ff49013dd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:25.436 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 1 monclient: found mon.noname-i 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.422+0000 7ff48ffff640 1 -- 172.21.15.190:0/1169525876 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff490141540 con 0x7ff490139e10 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff48ffff640 1 -- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff490139e10 msgr2=0x7ff49013e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff48ffff640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff490139e10 0x7ff49013e280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b9a0 tx=0x7ff48800be60 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 1 -- 172.21.15.190:0/1169525876 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff488019070 con 0x7ff490139e10 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 10 monclient: dump: 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:25.437 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:25.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 1 -- 172.21.15.190:0/1169525876 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48800cbb0 con 0x7ff490139e10 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff496046640 5 monclient: authenticate success, global_id 15343 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 1 -- 172.21.15.190:0/1169525876 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff48800cd60 con 0x7ff490139e10 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 10 monclient: dump: 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:25.439 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff48dffb640 4 set_mon_vals no callback set 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff496046640 10 monclient: _renew_subs 2024-08-19T10:44:25.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff496046640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff496046640 1 -- 172.21.15.190:0/1169525876 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4900a2800 con 0x7ff490139e10 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff496046640 10 monclient: _renew_subs 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff496046640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff496046640 1 -- 172.21.15.190:0/1169525876 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff490141260 con 0x7ff490139e10 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff496046640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff496046640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff496046640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff496046640 1 -- 172.21.15.190:0/1169525876 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4900a5420 con 0x7ff490139e10 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.426+0000 7ff477fff640 1 -- 172.21.15.190:0/1169525876 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff488006d10 con 0x7ff490139e10 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.430+0000 7ff477fff640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff47c03be00 0x7ff47c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.430+0000 7ff477fff640 1 -- 172.21.15.190:0/1169525876 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(213..213 src has 1..213) ==== 11789+0+0 (secure 0 0 0) 0x7ff48804ff10 con 0x7ff490139e10 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.430+0000 7ff48f7fe640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff47c03be00 0x7ff47c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:25.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.430+0000 7ff48f7fe640 10 monclient: get_auth_request con 0x7ff47c03be00 auth_method 0 2024-08-19T10:44:25.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.430+0000 7ff48f7fe640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff47c03be00 0x7ff47c03e2c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff480006fd0 tx=0x7ff480006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:25.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.434+0000 7ff477fff640 1 -- 172.21.15.190:0/1169525876 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff48801cdf0 con 0x7ff490139e10 2024-08-19T10:44:25.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.434+0000 7ff477fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:25.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.434+0000 7ff477fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.606+0000 7ff496046640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.606+0000 7ff496046640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.606+0000 7ff496046640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:25.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:25.606+0000 7ff496046640 1 -- 172.21.15.190:0/1169525876 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7ff49006ed90 con 0x7ff490139e10 2024-08-19T10:44:27.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.082+0000 7ff477fff640 1 -- 172.21.15.190:0/1169525876 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}]=0 added bucket h18 type host to location {rack=r1} v214) ==== 168+0+0 (secure 0 0 0) 0x7ff48804cda0 con 0x7ff490139e10 2024-08-19T10:44:27.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.082+0000 7ff477fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:27.084 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.082+0000 7ff477fff640 10 monclient: _finish_command 2 = system:0 added bucket h18 type host to location {rack=r1} 2024-08-19T10:44:27.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.190+0000 7ff496046640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:27.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.190+0000 7ff496046640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:27.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.190+0000 7ff496046640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:27.192 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.190+0000 7ff496046640 1 -- 172.21.15.190:0/1169525876 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7ff49013ffa0 con 0x7ff490139e10 2024-08-19T10:44:27.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff477fff640 1 -- 172.21.15.190:0/1169525876 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h18' already exists v214) ==== 147+0+0 (secure 0 0 0) 0x7ff488007350 con 0x7ff490139e10 2024-08-19T10:44:27.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff477fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h18", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:27.194 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff477fff640 10 monclient: _finish_command 3 = system:0 bucket 'h18' already exists 2024-08-19T10:44:27.194 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h18' already exists 2024-08-19T10:44:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 -- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff47c03be00 msgr2=0x7ff47c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff47c03be00 0x7ff47c03e2c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff480006fd0 tx=0x7ff480006d40 comp rx=0 tx=0).stop 2024-08-19T10:44:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 10 monclient: shutdown 2024-08-19T10:44:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 -- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff490139e10 msgr2=0x7ff49013e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff490139e10 0x7ff49013e280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b9a0 tx=0x7ff48800be60 comp rx=0 tx=0).stop 2024-08-19T10:44:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 -- 172.21.15.190:0/1169525876 shutdown_connections 2024-08-19T10:44:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff47c03be00 0x7ff47c03e2c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff490139e10 0x7ff49013e280 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900a5040 0x7ff49013dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.197 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 --2- 172.21.15.190:0/1169525876 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4900a4720 0x7ff49013d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 -- 172.21.15.190:0/1169525876 >> 172.21.15.190:0/1169525876 conn(0x7ff4900a9310 msgr2=0x7ff4900a6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 -- 172.21.15.190:0/1169525876 shutdown_connections 2024-08-19T10:44:27.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.194+0000 7ff496046640 1 -- 172.21.15.190:0/1169525876 wait complete. 2024-08-19T10:44:27.212 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:27.212 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h19 host rack=r1 2024-08-19T10:44:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.330+0000 7f9190d79640 10 monclient: get_monmap_and_config 2024-08-19T10:44:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient: build_initial_monmap 2024-08-19T10:44:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient: monmap: 2024-08-19T10:44:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:27.335616+0000 2024-08-19T10:44:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:27.335616+0000 2024-08-19T10:44:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:27.337 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:27.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 1 Processor -- start 2024-08-19T10:44:27.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 1 -- start start 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient: init 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient: _add_conns ranks=[4,2,8,5,10,1,6,9,7,0,3] 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918c1396a0 0x7f918c139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient(hunting): picked mon.noname-c con 0x7f918c1396a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918c139fc0 0x7f918c0a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient(hunting): picked mon.noname-b con 0x7f918c139fc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f918c0a6e20 0x7f918c0a7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient(hunting): picked mon.noname-e con 0x7f918c0a6e20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f918c0a35e0 con 0x7f918c139fc0 2024-08-19T10:44:27.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f918c0a8650 con 0x7f918c1396a0 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f918c0a8410 con 0x7f918c0a6e20 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9190d79640 10 monclient(hunting): authenticate will time out at 1312.905722s 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918a59c640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918c1396a0 msgr2=0x7f918c139a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918a59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918c1396a0 0x7f918c139a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9189d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918c139fc0 0x7f918c0a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f918c0a6e20 0x7f918c0a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9189d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918c139fc0 0x7f918c0a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:34508/0 (socket says 172.21.15.190:34508) 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9189d9b640 1 -- 172.21.15.190:0/2149270382 learned_addr learned my addr 172.21.15.190:0/2149270382 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9189d9b640 10 monclient(hunting): get_auth_request con 0x7f918c139fc0 auth_method 0 2024-08-19T10:44:27.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9189d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:27.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9189d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:27.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9189d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:27.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 10 monclient(hunting): get_auth_request con 0x7f918c0a6e20 auth_method 0 2024-08-19T10:44:27.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:27.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:27.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:27.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9189d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:27.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9189d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:27.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9189d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:27.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:27.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:27.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:27.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 10 monclient(hunting): handle_auth_done global_id 5885 payload 274 2024-08-19T10:44:27.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 1 -- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918c1396a0 msgr2=0x7f918c139a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:27.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 1 --2- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918c1396a0 0x7f918c139a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 1 -- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918c139fc0 msgr2=0x7f918c0a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:27.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 1 --2- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918c139fc0 0x7f918c0a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 10 monclient: _finish_hunting 0 2024-08-19T10:44:27.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 1 monclient: found mon.noname-e 2024-08-19T10:44:27.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:27.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:27.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f918ad9d640 1 -- 172.21.15.190:0/2149270382 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c0a4c40 con 0x7f918c0a6e20 2024-08-19T10:44:27.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.334+0000 7f9189d9b640 1 --2- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918c139fc0 0x7f918c0a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:27.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.338+0000 7f918ad9d640 1 -- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f918c0a6e20 msgr2=0x7f918c0a7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:27.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.338+0000 7f918ad9d640 1 --2- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f918c0a6e20 0x7f918c0a7280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f917c009820 tx=0x7f917c01c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=782e30ec6064ed62 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:27.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.338+0000 7f918959a640 1 -- 172.21.15.190:0/2149270382 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f917c0b0070 con 0x7f918c0a6e20 2024-08-19T10:44:27.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.338+0000 7f918959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:27.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.338+0000 7f918959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:27.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.338+0000 7f918959a640 10 monclient: dump: 2024-08-19T10:44:27.349 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:27.350 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918959a640 10 monclient: _finish_auth 0 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918959a640 10 monclient: _check_auth_tickets 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 5 monclient: authenticate success, global_id 5885 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918959a640 1 -- 172.21.15.190:0/2149270382 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f917c0aa810 con 0x7f918c0a6e20 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 4 set_mon_vals no callback set 2024-08-19T10:44:27.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918959a640 1 -- 172.21.15.190:0/2149270382 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f917c0a4e60 con 0x7f918c0a6e20 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918959a640 10 monclient: dump: 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:27.352 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:27.353 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:27.353 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:27.353 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:27.353 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:27.353 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:27.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient: shutdown 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 -- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f918c0a6e20 msgr2=0x7f918c0a7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 --2- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f918c0a6e20 0x7f918c0a7280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f917c009820 tx=0x7f917c01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 -- 172.21.15.190:0/2149270382 shutdown_connections 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 --2- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f918c0a6e20 0x7f918c0a7280 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 --2- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f918c139fc0 0x7f918c0a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 --2- 172.21.15.190:0/2149270382 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f918c1396a0 0x7f918c139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 -- 172.21.15.190:0/2149270382 >> 172.21.15.190:0/2149270382 conn(0x7f918c0abee0 msgr2=0x7f918c0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 -- 172.21.15.190:0/2149270382 shutdown_connections 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 -- 172.21.15.190:0/2149270382 wait complete. 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient: build_initial_monmap 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient: monmap: 2024-08-19T10:44:27.354 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:27.355 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 Processor -- start 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 -- start start 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient: init 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient: _add_conns ranks=[5,3,1,2,6,4,0,8,7] 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f918c0a6e20 0x7f918c1348b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient(hunting): picked mon.noname-h con 0x7f918c0a6e20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f918c1396a0 0x7f918c134df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient(hunting): picked mon.noname-g con 0x7f918c1396a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f918c139fc0 0x7f918c09ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient(hunting): picked mon.noname-f con 0x7f918c139fc0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:27.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f918c0a5820 con 0x7f918c139fc0 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f918c0a56a0 con 0x7f918c1396a0 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f918c143320 con 0x7f918c0a6e20 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9190d79640 10 monclient(hunting): authenticate will time out at 1312.914227s 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918ad9d640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f918c139fc0 msgr2=0x7f918c09ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f918c139fc0 0x7f918c09ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f918c0a6e20 0x7f918c1348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9189d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f918c1396a0 0x7f918c134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:27.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9189d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f918c1396a0 0x7f918c134df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:39290/0 (socket says 172.21.15.190:39290) 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f918c0a6e20 0x7f918c1348b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:45588/0 (socket says 172.21.15.190:45588) 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9189d9b640 1 -- 172.21.15.190:0/2499012979 learned_addr learned my addr 172.21.15.190:0/2499012979 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9189d9b640 10 monclient(hunting): get_auth_request con 0x7f918c1396a0 auth_method 0 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9189d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9189d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9189d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 10 monclient(hunting): get_auth_request con 0x7f918c0a6e20 auth_method 0 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:27.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9189d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9189d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9189d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 10 monclient(hunting): handle_auth_done global_id 6080 payload 824 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 1 -- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f918c1396a0 msgr2=0x7f918c134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f918c1396a0 0x7f918c134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 1 -- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f918c139fc0 msgr2=0x7f918c09ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f918c139fc0 0x7f918c09ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 1 monclient: found mon.noname-h 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:27.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f918a59c640 1 -- 172.21.15.190:0/2499012979 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c1356f0 con 0x7f918c0a6e20 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.342+0000 7f9189d9b640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f918c1396a0 0x7f918c134df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f918a59c640 1 -- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f918c0a6e20 msgr2=0x7f918c1348b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f918a59c640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f918c0a6e20 0x7f918c1348b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f918400bdd0 tx=0x7f9184009f70 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 1 -- 172.21.15.190:0/2499012979 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f91840158e0 con 0x7f918c0a6e20 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 10 monclient: dump: 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:27.360 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:27.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 1 -- 172.21.15.190:0/2499012979 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9184010070 con 0x7f918c0a6e20 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9190d79640 5 monclient: authenticate success, global_id 6080 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9190d79640 10 monclient: _renew_subs 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9190d79640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9190d79640 1 -- 172.21.15.190:0/2499012979 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c140840 con 0x7f918c0a6e20 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 1 -- 172.21.15.190:0/2499012979 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9184013370 con 0x7f918c0a6e20 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 10 monclient: dump: 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:27.362 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:27.363 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:27.363 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:27.363 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:27.363 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:27.363 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:27.363 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:27.363 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:27.363 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9188d99640 4 set_mon_vals no callback set 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9190d79640 10 monclient: _renew_subs 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9190d79640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9190d79640 1 -- 172.21.15.190:0/2499012979 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c09ef10 con 0x7f918c0a6e20 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9190d79640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9190d79640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9190d79640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:27.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9190d79640 1 -- 172.21.15.190:0/2499012979 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c139a80 con 0x7f918c0a6e20 2024-08-19T10:44:27.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 1 -- 172.21.15.190:0/2499012979 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9184015470 con 0x7f918c0a6e20 2024-08-19T10:44:27.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f916c03be30 0x7f916c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:27.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9182ffd640 1 -- 172.21.15.190:0/2499012979 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(214..214 src has 1..214) ==== 12002+0+0 (secure 0 0 0) 0x7f9184050a10 con 0x7f918c0a6e20 2024-08-19T10:44:27.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9189d9b640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f916c03be30 0x7f916c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:27.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.346+0000 7f9189d9b640 10 monclient: get_auth_request con 0x7f916c03be30 auth_method 0 2024-08-19T10:44:27.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.350+0000 7f9189d9b640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f916c03be30 0x7f916c03e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f918c09e040 tx=0x7f9174009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:27.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.350+0000 7f9182ffd640 1 -- 172.21.15.190:0/2499012979 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f918c1356f0 con 0x7f918c0a6e20 2024-08-19T10:44:27.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.350+0000 7f9182ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:27.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.350+0000 7f9182ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:27.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.530+0000 7f9190d79640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:27.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.530+0000 7f9190d79640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:27.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.530+0000 7f9190d79640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:27.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:27.530+0000 7f9190d79640 1 -- 172.21.15.190:0/2499012979 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f918c0a0670 con 0x7f918c0a6e20 2024-08-19T10:44:28.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.086+0000 7f9182ffd640 1 -- 172.21.15.190:0/2499012979 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}]=0 added bucket h19 type host to location {rack=r1} v215) ==== 168+0+0 (secure 0 0 0) 0x7f91840143d0 con 0x7f918c0a6e20 2024-08-19T10:44:28.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.086+0000 7f9182ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:28.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.086+0000 7f9182ffd640 10 monclient: _finish_command 2 = system:0 added bucket h19 type host to location {rack=r1} 2024-08-19T10:44:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.194+0000 7f9190d79640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.198+0000 7f9190d79640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.198+0000 7f9190d79640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:28.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.198+0000 7f9190d79640 1 -- 172.21.15.190:0/2499012979 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f918c0008e0 con 0x7f918c0a6e20 2024-08-19T10:44:28.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.198+0000 7f9182ffd640 1 -- 172.21.15.190:0/2499012979 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h19' already exists v215) ==== 147+0+0 (secure 0 0 0) 0x7f918401fe10 con 0x7f918c0a6e20 2024-08-19T10:44:28.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9182ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h19", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:28.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9182ffd640 10 monclient: _finish_command 3 = system:0 bucket 'h19' already exists 2024-08-19T10:44:28.202 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h19' already exists 2024-08-19T10:44:28.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 -- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f916c03be30 msgr2=0x7f916c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:28.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f916c03be30 0x7f916c03e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f918c09e040 tx=0x7f9174009240 comp rx=0 tx=0).stop 2024-08-19T10:44:28.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 10 monclient: shutdown 2024-08-19T10:44:28.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:28.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 -- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f918c0a6e20 msgr2=0x7f918c1348b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:28.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f918c0a6e20 0x7f918c1348b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f918400bdd0 tx=0x7f9184009f70 comp rx=0 tx=0).stop 2024-08-19T10:44:28.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 -- 172.21.15.190:0/2499012979 shutdown_connections 2024-08-19T10:44:28.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f916c03be30 0x7f916c03e2f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f918c139fc0 0x7f918c09ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f918c1396a0 0x7f918c134df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 --2- 172.21.15.190:0/2499012979 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f918c0a6e20 0x7f918c1348b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 -- 172.21.15.190:0/2499012979 >> 172.21.15.190:0/2499012979 conn(0x7f918c0abee0 msgr2=0x7f918c0ab470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:28.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 -- 172.21.15.190:0/2499012979 shutdown_connections 2024-08-19T10:44:28.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.202+0000 7f9190d79640 1 -- 172.21.15.190:0/2499012979 wait complete. 2024-08-19T10:44:28.220 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:28.221 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h20 host rack=r1 2024-08-19T10:44:28.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.334+0000 7fd49a50a640 10 monclient: get_monmap_and_config 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.334+0000 7fd49a50a640 10 monclient: build_initial_monmap 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.334+0000 7fd49a50a640 10 monclient: monmap: 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:28.338377+0000 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:28.338377+0000 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:28.337 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:28.338 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:28.338 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:28.338 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:28.338 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:28.338 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:28.338 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:28.338 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:28.338 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:28.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.334+0000 7fd49a50a640 1 Processor -- start 2024-08-19T10:44:28.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.334+0000 7fd49a50a640 1 -- start start 2024-08-19T10:44:28.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.334+0000 7fd49a50a640 10 monclient: init 2024-08-19T10:44:28.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:28.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 10 monclient: _add_conns ranks=[2,3,8,9,1,7,4,6,0,10,5] 2024-08-19T10:44:28.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4941396a0 0x7fd494139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:28.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 10 monclient(hunting): picked mon.noname-b con 0x7fd4941396a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:28.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd494139fc0 0x7fd4940a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:28.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 10 monclient(hunting): picked mon.noname-g con 0x7fd494139fc0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4940a6e20 0x7fd4940a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 10 monclient(hunting): picked mon.noname-e con 0x7fd4940a6e20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd4940a4f20 con 0x7fd4941396a0 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd4940a83e0 con 0x7fd494139fc0 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd4940a2f30 con 0x7fd4940a6e20 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd49a50a640 10 monclient(hunting): authenticate will time out at 1313.908214s 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd494139fc0 0x7fd4940a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:28.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd494139fc0 0x7fd4940a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:39296/0 (socket says 172.21.15.190:39296) 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 1 -- 172.21.15.190:0/391991722 learned_addr learned my addr 172.21.15.190:0/391991722 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 10 monclient(hunting): get_auth_request con 0x7fd494139fc0 auth_method 0 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd493fff640 1 --2- 172.21.15.190:0/391991722 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4941396a0 0x7fd494139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd498aa7640 1 --2- 172.21.15.190:0/391991722 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4940a6e20 0x7fd4940a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd498aa7640 10 monclient(hunting): get_auth_request con 0x7fd4940a6e20 auth_method 0 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd498aa7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:28.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd498aa7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd498aa7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd493fff640 10 monclient(hunting): get_auth_request con 0x7fd4941396a0 auth_method 0 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd493fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd493fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd493fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 10 monclient(hunting): handle_auth_done global_id 34437 payload 274 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 1 -- 172.21.15.190:0/391991722 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4940a6e20 msgr2=0x7fd4940a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 1 --2- 172.21.15.190:0/391991722 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4940a6e20 0x7fd4940a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 1 -- 172.21.15.190:0/391991722 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4941396a0 msgr2=0x7fd494139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 1 --2- 172.21.15.190:0/391991722 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4941396a0 0x7fd494139a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 1 monclient: found mon.noname-g 2024-08-19T10:44:28.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:28.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:28.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.338+0000 7fd4937fe640 1 -- 172.21.15.190:0/391991722 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4940a35e0 con 0x7fd494139fc0 2024-08-19T10:44:28.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd4937fe640 1 -- 172.21.15.190:0/391991722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd494139fc0 msgr2=0x7fd4940a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:28.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd4937fe640 1 --2- 172.21.15.190:0/391991722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd494139fc0 0x7fd4940a68e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098d0 tx=0x7fd48801c440 comp rx=0 tx=0).ready entity=mon.3 client_cookie=9afabf329c54c842 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:28.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 1 -- 172.21.15.190:0/391991722 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd4880b0070 con 0x7fd494139fc0 2024-08-19T10:44:28.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:28.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:28.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 10 monclient: dump: 2024-08-19T10:44:28.351 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:28.351 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:28.351 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:28.351 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:28.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 5 monclient: authenticate success, global_id 34437 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 1 -- 172.21.15.190:0/391991722 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4880aa810 con 0x7fd494139fc0 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 4 set_mon_vals no callback set 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 1 -- 172.21.15.190:0/391991722 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd4880a9300 con 0x7fd494139fc0 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd492ffd640 10 monclient: dump: 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:28.353 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:28.354 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient: shutdown 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 -- 172.21.15.190:0/391991722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd494139fc0 msgr2=0x7fd4940a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 --2- 172.21.15.190:0/391991722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd494139fc0 0x7fd4940a68e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098d0 tx=0x7fd48801c440 comp rx=0 tx=0).stop 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 -- 172.21.15.190:0/391991722 shutdown_connections 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 --2- 172.21.15.190:0/391991722 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4940a6e20 0x7fd4940a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 --2- 172.21.15.190:0/391991722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd494139fc0 0x7fd4940a68e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 --2- 172.21.15.190:0/391991722 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4941396a0 0x7fd494139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 -- 172.21.15.190:0/391991722 >> 172.21.15.190:0/391991722 conn(0x7fd4940abee0 msgr2=0x7fd4940ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 -- 172.21.15.190:0/391991722 shutdown_connections 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 -- 172.21.15.190:0/391991722 wait complete. 2024-08-19T10:44:28.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient: build_initial_monmap 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient: monmap: 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:28.356 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 Processor -- start 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 -- start start 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient: init 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient: _add_conns ranks=[0,8,1,2,6,5,7,4,3] 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4940a6e20 0x7fd494134c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient(hunting): picked mon.noname-a con 0x7fd4940a6e20 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4941396a0 0x7fd49409ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:28.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient(hunting): picked mon.noname-e con 0x7fd4941396a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd494139fc0 0x7fd49409cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient(hunting): picked mon.noname-f con 0x7fd494139fc0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd49413a3a0 con 0x7fd4940a6e20 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd494143390 con 0x7fd494139fc0 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd4940ac580 con 0x7fd4941396a0 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd49a50a640 10 monclient(hunting): authenticate will time out at 1313.915483s 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd498aa7640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd494139fc0 msgr2=0x7fd49409cfa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:28.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd498aa7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd494139fc0 0x7fd49409cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd493fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4940a6e20 0x7fd494134c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4941396a0 0x7fd49409ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd493fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4940a6e20 0x7fd494134c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:53216/0 (socket says 172.21.15.190:53216) 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd493fff640 1 -- 172.21.15.190:0/1907792803 learned_addr learned my addr 172.21.15.190:0/1907792803 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4941396a0 0x7fd49409ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:39348/0 (socket says 172.21.15.190:39348) 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd493fff640 10 monclient(hunting): get_auth_request con 0x7fd4940a6e20 auth_method 0 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd493fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd493fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd493fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd4937fe640 10 monclient(hunting): get_auth_request con 0x7fd4941396a0 auth_method 0 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd4937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:28.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd4937fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.342+0000 7fd4937fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd493fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd493fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd493fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 10 monclient(hunting): handle_auth_done global_id 5903 payload 824 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd494139fc0 msgr2=0x7fd49409cfa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd494139fc0 0x7fd49409cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4940a6e20 msgr2=0x7fd494134c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:28.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4940a6e20 0x7fd494134c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 monclient: found mon.noname-e 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 -- 172.21.15.190:0/1907792803 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd494135730 con 0x7fd4941396a0 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4941396a0 msgr2=0x7fd49409ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4941396a0 0x7fd49409ca60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd4940a8560 tx=0x7fd4880191e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4941396a0 msgr2=0x7fd49409ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4941396a0 msgr2=0x7fd49409ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4941396a0 0x7fd49409ca60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd4940a8560 tx=0x7fd4880191e0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4941396a0 0x7fd49409ca60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd4940a8560 tx=0x7fd4880191e0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 10 monclient: ms_handle_reset current mon [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:28.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:28.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4941396a0 msgr2=0x7fd49409ca60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:28.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4941396a0 0x7fd49409ca60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 10 monclient: _add_conns ranks=[2,4,7,5,3,6] 2024-08-19T10:44:28.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4800035e0 0x7fd480005a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:28.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 10 monclient(hunting): picked mon.noname-b con 0x7fd4800035e0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:28.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd480006530 0x7fd4800089c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:28.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 10 monclient(hunting): picked mon.noname-c con 0x7fd480006530 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:28.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd480009560 0x7fd48000ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:28.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 10 monclient(hunting): picked mon.noname-i con 0x7fd480009560 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:28.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd48000c5d0 con 0x7fd4800035e0 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd48000c750 con 0x7fd480006530 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd48000c950 con 0x7fd480009560 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd490ff9640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd480006530 msgr2=0x7fd4800089c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd4937fe640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd480006530 0x7fd4800089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd480009560 0x7fd48000ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 10 monclient(hunting): get_auth_request con 0x7fd480009560 auth_method 0 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:28.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd493fff640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4800035e0 0x7fd480005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd493fff640 10 monclient(hunting): get_auth_request con 0x7fd4800035e0 auth_method 0 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd493fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd493fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd493fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 10 monclient(hunting): handle_auth_done global_id 5903 payload 293 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd480006530 msgr2=0x7fd4800089c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:28.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd480006530 0x7fd4800089c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4800035e0 msgr2=0x7fd480005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4800035e0 0x7fd480005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 10 monclient: _finish_hunting 0 2024-08-19T10:44:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 1 monclient: found mon.noname-i 2024-08-19T10:44:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 1 -- 172.21.15.190:0/1907792803 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48000cc10 con 0x7fd480009560 2024-08-19T10:44:28.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.346+0000 7fd498aa7640 1 -- 172.21.15.190:0/1907792803 reap_dead start 2024-08-19T10:44:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd498aa7640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd480009560 msgr2=0x7fd48000ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd498aa7640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd480009560 0x7fd48000ba20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009f60 tx=0x7fd4880a6000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd48c009b60 con 0x7fd480009560 2024-08-19T10:44:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:28.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 10 monclient: dump: 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:28.368 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd49a50a640 5 monclient: authenticate success, global_id 5903 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd49a50a640 10 monclient: _renew_subs 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd49a50a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48c012980 con 0x7fd480009560 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd49a50a640 1 -- 172.21.15.190:0/1907792803 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd494140920 con 0x7fd480009560 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:28.369 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd48c012510 con 0x7fd480009560 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 10 monclient: dump: 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:28.370 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd491ffb640 4 set_mon_vals no callback set 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd49a50a640 10 monclient: _renew_subs 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd49a50a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd49a50a640 1 -- 172.21.15.190:0/1907792803 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4940a0600 con 0x7fd480009560 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd49a50a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd49a50a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:28.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd49a50a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:28.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd49a50a640 1 -- 172.21.15.190:0/1907792803 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd494139a80 con 0x7fd480009560 2024-08-19T10:44:28.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd48c012b30 con 0x7fd480009560 2024-08-19T10:44:28.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4800407c0 0x7fd480042c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:28.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(215..215 src has 1..215) ==== 12215+0+0 (secure 0 0 0) 0x7fd48c04f420 con 0x7fd480009560 2024-08-19T10:44:28.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd493fff640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4800407c0 0x7fd480042c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:28.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.350+0000 7fd493fff640 10 monclient: get_auth_request con 0x7fd4800407c0 auth_method 0 2024-08-19T10:44:28.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.354+0000 7fd493fff640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4800407c0 0x7fd480042c80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd484007910 tx=0x7fd484008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:28.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.358+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd48c012e20 con 0x7fd480009560 2024-08-19T10:44:28.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.358+0000 7fd490ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:28.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.358+0000 7fd490ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:28.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.530+0000 7fd49a50a640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:28.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.530+0000 7fd49a50a640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:28.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.530+0000 7fd49a50a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:28.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:28.530+0000 7fd49a50a640 1 -- 172.21.15.190:0/1907792803 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fd49409ee00 con 0x7fd480009560 2024-08-19T10:44:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.230+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}]=0 added bucket h20 type host to location {rack=r1} v216) ==== 168+0+0 (secure 0 0 0) 0x7fd48c009920 con 0x7fd480009560 2024-08-19T10:44:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.230+0000 7fd490ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:29.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.230+0000 7fd490ff9640 10 monclient: _finish_command 2 = system:0 added bucket h20 type host to location {rack=r1} 2024-08-19T10:44:29.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.338+0000 7fd49a50a640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:29.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.338+0000 7fd49a50a640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:29.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.338+0000 7fd49a50a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:29.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.338+0000 7fd49a50a640 1 -- 172.21.15.190:0/1907792803 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fd4940008e0 con 0x7fd480009560 2024-08-19T10:44:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.338+0000 7fd490ff9640 1 -- 172.21.15.190:0/1907792803 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h20' already exists v216) ==== 147+0+0 (secure 0 0 0) 0x7fd48c04f1a0 con 0x7fd480009560 2024-08-19T10:44:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.338+0000 7fd490ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h20", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.338+0000 7fd490ff9640 10 monclient: _finish_command 3 = system:0 bucket 'h20' already exists 2024-08-19T10:44:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h20' already exists 2024-08-19T10:44:29.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4800407c0 msgr2=0x7fd480042c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:29.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4800407c0 0x7fd480042c80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd484007910 tx=0x7fd484008040 comp rx=0 tx=0).stop 2024-08-19T10:44:29.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 10 monclient: shutdown 2024-08-19T10:44:29.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:29.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 1 -- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd480009560 msgr2=0x7fd48000ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:29.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd480009560 0x7fd48000ba20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009f60 tx=0x7fd4880a6000 comp rx=0 tx=0).stop 2024-08-19T10:44:29.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 1 -- 172.21.15.190:0/1907792803 shutdown_connections 2024-08-19T10:44:29.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4800407c0 0x7fd480042c80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:29.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 1 --2- 172.21.15.190:0/1907792803 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd480009560 0x7fd48000ba20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:29.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 1 -- 172.21.15.190:0/1907792803 >> 172.21.15.190:0/1907792803 conn(0x7fd4940abee0 msgr2=0x7fd494138ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:29.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 1 -- 172.21.15.190:0/1907792803 shutdown_connections 2024-08-19T10:44:29.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.342+0000 7fd49a50a640 1 -- 172.21.15.190:0/1907792803 wait complete. 2024-08-19T10:44:29.359 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:29.359 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h21 host rack=r1 2024-08-19T10:44:29.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient: get_monmap_and_config 2024-08-19T10:44:29.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient: build_initial_monmap 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient: monmap: 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:29.484166+0000 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:29.484166+0000 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:29.483 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:29.484 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:29.484 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:29.484 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:29.484 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:29.484 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:29.484 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:29.484 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:29.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 1 Processor -- start 2024-08-19T10:44:29.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 1 -- start start 2024-08-19T10:44:29.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient: init 2024-08-19T10:44:29.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:29.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient: _add_conns ranks=[2,7,0,1,6,8,4,10,3,5,9] 2024-08-19T10:44:29.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7dd81396a0 0x7f7dd8139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:29.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient(hunting): picked mon.noname-b con 0x7f7dd81396a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:29.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd8139fc0 0x7f7dd80a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient(hunting): picked mon.noname-i con 0x7f7dd8139fc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7dd80a6e20 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd80a83e0 con 0x7f7dd80a6e20 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd80a4f20 con 0x7f7dd81396a0 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7dd80a2f30 con 0x7f7dd8139fc0 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7de0657640 10 monclient(hunting): authenticate will time out at 1315.054292s 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd8139fc0 0x7f7dd80a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd8139fc0 0x7f7dd80a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42158/0 (socket says 172.21.15.190:42158) 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 1 -- 172.21.15.190:0/2701147545 learned_addr learned my addr 172.21.15.190:0/2701147545 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:29.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 10 monclient(hunting): get_auth_request con 0x7f7dd8139fc0 auth_method 0 2024-08-19T10:44:29.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:29.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:29.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:29.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:29.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:29.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:29.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dde3f3640 1 --2- 172.21.15.190:0/2701147545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7dd81396a0 0x7f7dd8139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:29.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 10 monclient(hunting): handle_auth_done global_id 15352 payload 274 2024-08-19T10:44:29.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 1 -- 172.21.15.190:0/2701147545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7dd81396a0 msgr2=0x7f7dd8139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:29.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 1 --2- 172.21.15.190:0/2701147545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7dd81396a0 0x7f7dd8139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:29.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 1 -- 172.21.15.190:0/2701147545 >> v1:172.21.15.89:6789/0 conn(0x7f7dd80a6e20 legacy=0x7f7dd80a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:29.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 10 monclient: _finish_hunting 0 2024-08-19T10:44:29.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 1 monclient: found mon.noname-i 2024-08-19T10:44:29.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:29.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:29.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.482+0000 7f7dddbf2640 1 -- 172.21.15.190:0/2701147545 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd80a35e0 con 0x7f7dd8139fc0 2024-08-19T10:44:29.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7dddbf2640 1 -- 172.21.15.190:0/2701147545 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd8139fc0 msgr2=0x7f7dd80a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:29.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7dddbf2640 1 --2- 172.21.15.190:0/2701147545 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd8139fc0 0x7f7dd80a68e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40098a0 tx=0x7f7dd401c440 comp rx=0 tx=0).ready entity=mon.7 client_cookie=38a2ec9ebe2e805b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:29.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 1 -- 172.21.15.190:0/2701147545 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7dd40b0070 con 0x7f7dd8139fc0 2024-08-19T10:44:29.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:29.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:29.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 10 monclient: dump: 2024-08-19T10:44:29.493 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:29.493 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:29.493 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:29.493 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:29.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 10 monclient: _finish_auth 0 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 10 monclient: _check_auth_tickets 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 1 -- 172.21.15.190:0/2701147545 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd40aa810 con 0x7f7dd8139fc0 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7de0657640 5 monclient: authenticate success, global_id 15352 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7de0657640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7de0657640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 1 -- 172.21.15.190:0/2701147545 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7dd40a93e0 con 0x7f7dd8139fc0 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7de0657640 4 set_mon_vals no callback set 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7ddd3f1640 10 monclient: dump: 2024-08-19T10:44:29.495 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:29.496 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7de0657640 10 monclient: shutdown 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7de0657640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7de0657640 1 -- 172.21.15.190:0/2701147545 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd8139fc0 msgr2=0x7f7dd80a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.486+0000 7f7de0657640 1 --2- 172.21.15.190:0/2701147545 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd8139fc0 0x7f7dd80a68e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40098a0 tx=0x7f7dd401c440 comp rx=0 tx=0).stop 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 -- 172.21.15.190:0/2701147545 shutdown_connections 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 --2- 172.21.15.190:0/2701147545 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd8139fc0 0x7f7dd80a68e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 --2- 172.21.15.190:0/2701147545 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7dd81396a0 0x7f7dd8139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 -- 172.21.15.190:0/2701147545 >> 172.21.15.190:0/2701147545 conn(0x7f7dd80abee0 msgr2=0x7f7dd80ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 -- 172.21.15.190:0/2701147545 shutdown_connections 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 -- 172.21.15.190:0/2701147545 wait complete. 2024-08-19T10:44:29.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient: build_initial_monmap 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient: monmap: 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:29.498 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 Processor -- start 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 -- start start 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient: init 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient: _add_conns ranks=[8,7,0,6,2,5,4,3,1] 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7dd80a6e20 0x7f7dd81d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient(hunting): picked mon.noname-e con 0x7f7dd80a6e20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:29.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd81396a0 0x7f7dd81387b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:29.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient(hunting): picked mon.noname-i con 0x7f7dd81396a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:29.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd8139fc0 0x7f7dd813d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:29.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient(hunting): picked mon.noname-a con 0x7f7dd8139fc0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:29.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:29.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd813a3a0 con 0x7f7dd8139fc0 2024-08-19T10:44:29.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:29.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7dd80a7930 con 0x7f7dd81396a0 2024-08-19T10:44:29.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:29.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7dd81228a0 con 0x7f7dd80a6e20 2024-08-19T10:44:29.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:29.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7de0657640 10 monclient(hunting): authenticate will time out at 1315.061953s 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd81396a0 0x7f7dd81387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd81396a0 0x7f7dd81387b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42170/0 (socket says 172.21.15.190:42170) 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 1 -- 172.21.15.190:0/3130115264 learned_addr learned my addr 172.21.15.190:0/3130115264 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 10 monclient(hunting): get_auth_request con 0x7f7dd81396a0 auth_method 0 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7ddebf4640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd8139fc0 0x7f7dd813d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dde3f3640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7dd80a6e20 0x7f7dd81d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:29.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7ddebf4640 10 monclient(hunting): get_auth_request con 0x7f7dd8139fc0 auth_method 0 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7ddebf4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7ddebf4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7ddebf4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dde3f3640 10 monclient(hunting): get_auth_request con 0x7f7dd80a6e20 auth_method 0 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dde3f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dde3f3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dde3f3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 10 monclient(hunting): handle_auth_done global_id 15361 payload 824 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 1 -- 172.21.15.190:0/3130115264 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7dd80a6e20 msgr2=0x7f7dd81d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7dd80a6e20 0x7f7dd81d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 1 -- 172.21.15.190:0/3130115264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd8139fc0 msgr2=0x7f7dd813d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:29.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd8139fc0 0x7f7dd813d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:29.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 10 monclient: _finish_hunting 0 2024-08-19T10:44:29.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 1 monclient: found mon.noname-i 2024-08-19T10:44:29.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:29.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:29.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.490+0000 7f7dddbf2640 1 -- 172.21.15.190:0/3130115264 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd80ac470 con 0x7f7dd81396a0 2024-08-19T10:44:29.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dddbf2640 1 -- 172.21.15.190:0/3130115264 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd81396a0 msgr2=0x7f7dd81387b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dddbf2640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd81396a0 0x7f7dd81387b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40ae040 tx=0x7f7dd40191e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 1 -- 172.21.15.190:0/3130115264 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7dd40b0070 con 0x7f7dd81396a0 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 10 monclient: dump: 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:29.505 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:29.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 1 -- 172.21.15.190:0/3130115264 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd40b1040 con 0x7f7dd81396a0 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7de0657640 5 monclient: authenticate success, global_id 15361 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7de0657640 10 monclient: _renew_subs 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7de0657640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7de0657640 1 -- 172.21.15.190:0/3130115264 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd80a26e0 con 0x7f7dd81396a0 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 1 -- 172.21.15.190:0/3130115264 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7dd40044b0 con 0x7f7dd81396a0 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 10 monclient: dump: 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:29.507 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:29.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7de0657640 10 monclient: _renew_subs 2024-08-19T10:44:29.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7de0657640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:29.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7de0657640 1 -- 172.21.15.190:0/3130115264 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd81d2150 con 0x7f7dd81396a0 2024-08-19T10:44:29.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7ddcbf0640 4 set_mon_vals no callback set 2024-08-19T10:44:29.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7de0657640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:29.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7de0657640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:29.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7de0657640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:29.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7de0657640 1 -- 172.21.15.190:0/3130115264 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd8139a80 con 0x7f7dd81396a0 2024-08-19T10:44:29.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 1 -- 172.21.15.190:0/3130115264 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7dd4004040 con 0x7f7dd81396a0 2024-08-19T10:44:29.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dbc03be00 0x7f7dbc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:29.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dceffd640 1 -- 172.21.15.190:0/3130115264 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(216..216 src has 1..216) ==== 12428+0+0 (secure 0 0 0) 0x7f7dd40ea430 con 0x7f7dd81396a0 2024-08-19T10:44:29.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dde3f3640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dbc03be00 0x7f7dbc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:29.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.494+0000 7f7dde3f3640 10 monclient: get_auth_request con 0x7f7dbc03be00 auth_method 0 2024-08-19T10:44:29.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.498+0000 7f7dde3f3640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dbc03be00 0x7f7dbc03e2c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800a970 tx=0x7f7dc8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:29.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.498+0000 7f7dceffd640 1 -- 172.21.15.190:0/3130115264 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7dd4016150 con 0x7f7dd81396a0 2024-08-19T10:44:29.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.498+0000 7f7dceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:29.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.498+0000 7f7dceffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:29.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.666+0000 7f7de0657640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:29.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.666+0000 7f7de0657640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:29.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.666+0000 7f7de0657640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:29.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:29.666+0000 7f7de0657640 1 -- 172.21.15.190:0/3130115264 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f7dd80a3de0 con 0x7f7dd81396a0 2024-08-19T10:44:30.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.450+0000 7f7dceffd640 1 -- 172.21.15.190:0/3130115264 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}]=0 added bucket h21 type host to location {rack=r1} v217) ==== 168+0+0 (secure 0 0 0) 0x7f7dd40a9df0 con 0x7f7dd81396a0 2024-08-19T10:44:30.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.450+0000 7f7dceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:30.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.450+0000 7f7dceffd640 10 monclient: _finish_command 2 = system:0 added bucket h21 type host to location {rack=r1} 2024-08-19T10:44:30.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.558+0000 7f7de0657640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:30.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.558+0000 7f7de0657640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:30.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.558+0000 7f7de0657640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:30.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.558+0000 7f7de0657640 1 -- 172.21.15.190:0/3130115264 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f7dd80008e0 con 0x7f7dd81396a0 2024-08-19T10:44:30.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.558+0000 7f7dceffd640 1 -- 172.21.15.190:0/3130115264 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h21' already exists v217) ==== 147+0+0 (secure 0 0 0) 0x7f7dd40e7040 con 0x7f7dd81396a0 2024-08-19T10:44:30.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.558+0000 7f7dceffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h21", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:30.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.558+0000 7f7dceffd640 10 monclient: _finish_command 3 = system:0 bucket 'h21' already exists 2024-08-19T10:44:30.560 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h21' already exists 2024-08-19T10:44:30.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 -- 172.21.15.190:0/3130115264 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dbc03be00 msgr2=0x7f7dbc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:30.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dbc03be00 0x7f7dbc03e2c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800a970 tx=0x7f7dc8008040 comp rx=0 tx=0).stop 2024-08-19T10:44:30.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 10 monclient: shutdown 2024-08-19T10:44:30.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:30.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 -- 172.21.15.190:0/3130115264 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd81396a0 msgr2=0x7f7dd81387b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:30.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd81396a0 0x7f7dd81387b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40ae040 tx=0x7f7dd40191e0 comp rx=0 tx=0).stop 2024-08-19T10:44:30.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 -- 172.21.15.190:0/3130115264 shutdown_connections 2024-08-19T10:44:30.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7dd8139fc0 0x7f7dd813d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:30.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7dd81396a0 0x7f7dd81387b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:30.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dbc03be00 0x7f7dbc03e2c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:30.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 --2- 172.21.15.190:0/3130115264 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7dd80a6e20 0x7f7dd81d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:30.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 -- 172.21.15.190:0/3130115264 >> 172.21.15.190:0/3130115264 conn(0x7f7dd80abee0 msgr2=0x7f7dd80aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:30.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 -- 172.21.15.190:0/3130115264 shutdown_connections 2024-08-19T10:44:30.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.562+0000 7f7de0657640 1 -- 172.21.15.190:0/3130115264 wait complete. 2024-08-19T10:44:30.578 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:30.578 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h22 host rack=r1 2024-08-19T10:44:30.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient: get_monmap_and_config 2024-08-19T10:44:30.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient: build_initial_monmap 2024-08-19T10:44:30.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient: monmap: 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:30.700079+0000 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:30.700079+0000 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:30.699 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:30.700 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:30.700 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:30.700 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:30.700 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:30.700 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:30.700 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:30.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 1 Processor -- start 2024-08-19T10:44:30.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 1 -- start start 2024-08-19T10:44:30.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient: init 2024-08-19T10:44:30.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:30.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient: _add_conns ranks=[8,1,5,2,3,10,6,9,7,4,0] 2024-08-19T10:44:30.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34201379b0 0x7f3420137d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:30.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient(hunting): picked mon.noname-e con 0x7f34201379b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:30.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f34200a1840 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:30.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34200a5f50 0x7f342013f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:30.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient(hunting): picked mon.noname-h con 0x7f34200a5f50 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:30.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3420072240 con 0x7f34200a1840 2024-08-19T10:44:30.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:30.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f3420073ff0 con 0x7f34200a5f50 2024-08-19T10:44:30.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:30.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3420074240 con 0x7f34201379b0 2024-08-19T10:44:30.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f3425fbc640 10 monclient(hunting): authenticate will time out at 1316.270323s 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341effd640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7f34200a1840 legacy=0x7f34200a5840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34200a5f50 0x7f342013f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34200a5f50 0x7f342013f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:45594/0 (socket says 172.21.15.190:45594) 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 1 -- 172.21.15.190:0/1154497300 learned_addr learned my addr 172.21.15.190:0/1154497300 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 10 monclient(hunting): get_auth_request con 0x7f34200a5f50 auth_method 0 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:30.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 10 monclient(hunting): handle_auth_done global_id 6089 payload 274 2024-08-19T10:44:30.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 1 -- 172.21.15.190:0/1154497300 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34201379b0 msgr2=0x7f3420137d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:30.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 1 --2- 172.21.15.190:0/1154497300 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34201379b0 0x7f3420137d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:30.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 1 -- 172.21.15.190:0/1154497300 >> v1:172.21.15.190:6789/0 conn(0x7f34200a1840 legacy=0x7f34200a5840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:30.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:30.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 1 monclient: found mon.noname-h 2024-08-19T10:44:30.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:30.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:30.705 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.698+0000 7f341ffff640 1 -- 172.21.15.190:0/1154497300 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3420072900 con 0x7f34200a5f50 2024-08-19T10:44:30.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341ffff640 1 -- 172.21.15.190:0/1154497300 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34200a5f50 msgr2=0x7f342013f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:30.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341ffff640 1 --2- 172.21.15.190:0/1154497300 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34200a5f50 0x7f342013f850 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f34100098a0 tx=0x7f341001c3e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=3baa6e8075d07325 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:30.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 1 -- 172.21.15.190:0/1154497300 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34100b0070 con 0x7f34200a5f50 2024-08-19T10:44:30.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:30.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 10 monclient: dump: 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:30.710 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f3425fbc640 5 monclient: authenticate success, global_id 6089 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f3425fbc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 1 -- 172.21.15.190:0/1154497300 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34100aa3a0 con 0x7f34200a5f50 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:30.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 1 -- 172.21.15.190:0/1154497300 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34100aaa00 con 0x7f34200a5f50 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f341e7fc640 10 monclient: dump: 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:30.712 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f3425fbc640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.702+0000 7f3425fbc640 4 set_mon_vals no callback set 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient: shutdown 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 -- 172.21.15.190:0/1154497300 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34200a5f50 msgr2=0x7f342013f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:30.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 --2- 172.21.15.190:0/1154497300 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34200a5f50 0x7f342013f850 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f34100098a0 tx=0x7f341001c3e0 comp rx=0 tx=0).stop 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 -- 172.21.15.190:0/1154497300 shutdown_connections 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 --2- 172.21.15.190:0/1154497300 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34200a5f50 0x7f342013f850 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 --2- 172.21.15.190:0/1154497300 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34201379b0 0x7f3420137d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 -- 172.21.15.190:0/1154497300 >> 172.21.15.190:0/1154497300 conn(0x7f34200aa430 msgr2=0x7f34200a6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 -- 172.21.15.190:0/1154497300 shutdown_connections 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 -- 172.21.15.190:0/1154497300 wait complete. 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient: build_initial_monmap 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient: monmap: 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:30.714 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:30.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 Processor -- start 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 -- start start 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient: init 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient: _add_conns ranks=[4,8,7,0,1,6,2,5,3] 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34200a1840 0x7f342013e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient(hunting): picked mon.noname-c con 0x7f34200a1840 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34200a5f50 0x7f3420136500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient(hunting): picked mon.noname-e con 0x7f34200a5f50 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34201379b0 0x7f342013c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient(hunting): picked mon.noname-i con 0x7f34201379b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f34201382d0 con 0x7f34200a1840 2024-08-19T10:44:30.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f34200a7330 con 0x7f34201379b0 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3420138450 con 0x7f34200a5f50 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f3425fbc640 10 monclient(hunting): authenticate will time out at 1316.277816s 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34201379b0 0x7f342013c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341f7fe640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34200a1840 msgr2=0x7f342013e8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341f7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34200a1840 0x7f342013e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34201379b0 0x7f342013c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42178/0 (socket says 172.21.15.190:42178) 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 1 -- 172.21.15.190:0/60424587 learned_addr learned my addr 172.21.15.190:0/60424587 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 10 monclient(hunting): get_auth_request con 0x7f34201379b0 auth_method 0 2024-08-19T10:44:30.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341effd640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34200a5f50 0x7f3420136500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 10 monclient(hunting): handle_auth_done global_id 15370 payload 824 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 1 -- 172.21.15.190:0/60424587 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34200a5f50 msgr2=0x7f3420136500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34200a5f50 0x7f3420136500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 1 -- 172.21.15.190:0/60424587 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34200a1840 msgr2=0x7f342013e8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34200a1840 0x7f342013e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:30.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:30.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 1 monclient: found mon.noname-i 2024-08-19T10:44:30.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:30.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:30.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341ffff640 1 -- 172.21.15.190:0/60424587 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3420120e90 con 0x7f34201379b0 2024-08-19T10:44:30.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341effd640 10 monclient: get_auth_request con 0x7f34200a5f50 auth_method 0 2024-08-19T10:44:30.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.706+0000 7f341effd640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34200a5f50 0x7f3420136500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:30.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f341ffff640 1 -- 172.21.15.190:0/60424587 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34201379b0 msgr2=0x7f342013c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:30.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f341ffff640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34201379b0 0x7f342013c2e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f34100ae040 tx=0x7f34100191e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:30.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 1 -- 172.21.15.190:0/60424587 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34100b9070 con 0x7f34201379b0 2024-08-19T10:44:30.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:30.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:30.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 10 monclient: dump: 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:30.722 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3425fbc640 5 monclient: authenticate success, global_id 15370 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3425fbc640 10 monclient: _renew_subs 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3425fbc640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3425fbc640 1 -- 172.21.15.190:0/60424587 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34200a3380 con 0x7f34201379b0 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3425fbc640 10 monclient: _renew_subs 2024-08-19T10:44:30.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3425fbc640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3425fbc640 1 -- 172.21.15.190:0/60424587 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f342013f280 con 0x7f34201379b0 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 1 -- 172.21.15.190:0/60424587 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34100039d0 con 0x7f34201379b0 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 1 -- 172.21.15.190:0/60424587 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34100b0040 con 0x7f34201379b0 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f341dffb640 4 set_mon_vals no callback set 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 10 monclient: dump: 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:30.724 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:30.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3425fbc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3425fbc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3425fbc640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3425fbc640 1 -- 172.21.15.190:0/60424587 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3404005090 con 0x7f34201379b0 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 1 -- 172.21.15.190:0/60424587 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3410003b80 con 0x7f34201379b0 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f340003be00 0x7f340003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f3417fff640 1 -- 172.21.15.190:0/60424587 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(217..217 src has 1..217) ==== 12641+0+0 (secure 0 0 0) 0x7f34100e9b50 con 0x7f34201379b0 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f341f7fe640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f340003be00 0x7f340003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.710+0000 7f341f7fe640 10 monclient: get_auth_request con 0x7f340003be00 auth_method 0 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.714+0000 7f341f7fe640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f340003be00 0x7f340003e2c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3418007fd0 tx=0x7f3418007ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.714+0000 7f3417fff640 1 -- 172.21.15.190:0/60424587 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3410016150 con 0x7f34201379b0 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.714+0000 7f3417fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:30.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.714+0000 7f3417fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:30.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.890+0000 7f3425fbc640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:30.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.890+0000 7f3425fbc640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:30.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.890+0000 7f3425fbc640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:30.892 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:30.890+0000 7f3425fbc640 1 -- 172.21.15.190:0/60424587 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f3404005380 con 0x7f34201379b0 2024-08-19T10:44:31.765 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.762+0000 7f3417fff640 1 -- 172.21.15.190:0/60424587 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}]=0 added bucket h22 type host to location {rack=r1} v218) ==== 168+0+0 (secure 0 0 0) 0x7f34100e9e20 con 0x7f34201379b0 2024-08-19T10:44:31.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.762+0000 7f3417fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:31.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.762+0000 7f3417fff640 10 monclient: _finish_command 2 = system:0 added bucket h22 type host to location {rack=r1} 2024-08-19T10:44:31.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.870+0000 7f3425fbc640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:31.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.870+0000 7f3425fbc640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:31.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.870+0000 7f3425fbc640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:31.873 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.870+0000 7f3425fbc640 1 -- 172.21.15.190:0/60424587 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f34040022c0 con 0x7f34201379b0 2024-08-19T10:44:31.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.874+0000 7f3417fff640 1 -- 172.21.15.190:0/60424587 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h22' already exists v218) ==== 147+0+0 (secure 0 0 0) 0x7f34100bcd70 con 0x7f34201379b0 2024-08-19T10:44:31.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.874+0000 7f3417fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h22", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:31.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.874+0000 7f3417fff640 10 monclient: _finish_command 3 = system:0 bucket 'h22' already exists 2024-08-19T10:44:31.876 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h22' already exists 2024-08-19T10:44:31.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 -- 172.21.15.190:0/60424587 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f340003be00 msgr2=0x7f340003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:31.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f340003be00 0x7f340003e2c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3418007fd0 tx=0x7f3418007ac0 comp rx=0 tx=0).stop 2024-08-19T10:44:31.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 10 monclient: shutdown 2024-08-19T10:44:31.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:31.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 -- 172.21.15.190:0/60424587 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34201379b0 msgr2=0x7f342013c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:31.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34201379b0 0x7f342013c2e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f34100ae040 tx=0x7f34100191e0 comp rx=0 tx=0).stop 2024-08-19T10:44:31.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 -- 172.21.15.190:0/60424587 shutdown_connections 2024-08-19T10:44:31.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34201379b0 0x7f342013c2e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:31.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f340003be00 0x7f340003e2c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:31.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34200a5f50 0x7f3420136500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:31.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 --2- 172.21.15.190:0/60424587 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f34200a1840 0x7f342013e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:31.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 -- 172.21.15.190:0/60424587 >> 172.21.15.190:0/60424587 conn(0x7f34200aa430 msgr2=0x7f34200a1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:31.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 -- 172.21.15.190:0/60424587 shutdown_connections 2024-08-19T10:44:31.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:31.878+0000 7f3425fbc640 1 -- 172.21.15.190:0/60424587 wait complete. 2024-08-19T10:44:31.894 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:31.894 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h23 host rack=r1 2024-08-19T10:44:32.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient: get_monmap_and_config 2024-08-19T10:44:32.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient: build_initial_monmap 2024-08-19T10:44:32.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient: monmap: 2024-08-19T10:44:32.006 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:32.007 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:32.007 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:32.007982+0000 2024-08-19T10:44:32.007 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:32.007982+0000 2024-08-19T10:44:32.007 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:32.007 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:32.007 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:32.007 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:32.007 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:32.007 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:32.007 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:32.008 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:32.008 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:32.008 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:32.008 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:32.008 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:32.008 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:32.008 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:32.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 1 Processor -- start 2024-08-19T10:44:32.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 1 -- start start 2024-08-19T10:44:32.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient: init 2024-08-19T10:44:32.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:32.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient: _add_conns ranks=[0,7,8,9,3,1,2,10,6,4,5] 2024-08-19T10:44:32.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f35781396a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:32.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f357813a190 0x7f357813eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:32.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient(hunting): picked mon.noname-i con 0x7f357813a190 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:32.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f357813f0a0 0x7f3578141490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:32.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient(hunting): picked mon.noname-e con 0x7f357813f0a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:32.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35780a83e0 con 0x7f35781396a0 2024-08-19T10:44:32.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:32.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f35780a4f20 con 0x7f357813a190 2024-08-19T10:44:32.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:32.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f35780a2f30 con 0x7f357813f0a0 2024-08-19T10:44:32.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:32.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357fb7f640 10 monclient(hunting): authenticate will time out at 1317.578090s 2024-08-19T10:44:32.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f357813a190 0x7f357813eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:32.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f357813a190 0x7f357813eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42184/0 (socket says 172.21.15.190:42184) 2024-08-19T10:44:32.012 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 1 -- 172.21.15.190:0/166989162 learned_addr learned my addr 172.21.15.190:0/166989162 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 10 monclient(hunting): get_auth_request con 0x7f357813a190 auth_method 0 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357e11c640 1 --2- 172.21.15.190:0/166989162 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f357813f0a0 0x7f3578141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357e11c640 10 monclient(hunting): get_auth_request con 0x7f357813f0a0 auth_method 0 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357e11c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357e11c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:32.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357e11c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:32.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 10 monclient(hunting): handle_auth_done global_id 15379 payload 274 2024-08-19T10:44:32.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 1 -- 172.21.15.190:0/166989162 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f357813f0a0 msgr2=0x7f3578141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:32.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 1 --2- 172.21.15.190:0/166989162 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f357813f0a0 0x7f3578141490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:32.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 1 -- 172.21.15.190:0/166989162 >> v1:172.21.15.89:6789/0 conn(0x7f35781396a0 legacy=0x7f3578139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:32.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 10 monclient: _finish_hunting 0 2024-08-19T10:44:32.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 1 monclient: found mon.noname-i 2024-08-19T10:44:32.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:32.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:32.014 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.006+0000 7f357d11a640 1 -- 172.21.15.190:0/166989162 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35780a35e0 con 0x7f357813a190 2024-08-19T10:44:32.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 1 -- 172.21.15.190:0/166989162 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f357813a190 msgr2=0x7f357813eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:32.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 1 --2- 172.21.15.190:0/166989162 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f357813a190 0x7f357813eb60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f35740098d0 tx=0x7f357401c440 comp rx=0 tx=0).ready entity=mon.7 client_cookie=11c48204b157c0a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:32.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 1 -- 172.21.15.190:0/166989162 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f35740b0070 con 0x7f357813a190 2024-08-19T10:44:32.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:32.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:32.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 10 monclient: dump: 2024-08-19T10:44:32.018 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:32.018 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:32.019 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 10 monclient: _finish_auth 0 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 10 monclient: _check_auth_tickets 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 5 monclient: authenticate success, global_id 15379 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 1 -- 172.21.15.190:0/166989162 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35740aa810 con 0x7f357813a190 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 1 -- 172.21.15.190:0/166989162 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f35740a9300 con 0x7f357813a190 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:32.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 4 set_mon_vals no callback set 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357c919640 10 monclient: dump: 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:32.021 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient: shutdown 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 -- 172.21.15.190:0/166989162 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f357813a190 msgr2=0x7f357813eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 --2- 172.21.15.190:0/166989162 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f357813a190 0x7f357813eb60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f35740098d0 tx=0x7f357401c440 comp rx=0 tx=0).stop 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 -- 172.21.15.190:0/166989162 shutdown_connections 2024-08-19T10:44:32.022 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 --2- 172.21.15.190:0/166989162 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f357813f0a0 0x7f3578141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 --2- 172.21.15.190:0/166989162 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f357813a190 0x7f357813eb60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 -- 172.21.15.190:0/166989162 >> 172.21.15.190:0/166989162 conn(0x7f35780abee0 msgr2=0x7f35780ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 -- 172.21.15.190:0/166989162 shutdown_connections 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 -- 172.21.15.190:0/166989162 wait complete. 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient: build_initial_monmap 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient: monmap: 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:32.023 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 Processor -- start 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 -- start start 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient: init 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:32.024 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient: _add_conns ranks=[8,3,2,1,4,7,5,0,6] 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f35781396a0 0x7f357812e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient(hunting): picked mon.noname-e con 0x7f35781396a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357813a190 0x7f357812edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient(hunting): picked mon.noname-g con 0x7f357813a190 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357813f0a0 0x7f3578136720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient(hunting): picked mon.noname-b con 0x7f357813f0a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f35781441f0 con 0x7f357813f0a0 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f35780a78f0 con 0x7f357813a190 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:32.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3578144370 con 0x7f35781396a0 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357fb7f640 10 monclient(hunting): authenticate will time out at 1317.586747s 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357813a190 0x7f357812edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357813a190 0x7f357812edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:39304/0 (socket says 172.21.15.190:39304) 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 1 -- 172.21.15.190:0/405532559 learned_addr learned my addr 172.21.15.190:0/405532559 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 10 monclient(hunting): get_auth_request con 0x7f357813a190 auth_method 0 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:32.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d91b640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f35781396a0 0x7f357812e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357e11c640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357813f0a0 0x7f3578136720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d91b640 10 monclient(hunting): get_auth_request con 0x7f35781396a0 auth_method 0 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d91b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d91b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d91b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 10 monclient(hunting): handle_auth_done global_id 34446 payload 824 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 1 -- 172.21.15.190:0/405532559 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f35781396a0 msgr2=0x7f357812e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f35781396a0 0x7f357812e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 1 -- 172.21.15.190:0/405532559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357813f0a0 msgr2=0x7f3578136720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357813f0a0 0x7f3578136720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 10 monclient: _finish_hunting 0 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 1 monclient: found mon.noname-g 2024-08-19T10:44:32.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:32.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:32.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357d11a640 1 -- 172.21.15.190:0/405532559 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35780a7030 con 0x7f357813a190 2024-08-19T10:44:32.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357e11c640 10 monclient: get_auth_request con 0x7f357813f0a0 auth_method 0 2024-08-19T10:44:32.028 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.014+0000 7f357e11c640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357813f0a0 0x7f3578136720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:32.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357d11a640 1 -- 172.21.15.190:0/405532559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357813a190 msgr2=0x7f357812edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:32.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357d11a640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357813a190 0x7f357812edf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f35740ae040 tx=0x7f35740191e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:32.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 1 -- 172.21.15.190:0/405532559 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f35740b9070 con 0x7f357813a190 2024-08-19T10:44:32.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:32.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 10 monclient: dump: 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:32.031 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 1 -- 172.21.15.190:0/405532559 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35740044b0 con 0x7f357813a190 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 1 -- 172.21.15.190:0/405532559 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f35740b0040 con 0x7f357813a190 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 10 monclient: dump: 2024-08-19T10:44:32.032 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:32.033 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357fb7f640 5 monclient: authenticate success, global_id 34446 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356ffff640 4 set_mon_vals no callback set 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357fb7f640 10 monclient: _renew_subs 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357fb7f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357fb7f640 1 -- 172.21.15.190:0/405532559 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35780a7460 con 0x7f357813a190 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357fb7f640 10 monclient: _renew_subs 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357fb7f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357fb7f640 1 -- 172.21.15.190:0/405532559 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f357813b090 con 0x7f357813a190 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357fb7f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357fb7f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357fb7f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:32.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357fb7f640 1 -- 172.21.15.190:0/405532559 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3578139a80 con 0x7f357813a190 2024-08-19T10:44:32.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 1 -- 172.21.15.190:0/405532559 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f35740aac70 con 0x7f357813a190 2024-08-19T10:44:32.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f356403bdf0 0x7f356403e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:32.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f356e7fc640 1 -- 172.21.15.190:0/405532559 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(218..218 src has 1..218) ==== 12854+0+0 (secure 0 0 0) 0x7f35740e99c0 con 0x7f357813a190 2024-08-19T10:44:32.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357d91b640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f356403bdf0 0x7f356403e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:32.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.018+0000 7f357d91b640 10 monclient: get_auth_request con 0x7f356403bdf0 auth_method 0 2024-08-19T10:44:32.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.022+0000 7f357d91b640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f356403bdf0 0x7f356403e2b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3570005530 tx=0x7f3570009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:32.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.022+0000 7f356e7fc640 1 -- 172.21.15.190:0/405532559 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3574016150 con 0x7f357813a190 2024-08-19T10:44:32.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.022+0000 7f356e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:32.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.022+0000 7f356e7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:32.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.186+0000 7f357fb7f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:32.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.186+0000 7f357fb7f640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:32.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.186+0000 7f357fb7f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:32.189 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.186+0000 7f357fb7f640 1 -- 172.21.15.190:0/405532559 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f3578138110 con 0x7f357813a190 2024-08-19T10:44:32.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.782+0000 7f356e7fc640 1 -- 172.21.15.190:0/405532559 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}]=0 added bucket h23 type host to location {rack=r1} v219) ==== 168+0+0 (secure 0 0 0) 0x7f35740e6420 con 0x7f357813a190 2024-08-19T10:44:32.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.782+0000 7f356e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:32.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.782+0000 7f356e7fc640 10 monclient: _finish_command 2 = system:0 added bucket h23 type host to location {rack=r1} 2024-08-19T10:44:32.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.882+0000 7f357fb7f640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:32.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.882+0000 7f357fb7f640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:32.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.882+0000 7f357fb7f640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:32.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.882+0000 7f357fb7f640 1 -- 172.21.15.190:0/405532559 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f357813b470 con 0x7f357813a190 2024-08-19T10:44:32.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.886+0000 7f356e7fc640 1 -- 172.21.15.190:0/405532559 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h23' already exists v219) ==== 147+0+0 (secure 0 0 0) 0x7f35740abe00 con 0x7f357813a190 2024-08-19T10:44:32.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.886+0000 7f356e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h23", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:32.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.886+0000 7f356e7fc640 10 monclient: _finish_command 3 = system:0 bucket 'h23' already exists 2024-08-19T10:44:32.887 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h23' already exists 2024-08-19T10:44:32.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.886+0000 7f357fb7f640 1 -- 172.21.15.190:0/405532559 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f356403bdf0 msgr2=0x7f356403e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:32.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f356403bdf0 0x7f356403e2b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3570005530 tx=0x7f3570009040 comp rx=0 tx=0).stop 2024-08-19T10:44:32.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 10 monclient: shutdown 2024-08-19T10:44:32.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:32.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 1 -- 172.21.15.190:0/405532559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357813a190 msgr2=0x7f357812edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:32.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357813a190 0x7f357812edf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f35740ae040 tx=0x7f35740191e0 comp rx=0 tx=0).stop 2024-08-19T10:44:32.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 1 -- 172.21.15.190:0/405532559 shutdown_connections 2024-08-19T10:44:32.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f357813f0a0 0x7f3578136720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:32.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f357813a190 0x7f357812edf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:32.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f356403bdf0 0x7f356403e2b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:32.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 1 --2- 172.21.15.190:0/405532559 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f35781396a0 0x7f357812e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:32.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 1 -- 172.21.15.190:0/405532559 >> 172.21.15.190:0/405532559 conn(0x7f35780abee0 msgr2=0x7f3578134ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:32.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 1 -- 172.21.15.190:0/405532559 shutdown_connections 2024-08-19T10:44:32.891 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:32.890+0000 7f357fb7f640 1 -- 172.21.15.190:0/405532559 wait complete. 2024-08-19T10:44:32.905 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:32.905 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h24 host rack=r1 2024-08-19T10:44:33.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.022+0000 7f7f660af640 10 monclient: get_monmap_and_config 2024-08-19T10:44:33.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient: build_initial_monmap 2024-08-19T10:44:33.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient: monmap: 2024-08-19T10:44:33.026 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:33.026 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:33.026 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:33.027611+0000 2024-08-19T10:44:33.026 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:33.027611+0000 2024-08-19T10:44:33.026 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:33.026 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:33.027 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:33.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 1 Processor -- start 2024-08-19T10:44:33.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 1 -- start start 2024-08-19T10:44:33.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient: init 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient: _add_conns ranks=[8,5,7,0,6,4,1,2,9,3,10] 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7f601396a0 0x7f7f60139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient(hunting): picked mon.noname-e con 0x7f7f601396a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7f60139fc0 0x7f7f600a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient(hunting): picked mon.noname-h con 0x7f7f60139fc0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f600a6e20 0x7f7f600a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient(hunting): picked mon.noname-i con 0x7f7f600a6e20 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7f600a4f20 con 0x7f7f60139fc0 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:33.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7f600a83e0 con 0x7f7f600a6e20 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7f600a2f30 con 0x7f7f601396a0 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f660af640 10 monclient(hunting): authenticate will time out at 1318.597463s 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f600a6e20 0x7f7f600a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f600a6e20 0x7f7f600a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:42188/0 (socket says 172.21.15.190:42188) 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 1 -- 172.21.15.190:0/4183895345 learned_addr learned my addr 172.21.15.190:0/4183895345 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 10 monclient(hunting): get_auth_request con 0x7f7f600a6e20 auth_method 0 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5effd640 1 --2- 172.21.15.190:0/4183895345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7f60139fc0 0x7f7f600a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:33.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5effd640 10 monclient(hunting): get_auth_request con 0x7f7f60139fc0 auth_method 0 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5f7fe640 1 --2- 172.21.15.190:0/4183895345 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7f601396a0 0x7f7f60139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 10 monclient(hunting): handle_auth_done global_id 15388 payload 274 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 1 -- 172.21.15.190:0/4183895345 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7f601396a0 msgr2=0x7f7f60139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:33.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 1 --2- 172.21.15.190:0/4183895345 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7f601396a0 0x7f7f60139a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 1 -- 172.21.15.190:0/4183895345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7f60139fc0 msgr2=0x7f7f600a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:33.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 1 --2- 172.21.15.190:0/4183895345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7f60139fc0 0x7f7f600a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:33.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 1 monclient: found mon.noname-i 2024-08-19T10:44:33.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:33.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:33.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5ffff640 1 -- 172.21.15.190:0/4183895345 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f600a35e0 con 0x7f7f600a6e20 2024-08-19T10:44:33.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5f7fe640 10 monclient: get_auth_request con 0x7f7f601396a0 auth_method 0 2024-08-19T10:44:33.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.026+0000 7f7f5f7fe640 1 --2- 172.21.15.190:0/4183895345 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7f601396a0 0x7f7f60139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:33.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5ffff640 1 -- 172.21.15.190:0/4183895345 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f600a6e20 msgr2=0x7f7f600a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:33.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5ffff640 1 --2- 172.21.15.190:0/4183895345 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f600a6e20 0x7f7f600a7220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009eb0 tx=0x7f7f5801c8f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=88e6238e56ba4903 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:33.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 1 -- 172.21.15.190:0/4183895345 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7f580090d0 con 0x7f7f600a6e20 2024-08-19T10:44:33.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:33.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:33.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 10 monclient: dump: 2024-08-19T10:44:33.037 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:33.037 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:33.038 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 5 monclient: authenticate success, global_id 15388 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 1 -- 172.21.15.190:0/4183895345 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f580a6950 con 0x7f7f600a6e20 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 4 set_mon_vals no callback set 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 1 -- 172.21.15.190:0/4183895345 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7f580af480 con 0x7f7f600a6e20 2024-08-19T10:44:33.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f5e7fc640 10 monclient: dump: 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:33.040 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 10 monclient: shutdown 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 1 -- 172.21.15.190:0/4183895345 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f600a6e20 msgr2=0x7f7f600a7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 1 --2- 172.21.15.190:0/4183895345 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f600a6e20 0x7f7f600a7220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009eb0 tx=0x7f7f5801c8f0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 1 -- 172.21.15.190:0/4183895345 shutdown_connections 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 1 --2- 172.21.15.190:0/4183895345 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f600a6e20 0x7f7f600a7220 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 1 --2- 172.21.15.190:0/4183895345 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7f60139fc0 0x7f7f600a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 1 --2- 172.21.15.190:0/4183895345 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7f601396a0 0x7f7f60139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 1 -- 172.21.15.190:0/4183895345 >> 172.21.15.190:0/4183895345 conn(0x7f7f600abee0 msgr2=0x7f7f600ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 1 -- 172.21.15.190:0/4183895345 shutdown_connections 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.030+0000 7f7f660af640 1 -- 172.21.15.190:0/4183895345 wait complete. 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient: build_initial_monmap 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient: monmap: 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:33.042 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 1 Processor -- start 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 1 -- start start 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient: init 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient: _add_conns ranks=[1,3,2,4,5,7,0,8,6] 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f600a6e20 0x7f7f60134910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:33.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient(hunting): picked mon.noname-f con 0x7f7f600a6e20 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f601396a0 0x7f7f60134e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient(hunting): picked mon.noname-g con 0x7f7f601396a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f60139fc0 0x7f7f6009ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient(hunting): picked mon.noname-b con 0x7f7f60139fc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6013a3a0 con 0x7f7f600a6e20 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7f60143390 con 0x7f7f60139fc0 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7f601408e0 con 0x7f7f601396a0 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f660af640 10 monclient(hunting): authenticate will time out at 1318.605192s 2024-08-19T10:44:33.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5f7fe640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f600a6e20 msgr2=0x7f7f60134910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f600a6e20 0x7f7f60134910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f601396a0 0x7f7f60134e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f601396a0 0x7f7f60134e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:39316/0 (socket says 172.21.15.190:39316) 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 1 -- 172.21.15.190:0/2824256139 learned_addr learned my addr 172.21.15.190:0/2824256139 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 10 monclient(hunting): get_auth_request con 0x7f7f601396a0 auth_method 0 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5ffff640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f60139fc0 0x7f7f6009ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:33.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 10 monclient(hunting): handle_auth_done global_id 34455 payload 824 2024-08-19T10:44:33.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 1 -- 172.21.15.190:0/2824256139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f60139fc0 msgr2=0x7f7f6009ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:33.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f60139fc0 0x7f7f6009ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 1 -- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f600a6e20 msgr2=0x7f7f60134910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:33.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f600a6e20 0x7f7f60134910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:33.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 1 monclient: found mon.noname-g 2024-08-19T10:44:33.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:33.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:33.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 1 -- 172.21.15.190:0/2824256139 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f60135780 con 0x7f7f601396a0 2024-08-19T10:44:33.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5ffff640 10 monclient: get_auth_request con 0x7f7f60139fc0 auth_method 0 2024-08-19T10:44:33.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5ffff640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f60139fc0 0x7f7f6009ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:33.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 1 -- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f601396a0 msgr2=0x7f7f60134e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:33.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.034+0000 7f7f5effd640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f601396a0 0x7f7f60134e50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400e870 tx=0x7f7f5400ec70 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 1 -- 172.21.15.190:0/2824256139 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7f54017070 con 0x7f7f601396a0 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 10 monclient: dump: 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:33.049 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:33.050 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:33.050 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:33.050 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:33.050 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:33.050 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:33.050 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:33.050 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:33.050 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:33.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:33.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 1 -- 172.21.15.190:0/2824256139 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f54013470 con 0x7f7f601396a0 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 1 -- 172.21.15.190:0/2824256139 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7f54013a90 con 0x7f7f601396a0 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 10 monclient: dump: 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:33.051 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:33.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f5dffb640 4 set_mon_vals no callback set 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f660af640 5 monclient: authenticate success, global_id 34455 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f660af640 10 monclient: _renew_subs 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f660af640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f660af640 1 -- 172.21.15.190:0/2824256139 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f600ac5a0 con 0x7f7f601396a0 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f660af640 10 monclient: _renew_subs 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f660af640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f660af640 1 -- 172.21.15.190:0/2824256139 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6009ffd0 con 0x7f7f601396a0 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f660af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f660af640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f660af640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f660af640 1 -- 172.21.15.190:0/2824256139 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f600ac7d0 con 0x7f7f601396a0 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 1 -- 172.21.15.190:0/2824256139 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7f5401a070 con 0x7f7f601396a0 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f4c03be30 0x7f7f4c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:33.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f47fff640 1 -- 172.21.15.190:0/2824256139 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(219..219 src has 1..219) ==== 13067+0+0 (secure 0 0 0) 0x7f7f5400f040 con 0x7f7f601396a0 2024-08-19T10:44:33.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f5f7fe640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f4c03be30 0x7f7f4c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:33.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.038+0000 7f7f5f7fe640 10 monclient: get_auth_request con 0x7f7f4c03be30 auth_method 0 2024-08-19T10:44:33.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.042+0000 7f7f5f7fe640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f4c03be30 0x7f7f4c03e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7f50006fd0 tx=0x7f7f50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:33.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.042+0000 7f7f47fff640 1 -- 172.21.15.190:0/2824256139 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7f600ac7d0 con 0x7f7f601396a0 2024-08-19T10:44:33.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.042+0000 7f7f47fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:33.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.042+0000 7f7f47fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:33.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.214+0000 7f7f660af640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:33.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.214+0000 7f7f660af640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:33.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.214+0000 7f7f660af640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:33.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.214+0000 7f7f660af640 1 -- 172.21.15.190:0/2824256139 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f7f60139a80 con 0x7f7f601396a0 2024-08-19T10:44:33.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.782+0000 7f7f47fff640 1 -- 172.21.15.190:0/2824256139 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}]=0 added bucket h24 type host to location {rack=r1} v220) ==== 168+0+0 (secure 0 0 0) 0x7f7f54021030 con 0x7f7f601396a0 2024-08-19T10:44:33.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.782+0000 7f7f47fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:33.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.782+0000 7f7f47fff640 10 monclient: _finish_command 2 = system:0 added bucket h24 type host to location {rack=r1} 2024-08-19T10:44:33.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.882+0000 7f7f660af640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:33.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.882+0000 7f7f660af640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:33.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.882+0000 7f7f660af640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:33.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.882+0000 7f7f660af640 1 -- 172.21.15.190:0/2824256139 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f7f600008e0 con 0x7f7f601396a0 2024-08-19T10:44:33.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f47fff640 1 -- 172.21.15.190:0/2824256139 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h24' already exists v220) ==== 147+0+0 (secure 0 0 0) 0x7f7f5404da20 con 0x7f7f601396a0 2024-08-19T10:44:33.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f47fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h24", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:33.887 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f47fff640 10 monclient: _finish_command 3 = system:0 bucket 'h24' already exists 2024-08-19T10:44:33.887 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h24' already exists 2024-08-19T10:44:33.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 -- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f4c03be30 msgr2=0x7f7f4c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:33.889 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f4c03be30 0x7f7f4c03e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7f50006fd0 tx=0x7f7f50008040 comp rx=0 tx=0).stop 2024-08-19T10:44:33.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 10 monclient: shutdown 2024-08-19T10:44:33.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:33.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 -- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f601396a0 msgr2=0x7f7f60134e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:33.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f601396a0 0x7f7f60134e50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400e870 tx=0x7f7f5400ec70 comp rx=0 tx=0).stop 2024-08-19T10:44:33.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 -- 172.21.15.190:0/2824256139 shutdown_connections 2024-08-19T10:44:33.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f4c03be30 0x7f7f4c03e2f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f60139fc0 0x7f7f6009ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f601396a0 0x7f7f60134e50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 --2- 172.21.15.190:0/2824256139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f600a6e20 0x7f7f60134910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:33.890 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 -- 172.21.15.190:0/2824256139 >> 172.21.15.190:0/2824256139 conn(0x7f7f600abee0 msgr2=0x7f7f60138b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:33.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 -- 172.21.15.190:0/2824256139 shutdown_connections 2024-08-19T10:44:33.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:33.886+0000 7f7f660af640 1 -- 172.21.15.190:0/2824256139 wait complete. 2024-08-19T10:44:33.903 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:33.904 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h25 host rack=r1 2024-08-19T10:44:34.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient: get_monmap_and_config 2024-08-19T10:44:34.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient: build_initial_monmap 2024-08-19T10:44:34.023 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient: monmap: 2024-08-19T10:44:34.023 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:34.023 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:34.024 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:34.024261+0000 2024-08-19T10:44:34.024 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:34.024261+0000 2024-08-19T10:44:34.024 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:34.024 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:34.024 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:34.024 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:34.024 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:34.024 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:34.025 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:34.025 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:34.025 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:34.025 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:34.025 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:34.025 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:34.025 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:34.025 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:34.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 1 Processor -- start 2024-08-19T10:44:34.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 1 -- start start 2024-08-19T10:44:34.026 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient: init 2024-08-19T10:44:34.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:34.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient: _add_conns ranks=[3,1,6,7,4,2,10,5,0,8,9] 2024-08-19T10:44:34.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac139a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:34.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient(hunting): picked mon.noname-g con 0x7fcdac1396a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:34.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcdac139fc0 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:34.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 0x7fcdac0a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:34.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient(hunting): picked mon.noname-d con 0x7fcdac0a6ff0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:34.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdac0a83e0 con 0x7fcdac139fc0 2024-08-19T10:44:34.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:34.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcdac0a4f20 con 0x7fcdac1396a0 2024-08-19T10:44:34.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:34.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fcdac0a2f30 con 0x7fcdac0a6ff0 2024-08-19T10:44:34.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:34.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb3a57640 10 monclient(hunting): authenticate will time out at 1319.594122s 2024-08-19T10:44:34.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb0ff2640 1 -- >> v1:172.21.15.190:6789/0 conn(0x7fcdac139fc0 legacy=0x7fcdac0a68e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.190:6789/0 2024-08-19T10:44:34.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:34.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb1ff4640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 msgr2=0x7fcdac0a73f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:34.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb1ff4640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 0x7fcdac0a73f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:34.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:39326/0 (socket says 172.21.15.190:39326) 2024-08-19T10:44:34.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 1 -- 172.21.15.190:0/1985762153 learned_addr learned my addr 172.21.15.190:0/1985762153 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:34.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 10 monclient(hunting): get_auth_request con 0x7fcdac1396a0 auth_method 0 2024-08-19T10:44:34.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:34.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:34.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:34.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:34.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:34.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:34.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 10 monclient(hunting): handle_auth_done global_id 34464 payload 274 2024-08-19T10:44:34.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 1 -- 172.21.15.190:0/1985762153 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 msgr2=0x7fcdac0a73f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:34.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 1 --2- 172.21.15.190:0/1985762153 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 0x7fcdac0a73f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:34.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 1 -- 172.21.15.190:0/1985762153 >> v1:172.21.15.190:6789/0 conn(0x7fcdac139fc0 legacy=0x7fcdac0a68e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:34.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 10 monclient: _finish_hunting 0 2024-08-19T10:44:34.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 1 monclient: found mon.noname-g 2024-08-19T10:44:34.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:34.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:34.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.022+0000 7fcdb17f3640 1 -- 172.21.15.190:0/1985762153 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdac0a35e0 con 0x7fcdac1396a0 2024-08-19T10:44:34.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb17f3640 1 -- 172.21.15.190:0/1985762153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 msgr2=0x7fcdac139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:34.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb17f3640 1 --2- 172.21.15.190:0/1985762153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac139a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcda00098d0 tx=0x7fcda001c460 comp rx=0 tx=0).ready entity=mon.3 client_cookie=9258c5d2bae689a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:34.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 1 -- 172.21.15.190:0/1985762153 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcda00b0070 con 0x7fcdac1396a0 2024-08-19T10:44:34.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:34.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:34.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 10 monclient: dump: 2024-08-19T10:44:34.040 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:34.040 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:34.040 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:34.040 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:34.040 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:34.040 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:34.040 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:34.040 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 10 monclient: _finish_auth 0 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 1 -- 172.21.15.190:0/1985762153 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda00aa810 con 0x7fcdac1396a0 2024-08-19T10:44:34.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 5 monclient: authenticate success, global_id 34464 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 1 -- 172.21.15.190:0/1985762153 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcda00a93e0 con 0x7fcdac1396a0 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 4 set_mon_vals no callback set 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcd9ffff640 10 monclient: dump: 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:34.042 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:34.043 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient: shutdown 2024-08-19T10:44:34.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:34.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 -- 172.21.15.190:0/1985762153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 msgr2=0x7fcdac139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:34.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 --2- 172.21.15.190:0/1985762153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac139a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcda00098d0 tx=0x7fcda001c460 comp rx=0 tx=0).stop 2024-08-19T10:44:34.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 -- 172.21.15.190:0/1985762153 shutdown_connections 2024-08-19T10:44:34.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 --2- 172.21.15.190:0/1985762153 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 0x7fcdac0a73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:34.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 --2- 172.21.15.190:0/1985762153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac139a80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:34.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 -- 172.21.15.190:0/1985762153 >> 172.21.15.190:0/1985762153 conn(0x7fcdac0abee0 msgr2=0x7fcdac0ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:34.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 -- 172.21.15.190:0/1985762153 shutdown_connections 2024-08-19T10:44:34.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 -- 172.21.15.190:0/1985762153 wait complete. 2024-08-19T10:44:34.044 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient: build_initial_monmap 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient: monmap: 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:34.045 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 Processor -- start 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 -- start start 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient: init 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient: _add_conns ranks=[6,3,1,4,5,8,0,7,2] 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 0x7fcdac1d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient(hunting): picked mon.noname-d con 0x7fcdac0a6ff0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:34.046 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac138950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient(hunting): picked mon.noname-g con 0x7fcdac1396a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdac139fc0 0x7fcdac13dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient(hunting): picked mon.noname-f con 0x7fcdac139fc0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac13a3a0 con 0x7fcdac139fc0 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcdac0a7930 con 0x7fcdac1396a0 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fcdac140110 con 0x7fcdac0a6ff0 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb3a57640 10 monclient(hunting): authenticate will time out at 1319.603344s 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb1ff4640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdac139fc0 msgr2=0x7fcdac13dc30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:34.047 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac138950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb1ff4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdac139fc0 0x7fcdac13dc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac138950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:39336/0 (socket says 172.21.15.190:39336) 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 1 -- 172.21.15.190:0/233236758 learned_addr learned my addr 172.21.15.190:0/233236758 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 10 monclient(hunting): get_auth_request con 0x7fcdac1396a0 auth_method 0 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb17f3640 1 -- 172.21.15.190:0/233236758 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 msgr2=0x7fcdac1d13b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb17f3640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 0x7fcdac1d13b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:34.048 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 10 monclient(hunting): handle_auth_done global_id 34473 payload 824 2024-08-19T10:44:34.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 1 -- 172.21.15.190:0/233236758 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 msgr2=0x7fcdac1d13b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:34.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 0x7fcdac1d13b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:34.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 1 -- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdac139fc0 msgr2=0x7fcdac13dc30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:34.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdac139fc0 0x7fcdac13dc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:34.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 10 monclient: _finish_hunting 0 2024-08-19T10:44:34.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 1 monclient: found mon.noname-g 2024-08-19T10:44:34.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:34.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:34.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.030+0000 7fcdb0ff2640 1 -- 172.21.15.190:0/233236758 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdac13f5b0 con 0x7fcdac1396a0 2024-08-19T10:44:34.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb0ff2640 1 -- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 msgr2=0x7fcdac138950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb0ff2640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac138950 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcda800b580 tx=0x7fcda800ba40 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 1 -- 172.21.15.190:0/233236758 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcda8016070 con 0x7fcdac1396a0 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 10 monclient: dump: 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:34.050 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 1 -- 172.21.15.190:0/233236758 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda8003de0 con 0x7fcdac1396a0 2024-08-19T10:44:34.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb3a57640 5 monclient: authenticate success, global_id 34473 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb3a57640 10 monclient: _renew_subs 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb3a57640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb3a57640 1 -- 172.21.15.190:0/233236758 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdac122a60 con 0x7fcdac1396a0 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 1 -- 172.21.15.190:0/233236758 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcda800f3e0 con 0x7fcdac1396a0 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 10 monclient: dump: 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:34.052 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:34.053 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9effd640 4 set_mon_vals no callback set 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb3a57640 10 monclient: _renew_subs 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb3a57640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb3a57640 1 -- 172.21.15.190:0/233236758 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdac1d21d0 con 0x7fcdac1396a0 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb3a57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb3a57640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb3a57640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb3a57640 1 -- 172.21.15.190:0/233236758 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdac0ac6d0 con 0x7fcdac1396a0 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 1 -- 172.21.15.190:0/233236758 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fcda80050d0 con 0x7fcdac1396a0 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd9003be00 0x7fcd9003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcd9dffb640 1 -- 172.21.15.190:0/233236758 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(220..220 src has 1..220) ==== 13280+0+0 (secure 0 0 0) 0x7fcda804ff60 con 0x7fcdac1396a0 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb17f3640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd9003be00 0x7fcd9003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:34.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.034+0000 7fcdb17f3640 10 monclient: get_auth_request con 0x7fcd9003be00 auth_method 0 2024-08-19T10:44:34.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.038+0000 7fcdb17f3640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd9003be00 0x7fcd9003e2c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0a56b0 tx=0x7fcda0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:34.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.042+0000 7fcd9dffb640 1 -- 172.21.15.190:0/233236758 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fcdac0ac6d0 con 0x7fcdac1396a0 2024-08-19T10:44:34.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.042+0000 7fcd9dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:34.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.042+0000 7fcd9dffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:34.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.218+0000 7fcdb3a57640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:34.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.218+0000 7fcdb3a57640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:34.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.218+0000 7fcdb3a57640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:34.219 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.218+0000 7fcdb3a57640 1 -- 172.21.15.190:0/233236758 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fcdac0a7bb0 con 0x7fcdac1396a0 2024-08-19T10:44:34.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.810+0000 7fcd9dffb640 1 -- 172.21.15.190:0/233236758 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}]=0 added bucket h25 type host to location {rack=r1} v221) ==== 168+0+0 (secure 0 0 0) 0x7fcda804c840 con 0x7fcdac1396a0 2024-08-19T10:44:34.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.810+0000 7fcd9dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:34.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.810+0000 7fcd9dffb640 10 monclient: _finish_command 2 = system:0 added bucket h25 type host to location {rack=r1} 2024-08-19T10:44:34.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.918+0000 7fcdb3a57640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:34.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.918+0000 7fcdb3a57640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:34.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.918+0000 7fcdb3a57640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:34.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.918+0000 7fcdb3a57640 1 -- 172.21.15.190:0/233236758 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7fcdac123120 con 0x7fcdac1396a0 2024-08-19T10:44:34.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.922+0000 7fcd9dffb640 1 -- 172.21.15.190:0/233236758 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h25' already exists v221) ==== 147+0+0 (secure 0 0 0) 0x7fcda800f640 con 0x7fcdac1396a0 2024-08-19T10:44:34.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.922+0000 7fcd9dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h25", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:34.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.922+0000 7fcd9dffb640 10 monclient: _finish_command 3 = system:0 bucket 'h25' already exists 2024-08-19T10:44:34.926 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h25' already exists 2024-08-19T10:44:34.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.922+0000 7fcdb3a57640 1 -- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd9003be00 msgr2=0x7fcd9003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:34.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.922+0000 7fcdb3a57640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd9003be00 0x7fcd9003e2c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0a56b0 tx=0x7fcda0008040 comp rx=0 tx=0).stop 2024-08-19T10:44:34.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.922+0000 7fcdb3a57640 10 monclient: shutdown 2024-08-19T10:44:34.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.922+0000 7fcdb3a57640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:34.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.922+0000 7fcdb3a57640 1 -- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 msgr2=0x7fcdac138950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:34.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.922+0000 7fcdb3a57640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac138950 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcda800b580 tx=0x7fcda800ba40 comp rx=0 tx=0).stop 2024-08-19T10:44:34.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.926+0000 7fcdb3a57640 1 -- 172.21.15.190:0/233236758 shutdown_connections 2024-08-19T10:44:34.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.926+0000 7fcdb3a57640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd9003be00 0x7fcd9003e2c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:34.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.926+0000 7fcdb3a57640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcdac139fc0 0x7fcdac13dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:34.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.926+0000 7fcdb3a57640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcdac1396a0 0x7fcdac138950 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:34.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.926+0000 7fcdb3a57640 1 --2- 172.21.15.190:0/233236758 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcdac0a6ff0 0x7fcdac1d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:34.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.926+0000 7fcdb3a57640 1 -- 172.21.15.190:0/233236758 >> 172.21.15.190:0/233236758 conn(0x7fcdac0abee0 msgr2=0x7fcdac0aab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:34.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.926+0000 7fcdb3a57640 1 -- 172.21.15.190:0/233236758 shutdown_connections 2024-08-19T10:44:34.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:34.926+0000 7fcdb3a57640 1 -- 172.21.15.190:0/233236758 wait complete. 2024-08-19T10:44:34.942 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:34.942 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h26 host rack=r1 2024-08-19T10:44:35.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient: get_monmap_and_config 2024-08-19T10:44:35.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient: build_initial_monmap 2024-08-19T10:44:35.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient: monmap: 2024-08-19T10:44:35.062 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:35.062 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:35.062 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:35.063753+0000 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:35.063753+0000 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:35.063 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:35.064 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:35.064 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:35.064 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:35.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 1 Processor -- start 2024-08-19T10:44:35.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 1 -- start start 2024-08-19T10:44:35.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient: init 2024-08-19T10:44:35.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient: _add_conns ranks=[0,3,8,1,10,5,6,4,7,9,2] 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f65840a68e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a6cc0 0x7f65840a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient(hunting): picked mon.noname-g con 0x7f65840a6cc0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f658413eaf0 0x7f6584140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient(hunting): picked mon.noname-e con 0x7f658413eaf0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65840a86c0 con 0x7f65840a68e0 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f65840a8410 con 0x7f65840a6cc0 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f65840a63a0 con 0x7f658413eaf0 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:35.067 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658c1a3640 10 monclient(hunting): authenticate will time out at 1320.633935s 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a6cc0 0x7f65840a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a6cc0 0x7f65840a70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:39342/0 (socket says 172.21.15.190:39342) 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 1 -- 172.21.15.190:0/2027864052 learned_addr learned my addr 172.21.15.190:0/2027864052 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 10 monclient(hunting): get_auth_request con 0x7f65840a6cc0 auth_method 0 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658a740640 1 --2- 172.21.15.190:0/2027864052 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f658413eaf0 0x7f6584140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:35.068 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658a740640 10 monclient(hunting): get_auth_request con 0x7f658413eaf0 auth_method 0 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658a740640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658a740640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658a740640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 10 monclient(hunting): handle_auth_done global_id 34482 payload 274 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 1 -- 172.21.15.190:0/2027864052 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f658413eaf0 msgr2=0x7f6584140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 1 --2- 172.21.15.190:0/2027864052 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f658413eaf0 0x7f6584140ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 1 -- 172.21.15.190:0/2027864052 >> v1:172.21.15.89:6789/0 conn(0x7f65840a68e0 legacy=0x7f65840a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 10 monclient: _finish_hunting 0 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 1 monclient: found mon.noname-g 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658973e640 1 -- 172.21.15.190:0/2027864052 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65840a4680 con 0x7f65840a6cc0 2024-08-19T10:44:35.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.062+0000 7f658a740640 1 --2- 172.21.15.190:0/2027864052 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f658413eaf0 0x7f6584140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:44:35.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658973e640 1 -- 172.21.15.190:0/2027864052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a6cc0 msgr2=0x7f65840a70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:35.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658973e640 1 --2- 172.21.15.190:0/2027864052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a6cc0 0x7f65840a70a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f65740098a0 tx=0x7f657401c120 comp rx=0 tx=0).ready entity=mon.3 client_cookie=ceb854041486c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:35.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 1 -- 172.21.15.190:0/2027864052 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65740b1070 con 0x7f65840a6cc0 2024-08-19T10:44:35.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:35.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:35.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 10 monclient: dump: 2024-08-19T10:44:35.074 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:35.074 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:35.074 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:35.074 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:35.074 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:35.074 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:35.074 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:35.074 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:35.074 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:35.074 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 10 monclient: _finish_auth 0 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 10 monclient: _check_auth_tickets 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 1 -- 172.21.15.190:0/2027864052 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65740a49f0 con 0x7f65840a6cc0 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:35.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 1 -- 172.21.15.190:0/2027864052 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65740a4c20 con 0x7f65840a6cc0 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 5 monclient: authenticate success, global_id 34482 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 4 set_mon_vals no callback set 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f6588f3d640 10 monclient: dump: 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:35.076 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 10 monclient: shutdown 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 1 -- 172.21.15.190:0/2027864052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a6cc0 msgr2=0x7f65840a70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:35.077 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 1 --2- 172.21.15.190:0/2027864052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a6cc0 0x7f65840a70a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f65740098a0 tx=0x7f657401c120 comp rx=0 tx=0).stop 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 1 -- 172.21.15.190:0/2027864052 shutdown_connections 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 1 --2- 172.21.15.190:0/2027864052 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f658413eaf0 0x7f6584140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 1 --2- 172.21.15.190:0/2027864052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a6cc0 0x7f65840a70a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 1 -- 172.21.15.190:0/2027864052 >> 172.21.15.190:0/2027864052 conn(0x7f65840abee0 msgr2=0x7f65840ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 1 -- 172.21.15.190:0/2027864052 shutdown_connections 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.066+0000 7f658c1a3640 1 -- 172.21.15.190:0/2027864052 wait complete. 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient: build_initial_monmap 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient: monmap: 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:35.078 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:35.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 1 Processor -- start 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 1 -- start start 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient: init 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient: _add_conns ranks=[3,8,1,2,0,4,7,5,6] 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a68e0 0x7f6584136670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient(hunting): picked mon.noname-g con 0x7f65840a68e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f65840a6cc0 0x7f6584136bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient(hunting): picked mon.noname-e con 0x7f65840a6cc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f658413eaf0 0x7f65841370f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient(hunting): picked mon.noname-f con 0x7f658413eaf0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f65841445e0 con 0x7f658413eaf0 2024-08-19T10:44:35.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6584144460 con 0x7f65840a68e0 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6584144760 con 0x7f65840a6cc0 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658c1a3640 10 monclient(hunting): authenticate will time out at 1320.641810s 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658a740640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f658413eaf0 msgr2=0x7f65841370f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3300/0 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658a740640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f658413eaf0 0x7f65841370f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a68e0 0x7f6584136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a68e0 0x7f6584136670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:39346/0 (socket says 172.21.15.190:39346) 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 1 -- 172.21.15.190:0/1363403120 learned_addr learned my addr 172.21.15.190:0/1363403120 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 10 monclient(hunting): get_auth_request con 0x7f65840a68e0 auth_method 0 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:35.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658973e640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f65840a6cc0 0x7f6584136bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658973e640 10 monclient(hunting): get_auth_request con 0x7f65840a6cc0 auth_method 0 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658973e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658973e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f658973e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 10 monclient(hunting): handle_auth_done global_id 34491 payload 824 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 1 -- 172.21.15.190:0/1363403120 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f65840a6cc0 msgr2=0x7f6584136bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:35.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f65840a6cc0 0x7f6584136bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:35.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 1 -- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f658413eaf0 msgr2=0x7f65841370f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:35.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f658413eaf0 0x7f65841370f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:35.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 10 monclient: _finish_hunting 0 2024-08-19T10:44:35.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 1 monclient: found mon.noname-g 2024-08-19T10:44:35.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:35.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:35.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.070+0000 7f6589f3f640 1 -- 172.21.15.190:0/1363403120 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658413a1e0 con 0x7f65840a68e0 2024-08-19T10:44:35.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f6589f3f640 1 -- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a68e0 msgr2=0x7f6584136670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:35.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f6589f3f640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a68e0 0x7f6584136670 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f657c00c7c0 tx=0x7f657c00cc80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:35.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 1 -- 172.21.15.190:0/1363403120 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:44:35.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 1 -- 172.21.15.190:0/1363403120 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f657c016020 con 0x7f65840a68e0 2024-08-19T10:44:35.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:35.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:35.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 10 monclient: dump: 2024-08-19T10:44:35.085 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:35.086 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:35.087 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:35.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:35.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:35.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:35.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 1 -- 172.21.15.190:0/1363403120 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657c012420 con 0x7f65840a68e0 2024-08-19T10:44:35.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:35.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 1 -- 172.21.15.190:0/1363403120 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f657c012a40 con 0x7f65840a68e0 2024-08-19T10:44:35.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:35.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:35.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 10 monclient: dump: 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:35.088 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657b7fe640 4 set_mon_vals no callback set 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f658c1a3640 5 monclient: authenticate success, global_id 34491 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f658c1a3640 10 monclient: _renew_subs 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f658c1a3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f658c1a3640 1 -- 172.21.15.190:0/1363403120 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658413dad0 con 0x7f65840a68e0 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f658c1a3640 10 monclient: _renew_subs 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f658c1a3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f658c1a3640 1 -- 172.21.15.190:0/1363403120 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f658413bf80 con 0x7f65840a68e0 2024-08-19T10:44:35.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f658c1a3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f658c1a3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f658c1a3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f657a7fc640 1 -- 172.21.15.190:0/1363403120 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f657c019020 con 0x7f65840a68e0 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.074+0000 7f658c1a3640 1 -- 172.21.15.190:0/1363403120 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65840a70a0 con 0x7f65840a68e0 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.078+0000 7f657a7fc640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f656803be00 0x7f656803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.078+0000 7f657a7fc640 1 -- 172.21.15.190:0/1363403120 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(221..221 src has 1..221) ==== 13493+0+0 (secure 0 0 0) 0x7f657c050c80 con 0x7f65840a68e0 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.078+0000 7f658973e640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f656803be00 0x7f656803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.078+0000 7f658973e640 10 monclient: get_auth_request con 0x7f656803be00 auth_method 0 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.078+0000 7f658973e640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f656803be00 0x7f656803e2c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f65840a43a0 tx=0x7f65740a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.082+0000 7f657a7fc640 1 -- 172.21.15.190:0/1363403120 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f657c020030 con 0x7f65840a68e0 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.082+0000 7f657a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:35.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.082+0000 7f657a7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:35.255 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.254+0000 7f658c1a3640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:35.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.254+0000 7f658c1a3640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:35.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.254+0000 7f658c1a3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:35.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.254+0000 7f658c1a3640 1 -- 172.21.15.190:0/1363403120 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f658406c9c0 con 0x7f65840a68e0 2024-08-19T10:44:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.810+0000 7f657a7fc640 1 -- 172.21.15.190:0/1363403120 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}]=0 added bucket h26 type host to location {rack=r1} v222) ==== 168+0+0 (secure 0 0 0) 0x7f657c011380 con 0x7f65840a68e0 2024-08-19T10:44:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.810+0000 7f657a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.810+0000 7f657a7fc640 10 monclient: _finish_command 2 = system:0 added bucket h26 type host to location {rack=r1} 2024-08-19T10:44:35.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.922+0000 7f658c1a3640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:35.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.922+0000 7f658c1a3640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:35.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.922+0000 7f658c1a3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:35.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.922+0000 7f658c1a3640 1 -- 172.21.15.190:0/1363403120 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f658413ad40 con 0x7f65840a68e0 2024-08-19T10:44:35.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.922+0000 7f657a7fc640 1 -- 172.21.15.190:0/1363403120 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h26' already exists v222) ==== 147+0+0 (secure 0 0 0) 0x7f657c04d460 con 0x7f65840a68e0 2024-08-19T10:44:35.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.922+0000 7f657a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h26", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:35.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.922+0000 7f657a7fc640 10 monclient: _finish_command 3 = system:0 bucket 'h26' already exists 2024-08-19T10:44:35.924 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h26' already exists 2024-08-19T10:44:35.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 -- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f656803be00 msgr2=0x7f656803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:35.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f656803be00 0x7f656803e2c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f65840a43a0 tx=0x7f65740a6070 comp rx=0 tx=0).stop 2024-08-19T10:44:35.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 10 monclient: shutdown 2024-08-19T10:44:35.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:35.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 -- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a68e0 msgr2=0x7f6584136670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:35.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a68e0 0x7f6584136670 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f657c00c7c0 tx=0x7f657c00cc80 comp rx=0 tx=0).stop 2024-08-19T10:44:35.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 -- 172.21.15.190:0/1363403120 shutdown_connections 2024-08-19T10:44:35.927 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f658413eaf0 0x7f65841370f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:35.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f656803be00 0x7f656803e2c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:35.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f65840a6cc0 0x7f6584136bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:35.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 --2- 172.21.15.190:0/1363403120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65840a68e0 0x7f6584136670 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:35.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 -- 172.21.15.190:0/1363403120 >> 172.21.15.190:0/1363403120 conn(0x7f65840abee0 msgr2=0x7f658413ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:35.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 -- 172.21.15.190:0/1363403120 shutdown_connections 2024-08-19T10:44:35.928 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:35.926+0000 7f658c1a3640 1 -- 172.21.15.190:0/1363403120 wait complete. 2024-08-19T10:44:35.943 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:35.943 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h27 host rack=r1 2024-08-19T10:44:36.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient: get_monmap_and_config 2024-08-19T10:44:36.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient: build_initial_monmap 2024-08-19T10:44:36.066 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient: monmap: 2024-08-19T10:44:36.066 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:36.066 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:36.066 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:36.067652+0000 2024-08-19T10:44:36.066 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:36.067652+0000 2024-08-19T10:44:36.066 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:36.066 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:36.067 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:36.067 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:36.067 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:36.067 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:36.067 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:36.067 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:36.067 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:36.068 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:36.068 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:36.068 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:36.068 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:36.068 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:36.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 1 Processor -- start 2024-08-19T10:44:36.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 1 -- start start 2024-08-19T10:44:36.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient: init 2024-08-19T10:44:36.070 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:36.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient: _add_conns ranks=[0,4,7,2,6,10,8,3,9,1,5] 2024-08-19T10:44:36.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f932c0a4720 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:36.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a2f80 0x7f932c13c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient(hunting): picked mon.noname-c con 0x7f932c0a2f80 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:36.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f932c0a28c0 0x7f932c13ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient(hunting): picked mon.noname-i con 0x7f932c0a28c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:36.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c0a2d40 con 0x7f932c0a4720 2024-08-19T10:44:36.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:36.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f932c09cbe0 con 0x7f932c0a2f80 2024-08-19T10:44:36.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f932c06d650 con 0x7f932c0a28c0 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f933338c640 10 monclient(hunting): authenticate will time out at 1321.637651s 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f932c0a28c0 0x7f932c13ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9330927640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a2f80 msgr2=0x7f932c13c720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f932c0a28c0 0x7f932c13ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:49204/0 (socket says 172.21.15.190:49204) 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9330927640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a2f80 0x7f932c13c720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 1 -- 172.21.15.190:0/196458715 learned_addr learned my addr 172.21.15.190:0/196458715 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 10 monclient(hunting): get_auth_request con 0x7f932c0a28c0 auth_method 0 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:36.072 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:36.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:36.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:36.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:36.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 10 monclient(hunting): handle_auth_done global_id 15397 payload 274 2024-08-19T10:44:36.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 1 -- 172.21.15.190:0/196458715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a2f80 msgr2=0x7f932c13c720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:36.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 1 --2- 172.21.15.190:0/196458715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a2f80 0x7f932c13c720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 1 -- 172.21.15.190:0/196458715 >> v1:172.21.15.89:6789/0 conn(0x7f932c0a4720 legacy=0x7f932c0a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:36.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 10 monclient: _finish_hunting 0 2024-08-19T10:44:36.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 1 monclient: found mon.noname-i 2024-08-19T10:44:36.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:36.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:36.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.066+0000 7f9331929640 1 -- 172.21.15.190:0/196458715 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c074570 con 0x7f932c0a28c0 2024-08-19T10:44:36.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f9331929640 1 -- 172.21.15.190:0/196458715 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f932c0a28c0 msgr2=0x7f932c13ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:36.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f9331929640 1 --2- 172.21.15.190:0/196458715 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f932c0a28c0 0x7f932c13ec70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098a0 tx=0x7f931c01c460 comp rx=0 tx=0).ready entity=mon.7 client_cookie=9f64f9b4befa3c5d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 1 -- 172.21.15.190:0/196458715 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f931c0b0070 con 0x7f932c0a28c0 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 10 monclient: dump: 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:36.079 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 10 monclient: _finish_auth 0 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 1 -- 172.21.15.190:0/196458715 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c0a9a60 con 0x7f932c0a28c0 2024-08-19T10:44:36.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 1 -- 172.21.15.190:0/196458715 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f931c0a9c50 con 0x7f932c0a28c0 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 5 monclient: authenticate success, global_id 15397 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 4 set_mon_vals no callback set 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f932bfff640 10 monclient: dump: 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:36.081 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:36.082 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:36.082 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:36.082 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:36.082 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:36.082 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:36.082 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:36.082 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:36.082 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:36.082 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:36.082 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:36.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 10 monclient: shutdown 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 1 -- 172.21.15.190:0/196458715 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f932c0a28c0 msgr2=0x7f932c13ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 1 --2- 172.21.15.190:0/196458715 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f932c0a28c0 0x7f932c13ec70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098a0 tx=0x7f931c01c460 comp rx=0 tx=0).stop 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 1 -- 172.21.15.190:0/196458715 shutdown_connections 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 1 --2- 172.21.15.190:0/196458715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a2f80 0x7f932c13c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 1 --2- 172.21.15.190:0/196458715 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f932c0a28c0 0x7f932c13ec70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 1 -- 172.21.15.190:0/196458715 >> 172.21.15.190:0/196458715 conn(0x7f932c0a9310 msgr2=0x7f932c0a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 1 -- 172.21.15.190:0/196458715 shutdown_connections 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 1 -- 172.21.15.190:0/196458715 wait complete. 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 10 monclient: build_initial_monmap 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.070+0000 7f933338c640 10 monclient: monmap: 2024-08-19T10:44:36.083 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:36.084 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 1 Processor -- start 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 1 -- start start 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 10 monclient: init 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 10 monclient: _add_conns ranks=[4,2,6,8,3,0,1,5,7] 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a28c0 0x7f932c1d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 10 monclient(hunting): picked mon.noname-c con 0x7f932c0a28c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f932c0a2f80 0x7f932c13e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 10 monclient(hunting): picked mon.noname-b con 0x7f932c0a2f80 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:36.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f932c0a4720 0x7f932c13eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 10 monclient(hunting): picked mon.noname-d con 0x7f932c0a4720 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:36.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:36.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f932c141b50 con 0x7f932c0a2f80 2024-08-19T10:44:36.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:36.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f932c1419d0 con 0x7f932c0a28c0 2024-08-19T10:44:36.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:36.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f932c141cd0 con 0x7f932c0a4720 2024-08-19T10:44:36.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:36.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f933338c640 10 monclient(hunting): authenticate will time out at 1321.644830s 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9331929640 1 -- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f932c0a4720 msgr2=0x7f932c13eaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3303/0 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9331128640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a28c0 msgr2=0x7f932c1d0f80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9331929640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f932c0a4720 0x7f932c13eaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9331128640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a28c0 0x7f932c1d0f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f932c0a2f80 0x7f932c13e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f932c0a2f80 0x7f932c13e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:41820/0 (socket says 172.21.15.190:41820) 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 1 -- 172.21.15.190:0/2291628555 learned_addr learned my addr 172.21.15.190:0/2291628555 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 10 monclient(hunting): get_auth_request con 0x7f932c0a2f80 auth_method 0 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:36.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 10 monclient(hunting): handle_auth_done global_id 6068 payload 824 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 1 -- 172.21.15.190:0/2291628555 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f932c0a4720 msgr2=0x7f932c13eaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f932c0a4720 0x7f932c13eaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 1 -- 172.21.15.190:0/2291628555 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a28c0 msgr2=0x7f932c1d0f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a28c0 0x7f932c1d0f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 10 monclient: _finish_hunting 0 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 1 monclient: found mon.noname-b 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.074+0000 7f9330927640 1 -- 172.21.15.190:0/2291628555 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c0a8840 con 0x7f932c0a2f80 2024-08-19T10:44:36.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9330927640 1 -- 172.21.15.190:0/2291628555 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f932c0a2f80 msgr2=0x7f932c13e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9330927640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f932c0a2f80 0x7f932c13e560 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f932000be50 tx=0x7f932000bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 1 -- 172.21.15.190:0/2291628555 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93200158e0 con 0x7f932c0a2f80 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 10 monclient: dump: 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:36.089 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:36.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f933338c640 5 monclient: authenticate success, global_id 6068 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f933338c640 10 monclient: _renew_subs 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f933338c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f933338c640 1 -- 172.21.15.190:0/2291628555 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c12a970 con 0x7f932c0a2f80 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 1 -- 172.21.15.190:0/2291628555 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9320010070 con 0x7f932c0a2f80 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f933338c640 10 monclient: _renew_subs 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f933338c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f933338c640 1 -- 172.21.15.190:0/2291628555 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c1294b0 con 0x7f932c0a2f80 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 1 -- 172.21.15.190:0/2291628555 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f932001a020 con 0x7f932c0a2f80 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f932b7fe640 4 set_mon_vals no callback set 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:36.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 10 monclient: dump: 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:36.092 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:36.093 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:36.093 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:36.093 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:36.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f933338c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:36.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f933338c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:36.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f933338c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:36.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f933338c640 1 -- 172.21.15.190:0/2291628555 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c129730 con 0x7f932c0a2f80 2024-08-19T10:44:36.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 1 -- 172.21.15.190:0/2291628555 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9320015470 con 0x7f932c0a2f80 2024-08-19T10:44:36.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f931803be00 0x7f931803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9329ffb640 1 -- 172.21.15.190:0/2291628555 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(222..222 src has 1..222) ==== 13706+0+0 (secure 0 0 0) 0x7f93200513f0 con 0x7f932c0a2f80 2024-08-19T10:44:36.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9331128640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f931803be00 0x7f931803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:36.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.078+0000 7f9331128640 10 monclient: get_auth_request con 0x7f931803be00 auth_method 0 2024-08-19T10:44:36.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.082+0000 7f9331128640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f931803be00 0x7f931803e2c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9324007fd0 tx=0x7f93240079a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:36.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.090+0000 7f9329ffb640 1 -- 172.21.15.190:0/2291628555 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f932c0a8840 con 0x7f932c0a2f80 2024-08-19T10:44:36.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.090+0000 7f9329ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:36.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.090+0000 7f9329ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:36.265 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.262+0000 7f933338c640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:36.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.262+0000 7f933338c640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:36.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.262+0000 7f933338c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:36.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.262+0000 7f933338c640 1 -- 172.21.15.190:0/2291628555 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f932c09eec0 con 0x7f932c0a2f80 2024-08-19T10:44:36.783 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-08-19T10:44:36.783 INFO:tasks.ceph.mon.f:Restarting daemon 2024-08-19T10:44:36.784 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-08-19T10:44:36.788 INFO:tasks.ceph.mon.f:Started 2024-08-19T10:44:36.788 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-08-19T10:44:36.788 INFO:tasks.ceph.mon.c:Restarting daemon 2024-08-19T10:44:36.788 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-08-19T10:44:36.790 INFO:tasks.ceph.mon.c:Started 2024-08-19T10:44:36.791 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.d 2024-08-19T10:44:36.791 INFO:tasks.ceph.mon.d:Restarting daemon 2024-08-19T10:44:36.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-08-19T10:44:36.793 INFO:tasks.ceph.mon.d:Started 2024-08-19T10:44:36.793 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:44:36.793 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:44:36.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.830+0000 7f9329ffb640 1 -- 172.21.15.190:0/2291628555 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}]=0 added bucket h27 type host to location {rack=r1} v223) ==== 168+0+0 (secure 0 0 0) 0x7f932001ede0 con 0x7f932c0a2f80 2024-08-19T10:44:36.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.830+0000 7f9329ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:36.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.830+0000 7f9329ffb640 10 monclient: _finish_command 2 = system:0 added bucket h27 type host to location {rack=r1} 2024-08-19T10:44:36.889 INFO:tasks.ceph.mon.d.smithi089.stderr:2024-08-19T10:44:36.885+0000 7f8c22683dc0 -1 mon.d@-1(???).log v0 log_external_backlog rewinding external_log_to from 208 -> 0 (sync_force? mon rebuild?) 2024-08-19T10:44:36.889 INFO:tasks.ceph.mon.f.smithi190.stderr:2024-08-19T10:44:36.886+0000 7fa7cd7f1dc0 -1 mon.f@-1(???).log v0 log_external_backlog rewinding external_log_to from 208 -> 0 (sync_force? mon rebuild?) 2024-08-19T10:44:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.913+0000 7efd33890640 10 monclient: get_monmap_and_config 2024-08-19T10:44:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.913+0000 7efd33890640 10 monclient: build_initial_monmap 2024-08-19T10:44:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.913+0000 7efd33890640 10 monclient: monmap: 2024-08-19T10:44:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:36.919 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:36.920498+0000 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:36.920498+0000 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:36.920 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:36.921 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:36.921 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 1 Processor -- start 2024-08-19T10:44:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 1 -- start start 2024-08-19T10:44:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 10 monclient: init 2024-08-19T10:44:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 10 monclient: _add_conns ranks=[8,9,4,10,1,2,6,3,0,5,7] 2024-08-19T10:44:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efd2c0a6860 0x7efd2c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 10 monclient(hunting): picked mon.noname-e con 0x7efd2c0a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efd2c0a7180 0x7efd2c0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 10 monclient(hunting): picked mon.noname-a con 0x7efd2c0a7180 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:36.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd2c139620 0x7efd2c139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 10 monclient(hunting): picked mon.noname-c con 0x7efd2c139620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7efd2c0a8190 con 0x7efd2c0a7180 2024-08-19T10:44:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7efd2c0a33d0 con 0x7efd2c139620 2024-08-19T10:44:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:36.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7efd2c0a2ec0 con 0x7efd2c0a6860 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd33890640 10 monclient(hunting): authenticate will time out at 1326.736758s 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd31e2d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd2c139620 0x7efd2c139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efd2c0a6860 0x7efd2c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd31e2d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd2c139620 0x7efd2c139ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58358/0 (socket says 172.21.15.89:58358) 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efd2c0a6860 0x7efd2c0a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:42782/0 (socket says 172.21.15.89:42782) 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd31e2d640 1 -- 172.21.15.89:0/1877572132 learned_addr learned my addr 172.21.15.89:0/1877572132 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd31e2d640 10 monclient(hunting): get_auth_request con 0x7efd2c139620 auth_method 0 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd31e2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd31e2d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd31e2d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:36.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 10 monclient(hunting): get_auth_request con 0x7efd2c0a6860 auth_method 0 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd30e2b640 1 --2- 172.21.15.89:0/1877572132 >> v2:172.21.15.89:3300/0 conn(0x7efd2c0a7180 0x7efd2c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd30e2b640 10 monclient(hunting): get_auth_request con 0x7efd2c0a7180 auth_method 0 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd30e2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd30e2b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd30e2b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd31e2d640 1 -- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd2c139620 msgr2=0x7efd2c139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd31e2d640 1 -- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd2c139620 msgr2=0x7efd2c139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd31e2d640 1 --2- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd2c139620 0x7efd2c139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:36.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd31e2d640 1 --2- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd2c139620 0x7efd2c139ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd30e2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd30e2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd30e2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 10 monclient(hunting): handle_auth_done global_id 5957 payload 274 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 1 -- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd2c139620 msgr2=0x7efd2c139ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 1 --2- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd2c139620 0x7efd2c139ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 1 -- 172.21.15.89:0/1877572132 >> v2:172.21.15.89:3300/0 conn(0x7efd2c0a7180 msgr2=0x7efd2c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 1 --2- 172.21.15.89:0/1877572132 >> v2:172.21.15.89:3300/0 conn(0x7efd2c0a7180 0x7efd2c0a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 10 monclient: _finish_hunting 0 2024-08-19T10:44:36.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 1 monclient: found mon.noname-e 2024-08-19T10:44:36.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:36.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:36.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd3162c640 1 -- 172.21.15.89:0/1877572132 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c0a4cd0 con 0x7efd2c0a6860 2024-08-19T10:44:36.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.917+0000 7efd30e2b640 1 --2- 172.21.15.89:0/1877572132 >> v2:172.21.15.89:3300/0 conn(0x7efd2c0a7180 0x7efd2c0a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd3162c640 1 -- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efd2c0a6860 msgr2=0x7efd2c0a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd3162c640 1 --2- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efd2c0a6860 0x7efd2c0a6c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efd280097c0 tx=0x7efd2801c360 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ac6940ccb2981ea7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 1 -- 172.21.15.89:0/1877572132 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efd280b0070 con 0x7efd2c0a6860 2024-08-19T10:44:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 10 monclient: dump: 2024-08-19T10:44:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:36.936 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:36.937 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 1 -- 172.21.15.89:0/1877572132 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd280aa810 con 0x7efd2c0a6860 2024-08-19T10:44:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:36.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 1 -- 172.21.15.89:0/1877572132 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efd280a9300 con 0x7efd2c0a6860 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 5 monclient: authenticate success, global_id 5957 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 4 set_mon_vals no callback set 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd23fff640 10 monclient: dump: 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:36.939 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 10 monclient: shutdown 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 -- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efd2c0a6860 msgr2=0x7efd2c0a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:36.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 --2- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efd2c0a6860 0x7efd2c0a6c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efd280097c0 tx=0x7efd2801c360 comp rx=0 tx=0).stop 2024-08-19T10:44:36.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 -- 172.21.15.89:0/1877572132 shutdown_connections 2024-08-19T10:44:36.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 --2- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7efd2c139620 0x7efd2c139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 --2- 172.21.15.89:0/1877572132 >> v2:172.21.15.89:3300/0 conn(0x7efd2c0a7180 0x7efd2c0a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 --2- 172.21.15.89:0/1877572132 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efd2c0a6860 0x7efd2c0a6c40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 -- 172.21.15.89:0/1877572132 >> 172.21.15.89:0/1877572132 conn(0x7efd2c074b40 msgr2=0x7efd2c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:36.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 -- 172.21.15.89:0/1877572132 shutdown_connections 2024-08-19T10:44:36.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 -- 172.21.15.89:0/1877572132 wait complete. 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 10 monclient: build_initial_monmap 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 10 monclient: monmap: 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:36.942 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:36.943 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 Processor -- start 2024-08-19T10:44:36.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 -- start start 2024-08-19T10:44:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 10 monclient: init 2024-08-19T10:44:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 10 monclient: _add_conns ranks=[2,1,3,0,5,6,7,4,8] 2024-08-19T10:44:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd2c0a6860 0x7efd2c1d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 10 monclient(hunting): picked mon.noname-b con 0x7efd2c0a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd2c0a7180 0x7efd2c0a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 10 monclient(hunting): picked mon.noname-f con 0x7efd2c0a7180 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd2c139620 0x7efd2c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 10 monclient(hunting): picked mon.noname-g con 0x7efd2c139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.921+0000 7efd33890640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c143210 con 0x7efd2c0a7180 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7efd2c143090 con 0x7efd2c0a6860 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7efd2c143390 con 0x7efd2c139620 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 10 monclient(hunting): authenticate will time out at 1326.744023s 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd2c0a6860 0x7efd2c1d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:36.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd2c0a6860 0x7efd2c1d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:37746/0 (socket says 172.21.15.89:37746) 2024-08-19T10:44:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 1 -- 172.21.15.89:0/500502886 learned_addr learned my addr 172.21.15.89:0/500502886 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 10 monclient(hunting): get_auth_request con 0x7efd2c0a6860 auth_method 0 2024-08-19T10:44:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd30e2b640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd2c0a7180 0x7efd2c0a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd30e2b640 10 monclient(hunting): get_auth_request con 0x7efd2c0a7180 auth_method 0 2024-08-19T10:44:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd30e2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd30e2b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:36.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd30e2b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd31e2d640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd2c139620 0x7efd2c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd31e2d640 10 monclient(hunting): get_auth_request con 0x7efd2c139620 auth_method 0 2024-08-19T10:44:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd31e2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd31e2d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd31e2d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd30e2b640 1 -- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd2c0a7180 msgr2=0x7efd2c0a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:44:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd30e2b640 1 -- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd2c0a7180 msgr2=0x7efd2c0a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:44:36.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd30e2b640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd2c0a7180 0x7efd2c0a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd30e2b640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd2c0a7180 0x7efd2c0a1ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd31e2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd31e2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd31e2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 10 monclient(hunting): handle_auth_done global_id 6077 payload 824 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 1 -- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd2c139620 msgr2=0x7efd2c0a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd2c139620 0x7efd2c0a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 1 -- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd2c0a7180 msgr2=0x7efd2c0a1ed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd2c0a7180 0x7efd2c0a1ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 10 monclient: _finish_hunting 0 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 1 monclient: found mon.noname-b 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:36.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 1 -- 172.21.15.89:0/500502886 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c09e640 con 0x7efd2c0a6860 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 1 -- 172.21.15.89:0/500502886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd2c0a6860 msgr2=0x7efd2c1d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd3162c640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd2c0a6860 0x7efd2c1d18c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd280ae040 tx=0x7efd28019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 1 -- 172.21.15.89:0/500502886 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efd280b0040 con 0x7efd2c0a6860 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 10 monclient: dump: 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:36.949 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:36.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 5 monclient: authenticate success, global_id 6077 2024-08-19T10:44:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 10 monclient: _renew_subs 2024-08-19T10:44:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 1 -- 172.21.15.89:0/500502886 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd280044b0 con 0x7efd2c0a6860 2024-08-19T10:44:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 1 -- 172.21.15.89:0/500502886 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2c0ac080 con 0x7efd2c0a6860 2024-08-19T10:44:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 1 -- 172.21.15.89:0/500502886 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efd28004040 con 0x7efd2c0a6860 2024-08-19T10:44:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:36.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd21ffb640 10 monclient: dump: 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:36.952 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd237fe640 4 set_mon_vals no callback set 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 10 monclient: _renew_subs 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.925+0000 7efd33890640 1 -- 172.21.15.89:0/500502886 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2c0a09a0 con 0x7efd2c0a6860 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.929+0000 7efd21ffb640 1 -- 172.21.15.89:0/500502886 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efd28003620 con 0x7efd2c0a6860 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.929+0000 7efd33890640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.929+0000 7efd33890640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.929+0000 7efd33890640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.929+0000 7efd33890640 1 -- 172.21.15.89:0/500502886 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd2c0ac2b0 con 0x7efd2c0a6860 2024-08-19T10:44:36.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.929+0000 7efd21ffb640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efd1003bdf0 0x7efd1003e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:36.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.929+0000 7efd21ffb640 1 -- 172.21.15.89:0/500502886 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(223..223 src has 1..223) ==== 13919+0+0 (secure 0 0 0) 0x7efd280ea8c0 con 0x7efd2c0a6860 2024-08-19T10:44:36.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.929+0000 7efd30e2b640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efd1003bdf0 0x7efd1003e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:36.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.929+0000 7efd30e2b640 10 monclient: get_auth_request con 0x7efd1003bdf0 auth_method 0 2024-08-19T10:44:36.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.929+0000 7efd30e2b640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efd1003bdf0 0x7efd1003e2b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efd2c13faf0 tx=0x7efd1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:36.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.933+0000 7efd21ffb640 1 -- 172.21.15.89:0/500502886 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7efd2c0ac2b0 con 0x7efd2c0a6860 2024-08-19T10:44:36.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.933+0000 7efd21ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:36.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:36.933+0000 7efd21ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:36.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:36.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:36.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:36.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 -- 172.21.15.190:0/2291628555 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f932c09e970 con 0x7f932c0a2f80 2024-08-19T10:44:36.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f9329ffb640 1 -- 172.21.15.190:0/2291628555 <== mon.2 v2:172.21.15.89:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h27' already exists v223) ==== 147+0+0 (secure 0 0 0) 0x7f932004db00 con 0x7f932c0a2f80 2024-08-19T10:44:36.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f9329ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h27", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:36.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f9329ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h27' already exists 2024-08-19T10:44:36.956 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h27' already exists 2024-08-19T10:44:36.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 -- 172.21.15.190:0/2291628555 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f931803be00 msgr2=0x7f931803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f931803be00 0x7f931803e2c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9324007fd0 tx=0x7f93240079a0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 10 monclient: shutdown 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 -- 172.21.15.190:0/2291628555 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f932c0a2f80 msgr2=0x7f932c13e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f932c0a2f80 0x7f932c13e560 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f932000be50 tx=0x7f932000bf40 comp rx=0 tx=0).stop 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 -- 172.21.15.190:0/2291628555 shutdown_connections 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f931803be00 0x7f931803e2c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f932c0a4720 0x7f932c13eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f932c0a2f80 0x7f932c13e560 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 --2- 172.21.15.190:0/2291628555 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f932c0a28c0 0x7f932c1d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 -- 172.21.15.190:0/2291628555 >> 172.21.15.190:0/2291628555 conn(0x7f932c0a9310 msgr2=0x7f932c070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.946+0000 7f933338c640 1 -- 172.21.15.190:0/2291628555 shutdown_connections 2024-08-19T10:44:36.957 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:36.950+0000 7f933338c640 1 -- 172.21.15.190:0/2291628555 wait complete. 2024-08-19T10:44:36.967 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:36.967 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h28 host rack=r1 2024-08-19T10:44:37.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient: get_monmap_and_config 2024-08-19T10:44:37.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient: build_initial_monmap 2024-08-19T10:44:37.071 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient: monmap: 2024-08-19T10:44:37.071 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:37.072 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:37.072 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:37.072396+0000 2024-08-19T10:44:37.072 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:37.072396+0000 2024-08-19T10:44:37.072 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:37.072 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:37.072 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:37.073 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:37.073 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:37.073 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:37.073 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:37.073 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:37.073 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:37.074 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:37.074 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:37.074 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:37.074 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:37.074 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:37.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 1 Processor -- start 2024-08-19T10:44:37.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 1 -- start start 2024-08-19T10:44:37.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient: init 2024-08-19T10:44:37.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:37.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient: _add_conns ranks=[7,1,8,5,10,3,9,0,6,2,4] 2024-08-19T10:44:37.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88340a36a0 0x7f88340a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:37.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient(hunting): picked mon.noname-i con 0x7f88340a36a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:37.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f88340a19d0 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:37.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88340a3fc0 0x7f8834137b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:37.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient(hunting): picked mon.noname-e con 0x7f88340a3fc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:37.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f883409d050 con 0x7f88340a19d0 2024-08-19T10:44:37.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:37.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f883406f250 con 0x7f88340a36a0 2024-08-19T10:44:37.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:37.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f883409dec0 con 0x7f88340a3fc0 2024-08-19T10:44:37.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:37.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883bab1640 10 monclient(hunting): authenticate will time out at 1322.642157s 2024-08-19T10:44:37.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88340a36a0 0x7f88340a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:37.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88340a36a0 0x7f88340a3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:49224/0 (socket says 172.21.15.190:49224) 2024-08-19T10:44:37.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 1 -- 172.21.15.190:0/1746385619 learned_addr learned my addr 172.21.15.190:0/1746385619 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:37.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883904c640 1 --1- 172.21.15.190:0/1746385619 >> v1:172.21.15.190:6789/0 conn(0x7f88340a19d0 0x7f8834137470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:51172/0 (socket says 172.21.15.190:51172) 2024-08-19T10:44:37.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 10 monclient(hunting): get_auth_request con 0x7f88340a36a0 auth_method 0 2024-08-19T10:44:37.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:37.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:37.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:37.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:37.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:37.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:37.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 10 monclient(hunting): handle_auth_done global_id 15406 payload 274 2024-08-19T10:44:37.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 1 -- 172.21.15.190:0/1746385619 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88340a3fc0 msgr2=0x7f8834137b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:37.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883a04e640 1 --2- 172.21.15.190:0/1746385619 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88340a3fc0 0x7f8834137b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:37.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 1 --2- 172.21.15.190:0/1746385619 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88340a3fc0 0x7f8834137b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:37.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 1 -- 172.21.15.190:0/1746385619 >> v1:172.21.15.190:6789/0 conn(0x7f88340a19d0 legacy=0x7f8834137470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:37.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 10 monclient: _finish_hunting 0 2024-08-19T10:44:37.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 1 monclient: found mon.noname-i 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.070+0000 7f883984d640 1 -- 172.21.15.190:0/1746385619 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883409d620 con 0x7f88340a36a0 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883984d640 1 -- 172.21.15.190:0/1746385619 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88340a36a0 msgr2=0x7f88340a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883984d640 1 --2- 172.21.15.190:0/1746385619 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88340a36a0 0x7f88340a3a80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f88240098a0 tx=0x7f882401c440 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2e5408a2e4b69af0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 1 -- 172.21.15.190:0/1746385619 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88240b0070 con 0x7f88340a36a0 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 10 monclient: dump: 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:37.091 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 10 monclient: _finish_auth 0 2024-08-19T10:44:37.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 10 monclient: _check_auth_tickets 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 5 monclient: authenticate success, global_id 15406 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 1 -- 172.21.15.190:0/1746385619 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88240aa810 con 0x7f88340a36a0 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 4 set_mon_vals no callback set 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 1 -- 172.21.15.190:0/1746385619 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88240a93e0 con 0x7f88340a36a0 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883884b640 10 monclient: dump: 2024-08-19T10:44:37.093 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:37.094 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:37.094 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:37.094 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:37.094 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:37.094 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:37.094 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:37.094 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:37.094 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:37.094 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:37.094 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 10 monclient: shutdown 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 1 -- 172.21.15.190:0/1746385619 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88340a36a0 msgr2=0x7f88340a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 1 --2- 172.21.15.190:0/1746385619 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88340a36a0 0x7f88340a3a80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f88240098a0 tx=0x7f882401c440 comp rx=0 tx=0).stop 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 1 -- 172.21.15.190:0/1746385619 shutdown_connections 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 1 --2- 172.21.15.190:0/1746385619 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88340a3fc0 0x7f8834137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:37.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 1 --2- 172.21.15.190:0/1746385619 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f88340a36a0 0x7f88340a3a80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 1 -- 172.21.15.190:0/1746385619 >> 172.21.15.190:0/1746385619 conn(0x7f88340a4790 msgr2=0x7f88340a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 1 -- 172.21.15.190:0/1746385619 shutdown_connections 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.074+0000 7f883bab1640 1 -- 172.21.15.190:0/1746385619 wait complete. 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient: build_initial_monmap 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient: monmap: 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:37.096 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 1 Processor -- start 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 1 -- start start 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient: init 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient: _add_conns ranks=[1,5,6,3,2,0,7,8,4] 2024-08-19T10:44:37.097 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88340a19d0 0x7f8834136cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient(hunting): picked mon.noname-f con 0x7f88340a19d0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88340a3fc0 0x7f8834137200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient(hunting): picked mon.noname-h con 0x7f88340a3fc0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88340a0ec0 0x7f883409f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient(hunting): picked mon.noname-d con 0x7f88340a0ec0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f88341380c0 con 0x7f88340a19d0 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f883409ca60 con 0x7f88340a3fc0 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8834138240 con 0x7f88340a0ec0 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:37.098 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient(hunting): authenticate will time out at 1322.648969s 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88340a3fc0 0x7f8834137200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883984d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88340a19d0 0x7f8834136cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88340a3fc0 0x7f8834137200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:54898/0 (socket says 172.21.15.190:54898) 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 1 -- 172.21.15.190:0/3217785971 learned_addr learned my addr 172.21.15.190:0/3217785971 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 10 monclient(hunting): get_auth_request con 0x7f88340a3fc0 auth_method 0 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883984d640 10 monclient(hunting): get_auth_request con 0x7f88340a19d0 auth_method 0 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883984d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883984d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883984d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:37.099 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:37.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:37.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883984d640 1 -- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88340a19d0 msgr2=0x7f8834136cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:44:37.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:37.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883984d640 1 -- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88340a19d0 msgr2=0x7f8834136cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:44:37.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883984d640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88340a19d0 0x7f8834136cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:37.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883984d640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88340a19d0 0x7f8834136cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:37.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883a04e640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88340a0ec0 0x7f883409f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:37.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 10 monclient(hunting): handle_auth_done global_id 6107 payload 824 2024-08-19T10:44:37.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 1 -- 172.21.15.190:0/3217785971 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88340a0ec0 msgr2=0x7f883409f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:37.100 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88340a0ec0 0x7f883409f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 1 -- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88340a19d0 msgr2=0x7f8834136cc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88340a19d0 0x7f8834136cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 10 monclient: _finish_hunting 0 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 1 monclient: found mon.noname-h 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 1 -- 172.21.15.190:0/3217785971 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88340a31b0 con 0x7f88340a3fc0 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883a04e640 10 monclient: get_auth_request con 0x7f88340a0ec0 auth_method 0 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883a04e640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88340a0ec0 0x7f883409f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 1 -- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88340a3fc0 msgr2=0x7f8834137200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883904c640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88340a3fc0 0x7f8834137200 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f883000c480 tx=0x7f883000c940 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:37.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 1 -- 172.21.15.190:0/3217785971 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8830017070 con 0x7f88340a3fc0 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 10 monclient: dump: 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:37.102 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 5 monclient: authenticate success, global_id 6107 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient: _renew_subs 2024-08-19T10:44:37.103 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:37.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 1 -- 172.21.15.190:0/3217785971 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8834123130 con 0x7f88340a3fc0 2024-08-19T10:44:37.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 1 -- 172.21.15.190:0/3217785971 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88300074e0 con 0x7f88340a3fc0 2024-08-19T10:44:37.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:37.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 1 -- 172.21.15.190:0/3217785971 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8830005c40 con 0x7f88340a3fc0 2024-08-19T10:44:37.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:37.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:37.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882bfff640 4 set_mon_vals no callback set 2024-08-19T10:44:37.104 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f882a7fc640 10 monclient: dump: 2024-08-19T10:44:37.104 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:37.104 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:37.104 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:37.105 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient: _renew_subs 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.078+0000 7f883bab1640 1 -- 172.21.15.190:0/3217785971 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f8834137740 con 0x7f88340a3fc0 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.082+0000 7f882a7fc640 1 -- 172.21.15.190:0/3217785971 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8830007070 con 0x7f88340a3fc0 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.082+0000 7f883bab1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.082+0000 7f883bab1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.082+0000 7f883bab1640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.082+0000 7f883bab1640 1 -- 172.21.15.190:0/3217785971 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88340a3a80 con 0x7f88340a3fc0 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.082+0000 7f882a7fc640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f881803be00 0x7f881803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.082+0000 7f882a7fc640 1 -- 172.21.15.190:0/3217785971 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(223..223 src has 1..223) ==== 13919+0+0 (secure 0 0 0) 0x7f88300501f0 con 0x7f88340a3fc0 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.082+0000 7f883984d640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f881803be00 0x7f881803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:37.106 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.082+0000 7f883984d640 10 monclient: get_auth_request con 0x7f881803be00 auth_method 0 2024-08-19T10:44:37.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.082+0000 7f883984d640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f881803be00 0x7f881803e2c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f883406f1c0 tx=0x7f88240a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:37.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.086+0000 7f882a7fc640 1 -- 172.21.15.190:0/3217785971 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f883004c830 con 0x7f88340a3fc0 2024-08-19T10:44:37.107 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.086+0000 7f882a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:37.108 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.086+0000 7f882a7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:37.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:37.173+0000 7efd33890640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:44:37.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:37.173+0000 7efd33890640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:44:37.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:37.173+0000 7efd33890640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:37.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:37.173+0000 7efd33890640 1 -- 172.21.15.89:0/500502886 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efd2c0a6c40 con 0x7efd2c0a6860 2024-08-19T10:44:37.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.258+0000 7f883bab1640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:37.261 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.258+0000 7f883bab1640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:37.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.258+0000 7f883bab1640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:37.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:37.258+0000 7f883bab1640 1 -- 172.21.15.190:0/3217785971 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f883406dba0 con 0x7f88340a3fc0 2024-08-19T10:44:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:39.925+0000 7efd20ff9640 10 monclient: tick 2024-08-19T10:44:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:39.925+0000 7efd20ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:39.925+0000 7efd20ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:39.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:39.925+0000 7efd20ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:40.078 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:40.077+0000 7f88297fa640 10 monclient: tick 2024-08-19T10:44:40.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:40.077+0000 7f88297fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:40.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:40.077+0000 7f88297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:40.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:40.077+0000 7f88297fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:40.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:40.925+0000 7efd20ff9640 10 monclient: tick 2024-08-19T10:44:40.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:40.925+0000 7efd20ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:40.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:40.925+0000 7efd20ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:41.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:41.077+0000 7f88297fa640 10 monclient: tick 2024-08-19T10:44:41.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:41.077+0000 7f88297fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:41.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:41.077+0000 7f88297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:41.925+0000 7efd20ff9640 10 monclient: tick 2024-08-19T10:44:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:41.925+0000 7efd20ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:41.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:41.925+0000 7efd20ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:42.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:42.077+0000 7f88297fa640 10 monclient: tick 2024-08-19T10:44:42.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:42.077+0000 7f88297fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:42.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:42.077+0000 7f88297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:42.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.145+0000 7efd21ffb640 1 -- 172.21.15.89:0/500502886 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3174 (secure 0 0 0) 0x7efd28016070 con 0x7efd2c0a6860 2024-08-19T10:44:42.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.145+0000 7efd21ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:44:42.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.145+0000 7efd21ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:42.150 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":64,"quorum":[0,2,3,5,7,8],"quorum_names":["a","b","g","h","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:44:42.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 -- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efd1003bdf0 msgr2=0x7efd1003e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:42.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efd1003bdf0 0x7efd1003e2b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efd2c13faf0 tx=0x7efd1c008040 comp rx=0 tx=0).stop 2024-08-19T10:44:42.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 10 monclient: shutdown 2024-08-19T10:44:42.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:42.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 -- 172.21.15.89:0/500502886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd2c0a6860 msgr2=0x7efd2c1d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:42.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd2c0a6860 0x7efd2c1d18c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd280ae040 tx=0x7efd28019100 comp rx=0 tx=0).stop 2024-08-19T10:44:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 -- 172.21.15.89:0/500502886 shutdown_connections 2024-08-19T10:44:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efd1003bdf0 0x7efd1003e2b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd2c0a7180 0x7efd2c0a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd2c139620 0x7efd2c0a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 --2- 172.21.15.89:0/500502886 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efd2c0a6860 0x7efd2c1d18c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 -- 172.21.15.89:0/500502886 >> 172.21.15.89:0/500502886 conn(0x7efd2c074b40 msgr2=0x7efd2c0ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 -- 172.21.15.89:0/500502886 shutdown_connections 2024-08-19T10:44:42.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:42.149+0000 7efd33890640 1 -- 172.21.15.89:0/500502886 wait complete. 2024-08-19T10:44:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:43.077+0000 7f88297fa640 10 monclient: tick 2024-08-19T10:44:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:43.077+0000 7f88297fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:43.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:43.077+0000 7f88297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:44.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:44.077+0000 7f88297fa640 10 monclient: tick 2024-08-19T10:44:44.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:44.077+0000 7f88297fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:44.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:44.077+0000 7f88297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:45.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:45.077+0000 7f88297fa640 10 monclient: tick 2024-08-19T10:44:45.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:45.077+0000 7f88297fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:45.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:45.077+0000 7f88297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:45.176 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:44:45.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient: get_monmap_and_config 2024-08-19T10:44:45.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient: build_initial_monmap 2024-08-19T10:44:45.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient: monmap: 2024-08-19T10:44:45.305 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:45.306834+0000 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:45.306834+0000 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:45.306 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:45.307 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:45.307 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:45.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 1 Processor -- start 2024-08-19T10:44:45.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 1 -- start start 2024-08-19T10:44:45.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient: init 2024-08-19T10:44:45.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:45.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient: _add_conns ranks=[9,10,7,3,0,2,4,6,8,5,1] 2024-08-19T10:44:45.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient(hunting): picked mon.noname-a con 0x7f27fc0a3c90 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient(hunting): picked mon.noname-f con 0x7f27fc0a45b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient(hunting): picked mon.noname-i con 0x7f27fc139e30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f27fc0a0b60 con 0x7f27fc0a3c90 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f27fc0a1090 con 0x7f27fc0a45b0 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f27fc0a1370 con 0x7f27fc139e30 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:45.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2803aca640 10 monclient(hunting): authenticate will time out at 1335.123528s 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2801866640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2801866640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40214/0 (socket says 172.21.15.89:40214) 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2801866640 1 -- 172.21.15.89:0/3502914817 learned_addr learned my addr 172.21.15.89:0/3502914817 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2801866640 10 monclient(hunting): get_auth_request con 0x7f27fc0a3c90 auth_method 0 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2801866640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2801866640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2801866640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2801866640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 msgr2=0x7f27fc0a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2801866640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 msgr2=0x7f27fc0a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.301+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:45.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2801065640 10 monclient(hunting): get_auth_request con 0x7f27fc0a45b0 auth_method 0 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2801065640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2801065640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2801065640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2802067640 10 monclient(hunting): get_auth_request con 0x7f27fc139e30 auth_method 0 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2802067640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2802067640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2802067640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2802067640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 msgr2=0x7f27fc13a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2802067640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 msgr2=0x7f27fc13a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 msgr2=0x7f27fc1398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:44:45.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:45.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 msgr2=0x7f27fc1398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:45.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:45.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.305+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:45.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:45.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801866640 10 monclient(hunting): get_auth_request con 0x7f27fc0a3c90 auth_method 0 2024-08-19T10:44:45.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801866640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:45.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801866640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:45.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801866640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:44:45.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801866640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 msgr2=0x7f27fc0a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:44:45.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801866640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 msgr2=0x7f27fc0a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:45.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:45.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:45.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2802067640 10 monclient(hunting): get_auth_request con 0x7f27fc139e30 auth_method 0 2024-08-19T10:44:45.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2802067640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2802067640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2802067640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801065640 10 monclient(hunting): get_auth_request con 0x7f27fc0a45b0 auth_method 0 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801065640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801065640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801065640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2802067640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 msgr2=0x7f27fc13a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2802067640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 msgr2=0x7f27fc13a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:44:45.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 msgr2=0x7f27fc1398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:44:45.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 msgr2=0x7f27fc1398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:45.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:45.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.509+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:44:45.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:45.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801866640 10 monclient(hunting): get_auth_request con 0x7f27fc0a3c90 auth_method 0 2024-08-19T10:44:45.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801866640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:45.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801866640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:45.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801866640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:44:45.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:45.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801866640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 msgr2=0x7f27fc0a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:44:45.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801866640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 msgr2=0x7f27fc0a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2802067640 10 monclient(hunting): get_auth_request con 0x7f27fc139e30 auth_method 0 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2802067640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2802067640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2802067640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801065640 10 monclient(hunting): get_auth_request con 0x7f27fc0a45b0 auth_method 0 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801065640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801065640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801065640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2802067640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 msgr2=0x7f27fc13a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:44:45.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2802067640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 msgr2=0x7f27fc13a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:45.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:45.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:44:45.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 msgr2=0x7f27fc1398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:44:45.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 msgr2=0x7f27fc1398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:45.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:45.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:45.913+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:44:46.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:46.077+0000 7f88297fa640 10 monclient: tick 2024-08-19T10:44:46.079 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:46.077+0000 7f88297fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:46.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:46.077+0000 7f88297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801866640 10 monclient(hunting): get_auth_request con 0x7f27fc0a3c90 auth_method 0 2024-08-19T10:44:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801866640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801866640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801866640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:44:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801866640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 msgr2=0x7f27fc0a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:44:46.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801866640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 msgr2=0x7f27fc0a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2802067640 10 monclient(hunting): get_auth_request con 0x7f27fc139e30 auth_method 0 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2802067640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2802067640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2802067640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801065640 10 monclient(hunting): get_auth_request con 0x7f27fc0a45b0 auth_method 0 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801065640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801065640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801065640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:44:46.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2802067640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 msgr2=0x7f27fc13a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:44:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2802067640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 msgr2=0x7f27fc13a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 msgr2=0x7f27fc1398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:44:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 msgr2=0x7f27fc1398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:44:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2802067640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:44:46.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:46.717+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:44:47.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:47.077+0000 7f88297fa640 10 monclient: tick 2024-08-19T10:44:47.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:47.077+0000 7f88297fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:47.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:47.077+0000 7f88297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:48.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.077+0000 7f88297fa640 10 monclient: tick 2024-08-19T10:44:48.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.077+0000 7f88297fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:48.080 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.077+0000 7f88297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:48.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.181+0000 7f882a7fc640 1 -- 172.21.15.190:0/3217785971 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h28' already exists v224) ==== 147+0+0 (secure 0 0 0) 0x7f8830011d80 con 0x7f88340a3fc0 2024-08-19T10:44:48.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.181+0000 7f882a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:48.182 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.181+0000 7f882a7fc640 10 monclient: _finish_command 2 = system:0 bucket 'h28' already exists 2024-08-19T10:44:48.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.289+0000 7f883bab1640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:48.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.289+0000 7f883bab1640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:48.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.289+0000 7f883bab1640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:48.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.289+0000 7f883bab1640 1 -- 172.21.15.190:0/3217785971 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f8834137d50 con 0x7f88340a3fc0 2024-08-19T10:44:48.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f882a7fc640 1 -- 172.21.15.190:0/3217785971 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h28' already exists v224) ==== 147+0+0 (secure 0 0 0) 0x7f883001a360 con 0x7f88340a3fc0 2024-08-19T10:44:48.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f882a7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h28", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:48.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f882a7fc640 10 monclient: _finish_command 3 = system:0 bucket 'h28' already exists 2024-08-19T10:44:48.294 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h28' already exists 2024-08-19T10:44:48.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 -- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f881803be00 msgr2=0x7f881803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:48.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f881803be00 0x7f881803e2c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f883406f1c0 tx=0x7f88240a6000 comp rx=0 tx=0).stop 2024-08-19T10:44:48.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 10 monclient: shutdown 2024-08-19T10:44:48.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:48.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 -- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88340a3fc0 msgr2=0x7f8834137200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:48.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88340a3fc0 0x7f8834137200 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f883000c480 tx=0x7f883000c940 comp rx=0 tx=0).stop 2024-08-19T10:44:48.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 -- 172.21.15.190:0/3217785971 shutdown_connections 2024-08-19T10:44:48.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f881803be00 0x7f881803e2c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88340a0ec0 0x7f883409f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88340a3fc0 0x7f8834137200 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 --2- 172.21.15.190:0/3217785971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88340a19d0 0x7f8834136cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 -- 172.21.15.190:0/3217785971 >> 172.21.15.190:0/3217785971 conn(0x7f88340a4790 msgr2=0x7f883413f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:48.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 -- 172.21.15.190:0/3217785971 shutdown_connections 2024-08-19T10:44:48.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.293+0000 7f883bab1640 1 -- 172.21.15.190:0/3217785971 wait complete. 2024-08-19T10:44:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 10 monclient(hunting): tick 2024-08-19T10:44:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 monclient(hunting): continuing hunt 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 msgr2=0x7f27fc13a2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27fc139e30 0x7f27fc13a2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 msgr2=0x7f27fc1398f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.190:3300/0 conn(0x7f27fc0a45b0 0x7f27fc1398f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 -- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 msgr2=0x7f27fc0a4070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 --2- 172.21.15.89:0/3502914817 >> v2:172.21.15.89:3300/0 conn(0x7f27fc0a3c90 0x7f27fc0a4070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 10 monclient: _add_conns ranks=[3,5,1,4,0,8,6,2] 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27e40037e0 0x7f27e4005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 10 monclient(hunting): picked mon.noname-g con 0x7f27e40037e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27e4006260 0x7f27e40086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 10 monclient(hunting): picked mon.noname-h con 0x7f27e4006260 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f27e4008dc0 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 -- 172.21.15.89:0/3502914817 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27fc06cf40 con 0x7f27e4008dc0 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 -- 172.21.15.89:0/3502914817 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f27fc0a0b60 con 0x7f27e40037e0 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.301+0000 7f27fb7fe640 1 -- 172.21.15.89:0/3502914817 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f27fc0a1090 con 0x7f27e4006260 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f27fb7fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27e4006260 0x7f27e40086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27e40037e0 0x7f27e4005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 10 monclient(hunting): get_auth_request con 0x7f27e4006260 auth_method 0 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801866640 10 monclient(hunting): get_auth_request con 0x7f27e40037e0 auth_method 0 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801866640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801866640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801866640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801866640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801866640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801866640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 10 monclient(hunting): handle_auth_done global_id 6116 payload 274 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27e40037e0 msgr2=0x7f27e4005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27e40037e0 0x7f27e4005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 >> v1:172.21.15.190:6789/0 conn(0x7f27e4008dc0 legacy=0x7f27e400b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:48.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 10 monclient: _finish_hunting 0 2024-08-19T10:44:48.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 1 monclient: found mon.noname-h 2024-08-19T10:44:48.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:48.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:48.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e400dd00 con 0x7f27e4006260 2024-08-19T10:44:48.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2802067640 1 -- 172.21.15.89:0/3502914817 reap_dead start 2024-08-19T10:44:48.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.305+0000 7f2801866640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27e40037e0 0x7f27e4005c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:48.314 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:48.314 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h29 host rack=r1 2024-08-19T10:44:48.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2801065640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27e4006260 msgr2=0x7f27e40086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:48.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2801065640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27e4006260 0x7f27e40086f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0099e0 tx=0x7f27ec01d3e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=89624a7482857ba4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:48.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 1 -- 172.21.15.89:0/3502914817 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f27ec0b3070 con 0x7f27e4006260 2024-08-19T10:44:48.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:48.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:48.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 10 monclient: dump: 2024-08-19T10:44:48.317 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:48.317 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 10 monclient: _finish_auth 0 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 10 monclient: _check_auth_tickets 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 1 -- 172.21.15.89:0/3502914817 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27ec0ac780 con 0x7f27e4006260 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 1 -- 172.21.15.89:0/3502914817 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f27ec0acde0 con 0x7f27e4006260 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2800864640 10 monclient: dump: 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.319 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 5 monclient: authenticate success, global_id 6116 2024-08-19T10:44:48.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:48.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 4 set_mon_vals no callback set 2024-08-19T10:44:48.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 10 monclient: shutdown 2024-08-19T10:44:48.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:48.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 1 -- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27e4006260 msgr2=0x7f27e40086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:48.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27e4006260 0x7f27e40086f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0099e0 tx=0x7f27ec01d3e0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 1 -- 172.21.15.89:0/3502914817 shutdown_connections 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 1 --2- 172.21.15.89:0/3502914817 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27e4006260 0x7f27e40086f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 1 -- 172.21.15.89:0/3502914817 >> 172.21.15.89:0/3502914817 conn(0x7f27fc06de40 msgr2=0x7f27fc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 1 -- 172.21.15.89:0/3502914817 shutdown_connections 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 1 -- 172.21.15.89:0/3502914817 wait complete. 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 10 monclient: build_initial_monmap 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.313+0000 7f2803aca640 10 monclient: monmap: 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:48.324 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 1 Processor -- start 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 1 -- start start 2024-08-19T10:44:48.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 10 monclient: init 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 10 monclient: _add_conns ranks=[7,3,5,1,6,0,8,4,2] 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27e4006260 0x7f27fc121ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 10 monclient(hunting): picked mon.noname-i con 0x7f27e4006260 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27fc122010 0x7f27fc122490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 10 monclient(hunting): picked mon.noname-g con 0x7f27fc122010 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27fc1229d0 0x7f27fc1dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 10 monclient(hunting): picked mon.noname-h con 0x7f27fc1229d0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f27fc06d7c0 con 0x7f27fc122010 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:48.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f27fc0a53f0 con 0x7f27fc1229d0 2024-08-19T10:44:48.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:48.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f27fc0a49c0 con 0x7f27e4006260 2024-08-19T10:44:48.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:48.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2803aca640 10 monclient(hunting): authenticate will time out at 1338.137387s 2024-08-19T10:44:48.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801065640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27fc122010 0x7f27fc122490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:48.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801065640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27fc122010 0x7f27fc122490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:32776/0 (socket says 172.21.15.89:32776) 2024-08-19T10:44:48.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2802067640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27fc1229d0 0x7f27fc1dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:48.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27e4006260 0x7f27fc121ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:48.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801065640 1 -- 172.21.15.89:0/1404937261 learned_addr learned my addr 172.21.15.89:0/1404937261 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:48.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801065640 10 monclient(hunting): get_auth_request con 0x7f27fc122010 auth_method 0 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801065640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801065640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801065640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 10 monclient(hunting): get_auth_request con 0x7f27e4006260 auth_method 0 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2802067640 10 monclient(hunting): get_auth_request con 0x7f27fc1229d0 auth_method 0 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2802067640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2802067640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2802067640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:48.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801065640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801065640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801065640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2802067640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2802067640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2802067640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 10 monclient(hunting): handle_auth_done global_id 15415 payload 824 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 1 -- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27fc1229d0 msgr2=0x7f27fc1dc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27fc1229d0 0x7f27fc1dc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 1 -- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27fc122010 msgr2=0x7f27fc122490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:48.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27fc122010 0x7f27fc122490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 10 monclient: _finish_hunting 0 2024-08-19T10:44:48.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 1 monclient: found mon.noname-i 2024-08-19T10:44:48.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:48.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:48.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.317+0000 7f2801866640 1 -- 172.21.15.89:0/1404937261 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc12c120 con 0x7f27e4006260 2024-08-19T10:44:48.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2801866640 1 -- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27e4006260 msgr2=0x7f27fc121ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:48.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2801866640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27e4006260 0x7f27fc121ad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0a4e70 tx=0x7f27f0017e10 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 1 -- 172.21.15.89:0/1404937261 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f27f000f040 con 0x7f27e4006260 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 10 monclient: dump: 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:48.333 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:48.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 1 -- 172.21.15.89:0/1404937261 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27f0018450 con 0x7f27e4006260 2024-08-19T10:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 1 -- 172.21.15.89:0/1404937261 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f27f0018a70 con 0x7f27e4006260 2024-08-19T10:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2803aca640 5 monclient: authenticate success, global_id 15415 2024-08-19T10:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2803aca640 10 monclient: _renew_subs 2024-08-19T10:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2803aca640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2803aca640 1 -- 172.21.15.89:0/1404937261 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc12cbe0 con 0x7f27e4006260 2024-08-19T10:44:48.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fbfff640 4 set_mon_vals no callback set 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 10 monclient: dump: 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:48.336 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2803aca640 10 monclient: _renew_subs 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2803aca640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2803aca640 1 -- 172.21.15.89:0/1404937261 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc12cea0 con 0x7f27e4006260 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2803aca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2803aca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2803aca640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2803aca640 1 -- 172.21.15.89:0/1404937261 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27fc064590 con 0x7f27e4006260 2024-08-19T10:44:48.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 1 -- 172.21.15.89:0/1404937261 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f27f0018c90 con 0x7f27e4006260 2024-08-19T10:44:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f27e403d370 0x7f27e403f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f27fa7fc640 1 -- 172.21.15.89:0/1404937261 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(224..224 src has 1..224) ==== 14132+0+0 (secure 0 0 0) 0x7f27f0052050 con 0x7f27e4006260 2024-08-19T10:44:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2801065640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f27e403d370 0x7f27e403f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.321+0000 7f2801065640 10 monclient: get_auth_request con 0x7f27e403d370 auth_method 0 2024-08-19T10:44:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.325+0000 7f2801065640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f27e403d370 0x7f27e403f830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f27ec01d420 tx=0x7f27ec0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.337+0000 7f27fa7fc640 1 -- 172.21.15.89:0/1404937261 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f27fc12cbe0 con 0x7f27e4006260 2024-08-19T10:44:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.337+0000 7f27fa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:48.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.337+0000 7f27fa7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:48.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient: get_monmap_and_config 2024-08-19T10:44:48.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient: build_initial_monmap 2024-08-19T10:44:48.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient: monmap: 2024-08-19T10:44:48.434 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:48.434 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:48.434 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:48.436036+0000 2024-08-19T10:44:48.434 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:48.436036+0000 2024-08-19T10:44:48.434 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:48.435 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:48.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 1 Processor -- start 2024-08-19T10:44:48.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 1 -- start start 2024-08-19T10:44:48.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient: init 2024-08-19T10:44:48.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:48.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient: _add_conns ranks=[1,9,0,3,4,10,6,2,5,7,8] 2024-08-19T10:44:48.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9f840a69c0 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9f84139e90 0x7f9f8413e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient(hunting): picked mon.noname-a con 0x7f9f84139e90 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9f8413a2a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f840a46b0 con 0x7f9f8413a2a0 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f840a4990 con 0x7f9f840a69c0 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f9f840a4fe0 con 0x7f9f84139e90 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f88a6e640 10 monclient(hunting): authenticate will time out at 1334.006725s 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f8259c640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f9f840a69c0 0x7f9f84139780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:35898/0 (socket says 172.21.15.190:35898) 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f8259c640 1 -- 172.21.15.190:0/1878875183 learned_addr learned my addr 172.21.15.190:0/1878875183 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:44:48.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 1 --2- 172.21.15.190:0/1878875183 >> v2:172.21.15.89:3300/0 conn(0x7f9f84139e90 0x7f9f8413e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:48.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 10 monclient(hunting): get_auth_request con 0x7f9f84139e90 auth_method 0 2024-08-19T10:44:48.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:48.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:48.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:48.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:48.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:48.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:48.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 10 monclient(hunting): handle_auth_done global_id 6111 payload 274 2024-08-19T10:44:48.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 1 -- 172.21.15.190:0/1878875183 >> v1:172.21.15.190:6789/0 conn(0x7f9f840a69c0 legacy=0x7f9f84139780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:48.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 1 -- 172.21.15.190:0/1878875183 >> v1:172.21.15.89:6789/0 conn(0x7f9f8413a2a0 legacy=0x7f9f84140f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:48.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:44:48.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 1 monclient: found mon.noname-a 2024-08-19T10:44:48.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:48.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:48.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.433+0000 7f9f81d9b640 1 -- 172.21.15.190:0/1878875183 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f840a4c70 con 0x7f9f84139e90 2024-08-19T10:44:48.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 1 -- 172.21.15.190:0/1878875183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84139e90 msgr2=0x7f9f8413e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:48.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 1 --2- 172.21.15.190:0/1878875183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84139e90 0x7f9f8413e9c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9f780097c0 tx=0x7f9f7801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446f1f89a543c70a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:48.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 1 -- 172.21.15.190:0/1878875183 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9f780b0070 con 0x7f9f84139e90 2024-08-19T10:44:48.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:48.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:48.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 10 monclient: dump: 2024-08-19T10:44:48.447 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:48.447 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:48.447 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:48.448 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 10 monclient: _finish_auth 0 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 10 monclient: _check_auth_tickets 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 5 monclient: authenticate success, global_id 6111 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 1 -- 172.21.15.190:0/1878875183 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f780aa3a0 con 0x7f9f84139e90 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 1 -- 172.21.15.190:0/1878875183 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9f780aaa40 con 0x7f9f84139e90 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 4 set_mon_vals no callback set 2024-08-19T10:44:48.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:48.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8159a640 10 monclient: dump: 2024-08-19T10:44:48.450 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:48.450 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:48.450 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.450 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.450 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:48.450 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:48.450 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:48.450 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:48.451 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:48.451 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:48.451 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:48.451 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:48.451 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:48.451 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:48.451 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:48.451 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:48.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient: shutdown 2024-08-19T10:44:48.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:48.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 -- 172.21.15.190:0/1878875183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84139e90 msgr2=0x7f9f8413e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:48.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 --2- 172.21.15.190:0/1878875183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84139e90 0x7f9f8413e9c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9f780097c0 tx=0x7f9f7801c3d0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 -- 172.21.15.190:0/1878875183 shutdown_connections 2024-08-19T10:44:48.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 --2- 172.21.15.190:0/1878875183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f84139e90 0x7f9f8413e9c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 -- 172.21.15.190:0/1878875183 >> 172.21.15.190:0/1878875183 conn(0x7f9f840abfc0 msgr2=0x7f9f840ac3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:48.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 -- 172.21.15.190:0/1878875183 shutdown_connections 2024-08-19T10:44:48.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 -- 172.21.15.190:0/1878875183 wait complete. 2024-08-19T10:44:48.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient: build_initial_monmap 2024-08-19T10:44:48.453 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient: monmap: 2024-08-19T10:44:48.453 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:48.453 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:48.453 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:48.453 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:48.453 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:48.453 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 Processor -- start 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 -- start start 2024-08-19T10:44:48.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient: init 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient: _add_conns ranks=[0,3,2,6,4,7,5,1,8] 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f840a69c0 0x7f9f841d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient(hunting): picked mon.noname-a con 0x7f9f840a69c0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f84139e90 0x7f9f84138920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient(hunting): picked mon.noname-g con 0x7f9f84139e90 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8413a2a0 0x7f9f8413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient(hunting): picked mon.noname-b con 0x7f9f8413a2a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9f84050c20 con 0x7f9f840a69c0 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9f84144e50 con 0x7f9f8413a2a0 2024-08-19T10:44:48.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8406a540 con 0x7f9f84139e90 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f88a6e640 10 monclient(hunting): authenticate will time out at 1334.015040s 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f84139e90 0x7f9f84138920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f84139e90 0x7f9f84138920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:55574/0 (socket says 172.21.15.190:55574) 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 1 -- 172.21.15.190:0/3181142882 learned_addr learned my addr 172.21.15.190:0/3181142882 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 10 monclient(hunting): get_auth_request con 0x7f9f84139e90 auth_method 0 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f840a69c0 0x7f9f841d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 1 -- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f84139e90 msgr2=0x7f9f84138920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:44:48.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 1 -- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f84139e90 msgr2=0x7f9f84138920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f84139e90 0x7f9f84138920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 10 monclient(hunting): get_auth_request con 0x7f9f840a69c0 auth_method 0 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f81d9b640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f84139e90 0x7f9f84138920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f82d9d640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8413a2a0 0x7f9f8413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f82d9d640 10 monclient(hunting): get_auth_request con 0x7f9f8413a2a0 auth_method 0 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f82d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f82d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f82d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:48.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f82d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f82d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f82d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 10 monclient(hunting): handle_auth_done global_id 6120 payload 824 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 1 -- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f84139e90 msgr2=0x7f9f84138920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f84139e90 0x7f9f84138920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 1 -- 172.21.15.190:0/3181142882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8413a2a0 msgr2=0x7f9f8413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8413a2a0 0x7f9f8413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 10 monclient: _finish_hunting 0 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 1 monclient: found mon.noname-a 2024-08-19T10:44:48.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:48.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:48.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.441+0000 7f9f8259c640 1 -- 172.21.15.190:0/3181142882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f84141670 con 0x7f9f840a69c0 2024-08-19T10:44:48.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f8259c640 1 -- 172.21.15.190:0/3181142882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f840a69c0 msgr2=0x7f9f841d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:48.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f8259c640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f840a69c0 0x7f9f841d0f30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400c440 tx=0x7f9f7400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:48.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 1 -- 172.21.15.190:0/3181142882 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9f74017070 con 0x7f9f840a69c0 2024-08-19T10:44:48.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:48.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:48.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 10 monclient: dump: 2024-08-19T10:44:48.461 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:48.461 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:48.462 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:48.463 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:48.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:48.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:48.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:48.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 1 -- 172.21.15.190:0/3181142882 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f74005d00 con 0x7f9f840a69c0 2024-08-19T10:44:48.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:48.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f88a6e640 5 monclient: authenticate success, global_id 6120 2024-08-19T10:44:48.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f88a6e640 10 monclient: _renew_subs 2024-08-19T10:44:48.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f88a6e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f88a6e640 1 -- 172.21.15.190:0/3181142882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f840a77a0 con 0x7f9f840a69c0 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f73fff640 4 set_mon_vals no callback set 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f88a6e640 10 monclient: _renew_subs 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f88a6e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 1 -- 172.21.15.190:0/3181142882 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9f74007540 con 0x7f9f840a69c0 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f88a6e640 1 -- 172.21.15.190:0/3181142882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f84141390 con 0x7f9f840a69c0 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 10 monclient: dump: 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.464 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:48.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f88a6e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:48.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f88a6e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:48.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f88a6e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:48.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f88a6e640 1 -- 172.21.15.190:0/3181142882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f84139780 con 0x7f9f840a69c0 2024-08-19T10:44:48.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.445+0000 7f9f72ffd640 1 -- 172.21.15.190:0/3181142882 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9f74007070 con 0x7f9f840a69c0 2024-08-19T10:44:48.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.449+0000 7f9f72ffd640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9f6403be00 0x7f9f6403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:48.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.449+0000 7f9f72ffd640 1 -- 172.21.15.190:0/3181142882 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(224..224 src has 1..224) ==== 14132+0+0 (secure 0 0 0) 0x7f9f74050650 con 0x7f9f840a69c0 2024-08-19T10:44:48.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.449+0000 7f9f81d9b640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9f6403be00 0x7f9f6403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:48.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.449+0000 7f9f81d9b640 10 monclient: get_auth_request con 0x7f9f6403be00 auth_method 0 2024-08-19T10:44:48.466 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.449+0000 7f9f81d9b640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9f6403be00 0x7f9f6403e2c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9f7801c200 tx=0x7f9f780a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:48.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.461+0000 7f9f72ffd640 1 -- 172.21.15.190:0/3181142882 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9f74011d80 con 0x7f9f840a69c0 2024-08-19T10:44:48.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.461+0000 7f9f72ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:48.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.461+0000 7f9f72ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.569+0000 7f2803aca640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:44:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.569+0000 7f2803aca640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:44:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.569+0000 7f2803aca640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:48.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.569+0000 7f2803aca640 1 -- 172.21.15.89:0/1404937261 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f27fc12d630 con 0x7f27e4006260 2024-08-19T10:44:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f27fa7fc640 1 -- 172.21.15.89:0/1404937261 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3174 (secure 0 0 0) 0x7f27f0051d80 con 0x7f27e4006260 2024-08-19T10:44:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f27fa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:44:48.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f27fa7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:48.576 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":68,"quorum":[0,2,3,5,7,8],"quorum_names":["a","b","g","h","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:44:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 -- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f27e403d370 msgr2=0x7f27e403f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f27e403d370 0x7f27e403f830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f27ec01d420 tx=0x7f27ec0a7000 comp rx=0 tx=0).stop 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 10 monclient: shutdown 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 -- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27e4006260 msgr2=0x7f27fc121ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27e4006260 0x7f27fc121ad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0a4e70 tx=0x7f27f0017e10 comp rx=0 tx=0).stop 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 -- 172.21.15.89:0/1404937261 shutdown_connections 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f27e403d370 0x7f27e403f830 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f27fc1229d0 0x7f27fc1dc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27fc122010 0x7f27fc122490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 --2- 172.21.15.89:0/1404937261 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f27e4006260 0x7f27fc121ad0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 -- 172.21.15.89:0/1404937261 >> 172.21.15.89:0/1404937261 conn(0x7f27fc06de40 msgr2=0x7f27fc070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 -- 172.21.15.89:0/1404937261 shutdown_connections 2024-08-19T10:44:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:48.573+0000 7f2803aca640 1 -- 172.21.15.89:0/1404937261 wait complete. 2024-08-19T10:44:48.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.637+0000 7f9f88a6e640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:48.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.637+0000 7f9f88a6e640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:48.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.637+0000 7f9f88a6e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:48.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:48.637+0000 7f9f88a6e640 1 -- 172.21.15.190:0/3181142882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f9f84140a40 con 0x7f9f840a69c0 2024-08-19T10:44:49.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.201+0000 7f9f72ffd640 1 -- 172.21.15.190:0/3181142882 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}]=0 added bucket h29 type host to location {rack=r1} v225) ==== 168+0+0 (secure 0 0 0) 0x7f9f7404bc00 con 0x7f9f840a69c0 2024-08-19T10:44:49.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.201+0000 7f9f72ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:49.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.201+0000 7f9f72ffd640 10 monclient: _finish_command 2 = system:0 added bucket h29 type host to location {rack=r1} 2024-08-19T10:44:49.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.309+0000 7f9f88a6e640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:49.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.309+0000 7f9f88a6e640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:49.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.309+0000 7f9f88a6e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:49.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.309+0000 7f9f88a6e640 1 -- 172.21.15.190:0/3181142882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f9f841402b0 con 0x7f9f840a69c0 2024-08-19T10:44:49.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f72ffd640 1 -- 172.21.15.190:0/3181142882 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h29' already exists v225) ==== 147+0+0 (secure 0 0 0) 0x7f9f7404bf80 con 0x7f9f840a69c0 2024-08-19T10:44:49.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f72ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h29", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:49.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f72ffd640 10 monclient: _finish_command 3 = system:0 bucket 'h29' already exists 2024-08-19T10:44:49.315 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h29' already exists 2024-08-19T10:44:49.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 1 -- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9f6403be00 msgr2=0x7f9f6403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:49.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9f6403be00 0x7f9f6403e2c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9f7801c200 tx=0x7f9f780a6000 comp rx=0 tx=0).stop 2024-08-19T10:44:49.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 10 monclient: shutdown 2024-08-19T10:44:49.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:49.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 1 -- 172.21.15.190:0/3181142882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f840a69c0 msgr2=0x7f9f841d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:49.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f840a69c0 0x7f9f841d0f30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400c440 tx=0x7f9f7400c900 comp rx=0 tx=0).stop 2024-08-19T10:44:49.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 1 -- 172.21.15.190:0/3181142882 shutdown_connections 2024-08-19T10:44:49.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9f6403be00 0x7f9f6403e2c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f8413a2a0 0x7f9f8413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f84139e90 0x7f9f84138920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 1 --2- 172.21.15.190:0/3181142882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f840a69c0 0x7f9f841d0f30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.313+0000 7f9f88a6e640 1 -- 172.21.15.190:0/3181142882 >> 172.21.15.190:0/3181142882 conn(0x7f9f840abfc0 msgr2=0x7f9f840ab290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:49.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.317+0000 7f9f88a6e640 1 -- 172.21.15.190:0/3181142882 shutdown_connections 2024-08-19T10:44:49.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.317+0000 7f9f88a6e640 1 -- 172.21.15.190:0/3181142882 wait complete. 2024-08-19T10:44:49.333 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:49.333 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h30 host rack=r1 2024-08-19T10:44:49.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient: get_monmap_and_config 2024-08-19T10:44:49.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient: build_initial_monmap 2024-08-19T10:44:49.455 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient: monmap: 2024-08-19T10:44:49.455 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:49.455 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:49.455 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:49.456085+0000 2024-08-19T10:44:49.456 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:49.456085+0000 2024-08-19T10:44:49.456 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:49.456 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:49.456 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:49.456 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:49.456 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:49.456 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:49.456 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:49.456 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:49.456 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:49.457 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:49.457 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:49.457 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:49.457 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:49.457 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:49.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 1 Processor -- start 2024-08-19T10:44:49.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 1 -- start start 2024-08-19T10:44:49.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient: init 2024-08-19T10:44:49.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:49.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient: _add_conns ranks=[9,3,8,6,5,0,2,4,1,10,7] 2024-08-19T10:44:49.460 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f330c0a3cb0 0x7f330c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:49.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient(hunting): picked mon.noname-a con 0x7f330c0a3cb0 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:49.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f330c0a45d0 0x7f330c13d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:49.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient(hunting): picked mon.noname-g con 0x7f330c0a45d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:49.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f330c13ded0 0x7f330c1402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:49.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient(hunting): picked mon.noname-e con 0x7f330c13ded0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:49.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:49.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f330c0a4fb0 con 0x7f330c0a3cb0 2024-08-19T10:44:49.461 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:49.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f330c0a0670 con 0x7f330c0a45d0 2024-08-19T10:44:49.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:49.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f330c0a0b30 con 0x7f330c13ded0 2024-08-19T10:44:49.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:49.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3310925640 10 monclient(hunting): authenticate will time out at 1335.026213s 2024-08-19T10:44:49.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f330c0a45d0 0x7f330c13d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:49.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f330c0a45d0 0x7f330c13d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:55588/0 (socket says 172.21.15.190:55588) 2024-08-19T10:44:49.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 1 -- 172.21.15.190:0/4110385850 learned_addr learned my addr 172.21.15.190:0/4110385850 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:49.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 10 monclient(hunting): get_auth_request con 0x7f330c0a45d0 auth_method 0 2024-08-19T10:44:49.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330a59c640 1 --2- 172.21.15.190:0/4110385850 >> v2:172.21.15.89:3300/0 conn(0x7f330c0a3cb0 0x7f330c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:49.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:49.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:49.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:49.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330ad9d640 1 --2- 172.21.15.190:0/4110385850 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f330c13ded0 0x7f330c1402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:49.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330ad9d640 10 monclient(hunting): get_auth_request con 0x7f330c13ded0 auth_method 0 2024-08-19T10:44:49.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:49.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:49.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:49.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330a59c640 10 monclient(hunting): get_auth_request con 0x7f330c0a3cb0 auth_method 0 2024-08-19T10:44:49.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:49.463 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:49.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:49.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:49.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:49.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:49.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:49.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:49.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:49.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:49.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:49.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f330a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:49.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 10 monclient(hunting): handle_auth_done global_id 34536 payload 274 2024-08-19T10:44:49.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 1 -- 172.21.15.190:0/4110385850 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f330c13ded0 msgr2=0x7f330c1402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:49.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 1 --2- 172.21.15.190:0/4110385850 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f330c13ded0 0x7f330c1402c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 1 -- 172.21.15.190:0/4110385850 >> v2:172.21.15.89:3300/0 conn(0x7f330c0a3cb0 msgr2=0x7f330c0a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:49.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 1 --2- 172.21.15.190:0/4110385850 >> v2:172.21.15.89:3300/0 conn(0x7f330c0a3cb0 0x7f330c0a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:44:49.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 1 monclient: found mon.noname-g 2024-08-19T10:44:49.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:49.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:49.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.453+0000 7f3309d9b640 1 -- 172.21.15.190:0/4110385850 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c0a0e10 con 0x7f330c0a45d0 2024-08-19T10:44:49.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.457+0000 7f3309d9b640 1 -- 172.21.15.190:0/4110385850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f330c0a45d0 msgr2=0x7f330c13d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:49.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.457+0000 7f3309d9b640 1 --2- 172.21.15.190:0/4110385850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f330c0a45d0 0x7f330c13d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3300009e50 tx=0x7f330001c8f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=7cc753dc918a1d1f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:49.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 1 -- 172.21.15.190:0/4110385850 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33000090d0 con 0x7f330c0a45d0 2024-08-19T10:44:49.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:49.472 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:49.473 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 10 monclient: dump: 2024-08-19T10:44:49.473 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:49.473 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:49.473 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:49.473 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:49.473 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:49.473 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:49.473 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:49.473 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:49.473 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:49.474 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:49.474 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:49.474 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:49.474 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:49.474 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:49.474 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:49.474 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:49.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 10 monclient: _finish_auth 0 2024-08-19T10:44:49.474 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 10 monclient: _check_auth_tickets 2024-08-19T10:44:49.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:49.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 5 monclient: authenticate success, global_id 34536 2024-08-19T10:44:49.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 1 -- 172.21.15.190:0/4110385850 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33000a5a30 con 0x7f330c0a45d0 2024-08-19T10:44:49.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:49.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:49.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 1 -- 172.21.15.190:0/4110385850 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33000a5be0 con 0x7f330c0a45d0 2024-08-19T10:44:49.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 4 set_mon_vals no callback set 2024-08-19T10:44:49.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:49.475 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330959a640 10 monclient: dump: 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:49.476 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:49.477 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:49.477 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:49.477 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:49.477 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:49.477 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:49.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient: shutdown 2024-08-19T10:44:49.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:49.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 -- 172.21.15.190:0/4110385850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f330c0a45d0 msgr2=0x7f330c13d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:49.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 --2- 172.21.15.190:0/4110385850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f330c0a45d0 0x7f330c13d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3300009e50 tx=0x7f330001c8f0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.477 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 -- 172.21.15.190:0/4110385850 shutdown_connections 2024-08-19T10:44:49.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 --2- 172.21.15.190:0/4110385850 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f330c13ded0 0x7f330c1402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 --2- 172.21.15.190:0/4110385850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f330c0a45d0 0x7f330c13d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 --2- 172.21.15.190:0/4110385850 >> v2:172.21.15.89:3300/0 conn(0x7f330c0a3cb0 0x7f330c0a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 -- 172.21.15.190:0/4110385850 >> 172.21.15.190:0/4110385850 conn(0x7f330c06de10 msgr2=0x7f330c06e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:49.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 -- 172.21.15.190:0/4110385850 shutdown_connections 2024-08-19T10:44:49.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 -- 172.21.15.190:0/4110385850 wait complete. 2024-08-19T10:44:49.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient: build_initial_monmap 2024-08-19T10:44:49.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient: monmap: 2024-08-19T10:44:49.478 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:49.478 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:49.479 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:49.479 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:49.479 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:49.479 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:49.479 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:49.479 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:49.480 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:49.480 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:49.480 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:49.480 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:49.480 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:49.480 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:49.480 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:49.480 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:49.480 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 Processor -- start 2024-08-19T10:44:49.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 -- start start 2024-08-19T10:44:49.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient: init 2024-08-19T10:44:49.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:49.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient: _add_conns ranks=[5,0,4,2,6,1,7,3,8] 2024-08-19T10:44:49.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330c0a3cb0 0x7f330c139820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:49.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient(hunting): picked mon.noname-h con 0x7f330c0a3cb0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:49.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f330c0a45d0 0x7f330c139d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:49.481 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient(hunting): picked mon.noname-a con 0x7f330c0a45d0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:49.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f330c13ded0 0x7f330c071a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:49.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient(hunting): picked mon.noname-c con 0x7f330c13ded0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:49.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:49.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0a0320 con 0x7f330c0a45d0 2024-08-19T10:44:49.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:49.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f330c0a4a30 con 0x7f330c13ded0 2024-08-19T10:44:49.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:49.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f330c0a1da0 con 0x7f330c0a3cb0 2024-08-19T10:44:49.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:49.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3310925640 10 monclient(hunting): authenticate will time out at 1335.034458s 2024-08-19T10:44:49.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330c0a3cb0 0x7f330c139820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:49.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330c0a3cb0 0x7f330c139820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:52770/0 (socket says 172.21.15.190:52770) 2024-08-19T10:44:49.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 1 -- 172.21.15.190:0/576254810 learned_addr learned my addr 172.21.15.190:0/576254810 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:49.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 10 monclient(hunting): get_auth_request con 0x7f330c0a3cb0 auth_method 0 2024-08-19T10:44:49.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:49.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:49.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:49.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:49.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:49.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:49.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 10 monclient(hunting): handle_auth_done global_id 6134 payload 824 2024-08-19T10:44:49.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f3309d9b640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f330c0a45d0 0x7f330c139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:49.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 1 -- 172.21.15.190:0/576254810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f330c13ded0 msgr2=0x7f330c071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:49.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330ad9d640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f330c13ded0 0x7f330c071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:49.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f330c13ded0 0x7f330c071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 1 -- 172.21.15.190:0/576254810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f330c0a45d0 msgr2=0x7f330c139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:49.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f330c0a45d0 0x7f330c139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:49.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:44:49.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 1 monclient: found mon.noname-h 2024-08-19T10:44:49.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:49.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:49.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.461+0000 7f330a59c640 1 -- 172.21.15.190:0/576254810 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c13fca0 con 0x7f330c0a3cb0 2024-08-19T10:44:49.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f330a59c640 1 -- 172.21.15.190:0/576254810 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330c0a3cb0 msgr2=0x7f330c139820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:49.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f330a59c640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330c0a3cb0 0x7f330c139820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00e930 tx=0x7f32fc00edf0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:49.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 1 -- 172.21.15.190:0/576254810 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32fc017070 con 0x7f330c0a3cb0 2024-08-19T10:44:49.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:49.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:49.486 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 10 monclient: dump: 2024-08-19T10:44:49.486 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:49.486 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:49.486 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:49.486 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:49.486 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:49.486 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:49.487 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:49.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:49.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3310925640 5 monclient: authenticate success, global_id 6134 2024-08-19T10:44:49.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 1 -- 172.21.15.190:0/576254810 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32fc00cc10 con 0x7f330c0a3cb0 2024-08-19T10:44:49.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:49.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 1 -- 172.21.15.190:0/576254810 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32fc00cdc0 con 0x7f330c0a3cb0 2024-08-19T10:44:49.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:49.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:49.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3308d99640 4 set_mon_vals no callback set 2024-08-19T10:44:49.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 10 monclient: dump: 2024-08-19T10:44:49.488 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:49.488 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:49.489 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:49.489 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:49.489 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:49.489 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:49.489 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:49.489 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:49.490 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:49.490 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:49.490 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:49.490 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:49.491 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:49.491 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:49.491 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:49.491 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:49.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3310925640 10 monclient: _renew_subs 2024-08-19T10:44:49.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3310925640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:49.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3310925640 1 -- 172.21.15.190:0/576254810 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c124f30 con 0x7f330c0a3cb0 2024-08-19T10:44:49.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3310925640 10 monclient: _renew_subs 2024-08-19T10:44:49.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3310925640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:49.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3310925640 1 -- 172.21.15.190:0/576254810 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c124c50 con 0x7f330c0a3cb0 2024-08-19T10:44:49.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3310925640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:49.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3310925640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:49.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3310925640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:49.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3310925640 1 -- 172.21.15.190:0/576254810 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c073b00 con 0x7f330c0a3cb0 2024-08-19T10:44:49.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 1 -- 172.21.15.190:0/576254810 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f32fc01a070 con 0x7f330c0a3cb0 2024-08-19T10:44:49.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32ec03be00 0x7f32ec03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:49.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f32faffd640 1 -- 172.21.15.190:0/576254810 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(225..225 src has 1..225) ==== 14345+0+0 (secure 0 0 0) 0x7f32fc00f040 con 0x7f330c0a3cb0 2024-08-19T10:44:49.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3309d9b640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32ec03be00 0x7f32ec03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:49.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.465+0000 7f3309d9b640 10 monclient: get_auth_request con 0x7f32ec03be00 auth_method 0 2024-08-19T10:44:49.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.469+0000 7f3309d9b640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32ec03be00 0x7f32ec03e2c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f330c0746f0 tx=0x7f33000a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:49.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.469+0000 7f32faffd640 1 -- 172.21.15.190:0/576254810 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f32fc021020 con 0x7f330c0a3cb0 2024-08-19T10:44:49.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.469+0000 7f32faffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:49.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.469+0000 7f32faffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:49.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.649+0000 7f3310925640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:49.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.649+0000 7f3310925640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:49.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.649+0000 7f3310925640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:49.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:49.649+0000 7f3310925640 1 -- 172.21.15.190:0/576254810 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f330c0a4090 con 0x7f330c0a3cb0 2024-08-19T10:44:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.713+0000 7f32faffd640 1 -- 172.21.15.190:0/576254810 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}]=0 added bucket h30 type host to location {rack=r1} v226) ==== 168+0+0 (secure 0 0 0) 0x7f330c13fca0 con 0x7f330c0a3cb0 2024-08-19T10:44:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.713+0000 7f32faffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:50.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.713+0000 7f32faffd640 10 monclient: _finish_command 2 = system:0 added bucket h30 type host to location {rack=r1} 2024-08-19T10:44:50.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.821+0000 7f3310925640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:50.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.821+0000 7f3310925640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:50.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.821+0000 7f3310925640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:50.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.821+0000 7f3310925640 1 -- 172.21.15.190:0/576254810 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f330c0008e0 con 0x7f330c0a3cb0 2024-08-19T10:44:50.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.821+0000 7f32faffd640 1 -- 172.21.15.190:0/576254810 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h30' already exists v226) ==== 147+0+0 (secure 0 0 0) 0x7f32fc01ac80 con 0x7f330c0a3cb0 2024-08-19T10:44:50.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.821+0000 7f32faffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h30", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:50.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.821+0000 7f32faffd640 10 monclient: _finish_command 3 = system:0 bucket 'h30' already exists 2024-08-19T10:44:50.825 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h30' already exists 2024-08-19T10:44:50.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 -- 172.21.15.190:0/576254810 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32ec03be00 msgr2=0x7f32ec03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:50.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32ec03be00 0x7f32ec03e2c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f330c0746f0 tx=0x7f33000a7070 comp rx=0 tx=0).stop 2024-08-19T10:44:50.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 10 monclient: shutdown 2024-08-19T10:44:50.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:50.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 -- 172.21.15.190:0/576254810 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330c0a3cb0 msgr2=0x7f330c139820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:50.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330c0a3cb0 0x7f330c139820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00e930 tx=0x7f32fc00edf0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 -- 172.21.15.190:0/576254810 shutdown_connections 2024-08-19T10:44:50.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f32ec03be00 0x7f32ec03e2c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f330c13ded0 0x7f330c071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f330c0a45d0 0x7f330c139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 --2- 172.21.15.190:0/576254810 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f330c0a3cb0 0x7f330c139820 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 -- 172.21.15.190:0/576254810 >> 172.21.15.190:0/576254810 conn(0x7f330c06de10 msgr2=0x7f330c0a7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:50.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 -- 172.21.15.190:0/576254810 shutdown_connections 2024-08-19T10:44:50.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.825+0000 7f3310925640 1 -- 172.21.15.190:0/576254810 wait complete. 2024-08-19T10:44:50.843 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:50.844 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h31 host rack=r1 2024-08-19T10:44:50.963 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.961+0000 7f78949f6640 10 monclient: get_monmap_and_config 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.961+0000 7f78949f6640 10 monclient: build_initial_monmap 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.961+0000 7f78949f6640 10 monclient: monmap: 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:50.965722+0000 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:50.965722+0000 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:50.964 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:50.965 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:50.965 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:50.965 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:50.965 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:50.965 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:50.965 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:50.965 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:50.965 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:50.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.961+0000 7f78949f6640 1 Processor -- start 2024-08-19T10:44:50.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.961+0000 7f78949f6640 1 -- start start 2024-08-19T10:44:50.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.961+0000 7f78949f6640 10 monclient: init 2024-08-19T10:44:50.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:50.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 10 monclient: _add_conns ranks=[8,2,5,7,9,4,10,3,6,1,0] 2024-08-19T10:44:50.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f78900a4720 0x7f78900a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:50.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 10 monclient(hunting): picked mon.noname-e con 0x7f78900a4720 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:50.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78900a2f30 0x7f78900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:50.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 10 monclient(hunting): picked mon.noname-b con 0x7f78900a2f30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:50.968 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7890136720 0x7f7890136b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 10 monclient(hunting): picked mon.noname-h con 0x7f7890136720 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f78900a0bc0 con 0x7f78900a2f30 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7890074350 con 0x7f7890136720 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f78900a5580 con 0x7f78900a4720 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f78949f6640 10 monclient(hunting): authenticate will time out at 1336.536801s 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7890136720 0x7f7890136b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7890136720 0x7f7890136b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:52784/0 (socket says 172.21.15.190:52784) 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 1 -- 172.21.15.190:0/1209884262 learned_addr learned my addr 172.21.15.190:0/1209884262 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:50.969 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 10 monclient(hunting): get_auth_request con 0x7f7890136720 auth_method 0 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f7885d9b640 1 --2- 172.21.15.190:0/1209884262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78900a2f30 0x7f78900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788e59c640 1 --2- 172.21.15.190:0/1209884262 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f78900a4720 0x7f78900a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f7885d9b640 10 monclient(hunting): get_auth_request con 0x7f78900a2f30 auth_method 0 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f7885d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f7885d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f7885d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:50.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788e59c640 10 monclient(hunting): get_auth_request con 0x7f78900a4720 auth_method 0 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 10 monclient(hunting): handle_auth_done global_id 6143 payload 274 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 1 -- 172.21.15.190:0/1209884262 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f78900a4720 msgr2=0x7f78900a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 1 --2- 172.21.15.190:0/1209884262 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f78900a4720 0x7f78900a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 1 -- 172.21.15.190:0/1209884262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78900a2f30 msgr2=0x7f78900a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 1 --2- 172.21.15.190:0/1209884262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78900a2f30 0x7f78900a5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 1 monclient: found mon.noname-h 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:50.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.965+0000 7f788ed9d640 1 -- 172.21.15.190:0/1209884262 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78900a2500 con 0x7f7890136720 2024-08-19T10:44:50.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788ed9d640 1 -- 172.21.15.190:0/1209884262 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7890136720 msgr2=0x7f7890136b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:50.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788ed9d640 1 --2- 172.21.15.190:0/1209884262 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7890136720 0x7f7890136b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7888009eb0 tx=0x7f788801c8f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=64c8b94fd8de02f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:50.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 1 -- 172.21.15.190:0/1209884262 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78880090d0 con 0x7f7890136720 2024-08-19T10:44:50.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:50.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:50.979 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 10 monclient: dump: 2024-08-19T10:44:50.979 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:50.979 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:50.979 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:50.980 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:50.981 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:50.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 10 monclient: _finish_auth 0 2024-08-19T10:44:50.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 10 monclient: _check_auth_tickets 2024-08-19T10:44:50.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:50.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 1 -- 172.21.15.190:0/1209884262 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78880a5a30 con 0x7f7890136720 2024-08-19T10:44:50.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:50.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 1 -- 172.21.15.190:0/1209884262 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78880a5be0 con 0x7f7890136720 2024-08-19T10:44:50.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 5 monclient: authenticate success, global_id 6143 2024-08-19T10:44:50.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:50.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 4 set_mon_vals no callback set 2024-08-19T10:44:50.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f788dd9b640 10 monclient: dump: 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:50.982 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient: shutdown 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 -- 172.21.15.190:0/1209884262 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7890136720 msgr2=0x7f7890136b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 --2- 172.21.15.190:0/1209884262 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7890136720 0x7f7890136b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7888009eb0 tx=0x7f788801c8f0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 -- 172.21.15.190:0/1209884262 shutdown_connections 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 --2- 172.21.15.190:0/1209884262 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7890136720 0x7f7890136b80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 --2- 172.21.15.190:0/1209884262 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f78900a2f30 0x7f78900a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.983 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 --2- 172.21.15.190:0/1209884262 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f78900a4720 0x7f78900a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 -- 172.21.15.190:0/1209884262 >> 172.21.15.190:0/1209884262 conn(0x7f78900a9310 msgr2=0x7f78900a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 -- 172.21.15.190:0/1209884262 shutdown_connections 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 -- 172.21.15.190:0/1209884262 wait complete. 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient: build_initial_monmap 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient: monmap: 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:50.984 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 Processor -- start 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 -- start start 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient: init 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:50.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient: _add_conns ranks=[4,0,6,7,8,5,3,2,1] 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78900a2f30 0x7f7890133140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient(hunting): picked mon.noname-c con 0x7f78900a2f30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78900a4720 0x7f7890133680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient(hunting): picked mon.noname-a con 0x7f78900a4720 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7890136720 0x7f789011f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient(hunting): picked mon.noname-d con 0x7f7890136720 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f78900a6270 con 0x7f78900a4720 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f78901374d0 con 0x7f78900a2f30 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:50.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7890140200 con 0x7f7890136720 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.969+0000 7f78949f6640 10 monclient(hunting): authenticate will time out at 1336.543899s 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78900a4720 0x7f7890133680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788e59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78900a2f30 0x7f7890133140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788e59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78900a2f30 0x7f7890133140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:57010/0 (socket says 172.21.15.190:57010) 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788ed9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7890136720 0x7f789011f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78900a4720 0x7f7890133680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:42296/0 (socket says 172.21.15.190:42296) 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788e59c640 1 -- 172.21.15.190:0/2305491345 learned_addr learned my addr 172.21.15.190:0/2305491345 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788e59c640 10 monclient(hunting): get_auth_request con 0x7f78900a2f30 auth_method 0 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 10 monclient(hunting): get_auth_request con 0x7f78900a4720 auth_method 0 2024-08-19T10:44:50.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788ed9d640 10 monclient(hunting): get_auth_request con 0x7f7890136720 auth_method 0 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788e59c640 1 -- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78900a2f30 msgr2=0x7f7890133140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788e59c640 1 -- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78900a2f30 msgr2=0x7f7890133140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788e59c640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78900a2f30 0x7f7890133140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:50.988 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788e59c640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78900a2f30 0x7f7890133140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788ed9d640 1 -- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7890136720 msgr2=0x7f789011f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788ed9d640 1 -- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7890136720 msgr2=0x7f789011f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788ed9d640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7890136720 0x7f789011f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788ed9d640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7890136720 0x7f789011f360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 10 monclient(hunting): handle_auth_done global_id 6138 payload 824 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 1 -- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7890136720 msgr2=0x7f789011f360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7890136720 0x7f789011f360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 1 -- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78900a2f30 msgr2=0x7f7890133140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78900a2f30 0x7f7890133140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 1 monclient: found mon.noname-a 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:50.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 1 -- 172.21.15.190:0/2305491345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7890121eb0 con 0x7f78900a4720 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 1 -- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78900a4720 msgr2=0x7f7890133680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f7885d9b640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78900a4720 0x7f7890133680 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f787800d5a0 tx=0x7f787800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 1 -- 172.21.15.190:0/2305491345 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7878019070 con 0x7f78900a4720 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 10 monclient: dump: 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:50.990 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 10 monclient: _finish_auth 0 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 10 monclient: _check_auth_tickets 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78949f6640 5 monclient: authenticate success, global_id 6138 2024-08-19T10:44:50.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78949f6640 10 monclient: _renew_subs 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78949f6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 1 -- 172.21.15.190:0/2305491345 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7878004450 con 0x7f78900a4720 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78949f6640 1 -- 172.21.15.190:0/2305491345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789009f3e0 con 0x7f78900a4720 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f788d59a640 4 set_mon_vals no callback set 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78949f6640 10 monclient: _renew_subs 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 1 -- 172.21.15.190:0/2305491345 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7878003a90 con 0x7f78900a4720 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78949f6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78949f6640 1 -- 172.21.15.190:0/2305491345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7890133e70 con 0x7f78900a4720 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.973+0000 7f78877fe640 10 monclient: dump: 2024-08-19T10:44:50.992 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:50.993 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:50.994 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:50.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.977+0000 7f78949f6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:50.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.977+0000 7f78949f6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:50.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.977+0000 7f78949f6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:50.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.977+0000 7f78949f6640 1 -- 172.21.15.190:0/2305491345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7890136b80 con 0x7f78900a4720 2024-08-19T10:44:50.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.977+0000 7f78877fe640 1 -- 172.21.15.190:0/2305491345 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7878003c30 con 0x7f78900a4720 2024-08-19T10:44:50.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.977+0000 7f78877fe640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f787003be00 0x7f787003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:50.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.977+0000 7f78877fe640 1 -- 172.21.15.190:0/2305491345 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(226..226 src has 1..226) ==== 14558+0+0 (secure 0 0 0) 0x7f787800f040 con 0x7f78900a4720 2024-08-19T10:44:50.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.977+0000 7f788e59c640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f787003be00 0x7f787003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:50.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.977+0000 7f788e59c640 10 monclient: get_auth_request con 0x7f787003be00 auth_method 0 2024-08-19T10:44:50.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.981+0000 7f788e59c640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f787003be00 0x7f787003e2c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f78800071c0 tx=0x7f7880008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:50.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.989+0000 7f78877fe640 1 -- 172.21.15.190:0/2305491345 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7878014e10 con 0x7f78900a4720 2024-08-19T10:44:50.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.989+0000 7f78877fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:50.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:50.989+0000 7f78877fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:51.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.161+0000 7f78949f6640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:51.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.161+0000 7f78949f6640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:51.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.161+0000 7f78949f6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:51.164 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.161+0000 7f78949f6640 1 -- 172.21.15.190:0/2305491345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f7890074550 con 0x7f78900a4720 2024-08-19T10:44:51.594 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:44:51.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.709+0000 7f78877fe640 1 -- 172.21.15.190:0/2305491345 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}]=0 added bucket h31 type host to location {rack=r1} v227) ==== 168+0+0 (secure 0 0 0) 0x7f787804c2f0 con 0x7f78900a4720 2024-08-19T10:44:51.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.709+0000 7f78877fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:51.711 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.709+0000 7f78877fe640 10 monclient: _finish_command 2 = system:0 added bucket h31 type host to location {rack=r1} 2024-08-19T10:44:51.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.721+0000 7f4e1ce91640 10 monclient: get_monmap_and_config 2024-08-19T10:44:51.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.721+0000 7f4e1ce91640 10 monclient: build_initial_monmap 2024-08-19T10:44:51.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.721+0000 7f4e1ce91640 10 monclient: monmap: 2024-08-19T10:44:51.726 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:51.726 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:51.726 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:51.727485+0000 2024-08-19T10:44:51.726 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:51.727485+0000 2024-08-19T10:44:51.726 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:51.726 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:51.727 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.721+0000 7f4e1ce91640 1 Processor -- start 2024-08-19T10:44:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.721+0000 7f4e1ce91640 1 -- start start 2024-08-19T10:44:51.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.721+0000 7f4e1ce91640 10 monclient: init 2024-08-19T10:44:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 10 monclient: _add_conns ranks=[2,7,8,1,4,6,3,5,9,10,0] 2024-08-19T10:44:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e180a3b30 0x7f4e180a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 10 monclient(hunting): picked mon.noname-b con 0x7f4e180a3b30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e180a3f10 0x7f4e180a4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 10 monclient(hunting): picked mon.noname-i con 0x7f4e180a3f10 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e181364f0 0x7f4e18136970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 10 monclient(hunting): picked mon.noname-e con 0x7f4e181364f0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4e180a08b0 con 0x7f4e180a3b30 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4e18074170 con 0x7f4e180a3f10 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4e180a0630 con 0x7f4e181364f0 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1ce91640 10 monclient(hunting): authenticate will time out at 1341.544598s 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e180a3b30 0x7f4e180a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e181364f0 0x7f4e18136970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e180a3b30 0x7f4e180a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56326/0 (socket says 172.21.15.89:56326) 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 10 monclient(hunting): get_auth_request con 0x7f4e181364f0 auth_method 0 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:51.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1659c640 1 -- 172.21.15.89:0/433925729 learned_addr learned my addr 172.21.15.89:0/433925729 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1659c640 10 monclient(hunting): get_auth_request con 0x7f4e180a3b30 auth_method 0 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:51.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 10 monclient(hunting): handle_auth_done global_id 5984 payload 274 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 1 -- 172.21.15.89:0/433925729 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e180a3f10 msgr2=0x7f4e180a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e15d9b640 1 --2- 172.21.15.89:0/433925729 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e180a3f10 0x7f4e180a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 1 --2- 172.21.15.89:0/433925729 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e180a3f10 0x7f4e180a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 1 -- 172.21.15.89:0/433925729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e180a3b30 msgr2=0x7f4e180a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 1 --2- 172.21.15.89:0/433925729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e180a3b30 0x7f4e180a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 1 monclient: found mon.noname-e 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e16d9d640 1 -- 172.21.15.89:0/433925729 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e180a1950 con 0x7f4e181364f0 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e1659c640 1 --2- 172.21.15.89:0/433925729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e180a3b30 0x7f4e180a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e15d9b640 10 monclient: get_auth_request con 0x7f4e180a3f10 auth_method 0 2024-08-19T10:44:51.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.725+0000 7f4e15d9b640 1 --2- 172.21.15.89:0/433925729 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e180a3f10 0x7f4e180a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e16d9d640 1 -- 172.21.15.89:0/433925729 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e181364f0 msgr2=0x7f4e18136970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:51.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e16d9d640 1 --2- 172.21.15.89:0/433925729 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e181364f0 0x7f4e18136970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000be90 tx=0x7f4e1001c8c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6f237e9b4a0de5b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 1 -- 172.21.15.89:0/433925729 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e100ac070 con 0x7f4e181364f0 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 10 monclient: dump: 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:51.737 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 10 monclient: _finish_auth 0 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 10 monclient: _check_auth_tickets 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 5 monclient: authenticate success, global_id 5984 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:51.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 1 -- 172.21.15.89:0/433925729 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e10009540 con 0x7f4e181364f0 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 1 -- 172.21.15.89:0/433925729 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e100096f0 con 0x7f4e181364f0 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 4 set_mon_vals no callback set 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1559a640 10 monclient: dump: 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:51.739 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:51.740 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 10 monclient: shutdown 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 1 -- 172.21.15.89:0/433925729 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e181364f0 msgr2=0x7f4e18136970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 1 --2- 172.21.15.89:0/433925729 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e181364f0 0x7f4e18136970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000be90 tx=0x7f4e1001c8c0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 1 -- 172.21.15.89:0/433925729 shutdown_connections 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 1 --2- 172.21.15.89:0/433925729 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4e181364f0 0x7f4e18136970 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 1 --2- 172.21.15.89:0/433925729 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4e180a3f10 0x7f4e180a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 1 --2- 172.21.15.89:0/433925729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4e180a3b30 0x7f4e180a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 1 -- 172.21.15.89:0/433925729 >> 172.21.15.89:0/433925729 conn(0x7f4e180a8f60 msgr2=0x7f4e180a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 1 -- 172.21.15.89:0/433925729 shutdown_connections 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 1 -- 172.21.15.89:0/433925729 wait complete. 2024-08-19T10:44:51.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 10 monclient: build_initial_monmap 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 10 monclient: monmap: 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:51.742 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 1 Processor -- start 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 1 -- start start 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.729+0000 7f4e1ce91640 10 monclient: init 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient: _add_conns ranks=[3,5,0,7,6,8,1,4,2] 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e180a3b30 0x7f4e1813a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient(hunting): picked mon.noname-g con 0x7f4e180a3b30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e180a3f10 0x7f4e1813a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient(hunting): picked mon.noname-h con 0x7f4e180a3f10 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e181364f0 0x7f4e1813adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient(hunting): picked mon.noname-a con 0x7f4e181364f0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1806d140 con 0x7f4e181364f0 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4e180a4870 con 0x7f4e180a3b30 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4e1806d2c0 con 0x7f4e180a3f10 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient(hunting): authenticate will time out at 1341.552194s 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e181364f0 0x7f4e1813adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e181364f0 0x7f4e1813adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51902/0 (socket says 172.21.15.89:51902) 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 1 -- 172.21.15.89:0/1580822295 learned_addr learned my addr 172.21.15.89:0/1580822295 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 10 monclient(hunting): get_auth_request con 0x7f4e181364f0 auth_method 0 2024-08-19T10:44:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e15d9b640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e180a3f10 0x7f4e1813a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1659c640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e180a3b30 0x7f4e1813a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1659c640 10 monclient(hunting): get_auth_request con 0x7f4e180a3b30 auth_method 0 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 10 monclient(hunting): handle_auth_done global_id 6147 payload 824 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 1 -- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e180a3f10 msgr2=0x7f4e1813a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:51.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e180a3f10 0x7f4e1813a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 1 -- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e180a3b30 msgr2=0x7f4e1813a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e180a3b30 0x7f4e1813a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:44:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 1 monclient: found mon.noname-a 2024-08-19T10:44:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 1 -- 172.21.15.89:0/1580822295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e181203c0 con 0x7f4e181364f0 2024-08-19T10:44:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e15d9b640 10 monclient: get_auth_request con 0x7f4e180a3f10 auth_method 0 2024-08-19T10:44:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e15d9b640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e180a3f10 0x7f4e1813a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 1 -- 172.21.15.89:0/1580822295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e181364f0 msgr2=0x7f4e1813adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e16d9d640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e181364f0 0x7f4e1813adb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000b040 tx=0x7f4e100092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 1 -- 172.21.15.89:0/1580822295 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e100ac070 con 0x7f4e181364f0 2024-08-19T10:44:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 10 monclient: dump: 2024-08-19T10:44:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 5 monclient: authenticate success, global_id 6147 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 1 -- 172.21.15.89:0/1580822295 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e10009d70 con 0x7f4e181364f0 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient: _renew_subs 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 1 -- 172.21.15.89:0/1580822295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1813c5d0 con 0x7f4e181364f0 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 1 -- 172.21.15.89:0/1580822295 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e100044b0 con 0x7f4e181364f0 2024-08-19T10:44:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e07fff640 4 set_mon_vals no callback set 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e06ffd640 10 monclient: dump: 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient: _renew_subs 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 1 -- 172.21.15.89:0/1580822295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e181362e0 con 0x7f4e181364f0 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.733+0000 7f4e1ce91640 1 -- 172.21.15.89:0/1580822295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e180a5f80 con 0x7f4e181364f0 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.737+0000 7f4e06ffd640 1 -- 172.21.15.89:0/1580822295 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4e100af070 con 0x7f4e181364f0 2024-08-19T10:44:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.737+0000 7f4e06ffd640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4df803be00 0x7f4df803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.737+0000 7f4e06ffd640 1 -- 172.21.15.89:0/1580822295 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 14771+0+0 (secure 0 0 0) 0x7f4e100eb510 con 0x7f4e181364f0 2024-08-19T10:44:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.737+0000 7f4e1659c640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4df803be00 0x7f4df803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.737+0000 7f4e1659c640 10 monclient: get_auth_request con 0x7f4df803be00 auth_method 0 2024-08-19T10:44:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.737+0000 7f4e1659c640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4df803be00 0x7f4df803e2c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00d440 tx=0x7f4e0c00f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.741+0000 7f4e06ffd640 1 -- 172.21.15.89:0/1580822295 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4e180a5f80 con 0x7f4e181364f0 2024-08-19T10:44:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.741+0000 7f4e06ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.741+0000 7f4e06ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:51.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.821+0000 7f78949f6640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:51.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.821+0000 7f78949f6640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:51.824 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.821+0000 7f78949f6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:51.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.821+0000 7f78949f6640 1 -- 172.21.15.190:0/2305491345 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f78901216f0 con 0x7f78900a4720 2024-08-19T10:44:51.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.821+0000 7f78877fe640 1 -- 172.21.15.190:0/2305491345 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h31' already exists v227) ==== 147+0+0 (secure 0 0 0) 0x7f787804c670 con 0x7f78900a4720 2024-08-19T10:44:51.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.821+0000 7f78877fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h31", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:51.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.821+0000 7f78877fe640 10 monclient: _finish_command 3 = system:0 bucket 'h31' already exists 2024-08-19T10:44:51.825 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h31' already exists 2024-08-19T10:44:51.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 -- 172.21.15.190:0/2305491345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f787003be00 msgr2=0x7f787003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:51.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f787003be00 0x7f787003e2c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f78800071c0 tx=0x7f7880008040 comp rx=0 tx=0).stop 2024-08-19T10:44:51.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 10 monclient: shutdown 2024-08-19T10:44:51.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:51.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 -- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78900a4720 msgr2=0x7f7890133680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:51.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78900a4720 0x7f7890133680 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f787800d5a0 tx=0x7f787800da60 comp rx=0 tx=0).stop 2024-08-19T10:44:51.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 -- 172.21.15.190:0/2305491345 shutdown_connections 2024-08-19T10:44:51.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f787003be00 0x7f787003e2c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7890136720 0x7f789011f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78900a4720 0x7f7890133680 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 --2- 172.21.15.190:0/2305491345 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f78900a2f30 0x7f7890133140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 -- 172.21.15.190:0/2305491345 >> 172.21.15.190:0/2305491345 conn(0x7f78900a9310 msgr2=0x7f789013bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:51.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 -- 172.21.15.190:0/2305491345 shutdown_connections 2024-08-19T10:44:51.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.825+0000 7f78949f6640 1 -- 172.21.15.190:0/2305491345 wait complete. 2024-08-19T10:44:51.844 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:51.844 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush add-bucket h32 host rack=r1 2024-08-19T10:44:51.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient: get_monmap_and_config 2024-08-19T10:44:51.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient: build_initial_monmap 2024-08-19T10:44:51.966 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient: monmap: 2024-08-19T10:44:51.966 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:51.966 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:51.966 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:51.967983+0000 2024-08-19T10:44:51.966 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:51.967983+0000 2024-08-19T10:44:51.966 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:51.966 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:51.966 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:51.967 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:51.967 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:51.967 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:51.967 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:51.967 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:51.967 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:51.967 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:51.967 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:51.967 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:51.967 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:51.967 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:51.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 1 Processor -- start 2024-08-19T10:44:51.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 1 -- start start 2024-08-19T10:44:51.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient: init 2024-08-19T10:44:51.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:51.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient: _add_conns ranks=[0,2,9,1,10,5,4,7,3,6,8] 2024-08-19T10:44:51.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1fd01396a0 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:51.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fd013a190 0x7f1fd013eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient(hunting): picked mon.noname-b con 0x7f1fd013a190 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:51.970 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1fd013f0a0 0x7f1fd0141490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient(hunting): picked mon.noname-a con 0x7f1fd013f0a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fd00a83e0 con 0x7f1fd01396a0 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f1fd00a4f20 con 0x7f1fd013f0a0 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd00a2f30 con 0x7f1fd013a190 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd7494640 10 monclient(hunting): authenticate will time out at 1337.538019s 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fd013a190 0x7f1fd013eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fd013a190 0x7f1fd013eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.190:34006/0 (socket says 172.21.15.190:34006) 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 1 -- 172.21.15.190:0/2995858791 learned_addr learned my addr 172.21.15.190:0/2995858791 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 10 monclient(hunting): get_auth_request con 0x7f1fd013a190 auth_method 0 2024-08-19T10:44:51.971 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd5230640 1 --1- 172.21.15.190:0/2995858791 >> v1:172.21.15.89:6789/0 conn(0x7f1fd01396a0 0x7f1fd0139a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.190:37718/0 (socket says 172.21.15.190:37718) 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd5a31640 1 --2- 172.21.15.190:0/2995858791 >> v2:172.21.15.89:3300/0 conn(0x7f1fd013f0a0 0x7f1fd0141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd5a31640 10 monclient(hunting): get_auth_request con 0x7f1fd013f0a0 auth_method 0 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd5a31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd5a31640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd5a31640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 10 monclient(hunting): handle_auth_done global_id 6113 payload 274 2024-08-19T10:44:51.972 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 1 -- 172.21.15.190:0/2995858791 >> v2:172.21.15.89:3300/0 conn(0x7f1fd013f0a0 msgr2=0x7f1fd0141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:51.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 1 --2- 172.21.15.190:0/2995858791 >> v2:172.21.15.89:3300/0 conn(0x7f1fd013f0a0 0x7f1fd0141490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 1 -- 172.21.15.190:0/2995858791 >> v1:172.21.15.89:6789/0 conn(0x7f1fd01396a0 legacy=0x7f1fd0139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:51.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 10 monclient: _finish_hunting 0 2024-08-19T10:44:51.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 1 monclient: found mon.noname-b 2024-08-19T10:44:51.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:51.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:44:51.973 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.965+0000 7f1fd4a2f640 1 -- 172.21.15.190:0/2995858791 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd00a35e0 con 0x7f1fd013a190 2024-08-19T10:44:51.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.969+0000 7f1fd4a2f640 1 -- 172.21.15.190:0/2995858791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fd013a190 msgr2=0x7f1fd013eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:51.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.969+0000 7f1fd4a2f640 1 --2- 172.21.15.190:0/2995858791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fd013a190 0x7f1fd013eb60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009870 tx=0x7f1fc001c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e10fd9f62b45c49 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:51.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.969+0000 7f1fc7fff640 1 -- 172.21.15.190:0/2995858791 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1fc00b0070 con 0x7f1fd013a190 2024-08-19T10:44:51.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.969+0000 7f1fc7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:51.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.969+0000 7f1fc7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:44:51.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.969+0000 7f1fc7fff640 10 monclient: dump: 2024-08-19T10:44:51.982 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:51.982 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:51.982 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.983 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.983 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:51.983 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:51.983 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:51.983 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:51.983 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:51.983 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:51.983 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.969+0000 7f1fc7fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.969+0000 7f1fc7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.969+0000 7f1fc7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 5 monclient: authenticate success, global_id 6113 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fc7fff640 1 -- 172.21.15.190:0/2995858791 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc00aa810 con 0x7f1fd013a190 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:51.984 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fc7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:51.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:51.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fc7fff640 1 -- 172.21.15.190:0/2995858791 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1fc00a9300 con 0x7f1fd013a190 2024-08-19T10:44:51.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fc7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:51.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 4 set_mon_vals no callback set 2024-08-19T10:44:51.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fc7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:51.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fc7fff640 10 monclient: dump: 2024-08-19T10:44:51.985 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:51.985 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:51.985 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.985 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.985 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:51.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient: shutdown 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 -- 172.21.15.190:0/2995858791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fd013a190 msgr2=0x7f1fd013eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 --2- 172.21.15.190:0/2995858791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fd013a190 0x7f1fd013eb60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009870 tx=0x7f1fc001c420 comp rx=0 tx=0).stop 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 -- 172.21.15.190:0/2995858791 shutdown_connections 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 --2- 172.21.15.190:0/2995858791 >> v2:172.21.15.89:3300/0 conn(0x7f1fd013f0a0 0x7f1fd0141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 --2- 172.21.15.190:0/2995858791 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1fd013a190 0x7f1fd013eb60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 -- 172.21.15.190:0/2995858791 >> 172.21.15.190:0/2995858791 conn(0x7f1fd00abee0 msgr2=0x7f1fd00ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 -- 172.21.15.190:0/2995858791 shutdown_connections 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 -- 172.21.15.190:0/2995858791 wait complete. 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient: build_initial_monmap 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient: monmap: 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:51.987 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:51.988 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 Processor -- start 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 -- start start 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient: init 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient: _add_conns ranks=[4,8,6,5,1,0,3,7,2] 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fd01396a0 0x7f1fd013a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient(hunting): picked mon.noname-c con 0x7f1fd01396a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1fd013a190 0x7f1fd013ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient(hunting): picked mon.noname-e con 0x7f1fd013a190 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1fd013f0a0 0x7f1fd013b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient(hunting): picked mon.noname-d con 0x7f1fd013f0a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:51.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f1fd01441f0 con 0x7f1fd01396a0 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1fd00a78f0 con 0x7f1fd013f0a0 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f1fd0144370 con 0x7f1fd013a190 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd7494640 10 monclient(hunting): authenticate will time out at 1337.546229s 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1fd013f0a0 0x7f1fd013b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1fd013f0a0 0x7f1fd013b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:36294/0 (socket says 172.21.15.190:36294) 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 1 -- 172.21.15.190:0/343016576 learned_addr learned my addr 172.21.15.190:0/343016576 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 10 monclient(hunting): get_auth_request con 0x7f1fd013f0a0 auth_method 0 2024-08-19T10:44:51.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5230640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fd01396a0 0x7f1fd013a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd4a2f640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1fd013a190 0x7f1fd013ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5230640 10 monclient(hunting): get_auth_request con 0x7f1fd01396a0 auth_method 0 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5230640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5230640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5230640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd4a2f640 10 monclient(hunting): get_auth_request con 0x7f1fd013a190 auth_method 0 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd4a2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd4a2f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd4a2f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:51.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:51.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:51.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:51.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd4a2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:51.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd4a2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:51.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd4a2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:51.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5230640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:51.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5230640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:51.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5230640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:51.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 10 monclient(hunting): handle_auth_done global_id 44106 payload 824 2024-08-19T10:44:51.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 1 -- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1fd013a190 msgr2=0x7f1fd013ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:51.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1fd013a190 0x7f1fd013ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 1 -- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fd01396a0 msgr2=0x7f1fd013a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:51.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fd01396a0 0x7f1fd013a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:51.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 10 monclient: _finish_hunting 0 2024-08-19T10:44:51.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 1 monclient: found mon.noname-d 2024-08-19T10:44:51.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:51.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:44:51.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.973+0000 7f1fd5a31640 1 -- 172.21.15.190:0/343016576 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd013d1a0 con 0x7f1fd013f0a0 2024-08-19T10:44:51.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd5a31640 1 -- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1fd013f0a0 msgr2=0x7f1fd013b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:51.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd5a31640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1fd013f0a0 0x7f1fd013b3b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8007e90 tx=0x7f1fc800e4b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 1 -- 172.21.15.190:0/343016576 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1fc8014890 con 0x7f1fd013f0a0 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 10 monclient: dump: 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:51.994 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 1 -- 172.21.15.190:0/343016576 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc8019070 con 0x7f1fd013f0a0 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:51.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 1 -- 172.21.15.190:0/343016576 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1fc8013380 con 0x7f1fd013f0a0 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 10 monclient: dump: 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:51.996 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd7494640 5 monclient: authenticate success, global_id 44106 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd7494640 10 monclient: _renew_subs 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd7494640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd7494640 1 -- 172.21.15.190:0/343016576 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd013c940 con 0x7f1fd013f0a0 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc6ffd640 4 set_mon_vals no callback set 2024-08-19T10:44:51.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd7494640 10 monclient: _renew_subs 2024-08-19T10:44:51.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd7494640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:44:51.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd7494640 1 -- 172.21.15.190:0/343016576 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd00a7330 con 0x7f1fd013f0a0 2024-08-19T10:44:51.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd7494640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:51.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd7494640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:51.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd7494640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:44:51.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd7494640 1 -- 172.21.15.190:0/343016576 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd0139a80 con 0x7f1fd013f0a0 2024-08-19T10:44:51.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 1 -- 172.21.15.190:0/343016576 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1fc8014420 con 0x7f1fd013f0a0 2024-08-19T10:44:51.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fb403be00 0x7f1fb403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:51.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fc5ffb640 1 -- 172.21.15.190:0/343016576 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(227..227 src has 1..227) ==== 14771+0+0 (secure 0 0 0) 0x7f1fc8051980 con 0x7f1fd013f0a0 2024-08-19T10:44:51.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd5230640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fb403be00 0x7f1fb403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:51.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.977+0000 7f1fd5230640 10 monclient: get_auth_request con 0x7f1fb403be00 auth_method 0 2024-08-19T10:44:51.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.981+0000 7f1fd5230640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fb403be00 0x7f1fb403e2c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc007240 tx=0x7f1fcc007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:51.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.989+0000 7f1fc5ffb640 1 -- 172.21.15.190:0/343016576 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1fc804dc60 con 0x7f1fd013f0a0 2024-08-19T10:44:51.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.989+0000 7f1fc5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:51.999 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:51.989+0000 7f1fc5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.977+0000 7f4e1ce91640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:44:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.977+0000 7f4e1ce91640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:44:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.977+0000 7f4e1ce91640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.977+0000 7f4e1ce91640 1 -- 172.21.15.89:0/1580822295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4e180733a0 con 0x7f4e181364f0 2024-08-19T10:44:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.977+0000 7f4e06ffd640 1 -- 172.21.15.89:0/1580822295 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7f4e100b6070 con 0x7f4e181364f0 2024-08-19T10:44:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.977+0000 7f4e06ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:44:52.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.977+0000 7f4e06ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:52.001 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":72,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:44:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 -- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4df803be00 msgr2=0x7f4df803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4df803be00 0x7f4df803e2c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00d440 tx=0x7f4e0c00f040 comp rx=0 tx=0).stop 2024-08-19T10:44:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 10 monclient: shutdown 2024-08-19T10:44:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 -- 172.21.15.89:0/1580822295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e181364f0 msgr2=0x7f4e1813adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e181364f0 0x7f4e1813adb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000b040 tx=0x7f4e100092f0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 -- 172.21.15.89:0/1580822295 shutdown_connections 2024-08-19T10:44:52.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4df803be00 0x7f4df803e2c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e181364f0 0x7f4e1813adb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4e180a3f10 0x7f4e1813a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 --2- 172.21.15.89:0/1580822295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e180a3b30 0x7f4e1813a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 -- 172.21.15.89:0/1580822295 >> 172.21.15.89:0/1580822295 conn(0x7f4e180a8f60 msgr2=0x7f4e180a8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:52.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 -- 172.21.15.89:0/1580822295 shutdown_connections 2024-08-19T10:44:52.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:51.981+0000 7f4e1ce91640 1 -- 172.21.15.89:0/1580822295 wait complete. 2024-08-19T10:44:52.004 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:44:52.004 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:44:52.004 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:44:52.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient: get_monmap_and_config 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient: build_initial_monmap 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient: monmap: 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:52.129163+0000 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:52.129163+0000 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:52.128 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:52.129 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:52.129 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 1 Processor -- start 2024-08-19T10:44:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 1 -- start start 2024-08-19T10:44:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient: init 2024-08-19T10:44:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient: _add_conns ranks=[3,8,2,4,1,7,10,6,5,0,9] 2024-08-19T10:44:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dc00a4b30 0x7f5dc00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient(hunting): picked mon.noname-g con 0x7f5dc00a4b30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5dc00720c0 0x7f5dc00a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient(hunting): picked mon.noname-e con 0x7f5dc00720c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:52.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5dc00a5450 0x7f5dc013d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient(hunting): picked mon.noname-b con 0x7f5dc00a5450 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc0073f30 con 0x7f5dc00a5450 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc0073600 con 0x7f5dc00a4b30 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f5dc00738e0 con 0x7f5dc00720c0 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dc58c0640 10 monclient(hunting): authenticate will time out at 1341.946024s 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbf7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5dc00a5450 0x7f5dc013d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5dc00720c0 0x7f5dc00a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5dc00720c0 0x7f5dc00a4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50522/0 (socket says 172.21.15.89:50522) 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbf7fe640 10 monclient(hunting): get_auth_request con 0x7f5dc00a5450 auth_method 0 2024-08-19T10:44:52.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 1 -- 172.21.15.89:0/3402774709 learned_addr learned my addr 172.21.15.89:0/3402774709 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 10 monclient(hunting): get_auth_request con 0x7f5dc00720c0 auth_method 0 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbeffd640 1 --2- 172.21.15.89:0/3402774709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dc00a4b30 0x7f5dc00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbeffd640 10 monclient(hunting): get_auth_request con 0x7f5dc00a4b30 auth_method 0 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 10 monclient(hunting): handle_auth_done global_id 6002 payload 274 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 1 -- 172.21.15.89:0/3402774709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dc00a4b30 msgr2=0x7f5dc00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 1 --2- 172.21.15.89:0/3402774709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dc00a4b30 0x7f5dc00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 1 -- 172.21.15.89:0/3402774709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5dc00a5450 msgr2=0x7f5dc013d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 1 --2- 172.21.15.89:0/3402774709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5dc00a5450 0x7f5dc013d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 1 monclient: found mon.noname-e 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:52.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:52.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.125+0000 7f5dbe7fc640 1 -- 172.21.15.89:0/3402774709 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc00a42a0 con 0x7f5dc00720c0 2024-08-19T10:44:52.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbe7fc640 1 -- 172.21.15.89:0/3402774709 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5dc00720c0 msgr2=0x7f5dc00a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbe7fc640 1 --2- 172.21.15.89:0/3402774709 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5dc00720c0 0x7f5dc00a4f10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009e80 tx=0x7f5db401c910 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f10f37f6780d1a28 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 1 -- 172.21.15.89:0/3402774709 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5db40090d0 con 0x7f5dc00720c0 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 10 monclient: dump: 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.139 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 5 monclient: authenticate success, global_id 6002 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:52.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 1 -- 172.21.15.89:0/3402774709 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db40a5b80 con 0x7f5dc00720c0 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 4 set_mon_vals no callback set 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 1 -- 172.21.15.89:0/3402774709 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5db40a5d30 con 0x7f5dc00720c0 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dbdffb640 10 monclient: dump: 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.141 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 10 monclient: shutdown 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:52.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3402774709 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5dc00720c0 msgr2=0x7f5dc00a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 1 --2- 172.21.15.89:0/3402774709 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5dc00720c0 0x7f5dc00a4f10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009e80 tx=0x7f5db401c910 comp rx=0 tx=0).stop 2024-08-19T10:44:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3402774709 shutdown_connections 2024-08-19T10:44:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 1 --2- 172.21.15.89:0/3402774709 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5dc00720c0 0x7f5dc00a4f10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 1 --2- 172.21.15.89:0/3402774709 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5dc00a5450 0x7f5dc013d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 1 --2- 172.21.15.89:0/3402774709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dc00a4b30 0x7f5dc00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3402774709 >> 172.21.15.89:0/3402774709 conn(0x7f5dc00aa240 msgr2=0x7f5dc00aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3402774709 shutdown_connections 2024-08-19T10:44:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.129+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3402774709 wait complete. 2024-08-19T10:44:52.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient: build_initial_monmap 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient: monmap: 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:52.144 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:52.145 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:52.145 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 1 Processor -- start 2024-08-19T10:44:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 1 -- start start 2024-08-19T10:44:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient: init 2024-08-19T10:44:52.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient: _add_conns ranks=[7,4,5,8,3,0,6,1,2] 2024-08-19T10:44:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5dc00720c0 0x7f5dc009f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient(hunting): picked mon.noname-i con 0x7f5dc00720c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5dc00a4b30 0x7f5dc0137940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient(hunting): picked mon.noname-c con 0x7f5dc00a4b30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:52.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5dc00a5450 0x7f5dc01ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient(hunting): picked mon.noname-h con 0x7f5dc00a5450 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f5dc0140b30 con 0x7f5dc00a4b30 2024-08-19T10:44:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f5dc01409b0 con 0x7f5dc00a5450 2024-08-19T10:44:52.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5dc0140cb0 con 0x7f5dc00720c0 2024-08-19T10:44:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dc58c0640 10 monclient(hunting): authenticate will time out at 1341.953213s 2024-08-19T10:44:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5dc00a4b30 0x7f5dc0137940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5dc00a4b30 0x7f5dc0137940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36558/0 (socket says 172.21.15.89:36558) 2024-08-19T10:44:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 1 -- 172.21.15.89:0/3705809955 learned_addr learned my addr 172.21.15.89:0/3705809955 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:52.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 10 monclient(hunting): get_auth_request con 0x7f5dc00a4b30 auth_method 0 2024-08-19T10:44:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbeffd640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5dc00720c0 0x7f5dc009f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbf7fe640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5dc00a5450 0x7f5dc01ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbeffd640 10 monclient(hunting): get_auth_request con 0x7f5dc00720c0 auth_method 0 2024-08-19T10:44:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbf7fe640 10 monclient(hunting): get_auth_request con 0x7f5dc00a5450 auth_method 0 2024-08-19T10:44:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 10 monclient(hunting): handle_auth_done global_id 44113 payload 824 2024-08-19T10:44:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 1 -- 172.21.15.89:0/3705809955 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5dc00720c0 msgr2=0x7f5dc009f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5dc00720c0 0x7f5dc009f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 1 -- 172.21.15.89:0/3705809955 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5dc00a5450 msgr2=0x7f5dc01ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5dc00a5450 0x7f5dc01ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:44:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 1 monclient: found mon.noname-c 2024-08-19T10:44:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:44:52.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.133+0000 7f5dbe7fc640 1 -- 172.21.15.89:0/3705809955 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc00a01c0 con 0x7f5dc00a4b30 2024-08-19T10:44:52.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbe7fc640 1 -- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5dc00a4b30 msgr2=0x7f5dc0137940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbe7fc640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5dc00a4b30 0x7f5dc0137940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5db40af040 tx=0x7f5db40196b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5db40bc020 con 0x7f5dc00a4b30 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 10 monclient: dump: 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.162 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db4003bb0 con 0x7f5dc00a4b30 2024-08-19T10:44:52.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:52.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5db4003d60 con 0x7f5dc00a4b30 2024-08-19T10:44:52.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:52.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 10 monclient: dump: 2024-08-19T10:44:52.164 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.164 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.164 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.164 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.165 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.166 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbd7fa640 4 set_mon_vals no callback set 2024-08-19T10:44:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dc58c0640 5 monclient: authenticate success, global_id 44113 2024-08-19T10:44:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dc58c0640 10 monclient: _renew_subs 2024-08-19T10:44:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dc58c0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:44:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3705809955 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc01232e0 con 0x7f5dc00a4b30 2024-08-19T10:44:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dc58c0640 10 monclient: _renew_subs 2024-08-19T10:44:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dc58c0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:44:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3705809955 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc0121ae0 con 0x7f5dc00a4b30 2024-08-19T10:44:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dc58c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:52.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dc58c0640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc0137e80 0x7f5dc01cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dc58c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3705809955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc0064590 con 0x7f5dc0137e80 2024-08-19T10:44:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbeffd640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc0137e80 0x7f5dc01cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbeffd640 10 monclient: get_auth_request con 0x7f5dc0137e80 auth_method 0 2024-08-19T10:44:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5db40bf020 con 0x7f5dc00a4b30 2024-08-19T10:44:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbeffd640 10 monclient(hunting): handle_auth_done global_id 6165 payload 824 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbeffd640 1 -- 172.21.15.89:0/3705809955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5dc00a25d0 con 0x7f5dc0137e80 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5dac03bdf0 0x7f5dac03e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(227..227 src has 1..227) ==== 14771+0+0 (secure 0 0 0) 0x7f5db40ebe70 con 0x7f5dc00a4b30 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbf7fe640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5dac03bdf0 0x7f5dac03e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.137+0000 7f5dbf7fe640 10 monclient: get_auth_request con 0x7f5dac03bdf0 auth_method 0 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5dbeffd640 1 -- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc0137e80 msgr2=0x7f5dc01cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5dbeffd640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc0137e80 0x7f5dc01cf400 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5db000f4b0 tx=0x7f5db000f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5db0019020 con 0x7f5dc0137e80 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5da7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5da7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:52.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5da7fff640 10 monclient: dump: 2024-08-19T10:44:52.170 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.170 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.170 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.170 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.170 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.170 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.170 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.171 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.171 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.171 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.171 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.171 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.171 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.171 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.171 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.173 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5db0014880 con 0x7f5dc0137e80 2024-08-19T10:44:52.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5da7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:52.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5da7fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:52.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc0137e80 msgr2=0x7f5dc01cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5da7fff640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc0137e80 0x7f5dc01cf400 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5db000f4b0 tx=0x7f5db000f970 comp rx=0 tx=0).stop 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc0137e80 msgr2=0x7f5dc01cf400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5da7fff640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc0137e80 0x7f5dc01cf400 secure :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5db000f4b0 tx=0x7f5db000f970 comp rx=0 tx=0).stop 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.141+0000 7f5dbf7fe640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5dac03bdf0 0x7f5dac03e2b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00a4b00 tx=0x7f5db8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dc58c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dc58c0640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc01cf940 0x7f5dc013be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dc58c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3705809955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01cfd50 con 0x7f5dc01cf940 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dbeffd640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc01cf940 0x7f5dc013be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dbeffd640 10 monclient: get_auth_request con 0x7f5dc01cf940 auth_method 0 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dbeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dbeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dbeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dbeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dbeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dbeffd640 10 monclient(hunting): handle_auth_done global_id 6174 payload 824 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.145+0000 7f5dbeffd640 1 -- 172.21.15.89:0/3705809955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5dc00a9540 con 0x7f5dc01cf940 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dbeffd640 1 -- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc01cf940 msgr2=0x7f5dc013be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dbeffd640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc01cf940 0x7f5dc013be30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5db000f4b0 tx=0x7f5db00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5db0019040 con 0x7f5dc01cf940 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5da7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5da7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5da7fff640 10 monclient: dump: 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.179 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9269 (secure 0 0 0) 0x7f5db0004030 con 0x7f5dc01cf940 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5da7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5da7fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:52.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc01cf940 msgr2=0x7f5dc013be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5da7fff640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc01cf940 0x7f5dc013be30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5db000f4b0 tx=0x7f5db00046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5da7fff640 1 -- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc01cf940 msgr2=0x7f5dc013be30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:52.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5da7fff640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dc01cf940 0x7f5dc013be30 secure :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5db000f4b0 tx=0x7f5db00046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.181 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:52.181 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:52.181 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:52.182 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:44:52.182 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:52.182 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 421962, 2024-08-19T10:44:52.182 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:52.182 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:52.182 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:52.183 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:52.183 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:52.183 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:52.183 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:52.183 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:52.183 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:52.183 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:52.183 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.183 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.184 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.185 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:52.186 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.187 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.188 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.189 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:52.190 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.191 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.192 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:52.193 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.194 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.195 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.196 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.198 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.199 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.201 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.203 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:44:52.204 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.205 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.205 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:52.205 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:52.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:52.165+0000 7f1fd7494640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:52.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:52.165+0000 7f1fd7494640 10 monclient: _send_command 2 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:52.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:52.165+0000 7f1fd7494640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:44:52.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:52.165+0000 7f1fd7494640 1 -- 172.21.15.190:0/343016576 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f1fd006c9c0 con 0x7f1fd013f0a0 2024-08-19T10:44:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3705809955 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5dac03bdf0 msgr2=0x7f5dac03e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dc58c0640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5dac03bdf0 0x7f5dac03e2b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00a4b00 tx=0x7f5db8008040 comp rx=0 tx=0).stop 2024-08-19T10:44:52.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dc58c0640 10 monclient: shutdown 2024-08-19T10:44:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dc58c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5dc00a4b30 msgr2=0x7f5dc0137940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dc58c0640 1 --2- 172.21.15.89:0/3705809955 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5dc00a4b30 0x7f5dc0137940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5db40af040 tx=0x7f5db40196b0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dbf7fe640 1 -- 172.21.15.89:0/3705809955 reap_dead start 2024-08-19T10:44:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dbf7fe640 1 -- 172.21.15.89:0/3705809955 reap_dead start 2024-08-19T10:44:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3705809955 shutdown_connections 2024-08-19T10:44:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3705809955 >> 172.21.15.89:0/3705809955 conn(0x7f5dc00aa240 msgr2=0x7f5dc00a7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3705809955 shutdown_connections 2024-08-19T10:44:52.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.149+0000 7f5dc58c0640 1 -- 172.21.15.89:0/3705809955 wait complete. 2024-08-19T10:44:52.209 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:52.209 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:44:52.209 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:44:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient: get_monmap_and_config 2024-08-19T10:44:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient: build_initial_monmap 2024-08-19T10:44:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient: monmap: 2024-08-19T10:44:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:52.329400+0000 2024-08-19T10:44:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:52.329400+0000 2024-08-19T10:44:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.328 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:52.329 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:52.329 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 1 Processor -- start 2024-08-19T10:44:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 1 -- start start 2024-08-19T10:44:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient: init 2024-08-19T10:44:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient: _add_conns ranks=[9,0,4,1,5,3,10,2,6,7,8] 2024-08-19T10:44:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ac00a3c70 0x7f7ac00a4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient(hunting): picked mon.noname-a con 0x7f7ac00a3c70 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:52.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7ac00a4590 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:52.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ac013a070 0x7f7ac0141c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient(hunting): picked mon.noname-c con 0x7f7ac013a070 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac00a5640 con 0x7f7ac00a4590 2024-08-19T10:44:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f7ac006cfb0 con 0x7f7ac00a3c70 2024-08-19T10:44:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f7ac00a0b40 con 0x7f7ac013a070 2024-08-19T10:44:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac845c640 10 monclient(hunting): authenticate will time out at 1342.146429s 2024-08-19T10:44:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ac013a070 0x7f7ac0141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac61f8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7ac00a3c70 0x7f7ac00a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac59f7640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7ac00a4590 0x7f7ac0139960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58306/0 (socket says 172.21.15.89:58306) 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac59f7640 1 -- 172.21.15.89:0/4084512974 learned_addr learned my addr 172.21.15.89:0/4084512974 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 10 monclient(hunting): get_auth_request con 0x7f7ac013a070 auth_method 0 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac61f8640 10 monclient(hunting): get_auth_request con 0x7f7ac00a3c70 auth_method 0 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac61f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac61f8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac61f8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac61f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac61f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac61f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 10 monclient(hunting): handle_auth_done global_id 44122 payload 274 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 1 -- 172.21.15.89:0/4084512974 >> v2:172.21.15.89:3300/0 conn(0x7f7ac00a3c70 msgr2=0x7f7ac00a4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 1 --2- 172.21.15.89:0/4084512974 >> v2:172.21.15.89:3300/0 conn(0x7f7ac00a3c70 0x7f7ac00a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 1 -- 172.21.15.89:0/4084512974 >> v1:172.21.15.89:6789/0 conn(0x7f7ac00a4590 legacy=0x7f7ac0139960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 10 monclient: _finish_hunting 0 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 1 monclient: found mon.noname-c 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac69f9640 1 -- 172.21.15.89:0/4084512974 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac00a0e20 con 0x7f7ac013a070 2024-08-19T10:44:52.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.325+0000 7f7ac61f8640 1 --2- 172.21.15.89:0/4084512974 >> v2:172.21.15.89:3300/0 conn(0x7f7ac00a3c70 0x7f7ac00a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac69f9640 1 -- 172.21.15.89:0/4084512974 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ac013a070 msgr2=0x7f7ac0141c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac69f9640 1 --2- 172.21.15.89:0/4084512974 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ac013a070 0x7f7ac0141c30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009d90 tx=0x7f7ab001c830 comp rx=0 tx=0).ready entity=mon.4 client_cookie=b39e87e00b5dac2f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 1 -- 172.21.15.89:0/4084512974 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7ab00b2070 con 0x7f7ac013a070 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 10 monclient: dump: 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.343 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 10 monclient: _finish_auth 0 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 10 monclient: _check_auth_tickets 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 1 -- 172.21.15.89:0/4084512974 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab00a59f0 con 0x7f7ac013a070 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac845c640 5 monclient: authenticate success, global_id 44122 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac845c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:52.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac845c640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 1 -- 172.21.15.89:0/4084512974 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7ab00a5be0 con 0x7f7ac013a070 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac845c640 4 set_mon_vals no callback set 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac51f6640 10 monclient: dump: 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.345 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac845c640 10 monclient: shutdown 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac845c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac845c640 1 -- 172.21.15.89:0/4084512974 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ac013a070 msgr2=0x7f7ac0141c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.329+0000 7f7ac845c640 1 --2- 172.21.15.89:0/4084512974 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ac013a070 0x7f7ac0141c30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009d90 tx=0x7f7ab001c830 comp rx=0 tx=0).stop 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 -- 172.21.15.89:0/4084512974 shutdown_connections 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 --2- 172.21.15.89:0/4084512974 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7ac013a070 0x7f7ac0141c30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 --2- 172.21.15.89:0/4084512974 >> v2:172.21.15.89:3300/0 conn(0x7f7ac00a3c70 0x7f7ac00a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 -- 172.21.15.89:0/4084512974 >> 172.21.15.89:0/4084512974 conn(0x7f7ac006de40 msgr2=0x7f7ac006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 -- 172.21.15.89:0/4084512974 shutdown_connections 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 -- 172.21.15.89:0/4084512974 wait complete. 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient: build_initial_monmap 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient: monmap: 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:52.347 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 Processor -- start 2024-08-19T10:44:52.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 -- start start 2024-08-19T10:44:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient: init 2024-08-19T10:44:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient: _add_conns ranks=[1,7,2,8,6,5,4,0,3] 2024-08-19T10:44:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ac00a3c70 0x7f7ac01404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient(hunting): picked mon.noname-f con 0x7f7ac00a3c70 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ac00a4590 0x7f7ac01d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient(hunting): picked mon.noname-i con 0x7f7ac00a4590 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013a070 0x7f7ac01d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient(hunting): picked mon.noname-b con 0x7f7ac013a070 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac006d3b0 con 0x7f7ac00a3c70 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac006d230 con 0x7f7ac013a070 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7ac0144990 con 0x7f7ac00a4590 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac845c640 10 monclient(hunting): authenticate will time out at 1342.153584s 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013a070 0x7f7ac01d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013a070 0x7f7ac01d22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56354/0 (socket says 172.21.15.89:56354) 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 1 -- 172.21.15.89:0/944519404 learned_addr learned my addr 172.21.15.89:0/944519404 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 10 monclient(hunting): get_auth_request con 0x7f7ac013a070 auth_method 0 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 10 monclient(hunting): handle_auth_done global_id 6131 payload 824 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 1 -- 172.21.15.89:0/944519404 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ac00a4590 msgr2=0x7f7ac01d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ac00a4590 0x7f7ac01d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 1 -- 172.21.15.89:0/944519404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ac00a3c70 msgr2=0x7f7ac01404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ac00a3c70 0x7f7ac01404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 10 monclient: _finish_hunting 0 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 1 monclient: found mon.noname-b 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:52.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.333+0000 7f7ac69f9640 1 -- 172.21.15.89:0/944519404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac01416f0 con 0x7f7ac013a070 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac69f9640 1 -- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013a070 msgr2=0x7f7ac01d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac69f9640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013a070 0x7f7ac01d22a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7ab001d0d0 tx=0x7f7ab00195d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7ab00b2050 con 0x7f7ac013a070 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 10 monclient: dump: 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.352 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:52.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab00be070 con 0x7f7ac013a070 2024-08-19T10:44:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac845c640 5 monclient: authenticate success, global_id 6131 2024-08-19T10:44:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac845c640 10 monclient: _renew_subs 2024-08-19T10:44:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac845c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac845c640 1 -- 172.21.15.89:0/944519404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac01419d0 con 0x7f7ac013a070 2024-08-19T10:44:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7ab0003dc0 con 0x7f7ac013a070 2024-08-19T10:44:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 10 monclient: dump: 2024-08-19T10:44:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.354 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.356 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac845c640 10 monclient: _renew_subs 2024-08-19T10:44:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac49f5640 4 set_mon_vals no callback set 2024-08-19T10:44:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac845c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac845c640 1 -- 172.21.15.89:0/944519404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac00a5f80 con 0x7f7ac013a070 2024-08-19T10:44:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac845c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac845c640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013dee0 0x7f7ac013e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac845c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac845c640 1 -- 172.21.15.89:0/944519404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac0069f50 con 0x7f7ac013dee0 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013dee0 0x7f7ac013e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 10 monclient: get_auth_request con 0x7f7ac013dee0 auth_method 0 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7ab0004040 con 0x7f7ac013a070 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 10 monclient(hunting): handle_auth_done global_id 6140 payload 824 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 1 -- 172.21.15.89:0/944519404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ac009f0b0 con 0x7f7ac013dee0 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7aac03bd30 0x7f7aac03e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(227..227 src has 1..227) ==== 14771+0+0 (secure 0 0 0) 0x7f7ab00ebf90 con 0x7f7ac013a070 2024-08-19T10:44:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac59f7640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7aac03bd30 0x7f7aac03e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac59f7640 10 monclient: get_auth_request con 0x7f7aac03bd30 auth_method 0 2024-08-19T10:44:52.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 1 -- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013dee0 msgr2=0x7f7ac013e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.337+0000 7f7ac61f8640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013dee0 0x7f7ac013e2c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800e7a0 tx=0x7f7ab800ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7ab8017070 con 0x7f7ac013dee0 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7abeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7abeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7abeffd640 10 monclient: dump: 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7ab800cae0 con 0x7f7ac013dee0 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7abeffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7abeffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013dee0 msgr2=0x7f7ac013e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7abeffd640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013dee0 0x7f7ac013e2c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800e7a0 tx=0x7f7ab800ec60 comp rx=0 tx=0).stop 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013dee0 msgr2=0x7f7ac013e2c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7abeffd640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013dee0 0x7f7ac013e2c0 secure :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800e7a0 tx=0x7f7ab800ec60 comp rx=0 tx=0).stop 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.341+0000 7f7ac59f7640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7aac03bd30 0x7f7aac03e1f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ac013ffc0 tx=0x7f7ab4007ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac845c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac845c640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013e800 0x7f7ac009d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac845c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac845c640 1 -- 172.21.15.89:0/944519404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac0139e80 con 0x7f7ac013e800 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac61f8640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013e800 0x7f7ac009d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac61f8640 10 monclient: get_auth_request con 0x7f7ac013e800 auth_method 0 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac61f8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac61f8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac61f8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac61f8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac61f8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac61f8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac61f8640 10 monclient(hunting): handle_auth_done global_id 6149 payload 824 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.345+0000 7f7ac61f8640 1 -- 172.21.15.89:0/944519404 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7ac00a6650 con 0x7f7ac013e800 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac61f8640 1 -- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013e800 msgr2=0x7f7ac009d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac61f8640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013e800 0x7f7ac009d8d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8019000 tx=0x7f7ab80046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7ab8017070 con 0x7f7ac013e800 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7abeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7abeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7abeffd640 10 monclient: dump: 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f7ab8006650 con 0x7f7ac013e800 2024-08-19T10:44:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7abeffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7abeffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013e800 msgr2=0x7f7ac009d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7abeffd640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013e800 0x7f7ac009d8d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8019000 tx=0x7f7ab80046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7abeffd640 1 -- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013e800 msgr2=0x7f7ac009d8d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7abeffd640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013e800 0x7f7ac009d8d0 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8019000 tx=0x7f7ab80046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.367 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:52.367 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:52.367 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:52.367 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:52.367 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:52.367 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 422178, 2024-08-19T10:44:52.367 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:52.367 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:52.368 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:52.368 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:52.368 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:52.368 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:52.368 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:52.368 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:52.368 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:52.368 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:52.368 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.369 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.370 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.370 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.370 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.370 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:52.370 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:52.370 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.370 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.370 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.370 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.370 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.370 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:52.371 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.372 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:52.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:52.374 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.376 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.377 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:52.378 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:52.378 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:52.379 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.382 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.384 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.385 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.386 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:52.388 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.389 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:44:52.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.392 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:52.392 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:52.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac845c640 1 -- 172.21.15.89:0/944519404 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7aac03bd30 msgr2=0x7f7aac03e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac845c640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7aac03bd30 0x7f7aac03e1f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ac013ffc0 tx=0x7f7ab4007ab0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac845c640 10 monclient: shutdown 2024-08-19T10:44:52.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac845c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:52.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac845c640 1 -- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013a070 msgr2=0x7f7ac01d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac845c640 1 --2- 172.21.15.89:0/944519404 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7ac013a070 0x7f7ac01d22a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7ab001d0d0 tx=0x7f7ab00195d0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac69f9640 1 -- 172.21.15.89:0/944519404 reap_dead start 2024-08-19T10:44:52.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac69f9640 1 -- 172.21.15.89:0/944519404 reap_dead start 2024-08-19T10:44:52.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac845c640 1 -- 172.21.15.89:0/944519404 shutdown_connections 2024-08-19T10:44:52.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac845c640 1 -- 172.21.15.89:0/944519404 >> 172.21.15.89:0/944519404 conn(0x7f7ac006de40 msgr2=0x7f7ac009ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:52.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac845c640 1 -- 172.21.15.89:0/944519404 shutdown_connections 2024-08-19T10:44:52.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.349+0000 7f7ac845c640 1 -- 172.21.15.89:0/944519404 wait complete. 2024-08-19T10:44:52.394 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:52.394 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:44:52.394 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:44:52.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient: get_monmap_and_config 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient: build_initial_monmap 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient: monmap: 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:52.510270+0000 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:52.510270+0000 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:52.509 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:52.510 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:52.510 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:52.510 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:52.510 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:52.510 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:52.510 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:52.510 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:52.510 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 1 Processor -- start 2024-08-19T10:44:52.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 1 -- start start 2024-08-19T10:44:52.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient: init 2024-08-19T10:44:52.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:52.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient: _add_conns ranks=[7,9,0,8,5,3,2,10,6,1,4] 2024-08-19T10:44:52.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff3340a67d0 0x7ff3340a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient(hunting): picked mon.noname-i con 0x7ff3340a67d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:52.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3340a6bb0 0x7ff3340a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient(hunting): picked mon.noname-a con 0x7ff3340a6bb0 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff334139590 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3340a8250 con 0x7ff334139590 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff3340a8740 con 0x7ff3340a6bb0 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff3340a3380 con 0x7ff3340a67d0 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff339f89640 10 monclient(hunting): authenticate will time out at 1342.327008s 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3340a6bb0 0x7ff3340a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff3340a6bb0 0x7ff3340a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51928/0 (socket says 172.21.15.89:51928) 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 1 -- 172.21.15.89:0/1033620159 learned_addr learned my addr 172.21.15.89:0/1033620159 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:52.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 10 monclient(hunting): get_auth_request con 0x7ff3340a6bb0 auth_method 0 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 10 monclient(hunting): handle_auth_done global_id 6192 payload 274 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 1 -- 172.21.15.89:0/1033620159 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff3340a67d0 msgr2=0x7ff3340a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff3337fe640 1 --2- 172.21.15.89:0/1033620159 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff3340a67d0 0x7ff3340a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 1 --2- 172.21.15.89:0/1033620159 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff3340a67d0 0x7ff3340a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 1 -- 172.21.15.89:0/1033620159 >> v1:172.21.15.89:6789/0 conn(0x7ff334139590 legacy=0x7ff334139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:52.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:52.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 1 monclient: found mon.noname-a 2024-08-19T10:44:52.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:52.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:52.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.505+0000 7ff332ffd640 1 -- 172.21.15.89:0/1033620159 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3340a4cc0 con 0x7ff3340a6bb0 2024-08-19T10:44:52.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff332ffd640 1 -- 172.21.15.89:0/1033620159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3340a6bb0 msgr2=0x7ff3340a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff332ffd640 1 --2- 172.21.15.89:0/1033620159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3340a6bb0 0x7ff3340a6f90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff328009870 tx=0x7ff32801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be3d73d870204b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff3327fc640 1 -- 172.21.15.89:0/1033620159 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff3280b0070 con 0x7ff3340a6bb0 2024-08-19T10:44:52.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff3327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff3327fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff3327fc640 10 monclient: dump: 2024-08-19T10:44:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.518 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff3327fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff3327fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff3327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:52.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff3327fc640 1 -- 172.21.15.89:0/1033620159 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3280aa810 con 0x7ff3340a6bb0 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff3327fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff3327fc640 1 -- 172.21.15.89:0/1033620159 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff3280a93e0 con 0x7ff3340a6bb0 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.509+0000 7ff3327fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3327fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3327fc640 10 monclient: dump: 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.520 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 5 monclient: authenticate success, global_id 6192 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 4 set_mon_vals no callback set 2024-08-19T10:44:52.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient: shutdown 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 -- 172.21.15.89:0/1033620159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3340a6bb0 msgr2=0x7ff3340a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 --2- 172.21.15.89:0/1033620159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3340a6bb0 0x7ff3340a6f90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff328009870 tx=0x7ff32801c4b0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 -- 172.21.15.89:0/1033620159 shutdown_connections 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 --2- 172.21.15.89:0/1033620159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3340a6bb0 0x7ff3340a6f90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 --2- 172.21.15.89:0/1033620159 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff3340a67d0 0x7ff3340a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 -- 172.21.15.89:0/1033620159 >> 172.21.15.89:0/1033620159 conn(0x7ff334069ae0 msgr2=0x7ff3340abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 -- 172.21.15.89:0/1033620159 shutdown_connections 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 -- 172.21.15.89:0/1033620159 wait complete. 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient: build_initial_monmap 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient: monmap: 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:52.522 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:52.523 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:52.524 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 Processor -- start 2024-08-19T10:44:52.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 -- start start 2024-08-19T10:44:52.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient: init 2024-08-19T10:44:52.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:52.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient: _add_conns ranks=[6,3,2,4,5,0,7,1,8] 2024-08-19T10:44:52.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff3340a67d0 0x7ff334138260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient(hunting): picked mon.noname-d con 0x7ff3340a67d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:52.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3340a6bb0 0x7ff33413d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient(hunting): picked mon.noname-g con 0x7ff3340a6bb0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:52.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff334139590 0x7ff33413da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient(hunting): picked mon.noname-b con 0x7ff334139590 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:52.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff3341445b0 con 0x7ff334139590 2024-08-19T10:44:52.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff334144430 con 0x7ff3340a6bb0 2024-08-19T10:44:52.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff334144730 con 0x7ff3340a67d0 2024-08-19T10:44:52.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:52.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff339f89640 10 monclient(hunting): authenticate will time out at 1342.334404s 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3337fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff3340a67d0 0x7ff334138260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3337fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff3340a67d0 0x7ff334138260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:33060/0 (socket says 172.21.15.89:33060) 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff334139590 0x7ff33413da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3337fe640 1 -- 172.21.15.89:0/3725550292 learned_addr learned my addr 172.21.15.89:0/3725550292 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3337fe640 10 monclient(hunting): get_auth_request con 0x7ff3340a67d0 auth_method 0 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3337fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3337fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 10 monclient(hunting): get_auth_request con 0x7ff334139590 auth_method 0 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff332ffd640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3340a6bb0 0x7ff33413d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff332ffd640 10 monclient(hunting): get_auth_request con 0x7ff3340a6bb0 auth_method 0 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff332ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff332ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff332ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 10 monclient(hunting): handle_auth_done global_id 6158 payload 824 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff3340a67d0 msgr2=0x7ff334138260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff3340a67d0 0x7ff334138260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3340a6bb0 msgr2=0x7ff33413d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3340a6bb0 0x7ff33413d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 1 monclient: found mon.noname-b 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff333fff640 1 -- 172.21.15.89:0/3725550292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3340ab310 con 0x7ff334139590 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.513+0000 7ff3337fe640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff3340a67d0 0x7ff334138260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff333fff640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff334139590 msgr2=0x7ff33413da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff333fff640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff334139590 0x7ff33413da20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff32400b3e0 tx=0x7ff32400b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff324015020 con 0x7ff334139590 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:44:52.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 10 monclient: dump: 2024-08-19T10:44:52.534 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.534 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.534 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.534 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.534 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.534 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.534 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.534 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.534 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.535 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.535 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.535 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.535 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.535 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.535 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.535 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 10 monclient: _finish_auth 0 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff324011840 con 0x7ff334139590 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff339f89640 5 monclient: authenticate success, global_id 6158 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff339f89640 10 monclient: _renew_subs 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff339f89640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff339f89640 1 -- 172.21.15.89:0/3725550292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff334122a80 con 0x7ff334139590 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff324010380 con 0x7ff334139590 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:52.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 10 monclient: dump: 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.537 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff339f89640 10 monclient: _renew_subs 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff331ffb640 4 set_mon_vals no callback set 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff339f89640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff339f89640 1 -- 172.21.15.89:0/3725550292 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33413fb30 con 0x7ff334139590 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff339f89640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff339f89640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 0x7ff33402d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff339f89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff339f89640 1 -- 172.21.15.89:0/3725550292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff3340a5500 con 0x7ff3341387a0 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff3337fe640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 0x7ff33402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff3337fe640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 msgr2=0x7ff33402d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:44:52.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff3337fe640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 msgr2=0x7ff33402d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:44:52.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff3337fe640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 0x7ff33402d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:52.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff3337fe640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 0x7ff33402d1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:52.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff324018020 con 0x7ff334139590 2024-08-19T10:44:52.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff32003be00 0x7ff32003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(227..227 src has 1..227) ==== 14771+0+0 (secure 0 0 0) 0x7ff3240508f0 con 0x7ff334139590 2024-08-19T10:44:52.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff332ffd640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff32003be00 0x7ff32003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.517+0000 7ff332ffd640 10 monclient: get_auth_request con 0x7ff32003be00 auth_method 0 2024-08-19T10:44:52.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.521+0000 7ff332ffd640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff32003be00 0x7ff32003e2c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff3340a9c10 tx=0x7ff3280a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.721+0000 7ff3337fe640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 0x7ff33402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.721+0000 7ff3337fe640 10 monclient: get_auth_request con 0x7ff3341387a0 auth_method 0 2024-08-19T10:44:52.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.721+0000 7ff3337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.721+0000 7ff3337fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.721+0000 7ff3337fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.721+0000 7ff3337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.721+0000 7ff3337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.721+0000 7ff3337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.721+0000 7ff3337fe640 10 monclient(hunting): handle_auth_done global_id 44131 payload 824 2024-08-19T10:44:52.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.721+0000 7ff3337fe640 1 -- 172.21.15.89:0/3725550292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff334064590 con 0x7ff3341387a0 2024-08-19T10:44:52.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.725+0000 7ff3337fe640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 msgr2=0x7ff33402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.725+0000 7ff3337fe640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 0x7ff33402d1d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0109e0 tx=0x7ff32c010ea0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.729+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff32c00cdb0 con 0x7ff3341387a0 2024-08-19T10:44:52.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.729+0000 7ff31bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.729+0000 7ff31bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.729+0000 7ff31bfff640 10 monclient: dump: 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.733 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.734 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.734 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.734 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.734 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.734 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.729+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff32c00cdb0 con 0x7ff3341387a0 2024-08-19T10:44:52.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.729+0000 7ff31bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:52.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.729+0000 7ff31bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:52.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.729+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 msgr2=0x7ff33402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.729+0000 7ff31bfff640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 0x7ff33402d1d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0109e0 tx=0x7ff32c010ea0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.729+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 msgr2=0x7ff33402d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:52.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.729+0000 7ff31bfff640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3341387a0 0x7ff33402d1d0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0109e0 tx=0x7ff32c010ea0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff339f89640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:52.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff339f89640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff33402d710 0x7ff33402db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff339f89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff339f89640 1 -- 172.21.15.89:0/3725550292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff3340a01e0 con 0x7ff33402d710 2024-08-19T10:44:52.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff3337fe640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff33402d710 0x7ff33402db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff3337fe640 10 monclient: get_auth_request con 0x7ff33402d710 auth_method 0 2024-08-19T10:44:52.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff3337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff3337fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff3337fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff3337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff3337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff3337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff3337fe640 10 monclient(hunting): handle_auth_done global_id 44140 payload 824 2024-08-19T10:44:52.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.737+0000 7ff3337fe640 1 -- 172.21.15.89:0/3725550292 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff33413ff70 con 0x7ff33402d710 2024-08-19T10:44:52.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff3337fe640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff33402d710 msgr2=0x7ff33402db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff3337fe640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff33402d710 0x7ff33402db20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0109e0 tx=0x7ff32c004ac0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff32c01e070 con 0x7ff33402d710 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff31bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff31bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff31bfff640 10 monclient: dump: 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.745 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7ff32c003e90 con 0x7ff33402d710 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff31bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff31bfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff33402d710 msgr2=0x7ff33402db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff31bfff640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff33402d710 0x7ff33402db20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0109e0 tx=0x7ff32c004ac0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff31bfff640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff33402d710 msgr2=0x7ff33402db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:52.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff31bfff640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff33402d710 0x7ff33402db20 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0109e0 tx=0x7ff32c004ac0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.749 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:52.749 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:52.749 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:52.749 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:52.749 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:52.749 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 15867, 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:44:52.750 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.751 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.752 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:52.753 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.754 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.755 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.756 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.758 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:52.759 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.760 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.761 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.762 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.763 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:52.765 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:52.766 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.767 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.768 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:52.771 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.773 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:52.773 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff339f89640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff32003be00 msgr2=0x7ff32003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff339f89640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff32003be00 0x7ff32003e2c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff3340a9c10 tx=0x7ff3280a6000 comp rx=0 tx=0).stop 2024-08-19T10:44:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff339f89640 10 monclient: shutdown 2024-08-19T10:44:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff339f89640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:52.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff339f89640 1 -- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff334139590 msgr2=0x7ff33413da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff339f89640 1 --2- 172.21.15.89:0/3725550292 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff334139590 0x7ff33413da20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff32400b3e0 tx=0x7ff32400b8a0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff333fff640 1 -- 172.21.15.89:0/3725550292 reap_dead start 2024-08-19T10:44:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff333fff640 1 -- 172.21.15.89:0/3725550292 reap_dead start 2024-08-19T10:44:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff339f89640 1 -- 172.21.15.89:0/3725550292 shutdown_connections 2024-08-19T10:44:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff339f89640 1 -- 172.21.15.89:0/3725550292 >> 172.21.15.89:0/3725550292 conn(0x7ff334069ae0 msgr2=0x7ff334140c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff339f89640 1 -- 172.21.15.89:0/3725550292 shutdown_connections 2024-08-19T10:44:52.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.741+0000 7ff339f89640 1 -- 172.21.15.89:0/3725550292 wait complete. 2024-08-19T10:44:52.775 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:52.775 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:44:52.776 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:44:52.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient: get_monmap_and_config 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient: build_initial_monmap 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient: monmap: 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:52.898098+0000 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:52.898098+0000 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:52.897 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:52.898 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:52.898 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:52.898 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:52.898 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:52.898 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:52.898 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:52.898 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:52.898 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 1 Processor -- start 2024-08-19T10:44:52.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 1 -- start start 2024-08-19T10:44:52.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient: init 2024-08-19T10:44:52.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:52.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient: _add_conns ranks=[10,6,8,4,3,9,7,2,5,1,0] 2024-08-19T10:44:52.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2bc0a5270 0x7fd2bc0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient(hunting): picked mon.noname-f con 0x7fd2bc0a5270 addr v2:172.21.15.190:3300/0 2024-08-19T10:44:52.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2bc0a6d30 0x7fd2bc0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient(hunting): picked mon.noname-d con 0x7fd2bc0a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2bc1395b0 0x7fd2bc139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient(hunting): picked mon.noname-e con 0x7fd2bc1395b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd2bc0a7650 con 0x7fd2bc0a5270 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd2bc0a8710 con 0x7fd2bc0a6d30 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd2bc0a5a40 con 0x7fd2bc1395b0 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2c1875640 10 monclient(hunting): authenticate will time out at 1342.714660s 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2bc1395b0 0x7fd2bc139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2ba7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2bc0a6d30 0x7fd2bc0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2bc1395b0 0x7fd2bc139a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50530/0 (socket says 172.21.15.89:50530) 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2ba7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2bc0a6d30 0x7fd2bc0a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:33074/0 (socket says 172.21.15.89:33074) 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 1 -- 172.21.15.89:0/1862420936 learned_addr learned my addr 172.21.15.89:0/1862420936 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 10 monclient(hunting): get_auth_request con 0x7fd2bc1395b0 auth_method 0 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2ba7fc640 10 monclient(hunting): get_auth_request con 0x7fd2bc0a6d30 auth_method 0 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2ba7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2baffd640 1 --2- 172.21.15.89:0/1862420936 >> v2:172.21.15.190:3300/0 conn(0x7fd2bc0a5270 0x7fd2bc0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2baffd640 10 monclient(hunting): get_auth_request con 0x7fd2bc0a5270 auth_method 0 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2baffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2baffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 10 monclient(hunting): handle_auth_done global_id 6011 payload 274 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 1 -- 172.21.15.89:0/1862420936 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2bc0a6d30 msgr2=0x7fd2bc0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 1 --2- 172.21.15.89:0/1862420936 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2bc0a6d30 0x7fd2bc0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 1 -- 172.21.15.89:0/1862420936 >> v2:172.21.15.190:3300/0 conn(0x7fd2bc0a5270 msgr2=0x7fd2bc0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:52.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 1 --2- 172.21.15.89:0/1862420936 >> v2:172.21.15.190:3300/0 conn(0x7fd2bc0a5270 0x7fd2bc0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:52.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 1 monclient: found mon.noname-e 2024-08-19T10:44:52.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:52.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:52.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2bb7fe640 1 -- 172.21.15.89:0/1862420936 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc0a81d0 con 0x7fd2bc1395b0 2024-08-19T10:44:52.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.893+0000 7fd2ba7fc640 1 --2- 172.21.15.89:0/1862420936 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2bc0a6d30 0x7fd2bc0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:52.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2bb7fe640 1 -- 172.21.15.89:0/1862420936 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2bc1395b0 msgr2=0x7fd2bc139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2bb7fe640 1 --2- 172.21.15.89:0/1862420936 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2bc1395b0 0x7fd2bc139a40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009870 tx=0x7fd2ac01c420 comp rx=0 tx=0).ready entity=mon.8 client_cookie=967ba6ecc62bed69 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 1 -- 172.21.15.89:0/1862420936 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2ac0b1070 con 0x7fd2bc1395b0 2024-08-19T10:44:52.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 10 monclient: dump: 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.911 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 1 -- 172.21.15.89:0/1862420936 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac0ab810 con 0x7fd2bc1395b0 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 1 -- 172.21.15.89:0/1862420936 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2ac0aa300 con 0x7fd2bc1395b0 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:52.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2b9ffb640 10 monclient: dump: 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.913 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 5 monclient: authenticate success, global_id 6011 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 4 set_mon_vals no callback set 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient: shutdown 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 -- 172.21.15.89:0/1862420936 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2bc1395b0 msgr2=0x7fd2bc139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 --2- 172.21.15.89:0/1862420936 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2bc1395b0 0x7fd2bc139a40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009870 tx=0x7fd2ac01c420 comp rx=0 tx=0).stop 2024-08-19T10:44:52.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 -- 172.21.15.89:0/1862420936 shutdown_connections 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 --2- 172.21.15.89:0/1862420936 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2bc1395b0 0x7fd2bc139a40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 --2- 172.21.15.89:0/1862420936 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2bc0a6d30 0x7fd2bc0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 --2- 172.21.15.89:0/1862420936 >> v2:172.21.15.190:3300/0 conn(0x7fd2bc0a5270 0x7fd2bc0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 -- 172.21.15.89:0/1862420936 >> 172.21.15.89:0/1862420936 conn(0x7fd2bc069a80 msgr2=0x7fd2bc0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 -- 172.21.15.89:0/1862420936 shutdown_connections 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 -- 172.21.15.89:0/1862420936 wait complete. 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient: build_initial_monmap 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient: monmap: 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:52.915 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 Processor -- start 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 -- start start 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient: init 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient: _add_conns ranks=[2,7,4,3,1,0,5,6,8] 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2bc0a5270 0x7fd2bc1d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient(hunting): picked mon.noname-b con 0x7fd2bc0a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2bc0a6d30 0x7fd2bc0a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient(hunting): picked mon.noname-i con 0x7fd2bc0a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2bc1395b0 0x7fd2bc0a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient(hunting): picked mon.noname-c con 0x7fd2bc1395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd2bc0a5650 con 0x7fd2bc0a5270 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd2bc0a4a70 con 0x7fd2bc1395b0 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd2bc0a57d0 con 0x7fd2bc0a6d30 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2c1875640 10 monclient(hunting): authenticate will time out at 1342.722477s 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2bc0a5270 0x7fd2bc1d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2bc0a5270 0x7fd2bc1d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56394/0 (socket says 172.21.15.89:56394) 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 1 -- 172.21.15.89:0/519553081 learned_addr learned my addr 172.21.15.89:0/519553081 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 10 monclient(hunting): get_auth_request con 0x7fd2bc0a5270 auth_method 0 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2bb7fe640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2bc1395b0 0x7fd2bc0a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2bb7fe640 10 monclient(hunting): get_auth_request con 0x7fd2bc1395b0 auth_method 0 2024-08-19T10:44:52.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2bb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 10 monclient(hunting): handle_auth_done global_id 6167 payload 824 2024-08-19T10:44:52.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2ba7fc640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2bc0a6d30 0x7fd2bc0a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 1 -- 172.21.15.89:0/519553081 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2bc0a6d30 msgr2=0x7fd2bc0a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2bc0a6d30 0x7fd2bc0a1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 1 -- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2bc1395b0 msgr2=0x7fd2bc0a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2bc1395b0 0x7fd2bc0a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 1 monclient: found mon.noname-b 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2baffd640 1 -- 172.21.15.89:0/519553081 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc09e510 con 0x7fd2bc0a5270 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.901+0000 7fd2bb7fe640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd2bc1395b0 0x7fd2bc0a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2baffd640 1 -- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2bc0a5270 msgr2=0x7fd2bc1d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2baffd640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2bc0a5270 0x7fd2bc1d1140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4007e40 tx=0x7fd2b400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2b4013a00 con 0x7fd2bc0a5270 2024-08-19T10:44:52.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 10 monclient: dump: 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.922 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2c1875640 5 monclient: authenticate success, global_id 6167 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2c1875640 10 monclient: _renew_subs 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2c1875640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2c1875640 1 -- 172.21.15.89:0/519553081 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2bc0a27c0 con 0x7fd2bc0a5270 2024-08-19T10:44:52.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2c1875640 10 monclient: _renew_subs 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2c1875640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2c1875640 1 -- 172.21.15.89:0/519553081 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2bc0aad90 con 0x7fd2bc0a5270 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b4019070 con 0x7fd2bc0a5270 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2b4013ba0 con 0x7fd2bc0a5270 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b97fa640 4 set_mon_vals no callback set 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 10 monclient: dump: 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.924 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2c1875640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2c1875640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0002890 0x7fd2a0004d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2c1875640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2c1875640 1 -- 172.21.15.89:0/519553081 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd2a0007cc0 con 0x7fd2a0002890 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2ba7fc640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0002890 0x7fd2a0004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2ba7fc640 10 monclient: get_auth_request con 0x7fd2a0002890 auth_method 0 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2ba7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd2b4013590 con 0x7fd2bc0a5270 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2ba7fc640 10 monclient(hunting): handle_auth_done global_id 44133 payload 824 2024-08-19T10:44:52.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2ba7fc640 1 -- 172.21.15.89:0/519553081 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2a0007ec0 con 0x7fd2a0002890 2024-08-19T10:44:52.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd29c03be00 0x7fd29c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(227..227 src has 1..227) ==== 14771+0+0 (secure 0 0 0) 0x7fd2b40518a0 con 0x7fd2bc0a5270 2024-08-19T10:44:52.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2bb7fe640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd29c03be00 0x7fd29c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.905+0000 7fd2bb7fe640 10 monclient: get_auth_request con 0x7fd29c03be00 auth_method 0 2024-08-19T10:44:52.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2ba7fc640 1 -- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0002890 msgr2=0x7fd2a0004d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2ba7fc640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0002890 0x7fd2a0004d30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40107c0 tx=0x7fd2a4010c80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2a400f040 con 0x7fd2a0002890 2024-08-19T10:44:52.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2b3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:44:52.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2b3fff640 10 monclient: dump: 2024-08-19T10:44:52.930 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.930 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.930 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.930 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.930 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.930 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.930 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.930 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.930 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd2a4010e90 con 0x7fd2a0002890 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2b3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2b3fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0002890 msgr2=0x7fd2a0004d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2b3fff640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0002890 0x7fd2a0004d30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40107c0 tx=0x7fd2a4010c80 comp rx=0 tx=0).stop 2024-08-19T10:44:52.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0002890 msgr2=0x7fd2a0004d30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2b3fff640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0002890 0x7fd2a0004d30 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40107c0 tx=0x7fd2a4010c80 comp rx=0 tx=0).stop 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.909+0000 7fd2bb7fe640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd29c03be00 0x7fd29c03e2c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0aceb0 tx=0x7fd2ac0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2c1875640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2c1875640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0006d30 0x7fd2a00071c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2c1875640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2c1875640 1 -- 172.21.15.89:0/519553081 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd2a000a1e0 con 0x7fd2a0006d30 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2ba7fc640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0006d30 0x7fd2a00071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2ba7fc640 10 monclient: get_auth_request con 0x7fd2a0006d30 auth_method 0 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2ba7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:52.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2ba7fc640 10 monclient(hunting): handle_auth_done global_id 44142 payload 824 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.913+0000 7fd2ba7fc640 1 -- 172.21.15.89:0/519553081 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd2a0007700 con 0x7fd2a0006d30 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2ba7fc640 1 -- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0006d30 msgr2=0x7fd2a00071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2ba7fc640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0006d30 0x7fd2a00071c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2a401b000 tx=0x7fd2a40046d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2a401cd00 con 0x7fd2a0006d30 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2b3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2b3fff640 10 monclient: dump: 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.933 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7fd2a401c890 con 0x7fd2a0006d30 2024-08-19T10:44:52.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2b3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:52.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2b3fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:52.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0006d30 msgr2=0x7fd2a00071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2b3fff640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0006d30 0x7fd2a00071c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2a401b000 tx=0x7fd2a40046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2b3fff640 1 -- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0006d30 msgr2=0x7fd2a00071c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:52.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2b3fff640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2a0006d30 0x7fd2a00071c0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2a401b000 tx=0x7fd2a40046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.935 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:52.935 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:52.935 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 16043, 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:52.936 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.937 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.938 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:52.939 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:52.940 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:52.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.942 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:52.943 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.944 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.948 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.949 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.951 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:52.952 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.952 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.952 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.953 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:52.953 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:52.953 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.953 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:52.956 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.957 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.957 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.957 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:52.957 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:52.957 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.957 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.957 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:52.958 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:52.959 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:52.960 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:52.961 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:52.961 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:52.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2c1875640 1 -- 172.21.15.89:0/519553081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd29c03be00 msgr2=0x7fd29c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2c1875640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd29c03be00 0x7fd29c03e2c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0aceb0 tx=0x7fd2ac0a7000 comp rx=0 tx=0).stop 2024-08-19T10:44:52.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2c1875640 10 monclient: shutdown 2024-08-19T10:44:52.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2c1875640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:52.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2c1875640 1 -- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2bc0a5270 msgr2=0x7fd2bc1d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:52.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2c1875640 1 --2- 172.21.15.89:0/519553081 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2bc0a5270 0x7fd2bc1d1140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4007e40 tx=0x7fd2b400e4b0 comp rx=0 tx=0).stop 2024-08-19T10:44:52.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2bb7fe640 1 -- 172.21.15.89:0/519553081 reap_dead start 2024-08-19T10:44:52.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2bb7fe640 1 -- 172.21.15.89:0/519553081 reap_dead start 2024-08-19T10:44:52.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2c1875640 1 -- 172.21.15.89:0/519553081 shutdown_connections 2024-08-19T10:44:52.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2c1875640 1 -- 172.21.15.89:0/519553081 >> 172.21.15.89:0/519553081 conn(0x7fd2bc069a80 msgr2=0x7fd2bc0aa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:52.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2c1875640 1 -- 172.21.15.89:0/519553081 shutdown_connections 2024-08-19T10:44:52.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:52.917+0000 7fd2c1875640 1 -- 172.21.15.89:0/519553081 wait complete. 2024-08-19T10:44:52.964 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:52.964 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:44:52.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:44:53.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient: get_monmap_and_config 2024-08-19T10:44:53.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient: monmap: 2024-08-19T10:44:53.080 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:53.081692+0000 2024-08-19T10:44:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:53.081692+0000 2024-08-19T10:44:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.081 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.082 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.082 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.082 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.082 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:53.082 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:53.082 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 1 Processor -- start 2024-08-19T10:44:53.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 1 -- start start 2024-08-19T10:44:53.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient: init 2024-08-19T10:44:53.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient: _add_conns ranks=[7,6,9,0,3,4,1,8,2,5,10] 2024-08-19T10:44:53.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79b80a3620 0x7f79b80a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient(hunting): picked mon.noname-i con 0x7f79b80a3620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:53.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 0x7f79b80a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient(hunting): picked mon.noname-d con 0x7f79b809c9e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:53.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f79b81373f0 0x7f79b8137830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient(hunting): picked mon.noname-a con 0x7f79b81373f0 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:53.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f79b809dab0 con 0x7f79b81373f0 2024-08-19T10:44:53.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f79b80a1f20 con 0x7f79b809c9e0 2024-08-19T10:44:53.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f79b80a4480 con 0x7f79b80a3620 2024-08-19T10:44:53.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79be468640 10 monclient(hunting): authenticate will time out at 1342.898207s 2024-08-19T10:44:53.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 0x7f79b80a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f79b81373f0 0x7f79b8137830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 0x7f79b80a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:33108/0 (socket says 172.21.15.89:33108) 2024-08-19T10:44:53.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f79b81373f0 0x7f79b8137830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51938/0 (socket says 172.21.15.89:51938) 2024-08-19T10:44:53.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b77fe640 1 -- 172.21.15.89:0/1624774588 learned_addr learned my addr 172.21.15.89:0/1624774588 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:53.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b77fe640 10 monclient(hunting): get_auth_request con 0x7f79b809c9e0 auth_method 0 2024-08-19T10:44:53.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 10 monclient(hunting): get_auth_request con 0x7f79b81373f0 auth_method 0 2024-08-19T10:44:53.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b7fff640 1 --2- 172.21.15.89:0/1624774588 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79b80a3620 0x7f79b80a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b7fff640 10 monclient(hunting): get_auth_request con 0x7f79b80a3620 auth_method 0 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 10 monclient(hunting): handle_auth_done global_id 6201 payload 274 2024-08-19T10:44:53.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 1 -- 172.21.15.89:0/1624774588 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79b80a3620 msgr2=0x7f79b80a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 1 --2- 172.21.15.89:0/1624774588 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79b80a3620 0x7f79b80a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 1 -- 172.21.15.89:0/1624774588 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 msgr2=0x7f79b80a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 1 --2- 172.21.15.89:0/1624774588 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 0x7f79b80a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 1 monclient: found mon.noname-a 2024-08-19T10:44:53.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:53.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79bca05640 1 -- 172.21.15.89:0/1624774588 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b80a1790 con 0x7f79b81373f0 2024-08-19T10:44:53.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.077+0000 7f79b77fe640 1 --2- 172.21.15.89:0/1624774588 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 0x7f79b80a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:53.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79bca05640 1 -- 172.21.15.89:0/1624774588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79b81373f0 msgr2=0x7f79b8137830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79bca05640 1 --2- 172.21.15.89:0/1624774588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79b81373f0 0x7f79b8137830 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f79a00097c0 tx=0x7f79a001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd9256a1eeb6517 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 1 -- 172.21.15.89:0/1624774588 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79a00b0070 con 0x7f79b81373f0 2024-08-19T10:44:53.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:53.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 10 monclient: dump: 2024-08-19T10:44:53.100 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.101 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.102 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 1 -- 172.21.15.89:0/1624774588 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a00aa810 con 0x7f79b81373f0 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 5 monclient: authenticate success, global_id 6201 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 4 set_mon_vals no callback set 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 1 -- 172.21.15.89:0/1624774588 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79a00a9300 con 0x7f79b81373f0 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79b6ffd640 10 monclient: dump: 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.103 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.104 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.105 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 10 monclient: shutdown 2024-08-19T10:44:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 1 -- 172.21.15.89:0/1624774588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79b81373f0 msgr2=0x7f79b8137830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 1 --2- 172.21.15.89:0/1624774588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79b81373f0 0x7f79b8137830 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f79a00097c0 tx=0x7f79a001c330 comp rx=0 tx=0).stop 2024-08-19T10:44:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 1 -- 172.21.15.89:0/1624774588 shutdown_connections 2024-08-19T10:44:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 1 --2- 172.21.15.89:0/1624774588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79b81373f0 0x7f79b8137830 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 1 --2- 172.21.15.89:0/1624774588 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 0x7f79b80a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 1 --2- 172.21.15.89:0/1624774588 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79b80a3620 0x7f79b80a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.081+0000 7f79be468640 1 -- 172.21.15.89:0/1624774588 >> 172.21.15.89:0/1624774588 conn(0x7f79b80a8210 msgr2=0x7f79b80a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 1 -- 172.21.15.89:0/1624774588 shutdown_connections 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 1 -- 172.21.15.89:0/1624774588 wait complete. 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient: monmap: 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:53.106 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 1 Processor -- start 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 1 -- start start 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient: init 2024-08-19T10:44:53.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient: _add_conns ranks=[6,2,7,4,3,8,5,1,0] 2024-08-19T10:44:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 0x7f79b8136330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient(hunting): picked mon.noname-d con 0x7f79b809c9e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79b80a3620 0x7f79b811e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient(hunting): picked mon.noname-b con 0x7f79b80a3620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79b81373f0 0x7f79b811e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient(hunting): picked mon.noname-i con 0x7f79b81373f0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f79b809df40 con 0x7f79b80a3620 2024-08-19T10:44:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f79b8138180 con 0x7f79b809c9e0 2024-08-19T10:44:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f79b8140e60 con 0x7f79b81373f0 2024-08-19T10:44:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79be468640 10 monclient(hunting): authenticate will time out at 1342.905847s 2024-08-19T10:44:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 0x7f79b8136330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79b80a3620 0x7f79b811e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 0x7f79b8136330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:33116/0 (socket says 172.21.15.89:33116) 2024-08-19T10:44:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 1 -- 172.21.15.89:0/3754625672 learned_addr learned my addr 172.21.15.89:0/3754625672 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b77fe640 10 monclient(hunting): get_auth_request con 0x7f79b80a3620 auth_method 0 2024-08-19T10:44:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 10 monclient(hunting): get_auth_request con 0x7f79b809c9e0 auth_method 0 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79bca05640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79b81373f0 0x7f79b811e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79bca05640 10 monclient(hunting): get_auth_request con 0x7f79b81373f0 auth_method 0 2024-08-19T10:44:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79bca05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79bca05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79bca05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 10 monclient(hunting): handle_auth_done global_id 44160 payload 824 2024-08-19T10:44:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 1 -- 172.21.15.89:0/3754625672 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79b81373f0 msgr2=0x7f79b811e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79b81373f0 0x7f79b811e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 1 -- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79b80a3620 msgr2=0x7f79b811e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79b80a3620 0x7f79b811e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 1 monclient: found mon.noname-d 2024-08-19T10:44:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:44:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.085+0000 7f79b7fff640 1 -- 172.21.15.89:0/3754625672 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b81370d0 con 0x7f79b809c9e0 2024-08-19T10:44:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b7fff640 1 -- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 msgr2=0x7f79b8136330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b7fff640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 0x7f79b8136330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f79b0007ec0 tx=0x7f79b000e4b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79b0013a00 con 0x7f79b809c9e0 2024-08-19T10:44:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:44:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 10 monclient: dump: 2024-08-19T10:44:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.112 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.113 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b0019070 con 0x7f79b809c9e0 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79b0013c00 con 0x7f79b809c9e0 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 10 monclient: dump: 2024-08-19T10:44:53.114 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.115 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.116 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b5ffb640 4 set_mon_vals no callback set 2024-08-19T10:44:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79be468640 5 monclient: authenticate success, global_id 44160 2024-08-19T10:44:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79be468640 10 monclient: _renew_subs 2024-08-19T10:44:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79be468640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:44:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79be468640 1 -- 172.21.15.89:0/3754625672 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b811f970 con 0x7f79b809c9e0 2024-08-19T10:44:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79be468640 10 monclient: _renew_subs 2024-08-19T10:44:53.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79be468640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:44:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79be468640 1 -- 172.21.15.89:0/3754625672 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b81204d0 con 0x7f79b809c9e0 2024-08-19T10:44:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79be468640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79be468640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b8136870 0x7f79b811ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79be468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79be468640 1 -- 172.21.15.89:0/3754625672 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f79b80a6e40 con 0x7f79b8136870 2024-08-19T10:44:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b77fe640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b8136870 0x7f79b811ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b77fe640 10 monclient: get_auth_request con 0x7f79b8136870 auth_method 0 2024-08-19T10:44:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f79b0013590 con 0x7f79b809c9e0 2024-08-19T10:44:53.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b77fe640 10 monclient(hunting): handle_auth_done global_id 6020 payload 824 2024-08-19T10:44:53.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b77fe640 1 -- 172.21.15.89:0/3754625672 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79b80a3d00 con 0x7f79b8136870 2024-08-19T10:44:53.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f799803be00 0x7f799803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(227..227 src has 1..227) ==== 14771+0+0 (secure 0 0 0) 0x7f79b0051a20 con 0x7f79b809c9e0 2024-08-19T10:44:53.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79bca05640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f799803be00 0x7f799803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.089+0000 7f79bca05640 10 monclient: get_auth_request con 0x7f799803be00 auth_method 0 2024-08-19T10:44:53.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b77fe640 1 -- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b8136870 msgr2=0x7f79b811ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b77fe640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b8136870 0x7f79b811ece0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f79b811ffe0 tx=0x7f79a8011d80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79a800fd60 con 0x7f79b8136870 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b4ff9640 10 monclient: dump: 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.119 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.120 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.120 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.120 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.120 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.120 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.120 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.120 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.120 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.120 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f79a800fd60 con 0x7f79b8136870 2024-08-19T10:44:53.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b4ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:53.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b8136870 msgr2=0x7f79b811ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b4ff9640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b8136870 0x7f79b811ece0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f79b811ffe0 tx=0x7f79a8011d80 comp rx=0 tx=0).stop 2024-08-19T10:44:53.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b8136870 msgr2=0x7f79b811ece0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:53.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79b4ff9640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b8136870 0x7f79b811ece0 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f79b811ffe0 tx=0x7f79a8011d80 comp rx=0 tx=0).stop 2024-08-19T10:44:53.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.093+0000 7f79bca05640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f799803be00 0x7f799803e2c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f79b80a8c40 tx=0x7f79a00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79be468640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:53.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79be468640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b811f220 0x7f79b8138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79be468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79be468640 1 -- 172.21.15.89:0/3754625672 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f79b811f630 con 0x7f79b811f220 2024-08-19T10:44:53.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79b77fe640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b811f220 0x7f79b8138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79b77fe640 10 monclient: get_auth_request con 0x7f79b811f220 auth_method 0 2024-08-19T10:44:53.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79b77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79b77fe640 10 monclient(hunting): handle_auth_done global_id 6029 payload 824 2024-08-19T10:44:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.097+0000 7f79b77fe640 1 -- 172.21.15.89:0/3754625672 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f79b81217f0 con 0x7f79b811f220 2024-08-19T10:44:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b77fe640 1 -- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b811f220 msgr2=0x7f79b8138400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b77fe640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b811f220 0x7f79b8138400 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f79b811ffe0 tx=0x7f79a80046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79a8014ac0 con 0x7f79b811f220 2024-08-19T10:44:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b4ff9640 10 monclient: dump: 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.125 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.126 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f79a802b470 con 0x7f79b811f220 2024-08-19T10:44:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b4ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b811f220 msgr2=0x7f79b8138400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b4ff9640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b811f220 0x7f79b8138400 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f79b811ffe0 tx=0x7f79a80046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b4ff9640 1 -- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b811f220 msgr2=0x7f79b8138400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:53.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79b4ff9640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79b811f220 0x7f79b8138400 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f79b811ffe0 tx=0x7f79a80046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.127 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:53.127 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:53.127 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:53.127 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:53.127 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:53.127 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 422894, 2024-08-19T10:44:53.127 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:44:53.128 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:53.129 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:53.129 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:53.129 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.129 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.129 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.129 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.129 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.129 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.129 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.129 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.129 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.130 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.130 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.130 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.130 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:53.130 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:53.130 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.130 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.130 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.130 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.130 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.131 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.131 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:53.132 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.133 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.134 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.135 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.136 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:53.136 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:53.136 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.136 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.136 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.136 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:53.136 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:53.136 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.137 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.137 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.137 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:53.137 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.139 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:53.140 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.140 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.140 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.140 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:53.140 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:53.140 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.140 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.140 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:53.141 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:53.142 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.143 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.144 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.145 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.146 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.147 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:53.147 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.147 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.147 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.147 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:53.147 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:53.147 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.147 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.149 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.150 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.151 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.152 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.153 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.153 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:53.153 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.153 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.153 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:53.153 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:53.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79be468640 1 -- 172.21.15.89:0/3754625672 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f799803be00 msgr2=0x7f799803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79be468640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f799803be00 0x7f799803e2c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f79b80a8c40 tx=0x7f79a00a6000 comp rx=0 tx=0).stop 2024-08-19T10:44:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79be468640 10 monclient: shutdown 2024-08-19T10:44:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79be468640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79be468640 1 -- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 msgr2=0x7f79b8136330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79be468640 1 --2- 172.21.15.89:0/3754625672 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79b809c9e0 0x7f79b8136330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f79b0007ec0 tx=0x7f79b000e4b0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79bca05640 1 -- 172.21.15.89:0/3754625672 reap_dead start 2024-08-19T10:44:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79bca05640 1 -- 172.21.15.89:0/3754625672 reap_dead start 2024-08-19T10:44:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79be468640 1 -- 172.21.15.89:0/3754625672 shutdown_connections 2024-08-19T10:44:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79be468640 1 -- 172.21.15.89:0/3754625672 >> 172.21.15.89:0/3754625672 conn(0x7f79b80a8210 msgr2=0x7f79b813dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79be468640 1 -- 172.21.15.89:0/3754625672 shutdown_connections 2024-08-19T10:44:53.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.101+0000 7f79be468640 1 -- 172.21.15.89:0/3754625672 wait complete. 2024-08-19T10:44:53.156 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:53.157 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:44:53.157 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:44:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient: get_monmap_and_config 2024-08-19T10:44:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient: monmap: 2024-08-19T10:44:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:53.280 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:53.281789+0000 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:53.281789+0000 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.281 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:53.282 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:53.282 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 1 Processor -- start 2024-08-19T10:44:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 1 -- start start 2024-08-19T10:44:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient: init 2024-08-19T10:44:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient: _add_conns ranks=[2,4,5,9,3,10,0,1,8,6,7] 2024-08-19T10:44:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3940a5270 0x7ff3940a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient(hunting): picked mon.noname-b con 0x7ff3940a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3940a6d30 0x7ff3940a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient(hunting): picked mon.noname-c con 0x7ff3940a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff3941395b0 0x7ff3941399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient(hunting): picked mon.noname-h con 0x7ff3941395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff3940a7650 con 0x7ff3940a5270 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff3940a8710 con 0x7ff3940a6d30 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff3940a5a40 con 0x7ff3941395b0 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39abae640 10 monclient(hunting): authenticate will time out at 1343.099150s 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3940a6d30 0x7ff3940a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39894a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3940a5270 0x7ff3940a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3940a6d30 0x7ff3940a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36608/0 (socket says 172.21.15.89:36608) 2024-08-19T10:44:53.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 1 -- 172.21.15.89:0/2344122568 learned_addr learned my addr 172.21.15.89:0/2344122568 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39894a640 10 monclient(hunting): get_auth_request con 0x7ff3940a5270 auth_method 0 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39894a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39894a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39894a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 10 monclient(hunting): get_auth_request con 0x7ff3940a6d30 auth_method 0 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39894a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39894a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39894a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39914b640 1 --2- 172.21.15.89:0/2344122568 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff3941395b0 0x7ff3941399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 10 monclient(hunting): handle_auth_done global_id 44158 payload 274 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 1 -- 172.21.15.89:0/2344122568 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff3941395b0 msgr2=0x7ff3941399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 1 --2- 172.21.15.89:0/2344122568 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff3941395b0 0x7ff3941399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 1 -- 172.21.15.89:0/2344122568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3940a5270 msgr2=0x7ff3940a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 1 --2- 172.21.15.89:0/2344122568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3940a5270 0x7ff3940a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 1 monclient: found mon.noname-c 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff393fff640 1 -- 172.21.15.89:0/2344122568 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3940a81d0 con 0x7ff3940a6d30 2024-08-19T10:44:53.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.277+0000 7ff39894a640 1 --2- 172.21.15.89:0/2344122568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3940a5270 0x7ff3940a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:53.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.281+0000 7ff393fff640 1 -- 172.21.15.89:0/2344122568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3940a6d30 msgr2=0x7ff3940a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.281+0000 7ff393fff640 1 --2- 172.21.15.89:0/2344122568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3940a6d30 0x7ff3940a7110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3880098a0 tx=0x7ff38801c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=bdc29c56f04485fc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 1 -- 172.21.15.89:0/2344122568 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff3880b0070 con 0x7ff3940a6d30 2024-08-19T10:44:53.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:44:53.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 10 monclient: dump: 2024-08-19T10:44:53.291 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.291 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.291 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 5 monclient: authenticate success, global_id 44158 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 1 -- 172.21.15.89:0/2344122568 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3880aa810 con 0x7ff3940a6d30 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 4 set_mon_vals no callback set 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 1 -- 172.21.15.89:0/2344122568 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff3880a9300 con 0x7ff3940a6d30 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff3937fe640 10 monclient: dump: 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.293 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.294 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient: shutdown 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 -- 172.21.15.89:0/2344122568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3940a6d30 msgr2=0x7ff3940a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 --2- 172.21.15.89:0/2344122568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3940a6d30 0x7ff3940a7110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3880098a0 tx=0x7ff38801c440 comp rx=0 tx=0).stop 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 -- 172.21.15.89:0/2344122568 shutdown_connections 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 --2- 172.21.15.89:0/2344122568 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff3941395b0 0x7ff3941399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 --2- 172.21.15.89:0/2344122568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff3940a6d30 0x7ff3940a7110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 --2- 172.21.15.89:0/2344122568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff3940a5270 0x7ff3940a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 -- 172.21.15.89:0/2344122568 >> 172.21.15.89:0/2344122568 conn(0x7ff394069a80 msgr2=0x7ff3940abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 -- 172.21.15.89:0/2344122568 shutdown_connections 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 -- 172.21.15.89:0/2344122568 wait complete. 2024-08-19T10:44:53.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient: monmap: 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.296 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 Processor -- start 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 -- start start 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient: init 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient: _add_conns ranks=[7,0,5,1,4,8,3,6,2] 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff3940a5270 0x7ff3940a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient(hunting): picked mon.noname-i con 0x7ff3940a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3940a6d30 0x7ff3940a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient(hunting): picked mon.noname-a con 0x7ff3940a6d30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff3941395b0 0x7ff3941d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient(hunting): picked mon.noname-h con 0x7ff3941395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff3940a57d0 con 0x7ff3940a6d30 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff3940a5650 con 0x7ff3941395b0 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff394143230 con 0x7ff3940a5270 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39abae640 10 monclient(hunting): authenticate will time out at 1343.106218s 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3940a6d30 0x7ff3940a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3940a6d30 0x7ff3940a2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51940/0 (socket says 172.21.15.89:51940) 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 1 -- 172.21.15.89:0/1556188536 learned_addr learned my addr 172.21.15.89:0/1556188536 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:53.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 10 monclient(hunting): get_auth_request con 0x7ff3940a6d30 auth_method 0 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39914b640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff3941395b0 0x7ff3941d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39894a640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff3940a5270 0x7ff3940a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39914b640 10 monclient(hunting): get_auth_request con 0x7ff3941395b0 auth_method 0 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39914b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39914b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39914b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39894a640 10 monclient(hunting): get_auth_request con 0x7ff3940a5270 auth_method 0 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39894a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39894a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39894a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 10 monclient(hunting): handle_auth_done global_id 6210 payload 824 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 1 -- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff3940a5270 msgr2=0x7ff3940a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff3940a5270 0x7ff3940a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 1 -- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff3941395b0 msgr2=0x7ff3941d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff3941395b0 0x7ff3941d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 1 monclient: found mon.noname-a 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff393fff640 1 -- 172.21.15.89:0/1556188536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39409f7c0 con 0x7ff3940a6d30 2024-08-19T10:44:53.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.285+0000 7ff39914b640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff3941395b0 0x7ff3941d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:44:53.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff393fff640 1 -- 172.21.15.89:0/1556188536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3940a6d30 msgr2=0x7ff3940a2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff393fff640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3940a6d30 0x7ff3940a2240 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff3880ae040 tx=0x7ff3880191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff3880b9070 con 0x7ff3940a6d30 2024-08-19T10:44:53.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 10 monclient: dump: 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39abae640 5 monclient: authenticate success, global_id 6210 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39abae640 10 monclient: _renew_subs 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39abae640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39abae640 1 -- 172.21.15.89:0/1556188536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39409cef0 con 0x7ff3940a6d30 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3880044b0 con 0x7ff3940a6d30 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff3880b0040 con 0x7ff3940a6d30 2024-08-19T10:44:53.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 10 monclient: dump: 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39abae640 10 monclient: _renew_subs 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39abae640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39abae640 1 -- 172.21.15.89:0/1556188536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff39409ef20 con 0x7ff3940a6d30 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff392ffd640 4 set_mon_vals no callback set 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39abae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39abae640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39409f160 0x7ff3941d1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39abae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39abae640 1 -- 172.21.15.89:0/1556188536 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff39409d970 con 0x7ff39409f160 2024-08-19T10:44:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff3880bc060 con 0x7ff3940a6d30 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39894a640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39409f160 0x7ff3941d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39894a640 10 monclient: get_auth_request con 0x7ff39409f160 auth_method 0 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39894a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39894a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39894a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff38003bdf0 0x7ff38003e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39894a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39894a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39894a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(227..227 src has 1..227) ==== 14771+0+0 (secure 0 0 0) 0x7ff3880ea090 con 0x7ff3940a6d30 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39914b640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff38003bdf0 0x7ff38003e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39894a640 10 monclient(hunting): handle_auth_done global_id 44110 payload 824 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39894a640 1 -- 172.21.15.89:0/1556188536 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff394140640 con 0x7ff39409f160 2024-08-19T10:44:53.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.289+0000 7ff39914b640 10 monclient: get_auth_request con 0x7ff38003bdf0 auth_method 0 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff39894a640 1 -- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39409f160 msgr2=0x7ff3941d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff39894a640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39409f160 0x7ff3941d1700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff384011510 tx=0x7ff3840119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff39914b640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff38003bdf0 0x7ff38003e2b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff38c007500 tx=0x7ff38c007490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff384015dd0 con 0x7ff39409f160 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff3917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff3917fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff3917fa640 10 monclient: dump: 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff384015960 con 0x7ff39409f160 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff3917fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff3917fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39409f160 msgr2=0x7ff3941d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff3917fa640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39409f160 0x7ff3941d1700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff384011510 tx=0x7ff3840119d0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39409f160 msgr2=0x7ff3941d1700 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.293+0000 7ff3917fa640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39409f160 0x7ff3941d1700 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff384011510 tx=0x7ff3840119d0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39abae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:53.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39abae640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39413f010 0x7ff3941d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39abae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39abae640 1 -- 172.21.15.89:0/1556188536 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff3940a67f0 con 0x7ff39413f010 2024-08-19T10:44:53.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39894a640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39413f010 0x7ff3941d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39894a640 10 monclient: get_auth_request con 0x7ff39413f010 auth_method 0 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39894a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39894a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39894a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39894a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39894a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39894a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39894a640 10 monclient(hunting): handle_auth_done global_id 44119 payload 824 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.301+0000 7ff39894a640 1 -- 172.21.15.89:0/1556188536 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff394064590 con 0x7ff39413f010 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39894a640 1 -- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39413f010 msgr2=0x7ff3941d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39894a640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39413f010 0x7ff3941d1c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff384011510 tx=0x7ff384004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff384003620 con 0x7ff39413f010 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff3917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff3917fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff3917fa640 10 monclient: dump: 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.315 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7ff384004240 con 0x7ff39413f010 2024-08-19T10:44:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff3917fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff3917fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39413f010 msgr2=0x7ff3941d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff3917fa640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39413f010 0x7ff3941d1c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff384011510 tx=0x7ff384004740 comp rx=0 tx=0).stop 2024-08-19T10:44:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff3917fa640 1 -- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39413f010 msgr2=0x7ff3941d1c40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:53.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff3917fa640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39413f010 0x7ff3941d1c40 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff384011510 tx=0x7ff384004740 comp rx=0 tx=0).stop 2024-08-19T10:44:53.315 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 16431, 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:53.316 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.317 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.318 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:53.319 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.320 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:53.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:53.322 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:53.325 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:53.327 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:53.329 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:53.330 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.332 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:53.332 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:53.332 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.332 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:53.332 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.334 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:53.334 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:53.334 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:53.334 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.336 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:53.336 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:53.336 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.336 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.336 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.336 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:53.336 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.338 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:53.339 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.339 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.339 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:53.339 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:53.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39abae640 1 -- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff38003bdf0 msgr2=0x7ff38003e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39abae640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff38003bdf0 0x7ff38003e2b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff38c007500 tx=0x7ff38c007490 comp rx=0 tx=0).stop 2024-08-19T10:44:53.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39abae640 10 monclient: shutdown 2024-08-19T10:44:53.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39abae640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39abae640 1 -- 172.21.15.89:0/1556188536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3940a6d30 msgr2=0x7ff3940a2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39abae640 1 --2- 172.21.15.89:0/1556188536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3940a6d30 0x7ff3940a2240 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff3880ae040 tx=0x7ff3880191e0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39914b640 1 -- 172.21.15.89:0/1556188536 reap_dead start 2024-08-19T10:44:53.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39914b640 1 -- 172.21.15.89:0/1556188536 reap_dead start 2024-08-19T10:44:53.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39abae640 1 -- 172.21.15.89:0/1556188536 shutdown_connections 2024-08-19T10:44:53.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39abae640 1 -- 172.21.15.89:0/1556188536 >> 172.21.15.89:0/1556188536 conn(0x7ff394069a80 msgr2=0x7ff394134660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39abae640 1 -- 172.21.15.89:0/1556188536 shutdown_connections 2024-08-19T10:44:53.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.305+0000 7ff39abae640 1 -- 172.21.15.89:0/1556188536 wait complete. 2024-08-19T10:44:53.341 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:53.341 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:44:53.342 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:44:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient: get_monmap_and_config 2024-08-19T10:44:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient: monmap: 2024-08-19T10:44:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:53.457135+0000 2024-08-19T10:44:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:53.457135+0000 2024-08-19T10:44:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:53.456 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:53.457 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:53.457 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 1 Processor -- start 2024-08-19T10:44:53.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 1 -- start start 2024-08-19T10:44:53.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient: init 2024-08-19T10:44:53.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient: _add_conns ranks=[7,2,6,1,10,5,3,0,9,8,4] 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93d80a4b30 0x7f93d80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient(hunting): picked mon.noname-i con 0x7f93d80a4b30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93d80720c0 0x7f93d80a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient(hunting): picked mon.noname-b con 0x7f93d80720c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93d80a5450 0x7f93d813d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient(hunting): picked mon.noname-d con 0x7f93d80a5450 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f93d8073f30 con 0x7f93d80720c0 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f93d8073600 con 0x7f93d80a5450 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f93d80738e0 con 0x7f93d80a4b30 2024-08-19T10:44:53.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93ddbf5640 10 monclient(hunting): authenticate will time out at 1343.273843s 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93d80a5450 0x7f93d813d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93d80a5450 0x7f93d813d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:33120/0 (socket says 172.21.15.89:33120) 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 1 -- 172.21.15.89:0/67444378 learned_addr learned my addr 172.21.15.89:0/67444378 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 10 monclient(hunting): get_auth_request con 0x7f93d80a5450 auth_method 0 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d6ffd640 1 --2- 172.21.15.89:0/67444378 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93d80720c0 0x7f93d80a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d6ffd640 10 monclient(hunting): get_auth_request con 0x7f93d80720c0 auth_method 0 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d77fe640 1 --2- 172.21.15.89:0/67444378 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93d80a4b30 0x7f93d80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d77fe640 10 monclient(hunting): get_auth_request con 0x7f93d80a4b30 auth_method 0 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 10 monclient(hunting): handle_auth_done global_id 44169 payload 274 2024-08-19T10:44:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 1 -- 172.21.15.89:0/67444378 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93d80a4b30 msgr2=0x7f93d80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 1 --2- 172.21.15.89:0/67444378 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93d80a4b30 0x7f93d80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 1 -- 172.21.15.89:0/67444378 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93d80720c0 msgr2=0x7f93d80a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 1 --2- 172.21.15.89:0/67444378 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93d80720c0 0x7f93d80a4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 1 monclient: found mon.noname-d 2024-08-19T10:44:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:44:53.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.453+0000 7f93d7fff640 1 -- 172.21.15.89:0/67444378 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d80a42a0 con 0x7f93d80a5450 2024-08-19T10:44:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d7fff640 1 -- 172.21.15.89:0/67444378 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93d80a5450 msgr2=0x7f93d813d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d7fff640 1 --2- 172.21.15.89:0/67444378 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93d80a5450 0x7f93d813d630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93c80098a0 tx=0x7f93c801c440 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d9e97b5e77a274d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 1 -- 172.21.15.89:0/67444378 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93c80b2070 con 0x7f93d80a5450 2024-08-19T10:44:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:44:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 10 monclient: dump: 2024-08-19T10:44:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.469 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 1 -- 172.21.15.89:0/67444378 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c80ab7f0 con 0x7f93d80a5450 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 5 monclient: authenticate success, global_id 44169 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 1 -- 172.21.15.89:0/67444378 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93c80abe10 con 0x7f93d80a5450 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 4 set_mon_vals no callback set 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93d67fc640 10 monclient: dump: 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.471 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.472 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 10 monclient: shutdown 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 1 -- 172.21.15.89:0/67444378 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93d80a5450 msgr2=0x7f93d813d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 1 --2- 172.21.15.89:0/67444378 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93d80a5450 0x7f93d813d630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93c80098a0 tx=0x7f93c801c440 comp rx=0 tx=0).stop 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 1 -- 172.21.15.89:0/67444378 shutdown_connections 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 1 --2- 172.21.15.89:0/67444378 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93d80a5450 0x7f93d813d630 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 1 --2- 172.21.15.89:0/67444378 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f93d80720c0 0x7f93d80a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 1 --2- 172.21.15.89:0/67444378 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93d80a4b30 0x7f93d80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 1 -- 172.21.15.89:0/67444378 >> 172.21.15.89:0/67444378 conn(0x7f93d80aa240 msgr2=0x7f93d80aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 1 -- 172.21.15.89:0/67444378 shutdown_connections 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.457+0000 7f93ddbf5640 1 -- 172.21.15.89:0/67444378 wait complete. 2024-08-19T10:44:53.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient: monmap: 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.474 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 1 Processor -- start 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 1 -- start start 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient: init 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient: _add_conns ranks=[7,8,5,2,4,6,0,3,1] 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93d80720c0 0x7f93d8137540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient(hunting): picked mon.noname-i con 0x7f93d80720c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f93d80a4b30 0x7f93d8120490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient(hunting): picked mon.noname-e con 0x7f93d80a4b30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f93d80a5450 0x7f93d81209d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient(hunting): picked mon.noname-h con 0x7f93d80a5450 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f93d8140b30 con 0x7f93d80a5450 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f93d81409b0 con 0x7f93d80720c0 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f93d8140cb0 con 0x7f93d80a4b30 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93ddbf5640 10 monclient(hunting): authenticate will time out at 1343.281168s 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f93d80a4b30 0x7f93d8120490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f93d80a4b30 0x7f93d8120490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50554/0 (socket says 172.21.15.89:50554) 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 1 -- 172.21.15.89:0/2557453322 learned_addr learned my addr 172.21.15.89:0/2557453322 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 10 monclient(hunting): get_auth_request con 0x7f93d80a4b30 auth_method 0 2024-08-19T10:44:53.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d7fff640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f93d80a5450 0x7f93d81209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d77fe640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93d80720c0 0x7f93d8137540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 10 monclient(hunting): handle_auth_done global_id 6038 payload 824 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 1 -- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93d80720c0 msgr2=0x7f93d8137540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93d80720c0 0x7f93d8137540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 1 -- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f93d80a5450 msgr2=0x7f93d81209d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f93d80a5450 0x7f93d81209d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 1 monclient: found mon.noname-e 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 1 -- 172.21.15.89:0/2557453322 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d80a9500 con 0x7f93d80a4b30 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 1 -- 172.21.15.89:0/2557453322 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f93d80a4b30 msgr2=0x7f93d8120490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.461+0000 7f93d6ffd640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f93d80a4b30 0x7f93d8120490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00efc0 tx=0x7f93cc00c490 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93cc018020 con 0x7f93d80a4b30 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 10 monclient: dump: 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.478 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93ddbf5640 5 monclient: authenticate success, global_id 6038 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93ddbf5640 10 monclient: _renew_subs 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93cc0148f0 con 0x7f93d80a4b30 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93ddbf5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93ddbf5640 1 -- 172.21.15.89:0/2557453322 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d81382e0 con 0x7f93d80a4b30 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93cc014480 con 0x7f93d80a4b30 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 10 monclient: dump: 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.480 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d5ffb640 4 set_mon_vals no callback set 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93ddbf5640 10 monclient: _renew_subs 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93ddbf5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93ddbf5640 1 -- 172.21.15.89:0/2557453322 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d8122c00 con 0x7f93d80a4b30 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93ddbf5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93ddbf5640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d81216e0 0x7f93d8120f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93ddbf5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93ddbf5640 1 -- 172.21.15.89:0/2557453322 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f93d8064590 con 0x7f93d81216e0 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f93cc01b020 con 0x7f93d80a4b30 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d77fe640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d81216e0 0x7f93d8120f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d77fe640 10 monclient: get_auth_request con 0x7f93d81216e0 auth_method 0 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f93c403be00 0x7f93c403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(227..227 src has 1..227) ==== 14771+0+0 (secure 0 0 0) 0x7f93cc052080 con 0x7f93d80a4b30 2024-08-19T10:44:53.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d7fff640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f93c403be00 0x7f93c403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d77fe640 10 monclient(hunting): handle_auth_done global_id 34572 payload 824 2024-08-19T10:44:53.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d77fe640 1 -- 172.21.15.89:0/2557453322 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93d80a25d0 con 0x7f93d81216e0 2024-08-19T10:44:53.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.465+0000 7f93d7fff640 10 monclient: get_auth_request con 0x7f93c403be00 auth_method 0 2024-08-19T10:44:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93d77fe640 1 -- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d81216e0 msgr2=0x7f93d8120f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93d77fe640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d81216e0 0x7f93d8120f10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93d000e510 tx=0x7f93d000e9d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93d0017070 con 0x7f93d81216e0 2024-08-19T10:44:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93bffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93bffff640 10 monclient: dump: 2024-08-19T10:44:53.486 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.487 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93d7fff640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f93c403be00 0x7f93c403e2c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009870 tx=0x7f93c80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f93d0013700 con 0x7f93d81216e0 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93bffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93bffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d81216e0 msgr2=0x7f93d8120f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93bffff640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d81216e0 0x7f93d8120f10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93d000e510 tx=0x7f93d000e9d0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d81216e0 msgr2=0x7f93d8120f10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.469+0000 7f93bffff640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d81216e0 0x7f93d8120f10 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93d000e510 tx=0x7f93d000e9d0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93ddbf5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93ddbf5640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d8137a80 0x7f93d813bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93ddbf5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93ddbf5640 1 -- 172.21.15.89:0/2557453322 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f93d80a4f10 con 0x7f93d8137a80 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93d77fe640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d8137a80 0x7f93d813bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93d77fe640 10 monclient: get_auth_request con 0x7f93d8137a80 auth_method 0 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93d77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.473+0000 7f93d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93d77fe640 10 monclient(hunting): handle_auth_done global_id 34581 payload 824 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93d77fe640 1 -- 172.21.15.89:0/2557453322 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f93d806fdf0 con 0x7f93d8137a80 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93d77fe640 1 -- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d8137a80 msgr2=0x7f93d813bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93d77fe640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d8137a80 0x7f93d813bc10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93d000e510 tx=0x7f93d00046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93d000f040 con 0x7f93d8137a80 2024-08-19T10:44:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93bffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93bffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93bffff640 10 monclient: dump: 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.490 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f93d0013e20 con 0x7f93d8137a80 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93bffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93bffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d8137a80 msgr2=0x7f93d813bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93bffff640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d8137a80 0x7f93d813bc10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93d000e510 tx=0x7f93d00046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93bffff640 1 -- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d8137a80 msgr2=0x7f93d813bc10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:53.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.477+0000 7f93bffff640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93d8137a80 0x7f93d813bc10 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93d000e510 tx=0x7f93d00046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.492 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 138525, 2024-08-19T10:44:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:53.492 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:53.493 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:53.494 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:53.496 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.497 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.499 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.500 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.501 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.502 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:53.504 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.512 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.513 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.514 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:53.515 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:53.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.489+0000 7f1fc5ffb640 1 -- 172.21.15.190:0/343016576 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}]=0 added bucket h32 type host to location {rack=r1} v228) ==== 168+0+0 (secure 0 0 0) 0x7f1fc8013520 con 0x7f1fd013f0a0 2024-08-19T10:44:53.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.489+0000 7f1fc5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:53.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.489+0000 7f1fc5ffb640 10 monclient: _finish_command 2 = system:0 added bucket h32 type host to location {rack=r1} 2024-08-19T10:44:53.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93ddbf5640 1 -- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f93c403be00 msgr2=0x7f93c403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93ddbf5640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f93c403be00 0x7f93c403e2c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009870 tx=0x7f93c80a7000 comp rx=0 tx=0).stop 2024-08-19T10:44:53.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93ddbf5640 10 monclient: shutdown 2024-08-19T10:44:53.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93ddbf5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93ddbf5640 1 -- 172.21.15.89:0/2557453322 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f93d80a4b30 msgr2=0x7f93d8120490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93ddbf5640 1 --2- 172.21.15.89:0/2557453322 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f93d80a4b30 0x7f93d8120490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00efc0 tx=0x7f93cc00c490 comp rx=0 tx=0).stop 2024-08-19T10:44:53.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93d7fff640 1 -- 172.21.15.89:0/2557453322 reap_dead start 2024-08-19T10:44:53.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93d7fff640 1 -- 172.21.15.89:0/2557453322 reap_dead start 2024-08-19T10:44:53.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93ddbf5640 1 -- 172.21.15.89:0/2557453322 shutdown_connections 2024-08-19T10:44:53.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93ddbf5640 1 -- 172.21.15.89:0/2557453322 >> 172.21.15.89:0/2557453322 conn(0x7f93d80aa240 msgr2=0x7f93d813a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93ddbf5640 1 -- 172.21.15.89:0/2557453322 shutdown_connections 2024-08-19T10:44:53.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.481+0000 7f93ddbf5640 1 -- 172.21.15.89:0/2557453322 wait complete. 2024-08-19T10:44:53.519 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:53.519 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:44:53.519 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:44:53.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.601+0000 7f1fd7494640 10 start_mon_command cmd=[{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:53.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.601+0000 7f1fd7494640 10 monclient: _send_command 3 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:53.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.601+0000 7f1fd7494640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:44:53.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.601+0000 7f1fd7494640 1 -- 172.21.15.190:0/343016576 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]} v 0) -- 0x7f1fd0138670 con 0x7f1fd013f0a0 2024-08-19T10:44:53.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.601+0000 7f1fc5ffb640 1 -- 172.21.15.190:0/343016576 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}]=0 bucket 'h32' already exists v228) ==== 147+0+0 (secure 0 0 0) 0x7f1fc8014c50 con 0x7f1fd013f0a0 2024-08-19T10:44:53.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.601+0000 7f1fc5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush add-bucket", "name": "h32", "type": "host", "args": ["rack=r1"]}] 2024-08-19T10:44:53.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.601+0000 7f1fc5ffb640 10 monclient: _finish_command 3 = system:0 bucket 'h32' already exists 2024-08-19T10:44:53.604 INFO:tasks.workunit.client.0.smithi190.stderr:bucket 'h32' already exists 2024-08-19T10:44:53.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 -- 172.21.15.190:0/343016576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fb403be00 msgr2=0x7f1fb403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fb403be00 0x7f1fb403e2c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc007240 tx=0x7f1fcc007e70 comp rx=0 tx=0).stop 2024-08-19T10:44:53.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 10 monclient: shutdown 2024-08-19T10:44:53.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 -- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1fd013f0a0 msgr2=0x7f1fd013b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1fd013f0a0 0x7f1fd013b3b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8007e90 tx=0x7f1fc800e4b0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 -- 172.21.15.190:0/343016576 shutdown_connections 2024-08-19T10:44:53.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1fd013f0a0 0x7f1fd013b3b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1fb403be00 0x7f1fb403e2c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1fd013a190 0x7f1fd013ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 --2- 172.21.15.190:0/343016576 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1fd01396a0 0x7f1fd013a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 -- 172.21.15.190:0/343016576 >> 172.21.15.190:0/343016576 conn(0x7f1fd00abee0 msgr2=0x7f1fd013f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 -- 172.21.15.190:0/343016576 shutdown_connections 2024-08-19T10:44:53.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.605+0000 7f1fd7494640 1 -- 172.21.15.190:0/343016576 wait complete. 2024-08-19T10:44:53.622 INFO:tasks.workunit.client.0.smithi190.stderr:++ seq 1 32 2024-08-19T10:44:53.622 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:53.622 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h1 2024-08-19T10:44:53.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.629+0000 7fe93d256640 10 monclient: get_monmap_and_config 2024-08-19T10:44:53.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.629+0000 7fe93d256640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.629+0000 7fe93d256640 10 monclient: monmap: 2024-08-19T10:44:53.634 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:53.634 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.634 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:53.635832+0000 2024-08-19T10:44:53.634 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:53.635832+0000 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:53.635 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.629+0000 7fe93d256640 1 Processor -- start 2024-08-19T10:44:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.629+0000 7fe93d256640 1 -- start start 2024-08-19T10:44:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.629+0000 7fe93d256640 10 monclient: init 2024-08-19T10:44:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 10 monclient: _add_conns ranks=[2,3,0,6,4,8,7,9,5,1,10] 2024-08-19T10:44:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9380a6860 0x7fe9380a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 10 monclient(hunting): picked mon.noname-b con 0x7fe9380a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9380a7180 0x7fe9380a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 10 monclient(hunting): picked mon.noname-g con 0x7fe9380a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe938139620 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9380a33d0 con 0x7fe938139620 2024-08-19T10:44:53.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fe9380a8190 con 0x7fe9380a6860 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe9380a2ec0 con 0x7fe9380a7180 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93d256640 10 monclient(hunting): authenticate will time out at 1343.452442s 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93759e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe938139620 0x7fe938139a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58326/0 (socket says 172.21.15.89:58326) 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93759e640 1 -- 172.21.15.89:0/433080424 learned_addr learned my addr 172.21.15.89:0/433080424 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9380a6860 0x7fe9380a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 10 monclient(hunting): get_auth_request con 0x7fe9380a6860 auth_method 0 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93659c640 1 --2- 172.21.15.89:0/433080424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9380a7180 0x7fe9380a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93659c640 10 monclient(hunting): get_auth_request con 0x7fe9380a7180 auth_method 0 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 10 monclient(hunting): handle_auth_done global_id 6203 payload 274 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 1 -- 172.21.15.89:0/433080424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9380a7180 msgr2=0x7fe9380a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 1 --2- 172.21.15.89:0/433080424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9380a7180 0x7fe9380a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 1 -- 172.21.15.89:0/433080424 >> v1:172.21.15.89:6789/0 conn(0x7fe938139620 legacy=0x7fe938139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 1 monclient: found mon.noname-b 2024-08-19T10:44:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:44:53.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.633+0000 7fe936d9d640 1 -- 172.21.15.89:0/433080424 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9380a4cd0 con 0x7fe9380a6860 2024-08-19T10:44:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe936d9d640 1 -- 172.21.15.89:0/433080424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9380a6860 msgr2=0x7fe9380a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe936d9d640 1 --2- 172.21.15.89:0/433080424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9380a6860 0x7fe9380a6c40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe9280097f0 tx=0x7fe92801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=580e62da50bfc395 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 1 -- 172.21.15.89:0/433080424 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe9280b0070 con 0x7fe9380a6860 2024-08-19T10:44:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:44:53.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 10 monclient: dump: 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.644 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 5 monclient: authenticate success, global_id 6203 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 1 -- 172.21.15.89:0/433080424 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9280aa810 con 0x7fe9380a6860 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 4 set_mon_vals no callback set 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 1 -- 172.21.15.89:0/433080424 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe9280a9300 con 0x7fe9380a6860 2024-08-19T10:44:53.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe935d9b640 10 monclient: dump: 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.646 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 10 monclient: shutdown 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 1 -- 172.21.15.89:0/433080424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9380a6860 msgr2=0x7fe9380a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 1 --2- 172.21.15.89:0/433080424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9380a6860 0x7fe9380a6c40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe9280097f0 tx=0x7fe92801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 1 -- 172.21.15.89:0/433080424 shutdown_connections 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 1 --2- 172.21.15.89:0/433080424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9380a7180 0x7fe9380a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 1 --2- 172.21.15.89:0/433080424 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe9380a6860 0x7fe9380a6c40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 1 -- 172.21.15.89:0/433080424 >> 172.21.15.89:0/433080424 conn(0x7fe938074b40 msgr2=0x7fe9380abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 1 -- 172.21.15.89:0/433080424 shutdown_connections 2024-08-19T10:44:53.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.637+0000 7fe93d256640 1 -- 172.21.15.89:0/433080424 wait complete. 2024-08-19T10:44:53.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient: monmap: 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.652 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 1 Processor -- start 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 1 -- start start 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient: init 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient: _add_conns ranks=[0,5,3,1,7,6,8,4,2] 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9380a6860 0x7fe9381d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient(hunting): picked mon.noname-a con 0x7fe9380a6860 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe9380a7180 0x7fe9381387a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient(hunting): picked mon.noname-h con 0x7fe9380a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:53.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe938139620 0x7fe93813d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient(hunting): picked mon.noname-g con 0x7fe938139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe938144430 con 0x7fe9380a6860 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe9381442b0 con 0x7fe938139620 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe9381445b0 con 0x7fe9380a7180 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93d256640 10 monclient(hunting): authenticate will time out at 1343.461774s 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9380a6860 0x7fe9381d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9380a6860 0x7fe9381d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51954/0 (socket says 172.21.15.89:51954) 2024-08-19T10:44:53.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 1 -- 172.21.15.89:0/1625276573 learned_addr learned my addr 172.21.15.89:0/1625276573 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 10 monclient(hunting): get_auth_request con 0x7fe9380a6860 auth_method 0 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93759e640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe938139620 0x7fe93813d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93659c640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe9380a7180 0x7fe9381387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93759e640 10 monclient(hunting): get_auth_request con 0x7fe938139620 auth_method 0 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93759e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93759e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93659c640 10 monclient(hunting): get_auth_request con 0x7fe9380a7180 auth_method 0 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 10 monclient(hunting): handle_auth_done global_id 6219 payload 824 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 1 -- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe9380a7180 msgr2=0x7fe9381387a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe9380a7180 0x7fe9381387a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 1 -- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe938139620 msgr2=0x7fe93813d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe938139620 0x7fe93813d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 1 monclient: found mon.noname-a 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.641+0000 7fe936d9d640 1 -- 172.21.15.89:0/1625276573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93813f170 con 0x7fe9380a6860 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe936d9d640 1 -- 172.21.15.89:0/1625276573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9380a6860 msgr2=0x7fe9381d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe936d9d640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9380a6860 0x7fe9381d16a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe9280097f0 tx=0x7fe928019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe9280b0040 con 0x7fe9380a6860 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 10 monclient: dump: 2024-08-19T10:44:53.657 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.658 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.659 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9280044b0 con 0x7fe9380a6860 2024-08-19T10:44:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe928003a90 con 0x7fe9380a6860 2024-08-19T10:44:53.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93d256640 5 monclient: authenticate success, global_id 6219 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93d256640 10 monclient: _renew_subs 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93d256640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93d256640 1 -- 172.21.15.89:0/1625276573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe938140850 con 0x7fe9380a6860 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93559a640 4 set_mon_vals no callback set 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 10 monclient: dump: 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.660 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93d256640 10 monclient: _renew_subs 2024-08-19T10:44:53.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93d256640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93d256640 1 -- 172.21.15.89:0/1625276573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9380aae40 con 0x7fe9380a6860 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93d256640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93d256640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe938138ce0 0x7fe93813dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93d256640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93d256640 1 -- 172.21.15.89:0/1625276573 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe938076ca0 con 0x7fe938138ce0 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93659c640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe938138ce0 0x7fe93813dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93659c640 10 monclient: get_auth_request con 0x7fe938138ce0 auth_method 0 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe928003c50 con 0x7fe9380a6860 2024-08-19T10:44:53.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93659c640 10 monclient(hunting): handle_auth_done global_id 6179 payload 824 2024-08-19T10:44:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe92403be00 0x7fe92403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93659c640 1 -- 172.21.15.89:0/1625276573 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9380ab4e0 con 0x7fe938138ce0 2024-08-19T10:44:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(228..228 src has 1..228) ==== 14984+0+0 (secure 0 0 0) 0x7fe9280eaca0 con 0x7fe9380a6860 2024-08-19T10:44:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93759e640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe92403be00 0x7fe92403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.645+0000 7fe93759e640 10 monclient: get_auth_request con 0x7fe92403be00 auth_method 0 2024-08-19T10:44:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe93659c640 1 -- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe938138ce0 msgr2=0x7fe93813dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe93659c640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe938138ce0 0x7fe93813dc40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe93813fc60 tx=0x7fe92c011d60 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe92c019070 con 0x7fe938138ce0 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe91f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe91f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe91f7fe640 10 monclient: dump: 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.667 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe93759e640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe92403be00 0x7fe92403e2c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe9300055b0 tx=0x7fe930005540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe92c00f890 con 0x7fe938138ce0 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe91f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe91f7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe938138ce0 msgr2=0x7fe93813dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe91f7fe640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe938138ce0 0x7fe93813dc40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe93813fc60 tx=0x7fe92c011d60 comp rx=0 tx=0).stop 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe938138ce0 msgr2=0x7fe93813dc40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.649+0000 7fe91f7fe640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe938138ce0 0x7fe93813dc40 secure :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe93813fc60 tx=0x7fe92c011d60 comp rx=0 tx=0).stop 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.653+0000 7fe93d256640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.653+0000 7fe93d256640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe93813e180 0x7fe9380a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.653+0000 7fe93d256640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.653+0000 7fe93d256640 1 -- 172.21.15.89:0/1625276573 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe93813e590 con 0x7fe93813e180 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.653+0000 7fe93659c640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe93813e180 0x7fe9380a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.653+0000 7fe93659c640 10 monclient: get_auth_request con 0x7fe93813e180 auth_method 0 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.653+0000 7fe93659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.653+0000 7fe93659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.653+0000 7fe93659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93659c640 10 monclient(hunting): handle_auth_done global_id 6188 payload 824 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93659c640 1 -- 172.21.15.89:0/1625276573 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe9380a6c40 con 0x7fe93813e180 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93659c640 1 -- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe93813e180 msgr2=0x7fe9380a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93659c640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe93813e180 0x7fe9380a0110 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe93813fc60 tx=0x7fe92c0046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe92c019040 con 0x7fe93813e180 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe91f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe91f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe91f7fe640 10 monclient: dump: 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.670 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9601 (secure 0 0 0) 0x7fe92c003e10 con 0x7fe93813e180 2024-08-19T10:44:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe91f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe91f7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe93813e180 msgr2=0x7fe9380a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe91f7fe640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe93813e180 0x7fe9380a0110 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe93813fc60 tx=0x7fe92c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe91f7fe640 1 -- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe93813e180 msgr2=0x7fe9380a0110 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:53.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe91f7fe640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe93813e180 0x7fe9380a0110 secure :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe93813fc60 tx=0x7fe92c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.672 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:53.672 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:53.672 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 423432, 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:53.673 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.674 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.675 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:53.676 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.677 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:53.678 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.680 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:53.681 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:53.684 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.685 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.687 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.689 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:53.690 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.690 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.690 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:53.690 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.690 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.690 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:53.690 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:53.691 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.692 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.693 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.694 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.695 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.696 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.697 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:53.697 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93d256640 1 -- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe92403be00 msgr2=0x7fe92403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93d256640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe92403be00 0x7fe92403e2c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe9300055b0 tx=0x7fe930005540 comp rx=0 tx=0).stop 2024-08-19T10:44:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93d256640 10 monclient: shutdown 2024-08-19T10:44:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93d256640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93d256640 1 -- 172.21.15.89:0/1625276573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9380a6860 msgr2=0x7fe9381d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93d256640 1 --2- 172.21.15.89:0/1625276573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9380a6860 0x7fe9381d16a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe9280097f0 tx=0x7fe928019140 comp rx=0 tx=0).stop 2024-08-19T10:44:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93759e640 1 -- 172.21.15.89:0/1625276573 reap_dead start 2024-08-19T10:44:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93759e640 1 -- 172.21.15.89:0/1625276573 reap_dead start 2024-08-19T10:44:53.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93d256640 1 -- 172.21.15.89:0/1625276573 shutdown_connections 2024-08-19T10:44:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93d256640 1 -- 172.21.15.89:0/1625276573 >> 172.21.15.89:0/1625276573 conn(0x7fe938074b40 msgr2=0x7fe9380a9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93d256640 1 -- 172.21.15.89:0/1625276573 shutdown_connections 2024-08-19T10:44:53.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.657+0000 7fe93d256640 1 -- 172.21.15.89:0/1625276573 wait complete. 2024-08-19T10:44:53.699 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:53.699 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:44:53.699 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:44:53.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient: get_monmap_and_config 2024-08-19T10:44:53.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient: monmap: 2024-08-19T10:44:53.744 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:53.744 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.744 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:53.746054+0000 2024-08-19T10:44:53.744 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:53.746054+0000 2024-08-19T10:44:53.744 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.744 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:53.745 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:53.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 1 Processor -- start 2024-08-19T10:44:53.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 1 -- start start 2024-08-19T10:44:53.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient: init 2024-08-19T10:44:53.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient: _add_conns ranks=[0,6,7,1,5,10,3,9,8,4,2] 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1f9c0a6740 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1f9c0a6b20 0x7f1f9c0a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient(hunting): picked mon.noname-d con 0x7f1f9c0a6b20 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c13eaa0 0x7f1f9c140e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient(hunting): picked mon.noname-i con 0x7f1f9c13eaa0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f9c0a5d90 con 0x7f1f9c0a6740 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1f9c0a4c20 con 0x7f1f9c0a6b20 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f1f9c0a3530 con 0x7f1f9c13eaa0 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.741+0000 7f1fa2468640 10 monclient(hunting): authenticate will time out at 1339.315935s 2024-08-19T10:44:53.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c13eaa0 0x7f1f9c140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c13eaa0 0x7f1f9c140e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34168/0 (socket says 172.21.15.190:34168) 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 1 -- 172.21.15.190:0/2955493866 learned_addr learned my addr 172.21.15.190:0/2955493866 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 10 monclient(hunting): get_auth_request con 0x7f1f9c13eaa0 auth_method 0 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1f9b7fe640 1 --2- 172.21.15.190:0/2955493866 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1f9c0a6b20 0x7f1f9c0a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1f9b7fe640 10 monclient(hunting): get_auth_request con 0x7f1f9c0a6b20 auth_method 0 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1f9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1f9b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1f9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1f9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 10 monclient(hunting): handle_auth_done global_id 15469 payload 274 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 1 -- 172.21.15.190:0/2955493866 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1f9c0a6b20 msgr2=0x7f1f9c0a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 1 --2- 172.21.15.190:0/2955493866 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1f9c0a6b20 0x7f1f9c0a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 1 -- 172.21.15.190:0/2955493866 >> v1:172.21.15.89:6789/0 conn(0x7f1f9c0a6740 legacy=0x7f1f9c0a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 1 monclient: found mon.noname-i 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:53.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.745+0000 7f1fa0a05640 1 -- 172.21.15.190:0/2955493866 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c0a3740 con 0x7f1f9c13eaa0 2024-08-19T10:44:53.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa0a05640 1 -- 172.21.15.190:0/2955493866 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c13eaa0 msgr2=0x7f1f9c140e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa0a05640 1 --2- 172.21.15.190:0/2955493866 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c13eaa0 0x7f1f9c140e90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009eb0 tx=0x7f1f9401c8f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=b55831250ca6eae6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 1 -- 172.21.15.190:0/2955493866 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f940090d0 con 0x7f1f9c13eaa0 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 10 monclient: dump: 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.755 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 5 monclient: authenticate success, global_id 15469 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 1 -- 172.21.15.190:0/2955493866 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f940a55f0 con 0x7f1f9c13eaa0 2024-08-19T10:44:53.756 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 1 -- 172.21.15.190:0/2955493866 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f94009270 con 0x7f1f9c13eaa0 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9affd640 10 monclient: dump: 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.757 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 4 set_mon_vals no callback set 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient: shutdown 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 -- 172.21.15.190:0/2955493866 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c13eaa0 msgr2=0x7f1f9c140e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 --2- 172.21.15.190:0/2955493866 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c13eaa0 0x7f1f9c140e90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009eb0 tx=0x7f1f9401c8f0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 -- 172.21.15.190:0/2955493866 shutdown_connections 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 --2- 172.21.15.190:0/2955493866 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1f9c0a6b20 0x7f1f9c0a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 --2- 172.21.15.190:0/2955493866 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c13eaa0 0x7f1f9c140e90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 -- 172.21.15.190:0/2955493866 >> 172.21.15.190:0/2955493866 conn(0x7f1f9c0abd40 msgr2=0x7f1f9c0ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 -- 172.21.15.190:0/2955493866 shutdown_connections 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 -- 172.21.15.190:0/2955493866 wait complete. 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient: monmap: 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:53.759 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 Processor -- start 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 -- start start 2024-08-19T10:44:53.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient: init 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient: _add_conns ranks=[7,0,3,4,1,5,6,2,8] 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c0a6740 0x7f1f9c139460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient(hunting): picked mon.noname-i con 0x7f1f9c0a6740 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9c0a6b20 0x7f1f9c0a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient(hunting): picked mon.noname-a con 0x7f1f9c0a6b20 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f9c13eaa0 0x7f1f9c0a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient(hunting): picked mon.noname-g con 0x7f1f9c13eaa0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c143d70 con 0x7f1f9c0a6b20 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9c143bf0 con 0x7f1f9c13eaa0 2024-08-19T10:44:53.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f1f9c143ef0 con 0x7f1f9c0a6740 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa2468640 10 monclient(hunting): authenticate will time out at 1339.323185s 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c0a6740 0x7f1f9c139460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa0a05640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f9c13eaa0 0x7f1f9c0a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c0a6740 0x7f1f9c139460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34180/0 (socket says 172.21.15.190:34180) 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 1 -- 172.21.15.190:0/3109687722 learned_addr learned my addr 172.21.15.190:0/3109687722 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa0a05640 10 monclient(hunting): get_auth_request con 0x7f1f9c13eaa0 auth_method 0 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa0a05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa0a05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa0a05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 10 monclient(hunting): get_auth_request con 0x7f1f9c0a6740 auth_method 0 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa0a05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa0a05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa0a05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9b7fe640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9c0a6b20 0x7f1f9c0a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9b7fe640 10 monclient(hunting): get_auth_request con 0x7f1f9c0a6b20 auth_method 0 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.763 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 10 monclient(hunting): handle_auth_done global_id 15478 payload 824 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 1 -- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f9c13eaa0 msgr2=0x7f1f9c0a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f9c13eaa0 0x7f1f9c0a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 1 -- 172.21.15.190:0/3109687722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9c0a6b20 msgr2=0x7f1f9c0a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9c0a6b20 0x7f1f9c0a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 1 monclient: found mon.noname-i 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1f9bfff640 1 -- 172.21.15.190:0/3109687722 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c13db40 con 0x7f1f9c0a6740 2024-08-19T10:44:53.764 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.749+0000 7f1fa0a05640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f9c13eaa0 0x7f1f9c0a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:53.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f9bfff640 1 -- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c0a6740 msgr2=0x7f1f9c139460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f9bfff640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c0a6740 0x7f1f9c139460 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c450 tx=0x7f1f8c00c910 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 1 -- 172.21.15.190:0/3109687722 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f8c012cb0 con 0x7f1f9c0a6740 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 10 monclient: dump: 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.767 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 1 -- 172.21.15.190:0/3109687722 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c017070 con 0x7f1f9c0a6740 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1fa2468640 5 monclient: authenticate success, global_id 15478 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1fa2468640 10 monclient: _renew_subs 2024-08-19T10:44:53.768 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1fa2468640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1fa2468640 1 -- 172.21.15.190:0/3109687722 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c0ac340 con 0x7f1f9c0a6740 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 1 -- 172.21.15.190:0/3109687722 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1f8c0117f0 con 0x7f1f9c0a6740 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 10 monclient: dump: 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:53.769 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f9a7fc640 4 set_mon_vals no callback set 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1fa2468640 10 monclient: _renew_subs 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1fa2468640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1fa2468640 1 -- 172.21.15.190:0/3109687722 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c0a2320 con 0x7f1f9c0a6740 2024-08-19T10:44:53.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1fa2468640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1fa2468640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1fa2468640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:53.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1fa2468640 1 -- 172.21.15.190:0/3109687722 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f9c0a6f00 con 0x7f1f9c0a6740 2024-08-19T10:44:53.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 1 -- 172.21.15.190:0/3109687722 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1f8c011380 con 0x7f1f9c0a6740 2024-08-19T10:44:53.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1f8803be00 0x7f1f8803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f98ff9640 1 -- 172.21.15.190:0/3109687722 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(228..228 src has 1..228) ==== 14984+0+0 (secure 0 0 0) 0x7f1f8c050f50 con 0x7f1f9c0a6740 2024-08-19T10:44:53.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f9b7fe640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1f8803be00 0x7f1f8803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.753+0000 7f1f9b7fe640 10 monclient: get_auth_request con 0x7f1f8803be00 auth_method 0 2024-08-19T10:44:53.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.757+0000 7f1f9b7fe640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1f8803be00 0x7f1f8803e2c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0a0830 tx=0x7f1f90009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.761+0000 7f1f98ff9640 1 -- 172.21.15.190:0/3109687722 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1f8c04d160 con 0x7f1f9c0a6740 2024-08-19T10:44:53.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.761+0000 7f1f98ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.761+0000 7f1f98ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient: get_monmap_and_config 2024-08-19T10:44:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient: monmap: 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:53.818085+0000 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:53.818085+0000 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:53.818 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 1 Processor -- start 2024-08-19T10:44:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 1 -- start start 2024-08-19T10:44:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient: init 2024-08-19T10:44:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient: _add_conns ranks=[0,6,3,2,4,9,8,10,7,1,5] 2024-08-19T10:44:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f54900a67d0 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f54900a72c0 0x7f54900a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient(hunting): picked mon.noname-d con 0x7f54900a72c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f549013e9e0 0x7f5490140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient(hunting): picked mon.noname-g con 0x7f549013e9e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54900a8250 con 0x7f54900a67d0 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f54900a8740 con 0x7f549013e9e0 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f54900a3380 con 0x7f54900a72c0 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5496562640 10 monclient(hunting): authenticate will time out at 1343.634714s 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f54900a72c0 0x7f54900a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548ffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f54900a67d0 0x7f54900a6bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58342/0 (socket says 172.21.15.89:58342) 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548ffff640 1 -- 172.21.15.89:0/983710159 learned_addr learned my addr 172.21.15.89:0/983710159 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 10 monclient(hunting): get_auth_request con 0x7f54900a72c0 auth_method 0 2024-08-19T10:44:53.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5494aff640 1 --2- 172.21.15.89:0/983710159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f549013e9e0 0x7f5490140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5494aff640 10 monclient(hunting): get_auth_request con 0x7f549013e9e0 auth_method 0 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5494aff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5494aff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f5494aff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 10 monclient(hunting): handle_auth_done global_id 44187 payload 274 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 1 -- 172.21.15.89:0/983710159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f549013e9e0 msgr2=0x7f5490140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 1 --2- 172.21.15.89:0/983710159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f549013e9e0 0x7f5490140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 1 -- 172.21.15.89:0/983710159 >> v1:172.21.15.89:6789/0 conn(0x7f54900a67d0 legacy=0x7f54900a6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 1 monclient: found mon.noname-d 2024-08-19T10:44:53.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:44:53.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.813+0000 7f548f7fe640 1 -- 172.21.15.89:0/983710159 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54900a4cc0 con 0x7f54900a72c0 2024-08-19T10:44:53.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548f7fe640 1 -- 172.21.15.89:0/983710159 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f54900a72c0 msgr2=0x7f54900a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548f7fe640 1 --2- 172.21.15.89:0/983710159 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f54900a72c0 0x7f54900a5500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5480009dc0 tx=0x7f548001c880 comp rx=0 tx=0).ready entity=mon.6 client_cookie=fe944cc3fa2a53ed server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 1 -- 172.21.15.89:0/983710159 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54800a6940 con 0x7f54900a72c0 2024-08-19T10:44:53.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:44:53.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 10 monclient: dump: 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.828 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f5496562640 5 monclient: authenticate success, global_id 44187 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 1 -- 172.21.15.89:0/983710159 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54800a6b00 con 0x7f54900a72c0 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f5496562640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f5496562640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f5496562640 4 set_mon_vals no callback set 2024-08-19T10:44:53.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 1 -- 172.21.15.89:0/983710159 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54800a6cb0 con 0x7f54900a72c0 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.817+0000 7f548effd640 10 monclient: dump: 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.830 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient: shutdown 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 -- 172.21.15.89:0/983710159 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f54900a72c0 msgr2=0x7f54900a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 --2- 172.21.15.89:0/983710159 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f54900a72c0 0x7f54900a5500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5480009dc0 tx=0x7f548001c880 comp rx=0 tx=0).stop 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 -- 172.21.15.89:0/983710159 shutdown_connections 2024-08-19T10:44:53.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 --2- 172.21.15.89:0/983710159 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f54900a72c0 0x7f54900a5500 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 --2- 172.21.15.89:0/983710159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f549013e9e0 0x7f5490140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 -- 172.21.15.89:0/983710159 >> 172.21.15.89:0/983710159 conn(0x7f5490069ae0 msgr2=0x7f54900abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 -- 172.21.15.89:0/983710159 shutdown_connections 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 -- 172.21.15.89:0/983710159 wait complete. 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient: monmap: 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:53.832 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 Processor -- start 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 -- start start 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient: init 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient: _add_conns ranks=[8,2,0,4,5,7,3,1,6] 2024-08-19T10:44:53.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54900a67d0 0x7f5490125900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient(hunting): picked mon.noname-e con 0x7f54900a67d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54900a72c0 0x7f54901d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient(hunting): picked mon.noname-b con 0x7f54900a72c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549013e9e0 0x7f54901232b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient(hunting): picked mon.noname-a con 0x7f549013e9e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5490143ec0 con 0x7f549013e9e0 2024-08-19T10:44:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f5490143d40 con 0x7f54900a72c0 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f5490144040 con 0x7f54900a67d0 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5496562640 10 monclient(hunting): authenticate will time out at 1343.642177s 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54900a67d0 0x7f5490125900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54900a72c0 0x7f54901d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54900a67d0 0x7f5490125900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50566/0 (socket says 172.21.15.89:50566) 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54900a72c0 0x7f54901d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56428/0 (socket says 172.21.15.89:56428) 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 1 -- 172.21.15.89:0/2274717451 learned_addr learned my addr 172.21.15.89:0/2274717451 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 10 monclient(hunting): get_auth_request con 0x7f54900a67d0 auth_method 0 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548f7fe640 10 monclient(hunting): get_auth_request con 0x7f54900a72c0 auth_method 0 2024-08-19T10:44:53.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5494aff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549013e9e0 0x7f54901232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5494aff640 10 monclient(hunting): get_auth_request con 0x7f549013e9e0 auth_method 0 2024-08-19T10:44:53.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5494aff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5494aff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5494aff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5494aff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5494aff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f5494aff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 10 monclient(hunting): handle_auth_done global_id 6047 payload 824 2024-08-19T10:44:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 1 -- 172.21.15.89:0/2274717451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54900a72c0 msgr2=0x7f54901d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54900a72c0 0x7f54901d15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 1 -- 172.21.15.89:0/2274717451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549013e9e0 msgr2=0x7f54901232b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f549013e9e0 0x7f54901232b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 1 monclient: found mon.noname-e 2024-08-19T10:44:53.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:53.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548ffff640 1 -- 172.21.15.89:0/2274717451 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54900ac290 con 0x7f54900a67d0 2024-08-19T10:44:53.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.821+0000 7f548f7fe640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f54900a72c0 0x7f54901d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:53.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548ffff640 1 -- 172.21.15.89:0/2274717451 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54900a67d0 msgr2=0x7f5490125900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548ffff640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54900a67d0 0x7f5490125900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f548800c470 tx=0x7f548800c870 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5488017070 con 0x7f54900a67d0 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 10 monclient: dump: 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.845 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5496562640 5 monclient: authenticate success, global_id 6047 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5496562640 10 monclient: _renew_subs 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5488005d00 con 0x7f54900a67d0 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5496562640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:53.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5496562640 1 -- 172.21.15.89:0/2274717451 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5490126e20 con 0x7f54900a67d0 2024-08-19T10:44:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5488007540 con 0x7f54900a67d0 2024-08-19T10:44:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548e7fc640 4 set_mon_vals no callback set 2024-08-19T10:44:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 10 monclient: dump: 2024-08-19T10:44:53.847 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.848 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5496562640 10 monclient: _renew_subs 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5496562640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5496562640 1 -- 172.21.15.89:0/2274717451 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f549013db80 con 0x7f54900a67d0 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5496562640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5496562640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54901d1b00 0x7f549002d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5496562640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5496562640 1 -- 172.21.15.89:0/2274717451 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f54900a9320 con 0x7f54901d1b00 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5488007070 con 0x7f54900a67d0 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548f7fe640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54901d1b00 0x7f549002d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548f7fe640 10 monclient: get_auth_request con 0x7f54901d1b00 auth_method 0 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f547003be30 0x7f547003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(228..228 src has 1..228) ==== 14984+0+0 (secure 0 0 0) 0x7f5488050690 con 0x7f54900a67d0 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5494aff640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f547003be30 0x7f547003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f5494aff640 10 monclient: get_auth_request con 0x7f547003be30 auth_method 0 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548f7fe640 10 monclient(hunting): handle_auth_done global_id 15487 payload 824 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.825+0000 7f548f7fe640 1 -- 172.21.15.89:0/2274717451 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54900a5500 con 0x7f54901d1b00 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548f7fe640 1 -- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54901d1b00 msgr2=0x7f549002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548f7fe640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54901d1b00 0x7f549002d1d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f54800af000 tx=0x7f5480019680 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f5494aff640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f547003be30 0x7f547003e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5490125620 tx=0x7f5478008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54800be070 con 0x7f54901d1b00 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548cff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548cff9640 10 monclient: dump: 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.851 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f54800a6e20 con 0x7f54901d1b00 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548cff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54901d1b00 msgr2=0x7f549002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548cff9640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54901d1b00 0x7f549002d1d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f54800af000 tx=0x7f5480019680 comp rx=0 tx=0).stop 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54901d1b00 msgr2=0x7f549002d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:53.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.829+0000 7f548cff9640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f54901d1b00 0x7f549002d1d0 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f54800af000 tx=0x7f5480019680 comp rx=0 tx=0).stop 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f5496562640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f5496562640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f549002d710 0x7f549002db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f5496562640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f5496562640 1 -- 172.21.15.89:0/2274717451 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5490064590 con 0x7f549002d710 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f549002d710 0x7f549002db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 10 monclient: get_auth_request con 0x7f549002d710 auth_method 0 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 10 monclient(hunting): handle_auth_done global_id 15496 payload 824 2024-08-19T10:44:53.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 1 -- 172.21.15.89:0/2274717451 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5490126290 con 0x7f549002d710 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 1 -- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f549002d710 msgr2=0x7f549002db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548f7fe640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f549002d710 0x7f549002db20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f548001d120 tx=0x7f54800a9000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54800c8bf0 con 0x7f549002d710 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548cff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.837+0000 7f548cff9640 10 monclient: dump: 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.854 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f54800ae420 con 0x7f549002d710 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f548cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f548cff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f549002d710 msgr2=0x7f549002db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f548cff9640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f549002d710 0x7f549002db20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f548001d120 tx=0x7f54800a9000 comp rx=0 tx=0).stop 2024-08-19T10:44:53.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f548cff9640 1 -- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f549002d710 msgr2=0x7f549002db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:53.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f548cff9640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f549002d710 0x7f549002db20 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f548001d120 tx=0x7f54800a9000 comp rx=0 tx=0).stop 2024-08-19T10:44:53.856 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:53.856 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:53.856 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:53.856 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:53.856 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:53.856 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 271430, 2024-08-19T10:44:53.856 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:53.856 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:53.857 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.858 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.860 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:53.860 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:53.860 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:53.860 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:53.860 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:53.860 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:53.860 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:53.860 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:53.860 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:53.860 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:53.861 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:53.864 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:53.867 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.870 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:53.872 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.873 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.874 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.875 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.876 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.877 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:53.878 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:53.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:53.879 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:53.879 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:53.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5496562640 1 -- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f547003be30 msgr2=0x7f547003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5496562640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f547003be30 0x7f547003e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5490125620 tx=0x7f5478008040 comp rx=0 tx=0).stop 2024-08-19T10:44:53.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5496562640 10 monclient: shutdown 2024-08-19T10:44:53.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5496562640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:53.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5496562640 1 -- 172.21.15.89:0/2274717451 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54900a67d0 msgr2=0x7f5490125900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:53.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5496562640 1 --2- 172.21.15.89:0/2274717451 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54900a67d0 0x7f5490125900 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f548800c470 tx=0x7f548800c870 comp rx=0 tx=0).stop 2024-08-19T10:44:53.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5494aff640 1 -- 172.21.15.89:0/2274717451 reap_dead start 2024-08-19T10:44:53.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5494aff640 1 -- 172.21.15.89:0/2274717451 reap_dead start 2024-08-19T10:44:53.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5496562640 1 -- 172.21.15.89:0/2274717451 shutdown_connections 2024-08-19T10:44:53.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5496562640 1 -- 172.21.15.89:0/2274717451 >> 172.21.15.89:0/2274717451 conn(0x7f5490069ae0 msgr2=0x7f54900aae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:53.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5496562640 1 -- 172.21.15.89:0/2274717451 shutdown_connections 2024-08-19T10:44:53.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.841+0000 7f5496562640 1 -- 172.21.15.89:0/2274717451 wait complete. 2024-08-19T10:44:53.881 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:53.881 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-08-19T10:44:53.882 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-08-19T10:44:53.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.929+0000 7f1fa2468640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h1"}] 2024-08-19T10:44:53.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.929+0000 7f1fa2468640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h1"}] 2024-08-19T10:44:53.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.929+0000 7f1fa2468640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:53.933 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:53.929+0000 7f1fa2468640 1 -- 172.21.15.190:0/3109687722 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h1"} v 0) -- 0x7f1f9c123bf0 con 0x7f1f9c0a6740 2024-08-19T10:44:53.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient: get_monmap_and_config 2024-08-19T10:44:53.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient: build_initial_monmap 2024-08-19T10:44:53.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient: monmap: 2024-08-19T10:44:53.985 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:53.985 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.985 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:53.986560+0000 2024-08-19T10:44:53.985 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:53.986560+0000 2024-08-19T10:44:53.985 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:53.986 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 1 Processor -- start 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 1 -- start start 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient: init 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient: _add_conns ranks=[10,8,5,3,1,7,2,4,6,9,0] 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc60c0a6860 0x7fc60c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient(hunting): picked mon.noname-f con 0x7fc60c0a6860 addr v2:172.21.15.190:3300/0 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc60c0a7180 0x7fc60c0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient(hunting): picked mon.noname-e con 0x7fc60c0a7180 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc60c139620 0x7fc60c139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient(hunting): picked mon.noname-h con 0x7fc60c139620 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:53.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc60c0a8190 con 0x7fc60c0a6860 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc60c0a33d0 con 0x7fc60c139620 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc60c0a2ec0 con 0x7fc60c0a7180 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.981+0000 7fc612a73640 10 monclient(hunting): authenticate will time out at 1343.803826s 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc60c0a7180 0x7fc60c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc60c0a7180 0x7fc60c0a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50582/0 (socket says 172.21.15.89:50582) 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 1 -- 172.21.15.89:0/2831985879 learned_addr learned my addr 172.21.15.89:0/2831985879 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 10 monclient(hunting): get_auth_request con 0x7fc60c0a7180 auth_method 0 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:53.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:53.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:53.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc61080f640 1 --2- 172.21.15.89:0/2831985879 >> v2:172.21.15.190:3300/0 conn(0x7fc60c0a6860 0x7fc60c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc611010640 1 --2- 172.21.15.89:0/2831985879 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc60c139620 0x7fc60c139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:53.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc61080f640 10 monclient(hunting): get_auth_request con 0x7fc60c0a6860 auth_method 0 2024-08-19T10:44:53.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc61080f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:53.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc61080f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:53.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc61080f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 10 monclient(hunting): handle_auth_done global_id 6056 payload 274 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 1 -- 172.21.15.89:0/2831985879 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc60c139620 msgr2=0x7fc60c139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 1 --2- 172.21.15.89:0/2831985879 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc60c139620 0x7fc60c139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 1 -- 172.21.15.89:0/2831985879 >> v2:172.21.15.190:3300/0 conn(0x7fc60c0a6860 msgr2=0x7fc60c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 1 --2- 172.21.15.89:0/2831985879 >> v2:172.21.15.190:3300/0 conn(0x7fc60c0a6860 0x7fc60c0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 1 monclient: found mon.noname-e 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc60bfff640 1 -- 172.21.15.89:0/2831985879 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c0a4cd0 con 0x7fc60c0a7180 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc611010640 10 monclient: get_auth_request con 0x7fc60c139620 auth_method 0 2024-08-19T10:44:53.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.985+0000 7fc611010640 1 --2- 172.21.15.89:0/2831985879 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc60c139620 0x7fc60c139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:53.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60bfff640 1 -- 172.21.15.89:0/2831985879 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc60c0a7180 msgr2=0x7fc60c0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:53.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60bfff640 1 --2- 172.21.15.89:0/2831985879 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc60c0a7180 0x7fc60c0a7560 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc600009870 tx=0x7fc60001c420 comp rx=0 tx=0).ready entity=mon.8 client_cookie=766443a8079c483f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:53.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 1 -- 172.21.15.89:0/2831985879 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc6000b0070 con 0x7fc60c0a7180 2024-08-19T10:44:53.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:53.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 10 monclient: dump: 2024-08-19T10:44:53.996 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.996 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.996 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.996 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:44:53.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:44:53.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:53.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 1 -- 172.21.15.89:0/2831985879 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6000aa810 con 0x7fc60c0a7180 2024-08-19T10:44:53.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 5 monclient: authenticate success, global_id 6056 2024-08-19T10:44:53.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:53.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:53.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:53.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 4 set_mon_vals no callback set 2024-08-19T10:44:53.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 1 -- 172.21.15.89:0/2831985879 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc6000a9300 con 0x7fc60c0a7180 2024-08-19T10:44:53.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:53.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60b7fe640 10 monclient: dump: 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:53.999 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient: shutdown 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 -- 172.21.15.89:0/2831985879 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc60c0a7180 msgr2=0x7fc60c0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 --2- 172.21.15.89:0/2831985879 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc60c0a7180 0x7fc60c0a7560 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc600009870 tx=0x7fc60001c420 comp rx=0 tx=0).stop 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 -- 172.21.15.89:0/2831985879 shutdown_connections 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 --2- 172.21.15.89:0/2831985879 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc60c139620 0x7fc60c139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 --2- 172.21.15.89:0/2831985879 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc60c0a7180 0x7fc60c0a7560 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 --2- 172.21.15.89:0/2831985879 >> v2:172.21.15.190:3300/0 conn(0x7fc60c0a6860 0x7fc60c0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 -- 172.21.15.89:0/2831985879 >> 172.21.15.89:0/2831985879 conn(0x7fc60c074b40 msgr2=0x7fc60c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 -- 172.21.15.89:0/2831985879 shutdown_connections 2024-08-19T10:44:54.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 -- 172.21.15.89:0/2831985879 wait complete. 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient: build_initial_monmap 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient: monmap: 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:54.001 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 Processor -- start 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 -- start start 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient: init 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient: _add_conns ranks=[4,3,0,8,1,7,6,5,2] 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60c0a6860 0x7fc60c1d1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient(hunting): picked mon.noname-c con 0x7fc60c0a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc60c0a7180 0x7fc60c0a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:54.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient(hunting): picked mon.noname-g con 0x7fc60c0a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60c139620 0x7fc60c0a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient(hunting): picked mon.noname-a con 0x7fc60c139620 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c143210 con 0x7fc60c139620 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc60c143090 con 0x7fc60c0a7180 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc60c143390 con 0x7fc60c0a6860 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc612a73640 10 monclient(hunting): authenticate will time out at 1343.810670s 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60c0a6860 0x7fc60c1d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc611010640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60c139620 0x7fc60c0a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60c0a6860 0x7fc60c1d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36614/0 (socket says 172.21.15.89:36614) 2024-08-19T10:44:54.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 1 -- 172.21.15.89:0/2238469827 learned_addr learned my addr 172.21.15.89:0/2238469827 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc611010640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60c139620 0x7fc60c0a2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51962/0 (socket says 172.21.15.89:51962) 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 10 monclient(hunting): get_auth_request con 0x7fc60c0a6860 auth_method 0 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc611010640 10 monclient(hunting): get_auth_request con 0x7fc60c139620 auth_method 0 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc611010640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc611010640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc611010640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc611010640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:54.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc611010640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc611010640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60bfff640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc60c0a7180 0x7fc60c0a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 10 monclient(hunting): handle_auth_done global_id 44167 payload 824 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 1 -- 172.21.15.89:0/2238469827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc60c0a7180 msgr2=0x7fc60c0a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc60c0a7180 0x7fc60c0a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 1 -- 172.21.15.89:0/2238469827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60c139620 msgr2=0x7fc60c0a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60c139620 0x7fc60c0a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 10 monclient: _finish_hunting 0 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 1 monclient: found mon.noname-c 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc61080f640 1 -- 172.21.15.89:0/2238469827 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c0a2540 con 0x7fc60c0a6860 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc611010640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60c139620 0x7fc60c0a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:54.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60bfff640 10 monclient: get_auth_request con 0x7fc60c0a7180 auth_method 0 2024-08-19T10:44:54.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.989+0000 7fc60bfff640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc60c0a7180 0x7fc60c0a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:54.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc61080f640 1 -- 172.21.15.89:0/2238469827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60c0a6860 msgr2=0x7fc60c1d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:54.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc61080f640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60c0a6860 0x7fc60c1d1580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc007ef0 tx=0x7fc5fc00e4b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:54.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 1 -- 172.21.15.89:0/2238469827 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5fc013a00 con 0x7fc60c0a6860 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 10 monclient: dump: 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:54.008 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 10 monclient: _finish_auth 0 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 1 -- 172.21.15.89:0/2238469827 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5fc019070 con 0x7fc60c0a6860 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc612a73640 5 monclient: authenticate success, global_id 44167 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc612a73640 10 monclient: _renew_subs 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc612a73640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:44:54.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc612a73640 1 -- 172.21.15.89:0/2238469827 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c09e230 con 0x7fc60c0a6860 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 1 -- 172.21.15.89:0/2238469827 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5fc013c00 con 0x7fc60c0a6860 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 10 monclient: dump: 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:54.010 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc612a73640 10 monclient: _renew_subs 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc612a73640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc612a73640 1 -- 172.21.15.89:0/2238469827 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60c09fcf0 con 0x7fc60c0a6860 2024-08-19T10:44:54.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc60affd640 4 set_mon_vals no callback set 2024-08-19T10:44:54.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 1 -- 172.21.15.89:0/2238469827 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc5fc013590 con 0x7fc60c0a6860 2024-08-19T10:44:54.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc612a73640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:54.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc612a73640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:54.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc612a73640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:44:54.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc612a73640 1 -- 172.21.15.89:0/2238469827 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60c0ac270 con 0x7fc60c0a6860 2024-08-19T10:44:54.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc5f803be00 0x7fc5f803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:54.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc6097fa640 1 -- 172.21.15.89:0/2238469827 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(228..228 src has 1..228) ==== 14984+0+0 (secure 0 0 0) 0x7fc5fc051b40 con 0x7fc60c0a6860 2024-08-19T10:44:54.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc60bfff640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc5f803be00 0x7fc5f803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:54.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.993+0000 7fc60bfff640 10 monclient: get_auth_request con 0x7fc5f803be00 auth_method 0 2024-08-19T10:44:54.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.997+0000 7fc60bfff640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc5f803be00 0x7fc5f803e2c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc60c06cff0 tx=0x7fc6000a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:54.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.997+0000 7fc6097fa640 1 -- 172.21.15.89:0/2238469827 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc60c0ac270 con 0x7fc60c0a6860 2024-08-19T10:44:54.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.997+0000 7fc6097fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:54.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:53.997+0000 7fc6097fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:54.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.225+0000 7fc612a73640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-08-19T10:44:54.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.225+0000 7fc612a73640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-08-19T10:44:54.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.225+0000 7fc612a73640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:44:54.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.225+0000 7fc612a73640 1 -- 172.21.15.89:0/2238469827 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fc60c0a6c40 con 0x7fc60c0a6860 2024-08-19T10:44:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc6097fa640 1 -- 172.21.15.89:0/2238469827 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fc5fc04dd50 con 0x7fc60c0a6860 2024-08-19T10:44:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc6097fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-08-19T10:44:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc6097fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:54.234 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.234 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=472348434,config=2586702516,health=1258394751,kv=1101891974,logm=2638111592}) 2024-08-19T10:44:54.234 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=10,kv=10,logm=68} crc {auth=2258319453,config=2586702516,health=1517920900,kv=2412645021,logm=3363385850}) 2024-08-19T10:44:54.234 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.234 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=472348434,config=2586702516,health=1258394751,kv=1101891974,logm=2638111592}) 2024-08-19T10:44:54.234 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=3607783594,config=2586702516,health=1186785469,kv=1101891974,logm=4187126183}) 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=472348434,config=2586702516,health=1258394751,kv=1101891974,logm=2638111592}) 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=10,config=2,health=12,kv=9,logm=67} crc {auth=1525643151,config=2586702516,health=1258394751,kv=2672888601,logm=4120757658}) 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=472348434,config=2586702516,health=1258394751,kv=1101891974,logm=2638111592}) 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=11,config=2,health=12,kv=12,logm=63} crc {auth=2245386439,config=2586702516,health=1258394751,kv=1101891974,logm=3434045858}) 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=472348434,config=2586702516,health=1258394751,kv=1101891974,logm=2638111592}) 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=11,kv=11,logm=65} crc {auth=2245386439,config=2586702516,health=1083800488,kv=1523842694,logm=2102381630}) 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=472348434,config=2586702516,health=1258394751,kv=1101891974,logm=2638111592}) 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=11,config=2,health=11,kv=11,logm=65} crc {auth=2245386439,config=2586702516,health=49033560,kv=1481264099,logm=2808775074}) 2024-08-19T10:44:54.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=472348434,config=2586702516,health=1258394751,kv=1101891974,logm=2638111592}) 2024-08-19T10:44:54.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=2258319453,config=2586702516,health=1083800488,kv=1101891974,logm=3870856306}) 2024-08-19T10:44:54.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=472348434,config=2586702516,health=1258394751,kv=1101891974,logm=2638111592}) 2024-08-19T10:44:54.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.229+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=3607783594,config=2586702516,health=2619505822,kv=1494960071,logm=4279811521}) 2024-08-19T10:44:54.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 1 -- 172.21.15.89:0/2238469827 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc5f803be00 msgr2=0x7fc5f803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc5f803be00 0x7fc5f803e2c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc60c06cff0 tx=0x7fc6000a6000 comp rx=0 tx=0).stop 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 10 monclient: shutdown 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 1 -- 172.21.15.89:0/2238469827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60c0a6860 msgr2=0x7fc60c1d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60c0a6860 0x7fc60c1d1580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc007ef0 tx=0x7fc5fc00e4b0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 1 -- 172.21.15.89:0/2238469827 shutdown_connections 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc5f803be00 0x7fc5f803e2c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc60c139620 0x7fc60c0a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc60c0a7180 0x7fc60c0a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 1 --2- 172.21.15.89:0/2238469827 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc60c0a6860 0x7fc60c1d1580 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 1 -- 172.21.15.89:0/2238469827 >> 172.21.15.89:0/2238469827 conn(0x7fc60c074b40 msgr2=0x7fc60c0ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:54.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.229+0000 7fc612a73640 1 -- 172.21.15.89:0/2238469827 shutdown_connections 2024-08-19T10:44:54.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:54.233+0000 7fc612a73640 1 -- 172.21.15.89:0/2238469827 wait complete. 2024-08-19T10:44:54.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827491072}) 2024-08-19T10:44:54.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=540783386}) 2024-08-19T10:44:54.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827491072}) 2024-08-19T10:44:54.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1929176835}) 2024-08-19T10:44:54.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827491072}) 2024-08-19T10:44:54.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3152945361}) 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827491072}) 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2500595701}) 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827491072}) 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3924369443}) 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827491072}) 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2117465427}) 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827491072}) 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2290465049}) 2024-08-19T10:44:54.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.241 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827491072}) 2024-08-19T10:44:54.241 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.233+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1464545820}) 2024-08-19T10:44:54.242 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.242 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231901661}) 2024-08-19T10:44:54.242 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=383694601}) 2024-08-19T10:44:54.242 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.242 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231901661}) 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=458266233}) 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231901661}) 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1089700120}) 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231901661}) 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=163010023}) 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231901661}) 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2125513976}) 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231901661}) 2024-08-19T10:44:54.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1034926497}) 2024-08-19T10:44:54.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231901661}) 2024-08-19T10:44:54.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4067643973}) 2024-08-19T10:44:54.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=231901661}) 2024-08-19T10:44:54.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.237+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2568630024}) 2024-08-19T10:44:54.253 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-08-19T10:44:54.731 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.731 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114554859}) 2024-08-19T10:44:54.731 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837007527}) 2024-08-19T10:44:54.731 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.731 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114554859}) 2024-08-19T10:44:54.731 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=578962192}) 2024-08-19T10:44:54.731 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.731 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114554859}) 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=76463638}) 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114554859}) 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2764806994}) 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114554859}) 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=783965348}) 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114554859}) 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2970656364}) 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114554859}) 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=796013451}) 2024-08-19T10:44:54.732 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1114554859}) 2024-08-19T10:44:54.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.725+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2517608196}) 2024-08-19T10:44:54.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=103730180}) 2024-08-19T10:44:54.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2625461478}) 2024-08-19T10:44:54.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=103730180}) 2024-08-19T10:44:54.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=644787770}) 2024-08-19T10:44:54.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=103730180}) 2024-08-19T10:44:54.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4034550064}) 2024-08-19T10:44:54.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=103730180}) 2024-08-19T10:44:54.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2236148314}) 2024-08-19T10:44:54.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=103730180}) 2024-08-19T10:44:54.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3425327586}) 2024-08-19T10:44:54.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=103730180}) 2024-08-19T10:44:54.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3967325332}) 2024-08-19T10:44:54.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=103730180}) 2024-08-19T10:44:54.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4020363172}) 2024-08-19T10:44:54.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.750 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=103730180}) 2024-08-19T10:44:54.750 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.741+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3794719404}) 2024-08-19T10:44:54.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.757+0000 7f1f98ff9640 1 -- 172.21.15.190:0/3109687722 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h1"}]=0 removed item id -19 name 'h1' from crush map v229) ==== 118+0+0 (secure 0 0 0) 0x7f1f8c01c460 con 0x7f1f9c0a6740 2024-08-19T10:44:54.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.757+0000 7f1f98ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h1"}] 2024-08-19T10:44:54.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.757+0000 7f1f98ff9640 10 monclient: _finish_command 2 = system:0 removed item id -19 name 'h1' from crush map 2024-08-19T10:44:54.762 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.762 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465030004}) 2024-08-19T10:44:54.762 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1711277174}) 2024-08-19T10:44:54.762 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.762 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465030004}) 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1374683713}) 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465030004}) 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=7885644}) 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465030004}) 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2342097018}) 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465030004}) 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1899826581}) 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465030004}) 2024-08-19T10:44:54.763 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4001125062}) 2024-08-19T10:44:54.764 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.764 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465030004}) 2024-08-19T10:44:54.764 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2566799376}) 2024-08-19T10:44:54.764 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:54.764 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465030004}) 2024-08-19T10:44:54.764 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:54.757+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3686217928}) 2024-08-19T10:44:54.875 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.873+0000 7f1fa2468640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h1"}] 2024-08-19T10:44:54.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.873+0000 7f1fa2468640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h1"}] 2024-08-19T10:44:54.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.873+0000 7f1fa2468640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:54.876 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.873+0000 7f1fa2468640 1 -- 172.21.15.190:0/3109687722 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h1"} v 0) -- 0x7f1f9c1d1c40 con 0x7f1f9c0a6740 2024-08-19T10:44:54.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1f98ff9640 1 -- 172.21.15.190:0/3109687722 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h1"}]=0 device 'h1' does not appear in the crush map v229) ==== 118+0+0 (secure 0 0 0) 0x7f1f8c010d80 con 0x7f1f9c0a6740 2024-08-19T10:44:54.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1f98ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h1"}] 2024-08-19T10:44:54.878 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1f98ff9640 10 monclient: _finish_command 3 = system:0 device 'h1' does not appear in the crush map 2024-08-19T10:44:54.878 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h1' does not appear in the crush map 2024-08-19T10:44:54.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 -- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1f8803be00 msgr2=0x7f1f8803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:54.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1f8803be00 0x7f1f8803e2c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0a0830 tx=0x7f1f90009190 comp rx=0 tx=0).stop 2024-08-19T10:44:54.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 10 monclient: shutdown 2024-08-19T10:44:54.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:54.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 -- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c0a6740 msgr2=0x7f1f9c139460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:54.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c0a6740 0x7f1f9c139460 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c450 tx=0x7f1f8c00c910 comp rx=0 tx=0).stop 2024-08-19T10:44:54.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 -- 172.21.15.190:0/3109687722 shutdown_connections 2024-08-19T10:44:54.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1f8803be00 0x7f1f8803e2c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1f9c0a6b20 0x7f1f9c0a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f9c13eaa0 0x7f1f9c0a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 --2- 172.21.15.190:0/3109687722 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1f9c0a6740 0x7f1f9c139460 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:54.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 -- 172.21.15.190:0/3109687722 >> 172.21.15.190:0/3109687722 conn(0x7f1f9c0abd40 msgr2=0x7f1f9c13fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:54.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 -- 172.21.15.190:0/3109687722 shutdown_connections 2024-08-19T10:44:54.881 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:54.877+0000 7f1fa2468640 1 -- 172.21.15.190:0/3109687722 wait complete. 2024-08-19T10:44:54.897 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:54.897 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h2 2024-08-19T10:44:55.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient: get_monmap_and_config 2024-08-19T10:44:55.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient: build_initial_monmap 2024-08-19T10:44:55.011 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient: monmap: 2024-08-19T10:44:55.011 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:55.011 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.011 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:55.012907+0000 2024-08-19T10:44:55.011 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:55.012907+0000 2024-08-19T10:44:55.011 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:55.012 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:55.013 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:55.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 1 Processor -- start 2024-08-19T10:44:55.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 1 -- start start 2024-08-19T10:44:55.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient: init 2024-08-19T10:44:55.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:55.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient: _add_conns ranks=[9,7,3,8,6,1,4,10,5,2,0] 2024-08-19T10:44:55.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbc4c09b6a0 0x7fbc4c09ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient(hunting): picked mon.noname-a con 0x7fbc4c09b6a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:55.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c0a12b0 0x7fbc4c0a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient(hunting): picked mon.noname-i con 0x7fbc4c0a12b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc4c0a3320 0x7fbc4c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient(hunting): picked mon.noname-g con 0x7fbc4c0a3320 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fbc4c0a28e0 con 0x7fbc4c09b6a0 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbc4c0a0f00 con 0x7fbc4c0a3320 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fbc4c0a05f0 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc511cd640 10 monclient(hunting): authenticate will time out at 1340.583717s 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4b59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc4c0a3320 0x7fbc4c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.016 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4259c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c0a12b0 0x7fbc4c0a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4b59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc4c0a3320 0x7fbc4c0a37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:55594/0 (socket says 172.21.15.190:55594) 2024-08-19T10:44:55.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4b59e640 1 -- 172.21.15.190:0/4091661013 learned_addr learned my addr 172.21.15.190:0/4091661013 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:55.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4b59e640 10 monclient(hunting): get_auth_request con 0x7fbc4c0a3320 auth_method 0 2024-08-19T10:44:55.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4b59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4b59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4259c640 10 monclient(hunting): get_auth_request con 0x7fbc4c0a12b0 auth_method 0 2024-08-19T10:44:55.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.009+0000 7fbc4259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4ad9d640 1 --2- 172.21.15.190:0/4091661013 >> v2:172.21.15.89:3300/0 conn(0x7fbc4c09b6a0 0x7fbc4c09ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4ad9d640 10 monclient(hunting): get_auth_request con 0x7fbc4c09b6a0 auth_method 0 2024-08-19T10:44:55.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 10 monclient(hunting): handle_auth_done global_id 34626 payload 274 2024-08-19T10:44:55.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 1 -- 172.21.15.190:0/4091661013 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c0a12b0 msgr2=0x7fbc4c0a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:55.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 1 --2- 172.21.15.190:0/4091661013 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c0a12b0 0x7fbc4c0a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 1 -- 172.21.15.190:0/4091661013 >> v2:172.21.15.89:3300/0 conn(0x7fbc4c09b6a0 msgr2=0x7fbc4c09ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:55.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 1 --2- 172.21.15.190:0/4091661013 >> v2:172.21.15.89:3300/0 conn(0x7fbc4c09b6a0 0x7fbc4c09ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 10 monclient: _finish_hunting 0 2024-08-19T10:44:55.019 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 1 monclient: found mon.noname-g 2024-08-19T10:44:55.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:55.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:55.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.013+0000 7fbc4b59e640 1 -- 172.21.15.190:0/4091661013 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c0a4860 con 0x7fbc4c0a3320 2024-08-19T10:44:55.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4b59e640 1 -- 172.21.15.190:0/4091661013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc4c0a3320 msgr2=0x7fbc4c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:55.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4b59e640 1 --2- 172.21.15.190:0/4091661013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc4c0a3320 0x7fbc4c0a37a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009e80 tx=0x7fbc4401c960 comp rx=0 tx=0).ready entity=mon.3 client_cookie=416aff09a970def2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 1 -- 172.21.15.190:0/4091661013 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbc440ac020 con 0x7fbc4c0a3320 2024-08-19T10:44:55.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:55.025 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 10 monclient: dump: 2024-08-19T10:44:55.025 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:55.025 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.026 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 10 monclient: _finish_auth 0 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 10 monclient: _check_auth_tickets 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 5 monclient: authenticate success, global_id 34626 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 1 -- 172.21.15.190:0/4091661013 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc44009110 con 0x7fbc4c0a3320 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 4 set_mon_vals no callback set 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 1 -- 172.21.15.190:0/4091661013 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbc440a6d80 con 0x7fbc4c0a3320 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:55.027 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4a59c640 10 monclient: dump: 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.028 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient: shutdown 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 -- 172.21.15.190:0/4091661013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc4c0a3320 msgr2=0x7fbc4c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 --2- 172.21.15.190:0/4091661013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc4c0a3320 0x7fbc4c0a37a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009e80 tx=0x7fbc4401c960 comp rx=0 tx=0).stop 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 -- 172.21.15.190:0/4091661013 shutdown_connections 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 --2- 172.21.15.190:0/4091661013 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc4c0a3320 0x7fbc4c0a37a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 --2- 172.21.15.190:0/4091661013 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c0a12b0 0x7fbc4c0a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 --2- 172.21.15.190:0/4091661013 >> v2:172.21.15.89:3300/0 conn(0x7fbc4c09b6a0 0x7fbc4c09ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.029 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 -- 172.21.15.190:0/4091661013 >> 172.21.15.190:0/4091661013 conn(0x7fbc4c0a85b0 msgr2=0x7fbc4c0a89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 -- 172.21.15.190:0/4091661013 shutdown_connections 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 -- 172.21.15.190:0/4091661013 wait complete. 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient: build_initial_monmap 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient: monmap: 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:55.030 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 Processor -- start 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 -- start start 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient: init 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient: _add_conns ranks=[7,5,8,2,6,1,3,4,0] 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c09b6a0 0x7fbc4c09aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient(hunting): picked mon.noname-i con 0x7fbc4c09b6a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:55.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbc4c0a12b0 0x7fbc4c095370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient(hunting): picked mon.noname-h con 0x7fbc4c0a12b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:55.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbc4c0a3320 0x7fbc4c0958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient(hunting): picked mon.noname-e con 0x7fbc4c0a3320 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:55.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fbc4c0a4c60 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fbc4c0a4ae0 con 0x7fbc4c09b6a0 2024-08-19T10:44:55.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.032 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fbc4c142dc0 con 0x7fbc4c0a3320 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc511cd640 10 monclient(hunting): authenticate will time out at 1340.590781s 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4ad9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c09b6a0 0x7fbc4c09aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbc4c0a12b0 0x7fbc4c095370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4ad9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c09b6a0 0x7fbc4c09aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34210/0 (socket says 172.21.15.190:34210) 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4ad9d640 1 -- 172.21.15.190:0/4189817661 learned_addr learned my addr 172.21.15.190:0/4189817661 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4ad9d640 10 monclient(hunting): get_auth_request con 0x7fbc4c09b6a0 auth_method 0 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 10 monclient(hunting): get_auth_request con 0x7fbc4c0a12b0 auth_method 0 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.033 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4b59e640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbc4c0a3320 0x7fbc4c0958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 10 monclient(hunting): handle_auth_done global_id 6197 payload 824 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 1 -- 172.21.15.190:0/4189817661 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbc4c0a3320 msgr2=0x7fbc4c0958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbc4c0a3320 0x7fbc4c0958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.034 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 1 -- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c09b6a0 msgr2=0x7fbc4c09aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c09b6a0 0x7fbc4c09aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 10 monclient: _finish_hunting 0 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 1 monclient: found mon.noname-h 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4259c640 1 -- 172.21.15.190:0/4189817661 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c099190 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4b59e640 10 monclient: get_auth_request con 0x7fbc4c0a3320 auth_method 0 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4b59e640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbc4c0a3320 0x7fbc4c0958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.017+0000 7fbc4ad9d640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c09b6a0 0x7fbc4c09aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc4259c640 1 -- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbc4c0a12b0 msgr2=0x7fbc4c095370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc4259c640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbc4c0a12b0 0x7fbc4c095370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400e840 tx=0x7fbc3400ed00 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 1 -- 172.21.15.190:0/4189817661 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbc34017070 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.035 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 10 monclient: dump: 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.036 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc511cd640 5 monclient: authenticate success, global_id 6197 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc511cd640 10 monclient: _renew_subs 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc511cd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 1 -- 172.21.15.190:0/4189817661 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc34013470 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc511cd640 1 -- 172.21.15.190:0/4189817661 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4c0a5630 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.037 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 1 -- 172.21.15.190:0/4189817661 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbc34013a90 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 10 monclient: dump: 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.038 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc511cd640 10 monclient: _renew_subs 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc511cd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc49d9b640 4 set_mon_vals no callback set 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc511cd640 1 -- 172.21.15.190:0/4189817661 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4c098eb0 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc511cd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc511cd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:55.039 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc511cd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:55.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc511cd640 1 -- 172.21.15.190:0/4189817661 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc4c074450 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 1 -- 172.21.15.190:0/4189817661 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbc3401a070 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbc2c03be00 0x7fbc2c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc43fff640 1 -- 172.21.15.190:0/4189817661 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(229..229 src has 1..229) ==== 14774+0+0 (secure 0 0 0) 0x7fbc3400f040 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc4ad9d640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbc2c03be00 0x7fbc2c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.040 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.021+0000 7fbc4ad9d640 10 monclient: get_auth_request con 0x7fbc2c03be00 auth_method 0 2024-08-19T10:44:55.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.025+0000 7fbc4ad9d640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbc2c03be00 0x7fbc2c03e2c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c007f00 tx=0x7fbc3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.025+0000 7fbc43fff640 1 -- 172.21.15.190:0/4189817661 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fbc3404d5c0 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.025+0000 7fbc43fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:55.041 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.025+0000 7fbc43fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:55.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011473112}) 2024-08-19T10:44:55.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=645702442}) 2024-08-19T10:44:55.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011473112}) 2024-08-19T10:44:55.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=874620673}) 2024-08-19T10:44:55.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011473112}) 2024-08-19T10:44:55.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=959221980}) 2024-08-19T10:44:55.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011473112}) 2024-08-19T10:44:55.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2684622134}) 2024-08-19T10:44:55.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011473112}) 2024-08-19T10:44:55.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=459053589}) 2024-08-19T10:44:55.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011473112}) 2024-08-19T10:44:55.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3038514454}) 2024-08-19T10:44:55.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011473112}) 2024-08-19T10:44:55.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2290460987}) 2024-08-19T10:44:55.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011473112}) 2024-08-19T10:44:55.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.033+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=655217233}) 2024-08-19T10:44:55.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738066383}) 2024-08-19T10:44:55.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3017968249}) 2024-08-19T10:44:55.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738066383}) 2024-08-19T10:44:55.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3781612830}) 2024-08-19T10:44:55.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738066383}) 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2167285619}) 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738066383}) 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=351776726}) 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738066383}) 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2390673268}) 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738066383}) 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=796710114}) 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738066383}) 2024-08-19T10:44:55.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.037+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2625303050}) 2024-08-19T10:44:55.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738066383}) 2024-08-19T10:44:55.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2140448378}) 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=933297701}) 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=640072623}) 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=933297701}) 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=924710675}) 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=933297701}) 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1832337800}) 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.041+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=933297701}) 2024-08-19T10:44:55.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=772677230}) 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=933297701}) 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3724267973}) 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=933297701}) 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1713524677}) 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=933297701}) 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3739292366}) 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=933297701}) 2024-08-19T10:44:55.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.045+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3963979703}) 2024-08-19T10:44:55.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.201+0000 7fbc511cd640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h2"}] 2024-08-19T10:44:55.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.201+0000 7fbc511cd640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h2"}] 2024-08-19T10:44:55.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.201+0000 7fbc511cd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:55.205 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.201+0000 7fbc511cd640 1 -- 172.21.15.190:0/4189817661 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h2"} v 0) -- 0x7fbc4c0a2de0 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189638305}) 2024-08-19T10:44:55.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3725456008}) 2024-08-19T10:44:55.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189638305}) 2024-08-19T10:44:55.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3780431177}) 2024-08-19T10:44:55.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189638305}) 2024-08-19T10:44:55.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4137662964}) 2024-08-19T10:44:55.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189638305}) 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2800990401}) 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189638305}) 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=842585071}) 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189638305}) 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=940609459}) 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189638305}) 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4035458004}) 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189638305}) 2024-08-19T10:44:55.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1555307635}) 2024-08-19T10:44:55.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604107628}) 2024-08-19T10:44:55.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1449492447}) 2024-08-19T10:44:55.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604107628}) 2024-08-19T10:44:55.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2480414917}) 2024-08-19T10:44:55.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604107628}) 2024-08-19T10:44:55.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1090041657}) 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604107628}) 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3359911893}) 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604107628}) 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1715459373}) 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604107628}) 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=49392985}) 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604107628}) 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1198620207}) 2024-08-19T10:44:55.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604107628}) 2024-08-19T10:44:55.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.225+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1713597719}) 2024-08-19T10:44:55.254 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:44:55.254 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:44:55.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.373+0000 7f5c6577c640 10 monclient: get_monmap_and_config 2024-08-19T10:44:55.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.373+0000 7f5c6577c640 10 monclient: build_initial_monmap 2024-08-19T10:44:55.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.373+0000 7f5c6577c640 10 monclient: monmap: 2024-08-19T10:44:55.378 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:55.378 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.378 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:55.379226+0000 2024-08-19T10:44:55.378 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:55.379226+0000 2024-08-19T10:44:55.378 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:55.378 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.378 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:55.378 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:55.378 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:55.379 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:55.379 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:55.379 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:55.379 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:55.379 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:55.379 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:55.379 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:55.379 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:55.379 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.373+0000 7f5c6577c640 1 Processor -- start 2024-08-19T10:44:55.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.373+0000 7f5c6577c640 1 -- start start 2024-08-19T10:44:55.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.373+0000 7f5c6577c640 10 monclient: init 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.373+0000 7f5c6577c640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.373+0000 7f5c6577c640 10 monclient: _add_conns ranks=[5,1,10,6,0,9,3,8,4,2,7] 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.373+0000 7f5c6577c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a5270 0x7f5c600a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.373+0000 7f5c6577c640 10 monclient(hunting): picked mon.noname-h con 0x7f5c600a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c6577c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5c600a6d30 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c6577c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5c600a7110 0x7f5c60139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c6577c640 10 monclient(hunting): picked mon.noname-f con 0x7f5c600a7110 addr v2:172.21.15.190:3300/0 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c6577c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c600a8710 con 0x7f5c600a6d30 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c6577c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c6577c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f5c600a5a40 con 0x7f5c600a7110 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c6577c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c6577c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f5c600a81d0 con 0x7f5c600a5270 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c6577c640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c6577c640 10 monclient(hunting): authenticate will time out at 1345.196089s 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5c600a7110 0x7f5c60139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5effd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a5270 0x7f5c600a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5e7fc640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f5c600a6d30 0x7f5c601395b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.89:49242/0 (socket says 172.21.15.89:49242) 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5e7fc640 1 -- 172.21.15.89:0/1030335763 learned_addr learned my addr 172.21.15.89:0/1030335763 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5effd640 10 monclient(hunting): get_auth_request con 0x7f5c600a5270 auth_method 0 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 10 monclient(hunting): get_auth_request con 0x7f5c600a7110 auth_method 0 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 10 monclient(hunting): handle_auth_done global_id 44137 payload 274 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 1 -- 172.21.15.89:0/1030335763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a5270 msgr2=0x7f5c600a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 1 --2- 172.21.15.89:0/1030335763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a5270 0x7f5c600a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 1 -- 172.21.15.89:0/1030335763 >> v1:172.21.15.190:6789/0 conn(0x7f5c600a6d30 legacy=0x7f5c601395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 1 monclient: found mon.noname-f 2024-08-19T10:44:55.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:44:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5f7fe640 1 -- 172.21.15.89:0/1030335763 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c600a4790 con 0x7f5c600a7110 2024-08-19T10:44:55.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.377+0000 7f5c5effd640 1 --2- 172.21.15.89:0/1030335763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a5270 0x7f5c600a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5f7fe640 1 -- 172.21.15.89:0/1030335763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c600a7110 msgr2=0x7f5c60139cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5f7fe640 1 --2- 172.21.15.89:0/1030335763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c600a7110 0x7f5c60139cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5c580097c0 tx=0x7f5c5801c150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=942bbe9d3731925 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 1 -- 172.21.15.89:0/1030335763 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c580b0070 con 0x7f5c600a7110 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 10 monclient: dump: 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.389 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:55.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 5 monclient: authenticate success, global_id 44137 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 1 -- 172.21.15.89:0/1030335763 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c580aa3a0 con 0x7f5c600a7110 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 4 set_mon_vals no callback set 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 1 -- 172.21.15.89:0/1030335763 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c580aaa40 con 0x7f5c600a7110 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c5dffb640 10 monclient: dump: 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.391 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.392 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 10 monclient: shutdown 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 1 -- 172.21.15.89:0/1030335763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c600a7110 msgr2=0x7f5c60139cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 1 --2- 172.21.15.89:0/1030335763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c600a7110 0x7f5c60139cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5c580097c0 tx=0x7f5c5801c150 comp rx=0 tx=0).stop 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 1 -- 172.21.15.89:0/1030335763 shutdown_connections 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 1 --2- 172.21.15.89:0/1030335763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c600a7110 0x7f5c60139cc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 1 --2- 172.21.15.89:0/1030335763 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a5270 0x7f5c600a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 1 -- 172.21.15.89:0/1030335763 >> 172.21.15.89:0/1030335763 conn(0x7f5c60069a80 msgr2=0x7f5c600abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 1 -- 172.21.15.89:0/1030335763 shutdown_connections 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 1 -- 172.21.15.89:0/1030335763 wait complete. 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 10 monclient: build_initial_monmap 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 10 monclient: monmap: 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:55.393 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:55.394 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 1 Processor -- start 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 1 -- start start 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.381+0000 7f5c6577c640 10 monclient: init 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 10 monclient: _add_conns ranks=[3,5,7,6,2,1,4,8,0] 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c600a5270 0x7f5c601387b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 10 monclient(hunting): picked mon.noname-g con 0x7f5c600a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a6d30 0x7f5c60138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 10 monclient(hunting): picked mon.noname-h con 0x7f5c600a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c600a7110 0x7f5c6013e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 10 monclient(hunting): picked mon.noname-i con 0x7f5c600a7110 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:55.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5c600a57d0 con 0x7f5c600a5270 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f5c600a5650 con 0x7f5c600a6d30 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5c6013a200 con 0x7f5c600a7110 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c6577c640 10 monclient(hunting): authenticate will time out at 1345.204146s 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5e7fc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a6d30 0x7f5c60138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c600a5270 0x7f5c601387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c600a7110 0x7f5c6013e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5e7fc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a6d30 0x7f5c60138cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:35416/0 (socket says 172.21.15.89:35416) 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5e7fc640 1 -- 172.21.15.89:0/1368051161 learned_addr learned my addr 172.21.15.89:0/1368051161 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:55.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5e7fc640 10 monclient(hunting): get_auth_request con 0x7f5c600a6d30 auth_method 0 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 10 monclient(hunting): get_auth_request con 0x7f5c600a7110 auth_method 0 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5effd640 10 monclient(hunting): get_auth_request con 0x7f5c600a5270 auth_method 0 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 10 monclient(hunting): handle_auth_done global_id 15523 payload 824 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 1 -- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a6d30 msgr2=0x7f5c60138cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a6d30 0x7f5c60138cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 1 -- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c600a5270 msgr2=0x7f5c601387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c600a5270 0x7f5c601387b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:55.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 1 monclient: found mon.noname-i 2024-08-19T10:44:55.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:55.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:55.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 1 -- 172.21.15.89:0/1368051161 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c60123600 con 0x7f5c600a7110 2024-08-19T10:44:55.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 1 -- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c600a7110 msgr2=0x7f5c6013e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:55.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.385+0000 7f5c5f7fe640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c600a7110 0x7f5c6013e260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5c580ab000 tx=0x7f5c58018ef0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 1 -- 172.21.15.89:0/1368051161 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c580b0050 con 0x7f5c600a7110 2024-08-19T10:44:55.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:55.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 10 monclient: dump: 2024-08-19T10:44:55.401 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.401 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.401 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.401 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 10 monclient: _finish_auth 0 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 10 monclient: _check_auth_tickets 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c6577c640 5 monclient: authenticate success, global_id 15523 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c6577c640 10 monclient: _renew_subs 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c6577c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c6577c640 1 -- 172.21.15.89:0/1368051161 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c601238e0 con 0x7f5c600a7110 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c6577c640 10 monclient: _renew_subs 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c6577c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c6577c640 1 -- 172.21.15.89:0/1368051161 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c600a1ed0 con 0x7f5c600a7110 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 1 -- 172.21.15.89:0/1368051161 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c580b1040 con 0x7f5c600a7110 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 1 -- 172.21.15.89:0/1368051161 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c580aa550 con 0x7f5c600a7110 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c5d7fa640 4 set_mon_vals no callback set 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 10 monclient: dump: 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c6577c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c6577c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c6577c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c6577c640 1 -- 172.21.15.89:0/1368051161 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c44005090 con 0x7f5c600a7110 2024-08-19T10:44:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 1 -- 172.21.15.89:0/1368051161 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5c580be030 con 0x7f5c600a7110 2024-08-19T10:44:55.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c4003be30 0x7f5c4003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c577fe640 1 -- 172.21.15.89:0/1368051161 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(229..229 src has 1..229) ==== 14774+0+0 (secure 0 0 0) 0x7f5c580bc020 con 0x7f5c600a7110 2024-08-19T10:44:55.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c5effd640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c4003be30 0x7f5c4003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.389+0000 7f5c5effd640 10 monclient: get_auth_request con 0x7f5c4003be30 auth_method 0 2024-08-19T10:44:55.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.393+0000 7f5c5effd640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c4003be30 0x7f5c4003e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009580 tx=0x7f5c50009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.401+0000 7f5c577fe640 1 -- 172.21.15.89:0/1368051161 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f5c580c5020 con 0x7f5c600a7110 2024-08-19T10:44:55.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.401+0000 7f5c577fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:55.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.401+0000 7f5c577fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:55.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.637+0000 7f5c6577c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:44:55.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.637+0000 7f5c6577c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:44:55.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.637+0000 7f5c6577c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:55.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.637+0000 7f5c6577c640 1 -- 172.21.15.89:0/1368051161 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5c44005650 con 0x7f5c600a7110 2024-08-19T10:44:55.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.637+0000 7f5c577fe640 1 -- 172.21.15.89:0/1368051161 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7f5c600a8710 con 0x7f5c600a7110 2024-08-19T10:44:55.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.637+0000 7f5c577fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:44:55.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.637+0000 7f5c577fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:55.641 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":72,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:44:55.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 -- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c4003be30 msgr2=0x7f5c4003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c4003be30 0x7f5c4003e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009580 tx=0x7f5c50009190 comp rx=0 tx=0).stop 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 10 monclient: shutdown 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 -- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c600a7110 msgr2=0x7f5c6013e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c600a7110 0x7f5c6013e260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5c580ab000 tx=0x7f5c58018ef0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 -- 172.21.15.89:0/1368051161 shutdown_connections 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c4003be30 0x7f5c4003e2f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c600a6d30 0x7f5c60138cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c600a7110 0x7f5c6013e260 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 --2- 172.21.15.89:0/1368051161 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c600a5270 0x7f5c601387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 -- 172.21.15.89:0/1368051161 >> 172.21.15.89:0/1368051161 conn(0x7f5c60069a80 msgr2=0x7f5c600aa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 -- 172.21.15.89:0/1368051161 shutdown_connections 2024-08-19T10:44:55.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.641+0000 7f5c6577c640 1 -- 172.21.15.89:0/1368051161 wait complete. 2024-08-19T10:44:55.662 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:44:55.662 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-08-19T10:44:55.662 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:44:55.662 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:44:55.740 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.740 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1773032515}) 2024-08-19T10:44:55.740 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2060202679}) 2024-08-19T10:44:55.740 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.740 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1773032515}) 2024-08-19T10:44:55.740 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2951744019}) 2024-08-19T10:44:55.740 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.740 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1773032515}) 2024-08-19T10:44:55.740 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3066274240}) 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1773032515}) 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=281982634}) 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1773032515}) 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2245413214}) 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1773032515}) 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3811693654}) 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1773032515}) 2024-08-19T10:44:55.741 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2213671386}) 2024-08-19T10:44:55.742 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.742 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1773032515}) 2024-08-19T10:44:55.742 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.733+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3494823500}) 2024-08-19T10:44:55.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=976138479}) 2024-08-19T10:44:55.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=503959124}) 2024-08-19T10:44:55.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=976138479}) 2024-08-19T10:44:55.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2263941279}) 2024-08-19T10:44:55.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=976138479}) 2024-08-19T10:44:55.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1047982911}) 2024-08-19T10:44:55.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=976138479}) 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3925735996}) 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=976138479}) 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3663427167}) 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=976138479}) 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2618976399}) 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=976138479}) 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3897570692}) 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.745+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=976138479}) 2024-08-19T10:44:55.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3885775885}) 2024-08-19T10:44:55.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.785+0000 7efcbe8c9640 10 monclient: get_monmap_and_config 2024-08-19T10:44:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.785+0000 7efcbe8c9640 10 monclient: build_initial_monmap 2024-08-19T10:44:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.785+0000 7efcbe8c9640 10 monclient: monmap: 2024-08-19T10:44:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:55.792296+0000 2024-08-19T10:44:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:55.792296+0000 2024-08-19T10:44:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:55.791 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:55.792 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:55.792 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 1 Processor -- start 2024-08-19T10:44:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 1 -- start start 2024-08-19T10:44:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 10 monclient: init 2024-08-19T10:44:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 10 monclient: _add_conns ranks=[0,8,6,5,3,1,7,4,9,10,2] 2024-08-19T10:44:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efcb8139940 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb806e1b0 0x7efcb80a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 10 monclient(hunting): picked mon.noname-e con 0x7efcb806e1b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcb813e860 0x7efcb8140c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 10 monclient(hunting): picked mon.noname-d con 0x7efcb813e860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb806f8b0 con 0x7efcb8139940 2024-08-19T10:44:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7efcb806df70 con 0x7efcb813e860 2024-08-19T10:44:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7efcb806f2f0 con 0x7efcb806e1b0 2024-08-19T10:44:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbe8c9640 10 monclient(hunting): authenticate will time out at 1345.608764s 2024-08-19T10:44:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb806e1b0 0x7efcb80a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcb813e860 0x7efcb8140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb806e1b0 0x7efcb80a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50590/0 (socket says 172.21.15.89:50590) 2024-08-19T10:44:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcaf7fe640 1 -- 172.21.15.89:0/3232071475 learned_addr learned my addr 172.21.15.89:0/3232071475 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcb7fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efcb8139940 0x7efcb8139d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58346/0 (socket says 172.21.15.89:58346) 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcaf7fe640 10 monclient(hunting): get_auth_request con 0x7efcb806e1b0 auth_method 0 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcaf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 10 monclient(hunting): get_auth_request con 0x7efcb813e860 auth_method 0 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 10 monclient(hunting): handle_auth_done global_id 44196 payload 274 2024-08-19T10:44:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 1 -- 172.21.15.89:0/3232071475 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb806e1b0 msgr2=0x7efcb80a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 1 --2- 172.21.15.89:0/3232071475 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb806e1b0 0x7efcb80a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 1 -- 172.21.15.89:0/3232071475 >> v1:172.21.15.89:6789/0 conn(0x7efcb8139940 legacy=0x7efcb8139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 10 monclient: _finish_hunting 0 2024-08-19T10:44:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 1 monclient: found mon.noname-d 2024-08-19T10:44:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:44:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcbce66640 1 -- 172.21.15.89:0/3232071475 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb80a66f0 con 0x7efcb813e860 2024-08-19T10:44:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.789+0000 7efcaf7fe640 1 --2- 172.21.15.89:0/3232071475 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb806e1b0 0x7efcb80a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbce66640 1 -- 172.21.15.89:0/3232071475 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcb813e860 msgr2=0x7efcb8140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbce66640 1 --2- 172.21.15.89:0/3232071475 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcb813e860 0x7efcb8140c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009dc0 tx=0x7efcb001c850 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d5029863e6bea3e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 1 -- 172.21.15.89:0/3232071475 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efcb00b2070 con 0x7efcb813e860 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 10 monclient: dump: 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 10 monclient: _finish_auth 0 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:44:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 1 -- 172.21.15.89:0/3232071475 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb00a59b0 con 0x7efcb813e860 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 5 monclient: authenticate success, global_id 44196 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 4 set_mon_vals no callback set 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 1 -- 172.21.15.89:0/3232071475 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efcb00a5ba0 con 0x7efcb813e860 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb77fe640 10 monclient: dump: 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient: shutdown 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 -- 172.21.15.89:0/3232071475 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcb813e860 msgr2=0x7efcb8140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 --2- 172.21.15.89:0/3232071475 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcb813e860 0x7efcb8140c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009dc0 tx=0x7efcb001c850 comp rx=0 tx=0).stop 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 -- 172.21.15.89:0/3232071475 shutdown_connections 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 --2- 172.21.15.89:0/3232071475 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcb813e860 0x7efcb8140c50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 --2- 172.21.15.89:0/3232071475 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb806e1b0 0x7efcb80a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 -- 172.21.15.89:0/3232071475 >> 172.21.15.89:0/3232071475 conn(0x7efcb80aa240 msgr2=0x7efcb80aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 -- 172.21.15.89:0/3232071475 shutdown_connections 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 -- 172.21.15.89:0/3232071475 wait complete. 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient: build_initial_monmap 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient: monmap: 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:55.811 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 Processor -- start 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 -- start start 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient: init 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient: _add_conns ranks=[0,2,8,1,4,7,3,5,6] 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb806e1b0 0x7efcb81d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient(hunting): picked mon.noname-a con 0x7efcb806e1b0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcb8139940 0x7efcb8135800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient(hunting): picked mon.noname-b con 0x7efcb8139940 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb813e860 0x7efcb813a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient(hunting): picked mon.noname-e con 0x7efcb813e860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8143d40 con 0x7efcb806e1b0 2024-08-19T10:44:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7efcb8143bc0 con 0x7efcb8139940 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7efcb8143ec0 con 0x7efcb813e860 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbe8c9640 10 monclient(hunting): authenticate will time out at 1345.615671s 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb806e1b0 0x7efcb81d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcaf7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcb8139940 0x7efcb8135800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb806e1b0 0x7efcb81d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51964/0 (socket says 172.21.15.89:51964) 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbce66640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb813e860 0x7efcb813a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb7fff640 1 -- 172.21.15.89:0/434500362 learned_addr learned my addr 172.21.15.89:0/434500362 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb7fff640 10 monclient(hunting): get_auth_request con 0x7efcb806e1b0 auth_method 0 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcaf7fe640 10 monclient(hunting): get_auth_request con 0x7efcb8139940 auth_method 0 2024-08-19T10:44:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcaf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbce66640 10 monclient(hunting): get_auth_request con 0x7efcb813e860 auth_method 0 2024-08-19T10:44:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbce66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbce66640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbce66640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbce66640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbce66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcbce66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.793+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 10 monclient(hunting): handle_auth_done global_id 6264 payload 824 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 1 -- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb813e860 msgr2=0x7efcb813a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb813e860 0x7efcb813a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 1 -- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcb8139940 msgr2=0x7efcb8135800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcb8139940 0x7efcb8135800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 1 monclient: found mon.noname-a 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 1 -- 172.21.15.89:0/434500362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb813e500 con 0x7efcb806e1b0 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbce66640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcb813e860 0x7efcb813a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcaf7fe640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7efcb8139940 0x7efcb8135800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 1 -- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb806e1b0 msgr2=0x7efcb81d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb7fff640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb806e1b0 0x7efcb81d0fb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efca800c460 tx=0x7efca800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efca8017070 con 0x7efcb806e1b0 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 10 monclient: dump: 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 10 monclient: _finish_auth 0 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbe8c9640 5 monclient: authenticate success, global_id 6264 2024-08-19T10:44:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbe8c9640 10 monclient: _renew_subs 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca8005c90 con 0x7efcb806e1b0 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbe8c9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbe8c9640 1 -- 172.21.15.89:0/434500362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb81d1ea0 con 0x7efcb806e1b0 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efca8005e40 con 0x7efcb806e1b0 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb6ffd640 4 set_mon_vals no callback set 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 10 monclient: dump: 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbe8c9640 10 monclient: _renew_subs 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbe8c9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbe8c9640 1 -- 172.21.15.89:0/434500362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb813c1a0 con 0x7efcb806e1b0 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbe8c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbe8c9640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb8135d40 0x7efcb813ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbe8c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcbe8c9640 1 -- 172.21.15.89:0/434500362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8072a40 con 0x7efcb8135d40 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcaf7fe640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb8135d40 0x7efcb813ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcaf7fe640 10 monclient: get_auth_request con 0x7efcb8135d40 auth_method 0 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcaf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efca8007490 con 0x7efcb806e1b0 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_done global_id 6273 payload 824 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcaf7fe640 1 -- 172.21.15.89:0/434500362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efcb8139d20 con 0x7efcb8135d40 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efc9803be00 0x7efc9803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.797+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(229..229 src has 1..229) ==== 14774+0+0 (secure 0 0 0) 0x7efca80504a0 con 0x7efcb806e1b0 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcbce66640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efc9803be00 0x7efc9803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcbce66640 10 monclient: get_auth_request con 0x7efc9803be00 auth_method 0 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcaf7fe640 1 -- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb8135d40 msgr2=0x7efcb813ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcaf7fe640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb8135d40 0x7efcb813ae30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efcb81365c0 tx=0x7efca000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efca0014ac0 con 0x7efcb8135d40 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcb57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcb57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcb57fa640 10 monclient: dump: 2024-08-19T10:44:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.824 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efca0014650 con 0x7efcb8135d40 2024-08-19T10:44:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcb57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcb57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb8135d40 msgr2=0x7efcb813ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcb57fa640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb8135d40 0x7efcb813ae30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efcb81365c0 tx=0x7efca000f610 comp rx=0 tx=0).stop 2024-08-19T10:44:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb8135d40 msgr2=0x7efcb813ae30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcb57fa640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb8135d40 0x7efcb813ae30 secure :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efcb81365c0 tx=0x7efca000f610 comp rx=0 tx=0).stop 2024-08-19T10:44:55.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.801+0000 7efcbce66640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efc9803be00 0x7efc9803e2c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efcb8073bb0 tx=0x7efcb00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:55.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb813dd80 0x7efcb813b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 1 -- 172.21.15.89:0/434500362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8064590 con 0x7efcb813dd80 2024-08-19T10:44:55.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb813dd80 0x7efcb813b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 10 monclient: get_auth_request con 0x7efcb813dd80 auth_method 0 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 10 monclient(hunting): handle_auth_done global_id 6282 payload 824 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 1 -- 172.21.15.89:0/434500362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efcb813cd00 con 0x7efcb813dd80 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 1 -- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb813dd80 msgr2=0x7efcb813b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcaf7fe640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb813dd80 0x7efcb813b370 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efca001c000 tx=0x7efca00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efca0022c40 con 0x7efcb813dd80 2024-08-19T10:44:55.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcb57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcb57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcb57fa640 10 monclient: dump: 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.828 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.829 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9269 (secure 0 0 0) 0x7efca0022c40 con 0x7efcb813dd80 2024-08-19T10:44:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcb57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcb57fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb813dd80 msgr2=0x7efcb813b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcb57fa640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb813dd80 0x7efcb813b370 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efca001c000 tx=0x7efca00046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcb57fa640 1 -- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb813dd80 msgr2=0x7efcb813b370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcb57fa640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb813dd80 0x7efcb813b370 secure :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efca001c000 tx=0x7efca00046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 425622, 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:55.852 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:55.853 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:55.853 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:44:55.853 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:55.853 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:55.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:55.853 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:55.853 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:55.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.841+0000 7fbc43fff640 1 -- 172.21.15.190:0/4189817661 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h2"}]=0 removed item id -22 name 'h2' from crush map v230) ==== 118+0+0 (secure 0 0 0) 0x7fbc34013c30 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.841+0000 7fbc43fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h2"}] 2024-08-19T10:44:55.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.841+0000 7fbc43fff640 10 monclient: _finish_command 2 = system:0 removed item id -22 name 'h2' from crush map 2024-08-19T10:44:55.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 1 -- 172.21.15.89:0/434500362 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efc9803be00 msgr2=0x7efc9803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efc9803be00 0x7efc9803e2c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efcb8073bb0 tx=0x7efcb00a7070 comp rx=0 tx=0).stop 2024-08-19T10:44:55.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 10 monclient: shutdown 2024-08-19T10:44:55.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:55.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 1 -- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb806e1b0 msgr2=0x7efcb81d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 1 --2- 172.21.15.89:0/434500362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcb806e1b0 0x7efcb81d0fb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efca800c460 tx=0x7efca800c920 comp rx=0 tx=0).stop 2024-08-19T10:44:55.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbce66640 1 -- 172.21.15.89:0/434500362 reap_dead start 2024-08-19T10:44:55.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbce66640 1 -- 172.21.15.89:0/434500362 reap_dead start 2024-08-19T10:44:55.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 1 -- 172.21.15.89:0/434500362 shutdown_connections 2024-08-19T10:44:55.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 1 -- 172.21.15.89:0/434500362 >> 172.21.15.89:0/434500362 conn(0x7efcb80aa240 msgr2=0x7efcb8138810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:55.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 1 -- 172.21.15.89:0/434500362 shutdown_connections 2024-08-19T10:44:55.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.809+0000 7efcbe8c9640 1 -- 172.21.15.89:0/434500362 wait complete. 2024-08-19T10:44:55.856 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:55.856 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:44:55.857 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:44:55.861 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.861 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823931878}) 2024-08-19T10:44:55.861 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2814623719}) 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823931878}) 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=37485054}) 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823931878}) 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3440315296}) 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823931878}) 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3356098073}) 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823931878}) 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3848352741}) 2024-08-19T10:44:55.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823931878}) 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4016670640}) 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823931878}) 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3892578155}) 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2823931878}) 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.829+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4192732938}) 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186931885}) 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2666257635}) 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186931885}) 2024-08-19T10:44:55.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4249400356}) 2024-08-19T10:44:55.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186931885}) 2024-08-19T10:44:55.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=7189309}) 2024-08-19T10:44:55.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186931885}) 2024-08-19T10:44:55.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1802652138}) 2024-08-19T10:44:55.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186931885}) 2024-08-19T10:44:55.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1725767870}) 2024-08-19T10:44:55.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186931885}) 2024-08-19T10:44:55.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3550727740}) 2024-08-19T10:44:55.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186931885}) 2024-08-19T10:44:55.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1141352113}) 2024-08-19T10:44:55.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:55.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186931885}) 2024-08-19T10:44:55.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:55.841+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3311236695}) 2024-08-19T10:44:55.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.953+0000 7fbc511cd640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h2"}] 2024-08-19T10:44:55.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.953+0000 7fbc511cd640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h2"}] 2024-08-19T10:44:55.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.953+0000 7fbc511cd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:44:55.956 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.953+0000 7fbc511cd640 1 -- 172.21.15.190:0/4189817661 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h2"} v 0) -- 0x7fbc4c09b0e0 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc43fff640 1 -- 172.21.15.190:0/4189817661 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h2"}]=0 device 'h2' does not appear in the crush map v230) ==== 118+0+0 (secure 0 0 0) 0x7fbc34023880 con 0x7fbc4c0a12b0 2024-08-19T10:44:55.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc43fff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h2"}] 2024-08-19T10:44:55.959 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h2' does not appear in the crush map 2024-08-19T10:44:55.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc43fff640 10 monclient: _finish_command 3 = system:0 device 'h2' does not appear in the crush map 2024-08-19T10:44:55.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 -- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbc2c03be00 msgr2=0x7fbc2c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbc2c03be00 0x7fbc2c03e2c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c007f00 tx=0x7fbc3c008040 comp rx=0 tx=0).stop 2024-08-19T10:44:55.960 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 10 monclient: shutdown 2024-08-19T10:44:55.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:55.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 -- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbc4c0a12b0 msgr2=0x7fbc4c095370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbc4c0a12b0 0x7fbc4c095370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400e840 tx=0x7fbc3400ed00 comp rx=0 tx=0).stop 2024-08-19T10:44:55.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 -- 172.21.15.190:0/4189817661 shutdown_connections 2024-08-19T10:44:55.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbc2c03be00 0x7fbc2c03e2c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbc4c0a3320 0x7fbc4c0958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbc4c0a12b0 0x7fbc4c095370 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 --2- 172.21.15.190:0/4189817661 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbc4c09b6a0 0x7fbc4c09aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 -- 172.21.15.190:0/4189817661 >> 172.21.15.190:0/4189817661 conn(0x7fbc4c0a85b0 msgr2=0x7fbc4c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:55.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 -- 172.21.15.190:0/4189817661 shutdown_connections 2024-08-19T10:44:55.961 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:55.957+0000 7fbc511cd640 1 -- 172.21.15.190:0/4189817661 wait complete. 2024-08-19T10:44:55.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient: get_monmap_and_config 2024-08-19T10:44:55.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient: build_initial_monmap 2024-08-19T10:44:55.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient: monmap: 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:55.973047+0000 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:55.973047+0000 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:55.972 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:55.973 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:55.973 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:55.973 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:55.973 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:55.973 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 1 Processor -- start 2024-08-19T10:44:55.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 1 -- start start 2024-08-19T10:44:55.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient: init 2024-08-19T10:44:55.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient: _add_conns ranks=[8,3,1,2,5,4,6,0,7,10,9] 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc40a6860 0x7f1dc40a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient(hunting): picked mon.noname-e con 0x7f1dc40a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc40a7180 0x7f1dc40a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient(hunting): picked mon.noname-g con 0x7f1dc40a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1dc4139620 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc40a33d0 con 0x7f1dc4139620 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc40a8190 con 0x7f1dc40a7180 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f1dc40a2ec0 con 0x7f1dc40a6860 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dca787640 10 monclient(hunting): authenticate will time out at 1345.789656s 2024-08-19T10:44:55.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc40a6860 0x7f1dc40a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc40a6860 0x7f1dc40a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50614/0 (socket says 172.21.15.89:50614) 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 1 -- 172.21.15.89:0/3160330668 learned_addr learned my addr 172.21.15.89:0/3160330668 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 10 monclient(hunting): get_auth_request con 0x7f1dc40a6860 auth_method 0 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc37fe640 1 --2- 172.21.15.89:0/3160330668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc40a7180 0x7f1dc40a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_done global_id 6083 payload 274 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 1 -- 172.21.15.89:0/3160330668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc40a7180 msgr2=0x7f1dc40a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:55.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 1 --2- 172.21.15.89:0/3160330668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc40a7180 0x7f1dc40a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 1 -- 172.21.15.89:0/3160330668 >> v1:172.21.15.190:6789/0 conn(0x7f1dc4139620 legacy=0x7f1dc4139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:55.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:55.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 1 monclient: found mon.noname-e 2024-08-19T10:44:55.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:55.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:55.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc3fff640 1 -- 172.21.15.89:0/3160330668 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc40a4cd0 con 0x7f1dc40a6860 2024-08-19T10:44:55.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc37fe640 10 monclient: get_auth_request con 0x7f1dc40a7180 auth_method 0 2024-08-19T10:44:55.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.969+0000 7f1dc37fe640 1 --2- 172.21.15.89:0/3160330668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc40a7180 0x7f1dc40a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:55.979 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:55.979 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h3 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc3fff640 1 -- 172.21.15.89:0/3160330668 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc40a6860 msgr2=0x7f1dc40a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc3fff640 1 --2- 172.21.15.89:0/3160330668 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc40a6860 0x7f1dc40a6c40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1db80098a0 tx=0x7f1db801c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=8fc309fb5c7ef435 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 1 -- 172.21.15.89:0/3160330668 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1db80b0070 con 0x7f1dc40a6860 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 10 monclient: dump: 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.984 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:55.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 1 -- 172.21.15.89:0/3160330668 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db80aa810 con 0x7f1dc40a6860 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 5 monclient: authenticate success, global_id 6083 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 4 set_mon_vals no callback set 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 1 -- 172.21.15.89:0/3160330668 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1db80a93e0 con 0x7f1dc40a6860 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dc2ffd640 10 monclient: dump: 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.986 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 10 monclient: shutdown 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 1 -- 172.21.15.89:0/3160330668 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc40a6860 msgr2=0x7f1dc40a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 1 --2- 172.21.15.89:0/3160330668 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc40a6860 0x7f1dc40a6c40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1db80098a0 tx=0x7f1db801c440 comp rx=0 tx=0).stop 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 1 -- 172.21.15.89:0/3160330668 shutdown_connections 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 1 --2- 172.21.15.89:0/3160330668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc40a7180 0x7f1dc40a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 1 --2- 172.21.15.89:0/3160330668 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc40a6860 0x7f1dc40a6c40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 1 -- 172.21.15.89:0/3160330668 >> 172.21.15.89:0/3160330668 conn(0x7f1dc4074b40 msgr2=0x7f1dc40abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 1 -- 172.21.15.89:0/3160330668 shutdown_connections 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 1 -- 172.21.15.89:0/3160330668 wait complete. 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 10 monclient: build_initial_monmap 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 10 monclient: monmap: 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:55.988 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 1 Processor -- start 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 1 -- start start 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.973+0000 7f1dca787640 10 monclient: init 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient: _add_conns ranks=[2,4,7,6,3,1,0,5,8] 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc40a6860 0x7f1dc413fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient(hunting): picked mon.noname-b con 0x7f1dc40a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc40a7180 0x7f1dc4138530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient(hunting): picked mon.noname-c con 0x7f1dc40a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1dc4139620 0x7f1dc413d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient(hunting): picked mon.noname-i con 0x7f1dc4139620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc4144430 con 0x7f1dc40a6860 2024-08-19T10:44:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f1dc41442b0 con 0x7f1dc40a7180 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f1dc41445b0 con 0x7f1dc4139620 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient(hunting): authenticate will time out at 1345.796514s 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc3fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc40a6860 0x7f1dc413fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc3fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc40a6860 0x7f1dc413fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56448/0 (socket says 172.21.15.89:56448) 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc3fff640 1 -- 172.21.15.89:0/2942414713 learned_addr learned my addr 172.21.15.89:0/2942414713 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc3fff640 10 monclient(hunting): get_auth_request con 0x7f1dc40a6860 auth_method 0 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc40a7180 0x7f1dc4138530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 10 monclient(hunting): get_auth_request con 0x7f1dc40a7180 auth_method 0 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc8d24640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1dc4139620 0x7f1dc413d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc8d24640 10 monclient(hunting): get_auth_request con 0x7f1dc4139620 auth_method 0 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc8d24640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc8d24640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc8d24640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 10 monclient(hunting): handle_auth_done global_id 44176 payload 824 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 1 -- 172.21.15.89:0/2942414713 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1dc4139620 msgr2=0x7f1dc413d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1dc4139620 0x7f1dc413d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 1 -- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc40a6860 msgr2=0x7f1dc413fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc40a6860 0x7f1dc413fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 1 monclient: found mon.noname-c 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 1 -- 172.21.15.89:0/2942414713 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc4122ec0 con 0x7f1dc40a7180 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc3fff640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc40a6860 0x7f1dc413fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 1 -- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc40a7180 msgr2=0x7f1dc4138530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc37fe640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc40a7180 0x7f1dc4138530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1db400c7f0 tx=0x7f1db400ccb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1db4015070 con 0x7f1dc40a7180 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 10 monclient: dump: 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 5 monclient: authenticate success, global_id 44176 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient: _renew_subs 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 1 -- 172.21.15.89:0/2942414713 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc40aae40 con 0x7f1dc40a7180 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db4007ca0 con 0x7f1dc40a7180 2024-08-19T10:44:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1db4007e50 con 0x7f1dc40a7180 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc1ffb640 4 set_mon_vals no callback set 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dc0ff9640 10 monclient: dump: 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient: _renew_subs 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.977+0000 7f1dca787640 1 -- 172.21.15.89:0/2942414713 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc4122620 con 0x7f1dc40a7180 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dca787640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dca787640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc4138a70 0x7f1dc413db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:55.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dca787640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dca787640 1 -- 172.21.15.89:0/2942414713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc41c9560 con 0x7f1dc4138a70 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc4138a70 0x7f1dc413db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 10 monclient: get_auth_request con 0x7f1dc4138a70 auth_method 0 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1db4007830 con 0x7f1dc40a7180 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_done global_id 6239 payload 824 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 1 -- 172.21.15.89:0/2942414713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1dc4076ca0 con 0x7f1dc4138a70 2024-08-19T10:44:55.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1db003bfa0 0x7f1db003e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(230..230 src has 1..230) ==== 14564+0+0 (secure 0 0 0) 0x7f1db4050e70 con 0x7f1dc40a7180 2024-08-19T10:44:56.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc8d24640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1db003bfa0 0x7f1db003e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc8d24640 10 monclient: get_auth_request con 0x7f1db003bfa0 auth_method 0 2024-08-19T10:44:56.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 1 -- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc4138a70 msgr2=0x7f1dc413db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc3fff640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc4138a70 0x7f1dc413db80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1db80097c0 tx=0x7f1db80191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1db80ba020 con 0x7f1dc4138a70 2024-08-19T10:44:56.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:56.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 10 monclient: dump: 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.003 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1db80aa830 con 0x7f1dc4138a70 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc4138a70 msgr2=0x7f1dc413db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc4138a70 0x7f1dc413db80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1db80097c0 tx=0x7f1db80191e0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc4138a70 msgr2=0x7f1dc413db80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc0ff9640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc4138a70 0x7f1dc413db80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.981+0000 7f1dc8d24640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1db003bfa0 0x7f1db003e460 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1dc413ed20 tx=0x7f1dbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dca787640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dca787640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc413e0c0 0x7f1dc40a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dca787640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dca787640 1 -- 172.21.15.89:0/2942414713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc4122ab0 con 0x7f1dc413e0c0 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc413e0c0 0x7f1dc40a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 10 monclient: get_auth_request con 0x7f1dc413e0c0 auth_method 0 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 10 monclient(hunting): handle_auth_done global_id 6248 payload 824 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 1 -- 172.21.15.89:0/2942414713 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1dc413e4d0 con 0x7f1dc413e0c0 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 1 -- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc413e0c0 msgr2=0x7f1dc40a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc3fff640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc413e0c0 0x7f1dc40a01f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1db8000c10 tx=0x7f1db80a8000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1db80ba040 con 0x7f1dc413e0c0 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc0ff9640 10 monclient: dump: 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f1db80c1020 con 0x7f1dc413e0c0 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc0ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc413e0c0 msgr2=0x7f1dc40a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc0ff9640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc413e0c0 0x7f1dc40a01f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1db8000c10 tx=0x7f1db80a8000 comp rx=0 tx=0).stop 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc0ff9640 1 -- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc413e0c0 msgr2=0x7f1dc40a01f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.989+0000 7f1dc0ff9640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc413e0c0 0x7f1dc40a01f0 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1db8000c10 tx=0x7f1db80a8000 comp rx=0 tx=0).stop 2024-08-19T10:44:56.008 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:56.008 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:56.008 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:56.008 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:56.008 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:56.008 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 425818, 2024-08-19T10:44:56.008 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:56.008 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:56.008 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.009 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:56.010 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.011 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:56.012 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.013 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.015 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:56.018 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.019 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.020 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.021 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:56.024 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.025 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.026 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:56.027 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:56.029 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:56.030 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:56.032 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:44:56.033 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.034 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.034 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:56.034 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:56.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dca787640 1 -- 172.21.15.89:0/2942414713 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1db003bfa0 msgr2=0x7f1db003e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dca787640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1db003bfa0 0x7f1db003e460 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1dc413ed20 tx=0x7f1dbc009040 comp rx=0 tx=0).stop 2024-08-19T10:44:56.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dca787640 10 monclient: shutdown 2024-08-19T10:44:56.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dca787640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dca787640 1 -- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc40a7180 msgr2=0x7f1dc4138530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dca787640 1 --2- 172.21.15.89:0/2942414713 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc40a7180 0x7f1dc4138530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1db400c7f0 tx=0x7f1db400ccb0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dc8d24640 1 -- 172.21.15.89:0/2942414713 reap_dead start 2024-08-19T10:44:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dc8d24640 1 -- 172.21.15.89:0/2942414713 reap_dead start 2024-08-19T10:44:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dca787640 1 -- 172.21.15.89:0/2942414713 shutdown_connections 2024-08-19T10:44:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dca787640 1 -- 172.21.15.89:0/2942414713 >> 172.21.15.89:0/2942414713 conn(0x7f1dc4074b40 msgr2=0x7f1dc40a9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dca787640 1 -- 172.21.15.89:0/2942414713 shutdown_connections 2024-08-19T10:44:56.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:55.993+0000 7f1dca787640 1 -- 172.21.15.89:0/2942414713 wait complete. 2024-08-19T10:44:56.036 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:56.037 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:44:56.037 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:44:56.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2993016835}) 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4070148358}) 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2993016835}) 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1800075533}) 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2993016835}) 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1949152462}) 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2993016835}) 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=437844555}) 2024-08-19T10:44:56.063 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2993016835}) 2024-08-19T10:44:56.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=540178297}) 2024-08-19T10:44:56.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2993016835}) 2024-08-19T10:44:56.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=166377709}) 2024-08-19T10:44:56.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2993016835}) 2024-08-19T10:44:56.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3768886473}) 2024-08-19T10:44:56.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2993016835}) 2024-08-19T10:44:56.064 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.057+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2776121341}) 2024-08-19T10:44:56.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1799019500}) 2024-08-19T10:44:56.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=500113577}) 2024-08-19T10:44:56.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1799019500}) 2024-08-19T10:44:56.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1075647034}) 2024-08-19T10:44:56.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1799019500}) 2024-08-19T10:44:56.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2159201095}) 2024-08-19T10:44:56.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1799019500}) 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=407959474}) 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1799019500}) 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2725914568}) 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1799019500}) 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2602635279}) 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1799019500}) 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4273400445}) 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1799019500}) 2024-08-19T10:44:56.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.061+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2001932205}) 2024-08-19T10:44:56.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3211409176}) 2024-08-19T10:44:56.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3763209611}) 2024-08-19T10:44:56.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3211409176}) 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1695071844}) 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3211409176}) 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2168501674}) 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3211409176}) 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3968168510}) 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3211409176}) 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1216340375}) 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3211409176}) 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2904369014}) 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3211409176}) 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1108615246}) 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3211409176}) 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2175940772}) 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4254034871}) 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2862784730}) 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4254034871}) 2024-08-19T10:44:56.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3831668601}) 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4254034871}) 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3211409141}) 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4254034871}) 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1550761648}) 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4254034871}) 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4065229309}) 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4254034871}) 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1714354355}) 2024-08-19T10:44:56.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4254034871}) 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3252659579}) 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4254034871}) 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.065+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=162242623}) 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3417201461}) 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3773239205}) 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3417201461}) 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3172116832}) 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3417201461}) 2024-08-19T10:44:56.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2975701762}) 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3417201461}) 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1553341778}) 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3417201461}) 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2348342512}) 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3417201461}) 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2961390526}) 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3417201461}) 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2672234032}) 2024-08-19T10:44:56.078 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.079 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3417201461}) 2024-08-19T10:44:56.079 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.069+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3031806524}) 2024-08-19T10:44:56.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2180442750}) 2024-08-19T10:44:56.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=893468628}) 2024-08-19T10:44:56.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2180442750}) 2024-08-19T10:44:56.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3552329829}) 2024-08-19T10:44:56.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2180442750}) 2024-08-19T10:44:56.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=66954248}) 2024-08-19T10:44:56.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2180442750}) 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3720819159}) 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2180442750}) 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=855848957}) 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2180442750}) 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1316968795}) 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2180442750}) 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=240921077}) 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2180442750}) 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1330593216}) 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1056309832}) 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2841421584}) 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1056309832}) 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=342561198}) 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1056309832}) 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2406055793}) 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1056309832}) 2024-08-19T10:44:56.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1767513032}) 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1056309832}) 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3434056776}) 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1056309832}) 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3366523570}) 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1056309832}) 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2936037934}) 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1056309832}) 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.073+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1603343053}) 2024-08-19T10:44:56.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2383903898}) 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=837646223}) 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2383903898}) 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536982786}) 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2383903898}) 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1882055634}) 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2383903898}) 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2792577566}) 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2383903898}) 2024-08-19T10:44:56.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=941746046}) 2024-08-19T10:44:56.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2383903898}) 2024-08-19T10:44:56.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=992286613}) 2024-08-19T10:44:56.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2383903898}) 2024-08-19T10:44:56.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2010027593}) 2024-08-19T10:44:56.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2383903898}) 2024-08-19T10:44:56.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.077+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1872772647}) 2024-08-19T10:44:56.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284607334}) 2024-08-19T10:44:56.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2994582370}) 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284607334}) 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=172750639}) 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284607334}) 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3483842854}) 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284607334}) 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=702007422}) 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284607334}) 2024-08-19T10:44:56.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=100551968}) 2024-08-19T10:44:56.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284607334}) 2024-08-19T10:44:56.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=46899457}) 2024-08-19T10:44:56.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284607334}) 2024-08-19T10:44:56.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2996484156}) 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284607334}) 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.081+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1227889043}) 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219534179}) 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2522198993}) 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219534179}) 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=516465706}) 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219534179}) 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=974492171}) 2024-08-19T10:44:56.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219534179}) 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1199006772}) 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219534179}) 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=89617158}) 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219534179}) 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=254638615}) 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219534179}) 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2025120752}) 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.097 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219534179}) 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=623131187}) 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1411783913}) 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585012059}) 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1411783913}) 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1661420723}) 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1411783913}) 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=680811087}) 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.098 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1411783913}) 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=109544709}) 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1411783913}) 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3312920347}) 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1411783913}) 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3948325731}) 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1411783913}) 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2002682223}) 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1411783913}) 2024-08-19T10:44:56.099 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.085+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2920771011}) 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2278348006}) 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=548441333}) 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2278348006}) 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2883314809}) 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2278348006}) 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2243291573}) 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2278348006}) 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2020261183}) 2024-08-19T10:44:56.100 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.101 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2278348006}) 2024-08-19T10:44:56.101 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=620204994}) 2024-08-19T10:44:56.101 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.101 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2278348006}) 2024-08-19T10:44:56.101 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1877251158}) 2024-08-19T10:44:56.101 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.101 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2278348006}) 2024-08-19T10:44:56.101 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3263312361}) 2024-08-19T10:44:56.101 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.101 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2278348006}) 2024-08-19T10:44:56.101 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=602700399}) 2024-08-19T10:44:56.109 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.109 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1893770475}) 2024-08-19T10:44:56.109 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=973714539}) 2024-08-19T10:44:56.109 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.109 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1893770475}) 2024-08-19T10:44:56.109 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3828544029}) 2024-08-19T10:44:56.109 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.109 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1893770475}) 2024-08-19T10:44:56.109 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3385624232}) 2024-08-19T10:44:56.109 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.109 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1893770475}) 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2457740593}) 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1893770475}) 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1991965737}) 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1893770475}) 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2892076097}) 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1893770475}) 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1506691642}) 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1893770475}) 2024-08-19T10:44:56.110 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.089+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1902859219}) 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3280471615}) 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=426822462}) 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3280471615}) 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1596060077}) 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3280471615}) 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=195899945}) 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3280471615}) 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=123320735}) 2024-08-19T10:44:56.111 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.112 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3280471615}) 2024-08-19T10:44:56.112 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1914259628}) 2024-08-19T10:44:56.112 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.112 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3280471615}) 2024-08-19T10:44:56.112 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=204055087}) 2024-08-19T10:44:56.112 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.112 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3280471615}) 2024-08-19T10:44:56.112 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1731413800}) 2024-08-19T10:44:56.112 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.112 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3280471615}) 2024-08-19T10:44:56.112 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.093+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=384632101}) 2024-08-19T10:44:56.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.093+0000 7f2f59ed3640 10 monclient: get_monmap_and_config 2024-08-19T10:44:56.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.093+0000 7f2f59ed3640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.093+0000 7f2f59ed3640 10 monclient: monmap: 2024-08-19T10:44:56.116 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:56.099460+0000 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:56.099460+0000 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.117 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 1 Processor -- start 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 1 -- start start 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 10 monclient: init 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 10 monclient: _add_conns ranks=[7,6,8,2,3,9,4,10,0,5,1] 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f54134130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 10 monclient(hunting): picked mon.noname-i con 0x7f2f540a1d50 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2f540a2130 0x7f2f540a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.118 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 10 monclient(hunting): picked mon.noname-d con 0x7f2f540a2130 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2f540a2510 0x7f2f540a3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 10 monclient(hunting): picked mon.noname-e con 0x7f2f540a2510 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f54134130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f2f54069b00 con 0x7f2f540a2130 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2f5409fd70 con 0x7f2f540a1d50 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2f540a0050 con 0x7f2f540a2510 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f59ed3640 10 monclient(hunting): authenticate will time out at 1341.669934s 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f54134130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:48890/0 (socket says 172.21.15.190:48890) 2024-08-19T10:44:56.119 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 1 -- 172.21.15.190:0/3009283018 learned_addr learned my addr 172.21.15.190:0/3009283018 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 10 monclient(hunting): get_auth_request con 0x7f2f540a1d50 auth_method 0 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f52ffd640 1 --2- 172.21.15.190:0/3009283018 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2f540a2130 0x7f2f540a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f53fff640 1 --2- 172.21.15.190:0/3009283018 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2f540a2510 0x7f2f540a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f53fff640 10 monclient(hunting): get_auth_request con 0x7f2f540a2510 auth_method 0 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f53fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f53fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f52ffd640 10 monclient(hunting): get_auth_request con 0x7f2f540a2130 auth_method 0 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f52ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 10 monclient(hunting): handle_auth_done global_id 15541 payload 274 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 1 -- 172.21.15.190:0/3009283018 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2f540a2510 msgr2=0x7f2f540a3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 1 --2- 172.21.15.190:0/3009283018 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2f540a2510 0x7f2f540a3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.121 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 1 -- 172.21.15.190:0/3009283018 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2f540a2130 msgr2=0x7f2f540a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 1 --2- 172.21.15.190:0/3009283018 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2f540a2130 0x7f2f540a2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 1 monclient: found mon.noname-i 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.097+0000 7f2f537fe640 1 -- 172.21.15.190:0/3009283018 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5409efc0 con 0x7f2f540a1d50 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f537fe640 1 -- 172.21.15.190:0/3009283018 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 msgr2=0x7f2f54134130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f537fe640 1 --2- 172.21.15.190:0/3009283018 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f54134130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2f5409ff80 tx=0x7f2f3c01c440 comp rx=0 tx=0).ready entity=mon.7 client_cookie=a800ebd188f31894 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 1 -- 172.21.15.190:0/3009283018 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f3c0b2070 con 0x7f2f540a1d50 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:56.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 10 monclient: dump: 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.123 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 1 -- 172.21.15.190:0/3009283018 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f3c0ab7f0 con 0x7f2f540a1d50 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 5 monclient: authenticate success, global_id 15541 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 1 -- 172.21.15.190:0/3009283018 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f3c0abe10 con 0x7f2f540a1d50 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 4 set_mon_vals no callback set 2024-08-19T10:44:56.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f527fc640 10 monclient: dump: 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.125 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 10 monclient: shutdown 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 1 -- 172.21.15.190:0/3009283018 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 msgr2=0x7f2f54134130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 1 --2- 172.21.15.190:0/3009283018 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f54134130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2f5409ff80 tx=0x7f2f3c01c440 comp rx=0 tx=0).stop 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 1 -- 172.21.15.190:0/3009283018 shutdown_connections 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 1 --2- 172.21.15.190:0/3009283018 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2f540a2510 0x7f2f540a3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 1 --2- 172.21.15.190:0/3009283018 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2f540a2130 0x7f2f540a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 1 --2- 172.21.15.190:0/3009283018 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f54134130 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.126 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 1 -- 172.21.15.190:0/3009283018 >> 172.21.15.190:0/3009283018 conn(0x7f2f540a84b0 msgr2=0x7f2f540a88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 1 -- 172.21.15.190:0/3009283018 shutdown_connections 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.101+0000 7f2f59ed3640 1 -- 172.21.15.190:0/3009283018 wait complete. 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient: monmap: 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.127 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 1 Processor -- start 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 1 -- start start 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient: init 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient: _add_conns ranks=[7,8,4,5,3,0,1,2,6] 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f5413d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient(hunting): picked mon.noname-i con 0x7f2f540a1d50 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:56.128 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2f540a2130 0x7f2f5413daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient(hunting): picked mon.noname-e con 0x7f2f540a2130 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f540a2510 0x7f2f54137150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient(hunting): picked mon.noname-c con 0x7f2f540a2510 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f2f54140ef0 con 0x7f2f540a2510 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2f54140d70 con 0x7f2f540a1d50 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2f54141070 con 0x7f2f540a2130 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f59ed3640 10 monclient(hunting): authenticate will time out at 1341.677181s 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f5413d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f5413d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:48894/0 (socket says 172.21.15.190:48894) 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 1 -- 172.21.15.190:0/220535232 learned_addr learned my addr 172.21.15.190:0/220535232 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 10 monclient(hunting): get_auth_request con 0x7f2f540a1d50 auth_method 0 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f52ffd640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2f540a2130 0x7f2f5413daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f53fff640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f540a2510 0x7f2f54137150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f52ffd640 10 monclient(hunting): get_auth_request con 0x7f2f540a2130 auth_method 0 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f52ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f52ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f52ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f53fff640 10 monclient(hunting): get_auth_request con 0x7f2f540a2510 auth_method 0 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f53fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f53fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f52ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f52ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f52ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 10 monclient(hunting): handle_auth_done global_id 15550 payload 824 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 1 -- 172.21.15.190:0/220535232 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2f540a2130 msgr2=0x7f2f5413daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2f540a2130 0x7f2f5413daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 1 -- 172.21.15.190:0/220535232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f540a2510 msgr2=0x7f2f54137150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f540a2510 0x7f2f54137150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 1 monclient: found mon.noname-i 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 1 -- 172.21.15.190:0/220535232 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f541368f0 con 0x7f2f540a1d50 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 1 -- 172.21.15.190:0/220535232 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 msgr2=0x7f2f5413d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.105+0000 7f2f537fe640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f5413d5b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0b0040 tx=0x7f2f3c0191e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 1 -- 172.21.15.190:0/220535232 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f3c0b2040 con 0x7f2f540a1d50 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:56.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 10 monclient: dump: 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.133 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f59ed3640 5 monclient: authenticate success, global_id 15550 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 1 -- 172.21.15.190:0/220535232 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f3c0bb070 con 0x7f2f540a1d50 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 1 -- 172.21.15.190:0/220535232 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f3c0044b0 con 0x7f2f540a1d50 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f51ffb640 4 set_mon_vals no callback set 2024-08-19T10:44:56.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 10 monclient: dump: 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.135 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f59ed3640 10 monclient: _renew_subs 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f59ed3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f59ed3640 1 -- 172.21.15.190:0/220535232 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5409f4d0 con 0x7f2f540a1d50 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f59ed3640 10 monclient: _renew_subs 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f59ed3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f59ed3640 1 -- 172.21.15.190:0/220535232 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5413a060 con 0x7f2f540a1d50 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f59ed3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f59ed3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 1 -- 172.21.15.190:0/220535232 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2f3c0abbb0 con 0x7f2f540a1d50 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f59ed3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f59ed3640 1 -- 172.21.15.190:0/220535232 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f54134130 con 0x7f2f540a1d50 2024-08-19T10:44:56.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f3403be00 0x7f2f3403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.137 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f4bfff640 1 -- 172.21.15.190:0/220535232 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(230..230 src has 1..230) ==== 14564+0+0 (secure 0 0 0) 0x7f2f3c0eac50 con 0x7f2f540a1d50 2024-08-19T10:44:56.137 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f52ffd640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f3403be00 0x7f2f3403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.137 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.109+0000 7f2f52ffd640 10 monclient: get_auth_request con 0x7f2f3403be00 auth_method 0 2024-08-19T10:44:56.137 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.113+0000 7f2f52ffd640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f3403be00 0x7f2f3403e2c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2f54074610 tx=0x7f2f44009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.137 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.113+0000 7f2f4bfff640 1 -- 172.21.15.190:0/220535232 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f2f3c016150 con 0x7f2f540a1d50 2024-08-19T10:44:56.137 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.113+0000 7f2f4bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.137 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.113+0000 7f2f4bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:56.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient: get_monmap_and_config 2024-08-19T10:44:56.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient: monmap: 2024-08-19T10:44:56.152 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:56.152 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.152 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:56.153677+0000 2024-08-19T10:44:56.152 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:56.153677+0000 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:56.153 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 1 Processor -- start 2024-08-19T10:44:56.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 1 -- start start 2024-08-19T10:44:56.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient: init 2024-08-19T10:44:56.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient: _add_conns ranks=[7,2,10,0,1,9,6,4,3,8,5] 2024-08-19T10:44:56.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f57b00a56b0 0x7f57b00a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient(hunting): picked mon.noname-i con 0x7f57b00a56b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:56.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57b00a1660 0x7f57b00a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient(hunting): picked mon.noname-b con 0x7f57b00a1660 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:56.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f57b0072250 0x7f57b0137460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient(hunting): picked mon.noname-f con 0x7f57b0072250 addr v2:172.21.15.190:3300/0 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f57b00a6510 con 0x7f57b0072250 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f57b0074010 con 0x7f57b00a1660 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f57b0072010 con 0x7f57b00a56b0 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b658c640 10 monclient(hunting): authenticate will time out at 1345.970134s 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57b00a1660 0x7f57b00a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57b00a1660 0x7f57b00a5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56478/0 (socket says 172.21.15.89:56478) 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 1 -- 172.21.15.89:0/1706246197 learned_addr learned my addr 172.21.15.89:0/1706246197 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 10 monclient(hunting): get_auth_request con 0x7f57b00a1660 auth_method 0 2024-08-19T10:44:56.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b4b29640 1 --2- 172.21.15.89:0/1706246197 >> v2:172.21.15.190:3300/0 conn(0x7f57b0072250 0x7f57b0137460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57affff640 1 --2- 172.21.15.89:0/1706246197 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f57b00a56b0 0x7f57b00a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b4b29640 10 monclient(hunting): get_auth_request con 0x7f57b0072250 auth_method 0 2024-08-19T10:44:56.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b4b29640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b4b29640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b4b29640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57affff640 10 monclient(hunting): get_auth_request con 0x7f57b00a56b0 auth_method 0 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b4b29640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b4b29640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57b4b29640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 10 monclient(hunting): handle_auth_done global_id 6257 payload 274 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 1 -- 172.21.15.89:0/1706246197 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f57b00a56b0 msgr2=0x7f57b00a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 1 --2- 172.21.15.89:0/1706246197 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f57b00a56b0 0x7f57b00a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 1 -- 172.21.15.89:0/1706246197 >> v2:172.21.15.190:3300/0 conn(0x7f57b0072250 msgr2=0x7f57b0137460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 1 --2- 172.21.15.89:0/1706246197 >> v2:172.21.15.190:3300/0 conn(0x7f57b0072250 0x7f57b0137460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 1 monclient: found mon.noname-b 2024-08-19T10:44:56.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:44:56.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.149+0000 7f57af7fe640 1 -- 172.21.15.89:0/1706246197 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b0073d30 con 0x7f57b00a1660 2024-08-19T10:44:56.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57af7fe640 1 -- 172.21.15.89:0/1706246197 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57b00a1660 msgr2=0x7f57b00a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57af7fe640 1 --2- 172.21.15.89:0/1706246197 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57b00a1660 0x7f57b00a5fd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009870 tx=0x7f57a401c190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da6812d263dbad9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 1 -- 172.21.15.89:0/1706246197 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f57a40b0070 con 0x7f57b00a1660 2024-08-19T10:44:56.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:44:56.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 10 monclient: dump: 2024-08-19T10:44:56.166 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.166 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.166 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.166 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 1 -- 172.21.15.89:0/1706246197 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a40aa810 con 0x7f57b00a1660 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 5 monclient: authenticate success, global_id 6257 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 1 -- 172.21.15.89:0/1706246197 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f57a40a9300 con 0x7f57b00a1660 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 4 set_mon_vals no callback set 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57aeffd640 10 monclient: dump: 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.168 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.169 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 10 monclient: shutdown 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 1 -- 172.21.15.89:0/1706246197 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57b00a1660 msgr2=0x7f57b00a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 1 --2- 172.21.15.89:0/1706246197 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57b00a1660 0x7f57b00a5fd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009870 tx=0x7f57a401c190 comp rx=0 tx=0).stop 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 1 -- 172.21.15.89:0/1706246197 shutdown_connections 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 1 --2- 172.21.15.89:0/1706246197 >> v2:172.21.15.190:3300/0 conn(0x7f57b0072250 0x7f57b0137460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 1 --2- 172.21.15.89:0/1706246197 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f57b00a1660 0x7f57b00a5fd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 1 --2- 172.21.15.89:0/1706246197 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f57b00a56b0 0x7f57b00a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 1 -- 172.21.15.89:0/1706246197 >> 172.21.15.89:0/1706246197 conn(0x7f57b00aa2a0 msgr2=0x7f57b00a67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 1 -- 172.21.15.89:0/1706246197 shutdown_connections 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.153+0000 7f57b658c640 1 -- 172.21.15.89:0/1706246197 wait complete. 2024-08-19T10:44:56.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient: monmap: 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.171 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 1 Processor -- start 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 1 -- start start 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient: init 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient: _add_conns ranks=[1,5,6,7,4,8,0,3,2] 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b00a56b0 0x7f57b01cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient(hunting): picked mon.noname-f con 0x7f57b00a56b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f57b0136520 0x7f57b01369a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient(hunting): picked mon.noname-h con 0x7f57b0136520 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:56.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57b013b4f0 0x7f57b013b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient(hunting): picked mon.noname-d con 0x7f57b013b4f0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f57b0141260 con 0x7f57b00a56b0 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f57b01410e0 con 0x7f57b0136520 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f57b01413e0 con 0x7f57b013b4f0 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b658c640 10 monclient(hunting): authenticate will time out at 1345.977107s 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57b013b4f0 0x7f57b013b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57affff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b00a56b0 0x7f57b01cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57b013b4f0 0x7f57b013b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:33138/0 (socket says 172.21.15.89:33138) 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 1 -- 172.21.15.89:0/1444461537 learned_addr learned my addr 172.21.15.89:0/1444461537 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 10 monclient(hunting): get_auth_request con 0x7f57b013b4f0 auth_method 0 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57affff640 10 monclient(hunting): get_auth_request con 0x7f57b00a56b0 auth_method 0 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57af7fe640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f57b0136520 0x7f57b01369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57af7fe640 10 monclient(hunting): get_auth_request con 0x7f57b0136520 auth_method 0 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57af7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 10 monclient(hunting): handle_auth_done global_id 44214 payload 824 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 1 -- 172.21.15.89:0/1444461537 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f57b0136520 msgr2=0x7f57b01369a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f57b0136520 0x7f57b01369a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 1 -- 172.21.15.89:0/1444461537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b00a56b0 msgr2=0x7f57b01cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57b00a56b0 0x7f57b01cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 1 monclient: found mon.noname-d 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 1 -- 172.21.15.89:0/1444461537 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b013cfb0 con 0x7f57b013b4f0 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 1 -- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57b013b4f0 msgr2=0x7f57b013b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.157+0000 7f57b4b29640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57b013b4f0 0x7f57b013b8d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009dd0 tx=0x7f57a800c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f57a8018020 con 0x7f57b013b4f0 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:44:56.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 10 monclient: dump: 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.177 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a80148f0 con 0x7f57b013b4f0 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57b658c640 5 monclient: authenticate success, global_id 44214 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57b658c640 10 monclient: _renew_subs 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57b658c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57b658c640 1 -- 172.21.15.89:0/1444461537 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b01cf920 con 0x7f57b013b4f0 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f57a80133e0 con 0x7f57b013b4f0 2024-08-19T10:44:56.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 10 monclient: dump: 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.179 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57ae7fc640 4 set_mon_vals no callback set 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57b658c640 10 monclient: _renew_subs 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57b658c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57b658c640 1 -- 172.21.15.89:0/1444461537 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b0121620 con 0x7f57b013b4f0 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57b658c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57b658c640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0072250 0x7f57b013c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57b658c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57b658c640 1 -- 172.21.15.89:0/1444461537 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f57b00a3aa0 con 0x7f57b0072250 2024-08-19T10:44:56.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f57a801b020 con 0x7f57b013b4f0 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0072250 0x7f57b013c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 10 monclient: get_auth_request con 0x7f57b0072250 auth_method 0 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f579c03be30 0x7f579c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(230..230 src has 1..230) ==== 14564+0+0 (secure 0 0 0) 0x7f57a8051b20 con 0x7f57b013b4f0 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 10 monclient(hunting): handle_auth_done global_id 44194 payload 824 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 1 -- 172.21.15.89:0/1444461537 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57b0064590 con 0x7f57b0072250 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57af7fe640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f579c03be30 0x7f579c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57af7fe640 10 monclient: get_auth_request con 0x7f579c03be30 auth_method 0 2024-08-19T10:44:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 1 -- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0072250 msgr2=0x7f57b013c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57affff640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0072250 0x7f57b013c5d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f57a0011890 tx=0x7f57a0011d50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f57a0019070 con 0x7f57b0072250 2024-08-19T10:44:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.161+0000 7f57acff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.165+0000 7f57acff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.165+0000 7f57acff9640 10 monclient: dump: 2024-08-19T10:44:56.184 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.185 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.165+0000 7f57af7fe640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f579c03be30 0x7f579c03e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f57b00aac70 tx=0x7f57a40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.165+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f57a000f890 con 0x7f57b0072250 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.165+0000 7f57acff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.165+0000 7f57acff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.165+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0072250 msgr2=0x7f57b013c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.165+0000 7f57acff9640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0072250 0x7f57b013c5d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f57a0011890 tx=0x7f57a0011d50 comp rx=0 tx=0).stop 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.165+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0072250 msgr2=0x7f57b013c5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.165+0000 7f57acff9640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0072250 0x7f57b013c5d0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f57a0011890 tx=0x7f57a0011d50 comp rx=0 tx=0).stop 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57b658c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57b658c640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0121ca0 0x7f57b002d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57b658c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57b658c640 1 -- 172.21.15.89:0/1444461537 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f57b00a0100 con 0x7f57b0121ca0 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57affff640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0121ca0 0x7f57b002d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57affff640 10 monclient: get_auth_request con 0x7f57b0121ca0 auth_method 0 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57affff640 10 monclient(hunting): handle_auth_done global_id 44203 payload 824 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.169+0000 7f57affff640 1 -- 172.21.15.89:0/1444461537 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f57b0121820 con 0x7f57b0121ca0 2024-08-19T10:44:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57affff640 1 -- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0121ca0 msgr2=0x7f57b002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57affff640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0121ca0 0x7f57b002d1d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f57a0011890 tx=0x7f57a00046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f57a0019040 con 0x7f57b0121ca0 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57acff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57acff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57acff9640 10 monclient: dump: 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7f57a001e450 con 0x7f57b0121ca0 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57acff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57acff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0121ca0 msgr2=0x7f57b002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57acff9640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0121ca0 0x7f57b002d1d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f57a0011890 tx=0x7f57a00046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57acff9640 1 -- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0121ca0 msgr2=0x7f57b002d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57acff9640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f57b0121ca0 0x7f57b002d1d0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f57a0011890 tx=0x7f57a00046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.190 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 19299, 2024-08-19T10:44:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:56.190 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:56.191 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:56.192 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:56.193 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:56.194 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.195 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.196 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.196 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:56.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.196 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:56.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.196 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:56.196 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:56.196 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.197 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:56.198 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:56.201 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.204 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.206 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.207 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.209 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.211 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.212 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:56.213 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.213 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.213 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:56.213 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b658c640 1 -- 172.21.15.89:0/1444461537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f579c03be30 msgr2=0x7f579c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b658c640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f579c03be30 0x7f579c03e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f57b00aac70 tx=0x7f57a40a6000 comp rx=0 tx=0).stop 2024-08-19T10:44:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b658c640 10 monclient: shutdown 2024-08-19T10:44:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b658c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b658c640 1 -- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57b013b4f0 msgr2=0x7f57b013b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b658c640 1 --2- 172.21.15.89:0/1444461537 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57b013b4f0 0x7f57b013b8d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009dd0 tx=0x7f57a800c490 comp rx=0 tx=0).stop 2024-08-19T10:44:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b4b29640 1 -- 172.21.15.89:0/1444461537 reap_dead start 2024-08-19T10:44:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b4b29640 1 -- 172.21.15.89:0/1444461537 reap_dead start 2024-08-19T10:44:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b658c640 1 -- 172.21.15.89:0/1444461537 shutdown_connections 2024-08-19T10:44:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b658c640 1 -- 172.21.15.89:0/1444461537 >> 172.21.15.89:0/1444461537 conn(0x7f57b00aa2a0 msgr2=0x7f57b00a8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b658c640 1 -- 172.21.15.89:0/1444461537 shutdown_connections 2024-08-19T10:44:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.173+0000 7f57b658c640 1 -- 172.21.15.89:0/1444461537 wait complete. 2024-08-19T10:44:56.215 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:56.215 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:44:56.216 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:44:56.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2795728686}) 2024-08-19T10:44:56.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2440361385}) 2024-08-19T10:44:56.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2795728686}) 2024-08-19T10:44:56.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3155869284}) 2024-08-19T10:44:56.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2795728686}) 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2029027337}) 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2795728686}) 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3479214911}) 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2795728686}) 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=998127322}) 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2795728686}) 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3902825992}) 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2795728686}) 2024-08-19T10:44:56.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3704752433}) 2024-08-19T10:44:56.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2795728686}) 2024-08-19T10:44:56.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.213+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=464339429}) 2024-08-19T10:44:56.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176306298}) 2024-08-19T10:44:56.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1766463885}) 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176306298}) 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1353052402}) 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176306298}) 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1460503023}) 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176306298}) 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2292883767}) 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176306298}) 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=463377261}) 2024-08-19T10:44:56.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176306298}) 2024-08-19T10:44:56.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=699151080}) 2024-08-19T10:44:56.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176306298}) 2024-08-19T10:44:56.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=911677225}) 2024-08-19T10:44:56.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176306298}) 2024-08-19T10:44:56.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.221+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3406604046}) 2024-08-19T10:44:56.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.289+0000 7f2f59ed3640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h3"}] 2024-08-19T10:44:56.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.289+0000 7f2f59ed3640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h3"}] 2024-08-19T10:44:56.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.289+0000 7f2f59ed3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:56.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:56.289+0000 7f2f59ed3640 1 -- 172.21.15.190:0/220535232 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h3"} v 0) -- 0x7f2f5413e330 con 0x7f2f540a1d50 2024-08-19T10:44:56.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient: get_monmap_and_config 2024-08-19T10:44:56.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient: monmap: 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:56.337101+0000 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:56.337101+0000 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.336 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.337 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.337 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.337 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:56.337 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:56.337 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 1 Processor -- start 2024-08-19T10:44:56.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 1 -- start start 2024-08-19T10:44:56.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient: init 2024-08-19T10:44:56.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient: _add_conns ranks=[9,3,4,0,6,2,5,1,7,8,10] 2024-08-19T10:44:56.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff69c0a6860 0x7ff69c0a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient(hunting): picked mon.noname-a con 0x7ff69c0a6860 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c0a6c40 0x7ff69c0a7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient(hunting): picked mon.noname-g con 0x7ff69c0a6c40 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69c139620 0x7ff69c139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient(hunting): picked mon.noname-c con 0x7ff69c139620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff69c0a61a0 con 0x7ff69c0a6860 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff69c0a81b0 con 0x7ff69c0a6c40 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff69c0a4c90 con 0x7ff69c139620 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a43ac640 10 monclient(hunting): authenticate will time out at 1346.153803s 2024-08-19T10:44:56.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff69c0a6860 0x7ff69c0a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff69c0a6860 0x7ff69c0a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51988/0 (socket says 172.21.15.89:51988) 2024-08-19T10:44:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 1 -- 172.21.15.89:0/1346345511 learned_addr learned my addr 172.21.15.89:0/1346345511 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 10 monclient(hunting): get_auth_request con 0x7ff69c0a6860 auth_method 0 2024-08-19T10:44:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2949640 1 --2- 172.21.15.89:0/1346345511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69c139620 0x7ff69c139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2949640 10 monclient(hunting): get_auth_request con 0x7ff69c139620 auth_method 0 2024-08-19T10:44:56.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2949640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2949640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2949640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2949640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2949640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2949640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a1947640 1 --2- 172.21.15.89:0/1346345511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c0a6c40 0x7ff69c0a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 10 monclient(hunting): handle_auth_done global_id 6291 payload 274 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 1 -- 172.21.15.89:0/1346345511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69c139620 msgr2=0x7ff69c139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 1 --2- 172.21.15.89:0/1346345511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69c139620 0x7ff69c139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 1 -- 172.21.15.89:0/1346345511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c0a6c40 msgr2=0x7ff69c0a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 1 --2- 172.21.15.89:0/1346345511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c0a6c40 0x7ff69c0a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 1 monclient: found mon.noname-a 2024-08-19T10:44:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a2148640 1 -- 172.21.15.89:0/1346345511 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c0a85c0 con 0x7ff69c0a6860 2024-08-19T10:44:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a1947640 10 monclient: get_auth_request con 0x7ff69c0a6c40 auth_method 0 2024-08-19T10:44:56.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.333+0000 7ff6a1947640 1 --2- 172.21.15.89:0/1346345511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c0a6c40 0x7ff69c0a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:56.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a2148640 1 -- 172.21.15.89:0/1346345511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c0a6860 msgr2=0x7ff69c0a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a2148640 1 --2- 172.21.15.89:0/1346345511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c0a6860 0x7ff69c0a54d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff694009e50 tx=0x7ff69401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c2690eadf8ddaa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 1 -- 172.21.15.89:0/1346345511 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6940090d0 con 0x7ff69c0a6860 2024-08-19T10:44:56.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 10 monclient: dump: 2024-08-19T10:44:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.345 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.346 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.347 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 1 -- 172.21.15.89:0/1346345511 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6940a59a0 con 0x7ff69c0a6860 2024-08-19T10:44:56.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 5 monclient: authenticate success, global_id 6291 2024-08-19T10:44:56.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:56.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:56.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 1 -- 172.21.15.89:0/1346345511 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6940ad3f0 con 0x7ff69c0a6860 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 4 set_mon_vals no callback set 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a1146640 10 monclient: dump: 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.348 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.349 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.349 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.349 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.349 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.349 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.349 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.349 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 10 monclient: shutdown 2024-08-19T10:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 1 -- 172.21.15.89:0/1346345511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c0a6860 msgr2=0x7ff69c0a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 1 --2- 172.21.15.89:0/1346345511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c0a6860 0x7ff69c0a54d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff694009e50 tx=0x7ff69401c8f0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 1 -- 172.21.15.89:0/1346345511 shutdown_connections 2024-08-19T10:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 1 --2- 172.21.15.89:0/1346345511 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69c139620 0x7ff69c139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 1 --2- 172.21.15.89:0/1346345511 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c0a6c40 0x7ff69c0a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 1 --2- 172.21.15.89:0/1346345511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c0a6860 0x7ff69c0a54d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 1 -- 172.21.15.89:0/1346345511 >> 172.21.15.89:0/1346345511 conn(0x7ff69c0699d0 msgr2=0x7ff69c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 1 -- 172.21.15.89:0/1346345511 shutdown_connections 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 1 -- 172.21.15.89:0/1346345511 wait complete. 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 10 monclient: monmap: 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 1 Processor -- start 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 1 -- start start 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 10 monclient: init 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.341+0000 7ff6a43ac640 10 monclient: _add_conns ranks=[4,8,0,1,6,7,3,2,5] 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69c0a6860 0x7ff69c134860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient(hunting): picked mon.noname-c con 0x7ff69c0a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff69c0a6c40 0x7ff69c134da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient(hunting): picked mon.noname-e con 0x7ff69c0a6c40 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c139620 0x7ff69c09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient(hunting): picked mon.noname-a con 0x7ff69c139620 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c143210 con 0x7ff69c139620 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff69c143090 con 0x7ff69c0a6860 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff69c143390 con 0x7ff69c0a6c40 2024-08-19T10:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient(hunting): authenticate will time out at 1346.164110s 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff69c0a6c40 0x7ff69c134da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2949640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c139620 0x7ff69c09c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2148640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69c0a6860 0x7ff69c134860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff69c0a6c40 0x7ff69c134da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50622/0 (socket says 172.21.15.89:50622) 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 1 -- 172.21.15.89:0/196193507 learned_addr learned my addr 172.21.15.89:0/196193507 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 10 monclient(hunting): get_auth_request con 0x7ff69c0a6c40 auth_method 0 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2949640 10 monclient(hunting): get_auth_request con 0x7ff69c139620 auth_method 0 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2949640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2949640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2949640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2148640 10 monclient(hunting): get_auth_request con 0x7ff69c0a6860 auth_method 0 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2148640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2148640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2148640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2949640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2949640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2949640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 10 monclient(hunting): handle_auth_done global_id 6110 payload 824 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 1 -- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69c0a6860 msgr2=0x7ff69c134860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff69c0a6860 0x7ff69c134860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 1 -- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c139620 msgr2=0x7ff69c09c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c139620 0x7ff69c09c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 1 monclient: found mon.noname-e 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 1 -- 172.21.15.89:0/196193507 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c0a0440 con 0x7ff69c0a6c40 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2949640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff69c139620 0x7ff69c09c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 1 -- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff69c0a6c40 msgr2=0x7ff69c134da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a1947640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff69c0a6c40 0x7ff69c134da0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff69000c7f0 tx=0x7ff69000ccb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6900113f0 con 0x7ff69c0a6c40 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 10 monclient: dump: 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.362 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6900115d0 con 0x7ff69c0a6c40 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 5 monclient: authenticate success, global_id 6110 2024-08-19T10:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff690011780 con 0x7ff69c0a6c40 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a0945640 4 set_mon_vals no callback set 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff69affd640 10 monclient: dump: 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient: _renew_subs 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 1 -- 172.21.15.89:0/196193507 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c09e100 con 0x7ff69c0a6c40 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient: _renew_subs 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 1 -- 172.21.15.89:0/196193507 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c0a0d00 con 0x7ff69c0a6c40 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09cf20 0x7ff69c09d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a43ac640 1 -- 172.21.15.89:0/196193507 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff69c09fdc0 con 0x7ff69c09cf20 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.345+0000 7ff6a2148640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09cf20 0x7ff69c09d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2148640 10 monclient: get_auth_request con 0x7ff69c09cf20 auth_method 0 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2148640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2148640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2148640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff690007a40 con 0x7ff69c0a6c40 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2148640 10 monclient(hunting): handle_auth_done global_id 44223 payload 824 2024-08-19T10:44:56.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2148640 1 -- 172.21.15.89:0/196193507 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff69c0a3cf0 con 0x7ff69c09cf20 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff68803be00 0x7ff68803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(230..230 src has 1..230) ==== 14564+0+0 (secure 0 0 0) 0x7ff690051340 con 0x7ff69c0a6c40 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2949640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff68803be00 0x7ff68803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2949640 10 monclient: get_auth_request con 0x7ff68803be00 auth_method 0 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2148640 1 -- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09cf20 msgr2=0x7ff69c09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2148640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09cf20 0x7ff69c09d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff6940ae000 tx=0x7ff694019690 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6940046c0 con 0x7ff69c09cf20 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 10 monclient: dump: 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff694004250 con 0x7ff69c09cf20 2024-08-19T10:44:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09cf20 msgr2=0x7ff69c09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09cf20 0x7ff69c09d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff6940ae000 tx=0x7ff694019690 comp rx=0 tx=0).stop 2024-08-19T10:44:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09cf20 msgr2=0x7ff69c09d300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff69affd640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09cf20 0x7ff69c09d300 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff6940ae000 tx=0x7ff694019690 comp rx=0 tx=0).stop 2024-08-19T10:44:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.349+0000 7ff6a2949640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff68803be00 0x7ff68803e2c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009e30 tx=0x7ff68c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a43ac640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a43ac640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09d840 0x7ff69c13a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a43ac640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a43ac640 1 -- 172.21.15.89:0/196193507 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff69c0a7020 con 0x7ff69c09d840 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09d840 0x7ff69c13a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 10 monclient: get_auth_request con 0x7ff69c09d840 auth_method 0 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 10 monclient(hunting): handle_auth_done global_id 44232 payload 824 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 1 -- 172.21.15.89:0/196193507 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff69c064590 con 0x7ff69c09d840 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 1 -- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09d840 msgr2=0x7ff69c13a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.357+0000 7ff6a2148640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09d840 0x7ff69c13a9f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff69401c720 tx=0x7ff6940a7000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6940a59a0 con 0x7ff69c09d840 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff69affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff69affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff69affd640 10 monclient: dump: 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.372 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7ff6940a5530 con 0x7ff69c09d840 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff69affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:56.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff69affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09d840 msgr2=0x7ff69c13a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff69affd640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09d840 0x7ff69c13a9f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff69401c720 tx=0x7ff6940a7000 comp rx=0 tx=0).stop 2024-08-19T10:44:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff69affd640 1 -- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09d840 msgr2=0x7ff69c13a9f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff69affd640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff69c09d840 0x7ff69c13a9f0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff69401c720 tx=0x7ff6940a7000 comp rx=0 tx=0).stop 2024-08-19T10:44:56.374 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:56.374 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:56.374 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:56.374 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:56.374 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 19487, 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:44:56.375 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.376 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.377 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:56.378 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.379 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.381 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:56.384 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.386 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.388 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:56.390 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.393 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.395 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:56.396 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:56.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.396 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.396 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.396 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:56.396 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.396 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.397 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:56.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.397 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.397 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.397 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:56.397 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.397 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.397 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:56.397 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff698ff9640 1 -- 172.21.15.89:0/196193507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff68803be00 msgr2=0x7ff68803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff698ff9640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff68803be00 0x7ff68803e2c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009e30 tx=0x7ff68c009190 comp rx=0 tx=0).stop 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff698ff9640 10 monclient: shutdown 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff698ff9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff698ff9640 1 -- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff69c0a6c40 msgr2=0x7ff69c134da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff698ff9640 1 --2- 172.21.15.89:0/196193507 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff69c0a6c40 0x7ff69c134da0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff69000c7f0 tx=0x7ff69000ccb0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff6a2949640 1 -- 172.21.15.89:0/196193507 reap_dead start 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff6a2949640 1 -- 172.21.15.89:0/196193507 reap_dead start 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff698ff9640 1 -- 172.21.15.89:0/196193507 shutdown_connections 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff698ff9640 1 -- 172.21.15.89:0/196193507 >> 172.21.15.89:0/196193507 conn(0x7ff69c0699d0 msgr2=0x7ff69c138a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff698ff9640 1 -- 172.21.15.89:0/196193507 shutdown_connections 2024-08-19T10:44:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.361+0000 7ff698ff9640 1 -- 172.21.15.89:0/196193507 wait complete. 2024-08-19T10:44:56.400 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:56.400 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:44:56.400 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:44:56.524 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.524 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3046714870}) 2024-08-19T10:44:56.524 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1266455801}) 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3046714870}) 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=584235642}) 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3046714870}) 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2473741293}) 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3046714870}) 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3244940907}) 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3046714870}) 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1690270078}) 2024-08-19T10:44:56.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3046714870}) 2024-08-19T10:44:56.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3809828519}) 2024-08-19T10:44:56.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3046714870}) 2024-08-19T10:44:56.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1249480810}) 2024-08-19T10:44:56.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3046714870}) 2024-08-19T10:44:56.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.521+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=475761870}) 2024-08-19T10:44:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient: get_monmap_and_config 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient: monmap: 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:56.573276+0000 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:56.573276+0000 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:56.573 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 1 Processor -- start 2024-08-19T10:44:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 1 -- start start 2024-08-19T10:44:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient: init 2024-08-19T10:44:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient: _add_conns ranks=[3,0,7,6,9,1,2,5,4,8,10] 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33800a6860 0x7f33800a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient(hunting): picked mon.noname-g con 0x7f33800a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f33800a7180 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33800a7560 0x7f3380139d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient(hunting): picked mon.noname-i con 0x7f33800a7560 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33800a33d0 con 0x7f33800a7180 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f33800a8190 con 0x7f33800a6860 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f33800a79f0 con 0x7f33800a7560 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338737f640 10 monclient(hunting): authenticate will time out at 1346.389799s 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338491a640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f33800a7180 0x7f3380139620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:58348/0 (socket says 172.21.15.89:58348) 2024-08-19T10:44:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338491a640 1 -- 172.21.15.89:0/4182297974 learned_addr learned my addr 172.21.15.89:0/4182297974 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338511b640 1 --2- 172.21.15.89:0/4182297974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33800a6860 0x7f33800a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 1 --2- 172.21.15.89:0/4182297974 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33800a7560 0x7f3380139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 10 monclient(hunting): get_auth_request con 0x7f33800a7560 auth_method 0 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338511b640 10 monclient(hunting): get_auth_request con 0x7f33800a6860 auth_method 0 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338511b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338511b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338511b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 10 monclient(hunting): handle_auth_done global_id 15568 payload 274 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 1 -- 172.21.15.89:0/4182297974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33800a6860 msgr2=0x7f33800a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 1 --2- 172.21.15.89:0/4182297974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33800a6860 0x7f33800a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 1 -- 172.21.15.89:0/4182297974 >> v1:172.21.15.89:6789/0 conn(0x7f33800a7180 legacy=0x7f3380139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 1 monclient: found mon.noname-i 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:44:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.569+0000 7f338591c640 1 -- 172.21.15.89:0/4182297974 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33800a2ec0 con 0x7f33800a7560 2024-08-19T10:44:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.573+0000 7f338591c640 1 -- 172.21.15.89:0/4182297974 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33800a7560 msgr2=0x7f3380139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.573+0000 7f338591c640 1 --2- 172.21.15.89:0/4182297974 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33800a7560 0x7f3380139d30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f33780097f0 tx=0x7f337801c3a0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=3a907acfdc9caf7f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.573+0000 7f3373fff640 1 -- 172.21.15.89:0/4182297974 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33780b0070 con 0x7f33800a7560 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.573+0000 7f3373fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.573+0000 7f3373fff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.573+0000 7f3373fff640 10 monclient: dump: 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f3373fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f3373fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f3373fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 5 monclient: authenticate success, global_id 15568 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f3373fff640 1 -- 172.21.15.89:0/4182297974 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33780aa810 con 0x7f33800a7560 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f3373fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 4 set_mon_vals no callback set 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f3373fff640 1 -- 172.21.15.89:0/4182297974 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f33780aa9c0 con 0x7f33800a7560 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f3373fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f3373fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f3373fff640 10 monclient: dump: 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.583 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient: shutdown 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 -- 172.21.15.89:0/4182297974 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33800a7560 msgr2=0x7f3380139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 --2- 172.21.15.89:0/4182297974 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33800a7560 0x7f3380139d30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f33780097f0 tx=0x7f337801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 -- 172.21.15.89:0/4182297974 shutdown_connections 2024-08-19T10:44:56.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 --2- 172.21.15.89:0/4182297974 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f33800a7560 0x7f3380139d30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 --2- 172.21.15.89:0/4182297974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33800a6860 0x7f33800a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 -- 172.21.15.89:0/4182297974 >> 172.21.15.89:0/4182297974 conn(0x7f3380074b40 msgr2=0x7f33800abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 -- 172.21.15.89:0/4182297974 shutdown_connections 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 -- 172.21.15.89:0/4182297974 wait complete. 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient: monmap: 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.586 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 Processor -- start 2024-08-19T10:44:56.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 -- start start 2024-08-19T10:44:56.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient: init 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient: _add_conns ranks=[2,4,3,8,1,7,0,6,5] 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33800a6860 0x7f3380138770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient(hunting): picked mon.noname-b con 0x7f33800a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33800a7180 0x7f3380138cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient(hunting): picked mon.noname-c con 0x7f33800a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33800a7560 0x7f338013e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient(hunting): picked mon.noname-g con 0x7f33800a7560 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f338013a3f0 con 0x7f33800a6860 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f338013a270 con 0x7f33800a7560 2024-08-19T10:44:56.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f33801440a0 con 0x7f33800a7180 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338737f640 10 monclient(hunting): authenticate will time out at 1346.398709s 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338511b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33800a6860 0x7f3380138770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33800a7180 0x7f3380138cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338511b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33800a6860 0x7f3380138770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56494/0 (socket says 172.21.15.89:56494) 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33800a7180 0x7f3380138cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36676/0 (socket says 172.21.15.89:36676) 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338511b640 1 -- 172.21.15.89:0/1773445401 learned_addr learned my addr 172.21.15.89:0/1773445401 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338511b640 10 monclient(hunting): get_auth_request con 0x7f33800a6860 auth_method 0 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338511b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338511b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338511b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 10 monclient(hunting): get_auth_request con 0x7f33800a7180 auth_method 0 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338591c640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33800a7560 0x7f338013e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338591c640 10 monclient(hunting): get_auth_request con 0x7f33800a7560 auth_method 0 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338591c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338591c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338591c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 10 monclient(hunting): handle_auth_done global_id 44230 payload 824 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 1 -- 172.21.15.89:0/1773445401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33800a7560 msgr2=0x7f338013e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33800a7560 0x7f338013e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 1 -- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33800a6860 msgr2=0x7f3380138770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33800a6860 0x7f3380138770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 1 monclient: found mon.noname-c 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338491a640 1 -- 172.21.15.89:0/1773445401 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3380123360 con 0x7f33800a7180 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.577+0000 7f338511b640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f33800a6860 0x7f3380138770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338491a640 1 -- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33800a7180 msgr2=0x7f3380138cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338491a640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33800a7180 0x7f3380138cb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f337400caf0 tx=0x7f337400a060 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f337400acb0 con 0x7f33800a7180 2024-08-19T10:44:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:44:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 10 monclient: dump: 2024-08-19T10:44:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.597 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.598 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338737f640 5 monclient: authenticate success, global_id 44230 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338737f640 10 monclient: _renew_subs 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338737f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338737f640 1 -- 172.21.15.89:0/1773445401 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3380123640 con 0x7f33800a7180 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f337400ae50 con 0x7f33800a7180 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3374012870 con 0x7f33800a7180 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f33737fe640 4 set_mon_vals no callback set 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 10 monclient: dump: 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.600 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338737f640 10 monclient: _renew_subs 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338737f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338737f640 1 -- 172.21.15.89:0/1773445401 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f3380123080 con 0x7f33800a7180 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338737f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338737f640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013e6f0 0x7f338013ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338737f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338737f640 1 -- 172.21.15.89:0/1773445401 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f33800a26b0 con 0x7f338013e6f0 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3374012400 con 0x7f33800a7180 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338511b640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013e6f0 0x7f338013ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338511b640 10 monclient: get_auth_request con 0x7f338013e6f0 auth_method 0 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338511b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338511b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338511b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f336403be30 0x7f336403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(230..230 src has 1..230) ==== 14564+0+0 (secure 0 0 0) 0x7f3374051640 con 0x7f33800a7180 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338511b640 10 monclient(hunting): handle_auth_done global_id 6119 payload 824 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338511b640 1 -- 172.21.15.89:0/1773445401 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f33800a2110 con 0x7f338013e6f0 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338591c640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f336403be30 0x7f336403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.581+0000 7f338591c640 10 monclient: get_auth_request con 0x7f336403be30 auth_method 0 2024-08-19T10:44:56.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f338511b640 1 -- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013e6f0 msgr2=0x7f338013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f338511b640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013e6f0 0x7f338013ead0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f337c009ec0 tx=0x7f337c00f610 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f337c014ac0 con 0x7f338013e6f0 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f3371ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f3371ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f3371ffb640 10 monclient: dump: 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.603 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f337c014650 con 0x7f338013e6f0 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f3371ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f3371ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013e6f0 msgr2=0x7f338013ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f3371ffb640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013e6f0 0x7f338013ead0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f337c009ec0 tx=0x7f337c00f610 comp rx=0 tx=0).stop 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013e6f0 msgr2=0x7f338013ead0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f3371ffb640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013e6f0 0x7f338013ead0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.585+0000 7f338591c640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f336403be30 0x7f336403e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f33800a5460 tx=0x7f33780a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013f010 0x7f338013a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 1 -- 172.21.15.89:0/1773445401 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f33800a2400 con 0x7f338013f010 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013f010 0x7f338013a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 10 monclient: get_auth_request con 0x7f338013f010 auth_method 0 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 10 monclient(hunting): handle_auth_done global_id 6128 payload 824 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 1 -- 172.21.15.89:0/1773445401 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3380064590 con 0x7f338013f010 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 1 -- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013f010 msgr2=0x7f338013a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338511b640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013f010 0x7f338013a630 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f337c01b000 tx=0x7f337c0046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f337c004490 con 0x7f338013f010 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f3371ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f3371ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f3371ffb640 10 monclient: dump: 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.608 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f337c0273f0 con 0x7f338013f010 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f3371ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f3371ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:56.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013f010 msgr2=0x7f338013a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f3371ffb640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013f010 0x7f338013a630 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f337c01b000 tx=0x7f337c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f3371ffb640 1 -- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013f010 msgr2=0x7f338013a630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f3371ffb640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f338013f010 0x7f338013a630 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f337c01b000 tx=0x7f337c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.610 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 426386, 2024-08-19T10:44:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 7, 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.616 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.617 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.619 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.620 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.622 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.624 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.624 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:56.624 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.624 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.624 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.624 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.624 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.624 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.624 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.625 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.626 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.627 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.629 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.631 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.632 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:56.632 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:56.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.632 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.632 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.632 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:56.633 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:56.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.635 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.635 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:56.635 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:56.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 1 -- 172.21.15.89:0/1773445401 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f336403be30 msgr2=0x7f336403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f336403be30 0x7f336403e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f33800a5460 tx=0x7f33780a6000 comp rx=0 tx=0).stop 2024-08-19T10:44:56.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 10 monclient: shutdown 2024-08-19T10:44:56.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:56.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 1 -- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33800a7180 msgr2=0x7f3380138cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 1 --2- 172.21.15.89:0/1773445401 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f33800a7180 0x7f3380138cb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f337400caf0 tx=0x7f337400a060 comp rx=0 tx=0).stop 2024-08-19T10:44:56.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338591c640 1 -- 172.21.15.89:0/1773445401 reap_dead start 2024-08-19T10:44:56.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338591c640 1 -- 172.21.15.89:0/1773445401 reap_dead start 2024-08-19T10:44:56.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 1 -- 172.21.15.89:0/1773445401 shutdown_connections 2024-08-19T10:44:56.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 1 -- 172.21.15.89:0/1773445401 >> 172.21.15.89:0/1773445401 conn(0x7f3380074b40 msgr2=0x7f3380141090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:56.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 1 -- 172.21.15.89:0/1773445401 shutdown_connections 2024-08-19T10:44:56.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.593+0000 7f338737f640 1 -- 172.21.15.89:0/1773445401 wait complete. 2024-08-19T10:44:56.637 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:56.637 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:44:56.638 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:44:56.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4089731991}) 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3169231239}) 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4089731991}) 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4145577192}) 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4089731991}) 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2114457764}) 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4089731991}) 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3467567669}) 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4089731991}) 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=922336513}) 2024-08-19T10:44:56.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4089731991}) 2024-08-19T10:44:56.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2213209409}) 2024-08-19T10:44:56.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4089731991}) 2024-08-19T10:44:56.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3013249903}) 2024-08-19T10:44:56.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4089731991}) 2024-08-19T10:44:56.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.749+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3209101325}) 2024-08-19T10:44:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient: get_monmap_and_config 2024-08-19T10:44:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient: monmap: 2024-08-19T10:44:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:56.753778+0000 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:56.753778+0000 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.757 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 1 Processor -- start 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 1 -- start start 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient: init 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient: _add_conns ranks=[9,4,7,5,0,1,2,3,10,8,6] 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff9480a3af0 0x7ff9480a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient(hunting): picked mon.noname-a con 0x7ff9480a3af0 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9480a4410 0x7ff94813d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient(hunting): picked mon.noname-c con 0x7ff9480a4410 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff94813dec0 0x7ff9481402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient(hunting): picked mon.noname-i con 0x7ff94813dec0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff9480a02a0 con 0x7ff9480a3af0 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff9480a31e0 con 0x7ff9480a4410 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff9480a0ed0 con 0x7ff94813dec0 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff94dde8640 10 monclient(hunting): authenticate will time out at 1346.570669s 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9480a4410 0x7ff94813d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9480a4410 0x7ff94813d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36692/0 (socket says 172.21.15.89:36692) 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 1 -- 172.21.15.89:0/4027696027 learned_addr learned my addr 172.21.15.89:0/4027696027 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 10 monclient(hunting): get_auth_request con 0x7ff9480a4410 auth_method 0 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff947fff640 1 --2- 172.21.15.89:0/4027696027 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff94813dec0 0x7ff9481402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff947fff640 10 monclient(hunting): get_auth_request con 0x7ff94813dec0 auth_method 0 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff947fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff947fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff947fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 10 monclient(hunting): handle_auth_done global_id 44239 payload 274 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 1 -- 172.21.15.89:0/4027696027 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff94813dec0 msgr2=0x7ff9481402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 1 --2- 172.21.15.89:0/4027696027 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff94813dec0 0x7ff9481402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 1 -- 172.21.15.89:0/4027696027 >> v2:172.21.15.89:3300/0 conn(0x7ff9480a3af0 msgr2=0x7ff9480a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 1 --2- 172.21.15.89:0/4027696027 >> v2:172.21.15.89:3300/0 conn(0x7ff9480a3af0 0x7ff9480a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 1 monclient: found mon.noname-c 2024-08-19T10:44:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:44:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.749+0000 7ff946ffd640 1 -- 172.21.15.89:0/4027696027 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9480a11b0 con 0x7ff9480a4410 2024-08-19T10:44:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff946ffd640 1 -- 172.21.15.89:0/4027696027 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9480a4410 msgr2=0x7ff94813d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff946ffd640 1 --2- 172.21.15.89:0/4027696027 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9480a4410 0x7ff94813d980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0098a0 tx=0x7ff93c01c400 comp rx=0 tx=0).ready entity=mon.4 client_cookie=61a8a46486e09ef8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 1 -- 172.21.15.89:0/4027696027 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff93c0b0070 con 0x7ff9480a4410 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 10 monclient: dump: 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff94dde8640 5 monclient: authenticate success, global_id 44239 2024-08-19T10:44:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff94dde8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 1 -- 172.21.15.89:0/4027696027 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c0aa810 con 0x7ff9480a4410 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff94dde8640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 1 -- 172.21.15.89:0/4027696027 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff93c0aaa00 con 0x7ff9480a4410 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff94dde8640 4 set_mon_vals no callback set 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.753+0000 7ff9467fc640 10 monclient: dump: 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.757+0000 7ff94dde8640 10 monclient: shutdown 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.757+0000 7ff94dde8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.757+0000 7ff94dde8640 1 -- 172.21.15.89:0/4027696027 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9480a4410 msgr2=0x7ff94813d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.757+0000 7ff94dde8640 1 --2- 172.21.15.89:0/4027696027 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9480a4410 0x7ff94813d980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0098a0 tx=0x7ff93c01c400 comp rx=0 tx=0).stop 2024-08-19T10:44:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.757+0000 7ff94dde8640 1 -- 172.21.15.89:0/4027696027 shutdown_connections 2024-08-19T10:44:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.757+0000 7ff94dde8640 1 --2- 172.21.15.89:0/4027696027 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff94813dec0 0x7ff9481402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.757+0000 7ff94dde8640 1 --2- 172.21.15.89:0/4027696027 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9480a4410 0x7ff94813d980 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.757+0000 7ff94dde8640 1 --2- 172.21.15.89:0/4027696027 >> v2:172.21.15.89:3300/0 conn(0x7ff9480a3af0 0x7ff9480a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.757+0000 7ff94dde8640 1 -- 172.21.15.89:0/4027696027 >> 172.21.15.89:0/4027696027 conn(0x7ff94806de40 msgr2=0x7ff94806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.757+0000 7ff94dde8640 1 -- 172.21.15.89:0/4027696027 shutdown_connections 2024-08-19T10:44:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.757+0000 7ff94dde8640 1 -- 172.21.15.89:0/4027696027 wait complete. 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.761+0000 7ff94dde8640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.761+0000 7ff94dde8640 10 monclient: monmap: 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.769 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.761+0000 7ff94dde8640 1 Processor -- start 2024-08-19T10:44:56.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 1 -- start start 2024-08-19T10:44:56.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 10 monclient: init 2024-08-19T10:44:56.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 10 monclient: _add_conns ranks=[4,7,0,1,6,5,8,3,2] 2024-08-19T10:44:56.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9480a3af0 0x7ff948122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 10 monclient(hunting): picked mon.noname-c con 0x7ff9480a3af0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:56.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff9480a4410 0x7ff948070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 10 monclient(hunting): picked mon.noname-i con 0x7ff9480a4410 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:56.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff94813dec0 0x7ff948071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 10 monclient(hunting): picked mon.noname-a con 0x7ff94813dec0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff94806cbf0 con 0x7ff94813dec0 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff948074400 con 0x7ff9480a3af0 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff94813fa80 con 0x7ff9480a4410 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff94dde8640 10 monclient(hunting): authenticate will time out at 1346.586153s 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff94813dec0 0x7ff948071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff94813dec0 0x7ff948071300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52026/0 (socket says 172.21.15.89:52026) 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 1 -- 172.21.15.89:0/300964836 learned_addr learned my addr 172.21.15.89:0/300964836 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 10 monclient(hunting): get_auth_request con 0x7ff94813dec0 auth_method 0 2024-08-19T10:44:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff946ffd640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff9480a4410 0x7ff948070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff946ffd640 10 monclient(hunting): get_auth_request con 0x7ff9480a4410 auth_method 0 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff946ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff946ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff946ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 10 monclient(hunting): handle_auth_done global_id 6309 payload 824 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 1 -- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff9480a4410 msgr2=0x7ff948070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff9480a4410 0x7ff948070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 1 -- 172.21.15.89:0/300964836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9480a3af0 msgr2=0x7ff948122770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff9480a3af0 0x7ff948122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 1 monclient: found mon.noname-a 2024-08-19T10:44:56.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:56.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.765+0000 7ff947fff640 1 -- 172.21.15.89:0/300964836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9480a1fd0 con 0x7ff94813dec0 2024-08-19T10:44:56.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.769+0000 7ff947fff640 1 -- 172.21.15.89:0/300964836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff94813dec0 msgr2=0x7ff948071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.769+0000 7ff947fff640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff94813dec0 0x7ff948071300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff940007d80 tx=0x7ff94000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4082627901}) 2024-08-19T10:44:56.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4024194232}) 2024-08-19T10:44:56.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4082627901}) 2024-08-19T10:44:56.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3188829107}) 2024-08-19T10:44:56.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4082627901}) 2024-08-19T10:44:56.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1529895060}) 2024-08-19T10:44:56.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4082627901}) 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2488709799}) 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4082627901}) 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1165345279}) 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4082627901}) 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2222097703}) 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4082627901}) 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1097836407}) 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4082627901}) 2024-08-19T10:44:56.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:56.773+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=465611311}) 2024-08-19T10:44:56.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff940014890 con 0x7ff94813dec0 2024-08-19T10:44:56.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:56.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 10 monclient: dump: 2024-08-19T10:44:56.786 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.786 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.786 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.786 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.786 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.787 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.787 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.787 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.787 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.787 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.787 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.787 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.787 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.787 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.787 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.787 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff940019070 con 0x7ff94813dec0 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff94dde8640 5 monclient: authenticate success, global_id 6309 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff940013380 con 0x7ff94813dec0 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 10 monclient: dump: 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.788 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.789 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.789 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.789 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.789 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.789 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.789 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.789 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.789 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.789 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.789 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.789 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.790 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff945ffb640 4 set_mon_vals no callback set 2024-08-19T10:44:56.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff94dde8640 10 monclient: _renew_subs 2024-08-19T10:44:56.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff94dde8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:56.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff94dde8640 1 -- 172.21.15.89:0/300964836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94809f530 con 0x7ff94813dec0 2024-08-19T10:44:56.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff94dde8640 10 monclient: _renew_subs 2024-08-19T10:44:56.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff94dde8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:44:56.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff94dde8640 1 -- 172.21.15.89:0/300964836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff948073240 con 0x7ff94813dec0 2024-08-19T10:44:56.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff94dde8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff94dde8640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff948071840 0x7ff94813e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff94dde8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff94dde8640 1 -- 172.21.15.89:0/300964836 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff94806cd70 con 0x7ff948071840 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff940014420 con 0x7ff94813dec0 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff93403be30 0x7ff93403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(230..230 src has 1..230) ==== 14564+0+0 (secure 0 0 0) 0x7ff9400518a0 con 0x7ff94813dec0 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff9477fe640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff948071840 0x7ff94813e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff9477fe640 10 monclient: get_auth_request con 0x7ff948071840 auth_method 0 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff9477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff9477fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.781+0000 7ff9477fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.785+0000 7ff9477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.785+0000 7ff9477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.785+0000 7ff9477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.785+0000 7ff946ffd640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff93403be30 0x7ff93403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.785+0000 7ff946ffd640 10 monclient: get_auth_request con 0x7ff93403be30 auth_method 0 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.785+0000 7ff9477fe640 10 monclient(hunting): handle_auth_done global_id 44164 payload 824 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.785+0000 7ff9477fe640 1 -- 172.21.15.89:0/300964836 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff948071c20 con 0x7ff948071840 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff946ffd640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff93403be30 0x7ff93403e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff94806cf00 tx=0x7ff93c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff9477fe640 1 -- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff948071840 msgr2=0x7ff94813e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff9477fe640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff948071840 0x7ff94813e2a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b320 tx=0x7ff93800b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff938015020 con 0x7ff948071840 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff92ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff92ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff92ffff640 10 monclient: dump: 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.795 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff938011920 con 0x7ff948071840 2024-08-19T10:44:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff92ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff92ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff948071840 msgr2=0x7ff94813e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff92ffff640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff948071840 0x7ff94813e2a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b320 tx=0x7ff93800b7e0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff948071840 msgr2=0x7ff94813e2a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.789+0000 7ff92ffff640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff948071840 0x7ff94813e2a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff94dde8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff94dde8640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff94813e7e0 0x7ff94813ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff94dde8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff94dde8640 1 -- 172.21.15.89:0/300964836 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff9480a3ed0 con 0x7ff94813e7e0 2024-08-19T10:44:56.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff94813e7e0 0x7ff94813ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 10 monclient: get_auth_request con 0x7ff94813e7e0 auth_method 0 2024-08-19T10:44:56.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 10 monclient(hunting): handle_auth_done global_id 44173 payload 824 2024-08-19T10:44:56.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 1 -- 172.21.15.89:0/300964836 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff948064590 con 0x7ff94813e7e0 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 1 -- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff94813e7e0 msgr2=0x7ff94813ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff9477fe640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff94813e7e0 0x7ff94813ebf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b320 tx=0x7ff9380047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff938015040 con 0x7ff94813e7e0 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff92ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff92ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.797+0000 7ff92ffff640 10 monclient: dump: 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.803 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.804 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.804 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.804 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.804 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.804 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.804 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.804 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.804 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.804 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7ff938011e10 con 0x7ff94813e7e0 2024-08-19T10:44:56.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff92ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:56.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff92ffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:56.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff94813e7e0 msgr2=0x7ff94813ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff92ffff640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff94813e7e0 0x7ff94813ebf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b320 tx=0x7ff9380047e0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff92ffff640 1 -- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff94813e7e0 msgr2=0x7ff94813ebf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff92ffff640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff94813e7e0 0x7ff94813ebf0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b320 tx=0x7ff9380047e0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.805 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 19927, 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:56.806 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 7, 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.807 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.808 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:56.809 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.810 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.811 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:56.812 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:56.812 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.812 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.812 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.813 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:56.813 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.813 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.813 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.813 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:56.813 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.814 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.814 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.814 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.814 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.815 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.815 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.815 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.816 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.817 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:56.818 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:56.818 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.818 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.818 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.818 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.818 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:56.818 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.820 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:56.820 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:56.820 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.820 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.820 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.820 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:56.820 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:56.822 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:56.823 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.826 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:56.829 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:56.830 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.832 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:56.832 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:56.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff94dde8640 1 -- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff93403be30 msgr2=0x7ff93403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff94dde8640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff93403be30 0x7ff93403e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff94806cf00 tx=0x7ff93c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:44:56.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff94dde8640 10 monclient: shutdown 2024-08-19T10:44:56.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff94dde8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:56.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff94dde8640 1 -- 172.21.15.89:0/300964836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff94813dec0 msgr2=0x7ff948071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff94dde8640 1 --2- 172.21.15.89:0/300964836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff94813dec0 0x7ff948071300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff940007d80 tx=0x7ff94000e4b0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff947fff640 1 -- 172.21.15.89:0/300964836 reap_dead start 2024-08-19T10:44:56.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff947fff640 1 -- 172.21.15.89:0/300964836 reap_dead start 2024-08-19T10:44:56.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff94dde8640 1 -- 172.21.15.89:0/300964836 shutdown_connections 2024-08-19T10:44:56.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff94dde8640 1 -- 172.21.15.89:0/300964836 >> 172.21.15.89:0/300964836 conn(0x7ff94806de40 msgr2=0x7ff9480a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:56.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff94dde8640 1 -- 172.21.15.89:0/300964836 shutdown_connections 2024-08-19T10:44:56.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.801+0000 7ff94dde8640 1 -- 172.21.15.89:0/300964836 wait complete. 2024-08-19T10:44:56.834 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:56.834 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:44:56.834 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:44:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient: get_monmap_and_config 2024-08-19T10:44:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient: monmap: 2024-08-19T10:44:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:56.941779+0000 2024-08-19T10:44:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:56.941779+0000 2024-08-19T10:44:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:56.941 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 1 Processor -- start 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 1 -- start start 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient: init 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient: _add_conns ranks=[9,1,8,5,4,7,10,3,6,2,0] 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa4b006c930 0x7fa4b006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient(hunting): picked mon.noname-a con 0x7fa4b006c930 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa4b006d250 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b0137b70 0x7fa4b013f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient(hunting): picked mon.noname-e con 0x7fa4b0137b70 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b00a3960 con 0x7fa4b006d250 2024-08-19T10:44:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b00a3e00 con 0x7fa4b006c930 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa4b00a3b80 con 0x7fa4b0137b70 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b676d640 10 monclient(hunting): authenticate will time out at 1346.758321s 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa4b006c930 0x7fa4b006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b4d0a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b0137b70 0x7fa4b013f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa4b006c930 0x7fa4b006cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52034/0 (socket says 172.21.15.89:52034) 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 1 -- 172.21.15.89:0/1913586213 learned_addr learned my addr 172.21.15.89:0/1913586213 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 10 monclient(hunting): get_auth_request con 0x7fa4b006c930 auth_method 0 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b4d0a640 10 monclient(hunting): get_auth_request con 0x7fa4b0137b70 auth_method 0 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b4d0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b4d0a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b4d0a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b4d0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b4d0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b4d0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 10 monclient(hunting): handle_auth_done global_id 6318 payload 274 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 1 -- 172.21.15.89:0/1913586213 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b0137b70 msgr2=0x7fa4b013f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 1 --2- 172.21.15.89:0/1913586213 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b0137b70 0x7fa4b013f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 1 -- 172.21.15.89:0/1913586213 >> v1:172.21.15.190:6789/0 conn(0x7fa4b006d250 legacy=0x7fa4b0137460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 1 monclient: found mon.noname-a 2024-08-19T10:44:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4affff640 1 -- 172.21.15.89:0/1913586213 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b00a4ea0 con 0x7fa4b006c930 2024-08-19T10:44:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.937+0000 7fa4b4d0a640 1 --2- 172.21.15.89:0/1913586213 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b0137b70 0x7fa4b013f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4affff640 1 -- 172.21.15.89:0/1913586213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4b006c930 msgr2=0x7fa4b006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4affff640 1 --2- 172.21.15.89:0/1913586213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4b006c930 0x7fa4b006cd10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009e50 tx=0x7fa4a001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84a636cf664e272 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 1 -- 172.21.15.89:0/1913586213 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4a00090d0 con 0x7fa4b006c930 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 10 monclient: dump: 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 1 -- 172.21.15.89:0/1913586213 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a00a5a40 con 0x7fa4b006c930 2024-08-19T10:44:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 1 -- 172.21.15.89:0/1913586213 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4a00a5bf0 con 0x7fa4b006c930 2024-08-19T10:44:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4b676d640 5 monclient: authenticate success, global_id 6318 2024-08-19T10:44:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4b676d640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4b676d640 4 set_mon_vals no callback set 2024-08-19T10:44:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.941+0000 7fa4aeffd640 10 monclient: dump: 2024-08-19T10:44:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient: shutdown 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 -- 172.21.15.89:0/1913586213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4b006c930 msgr2=0x7fa4b006cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 --2- 172.21.15.89:0/1913586213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4b006c930 0x7fa4b006cd10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009e50 tx=0x7fa4a001c980 comp rx=0 tx=0).stop 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 -- 172.21.15.89:0/1913586213 shutdown_connections 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 --2- 172.21.15.89:0/1913586213 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b0137b70 0x7fa4b013f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 --2- 172.21.15.89:0/1913586213 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4b006c930 0x7fa4b006cd10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 -- 172.21.15.89:0/1913586213 >> 172.21.15.89:0/1913586213 conn(0x7fa4b00a9090 msgr2=0x7fa4b00a94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 -- 172.21.15.89:0/1913586213 shutdown_connections 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 -- 172.21.15.89:0/1913586213 wait complete. 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient: build_initial_monmap 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient: monmap: 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 Processor -- start 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 -- start start 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient: init 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient: _add_conns ranks=[2,8,0,1,6,4,3,5,7] 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4b006c930 0x7fa4b013a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient(hunting): picked mon.noname-b con 0x7fa4b006c930 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b006d250 0x7fa4b013a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient(hunting): picked mon.noname-e con 0x7fa4b006d250 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4b0137b70 0x7fa4b013ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient(hunting): picked mon.noname-a con 0x7fa4b0137b70 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b009fe00 con 0x7fa4b0137b70 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b009fc80 con 0x7fa4b006c930 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa4b0142420 con 0x7fa4b006d250 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b676d640 10 monclient(hunting): authenticate will time out at 1346.765865s 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4affff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4b006c930 0x7fa4b013a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b006d250 0x7fa4b013a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4affff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4b006c930 0x7fa4b013a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56508/0 (socket says 172.21.15.89:56508) 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4affff640 1 -- 172.21.15.89:0/1476950452 learned_addr learned my addr 172.21.15.89:0/1476950452 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4affff640 10 monclient(hunting): get_auth_request con 0x7fa4b006c930 auth_method 0 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 10 monclient(hunting): get_auth_request con 0x7fa4b006d250 auth_method 0 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4affff640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4b006c930 msgr2=0x7fa4b013a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b4d0a640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4b0137b70 0x7fa4b013ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4affff640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4b006c930 msgr2=0x7fa4b013a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4affff640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4b006c930 0x7fa4b013a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b4d0a640 10 monclient(hunting): get_auth_request con 0x7fa4b0137b70 auth_method 0 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b4d0a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b4d0a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b4d0a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4affff640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4b006c930 0x7fa4b013a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b4d0a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b4d0a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b4d0a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 10 monclient(hunting): handle_auth_done global_id 6146 payload 824 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4b006c930 msgr2=0x7fa4b013a100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4b006c930 0x7fa4b013a100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4b0137b70 msgr2=0x7fa4b013ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4b0137b70 0x7fa4b013ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:44:56.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 1 monclient: found mon.noname-e 2024-08-19T10:44:56.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:56.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:56.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4af7fe640 1 -- 172.21.15.89:0/1476950452 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b013d960 con 0x7fa4b006d250 2024-08-19T10:44:56.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.945+0000 7fa4b4d0a640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4b0137b70 0x7fa4b013ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:56.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4af7fe640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b006d250 msgr2=0x7fa4b013a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4af7fe640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b006d250 0x7fa4b013a640 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa49800c3e0 tx=0x7fa49800c7e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa498011c10 con 0x7fa4b006d250 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 10 monclient: dump: 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.965 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 10 monclient: _finish_auth 0 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 10 monclient: _check_auth_tickets 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b676d640 5 monclient: authenticate success, global_id 6146 2024-08-19T10:44:56.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa498017070 con 0x7fa4b006d250 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b676d640 10 monclient: _renew_subs 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa498011e30 con 0x7fa4b006d250 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b676d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4ae7fc640 4 set_mon_vals no callback set 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b676d640 1 -- 172.21.15.89:0/1476950452 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b013dc40 con 0x7fa4b006d250 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 10 monclient: dump: 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b676d640 10 monclient: _renew_subs 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b676d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:56.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b676d640 1 -- 172.21.15.89:0/1476950452 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b013d0b0 con 0x7fa4b006d250 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b676d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b676d640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfaf0 0x7fa4b002d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa4980117a0 con 0x7fa4b006d250 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b676d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b676d640 1 -- 172.21.15.89:0/1476950452 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b006cd10 con 0x7fa4b01cfaf0 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa49003bdf0 0x7fa49003e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(230..230 src has 1..230) ==== 14564+0+0 (secure 0 0 0) 0x7fa4980508b0 con 0x7fa4b006d250 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4affff640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfaf0 0x7fa4b002d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b4d0a640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa49003bdf0 0x7fa49003e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4affff640 10 monclient: get_auth_request con 0x7fa4b01cfaf0 auth_method 0 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4b4d0a640 10 monclient: get_auth_request con 0x7fa49003bdf0 auth_method 0 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4affff640 10 monclient(hunting): handle_auth_done global_id 34662 payload 824 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.949+0000 7fa4affff640 1 -- 172.21.15.89:0/1476950452 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4b0064590 con 0x7fa4b01cfaf0 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4b4d0a640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa49003bdf0 0x7fa49003e2b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80095a0 tx=0x7fa4a8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4affff640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfaf0 msgr2=0x7fa4b002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4affff640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfaf0 0x7fa4b002d1d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00af040 tx=0x7fa4a0019720 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4a00bd070 con 0x7fa4b01cfaf0 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4acff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4acff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4acff9640 10 monclient: dump: 2024-08-19T10:44:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa4a0004250 con 0x7fa4b01cfaf0 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4acff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4acff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfaf0 msgr2=0x7fa4b002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4acff9640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfaf0 0x7fa4b002d1d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00af040 tx=0x7fa4a0019720 comp rx=0 tx=0).stop 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfaf0 msgr2=0x7fa4b002d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.953+0000 7fa4acff9640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfaf0 0x7fa4b002d1d0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00af040 tx=0x7fa4a0019720 comp rx=0 tx=0).stop 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4b676d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4b676d640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfed0 0x7fa4b002d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4b676d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4b676d640 1 -- 172.21.15.89:0/1476950452 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b013d680 con 0x7fa4b01cfed0 2024-08-19T10:44:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4affff640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfed0 0x7fa4b002d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4affff640 10 monclient: get_auth_request con 0x7fa4b01cfed0 auth_method 0 2024-08-19T10:44:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4affff640 10 monclient(hunting): handle_auth_done global_id 34671 payload 824 2024-08-19T10:44:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.961+0000 7fa4affff640 1 -- 172.21.15.89:0/1476950452 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa4b0135d50 con 0x7fa4b01cfed0 2024-08-19T10:44:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4affff640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfed0 msgr2=0x7fa4b002d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4affff640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfed0 0x7fa4b002d710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa4b006ce90 tx=0x7fa4a00a7000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4a00a5a40 con 0x7fa4b01cfed0 2024-08-19T10:44:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4acff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4acff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4acff9640 10 monclient: dump: 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7fa4a00a55d0 con 0x7fa4b01cfed0 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4acff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4acff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfed0 msgr2=0x7fa4b002d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4acff9640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfed0 0x7fa4b002d710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa4b006ce90 tx=0x7fa4a00a7000 comp rx=0 tx=0).stop 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4acff9640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfed0 msgr2=0x7fa4b002d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4acff9640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4b01cfed0 0x7fa4b002d710 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa4b006ce90 tx=0x7fa4a00a7000 comp rx=0 tx=0).stop 2024-08-19T10:44:56.977 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 142009, 2024-08-19T10:44:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:56.977 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 7, 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.978 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.979 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:56.980 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:56.982 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:56.985 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:56.986 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:56.988 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.989 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.990 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:56.991 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.993 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.994 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.995 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:56.996 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.998 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:56.999 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b676d640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa49003bdf0 msgr2=0x7fa49003e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b676d640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa49003bdf0 0x7fa49003e2b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80095a0 tx=0x7fa4a8009190 comp rx=0 tx=0).stop 2024-08-19T10:44:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b676d640 10 monclient: shutdown 2024-08-19T10:44:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b676d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b676d640 1 -- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b006d250 msgr2=0x7fa4b013a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b676d640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b006d250 0x7fa4b013a640 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa49800c3e0 tx=0x7fa49800c7e0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b4d0a640 1 -- 172.21.15.89:0/1476950452 reap_dead start 2024-08-19T10:44:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b676d640 1 -- 172.21.15.89:0/1476950452 shutdown_connections 2024-08-19T10:44:57.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b676d640 1 --2- 172.21.15.89:0/1476950452 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa4b006d250 0x7fa4b013a640 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b676d640 1 -- 172.21.15.89:0/1476950452 >> 172.21.15.89:0/1476950452 conn(0x7fa4b00a9090 msgr2=0x7fa4b0072d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:57.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b676d640 1 -- 172.21.15.89:0/1476950452 shutdown_connections 2024-08-19T10:44:57.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:56.965+0000 7fa4b676d640 1 -- 172.21.15.89:0/1476950452 wait complete. 2024-08-19T10:44:57.002 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:57.002 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:44:57.002 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:44:57.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient: get_monmap_and_config 2024-08-19T10:44:57.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient: build_initial_monmap 2024-08-19T10:44:57.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient: monmap: 2024-08-19T10:44:57.116 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:57.116 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.116 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:57.117567+0000 2024-08-19T10:44:57.116 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:57.117567+0000 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:57.117 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:57.118 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 1 Processor -- start 2024-08-19T10:44:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 1 -- start start 2024-08-19T10:44:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient: init 2024-08-19T10:44:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient: _add_conns ranks=[2,9,1,3,8,6,4,0,7,10,5] 2024-08-19T10:44:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda7c0a6860 0x7fda7c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient(hunting): picked mon.noname-b con 0x7fda7c0a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:57.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda7c0a7180 0x7fda7c0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient(hunting): picked mon.noname-a con 0x7fda7c0a7180 addr v2:172.21.15.89:3300/0 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fda7c139620 addr v1:172.21.15.190:6789/0 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda7c0a33d0 con 0x7fda7c139620 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fda7c0a8190 con 0x7fda7c0a7180 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c0a2ec0 con 0x7fda7c0a6860 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda821cc640 10 monclient(hunting): authenticate will time out at 1346.934650s 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda7c0a6860 0x7fda7c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda7c0a7180 0x7fda7c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda7c0a6860 0x7fda7c0a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56524/0 (socket says 172.21.15.89:56524) 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7b7fe640 1 -- 172.21.15.89:0/1464748496 learned_addr learned my addr 172.21.15.89:0/1464748496 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fda7c0a7180 0x7fda7c0a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52056/0 (socket says 172.21.15.89:52056) 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7b7fe640 10 monclient(hunting): get_auth_request con 0x7fda7c0a6860 auth_method 0 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 10 monclient(hunting): get_auth_request con 0x7fda7c0a7180 auth_method 0 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 10 monclient(hunting): handle_auth_done global_id 6336 payload 274 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 1 -- 172.21.15.89:0/1464748496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda7c0a6860 msgr2=0x7fda7c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 1 --2- 172.21.15.89:0/1464748496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda7c0a6860 0x7fda7c0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 1 -- 172.21.15.89:0/1464748496 >> v1:172.21.15.190:6789/0 conn(0x7fda7c139620 legacy=0x7fda7c139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 1 monclient: found mon.noname-a 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7affd640 1 -- 172.21.15.89:0/1464748496 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c0a4cd0 con 0x7fda7c0a7180 2024-08-19T10:44:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.113+0000 7fda7b7fe640 1 --2- 172.21.15.89:0/1464748496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda7c0a6860 0x7fda7c0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7affd640 1 -- 172.21.15.89:0/1464748496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c0a7180 msgr2=0x7fda7c0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7affd640 1 --2- 172.21.15.89:0/1464748496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c0a7180 0x7fda7c0a7560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fda700097c0 tx=0x7fda7001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34a5ddbc18928a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 1 -- 172.21.15.89:0/1464748496 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda700b0070 con 0x7fda7c0a7180 2024-08-19T10:44:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:44:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 10 monclient: dump: 2024-08-19T10:44:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.127 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.128 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda821cc640 5 monclient: authenticate success, global_id 6336 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda821cc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 1 -- 172.21.15.89:0/1464748496 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda700aa810 con 0x7fda7c0a7180 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 1 -- 172.21.15.89:0/1464748496 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda700a9300 con 0x7fda7c0a7180 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda821cc640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda821cc640 4 set_mon_vals no callback set 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:44:57.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.117+0000 7fda7a7fc640 10 monclient: dump: 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.131 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient: shutdown 2024-08-19T10:44:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 -- 172.21.15.89:0/1464748496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c0a7180 msgr2=0x7fda7c0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 --2- 172.21.15.89:0/1464748496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c0a7180 0x7fda7c0a7560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fda700097c0 tx=0x7fda7001c360 comp rx=0 tx=0).stop 2024-08-19T10:44:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 -- 172.21.15.89:0/1464748496 shutdown_connections 2024-08-19T10:44:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 --2- 172.21.15.89:0/1464748496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda7c0a7180 0x7fda7c0a7560 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 --2- 172.21.15.89:0/1464748496 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda7c0a6860 0x7fda7c0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 -- 172.21.15.89:0/1464748496 >> 172.21.15.89:0/1464748496 conn(0x7fda7c074b40 msgr2=0x7fda7c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 -- 172.21.15.89:0/1464748496 shutdown_connections 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 -- 172.21.15.89:0/1464748496 wait complete. 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient: build_initial_monmap 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient: monmap: 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.132 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 Processor -- start 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 -- start start 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient: init 2024-08-19T10:44:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient: _add_conns ranks=[7,8,3,4,5,6,0,2,1] 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fda7c0a6860 0x7fda7c13f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient(hunting): picked mon.noname-i con 0x7fda7c0a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda7c0a7180 0x7fda7c1d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient(hunting): picked mon.noname-e con 0x7fda7c0a7180 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda7c139620 0x7fda7c13d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient(hunting): picked mon.noname-g con 0x7fda7c139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c144430 con 0x7fda7c139620 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fda7c1442b0 con 0x7fda7c0a6860 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fda7c1445b0 con 0x7fda7c0a7180 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda821cc640 10 monclient(hunting): authenticate will time out at 1346.941780s 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda7c0a7180 0x7fda7c1d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda7c0a7180 0x7fda7c1d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50646/0 (socket says 172.21.15.89:50646) 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 1 -- 172.21.15.89:0/1118723820 learned_addr learned my addr 172.21.15.89:0/1118723820 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 10 monclient(hunting): get_auth_request con 0x7fda7c0a7180 auth_method 0 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 10 monclient(hunting): handle_auth_done global_id 6155 payload 824 2024-08-19T10:44:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7bfff640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda7c139620 0x7fda7c13d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 1 -- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fda7c0a6860 msgr2=0x7fda7c13f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7b7fe640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fda7c0a6860 0x7fda7c13f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fda7c0a6860 0x7fda7c13f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 1 -- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda7c139620 msgr2=0x7fda7c13d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda7c139620 0x7fda7c13d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 10 monclient: _finish_hunting 0 2024-08-19T10:44:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 1 monclient: found mon.noname-e 2024-08-19T10:44:57.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:57.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:57.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7affd640 1 -- 172.21.15.89:0/1118723820 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c13f420 con 0x7fda7c0a7180 2024-08-19T10:44:57.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7bfff640 10 monclient: get_auth_request con 0x7fda7c139620 auth_method 0 2024-08-19T10:44:57.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.121+0000 7fda7bfff640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda7c139620 0x7fda7c13d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:57.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7affd640 1 -- 172.21.15.89:0/1118723820 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda7c0a7180 msgr2=0x7fda7c1d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7affd640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda7c0a7180 0x7fda7c1d1a80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fda7001cc00 tx=0x7fda70019100 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda700b0040 con 0x7fda7c0a7180 2024-08-19T10:44:57.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:57.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 10 monclient: dump: 2024-08-19T10:44:57.141 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda821cc640 5 monclient: authenticate success, global_id 6155 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda821cc640 10 monclient: _renew_subs 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda821cc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda821cc640 1 -- 172.21.15.89:0/1118723820 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7c122cc0 con 0x7fda7c0a7180 2024-08-19T10:44:57.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda700b9070 con 0x7fda7c0a7180 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda700044b0 con 0x7fda7c0a7180 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 10 monclient: dump: 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.144 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda821cc640 10 monclient: _renew_subs 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda79ffb640 4 set_mon_vals no callback set 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda821cc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda821cc640 1 -- 172.21.15.89:0/1118723820 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7c140230 con 0x7fda7c0a7180 2024-08-19T10:44:57.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda821cc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda821cc640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c1d1fc0 0x7fda7c13d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda821cc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda821cc640 1 -- 172.21.15.89:0/1118723820 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fda7c0a6c40 con 0x7fda7c1d1fc0 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fda70004040 con 0x7fda7c0a7180 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7b7fe640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c1d1fc0 0x7fda7c13d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7b7fe640 10 monclient: get_auth_request con 0x7fda7c1d1fc0 auth_method 0 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda6803bdf0 0x7fda6803e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(230..230 src has 1..230) ==== 14564+0+0 (secure 0 0 0) 0x7fda700eadb0 con 0x7fda7c0a7180 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7bfff640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda6803bdf0 0x7fda6803e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7bfff640 10 monclient: get_auth_request con 0x7fda6803bdf0 auth_method 0 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7b7fe640 10 monclient(hunting): handle_auth_done global_id 6233 payload 824 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.125+0000 7fda7b7fe640 1 -- 172.21.15.89:0/1118723820 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda7c064590 con 0x7fda7c1d1fc0 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda7b7fe640 1 -- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c1d1fc0 msgr2=0x7fda7c13d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda7b7fe640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c1d1fc0 0x7fda7c13d870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0107c0 tx=0x7fda6c010bc0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda6c00f040 con 0x7fda7c1d1fc0 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda63fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda63fff640 10 monclient: dump: 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.147 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.148 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda7bfff640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda6803bdf0 0x7fda6803e2b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda74007e10 tx=0x7fda74007320 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fda6c01c400 con 0x7fda7c1d1fc0 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda63fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda63fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c1d1fc0 msgr2=0x7fda7c13d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda63fff640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c1d1fc0 0x7fda7c13d870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0107c0 tx=0x7fda6c010bc0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c1d1fc0 msgr2=0x7fda7c13d870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.129+0000 7fda63fff640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c1d1fc0 0x7fda7c13d870 secure :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0107c0 tx=0x7fda6c010bc0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.133+0000 7fda821cc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.133+0000 7fda821cc640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c13ddb0 0x7fda7c09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.133+0000 7fda821cc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.133+0000 7fda821cc640 1 -- 172.21.15.89:0/1118723820 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fda7c13e1c0 con 0x7fda7c13ddb0 2024-08-19T10:44:57.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c13ddb0 0x7fda7c09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 10 monclient: get_auth_request con 0x7fda7c13ddb0 auth_method 0 2024-08-19T10:44:57.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 10 monclient(hunting): handle_auth_done global_id 6242 payload 824 2024-08-19T10:44:57.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 1 -- 172.21.15.89:0/1118723820 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fda7c0008e0 con 0x7fda7c13ddb0 2024-08-19T10:44:57.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 1 -- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c13ddb0 msgr2=0x7fda7c09fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda7b7fe640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c13ddb0 0x7fda7c09fdf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0107c0 tx=0x7fda6c0046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda6c004490 con 0x7fda7c13ddb0 2024-08-19T10:44:57.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda63fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:44:57.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.137+0000 7fda63fff640 10 monclient: dump: 2024-08-19T10:44:57.152 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.152 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.152 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.152 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9601 (secure 0 0 0) 0x7fda6c004020 con 0x7fda7c13ddb0 2024-08-19T10:44:57.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda63fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:57.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda63fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:57.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c13ddb0 msgr2=0x7fda7c09fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda63fff640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c13ddb0 0x7fda7c09fdf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0107c0 tx=0x7fda6c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda63fff640 1 -- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c13ddb0 msgr2=0x7fda7c09fdf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:57.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda63fff640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fda7c13ddb0 0x7fda7c09fdf0 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0107c0 tx=0x7fda6c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.154 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:57.154 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:57.154 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 426912, 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:57.155 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 7, 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:57.156 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:57.157 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:57.158 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:57.159 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:57.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:57.161 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:57.162 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.164 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.165 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:57.168 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.169 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.170 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.171 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:57.172 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.173 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:57.174 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:57.177 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.178 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:57.179 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:57.179 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:57.179 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.179 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.179 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:57.179 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:57.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda821cc640 1 -- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda6803bdf0 msgr2=0x7fda6803e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda821cc640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda6803bdf0 0x7fda6803e2b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda74007e10 tx=0x7fda74007320 comp rx=0 tx=0).stop 2024-08-19T10:44:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda821cc640 10 monclient: shutdown 2024-08-19T10:44:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda821cc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda821cc640 1 -- 172.21.15.89:0/1118723820 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda7c0a7180 msgr2=0x7fda7c1d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda821cc640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda7c0a7180 0x7fda7c1d1a80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fda7001cc00 tx=0x7fda70019100 comp rx=0 tx=0).stop 2024-08-19T10:44:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda7bfff640 1 -- 172.21.15.89:0/1118723820 reap_dead start 2024-08-19T10:44:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda821cc640 1 -- 172.21.15.89:0/1118723820 shutdown_connections 2024-08-19T10:44:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda821cc640 1 --2- 172.21.15.89:0/1118723820 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda7c0a7180 0x7fda7c1d1a80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda821cc640 1 -- 172.21.15.89:0/1118723820 >> 172.21.15.89:0/1118723820 conn(0x7fda7c074b40 msgr2=0x7fda7c0a9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda821cc640 1 -- 172.21.15.89:0/1118723820 shutdown_connections 2024-08-19T10:44:57.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.141+0000 7fda821cc640 1 -- 172.21.15.89:0/1118723820 wait complete. 2024-08-19T10:44:57.182 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:57.182 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:44:57.182 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:44:57.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.173+0000 7f2f4bfff640 1 -- 172.21.15.190:0/220535232 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h3"}]=0 removed item id -25 name 'h3' from crush map v231) ==== 118+0+0 (secure 0 0 0) 0x7f2f3c0e7000 con 0x7f2f540a1d50 2024-08-19T10:44:57.183 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.173+0000 7f2f4bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h3"}] 2024-08-19T10:44:57.184 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.173+0000 7f2f4bfff640 10 monclient: _finish_command 2 = system:0 removed item id -25 name 'h3' from crush map 2024-08-19T10:44:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3785608554}) 2024-08-19T10:44:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1772716303}) 2024-08-19T10:44:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3785608554}) 2024-08-19T10:44:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1978944947}) 2024-08-19T10:44:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3785608554}) 2024-08-19T10:44:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=556821593}) 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3785608554}) 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1987180053}) 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3785608554}) 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4252992043}) 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3785608554}) 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=639360499}) 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3785608554}) 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1864732259}) 2024-08-19T10:44:57.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3785608554}) 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.161+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2988946943}) 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505971059}) 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2721250149}) 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505971059}) 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=588773349}) 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505971059}) 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3125968009}) 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505971059}) 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3559815072}) 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505971059}) 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3737944016}) 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505971059}) 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2236286580}) 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505971059}) 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2032236837}) 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505971059}) 2024-08-19T10:44:57.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.165+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3262363094}) 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=261045554}) 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2896718908}) 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=261045554}) 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1594756986}) 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=261045554}) 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=699708079}) 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=261045554}) 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2564081471}) 2024-08-19T10:44:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=261045554}) 2024-08-19T10:44:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2630626657}) 2024-08-19T10:44:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=261045554}) 2024-08-19T10:44:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4276353728}) 2024-08-19T10:44:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=261045554}) 2024-08-19T10:44:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1082319903}) 2024-08-19T10:44:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=261045554}) 2024-08-19T10:44:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.177+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2819991210}) 2024-08-19T10:44:57.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.281+0000 7f2f59ed3640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h3"}] 2024-08-19T10:44:57.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.281+0000 7f2f59ed3640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h3"}] 2024-08-19T10:44:57.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.281+0000 7f2f59ed3640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:44:57.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.281+0000 7f2f59ed3640 1 -- 172.21.15.190:0/220535232 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h3"} v 0) -- 0x7f2f54138860 con 0x7f2f540a1d50 2024-08-19T10:44:57.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f4bfff640 1 -- 172.21.15.190:0/220535232 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h3"}]=0 device 'h3' does not appear in the crush map v231) ==== 118+0+0 (secure 0 0 0) 0x7f2f3c0b5b10 con 0x7f2f540a1d50 2024-08-19T10:44:57.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f4bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h3"}] 2024-08-19T10:44:57.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f4bfff640 10 monclient: _finish_command 3 = system:0 device 'h3' does not appear in the crush map 2024-08-19T10:44:57.286 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h3' does not appear in the crush map 2024-08-19T10:44:57.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 -- 172.21.15.190:0/220535232 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f3403be00 msgr2=0x7f2f3403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f3403be00 0x7f2f3403e2c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2f54074610 tx=0x7f2f44009190 comp rx=0 tx=0).stop 2024-08-19T10:44:57.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 10 monclient: shutdown 2024-08-19T10:44:57.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:57.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 -- 172.21.15.190:0/220535232 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 msgr2=0x7f2f5413d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f5413d5b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0b0040 tx=0x7f2f3c0191e0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 -- 172.21.15.190:0/220535232 shutdown_connections 2024-08-19T10:44:57.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f540a2510 0x7f2f54137150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f3403be00 0x7f2f3403e2c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2f540a2130 0x7f2f5413daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 --2- 172.21.15.190:0/220535232 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f540a1d50 0x7f2f5413d5b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 -- 172.21.15.190:0/220535232 >> 172.21.15.190:0/220535232 conn(0x7f2f540a84b0 msgr2=0x7f2f54135760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:57.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 -- 172.21.15.190:0/220535232 shutdown_connections 2024-08-19T10:44:57.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.285+0000 7f2f59ed3640 1 -- 172.21.15.190:0/220535232 wait complete. 2024-08-19T10:44:57.303 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:44:57.304 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h4 2024-08-19T10:44:57.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient: get_monmap_and_config 2024-08-19T10:44:57.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient: build_initial_monmap 2024-08-19T10:44:57.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient: monmap: 2024-08-19T10:44:57.307 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:57.307 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:57.309040+0000 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:57.309040+0000 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:57.308 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:57.309 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 1 Processor -- start 2024-08-19T10:44:57.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 1 -- start start 2024-08-19T10:44:57.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient: init 2024-08-19T10:44:57.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:57.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient: _add_conns ranks=[8,5,7,2,0,9,6,10,1,4,3] 2024-08-19T10:44:57.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63d40a3d00 0x7f63d40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient(hunting): picked mon.noname-e con 0x7f63d40a3d00 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63d40a1a00 0x7f63d40a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient(hunting): picked mon.noname-h con 0x7f63d40a1a00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d409b1d0 0x7f63d409b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient(hunting): picked mon.noname-i con 0x7f63d409b1d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f63d409d3d0 con 0x7f63d40a1a00 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f63d406fd00 con 0x7f63d409b1d0 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f63d40a4b60 con 0x7f63d40a3d00 2024-08-19T10:44:57.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63db78b640 10 monclient(hunting): authenticate will time out at 1347.125535s 2024-08-19T10:44:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63d40a3d00 0x7f63d40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63d40a3d00 0x7f63d40a40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50654/0 (socket says 172.21.15.89:50654) 2024-08-19T10:44:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 1 -- 172.21.15.89:0/545913957 learned_addr learned my addr 172.21.15.89:0/545913957 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 10 monclient(hunting): get_auth_request con 0x7f63d40a3d00 auth_method 0 2024-08-19T10:44:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9d28640 1 --2- 172.21.15.89:0/545913957 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d409b1d0 0x7f63d409b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d8d26640 1 --2- 172.21.15.89:0/545913957 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63d40a1a00 0x7f63d40a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 10 monclient(hunting): handle_auth_done global_id 6164 payload 274 2024-08-19T10:44:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 1 -- 172.21.15.89:0/545913957 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d409b1d0 msgr2=0x7f63d409b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 1 --2- 172.21.15.89:0/545913957 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d409b1d0 0x7f63d409b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 1 -- 172.21.15.89:0/545913957 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63d40a1a00 msgr2=0x7f63d40a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 1 --2- 172.21.15.89:0/545913957 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63d40a1a00 0x7f63d40a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 10 monclient: _finish_hunting 0 2024-08-19T10:44:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 1 monclient: found mon.noname-e 2024-08-19T10:44:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:57.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9527640 1 -- 172.21.15.89:0/545913957 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d406db50 con 0x7f63d40a3d00 2024-08-19T10:44:57.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d8d26640 10 monclient: get_auth_request con 0x7f63d40a1a00 auth_method 0 2024-08-19T10:44:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9d28640 10 monclient: get_auth_request con 0x7f63d409b1d0 auth_method 0 2024-08-19T10:44:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d8d26640 1 --2- 172.21.15.89:0/545913957 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63d40a1a00 0x7f63d40a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:57.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.305+0000 7f63d9d28640 1 --2- 172.21.15.89:0/545913957 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d409b1d0 0x7f63d409b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:57.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63d9527640 1 -- 172.21.15.89:0/545913957 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63d40a3d00 msgr2=0x7f63d40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63d9527640 1 --2- 172.21.15.89:0/545913957 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63d40a3d00 0x7f63d40a40e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009eb0 tx=0x7f63cc01c8f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=8b7a22e0d1295101 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 1 -- 172.21.15.89:0/545913957 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63cc0091b0 con 0x7f63d40a3d00 2024-08-19T10:44:57.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 10 monclient: dump: 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.325 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 10 monclient: _finish_auth 0 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 1 -- 172.21.15.89:0/545913957 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63cc0a6950 con 0x7f63d40a3d00 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 5 monclient: authenticate success, global_id 6164 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 1 -- 172.21.15.89:0/545913957 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63cc0af480 con 0x7f63d40a3d00 2024-08-19T10:44:57.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 4 set_mon_vals no callback set 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63cbfff640 10 monclient: dump: 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.327 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 10 monclient: shutdown 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 1 -- 172.21.15.89:0/545913957 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63d40a3d00 msgr2=0x7f63d40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 1 --2- 172.21.15.89:0/545913957 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63d40a3d00 0x7f63d40a40e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009eb0 tx=0x7f63cc01c8f0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 1 -- 172.21.15.89:0/545913957 shutdown_connections 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 1 --2- 172.21.15.89:0/545913957 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d409b1d0 0x7f63d409b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 1 --2- 172.21.15.89:0/545913957 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63d40a1a00 0x7f63d40a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 1 --2- 172.21.15.89:0/545913957 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63d40a3d00 0x7f63d40a40e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 1 -- 172.21.15.89:0/545913957 >> 172.21.15.89:0/545913957 conn(0x7f63d40a4df0 msgr2=0x7f63d40a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 1 -- 172.21.15.89:0/545913957 shutdown_connections 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.309+0000 7f63db78b640 1 -- 172.21.15.89:0/545913957 wait complete. 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient: build_initial_monmap 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient: monmap: 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:57.329 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 1 Processor -- start 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 1 -- start start 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient: init 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient: _add_conns ranks=[6,5,2,3,0,7,8,1,4] 2024-08-19T10:44:57.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63d409b1d0 0x7f63d413ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient(hunting): picked mon.noname-d con 0x7f63d409b1d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63d40a1a00 0x7f63d409f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient(hunting): picked mon.noname-h con 0x7f63d40a1a00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63d40a3d00 0x7f63d409fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient(hunting): picked mon.noname-b con 0x7f63d40a3d00 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f63d40a5bd0 con 0x7f63d40a3d00 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f63d409bf60 con 0x7f63d40a1a00 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f63d4141140 con 0x7f63d409b1d0 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:57.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63db78b640 10 monclient(hunting): authenticate will time out at 1347.133077s 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63d40a3d00 0x7f63d409fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63d40a3d00 0x7f63d409fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:56530/0 (socket says 172.21.15.89:56530) 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 1 -- 172.21.15.89:0/1174418706 learned_addr learned my addr 172.21.15.89:0/1174418706 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 10 monclient(hunting): get_auth_request con 0x7f63d40a3d00 auth_method 0 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9527640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63d409b1d0 0x7f63d413ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9527640 10 monclient(hunting): get_auth_request con 0x7f63d409b1d0 auth_method 0 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9527640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9527640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9527640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d8d26640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63d40a1a00 0x7f63d409f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d8d26640 10 monclient(hunting): get_auth_request con 0x7f63d40a1a00 auth_method 0 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d8d26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d8d26640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d8d26640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 10 monclient(hunting): handle_auth_done global_id 6284 payload 824 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 1 -- 172.21.15.89:0/1174418706 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63d409b1d0 msgr2=0x7f63d413ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f63d409b1d0 0x7f63d413ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 1 -- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63d40a1a00 msgr2=0x7f63d409f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63d40a1a00 0x7f63d409f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 10 monclient: _finish_hunting 0 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 1 monclient: found mon.noname-b 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 1 -- 172.21.15.89:0/1174418706 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d40a6740 con 0x7f63d40a3d00 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d8d26640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63d40a1a00 0x7f63d409f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 1 -- 172.21.15.89:0/1174418706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63d40a3d00 msgr2=0x7f63d409fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63d9d28640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63d40a3d00 0x7f63d409fd30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c7f0 tx=0x7f63c400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63c4015070 con 0x7f63d40a3d00 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63c9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:44:57.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.313+0000 7f63c9ffb640 10 monclient: dump: 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.335 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63c4007ca0 con 0x7f63d40a3d00 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63db78b640 5 monclient: authenticate success, global_id 6284 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63c4007e50 con 0x7f63d40a3d00 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63cb7fe640 4 set_mon_vals no callback set 2024-08-19T10:44:57.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 10 monclient: dump: 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.337 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63db78b640 10 monclient: _renew_subs 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63db78b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63db78b640 1 -- 172.21.15.89:0/1174418706 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d40a61f0 con 0x7f63d40a3d00 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63db78b640 10 monclient: _renew_subs 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63db78b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63db78b640 1 -- 172.21.15.89:0/1174418706 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d413d200 con 0x7f63d40a3d00 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63db78b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63db78b640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d40a0270 0x7f63d4099230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63db78b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63db78b640 1 -- 172.21.15.89:0/1174418706 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f63d413daf0 con 0x7f63d40a0270 2024-08-19T10:44:57.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f63c4018070 con 0x7f63d40a3d00 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d9527640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d40a0270 0x7f63d4099230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d9527640 10 monclient: get_auth_request con 0x7f63d40a0270 auth_method 0 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d9527640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d9527640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d9527640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63c003bfa0 0x7f63c003e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(231..231 src has 1..231) ==== 14354+0+0 (secure 0 0 0) 0x7f63c4050fa0 con 0x7f63d40a3d00 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d9527640 10 monclient(hunting): handle_auth_done global_id 15595 payload 824 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d9527640 1 -- 172.21.15.89:0/1174418706 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63d409b5e0 con 0x7f63d40a0270 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d8d26640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63c003bfa0 0x7f63c003e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.317+0000 7f63d8d26640 10 monclient: get_auth_request con 0x7f63c003bfa0 auth_method 0 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63d9527640 1 -- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d40a0270 msgr2=0x7f63d4099230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63d9527640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d40a0270 0x7f63d4099230 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009e80 tx=0x7f63cc019690 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63cc0044b0 con 0x7f63d40a0270 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63c9ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63c9ffb640 10 monclient: dump: 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63d8d26640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63c003bfa0 0x7f63c003e460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f63d40a8170 tx=0x7f63d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f63cc004040 con 0x7f63d40a0270 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63c9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63c9ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d40a0270 msgr2=0x7f63d4099230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63c9ffb640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d40a0270 0x7f63d4099230 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009e80 tx=0x7f63cc019690 comp rx=0 tx=0).stop 2024-08-19T10:44:57.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d40a0270 msgr2=0x7f63d4099230 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:57.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.321+0000 7f63c9ffb640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d40a0270 0x7f63d4099230 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009e80 tx=0x7f63cc019690 comp rx=0 tx=0).stop 2024-08-19T10:44:57.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63db78b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63db78b640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d4099770 0x7f63d4099b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63db78b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63db78b640 1 -- 172.21.15.89:0/1174418706 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f63d4064590 con 0x7f63d4099770 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63d9527640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d4099770 0x7f63d4099b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63d9527640 10 monclient: get_auth_request con 0x7f63d4099770 auth_method 0 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63d9527640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63d9527640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63d9527640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63d9527640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63d9527640 10 monclient(hunting): handle_auth_done global_id 15604 payload 824 2024-08-19T10:44:57.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.325+0000 7f63d9527640 1 -- 172.21.15.89:0/1174418706 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f63d413d7a0 con 0x7f63d4099770 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63d9527640 1 -- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d4099770 msgr2=0x7f63d4099b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63d9527640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d4099770 0x7f63d4099b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f63cc01c720 tx=0x7f63cc0a9000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63cc0a6bb0 con 0x7f63d4099770 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63c9ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63c9ffb640 10 monclient: dump: 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.345 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f63cc0c6070 con 0x7f63d4099770 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63c9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63c9ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d4099770 msgr2=0x7f63d4099b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63c9ffb640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d4099770 0x7f63d4099b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f63cc01c720 tx=0x7f63cc0a9000 comp rx=0 tx=0).stop 2024-08-19T10:44:57.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63c9ffb640 1 -- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d4099770 msgr2=0x7f63d4099b80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:44:57.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63c9ffb640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63d4099770 0x7f63d4099b80 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f63cc01c720 tx=0x7f63cc0a9000 comp rx=0 tx=0).stop 2024-08-19T10:44:57.347 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:44:57.347 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:57.347 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:57.347 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:44:57.347 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 72, 2024-08-19T10:44:57.347 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 274921, 2024-08-19T10:44:57.347 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 7, 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:44:57.348 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:57.349 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.350 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.351 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:44:57.351 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:44:57.351 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:44:57.351 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:44:57.351 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:44:57.351 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:44:57.351 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:57.351 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:44:57.351 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:44:57.352 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:44:57.353 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:44:57.353 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:44:57.353 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:44:57.353 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:44:57.353 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:44:57.353 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:57.353 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:44:57.353 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.353 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:44:57.353 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.353 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:44:57.354 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:44:57.354 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.354 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.354 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.354 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.354 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:44:57.354 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.354 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.354 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.354 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:44:57.355 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.356 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.357 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.357 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.357 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:44:57.357 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:44:57.357 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.357 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.357 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.357 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.357 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.357 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:44:57.357 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.358 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.359 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:44:57.360 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.361 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.362 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:44:57.363 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.364 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:44:57.365 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:44:57.366 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.366 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.366 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.367 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.367 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:44:57.367 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:44:57.367 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.367 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.367 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.367 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.367 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:44:57.367 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:44:57.369 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:44:57.370 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:44:57.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:44:57.372 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:44:57.372 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:44:57.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63db78b640 1 -- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63c003bfa0 msgr2=0x7f63c003e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63db78b640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63c003bfa0 0x7f63c003e460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f63d40a8170 tx=0x7f63d0008040 comp rx=0 tx=0).stop 2024-08-19T10:44:57.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63db78b640 10 monclient: shutdown 2024-08-19T10:44:57.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63db78b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:57.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63db78b640 1 -- 172.21.15.89:0/1174418706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63d40a3d00 msgr2=0x7f63d409fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63d9d28640 1 -- 172.21.15.89:0/1174418706 reap_dead start 2024-08-19T10:44:57.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63db78b640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63d40a3d00 0x7f63d409fd30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c7f0 tx=0x7f63c400ccb0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63db78b640 1 -- 172.21.15.89:0/1174418706 shutdown_connections 2024-08-19T10:44:57.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63db78b640 1 --2- 172.21.15.89:0/1174418706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63d40a3d00 0x7f63d409fd30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63db78b640 1 -- 172.21.15.89:0/1174418706 >> 172.21.15.89:0/1174418706 conn(0x7f63d40a4df0 msgr2=0x7f63d4073230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:57.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63db78b640 1 -- 172.21.15.89:0/1174418706 shutdown_connections 2024-08-19T10:44:57.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.329+0000 7f63db78b640 1 -- 172.21.15.89:0/1174418706 wait complete. 2024-08-19T10:44:57.374 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:44:57.375 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-08-19T10:44:57.375 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-08-19T10:44:57.375 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-08-19T10:44:57.375 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-08-19T10:44:57.375 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-08-19T10:44:57.375 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:44:57.388 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.388 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139384214}) 2024-08-19T10:44:57.388 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3234069806}) 2024-08-19T10:44:57.388 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.388 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139384214}) 2024-08-19T10:44:57.388 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2756716281}) 2024-08-19T10:44:57.388 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.388 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139384214}) 2024-08-19T10:44:57.388 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4045428027}) 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139384214}) 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=435935511}) 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139384214}) 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1520989208}) 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139384214}) 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1969203861}) 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139384214}) 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3292489245}) 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:44:57.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139384214}) 2024-08-19T10:44:57.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:44:57.381+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3201136639}) 2024-08-19T10:44:57.390 INFO:tasks.ceph.mon.b:Stopped 2024-08-19T10:44:57.391 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-08-19T10:44:57.391 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:44:57.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.417+0000 7f2f738f4640 10 monclient: get_monmap_and_config 2024-08-19T10:44:57.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.417+0000 7f2f738f4640 10 monclient: build_initial_monmap 2024-08-19T10:44:57.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient: monmap: 2024-08-19T10:44:57.422 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:57.422 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.422 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:44:57.423658+0000 2024-08-19T10:44:57.422 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:44:57.423658+0000 2024-08-19T10:44:57.422 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:57.422 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:57.422 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:57.422 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:57.423 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:57.423 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:57.423 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:57.423 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:57.423 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:57.423 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:57.423 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:57.423 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:57.423 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:57.423 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:57.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 1 Processor -- start 2024-08-19T10:44:57.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 1 -- start start 2024-08-19T10:44:57.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient: init 2024-08-19T10:44:57.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:57.426 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient: _add_conns ranks=[7,3,5,10,6,2,8,9,1,4,0] 2024-08-19T10:44:57.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f6c069410 0x7f2f6c0a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient(hunting): picked mon.noname-i con 0x7f2f6c069410 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:44:57.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c0a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient(hunting): picked mon.noname-g con 0x7f2f6c0a41c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:57.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2f6c13aa60 0x7f2f6c13ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient(hunting): picked mon.noname-h con 0x7f2f6c13aa60 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:57.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c09e520 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.427 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2f6c0a4ae0 con 0x7f2f6c13aa60 2024-08-19T10:44:57.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2f6c0697f0 con 0x7f2f6c069410 2024-08-19T10:44:57.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:57.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f738f4640 10 monclient(hunting): authenticate will time out at 1342.994045s 2024-08-19T10:44:57.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c0a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71690640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f6c069410 0x7f2f6c0a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c0a45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46182/0 (socket says 172.21.15.190:46182) 2024-08-19T10:44:57.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71690640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f6c069410 0x7f2f6c0a3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:48910/0 (socket says 172.21.15.190:48910) 2024-08-19T10:44:57.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 1 -- 172.21.15.190:0/2697210239 learned_addr learned my addr 172.21.15.190:0/2697210239 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:57.428 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 10 monclient(hunting): get_auth_request con 0x7f2f6c0a41c0 auth_method 0 2024-08-19T10:44:57.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71690640 10 monclient(hunting): get_auth_request con 0x7f2f6c069410 auth_method 0 2024-08-19T10:44:57.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71690640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71690640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71690640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71e91640 1 --2- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2f6c13aa60 0x7f2f6c13ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71e91640 10 monclient(hunting): get_auth_request con 0x7f2f6c13aa60 auth_method 0 2024-08-19T10:44:57.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71e91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71e91640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71e91640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71690640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71690640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71690640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71e91640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71e91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71e91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 10 monclient(hunting): handle_auth_done global_id 34680 payload 274 2024-08-19T10:44:57.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 1 -- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f6c069410 msgr2=0x7f2f6c0a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:57.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 1 --2- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f6c069410 0x7f2f6c0a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 1 -- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2f6c13aa60 msgr2=0x7f2f6c13ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:57.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 1 --2- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2f6c13aa60 0x7f2f6c13ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 10 monclient: _finish_hunting 0 2024-08-19T10:44:57.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 1 monclient: found mon.noname-g 2024-08-19T10:44:57.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:57.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:57.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f70e8f640 1 -- 172.21.15.190:0/2697210239 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c074a20 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.421+0000 7f2f71690640 1 --2- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f6c069410 0x7f2f6c0a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f70e8f640 1 -- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 msgr2=0x7f2f6c0a45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f70e8f640 1 --2- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c0a45a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0098d0 tx=0x7f2f5c01c4d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=48a686fe3738508a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 1 -- 172.21.15.190:0/2697210239 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f5c0b0070 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 10 monclient: dump: 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.437 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 10 monclient: _finish_auth 0 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 10 monclient: _check_auth_tickets 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:57.438 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 5 monclient: authenticate success, global_id 34680 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 1 -- 172.21.15.190:0/2697210239 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c0aa810 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 4 set_mon_vals no callback set 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 1 -- 172.21.15.190:0/2697210239 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f5c0aae30 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f63fff640 10 monclient: dump: 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.439 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 10 monclient: shutdown 2024-08-19T10:44:57.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 1 -- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 msgr2=0x7f2f6c0a45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 1 --2- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c0a45a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0098d0 tx=0x7f2f5c01c4d0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 1 -- 172.21.15.190:0/2697210239 shutdown_connections 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 1 --2- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2f6c13aa60 0x7f2f6c13ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 1 --2- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c0a45a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 1 --2- 172.21.15.190:0/2697210239 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2f6c069410 0x7f2f6c0a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 1 -- 172.21.15.190:0/2697210239 >> 172.21.15.190:0/2697210239 conn(0x7f2f6c0a4d70 msgr2=0x7f2f6c0a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 1 -- 172.21.15.190:0/2697210239 shutdown_connections 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.425+0000 7f2f738f4640 1 -- 172.21.15.190:0/2697210239 wait complete. 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient: build_initial_monmap 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient: monmap: 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:44:57.441 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:57.442 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 1 Processor -- start 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 1 -- start start 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient: init 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient: _add_conns ranks=[5,3,4,7,8,0,1,2,6] 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2f6c069410 0x7f2f6c070c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient(hunting): picked mon.noname-h con 0x7f2f6c069410 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient(hunting): picked mon.noname-g con 0x7f2f6c0a41c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6c13aa60 0x7f2f6c0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient(hunting): picked mon.noname-c con 0x7f2f6c13aa60 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:57.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c09e290 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f2f6c09e110 con 0x7f2f6c13aa60 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2f6c09d270 con 0x7f2f6c069410 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f738f4640 10 monclient(hunting): authenticate will time out at 1343.001196s 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71690640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2f6c069410 0x7f2f6c070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71690640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2f6c069410 0x7f2f6c070c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:42618/0 (socket says 172.21.15.190:42618) 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c071170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46184/0 (socket says 172.21.15.190:46184) 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71690640 1 -- 172.21.15.190:0/860113610 learned_addr learned my addr 172.21.15.190:0/860113610 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71690640 10 monclient(hunting): get_auth_request con 0x7f2f6c069410 auth_method 0 2024-08-19T10:44:57.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71690640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71690640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71690640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 10 monclient(hunting): get_auth_request con 0x7f2f6c0a41c0 auth_method 0 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71690640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71690640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71690640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 10 monclient(hunting): handle_auth_done global_id 34689 payload 824 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 1 -- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2f6c069410 msgr2=0x7f2f6c070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2f6c069410 0x7f2f6c070c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 1 -- 172.21.15.190:0/860113610 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6c13aa60 msgr2=0x7f2f6c0716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71e91640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6c13aa60 0x7f2f6c0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6c13aa60 0x7f2f6c0716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 10 monclient: _finish_hunting 0 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 1 monclient: found mon.noname-g 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 1 -- 172.21.15.190:0/860113610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c13d110 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71e91640 10 monclient: get_auth_request con 0x7f2f6c13aa60 auth_method 0 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f71e91640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6c13aa60 0x7f2f6c0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:44:57.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 1 -- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 msgr2=0x7f2f6c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.429+0000 7f2f70e8f640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c071170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0ad000 tx=0x7f2f5c019270 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 1 -- 172.21.15.190:0/860113610 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f5c0b0040 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 10 monclient: dump: 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.447 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 10 monclient: _finish_auth 0 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f738f4640 5 monclient: authenticate success, global_id 34689 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f738f4640 10 monclient: _renew_subs 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 1 -- 172.21.15.190:0/860113610 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c0044b0 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.448 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f738f4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f738f4640 1 -- 172.21.15.190:0/860113610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c0a4810 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 1 -- 172.21.15.190:0/860113610 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2f5c0aad10 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 10 monclient: dump: 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:44:57.449 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f637fe640 4 set_mon_vals no callback set 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f738f4640 10 monclient: _renew_subs 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f738f4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f738f4640 1 -- 172.21.15.190:0/860113610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c074160 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.450 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f738f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:57.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f738f4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:57.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f738f4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:57.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f738f4640 1 -- 172.21.15.190:0/860113610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c09d5a0 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f70e8f640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c071170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0ad000 tx=0x7f2f5c019270 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f2f5c0ea100 osd_map(231..231 src has 1..231) 2024-08-19T10:44:57.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 1 -- 172.21.15.190:0/860113610 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2f5c004040 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f61ffb640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f5003be00 0x7f2f5003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f71690640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f5003be00 0x7f2f5003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.451 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.433+0000 7f2f71690640 10 monclient: get_auth_request con 0x7f2f5003be00 auth_method 0 2024-08-19T10:44:57.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.437+0000 7f2f71690640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f5003be00 0x7f2f5003e2c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800b820 tx=0x7f2f6800b3a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.452 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:57.437+0000 7f2f61ffb640 1 -- 172.21.15.190:0/860113610 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(231..231 src has 1..231) ==== 14354+0+0 (secure 0 0 0) 0x7f2f5c0ea100 con 0x7f2f6c0a41c0 2024-08-19T10:44:57.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient: get_monmap_and_config 2024-08-19T10:44:57.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient: build_initial_monmap 2024-08-19T10:44:57.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient: monmap: 2024-08-19T10:44:57.573 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:57.573 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.573 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:44:57.574676+0000 2024-08-19T10:44:57.573 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:44:57.574676+0000 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:44:57.574 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:44:57.575 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 1 Processor -- start 2024-08-19T10:44:57.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 1 -- start start 2024-08-19T10:44:57.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient: init 2024-08-19T10:44:57.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:57.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient: _add_conns ranks=[4,10,0,5,3,9,7,8,2,6,1] 2024-08-19T10:44:57.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f278c09b8e0 0x7f278c09bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient(hunting): picked mon.noname-c con 0x7f278c09b8e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f278c09ddd0 0x7f278c0a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient(hunting): picked mon.noname-f con 0x7f278c09ddd0 addr v2:172.21.15.190:3300/0 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f278c0a4240 addr v1:172.21.15.89:6789/0 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f278c06db50 con 0x7f278c0a4240 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f278c09d760 con 0x7f278c09ddd0 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f278c09d9b0 con 0x7f278c09b8e0 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2793e11640 10 monclient(hunting): authenticate will time out at 1347.391281s 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f278c09b8e0 0x7f278c09bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f278c09b8e0 0x7f278c09bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:36728/0 (socket says 172.21.15.89:36728) 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 1 -- 172.21.15.89:0/1651618788 learned_addr learned my addr 172.21.15.89:0/1651618788 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 10 monclient(hunting): get_auth_request con 0x7f278c09b8e0 auth_method 0 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 10 monclient(hunting): handle_auth_done global_id 44248 payload 274 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 1 -- 172.21.15.89:0/1651618788 >> v2:172.21.15.190:3300/0 conn(0x7f278c09ddd0 msgr2=0x7f278c0a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f27913ac640 1 --2- 172.21.15.89:0/1651618788 >> v2:172.21.15.190:3300/0 conn(0x7f278c09ddd0 0x7f278c0a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 1 --2- 172.21.15.89:0/1651618788 >> v2:172.21.15.190:3300/0 conn(0x7f278c09ddd0 0x7f278c0a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 1 -- 172.21.15.89:0/1651618788 >> v1:172.21.15.89:6789/0 conn(0x7f278c0a4240 legacy=0x7f278c1404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:44:57.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 10 monclient: _finish_hunting 0 2024-08-19T10:44:57.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 1 monclient: found mon.noname-c 2024-08-19T10:44:57.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:57.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:44:57.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.569+0000 7f2791bad640 1 -- 172.21.15.89:0/1651618788 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c06fd30 con 0x7f278c09b8e0 2024-08-19T10:44:57.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 1 -- 172.21.15.89:0/1651618788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f278c09b8e0 msgr2=0x7f278c09bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 1 --2- 172.21.15.89:0/1651618788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f278c09b8e0 0x7f278c09bcc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f277c009870 tx=0x7f277c01c420 comp rx=0 tx=0).ready entity=mon.4 client_cookie=7af4bfcac8e6180c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 1 -- 172.21.15.89:0/1651618788 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f277c0b0070 con 0x7f278c09b8e0 2024-08-19T10:44:57.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:44:57.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 10 monclient: dump: 2024-08-19T10:44:57.581 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.581 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.581 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.581 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.581 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 10 monclient: _finish_auth 0 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 10 monclient: _check_auth_tickets 2024-08-19T10:44:57.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:57.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 5 monclient: authenticate success, global_id 44248 2024-08-19T10:44:57.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:44:57.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 1 -- 172.21.15.89:0/1651618788 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f277c0aa810 con 0x7f278c09b8e0 2024-08-19T10:44:57.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:57.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient: get_monmap_and_config success 2024-08-19T10:44:57.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 4 set_mon_vals no callback set 2024-08-19T10:44:57.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 1 -- 172.21.15.89:0/1651618788 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f277c0a93e0 con 0x7f278c09b8e0 2024-08-19T10:44:57.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:44:57.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2790bab640 10 monclient: dump: 2024-08-19T10:44:57.584 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.584 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.584 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.584 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.584 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.584 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.584 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.584 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.584 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient: shutdown 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 -- 172.21.15.89:0/1651618788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f278c09b8e0 msgr2=0x7f278c09bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 --2- 172.21.15.89:0/1651618788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f278c09b8e0 0x7f278c09bcc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f277c009870 tx=0x7f277c01c420 comp rx=0 tx=0).stop 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 -- 172.21.15.89:0/1651618788 shutdown_connections 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 --2- 172.21.15.89:0/1651618788 >> v2:172.21.15.190:3300/0 conn(0x7f278c09ddd0 0x7f278c0a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 --2- 172.21.15.89:0/1651618788 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f278c09b8e0 0x7f278c09bcc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 -- 172.21.15.89:0/1651618788 >> 172.21.15.89:0/1651618788 conn(0x7f278c0a4df0 msgr2=0x7f278c0a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 -- 172.21.15.89:0/1651618788 shutdown_connections 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 -- 172.21.15.89:0/1651618788 wait complete. 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient: build_initial_monmap 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient: monmap: 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:44:57.586 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:44:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:44:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:44:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:44:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:44:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:44:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:44:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:44:57.587 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 Processor -- start 2024-08-19T10:44:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 -- start start 2024-08-19T10:44:57.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient: init 2024-08-19T10:44:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient: _reopen_session rank -1 2024-08-19T10:44:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient: _add_conns ranks=[8,2,1,6,3,4,0,5,7] 2024-08-19T10:44:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f278c09b8e0 0x7f278c09a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient(hunting): picked mon.noname-e con 0x7f278c09b8e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:44:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f278c09ddd0 0x7f278c09acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient(hunting): picked mon.noname-b con 0x7f278c09ddd0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:44:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f278c0a4240 0x7f278c092380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient(hunting): picked mon.noname-f con 0x7f278c0a4240 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f278c069300 con 0x7f278c0a4240 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f278c0a5c00 con 0x7f278c09ddd0 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient(hunting): start opening mon connection 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f278c1443f0 con 0x7f278c09b8e0 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient(hunting): _renew_subs 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2793e11640 10 monclient(hunting): authenticate will time out at 1347.398629s 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f27913ac640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f278c09ddd0 msgr2=0x7f278c09acc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f27913ac640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f278c09ddd0 0x7f278c09acc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f278c09b8e0 0x7f278c09a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f278c09b8e0 0x7f278c09a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50660/0 (socket says 172.21.15.89:50660) 2024-08-19T10:44:57.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 1 -- 172.21.15.89:0/594764920 learned_addr learned my addr 172.21.15.89:0/594764920 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 10 monclient(hunting): get_auth_request con 0x7f278c09b8e0 auth_method 0 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 10 monclient(hunting): handle_auth_done global_id 6173 payload 824 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 1 -- 172.21.15.89:0/594764920 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f278c09ddd0 msgr2=0x7f278c09acc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f278c09ddd0 0x7f278c09acc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 1 -- 172.21.15.89:0/594764920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f278c0a4240 msgr2=0x7f278c092380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f27923ae640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f278c0a4240 0x7f278c092380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f278c0a4240 0x7f278c092380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:57.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 10 monclient: _finish_hunting 0 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 1 monclient: found mon.noname-e 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.577+0000 7f2791bad640 1 -- 172.21.15.89:0/594764920 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c1d1a40 con 0x7f278c09b8e0 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2791bad640 1 -- 172.21.15.89:0/594764920 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f278c09b8e0 msgr2=0x7f278c09a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2791bad640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f278c09b8e0 0x7f278c09a780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f277c0097c0 tx=0x7f277c0191c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2791bad640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f278c09b8e0 0x7f278c09a780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f277c0097c0 tx=0x7f277c0191c0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f277c0b9070 mon_map magic: 0 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 1 -- 172.21.15.89:0/594764920 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f277c0b9070 con 0x7f278c09b8e0 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 10 monclient: dump: 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.594 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.595 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 10 monclient: _finish_auth 0 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 10 monclient: _check_auth_tickets 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2793e11640 5 monclient: authenticate success, global_id 6173 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 1 -- 172.21.15.89:0/594764920 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f277c0044b0 con 0x7f278c09b8e0 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2793e11640 10 monclient: _renew_subs 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2793e11640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2793e11640 1 -- 172.21.15.89:0/594764920 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c094f30 con 0x7f278c09b8e0 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2793e11640 10 monclient: _renew_subs 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2793e11640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27837fe640 4 set_mon_vals no callback set 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2793e11640 1 -- 172.21.15.89:0/594764920 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c093ed0 con 0x7f278c09b8e0 2024-08-19T10:44:57.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2793e11640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2793e11640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2793e11640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f2793e11640 1 -- 172.21.15.89:0/594764920 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278c0a37d0 con 0x7f278c09b8e0 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 1 -- 172.21.15.89:0/594764920 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f277c0b0040 con 0x7f278c09b8e0 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 10 monclient: dump: 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:44:57.597 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 1 -- 172.21.15.89:0/594764920 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f277c004040 con 0x7f278c09b8e0 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27827fc640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f277003be40 0x7f277003e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27913ac640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f277003be40 0x7f277003e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:44:57.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.581+0000 7f27913ac640 10 monclient: get_auth_request con 0x7f277003be40 auth_method 0 2024-08-19T10:44:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.585+0000 7f27827fc640 1 -- 172.21.15.89:0/594764920 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:44:57.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.585+0000 7f27913ac640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f277003be40 0x7f277003e300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f278c0a17b0 tx=0x7f2788008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:44:57.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:57.589+0000 7f27827fc640 1 -- 172.21.15.89:0/594764920 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(231..231 src has 1..231) ==== 14354+0+0 (secure 0 0 0) 0x7f277c0ea830 con 0x7f278c09b8e0 2024-08-19T10:44:58.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.205+0000 7f27827fc640 1 -- 172.21.15.89:0/594764920 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f278c0a37d0 con 0x7f278c09b8e0 2024-08-19T10:44:58.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.205+0000 7f27827fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:58.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.205+0000 7f27827fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:58.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:58.261+0000 7f2f61ffb640 1 -- 172.21.15.190:0/860113610 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f2f5c016150 con 0x7f2f6c0a41c0 2024-08-19T10:44:58.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:58.261+0000 7f2f61ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:44:58.266 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:58.261+0000 7f2f61ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:44:58.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.437+0000 7f2793e11640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:44:58.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.437+0000 7f2793e11640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:44:58.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.437+0000 7f2793e11640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:44:58.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.437+0000 7f2793e11640 1 -- 172.21.15.89:0/594764920 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f278c076bd0 con 0x7f278c09b8e0 2024-08-19T10:44:58.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.437+0000 7f27827fc640 1 -- 172.21.15.89:0/594764920 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7f278c076bd0 con 0x7f278c09b8e0 2024-08-19T10:44:58.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.437+0000 7f27827fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:44:58.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.437+0000 7f27827fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:44:58.442 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":72,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:44:58.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 -- 172.21.15.89:0/594764920 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f277003be40 msgr2=0x7f277003e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:58.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f277003be40 0x7f277003e300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f278c0a17b0 tx=0x7f2788008040 comp rx=0 tx=0).stop 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 10 monclient: shutdown 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 -- 172.21.15.89:0/594764920 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f278c09b8e0 msgr2=0x7f278c09a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f278c09b8e0 0x7f278c09a780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f277c0097c0 tx=0x7f277c0191c0 comp rx=0 tx=0).stop 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 -- 172.21.15.89:0/594764920 shutdown_connections 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f278c0a4240 0x7f278c092380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f278c09ddd0 0x7f278c09acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f277003be40 0x7f277003e300 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 --2- 172.21.15.89:0/594764920 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f278c09b8e0 0x7f278c09a780 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 -- 172.21.15.89:0/594764920 >> 172.21.15.89:0/594764920 conn(0x7f278c0a4df0 msgr2=0x7f278c0a7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 -- 172.21.15.89:0/594764920 shutdown_connections 2024-08-19T10:44:58.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:44:58.441+0000 7f2793e11640 1 -- 172.21.15.89:0/594764920 wait complete. 2024-08-19T10:44:58.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:58.449+0000 7f2f738f4640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h4"}] 2024-08-19T10:44:58.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:58.449+0000 7f2f738f4640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h4"}] 2024-08-19T10:44:58.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:58.449+0000 7f2f738f4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:44:58.449 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:44:58.449+0000 7f2f738f4640 1 -- 172.21.15.190:0/860113610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h4"} v 0) -- 0x7f2f6c11ea90 con 0x7f2f6c0a41c0 2024-08-19T10:45:00.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:00.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:00.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:00.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:00.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:00.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:00.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:00.429+0000 7f2f60ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:01.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:01.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:01.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:01.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:01.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:01.459 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:45:01.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient: get_monmap_and_config 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient: build_initial_monmap 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient: monmap: 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:45:01.591264+0000 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:45:01.591264+0000 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:01.590 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:01.591 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:01.591 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:01.591 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:01.591 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:01.591 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:01.591 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:01.591 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:01.591 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:01.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 1 Processor -- start 2024-08-19T10:45:01.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 1 -- start start 2024-08-19T10:45:01.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient: init 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient: _add_conns ranks=[3,4,2,9,0,5,1,6,8,10,7] 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d40a3d70 0x7f79d40a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient(hunting): picked mon.noname-g con 0x7f79d40a3d70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79d40a4150 0x7f79d40a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient(hunting): picked mon.noname-c con 0x7f79d40a4150 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 0x7f79d409b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient(hunting): picked mon.noname-b con 0x7f79d409b040 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f79d406f8e0 con 0x7f79d409b040 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f79d40a0780 con 0x7f79d40a3d70 2024-08-19T10:45:01.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:01.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f79d409fb40 con 0x7f79d40a4150 2024-08-19T10:45:01.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.585+0000 7f79dac5f640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:01.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d91fc640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 msgr2=0x7f79d409b4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:01.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79dac5f640 10 monclient(hunting): authenticate will time out at 1351.408024s 2024-08-19T10:45:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d91fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 0x7f79d409b4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79d40a4150 0x7f79d40a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79d40a4150 0x7f79d40a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41576/0 (socket says 172.21.15.89:41576) 2024-08-19T10:45:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 1 -- 172.21.15.89:0/3778575145 learned_addr learned my addr 172.21.15.89:0/3778575145 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 10 monclient(hunting): get_auth_request con 0x7f79d40a4150 auth_method 0 2024-08-19T10:45:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:01.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d89fb640 1 --2- 172.21.15.89:0/3778575145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d40a3d70 0x7f79d40a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d89fb640 10 monclient(hunting): get_auth_request con 0x7f79d40a3d70 auth_method 0 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d89fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d89fb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d89fb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 10 monclient(hunting): handle_auth_done global_id 44257 payload 274 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 1 -- 172.21.15.89:0/3778575145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d40a3d70 msgr2=0x7f79d40a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 1 --2- 172.21.15.89:0/3778575145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d40a3d70 0x7f79d40a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 1 -- 172.21.15.89:0/3778575145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 msgr2=0x7f79d409b4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 1 --2- 172.21.15.89:0/3778575145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 0x7f79d409b4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:01.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 1 monclient: found mon.noname-c 2024-08-19T10:45:01.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:01.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:45:01.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.589+0000 7f79d3fff640 1 -- 172.21.15.89:0/3778575145 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d409fe90 con 0x7f79d40a4150 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 1 -- 172.21.15.89:0/3778575145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79d40a4150 msgr2=0x7f79d40a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 1 --2- 172.21.15.89:0/3778575145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79d40a4150 0x7f79d40a4570 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009900 tx=0x7f79c001c4d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f8b15a0f245dc837 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 1 -- 172.21.15.89:0/3778575145 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79c00b0070 con 0x7f79d40a4150 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 10 monclient: dump: 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:01.604 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:01.605 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:01.605 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:01.605 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:01.605 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:01.605 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:01.605 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:01.605 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:01.605 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:01.605 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:01.606 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:01.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:01.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:01.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:01.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 5 monclient: authenticate success, global_id 44257 2024-08-19T10:45:01.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:01.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 1 -- 172.21.15.89:0/3778575145 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c00aa3a0 con 0x7f79d40a4150 2024-08-19T10:45:01.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 1 -- 172.21.15.89:0/3778575145 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79c00aa9c0 con 0x7f79d40a4150 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d37fe640 10 monclient: dump: 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:01.607 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 4 set_mon_vals no callback set 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient: shutdown 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 -- 172.21.15.89:0/3778575145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79d40a4150 msgr2=0x7f79d40a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:01.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 --2- 172.21.15.89:0/3778575145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79d40a4150 0x7f79d40a4570 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009900 tx=0x7f79c001c4d0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 -- 172.21.15.89:0/3778575145 shutdown_connections 2024-08-19T10:45:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 --2- 172.21.15.89:0/3778575145 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79d40a4150 0x7f79d40a4570 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 --2- 172.21.15.89:0/3778575145 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 0x7f79d409b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 --2- 172.21.15.89:0/3778575145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79d40a3d70 0x7f79d40a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 -- 172.21.15.89:0/3778575145 >> 172.21.15.89:0/3778575145 conn(0x7f79d40a4e60 msgr2=0x7f79d40a5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 -- 172.21.15.89:0/3778575145 shutdown_connections 2024-08-19T10:45:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 -- 172.21.15.89:0/3778575145 wait complete. 2024-08-19T10:45:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient: build_initial_monmap 2024-08-19T10:45:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient: monmap: 2024-08-19T10:45:01.609 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:01.610 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 Processor -- start 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 -- start start 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient: init 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient: _add_conns ranks=[2,8,7,5,0,4,3,1,6] 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 0x7f79d41cf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient(hunting): picked mon.noname-b con 0x7f79d409b040 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79d40a3d70 0x7f79d4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient(hunting): picked mon.noname-e con 0x7f79d40a3d70 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79d40a4150 0x7f79d4136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient(hunting): picked mon.noname-i con 0x7f79d40a4150 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:01.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:01.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f79d4140f30 con 0x7f79d409b040 2024-08-19T10:45:01.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:01.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f79d40a4ab0 con 0x7f79d40a4150 2024-08-19T10:45:01.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:01.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d89fb640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 msgr2=0x7f79d41cf620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:01.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f79d41410b0 con 0x7f79d40a3d70 2024-08-19T10:45:01.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:01.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79dac5f640 10 monclient(hunting): authenticate will time out at 1351.415293s 2024-08-19T10:45:01.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d89fb640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 0x7f79d41cf620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79d40a3d70 0x7f79d4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79d40a3d70 0x7f79d4136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47472/0 (socket says 172.21.15.89:47472) 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 1 -- 172.21.15.89:0/783098706 learned_addr learned my addr 172.21.15.89:0/783098706 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 10 monclient(hunting): get_auth_request con 0x7f79d40a3d70 auth_method 0 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 10 monclient(hunting): handle_auth_done global_id 6182 payload 824 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 1 -- 172.21.15.89:0/783098706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79d40a4150 msgr2=0x7f79d4136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:01.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79d40a4150 0x7f79d4136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 1 -- 172.21.15.89:0/783098706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 msgr2=0x7f79d41cf620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 0x7f79d41cf620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 1 monclient: found mon.noname-e 2024-08-19T10:45:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.593+0000 7f79d3fff640 1 -- 172.21.15.89:0/783098706 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d40a2880 con 0x7f79d40a3d70 2024-08-19T10:45:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d3fff640 1 -- 172.21.15.89:0/783098706 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79d40a3d70 msgr2=0x7f79d4136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:01.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d3fff640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79d40a3d70 0x7f79d4136aa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f79d41cfc40 tx=0x7f79c0019270 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 1 -- 172.21.15.89:0/783098706 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79c00b0050 con 0x7f79d40a3d70 2024-08-19T10:45:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 10 monclient: dump: 2024-08-19T10:45:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:01.615 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:01.616 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:01.616 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:01.616 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:01.616 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:01.616 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:01.616 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:01.616 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:01.616 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:01.617 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:01.617 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:01.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 10 monclient: _finish_auth 0 2024-08-19T10:45:01.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 10 monclient: _check_auth_tickets 2024-08-19T10:45:01.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:01.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 1 -- 172.21.15.89:0/783098706 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c00b1040 con 0x7f79d40a3d70 2024-08-19T10:45:01.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:01.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 1 -- 172.21.15.89:0/783098706 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79c00aae30 con 0x7f79d40a3d70 2024-08-19T10:45:01.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:01.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:01.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 10 monclient: dump: 2024-08-19T10:45:01.618 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:01.618 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:01.618 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:01.618 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:01.618 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:01.618 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:01.619 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d2ffd640 4 set_mon_vals no callback set 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79dac5f640 5 monclient: authenticate success, global_id 6182 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79dac5f640 10 monclient: _renew_subs 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79dac5f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79dac5f640 1 -- 172.21.15.89:0/783098706 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d412f2e0 con 0x7f79d40a3d70 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79dac5f640 10 monclient: _renew_subs 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79dac5f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79dac5f640 1 -- 172.21.15.89:0/783098706 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d40749d0 con 0x7f79d40a3d70 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79dac5f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79dac5f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79dac5f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:01.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79dac5f640 1 -- 172.21.15.89:0/783098706 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d40a0240 con 0x7f79d40a3d70 2024-08-19T10:45:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 1 -- 172.21.15.89:0/783098706 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f79c00be030 con 0x7f79d40a3d70 2024-08-19T10:45:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f79b803be00 0x7f79b803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d17fa640 1 -- 172.21.15.89:0/783098706 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(231..231 src has 1..231) ==== 14354+0+0 (secure 0 0 0) 0x7f79c00bc020 con 0x7f79d40a3d70 2024-08-19T10:45:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d89fb640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f79b803be00 0x7f79b803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.597+0000 7f79d89fb640 10 monclient: get_auth_request con 0x7f79b803be00 auth_method 0 2024-08-19T10:45:01.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.601+0000 7f79d89fb640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f79b803be00 0x7f79b803e2c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f79c8000f00 tx=0x7f79c8006e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:01.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.601+0000 7f79d17fa640 1 -- 172.21.15.89:0/783098706 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f79c00f1080 con 0x7f79d40a3d70 2024-08-19T10:45:01.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.601+0000 7f79d17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:01.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.601+0000 7f79d17fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:01.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.841+0000 7f79dac5f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:45:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.841+0000 7f79dac5f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:45:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.841+0000 7f79dac5f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.841+0000 7f79dac5f640 1 -- 172.21.15.89:0/783098706 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f79d4064590 con 0x7f79d40a3d70 2024-08-19T10:45:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79d17fa640 1 -- 172.21.15.89:0/783098706 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7f79c00c5020 con 0x7f79d40a3d70 2024-08-19T10:45:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79d17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:45:01.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79d17fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:01.848 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":72,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:45:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 -- 172.21.15.89:0/783098706 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f79b803be00 msgr2=0x7f79b803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f79b803be00 0x7f79b803e2c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f79c8000f00 tx=0x7f79c8006e20 comp rx=0 tx=0).stop 2024-08-19T10:45:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 10 monclient: shutdown 2024-08-19T10:45:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 -- 172.21.15.89:0/783098706 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79d40a3d70 msgr2=0x7f79d4136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79d40a3d70 0x7f79d4136aa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f79d41cfc40 tx=0x7f79c0019270 comp rx=0 tx=0).stop 2024-08-19T10:45:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 -- 172.21.15.89:0/783098706 shutdown_connections 2024-08-19T10:45:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f79b803be00 0x7f79b803e2c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f79d40a3d70 0x7f79d4136aa0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f79d40a4150 0x7f79d4136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 --2- 172.21.15.89:0/783098706 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79d409b040 0x7f79d41cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 -- 172.21.15.89:0/783098706 >> 172.21.15.89:0/783098706 conn(0x7f79d40a4e60 msgr2=0x7f79d413cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 -- 172.21.15.89:0/783098706 shutdown_connections 2024-08-19T10:45:01.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:01.845+0000 7f79dac5f640 1 -- 172.21.15.89:0/783098706 wait complete. 2024-08-19T10:45:02.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:02.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:02.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:02.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:02.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:02.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:03.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:03.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:03.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:03.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:03.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:03.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:04.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:04.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:04.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:04.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:04.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:04.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:04.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:45:04.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient: get_monmap_and_config 2024-08-19T10:45:04.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient: build_initial_monmap 2024-08-19T10:45:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient: monmap: 2024-08-19T10:45:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:45:04.998184+0000 2024-08-19T10:45:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:45:04.998184+0000 2024-08-19T10:45:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:04.997 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:04.998 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:04.998 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:05.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 1 Processor -- start 2024-08-19T10:45:05.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 1 -- start start 2024-08-19T10:45:05.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient: init 2024-08-19T10:45:05.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:05.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient: _add_conns ranks=[5,9,0,4,1,7,8,6,3,10,2] 2024-08-19T10:45:05.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88f00a5270 0x7f88f00a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:05.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient(hunting): picked mon.noname-h con 0x7f88f00a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:05.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f88f00a6d30 0x7f88f00a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:05.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient(hunting): picked mon.noname-a con 0x7f88f00a6d30 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:05.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f88f01395b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:05.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88f00a8710 con 0x7f88f01395b0 2024-08-19T10:45:05.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:05.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f88f00a7650 con 0x7f88f00a6d30 2024-08-19T10:45:05.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:05.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f88f00a5a40 con 0x7f88f00a5270 2024-08-19T10:45:05.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:05.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f66ce640 10 monclient(hunting): authenticate will time out at 1354.814977s 2024-08-19T10:45:05.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f88f00a6d30 0x7f88f00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:05.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f88f00a6d30 0x7f88f00a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42144/0 (socket says 172.21.15.89:42144) 2024-08-19T10:45:05.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88f4c6b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f88f01395b0 0x7f88f0139a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54192/0 (socket says 172.21.15.89:54192) 2024-08-19T10:45:05.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 1 -- 172.21.15.89:0/1884331647 learned_addr learned my addr 172.21.15.89:0/1884331647 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:05.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 10 monclient(hunting): get_auth_request con 0x7f88f00a6d30 auth_method 0 2024-08-19T10:45:05.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:05.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:05.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:05.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:05.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:05.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:05.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88effff640 1 --2- 172.21.15.89:0/1884331647 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88f00a5270 0x7f88f00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:05.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88effff640 10 monclient(hunting): get_auth_request con 0x7f88f00a5270 auth_method 0 2024-08-19T10:45:05.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:05.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88effff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:05.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88effff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:05.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 10 monclient(hunting): handle_auth_done global_id 6345 payload 274 2024-08-19T10:45:05.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 1 -- 172.21.15.89:0/1884331647 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88f00a5270 msgr2=0x7f88f00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:05.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 1 --2- 172.21.15.89:0/1884331647 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88f00a5270 0x7f88f00a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:05.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 1 -- 172.21.15.89:0/1884331647 >> v1:172.21.15.89:6789/0 conn(0x7f88f01395b0 legacy=0x7f88f0139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:05.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:05.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 1 monclient: found mon.noname-a 2024-08-19T10:45:05.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:05.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:45:05.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:04.993+0000 7f88ef7fe640 1 -- 172.21.15.89:0/1884331647 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f00a81d0 con 0x7f88f00a6d30 2024-08-19T10:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88ef7fe640 1 -- 172.21.15.89:0/1884331647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88f00a6d30 msgr2=0x7f88f00a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88ef7fe640 1 --2- 172.21.15.89:0/1884331647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88f00a6d30 0x7f88f00a7110 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009870 tx=0x7f88e001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33c58f4f0fa7efc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 1 -- 172.21.15.89:0/1884331647 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88e00b0070 con 0x7f88f00a6d30 2024-08-19T10:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 10 monclient: dump: 2024-08-19T10:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:05.013 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 10 monclient: _finish_auth 0 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 1 -- 172.21.15.89:0/1884331647 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e00a4e50 con 0x7f88f00a6d30 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 5 monclient: authenticate success, global_id 6345 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 1 -- 172.21.15.89:0/1884331647 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88e00aac80 con 0x7f88f00a6d30 2024-08-19T10:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88eeffd640 10 monclient: dump: 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 4 set_mon_vals no callback set 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 10 monclient: shutdown 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 1 -- 172.21.15.89:0/1884331647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88f00a6d30 msgr2=0x7f88f00a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 1 --2- 172.21.15.89:0/1884331647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88f00a6d30 0x7f88f00a7110 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009870 tx=0x7f88e001c240 comp rx=0 tx=0).stop 2024-08-19T10:45:05.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 1 -- 172.21.15.89:0/1884331647 shutdown_connections 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 1 --2- 172.21.15.89:0/1884331647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88f00a6d30 0x7f88f00a7110 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 1 --2- 172.21.15.89:0/1884331647 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f88f00a5270 0x7f88f00a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 1 -- 172.21.15.89:0/1884331647 >> 172.21.15.89:0/1884331647 conn(0x7f88f0069a80 msgr2=0x7f88f00abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 1 -- 172.21.15.89:0/1884331647 shutdown_connections 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 1 -- 172.21.15.89:0/1884331647 wait complete. 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 10 monclient: build_initial_monmap 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 10 monclient: monmap: 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 1 Processor -- start 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 1 -- start start 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.001+0000 7f88f66ce640 10 monclient: init 2024-08-19T10:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 10 monclient: _add_conns ranks=[8,6,3,7,1,0,5,2,4] 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88f00a5270 0x7f88f01d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 10 monclient(hunting): picked mon.noname-e con 0x7f88f00a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88f00a6d30 0x7f88f0138650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 10 monclient(hunting): picked mon.noname-d con 0x7f88f00a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f01395b0 0x7f88f013da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 10 monclient(hunting): picked mon.noname-g con 0x7f88f01395b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f88f00a57d0 con 0x7f88f01395b0 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f88f00a5650 con 0x7f88f00a6d30 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f88f0144240 con 0x7f88f00a5270 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f66ce640 10 monclient(hunting): authenticate will time out at 1354.824419s 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ef7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88f00a6d30 0x7f88f0138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88f00a5270 0x7f88f01d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ef7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88f00a6d30 0x7f88f0138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:35636/0 (socket says 172.21.15.89:35636) 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ef7fe640 1 -- 172.21.15.89:0/4170572327 learned_addr learned my addr 172.21.15.89:0/4170572327 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ef7fe640 10 monclient(hunting): get_auth_request con 0x7f88f00a6d30 auth_method 0 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ef7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 10 monclient(hunting): get_auth_request con 0x7f88f00a5270 auth_method 0 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f4c6b640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f01395b0 0x7f88f013da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 10 monclient(hunting): handle_auth_done global_id 6191 payload 824 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 1 -- 172.21.15.89:0/4170572327 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88f00a6d30 msgr2=0x7f88f0138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88f00a6d30 0x7f88f0138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 1 -- 172.21.15.89:0/4170572327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f01395b0 msgr2=0x7f88f013da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f01395b0 0x7f88f013da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 1 monclient: found mon.noname-e 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 1 -- 172.21.15.89:0/4170572327 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f0122a20 con 0x7f88f00a5270 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f4c6b640 10 monclient: get_auth_request con 0x7f88f01395b0 auth_method 0 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ef7fe640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88f00a6d30 0x7f88f0138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88f4c6b640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f01395b0 0x7f88f013da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 1 -- 172.21.15.89:0/4170572327 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88f00a5270 msgr2=0x7f88f01d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88effff640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88f00a5270 0x7f88f01d12c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f88d800d600 tx=0x7f88d800dac0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 1 -- 172.21.15.89:0/4170572327 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88d8012c70 con 0x7f88f00a5270 2024-08-19T10:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 10 monclient: dump: 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:05.024 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 10 monclient: _finish_auth 0 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 1 -- 172.21.15.89:0/4170572327 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d8019070 con 0x7f88f00a5270 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 1 -- 172.21.15.89:0/4170572327 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f88d8012e30 con 0x7f88f00a5270 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ecff9640 10 monclient: dump: 2024-08-19T10:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.005+0000 7f88ee7fc640 4 set_mon_vals no callback set 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88f66ce640 5 monclient: authenticate success, global_id 6191 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88f66ce640 10 monclient: _renew_subs 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88f66ce640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88f66ce640 1 -- 172.21.15.89:0/4170572327 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f013fed0 con 0x7f88f00a5270 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88f66ce640 10 monclient: _renew_subs 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88f66ce640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88f66ce640 1 -- 172.21.15.89:0/4170572327 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f0140b60 con 0x7f88f00a5270 2024-08-19T10:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88f66ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88f66ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88f66ce640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88f66ce640 1 -- 172.21.15.89:0/4170572327 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f00a25e0 con 0x7f88f00a5270 2024-08-19T10:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88ecff9640 1 -- 172.21.15.89:0/4170572327 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f88d8012800 con 0x7f88f00a5270 2024-08-19T10:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88ecff9640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88d003be00 0x7f88d003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:05.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88ecff9640 1 -- 172.21.15.89:0/4170572327 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(231..231 src has 1..231) ==== 14354+0+0 (secure 0 0 0) 0x7f88d800f040 con 0x7f88f00a5270 2024-08-19T10:45:05.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88ef7fe640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88d003be00 0x7f88d003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:05.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.009+0000 7f88ef7fe640 10 monclient: get_auth_request con 0x7f88d003be00 auth_method 0 2024-08-19T10:45:05.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.013+0000 7f88ef7fe640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88d003be00 0x7f88d003e2c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88e001cae0 tx=0x7f88e00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:05.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.013+0000 7f88ecff9640 1 -- 172.21.15.89:0/4170572327 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f88d804d9a0 con 0x7f88f00a5270 2024-08-19T10:45:05.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.013+0000 7f88ecff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:05.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.013+0000 7f88ecff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:05.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.245+0000 7f88f66ce640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:45:05.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.245+0000 7f88f66ce640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:45:05.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.245+0000 7f88f66ce640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:05.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.245+0000 7f88f66ce640 1 -- 172.21.15.89:0/4170572327 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f88f00aae20 con 0x7f88f00a5270 2024-08-19T10:45:05.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.245+0000 7f88ecff9640 1 -- 172.21.15.89:0/4170572327 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7f88d800bda0 con 0x7f88f00a5270 2024-08-19T10:45:05.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.245+0000 7f88ecff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:45:05.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.245+0000 7f88ecff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:05.250 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":72,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:45:05.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 -- 172.21.15.89:0/4170572327 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88d003be00 msgr2=0x7f88d003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:05.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88d003be00 0x7f88d003e2c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88e001cae0 tx=0x7f88e00a6000 comp rx=0 tx=0).stop 2024-08-19T10:45:05.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 10 monclient: shutdown 2024-08-19T10:45:05.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:05.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 -- 172.21.15.89:0/4170572327 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88f00a5270 msgr2=0x7f88f01d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:05.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88f00a5270 0x7f88f01d12c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f88d800d600 tx=0x7f88d800dac0 comp rx=0 tx=0).stop 2024-08-19T10:45:05.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 -- 172.21.15.89:0/4170572327 shutdown_connections 2024-08-19T10:45:05.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f01395b0 0x7f88f013da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:05.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f88f00a6d30 0x7f88f0138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:05.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f88d003be00 0x7f88d003e2c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:05.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 --2- 172.21.15.89:0/4170572327 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f88f00a5270 0x7f88f01d12c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:05.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 -- 172.21.15.89:0/4170572327 >> 172.21.15.89:0/4170572327 conn(0x7f88f0069a80 msgr2=0x7f88f00ab150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:05.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 -- 172.21.15.89:0/4170572327 shutdown_connections 2024-08-19T10:45:05.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:05.249+0000 7f88f66ce640 1 -- 172.21.15.89:0/4170572327 wait complete. 2024-08-19T10:45:05.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:05.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:05.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:05.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:05.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:05.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:06.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:06.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:06.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:06.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:06.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:06.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:07.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:07.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:07.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:07.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:07.476 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:07.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:08.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:45:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient: get_monmap_and_config 2024-08-19T10:45:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient: build_initial_monmap 2024-08-19T10:45:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient: monmap: 2024-08-19T10:45:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:08.397 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:45:08.398880+0000 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:45:08.398880+0000 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:08.398 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:08.399 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:08.399 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:08.399 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:08.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 1 Processor -- start 2024-08-19T10:45:08.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 1 -- start start 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient: init 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient: _add_conns ranks=[9,4,1,7,0,8,5,3,10,2,6] 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient(hunting): picked mon.noname-a con 0x7f6d200a5270 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient(hunting): picked mon.noname-c con 0x7f6d200a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6d201395b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d200a8710 con 0x7f6d201395b0 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f6d200a7650 con 0x7f6d200a5270 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:08.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6d200a5a40 con 0x7f6d200a6d30 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d253fe640 10 monclient(hunting): authenticate will time out at 1358.215602s 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1effd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1e7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1effd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42160/0 (socket says 172.21.15.89:42160) 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 learned_addr learned my addr 172.21.15.89:0/2683474876 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1e7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:41592/0 (socket says 172.21.15.89:41592) 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1effd640 10 monclient(hunting): get_auth_request con 0x7f6d200a5270 auth_method 0 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request con 0x7f6d200a6d30 auth_method 0 2024-08-19T10:45:08.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:08.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:08.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:08.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 msgr2=0x7f6d200a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:45:08.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 msgr2=0x7f6d200a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:08.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:08.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 msgr2=0x7f6d200a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:45:08.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 msgr2=0x7f6d200a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:08.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:08.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:08.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.393+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:08.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.397+0000 7f6d1f7fe640 1 -- 172.21.15.89:0/2683474876 >> v1:172.21.15.190:6789/0 conn(0x7f6d201395b0 legacy=0x7f6d20139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:08.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:08.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:08.431 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:08.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:08.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:08.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1effd640 10 monclient(hunting): get_auth_request con 0x7f6d200a5270 auth_method 0 2024-08-19T10:45:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:45:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request con 0x7f6d200a6d30 auth_method 0 2024-08-19T10:45:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:08.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:08.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:45:08.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 msgr2=0x7f6d200a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:45:08.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 msgr2=0x7f6d200a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:08.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:08.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 msgr2=0x7f6d200a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:45:08.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 msgr2=0x7f6d200a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:08.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:08.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:45:08.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:08.597+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:45:09.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:09.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request con 0x7f6d200a6d30 auth_method 0 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1effd640 10 monclient(hunting): get_auth_request con 0x7f6d200a5270 auth_method 0 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 msgr2=0x7f6d200a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 msgr2=0x7f6d200a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 msgr2=0x7f6d200a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:45:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 msgr2=0x7f6d200a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:45:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.001+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:45:09.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:09.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:09.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:09.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:09.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:09.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request con 0x7f6d200a6d30 auth_method 0 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1effd640 10 monclient(hunting): get_auth_request con 0x7f6d200a5270 auth_method 0 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 msgr2=0x7f6d200a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 msgr2=0x7f6d200a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 msgr2=0x7f6d200a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:45:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 msgr2=0x7f6d200a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:45:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:09.805+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:45:10.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:10.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:10.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:10.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:10.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:10.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:10.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:10.429+0000 7f2f60ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:11.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 10 monclient(hunting): tick 2024-08-19T10:45:11.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:45:11.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 monclient(hunting): continuing hunt 2024-08-19T10:45:11.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 msgr2=0x7f6d200a7110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6d200a6d30 0x7f6d200a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 msgr2=0x7f6d200a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.89:3300/0 conn(0x7f6d200a5270 0x7f6d200a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 -- 172.21.15.89:0/2683474876 >> v1:172.21.15.190:6789/0 conn(0x7f6d201395b0 legacy=0x7f6d20139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 10 monclient: _add_conns ranks=[2,8,10,6,5,7,3,0] 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d0c003880 0x7f6d0c005cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 10 monclient(hunting): picked mon.noname-b con 0x7f6d0c003880 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 10 monclient(hunting): picked mon.noname-e con 0x7f6d0c006350 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 0x7f6d0c00b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 10 monclient(hunting): picked mon.noname-f con 0x7f6d0c008e90 addr v2:172.21.15.190:3300/0 2024-08-19T10:45:11.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 -- 172.21.15.89:0/2683474876 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6d200a7650 con 0x7f6d0c008e90 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 -- 172.21.15.89:0/2683474876 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6d200a5a40 con 0x7f6d0c003880 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 1 -- 172.21.15.89:0/2683474876 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6d0c00baa0 con 0x7f6d0c006350 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1cff9640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.393+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d0c003880 msgr2=0x7f6d0c005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d0c003880 0x7f6d0c005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1f7fe640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 0x7f6d0c00b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1f7fe640 10 monclient(hunting): get_auth_request con 0x7f6d0c008e90 auth_method 0 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:11.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request con 0x7f6d0c006350 auth_method 0 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1f7fe640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 msgr2=0x7f6d0c00b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1f7fe640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 msgr2=0x7f6d0c00b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1f7fe640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 0x7f6d0c00b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 msgr2=0x7f6d0c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 msgr2=0x7f6d0c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1f7fe640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 0x7f6d0c00b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.397+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:11.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:11.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:11.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:11.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:11.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:11.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:11.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:11.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d0c003880 msgr2=0x7f6d0c005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request con 0x7f6d0c006350 auth_method 0 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d0c003880 0x7f6d0c005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1f7fe640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 0x7f6d0c00b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1f7fe640 10 monclient(hunting): get_auth_request con 0x7f6d0c008e90 auth_method 0 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 msgr2=0x7f6d0c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:45:11.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 msgr2=0x7f6d0c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1f7fe640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 msgr2=0x7f6d0c00b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:45:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1f7fe640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 msgr2=0x7f6d0c00b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:45:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1f7fe640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 0x7f6d0c00b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:11.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:11.601+0000 7f6d1f7fe640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 0x7f6d0c00b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:45:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d0c003880 msgr2=0x7f6d0c005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d0c003880 0x7f6d0c005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:45:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request con 0x7f6d0c006350 auth_method 0 2024-08-19T10:45:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:45:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 msgr2=0x7f6d0c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:45:12.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 msgr2=0x7f6d0c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:45:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1f7fe640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 0x7f6d0c00b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1f7fe640 10 monclient(hunting): get_auth_request con 0x7f6d0c008e90 auth_method 0 2024-08-19T10:45:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:45:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1f7fe640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 msgr2=0x7f6d0c00b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:45:12.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1f7fe640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 msgr2=0x7f6d0c00b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:12.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1f7fe640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 0x7f6d0c00b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:12.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.005+0000 7f6d1f7fe640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 0x7f6d0c00b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:45:12.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:12.429+0000 7f2f60ff9640 10 monclient: tick 2024-08-19T10:45:12.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:12.429+0000 7f2f60ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:12.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:12.429+0000 7f2f60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:12.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1effd640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d0c003880 msgr2=0x7f6d0c005cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1effd640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d0c003880 0x7f6d0c005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request con 0x7f6d0c006350 auth_method 0 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 10 monclient(hunting): handle_auth_done global_id 6200 payload 274 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d0c003880 msgr2=0x7f6d0c005cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6d0c003880 0x7f6d0c005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 msgr2=0x7f6d0c00b350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> v2:172.21.15.190:3300/0 conn(0x7f6d0c008e90 0x7f6d0c00b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:45:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 1 monclient: found mon.noname-e 2024-08-19T10:45:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1f7fe640 1 -- 172.21.15.89:0/2683474876 reap_dead start 2024-08-19T10:45:12.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.809+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d200a81d0 con 0x7f6d0c006350 2024-08-19T10:45:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1e7fc640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 msgr2=0x7f6d0c0087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:12.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6d14000ee0 tx=0x7f6d1401dce0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=181e53a8fd94043e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 1 -- 172.21.15.89:0/2683474876 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d140ac070 con 0x7f6d0c006350 2024-08-19T10:45:12.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 10 monclient: dump: 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:12.819 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 1 -- 172.21.15.89:0/2683474876 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d140044a0 con 0x7f6d0c006350 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 1 -- 172.21.15.89:0/2683474876 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d1400f040 con 0x7f6d0c006350 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d253fe640 5 monclient: authenticate success, global_id 6200 2024-08-19T10:45:12.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d253fe640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d253fe640 4 set_mon_vals no callback set 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.813+0000 7f6d1dffb640 10 monclient: dump: 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:12.821 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:12.822 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:12.822 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 10 monclient: shutdown 2024-08-19T10:45:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 1 -- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 msgr2=0x7f6d0c0087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6d14000ee0 tx=0x7f6d1401dce0 comp rx=0 tx=0).stop 2024-08-19T10:45:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 1 -- 172.21.15.89:0/2683474876 shutdown_connections 2024-08-19T10:45:12.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 1 --2- 172.21.15.89:0/2683474876 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d0c0087c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 1 -- 172.21.15.89:0/2683474876 >> 172.21.15.89:0/2683474876 conn(0x7f6d20069a80 msgr2=0x7f6d200abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 1 -- 172.21.15.89:0/2683474876 shutdown_connections 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 1 -- 172.21.15.89:0/2683474876 wait complete. 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 10 monclient: build_initial_monmap 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 10 monclient: monmap: 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 1 Processor -- start 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 1 -- start start 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 10 monclient: init 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.817+0000 7f6d253fe640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient: _add_conns ranks=[8,1,7,3,2,4,0,5,6] 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d201d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient(hunting): picked mon.noname-e con 0x7f6d0c006350 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d201d21b0 0x7f6d20126f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient(hunting): picked mon.noname-f con 0x7f6d201d21b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6d201d25f0 0x7f6d20127460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient(hunting): picked mon.noname-i con 0x7f6d201d25f0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6d200a57d0 con 0x7f6d201d21b0 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6d200a5650 con 0x7f6d201d25f0 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:12.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6d2013a1a0 con 0x7f6d0c006350 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d201d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient(hunting): authenticate will time out at 1362.640313s 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d201d1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:45212/0 (socket says 172.21.15.89:45212) 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 1 -- 172.21.15.89:0/33159023 learned_addr learned my addr 172.21.15.89:0/33159023 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 10 monclient(hunting): get_auth_request con 0x7f6d0c006350 auth_method 0 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:12.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1f7fe640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6d201d25f0 0x7f6d20127460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d201d21b0 0x7f6d20126f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1f7fe640 10 monclient(hunting): get_auth_request con 0x7f6d201d25f0 auth_method 0 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request con 0x7f6d201d21b0 auth_method 0 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 10 monclient(hunting): handle_auth_done global_id 6209 payload 824 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 1 -- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6d201d25f0 msgr2=0x7f6d20127460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6d201d25f0 0x7f6d20127460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:12.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 1 -- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d201d21b0 msgr2=0x7f6d20126f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:12.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d201d21b0 0x7f6d20126f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:12.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 10 monclient: _finish_hunting 0 2024-08-19T10:45:12.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 1 monclient: found mon.noname-e 2024-08-19T10:45:12.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:12.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:12.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 1 -- 172.21.15.89:0/33159023 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2012a8c0 con 0x7f6d0c006350 2024-08-19T10:45:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 1 -- 172.21.15.89:0/33159023 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 msgr2=0x7f6d201d1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1effd640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d201d1c70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6d10010690 tx=0x7f6d10010b50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 1 -- 172.21.15.89:0/33159023 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d1001b030 con 0x7f6d0c006350 2024-08-19T10:45:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 10 monclient: dump: 2024-08-19T10:45:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:12.837 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:12.838 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 5 monclient: authenticate success, global_id 6209 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient: _renew_subs 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 1 -- 172.21.15.89:0/33159023 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d10014ae0 con 0x7f6d0c006350 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 1 -- 172.21.15.89:0/33159023 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d200a25a0 con 0x7f6d0c006350 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 1 -- 172.21.15.89:0/33159023 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6d1000f040 con 0x7f6d0c006350 2024-08-19T10:45:12.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d077fe640 10 monclient: dump: 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:12.840 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d1d7fa640 4 set_mon_vals no callback set 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient: _renew_subs 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.821+0000 7f6d253fe640 1 -- 172.21.15.89:0/33159023 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d20139150 con 0x7f6d0c006350 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.825+0000 7f6d077fe640 1 -- 172.21.15.89:0/33159023 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6d1001e030 con 0x7f6d0c006350 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.825+0000 7f6d077fe640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0c03d350 0x7f6d0c03f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.825+0000 7f6d077fe640 1 -- 172.21.15.89:0/33159023 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(231..231 src has 1..231) ==== 14354+0+0 (secure 0 0 0) 0x7f6d1001c070 con 0x7f6d0c006350 2024-08-19T10:45:12.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.825+0000 7f6d253fe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:12.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.825+0000 7f6d253fe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:12.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.825+0000 7f6d253fe640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:12.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.825+0000 7f6d253fe640 1 -- 172.21.15.89:0/33159023 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d2013c9f0 con 0x7f6d0c006350 2024-08-19T10:45:12.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.825+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0c03d350 0x7f6d0c03f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:12.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.825+0000 7f6d1e7fc640 10 monclient: get_auth_request con 0x7f6d0c03d350 auth_method 0 2024-08-19T10:45:12.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.829+0000 7f6d1e7fc640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0c03d350 0x7f6d0c03f810 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c008e60 tx=0x7f6d140a8070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:12.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.829+0000 7f6d077fe640 1 -- 172.21.15.89:0/33159023 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6d10015360 con 0x7f6d0c006350 2024-08-19T10:45:12.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.829+0000 7f6d077fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:12.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:12.829+0000 7f6d077fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:13.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:45:13.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:45:13.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:13.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 -- 172.21.15.89:0/33159023 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6d20064590 con 0x7f6d0c006350 2024-08-19T10:45:13.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d077fe640 1 -- 172.21.15.89:0/33159023 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3186 (secure 0 0 0) 0x7f6d1004dbb0 con 0x7f6d0c006350 2024-08-19T10:45:13.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d077fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:45:13.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d077fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:13.068 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":80,"quorum":[0,1,3,4,5,6,7,8],"quorum_names":["a","f","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:45:13.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 -- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0c03d350 msgr2=0x7f6d0c03f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0c03d350 0x7f6d0c03f810 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c008e60 tx=0x7f6d140a8070 comp rx=0 tx=0).stop 2024-08-19T10:45:13.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 10 monclient: shutdown 2024-08-19T10:45:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 -- 172.21.15.89:0/33159023 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 msgr2=0x7f6d201d1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d201d1c70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6d10010690 tx=0x7f6d10010b50 comp rx=0 tx=0).stop 2024-08-19T10:45:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 -- 172.21.15.89:0/33159023 shutdown_connections 2024-08-19T10:45:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6d201d25f0 0x7f6d20127460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d201d21b0 0x7f6d20126f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6d0c03d350 0x7f6d0c03f810 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 --2- 172.21.15.89:0/33159023 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6d0c006350 0x7f6d201d1c70 secure :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6d10010690 tx=0x7f6d10010b50 comp rx=0 tx=0).stop 2024-08-19T10:45:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 -- 172.21.15.89:0/33159023 >> 172.21.15.89:0/33159023 conn(0x7f6d20069a80 msgr2=0x7f6d2009c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 -- 172.21.15.89:0/33159023 shutdown_connections 2024-08-19T10:45:13.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.065+0000 7f6d253fe640 1 -- 172.21.15.89:0/33159023 wait complete. 2024-08-19T10:45:13.086 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-08-19T10:45:13.086 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-08-19T10:45:13.087 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:45:13.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient: get_monmap_and_config 2024-08-19T10:45:13.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient: monmap: 2024-08-19T10:45:13.202 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:13.202 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.202 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:45:13.203567+0000 2024-08-19T10:45:13.202 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:45:13.203567+0000 2024-08-19T10:45:13.202 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:13.203 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 1 Processor -- start 2024-08-19T10:45:13.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 1 -- start start 2024-08-19T10:45:13.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient: init 2024-08-19T10:45:13.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient: _add_conns ranks=[6,5,2,7,10,8,4,1,9,3,0] 2024-08-19T10:45:13.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f89b80a3620 0x7f89b80a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient(hunting): picked mon.noname-d con 0x7f89b80a3620 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:13.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f89b809c9e0 0x7f89b80a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient(hunting): picked mon.noname-h con 0x7f89b809c9e0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:13.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89b81373f0 0x7f89b8137810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient(hunting): picked mon.noname-b con 0x7f89b81373f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:13.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f89b809dab0 con 0x7f89b81373f0 2024-08-19T10:45:13.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f89b80a1f20 con 0x7f89b809c9e0 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f89b80a4480 con 0x7f89b80a3620 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bf16f640 10 monclient(hunting): authenticate will time out at 1363.019812s 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bd70c640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89b81373f0 msgr2=0x7f89b8137810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.197+0000 7f89bd70c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89b81373f0 0x7f89b8137810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f89b80a3620 0x7f89b80a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f89b80a3620 0x7f89b80a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:53088/0 (socket says 172.21.15.89:53088) 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 1 -- 172.21.15.89:0/786813727 learned_addr learned my addr 172.21.15.89:0/786813727 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 10 monclient(hunting): get_auth_request con 0x7f89b80a3620 auth_method 0 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89b7fff640 1 --2- 172.21.15.89:0/786813727 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f89b809c9e0 0x7f89b80a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89b7fff640 10 monclient(hunting): get_auth_request con 0x7f89b809c9e0 auth_method 0 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89b7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_done global_id 44259 payload 274 2024-08-19T10:45:13.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 1 -- 172.21.15.89:0/786813727 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f89b809c9e0 msgr2=0x7f89b80a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:13.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 1 --2- 172.21.15.89:0/786813727 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f89b809c9e0 0x7f89b80a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 1 -- 172.21.15.89:0/786813727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89b81373f0 msgr2=0x7f89b8137810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:13.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 1 --2- 172.21.15.89:0/786813727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89b81373f0 0x7f89b8137810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 1 monclient: found mon.noname-d 2024-08-19T10:45:13.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:45:13.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.201+0000 7f89bcf0b640 1 -- 172.21.15.89:0/786813727 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b80a1790 con 0x7f89b80a3620 2024-08-19T10:45:13.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bcf0b640 1 -- 172.21.15.89:0/786813727 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f89b80a3620 msgr2=0x7f89b80a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bcf0b640 1 --2- 172.21.15.89:0/786813727 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f89b80a3620 0x7f89b80a3a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89a40098d0 tx=0x7f89a401c460 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e05375360d1e3bd9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 1 -- 172.21.15.89:0/786813727 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f89a40b0070 con 0x7f89b80a3620 2024-08-19T10:45:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:45:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 10 monclient: dump: 2024-08-19T10:45:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.216 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 5 monclient: authenticate success, global_id 44259 2024-08-19T10:45:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 1 -- 172.21.15.89:0/786813727 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a40aa810 con 0x7f89b80a3620 2024-08-19T10:45:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 4 set_mon_vals no callback set 2024-08-19T10:45:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 1 -- 172.21.15.89:0/786813727 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f89a40a9300 con 0x7f89b80a3620 2024-08-19T10:45:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:45:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b77fe640 10 monclient: dump: 2024-08-19T10:45:13.218 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.219 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient: shutdown 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 -- 172.21.15.89:0/786813727 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f89b80a3620 msgr2=0x7f89b80a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 --2- 172.21.15.89:0/786813727 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f89b80a3620 0x7f89b80a3a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89a40098d0 tx=0x7f89a401c460 comp rx=0 tx=0).stop 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 -- 172.21.15.89:0/786813727 shutdown_connections 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 --2- 172.21.15.89:0/786813727 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89b81373f0 0x7f89b8137810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 --2- 172.21.15.89:0/786813727 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f89b809c9e0 0x7f89b80a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 --2- 172.21.15.89:0/786813727 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f89b80a3620 0x7f89b80a3a00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 -- 172.21.15.89:0/786813727 >> 172.21.15.89:0/786813727 conn(0x7f89b80a8210 msgr2=0x7f89b80a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 -- 172.21.15.89:0/786813727 shutdown_connections 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 -- 172.21.15.89:0/786813727 wait complete. 2024-08-19T10:45:13.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient: monmap: 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.221 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 Processor -- start 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 -- start start 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient: init 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient: _add_conns ranks=[2,4,8,5,3,6,0,1,7] 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89b809c9e0 0x7f89b8136320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient(hunting): picked mon.noname-b con 0x7f89b809c9e0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89b80a3620 0x7f89b811e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient(hunting): picked mon.noname-c con 0x7f89b80a3620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:13.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f89b81373f0 0x7f89b811e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient(hunting): picked mon.noname-e con 0x7f89b81373f0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f89b8140e60 con 0x7f89b809c9e0 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f89b809df40 con 0x7f89b80a3620 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f89b8140fe0 con 0x7f89b81373f0 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bf16f640 10 monclient(hunting): authenticate will time out at 1363.027235s 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bcf0b640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89b809c9e0 msgr2=0x7f89b8136320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bcf0b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89b809c9e0 0x7f89b8136320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89b80a3620 0x7f89b811e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bd70c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f89b81373f0 0x7f89b811e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89b80a3620 0x7f89b811e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54360/0 (socket says 172.21.15.89:54360) 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bd70c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f89b81373f0 0x7f89b811e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:45220/0 (socket says 172.21.15.89:45220) 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 1 -- 172.21.15.89:0/2402680748 learned_addr learned my addr 172.21.15.89:0/2402680748 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 10 monclient(hunting): get_auth_request con 0x7f89b80a3620 auth_method 0 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bd70c640 10 monclient(hunting): get_auth_request con 0x7f89b81373f0 auth_method 0 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bd70c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bd70c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bd70c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bd70c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bd70c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bd70c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 10 monclient(hunting): handle_auth_done global_id 44266 payload 824 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 1 -- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f89b81373f0 msgr2=0x7f89b811e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f89b81373f0 0x7f89b811e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 1 -- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89b809c9e0 msgr2=0x7f89b8136320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f89b809c9e0 0x7f89b8136320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 1 monclient: found mon.noname-c 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89b7fff640 1 -- 172.21.15.89:0/2402680748 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b80a6a90 con 0x7f89b80a3620 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.205+0000 7f89bd70c640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f89b81373f0 0x7f89b811e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b7fff640 1 -- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89b80a3620 msgr2=0x7f89b811e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b7fff640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89b80a3620 0x7f89b811e260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00e930 tx=0x7f89ac00edf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f89ac017070 con 0x7f89b80a3620 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 10 monclient: dump: 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.226 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bf16f640 5 monclient: authenticate success, global_id 44266 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bf16f640 10 monclient: _renew_subs 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac00cc10 con 0x7f89b80a3620 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bf16f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bf16f640 1 -- 172.21.15.89:0/2402680748 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b81370c0 con 0x7f89b80a3620 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f89ac00cdc0 con 0x7f89b80a3620 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b6ffd640 4 set_mon_vals no callback set 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 10 monclient: dump: 2024-08-19T10:45:13.228 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.229 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bf16f640 10 monclient: _renew_subs 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bf16f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bf16f640 1 -- 172.21.15.89:0/2402680748 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b81204c0 con 0x7f89b80a3620 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bf16f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bf16f640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b8136860 0x7f89b802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bf16f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bf16f640 1 -- 172.21.15.89:0/2402680748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f89b813d820 con 0x7f89b8136860 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bcf0b640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b8136860 0x7f89b802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f89ac01a070 con 0x7f89b80a3620 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bcf0b640 10 monclient: get_auth_request con 0x7f89b8136860 auth_method 0 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bcf0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bcf0b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bcf0b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_done global_id 6354 payload 824 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f899c03be30 0x7f899c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bcf0b640 1 -- 172.21.15.89:0/2402680748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89b8137810 con 0x7f89b8136860 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(231..231 src has 1..231) ==== 14354+0+0 (secure 0 0 0) 0x7f89ac00f040 con 0x7f89b80a3620 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bd70c640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f899c03be30 0x7f899c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.209+0000 7f89bd70c640 10 monclient: get_auth_request con 0x7f899c03be30 auth_method 0 2024-08-19T10:45:13.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89bcf0b640 1 -- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b8136860 msgr2=0x7f89b802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89bcf0b640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b8136860 0x7f89b802d1d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f89a40ad000 tx=0x7f89a401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f89a40b0040 con 0x7f89b8136860 2024-08-19T10:45:13.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89b57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:45:13.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89b57fa640 10 monclient: dump: 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.235 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f89a40bb070 con 0x7f89b8136860 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89b57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89b57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b8136860 msgr2=0x7f89b802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89b57fa640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b8136860 0x7f89b802d1d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f89a40ad000 tx=0x7f89a401aca0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b8136860 msgr2=0x7f89b802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89b57fa640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b8136860 0x7f89b802d1d0 secure :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f89a40ad000 tx=0x7f89a401aca0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.213+0000 7f89bd70c640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f899c03be30 0x7f899c03e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f89b0003cc0 tx=0x7f89b000d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bf16f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bf16f640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b802d710 0x7f89b802db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bf16f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bf16f640 1 -- 172.21.15.89:0/2402680748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8064590 con 0x7f89b802d710 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b802d710 0x7f89b802db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 10 monclient: get_auth_request con 0x7f89b802d710 auth_method 0 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 10 monclient(hunting): handle_auth_done global_id 6363 payload 824 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 1 -- 172.21.15.89:0/2402680748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f89b80a6ec0 con 0x7f89b802d710 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 1 -- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b802d710 msgr2=0x7f89b802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89bcf0b640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b802d710 0x7f89b802db20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f89a40097f0 tx=0x7f89a4019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f89a40b0040 con 0x7f89b802d710 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89b57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89b57fa640 10 monclient: dump: 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.238 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9425 (secure 0 0 0) 0x7f89a40c0260 con 0x7f89b802d710 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89b57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89b57fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b802d710 msgr2=0x7f89b802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89b57fa640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b802d710 0x7f89b802db20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f89a40097f0 tx=0x7f89a4019200 comp rx=0 tx=0).stop 2024-08-19T10:45:13.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89b57fa640 1 -- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b802d710 msgr2=0x7f89b802db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:13.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.221+0000 7f89b57fa640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89b802d710 0x7f89b802db20 secure :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f89a40097f0 tx=0x7f89a4019200 comp rx=0 tx=0).stop 2024-08-19T10:45:13.240 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:45:13.240 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:45:13.240 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:45:13.240 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:45:13.240 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 80, 2024-08-19T10:45:13.240 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 443034, 2024-08-19T10:45:13.240 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:45:13.240 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:45:13.241 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.242 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:45:13.246 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:45:13.246 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.246 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:45:13.247 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:45:13.248 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.249 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.250 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.251 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:45:13.254 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:45:13.255 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.256 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.257 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.258 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.260 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.261 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.262 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:45:13.263 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:45:13.264 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.265 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:45:13.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.268 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:45:13.268 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:45:13.268 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.249+0000 7f2f61ffb640 1 -- 172.21.15.190:0/860113610 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h4"}]=0 removed item id -28 name 'h4' from crush map v232) ==== 118+0+0 (secure 0 0 0) 0x7f2f5c0e6580 con 0x7f2f6c0a41c0 2024-08-19T10:45:13.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.249+0000 7f2f61ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h4"}] 2024-08-19T10:45:13.269 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.249+0000 7f2f61ffb640 10 monclient: _finish_command 2 = system:0 removed item id -28 name 'h4' from crush map 2024-08-19T10:45:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bf16f640 1 -- 172.21.15.89:0/2402680748 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f899c03be30 msgr2=0x7f899c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bf16f640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f899c03be30 0x7f899c03e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f89b0003cc0 tx=0x7f89b000d040 comp rx=0 tx=0).stop 2024-08-19T10:45:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bf16f640 10 monclient: shutdown 2024-08-19T10:45:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bf16f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bf16f640 1 -- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89b80a3620 msgr2=0x7f89b811e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bf16f640 1 --2- 172.21.15.89:0/2402680748 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f89b80a3620 0x7f89b811e260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00e930 tx=0x7f89ac00edf0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bd70c640 1 -- 172.21.15.89:0/2402680748 reap_dead start 2024-08-19T10:45:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bd70c640 1 -- 172.21.15.89:0/2402680748 reap_dead start 2024-08-19T10:45:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bf16f640 1 -- 172.21.15.89:0/2402680748 shutdown_connections 2024-08-19T10:45:13.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bf16f640 1 -- 172.21.15.89:0/2402680748 >> 172.21.15.89:0/2402680748 conn(0x7f89b80a8210 msgr2=0x7f89b80727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bf16f640 1 -- 172.21.15.89:0/2402680748 shutdown_connections 2024-08-19T10:45:13.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.225+0000 7f89bf16f640 1 -- 172.21.15.89:0/2402680748 wait complete. 2024-08-19T10:45:13.271 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:45:13.271 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-08-19T10:45:13.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:45:13.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.361+0000 7f2f738f4640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h4"}] 2024-08-19T10:45:13.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.361+0000 7f2f738f4640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h4"}] 2024-08-19T10:45:13.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.361+0000 7f2f738f4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:13.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.361+0000 7f2f738f4640 1 -- 172.21.15.190:0/860113610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h4"} v 0) -- 0x7f2f6c0008e0 con 0x7f2f6c0a41c0 2024-08-19T10:45:13.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.361+0000 7f2f61ffb640 1 -- 172.21.15.190:0/860113610 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h4"}]=0 device 'h4' does not appear in the crush map v232) ==== 118+0+0 (secure 0 0 0) 0x7f2f5c0bbab0 con 0x7f2f6c0a41c0 2024-08-19T10:45:13.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.361+0000 7f2f61ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h4"}] 2024-08-19T10:45:13.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.361+0000 7f2f61ffb640 10 monclient: _finish_command 3 = system:0 device 'h4' does not appear in the crush map 2024-08-19T10:45:13.365 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h4' does not appear in the crush map 2024-08-19T10:45:13.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 -- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f5003be00 msgr2=0x7f2f5003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f5003be00 0x7f2f5003e2c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800b820 tx=0x7f2f6800b3a0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 10 monclient: shutdown 2024-08-19T10:45:13.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 -- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 msgr2=0x7f2f6c071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c071170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0ad000 tx=0x7f2f5c019270 comp rx=0 tx=0).stop 2024-08-19T10:45:13.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 -- 172.21.15.190:0/860113610 shutdown_connections 2024-08-19T10:45:13.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2f5003be00 0x7f2f5003e2c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2f6c13aa60 0x7f2f6c0716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f6c0a41c0 0x7f2f6c071170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 --2- 172.21.15.190:0/860113610 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2f6c069410 0x7f2f6c070c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 -- 172.21.15.190:0/860113610 >> 172.21.15.190:0/860113610 conn(0x7f2f6c0a4d70 msgr2=0x7f2f6c0a6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 -- 172.21.15.190:0/860113610 shutdown_connections 2024-08-19T10:45:13.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.365+0000 7f2f738f4640 1 -- 172.21.15.190:0/860113610 wait complete. 2024-08-19T10:45:13.382 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:13.382 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h5 2024-08-19T10:45:13.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient: get_monmap_and_config 2024-08-19T10:45:13.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient: monmap: 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:45:13.389507+0000 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:45:13.389507+0000 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.388 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.389 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.389 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:13.389 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:13.389 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 1 Processor -- start 2024-08-19T10:45:13.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 1 -- start start 2024-08-19T10:45:13.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient: init 2024-08-19T10:45:13.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient: _add_conns ranks=[10,5,2,8,1,9,0,7,4,3,6] 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4a100a5270 0x7f4a100a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient(hunting): picked mon.noname-f con 0x7f4a100a5270 addr v2:172.21.15.190:3300/0 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a100a6d30 0x7f4a100a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient(hunting): picked mon.noname-h con 0x7f4a100a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a101395b0 0x7f4a10139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient(hunting): picked mon.noname-b con 0x7f4a101395b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f4a100a7650 con 0x7f4a100a5270 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4a100a8710 con 0x7f4a101395b0 2024-08-19T10:45:13.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4a100a5a40 con 0x7f4a100a6d30 2024-08-19T10:45:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a16a9a640 10 monclient(hunting): authenticate will time out at 1363.206231s 2024-08-19T10:45:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a15037640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a101395b0 msgr2=0x7f4a10139a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a15037640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a101395b0 0x7f4a10139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a14836640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4a100a5270 0x7f4a100a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a14836640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4a100a5270 0x7f4a100a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:49110/0 (socket says 172.21.15.89:49110) 2024-08-19T10:45:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a14836640 1 -- 172.21.15.89:0/1345608865 learned_addr learned my addr 172.21.15.89:0/1345608865 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a14836640 10 monclient(hunting): get_auth_request con 0x7f4a100a5270 auth_method 0 2024-08-19T10:45:13.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a14836640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a14836640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a14836640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 1 --2- 172.21.15.89:0/1345608865 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a100a6d30 0x7f4a100a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 10 monclient(hunting): get_auth_request con 0x7f4a100a6d30 auth_method 0 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a14836640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a14836640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a14836640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_done global_id 6296 payload 274 2024-08-19T10:45:13.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 1 -- 172.21.15.89:0/1345608865 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a101395b0 msgr2=0x7f4a10139a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:13.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 1 --2- 172.21.15.89:0/1345608865 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a101395b0 0x7f4a10139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 1 -- 172.21.15.89:0/1345608865 >> v2:172.21.15.190:3300/0 conn(0x7f4a100a5270 msgr2=0x7f4a100a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:13.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 1 --2- 172.21.15.89:0/1345608865 >> v2:172.21.15.190:3300/0 conn(0x7f4a100a5270 0x7f4a100a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 1 monclient: found mon.noname-h 2024-08-19T10:45:13.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:13.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.385+0000 7f4a0ffff640 1 -- 172.21.15.89:0/1345608865 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a100a81d0 con 0x7f4a100a6d30 2024-08-19T10:45:13.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0ffff640 1 -- 172.21.15.89:0/1345608865 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a100a6d30 msgr2=0x7f4a100a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0ffff640 1 --2- 172.21.15.89:0/1345608865 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a100a6d30 0x7f4a100a7110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009d90 tx=0x7f49f801c850 comp rx=0 tx=0).ready entity=mon.5 client_cookie=1040fff875c679a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 1 -- 172.21.15.89:0/1345608865 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49f80b2070 con 0x7f4a100a6d30 2024-08-19T10:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 10 monclient: dump: 2024-08-19T10:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.403 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 1 -- 172.21.15.89:0/1345608865 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f80a5b00 con 0x7f4a100a6d30 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a16a9a640 5 monclient: authenticate success, global_id 6296 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a16a9a640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a16a9a640 4 set_mon_vals no callback set 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 1 -- 172.21.15.89:0/1345608865 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49f80a5cb0 con 0x7f4a100a6d30 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.389+0000 7f4a0f7fe640 10 monclient: dump: 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.405 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient: shutdown 2024-08-19T10:45:13.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1345608865 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a100a6d30 msgr2=0x7f4a100a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 --2- 172.21.15.89:0/1345608865 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a100a6d30 0x7f4a100a7110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009d90 tx=0x7f49f801c850 comp rx=0 tx=0).stop 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1345608865 shutdown_connections 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 --2- 172.21.15.89:0/1345608865 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4a101395b0 0x7f4a10139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 --2- 172.21.15.89:0/1345608865 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4a100a6d30 0x7f4a100a7110 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 --2- 172.21.15.89:0/1345608865 >> v2:172.21.15.190:3300/0 conn(0x7f4a100a5270 0x7f4a100a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1345608865 >> 172.21.15.89:0/1345608865 conn(0x7f4a10069a80 msgr2=0x7f4a100abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1345608865 shutdown_connections 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1345608865 wait complete. 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient: monmap: 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:13.407 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.408 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 Processor -- start 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 -- start start 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient: init 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient: _add_conns ranks=[0,7,6,5,2,4,3,8,1] 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a100a5270 0x7f4a10131d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient(hunting): picked mon.noname-a con 0x7f4a100a5270 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a100a6d30 0x7f4a1013cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient(hunting): picked mon.noname-i con 0x7f4a100a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a101395b0 0x7f4a1013d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient(hunting): picked mon.noname-d con 0x7f4a101395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:13.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4a100a5650 con 0x7f4a100a5270 2024-08-19T10:45:13.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4a100a4a70 con 0x7f4a101395b0 2024-08-19T10:45:13.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4a100a57d0 con 0x7f4a100a6d30 2024-08-19T10:45:13.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a16a9a640 10 monclient(hunting): authenticate will time out at 1363.213870s 2024-08-19T10:45:13.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a100a5270 0x7f4a10131d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a15037640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a101395b0 0x7f4a1013d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a100a5270 0x7f4a10131d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42912/0 (socket says 172.21.15.89:42912) 2024-08-19T10:45:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 1 -- 172.21.15.89:0/1498043190 learned_addr learned my addr 172.21.15.89:0/1498043190 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 10 monclient(hunting): get_auth_request con 0x7f4a100a5270 auth_method 0 2024-08-19T10:45:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a15037640 10 monclient(hunting): get_auth_request con 0x7f4a101395b0 auth_method 0 2024-08-19T10:45:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a15037640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a15037640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a15037640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a0ffff640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a100a6d30 0x7f4a1013cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a15037640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a15037640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a15037640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a0ffff640 10 monclient(hunting): get_auth_request con 0x7f4a100a6d30 auth_method 0 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a0ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 10 monclient(hunting): handle_auth_done global_id 6372 payload 824 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 1 -- 172.21.15.89:0/1498043190 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a100a6d30 msgr2=0x7f4a1013cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4a100a6d30 0x7f4a1013cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 1 -- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a101395b0 msgr2=0x7f4a1013d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a101395b0 0x7f4a1013d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 1 monclient: found mon.noname-a 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a14836640 1 -- 172.21.15.89:0/1498043190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1013e8e0 con 0x7f4a100a5270 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.393+0000 7f4a15037640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4a101395b0 0x7f4a1013d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:13.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a14836640 1 -- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a100a5270 msgr2=0x7f4a10131d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a14836640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a100a5270 0x7f4a10131d20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800e7f0 tx=0x7f4a0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a08017070 con 0x7f4a100a5270 2024-08-19T10:45:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:45:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 10 monclient: dump: 2024-08-19T10:45:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.414 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.415 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.415 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.415 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.415 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.415 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.415 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.415 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.415 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.415 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.415 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a16a9a640 5 monclient: authenticate success, global_id 6372 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a16a9a640 10 monclient: _renew_subs 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a16a9a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1498043190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1009d7b0 con 0x7f4a100a5270 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a0800cb90 con 0x7f4a100a5270 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4a0800cd40 con 0x7f4a100a5270 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:45:13.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 10 monclient: dump: 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.417 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0effd640 4 set_mon_vals no callback set 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a16a9a640 10 monclient: _renew_subs 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a16a9a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1498043190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a100a6a60 con 0x7f4a100a5270 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a16a9a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a16a9a640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a10132260 0x7f4a1013d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a16a9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1498043190 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f4a10135820 con 0x7f4a10132260 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0ffff640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a10132260 0x7f4a1013d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0ffff640 10 monclient: get_auth_request con 0x7f4a10132260 auth_method 0 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4a0801a070 con 0x7f4a100a5270 2024-08-19T10:45:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_done global_id 44275 payload 824 2024-08-19T10:45:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0ffff640 1 -- 172.21.15.89:0/1498043190 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a100a7110 con 0x7f4a10132260 2024-08-19T10:45:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f49f003be00 0x7f49f003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(232..232 src has 1..232) ==== 14144+0+0 (secure 0 0 0) 0x7f4a0800f040 con 0x7f4a100a5270 2024-08-19T10:45:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a15037640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f49f003be00 0x7f49f003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.397+0000 7f4a15037640 10 monclient: get_auth_request con 0x7f49f003be00 auth_method 0 2024-08-19T10:45:13.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0ffff640 1 -- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a10132260 msgr2=0x7f4a1013d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0ffff640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a10132260 0x7f4a1013d870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a100aceb0 tx=0x7f49f80195f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49f80bc030 con 0x7f4a10132260 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0d7fa640 10 monclient: dump: 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.423 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f49f80b3040 con 0x7f4a10132260 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a10132260 msgr2=0x7f4a1013d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0d7fa640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a10132260 0x7f4a1013d870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a100aceb0 tx=0x7f49f80195f0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a10132260 msgr2=0x7f4a1013d870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a0d7fa640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a10132260 0x7f4a1013d870 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a100aceb0 tx=0x7f49f80195f0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.401+0000 7f4a15037640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f49f003be00 0x7f49f003e2c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4a1013f100 tx=0x7f4a00009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a16a9a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:45:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a16a9a640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1013ddb0 0x7f4a10137a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a16a9a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1498043190 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f4a1009d0a0 con 0x7f4a1013ddb0 2024-08-19T10:45:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a0ffff640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1013ddb0 0x7f4a10137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a0ffff640 10 monclient: get_auth_request con 0x7f4a1013ddb0 auth_method 0 2024-08-19T10:45:13.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a0ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a0ffff640 10 monclient(hunting): handle_auth_done global_id 44284 payload 824 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.405+0000 7f4a0ffff640 1 -- 172.21.15.89:0/1498043190 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4a10064590 con 0x7f4a1013ddb0 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0ffff640 1 -- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1013ddb0 msgr2=0x7f4a10137a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0ffff640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1013ddb0 0x7f4a10137a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f49f801c680 tx=0x7f49f80a7000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f49f80bc030 con 0x7f4a1013ddb0 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0d7fa640 10 monclient: dump: 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.427 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.428 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.428 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.428 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.428 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9255 (secure 0 0 0) 0x7f49f80c1670 con 0x7f4a1013ddb0 2024-08-19T10:45:13.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:45:13.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0d7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:13.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1013ddb0 msgr2=0x7f4a10137a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0d7fa640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1013ddb0 0x7f4a10137a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f49f801c680 tx=0x7f49f80a7000 comp rx=0 tx=0).stop 2024-08-19T10:45:13.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0d7fa640 1 -- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1013ddb0 msgr2=0x7f4a10137a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:13.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a0d7fa640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4a1013ddb0 0x7f4a10137a40 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f49f801c680 tx=0x7f49f80a7000 comp rx=0 tx=0).stop 2024-08-19T10:45:13.429 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:45:13.429 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:45:13.429 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:45:13.429 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:45:13.429 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 80, 2024-08-19T10:45:13.429 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 36535, 2024-08-19T10:45:13.429 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:45:13.429 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:45:13.429 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:45:13.429 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:45:13.430 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:45:13.430 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:45:13.430 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:45:13.430 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:45:13.430 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:45:13.430 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.430 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:45:13.430 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:13.430 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:45:13.430 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:45:13.430 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.431 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.431 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.431 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.431 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.431 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.431 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.431 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.432 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:45:13.433 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:45:13.434 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:45:13.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.436 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.437 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.438 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.439 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.440 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.441 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.442 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:45:13.445 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.446 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.446 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.446 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:45:13.446 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:45:13.446 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.446 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.446 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.446 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:45:13.450 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:45:13.450 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.450 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:45:13.451 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.453 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:13.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.457 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.457 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:45:13.457 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:45:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1498043190 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f49f003be00 msgr2=0x7f49f003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a16a9a640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f49f003be00 0x7f49f003e2c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4a1013f100 tx=0x7f4a00009060 comp rx=0 tx=0).stop 2024-08-19T10:45:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a16a9a640 10 monclient: shutdown 2024-08-19T10:45:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a16a9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a100a5270 msgr2=0x7f4a10131d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a16a9a640 1 --2- 172.21.15.89:0/1498043190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a100a5270 0x7f4a10131d20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800e7f0 tx=0x7f4a0800ecb0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a15037640 1 -- 172.21.15.89:0/1498043190 reap_dead start 2024-08-19T10:45:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a15037640 1 -- 172.21.15.89:0/1498043190 reap_dead start 2024-08-19T10:45:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1498043190 shutdown_connections 2024-08-19T10:45:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1498043190 >> 172.21.15.89:0/1498043190 conn(0x7f4a10069a80 msgr2=0x7f4a100ab130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1498043190 shutdown_connections 2024-08-19T10:45:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.409+0000 7f4a16a9a640 1 -- 172.21.15.89:0/1498043190 wait complete. 2024-08-19T10:45:13.460 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:45:13.460 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-08-19T10:45:13.460 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:45:13.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient: get_monmap_and_config 2024-08-19T10:45:13.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient: monmap: 2024-08-19T10:45:13.495 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:13.495 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.495 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:13.497338+0000 2024-08-19T10:45:13.495 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:13.497338+0000 2024-08-19T10:45:13.495 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.495 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:13.495 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:13.495 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:13.495 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.496 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.496 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.496 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.496 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.496 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.496 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.496 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:13.496 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:13.496 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:13.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 1 Processor -- start 2024-08-19T10:45:13.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 1 -- start start 2024-08-19T10:45:13.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient: init 2024-08-19T10:45:13.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient: _add_conns ranks=[3,0,6,4,7,8,9,1,5,2,10] 2024-08-19T10:45:13.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient(hunting): picked mon.noname-g con 0x7f632c1377c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:13.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f632c0703c0 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:13.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f632c09ca60 0x7f632c0a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient(hunting): picked mon.noname-d con 0x7f632c09ca60 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:13.499 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f632c0a14d0 con 0x7f632c0703c0 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f632c0743f0 con 0x7f632c1377c0 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f632c070840 con 0x7f632c09ca60 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6334497640 10 monclient(hunting): authenticate will time out at 1359.066992s 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:35322/0 (socket says 172.21.15.190:35322) 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 1 -- 172.21.15.190:0/2956417572 learned_addr learned my addr 172.21.15.190:0/2956417572 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 10 monclient(hunting): get_auth_request con 0x7f632c1377c0 auth_method 0 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332a34640 1 --2- 172.21.15.190:0/2956417572 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f632c09ca60 0x7f632c0a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332a34640 10 monclient(hunting): get_auth_request con 0x7f632c09ca60 auth_method 0 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332a34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332a34640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332a34640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 10 monclient(hunting): handle_auth_done global_id 34707 payload 274 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 1 -- 172.21.15.190:0/2956417572 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f632c09ca60 msgr2=0x7f632c0a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 1 --2- 172.21.15.190:0/2956417572 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f632c09ca60 0x7f632c0a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 1 -- 172.21.15.190:0/2956417572 >> v1:172.21.15.89:6789/0 conn(0x7f632c0703c0 legacy=0x7f632c0a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 1 monclient: found mon.noname-g 2024-08-19T10:45:13.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:13.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332233640 1 -- 172.21.15.190:0/2956417572 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c074110 con 0x7f632c1377c0 2024-08-19T10:45:13.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.493+0000 7f6332a34640 1 --2- 172.21.15.190:0/2956417572 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f632c09ca60 0x7f632c0a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6332233640 1 -- 172.21.15.190:0/2956417572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 msgr2=0x7f632c137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6332233640 1 --2- 172.21.15.190:0/2956417572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c137ba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f631c0098a0 tx=0x7f631c01c260 comp rx=0 tx=0).ready entity=mon.3 client_cookie=fe7cb9ece335dcaf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 1 -- 172.21.15.190:0/2956417572 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f631c0b0070 con 0x7f632c1377c0 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 10 monclient: dump: 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.508 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 1 -- 172.21.15.190:0/2956417572 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f631c0aa810 con 0x7f632c1377c0 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 1 -- 172.21.15.190:0/2956417572 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f631c0a9300 con 0x7f632c1377c0 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6334497640 5 monclient: authenticate success, global_id 34707 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6334497640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6334497640 4 set_mon_vals no callback set 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.497+0000 7f6331231640 10 monclient: dump: 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.510 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.511 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient: shutdown 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 -- 172.21.15.190:0/2956417572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 msgr2=0x7f632c137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 --2- 172.21.15.190:0/2956417572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c137ba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f631c0098a0 tx=0x7f631c01c260 comp rx=0 tx=0).stop 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 -- 172.21.15.190:0/2956417572 shutdown_connections 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 --2- 172.21.15.190:0/2956417572 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f632c09ca60 0x7f632c0a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 --2- 172.21.15.190:0/2956417572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c137ba0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 -- 172.21.15.190:0/2956417572 >> 172.21.15.190:0/2956417572 conn(0x7f632c0a4700 msgr2=0x7f632c0a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 -- 172.21.15.190:0/2956417572 shutdown_connections 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 -- 172.21.15.190:0/2956417572 wait complete. 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient: monmap: 2024-08-19T10:45:13.513 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:13.513 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:13.513 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:13.513 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:13.513 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.513 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:13.513 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:13.513 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 Processor -- start 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 -- start start 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient: init 2024-08-19T10:45:13.514 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient: _add_conns ranks=[4,0,3,2,7,6,5,1,8] 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632c0703c0 0x7f632c1cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient(hunting): picked mon.noname-c con 0x7f632c0703c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632c09ca60 0x7f632c13e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient(hunting): picked mon.noname-a con 0x7f632c09ca60 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c13ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient(hunting): picked mon.noname-g con 0x7f632c1377c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f632c0a43e0 con 0x7f632c09ca60 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f632c0a4260 con 0x7f632c1377c0 2024-08-19T10:45:13.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f632c141e60 con 0x7f632c0703c0 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6334497640 10 monclient(hunting): authenticate will time out at 1359.074271s 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c13ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c13ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:35324/0 (socket says 172.21.15.190:35324) 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 1 -- 172.21.15.190:0/2124452460 learned_addr learned my addr 172.21.15.190:0/2124452460 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 10 monclient(hunting): get_auth_request con 0x7f632c1377c0 auth_method 0 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332233640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632c0703c0 0x7f632c1cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332233640 10 monclient(hunting): get_auth_request con 0x7f632c0703c0 auth_method 0 2024-08-19T10:45:13.516 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6331a32640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632c09ca60 0x7f632c13e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332233640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332233640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332233640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6331a32640 10 monclient(hunting): get_auth_request con 0x7f632c09ca60 auth_method 0 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6331a32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6331a32640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6331a32640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332233640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332233640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332233640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6331a32640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6331a32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6331a32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 10 monclient(hunting): handle_auth_done global_id 34716 payload 824 2024-08-19T10:45:13.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 1 -- 172.21.15.190:0/2124452460 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632c0703c0 msgr2=0x7f632c1cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632c0703c0 0x7f632c1cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 1 -- 172.21.15.190:0/2124452460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632c09ca60 msgr2=0x7f632c13e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632c09ca60 0x7f632c13e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 1 monclient: found mon.noname-g 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.501+0000 7f6332a34640 1 -- 172.21.15.190:0/2124452460 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c13ca40 con 0x7f632c1377c0 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6332a34640 1 -- 172.21.15.190:0/2124452460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 msgr2=0x7f632c13ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6332a34640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c13ed50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6324007f20 tx=0x7f632400d560 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 1 -- 172.21.15.190:0/2124452460 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f632401a070 con 0x7f632c1377c0 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 10 monclient: dump: 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.519 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.520 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:13.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 1 -- 172.21.15.190:0/2124452460 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6324003e10 con 0x7f632c1377c0 2024-08-19T10:45:13.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 1 -- 172.21.15.190:0/2124452460 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63240044a0 con 0x7f632c1377c0 2024-08-19T10:45:13.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:13.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 10 monclient: dump: 2024-08-19T10:45:13.522 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:13.522 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.522 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.522 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.522 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.522 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:13.522 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.522 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.523 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.523 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.523 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.523 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.523 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.523 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.523 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.523 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:13.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6334497640 5 monclient: authenticate success, global_id 34716 2024-08-19T10:45:13.523 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6334497640 10 monclient: _renew_subs 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6334497640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6330a30640 4 set_mon_vals no callback set 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6334497640 1 -- 172.21.15.190:0/2124452460 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632c0a6ae0 con 0x7f632c1377c0 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6334497640 10 monclient: _renew_subs 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6334497640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6334497640 1 -- 172.21.15.190:0/2124452460 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f632c13d2e0 con 0x7f632c1377c0 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6334497640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6334497640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6334497640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6334497640 1 -- 172.21.15.190:0/2124452460 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632c0a7010 con 0x7f632c1377c0 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 1 -- 172.21.15.190:0/2124452460 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6324004680 con 0x7f632c1377c0 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f631803be00 0x7f631803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.524 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f632affd640 1 -- 172.21.15.190:0/2124452460 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(232..232 src has 1..232) ==== 14144+0+0 (secure 0 0 0) 0x7f6324051150 con 0x7f632c1377c0 2024-08-19T10:45:13.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6332233640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f631803be00 0x7f631803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.505+0000 7f6332233640 10 monclient: get_auth_request con 0x7f631803be00 auth_method 0 2024-08-19T10:45:13.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.509+0000 7f6332233640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f631803be00 0x7f631803e2c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f632c0a5480 tx=0x7f631c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.509+0000 7f632affd640 1 -- 172.21.15.190:0/2124452460 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f632c13ca40 con 0x7f632c1377c0 2024-08-19T10:45:13.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.509+0000 7f632affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.525 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.509+0000 7f632affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:13.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient: get_monmap_and_config 2024-08-19T10:45:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient: monmap: 2024-08-19T10:45:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:45:13.577603+0000 2024-08-19T10:45:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:45:13.577603+0000 2024-08-19T10:45:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:13.576 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.577 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:13.578 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:13.578 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 1 Processor -- start 2024-08-19T10:45:13.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 1 -- start start 2024-08-19T10:45:13.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient: init 2024-08-19T10:45:13.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient: _add_conns ranks=[0,10,7,3,9,4,8,6,1,2,5] 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f76e40a6860 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f76e40a81b0 0x7f76e40a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient(hunting): picked mon.noname-f con 0x7f76e40a81b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f76e40a7180 0x7f76e41408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient(hunting): picked mon.noname-i con 0x7f76e40a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76e40a4c90 con 0x7f76e40a6860 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f76e40a7630 con 0x7f76e40a81b0 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f76e40a8590 con 0x7f76e40a7180 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76eb692640 10 monclient(hunting): authenticate will time out at 1363.394132s 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e942e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f76e40a6860 0x7f76e40a54d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39796/0 (socket says 172.21.15.89:39796) 2024-08-19T10:45:13.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e942e640 1 -- 172.21.15.89:0/2180205843 learned_addr learned my addr 172.21.15.89:0/2180205843 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:45:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 1 --2- 172.21.15.89:0/2180205843 >> v2:172.21.15.190:3300/0 conn(0x7f76e40a81b0 0x7f76e40a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e9c2f640 1 --2- 172.21.15.89:0/2180205843 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f76e40a7180 0x7f76e41408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 10 monclient(hunting): get_auth_request con 0x7f76e40a81b0 auth_method 0 2024-08-19T10:45:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e9c2f640 10 monclient(hunting): get_auth_request con 0x7f76e40a7180 auth_method 0 2024-08-19T10:45:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e9c2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e9c2f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e9c2f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e9c2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e9c2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e9c2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 10 monclient(hunting): handle_auth_done global_id 44200 payload 274 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 1 -- 172.21.15.89:0/2180205843 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f76e40a7180 msgr2=0x7f76e41408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 1 --2- 172.21.15.89:0/2180205843 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f76e40a7180 0x7f76e41408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 1 -- 172.21.15.89:0/2180205843 >> v1:172.21.15.89:6789/0 conn(0x7f76e40a6860 legacy=0x7f76e40a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 1 monclient: found mon.noname-f 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:45:13.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.573+0000 7f76e8c2d640 1 -- 172.21.15.89:0/2180205843 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e4141e30 con 0x7f76e40a81b0 2024-08-19T10:45:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.577+0000 7f76e8c2d640 1 -- 172.21.15.89:0/2180205843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76e40a81b0 msgr2=0x7f76e40a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.577+0000 7f76e8c2d640 1 --2- 172.21.15.89:0/2180205843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76e40a81b0 0x7f76e40a6c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009d90 tx=0x7f76d801c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0a783a328cb2819 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.577+0000 7f76e3fff640 1 -- 172.21.15.89:0/2180205843 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76d80a6b90 con 0x7f76e40a81b0 2024-08-19T10:45:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.577+0000 7f76e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.577+0000 7f76e3fff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:45:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.577+0000 7f76e3fff640 10 monclient: dump: 2024-08-19T10:45:13.592 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.593 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e3fff640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 5 monclient: authenticate success, global_id 44200 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e3fff640 1 -- 172.21.15.89:0/2180205843 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d80a6d50 con 0x7f76e40a81b0 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e3fff640 1 -- 172.21.15.89:0/2180205843 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76d80a58b0 con 0x7f76e40a81b0 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e3fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e3fff640 10 monclient: dump: 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.595 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 4 set_mon_vals no callback set 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient: shutdown 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 -- 172.21.15.89:0/2180205843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76e40a81b0 msgr2=0x7f76e40a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 --2- 172.21.15.89:0/2180205843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76e40a81b0 0x7f76e40a6c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009d90 tx=0x7f76d801c8c0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 -- 172.21.15.89:0/2180205843 shutdown_connections 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 --2- 172.21.15.89:0/2180205843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76e40a81b0 0x7f76e40a6c40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 --2- 172.21.15.89:0/2180205843 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f76e40a7180 0x7f76e41408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 -- 172.21.15.89:0/2180205843 >> 172.21.15.89:0/2180205843 conn(0x7f76e4069a00 msgr2=0x7f76e40abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 -- 172.21.15.89:0/2180205843 shutdown_connections 2024-08-19T10:45:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 -- 172.21.15.89:0/2180205843 wait complete. 2024-08-19T10:45:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient: monmap: 2024-08-19T10:45:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:45:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:45:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.597 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.598 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 Processor -- start 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 -- start start 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient: init 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient: _add_conns ranks=[5,7,6,0,1,4,2,3,8] 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f76e40a6860 0x7f76e41315a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient(hunting): picked mon.noname-h con 0x7f76e40a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f76e40a7180 0x7f76e4131ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient(hunting): picked mon.noname-i con 0x7f76e40a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a81b0 0x7f76e4132020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient(hunting): picked mon.noname-d con 0x7f76e40a81b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:13.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f76e41419c0 con 0x7f76e40a6860 2024-08-19T10:45:13.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f76e4141840 con 0x7f76e40a81b0 2024-08-19T10:45:13.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f76e4141b40 con 0x7f76e40a7180 2024-08-19T10:45:13.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76eb692640 10 monclient(hunting): authenticate will time out at 1363.402238s 2024-08-19T10:45:13.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a81b0 0x7f76e4132020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a81b0 0x7f76e4132020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:53096/0 (socket says 172.21.15.89:53096) 2024-08-19T10:45:13.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 1 -- 172.21.15.89:0/4070460354 learned_addr learned my addr 172.21.15.89:0/4070460354 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 10 monclient(hunting): get_auth_request con 0x7f76e40a81b0 auth_method 0 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e8c2d640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f76e40a7180 0x7f76e4131ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e942e640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f76e40a6860 0x7f76e41315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e8c2d640 10 monclient(hunting): get_auth_request con 0x7f76e40a7180 auth_method 0 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e8c2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e8c2d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e8c2d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e942e640 10 monclient(hunting): get_auth_request con 0x7f76e40a6860 auth_method 0 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e942e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e942e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e942e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e8c2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e8c2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e8c2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 10 monclient(hunting): handle_auth_done global_id 44286 payload 824 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 1 -- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f76e40a7180 msgr2=0x7f76e4131ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f76e40a7180 0x7f76e4131ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 1 -- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f76e40a6860 msgr2=0x7f76e41315a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f76e40a6860 0x7f76e41315a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 1 monclient: found mon.noname-d 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e9c2f640 1 -- 172.21.15.89:0/4070460354 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e4139940 con 0x7f76e40a81b0 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.581+0000 7f76e942e640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f76e40a6860 0x7f76e41315a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e9c2f640 1 -- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a81b0 msgr2=0x7f76e4132020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e9c2f640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a81b0 0x7f76e4132020 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76d400e890 tx=0x7f76d400ed50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76d4017070 con 0x7f76e40a81b0 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:45:13.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 10 monclient: dump: 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.604 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d4013470 con 0x7f76e40a81b0 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76eb692640 5 monclient: authenticate success, global_id 44286 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76eb692640 10 monclient: _renew_subs 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76eb692640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76eb692640 1 -- 172.21.15.89:0/4070460354 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e4139c20 con 0x7f76e40a81b0 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76d4013a90 con 0x7f76e40a81b0 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 10 monclient: dump: 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.606 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e37fe640 4 set_mon_vals no callback set 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76eb692640 10 monclient: _renew_subs 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76eb692640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76eb692640 1 -- 172.21.15.89:0/4070460354 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e4140680 con 0x7f76e40a81b0 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76eb692640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76eb692640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a9c60 0x7f76e40aa040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76eb692640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76eb692640 1 -- 172.21.15.89:0/4070460354 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f76e40ab280 con 0x7f76e40a9c60 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e942e640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a9c60 0x7f76e40aa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e942e640 10 monclient: get_auth_request con 0x7f76e40a9c60 auth_method 0 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e942e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e942e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e942e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e942e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e942e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e942e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e942e640 10 monclient(hunting): handle_auth_done global_id 44295 payload 824 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e942e640 1 -- 172.21.15.89:0/4070460354 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76e4140910 con 0x7f76e40a9c60 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f76d401a070 con 0x7f76e40a81b0 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f76d003be00 0x7f76d003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(232..232 src has 1..232) ==== 14144+0+0 (secure 0 0 0) 0x7f76d400f040 con 0x7f76e40a81b0 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e8c2d640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f76d003be00 0x7f76d003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.585+0000 7f76e8c2d640 10 monclient: get_auth_request con 0x7f76d003be00 auth_method 0 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e942e640 1 -- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a9c60 msgr2=0x7f76e40aa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e942e640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a9c60 0x7f76e40aa040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00e730 tx=0x7f76dc00ebf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76dc0070d0 con 0x7f76e40a9c60 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e1ffb640 10 monclient: dump: 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.609 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f76dc018070 con 0x7f76e40a9c60 2024-08-19T10:45:13.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e1ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:13.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a9c60 msgr2=0x7f76e40aa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e1ffb640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a9c60 0x7f76e40aa040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00e730 tx=0x7f76dc00ebf0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a9c60 msgr2=0x7f76e40aa040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:13.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e1ffb640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a9c60 0x7f76e40aa040 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00e730 tx=0x7f76dc00ebf0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.589+0000 7f76e8c2d640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f76d003be00 0x7f76d003e2c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f76e40a6330 tx=0x7f76d80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76eb692640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:45:13.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76eb692640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40aa580 0x7f76e4136f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76eb692640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76eb692640 1 -- 172.21.15.89:0/4070460354 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f76e40a6c40 con 0x7f76e40aa580 2024-08-19T10:45:13.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76e942e640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40aa580 0x7f76e4136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76e942e640 10 monclient: get_auth_request con 0x7f76e40aa580 auth_method 0 2024-08-19T10:45:13.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76e942e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76e942e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76e942e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76e942e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76e942e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76e942e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76e942e640 10 monclient(hunting): handle_auth_done global_id 44304 payload 824 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.593+0000 7f76e942e640 1 -- 172.21.15.89:0/4070460354 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f76e4064590 con 0x7f76e40aa580 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e942e640 1 -- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40aa580 msgr2=0x7f76e4136f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e942e640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40aa580 0x7f76e4136f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00e700 tx=0x7f76dc004950 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76dc007540 con 0x7f76e40aa580 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e1ffb640 10 monclient: dump: 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.614 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.615 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.616 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9422 (secure 0 0 0) 0x7f76dc0070d0 con 0x7f76e40aa580 2024-08-19T10:45:13.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:45:13.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e1ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:13.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40aa580 msgr2=0x7f76e4136f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e1ffb640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40aa580 0x7f76e4136f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00e700 tx=0x7f76dc004950 comp rx=0 tx=0).stop 2024-08-19T10:45:13.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e1ffb640 1 -- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40aa580 msgr2=0x7f76e4136f50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:13.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e1ffb640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40aa580 0x7f76e4136f50 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00e700 tx=0x7f76dc004950 comp rx=0 tx=0).stop 2024-08-19T10:45:13.616 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 80, 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 36723, 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:45:13.617 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:45:13.618 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:45:13.618 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.618 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:45:13.618 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:13.618 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:45:13.618 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:45:13.618 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.618 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.618 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.618 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.618 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.619 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:45:13.620 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:45:13.621 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.622 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:45:13.623 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.624 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.625 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:45:13.627 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:45:13.628 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.629 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:45:13.631 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.634 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:45:13.635 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.636 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.637 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.638 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.639 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.640 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:45:13.641 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.641 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.641 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:45:13.641 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:45:13.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76eb692640 1 -- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f76d003be00 msgr2=0x7f76d003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76eb692640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f76d003be00 0x7f76d003e2c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f76e40a6330 tx=0x7f76d80a7000 comp rx=0 tx=0).stop 2024-08-19T10:45:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76eb692640 10 monclient: shutdown 2024-08-19T10:45:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76eb692640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76eb692640 1 -- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a81b0 msgr2=0x7f76e4132020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76eb692640 1 --2- 172.21.15.89:0/4070460354 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f76e40a81b0 0x7f76e4132020 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76d400e890 tx=0x7f76d400ed50 comp rx=0 tx=0).stop 2024-08-19T10:45:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e9c2f640 1 -- 172.21.15.89:0/4070460354 reap_dead start 2024-08-19T10:45:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76e9c2f640 1 -- 172.21.15.89:0/4070460354 reap_dead start 2024-08-19T10:45:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76eb692640 1 -- 172.21.15.89:0/4070460354 shutdown_connections 2024-08-19T10:45:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76eb692640 1 -- 172.21.15.89:0/4070460354 >> 172.21.15.89:0/4070460354 conn(0x7f76e4069a00 msgr2=0x7f76e4135c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76eb692640 1 -- 172.21.15.89:0/4070460354 shutdown_connections 2024-08-19T10:45:13.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.597+0000 7f76eb692640 1 -- 172.21.15.89:0/4070460354 wait complete. 2024-08-19T10:45:13.644 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:45:13.644 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-08-19T10:45:13.644 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:45:13.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.685+0000 7f6334497640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h5"}] 2024-08-19T10:45:13.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.685+0000 7f6334497640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h5"}] 2024-08-19T10:45:13.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.685+0000 7f6334497640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:13.687 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:13.685+0000 7f6334497640 1 -- 172.21.15.190:0/2124452460 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h5"} v 0) -- 0x7f632c137ba0 con 0x7f632c1377c0 2024-08-19T10:45:13.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient: get_monmap_and_config 2024-08-19T10:45:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient: monmap: 2024-08-19T10:45:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:45:13.749440+0000 2024-08-19T10:45:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:45:13.749440+0000 2024-08-19T10:45:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:13.748 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:13.749 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:13.749 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 1 Processor -- start 2024-08-19T10:45:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 1 -- start start 2024-08-19T10:45:13.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient: init 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient: _add_conns ranks=[6,4,0,9,5,3,7,1,10,2,8] 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f458c0a6860 0x7f458c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient(hunting): picked mon.noname-d con 0x7f458c0a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f458c0a7180 0x7f458c0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient(hunting): picked mon.noname-c con 0x7f458c0a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f458c139620 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c0a33d0 con 0x7f458c139620 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f458c0a8190 con 0x7f458c0a7180 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f458c0a2ec0 con 0x7f458c0a6860 2024-08-19T10:45:13.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f4591b9e640 10 monclient(hunting): authenticate will time out at 1363.566193s 2024-08-19T10:45:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f458c0a6860 0x7f458c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458affd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f458c0a7180 0x7f458c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458bfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f458c139620 0x7f458c139a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39800/0 (socket says 172.21.15.89:39800) 2024-08-19T10:45:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 10 monclient(hunting): get_auth_request con 0x7f458c0a6860 auth_method 0 2024-08-19T10:45:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458bfff640 1 -- 172.21.15.89:0/2167214695 learned_addr learned my addr 172.21.15.89:0/2167214695 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:45:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458affd640 10 monclient(hunting): get_auth_request con 0x7f458c0a7180 auth_method 0 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 10 monclient(hunting): handle_auth_done global_id 44313 payload 274 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 1 -- 172.21.15.89:0/2167214695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f458c0a7180 msgr2=0x7f458c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:13.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 1 --2- 172.21.15.89:0/2167214695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f458c0a7180 0x7f458c0a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 1 -- 172.21.15.89:0/2167214695 >> v1:172.21.15.89:6789/0 conn(0x7f458c139620 legacy=0x7f458c139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:13.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 1 monclient: found mon.noname-d 2024-08-19T10:45:13.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:45:13.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.745+0000 7f458b7fe640 1 -- 172.21.15.89:0/2167214695 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c0a4cd0 con 0x7f458c0a6860 2024-08-19T10:45:13.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458b7fe640 1 -- 172.21.15.89:0/2167214695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f458c0a6860 msgr2=0x7f458c0a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458b7fe640 1 --2- 172.21.15.89:0/2167214695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f458c0a6860 0x7f458c0a6c40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f457c0097f0 tx=0x7f457c01c3a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c551350897a008d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 1 -- 172.21.15.89:0/2167214695 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f457c0b0070 con 0x7f458c0a6860 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 10 monclient: dump: 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.761 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 5 monclient: authenticate success, global_id 44313 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 1 -- 172.21.15.89:0/2167214695 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f457c0aa810 con 0x7f458c0a6860 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 1 -- 172.21.15.89:0/2167214695 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f457c0aa9c0 con 0x7f458c0a6860 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 4 set_mon_vals no callback set 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f458a7fc640 10 monclient: dump: 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.763 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 10 monclient: shutdown 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 1 -- 172.21.15.89:0/2167214695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f458c0a6860 msgr2=0x7f458c0a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 1 --2- 172.21.15.89:0/2167214695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f458c0a6860 0x7f458c0a6c40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f457c0097f0 tx=0x7f457c01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 1 -- 172.21.15.89:0/2167214695 shutdown_connections 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 1 --2- 172.21.15.89:0/2167214695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f458c0a7180 0x7f458c0a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 1 --2- 172.21.15.89:0/2167214695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f458c0a6860 0x7f458c0a6c40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 1 -- 172.21.15.89:0/2167214695 >> 172.21.15.89:0/2167214695 conn(0x7f458c074b40 msgr2=0x7f458c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 1 -- 172.21.15.89:0/2167214695 shutdown_connections 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.749+0000 7f4591b9e640 1 -- 172.21.15.89:0/2167214695 wait complete. 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient: monmap: 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:13.765 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.766 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 1 Processor -- start 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 1 -- start start 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient: init 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient: _add_conns ranks=[8,3,5,0,1,4,2,6,7] 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c0a6860 0x7f458c138480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient(hunting): picked mon.noname-e con 0x7f458c0a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f458c0a7180 0x7f458c13d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient(hunting): picked mon.noname-g con 0x7f458c0a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f458c139620 0x7f458c13db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient(hunting): picked mon.noname-h con 0x7f458c139620 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:13.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f458c144430 con 0x7f458c0a7180 2024-08-19T10:45:13.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f458c1442b0 con 0x7f458c139620 2024-08-19T10:45:13.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f458c1445b0 con 0x7f458c0a6860 2024-08-19T10:45:13.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient(hunting): authenticate will time out at 1363.573031s 2024-08-19T10:45:13.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c0a6860 0x7f458c138480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c0a6860 0x7f458c138480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:45224/0 (socket says 172.21.15.89:45224) 2024-08-19T10:45:13.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 1 -- 172.21.15.89:0/1271414282 learned_addr learned my addr 172.21.15.89:0/1271414282 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:13.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 10 monclient(hunting): get_auth_request con 0x7f458c0a6860 auth_method 0 2024-08-19T10:45:13.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 10 monclient(hunting): handle_auth_done global_id 6227 payload 824 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 1 -- 172.21.15.89:0/1271414282 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f458c139620 msgr2=0x7f458c13db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f458c139620 0x7f458c13db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 1 -- 172.21.15.89:0/1271414282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f458c0a7180 msgr2=0x7f458c13d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f458c0a7180 0x7f458c13d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 1 monclient: found mon.noname-e 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 1 -- 172.21.15.89:0/1271414282 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c13fde0 con 0x7f458c0a6860 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 1 -- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c0a6860 msgr2=0x7f458c138480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f458b7fe640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c0a6860 0x7f458c138480 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f457c01c1d0 tx=0x7f457c019140 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f457c0b0040 con 0x7f458c0a6860 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 10 monclient: dump: 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.771 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f457c0b9070 con 0x7f458c0a6860 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f457c0044b0 con 0x7f458c0a6860 2024-08-19T10:45:13.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4573fff640 10 monclient: dump: 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.773 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4589ffb640 4 set_mon_vals no callback set 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 5 monclient: authenticate success, global_id 6227 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient: _renew_subs 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.753+0000 7f4591b9e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4591b9e640 1 -- 172.21.15.89:0/1271414282 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c0ab2b0 con 0x7f458c0a6860 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4591b9e640 10 monclient: _renew_subs 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4591b9e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:13.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4591b9e640 1 -- 172.21.15.89:0/1271414282 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c122c50 con 0x7f458c0a6860 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4591b9e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4591b9e640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c1389c0 0x7f458c13e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4591b9e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4591b9e640 1 -- 172.21.15.89:0/1271414282 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f458c140910 con 0x7f458c1389c0 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c1389c0 0x7f458c13e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 10 monclient: get_auth_request con 0x7f458c1389c0 auth_method 0 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f457c004040 con 0x7f458c0a6860 2024-08-19T10:45:13.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 10 monclient(hunting): handle_auth_done global_id 6236 payload 824 2024-08-19T10:45:13.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 1 -- 172.21.15.89:0/1271414282 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f458c0a6c40 con 0x7f458c1389c0 2024-08-19T10:45:13.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f457803be00 0x7f457803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(232..232 src has 1..232) ==== 14144+0+0 (secure 0 0 0) 0x7f457c0eace0 con 0x7f458c0a6860 2024-08-19T10:45:13.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458bfff640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f457803be00 0x7f457803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458bfff640 10 monclient: get_auth_request con 0x7f457803be00 auth_method 0 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 1 -- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c1389c0 msgr2=0x7f458c13e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f458affd640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c1389c0 0x7f458c13e0c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f458c1395f0 tx=0x7f458000ec50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4580017070 con 0x7f458c1389c0 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 10 monclient: dump: 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.779 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f458000cb40 con 0x7f458c1389c0 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c1389c0 msgr2=0x7f458c13e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c1389c0 0x7f458c13e0c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f458c1395f0 tx=0x7f458000ec50 comp rx=0 tx=0).stop 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c1389c0 msgr2=0x7f458c13e0c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.757+0000 7f4573fff640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c1389c0 0x7f458c13e0c0 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f458c1395f0 tx=0x7f458000ec50 comp rx=0 tx=0).stop 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.761+0000 7f458bfff640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f457803be00 0x7f457803e2c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f458c0a60b0 tx=0x7f4584004fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f4591b9e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f4591b9e640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c13e600 0x7f458c0a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f4591b9e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f4591b9e640 1 -- 172.21.15.89:0/1271414282 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f458c13f030 con 0x7f458c13e600 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f458affd640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c13e600 0x7f458c0a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f458affd640 10 monclient: get_auth_request con 0x7f458c13e600 auth_method 0 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f458affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f458affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f458affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f458affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f458affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f458affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f458affd640 10 monclient(hunting): handle_auth_done global_id 6245 payload 824 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.765+0000 7f458affd640 1 -- 172.21.15.89:0/1271414282 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f458c1228d0 con 0x7f458c13e600 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f458affd640 1 -- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c13e600 msgr2=0x7f458c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f458affd640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c13e600 0x7f458c0a01f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f458000ec90 tx=0x7f45800046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4580017030 con 0x7f458c13e600 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4573fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4573fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4573fff640 10 monclient: dump: 2024-08-19T10:45:13.782 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.783 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.784 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9423 (secure 0 0 0) 0x7f4580003620 con 0x7f458c13e600 2024-08-19T10:45:13.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4573fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:45:13.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4573fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:13.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c13e600 msgr2=0x7f458c0a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4573fff640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c13e600 0x7f458c0a01f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f458000ec90 tx=0x7f45800046d0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4573fff640 1 -- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c13e600 msgr2=0x7f458c0a01f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:13.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4573fff640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c13e600 0x7f458c0a01f0 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f458000ec90 tx=0x7f45800046d0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.784 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 80, 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 443562, 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:45:13.785 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:45:13.786 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:45:13.786 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:45:13.786 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.786 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:45:13.786 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:13.786 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:45:13.786 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:45:13.786 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.786 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.786 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.786 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.787 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:45:13.788 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.790 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.792 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.793 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:45:13.795 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:45:13.796 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.797 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.798 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:45:13.799 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.801 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.802 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.803 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:45:13.804 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:45:13.805 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.806 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.807 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:45:13.808 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.809 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.809 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.809 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:45:13.809 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.809 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.809 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:45:13.809 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:45:13.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4591b9e640 1 -- 172.21.15.89:0/1271414282 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f457803be00 msgr2=0x7f457803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4591b9e640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f457803be00 0x7f457803e2c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f458c0a60b0 tx=0x7f4584004fc0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4591b9e640 10 monclient: shutdown 2024-08-19T10:45:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4591b9e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4591b9e640 1 -- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c0a6860 msgr2=0x7f458c138480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4591b9e640 1 --2- 172.21.15.89:0/1271414282 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f458c0a6860 0x7f458c138480 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f457c01c1d0 tx=0x7f457c019140 comp rx=0 tx=0).stop 2024-08-19T10:45:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f458bfff640 1 -- 172.21.15.89:0/1271414282 reap_dead start 2024-08-19T10:45:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f458bfff640 1 -- 172.21.15.89:0/1271414282 reap_dead start 2024-08-19T10:45:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4591b9e640 1 -- 172.21.15.89:0/1271414282 shutdown_connections 2024-08-19T10:45:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4591b9e640 1 -- 172.21.15.89:0/1271414282 >> 172.21.15.89:0/1271414282 conn(0x7f458c074b40 msgr2=0x7f458c0a9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4591b9e640 1 -- 172.21.15.89:0/1271414282 shutdown_connections 2024-08-19T10:45:13.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.769+0000 7f4591b9e640 1 -- 172.21.15.89:0/1271414282 wait complete. 2024-08-19T10:45:13.812 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:45:13.812 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-08-19T10:45:13.812 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:45:13.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.921+0000 7f97acc2d640 10 monclient: get_monmap_and_config 2024-08-19T10:45:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.921+0000 7f97acc2d640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.921+0000 7f97acc2d640 10 monclient: monmap: 2024-08-19T10:45:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:45:13.928606+0000 2024-08-19T10:45:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:45:13.928606+0000 2024-08-19T10:45:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:13.927 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:13.928 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:13.928 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 1 Processor -- start 2024-08-19T10:45:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 1 -- start start 2024-08-19T10:45:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 10 monclient: init 2024-08-19T10:45:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 10 monclient: _add_conns ranks=[8,4,9,10,7,6,5,3,2,1,0] 2024-08-19T10:45:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a5270 0x7f97a80a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 10 monclient(hunting): picked mon.noname-e con 0x7f97a80a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97a80a6d30 0x7f97a80a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 10 monclient(hunting): picked mon.noname-c con 0x7f97a80a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97a81395b0 0x7f97a8139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 10 monclient(hunting): picked mon.noname-a con 0x7f97a81395b0 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f97a80a7650 con 0x7f97a81395b0 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f97a80a8710 con 0x7f97a80a6d30 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f97a80a5a40 con 0x7f97a80a5270 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97acc2d640 10 monclient(hunting): authenticate will time out at 1363.745335s 2024-08-19T10:45:13.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a6d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97a81395b0 0x7f97a8139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a659c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a5270 0x7f97a80a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a6d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f97a81395b0 0x7f97a8139a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42918/0 (socket says 172.21.15.89:42918) 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97a80a6d30 0x7f97a80a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a6d9d640 1 -- 172.21.15.89:0/2384477019 learned_addr learned my addr 172.21.15.89:0/2384477019 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a6d9d640 10 monclient(hunting): get_auth_request con 0x7f97a81395b0 auth_method 0 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a659c640 10 monclient(hunting): get_auth_request con 0x7f97a80a5270 auth_method 0 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 10 monclient(hunting): get_auth_request con 0x7f97a80a6d30 auth_method 0 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 10 monclient(hunting): handle_auth_done global_id 44311 payload 274 2024-08-19T10:45:13.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 1 -- 172.21.15.89:0/2384477019 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a5270 msgr2=0x7f97a80a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 1 --2- 172.21.15.89:0/2384477019 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a5270 0x7f97a80a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 1 -- 172.21.15.89:0/2384477019 >> v2:172.21.15.89:3300/0 conn(0x7f97a81395b0 msgr2=0x7f97a8139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 1 --2- 172.21.15.89:0/2384477019 >> v2:172.21.15.89:3300/0 conn(0x7f97a81395b0 0x7f97a8139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 1 monclient: found mon.noname-c 2024-08-19T10:45:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:45:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a5d9b640 1 -- 172.21.15.89:0/2384477019 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a80a81d0 con 0x7f97a80a6d30 2024-08-19T10:45:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a6d9d640 1 --2- 172.21.15.89:0/2384477019 >> v2:172.21.15.89:3300/0 conn(0x7f97a81395b0 0x7f97a8139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:13.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.925+0000 7f97a659c640 1 --2- 172.21.15.89:0/2384477019 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a5270 0x7f97a80a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a5d9b640 1 -- 172.21.15.89:0/2384477019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97a80a6d30 msgr2=0x7f97a80a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a5d9b640 1 --2- 172.21.15.89:0/2384477019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97a80a6d30 0x7f97a80a7110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f979c0097c0 tx=0x7f979c01c360 comp rx=0 tx=0).ready entity=mon.4 client_cookie=55409820b2863a96 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 1 -- 172.21.15.89:0/2384477019 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f979c0b0070 con 0x7f97a80a6d30 2024-08-19T10:45:13.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 10 monclient: dump: 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.938 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 1 -- 172.21.15.89:0/2384477019 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979c0aa810 con 0x7f97a80a6d30 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 1 -- 172.21.15.89:0/2384477019 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f979c0a9300 con 0x7f97a80a6d30 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 5 monclient: authenticate success, global_id 44311 2024-08-19T10:45:13.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 4 set_mon_vals no callback set 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97a559a640 10 monclient: dump: 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.940 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 10 monclient: shutdown 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 1 -- 172.21.15.89:0/2384477019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97a80a6d30 msgr2=0x7f97a80a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 1 --2- 172.21.15.89:0/2384477019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97a80a6d30 0x7f97a80a7110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f979c0097c0 tx=0x7f979c01c360 comp rx=0 tx=0).stop 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 1 -- 172.21.15.89:0/2384477019 shutdown_connections 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 1 --2- 172.21.15.89:0/2384477019 >> v2:172.21.15.89:3300/0 conn(0x7f97a81395b0 0x7f97a8139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 1 --2- 172.21.15.89:0/2384477019 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f97a80a6d30 0x7f97a80a7110 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 1 --2- 172.21.15.89:0/2384477019 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a5270 0x7f97a80a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 1 -- 172.21.15.89:0/2384477019 >> 172.21.15.89:0/2384477019 conn(0x7f97a8069a80 msgr2=0x7f97a80abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 1 -- 172.21.15.89:0/2384477019 shutdown_connections 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 1 -- 172.21.15.89:0/2384477019 wait complete. 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 10 monclient: build_initial_monmap 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 10 monclient: monmap: 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:13.942 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 1 Processor -- start 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 1 -- start start 2024-08-19T10:45:13.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.929+0000 7f97acc2d640 10 monclient: init 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient: _add_conns ranks=[1,8,6,3,4,0,2,7,5] 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a80a5270 0x7f97a813ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient(hunting): picked mon.noname-f con 0x7f97a80a5270 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a6d30 0x7f97a809cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient(hunting): picked mon.noname-e con 0x7f97a80a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f97a81395b0 0x7f97a809d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient(hunting): picked mon.noname-d con 0x7f97a81395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f97a80a5650 con 0x7f97a80a5270 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f97a813a350 con 0x7f97a81395b0 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f97a80a57d0 con 0x7f97a80a6d30 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient(hunting): authenticate will time out at 1363.752443s 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a6d30 0x7f97a809cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a6d30 0x7f97a809cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:45272/0 (socket says 172.21.15.89:45272) 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 1 -- 172.21.15.89:0/3595994512 learned_addr learned my addr 172.21.15.89:0/3595994512 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 10 monclient(hunting): get_auth_request con 0x7f97a80a6d30 auth_method 0 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a6d9d640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f97a81395b0 0x7f97a809d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a6d9d640 10 monclient(hunting): get_auth_request con 0x7f97a81395b0 auth_method 0 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 10 monclient(hunting): handle_auth_done global_id 6263 payload 824 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 1 -- 172.21.15.89:0/3595994512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f97a81395b0 msgr2=0x7f97a809d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f97a81395b0 0x7f97a809d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 1 -- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a80a5270 msgr2=0x7f97a813ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a80a5270 0x7f97a813ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:45:13.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 1 monclient: found mon.noname-e 2024-08-19T10:45:13.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:13.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:13.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 1 -- 172.21.15.89:0/3595994512 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a8139be0 con 0x7f97a80a6d30 2024-08-19T10:45:13.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 1 -- 172.21.15.89:0/3595994512 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a6d30 msgr2=0x7f97a809cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a5d9b640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a6d30 0x7f97a809cef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f979c01cc00 tx=0x7f979c019100 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f979c0b0040 con 0x7f97a80a6d30 2024-08-19T10:45:13.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:13.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 10 monclient: dump: 2024-08-19T10:45:13.951 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.952 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 10 monclient: _finish_auth 0 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979c0aae30 con 0x7f97a80a6d30 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 5 monclient: authenticate success, global_id 6263 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f979c004250 con 0x7f97a80a6d30 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient: _renew_subs 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:13.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 1 -- 172.21.15.89:0/3595994512 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a809f550 con 0x7f97a80a6d30 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97a4d99640 4 set_mon_vals no callback set 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f9796ffd640 10 monclient: dump: 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.954 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient: _renew_subs 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 1 -- 172.21.15.89:0/3595994512 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a809e030 con 0x7f97a80a6d30 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d10b0 0x7f97a81d1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.933+0000 7f97acc2d640 1 -- 172.21.15.89:0/3595994512 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f97a809d9d0 con 0x7f97a81d10b0 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f979c003620 con 0x7f97a80a6d30 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f9796ffd640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f978803be00 0x7f978803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d10b0 0x7f97a81d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(232..232 src has 1..232) ==== 14144+0+0 (secure 0 0 0) 0x7f979c0eaa70 con 0x7f97a80a6d30 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 10 monclient: get_auth_request con 0x7f97a81d10b0 auth_method 0 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a6d9d640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f978803be00 0x7f978803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a6d9d640 10 monclient: get_auth_request con 0x7f978803be00 auth_method 0 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 10 monclient(hunting): handle_auth_done global_id 44209 payload 824 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 1 -- 172.21.15.89:0/3595994512 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97a80a7110 con 0x7f97a81d10b0 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 1 -- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d10b0 msgr2=0x7f97a81d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a659c640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d10b0 0x7f97a81d1490 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f979800e7a0 tx=0x7f979800ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f97a6d9d640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f978803be00 0x7f978803e2c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009c90 tx=0x7f97a000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9798017070 con 0x7f97a81d10b0 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f9796ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f9796ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.937+0000 7f9796ffd640 10 monclient: dump: 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.957 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.958 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.941+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f979800cb40 con 0x7f97a81d10b0 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.941+0000 7f9796ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.941+0000 7f9796ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.941+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d10b0 msgr2=0x7f97a81d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.941+0000 7f9796ffd640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d10b0 0x7f97a81d1490 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f979800e7a0 tx=0x7f979800ec60 comp rx=0 tx=0).stop 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.941+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d10b0 msgr2=0x7f97a81d1490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.941+0000 7f9796ffd640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d10b0 0x7f97a81d1490 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f979800e7a0 tx=0x7f979800ec60 comp rx=0 tx=0).stop 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97acc2d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97acc2d640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d19d0 0x7f97a813adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97acc2d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97acc2d640 1 -- 172.21.15.89:0/3595994512 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f97a8064590 con 0x7f97a81d19d0 2024-08-19T10:45:13.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97a659c640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d19d0 0x7f97a813adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97a659c640 10 monclient: get_auth_request con 0x7f97a81d19d0 auth_method 0 2024-08-19T10:45:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97a659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97a659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97a659c640 10 monclient(hunting): handle_auth_done global_id 44218 payload 824 2024-08-19T10:45:13.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.945+0000 7f97a659c640 1 -- 172.21.15.89:0/3595994512 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f97a813f8b0 con 0x7f97a81d19d0 2024-08-19T10:45:13.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97a659c640 1 -- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d19d0 msgr2=0x7f97a813adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:13.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97a659c640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d19d0 0x7f97a813adc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f979800e7a0 tx=0x7f97980046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:13.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9798017070 con 0x7f97a81d19d0 2024-08-19T10:45:13.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f9796ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:13.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f9796ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:13.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f9796ffd640 10 monclient: dump: 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:13.962 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:13.963 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:13.963 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:13.963 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:13.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9255 (secure 0 0 0) 0x7f9798006650 con 0x7f97a81d19d0 2024-08-19T10:45:13.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f9796ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:45:13.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f9796ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:13.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d19d0 msgr2=0x7f97a813adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f9796ffd640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d19d0 0x7f97a813adc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f979800e7a0 tx=0x7f97980046d0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f9796ffd640 1 -- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d19d0 msgr2=0x7f97a813adc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:13.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f9796ffd640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97a81d19d0 0x7f97a813adc0 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f979800e7a0 tx=0x7f97980046d0 comp rx=0 tx=0).stop 2024-08-19T10:45:13.963 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:45:13.964 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:45:13.964 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:45:13.964 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:45:13.964 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 80, 2024-08-19T10:45:13.964 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 37079, 2024-08-19T10:45:13.964 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:45:13.964 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:45:13.964 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:45:13.964 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:45:13.964 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:45:13.964 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.965 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.966 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:45:13.967 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:45:13.968 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:13.969 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:45:13.969 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:13.969 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:13.969 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:13.969 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:13.969 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:13.969 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:13.969 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:13.969 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.970 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.971 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.972 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.973 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.976 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:45:13.977 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.978 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.979 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.980 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:45:13.981 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.982 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.984 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:13.985 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:45:13.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:13.987 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:45:13.988 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:45:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97acc2d640 1 -- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f978803be00 msgr2=0x7f978803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97acc2d640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f978803be00 0x7f978803e2c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009c90 tx=0x7f97a000b040 comp rx=0 tx=0).stop 2024-08-19T10:45:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97acc2d640 10 monclient: shutdown 2024-08-19T10:45:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97acc2d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97acc2d640 1 -- 172.21.15.89:0/3595994512 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a6d30 msgr2=0x7f97a809cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97acc2d640 1 --2- 172.21.15.89:0/3595994512 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f97a80a6d30 0x7f97a809cef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f979c01cc00 tx=0x7f979c019100 comp rx=0 tx=0).stop 2024-08-19T10:45:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97a6d9d640 1 -- 172.21.15.89:0/3595994512 reap_dead start 2024-08-19T10:45:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97a6d9d640 1 -- 172.21.15.89:0/3595994512 reap_dead start 2024-08-19T10:45:13.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97acc2d640 1 -- 172.21.15.89:0/3595994512 shutdown_connections 2024-08-19T10:45:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97acc2d640 1 -- 172.21.15.89:0/3595994512 >> 172.21.15.89:0/3595994512 conn(0x7f97a8069a80 msgr2=0x7f97a80ab3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97acc2d640 1 -- 172.21.15.89:0/3595994512 shutdown_connections 2024-08-19T10:45:13.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:13.949+0000 7f97acc2d640 1 -- 172.21.15.89:0/3595994512 wait complete. 2024-08-19T10:45:13.990 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:45:13.990 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-08-19T10:45:13.990 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:45:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient: get_monmap_and_config 2024-08-19T10:45:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient: build_initial_monmap 2024-08-19T10:45:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient: monmap: 2024-08-19T10:45:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:45:14.105250+0000 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:45:14.105250+0000 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:14.106 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 1 Processor -- start 2024-08-19T10:45:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 1 -- start start 2024-08-19T10:45:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient: init 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient: _add_conns ranks=[1,4,3,0,6,7,9,10,2,8,5] 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f10640a6860 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10640a7350 0x7f106413e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient(hunting): picked mon.noname-c con 0x7f10640a7350 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10640a7730 0x7f1064140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient(hunting): picked mon.noname-g con 0x7f10640a7730 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10640a33d0 con 0x7f10640a6860 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f10640a8190 con 0x7f10640a7730 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f10640a2ec0 con 0x7f10640a7350 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1069c23640 10 monclient(hunting): authenticate will time out at 1363.921909s 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10640a7350 0x7f106413e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10640a7350 0x7f106413e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:54408/0 (socket says 172.21.15.89:54408) 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 1 -- 172.21.15.89:0/4188111973 learned_addr learned my addr 172.21.15.89:0/4188111973 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 10 monclient(hunting): get_auth_request con 0x7f10640a7350 auth_method 0 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1063fff640 1 --2- 172.21.15.89:0/4188111973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10640a7730 0x7f1064140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1063fff640 10 monclient(hunting): get_auth_request con 0x7f10640a7730 auth_method 0 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1063fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1063fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1063fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 10 monclient(hunting): handle_auth_done global_id 44320 payload 274 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 1 -- 172.21.15.89:0/4188111973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10640a7730 msgr2=0x7f1064140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 1 --2- 172.21.15.89:0/4188111973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10640a7730 0x7f1064140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 1 -- 172.21.15.89:0/4188111973 >> v1:172.21.15.190:6789/0 conn(0x7f10640a6860 legacy=0x7f10640a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 1 monclient: found mon.noname-c 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:45:14.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1062ffd640 1 -- 172.21.15.89:0/4188111973 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10640a4cd0 con 0x7f10640a7350 2024-08-19T10:45:14.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.101+0000 7f1063fff640 1 --2- 172.21.15.89:0/4188111973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10640a7730 0x7f1064140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:45:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1062ffd640 1 -- 172.21.15.89:0/4188111973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10640a7350 msgr2=0x7f106413e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1062ffd640 1 --2- 172.21.15.89:0/4188111973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10640a7350 0x7f106413e860 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f104c0098a0 tx=0x7f104c01cea0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=56b63f00d2ed18d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 1 -- 172.21.15.89:0/4188111973 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f104c0b0070 con 0x7f10640a7350 2024-08-19T10:45:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:45:14.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 10 monclient: dump: 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.113 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 1 -- 172.21.15.89:0/4188111973 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f104c0aa810 con 0x7f10640a7350 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 5 monclient: authenticate success, global_id 44320 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 1 -- 172.21.15.89:0/4188111973 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f104c0a9300 con 0x7f10640a7350 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 4 set_mon_vals no callback set 2024-08-19T10:45:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10627fc640 10 monclient: dump: 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient: shutdown 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 -- 172.21.15.89:0/4188111973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10640a7350 msgr2=0x7f106413e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 --2- 172.21.15.89:0/4188111973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10640a7350 0x7f106413e860 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f104c0098a0 tx=0x7f104c01cea0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 -- 172.21.15.89:0/4188111973 shutdown_connections 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 --2- 172.21.15.89:0/4188111973 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10640a7730 0x7f1064140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 --2- 172.21.15.89:0/4188111973 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f10640a7350 0x7f106413e860 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 -- 172.21.15.89:0/4188111973 >> 172.21.15.89:0/4188111973 conn(0x7f1064074b40 msgr2=0x7f10640abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 -- 172.21.15.89:0/4188111973 shutdown_connections 2024-08-19T10:45:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 -- 172.21.15.89:0/4188111973 wait complete. 2024-08-19T10:45:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient: build_initial_monmap 2024-08-19T10:45:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient: monmap: 2024-08-19T10:45:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:45:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:45:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 Processor -- start 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 -- start start 2024-08-19T10:45:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient: init 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient: _add_conns ranks=[6,2,0,8,5,4,3,7,1] 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10640a6860 0x7f10641287b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient(hunting): picked mon.noname-d con 0x7f10640a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10640a7350 0x7f10641d1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient(hunting): picked mon.noname-b con 0x7f10640a7350 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10640a7730 0x7f1064126160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient(hunting): picked mon.noname-a con 0x7f10640a7730 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1064143ea0 con 0x7f10640a7730 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1064143d20 con 0x7f10640a7350 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1064144020 con 0x7f10640a6860 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1069c23640 10 monclient(hunting): authenticate will time out at 1363.930925s 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1062ffd640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10640a7350 msgr2=0x7f10641d1520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1062ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10640a7350 0x7f10641d1520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10637fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10640a6860 0x7f10641287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10640a7730 0x7f1064126160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10640a7730 0x7f1064126160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42928/0 (socket says 172.21.15.89:42928) 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 1 -- 172.21.15.89:0/1412735315 learned_addr learned my addr 172.21.15.89:0/1412735315 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 10 monclient(hunting): get_auth_request con 0x7f10640a7730 auth_method 0 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10637fe640 10 monclient(hunting): get_auth_request con 0x7f10640a6860 auth_method 0 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10637fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10637fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 10 monclient(hunting): handle_auth_done global_id 6399 payload 824 2024-08-19T10:45:14.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 1 -- 172.21.15.89:0/1412735315 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10640a6860 msgr2=0x7f10641287b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10640a6860 0x7f10641287b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 1 -- 172.21.15.89:0/1412735315 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10640a7350 msgr2=0x7f10641d1520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f10640a7350 0x7f10641d1520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 1 monclient: found mon.noname-a 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f1063fff640 1 -- 172.21.15.89:0/1412735315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1064129140 con 0x7f10640a7730 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.109+0000 7f10637fe640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f10640a6860 0x7f10641287b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1063fff640 1 -- 172.21.15.89:0/1412735315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10640a7730 msgr2=0x7f1064126160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1063fff640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10640a7730 0x7f1064126160 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f105400d800 tx=0x7f105400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f10540073a0 con 0x7f10640a7730 2024-08-19T10:45:14.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 10 monclient: dump: 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.127 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 10 monclient: _finish_auth 0 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1069c23640 5 monclient: authenticate success, global_id 6399 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1069c23640 10 monclient: _renew_subs 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1069c23640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1069c23640 1 -- 172.21.15.89:0/1412735315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1064129710 con 0x7f10640a7730 2024-08-19T10:45:14.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1054019070 con 0x7f10640a7730 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f105400bd40 con 0x7f10640a7730 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 10 monclient: dump: 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.129 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1061ffb640 4 set_mon_vals no callback set 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1069c23640 10 monclient: _renew_subs 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1069c23640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1069c23640 1 -- 172.21.15.89:0/1412735315 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1064129fb0 con 0x7f10640a7730 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1069c23640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1069c23640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10641d1a60 0x7f10641266a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1069c23640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1069c23640 1 -- 172.21.15.89:0/1412735315 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f10640a92f0 con 0x7f10641d1a60 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f105400b8d0 con 0x7f10640a7730 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f10637fe640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10641d1a60 0x7f10641266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f10637fe640 10 monclient: get_auth_request con 0x7f10641d1a60 auth_method 0 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f10637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f10637fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f10637fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f104403be00 0x7f104403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f10637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f10637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(232..232 src has 1..232) ==== 14144+0+0 (secure 0 0 0) 0x7f105400f040 con 0x7f10640a7730 2024-08-19T10:45:14.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f10637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1062ffd640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f104403be00 0x7f104403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f10637fe640 10 monclient(hunting): handle_auth_done global_id 34734 payload 824 2024-08-19T10:45:14.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f10637fe640 1 -- 172.21.15.89:0/1412735315 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10640a6c40 con 0x7f10641d1a60 2024-08-19T10:45:14.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.113+0000 7f1062ffd640 10 monclient: get_auth_request con 0x7f104403be00 auth_method 0 2024-08-19T10:45:14.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f10637fe640 1 -- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10641d1a60 msgr2=0x7f10641266a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f10637fe640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10641d1a60 0x7f10641266a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f105c0048e0 tx=0x7f105c00d520 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f105c004de0 con 0x7f10641d1a60 2024-08-19T10:45:14.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f105bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f105bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:14.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f105bfff640 10 monclient: dump: 2024-08-19T10:45:14.134 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.134 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.135 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f1062ffd640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f104403be00 0x7f104403e2c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f106406cff0 tx=0x7f104c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f105c004970 con 0x7f10641d1a60 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f105bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f105bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10641d1a60 msgr2=0x7f10641266a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f105bfff640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10641d1a60 0x7f10641266a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f105c0048e0 tx=0x7f105c00d520 comp rx=0 tx=0).stop 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10641d1a60 msgr2=0x7f10641266a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.117+0000 7f105bfff640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10641d1a60 0x7f10641266a0 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f105c0048e0 tx=0x7f105c00d520 comp rx=0 tx=0).stop 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f1069c23640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f1069c23640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1064126be0 0x7f10641224b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f1069c23640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f1069c23640 1 -- 172.21.15.89:0/1412735315 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1064126ff0 con 0x7f1064126be0 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f10637fe640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1064126be0 0x7f10641224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f10637fe640 10 monclient: get_auth_request con 0x7f1064126be0 auth_method 0 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f10637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f10637fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f10637fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f10637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f10637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f10637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f10637fe640 10 monclient(hunting): handle_auth_done global_id 34743 payload 824 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.125+0000 7f10637fe640 1 -- 172.21.15.89:0/1412735315 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1064064590 con 0x7f1064126be0 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f10637fe640 1 -- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1064126be0 msgr2=0x7f10641224b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f10637fe640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1064126be0 0x7f10641224b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f105c0048e0 tx=0x7f105c003af0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f105c01e030 con 0x7f1064126be0 2024-08-19T10:45:14.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f105bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f105bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f105bfff640 10 monclient: dump: 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.138 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9256 (secure 0 0 0) 0x7f105c01e030 con 0x7f1064126be0 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f105bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f105bfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1064126be0 msgr2=0x7f10641224b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f105bfff640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1064126be0 0x7f10641224b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f105c0048e0 tx=0x7f105c003af0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f105bfff640 1 -- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1064126be0 msgr2=0x7f10641224b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:14.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f105bfff640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1064126be0 0x7f10641224b0 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f105c0048e0 tx=0x7f105c003af0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 80, 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 159173, 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:45:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:45:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:45:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:45:14.148 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:45:14.149 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.151 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:45:14.152 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.153 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.154 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:45:14.155 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.156 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.157 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:45:14.159 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.161 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.162 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:14.163 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:14.163 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:45:14.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.163 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.163 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:45:14.163 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:45:14.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1069c23640 1 -- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f104403be00 msgr2=0x7f104403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1069c23640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f104403be00 0x7f104403e2c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f106406cff0 tx=0x7f104c008040 comp rx=0 tx=0).stop 2024-08-19T10:45:14.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1069c23640 10 monclient: shutdown 2024-08-19T10:45:14.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1069c23640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:14.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1069c23640 1 -- 172.21.15.89:0/1412735315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10640a7730 msgr2=0x7f1064126160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1069c23640 1 --2- 172.21.15.89:0/1412735315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10640a7730 0x7f1064126160 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f105400d800 tx=0x7f105400dcc0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1063fff640 1 -- 172.21.15.89:0/1412735315 reap_dead start 2024-08-19T10:45:14.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1063fff640 1 -- 172.21.15.89:0/1412735315 reap_dead start 2024-08-19T10:45:14.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1069c23640 1 -- 172.21.15.89:0/1412735315 shutdown_connections 2024-08-19T10:45:14.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1069c23640 1 -- 172.21.15.89:0/1412735315 >> 172.21.15.89:0/1412735315 conn(0x7f1064074b40 msgr2=0x7f106409f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:14.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1069c23640 1 -- 172.21.15.89:0/1412735315 shutdown_connections 2024-08-19T10:45:14.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.129+0000 7f1069c23640 1 -- 172.21.15.89:0/1412735315 wait complete. 2024-08-19T10:45:14.166 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:45:14.166 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-08-19T10:45:14.166 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:45:14.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.261+0000 7f632affd640 1 -- 172.21.15.190:0/2124452460 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h5"}]=0 removed item id -31 name 'h5' from crush map v233) ==== 118+0+0 (secure 0 0 0) 0x7f6324010070 con 0x7f632c1377c0 2024-08-19T10:45:14.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.261+0000 7f632affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h5"}] 2024-08-19T10:45:14.264 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.261+0000 7f632affd640 10 monclient: _finish_command 2 = system:0 removed item id -31 name 'h5' from crush map 2024-08-19T10:45:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.273+0000 7f48d6f50640 10 monclient: get_monmap_and_config 2024-08-19T10:45:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.273+0000 7f48d6f50640 10 monclient: build_initial_monmap 2024-08-19T10:45:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.273+0000 7f48d6f50640 10 monclient: monmap: 2024-08-19T10:45:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:45:14.280053+0000 2024-08-19T10:45:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:45:14.280053+0000 2024-08-19T10:45:14.278 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:14.279 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.273+0000 7f48d6f50640 1 Processor -- start 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.273+0000 7f48d6f50640 1 -- start start 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.273+0000 7f48d6f50640 10 monclient: init 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 10 monclient: _add_conns ranks=[9,1,7,8,6,3,2,0,10,4,5] 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48d00a45a0 0x7f48d00a4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 10 monclient(hunting): picked mon.noname-a con 0x7f48d00a45a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f48d006d520 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48d013c820 0x7f48d013ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 10 monclient(hunting): picked mon.noname-i con 0x7f48d013c820 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d006c9e0 con 0x7f48d006d520 2024-08-19T10:45:14.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f48d006d2e0 con 0x7f48d00a45a0 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f48d00738b0 con 0x7f48d013c820 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d6f50640 10 monclient(hunting): authenticate will time out at 1364.096552s 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48d00a45a0 0x7f48d00a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48d00a45a0 0x7f48d00a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42938/0 (socket says 172.21.15.89:42938) 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 1 -- 172.21.15.89:0/1774199414 learned_addr learned my addr 172.21.15.89:0/1774199414 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 10 monclient(hunting): get_auth_request con 0x7f48d00a45a0 auth_method 0 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d54ed640 1 --2- 172.21.15.89:0/1774199414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48d013c820 0x7f48d013ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d54ed640 10 monclient(hunting): get_auth_request con 0x7f48d013c820 auth_method 0 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d54ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d54ed640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d54ed640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 10 monclient(hunting): handle_auth_done global_id 6408 payload 274 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 1 -- 172.21.15.89:0/1774199414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48d013c820 msgr2=0x7f48d013ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 1 --2- 172.21.15.89:0/1774199414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48d013c820 0x7f48d013ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 1 -- 172.21.15.89:0/1774199414 >> v1:172.21.15.190:6789/0 conn(0x7f48d006d520 legacy=0x7f48d00a4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 10 monclient: _finish_hunting 0 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 1 monclient: found mon.noname-a 2024-08-19T10:45:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:45:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.277+0000 7f48d4cec640 1 -- 172.21.15.89:0/1774199414 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d00a60c0 con 0x7f48d00a45a0 2024-08-19T10:45:14.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d4cec640 1 -- 172.21.15.89:0/1774199414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48d00a45a0 msgr2=0x7f48d00a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d4cec640 1 --2- 172.21.15.89:0/1774199414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48d00a45a0 0x7f48d00a4980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009870 tx=0x7f48bc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8250aa42f2b11b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 1 -- 172.21.15.89:0/1774199414 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48bc0b0070 con 0x7f48d00a45a0 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 10 monclient: dump: 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 1 -- 172.21.15.89:0/1774199414 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48bc0aa810 con 0x7f48d00a45a0 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 5 monclient: authenticate success, global_id 6408 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 1 -- 172.21.15.89:0/1774199414 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48bc0a9300 con 0x7f48d00a45a0 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 4 set_mon_vals no callback set 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cf7fe640 10 monclient: dump: 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient: shutdown 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 -- 172.21.15.89:0/1774199414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48d00a45a0 msgr2=0x7f48d00a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 --2- 172.21.15.89:0/1774199414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48d00a45a0 0x7f48d00a4980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009870 tx=0x7f48bc01c420 comp rx=0 tx=0).stop 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 -- 172.21.15.89:0/1774199414 shutdown_connections 2024-08-19T10:45:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 --2- 172.21.15.89:0/1774199414 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f48d013c820 0x7f48d013ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 --2- 172.21.15.89:0/1774199414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48d00a45a0 0x7f48d00a4980 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 -- 172.21.15.89:0/1774199414 >> 172.21.15.89:0/1774199414 conn(0x7f48d00a9190 msgr2=0x7f48d00a5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 -- 172.21.15.89:0/1774199414 shutdown_connections 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 -- 172.21.15.89:0/1774199414 wait complete. 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient: build_initial_monmap 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient: monmap: 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 Processor -- start 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 -- start start 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient: init 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient: _add_conns ranks=[3,1,8,7,6,4,0,2,5] 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d006d520 0x7f48d0123920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient(hunting): picked mon.noname-g con 0x7f48d006d520 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d00a45a0 0x7f48d01cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient(hunting): picked mon.noname-f con 0x7f48d00a45a0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48d013c820 0x7f48d0121310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient(hunting): picked mon.noname-e con 0x7f48d013c820 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48d0141d00 con 0x7f48d00a45a0 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f48d0141b80 con 0x7f48d006d520 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f48d0141e80 con 0x7f48d013c820 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d6f50640 10 monclient(hunting): authenticate will time out at 1364.103219s 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48d013c820 0x7f48d0121310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48d013c820 0x7f48d0121310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:45282/0 (socket says 172.21.15.89:45282) 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 1 -- 172.21.15.89:0/3311528457 learned_addr learned my addr 172.21.15.89:0/3311528457 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 10 monclient(hunting): get_auth_request con 0x7f48d013c820 auth_method 0 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d4cec640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d006d520 0x7f48d0123920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cffff640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d00a45a0 0x7f48d01cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d4cec640 10 monclient(hunting): get_auth_request con 0x7f48d006d520 auth_method 0 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d4cec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d4cec640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d4cec640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cffff640 10 monclient(hunting): get_auth_request con 0x7f48d00a45a0 auth_method 0 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48cffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 10 monclient(hunting): handle_auth_done global_id 6272 payload 824 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 1 -- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d006d520 msgr2=0x7f48d0123920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d006d520 0x7f48d0123920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 1 -- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d00a45a0 msgr2=0x7f48d01cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d00a45a0 0x7f48d01cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 10 monclient: _finish_hunting 0 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 1 monclient: found mon.noname-e 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.281+0000 7f48d54ed640 1 -- 172.21.15.89:0/3311528457 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d00a7f50 con 0x7f48d013c820 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d54ed640 1 -- 172.21.15.89:0/3311528457 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48d013c820 msgr2=0x7f48d0121310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d54ed640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48d013c820 0x7f48d0121310 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f48c8007cd0 tx=0x7f48c800e4b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48c8009890 con 0x7f48d013c820 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 10 monclient: dump: 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.300 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 10 monclient: _finish_auth 0 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d6f50640 5 monclient: authenticate success, global_id 6272 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c8019070 con 0x7f48d013c820 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d6f50640 10 monclient: _renew_subs 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48c8014d00 con 0x7f48d013c820 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d6f50640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d6f50640 1 -- 172.21.15.89:0/3311528457 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d0124e40 con 0x7f48d013c820 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:14.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48ceffd640 4 set_mon_vals no callback set 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 10 monclient: dump: 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.302 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.303 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d6f50640 10 monclient: _renew_subs 2024-08-19T10:45:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d6f50640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d6f50640 1 -- 172.21.15.89:0/3311528457 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d00a84a0 con 0x7f48d013c820 2024-08-19T10:45:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d6f50640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d6f50640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002d1d0 0x7f48d002d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d6f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d6f50640 1 -- 172.21.15.89:0/3311528457 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f48d01c6d10 con 0x7f48d002d1d0 2024-08-19T10:45:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d4cec640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002d1d0 0x7f48d002d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d4cec640 10 monclient: get_auth_request con 0x7f48d002d1d0 auth_method 0 2024-08-19T10:45:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d4cec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d4cec640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d4cec640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f48c8014420 con 0x7f48d013c820 2024-08-19T10:45:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d4cec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d4cec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d4cec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f48b403be00 0x7f48b403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(233..233 src has 1..233) ==== 13934+0+0 (secure 0 0 0) 0x7f48c8051640 con 0x7f48d013c820 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d4cec640 10 monclient(hunting): handle_auth_done global_id 6314 payload 824 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48d4cec640 1 -- 172.21.15.89:0/3311528457 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48d00a4980 con 0x7f48d002d1d0 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cffff640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f48b403be00 0x7f48b403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.285+0000 7f48cffff640 10 monclient: get_auth_request con 0x7f48b403be00 auth_method 0 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48d4cec640 1 -- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002d1d0 msgr2=0x7f48d002d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48d4cec640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002d1d0 0x7f48d002d5b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0ad000 tx=0x7f48bc0191c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48bc0b9020 con 0x7f48d002d1d0 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cd7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cd7fa640 10 monclient: dump: 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.309 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cffff640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f48b403be00 0x7f48b403e2c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f48c4004e30 tx=0x7f48c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f48bc0b1040 con 0x7f48d002d1d0 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cd7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cd7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002d1d0 msgr2=0x7f48d002d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cd7fa640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002d1d0 0x7f48d002d5b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0ad000 tx=0x7f48bc0191c0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002d1d0 msgr2=0x7f48d002d5b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.289+0000 7f48cd7fa640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002d1d0 0x7f48d002d5b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d6f50640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d6f50640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002daf0 0x7f48d002df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d6f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d6f50640 1 -- 172.21.15.89:0/3311528457 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f48d0064590 con 0x7f48d002daf0 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002daf0 0x7f48d002df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 10 monclient: get_auth_request con 0x7f48d002daf0 auth_method 0 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 10 monclient(hunting): handle_auth_done global_id 6323 payload 824 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 1 -- 172.21.15.89:0/3311528457 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f48d01242b0 con 0x7f48d002daf0 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 1 -- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002daf0 msgr2=0x7f48d002df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48d4cec640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002daf0 0x7f48d002df00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009870 tx=0x7f48bc0a8000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48bc0b9040 con 0x7f48d002daf0 2024-08-19T10:45:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48cd7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48cd7fa640 10 monclient: dump: 2024-08-19T10:45:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9590 (secure 0 0 0) 0x7f48bc0be630 con 0x7f48d002daf0 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48cd7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48cd7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002daf0 msgr2=0x7f48d002df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48cd7fa640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002daf0 0x7f48d002df00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009870 tx=0x7f48bc0a8000 comp rx=0 tx=0).stop 2024-08-19T10:45:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48cd7fa640 1 -- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002daf0 msgr2=0x7f48d002df00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.297+0000 7f48cd7fa640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f48d002daf0 0x7f48d002df00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.314 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:45:14.314 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:45:14.314 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:45:14.314 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:45:14.314 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 80, 2024-08-19T10:45:14.314 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 444071, 2024-08-19T10:45:14.314 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:45:14.315 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.316 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:14.317 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:45:14.318 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:45:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:45:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:45:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:45:14.339 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:45:14.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d6f50640 1 -- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f48b403be00 msgr2=0x7f48b403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d6f50640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f48b403be00 0x7f48b403e2c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f48c4004e30 tx=0x7f48c4009040 comp rx=0 tx=0).stop 2024-08-19T10:45:14.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d6f50640 10 monclient: shutdown 2024-08-19T10:45:14.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d6f50640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:14.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d6f50640 1 -- 172.21.15.89:0/3311528457 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48d013c820 msgr2=0x7f48d0121310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d6f50640 1 --2- 172.21.15.89:0/3311528457 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48d013c820 0x7f48d0121310 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f48c8007cd0 tx=0x7f48c800e4b0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d54ed640 1 -- 172.21.15.89:0/3311528457 reap_dead start 2024-08-19T10:45:14.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d54ed640 1 -- 172.21.15.89:0/3311528457 reap_dead start 2024-08-19T10:45:14.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d6f50640 1 -- 172.21.15.89:0/3311528457 shutdown_connections 2024-08-19T10:45:14.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d6f50640 1 -- 172.21.15.89:0/3311528457 >> 172.21.15.89:0/3311528457 conn(0x7f48d00a9190 msgr2=0x7f48d013daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:14.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d6f50640 1 -- 172.21.15.89:0/3311528457 shutdown_connections 2024-08-19T10:45:14.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.301+0000 7f48d6f50640 1 -- 172.21.15.89:0/3311528457 wait complete. 2024-08-19T10:45:14.341 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:45:14.341 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-08-19T10:45:14.342 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:45:14.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.373+0000 7f6334497640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h5"}] 2024-08-19T10:45:14.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.373+0000 7f6334497640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h5"}] 2024-08-19T10:45:14.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.373+0000 7f6334497640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:14.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.373+0000 7f6334497640 1 -- 172.21.15.190:0/2124452460 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h5"} v 0) -- 0x7f632c070af0 con 0x7f632c1377c0 2024-08-19T10:45:14.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f632affd640 1 -- 172.21.15.190:0/2124452460 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h5"}]=0 device 'h5' does not appear in the crush map v233) ==== 118+0+0 (secure 0 0 0) 0x7f6324004210 con 0x7f632c1377c0 2024-08-19T10:45:14.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f632affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h5"}] 2024-08-19T10:45:14.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f632affd640 10 monclient: _finish_command 3 = system:0 device 'h5' does not appear in the crush map 2024-08-19T10:45:14.377 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h5' does not appear in the crush map 2024-08-19T10:45:14.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 -- 172.21.15.190:0/2124452460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f631803be00 msgr2=0x7f631803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f631803be00 0x7f631803e2c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f632c0a5480 tx=0x7f631c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:45:14.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 10 monclient: shutdown 2024-08-19T10:45:14.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:14.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 -- 172.21.15.190:0/2124452460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 msgr2=0x7f632c13ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c13ed50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6324007f20 tx=0x7f632400d560 comp rx=0 tx=0).stop 2024-08-19T10:45:14.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 -- 172.21.15.190:0/2124452460 shutdown_connections 2024-08-19T10:45:14.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f631803be00 0x7f631803e2c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c1377c0 0x7f632c13ed50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f632c09ca60 0x7f632c13e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 --2- 172.21.15.190:0/2124452460 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f632c0703c0 0x7f632c1cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 -- 172.21.15.190:0/2124452460 >> 172.21.15.190:0/2124452460 conn(0x7f632c0a4700 msgr2=0x7f632c0a62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:14.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 -- 172.21.15.190:0/2124452460 shutdown_connections 2024-08-19T10:45:14.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.377+0000 7f6334497640 1 -- 172.21.15.190:0/2124452460 wait complete. 2024-08-19T10:45:14.395 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:14.395 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h6 2024-08-19T10:45:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient: get_monmap_and_config 2024-08-19T10:45:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient: build_initial_monmap 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient: monmap: 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:45:14.449390+0000 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:45:14.449390+0000 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:14.448 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:14.449 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:14.449 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 1 Processor -- start 2024-08-19T10:45:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 1 -- start start 2024-08-19T10:45:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient: init 2024-08-19T10:45:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:14.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient: _add_conns ranks=[0,7,2,1,5,3,6,8,9,4,10] 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f414c0a5270 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a6f00 0x7f414c13ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient(hunting): picked mon.noname-i con 0x7f414c0a6f00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f414c0a72e0 0x7f414c140f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient(hunting): picked mon.noname-b con 0x7f414c0a72e0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f414c0a8710 con 0x7f414c0a5270 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f414c0a5a40 con 0x7f414c0a72e0 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f414c0a81d0 con 0x7f414c0a6f00 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f41527e6640 10 monclient(hunting): authenticate will time out at 1364.265895s 2024-08-19T10:45:14.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f4150d83640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f414c0a72e0 msgr2=0x7f414c140f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f4150d83640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f414c0a72e0 0x7f414c140f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414bfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f414c0a5270 0x7f414c0a67f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:39814/0 (socket says 172.21.15.89:39814) 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414bfff640 1 -- 172.21.15.89:0/3558503457 learned_addr learned my addr 172.21.15.89:0/3558503457 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 1 --2- 172.21.15.89:0/3558503457 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a6f00 0x7f414c13ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 10 monclient(hunting): get_auth_request con 0x7f414c0a6f00 auth_method 0 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 10 monclient(hunting): handle_auth_done global_id 15667 payload 274 2024-08-19T10:45:14.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 1 -- 172.21.15.89:0/3558503457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f414c0a72e0 msgr2=0x7f414c140f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:14.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 1 --2- 172.21.15.89:0/3558503457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f414c0a72e0 0x7f414c140f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 1 -- 172.21.15.89:0/3558503457 >> v1:172.21.15.89:6789/0 conn(0x7f414c0a5270 legacy=0x7f414c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:14.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:14.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 1 monclient: found mon.noname-i 2024-08-19T10:45:14.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:14.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:14.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.445+0000 7f414b7fe640 1 -- 172.21.15.89:0/3558503457 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c0a4790 con 0x7f414c0a6f00 2024-08-19T10:45:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414b7fe640 1 -- 172.21.15.89:0/3558503457 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a6f00 msgr2=0x7f414c13ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414b7fe640 1 --2- 172.21.15.89:0/3558503457 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a6f00 0x7f414c13ea00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f413c0097f0 tx=0x7f413c01c1b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=8e559aef7309b546 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 1 -- 172.21.15.89:0/3558503457 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f413c0b0070 con 0x7f414c0a6f00 2024-08-19T10:45:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 10 monclient: dump: 2024-08-19T10:45:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 10 monclient: _finish_auth 0 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 5 monclient: authenticate success, global_id 15667 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 1 -- 172.21.15.89:0/3558503457 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f413c0aa3a0 con 0x7f414c0a6f00 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 4 set_mon_vals no callback set 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 1 -- 172.21.15.89:0/3558503457 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f413c0aa9c0 con 0x7f414c0a6f00 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f414affd640 10 monclient: dump: 2024-08-19T10:45:14.460 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.461 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 10 monclient: shutdown 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 1 -- 172.21.15.89:0/3558503457 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a6f00 msgr2=0x7f414c13ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 1 --2- 172.21.15.89:0/3558503457 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a6f00 0x7f414c13ea00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f413c0097f0 tx=0x7f413c01c1b0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 1 -- 172.21.15.89:0/3558503457 shutdown_connections 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 1 --2- 172.21.15.89:0/3558503457 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f414c0a72e0 0x7f414c140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 1 --2- 172.21.15.89:0/3558503457 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a6f00 0x7f414c13ea00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 1 -- 172.21.15.89:0/3558503457 >> 172.21.15.89:0/3558503457 conn(0x7f414c069a80 msgr2=0x7f414c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 1 -- 172.21.15.89:0/3558503457 shutdown_connections 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.449+0000 7f41527e6640 1 -- 172.21.15.89:0/3558503457 wait complete. 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient: build_initial_monmap 2024-08-19T10:45:14.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient: monmap: 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:14.463 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:14.464 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:14.464 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 1 Processor -- start 2024-08-19T10:45:14.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 1 -- start start 2024-08-19T10:45:14.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient: init 2024-08-19T10:45:14.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:14.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient: _add_conns ranks=[7,0,5,6,1,4,2,8,3] 2024-08-19T10:45:14.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a5270 0x7f414c1368d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient(hunting): picked mon.noname-i con 0x7f414c0a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:14.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f414c0a6f00 0x7f414c136e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient(hunting): picked mon.noname-a con 0x7f414c0a6f00 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:14.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f414c0a72e0 0x7f414c13b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient(hunting): picked mon.noname-h con 0x7f414c0a72e0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:14.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f414c143e30 con 0x7f414c0a6f00 2024-08-19T10:45:14.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f414c143cb0 con 0x7f414c0a72e0 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f414c143fb0 con 0x7f414c0a5270 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f41527e6640 10 monclient(hunting): authenticate will time out at 1364.273305s 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f414c0a6f00 0x7f414c136e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f414c0a6f00 0x7f414c136e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42946/0 (socket says 172.21.15.89:42946) 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 1 -- 172.21.15.89:0/1093589284 learned_addr learned my addr 172.21.15.89:0/1093589284 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 10 monclient(hunting): get_auth_request con 0x7f414c0a6f00 auth_method 0 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414bfff640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a5270 0x7f414c1368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414bfff640 10 monclient(hunting): get_auth_request con 0x7f414c0a5270 auth_method 0 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f4150d83640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f414c0a72e0 0x7f414c13b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 10 monclient(hunting): handle_auth_done global_id 6417 payload 824 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 1 -- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a5270 msgr2=0x7f414c1368d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a5270 0x7f414c1368d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 1 -- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f414c0a72e0 msgr2=0x7f414c13b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f414c0a72e0 0x7f414c13b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 1 monclient: found mon.noname-a 2024-08-19T10:45:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:45:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 1 -- 172.21.15.89:0/1093589284 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c13f100 con 0x7f414c0a6f00 2024-08-19T10:45:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f4150d83640 10 monclient: get_auth_request con 0x7f414c0a72e0 auth_method 0 2024-08-19T10:45:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414bfff640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c0a5270 0x7f414c1368d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:45:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f4150d83640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f414c0a72e0 0x7f414c13b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 1 -- 172.21.15.89:0/1093589284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f414c0a6f00 msgr2=0x7f414c136e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f414b7fe640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f414c0a6f00 0x7f414c136e10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f413c009710 tx=0x7f413c018f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.453+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:45:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f413c0b0040 con 0x7f414c0a6f00 2024-08-19T10:45:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:45:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 10 monclient: dump: 2024-08-19T10:45:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.474 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 10 monclient: _finish_auth 0 2024-08-19T10:45:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f413c0aa550 con 0x7f414c0a6f00 2024-08-19T10:45:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:14.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f413c003e60 con 0x7f414c0a6f00 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f41527e6640 5 monclient: authenticate success, global_id 6417 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 10 monclient: dump: 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.476 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f41527e6640 10 monclient: _renew_subs 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f414a7fc640 4 set_mon_vals no callback set 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f41527e6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f41527e6640 1 -- 172.21.15.89:0/1093589284 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414c13e8a0 con 0x7f414c0a6f00 2024-08-19T10:45:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f41527e6640 10 monclient: _renew_subs 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f41527e6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f41527e6640 1 -- 172.21.15.89:0/1093589284 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414c140bf0 con 0x7f414c0a6f00 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f41527e6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f41527e6640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c13bc10 0x7f414c13bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f413c0039f0 con 0x7f414c0a6f00 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f41527e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f41527e6640 1 -- 172.21.15.89:0/1093589284 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f414c13c530 con 0x7f414c13bc10 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f413803be30 0x7f413803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 13934+0+0 (secure 0 0 0) 0x7f413c0eaaf0 con 0x7f414c0a6f00 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f414bfff640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c13bc10 0x7f414c13bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f414bfff640 10 monclient: get_auth_request con 0x7f414c13bc10 auth_method 0 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f414bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f414bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f414bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f414bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f414bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f414bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4150d83640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f413803be30 0x7f413803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f414bfff640 10 monclient(hunting): handle_auth_done global_id 15685 payload 824 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f414bfff640 1 -- 172.21.15.89:0/1093589284 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f414c0a67f0 con 0x7f414c13bc10 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.457+0000 7f4150d83640 10 monclient: get_auth_request con 0x7f413803be30 auth_method 0 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f414bfff640 1 -- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c13bc10 msgr2=0x7f414c13bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f414bfff640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c13bc10 0x7f414c13bff0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f414000d700 tx=0x7f414000dbc0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4140018020 con 0x7f414c13bc10 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4148ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4148ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4148ff9640 10 monclient: dump: 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4150d83640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f413803be30 0x7f413803e2f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4144003cc0 tx=0x7f4144007320 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4140009940 con 0x7f414c13bc10 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4148ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4148ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c13bc10 msgr2=0x7f414c13bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4148ff9640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c13bc10 0x7f414c13bff0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f414000d700 tx=0x7f414000dbc0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c13bc10 msgr2=0x7f414c13bff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.461+0000 7f4148ff9640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c13bc10 0x7f414c13bff0 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f414000d700 tx=0x7f414000dbc0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f41527e6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:45:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f41527e6640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c1377a0 0x7f414c137bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f41527e6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f41527e6640 1 -- 172.21.15.89:0/1093589284 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f414c064590 con 0x7f414c1377a0 2024-08-19T10:45:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c1377a0 0x7f414c137bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 10 monclient: get_auth_request con 0x7f414c1377a0 auth_method 0 2024-08-19T10:45:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 10 monclient(hunting): handle_auth_done global_id 15694 payload 824 2024-08-19T10:45:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 1 -- 172.21.15.89:0/1093589284 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f414c137350 con 0x7f414c1377a0 2024-08-19T10:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 1 -- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c1377a0 msgr2=0x7f414c137bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f414bfff640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c1377a0 0x7f414c137bb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4140019000 tx=0x7f41400058e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4140018040 con 0x7f414c1377a0 2024-08-19T10:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f4148ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f4148ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.469+0000 7f4148ff9640 10 monclient: dump: 2024-08-19T10:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9256 (secure 0 0 0) 0x7f414001f860 con 0x7f414c1377a0 2024-08-19T10:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f4148ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f4148ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c1377a0 msgr2=0x7f414c137bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f4148ff9640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c1377a0 0x7f414c137bb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4140019000 tx=0x7f41400058e0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f4148ff9640 1 -- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c1377a0 msgr2=0x7f414c137bb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f4148ff9640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f414c1377a0 0x7f414c137bb0 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4140019000 tx=0x7f41400058e0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.486 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 80, 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 292061, 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:45:14.487 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:14.488 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:45:14.490 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:45:14.491 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:45:14.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:45:14.493 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.494 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.494 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.494 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:45:14.494 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:45:14.494 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.494 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.494 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.495 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:45:14.496 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:45:14.499 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:45:14.499 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.499 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.499 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:45:14.500 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:45:14.500 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.500 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.500 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.501 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:45:14.501 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:45:14.501 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:45:14.501 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.502 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:45:14.502 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.502 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.502 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:45:14.502 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:45:14.502 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.502 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.502 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:45:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:45:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:45:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:45:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:45:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:45:14.511 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:45:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f41527e6640 1 -- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f413803be30 msgr2=0x7f413803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f41527e6640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f413803be30 0x7f413803e2f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4144003cc0 tx=0x7f4144007320 comp rx=0 tx=0).stop 2024-08-19T10:45:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f41527e6640 10 monclient: shutdown 2024-08-19T10:45:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f41527e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:14.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f41527e6640 1 -- 172.21.15.89:0/1093589284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f414c0a6f00 msgr2=0x7f414c136e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f41527e6640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f414c0a6f00 0x7f414c136e10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f413c009710 tx=0x7f413c018f50 comp rx=0 tx=0).stop 2024-08-19T10:45:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f4150d83640 1 -- 172.21.15.89:0/1093589284 reap_dead start 2024-08-19T10:45:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f41527e6640 1 -- 172.21.15.89:0/1093589284 shutdown_connections 2024-08-19T10:45:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f41527e6640 1 --2- 172.21.15.89:0/1093589284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f414c0a6f00 0x7f414c136e10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f41527e6640 1 -- 172.21.15.89:0/1093589284 >> 172.21.15.89:0/1093589284 conn(0x7f414c069a80 msgr2=0x7f414c0aa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f41527e6640 1 -- 172.21.15.89:0/1093589284 shutdown_connections 2024-08-19T10:45:14.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:45:14.473+0000 7f41527e6640 1 -- 172.21.15.89:0/1093589284 wait complete. 2024-08-19T10:45:14.513 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:45:14.513 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-08-19T10:45:14.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient: get_monmap_and_config 2024-08-19T10:45:14.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient: build_initial_monmap 2024-08-19T10:45:14.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient: monmap: 2024-08-19T10:45:14.515 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:14.515 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:14.518103+0000 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:14.518103+0000 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:14.516 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:14.517 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:14.517 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:14.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 1 Processor -- start 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 1 -- start start 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient: init 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient: _add_conns ranks=[2,5,4,10,8,3,6,7,1,9,0] 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc0139980 0x7f1dc00a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient(hunting): picked mon.noname-b con 0x7f1dc0139980 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139d60 0x7f1dc00a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient(hunting): picked mon.noname-h con 0x7f1dc0139d60 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc013a140 0x7f1dc00a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient(hunting): picked mon.noname-c con 0x7f1dc013a140 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc00a8620 con 0x7f1dc0139980 2024-08-19T10:45:14.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f1dc00a5b50 con 0x7f1dc013a140 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1dc00a45b0 con 0x7f1dc0139d60 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.513+0000 7f1dc735e640 10 monclient(hunting): authenticate will time out at 1360.087981s 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139d60 0x7f1dc00a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc50fa640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc0139980 msgr2=0x7f1dc00a5610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139d60 0x7f1dc00a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:50114/0 (socket says 172.21.15.190:50114) 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc50fa640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc0139980 0x7f1dc00a5610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 1 -- 172.21.15.190:0/1728368468 learned_addr learned my addr 172.21.15.190:0/1728368468 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 10 monclient(hunting): get_auth_request con 0x7f1dc0139d60 auth_method 0 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.520 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc58fb640 1 --2- 172.21.15.190:0/1728368468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc013a140 0x7f1dc00a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 10 monclient(hunting): handle_auth_done global_id 6332 payload 274 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 1 -- 172.21.15.190:0/1728368468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc013a140 msgr2=0x7f1dc00a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 1 --2- 172.21.15.190:0/1728368468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc013a140 0x7f1dc00a6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 1 -- 172.21.15.190:0/1728368468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc0139980 msgr2=0x7f1dc00a5610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 1 --2- 172.21.15.190:0/1728368468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc0139980 0x7f1dc00a5610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 10 monclient: _finish_hunting 0 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 1 monclient: found mon.noname-h 2024-08-19T10:45:14.521 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:14.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:14.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc48f9640 1 -- 172.21.15.190:0/1728368468 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc00a4890 con 0x7f1dc0139d60 2024-08-19T10:45:14.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc58fb640 10 monclient: get_auth_request con 0x7f1dc013a140 auth_method 0 2024-08-19T10:45:14.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.517+0000 7f1dc58fb640 1 --2- 172.21.15.190:0/1728368468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc013a140 0x7f1dc00a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:14.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc48f9640 1 -- 172.21.15.190:0/1728368468 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139d60 msgr2=0x7f1dc00a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc48f9640 1 --2- 172.21.15.190:0/1728368468 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139d60 0x7f1dc00a67e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc0098a0 tx=0x7f1dbc01c410 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d637cce4f9fdd56b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.526 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 1 -- 172.21.15.190:0/1728368468 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1dbc0b0070 con 0x7f1dc0139d60 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 10 monclient: dump: 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.527 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.528 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.528 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.528 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.528 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.528 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.528 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:14.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 10 monclient: _finish_auth 0 2024-08-19T10:45:14.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:14.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:14.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 5 monclient: authenticate success, global_id 6332 2024-08-19T10:45:14.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 1 -- 172.21.15.190:0/1728368468 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dbc0aa810 con 0x7f1dc0139d60 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 4 set_mon_vals no callback set 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 1 -- 172.21.15.190:0/1728368468 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1dbc0a93e0 con 0x7f1dc0139d60 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1db7fff640 10 monclient: dump: 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.529 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:14.530 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient: shutdown 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 -- 172.21.15.190:0/1728368468 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139d60 msgr2=0x7f1dc00a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 --2- 172.21.15.190:0/1728368468 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139d60 0x7f1dc00a67e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc0098a0 tx=0x7f1dbc01c410 comp rx=0 tx=0).stop 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 -- 172.21.15.190:0/1728368468 shutdown_connections 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 --2- 172.21.15.190:0/1728368468 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc013a140 0x7f1dc00a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 --2- 172.21.15.190:0/1728368468 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139d60 0x7f1dc00a67e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 --2- 172.21.15.190:0/1728368468 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1dc0139980 0x7f1dc00a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 -- 172.21.15.190:0/1728368468 >> 172.21.15.190:0/1728368468 conn(0x7f1dc00abde0 msgr2=0x7f1dc00ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 -- 172.21.15.190:0/1728368468 shutdown_connections 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 -- 172.21.15.190:0/1728368468 wait complete. 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient: build_initial_monmap 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient: monmap: 2024-08-19T10:45:14.531 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:14.532 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 Processor -- start 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 -- start start 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient: init 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient: _add_conns ranks=[5,4,8,0,7,1,2,3,6] 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139980 0x7f1dc01348b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient(hunting): picked mon.noname-h con 0x7f1dc0139980 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc013a140 0x7f1dc0134df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient(hunting): picked mon.noname-c con 0x7f1dc013a140 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc013a5b0 0x7f1dc013aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient(hunting): picked mon.noname-e con 0x7f1dc013a5b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:14.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f1dc0143190 con 0x7f1dc013a140 2024-08-19T10:45:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1dc0143010 con 0x7f1dc0139980 2024-08-19T10:45:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f1dc0143310 con 0x7f1dc013a5b0 2024-08-19T10:45:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc735e640 10 monclient(hunting): authenticate will time out at 1360.095404s 2024-08-19T10:45:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139980 0x7f1dc01348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139980 0x7f1dc01348b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:50128/0 (socket says 172.21.15.190:50128) 2024-08-19T10:45:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 1 -- 172.21.15.190:0/1849259863 learned_addr learned my addr 172.21.15.190:0/1849259863 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:14.534 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 10 monclient(hunting): get_auth_request con 0x7f1dc0139980 auth_method 0 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc48f9640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc013a140 0x7f1dc0134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc58fb640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc013a5b0 0x7f1dc013aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc48f9640 10 monclient(hunting): get_auth_request con 0x7f1dc013a140 auth_method 0 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc48f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc48f9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc48f9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.535 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc58fb640 10 monclient(hunting): get_auth_request con 0x7f1dc013a5b0 auth_method 0 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc58fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc58fb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc58fb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc48f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc48f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc48f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc58fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc58fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc58fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 10 monclient(hunting): handle_auth_done global_id 6341 payload 824 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 1 -- 172.21.15.190:0/1849259863 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc013a5b0 msgr2=0x7f1dc013aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc013a5b0 0x7f1dc013aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.536 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 1 -- 172.21.15.190:0/1849259863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc013a140 msgr2=0x7f1dc0134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:14.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc013a140 0x7f1dc0134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:14.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 10 monclient: _finish_hunting 0 2024-08-19T10:45:14.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 1 monclient: found mon.noname-h 2024-08-19T10:45:14.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:14.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:14.537 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.521+0000 7f1dc50fa640 1 -- 172.21.15.190:0/1849259863 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc013bd60 con 0x7f1dc0139980 2024-08-19T10:45:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc50fa640 1 -- 172.21.15.190:0/1849259863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139980 msgr2=0x7f1dc01348b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc50fa640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139980 0x7f1dc01348b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1dc013bf30 tx=0x7f1db000b9c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 1 -- 172.21.15.190:0/1849259863 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1db0003de0 con 0x7f1dc0139980 2024-08-19T10:45:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 10 monclient: dump: 2024-08-19T10:45:14.539 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.540 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 1 -- 172.21.15.190:0/1849259863 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db0015070 con 0x7f1dc0139980 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 1 -- 172.21.15.190:0/1849259863 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1db0010420 con 0x7f1dc0139980 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db5ffb640 10 monclient: dump: 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.541 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:14.542 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1db77fe640 4 set_mon_vals no callback set 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc735e640 5 monclient: authenticate success, global_id 6341 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc735e640 10 monclient: _renew_subs 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc735e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc735e640 1 -- 172.21.15.190:0/1849259863 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc00a2750 con 0x7f1dc0139980 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc735e640 10 monclient: _renew_subs 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc735e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc735e640 1 -- 172.21.15.190:0/1849259863 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc013c8a0 con 0x7f1dc0139980 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc735e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc735e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc735e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.525+0000 7f1dc735e640 1 -- 172.21.15.190:0/1849259863 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc00a5610 con 0x7f1dc0139980 2024-08-19T10:45:14.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.529+0000 7f1db5ffb640 1 -- 172.21.15.190:0/1849259863 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1db0005070 con 0x7f1dc0139980 2024-08-19T10:45:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.529+0000 7f1db5ffb640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1da403be00 0x7f1da403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.529+0000 7f1db5ffb640 1 -- 172.21.15.190:0/1849259863 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(233..233 src has 1..233) ==== 13934+0+0 (secure 0 0 0) 0x7f1db000d040 con 0x7f1dc0139980 2024-08-19T10:45:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.529+0000 7f1dc48f9640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1da403be00 0x7f1da403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.529+0000 7f1dc48f9640 10 monclient: get_auth_request con 0x7f1da403be00 auth_method 0 2024-08-19T10:45:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.529+0000 7f1dc48f9640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1da403be00 0x7f1da403e2c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009870 tx=0x7f1dbc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.533+0000 7f1db5ffb640 1 -- 172.21.15.190:0/1849259863 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1db004d130 con 0x7f1dc0139980 2024-08-19T10:45:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.533+0000 7f1db5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:14.544 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.533+0000 7f1db5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:14.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.709+0000 7f1dc735e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h6"}] 2024-08-19T10:45:14.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.709+0000 7f1dc735e640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h6"}] 2024-08-19T10:45:14.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.709+0000 7f1dc735e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:14.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:14.709+0000 7f1dc735e640 1 -- 172.21.15.190:0/1849259863 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h6"} v 0) -- 0x7f1dc006c8d0 con 0x7f1dc0139980 2024-08-19T10:45:16.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.029+0000 7f1db5ffb640 1 -- 172.21.15.190:0/1849259863 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h6"}]=0 removed item id -34 name 'h6' from crush map v234) ==== 118+0+0 (secure 0 0 0) 0x7f1db000d300 con 0x7f1dc0139980 2024-08-19T10:45:16.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.029+0000 7f1db5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h6"}] 2024-08-19T10:45:16.031 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.029+0000 7f1db5ffb640 10 monclient: _finish_command 2 = system:0 removed item id -34 name 'h6' from crush map 2024-08-19T10:45:16.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.137+0000 7f1dc735e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h6"}] 2024-08-19T10:45:16.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.137+0000 7f1dc735e640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h6"}] 2024-08-19T10:45:16.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.137+0000 7f1dc735e640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:16.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.137+0000 7f1dc735e640 1 -- 172.21.15.190:0/1849259863 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h6"} v 0) -- 0x7f1dc013ead0 con 0x7f1dc0139980 2024-08-19T10:45:16.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1db5ffb640 1 -- 172.21.15.190:0/1849259863 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h6"}]=0 device 'h6' does not appear in the crush map v234) ==== 118+0+0 (secure 0 0 0) 0x7f1db000f380 con 0x7f1dc0139980 2024-08-19T10:45:16.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1db5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h6"}] 2024-08-19T10:45:16.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1db5ffb640 10 monclient: _finish_command 3 = system:0 device 'h6' does not appear in the crush map 2024-08-19T10:45:16.141 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h6' does not appear in the crush map 2024-08-19T10:45:16.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 -- 172.21.15.190:0/1849259863 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1da403be00 msgr2=0x7f1da403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:16.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1da403be00 0x7f1da403e2c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009870 tx=0x7f1dbc0a6000 comp rx=0 tx=0).stop 2024-08-19T10:45:16.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 10 monclient: shutdown 2024-08-19T10:45:16.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:16.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 -- 172.21.15.190:0/1849259863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139980 msgr2=0x7f1dc01348b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:16.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139980 0x7f1dc01348b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1dc013bf30 tx=0x7f1db000b9c0 comp rx=0 tx=0).stop 2024-08-19T10:45:16.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 -- 172.21.15.190:0/1849259863 shutdown_connections 2024-08-19T10:45:16.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1da403be00 0x7f1da403e2c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:16.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1dc013a5b0 0x7f1dc013aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:16.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1dc013a140 0x7f1dc0134df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:16.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 --2- 172.21.15.190:0/1849259863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1dc0139980 0x7f1dc01348b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:16.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 -- 172.21.15.190:0/1849259863 >> 172.21.15.190:0/1849259863 conn(0x7f1dc00abde0 msgr2=0x7f1dc0138c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:16.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 -- 172.21.15.190:0/1849259863 shutdown_connections 2024-08-19T10:45:16.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.141+0000 7f1dc735e640 1 -- 172.21.15.190:0/1849259863 wait complete. 2024-08-19T10:45:16.160 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:16.160 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h7 2024-08-19T10:45:16.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient: get_monmap_and_config 2024-08-19T10:45:16.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient: build_initial_monmap 2024-08-19T10:45:16.278 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient: monmap: 2024-08-19T10:45:16.278 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:16.278 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:16.278 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:16.280663+0000 2024-08-19T10:45:16.278 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:16.280663+0000 2024-08-19T10:45:16.278 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:16.279 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:16.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 1 Processor -- start 2024-08-19T10:45:16.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 1 -- start start 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient: init 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient: _add_conns ranks=[1,8,9,7,3,4,2,0,5,10,6] 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4dbc0a6850 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4dbc0a6c30 0x7f4dbc0a7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient(hunting): picked mon.noname-e con 0x7f4dbc0a6c30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4dbc13ebb0 0x7f4dbc140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient(hunting): picked mon.noname-a con 0x7f4dbc13ebb0 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dbc0a5f00 con 0x7f4dbc0a6850 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f4dbc0a3640 con 0x7f4dbc13ebb0 2024-08-19T10:45:16.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4dbc0a3850 con 0x7f4dbc0a6c30 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dc112c640 10 monclient(hunting): authenticate will time out at 1361.850512s 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dbad9d640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f4dbc0a6850 0x7f4dbc0a57f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:59340/0 (socket says 172.21.15.190:59340) 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dbad9d640 1 -- 172.21.15.190:0/12441858 learned_addr learned my addr 172.21.15.190:0/12441858 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dbb59e640 1 --2- 172.21.15.190:0/12441858 >> v2:172.21.15.89:3300/0 conn(0x7f4dbc13ebb0 0x7f4dbc140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 1 --2- 172.21.15.190:0/12441858 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4dbc0a6c30 0x7f4dbc0a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dbb59e640 10 monclient(hunting): get_auth_request con 0x7f4dbc13ebb0 auth_method 0 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dbb59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dbb59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dbb59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:16.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 10 monclient(hunting): get_auth_request con 0x7f4dbc0a6c30 auth_method 0 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dbb59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dbb59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dbb59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 10 monclient(hunting): handle_auth_done global_id 6290 payload 274 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 1 -- 172.21.15.190:0/12441858 >> v2:172.21.15.89:3300/0 conn(0x7f4dbc13ebb0 msgr2=0x7f4dbc140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 1 --2- 172.21.15.190:0/12441858 >> v2:172.21.15.89:3300/0 conn(0x7f4dbc13ebb0 0x7f4dbc140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:16.284 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 1 -- 172.21.15.190:0/12441858 >> v1:172.21.15.190:6789/0 conn(0x7f4dbc0a6850 legacy=0x7f4dbc0a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:16.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 10 monclient: _finish_hunting 0 2024-08-19T10:45:16.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 1 monclient: found mon.noname-e 2024-08-19T10:45:16.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:16.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:16.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dba59c640 1 -- 172.21.15.190:0/12441858 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc0a46e0 con 0x7f4dbc0a6c30 2024-08-19T10:45:16.285 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.277+0000 7f4dbb59e640 1 --2- 172.21.15.190:0/12441858 >> v2:172.21.15.89:3300/0 conn(0x7f4dbc13ebb0 0x7f4dbc140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:16.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dba59c640 1 -- 172.21.15.190:0/12441858 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4dbc0a6c30 msgr2=0x7f4dbc0a7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dba59c640 1 --2- 172.21.15.190:0/12441858 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4dbc0a6c30 0x7f4dbc0a7010 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4db00097c0 tx=0x7f4db001c360 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2144e715602b307e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 1 -- 172.21.15.190:0/12441858 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4db00b0070 con 0x7f4dbc0a6c30 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 10 monclient: dump: 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:16.287 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 10 monclient: _finish_auth 0 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:16.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 1 -- 172.21.15.190:0/12441858 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db00aa810 con 0x7f4dbc0a6c30 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 1 -- 172.21.15.190:0/12441858 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4db00a9300 con 0x7f4dbc0a6c30 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 5 monclient: authenticate success, global_id 6290 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 4 set_mon_vals no callback set 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4db9d9b640 10 monclient: dump: 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:16.289 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:16.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient: shutdown 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 -- 172.21.15.190:0/12441858 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4dbc0a6c30 msgr2=0x7f4dbc0a7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 --2- 172.21.15.190:0/12441858 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4dbc0a6c30 0x7f4dbc0a7010 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4db00097c0 tx=0x7f4db001c360 comp rx=0 tx=0).stop 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 -- 172.21.15.190:0/12441858 shutdown_connections 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 --2- 172.21.15.190:0/12441858 >> v2:172.21.15.89:3300/0 conn(0x7f4dbc13ebb0 0x7f4dbc140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 --2- 172.21.15.190:0/12441858 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4dbc0a6c30 0x7f4dbc0a7010 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 -- 172.21.15.190:0/12441858 >> 172.21.15.190:0/12441858 conn(0x7f4dbc0abe50 msgr2=0x7f4dbc0ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 -- 172.21.15.190:0/12441858 shutdown_connections 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 -- 172.21.15.190:0/12441858 wait complete. 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient: build_initial_monmap 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient: monmap: 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:16.291 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:16.292 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:16.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 Processor -- start 2024-08-19T10:45:16.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 -- start start 2024-08-19T10:45:16.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient: init 2024-08-19T10:45:16.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:16.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient: _add_conns ranks=[7,0,4,6,5,2,8,1,3] 2024-08-19T10:45:16.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4dbc0a6850 0x7f4dbc1d1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient(hunting): picked mon.noname-i con 0x7f4dbc0a6850 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dbc0a6c30 0x7f4dbc139a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient(hunting): picked mon.noname-a con 0x7f4dbc0a6c30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4dbc13ebb0 0x7f4dbc0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient(hunting): picked mon.noname-c con 0x7f4dbc13ebb0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc143e80 con 0x7f4dbc0a6c30 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f4dbc143d00 con 0x7f4dbc13ebb0 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4dbc144000 con 0x7f4dbc0a6850 2024-08-19T10:45:16.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dc112c640 10 monclient(hunting): authenticate will time out at 1361.858635s 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4dbc0a6850 0x7f4dbc1d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4dbc0a6850 0x7f4dbc1d1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:53878/0 (socket says 172.21.15.190:53878) 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 1 -- 172.21.15.190:0/1168919046 learned_addr learned my addr 172.21.15.190:0/1168919046 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 10 monclient(hunting): get_auth_request con 0x7f4dbc0a6850 auth_method 0 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 10 monclient(hunting): handle_auth_done global_id 15703 payload 824 2024-08-19T10:45:16.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dba59c640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dbc0a6c30 0x7f4dbc139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbb59e640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4dbc13ebb0 0x7f4dbc0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 1 -- 172.21.15.190:0/1168919046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4dbc13ebb0 msgr2=0x7f4dbc0a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4dbc13ebb0 0x7f4dbc0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 1 -- 172.21.15.190:0/1168919046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dbc0a6c30 msgr2=0x7f4dbc139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dbc0a6c30 0x7f4dbc139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 10 monclient: _finish_hunting 0 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 1 monclient: found mon.noname-i 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.285+0000 7f4dbad9d640 1 -- 172.21.15.190:0/1168919046 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc13dbc0 con 0x7f4dbc0a6850 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dbad9d640 1 -- 172.21.15.190:0/1168919046 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4dbc0a6850 msgr2=0x7f4dbc1d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:16.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dbad9d640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4dbc0a6850 0x7f4dbc1d1390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00cc70 tx=0x7f4dac00a430 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 1 -- 172.21.15.190:0/1168919046 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4dac00ce20 con 0x7f4dbc0a6850 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 10 monclient: dump: 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:16.300 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:16.301 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:16.301 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:16.301 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:16.301 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:16.301 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:16.301 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:16.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:16.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:16.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:16.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dc112c640 5 monclient: authenticate success, global_id 15703 2024-08-19T10:45:16.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 1 -- 172.21.15.190:0/1168919046 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dac012870 con 0x7f4dbc0a6850 2024-08-19T10:45:16.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dc112c640 10 monclient: _renew_subs 2024-08-19T10:45:16.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dc112c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:16.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dc112c640 1 -- 172.21.15.190:0/1168919046 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dbc141140 con 0x7f4dbc0a6850 2024-08-19T10:45:16.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:16.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 1 -- 172.21.15.190:0/1168919046 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4dac007070 con 0x7f4dbc0a6850 2024-08-19T10:45:16.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:16.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:16.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 10 monclient: dump: 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:16.303 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dc112c640 10 monclient: _renew_subs 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dc112c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dc112c640 1 -- 172.21.15.190:0/1168919046 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dbc124130 con 0x7f4dbc0a6850 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4db959a640 4 set_mon_vals no callback set 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dc112c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dc112c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dc112c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dc112c640 1 -- 172.21.15.190:0/1168919046 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dbc0a27b0 con 0x7f4dbc0a6850 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 1 -- 172.21.15.190:0/1168919046 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4dac012400 con 0x7f4dbc0a6850 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4da803be00 0x7f4da803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:16.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4da37fe640 1 -- 172.21.15.190:0/1168919046 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(234..234 src has 1..234) ==== 13724+0+0 (secure 0 0 0) 0x7f4dac051680 con 0x7f4dbc0a6850 2024-08-19T10:45:16.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dba59c640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4da803be00 0x7f4da803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:16.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.289+0000 7f4dba59c640 10 monclient: get_auth_request con 0x7f4da803be00 auth_method 0 2024-08-19T10:45:16.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.293+0000 7f4dba59c640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4da803be00 0x7f4da803e2c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4db00ad000 tx=0x7f4db00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:16.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.293+0000 7f4da37fe640 1 -- 172.21.15.190:0/1168919046 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4dbc141140 con 0x7f4dbc0a6850 2024-08-19T10:45:16.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.293+0000 7f4da37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:16.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.293+0000 7f4da37fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:16.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.469+0000 7f4dc112c640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h7"}] 2024-08-19T10:45:16.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.469+0000 7f4dc112c640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h7"}] 2024-08-19T10:45:16.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.469+0000 7f4dc112c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:16.471 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:16.469+0000 7f4dc112c640 1 -- 172.21.15.190:0/1168919046 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h7"} v 0) -- 0x7f4dbc0ac610 con 0x7f4dbc0a6850 2024-08-19T10:45:17.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.041+0000 7f4da37fe640 1 -- 172.21.15.190:0/1168919046 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h7"}]=0 removed item id -37 name 'h7' from crush map v235) ==== 118+0+0 (secure 0 0 0) 0x7f4dbc0ac610 con 0x7f4dbc0a6850 2024-08-19T10:45:17.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.041+0000 7f4da37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h7"}] 2024-08-19T10:45:17.045 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.041+0000 7f4da37fe640 10 monclient: _finish_command 2 = system:0 removed item id -37 name 'h7' from crush map 2024-08-19T10:45:17.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.153+0000 7f4dc112c640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h7"}] 2024-08-19T10:45:17.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.153+0000 7f4dc112c640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h7"}] 2024-08-19T10:45:17.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.153+0000 7f4dc112c640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:17.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.153+0000 7f4dc112c640 1 -- 172.21.15.190:0/1168919046 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h7"} v 0) -- 0x7f4dbc0777a0 con 0x7f4dbc0a6850 2024-08-19T10:45:17.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4da37fe640 1 -- 172.21.15.190:0/1168919046 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h7"}]=0 device 'h7' does not appear in the crush map v235) ==== 118+0+0 (secure 0 0 0) 0x7f4dbc0777a0 con 0x7f4dbc0a6850 2024-08-19T10:45:17.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4da37fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h7"}] 2024-08-19T10:45:17.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4da37fe640 10 monclient: _finish_command 3 = system:0 device 'h7' does not appear in the crush map 2024-08-19T10:45:17.157 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h7' does not appear in the crush map 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 -- 172.21.15.190:0/1168919046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4da803be00 msgr2=0x7f4da803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4da803be00 0x7f4da803e2c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4db00ad000 tx=0x7f4db00a6000 comp rx=0 tx=0).stop 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 10 monclient: shutdown 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 -- 172.21.15.190:0/1168919046 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4dbc0a6850 msgr2=0x7f4dbc1d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4dbc0a6850 0x7f4dbc1d1390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00cc70 tx=0x7f4dac00a430 comp rx=0 tx=0).stop 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 -- 172.21.15.190:0/1168919046 shutdown_connections 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4da803be00 0x7f4da803e2c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4dbc13ebb0 0x7f4dbc0a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4dbc0a6c30 0x7f4dbc139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 --2- 172.21.15.190:0/1168919046 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4dbc0a6850 0x7f4dbc1d1390 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:17.160 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 -- 172.21.15.190:0/1168919046 >> 172.21.15.190:0/1168919046 conn(0x7f4dbc0abe50 msgr2=0x7f4dbc13fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:17.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 -- 172.21.15.190:0/1168919046 shutdown_connections 2024-08-19T10:45:17.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.157+0000 7f4dc112c640 1 -- 172.21.15.190:0/1168919046 wait complete. 2024-08-19T10:45:17.175 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:17.175 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h8 2024-08-19T10:45:17.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient: get_monmap_and_config 2024-08-19T10:45:17.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient: build_initial_monmap 2024-08-19T10:45:17.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient: monmap: 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:17.296928+0000 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:17.296928+0000 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:17.295 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:17.296 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:17.296 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:17.296 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:17.296 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:17.296 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:17.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 1 Processor -- start 2024-08-19T10:45:17.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 1 -- start start 2024-08-19T10:45:17.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient: init 2024-08-19T10:45:17.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:17.298 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient: _add_conns ranks=[8,6,3,2,10,0,9,4,1,5,7] 2024-08-19T10:45:17.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f176c0a56c0 0x7f176c0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:17.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient(hunting): picked mon.noname-e con 0x7f176c0a56c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:17.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f176c0a16e0 0x7f176c0a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:17.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient(hunting): picked mon.noname-d con 0x7f176c0a16e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:17.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c137470 0x7f176c1378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:17.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient(hunting): picked mon.noname-g con 0x7f176c137470 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:17.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:17.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f176c0a4090 con 0x7f176c137470 2024-08-19T10:45:17.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:17.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f176c071e20 con 0x7f176c0a16e0 2024-08-19T10:45:17.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:17.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f176c0737c0 con 0x7f176c0a56c0 2024-08-19T10:45:17.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:17.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f17733c6640 10 monclient(hunting): authenticate will time out at 1362.867531s 2024-08-19T10:45:17.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c137470 0x7f176c1378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:17.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c137470 0x7f176c1378d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:57756/0 (socket says 172.21.15.190:57756) 2024-08-19T10:45:17.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 1 -- 172.21.15.190:0/64282158 learned_addr learned my addr 172.21.15.190:0/64282158 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:17.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 10 monclient(hunting): get_auth_request con 0x7f176c137470 auth_method 0 2024-08-19T10:45:17.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:17.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:17.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:17.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 10 monclient(hunting): handle_auth_done global_id 34761 payload 274 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771162640 1 --2- 172.21.15.190:0/64282158 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f176c0a56c0 0x7f176c0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.293+0000 7f1771963640 1 -- 172.21.15.190:0/64282158 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f176c0a56c0 msgr2=0x7f176c0a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.297+0000 7f1770961640 1 --2- 172.21.15.190:0/64282158 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f176c0a16e0 0x7f176c0a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.297+0000 7f1771963640 1 --2- 172.21.15.190:0/64282158 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f176c0a56c0 0x7f176c0a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.297+0000 7f1771963640 1 -- 172.21.15.190:0/64282158 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f176c0a16e0 msgr2=0x7f176c0a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.297+0000 7f1771963640 1 --2- 172.21.15.190:0/64282158 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f176c0a16e0 0x7f176c0a5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.297+0000 7f1771963640 10 monclient: _finish_hunting 0 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.297+0000 7f1771963640 1 monclient: found mon.noname-g 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.297+0000 7f1771963640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:17.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.297+0000 7f1771963640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:17.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.297+0000 7f1771963640 1 -- 172.21.15.190:0/64282158 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c073aa0 con 0x7f176c137470 2024-08-19T10:45:17.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.297+0000 7f1771162640 10 monclient: get_auth_request con 0x7f176c0a56c0 auth_method 0 2024-08-19T10:45:17.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.297+0000 7f1771162640 1 --2- 172.21.15.190:0/64282158 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f176c0a56c0 0x7f176c0a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:17.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 1 -- 172.21.15.190:0/64282158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c137470 msgr2=0x7f176c1378d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:17.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 1 --2- 172.21.15.190:0/64282158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c137470 0x7f176c1378d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1764009e80 tx=0x7f176401c880 comp rx=0 tx=0).ready entity=mon.3 client_cookie=310ba08a2083bb7f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:17.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 1 -- 172.21.15.190:0/64282158 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17640091b0 con 0x7f176c137470 2024-08-19T10:45:17.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:17.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:17.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 10 monclient: dump: 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:17.306 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 10 monclient: _finish_auth 0 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 1 -- 172.21.15.190:0/64282158 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17640a5b50 con 0x7f176c137470 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 1 -- 172.21.15.190:0/64282158 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17640a5d00 con 0x7f176c137470 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 5 monclient: authenticate success, global_id 34761 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 4 set_mon_vals no callback set 2024-08-19T10:45:17.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1763fff640 10 monclient: dump: 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:17.308 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient: shutdown 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 -- 172.21.15.190:0/64282158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c137470 msgr2=0x7f176c1378d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 --2- 172.21.15.190:0/64282158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c137470 0x7f176c1378d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1764009e80 tx=0x7f176401c880 comp rx=0 tx=0).stop 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 -- 172.21.15.190:0/64282158 shutdown_connections 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 --2- 172.21.15.190:0/64282158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c137470 0x7f176c1378d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 --2- 172.21.15.190:0/64282158 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f176c0a16e0 0x7f176c0a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:17.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 --2- 172.21.15.190:0/64282158 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f176c0a56c0 0x7f176c0a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 -- 172.21.15.190:0/64282158 >> 172.21.15.190:0/64282158 conn(0x7f176c0aa2b0 msgr2=0x7f176c0a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 -- 172.21.15.190:0/64282158 shutdown_connections 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 -- 172.21.15.190:0/64282158 wait complete. 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient: build_initial_monmap 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient: monmap: 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:17.310 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 Processor -- start 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 -- start start 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient: init 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:17.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient: _add_conns ranks=[5,0,1,3,4,8,2,7,6] 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f176c0a56c0 0x7f176c1cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient(hunting): picked mon.noname-h con 0x7f176c0a56c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f176c137470 0x7f176c1cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient(hunting): picked mon.noname-a con 0x7f176c137470 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c13b230 0x7f176c13b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient(hunting): picked mon.noname-f con 0x7f176c13b230 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f176c0a71b0 con 0x7f176c137470 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f176c138220 con 0x7f176c13b230 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f176c1410f0 con 0x7f176c0a56c0 2024-08-19T10:45:17.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f17733c6640 10 monclient(hunting): authenticate will time out at 1362.874888s 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771162640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f176c0a56c0 0x7f176c1cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c13b230 0x7f176c13b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771162640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f176c0a56c0 0x7f176c1cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:35818/0 (socket says 172.21.15.190:35818) 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 10 monclient(hunting): get_auth_request con 0x7f176c13b230 auth_method 0 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771162640 1 -- 172.21.15.190:0/2948285025 learned_addr learned my addr 172.21.15.190:0/2948285025 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771162640 10 monclient(hunting): get_auth_request con 0x7f176c0a56c0 auth_method 0 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771162640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:17.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771162640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771162640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771162640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771162640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771162640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1770961640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f176c137470 0x7f176c1cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1770961640 10 monclient(hunting): get_auth_request con 0x7f176c137470 auth_method 0 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1770961640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1770961640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1770961640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:17.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 10 monclient(hunting): handle_auth_done global_id 44236 payload 824 2024-08-19T10:45:17.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 1 -- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f176c0a56c0 msgr2=0x7f176c1cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:17.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f176c0a56c0 0x7f176c1cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:17.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 1 -- 172.21.15.190:0/2948285025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f176c137470 msgr2=0x7f176c1cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:17.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f176c137470 0x7f176c1cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:17.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 10 monclient: _finish_hunting 0 2024-08-19T10:45:17.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 1 monclient: found mon.noname-f 2024-08-19T10:45:17.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:17.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:45:17.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771963640 1 -- 172.21.15.190:0/2948285025 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c0a3650 con 0x7f176c13b230 2024-08-19T10:45:17.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.301+0000 7f1771162640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f176c0a56c0 0x7f176c1cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:17.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1771963640 1 -- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c13b230 msgr2=0x7f176c13b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:17.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1771963640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c13b230 0x7f176c13b690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f17640ab000 tx=0x7f1764019680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 1 -- 172.21.15.190:0/2948285025 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17640bc030 con 0x7f176c13b230 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 10 monclient: dump: 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:17.318 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 1 -- 172.21.15.190:0/2948285025 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1764003dc0 con 0x7f176c13b230 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:17.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 1 -- 172.21.15.190:0/2948285025 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f17640b7450 con 0x7f176c13b230 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f17733c6640 5 monclient: authenticate success, global_id 44236 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f17733c6640 10 monclient: _renew_subs 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f17733c6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f17733c6640 1 -- 172.21.15.190:0/2948285025 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176c0a8ce0 con 0x7f176c13b230 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1762ffd640 4 set_mon_vals no callback set 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 10 monclient: dump: 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:17.320 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f17733c6640 10 monclient: _renew_subs 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f17733c6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f17733c6640 1 -- 172.21.15.190:0/2948285025 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176c1216c0 con 0x7f176c13b230 2024-08-19T10:45:17.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f17733c6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f17733c6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f17733c6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f17733c6640 1 -- 172.21.15.190:0/2948285025 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176c1378d0 con 0x7f176c13b230 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 1 -- 172.21.15.190:0/2948285025 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f17640b7670 con 0x7f176c13b230 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f175003bdf0 0x7f175003e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1761ffb640 1 -- 172.21.15.190:0/2948285025 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 13514+0+0 (secure 0 0 0) 0x7f17640bd070 con 0x7f176c13b230 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1771162640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f175003bdf0 0x7f175003e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.305+0000 7f1771162640 10 monclient: get_auth_request con 0x7f175003bdf0 auth_method 0 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.309+0000 7f1771162640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f175003bdf0 0x7f175003e2b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1768006fd0 tx=0x7f1768008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.313+0000 7f1761ffb640 1 -- 172.21.15.190:0/2948285025 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f17640c5020 con 0x7f176c13b230 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.313+0000 7f1761ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:17.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.313+0000 7f1761ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:17.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.489+0000 7f17733c6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h8"}] 2024-08-19T10:45:17.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.489+0000 7f17733c6640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h8"}] 2024-08-19T10:45:17.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.489+0000 7f17733c6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:17.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:17.489+0000 7f17733c6640 1 -- 172.21.15.190:0/2948285025 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h8"} v 0) -- 0x7f176c06fd10 con 0x7f176c13b230 2024-08-19T10:45:18.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.049+0000 7f1761ffb640 1 -- 172.21.15.190:0/2948285025 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h8"}]=0 removed item id -40 name 'h8' from crush map v236) ==== 118+0+0 (secure 0 0 0) 0x7f176c0a8ce0 con 0x7f176c13b230 2024-08-19T10:45:18.049 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.049+0000 7f1761ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h8"}] 2024-08-19T10:45:18.050 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.049+0000 7f1761ffb640 10 monclient: _finish_command 2 = system:0 removed item id -40 name 'h8' from crush map 2024-08-19T10:45:18.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.157+0000 7f17733c6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h8"}] 2024-08-19T10:45:18.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.157+0000 7f17733c6640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h8"}] 2024-08-19T10:45:18.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.157+0000 7f17733c6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:18.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.157+0000 7f17733c6640 1 -- 172.21.15.190:0/2948285025 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h8"} v 0) -- 0x7f176c1cfd20 con 0x7f176c13b230 2024-08-19T10:45:18.230 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.229+0000 7f1761ffb640 1 -- 172.21.15.190:0/2948285025 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h8"}]=0 device 'h8' does not appear in the crush map v236) ==== 118+0+0 (secure 0 0 0) 0x7f176c0a3650 con 0x7f176c13b230 2024-08-19T10:45:18.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.229+0000 7f1761ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h8"}] 2024-08-19T10:45:18.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.229+0000 7f1761ffb640 10 monclient: _finish_command 3 = system:0 device 'h8' does not appear in the crush map 2024-08-19T10:45:18.231 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h8' does not appear in the crush map 2024-08-19T10:45:18.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 -- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f175003bdf0 msgr2=0x7f175003e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:18.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f175003bdf0 0x7f175003e2b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1768006fd0 tx=0x7f1768008040 comp rx=0 tx=0).stop 2024-08-19T10:45:18.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 10 monclient: shutdown 2024-08-19T10:45:18.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:18.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 -- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c13b230 msgr2=0x7f176c13b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:18.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c13b230 0x7f176c13b690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f17640ab000 tx=0x7f1764019680 comp rx=0 tx=0).stop 2024-08-19T10:45:18.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 -- 172.21.15.190:0/2948285025 shutdown_connections 2024-08-19T10:45:18.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f175003bdf0 0x7f175003e2b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:18.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c13b230 0x7f176c13b690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:18.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f176c137470 0x7f176c1cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:18.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 --2- 172.21.15.190:0/2948285025 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f176c0a56c0 0x7f176c1cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:18.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 -- 172.21.15.190:0/2948285025 >> 172.21.15.190:0/2948285025 conn(0x7f176c0aa2b0 msgr2=0x7f176c0a1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:18.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 -- 172.21.15.190:0/2948285025 shutdown_connections 2024-08-19T10:45:18.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.233+0000 7f17733c6640 1 -- 172.21.15.190:0/2948285025 wait complete. 2024-08-19T10:45:18.251 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:18.251 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h9 2024-08-19T10:45:18.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient: get_monmap_and_config 2024-08-19T10:45:18.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient: build_initial_monmap 2024-08-19T10:45:18.370 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient: monmap: 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:18.373005+0000 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:18.373005+0000 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:18.371 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:18.372 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:18.372 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:18.372 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:18.372 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:18.372 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:18.372 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:18.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 1 Processor -- start 2024-08-19T10:45:18.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 1 -- start start 2024-08-19T10:45:18.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient: init 2024-08-19T10:45:18.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:18.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient: _add_conns ranks=[0,4,10,2,1,7,8,9,6,5,3] 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f08c409b960 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08c409c6a0 0x7f08c409bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient(hunting): picked mon.noname-c con 0x7f08c409c6a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f08c406cfe0 0x7f08c406d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient(hunting): picked mon.noname-f con 0x7f08c406cfe0 addr v2:172.21.15.190:3300/0 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c40a9180 con 0x7f08c409b960 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f08c409d0f0 con 0x7f08c406cfe0 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f08c409c3e0 con 0x7f08c409c6a0 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08cb6a7640 10 monclient(hunting): authenticate will time out at 1363.943543s 2024-08-19T10:45:18.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08c9c44640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f08c406cfe0 0x7f08c406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08c9c44640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f08c406cfe0 0x7f08c406d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:33150/0 (socket says 172.21.15.190:33150) 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08c9c44640 1 -- 172.21.15.190:0/3604246568 learned_addr learned my addr 172.21.15.190:0/3604246568 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08c9c44640 10 monclient(hunting): get_auth_request con 0x7f08c406cfe0 auth_method 0 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08c9c44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08c9c44640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08c9c44640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08c9c44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08c9c44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08c9c44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.369+0000 7f08c8c42640 1 --2- 172.21.15.190:0/3604246568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08c409c6a0 0x7f08c409bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c8c42640 10 monclient(hunting): get_auth_request con 0x7f08c409c6a0 auth_method 0 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c8c42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:18.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c8c42640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c8c42640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c9c44640 10 monclient(hunting): handle_auth_done global_id 44245 payload 274 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c9c44640 1 -- 172.21.15.190:0/3604246568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08c409c6a0 msgr2=0x7f08c409bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c9c44640 1 --2- 172.21.15.190:0/3604246568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08c409c6a0 0x7f08c409bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c9c44640 1 -- 172.21.15.190:0/3604246568 >> v1:172.21.15.89:6789/0 conn(0x7f08c409b960 legacy=0x7f08c406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c9c44640 10 monclient: _finish_hunting 0 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c9c44640 1 monclient: found mon.noname-f 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c9c44640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c9c44640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.373+0000 7f08c9c44640 1 -- 172.21.15.190:0/3604246568 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c4069a50 con 0x7f08c406cfe0 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 1 -- 172.21.15.190:0/3604246568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08c406cfe0 msgr2=0x7f08c406d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:18.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 1 --2- 172.21.15.190:0/3604246568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08c406cfe0 0x7f08c406d470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009e50 tx=0x7f08bc01c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ea69cd8de6e2be2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 1 -- 172.21.15.190:0/3604246568 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08bc0a59a0 con 0x7f08c406cfe0 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 10 monclient: dump: 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:18.392 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 10 monclient: _finish_auth 0 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 1 -- 172.21.15.190:0/3604246568 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08bc0a5b60 con 0x7f08c406cfe0 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 1 -- 172.21.15.190:0/3604246568 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08bc0a5d10 con 0x7f08c406cfe0 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 5 monclient: authenticate success, global_id 44245 2024-08-19T10:45:18.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 4 set_mon_vals no callback set 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08bbfff640 10 monclient: dump: 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:18.394 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient: shutdown 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 -- 172.21.15.190:0/3604246568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08c406cfe0 msgr2=0x7f08c406d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 --2- 172.21.15.190:0/3604246568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08c406cfe0 0x7f08c406d470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009e50 tx=0x7f08bc01c8f0 comp rx=0 tx=0).stop 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 -- 172.21.15.190:0/3604246568 shutdown_connections 2024-08-19T10:45:18.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 --2- 172.21.15.190:0/3604246568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08c406cfe0 0x7f08c406d470 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 --2- 172.21.15.190:0/3604246568 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08c409c6a0 0x7f08c409bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 -- 172.21.15.190:0/3604246568 >> 172.21.15.190:0/3604246568 conn(0x7f08c40a8370 msgr2=0x7f08c40a8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 -- 172.21.15.190:0/3604246568 shutdown_connections 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 -- 172.21.15.190:0/3604246568 wait complete. 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient: build_initial_monmap 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient: monmap: 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:18.396 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 Processor -- start 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 -- start start 2024-08-19T10:45:18.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient: init 2024-08-19T10:45:18.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:18.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient: _add_conns ranks=[2,8,6,5,1,3,7,0,4] 2024-08-19T10:45:18.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08c406cfe0 0x7f08c4136750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:18.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient(hunting): picked mon.noname-b con 0x7f08c406cfe0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:18.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f08c409b960 0x7f08c4136c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:18.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient(hunting): picked mon.noname-e con 0x7f08c409b960 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:18.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08c409c6a0 0x7f08c413e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:18.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient(hunting): picked mon.noname-d con 0x7f08c409c6a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:18.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f08c4143ee0 con 0x7f08c406cfe0 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f08c4143d60 con 0x7f08c409c6a0 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f08c4144060 con 0x7f08c409b960 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08cb6a7640 10 monclient(hunting): authenticate will time out at 1363.950967s 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9443640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08c406cfe0 msgr2=0x7f08c4136750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9443640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08c406cfe0 0x7f08c4136750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08c409c6a0 0x7f08c413e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c8c42640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f08c409b960 0x7f08c4136c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08c409c6a0 0x7f08c413e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:34818/0 (socket says 172.21.15.190:34818) 2024-08-19T10:45:18.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 1 -- 172.21.15.190:0/2612883887 learned_addr learned my addr 172.21.15.190:0/2612883887 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 10 monclient(hunting): get_auth_request con 0x7f08c409c6a0 auth_method 0 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c8c42640 10 monclient(hunting): get_auth_request con 0x7f08c409b960 auth_method 0 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c8c42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c8c42640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c8c42640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c8c42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:18.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c8c42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c8c42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 10 monclient(hunting): handle_auth_done global_id 44340 payload 824 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 1 -- 172.21.15.190:0/2612883887 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f08c409b960 msgr2=0x7f08c4136c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f08c409b960 0x7f08c4136c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 1 -- 172.21.15.190:0/2612883887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08c406cfe0 msgr2=0x7f08c4136750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08c406cfe0 0x7f08c4136750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 10 monclient: _finish_hunting 0 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 1 monclient: found mon.noname-d 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.377+0000 7f08c9c44640 1 -- 172.21.15.190:0/2612883887 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c40a2690 con 0x7f08c409c6a0 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08c9c44640 1 -- 172.21.15.190:0/2612883887 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08c409c6a0 msgr2=0x7f08c413e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:18.401 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08c9c44640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08c409c6a0 0x7f08c413e2c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08bc01c720 tx=0x7f08bc019690 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 1 -- 172.21.15.190:0/2612883887 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08bc0bc020 con 0x7f08c409c6a0 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 10 monclient: dump: 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:18.402 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08cb6a7640 5 monclient: authenticate success, global_id 44340 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08cb6a7640 10 monclient: _renew_subs 2024-08-19T10:45:18.403 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08cb6a7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08cb6a7640 1 -- 172.21.15.190:0/2612883887 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c40a5460 con 0x7f08c409c6a0 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 1 -- 172.21.15.190:0/2612883887 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08bc003dc0 con 0x7f08c409c6a0 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08cb6a7640 10 monclient: _renew_subs 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08cb6a7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08cb6a7640 1 -- 172.21.15.190:0/2612883887 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c4070ce0 con 0x7f08c409c6a0 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08bb7fe640 4 set_mon_vals no callback set 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 1 -- 172.21.15.190:0/2612883887 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08bc0044b0 con 0x7f08c409c6a0 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08b9ffb640 10 monclient: dump: 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:18.404 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:18.405 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:18.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08cb6a7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:18.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08cb6a7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:18.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08cb6a7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:18.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.381+0000 7f08cb6a7640 1 -- 172.21.15.190:0/2612883887 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c406d470 con 0x7f08c409c6a0 2024-08-19T10:45:18.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.385+0000 7f08b9ffb640 1 -- 172.21.15.190:0/2612883887 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f08bc0bf020 con 0x7f08c409c6a0 2024-08-19T10:45:18.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.385+0000 7f08b9ffb640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08a803be30 0x7f08a803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:18.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.385+0000 7f08b9ffb640 1 -- 172.21.15.190:0/2612883887 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(236..236 src has 1..236) ==== 13304+0+0 (secure 0 0 0) 0x7f08bc0ec450 con 0x7f08c409c6a0 2024-08-19T10:45:18.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.385+0000 7f08c9443640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08a803be30 0x7f08a803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:18.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.385+0000 7f08c9443640 10 monclient: get_auth_request con 0x7f08a803be30 auth_method 0 2024-08-19T10:45:18.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.385+0000 7f08c9443640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08a803be30 0x7f08a803e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f08c0003730 tx=0x7f08c0007a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:18.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.397+0000 7f08b9ffb640 1 -- 172.21.15.190:0/2612883887 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f08bc016600 con 0x7f08c409c6a0 2024-08-19T10:45:18.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.397+0000 7f08b9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:18.407 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.397+0000 7f08b9ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:18.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.573+0000 7f08cb6a7640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h9"}] 2024-08-19T10:45:18.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.573+0000 7f08cb6a7640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h9"}] 2024-08-19T10:45:18.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.573+0000 7f08cb6a7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:18.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:18.573+0000 7f08cb6a7640 1 -- 172.21.15.190:0/2612883887 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h9"} v 0) -- 0x7f08c406dae0 con 0x7f08c409c6a0 2024-08-19T10:45:19.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.229+0000 7f08b9ffb640 1 -- 172.21.15.190:0/2612883887 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h9"}]=0 removed item id -43 name 'h9' from crush map v237) ==== 118+0+0 (secure 0 0 0) 0x7f08bc0ec170 con 0x7f08c409c6a0 2024-08-19T10:45:19.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.233+0000 7f08b9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h9"}] 2024-08-19T10:45:19.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.233+0000 7f08b9ffb640 10 monclient: _finish_command 2 = system:0 removed item id -43 name 'h9' from crush map 2024-08-19T10:45:19.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.337+0000 7f08cb6a7640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h9"}] 2024-08-19T10:45:19.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.337+0000 7f08cb6a7640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h9"}] 2024-08-19T10:45:19.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.337+0000 7f08cb6a7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:19.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.337+0000 7f08cb6a7640 1 -- 172.21.15.190:0/2612883887 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h9"} v 0) -- 0x7f08c406dae0 con 0x7f08c409c6a0 2024-08-19T10:45:19.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08b9ffb640 1 -- 172.21.15.190:0/2612883887 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h9"}]=0 device 'h9' does not appear in the crush map v237) ==== 118+0+0 (secure 0 0 0) 0x7f08bc0b75c0 con 0x7f08c409c6a0 2024-08-19T10:45:19.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08b9ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h9"}] 2024-08-19T10:45:19.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08b9ffb640 10 monclient: _finish_command 3 = system:0 device 'h9' does not appear in the crush map 2024-08-19T10:45:19.342 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h9' does not appear in the crush map 2024-08-19T10:45:19.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 1 -- 172.21.15.190:0/2612883887 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08a803be30 msgr2=0x7f08a803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:19.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08a803be30 0x7f08a803e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f08c0003730 tx=0x7f08c0007a20 comp rx=0 tx=0).stop 2024-08-19T10:45:19.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 10 monclient: shutdown 2024-08-19T10:45:19.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:19.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 1 -- 172.21.15.190:0/2612883887 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08c409c6a0 msgr2=0x7f08c413e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:19.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08c409c6a0 0x7f08c413e2c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08bc01c720 tx=0x7f08bc019690 comp rx=0 tx=0).stop 2024-08-19T10:45:19.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 1 -- 172.21.15.190:0/2612883887 shutdown_connections 2024-08-19T10:45:19.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08c409c6a0 0x7f08c413e2c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:19.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08a803be30 0x7f08a803e2f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:19.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f08c409b960 0x7f08c4136c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:19.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 1 --2- 172.21.15.190:0/2612883887 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f08c406cfe0 0x7f08c4136750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:19.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 1 -- 172.21.15.190:0/2612883887 >> 172.21.15.190:0/2612883887 conn(0x7f08c40a8370 msgr2=0x7f08c413cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:19.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.341+0000 7f08cb6a7640 1 -- 172.21.15.190:0/2612883887 shutdown_connections 2024-08-19T10:45:19.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.345+0000 7f08cb6a7640 1 -- 172.21.15.190:0/2612883887 wait complete. 2024-08-19T10:45:19.361 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:19.361 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h10 2024-08-19T10:45:19.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient: get_monmap_and_config 2024-08-19T10:45:19.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient: build_initial_monmap 2024-08-19T10:45:19.478 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient: monmap: 2024-08-19T10:45:19.478 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:19.479 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:19.479 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:19.480919+0000 2024-08-19T10:45:19.479 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:19.480919+0000 2024-08-19T10:45:19.479 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:19.479 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:19.479 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:19.479 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:19.479 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:19.479 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:19.480 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:19.480 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:19.480 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:19.480 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:19.480 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:19.480 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:19.480 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:19.480 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:19.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 1 Processor -- start 2024-08-19T10:45:19.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 1 -- start start 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient: init 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient: _add_conns ranks=[5,1,0,4,10,3,8,6,7,9,2] 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe138139850 0x7fe138071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient(hunting): picked mon.noname-h con 0x7fe138139850 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe138139c30 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe13813a010 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1380a64e0 con 0x7fe13813a010 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe13806e0e0 con 0x7fe138139c30 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe13806f180 con 0x7fe138139850 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:19.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe13dd8b640 10 monclient(hunting): authenticate will time out at 1365.051393s 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe138139850 0x7fe138071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe138139850 0x7fe138071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:35832/0 (socket says 172.21.15.190:35832) 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 1 -- 172.21.15.190:0/566822072 learned_addr learned my addr 172.21.15.190:0/566822072 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 10 monclient(hunting): get_auth_request con 0x7fe138139850 auth_method 0 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 10 monclient(hunting): handle_auth_done global_id 6359 payload 274 2024-08-19T10:45:19.484 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 1 -- 172.21.15.190:0/566822072 >> v1:172.21.15.190:6789/0 conn(0x7fe138139c30 legacy=0x7fe1380a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:19.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 1 -- 172.21.15.190:0/566822072 >> v1:172.21.15.89:6789/0 conn(0x7fe13813a010 legacy=0x7fe1380a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:19.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:19.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 1 monclient: found mon.noname-h 2024-08-19T10:45:19.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:19.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:19.485 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.477+0000 7fe1377fe640 1 -- 172.21.15.190:0/566822072 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13806f460 con 0x7fe138139850 2024-08-19T10:45:19.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 1 -- 172.21.15.190:0/566822072 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe138139850 msgr2=0x7fe138071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:19.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 1 --2- 172.21.15.190:0/566822072 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe138139850 0x7fe138071fe0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe1280098a0 tx=0x7fe12801c3e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=c563023238c19369 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:19.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 1 -- 172.21.15.190:0/566822072 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe1280b0070 con 0x7fe138139850 2024-08-19T10:45:19.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:19.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:19.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 10 monclient: dump: 2024-08-19T10:45:19.491 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:19.491 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:19.491 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:19.491 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:19.491 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:19.491 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:19.492 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:19.492 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:19.492 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:19.492 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:19.492 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:19.492 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:19.492 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:19.493 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:19.493 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:19.493 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:19.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:19.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:19.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:19.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 5 monclient: authenticate success, global_id 6359 2024-08-19T10:45:19.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:19.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 1 -- 172.21.15.190:0/566822072 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1280aa810 con 0x7fe138139850 2024-08-19T10:45:19.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:19.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 1 -- 172.21.15.190:0/566822072 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe1280a93e0 con 0x7fe138139850 2024-08-19T10:45:19.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:19.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 4 set_mon_vals no callback set 2024-08-19T10:45:19.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:19.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:19.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1367fc640 10 monclient: dump: 2024-08-19T10:45:19.495 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:19.495 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:19.495 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:19.495 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:19.496 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:19.496 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:19.496 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:19.496 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:19.496 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:19.496 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:19.496 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:19.497 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:19.497 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:19.497 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:19.497 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:19.497 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:19.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient: shutdown 2024-08-19T10:45:19.497 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:19.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 -- 172.21.15.190:0/566822072 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe138139850 msgr2=0x7fe138071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:19.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 --2- 172.21.15.190:0/566822072 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe138139850 0x7fe138071fe0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe1280098a0 tx=0x7fe12801c3e0 comp rx=0 tx=0).stop 2024-08-19T10:45:19.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 -- 172.21.15.190:0/566822072 shutdown_connections 2024-08-19T10:45:19.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 --2- 172.21.15.190:0/566822072 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe138139850 0x7fe138071fe0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:19.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 -- 172.21.15.190:0/566822072 >> 172.21.15.190:0/566822072 conn(0x7fe1380aa2b0 msgr2=0x7fe1380a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:19.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 -- 172.21.15.190:0/566822072 shutdown_connections 2024-08-19T10:45:19.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 -- 172.21.15.190:0/566822072 wait complete. 2024-08-19T10:45:19.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient: build_initial_monmap 2024-08-19T10:45:19.498 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient: monmap: 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:19.499 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 Processor -- start 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 -- start start 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient: init 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient: _add_conns ranks=[3,4,8,0,7,1,6,2,5] 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe138139850 0x7fe1380a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient(hunting): picked mon.noname-g con 0x7fe138139850 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe138139c30 0x7fe13809ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient(hunting): picked mon.noname-c con 0x7fe138139c30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe13813a010 0x7fe13809cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:19.500 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient(hunting): picked mon.noname-e con 0x7fe13813a010 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe13806e980 con 0x7fe138139850 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fe13806e800 con 0x7fe138139c30 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fe13806b6c0 con 0x7fe13813a010 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe13dd8b640 10 monclient(hunting): authenticate will time out at 1365.058436s 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe138139850 0x7fe1380a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe138139850 0x7fe1380a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:57768/0 (socket says 172.21.15.190:57768) 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 1 -- 172.21.15.190:0/1733415289 learned_addr learned my addr 172.21.15.190:0/1733415289 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 10 monclient(hunting): get_auth_request con 0x7fe138139850 auth_method 0 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:19.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe137fff640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe13813a010 0x7fe13809cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe137fff640 10 monclient(hunting): get_auth_request con 0x7fe13813a010 auth_method 0 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe137fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe137fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe137fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe136ffd640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe138139c30 0x7fe13809ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 10 monclient(hunting): handle_auth_done global_id 34770 payload 824 2024-08-19T10:45:19.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 1 -- 172.21.15.190:0/1733415289 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe13813a010 msgr2=0x7fe13809cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:19.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe13813a010 0x7fe13809cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:19.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 1 -- 172.21.15.190:0/1733415289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe138139c30 msgr2=0x7fe13809ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:19.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe138139c30 0x7fe13809ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:19.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:19.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 1 monclient: found mon.noname-g 2024-08-19T10:45:19.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:19.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:19.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe1377fe640 1 -- 172.21.15.190:0/1733415289 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1381335c0 con 0x7fe138139850 2024-08-19T10:45:19.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe136ffd640 10 monclient: get_auth_request con 0x7fe138139c30 auth_method 0 2024-08-19T10:45:19.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.485+0000 7fe136ffd640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe138139c30 0x7fe13809ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:19.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe1377fe640 1 -- 172.21.15.190:0/1733415289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe138139850 msgr2=0x7fe1380a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:19.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe1377fe640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe138139850 0x7fe1380a0720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe128000c10 tx=0x7fe1280191e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:19.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 1 -- 172.21.15.190:0/1733415289 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe1280b9070 con 0x7fe138139850 2024-08-19T10:45:19.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:19.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:19.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 10 monclient: dump: 2024-08-19T10:45:19.506 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:19.506 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:19.506 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:19.507 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 10 monclient: _finish_auth 0 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 1 -- 172.21.15.190:0/1733415289 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1280044b0 con 0x7fe138139850 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe13dd8b640 5 monclient: authenticate success, global_id 34770 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe13dd8b640 10 monclient: _renew_subs 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe13dd8b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe13dd8b640 1 -- 172.21.15.190:0/1733415289 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13809f450 con 0x7fe138139850 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 1 -- 172.21.15.190:0/1733415289 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe1280b0040 con 0x7fe138139850 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:19.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 10 monclient: dump: 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:19.509 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe13dd8b640 10 monclient: _renew_subs 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe135ffb640 4 set_mon_vals no callback set 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe13dd8b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe13dd8b640 1 -- 172.21.15.190:0/1733415289 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1380a55b0 con 0x7fe138139850 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe13dd8b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe13dd8b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe13dd8b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe13dd8b640 1 -- 172.21.15.190:0/1733415289 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe138071fe0 con 0x7fe138139850 2024-08-19T10:45:19.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 1 -- 172.21.15.190:0/1733415289 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe128004040 con 0x7fe138139850 2024-08-19T10:45:19.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe12403bdf0 0x7fe12403e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:19.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe11ffff640 1 -- 172.21.15.190:0/1733415289 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(237..237 src has 1..237) ==== 13094+0+0 (secure 0 0 0) 0x7fe1280e9c80 con 0x7fe138139850 2024-08-19T10:45:19.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe136ffd640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe12403bdf0 0x7fe12403e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:19.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.489+0000 7fe136ffd640 10 monclient: get_auth_request con 0x7fe12403bdf0 auth_method 0 2024-08-19T10:45:19.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.493+0000 7fe136ffd640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe12403bdf0 0x7fe12403e2b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe138137510 tx=0x7fe12c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:19.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.493+0000 7fe11ffff640 1 -- 172.21.15.190:0/1733415289 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe1381335c0 con 0x7fe138139850 2024-08-19T10:45:19.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.493+0000 7fe11ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:19.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.493+0000 7fe11ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:19.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.669+0000 7fe13dd8b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h10"}] 2024-08-19T10:45:19.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.669+0000 7fe13dd8b640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h10"}] 2024-08-19T10:45:19.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.669+0000 7fe13dd8b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:19.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:19.669+0000 7fe13dd8b640 1 -- 172.21.15.190:0/1733415289 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h10"} v 0) -- 0x7fe13806fcc0 con 0x7fe138139850 2024-08-19T10:45:21.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.289+0000 7fe11ffff640 1 -- 172.21.15.190:0/1733415289 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h10"}]=0 removed item id -46 name 'h10' from crush map v238) ==== 120+0+0 (secure 0 0 0) 0x7fe128016150 con 0x7fe138139850 2024-08-19T10:45:21.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.289+0000 7fe11ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h10"}] 2024-08-19T10:45:21.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.289+0000 7fe11ffff640 10 monclient: _finish_command 2 = system:0 removed item id -46 name 'h10' from crush map 2024-08-19T10:45:21.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.417+0000 7fe13dd8b640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h10"}] 2024-08-19T10:45:21.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.417+0000 7fe13dd8b640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h10"}] 2024-08-19T10:45:21.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.417+0000 7fe13dd8b640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:21.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.417+0000 7fe13dd8b640 1 -- 172.21.15.190:0/1733415289 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h10"} v 0) -- 0x7fe138138580 con 0x7fe138139850 2024-08-19T10:45:21.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe11ffff640 1 -- 172.21.15.190:0/1733415289 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h10"}]=0 device 'h10' does not appear in the crush map v238) ==== 120+0+0 (secure 0 0 0) 0x7fe1280e65f0 con 0x7fe138139850 2024-08-19T10:45:21.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe11ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h10"}] 2024-08-19T10:45:21.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe11ffff640 10 monclient: _finish_command 3 = system:0 device 'h10' does not appear in the crush map 2024-08-19T10:45:21.422 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h10' does not appear in the crush map 2024-08-19T10:45:21.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 -- 172.21.15.190:0/1733415289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe12403bdf0 msgr2=0x7fe12403e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:21.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe12403bdf0 0x7fe12403e2b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe138137510 tx=0x7fe12c008040 comp rx=0 tx=0).stop 2024-08-19T10:45:21.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 10 monclient: shutdown 2024-08-19T10:45:21.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:21.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 -- 172.21.15.190:0/1733415289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe138139850 msgr2=0x7fe1380a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:21.424 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe138139850 0x7fe1380a0720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe128000c10 tx=0x7fe1280191e0 comp rx=0 tx=0).stop 2024-08-19T10:45:21.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 -- 172.21.15.190:0/1733415289 shutdown_connections 2024-08-19T10:45:21.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe12403bdf0 0x7fe12403e2b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:21.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe13813a010 0x7fe13809cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:21.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe138139c30 0x7fe13809ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:21.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 --2- 172.21.15.190:0/1733415289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe138139850 0x7fe1380a0720 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:21.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 -- 172.21.15.190:0/1733415289 >> 172.21.15.190:0/1733415289 conn(0x7fe1380aa2b0 msgr2=0x7fe1380a85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:21.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 -- 172.21.15.190:0/1733415289 shutdown_connections 2024-08-19T10:45:21.425 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.421+0000 7fe13dd8b640 1 -- 172.21.15.190:0/1733415289 wait complete. 2024-08-19T10:45:21.439 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:21.439 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h11 2024-08-19T10:45:21.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 10 monclient: get_monmap_and_config 2024-08-19T10:45:21.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 10 monclient: build_initial_monmap 2024-08-19T10:45:21.551 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 10 monclient: monmap: 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:21.554134+0000 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:21.554134+0000 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:21.552 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:21.553 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:21.553 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:21.553 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:21.553 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:21.553 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:21.555 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 1 Processor -- start 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 1 -- start start 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 10 monclient: init 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 10 monclient: _add_conns ranks=[7,4,0,1,10,2,3,5,9,8,6] 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8e74139980 0x7f8e74139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 10 monclient(hunting): picked mon.noname-i con 0x7f8e74139980 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a2a0 0x7f8e740a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.549+0000 7f8e7ad0a640 10 monclient(hunting): picked mon.noname-c con 0x7f8e7413a2a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e7ad0a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8e740a5c70 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e7ad0a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e740a8470 con 0x7f8e740a5c70 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e7ad0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:21.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e7ad0a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f8e7409ca60 con 0x7f8e7413a2a0 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e7ad0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e7ad0a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8e740a3640 con 0x7f8e74139980 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e7ad0a640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e7ad0a640 10 monclient(hunting): authenticate will time out at 1367.124057s 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8e74139980 0x7f8e74139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8e74139980 0x7f8e74139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:53880/0 (socket says 172.21.15.190:53880) 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 1 -- 172.21.15.190:0/984870429 learned_addr learned my addr 172.21.15.190:0/984870429 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 10 monclient(hunting): get_auth_request con 0x7f8e74139980 auth_method 0 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:21.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e73fff640 1 --2- 172.21.15.190:0/984870429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a2a0 0x7f8e740a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 10 monclient(hunting): handle_auth_done global_id 15712 payload 274 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 1 -- 172.21.15.190:0/984870429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a2a0 msgr2=0x7f8e740a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 1 --2- 172.21.15.190:0/984870429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a2a0 0x7f8e740a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 1 -- 172.21.15.190:0/984870429 >> v1:172.21.15.89:6789/0 conn(0x7f8e740a5c70 legacy=0x7f8e740a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 10 monclient: _finish_hunting 0 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 1 monclient: found mon.noname-i 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e78aa6640 1 -- 172.21.15.190:0/984870429 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e740a3850 con 0x7f8e74139980 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e73fff640 10 monclient: get_auth_request con 0x7f8e7413a2a0 auth_method 0 2024-08-19T10:45:21.558 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.553+0000 7f8e73fff640 1 --2- 172.21.15.190:0/984870429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a2a0 0x7f8e740a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:21.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 1 -- 172.21.15.190:0/984870429 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8e74139980 msgr2=0x7f8e74139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 1 --2- 172.21.15.190:0/984870429 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8e74139980 0x7f8e74139d60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8e640098a0 tx=0x7f8e6401c410 comp rx=0 tx=0).ready entity=mon.7 client_cookie=ba79339bec748b8f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 1 -- 172.21.15.190:0/984870429 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e640b0070 con 0x7f8e74139980 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 10 monclient: dump: 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:21.562 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:21.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 5 monclient: authenticate success, global_id 15712 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 1 -- 172.21.15.190:0/984870429 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e640aa810 con 0x7f8e74139980 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 4 set_mon_vals no callback set 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 1 -- 172.21.15.190:0/984870429 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e640aae30 con 0x7f8e74139980 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e737fe640 10 monclient: dump: 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:21.564 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:21.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient: shutdown 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/984870429 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8e74139980 msgr2=0x7f8e74139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 --2- 172.21.15.190:0/984870429 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8e74139980 0x7f8e74139d60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8e640098a0 tx=0x7f8e6401c410 comp rx=0 tx=0).stop 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/984870429 shutdown_connections 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 --2- 172.21.15.190:0/984870429 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a2a0 0x7f8e740a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 --2- 172.21.15.190:0/984870429 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8e74139980 0x7f8e74139d60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/984870429 >> 172.21.15.190:0/984870429 conn(0x7f8e740abde0 msgr2=0x7f8e740ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/984870429 shutdown_connections 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/984870429 wait complete. 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient: build_initial_monmap 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient: monmap: 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 Processor -- start 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 -- start start 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient: init 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient: _add_conns ranks=[3,0,4,8,1,5,7,6,2] 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e740a5c70 0x7f8e74133890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient(hunting): picked mon.noname-g con 0x7f8e740a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e74139980 0x7f8e74133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient(hunting): picked mon.noname-a con 0x7f8e74139980 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a690 0x7f8e7413ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:21.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient(hunting): picked mon.noname-c con 0x7f8e7413a690 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74144280 con 0x7f8e74139980 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8e74144100 con 0x7f8e740a5c70 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f8e74144400 con 0x7f8e7413a690 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e7ad0a640 10 monclient(hunting): authenticate will time out at 1367.131200s 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e740a5c70 0x7f8e74133890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e740a5c70 0x7f8e74133890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:57782/0 (socket says 172.21.15.190:57782) 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 1 -- 172.21.15.190:0/52522508 learned_addr learned my addr 172.21.15.190:0/52522508 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 10 monclient(hunting): get_auth_request con 0x7f8e740a5c70 auth_method 0 2024-08-19T10:45:21.569 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e73fff640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e74139980 0x7f8e74133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e792a7640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a690 0x7f8e7413ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 10 monclient(hunting): handle_auth_done global_id 34779 payload 824 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 1 -- 172.21.15.190:0/52522508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a690 msgr2=0x7f8e7413ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a690 0x7f8e7413ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 1 -- 172.21.15.190:0/52522508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e74139980 msgr2=0x7f8e74133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e74139980 0x7f8e74133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 10 monclient: _finish_hunting 0 2024-08-19T10:45:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 1 monclient: found mon.noname-g 2024-08-19T10:45:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e78aa6640 1 -- 172.21.15.190:0/52522508 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7413d680 con 0x7f8e740a5c70 2024-08-19T10:45:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e792a7640 10 monclient: get_auth_request con 0x7f8e7413a690 auth_method 0 2024-08-19T10:45:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.557+0000 7f8e792a7640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a690 0x7f8e7413ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.561+0000 7f8e78aa6640 1 -- 172.21.15.190:0/52522508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e740a5c70 msgr2=0x7f8e74133890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.561+0000 7f8e78aa6640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e740a5c70 0x7f8e74133890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009870 tx=0x7f8e640191b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.561+0000 7f8e717fa640 1 -- 172.21.15.190:0/52522508 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:45:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.561+0000 7f8e717fa640 1 -- 172.21.15.190:0/52522508 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e640b9070 con 0x7f8e740a5c70 2024-08-19T10:45:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.561+0000 7f8e717fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 10 monclient: dump: 2024-08-19T10:45:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:21.574 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 10 monclient: _finish_auth 0 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 10 monclient: _check_auth_tickets 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 1 -- 172.21.15.190:0/52522508 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e64004250 con 0x7f8e740a5c70 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e7ad0a640 5 monclient: authenticate success, global_id 34779 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e7ad0a640 10 monclient: _renew_subs 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e7ad0a640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/52522508 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e740ac4a0 con 0x7f8e740a5c70 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 1 -- 172.21.15.190:0/52522508 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8e640b0040 con 0x7f8e740a5c70 2024-08-19T10:45:21.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 10 monclient: dump: 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:21.576 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e72ffd640 4 set_mon_vals no callback set 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e7ad0a640 10 monclient: _renew_subs 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e7ad0a640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/52522508 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7413d3a0 con 0x7f8e740a5c70 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e7ad0a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e7ad0a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e7ad0a640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/52522508 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e740ac6d0 con 0x7f8e740a5c70 2024-08-19T10:45:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 1 -- 172.21.15.190:0/52522508 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8e64003620 con 0x7f8e740a5c70 2024-08-19T10:45:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8e6003bdf0 0x7f8e6003e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e717fa640 1 -- 172.21.15.190:0/52522508 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(238..238 src has 150..238) ==== 12881+0+0 (secure 0 0 0) 0x7f8e640ea470 con 0x7f8e740a5c70 2024-08-19T10:45:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e73fff640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8e6003bdf0 0x7f8e6003e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.565+0000 7f8e73fff640 10 monclient: get_auth_request con 0x7f8e6003bdf0 auth_method 0 2024-08-19T10:45:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.569+0000 7f8e73fff640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8e6003bdf0 0x7f8e6003e2b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8e68006fd0 tx=0x7f8e68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.569+0000 7f8e717fa640 1 -- 172.21.15.190:0/52522508 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8e740ac6d0 con 0x7f8e740a5c70 2024-08-19T10:45:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.569+0000 7f8e717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.569+0000 7f8e717fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:21.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.745+0000 7f8e7ad0a640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h11"}] 2024-08-19T10:45:21.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.745+0000 7f8e7ad0a640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h11"}] 2024-08-19T10:45:21.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.745+0000 7f8e7ad0a640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:21.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:21.745+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/52522508 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h11"} v 0) -- 0x7f8e740a97e0 con 0x7f8e740a5c70 2024-08-19T10:45:22.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.297+0000 7f8e717fa640 1 -- 172.21.15.190:0/52522508 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h11"}]=0 removed item id -49 name 'h11' from crush map v239) ==== 120+0+0 (secure 0 0 0) 0x7f8e640c2020 con 0x7f8e740a5c70 2024-08-19T10:45:22.300 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.297+0000 7f8e717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h11"}] 2024-08-19T10:45:22.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.297+0000 7f8e717fa640 10 monclient: _finish_command 2 = system:0 removed item id -49 name 'h11' from crush map 2024-08-19T10:45:22.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.409+0000 7f8e7ad0a640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h11"}] 2024-08-19T10:45:22.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.409+0000 7f8e7ad0a640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h11"}] 2024-08-19T10:45:22.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.409+0000 7f8e7ad0a640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:22.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.409+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/52522508 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h11"} v 0) -- 0x7f8e74139d60 con 0x7f8e740a5c70 2024-08-19T10:45:22.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.413+0000 7f8e717fa640 1 -- 172.21.15.190:0/52522508 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h11"}]=0 device 'h11' does not appear in the crush map v239) ==== 120+0+0 (secure 0 0 0) 0x7f8e640d8090 con 0x7f8e740a5c70 2024-08-19T10:45:22.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.413+0000 7f8e717fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h11"}] 2024-08-19T10:45:22.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.413+0000 7f8e717fa640 10 monclient: _finish_command 3 = system:0 device 'h11' does not appear in the crush map 2024-08-19T10:45:22.415 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h11' does not appear in the crush map 2024-08-19T10:45:22.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/52522508 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8e6003bdf0 msgr2=0x7f8e6003e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:22.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8e6003bdf0 0x7f8e6003e2b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8e68006fd0 tx=0x7f8e68008040 comp rx=0 tx=0).stop 2024-08-19T10:45:22.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 10 monclient: shutdown 2024-08-19T10:45:22.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:22.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/52522508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e740a5c70 msgr2=0x7f8e74133890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:22.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e740a5c70 0x7f8e74133890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009870 tx=0x7f8e640191b0 comp rx=0 tx=0).stop 2024-08-19T10:45:22.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/52522508 shutdown_connections 2024-08-19T10:45:22.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8e6003bdf0 0x7f8e6003e2b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:22.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8e7413a690 0x7f8e7413ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:22.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e74139980 0x7f8e74133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:22.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 --2- 172.21.15.190:0/52522508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e740a5c70 0x7f8e74133890 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:22.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/52522508 >> 172.21.15.190:0/52522508 conn(0x7f8e740abde0 msgr2=0x7f8e74135170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:22.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/52522508 shutdown_connections 2024-08-19T10:45:22.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.417+0000 7f8e7ad0a640 1 -- 172.21.15.190:0/52522508 wait complete. 2024-08-19T10:45:22.433 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:22.433 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h12 2024-08-19T10:45:22.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient: get_monmap_and_config 2024-08-19T10:45:22.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient: build_initial_monmap 2024-08-19T10:45:22.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient: monmap: 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:22.557066+0000 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:22.557066+0000 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:22.555 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:22.556 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:22.556 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:22.556 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:22.556 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:22.556 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:22.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 1 Processor -- start 2024-08-19T10:45:22.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 1 -- start start 2024-08-19T10:45:22.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient: init 2024-08-19T10:45:22.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:22.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient: _add_conns ranks=[0,1,3,6,10,8,9,4,5,2,7] 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd8380a6850 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd8380a6c30 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 0x7fd838141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient(hunting): picked mon.noname-g con 0x7fd83813fbc0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8380a5f00 con 0x7fd8380a6850 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8380a3850 con 0x7fd8380a6c30 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd8380a3640 con 0x7fd83813fbc0 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd83e0bb640 10 monclient(hunting): authenticate will time out at 1368.127134s 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 0x7fd838141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:22.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd836ffd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd8380a6c30 0x7fd8380a7010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:59356/0 (socket says 172.21.15.190:59356) 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd836ffd640 1 -- 172.21.15.190:0/971536600 learned_addr learned my addr 172.21.15.190:0/971536600 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 10 monclient(hunting): get_auth_request con 0x7fd83813fbc0 auth_method 0 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 10 monclient(hunting): handle_auth_done global_id 34788 payload 274 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 1 -- 172.21.15.190:0/971536600 >> v1:172.21.15.190:6789/0 conn(0x7fd8380a6c30 legacy=0x7fd8380a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 1 -- 172.21.15.190:0/971536600 >> v1:172.21.15.89:6789/0 conn(0x7fd8380a6850 legacy=0x7fd8380a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 1 monclient: found mon.noname-g 2024-08-19T10:45:22.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:22.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:22.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.553+0000 7fd837fff640 1 -- 172.21.15.190:0/971536600 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8380a46e0 con 0x7fd83813fbc0 2024-08-19T10:45:22.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.557+0000 7fd837fff640 1 -- 172.21.15.190:0/971536600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 msgr2=0x7fd838141fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:22.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.557+0000 7fd837fff640 1 --2- 172.21.15.190:0/971536600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 0x7fd838141fb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd830009dc0 tx=0x7fd83001c7f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=519ec079546329f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:22.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 1 -- 172.21.15.190:0/971536600 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8300ac070 con 0x7fd83813fbc0 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 10 monclient: dump: 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:22.566 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 1 -- 172.21.15.190:0/971536600 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8300a6990 con 0x7fd83813fbc0 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 5 monclient: authenticate success, global_id 34788 2024-08-19T10:45:22.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 4 set_mon_vals no callback set 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 1 -- 172.21.15.190:0/971536600 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8300a7420 con 0x7fd83813fbc0 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8367fc640 10 monclient: dump: 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:22.568 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:22.569 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:22.569 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:22.569 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:22.569 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:22.569 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:22.569 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:22.569 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:22.569 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient: shutdown 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 -- 172.21.15.190:0/971536600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 msgr2=0x7fd838141fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 --2- 172.21.15.190:0/971536600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 0x7fd838141fb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd830009dc0 tx=0x7fd83001c7f0 comp rx=0 tx=0).stop 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 -- 172.21.15.190:0/971536600 shutdown_connections 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 --2- 172.21.15.190:0/971536600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 0x7fd838141fb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 -- 172.21.15.190:0/971536600 >> 172.21.15.190:0/971536600 conn(0x7fd8380abe50 msgr2=0x7fd8380ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 -- 172.21.15.190:0/971536600 shutdown_connections 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 -- 172.21.15.190:0/971536600 wait complete. 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient: build_initial_monmap 2024-08-19T10:45:22.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient: monmap: 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:22.571 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 Processor -- start 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 -- start start 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient: init 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient: _add_conns ranks=[7,0,3,5,2,8,6,1,4] 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8380a6850 0x7fd838135a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient(hunting): picked mon.noname-i con 0x7fd8380a6850 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8380a6c30 0x7fd838135f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient(hunting): picked mon.noname-a con 0x7fd8380a6c30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 0x7fd83809cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:22.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient(hunting): picked mon.noname-g con 0x7fd83813fbc0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd83806b6c0 con 0x7fd8380a6c30 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd838144d10 con 0x7fd83813fbc0 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd838144e90 con 0x7fd8380a6850 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd83e0bb640 10 monclient(hunting): authenticate will time out at 1368.135091s 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 0x7fd83809cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 0x7fd83809cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:57800/0 (socket says 172.21.15.190:57800) 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 1 -- 172.21.15.190:0/238830033 learned_addr learned my addr 172.21.15.190:0/238830033 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 10 monclient(hunting): get_auth_request con 0x7fd83813fbc0 auth_method 0 2024-08-19T10:45:22.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8377fe640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8380a6850 0x7fd838135a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8377fe640 10 monclient(hunting): get_auth_request con 0x7fd8380a6850 auth_method 0 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8377fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8377fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:22.574 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd8377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd836ffd640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8380a6c30 0x7fd838135f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd836ffd640 10 monclient(hunting): get_auth_request con 0x7fd8380a6c30 auth_method 0 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd836ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd836ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd836ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 10 monclient(hunting): handle_auth_done global_id 34797 payload 824 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 1 -- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8380a6850 msgr2=0x7fd838135a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8380a6850 0x7fd838135a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 1 -- 172.21.15.190:0/238830033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8380a6c30 msgr2=0x7fd838135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8380a6c30 0x7fd838135f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 1 monclient: found mon.noname-g 2024-08-19T10:45:22.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:22.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:22.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.561+0000 7fd837fff640 1 -- 172.21.15.190:0/238830033 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8380a9330 con 0x7fd83813fbc0 2024-08-19T10:45:22.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd837fff640 1 -- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 msgr2=0x7fd83809cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:22.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd837fff640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 0x7fd83809cb20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd83001c620 tx=0x7fd8300195f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:22.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 1 -- 172.21.15.190:0/238830033 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8300ac070 con 0x7fd83813fbc0 2024-08-19T10:45:22.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:22.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:22.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 10 monclient: dump: 2024-08-19T10:45:22.578 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:22.578 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:22.578 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:22.579 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 10 monclient: _finish_auth 0 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd83e0bb640 5 monclient: authenticate success, global_id 34797 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 1 -- 172.21.15.190:0/238830033 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8300a6790 con 0x7fd83813fbc0 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 1 -- 172.21.15.190:0/238830033 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd830003bb0 con 0x7fd83813fbc0 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd83e0bb640 10 monclient: _renew_subs 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd83e0bb640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd83e0bb640 1 -- 172.21.15.190:0/238830033 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83809f580 con 0x7fd83813fbc0 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:22.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd835ffb640 4 set_mon_vals no callback set 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 10 monclient: dump: 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:22.581 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd83e0bb640 10 monclient: _renew_subs 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd83e0bb640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd83e0bb640 1 -- 172.21.15.190:0/238830033 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd83809f2a0 con 0x7fd83813fbc0 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 1 -- 172.21.15.190:0/238830033 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd8300af070 con 0x7fd83813fbc0 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd83e0bb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd83e0bb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd83e0bb640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd83e0bb640 1 -- 172.21.15.190:0/238830033 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8380a57f0 con 0x7fd83813fbc0 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd82403be30 0x7fd82403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:22.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd81ffff640 1 -- 172.21.15.190:0/238830033 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(239..239 src has 150..239) ==== 12668+0+0 (secure 0 0 0) 0x7fd8300b7070 con 0x7fd83813fbc0 2024-08-19T10:45:22.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd8377fe640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd82403be30 0x7fd82403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:22.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.565+0000 7fd8377fe640 10 monclient: get_auth_request con 0x7fd82403be30 auth_method 0 2024-08-19T10:45:22.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.569+0000 7fd8377fe640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd82403be30 0x7fd82403e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd8280098e0 tx=0x7fd828009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:22.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.573+0000 7fd81ffff640 1 -- 172.21.15.190:0/238830033 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd8380a9330 con 0x7fd83813fbc0 2024-08-19T10:45:22.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.573+0000 7fd81ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:22.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.573+0000 7fd81ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:22.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.745+0000 7fd83e0bb640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h12"}] 2024-08-19T10:45:22.748 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.745+0000 7fd83e0bb640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h12"}] 2024-08-19T10:45:22.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.745+0000 7fd83e0bb640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:22.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:22.745+0000 7fd83e0bb640 1 -- 172.21.15.190:0/238830033 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h12"} v 0) -- 0x7fd8380ac750 con 0x7fd83813fbc0 2024-08-19T10:45:23.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.329+0000 7fd81ffff640 1 -- 172.21.15.190:0/238830033 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h12"}]=0 removed item id -52 name 'h12' from crush map v240) ==== 120+0+0 (secure 0 0 0) 0x7fd830016560 con 0x7fd83813fbc0 2024-08-19T10:45:23.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.329+0000 7fd81ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h12"}] 2024-08-19T10:45:23.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.329+0000 7fd81ffff640 10 monclient: _finish_command 2 = system:0 removed item id -52 name 'h12' from crush map 2024-08-19T10:45:23.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.437+0000 7fd83e0bb640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h12"}] 2024-08-19T10:45:23.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.437+0000 7fd83e0bb640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h12"}] 2024-08-19T10:45:23.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.437+0000 7fd83e0bb640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:23.440 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.437+0000 7fd83e0bb640 1 -- 172.21.15.190:0/238830033 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h12"} v 0) -- 0x7fd83809e060 con 0x7fd83813fbc0 2024-08-19T10:45:23.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd81ffff640 1 -- 172.21.15.190:0/238830033 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h12"}]=0 device 'h12' does not appear in the crush map v240) ==== 120+0+0 (secure 0 0 0) 0x7fd830003e10 con 0x7fd83813fbc0 2024-08-19T10:45:23.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd81ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h12"}] 2024-08-19T10:45:23.442 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd81ffff640 10 monclient: _finish_command 3 = system:0 device 'h12' does not appear in the crush map 2024-08-19T10:45:23.442 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h12' does not appear in the crush map 2024-08-19T10:45:23.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 -- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd82403be30 msgr2=0x7fd82403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:23.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd82403be30 0x7fd82403e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd8280098e0 tx=0x7fd828009060 comp rx=0 tx=0).stop 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 10 monclient: shutdown 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 -- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 msgr2=0x7fd83809cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 0x7fd83809cb20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd83001c620 tx=0x7fd8300195f0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 -- 172.21.15.190:0/238830033 shutdown_connections 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd82403be30 0x7fd82403e2f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8380a6c30 0x7fd838135f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd83813fbc0 0x7fd83809cb20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 --2- 172.21.15.190:0/238830033 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd8380a6850 0x7fd838135a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 -- 172.21.15.190:0/238830033 >> 172.21.15.190:0/238830033 conn(0x7fd8380abe50 msgr2=0x7fd8381412c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 -- 172.21.15.190:0/238830033 shutdown_connections 2024-08-19T10:45:23.445 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.441+0000 7fd83e0bb640 1 -- 172.21.15.190:0/238830033 wait complete. 2024-08-19T10:45:23.459 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:23.459 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h13 2024-08-19T10:45:23.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient: get_monmap_and_config 2024-08-19T10:45:23.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient: build_initial_monmap 2024-08-19T10:45:23.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient: monmap: 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:23.581043+0000 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:23.581043+0000 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:23.579 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:23.580 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:23.580 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:23.580 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:23.580 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:23.580 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:23.580 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:23.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 1 Processor -- start 2024-08-19T10:45:23.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 1 -- start start 2024-08-19T10:45:23.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient: init 2024-08-19T10:45:23.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:23.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient: _add_conns ranks=[3,6,8,5,4,0,10,2,7,1,9] 2024-08-19T10:45:23.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bb00a6740 0x7f6bb00a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:23.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient(hunting): picked mon.noname-g con 0x7f6bb00a6740 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:23.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bb00a5bc0 0x7f6bb00a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient(hunting): picked mon.noname-d con 0x7f6bb00a5bc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6bb00a7060 0x7f6bb0139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient(hunting): picked mon.noname-e con 0x7f6bb00a7060 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb00a4c20 con 0x7f6bb00a6740 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6bb00a3530 con 0x7f6bb00a5bc0 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6bb00a3740 con 0x7f6bb00a7060 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb7168640 10 monclient(hunting): authenticate will time out at 1369.150851s 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bb00a6740 0x7f6bb00a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:23.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bb00a6740 0x7f6bb00a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:57806/0 (socket says 172.21.15.190:57806) 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 1 -- 172.21.15.190:0/184966797 learned_addr learned my addr 172.21.15.190:0/184966797 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 10 monclient(hunting): get_auth_request con 0x7f6bb00a6740 auth_method 0 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6baffff640 1 --2- 172.21.15.190:0/184966797 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bb00a5bc0 0x7f6bb00a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb5705640 1 --2- 172.21.15.190:0/184966797 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6bb00a7060 0x7f6bb0139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 10 monclient(hunting): handle_auth_done global_id 34806 payload 274 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 1 -- 172.21.15.190:0/184966797 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6bb00a7060 msgr2=0x7f6bb0139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 1 --2- 172.21.15.190:0/184966797 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6bb00a7060 0x7f6bb0139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 1 -- 172.21.15.190:0/184966797 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bb00a5bc0 msgr2=0x7f6bb00a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:23.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 1 --2- 172.21.15.190:0/184966797 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bb00a5bc0 0x7f6bb00a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 10 monclient: _finish_hunting 0 2024-08-19T10:45:23.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 1 monclient: found mon.noname-g 2024-08-19T10:45:23.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:23.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:23.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb4f04640 1 -- 172.21.15.190:0/184966797 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb00a3240 con 0x7f6bb00a6740 2024-08-19T10:45:23.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb5705640 10 monclient: get_auth_request con 0x7f6bb00a7060 auth_method 0 2024-08-19T10:45:23.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6baffff640 10 monclient: get_auth_request con 0x7f6bb00a5bc0 auth_method 0 2024-08-19T10:45:23.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6bb5705640 1 --2- 172.21.15.190:0/184966797 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6bb00a7060 0x7f6bb0139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:23.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.577+0000 7f6baffff640 1 --2- 172.21.15.190:0/184966797 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bb00a5bc0 0x7f6bb00a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:23.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 1 -- 172.21.15.190:0/184966797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bb00a6740 msgr2=0x7f6bb00a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:23.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 1 --2- 172.21.15.190:0/184966797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bb00a6740 0x7f6bb00a5680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80098d0 tx=0x7f6ba801c440 comp rx=0 tx=0).ready entity=mon.3 client_cookie=abe5dc4c502f4f7b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:23.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 1 -- 172.21.15.190:0/184966797 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ba80b0070 con 0x7f6bb00a6740 2024-08-19T10:45:23.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:23.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:23.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 10 monclient: dump: 2024-08-19T10:45:23.587 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:23.587 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:23.588 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:23.589 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:23.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:23.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:23.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:23.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 1 -- 172.21.15.190:0/184966797 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba80aa810 con 0x7f6bb00a6740 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 5 monclient: authenticate success, global_id 34806 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 1 -- 172.21.15.190:0/184966797 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ba80a93e0 con 0x7f6bb00a6740 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 4 set_mon_vals no callback set 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6baf7fe640 10 monclient: dump: 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:23.590 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:23.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient: shutdown 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 -- 172.21.15.190:0/184966797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bb00a6740 msgr2=0x7f6bb00a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 --2- 172.21.15.190:0/184966797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bb00a6740 0x7f6bb00a5680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80098d0 tx=0x7f6ba801c440 comp rx=0 tx=0).stop 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 -- 172.21.15.190:0/184966797 shutdown_connections 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 --2- 172.21.15.190:0/184966797 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6bb00a7060 0x7f6bb0139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 --2- 172.21.15.190:0/184966797 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bb00a5bc0 0x7f6bb00a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 --2- 172.21.15.190:0/184966797 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bb00a6740 0x7f6bb00a5680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 -- 172.21.15.190:0/184966797 >> 172.21.15.190:0/184966797 conn(0x7f6bb00abd40 msgr2=0x7f6bb00ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 -- 172.21.15.190:0/184966797 shutdown_connections 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 -- 172.21.15.190:0/184966797 wait complete. 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient: build_initial_monmap 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient: monmap: 2024-08-19T10:45:23.592 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:23.593 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:23.594 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:23.594 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:23.594 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:23.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 Processor -- start 2024-08-19T10:45:23.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 -- start start 2024-08-19T10:45:23.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient: init 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient: _add_conns ranks=[7,8,4,0,3,5,6,1,2] 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bb00a6740 0x7f6bb01338f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient(hunting): picked mon.noname-i con 0x7f6bb00a6740 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6bb00a7060 0x7f6bb0133e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient(hunting): picked mon.noname-e con 0x7f6bb00a7060 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009cad0 0x7f6bb009cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient(hunting): picked mon.noname-c con 0x7f6bb009cad0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6bb0143450 con 0x7f6bb009cad0 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:23.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6bb01432d0 con 0x7f6bb00a6740 2024-08-19T10:45:23.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:23.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6bb01435d0 con 0x7f6bb00a7060 2024-08-19T10:45:23.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:23.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb7168640 10 monclient(hunting): authenticate will time out at 1369.159637s 2024-08-19T10:45:23.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bb00a6740 0x7f6bb01338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:23.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bb00a6740 0x7f6bb01338f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:53896/0 (socket says 172.21.15.190:53896) 2024-08-19T10:45:23.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 1 -- 172.21.15.190:0/1408238985 learned_addr learned my addr 172.21.15.190:0/1408238985 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:23.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 10 monclient(hunting): get_auth_request con 0x7f6bb00a6740 auth_method 0 2024-08-19T10:45:23.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:23.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:23.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.585+0000 7f6bb4f04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb5705640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009cad0 0x7f6bb009cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb5705640 10 monclient(hunting): get_auth_request con 0x7f6bb009cad0 auth_method 0 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb5705640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb5705640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb5705640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6baffff640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6bb00a7060 0x7f6bb0133e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 10 monclient(hunting): handle_auth_done global_id 15730 payload 824 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 1 -- 172.21.15.190:0/1408238985 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6bb00a7060 msgr2=0x7f6bb0133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6bb00a7060 0x7f6bb0133e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 1 -- 172.21.15.190:0/1408238985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009cad0 msgr2=0x7f6bb009cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:23.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009cad0 0x7f6bb009cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:23.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 10 monclient: _finish_hunting 0 2024-08-19T10:45:23.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 1 monclient: found mon.noname-i 2024-08-19T10:45:23.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:23.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:23.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 1 -- 172.21.15.190:0/1408238985 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb009f5b0 con 0x7f6bb00a6740 2024-08-19T10:45:23.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 1 -- 172.21.15.190:0/1408238985 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bb00a6740 msgr2=0x7f6bb01338f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:23.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb4f04640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bb00a6740 0x7f6bb01338f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80098d0 tx=0x7f6ba80191e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:23.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 1 -- 172.21.15.190:0/1408238985 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ba80b9070 con 0x7f6bb00a6740 2024-08-19T10:45:23.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:23.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 10 monclient: dump: 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:23.603 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 10 monclient: _finish_auth 0 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 1 -- 172.21.15.190:0/1408238985 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba8004250 con 0x7f6bb00a6740 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb7168640 5 monclient: authenticate success, global_id 15730 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb7168640 10 monclient: _renew_subs 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb7168640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb7168640 1 -- 172.21.15.190:0/1408238985 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb013f1a0 con 0x7f6bb00a6740 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 1 -- 172.21.15.190:0/1408238985 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ba80b0040 con 0x7f6bb00a6740 2024-08-19T10:45:23.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 10 monclient: dump: 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:23.605 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6baeffd640 4 set_mon_vals no callback set 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb7168640 10 monclient: _renew_subs 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb7168640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb7168640 1 -- 172.21.15.190:0/1408238985 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb00a2480 con 0x7f6bb00a6740 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb7168640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb7168640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb7168640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:23.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bb7168640 1 -- 172.21.15.190:0/1408238985 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb00a5680 con 0x7f6bb00a6740 2024-08-19T10:45:23.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 1 -- 172.21.15.190:0/1408238985 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6ba80bc060 con 0x7f6bb00a6740 2024-08-19T10:45:23.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.589+0000 7f6bad7fa640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b9003be30 0x7f6b9003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:23.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.593+0000 7f6bad7fa640 1 -- 172.21.15.190:0/1408238985 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(240..240 src has 150..240) ==== 12455+0+0 (secure 0 0 0) 0x7f6ba80e9cb0 con 0x7f6bb00a6740 2024-08-19T10:45:23.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.593+0000 7f6baffff640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b9003be30 0x7f6b9003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:23.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.593+0000 7f6baffff640 10 monclient: get_auth_request con 0x7f6b9003be30 auth_method 0 2024-08-19T10:45:23.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.593+0000 7f6baffff640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b9003be30 0x7f6b9003e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6b98006fd0 tx=0x7f6b98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:23.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.597+0000 7f6bad7fa640 1 -- 172.21.15.190:0/1408238985 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6ba8016150 con 0x7f6bb00a6740 2024-08-19T10:45:23.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.597+0000 7f6bad7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:23.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.597+0000 7f6bad7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:23.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.769+0000 7f6bb7168640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h13"}] 2024-08-19T10:45:23.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.769+0000 7f6bb7168640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h13"}] 2024-08-19T10:45:23.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.769+0000 7f6bb7168640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:23.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:23.769+0000 7f6bb7168640 1 -- 172.21.15.190:0/1408238985 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h13"} v 0) -- 0x7f6bb0134370 con 0x7f6bb00a6740 2024-08-19T10:45:24.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:24.977+0000 7f6bad7fa640 1 -- 172.21.15.190:0/1408238985 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h13"}]=0 removed item id -55 name 'h13' from crush map v241) ==== 120+0+0 (secure 0 0 0) 0x7f6ba80aa7e0 con 0x7f6bb00a6740 2024-08-19T10:45:24.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:24.977+0000 7f6bad7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h13"}] 2024-08-19T10:45:24.980 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:24.977+0000 7f6bad7fa640 10 monclient: _finish_command 2 = system:0 removed item id -55 name 'h13' from crush map 2024-08-19T10:45:25.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.089+0000 7f6bb7168640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h13"}] 2024-08-19T10:45:25.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.089+0000 7f6bb7168640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h13"}] 2024-08-19T10:45:25.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.089+0000 7f6bb7168640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:25.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.089+0000 7f6bb7168640 1 -- 172.21.15.190:0/1408238985 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h13"} v 0) -- 0x7f6bb00008e0 con 0x7f6bb00a6740 2024-08-19T10:45:25.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.089+0000 7f6bad7fa640 1 -- 172.21.15.190:0/1408238985 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h13"}]=0 device 'h13' does not appear in the crush map v241) ==== 120+0+0 (secure 0 0 0) 0x7f6ba80bc8b0 con 0x7f6bb00a6740 2024-08-19T10:45:25.091 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.089+0000 7f6bad7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h13"}] 2024-08-19T10:45:25.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.089+0000 7f6bad7fa640 10 monclient: _finish_command 3 = system:0 device 'h13' does not appear in the crush map 2024-08-19T10:45:25.092 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h13' does not appear in the crush map 2024-08-19T10:45:25.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 -- 172.21.15.190:0/1408238985 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b9003be30 msgr2=0x7f6b9003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:25.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b9003be30 0x7f6b9003e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6b98006fd0 tx=0x7f6b98008040 comp rx=0 tx=0).stop 2024-08-19T10:45:25.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 10 monclient: shutdown 2024-08-19T10:45:25.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:25.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 -- 172.21.15.190:0/1408238985 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bb00a6740 msgr2=0x7f6bb01338f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:25.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bb00a6740 0x7f6bb01338f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80098d0 tx=0x7f6ba80191e0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 -- 172.21.15.190:0/1408238985 shutdown_connections 2024-08-19T10:45:25.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6bb009cad0 0x7f6bb009cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b9003be30 0x7f6b9003e2f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6bb00a7060 0x7f6bb0133e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 --2- 172.21.15.190:0/1408238985 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bb00a6740 0x7f6bb01338f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 -- 172.21.15.190:0/1408238985 >> 172.21.15.190:0/1408238985 conn(0x7f6bb00abd40 msgr2=0x7f6bb01366d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:25.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 -- 172.21.15.190:0/1408238985 shutdown_connections 2024-08-19T10:45:25.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.093+0000 7f6bb7168640 1 -- 172.21.15.190:0/1408238985 wait complete. 2024-08-19T10:45:25.109 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:25.109 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h14 2024-08-19T10:45:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.225+0000 7f8cfc92f640 10 monclient: get_monmap_and_config 2024-08-19T10:45:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.225+0000 7f8cfc92f640 10 monclient: build_initial_monmap 2024-08-19T10:45:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.225+0000 7f8cfc92f640 10 monclient: monmap: 2024-08-19T10:45:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:25.230545+0000 2024-08-19T10:45:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:25.230545+0000 2024-08-19T10:45:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:25.228 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:25.229 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:25.229 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:25.231 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 1 Processor -- start 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 1 -- start start 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 10 monclient: init 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 10 monclient: _add_conns ranks=[3,2,6,4,5,9,7,0,10,8,1] 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf8139980 0x7f8cf8139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 10 monclient(hunting): picked mon.noname-g con 0x7f8cf8139980 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cf813a2a0 0x7f8cf80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 10 monclient(hunting): picked mon.noname-b con 0x7f8cf813a2a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8cf80a5c70 0x7f8cf80a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 10 monclient(hunting): picked mon.noname-d con 0x7f8cf80a5c70 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:25.232 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf809ca60 con 0x7f8cf813a2a0 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf80a3640 con 0x7f8cf8139980 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8cf80a8470 con 0x7f8cf80a5c70 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cfc92f640 10 monclient(hunting): authenticate will time out at 1370.800819s 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf8139980 0x7f8cf8139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cedd9b640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cf813a2a0 msgr2=0x7f8cf80a5730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf8139980 0x7f8cf8139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:57812/0 (socket says 172.21.15.190:57812) 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cedd9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cf813a2a0 0x7f8cf80a5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 1 -- 172.21.15.190:0/3035100529 learned_addr learned my addr 172.21.15.190:0/3035100529 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 10 monclient(hunting): get_auth_request con 0x7f8cf8139980 auth_method 0 2024-08-19T10:45:25.233 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf6d9d640 1 --2- 172.21.15.190:0/3035100529 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8cf80a5c70 0x7f8cf80a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf6d9d640 10 monclient(hunting): get_auth_request con 0x7f8cf80a5c70 auth_method 0 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 10 monclient(hunting): handle_auth_done global_id 34815 payload 274 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 1 -- 172.21.15.190:0/3035100529 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8cf80a5c70 msgr2=0x7f8cf80a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:25.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 1 --2- 172.21.15.190:0/3035100529 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8cf80a5c70 0x7f8cf80a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 1 -- 172.21.15.190:0/3035100529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cf813a2a0 msgr2=0x7f8cf80a5730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:25.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 1 --2- 172.21.15.190:0/3035100529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cf813a2a0 0x7f8cf80a5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 10 monclient: _finish_hunting 0 2024-08-19T10:45:25.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 1 monclient: found mon.noname-g 2024-08-19T10:45:25.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:25.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:25.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.229+0000 7f8cf659c640 1 -- 172.21.15.190:0/3035100529 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf80a3850 con 0x7f8cf8139980 2024-08-19T10:45:25.238 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf659c640 1 -- 172.21.15.190:0/3035100529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf8139980 msgr2=0x7f8cf8139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf659c640 1 --2- 172.21.15.190:0/3035100529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf8139980 0x7f8cf8139d60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80098d0 tx=0x7f8ce801c170 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c12e4a828785e99 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 1 -- 172.21.15.190:0/3035100529 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ce80b0070 con 0x7f8cf8139980 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 10 monclient: dump: 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:25.239 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 10 monclient: _finish_auth 0 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:25.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 5 monclient: authenticate success, global_id 34815 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 1 -- 172.21.15.190:0/3035100529 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce80aa810 con 0x7f8cf8139980 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 4 set_mon_vals no callback set 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 1 -- 172.21.15.190:0/3035100529 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ce80aae30 con 0x7f8cf8139980 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cf5d9b640 10 monclient: dump: 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:25.241 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:25.242 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 10 monclient: shutdown 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3035100529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf8139980 msgr2=0x7f8cf8139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 1 --2- 172.21.15.190:0/3035100529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf8139980 0x7f8cf8139d60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80098d0 tx=0x7f8ce801c170 comp rx=0 tx=0).stop 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3035100529 shutdown_connections 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 1 --2- 172.21.15.190:0/3035100529 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8cf80a5c70 0x7f8cf80a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 1 --2- 172.21.15.190:0/3035100529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8cf813a2a0 0x7f8cf80a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 1 --2- 172.21.15.190:0/3035100529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8cf8139980 0x7f8cf8139d60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3035100529 >> 172.21.15.190:0/3035100529 conn(0x7f8cf80abde0 msgr2=0x7f8cf80ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3035100529 shutdown_connections 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3035100529 wait complete. 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 10 monclient: build_initial_monmap 2024-08-19T10:45:25.243 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.233+0000 7f8cfc92f640 10 monclient: monmap: 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:25.244 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 1 Processor -- start 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 1 -- start start 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 10 monclient: init 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 10 monclient: _add_conns ranks=[6,0,8,5,3,1,2,4,7] 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8cf80a5c70 0x7f8cf8133540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 10 monclient(hunting): picked mon.noname-d con 0x7f8cf80a5c70 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cf8139980 0x7f8cf8133a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 10 monclient(hunting): picked mon.noname-a con 0x7f8cf8139980 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8cf8133fc0 0x7f8cf813a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:25.245 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 10 monclient(hunting): picked mon.noname-e con 0x7f8cf8133fc0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8143270 con 0x7f8cf8139980 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8cf81430f0 con 0x7f8cf80a5c70 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f8cf81433f0 con 0x7f8cf8133fc0 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cfc92f640 10 monclient(hunting): authenticate will time out at 1370.808722s 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf659c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8cf80a5c70 0x7f8cf8133540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf6d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8cf8133fc0 0x7f8cf813a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cf8139980 0x7f8cf8133a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:25.246 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cf8139980 0x7f8cf8133a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:59418/0 (socket says 172.21.15.190:59418) 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 1 -- 172.21.15.190:0/3860536176 learned_addr learned my addr 172.21.15.190:0/3860536176 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 10 monclient(hunting): get_auth_request con 0x7f8cf8139980 auth_method 0 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf6d9d640 10 monclient(hunting): get_auth_request con 0x7f8cf8133fc0 auth_method 0 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf659c640 10 monclient(hunting): get_auth_request con 0x7f8cf80a5c70 auth_method 0 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:25.247 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cf659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 10 monclient(hunting): handle_auth_done global_id 6453 payload 824 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 1 -- 172.21.15.190:0/3860536176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8cf8133fc0 msgr2=0x7f8cf813a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8cf8133fc0 0x7f8cf813a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 1 -- 172.21.15.190:0/3860536176 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8cf80a5c70 msgr2=0x7f8cf8133540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:25.248 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8cf80a5c70 0x7f8cf8133540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 10 monclient: _finish_hunting 0 2024-08-19T10:45:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 1 monclient: found mon.noname-a 2024-08-19T10:45:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:45:25.249 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.237+0000 7f8cedd9b640 1 -- 172.21.15.190:0/3860536176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf813d1e0 con 0x7f8cf8139980 2024-08-19T10:45:25.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cedd9b640 1 -- 172.21.15.190:0/3860536176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cf8139980 msgr2=0x7f8cf8133a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:25.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cedd9b640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cf8139980 0x7f8cf8133a80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000b560 tx=0x7f8ce000ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:25.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 1 -- 172.21.15.190:0/3860536176 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ce0013020 con 0x7f8cf8139980 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 10 monclient: dump: 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:25.252 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cfc92f640 5 monclient: authenticate success, global_id 6453 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cfc92f640 10 monclient: _renew_subs 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 1 -- 172.21.15.190:0/3860536176 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce0003de0 con 0x7f8cf8139980 2024-08-19T10:45:25.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cfc92f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3860536176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf80a1b30 con 0x7f8cf8139980 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cf559a640 4 set_mon_vals no callback set 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cfc92f640 10 monclient: _renew_subs 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cfc92f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3860536176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf813c920 con 0x7f8cf8139980 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 1 -- 172.21.15.190:0/3860536176 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ce00054e0 con 0x7f8cf8139980 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 10 monclient: dump: 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:25.254 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:25.255 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:25.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cfc92f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:25.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cfc92f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:25.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cfc92f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:45:25.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3860536176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf8139d60 con 0x7f8cf8139980 2024-08-19T10:45:25.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 1 -- 172.21.15.190:0/3860536176 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8ce0016020 con 0x7f8cf8139980 2024-08-19T10:45:25.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8cd803be00 0x7f8cd803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:25.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cef7fe640 1 -- 172.21.15.190:0/3860536176 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(241..241 src has 150..241) ==== 12242+0+0 (secure 0 0 0) 0x7f8ce0050590 con 0x7f8cf8139980 2024-08-19T10:45:25.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cf659c640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8cd803be00 0x7f8cd803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:25.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.241+0000 7f8cf659c640 10 monclient: get_auth_request con 0x7f8cd803be00 auth_method 0 2024-08-19T10:45:25.256 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.245+0000 7f8cf659c640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8cd803be00 0x7f8cd803e2c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80098d0 tx=0x7f8ce80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:25.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.253+0000 7f8cef7fe640 1 -- 172.21.15.190:0/3860536176 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8ce001d030 con 0x7f8cf8139980 2024-08-19T10:45:25.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.253+0000 7f8cef7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:25.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.253+0000 7f8cef7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:25.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.429+0000 7f8cfc92f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h14"}] 2024-08-19T10:45:25.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.429+0000 7f8cfc92f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h14"}] 2024-08-19T10:45:25.432 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.429+0000 7f8cfc92f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:45:25.433 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:25.429+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3860536176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h14"} v 0) -- 0x7f8cf806c8d0 con 0x7f8cf8139980 2024-08-19T10:45:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.017+0000 7f8cef7fe640 1 -- 172.21.15.190:0/3860536176 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h14"}]=0 removed item id -58 name 'h14' from crush map v242) ==== 120+0+0 (secure 0 0 0) 0x7f8ce004c2a0 con 0x7f8cf8139980 2024-08-19T10:45:26.020 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.017+0000 7f8cef7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h14"}] 2024-08-19T10:45:26.021 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.017+0000 7f8cef7fe640 10 monclient: _finish_command 2 = system:0 removed item id -58 name 'h14' from crush map 2024-08-19T10:45:26.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.133+0000 7f8cfc92f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h14"}] 2024-08-19T10:45:26.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.133+0000 7f8cfc92f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h14"}] 2024-08-19T10:45:26.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.133+0000 7f8cfc92f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:45:26.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.133+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3860536176 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h14"} v 0) -- 0x7f8cf80a13c0 con 0x7f8cf8139980 2024-08-19T10:45:26.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.133+0000 7f8cef7fe640 1 -- 172.21.15.190:0/3860536176 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h14"}]=0 device 'h14' does not appear in the crush map v242) ==== 120+0+0 (secure 0 0 0) 0x7f8ce0020580 con 0x7f8cf8139980 2024-08-19T10:45:26.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.133+0000 7f8cef7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h14"}] 2024-08-19T10:45:26.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.133+0000 7f8cef7fe640 10 monclient: _finish_command 3 = system:0 device 'h14' does not appear in the crush map 2024-08-19T10:45:26.137 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h14' does not appear in the crush map 2024-08-19T10:45:26.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3860536176 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8cd803be00 msgr2=0x7f8cd803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:26.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8cd803be00 0x7f8cd803e2c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80098d0 tx=0x7f8ce80a6000 comp rx=0 tx=0).stop 2024-08-19T10:45:26.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 10 monclient: shutdown 2024-08-19T10:45:26.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:26.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3860536176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cf8139980 msgr2=0x7f8cf8133a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:26.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cf8139980 0x7f8cf8133a80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000b560 tx=0x7f8ce000ba20 comp rx=0 tx=0).stop 2024-08-19T10:45:26.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3860536176 shutdown_connections 2024-08-19T10:45:26.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8cd803be00 0x7f8cd803e2c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8cf8133fc0 0x7f8cf813a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8cf8139980 0x7f8cf8133a80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 --2- 172.21.15.190:0/3860536176 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8cf80a5c70 0x7f8cf8133540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3860536176 >> 172.21.15.190:0/3860536176 conn(0x7f8cf80abde0 msgr2=0x7f8cf8134d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:26.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3860536176 shutdown_connections 2024-08-19T10:45:26.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.137+0000 7f8cfc92f640 1 -- 172.21.15.190:0/3860536176 wait complete. 2024-08-19T10:45:26.154 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:26.155 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h15 2024-08-19T10:45:26.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.273+0000 7f29d26ba640 10 monclient: get_monmap_and_config 2024-08-19T10:45:26.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.273+0000 7f29d26ba640 10 monclient: build_initial_monmap 2024-08-19T10:45:26.275 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.273+0000 7f29d26ba640 10 monclient: monmap: 2024-08-19T10:45:26.275 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:26.275 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:26.276 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:26.277968+0000 2024-08-19T10:45:26.276 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:26.277968+0000 2024-08-19T10:45:26.276 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:26.276 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:26.276 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:26.276 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:26.276 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:26.276 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:26.276 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:26.276 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:26.276 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:26.277 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:26.277 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:26.277 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:26.277 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:26.277 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:26.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.273+0000 7f29d26ba640 1 Processor -- start 2024-08-19T10:45:26.279 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.273+0000 7f29d26ba640 1 -- start start 2024-08-19T10:45:26.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.273+0000 7f29d26ba640 10 monclient: init 2024-08-19T10:45:26.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:26.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 10 monclient: _add_conns ranks=[2,4,3,0,9,10,1,7,6,5,8] 2024-08-19T10:45:26.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29cc0a3a40 0x7f29cc0a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:26.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 10 monclient(hunting): picked mon.noname-b con 0x7f29cc0a3a40 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:26.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29cc0a0320 0x7f29cc0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:26.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 10 monclient(hunting): picked mon.noname-c con 0x7f29cc0a0320 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:26.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc139dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:26.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 10 monclient(hunting): picked mon.noname-g con 0x7f29cc139970 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:26.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:26.280 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f29cc0a1380 con 0x7f29cc0a3a40 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f29cc0a48a0 con 0x7f29cc139970 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f29cc0a0ae0 con 0x7f29cc0a0320 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d26ba640 10 monclient(hunting): authenticate will time out at 1371.848627s 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc139dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29cbfff640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29cc0a3a40 msgr2=0x7f29cc0a3e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc139dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:41836/0 (socket says 172.21.15.190:41836) 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29cbfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29cc0a3a40 0x7f29cc0a3e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 1 -- 172.21.15.190:0/3762764883 learned_addr learned my addr 172.21.15.190:0/3762764883 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:26.281 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 10 monclient(hunting): get_auth_request con 0x7f29cc139970 auth_method 0 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29cb7fe640 1 --2- 172.21.15.190:0/3762764883 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29cc0a0320 0x7f29cc0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 10 monclient(hunting): handle_auth_done global_id 34824 payload 274 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 1 -- 172.21.15.190:0/3762764883 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29cc0a0320 msgr2=0x7f29cc0a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 1 --2- 172.21.15.190:0/3762764883 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29cc0a0320 0x7f29cc0a4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 1 -- 172.21.15.190:0/3762764883 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29cc0a3a40 msgr2=0x7f29cc0a3e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:26.282 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 1 --2- 172.21.15.190:0/3762764883 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29cc0a3a40 0x7f29cc0a3e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 10 monclient: _finish_hunting 0 2024-08-19T10:45:26.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 1 monclient: found mon.noname-g 2024-08-19T10:45:26.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:26.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:26.283 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.277+0000 7f29d0c57640 1 -- 172.21.15.190:0/3762764883 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc0a0dc0 con 0x7f29cc139970 2024-08-19T10:45:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d0c57640 1 -- 172.21.15.190:0/3762764883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 msgr2=0x7f29cc139dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d0c57640 1 --2- 172.21.15.190:0/3762764883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc139dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29c40098a0 tx=0x7f29c401c4f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c7861e73ce5d4509 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 1 -- 172.21.15.190:0/3762764883 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f29c40b0070 con 0x7f29cc139970 2024-08-19T10:45:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 10 monclient: dump: 2024-08-19T10:45:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:26.286 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:26.287 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 10 monclient: _finish_auth 0 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 5 monclient: authenticate success, global_id 34824 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 1 -- 172.21.15.190:0/3762764883 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c40aa3a0 con 0x7f29cc139970 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 1 -- 172.21.15.190:0/3762764883 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f29c40aaa00 con 0x7f29cc139970 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 4 set_mon_vals no callback set 2024-08-19T10:45:26.288 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29caffd640 10 monclient: dump: 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:26.289 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient: shutdown 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 -- 172.21.15.190:0/3762764883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 msgr2=0x7f29cc139dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 --2- 172.21.15.190:0/3762764883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc139dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29c40098a0 tx=0x7f29c401c4f0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 -- 172.21.15.190:0/3762764883 shutdown_connections 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 --2- 172.21.15.190:0/3762764883 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29cc0a0320 0x7f29cc0a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.290 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 --2- 172.21.15.190:0/3762764883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc139dd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 --2- 172.21.15.190:0/3762764883 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f29cc0a3a40 0x7f29cc0a3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 -- 172.21.15.190:0/3762764883 >> 172.21.15.190:0/3762764883 conn(0x7f29cc06dde0 msgr2=0x7f29cc06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:26.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 -- 172.21.15.190:0/3762764883 shutdown_connections 2024-08-19T10:45:26.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 -- 172.21.15.190:0/3762764883 wait complete. 2024-08-19T10:45:26.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient: build_initial_monmap 2024-08-19T10:45:26.291 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient: monmap: 2024-08-19T10:45:26.291 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:26.291 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:26.291 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:26.291 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:26.291 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:26.292 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 Processor -- start 2024-08-19T10:45:26.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 -- start start 2024-08-19T10:45:26.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient: init 2024-08-19T10:45:26.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:26.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient: _add_conns ranks=[4,5,3,2,6,8,1,0,7] 2024-08-19T10:45:26.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29cc0a0320 0x7f29cc138500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:26.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient(hunting): picked mon.noname-c con 0x7f29cc0a0320 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:26.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29cc0a3a40 0x7f29cc138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:26.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient(hunting): picked mon.noname-h con 0x7f29cc0a3a40 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:26.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc13c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:26.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient(hunting): picked mon.noname-g con 0x7f29cc139970 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:26.293 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:26.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f29cc0a4f10 con 0x7f29cc139970 2024-08-19T10:45:26.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:26.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f29cc06cbf0 con 0x7f29cc0a0320 2024-08-19T10:45:26.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:26.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f29cc0a6db0 con 0x7f29cc0a3a40 2024-08-19T10:45:26.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:26.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.281+0000 7f29d26ba640 10 monclient(hunting): authenticate will time out at 1371.855957s 2024-08-19T10:45:26.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc13c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:26.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cb7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29cc0a3a40 0x7f29cc138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:26.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc13c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:41840/0 (socket says 172.21.15.190:41840) 2024-08-19T10:45:26.294 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cb7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29cc0a3a40 0x7f29cc138a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:39830/0 (socket says 172.21.15.190:39830) 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 1 -- 172.21.15.190:0/970046197 learned_addr learned my addr 172.21.15.190:0/970046197 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 10 monclient(hunting): get_auth_request con 0x7f29cc139970 auth_method 0 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cb7fe640 10 monclient(hunting): get_auth_request con 0x7f29cc0a3a40 auth_method 0 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:26.295 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cbfff640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29cc0a0320 0x7f29cc138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 10 monclient(hunting): handle_auth_done global_id 34833 payload 824 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 1 -- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29cc0a3a40 msgr2=0x7f29cc138a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29cc0a3a40 0x7f29cc138a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 1 -- 172.21.15.190:0/970046197 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29cc0a0320 msgr2=0x7f29cc138500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29cc0a0320 0x7f29cc138500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 10 monclient: _finish_hunting 0 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 1 monclient: found mon.noname-g 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:26.296 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 1 -- 172.21.15.190:0/970046197 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc123f00 con 0x7f29cc139970 2024-08-19T10:45:26.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cbfff640 10 monclient: get_auth_request con 0x7f29cc0a0320 auth_method 0 2024-08-19T10:45:26.297 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29cbfff640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29cc0a0320 0x7f29cc138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:26.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 1 -- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 msgr2=0x7f29cc13c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:26.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d0c57640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc13c7d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f29c40ae040 tx=0x7f29c4019290 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:26.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 1 -- 172.21.15.190:0/970046197 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f29c40b0070 con 0x7f29cc139970 2024-08-19T10:45:26.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:26.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:26.299 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 10 monclient: dump: 2024-08-19T10:45:26.299 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:26.299 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:26.299 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:26.300 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:26.300 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:26.300 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:26.300 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:26.300 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:26.300 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:26.300 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:26.300 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:26.300 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:26.300 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:26.301 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:26.301 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:26.301 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:26.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 10 monclient: _finish_auth 0 2024-08-19T10:45:26.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:26.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:26.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 1 -- 172.21.15.190:0/970046197 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c40ba070 con 0x7f29cc139970 2024-08-19T10:45:26.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d26ba640 5 monclient: authenticate success, global_id 34833 2024-08-19T10:45:26.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d26ba640 10 monclient: _renew_subs 2024-08-19T10:45:26.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d26ba640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:26.301 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d26ba640 1 -- 172.21.15.190:0/970046197 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29cc1395e0 con 0x7f29cc139970 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 1 -- 172.21.15.190:0/970046197 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f29c40aa9c0 con 0x7f29cc139970 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29c8ff9640 10 monclient: dump: 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:26.302 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29ca7fc640 4 set_mon_vals no callback set 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d26ba640 10 monclient: _renew_subs 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d26ba640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:26.303 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d26ba640 1 -- 172.21.15.190:0/970046197 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f29cc138f80 con 0x7f29cc139970 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d26ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d26ba640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d26ba640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.285+0000 7f29d26ba640 1 -- 172.21.15.190:0/970046197 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29cc139dd0 con 0x7f29cc139970 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.289+0000 7f29c8ff9640 1 -- 172.21.15.190:0/970046197 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f29c40aaba0 con 0x7f29cc139970 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.289+0000 7f29c8ff9640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f29ac03be00 0x7f29ac03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.289+0000 7f29c8ff9640 1 -- 172.21.15.190:0/970046197 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(242..242 src has 150..242) ==== 12029+0+0 (secure 0 0 0) 0x7f29c40ea6e0 con 0x7f29cc139970 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.289+0000 7f29cbfff640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f29ac03be00 0x7f29ac03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.289+0000 7f29cbfff640 10 monclient: get_auth_request con 0x7f29ac03be00 auth_method 0 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.289+0000 7f29cbfff640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f29ac03be00 0x7f29ac03e2c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f29bc003cc0 tx=0x7f29bc008330 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.293+0000 7f29c8ff9640 1 -- 172.21.15.190:0/970046197 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f29c4016170 con 0x7f29cc139970 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.293+0000 7f29c8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:26.304 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.293+0000 7f29c8ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:26.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.469+0000 7f29d26ba640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h15"}] 2024-08-19T10:45:26.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.469+0000 7f29d26ba640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h15"}] 2024-08-19T10:45:26.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.469+0000 7f29d26ba640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:26.470 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:26.469+0000 7f29d26ba640 1 -- 172.21.15.190:0/970046197 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h15"} v 0) -- 0x7f29cc073fd0 con 0x7f29cc139970 2024-08-19T10:45:27.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.061+0000 7f29c8ff9640 1 -- 172.21.15.190:0/970046197 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h15"}]=0 removed item id -61 name 'h15' from crush map v243) ==== 120+0+0 (secure 0 0 0) 0x7f29c4003d90 con 0x7f29cc139970 2024-08-19T10:45:27.062 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.061+0000 7f29c8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h15"}] 2024-08-19T10:45:27.063 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.061+0000 7f29c8ff9640 10 monclient: _finish_command 2 = system:0 removed item id -61 name 'h15' from crush map 2024-08-19T10:45:27.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.169+0000 7f29d26ba640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h15"}] 2024-08-19T10:45:27.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.169+0000 7f29d26ba640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h15"}] 2024-08-19T10:45:27.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.169+0000 7f29d26ba640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:27.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.169+0000 7f29d26ba640 1 -- 172.21.15.190:0/970046197 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h15"} v 0) -- 0x7f29cc0008e0 con 0x7f29cc139970 2024-08-19T10:45:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29c8ff9640 1 -- 172.21.15.190:0/970046197 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h15"}]=0 device 'h15' does not appear in the crush map v243) ==== 120+0+0 (secure 0 0 0) 0x7f29c40e7480 con 0x7f29cc139970 2024-08-19T10:45:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29c8ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h15"}] 2024-08-19T10:45:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29c8ff9640 10 monclient: _finish_command 3 = system:0 device 'h15' does not appear in the crush map 2024-08-19T10:45:27.173 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h15' does not appear in the crush map 2024-08-19T10:45:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 -- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f29ac03be00 msgr2=0x7f29ac03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f29ac03be00 0x7f29ac03e2c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f29bc003cc0 tx=0x7f29bc008330 comp rx=0 tx=0).stop 2024-08-19T10:45:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 10 monclient: shutdown 2024-08-19T10:45:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 -- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 msgr2=0x7f29cc13c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc13c7d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f29c40ae040 tx=0x7f29c4019290 comp rx=0 tx=0).stop 2024-08-19T10:45:27.176 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 -- 172.21.15.190:0/970046197 shutdown_connections 2024-08-19T10:45:27.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f29ac03be00 0x7f29ac03e2c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:27.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29cc139970 0x7f29cc13c7d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:27.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f29cc0a3a40 0x7f29cc138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:27.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 --2- 172.21.15.190:0/970046197 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f29cc0a0320 0x7f29cc138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:27.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 -- 172.21.15.190:0/970046197 >> 172.21.15.190:0/970046197 conn(0x7f29cc06dde0 msgr2=0x7f29cc13fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:27.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 -- 172.21.15.190:0/970046197 shutdown_connections 2024-08-19T10:45:27.177 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.173+0000 7f29d26ba640 1 -- 172.21.15.190:0/970046197 wait complete. 2024-08-19T10:45:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:27.191 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h16 2024-08-19T10:45:27.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient: get_monmap_and_config 2024-08-19T10:45:27.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient: build_initial_monmap 2024-08-19T10:45:27.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient: monmap: 2024-08-19T10:45:27.310 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:27.310 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:27.310 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:27.312911+0000 2024-08-19T10:45:27.310 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:27.312911+0000 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:27.311 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:27.312 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:27.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 1 Processor -- start 2024-08-19T10:45:27.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 1 -- start start 2024-08-19T10:45:27.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient: init 2024-08-19T10:45:27.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:27.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient: _add_conns ranks=[4,10,0,3,5,6,8,2,7,1,9] 2024-08-19T10:45:27.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09100a5eb0 0x7f09100a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:27.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient(hunting): picked mon.noname-c con 0x7f09100a5eb0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:27.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09100a4bf0 0x7f09100a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:27.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient(hunting): picked mon.noname-f con 0x7f09100a4bf0 addr v2:172.21.15.190:3300/0 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0910139860 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f091006ca60 con 0x7f0910139860 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f091006d120 con 0x7f09100a4bf0 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f09100a5510 con 0x7f09100a5eb0 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f0916c36640 10 monclient(hunting): authenticate will time out at 1372.883446s 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09100a4bf0 0x7f09100a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09100a4bf0 0x7f09100a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:52878/0 (socket says 172.21.15.190:52878) 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 1 -- 172.21.15.190:0/1083602557 learned_addr learned my addr 172.21.15.190:0/1083602557 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 10 monclient(hunting): get_auth_request con 0x7f09100a4bf0 auth_method 0 2024-08-19T10:45:27.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f09149d2640 1 --2- 172.21.15.190:0/1083602557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09100a5eb0 0x7f09100a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f09149d2640 10 monclient(hunting): get_auth_request con 0x7f09100a5eb0 auth_method 0 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f09149d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f09149d2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f09149d2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 10 monclient(hunting): handle_auth_done global_id 44254 payload 274 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.309+0000 7f090ffff640 1 -- 172.21.15.190:0/1083602557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09100a5eb0 msgr2=0x7f09100a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.313+0000 7f090ffff640 1 --2- 172.21.15.190:0/1083602557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09100a5eb0 0x7f09100a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:27.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.313+0000 7f090ffff640 1 -- 172.21.15.190:0/1083602557 >> v1:172.21.15.89:6789/0 conn(0x7f0910139860 legacy=0x7f0910139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:27.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.313+0000 7f090ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:27.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.313+0000 7f090ffff640 1 monclient: found mon.noname-f 2024-08-19T10:45:27.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.313+0000 7f090ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:27.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.313+0000 7f090ffff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:45:27.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.313+0000 7f090ffff640 1 -- 172.21.15.190:0/1083602557 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09100a0210 con 0x7f09100a4bf0 2024-08-19T10:45:27.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090ffff640 1 -- 172.21.15.190:0/1083602557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09100a4bf0 msgr2=0x7f09100a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:27.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090ffff640 1 --2- 172.21.15.190:0/1083602557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09100a4bf0 0x7f09100a4fd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009870 tx=0x7f08fc01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69a7a065d18a0275 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:27.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 1 -- 172.21.15.190:0/1083602557 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08fc0b0070 con 0x7f09100a4bf0 2024-08-19T10:45:27.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:27.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:45:27.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 10 monclient: dump: 2024-08-19T10:45:27.321 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:27.321 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:27.321 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:27.321 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:27.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 1 -- 172.21.15.190:0/1083602557 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08fc0aa810 con 0x7f09100a4bf0 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 1 -- 172.21.15.190:0/1083602557 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08fc0a9300 con 0x7f09100a4bf0 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090f7fe640 10 monclient: dump: 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:27.323 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 5 monclient: authenticate success, global_id 44254 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 4 set_mon_vals no callback set 2024-08-19T10:45:27.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient: shutdown 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 -- 172.21.15.190:0/1083602557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09100a4bf0 msgr2=0x7f09100a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 --2- 172.21.15.190:0/1083602557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09100a4bf0 0x7f09100a4fd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009870 tx=0x7f08fc01c420 comp rx=0 tx=0).stop 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 -- 172.21.15.190:0/1083602557 shutdown_connections 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 --2- 172.21.15.190:0/1083602557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09100a4bf0 0x7f09100a4fd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 --2- 172.21.15.190:0/1083602557 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09100a5eb0 0x7f09100a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 -- 172.21.15.190:0/1083602557 >> 172.21.15.190:0/1083602557 conn(0x7f09100a92a0 msgr2=0x7f09100a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 -- 172.21.15.190:0/1083602557 shutdown_connections 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 -- 172.21.15.190:0/1083602557 wait complete. 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient: build_initial_monmap 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient: monmap: 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:27.325 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:27.326 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 Processor -- start 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 -- start start 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient: init 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient: _add_conns ranks=[5,4,7,0,6,3,1,8,2] 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09100a4bf0 0x7f091013ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient(hunting): picked mon.noname-h con 0x7f09100a4bf0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09100a5eb0 0x7f091013cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient(hunting): picked mon.noname-c con 0x7f09100a5eb0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0910139860 0x7f0910138a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient(hunting): picked mon.noname-i con 0x7f0910139860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:27.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f09101442e0 con 0x7f09100a5eb0 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f09100a1030 con 0x7f09100a4bf0 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0910144460 con 0x7f0910139860 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f0916c36640 10 monclient(hunting): authenticate will time out at 1372.890726s 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09100a4bf0 0x7f091013ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09151d3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0910139860 0x7f0910138a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09100a4bf0 0x7f091013ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:39834/0 (socket says 172.21.15.190:39834) 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 1 -- 172.21.15.190:0/2463917336 learned_addr learned my addr 172.21.15.190:0/2463917336 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 10 monclient(hunting): get_auth_request con 0x7f09100a4bf0 auth_method 0 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:27.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09151d3640 10 monclient(hunting): get_auth_request con 0x7f0910139860 auth_method 0 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09151d3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09151d3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09151d3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09151d3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09151d3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09151d3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 10 monclient(hunting): handle_auth_done global_id 6377 payload 824 2024-08-19T10:45:27.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 1 -- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0910139860 msgr2=0x7f0910138a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:27.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0910139860 0x7f0910138a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:27.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 1 -- 172.21.15.190:0/2463917336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09100a5eb0 msgr2=0x7f091013cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:27.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f090ffff640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09100a5eb0 0x7f091013cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:27.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09100a5eb0 0x7f091013cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:27.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 10 monclient: _finish_hunting 0 2024-08-19T10:45:27.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 1 monclient: found mon.noname-h 2024-08-19T10:45:27.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:27.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:27.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09149d2640 1 -- 172.21.15.190:0/2463917336 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09100a6c10 con 0x7f09100a4bf0 2024-08-19T10:45:27.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.317+0000 7f09151d3640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0910139860 0x7f0910138a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:27.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f09149d2640 1 -- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09100a4bf0 msgr2=0x7f091013ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:27.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f09149d2640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09100a4bf0 0x7f091013ef80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f090400d800 tx=0x7f090400dcc0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 1 -- 172.21.15.190:0/2463917336 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f090400bd40 con 0x7f09100a4bf0 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 10 monclient: dump: 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:27.333 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 1 -- 172.21.15.190:0/2463917336 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0904019070 con 0x7f09100a4bf0 2024-08-19T10:45:27.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 1 -- 172.21.15.190:0/2463917336 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09040148a0 con 0x7f09100a4bf0 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 10 monclient: dump: 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:27.335 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090e7fc640 4 set_mon_vals no callback set 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f0916c36640 5 monclient: authenticate success, global_id 6377 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f0916c36640 10 monclient: _renew_subs 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f0916c36640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f0916c36640 1 -- 172.21.15.190:0/2463917336 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09100a8900 con 0x7f09100a4bf0 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f0916c36640 10 monclient: _renew_subs 2024-08-19T10:45:27.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f0916c36640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f0916c36640 1 -- 172.21.15.190:0/2463917336 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f0910138f60 con 0x7f09100a4bf0 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f0916c36640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f0916c36640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f0916c36640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f0916c36640 1 -- 172.21.15.190:0/2463917336 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09100a4fd0 con 0x7f09100a4bf0 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 1 -- 172.21.15.190:0/2463917336 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0904014430 con 0x7f09100a4bf0 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08f403bd60 0x7f08f403e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090d7fa640 1 -- 172.21.15.190:0/2463917336 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(243..243 src has 150..243) ==== 11816+0+0 (secure 0 0 0) 0x7f090400f040 con 0x7f09100a4bf0 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090ffff640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08f403bd60 0x7f08f403e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.321+0000 7f090ffff640 10 monclient: get_auth_request con 0x7f08f403bd60 auth_method 0 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.325+0000 7f090ffff640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08f403bd60 0x7f08f403e220 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f09100a04e0 tx=0x7f08fc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:27.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.325+0000 7f090d7fa640 1 -- 172.21.15.190:0/2463917336 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f090404db50 con 0x7f09100a4bf0 2024-08-19T10:45:27.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.329+0000 7f090d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:27.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.329+0000 7f090d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:27.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.505+0000 7f0916c36640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h16"}] 2024-08-19T10:45:27.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.505+0000 7f0916c36640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h16"}] 2024-08-19T10:45:27.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.505+0000 7f0916c36640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:27.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:27.505+0000 7f0916c36640 1 -- 172.21.15.190:0/2463917336 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h16"} v 0) -- 0x7f0910139160 con 0x7f09100a4bf0 2024-08-19T10:45:28.058 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.057+0000 7f090d7fa640 1 -- 172.21.15.190:0/2463917336 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h16"}]=0 removed item id -64 name 'h16' from crush map v244) ==== 120+0+0 (secure 0 0 0) 0x7f090401dce0 con 0x7f09100a4bf0 2024-08-19T10:45:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.057+0000 7f090d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h16"}] 2024-08-19T10:45:28.059 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.057+0000 7f090d7fa640 10 monclient: _finish_command 2 = system:0 removed item id -64 name 'h16' from crush map 2024-08-19T10:45:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.169+0000 7f0916c36640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h16"}] 2024-08-19T10:45:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.169+0000 7f0916c36640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h16"}] 2024-08-19T10:45:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.169+0000 7f0916c36640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:28.169 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.169+0000 7f0916c36640 1 -- 172.21.15.190:0/2463917336 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h16"} v 0) -- 0x7f0910141400 con 0x7f09100a4bf0 2024-08-19T10:45:28.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.169+0000 7f090d7fa640 1 -- 172.21.15.190:0/2463917336 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h16"}]=0 device 'h16' does not appear in the crush map v244) ==== 120+0+0 (secure 0 0 0) 0x7f0904014a40 con 0x7f09100a4bf0 2024-08-19T10:45:28.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.169+0000 7f090d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h16"}] 2024-08-19T10:45:28.171 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.169+0000 7f090d7fa640 10 monclient: _finish_command 3 = system:0 device 'h16' does not appear in the crush map 2024-08-19T10:45:28.171 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h16' does not appear in the crush map 2024-08-19T10:45:28.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 -- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08f403bd60 msgr2=0x7f08f403e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:28.173 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08f403bd60 0x7f08f403e220 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f09100a04e0 tx=0x7f08fc0a6000 comp rx=0 tx=0).stop 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 10 monclient: shutdown 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 -- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09100a4bf0 msgr2=0x7f091013ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09100a4bf0 0x7f091013ef80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f090400d800 tx=0x7f090400dcc0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 -- 172.21.15.190:0/2463917336 shutdown_connections 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08f403bd60 0x7f08f403e220 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0910139860 0x7f0910138a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09100a5eb0 0x7f091013cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 --2- 172.21.15.190:0/2463917336 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09100a4bf0 0x7f091013ef80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 -- 172.21.15.190:0/2463917336 >> 172.21.15.190:0/2463917336 conn(0x7f09100a92a0 msgr2=0x7f0910140cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 -- 172.21.15.190:0/2463917336 shutdown_connections 2024-08-19T10:45:28.174 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.173+0000 7f0916c36640 1 -- 172.21.15.190:0/2463917336 wait complete. 2024-08-19T10:45:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:28.189 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h17 2024-08-19T10:45:28.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient: get_monmap_and_config 2024-08-19T10:45:28.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient: build_initial_monmap 2024-08-19T10:45:28.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient: monmap: 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:28.309371+0000 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:28.309371+0000 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:28.307 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:28.308 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:28.308 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:28.308 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:28.308 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:28.308 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:28.308 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:28.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 1 Processor -- start 2024-08-19T10:45:28.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 1 -- start start 2024-08-19T10:45:28.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient: init 2024-08-19T10:45:28.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:28.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient: _add_conns ranks=[5,9,4,6,7,2,3,8,1,10,0] 2024-08-19T10:45:28.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff0900a4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:28.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient(hunting): picked mon.noname-h con 0x7ff0900a3c80 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:28.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff0900a45a0 0x7ff09013ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:28.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient(hunting): picked mon.noname-a con 0x7ff0900a45a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff09013bfb0 0x7ff09013e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient(hunting): picked mon.noname-c con 0x7ff09013bfb0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff09009e9b0 con 0x7ff0900a45a0 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff0900a22f0 con 0x7ff09013bfb0 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff09009dd30 con 0x7ff0900a3c80 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff096bd7640 10 monclient(hunting): authenticate will time out at 1373.879552s 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff094973640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff0900a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff094973640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff0900a4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:39850/0 (socket says 172.21.15.190:39850) 2024-08-19T10:45:28.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff094973640 1 -- 172.21.15.190:0/1675820359 learned_addr learned my addr 172.21.15.190:0/1675820359 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff094973640 10 monclient(hunting): get_auth_request con 0x7ff0900a3c80 auth_method 0 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff094973640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff094973640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.305+0000 7ff094973640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff095174640 1 --2- 172.21.15.190:0/1675820359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff09013bfb0 0x7ff09013e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff095174640 10 monclient(hunting): get_auth_request con 0x7ff09013bfb0 auth_method 0 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff095174640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff095174640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff095174640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff08ffff640 1 --2- 172.21.15.190:0/1675820359 >> v2:172.21.15.89:3300/0 conn(0x7ff0900a45a0 0x7ff09013ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:28.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff08ffff640 10 monclient(hunting): get_auth_request con 0x7ff0900a45a0 auth_method 0 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff08ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff08ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff08ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff095174640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff095174640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff095174640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 10 monclient(hunting): handle_auth_done global_id 6386 payload 274 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 1 -- 172.21.15.190:0/1675820359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff09013bfb0 msgr2=0x7ff09013e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 1 --2- 172.21.15.190:0/1675820359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff09013bfb0 0x7ff09013e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 1 -- 172.21.15.190:0/1675820359 >> v2:172.21.15.89:3300/0 conn(0x7ff0900a45a0 msgr2=0x7ff09013ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:28.314 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 1 --2- 172.21.15.190:0/1675820359 >> v2:172.21.15.89:3300/0 conn(0x7ff0900a45a0 0x7ff09013ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 10 monclient: _finish_hunting 0 2024-08-19T10:45:28.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 1 monclient: found mon.noname-h 2024-08-19T10:45:28.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:28.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:28.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff094973640 1 -- 172.21.15.190:0/1675820359 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0900a2070 con 0x7ff0900a3c80 2024-08-19T10:45:28.315 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.309+0000 7ff08ffff640 1 --2- 172.21.15.190:0/1675820359 >> v2:172.21.15.89:3300/0 conn(0x7ff0900a45a0 0x7ff09013ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:45:28.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff094973640 1 -- 172.21.15.190:0/1675820359 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 msgr2=0x7ff0900a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:28.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff094973640 1 --2- 172.21.15.190:0/1675820359 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff0900a4060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff080009870 tx=0x7ff08001c420 comp rx=0 tx=0).ready entity=mon.5 client_cookie=ada50b73a6942931 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:28.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 1 -- 172.21.15.190:0/1675820359 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0800b0070 con 0x7ff0900a3c80 2024-08-19T10:45:28.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:28.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:28.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 10 monclient: dump: 2024-08-19T10:45:28.319 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:28.319 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:28.319 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:28.319 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:28.319 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:28.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 5 monclient: authenticate success, global_id 6386 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 1 -- 172.21.15.190:0/1675820359 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0800aa810 con 0x7ff0900a3c80 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 4 set_mon_vals no callback set 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 1 -- 172.21.15.190:0/1675820359 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0800a9300 con 0x7ff0900a3c80 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff08f7fe640 10 monclient: dump: 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:28.321 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:28.322 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:28.322 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:28.322 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:28.322 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:28.322 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:28.322 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:28.322 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:28.322 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:28.322 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 10 monclient: shutdown 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 1 -- 172.21.15.190:0/1675820359 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 msgr2=0x7ff0900a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 1 --2- 172.21.15.190:0/1675820359 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff0900a4060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff080009870 tx=0x7ff08001c420 comp rx=0 tx=0).stop 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 1 -- 172.21.15.190:0/1675820359 shutdown_connections 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 1 --2- 172.21.15.190:0/1675820359 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff09013bfb0 0x7ff09013e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 1 --2- 172.21.15.190:0/1675820359 >> v2:172.21.15.89:3300/0 conn(0x7ff0900a45a0 0x7ff09013ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 1 --2- 172.21.15.190:0/1675820359 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff0900a4060 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 1 -- 172.21.15.190:0/1675820359 >> 172.21.15.190:0/1675820359 conn(0x7ff0900a4f40 msgr2=0x7ff0900a5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 1 -- 172.21.15.190:0/1675820359 shutdown_connections 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 1 -- 172.21.15.190:0/1675820359 wait complete. 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 10 monclient: build_initial_monmap 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.313+0000 7ff096bd7640 10 monclient: monmap: 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:28.324 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 1 Processor -- start 2024-08-19T10:45:28.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 1 -- start start 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient: init 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient: _add_conns ranks=[5,0,2,3,1,6,7,8,4] 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff09011f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient(hunting): picked mon.noname-h con 0x7ff0900a3c80 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0900a45a0 0x7ff09011f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient(hunting): picked mon.noname-a con 0x7ff0900a45a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff09013bfb0 0x7ff09011fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient(hunting): picked mon.noname-b con 0x7ff09013bfb0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff0900a4980 con 0x7ff0900a45a0 2024-08-19T10:45:28.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff090069720 con 0x7ff09013bfb0 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff0900a3880 con 0x7ff0900a3c80 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient(hunting): authenticate will time out at 1373.888638s 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff09011f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff09011f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:39854/0 (socket says 172.21.15.190:39854) 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 1 -- 172.21.15.190:0/717536557 learned_addr learned my addr 172.21.15.190:0/717536557 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 10 monclient(hunting): get_auth_request con 0x7ff0900a3c80 auth_method 0 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:28.327 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff095174640 1 -- 172.21.15.190:0/717536557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff09013bfb0 msgr2=0x7ff09011fd40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff095174640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff09013bfb0 0x7ff09011fd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 10 monclient(hunting): handle_auth_done global_id 6395 payload 824 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08ffff640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0900a45a0 0x7ff09011f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 1 -- 172.21.15.190:0/717536557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff09013bfb0 msgr2=0x7ff09011fd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff09013bfb0 0x7ff09011fd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 1 -- 172.21.15.190:0/717536557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0900a45a0 msgr2=0x7ff09011f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0900a45a0 0x7ff09011f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 10 monclient: _finish_hunting 0 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 1 monclient: found mon.noname-h 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:28.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 1 -- 172.21.15.190:0/717536557 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0900a33c0 con 0x7ff0900a3c80 2024-08-19T10:45:28.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 1 -- 172.21.15.190:0/717536557 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 msgr2=0x7ff09011f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:28.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff094973640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff09011f2c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff0800097c0 tx=0x7ff0800191c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:28.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 1 -- 172.21.15.190:0/717536557 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0800b9070 con 0x7ff0900a3c80 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 10 monclient: dump: 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:28.331 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 1 -- 172.21.15.190:0/717536557 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0800044b0 con 0x7ff0900a3c80 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 5 monclient: authenticate success, global_id 6395 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient: _renew_subs 2024-08-19T10:45:28.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 1 -- 172.21.15.190:0/717536557 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff090122cc0 con 0x7ff0900a3c80 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 1 -- 172.21.15.190:0/717536557 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0800b0040 con 0x7ff0900a3c80 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08d7fa640 10 monclient: dump: 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:28.333 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient: _renew_subs 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff096bd7640 1 -- 172.21.15.190:0/717536557 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0901229e0 con 0x7ff0900a3c80 2024-08-19T10:45:28.334 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.317+0000 7ff08effd640 4 set_mon_vals no callback set 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.321+0000 7ff096bd7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.321+0000 7ff096bd7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.321+0000 7ff096bd7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.321+0000 7ff096bd7640 1 -- 172.21.15.190:0/717536557 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09009cb80 con 0x7ff0900a3c80 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.321+0000 7ff08d7fa640 1 -- 172.21.15.190:0/717536557 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff080004040 con 0x7ff0900a3c80 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.321+0000 7ff08d7fa640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff07c03be00 0x7ff07c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.321+0000 7ff08d7fa640 1 -- 172.21.15.190:0/717536557 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(244..244 src has 150..244) ==== 11603+0+0 (secure 0 0 0) 0x7ff0800e9500 con 0x7ff0900a3c80 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.321+0000 7ff08ffff640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff07c03be00 0x7ff07c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.321+0000 7ff08ffff640 10 monclient: get_auth_request con 0x7ff07c03be00 auth_method 0 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.321+0000 7ff08ffff640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff07c03be00 0x7ff07c03e2c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff0901212b0 tx=0x7ff084008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.325+0000 7ff08d7fa640 1 -- 172.21.15.190:0/717536557 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff0900a33c0 con 0x7ff0900a3c80 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.325+0000 7ff08d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:28.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.325+0000 7ff08d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:28.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.501+0000 7ff096bd7640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h17"}] 2024-08-19T10:45:28.501 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.501+0000 7ff096bd7640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h17"}] 2024-08-19T10:45:28.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.501+0000 7ff096bd7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:28.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:28.501+0000 7ff096bd7640 1 -- 172.21.15.190:0/717536557 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h17"} v 0) -- 0x7ff09006e4d0 con 0x7ff0900a3c80 2024-08-19T10:45:29.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.085+0000 7ff08d7fa640 1 -- 172.21.15.190:0/717536557 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h17"}]=0 removed item id -67 name 'h17' from crush map v245) ==== 120+0+0 (secure 0 0 0) 0x7ff080016130 con 0x7ff0900a3c80 2024-08-19T10:45:29.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.085+0000 7ff08d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h17"}] 2024-08-19T10:45:29.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.085+0000 7ff08d7fa640 10 monclient: _finish_command 2 = system:0 removed item id -67 name 'h17' from crush map 2024-08-19T10:45:29.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.193+0000 7ff096bd7640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h17"}] 2024-08-19T10:45:29.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.193+0000 7ff096bd7640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h17"}] 2024-08-19T10:45:29.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.193+0000 7ff096bd7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:29.196 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.193+0000 7ff096bd7640 1 -- 172.21.15.190:0/717536557 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h17"} v 0) -- 0x7ff090121a30 con 0x7ff0900a3c80 2024-08-19T10:45:29.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.197+0000 7ff08d7fa640 1 -- 172.21.15.190:0/717536557 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h17"}]=0 device 'h17' does not appear in the crush map v245) ==== 120+0+0 (secure 0 0 0) 0x7ff0800c2090 con 0x7ff0900a3c80 2024-08-19T10:45:29.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.197+0000 7ff08d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h17"}] 2024-08-19T10:45:29.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.197+0000 7ff08d7fa640 10 monclient: _finish_command 3 = system:0 device 'h17' does not appear in the crush map 2024-08-19T10:45:29.199 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h17' does not appear in the crush map 2024-08-19T10:45:29.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 -- 172.21.15.190:0/717536557 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff07c03be00 msgr2=0x7ff07c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:29.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff07c03be00 0x7ff07c03e2c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff0901212b0 tx=0x7ff084008040 comp rx=0 tx=0).stop 2024-08-19T10:45:29.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 10 monclient: shutdown 2024-08-19T10:45:29.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:29.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 -- 172.21.15.190:0/717536557 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 msgr2=0x7ff09011f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:29.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff09011f2c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff0800097c0 tx=0x7ff0800191c0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 -- 172.21.15.190:0/717536557 shutdown_connections 2024-08-19T10:45:29.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff07c03be00 0x7ff07c03e2c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff09013bfb0 0x7ff09011fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0900a45a0 0x7ff09011f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 --2- 172.21.15.190:0/717536557 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0900a3c80 0x7ff09011f2c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 -- 172.21.15.190:0/717536557 >> 172.21.15.190:0/717536557 conn(0x7ff0900a4f40 msgr2=0x7ff09013d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:29.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 -- 172.21.15.190:0/717536557 shutdown_connections 2024-08-19T10:45:29.202 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.201+0000 7ff096bd7640 1 -- 172.21.15.190:0/717536557 wait complete. 2024-08-19T10:45:29.216 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:29.217 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h18 2024-08-19T10:45:29.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient: get_monmap_and_config 2024-08-19T10:45:29.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient: build_initial_monmap 2024-08-19T10:45:29.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient: monmap: 2024-08-19T10:45:29.335 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:29.335 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:29.335 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:29.337276+0000 2024-08-19T10:45:29.335 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:29.337276+0000 2024-08-19T10:45:29.336 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:29.336 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:29.336 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:29.336 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:29.336 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:29.336 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:29.336 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:29.336 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:29.336 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:29.337 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:29.337 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:29.337 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:29.337 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:29.337 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:29.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 1 Processor -- start 2024-08-19T10:45:29.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 1 -- start start 2024-08-19T10:45:29.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient: init 2024-08-19T10:45:29.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:29.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient: _add_conns ranks=[7,9,2,4,8,1,6,0,5,3,10] 2024-08-19T10:45:29.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac40a6740 0x7f8ac40a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:29.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient(hunting): picked mon.noname-i con 0x7f8ac40a6740 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:29.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8ac40a5bc0 0x7f8ac40a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:29.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient(hunting): picked mon.noname-a con 0x7f8ac40a5bc0 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:29.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ac40a7060 0x7f8ac4139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:29.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient(hunting): picked mon.noname-b con 0x7f8ac40a7060 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:29.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f8ac40a4c20 con 0x7f8ac40a5bc0 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac40a3530 con 0x7f8ac40a7060 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8ac40a3740 con 0x7f8ac40a6740 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac8f50640 10 monclient(hunting): authenticate will time out at 1374.907241s 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac2d9d640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ac40a7060 msgr2=0x7f8ac4139590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac2d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ac40a7060 0x7f8ac4139590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac40a6740 0x7f8ac40a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac40a6740 0x7f8ac40a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:48282/0 (socket says 172.21.15.190:48282) 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 1 -- 172.21.15.190:0/3838163718 learned_addr learned my addr 172.21.15.190:0/3838163718 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:29.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 10 monclient(hunting): get_auth_request con 0x7f8ac40a6740 auth_method 0 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac1d9b640 1 --2- 172.21.15.190:0/3838163718 >> v2:172.21.15.89:3300/0 conn(0x7f8ac40a5bc0 0x7f8ac40a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac1d9b640 10 monclient(hunting): get_auth_request con 0x7f8ac40a5bc0 auth_method 0 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:29.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:29.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:29.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 10 monclient(hunting): handle_auth_done global_id 15748 payload 274 2024-08-19T10:45:29.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 1 -- 172.21.15.190:0/3838163718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ac40a7060 msgr2=0x7f8ac4139590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:29.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 1 --2- 172.21.15.190:0/3838163718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ac40a7060 0x7f8ac4139590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 1 -- 172.21.15.190:0/3838163718 >> v2:172.21.15.89:3300/0 conn(0x7f8ac40a5bc0 msgr2=0x7f8ac40a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:29.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 1 --2- 172.21.15.190:0/3838163718 >> v2:172.21.15.89:3300/0 conn(0x7f8ac40a5bc0 0x7f8ac40a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 10 monclient: _finish_hunting 0 2024-08-19T10:45:29.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 1 monclient: found mon.noname-i 2024-08-19T10:45:29.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:29.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:29.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.333+0000 7f8ac259c640 1 -- 172.21.15.190:0/3838163718 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac40a74d0 con 0x7f8ac40a6740 2024-08-19T10:45:29.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 1 -- 172.21.15.190:0/3838163718 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac40a6740 msgr2=0x7f8ac40a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:29.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 1 --2- 172.21.15.190:0/3838163718 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac40a6740 0x7f8ac40a5680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40098a0 tx=0x7f8ab401c420 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2b50a438c959118f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:29.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 1 -- 172.21.15.190:0/3838163718 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ab40b1070 con 0x7f8ac40a6740 2024-08-19T10:45:29.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:29.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:29.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 10 monclient: dump: 2024-08-19T10:45:29.349 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:29.349 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:29.350 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:29.351 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:29.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 10 monclient: _finish_auth 0 2024-08-19T10:45:29.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 10 monclient: _check_auth_tickets 2024-08-19T10:45:29.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:29.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 1 -- 172.21.15.190:0/3838163718 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab40ab830 con 0x7f8ac40a6740 2024-08-19T10:45:29.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 5 monclient: authenticate success, global_id 15748 2024-08-19T10:45:29.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:29.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:29.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:29.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 1 -- 172.21.15.190:0/3838163718 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8ab40aa320 con 0x7f8ac40a6740 2024-08-19T10:45:29.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 4 set_mon_vals no callback set 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac159a640 10 monclient: dump: 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:29.352 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient: shutdown 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 -- 172.21.15.190:0/3838163718 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac40a6740 msgr2=0x7f8ac40a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 --2- 172.21.15.190:0/3838163718 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac40a6740 0x7f8ac40a5680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40098a0 tx=0x7f8ab401c420 comp rx=0 tx=0).stop 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 -- 172.21.15.190:0/3838163718 shutdown_connections 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 --2- 172.21.15.190:0/3838163718 >> v2:172.21.15.89:3300/0 conn(0x7f8ac40a5bc0 0x7f8ac40a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 --2- 172.21.15.190:0/3838163718 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8ac40a7060 0x7f8ac4139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 --2- 172.21.15.190:0/3838163718 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac40a6740 0x7f8ac40a5680 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 -- 172.21.15.190:0/3838163718 >> 172.21.15.190:0/3838163718 conn(0x7f8ac40abd40 msgr2=0x7f8ac40ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 -- 172.21.15.190:0/3838163718 shutdown_connections 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 -- 172.21.15.190:0/3838163718 wait complete. 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient: build_initial_monmap 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient: monmap: 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:29.354 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 Processor -- start 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 -- start start 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient: init 2024-08-19T10:45:29.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient: _add_conns ranks=[1,3,7,8,4,6,5,2,0] 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac40a6740 0x7f8ac40a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient(hunting): picked mon.noname-f con 0x7f8ac40a6740 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac40a7060 0x7f8ac40a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient(hunting): picked mon.noname-g con 0x7f8ac40a7060 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac41d1800 0x7f8ac41d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient(hunting): picked mon.noname-i con 0x7f8ac41d1800 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4143450 con 0x7f8ac40a6740 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac41432d0 con 0x7f8ac40a7060 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:29.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8ac41435d0 con 0x7f8ac41d1800 2024-08-19T10:45:29.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:29.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac8f50640 10 monclient(hunting): authenticate will time out at 1374.915027s 2024-08-19T10:45:29.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac40a6740 0x7f8ac40a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:29.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac41d1800 0x7f8ac41d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:29.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac40a6740 0x7f8ac40a20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:52882/0 (socket says 172.21.15.190:52882) 2024-08-19T10:45:29.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 1 -- 172.21.15.190:0/1046768136 learned_addr learned my addr 172.21.15.190:0/1046768136 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:29.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 10 monclient(hunting): get_auth_request con 0x7f8ac40a6740 auth_method 0 2024-08-19T10:45:29.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:29.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:29.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:29.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac1d9b640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac40a7060 0x7f8ac40a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:29.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 10 monclient(hunting): get_auth_request con 0x7f8ac41d1800 auth_method 0 2024-08-19T10:45:29.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:29.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:29.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:29.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac1d9b640 10 monclient(hunting): get_auth_request con 0x7f8ac40a7060 auth_method 0 2024-08-19T10:45:29.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:29.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:29.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:29.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:29.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:29.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:29.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:29.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:29.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:29.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:29.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:29.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 10 monclient(hunting): handle_auth_done global_id 15757 payload 824 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 1 -- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac40a7060 msgr2=0x7f8ac40a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac40a7060 0x7f8ac40a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 1 -- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac40a6740 msgr2=0x7f8ac40a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac40a6740 0x7f8ac40a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 1 monclient: found mon.noname-i 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.341+0000 7f8ac2d9d640 1 -- 172.21.15.190:0/1046768136 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac40a2b30 con 0x7f8ac41d1800 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac2d9d640 1 -- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac41d1800 msgr2=0x7f8ac41d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac2d9d640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac41d1800 0x7f8ac41d1c80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00bd90 tx=0x7f8abc009eb0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:29.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 1 -- 172.21.15.190:0/1046768136 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8abc006960 con 0x7f8ac41d1800 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 10 monclient: dump: 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:29.361 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 1 -- 172.21.15.190:0/1046768136 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8abc006b40 con 0x7f8ac41d1800 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac8f50640 5 monclient: authenticate success, global_id 15757 2024-08-19T10:45:29.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac8f50640 10 monclient: _renew_subs 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac8f50640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac8f50640 1 -- 172.21.15.190:0/1046768136 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac413fe00 con 0x7f8ac41d1800 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 1 -- 172.21.15.190:0/1046768136 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8abc0158b0 con 0x7f8ac41d1800 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 10 monclient: dump: 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:29.363 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac8f50640 10 monclient: _renew_subs 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac8f50640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac0d99640 4 set_mon_vals no callback set 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac8f50640 1 -- 172.21.15.190:0/1046768136 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac409f890 con 0x7f8ac41d1800 2024-08-19T10:45:29.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac8f50640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:29.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac8f50640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:29.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac8f50640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:29.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac8f50640 1 -- 172.21.15.190:0/1046768136 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac409e370 con 0x7f8ac41d1800 2024-08-19T10:45:29.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 1 -- 172.21.15.190:0/1046768136 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8abc015440 con 0x7f8ac41d1800 2024-08-19T10:45:29.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8aa403be60 0x7f8aa403e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:29.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ab2ffd640 1 -- 172.21.15.190:0/1046768136 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(245..245 src has 150..245) ==== 11390+0+0 (secure 0 0 0) 0x7f8abc051230 con 0x7f8ac41d1800 2024-08-19T10:45:29.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac259c640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8aa403be60 0x7f8aa403e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:29.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.345+0000 7f8ac259c640 10 monclient: get_auth_request con 0x7f8aa403be60 auth_method 0 2024-08-19T10:45:29.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.349+0000 7f8ac259c640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8aa403be60 0x7f8aa403e320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8ab401ccc0 tx=0x7f8ab4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:29.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.353+0000 7f8ab2ffd640 1 -- 172.21.15.190:0/1046768136 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8abc012330 con 0x7f8ac41d1800 2024-08-19T10:45:29.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.353+0000 7f8ab2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:29.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.353+0000 7f8ab2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:29.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.525+0000 7f8ac8f50640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h18"}] 2024-08-19T10:45:29.527 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.525+0000 7f8ac8f50640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h18"}] 2024-08-19T10:45:29.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.525+0000 7f8ac8f50640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:29.528 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:29.525+0000 7f8ac8f50640 1 -- 172.21.15.190:0/1046768136 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h18"} v 0) -- 0x7f8ac40a5680 con 0x7f8ac41d1800 2024-08-19T10:45:30.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.085+0000 7f8ab2ffd640 1 -- 172.21.15.190:0/1046768136 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h18"}]=0 removed item id -70 name 'h18' from crush map v246) ==== 120+0+0 (secure 0 0 0) 0x7f8abc04e250 con 0x7f8ac41d1800 2024-08-19T10:45:30.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.085+0000 7f8ab2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h18"}] 2024-08-19T10:45:30.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.085+0000 7f8ab2ffd640 10 monclient: _finish_command 2 = system:0 removed item id -70 name 'h18' from crush map 2024-08-19T10:45:30.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.197+0000 7f8ac8f50640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h18"}] 2024-08-19T10:45:30.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.197+0000 7f8ac8f50640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h18"}] 2024-08-19T10:45:30.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.197+0000 7f8ac8f50640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:30.198 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.197+0000 7f8ac8f50640 1 -- 172.21.15.190:0/1046768136 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h18"} v 0) -- 0x7f8ac40008e0 con 0x7f8ac41d1800 2024-08-19T10:45:30.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ab2ffd640 1 -- 172.21.15.190:0/1046768136 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h18"}]=0 device 'h18' does not appear in the crush map v246) ==== 120+0+0 (secure 0 0 0) 0x7f8abc01c4b0 con 0x7f8ac41d1800 2024-08-19T10:45:30.200 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ab2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h18"}] 2024-08-19T10:45:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ab2ffd640 10 monclient: _finish_command 3 = system:0 device 'h18' does not appear in the crush map 2024-08-19T10:45:30.201 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h18' does not appear in the crush map 2024-08-19T10:45:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 -- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8aa403be60 msgr2=0x7f8aa403e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8aa403be60 0x7f8aa403e320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8ab401ccc0 tx=0x7f8ab4008040 comp rx=0 tx=0).stop 2024-08-19T10:45:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 10 monclient: shutdown 2024-08-19T10:45:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:30.203 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 -- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac41d1800 msgr2=0x7f8ac41d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac41d1800 0x7f8ac41d1c80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00bd90 tx=0x7f8abc009eb0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 -- 172.21.15.190:0/1046768136 shutdown_connections 2024-08-19T10:45:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8aa403be60 0x7f8aa403e320 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8ac41d1800 0x7f8ac41d1c80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ac40a7060 0x7f8ac40a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 --2- 172.21.15.190:0/1046768136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ac40a6740 0x7f8ac40a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 -- 172.21.15.190:0/1046768136 >> 172.21.15.190:0/1046768136 conn(0x7f8ac40abd40 msgr2=0x7f8ac4136640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 -- 172.21.15.190:0/1046768136 shutdown_connections 2024-08-19T10:45:30.204 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.201+0000 7f8ac8f50640 1 -- 172.21.15.190:0/1046768136 wait complete. 2024-08-19T10:45:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:30.219 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h19 2024-08-19T10:45:30.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.329+0000 7f8c3d863640 10 monclient: get_monmap_and_config 2024-08-19T10:45:30.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.329+0000 7f8c3d863640 10 monclient: build_initial_monmap 2024-08-19T10:45:30.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.329+0000 7f8c3d863640 10 monclient: monmap: 2024-08-19T10:45:30.333 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:30.333 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:30.333 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:30.335065+0000 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:30.335065+0000 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:30.334 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:30.335 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:30.335 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:30.335 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 1 Processor -- start 2024-08-19T10:45:30.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 1 -- start start 2024-08-19T10:45:30.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 10 monclient: init 2024-08-19T10:45:30.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:30.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 10 monclient: _add_conns ranks=[3,2,9,4,1,10,7,5,0,8,6] 2024-08-19T10:45:30.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c380a3610 0x7f8c38073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:30.336 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 10 monclient(hunting): picked mon.noname-g con 0x7f8c380a3610 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c380741f0 0x7f8c38135430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 10 monclient(hunting): picked mon.noname-b con 0x7f8c380741f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8c38135970 0x7f8c38135d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 10 monclient(hunting): picked mon.noname-a con 0x7f8c38135970 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f8c380a44c0 con 0x7f8c38135970 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8c380a39f0 con 0x7f8c380741f0 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8c380706c0 con 0x7f8c380a3610 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c3d863640 10 monclient(hunting): authenticate will time out at 1375.905576s 2024-08-19T10:45:30.337 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c2ffff640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c380741f0 msgr2=0x7f8c38135430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c2ffff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c380741f0 0x7f8c38135430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c380a3610 0x7f8c38073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c380a3610 0x7f8c38073cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:41852/0 (socket says 172.21.15.190:41852) 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 1 -- 172.21.15.190:0/3929769405 learned_addr learned my addr 172.21.15.190:0/3929769405 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 10 monclient(hunting): get_auth_request con 0x7f8c380a3610 auth_method 0 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c377fe640 1 --2- 172.21.15.190:0/3929769405 >> v2:172.21.15.89:3300/0 conn(0x7f8c38135970 0x7f8c38135d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:30.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 10 monclient(hunting): handle_auth_done global_id 34851 payload 274 2024-08-19T10:45:30.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 1 -- 172.21.15.190:0/3929769405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c380741f0 msgr2=0x7f8c38135430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:30.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 1 --2- 172.21.15.190:0/3929769405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c380741f0 0x7f8c38135430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 1 -- 172.21.15.190:0/3929769405 >> v2:172.21.15.89:3300/0 conn(0x7f8c38135970 msgr2=0x7f8c38135d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:30.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 1 --2- 172.21.15.190:0/3929769405 >> v2:172.21.15.89:3300/0 conn(0x7f8c38135970 0x7f8c38135d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:45:30.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 1 monclient: found mon.noname-g 2024-08-19T10:45:30.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:30.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:30.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c36ffd640 1 -- 172.21.15.190:0/3929769405 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c38071cb0 con 0x7f8c380a3610 2024-08-19T10:45:30.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c377fe640 10 monclient: get_auth_request con 0x7f8c38135970 auth_method 0 2024-08-19T10:45:30.339 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.333+0000 7f8c377fe640 1 --2- 172.21.15.190:0/3929769405 >> v2:172.21.15.89:3300/0 conn(0x7f8c38135970 0x7f8c38135d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:30.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.337+0000 7f8c36ffd640 1 -- 172.21.15.190:0/3929769405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c380a3610 msgr2=0x7f8c38073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:30.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.337+0000 7f8c36ffd640 1 --2- 172.21.15.190:0/3929769405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c380a3610 0x7f8c38073cb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800fb40 tx=0x7f8c28020730 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3912a7de0b4bde03 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:30.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 1 -- 172.21.15.190:0/3929769405 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c280b3070 con 0x7f8c380a3610 2024-08-19T10:45:30.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:30.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:30.341 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 10 monclient: dump: 2024-08-19T10:45:30.341 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:30.341 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:30.341 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:30.341 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:30.341 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:30.341 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:30.341 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:30.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 5 monclient: authenticate success, global_id 34851 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 1 -- 172.21.15.190:0/3929769405 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c2800e660 con 0x7f8c380a3610 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 1 -- 172.21.15.190:0/3929769405 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c2800ec80 con 0x7f8c380a3610 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c367fc640 10 monclient: dump: 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:30.343 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:30.344 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:30.344 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:30.344 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:30.344 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:30.344 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:30.344 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:30.344 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:30.344 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:30.344 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:30.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:30.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 4 set_mon_vals no callback set 2024-08-19T10:45:30.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 10 monclient: shutdown 2024-08-19T10:45:30.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:30.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 1 -- 172.21.15.190:0/3929769405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c380a3610 msgr2=0x7f8c38073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:30.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 1 --2- 172.21.15.190:0/3929769405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c380a3610 0x7f8c38073cb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800fb40 tx=0x7f8c28020730 comp rx=0 tx=0).stop 2024-08-19T10:45:30.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 1 -- 172.21.15.190:0/3929769405 shutdown_connections 2024-08-19T10:45:30.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 1 --2- 172.21.15.190:0/3929769405 >> v2:172.21.15.89:3300/0 conn(0x7f8c38135970 0x7f8c38135d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 1 --2- 172.21.15.190:0/3929769405 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8c380741f0 0x7f8c38135430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 1 --2- 172.21.15.190:0/3929769405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c380a3610 0x7f8c38073cb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 1 -- 172.21.15.190:0/3929769405 >> 172.21.15.190:0/3929769405 conn(0x7f8c380a4700 msgr2=0x7f8c380a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 1 -- 172.21.15.190:0/3929769405 shutdown_connections 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 1 -- 172.21.15.190:0/3929769405 wait complete. 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 10 monclient: build_initial_monmap 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.341+0000 7f8c3d863640 10 monclient: monmap: 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:30.346 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:30.347 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:30.347 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:30.347 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:30.347 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:30.347 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:30.347 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:30.347 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:30.347 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:30.347 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 1 Processor -- start 2024-08-19T10:45:30.348 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 1 -- start start 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 10 monclient: init 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 10 monclient: _add_conns ranks=[0,6,8,7,4,1,5,2,3] 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c380741f0 0x7f8c381d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 10 monclient(hunting): picked mon.noname-a con 0x7f8c380741f0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8c38135970 0x7f8c38134160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 10 monclient(hunting): picked mon.noname-d con 0x7f8c38135970 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8c381346a0 0x7f8c381d5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 10 monclient(hunting): picked mon.noname-e con 0x7f8c381346a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8c380745d0 con 0x7f8c380741f0 2024-08-19T10:45:30.349 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8c380703b0 con 0x7f8c38135970 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f8c3813ef80 con 0x7f8c381346a0 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c3d863640 10 monclient(hunting): authenticate will time out at 1375.917515s 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c36ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c380741f0 0x7f8c381d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c36ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c380741f0 0x7f8c381d1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:34340/0 (socket says 172.21.15.190:34340) 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c36ffd640 1 -- 172.21.15.190:0/2242913925 learned_addr learned my addr 172.21.15.190:0/2242913925 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c36ffd640 10 monclient(hunting): get_auth_request con 0x7f8c380741f0 auth_method 0 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c36ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c36ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8c38135970 0x7f8c38134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:30.350 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 10 monclient(hunting): get_auth_request con 0x7f8c38135970 auth_method 0 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 10 monclient(hunting): handle_auth_done global_id 44367 payload 824 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 1 -- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8c381346a0 msgr2=0x7f8c381d5510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8c381346a0 0x7f8c381d5510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 1 -- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c380741f0 msgr2=0x7f8c381d1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:30.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c380741f0 0x7f8c381d1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:30.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:30.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 1 monclient: found mon.noname-d 2024-08-19T10:45:30.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:30.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:45:30.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c2ffff640 1 -- 172.21.15.190:0/2242913925 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c38127b80 con 0x7f8c38135970 2024-08-19T10:45:30.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.345+0000 7f8c36ffd640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c380741f0 0x7f8c381d1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:30.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c2ffff640 1 -- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8c38135970 msgr2=0x7f8c38134160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:30.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c2ffff640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8c38135970 0x7f8c38134160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000c7a0 tx=0x7f8c2000cc60 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:30.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 1 -- 172.21.15.190:0/2242913925 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c20007bd0 con 0x7f8c38135970 2024-08-19T10:45:30.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:30.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:45:30.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 10 monclient: dump: 2024-08-19T10:45:30.354 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:30.354 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:30.355 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 10 monclient: _finish_auth 0 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c3d863640 5 monclient: authenticate success, global_id 44367 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 1 -- 172.21.15.190:0/2242913925 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c20007d90 con 0x7f8c38135970 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c3d863640 10 monclient: _renew_subs 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c3d863640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c3d863640 1 -- 172.21.15.190:0/2242913925 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3813c000 con 0x7f8c38135970 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 1 -- 172.21.15.190:0/2242913925 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8c2000ed00 con 0x7f8c38135970 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:45:30.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c34ff9640 10 monclient: dump: 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:30.357 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c3d863640 10 monclient: _renew_subs 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c3d863640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c3d863640 1 -- 172.21.15.190:0/2242913925 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c381278a0 con 0x7f8c38135970 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c3d863640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c3d863640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c3d863640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.349+0000 7f8c3d863640 1 -- 172.21.15.190:0/2242913925 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c38073cb0 con 0x7f8c38135970 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.353+0000 7f8c357fa640 4 set_mon_vals no callback set 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.353+0000 7f8c34ff9640 1 -- 172.21.15.190:0/2242913925 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8c20015020 con 0x7f8c38135970 2024-08-19T10:45:30.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.353+0000 7f8c34ff9640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8c1c03be00 0x7f8c1c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:30.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.353+0000 7f8c36ffd640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8c1c03be00 0x7f8c1c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:30.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.353+0000 7f8c34ff9640 1 -- 172.21.15.190:0/2242913925 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(247..247 src has 150..247) ==== 11177+0+0 (secure 0 0 0) 0x7f8c20050cf0 con 0x7f8c38135970 2024-08-19T10:45:30.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.353+0000 7f8c36ffd640 10 monclient: get_auth_request con 0x7f8c1c03be00 auth_method 0 2024-08-19T10:45:30.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.357+0000 7f8c36ffd640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8c1c03be00 0x7f8c1c03e2c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c28020fd0 tx=0x7f8c280ab000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:30.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.365+0000 7f8c34ff9640 1 -- 172.21.15.190:0/2242913925 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8c20012030 con 0x7f8c38135970 2024-08-19T10:45:30.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.365+0000 7f8c34ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:30.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.365+0000 7f8c34ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.541+0000 7f8c3d863640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h19"}] 2024-08-19T10:45:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.541+0000 7f8c3d863640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h19"}] 2024-08-19T10:45:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.541+0000 7f8c3d863640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:30.543 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:30.541+0000 7f8c3d863640 1 -- 172.21.15.190:0/2242913925 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h19"} v 0) -- 0x7f8c380008e0 con 0x7f8c38135970 2024-08-19T10:45:32.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.385+0000 7f8c34ff9640 1 -- 172.21.15.190:0/2242913925 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h19"}]=0 removed item id -73 name 'h19' from crush map v248) ==== 120+0+0 (secure 0 0 0) 0x7f8c2004dde0 con 0x7f8c38135970 2024-08-19T10:45:32.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.385+0000 7f8c34ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h19"}] 2024-08-19T10:45:32.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.385+0000 7f8c34ff9640 10 monclient: _finish_command 2 = system:0 removed item id -73 name 'h19' from crush map 2024-08-19T10:45:32.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.493+0000 7f8c3d863640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h19"}] 2024-08-19T10:45:32.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.493+0000 7f8c3d863640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h19"}] 2024-08-19T10:45:32.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.493+0000 7f8c3d863640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:32.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.493+0000 7f8c3d863640 1 -- 172.21.15.190:0/2242913925 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h19"} v 0) -- 0x7f8c3806db30 con 0x7f8c38135970 2024-08-19T10:45:32.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.505+0000 7f8c34ff9640 1 -- 172.21.15.190:0/2242913925 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h19"}]=0 device 'h19' does not appear in the crush map v248) ==== 120+0+0 (secure 0 0 0) 0x7f8c20015d40 con 0x7f8c38135970 2024-08-19T10:45:32.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.505+0000 7f8c34ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h19"}] 2024-08-19T10:45:32.508 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.505+0000 7f8c34ff9640 10 monclient: _finish_command 3 = system:0 device 'h19' does not appear in the crush map 2024-08-19T10:45:32.509 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h19' does not appear in the crush map 2024-08-19T10:45:32.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 -- 172.21.15.190:0/2242913925 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8c1c03be00 msgr2=0x7f8c1c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:32.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8c1c03be00 0x7f8c1c03e2c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c28020fd0 tx=0x7f8c280ab000 comp rx=0 tx=0).stop 2024-08-19T10:45:32.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 10 monclient: shutdown 2024-08-19T10:45:32.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:32.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 -- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8c38135970 msgr2=0x7f8c38134160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:32.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8c38135970 0x7f8c38134160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000c7a0 tx=0x7f8c2000cc60 comp rx=0 tx=0).stop 2024-08-19T10:45:32.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 -- 172.21.15.190:0/2242913925 shutdown_connections 2024-08-19T10:45:32.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8c1c03be00 0x7f8c1c03e2c0 secure :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c28020fd0 tx=0x7f8c280ab000 comp rx=0 tx=0).stop 2024-08-19T10:45:32.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8c381346a0 0x7f8c381d5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:32.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8c38135970 0x7f8c38134160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:32.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 --2- 172.21.15.190:0/2242913925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c380741f0 0x7f8c381d1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:32.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 -- 172.21.15.190:0/2242913925 >> 172.21.15.190:0/2242913925 conn(0x7f8c380a4700 msgr2=0x7f8c38135070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:32.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 -- 172.21.15.190:0/2242913925 shutdown_connections 2024-08-19T10:45:32.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.509+0000 7f8c3d863640 1 -- 172.21.15.190:0/2242913925 wait complete. 2024-08-19T10:45:32.526 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:32.527 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h20 2024-08-19T10:45:32.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient: get_monmap_and_config 2024-08-19T10:45:32.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient: build_initial_monmap 2024-08-19T10:45:32.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient: monmap: 2024-08-19T10:45:32.646 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:32.646 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:32.646 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:32.648664+0000 2024-08-19T10:45:32.646 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:32.648664+0000 2024-08-19T10:45:32.646 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:32.646 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:32.646 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:32.647 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:32.647 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:32.647 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:32.647 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:32.647 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:32.647 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:32.647 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:32.647 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:32.647 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:32.647 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:32.647 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:32.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 1 Processor -- start 2024-08-19T10:45:32.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 1 -- start start 2024-08-19T10:45:32.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient: init 2024-08-19T10:45:32.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:32.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient: _add_conns ranks=[7,5,3,10,1,2,4,6,0,8,9] 2024-08-19T10:45:32.650 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc970139980 0x7fc970139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient(hunting): picked mon.noname-i con 0x7fc970139980 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc97013a2a0 0x7fc9700a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient(hunting): picked mon.noname-h con 0x7fc97013a2a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9700a5c70 0x7fc9700a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient(hunting): picked mon.noname-g con 0x7fc9700a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc97009ca60 con 0x7fc9700a5c70 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc9700a3640 con 0x7fc97013a2a0 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc9700a8470 con 0x7fc970139980 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:32.651 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc975442640 10 monclient(hunting): authenticate will time out at 1378.218673s 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9700a5c70 0x7fc9700a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9700a5c70 0x7fc9700a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:41858/0 (socket says 172.21.15.190:41858) 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 1 -- 172.21.15.190:0/91285366 learned_addr learned my addr 172.21.15.190:0/91285366 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96e7fc640 1 --2- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc97013a2a0 0x7fc9700a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 10 monclient(hunting): get_auth_request con 0x7fc9700a5c70 auth_method 0 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96e7fc640 10 monclient(hunting): get_auth_request con 0x7fc97013a2a0 auth_method 0 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:32.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96effd640 1 --2- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc970139980 0x7fc970139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96effd640 10 monclient(hunting): get_auth_request con 0x7fc970139980 auth_method 0 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:32.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 10 monclient(hunting): handle_auth_done global_id 34860 payload 274 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 1 -- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc970139980 msgr2=0x7fc970139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 1 --2- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc970139980 0x7fc970139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 1 -- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc97013a2a0 msgr2=0x7fc9700a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 1 --2- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc97013a2a0 0x7fc9700a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 1 monclient: found mon.noname-g 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96f7fe640 1 -- 172.21.15.190:0/91285366 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9700a3850 con 0x7fc9700a5c70 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96e7fc640 1 --2- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc97013a2a0 0x7fc9700a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:32.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.645+0000 7fc96effd640 1 --2- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc970139980 0x7fc970139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:32.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96f7fe640 1 -- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9700a5c70 msgr2=0x7fc9700a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:32.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96f7fe640 1 --2- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9700a5c70 0x7fc9700a67e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc968009eb0 tx=0x7fc96801c600 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c3d4b440f3bcb727 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:32.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 1 -- 172.21.15.190:0/91285366 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9680ac020 con 0x7fc9700a5c70 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 10 monclient: dump: 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:32.661 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 1 -- 172.21.15.190:0/91285366 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc968009110 con 0x7fc9700a5c70 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc975442640 5 monclient: authenticate success, global_id 34860 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc975442640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:32.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 1 -- 172.21.15.190:0/91285366 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9680a6cf0 con 0x7fc9700a5c70 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc975442640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc975442640 4 set_mon_vals no callback set 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.649+0000 7fc96dffb640 10 monclient: dump: 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:32.663 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient: shutdown 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 -- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9700a5c70 msgr2=0x7fc9700a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 --2- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9700a5c70 0x7fc9700a67e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc968009eb0 tx=0x7fc96801c600 comp rx=0 tx=0).stop 2024-08-19T10:45:32.664 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 -- 172.21.15.190:0/91285366 shutdown_connections 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 --2- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc97013a2a0 0x7fc9700a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 --2- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9700a5c70 0x7fc9700a67e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 --2- 172.21.15.190:0/91285366 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc970139980 0x7fc970139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 -- 172.21.15.190:0/91285366 >> 172.21.15.190:0/91285366 conn(0x7fc9700abde0 msgr2=0x7fc9700ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 -- 172.21.15.190:0/91285366 shutdown_connections 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 -- 172.21.15.190:0/91285366 wait complete. 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient: build_initial_monmap 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient: monmap: 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:32.665 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 Processor -- start 2024-08-19T10:45:32.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 -- start start 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient: init 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient: _add_conns ranks=[8,6,4,5,2,0,7,1,3] 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9700a5c70 0x7fc970134d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient(hunting): picked mon.noname-e con 0x7fc9700a5c70 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc970139980 0x7fc970135240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient(hunting): picked mon.noname-d con 0x7fc970139980 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97013a690 0x7fc97013aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient(hunting): picked mon.noname-c con 0x7fc97013a690 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc970143270 con 0x7fc97013a690 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:32.667 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fc9701430f0 con 0x7fc970139980 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc9701433f0 con 0x7fc9700a5c70 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc975442640 10 monclient(hunting): authenticate will time out at 1378.226076s 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96e7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc970139980 0x7fc970135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96f7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97013a690 0x7fc97013aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9700a5c70 0x7fc970134d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96e7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc970139980 0x7fc970135240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:33638/0 (socket says 172.21.15.190:33638) 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96e7fc640 1 -- 172.21.15.190:0/3709136347 learned_addr learned my addr 172.21.15.190:0/3709136347 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96e7fc640 10 monclient(hunting): get_auth_request con 0x7fc970139980 auth_method 0 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:32.668 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96f7fe640 10 monclient(hunting): get_auth_request con 0x7fc97013a690 auth_method 0 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 10 monclient(hunting): get_auth_request con 0x7fc9700a5c70 auth_method 0 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:32.669 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 10 monclient(hunting): handle_auth_done global_id 6326 payload 824 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 1 -- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc970139980 msgr2=0x7fc970135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc970139980 0x7fc970135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 1 -- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97013a690 msgr2=0x7fc97013aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97013a690 0x7fc97013aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 10 monclient: _finish_hunting 0 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 1 monclient: found mon.noname-e 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:32.670 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.653+0000 7fc96effd640 1 -- 172.21.15.190:0/3709136347 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9700a5790 con 0x7fc9700a5c70 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc96effd640 1 -- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9700a5c70 msgr2=0x7fc970134d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc96effd640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9700a5c70 0x7fc970134d00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc96000e9e0 tx=0x7fc96000eea0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 1 -- 172.21.15.190:0/3709136347 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc960017070 con 0x7fc9700a5c70 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 10 monclient: dump: 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:32.671 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:32.672 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc975442640 5 monclient: authenticate success, global_id 6326 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc975442640 10 monclient: _renew_subs 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc975442640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc975442640 1 -- 172.21.15.190:0/3709136347 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97013be90 con 0x7fc9700a5c70 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc975442640 10 monclient: _renew_subs 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 1 -- 172.21.15.190:0/3709136347 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96000cbe0 con 0x7fc9700a5c70 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc96effd640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9700a5c70 0x7fc970134d00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc96000e9e0 tx=0x7fc96000eea0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fc96000cd90 mon_map magic: 0 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc975442640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc975442640 1 -- 172.21.15.190:0/3709136347 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97013c9f0 con 0x7fc9700a5c70 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc96d7fa640 4 set_mon_vals no callback set 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc975442640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc975442640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:32.673 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc975442640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc975442640 1 -- 172.21.15.190:0/3709136347 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc970139d60 con 0x7fc9700a5c70 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 1 -- 172.21.15.190:0/3709136347 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc96000cd90 con 0x7fc9700a5c70 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 10 monclient: dump: 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:32.674 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 1 -- 172.21.15.190:0/3709136347 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc96001a070 con 0x7fc9700a5c70 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc9577fe640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc95c03be50 0x7fc95c03e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc96e7fc640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc95c03be50 0x7fc95c03e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:32.675 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.657+0000 7fc96e7fc640 10 monclient: get_auth_request con 0x7fc95c03be50 auth_method 0 2024-08-19T10:45:32.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.661+0000 7fc96e7fc640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc95c03be50 0x7fc95c03e310 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc964009600 tx=0x7fc964009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:32.676 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:32.673+0000 7fc9577fe640 1 -- 172.21.15.190:0/3709136347 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(248..248 src has 150..248) ==== 10964+0+0 (secure 0 0 0) 0x7fc96002ee00 con 0x7fc9700a5c70 2024-08-19T10:45:33.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:33.485+0000 7fc9577fe640 1 -- 172.21.15.190:0/3709136347 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc960021070 con 0x7fc9700a5c70 2024-08-19T10:45:33.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:33.485+0000 7fc9577fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:33.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:33.485+0000 7fc9577fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:33.678 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:33.677+0000 7fc975442640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h20"}] 2024-08-19T10:45:33.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:33.677+0000 7fc975442640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h20"}] 2024-08-19T10:45:33.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:33.677+0000 7fc975442640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:33.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:33.677+0000 7fc975442640 1 -- 172.21.15.190:0/3709136347 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush rm", "name": "h20"} v 0) -- 0x7fc97006c8d0 con 0x7fc9700a5c70 2024-08-19T10:45:34.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.521+0000 7fc9577fe640 1 -- 172.21.15.190:0/3709136347 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h20"}]=0 removed item id -76 name 'h20' from crush map v249) ==== 120+0+0 (secure 0 0 0) 0x7fc960009300 con 0x7fc9700a5c70 2024-08-19T10:45:34.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.521+0000 7fc9577fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h20"}] 2024-08-19T10:45:34.522 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.521+0000 7fc9577fe640 10 monclient: _finish_command 2 = system:0 removed item id -76 name 'h20' from crush map 2024-08-19T10:45:34.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.629+0000 7fc975442640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h20"}] 2024-08-19T10:45:34.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.629+0000 7fc975442640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h20"}] 2024-08-19T10:45:34.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.629+0000 7fc975442640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:34.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.629+0000 7fc975442640 1 -- 172.21.15.190:0/3709136347 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush rm", "name": "h20"} v 0) -- 0x7fc97013ebf0 con 0x7fc9700a5c70 2024-08-19T10:45:34.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.633+0000 7fc9577fe640 1 -- 172.21.15.190:0/3709136347 <== mon.8 v2:172.21.15.89:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h20"}]=0 device 'h20' does not appear in the crush map v249) ==== 120+0+0 (secure 0 0 0) 0x7fc96001bb00 con 0x7fc9700a5c70 2024-08-19T10:45:34.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.633+0000 7fc9577fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h20"}] 2024-08-19T10:45:34.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.633+0000 7fc9577fe640 10 monclient: _finish_command 3 = system:0 device 'h20' does not appear in the crush map 2024-08-19T10:45:34.635 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h20' does not appear in the crush map 2024-08-19T10:45:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 -- 172.21.15.190:0/3709136347 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc95c03be50 msgr2=0x7fc95c03e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc95c03be50 0x7fc95c03e310 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc964009600 tx=0x7fc964009190 comp rx=0 tx=0).stop 2024-08-19T10:45:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 10 monclient: shutdown 2024-08-19T10:45:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 -- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9700a5c70 msgr2=0x7fc970134d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:34.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9700a5c70 0x7fc970134d00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc96000e9e0 tx=0x7fc96000eea0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 -- 172.21.15.190:0/3709136347 shutdown_connections 2024-08-19T10:45:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc97013a690 0x7fc97013aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fc970139980 0x7fc970135240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc95c03be50 0x7fc95c03e310 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 --2- 172.21.15.190:0/3709136347 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc9700a5c70 0x7fc970134d00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 -- 172.21.15.190:0/3709136347 >> 172.21.15.190:0/3709136347 conn(0x7fc9700abde0 msgr2=0x7fc970138cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 -- 172.21.15.190:0/3709136347 shutdown_connections 2024-08-19T10:45:34.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.637+0000 7fc975442640 1 -- 172.21.15.190:0/3709136347 wait complete. 2024-08-19T10:45:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:34.653 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h21 2024-08-19T10:45:34.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient: get_monmap_and_config 2024-08-19T10:45:34.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient: build_initial_monmap 2024-08-19T10:45:34.766 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient: monmap: 2024-08-19T10:45:34.766 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:34.766 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:34.766 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:34.769007+0000 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:34.769007+0000 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:34.767 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:34.768 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:34.768 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:34.769 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 1 Processor -- start 2024-08-19T10:45:34.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 1 -- start start 2024-08-19T10:45:34.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient: init 2024-08-19T10:45:34.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:34.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient: _add_conns ranks=[8,4,3,6,5,9,0,7,1,10,2] 2024-08-19T10:45:34.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0104139980 0x7f0104139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:34.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient(hunting): picked mon.noname-e con 0x7f0104139980 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:34.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f010413a2a0 0x7f01040a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:34.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient(hunting): picked mon.noname-c con 0x7f010413a2a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:34.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01040a5c70 0x7f01040a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:34.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient(hunting): picked mon.noname-g con 0x7f01040a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:34.770 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f010409ca60 con 0x7f01040a5c70 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f01040a3640 con 0x7f010413a2a0 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f01040a8470 con 0x7f0104139980 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010c242640 10 monclient(hunting): authenticate will time out at 1380.338898s 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01040a5c70 0x7f01040a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01040a5c70 0x7f01040a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:41870/0 (socket says 172.21.15.190:41870) 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 1 -- 172.21.15.190:0/392958138 learned_addr learned my addr 172.21.15.190:0/392958138 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 10 monclient(hunting): get_auth_request con 0x7f01040a5c70 auth_method 0 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:34.771 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f01097dd640 1 --2- 172.21.15.190:0/392958138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f010413a2a0 0x7f01040a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f0109fde640 1 --2- 172.21.15.190:0/392958138 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0104139980 0x7f0104139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f01097dd640 10 monclient(hunting): get_auth_request con 0x7f010413a2a0 auth_method 0 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f01097dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f01097dd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f01097dd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f0109fde640 10 monclient(hunting): get_auth_request con 0x7f0104139980 auth_method 0 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f0109fde640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f0109fde640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:34.772 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f0109fde640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:34.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 10 monclient(hunting): handle_auth_done global_id 34869 payload 274 2024-08-19T10:45:34.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 1 -- 172.21.15.190:0/392958138 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0104139980 msgr2=0x7f0104139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:34.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 1 --2- 172.21.15.190:0/392958138 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0104139980 0x7f0104139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 1 -- 172.21.15.190:0/392958138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f010413a2a0 msgr2=0x7f01040a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:34.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 1 --2- 172.21.15.190:0/392958138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f010413a2a0 0x7f01040a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 10 monclient: _finish_hunting 0 2024-08-19T10:45:34.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 1 monclient: found mon.noname-g 2024-08-19T10:45:34.773 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:34.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:34.774 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.765+0000 7f010a7df640 1 -- 172.21.15.190:0/392958138 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01040a3850 con 0x7f01040a5c70 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.769+0000 7f010a7df640 1 -- 172.21.15.190:0/392958138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01040a5c70 msgr2=0x7f01040a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 1 --2- 172.21.15.190:0/392958138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01040a5c70 0x7f01040a67e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f00f80098d0 tx=0x7f00f801c150 comp rx=0 tx=0).ready entity=mon.3 client_cookie=804d1508880dbd1b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 1 -- 172.21.15.190:0/392958138 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00f80b0070 con 0x7f01040a5c70 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 10 monclient: dump: 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:34.780 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 10 monclient: _finish_auth 0 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:34.781 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 1 -- 172.21.15.190:0/392958138 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f80aa810 con 0x7f01040a5c70 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 1 -- 172.21.15.190:0/392958138 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00f80aae30 con 0x7f01040a5c70 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 5 monclient: authenticate success, global_id 34869 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 4 set_mon_vals no callback set 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0108fdc640 10 monclient: dump: 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:34.782 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient: shutdown 2024-08-19T10:45:34.783 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:34.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 -- 172.21.15.190:0/392958138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01040a5c70 msgr2=0x7f01040a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:34.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 --2- 172.21.15.190:0/392958138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01040a5c70 0x7f01040a67e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f00f80098d0 tx=0x7f00f801c150 comp rx=0 tx=0).stop 2024-08-19T10:45:34.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 -- 172.21.15.190:0/392958138 shutdown_connections 2024-08-19T10:45:34.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 --2- 172.21.15.190:0/392958138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f01040a5c70 0x7f01040a67e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 --2- 172.21.15.190:0/392958138 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f010413a2a0 0x7f01040a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.784 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 --2- 172.21.15.190:0/392958138 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0104139980 0x7f0104139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 -- 172.21.15.190:0/392958138 >> 172.21.15.190:0/392958138 conn(0x7f01040abde0 msgr2=0x7f01040ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 -- 172.21.15.190:0/392958138 shutdown_connections 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 -- 172.21.15.190:0/392958138 wait complete. 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient: build_initial_monmap 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient: monmap: 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:34.785 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 Processor -- start 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 -- start start 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient: init 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:34.786 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient: _add_conns ranks=[0,6,1,7,2,5,8,4,3] 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01040a5c70 0x7f01041347e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient(hunting): picked mon.noname-a con 0x7f01040a5c70 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0104139980 0x7f0104134d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient(hunting): picked mon.noname-d con 0x7f0104139980 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010413a690 0x7f010413aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient(hunting): picked mon.noname-f con 0x7f010413a690 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0104143270 con 0x7f01040a5c70 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f01041430f0 con 0x7f010413a690 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:34.787 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f01041433f0 con 0x7f0104139980 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010c242640 10 monclient(hunting): authenticate will time out at 1380.346473s 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010413a690 0x7f010413aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010413a690 0x7f010413aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:52890/0 (socket says 172.21.15.190:52890) 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 1 -- 172.21.15.190:0/4115860950 learned_addr learned my addr 172.21.15.190:0/4115860950 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 10 monclient(hunting): get_auth_request con 0x7f010413a690 auth_method 0 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:34.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f01097dd640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0104139980 0x7f0104134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0109fde640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01040a5c70 0x7f01041347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f01097dd640 10 monclient(hunting): get_auth_request con 0x7f0104139980 auth_method 0 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f01097dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f01097dd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f01097dd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0109fde640 10 monclient(hunting): get_auth_request con 0x7f01040a5c70 auth_method 0 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0109fde640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0109fde640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f0109fde640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 10 monclient(hunting): handle_auth_done global_id 44272 payload 824 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 1 -- 172.21.15.190:0/4115860950 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0104139980 msgr2=0x7f0104134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0104139980 0x7f0104134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.789 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 1 -- 172.21.15.190:0/4115860950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01040a5c70 msgr2=0x7f01041347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:34.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01040a5c70 0x7f01041347e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:34.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 10 monclient: _finish_hunting 0 2024-08-19T10:45:34.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 1 monclient: found mon.noname-f 2024-08-19T10:45:34.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:34.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:45:34.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.773+0000 7f010a7df640 1 -- 172.21.15.190:0/4115860950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010413dda0 con 0x7f010413a690 2024-08-19T10:45:34.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010a7df640 1 -- 172.21.15.190:0/4115860950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010413a690 msgr2=0x7f010413aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:34.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010a7df640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010413a690 0x7f010413aad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f00f801c190 tx=0x7f00f8018ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:34.790 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 1 -- 172.21.15.190:0/4115860950 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00f80b0040 con 0x7f010413a690 2024-08-19T10:45:34.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:34.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:45:34.791 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 10 monclient: dump: 2024-08-19T10:45:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:34.794 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:34.795 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:34.795 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:34.795 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:34.795 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:34.795 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:34.795 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:34.795 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:34.795 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:34.795 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:34.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:34.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:34.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:34.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010c242640 5 monclient: authenticate success, global_id 44272 2024-08-19T10:45:34.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 1 -- 172.21.15.190:0/4115860950 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f8004250 con 0x7f010413a690 2024-08-19T10:45:34.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010c242640 10 monclient: _renew_subs 2024-08-19T10:45:34.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010c242640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:34.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010c242640 1 -- 172.21.15.190:0/4115860950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010413cb60 con 0x7f010413a690 2024-08-19T10:45:34.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:34.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 1 -- 172.21.15.190:0/4115860950 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00f8004400 con 0x7f010413a690 2024-08-19T10:45:34.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:34.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:34.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f0103fff640 4 set_mon_vals no callback set 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 10 monclient: dump: 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:34.798 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:34.799 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010c242640 10 monclient: _renew_subs 2024-08-19T10:45:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010c242640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010c242640 1 -- 172.21.15.190:0/4115860950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01040a57b0 con 0x7f010413a690 2024-08-19T10:45:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010c242640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010c242640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:34.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010c242640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f010c242640 1 -- 172.21.15.190:0/4115860950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01040a6870 con 0x7f010413a690 2024-08-19T10:45:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 1 -- 172.21.15.190:0/4115860950 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f00f8003620 con 0x7f010413a690 2024-08-19T10:45:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f00f003be00 0x7f00f003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f01027fc640 1 -- 172.21.15.190:0/4115860950 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(249..249 src has 150..249) ==== 10751+0+0 (secure 0 0 0) 0x7f00f80ea000 con 0x7f010413a690 2024-08-19T10:45:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f0109fde640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f00f003be00 0x7f00f003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:34.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.777+0000 7f0109fde640 10 monclient: get_auth_request con 0x7f00f003be00 auth_method 0 2024-08-19T10:45:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.781+0000 7f0109fde640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f00f003be00 0x7f00f003e2c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f00f4003fd0 tx=0x7f00f40039a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.781+0000 7f01027fc640 1 -- 172.21.15.190:0/4115860950 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f00f8016150 con 0x7f010413a690 2024-08-19T10:45:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.781+0000 7f01027fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:34.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.781+0000 7f01027fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:34.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.957+0000 7f010c242640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h21"}] 2024-08-19T10:45:34.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.957+0000 7f010c242640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h21"}] 2024-08-19T10:45:34.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.957+0000 7f010c242640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:34.959 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:34.957+0000 7f010c242640 1 -- 172.21.15.190:0/4115860950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h21"} v 0) -- 0x7f010413e450 con 0x7f010413a690 2024-08-19T10:45:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.545+0000 7f01027fc640 1 -- 172.21.15.190:0/4115860950 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h21"}]=0 removed item id -79 name 'h21' from crush map v250) ==== 120+0+0 (secure 0 0 0) 0x7f00f80b2dd0 con 0x7f010413a690 2024-08-19T10:45:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.545+0000 7f01027fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h21"}] 2024-08-19T10:45:35.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.545+0000 7f01027fc640 10 monclient: _finish_command 2 = system:0 removed item id -79 name 'h21' from crush map 2024-08-19T10:45:35.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.653+0000 7f010c242640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h21"}] 2024-08-19T10:45:35.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.653+0000 7f010c242640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h21"}] 2024-08-19T10:45:35.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.653+0000 7f010c242640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:35.655 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.653+0000 7f010c242640 1 -- 172.21.15.190:0/4115860950 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h21"} v 0) -- 0x7f0104139d60 con 0x7f010413a690 2024-08-19T10:45:35.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f01027fc640 1 -- 172.21.15.190:0/4115860950 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h21"}]=0 device 'h21' does not appear in the crush map v250) ==== 120+0+0 (secure 0 0 0) 0x7f00f80e72a0 con 0x7f010413a690 2024-08-19T10:45:35.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f01027fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h21"}] 2024-08-19T10:45:35.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f01027fc640 10 monclient: _finish_command 3 = system:0 device 'h21' does not appear in the crush map 2024-08-19T10:45:35.657 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h21' does not appear in the crush map 2024-08-19T10:45:35.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 -- 172.21.15.190:0/4115860950 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f00f003be00 msgr2=0x7f00f003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:35.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f00f003be00 0x7f00f003e2c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f00f4003fd0 tx=0x7f00f40039a0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 10 monclient: shutdown 2024-08-19T10:45:35.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:35.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 -- 172.21.15.190:0/4115860950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010413a690 msgr2=0x7f010413aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:35.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010413a690 0x7f010413aad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f00f801c190 tx=0x7f00f8018ef0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 -- 172.21.15.190:0/4115860950 shutdown_connections 2024-08-19T10:45:35.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f00f003be00 0x7f00f003e2c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.660 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f010413a690 0x7f010413aad0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0104139980 0x7f0104134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 --2- 172.21.15.190:0/4115860950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01040a5c70 0x7f01041347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 -- 172.21.15.190:0/4115860950 >> 172.21.15.190:0/4115860950 conn(0x7f01040abde0 msgr2=0x7f0104138d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:35.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 -- 172.21.15.190:0/4115860950 shutdown_connections 2024-08-19T10:45:35.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.657+0000 7f010c242640 1 -- 172.21.15.190:0/4115860950 wait complete. 2024-08-19T10:45:35.675 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:35.676 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h22 2024-08-19T10:45:35.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.789+0000 7f4f29caa640 10 monclient: get_monmap_and_config 2024-08-19T10:45:35.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.789+0000 7f4f29caa640 10 monclient: build_initial_monmap 2024-08-19T10:45:35.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.789+0000 7f4f29caa640 10 monclient: monmap: 2024-08-19T10:45:35.792 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:35.792 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:35.792 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:35.794665+0000 2024-08-19T10:45:35.792 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:35.794665+0000 2024-08-19T10:45:35.792 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:35.793 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:35.794 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:35.794 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.789+0000 7f4f29caa640 1 Processor -- start 2024-08-19T10:45:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 1 -- start start 2024-08-19T10:45:35.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 10 monclient: init 2024-08-19T10:45:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 10 monclient: _add_conns ranks=[5,8,2,0,9,1,3,7,10,6,4] 2024-08-19T10:45:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f240a3ae0 0x7f4f240a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 10 monclient(hunting): picked mon.noname-h con 0x7f4f240a3ae0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4f240a4400 0x7f4f2413d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 10 monclient(hunting): picked mon.noname-e con 0x7f4f240a4400 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f2413ded0 0x7f4f241402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 10 monclient(hunting): picked mon.noname-b con 0x7f4f2413ded0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:35.797 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2406d780 con 0x7f4f2413ded0 2024-08-19T10:45:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4f240a0dc0 con 0x7f4f240a3ae0 2024-08-19T10:45:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4f24074510 con 0x7f4f240a4400 2024-08-19T10:45:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f29caa640 10 monclient(hunting): authenticate will time out at 1381.364715s 2024-08-19T10:45:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f240a3ae0 0x7f4f240a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f23fff640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f2413ded0 msgr2=0x7f4f241402c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f240a3ae0 0x7f4f240a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:56486/0 (socket says 172.21.15.190:56486) 2024-08-19T10:45:35.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f23fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f2413ded0 0x7f4f241402c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 1 -- 172.21.15.190:0/742822193 learned_addr learned my addr 172.21.15.190:0/742822193 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 10 monclient(hunting): get_auth_request con 0x7f4f240a3ae0 auth_method 0 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f22ffd640 1 --2- 172.21.15.190:0/742822193 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4f240a4400 0x7f4f2413d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f22ffd640 10 monclient(hunting): get_auth_request con 0x7f4f240a4400 auth_method 0 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f22ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:35.799 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f22ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:35.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f22ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:35.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 10 monclient(hunting): handle_auth_done global_id 6413 payload 274 2024-08-19T10:45:35.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 1 -- 172.21.15.190:0/742822193 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4f240a4400 msgr2=0x7f4f2413d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:35.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 1 --2- 172.21.15.190:0/742822193 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4f240a4400 0x7f4f2413d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 1 -- 172.21.15.190:0/742822193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f2413ded0 msgr2=0x7f4f241402c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:35.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 1 --2- 172.21.15.190:0/742822193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f2413ded0 0x7f4f241402c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:35.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 1 monclient: found mon.noname-h 2024-08-19T10:45:35.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:35.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:35.800 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.793+0000 7f4f237fe640 1 -- 172.21.15.190:0/742822193 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f240a05e0 con 0x7f4f240a3ae0 2024-08-19T10:45:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f237fe640 1 -- 172.21.15.190:0/742822193 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f240a3ae0 msgr2=0x7f4f240a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f237fe640 1 --2- 172.21.15.190:0/742822193 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f240a3ae0 0x7f4f240a3ec0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4f180098a0 tx=0x7f4f1801c400 comp rx=0 tx=0).ready entity=mon.5 client_cookie=34082241b3e265c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 1 -- 172.21.15.190:0/742822193 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4f180b0070 con 0x7f4f240a3ae0 2024-08-19T10:45:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 10 monclient: dump: 2024-08-19T10:45:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:35.806 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:35.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 5 monclient: authenticate success, global_id 6413 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 1 -- 172.21.15.190:0/742822193 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f180aa810 con 0x7f4f240a3ae0 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 1 -- 172.21.15.190:0/742822193 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4f180aae30 con 0x7f4f240a3ae0 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f227fc640 10 monclient: dump: 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:35.808 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:35.809 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:35.810 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 4 set_mon_vals no callback set 2024-08-19T10:45:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 10 monclient: shutdown 2024-08-19T10:45:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 1 -- 172.21.15.190:0/742822193 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f240a3ae0 msgr2=0x7f4f240a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 1 --2- 172.21.15.190:0/742822193 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f240a3ae0 0x7f4f240a3ec0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4f180098a0 tx=0x7f4f1801c400 comp rx=0 tx=0).stop 2024-08-19T10:45:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 1 -- 172.21.15.190:0/742822193 shutdown_connections 2024-08-19T10:45:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 1 --2- 172.21.15.190:0/742822193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4f2413ded0 0x7f4f241402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 1 --2- 172.21.15.190:0/742822193 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4f240a4400 0x7f4f2413d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 1 --2- 172.21.15.190:0/742822193 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f240a3ae0 0x7f4f240a3ec0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 1 -- 172.21.15.190:0/742822193 >> 172.21.15.190:0/742822193 conn(0x7f4f2406dde0 msgr2=0x7f4f2406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 1 -- 172.21.15.190:0/742822193 shutdown_connections 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.797+0000 7f4f29caa640 1 -- 172.21.15.190:0/742822193 wait complete. 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient: build_initial_monmap 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient: monmap: 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:35.811 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 1 Processor -- start 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 1 -- start start 2024-08-19T10:45:35.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient: init 2024-08-19T10:45:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient: _add_conns ranks=[3,7,5,8,1,4,6,2,0] 2024-08-19T10:45:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240a3ae0 0x7f4f2413a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient(hunting): picked mon.noname-g con 0x7f4f240a3ae0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4f240a4400 0x7f4f2413aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient(hunting): picked mon.noname-i con 0x7f4f240a4400 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f2413ded0 0x7f4f24123810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient(hunting): picked mon.noname-h con 0x7f4f2413ded0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4f240a47e0 con 0x7f4f240a3ae0 2024-08-19T10:45:35.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:35.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4f2406cbc0 con 0x7f4f2413ded0 2024-08-19T10:45:35.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:35.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4f241249b0 con 0x7f4f240a4400 2024-08-19T10:45:35.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:35.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f29caa640 10 monclient(hunting): authenticate will time out at 1381.373167s 2024-08-19T10:45:35.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240a3ae0 0x7f4f2413a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:35.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f23fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f2413ded0 0x7f4f24123810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:35.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f22ffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4f240a4400 0x7f4f2413aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:35.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240a3ae0 0x7f4f2413a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46550/0 (socket says 172.21.15.190:46550) 2024-08-19T10:45:35.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 1 -- 172.21.15.190:0/1545717575 learned_addr learned my addr 172.21.15.190:0/1545717575 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:35.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 10 monclient(hunting): get_auth_request con 0x7f4f240a3ae0 auth_method 0 2024-08-19T10:45:35.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:35.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:35.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:35.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f23fff640 10 monclient(hunting): get_auth_request con 0x7f4f2413ded0 auth_method 0 2024-08-19T10:45:35.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f23fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:35.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f23fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:35.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f23fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:35.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f22ffd640 10 monclient(hunting): get_auth_request con 0x7f4f240a4400 auth_method 0 2024-08-19T10:45:35.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f22ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:35.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f22ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f22ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f23fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f23fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f23fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f22ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f22ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f22ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 10 monclient(hunting): handle_auth_done global_id 34878 payload 824 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 1 -- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4f240a4400 msgr2=0x7f4f2413aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:35.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4f240a4400 0x7f4f2413aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 1 -- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f2413ded0 msgr2=0x7f4f24123810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f2413ded0 0x7f4f24123810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 1 monclient: found mon.noname-g 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f237fe640 1 -- 172.21.15.190:0/1545717575 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f24125ed0 con 0x7f4f240a3ae0 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.801+0000 7f4f23fff640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f2413ded0 0x7f4f24123810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f237fe640 1 -- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240a3ae0 msgr2=0x7f4f2413a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f237fe640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240a3ae0 0x7f4f2413a4f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f1801ab10 tx=0x7f4f18019200 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 1 -- 172.21.15.190:0/1545717575 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4f180ba070 con 0x7f4f240a3ae0 2024-08-19T10:45:35.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:35.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:35.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 10 monclient: dump: 2024-08-19T10:45:35.821 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:35.821 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:35.821 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:35.821 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:35.821 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:35.821 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:35.821 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:35.821 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:35.821 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:35.822 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:35.822 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:35.822 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:35.822 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:35.822 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:35.822 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:35.822 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:35.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 10 monclient: _finish_auth 0 2024-08-19T10:45:35.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:35.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:35.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 1 -- 172.21.15.190:0/1545717575 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f180b1040 con 0x7f4f240a3ae0 2024-08-19T10:45:35.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f29caa640 5 monclient: authenticate success, global_id 34878 2024-08-19T10:45:35.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f29caa640 10 monclient: _renew_subs 2024-08-19T10:45:35.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f29caa640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:35.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f29caa640 1 -- 172.21.15.190:0/1545717575 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f240738a0 con 0x7f4f240a3ae0 2024-08-19T10:45:35.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:35.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 1 -- 172.21.15.190:0/1545717575 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4f180b0070 con 0x7f4f240a3ae0 2024-08-19T10:45:35.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:35.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:35.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 10 monclient: dump: 2024-08-19T10:45:35.823 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:35.823 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:35.824 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:35.824 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:35.824 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:35.824 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:35.824 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:35.824 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:35.824 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:35.825 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:35.825 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:35.825 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:35.825 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:35.825 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:35.825 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:35.825 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:35.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f29caa640 10 monclient: _renew_subs 2024-08-19T10:45:35.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f21ffb640 4 set_mon_vals no callback set 2024-08-19T10:45:35.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f29caa640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:35.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f29caa640 1 -- 172.21.15.190:0/1545717575 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f2413fde0 con 0x7f4f240a3ae0 2024-08-19T10:45:35.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f29caa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:35.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f29caa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:35.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f29caa640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:35.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f29caa640 1 -- 172.21.15.190:0/1545717575 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f2413f900 con 0x7f4f240a3ae0 2024-08-19T10:45:35.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 1 -- 172.21.15.190:0/1545717575 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4f18004250 con 0x7f4f240a3ae0 2024-08-19T10:45:35.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f1003be00 0x7f4f1003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:35.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f0bfff640 1 -- 172.21.15.190:0/1545717575 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(250..250 src has 150..250) ==== 10538+0+0 (secure 0 0 0) 0x7f4f180e95c0 con 0x7f4f240a3ae0 2024-08-19T10:45:35.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f22ffd640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f1003be00 0x7f4f1003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:35.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.805+0000 7f4f22ffd640 10 monclient: get_auth_request con 0x7f4f1003be00 auth_method 0 2024-08-19T10:45:35.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.809+0000 7f4f22ffd640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f1003be00 0x7f4f1003e2c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4f24072ab0 tx=0x7f4f14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:35.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.809+0000 7f4f0bfff640 1 -- 172.21.15.190:0/1545717575 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4f18016170 con 0x7f4f240a3ae0 2024-08-19T10:45:35.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.809+0000 7f4f0bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:35.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.809+0000 7f4f0bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:35.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.985+0000 7f4f29caa640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h22"}] 2024-08-19T10:45:35.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.985+0000 7f4f29caa640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h22"}] 2024-08-19T10:45:35.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.985+0000 7f4f29caa640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:35.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:35.985+0000 7f4f29caa640 1 -- 172.21.15.190:0/1545717575 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h22"} v 0) -- 0x7f4f240a22f0 con 0x7f4f240a3ae0 2024-08-19T10:45:36.549 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.549+0000 7f4f0bfff640 1 -- 172.21.15.190:0/1545717575 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h22"}]=0 removed item id -82 name 'h22' from crush map v251) ==== 120+0+0 (secure 0 0 0) 0x7f4f18003dd0 con 0x7f4f240a3ae0 2024-08-19T10:45:36.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.549+0000 7f4f0bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h22"}] 2024-08-19T10:45:36.550 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.549+0000 7f4f0bfff640 10 monclient: _finish_command 2 = system:0 removed item id -82 name 'h22' from crush map 2024-08-19T10:45:36.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.661+0000 7f4f29caa640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h22"}] 2024-08-19T10:45:36.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.661+0000 7f4f29caa640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h22"}] 2024-08-19T10:45:36.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.661+0000 7f4f29caa640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:36.661 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.661+0000 7f4f29caa640 1 -- 172.21.15.190:0/1545717575 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h22"} v 0) -- 0x7f4f240a4f30 con 0x7f4f240a3ae0 2024-08-19T10:45:36.662 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.661+0000 7f4f0bfff640 1 -- 172.21.15.190:0/1545717575 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h22"}]=0 device 'h22' does not appear in the crush map v251) ==== 120+0+0 (secure 0 0 0) 0x7f4f180bca80 con 0x7f4f240a3ae0 2024-08-19T10:45:36.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.661+0000 7f4f0bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h22"}] 2024-08-19T10:45:36.663 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.661+0000 7f4f0bfff640 10 monclient: _finish_command 3 = system:0 device 'h22' does not appear in the crush map 2024-08-19T10:45:36.663 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h22' does not appear in the crush map 2024-08-19T10:45:36.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 -- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f1003be00 msgr2=0x7f4f1003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:36.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f1003be00 0x7f4f1003e2c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4f24072ab0 tx=0x7f4f14008040 comp rx=0 tx=0).stop 2024-08-19T10:45:36.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 10 monclient: shutdown 2024-08-19T10:45:36.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:36.665 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 -- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240a3ae0 msgr2=0x7f4f2413a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:36.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240a3ae0 0x7f4f2413a4f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f1801ab10 tx=0x7f4f18019200 comp rx=0 tx=0).stop 2024-08-19T10:45:36.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 -- 172.21.15.190:0/1545717575 shutdown_connections 2024-08-19T10:45:36.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4f1003be00 0x7f4f1003e2c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:36.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4f2413ded0 0x7f4f24123810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:36.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4f240a4400 0x7f4f2413aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:36.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 --2- 172.21.15.190:0/1545717575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240a3ae0 0x7f4f2413a4f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:36.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 -- 172.21.15.190:0/1545717575 >> 172.21.15.190:0/1545717575 conn(0x7f4f2406dde0 msgr2=0x7f4f240a7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:36.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 -- 172.21.15.190:0/1545717575 shutdown_connections 2024-08-19T10:45:36.666 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.665+0000 7f4f29caa640 1 -- 172.21.15.190:0/1545717575 wait complete. 2024-08-19T10:45:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:36.681 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h23 2024-08-19T10:45:36.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient: get_monmap_and_config 2024-08-19T10:45:36.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient: build_initial_monmap 2024-08-19T10:45:36.798 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient: monmap: 2024-08-19T10:45:36.798 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:36.798 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:36.801145+0000 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:36.801145+0000 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:36.799 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:36.800 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:36.800 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:36.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 1 Processor -- start 2024-08-19T10:45:36.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 1 -- start start 2024-08-19T10:45:36.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient: init 2024-08-19T10:45:36.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:36.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient: _add_conns ranks=[10,3,5,6,2,1,9,4,8,7,0] 2024-08-19T10:45:36.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd6d0139a60 0x7fd6d0139e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:36.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient(hunting): picked mon.noname-f con 0x7fd6d0139a60 addr v2:172.21.15.190:3300/0 2024-08-19T10:45:36.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6d00a5730 0x7fd6d00a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:36.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient(hunting): picked mon.noname-g con 0x7fd6d00a5730 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:36.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6d00a6850 0x7fd6d00a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:36.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient(hunting): picked mon.noname-h con 0x7fd6d00a6850 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d009ca60 con 0x7fd6d0139a60 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d00a33c0 con 0x7fd6d00a5730 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd6d00a3640 con 0x7fd6d00a6850 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6d50b6640 10 monclient(hunting): authenticate will time out at 1382.371068s 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6cf59e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6d00a6850 0x7fd6d00a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6ced9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd6d0139a60 0x7fd6d0139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6d00a5730 0x7fd6d00a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6cf59e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6d00a6850 0x7fd6d00a6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:56512/0 (socket says 172.21.15.190:56512) 2024-08-19T10:45:36.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6cf59e640 1 -- 172.21.15.190:0/2946567850 learned_addr learned my addr 172.21.15.190:0/2946567850 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6cf59e640 10 monclient(hunting): get_auth_request con 0x7fd6d00a6850 auth_method 0 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6ced9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd6d0139a60 0x7fd6d0139e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60946/0 (socket says 172.21.15.190:60946) 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6cf59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6cf59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6cf59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 10 monclient(hunting): get_auth_request con 0x7fd6d00a5730 auth_method 0 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6ced9d640 10 monclient(hunting): get_auth_request con 0x7fd6d0139a60 auth_method 0 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6ced9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:36.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6ced9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6cf59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6cf59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6cf59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6ced9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6ced9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6ced9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 10 monclient(hunting): handle_auth_done global_id 34887 payload 274 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 1 -- 172.21.15.190:0/2946567850 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6d00a6850 msgr2=0x7fd6d00a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 1 --2- 172.21.15.190:0/2946567850 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6d00a6850 0x7fd6d00a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 1 -- 172.21.15.190:0/2946567850 >> v2:172.21.15.190:3300/0 conn(0x7fd6d0139a60 msgr2=0x7fd6d0139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:36.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 1 --2- 172.21.15.190:0/2946567850 >> v2:172.21.15.190:3300/0 conn(0x7fd6d0139a60 0x7fd6d0139e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:36.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:36.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 1 monclient: found mon.noname-g 2024-08-19T10:45:36.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:36.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:36.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.797+0000 7fd6c7fff640 1 -- 172.21.15.190:0/2946567850 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d00a46e0 con 0x7fd6d00a5730 2024-08-19T10:45:36.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.801+0000 7fd6c7fff640 1 -- 172.21.15.190:0/2946567850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6d00a5730 msgr2=0x7fd6d00a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:36.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.801+0000 7fd6c7fff640 1 --2- 172.21.15.190:0/2946567850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6d00a5730 0x7fd6d00a5b10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80097c0 tx=0x7fd6b801c360 comp rx=0 tx=0).ready entity=mon.3 client_cookie=6bb6229d565e50d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:36.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 1 -- 172.21.15.190:0/2946567850 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6b80b0070 con 0x7fd6d00a5730 2024-08-19T10:45:36.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:36.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 10 monclient: dump: 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:36.810 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 10 monclient: _finish_auth 0 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 10 monclient: _check_auth_tickets 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 1 -- 172.21.15.190:0/2946567850 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b80aa810 con 0x7fd6d00a5730 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 1 -- 172.21.15.190:0/2946567850 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6b80a9300 con 0x7fd6d00a5730 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 5 monclient: authenticate success, global_id 34887 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:36.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:36.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 4 set_mon_vals no callback set 2024-08-19T10:45:36.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:36.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ce59c640 10 monclient: dump: 2024-08-19T10:45:36.812 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:36.812 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:36.812 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:36.812 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:36.812 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:36.812 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:36.812 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:36.812 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient: shutdown 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 -- 172.21.15.190:0/2946567850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6d00a5730 msgr2=0x7fd6d00a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 --2- 172.21.15.190:0/2946567850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6d00a5730 0x7fd6d00a5b10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80097c0 tx=0x7fd6b801c360 comp rx=0 tx=0).stop 2024-08-19T10:45:36.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 -- 172.21.15.190:0/2946567850 shutdown_connections 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 --2- 172.21.15.190:0/2946567850 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd6d00a6850 0x7fd6d00a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 --2- 172.21.15.190:0/2946567850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6d00a5730 0x7fd6d00a5b10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 --2- 172.21.15.190:0/2946567850 >> v2:172.21.15.190:3300/0 conn(0x7fd6d0139a60 0x7fd6d0139e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 -- 172.21.15.190:0/2946567850 >> 172.21.15.190:0/2946567850 conn(0x7fd6d00abe50 msgr2=0x7fd6d00ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 -- 172.21.15.190:0/2946567850 shutdown_connections 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 -- 172.21.15.190:0/2946567850 wait complete. 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient: build_initial_monmap 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient: monmap: 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:36.814 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 Processor -- start 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 -- start start 2024-08-19T10:45:36.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient: init 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient: _add_conns ranks=[6,1,0,7,3,4,2,5,8] 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6d00a5730 0x7fd6d0131850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient(hunting): picked mon.noname-d con 0x7fd6d00a5730 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d00a6850 0x7fd6d0131d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient(hunting): picked mon.noname-f con 0x7fd6d00a6850 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6d0139a60 0x7fd6d009efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient(hunting): picked mon.noname-a con 0x7fd6d0139a60 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0143270 con 0x7fd6d0139a60 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01430f0 con 0x7fd6d00a6850 2024-08-19T10:45:36.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd6d01433f0 con 0x7fd6d00a5730 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6d50b6640 10 monclient(hunting): authenticate will time out at 1382.378629s 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d00a6850 0x7fd6d0131d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d00a6850 0x7fd6d0131d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60956/0 (socket says 172.21.15.190:60956) 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 1 -- 172.21.15.190:0/3936925296 learned_addr learned my addr 172.21.15.190:0/3936925296 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 10 monclient(hunting): get_auth_request con 0x7fd6d00a6850 auth_method 0 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:36.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6cf59e640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6d0139a60 0x7fd6d009efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6ced9d640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6d00a5730 0x7fd6d0131850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 10 monclient(hunting): handle_auth_done global_id 44290 payload 824 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 1 -- 172.21.15.190:0/3936925296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6d00a5730 msgr2=0x7fd6d0131850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6d00a5730 0x7fd6d0131850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 1 -- 172.21.15.190:0/3936925296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6d0139a60 msgr2=0x7fd6d009efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6d0139a60 0x7fd6d009efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 1 monclient: found mon.noname-f 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6c7fff640 1 -- 172.21.15.190:0/3936925296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d013b0a0 con 0x7fd6d00a6850 2024-08-19T10:45:36.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6cf59e640 10 monclient: get_auth_request con 0x7fd6d0139a60 auth_method 0 2024-08-19T10:45:36.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.805+0000 7fd6cf59e640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6d0139a60 0x7fd6d009efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:36.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c7fff640 1 -- 172.21.15.190:0/3936925296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d00a6850 msgr2=0x7fd6d0131d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:36.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c7fff640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d00a6850 0x7fd6d0131d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6b801cc00 tx=0x7fd6b8019100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:36.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 1 -- 172.21.15.190:0/3936925296 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6b80b0040 con 0x7fd6d00a6850 2024-08-19T10:45:36.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:36.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:45:36.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 10 monclient: dump: 2024-08-19T10:45:36.821 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:36.821 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:36.821 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:36.821 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:36.821 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:36.822 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:36.822 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:36.822 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:36.822 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:36.822 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:36.822 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:36.822 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:36.822 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:36.822 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:36.822 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:36.822 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 1 -- 172.21.15.190:0/3936925296 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b80b9070 con 0x7fd6d00a6850 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6d50b6640 5 monclient: authenticate success, global_id 44290 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6d50b6640 10 monclient: _renew_subs 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6d50b6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6d50b6640 1 -- 172.21.15.190:0/3936925296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d01358f0 con 0x7fd6d00a6850 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 1 -- 172.21.15.190:0/3936925296 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd6b80044b0 con 0x7fd6d00a6850 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:36.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 10 monclient: dump: 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:36.824 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6d50b6640 10 monclient: _renew_subs 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6cd59a640 4 set_mon_vals no callback set 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6d50b6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6d50b6640 1 -- 172.21.15.190:0/3936925296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d00a0960 con 0x7fd6d00a6850 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 1 -- 172.21.15.190:0/3936925296 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd6b8004040 con 0x7fd6d00a6850 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6d50b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6d50b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6d50b6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:36.825 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6d50b6640 1 -- 172.21.15.190:0/3936925296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d00a6cb0 con 0x7fd6d00a6850 2024-08-19T10:45:36.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6b003be00 0x7fd6b003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:36.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6c77fe640 1 -- 172.21.15.190:0/3936925296 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(251..251 src has 150..251) ==== 10325+0+0 (secure 0 0 0) 0x7fd6b80e94b0 con 0x7fd6d00a6850 2024-08-19T10:45:36.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6ced9d640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6b003be00 0x7fd6b003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:36.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.809+0000 7fd6ced9d640 10 monclient: get_auth_request con 0x7fd6b003be00 auth_method 0 2024-08-19T10:45:36.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.813+0000 7fd6ced9d640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6b003be00 0x7fd6b003e2c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009910 tx=0x7fd6c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:36.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.813+0000 7fd6c77fe640 1 -- 172.21.15.190:0/3936925296 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd6b80c2060 con 0x7fd6d00a6850 2024-08-19T10:45:36.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.813+0000 7fd6c77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:36.826 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.813+0000 7fd6c77fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:36.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.989+0000 7fd6d50b6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h23"}] 2024-08-19T10:45:36.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.989+0000 7fd6d50b6640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h23"}] 2024-08-19T10:45:36.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.989+0000 7fd6d50b6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:36.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:36.989+0000 7fd6d50b6640 1 -- 172.21.15.190:0/3936925296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h23"} v 0) -- 0x7fd6d00a14c0 con 0x7fd6d00a6850 2024-08-19T10:45:37.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.573+0000 7fd6c77fe640 1 -- 172.21.15.190:0/3936925296 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h23"}]=0 removed item id -85 name 'h23' from crush map v252) ==== 120+0+0 (secure 0 0 0) 0x7fd6b8016070 con 0x7fd6d00a6850 2024-08-19T10:45:37.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.573+0000 7fd6c77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h23"}] 2024-08-19T10:45:37.575 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.573+0000 7fd6c77fe640 10 monclient: _finish_command 2 = system:0 removed item id -85 name 'h23' from crush map 2024-08-19T10:45:37.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.677+0000 7fd6d50b6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h23"}] 2024-08-19T10:45:37.679 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.677+0000 7fd6d50b6640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h23"}] 2024-08-19T10:45:37.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.677+0000 7fd6d50b6640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:37.680 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.677+0000 7fd6d50b6640 1 -- 172.21.15.190:0/3936925296 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h23"} v 0) -- 0x7fd6d00008e0 con 0x7fd6d00a6850 2024-08-19T10:45:37.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6c77fe640 1 -- 172.21.15.190:0/3936925296 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h23"}]=0 device 'h23' does not appear in the crush map v252) ==== 120+0+0 (secure 0 0 0) 0x7fd6b80aa7e0 con 0x7fd6d00a6850 2024-08-19T10:45:37.681 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6c77fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h23"}] 2024-08-19T10:45:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6c77fe640 10 monclient: _finish_command 3 = system:0 device 'h23' does not appear in the crush map 2024-08-19T10:45:37.682 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h23' does not appear in the crush map 2024-08-19T10:45:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 -- 172.21.15.190:0/3936925296 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6b003be00 msgr2=0x7fd6b003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6b003be00 0x7fd6b003e2c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009910 tx=0x7fd6c0008040 comp rx=0 tx=0).stop 2024-08-19T10:45:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 10 monclient: shutdown 2024-08-19T10:45:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 -- 172.21.15.190:0/3936925296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d00a6850 msgr2=0x7fd6d0131d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:37.684 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d00a6850 0x7fd6d0131d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6b801cc00 tx=0x7fd6b8019100 comp rx=0 tx=0).stop 2024-08-19T10:45:37.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 -- 172.21.15.190:0/3936925296 shutdown_connections 2024-08-19T10:45:37.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6b003be00 0x7fd6b003e2c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:37.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6d0139a60 0x7fd6d009efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:37.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d00a6850 0x7fd6d0131d90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:37.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 --2- 172.21.15.190:0/3936925296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd6d00a5730 0x7fd6d0131850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:37.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 -- 172.21.15.190:0/3936925296 >> 172.21.15.190:0/3936925296 conn(0x7fd6d00abe50 msgr2=0x7fd6d00ab300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:37.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 -- 172.21.15.190:0/3936925296 shutdown_connections 2024-08-19T10:45:37.685 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.681+0000 7fd6d50b6640 1 -- 172.21.15.190:0/3936925296 wait complete. 2024-08-19T10:45:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:37.699 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h24 2024-08-19T10:45:37.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient: get_monmap_and_config 2024-08-19T10:45:37.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient: build_initial_monmap 2024-08-19T10:45:37.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient: monmap: 2024-08-19T10:45:37.815 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:37.815 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:37.815 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:37.817785+0000 2024-08-19T10:45:37.815 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:37.817785+0000 2024-08-19T10:45:37.815 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:37.815 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:37.816 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:37.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 1 Processor -- start 2024-08-19T10:45:37.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 1 -- start start 2024-08-19T10:45:37.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient: init 2024-08-19T10:45:37.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:37.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient: _add_conns ranks=[7,10,1,2,6,8,0,5,9,3,4] 2024-08-19T10:45:37.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7bf41377c0 0x7f7bf4137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:37.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient(hunting): picked mon.noname-i con 0x7f7bf41377c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:37.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7bf40703c0 0x7f7bf40a3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:37.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient(hunting): picked mon.noname-f con 0x7f7bf40703c0 addr v2:172.21.15.190:3300/0 2024-08-19T10:45:37.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7bf409ca60 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:37.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bf40a14d0 con 0x7f7bf409ca60 2024-08-19T10:45:37.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:37.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf40743f0 con 0x7f7bf40703c0 2024-08-19T10:45:37.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:37.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7bf4070840 con 0x7f7bf41377c0 2024-08-19T10:45:37.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf9f73640 10 monclient(hunting): authenticate will time out at 1383.387665s 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf37fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7bf41377c0 0x7f7bf4137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf3fff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f7bf409ca60 0x7f7bf40a3b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:48800/0 (socket says 172.21.15.190:48800) 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf3fff640 1 -- 172.21.15.190:0/1016151177 learned_addr learned my addr 172.21.15.190:0/1016151177 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf37fe640 10 monclient(hunting): get_auth_request con 0x7f7bf41377c0 auth_method 0 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf2ffd640 1 --2- 172.21.15.190:0/1016151177 >> v2:172.21.15.190:3300/0 conn(0x7f7bf40703c0 0x7f7bf40a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.813+0000 7f7bf37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:37.821 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:37.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf2ffd640 10 monclient(hunting): get_auth_request con 0x7f7bf40703c0 auth_method 0 2024-08-19T10:45:37.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:37.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:37.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:37.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:37.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:37.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:37.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf37fe640 10 monclient(hunting): handle_auth_done global_id 15784 payload 274 2024-08-19T10:45:37.822 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf37fe640 1 -- 172.21.15.190:0/1016151177 >> v2:172.21.15.190:3300/0 conn(0x7f7bf40703c0 msgr2=0x7f7bf40a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:37.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf37fe640 1 --2- 172.21.15.190:0/1016151177 >> v2:172.21.15.190:3300/0 conn(0x7f7bf40703c0 0x7f7bf40a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:37.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf37fe640 1 -- 172.21.15.190:0/1016151177 >> v1:172.21.15.190:6789/0 conn(0x7f7bf409ca60 legacy=0x7f7bf40a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:37.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:37.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf37fe640 1 monclient: found mon.noname-i 2024-08-19T10:45:37.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:37.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf37fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:37.823 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.817+0000 7f7bf37fe640 1 -- 172.21.15.190:0/1016151177 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf4074110 con 0x7f7bf41377c0 2024-08-19T10:45:37.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf37fe640 1 -- 172.21.15.190:0/1016151177 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7bf41377c0 msgr2=0x7f7bf4137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:37.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf37fe640 1 --2- 172.21.15.190:0/1016151177 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7bf41377c0 0x7f7bf4137ba0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7be80097c0 tx=0x7f7be801c180 comp rx=0 tx=0).ready entity=mon.7 client_cookie=b041a30489672ec9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:37.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 1 -- 172.21.15.190:0/1016151177 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7be80b1070 con 0x7f7bf41377c0 2024-08-19T10:45:37.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:37.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:37.829 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 10 monclient: dump: 2024-08-19T10:45:37.829 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:37.830 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 5 monclient: authenticate success, global_id 15784 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 1 -- 172.21.15.190:0/1016151177 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be80a4cc0 con 0x7f7bf41377c0 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 4 set_mon_vals no callback set 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 1 -- 172.21.15.190:0/1016151177 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7be80aac20 con 0x7f7bf41377c0 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:37.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf27fc640 10 monclient: dump: 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:37.832 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient: shutdown 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 -- 172.21.15.190:0/1016151177 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7bf41377c0 msgr2=0x7f7bf4137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 --2- 172.21.15.190:0/1016151177 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7bf41377c0 0x7f7bf4137ba0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7be80097c0 tx=0x7f7be801c180 comp rx=0 tx=0).stop 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 -- 172.21.15.190:0/1016151177 shutdown_connections 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 --2- 172.21.15.190:0/1016151177 >> v2:172.21.15.190:3300/0 conn(0x7f7bf40703c0 0x7f7bf40a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 --2- 172.21.15.190:0/1016151177 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7bf41377c0 0x7f7bf4137ba0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 -- 172.21.15.190:0/1016151177 >> 172.21.15.190:0/1016151177 conn(0x7f7bf40a4700 msgr2=0x7f7bf40a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 -- 172.21.15.190:0/1016151177 shutdown_connections 2024-08-19T10:45:37.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 -- 172.21.15.190:0/1016151177 wait complete. 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient: build_initial_monmap 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient: monmap: 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:37.834 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 Processor -- start 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 -- start start 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient: init 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient: _add_conns ranks=[6,1,3,7,2,5,0,8,4] 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7bf40703c0 0x7f7bf41cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient(hunting): picked mon.noname-d con 0x7f7bf40703c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:37.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf409ca60 0x7f7bf413e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient(hunting): picked mon.noname-f con 0x7f7bf409ca60 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bf41377c0 0x7f7bf413ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient(hunting): picked mon.noname-g con 0x7f7bf41377c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf40a43e0 con 0x7f7bf409ca60 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf40a4260 con 0x7f7bf41377c0 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7bf4141e60 con 0x7f7bf40703c0 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf9f73640 10 monclient(hunting): authenticate will time out at 1383.394973s 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf409ca60 0x7f7bf413e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:37.836 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf409ca60 0x7f7bf413e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60978/0 (socket says 172.21.15.190:60978) 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bf41377c0 0x7f7bf413ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 1 -- 172.21.15.190:0/2639539316 learned_addr learned my addr 172.21.15.190:0/2639539316 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 10 monclient(hunting): get_auth_request con 0x7f7bf409ca60 auth_method 0 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf3fff640 10 monclient(hunting): get_auth_request con 0x7f7bf41377c0 auth_method 0 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:37.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 10 monclient(hunting): handle_auth_done global_id 44308 payload 824 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 1 -- 172.21.15.190:0/2639539316 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7bf40703c0 msgr2=0x7f7bf41cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf37fe640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7bf40703c0 0x7f7bf41cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7bf40703c0 0x7f7bf41cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 1 -- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bf41377c0 msgr2=0x7f7bf413ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bf41377c0 0x7f7bf413ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 1 monclient: found mon.noname-f 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:37.838 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.821+0000 7f7bf2ffd640 1 -- 172.21.15.190:0/2639539316 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf413aee0 con 0x7f7bf409ca60 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf2ffd640 1 -- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf409ca60 msgr2=0x7f7bf413e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf2ffd640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf409ca60 0x7f7bf413e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7be400e8d0 tx=0x7f7be400ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 1 -- 172.21.15.190:0/2639539316 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7be4017070 con 0x7f7bf409ca60 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 10 monclient: dump: 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:37.839 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 10 monclient: _finish_auth 0 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:37.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 1 -- 172.21.15.190:0/2639539316 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be4013470 con 0x7f7bf409ca60 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf9f73640 5 monclient: authenticate success, global_id 44308 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf9f73640 10 monclient: _renew_subs 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf9f73640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf9f73640 1 -- 172.21.15.190:0/2639539316 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf40a6ae0 con 0x7f7bf409ca60 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 1 -- 172.21.15.190:0/2639539316 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7be4013a90 con 0x7f7bf409ca60 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 10 monclient: dump: 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:37.841 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:37.842 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf1ffb640 4 set_mon_vals no callback set 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf9f73640 10 monclient: _renew_subs 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf9f73640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf9f73640 1 -- 172.21.15.190:0/2639539316 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf413c9a0 con 0x7f7bf409ca60 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf9f73640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf9f73640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf9f73640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf9f73640 1 -- 172.21.15.190:0/2639539316 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf4137ba0 con 0x7f7bf409ca60 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 1 -- 172.21.15.190:0/2639539316 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7be401a070 con 0x7f7bf409ca60 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7be003be00 0x7f7be003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bdbfff640 1 -- 172.21.15.190:0/2639539316 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(252..252 src has 150..252) ==== 10112+0+0 (secure 0 0 0) 0x7f7be400f040 con 0x7f7bf409ca60 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf37fe640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7be003be00 0x7f7be003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:37.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.825+0000 7f7bf37fe640 10 monclient: get_auth_request con 0x7f7be003be00 auth_method 0 2024-08-19T10:45:37.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.829+0000 7f7bf37fe640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7be003be00 0x7f7be003e2c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4070260 tx=0x7f7be80a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:37.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.829+0000 7f7bdbfff640 1 -- 172.21.15.190:0/2639539316 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7be4013c70 con 0x7f7bf409ca60 2024-08-19T10:45:37.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.829+0000 7f7bdbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:37.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.829+0000 7f7bdbfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:37.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.997+0000 7f7bf9f73640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h24"}] 2024-08-19T10:45:37.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.997+0000 7f7bf9f73640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h24"}] 2024-08-19T10:45:37.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.997+0000 7f7bf9f73640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:37.998 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:37.997+0000 7f7bf9f73640 1 -- 172.21.15.190:0/2639539316 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h24"} v 0) -- 0x7f7bf4070af0 con 0x7f7bf409ca60 2024-08-19T10:45:38.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.577+0000 7f7bdbfff640 1 -- 172.21.15.190:0/2639539316 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h24"}]=0 removed item id -88 name 'h24' from crush map v253) ==== 120+0+0 (secure 0 0 0) 0x7f7be404d6f0 con 0x7f7bf409ca60 2024-08-19T10:45:38.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.577+0000 7f7bdbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h24"}] 2024-08-19T10:45:38.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.577+0000 7f7bdbfff640 10 monclient: _finish_command 2 = system:0 removed item id -88 name 'h24' from crush map 2024-08-19T10:45:38.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.689+0000 7f7bf9f73640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h24"}] 2024-08-19T10:45:38.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.689+0000 7f7bf9f73640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h24"}] 2024-08-19T10:45:38.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.689+0000 7f7bf9f73640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:38.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.689+0000 7f7bf9f73640 1 -- 172.21.15.190:0/2639539316 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h24"} v 0) -- 0x7f7bf413cd80 con 0x7f7bf409ca60 2024-08-19T10:45:38.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.689+0000 7f7bdbfff640 1 -- 172.21.15.190:0/2639539316 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h24"}]=0 device 'h24' does not appear in the crush map v253) ==== 120+0+0 (secure 0 0 0) 0x7f7be4013620 con 0x7f7bf409ca60 2024-08-19T10:45:38.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.689+0000 7f7bdbfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h24"}] 2024-08-19T10:45:38.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.689+0000 7f7bdbfff640 10 monclient: _finish_command 3 = system:0 device 'h24' does not appear in the crush map 2024-08-19T10:45:38.692 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h24' does not appear in the crush map 2024-08-19T10:45:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 -- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7be003be00 msgr2=0x7f7be003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7be003be00 0x7f7be003e2c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4070260 tx=0x7f7be80a6070 comp rx=0 tx=0).stop 2024-08-19T10:45:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 10 monclient: shutdown 2024-08-19T10:45:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 -- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf409ca60 msgr2=0x7f7bf413e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf409ca60 0x7f7bf413e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7be400e8d0 tx=0x7f7be400ed90 comp rx=0 tx=0).stop 2024-08-19T10:45:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 -- 172.21.15.190:0/2639539316 shutdown_connections 2024-08-19T10:45:38.695 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7be003be00 0x7f7be003e2c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7bf41377c0 0x7f7bf413ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf409ca60 0x7f7bf413e770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 --2- 172.21.15.190:0/2639539316 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7bf40703c0 0x7f7bf41cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 -- 172.21.15.190:0/2639539316 >> 172.21.15.190:0/2639539316 conn(0x7f7bf40a4700 msgr2=0x7f7bf40a62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 -- 172.21.15.190:0/2639539316 shutdown_connections 2024-08-19T10:45:38.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.693+0000 7f7bf9f73640 1 -- 172.21.15.190:0/2639539316 wait complete. 2024-08-19T10:45:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:38.710 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h25 2024-08-19T10:45:38.827 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.825+0000 7f87a2d1f640 10 monclient: get_monmap_and_config 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.825+0000 7f87a2d1f640 10 monclient: build_initial_monmap 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.825+0000 7f87a2d1f640 10 monclient: monmap: 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:38.830780+0000 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:38.830780+0000 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:38.828 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:38.829 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:38.829 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:38.829 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:38.829 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:38.829 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:38.829 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:38.829 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:38.829 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:38.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 1 Processor -- start 2024-08-19T10:45:38.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 1 -- start start 2024-08-19T10:45:38.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 10 monclient: init 2024-08-19T10:45:38.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:38.831 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 10 monclient: _add_conns ranks=[2,6,9,3,7,1,10,8,0,5,4] 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f879c0a5eb0 0x7f879c0a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 10 monclient(hunting): picked mon.noname-b con 0x7f879c0a5eb0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a4bf0 0x7f879c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 10 monclient(hunting): picked mon.noname-d con 0x7f879c0a4bf0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f879c139860 0x7f879c139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 10 monclient(hunting): picked mon.noname-a con 0x7f879c139860 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f879c06d120 con 0x7f879c139860 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f879c0a5510 con 0x7f879c0a5eb0 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f879c0a6290 con 0x7f879c0a4bf0 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a2d1f640 10 monclient(hunting): authenticate will time out at 1384.401115s 2024-08-19T10:45:38.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a0abb640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f879c0a5eb0 msgr2=0x7f879c0a46b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a0abb640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f879c0a5eb0 0x7f879c0a46b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f879bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a4bf0 0x7f879c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f879c139860 0x7f879c139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f879c139860 0x7f879c139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.190:33534/0 (socket says 172.21.15.190:33534) 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 1 -- 172.21.15.190:0/4253525542 learned_addr learned my addr 172.21.15.190:0/4253525542 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 10 monclient(hunting): get_auth_request con 0x7f879c139860 auth_method 0 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f879bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a4bf0 0x7f879c0a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:56774/0 (socket says 172.21.15.190:56774) 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f879bfff640 10 monclient(hunting): get_auth_request con 0x7f879c0a4bf0 auth_method 0 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f879bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f879bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:38.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f879bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f879bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f879bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f879bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 10 monclient(hunting): handle_auth_done global_id 6498 payload 274 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 1 -- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a4bf0 msgr2=0x7f879c0a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 1 --2- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a4bf0 0x7f879c0a4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 1 -- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f879c0a5eb0 msgr2=0x7f879c0a46b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 1 --2- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f879c0a5eb0 0x7f879c0a46b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 10 monclient: _finish_hunting 0 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 1 monclient: found mon.noname-a 2024-08-19T10:45:38.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:38.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:45:38.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f87a12bc640 1 -- 172.21.15.190:0/4253525542 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c06ca60 con 0x7f879c139860 2024-08-19T10:45:38.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.829+0000 7f879bfff640 1 --2- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a4bf0 0x7f879c0a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:38.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f87a12bc640 1 -- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f879c139860 msgr2=0x7f879c139cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:38.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f87a12bc640 1 --2- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f879c139860 0x7f879c139cf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8794009e80 tx=0x7f879401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec37c476d1ae460 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:38.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 1 -- 172.21.15.190:0/4253525542 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87940090d0 con 0x7f879c139860 2024-08-19T10:45:38.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:38.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:45:38.837 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 10 monclient: dump: 2024-08-19T10:45:38.837 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:38.838 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f87a2d1f640 5 monclient: authenticate success, global_id 6498 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f87a2d1f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 1 -- 172.21.15.190:0/4253525542 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87940a59a0 con 0x7f879c139860 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 1 -- 172.21.15.190:0/4253525542 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87940a5b50 con 0x7f879c139860 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:45:38.839 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f879b7fe640 10 monclient: dump: 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:38.840 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f87a2d1f640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.833+0000 7f87a2d1f640 4 set_mon_vals no callback set 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient: shutdown 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 -- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f879c139860 msgr2=0x7f879c139cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 --2- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f879c139860 0x7f879c139cf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8794009e80 tx=0x7f879401c8f0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 -- 172.21.15.190:0/4253525542 shutdown_connections 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 --2- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f879c139860 0x7f879c139cf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 --2- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a4bf0 0x7f879c0a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 --2- 172.21.15.190:0/4253525542 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f879c0a5eb0 0x7f879c0a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 -- 172.21.15.190:0/4253525542 >> 172.21.15.190:0/4253525542 conn(0x7f879c0a92a0 msgr2=0x7f879c0a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 -- 172.21.15.190:0/4253525542 shutdown_connections 2024-08-19T10:45:38.841 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 -- 172.21.15.190:0/4253525542 wait complete. 2024-08-19T10:45:38.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient: build_initial_monmap 2024-08-19T10:45:38.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient: monmap: 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:38.847 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 Processor -- start 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 -- start start 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient: init 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient: _add_conns ranks=[3,6,5,2,1,4,7,0,8] 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879c0a4bf0 0x7f879c138470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient(hunting): picked mon.noname-g con 0x7f879c0a4bf0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a5eb0 0x7f879c1389b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient(hunting): picked mon.noname-d con 0x7f879c0a5eb0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f879c139860 0x7f879c13c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:38.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient(hunting): picked mon.noname-h con 0x7f879c139860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f879c06cd80 con 0x7f879c0a4bf0 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f879c0a0a40 con 0x7f879c139860 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f879c06cf00 con 0x7f879c0a5eb0 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a2d1f640 10 monclient(hunting): authenticate will time out at 1384.410342s 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a0abb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879c0a4bf0 0x7f879c138470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f879c139860 0x7f879c13c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a0abb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879c0a4bf0 0x7f879c138470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46584/0 (socket says 172.21.15.190:46584) 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a0abb640 1 -- 172.21.15.190:0/24270578 learned_addr learned my addr 172.21.15.190:0/24270578 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:38.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 10 monclient(hunting): get_auth_request con 0x7f879c139860 auth_method 0 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a0abb640 10 monclient(hunting): get_auth_request con 0x7f879c0a4bf0 auth_method 0 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a0abb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a0abb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a0abb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a0abb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a0abb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:38.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a0abb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f879bfff640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a5eb0 0x7f879c1389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f879bfff640 10 monclient(hunting): get_auth_request con 0x7f879c0a5eb0 auth_method 0 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f879bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f879bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f879bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 10 monclient(hunting): handle_auth_done global_id 6440 payload 824 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 1 -- 172.21.15.190:0/24270578 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a5eb0 msgr2=0x7f879c1389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a5eb0 0x7f879c1389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 1 -- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879c0a4bf0 msgr2=0x7f879c138470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879c0a4bf0 0x7f879c138470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 10 monclient: _finish_hunting 0 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 1 monclient: found mon.noname-h 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:38.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a12bc640 1 -- 172.21.15.190:0/24270578 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c123ca0 con 0x7f879c139860 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f879bfff640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a5eb0 0x7f879c1389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.837+0000 7f87a0abb640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879c0a4bf0 0x7f879c138470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a12bc640 1 -- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f879c139860 msgr2=0x7f879c13c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a12bc640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f879c139860 0x7f879c13c9d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f87940ab000 tx=0x7f8794019690 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 1 -- 172.21.15.190:0/24270578 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87940bb020 con 0x7f879c139860 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 10 monclient: dump: 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:38.852 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:38.853 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 10 monclient: _finish_auth 0 2024-08-19T10:45:38.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 10 monclient: _check_auth_tickets 2024-08-19T10:45:38.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:38.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a2d1f640 5 monclient: authenticate success, global_id 6440 2024-08-19T10:45:38.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 1 -- 172.21.15.190:0/24270578 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87940a5e20 con 0x7f879c139860 2024-08-19T10:45:38.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a2d1f640 10 monclient: _renew_subs 2024-08-19T10:45:38.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a2d1f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:38.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a2d1f640 1 -- 172.21.15.190:0/24270578 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879c13fbd0 con 0x7f879c139860 2024-08-19T10:45:38.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:38.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 1 -- 172.21.15.190:0/24270578 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8794003bb0 con 0x7f879c139860 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f879a7fc640 4 set_mon_vals no callback set 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 10 monclient: dump: 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:38.855 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a2d1f640 10 monclient: _renew_subs 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a2d1f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a2d1f640 1 -- 172.21.15.190:0/24270578 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f879c0a5210 con 0x7f879c139860 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a2d1f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a2d1f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a2d1f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:38.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a2d1f640 1 -- 172.21.15.190:0/24270578 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879c0a46b0 con 0x7f879c139860 2024-08-19T10:45:38.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 1 -- 172.21.15.190:0/24270578 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8794004040 con 0x7f879c139860 2024-08-19T10:45:38.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878003be00 0x7f878003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:38.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87997fa640 1 -- 172.21.15.190:0/24270578 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(253..253 src has 150..253) ==== 9899+0+0 (secure 0 0 0) 0x7f87940e9ea0 con 0x7f879c139860 2024-08-19T10:45:38.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a0abb640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878003be00 0x7f878003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:38.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.841+0000 7f87a0abb640 10 monclient: get_auth_request con 0x7f878003be00 auth_method 0 2024-08-19T10:45:38.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.845+0000 7f87a0abb640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878003be00 0x7f878003e2c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8788003fd0 tx=0x7f8788003ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:38.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.845+0000 7f87997fa640 1 -- 172.21.15.190:0/24270578 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f8794016600 con 0x7f879c139860 2024-08-19T10:45:38.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.845+0000 7f87997fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:38.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:38.845+0000 7f87997fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:39.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.009+0000 7f87a2d1f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h25"}] 2024-08-19T10:45:39.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.009+0000 7f87a2d1f640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h25"}] 2024-08-19T10:45:39.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.009+0000 7f87a2d1f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:39.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.009+0000 7f87a2d1f640 1 -- 172.21.15.190:0/24270578 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h25"} v 0) -- 0x7f879c06ec50 con 0x7f879c139860 2024-08-19T10:45:39.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.593+0000 7f87997fa640 1 -- 172.21.15.190:0/24270578 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h25"}]=0 removed item id -91 name 'h25' from crush map v254) ==== 120+0+0 (secure 0 0 0) 0x7f87940a5970 con 0x7f879c139860 2024-08-19T10:45:39.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.597+0000 7f87997fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h25"}] 2024-08-19T10:45:39.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.597+0000 7f87997fa640 10 monclient: _finish_command 2 = system:0 removed item id -91 name 'h25' from crush map 2024-08-19T10:45:39.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.705+0000 7f87a2d1f640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h25"}] 2024-08-19T10:45:39.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.709+0000 7f87a2d1f640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h25"}] 2024-08-19T10:45:39.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.709+0000 7f87a2d1f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:39.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.709+0000 7f87a2d1f640 1 -- 172.21.15.190:0/24270578 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush rm", "name": "h25"} v 0) -- 0x7f879c1239c0 con 0x7f879c139860 2024-08-19T10:45:39.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.985+0000 7f87997fa640 1 -- 172.21.15.190:0/24270578 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h25"}]=0 device 'h25' does not appear in the crush map v254) ==== 120+0+0 (secure 0 0 0) 0x7f8794003df0 con 0x7f879c139860 2024-08-19T10:45:39.985 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.985+0000 7f87997fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h25"}] 2024-08-19T10:45:39.986 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.985+0000 7f87997fa640 10 monclient: _finish_command 3 = system:0 device 'h25' does not appear in the crush map 2024-08-19T10:45:39.986 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h25' does not appear in the crush map 2024-08-19T10:45:39.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 -- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878003be00 msgr2=0x7f878003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:39.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878003be00 0x7f878003e2c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8788003fd0 tx=0x7f8788003ba0 comp rx=0 tx=0).stop 2024-08-19T10:45:39.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 10 monclient: shutdown 2024-08-19T10:45:39.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:39.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 -- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f879c139860 msgr2=0x7f879c13c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:39.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f879c139860 0x7f879c13c9d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f87940ab000 tx=0x7f8794019690 comp rx=0 tx=0).stop 2024-08-19T10:45:39.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 -- 172.21.15.190:0/24270578 shutdown_connections 2024-08-19T10:45:39.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f878003be00 0x7f878003e2c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:39.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f879c139860 0x7f879c13c9d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:39.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f879c0a5eb0 0x7f879c1389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:39.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 --2- 172.21.15.190:0/24270578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879c0a4bf0 0x7f879c138470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:39.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 -- 172.21.15.190:0/24270578 >> 172.21.15.190:0/24270578 conn(0x7f879c0a92a0 msgr2=0x7f879c13fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:39.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 -- 172.21.15.190:0/24270578 shutdown_connections 2024-08-19T10:45:39.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:39.989+0000 7f87a2d1f640 1 -- 172.21.15.190:0/24270578 wait complete. 2024-08-19T10:45:40.012 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:40.012 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h26 2024-08-19T10:45:40.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient: get_monmap_and_config 2024-08-19T10:45:40.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient: build_initial_monmap 2024-08-19T10:45:40.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient: monmap: 2024-08-19T10:45:40.134 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:40.134 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:40.134 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:40.136946+0000 2024-08-19T10:45:40.134 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:40.136946+0000 2024-08-19T10:45:40.134 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:40.135 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:40.135 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:40.135 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:40.135 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:40.135 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:40.135 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:40.135 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:40.135 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:40.135 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:40.135 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:40.136 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:40.136 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:40.136 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:40.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 1 Processor -- start 2024-08-19T10:45:40.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 1 -- start start 2024-08-19T10:45:40.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient: init 2024-08-19T10:45:40.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:40.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient: _add_conns ranks=[9,1,6,8,7,5,0,2,3,10,4] 2024-08-19T10:45:40.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff624139980 0x7ff624139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:40.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient(hunting): picked mon.noname-a con 0x7ff624139980 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:40.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff62413a2a0 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:40.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff62413e8c0 0x7ff624140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:40.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient(hunting): picked mon.noname-d con 0x7ff62413e8c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:40.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6240a8470 con 0x7ff62413a2a0 2024-08-19T10:45:40.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:40.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff62409ca60 con 0x7ff624139980 2024-08-19T10:45:40.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff6240a3640 con 0x7ff62413e8c0 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff629d72640 10 monclient(hunting): authenticate will time out at 1385.706873s 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff622ffd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff62413a2a0 0x7ff6240a5730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:48816/0 (socket says 172.21.15.190:48816) 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff622ffd640 1 -- 172.21.15.190:0/4139555309 learned_addr learned my addr 172.21.15.190:0/4139555309 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 1 --2- 172.21.15.190:0/4139555309 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff62413e8c0 0x7ff624140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff6237fe640 1 --2- 172.21.15.190:0/4139555309 >> v2:172.21.15.89:3300/0 conn(0x7ff624139980 0x7ff624139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 10 monclient(hunting): get_auth_request con 0x7ff62413e8c0 auth_method 0 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:40.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff6237fe640 10 monclient(hunting): get_auth_request con 0x7ff624139980 auth_method 0 2024-08-19T10:45:40.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff6237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:40.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff6237fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:40.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff6237fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff6237fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff6237fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff6237fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 10 monclient(hunting): handle_auth_done global_id 44412 payload 274 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 1 -- 172.21.15.190:0/4139555309 >> v2:172.21.15.89:3300/0 conn(0x7ff624139980 msgr2=0x7ff624139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 1 --2- 172.21.15.190:0/4139555309 >> v2:172.21.15.89:3300/0 conn(0x7ff624139980 0x7ff624139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 1 -- 172.21.15.190:0/4139555309 >> v1:172.21.15.190:6789/0 conn(0x7ff62413a2a0 legacy=0x7ff6240a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:40.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 1 monclient: found mon.noname-d 2024-08-19T10:45:40.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:40.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:45:40.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.133+0000 7ff623fff640 1 -- 172.21.15.190:0/4139555309 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6240a3850 con 0x7ff62413e8c0 2024-08-19T10:45:40.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 1 -- 172.21.15.190:0/4139555309 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff62413e8c0 msgr2=0x7ff624140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 1 --2- 172.21.15.190:0/4139555309 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff62413e8c0 0x7ff624140cb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009d90 tx=0x7ff61c01c830 comp rx=0 tx=0).ready entity=mon.6 client_cookie=8553f9f68331b90f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 1 -- 172.21.15.190:0/4139555309 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff61c0b2070 con 0x7ff62413e8c0 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 10 monclient: dump: 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:40.148 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 5 monclient: authenticate success, global_id 44412 2024-08-19T10:45:40.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 1 -- 172.21.15.190:0/4139555309 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61c0a59d0 con 0x7ff62413e8c0 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 4 set_mon_vals no callback set 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 1 -- 172.21.15.190:0/4139555309 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff61c0ab340 con 0x7ff62413e8c0 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6227fc640 10 monclient: dump: 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:40.150 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient: shutdown 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:40.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 -- 172.21.15.190:0/4139555309 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff62413e8c0 msgr2=0x7ff624140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 --2- 172.21.15.190:0/4139555309 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff62413e8c0 0x7ff624140cb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009d90 tx=0x7ff61c01c830 comp rx=0 tx=0).stop 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 -- 172.21.15.190:0/4139555309 shutdown_connections 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 --2- 172.21.15.190:0/4139555309 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff62413e8c0 0x7ff624140cb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 --2- 172.21.15.190:0/4139555309 >> v2:172.21.15.89:3300/0 conn(0x7ff624139980 0x7ff624139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 -- 172.21.15.190:0/4139555309 >> 172.21.15.190:0/4139555309 conn(0x7ff6240abde0 msgr2=0x7ff6240ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 -- 172.21.15.190:0/4139555309 shutdown_connections 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 -- 172.21.15.190:0/4139555309 wait complete. 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient: build_initial_monmap 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient: monmap: 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:40.152 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 Processor -- start 2024-08-19T10:45:40.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 -- start start 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient: init 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient: _add_conns ranks=[4,2,1,6,7,5,8,3,0] 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff624139980 0x7ff6241358e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient(hunting): picked mon.noname-c con 0x7ff624139980 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff62413e8c0 0x7ff624135e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient(hunting): picked mon.noname-b con 0x7ff62413e8c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff624136360 0x7ff6241d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient(hunting): picked mon.noname-f con 0x7ff624136360 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff624143b90 con 0x7ff624136360 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:40.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff624143a10 con 0x7ff62413e8c0 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff624143d10 con 0x7ff624139980 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff629d72640 10 monclient(hunting): authenticate will time out at 1385.715006s 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff624136360 0x7ff6241d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff622ffd640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff62413e8c0 msgr2=0x7ff624135e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff622ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff62413e8c0 0x7ff624135e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff624136360 0x7ff6241d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60982/0 (socket says 172.21.15.190:60982) 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 1 -- 172.21.15.190:0/4152892924 learned_addr learned my addr 172.21.15.190:0/4152892924 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 10 monclient(hunting): get_auth_request con 0x7ff624136360 auth_method 0 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:40.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6237fe640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff624139980 0x7ff6241358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6237fe640 10 monclient(hunting): get_auth_request con 0x7ff624139980 auth_method 0 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6237fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff6237fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 10 monclient(hunting): handle_auth_done global_id 44317 payload 824 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 1 -- 172.21.15.190:0/4152892924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff624139980 msgr2=0x7ff6241358e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff624139980 0x7ff6241358e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 1 -- 172.21.15.190:0/4152892924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff62413e8c0 msgr2=0x7ff624135e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff62413e8c0 0x7ff624135e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:40.156 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 1 monclient: found mon.noname-f 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.141+0000 7ff623fff640 1 -- 172.21.15.190:0/4152892924 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff624134700 con 0x7ff624136360 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff623fff640 1 -- 172.21.15.190:0/4152892924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff624136360 msgr2=0x7ff6241d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff623fff640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff624136360 0x7ff6241d0f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff61c01d0d0 tx=0x7ff61c0195d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 1 -- 172.21.15.190:0/4152892924 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff61c0b2050 con 0x7ff624136360 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 10 monclient: dump: 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:40.157 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:40.158 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 10 monclient: _finish_auth 0 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 1 -- 172.21.15.190:0/4152892924 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61c0be070 con 0x7ff624136360 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff629d72640 5 monclient: authenticate success, global_id 44317 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff629d72640 10 monclient: _renew_subs 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff629d72640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff629d72640 1 -- 172.21.15.190:0/4152892924 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62409d6c0 con 0x7ff624136360 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 1 -- 172.21.15.190:0/4152892924 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff61c003bb0 con 0x7ff624136360 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 10 monclient: dump: 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:40.159 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:40.160 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff629d72640 10 monclient: _renew_subs 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff621ffb640 4 set_mon_vals no callback set 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff629d72640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff629d72640 1 -- 172.21.15.190:0/4152892924 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62409cbf0 con 0x7ff624136360 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 1 -- 172.21.15.190:0/4152892924 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff61c004040 con 0x7ff624136360 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff629d72640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff629d72640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff629d72640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff629d72640 1 -- 172.21.15.190:0/4152892924 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6240008e0 con 0x7ff624136360 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff61003be00 0x7ff61003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff60bfff640 1 -- 172.21.15.190:0/4152892924 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(254..254 src has 150..254) ==== 9686+0+0 (secure 0 0 0) 0x7ff61c0eacc0 con 0x7ff624136360 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff6237fe640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff61003be00 0x7ff61003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:40.161 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.145+0000 7ff6237fe640 10 monclient: get_auth_request con 0x7ff61003be00 auth_method 0 2024-08-19T10:45:40.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.149+0000 7ff6237fe640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff61003be00 0x7ff61003e2c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff614009af0 tx=0x7ff614008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:40.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.149+0000 7ff60bfff640 1 -- 172.21.15.190:0/4152892924 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff624134700 con 0x7ff624136360 2024-08-19T10:45:40.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.149+0000 7ff60bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:40.162 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.149+0000 7ff60bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:40.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.325+0000 7ff629d72640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h26"}] 2024-08-19T10:45:40.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.325+0000 7ff629d72640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h26"}] 2024-08-19T10:45:40.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.325+0000 7ff629d72640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:40.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:40.325+0000 7ff629d72640 1 -- 172.21.15.190:0/4152892924 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h26"} v 0) -- 0x7ff624139d60 con 0x7ff624136360 2024-08-19T10:45:41.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.453+0000 7ff60bfff640 1 -- 172.21.15.190:0/4152892924 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h26"}]=0 removed item id -94 name 'h26' from crush map v256) ==== 120+0+0 (secure 0 0 0) 0x7ff61c016540 con 0x7ff624136360 2024-08-19T10:45:41.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.453+0000 7ff60bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h26"}] 2024-08-19T10:45:41.456 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.453+0000 7ff60bfff640 10 monclient: _finish_command 2 = system:0 removed item id -94 name 'h26' from crush map 2024-08-19T10:45:41.565 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.565+0000 7ff629d72640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h26"}] 2024-08-19T10:45:41.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.565+0000 7ff629d72640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h26"}] 2024-08-19T10:45:41.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.565+0000 7ff629d72640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:41.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.565+0000 7ff629d72640 1 -- 172.21.15.190:0/4152892924 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd crush rm", "name": "h26"} v 0) -- 0x7ff62413a2a0 con 0x7ff624136360 2024-08-19T10:45:41.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.565+0000 7ff60bfff640 1 -- 172.21.15.190:0/4152892924 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h26"}]=0 device 'h26' does not appear in the crush map v256) ==== 120+0+0 (secure 0 0 0) 0x7ff61c0e8380 con 0x7ff624136360 2024-08-19T10:45:41.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.565+0000 7ff60bfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h26"}] 2024-08-19T10:45:41.567 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.565+0000 7ff60bfff640 10 monclient: _finish_command 3 = system:0 device 'h26' does not appear in the crush map 2024-08-19T10:45:41.567 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h26' does not appear in the crush map 2024-08-19T10:45:41.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 -- 172.21.15.190:0/4152892924 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff61003be00 msgr2=0x7ff61003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:41.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff61003be00 0x7ff61003e2c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff614009af0 tx=0x7ff614008040 comp rx=0 tx=0).stop 2024-08-19T10:45:41.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 10 monclient: shutdown 2024-08-19T10:45:41.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:41.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 -- 172.21.15.190:0/4152892924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff624136360 msgr2=0x7ff6241d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:41.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff624136360 0x7ff6241d0f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff61c01d0d0 tx=0x7ff61c0195d0 comp rx=0 tx=0).stop 2024-08-19T10:45:41.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 -- 172.21.15.190:0/4152892924 shutdown_connections 2024-08-19T10:45:41.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff61003be00 0x7ff61003e2c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:41.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff62413e8c0 0x7ff624135e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:41.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff624136360 0x7ff6241d0f20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:41.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 --2- 172.21.15.190:0/4152892924 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff624139980 0x7ff6241358e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:41.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 -- 172.21.15.190:0/4152892924 >> 172.21.15.190:0/4152892924 conn(0x7ff6240abde0 msgr2=0x7ff6240a9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:41.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 -- 172.21.15.190:0/4152892924 shutdown_connections 2024-08-19T10:45:41.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.569+0000 7ff629d72640 1 -- 172.21.15.190:0/4152892924 wait complete. 2024-08-19T10:45:41.586 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:41.586 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h27 2024-08-19T10:45:41.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.701+0000 7f9c98a5e640 10 monclient: get_monmap_and_config 2024-08-19T10:45:41.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.701+0000 7f9c98a5e640 10 monclient: build_initial_monmap 2024-08-19T10:45:41.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.701+0000 7f9c98a5e640 10 monclient: monmap: 2024-08-19T10:45:41.703 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:41.703 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:41.703 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:41.706243+0000 2024-08-19T10:45:41.703 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:41.706243+0000 2024-08-19T10:45:41.703 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:41.704 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:41.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.701+0000 7f9c98a5e640 1 Processor -- start 2024-08-19T10:45:41.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.701+0000 7f9c98a5e640 1 -- start start 2024-08-19T10:45:41.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.701+0000 7f9c98a5e640 10 monclient: init 2024-08-19T10:45:41.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:41.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 10 monclient: _add_conns ranks=[10,3,1,6,5,7,2,9,4,8,0] 2024-08-19T10:45:41.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9c94139980 0x7f9c94139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:41.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 10 monclient(hunting): picked mon.noname-f con 0x7f9c94139980 addr v2:172.21.15.190:3300/0 2024-08-19T10:45:41.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c9413a2a0 0x7f9c940a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:41.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 10 monclient(hunting): picked mon.noname-g con 0x7f9c9413a2a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:41.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9c940a5c70 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:41.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c940a8470 con 0x7f9c940a5c70 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9c9409ca60 con 0x7f9c94139980 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9c940a3640 con 0x7f9c9413a2a0 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c98a5e640 10 monclient(hunting): authenticate will time out at 1387.276263s 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c9259c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9c94139980 0x7f9c94139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c92d9d640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f9c940a5c70 0x7f9c940a67e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:48832/0 (socket says 172.21.15.190:48832) 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c92d9d640 1 -- 172.21.15.190:0/445296087 learned_addr learned my addr 172.21.15.190:0/445296087 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c9259c640 10 monclient(hunting): get_auth_request con 0x7f9c94139980 auth_method 0 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c9259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:41.708 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c9259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 1 --2- 172.21.15.190:0/445296087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c9413a2a0 0x7f9c940a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 10 monclient(hunting): get_auth_request con 0x7f9c9413a2a0 auth_method 0 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c9259c640 1 -- 172.21.15.190:0/445296087 >> v2:172.21.15.190:3300/0 conn(0x7f9c94139980 msgr2=0x7f9c94139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c9259c640 1 -- 172.21.15.190:0/445296087 >> v2:172.21.15.190:3300/0 conn(0x7f9c94139980 msgr2=0x7f9c94139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c9259c640 1 --2- 172.21.15.190:0/445296087 >> v2:172.21.15.190:3300/0 conn(0x7f9c94139980 0x7f9c94139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c9259c640 1 --2- 172.21.15.190:0/445296087 >> v2:172.21.15.190:3300/0 conn(0x7f9c94139980 0x7f9c94139d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:41.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 10 monclient(hunting): handle_auth_done global_id 34914 payload 274 2024-08-19T10:45:41.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 1 -- 172.21.15.190:0/445296087 >> v2:172.21.15.190:3300/0 conn(0x7f9c94139980 msgr2=0x7f9c94139d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:41.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 1 --2- 172.21.15.190:0/445296087 >> v2:172.21.15.190:3300/0 conn(0x7f9c94139980 0x7f9c94139d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:41.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 1 -- 172.21.15.190:0/445296087 >> v1:172.21.15.190:6789/0 conn(0x7f9c940a5c70 legacy=0x7f9c940a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:41.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:45:41.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 1 monclient: found mon.noname-g 2024-08-19T10:45:41.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:41.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:41.710 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.705+0000 7f9c91d9b640 1 -- 172.21.15.190:0/445296087 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c940a3850 con 0x7f9c9413a2a0 2024-08-19T10:45:41.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 1 -- 172.21.15.190:0/445296087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c9413a2a0 msgr2=0x7f9c940a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:41.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 1 --2- 172.21.15.190:0/445296087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c9413a2a0 0x7f9c940a5730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c840097c0 tx=0x7f9c8401c360 comp rx=0 tx=0).ready entity=mon.3 client_cookie=ae6d22d9c4641f7a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:41.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 1 -- 172.21.15.190:0/445296087 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9c840b0070 con 0x7f9c9413a2a0 2024-08-19T10:45:41.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:41.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:41.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 10 monclient: dump: 2024-08-19T10:45:41.714 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:41.714 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:41.714 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:41.714 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:41.714 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:41.714 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:41.714 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:41.714 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 10 monclient: _finish_auth 0 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 10 monclient: _check_auth_tickets 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 1 -- 172.21.15.190:0/445296087 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c840aa810 con 0x7f9c9413a2a0 2024-08-19T10:45:41.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 1 -- 172.21.15.190:0/445296087 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9c840aae30 con 0x7f9c9413a2a0 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9159a640 10 monclient: dump: 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:41.716 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 5 monclient: authenticate success, global_id 34914 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 4 set_mon_vals no callback set 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient: shutdown 2024-08-19T10:45:41.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 -- 172.21.15.190:0/445296087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c9413a2a0 msgr2=0x7f9c940a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 --2- 172.21.15.190:0/445296087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c9413a2a0 0x7f9c940a5730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c840097c0 tx=0x7f9c8401c360 comp rx=0 tx=0).stop 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 -- 172.21.15.190:0/445296087 shutdown_connections 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 --2- 172.21.15.190:0/445296087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c9413a2a0 0x7f9c940a5730 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 --2- 172.21.15.190:0/445296087 >> v2:172.21.15.190:3300/0 conn(0x7f9c94139980 0x7f9c94139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 -- 172.21.15.190:0/445296087 >> 172.21.15.190:0/445296087 conn(0x7f9c940abde0 msgr2=0x7f9c940ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 -- 172.21.15.190:0/445296087 shutdown_connections 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 -- 172.21.15.190:0/445296087 wait complete. 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient: build_initial_monmap 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient: monmap: 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:41.718 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:41.719 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 Processor -- start 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 -- start start 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient: init 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient: _add_conns ranks=[3,7,4,0,1,6,2,8,5] 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c940a5c70 0x7f9c94133990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient(hunting): picked mon.noname-g con 0x7f9c940a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9c94139980 0x7f9c94133ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient(hunting): picked mon.noname-i con 0x7f9c94139980 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c9413a690 0x7f9c9413ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient(hunting): picked mon.noname-c con 0x7f9c9413a690 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:41.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9c94144280 con 0x7f9c940a5c70 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9c94144100 con 0x7f9c9413a690 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9c94144400 con 0x7f9c94139980 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c98a5e640 10 monclient(hunting): authenticate will time out at 1387.283618s 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c940a5c70 0x7f9c94133990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9c94139980 0x7f9c94133ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9c94139980 0x7f9c94133ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34198/0 (socket says 172.21.15.190:34198) 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c940a5c70 0x7f9c94133990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46608/0 (socket says 172.21.15.190:46608) 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 1 -- 172.21.15.190:0/2196889908 learned_addr learned my addr 172.21.15.190:0/2196889908 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 10 monclient(hunting): get_auth_request con 0x7f9c94139980 auth_method 0 2024-08-19T10:45:41.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9259c640 10 monclient(hunting): get_auth_request con 0x7f9c940a5c70 auth_method 0 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c9259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:41.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.709+0000 7f9c91d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:41.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 10 monclient(hunting): handle_auth_done global_id 34923 payload 824 2024-08-19T10:45:41.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 1 -- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9c94139980 msgr2=0x7f9c94133ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:41.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9c94139980 0x7f9c94133ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:41.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 1 -- 172.21.15.190:0/2196889908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c9413a690 msgr2=0x7f9c9413ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:41.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c9413a690 0x7f9c9413ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:41.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 10 monclient: _finish_hunting 0 2024-08-19T10:45:41.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 1 monclient: found mon.noname-g 2024-08-19T10:45:41.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:41.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:41.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 1 -- 172.21.15.190:0/2196889908 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c940777a0 con 0x7f9c940a5c70 2024-08-19T10:45:41.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c91d9b640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9c94139980 0x7f9c94133ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 1 -- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c940a5c70 msgr2=0x7f9c94133990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c9259c640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c940a5c70 0x7f9c94133990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00e810 tx=0x7f9c7c00ecd0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 1 -- 172.21.15.190:0/2196889908 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9c7c017030 con 0x7f9c940a5c70 2024-08-19T10:45:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 10 monclient: dump: 2024-08-19T10:45:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:41.726 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 10 monclient: _finish_auth 0 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:41.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c98a5e640 5 monclient: authenticate success, global_id 34923 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 1 -- 172.21.15.190:0/2196889908 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c00cb40 con 0x7f9c940a5c70 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c98a5e640 10 monclient: _renew_subs 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c98a5e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c98a5e640 1 -- 172.21.15.190:0/2196889908 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9413c960 con 0x7f9c940a5c70 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 1 -- 172.21.15.190:0/2196889908 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9c7c00f040 con 0x7f9c940a5c70 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 10 monclient: dump: 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:41.728 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:41.729 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:41.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c90d99640 4 set_mon_vals no callback set 2024-08-19T10:45:41.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c98a5e640 10 monclient: _renew_subs 2024-08-19T10:45:41.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c98a5e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:41.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c98a5e640 1 -- 172.21.15.190:0/2196889908 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c940a1320 con 0x7f9c940a5c70 2024-08-19T10:45:41.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c98a5e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:41.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c98a5e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:41.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c98a5e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:41.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c98a5e640 1 -- 172.21.15.190:0/2196889908 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c94139d60 con 0x7f9c940a5c70 2024-08-19T10:45:41.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 1 -- 172.21.15.190:0/2196889908 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9c7c01a070 con 0x7f9c940a5c70 2024-08-19T10:45:41.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9c7403be00 0x7f9c7403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:41.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.713+0000 7f9c8affd640 1 -- 172.21.15.190:0/2196889908 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(256..256 src has 150..256) ==== 9473+0+0 (secure 0 0 0) 0x7f9c7c018070 con 0x7f9c940a5c70 2024-08-19T10:45:41.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.717+0000 7f9c91d9b640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9c7403be00 0x7f9c7403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:41.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.717+0000 7f9c91d9b640 10 monclient: get_auth_request con 0x7f9c7403be00 auth_method 0 2024-08-19T10:45:41.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.717+0000 7f9c91d9b640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9c7403be00 0x7f9c7403e2c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9c84000c10 tx=0x7f9c840a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:41.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.721+0000 7f9c8affd640 1 -- 172.21.15.190:0/2196889908 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9c7c021020 con 0x7f9c940a5c70 2024-08-19T10:45:41.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.721+0000 7f9c8affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:41.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.721+0000 7f9c8affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:41.895 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.893+0000 7f9c98a5e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h27"}] 2024-08-19T10:45:41.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.893+0000 7f9c98a5e640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h27"}] 2024-08-19T10:45:41.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.893+0000 7f9c98a5e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:41.896 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:41.893+0000 7f9c98a5e640 1 -- 172.21.15.190:0/2196889908 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h27"} v 0) -- 0x7f9c941391b0 con 0x7f9c940a5c70 2024-08-19T10:45:42.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.465+0000 7f9c8affd640 1 -- 172.21.15.190:0/2196889908 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h27"}]=0 removed item id -97 name 'h27' from crush map v257) ==== 120+0+0 (secure 0 0 0) 0x7f9c7c04c850 con 0x7f9c940a5c70 2024-08-19T10:45:42.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.465+0000 7f9c8affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h27"}] 2024-08-19T10:45:42.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.465+0000 7f9c8affd640 10 monclient: _finish_command 2 = system:0 removed item id -97 name 'h27' from crush map 2024-08-19T10:45:42.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.577+0000 7f9c98a5e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h27"}] 2024-08-19T10:45:42.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.577+0000 7f9c98a5e640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h27"}] 2024-08-19T10:45:42.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.577+0000 7f9c98a5e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:42.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.577+0000 7f9c98a5e640 1 -- 172.21.15.190:0/2196889908 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h27"} v 0) -- 0x7f9c940008e0 con 0x7f9c940a5c70 2024-08-19T10:45:42.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.577+0000 7f9c8affd640 1 -- 172.21.15.190:0/2196889908 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h27"}]=0 device 'h27' does not appear in the crush map v257) ==== 120+0+0 (secure 0 0 0) 0x7f9c7c01ad80 con 0x7f9c940a5c70 2024-08-19T10:45:42.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.577+0000 7f9c8affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h27"}] 2024-08-19T10:45:42.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.577+0000 7f9c8affd640 10 monclient: _finish_command 3 = system:0 device 'h27' does not appear in the crush map 2024-08-19T10:45:42.580 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h27' does not appear in the crush map 2024-08-19T10:45:42.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 -- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9c7403be00 msgr2=0x7f9c7403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:42.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9c7403be00 0x7f9c7403e2c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9c84000c10 tx=0x7f9c840a6000 comp rx=0 tx=0).stop 2024-08-19T10:45:42.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 10 monclient: shutdown 2024-08-19T10:45:42.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:42.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 -- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c940a5c70 msgr2=0x7f9c94133990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:42.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c940a5c70 0x7f9c94133990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00e810 tx=0x7f9c7c00ecd0 comp rx=0 tx=0).stop 2024-08-19T10:45:42.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 -- 172.21.15.190:0/2196889908 shutdown_connections 2024-08-19T10:45:42.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9c7403be00 0x7f9c7403e2c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:42.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9c9413a690 0x7f9c9413ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:42.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9c94139980 0x7f9c94133ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:42.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 --2- 172.21.15.190:0/2196889908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c940a5c70 0x7f9c94133990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:42.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 -- 172.21.15.190:0/2196889908 >> 172.21.15.190:0/2196889908 conn(0x7f9c940abde0 msgr2=0x7f9c94134ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:42.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 -- 172.21.15.190:0/2196889908 shutdown_connections 2024-08-19T10:45:42.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.581+0000 7f9c98a5e640 1 -- 172.21.15.190:0/2196889908 wait complete. 2024-08-19T10:45:42.598 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:42.598 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h28 2024-08-19T10:45:42.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient: get_monmap_and_config 2024-08-19T10:45:42.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient: build_initial_monmap 2024-08-19T10:45:42.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient: monmap: 2024-08-19T10:45:42.714 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:42.717308+0000 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:42.717308+0000 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:42.715 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:42.716 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:42.716 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:42.716 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:42.716 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:42.716 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:42.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 1 Processor -- start 2024-08-19T10:45:42.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 1 -- start start 2024-08-19T10:45:42.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient: init 2024-08-19T10:45:42.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:42.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient: _add_conns ranks=[5,10,1,9,6,7,8,4,0,2,3] 2024-08-19T10:45:42.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f46a40a56c0 0x7f46a4071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:42.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient(hunting): picked mon.noname-h con 0x7f46a40a56c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f46a40a5aa0 0x7f46a40a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient(hunting): picked mon.noname-f con 0x7f46a40a5aa0 addr v2:172.21.15.190:3300/0 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f46a4139490 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a406de60 con 0x7f46a4139490 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f46a406e0e0 con 0x7f46a40a5aa0 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f46a406f180 con 0x7f46a40a56c0 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46ab538640 10 monclient(hunting): authenticate will time out at 1388.287734s 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a92d4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f46a40a56c0 0x7f46a4071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a8ad3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f46a40a5aa0 0x7f46a40a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:42.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a9ad5640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f46a4139490 0x7f46a4139940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:48848/0 (socket says 172.21.15.190:48848) 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a92d4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f46a40a56c0 0x7f46a4071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:56528/0 (socket says 172.21.15.190:56528) 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a9ad5640 1 -- 172.21.15.190:0/3283298061 learned_addr learned my addr 172.21.15.190:0/3283298061 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a92d4640 10 monclient(hunting): get_auth_request con 0x7f46a40a56c0 auth_method 0 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a92d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a92d4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a92d4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a8ad3640 10 monclient(hunting): get_auth_request con 0x7f46a40a5aa0 auth_method 0 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a8ad3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a8ad3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a8ad3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.713+0000 7f46a92d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a92d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a92d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:42.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a8ad3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:42.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a8ad3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:42.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a8ad3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:42.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a92d4640 10 monclient(hunting): handle_auth_done global_id 6449 payload 274 2024-08-19T10:45:42.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a92d4640 1 -- 172.21.15.190:0/3283298061 >> v2:172.21.15.190:3300/0 conn(0x7f46a40a5aa0 msgr2=0x7f46a40a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:42.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a92d4640 1 --2- 172.21.15.190:0/3283298061 >> v2:172.21.15.190:3300/0 conn(0x7f46a40a5aa0 0x7f46a40a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:42.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a92d4640 1 -- 172.21.15.190:0/3283298061 >> v1:172.21.15.190:6789/0 conn(0x7f46a4139490 legacy=0x7f46a4139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:42.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a92d4640 10 monclient: _finish_hunting 0 2024-08-19T10:45:42.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a92d4640 1 monclient: found mon.noname-h 2024-08-19T10:45:42.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a92d4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:42.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a92d4640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:42.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.717+0000 7f46a92d4640 1 -- 172.21.15.190:0/3283298061 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a406f460 con 0x7f46a40a56c0 2024-08-19T10:45:42.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 1 -- 172.21.15.190:0/3283298061 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f46a40a56c0 msgr2=0x7f46a4071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:42.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 1 --2- 172.21.15.190:0/3283298061 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f46a40a56c0 0x7f46a4071fe0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f46a00097c0 tx=0x7f46a001c3f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=cbed559c656d3668 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:42.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 1 -- 172.21.15.190:0/3283298061 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46a00b0070 con 0x7f46a40a56c0 2024-08-19T10:45:42.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:42.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:42.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 10 monclient: dump: 2024-08-19T10:45:42.727 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:42.727 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:42.727 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:42.727 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:42.727 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:42.727 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:42.727 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 10 monclient: _finish_auth 0 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:42.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 5 monclient: authenticate success, global_id 6449 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 1 -- 172.21.15.190:0/3283298061 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a00aa810 con 0x7f46a40a56c0 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 4 set_mon_vals no callback set 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 1 -- 172.21.15.190:0/3283298061 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46a00a9300 con 0x7f46a40a56c0 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f469bfff640 10 monclient: dump: 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:42.729 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:42.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient: shutdown 2024-08-19T10:45:42.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:42.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 -- 172.21.15.190:0/3283298061 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f46a40a56c0 msgr2=0x7f46a4071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:42.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 --2- 172.21.15.190:0/3283298061 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f46a40a56c0 0x7f46a4071fe0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f46a00097c0 tx=0x7f46a001c3f0 comp rx=0 tx=0).stop 2024-08-19T10:45:42.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 -- 172.21.15.190:0/3283298061 shutdown_connections 2024-08-19T10:45:42.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 --2- 172.21.15.190:0/3283298061 >> v2:172.21.15.190:3300/0 conn(0x7f46a40a5aa0 0x7f46a40a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:42.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 --2- 172.21.15.190:0/3283298061 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f46a40a56c0 0x7f46a4071fe0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:42.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 -- 172.21.15.190:0/3283298061 >> 172.21.15.190:0/3283298061 conn(0x7f46a40aa2b0 msgr2=0x7f46a40a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:42.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 -- 172.21.15.190:0/3283298061 shutdown_connections 2024-08-19T10:45:42.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 -- 172.21.15.190:0/3283298061 wait complete. 2024-08-19T10:45:42.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient: build_initial_monmap 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient: monmap: 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:42.732 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 Processor -- start 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 -- start start 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient: init 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient: _add_conns ranks=[8,2,6,0,4,7,5,3,1] 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f46a40a56c0 0x7f46a413f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient(hunting): picked mon.noname-e con 0x7f46a40a56c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:42.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46a40a5aa0 0x7f46a4120300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient(hunting): picked mon.noname-b con 0x7f46a40a5aa0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f46a4139490 0x7f46a4120840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient(hunting): picked mon.noname-d con 0x7f46a4139490 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f46a4144190 con 0x7f46a40a5aa0 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f46a40a6400 con 0x7f46a4139490 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f46a4144310 con 0x7f46a40a56c0 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46ab538640 10 monclient(hunting): authenticate will time out at 1388.294729s 2024-08-19T10:45:42.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a8ad3640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46a40a5aa0 msgr2=0x7f46a4120300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a8ad3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46a40a5aa0 0x7f46a4120300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f46a40a56c0 0x7f46a413f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f46a40a56c0 0x7f46a413f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.190:50302/0 (socket says 172.21.15.190:50302) 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 1 -- 172.21.15.190:0/3930606328 learned_addr learned my addr 172.21.15.190:0/3930606328 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 10 monclient(hunting): get_auth_request con 0x7f46a40a56c0 auth_method 0 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a9ad5640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f46a4139490 0x7f46a4120840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a9ad5640 10 monclient(hunting): get_auth_request con 0x7f46a4139490 auth_method 0 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a9ad5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a9ad5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a9ad5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:42.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a9ad5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a9ad5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a9ad5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 10 monclient(hunting): handle_auth_done global_id 6353 payload 824 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 1 -- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f46a4139490 msgr2=0x7f46a4120840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f46a4139490 0x7f46a4120840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 1 -- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46a40a5aa0 msgr2=0x7f46a4120300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46a40a5aa0 0x7f46a4120300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 10 monclient: _finish_hunting 0 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 1 monclient: found mon.noname-e 2024-08-19T10:45:42.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:42.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:42.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.721+0000 7f46a92d4640 1 -- 172.21.15.190:0/3930606328 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a41228e0 con 0x7f46a40a56c0 2024-08-19T10:45:42.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.725+0000 7f46a92d4640 1 -- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f46a40a56c0 msgr2=0x7f46a413f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:42.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.725+0000 7f46a92d4640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f46a40a56c0 0x7f46a413f4d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f46a00ae040 tx=0x7f46a0019190 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:42.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 1 -- 172.21.15.190:0/3930606328 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46a00b0040 con 0x7f46a40a56c0 2024-08-19T10:45:42.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:42.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:42.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 10 monclient: dump: 2024-08-19T10:45:42.739 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:42.739 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:42.739 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:42.739 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:42.739 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:42.739 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:42.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:42.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 1 -- 172.21.15.190:0/3930606328 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a00b9070 con 0x7f46a40a56c0 2024-08-19T10:45:42.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:42.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 1 -- 172.21.15.190:0/3930606328 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46a00044b0 con 0x7f46a40a56c0 2024-08-19T10:45:42.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:42.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:42.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 10 monclient: dump: 2024-08-19T10:45:42.741 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:42.741 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:42.741 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:42.741 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:42.742 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46ab538640 5 monclient: authenticate success, global_id 6353 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f469b7fe640 4 set_mon_vals no callback set 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46ab538640 10 monclient: _renew_subs 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46ab538640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46ab538640 1 -- 172.21.15.190:0/3930606328 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a4122600 con 0x7f46a40a56c0 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46ab538640 10 monclient: _renew_subs 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46ab538640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46ab538640 1 -- 172.21.15.190:0/3930606328 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a4120d80 con 0x7f46a40a56c0 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46ab538640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46ab538640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46ab538640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46ab538640 1 -- 172.21.15.190:0/3930606328 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a4071fe0 con 0x7f46a40a56c0 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 1 -- 172.21.15.190:0/3930606328 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f46a0004040 con 0x7f46a40a56c0 2024-08-19T10:45:42.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f468803be30 0x7f468803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:42.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f4699ffb640 1 -- 172.21.15.190:0/3930606328 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(257..257 src has 150..257) ==== 9260+0+0 (secure 0 0 0) 0x7f46a00e9780 con 0x7f46a40a56c0 2024-08-19T10:45:42.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46a8ad3640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f468803be30 0x7f468803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:42.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.729+0000 7f46a8ad3640 10 monclient: get_auth_request con 0x7f468803be30 auth_method 0 2024-08-19T10:45:42.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.733+0000 7f46a8ad3640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f468803be30 0x7f468803e2f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f46a413f3b0 tx=0x7f4694009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:42.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.741+0000 7f4699ffb640 1 -- 172.21.15.190:0/3930606328 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f46a0016070 con 0x7f46a40a56c0 2024-08-19T10:45:42.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.741+0000 7f4699ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:42.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.741+0000 7f4699ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:42.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.917+0000 7f46ab538640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h28"}] 2024-08-19T10:45:42.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.917+0000 7f46ab538640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h28"}] 2024-08-19T10:45:42.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.917+0000 7f46ab538640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:42.920 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:42.917+0000 7f46ab538640 1 -- 172.21.15.190:0/3930606328 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush rm", "name": "h28"} v 0) -- 0x7f46a40a97d0 con 0x7f46a40a56c0 2024-08-19T10:45:43.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.481+0000 7f4699ffb640 1 -- 172.21.15.190:0/3930606328 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h28"}]=0 removed item id -100 name 'h28' from crush map v258) ==== 121+0+0 (secure 0 0 0) 0x7f46a00e6ff0 con 0x7f46a40a56c0 2024-08-19T10:45:43.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.481+0000 7f4699ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h28"}] 2024-08-19T10:45:43.483 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.481+0000 7f4699ffb640 10 monclient: _finish_command 2 = system:0 removed item id -100 name 'h28' from crush map 2024-08-19T10:45:43.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.585+0000 7f46ab538640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h28"}] 2024-08-19T10:45:43.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.585+0000 7f46ab538640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h28"}] 2024-08-19T10:45:43.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.585+0000 7f46ab538640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:43.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.585+0000 7f46ab538640 1 -- 172.21.15.190:0/3930606328 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd crush rm", "name": "h28"} v 0) -- 0x7f46a406fca0 con 0x7f46a40a56c0 2024-08-19T10:45:43.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f4699ffb640 1 -- 172.21.15.190:0/3930606328 <== mon.8 v2:172.21.15.89:3304/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h28"}]=0 device 'h28' does not appear in the crush map v258) ==== 120+0+0 (secure 0 0 0) 0x7f46a00b3aa0 con 0x7f46a40a56c0 2024-08-19T10:45:43.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f4699ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h28"}] 2024-08-19T10:45:43.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f4699ffb640 10 monclient: _finish_command 3 = system:0 device 'h28' does not appear in the crush map 2024-08-19T10:45:43.589 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h28' does not appear in the crush map 2024-08-19T10:45:43.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 -- 172.21.15.190:0/3930606328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f468803be30 msgr2=0x7f468803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:43.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f468803be30 0x7f468803e2f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f46a413f3b0 tx=0x7f4694009190 comp rx=0 tx=0).stop 2024-08-19T10:45:43.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 10 monclient: shutdown 2024-08-19T10:45:43.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:43.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 -- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f46a40a56c0 msgr2=0x7f46a413f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:43.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f46a40a56c0 0x7f46a413f4d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f46a00ae040 tx=0x7f46a0019190 comp rx=0 tx=0).stop 2024-08-19T10:45:43.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 -- 172.21.15.190:0/3930606328 shutdown_connections 2024-08-19T10:45:43.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f46a4139490 0x7f46a4120840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:43.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46a40a5aa0 0x7f46a4120300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:43.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f468803be30 0x7f468803e2f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:43.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 --2- 172.21.15.190:0/3930606328 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f46a40a56c0 0x7f46a413f4d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:43.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 -- 172.21.15.190:0/3930606328 >> 172.21.15.190:0/3930606328 conn(0x7f46a40aa2b0 msgr2=0x7f46a40a45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:43.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 -- 172.21.15.190:0/3930606328 shutdown_connections 2024-08-19T10:45:43.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.589+0000 7f46ab538640 1 -- 172.21.15.190:0/3930606328 wait complete. 2024-08-19T10:45:43.605 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:43.605 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h29 2024-08-19T10:45:43.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.717+0000 7f80ab6d6640 10 monclient: get_monmap_and_config 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.717+0000 7f80ab6d6640 10 monclient: build_initial_monmap 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.717+0000 7f80ab6d6640 10 monclient: monmap: 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:43.722638+0000 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:43.722638+0000 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:43.720 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:43.721 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:43.721 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:43.721 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:43.721 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:43.721 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:43.721 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:43.721 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:43.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 1 Processor -- start 2024-08-19T10:45:43.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 1 -- start start 2024-08-19T10:45:43.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 10 monclient: init 2024-08-19T10:45:43.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 10 monclient: _add_conns ranks=[3,0,9,4,6,8,7,1,2,5,10] 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80a40a3ae0 0x7f80a40a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 10 monclient(hunting): picked mon.noname-g con 0x7f80a40a3ae0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f80a40a4400 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f80a413f0b0 0x7f80a41414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 10 monclient(hunting): picked mon.noname-a con 0x7f80a413f0b0 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a4074510 con 0x7f80a40a4400 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f80a406d780 con 0x7f80a413f0b0 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f80a40a0dc0 con 0x7f80a40a3ae0 2024-08-19T10:45:43.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80ab6d6640 10 monclient(hunting): authenticate will time out at 1389.293186s 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80a40a3ae0 0x7f80a40a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80a40a3ae0 0x7f80a40a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46616/0 (socket says 172.21.15.190:46616) 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 1 -- 172.21.15.190:0/1495078500 learned_addr learned my addr 172.21.15.190:0/1495078500 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 10 monclient(hunting): get_auth_request con 0x7f80a40a3ae0 auth_method 0 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9c73640 1 --2- 172.21.15.190:0/1495078500 >> v2:172.21.15.89:3300/0 conn(0x7f80a413f0b0 0x7f80a41414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:43.725 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9c73640 10 monclient(hunting): get_auth_request con 0x7f80a413f0b0 auth_method 0 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9c73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9c73640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9c73640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 10 monclient(hunting): handle_auth_done global_id 34932 payload 274 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 1 -- 172.21.15.190:0/1495078500 >> v2:172.21.15.89:3300/0 conn(0x7f80a413f0b0 msgr2=0x7f80a41414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 1 --2- 172.21.15.190:0/1495078500 >> v2:172.21.15.89:3300/0 conn(0x7f80a413f0b0 0x7f80a41414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 1 -- 172.21.15.190:0/1495078500 >> v1:172.21.15.89:6789/0 conn(0x7f80a40a4400 legacy=0x7f80a413d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 10 monclient: _finish_hunting 0 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 1 monclient: found mon.noname-g 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:43.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.721+0000 7f80a9472640 1 -- 172.21.15.190:0/1495078500 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a40a05e0 con 0x7f80a40a3ae0 2024-08-19T10:45:43.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80a9472640 1 -- 172.21.15.190:0/1495078500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80a40a3ae0 msgr2=0x7f80a40a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:43.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80a9472640 1 --2- 172.21.15.190:0/1495078500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80a40a3ae0 0x7f80a40a3ec0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009870 tx=0x7f80a001c240 comp rx=0 tx=0).ready entity=mon.3 client_cookie=f7928eff8f8f6e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:43.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 1 -- 172.21.15.190:0/1495078500 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80a00b0070 con 0x7f80a40a3ae0 2024-08-19T10:45:43.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:43.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:43.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 10 monclient: dump: 2024-08-19T10:45:43.728 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:43.728 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:43.728 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:43.728 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:43.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 10 monclient: _finish_auth 0 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 1 -- 172.21.15.190:0/1495078500 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a00aa810 con 0x7f80a40a3ae0 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 1 -- 172.21.15.190:0/1495078500 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f80a00a9300 con 0x7f80a40a3ae0 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 5 monclient: authenticate success, global_id 34932 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 4 set_mon_vals no callback set 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f809bfff640 10 monclient: dump: 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:43.730 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:43.731 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:43.731 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:43.731 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:43.731 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:43.731 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:43.731 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:43.731 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:43.731 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 10 monclient: shutdown 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 1 -- 172.21.15.190:0/1495078500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80a40a3ae0 msgr2=0x7f80a40a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 1 --2- 172.21.15.190:0/1495078500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80a40a3ae0 0x7f80a40a3ec0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009870 tx=0x7f80a001c240 comp rx=0 tx=0).stop 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 1 -- 172.21.15.190:0/1495078500 shutdown_connections 2024-08-19T10:45:43.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 1 --2- 172.21.15.190:0/1495078500 >> v2:172.21.15.89:3300/0 conn(0x7f80a413f0b0 0x7f80a41414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 1 --2- 172.21.15.190:0/1495078500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80a40a3ae0 0x7f80a40a3ec0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 1 -- 172.21.15.190:0/1495078500 >> 172.21.15.190:0/1495078500 conn(0x7f80a406dde0 msgr2=0x7f80a406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 1 -- 172.21.15.190:0/1495078500 shutdown_connections 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 1 -- 172.21.15.190:0/1495078500 wait complete. 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 10 monclient: build_initial_monmap 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.725+0000 7f80ab6d6640 10 monclient: monmap: 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:43.733 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:43.734 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:43.734 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:43.734 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:43.734 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:43.734 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:43.734 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:43.734 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:43.734 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:43.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 1 Processor -- start 2024-08-19T10:45:43.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 1 -- start start 2024-08-19T10:45:43.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 10 monclient: init 2024-08-19T10:45:43.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:43.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 10 monclient: _add_conns ranks=[4,7,6,8,1,2,0,3,5] 2024-08-19T10:45:43.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80a40a3ae0 0x7f80a413a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:43.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 10 monclient(hunting): picked mon.noname-c con 0x7f80a40a3ae0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:43.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f80a40a4400 0x7f80a413a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 10 monclient(hunting): picked mon.noname-i con 0x7f80a40a4400 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f80a413f0b0 0x7f80a413ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 10 monclient(hunting): picked mon.noname-d con 0x7f80a413f0b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f80a40a4800 con 0x7f80a40a3ae0 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f80a406cbc0 con 0x7f80a413f0b0 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f80a40a4f30 con 0x7f80a40a4400 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80ab6d6640 10 monclient(hunting): authenticate will time out at 1389.301175s 2024-08-19T10:45:43.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f80a40a4400 0x7f80a413a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f80a40a4400 0x7f80a413a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:34206/0 (socket says 172.21.15.190:34206) 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 1 -- 172.21.15.190:0/832597541 learned_addr learned my addr 172.21.15.190:0/832597541 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 10 monclient(hunting): get_auth_request con 0x7f80a40a4400 auth_method 0 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a9472640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80a40a3ae0 0x7f80a413a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a9472640 10 monclient(hunting): get_auth_request con 0x7f80a40a3ae0 auth_method 0 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a9472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:43.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a9472640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a9472640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 10 monclient(hunting): handle_auth_done global_id 15802 payload 824 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a9c73640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f80a413f0b0 0x7f80a413ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 1 -- 172.21.15.190:0/832597541 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f80a413f0b0 msgr2=0x7f80a413ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f80a413f0b0 0x7f80a413ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 1 -- 172.21.15.190:0/832597541 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80a40a3ae0 msgr2=0x7f80a413a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80a40a3ae0 0x7f80a413a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 10 monclient: _finish_hunting 0 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 1 monclient: found mon.noname-i 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:43.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 1 -- 172.21.15.190:0/832597541 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a40a2330 con 0x7f80a40a4400 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 1 -- 172.21.15.190:0/832597541 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f80a40a4400 msgr2=0x7f80a413a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.729+0000 7f80a8c71640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f80a40a4400 0x7f80a413a850 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f809400c440 tx=0x7f809400c900 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 1 -- 172.21.15.190:0/832597541 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8094016020 con 0x7f80a40a4400 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 10 monclient: dump: 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:43.742 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:43.743 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80ab6d6640 5 monclient: authenticate success, global_id 15802 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80ab6d6640 10 monclient: _renew_subs 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 1 -- 172.21.15.190:0/832597541 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8094012cb0 con 0x7f80a40a4400 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80ab6d6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80ab6d6640 1 -- 172.21.15.190:0/832597541 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a41251d0 con 0x7f80a40a4400 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 1 -- 172.21.15.190:0/832597541 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8094012e60 con 0x7f80a40a4400 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f809b7fe640 4 set_mon_vals no callback set 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 10 monclient: dump: 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:43.744 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:43.745 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80ab6d6640 10 monclient: _renew_subs 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80ab6d6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80ab6d6640 1 -- 172.21.15.190:0/832597541 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a4124930 con 0x7f80a40a4400 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80ab6d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80ab6d6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80ab6d6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80ab6d6640 1 -- 172.21.15.190:0/832597541 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a4140ca0 con 0x7f80a40a4400 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 1 -- 172.21.15.190:0/832597541 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8094019020 con 0x7f80a40a4400 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f808803be00 0x7f808803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f8099ffb640 1 -- 172.21.15.190:0/832597541 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(258..258 src has 150..258) ==== 9047+0+0 (secure 0 0 0) 0x7f809404f8c0 con 0x7f80a40a4400 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80a9472640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f808803be00 0x7f808803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.733+0000 7f80a9472640 10 monclient: get_auth_request con 0x7f808803be00 auth_method 0 2024-08-19T10:45:43.746 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.737+0000 7f80a9472640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f808803be00 0x7f808803e2c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f80a406cb30 tx=0x7f80a00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:43.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.741+0000 7f8099ffb640 1 -- 172.21.15.190:0/832597541 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f80a4140ca0 con 0x7f80a40a4400 2024-08-19T10:45:43.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.741+0000 7f8099ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:43.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.741+0000 7f8099ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:43.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.901+0000 7f80ab6d6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h29"}] 2024-08-19T10:45:43.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.901+0000 7f80ab6d6640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h29"}] 2024-08-19T10:45:43.901 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.901+0000 7f80ab6d6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:43.902 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:43.901+0000 7f80ab6d6640 1 -- 172.21.15.190:0/832597541 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h29"} v 0) -- 0x7f80a40a3ec0 con 0x7f80a40a4400 2024-08-19T10:45:44.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.489+0000 7f8099ffb640 1 -- 172.21.15.190:0/832597541 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h29"}]=0 removed item id -103 name 'h29' from crush map v259) ==== 121+0+0 (secure 0 0 0) 0x7f8094020030 con 0x7f80a40a4400 2024-08-19T10:45:44.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.489+0000 7f8099ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h29"}] 2024-08-19T10:45:44.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.489+0000 7f8099ffb640 10 monclient: _finish_command 2 = system:0 removed item id -103 name 'h29' from crush map 2024-08-19T10:45:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.593+0000 7f80ab6d6640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h29"}] 2024-08-19T10:45:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.593+0000 7f80ab6d6640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h29"}] 2024-08-19T10:45:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.593+0000 7f80ab6d6640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:44.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.593+0000 7f80ab6d6640 1 -- 172.21.15.190:0/832597541 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h29"} v 0) -- 0x7f80a4140a30 con 0x7f80a40a4400 2024-08-19T10:45:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f8099ffb640 1 -- 172.21.15.190:0/832597541 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h29"}]=0 device 'h29' does not appear in the crush map v259) ==== 120+0+0 (secure 0 0 0) 0x7f80a40a2330 con 0x7f80a40a4400 2024-08-19T10:45:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f8099ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h29"}] 2024-08-19T10:45:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f8099ffb640 10 monclient: _finish_command 3 = system:0 device 'h29' does not appear in the crush map 2024-08-19T10:45:44.597 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h29' does not appear in the crush map 2024-08-19T10:45:44.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 -- 172.21.15.190:0/832597541 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f808803be00 msgr2=0x7f808803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:44.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f808803be00 0x7f808803e2c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f80a406cb30 tx=0x7f80a00a6000 comp rx=0 tx=0).stop 2024-08-19T10:45:44.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 10 monclient: shutdown 2024-08-19T10:45:44.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:44.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 -- 172.21.15.190:0/832597541 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f80a40a4400 msgr2=0x7f80a413a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:44.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f80a40a4400 0x7f80a413a850 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f809400c440 tx=0x7f809400c900 comp rx=0 tx=0).stop 2024-08-19T10:45:44.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 -- 172.21.15.190:0/832597541 shutdown_connections 2024-08-19T10:45:44.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f808803be00 0x7f808803e2c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:44.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f80a413f0b0 0x7f80a413ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:44.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f80a40a4400 0x7f80a413a850 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:44.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 --2- 172.21.15.190:0/832597541 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f80a40a3ae0 0x7f80a413a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:44.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 -- 172.21.15.190:0/832597541 >> 172.21.15.190:0/832597541 conn(0x7f80a406dde0 msgr2=0x7f80a40a6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:44.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 -- 172.21.15.190:0/832597541 shutdown_connections 2024-08-19T10:45:44.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.597+0000 7f80ab6d6640 1 -- 172.21.15.190:0/832597541 wait complete. 2024-08-19T10:45:44.614 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:44.614 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h30 2024-08-19T10:45:44.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient: get_monmap_and_config 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient: build_initial_monmap 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient: monmap: 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:44.736752+0000 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:44.736752+0000 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:44.734 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:44.735 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:44.735 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:44.735 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:44.735 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:44.735 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:44.735 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:44.735 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:44.735 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:44.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 1 Processor -- start 2024-08-19T10:45:44.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 1 -- start start 2024-08-19T10:45:44.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient: init 2024-08-19T10:45:44.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:44.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient: _add_conns ranks=[1,5,4,2,8,0,10,3,7,9,6] 2024-08-19T10:45:44.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe318137830 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:44.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3180a2f20 0x7fe31813c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient(hunting): picked mon.noname-h con 0x7fe3180a2f20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3180a2860 0x7fe31813ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient(hunting): picked mon.noname-c con 0x7fe3180a2860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3180a25d0 con 0x7fe318137830 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fe31809fb20 con 0x7fe3180a2860 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fe3180a2ce0 con 0x7fe3180a2f20 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe32020e640 10 monclient(hunting): authenticate will time out at 1390.306654s 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31dfaa640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe318137830 0x7fe318137c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:48864/0 (socket says 172.21.15.190:48864) 2024-08-19T10:45:44.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31dfaa640 1 -- 172.21.15.190:0/1828004200 learned_addr learned my addr 172.21.15.190:0/1828004200 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 1 --2- 172.21.15.190:0/1828004200 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3180a2f20 0x7fe31813c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 10 monclient(hunting): get_auth_request con 0x7fe3180a2f20 auth_method 0 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31e7ab640 1 --2- 172.21.15.190:0/1828004200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3180a2860 0x7fe31813ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31e7ab640 10 monclient(hunting): get_auth_request con 0x7fe3180a2860 auth_method 0 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31e7ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31e7ab640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:44.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31e7ab640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:44.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 10 monclient(hunting): handle_auth_done global_id 6458 payload 274 2024-08-19T10:45:44.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 1 -- 172.21.15.190:0/1828004200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3180a2860 msgr2=0x7fe31813ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:44.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 1 --2- 172.21.15.190:0/1828004200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3180a2860 0x7fe31813ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:44.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 1 -- 172.21.15.190:0/1828004200 >> v1:172.21.15.190:6789/0 conn(0x7fe318137830 legacy=0x7fe318137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:44.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 10 monclient: _finish_hunting 0 2024-08-19T10:45:44.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 1 monclient: found mon.noname-h 2024-08-19T10:45:44.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:44.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:44.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.733+0000 7fe31d7a9640 1 -- 172.21.15.190:0/1828004200 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31809cbe0 con 0x7fe3180a2f20 2024-08-19T10:45:44.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31d7a9640 1 -- 172.21.15.190:0/1828004200 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3180a2f20 msgr2=0x7fe31813c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:44.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31d7a9640 1 --2- 172.21.15.190:0/1828004200 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3180a2f20 0x7fe31813c6b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0097c0 tx=0x7fe30c01c360 comp rx=0 tx=0).ready entity=mon.5 client_cookie=3b9d5ec8c2ff018d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:44.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 1 -- 172.21.15.190:0/1828004200 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe30c0b0070 con 0x7fe3180a2f20 2024-08-19T10:45:44.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:44.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:44.745 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 10 monclient: dump: 2024-08-19T10:45:44.745 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:44.746 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 10 monclient: _finish_auth 0 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 10 monclient: _check_auth_tickets 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 1 -- 172.21.15.190:0/1828004200 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30c0aa810 con 0x7fe3180a2f20 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 5 monclient: authenticate success, global_id 6458 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 1 -- 172.21.15.190:0/1828004200 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe30c0a9300 con 0x7fe3180a2f20 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 4 set_mon_vals no callback set 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:44.747 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31cfa8640 10 monclient: dump: 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:44.748 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient: shutdown 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 -- 172.21.15.190:0/1828004200 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3180a2f20 msgr2=0x7fe31813c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 --2- 172.21.15.190:0/1828004200 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3180a2f20 0x7fe31813c6b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0097c0 tx=0x7fe30c01c360 comp rx=0 tx=0).stop 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 -- 172.21.15.190:0/1828004200 shutdown_connections 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 --2- 172.21.15.190:0/1828004200 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3180a2860 0x7fe31813ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 --2- 172.21.15.190:0/1828004200 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fe3180a2f20 0x7fe31813c6b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 -- 172.21.15.190:0/1828004200 >> 172.21.15.190:0/1828004200 conn(0x7fe3180a92a0 msgr2=0x7fe3180a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:44.749 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 -- 172.21.15.190:0/1828004200 shutdown_connections 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 -- 172.21.15.190:0/1828004200 wait complete. 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient: build_initial_monmap 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient: monmap: 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:44.750 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 Processor -- start 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 -- start start 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient: init 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient: _add_conns ranks=[4,2,3,5,8,1,0,6,7] 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3180a2860 0x7fe318136760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient(hunting): picked mon.noname-c con 0x7fe3180a2860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:44.751 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3180a2f20 0x7fe31811f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient(hunting): picked mon.noname-b con 0x7fe3180a2f20 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe318137830 0x7fe31811f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient(hunting): picked mon.noname-g con 0x7fe318137830 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fe318141ae0 con 0x7fe3180a2f20 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe318141960 con 0x7fe318137830 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fe318141c60 con 0x7fe3180a2860 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe32020e640 10 monclient(hunting): authenticate will time out at 1390.315245s 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31d7a9640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3180a2f20 msgr2=0x7fe31811f2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:44.752 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe318137830 0x7fe31811f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31d7a9640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3180a2f20 0x7fe31811f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe318137830 0x7fe31811f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:46628/0 (socket says 172.21.15.190:46628) 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 1 -- 172.21.15.190:0/436401780 learned_addr learned my addr 172.21.15.190:0/436401780 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 10 monclient(hunting): get_auth_request con 0x7fe318137830 auth_method 0 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31dfaa640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3180a2860 0x7fe318136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31dfaa640 10 monclient(hunting): get_auth_request con 0x7fe3180a2860 auth_method 0 2024-08-19T10:45:44.753 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31dfaa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31dfaa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31dfaa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31dfaa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31dfaa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31dfaa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 10 monclient(hunting): handle_auth_done global_id 34941 payload 824 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 1 -- 172.21.15.190:0/436401780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3180a2860 msgr2=0x7fe318136760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3180a2860 0x7fe318136760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 1 -- 172.21.15.190:0/436401780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3180a2f20 msgr2=0x7fe31811f2f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3180a2f20 0x7fe31811f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 10 monclient: _finish_hunting 0 2024-08-19T10:45:44.754 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 1 monclient: found mon.noname-g 2024-08-19T10:45:44.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:44.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:44.755 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.741+0000 7fe31e7ab640 1 -- 172.21.15.190:0/436401780 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe318121400 con 0x7fe318137830 2024-08-19T10:45:44.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe31e7ab640 1 -- 172.21.15.190:0/436401780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe318137830 msgr2=0x7fe31811f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:44.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe31e7ab640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe318137830 0x7fe31811f830 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe308007e40 tx=0x7fe30800e4b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:44.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 1 -- 172.21.15.190:0/436401780 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe308013a00 con 0x7fe318137830 2024-08-19T10:45:44.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 10 monclient: dump: 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:44.758 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 1 -- 172.21.15.190:0/436401780 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe308019070 con 0x7fe318137830 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 1 -- 172.21.15.190:0/436401780 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe308013c00 con 0x7fe318137830 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:44.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 10 monclient: dump: 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:44.760 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe317fff640 4 set_mon_vals no callback set 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe32020e640 5 monclient: authenticate success, global_id 34941 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe32020e640 10 monclient: _renew_subs 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe32020e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe32020e640 1 -- 172.21.15.190:0/436401780 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3180a8700 con 0x7fe318137830 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe32020e640 10 monclient: _renew_subs 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe32020e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe32020e640 1 -- 172.21.15.190:0/436401780 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3180a82f0 con 0x7fe318137830 2024-08-19T10:45:44.761 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe32020e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe32020e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe32020e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe32020e640 1 -- 172.21.15.190:0/436401780 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe318137c10 con 0x7fe318137830 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.745+0000 7fe3167fc640 1 -- 172.21.15.190:0/436401780 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe308013590 con 0x7fe318137830 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.749+0000 7fe3167fc640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30403be00 0x7fe30403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.749+0000 7fe3167fc640 1 -- 172.21.15.190:0/436401780 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(259..259 src has 150..259) ==== 8834+0+0 (secure 0 0 0) 0x7fe3080502f0 con 0x7fe318137830 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.749+0000 7fe31dfaa640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30403be00 0x7fe30403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.749+0000 7fe31dfaa640 10 monclient: get_auth_request con 0x7fe30403be00 auth_method 0 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.749+0000 7fe31dfaa640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30403be00 0x7fe30403e2c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe3100092c0 tx=0x7fe310009250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.753+0000 7fe3167fc640 1 -- 172.21.15.190:0/436401780 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe308013da0 con 0x7fe318137830 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.753+0000 7fe3167fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:44.762 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.753+0000 7fe3167fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:44.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.913+0000 7fe32020e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h30"}] 2024-08-19T10:45:44.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.913+0000 7fe32020e640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h30"}] 2024-08-19T10:45:44.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.913+0000 7fe32020e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:44.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:44.913+0000 7fe32020e640 1 -- 172.21.15.190:0/436401780 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h30"} v 0) -- 0x7fe318121680 con 0x7fe318137830 2024-08-19T10:45:46.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.101+0000 7fe3167fc640 1 -- 172.21.15.190:0/436401780 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h30"}]=0 removed item id -106 name 'h30' from crush map v260) ==== 121+0+0 (secure 0 0 0) 0x7fe30804dd00 con 0x7fe318137830 2024-08-19T10:45:46.101 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.101+0000 7fe3167fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h30"}] 2024-08-19T10:45:46.102 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.101+0000 7fe3167fc640 10 monclient: _finish_command 2 = system:0 removed item id -106 name 'h30' from crush map 2024-08-19T10:45:46.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.205+0000 7fe32020e640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h30"}] 2024-08-19T10:45:46.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.205+0000 7fe32020e640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h30"}] 2024-08-19T10:45:46.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.205+0000 7fe32020e640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:46.206 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.205+0000 7fe32020e640 1 -- 172.21.15.190:0/436401780 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h30"} v 0) -- 0x7fe3180008e0 con 0x7fe318137830 2024-08-19T10:45:46.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.205+0000 7fe3167fc640 1 -- 172.21.15.190:0/436401780 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h30"}]=0 device 'h30' does not appear in the crush map v260) ==== 120+0+0 (secure 0 0 0) 0x7fe308012870 con 0x7fe318137830 2024-08-19T10:45:46.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.205+0000 7fe3167fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h30"}] 2024-08-19T10:45:46.208 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.205+0000 7fe3167fc640 10 monclient: _finish_command 3 = system:0 device 'h30' does not appear in the crush map 2024-08-19T10:45:46.208 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h30' does not appear in the crush map 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 -- 172.21.15.190:0/436401780 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30403be00 msgr2=0x7fe30403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30403be00 0x7fe30403e2c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe3100092c0 tx=0x7fe310009250 comp rx=0 tx=0).stop 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 10 monclient: shutdown 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 -- 172.21.15.190:0/436401780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe318137830 msgr2=0x7fe31811f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe318137830 0x7fe31811f830 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe308007e40 tx=0x7fe30800e4b0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 -- 172.21.15.190:0/436401780 shutdown_connections 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe30403be00 0x7fe30403e2c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe318137830 0x7fe31811f830 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fe3180a2f20 0x7fe31811f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 --2- 172.21.15.190:0/436401780 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fe3180a2860 0x7fe318136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 -- 172.21.15.190:0/436401780 >> 172.21.15.190:0/436401780 conn(0x7fe3180a92a0 msgr2=0x7fe31809e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:46.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 -- 172.21.15.190:0/436401780 shutdown_connections 2024-08-19T10:45:46.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.209+0000 7fe32020e640 1 -- 172.21.15.190:0/436401780 wait complete. 2024-08-19T10:45:46.226 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:46.227 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h31 2024-08-19T10:45:46.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient: get_monmap_and_config 2024-08-19T10:45:46.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient: build_initial_monmap 2024-08-19T10:45:46.338 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient: monmap: 2024-08-19T10:45:46.338 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:46.338 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:46.338 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:46.341146+0000 2024-08-19T10:45:46.338 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:46.341146+0000 2024-08-19T10:45:46.338 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:46.339 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:46.339 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:46.339 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:46.339 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:46.339 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:46.339 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:46.339 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:46.339 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:46.339 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:46.339 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:46.340 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:46.340 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:46.340 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:46.342 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 1 Processor -- start 2024-08-19T10:45:46.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 1 -- start start 2024-08-19T10:45:46.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient: init 2024-08-19T10:45:46.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:46.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient: _add_conns ranks=[4,3,2,9,10,0,5,1,7,6,8] 2024-08-19T10:45:46.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd058139980 0x7fd058139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:46.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient(hunting): picked mon.noname-c con 0x7fd058139980 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:46.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05813a2a0 0x7fd0580a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:46.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient(hunting): picked mon.noname-g con 0x7fd05813a2a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:46.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0580a5c70 0x7fd0580a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:46.343 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient(hunting): picked mon.noname-b con 0x7fd0580a5c70 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd05809ca60 con 0x7fd0580a5c70 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd0580a3640 con 0x7fd05813a2a0 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd0580a8470 con 0x7fd058139980 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05c8e4640 10 monclient(hunting): authenticate will time out at 1391.911017s 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd056d9d640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0580a5c70 msgr2=0x7fd0580a67e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05813a2a0 0x7fd0580a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd056d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0580a5c70 0x7fd0580a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:46.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05813a2a0 0x7fd0580a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47264/0 (socket says 172.21.15.190:47264) 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 1 -- 172.21.15.190:0/510339068 learned_addr learned my addr 172.21.15.190:0/510339068 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 10 monclient(hunting): get_auth_request con 0x7fd05813a2a0 auth_method 0 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05659c640 1 --2- 172.21.15.190:0/510339068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd058139980 0x7fd058139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05659c640 10 monclient(hunting): get_auth_request con 0x7fd058139980 auth_method 0 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:46.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd05659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:46.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 10 monclient(hunting): handle_auth_done global_id 34950 payload 274 2024-08-19T10:45:46.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 1 -- 172.21.15.190:0/510339068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd058139980 msgr2=0x7fd058139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:46.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 1 --2- 172.21.15.190:0/510339068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd058139980 0x7fd058139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 1 -- 172.21.15.190:0/510339068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0580a5c70 msgr2=0x7fd0580a67e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:46.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 1 --2- 172.21.15.190:0/510339068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0580a5c70 0x7fd0580a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:45:46.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 1 monclient: found mon.noname-g 2024-08-19T10:45:46.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:46.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:46.346 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.337+0000 7fd04ffff640 1 -- 172.21.15.190:0/510339068 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0580a3850 con 0x7fd05813a2a0 2024-08-19T10:45:46.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd04ffff640 1 -- 172.21.15.190:0/510339068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05813a2a0 msgr2=0x7fd0580a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:46.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd04ffff640 1 --2- 172.21.15.190:0/510339068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05813a2a0 0x7fd0580a5730 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd0400098d0 tx=0x7fd04001c170 comp rx=0 tx=0).ready entity=mon.3 client_cookie=e82647bf31cb3a79 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:46.351 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 1 -- 172.21.15.190:0/510339068 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd0400b0070 con 0x7fd05813a2a0 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 10 monclient: dump: 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:46.352 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 10 monclient: _finish_auth 0 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 1 -- 172.21.15.190:0/510339068 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0400aa810 con 0x7fd05813a2a0 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 5 monclient: authenticate success, global_id 34950 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:46.353 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 1 -- 172.21.15.190:0/510339068 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd0400aae30 con 0x7fd05813a2a0 2024-08-19T10:45:46.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:46.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 4 set_mon_vals no callback set 2024-08-19T10:45:46.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:46.354 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd055d9b640 10 monclient: dump: 2024-08-19T10:45:46.354 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:46.354 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:46.354 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:46.354 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:46.354 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:46.354 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:46.354 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient: shutdown 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 -- 172.21.15.190:0/510339068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05813a2a0 msgr2=0x7fd0580a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:46.355 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 --2- 172.21.15.190:0/510339068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05813a2a0 0x7fd0580a5730 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd0400098d0 tx=0x7fd04001c170 comp rx=0 tx=0).stop 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 -- 172.21.15.190:0/510339068 shutdown_connections 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 --2- 172.21.15.190:0/510339068 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd0580a5c70 0x7fd0580a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 --2- 172.21.15.190:0/510339068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05813a2a0 0x7fd0580a5730 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 --2- 172.21.15.190:0/510339068 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd058139980 0x7fd058139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 -- 172.21.15.190:0/510339068 >> 172.21.15.190:0/510339068 conn(0x7fd0580abde0 msgr2=0x7fd0580ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 -- 172.21.15.190:0/510339068 shutdown_connections 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 -- 172.21.15.190:0/510339068 wait complete. 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient: build_initial_monmap 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient: monmap: 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:46.356 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 Processor -- start 2024-08-19T10:45:46.357 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 -- start start 2024-08-19T10:45:46.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient: init 2024-08-19T10:45:46.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:46.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient: _add_conns ranks=[6,2,3,4,1,5,8,0,7] 2024-08-19T10:45:46.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd0580a5c70 0x7fd05809f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:46.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient(hunting): picked mon.noname-d con 0x7fd0580a5c70 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:46.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd058139980 0x7fd05809f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:46.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient(hunting): picked mon.noname-b con 0x7fd058139980 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:46.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05809fe50 0x7fd0581d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:46.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient(hunting): picked mon.noname-g con 0x7fd05809fe50 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:46.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:46.358 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd058143270 con 0x7fd058139980 2024-08-19T10:45:46.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:46.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd0581430f0 con 0x7fd05809fe50 2024-08-19T10:45:46.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:46.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd0581433f0 con 0x7fd0580a5c70 2024-08-19T10:45:46.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:46.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05c8e4640 10 monclient(hunting): authenticate will time out at 1391.918727s 2024-08-19T10:45:46.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05809fe50 0x7fd0581d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:46.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd04ffff640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd058139980 msgr2=0x7fd05809f910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:46.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05809fe50 0x7fd0581d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47266/0 (socket says 172.21.15.190:47266) 2024-08-19T10:45:46.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 1 -- 172.21.15.190:0/1225725988 learned_addr learned my addr 172.21.15.190:0/1225725988 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:46.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 10 monclient(hunting): get_auth_request con 0x7fd05809fe50 auth_method 0 2024-08-19T10:45:46.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:46.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:46.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:46.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd04ffff640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd058139980 0x7fd05809f910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:46.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:46.360 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 10 monclient(hunting): handle_auth_done global_id 34959 payload 824 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd05659c640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd0580a5c70 0x7fd05809f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 1 -- 172.21.15.190:0/1225725988 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd0580a5c70 msgr2=0x7fd05809f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd0580a5c70 0x7fd05809f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 1 -- 172.21.15.190:0/1225725988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd058139980 msgr2=0x7fd05809f910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd058139980 0x7fd05809f910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 1 monclient: found mon.noname-g 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:46.361 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.345+0000 7fd056d9d640 1 -- 172.21.15.190:0/1225725988 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05813a690 con 0x7fd05809fe50 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd056d9d640 1 -- 172.21.15.190:0/1225725988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05809fe50 msgr2=0x7fd0581d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd056d9d640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05809fe50 0x7fd0581d1410 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd05000bdc0 tx=0x7fd050009eb0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 1 -- 172.21.15.190:0/1225725988 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd050006960 con 0x7fd05809fe50 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 10 monclient: dump: 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:46.362 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:46.363 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:46.363 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:46.363 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:46.363 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:46.363 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:46.363 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:46.363 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:46.363 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:46.363 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:46.363 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:46.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 10 monclient: _finish_auth 0 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 1 -- 172.21.15.190:0/1225725988 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd050006b60 con 0x7fd05809fe50 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05c8e4640 5 monclient: authenticate success, global_id 34959 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05c8e4640 10 monclient: _renew_subs 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05c8e4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05c8e4640 1 -- 172.21.15.190:0/1225725988 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0580a0dd0 con 0x7fd05809fe50 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 1 -- 172.21.15.190:0/1225725988 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd0500158f0 con 0x7fd05809fe50 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd054d99640 4 set_mon_vals no callback set 2024-08-19T10:45:46.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 10 monclient: dump: 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:46.365 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:46.366 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:46.366 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:46.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05c8e4640 10 monclient: _renew_subs 2024-08-19T10:45:46.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05c8e4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:46.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05c8e4640 1 -- 172.21.15.190:0/1225725988 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05813aa50 con 0x7fd05809fe50 2024-08-19T10:45:46.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05c8e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:46.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05c8e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:46.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05c8e4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:46.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05c8e4640 1 -- 172.21.15.190:0/1225725988 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd058139d60 con 0x7fd05809fe50 2024-08-19T10:45:46.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 1 -- 172.21.15.190:0/1225725988 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd0500064f0 con 0x7fd05809fe50 2024-08-19T10:45:46.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd03803be00 0x7fd03803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:46.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd04effd640 1 -- 172.21.15.190:0/1225725988 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(260..260 src has 150..260) ==== 8621+0+0 (secure 0 0 0) 0x7fd0500509b0 con 0x7fd05809fe50 2024-08-19T10:45:46.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05659c640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd03803be00 0x7fd03803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:46.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.349+0000 7fd05659c640 10 monclient: get_auth_request con 0x7fd03803be00 auth_method 0 2024-08-19T10:45:46.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.353+0000 7fd05659c640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd03803be00 0x7fd03803e2c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd048009790 tx=0x7fd048008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:46.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.353+0000 7fd04effd640 1 -- 172.21.15.190:0/1225725988 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd050012330 con 0x7fd05809fe50 2024-08-19T10:45:46.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.353+0000 7fd04effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:46.368 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.353+0000 7fd04effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:46.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.533+0000 7fd05c8e4640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h31"}] 2024-08-19T10:45:46.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.533+0000 7fd05c8e4640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h31"}] 2024-08-19T10:45:46.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.533+0000 7fd05c8e4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:46.533 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:46.533+0000 7fd05c8e4640 1 -- 172.21.15.190:0/1225725988 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h31"} v 0) -- 0x7fd0581405f0 con 0x7fd05809fe50 2024-08-19T10:45:47.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.117+0000 7fd04effd640 1 -- 172.21.15.190:0/1225725988 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h31"}]=0 removed item id -109 name 'h31' from crush map v261) ==== 121+0+0 (secure 0 0 0) 0x7fd050013db0 con 0x7fd05809fe50 2024-08-19T10:45:47.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.117+0000 7fd04effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h31"}] 2024-08-19T10:45:47.120 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.117+0000 7fd04effd640 10 monclient: _finish_command 2 = system:0 removed item id -109 name 'h31' from crush map 2024-08-19T10:45:47.223 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.221+0000 7fd05c8e4640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h31"}] 2024-08-19T10:45:47.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.221+0000 7fd05c8e4640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h31"}] 2024-08-19T10:45:47.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.221+0000 7fd05c8e4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:47.224 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.221+0000 7fd05c8e4640 1 -- 172.21.15.190:0/1225725988 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd crush rm", "name": "h31"} v 0) -- 0x7fd0580008e0 con 0x7fd05809fe50 2024-08-19T10:45:47.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.225+0000 7fd04effd640 1 -- 172.21.15.190:0/1225725988 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h31"}]=0 device 'h31' does not appear in the crush map v261) ==== 120+0+0 (secure 0 0 0) 0x7fd05004e4a0 con 0x7fd05809fe50 2024-08-19T10:45:47.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.225+0000 7fd04effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h31"}] 2024-08-19T10:45:47.226 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.225+0000 7fd04effd640 10 monclient: _finish_command 3 = system:0 device 'h31' does not appear in the crush map 2024-08-19T10:45:47.226 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h31' does not appear in the crush map 2024-08-19T10:45:47.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.225+0000 7fd05c8e4640 1 -- 172.21.15.190:0/1225725988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd03803be00 msgr2=0x7fd03803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:47.228 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.225+0000 7fd05c8e4640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd03803be00 0x7fd03803e2c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd048009790 tx=0x7fd048008040 comp rx=0 tx=0).stop 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.225+0000 7fd05c8e4640 10 monclient: shutdown 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.225+0000 7fd05c8e4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.225+0000 7fd05c8e4640 1 -- 172.21.15.190:0/1225725988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05809fe50 msgr2=0x7fd0581d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.225+0000 7fd05c8e4640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05809fe50 0x7fd0581d1410 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd05000bdc0 tx=0x7fd050009eb0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.229+0000 7fd05c8e4640 1 -- 172.21.15.190:0/1225725988 shutdown_connections 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.229+0000 7fd05c8e4640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd03803be00 0x7fd03803e2c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.229+0000 7fd05c8e4640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05809fe50 0x7fd0581d1410 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.229+0000 7fd05c8e4640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd058139980 0x7fd05809f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.229+0000 7fd05c8e4640 1 --2- 172.21.15.190:0/1225725988 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd0580a5c70 0x7fd05809f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.229+0000 7fd05c8e4640 1 -- 172.21.15.190:0/1225725988 >> 172.21.15.190:0/1225725988 conn(0x7fd0580abde0 msgr2=0x7fd0580a2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.229+0000 7fd05c8e4640 1 -- 172.21.15.190:0/1225725988 shutdown_connections 2024-08-19T10:45:47.229 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.229+0000 7fd05c8e4640 1 -- 172.21.15.190:0/1225725988 wait complete. 2024-08-19T10:45:47.243 INFO:tasks.workunit.client.0.smithi190.stderr:+ for f in `seq 1 32` 2024-08-19T10:45:47.243 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm h32 2024-08-19T10:45:47.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.357+0000 7f0e705de640 10 monclient: get_monmap_and_config 2024-08-19T10:45:47.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.357+0000 7f0e705de640 10 monclient: build_initial_monmap 2024-08-19T10:45:47.359 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.357+0000 7f0e705de640 10 monclient: monmap: 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:47.362518+0000 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:47.362518+0000 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:47.360 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:47.361 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:47.361 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:47.361 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:47.361 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:47.361 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:47.361 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:47.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.357+0000 7f0e705de640 1 Processor -- start 2024-08-19T10:45:47.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.357+0000 7f0e705de640 1 -- start start 2024-08-19T10:45:47.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.357+0000 7f0e705de640 10 monclient: init 2024-08-19T10:45:47.363 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 10 monclient: _add_conns ranks=[10,7,9,2,4,1,0,8,3,5,6] 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e680a6740 0x7f0e680a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 10 monclient(hunting): picked mon.noname-f con 0x7f0e680a6740 addr v2:172.21.15.190:3300/0 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a5bc0 0x7f0e680a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 10 monclient(hunting): picked mon.noname-i con 0x7f0e680a5bc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0e680a7060 0x7f0e68139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 10 monclient(hunting): picked mon.noname-a con 0x7f0e680a7060 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f0e680a4c20 con 0x7f0e680a7060 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0e680a3530 con 0x7f0e680a6740 2024-08-19T10:45:47.364 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0e680a3740 con 0x7f0e680a5bc0 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e705de640 10 monclient(hunting): authenticate will time out at 1392.932528s 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6e37a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e680a6740 0x7f0e680a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a5bc0 0x7f0e680a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6e37a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e680a6740 0x7f0e680a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:47280/0 (socket says 172.21.15.190:47280) 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6e37a640 1 -- 172.21.15.190:0/2611055687 learned_addr learned my addr 172.21.15.190:0/2611055687 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6e37a640 10 monclient(hunting): get_auth_request con 0x7f0e680a6740 auth_method 0 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6e37a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6e37a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6e37a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 10 monclient(hunting): get_auth_request con 0x7f0e680a5bc0 auth_method 0 2024-08-19T10:45:47.365 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6e37a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6e37a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6e37a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 10 monclient(hunting): handle_auth_done global_id 15811 payload 274 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6eb7b640 1 --2- 172.21.15.190:0/2611055687 >> v2:172.21.15.89:3300/0 conn(0x7f0e680a7060 0x7f0e68139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 1 -- 172.21.15.190:0/2611055687 >> v2:172.21.15.190:3300/0 conn(0x7f0e680a6740 msgr2=0x7f0e680a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 1 --2- 172.21.15.190:0/2611055687 >> v2:172.21.15.190:3300/0 conn(0x7f0e680a6740 0x7f0e680a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.366 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 1 -- 172.21.15.190:0/2611055687 >> v2:172.21.15.89:3300/0 conn(0x7f0e680a7060 msgr2=0x7f0e68139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:47.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 1 --2- 172.21.15.190:0/2611055687 >> v2:172.21.15.89:3300/0 conn(0x7f0e680a7060 0x7f0e68139590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 10 monclient: _finish_hunting 0 2024-08-19T10:45:47.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 1 monclient: found mon.noname-i 2024-08-19T10:45:47.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:47.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:47.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6db79640 1 -- 172.21.15.190:0/2611055687 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e680a74d0 con 0x7f0e680a5bc0 2024-08-19T10:45:47.367 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.361+0000 7f0e6e37a640 1 --2- 172.21.15.190:0/2611055687 >> v2:172.21.15.190:3300/0 conn(0x7f0e680a6740 0x7f0e680a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:47.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6db79640 1 -- 172.21.15.190:0/2611055687 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a5bc0 msgr2=0x7f0e680a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:47.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6db79640 1 --2- 172.21.15.190:0/2611055687 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a5bc0 0x7f0e680a6b20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0e680a3940 tx=0x7f0e5801c300 comp rx=0 tx=0).ready entity=mon.7 client_cookie=f4f542e1d022c8bb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:47.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 1 -- 172.21.15.190:0/2611055687 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e580b0070 con 0x7f0e680a5bc0 2024-08-19T10:45:47.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 10 monclient: dump: 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:47.372 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 10 monclient: _finish_auth 0 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 10 monclient: _check_auth_tickets 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 1 -- 172.21.15.190:0/2611055687 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e580aa810 con 0x7f0e680a5bc0 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 1 -- 172.21.15.190:0/2611055687 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e580aae30 con 0x7f0e680a5bc0 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:47.373 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6d378640 10 monclient: dump: 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:47.374 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 5 monclient: authenticate success, global_id 15811 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 4 set_mon_vals no callback set 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient: shutdown 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 -- 172.21.15.190:0/2611055687 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a5bc0 msgr2=0x7f0e680a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 --2- 172.21.15.190:0/2611055687 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a5bc0 0x7f0e680a6b20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0e680a3940 tx=0x7f0e5801c300 comp rx=0 tx=0).stop 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 -- 172.21.15.190:0/2611055687 shutdown_connections 2024-08-19T10:45:47.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 --2- 172.21.15.190:0/2611055687 >> v2:172.21.15.89:3300/0 conn(0x7f0e680a7060 0x7f0e68139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 --2- 172.21.15.190:0/2611055687 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a5bc0 0x7f0e680a6b20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 --2- 172.21.15.190:0/2611055687 >> v2:172.21.15.190:3300/0 conn(0x7f0e680a6740 0x7f0e680a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 -- 172.21.15.190:0/2611055687 >> 172.21.15.190:0/2611055687 conn(0x7f0e680abd40 msgr2=0x7f0e680ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 -- 172.21.15.190:0/2611055687 shutdown_connections 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 -- 172.21.15.190:0/2611055687 wait complete. 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient: build_initial_monmap 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient: monmap: 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:47.376 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 Processor -- start 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 -- start start 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient: init 2024-08-19T10:45:47.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient: _add_conns ranks=[7,6,2,0,8,3,5,4,1] 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a6740 0x7f0e680a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient(hunting): picked mon.noname-i con 0x7f0e680a6740 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0e680a7060 0x7f0e680a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient(hunting): picked mon.noname-d con 0x7f0e680a7060 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e6809cad0 0x7f0e6809ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient(hunting): picked mon.noname-b con 0x7f0e6809cad0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f0e68143450 con 0x7f0e6809cad0 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f0e681432d0 con 0x7f0e680a7060 2024-08-19T10:45:47.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0e681435d0 con 0x7f0e680a6740 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e705de640 10 monclient(hunting): authenticate will time out at 1392.939841s 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6e37a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a6740 0x7f0e680a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6e37a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a6740 0x7f0e680a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:59362/0 (socket says 172.21.15.190:59362) 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6e37a640 1 -- 172.21.15.190:0/2434383576 learned_addr learned my addr 172.21.15.190:0/2434383576 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6e37a640 10 monclient(hunting): get_auth_request con 0x7f0e680a6740 auth_method 0 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6e37a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6e37a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6e37a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6eb7b640 1 -- 172.21.15.190:0/2434383576 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e6809cad0 msgr2=0x7f0e6809ceb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.365+0000 7f0e6eb7b640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e6809cad0 0x7f0e6809ceb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:47.379 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 10 monclient(hunting): handle_auth_done global_id 15820 payload 824 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 1 -- 172.21.15.190:0/2434383576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0e680a7060 msgr2=0x7f0e680a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0e680a7060 0x7f0e680a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 1 -- 172.21.15.190:0/2434383576 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e6809cad0 msgr2=0x7f0e6809ceb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e6809cad0 0x7f0e6809ceb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 10 monclient: _finish_hunting 0 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 1 monclient: found mon.noname-i 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 1 -- 172.21.15.190:0/2434383576 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6809f770 con 0x7f0e680a6740 2024-08-19T10:45:47.380 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 1 -- 172.21.15.190:0/2434383576 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a6740 msgr2=0x7f0e680a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6e37a640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a6740 0x7f0e680a1f90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400e950 tx=0x7f0e6400ee10 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 1 -- 172.21.15.190:0/2434383576 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e64017070 con 0x7f0e680a6740 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 10 monclient: dump: 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:47.381 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 10 monclient: _finish_auth 0 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 1 -- 172.21.15.190:0/2434383576 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6400cc10 con 0x7f0e680a6740 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:47.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 1 -- 172.21.15.190:0/2434383576 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e6400cdc0 con 0x7f0e680a6740 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 10 monclient: dump: 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:47.383 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e6cb77640 4 set_mon_vals no callback set 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e705de640 5 monclient: authenticate success, global_id 15820 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e705de640 10 monclient: _renew_subs 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e705de640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e705de640 1 -- 172.21.15.190:0/2434383576 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6809fa50 con 0x7f0e680a6740 2024-08-19T10:45:47.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e705de640 10 monclient: _renew_subs 2024-08-19T10:45:47.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e705de640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:47.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e705de640 1 -- 172.21.15.190:0/2434383576 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e680ab300 con 0x7f0e680a6740 2024-08-19T10:45:47.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e705de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:47.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e705de640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:47.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e5effd640 1 -- 172.21.15.190:0/2434383576 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0e6401a070 con 0x7f0e680a6740 2024-08-19T10:45:47.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e705de640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:47.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.369+0000 7f0e705de640 1 -- 172.21.15.190:0/2434383576 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e680a2a10 con 0x7f0e680a6740 2024-08-19T10:45:47.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.373+0000 7f0e5effd640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e4c03be00 0x7f0e4c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:47.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.373+0000 7f0e5effd640 1 -- 172.21.15.190:0/2434383576 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(261..261 src has 150..261) ==== 8408+0+0 (secure 0 0 0) 0x7f0e6400f040 con 0x7f0e680a6740 2024-08-19T10:45:47.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.373+0000 7f0e6db79640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e4c03be00 0x7f0e4c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:47.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.373+0000 7f0e6db79640 10 monclient: get_auth_request con 0x7f0e4c03be00 auth_method 0 2024-08-19T10:45:47.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.373+0000 7f0e6db79640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e4c03be00 0x7f0e4c03e2c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0e580ab000 tx=0x7f0e58004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:47.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.377+0000 7f0e5effd640 1 -- 172.21.15.190:0/2434383576 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0e64021070 con 0x7f0e680a6740 2024-08-19T10:45:47.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.377+0000 7f0e5effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:47.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.377+0000 7f0e5effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:47.541 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.541+0000 7f0e705de640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h32"}] 2024-08-19T10:45:47.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.541+0000 7f0e705de640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "h32"}] 2024-08-19T10:45:47.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.541+0000 7f0e705de640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:47.542 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:47.541+0000 7f0e705de640 1 -- 172.21.15.190:0/2434383576 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h32"} v 0) -- 0x7f0e680a5680 con 0x7f0e680a6740 2024-08-19T10:45:48.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.129+0000 7f0e5effd640 1 -- 172.21.15.190:0/2434383576 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h32"}]=0 removed item id -112 name 'h32' from crush map v262) ==== 121+0+0 (secure 0 0 0) 0x7f0e6404d5f0 con 0x7f0e680a6740 2024-08-19T10:45:48.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.129+0000 7f0e5effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "h32"}] 2024-08-19T10:45:48.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.129+0000 7f0e5effd640 10 monclient: _finish_command 2 = system:0 removed item id -112 name 'h32' from crush map 2024-08-19T10:45:48.234 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.233+0000 7f0e705de640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "h32"}] 2024-08-19T10:45:48.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.233+0000 7f0e705de640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "h32"}] 2024-08-19T10:45:48.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.233+0000 7f0e705de640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:48.235 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.233+0000 7f0e705de640 1 -- 172.21.15.190:0/2434383576 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "h32"} v 0) -- 0x7f0e6809e600 con 0x7f0e680a6740 2024-08-19T10:45:48.236 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e5effd640 1 -- 172.21.15.190:0/2434383576 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "h32"}]=0 device 'h32' does not appear in the crush map v262) ==== 120+0+0 (secure 0 0 0) 0x7f0e6401b940 con 0x7f0e680a6740 2024-08-19T10:45:48.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e5effd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "h32"}] 2024-08-19T10:45:48.237 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e5effd640 10 monclient: _finish_command 3 = system:0 device 'h32' does not appear in the crush map 2024-08-19T10:45:48.237 INFO:tasks.workunit.client.0.smithi190.stderr:device 'h32' does not appear in the crush map 2024-08-19T10:45:48.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 -- 172.21.15.190:0/2434383576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e4c03be00 msgr2=0x7f0e4c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:48.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e4c03be00 0x7f0e4c03e2c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0e580ab000 tx=0x7f0e58004740 comp rx=0 tx=0).stop 2024-08-19T10:45:48.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 10 monclient: shutdown 2024-08-19T10:45:48.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:48.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 -- 172.21.15.190:0/2434383576 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a6740 msgr2=0x7f0e680a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:48.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a6740 0x7f0e680a1f90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400e950 tx=0x7f0e6400ee10 comp rx=0 tx=0).stop 2024-08-19T10:45:48.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 -- 172.21.15.190:0/2434383576 shutdown_connections 2024-08-19T10:45:48.239 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e4c03be00 0x7f0e4c03e2c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:48.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0e680a7060 0x7f0e680a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:48.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e6809cad0 0x7f0e6809ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:48.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 --2- 172.21.15.190:0/2434383576 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e680a6740 0x7f0e680a1f90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:48.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 -- 172.21.15.190:0/2434383576 >> 172.21.15.190:0/2434383576 conn(0x7f0e680abd40 msgr2=0x7f0e68136770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:48.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 -- 172.21.15.190:0/2434383576 shutdown_connections 2024-08-19T10:45:48.240 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.237+0000 7f0e705de640 1 -- 172.21.15.190:0/2434383576 wait complete. 2024-08-19T10:45:48.254 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush rm r1 2024-08-19T10:45:48.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.369+0000 7f828ec0a640 10 monclient: get_monmap_and_config 2024-08-19T10:45:48.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.369+0000 7f828ec0a640 10 monclient: build_initial_monmap 2024-08-19T10:45:48.371 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.369+0000 7f828ec0a640 10 monclient: monmap: 2024-08-19T10:45:48.371 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:48.371 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:48.371 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:48.374360+0000 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:48.374360+0000 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:48.372 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:48.373 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:48.373 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:48.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.369+0000 7f828ec0a640 1 Processor -- start 2024-08-19T10:45:48.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.369+0000 7f828ec0a640 1 -- start start 2024-08-19T10:45:48.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.369+0000 7f828ec0a640 10 monclient: init 2024-08-19T10:45:48.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:48.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 10 monclient: _add_conns ranks=[5,0,4,7,3,10,1,2,6,9,8] 2024-08-19T10:45:48.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82880a56c0 0x7f8288071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:48.375 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 10 monclient(hunting): picked mon.noname-h con 0x7f82880a56c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f82880a5aa0 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f828813e950 0x7f8288140d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 10 monclient(hunting): picked mon.noname-c con 0x7f828813e950 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f828806e930 con 0x7f82880a5aa0 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f828806de60 con 0x7f828813e950 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f828806e0e0 con 0x7f82880a56c0 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828ec0a640 10 monclient(hunting): authenticate will time out at 1393.944361s 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82880a56c0 0x7f8288071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82880a56c0 0x7f8288071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:56830/0 (socket says 172.21.15.190:56830) 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 1 -- 172.21.15.190:0/4039957298 learned_addr learned my addr 172.21.15.190:0/4039957298 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:48.376 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 10 monclient(hunting): get_auth_request con 0x7f82880a56c0 auth_method 0 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 10 monclient(hunting): handle_auth_done global_id 6467 payload 274 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828d1a7640 1 --2- 172.21.15.190:0/4039957298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f828813e950 0x7f8288140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 1 -- 172.21.15.190:0/4039957298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f828813e950 msgr2=0x7f8288140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 1 --2- 172.21.15.190:0/4039957298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f828813e950 0x7f8288140d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 1 -- 172.21.15.190:0/4039957298 >> v1:172.21.15.89:6789/0 conn(0x7f82880a5aa0 legacy=0x7f82880a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 10 monclient: _finish_hunting 0 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 1 monclient: found mon.noname-h 2024-08-19T10:45:48.377 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:48.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:48.378 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.373+0000 7f828c9a6640 1 -- 172.21.15.190:0/4039957298 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828806f180 con 0x7f82880a56c0 2024-08-19T10:45:48.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828c9a6640 1 -- 172.21.15.190:0/4039957298 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82880a56c0 msgr2=0x7f8288071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:48.381 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828c9a6640 1 --2- 172.21.15.190:0/4039957298 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82880a56c0 0x7f8288071fe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098a0 tx=0x7f827c01c440 comp rx=0 tx=0).ready entity=mon.5 client_cookie=e3eff005ab347e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 1 -- 172.21.15.190:0/4039957298 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f827c0b0070 con 0x7f82880a56c0 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 10 monclient: dump: 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:48.382 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 1 -- 172.21.15.190:0/4039957298 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f827c0aa810 con 0x7f82880a56c0 2024-08-19T10:45:48.383 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 5 monclient: authenticate success, global_id 6467 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 4 set_mon_vals no callback set 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 1 -- 172.21.15.190:0/4039957298 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f827c0aae30 con 0x7f82880a56c0 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f82877fe640 10 monclient: dump: 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:48.384 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient: shutdown 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 -- 172.21.15.190:0/4039957298 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82880a56c0 msgr2=0x7f8288071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:48.385 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 --2- 172.21.15.190:0/4039957298 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82880a56c0 0x7f8288071fe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098a0 tx=0x7f827c01c440 comp rx=0 tx=0).stop 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 -- 172.21.15.190:0/4039957298 shutdown_connections 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 --2- 172.21.15.190:0/4039957298 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f828813e950 0x7f8288140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 --2- 172.21.15.190:0/4039957298 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f82880a56c0 0x7f8288071fe0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 -- 172.21.15.190:0/4039957298 >> 172.21.15.190:0/4039957298 conn(0x7f82880aa2b0 msgr2=0x7f82880a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 -- 172.21.15.190:0/4039957298 shutdown_connections 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 -- 172.21.15.190:0/4039957298 wait complete. 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient: build_initial_monmap 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient: monmap: 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:48.386 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:48.387 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 Processor -- start 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 -- start start 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient: init 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient: _add_conns ranks=[0,2,6,8,1,3,5,4,7] 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82880a56c0 0x7f828813cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient(hunting): picked mon.noname-a con 0x7f82880a56c0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82880a5aa0 0x7f82881d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient(hunting): picked mon.noname-b con 0x7f82880a5aa0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f828813e950 0x7f828813a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient(hunting): picked mon.noname-d con 0x7f828813e950 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8288143c20 con 0x7f82880a56c0 2024-08-19T10:45:48.388 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f8288143aa0 con 0x7f82880a5aa0 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8288143da0 con 0x7f828813e950 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828ec0a640 10 monclient(hunting): authenticate will time out at 1393.951572s 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f8287fff640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82880a5aa0 msgr2=0x7f82881d1410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f8287fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82880a5aa0 0x7f82881d1410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828c9a6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82880a56c0 0x7f828813cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.377+0000 7f828d1a7640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f828813e950 0x7f828813a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f828813e950 0x7f828813a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:35688/0 (socket says 172.21.15.190:35688) 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 1 -- 172.21.15.190:0/2835301119 learned_addr learned my addr 172.21.15.190:0/2835301119 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 10 monclient(hunting): get_auth_request con 0x7f828813e950 auth_method 0 2024-08-19T10:45:48.389 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828c9a6640 10 monclient(hunting): get_auth_request con 0x7f82880a56c0 auth_method 0 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828c9a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828c9a6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828c9a6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828c9a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828c9a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828c9a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:48.390 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 10 monclient(hunting): handle_auth_done global_id 44430 payload 824 2024-08-19T10:45:48.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 1 -- 172.21.15.190:0/2835301119 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82880a5aa0 msgr2=0x7f82881d1410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:48.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82880a5aa0 0x7f82881d1410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:48.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 1 -- 172.21.15.190:0/2835301119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82880a56c0 msgr2=0x7f828813cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:48.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82880a56c0 0x7f828813cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:48.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 10 monclient: _finish_hunting 0 2024-08-19T10:45:48.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 1 monclient: found mon.noname-d 2024-08-19T10:45:48.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:48.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:45:48.391 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 1 -- 172.21.15.190:0/2835301119 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8288120c60 con 0x7f828813e950 2024-08-19T10:45:48.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 1 -- 172.21.15.190:0/2835301119 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f828813e950 msgr2=0x7f828813a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:48.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828d1a7640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f828813e950 0x7f828813a640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f828000cc80 tx=0x7f828000a5c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:48.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 1 -- 172.21.15.190:0/2835301119 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f828000ce30 con 0x7f828813e950 2024-08-19T10:45:48.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:48.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:45:48.393 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 10 monclient: dump: 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:48.394 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 10 monclient: _finish_auth 0 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 10 monclient: _check_auth_tickets 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 1 -- 172.21.15.190:0/2835301119 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828000fa70 con 0x7f828813e950 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828ec0a640 5 monclient: authenticate success, global_id 44430 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828ec0a640 10 monclient: _renew_subs 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 1 -- 172.21.15.190:0/2835301119 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f828000fc20 con 0x7f828813e950 2024-08-19T10:45:48.395 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828ec0a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828ec0a640 1 -- 172.21.15.190:0/2835301119 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82880a8240 con 0x7f828813e950 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f8286ffd640 4 set_mon_vals no callback set 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f82857fa640 10 monclient: dump: 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:48.396 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828ec0a640 10 monclient: _renew_subs 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828ec0a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.381+0000 7f828ec0a640 1 -- 172.21.15.190:0/2835301119 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f8288120980 con 0x7f828813e950 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.385+0000 7f828ec0a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:48.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.385+0000 7f828ec0a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:48.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.385+0000 7f828ec0a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:48.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.385+0000 7f828ec0a640 1 -- 172.21.15.190:0/2835301119 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8288071fe0 con 0x7f828813e950 2024-08-19T10:45:48.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.385+0000 7f82857fa640 1 -- 172.21.15.190:0/2835301119 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f828000f600 con 0x7f828813e950 2024-08-19T10:45:48.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.385+0000 7f82857fa640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f826c03be00 0x7f826c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:48.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.385+0000 7f82857fa640 1 -- 172.21.15.190:0/2835301119 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(262..262 src has 150..262) ==== 8195+0+0 (secure 0 0 0) 0x7f828002ae00 con 0x7f828813e950 2024-08-19T10:45:48.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.385+0000 7f828c9a6640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f826c03be00 0x7f826c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:48.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.385+0000 7f828c9a6640 10 monclient: get_auth_request con 0x7f826c03be00 auth_method 0 2024-08-19T10:45:48.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.385+0000 7f828c9a6640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f826c03be00 0x7f826c03e2c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098a0 tx=0x7f827c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:48.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.397+0000 7f82857fa640 1 -- 172.21.15.190:0/2835301119 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f828001d070 con 0x7f828813e950 2024-08-19T10:45:48.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.397+0000 7f82857fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:48.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.397+0000 7f82857fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:48.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.573+0000 7f828ec0a640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "r1"}] 2024-08-19T10:45:48.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.573+0000 7f828ec0a640 10 monclient: _send_command 2 [{"prefix": "osd crush rm", "name": "r1"}] 2024-08-19T10:45:48.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.573+0000 7f828ec0a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:48.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:48.573+0000 7f828ec0a640 1 -- 172.21.15.190:0/2835301119 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "r1"} v 0) -- 0x7f828806fca0 con 0x7f828813e950 2024-08-19T10:45:49.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.141+0000 7f82857fa640 1 -- 172.21.15.190:0/2835301119 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "r1"}]=0 removed item id -16 name 'r1' from crush map v263) ==== 118+0+0 (secure 0 0 0) 0x7f8280014030 con 0x7f828813e950 2024-08-19T10:45:49.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.141+0000 7f82857fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush rm", "name": "r1"}] 2024-08-19T10:45:49.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.141+0000 7f82857fa640 10 monclient: _finish_command 2 = system:0 removed item id -16 name 'r1' from crush map 2024-08-19T10:45:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.249+0000 7f828ec0a640 10 start_mon_command cmd=[{"prefix": "osd crush rm", "name": "r1"}] 2024-08-19T10:45:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.249+0000 7f828ec0a640 10 monclient: _send_command 3 [{"prefix": "osd crush rm", "name": "r1"}] 2024-08-19T10:45:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.249+0000 7f828ec0a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:45:49.251 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.249+0000 7f828ec0a640 1 -- 172.21.15.190:0/2835301119 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd crush rm", "name": "r1"} v 0) -- 0x7f8288136820 con 0x7f828813e950 2024-08-19T10:45:49.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f82857fa640 1 -- 172.21.15.190:0/2835301119 <== mon.6 v2:172.21.15.89:3303/0 8 ==== mon_command_ack([{"prefix": "osd crush rm", "name": "r1"}]=0 device 'r1' does not appear in the crush map v263) ==== 118+0+0 (secure 0 0 0) 0x7f82800152b0 con 0x7f828813e950 2024-08-19T10:45:49.253 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f82857fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush rm", "name": "r1"}] 2024-08-19T10:45:49.254 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f82857fa640 10 monclient: _finish_command 3 = system:0 device 'r1' does not appear in the crush map 2024-08-19T10:45:49.254 INFO:tasks.workunit.client.0.smithi190.stderr:device 'r1' does not appear in the crush map 2024-08-19T10:45:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 -- 172.21.15.190:0/2835301119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f826c03be00 msgr2=0x7f826c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f826c03be00 0x7f826c03e2c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f827c0098a0 tx=0x7f827c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:45:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 10 monclient: shutdown 2024-08-19T10:45:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 -- 172.21.15.190:0/2835301119 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f828813e950 msgr2=0x7f828813a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:49.257 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f828813e950 0x7f828813a640 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f828000cc80 tx=0x7f828000a5c0 comp rx=0 tx=0).stop 2024-08-19T10:45:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 -- 172.21.15.190:0/2835301119 shutdown_connections 2024-08-19T10:45:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f826c03be00 0x7f826c03e2c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f828813e950 0x7f828813a640 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f82880a5aa0 0x7f82881d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 --2- 172.21.15.190:0/2835301119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82880a56c0 0x7f828813cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 -- 172.21.15.190:0/2835301119 >> 172.21.15.190:0/2835301119 conn(0x7f82880aa2b0 msgr2=0x7f82880a8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:49.258 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 -- 172.21.15.190:0/2835301119 shutdown_connections 2024-08-19T10:45:49.259 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.253+0000 7f828ec0a640 1 -- 172.21.15.190:0/2835301119 wait complete. 2024-08-19T10:45:49.272 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush weight-set rm-compat 2024-08-19T10:45:49.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.389+0000 7fa570266640 10 monclient: get_monmap_and_config 2024-08-19T10:45:49.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.389+0000 7fa570266640 10 monclient: build_initial_monmap 2024-08-19T10:45:49.392 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.389+0000 7fa570266640 10 monclient: monmap: 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:49.395427+0000 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:49.395427+0000 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:49.393 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:49.394 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:49.394 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:49.394 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:49.394 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:49.394 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:49.394 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:49.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 1 Processor -- start 2024-08-19T10:45:49.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 1 -- start start 2024-08-19T10:45:49.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 10 monclient: init 2024-08-19T10:45:49.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:49.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 10 monclient: _add_conns ranks=[3,0,7,8,9,5,4,2,6,1,10] 2024-08-19T10:45:49.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa568069410 0x7fa568074b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:49.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 10 monclient(hunting): picked mon.noname-g con 0x7fa568069410 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:49.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa5680a3c80 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:49.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5680a47d0 0x7fa56813ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:49.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 10 monclient(hunting): picked mon.noname-i con 0x7fa5680a47d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:49.397 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5680a20b0 con 0x7fa5680a3c80 2024-08-19T10:45:49.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:49.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa5680697f0 con 0x7fa568069410 2024-08-19T10:45:49.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:49.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa56809d500 con 0x7fa5680a47d0 2024-08-19T10:45:49.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:49.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa570266640 10 monclient(hunting): authenticate will time out at 1394.965572s 2024-08-19T10:45:49.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e803640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5680a47d0 0x7fa56813ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:49.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa568069410 0x7fa568074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:49.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa568069410 0x7fa568074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47278/0 (socket says 172.21.15.190:47278) 2024-08-19T10:45:49.398 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e803640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5680a47d0 0x7fa56813ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:59372/0 (socket says 172.21.15.190:59372) 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 1 -- 172.21.15.190:0/890429947 learned_addr learned my addr 172.21.15.190:0/890429947 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 10 monclient(hunting): get_auth_request con 0x7fa568069410 auth_method 0 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e803640 10 monclient(hunting): get_auth_request con 0x7fa5680a47d0 auth_method 0 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e803640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e803640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e803640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e803640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:49.399 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e803640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:49.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e803640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:49.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 10 monclient(hunting): handle_auth_done global_id 34968 payload 274 2024-08-19T10:45:49.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 1 -- 172.21.15.190:0/890429947 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5680a47d0 msgr2=0x7fa56813ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:49.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 1 --2- 172.21.15.190:0/890429947 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5680a47d0 0x7fa56813ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:49.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 1 -- 172.21.15.190:0/890429947 >> v1:172.21.15.89:6789/0 conn(0x7fa5680a3c80 legacy=0x7fa5680a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:49.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 10 monclient: _finish_hunting 0 2024-08-19T10:45:49.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 1 monclient: found mon.noname-g 2024-08-19T10:45:49.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:49.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:49.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e002640 1 -- 172.21.15.190:0/890429947 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56809d7e0 con 0x7fa568069410 2024-08-19T10:45:49.400 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.393+0000 7fa56e803640 1 --2- 172.21.15.190:0/890429947 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5680a47d0 0x7fa56813ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:49.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56e002640 1 -- 172.21.15.190:0/890429947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa568069410 msgr2=0x7fa568074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:49.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56e002640 1 --2- 172.21.15.190:0/890429947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa568069410 0x7fa568074b90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa5580098a0 tx=0x7fa55801c410 comp rx=0 tx=0).ready entity=mon.3 client_cookie=f297b399c0c5837e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:49.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 1 -- 172.21.15.190:0/890429947 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5580b0070 con 0x7fa568069410 2024-08-19T10:45:49.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:49.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:49.406 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 10 monclient: dump: 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:49.407 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 10 monclient: _finish_auth 0 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 10 monclient: _check_auth_tickets 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 1 -- 172.21.15.190:0/890429947 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5580044a0 con 0x7fa568069410 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 1 -- 172.21.15.190:0/890429947 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa558003a90 con 0x7fa568069410 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 5 monclient: authenticate success, global_id 34968 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:49.408 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 4 set_mon_vals no callback set 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa56d000640 10 monclient: dump: 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:49.409 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:49.410 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:49.410 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:49.410 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:49.410 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:49.410 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:49.410 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:49.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 10 monclient: shutdown 2024-08-19T10:45:49.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:49.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 1 -- 172.21.15.190:0/890429947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa568069410 msgr2=0x7fa568074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:49.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 1 --2- 172.21.15.190:0/890429947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa568069410 0x7fa568074b90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa5580098a0 tx=0x7fa55801c410 comp rx=0 tx=0).stop 2024-08-19T10:45:49.410 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 1 -- 172.21.15.190:0/890429947 shutdown_connections 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 1 --2- 172.21.15.190:0/890429947 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa5680a47d0 0x7fa56813ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 1 --2- 172.21.15.190:0/890429947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa568069410 0x7fa568074b90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 1 -- 172.21.15.190:0/890429947 >> 172.21.15.190:0/890429947 conn(0x7fa5680a4d70 msgr2=0x7fa5680a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 1 -- 172.21.15.190:0/890429947 shutdown_connections 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 1 -- 172.21.15.190:0/890429947 wait complete. 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 10 monclient: build_initial_monmap 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.397+0000 7fa570266640 10 monclient: monmap: 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:49.411 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 1 Processor -- start 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 1 -- start start 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient: init 2024-08-19T10:45:49.412 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient: _add_conns ranks=[5,2,1,4,8,0,7,3,6] 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa568069410 0x7fa5681d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient(hunting): picked mon.noname-h con 0x7fa568069410 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5680a3c80 0x7fa568139d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient(hunting): picked mon.noname-b con 0x7fa5680a3c80 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680a47d0 0x7fa56813a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient(hunting): picked mon.noname-f con 0x7fa5680a47d0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa568141970 con 0x7fa5680a47d0 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa5681417f0 con 0x7fa5680a3c80 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa568141af0 con 0x7fa568069410 2024-08-19T10:45:49.413 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient(hunting): authenticate will time out at 1394.972710s 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa568069410 0x7fa5681d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa568069410 0x7fa5681d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:56844/0 (socket says 172.21.15.190:56844) 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 1 -- 172.21.15.190:0/2661668629 learned_addr learned my addr 172.21.15.190:0/2661668629 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 10 monclient(hunting): get_auth_request con 0x7fa568069410 auth_method 0 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e803640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680a47d0 0x7fa56813a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56d801640 1 -- 172.21.15.190:0/2661668629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5680a3c80 msgr2=0x7fa568139d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e803640 10 monclient(hunting): get_auth_request con 0x7fa5680a47d0 auth_method 0 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e803640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:49.414 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e803640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:49.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e803640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:49.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56d801640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5680a3c80 0x7fa568139d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:49.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:49.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:49.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:49.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e803640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:49.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e803640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:49.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e803640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:49.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 10 monclient(hunting): handle_auth_done global_id 6476 payload 824 2024-08-19T10:45:49.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 1 -- 172.21.15.190:0/2661668629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5680a3c80 msgr2=0x7fa568139d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:49.415 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5680a3c80 0x7fa568139d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 1 -- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680a47d0 msgr2=0x7fa56813a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680a47d0 0x7fa56813a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 10 monclient: _finish_hunting 0 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 1 monclient: found mon.noname-h 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 1 -- 172.21.15.190:0/2661668629 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa568124c00 con 0x7fa568069410 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e803640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680a47d0 0x7fa56813a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 1 -- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa568069410 msgr2=0x7fa5681d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa56e002640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa568069410 0x7fa5681d0c90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa5580ae040 tx=0x7fa5580191b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 1 -- 172.21.15.190:0/2661668629 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5580b0050 con 0x7fa568069410 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:49.416 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 10 monclient: dump: 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:49.417 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 5 monclient: authenticate success, global_id 6476 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 1 -- 172.21.15.190:0/2661668629 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5580ba070 con 0x7fa568069410 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient: _renew_subs 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 1 -- 172.21.15.190:0/2661668629 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5680a10d0 con 0x7fa568069410 2024-08-19T10:45:49.418 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 1 -- 172.21.15.190:0/2661668629 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa5580b5d20 con 0x7fa568069410 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55e7fc640 10 monclient: dump: 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:49.419 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient: _renew_subs 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa55ffff640 4 set_mon_vals no callback set 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.401+0000 7fa570266640 1 -- 172.21.15.190:0/2661668629 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fa568124eb0 con 0x7fa568069410 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.405+0000 7fa570266640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.405+0000 7fa570266640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:49.420 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.405+0000 7fa570266640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:49.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.405+0000 7fa570266640 1 -- 172.21.15.190:0/2661668629 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa568074b90 con 0x7fa568069410 2024-08-19T10:45:49.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.405+0000 7fa55e7fc640 1 -- 172.21.15.190:0/2661668629 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa558003be0 con 0x7fa568069410 2024-08-19T10:45:49.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.405+0000 7fa55e7fc640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa54c03bdf0 0x7fa54c03e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:49.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.405+0000 7fa55e7fc640 1 -- 172.21.15.190:0/2661668629 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(263..263 src has 150..263) ==== 7985+0+0 (secure 0 0 0) 0x7fa5580e95e0 con 0x7fa568069410 2024-08-19T10:45:49.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.405+0000 7fa56d801640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa54c03bdf0 0x7fa54c03e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:49.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.405+0000 7fa56d801640 10 monclient: get_auth_request con 0x7fa54c03bdf0 auth_method 0 2024-08-19T10:45:49.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.405+0000 7fa56d801640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa54c03bdf0 0x7fa54c03e2b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa564007fd0 tx=0x7fa564007da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:49.421 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.409+0000 7fa55e7fc640 1 -- 172.21.15.190:0/2661668629 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fa558016150 con 0x7fa568069410 2024-08-19T10:45:49.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.409+0000 7fa55e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:49.422 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.409+0000 7fa55e7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:49.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.585+0000 7fa570266640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:45:49.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.585+0000 7fa570266640 10 monclient: _send_command 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:45:49.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.585+0000 7fa570266640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:49.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:49.585+0000 7fa570266640 1 -- 172.21.15.190:0/2661668629 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7fa5680a5370 con 0x7fa568069410 2024-08-19T10:45:50.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:50.149+0000 7fa55e7fc640 1 -- 172.21.15.190:0/2661668629 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v264) ==== 78+0+0 (secure 0 0 0) 0x7fa5580becc0 con 0x7fa568069410 2024-08-19T10:45:50.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:50.149+0000 7fa55e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:45:50.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:50.149+0000 7fa55e7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:50.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:50.261+0000 7fa570266640 10 start_mon_command cmd=[{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:45:50.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:50.261+0000 7fa570266640 10 monclient: _send_command 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:45:50.262 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:50.261+0000 7fa570266640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:50.263 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:50.261+0000 7fa570266640 1 -- 172.21.15.190:0/2661668629 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd crush weight-set rm-compat"} v 0) -- 0x7fa568126050 con 0x7fa568069410 2024-08-19T10:45:52.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.209+0000 7fa55e7fc640 1 -- 172.21.15.190:0/2661668629 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd crush weight-set rm-compat"}]=0 v266) ==== 78+0+0 (secure 0 0 0) 0x7fa5580bda50 con 0x7fa568069410 2024-08-19T10:45:52.211 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.209+0000 7fa55e7fc640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd crush weight-set rm-compat"}] 2024-08-19T10:45:52.212 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.209+0000 7fa55e7fc640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:45:52.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 -- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa54c03bdf0 msgr2=0x7fa54c03e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:52.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa54c03bdf0 0x7fa54c03e2b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa564007fd0 tx=0x7fa564007da0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 10 monclient: shutdown 2024-08-19T10:45:52.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:52.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 -- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa568069410 msgr2=0x7fa5681d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:52.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa568069410 0x7fa5681d0c90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa5580ae040 tx=0x7fa5580191b0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 -- 172.21.15.190:0/2661668629 shutdown_connections 2024-08-19T10:45:52.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa54c03bdf0 0x7fa54c03e2b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680a47d0 0x7fa56813a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.216 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa5680a3c80 0x7fa568139d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 --2- 172.21.15.190:0/2661668629 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa568069410 0x7fa5681d0c90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 -- 172.21.15.190:0/2661668629 >> 172.21.15.190:0/2661668629 conn(0x7fa5680a4d70 msgr2=0x7fa568070b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:52.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 -- 172.21.15.190:0/2661668629 shutdown_connections 2024-08-19T10:45:52.217 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.213+0000 7fa570266640 1 -- 172.21.15.190:0/2661668629 wait complete. 2024-08-19T10:45:52.248 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo OK 2024-08-19T10:45:52.249 INFO:tasks.workunit.client.0.smithi190.stdout:OK 2024-08-19T10:45:52.250 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-19T10:45:52.250 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-19T10:45:52.307 INFO:tasks.workunit:Running workunits matching mon/osd.sh on client.0... 2024-08-19T10:45:52.309 INFO:tasks.workunit:Running workunit mon/osd.sh... 2024-08-19T10:45:52.309 DEBUG:teuthology.orchestra.run.smithi190:workunit test mon/osd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=964adda23da9f7b057cb34fdc0b82aeb7ed65324 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/osd.sh 2024-08-19T10:45:52.359 INFO:tasks.workunit.client.0.smithi190.stderr:+ set -e 2024-08-19T10:45:52.360 INFO:tasks.workunit.client.0.smithi190.stderr:+ uuidgen 2024-08-19T10:45:52.360 INFO:tasks.workunit.client.0.smithi190.stderr:+ ua=81c92c7b-73dd-42d5-ae6d-11d899af5fe3 2024-08-19T10:45:52.361 INFO:tasks.workunit.client.0.smithi190.stderr:+ uuidgen 2024-08-19T10:45:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:+ ub=f0a75527-a0ce-4e7b-85de-2da6aea4d069 2024-08-19T10:45:52.362 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd create 81c92c7b-73dd-42d5-ae6d-11d899af5fe3 2024-08-19T10:45:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient: get_monmap_and_config 2024-08-19T10:45:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient: build_initial_monmap 2024-08-19T10:45:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient: monmap: 2024-08-19T10:45:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:52.482 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:52.485224+0000 2024-08-19T10:45:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:52.485224+0000 2024-08-19T10:45:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:52.483 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:52.484 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:52.484 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 1 Processor -- start 2024-08-19T10:45:52.488 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 1 -- start start 2024-08-19T10:45:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient: init 2024-08-19T10:45:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient: _add_conns ranks=[1,6,8,2,4,10,0,3,9,5,7] 2024-08-19T10:45:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa8780a1700 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa87809ca60 0x7fa87813c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:52.489 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient(hunting): picked mon.noname-d con 0x7fa87809ca60 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8780a3c70 0x7fa87813edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient(hunting): picked mon.noname-e con 0x7fa8780a3c70 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87809da20 con 0x7fa8780a1700 2024-08-19T10:45:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa87809d460 con 0x7fa87809ca60 2024-08-19T10:45:52.490 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa87809ef40 con 0x7fa8780a3c70 2024-08-19T10:45:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa87d28b640 10 monclient(hunting): authenticate will time out at 1398.055329s 2024-08-19T10:45:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa876ffd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa8780a1700 0x7fa8780a1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:49020/0 (socket says 172.21.15.190:49020) 2024-08-19T10:45:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa876ffd640 1 -- 172.21.15.190:0/1831635128 learned_addr learned my addr 172.21.15.190:0/1831635128 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:45:52.491 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8767fc640 1 --2- 172.21.15.190:0/1831635128 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa87809ca60 0x7fa87813c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8777fe640 1 --2- 172.21.15.190:0/1831635128 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8780a3c70 0x7fa87813edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8767fc640 10 monclient(hunting): get_auth_request con 0x7fa87809ca60 auth_method 0 2024-08-19T10:45:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8767fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8767fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8777fe640 10 monclient(hunting): get_auth_request con 0x7fa8780a3c70 auth_method 0 2024-08-19T10:45:52.492 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:52.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8777fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:52.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8777fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:52.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8767fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:52.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8767fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:52.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.481+0000 7fa8767fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:52.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:52.493 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:52.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:52.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8767fc640 10 monclient(hunting): handle_auth_done global_id 44439 payload 274 2024-08-19T10:45:52.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8767fc640 1 -- 172.21.15.190:0/1831635128 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8780a3c70 msgr2=0x7fa87813edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:52.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8767fc640 1 --2- 172.21.15.190:0/1831635128 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8780a3c70 0x7fa87813edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8767fc640 1 -- 172.21.15.190:0/1831635128 >> v1:172.21.15.190:6789/0 conn(0x7fa8780a1700 legacy=0x7fa8780a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:52.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8767fc640 10 monclient: _finish_hunting 0 2024-08-19T10:45:52.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8767fc640 1 monclient: found mon.noname-d 2024-08-19T10:45:52.494 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:52.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8767fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:45:52.495 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.485+0000 7fa8767fc640 1 -- 172.21.15.190:0/1831635128 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8780a5560 con 0x7fa87809ca60 2024-08-19T10:45:52.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa8767fc640 1 -- 172.21.15.190:0/1831635128 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa87809ca60 msgr2=0x7fa87813c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:52.502 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa8767fc640 1 --2- 172.21.15.190:0/1831635128 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa87809ca60 0x7fa87813c870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0097f0 tx=0x7fa86c01c3a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=dac2dfa9f4cc1606 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 1 -- 172.21.15.190:0/1831635128 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa86c0b0070 con 0x7fa87809ca60 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 10 monclient: dump: 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:52.503 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 1 -- 172.21.15.190:0/1831635128 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86c0a9980 con 0x7fa87809ca60 2024-08-19T10:45:52.504 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 1 -- 172.21.15.190:0/1831635128 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa86c0aa3a0 con 0x7fa87809ca60 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 5 monclient: authenticate success, global_id 44439 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 4 set_mon_vals no callback set 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa875ffb640 10 monclient: dump: 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:52.505 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:52.506 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:52.506 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:52.506 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:52.506 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:52.506 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:52.506 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:52.506 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:52.506 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:52.506 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:52.506 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:52.506 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient: shutdown 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 -- 172.21.15.190:0/1831635128 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa87809ca60 msgr2=0x7fa87813c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 --2- 172.21.15.190:0/1831635128 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa87809ca60 0x7fa87813c870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0097f0 tx=0x7fa86c01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 -- 172.21.15.190:0/1831635128 shutdown_connections 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 --2- 172.21.15.190:0/1831635128 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa8780a3c70 0x7fa87813edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 --2- 172.21.15.190:0/1831635128 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa87809ca60 0x7fa87813c870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 -- 172.21.15.190:0/1831635128 >> 172.21.15.190:0/1831635128 conn(0x7fa87806ced0 msgr2=0x7fa8780a4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 -- 172.21.15.190:0/1831635128 shutdown_connections 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 -- 172.21.15.190:0/1831635128 wait complete. 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient: build_initial_monmap 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient: monmap: 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:52.507 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:52.508 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 Processor -- start 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 -- start start 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient: init 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient: _add_conns ranks=[8,2,4,3,5,6,0,7,1] 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa87809ca60 0x7fa8781d1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient(hunting): picked mon.noname-e con 0x7fa87809ca60 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8780a1700 0x7fa87813e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient(hunting): picked mon.noname-b con 0x7fa8780a1700 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:52.509 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8780a3c70 0x7fa87813eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient(hunting): picked mon.noname-c con 0x7fa8780a3c70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa878141ca0 con 0x7fa8780a1700 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa878141b20 con 0x7fa8780a3c70 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa878141e20 con 0x7fa87809ca60 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa87d28b640 10 monclient(hunting): authenticate will time out at 1398.063824s 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa8767fc640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8780a1700 msgr2=0x7fa87813e610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.489+0000 7fa8767fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8780a1700 0x7fa87813e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:52.510 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa8777fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8780a3c70 0x7fa87813eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa87809ca60 0x7fa8781d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa8777fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8780a3c70 0x7fa87813eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.190:56112/0 (socket says 172.21.15.190:56112) 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa8777fe640 1 -- 172.21.15.190:0/3226699398 learned_addr learned my addr 172.21.15.190:0/3226699398 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa8777fe640 10 monclient(hunting): get_auth_request con 0x7fa8780a3c70 auth_method 0 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa8777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa8777fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa8777fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 10 monclient(hunting): get_auth_request con 0x7fa87809ca60 auth_method 0 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:52.511 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa8777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa8777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa8777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 10 monclient(hunting): handle_auth_done global_id 6371 payload 824 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 1 -- 172.21.15.190:0/3226699398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8780a3c70 msgr2=0x7fa87813eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8780a3c70 0x7fa87813eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 1 -- 172.21.15.190:0/3226699398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8780a1700 msgr2=0x7fa87813e610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8780a1700 0x7fa87813e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:45:52.512 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 1 monclient: found mon.noname-e 2024-08-19T10:45:52.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:52.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:52.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 1 -- 172.21.15.190:0/3226699398 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8780739a0 con 0x7fa87809ca60 2024-08-19T10:45:52.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 1 -- 172.21.15.190:0/3226699398 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa87809ca60 msgr2=0x7fa8781d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:52.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa876ffd640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa87809ca60 0x7fa8781d1040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa86800c410 tx=0x7fa86800c8d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:52.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 1 -- 172.21.15.190:0/3226699398 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa868017070 con 0x7fa87809ca60 2024-08-19T10:45:52.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:52.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:52.513 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 10 monclient: dump: 2024-08-19T10:45:52.513 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:52.513 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:52.514 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 1 -- 172.21.15.190:0/3226699398 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa868005d00 con 0x7fa87809ca60 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 1 -- 172.21.15.190:0/3226699398 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa868007540 con 0x7fa87809ca60 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa8757fa640 4 set_mon_vals no callback set 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa85f7fe640 10 monclient: dump: 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:52.515 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:52.516 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa87d28b640 5 monclient: authenticate success, global_id 6371 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa87d28b640 10 monclient: _renew_subs 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa87d28b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa87d28b640 1 -- 172.21.15.190:0/3226699398 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87812ace0 con 0x7fa87809ca60 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa87d28b640 10 monclient: _renew_subs 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa87d28b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.493+0000 7fa87d28b640 1 -- 172.21.15.190:0/3226699398 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8780a0f00 con 0x7fa87809ca60 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.497+0000 7fa87d28b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.497+0000 7fa87d28b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.497+0000 7fa87d28b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.497+0000 7fa87d28b640 1 -- 172.21.15.190:0/3226699398 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8780a1ae0 con 0x7fa87809ca60 2024-08-19T10:45:52.517 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.497+0000 7fa85f7fe640 1 -- 172.21.15.190:0/3226699398 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa868007070 con 0x7fa87809ca60 2024-08-19T10:45:52.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.497+0000 7fa85f7fe640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa86403be00 0x7fa86403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:52.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.497+0000 7fa85f7fe640 1 -- 172.21.15.190:0/3226699398 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(266..266 src has 150..266) ==== 7657+0+0 (secure 0 0 0) 0x7fa86804ece0 con 0x7fa87809ca60 2024-08-19T10:45:52.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.497+0000 7fa8767fc640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa86403be00 0x7fa86403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:52.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.497+0000 7fa8767fc640 10 monclient: get_auth_request con 0x7fa86403be00 auth_method 0 2024-08-19T10:45:52.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.501+0000 7fa8767fc640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa86403be00 0x7fa86403e2c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa87806d960 tx=0x7fa86c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:52.518 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.509+0000 7fa85f7fe640 1 -- 172.21.15.190:0/3226699398 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fa8780739a0 con 0x7fa87809ca60 2024-08-19T10:45:52.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.509+0000 7fa85f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:52.519 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.509+0000 7fa85f7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:52.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.689+0000 7fa87d28b640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:52.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.689+0000 7fa87d28b640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:52.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.689+0000 7fa87d28b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:52.691 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:52.689+0000 7fa87d28b640 1 -- 172.21.15.190:0/3226699398 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"} v 0) -- 0x7fa87809e5e0 con 0x7fa87809ca60 2024-08-19T10:45:53.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.437+0000 7fa85f7fe640 1 -- 172.21.15.190:0/3226699398 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}]=0 v267) ==== 106+0+3 (secure 0 0 0) 0x7fa8680200b0 con 0x7fa87809ca60 2024-08-19T10:45:53.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.437+0000 7fa85f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:53.437 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.437+0000 7fa85f7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:53.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.553+0000 7fa87d28b640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:53.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.553+0000 7fa87d28b640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:53.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.553+0000 7fa87d28b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:45:53.556 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.553+0000 7fa87d28b640 1 -- 172.21.15.190:0/3226699398 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"} v 0) -- 0x7fa8780008e0 con 0x7fa87809ca60 2024-08-19T10:45:53.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa85f7fe640 1 -- 172.21.15.190:0/3226699398 <== mon.8 v2:172.21.15.89:3304/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}]=0 v267) ==== 106+0+3 (secure 0 0 0) 0x7fa868010db0 con 0x7fa87809ca60 2024-08-19T10:45:53.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa85f7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:53.557 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa85f7fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:45:53.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 1 -- 172.21.15.190:0/3226699398 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa86403be00 msgr2=0x7fa86403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:53.560 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa86403be00 0x7fa86403e2c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa87806d960 tx=0x7fa86c008040 comp rx=0 tx=0).stop 2024-08-19T10:45:53.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 10 monclient: shutdown 2024-08-19T10:45:53.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:53.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 1 -- 172.21.15.190:0/3226699398 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa87809ca60 msgr2=0x7fa8781d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:53.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa87809ca60 0x7fa8781d1040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa86800c410 tx=0x7fa86800c8d0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 1 -- 172.21.15.190:0/3226699398 shutdown_connections 2024-08-19T10:45:53.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa8780a3c70 0x7fa87813eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa8780a1700 0x7fa87813e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa86403be00 0x7fa86403e2c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 1 --2- 172.21.15.190:0/3226699398 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa87809ca60 0x7fa8781d1040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.557+0000 7fa87d28b640 1 -- 172.21.15.190:0/3226699398 >> 172.21.15.190:0/3226699398 conn(0x7fa87806ced0 msgr2=0x7fa878072020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:53.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.561+0000 7fa87d28b640 1 -- 172.21.15.190:0/3226699398 shutdown_connections 2024-08-19T10:45:53.562 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.561+0000 7fa87d28b640 1 -- 172.21.15.190:0/3226699398 wait complete. 2024-08-19T10:45:53.576 INFO:tasks.workunit.client.0.smithi190.stderr:+ na=12 2024-08-19T10:45:53.576 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd create 81c92c7b-73dd-42d5-ae6d-11d899af5fe3 2024-08-19T10:45:53.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.689+0000 7f1490aa2640 10 monclient: get_monmap_and_config 2024-08-19T10:45:53.692 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.689+0000 7f1490aa2640 10 monclient: build_initial_monmap 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient: monmap: 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:53.695667+0000 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:53.695667+0000 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:53.693 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:53.694 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:53.694 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:53.694 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:53.694 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:53.694 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:53.694 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:53.694 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:53.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 1 Processor -- start 2024-08-19T10:45:53.696 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 1 -- start start 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient: init 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient: _add_conns ranks=[4,3,5,9,1,10,0,7,8,6,2] 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f148c072550 0x7f148c06dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient(hunting): picked mon.noname-c con 0x7f148c072550 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient(hunting): picked mon.noname-g con 0x7f148c06e1c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f148c1395e0 0x7f148c1399e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient(hunting): picked mon.noname-h con 0x7f148c1395e0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f148c072930 con 0x7f148c06e1c0 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:53.697 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f148c06d9e0 con 0x7f148c072550 2024-08-19T10:45:53.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:53.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f148c06f900 con 0x7f148c1395e0 2024-08-19T10:45:53.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:53.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1490aa2640 10 monclient(hunting): authenticate will time out at 1399.265739s 2024-08-19T10:45:53.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:53.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148ad9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f148c1395e0 0x7f148c1399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:53.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c06e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47290/0 (socket says 172.21.15.190:47290) 2024-08-19T10:45:53.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148ad9d640 10 monclient(hunting): get_auth_request con 0x7f148c1395e0 auth_method 0 2024-08-19T10:45:53.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 1 -- 172.21.15.190:0/3367203755 learned_addr learned my addr 172.21.15.190:0/3367203755 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:53.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:53.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:53.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:53.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 10 monclient(hunting): get_auth_request con 0x7f148c06e1c0 auth_method 0 2024-08-19T10:45:53.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:53.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:53.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:53.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:53.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:53.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:53.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:53.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:53.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:53.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148a59c640 1 --2- 172.21.15.190:0/3367203755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f148c072550 0x7f148c06dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:53.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 10 monclient(hunting): handle_auth_done global_id 34977 payload 274 2024-08-19T10:45:53.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 1 -- 172.21.15.190:0/3367203755 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f148c1395e0 msgr2=0x7f148c1399e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:53.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 1 --2- 172.21.15.190:0/3367203755 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f148c1395e0 0x7f148c1399e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 1 -- 172.21.15.190:0/3367203755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f148c072550 msgr2=0x7f148c06dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:53.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 1 --2- 172.21.15.190:0/3367203755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f148c072550 0x7f148c06dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:45:53.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 1 monclient: found mon.noname-g 2024-08-19T10:45:53.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:53.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:53.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f1489d9b640 1 -- 172.21.15.190:0/3367203755 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c071d50 con 0x7f148c06e1c0 2024-08-19T10:45:53.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148a59c640 10 monclient: get_auth_request con 0x7f148c072550 auth_method 0 2024-08-19T10:45:53.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148a59c640 1 --2- 172.21.15.190:0/3367203755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f148c072550 0x7f148c06dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:53.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.693+0000 7f148ad9d640 1 --2- 172.21.15.190:0/3367203755 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f148c1395e0 0x7f148c1399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:53.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1489d9b640 1 -- 172.21.15.190:0/3367203755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 msgr2=0x7f148c06e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:53.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1489d9b640 1 --2- 172.21.15.190:0/3367203755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c06e5a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f14800098a0 tx=0x7f148001c4a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c03b1ed3d1e0794e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:53.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 1 -- 172.21.15.190:0/3367203755 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14800b1070 con 0x7f148c06e1c0 2024-08-19T10:45:53.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:53.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:53.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 10 monclient: dump: 2024-08-19T10:45:53.713 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:53.713 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:53.713 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:53.713 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:53.713 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:53.714 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:53.714 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:53.714 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:53.714 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:53.714 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:53.715 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:53.715 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:53.715 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:53.715 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:53.715 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:53.715 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:53.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 10 monclient: _finish_auth 0 2024-08-19T10:45:53.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 10 monclient: _check_auth_tickets 2024-08-19T10:45:53.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:53.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 5 monclient: authenticate success, global_id 34977 2024-08-19T10:45:53.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:53.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 1 -- 172.21.15.190:0/3367203755 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14800a8050 con 0x7f148c06e1c0 2024-08-19T10:45:53.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:53.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 1 -- 172.21.15.190:0/3367203755 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14800ab850 con 0x7f148c06e1c0 2024-08-19T10:45:53.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:53.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:53.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f148959a640 10 monclient: dump: 2024-08-19T10:45:53.717 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:53.718 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:53.718 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:53.718 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:53.718 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:53.718 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:53.718 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:53.719 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:53.719 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:53.719 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:53.719 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:53.719 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:53.720 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:53.720 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:53.720 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:53.720 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:53.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:53.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 4 set_mon_vals no callback set 2024-08-19T10:45:53.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 10 monclient: shutdown 2024-08-19T10:45:53.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:53.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 1 -- 172.21.15.190:0/3367203755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 msgr2=0x7f148c06e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:53.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 1 --2- 172.21.15.190:0/3367203755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c06e5a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f14800098a0 tx=0x7f148001c4a0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 1 -- 172.21.15.190:0/3367203755 shutdown_connections 2024-08-19T10:45:53.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 1 --2- 172.21.15.190:0/3367203755 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f148c1395e0 0x7f148c1399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 1 --2- 172.21.15.190:0/3367203755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c06e5a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 1 --2- 172.21.15.190:0/3367203755 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f148c072550 0x7f148c06dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 1 -- 172.21.15.190:0/3367203755 >> 172.21.15.190:0/3367203755 conn(0x7f148c0a6720 msgr2=0x7f148c0a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:53.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 1 -- 172.21.15.190:0/3367203755 shutdown_connections 2024-08-19T10:45:53.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.697+0000 7f1490aa2640 1 -- 172.21.15.190:0/3367203755 wait complete. 2024-08-19T10:45:53.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient: build_initial_monmap 2024-08-19T10:45:53.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient: monmap: 2024-08-19T10:45:53.723 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:53.723 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:53.724 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:53.724 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:53.724 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:53.724 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:53.724 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:53.724 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:53.725 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:53.725 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:53.725 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:53.725 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:53.725 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:53.725 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:53.726 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:53.726 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:53.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 1 Processor -- start 2024-08-19T10:45:53.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 1 -- start start 2024-08-19T10:45:53.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient: init 2024-08-19T10:45:53.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:53.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient: _add_conns ranks=[3,0,8,2,5,4,1,7,6] 2024-08-19T10:45:53.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c138850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:53.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient(hunting): picked mon.noname-g con 0x7f148c06e1c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:53.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f148c072550 0x7f148c138d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:53.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient(hunting): picked mon.noname-a con 0x7f148c072550 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:53.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f148c1395e0 0x7f148c1206a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:53.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient(hunting): picked mon.noname-e con 0x7f148c1395e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:53.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:53.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f148c0a5a90 con 0x7f148c072550 2024-08-19T10:45:53.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:53.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f148c0a5910 con 0x7f148c06e1c0 2024-08-19T10:45:53.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:53.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f148c143130 con 0x7f148c1395e0 2024-08-19T10:45:53.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:53.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1490aa2640 10 monclient(hunting): authenticate will time out at 1399.273066s 2024-08-19T10:45:53.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c138850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:53.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c138850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47300/0 (socket says 172.21.15.190:47300) 2024-08-19T10:45:53.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 1 -- 172.21.15.190:0/1941032027 learned_addr learned my addr 172.21.15.190:0/1941032027 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:53.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 10 monclient(hunting): get_auth_request con 0x7f148c06e1c0 auth_method 0 2024-08-19T10:45:53.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:53.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:53.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:53.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:53.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:53.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:53.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148ad9d640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f148c1395e0 0x7f148c1206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:53.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148ad9d640 10 monclient(hunting): get_auth_request con 0x7f148c1395e0 auth_method 0 2024-08-19T10:45:53.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:53.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:53.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:53.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1489d9b640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f148c072550 0x7f148c138d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:53.732 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 10 monclient(hunting): handle_auth_done global_id 34986 payload 824 2024-08-19T10:45:53.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 1 -- 172.21.15.190:0/1941032027 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f148c1395e0 msgr2=0x7f148c1206a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:53.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f148c1395e0 0x7f148c1206a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 1 -- 172.21.15.190:0/1941032027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f148c072550 msgr2=0x7f148c138d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:53.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f148c072550 0x7f148c138d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:45:53.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 1 monclient: found mon.noname-g 2024-08-19T10:45:53.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:53.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:53.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 1 -- 172.21.15.190:0/1941032027 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c1242f0 con 0x7f148c06e1c0 2024-08-19T10:45:53.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1489d9b640 10 monclient: get_auth_request con 0x7f148c072550 auth_method 0 2024-08-19T10:45:53.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f1489d9b640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f148c072550 0x7f148c138d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:53.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 1 -- 172.21.15.190:0/1941032027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 msgr2=0x7f148c138850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:53.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.701+0000 7f148a59c640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c138850 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f147c00c7c0 tx=0x7f147c00cc80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:53.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 1 -- 172.21.15.190:0/1941032027 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f147c0113f0 con 0x7f148c06e1c0 2024-08-19T10:45:53.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:53.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:53.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 10 monclient: dump: 2024-08-19T10:45:53.735 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:53.735 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:53.735 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:53.735 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:53.735 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:53.736 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:53.736 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:53.736 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:53.736 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:53.736 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:53.736 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:53.736 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:53.736 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:53.737 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:53.737 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:53.737 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:53.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 10 monclient: _finish_auth 0 2024-08-19T10:45:53.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:53.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:53.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 1 -- 172.21.15.190:0/1941032027 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c0115d0 con 0x7f148c06e1c0 2024-08-19T10:45:53.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1490aa2640 5 monclient: authenticate success, global_id 34986 2024-08-19T10:45:53.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:53.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 1 -- 172.21.15.190:0/1941032027 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f147c011bf0 con 0x7f148c06e1c0 2024-08-19T10:45:53.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1490aa2640 10 monclient: _renew_subs 2024-08-19T10:45:53.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1490aa2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:53.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1490aa2640 1 -- 172.21.15.190:0/1941032027 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148c123a50 con 0x7f148c06e1c0 2024-08-19T10:45:53.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:53.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:53.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 10 monclient: dump: 2024-08-19T10:45:53.738 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:53.739 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1488d99640 4 set_mon_vals no callback set 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1490aa2640 10 monclient: _renew_subs 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1490aa2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1490aa2640 1 -- 172.21.15.190:0/1941032027 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f148c1d18f0 con 0x7f148c06e1c0 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1490aa2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1490aa2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1490aa2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1490aa2640 1 -- 172.21.15.190:0/1941032027 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f148c06dc80 con 0x7f148c06e1c0 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 1 -- 172.21.15.190:0/1941032027 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f147c011780 con 0x7f148c06e1c0 2024-08-19T10:45:53.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f146c03be30 0x7f146c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:53.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f147affd640 1 -- 172.21.15.190:0/1941032027 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(267..267 src has 150..267) ==== 7657+0+0 (secure 0 0 0) 0x7f147c04f7b0 con 0x7f148c06e1c0 2024-08-19T10:45:53.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1489d9b640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f146c03be30 0x7f146c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:53.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.705+0000 7f1489d9b640 10 monclient: get_auth_request con 0x7f146c03be30 auth_method 0 2024-08-19T10:45:53.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.709+0000 7f1489d9b640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f146c03be30 0x7f146c03e2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f14800098a0 tx=0x7f14800a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:53.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.709+0000 7f147affd640 1 -- 172.21.15.190:0/1941032027 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f147c015020 con 0x7f148c06e1c0 2024-08-19T10:45:53.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.709+0000 7f147affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:53.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.709+0000 7f147affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:53.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.881+0000 7f1490aa2640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:53.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.881+0000 7f1490aa2640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:53.884 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.881+0000 7f1490aa2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:53.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.881+0000 7f1490aa2640 1 -- 172.21.15.190:0/1941032027 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"} v 0) -- 0x7f148c0a4e90 con 0x7f148c06e1c0 2024-08-19T10:45:53.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.885+0000 7f147affd640 1 -- 172.21.15.190:0/1941032027 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}]=0 v267) ==== 106+0+3 (secure 0 0 0) 0x7f147c018d60 con 0x7f148c06e1c0 2024-08-19T10:45:53.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.885+0000 7f147affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:53.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.885+0000 7f147affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:53.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.989+0000 7f1490aa2640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:53.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.989+0000 7f1490aa2640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:53.990 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.989+0000 7f1490aa2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:53.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.989+0000 7f1490aa2640 1 -- 172.21.15.190:0/1941032027 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"} v 0) -- 0x7f148c1393c0 con 0x7f148c06e1c0 2024-08-19T10:45:53.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f147affd640 1 -- 172.21.15.190:0/1941032027 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}]=0 v267) ==== 106+0+3 (secure 0 0 0) 0x7f147c007a40 con 0x7f148c06e1c0 2024-08-19T10:45:53.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f147affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:53.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f147affd640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:45:53.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 -- 172.21.15.190:0/1941032027 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f146c03be30 msgr2=0x7f146c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:53.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f146c03be30 0x7f146c03e2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f14800098a0 tx=0x7f14800a7000 comp rx=0 tx=0).stop 2024-08-19T10:45:53.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 10 monclient: shutdown 2024-08-19T10:45:53.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:53.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 -- 172.21.15.190:0/1941032027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 msgr2=0x7f148c138850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:53.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c138850 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f147c00c7c0 tx=0x7f147c00cc80 comp rx=0 tx=0).stop 2024-08-19T10:45:53.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 -- 172.21.15.190:0/1941032027 shutdown_connections 2024-08-19T10:45:53.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f146c03be30 0x7f146c03e2f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f148c1395e0 0x7f148c1206a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f148c072550 0x7f148c138d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 --2- 172.21.15.190:0/1941032027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f148c06e1c0 0x7f148c138850 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:53.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 -- 172.21.15.190:0/1941032027 >> 172.21.15.190:0/1941032027 conn(0x7f148c0a6720 msgr2=0x7f148c0a7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:53.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 -- 172.21.15.190:0/1941032027 shutdown_connections 2024-08-19T10:45:53.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:53.993+0000 7f1490aa2640 1 -- 172.21.15.190:0/1941032027 wait complete. 2024-08-19T10:45:54.008 INFO:tasks.workunit.client.0.smithi190.stderr:+ test 12 -eq 12 2024-08-19T10:45:54.009 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd create f0a75527-a0ce-4e7b-85de-2da6aea4d069 2024-08-19T10:45:54.116 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient: get_monmap_and_config 2024-08-19T10:45:54.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient: build_initial_monmap 2024-08-19T10:45:54.117 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient: monmap: 2024-08-19T10:45:54.117 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:54.117 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:54.117 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:54.119882+0000 2024-08-19T10:45:54.117 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:54.119882+0000 2024-08-19T10:45:54.117 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:54.117 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.117 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:54.117 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:54.117 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:54.118 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:54.118 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:54.118 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:54.118 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:54.118 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:54.118 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:54.118 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:54.118 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:54.118 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 1 Processor -- start 2024-08-19T10:45:54.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 1 -- start start 2024-08-19T10:45:54.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient: init 2024-08-19T10:45:54.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:54.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient: _add_conns ranks=[9,8,1,0,3,2,7,5,6,10,4] 2024-08-19T10:45:54.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4bc8139980 0x7f4bc8139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.122 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient(hunting): picked mon.noname-a con 0x7f4bc8139980 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:54.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4bc813a2a0 0x7f4bc80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient(hunting): picked mon.noname-e con 0x7f4bc813a2a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:45:54.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4bc80a5c70 addr v1:172.21.15.190:6789/0 2024-08-19T10:45:54.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc80a8470 con 0x7f4bc80a5c70 2024-08-19T10:45:54.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:54.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc809ca60 con 0x7f4bc8139980 2024-08-19T10:45:54.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:54.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4bc80a3640 con 0x7f4bc813a2a0 2024-08-19T10:45:54.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:54.123 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bcdc61640 10 monclient(hunting): authenticate will time out at 1399.689909s 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc7fff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f4bc80a5c70 0x7f4bc80a67e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:49034/0 (socket says 172.21.15.190:49034) 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc7fff640 1 -- 172.21.15.190:0/872704746 learned_addr learned my addr 172.21.15.190:0/872704746 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 1 --2- 172.21.15.190:0/872704746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4bc813a2a0 0x7f4bc80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc77fe640 1 --2- 172.21.15.190:0/872704746 >> v2:172.21.15.89:3300/0 conn(0x7f4bc8139980 0x7f4bc8139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 10 monclient(hunting): get_auth_request con 0x7f4bc813a2a0 auth_method 0 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc77fe640 10 monclient(hunting): get_auth_request con 0x7f4bc8139980 auth_method 0 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:54.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 10 monclient(hunting): handle_auth_done global_id 6389 payload 274 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 1 -- 172.21.15.190:0/872704746 >> v2:172.21.15.89:3300/0 conn(0x7f4bc8139980 msgr2=0x7f4bc8139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 1 --2- 172.21.15.190:0/872704746 >> v2:172.21.15.89:3300/0 conn(0x7f4bc8139980 0x7f4bc8139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 1 -- 172.21.15.190:0/872704746 >> v1:172.21.15.190:6789/0 conn(0x7f4bc80a5c70 legacy=0x7f4bc80a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 1 monclient: found mon.noname-e 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:45:54.125 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.117+0000 7f4bc6ffd640 1 -- 172.21.15.190:0/872704746 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc80a3850 con 0x7f4bc813a2a0 2024-08-19T10:45:54.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc6ffd640 1 -- 172.21.15.190:0/872704746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4bc813a2a0 msgr2=0x7f4bc80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:54.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc6ffd640 1 --2- 172.21.15.190:0/872704746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4bc813a2a0 0x7f4bc80a5730 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009d90 tx=0x7f4bb801c830 comp rx=0 tx=0).ready entity=mon.8 client_cookie=66089ee1e7cd0296 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 1 -- 172.21.15.190:0/872704746 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4bb80a6920 con 0x7f4bc813a2a0 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 10 monclient: dump: 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:54.132 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 10 monclient: _finish_auth 0 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 1 -- 172.21.15.190:0/872704746 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb80a6ae0 con 0x7f4bc813a2a0 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:54.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 1 -- 172.21.15.190:0/872704746 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4bb80a6c90 con 0x7f4bc813a2a0 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bcdc61640 5 monclient: authenticate success, global_id 6389 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bcdc61640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bcdc61640 4 set_mon_vals no callback set 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.121+0000 7f4bc67fc640 10 monclient: dump: 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.134 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient: shutdown 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 -- 172.21.15.190:0/872704746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4bc813a2a0 msgr2=0x7f4bc80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 --2- 172.21.15.190:0/872704746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4bc813a2a0 0x7f4bc80a5730 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009d90 tx=0x7f4bb801c830 comp rx=0 tx=0).stop 2024-08-19T10:45:54.135 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 -- 172.21.15.190:0/872704746 shutdown_connections 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 --2- 172.21.15.190:0/872704746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4bc813a2a0 0x7f4bc80a5730 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 --2- 172.21.15.190:0/872704746 >> v2:172.21.15.89:3300/0 conn(0x7f4bc8139980 0x7f4bc8139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 -- 172.21.15.190:0/872704746 >> 172.21.15.190:0/872704746 conn(0x7f4bc80abde0 msgr2=0x7f4bc80ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 -- 172.21.15.190:0/872704746 shutdown_connections 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 -- 172.21.15.190:0/872704746 wait complete. 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient: build_initial_monmap 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient: monmap: 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:54.136 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.137 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:54.137 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:54.137 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:54.137 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:54.137 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:54.137 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:54.137 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:54.137 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:54.137 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:54.137 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 Processor -- start 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 -- start start 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient: init 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient: _add_conns ranks=[3,5,0,8,2,4,1,7,6] 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc80a5c70 0x7f4bc8140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient(hunting): picked mon.noname-g con 0x7f4bc80a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4bc8139980 0x7f4bc8140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient(hunting): picked mon.noname-h con 0x7f4bc8139980 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bc813a690 0x7f4bc813aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient(hunting): picked mon.noname-a con 0x7f4bc813a690 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:54.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc8144280 con 0x7f4bc813a690 2024-08-19T10:45:54.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:54.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc8144100 con 0x7f4bc80a5c70 2024-08-19T10:45:54.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:54.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4bc8144400 con 0x7f4bc8139980 2024-08-19T10:45:54.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:54.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bcdc61640 10 monclient(hunting): authenticate will time out at 1399.698692s 2024-08-19T10:45:54.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc80a5c70 0x7f4bc8140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc80a5c70 0x7f4bc8140890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47306/0 (socket says 172.21.15.190:47306) 2024-08-19T10:45:54.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc6ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4bc8139980 0x7f4bc8140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 1 -- 172.21.15.190:0/3020017878 learned_addr learned my addr 172.21.15.190:0/3020017878 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:54.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 10 monclient(hunting): get_auth_request con 0x7f4bc80a5c70 auth_method 0 2024-08-19T10:45:54.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:54.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:54.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:54.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc6ffd640 10 monclient(hunting): get_auth_request con 0x7f4bc8139980 auth_method 0 2024-08-19T10:45:54.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:54.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:54.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 10 monclient(hunting): handle_auth_done global_id 34995 payload 824 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 1 -- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4bc8139980 msgr2=0x7f4bc8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc7fff640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bc813a690 0x7f4bc813aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4bc8139980 0x7f4bc8140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 1 -- 172.21.15.190:0/3020017878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bc813a690 msgr2=0x7f4bc813aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:54.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bc813a690 0x7f4bc813aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:45:54.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 1 monclient: found mon.noname-g 2024-08-19T10:45:54.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:54.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:54.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc77fe640 1 -- 172.21.15.190:0/3020017878 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc80ac430 con 0x7f4bc80a5c70 2024-08-19T10:45:54.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc7fff640 10 monclient: get_auth_request con 0x7f4bc813a690 auth_method 0 2024-08-19T10:45:54.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.125+0000 7f4bc7fff640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bc813a690 0x7f4bc813aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:54.142 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bc77fe640 1 -- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc80a5c70 msgr2=0x7f4bc8140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bc77fe640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc80a5c70 0x7f4bc8140890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000e930 tx=0x7f4bb000edf0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 1 -- 172.21.15.190:0/3020017878 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4bb0017070 con 0x7f4bc80a5c70 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 10 monclient: dump: 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:54.143 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 10 monclient: _finish_auth 0 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 1 -- 172.21.15.190:0/3020017878 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb000cbb0 con 0x7f4bc80a5c70 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bcdc61640 5 monclient: authenticate success, global_id 34995 2024-08-19T10:45:54.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bcdc61640 10 monclient: _renew_subs 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bcdc61640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bcdc61640 1 -- 172.21.15.190:0/3020017878 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc813d060 con 0x7f4bc80a5c70 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 1 -- 172.21.15.190:0/3020017878 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4bb000cd60 con 0x7f4bc80a5c70 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bc57fa640 4 set_mon_vals no callback set 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 10 monclient: dump: 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:54.145 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.146 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:54.146 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:54.146 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:54.146 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:54.146 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:54.146 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:54.146 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:54.146 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:54.146 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:54.146 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bcdc61640 10 monclient: _renew_subs 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bcdc61640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bcdc61640 1 -- 172.21.15.190:0/3020017878 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc813bb40 con 0x7f4bc80a5c70 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bcdc61640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bcdc61640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bcdc61640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bcdc61640 1 -- 172.21.15.190:0/3020017878 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc80ac660 con 0x7f4bc80a5c70 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 1 -- 172.21.15.190:0/3020017878 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4bb001a070 con 0x7f4bc80a5c70 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ba803be00 0x7f4ba803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bbffff640 1 -- 172.21.15.190:0/3020017878 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(267..267 src has 150..267) ==== 7657+0+0 (secure 0 0 0) 0x7f4bb000f040 con 0x7f4bc80a5c70 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bc6ffd640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ba803be00 0x7f4ba803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.129+0000 7f4bc6ffd640 10 monclient: get_auth_request con 0x7f4ba803be00 auth_method 0 2024-08-19T10:45:54.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.133+0000 7f4bc6ffd640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ba803be00 0x7f4ba803e2c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009d90 tx=0x7f4bb80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:54.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.133+0000 7f4bbffff640 1 -- 172.21.15.190:0/3020017878 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4bc80ac660 con 0x7f4bc80a5c70 2024-08-19T10:45:54.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.133+0000 7f4bbffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:54.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.133+0000 7f4bbffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:54.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.317+0000 7f4bcdc61640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.317+0000 7f4bcdc61640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.317+0000 7f4bcdc61640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:54.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.317+0000 7f4bcdc61640 1 -- 172.21.15.190:0/3020017878 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"} v 0) -- 0x7f4bc80a2a40 con 0x7f4bc80a5c70 2024-08-19T10:45:54.443 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.441+0000 7f4bbffff640 1 -- 172.21.15.190:0/3020017878 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}]=0 v268) ==== 106+0+3 (secure 0 0 0) 0x7f4bb0021020 con 0x7f4bc80a5c70 2024-08-19T10:45:54.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.441+0000 7f4bbffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.444 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.441+0000 7f4bbffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:54.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.557+0000 7f4bcdc61640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.557+0000 7f4bcdc61640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.557+0000 7f4bcdc61640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:54.559 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.557+0000 7f4bcdc61640 1 -- 172.21.15.190:0/3020017878 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"} v 0) -- 0x7f4bc80a1aa0 con 0x7f4bc80a5c70 2024-08-19T10:45:54.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bbffff640 1 -- 172.21.15.190:0/3020017878 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}]=0 v268) ==== 106+0+3 (secure 0 0 0) 0x7f4bc80ac430 con 0x7f4bc80a5c70 2024-08-19T10:45:54.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bbffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.561 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bbffff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:45:54.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 -- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ba803be00 msgr2=0x7f4ba803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:54.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ba803be00 0x7f4ba803e2c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009d90 tx=0x7f4bb80a7000 comp rx=0 tx=0).stop 2024-08-19T10:45:54.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 10 monclient: shutdown 2024-08-19T10:45:54.563 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:54.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 -- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc80a5c70 msgr2=0x7f4bc8140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:54.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc80a5c70 0x7f4bc8140890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000e930 tx=0x7f4bb000edf0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 -- 172.21.15.190:0/3020017878 shutdown_connections 2024-08-19T10:45:54.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ba803be00 0x7f4ba803e2c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4bc813a690 0x7f4bc813aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4bc8139980 0x7f4bc8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 --2- 172.21.15.190:0/3020017878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4bc80a5c70 0x7f4bc8140890 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 -- 172.21.15.190:0/3020017878 >> 172.21.15.190:0/3020017878 conn(0x7f4bc80abde0 msgr2=0x7f4bc8134c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:54.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 -- 172.21.15.190:0/3020017878 shutdown_connections 2024-08-19T10:45:54.564 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.561+0000 7f4bcdc61640 1 -- 172.21.15.190:0/3020017878 wait complete. 2024-08-19T10:45:54.579 INFO:tasks.workunit.client.0.smithi190.stderr:+ nb=14 2024-08-19T10:45:54.579 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd create f0a75527-a0ce-4e7b-85de-2da6aea4d069 2024-08-19T10:45:54.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient: get_monmap_and_config 2024-08-19T10:45:54.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient: build_initial_monmap 2024-08-19T10:45:54.690 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient: monmap: 2024-08-19T10:45:54.691 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:54.691 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:54.691 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:54.693453+0000 2024-08-19T10:45:54.691 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:54.693453+0000 2024-08-19T10:45:54.691 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:54.691 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.692 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:54.692 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:54.692 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:54.692 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:54.692 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:54.692 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:54.692 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:54.693 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:54.693 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:54.693 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:54.693 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:54.693 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 1 Processor -- start 2024-08-19T10:45:54.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 1 -- start start 2024-08-19T10:45:54.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient: init 2024-08-19T10:45:54.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:54.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient: _add_conns ranks=[3,4,9,2,10,7,5,6,8,0,1] 2024-08-19T10:45:54.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0068139980 0x7f0068139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient(hunting): picked mon.noname-g con 0x7f0068139980 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:54.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006813a2a0 0x7f00680a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient(hunting): picked mon.noname-c con 0x7f006813a2a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:54.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f00680a5c70 0x7f00680a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.698 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient(hunting): picked mon.noname-a con 0x7f00680a5c70 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:54.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:54.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f006809ca60 con 0x7f00680a5c70 2024-08-19T10:45:54.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:54.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f00680a3640 con 0x7f0068139980 2024-08-19T10:45:54.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:54.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f00680a8470 con 0x7f006813a2a0 2024-08-19T10:45:54.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:54.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.689+0000 7f006f5ee640 10 monclient(hunting): authenticate will time out at 1400.263911s 2024-08-19T10:45:54.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0068139980 0x7f0068139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.699 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0068139980 0x7f0068139d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47310/0 (socket says 172.21.15.190:47310) 2024-08-19T10:45:54.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 1 -- 172.21.15.190:0/1103735735 learned_addr learned my addr 172.21.15.190:0/1103735735 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:54.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 10 monclient(hunting): get_auth_request con 0x7f0068139980 auth_method 0 2024-08-19T10:45:54.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:54.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:54.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:54.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:54.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:54.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:54.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 10 monclient(hunting): handle_auth_done global_id 35004 payload 274 2024-08-19T10:45:54.700 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006cb89640 1 --2- 172.21.15.190:0/1103735735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006813a2a0 0x7f00680a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 1 -- 172.21.15.190:0/1103735735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006813a2a0 msgr2=0x7f00680a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006db8b640 1 --2- 172.21.15.190:0/1103735735 >> v2:172.21.15.89:3300/0 conn(0x7f00680a5c70 0x7f00680a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 1 --2- 172.21.15.190:0/1103735735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006813a2a0 0x7f00680a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 1 -- 172.21.15.190:0/1103735735 >> v2:172.21.15.89:3300/0 conn(0x7f00680a5c70 msgr2=0x7f00680a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 1 --2- 172.21.15.190:0/1103735735 >> v2:172.21.15.89:3300/0 conn(0x7f00680a5c70 0x7f00680a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 10 monclient: _finish_hunting 0 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 1 monclient: found mon.noname-g 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006d38a640 1 -- 172.21.15.190:0/1103735735 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00680a3850 con 0x7f0068139980 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006cb89640 10 monclient: get_auth_request con 0x7f006813a2a0 auth_method 0 2024-08-19T10:45:54.701 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.693+0000 7f006cb89640 1 --2- 172.21.15.190:0/1103735735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006813a2a0 0x7f00680a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006d38a640 1 -- 172.21.15.190:0/1103735735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0068139980 msgr2=0x7f0068139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006d38a640 1 --2- 172.21.15.190:0/1103735735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0068139980 0x7f0068139d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0064009870 tx=0x7f006401c160 comp rx=0 tx=0).ready entity=mon.3 client_cookie=cba4cfbb33f41e6e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 1 -- 172.21.15.190:0/1103735735 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00640b0070 con 0x7f0068139980 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 10 monclient: dump: 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:54.702 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 10 monclient: _finish_auth 0 2024-08-19T10:45:54.703 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 5 monclient: authenticate success, global_id 35004 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 1 -- 172.21.15.190:0/1103735735 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00640aa810 con 0x7f0068139980 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 1 -- 172.21.15.190:0/1103735735 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00640aae30 con 0x7f0068139980 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 4 set_mon_vals no callback set 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f005ffff640 10 monclient: dump: 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:54.704 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:54.705 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:54.706 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient: shutdown 2024-08-19T10:45:54.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:54.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 -- 172.21.15.190:0/1103735735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0068139980 msgr2=0x7f0068139d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:54.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 --2- 172.21.15.190:0/1103735735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0068139980 0x7f0068139d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0064009870 tx=0x7f006401c160 comp rx=0 tx=0).stop 2024-08-19T10:45:54.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 -- 172.21.15.190:0/1103735735 shutdown_connections 2024-08-19T10:45:54.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 --2- 172.21.15.190:0/1103735735 >> v2:172.21.15.89:3300/0 conn(0x7f00680a5c70 0x7f00680a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 --2- 172.21.15.190:0/1103735735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f006813a2a0 0x7f00680a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 --2- 172.21.15.190:0/1103735735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0068139980 0x7f0068139d60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 -- 172.21.15.190:0/1103735735 >> 172.21.15.190:0/1103735735 conn(0x7f00680abde0 msgr2=0x7f00680ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:54.706 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 -- 172.21.15.190:0/1103735735 shutdown_connections 2024-08-19T10:45:54.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 -- 172.21.15.190:0/1103735735 wait complete. 2024-08-19T10:45:54.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient: build_initial_monmap 2024-08-19T10:45:54.707 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient: monmap: 2024-08-19T10:45:54.707 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:54.707 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:54.707 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:54.707 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:54.707 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:54.707 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.707 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:54.707 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:54.708 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:54.708 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:54.708 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:54.708 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:54.708 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:54.708 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:54.708 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:54.708 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 Processor -- start 2024-08-19T10:45:54.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 -- start start 2024-08-19T10:45:54.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient: init 2024-08-19T10:45:54.712 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient: _add_conns ranks=[0,7,5,2,8,1,3,6,4] 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00680a5c70 0x7f00681d1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient(hunting): picked mon.noname-a con 0x7f00680a5c70 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0068139980 0x7f006813fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient(hunting): picked mon.noname-i con 0x7f0068139980 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0068140310 0x7f006809f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient(hunting): picked mon.noname-h con 0x7f0068140310 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0068143270 con 0x7f00680a5c70 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f00681430f0 con 0x7f0068140310 2024-08-19T10:45:54.713 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f00681433f0 con 0x7f0068139980 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006f5ee640 10 monclient(hunting): authenticate will time out at 1400.271017s 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006cb89640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0068139980 0x7f006813fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0068140310 0x7f006809f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006cb89640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0068139980 0x7f006813fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:59384/0 (socket says 172.21.15.190:59384) 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0068140310 0x7f006809f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:56888/0 (socket says 172.21.15.190:56888) 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006cb89640 1 -- 172.21.15.190:0/2997953426 learned_addr learned my addr 172.21.15.190:0/2997953426 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006cb89640 10 monclient(hunting): get_auth_request con 0x7f0068139980 auth_method 0 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006cb89640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006cb89640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:54.714 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006cb89640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 10 monclient(hunting): get_auth_request con 0x7f0068140310 auth_method 0 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006cb89640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006cb89640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006cb89640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006d38a640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00680a5c70 0x7f00681d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.715 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006d38a640 10 monclient(hunting): get_auth_request con 0x7f00680a5c70 auth_method 0 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006d38a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006d38a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006d38a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 10 monclient(hunting): handle_auth_done global_id 6503 payload 824 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 1 -- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0068139980 msgr2=0x7f006813fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0068139980 0x7f006813fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 1 -- 172.21.15.190:0/2997953426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00680a5c70 msgr2=0x7f00681d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00680a5c70 0x7f00681d1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 10 monclient: _finish_hunting 0 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 1 monclient: found mon.noname-h 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:54.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006db8b640 1 -- 172.21.15.190:0/2997953426 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00680a90e0 con 0x7f0068140310 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.697+0000 7f006cb89640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0068139980 0x7f006813fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006db8b640 1 -- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0068140310 msgr2=0x7f006809f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006db8b640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0068140310 0x7f006809f970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f006000cc80 tx=0x7f006000a660 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 1 -- 172.21.15.190:0/2997953426 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f006000ea70 con 0x7f0068140310 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 10 monclient: dump: 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.717 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:54.718 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 1 -- 172.21.15.190:0/2997953426 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f006000ec70 con 0x7f0068140310 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006f5ee640 5 monclient: authenticate success, global_id 6503 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006f5ee640 10 monclient: _renew_subs 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006f5ee640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006f5ee640 1 -- 172.21.15.190:0/2997953426 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006813aff0 con 0x7f0068140310 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 1 -- 172.21.15.190:0/2997953426 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f006000ee20 con 0x7f0068140310 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:54.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 10 monclient: dump: 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:54.720 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:54.721 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:54.721 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:54.721 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:54.721 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:54.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005effd640 4 set_mon_vals no callback set 2024-08-19T10:45:54.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006f5ee640 10 monclient: _renew_subs 2024-08-19T10:45:54.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006f5ee640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:54.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006f5ee640 1 -- 172.21.15.190:0/2997953426 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f006813b5b0 con 0x7f0068140310 2024-08-19T10:45:54.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006f5ee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:54.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006f5ee640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:54.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006f5ee640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:54.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 1 -- 172.21.15.190:0/2997953426 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f006000e600 con 0x7f0068140310 2024-08-19T10:45:54.722 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006f5ee640 1 -- 172.21.15.190:0/2997953426 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006813b7b0 con 0x7f0068140310 2024-08-19T10:45:54.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f004c03be30 0x7f004c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:54.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f005dffb640 1 -- 172.21.15.190:0/2997953426 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(268..268 src has 150..268) ==== 7657+0+0 (secure 0 0 0) 0x7f0060050010 con 0x7f0068140310 2024-08-19T10:45:54.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006d38a640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f004c03be30 0x7f004c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:54.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.701+0000 7f006d38a640 10 monclient: get_auth_request con 0x7f004c03be30 auth_method 0 2024-08-19T10:45:54.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.705+0000 7f006d38a640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f004c03be30 0x7f004c03e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0064009870 tx=0x7f00640a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:54.723 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.705+0000 7f005dffb640 1 -- 172.21.15.190:0/2997953426 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f006004ded0 con 0x7f0068140310 2024-08-19T10:45:54.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.705+0000 7f005dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:54.724 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.705+0000 7f005dffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:54.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.885+0000 7f006f5ee640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.885+0000 7f006f5ee640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.885+0000 7f006f5ee640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:54.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.885+0000 7f006f5ee640 1 -- 172.21.15.190:0/2997953426 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"} v 0) -- 0x7f0068139d60 con 0x7f0068140310 2024-08-19T10:45:54.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.889+0000 7f005dffb640 1 -- 172.21.15.190:0/2997953426 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}]=0 v268) ==== 106+0+3 (secure 0 0 0) 0x7f0060016430 con 0x7f0068140310 2024-08-19T10:45:54.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.889+0000 7f005dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.888 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.889+0000 7f005dffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:54.991 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.989+0000 7f006f5ee640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.989+0000 7f006f5ee640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.989+0000 7f006f5ee640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:54.992 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.989+0000 7f006f5ee640 1 -- 172.21.15.190:0/2997953426 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"} v 0) -- 0x7f00680008e0 con 0x7f0068140310 2024-08-19T10:45:54.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f005dffb640 1 -- 172.21.15.190:0/2997953426 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}]=0 v268) ==== 106+0+3 (secure 0 0 0) 0x7f0060014a20 con 0x7f0068140310 2024-08-19T10:45:54.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f005dffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "f0a75527-a0ce-4e7b-85de-2da6aea4d069"}] 2024-08-19T10:45:54.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f005dffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:45:54.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 1 -- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f004c03be30 msgr2=0x7f004c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:54.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f004c03be30 0x7f004c03e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0064009870 tx=0x7f00640a6000 comp rx=0 tx=0).stop 2024-08-19T10:45:54.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 10 monclient: shutdown 2024-08-19T10:45:54.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:54.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 1 -- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0068140310 msgr2=0x7f006809f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:54.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0068140310 0x7f006809f970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f006000cc80 tx=0x7f006000a660 comp rx=0 tx=0).stop 2024-08-19T10:45:54.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 1 -- 172.21.15.190:0/2997953426 shutdown_connections 2024-08-19T10:45:54.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f004c03be30 0x7f004c03e2f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0068140310 0x7f006809f970 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0068139980 0x7f006813fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 1 --2- 172.21.15.190:0/2997953426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00680a5c70 0x7f00681d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:54.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.993+0000 7f006f5ee640 1 -- 172.21.15.190:0/2997953426 >> 172.21.15.190:0/2997953426 conn(0x7f00680abde0 msgr2=0x7f00680ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:54.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.997+0000 7f006f5ee640 1 -- 172.21.15.190:0/2997953426 shutdown_connections 2024-08-19T10:45:54.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:54.997+0000 7f006f5ee640 1 -- 172.21.15.190:0/2997953426 wait complete. 2024-08-19T10:45:55.010 INFO:tasks.workunit.client.0.smithi190.stderr:+ test 14 -eq 14 2024-08-19T10:45:55.010 INFO:tasks.workunit.client.0.smithi190.stderr:+ test 14 -ne 12 2024-08-19T10:45:55.010 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd rm 12 2024-08-19T10:45:55.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.121+0000 7fa4c8b27640 10 monclient: get_monmap_and_config 2024-08-19T10:45:55.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.121+0000 7fa4c8b27640 10 monclient: build_initial_monmap 2024-08-19T10:45:55.124 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.121+0000 7fa4c8b27640 10 monclient: monmap: 2024-08-19T10:45:55.124 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:55.124 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:55.125 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:55.127397+0000 2024-08-19T10:45:55.125 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:55.127397+0000 2024-08-19T10:45:55.125 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:55.125 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.125 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:55.126 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:55.126 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:55.126 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:55.126 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:55.126 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:55.126 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:55.126 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:55.126 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:55.126 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:55.126 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:55.126 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 1 Processor -- start 2024-08-19T10:45:55.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 1 -- start start 2024-08-19T10:45:55.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 10 monclient: init 2024-08-19T10:45:55.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:55.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 10 monclient: _add_conns ranks=[9,2,7,3,8,5,0,10,1,6,4] 2024-08-19T10:45:55.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa4c4139980 0x7fa4c4139d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 10 monclient(hunting): picked mon.noname-a con 0x7fa4c4139980 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:55.129 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4c413a2a0 0x7fa4c40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 10 monclient(hunting): picked mon.noname-b con 0x7fa4c413a2a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:55.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa4c40a5c70 0x7fa4c40a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 10 monclient(hunting): picked mon.noname-i con 0x7fa4c40a5c70 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:55.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fa4c409ca60 con 0x7fa4c4139980 2024-08-19T10:45:55.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c40a3640 con 0x7fa4c413a2a0 2024-08-19T10:45:55.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa4c40a8470 con 0x7fa4c40a5c70 2024-08-19T10:45:55.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:55.130 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c8b27640 10 monclient(hunting): authenticate will time out at 1400.697270s 2024-08-19T10:45:55.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c1d9b640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4c413a2a0 msgr2=0x7fa4c40a5730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:55.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa4c40a5c70 0x7fa4c40a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c1d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4c413a2a0 0x7fa4c40a5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:55.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa4c40a5c70 0x7fa4c40a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:59398/0 (socket says 172.21.15.190:59398) 2024-08-19T10:45:55.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 1 -- 172.21.15.190:0/2472300601 learned_addr learned my addr 172.21.15.190:0/2472300601 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:55.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 10 monclient(hunting): get_auth_request con 0x7fa4c40a5c70 auth_method 0 2024-08-19T10:45:55.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:55.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:55.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:55.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:55.131 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c259c640 1 --2- 172.21.15.190:0/2472300601 >> v2:172.21.15.89:3300/0 conn(0x7fa4c4139980 0x7fa4c4139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c259c640 10 monclient(hunting): get_auth_request con 0x7fa4c4139980 auth_method 0 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 10 monclient(hunting): handle_auth_done global_id 15847 payload 274 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 1 -- 172.21.15.190:0/2472300601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4c413a2a0 msgr2=0x7fa4c40a5730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 1 --2- 172.21.15.190:0/2472300601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4c413a2a0 0x7fa4c40a5730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 1 -- 172.21.15.190:0/2472300601 >> v2:172.21.15.89:3300/0 conn(0x7fa4c4139980 msgr2=0x7fa4c4139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 1 --2- 172.21.15.190:0/2472300601 >> v2:172.21.15.89:3300/0 conn(0x7fa4c4139980 0x7fa4c4139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.132 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:45:55.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 1 monclient: found mon.noname-i 2024-08-19T10:45:55.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:55.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:55.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c2d9d640 1 -- 172.21.15.190:0/2472300601 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c40a3850 con 0x7fa4c40a5c70 2024-08-19T10:45:55.133 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.125+0000 7fa4c259c640 1 --2- 172.21.15.190:0/2472300601 >> v2:172.21.15.89:3300/0 conn(0x7fa4c4139980 0x7fa4c4139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:45:55.138 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c2d9d640 1 -- 172.21.15.190:0/2472300601 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa4c40a5c70 msgr2=0x7fa4c40a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:55.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c2d9d640 1 --2- 172.21.15.190:0/2472300601 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa4c40a5c70 0x7fa4c40a67e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009e80 tx=0x7fa4bc01c630 comp rx=0 tx=0).ready entity=mon.7 client_cookie=6878c34efedb953e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:55.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 1 -- 172.21.15.190:0/2472300601 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4bc0ac070 con 0x7fa4c40a5c70 2024-08-19T10:45:55.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:55.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:55.139 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 10 monclient: dump: 2024-08-19T10:45:55.139 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:55.139 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:55.139 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.139 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.139 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:55.139 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.140 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:55.140 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:55.140 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:55.140 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:55.140 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:55.140 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:55.140 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:55.140 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:55.140 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:55.140 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.140 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 10 monclient: _finish_auth 0 2024-08-19T10:45:55.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 10 monclient: _check_auth_tickets 2024-08-19T10:45:55.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:55.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 1 -- 172.21.15.190:0/2472300601 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc0090d0 con 0x7fa4c40a5c70 2024-08-19T10:45:55.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:55.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 1 -- 172.21.15.190:0/2472300601 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4bc007cd0 con 0x7fa4c40a5c70 2024-08-19T10:45:55.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 5 monclient: authenticate success, global_id 15847 2024-08-19T10:45:55.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:55.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 4 set_mon_vals no callback set 2024-08-19T10:45:55.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:55.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:55.141 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c159a640 10 monclient: dump: 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:55.142 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:55.143 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:55.143 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:55.143 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:55.143 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 10 monclient: shutdown 2024-08-19T10:45:55.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:55.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 1 -- 172.21.15.190:0/2472300601 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa4c40a5c70 msgr2=0x7fa4c40a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:55.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 1 --2- 172.21.15.190:0/2472300601 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa4c40a5c70 0x7fa4c40a67e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009e80 tx=0x7fa4bc01c630 comp rx=0 tx=0).stop 2024-08-19T10:45:55.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 1 -- 172.21.15.190:0/2472300601 shutdown_connections 2024-08-19T10:45:55.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 1 --2- 172.21.15.190:0/2472300601 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa4c40a5c70 0x7fa4c40a67e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.143 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 1 --2- 172.21.15.190:0/2472300601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa4c413a2a0 0x7fa4c40a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 1 --2- 172.21.15.190:0/2472300601 >> v2:172.21.15.89:3300/0 conn(0x7fa4c4139980 0x7fa4c4139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 1 -- 172.21.15.190:0/2472300601 >> 172.21.15.190:0/2472300601 conn(0x7fa4c40abde0 msgr2=0x7fa4c40ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 1 -- 172.21.15.190:0/2472300601 shutdown_connections 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 1 -- 172.21.15.190:0/2472300601 wait complete. 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 10 monclient: build_initial_monmap 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.129+0000 7fa4c8b27640 10 monclient: monmap: 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:55.144 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.145 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:55.145 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:55.145 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:55.145 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:55.145 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:55.145 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:55.145 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:55.145 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:55.145 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:55.145 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 1 Processor -- start 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 1 -- start start 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 10 monclient: init 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 10 monclient: _add_conns ranks=[3,6,4,1,7,8,5,0,2] 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4c40a5c70 0x7fa4c4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 10 monclient(hunting): picked mon.noname-g con 0x7fa4c40a5c70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa4c4139980 0x7fa4c409f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 10 monclient(hunting): picked mon.noname-d con 0x7fa4c4139980 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4c4132270 0x7fa4c409f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 10 monclient(hunting): picked mon.noname-c con 0x7fa4c4132270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.146 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c4143270 con 0x7fa4c40a5c70 2024-08-19T10:45:55.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa4c41430f0 con 0x7fa4c4132270 2024-08-19T10:45:55.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa4c41433f0 con 0x7fa4c4139980 2024-08-19T10:45:55.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:55.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 10 monclient(hunting): authenticate will time out at 1400.704722s 2024-08-19T10:45:55.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4c40a5c70 0x7fa4c4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4c40a5c70 0x7fa4c4131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47318/0 (socket says 172.21.15.190:47318) 2024-08-19T10:45:55.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 1 -- 172.21.15.190:0/4111922926 learned_addr learned my addr 172.21.15.190:0/4111922926 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:55.147 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 10 monclient(hunting): get_auth_request con 0x7fa4c40a5c70 auth_method 0 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c1d9b640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa4c4139980 0x7fa4c409f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 10 monclient(hunting): handle_auth_done global_id 35013 payload 824 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c2d9d640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4c4132270 0x7fa4c409f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 1 -- 172.21.15.190:0/4111922926 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa4c4139980 msgr2=0x7fa4c409f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa4c4139980 0x7fa4c409f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.148 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 1 -- 172.21.15.190:0/4111922926 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4c4132270 msgr2=0x7fa4c409f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:55.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4c4132270 0x7fa4c409f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 10 monclient: _finish_hunting 0 2024-08-19T10:45:55.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 1 monclient: found mon.noname-g 2024-08-19T10:45:55.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:55.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:55.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 1 -- 172.21.15.190:0/4111922926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c41357f0 con 0x7fa4c40a5c70 2024-08-19T10:45:55.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 1 -- 172.21.15.190:0/4111922926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4c40a5c70 msgr2=0x7fa4c4131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:55.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c259c640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4c40a5c70 0x7fa4c4131d30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400e810 tx=0x7fa4b400ecd0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:55.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 1 -- 172.21.15.190:0/4111922926 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4b4017070 con 0x7fa4c40a5c70 2024-08-19T10:45:55.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:55.149 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 10 monclient: dump: 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:55.150 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:55.151 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:55.151 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:55.151 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:55.151 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:55.151 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:45:55.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:55.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:55.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c8b27640 5 monclient: authenticate success, global_id 35013 2024-08-19T10:45:55.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 1 -- 172.21.15.190:0/4111922926 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b400cd50 con 0x7fa4c40a5c70 2024-08-19T10:45:55.151 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 1 -- 172.21.15.190:0/4111922926 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa4b400f040 con 0x7fa4c40a5c70 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4c0d99640 4 set_mon_vals no callback set 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.133+0000 7fa4b2ffd640 10 monclient: dump: 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.152 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:55.153 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:55.153 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:55.153 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:55.153 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:55.153 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:55.153 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:55.153 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:55.153 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:55.153 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c8b27640 10 monclient: _renew_subs 2024-08-19T10:45:55.153 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c8b27640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c8b27640 1 -- 172.21.15.190:0/4111922926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c40a5950 con 0x7fa4c40a5c70 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c8b27640 10 monclient: _renew_subs 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c8b27640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c8b27640 1 -- 172.21.15.190:0/4111922926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c40a0cd0 con 0x7fa4c40a5c70 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c8b27640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c8b27640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c8b27640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c8b27640 1 -- 172.21.15.190:0/4111922926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c4139d60 con 0x7fa4c40a5c70 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4b2ffd640 1 -- 172.21.15.190:0/4111922926 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa4b401a070 con 0x7fa4c40a5c70 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4b2ffd640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa4a403be00 0x7fa4a403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4b2ffd640 1 -- 172.21.15.190:0/4111922926 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(268..268 src has 150..268) ==== 7657+0+0 (secure 0 0 0) 0x7fa4b404f2b0 con 0x7fa4c40a5c70 2024-08-19T10:45:55.154 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c1d9b640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa4a403be00 0x7fa4a403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c1d9b640 10 monclient: get_auth_request con 0x7fa4a403be00 auth_method 0 2024-08-19T10:45:55.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.137+0000 7fa4c1d9b640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa4a403be00 0x7fa4a403e2c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40a0940 tx=0x7fa4b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:55.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.141+0000 7fa4b2ffd640 1 -- 172.21.15.190:0/4111922926 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fa4c41357f0 con 0x7fa4c40a5c70 2024-08-19T10:45:55.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.141+0000 7fa4b2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:55.155 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.141+0000 7fa4b2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.321+0000 7fa4c8b27640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.321+0000 7fa4c8b27640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.321+0000 7fa4c8b27640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:55.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.321+0000 7fa4c8b27640 1 -- 172.21.15.190:0/4111922926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) -- 0x7fa4c406c8d0 con 0x7fa4c40a5c70 2024-08-19T10:45:55.322 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:45:55.324+0000 7f16cc065640 -1 mon.a@0(leader).osd e268 definitely_dead 0 2024-08-19T10:45:55.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.337+0000 7fa4b2ffd640 1 -- 172.21.15.190:0/4111922926 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 removed osd.12 v269) ==== 83+0+0 (secure 0 0 0) 0x7fa4b4012dc0 con 0x7fa4c40a5c70 2024-08-19T10:45:55.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.337+0000 7fa4b2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.340 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.337+0000 7fa4b2ffd640 10 monclient: _finish_command 2 = system:0 removed osd.12 2024-08-19T10:45:55.446 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.445+0000 7fa4c8b27640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.445+0000 7fa4c8b27640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.445+0000 7fa4c8b27640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:55.447 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.445+0000 7fa4c8b27640 1 -- 172.21.15.190:0/4111922926 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) -- 0x7fa4c40a1830 con 0x7fa4c40a5c70 2024-08-19T10:45:55.453 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:45:55.452+0000 7f16cc065640 -1 mon.a@0(leader).osd e269 definitely_dead 0 2024-08-19T10:45:55.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.453+0000 7fa4b2ffd640 1 -- 172.21.15.190:0/4111922926 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v269) ==== 92+0+0 (secure 0 0 0) 0x7fa4b404d170 con 0x7fa4c40a5c70 2024-08-19T10:45:55.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.453+0000 7fa4b2ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.454 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.453+0000 7fa4b2ffd640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-08-19T10:45:55.454 INFO:tasks.workunit.client.0.smithi190.stderr:osd.12 does not exist. 2024-08-19T10:45:55.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 -- 172.21.15.190:0/4111922926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa4a403be00 msgr2=0x7fa4a403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:55.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa4a403be00 0x7fa4a403e2c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40a0940 tx=0x7fa4b8008040 comp rx=0 tx=0).stop 2024-08-19T10:45:55.457 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 10 monclient: shutdown 2024-08-19T10:45:55.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:55.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 -- 172.21.15.190:0/4111922926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4c40a5c70 msgr2=0x7fa4c4131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:55.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4c40a5c70 0x7fa4c4131d30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400e810 tx=0x7fa4b400ecd0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 -- 172.21.15.190:0/4111922926 shutdown_connections 2024-08-19T10:45:55.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa4a403be00 0x7fa4a403e2c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa4c4132270 0x7fa4c409f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa4c4139980 0x7fa4c409f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 --2- 172.21.15.190:0/4111922926 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4c40a5c70 0x7fa4c4131d30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 -- 172.21.15.190:0/4111922926 >> 172.21.15.190:0/4111922926 conn(0x7fa4c40abde0 msgr2=0x7fa4c4134ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:55.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 -- 172.21.15.190:0/4111922926 shutdown_connections 2024-08-19T10:45:55.458 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.457+0000 7fa4c8b27640 1 -- 172.21.15.190:0/4111922926 wait complete. 2024-08-19T10:45:55.474 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd rm 12 2024-08-19T10:45:55.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.573+0000 7ff45899a640 10 monclient: get_monmap_and_config 2024-08-19T10:45:55.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.573+0000 7ff45899a640 10 monclient: build_initial_monmap 2024-08-19T10:45:55.576 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.573+0000 7ff45899a640 10 monclient: monmap: 2024-08-19T10:45:55.576 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:55.576 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:55.576 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:55.579293+0000 2024-08-19T10:45:55.576 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:55.579293+0000 2024-08-19T10:45:55.577 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:55.577 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.577 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:55.577 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:55.577 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:55.577 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:55.577 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:55.577 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:55.577 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:55.577 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:55.577 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:55.578 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:55.578 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:55.578 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 1 Processor -- start 2024-08-19T10:45:55.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 1 -- start start 2024-08-19T10:45:55.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 10 monclient: init 2024-08-19T10:45:55.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:55.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 10 monclient: _add_conns ranks=[7,6,5,8,4,3,9,10,2,0,1] 2024-08-19T10:45:55.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a6740 0x7ff4540a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 10 monclient(hunting): picked mon.noname-i con 0x7ff4540a6740 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:55.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4540a5bc0 0x7ff4540a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 10 monclient(hunting): picked mon.noname-d con 0x7ff4540a5bc0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:55.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4540a7060 0x7ff454139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 10 monclient(hunting): picked mon.noname-h con 0x7ff4540a7060 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:55.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff4540a4c20 con 0x7ff4540a7060 2024-08-19T10:45:55.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff4540a3530 con 0x7ff4540a5bc0 2024-08-19T10:45:55.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff4540a3740 con 0x7ff4540a6740 2024-08-19T10:45:55.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:55.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45899a640 10 monclient(hunting): authenticate will time out at 1401.149339s 2024-08-19T10:45:55.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45259c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a6740 0x7ff4540a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4540a7060 0x7ff454139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45259c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a6740 0x7ff4540a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:53402/0 (socket says 172.21.15.190:53402) 2024-08-19T10:45:55.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45259c640 1 -- 172.21.15.190:0/2388439420 learned_addr learned my addr 172.21.15.190:0/2388439420 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:55.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45259c640 10 monclient(hunting): get_auth_request con 0x7ff4540a6740 auth_method 0 2024-08-19T10:45:55.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:55.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:55.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:55.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 10 monclient(hunting): get_auth_request con 0x7ff4540a7060 auth_method 0 2024-08-19T10:45:55.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:55.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:55.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:55.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:55.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:55.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:55.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:55.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:55.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff45259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:55.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 10 monclient(hunting): handle_auth_done global_id 6512 payload 274 2024-08-19T10:45:55.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 1 -- 172.21.15.190:0/2388439420 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a6740 msgr2=0x7ff4540a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:55.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 1 --2- 172.21.15.190:0/2388439420 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a6740 0x7ff4540a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 1 -- 172.21.15.190:0/2388439420 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4540a5bc0 msgr2=0x7ff4540a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:55.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 1 --2- 172.21.15.190:0/2388439420 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4540a5bc0 0x7ff4540a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:45:55.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 1 monclient: found mon.noname-h 2024-08-19T10:45:55.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:55.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:55.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.577+0000 7ff452d9d640 1 -- 172.21.15.190:0/2388439420 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4540a3240 con 0x7ff4540a7060 2024-08-19T10:45:55.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff452d9d640 1 -- 172.21.15.190:0/2388439420 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4540a7060 msgr2=0x7ff454139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:55.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff452d9d640 1 --2- 172.21.15.190:0/2388439420 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4540a7060 0x7ff454139590 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009df0 tx=0x7ff44c01c850 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d231940ea11b9fc9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:55.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 1 -- 172.21.15.190:0/2388439420 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff44c0b2070 con 0x7ff4540a7060 2024-08-19T10:45:55.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:55.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:55.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 10 monclient: dump: 2024-08-19T10:45:55.592 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:55.592 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:55.592 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.592 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.593 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:55.593 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.593 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:55.593 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:55.593 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:55.593 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:55.593 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:55.593 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:55.593 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:55.593 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:55.594 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:55.594 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 10 monclient: _finish_auth 0 2024-08-19T10:45:55.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 10 monclient: _check_auth_tickets 2024-08-19T10:45:55.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:55.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 1 -- 172.21.15.190:0/2388439420 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff44c0a59c0 con 0x7ff4540a7060 2024-08-19T10:45:55.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 5 monclient: authenticate success, global_id 6512 2024-08-19T10:45:55.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:55.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:55.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:55.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 1 -- 172.21.15.190:0/2388439420 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff44c0a5bb0 con 0x7ff4540a7060 2024-08-19T10:45:55.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:55.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 4 set_mon_vals no callback set 2024-08-19T10:45:55.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:55.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45159a640 10 monclient: dump: 2024-08-19T10:45:55.595 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:55.595 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:55.595 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.595 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.595 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:55.595 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.595 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 10 monclient: shutdown 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:55.596 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 1 -- 172.21.15.190:0/2388439420 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4540a7060 msgr2=0x7ff454139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:55.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 1 --2- 172.21.15.190:0/2388439420 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4540a7060 0x7ff454139590 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009df0 tx=0x7ff44c01c850 comp rx=0 tx=0).stop 2024-08-19T10:45:55.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 1 -- 172.21.15.190:0/2388439420 shutdown_connections 2024-08-19T10:45:55.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 1 --2- 172.21.15.190:0/2388439420 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4540a7060 0x7ff454139590 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 1 --2- 172.21.15.190:0/2388439420 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4540a5bc0 0x7ff4540a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 1 --2- 172.21.15.190:0/2388439420 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a6740 0x7ff4540a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 1 -- 172.21.15.190:0/2388439420 >> 172.21.15.190:0/2388439420 conn(0x7ff4540abd40 msgr2=0x7ff4540ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:55.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 1 -- 172.21.15.190:0/2388439420 shutdown_connections 2024-08-19T10:45:55.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 1 -- 172.21.15.190:0/2388439420 wait complete. 2024-08-19T10:45:55.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 10 monclient: build_initial_monmap 2024-08-19T10:45:55.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.581+0000 7ff45899a640 10 monclient: monmap: 2024-08-19T10:45:55.597 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:55.598 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:55.598 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:55.598 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:55.598 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:55.598 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.598 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:55.598 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:55.598 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:55.598 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:55.598 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:55.598 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:55.599 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:55.599 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:55.599 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:55.599 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 1 Processor -- start 2024-08-19T10:45:55.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 1 -- start start 2024-08-19T10:45:55.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient: init 2024-08-19T10:45:55.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:55.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient: _add_conns ranks=[6,7,5,3,4,1,2,8,0] 2024-08-19T10:45:55.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4540a6740 0x7ff4540a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient(hunting): picked mon.noname-d con 0x7ff4540a6740 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a7060 0x7ff454135910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient(hunting): picked mon.noname-i con 0x7ff4540a7060 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff454135e50 0x7ff45409d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient(hunting): picked mon.noname-h con 0x7ff454135e50 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff454143450 con 0x7ff454135e50 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff4541432d0 con 0x7ff4540a6740 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff4541435d0 con 0x7ff4540a7060 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:55.600 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient(hunting): authenticate will time out at 1401.156718s 2024-08-19T10:45:55.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff454135e50 0x7ff45409d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff451d9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a7060 0x7ff454135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff454135e50 0x7ff45409d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:50468/0 (socket says 172.21.15.190:50468) 2024-08-19T10:45:55.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff451d9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a7060 0x7ff454135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:53404/0 (socket says 172.21.15.190:53404) 2024-08-19T10:45:55.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 1 -- 172.21.15.190:0/4110162974 learned_addr learned my addr 172.21.15.190:0/4110162974 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:55.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 10 monclient(hunting): get_auth_request con 0x7ff454135e50 auth_method 0 2024-08-19T10:45:55.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:55.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:55.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:55.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff451d9b640 10 monclient(hunting): get_auth_request con 0x7ff4540a7060 auth_method 0 2024-08-19T10:45:55.601 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff451d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:55.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff451d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:55.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff451d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:55.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:55.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:55.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:55.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff451d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:55.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff451d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:55.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff451d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:55.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 10 monclient(hunting): handle_auth_done global_id 6521 payload 824 2024-08-19T10:45:55.602 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 1 -- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a7060 msgr2=0x7ff454135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:55.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a7060 0x7ff454135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 1 -- 172.21.15.190:0/4110162974 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4540a6740 msgr2=0x7ff4540a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:55.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4540a6740 0x7ff4540a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:45:55.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 1 monclient: found mon.noname-h 2024-08-19T10:45:55.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:55.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:45:55.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 1 -- 172.21.15.190:0/4110162974 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45409f450 con 0x7ff454135e50 2024-08-19T10:45:55.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff451d9b640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a7060 0x7ff454135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:55.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 1 -- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff454135e50 msgr2=0x7ff45409d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:55.603 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff452d9d640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff454135e50 0x7ff45409d9b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009df0 tx=0x7ff44c0195f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:55.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 1 -- 172.21.15.190:0/4110162974 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff44c0b2050 con 0x7ff454135e50 2024-08-19T10:45:55.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:55.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:45:55.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 10 monclient: dump: 2024-08-19T10:45:55.604 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:55.604 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:55.604 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.604 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.604 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:55.604 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.604 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:55.605 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:55.605 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:55.605 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:55.605 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:55.605 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:55.605 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:55.605 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:55.605 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:55.605 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 10 monclient: _finish_auth 0 2024-08-19T10:45:55.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:55.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:55.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 1 -- 172.21.15.190:0/4110162974 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff44c0b3040 con 0x7ff454135e50 2024-08-19T10:45:55.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:55.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 1 -- 172.21.15.190:0/4110162974 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff44c003dc0 con 0x7ff454135e50 2024-08-19T10:45:55.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 5 monclient: authenticate success, global_id 6521 2024-08-19T10:45:55.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient: _renew_subs 2024-08-19T10:45:55.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:55.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 1 -- 172.21.15.190:0/4110162974 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4540ab3a0 con 0x7ff454135e50 2024-08-19T10:45:55.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff450d99640 4 set_mon_vals no callback set 2024-08-19T10:45:55.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:55.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:45:55.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff442ffd640 10 monclient: dump: 2024-08-19T10:45:55.607 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:55.607 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:55.607 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.607 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:55.607 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:55.607 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.607 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:55.607 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:55.608 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:55.608 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:55.608 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:55.608 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:55.608 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:55.608 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:55.608 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:55.608 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient: _renew_subs 2024-08-19T10:45:55.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:55.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.585+0000 7ff45899a640 1 -- 172.21.15.190:0/4110162974 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4540a17b0 con 0x7ff454135e50 2024-08-19T10:45:55.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.589+0000 7ff442ffd640 1 -- 172.21.15.190:0/4110162974 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff44c0c0030 con 0x7ff454135e50 2024-08-19T10:45:55.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.589+0000 7ff45899a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:55.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.589+0000 7ff45899a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:55.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.589+0000 7ff45899a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:55.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.589+0000 7ff45899a640 1 -- 172.21.15.190:0/4110162974 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4540a5680 con 0x7ff454135e50 2024-08-19T10:45:55.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.589+0000 7ff442ffd640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff43403be00 0x7ff43403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.589+0000 7ff442ffd640 1 -- 172.21.15.190:0/4110162974 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(269..269 src has 150..269) ==== 7657+0+0 (secure 0 0 0) 0x7ff44c0be020 con 0x7ff454135e50 2024-08-19T10:45:55.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.589+0000 7ff45259c640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff43403be00 0x7ff43403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.589+0000 7ff45259c640 10 monclient: get_auth_request con 0x7ff43403be00 auth_method 0 2024-08-19T10:45:55.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.589+0000 7ff45259c640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff43403be00 0x7ff43403e2c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff4440094a0 tx=0x7ff4440093f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:55.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.593+0000 7ff442ffd640 1 -- 172.21.15.190:0/4110162974 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff44c0c7060 con 0x7ff454135e50 2024-08-19T10:45:55.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.593+0000 7ff442ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:55.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.593+0000 7ff442ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.757+0000 7ff45899a640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.757+0000 7ff45899a640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.757+0000 7ff45899a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:55.757 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.757+0000 7ff45899a640 1 -- 172.21.15.190:0/4110162974 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) -- 0x7ff45406c8d0 con 0x7ff454135e50 2024-08-19T10:45:55.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:45:55.760+0000 7f16cc065640 -1 mon.a@0(leader).osd e269 definitely_dead 0 2024-08-19T10:45:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.757+0000 7ff442ffd640 1 -- 172.21.15.190:0/4110162974 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v269) ==== 92+0+0 (secure 0 0 0) 0x7ff44c016560 con 0x7ff454135e50 2024-08-19T10:45:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.757+0000 7ff442ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.759 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.757+0000 7ff442ffd640 10 monclient: _finish_command 2 = system:0 osd.12 does not exist. 2024-08-19T10:45:55.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.857+0000 7ff45899a640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.857+0000 7ff45899a640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.857+0000 7ff45899a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:45:55.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.857+0000 7ff45899a640 1 -- 172.21.15.190:0/4110162974 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "osd rm", "ids": ["12"]} v 0) -- 0x7ff4540a0f10 con 0x7ff454135e50 2024-08-19T10:45:55.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:45:55.860+0000 7f16cc065640 -1 mon.a@0(leader).osd e269 definitely_dead 0 2024-08-19T10:45:55.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff442ffd640 1 -- 172.21.15.190:0/4110162974 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["12"]}]=0 osd.12 does not exist. v269) ==== 92+0+0 (secure 0 0 0) 0x7ff44c0042d0 con 0x7ff454135e50 2024-08-19T10:45:55.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff442ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["12"]}] 2024-08-19T10:45:55.860 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff442ffd640 10 monclient: _finish_command 3 = system:0 osd.12 does not exist. 2024-08-19T10:45:55.860 INFO:tasks.workunit.client.0.smithi190.stderr:osd.12 does not exist. 2024-08-19T10:45:55.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 -- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff43403be00 msgr2=0x7ff43403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:55.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff43403be00 0x7ff43403e2c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff4440094a0 tx=0x7ff4440093f0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 10 monclient: shutdown 2024-08-19T10:45:55.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:55.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 -- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff454135e50 msgr2=0x7ff45409d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:55.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff454135e50 0x7ff45409d9b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009df0 tx=0x7ff44c0195f0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 -- 172.21.15.190:0/4110162974 shutdown_connections 2024-08-19T10:45:55.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff43403be00 0x7ff43403e2c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff454135e50 0x7ff45409d9b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.863 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4540a7060 0x7ff454135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 --2- 172.21.15.190:0/4110162974 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4540a6740 0x7ff4540a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 -- 172.21.15.190:0/4110162974 >> 172.21.15.190:0/4110162974 conn(0x7ff4540abd40 msgr2=0x7ff45413a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:55.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 -- 172.21.15.190:0/4110162974 shutdown_connections 2024-08-19T10:45:55.864 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.861+0000 7ff45899a640 1 -- 172.21.15.190:0/4110162974 wait complete. 2024-08-19T10:45:55.876 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd rm 14 2024-08-19T10:45:55.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient: get_monmap_and_config 2024-08-19T10:45:55.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient: build_initial_monmap 2024-08-19T10:45:55.989 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient: monmap: 2024-08-19T10:45:55.989 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:55.989 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:55.992541+0000 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:55.992541+0000 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:55.990 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:55.991 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:55.991 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:55.991 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:55.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 1 Processor -- start 2024-08-19T10:45:55.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 1 -- start start 2024-08-19T10:45:55.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient: init 2024-08-19T10:45:55.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:55.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient: _add_conns ranks=[3,5,4,6,9,8,0,1,7,10,2] 2024-08-19T10:45:55.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00a56c0 0x7fb4f00a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient(hunting): picked mon.noname-g con 0x7fb4f00a56c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:55.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb4f00722a0 0x7fb4f00a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.993 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient(hunting): picked mon.noname-h con 0x7fb4f00722a0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:55.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f00a5fe0 0x7fb4f01374e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:55.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient(hunting): picked mon.noname-c con 0x7fb4f00a5fe0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:55.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f0072720 con 0x7fb4f00a56c0 2024-08-19T10:45:55.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb4f00737c0 con 0x7fb4f00a5fe0 2024-08-19T10:45:55.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:55.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb4f00a71e0 con 0x7fb4f00722a0 2024-08-19T10:45:55.994 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:55.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f7508640 10 monclient(hunting): authenticate will time out at 1401.563290s 2024-08-19T10:45:55.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00a56c0 0x7fb4f00a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00a56c0 0x7fb4f00a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47254/0 (socket says 172.21.15.190:47254) 2024-08-19T10:45:55.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 1 -- 172.21.15.190:0/2290856412 learned_addr learned my addr 172.21.15.190:0/2290856412 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:55.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 10 monclient(hunting): get_auth_request con 0x7fb4f00a56c0 auth_method 0 2024-08-19T10:45:55.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:55.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:55.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:55.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f4aa3640 1 --2- 172.21.15.190:0/2290856412 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb4f00722a0 0x7fb4f00a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f4aa3640 10 monclient(hunting): get_auth_request con 0x7fb4f00722a0 auth_method 0 2024-08-19T10:45:55.995 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f4aa3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f4aa3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f4aa3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f4aa3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f4aa3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f4aa3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f5aa5640 1 --2- 172.21.15.190:0/2290856412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f00a5fe0 0x7fb4f01374e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 10 monclient(hunting): handle_auth_done global_id 35022 payload 274 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 1 -- 172.21.15.190:0/2290856412 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb4f00722a0 msgr2=0x7fb4f00a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 1 --2- 172.21.15.190:0/2290856412 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb4f00722a0 0x7fb4f00a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 1 -- 172.21.15.190:0/2290856412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f00a5fe0 msgr2=0x7fb4f01374e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:55.996 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 1 --2- 172.21.15.190:0/2290856412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f00a5fe0 0x7fb4f01374e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:55.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 10 monclient: _finish_hunting 0 2024-08-19T10:45:55.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 1 monclient: found mon.noname-g 2024-08-19T10:45:55.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:55.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:55.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f52a4640 1 -- 172.21.15.190:0/2290856412 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f00a6420 con 0x7fb4f00a56c0 2024-08-19T10:45:55.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f5aa5640 10 monclient: get_auth_request con 0x7fb4f00a5fe0 auth_method 0 2024-08-19T10:45:55.997 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.989+0000 7fb4f5aa5640 1 --2- 172.21.15.190:0/2290856412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f00a5fe0 0x7fb4f01374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:56.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.993+0000 7fb4f52a4640 1 -- 172.21.15.190:0/2290856412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00a56c0 msgr2=0x7fb4f00a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:56.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f52a4640 1 --2- 172.21.15.190:0/2290856412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00a56c0 0x7fb4f00a16e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0098a0 tx=0x7fb4ec01c180 comp rx=0 tx=0).ready entity=mon.3 client_cookie=7e39cb520fc1c712 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:56.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 1 -- 172.21.15.190:0/2290856412 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4ec0b1070 con 0x7fb4f00a56c0 2024-08-19T10:45:56.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:56.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:56.001 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 10 monclient: dump: 2024-08-19T10:45:56.001 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:56.001 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:56.001 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.001 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.001 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 10 monclient: _finish_auth 0 2024-08-19T10:45:56.002 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 5 monclient: authenticate success, global_id 35022 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 1 -- 172.21.15.190:0/2290856412 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0aa7b0 con 0x7fb4f00a56c0 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 1 -- 172.21.15.190:0/2290856412 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4ec0a4e60 con 0x7fb4f00a56c0 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 4 set_mon_vals no callback set 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4e7fff640 10 monclient: dump: 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:56.003 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:56.004 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient: shutdown 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 -- 172.21.15.190:0/2290856412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00a56c0 msgr2=0x7fb4f00a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 --2- 172.21.15.190:0/2290856412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00a56c0 0x7fb4f00a16e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0098a0 tx=0x7fb4ec01c180 comp rx=0 tx=0).stop 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 -- 172.21.15.190:0/2290856412 shutdown_connections 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 --2- 172.21.15.190:0/2290856412 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4f00a5fe0 0x7fb4f01374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 --2- 172.21.15.190:0/2290856412 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb4f00722a0 0x7fb4f00a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 --2- 172.21.15.190:0/2290856412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4f00a56c0 0x7fb4f00a16e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 -- 172.21.15.190:0/2290856412 >> 172.21.15.190:0/2290856412 conn(0x7fb4f00aa2b0 msgr2=0x7fb4f00a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 -- 172.21.15.190:0/2290856412 shutdown_connections 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 -- 172.21.15.190:0/2290856412 wait complete. 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient: build_initial_monmap 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient: monmap: 2024-08-19T10:45:56.005 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:56.006 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:56.006 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:56.006 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:56.006 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:56.006 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:56.006 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:56.006 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:56.006 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:56.006 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:56.006 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 Processor -- start 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 -- start start 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient: init 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient: _add_conns ranks=[0,1,5,8,2,4,7,3,6] 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4f00722a0 0x7fb4f01330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient(hunting): picked mon.noname-a con 0x7fb4f00722a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:56.007 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00a56c0 0x7fb4f01335e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient(hunting): picked mon.noname-f con 0x7fb4f00a56c0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb4f00a5fe0 0x7fb4f009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient(hunting): picked mon.noname-h con 0x7fb4f00a5fe0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0141140 con 0x7fb4f00722a0 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0140fc0 con 0x7fb4f00a56c0 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb4f01412c0 con 0x7fb4f00a5fe0 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f7508640 10 monclient(hunting): authenticate will time out at 1401.570586s 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00a56c0 0x7fb4f01335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:56.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00a56c0 0x7fb4f01335e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:44532/0 (socket says 172.21.15.190:44532) 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 1 -- 172.21.15.190:0/3823824715 learned_addr learned my addr 172.21.15.190:0/3823824715 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 10 monclient(hunting): get_auth_request con 0x7fb4f00a56c0 auth_method 0 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f5aa5640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb4f00a5fe0 0x7fb4f009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f5aa5640 10 monclient(hunting): get_auth_request con 0x7fb4f00a5fe0 auth_method 0 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f5aa5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f5aa5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f5aa5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:56.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f5aa5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f5aa5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f5aa5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 10 monclient(hunting): handle_auth_done global_id 44362 payload 824 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f52a4640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4f00722a0 0x7fb4f01330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 1 -- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb4f00a5fe0 msgr2=0x7fb4f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb4f00a5fe0 0x7fb4f009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 1 -- 172.21.15.190:0/3823824715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4f00722a0 msgr2=0x7fb4f01330a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4f00722a0 0x7fb4f01330a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 10 monclient: _finish_hunting 0 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 1 monclient: found mon.noname-f 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:45:56.010 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:55.997+0000 7fb4f4aa3640 1 -- 172.21.15.190:0/3823824715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f0120860 con 0x7fb4f00a56c0 2024-08-19T10:45:56.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f4aa3640 1 -- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00a56c0 msgr2=0x7fb4f01335e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:56.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f4aa3640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00a56c0 0x7fb4f01335e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000e8d0 tx=0x7fb4e000ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:56.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 1 -- 172.21.15.190:0/3823824715 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4e0017070 con 0x7fb4f00a56c0 2024-08-19T10:45:56.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:56.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:45:56.013 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 10 monclient: dump: 2024-08-19T10:45:56.013 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:56.013 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:56.013 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:56.014 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 1 -- 172.21.15.190:0/3823824715 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e0013470 con 0x7fb4f00a56c0 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 1 -- 172.21.15.190:0/3823824715 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb4e0013a90 con 0x7fb4f00a56c0 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f7508640 5 monclient: authenticate success, global_id 44362 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e77fe640 4 set_mon_vals no callback set 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 10 monclient: dump: 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:56.015 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:56.016 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:56.017 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:56.017 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:56.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f7508640 10 monclient: _renew_subs 2024-08-19T10:45:56.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f7508640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:56.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f7508640 1 -- 172.21.15.190:0/3823824715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f0120b40 con 0x7fb4f00a56c0 2024-08-19T10:45:56.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f7508640 10 monclient: _renew_subs 2024-08-19T10:45:56.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f7508640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:56.017 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f7508640 1 -- 172.21.15.190:0/3823824715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f0120580 con 0x7fb4f00a56c0 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f7508640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f7508640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f7508640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f7508640 1 -- 172.21.15.190:0/3823824715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f00a1670 con 0x7fb4f00a56c0 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 1 -- 172.21.15.190:0/3823824715 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb4e001a070 con 0x7fb4f00a56c0 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb4d403be00 0x7fb4d403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4e5ffb640 1 -- 172.21.15.190:0/3823824715 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(269..269 src has 150..269) ==== 7657+0+0 (secure 0 0 0) 0x7fb4e000f040 con 0x7fb4f00a56c0 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f52a4640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb4d403be00 0x7fb4d403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.001+0000 7fb4f52a4640 10 monclient: get_auth_request con 0x7fb4d403be00 auth_method 0 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.005+0000 7fb4f52a4640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb4d403be00 0x7fb4d403e2c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009870 tx=0x7fb4ec0a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.005+0000 7fb4e5ffb640 1 -- 172.21.15.190:0/3823824715 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb4e0021070 con 0x7fb4f00a56c0 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.005+0000 7fb4e5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:56.018 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.005+0000 7fb4e5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:56.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.185+0000 7fb4f7508640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["14"]}] 2024-08-19T10:45:56.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.185+0000 7fb4f7508640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["14"]}] 2024-08-19T10:45:56.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.185+0000 7fb4f7508640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:56.188 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.185+0000 7fb4f7508640 1 -- 172.21.15.190:0/3823824715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["14"]} v 0) -- 0x7fb4f006fd10 con 0x7fb4f00a56c0 2024-08-19T10:45:56.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:45:56.188+0000 7f16cc065640 -1 mon.a@0(leader).osd e269 definitely_dead 0 2024-08-19T10:45:56.467 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.465+0000 7fb4e5ffb640 1 -- 172.21.15.190:0/3823824715 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["14"]}]=0 removed osd.14 v270) ==== 83+0+0 (secure 0 0 0) 0x7fb4e000cb50 con 0x7fb4f00a56c0 2024-08-19T10:45:56.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.465+0000 7fb4e5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["14"]}] 2024-08-19T10:45:56.468 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.465+0000 7fb4e5ffb640 10 monclient: _finish_command 2 = system:0 removed osd.14 2024-08-19T10:45:56.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.581+0000 7fb4f7508640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["14"]}] 2024-08-19T10:45:56.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.581+0000 7fb4f7508640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["14"]}] 2024-08-19T10:45:56.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.581+0000 7fb4f7508640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:56.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.581+0000 7fb4f7508640 1 -- 172.21.15.190:0/3823824715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd rm", "ids": ["14"]} v 0) -- 0x7fb4f0133b20 con 0x7fb4f00a56c0 2024-08-19T10:45:56.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:45:56.584+0000 7f16cc065640 -1 mon.a@0(leader).osd e270 definitely_dead 0 2024-08-19T10:45:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4e5ffb640 1 -- 172.21.15.190:0/3823824715 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["14"]}]=0 osd.14 does not exist. v270) ==== 92+0+0 (secure 0 0 0) 0x7fb4e001ac80 con 0x7fb4f00a56c0 2024-08-19T10:45:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4e5ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["14"]}] 2024-08-19T10:45:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4e5ffb640 10 monclient: _finish_command 3 = system:0 osd.14 does not exist. 2024-08-19T10:45:56.585 INFO:tasks.workunit.client.0.smithi190.stderr:osd.14 does not exist. 2024-08-19T10:45:56.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 -- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb4d403be00 msgr2=0x7fb4d403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:56.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb4d403be00 0x7fb4d403e2c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009870 tx=0x7fb4ec0a6070 comp rx=0 tx=0).stop 2024-08-19T10:45:56.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 10 monclient: shutdown 2024-08-19T10:45:56.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:56.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 -- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00a56c0 msgr2=0x7fb4f01335e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:56.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00a56c0 0x7fb4f01335e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000e8d0 tx=0x7fb4e000ed90 comp rx=0 tx=0).stop 2024-08-19T10:45:56.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 -- 172.21.15.190:0/3823824715 shutdown_connections 2024-08-19T10:45:56.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb4d403be00 0x7fb4d403e2c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb4f00a5fe0 0x7fb4f009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4f00a56c0 0x7fb4f01335e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 --2- 172.21.15.190:0/3823824715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4f00722a0 0x7fb4f01330a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 -- 172.21.15.190:0/3823824715 >> 172.21.15.190:0/3823824715 conn(0x7fb4f00aa2b0 msgr2=0x7fb4f013cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:56.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 -- 172.21.15.190:0/3823824715 shutdown_connections 2024-08-19T10:45:56.588 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.585+0000 7fb4f7508640 1 -- 172.21.15.190:0/3823824715 wait complete. 2024-08-19T10:45:56.602 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd rm 1000 2024-08-19T10:45:56.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient: get_monmap_and_config 2024-08-19T10:45:56.716 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient: build_initial_monmap 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient: monmap: 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:56.716087+0000 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:56.716087+0000 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:56.717 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:56.718 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:56.718 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:56.718 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:56.718 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:56.718 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:56.718 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:56.718 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:56.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 1 Processor -- start 2024-08-19T10:45:56.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 1 -- start start 2024-08-19T10:45:56.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient: init 2024-08-19T10:45:56.718 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient: _add_conns ranks=[2,7,9,10,4,8,0,3,5,1,6] 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5680a3d30 0x7fc568136570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient(hunting): picked mon.noname-b con 0x7fc5680a3d30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc568136ab0 0x7fc568136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient(hunting): picked mon.noname-i con 0x7fc568136ab0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc5680a4110 0x7fc5680a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient(hunting): picked mon.noname-a con 0x7fc5680a4110 addr v2:172.21.15.89:3300/0 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fc5681373f0 con 0x7fc5680a4110 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc5680a0840 con 0x7fc5680a3d30 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:56.719 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc5680a0ac0 con 0x7fc568136ab0 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc570410640 10 monclient(hunting): authenticate will time out at 1402.285734s 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc568136ab0 0x7fc568136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc568136ab0 0x7fc568136eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:53410/0 (socket says 172.21.15.190:53410) 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 1 -- 172.21.15.190:0/3992617009 learned_addr learned my addr 172.21.15.190:0/3992617009 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 10 monclient(hunting): get_auth_request con 0x7fc568136ab0 auth_method 0 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56e1ac640 1 -- 172.21.15.190:0/3992617009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5680a3d30 msgr2=0x7fc568136570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56e1ac640 1 --2- 172.21.15.190:0/3992617009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5680a3d30 0x7fc568136570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:56.720 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56e9ad640 1 --2- 172.21.15.190:0/3992617009 >> v2:172.21.15.89:3300/0 conn(0x7fc5680a4110 0x7fc5680a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 10 monclient(hunting): handle_auth_done global_id 15874 payload 274 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 1 -- 172.21.15.190:0/3992617009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5680a3d30 msgr2=0x7fc568136570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 1 --2- 172.21.15.190:0/3992617009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5680a3d30 0x7fc568136570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 1 -- 172.21.15.190:0/3992617009 >> v2:172.21.15.89:3300/0 conn(0x7fc5680a4110 msgr2=0x7fc5680a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 1 --2- 172.21.15.190:0/3992617009 >> v2:172.21.15.89:3300/0 conn(0x7fc5680a4110 0x7fc5680a45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 10 monclient: _finish_hunting 0 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 1 monclient: found mon.noname-i 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:56.721 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.713+0000 7fc56d9ab640 1 -- 172.21.15.190:0/3992617009 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5680a1b60 con 0x7fc568136ab0 2024-08-19T10:45:56.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d9ab640 1 -- 172.21.15.190:0/3992617009 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc568136ab0 msgr2=0x7fc568136eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:56.726 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d9ab640 1 --2- 172.21.15.190:0/3992617009 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc568136ab0 0x7fc568136eb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc560009e50 tx=0x7fc56001c910 comp rx=0 tx=0).ready entity=mon.7 client_cookie=b7ffef22a75cd30c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 1 -- 172.21.15.190:0/3992617009 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5600ac020 con 0x7fc568136ab0 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 10 monclient: dump: 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:56.727 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 10 monclient: _finish_auth 0 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 10 monclient: _check_auth_tickets 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 1 -- 172.21.15.190:0/3992617009 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5600091f0 con 0x7fc568136ab0 2024-08-19T10:45:56.728 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 5 monclient: authenticate success, global_id 15874 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 1 -- 172.21.15.190:0/3992617009 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5600a6d40 con 0x7fc568136ab0 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 4 set_mon_vals no callback set 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc56d1aa640 10 monclient: dump: 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.729 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:56.730 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 10 monclient: shutdown 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 1 -- 172.21.15.190:0/3992617009 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc568136ab0 msgr2=0x7fc568136eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 1 --2- 172.21.15.190:0/3992617009 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc568136ab0 0x7fc568136eb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc560009e50 tx=0x7fc56001c910 comp rx=0 tx=0).stop 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 1 -- 172.21.15.190:0/3992617009 shutdown_connections 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 1 --2- 172.21.15.190:0/3992617009 >> v2:172.21.15.89:3300/0 conn(0x7fc5680a4110 0x7fc5680a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 1 --2- 172.21.15.190:0/3992617009 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc568136ab0 0x7fc568136eb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 1 --2- 172.21.15.190:0/3992617009 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5680a3d30 0x7fc568136570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 1 -- 172.21.15.190:0/3992617009 >> 172.21.15.190:0/3992617009 conn(0x7fc56806dde0 msgr2=0x7fc56806e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 1 -- 172.21.15.190:0/3992617009 shutdown_connections 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 1 -- 172.21.15.190:0/3992617009 wait complete. 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 10 monclient: build_initial_monmap 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.717+0000 7fc570410640 10 monclient: monmap: 2024-08-19T10:45:56.731 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:56.732 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 1 Processor -- start 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 1 -- start start 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient: init 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient: _add_conns ranks=[4,7,0,8,2,5,1,3,6] 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5680a3d30 0x7fc56813a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient(hunting): picked mon.noname-c con 0x7fc5680a3d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5680a4110 0x7fc56813a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient(hunting): picked mon.noname-i con 0x7fc5680a4110 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc568136ab0 0x7fc56813adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:56.733 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient(hunting): picked mon.noname-a con 0x7fc568136ab0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc5680a4ef0 con 0x7fc568136ab0 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc56806d0e0 con 0x7fc5680a3d30 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc568077980 con 0x7fc5680a4110 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient(hunting): authenticate will time out at 1402.292834s 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5680a4110 0x7fc56813a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5680a4110 0x7fc56813a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:53424/0 (socket says 172.21.15.190:53424) 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 -- 172.21.15.190:0/1433724669 learned_addr learned my addr 172.21.15.190:0/1433724669 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 10 monclient(hunting): get_auth_request con 0x7fc5680a4110 auth_method 0 2024-08-19T10:45:56.734 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 10 monclient(hunting): handle_auth_done global_id 15883 payload 824 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 -- 172.21.15.190:0/1433724669 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5680a3d30 msgr2=0x7fc56813a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5680a3d30 0x7fc56813a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 -- 172.21.15.190:0/1433724669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc568136ab0 msgr2=0x7fc56813adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc568136ab0 0x7fc56813adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 10 monclient: _finish_hunting 0 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 monclient: found mon.noname-i 2024-08-19T10:45:56.735 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 -- 172.21.15.190:0/1433724669 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56813bc90 con 0x7fc5680a4110 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 -- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5680a4110 msgr2=0x7fc56813a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5680a4110 0x7fc56813a8b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc56001d1b0 tx=0x7fc5600196b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56d9ab640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5680a4110 0x7fc56813a8b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc56001d1b0 tx=0x7fc5600196b0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fc5600a68d0 mon_map magic: 0 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc55affd640 1 -- 172.21.15.190:0/1433724669 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5600ac040 con 0x7fc5680a4110 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc55affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc55affd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc55affd640 10 monclient: dump: 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.736 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:56.737 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc55affd640 10 monclient: _finish_auth 0 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc55affd640 10 monclient: _check_auth_tickets 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc55affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc55affd640 1 -- 172.21.15.190:0/1433724669 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc560004860 con 0x7fc5680a4110 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 5 monclient: authenticate success, global_id 15883 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient: _renew_subs 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 1 -- 172.21.15.190:0/1433724669 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc568120c40 con 0x7fc5680a4110 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc55affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient: _renew_subs 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc56c9a9640 4 set_mon_vals no callback set 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.721+0000 7fc570410640 1 -- 172.21.15.190:0/1433724669 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fc56811f6d0 con 0x7fc5680a4110 2024-08-19T10:45:56.738 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc570410640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc570410640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc570410640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc570410640 1 -- 172.21.15.190:0/1433724669 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc56806d4a0 con 0x7fc5680a4110 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc55affd640 1 -- 172.21.15.190:0/1433724669 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5600a68d0 con 0x7fc5680a4110 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc55affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc55affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc55affd640 10 monclient: dump: 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:56.739 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc55affd640 1 -- 172.21.15.190:0/1433724669 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc5600af020 con 0x7fc5680a4110 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc55affd640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc54c03be50 0x7fc54c03e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:56.740 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc56e1ac640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc54c03be50 0x7fc54c03e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:56.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc56e1ac640 10 monclient: get_auth_request con 0x7fc54c03be50 auth_method 0 2024-08-19T10:45:56.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.725+0000 7fc56e1ac640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc54c03be50 0x7fc54c03e310 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc564006fd0 tx=0x7fc564006d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:56.741 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:56.729+0000 7fc55affd640 1 -- 172.21.15.190:0/1433724669 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(270..270 src has 150..270) ==== 7657+0+0 (secure 0 0 0) 0x7fc5600ea2d0 con 0x7fc5680a4110 2024-08-19T10:45:57.344 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.345+0000 7fc55affd640 1 -- 172.21.15.190:0/1433724669 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc560016620 con 0x7fc5680a4110 2024-08-19T10:45:57.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.345+0000 7fc55affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:57.345 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.345+0000 7fc55affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.537+0000 7fc570410640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["1000"]}] 2024-08-19T10:45:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.537+0000 7fc570410640 10 monclient: _send_command 2 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-08-19T10:45:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.537+0000 7fc570410640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:57.538 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.537+0000 7fc570410640 1 -- 172.21.15.190:0/1433724669 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd rm", "ids": ["1000"]} v 0) -- 0x7fc568120010 con 0x7fc5680a4110 2024-08-19T10:45:57.539 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:45:57.540+0000 7f16cc065640 -1 mon.a@0(leader).osd e270 definitely_dead 0 2024-08-19T10:45:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.537+0000 7fc55affd640 1 -- 172.21.15.190:0/1433724669 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["1000"]}]=0 osd.1000 does not exist. v270) ==== 96+0+0 (secure 0 0 0) 0x7fc5600b9e20 con 0x7fc5680a4110 2024-08-19T10:45:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.537+0000 7fc55affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-08-19T10:45:57.539 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.537+0000 7fc55affd640 10 monclient: _finish_command 2 = system:0 osd.1000 does not exist. 2024-08-19T10:45:57.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.653+0000 7fc570410640 10 start_mon_command cmd=[{"prefix": "osd rm", "ids": ["1000"]}] 2024-08-19T10:45:57.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.653+0000 7fc570410640 10 monclient: _send_command 3 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-08-19T10:45:57.652 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.653+0000 7fc570410640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:45:57.653 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.653+0000 7fc570410640 1 -- 172.21.15.190:0/1433724669 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "osd rm", "ids": ["1000"]} v 0) -- 0x7fc5680008e0 con 0x7fc5680a4110 2024-08-19T10:45:57.653 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:45:57.652+0000 7f16cc065640 -1 mon.a@0(leader).osd e270 definitely_dead 0 2024-08-19T10:45:57.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.653+0000 7fc55affd640 1 -- 172.21.15.190:0/1433724669 <== mon.7 v2:172.21.15.190:3303/0 8 ==== mon_command_ack([{"prefix": "osd rm", "ids": ["1000"]}]=0 osd.1000 does not exist. v270) ==== 96+0+0 (secure 0 0 0) 0x7fc5600e8190 con 0x7fc5680a4110 2024-08-19T10:45:57.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.653+0000 7fc55affd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd rm", "ids": ["1000"]}] 2024-08-19T10:45:57.654 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.653+0000 7fc55affd640 10 monclient: _finish_command 3 = system:0 osd.1000 does not exist. 2024-08-19T10:45:57.654 INFO:tasks.workunit.client.0.smithi190.stderr:osd.1000 does not exist. 2024-08-19T10:45:57.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 -- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc54c03be50 msgr2=0x7fc54c03e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:57.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc54c03be50 0x7fc54c03e310 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc564006fd0 tx=0x7fc564006d00 comp rx=0 tx=0).stop 2024-08-19T10:45:57.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 10 monclient: shutdown 2024-08-19T10:45:57.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:57.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 -- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5680a4110 msgr2=0x7fc56813a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:57.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5680a4110 0x7fc56813a8b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc56001d1b0 tx=0x7fc5600196b0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 -- 172.21.15.190:0/1433724669 shutdown_connections 2024-08-19T10:45:57.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc54c03be50 0x7fc54c03e310 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc568136ab0 0x7fc56813adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.657 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5680a4110 0x7fc56813a8b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 --2- 172.21.15.190:0/1433724669 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5680a3d30 0x7fc56813a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 -- 172.21.15.190:0/1433724669 >> 172.21.15.190:0/1433724669 conn(0x7fc56806dde0 msgr2=0x7fc5680a7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:57.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 -- 172.21.15.190:0/1433724669 shutdown_connections 2024-08-19T10:45:57.658 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.657+0000 7fc570410640 1 -- 172.21.15.190:0/1433724669 wait complete. 2024-08-19T10:45:57.672 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd create 81c92c7b-73dd-42d5-ae6d-11d899af5fe3 2024-08-19T10:45:57.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient: get_monmap_and_config 2024-08-19T10:45:57.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient: build_initial_monmap 2024-08-19T10:45:57.788 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient: monmap: 2024-08-19T10:45:57.788 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:57.788 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:57.791722+0000 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:57.791722+0000 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:57.789 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:57.790 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:57.790 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:57.790 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:57.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 1 Processor -- start 2024-08-19T10:45:57.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 1 -- start start 2024-08-19T10:45:57.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient: init 2024-08-19T10:45:57.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:57.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient: _add_conns ranks=[10,3,6,9,1,5,0,7,2,4,8] 2024-08-19T10:45:57.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0bc80a3d30 0x7f0bc8136570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:57.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient(hunting): picked mon.noname-f con 0x7f0bc80a3d30 addr v2:172.21.15.190:3300/0 2024-08-19T10:45:57.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc8136ab0 0x7f0bc8136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:57.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient(hunting): picked mon.noname-g con 0x7f0bc8136ab0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:57.792 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0bc80a4110 0x7f0bc80a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient(hunting): picked mon.noname-d con 0x7f0bc80a4110 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc81373f0 con 0x7f0bc80a3d30 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc80a0840 con 0x7f0bc8136ab0 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f0bc80a0ac0 con 0x7f0bc80a4110 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bcd3fe640 10 monclient(hunting): authenticate will time out at 1403.361599s 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc8136ab0 0x7f0bc8136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc6ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0bc80a3d30 0x7f0bc8136570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc8136ab0 0x7f0bc8136eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47258/0 (socket says 172.21.15.190:47258) 2024-08-19T10:45:57.793 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 1 -- 172.21.15.190:0/588785567 learned_addr learned my addr 172.21.15.190:0/588785567 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 10 monclient(hunting): get_auth_request con 0x7f0bc8136ab0 auth_method 0 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc6ffd640 10 monclient(hunting): get_auth_request con 0x7f0bc80a3d30 auth_method 0 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:57.794 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc77fe640 1 --2- 172.21.15.190:0/588785567 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0bc80a4110 0x7f0bc80a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 10 monclient(hunting): handle_auth_done global_id 35031 payload 274 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 1 -- 172.21.15.190:0/588785567 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0bc80a4110 msgr2=0x7f0bc80a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 1 --2- 172.21.15.190:0/588785567 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0bc80a4110 0x7f0bc80a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 1 -- 172.21.15.190:0/588785567 >> v2:172.21.15.190:3300/0 conn(0x7f0bc80a3d30 msgr2=0x7f0bc8136570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 1 --2- 172.21.15.190:0/588785567 >> v2:172.21.15.190:3300/0 conn(0x7f0bc80a3d30 0x7f0bc8136570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 10 monclient: _finish_hunting 0 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 1 monclient: found mon.noname-g 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:57.795 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc67fc640 1 -- 172.21.15.190:0/588785567 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc80a1b60 con 0x7f0bc8136ab0 2024-08-19T10:45:57.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc6ffd640 1 --2- 172.21.15.190:0/588785567 >> v2:172.21.15.190:3300/0 conn(0x7f0bc80a3d30 0x7f0bc8136570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:45:57.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc77fe640 10 monclient: get_auth_request con 0x7f0bc80a4110 auth_method 0 2024-08-19T10:45:57.796 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.789+0000 7f0bc77fe640 1 --2- 172.21.15.190:0/588785567 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0bc80a4110 0x7f0bc80a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:45:57.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc67fc640 1 -- 172.21.15.190:0/588785567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc8136ab0 msgr2=0x7f0bc8136eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:57.801 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc67fc640 1 --2- 172.21.15.190:0/588785567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc8136ab0 0x7f0bc8136eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009e50 tx=0x7f0bbc01c8f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c15db853da6d5978 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 1 -- 172.21.15.190:0/588785567 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0bbc0090d0 con 0x7f0bc8136ab0 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 10 monclient: dump: 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:57.802 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:57.803 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:57.803 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:57.803 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:57.803 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:57.803 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:57.803 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:57.803 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:57.803 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:57.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:57.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:57.803 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:57.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 5 monclient: authenticate success, global_id 35031 2024-08-19T10:45:57.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 1 -- 172.21.15.190:0/588785567 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bbc0a59a0 con 0x7f0bc8136ab0 2024-08-19T10:45:57.804 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:45:57.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:57.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:57.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 4 set_mon_vals no callback set 2024-08-19T10:45:57.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 1 -- 172.21.15.190:0/588785567 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0bbc0ad3f0 con 0x7f0bc8136ab0 2024-08-19T10:45:57.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:57.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:57.805 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bc5ffb640 10 monclient: dump: 2024-08-19T10:45:57.805 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:57.805 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:57.805 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:57.805 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 10 monclient: shutdown 2024-08-19T10:45:57.806 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:57.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/588785567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc8136ab0 msgr2=0x7f0bc8136eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:57.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 1 --2- 172.21.15.190:0/588785567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc8136ab0 0x7f0bc8136eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009e50 tx=0x7f0bbc01c8f0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/588785567 shutdown_connections 2024-08-19T10:45:57.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 1 --2- 172.21.15.190:0/588785567 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0bc80a4110 0x7f0bc80a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 1 --2- 172.21.15.190:0/588785567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc8136ab0 0x7f0bc8136eb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 1 --2- 172.21.15.190:0/588785567 >> v2:172.21.15.190:3300/0 conn(0x7f0bc80a3d30 0x7f0bc8136570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/588785567 >> 172.21.15.190:0/588785567 conn(0x7f0bc806dde0 msgr2=0x7f0bc806e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:57.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/588785567 shutdown_connections 2024-08-19T10:45:57.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/588785567 wait complete. 2024-08-19T10:45:57.807 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 10 monclient: build_initial_monmap 2024-08-19T10:45:57.808 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.793+0000 7f0bcd3fe640 10 monclient: monmap: 2024-08-19T10:45:57.808 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:57.808 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:57.808 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:57.808 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:57.808 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:57.808 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:57.808 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:57.808 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:57.808 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:57.808 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 1 Processor -- start 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 1 -- start start 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient: init 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient: _add_conns ranks=[1,0,5,4,7,3,8,6,2] 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc80a3d30 0x7f0bc811f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient(hunting): picked mon.noname-f con 0x7f0bc80a3d30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:57.809 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc80a4110 0x7f0bc811f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient(hunting): picked mon.noname-a con 0x7f0bc80a4110 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0bc8136ab0 0x7f0bc811fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient(hunting): picked mon.noname-h con 0x7f0bc8136ab0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc80a4ef0 con 0x7f0bc80a4110 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc813ffe0 con 0x7f0bc80a3d30 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0bc80a6270 con 0x7f0bc8136ab0 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient(hunting): authenticate will time out at 1403.368789s 2024-08-19T10:45:57.810 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc80a3d30 0x7f0bc811f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:57.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc80a3d30 0x7f0bc811f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:44554/0 (socket says 172.21.15.190:44554) 2024-08-19T10:45:57.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 1 -- 172.21.15.190:0/2499332814 learned_addr learned my addr 172.21.15.190:0/2499332814 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:57.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 10 monclient(hunting): get_auth_request con 0x7f0bc80a3d30 auth_method 0 2024-08-19T10:45:57.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:57.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:57.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:57.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc77fe640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0bc8136ab0 0x7f0bc811fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:57.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc77fe640 10 monclient(hunting): get_auth_request con 0x7f0bc8136ab0 auth_method 0 2024-08-19T10:45:57.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:57.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:57.811 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:57.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:57.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:57.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:57.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:57.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:57.812 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 10 monclient(hunting): handle_auth_done global_id 44380 payload 824 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 1 -- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0bc8136ab0 msgr2=0x7f0bc811fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0bc8136ab0 0x7f0bc811fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 1 -- 172.21.15.190:0/2499332814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc80a4110 msgr2=0x7f0bc811f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc80a4110 0x7f0bc811f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 1 monclient: found mon.noname-f 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 1 -- 172.21.15.190:0/2499332814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc8121f50 con 0x7f0bc80a3d30 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 1 -- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc80a3d30 msgr2=0x7f0bc811f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:57.813 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc6ffd640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc80a3d30 0x7f0bc811f400 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000ebe0 tx=0x7f0bc000c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:57.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 1 -- 172.21.15.190:0/2499332814 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0bc0017070 con 0x7f0bc80a3d30 2024-08-19T10:45:57.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:57.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:45:57.814 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 10 monclient: dump: 2024-08-19T10:45:57.814 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:57.814 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:57.814 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:57.814 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:57.814 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:45:57.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 5 monclient: authenticate success, global_id 44380 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 1 -- 172.21.15.190:0/2499332814 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc000ce40 con 0x7f0bc80a3d30 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient: _renew_subs 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/2499332814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc80a7de0 con 0x7f0bc80a3d30 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 1 -- 172.21.15.190:0/2499332814 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0bc0013d50 con 0x7f0bc80a3d30 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0baf7fe640 10 monclient: dump: 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:57.816 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:57.817 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bc57fa640 4 set_mon_vals no callback set 2024-08-19T10:45:57.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient: _renew_subs 2024-08-19T10:45:57.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:57.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.797+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/2499332814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc813cfa0 con 0x7f0bc80a3d30 2024-08-19T10:45:57.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.801+0000 7f0bcd3fe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:57.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.801+0000 7f0bcd3fe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:57.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.801+0000 7f0bcd3fe640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:57.818 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.801+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/2499332814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc80a45a0 con 0x7f0bc80a3d30 2024-08-19T10:45:57.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.801+0000 7f0baf7fe640 1 -- 172.21.15.190:0/2499332814 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0bc001a070 con 0x7f0bc80a3d30 2024-08-19T10:45:57.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.801+0000 7f0baf7fe640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0bb403be30 0x7f0bb403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:57.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.801+0000 7f0baf7fe640 1 -- 172.21.15.190:0/2499332814 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(270..270 src has 150..270) ==== 7657+0+0 (secure 0 0 0) 0x7f0bc000f040 con 0x7f0bc80a3d30 2024-08-19T10:45:57.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.801+0000 7f0bc67fc640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0bb403be30 0x7f0bb403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:57.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.801+0000 7f0bc67fc640 10 monclient: get_auth_request con 0x7f0bb403be30 auth_method 0 2024-08-19T10:45:57.819 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.801+0000 7f0bc67fc640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0bb403be30 0x7f0bb403e2f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0bc809fd90 tx=0x7f0bbc004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:57.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.805+0000 7f0baf7fe640 1 -- 172.21.15.190:0/2499332814 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0bc0021070 con 0x7f0bc80a3d30 2024-08-19T10:45:57.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.805+0000 7f0baf7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:57.820 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.805+0000 7f0baf7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:45:57.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.981+0000 7f0bcd3fe640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:57.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.981+0000 7f0bcd3fe640 10 monclient: _send_command 2 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:57.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.981+0000 7f0bcd3fe640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:57.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:57.981+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/2499332814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"} v 0) -- 0x7f0bc8121d30 con 0x7f0bc80a3d30 2024-08-19T10:45:58.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.597+0000 7f0baf7fe640 1 -- 172.21.15.190:0/2499332814 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}]=0 v271) ==== 106+0+3 (secure 0 0 0) 0x7f0bc001ac80 con 0x7f0bc80a3d30 2024-08-19T10:45:58.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.597+0000 7f0baf7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:58.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.597+0000 7f0baf7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:45:58.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.709+0000 7f0bcd3fe640 10 start_mon_command cmd=[{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:58.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.709+0000 7f0bcd3fe640 10 monclient: _send_command 3 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:58.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.709+0000 7f0bcd3fe640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:45:58.709 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.709+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/2499332814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"} v 0) -- 0x7f0bc809f400 con 0x7f0bc80a3d30 2024-08-19T10:45:58.976 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.977+0000 7f0baf7fe640 1 -- 172.21.15.190:0/2499332814 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}]=0 v271) ==== 106+0+3 (secure 0 0 0) 0x7f0bc001b3b0 con 0x7f0bc80a3d30 2024-08-19T10:45:58.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.977+0000 7f0baf7fe640 10 monclient: handle_mon_command_ack 3 [{"prefix": "osd create", "uuid": "81c92c7b-73dd-42d5-ae6d-11d899af5fe3"}] 2024-08-19T10:45:58.977 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.977+0000 7f0baf7fe640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:45:58.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0bb403be30 msgr2=0x7f0bb403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:58.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0bb403be30 0x7f0bb403e2f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0bc809fd90 tx=0x7f0bbc004ac0 comp rx=0 tx=0).stop 2024-08-19T10:45:58.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 10 monclient: shutdown 2024-08-19T10:45:58.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:58.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc80a3d30 msgr2=0x7f0bc811f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:58.981 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc80a3d30 0x7f0bc811f400 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000ebe0 tx=0x7f0bc000c520 comp rx=0 tx=0).stop 2024-08-19T10:45:58.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/2499332814 shutdown_connections 2024-08-19T10:45:58.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0bb403be30 0x7f0bb403e2f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:58.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0bc8136ab0 0x7f0bc811fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:58.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bc80a4110 0x7f0bc811f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:58.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 --2- 172.21.15.190:0/2499332814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc80a3d30 0x7f0bc811f400 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:58.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/2499332814 >> 172.21.15.190:0/2499332814 conn(0x7f0bc806dde0 msgr2=0x7f0bc8072e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:58.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/2499332814 shutdown_connections 2024-08-19T10:45:58.982 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:58.981+0000 7f0bcd3fe640 1 -- 172.21.15.190:0/2499332814 wait complete. 2024-08-19T10:45:59.003 INFO:tasks.workunit.client.0.smithi190.stdout:OK 2024-08-19T10:45:59.004 INFO:tasks.workunit.client.0.smithi190.stderr:+ na2=12 2024-08-19T10:45:59.004 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo OK 2024-08-19T10:45:59.004 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-19T10:45:59.004 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-19T10:45:59.066 INFO:tasks.workunit:Running workunits matching mon/caps.sh on client.0... 2024-08-19T10:45:59.067 INFO:tasks.workunit:Running workunit mon/caps.sh... 2024-08-19T10:45:59.067 DEBUG:teuthology.orchestra.run.smithi190:workunit test mon/caps.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=964adda23da9f7b057cb34fdc0b82aeb7ed65324 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh 2024-08-19T10:45:59.118 INFO:tasks.workunit.client.0.smithi190.stderr:+ tmp=/tmp/cephtest-mon-caps-madness 2024-08-19T10:45:59.118 INFO:tasks.workunit.client.0.smithi190.stderr:+ exit_on_error=1 2024-08-19T10:45:59.118 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ ! -z '' ]] 2024-08-19T10:45:59.118 INFO:tasks.workunit.client.0.smithi190.stderr:++ uname 2024-08-19T10:45:59.119 INFO:tasks.workunit.client.0.smithi190.stderr:+ '[' Linux = FreeBSD ']' 2024-08-19T10:45:59.119 INFO:tasks.workunit.client.0.smithi190.stderr:+ ETIMEDOUT=110 2024-08-19T10:45:59.119 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring' 0 2024-08-19T10:45:59.119 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring' 2024-08-19T10:45:59.119 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=0 2024-08-19T10:45:59.120 INFO:tasks.workunit.client.0.smithi190.stdout:ceph auth get-or-create client.bazar > /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-08-19T10:45:59.120 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph auth get-or-create client.bazar '>' /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-08-19T10:45:59.120 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph auth get-or-create client.bazar '>' /tmp/cephtest-mon-caps-madness.bazar.keyring 2024-08-19T10:45:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=0 2024-08-19T10:45:59.664 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 0 -ne 0 ]] 2024-08-19T10:45:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:45:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status' 13 2024-08-19T10:45:59.665 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-08-19T10:45:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status' 2024-08-19T10:45:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=13 2024-08-19T10:45:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-08-19T10:45:59.665 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bazar.keyring --user bazar quorum_status 2024-08-19T10:45:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=13 2024-08-19T10:45:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 13 -ne 13 ]] 2024-08-19T10:45:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:45:59.783 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph auth del client.bazar 2024-08-19T10:45:59.893 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient: get_monmap_and_config 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient: build_initial_monmap 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient: monmap: 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:45:59.896961+0000 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:45:59.896961+0000 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:59.894 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:59.895 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:59.895 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:59.895 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:59.895 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:59.895 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:59.895 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:45:59.895 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:45:59.895 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:59.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 1 Processor -- start 2024-08-19T10:45:59.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 1 -- start start 2024-08-19T10:45:59.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient: init 2024-08-19T10:45:59.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:59.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient: _add_conns ranks=[7,0,4,6,5,9,8,3,10,1,2] 2024-08-19T10:45:59.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd35c1399d0 0x7fd35c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:59.897 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient(hunting): picked mon.noname-i con 0x7fd35c1399d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd35c139db0 addr v1:172.21.15.89:6789/0 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd35c13a190 0x7fd35c0a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient(hunting): picked mon.noname-c con 0x7fd35c13a190 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd35c0a46e0 con 0x7fd35c139db0 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd35c0a3470 con 0x7fd35c13a190 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd35c0a49c0 con 0x7fd35c1399d0 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd363952640 10 monclient(hunting): authenticate will time out at 1405.466893s 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd35c1399d0 0x7fd35c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:59.898 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd35c1399d0 0x7fd35c0a5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.190:53436/0 (socket says 172.21.15.190:53436) 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 1 -- 172.21.15.190:0/2730006248 learned_addr learned my addr 172.21.15.190:0/2730006248 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 10 monclient(hunting): get_auth_request con 0x7fd35c1399d0 auth_method 0 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd361eef640 1 --2- 172.21.15.190:0/2730006248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd35c13a190 0x7fd35c0a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd361eef640 10 monclient(hunting): get_auth_request con 0x7fd35c13a190 auth_method 0 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd361eef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd361eef640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:59.899 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd361eef640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:59.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 10 monclient(hunting): handle_auth_done global_id 15901 payload 274 2024-08-19T10:45:59.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 1 -- 172.21.15.190:0/2730006248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd35c13a190 msgr2=0x7fd35c0a6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:59.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 1 --2- 172.21.15.190:0/2730006248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd35c13a190 0x7fd35c0a6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:59.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 1 -- 172.21.15.190:0/2730006248 >> v1:172.21.15.89:6789/0 conn(0x7fd35c139db0 legacy=0x7fd35c0a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:45:59.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 10 monclient: _finish_hunting 0 2024-08-19T10:45:59.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 1 monclient: found mon.noname-i 2024-08-19T10:45:59.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:59.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:45:59.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd3616ee640 1 -- 172.21.15.190:0/2730006248 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c0a4ca0 con 0x7fd35c1399d0 2024-08-19T10:45:59.900 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.893+0000 7fd361eef640 1 --2- 172.21.15.190:0/2730006248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd35c13a190 0x7fd35c0a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:45:59.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.897+0000 7fd3616ee640 1 -- 172.21.15.190:0/2730006248 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd35c1399d0 msgr2=0x7fd35c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:59.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.897+0000 7fd3616ee640 1 --2- 172.21.15.190:0/2730006248 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd35c1399d0 0x7fd35c0a5790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009e80 tx=0x7fd34c01c910 comp rx=0 tx=0).ready entity=mon.7 client_cookie=4250ec6a9461ec8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:59.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.897+0000 7fd35bfff640 1 -- 172.21.15.190:0/2730006248 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd34c0090d0 con 0x7fd35c1399d0 2024-08-19T10:45:59.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.897+0000 7fd35bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:59.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.897+0000 7fd35bfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:45:59.904 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.897+0000 7fd35bfff640 10 monclient: dump: 2024-08-19T10:45:59.904 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:59.904 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:59.905 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:59.906 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:59.906 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:59.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.897+0000 7fd35bfff640 10 monclient: _finish_auth 0 2024-08-19T10:45:59.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.897+0000 7fd35bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:45:59.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd35bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:59.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd35bfff640 1 -- 172.21.15.190:0/2730006248 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c0a5b80 con 0x7fd35c1399d0 2024-08-19T10:45:59.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd35bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:59.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd35bfff640 1 -- 172.21.15.190:0/2730006248 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd34c0a5d30 con 0x7fd35c1399d0 2024-08-19T10:45:59.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 5 monclient: authenticate success, global_id 15901 2024-08-19T10:45:59.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient: get_monmap_and_config success 2024-08-19T10:45:59.906 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 4 set_mon_vals no callback set 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd35bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd35bfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd35bfff640 10 monclient: dump: 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:59.907 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient: shutdown 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 -- 172.21.15.190:0/2730006248 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd35c1399d0 msgr2=0x7fd35c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 --2- 172.21.15.190:0/2730006248 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd35c1399d0 0x7fd35c0a5790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009e80 tx=0x7fd34c01c910 comp rx=0 tx=0).stop 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 -- 172.21.15.190:0/2730006248 shutdown_connections 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 --2- 172.21.15.190:0/2730006248 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd35c13a190 0x7fd35c0a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:59.908 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 --2- 172.21.15.190:0/2730006248 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd35c1399d0 0x7fd35c0a5790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:59.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 -- 172.21.15.190:0/2730006248 >> 172.21.15.190:0/2730006248 conn(0x7fd35c0abe50 msgr2=0x7fd35c0ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:45:59.909 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 -- 172.21.15.190:0/2730006248 shutdown_connections 2024-08-19T10:45:59.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 -- 172.21.15.190:0/2730006248 wait complete. 2024-08-19T10:45:59.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient: build_initial_monmap 2024-08-19T10:45:59.910 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient: monmap: 2024-08-19T10:45:59.910 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:45:59.910 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:45:59.910 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:45:59.910 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:45:59.910 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:45:59.910 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:59.910 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:45:59.910 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 Processor -- start 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 -- start start 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient: init 2024-08-19T10:45:59.911 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient: _reopen_session rank -1 2024-08-19T10:45:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient: _add_conns ranks=[3,6,1,8,2,0,7,5,4] 2024-08-19T10:45:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35c1399d0 0x7fd35c140920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient(hunting): picked mon.noname-g con 0x7fd35c1399d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:45:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd35c139db0 0x7fd35c140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient(hunting): picked mon.noname-d con 0x7fd35c139db0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:45:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd35c13a190 0x7fd35c13a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient(hunting): picked mon.noname-f con 0x7fd35c13a190 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:45:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c0a7620 con 0x7fd35c13a190 2024-08-19T10:45:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:59.912 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c0a74a0 con 0x7fd35c1399d0 2024-08-19T10:45:59.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient(hunting): start opening mon connection 2024-08-19T10:45:59.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd35c144170 con 0x7fd35c139db0 2024-08-19T10:45:59.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient(hunting): _renew_subs 2024-08-19T10:45:59.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd363952640 10 monclient(hunting): authenticate will time out at 1405.474247s 2024-08-19T10:45:59.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35c1399d0 0x7fd35c140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:59.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd361eef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd35c13a190 0x7fd35c13a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:59.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35c1399d0 0x7fd35c140920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:47292/0 (socket says 172.21.15.190:47292) 2024-08-19T10:45:59.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 1 -- 172.21.15.190:0/698483088 learned_addr learned my addr 172.21.15.190:0/698483088 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:45:59.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 10 monclient(hunting): get_auth_request con 0x7fd35c1399d0 auth_method 0 2024-08-19T10:45:59.913 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:59.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:59.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:59.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd361eef640 10 monclient(hunting): get_auth_request con 0x7fd35c13a190 auth_method 0 2024-08-19T10:45:59.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd361eef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:59.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd361eef640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:59.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd361eef640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:59.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:59.914 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:59.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:59.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd361eef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:45:59.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd361eef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:45:59.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd361eef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:45:59.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd360eed640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd35c139db0 0x7fd35c140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:59.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd360eed640 10 monclient(hunting): get_auth_request con 0x7fd35c139db0 auth_method 0 2024-08-19T10:45:59.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd360eed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:45:59.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd360eed640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:45:59.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd360eed640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:45:59.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 10 monclient(hunting): handle_auth_done global_id 35049 payload 824 2024-08-19T10:45:59.915 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 1 -- 172.21.15.190:0/698483088 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd35c139db0 msgr2=0x7fd35c140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:59.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd35c139db0 0x7fd35c140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:59.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 1 -- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd35c13a190 msgr2=0x7fd35c13a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:45:59.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd35c13a190 0x7fd35c13a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:45:59.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 10 monclient: _finish_hunting 0 2024-08-19T10:45:59.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 1 monclient: found mon.noname-g 2024-08-19T10:45:59.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:45:59.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:45:59.916 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.901+0000 7fd3616ee640 1 -- 172.21.15.190:0/698483088 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c13d3d0 con 0x7fd35c1399d0 2024-08-19T10:45:59.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd3616ee640 1 -- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35c1399d0 msgr2=0x7fd35c140920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:45:59.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd3616ee640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35c1399d0 0x7fd35c140920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0af040 tx=0x7fd34c0196b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:59.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 1 -- 172.21.15.190:0/698483088 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd34c0bc020 con 0x7fd35c1399d0 2024-08-19T10:45:59.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:59.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:45:59.919 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 10 monclient: dump: 2024-08-19T10:45:59.919 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:59.919 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:59.919 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:59.919 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:59.920 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:59.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 10 monclient: _finish_auth 0 2024-08-19T10:45:59.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:45:59.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:45:59.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd363952640 5 monclient: authenticate success, global_id 35049 2024-08-19T10:45:59.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd363952640 10 monclient: _renew_subs 2024-08-19T10:45:59.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 1 -- 172.21.15.190:0/698483088 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c003bb0 con 0x7fd35c1399d0 2024-08-19T10:45:59.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd363952640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:59.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd363952640 1 -- 172.21.15.190:0/698483088 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c1413a0 con 0x7fd35c1399d0 2024-08-19T10:45:59.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:45:59.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 1 -- 172.21.15.190:0/698483088 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd34c003d60 con 0x7fd35c1399d0 2024-08-19T10:45:59.921 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:45:59.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:45:59.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd35affd640 4 set_mon_vals no callback set 2024-08-19T10:45:59.922 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 10 monclient: dump: 2024-08-19T10:45:59.922 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:45:59.922 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:45:59.922 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:59.922 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:45:59.923 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:45:59.923 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:45:59.923 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:45:59.923 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:45:59.923 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:45:59.923 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:45:59.923 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:45:59.923 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:45:59.923 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:45:59.923 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:45:59.924 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:45:59.924 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:45:59.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd363952640 10 monclient: _renew_subs 2024-08-19T10:45:59.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd363952640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:59.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd363952640 1 -- 172.21.15.190:0/698483088 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c0a1910 con 0x7fd35c1399d0 2024-08-19T10:45:59.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd363952640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:59.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd363952640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:59.924 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd363952640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:45:59.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd363952640 1 -- 172.21.15.190:0/698483088 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35c0a5790 con 0x7fd35c1399d0 2024-08-19T10:45:59.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 1 -- 172.21.15.190:0/698483088 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd34c0bf020 con 0x7fd35c1399d0 2024-08-19T10:45:59.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd34803be00 0x7fd34803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:45:59.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd359ffb640 1 -- 172.21.15.190:0/698483088 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fd34c0ea3a0 con 0x7fd35c1399d0 2024-08-19T10:45:59.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd360eed640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd34803be00 0x7fd34803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:45:59.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.905+0000 7fd360eed640 10 monclient: get_auth_request con 0x7fd34803be00 auth_method 0 2024-08-19T10:45:59.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.909+0000 7fd360eed640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd34803be00 0x7fd34803e2c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd35c134cf0 tx=0x7fd350009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:45:59.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.909+0000 7fd359ffb640 1 -- 172.21.15.190:0/698483088 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd34c0c6030 con 0x7fd35c1399d0 2024-08-19T10:45:59.925 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.909+0000 7fd359ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:45:59.926 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:45:59.909+0000 7fd359ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:00.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.145+0000 7fd363952640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bazar"}] 2024-08-19T10:46:00.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.145+0000 7fd363952640 10 monclient: _send_command 2 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-08-19T10:46:00.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.145+0000 7fd363952640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:46:00.145 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.145+0000 7fd363952640 1 -- 172.21.15.190:0/698483088 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bazar"} v 0) -- 0x7fd35c13d050 con 0x7fd35c1399d0 2024-08-19T10:46:00.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.153+0000 7fd359ffb640 1 -- 172.21.15.190:0/698483088 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bazar"}]=0 v9) ==== 82+0+0 (secure 0 0 0) 0x7fd34c016620 con 0x7fd35c1399d0 2024-08-19T10:46:00.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.153+0000 7fd359ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-08-19T10:46:00.152 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.153+0000 7fd359ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:00.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.305+0000 7fd363952640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bazar"}] 2024-08-19T10:46:00.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.305+0000 7fd363952640 10 monclient: _send_command 3 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-08-19T10:46:00.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.305+0000 7fd363952640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:46:00.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.305+0000 7fd363952640 1 -- 172.21.15.190:0/698483088 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bazar"} v 0) -- 0x7fd35c13bbd0 con 0x7fd35c1399d0 2024-08-19T10:46:00.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.305+0000 7fd359ffb640 1 -- 172.21.15.190:0/698483088 <== mon.3 v2:172.21.15.190:3301/0 8 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bazar"}]=0 v9) ==== 82+0+0 (secure 0 0 0) 0x7fd34c0a5b50 con 0x7fd35c1399d0 2024-08-19T10:46:00.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.305+0000 7fd359ffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth del", "entity": "client.bazar"}] 2024-08-19T10:46:00.307 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.305+0000 7fd359ffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:46:00.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 -- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd34803be00 msgr2=0x7fd34803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:00.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd34803be00 0x7fd34803e2c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd35c134cf0 tx=0x7fd350009040 comp rx=0 tx=0).stop 2024-08-19T10:46:00.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 10 monclient: shutdown 2024-08-19T10:46:00.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:00.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 -- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35c1399d0 msgr2=0x7fd35c140920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:00.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35c1399d0 0x7fd35c140920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0af040 tx=0x7fd34c0196b0 comp rx=0 tx=0).stop 2024-08-19T10:46:00.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 -- 172.21.15.190:0/698483088 shutdown_connections 2024-08-19T10:46:00.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd34803be00 0x7fd34803e2c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:00.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd35c13a190 0x7fd35c13a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:00.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd35c139db0 0x7fd35c140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:00.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 --2- 172.21.15.190:0/698483088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd35c1399d0 0x7fd35c140920 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:00.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 -- 172.21.15.190:0/698483088 >> 172.21.15.190:0/698483088 conn(0x7fd35c0abe50 msgr2=0x7fd35c134d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:00.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 -- 172.21.15.190:0/698483088 shutdown_connections 2024-08-19T10:46:00.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:46:00.309+0000 7fd363952640 1 -- 172.21.15.190:0/698483088 wait complete. 2024-08-19T10:46:00.323 INFO:tasks.workunit.client.0.smithi190.stderr:+ c=''\''allow command "auth ls", allow command quorum_status'\''' 2024-08-19T10:46:00.323 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph auth get-or-create client.foo mon '\''allow command "auth ls", allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.foo.keyring' 0 2024-08-19T10:46:00.323 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph auth get-or-create client.foo mon '\''allow command "auth ls", allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.foo.keyring' 2024-08-19T10:46:00.323 INFO:tasks.workunit.client.0.smithi190.stdout:ceph auth get-or-create client.foo mon 'allow command "auth ls", allow command quorum_status' > /tmp/cephtest-mon-caps-madness.foo.keyring 2024-08-19T10:46:00.324 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=0 2024-08-19T10:46:00.324 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph auth get-or-create client.foo mon ''\''allow' command '"auth' 'ls",' allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.foo.keyring 2024-08-19T10:46:00.324 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph auth get-or-create client.foo mon ''\''allow' command '"auth' 'ls",' allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.foo.keyring 2024-08-19T10:46:01.684 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=0 2024-08-19T10:46:01.684 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 0 -ne 0 ]] 2024-08-19T10:46:01.685 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:46:01.685 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 0 2024-08-19T10:46:01.685 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 2024-08-19T10:46:01.685 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=0 2024-08-19T10:46:01.685 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-08-19T10:46:01.685 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-08-19T10:46:01.685 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-08-19T10:46:02.185 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=0 2024-08-19T10:46:02.185 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 0 -ne 0 ]] 2024-08-19T10:46:02.185 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:46:02.185 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls' 0 2024-08-19T10:46:02.185 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls' 2024-08-19T10:46:02.186 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-08-19T10:46:02.186 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=0 2024-08-19T10:46:02.186 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-08-19T10:46:02.186 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth ls 2024-08-19T10:46:03.444 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=0 2024-08-19T10:46:03.444 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 0 -ne 0 ]] 2024-08-19T10:46:03.444 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:46:03.444 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-08-19T10:46:03.445 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export' 13 2024-08-19T10:46:03.445 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export' 2024-08-19T10:46:03.445 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=13 2024-08-19T10:46:03.445 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-08-19T10:46:03.445 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth export 2024-08-19T10:46:03.795 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=13 2024-08-19T10:46:03.842 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 13 -ne 13 ]] 2024-08-19T10:46:03.842 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:46:03.842 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar' 13 2024-08-19T10:46:03.842 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar' 2024-08-19T10:46:03.842 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=13 2024-08-19T10:46:03.842 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-08-19T10:46:03.843 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-08-19T10:46:03.843 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo auth del client.bazar 2024-08-19T10:46:04.147 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=13 2024-08-19T10:46:04.147 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 13 -ne 13 ]] 2024-08-19T10:46:04.147 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:46:04.147 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-08-19T10:46:04.147 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump' 13 2024-08-19T10:46:04.148 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump' 2024-08-19T10:46:04.148 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=13 2024-08-19T10:46:04.148 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-08-19T10:46:04.148 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo osd dump 2024-08-19T10:46:04.451 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=13 2024-08-19T10:46:04.452 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 13 -ne 13 ]] 2024-08-19T10:46:04.452 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:46:04.452 INFO:tasks.workunit.client.0.smithi190.stderr:+ export CEPH_ARGS=--rados-mon-op-timeout=300 2024-08-19T10:46:04.452 INFO:tasks.workunit.client.0.smithi190.stderr:+ CEPH_ARGS=--rados-mon-op-timeout=300 2024-08-19T10:46:04.452 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump' 110 2024-08-19T10:46:04.452 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump' 2024-08-19T10:46:04.452 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=110 2024-08-19T10:46:04.452 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-08-19T10:46:04.453 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-08-19T10:46:04.453 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo pg dump 2024-08-19T10:46:44.514 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-08-19T10:46:44.515 INFO:tasks.ceph.mon.b:Restarting daemon 2024-08-19T10:46:44.515 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-08-19T10:46:44.519 INFO:tasks.ceph.mon.b:Started 2024-08-19T10:46:44.519 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:46:44.519 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:46:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.636+0000 7f1e7d933640 10 monclient: get_monmap_and_config 2024-08-19T10:46:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.636+0000 7f1e7d933640 10 monclient: build_initial_monmap 2024-08-19T10:46:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.636+0000 7f1e7d933640 10 monclient: monmap: 2024-08-19T10:46:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:44.637 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:44.640344+0000 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:44.640344+0000 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:44.638 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:44.639 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:44.639 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:44.639 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:44.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 1 Processor -- start 2024-08-19T10:46:44.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 1 -- start start 2024-08-19T10:46:44.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 10 monclient: init 2024-08-19T10:46:44.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:44.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 10 monclient: _add_conns ranks=[5,8,4,10,3,9,1,0,2,6,7] 2024-08-19T10:46:44.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e780a3aa0 0x7f1e780a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:44.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 10 monclient(hunting): picked mon.noname-h con 0x7f1e780a3aa0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:44.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e780a1760 0x7f1e780a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:44.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 10 monclient(hunting): picked mon.noname-e con 0x7f1e780a1760 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:44.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e7813d910 0x7f1e7813fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:44.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 10 monclient(hunting): picked mon.noname-c con 0x7f1e7813d910 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:44.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:44.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f1e7806d6d0 con 0x7f1e7813d910 2024-08-19T10:46:44.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:44.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1e780a4900 con 0x7f1e780a3aa0 2024-08-19T10:46:44.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:44.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f1e780a4b90 con 0x7f1e780a1760 2024-08-19T10:46:44.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:44.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e7d933640 10 monclient(hunting): authenticate will time out at 1454.456697s 2024-08-19T10:46:44.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e780a1760 0x7f1e780a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:44.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e777fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e7813d910 0x7f1e7813fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:44.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e777fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e7813d910 0x7f1e7813fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45612/0 (socket says 172.21.15.89:45612) 2024-08-19T10:46:44.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e780a1760 0x7f1e780a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:49094/0 (socket says 172.21.15.89:49094) 2024-08-19T10:46:44.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e777fe640 1 -- 172.21.15.89:0/1745511715 learned_addr learned my addr 172.21.15.89:0/1745511715 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:44.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e777fe640 10 monclient(hunting): get_auth_request con 0x7f1e7813d910 auth_method 0 2024-08-19T10:46:44.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:44.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e777fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:44.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e777fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:44.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 10 monclient(hunting): get_auth_request con 0x7f1e780a1760 auth_method 0 2024-08-19T10:46:44.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:44.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:44.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:44.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:44.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:44.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:44.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:44.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:44.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:44.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e76ffd640 1 --2- 172.21.15.89:0/1745511715 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e780a3aa0 0x7f1e780a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:44.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e76ffd640 10 monclient(hunting): get_auth_request con 0x7f1e780a3aa0 auth_method 0 2024-08-19T10:46:44.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e76ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:44.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e76ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:44.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:44.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 10 monclient(hunting): handle_auth_done global_id 6425 payload 274 2024-08-19T10:46:44.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 1 -- 172.21.15.89:0/1745511715 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e780a3aa0 msgr2=0x7f1e780a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:44.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 1 --2- 172.21.15.89:0/1745511715 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e780a3aa0 0x7f1e780a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:44.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 1 -- 172.21.15.89:0/1745511715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e7813d910 msgr2=0x7f1e7813fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:44.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 1 --2- 172.21.15.89:0/1745511715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e7813d910 0x7f1e7813fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:44.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 10 monclient: _finish_hunting 0 2024-08-19T10:46:44.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 1 monclient: found mon.noname-e 2024-08-19T10:46:44.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:44.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:46:44.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e767fc640 1 -- 172.21.15.89:0/1745511715 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e780a09c0 con 0x7f1e780a1760 2024-08-19T10:46:44.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.640+0000 7f1e777fe640 1 --2- 172.21.15.89:0/1745511715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e7813d910 0x7f1e7813fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:44.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e767fc640 1 -- 172.21.15.89:0/1745511715 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e780a1760 msgr2=0x7f1e780a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:44.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e767fc640 1 --2- 172.21.15.89:0/1745511715 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e780a1760 0x7f1e780a43c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0097f0 tx=0x7f1e6c01c370 comp rx=0 tx=0).ready entity=mon.8 client_cookie=612cd8f089fb72f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:44.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 1 -- 172.21.15.89:0/1745511715 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e6c0b0070 con 0x7f1e780a1760 2024-08-19T10:46:44.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:44.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:46:44.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 10 monclient: dump: 2024-08-19T10:46:44.664 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:44.664 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:44.666 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 10 monclient: _finish_auth 0 2024-08-19T10:46:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:46:44.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:44.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 1 -- 172.21.15.89:0/1745511715 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6c0aa810 con 0x7f1e780a1760 2024-08-19T10:46:44.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:44.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 1 -- 172.21.15.89:0/1745511715 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e6c0a9300 con 0x7f1e780a1760 2024-08-19T10:46:44.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 5 monclient: authenticate success, global_id 6425 2024-08-19T10:46:44.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:44.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 4 set_mon_vals no callback set 2024-08-19T10:46:44.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:44.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:44.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e75ffb640 10 monclient: dump: 2024-08-19T10:46:44.668 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:44.668 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:44.668 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:44.668 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:44.668 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:44.669 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:44.669 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:44.669 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:44.669 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:44.669 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:44.669 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:44.669 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:44.669 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient: shutdown 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 -- 172.21.15.89:0/1745511715 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e780a1760 msgr2=0x7f1e780a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 --2- 172.21.15.89:0/1745511715 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e780a1760 0x7f1e780a43c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0097f0 tx=0x7f1e6c01c370 comp rx=0 tx=0).stop 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 -- 172.21.15.89:0/1745511715 shutdown_connections 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 --2- 172.21.15.89:0/1745511715 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1e7813d910 0x7f1e7813fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 --2- 172.21.15.89:0/1745511715 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1e780a1760 0x7f1e780a43c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 --2- 172.21.15.89:0/1745511715 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1e780a3aa0 0x7f1e780a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:44.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 -- 172.21.15.89:0/1745511715 >> 172.21.15.89:0/1745511715 conn(0x7f1e780a9130 msgr2=0x7f1e780a9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 -- 172.21.15.89:0/1745511715 shutdown_connections 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 -- 172.21.15.89:0/1745511715 wait complete. 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient: build_initial_monmap 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient: monmap: 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:44.671 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:44.672 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:44.672 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:44.672 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:44.672 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:44.672 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:44.672 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:44.672 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:44.672 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:44.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 Processor -- start 2024-08-19T10:46:44.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 -- start start 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient: init 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient: _add_conns ranks=[1,2,6,3,5,0,8,4,7] 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e780a1760 0x7f1e7813c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient(hunting): picked mon.noname-f con 0x7f1e780a1760 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e780a3aa0 0x7f1e7813cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient(hunting): picked mon.noname-b con 0x7f1e780a3aa0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e7813d910 0x7f1e78072010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient(hunting): picked mon.noname-d con 0x7f1e7813d910 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78142a60 con 0x7f1e780a1760 2024-08-19T10:46:44.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:44.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1e780a50b0 con 0x7f1e780a3aa0 2024-08-19T10:46:44.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:44.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1e78142be0 con 0x7f1e7813d910 2024-08-19T10:46:44.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:44.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e7d933640 10 monclient(hunting): authenticate will time out at 1454.463790s 2024-08-19T10:46:44.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e777fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e7813d910 0x7f1e78072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:44.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e777fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e7813d910 0x7f1e78072010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:57360/0 (socket says 172.21.15.89:57360) 2024-08-19T10:46:44.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e767fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e780a3aa0 0x7f1e7813cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:44.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e777fe640 1 -- 172.21.15.89:0/3261045749 learned_addr learned my addr 172.21.15.89:0/3261045749 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e777fe640 10 monclient(hunting): get_auth_request con 0x7f1e7813d910 auth_method 0 2024-08-19T10:46:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e777fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e777fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e767fc640 10 monclient(hunting): get_auth_request con 0x7f1e780a3aa0 auth_method 0 2024-08-19T10:46:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e767fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.644+0000 7f1e767fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e767fc640 1 -- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e780a3aa0 msgr2=0x7f1e7813cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:46:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e767fc640 1 -- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e780a3aa0 msgr2=0x7f1e7813cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:46:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:44.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e767fc640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e780a3aa0 0x7f1e7813cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:46:44.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:44.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:44.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e767fc640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e780a3aa0 0x7f1e7813cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:46:44.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e76ffd640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e780a1760 0x7f1e7813c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:44.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e76ffd640 10 monclient(hunting): get_auth_request con 0x7f1e780a1760 auth_method 0 2024-08-19T10:46:44.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e76ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:44.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e76ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:44.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e76ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:44.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 10 monclient(hunting): handle_auth_done global_id 44484 payload 824 2024-08-19T10:46:44.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 1 -- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e780a3aa0 msgr2=0x7f1e7813cfd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:46:44.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e780a3aa0 0x7f1e7813cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:44.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 1 -- 172.21.15.89:0/3261045749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e780a1760 msgr2=0x7f1e7813c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:44.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e780a1760 0x7f1e7813c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:44.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 10 monclient: _finish_hunting 0 2024-08-19T10:46:44.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 1 monclient: found mon.noname-d 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 1 -- 172.21.15.89:0/3261045749 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e78073ab0 con 0x7f1e7813d910 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 1 -- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e7813d910 msgr2=0x7f1e78072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e777fe640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e7813d910 0x7f1e78072010 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000efc0 tx=0x7f1e7000c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 1 -- 172.21.15.89:0/3261045749 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e7000cd30 con 0x7f1e7813d910 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 10 monclient: dump: 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 10 monclient: _finish_auth 0 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e7d933640 5 monclient: authenticate success, global_id 44484 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 1 -- 172.21.15.89:0/3261045749 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e70019070 con 0x7f1e7813d910 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e7d933640 10 monclient: _renew_subs 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e7d933640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e7d933640 1 -- 172.21.15.89:0/3261045749 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e78136df0 con 0x7f1e7813d910 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 1 -- 172.21.15.89:0/3261045749 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1e70014d60 con 0x7f1e7813d910 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e757fa640 4 set_mon_vals no callback set 2024-08-19T10:46:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 10 monclient: dump: 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:44.682 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e7d933640 10 monclient: _renew_subs 2024-08-19T10:46:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e7d933640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e7d933640 1 -- 172.21.15.89:0/3261045749 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e7813b740 con 0x7f1e7813d910 2024-08-19T10:46:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e7d933640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e7d933640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e7d933640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e7d933640 1 -- 172.21.15.89:0/3261045749 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e780a3e80 con 0x7f1e7813d910 2024-08-19T10:46:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 1 -- 172.21.15.89:0/3261045749 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1e70012050 con 0x7f1e7813d910 2024-08-19T10:46:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.648+0000 7f1e5ffff640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e6403be30 0x7f1e6403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.652+0000 7f1e5ffff640 1 -- 172.21.15.89:0/3261045749 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f1e7004f9b0 con 0x7f1e7813d910 2024-08-19T10:46:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.652+0000 7f1e76ffd640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e6403be30 0x7f1e6403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.652+0000 7f1e76ffd640 10 monclient: get_auth_request con 0x7f1e6403be30 auth_method 0 2024-08-19T10:46:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.652+0000 7f1e76ffd640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e6403be30 0x7f1e6403e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b9d0 tx=0x7f1e68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.656+0000 7f1e5ffff640 1 -- 172.21.15.89:0/3261045749 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1e70010040 con 0x7f1e7813d910 2024-08-19T10:46:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.656+0000 7f1e5ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:44.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.656+0000 7f1e5ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:46:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:46:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:44.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 -- 172.21.15.89:0/3261045749 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1e78064590 con 0x7f1e7813d910 2024-08-19T10:46:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e5ffff640 1 -- 172.21.15.89:0/3261045749 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3187 (secure 0 0 0) 0x7f1e70009a30 con 0x7f1e7813d910 2024-08-19T10:46:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e5ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:46:44.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e5ffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:44.879 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":80,"quorum":[0,1,3,4,5,6,7,8],"quorum_names":["a","f","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":92,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:46:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 -- 172.21.15.89:0/3261045749 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e6403be30 msgr2=0x7f1e6403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e6403be30 0x7f1e6403e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b9d0 tx=0x7f1e68008040 comp rx=0 tx=0).stop 2024-08-19T10:46:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 10 monclient: shutdown 2024-08-19T10:46:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 -- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e7813d910 msgr2=0x7f1e78072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e7813d910 0x7f1e78072010 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000efc0 tx=0x7f1e7000c490 comp rx=0 tx=0).stop 2024-08-19T10:46:44.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 -- 172.21.15.89:0/3261045749 shutdown_connections 2024-08-19T10:46:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1e6403be30 0x7f1e6403e2f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1e7813d910 0x7f1e78072010 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1e780a3aa0 0x7f1e7813cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 --2- 172.21.15.89:0/3261045749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e780a1760 0x7f1e7813c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 -- 172.21.15.89:0/3261045749 >> 172.21.15.89:0/3261045749 conn(0x7f1e780a9130 msgr2=0x7f1e780a7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 -- 172.21.15.89:0/3261045749 shutdown_connections 2024-08-19T10:46:44.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:44.880+0000 7f1e7d933640 1 -- 172.21.15.89:0/3261045749 wait complete. 2024-08-19T10:46:47.898 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:46:48.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient: get_monmap_and_config 2024-08-19T10:46:48.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient: build_initial_monmap 2024-08-19T10:46:48.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient: monmap: 2024-08-19T10:46:48.028 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:48.028 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:48.028 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:48.030257+0000 2024-08-19T10:46:48.028 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:48.030257+0000 2024-08-19T10:46:48.028 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:48.029 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:48.029 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:48.029 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:48.029 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:48.029 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:48.029 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:48.030 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:48.030 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:48.030 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:48.030 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:48.030 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:48.031 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:48.031 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 1 Processor -- start 2024-08-19T10:46:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 1 -- start start 2024-08-19T10:46:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient: init 2024-08-19T10:46:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient: _add_conns ranks=[9,1,5,7,8,10,2,4,6,3,0] 2024-08-19T10:46:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd2440a5270 0x7fd2440a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient(hunting): picked mon.noname-a con 0x7fd2440a5270 addr v2:172.21.15.89:3300/0 2024-08-19T10:46:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd2440a6d30 addr v1:172.21.15.190:6789/0 2024-08-19T10:46:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2440a7140 0x7fd244139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:48.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient(hunting): picked mon.noname-h con 0x7fd2440a7140 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2440a8710 con 0x7fd2440a6d30 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fd2440a5a40 con 0x7fd2440a5270 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd2440a81d0 con 0x7fd2440a7140 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24c63e640 10 monclient(hunting): authenticate will time out at 1457.847645s 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24a3da640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd2440a5270 0x7fd2440a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24a3da640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd2440a5270 0x7fd2440a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41650/0 (socket says 172.21.15.89:41650) 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24a3da640 1 -- 172.21.15.89:0/3801021356 learned_addr learned my addr 172.21.15.89:0/3801021356 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24a3da640 10 monclient(hunting): get_auth_request con 0x7fd2440a5270 auth_method 0 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24a3da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:48.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24a3da640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24a3da640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24a3da640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24a3da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.028+0000 7fd24a3da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24abdb640 1 --2- 172.21.15.89:0/3801021356 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2440a7140 0x7fd244139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24abdb640 10 monclient(hunting): get_auth_request con 0x7fd2440a7140 auth_method 0 2024-08-19T10:46:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24abdb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24abdb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:48.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24abdb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24a3da640 10 monclient(hunting): handle_auth_done global_id 6588 payload 274 2024-08-19T10:46:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24a3da640 1 -- 172.21.15.89:0/3801021356 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2440a7140 msgr2=0x7fd244139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24a3da640 1 --2- 172.21.15.89:0/3801021356 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2440a7140 0x7fd244139cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24a3da640 1 -- 172.21.15.89:0/3801021356 >> v1:172.21.15.190:6789/0 conn(0x7fd2440a6d30 legacy=0x7fd2441395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24a3da640 10 monclient: _finish_hunting 0 2024-08-19T10:46:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24a3da640 1 monclient: found mon.noname-a 2024-08-19T10:46:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24a3da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24a3da640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:46:48.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.032+0000 7fd24a3da640 1 -- 172.21.15.89:0/3801021356 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2440a4790 con 0x7fd2440a5270 2024-08-19T10:46:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 1 -- 172.21.15.89:0/3801021356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2440a5270 msgr2=0x7fd2440a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 1 --2- 172.21.15.89:0/3801021356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2440a5270 0x7fd2440a67f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd240009e50 tx=0x7fd24001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40366ad2688ea6f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 1 -- 172.21.15.89:0/3801021356 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2400090d0 con 0x7fd2440a5270 2024-08-19T10:46:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:46:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 10 monclient: dump: 2024-08-19T10:46:48.042 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:48.043 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 10 monclient: _finish_auth 0 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 10 monclient: _check_auth_tickets 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 1 -- 172.21.15.89:0/3801021356 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2400a5990 con 0x7fd2440a5270 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 5 monclient: authenticate success, global_id 6588 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 1 -- 172.21.15.89:0/3801021356 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2400ad3f0 con 0x7fd2440a5270 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 4 set_mon_vals no callback set 2024-08-19T10:46:48.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd2493d8640 10 monclient: dump: 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:48.045 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient: shutdown 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 -- 172.21.15.89:0/3801021356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2440a5270 msgr2=0x7fd2440a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 --2- 172.21.15.89:0/3801021356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2440a5270 0x7fd2440a67f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd240009e50 tx=0x7fd24001c6e0 comp rx=0 tx=0).stop 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 -- 172.21.15.89:0/3801021356 shutdown_connections 2024-08-19T10:46:48.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 --2- 172.21.15.89:0/3801021356 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2440a7140 0x7fd244139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 --2- 172.21.15.89:0/3801021356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2440a5270 0x7fd2440a67f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 -- 172.21.15.89:0/3801021356 >> 172.21.15.89:0/3801021356 conn(0x7fd244069a80 msgr2=0x7fd2440abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 -- 172.21.15.89:0/3801021356 shutdown_connections 2024-08-19T10:46:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 -- 172.21.15.89:0/3801021356 wait complete. 2024-08-19T10:46:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient: build_initial_monmap 2024-08-19T10:46:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient: monmap: 2024-08-19T10:46:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:48.047 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:48.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 Processor -- start 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 -- start start 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient: init 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient: _add_conns ranks=[8,6,5,4,3,2,1,0,7] 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2440a5270 0x7fd2441d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient(hunting): picked mon.noname-e con 0x7fd2440a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2440a6d30 0x7fd2441d1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient(hunting): picked mon.noname-d con 0x7fd2440a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2440a7140 0x7fd2441d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient(hunting): picked mon.noname-h con 0x7fd2440a7140 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:48.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd2440a57d0 con 0x7fd2440a7140 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd2440a5650 con 0x7fd2440a6d30 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd24413a200 con 0x7fd2440a5270 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24c63e640 10 monclient(hunting): authenticate will time out at 1457.855086s 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2440a5270 0x7fd2441d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd249bd9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2440a6d30 0x7fd2441d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd249bd9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2440a6d30 0x7fd2441d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:57418/0 (socket says 172.21.15.89:57418) 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2440a5270 0x7fd2441d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:49130/0 (socket says 172.21.15.89:49130) 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd249bd9640 1 -- 172.21.15.89:0/174686723 learned_addr learned my addr 172.21.15.89:0/174686723 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:48.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd249bd9640 10 monclient(hunting): get_auth_request con 0x7fd2440a6d30 auth_method 0 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd249bd9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd249bd9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd249bd9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 10 monclient(hunting): get_auth_request con 0x7fd2440a5270 auth_method 0 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd249bd9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:48.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd249bd9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd249bd9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24abdb640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2440a7140 0x7fd2441d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24abdb640 10 monclient(hunting): get_auth_request con 0x7fd2440a7140 auth_method 0 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24abdb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24abdb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24abdb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 10 monclient(hunting): handle_auth_done global_id 6434 payload 824 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 1 -- 172.21.15.89:0/174686723 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2440a6d30 msgr2=0x7fd2441d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2440a6d30 0x7fd2441d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 1 -- 172.21.15.89:0/174686723 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2440a7140 msgr2=0x7fd2441d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2440a7140 0x7fd2441d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 10 monclient: _finish_hunting 0 2024-08-19T10:46:48.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 1 monclient: found mon.noname-e 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd24a3da640 1 -- 172.21.15.89:0/174686723 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2441233b0 con 0x7fd2440a5270 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.036+0000 7fd249bd9640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2440a6d30 0x7fd2441d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24a3da640 1 -- 172.21.15.89:0/174686723 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2440a5270 msgr2=0x7fd2441d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24a3da640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2440a5270 0x7fd2441d0fd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd2440a5c10 tx=0x7fd240019480 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 1 -- 172.21.15.89:0/174686723 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd240003e10 con 0x7fd2440a5270 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 10 monclient: dump: 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:48.053 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:48.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 10 monclient: _finish_auth 0 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 10 monclient: _check_auth_tickets 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24c63e640 5 monclient: authenticate success, global_id 6434 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24c63e640 10 monclient: _renew_subs 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 1 -- 172.21.15.89:0/174686723 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd240004040 con 0x7fd2440a5270 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24c63e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24c63e640 1 -- 172.21.15.89:0/174686723 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2440a23b0 con 0x7fd2440a5270 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 1 -- 172.21.15.89:0/174686723 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2400041f0 con 0x7fd2440a5270 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 10 monclient: dump: 2024-08-19T10:46:48.055 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:48.056 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd248bd7640 4 set_mon_vals no callback set 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24c63e640 10 monclient: _renew_subs 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24c63e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24c63e640 1 -- 172.21.15.89:0/174686723 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2441230d0 con 0x7fd2440a5270 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24c63e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24c63e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24c63e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd24c63e640 1 -- 172.21.15.89:0/174686723 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2440a67f0 con 0x7fd2440a5270 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 1 -- 172.21.15.89:0/174686723 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd2400be020 con 0x7fd2440a5270 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd22803be00 0x7fd22803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd23affd640 1 -- 172.21.15.89:0/174686723 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fd2400bc020 con 0x7fd2440a5270 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd249bd9640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd22803be00 0x7fd22803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.040+0000 7fd249bd9640 10 monclient: get_auth_request con 0x7fd22803be00 auth_method 0 2024-08-19T10:46:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.044+0000 7fd249bd9640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd22803be00 0x7fd22803e2c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd24413f5f0 tx=0x7fd234003700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.044+0000 7fd23affd640 1 -- 172.21.15.89:0/174686723 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd2440a8710 con 0x7fd2440a5270 2024-08-19T10:46:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.044+0000 7fd23affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.044+0000 7fd23affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.276+0000 7fd24c63e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:46:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.276+0000 7fd24c63e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:46:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.276+0000 7fd24c63e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.276+0000 7fd24c63e640 1 -- 172.21.15.89:0/174686723 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd244064590 con 0x7fd2440a5270 2024-08-19T10:46:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.276+0000 7fd23affd640 1 -- 172.21.15.89:0/174686723 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3187 (secure 0 0 0) 0x7fd2400a5960 con 0x7fd2440a5270 2024-08-19T10:46:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.276+0000 7fd23affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:46:48.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.276+0000 7fd23affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:48.276 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":80,"quorum":[0,1,3,4,5,6,7,8],"quorum_names":["a","f","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":96,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:46:48.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 -- 172.21.15.89:0/174686723 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd22803be00 msgr2=0x7fd22803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd22803be00 0x7fd22803e2c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd24413f5f0 tx=0x7fd234003700 comp rx=0 tx=0).stop 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 10 monclient: shutdown 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 -- 172.21.15.89:0/174686723 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2440a5270 msgr2=0x7fd2441d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2440a5270 0x7fd2441d0fd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd2440a5c10 tx=0x7fd240019480 comp rx=0 tx=0).stop 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 -- 172.21.15.89:0/174686723 shutdown_connections 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2440a7140 0x7fd2441d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd2440a6d30 0x7fd2441d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd22803be00 0x7fd22803e2c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 --2- 172.21.15.89:0/174686723 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd2440a5270 0x7fd2441d0fd0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 -- 172.21.15.89:0/174686723 >> 172.21.15.89:0/174686723 conn(0x7fd244069a80 msgr2=0x7fd244141010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 -- 172.21.15.89:0/174686723 shutdown_connections 2024-08-19T10:46:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:48.280+0000 7fd24c63e640 1 -- 172.21.15.89:0/174686723 wait complete. 2024-08-19T10:46:51.294 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:46:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient: get_monmap_and_config 2024-08-19T10:46:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient: build_initial_monmap 2024-08-19T10:46:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient: monmap: 2024-08-19T10:46:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:51.430197+0000 2024-08-19T10:46:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:51.430197+0000 2024-08-19T10:46:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:51.427 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:51.428 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 1 Processor -- start 2024-08-19T10:46:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 1 -- start start 2024-08-19T10:46:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient: init 2024-08-19T10:46:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient: _add_conns ranks=[10,8,3,7,4,9,1,5,2,0,6] 2024-08-19T10:46:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe3a80a67d0 0x7fe3a80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient(hunting): picked mon.noname-f con 0x7fe3a80a67d0 addr v2:172.21.15.190:3300/0 2024-08-19T10:46:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a6bb0 0x7fe3a80a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient(hunting): picked mon.noname-e con 0x7fe3a80a6bb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3a8139590 0x7fe3a8139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:51.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient(hunting): picked mon.noname-g con 0x7fe3a8139590 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe3a80a8740 con 0x7fe3a80a67d0 2024-08-19T10:46:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a80a3380 con 0x7fe3a8139590 2024-08-19T10:46:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fe3a80a8250 con 0x7fe3a80a6bb0 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3aeaa9640 10 monclient(hunting): authenticate will time out at 1461.246805s 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a6bb0 0x7fe3a80a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a6bb0 0x7fe3a80a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:36096/0 (socket says 172.21.15.89:36096) 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 1 -- 172.21.15.89:0/1016444861 learned_addr learned my addr 172.21.15.89:0/1016444861 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 10 monclient(hunting): get_auth_request con 0x7fe3a80a6bb0 auth_method 0 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ad046640 1 --2- 172.21.15.89:0/1016444861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3a8139590 0x7fe3a8139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:51.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ac845640 1 --2- 172.21.15.89:0/1016444861 >> v2:172.21.15.190:3300/0 conn(0x7fe3a80a67d0 0x7fe3a80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ad046640 10 monclient(hunting): get_auth_request con 0x7fe3a8139590 auth_method 0 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ad046640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ad046640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ad046640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ac845640 10 monclient(hunting): get_auth_request con 0x7fe3a80a67d0 auth_method 0 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ac845640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ac845640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ac845640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ad046640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ad046640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:51.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ad046640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ac845640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ac845640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3ac845640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 10 monclient(hunting): handle_auth_done global_id 6443 payload 274 2024-08-19T10:46:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 1 -- 172.21.15.89:0/1016444861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3a8139590 msgr2=0x7fe3a8139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 1 --2- 172.21.15.89:0/1016444861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3a8139590 0x7fe3a8139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 1 -- 172.21.15.89:0/1016444861 >> v2:172.21.15.190:3300/0 conn(0x7fe3a80a67d0 msgr2=0x7fe3a80a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:51.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 1 --2- 172.21.15.89:0/1016444861 >> v2:172.21.15.190:3300/0 conn(0x7fe3a80a67d0 0x7fe3a80a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:46:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 1 monclient: found mon.noname-e 2024-08-19T10:46:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:46:51.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.428+0000 7fe3a7fff640 1 -- 172.21.15.89:0/1016444861 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a80a4cc0 con 0x7fe3a80a6bb0 2024-08-19T10:46:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a7fff640 1 -- 172.21.15.89:0/1016444861 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a6bb0 msgr2=0x7fe3a80a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a7fff640 1 --2- 172.21.15.89:0/1016444861 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a6bb0 0x7fe3a80a6f90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0098a0 tx=0x7fe39c01c490 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9d8c4670529a5b2b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 1 -- 172.21.15.89:0/1016444861 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe39c0b0070 con 0x7fe3a80a6bb0 2024-08-19T10:46:51.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 10 monclient: dump: 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:51.443 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 10 monclient: _finish_auth 0 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 1 -- 172.21.15.89:0/1016444861 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe39c0aa810 con 0x7fe3a80a6bb0 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 5 monclient: authenticate success, global_id 6443 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:51.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 1 -- 172.21.15.89:0/1016444861 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe39c0a9300 con 0x7fe3a80a6bb0 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 4 set_mon_vals no callback set 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a77fe640 10 monclient: dump: 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:51.445 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient: shutdown 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/1016444861 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a6bb0 msgr2=0x7fe3a80a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 --2- 172.21.15.89:0/1016444861 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a6bb0 0x7fe3a80a6f90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0098a0 tx=0x7fe39c01c490 comp rx=0 tx=0).stop 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/1016444861 shutdown_connections 2024-08-19T10:46:51.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 --2- 172.21.15.89:0/1016444861 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3a8139590 0x7fe3a8139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 --2- 172.21.15.89:0/1016444861 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a6bb0 0x7fe3a80a6f90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 --2- 172.21.15.89:0/1016444861 >> v2:172.21.15.190:3300/0 conn(0x7fe3a80a67d0 0x7fe3a80a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/1016444861 >> 172.21.15.89:0/1016444861 conn(0x7fe3a8069ae0 msgr2=0x7fe3a80abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/1016444861 shutdown_connections 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/1016444861 wait complete. 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient: build_initial_monmap 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient: monmap: 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:51.447 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:51.448 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:51.448 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 Processor -- start 2024-08-19T10:46:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 -- start start 2024-08-19T10:46:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient: init 2024-08-19T10:46:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient: _add_conns ranks=[8,7,1,3,0,5,4,6,2] 2024-08-19T10:46:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a67d0 0x7fe3a81d1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient(hunting): picked mon.noname-e con 0x7fe3a80a67d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3a80a6bb0 0x7fe3a80a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient(hunting): picked mon.noname-i con 0x7fe3a80a6bb0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:46:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3a8139590 0x7fe3a80a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:51.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient(hunting): picked mon.noname-f con 0x7fe3a8139590 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:46:51.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:51.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8143390 con 0x7fe3a8139590 2024-08-19T10:46:51.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:51.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fe3a8143210 con 0x7fe3a80a6bb0 2024-08-19T10:46:51.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:51.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fe3a8143510 con 0x7fe3a80a67d0 2024-08-19T10:46:51.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:51.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3aeaa9640 10 monclient(hunting): authenticate will time out at 1461.255367s 2024-08-19T10:46:51.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a67d0 0x7fe3a81d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:51.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a67d0 0x7fe3a81d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:36102/0 (socket says 172.21.15.89:36102) 2024-08-19T10:46:51.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 1 -- 172.21.15.89:0/94487361 learned_addr learned my addr 172.21.15.89:0/94487361 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 10 monclient(hunting): get_auth_request con 0x7fe3a80a67d0 auth_method 0 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a7fff640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3a80a6bb0 0x7fe3a80a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ad046640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3a8139590 0x7fe3a80a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 10 monclient(hunting): handle_auth_done global_id 6452 payload 824 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 1 -- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3a80a6bb0 msgr2=0x7fe3a80a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3a80a6bb0 0x7fe3a80a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 1 -- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3a8139590 msgr2=0x7fe3a80a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:51.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3a8139590 0x7fe3a80a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 10 monclient: _finish_hunting 0 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 1 monclient: found mon.noname-e 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ac845640 1 -- 172.21.15.89:0/94487361 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a80a5620 con 0x7fe3a80a67d0 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a7fff640 10 monclient: get_auth_request con 0x7fe3a80a6bb0 auth_method 0 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ad046640 10 monclient: get_auth_request con 0x7fe3a8139590 auth_method 0 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3a7fff640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3a80a6bb0 0x7fe3a80a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.436+0000 7fe3ad046640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3a8139590 0x7fe3a80a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3ac845640 1 -- 172.21.15.89:0/94487361 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a67d0 msgr2=0x7fe3a81d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3ac845640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a67d0 0x7fe3a81d1510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe39800e930 tx=0x7fe39800edf0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:51.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 1 -- 172.21.15.89:0/94487361 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe398017070 con 0x7fe3a80a67d0 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 10 monclient: dump: 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:51.453 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:51.454 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:51.454 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:51.454 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:51.454 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:51.454 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:51.454 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:51.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 10 monclient: _finish_auth 0 2024-08-19T10:46:51.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:46:51.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:51.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3aeaa9640 5 monclient: authenticate success, global_id 6452 2024-08-19T10:46:51.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 1 -- 172.21.15.89:0/94487361 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe39800cc10 con 0x7fe3a80a67d0 2024-08-19T10:46:51.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:51.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 1 -- 172.21.15.89:0/94487361 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fe39800cdc0 con 0x7fe3a80a67d0 2024-08-19T10:46:51.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:51.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:51.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a6ffd640 4 set_mon_vals no callback set 2024-08-19T10:46:51.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 10 monclient: dump: 2024-08-19T10:46:51.455 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:51.456 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:51.457 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3aeaa9640 10 monclient: _renew_subs 2024-08-19T10:46:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3aeaa9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/94487361 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a809f120 con 0x7fe3a80a67d0 2024-08-19T10:46:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3aeaa9640 10 monclient: _renew_subs 2024-08-19T10:46:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3aeaa9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/94487361 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a80a0880 con 0x7fe3a80a67d0 2024-08-19T10:46:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3aeaa9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3aeaa9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3aeaa9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:51.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/94487361 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a80a6f90 con 0x7fe3a80a67d0 2024-08-19T10:46:51.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.440+0000 7fe3a57fa640 1 -- 172.21.15.89:0/94487361 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fe39801a070 con 0x7fe3a80a67d0 2024-08-19T10:46:51.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.444+0000 7fe3a57fa640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39403be00 0x7fe39403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:51.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.444+0000 7fe3a57fa640 1 -- 172.21.15.89:0/94487361 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fe39800f040 con 0x7fe3a80a67d0 2024-08-19T10:46:51.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.444+0000 7fe3a7fff640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39403be00 0x7fe39403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:51.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.444+0000 7fe3a7fff640 10 monclient: get_auth_request con 0x7fe39403be00 auth_method 0 2024-08-19T10:46:51.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.444+0000 7fe3a7fff640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39403be00 0x7fe39403e2c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80a3e80 tx=0x7fe39c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:51.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.448+0000 7fe3a57fa640 1 -- 172.21.15.89:0/94487361 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fe398021020 con 0x7fe3a80a67d0 2024-08-19T10:46:51.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.448+0000 7fe3a57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:51.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.448+0000 7fe3a57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:51.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.676+0000 7fe3aeaa9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:46:51.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.676+0000 7fe3aeaa9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:46:51.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.676+0000 7fe3aeaa9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:51.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.676+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/94487361 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe3a8064590 con 0x7fe3a80a67d0 2024-08-19T10:46:51.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.676+0000 7fe3a57fa640 1 -- 172.21.15.89:0/94487361 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3187 (secure 0 0 0) 0x7fe39801ac80 con 0x7fe3a80a67d0 2024-08-19T10:46:51.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.676+0000 7fe3a57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:46:51.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.676+0000 7fe3a57fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:51.676 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":80,"quorum":[0,1,3,4,5,6,7,8],"quorum_names":["a","f","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":99,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:46:51.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39403be00 msgr2=0x7fe39403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:51.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39403be00 0x7fe39403e2c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80a3e80 tx=0x7fe39c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:46:51.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 10 monclient: shutdown 2024-08-19T10:46:51.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:51.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/94487361 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a67d0 msgr2=0x7fe3a81d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:51.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a67d0 0x7fe3a81d1510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe39800e930 tx=0x7fe39800edf0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/94487361 shutdown_connections 2024-08-19T10:46:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3a8139590 0x7fe3a80a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fe3a80a6bb0 0x7fe3a80a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fe39403be00 0x7fe39403e2c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 --2- 172.21.15.89:0/94487361 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fe3a80a67d0 0x7fe3a81d1510 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/94487361 >> 172.21.15.89:0/94487361 conn(0x7fe3a8069ae0 msgr2=0x7fe3a80ab780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/94487361 shutdown_connections 2024-08-19T10:46:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:51.680+0000 7fe3aeaa9640 1 -- 172.21.15.89:0/94487361 wait complete. 2024-08-19T10:46:54.694 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:46:54.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.824+0000 7fcb0e750640 10 monclient: get_monmap_and_config 2024-08-19T10:46:54.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.824+0000 7fcb0e750640 10 monclient: build_initial_monmap 2024-08-19T10:46:54.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.824+0000 7fcb0e750640 10 monclient: monmap: 2024-08-19T10:46:54.824 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:54.824 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:54.824 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:54.827054+0000 2024-08-19T10:46:54.825 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:54.827054+0000 2024-08-19T10:46:54.825 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:54.825 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:54.825 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:54.825 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:54.825 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:54.825 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:54.825 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:54.826 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:54.826 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:54.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.824+0000 7fcb0e750640 1 Processor -- start 2024-08-19T10:46:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.824+0000 7fcb0e750640 1 -- start start 2024-08-19T10:46:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.824+0000 7fcb0e750640 10 monclient: init 2024-08-19T10:46:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 10 monclient: _add_conns ranks=[10,4,2,1,8,9,0,5,7,6,3] 2024-08-19T10:46:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcb0806c930 0x7fcb080a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 10 monclient(hunting): picked mon.noname-f con 0x7fcb0806c930 addr v2:172.21.15.190:3300/0 2024-08-19T10:46:54.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb080a3f40 0x7fcb080a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 10 monclient(hunting): picked mon.noname-c con 0x7fcb080a3f40 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb08139850 0x7fcb08139d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 10 monclient(hunting): picked mon.noname-b con 0x7fcb08139850 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fcb080a4860 con 0x7fcb0806c930 2024-08-19T10:46:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fcb0806cd10 con 0x7fcb08139850 2024-08-19T10:46:54.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fcb080a1bc0 con 0x7fcb080a3f40 2024-08-19T10:46:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0e750640 10 monclient(hunting): authenticate will time out at 1464.644191s 2024-08-19T10:46:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb077fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb080a3f40 0x7fcb080a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb08139850 0x7fcb08139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb077fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb080a3f40 0x7fcb080a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42158/0 (socket says 172.21.15.89:42158) 2024-08-19T10:46:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb08139850 0x7fcb08139d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44848/0 (socket says 172.21.15.89:44848) 2024-08-19T10:46:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb077fe640 1 -- 172.21.15.89:0/1766390575 learned_addr learned my addr 172.21.15.89:0/1766390575 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb077fe640 10 monclient(hunting): get_auth_request con 0x7fcb080a3f40 auth_method 0 2024-08-19T10:46:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb077fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb077fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 10 monclient(hunting): get_auth_request con 0x7fcb08139850 auth_method 0 2024-08-19T10:46:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb07fff640 1 --2- 172.21.15.89:0/1766390575 >> v2:172.21.15.190:3300/0 conn(0x7fcb0806c930 0x7fcb080a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb07fff640 10 monclient(hunting): get_auth_request con 0x7fcb0806c930 auth_method 0 2024-08-19T10:46:54.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:54.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb07fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:54.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb07fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:54.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 10 monclient(hunting): handle_auth_done global_id 54101 payload 274 2024-08-19T10:46:54.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 1 -- 172.21.15.89:0/1766390575 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb080a3f40 msgr2=0x7fcb080a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:54.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 1 --2- 172.21.15.89:0/1766390575 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb080a3f40 0x7fcb080a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:54.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 1 -- 172.21.15.89:0/1766390575 >> v2:172.21.15.190:3300/0 conn(0x7fcb0806c930 msgr2=0x7fcb080a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:54.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 1 --2- 172.21.15.89:0/1766390575 >> v2:172.21.15.190:3300/0 conn(0x7fcb0806c930 0x7fcb080a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:54.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 10 monclient: _finish_hunting 0 2024-08-19T10:46:54.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 1 monclient: found mon.noname-b 2024-08-19T10:46:54.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:54.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:46:54.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb0cced640 1 -- 172.21.15.89:0/1766390575 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb080a4b60 con 0x7fcb08139850 2024-08-19T10:46:54.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.828+0000 7fcb077fe640 1 --2- 172.21.15.89:0/1766390575 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb080a3f40 0x7fcb080a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:54.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0cced640 1 -- 172.21.15.89:0/1766390575 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb08139850 msgr2=0x7fcb08139d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:54.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0cced640 1 --2- 172.21.15.89:0/1766390575 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb08139850 0x7fcb08139d00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb000097c0 tx=0x7fcb0001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65a85c5d89f03c0c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:54.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 1 -- 172.21.15.89:0/1766390575 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb000b0070 con 0x7fcb08139850 2024-08-19T10:46:54.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:54.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:46:54.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 10 monclient: dump: 2024-08-19T10:46:54.843 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:54.843 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:54.843 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:54.843 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:54.843 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:54.843 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:54.843 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:54.843 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:54.844 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:54.844 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:54.844 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:54.844 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:54.844 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:54.844 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:54.844 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:54.844 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:54.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 10 monclient: _finish_auth 0 2024-08-19T10:46:54.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:46:54.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:54.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 1 -- 172.21.15.89:0/1766390575 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb000aa810 con 0x7fcb08139850 2024-08-19T10:46:54.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 5 monclient: authenticate success, global_id 54101 2024-08-19T10:46:54.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:46:54.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:54.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 1 -- 172.21.15.89:0/1766390575 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb000aae30 con 0x7fcb08139850 2024-08-19T10:46:54.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:54.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:46:54.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb06ffd640 10 monclient: dump: 2024-08-19T10:46:54.846 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:54.846 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:54.846 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:54.846 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:54.846 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:54.847 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:54.847 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:54.847 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:54.847 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:54.847 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:54.847 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:54.847 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:54.847 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:54.848 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:54.848 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:54.848 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:54.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:54.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 4 set_mon_vals no callback set 2024-08-19T10:46:54.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient: shutdown 2024-08-19T10:46:54.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:54.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 -- 172.21.15.89:0/1766390575 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb08139850 msgr2=0x7fcb08139d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:54.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 --2- 172.21.15.89:0/1766390575 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb08139850 0x7fcb08139d00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb000097c0 tx=0x7fcb0001c360 comp rx=0 tx=0).stop 2024-08-19T10:46:54.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 -- 172.21.15.89:0/1766390575 shutdown_connections 2024-08-19T10:46:54.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 --2- 172.21.15.89:0/1766390575 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb08139850 0x7fcb08139d00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:54.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 --2- 172.21.15.89:0/1766390575 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb080a3f40 0x7fcb080a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:54.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 --2- 172.21.15.89:0/1766390575 >> v2:172.21.15.190:3300/0 conn(0x7fcb0806c930 0x7fcb080a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:54.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 -- 172.21.15.89:0/1766390575 >> 172.21.15.89:0/1766390575 conn(0x7fcb080a9090 msgr2=0x7fcb080a94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:54.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 -- 172.21.15.89:0/1766390575 shutdown_connections 2024-08-19T10:46:54.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 -- 172.21.15.89:0/1766390575 wait complete. 2024-08-19T10:46:54.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient: build_initial_monmap 2024-08-19T10:46:54.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient: monmap: 2024-08-19T10:46:54.850 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:54.851 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:54.852 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:54.852 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:54.852 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:54.852 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:54.852 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:54.852 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:54.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 Processor -- start 2024-08-19T10:46:54.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 -- start start 2024-08-19T10:46:54.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient: init 2024-08-19T10:46:54.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:54.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient: _add_conns ranks=[4,6,8,0,5,7,2,3,1] 2024-08-19T10:46:54.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb0806c930 0x7fcb08137f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:54.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient(hunting): picked mon.noname-c con 0x7fcb0806c930 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:54.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcb080a3f40 0x7fcb08138460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:54.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient(hunting): picked mon.noname-d con 0x7fcb080a3f40 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:54.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcb08139850 0x7fcb0813c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:54.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient(hunting): picked mon.noname-e con 0x7fcb08139850 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:54.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:54.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fcb080a0f10 con 0x7fcb0806c930 2024-08-19T10:46:54.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:54.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fcb080a0c30 con 0x7fcb080a3f40 2024-08-19T10:46:54.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:54.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fcb080a0190 con 0x7fcb08139850 2024-08-19T10:46:54.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:54.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0e750640 10 monclient(hunting): authenticate will time out at 1464.651527s 2024-08-19T10:46:54.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb07fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb0806c930 0x7fcb08137f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:54.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0cced640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcb08139850 0x7fcb0813c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:54.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb077fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcb080a3f40 0x7fcb08138460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:54.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb07fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb0806c930 0x7fcb08137f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42166/0 (socket says 172.21.15.89:42166) 2024-08-19T10:46:54.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0cced640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcb08139850 0x7fcb0813c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:36108/0 (socket says 172.21.15.89:36108) 2024-08-19T10:46:54.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb07fff640 1 -- 172.21.15.89:0/453259005 learned_addr learned my addr 172.21.15.89:0/453259005 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:54.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb077fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcb080a3f40 0x7fcb08138460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:35498/0 (socket says 172.21.15.89:35498) 2024-08-19T10:46:54.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb077fe640 10 monclient(hunting): get_auth_request con 0x7fcb080a3f40 auth_method 0 2024-08-19T10:46:54.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:54.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb077fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:54.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb077fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:54.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0cced640 10 monclient(hunting): get_auth_request con 0x7fcb08139850 auth_method 0 2024-08-19T10:46:54.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0cced640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:54.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0cced640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:54.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0cced640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:54.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb07fff640 10 monclient(hunting): get_auth_request con 0x7fcb0806c930 auth_method 0 2024-08-19T10:46:54.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:54.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb07fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:54.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb07fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:54.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:54.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:54.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:54.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:54.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:54.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:54.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0cced640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:54.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0cced640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:54.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.832+0000 7fcb0cced640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:54.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 10 monclient(hunting): handle_auth_done global_id 44502 payload 824 2024-08-19T10:46:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 1 -- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcb08139850 msgr2=0x7fcb0813c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcb08139850 0x7fcb0813c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 1 -- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb0806c930 msgr2=0x7fcb08137f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb0806c930 0x7fcb08137f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 10 monclient: _finish_hunting 0 2024-08-19T10:46:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 1 monclient: found mon.noname-d 2024-08-19T10:46:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:46:54.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 1 -- 172.21.15.89:0/453259005 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0813dfa0 con 0x7fcb080a3f40 2024-08-19T10:46:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb07fff640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb0806c930 0x7fcb08137f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:54.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0cced640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcb08139850 0x7fcb0813c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 1 -- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcb080a3f40 msgr2=0x7fcb08138460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb077fe640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcb080a3f40 0x7fcb08138460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcb080a0880 tx=0x7fcafc00eda0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:54.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 1 -- 172.21.15.89:0/453259005 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcafc017020 con 0x7fcb080a3f40 2024-08-19T10:46:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:46:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 10 monclient: dump: 2024-08-19T10:46:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:54.867 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:54.868 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:54.868 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:54.868 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:54.868 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:54.868 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:54.868 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:54.868 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 10 monclient: _finish_auth 0 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 1 -- 172.21.15.89:0/453259005 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcafc004450 con 0x7fcb080a3f40 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 1 -- 172.21.15.89:0/453259005 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcafc00cba0 con 0x7fcb080a3f40 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0e750640 5 monclient: authenticate success, global_id 44502 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0e750640 10 monclient: _renew_subs 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0e750640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:54.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0e750640 1 -- 172.21.15.89:0/453259005 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb08123f90 con 0x7fcb080a3f40 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 10 monclient: dump: 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:54.870 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb067fc640 4 set_mon_vals no callback set 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0e750640 10 monclient: _renew_subs 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0e750640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0e750640 1 -- 172.21.15.89:0/453259005 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb08123cb0 con 0x7fcb080a3f40 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0e750640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:54.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0e750640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0e750640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb0e750640 1 -- 172.21.15.89:0/453259005 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb080a4320 con 0x7fcb080a3f40 2024-08-19T10:46:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 1 -- 172.21.15.89:0/453259005 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fcafc01a020 con 0x7fcb080a3f40 2024-08-19T10:46:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.836+0000 7fcb04ff9640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcaf403be00 0x7fcaf403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.840+0000 7fcb04ff9640 1 -- 172.21.15.89:0/453259005 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fcafc00f040 con 0x7fcb080a3f40 2024-08-19T10:46:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.840+0000 7fcb07fff640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcaf403be00 0x7fcaf403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.840+0000 7fcb07fff640 10 monclient: get_auth_request con 0x7fcaf403be00 auth_method 0 2024-08-19T10:46:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.840+0000 7fcb07fff640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcaf403be00 0x7fcaf403e2c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8003f30 tx=0x7fcaf80098d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.844+0000 7fcb04ff9640 1 -- 172.21.15.89:0/453259005 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fcafc021030 con 0x7fcb080a3f40 2024-08-19T10:46:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.844+0000 7fcb04ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:54.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:54.844+0000 7fcb04ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.076+0000 7fcb0e750640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:46:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.076+0000 7fcb0e750640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:46:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.076+0000 7fcb0e750640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:55.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.076+0000 7fcb0e750640 1 -- 172.21.15.89:0/453259005 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcb0813f720 con 0x7fcb080a3f40 2024-08-19T10:46:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.076+0000 7fcb04ff9640 1 -- 172.21.15.89:0/453259005 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7fcb0813dfa0 con 0x7fcb080a3f40 2024-08-19T10:46:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.076+0000 7fcb04ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:46:55.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.076+0000 7fcb04ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:55.076 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":82,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:46:55.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 -- 172.21.15.89:0/453259005 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcaf403be00 msgr2=0x7fcaf403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcaf403be00 0x7fcaf403e2c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8003f30 tx=0x7fcaf80098d0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 10 monclient: shutdown 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 -- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcb080a3f40 msgr2=0x7fcb08138460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcb080a3f40 0x7fcb08138460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcb080a0880 tx=0x7fcafc00eda0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 -- 172.21.15.89:0/453259005 shutdown_connections 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcaf403be00 0x7fcaf403e2c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcb08139850 0x7fcb0813c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcb080a3f40 0x7fcb08138460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 --2- 172.21.15.89:0/453259005 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb0806c930 0x7fcb08137f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 -- 172.21.15.89:0/453259005 >> 172.21.15.89:0/453259005 conn(0x7fcb080a9090 msgr2=0x7fcb08072d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 -- 172.21.15.89:0/453259005 shutdown_connections 2024-08-19T10:46:55.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.080+0000 7fcb0e750640 1 -- 172.21.15.89:0/453259005 wait complete. 2024-08-19T10:46:55.095 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:46:55.095 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:46:55.095 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:46:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient: get_monmap_and_config 2024-08-19T10:46:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient: build_initial_monmap 2024-08-19T10:46:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient: monmap: 2024-08-19T10:46:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:55.222127+0000 2024-08-19T10:46:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:55.222127+0000 2024-08-19T10:46:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:55.219 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:55.220 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:55.220 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 1 Processor -- start 2024-08-19T10:46:55.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 1 -- start start 2024-08-19T10:46:55.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient: init 2024-08-19T10:46:55.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:55.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient: _add_conns ranks=[1,10,3,6,9,5,7,8,0,4,2] 2024-08-19T10:46:55.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3fb80a5270 addr v1:172.21.15.190:6789/0 2024-08-19T10:46:55.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3fb80a6f00 0x7f3fb813ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient(hunting): picked mon.noname-f con 0x7f3fb80a6f00 addr v2:172.21.15.190:3300/0 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fb80a7310 0x7f3fb8140f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient(hunting): picked mon.noname-g con 0x7f3fb80a7310 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fb80a8710 con 0x7f3fb80a5270 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f3fb80a5a40 con 0x7f3fb80a6f00 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb80a81d0 con 0x7f3fb80a7310 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fbca32640 10 monclient(hunting): authenticate will time out at 1465.038689s 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fb80a7310 0x7f3fb8140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fadd9b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3fb80a6f00 0x7f3fb813ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fb80a7310 0x7f3fb8140f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:55732/0 (socket says 172.21.15.89:55732) 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 1 -- 172.21.15.89:0/1318164427 learned_addr learned my addr 172.21.15.89:0/1318164427 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 10 monclient(hunting): get_auth_request con 0x7f3fb80a7310 auth_method 0 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fadd9b640 10 monclient(hunting): get_auth_request con 0x7f3fb80a6f00 auth_method 0 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fadd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fadd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fadd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fadd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fadd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fadd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 10 monclient(hunting): handle_auth_done global_id 35103 payload 274 2024-08-19T10:46:55.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 1 -- 172.21.15.89:0/1318164427 >> v2:172.21.15.190:3300/0 conn(0x7f3fb80a6f00 msgr2=0x7f3fb813ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:55.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 1 --2- 172.21.15.89:0/1318164427 >> v2:172.21.15.190:3300/0 conn(0x7f3fb80a6f00 0x7f3fb813ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 1 -- 172.21.15.89:0/1318164427 >> v1:172.21.15.190:6789/0 conn(0x7f3fb80a5270 legacy=0x7f3fb80a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:55.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:46:55.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 1 monclient: found mon.noname-g 2024-08-19T10:46:55.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:55.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:46:55.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.220+0000 7f3fb6d9d640 1 -- 172.21.15.89:0/1318164427 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb80a4790 con 0x7f3fb80a7310 2024-08-19T10:46:55.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb6d9d640 1 -- 172.21.15.89:0/1318164427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fb80a7310 msgr2=0x7f3fb8140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb6d9d640 1 --2- 172.21.15.89:0/1318164427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fb80a7310 0x7f3fb8140f50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009e50 tx=0x7f3fb001c6e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=6affd46a57a79c49 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 1 -- 172.21.15.89:0/1318164427 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3fb00090d0 con 0x7f3fb80a7310 2024-08-19T10:46:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:46:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 10 monclient: dump: 2024-08-19T10:46:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.228 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 10 monclient: _finish_auth 0 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 1 -- 172.21.15.89:0/1318164427 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb00a5990 con 0x7f3fb80a7310 2024-08-19T10:46:55.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 1 -- 172.21.15.89:0/1318164427 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3fb00ad3f0 con 0x7f3fb80a7310 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fb5d9b640 10 monclient: dump: 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.230 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.231 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.231 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.231 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.231 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.231 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.231 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.231 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 5 monclient: authenticate success, global_id 35103 2024-08-19T10:46:55.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:55.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 4 set_mon_vals no callback set 2024-08-19T10:46:55.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 10 monclient: shutdown 2024-08-19T10:46:55.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:55.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 1 -- 172.21.15.89:0/1318164427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fb80a7310 msgr2=0x7f3fb8140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 1 --2- 172.21.15.89:0/1318164427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fb80a7310 0x7f3fb8140f50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009e50 tx=0x7f3fb001c6e0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 1 -- 172.21.15.89:0/1318164427 shutdown_connections 2024-08-19T10:46:55.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 1 --2- 172.21.15.89:0/1318164427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3fb80a7310 0x7f3fb8140f50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 1 --2- 172.21.15.89:0/1318164427 >> v2:172.21.15.190:3300/0 conn(0x7f3fb80a6f00 0x7f3fb813ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 1 -- 172.21.15.89:0/1318164427 >> 172.21.15.89:0/1318164427 conn(0x7f3fb8069a80 msgr2=0x7f3fb80abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:55.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 1 -- 172.21.15.89:0/1318164427 shutdown_connections 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 1 -- 172.21.15.89:0/1318164427 wait complete. 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 10 monclient: build_initial_monmap 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 10 monclient: monmap: 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:55.237 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 1 Processor -- start 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 1 -- start start 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.228+0000 7f3fbca32640 10 monclient: init 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient: _add_conns ranks=[2,4,6,8,1,3,5,0,7] 2024-08-19T10:46:55.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fb80a5270 0x7f3fb81289b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient(hunting): picked mon.noname-b con 0x7f3fb80a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fb80a6f00 0x7f3fb81d1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient(hunting): picked mon.noname-c con 0x7f3fb80a6f00 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3fb80a7310 0x7f3fb8126430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient(hunting): picked mon.noname-d con 0x7f3fb80a7310 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb8143e30 con 0x7f3fb80a5270 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3fb8143cb0 con 0x7f3fb80a6f00 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f3fb8143fb0 con 0x7f3fb80a7310 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:55.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient(hunting): authenticate will time out at 1465.048332s 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fb80a5270 0x7f3fb81289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb6d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3fb80a7310 0x7f3fb8126430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb6d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3fb80a7310 0x7f3fb8126430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:35510/0 (socket says 172.21.15.89:35510) 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fb80a5270 0x7f3fb81289b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44862/0 (socket says 172.21.15.89:44862) 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fb80a6f00 0x7f3fb81d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb6d9d640 1 -- 172.21.15.89:0/3798723473 learned_addr learned my addr 172.21.15.89:0/3798723473 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb6d9d640 10 monclient(hunting): get_auth_request con 0x7f3fb80a7310 auth_method 0 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb659c640 10 monclient(hunting): get_auth_request con 0x7f3fb80a5270 auth_method 0 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 10 monclient(hunting): get_auth_request con 0x7f3fb80a6f00 auth_method 0 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 10 monclient(hunting): handle_auth_done global_id 44500 payload 824 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 1 -- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3fb80a7310 msgr2=0x7f3fb8126430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3fb80a7310 0x7f3fb8126430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 1 -- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fb80a5270 msgr2=0x7f3fb81289b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3fb80a5270 0x7f3fb81289b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 10 monclient: _finish_hunting 0 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 1 monclient: found mon.noname-c 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 1 -- 172.21.15.89:0/3798723473 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb8129ed0 con 0x7f3fb80a6f00 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb6d9d640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3fb80a7310 0x7f3fb8126430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:55.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 1 -- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fb80a6f00 msgr2=0x7f3fb81d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fadd9b640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fb80a6f00 0x7f3fb81d1530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000e7f0 tx=0x7f3fa000ecb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3fa0017070 con 0x7f3fb80a6f00 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 10 monclient: dump: 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.243 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa0013470 con 0x7f3fb80a6f00 2024-08-19T10:46:55.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 5 monclient: authenticate success, global_id 44500 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient: _renew_subs 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 1 -- 172.21.15.89:0/3798723473 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb812a1b0 con 0x7f3fb80a6f00 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3fa0013a90 con 0x7f3fb80a6f00 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3faf7fe640 10 monclient: dump: 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.245 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fb559a640 4 set_mon_vals no callback set 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient: _renew_subs 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.232+0000 7f3fbca32640 1 -- 172.21.15.89:0/3798723473 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb8129340 con 0x7f3fb80a6f00 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fbca32640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fbca32640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb81d1a70 0x7f3fb8126970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fbca32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fbca32640 1 -- 172.21.15.89:0/3798723473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb80a92d0 con 0x7f3fb81d1a70 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb81d1a70 0x7f3fb8126970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 10 monclient: get_auth_request con 0x7f3fb81d1a70 auth_method 0 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3fa001a070 con 0x7f3fb80a6f00 2024-08-19T10:46:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 10 monclient(hunting): handle_auth_done global_id 6597 payload 824 2024-08-19T10:46:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 1 -- 172.21.15.89:0/3798723473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3fb80a2580 con 0x7f3fb81d1a70 2024-08-19T10:46:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f9803be30 0x7f3f9803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f3fa000f040 con 0x7f3fb80a6f00 2024-08-19T10:46:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb6d9d640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f9803be30 0x7f3f9803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb6d9d640 10 monclient: get_auth_request con 0x7f3f9803be30 auth_method 0 2024-08-19T10:46:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 1 -- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb81d1a70 msgr2=0x7f3fb8126970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3fb659c640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb81d1a70 0x7f3fb8126970 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e590 tx=0x7f3fa800ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3fa8017070 con 0x7f3fb81d1a70 2024-08-19T10:46:55.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 10 monclient: dump: 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.251 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3fa8013970 con 0x7f3fb81d1a70 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb81d1a70 msgr2=0x7f3fb8126970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb81d1a70 0x7f3fb8126970 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e590 tx=0x7f3fa800ea50 comp rx=0 tx=0).stop 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb81d1a70 msgr2=0x7f3fb8126970 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.236+0000 7f3faf7fe640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb81d1a70 0x7f3fb8126970 secure :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e590 tx=0x7f3fa800ea50 comp rx=0 tx=0).stop 2024-08-19T10:46:55.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.240+0000 7f3fb6d9d640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f9803be30 0x7f3f9803e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3fb001c720 tx=0x7f3fb00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fbca32640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fbca32640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb8126eb0 0x7f3fb8122440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fbca32640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fbca32640 1 -- 172.21.15.89:0/3798723473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81272c0 con 0x7f3fb8126eb0 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fb659c640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb8126eb0 0x7f3fb8122440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fb659c640 10 monclient: get_auth_request con 0x7f3fb8126eb0 auth_method 0 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fb659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fb659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fb659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fb659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fb659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fb659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fb659c640 10 monclient(hunting): handle_auth_done global_id 6606 payload 824 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.244+0000 7f3fb659c640 1 -- 172.21.15.89:0/3798723473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3fb80a67f0 con 0x7f3fb8126eb0 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fb659c640 1 -- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb8126eb0 msgr2=0x7f3fb8122440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fb659c640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb8126eb0 0x7f3fb8122440 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e590 tx=0x7f3fa8004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3fa8017030 con 0x7f3fb8126eb0 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3faf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3faf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3faf7fe640 10 monclient: dump: 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.254 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9436 (secure 0 0 0) 0x7f3fa8006050 con 0x7f3fb8126eb0 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3faf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:55.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3faf7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:55.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb8126eb0 msgr2=0x7f3fb8122440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3faf7fe640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb8126eb0 0x7f3fb8122440 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e590 tx=0x7f3fa8004700 comp rx=0 tx=0).stop 2024-08-19T10:46:55.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3faf7fe640 1 -- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb8126eb0 msgr2=0x7f3fb8122440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:55.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3faf7fe640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fb8126eb0 0x7f3fb8122440 secure :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e590 tx=0x7f3fa8004700 comp rx=0 tx=0).stop 2024-08-19T10:46:55.256 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:55.256 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:55.256 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:55.256 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:46:55.256 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 545061, 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.257 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.258 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.259 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:55.260 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.261 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:55.262 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.263 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:55.265 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:55.266 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.266 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.266 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.266 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:55.266 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:55.266 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.266 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:55.267 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:55.268 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.269 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.270 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.271 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.272 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:55.274 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.277 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.278 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.279 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.280 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:55.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.280 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:55.280 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:55.280 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:46:55.280 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.280 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.280 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:55.280 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:55.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fbca32640 1 -- 172.21.15.89:0/3798723473 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f9803be30 msgr2=0x7f3f9803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fbca32640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f9803be30 0x7f3f9803e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3fb001c720 tx=0x7f3fb00a7070 comp rx=0 tx=0).stop 2024-08-19T10:46:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fbca32640 10 monclient: shutdown 2024-08-19T10:46:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fbca32640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fbca32640 1 -- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fb80a6f00 msgr2=0x7f3fb81d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fbca32640 1 --2- 172.21.15.89:0/3798723473 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3fb80a6f00 0x7f3fb81d1530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000e7f0 tx=0x7f3fa000ecb0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fb6d9d640 1 -- 172.21.15.89:0/3798723473 reap_dead start 2024-08-19T10:46:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fb6d9d640 1 -- 172.21.15.89:0/3798723473 reap_dead start 2024-08-19T10:46:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fbca32640 1 -- 172.21.15.89:0/3798723473 shutdown_connections 2024-08-19T10:46:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fbca32640 1 -- 172.21.15.89:0/3798723473 >> 172.21.15.89:0/3798723473 conn(0x7f3fb8069a80 msgr2=0x7f3fb809f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fbca32640 1 -- 172.21.15.89:0/3798723473 shutdown_connections 2024-08-19T10:46:55.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.248+0000 7f3fbca32640 1 -- 172.21.15.89:0/3798723473 wait complete. 2024-08-19T10:46:55.283 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:55.283 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:46:55.283 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:46:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient: get_monmap_and_config 2024-08-19T10:46:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient: build_initial_monmap 2024-08-19T10:46:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient: monmap: 2024-08-19T10:46:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:55.409345+0000 2024-08-19T10:46:55.406 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:55.409345+0000 2024-08-19T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:55.407 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:55.408 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 1 Processor -- start 2024-08-19T10:46:55.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 1 -- start start 2024-08-19T10:46:55.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient: init 2024-08-19T10:46:55.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:55.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient: _add_conns ranks=[3,8,1,0,2,10,9,6,7,5,4] 2024-08-19T10:46:55.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda200a5270 0x7fda200a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient(hunting): picked mon.noname-g con 0x7fda200a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:55.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda200a6d30 0x7fda200a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient(hunting): picked mon.noname-e con 0x7fda200a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fda201395b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda200a8710 con 0x7fda201395b0 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fda200a7650 con 0x7fda200a5270 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fda200a5a40 con 0x7fda200a6d30 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda254c1640 10 monclient(hunting): authenticate will time out at 1465.226004s 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda200a6d30 0x7fda200a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda200a6d30 0x7fda200a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:36118/0 (socket says 172.21.15.89:36118) 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 1 -- 172.21.15.89:0/328550738 learned_addr learned my addr 172.21.15.89:0/328550738 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:55.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 10 monclient(hunting): get_auth_request con 0x7fda200a6d30 auth_method 0 2024-08-19T10:46:55.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1effd640 1 --2- 172.21.15.89:0/328550738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda200a5270 0x7fda200a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1effd640 10 monclient(hunting): get_auth_request con 0x7fda200a5270 auth_method 0 2024-08-19T10:46:55.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_done global_id 6470 payload 274 2024-08-19T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 1 -- 172.21.15.89:0/328550738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda200a5270 msgr2=0x7fda200a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 1 --2- 172.21.15.89:0/328550738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda200a5270 0x7fda200a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 1 -- 172.21.15.89:0/328550738 >> v1:172.21.15.190:6789/0 conn(0x7fda201395b0 legacy=0x7fda20139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 1 monclient: found mon.noname-e 2024-08-19T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:46:55.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.408+0000 7fda1e7fc640 1 -- 172.21.15.89:0/328550738 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda200a81d0 con 0x7fda200a6d30 2024-08-19T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1e7fc640 1 -- 172.21.15.89:0/328550738 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda200a6d30 msgr2=0x7fda200a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1e7fc640 1 --2- 172.21.15.89:0/328550738 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda200a6d30 0x7fda200a7110 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fda140098a0 tx=0x7fda1401c260 comp rx=0 tx=0).ready entity=mon.8 client_cookie=320e9c79f24c5fe9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 1 -- 172.21.15.89:0/328550738 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda140b0070 con 0x7fda200a6d30 2024-08-19T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 10 monclient: dump: 2024-08-19T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.417 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.418 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 10 monclient: _finish_auth 0 2024-08-19T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 5 monclient: authenticate success, global_id 6470 2024-08-19T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 1 -- 172.21.15.89:0/328550738 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda140aa810 con 0x7fda200a6d30 2024-08-19T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:55.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 4 set_mon_vals no callback set 2024-08-19T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 1 -- 172.21.15.89:0/328550738 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda140a9300 con 0x7fda200a6d30 2024-08-19T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda1dffb640 10 monclient: dump: 2024-08-19T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.420 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.421 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.421 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.421 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.421 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.421 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.421 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.421 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.421 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.421 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.421 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.421 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 10 monclient: shutdown 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 1 -- 172.21.15.89:0/328550738 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda200a6d30 msgr2=0x7fda200a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 1 --2- 172.21.15.89:0/328550738 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda200a6d30 0x7fda200a7110 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fda140098a0 tx=0x7fda1401c260 comp rx=0 tx=0).stop 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 1 -- 172.21.15.89:0/328550738 shutdown_connections 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 1 --2- 172.21.15.89:0/328550738 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fda200a6d30 0x7fda200a7110 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 1 --2- 172.21.15.89:0/328550738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda200a5270 0x7fda200a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 1 -- 172.21.15.89:0/328550738 >> 172.21.15.89:0/328550738 conn(0x7fda20069a80 msgr2=0x7fda200abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 1 -- 172.21.15.89:0/328550738 shutdown_connections 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.412+0000 7fda254c1640 1 -- 172.21.15.89:0/328550738 wait complete. 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient: build_initial_monmap 2024-08-19T10:46:55.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient: monmap: 2024-08-19T10:46:55.423 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:55.423 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:55.423 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:55.423 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:55.423 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:55.423 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.423 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:55.423 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:55.423 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:55.423 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:55.424 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:55.424 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:55.424 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:55.424 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:55.424 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:55.424 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 1 Processor -- start 2024-08-19T10:46:55.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 1 -- start start 2024-08-19T10:46:55.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient: init 2024-08-19T10:46:55.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:55.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient: _add_conns ranks=[0,7,2,1,6,3,8,5,4] 2024-08-19T10:46:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda200a5270 0x7fda201d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient(hunting): picked mon.noname-a con 0x7fda200a5270 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:46:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fda200a6d30 0x7fda201386a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient(hunting): picked mon.noname-i con 0x7fda200a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:46:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda201395b0 0x7fda2013da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient(hunting): picked mon.noname-b con 0x7fda201395b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fda200a57d0 con 0x7fda200a5270 2024-08-19T10:46:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fda200a5650 con 0x7fda201395b0 2024-08-19T10:46:55.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fda20144240 con 0x7fda200a6d30 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda254c1640 10 monclient(hunting): authenticate will time out at 1465.233166s 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda200a5270 0x7fda201d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda200a5270 0x7fda201d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50638/0 (socket says 172.21.15.89:50638) 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 1 -- 172.21.15.89:0/3196101268 learned_addr learned my addr 172.21.15.89:0/3196101268 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 10 monclient(hunting): get_auth_request con 0x7fda200a5270 auth_method 0 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1f7fe640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda201395b0 0x7fda2013da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1f7fe640 10 monclient(hunting): get_auth_request con 0x7fda201395b0 auth_method 0 2024-08-19T10:46:55.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 10 monclient(hunting): handle_auth_done global_id 6615 payload 824 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 1 -- 172.21.15.89:0/3196101268 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fda200a6d30 msgr2=0x7fda201386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fda200a6d30 0x7fda201386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 1 -- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda201395b0 msgr2=0x7fda2013da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda201395b0 0x7fda2013da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 10 monclient: _finish_hunting 0 2024-08-19T10:46:55.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 1 monclient: found mon.noname-a 2024-08-19T10:46:55.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:55.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:46:55.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 1 -- 172.21.15.89:0/3196101268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda201d21e0 con 0x7fda200a5270 2024-08-19T10:46:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 1 -- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda200a5270 msgr2=0x7fda201d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.416+0000 7fda1effd640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda200a5270 0x7fda201d12c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fda1800d5f0 tx=0x7fda1800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda18012c70 con 0x7fda200a5270 2024-08-19T10:46:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:46:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: dump: 2024-08-19T10:46:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.433 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: _finish_auth 0 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: _check_auth_tickets 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda254c1640 5 monclient: authenticate success, global_id 6615 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda18019070 con 0x7fda200a5270 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda254c1640 10 monclient: _renew_subs 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda254c1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda254c1640 1 -- 172.21.15.89:0/3196101268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda200a71a0 con 0x7fda200a5270 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda18012e30 con 0x7fda200a5270 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1d7fa640 4 set_mon_vals no callback set 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: dump: 2024-08-19T10:46:55.435 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.436 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda254c1640 10 monclient: _renew_subs 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda254c1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda254c1640 1 -- 172.21.15.89:0/3196101268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda20122d50 con 0x7fda200a5270 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda254c1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda254c1640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda20138be0 0x7fda2002d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda254c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda254c1640 1 -- 172.21.15.89:0/3196101268 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fda200a2570 con 0x7fda20138be0 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda20138be0 0x7fda2002d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 10 monclient: get_auth_request con 0x7fda20138be0 auth_method 0 2024-08-19T10:46:55.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fda18012800 con 0x7fda200a5270 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_done global_id 54128 payload 824 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 1 -- 172.21.15.89:0/3196101268 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda2013fea0 con 0x7fda20138be0 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda0c03be00 0x7fda0c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fda1800f040 con 0x7fda200a5270 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1f7fe640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda0c03be00 0x7fda0c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1f7fe640 10 monclient: get_auth_request con 0x7fda0c03be00 auth_method 0 2024-08-19T10:46:55.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 1 -- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda20138be0 msgr2=0x7fda2002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda1e7fc640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda20138be0 0x7fda2002d1d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fda140ad000 tx=0x7fda14003850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda140b0040 con 0x7fda20138be0 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.420+0000 7fda077fe640 10 monclient: dump: 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.439 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.424+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fda14003ad0 con 0x7fda20138be0 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.424+0000 7fda077fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.424+0000 7fda077fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:55.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.424+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda20138be0 msgr2=0x7fda2002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.424+0000 7fda077fe640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda20138be0 0x7fda2002d1d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fda140ad000 tx=0x7fda14003850 comp rx=0 tx=0).stop 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.424+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda20138be0 msgr2=0x7fda2002d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.424+0000 7fda077fe640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda20138be0 0x7fda2002d1d0 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fda140ad000 tx=0x7fda14003850 comp rx=0 tx=0).stop 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.424+0000 7fda1f7fe640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda0c03be00 0x7fda0c03e2c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fda10005600 tx=0x7fda10003d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda254c1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda254c1640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2002d710 0x7fda2002daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda254c1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda254c1640 1 -- 172.21.15.89:0/3196101268 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fda200a67f0 con 0x7fda2002d710 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda1e7fc640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2002d710 0x7fda2002daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda1e7fc640 10 monclient: get_auth_request con 0x7fda2002d710 auth_method 0 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda1e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda1e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda1e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda1e7fc640 10 monclient(hunting): handle_auth_done global_id 54137 payload 824 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.428+0000 7fda1e7fc640 1 -- 172.21.15.89:0/3196101268 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fda20140ad0 con 0x7fda2002d710 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda1e7fc640 1 -- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2002d710 msgr2=0x7fda2002daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda1e7fc640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2002d710 0x7fda2002daf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fda14000c10 tx=0x7fda140a8000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fda140b0040 con 0x7fda2002d710 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda077fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda077fe640 10 monclient: dump: 2024-08-19T10:46:55.442 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.443 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.444 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7fda140bc720 con 0x7fda2002d710 2024-08-19T10:46:55.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda077fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:55.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda077fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:55.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2002d710 msgr2=0x7fda2002daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda077fe640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2002d710 0x7fda2002daf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fda14000c10 tx=0x7fda140a8000 comp rx=0 tx=0).stop 2024-08-19T10:46:55.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda077fe640 1 -- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2002d710 msgr2=0x7fda2002daf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:55.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda077fe640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fda2002d710 0x7fda2002daf0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fda14000c10 tx=0x7fda140a8000 comp rx=0 tx=0).stop 2024-08-19T10:46:55.444 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:55.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:55.445 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:55.445 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:55.445 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:55.445 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 10827, 2024-08-19T10:46:55.445 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:55.445 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:55.445 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:55.445 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:55.446 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:55.446 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:55.446 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:55.446 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:55.446 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:55.446 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:55.446 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.446 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:46:55.446 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:55.446 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.447 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.448 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:55.449 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.450 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:55.451 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.452 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:55.455 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.456 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:55.459 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:55.460 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.461 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.462 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:55.464 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.465 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.465 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:55.465 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.465 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.465 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.465 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:55.465 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:55.465 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.467 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:55.468 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:55.469 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:55.469 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.469 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.469 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:55.469 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:55.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda254c1640 1 -- 172.21.15.89:0/3196101268 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda0c03be00 msgr2=0x7fda0c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda254c1640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fda0c03be00 0x7fda0c03e2c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fda10005600 tx=0x7fda10003d50 comp rx=0 tx=0).stop 2024-08-19T10:46:55.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda254c1640 10 monclient: shutdown 2024-08-19T10:46:55.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda254c1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:55.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda254c1640 1 -- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda200a5270 msgr2=0x7fda201d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda254c1640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda200a5270 0x7fda201d12c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fda1800d5f0 tx=0x7fda1800dab0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda1f7fe640 1 -- 172.21.15.89:0/3196101268 reap_dead start 2024-08-19T10:46:55.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda254c1640 1 -- 172.21.15.89:0/3196101268 shutdown_connections 2024-08-19T10:46:55.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda254c1640 1 --2- 172.21.15.89:0/3196101268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda200a5270 0x7fda201d12c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda254c1640 1 -- 172.21.15.89:0/3196101268 >> 172.21.15.89:0/3196101268 conn(0x7fda20069a80 msgr2=0x7fda200ab210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:55.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda254c1640 1 -- 172.21.15.89:0/3196101268 shutdown_connections 2024-08-19T10:46:55.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.432+0000 7fda254c1640 1 -- 172.21.15.89:0/3196101268 wait complete. 2024-08-19T10:46:55.475 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:55.475 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:46:55.475 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:46:55.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.596+0000 7f910a116640 10 monclient: get_monmap_and_config 2024-08-19T10:46:55.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.596+0000 7f910a116640 10 monclient: build_initial_monmap 2024-08-19T10:46:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.596+0000 7f910a116640 10 monclient: monmap: 2024-08-19T10:46:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:55.600846+0000 2024-08-19T10:46:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:55.600846+0000 2024-08-19T10:46:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.598 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:55.599 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:55.599 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:55.599 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:55.599 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:55.599 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:55.599 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:55.599 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:55.599 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:55.599 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:55.599 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:55.599 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 1 Processor -- start 2024-08-19T10:46:55.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 1 -- start start 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 10 monclient: init 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 10 monclient: _add_conns ranks=[1,9,4,2,7,0,5,6,3,10,8] 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f91040a6860 addr v1:172.21.15.190:6789/0 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f91040a6c40 0x7f91040a7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 10 monclient(hunting): picked mon.noname-a con 0x7f91040a6c40 addr v2:172.21.15.89:3300/0 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910413e860 0x7f9104140c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 10 monclient(hunting): picked mon.noname-c con 0x7f910413e860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91040a61a0 con 0x7f91040a6860 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f91040a81b0 con 0x7f91040a6c40 2024-08-19T10:46:55.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f91040a4c90 con 0x7f910413e860 2024-08-19T10:46:55.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:55.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f910a116640 10 monclient(hunting): authenticate will time out at 1465.417501s 2024-08-19T10:46:55.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910413e860 0x7f9104140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910413e860 0x7f9104140c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42188/0 (socket says 172.21.15.89:42188) 2024-08-19T10:46:55.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9102ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f91040a6c40 0x7f91040a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 1 -- 172.21.15.89:0/745446041 learned_addr learned my addr 172.21.15.89:0/745446041 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:55.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 10 monclient(hunting): get_auth_request con 0x7f910413e860 auth_method 0 2024-08-19T10:46:55.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9102ffd640 10 monclient(hunting): get_auth_request con 0x7f91040a6c40 auth_method 0 2024-08-19T10:46:55.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9102ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9102ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9102ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9102ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9102ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9102ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 10 monclient(hunting): handle_auth_done global_id 44509 payload 274 2024-08-19T10:46:55.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 1 -- 172.21.15.89:0/745446041 >> v2:172.21.15.89:3300/0 conn(0x7f91040a6c40 msgr2=0x7f91040a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:55.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 1 --2- 172.21.15.89:0/745446041 >> v2:172.21.15.89:3300/0 conn(0x7f91040a6c40 0x7f91040a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 1 -- 172.21.15.89:0/745446041 >> v1:172.21.15.190:6789/0 conn(0x7f91040a6860 legacy=0x7f91040a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:55.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 10 monclient: _finish_hunting 0 2024-08-19T10:46:55.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 1 monclient: found mon.noname-c 2024-08-19T10:46:55.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:55.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:46:55.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9103fff640 1 -- 172.21.15.89:0/745446041 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91040a85c0 con 0x7f910413e860 2024-08-19T10:46:55.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.600+0000 7f9102ffd640 1 --2- 172.21.15.89:0/745446041 >> v2:172.21.15.89:3300/0 conn(0x7f91040a6c40 0x7f91040a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:55.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f9103fff640 1 -- 172.21.15.89:0/745446041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910413e860 msgr2=0x7f9104140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f9103fff640 1 --2- 172.21.15.89:0/745446041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910413e860 0x7f9104140c50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f91040acc40 tx=0x7f90fc01c8b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=398a6651745e0661 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 1 -- 172.21.15.89:0/745446041 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90fc0090d0 con 0x7f910413e860 2024-08-19T10:46:55.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:46:55.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 10 monclient: dump: 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.614 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 10 monclient: _finish_auth 0 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 10 monclient: _check_auth_tickets 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 1 -- 172.21.15.89:0/745446041 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc0a6970 con 0x7f910413e860 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 1 -- 172.21.15.89:0/745446041 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90fc0af480 con 0x7f910413e860 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f91027fc640 10 monclient: dump: 2024-08-19T10:46:55.615 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.616 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.617 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.617 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.617 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 5 monclient: authenticate success, global_id 44509 2024-08-19T10:46:55.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:55.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 4 set_mon_vals no callback set 2024-08-19T10:46:55.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 10 monclient: shutdown 2024-08-19T10:46:55.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:55.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 1 -- 172.21.15.89:0/745446041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910413e860 msgr2=0x7f9104140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 1 --2- 172.21.15.89:0/745446041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910413e860 0x7f9104140c50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f91040acc40 tx=0x7f90fc01c8b0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 1 -- 172.21.15.89:0/745446041 shutdown_connections 2024-08-19T10:46:55.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 1 --2- 172.21.15.89:0/745446041 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910413e860 0x7f9104140c50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 1 --2- 172.21.15.89:0/745446041 >> v2:172.21.15.89:3300/0 conn(0x7f91040a6c40 0x7f91040a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 1 -- 172.21.15.89:0/745446041 >> 172.21.15.89:0/745446041 conn(0x7f91040699d0 msgr2=0x7f91040abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:55.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 1 -- 172.21.15.89:0/745446041 shutdown_connections 2024-08-19T10:46:55.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 1 -- 172.21.15.89:0/745446041 wait complete. 2024-08-19T10:46:55.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 10 monclient: build_initial_monmap 2024-08-19T10:46:55.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.604+0000 7f910a116640 10 monclient: monmap: 2024-08-19T10:46:55.618 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:55.619 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 1 Processor -- start 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 1 -- start start 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient: init 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient: _add_conns ranks=[4,0,1,8,6,7,2,3,5] 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91040a6860 0x7f91041258f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient(hunting): picked mon.noname-c con 0x7f91040a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91040a6c40 0x7f910413c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient(hunting): picked mon.noname-a con 0x7f91040a6c40 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910413e860 0x7f9104123380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient(hunting): picked mon.noname-f con 0x7f910413e860 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:46:55.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9104143d40 con 0x7f91040a6c40 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9104143bc0 con 0x7f910413e860 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9104143ec0 con 0x7f91040a6860 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient(hunting): authenticate will time out at 1465.424932s 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f91037fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91040a6860 0x7f91041258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91040a6c40 0x7f910413c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f91037fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91040a6860 0x7f91041258f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42200/0 (socket says 172.21.15.89:42200) 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f91037fe640 1 -- 172.21.15.89:0/2661961556 learned_addr learned my addr 172.21.15.89:0/2661961556 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f91037fe640 10 monclient(hunting): get_auth_request con 0x7f91040a6860 auth_method 0 2024-08-19T10:46:55.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f91037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f91037fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f91037fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 10 monclient(hunting): get_auth_request con 0x7f91040a6c40 auth_method 0 2024-08-19T10:46:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f91037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f91037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f91037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 10 monclient(hunting): handle_auth_done global_id 6633 payload 824 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 1 -- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91040a6860 msgr2=0x7f91041258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91040a6860 0x7f91041258f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 1 -- 172.21.15.89:0/2661961556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910413e860 msgr2=0x7f9104123380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9103fff640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910413e860 0x7f9104123380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f910413e860 0x7f9104123380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 1 monclient: found mon.noname-a 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 1 -- 172.21.15.89:0/2661961556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91041d1be0 con 0x7f91040a6c40 2024-08-19T10:46:55.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f91037fe640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f91040a6860 0x7f91041258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 1 -- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91040a6c40 msgr2=0x7f910413c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9102ffd640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91040a6c40 0x7f910413c760 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f90f800e9b0 tx=0x7f90f800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90f8017070 con 0x7f91040a6c40 2024-08-19T10:46:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:46:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 10 monclient: dump: 2024-08-19T10:46:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.624 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 10 monclient: _finish_auth 0 2024-08-19T10:46:55.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 10 monclient: _check_auth_tickets 2024-08-19T10:46:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 5 monclient: authenticate success, global_id 6633 2024-08-19T10:46:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient: _renew_subs 2024-08-19T10:46:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f800cbb0 con 0x7f91040a6c40 2024-08-19T10:46:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f910a116640 1 -- 172.21.15.89:0/2661961556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9104124d90 con 0x7f91040a6c40 2024-08-19T10:46:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90f800cd60 con 0x7f91040a6c40 2024-08-19T10:46:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f9101ffb640 4 set_mon_vals no callback set 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.608+0000 7f90ebfff640 10 monclient: dump: 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.627 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.628 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f910a116640 10 monclient: _renew_subs 2024-08-19T10:46:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f910a116640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f910a116640 1 -- 172.21.15.89:0/2661961556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9104126280 con 0x7f91040a6c40 2024-08-19T10:46:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f910a116640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f910a116640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402d1d0 0x7f910402d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f910a116640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f910a116640 1 -- 172.21.15.89:0/2661961556 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9104064590 con 0x7f910402d1d0 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402d1d0 0x7f910402d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 10 monclient: get_auth_request con 0x7f910402d1d0 auth_method 0 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f9102ffd640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91040a6c40 0x7f910413c760 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f90f800e9b0 tx=0x7f90f800ee70 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f90f800f040 osd_map(271..271 src has 150..271) 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f90f801a070 con 0x7f91040a6c40 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 10 monclient(hunting): handle_auth_done global_id 44527 payload 824 2024-08-19T10:46:55.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 1 -- 172.21.15.89:0/2661961556 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91040a0100 con 0x7f910402d1d0 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f90f003be00 0x7f90f003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f9103fff640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f90f003be00 0x7f90f003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f9103fff640 10 monclient: get_auth_request con 0x7f90f003be00 auth_method 0 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 1 -- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402d1d0 msgr2=0x7f910402d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f91037fe640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402d1d0 0x7f910402d5b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90f400fb10 tx=0x7f90f40045a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90f4018030 con 0x7f910402d1d0 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 10 monclient: dump: 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.630 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f90f400de50 con 0x7f910402d1d0 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:55.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:55.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402d1d0 msgr2=0x7f910402d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402d1d0 0x7f910402d5b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90f400fb10 tx=0x7f90f40045a0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402d1d0 msgr2=0x7f910402d5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:55.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.612+0000 7f90ebfff640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402d1d0 0x7f910402d5b0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90f400fb10 tx=0x7f90f40045a0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.616+0000 7f9103fff640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f90f003be00 0x7f90f003e2c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9104075070 tx=0x7f90fc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f910a116640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f910a116640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402daf0 0x7f910402df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f910a116640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f910a116640 1 -- 172.21.15.89:0/2661961556 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f91041c8d50 con 0x7f910402daf0 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f91037fe640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402daf0 0x7f910402df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f91037fe640 10 monclient: get_auth_request con 0x7f910402daf0 auth_method 0 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f91037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f91037fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f91037fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f91037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f91037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f91037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f91037fe640 10 monclient(hunting): handle_auth_done global_id 44536 payload 824 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.620+0000 7f91037fe640 1 -- 172.21.15.89:0/2661961556 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f91040a7020 con 0x7f910402daf0 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f91037fe640 1 -- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402daf0 msgr2=0x7f910402df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f91037fe640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402daf0 0x7f910402df00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f90f400fb10 tx=0x7f90f4007f90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f90f4007490 con 0x7f910402daf0 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 10 monclient: dump: 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.635 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f90f40047b0 con 0x7f910402daf0 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:55.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402daf0 msgr2=0x7f910402df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402daf0 0x7f910402df00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f90f400fb10 tx=0x7f90f4007f90 comp rx=0 tx=0).stop 2024-08-19T10:46:55.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402daf0 msgr2=0x7f910402df00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:55.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f910402daf0 0x7f910402df00 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f90f400fb10 tx=0x7f90f4007f90 comp rx=0 tx=0).stop 2024-08-19T10:46:55.637 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:55.637 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:55.637 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:55.637 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:55.637 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:55.637 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 138750, 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:46:55.638 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.639 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.640 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:55.642 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:55.642 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:55.642 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:55.642 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:55.642 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:55.642 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:55.642 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.642 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.642 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.642 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.642 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:55.643 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.644 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.645 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.645 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:55.645 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:55.645 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.645 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.645 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.646 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:55.647 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.649 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:55.650 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:55.655 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:55.658 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:55.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.661 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:55.661 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:55.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f910a116640 1 -- 172.21.15.89:0/2661961556 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f90f003be00 msgr2=0x7f90f003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f910a116640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f90f003be00 0x7f90f003e2c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9104075070 tx=0x7f90fc0a7000 comp rx=0 tx=0).stop 2024-08-19T10:46:55.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f910a116640 10 monclient: shutdown 2024-08-19T10:46:55.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f910a116640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:55.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f910a116640 1 -- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91040a6c40 msgr2=0x7f910413c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f910a116640 1 --2- 172.21.15.89:0/2661961556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91040a6c40 0x7f910413c760 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f90f800e9b0 tx=0x7f90f800ee70 comp rx=0 tx=0).stop 2024-08-19T10:46:55.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f90ebfff640 1 -- 172.21.15.89:0/2661961556 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (unknown 0 0 0) 0x7f90f800f040 con 0x7f91040a6c40 2024-08-19T10:46:55.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f9103fff640 1 -- 172.21.15.89:0/2661961556 reap_dead start 2024-08-19T10:46:55.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f9103fff640 1 -- 172.21.15.89:0/2661961556 reap_dead start 2024-08-19T10:46:55.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f910a116640 1 -- 172.21.15.89:0/2661961556 shutdown_connections 2024-08-19T10:46:55.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f910a116640 1 -- 172.21.15.89:0/2661961556 >> 172.21.15.89:0/2661961556 conn(0x7f91040699d0 msgr2=0x7f910413f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:55.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f910a116640 1 -- 172.21.15.89:0/2661961556 shutdown_connections 2024-08-19T10:46:55.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.624+0000 7f910a116640 1 -- 172.21.15.89:0/2661961556 wait complete. 2024-08-19T10:46:55.663 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:55.663 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:46:55.664 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:46:55.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient: get_monmap_and_config 2024-08-19T10:46:55.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient: build_initial_monmap 2024-08-19T10:46:55.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient: monmap: 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:55.781622+0000 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:55.781622+0000 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:55.779 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:55.780 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:55.780 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:55.780 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:55.780 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:55.780 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:55.780 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 1 Processor -- start 2024-08-19T10:46:55.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 1 -- start start 2024-08-19T10:46:55.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient: init 2024-08-19T10:46:55.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:55.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient: _add_conns ranks=[6,5,2,9,1,3,4,10,0,8,7] 2024-08-19T10:46:55.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c80a5270 0x7f09c80a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient(hunting): picked mon.noname-d con 0x7f09c80a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:55.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09c80a6d30 0x7f09c80a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient(hunting): picked mon.noname-h con 0x7f09c80a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:55.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09c81395b0 0x7f09c81399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient(hunting): picked mon.noname-b con 0x7f09c81395b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:55.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f09c80a7650 con 0x7f09c81395b0 2024-08-19T10:46:55.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f09c80a8710 con 0x7f09c80a6d30 2024-08-19T10:46:55.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f09c80a5a40 con 0x7f09c80a5270 2024-08-19T10:46:55.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:55.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ce965640 10 monclient(hunting): authenticate will time out at 1465.598295s 2024-08-19T10:46:55.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c80a5270 0x7f09c80a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c80a5270 0x7f09c80a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:35518/0 (socket says 172.21.15.89:35518) 2024-08-19T10:46:55.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 1 -- 172.21.15.89:0/1436538729 learned_addr learned my addr 172.21.15.89:0/1436538729 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:55.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 10 monclient(hunting): get_auth_request con 0x7f09c80a5270 auth_method 0 2024-08-19T10:46:55.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ccf02640 1 --2- 172.21.15.89:0/1436538729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09c81395b0 0x7f09c81399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ccf02640 10 monclient(hunting): get_auth_request con 0x7f09c81395b0 auth_method 0 2024-08-19T10:46:55.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ccf02640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ccf02640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ccf02640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ccf02640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ccf02640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ccf02640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c77fe640 1 --2- 172.21.15.89:0/1436538729 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09c80a6d30 0x7f09c80a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c77fe640 10 monclient(hunting): get_auth_request con 0x7f09c80a6d30 auth_method 0 2024-08-19T10:46:55.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 10 monclient(hunting): handle_auth_done global_id 44520 payload 274 2024-08-19T10:46:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 1 -- 172.21.15.89:0/1436538729 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09c80a6d30 msgr2=0x7f09c80a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 1 --2- 172.21.15.89:0/1436538729 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09c80a6d30 0x7f09c80a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 1 -- 172.21.15.89:0/1436538729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09c81395b0 msgr2=0x7f09c81399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 1 --2- 172.21.15.89:0/1436538729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09c81395b0 0x7f09c81399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:46:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 1 monclient: found mon.noname-d 2024-08-19T10:46:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:46:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09c7fff640 1 -- 172.21.15.89:0/1436538729 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c80a81d0 con 0x7f09c80a5270 2024-08-19T10:46:55.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.780+0000 7f09ccf02640 1 --2- 172.21.15.89:0/1436538729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09c81395b0 0x7f09c81399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c7fff640 1 -- 172.21.15.89:0/1436538729 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c80a5270 msgr2=0x7f09c80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c7fff640 1 --2- 172.21.15.89:0/1436538729 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c80a5270 0x7f09c80a67f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009e80 tx=0x7f09c001c910 comp rx=0 tx=0).ready entity=mon.6 client_cookie=3744e06e38c2276f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 1 -- 172.21.15.89:0/1436538729 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09c00090d0 con 0x7f09c80a5270 2024-08-19T10:46:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:46:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 10 monclient: dump: 2024-08-19T10:46:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.796 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.797 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.797 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:46:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:46:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 1 -- 172.21.15.89:0/1436538729 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c00a5a50 con 0x7f09c80a5270 2024-08-19T10:46:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 5 monclient: authenticate success, global_id 44520 2024-08-19T10:46:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:46:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:55.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 1 -- 172.21.15.89:0/1436538729 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09c00a5c00 con 0x7f09c80a5270 2024-08-19T10:46:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 4 set_mon_vals no callback set 2024-08-19T10:46:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:46:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09c6ffd640 10 monclient: dump: 2024-08-19T10:46:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.799 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.800 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.801 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 10 monclient: shutdown 2024-08-19T10:46:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 1 -- 172.21.15.89:0/1436538729 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c80a5270 msgr2=0x7f09c80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 1 --2- 172.21.15.89:0/1436538729 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c80a5270 0x7f09c80a67f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009e80 tx=0x7f09c001c910 comp rx=0 tx=0).stop 2024-08-19T10:46:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 1 -- 172.21.15.89:0/1436538729 shutdown_connections 2024-08-19T10:46:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 1 --2- 172.21.15.89:0/1436538729 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f09c81395b0 0x7f09c81399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 1 --2- 172.21.15.89:0/1436538729 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09c80a6d30 0x7f09c80a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 1 --2- 172.21.15.89:0/1436538729 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c80a5270 0x7f09c80a67f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 1 -- 172.21.15.89:0/1436538729 >> 172.21.15.89:0/1436538729 conn(0x7f09c8069a80 msgr2=0x7f09c80abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:55.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 1 -- 172.21.15.89:0/1436538729 shutdown_connections 2024-08-19T10:46:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.784+0000 7f09ce965640 1 -- 172.21.15.89:0/1436538729 wait complete. 2024-08-19T10:46:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient: build_initial_monmap 2024-08-19T10:46:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient: monmap: 2024-08-19T10:46:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:55.802 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 1 Processor -- start 2024-08-19T10:46:55.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 1 -- start start 2024-08-19T10:46:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient: init 2024-08-19T10:46:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient: _add_conns ranks=[3,0,5,6,2,8,1,4,7] 2024-08-19T10:46:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c80a5270 0x7f09c8132800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient(hunting): picked mon.noname-g con 0x7f09c80a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c80a6d30 0x7f09c8132d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient(hunting): picked mon.noname-a con 0x7f09c80a6d30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:46:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09c81395b0 0x7f09c813dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient(hunting): picked mon.noname-h con 0x7f09c81395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f09c80a57d0 con 0x7f09c80a6d30 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f09c80a5650 con 0x7f09c80a5270 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f09c8143230 con 0x7f09c81395b0 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ce965640 10 monclient(hunting): authenticate will time out at 1465.605462s 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c80a6d30 0x7f09c8132d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c80a6d30 0x7f09c8132d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50656/0 (socket says 172.21.15.89:50656) 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 1 -- 172.21.15.89:0/3517699745 learned_addr learned my addr 172.21.15.89:0/3517699745 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 10 monclient(hunting): get_auth_request con 0x7f09c80a6d30 auth_method 0 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ccf02640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09c81395b0 0x7f09c813dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c7fff640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c80a5270 0x7f09c8132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ccf02640 10 monclient(hunting): get_auth_request con 0x7f09c81395b0 auth_method 0 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ccf02640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ccf02640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ccf02640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c7fff640 10 monclient(hunting): get_auth_request con 0x7f09c80a5270 auth_method 0 2024-08-19T10:46:55.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ccf02640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ccf02640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09ccf02640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 10 monclient(hunting): handle_auth_done global_id 6642 payload 824 2024-08-19T10:46:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 1 -- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09c81395b0 msgr2=0x7f09c813dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f09c81395b0 0x7f09c813dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 1 -- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c80a5270 msgr2=0x7f09c8132800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c80a5270 0x7f09c8132800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:46:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 1 monclient: found mon.noname-a 2024-08-19T10:46:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:46:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c77fe640 1 -- 172.21.15.89:0/3517699745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c8140680 con 0x7f09c80a6d30 2024-08-19T10:46:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.788+0000 7f09c7fff640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09c80a5270 0x7f09c8132800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:46:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c77fe640 1 -- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c80a6d30 msgr2=0x7f09c8132d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c77fe640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c80a6d30 0x7f09c8132d40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f09b000d800 tx=0x7f09b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09b000bd40 con 0x7f09c80a6d30 2024-08-19T10:46:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:46:55.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 10 monclient: dump: 2024-08-19T10:46:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.809 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.810 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 10 monclient: _finish_auth 0 2024-08-19T10:46:55.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b0019070 con 0x7f09c80a6d30 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ce965640 5 monclient: authenticate success, global_id 6642 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ce965640 10 monclient: _renew_subs 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ce965640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ce965640 1 -- 172.21.15.89:0/3517699745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c813f710 con 0x7f09c80a6d30 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09b00148a0 con 0x7f09c80a6d30 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c67fc640 4 set_mon_vals no callback set 2024-08-19T10:46:55.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 10 monclient: dump: 2024-08-19T10:46:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.812 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.813 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ce965640 10 monclient: _renew_subs 2024-08-19T10:46:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ce965640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:55.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ce965640 1 -- 172.21.15.89:0/3517699745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c809c9e0 con 0x7f09c80a6d30 2024-08-19T10:46:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ce965640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ce965640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813e2a0 0x7f09c813e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ce965640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ce965640 1 -- 172.21.15.89:0/3517699745 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f09c80a73b0 con 0x7f09c813e2a0 2024-08-19T10:46:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f09b0014430 con 0x7f09c80a6d30 2024-08-19T10:46:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c7fff640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813e2a0 0x7f09c813e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c7fff640 10 monclient: get_auth_request con 0x7f09c813e2a0 auth_method 0 2024-08-19T10:46:55.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f09a803be30 0x7f09a803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f09b000f040 con 0x7f09c80a6d30 2024-08-19T10:46:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c7fff640 10 monclient(hunting): handle_auth_done global_id 44529 payload 824 2024-08-19T10:46:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09c7fff640 1 -- 172.21.15.89:0/3517699745 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09c80a67f0 con 0x7f09c813e2a0 2024-08-19T10:46:55.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ccf02640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f09a803be30 0x7f09a803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.792+0000 7f09ccf02640 10 monclient: get_auth_request con 0x7f09a803be30 auth_method 0 2024-08-19T10:46:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c7fff640 1 -- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813e2a0 msgr2=0x7f09c813e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c7fff640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813e2a0 0x7f09c813e680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09c00ae000 tx=0x7f09c00196b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09c00bd020 con 0x7f09c813e2a0 2024-08-19T10:46:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c4ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:46:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c4ff9640 10 monclient: dump: 2024-08-19T10:46:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.816 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.817 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.818 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f09c0004250 con 0x7f09c813e2a0 2024-08-19T10:46:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c4ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813e2a0 msgr2=0x7f09c813e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c4ff9640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813e2a0 0x7f09c813e680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f09c00ae000 tx=0x7f09c00196b0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813e2a0 msgr2=0x7f09c813e680 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:55.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09c4ff9640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813e2a0 0x7f09c813e680 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.796+0000 7f09ccf02640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f09a803be30 0x7f09a803e2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009580 tx=0x7f09b8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09ce965640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09ce965640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813ebc0 0x7f09c81379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09ce965640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09ce965640 1 -- 172.21.15.89:0/3517699745 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f09c8064590 con 0x7f09c813ebc0 2024-08-19T10:46:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09c7fff640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813ebc0 0x7f09c81379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09c7fff640 10 monclient: get_auth_request con 0x7f09c813ebc0 auth_method 0 2024-08-19T10:46:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09c7fff640 10 monclient(hunting): handle_auth_done global_id 44538 payload 824 2024-08-19T10:46:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.800+0000 7f09c7fff640 1 -- 172.21.15.89:0/3517699745 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f09c806c9e0 con 0x7f09c813ebc0 2024-08-19T10:46:55.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c7fff640 1 -- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813ebc0 msgr2=0x7f09c81379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c7fff640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813ebc0 0x7f09c81379b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009e80 tx=0x7f09c00a7000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09c00bd040 con 0x7f09c813ebc0 2024-08-19T10:46:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c4ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:46:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c4ff9640 10 monclient: dump: 2024-08-19T10:46:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:55.822 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:55.823 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f09c00c8070 con 0x7f09c813ebc0 2024-08-19T10:46:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c4ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813ebc0 msgr2=0x7f09c81379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c4ff9640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813ebc0 0x7f09c81379b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009e80 tx=0x7f09c00a7000 comp rx=0 tx=0).stop 2024-08-19T10:46:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c4ff9640 1 -- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813ebc0 msgr2=0x7f09c81379b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:55.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09c4ff9640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09c813ebc0 0x7f09c81379b0 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009e80 tx=0x7f09c00a7000 comp rx=0 tx=0).stop 2024-08-19T10:46:55.824 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:55.825 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:55.825 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:55.825 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:55.825 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:55.825 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 138930, 2024-08-19T10:46:55.825 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:55.825 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:55.825 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:55.825 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:55.825 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:55.825 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:55.826 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:55.826 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:55.826 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:55.826 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:55.826 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.826 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:46:55.826 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:55.826 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:55.826 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:55.826 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.826 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:55.827 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.828 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:55.829 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:55.830 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.831 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:55.832 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.833 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.841 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.843 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.845 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:55.849 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:55.850 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:55.851 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:55.851 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:55.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ce965640 1 -- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f09a803be30 msgr2=0x7f09a803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ce965640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f09a803be30 0x7f09a803e2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009580 tx=0x7f09b8009190 comp rx=0 tx=0).stop 2024-08-19T10:46:55.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ce965640 10 monclient: shutdown 2024-08-19T10:46:55.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ce965640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:55.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ce965640 1 -- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c80a6d30 msgr2=0x7f09c8132d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:55.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ce965640 1 --2- 172.21.15.89:0/3517699745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09c80a6d30 0x7f09c8132d40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f09b000d800 tx=0x7f09b000dcc0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ccf02640 1 -- 172.21.15.89:0/3517699745 reap_dead start 2024-08-19T10:46:55.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ccf02640 1 -- 172.21.15.89:0/3517699745 reap_dead start 2024-08-19T10:46:55.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ce965640 1 -- 172.21.15.89:0/3517699745 shutdown_connections 2024-08-19T10:46:55.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ce965640 1 -- 172.21.15.89:0/3517699745 >> 172.21.15.89:0/3517699745 conn(0x7f09c8069a80 msgr2=0x7f09c813ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:55.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ce965640 1 -- 172.21.15.89:0/3517699745 shutdown_connections 2024-08-19T10:46:55.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.804+0000 7f09ce965640 1 -- 172.21.15.89:0/3517699745 wait complete. 2024-08-19T10:46:55.854 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:55.854 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:46:55.855 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:46:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.988+0000 7feef8d82640 10 monclient: get_monmap_and_config 2024-08-19T10:46:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.988+0000 7feef8d82640 10 monclient: build_initial_monmap 2024-08-19T10:46:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.988+0000 7feef8d82640 10 monclient: monmap: 2024-08-19T10:46:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:55.992245+0000 2024-08-19T10:46:55.989 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:55.992245+0000 2024-08-19T10:46:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:55.990 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:55.991 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:55.991 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 1 Processor -- start 2024-08-19T10:46:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 1 -- start start 2024-08-19T10:46:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 10 monclient: init 2024-08-19T10:46:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 10 monclient: _add_conns ranks=[7,9,10,3,2,6,8,4,1,5,0] 2024-08-19T10:46:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7feef40a58c0 0x7feef4072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 10 monclient(hunting): picked mon.noname-i con 0x7feef40a58c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:46:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feef40a5ca0 0x7feef40a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 10 monclient(hunting): picked mon.noname-a con 0x7feef40a5ca0 addr v2:172.21.15.89:3300/0 2024-08-19T10:46:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feef4139690 0x7feef4139b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 10 monclient(hunting): picked mon.noname-f con 0x7feef4139690 addr v2:172.21.15.190:3300/0 2024-08-19T10:46:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7feef406dfe0 con 0x7feef40a5ca0 2024-08-19T10:46:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7feef406f360 con 0x7feef4139690 2024-08-19T10:46:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7feef406f080 con 0x7feef40a58c0 2024-08-19T10:46:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef8d82640 10 monclient(hunting): authenticate will time out at 1465.808745s 2024-08-19T10:46:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feef40a5ca0 0x7feef40a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7feef40a5ca0 0x7feef40a6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50670/0 (socket says 172.21.15.89:50670) 2024-08-19T10:46:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 1 -- 172.21.15.89:0/1303167287 learned_addr learned my addr 172.21.15.89:0/1303167287 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 10 monclient(hunting): get_auth_request con 0x7feef40a5ca0 auth_method 0 2024-08-19T10:46:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef259c640 1 --2- 172.21.15.89:0/1303167287 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7feef40a58c0 0x7feef4072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef2d9d640 1 --2- 172.21.15.89:0/1303167287 >> v2:172.21.15.190:3300/0 conn(0x7feef4139690 0x7feef4139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef259c640 10 monclient(hunting): get_auth_request con 0x7feef40a58c0 auth_method 0 2024-08-19T10:46:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef2d9d640 10 monclient(hunting): get_auth_request con 0x7feef4139690 auth_method 0 2024-08-19T10:46:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef2d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 10 monclient(hunting): handle_auth_done global_id 6651 payload 274 2024-08-19T10:46:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 1 -- 172.21.15.89:0/1303167287 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7feef40a58c0 msgr2=0x7feef4072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 1 --2- 172.21.15.89:0/1303167287 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7feef40a58c0 0x7feef4072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 1 -- 172.21.15.89:0/1303167287 >> v2:172.21.15.190:3300/0 conn(0x7feef4139690 msgr2=0x7feef4139b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 1 --2- 172.21.15.89:0/1303167287 >> v2:172.21.15.190:3300/0 conn(0x7feef4139690 0x7feef4139b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:46:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 1 monclient: found mon.noname-a 2024-08-19T10:46:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:46:55.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.992+0000 7feef1d9b640 1 -- 172.21.15.89:0/1303167287 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef406f640 con 0x7feef40a5ca0 2024-08-19T10:46:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef1d9b640 1 -- 172.21.15.89:0/1303167287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a5ca0 msgr2=0x7feef40a6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef1d9b640 1 --2- 172.21.15.89:0/1303167287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a5ca0 0x7feef40a6080 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feef4072570 tx=0x7feee801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7c59239d930588 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 1 -- 172.21.15.89:0/1303167287 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7feee80b1070 con 0x7feef40a5ca0 2024-08-19T10:46:56.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:46:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 10 monclient: dump: 2024-08-19T10:46:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.006 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.007 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 1 -- 172.21.15.89:0/1303167287 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee80ab810 con 0x7feef40a5ca0 2024-08-19T10:46:56.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 1 -- 172.21.15.89:0/1303167287 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7feee80aa300 con 0x7feef40a5ca0 2024-08-19T10:46:56.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 5 monclient: authenticate success, global_id 6651 2024-08-19T10:46:56.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:56.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 4 set_mon_vals no callback set 2024-08-19T10:46:56.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:56.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef159a640 10 monclient: dump: 2024-08-19T10:46:56.008 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.008 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.008 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.009 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.009 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.009 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.009 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.009 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.009 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.009 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.009 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.009 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.009 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.009 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.010 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.010 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 10 monclient: shutdown 2024-08-19T10:46:56.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:56.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 1 -- 172.21.15.89:0/1303167287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a5ca0 msgr2=0x7feef40a6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 1 --2- 172.21.15.89:0/1303167287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a5ca0 0x7feef40a6080 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feef4072570 tx=0x7feee801c3e0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 1 -- 172.21.15.89:0/1303167287 shutdown_connections 2024-08-19T10:46:56.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 1 --2- 172.21.15.89:0/1303167287 >> v2:172.21.15.190:3300/0 conn(0x7feef4139690 0x7feef4139b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 1 --2- 172.21.15.89:0/1303167287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a5ca0 0x7feef40a6080 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 1 --2- 172.21.15.89:0/1303167287 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7feef40a58c0 0x7feef4072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 1 -- 172.21.15.89:0/1303167287 >> 172.21.15.89:0/1303167287 conn(0x7feef40aa4b0 msgr2=0x7feef40a69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:56.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 1 -- 172.21.15.89:0/1303167287 shutdown_connections 2024-08-19T10:46:56.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 1 -- 172.21.15.89:0/1303167287 wait complete. 2024-08-19T10:46:56.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 10 monclient: build_initial_monmap 2024-08-19T10:46:56.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 10 monclient: monmap: 2024-08-19T10:46:56.011 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:56.011 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:56.011 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:56.011 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:56.011 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:56.011 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.011 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:56.012 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:56.012 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:56.012 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:56.012 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:56.012 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:56.012 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:56.012 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:56.012 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:56.012 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 1 Processor -- start 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 1 -- start start 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:55.996+0000 7feef8d82640 10 monclient: init 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient: _add_conns ranks=[0,5,2,8,6,7,1,4,3] 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a58c0 0x7feef41d1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient(hunting): picked mon.noname-a con 0x7feef40a58c0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7feef40a5ca0 0x7feef41386c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient(hunting): picked mon.noname-h con 0x7feef40a5ca0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feef4139690 0x7feef4120500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient(hunting): picked mon.noname-b con 0x7feef4139690 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:56.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7feef406e860 con 0x7feef40a58c0 2024-08-19T10:46:56.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7feef406e6e0 con 0x7feef4139690 2024-08-19T10:46:56.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7feef4143100 con 0x7feef40a5ca0 2024-08-19T10:46:56.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:56.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient(hunting): authenticate will time out at 1465.816201s 2024-08-19T10:46:56.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a58c0 0x7feef41d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a58c0 0x7feef41d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50678/0 (socket says 172.21.15.89:50678) 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 1 -- 172.21.15.89:0/2724315127 learned_addr learned my addr 172.21.15.89:0/2724315127 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 10 monclient(hunting): get_auth_request con 0x7feef40a58c0 auth_method 0 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef2d9d640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feef4139690 0x7feef4120500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef2d9d640 10 monclient(hunting): get_auth_request con 0x7feef4139690 auth_method 0 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef2d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 10 monclient(hunting): handle_auth_done global_id 6660 payload 824 2024-08-19T10:46:56.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef1d9b640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7feef40a5ca0 0x7feef41386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 1 -- 172.21.15.89:0/2724315127 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7feef40a5ca0 msgr2=0x7feef41386c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7feef40a5ca0 0x7feef41386c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 1 -- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feef4139690 msgr2=0x7feef4120500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7feef4139690 0x7feef4120500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 10 monclient: _finish_hunting 0 2024-08-19T10:46:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 1 monclient: found mon.noname-a 2024-08-19T10:46:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:46:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 1 -- 172.21.15.89:0/2724315127 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef40a8550 con 0x7feef40a58c0 2024-08-19T10:46:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 1 -- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a58c0 msgr2=0x7feef41d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef259c640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a58c0 0x7feef41d1090 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feee400d830 tx=0x7feee400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7feee400bd40 con 0x7feef40a58c0 2024-08-19T10:46:56.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:46:56.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 10 monclient: dump: 2024-08-19T10:46:56.018 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.018 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.018 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.018 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.018 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.018 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.018 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.018 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.019 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.019 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.019 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.019 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.019 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.019 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.019 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.019 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 5 monclient: authenticate success, global_id 6660 2024-08-19T10:46:56.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee4019070 con 0x7feef40a58c0 2024-08-19T10:46:56.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient: _renew_subs 2024-08-19T10:46:56.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:56.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 1 -- 172.21.15.89:0/2724315127 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef4139460 con 0x7feef40a58c0 2024-08-19T10:46:56.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7feee40148a0 con 0x7feef40a58c0 2024-08-19T10:46:56.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee3fff640 4 set_mon_vals no callback set 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feee2ffd640 10 monclient: dump: 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.021 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.022 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.022 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.022 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.022 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.022 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.022 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.022 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient: _renew_subs 2024-08-19T10:46:56.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:56.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.000+0000 7feef8d82640 1 -- 172.21.15.89:0/2724315127 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef4123aa0 con 0x7feef40a58c0 2024-08-19T10:46:56.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef8d82640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef8d82640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4138c00 0x7feef4120a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef8d82640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef8d82640 1 -- 172.21.15.89:0/2724315127 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7feef4069f50 con 0x7feef4138c00 2024-08-19T10:46:56.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4138c00 0x7feef4120a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 10 monclient: get_auth_request con 0x7feef4138c00 auth_method 0 2024-08-19T10:46:56.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7feee4014430 con 0x7feef40a58c0 2024-08-19T10:46:56.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 10 monclient(hunting): handle_auth_done global_id 6479 payload 824 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7feed403be30 0x7feed403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 1 -- 172.21.15.89:0/2724315127 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feef40a6080 con 0x7feef4138c00 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7feee400f040 con 0x7feef40a58c0 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef2d9d640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7feed403be30 0x7feed403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef2d9d640 10 monclient: get_auth_request con 0x7feed403be30 auth_method 0 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 1 -- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4138c00 msgr2=0x7feef4120a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef1d9b640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4138c00 0x7feef4120a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feee80ae000 tx=0x7feef40a75f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7feee80bc070 con 0x7feef4138c00 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 10 monclient: dump: 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.025 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.026 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.026 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.026 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.026 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.026 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7feee8003830 con 0x7feef4138c00 2024-08-19T10:46:56.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:56.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4138c00 msgr2=0x7feef4120a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4138c00 0x7feef4120a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feee80ae000 tx=0x7feef40a75f0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4138c00 msgr2=0x7feef4120a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:56.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feee2ffd640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4138c00 0x7feef4120a40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.004+0000 7feef2d9d640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7feed403be30 0x7feed403e2f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feeec0039a0 tx=0x7feeec009780 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef8d82640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:56.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef8d82640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4120f80 0x7feef413a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef8d82640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef8d82640 1 -- 172.21.15.89:0/2724315127 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7feef4121360 con 0x7feef4120f80 2024-08-19T10:46:56.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef1d9b640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4120f80 0x7feef413a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef1d9b640 10 monclient: get_auth_request con 0x7feef4120f80 auth_method 0 2024-08-19T10:46:56.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef1d9b640 10 monclient(hunting): handle_auth_done global_id 6488 payload 824 2024-08-19T10:46:56.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.012+0000 7feef1d9b640 1 -- 172.21.15.89:0/2724315127 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7feef4064590 con 0x7feef4120f80 2024-08-19T10:46:56.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef1d9b640 1 -- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4120f80 msgr2=0x7feef413a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef1d9b640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4120f80 0x7feef413a6a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feee8009870 tx=0x7feee80a7000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7feee80bc040 con 0x7feef4120f80 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feee2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feee2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feee2ffd640 10 monclient: dump: 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.031 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.032 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.032 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.032 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.032 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.032 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.032 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.032 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7feee80b78f0 con 0x7feef4120f80 2024-08-19T10:46:56.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feee2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:56.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feee2ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:56.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4120f80 msgr2=0x7feef413a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feee2ffd640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4120f80 0x7feef413a6a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feee8009870 tx=0x7feee80a7000 comp rx=0 tx=0).stop 2024-08-19T10:46:56.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feee2ffd640 1 -- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4120f80 msgr2=0x7feef413a6a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:56.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feee2ffd640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7feef4120f80 0x7feef413a6a0 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feee8009870 tx=0x7feee80a7000 comp rx=0 tx=0).stop 2024-08-19T10:46:56.033 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:56.034 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:56.034 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:56.034 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:56.034 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:56.034 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 545809, 2024-08-19T10:46:56.034 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:56.034 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:56.034 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:56.035 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:56.035 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:56.035 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:56.035 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:56.035 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:56.035 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:56.035 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:56.035 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.035 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:46:56.035 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:56.035 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.036 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.037 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.037 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:56.037 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:56.037 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.037 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.037 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.037 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.037 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.037 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.037 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.037 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.038 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.038 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.038 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.038 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.038 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:56.038 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:56.038 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:56.038 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:56.038 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:56.039 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:56.039 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:56.039 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:56.039 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:56.039 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:56.039 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:56.039 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:56.039 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:56.039 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:56.039 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:56.039 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:56.040 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:56.040 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.040 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.040 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.040 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.040 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.040 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.040 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.040 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.040 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.040 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.041 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.041 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.041 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:56.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.041 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:56.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.041 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:56.041 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:56.041 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.044 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.044 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:56.044 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.044 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.044 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:56.044 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.044 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.044 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.045 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:56.045 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:56.045 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.045 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.045 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.045 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:56.045 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:56.046 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.046 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.046 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.046 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.046 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:56.046 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.046 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.046 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.046 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:56.047 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.047 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.047 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:56.047 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:56.047 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.047 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.047 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.048 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:56.048 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:56.048 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.048 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.048 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.048 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:56.048 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.049 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.049 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:56.049 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.049 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.050 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.050 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:56.050 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:56.050 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.050 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.050 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.050 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.051 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:56.051 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:56.051 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.051 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:56.051 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.052 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.053 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.053 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:56.053 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:56.053 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.053 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.053 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.053 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:56.054 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.054 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.054 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.054 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:56.054 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.054 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.054 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.054 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:56.054 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:56.055 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.055 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.055 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.055 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.055 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.055 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:56.055 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:56.055 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.055 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.055 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.055 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:56.056 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.056 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.056 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:56.056 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.056 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.056 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.057 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:56.058 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:56.058 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.058 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.058 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.059 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:56.059 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:56.059 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.059 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.059 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.059 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:56.059 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:56.059 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.059 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.060 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.061 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:56.061 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:56.061 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.061 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.061 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.061 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.061 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.061 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.062 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.063 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:56.064 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:56.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef8d82640 1 -- 172.21.15.89:0/2724315127 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7feed403be30 msgr2=0x7feed403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef8d82640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7feed403be30 0x7feed403e2f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feeec0039a0 tx=0x7feeec009780 comp rx=0 tx=0).stop 2024-08-19T10:46:56.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef8d82640 10 monclient: shutdown 2024-08-19T10:46:56.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef8d82640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:56.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef8d82640 1 -- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a58c0 msgr2=0x7feef41d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef8d82640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a58c0 0x7feef41d1090 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feee400d830 tx=0x7feee400dcf0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef2d9d640 1 -- 172.21.15.89:0/2724315127 reap_dead start 2024-08-19T10:46:56.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef8d82640 1 -- 172.21.15.89:0/2724315127 shutdown_connections 2024-08-19T10:46:56.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef8d82640 1 --2- 172.21.15.89:0/2724315127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feef40a58c0 0x7feef41d1090 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef8d82640 1 -- 172.21.15.89:0/2724315127 >> 172.21.15.89:0/2724315127 conn(0x7feef40aa4b0 msgr2=0x7feef413fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:56.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef8d82640 1 -- 172.21.15.89:0/2724315127 shutdown_connections 2024-08-19T10:46:56.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.016+0000 7feef8d82640 1 -- 172.21.15.89:0/2724315127 wait complete. 2024-08-19T10:46:56.066 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:56.067 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:46:56.067 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:46:56.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient: get_monmap_and_config 2024-08-19T10:46:56.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient: build_initial_monmap 2024-08-19T10:46:56.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient: monmap: 2024-08-19T10:46:56.182 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:56.182 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.182 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:56.185182+0000 2024-08-19T10:46:56.182 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:56.185182+0000 2024-08-19T10:46:56.182 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:56.183 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 1 Processor -- start 2024-08-19T10:46:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 1 -- start start 2024-08-19T10:46:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient: init 2024-08-19T10:46:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient: _add_conns ranks=[3,7,4,6,1,8,10,9,2,0,5] 2024-08-19T10:46:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4080a6860 0x7fc4080a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient(hunting): picked mon.noname-g con 0x7fc4080a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc4080a7180 0x7fc4080a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient(hunting): picked mon.noname-i con 0x7fc4080a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc408139620 0x7fc408139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient(hunting): picked mon.noname-c con 0x7fc408139620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc4080a8190 con 0x7fc4080a6860 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc4080a33d0 con 0x7fc408139620 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc4080a2ec0 con 0x7fc4080a7180 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40cf68640 10 monclient(hunting): authenticate will time out at 1466.001750s 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc408139620 0x7fc408139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc408139620 0x7fc408139a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42230/0 (socket says 172.21.15.89:42230) 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 1 -- 172.21.15.89:0/2400722757 learned_addr learned my addr 172.21.15.89:0/2400722757 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 10 monclient(hunting): get_auth_request con 0x7fc408139620 auth_method 0 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc3fffff640 1 --2- 172.21.15.89:0/2400722757 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc4080a7180 0x7fc4080a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc3fffff640 10 monclient(hunting): get_auth_request con 0x7fc4080a7180 auth_method 0 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc3fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc3fffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc3fffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40659c640 1 --2- 172.21.15.89:0/2400722757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4080a6860 0x7fc4080a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40659c640 10 monclient(hunting): get_auth_request con 0x7fc4080a6860 auth_method 0 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc40659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 10 monclient(hunting): handle_auth_done global_id 44545 payload 274 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 1 -- 172.21.15.89:0/2400722757 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc4080a7180 msgr2=0x7fc4080a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 1 --2- 172.21.15.89:0/2400722757 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc4080a7180 0x7fc4080a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 1 -- 172.21.15.89:0/2400722757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4080a6860 msgr2=0x7fc4080a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 1 --2- 172.21.15.89:0/2400722757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4080a6860 0x7fc4080a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 1 monclient: found mon.noname-c 2024-08-19T10:46:56.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:56.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:46:56.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc406d9d640 1 -- 172.21.15.89:0/2400722757 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4080a4cd0 con 0x7fc408139620 2024-08-19T10:46:56.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.184+0000 7fc3fffff640 1 --2- 172.21.15.89:0/2400722757 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc4080a7180 0x7fc4080a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:46:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc406d9d640 1 -- 172.21.15.89:0/2400722757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc408139620 msgr2=0x7fc408139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc406d9d640 1 --2- 172.21.15.89:0/2400722757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc408139620 0x7fc408139a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc400009e80 tx=0x7fc40001c910 comp rx=0 tx=0).ready entity=mon.4 client_cookie=dbc384c393781e92 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 1 -- 172.21.15.89:0/2400722757 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4000090d0 con 0x7fc408139620 2024-08-19T10:46:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:46:56.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 10 monclient: dump: 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.196 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 1 -- 172.21.15.89:0/2400722757 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4000a6970 con 0x7fc408139620 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 1 -- 172.21.15.89:0/2400722757 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc4000af480 con 0x7fc408139620 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc405d9b640 10 monclient: dump: 2024-08-19T10:46:56.197 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.198 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 5 monclient: authenticate success, global_id 44545 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 4 set_mon_vals no callback set 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 10 monclient: shutdown 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 1 -- 172.21.15.89:0/2400722757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc408139620 msgr2=0x7fc408139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 1 --2- 172.21.15.89:0/2400722757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc408139620 0x7fc408139a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc400009e80 tx=0x7fc40001c910 comp rx=0 tx=0).stop 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 1 -- 172.21.15.89:0/2400722757 shutdown_connections 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 1 --2- 172.21.15.89:0/2400722757 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc408139620 0x7fc408139a60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 1 --2- 172.21.15.89:0/2400722757 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc4080a7180 0x7fc4080a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 1 --2- 172.21.15.89:0/2400722757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4080a6860 0x7fc4080a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 1 -- 172.21.15.89:0/2400722757 >> 172.21.15.89:0/2400722757 conn(0x7fc408074b40 msgr2=0x7fc4080abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 1 -- 172.21.15.89:0/2400722757 shutdown_connections 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 1 -- 172.21.15.89:0/2400722757 wait complete. 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 10 monclient: build_initial_monmap 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.188+0000 7fc40cf68640 10 monclient: monmap: 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.200 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 1 Processor -- start 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 1 -- start start 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient: init 2024-08-19T10:46:56.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient: _add_conns ranks=[2,4,8,6,5,7,0,1,3] 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4080a6860 0x7fc408134950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient(hunting): picked mon.noname-b con 0x7fc4080a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4080a7180 0x7fc408134e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient(hunting): picked mon.noname-c con 0x7fc4080a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc408139620 0x7fc40809c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient(hunting): picked mon.noname-e con 0x7fc408139620 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc408143210 con 0x7fc4080a6860 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc408143090 con 0x7fc4080a7180 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc408143390 con 0x7fc408139620 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient(hunting): authenticate will time out at 1466.008870s 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4080a7180 0x7fc408134e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4080a7180 0x7fc408134e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42236/0 (socket says 172.21.15.89:42236) 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc406d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc408139620 0x7fc40809c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 1 -- 172.21.15.89:0/3675942396 learned_addr learned my addr 172.21.15.89:0/3675942396 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 10 monclient(hunting): get_auth_request con 0x7fc4080a7180 auth_method 0 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40659c640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4080a6860 0x7fc408134950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc406d9d640 10 monclient(hunting): get_auth_request con 0x7fc408139620 auth_method 0 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc406d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc406d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc406d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40659c640 10 monclient(hunting): get_auth_request con 0x7fc4080a6860 auth_method 0 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc406d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc406d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc406d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 10 monclient(hunting): handle_auth_done global_id 44554 payload 824 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 1 -- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc408139620 msgr2=0x7fc40809c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc408139620 0x7fc40809c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 1 -- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4080a6860 msgr2=0x7fc408134950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4080a6860 0x7fc408134950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 10 monclient: _finish_hunting 0 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 1 monclient: found mon.noname-c 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:46:56.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 1 -- 172.21.15.89:0/3675942396 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4080a02b0 con 0x7fc4080a7180 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc406d9d640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc408139620 0x7fc40809c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40659c640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc4080a6860 0x7fc408134950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 1 -- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4080a7180 msgr2=0x7fc408134e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3fffff640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4080a7180 0x7fc408134e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e810 tx=0x7fc3f000ecd0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc3f0017070 con 0x7fc4080a7180 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 10 monclient: dump: 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.206 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f000cd50 con 0x7fc4080a7180 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc3f000f040 con 0x7fc4080a7180 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc3feffd640 10 monclient: dump: 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.208 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40559a640 4 set_mon_vals no callback set 2024-08-19T10:46:56.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 5 monclient: authenticate success, global_id 44554 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient: _renew_subs 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 1 -- 172.21.15.89:0/3675942396 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4080a27a0 con 0x7fc4080a7180 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.192+0000 7fc40cf68640 10 monclient: _renew_subs 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40cf68640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40cf68640 1 -- 172.21.15.89:0/3675942396 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40809fcf0 con 0x7fc4080a7180 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40cf68640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40cf68640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809cf20 0x7fc40809d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40cf68640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40cf68640 1 -- 172.21.15.89:0/3675942396 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc408064590 con 0x7fc40809cf20 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc3fffff640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4080a7180 0x7fc408134e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e810 tx=0x7fc3f000ecd0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fc3f001a070 mgrmap(e 5) 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc3f001a070 con 0x7fc4080a7180 2024-08-19T10:46:56.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40659c640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809cf20 0x7fc40809d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40659c640 10 monclient: get_auth_request con 0x7fc40809cf20 auth_method 0 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc3feffd640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc3e803be00 0x7fc3e803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc406d9d640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc3e803be00 0x7fc3e803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc406d9d640 10 monclient: get_auth_request con 0x7fc3e803be00 auth_method 0 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40659c640 10 monclient(hunting): handle_auth_done global_id 44497 payload 824 2024-08-19T10:46:56.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.196+0000 7fc40659c640 1 -- 172.21.15.89:0/3675942396 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc40809fef0 con 0x7fc40809cf20 2024-08-19T10:46:56.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc40659c640 1 -- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809cf20 msgr2=0x7fc40809d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc40659c640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809cf20 0x7fc40809d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8010b90 tx=0x7fc3f800e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc406d9d640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc3e803be00 0x7fc3e803e2c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc400009e50 tx=0x7fc4000a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc3f8018020 con 0x7fc40809cf20 2024-08-19T10:46:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc3feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc3feffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:46:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc3feffd640 10 monclient: dump: 2024-08-19T10:46:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.214 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc3f8010e90 con 0x7fc40809cf20 2024-08-19T10:46:56.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc3feffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc3feffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809cf20 msgr2=0x7fc40809d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc3feffd640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809cf20 0x7fc40809d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8010b90 tx=0x7fc3f800e5a0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809cf20 msgr2=0x7fc40809d300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.200+0000 7fc3feffd640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809cf20 0x7fc40809d300 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8010b90 tx=0x7fc3f800e5a0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.204+0000 7fc40cf68640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.204+0000 7fc40cf68640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809d840 0x7fc40813a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.204+0000 7fc40cf68640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.204+0000 7fc40cf68640 1 -- 172.21.15.89:0/3675942396 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc40809f6e0 con 0x7fc40809d840 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.204+0000 7fc40659c640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809d840 0x7fc40813a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc40659c640 10 monclient: get_auth_request con 0x7fc40809d840 auth_method 0 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc40659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc40659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc40659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc40659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc40659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc40659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc40659c640 10 monclient(hunting): handle_auth_done global_id 44506 payload 824 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc40659c640 1 -- 172.21.15.89:0/3675942396 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc40806cab0 con 0x7fc40809d840 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc40659c640 1 -- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809d840 msgr2=0x7fc40813a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc40659c640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809d840 0x7fc40813a8f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8019000 tx=0x7fc3f80046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc3f8018040 con 0x7fc40809d840 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc3feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc3feffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.208+0000 7fc3feffd640 10 monclient: dump: 2024-08-19T10:46:56.217 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.218 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.219 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7fc3f8003830 con 0x7fc40809d840 2024-08-19T10:46:56.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc3feffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:56.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc3feffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:56.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809d840 msgr2=0x7fc40813a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc3feffd640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809d840 0x7fc40813a8f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8019000 tx=0x7fc3f80046d0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809d840 msgr2=0x7fc40813a8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:56.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc3feffd640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc40809d840 0x7fc40813a8f0 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8019000 tx=0x7fc3f80046d0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.219 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 139334, 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:56.220 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.221 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.222 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:56.223 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:56.224 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:56.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.227 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.228 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.232 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.234 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.235 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.237 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.239 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.241 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.242 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.243 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.243 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:46:56.243 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.243 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.243 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:56.243 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:56.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc40cf68640 1 -- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc3e803be00 msgr2=0x7fc3e803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc40cf68640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc3e803be00 0x7fc3e803e2c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc400009e50 tx=0x7fc4000a7000 comp rx=0 tx=0).stop 2024-08-19T10:46:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc40cf68640 10 monclient: shutdown 2024-08-19T10:46:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc40cf68640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc40cf68640 1 -- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4080a7180 msgr2=0x7fc408134e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc40cf68640 1 --2- 172.21.15.89:0/3675942396 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc4080a7180 0x7fc408134e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e810 tx=0x7fc3f000ecd0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc406d9d640 1 -- 172.21.15.89:0/3675942396 reap_dead start 2024-08-19T10:46:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc3feffd640 1 -- 172.21.15.89:0/3675942396 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (unknown 0 0 0) 0x7fc3f004f0e0 con 0x7fc4080a7180 2024-08-19T10:46:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc406d9d640 1 -- 172.21.15.89:0/3675942396 reap_dead start 2024-08-19T10:46:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc40cf68640 1 -- 172.21.15.89:0/3675942396 shutdown_connections 2024-08-19T10:46:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc40cf68640 1 -- 172.21.15.89:0/3675942396 >> 172.21.15.89:0/3675942396 conn(0x7fc408074b40 msgr2=0x7fc4080ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc40cf68640 1 -- 172.21.15.89:0/3675942396 shutdown_connections 2024-08-19T10:46:56.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.212+0000 7fc40cf68640 1 -- 172.21.15.89:0/3675942396 wait complete. 2024-08-19T10:46:56.246 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:56.246 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:46:56.246 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:46:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient: get_monmap_and_config 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient: build_initial_monmap 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient: monmap: 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:56.366856+0000 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:56.366856+0000 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:56.364 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:56.365 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:56.365 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 1 Processor -- start 2024-08-19T10:46:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 1 -- start start 2024-08-19T10:46:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient: init 2024-08-19T10:46:56.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient: _add_conns ranks=[3,2,6,8,9,4,10,1,7,5,0] 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c0a4b30 0x7f7a5c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient(hunting): picked mon.noname-g con 0x7f7a5c0a4b30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a5c0720c0 0x7f7a5c0a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient(hunting): picked mon.noname-b con 0x7f7a5c0720c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7a5c0a5450 0x7f7a5c13d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient(hunting): picked mon.noname-d con 0x7f7a5c0a5450 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c073f30 con 0x7f7a5c0720c0 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c073600 con 0x7f7a5c0a4b30 2024-08-19T10:46:56.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7a5c0738e0 con 0x7f7a5c0a5450 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a6406a640 10 monclient(hunting): authenticate will time out at 1466.183464s 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a5c0720c0 0x7f7a5c0a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a5c0720c0 0x7f7a5c0a4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44934/0 (socket says 172.21.15.89:44934) 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 1 -- 172.21.15.89:0/4270714360 learned_addr learned my addr 172.21.15.89:0/4270714360 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 10 monclient(hunting): get_auth_request con 0x7f7a5c0720c0 auth_method 0 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a62607640 1 --2- 172.21.15.89:0/4270714360 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7a5c0a5450 0x7f7a5c13d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a62607640 10 monclient(hunting): get_auth_request con 0x7f7a5c0a5450 auth_method 0 2024-08-19T10:46:56.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a62607640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a62607640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a62607640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a62607640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a62607640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a62607640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61e06640 1 --2- 172.21.15.89:0/4270714360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c0a4b30 0x7f7a5c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 10 monclient(hunting): handle_auth_done global_id 54173 payload 274 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 1 -- 172.21.15.89:0/4270714360 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7a5c0a5450 msgr2=0x7f7a5c13d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:56.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 1 --2- 172.21.15.89:0/4270714360 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7a5c0a5450 0x7f7a5c13d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 1 -- 172.21.15.89:0/4270714360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c0a4b30 msgr2=0x7f7a5c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 1 --2- 172.21.15.89:0/4270714360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c0a4b30 0x7f7a5c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 10 monclient: _finish_hunting 0 2024-08-19T10:46:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 1 monclient: found mon.noname-b 2024-08-19T10:46:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:46:56.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.364+0000 7f7a61605640 1 -- 172.21.15.89:0/4270714360 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c0a42a0 con 0x7f7a5c0720c0 2024-08-19T10:46:56.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61605640 1 -- 172.21.15.89:0/4270714360 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a5c0720c0 msgr2=0x7f7a5c0a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61605640 1 --2- 172.21.15.89:0/4270714360 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a5c0720c0 0x7f7a5c0a4f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a580098a0 tx=0x7f7a5801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95b8c5e854b193d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 1 -- 172.21.15.89:0/4270714360 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7a580b2070 con 0x7f7a5c0720c0 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 10 monclient: dump: 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.378 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 5 monclient: authenticate success, global_id 54173 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 1 -- 172.21.15.89:0/4270714360 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a580ab7f0 con 0x7f7a5c0720c0 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 4 set_mon_vals no callback set 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 1 -- 172.21.15.89:0/4270714360 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7a580abe10 con 0x7f7a5c0720c0 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a60e04640 10 monclient: dump: 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.380 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.381 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient: shutdown 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 -- 172.21.15.89:0/4270714360 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a5c0720c0 msgr2=0x7f7a5c0a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 --2- 172.21.15.89:0/4270714360 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a5c0720c0 0x7f7a5c0a4f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a580098a0 tx=0x7f7a5801c440 comp rx=0 tx=0).stop 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 -- 172.21.15.89:0/4270714360 shutdown_connections 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 --2- 172.21.15.89:0/4270714360 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7a5c0a5450 0x7f7a5c13d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 --2- 172.21.15.89:0/4270714360 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7a5c0720c0 0x7f7a5c0a4f10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 --2- 172.21.15.89:0/4270714360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c0a4b30 0x7f7a5c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 -- 172.21.15.89:0/4270714360 >> 172.21.15.89:0/4270714360 conn(0x7f7a5c0aa240 msgr2=0x7f7a5c0aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 -- 172.21.15.89:0/4270714360 shutdown_connections 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 -- 172.21.15.89:0/4270714360 wait complete. 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient: build_initial_monmap 2024-08-19T10:46:56.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient: monmap: 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:56.383 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 Processor -- start 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 -- start start 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient: init 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient: _add_conns ranks=[8,6,3,0,5,1,7,4,2] 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7a5c0720c0 0x7f7a5c1d1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient(hunting): picked mon.noname-e con 0x7f7a5c0720c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:56.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7a5c0a4b30 0x7f7a5c13bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient(hunting): picked mon.noname-d con 0x7f7a5c0a4b30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c0a5450 0x7f7a5c125ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient(hunting): picked mon.noname-g con 0x7f7a5c0a5450 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c140b30 con 0x7f7a5c0a5450 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7a5c1409b0 con 0x7f7a5c0a4b30 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7a5c140cb0 con 0x7f7a5c0720c0 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a6406a640 10 monclient(hunting): authenticate will time out at 1466.190935s 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7a5c0720c0 0x7f7a5c1d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61605640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7a5c0a4b30 0x7f7a5c13bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7a5c0720c0 0x7f7a5c1d1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:36154/0 (socket says 172.21.15.89:36154) 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 1 -- 172.21.15.89:0/2344069532 learned_addr learned my addr 172.21.15.89:0/2344069532 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 10 monclient(hunting): get_auth_request con 0x7f7a5c0720c0 auth_method 0 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61605640 10 monclient(hunting): get_auth_request con 0x7f7a5c0a4b30 auth_method 0 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61605640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61605640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61605640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a62607640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c0a5450 0x7f7a5c125ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a62607640 10 monclient(hunting): get_auth_request con 0x7f7a5c0a5450 auth_method 0 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a62607640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a62607640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a62607640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 10 monclient(hunting): handle_auth_done global_id 6506 payload 824 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 1 -- 172.21.15.89:0/2344069532 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7a5c0a4b30 msgr2=0x7f7a5c13bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7a5c0a4b30 0x7f7a5c13bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 1 -- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c0a5450 msgr2=0x7f7a5c125ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c0a5450 0x7f7a5c125ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 10 monclient: _finish_hunting 0 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 1 monclient: found mon.noname-e 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61e06640 1 -- 172.21.15.89:0/2344069532 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c1295e0 con 0x7f7a5c0720c0 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.372+0000 7f7a61605640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7a5c0a4b30 0x7f7a5c13bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61e06640 1 -- 172.21.15.89:0/2344069532 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7a5c0720c0 msgr2=0x7f7a5c1d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61e06640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7a5c0720c0 0x7f7a5c1d1020 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000c820 tx=0x7f7a5000cce0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7a50015070 con 0x7f7a5c0720c0 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:46:56.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 10 monclient: dump: 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.389 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a6406a640 5 monclient: authenticate success, global_id 6506 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5000ce10 con 0x7f7a5c0720c0 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7a50011860 con 0x7f7a5c0720c0 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a6406a640 10 monclient: _renew_subs 2024-08-19T10:46:56.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a6406a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a6406a640 1 -- 172.21.15.89:0/2344069532 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c1298c0 con 0x7f7a5c0720c0 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4ffff640 4 set_mon_vals no callback set 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a6406a640 10 monclient: _renew_subs 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a6406a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a6406a640 1 -- 172.21.15.89:0/2344069532 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c0a2660 con 0x7f7a5c0720c0 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 10 monclient: dump: 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.391 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a6406a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a6406a640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c126000 0x7f7a5c1263e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a6406a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a6406a640 1 -- 172.21.15.89:0/2344069532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c064590 con 0x7f7a5c126000 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7a50018070 con 0x7f7a5c0720c0 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61605640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c126000 0x7f7a5c1263e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61605640 10 monclient: get_auth_request con 0x7f7a5c126000 auth_method 0 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61605640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61605640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61605640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7a4003be00 0x7f7a4003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f7a5004f530 con 0x7f7a5c0720c0 2024-08-19T10:46:56.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61605640 10 monclient(hunting): handle_auth_done global_id 35148 payload 824 2024-08-19T10:46:56.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a61605640 1 -- 172.21.15.89:0/2344069532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a5c128120 con 0x7f7a5c126000 2024-08-19T10:46:56.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a62607640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7a4003be00 0x7f7a4003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.376+0000 7f7a62607640 10 monclient: get_auth_request con 0x7f7a4003be00 auth_method 0 2024-08-19T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a61605640 1 -- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c126000 msgr2=0x7f7a5c1263e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a61605640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c126000 0x7f7a5c1263e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a580af000 tx=0x7f7a5801ac80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7a580b2050 con 0x7f7a5c126000 2024-08-19T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a4e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a4e7fc640 10 monclient: dump: 2024-08-19T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.396 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.397 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a62607640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7a4003be00 0x7f7a4003e2c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0a4b00 tx=0x7f7a5400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7a580b3040 con 0x7f7a5c126000 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a4e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a4e7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c126000 msgr2=0x7f7a5c1263e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a4e7fc640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c126000 0x7f7a5c1263e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a580af000 tx=0x7f7a5801ac80 comp rx=0 tx=0).stop 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c126000 msgr2=0x7f7a5c1263e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.380+0000 7f7a4e7fc640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c126000 0x7f7a5c1263e0 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a580af000 tx=0x7f7a5801ac80 comp rx=0 tx=0).stop 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a6406a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a6406a640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c09e9f0 0x7f7a5c09ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a6406a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a6406a640 1 -- 172.21.15.89:0/2344069532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c126920 con 0x7f7a5c09e9f0 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a61605640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c09e9f0 0x7f7a5c09ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a61605640 10 monclient: get_auth_request con 0x7f7a5c09e9f0 auth_method 0 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a61605640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a61605640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a61605640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a61605640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a61605640 10 monclient(hunting): handle_auth_done global_id 35157 payload 824 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.388+0000 7f7a61605640 1 -- 172.21.15.89:0/2344069532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7a5c0a4f10 con 0x7f7a5c09e9f0 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a61605640 1 -- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c09e9f0 msgr2=0x7f7a5c09ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a61605640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c09e9f0 0x7f7a5c09ee00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7a5801cce0 tx=0x7f7a580191e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7a580addb0 con 0x7f7a5c09e9f0 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a4e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a4e7fc640 10 monclient: dump: 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.400 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f7a580c0670 con 0x7f7a5c09e9f0 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a4e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a4e7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c09e9f0 msgr2=0x7f7a5c09ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a4e7fc640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c09e9f0 0x7f7a5c09ee00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7a5801cce0 tx=0x7f7a580191e0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a4e7fc640 1 -- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c09e9f0 msgr2=0x7f7a5c09ee00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a4e7fc640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a5c09e9f0 0x7f7a5c09ee00 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7a5801cce0 tx=0x7f7a580191e0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 261432, 2024-08-19T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:56.402 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:56.403 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:56.404 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:56.404 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.404 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.405 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.406 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:56.407 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:56.408 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:56.408 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:56.408 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:56.408 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:56.408 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:56.409 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:56.409 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:56.409 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:56.409 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:56.409 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:56.409 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.409 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.409 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:56.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:56.411 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:56.412 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:56.415 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:56.418 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:56.419 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:56.420 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:56.421 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:56.423 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:56.424 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.426 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:56.427 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a6406a640 1 -- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7a4003be00 msgr2=0x7f7a4003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a6406a640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7a4003be00 0x7f7a4003e2c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0a4b00 tx=0x7f7a5400b040 comp rx=0 tx=0).stop 2024-08-19T10:46:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a6406a640 10 monclient: shutdown 2024-08-19T10:46:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a6406a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a6406a640 1 -- 172.21.15.89:0/2344069532 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7a5c0720c0 msgr2=0x7f7a5c1d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a6406a640 1 --2- 172.21.15.89:0/2344069532 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7a5c0720c0 0x7f7a5c1d1020 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000c820 tx=0x7f7a5000cce0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a62607640 1 -- 172.21.15.89:0/2344069532 reap_dead start 2024-08-19T10:46:56.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a62607640 1 -- 172.21.15.89:0/2344069532 reap_dead start 2024-08-19T10:46:56.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a6406a640 1 -- 172.21.15.89:0/2344069532 shutdown_connections 2024-08-19T10:46:56.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a6406a640 1 -- 172.21.15.89:0/2344069532 >> 172.21.15.89:0/2344069532 conn(0x7f7a5c0aa240 msgr2=0x7f7a5c13ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:56.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a6406a640 1 -- 172.21.15.89:0/2344069532 shutdown_connections 2024-08-19T10:46:56.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.392+0000 7f7a6406a640 1 -- 172.21.15.89:0/2344069532 wait complete. 2024-08-19T10:46:56.430 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:56.430 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:46:56.431 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:46:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient: get_monmap_and_config 2024-08-19T10:46:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient: build_initial_monmap 2024-08-19T10:46:56.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient: monmap: 2024-08-19T10:46:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:56.553728+0000 2024-08-19T10:46:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:56.553728+0000 2024-08-19T10:46:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:56.551 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:56.552 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:56.553 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:56.553 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:56.553 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 1 Processor -- start 2024-08-19T10:46:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 1 -- start start 2024-08-19T10:46:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient: init 2024-08-19T10:46:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient: _add_conns ranks=[0,5,9,4,7,6,10,3,1,2,8] 2024-08-19T10:46:56.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd82809b390 addr v1:172.21.15.89:6789/0 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809ec20 0x7fd82813ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient(hunting): picked mon.noname-h con 0x7fd82809ec20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd82813b0a0 0x7fd82813d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient(hunting): picked mon.noname-a con 0x7fd82813b0a0 addr v2:172.21.15.89:3300/0 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd82806db50 con 0x7fd82809b390 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fd82809c900 con 0x7fd82813b0a0 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd82809be80 con 0x7fd82809ec20 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82fa50640 10 monclient(hunting): authenticate will time out at 1466.371103s 2024-08-19T10:46:56.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82d7ec640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd82809b390 0x7fd82809b770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44802/0 (socket says 172.21.15.89:44802) 2024-08-19T10:46:56.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd82813b0a0 0x7fd82813d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82d7ec640 1 -- 172.21.15.89:0/2762420396 learned_addr learned my addr 172.21.15.89:0/2762420396 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:46:56.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 10 monclient(hunting): get_auth_request con 0x7fd82813b0a0 auth_method 0 2024-08-19T10:46:56.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82cfeb640 1 --2- 172.21.15.89:0/2762420396 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809ec20 0x7fd82813ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82cfeb640 10 monclient(hunting): get_auth_request con 0x7fd82809ec20 auth_method 0 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82cfeb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82cfeb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82cfeb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82cfeb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82cfeb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82cfeb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 10 monclient(hunting): handle_auth_done global_id 6669 payload 274 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 1 -- 172.21.15.89:0/2762420396 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809ec20 msgr2=0x7fd82813ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 1 --2- 172.21.15.89:0/2762420396 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809ec20 0x7fd82813ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 1 -- 172.21.15.89:0/2762420396 >> v1:172.21.15.89:6789/0 conn(0x7fd82809b390 legacy=0x7fd82809b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 10 monclient: _finish_hunting 0 2024-08-19T10:46:56.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 1 monclient: found mon.noname-a 2024-08-19T10:46:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:46:56.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.552+0000 7fd82dfed640 1 -- 172.21.15.89:0/2762420396 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82809d9a0 con 0x7fd82813b0a0 2024-08-19T10:46:56.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.556+0000 7fd82dfed640 1 -- 172.21.15.89:0/2762420396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82813b0a0 msgr2=0x7fd82813d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.556+0000 7fd82dfed640 1 --2- 172.21.15.89:0/2762420396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82813b0a0 0x7fd82813d490 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd820009d90 tx=0x7fd82001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0ae826cdf6e805 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 1 -- 172.21.15.89:0/2762420396 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8200a6930 con 0x7fd82813b0a0 2024-08-19T10:46:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:46:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 10 monclient: dump: 2024-08-19T10:46:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.566 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 1 -- 172.21.15.89:0/2762420396 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8200a6af0 con 0x7fd82813b0a0 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 1 -- 172.21.15.89:0/2762420396 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8200a6ca0 con 0x7fd82813b0a0 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 5 monclient: authenticate success, global_id 6669 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 4 set_mon_vals no callback set 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd81ffff640 10 monclient: dump: 2024-08-19T10:46:56.568 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.569 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient: shutdown 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 -- 172.21.15.89:0/2762420396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82813b0a0 msgr2=0x7fd82813d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 --2- 172.21.15.89:0/2762420396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82813b0a0 0x7fd82813d490 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd820009d90 tx=0x7fd82001c800 comp rx=0 tx=0).stop 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 -- 172.21.15.89:0/2762420396 shutdown_connections 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 --2- 172.21.15.89:0/2762420396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd82813b0a0 0x7fd82813d490 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 --2- 172.21.15.89:0/2762420396 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809ec20 0x7fd82813ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 -- 172.21.15.89:0/2762420396 >> 172.21.15.89:0/2762420396 conn(0x7fd8280a4df0 msgr2=0x7fd8280a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 -- 172.21.15.89:0/2762420396 shutdown_connections 2024-08-19T10:46:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 -- 172.21.15.89:0/2762420396 wait complete. 2024-08-19T10:46:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient: build_initial_monmap 2024-08-19T10:46:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient: monmap: 2024-08-19T10:46:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:56.571 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 Processor -- start 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 -- start start 2024-08-19T10:46:56.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient: init 2024-08-19T10:46:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient: _add_conns ranks=[3,1,2,5,6,7,8,4,0] 2024-08-19T10:46:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82809b390 0x7fd828074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient(hunting): picked mon.noname-g con 0x7fd82809b390 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd82809ec20 0x7fd82813faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient(hunting): picked mon.noname-f con 0x7fd82809ec20 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:46:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82813b0a0 0x7fd828140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient(hunting): picked mon.noname-b con 0x7fd82813b0a0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd82809f000 con 0x7fd82809ec20 2024-08-19T10:46:56.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd8280a5bd0 con 0x7fd82813b0a0 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd828141d00 con 0x7fd82809b390 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82fa50640 10 monclient(hunting): authenticate will time out at 1466.378443s 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82813b0a0 0x7fd828140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82813b0a0 0x7fd828140030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44936/0 (socket says 172.21.15.89:44936) 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 1 -- 172.21.15.89:0/2294530499 learned_addr learned my addr 172.21.15.89:0/2294530499 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 10 monclient(hunting): get_auth_request con 0x7fd82813b0a0 auth_method 0 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82d7ec640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82809b390 0x7fd828074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82d7ec640 10 monclient(hunting): get_auth_request con 0x7fd82809b390 auth_method 0 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82d7ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82d7ec640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82d7ec640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82cfeb640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd82809ec20 0x7fd82813faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 10 monclient(hunting): handle_auth_done global_id 54182 payload 824 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 1 -- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82809b390 msgr2=0x7fd828074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82809b390 0x7fd828074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 1 -- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd82809ec20 msgr2=0x7fd82813faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd82809ec20 0x7fd82813faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 10 monclient: _finish_hunting 0 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 1 monclient: found mon.noname-b 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82dfed640 1 -- 172.21.15.89:0/2294530499 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd828141240 con 0x7fd82813b0a0 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82cfeb640 10 monclient: get_auth_request con 0x7fd82809ec20 auth_method 0 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.560+0000 7fd82cfeb640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd82809ec20 0x7fd82813faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82dfed640 1 -- 172.21.15.89:0/2294530499 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82813b0a0 msgr2=0x7fd828140030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82dfed640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82813b0a0 0x7fd828140030 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd82001d0a0 tx=0x7fd8200195a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8200bc070 con 0x7fd82813b0a0 2024-08-19T10:46:56.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 10 monclient: dump: 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.577 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82fa50640 5 monclient: authenticate success, global_id 54182 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd820003bb0 con 0x7fd82813b0a0 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82fa50640 10 monclient: _renew_subs 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82fa50640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82fa50640 1 -- 172.21.15.89:0/2294530499 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8280739b0 con 0x7fd82813b0a0 2024-08-19T10:46:56.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd820003d60 con 0x7fd82813b0a0 2024-08-19T10:46:56.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:46:56.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81f7fe640 4 set_mon_vals no callback set 2024-08-19T10:46:56.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 10 monclient: dump: 2024-08-19T10:46:56.579 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.579 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.579 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.579 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.579 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.580 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.580 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82fa50640 10 monclient: _renew_subs 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82fa50640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82fa50640 1 -- 172.21.15.89:0/2294530499 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82813abc0 con 0x7fd82813b0a0 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82fa50640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82fa50640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd828136f10 0x7fd828140570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82fa50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82fa50640 1 -- 172.21.15.89:0/2294530499 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd82813a690 con 0x7fd828136f10 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd8200bf070 con 0x7fd82813b0a0 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82d7ec640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd828136f10 0x7fd828140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd80c03be00 0x7fd80c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82d7ec640 10 monclient: get_auth_request con 0x7fd828136f10 auth_method 0 2024-08-19T10:46:56.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82d7ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82d7ec640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82d7ec640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fd8200ea6f0 con 0x7fd82813b0a0 2024-08-19T10:46:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82d7ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82d7ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82d7ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82cfeb640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd80c03be00 0x7fd80c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82cfeb640 10 monclient: get_auth_request con 0x7fd80c03be00 auth_method 0 2024-08-19T10:46:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82d7ec640 10 monclient(hunting): handle_auth_done global_id 6638 payload 824 2024-08-19T10:46:56.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.564+0000 7fd82d7ec640 1 -- 172.21.15.89:0/2294530499 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd82809b770 con 0x7fd828136f10 2024-08-19T10:46:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd82d7ec640 1 -- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd828136f10 msgr2=0x7fd828140570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd82d7ec640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd828136f10 0x7fd828140570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e500 tx=0x7fd82400e9c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd824004e40 con 0x7fd828136f10 2024-08-19T10:46:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd81dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd81dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:46:56.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd81dffb640 10 monclient: dump: 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.586 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd82cfeb640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd80c03be00 0x7fd80c03e2c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd828068f10 tx=0x7fd818008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd824018070 con 0x7fd828136f10 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd81dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd81dffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd828136f10 msgr2=0x7fd828140570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd81dffb640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd828136f10 0x7fd828140570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e500 tx=0x7fd82400e9c0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd828136f10 msgr2=0x7fd828140570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.568+0000 7fd81dffb640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd828136f10 0x7fd828140570 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82fa50640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:56.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82fa50640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809f6f0 0x7fd82809fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82fa50640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82fa50640 1 -- 172.21.15.89:0/2294530499 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd828064590 con 0x7fd82809f6f0 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82d7ec640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809f6f0 0x7fd82809fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82d7ec640 10 monclient: get_auth_request con 0x7fd82809f6f0 auth_method 0 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82d7ec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82d7ec640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82d7ec640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82d7ec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82d7ec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82d7ec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82d7ec640 10 monclient(hunting): handle_auth_done global_id 6647 payload 824 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.576+0000 7fd82d7ec640 1 -- 172.21.15.89:0/2294530499 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd8280721b0 con 0x7fd82809f6f0 2024-08-19T10:46:56.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82d7ec640 1 -- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809f6f0 msgr2=0x7fd82809fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82d7ec640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809f6f0 0x7fd82809fb00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd824017000 tx=0x7fd824004950 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd824028440 con 0x7fd82809f6f0 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd81dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd81dffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd81dffb640 10 monclient: dump: 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.589 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9601 (secure 0 0 0) 0x7fd8240064f0 con 0x7fd82809f6f0 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd81dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd81dffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809f6f0 msgr2=0x7fd82809fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd81dffb640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809f6f0 0x7fd82809fb00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd824017000 tx=0x7fd824004950 comp rx=0 tx=0).stop 2024-08-19T10:46:56.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd81dffb640 1 -- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809f6f0 msgr2=0x7fd82809fb00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:56.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd81dffb640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd82809f6f0 0x7fd82809fb00 secure :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd824017000 tx=0x7fd824004950 comp rx=0 tx=0).stop 2024-08-19T10:46:56.591 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:56.591 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:56.591 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:56.591 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:56.591 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:56.591 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 546347, 2024-08-19T10:46:56.591 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:56.592 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.593 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.594 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:56.595 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.596 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:56.598 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:56.601 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:56.602 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.604 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:56.605 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:56.608 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.609 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.610 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.611 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.612 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.613 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.614 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:46:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.615 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:56.615 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82fa50640 1 -- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd80c03be00 msgr2=0x7fd80c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82fa50640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd80c03be00 0x7fd80c03e2c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd828068f10 tx=0x7fd818008040 comp rx=0 tx=0).stop 2024-08-19T10:46:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82fa50640 10 monclient: shutdown 2024-08-19T10:46:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82fa50640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82fa50640 1 -- 172.21.15.89:0/2294530499 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82813b0a0 msgr2=0x7fd828140030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82fa50640 1 --2- 172.21.15.89:0/2294530499 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd82813b0a0 0x7fd828140030 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd82001d0a0 tx=0x7fd8200195a0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82dfed640 1 -- 172.21.15.89:0/2294530499 reap_dead start 2024-08-19T10:46:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82dfed640 1 -- 172.21.15.89:0/2294530499 reap_dead start 2024-08-19T10:46:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82fa50640 1 -- 172.21.15.89:0/2294530499 shutdown_connections 2024-08-19T10:46:56.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82fa50640 1 -- 172.21.15.89:0/2294530499 >> 172.21.15.89:0/2294530499 conn(0x7fd8280a4df0 msgr2=0x7fd828139790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:56.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82fa50640 1 -- 172.21.15.89:0/2294530499 shutdown_connections 2024-08-19T10:46:56.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.580+0000 7fd82fa50640 1 -- 172.21.15.89:0/2294530499 wait complete. 2024-08-19T10:46:56.618 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:56.618 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:46:56.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:46:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient: get_monmap_and_config 2024-08-19T10:46:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient: build_initial_monmap 2024-08-19T10:46:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient: monmap: 2024-08-19T10:46:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:56.737266+0000 2024-08-19T10:46:56.734 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:56.737266+0000 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:56.735 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:56.736 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 1 Processor -- start 2024-08-19T10:46:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 1 -- start start 2024-08-19T10:46:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient: init 2024-08-19T10:46:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient: _add_conns ranks=[10,0,7,6,5,2,8,9,3,4,1] 2024-08-19T10:46:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdaf40a6860 0x7fdaf40a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient(hunting): picked mon.noname-f con 0x7fdaf40a6860 addr v2:172.21.15.190:3300/0 2024-08-19T10:46:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdaf40a7180 addr v1:172.21.15.89:6789/0 2024-08-19T10:46:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf40a7560 0x7fdaf4139d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient(hunting): picked mon.noname-i con 0x7fdaf40a7560 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:46:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf40a33d0 con 0x7fdaf40a7180 2024-08-19T10:46:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf40a8190 con 0x7fdaf40a6860 2024-08-19T10:46:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fdaf40a2ec0 con 0x7fdaf40a7560 2024-08-19T10:46:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafbde0640 10 monclient(hunting): authenticate will time out at 1466.553872s 2024-08-19T10:46:56.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf937b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdaf40a7180 0x7fdaf4139620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44818/0 (socket says 172.21.15.89:44818) 2024-08-19T10:46:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf937b640 1 -- 172.21.15.89:0/1779224809 learned_addr learned my addr 172.21.15.89:0/1779224809 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:46:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 1 --2- 172.21.15.89:0/1779224809 >> v2:172.21.15.190:3300/0 conn(0x7fdaf40a6860 0x7fdaf40a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 10 monclient(hunting): get_auth_request con 0x7fdaf40a6860 auth_method 0 2024-08-19T10:46:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafa37d640 1 --2- 172.21.15.89:0/1779224809 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf40a7560 0x7fdaf4139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafa37d640 10 monclient(hunting): get_auth_request con 0x7fdaf40a7560 auth_method 0 2024-08-19T10:46:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafa37d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafa37d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafa37d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafa37d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafa37d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdafa37d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_done global_id 44515 payload 274 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 1 -- 172.21.15.89:0/1779224809 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf40a7560 msgr2=0x7fdaf4139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 1 --2- 172.21.15.89:0/1779224809 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf40a7560 0x7fdaf4139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 1 -- 172.21.15.89:0/1779224809 >> v1:172.21.15.89:6789/0 conn(0x7fdaf40a7180 legacy=0x7fdaf4139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 10 monclient: _finish_hunting 0 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 1 monclient: found mon.noname-f 2024-08-19T10:46:56.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:56.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:46:56.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.736+0000 7fdaf9b7c640 1 -- 172.21.15.89:0/1779224809 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf40a79a0 con 0x7fdaf40a6860 2024-08-19T10:46:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf9b7c640 1 -- 172.21.15.89:0/1779224809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf40a6860 msgr2=0x7fdaf40a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf9b7c640 1 --2- 172.21.15.89:0/1779224809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf40a6860 0x7fdaf40a6c40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0097c0 tx=0x7fdaec01c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa59675d9dc0ce1b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 1 -- 172.21.15.89:0/1779224809 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaec0b0070 con 0x7fdaf40a6860 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 10 monclient: dump: 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.746 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 1 -- 172.21.15.89:0/1779224809 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec0aa810 con 0x7fdaf40a6860 2024-08-19T10:46:56.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 1 -- 172.21.15.89:0/1779224809 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaec0a9300 con 0x7fdaf40a6860 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdaf8b7a640 10 monclient: dump: 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.748 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdafbde0640 5 monclient: authenticate success, global_id 44515 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdafbde0640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.740+0000 7fdafbde0640 4 set_mon_vals no callback set 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient: shutdown 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:56.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 -- 172.21.15.89:0/1779224809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf40a6860 msgr2=0x7fdaf40a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 --2- 172.21.15.89:0/1779224809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf40a6860 0x7fdaf40a6c40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0097c0 tx=0x7fdaec01c360 comp rx=0 tx=0).stop 2024-08-19T10:46:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 -- 172.21.15.89:0/1779224809 shutdown_connections 2024-08-19T10:46:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 --2- 172.21.15.89:0/1779224809 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf40a7560 0x7fdaf4139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 --2- 172.21.15.89:0/1779224809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdaf40a6860 0x7fdaf40a6c40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 -- 172.21.15.89:0/1779224809 >> 172.21.15.89:0/1779224809 conn(0x7fdaf4074b40 msgr2=0x7fdaf40abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 -- 172.21.15.89:0/1779224809 shutdown_connections 2024-08-19T10:46:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 -- 172.21.15.89:0/1779224809 wait complete. 2024-08-19T10:46:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient: build_initial_monmap 2024-08-19T10:46:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient: monmap: 2024-08-19T10:46:56.750 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:56.751 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:56.751 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:56.751 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:56.751 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:56.751 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.751 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:56.751 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:56.751 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 Processor -- start 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 -- start start 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient: init 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient: _add_conns ranks=[8,3,0,5,1,6,4,7,2] 2024-08-19T10:46:56.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdaf40a6860 0x7fdaf4138ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient(hunting): picked mon.noname-e con 0x7fdaf40a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdaf40a7180 0x7fdaf4139010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient(hunting): picked mon.noname-g con 0x7fdaf40a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf40a7560 0x7fdaf413df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient(hunting): picked mon.noname-a con 0x7fdaf40a7560 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf413a3f0 con 0x7fdaf40a7560 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf413a270 con 0x7fdaf40a7180 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fdaf41440a0 con 0x7fdaf40a6860 2024-08-19T10:46:56.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafbde0640 10 monclient(hunting): authenticate will time out at 1466.562530s 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf40a7560 0x7fdaf413df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf40a7560 0x7fdaf413df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50696/0 (socket says 172.21.15.89:50696) 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 1 -- 172.21.15.89:0/1707304240 learned_addr learned my addr 172.21.15.89:0/1707304240 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf9b7c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdaf40a6860 0x7fdaf4138ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 10 monclient(hunting): get_auth_request con 0x7fdaf40a7560 auth_method 0 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf9b7c640 10 monclient(hunting): get_auth_request con 0x7fdaf40a6860 auth_method 0 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf9b7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf9b7c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf9b7c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf937b640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdaf40a7180 0x7fdaf4139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 10 monclient(hunting): handle_auth_done global_id 6678 payload 824 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 1 -- 172.21.15.89:0/1707304240 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdaf40a6860 msgr2=0x7fdaf4138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdaf40a6860 0x7fdaf4138ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 1 -- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdaf40a7180 msgr2=0x7fdaf4139010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdaf40a7180 0x7fdaf4139010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 10 monclient: _finish_hunting 0 2024-08-19T10:46:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 1 monclient: found mon.noname-a 2024-08-19T10:46:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:46:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdafa37d640 1 -- 172.21.15.89:0/1707304240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf4122be0 con 0x7fdaf40a7560 2024-08-19T10:46:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf937b640 10 monclient: get_auth_request con 0x7fdaf40a7180 auth_method 0 2024-08-19T10:46:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf937b640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdaf40a7180 0x7fdaf4139010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:46:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.744+0000 7fdaf9b7c640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdaf40a6860 0x7fdaf4138ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafa37d640 1 -- 172.21.15.89:0/1707304240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf40a7560 msgr2=0x7fdaf413df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafa37d640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf40a7560 0x7fdaf413df90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdae400e7f0 tx=0x7fdae400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdae4017070 con 0x7fdaf40a7560 2024-08-19T10:46:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 10 monclient: dump: 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.761 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae4013470 con 0x7fdaf40a7560 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafbde0640 5 monclient: authenticate success, global_id 6678 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafbde0640 10 monclient: _renew_subs 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafbde0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:56.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafbde0640 1 -- 172.21.15.89:0/1707304240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf4123440 con 0x7fdaf40a7560 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdae400eea0 con 0x7fdaf40a7560 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 10 monclient: dump: 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.763 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf3fff640 4 set_mon_vals no callback set 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafbde0640 10 monclient: _renew_subs 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafbde0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafbde0640 1 -- 172.21.15.89:0/1707304240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf4140360 con 0x7fdaf40a7560 2024-08-19T10:46:56.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafbde0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafbde0640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413ec90 0x7fdaf402d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafbde0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdafbde0640 1 -- 172.21.15.89:0/1707304240 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fdaf4139a50 con 0x7fdaf413ec90 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fdae401a070 con 0x7fdaf40a7560 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf9b7c640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413ec90 0x7fdaf402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdae003be30 0x7fdae003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf9b7c640 10 monclient: get_auth_request con 0x7fdaf413ec90 auth_method 0 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf9b7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf9b7c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf9b7c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fdae400f040 con 0x7fdaf40a7560 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf937b640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdae003be30 0x7fdae003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf937b640 10 monclient: get_auth_request con 0x7fdae003be30 auth_method 0 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_done global_id 15982 payload 824 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.748+0000 7fdaf9b7c640 1 -- 172.21.15.89:0/1707304240 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdaf40a6c40 con 0x7fdaf413ec90 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf9b7c640 1 -- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413ec90 msgr2=0x7fdaf402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf9b7c640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413ec90 0x7fdaf402d1d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0ad000 tx=0x7fdaec019100 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf937b640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdae003be30 0x7fdae003e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdaf413f6f0 tx=0x7fdae8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaec0b0040 con 0x7fdaf413ec90 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf27fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:46:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf27fc640 10 monclient: dump: 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdaec0aab00 con 0x7fdaf413ec90 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf27fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413ec90 msgr2=0x7fdaf402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf27fc640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413ec90 0x7fdaf402d1d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0ad000 tx=0x7fdaec019100 comp rx=0 tx=0).stop 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413ec90 msgr2=0x7fdaf402d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.752+0000 7fdaf27fc640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413ec90 0x7fdaf402d1d0 secure :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0ad000 tx=0x7fdaec019100 comp rx=0 tx=0).stop 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdafbde0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdafbde0640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413f070 0x7fdaf402d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdafbde0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdafbde0640 1 -- 172.21.15.89:0/1707304240 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fdaf4064590 con 0x7fdaf413f070 2024-08-19T10:46:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdaf9b7c640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413f070 0x7fdaf402d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdaf9b7c640 10 monclient: get_auth_request con 0x7fdaf413f070 auth_method 0 2024-08-19T10:46:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdaf9b7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdaf9b7c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdaf9b7c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdaf9b7c640 10 monclient(hunting): handle_auth_done global_id 15991 payload 824 2024-08-19T10:46:56.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.760+0000 7fdaf9b7c640 1 -- 172.21.15.89:0/1707304240 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdaf4123680 con 0x7fdaf413f070 2024-08-19T10:46:56.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf9b7c640 1 -- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413f070 msgr2=0x7fdaf402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf9b7c640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413f070 0x7fdaf402d710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdaec01cc00 tx=0x7fdaec009190 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdaec0c9020 con 0x7fdaf413f070 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf27fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf27fc640 10 monclient: dump: 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.771 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7fdaec0b5660 con 0x7fdaf413f070 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf27fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413f070 msgr2=0x7fdaf402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf27fc640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413f070 0x7fdaf402d710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdaec01cc00 tx=0x7fdaec009190 comp rx=0 tx=0).stop 2024-08-19T10:46:56.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf27fc640 1 -- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413f070 msgr2=0x7fdaf402d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:56.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdaf27fc640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdaf413f070 0x7fdaf402d710 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdaec01cc00 tx=0x7fdaec009190 comp rx=0 tx=0).stop 2024-08-19T10:46:56.773 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 394348, 2024-08-19T10:46:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:56.773 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:56.774 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.775 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.776 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:56.777 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:56.778 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.779 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.780 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.782 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.785 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.786 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:56.789 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:56.790 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.791 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:56.793 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:56.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.795 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:56.796 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:56.797 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:56.797 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:56.797 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:56.797 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:56.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:56.797 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:56.797 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:56.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafbde0640 1 -- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdae003be30 msgr2=0x7fdae003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafbde0640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdae003be30 0x7fdae003e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdaf413f6f0 tx=0x7fdae8009040 comp rx=0 tx=0).stop 2024-08-19T10:46:56.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafbde0640 10 monclient: shutdown 2024-08-19T10:46:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafbde0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafbde0640 1 -- 172.21.15.89:0/1707304240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf40a7560 msgr2=0x7fdaf413df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafbde0640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf40a7560 0x7fdaf413df90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdae400e7f0 tx=0x7fdae400ecb0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafa37d640 1 -- 172.21.15.89:0/1707304240 reap_dead start 2024-08-19T10:46:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafbde0640 1 -- 172.21.15.89:0/1707304240 shutdown_connections 2024-08-19T10:46:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafbde0640 1 --2- 172.21.15.89:0/1707304240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdaf40a7560 0x7fdaf413df90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafbde0640 1 -- 172.21.15.89:0/1707304240 >> 172.21.15.89:0/1707304240 conn(0x7fdaf4074b40 msgr2=0x7fdaf40ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafbde0640 1 -- 172.21.15.89:0/1707304240 shutdown_connections 2024-08-19T10:46:56.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.764+0000 7fdafbde0640 1 -- 172.21.15.89:0/1707304240 wait complete. 2024-08-19T10:46:56.799 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:56.800 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-08-19T10:46:56.800 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-08-19T10:46:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.920+0000 7efcdbb42640 10 monclient: get_monmap_and_config 2024-08-19T10:46:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.920+0000 7efcdbb42640 10 monclient: build_initial_monmap 2024-08-19T10:46:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.920+0000 7efcdbb42640 10 monclient: monmap: 2024-08-19T10:46:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:56.923299+0000 2024-08-19T10:46:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:56.923299+0000 2024-08-19T10:46:56.920 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:56.921 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.920+0000 7efcdbb42640 1 Processor -- start 2024-08-19T10:46:56.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.920+0000 7efcdbb42640 1 -- start start 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.920+0000 7efcdbb42640 10 monclient: init 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 10 monclient: _add_conns ranks=[8,6,3,5,10,2,7,0,1,9,4] 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd40a6860 0x7efcd40a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 10 monclient(hunting): picked mon.noname-e con 0x7efcd40a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcd40a7180 0x7efcd40a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 10 monclient(hunting): picked mon.noname-d con 0x7efcd40a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcd4139620 0x7efcd4139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 10 monclient(hunting): picked mon.noname-g con 0x7efcd4139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7efcd40a8190 con 0x7efcd4139620 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7efcd40a33d0 con 0x7efcd40a7180 2024-08-19T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7efcd40a2ec0 con 0x7efcd40a6860 2024-08-19T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcdbb42640 10 monclient(hunting): authenticate will time out at 1466.740626s 2024-08-19T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd90dd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcd40a7180 0x7efcd40a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd40a6860 0x7efcd40a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd40a6860 0x7efcd40a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:36164/0 (socket says 172.21.15.89:36164) 2024-08-19T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd90dd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcd40a7180 0x7efcd40a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:35546/0 (socket says 172.21.15.89:35546) 2024-08-19T10:46:56.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 1 -- 172.21.15.89:0/2133537697 learned_addr learned my addr 172.21.15.89:0/2133537697 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 10 monclient(hunting): get_auth_request con 0x7efcd40a6860 auth_method 0 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd90dd640 10 monclient(hunting): get_auth_request con 0x7efcd40a7180 auth_method 0 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd90dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd90dd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd90dd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd90dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd90dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd90dd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcda0df640 1 --2- 172.21.15.89:0/2133537697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcd4139620 0x7efcd4139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcda0df640 10 monclient(hunting): get_auth_request con 0x7efcd4139620 auth_method 0 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcda0df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcda0df640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcda0df640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 10 monclient(hunting): handle_auth_done global_id 6524 payload 274 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 1 -- 172.21.15.89:0/2133537697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcd40a7180 msgr2=0x7efcd40a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 1 --2- 172.21.15.89:0/2133537697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcd40a7180 0x7efcd40a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 1 -- 172.21.15.89:0/2133537697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcd4139620 msgr2=0x7efcd4139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 1 --2- 172.21.15.89:0/2133537697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcd4139620 0x7efcd4139a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 10 monclient: _finish_hunting 0 2024-08-19T10:46:56.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 1 monclient: found mon.noname-e 2024-08-19T10:46:56.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:56.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:46:56.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.924+0000 7efcd98de640 1 -- 172.21.15.89:0/2133537697 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd40a4cd0 con 0x7efcd40a6860 2024-08-19T10:46:56.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd98de640 1 -- 172.21.15.89:0/2133537697 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd40a6860 msgr2=0x7efcd40a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd98de640 1 --2- 172.21.15.89:0/2133537697 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd40a6860 0x7efcd40a6c40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efcd00098a0 tx=0x7efcd001c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a1d0b75114262396 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 1 -- 172.21.15.89:0/2133537697 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efcd00b0070 con 0x7efcd40a6860 2024-08-19T10:46:56.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:46:56.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 10 monclient: dump: 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.932 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 1 -- 172.21.15.89:0/2133537697 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd00aa810 con 0x7efcd40a6860 2024-08-19T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 5 monclient: authenticate success, global_id 6524 2024-08-19T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 4 set_mon_vals no callback set 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 1 -- 172.21.15.89:0/2133537697 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efcd00a9300 con 0x7efcd40a6860 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcd88dc640 10 monclient: dump: 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.934 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient: shutdown 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 -- 172.21.15.89:0/2133537697 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd40a6860 msgr2=0x7efcd40a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 --2- 172.21.15.89:0/2133537697 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd40a6860 0x7efcd40a6c40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efcd00098a0 tx=0x7efcd001c440 comp rx=0 tx=0).stop 2024-08-19T10:46:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 -- 172.21.15.89:0/2133537697 shutdown_connections 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 --2- 172.21.15.89:0/2133537697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcd4139620 0x7efcd4139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 --2- 172.21.15.89:0/2133537697 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efcd40a7180 0x7efcd40a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 --2- 172.21.15.89:0/2133537697 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd40a6860 0x7efcd40a6c40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 -- 172.21.15.89:0/2133537697 >> 172.21.15.89:0/2133537697 conn(0x7efcd4074b40 msgr2=0x7efcd40abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 -- 172.21.15.89:0/2133537697 shutdown_connections 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 -- 172.21.15.89:0/2133537697 wait complete. 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient: build_initial_monmap 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient: monmap: 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 Processor -- start 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 -- start start 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient: init 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient: _add_conns ranks=[1,7,8,5,6,4,3,0,2] 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd40a6860 0x7efcd41d1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient(hunting): picked mon.noname-f con 0x7efcd40a6860 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efcd40a7180 0x7efcd40a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient(hunting): picked mon.noname-i con 0x7efcd40a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd4139620 0x7efcd40a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient(hunting): picked mon.noname-e con 0x7efcd4139620 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efcd4143210 con 0x7efcd40a6860 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7efcd4143090 con 0x7efcd40a7180 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7efcd4143390 con 0x7efcd4139620 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcdbb42640 10 monclient(hunting): authenticate will time out at 1466.747848s 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcda0df640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd4139620 0x7efcd40a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcda0df640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd4139620 0x7efcd40a20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:36170/0 (socket says 172.21.15.89:36170) 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.928+0000 7efcda0df640 1 -- 172.21.15.89:0/615620081 learned_addr learned my addr 172.21.15.89:0/615620081 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 10 monclient(hunting): get_auth_request con 0x7efcd4139620 auth_method 0 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcd90dd640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efcd40a7180 0x7efcd40a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcd98de640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd40a6860 0x7efcd41d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 10 monclient(hunting): handle_auth_done global_id 6533 payload 824 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 1 -- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efcd40a7180 msgr2=0x7efcd40a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efcd40a7180 0x7efcd40a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 1 -- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd40a6860 msgr2=0x7efcd41d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd40a6860 0x7efcd41d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 10 monclient: _finish_hunting 0 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 1 monclient: found mon.noname-e 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:46:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 1 -- 172.21.15.89:0/615620081 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd4140640 con 0x7efcd4139620 2024-08-19T10:46:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcd90dd640 10 monclient: get_auth_request con 0x7efcd40a7180 auth_method 0 2024-08-19T10:46:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcd98de640 10 monclient: get_auth_request con 0x7efcd40a6860 auth_method 0 2024-08-19T10:46:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcd90dd640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efcd40a7180 0x7efcd40a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:46:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcd98de640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd40a6860 0x7efcd41d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:46:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 1 -- 172.21.15.89:0/615620081 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd4139620 msgr2=0x7efcd40a20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcda0df640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd4139620 0x7efcd40a20a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efccc007d80 tx=0x7efccc00e4b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 1 -- 172.21.15.89:0/615620081 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efccc013a00 con 0x7efcd4139620 2024-08-19T10:46:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 10 monclient: dump: 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 10 monclient: _finish_auth 0 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 1 -- 172.21.15.89:0/615620081 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efccc019070 con 0x7efcd4139620 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 1 -- 172.21.15.89:0/615620081 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efccc013be0 con 0x7efcd4139620 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcdbb42640 5 monclient: authenticate success, global_id 6533 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcdbb42640 10 monclient: _renew_subs 2024-08-19T10:46:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcdbb42640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcdbb42640 1 -- 172.21.15.89:0/615620081 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd40a25e0 con 0x7efcd4139620 2024-08-19T10:46:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efccbfff640 4 set_mon_vals no callback set 2024-08-19T10:46:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 10 monclient: dump: 2024-08-19T10:46:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcdbb42640 10 monclient: _renew_subs 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcdbb42640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcdbb42640 1 -- 172.21.15.89:0/615620081 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd40a0070 con 0x7efcd4139620 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcdbb42640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcdbb42640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcdbb42640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcdbb42640 1 -- 172.21.15.89:0/615620081 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd40a94b0 con 0x7efcd4139620 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.932+0000 7efcca7fc640 1 -- 172.21.15.89:0/615620081 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efccc013590 con 0x7efcd4139620 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.936+0000 7efcca7fc640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcb803be00 0x7efcb803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.936+0000 7efcca7fc640 1 -- 172.21.15.89:0/615620081 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7efccc04fe60 con 0x7efcd4139620 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.936+0000 7efcd98de640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcb803be00 0x7efcb803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.936+0000 7efcd98de640 10 monclient: get_auth_request con 0x7efcb803be00 auth_method 0 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.936+0000 7efcd98de640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcb803be00 0x7efcb803e2c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.940+0000 7efcca7fc640 1 -- 172.21.15.89:0/615620081 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7efcd40a94b0 con 0x7efcd4139620 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.940+0000 7efcca7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:56.940+0000 7efcca7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:57.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.172+0000 7efcdbb42640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-08-19T10:46:57.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.172+0000 7efcdbb42640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-08-19T10:46:57.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.172+0000 7efcdbb42640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:46:57.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.172+0000 7efcdbb42640 1 -- 172.21.15.89:0/615620081 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7efcd40a7560 con 0x7efcd4139620 2024-08-19T10:46:57.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.176+0000 7efcca7fc640 1 -- 172.21.15.89:0/615620081 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7efccc053030 con 0x7efcd4139620 2024-08-19T10:46:57.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.176+0000 7efcca7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-08-19T10:46:57.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.176+0000 7efcca7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:57.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=604566036,config=2586702516,health=264905036,kv=4035213304,logm=639102778}) 2024-08-19T10:46:57.179 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=16,config=2,health=12,kv=11,logm=59} crc {auth=185483663,config=2586702517,health=264905036,kv=4035213304,logm=3631710347}) 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=604566036,config=2586702516,health=264905036,kv=4035213304,logm=639102778}) 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=15,config=2,health=12,kv=12,logm=59} crc {auth=2390603937,config=2586702516,health=264905036,kv=1101891974,logm=3661379017}) 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=604566036,config=2586702516,health=264905036,kv=4035213304,logm=639102778}) 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=1369310603,config=2586702516,health=3937121041,kv=1101891974,logm=1603679159}) 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=604566036,config=2586702516,health=264905036,kv=4035213304,logm=639102778}) 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=16,config=2,health=12,kv=11,logm=59} crc {auth=1369310603,config=2586702516,health=264905036,kv=2081965194,logm=2916073932}) 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=604566036,config=2586702516,health=264905036,kv=4035213304,logm=639102778}) 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=1369310603,config=2586702516,health=264905036,kv=1101891974,logm=1530080851}) 2024-08-19T10:46:57.180 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.181 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=604566036,config=2586702516,health=264905036,kv=4035213304,logm=639102778}) 2024-08-19T10:46:57.181 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=1369310603,config=2586702516,health=264905036,kv=1101891974,logm=2937023044}) 2024-08-19T10:46:57.181 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.181 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=604566036,config=2586702516,health=264905036,kv=4035213304,logm=639102778}) 2024-08-19T10:46:57.181 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=14,config=2,health=11,kv=10,logm=63} crc {auth=3429862150,config=2586702516,health=1179264661,kv=1979631103,logm=4085712799}) 2024-08-19T10:46:57.181 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.181 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=604566036,config=2586702516,health=264905036,kv=4035213304,logm=639102778}) 2024-08-19T10:46:57.181 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.180+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=14,config=2,health=11,kv=12,logm=61} crc {auth=2509598113,config=2586702516,health=1111782846,kv=1101891974,logm=3179934776}) 2024-08-19T10:46:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 -- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcb803be00 msgr2=0x7efcb803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcb803be00 0x7efcb803e2c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd00a6000 comp rx=0 tx=0).stop 2024-08-19T10:46:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 10 monclient: shutdown 2024-08-19T10:46:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 -- 172.21.15.89:0/615620081 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd4139620 msgr2=0x7efcd40a20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd4139620 0x7efcd40a20a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efccc007d80 tx=0x7efccc00e4b0 comp rx=0 tx=0).stop 2024-08-19T10:46:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 -- 172.21.15.89:0/615620081 shutdown_connections 2024-08-19T10:46:57.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efcb803be00 0x7efcb803e2c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7efcd4139620 0x7efcd40a20a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7efcd40a7180 0x7efcd40a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 --2- 172.21.15.89:0/615620081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd40a6860 0x7efcd41d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 -- 172.21.15.89:0/615620081 >> 172.21.15.89:0/615620081 conn(0x7efcd4074b40 msgr2=0x7efcd40a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 -- 172.21.15.89:0/615620081 shutdown_connections 2024-08-19T10:46:57.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:57.180+0000 7efcdbb42640 1 -- 172.21.15.89:0/615620081 wait complete. 2024-08-19T10:46:57.185 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.185 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265106580}) 2024-08-19T10:46:57.185 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523182613}) 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265106580}) 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3109817851}) 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265106580}) 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1488283057}) 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265106580}) 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=360095107}) 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265106580}) 2024-08-19T10:46:57.186 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1256025355}) 2024-08-19T10:46:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265106580}) 2024-08-19T10:46:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4246980407}) 2024-08-19T10:46:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265106580}) 2024-08-19T10:46:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2156131382}) 2024-08-19T10:46:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265106580}) 2024-08-19T10:46:57.187 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.184+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2213779039}) 2024-08-19T10:46:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.188+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.188+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2888334577}) 2024-08-19T10:46:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.188+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=578298883}) 2024-08-19T10:46:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.188+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.188+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2888334577}) 2024-08-19T10:46:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.188+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3048366506}) 2024-08-19T10:46:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2888334577}) 2024-08-19T10:46:57.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3882114497}) 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2888334577}) 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3753391255}) 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2888334577}) 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2641915978}) 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2888334577}) 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2618793876}) 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2888334577}) 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3351785923}) 2024-08-19T10:46:57.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2888334577}) 2024-08-19T10:46:57.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.192+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=14160884}) 2024-08-19T10:46:57.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493326549}) 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=675583525}) 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493326549}) 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2160812005}) 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493326549}) 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=824889467}) 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493326549}) 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=919481436}) 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493326549}) 2024-08-19T10:46:57.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3681650204}) 2024-08-19T10:46:57.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493326549}) 2024-08-19T10:46:57.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2360831403}) 2024-08-19T10:46:57.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493326549}) 2024-08-19T10:46:57.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2950576975}) 2024-08-19T10:46:57.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493326549}) 2024-08-19T10:46:57.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.196+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3586120707}) 2024-08-19T10:46:57.200 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.200 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232889658}) 2024-08-19T10:46:57.200 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2073278416}) 2024-08-19T10:46:57.200 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.200 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232889658}) 2024-08-19T10:46:57.200 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=358995590}) 2024-08-19T10:46:57.200 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.200 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232889658}) 2024-08-19T10:46:57.200 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=309304623}) 2024-08-19T10:46:57.200 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.200 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232889658}) 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1501070568}) 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232889658}) 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3412419858}) 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232889658}) 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4113750947}) 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232889658}) 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=500926281}) 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232889658}) 2024-08-19T10:46:57.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.200+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=808703256}) 2024-08-19T10:46:57.202 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-08-19T10:46:57.207 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.207 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4291744719}) 2024-08-19T10:46:57.207 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3066738915}) 2024-08-19T10:46:57.207 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.207 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4291744719}) 2024-08-19T10:46:57.207 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2868830308}) 2024-08-19T10:46:57.207 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.207 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4291744719}) 2024-08-19T10:46:57.207 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3401983176}) 2024-08-19T10:46:57.207 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.207 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4291744719}) 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1057389218}) 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4291744719}) 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=485529984}) 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4291744719}) 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3624215832}) 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4291744719}) 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=625327545}) 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4291744719}) 2024-08-19T10:46:57.208 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1546388704}) 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2199051796}) 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2263102524}) 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2199051796}) 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.204+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1776534576}) 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2199051796}) 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2849251462}) 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2199051796}) 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2072296745}) 2024-08-19T10:46:57.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2199051796}) 2024-08-19T10:46:57.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=897899636}) 2024-08-19T10:46:57.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2199051796}) 2024-08-19T10:46:57.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2286447358}) 2024-08-19T10:46:57.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2199051796}) 2024-08-19T10:46:57.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=334081657}) 2024-08-19T10:46:57.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2199051796}) 2024-08-19T10:46:57.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.208+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2586234275}) 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=32027443}) 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=74542318}) 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=32027443}) 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=662413411}) 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=32027443}) 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3586055179}) 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=32027443}) 2024-08-19T10:46:57.213 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1973443601}) 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=32027443}) 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2056104755}) 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=32027443}) 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2080488527}) 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=32027443}) 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2846719386}) 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=32027443}) 2024-08-19T10:46:57.214 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.212+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2686773383}) 2024-08-19T10:46:57.217 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.217 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3638312394}) 2024-08-19T10:46:57.217 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=22918952}) 2024-08-19T10:46:57.217 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.217 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3638312394}) 2024-08-19T10:46:57.217 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2198656664}) 2024-08-19T10:46:57.217 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.217 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3638312394}) 2024-08-19T10:46:57.217 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1283624759}) 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3638312394}) 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1261475010}) 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3638312394}) 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1077256819}) 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3638312394}) 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4014445519}) 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3638312394}) 2024-08-19T10:46:57.218 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2846246048}) 2024-08-19T10:46:57.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3638312394}) 2024-08-19T10:46:57.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.216+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3770530751}) 2024-08-19T10:46:57.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797675783}) 2024-08-19T10:46:57.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1441916050}) 2024-08-19T10:46:57.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797675783}) 2024-08-19T10:46:57.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1911885812}) 2024-08-19T10:46:57.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797675783}) 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3583550144}) 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797675783}) 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2057620152}) 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797675783}) 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3222256869}) 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797675783}) 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1698282948}) 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.222 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797675783}) 2024-08-19T10:46:57.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2957919966}) 2024-08-19T10:46:57.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3797675783}) 2024-08-19T10:46:57.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.220+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1974050536}) 2024-08-19T10:46:57.225 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.225 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56758236}) 2024-08-19T10:46:57.225 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3508215939}) 2024-08-19T10:46:57.225 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56758236}) 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3277129034}) 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56758236}) 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1847251073}) 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56758236}) 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3966599801}) 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56758236}) 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1592256555}) 2024-08-19T10:46:57.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56758236}) 2024-08-19T10:46:57.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1072019718}) 2024-08-19T10:46:57.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56758236}) 2024-08-19T10:46:57.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=670006340}) 2024-08-19T10:46:57.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56758236}) 2024-08-19T10:46:57.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.224+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1155672412}) 2024-08-19T10:46:57.229 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3666721095}) 2024-08-19T10:46:57.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=234803828}) 2024-08-19T10:46:57.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3666721095}) 2024-08-19T10:46:57.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1491433227}) 2024-08-19T10:46:57.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3666721095}) 2024-08-19T10:46:57.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=311145055}) 2024-08-19T10:46:57.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3666721095}) 2024-08-19T10:46:57.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2706548955}) 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3666721095}) 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1329006431}) 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3666721095}) 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2205862365}) 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3666721095}) 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=540413928}) 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3666721095}) 2024-08-19T10:46:57.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.228+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=475404736}) 2024-08-19T10:46:57.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=902761560}) 2024-08-19T10:46:57.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3103630282}) 2024-08-19T10:46:57.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=902761560}) 2024-08-19T10:46:57.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1492023225}) 2024-08-19T10:46:57.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=902761560}) 2024-08-19T10:46:57.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2903459414}) 2024-08-19T10:46:57.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=902761560}) 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=930751079}) 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=902761560}) 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2703608434}) 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=902761560}) 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3157627209}) 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=902761560}) 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2983269389}) 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=902761560}) 2024-08-19T10:46:57.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.232+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3645671920}) 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1712416698}) 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=99020347}) 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1712416698}) 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=592185872}) 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1712416698}) 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4159004166}) 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1712416698}) 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3282913917}) 2024-08-19T10:46:57.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1712416698}) 2024-08-19T10:46:57.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3366092060}) 2024-08-19T10:46:57.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1712416698}) 2024-08-19T10:46:57.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1284846907}) 2024-08-19T10:46:57.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1712416698}) 2024-08-19T10:46:57.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2548350300}) 2024-08-19T10:46:57.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1712416698}) 2024-08-19T10:46:57.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.236+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2932772895}) 2024-08-19T10:46:57.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1915577384}) 2024-08-19T10:46:57.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1828807043}) 2024-08-19T10:46:57.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1915577384}) 2024-08-19T10:46:57.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=219460658}) 2024-08-19T10:46:57.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1915577384}) 2024-08-19T10:46:57.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3107460218}) 2024-08-19T10:46:57.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1915577384}) 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=225966213}) 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1915577384}) 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1755000299}) 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1915577384}) 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3185152593}) 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1915577384}) 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3179975195}) 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.245 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1915577384}) 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.240+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2692002365}) 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578768731}) 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2519958752}) 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578768731}) 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=774265007}) 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578768731}) 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=589664570}) 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578768731}) 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3048000937}) 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578768731}) 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2677072871}) 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578768731}) 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1943375114}) 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578768731}) 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=415386548}) 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2578768731}) 2024-08-19T10:46:57.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3377743200}) 2024-08-19T10:46:57.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597625004}) 2024-08-19T10:46:57.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2746172257}) 2024-08-19T10:46:57.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597625004}) 2024-08-19T10:46:57.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3224781720}) 2024-08-19T10:46:57.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597625004}) 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.244+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=762203573}) 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597625004}) 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4022313312}) 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597625004}) 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=39869723}) 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597625004}) 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1380496543}) 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597625004}) 2024-08-19T10:46:57.252 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1804803288}) 2024-08-19T10:46:57.252 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.252 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597625004}) 2024-08-19T10:46:57.252 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.248+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=443999449}) 2024-08-19T10:46:57.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2342207934}) 2024-08-19T10:46:57.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3209452124}) 2024-08-19T10:46:57.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2342207934}) 2024-08-19T10:46:57.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1496031591}) 2024-08-19T10:46:57.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2342207934}) 2024-08-19T10:46:57.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=992649461}) 2024-08-19T10:46:57.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2342207934}) 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2725277081}) 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2342207934}) 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3284121066}) 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2342207934}) 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=534543824}) 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2342207934}) 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=187881946}) 2024-08-19T10:46:57.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2342207934}) 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.252+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4223379198}) 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3051453765}) 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1339687544}) 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3051453765}) 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=884811062}) 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3051453765}) 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=354399929}) 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3051453765}) 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=618186091}) 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3051453765}) 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3733138833}) 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3051453765}) 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2808281633}) 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3051453765}) 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2765166165}) 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3051453765}) 2024-08-19T10:46:57.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.256+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2640206023}) 2024-08-19T10:46:57.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2854031427}) 2024-08-19T10:46:57.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1396356981}) 2024-08-19T10:46:57.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2854031427}) 2024-08-19T10:46:57.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3138619484}) 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2854031427}) 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=500175193}) 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2854031427}) 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1502752052}) 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2854031427}) 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3793800161}) 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2854031427}) 2024-08-19T10:46:57.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=306563283}) 2024-08-19T10:46:57.282 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.282 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2854031427}) 2024-08-19T10:46:57.282 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3164207705}) 2024-08-19T10:46:57.282 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.282 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2854031427}) 2024-08-19T10:46:57.282 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.280+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3177702444}) 2024-08-19T10:46:57.284 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.284 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2214768978}) 2024-08-19T10:46:57.284 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2504530195}) 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2214768978}) 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2090799226}) 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2214768978}) 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2974475153}) 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2214768978}) 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3234211047}) 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2214768978}) 2024-08-19T10:46:57.285 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=130342881}) 2024-08-19T10:46:57.286 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.286 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2214768978}) 2024-08-19T10:46:57.286 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3369133788}) 2024-08-19T10:46:57.286 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.286 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2214768978}) 2024-08-19T10:46:57.286 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=753794242}) 2024-08-19T10:46:57.286 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.286 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2214768978}) 2024-08-19T10:46:57.286 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.284+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3071853801}) 2024-08-19T10:46:57.289 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.289 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698703392}) 2024-08-19T10:46:57.289 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1849805754}) 2024-08-19T10:46:57.289 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.289 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698703392}) 2024-08-19T10:46:57.289 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2289658426}) 2024-08-19T10:46:57.289 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.289 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698703392}) 2024-08-19T10:46:57.289 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2309327097}) 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698703392}) 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2814546941}) 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698703392}) 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4246731396}) 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698703392}) 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=171233777}) 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698703392}) 2024-08-19T10:46:57.290 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=908484589}) 2024-08-19T10:46:57.291 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.291 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2698703392}) 2024-08-19T10:46:57.291 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.288+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1070749253}) 2024-08-19T10:46:57.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974149150}) 2024-08-19T10:46:57.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2006600890}) 2024-08-19T10:46:57.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974149150}) 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1918083236}) 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974149150}) 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2866871479}) 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974149150}) 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=784043088}) 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974149150}) 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3323182086}) 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974149150}) 2024-08-19T10:46:57.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4036799683}) 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974149150}) 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3441313338}) 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1974149150}) 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1496307490}) 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.292+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614079699}) 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=229609073}) 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614079699}) 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3308866234}) 2024-08-19T10:46:57.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614079699}) 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=823309996}) 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614079699}) 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3133597205}) 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614079699}) 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2417283002}) 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614079699}) 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1905295474}) 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614079699}) 2024-08-19T10:46:57.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3150287225}) 2024-08-19T10:46:57.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2614079699}) 2024-08-19T10:46:57.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.296+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3539917746}) 2024-08-19T10:46:57.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483349830}) 2024-08-19T10:46:57.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1413959151}) 2024-08-19T10:46:57.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483349830}) 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=598040756}) 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483349830}) 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1445675126}) 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483349830}) 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3018214676}) 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483349830}) 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1199679034}) 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483349830}) 2024-08-19T10:46:57.304 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=543197596}) 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483349830}) 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3875059745}) 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3483349830}) 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=376639875}) 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=294504591}) 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=504904537}) 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=294504591}) 2024-08-19T10:46:57.305 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3928000779}) 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=294504591}) 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=81942899}) 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=294504591}) 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4172914042}) 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=294504591}) 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2019786622}) 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=294504591}) 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=715064718}) 2024-08-19T10:46:57.306 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.307 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=294504591}) 2024-08-19T10:46:57.307 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.300+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1603677928}) 2024-08-19T10:46:57.307 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.304+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.307 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.304+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=294504591}) 2024-08-19T10:46:57.307 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.304+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1184175872}) 2024-08-19T10:46:57.360 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3014381086}) 2024-08-19T10:46:57.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3519479960}) 2024-08-19T10:46:57.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3014381086}) 2024-08-19T10:46:57.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2280502137}) 2024-08-19T10:46:57.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3014381086}) 2024-08-19T10:46:57.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1056136773}) 2024-08-19T10:46:57.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3014381086}) 2024-08-19T10:46:57.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4224147610}) 2024-08-19T10:46:57.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3014381086}) 2024-08-19T10:46:57.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2773038080}) 2024-08-19T10:46:57.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3014381086}) 2024-08-19T10:46:57.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1260535425}) 2024-08-19T10:46:57.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3014381086}) 2024-08-19T10:46:57.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2872191921}) 2024-08-19T10:46:57.363 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.363 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3014381086}) 2024-08-19T10:46:57.363 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.360+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2807080309}) 2024-08-19T10:46:57.375 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1156499966}) 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2364014635}) 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1156499966}) 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1820290654}) 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1156499966}) 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=324190522}) 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1156499966}) 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2413559461}) 2024-08-19T10:46:57.376 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1156499966}) 2024-08-19T10:46:57.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2250224338}) 2024-08-19T10:46:57.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1156499966}) 2024-08-19T10:46:57.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=887206901}) 2024-08-19T10:46:57.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1156499966}) 2024-08-19T10:46:57.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=476800429}) 2024-08-19T10:46:57.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1156499966}) 2024-08-19T10:46:57.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.376+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2022716154}) 2024-08-19T10:46:57.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2792370719}) 2024-08-19T10:46:57.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1405049874}) 2024-08-19T10:46:57.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2792370719}) 2024-08-19T10:46:57.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=356580740}) 2024-08-19T10:46:57.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.634 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2792370719}) 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=334777433}) 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2792370719}) 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1698092561}) 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2792370719}) 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2653683143}) 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2792370719}) 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3288898978}) 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2792370719}) 2024-08-19T10:46:57.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3504396366}) 2024-08-19T10:46:57.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:57.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2792370719}) 2024-08-19T10:46:57.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:57.632+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=822442402}) 2024-08-19T10:46:58.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493521687}) 2024-08-19T10:46:58.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2943582299}) 2024-08-19T10:46:58.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493521687}) 2024-08-19T10:46:58.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1254774920}) 2024-08-19T10:46:58.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.124 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493521687}) 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=509049097}) 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493521687}) 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=305988622}) 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493521687}) 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=311096822}) 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493521687}) 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3676308741}) 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.125 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493521687}) 2024-08-19T10:46:58.126 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1473216317}) 2024-08-19T10:46:58.126 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.126 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3493521687}) 2024-08-19T10:46:58.126 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.124+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=556753543}) 2024-08-19T10:46:58.128 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.128 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3625969771}) 2024-08-19T10:46:58.128 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3394526887}) 2024-08-19T10:46:58.128 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.128 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3625969771}) 2024-08-19T10:46:58.128 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1265191138}) 2024-08-19T10:46:58.128 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.128 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3625969771}) 2024-08-19T10:46:58.128 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=367332287}) 2024-08-19T10:46:58.129 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.129 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3625969771}) 2024-08-19T10:46:58.129 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3321191387}) 2024-08-19T10:46:58.129 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.129 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3625969771}) 2024-08-19T10:46:58.129 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=361561748}) 2024-08-19T10:46:58.129 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.129 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3625969771}) 2024-08-19T10:46:58.129 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4237635456}) 2024-08-19T10:46:58.129 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.129 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3625969771}) 2024-08-19T10:46:58.130 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1191024156}) 2024-08-19T10:46:58.130 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.130 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3625969771}) 2024-08-19T10:46:58.130 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.128+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1532008889}) 2024-08-19T10:46:58.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3177129805}) 2024-08-19T10:46:58.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1094717432}) 2024-08-19T10:46:58.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.139 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3177129805}) 2024-08-19T10:46:58.140 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2870010615}) 2024-08-19T10:46:58.140 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.140 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3177129805}) 2024-08-19T10:46:58.140 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1385215911}) 2024-08-19T10:46:58.140 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.141 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3177129805}) 2024-08-19T10:46:58.141 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=962942256}) 2024-08-19T10:46:58.141 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.141 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3177129805}) 2024-08-19T10:46:58.141 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1042469066}) 2024-08-19T10:46:58.141 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.142 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3177129805}) 2024-08-19T10:46:58.142 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4234183196}) 2024-08-19T10:46:58.142 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.142 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3177129805}) 2024-08-19T10:46:58.142 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1104462801}) 2024-08-19T10:46:58.143 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.143 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3177129805}) 2024-08-19T10:46:58.143 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.132+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2323686297}) 2024-08-19T10:46:58.143 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.143 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3646160287}) 2024-08-19T10:46:58.143 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3644746245}) 2024-08-19T10:46:58.144 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.144 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3646160287}) 2024-08-19T10:46:58.144 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=297068160}) 2024-08-19T10:46:58.144 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.144 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3646160287}) 2024-08-19T10:46:58.145 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3102006364}) 2024-08-19T10:46:58.145 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.145 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3646160287}) 2024-08-19T10:46:58.145 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=925750663}) 2024-08-19T10:46:58.145 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.145 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3646160287}) 2024-08-19T10:46:58.146 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1879598306}) 2024-08-19T10:46:58.146 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.146 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3646160287}) 2024-08-19T10:46:58.146 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1262664325}) 2024-08-19T10:46:58.146 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.147 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3646160287}) 2024-08-19T10:46:58.147 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1504260926}) 2024-08-19T10:46:58.147 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.147 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3646160287}) 2024-08-19T10:46:58.147 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.136+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3257802600}) 2024-08-19T10:46:58.157 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.157 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=637008070}) 2024-08-19T10:46:58.157 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2907223381}) 2024-08-19T10:46:58.157 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.157 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=637008070}) 2024-08-19T10:46:58.157 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=426682168}) 2024-08-19T10:46:58.157 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.157 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=637008070}) 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2581604091}) 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=637008070}) 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1584435395}) 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=637008070}) 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2323815509}) 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=637008070}) 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2849610340}) 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=637008070}) 2024-08-19T10:46:58.158 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1271724518}) 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=637008070}) 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.140+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=922470022}) 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746904612}) 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1665349242}) 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746904612}) 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=323143366}) 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746904612}) 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=855238178}) 2024-08-19T10:46:58.159 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746904612}) 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3258251172}) 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746904612}) 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1565536574}) 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746904612}) 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3139630777}) 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746904612}) 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1687726882}) 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.160 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746904612}) 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.144+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=228055564}) 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3721081271}) 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3944925504}) 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3721081271}) 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2444127425}) 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3721081271}) 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3940992234}) 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3721081271}) 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2595946785}) 2024-08-19T10:46:58.161 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3721081271}) 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2942498872}) 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3721081271}) 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3466994298}) 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3721081271}) 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=493507209}) 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3721081271}) 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.148+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3730944224}) 2024-08-19T10:46:58.162 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105042544}) 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1467451529}) 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105042544}) 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2912106320}) 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105042544}) 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2107914377}) 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105042544}) 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=999561955}) 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.163 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105042544}) 2024-08-19T10:46:58.164 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1676739359}) 2024-08-19T10:46:58.164 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.164 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105042544}) 2024-08-19T10:46:58.164 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3637549633}) 2024-08-19T10:46:58.164 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.165 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105042544}) 2024-08-19T10:46:58.165 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4279746842}) 2024-08-19T10:46:58.165 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.165 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105042544}) 2024-08-19T10:46:58.165 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.152+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4001735103}) 2024-08-19T10:46:58.203 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:46:58.204 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:46:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient: get_monmap_and_config 2024-08-19T10:46:58.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient: build_initial_monmap 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient: monmap: 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:58.334762+0000 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:58.334762+0000 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:58.332 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:58.333 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:58.333 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 1 Processor -- start 2024-08-19T10:46:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 1 -- start start 2024-08-19T10:46:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient: init 2024-08-19T10:46:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient: _add_conns ranks=[6,4,0,3,7,5,8,10,1,2,9] 2024-08-19T10:46:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff5ec0a5270 0x7ff5ec0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient(hunting): picked mon.noname-d con 0x7ff5ec0a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5ec0a6d30 0x7ff5ec0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient(hunting): picked mon.noname-c con 0x7ff5ec0a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff5ec1395b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5ec0a8710 con 0x7ff5ec1395b0 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff5ec0a7650 con 0x7ff5ec0a6d30 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff5ec0a5a40 con 0x7ff5ec0a5270 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5f13b7640 10 monclient(hunting): authenticate will time out at 1468.151324s 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5e27fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5ec0a6d30 0x7ff5ec0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff5ec0a5270 0x7ff5ec0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5e27fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5ec0a6d30 0x7ff5ec0a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42262/0 (socket says 172.21.15.89:42262) 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5e27fc640 1 -- 172.21.15.89:0/3339583195 learned_addr learned my addr 172.21.15.89:0/3339583195 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:58.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 10 monclient(hunting): get_auth_request con 0x7ff5ec0a5270 auth_method 0 2024-08-19T10:46:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5e27fc640 10 monclient(hunting): get_auth_request con 0x7ff5ec0a6d30 auth_method 0 2024-08-19T10:46:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5e27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5e27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_done global_id 44574 payload 274 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 1 -- 172.21.15.89:0/3339583195 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5ec0a6d30 msgr2=0x7ff5ec0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 1 --2- 172.21.15.89:0/3339583195 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5ec0a6d30 0x7ff5ec0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 1 -- 172.21.15.89:0/3339583195 >> v1:172.21.15.89:6789/0 conn(0x7ff5ec1395b0 legacy=0x7ff5ec139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 10 monclient: _finish_hunting 0 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 1 monclient: found mon.noname-d 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:58.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:46:58.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5eaffd640 1 -- 172.21.15.89:0/3339583195 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec0a81d0 con 0x7ff5ec0a5270 2024-08-19T10:46:58.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.332+0000 7ff5e27fc640 1 --2- 172.21.15.89:0/3339583195 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5ec0a6d30 0x7ff5ec0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:46:58.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eaffd640 1 -- 172.21.15.89:0/3339583195 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff5ec0a5270 msgr2=0x7ff5ec0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:58.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eaffd640 1 --2- 172.21.15.89:0/3339583195 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff5ec0a5270 0x7ff5ec0a67f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40097f0 tx=0x7ff5d401c1c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a0ab2c4889269874 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 1 -- 172.21.15.89:0/3339583195 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5d40b0070 con 0x7ff5ec0a5270 2024-08-19T10:46:58.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 10 monclient: dump: 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.343 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 10 monclient: _finish_auth 0 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 1 -- 172.21.15.89:0/3339583195 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d40aa810 con 0x7ff5ec0a5270 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 1 -- 172.21.15.89:0/3339583195 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5d40a9300 con 0x7ff5ec0a5270 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 5 monclient: authenticate success, global_id 44574 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:58.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 4 set_mon_vals no callback set 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5ea7fc640 10 monclient: dump: 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.345 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient: shutdown 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3339583195 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff5ec0a5270 msgr2=0x7ff5ec0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 --2- 172.21.15.89:0/3339583195 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff5ec0a5270 0x7ff5ec0a67f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40097f0 tx=0x7ff5d401c1c0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3339583195 shutdown_connections 2024-08-19T10:46:58.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 --2- 172.21.15.89:0/3339583195 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff5ec0a6d30 0x7ff5ec0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 --2- 172.21.15.89:0/3339583195 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff5ec0a5270 0x7ff5ec0a67f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3339583195 >> 172.21.15.89:0/3339583195 conn(0x7ff5ec069a80 msgr2=0x7ff5ec0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3339583195 shutdown_connections 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3339583195 wait complete. 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient: build_initial_monmap 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient: monmap: 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:58.347 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 Processor -- start 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 -- start start 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient: init 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:58.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient: _add_conns ranks=[7,1,5,3,8,2,0,4,6] 2024-08-19T10:46:58.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5ec0a5270 0x7ff5ec13fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient(hunting): picked mon.noname-i con 0x7ff5ec0a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:46:58.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5ec0a6d30 0x7ff5ec1d1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient(hunting): picked mon.noname-f con 0x7ff5ec0a6d30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:46:58.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff5ec1395b0 0x7ff5ec13d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient(hunting): picked mon.noname-h con 0x7ff5ec1395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:58.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec0a57d0 con 0x7ff5ec0a6d30 2024-08-19T10:46:58.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff5ec0a5650 con 0x7ff5ec1395b0 2024-08-19T10:46:58.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff5ec144240 con 0x7ff5ec0a5270 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5f13b7640 10 monclient(hunting): authenticate will time out at 1468.158991s 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5e27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5ec0a6d30 0x7ff5ec1d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eaffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5ec0a5270 0x7ff5ec13fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff5ec1395b0 0x7ff5ec13d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff5ec1395b0 0x7ff5ec13d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:38688/0 (socket says 172.21.15.89:38688) 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eb7fe640 1 -- 172.21.15.89:0/3267313790 learned_addr learned my addr 172.21.15.89:0/3267313790 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eb7fe640 10 monclient(hunting): get_auth_request con 0x7ff5ec1395b0 auth_method 0 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eaffd640 10 monclient(hunting): get_auth_request con 0x7ff5ec0a5270 auth_method 0 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5e27fc640 10 monclient(hunting): get_auth_request con 0x7ff5ec0a6d30 auth_method 0 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5e27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5e27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5e27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5e27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5e27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.340+0000 7ff5eb7fe640 10 monclient(hunting): handle_auth_done global_id 6656 payload 824 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5eb7fe640 1 -- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5ec0a5270 msgr2=0x7ff5ec13fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5eb7fe640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5ec0a5270 0x7ff5ec13fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5eb7fe640 1 -- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5ec0a6d30 msgr2=0x7ff5ec1d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5eb7fe640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5ec0a6d30 0x7ff5ec1d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5eb7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5eb7fe640 1 monclient: found mon.noname-h 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5eb7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:46:58.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5eb7fe640 1 -- 172.21.15.89:0/3267313790 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec122d80 con 0x7ff5ec1395b0 2024-08-19T10:46:58.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5eb7fe640 1 -- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff5ec1395b0 msgr2=0x7ff5ec13d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:58.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5eb7fe640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff5ec1395b0 0x7ff5ec13d770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c780 tx=0x7ff5e400cc40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 1 -- 172.21.15.89:0/3267313790 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5e4004970 con 0x7ff5ec1395b0 2024-08-19T10:46:58.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 10 monclient: dump: 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.355 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 10 monclient: _finish_auth 0 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 10 monclient: _check_auth_tickets 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 1 -- 172.21.15.89:0/3267313790 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e4004b30 con 0x7ff5ec1395b0 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5f13b7640 5 monclient: authenticate success, global_id 6656 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5f13b7640 10 monclient: _renew_subs 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5f13b7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:46:58.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3267313790 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec140610 con 0x7ff5ec1395b0 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 1 -- 172.21.15.89:0/3267313790 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff5e4004ce0 con 0x7ff5ec1395b0 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e3fff640 10 monclient: dump: 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.357 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5e97fa640 4 set_mon_vals no callback set 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5f13b7640 10 monclient: _renew_subs 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5f13b7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:46:58.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3267313790 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec1389d0 con 0x7ff5ec1395b0 2024-08-19T10:46:58.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5f13b7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:58.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5f13b7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:58.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5f13b7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:46:58.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.344+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3267313790 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ec0a67f0 con 0x7ff5ec1395b0 2024-08-19T10:46:58.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.348+0000 7ff5e3fff640 1 -- 172.21.15.89:0/3267313790 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff5e4006b60 con 0x7ff5ec1395b0 2024-08-19T10:46:58.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.348+0000 7ff5e3fff640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff5cc03be00 0x7ff5cc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.348+0000 7ff5e3fff640 1 -- 172.21.15.89:0/3267313790 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7ff5e404ff70 con 0x7ff5ec1395b0 2024-08-19T10:46:58.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.348+0000 7ff5eaffd640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff5cc03be00 0x7ff5cc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.348+0000 7ff5eaffd640 10 monclient: get_auth_request con 0x7ff5cc03be00 auth_method 0 2024-08-19T10:46:58.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.348+0000 7ff5eaffd640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff5cc03be00 0x7ff5cc03e2c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0acc30 tx=0x7ff5d40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.360+0000 7ff5e3fff640 1 -- 172.21.15.89:0/3267313790 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff5e4020070 con 0x7ff5ec1395b0 2024-08-19T10:46:58.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.360+0000 7ff5e3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:58.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.360+0000 7ff5e3fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:58.513 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.513 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3495706297}) 2024-08-19T10:46:58.513 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=744385218}) 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3495706297}) 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=659598293}) 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3495706297}) 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3425870487}) 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3495706297}) 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=988519437}) 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3495706297}) 2024-08-19T10:46:58.514 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3950444196}) 2024-08-19T10:46:58.515 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.515 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3495706297}) 2024-08-19T10:46:58.515 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2135925569}) 2024-08-19T10:46:58.515 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.515 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3495706297}) 2024-08-19T10:46:58.515 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2180043082}) 2024-08-19T10:46:58.515 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.515 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3495706297}) 2024-08-19T10:46:58.515 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.512+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=463602860}) 2024-08-19T10:46:58.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1962670386}) 2024-08-19T10:46:58.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1694289910}) 2024-08-19T10:46:58.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1962670386}) 2024-08-19T10:46:58.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3862919927}) 2024-08-19T10:46:58.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1962670386}) 2024-08-19T10:46:58.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1489977379}) 2024-08-19T10:46:58.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1962670386}) 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1402671780}) 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1962670386}) 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3494176159}) 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1962670386}) 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3607061259}) 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1962670386}) 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1017459177}) 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1962670386}) 2024-08-19T10:46:58.595 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.592+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2451454857}) 2024-08-19T10:46:58.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.604+0000 7ff5f13b7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:46:58.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.604+0000 7ff5f13b7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:46:58.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.604+0000 7ff5f13b7640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:46:58.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.604+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3267313790 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff5ec064590 con 0x7ff5ec1395b0 2024-08-19T10:46:58.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.620+0000 7ff5e3fff640 1 -- 172.21.15.89:0/3267313790 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7ff5e401adb0 con 0x7ff5ec1395b0 2024-08-19T10:46:58.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.620+0000 7ff5e3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:46:58.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.620+0000 7ff5e3fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:58.621 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":82,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:46:58.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454049363}) 2024-08-19T10:46:58.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2403302322}) 2024-08-19T10:46:58.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454049363}) 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2114127995}) 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454049363}) 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3684982989}) 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454049363}) 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=607991100}) 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454049363}) 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=72736818}) 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454049363}) 2024-08-19T10:46:58.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3364522937}) 2024-08-19T10:46:58.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454049363}) 2024-08-19T10:46:58.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=162987400}) 2024-08-19T10:46:58.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1454049363}) 2024-08-19T10:46:58.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1649281937}) 2024-08-19T10:46:58.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff5cc03be00 msgr2=0x7ff5cc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff5cc03be00 0x7ff5cc03e2c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0acc30 tx=0x7ff5d40a6000 comp rx=0 tx=0).stop 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 10 monclient: shutdown 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff5ec1395b0 msgr2=0x7ff5ec13d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff5ec1395b0 0x7ff5ec13d770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c780 tx=0x7ff5e400cc40 comp rx=0 tx=0).stop 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3267313790 shutdown_connections 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff5cc03be00 0x7ff5cc03e2c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff5ec1395b0 0x7ff5ec13d770 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5ec0a6d30 0x7ff5ec1d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 --2- 172.21.15.89:0/3267313790 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff5ec0a5270 0x7ff5ec13fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3267313790 >> 172.21.15.89:0/3267313790 conn(0x7ff5ec069a80 msgr2=0x7ff5ec0a9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3267313790 shutdown_connections 2024-08-19T10:46:58.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.624+0000 7ff5f13b7640 1 -- 172.21.15.89:0/3267313790 wait complete. 2024-08-19T10:46:58.643 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:46:58.643 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-08-19T10:46:58.643 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:46:58.643 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:46:58.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient: get_monmap_and_config 2024-08-19T10:46:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient: build_initial_monmap 2024-08-19T10:46:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient: monmap: 2024-08-19T10:46:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:58.765091+0000 2024-08-19T10:46:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:58.765091+0000 2024-08-19T10:46:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:58.762 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:58.763 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:58.763 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:58.763 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:58.763 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:58.763 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:58.763 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:58.763 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:58.763 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:58.763 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:58.763 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 1 Processor -- start 2024-08-19T10:46:58.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 1 -- start start 2024-08-19T10:46:58.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient: init 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient: _add_conns ranks=[4,10,7,3,1,9,6,2,0,5,8] 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5e41378c0 0x7fd5e4137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient(hunting): picked mon.noname-c con 0x7fd5e41378c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5e41381e0 0x7fd5e40a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient(hunting): picked mon.noname-f con 0x7fd5e41381e0 addr v2:172.21.15.190:3300/0 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd5e4072250 0x7fd5e40a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient(hunting): picked mon.noname-i con 0x7fd5e4072250 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd5e40a37b0 con 0x7fd5e41381e0 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd5e40726d0 con 0x7fd5e41378c0 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd5e4074010 con 0x7fd5e4072250 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e96d4640 10 monclient(hunting): authenticate will time out at 1468.581635s 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5e41378c0 0x7fd5e4137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5e41378c0 0x7fd5e4137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42268/0 (socket says 172.21.15.89:42268) 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 1 -- 172.21.15.89:0/3841811425 learned_addr learned my addr 172.21.15.89:0/3841811425 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 10 monclient(hunting): get_auth_request con 0x7fd5e41378c0 auth_method 0 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e37fe640 1 --2- 172.21.15.89:0/3841811425 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd5e4072250 0x7fd5e40a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e37fe640 10 monclient(hunting): get_auth_request con 0x7fd5e4072250 auth_method 0 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_done global_id 44572 payload 274 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e27fc640 1 --2- 172.21.15.89:0/3841811425 >> v2:172.21.15.190:3300/0 conn(0x7fd5e41381e0 0x7fd5e40a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 1 -- 172.21.15.89:0/3841811425 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd5e4072250 msgr2=0x7fd5e40a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 1 --2- 172.21.15.89:0/3841811425 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd5e4072250 0x7fd5e40a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 1 -- 172.21.15.89:0/3841811425 >> v2:172.21.15.190:3300/0 conn(0x7fd5e41381e0 msgr2=0x7fd5e40a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 1 --2- 172.21.15.89:0/3841811425 >> v2:172.21.15.190:3300/0 conn(0x7fd5e41381e0 0x7fd5e40a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 1 monclient: found mon.noname-c 2024-08-19T10:46:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:58.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:46:58.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.764+0000 7fd5e2ffd640 1 -- 172.21.15.89:0/3841811425 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e40a4300 con 0x7fd5e41378c0 2024-08-19T10:46:58.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e2ffd640 1 -- 172.21.15.89:0/3841811425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5e41378c0 msgr2=0x7fd5e4137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:58.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e2ffd640 1 --2- 172.21.15.89:0/3841811425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5e41378c0 0x7fd5e4137ca0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009870 tx=0x7fd5d801c420 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1daf2ce0b89a7c24 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 1 -- 172.21.15.89:0/3841811425 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd5d80b0070 con 0x7fd5e41378c0 2024-08-19T10:46:58.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 10 monclient: dump: 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.771 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 5 monclient: authenticate success, global_id 44572 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 1 -- 172.21.15.89:0/3841811425 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d80aa810 con 0x7fd5e41378c0 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:58.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 1 -- 172.21.15.89:0/3841811425 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd5d80a9300 con 0x7fd5e41378c0 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 4 set_mon_vals no callback set 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e1ffb640 10 monclient: dump: 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.773 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.774 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.774 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.774 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.774 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.774 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.774 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.774 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient: shutdown 2024-08-19T10:46:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3841811425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5e41378c0 msgr2=0x7fd5e4137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 --2- 172.21.15.89:0/3841811425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5e41378c0 0x7fd5e4137ca0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009870 tx=0x7fd5d801c420 comp rx=0 tx=0).stop 2024-08-19T10:46:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3841811425 shutdown_connections 2024-08-19T10:46:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 --2- 172.21.15.89:0/3841811425 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd5e4072250 0x7fd5e40a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 --2- 172.21.15.89:0/3841811425 >> v2:172.21.15.190:3300/0 conn(0x7fd5e41381e0 0x7fd5e40a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 --2- 172.21.15.89:0/3841811425 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd5e41378c0 0x7fd5e4137ca0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3841811425 >> 172.21.15.89:0/3841811425 conn(0x7fd5e40aa240 msgr2=0x7fd5e40aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3841811425 shutdown_connections 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3841811425 wait complete. 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient: build_initial_monmap 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient: monmap: 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:58.781 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.781 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:58.781 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:58.781 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:58.781 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:58.781 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:58.781 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:58.781 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:58.781 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:58.781 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 Processor -- start 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 -- start start 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient: init 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient: _add_conns ranks=[5,2,6,7,3,0,8,4,1] 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd5e4072250 0x7fd5e4136900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient(hunting): picked mon.noname-h con 0x7fd5e4072250 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5e41378c0 0x7fd5e4136e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient(hunting): picked mon.noname-b con 0x7fd5e41378c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd5e41381e0 0x7fd5e4120490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient(hunting): picked mon.noname-d con 0x7fd5e41381e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e4140bf0 con 0x7fd5e41378c0 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd5e4140a70 con 0x7fd5e4072250 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd5e4140d70 con 0x7fd5e41381e0 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e96d4640 10 monclient(hunting): authenticate will time out at 1468.591303s 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5e41378c0 0x7fd5e4136e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e37fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd5e41381e0 0x7fd5e4120490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5e41378c0 0x7fd5e4136e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44958/0 (socket says 172.21.15.89:44958) 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 1 -- 172.21.15.89:0/3242699410 learned_addr learned my addr 172.21.15.89:0/3242699410 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:58.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 10 monclient(hunting): get_auth_request con 0x7fd5e41378c0 auth_method 0 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e37fe640 10 monclient(hunting): get_auth_request con 0x7fd5e41381e0 auth_method 0 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e2ffd640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd5e4072250 0x7fd5e4136900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e2ffd640 10 monclient(hunting): get_auth_request con 0x7fd5e4072250 auth_method 0 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 10 monclient(hunting): handle_auth_done global_id 54191 payload 824 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 1 -- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd5e41381e0 msgr2=0x7fd5e4120490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd5e41381e0 0x7fd5e4120490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 1 -- 172.21.15.89:0/3242699410 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd5e4072250 msgr2=0x7fd5e4136900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd5e4072250 0x7fd5e4136900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 10 monclient: _finish_hunting 0 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 1 monclient: found mon.noname-b 2024-08-19T10:46:58.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.772+0000 7fd5e27fc640 1 -- 172.21.15.89:0/3242699410 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e40a8de0 con 0x7fd5e41378c0 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e27fc640 1 -- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5e41378c0 msgr2=0x7fd5e4136e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e27fc640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5e41378c0 0x7fd5e4136e40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400c7f0 tx=0x7fd5d400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd5d4015070 con 0x7fd5e41378c0 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 10 monclient: dump: 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.786 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d4007ca0 con 0x7fd5e41378c0 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd5d4007e50 con 0x7fd5e41378c0 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 10 monclient: dump: 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.788 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e0ff9640 4 set_mon_vals no callback set 2024-08-19T10:46:58.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e96d4640 5 monclient: authenticate success, global_id 54191 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e96d4640 10 monclient: _renew_subs 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e96d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3242699410 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e413dcd0 con 0x7fd5e41378c0 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e96d4640 10 monclient: _renew_subs 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e96d4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3242699410 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41242c0 con 0x7fd5e41378c0 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e96d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e96d4640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e41d1470 0x7fd5e402d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e96d4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3242699410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4123f40 con 0x7fd5e41d1470 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e2ffd640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e41d1470 0x7fd5e402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd5d4018070 con 0x7fd5e41378c0 2024-08-19T10:46:58.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e2ffd640 10 monclient: get_auth_request con 0x7fd5e41d1470 auth_method 0 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.776+0000 7fd5cb7fe640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd5d003bfa0 0x7fd5d003e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_done global_id 6687 payload 824 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5e2ffd640 1 -- 172.21.15.89:0/3242699410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5e4137ca0 con 0x7fd5e41d1470 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fd5d404f340 con 0x7fd5e41378c0 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5e37fe640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd5d003bfa0 0x7fd5d003e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5e37fe640 10 monclient: get_auth_request con 0x7fd5d003bfa0 auth_method 0 2024-08-19T10:46:58.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5e2ffd640 1 -- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e41d1470 msgr2=0x7fd5e402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:58.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5e2ffd640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e41d1470 0x7fd5e402d1d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80ad000 tx=0x7fd5d80191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd5d80b9070 con 0x7fd5e41d1470 2024-08-19T10:46:58.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:58.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 10 monclient: dump: 2024-08-19T10:46:58.794 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.794 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.794 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.795 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd5d80b1040 con 0x7fd5e41d1470 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e41d1470 msgr2=0x7fd5e402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e41d1470 0x7fd5e402d1d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80ad000 tx=0x7fd5d80191c0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e41d1470 msgr2=0x7fd5e402d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5cb7fe640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e41d1470 0x7fd5e402d1d0 secure :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80ad000 tx=0x7fd5d80191c0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.780+0000 7fd5e37fe640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd5d003bfa0 0x7fd5d003e460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd5e41227e0 tx=0x7fd5dc009820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e96d4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e96d4640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e402d710 0x7fd5e402db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e96d4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3242699410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41d2140 con 0x7fd5e402d710 2024-08-19T10:46:58.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e402d710 0x7fd5e402db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 10 monclient: get_auth_request con 0x7fd5e402d710 auth_method 0 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 10 monclient(hunting): handle_auth_done global_id 6696 payload 824 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 1 -- 172.21.15.89:0/3242699410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd5e4064590 con 0x7fd5e402d710 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 1 -- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e402d710 msgr2=0x7fd5e402db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5e2ffd640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e402d710 0x7fd5e402db20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009870 tx=0x7fd5d80a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd5d80b0050 con 0x7fd5e402d710 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5cb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5cb7fe640 10 monclient: dump: 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.798 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.799 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.799 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.799 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.799 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9436 (secure 0 0 0) 0x7fd5d80c2070 con 0x7fd5e402d710 2024-08-19T10:46:58.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5cb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:58.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.788+0000 7fd5cb7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:58.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e402d710 msgr2=0x7fd5e402db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5cb7fe640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e402d710 0x7fd5e402db20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009870 tx=0x7fd5d80a8000 comp rx=0 tx=0).stop 2024-08-19T10:46:58.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5cb7fe640 1 -- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e402d710 msgr2=0x7fd5e402db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:58.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5cb7fe640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5e402d710 0x7fd5e402db20 secure :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009870 tx=0x7fd5d80a8000 comp rx=0 tx=0).stop 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 548601, 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:58.800 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:58.801 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:58.802 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:58.803 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:58.804 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:58.805 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:58.805 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:58.805 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:58.805 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:58.805 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:58.805 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:58.805 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:58.805 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:58.805 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:58.805 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.806 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.807 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.808 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.809 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.810 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:58.812 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:58.813 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.815 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:58.816 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.817 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.818 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.819 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:58.820 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.821 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:58.822 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.824 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:58.824 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3242699410 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd5d003bfa0 msgr2=0x7fd5d003e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e96d4640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd5d003bfa0 0x7fd5d003e460 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd5e41227e0 tx=0x7fd5dc009820 comp rx=0 tx=0).stop 2024-08-19T10:46:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e96d4640 10 monclient: shutdown 2024-08-19T10:46:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e96d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5e41378c0 msgr2=0x7fd5e4136e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e96d4640 1 --2- 172.21.15.89:0/3242699410 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd5e41378c0 0x7fd5e4136e40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400c7f0 tx=0x7fd5d400ccb0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e37fe640 1 -- 172.21.15.89:0/3242699410 reap_dead start 2024-08-19T10:46:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e37fe640 1 -- 172.21.15.89:0/3242699410 reap_dead start 2024-08-19T10:46:58.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3242699410 shutdown_connections 2024-08-19T10:46:58.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3242699410 >> 172.21.15.89:0/3242699410 conn(0x7fd5e40aa240 msgr2=0x7fd5e40a9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:58.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3242699410 shutdown_connections 2024-08-19T10:46:58.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.792+0000 7fd5e96d4640 1 -- 172.21.15.89:0/3242699410 wait complete. 2024-08-19T10:46:58.826 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:58.826 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:46:58.826 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:46:58.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1493343197}) 2024-08-19T10:46:58.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3058270227}) 2024-08-19T10:46:58.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1493343197}) 2024-08-19T10:46:58.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2241585522}) 2024-08-19T10:46:58.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1493343197}) 2024-08-19T10:46:58.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1785135796}) 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1493343197}) 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1284819555}) 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1493343197}) 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3741689588}) 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1493343197}) 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=686475077}) 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1493343197}) 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2283141425}) 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1493343197}) 2024-08-19T10:46:58.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2428155166}) 2024-08-19T10:46:58.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2948682135}) 2024-08-19T10:46:58.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=722886427}) 2024-08-19T10:46:58.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2948682135}) 2024-08-19T10:46:58.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3210975772}) 2024-08-19T10:46:58.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2948682135}) 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1064914088}) 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2948682135}) 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2545006773}) 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2948682135}) 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2553039842}) 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2948682135}) 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4173861149}) 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2948682135}) 2024-08-19T10:46:58.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1971588700}) 2024-08-19T10:46:58.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2948682135}) 2024-08-19T10:46:58.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2944661307}) 2024-08-19T10:46:58.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035892977}) 2024-08-19T10:46:58.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321771510}) 2024-08-19T10:46:58.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035892977}) 2024-08-19T10:46:58.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2593468847}) 2024-08-19T10:46:58.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035892977}) 2024-08-19T10:46:58.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2615787969}) 2024-08-19T10:46:58.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035892977}) 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1803105007}) 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035892977}) 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1321812838}) 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035892977}) 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1248872653}) 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035892977}) 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2899537691}) 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2035892977}) 2024-08-19T10:46:58.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:58.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3195342902}) 2024-08-19T10:46:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient: get_monmap_and_config 2024-08-19T10:46:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient: build_initial_monmap 2024-08-19T10:46:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient: monmap: 2024-08-19T10:46:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:58.947 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:58.949990+0000 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:58.949990+0000 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:58.948 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:58.949 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:58.949 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:58.949 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 1 Processor -- start 2024-08-19T10:46:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 1 -- start start 2024-08-19T10:46:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient: init 2024-08-19T10:46:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient: _add_conns ranks=[2,6,5,10,9,3,7,8,0,1,4] 2024-08-19T10:46:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92406c930 0x7fd92409d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient(hunting): picked mon.noname-b con 0x7fd92406c930 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd92406cd10 0x7fd92406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient(hunting): picked mon.noname-d con 0x7fd92406cd10 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd9241375e0 0x7fd924137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient(hunting): picked mon.noname-h con 0x7fd9241375e0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd9240a3ea0 con 0x7fd92406c930 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd9240a3180 con 0x7fd9241375e0 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd92409ca80 con 0x7fd92406cd10 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd928c76640 10 monclient(hunting): authenticate will time out at 1468.767192s 2024-08-19T10:46:58.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd92406cd10 0x7fd92406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd92259c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92406c930 0x7fd92409d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd92259c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92406c930 0x7fd92409d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44966/0 (socket says 172.21.15.89:44966) 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd92406cd10 0x7fd92406d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:35580/0 (socket says 172.21.15.89:35580) 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd92259c640 1 -- 172.21.15.89:0/2481416235 learned_addr learned my addr 172.21.15.89:0/2481416235 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd92259c640 10 monclient(hunting): get_auth_request con 0x7fd92406c930 auth_method 0 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd92259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd92259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd92259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 10 monclient(hunting): get_auth_request con 0x7fd92406cd10 auth_method 0 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd922d9d640 1 --2- 172.21.15.89:0/2481416235 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd9241375e0 0x7fd924137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 10 monclient(hunting): handle_auth_done global_id 44592 payload 274 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 1 -- 172.21.15.89:0/2481416235 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd9241375e0 msgr2=0x7fd924137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 1 --2- 172.21.15.89:0/2481416235 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd9241375e0 0x7fd924137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 1 -- 172.21.15.89:0/2481416235 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92406c930 msgr2=0x7fd92409d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 1 --2- 172.21.15.89:0/2481416235 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92406c930 0x7fd92409d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:46:58.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 1 monclient: found mon.noname-d 2024-08-19T10:46:58.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:58.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:46:58.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd921d9b640 1 -- 172.21.15.89:0/2481416235 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9240a4f40 con 0x7fd92406cd10 2024-08-19T10:46:58.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd92259c640 1 --2- 172.21.15.89:0/2481416235 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92406c930 0x7fd92409d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:58.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd922d9d640 10 monclient: get_auth_request con 0x7fd9241375e0 auth_method 0 2024-08-19T10:46:58.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.948+0000 7fd922d9d640 1 --2- 172.21.15.89:0/2481416235 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd9241375e0 0x7fd924137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:46:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd921d9b640 1 -- 172.21.15.89:0/2481416235 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd92406cd10 msgr2=0x7fd92406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd921d9b640 1 --2- 172.21.15.89:0/2481416235 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd92406cd10 0x7fd92406d0f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd918009820 tx=0x7fd91801c380 comp rx=0 tx=0).ready entity=mon.6 client_cookie=152dbe8107b56baa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 1 -- 172.21.15.89:0/2481416235 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd9180b0070 con 0x7fd92406cd10 2024-08-19T10:46:58.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:46:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 10 monclient: dump: 2024-08-19T10:46:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.959 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 10 monclient: _finish_auth 0 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 10 monclient: _check_auth_tickets 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:58.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 1 -- 172.21.15.89:0/2481416235 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9180aa810 con 0x7fd92406cd10 2024-08-19T10:46:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 5 monclient: authenticate success, global_id 44592 2024-08-19T10:46:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:46:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 4 set_mon_vals no callback set 2024-08-19T10:46:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 1 -- 172.21.15.89:0/2481416235 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd9180a9300 con 0x7fd92406cd10 2024-08-19T10:46:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:46:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92159a640 10 monclient: dump: 2024-08-19T10:46:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.961 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.962 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.963 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient: shutdown 2024-08-19T10:46:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 -- 172.21.15.89:0/2481416235 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd92406cd10 msgr2=0x7fd92406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 --2- 172.21.15.89:0/2481416235 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd92406cd10 0x7fd92406d0f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd918009820 tx=0x7fd91801c380 comp rx=0 tx=0).stop 2024-08-19T10:46:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 -- 172.21.15.89:0/2481416235 shutdown_connections 2024-08-19T10:46:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 --2- 172.21.15.89:0/2481416235 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd9241375e0 0x7fd924137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 --2- 172.21.15.89:0/2481416235 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd92406cd10 0x7fd92406d0f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 --2- 172.21.15.89:0/2481416235 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92406c930 0x7fd92409d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 -- 172.21.15.89:0/2481416235 >> 172.21.15.89:0/2481416235 conn(0x7fd9240a9130 msgr2=0x7fd9240a9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:58.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 -- 172.21.15.89:0/2481416235 shutdown_connections 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 -- 172.21.15.89:0/2481416235 wait complete. 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient: build_initial_monmap 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient: monmap: 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:58.964 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:58.965 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 Processor -- start 2024-08-19T10:46:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 -- start start 2024-08-19T10:46:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient: init 2024-08-19T10:46:58.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient: _add_conns ranks=[0,3,2,4,6,5,1,7,8] 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd92406c930 0x7fd924136130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient(hunting): picked mon.noname-a con 0x7fd92406c930 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd92406cd10 0x7fd924136670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient(hunting): picked mon.noname-g con 0x7fd92406cd10 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9241375e0 0x7fd92411f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient(hunting): picked mon.noname-b con 0x7fd9241375e0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd924141240 con 0x7fd92406c930 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd9241410c0 con 0x7fd9241375e0 2024-08-19T10:46:58.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd9241413c0 con 0x7fd92406cd10 2024-08-19T10:46:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd928c76640 10 monclient(hunting): authenticate will time out at 1468.775201s 2024-08-19T10:46:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd92406c930 0x7fd924136130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9241375e0 0x7fd92411f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd92406c930 0x7fd924136130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50734/0 (socket says 172.21.15.89:50734) 2024-08-19T10:46:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92259c640 1 -- 172.21.15.89:0/3082718680 learned_addr learned my addr 172.21.15.89:0/3082718680 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92259c640 10 monclient(hunting): get_auth_request con 0x7fd92406c930 auth_method 0 2024-08-19T10:46:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 10 monclient(hunting): get_auth_request con 0x7fd9241375e0 auth_method 0 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd921d9b640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd92406cd10 0x7fd924136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 10 monclient(hunting): handle_auth_done global_id 54209 payload 824 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 1 -- 172.21.15.89:0/3082718680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd92406cd10 msgr2=0x7fd924136670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd92406cd10 0x7fd924136670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 1 -- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd92406c930 msgr2=0x7fd924136130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd92406c930 0x7fd924136130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 1 monclient: found mon.noname-b 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd922d9d640 1 -- 172.21.15.89:0/3082718680 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd924121e10 con 0x7fd9241375e0 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd921d9b640 10 monclient: get_auth_request con 0x7fd92406cd10 auth_method 0 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd921d9b640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd92406cd10 0x7fd924136670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:46:58.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.956+0000 7fd92259c640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd92406c930 0x7fd924136130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd922d9d640 1 -- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9241375e0 msgr2=0x7fd92411f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd922d9d640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9241375e0 0x7fd92411f380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00cce0 tx=0x7fd91c00a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd91c00fb00 con 0x7fd9241375e0 2024-08-19T10:46:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 10 monclient: dump: 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.975 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 10 monclient: _finish_auth 0 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd91c00fd00 con 0x7fd9241375e0 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd928c76640 5 monclient: authenticate success, global_id 54209 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd928c76640 10 monclient: _renew_subs 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd928c76640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd928c76640 1 -- 172.21.15.89:0/3082718680 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9240a9610 con 0x7fd9241375e0 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd91c00e850 con 0x7fd9241375e0 2024-08-19T10:46:58.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 10 monclient: dump: 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.977 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd928c76640 10 monclient: _renew_subs 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd920d99640 4 set_mon_vals no callback set 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd928c76640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd928c76640 1 -- 172.21.15.89:0/3082718680 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd92413cfd0 con 0x7fd9241375e0 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd928c76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd928c76640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92411f8c0 0x7fd92411fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd928c76640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd928c76640 1 -- 172.21.15.89:0/3082718680 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd924064590 con 0x7fd92411f8c0 2024-08-19T10:46:58.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd91c00e3e0 con 0x7fd9241375e0 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd92259c640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92411f8c0 0x7fd92411fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd92259c640 10 monclient: get_auth_request con 0x7fd92411f8c0 auth_method 0 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd92259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd92259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd92259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd90403be30 0x7fd90403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fd91c04fe40 con 0x7fd9241375e0 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd92259c640 10 monclient(hunting): handle_auth_done global_id 54218 payload 824 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd92259c640 1 -- 172.21.15.89:0/3082718680 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd9240a9840 con 0x7fd92411f8c0 2024-08-19T10:46:58.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd921d9b640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd90403be30 0x7fd90403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.960+0000 7fd921d9b640 10 monclient: get_auth_request con 0x7fd90403be30 auth_method 0 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd92259c640 1 -- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92411f8c0 msgr2=0x7fd92411fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd92259c640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92411f8c0 0x7fd92411fca0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd914011db0 tx=0x7fd91400f610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd914014cd0 con 0x7fd92411f8c0 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd912ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd912ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd912ffd640 10 monclient: dump: 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.980 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd914014860 con 0x7fd92411f8c0 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd912ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:58.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd912ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92411f8c0 msgr2=0x7fd92411fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd912ffd640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92411f8c0 0x7fd92411fca0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd914011db0 tx=0x7fd91400f610 comp rx=0 tx=0).stop 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92411f8c0 msgr2=0x7fd92411fca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd912ffd640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd92411f8c0 0x7fd92411fca0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd914011db0 tx=0x7fd91400f610 comp rx=0 tx=0).stop 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.964+0000 7fd921d9b640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd90403be30 0x7fd90403e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd9180097f0 tx=0x7fd9180a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd928c76640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd928c76640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd924138660 0x7fd924138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd928c76640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd928c76640 1 -- 172.21.15.89:0/3082718680 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd9241201e0 con 0x7fd924138660 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd924138660 0x7fd924138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 10 monclient: get_auth_request con 0x7fd924138660 auth_method 0 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:58.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 10 monclient(hunting): handle_auth_done global_id 54227 payload 824 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 1 -- 172.21.15.89:0/3082718680 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd92409d4b0 con 0x7fd924138660 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 1 -- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd924138660 msgr2=0x7fd924138a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd92259c640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd924138660 0x7fd924138a40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd914011db0 tx=0x7fd9140046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd914022cd0 con 0x7fd924138660 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd912ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd912ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd912ffd640 10 monclient: dump: 2024-08-19T10:46:58.983 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:58.984 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7fd91402a410 con 0x7fd924138660 2024-08-19T10:46:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd912ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd912ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd924138660 msgr2=0x7fd924138a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd912ffd640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd924138660 0x7fd924138a40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd914011db0 tx=0x7fd9140046d0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd912ffd640 1 -- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd924138660 msgr2=0x7fd924138a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:58.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.972+0000 7fd912ffd640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd924138660 0x7fd924138a40 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd914011db0 tx=0x7fd9140046d0 comp rx=0 tx=0).stop 2024-08-19T10:46:58.985 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:58.985 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 14367, 2024-08-19T10:46:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:58.986 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:58.987 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:58.988 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:58.989 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:58.990 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:58.991 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.994 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.996 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:58.997 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:58.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:58.999 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.000 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.003 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:59.005 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:59.006 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.007 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.008 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:46:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.009 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:59.009 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd928c76640 1 -- 172.21.15.89:0/3082718680 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd90403be30 msgr2=0x7fd90403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd928c76640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd90403be30 0x7fd90403e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd9180097f0 tx=0x7fd9180a6000 comp rx=0 tx=0).stop 2024-08-19T10:46:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd928c76640 10 monclient: shutdown 2024-08-19T10:46:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd928c76640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:59.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd928c76640 1 -- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9241375e0 msgr2=0x7fd92411f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd928c76640 1 --2- 172.21.15.89:0/3082718680 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd9241375e0 0x7fd92411f380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00cce0 tx=0x7fd91c00a650 comp rx=0 tx=0).stop 2024-08-19T10:46:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd922d9d640 1 -- 172.21.15.89:0/3082718680 reap_dead start 2024-08-19T10:46:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd922d9d640 1 -- 172.21.15.89:0/3082718680 reap_dead start 2024-08-19T10:46:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd928c76640 1 -- 172.21.15.89:0/3082718680 shutdown_connections 2024-08-19T10:46:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd928c76640 1 -- 172.21.15.89:0/3082718680 >> 172.21.15.89:0/3082718680 conn(0x7fd9240a9130 msgr2=0x7fd92413e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd928c76640 1 -- 172.21.15.89:0/3082718680 shutdown_connections 2024-08-19T10:46:59.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:58.976+0000 7fd928c76640 1 -- 172.21.15.89:0/3082718680 wait complete. 2024-08-19T10:46:59.011 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:59.011 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:46:59.012 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:46:59.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=547593532}) 2024-08-19T10:46:59.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3488102791}) 2024-08-19T10:46:59.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=547593532}) 2024-08-19T10:46:59.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4215166654}) 2024-08-19T10:46:59.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=547593532}) 2024-08-19T10:46:59.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4249687950}) 2024-08-19T10:46:59.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=547593532}) 2024-08-19T10:46:59.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1686683461}) 2024-08-19T10:46:59.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=547593532}) 2024-08-19T10:46:59.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2541034499}) 2024-08-19T10:46:59.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=547593532}) 2024-08-19T10:46:59.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=527676347}) 2024-08-19T10:46:59.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=547593532}) 2024-08-19T10:46:59.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3021776233}) 2024-08-19T10:46:59.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=547593532}) 2024-08-19T10:46:59.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3476964334}) 2024-08-19T10:46:59.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564355746}) 2024-08-19T10:46:59.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=926360721}) 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564355746}) 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4049966652}) 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564355746}) 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3301016417}) 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564355746}) 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1280291306}) 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564355746}) 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=253523702}) 2024-08-19T10:46:59.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564355746}) 2024-08-19T10:46:59.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3271979724}) 2024-08-19T10:46:59.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564355746}) 2024-08-19T10:46:59.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3951732917}) 2024-08-19T10:46:59.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3564355746}) 2024-08-19T10:46:59.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.076+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1881182102}) 2024-08-19T10:46:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient: get_monmap_and_config 2024-08-19T10:46:59.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient: build_initial_monmap 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient: monmap: 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:59.181031+0000 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:59.181031+0000 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:59.178 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:59.179 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:59.179 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 1 Processor -- start 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 1 -- start start 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient: init 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient: _add_conns ranks=[8,3,4,0,1,2,5,9,10,6,7] 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54e40a39c0 0x7f54e40a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient(hunting): picked mon.noname-e con 0x7f54e40a39c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54e40a42e0 0x7f54e413d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient(hunting): picked mon.noname-g con 0x7f54e40a42e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e413de50 0x7f54e4140240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient(hunting): picked mon.noname-c con 0x7f54e413de50 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:59.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f54e40a02a0 con 0x7f54e40a42e0 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f54e40a4d30 con 0x7f54e413de50 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f54e40a4ab0 con 0x7f54e40a39c0 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ebc61640 10 monclient(hunting): authenticate will time out at 1468.997655s 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e99fd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54e40a39c0 0x7f54e40a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e413de50 0x7f54e4140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e413de50 0x7f54e4140240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42284/0 (socket says 172.21.15.89:42284) 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e99fd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54e40a39c0 0x7f54e40a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:36186/0 (socket says 172.21.15.89:36186) 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 1 -- 172.21.15.89:0/156149690 learned_addr learned my addr 172.21.15.89:0/156149690 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 10 monclient(hunting): get_auth_request con 0x7f54e413de50 auth_method 0 2024-08-19T10:46:59.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e99fd640 10 monclient(hunting): get_auth_request con 0x7f54e40a39c0 auth_method 0 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e99fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e99fd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e99fd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e99fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e99fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e99fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e91fc640 1 --2- 172.21.15.89:0/156149690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54e40a42e0 0x7f54e413d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e91fc640 10 monclient(hunting): get_auth_request con 0x7f54e40a42e0 auth_method 0 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e91fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e91fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e91fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 10 monclient(hunting): handle_auth_done global_id 44581 payload 274 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 1 -- 172.21.15.89:0/156149690 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54e40a39c0 msgr2=0x7f54e40a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 1 --2- 172.21.15.89:0/156149690 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54e40a39c0 0x7f54e40a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 1 -- 172.21.15.89:0/156149690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54e40a42e0 msgr2=0x7f54e413d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 1 --2- 172.21.15.89:0/156149690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54e40a42e0 0x7f54e413d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 10 monclient: _finish_hunting 0 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 1 monclient: found mon.noname-c 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:59.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:46:59.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54ea1fe640 1 -- 172.21.15.89:0/156149690 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e40a0910 con 0x7f54e413de50 2024-08-19T10:46:59.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.180+0000 7f54e99fd640 1 --2- 172.21.15.89:0/156149690 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54e40a39c0 0x7f54e40a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:59.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ea1fe640 1 -- 172.21.15.89:0/156149690 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e413de50 msgr2=0x7f54e4140240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ea1fe640 1 --2- 172.21.15.89:0/156149690 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e413de50 0x7f54e4140240 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54d40098a0 tx=0x7f54d401c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c7cec1188e98005 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 1 -- 172.21.15.89:0/156149690 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54d40b0070 con 0x7f54e413de50 2024-08-19T10:46:59.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 10 monclient: dump: 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.190 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 10 monclient: _finish_auth 0 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 10 monclient: _check_auth_tickets 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 1 -- 172.21.15.89:0/156149690 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d40aa810 con 0x7f54e413de50 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 1 -- 172.21.15.89:0/156149690 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54d40a9300 con 0x7f54e413de50 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 5 monclient: authenticate success, global_id 44581 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:59.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 4 set_mon_vals no callback set 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54e89fb640 10 monclient: dump: 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.192 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 10 monclient: shutdown 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 1 -- 172.21.15.89:0/156149690 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e413de50 msgr2=0x7f54e4140240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 1 --2- 172.21.15.89:0/156149690 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e413de50 0x7f54e4140240 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54d40098a0 tx=0x7f54d401c440 comp rx=0 tx=0).stop 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 1 -- 172.21.15.89:0/156149690 shutdown_connections 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 1 --2- 172.21.15.89:0/156149690 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e413de50 0x7f54e4140240 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 1 --2- 172.21.15.89:0/156149690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54e40a42e0 0x7f54e413d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 1 --2- 172.21.15.89:0/156149690 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f54e40a39c0 0x7f54e40a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 1 -- 172.21.15.89:0/156149690 >> 172.21.15.89:0/156149690 conn(0x7f54e40a8f60 msgr2=0x7f54e40a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 1 -- 172.21.15.89:0/156149690 shutdown_connections 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 1 -- 172.21.15.89:0/156149690 wait complete. 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 10 monclient: build_initial_monmap 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.184+0000 7f54ebc61640 10 monclient: monmap: 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:59.194 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:59.195 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:59.195 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:59.195 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:59.195 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:59.195 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:59.195 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:59.195 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:59.195 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 1 Processor -- start 2024-08-19T10:46:59.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 1 -- start start 2024-08-19T10:46:59.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient: init 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient: _add_conns ranks=[0,1,3,5,8,6,2,4,7] 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e40a39c0 0x7f54e409e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient(hunting): picked mon.noname-a con 0x7f54e40a39c0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e40a42e0 0x7f54e41d1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient(hunting): picked mon.noname-f con 0x7f54e40a42e0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54e413de50 0x7f54e4123790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient(hunting): picked mon.noname-g con 0x7f54e413de50 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40a46c0 con 0x7f54e40a39c0 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40743a0 con 0x7f54e40a42e0 2024-08-19T10:46:59.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f54e41c7cb0 con 0x7f54e413de50 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient(hunting): authenticate will time out at 1469.004814s 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e40a39c0 0x7f54e409e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e40a39c0 0x7f54e409e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50736/0 (socket says 172.21.15.89:50736) 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 1 -- 172.21.15.89:0/1085177836 learned_addr learned my addr 172.21.15.89:0/1085177836 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 10 monclient(hunting): get_auth_request con 0x7f54e40a39c0 auth_method 0 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e91fc640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e40a42e0 0x7f54e41d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 10 monclient(hunting): handle_auth_done global_id 6714 payload 824 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 1 -- 172.21.15.89:0/1085177836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54e413de50 msgr2=0x7f54e4123790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54e413de50 0x7f54e4123790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 1 -- 172.21.15.89:0/1085177836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e40a42e0 msgr2=0x7f54e41d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e40a42e0 0x7f54e41d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 10 monclient: _finish_hunting 0 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 1 monclient: found mon.noname-a 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 1 -- 172.21.15.89:0/1085177836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e4125cc0 con 0x7f54e40a39c0 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e91fc640 10 monclient: get_auth_request con 0x7f54e40a42e0 auth_method 0 2024-08-19T10:46:59.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e91fc640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54e40a42e0 0x7f54e41d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 1 -- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e40a39c0 msgr2=0x7f54e409e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e99fd640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e40a39c0 0x7f54e409e4f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00eb60 tx=0x7f54dc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54dc017070 con 0x7f54e40a39c0 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 10 monclient: dump: 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.199 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 10 monclient: _finish_auth 0 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54dc00ce40 con 0x7f54e40a39c0 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:59.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54dc013d50 con 0x7f54e40a39c0 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 5 monclient: authenticate success, global_id 6714 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient: _renew_subs 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 1 -- 172.21.15.89:0/1085177836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e4125fa0 con 0x7f54e40a39c0 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e37fe640 4 set_mon_vals no callback set 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54e27fc640 10 monclient: dump: 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.201 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient: _renew_subs 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:46:59.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.188+0000 7f54ebc61640 1 -- 172.21.15.89:0/1085177836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e41259e0 con 0x7f54e40a39c0 2024-08-19T10:46:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54ebc61640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54ebc61640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e41d1840 0x7f54e4123cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54ebc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54ebc61640 1 -- 172.21.15.89:0/1085177836 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f54e40a4840 con 0x7f54e41d1840 2024-08-19T10:46:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e41d1840 0x7f54e4123cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 10 monclient: get_auth_request con 0x7f54e41d1840 auth_method 0 2024-08-19T10:46:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f54dc01a070 con 0x7f54e40a39c0 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 10 monclient(hunting): handle_auth_done global_id 44590 payload 824 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 1 -- 172.21.15.89:0/1085177836 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54e413fbd0 con 0x7f54e41d1840 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f54d003be60 0x7f54d003e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f54dc00f040 con 0x7f54e40a39c0 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54ea1fe640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f54d003be60 0x7f54d003e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54ea1fe640 10 monclient: get_auth_request con 0x7f54d003be60 auth_method 0 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 1 -- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e41d1840 msgr2=0x7f54e4123cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e91fc640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e41d1840 0x7f54e4123cd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f54e40a9df0 tx=0x7f54d800f520 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54d8018020 con 0x7f54e41d1840 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 10 monclient: dump: 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.208 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f54d8014600 con 0x7f54e41d1840 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e41d1840 msgr2=0x7f54e4123cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e41d1840 0x7f54e4123cd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f54e40a9df0 tx=0x7f54d800f520 comp rx=0 tx=0).stop 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e41d1840 msgr2=0x7f54e4123cd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.192+0000 7f54e27fc640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e41d1840 0x7f54e4123cd0 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f54e40a9df0 tx=0x7f54d800f520 comp rx=0 tx=0).stop 2024-08-19T10:46:59.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.196+0000 7f54ea1fe640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f54d003be60 0x7f54d003e320 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009870 tx=0x7f54d4004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54ebc61640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54ebc61640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e4124210 0x7f54e4070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54ebc61640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54ebc61640 1 -- 172.21.15.89:0/1085177836 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f54e4124620 con 0x7f54e4124210 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54e91fc640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e4124210 0x7f54e4070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54e91fc640 10 monclient: get_auth_request con 0x7f54e4124210 auth_method 0 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54e91fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54e91fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54e91fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54e91fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54e91fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54e91fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54e91fc640 10 monclient(hunting): handle_auth_done global_id 44599 payload 824 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.200+0000 7f54e91fc640 1 -- 172.21.15.89:0/1085177836 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f54e40a3da0 con 0x7f54e4124210 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e91fc640 1 -- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e4124210 msgr2=0x7f54e4070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e91fc640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e4124210 0x7f54e4070e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f54e40a9df0 tx=0x7f54d8004a50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f54d8018040 con 0x7f54e4124210 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e27fc640 10 monclient: dump: 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.211 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f54d8014cc0 con 0x7f54e4124210 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e27fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e4124210 msgr2=0x7f54e4070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e27fc640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e4124210 0x7f54e4070e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f54e40a9df0 tx=0x7f54d8004a50 comp rx=0 tx=0).stop 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e27fc640 1 -- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e4124210 msgr2=0x7f54e4070e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:59.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54e27fc640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f54e4124210 0x7f54e4070e10 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f54e40a9df0 tx=0x7f54d8004a50 comp rx=0 tx=0).stop 2024-08-19T10:46:59.213 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:59.213 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:59.213 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:59.213 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:59.213 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:59.213 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 142330, 2024-08-19T10:46:59.213 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:59.213 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:59.213 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:59.214 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.215 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.216 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:59.217 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.218 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.222 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:59.224 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:59.227 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:59.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:59.231 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:59.233 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:59.233 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.233 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.233 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.233 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:59.233 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:59.233 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.234 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:59.235 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:59.236 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:59.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ebc61640 1 -- 172.21.15.89:0/1085177836 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f54d003be60 msgr2=0x7f54d003e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ebc61640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f54d003be60 0x7f54d003e320 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009870 tx=0x7f54d4004740 comp rx=0 tx=0).stop 2024-08-19T10:46:59.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ebc61640 10 monclient: shutdown 2024-08-19T10:46:59.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ebc61640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:59.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ebc61640 1 -- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e40a39c0 msgr2=0x7f54e409e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ebc61640 1 --2- 172.21.15.89:0/1085177836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e40a39c0 0x7f54e409e4f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00eb60 tx=0x7f54dc00c520 comp rx=0 tx=0).stop 2024-08-19T10:46:59.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ea1fe640 1 -- 172.21.15.89:0/1085177836 reap_dead start 2024-08-19T10:46:59.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ea1fe640 1 -- 172.21.15.89:0/1085177836 reap_dead start 2024-08-19T10:46:59.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ebc61640 1 -- 172.21.15.89:0/1085177836 shutdown_connections 2024-08-19T10:46:59.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ebc61640 1 -- 172.21.15.89:0/1085177836 >> 172.21.15.89:0/1085177836 conn(0x7f54e40a8f60 msgr2=0x7f54e40a7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:59.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ebc61640 1 -- 172.21.15.89:0/1085177836 shutdown_connections 2024-08-19T10:46:59.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.204+0000 7f54ebc61640 1 -- 172.21.15.89:0/1085177836 wait complete. 2024-08-19T10:46:59.239 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:59.239 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:46:59.239 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:46:59.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient: get_monmap_and_config 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient: build_initial_monmap 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient: monmap: 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:59.405054+0000 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:59.405054+0000 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:59.402 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:59.403 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:59.403 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:59.403 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:59.403 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:59.403 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:59.403 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:59.403 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:59.403 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 1 Processor -- start 2024-08-19T10:46:59.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 1 -- start start 2024-08-19T10:46:59.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient: init 2024-08-19T10:46:59.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient: _add_conns ranks=[3,0,5,7,10,9,6,1,4,8,2] 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f404406c930 0x7f40440a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient(hunting): picked mon.noname-g con 0x7f404406c930 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f40440a3f00 addr v1:172.21.15.89:6789/0 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f40440a4340 0x7f4044145810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient(hunting): picked mon.noname-h con 0x7f40440a4340 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f404409fb10 con 0x7f40440a3f00 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f404406cd10 con 0x7f404406c930 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f40440a11b0 con 0x7f40440a4340 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:59.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4049c33640 10 monclient(hunting): authenticate will time out at 1469.221597s 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4042ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f40440a3f00 0x7f40441420e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44834/0 (socket says 172.21.15.89:44834) 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4042ffd640 1 -- 172.21.15.89:0/1038532137 learned_addr learned my addr 172.21.15.89:0/1038532137 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 1 --2- 172.21.15.89:0/1038532137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f404406c930 0x7f40440a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4043fff640 1 --2- 172.21.15.89:0/1038532137 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f40440a4340 0x7f4044145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 10 monclient(hunting): get_auth_request con 0x7f404406c930 auth_method 0 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4043fff640 10 monclient(hunting): get_auth_request con 0x7f40440a4340 auth_method 0 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4043fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4043fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4043fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4043fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4043fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4043fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 10 monclient(hunting): handle_auth_done global_id 35193 payload 274 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 1 -- 172.21.15.89:0/1038532137 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f40440a4340 msgr2=0x7f4044145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 1 --2- 172.21.15.89:0/1038532137 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f40440a4340 0x7f4044145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 1 -- 172.21.15.89:0/1038532137 >> v1:172.21.15.89:6789/0 conn(0x7f40440a3f00 legacy=0x7f40441420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 10 monclient: _finish_hunting 0 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 1 monclient: found mon.noname-g 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:59.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:46:59.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f40437fe640 1 -- 172.21.15.89:0/1038532137 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40440a4d30 con 0x7f404406c930 2024-08-19T10:46:59.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.404+0000 7f4043fff640 1 --2- 172.21.15.89:0/1038532137 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f40440a4340 0x7f4044145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:59.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40437fe640 1 -- 172.21.15.89:0/1038532137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f404406c930 msgr2=0x7f40440a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40437fe640 1 --2- 172.21.15.89:0/1038532137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f404406c930 0x7f40440a39c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4034009df0 tx=0x7f403401c850 comp rx=0 tx=0).ready entity=mon.3 client_cookie=9cf1d3379cff22b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 1 -- 172.21.15.89:0/1038532137 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40340ac070 con 0x7f404406c930 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 10 monclient: dump: 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.411 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 10 monclient: _finish_auth 0 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 10 monclient: _check_auth_tickets 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 1 -- 172.21.15.89:0/1038532137 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40340a68b0 con 0x7f404406c930 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f4049c33640 5 monclient: authenticate success, global_id 35193 2024-08-19T10:46:59.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f4049c33640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f4049c33640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f4049c33640 4 set_mon_vals no callback set 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 1 -- 172.21.15.89:0/1038532137 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40340a7420 con 0x7f404406c930 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.408+0000 7f40427fc640 10 monclient: dump: 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.413 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient: shutdown 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 -- 172.21.15.89:0/1038532137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f404406c930 msgr2=0x7f40440a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 --2- 172.21.15.89:0/1038532137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f404406c930 0x7f40440a39c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4034009df0 tx=0x7f403401c850 comp rx=0 tx=0).stop 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 -- 172.21.15.89:0/1038532137 shutdown_connections 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 --2- 172.21.15.89:0/1038532137 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f40440a4340 0x7f4044145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 --2- 172.21.15.89:0/1038532137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f404406c930 0x7f40440a39c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 -- 172.21.15.89:0/1038532137 >> 172.21.15.89:0/1038532137 conn(0x7f40440a8f60 msgr2=0x7f40440a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 -- 172.21.15.89:0/1038532137 shutdown_connections 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 -- 172.21.15.89:0/1038532137 wait complete. 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient: build_initial_monmap 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient: monmap: 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:59.415 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 Processor -- start 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 -- start start 2024-08-19T10:46:59.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient: init 2024-08-19T10:46:59.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:59.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient: _add_conns ranks=[4,6,3,2,0,7,1,5,8] 2024-08-19T10:46:59.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f404406c930 0x7f40441d5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient(hunting): picked mon.noname-c con 0x7f404406c930 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:59.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40440a3f00 0x7f40441d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient(hunting): picked mon.noname-d con 0x7f40440a3f00 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40440a4340 0x7f4044142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient(hunting): picked mon.noname-g con 0x7f40440a4340 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f40440a0420 con 0x7f40440a4340 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f40440a02a0 con 0x7f404406c930 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4044148570 con 0x7f40440a3f00 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4049c33640 10 monclient(hunting): authenticate will time out at 1469.229832s 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f40437fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f404406c930 0x7f40441d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40440a3f00 0x7f40441d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f40437fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f404406c930 0x7f40441d5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42320/0 (socket says 172.21.15.89:42320) 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f40437fe640 1 -- 172.21.15.89:0/3026463479 learned_addr learned my addr 172.21.15.89:0/3026463479 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f40437fe640 10 monclient(hunting): get_auth_request con 0x7f404406c930 auth_method 0 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f40437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f40437fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f40437fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 10 monclient(hunting): get_auth_request con 0x7f40440a3f00 auth_method 0 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4043fff640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40440a4340 0x7f4044142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 10 monclient(hunting): handle_auth_done global_id 44601 payload 824 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 1 -- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f404406c930 msgr2=0x7f40441d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f404406c930 0x7f40441d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 1 -- 172.21.15.89:0/3026463479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40440a4340 msgr2=0x7f4044142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40440a4340 0x7f4044142980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 1 monclient: found mon.noname-d 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f4042ffd640 1 -- 172.21.15.89:0/3026463479 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404411fa70 con 0x7f40440a3f00 2024-08-19T10:46:59.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.412+0000 7f40437fe640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f404406c930 0x7f40441d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4042ffd640 1 -- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40440a3f00 msgr2=0x7f40441d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4042ffd640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40440a3f00 0x7f40441d5c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f403800c3e0 tx=0x7f403800c8a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4038017070 con 0x7f40440a3f00 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 10 monclient: dump: 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.425 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 10 monclient: _finish_auth 0 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4049c33640 5 monclient: authenticate success, global_id 44601 2024-08-19T10:46:59.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4038005d00 con 0x7f40440a3f00 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4049c33640 10 monclient: _renew_subs 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4049c33640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4049c33640 1 -- 172.21.15.89:0/3026463479 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404409f590 con 0x7f40440a3f00 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4038004a80 con 0x7f40440a3f00 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4041ffb640 4 set_mon_vals no callback set 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 10 monclient: dump: 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.427 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4049c33640 10 monclient: _renew_subs 2024-08-19T10:46:59.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4049c33640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4049c33640 1 -- 172.21.15.89:0/3026463479 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f404411f790 con 0x7f40440a3f00 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4049c33640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4049c33640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4044142ec0 0x7f40441432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4049c33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4049c33640 1 -- 172.21.15.89:0/3026463479 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4044144ed0 con 0x7f4044142ec0 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f40437fe640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4044142ec0 0x7f40441432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f40437fe640 10 monclient: get_auth_request con 0x7f4044142ec0 auth_method 0 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f40437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f40437fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f40437fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4042ffd640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40440a3f00 0x7f40441d5c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f403800c3e0 tx=0x7f403800c8a0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f4038007070 mgrmap(e 5) 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4038007070 con 0x7f40440a3f00 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f40437fe640 10 monclient(hunting): handle_auth_done global_id 44610 payload 824 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f40437fe640 1 -- 172.21.15.89:0/3026463479 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40440a39c0 con 0x7f4044142ec0 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f402bfff640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f403003bd30 0x7f403003e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4043fff640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f403003bd30 0x7f403003e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.416+0000 7f4043fff640 10 monclient: get_auth_request con 0x7f403003bd30 auth_method 0 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f40437fe640 1 -- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4044142ec0 msgr2=0x7f40441432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f40437fe640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4044142ec0 0x7f40441432a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4034000f90 tx=0x7f40340195f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40340ac050 con 0x7f4044142ec0 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f402bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f402bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:46:59.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f402bfff640 10 monclient: dump: 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.431 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.432 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.432 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4034004540 con 0x7f4044142ec0 2024-08-19T10:46:59.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f402bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:59.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f402bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:59.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4044142ec0 msgr2=0x7f40441432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f402bfff640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4044142ec0 0x7f40441432a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4034000f90 tx=0x7f40340195f0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4044142ec0 msgr2=0x7f40441432a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:59.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f402bfff640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4044142ec0 0x7f40441432a0 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4034000f90 tx=0x7f40340195f0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.420+0000 7f4043fff640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f403003bd30 0x7f403003e1f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4044143fc0 tx=0x7f403c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f4049c33640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:59.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f4049c33640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40441437e0 0x7f404409da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f4049c33640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f4049c33640 1 -- 172.21.15.89:0/3026463479 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4044064590 con 0x7f40441437e0 2024-08-19T10:46:59.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f40437fe640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40441437e0 0x7f404409da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f40437fe640 10 monclient: get_auth_request con 0x7f40441437e0 auth_method 0 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f40437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f40437fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f40437fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f40437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f40437fe640 10 monclient(hunting): handle_auth_done global_id 44619 payload 824 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.424+0000 7f40437fe640 1 -- 172.21.15.89:0/3026463479 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4044144a50 con 0x7f40441437e0 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f40437fe640 1 -- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40441437e0 msgr2=0x7f404409da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f40437fe640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40441437e0 0x7f404409da60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4044076d00 tx=0x7f40340a9000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40340b1ad0 con 0x7f40441437e0 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 10 monclient: dump: 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.436 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.437 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f40340b1660 con 0x7f40441437e0 2024-08-19T10:46:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40441437e0 msgr2=0x7f404409da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40441437e0 0x7f404409da60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4044076d00 tx=0x7f40340a9000 comp rx=0 tx=0).stop 2024-08-19T10:46:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40441437e0 msgr2=0x7f404409da60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:59.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40441437e0 0x7f404409da60 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4044076d00 tx=0x7f40340a9000 comp rx=0 tx=0).stop 2024-08-19T10:46:59.437 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 142554, 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.439 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.440 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:59.441 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.442 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:59.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:59.444 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.446 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:59.447 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:59.450 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.453 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:59.455 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.457 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.458 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:59.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.460 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:59.460 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:59.460 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:46:59.460 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.460 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.460 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:59.460 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:59.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4049c33640 1 -- 172.21.15.89:0/3026463479 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f403003bd30 msgr2=0x7f403003e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4049c33640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f403003bd30 0x7f403003e1f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4044143fc0 tx=0x7f403c008040 comp rx=0 tx=0).stop 2024-08-19T10:46:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4049c33640 10 monclient: shutdown 2024-08-19T10:46:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4049c33640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4049c33640 1 -- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40440a3f00 msgr2=0x7f40441d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4049c33640 1 --2- 172.21.15.89:0/3026463479 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40440a3f00 0x7f40441d5c90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f403800c3e0 tx=0x7f403800c8a0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4043fff640 1 -- 172.21.15.89:0/3026463479 reap_dead start 2024-08-19T10:46:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f402bfff640 1 -- 172.21.15.89:0/3026463479 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (unknown 0 0 0) 0x7f403804e8d0 con 0x7f40440a3f00 2024-08-19T10:46:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4043fff640 1 -- 172.21.15.89:0/3026463479 reap_dead start 2024-08-19T10:46:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4049c33640 1 -- 172.21.15.89:0/3026463479 shutdown_connections 2024-08-19T10:46:59.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4049c33640 1 -- 172.21.15.89:0/3026463479 >> 172.21.15.89:0/3026463479 conn(0x7f40440a8f60 msgr2=0x7f4044141640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:59.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4049c33640 1 -- 172.21.15.89:0/3026463479 shutdown_connections 2024-08-19T10:46:59.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.428+0000 7f4049c33640 1 -- 172.21.15.89:0/3026463479 wait complete. 2024-08-19T10:46:59.463 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:59.463 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:46:59.463 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:46:59.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient: get_monmap_and_config 2024-08-19T10:46:59.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient: build_initial_monmap 2024-08-19T10:46:59.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient: monmap: 2024-08-19T10:46:59.578 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:59.578 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.578 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:59.581411+0000 2024-08-19T10:46:59.578 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:59.581411+0000 2024-08-19T10:46:59.578 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:59.578 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:59.579 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 1 Processor -- start 2024-08-19T10:46:59.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 1 -- start start 2024-08-19T10:46:59.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient: init 2024-08-19T10:46:59.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:59.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient: _add_conns ranks=[7,5,0,6,10,1,2,3,4,8,9] 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b400a5270 0x7f8b400a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient(hunting): picked mon.noname-i con 0x7f8b400a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8b400a6d30 0x7f8b400a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient(hunting): picked mon.noname-h con 0x7f8b400a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8b401395b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b400a8710 con 0x7f8b401395b0 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f8b400a7650 con 0x7f8b400a6d30 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f8b400a5a40 con 0x7f8b400a5270 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b45088640 10 monclient(hunting): authenticate will time out at 1469.397994s 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3f59e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8b401395b0 0x7f8b40139a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44850/0 (socket says 172.21.15.89:44850) 2024-08-19T10:46:59.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3f59e640 1 -- 172.21.15.89:0/1976630416 learned_addr learned my addr 172.21.15.89:0/1976630416 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 1 --2- 172.21.15.89:0/1976630416 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b400a5270 0x7f8b400a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 10 monclient(hunting): get_auth_request con 0x7f8b400a5270 auth_method 0 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3e59c640 1 --2- 172.21.15.89:0/1976630416 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8b400a6d30 0x7f8b400a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3e59c640 10 monclient(hunting): get_auth_request con 0x7f8b400a6d30 auth_method 0 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_done global_id 16018 payload 274 2024-08-19T10:46:59.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 1 -- 172.21.15.89:0/1976630416 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8b400a6d30 msgr2=0x7f8b400a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:59.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 1 --2- 172.21.15.89:0/1976630416 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8b400a6d30 0x7f8b400a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 1 -- 172.21.15.89:0/1976630416 >> v1:172.21.15.89:6789/0 conn(0x7f8b401395b0 legacy=0x7f8b40139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:59.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:46:59.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 1 monclient: found mon.noname-i 2024-08-19T10:46:59.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:59.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:46:59.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.580+0000 7f8b3ed9d640 1 -- 172.21.15.89:0/1976630416 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b400a81d0 con 0x7f8b400a5270 2024-08-19T10:46:59.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3ed9d640 1 -- 172.21.15.89:0/1976630416 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b400a5270 msgr2=0x7f8b400a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3ed9d640 1 --2- 172.21.15.89:0/1976630416 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b400a5270 0x7f8b400a67f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8b340097f0 tx=0x7f8b3401c1c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=9f1c76658ed2da3d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 1 -- 172.21.15.89:0/1976630416 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b340b0070 con 0x7f8b400a5270 2024-08-19T10:46:59.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:46:59.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 10 monclient: dump: 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.590 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 10 monclient: _finish_auth 0 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 10 monclient: _check_auth_tickets 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 1 -- 172.21.15.89:0/1976630416 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b340aa810 con 0x7f8b400a5270 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b45088640 5 monclient: authenticate success, global_id 16018 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b45088640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b45088640 4 set_mon_vals no callback set 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 1 -- 172.21.15.89:0/1976630416 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b340aaa00 con 0x7f8b400a5270 2024-08-19T10:46:59.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.584+0000 7f8b3dd9b640 10 monclient: dump: 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.592 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient: shutdown 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 -- 172.21.15.89:0/1976630416 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b400a5270 msgr2=0x7f8b400a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 --2- 172.21.15.89:0/1976630416 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b400a5270 0x7f8b400a67f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8b340097f0 tx=0x7f8b3401c1c0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 -- 172.21.15.89:0/1976630416 shutdown_connections 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 --2- 172.21.15.89:0/1976630416 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f8b400a6d30 0x7f8b400a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 --2- 172.21.15.89:0/1976630416 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f8b400a5270 0x7f8b400a67f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 -- 172.21.15.89:0/1976630416 >> 172.21.15.89:0/1976630416 conn(0x7f8b40069a80 msgr2=0x7f8b400abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 -- 172.21.15.89:0/1976630416 shutdown_connections 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 -- 172.21.15.89:0/1976630416 wait complete. 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient: build_initial_monmap 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient: monmap: 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:59.594 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 Processor -- start 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 -- start start 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient: init 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient: _add_conns ranks=[0,1,6,5,7,8,2,3,4] 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b400a5270 0x7f8b4013fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient(hunting): picked mon.noname-a con 0x7f8b400a5270 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b400a6d30 0x7f8b401d1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient(hunting): picked mon.noname-f con 0x7f8b400a6d30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b401395b0 0x7f8b4013d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient(hunting): picked mon.noname-d con 0x7f8b401395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8b400a57d0 con 0x7f8b400a5270 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8b400a5650 con 0x7f8b400a6d30 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f8b40144240 con 0x7f8b401395b0 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b45088640 10 monclient(hunting): authenticate will time out at 1469.406617s 2024-08-19T10:46:59.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b400a5270 0x7f8b4013fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b401395b0 0x7f8b4013d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3ed9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b400a5270 0x7f8b4013fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50750/0 (socket says 172.21.15.89:50750) 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3ed9d640 1 -- 172.21.15.89:0/17400482 learned_addr learned my addr 172.21.15.89:0/17400482 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3ed9d640 10 monclient(hunting): get_auth_request con 0x7f8b400a5270 auth_method 0 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 10 monclient(hunting): get_auth_request con 0x7f8b401395b0 auth_method 0 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 10 monclient(hunting): handle_auth_done global_id 44628 payload 824 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 1 -- 172.21.15.89:0/17400482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b400a6d30 msgr2=0x7f8b401d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b400a6d30 0x7f8b401d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 1 -- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b400a5270 msgr2=0x7f8b4013fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b400a5270 0x7f8b4013fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 10 monclient: _finish_hunting 0 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 1 monclient: found mon.noname-d 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:59.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:46:59.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3f59e640 1 -- 172.21.15.89:0/17400482 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b40122da0 con 0x7f8b401395b0 2024-08-19T10:46:59.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.588+0000 7f8b3ed9d640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b400a5270 0x7f8b4013fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:59.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3f59e640 1 -- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b401395b0 msgr2=0x7f8b4013d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3f59e640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b401395b0 0x7f8b4013d5b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800cb20 tx=0x7f8b3800cf20 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b38017070 con 0x7f8b401395b0 2024-08-19T10:46:59.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:46:59.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 10 monclient: dump: 2024-08-19T10:46:59.602 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.602 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.603 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 10 monclient: _finish_auth 0 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 10 monclient: _check_auth_tickets 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3800abe0 con 0x7f8b401395b0 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b45088640 5 monclient: authenticate success, global_id 44628 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b45088640 10 monclient: _renew_subs 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b45088640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b45088640 1 -- 172.21.15.89:0/17400482 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b400aad70 con 0x7f8b401395b0 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b3800ad90 con 0x7f8b401395b0 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:46:59.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 10 monclient: dump: 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.605 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3d59a640 4 set_mon_vals no callback set 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b45088640 10 monclient: _renew_subs 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b45088640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b45088640 1 -- 172.21.15.89:0/17400482 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b40122ac0 con 0x7f8b401395b0 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b45088640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b45088640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b401d1d60 0x7f8b4002d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b45088640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b45088640 1 -- 172.21.15.89:0/17400482 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f8b40122740 con 0x7f8b401d1d60 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3ed9d640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b401d1d60 0x7f8b4002d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3ed9d640 10 monclient: get_auth_request con 0x7f8b401d1d60 auth_method 0 2024-08-19T10:46:59.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3f59e640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b401395b0 0x7f8b4013d5b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800cb20 tx=0x7f8b3800cf20 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f8b3804f210 osd_map(271..271 src has 150..271) 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8b38007070 con 0x7f8b401395b0 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_done global_id 6551 payload 824 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3ed9d640 1 -- 172.21.15.89:0/17400482 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b400a67f0 con 0x7f8b401d1d60 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b277fe640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b2c03be00 0x7f8b2c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3e59c640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b2c03be00 0x7f8b2c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.592+0000 7f8b3e59c640 10 monclient: get_auth_request con 0x7f8b2c03be00 auth_method 0 2024-08-19T10:46:59.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b3ed9d640 1 -- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b401d1d60 msgr2=0x7f8b4002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b3ed9d640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b401d1d60 0x7f8b4002d1d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009710 tx=0x7f8b34018f60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b340b0040 con 0x7f8b401d1d60 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b277fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b277fe640 10 monclient: dump: 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.608 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8b340aab70 con 0x7f8b401d1d60 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b277fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b277fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b401d1d60 msgr2=0x7f8b4002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b277fe640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b401d1d60 0x7f8b4002d1d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009710 tx=0x7f8b34018f60 comp rx=0 tx=0).stop 2024-08-19T10:46:59.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b401d1d60 msgr2=0x7f8b4002d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:59.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b277fe640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b401d1d60 0x7f8b4002d1d0 secure :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009710 tx=0x7f8b34018f60 comp rx=0 tx=0).stop 2024-08-19T10:46:59.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.596+0000 7f8b3e59c640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b2c03be00 0x7f8b2c03e2c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8b30006aa0 tx=0x7f8b30006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761558978}) 2024-08-19T10:46:59.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=719071611}) 2024-08-19T10:46:59.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761558978}) 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1947753120}) 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761558978}) 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2479896487}) 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761558978}) 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1031143823}) 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761558978}) 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2153159819}) 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761558978}) 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1796034821}) 2024-08-19T10:46:59.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761558978}) 2024-08-19T10:46:59.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1776828711}) 2024-08-19T10:46:59.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761558978}) 2024-08-19T10:46:59.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.600+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3925792556}) 2024-08-19T10:46:59.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b45088640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:59.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b45088640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b4002d710 0x7f8b4002db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b45088640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b45088640 1 -- 172.21.15.89:0/17400482 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f8b40064590 con 0x7f8b4002d710 2024-08-19T10:46:59.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b3ed9d640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b4002d710 0x7f8b4002db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b3ed9d640 10 monclient: get_auth_request con 0x7f8b4002d710 auth_method 0 2024-08-19T10:46:59.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b3ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b3ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b3ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b3ed9d640 10 monclient(hunting): handle_auth_done global_id 6560 payload 824 2024-08-19T10:46:59.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f8b3ed9d640 1 -- 172.21.15.89:0/17400482 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8b400a07c0 con 0x7f8b4002d710 2024-08-19T10:46:59.618 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.618 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1222546568}) 2024-08-19T10:46:59.618 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017457607}) 2024-08-19T10:46:59.618 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.618 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1222546568}) 2024-08-19T10:46:59.618 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=53660307}) 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1222546568}) 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3763245100}) 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1222546568}) 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3705109324}) 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1222546568}) 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2759325912}) 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1222546568}) 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3368083060}) 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.619 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1222546568}) 2024-08-19T10:46:59.620 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1902604465}) 2024-08-19T10:46:59.620 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.620 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1222546568}) 2024-08-19T10:46:59.620 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.604+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4132652021}) 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b3ed9d640 1 -- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b4002d710 msgr2=0x7f8b4002db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b3ed9d640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b4002d710 0x7f8b4002db20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b34000c10 tx=0x7f8b340a8000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8b340b5ad0 con 0x7f8b4002d710 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 10 monclient: dump: 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.621 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f8b340b5c70 con 0x7f8b4002d710 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:59.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b4002d710 msgr2=0x7f8b4002db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b4002d710 0x7f8b4002db20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b34000c10 tx=0x7f8b340a8000 comp rx=0 tx=0).stop 2024-08-19T10:46:59.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b4002d710 msgr2=0x7f8b4002db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:59.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8b4002d710 0x7f8b4002db20 secure :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b34000c10 tx=0x7f8b340a8000 comp rx=0 tx=0).stop 2024-08-19T10:46:59.623 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:59.623 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 549401, 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:46:59.624 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.625 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.626 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:59.626 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:59.626 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.626 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.626 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.626 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.626 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.626 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.626 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.626 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.626 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:59.627 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.628 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:59.629 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.630 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.630 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:59.630 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.630 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:59.630 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:59.631 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.633 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:59.634 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:59.635 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.636 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.637 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.638 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.639 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.640 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:59.642 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.643 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.644 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.645 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:46:59.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.647 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.647 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:59.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.647 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:59.647 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:59.647 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:59.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.647 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.647 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:59.647 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:59.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b45088640 1 -- 172.21.15.89:0/17400482 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b2c03be00 msgr2=0x7f8b2c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b45088640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f8b2c03be00 0x7f8b2c03e2c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8b30006aa0 tx=0x7f8b30006a30 comp rx=0 tx=0).stop 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b45088640 10 monclient: shutdown 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b45088640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b45088640 1 -- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b401395b0 msgr2=0x7f8b4013d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b45088640 1 --2- 172.21.15.89:0/17400482 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f8b401395b0 0x7f8b4013d5b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800cb20 tx=0x7f8b3800cf20 comp rx=0 tx=0).stop 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b3f59e640 1 -- 172.21.15.89:0/17400482 reap_dead start 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b3f59e640 1 -- 172.21.15.89:0/17400482 reap_dead start 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b277fe640 1 -- 172.21.15.89:0/17400482 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (unknown 0 0 0) 0x7f8b3804f210 con 0x7f8b401395b0 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b45088640 1 -- 172.21.15.89:0/17400482 shutdown_connections 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b45088640 1 -- 172.21.15.89:0/17400482 >> 172.21.15.89:0/17400482 conn(0x7f8b40069a80 msgr2=0x7f8b400ab180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b45088640 1 -- 172.21.15.89:0/17400482 shutdown_connections 2024-08-19T10:46:59.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.608+0000 7f8b45088640 1 -- 172.21.15.89:0/17400482 wait complete. 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279987507}) 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2142097379}) 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279987507}) 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2320864828}) 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279987507}) 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3151442611}) 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279987507}) 2024-08-19T10:46:59.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=955259311}) 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279987507}) 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1442723287}) 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279987507}) 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2628037029}) 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279987507}) 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3167100990}) 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=279987507}) 2024-08-19T10:46:59.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:46:59.624+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3047516386}) 2024-08-19T10:46:59.653 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:59.653 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:46:59.653 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:46:59.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient: get_monmap_and_config 2024-08-19T10:46:59.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient: build_initial_monmap 2024-08-19T10:46:59.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient: monmap: 2024-08-19T10:46:59.770 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:59.770 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.770 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:59.773141+0000 2024-08-19T10:46:59.771 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:59.773141+0000 2024-08-19T10:46:59.771 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:59.771 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.771 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:59.771 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:59.771 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:59.772 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:59.772 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 1 Processor -- start 2024-08-19T10:46:59.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 1 -- start start 2024-08-19T10:46:59.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient: init 2024-08-19T10:46:59.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:59.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient: _add_conns ranks=[5,10,4,9,3,1,7,0,6,8,2] 2024-08-19T10:46:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66e80a3c30 0x7f66e80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient(hunting): picked mon.noname-h con 0x7f66e80a3c30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f66e80a02a0 0x7f66e80a4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient(hunting): picked mon.noname-f con 0x7f66e80a02a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:46:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66e813d910 0x7f66e813fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient(hunting): picked mon.noname-c con 0x7f66e813d910 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:46:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f66e80a4a90 con 0x7f66e80a02a0 2024-08-19T10:46:59.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f66e80a0a60 con 0x7f66e813d910 2024-08-19T10:46:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f66e80a0d40 con 0x7f66e80a3c30 2024-08-19T10:46:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:59.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66ecbfa640 10 monclient(hunting): authenticate will time out at 1469.589860s 2024-08-19T10:46:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66e813d910 0x7f66e813fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66e813d910 0x7f66e813fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42328/0 (socket says 172.21.15.89:42328) 2024-08-19T10:46:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 1 -- 172.21.15.89:0/3495575759 learned_addr learned my addr 172.21.15.89:0/3495575759 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 10 monclient(hunting): get_auth_request con 0x7f66e813d910 auth_method 0 2024-08-19T10:46:59.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 10 monclient(hunting): handle_auth_done global_id 44617 payload 274 2024-08-19T10:46:59.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 1 -- 172.21.15.89:0/3495575759 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66e80a3c30 msgr2=0x7f66e80a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:59.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e5d9b640 1 --2- 172.21.15.89:0/3495575759 >> v2:172.21.15.190:3300/0 conn(0x7f66e80a02a0 0x7f66e80a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e659c640 1 --2- 172.21.15.89:0/3495575759 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66e80a3c30 0x7f66e80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 1 --2- 172.21.15.89:0/3495575759 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66e80a3c30 0x7f66e80a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 1 -- 172.21.15.89:0/3495575759 >> v2:172.21.15.190:3300/0 conn(0x7f66e80a02a0 msgr2=0x7f66e80a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:59.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 1 --2- 172.21.15.89:0/3495575759 >> v2:172.21.15.190:3300/0 conn(0x7f66e80a02a0 0x7f66e80a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:46:59.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 1 monclient: found mon.noname-c 2024-08-19T10:46:59.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:59.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:46:59.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e6d9d640 1 -- 172.21.15.89:0/3495575759 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e80a1020 con 0x7f66e813d910 2024-08-19T10:46:59.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e5d9b640 10 monclient: get_auth_request con 0x7f66e80a02a0 auth_method 0 2024-08-19T10:46:59.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e5d9b640 1 --2- 172.21.15.89:0/3495575759 >> v2:172.21.15.190:3300/0 conn(0x7f66e80a02a0 0x7f66e80a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:46:59.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e659c640 10 monclient: get_auth_request con 0x7f66e80a3c30 auth_method 0 2024-08-19T10:46:59.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.772+0000 7f66e659c640 1 --2- 172.21.15.89:0/3495575759 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66e80a3c30 0x7f66e80a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:46:59.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e6d9d640 1 -- 172.21.15.89:0/3495575759 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66e813d910 msgr2=0x7f66e813fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e6d9d640 1 --2- 172.21.15.89:0/3495575759 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66e813d910 0x7f66e813fd00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009e50 tx=0x7f66e001c980 comp rx=0 tx=0).ready entity=mon.4 client_cookie=6515e14feaa07f69 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 1 -- 172.21.15.89:0/3495575759 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66e00091b0 con 0x7f66e813d910 2024-08-19T10:46:59.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:46:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 10 monclient: dump: 2024-08-19T10:46:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.784 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 10 monclient: _finish_auth 0 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 10 monclient: _check_auth_tickets 2024-08-19T10:46:59.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 1 -- 172.21.15.89:0/3495575759 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e00a5a30 con 0x7f66e813d910 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 1 -- 172.21.15.89:0/3495575759 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66e00a5be0 con 0x7f66e813d910 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 5 monclient: authenticate success, global_id 44617 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 4 set_mon_vals no callback set 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66e559a640 10 monclient: dump: 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.786 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.787 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 10 monclient: shutdown 2024-08-19T10:46:59.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:59.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 1 -- 172.21.15.89:0/3495575759 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66e813d910 msgr2=0x7f66e813fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 1 --2- 172.21.15.89:0/3495575759 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66e813d910 0x7f66e813fd00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009e50 tx=0x7f66e001c980 comp rx=0 tx=0).stop 2024-08-19T10:46:59.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 1 -- 172.21.15.89:0/3495575759 shutdown_connections 2024-08-19T10:46:59.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 1 --2- 172.21.15.89:0/3495575759 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f66e813d910 0x7f66e813fd00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 1 --2- 172.21.15.89:0/3495575759 >> v2:172.21.15.190:3300/0 conn(0x7f66e80a02a0 0x7f66e80a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 1 --2- 172.21.15.89:0/3495575759 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66e80a3c30 0x7f66e80a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 1 -- 172.21.15.89:0/3495575759 >> 172.21.15.89:0/3495575759 conn(0x7f66e806de40 msgr2=0x7f66e806e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 1 -- 172.21.15.89:0/3495575759 shutdown_connections 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.776+0000 7f66ecbfa640 1 -- 172.21.15.89:0/3495575759 wait complete. 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient: build_initial_monmap 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient: monmap: 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:59.796 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:59.797 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:59.797 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 1 Processor -- start 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 1 -- start start 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient: init 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient: _add_conns ranks=[8,5,2,4,6,0,1,3,7] 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66e80a02a0 0x7f66e81387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient(hunting): picked mon.noname-e con 0x7f66e80a02a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66e80a3c30 0x7f66e8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient(hunting): picked mon.noname-h con 0x7f66e80a3c30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66e813d910 0x7f66e813a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient(hunting): picked mon.noname-b con 0x7f66e813d910 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:59.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f66e8142df0 con 0x7f66e813d910 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f66e8142c70 con 0x7f66e80a3c30 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f66e8142f70 con 0x7f66e80a02a0 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66ecbfa640 10 monclient(hunting): authenticate will time out at 1469.597134s 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e659c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66e80a02a0 0x7f66e81387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66e813d910 0x7f66e813a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e659c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66e80a02a0 0x7f66e81387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:36214/0 (socket says 172.21.15.89:36214) 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e659c640 1 -- 172.21.15.89:0/1173938409 learned_addr learned my addr 172.21.15.89:0/1173938409 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e659c640 10 monclient(hunting): get_auth_request con 0x7f66e80a02a0 auth_method 0 2024-08-19T10:46:59.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 10 monclient(hunting): get_auth_request con 0x7f66e813d910 auth_method 0 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e5d9b640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66e80a3c30 0x7f66e8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e5d9b640 10 monclient(hunting): get_auth_request con 0x7f66e80a3c30 auth_method 0 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 10 monclient(hunting): handle_auth_done global_id 54236 payload 824 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 1 -- 172.21.15.89:0/1173938409 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66e80a02a0 msgr2=0x7f66e81387d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66e80a02a0 0x7f66e81387d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 1 -- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66e80a3c30 msgr2=0x7f66e8138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66e80a3c30 0x7f66e8138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 1 monclient: found mon.noname-b 2024-08-19T10:46:59.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:59.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:46:59.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 1 -- 172.21.15.89:0/1173938409 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e8123f30 con 0x7f66e813d910 2024-08-19T10:46:59.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e659c640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f66e80a02a0 0x7f66e81387d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:59.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 1 -- 172.21.15.89:0/1173938409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66e813d910 msgr2=0x7f66e813a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.780+0000 7f66e6d9d640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66e813d910 0x7f66e813a6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f66e00ab000 tx=0x7f66e0019720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66e00bc070 con 0x7f66e813d910 2024-08-19T10:46:59.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:46:59.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 10 monclient: dump: 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e0003e10 con 0x7f66e813d910 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66e00044b0 con 0x7f66e813d910 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66ecbfa640 5 monclient: authenticate success, global_id 54236 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66ecbfa640 10 monclient: _renew_subs 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66ecbfa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66ecbfa640 1 -- 172.21.15.89:0/1173938409 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e8124210 con 0x7f66e813d910 2024-08-19T10:46:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e4d99640 4 set_mon_vals no callback set 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 10 monclient: dump: 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.805 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66ecbfa640 10 monclient: _renew_subs 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66ecbfa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66ecbfa640 1 -- 172.21.15.89:0/1173938409 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e80a74e0 con 0x7f66e813d910 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66ecbfa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66ecbfa640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e8123b00 0x7f66e813abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66ecbfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66ecbfa640 1 -- 172.21.15.89:0/1173938409 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8139980 con 0x7f66e8123b00 2024-08-19T10:46:59.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f66e0004710 con 0x7f66e813d910 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f66c803be00 0x7f66c803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e659c640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e8123b00 0x7f66e813abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f66e00b3040 con 0x7f66e813d910 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e659c640 10 monclient: get_auth_request con 0x7f66e8123b00 auth_method 0 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e5d9b640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f66c803be00 0x7f66c803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e5d9b640 10 monclient: get_auth_request con 0x7f66c803be00 auth_method 0 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e659c640 10 monclient(hunting): handle_auth_done global_id 44533 payload 824 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.784+0000 7f66e659c640 1 -- 172.21.15.89:0/1173938409 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66e80a4010 con 0x7f66e8123b00 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66e659c640 1 -- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e8123b00 msgr2=0x7f66e813abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66e659c640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e8123b00 0x7f66e813abe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66d800fc90 tx=0x7f66d800d590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66e5d9b640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f66c803be00 0x7f66c803e2c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f66e813bd80 tx=0x7f66dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66d800fea0 con 0x7f66e8123b00 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66d6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66d6ffd640 10 monclient: dump: 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.808 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f66d8013650 con 0x7f66e8123b00 2024-08-19T10:46:59.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66d6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66d6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e8123b00 msgr2=0x7f66e813abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66d6ffd640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e8123b00 0x7f66e813abe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66d800fc90 tx=0x7f66d800d590 comp rx=0 tx=0).stop 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e8123b00 msgr2=0x7f66e813abe0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.788+0000 7f66d6ffd640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e8123b00 0x7f66e813abe0 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66d800fc90 tx=0x7f66d800d590 comp rx=0 tx=0).stop 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66ecbfa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66ecbfa640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e813b120 0x7f66e813b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66ecbfa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66ecbfa640 1 -- 172.21.15.89:0/1173938409 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8064590 con 0x7f66e813b120 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66e659c640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e813b120 0x7f66e813b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66e659c640 10 monclient: get_auth_request con 0x7f66e813b120 auth_method 0 2024-08-19T10:46:59.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66e659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66e659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66e659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66e659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.792+0000 7f66e659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66e659c640 10 monclient(hunting): handle_auth_done global_id 44542 payload 824 2024-08-19T10:46:59.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66e659c640 1 -- 172.21.15.89:0/1173938409 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f66e813cb80 con 0x7f66e813b120 2024-08-19T10:46:59.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66e659c640 1 -- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e813b120 msgr2=0x7f66e813b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66e659c640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e813b120 0x7f66e813b530 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f66d8008010 tx=0x7f66d80046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66d80066c0 con 0x7f66e813b120 2024-08-19T10:46:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66d6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:46:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66d6ffd640 10 monclient: dump: 2024-08-19T10:46:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.813 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.814 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.815 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.816 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f66d8006250 con 0x7f66e813b120 2024-08-19T10:46:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66d6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:46:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66d6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:46:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e813b120 msgr2=0x7f66e813b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66d6ffd640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e813b120 0x7f66e813b530 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f66d8008010 tx=0x7f66d80046d0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66d6ffd640 1 -- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e813b120 msgr2=0x7f66e813b530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:46:59.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.796+0000 7f66d6ffd640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66e813b120 0x7f66e813b530 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f66d8008010 tx=0x7f66d80046d0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.817 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:46:59.817 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:59.817 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:59.817 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:46:59.818 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:46:59.818 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 142922, 2024-08-19T10:46:59.818 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:46:59.818 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:46:59.818 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:46:59.819 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:46:59.819 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:46:59.819 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:46:59.819 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:46:59.819 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:46:59.819 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:46:59.820 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:46:59.820 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.820 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:46:59.820 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:59.820 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:46:59.820 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.821 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.822 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:46:59.823 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:46:59.824 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:46:59.824 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:46:59.824 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:46:59.824 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:46:59.824 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:46:59.824 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:46:59.824 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:46:59.824 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:46:59.824 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:46:59.825 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:46:59.825 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:46:59.825 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:46:59.825 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:46:59.825 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:46:59.825 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:46:59.825 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:46:59.825 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:46:59.825 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:46:59.826 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:46:59.826 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.826 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:46:59.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.826 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:46:59.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.826 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:46:59.826 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:46:59.826 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.827 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:46:59.827 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.828 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.828 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:46:59.828 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.829 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.829 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:46:59.829 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:46:59.830 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.830 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.830 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.830 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:46:59.831 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:46:59.831 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.831 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:46:59.832 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.832 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:46:59.833 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.833 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.833 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:46:59.834 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:46:59.834 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.834 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.834 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.835 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:46:59.835 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:46:59.835 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:46:59.836 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.837 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:46:59.837 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.837 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.838 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:46:59.838 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:46:59.838 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.838 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.839 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.839 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:46:59.839 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:46:59.840 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:46:59.840 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.841 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.842 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.843 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.844 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.846 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.847 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.848 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:46:59.849 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:46:59.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:46:59.851 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:46:59.851 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:46:59.851 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:46:59.851 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:46:59.851 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:46:59.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:46:59.851 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:46:59.851 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:46:59.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66ecbfa640 1 -- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f66c803be00 msgr2=0x7f66c803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66ecbfa640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f66c803be00 0x7f66c803e2c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f66e813bd80 tx=0x7f66dc008040 comp rx=0 tx=0).stop 2024-08-19T10:46:59.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66ecbfa640 10 monclient: shutdown 2024-08-19T10:46:59.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66ecbfa640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:59.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66ecbfa640 1 -- 172.21.15.89:0/1173938409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66e813d910 msgr2=0x7f66e813a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66ecbfa640 1 --2- 172.21.15.89:0/1173938409 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66e813d910 0x7f66e813a6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f66e00ab000 tx=0x7f66e0019720 comp rx=0 tx=0).stop 2024-08-19T10:46:59.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66e6d9d640 1 -- 172.21.15.89:0/1173938409 reap_dead start 2024-08-19T10:46:59.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66e6d9d640 1 -- 172.21.15.89:0/1173938409 reap_dead start 2024-08-19T10:46:59.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66ecbfa640 1 -- 172.21.15.89:0/1173938409 shutdown_connections 2024-08-19T10:46:59.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66ecbfa640 1 -- 172.21.15.89:0/1173938409 >> 172.21.15.89:0/1173938409 conn(0x7f66e806de40 msgr2=0x7f66e80a5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:59.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66ecbfa640 1 -- 172.21.15.89:0/1173938409 shutdown_connections 2024-08-19T10:46:59.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.800+0000 7f66ecbfa640 1 -- 172.21.15.89:0/1173938409 wait complete. 2024-08-19T10:46:59.853 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:46:59.854 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:46:59.854 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:46:59.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient: get_monmap_and_config 2024-08-19T10:46:59.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient: build_initial_monmap 2024-08-19T10:46:59.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient: monmap: 2024-08-19T10:46:59.978 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:59.978 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:46:59.981112+0000 2024-08-19T10:46:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:46:59.981112+0000 2024-08-19T10:46:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:46:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:46:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:59.979 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:59.980 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:59.980 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:59.980 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:59.980 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:46:59.980 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:46:59.980 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 1 Processor -- start 2024-08-19T10:46:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 1 -- start start 2024-08-19T10:46:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient: init 2024-08-19T10:46:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient: _add_conns ranks=[8,9,2,1,5,7,0,10,4,3,6] 2024-08-19T10:46:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f744009b8e0 0x7f744009bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient(hunting): picked mon.noname-e con 0x7f744009b8e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:46:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f744009ddd0 0x7f74400a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient(hunting): picked mon.noname-a con 0x7f744009ddd0 addr v2:172.21.15.89:3300/0 2024-08-19T10:46:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74400a4240 0x7f74400a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:46:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient(hunting): picked mon.noname-b con 0x7f74400a4240 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:46:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f744009d760 con 0x7f744009ddd0 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f744009d9b0 con 0x7f74400a4240 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f744006fd00 con 0x7f744009b8e0 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient(hunting): _renew_subs 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f7446b18640 10 monclient(hunting): authenticate will time out at 1469.797710s 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f744009b8e0 0x7f744009bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f743ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f744009ddd0 0x7f74400a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74450b5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74400a4240 0x7f74400a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f744009b8e0 0x7f744009bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:36224/0 (socket says 172.21.15.89:36224) 2024-08-19T10:46:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 1 -- 172.21.15.89:0/2168120568 learned_addr learned my addr 172.21.15.89:0/2168120568 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 10 monclient(hunting): get_auth_request con 0x7f744009b8e0 auth_method 0 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f743ffff640 10 monclient(hunting): get_auth_request con 0x7f744009ddd0 auth_method 0 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f743ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f743ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f743ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74450b5640 10 monclient(hunting): get_auth_request con 0x7f74400a4240 auth_method 0 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74450b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74450b5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74450b5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:46:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74450b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74450b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74450b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f743ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f743ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f743ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 10 monclient(hunting): handle_auth_done global_id 6578 payload 274 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 1 -- 172.21.15.89:0/2168120568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74400a4240 msgr2=0x7f74400a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 1 --2- 172.21.15.89:0/2168120568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74400a4240 0x7f74400a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 1 -- 172.21.15.89:0/2168120568 >> v2:172.21.15.89:3300/0 conn(0x7f744009ddd0 msgr2=0x7f74400a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 1 --2- 172.21.15.89:0/2168120568 >> v2:172.21.15.89:3300/0 conn(0x7f744009ddd0 0x7f74400a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 10 monclient: _finish_hunting 0 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 1 monclient: found mon.noname-e 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74448b4640 1 -- 172.21.15.89:0/2168120568 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744006db50 con 0x7f744009b8e0 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f74450b5640 1 --2- 172.21.15.89:0/2168120568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74400a4240 0x7f74400a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.980+0000 7f743ffff640 1 --2- 172.21.15.89:0/2168120568 >> v2:172.21.15.89:3300/0 conn(0x7f744009ddd0 0x7f74400a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f74448b4640 1 -- 172.21.15.89:0/2168120568 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f744009b8e0 msgr2=0x7f744009bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f74448b4640 1 --2- 172.21.15.89:0/2168120568 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f744009b8e0 0x7f744009bcc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7438009870 tx=0x7f743801c420 comp rx=0 tx=0).ready entity=mon.8 client_cookie=29b92ddcd1ffc555 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 1 -- 172.21.15.89:0/2168120568 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74380b0070 con 0x7f744009b8e0 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 10 monclient: dump: 2024-08-19T10:46:59.993 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.994 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.995 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:46:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:46:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:46:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 1 -- 172.21.15.89:0/2168120568 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74380aa810 con 0x7f744009b8e0 2024-08-19T10:46:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:46:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 1 -- 172.21.15.89:0/2168120568 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74380a9300 con 0x7f744009b8e0 2024-08-19T10:46:59.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f743f7fe640 10 monclient: dump: 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:46:59.996 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 5 monclient: authenticate success, global_id 6578 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 10 monclient: get_monmap_and_config success 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 4 set_mon_vals no callback set 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 10 monclient: shutdown 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 1 -- 172.21.15.89:0/2168120568 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f744009b8e0 msgr2=0x7f744009bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 1 --2- 172.21.15.89:0/2168120568 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f744009b8e0 0x7f744009bcc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7438009870 tx=0x7f743801c420 comp rx=0 tx=0).stop 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 1 -- 172.21.15.89:0/2168120568 shutdown_connections 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 1 --2- 172.21.15.89:0/2168120568 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74400a4240 0x7f74400a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 1 --2- 172.21.15.89:0/2168120568 >> v2:172.21.15.89:3300/0 conn(0x7f744009ddd0 0x7f74400a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 1 --2- 172.21.15.89:0/2168120568 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f744009b8e0 0x7f744009bcc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 1 -- 172.21.15.89:0/2168120568 >> 172.21.15.89:0/2168120568 conn(0x7f74400a4df0 msgr2=0x7f74400a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 1 -- 172.21.15.89:0/2168120568 shutdown_connections 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.984+0000 7f7446b18640 1 -- 172.21.15.89:0/2168120568 wait complete. 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient: build_initial_monmap 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient: monmap: 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:46:59.998 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 1 Processor -- start 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 1 -- start start 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient: init 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient: _reopen_session rank -1 2024-08-19T10:46:59.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient: _add_conns ranks=[0,5,8,6,4,1,7,3,2] 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f744009b8e0 0x7f744009a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient(hunting): picked mon.noname-a con 0x7f744009b8e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f744009ddd0 0x7f7440095140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient(hunting): picked mon.noname-h con 0x7f744009ddd0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74400a4240 0x7f7440095680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient(hunting): picked mon.noname-e con 0x7f74400a4240 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7440069300 con 0x7f744009b8e0 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f74400a5c00 con 0x7f744009ddd0 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7440142cd0 con 0x7f74400a4240 2024-08-19T10:47:00.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f7446b18640 10 monclient(hunting): authenticate will time out at 1469.805332s 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74448b4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f744009b8e0 0x7f744009a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74448b4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f744009b8e0 0x7f744009a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50760/0 (socket says 172.21.15.89:50760) 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74448b4640 1 -- 172.21.15.89:0/2562505816 learned_addr learned my addr 172.21.15.89:0/2562505816 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74448b4640 10 monclient(hunting): get_auth_request con 0x7f744009b8e0 auth_method 0 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74448b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74448b4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74448b4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74400a4240 0x7f7440095680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 10 monclient(hunting): get_auth_request con 0x7f74400a4240 auth_method 0 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 10 monclient(hunting): handle_auth_done global_id 6587 payload 824 2024-08-19T10:47:00.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f743ffff640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f744009ddd0 0x7f7440095140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 1 -- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f744009ddd0 msgr2=0x7f7440095140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f744009ddd0 0x7f7440095140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 1 -- 172.21.15.89:0/2562505816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f744009b8e0 msgr2=0x7f744009a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f744009b8e0 0x7f744009a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 10 monclient: _finish_hunting 0 2024-08-19T10:47:00.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 1 monclient: found mon.noname-e 2024-08-19T10:47:00.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:00.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:47:00.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74450b5640 1 -- 172.21.15.89:0/2562505816 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74400a3560 con 0x7f74400a4240 2024-08-19T10:47:00.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.988+0000 7f74448b4640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f744009b8e0 0x7f744009a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:00.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74450b5640 1 -- 172.21.15.89:0/2562505816 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74400a4240 msgr2=0x7f7440095680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:00.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74450b5640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74400a4240 0x7f7440095680 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f743000efc0 tx=0x7f743000c490 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7430018030 con 0x7f74400a4240 2024-08-19T10:47:00.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:47:00.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 10 monclient: dump: 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.009 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74300148f0 con 0x7f74400a4240 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74300133e0 con 0x7f74400a4240 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f7446b18640 5 monclient: authenticate success, global_id 6587 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f7446b18640 10 monclient: _renew_subs 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f7446b18640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f7446b18640 1 -- 172.21.15.89:0/2562505816 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74400a47d0 con 0x7f74400a4240 2024-08-19T10:47:00.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743e7fc640 4 set_mon_vals no callback set 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 10 monclient: dump: 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.011 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f7446b18640 10 monclient: _renew_subs 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f7446b18640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f7446b18640 1 -- 172.21.15.89:0/2562505816 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f7440098cd0 con 0x7f74400a4240 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f7446b18640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f7446b18640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f744002e350 0x7f7440095bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f7446b18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f7446b18640 1 -- 172.21.15.89:0/2562505816 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7440096100 con 0x7f744002e350 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74450b5640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74400a4240 0x7f7440095680 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f743000efc0 tx=0x7f743000c490 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f7430019070 osd_map(271..271 src has 150..271) 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f743001b070 con 0x7f74400a4240 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743d7fa640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f742003be30 0x7f742003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74448b4640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f744002e350 0x7f7440095bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74448b4640 10 monclient: get_auth_request con 0x7f744002e350 auth_method 0 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74448b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74448b4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74448b4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743ffff640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f742003be30 0x7f742003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f743ffff640 10 monclient: get_auth_request con 0x7f742003be30 auth_method 0 2024-08-19T10:47:00.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74448b4640 10 monclient(hunting): handle_auth_done global_id 35202 payload 824 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.992+0000 7f74448b4640 1 -- 172.21.15.89:0/2562505816 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f744002e730 con 0x7f744002e350 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f74448b4640 1 -- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f744002e350 msgr2=0x7f7440095bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f74448b4640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f744002e350 0x7f7440095bc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f743801c250 tx=0x7f74380191c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743ffff640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f742003be30 0x7f742003e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f74400972c0 tx=0x7f7428008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74380b0040 con 0x7f744002e350 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743d7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743d7fa640 10 monclient: dump: 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.014 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f74380093d0 con 0x7f744002e350 2024-08-19T10:47:00.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f744002e350 msgr2=0x7f7440095bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743d7fa640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f744002e350 0x7f7440095bc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f743801c250 tx=0x7f74380191c0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f744002e350 msgr2=0x7f7440095bc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:46:59.996+0000 7f743d7fa640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f744002e350 0x7f7440095bc0 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f743801c250 tx=0x7f74380191c0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f7446b18640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f7446b18640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7440063b30 0x7f7440063f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f7446b18640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f7446b18640 1 -- 172.21.15.89:0/2562505816 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7440064590 con 0x7f7440063b30 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7440063b30 0x7f7440063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 10 monclient: get_auth_request con 0x7f7440063b30 auth_method 0 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 10 monclient(hunting): handle_auth_done global_id 35211 payload 824 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 1 -- 172.21.15.89:0/2562505816 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f74400a3d00 con 0x7f7440063b30 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 1 -- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7440063b30 msgr2=0x7f7440063f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.004+0000 7f74448b4640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7440063b30 0x7f7440063f40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7440076b60 tx=0x7f74380a8000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74380b0040 con 0x7f7440063b30 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 10 monclient: dump: 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.017 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.018 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f7438003620 con 0x7f7440063b30 2024-08-19T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7440063b30 msgr2=0x7f7440063f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7440063b30 0x7f7440063f40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7440076b60 tx=0x7f74380a8000 comp rx=0 tx=0).stop 2024-08-19T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7440063b30 msgr2=0x7f7440063f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7440063b30 0x7f7440063f40 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7440076b60 tx=0x7f74380a8000 comp rx=0 tx=0).stop 2024-08-19T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:47:00.019 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 265048, 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:47:00.020 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 7, 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:00.021 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:00.022 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:00.023 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:00.024 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.025 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:47:00.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.026 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:47:00.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.026 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:00.026 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:00.026 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.026 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.026 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:47:00.027 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:47:00.030 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:47:00.034 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:47:00.037 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.041 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:47:00.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.043 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:00.043 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:00.043 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:47:00.043 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.043 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.043 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:47:00.043 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:47:00.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f7446b18640 1 -- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f742003be30 msgr2=0x7f742003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f7446b18640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f742003be30 0x7f742003e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f74400972c0 tx=0x7f7428008040 comp rx=0 tx=0).stop 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f7446b18640 10 monclient: shutdown 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f7446b18640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f7446b18640 1 -- 172.21.15.89:0/2562505816 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74400a4240 msgr2=0x7f7440095680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f7446b18640 1 --2- 172.21.15.89:0/2562505816 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74400a4240 0x7f7440095680 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f743000efc0 tx=0x7f743000c490 comp rx=0 tx=0).stop 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f74450b5640 1 -- 172.21.15.89:0/2562505816 reap_dead start 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f74450b5640 1 -- 172.21.15.89:0/2562505816 reap_dead start 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f743d7fa640 1 -- 172.21.15.89:0/2562505816 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (unknown 0 0 0) 0x7f7430019070 con 0x7f74400a4240 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f7446b18640 1 -- 172.21.15.89:0/2562505816 shutdown_connections 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f7446b18640 1 -- 172.21.15.89:0/2562505816 >> 172.21.15.89:0/2562505816 conn(0x7f74400a4df0 msgr2=0x7f74400a08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f7446b18640 1 -- 172.21.15.89:0/2562505816 shutdown_connections 2024-08-19T10:47:00.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.008+0000 7f7446b18640 1 -- 172.21.15.89:0/2562505816 wait complete. 2024-08-19T10:47:00.046 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:47:00.046 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:47:00.046 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:47:00.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient: get_monmap_and_config 2024-08-19T10:47:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient: build_initial_monmap 2024-08-19T10:47:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient: monmap: 2024-08-19T10:47:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:47:00.169691+0000 2024-08-19T10:47:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:47:00.169691+0000 2024-08-19T10:47:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.234 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:47:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:47:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:47:00.235 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:47:00.235 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 1 Processor -- start 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 1 -- start start 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient: init 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient: _add_conns ranks=[0,6,5,9,1,3,7,10,8,2,4] 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f64480a6860 addr v1:172.21.15.89:6789/0 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f64480a8480 0x7f64480a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient(hunting): picked mon.noname-d con 0x7f64480a8480 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813e860 0x7f6448140c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient(hunting): picked mon.noname-h con 0x7f644813e860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64480a8240 con 0x7f64480a6860 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f64480a4d50 con 0x7f644813e860 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f64480a2ec0 con 0x7f64480a8480 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f644cceb640 10 monclient(hunting): authenticate will time out at 1469.986308s 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f64480a8480 0x7f64480a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f64480a8480 0x7f64480a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:35604/0 (socket says 172.21.15.89:35604) 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 1 -- 172.21.15.89:0/1189703884 learned_addr learned my addr 172.21.15.89:0/1189703884 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 10 monclient(hunting): get_auth_request con 0x7f64480a8480 auth_method 0 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 10 monclient(hunting): handle_auth_done global_id 44637 payload 274 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6446d9d640 1 --2- 172.21.15.89:0/1189703884 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813e860 0x7f6448140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 1 -- 172.21.15.89:0/1189703884 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813e860 msgr2=0x7f6448140c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 1 --2- 172.21.15.89:0/1189703884 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813e860 0x7f6448140c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 1 -- 172.21.15.89:0/1189703884 >> v1:172.21.15.89:6789/0 conn(0x7f64480a6860 legacy=0x7f64480a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 1 monclient: found mon.noname-d 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.168+0000 7f6445d9b640 1 -- 172.21.15.89:0/1189703884 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64480a4a70 con 0x7f64480a8480 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f6445d9b640 1 -- 172.21.15.89:0/1189703884 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f64480a8480 msgr2=0x7f64480a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f6445d9b640 1 --2- 172.21.15.89:0/1189703884 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f64480a8480 0x7f64480a7350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f64380098a0 tx=0x7f643801c440 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ed67bbe5352644d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 1 -- 172.21.15.89:0/1189703884 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64380b0070 con 0x7f64480a8480 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 10 monclient: dump: 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.242 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 10 monclient: _finish_auth 0 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 10 monclient: _check_auth_tickets 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 1 -- 172.21.15.89:0/1189703884 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64380aa810 con 0x7f64480a8480 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 1 -- 172.21.15.89:0/1189703884 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64380a93e0 con 0x7f64480a8480 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 5 monclient: authenticate success, global_id 44637 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 10 monclient: get_monmap_and_config success 2024-08-19T10:47:00.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 4 set_mon_vals no callback set 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644559a640 10 monclient: dump: 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.244 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 10 monclient: shutdown 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 1 -- 172.21.15.89:0/1189703884 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f64480a8480 msgr2=0x7f64480a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 1 --2- 172.21.15.89:0/1189703884 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f64480a8480 0x7f64480a7350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f64380098a0 tx=0x7f643801c440 comp rx=0 tx=0).stop 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 1 -- 172.21.15.89:0/1189703884 shutdown_connections 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 1 --2- 172.21.15.89:0/1189703884 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813e860 0x7f6448140c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 1 --2- 172.21.15.89:0/1189703884 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f64480a8480 0x7f64480a7350 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 1 -- 172.21.15.89:0/1189703884 >> 172.21.15.89:0/1189703884 conn(0x7f64480699d0 msgr2=0x7f64480abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:00.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 1 -- 172.21.15.89:0/1189703884 shutdown_connections 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.172+0000 7f644cceb640 1 -- 172.21.15.89:0/1189703884 wait complete. 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient: build_initial_monmap 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient: monmap: 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:00.246 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:00.247 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:00.247 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:00.247 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:00.247 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:00.247 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:00.247 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.248 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.248 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420233870}) 2024-08-19T10:47:00.248 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038880412}) 2024-08-19T10:47:00.248 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.248 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420233870}) 2024-08-19T10:47:00.249 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2006242921}) 2024-08-19T10:47:00.249 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.249 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420233870}) 2024-08-19T10:47:00.249 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2524042193}) 2024-08-19T10:47:00.249 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.249 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420233870}) 2024-08-19T10:47:00.249 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3512056897}) 2024-08-19T10:47:00.249 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.249 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420233870}) 2024-08-19T10:47:00.249 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=163634347}) 2024-08-19T10:47:00.249 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 1 Processor -- start 2024-08-19T10:47:00.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420233870}) 2024-08-19T10:47:00.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=679925311}) 2024-08-19T10:47:00.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420233870}) 2024-08-19T10:47:00.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 1 -- start start 2024-08-19T10:47:00.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient: init 2024-08-19T10:47:00.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1686496457}) 2024-08-19T10:47:00.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420233870}) 2024-08-19T10:47:00.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3487536546}) 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient: _add_conns ranks=[8,4,6,3,2,5,0,7,1] 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64480a6860 0x7f6448125990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient(hunting): picked mon.noname-e con 0x7f64480a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64480a8480 0x7f644813c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient(hunting): picked mon.noname-c con 0x7f64480a8480 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f644813e860 0x7f6448123370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient(hunting): picked mon.noname-d con 0x7f644813e860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6448143d40 con 0x7f64480a8480 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6448143bc0 con 0x7f644813e860 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6448143ec0 con 0x7f64480a6860 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644cceb640 10 monclient(hunting): authenticate will time out at 1469.993692s 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6446d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f644813e860 0x7f6448123370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6445d9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64480a8480 0x7f644813c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6446d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f644813e860 0x7f6448123370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:35616/0 (socket says 172.21.15.89:35616) 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6446d9d640 1 -- 172.21.15.89:0/2164723495 learned_addr learned my addr 172.21.15.89:0/2164723495 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6445d9b640 10 monclient(hunting): get_auth_request con 0x7f64480a8480 auth_method 0 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6445d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6445d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64480a6860 0x7f6448125990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6445d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6446d9d640 10 monclient(hunting): get_auth_request con 0x7f644813e860 auth_method 0 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6446d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6446d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6446d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 10 monclient(hunting): get_auth_request con 0x7f64480a6860 auth_method 0 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6446d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6446d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f6446d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 10 monclient(hunting): handle_auth_done global_id 6596 payload 824 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 1 -- 172.21.15.89:0/2164723495 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f644813e860 msgr2=0x7f6448123370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f644813e860 0x7f6448123370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 1 -- 172.21.15.89:0/2164723495 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64480a8480 msgr2=0x7f644813c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64480a8480 0x7f644813c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 10 monclient: _finish_hunting 0 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 1 monclient: found mon.noname-e 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.176+0000 7f644659c640 1 -- 172.21.15.89:0/2164723495 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6448127190 con 0x7f64480a6860 2024-08-19T10:47:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644659c640 1 -- 172.21.15.89:0/2164723495 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64480a6860 msgr2=0x7f6448125990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644659c640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64480a6860 0x7f6448125990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f643000c730 tx=0x7f643000cbf0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6430016020 con 0x7f64480a6860 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 10 monclient: dump: 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.261 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.262 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 10 monclient: _finish_auth 0 2024-08-19T10:47:00.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 10 monclient: _check_auth_tickets 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f643000ce40 con 0x7f64480a6860 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644cceb640 5 monclient: authenticate success, global_id 6596 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644cceb640 10 monclient: _renew_subs 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644cceb640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644cceb640 1 -- 172.21.15.89:0/2164723495 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64481d1ca0 con 0x7f64480a6860 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6430012890 con 0x7f64480a6860 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6444d99640 4 set_mon_vals no callback set 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 10 monclient: dump: 2024-08-19T10:47:00.263 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.264 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644cceb640 10 monclient: _renew_subs 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644cceb640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644cceb640 1 -- 172.21.15.89:0/2164723495 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f6448124e30 con 0x7f64480a6860 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644cceb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6430019020 con 0x7f64480a6860 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644cceb640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813cd40 0x7f644802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644cceb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f644cceb640 1 -- 172.21.15.89:0/2164723495 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f64480a6c40 con 0x7f644813cd40 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f642803be30 0x7f642803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f643004f870 con 0x7f64480a6860 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6445d9b640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813cd40 0x7f644802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6445d9b640 10 monclient: get_auth_request con 0x7f644813cd40 auth_method 0 2024-08-19T10:47:00.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6445d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6445d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6445d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6446d9d640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f642803be30 0x7f642803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6446d9d640 10 monclient: get_auth_request con 0x7f642803be30 auth_method 0 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6445d9b640 10 monclient(hunting): handle_auth_done global_id 6701 payload 824 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.180+0000 7f6445d9b640 1 -- 172.21.15.89:0/2164723495 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6448064590 con 0x7f644813cd40 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f6446d9d640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f642803be30 0x7f642803e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6440006fd0 tx=0x7f6440008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f6445d9b640 1 -- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813cd40 msgr2=0x7f644802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f6445d9b640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813cd40 0x7f644802d1d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f64380ad000 tx=0x7f64380191e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64380ba070 con 0x7f644813cd40 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f643effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f643effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f643effd640 10 monclient: dump: 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.267 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6438003910 con 0x7f644813cd40 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f643effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f643effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813cd40 msgr2=0x7f644802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f643effd640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813cd40 0x7f644802d1d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f64380ad000 tx=0x7f64380191e0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813cd40 msgr2=0x7f644802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.184+0000 7f643effd640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644813cd40 0x7f644802d1d0 secure :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f64380ad000 tx=0x7f64380191e0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f644cceb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:47:00.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f644cceb640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644802d710 0x7f644802db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f644cceb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f644cceb640 1 -- 172.21.15.89:0/2164723495 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f64480a0100 con 0x7f644802d710 2024-08-19T10:47:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f6445d9b640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644802d710 0x7f644802db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f6445d9b640 10 monclient: get_auth_request con 0x7f644802d710 auth_method 0 2024-08-19T10:47:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f6445d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f6445d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f6445d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f6445d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f6445d9b640 10 monclient(hunting): handle_auth_done global_id 6710 payload 824 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.192+0000 7f6445d9b640 1 -- 172.21.15.89:0/2164723495 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f644806ca50 con 0x7f644802d710 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f6445d9b640 1 -- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644802d710 msgr2=0x7f644802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f6445d9b640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644802d710 0x7f644802db20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f643801c270 tx=0x7f64380a8000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64380b0040 con 0x7f644802d710 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f643effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f643effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f643effd640 10 monclient: dump: 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.270 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9601 (secure 0 0 0) 0x7f64380c3070 con 0x7f644802d710 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f643effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f643effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:00.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644802d710 msgr2=0x7f644802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f643effd640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644802d710 0x7f644802db20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f643801c270 tx=0x7f64380a8000 comp rx=0 tx=0).stop 2024-08-19T10:47:00.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f643effd640 1 -- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644802d710 msgr2=0x7f644802db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:00.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f643effd640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f644802d710 0x7f644802db20 secure :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f643801c270 tx=0x7f64380a8000 comp rx=0 tx=0).stop 2024-08-19T10:47:00.272 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:47:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:47:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:47:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 549963, 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 7, 2024-08-19T10:47:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:47:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:47:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:47:00.295 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:47:00.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f644cceb640 1 -- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f642803be30 msgr2=0x7f642803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f644cceb640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f642803be30 0x7f642803e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6440006fd0 tx=0x7f6440008040 comp rx=0 tx=0).stop 2024-08-19T10:47:00.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f644cceb640 10 monclient: shutdown 2024-08-19T10:47:00.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f644cceb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:00.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f644cceb640 1 -- 172.21.15.89:0/2164723495 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64480a6860 msgr2=0x7f6448125990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f644cceb640 1 --2- 172.21.15.89:0/2164723495 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64480a6860 0x7f6448125990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f643000c730 tx=0x7f643000cbf0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f6446d9d640 1 -- 172.21.15.89:0/2164723495 reap_dead start 2024-08-19T10:47:00.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f6446d9d640 1 -- 172.21.15.89:0/2164723495 reap_dead start 2024-08-19T10:47:00.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f644cceb640 1 -- 172.21.15.89:0/2164723495 shutdown_connections 2024-08-19T10:47:00.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f644cceb640 1 -- 172.21.15.89:0/2164723495 >> 172.21.15.89:0/2164723495 conn(0x7f64480699d0 msgr2=0x7f644813f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:00.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f644cceb640 1 -- 172.21.15.89:0/2164723495 shutdown_connections 2024-08-19T10:47:00.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.196+0000 7f644cceb640 1 -- 172.21.15.89:0/2164723495 wait complete. 2024-08-19T10:47:00.298 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:47:00.298 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:47:00.298 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:47:00.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient: get_monmap_and_config 2024-08-19T10:47:00.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient: build_initial_monmap 2024-08-19T10:47:00.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient: monmap: 2024-08-19T10:47:00.426 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:00.426 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.426 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:47:00.429342+0000 2024-08-19T10:47:00.426 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:47:00.429342+0000 2024-08-19T10:47:00.426 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:47:00.427 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 1 Processor -- start 2024-08-19T10:47:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 1 -- start start 2024-08-19T10:47:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient: init 2024-08-19T10:47:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient: _add_conns ranks=[9,8,3,4,0,2,7,6,1,5,10] 2024-08-19T10:47:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18bc0a5270 0x7f18bc0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient(hunting): picked mon.noname-a con 0x7f18bc0a5270 addr v2:172.21.15.89:3300/0 2024-08-19T10:47:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f18bc0a6d30 0x7f18bc0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient(hunting): picked mon.noname-e con 0x7f18bc0a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18bc1395b0 0x7f18bc139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient(hunting): picked mon.noname-g con 0x7f18bc1395b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f18bc0a7650 con 0x7f18bc0a5270 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc0a8710 con 0x7f18bc1395b0 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f18bc0a5a40 con 0x7f18bc0a6d30 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c2975640 10 monclient(hunting): authenticate will time out at 1470.245855s 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bbfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18bc0a5270 0x7f18bc0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f18bc0a6d30 0x7f18bc0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f18bc0a6d30 0x7f18bc0a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:36252/0 (socket says 172.21.15.89:36252) 2024-08-19T10:47:00.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bbfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f18bc0a5270 0x7f18bc0a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50762/0 (socket says 172.21.15.89:50762) 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 1 -- 172.21.15.89:0/380635029 learned_addr learned my addr 172.21.15.89:0/380635029 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 10 monclient(hunting): get_auth_request con 0x7f18bc0a6d30 auth_method 0 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bbfff640 10 monclient(hunting): get_auth_request con 0x7f18bc0a5270 auth_method 0 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c0f12640 1 --2- 172.21.15.89:0/380635029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18bc1395b0 0x7f18bc139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c0f12640 10 monclient(hunting): get_auth_request con 0x7f18bc1395b0 auth_method 0 2024-08-19T10:47:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c0f12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c0f12640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18c0f12640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 10 monclient(hunting): handle_auth_done global_id 6605 payload 274 2024-08-19T10:47:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 1 -- 172.21.15.89:0/380635029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18bc1395b0 msgr2=0x7f18bc139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 1 --2- 172.21.15.89:0/380635029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18bc1395b0 0x7f18bc139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 1 -- 172.21.15.89:0/380635029 >> v2:172.21.15.89:3300/0 conn(0x7f18bc0a5270 msgr2=0x7f18bc0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:00.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 1 --2- 172.21.15.89:0/380635029 >> v2:172.21.15.89:3300/0 conn(0x7f18bc0a5270 0x7f18bc0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:47:00.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 1 monclient: found mon.noname-e 2024-08-19T10:47:00.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:00.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:47:00.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bb7fe640 1 -- 172.21.15.89:0/380635029 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc0a81d0 con 0x7f18bc0a6d30 2024-08-19T10:47:00.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.428+0000 7f18bbfff640 1 --2- 172.21.15.89:0/380635029 >> v2:172.21.15.89:3300/0 conn(0x7f18bc0a5270 0x7f18bc0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:00.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18bb7fe640 1 -- 172.21.15.89:0/380635029 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f18bc0a6d30 msgr2=0x7f18bc0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:00.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18bb7fe640 1 --2- 172.21.15.89:0/380635029 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f18bc0a6d30 0x7f18bc0a7110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009870 tx=0x7f18b001c420 comp rx=0 tx=0).ready entity=mon.8 client_cookie=bc81bc7760f46f98 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 1 -- 172.21.15.89:0/380635029 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18b00b1070 con 0x7f18bc0a6d30 2024-08-19T10:47:00.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 10 monclient: dump: 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.436 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 10 monclient: _finish_auth 0 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 10 monclient: _check_auth_tickets 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 1 -- 172.21.15.89:0/380635029 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b00ab810 con 0x7f18bc0a6d30 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 5 monclient: authenticate success, global_id 6605 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 10 monclient: get_monmap_and_config success 2024-08-19T10:47:00.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 4 set_mon_vals no callback set 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 1 -- 172.21.15.89:0/380635029 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18b00aa300 con 0x7f18bc0a6d30 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18baffd640 10 monclient: dump: 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.439 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.439 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.439 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.439 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.439 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.439 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193940696}) 2024-08-19T10:47:00.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=463141662}) 2024-08-19T10:47:00.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193940696}) 2024-08-19T10:47:00.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 10 monclient: shutdown 2024-08-19T10:47:00.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:00.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 1 -- 172.21.15.89:0/380635029 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f18bc0a6d30 msgr2=0x7f18bc0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3942596559}) 2024-08-19T10:47:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 1 --2- 172.21.15.89:0/380635029 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f18bc0a6d30 0x7f18bc0a7110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009870 tx=0x7f18b001c420 comp rx=0 tx=0).stop 2024-08-19T10:47:00.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193940696}) 2024-08-19T10:47:00.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1069508399}) 2024-08-19T10:47:00.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193940696}) 2024-08-19T10:47:00.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 1 -- 172.21.15.89:0/380635029 shutdown_connections 2024-08-19T10:47:00.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2869420867}) 2024-08-19T10:47:00.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 1 --2- 172.21.15.89:0/380635029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f18bc1395b0 0x7f18bc139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 1 --2- 172.21.15.89:0/380635029 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f18bc0a6d30 0x7f18bc0a7110 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 1 --2- 172.21.15.89:0/380635029 >> v2:172.21.15.89:3300/0 conn(0x7f18bc0a5270 0x7f18bc0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 1 -- 172.21.15.89:0/380635029 >> 172.21.15.89:0/380635029 conn(0x7f18bc069a80 msgr2=0x7f18bc0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:00.444 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.444 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193940696}) 2024-08-19T10:47:00.444 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1291907307}) 2024-08-19T10:47:00.444 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 1 -- 172.21.15.89:0/380635029 shutdown_connections 2024-08-19T10:47:00.444 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193940696}) 2024-08-19T10:47:00.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f18c2975640 1 -- 172.21.15.89:0/380635029 wait complete. 2024-08-19T10:47:00.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2136634808}) 2024-08-19T10:47:00.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193940696}) 2024-08-19T10:47:00.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=593628558}) 2024-08-19T10:47:00.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1193940696}) 2024-08-19T10:47:00.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.432+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=952878961}) 2024-08-19T10:47:00.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient: build_initial_monmap 2024-08-19T10:47:00.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient: monmap: 2024-08-19T10:47:00.448 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:00.448 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:47:00.448 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:47:00.448 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:47:00.448 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 1 Processor -- start 2024-08-19T10:47:00.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 1 -- start start 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient: init 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient: _add_conns ranks=[5,4,1,6,2,3,0,7,8] 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f18bc0a5270 0x7f18bc13d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient(hunting): picked mon.noname-h con 0x7f18bc0a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18bc0a6d30 0x7f18bc13d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient(hunting): picked mon.noname-c con 0x7f18bc0a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18bc1395b0 0x7f18bc1d1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient(hunting): picked mon.noname-f con 0x7f18bc1395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc0a5650 con 0x7f18bc1395b0 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f18bc0a4a70 con 0x7f18bc0a6d30 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f18bc0a57d0 con 0x7f18bc0a5270 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c2975640 10 monclient(hunting): authenticate will time out at 1470.253943s 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18bc0a6d30 0x7f18bc13d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18bc0a6d30 0x7f18bc13d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:42350/0 (socket says 172.21.15.89:42350) 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 1 -- 172.21.15.89:0/461816742 learned_addr learned my addr 172.21.15.89:0/461816742 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 10 monclient(hunting): get_auth_request con 0x7f18bc0a6d30 auth_method 0 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c0f12640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18bc1395b0 0x7f18bc1d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bbfff640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f18bc0a5270 0x7f18bc13d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bbfff640 10 monclient(hunting): get_auth_request con 0x7f18bc0a5270 auth_method 0 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c0f12640 10 monclient(hunting): get_auth_request con 0x7f18bc1395b0 auth_method 0 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c0f12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c0f12640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18c0f12640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 10 monclient(hunting): handle_auth_done global_id 44635 payload 824 2024-08-19T10:47:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 1 -- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f18bc0a5270 msgr2=0x7f18bc13d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f18bc0a5270 0x7f18bc13d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 1 -- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18bc1395b0 msgr2=0x7f18bc1d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18bc1395b0 0x7f18bc1d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:47:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 1 monclient: found mon.noname-c 2024-08-19T10:47:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:47:00.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f18bb7fe640 1 -- 172.21.15.89:0/461816742 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc1405f0 con 0x7f18bc0a6d30 2024-08-19T10:47:00.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777620245}) 2024-08-19T10:47:00.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2397450068}) 2024-08-19T10:47:00.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777620245}) 2024-08-19T10:47:00.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3635820932}) 2024-08-19T10:47:00.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777620245}) 2024-08-19T10:47:00.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1325359948}) 2024-08-19T10:47:00.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777620245}) 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3837475217}) 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777620245}) 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2316924054}) 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777620245}) 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3084158038}) 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777620245}) 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2814346190}) 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777620245}) 2024-08-19T10:47:00.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.436+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=806402544}) 2024-08-19T10:47:00.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bb7fe640 1 -- 172.21.15.89:0/461816742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18bc0a6d30 msgr2=0x7f18bc13d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:00.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bb7fe640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18bc0a6d30 0x7f18bc13d920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f18b00af040 tx=0x7f18b00191c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18b00b1040 con 0x7f18bc0a6d30 2024-08-19T10:47:00.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:47:00.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 10 monclient: dump: 2024-08-19T10:47:00.458 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.459 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 10 monclient: _finish_auth 0 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b00ba070 con 0x7f18bc0a6d30 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c2975640 5 monclient: authenticate success, global_id 44635 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c2975640 10 monclient: _renew_subs 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c2975640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c2975640 1 -- 172.21.15.89:0/461816742 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc13f990 con 0x7f18bc0a6d30 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18b00abe30 con 0x7f18bc0a6d30 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:47:00.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 10 monclient: dump: 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.461 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18ba7fc640 4 set_mon_vals no callback set 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c2975640 10 monclient: _renew_subs 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c2975640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c2975640 1 -- 172.21.15.89:0/461816742 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc0a28e0 con 0x7f18bc0a6d30 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c2975640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c2975640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc1d1f50 0x7f18bc02d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c2975640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c2975640 1 -- 172.21.15.89:0/461816742 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f18bc064590 con 0x7f18bc1d1f50 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bbfff640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc1d1f50 0x7f18bc02d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f18b0004250 con 0x7f18bc0a6d30 2024-08-19T10:47:00.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bbfff640 10 monclient: get_auth_request con 0x7f18bc1d1f50 auth_method 0 2024-08-19T10:47:00.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4016424963}) 2024-08-19T10:47:00.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3951261120}) 2024-08-19T10:47:00.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4016424963}) 2024-08-19T10:47:00.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1671899096}) 2024-08-19T10:47:00.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4016424963}) 2024-08-19T10:47:00.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3795010115}) 2024-08-19T10:47:00.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f18a803be00 0x7f18a803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.465 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4016424963}) 2024-08-19T10:47:00.465 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2671439880}) 2024-08-19T10:47:00.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.466 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.466 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4016424963}) 2024-08-19T10:47:00.466 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=860185454}) 2024-08-19T10:47:00.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f18b00e93b0 con 0x7f18bc0a6d30 2024-08-19T10:47:00.467 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.467 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4016424963}) 2024-08-19T10:47:00.467 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=952217049}) 2024-08-19T10:47:00.467 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.467 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4016424963}) 2024-08-19T10:47:00.467 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=51525894}) 2024-08-19T10:47:00.467 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.467 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4016424963}) 2024-08-19T10:47:00.467 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=459420146}) 2024-08-19T10:47:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c0f12640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f18a803be00 0x7f18a803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bbfff640 10 monclient(hunting): handle_auth_done global_id 16027 payload 824 2024-08-19T10:47:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18bbfff640 1 -- 172.21.15.89:0/461816742 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18bc139300 con 0x7f18bc1d1f50 2024-08-19T10:47:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.440+0000 7f18c0f12640 10 monclient: get_auth_request con 0x7f18a803be00 auth_method 0 2024-08-19T10:47:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18bbfff640 1 -- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc1d1f50 msgr2=0x7f18bc02d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18bbfff640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc1d1f50 0x7f18bc02d1d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f18ac011510 tx=0x7f18ac0119d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18ac015dd0 con 0x7f18bc1d1f50 2024-08-19T10:47:00.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18b8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.469 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.469 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3928123038}) 2024-08-19T10:47:00.469 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3913216550}) 2024-08-19T10:47:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18b8ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:47:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18b8ff9640 10 monclient: dump: 2024-08-19T10:47:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.469 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.470 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.471 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.471 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3928123038}) 2024-08-19T10:47:00.471 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3144764115}) 2024-08-19T10:47:00.471 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.471 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3928123038}) 2024-08-19T10:47:00.471 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2040219072}) 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3928123038}) 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1113266347}) 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3928123038}) 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2478217991}) 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3928123038}) 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2162949251}) 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3928123038}) 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3285756579}) 2024-08-19T10:47:00.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18c0f12640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f18a803be00 0x7f18a803e2c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009f00 tx=0x7f18b400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.473 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3928123038}) 2024-08-19T10:47:00.473 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=311564076}) 2024-08-19T10:47:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f18ac015960 con 0x7f18bc1d1f50 2024-08-19T10:47:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18b8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:00.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18b8ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:00.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc1d1f50 msgr2=0x7f18bc02d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18b8ff9640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc1d1f50 0x7f18bc02d1d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f18ac011510 tx=0x7f18ac0119d0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc1d1f50 msgr2=0x7f18bc02d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:00.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.444+0000 7f18b8ff9640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc1d1f50 0x7f18bc02d1d0 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f18ac011510 tx=0x7f18ac0119d0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.482 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.482 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966588712}) 2024-08-19T10:47:00.482 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1856900702}) 2024-08-19T10:47:00.482 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.482 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966588712}) 2024-08-19T10:47:00.482 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1416128571}) 2024-08-19T10:47:00.482 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.482 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966588712}) 2024-08-19T10:47:00.482 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4086258669}) 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966588712}) 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1485931427}) 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966588712}) 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2728052771}) 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966588712}) 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=21221572}) 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966588712}) 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4201239476}) 2024-08-19T10:47:00.483 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1966588712}) 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.448+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1028903172}) 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=628948167}) 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=61736936}) 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=628948167}) 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=158699322}) 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=628948167}) 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2819638344}) 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.484 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=628948167}) 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1856789746}) 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=628948167}) 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2412760351}) 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=628948167}) 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3888624111}) 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=628948167}) 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2770970837}) 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=628948167}) 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3068940232}) 2024-08-19T10:47:00.485 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.486 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228414342}) 2024-08-19T10:47:00.486 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3035404916}) 2024-08-19T10:47:00.486 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.486 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228414342}) 2024-08-19T10:47:00.486 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3495226399}) 2024-08-19T10:47:00.486 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.486 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228414342}) 2024-08-19T10:47:00.486 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2125462952}) 2024-08-19T10:47:00.486 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.487 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228414342}) 2024-08-19T10:47:00.487 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2399108551}) 2024-08-19T10:47:00.487 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.487 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228414342}) 2024-08-19T10:47:00.487 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1237081971}) 2024-08-19T10:47:00.487 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.487 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228414342}) 2024-08-19T10:47:00.487 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=352390433}) 2024-08-19T10:47:00.487 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.487 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228414342}) 2024-08-19T10:47:00.487 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1843048113}) 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228414342}) 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.452+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1663165859}) 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307359613}) 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2381319146}) 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307359613}) 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1158397896}) 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307359613}) 2024-08-19T10:47:00.488 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1900427897}) 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307359613}) 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2692853301}) 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307359613}) 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3827559285}) 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307359613}) 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=735817336}) 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307359613}) 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1698292229}) 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.489 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307359613}) 2024-08-19T10:47:00.490 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3191250101}) 2024-08-19T10:47:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18c2975640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:47:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18c2975640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc0a7110 0x7f18bc02d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18c2975640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18c2975640 1 -- 172.21.15.89:0/461816742 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f18bc06c9e0 con 0x7f18bc0a7110 2024-08-19T10:47:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18bbfff640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc0a7110 0x7f18bc02d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18bbfff640 10 monclient: get_auth_request con 0x7f18bc0a7110 auth_method 0 2024-08-19T10:47:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18bbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:00.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:00.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:00.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.456+0000 7f18bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:00.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f18bbfff640 10 monclient(hunting): handle_auth_done global_id 16036 payload 824 2024-08-19T10:47:00.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f18bbfff640 1 -- 172.21.15.89:0/461816742 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f18bc13f4d0 con 0x7f18bc0a7110 2024-08-19T10:47:00.493 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.493 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=207296395}) 2024-08-19T10:47:00.493 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4269428023}) 2024-08-19T10:47:00.493 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.493 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=207296395}) 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1206402243}) 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=207296395}) 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=766890084}) 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=207296395}) 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2941003275}) 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=207296395}) 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4024141290}) 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=207296395}) 2024-08-19T10:47:00.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3751074347}) 2024-08-19T10:47:00.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=207296395}) 2024-08-19T10:47:00.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=510394977}) 2024-08-19T10:47:00.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=207296395}) 2024-08-19T10:47:00.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2346609826}) 2024-08-19T10:47:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f18bbfff640 1 -- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc0a7110 msgr2=0x7f18bc02d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f18bbfff640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc0a7110 0x7f18bc02d710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f18ac011510 tx=0x7f18ac0046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f18ac0064b0 con 0x7f18bc0a7110 2024-08-19T10:47:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f18b8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f18b8ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:47:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.460+0000 7f18b8ff9640 10 monclient: dump: 2024-08-19T10:47:00.496 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:00.497 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:00.498 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f18ac005ef0 con 0x7f18bc0a7110 2024-08-19T10:47:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18b8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:47:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18b8ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc0a7110 msgr2=0x7f18bc02d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18b8ff9640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc0a7110 0x7f18bc02d710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f18ac011510 tx=0x7f18ac0046d0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18b8ff9640 1 -- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc0a7110 msgr2=0x7f18bc02d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:00.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18b8ff9640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f18bc0a7110 0x7f18bc02d710 secure :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f18ac011510 tx=0x7f18ac0046d0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.498 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 82, 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 398048, 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:47:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 7, 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:47:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:47:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:47:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:47:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:47:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:47:00.523 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:47:00.524 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.524 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2488587795}) 2024-08-19T10:47:00.524 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=34780865}) 2024-08-19T10:47:00.524 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.524 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2488587795}) 2024-08-19T10:47:00.524 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2394770021}) 2024-08-19T10:47:00.524 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2488587795}) 2024-08-19T10:47:00.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2608465344}) 2024-08-19T10:47:00.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2488587795}) 2024-08-19T10:47:00.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2922404117}) 2024-08-19T10:47:00.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c2975640 1 -- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f18a803be00 msgr2=0x7f18a803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2488587795}) 2024-08-19T10:47:00.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c2975640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f18a803be00 0x7f18a803e2c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009f00 tx=0x7f18b400b040 comp rx=0 tx=0).stop 2024-08-19T10:47:00.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c2975640 10 monclient: shutdown 2024-08-19T10:47:00.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c2975640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:00.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c2975640 1 -- 172.21.15.89:0/461816742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18bc0a6d30 msgr2=0x7f18bc13d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:00.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c2975640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18bc0a6d30 0x7f18bc13d920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f18b00af040 tx=0x7f18b00191c0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c0f12640 1 -- 172.21.15.89:0/461816742 reap_dead start 2024-08-19T10:47:00.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3888096167}) 2024-08-19T10:47:00.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2488587795}) 2024-08-19T10:47:00.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4220717584}) 2024-08-19T10:47:00.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2488587795}) 2024-08-19T10:47:00.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3055941831}) 2024-08-19T10:47:00.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2488587795}) 2024-08-19T10:47:00.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=47721996}) 2024-08-19T10:47:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c2975640 1 -- 172.21.15.89:0/461816742 shutdown_connections 2024-08-19T10:47:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c2975640 1 --2- 172.21.15.89:0/461816742 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f18bc0a6d30 0x7f18bc13d920 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c2975640 1 -- 172.21.15.89:0/461816742 >> 172.21.15.89:0/461816742 conn(0x7f18bc069a80 msgr2=0x7f18bc0ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c2975640 1 -- 172.21.15.89:0/461816742 shutdown_connections 2024-08-19T10:47:00.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f18c2975640 1 -- 172.21.15.89:0/461816742 wait complete. 2024-08-19T10:47:00.547 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.547 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3145636597}) 2024-08-19T10:47:00.547 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3931488025}) 2024-08-19T10:47:00.547 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.547 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3145636597}) 2024-08-19T10:47:00.547 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2949666421}) 2024-08-19T10:47:00.547 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.547 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3145636597}) 2024-08-19T10:47:00.547 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2110478948}) 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3145636597}) 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2937269183}) 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3145636597}) 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3696972480}) 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3145636597}) 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3455812595}) 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3145636597}) 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1408724449}) 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3145636597}) 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.464+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2811690555}) 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=90427588}) 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4061621193}) 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=90427588}) 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=505671470}) 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=90427588}) 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=393450466}) 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=90427588}) 2024-08-19T10:47:00.549 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=772189199}) 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=90427588}) 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=798381469}) 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=90427588}) 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=130327425}) 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=90427588}) 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=74906099}) 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=90427588}) 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.468+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1237020826}) 2024-08-19T10:47:00.550 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=764908707}) 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1102465421}) 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=764908707}) 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3963954716}) 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=764908707}) 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3371391924}) 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=764908707}) 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3287162271}) 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=764908707}) 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2967753602}) 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=764908707}) 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=873896346}) 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=764908707}) 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=395711159}) 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=764908707}) 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2826454511}) 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4011510653}) 2024-08-19T10:47:00.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=367982810}) 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4011510653}) 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=776053535}) 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4011510653}) 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2728219760}) 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4011510653}) 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=33249766}) 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4011510653}) 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3114221465}) 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4011510653}) 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3363275127}) 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4011510653}) 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2444322169}) 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4011510653}) 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.472+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1471557692}) 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3621417777}) 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1323710125}) 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3621417777}) 2024-08-19T10:47:00.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2295492560}) 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3621417777}) 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2358127962}) 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3621417777}) 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2209713404}) 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3621417777}) 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2830143152}) 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3621417777}) 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1472580424}) 2024-08-19T10:47:00.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3621417777}) 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2647407685}) 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3621417777}) 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.476+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2572377165}) 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=511517862}) 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2698450520}) 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=511517862}) 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1259916485}) 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.556 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=511517862}) 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3364162149}) 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=511517862}) 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3370326994}) 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=511517862}) 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1117608741}) 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=511517862}) 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2060973400}) 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=511517862}) 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=191931726}) 2024-08-19T10:47:00.557 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=511517862}) 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3493996221}) 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852505406}) 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3230751530}) 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852505406}) 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=45675254}) 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852505406}) 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3898723045}) 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.558 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852505406}) 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=661453282}) 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852505406}) 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2157947724}) 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852505406}) 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3307744603}) 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852505406}) 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2656384958}) 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852505406}) 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.480+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1711265746}) 2024-08-19T10:47:00.559 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3229805358}) 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1954710088}) 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3229805358}) 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1288325730}) 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3229805358}) 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3377488793}) 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3229805358}) 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2613009360}) 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.560 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3229805358}) 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3899530908}) 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3229805358}) 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1764707084}) 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3229805358}) 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2915223697}) 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3229805358}) 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2329708012}) 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2321226853}) 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3027587508}) 2024-08-19T10:47:00.561 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.562 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2321226853}) 2024-08-19T10:47:00.562 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=684877010}) 2024-08-19T10:47:00.562 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.562 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2321226853}) 2024-08-19T10:47:00.562 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2121117414}) 2024-08-19T10:47:00.562 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.562 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2321226853}) 2024-08-19T10:47:00.562 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1834204161}) 2024-08-19T10:47:00.562 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.562 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2321226853}) 2024-08-19T10:47:00.562 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2303798480}) 2024-08-19T10:47:00.563 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.563 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2321226853}) 2024-08-19T10:47:00.563 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4247924906}) 2024-08-19T10:47:00.563 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.563 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2321226853}) 2024-08-19T10:47:00.563 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4265562767}) 2024-08-19T10:47:00.563 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.563 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2321226853}) 2024-08-19T10:47:00.563 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.484+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=359863829}) 2024-08-19T10:47:00.564 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:47:00.565 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['i'] 2024-08-19T10:47:00.565 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a', 'g'] 2024-08-19T10:47:00.565 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-08-19T10:47:00.565 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-08-19T10:47:00.565 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-08-19T10:47:00.565 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:47:00.579 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.579 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3896838719}) 2024-08-19T10:47:00.579 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2979524551}) 2024-08-19T10:47:00.579 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.579 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3896838719}) 2024-08-19T10:47:00.579 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=259654736}) 2024-08-19T10:47:00.579 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.579 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3896838719}) 2024-08-19T10:47:00.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2056875577}) 2024-08-19T10:47:00.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3896838719}) 2024-08-19T10:47:00.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=784869644}) 2024-08-19T10:47:00.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3896838719}) 2024-08-19T10:47:00.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3520528139}) 2024-08-19T10:47:00.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3896838719}) 2024-08-19T10:47:00.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=648028145}) 2024-08-19T10:47:00.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3896838719}) 2024-08-19T10:47:00.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2488146056}) 2024-08-19T10:47:00.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3896838719}) 2024-08-19T10:47:00.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.488+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3175430790}) 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=625773963}) 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=372345580}) 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=625773963}) 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4092185675}) 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=625773963}) 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2915965191}) 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=625773963}) 2024-08-19T10:47:00.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3742523076}) 2024-08-19T10:47:00.583 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.583 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=625773963}) 2024-08-19T10:47:00.583 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1963976720}) 2024-08-19T10:47:00.583 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.583 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=625773963}) 2024-08-19T10:47:00.583 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2157054208}) 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=625773963}) 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2936390219}) 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=625773963}) 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2979309848}) 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776716164}) 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4060690540}) 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776716164}) 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2604821966}) 2024-08-19T10:47:00.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.496+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776716164}) 2024-08-19T10:47:00.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=753235823}) 2024-08-19T10:47:00.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776716164}) 2024-08-19T10:47:00.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=797353287}) 2024-08-19T10:47:00.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776716164}) 2024-08-19T10:47:00.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2827614851}) 2024-08-19T10:47:00.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776716164}) 2024-08-19T10:47:00.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4030977499}) 2024-08-19T10:47:00.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776716164}) 2024-08-19T10:47:00.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=388866734}) 2024-08-19T10:47:00.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1776716164}) 2024-08-19T10:47:00.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=162136605}) 2024-08-19T10:47:00.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=64,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12} crc {logm=542686635,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=868745944,mon_config_key=2521338541}) 2024-08-19T10:47:00.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=67,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11} crc {logm=836505977,mdsmap=2043744197,mgr=471976499,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=2921253974,mon_config_key=751381896}) 2024-08-19T10:47:00.587 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.587 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=64,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12} crc {logm=542686635,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=868745944,mon_config_key=2521338541}) 2024-08-19T10:47:00.587 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=69,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=7} crc {logm=1703475358,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=3133019967,mon_config_key=1665870468}) 2024-08-19T10:47:00.587 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.588 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=64,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12} crc {logm=542686635,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=868745944,mon_config_key=2521338541}) 2024-08-19T10:47:00.588 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=68,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=8} crc {logm=2251402060,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=2581569907,mon_config_key=2611799687}) 2024-08-19T10:47:00.588 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.588 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=64,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12} crc {logm=542686635,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=868745944,mon_config_key=2521338541}) 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=71,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5} crc {logm=3437171267,mdsmap=3140514009,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=1679846880,mon_config_key=2584219339}) 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=64,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12} crc {logm=542686635,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=868745944,mon_config_key=2521338541}) 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=70,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=6} crc {logm=3186053148,mdsmap=2043744197,mgr=544454686,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=2658756836,mon_config_key=1345839627}) 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=64,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12} crc {logm=542686635,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=868745944,mon_config_key=2521338541}) 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=68,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=10,mon_config_key=10} crc {logm=2510318748,mdsmap=2043744197,mgr=141335446,mgr_metadata=536852559,mgrstat=670447580,mon_config_key=1147720124}) 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=64,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12} crc {logm=542686635,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=868745944,mon_config_key=2521338541}) 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=68,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=9} crc {logm=4258932860,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=3185074579,mon_config_key=1705195315}) 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=64,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=12} crc {logm=542686635,mdsmap=2043744197,mgr=322451002,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=868745944,mon_config_key=2521338541}) 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.500+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=68,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=9} crc {logm=3491731012,mdsmap=3121089713,mgr=2558765597,mgr_command_descs=3328876062,mgr_metadata=536852559,mgrstat=1679846880,mon_config_key=2163398106}) 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=88} crc {monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=4281913764}) 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=1,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=88} crc {mon_config_key=4079344289,monmap=317198000,osd_metadata=2363964406,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=3279371078}) 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=88} crc {monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=4281913764}) 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=1,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=2,osdmap=88} crc {mon_config_key=4079344289,monmap=317198000,osd_metadata=2976096237,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=2735542444}) 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=88} crc {monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=4281913764}) 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mon_config_key=1,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=87} crc {mon_config_key=4079344290,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=1308395012}) 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=88} crc {monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=4281913764}) 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mon_config_key=1,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=87} crc {mon_config_key=4079344289,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=135464030}) 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=88} crc {monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=4281913764}) 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=1,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=87} crc {mon_config_key=4079344289,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=2465634389}) 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=88} crc {monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=4281913764}) 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=1,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=87} crc {mon_config_key=4079344289,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=687781427}) 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=88} crc {monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=4281913764}) 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mon_config_key=1,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=87} crc {mon_config_key=4079344289,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=3981754073}) 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=88} crc {monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=4281913764}) 2024-08-19T10:47:00.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.508+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mon_config_key=1,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=2,osdmap=87} crc {mon_config_key=4079344289,monmap=317198000,osd_metadata=1520236208,osd_pg_creating=4185911172,osd_snap=841190635,osdmap=1893902254}) 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4124755568}) 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3506014357}) 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4124755568}) 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=243627398}) 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4124755568}) 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=420116415}) 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4124755568}) 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=876345184}) 2024-08-19T10:47:00.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4124755568}) 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1909371581}) 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4124755568}) 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3506014357}) 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4124755568}) 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=242204536}) 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4124755568}) 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.516+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2145376284}) 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.532+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:47:00.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.532+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=6} crc {osdmap=2621755839}) 2024-08-19T10:47:00.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:47:00.532+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=6} crc {osdmap=371812806}) 2024-08-19T10:47:00.595 INFO:tasks.ceph.mon.i:Stopped 2024-08-19T10:47:00.595 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-08-19T10:47:00.595 INFO:tasks.ceph.mon.a:Sent signal 19 2024-08-19T10:47:00.595 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-08-19T10:47:00.595 INFO:tasks.ceph.mon.g:Sent signal 19 2024-08-19T10:47:00.595 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-08-19T10:47:15.596 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-08-19T10:47:15.597 INFO:tasks.ceph.mon.a:Sent signal 18 2024-08-19T10:47:15.597 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-08-19T10:47:15.597 INFO:tasks.ceph.mon.g:Sent signal 18 2024-08-19T10:47:15.598 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-08-19T10:47:15.598 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:47:15.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient: get_monmap_and_config 2024-08-19T10:47:15.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient: build_initial_monmap 2024-08-19T10:47:15.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient: monmap: 2024-08-19T10:47:15.739 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:15.739 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:15.739 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:47:15.742248+0000 2024-08-19T10:47:15.739 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:47:15.742248+0000 2024-08-19T10:47:15.739 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:15.739 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:15.739 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 1 Processor -- start 2024-08-19T10:47:15.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 1 -- start start 2024-08-19T10:47:15.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient: init 2024-08-19T10:47:15.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient: _add_conns ranks=[2,7,8,3,9,5,6,0,4,10,1] 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient(hunting): picked mon.noname-b con 0x7fb7000a3d00 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7000a1a00 0x7fb7000a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient(hunting): picked mon.noname-i con 0x7fb7000a1a00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient(hunting): picked mon.noname-e con 0x7fb70009b1d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb70009d3d0 con 0x7fb7000a3d00 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb70006fd00 con 0x7fb7000a1a00 2024-08-19T10:47:15.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb7000a4b60 con 0x7fb70009b1d0 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb708300640 10 monclient(hunting): authenticate will time out at 1485.558767s 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70609c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70689d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70609c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:44810/0 (socket says 172.21.15.89:44810) 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70689d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:33518/0 (socket says 172.21.15.89:33518) 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 learned_addr learned my addr 172.21.15.89:0/2161998606 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70609c640 10 monclient(hunting): get_auth_request con 0x7fb7000a3d00 auth_method 0 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70609c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70609c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:15.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70609c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:15.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70689d640 10 monclient(hunting): get_auth_request con 0x7fb70009b1d0 auth_method 0 2024-08-19T10:47:15.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7000a1a00 msgr2=0x7fb7000a4620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:47:15.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70689d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:15.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70689d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:15.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70689d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:15.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7000a1a00 0x7fb7000a4620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:15.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 msgr2=0x7fb7000a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:47:15.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 msgr2=0x7fb7000a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:15.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:15.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:15.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 msgr2=0x7fb70009b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:47:15.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 msgr2=0x7fb70009b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:15.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:15.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.740+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:15.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7000a1a00 msgr2=0x7fb7000a4620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7000a1a00 0x7fb7000a4620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70689d640 10 monclient(hunting): get_auth_request con 0x7fb70009b1d0 auth_method 0 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70689d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70689d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70689d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70609c640 10 monclient(hunting): get_auth_request con 0x7fb7000a3d00 auth_method 0 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70609c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70609c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70609c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:15.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 msgr2=0x7fb7000a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:47:15.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 msgr2=0x7fb7000a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:15.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:15.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 msgr2=0x7fb70009b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:47:15.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 msgr2=0x7fb70009b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:15.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:15.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:47:15.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:15.944+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:47:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:16.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70609c640 10 monclient(hunting): get_auth_request con 0x7fb7000a3d00 auth_method 0 2024-08-19T10:47:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70609c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70609c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70609c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70689d640 10 monclient(hunting): get_auth_request con 0x7fb70009b1d0 auth_method 0 2024-08-19T10:47:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70689d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70689d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70689d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 msgr2=0x7fb7000a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:47:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 msgr2=0x7fb7000a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:16.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:47:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 msgr2=0x7fb70009b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:47:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 msgr2=0x7fb70009b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7000a1a00 msgr2=0x7fb7000a4620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:47:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:47:16.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:16.348+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7000a1a00 0x7fb7000a4620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:47:17.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:17.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:17.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7000a1a00 msgr2=0x7fb7000a4620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:47:17.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70609c640 10 monclient(hunting): get_auth_request con 0x7fb7000a3d00 auth_method 0 2024-08-19T10:47:17.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70609c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70609c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70609c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7000a1a00 0x7fb7000a4620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70689d640 10 monclient(hunting): get_auth_request con 0x7fb70009b1d0 auth_method 0 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70689d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70689d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70689d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 msgr2=0x7fb7000a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 msgr2=0x7fb7000a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 msgr2=0x7fb70009b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 msgr2=0x7fb70009b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:17.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:17.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:47:17.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:17.152+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:47:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 10 monclient(hunting): tick 2024-08-19T10:47:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 monclient(hunting): continuing hunt 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 msgr2=0x7fb70009b610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb70009b1d0 0x7fb70009b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7000a1a00 msgr2=0x7fb7000a4620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7000a1a00 0x7fb7000a4620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 msgr2=0x7fb7000a40e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb7000a3d00 0x7fb7000a40e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 10 monclient: _add_conns ranks=[6,10,9,5,1,4,3,0] 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 10 monclient(hunting): picked mon.noname-d con 0x7fb6e4003890 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:47:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 10 monclient(hunting): picked mon.noname-f con 0x7fb6e40062a0 addr v2:172.21.15.190:3300/0 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 10 monclient(hunting): picked mon.noname-a con 0x7fb6e4008df0 addr v2:172.21.15.89:3300/0 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fb70006fd00 con 0x7fb6e4008df0 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb7000a4b60 con 0x7fb6e40062a0 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb6e400b970 con 0x7fb6e4003890 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb6f7fff640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70609c640 10 monclient(hunting): get_auth_request con 0x7fb6e4003890 auth_method 0 2024-08-19T10:47:18.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70609c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70609c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70609c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70689d640 10 monclient(hunting): get_auth_request con 0x7fb6e4008df0 auth_method 0 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70689d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70689d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70689d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70589b640 10 monclient(hunting): get_auth_request con 0x7fb6e40062a0 auth_method 0 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70589b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 msgr2=0x7fb6e4005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70589b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:18.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 msgr2=0x7fb6e4005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70589b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 msgr2=0x7fb6e400b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:47:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 msgr2=0x7fb6e400b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.739+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.743+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 msgr2=0x7fb6e4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:47:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.743+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 msgr2=0x7fb6e4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.743+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.743+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70689d640 10 monclient(hunting): get_auth_request con 0x7fb6e4008df0 auth_method 0 2024-08-19T10:47:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70689d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70689d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70689d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70609c640 10 monclient(hunting): get_auth_request con 0x7fb6e4003890 auth_method 0 2024-08-19T10:47:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70609c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70609c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70609c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:18.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 msgr2=0x7fb6e400b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:47:18.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 msgr2=0x7fb6e400b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:18.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:18.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 msgr2=0x7fb6e4005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:47:18.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 msgr2=0x7fb6e4005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:18.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:47:18.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:18.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.943+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:47:18.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.947+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.947+0000 7fb70589b640 10 monclient(hunting): get_auth_request con 0x7fb6e40062a0 auth_method 0 2024-08-19T10:47:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.947+0000 7fb70589b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.947+0000 7fb70589b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.947+0000 7fb70589b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.947+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 msgr2=0x7fb6e4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:47:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.947+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 msgr2=0x7fb6e4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.947+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:18.947+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:47:19.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:19.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70689d640 10 monclient(hunting): get_auth_request con 0x7fb6e4008df0 auth_method 0 2024-08-19T10:47:19.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70689d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:19.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70689d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:19.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70689d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:19.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:19.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70609c640 10 monclient(hunting): get_auth_request con 0x7fb6e4003890 auth_method 0 2024-08-19T10:47:19.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70609c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:19.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70609c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:19.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70609c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:19.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 msgr2=0x7fb6e400b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:47:19.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 msgr2=0x7fb6e400b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:19.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:19.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:47:19.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 msgr2=0x7fb6e4005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:47:19.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 msgr2=0x7fb6e4005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:19.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:19.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.347+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:47:19.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.351+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:19.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.351+0000 7fb70589b640 10 monclient(hunting): get_auth_request con 0x7fb6e40062a0 auth_method 0 2024-08-19T10:47:19.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.351+0000 7fb70589b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:19.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.351+0000 7fb70589b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:19.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.351+0000 7fb70589b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:19.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.351+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 msgr2=0x7fb6e4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:47:19.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.351+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 msgr2=0x7fb6e4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:19.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.351+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:19.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:19.351+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:47:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70689d640 10 monclient(hunting): get_auth_request con 0x7fb6e4008df0 auth_method 0 2024-08-19T10:47:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70689d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70689d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70689d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:20.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70609c640 10 monclient(hunting): get_auth_request con 0x7fb6e4003890 auth_method 0 2024-08-19T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70609c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70609c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70609c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 msgr2=0x7fb6e400b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 msgr2=0x7fb6e400b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 msgr2=0x7fb6e4005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70609c640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 msgr2=0x7fb6e4005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:20.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.151+0000 7fb70609c640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:47:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.155+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.155+0000 7fb70589b640 10 monclient(hunting): get_auth_request con 0x7fb6e40062a0 auth_method 0 2024-08-19T10:47:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.155+0000 7fb70589b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.155+0000 7fb70589b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.155+0000 7fb70589b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:47:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.155+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 msgr2=0x7fb6e4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:47:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.155+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 msgr2=0x7fb6e4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:47:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.155+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:47:20.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:20.155+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:47:21.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 10 monclient(hunting): tick 2024-08-19T10:47:21.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:47:21.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 monclient(hunting): continuing hunt 2024-08-19T10:47:21.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:47:21.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 msgr2=0x7fb6e4005cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:47:21.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb6e4003890 0x7fb6e4005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:21.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 msgr2=0x7fb6e4008730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.190:3300/0 conn(0x7fb6e40062a0 0x7fb6e4008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 msgr2=0x7fb6e400b2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 --2- 172.21.15.89:0/2161998606 >> v2:172.21.15.89:3300/0 conn(0x7fb6e4008df0 0x7fb6e400b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 reap_dead start 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 10 monclient: _add_conns ranks=[0,5,3,1,4] 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb70689d640 1 -- 172.21.15.89:0/2161998606 reap_dead start 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb6e4003cb0 addr v1:172.21.15.89:6789/0 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e400a9c0 0x7fb6e4008730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 10 monclient(hunting): picked mon.noname-h con 0x7fb6e400a9c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6e400adc0 0x7fb6e400b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 10 monclient(hunting): picked mon.noname-g con 0x7fb6e400adc0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb70006db50 con 0x7fb6e4003cb0 2024-08-19T10:47:21.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb70006fd00 con 0x7fb6e400adc0 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 1 -- 172.21.15.89:0/2161998606 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb7000a4b60 con 0x7fb6e400a9c0 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.739+0000 7fb6f7fff640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e400a9c0 0x7fb6e4008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6e400adc0 0x7fb6e400b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70689d640 10 monclient(hunting): get_auth_request con 0x7fb6e400adc0 auth_method 0 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70689d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70689d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70689d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 10 monclient(hunting): get_auth_request con 0x7fb6e400a9c0 auth_method 0 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:21.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70689d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70689d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70689d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 10 monclient(hunting): handle_auth_done global_id 6728 payload 274 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6e400adc0 msgr2=0x7fb6e400b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6e400adc0 0x7fb6e400b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> v1:172.21.15.89:6789/0 conn(0x7fb6e4003cb0 legacy=0x7fb6e40092a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 10 monclient: _finish_hunting 0 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 1 monclient: found mon.noname-h 2024-08-19T10:47:21.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:21.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:47:21.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e400f070 con 0x7fb6e400a9c0 2024-08-19T10:47:21.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.743+0000 7fb70689d640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6e400adc0 0x7fb6e400b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70589b640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e400a9c0 msgr2=0x7fb6e4008730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:21.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70589b640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e400a9c0 0x7fb6e4008730 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb700070680 tx=0x7fb6f001d850 comp rx=0 tx=0).ready entity=mon.5 client_cookie=c6d08af42f6662f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 1 -- 172.21.15.89:0/2161998606 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6f00ad890 con 0x7fb6e400a9c0 2024-08-19T10:47:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:47:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 10 monclient: dump: 2024-08-19T10:47:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:21.751 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 10 monclient: _finish_auth 0 2024-08-19T10:47:21.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 10 monclient: _check_auth_tickets 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 1 -- 172.21.15.89:0/2161998606 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f00b2070 con 0x7fb6e400a9c0 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 1 -- 172.21.15.89:0/2161998606 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6f00ac380 con 0x7fb6e400a9c0 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb70509a640 10 monclient: dump: 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:21.753 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 5 monclient: authenticate success, global_id 6728 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 10 monclient: get_monmap_and_config success 2024-08-19T10:47:21.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 4 set_mon_vals no callback set 2024-08-19T10:47:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 10 monclient: shutdown 2024-08-19T10:47:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 1 -- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e400a9c0 msgr2=0x7fb6e4008730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:21.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e400a9c0 0x7fb6e4008730 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb700070680 tx=0x7fb6f001d850 comp rx=0 tx=0).stop 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 1 -- 172.21.15.89:0/2161998606 shutdown_connections 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e400a9c0 0x7fb6e4008730 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 1 --2- 172.21.15.89:0/2161998606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb6e400adc0 0x7fb6e400b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 1 -- 172.21.15.89:0/2161998606 >> 172.21.15.89:0/2161998606 conn(0x7fb7000a4df0 msgr2=0x7fb7000a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 1 -- 172.21.15.89:0/2161998606 shutdown_connections 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.751+0000 7fb708300640 1 -- 172.21.15.89:0/2161998606 wait complete. 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient: build_initial_monmap 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient: monmap: 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:47:21.758 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 1 Processor -- start 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 1 -- start start 2024-08-19T10:47:21.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient: init 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient: _add_conns ranks=[5,1,2,6,8,7,0,3,4] 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e4003cb0 0x7fb7001cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient(hunting): picked mon.noname-h con 0x7fb6e4003cb0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6e400a9c0 0x7fb700137a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient(hunting): picked mon.noname-f con 0x7fb6e400a9c0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6e400adc0 0x7fb700137f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient(hunting): picked mon.noname-b con 0x7fb6e400adc0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb70009b5b0 con 0x7fb6e400a9c0 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb7000a5ba0 con 0x7fb6e400adc0 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:21.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb70009b730 con 0x7fb6e4003cb0 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb708300640 10 monclient(hunting): authenticate will time out at 1491.573978s 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6e400adc0 0x7fb700137f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6e400adc0 0x7fb700137f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35084/0 (socket says 172.21.15.89:35084) 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 1 -- 172.21.15.89:0/882567414 learned_addr learned my addr 172.21.15.89:0/882567414 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 10 monclient(hunting): get_auth_request con 0x7fb6e400adc0 auth_method 0 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70609c640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e4003cb0 0x7fb7001cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:21.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70589b640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6e400a9c0 0x7fb700137a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70589b640 10 monclient(hunting): get_auth_request con 0x7fb6e400a9c0 auth_method 0 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70589b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70589b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70589b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70609c640 10 monclient(hunting): get_auth_request con 0x7fb6e4003cb0 auth_method 0 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70609c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70609c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70609c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70589b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70589b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70589b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70609c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:21.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70609c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:21.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70609c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:21.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 10 monclient(hunting): handle_auth_done global_id 54254 payload 824 2024-08-19T10:47:21.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 1 -- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e4003cb0 msgr2=0x7fb7001cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:21.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e4003cb0 0x7fb7001cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:21.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 1 -- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6e400a9c0 msgr2=0x7fb700137a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:21.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6e400a9c0 0x7fb700137a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:21.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 10 monclient: _finish_hunting 0 2024-08-19T10:47:21.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 1 monclient: found mon.noname-b 2024-08-19T10:47:21.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:21.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:47:21.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.755+0000 7fb70689d640 1 -- 172.21.15.89:0/882567414 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7001321c0 con 0x7fb6e400adc0 2024-08-19T10:47:21.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb70689d640 1 -- 172.21.15.89:0/882567414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6e400adc0 msgr2=0x7fb700137f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:21.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb70689d640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6e400adc0 0x7fb700137f50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400b910 tx=0x7fb6f80187f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:21.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 1 -- 172.21.15.89:0/882567414 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6f8017a30 con 0x7fb6e400adc0 2024-08-19T10:47:21.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:21.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:47:21.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 10 monclient: dump: 2024-08-19T10:47:21.765 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:21.765 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:21.766 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 1 -- 172.21.15.89:0/882567414 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f8017bf0 con 0x7fb6e400adc0 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb708300640 5 monclient: authenticate success, global_id 54254 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb708300640 10 monclient: _renew_subs 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb708300640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb708300640 1 -- 172.21.15.89:0/882567414 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7000745a0 con 0x7fb6e400adc0 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 1 -- 172.21.15.89:0/882567414 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb6f8017da0 con 0x7fb6e400adc0 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:47:21.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb704899640 4 set_mon_vals no callback set 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 10 monclient: dump: 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb708300640 10 monclient: _renew_subs 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb708300640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb708300640 1 -- 172.21.15.89:0/882567414 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb700130e30 con 0x7fb6e400adc0 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 1 -- 172.21.15.89:0/882567414 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb6f8015040 con 0x7fb6e400adc0 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb708300640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb708300640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb708300640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb708300640 1 -- 172.21.15.89:0/882567414 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70009f460 con 0x7fb6e400adc0 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6e403e250 0x7fb6e4040710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb6f6ffd640 1 -- 172.21.15.89:0/882567414 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fb6f801e070 con 0x7fb6e400adc0 2024-08-19T10:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb70609c640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6e403e250 0x7fb6e4040710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:21.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.759+0000 7fb70609c640 10 monclient: get_auth_request con 0x7fb6e403e250 auth_method 0 2024-08-19T10:47:21.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.763+0000 7fb70609c640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6e403e250 0x7fb6e4040710 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40036e0 tx=0x7fb6fc01b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:21.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.767+0000 7fb6f6ffd640 1 -- 172.21.15.89:0/882567414 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb6f8028060 con 0x7fb6e400adc0 2024-08-19T10:47:21.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.767+0000 7fb6f6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:21.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:21.767+0000 7fb6f6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:22.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.007+0000 7fb708300640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:47:22.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.007+0000 7fb708300640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:47:22.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.007+0000 7fb708300640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:47:22.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.007+0000 7fb708300640 1 -- 172.21.15.89:0/882567414 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb70009ba60 con 0x7fb6e400adc0 2024-08-19T10:47:22.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.007+0000 7fb6f6ffd640 1 -- 172.21.15.89:0/882567414 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3186 (secure 0 0 0) 0x7fb6f804ec60 con 0x7fb6e400adc0 2024-08-19T10:47:22.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.007+0000 7fb6f6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:47:22.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.007+0000 7fb6f6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:22.007 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":90,"quorum":[0,1,2,3,4,5,6,8],"quorum_names":["a","f","b","g","c","h","d","e"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:47:22.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 -- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6e403e250 msgr2=0x7fb6e4040710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6e403e250 0x7fb6e4040710 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40036e0 tx=0x7fb6fc01b040 comp rx=0 tx=0).stop 2024-08-19T10:47:22.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 10 monclient: shutdown 2024-08-19T10:47:22.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:22.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 -- 172.21.15.89:0/882567414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6e400adc0 msgr2=0x7fb700137f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6e400adc0 0x7fb700137f50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400b910 tx=0x7fb6f80187f0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 -- 172.21.15.89:0/882567414 shutdown_connections 2024-08-19T10:47:22.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb6e403e250 0x7fb6e4040710 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb6e400adc0 0x7fb700137f50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6e400a9c0 0x7fb700137a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 --2- 172.21.15.89:0/882567414 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb6e4003cb0 0x7fb7001cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 -- 172.21.15.89:0/882567414 >> 172.21.15.89:0/882567414 conn(0x7fb7000a4df0 msgr2=0x7fb7001bf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:22.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 -- 172.21.15.89:0/882567414 shutdown_connections 2024-08-19T10:47:22.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.011+0000 7fb708300640 1 -- 172.21.15.89:0/882567414 wait complete. 2024-08-19T10:47:22.027 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-08-19T10:47:22.028 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-08-19T10:47:22.028 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:47:22.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient: get_monmap_and_config 2024-08-19T10:47:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient: build_initial_monmap 2024-08-19T10:47:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient: monmap: 2024-08-19T10:47:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:47:22.153479+0000 2024-08-19T10:47:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:47:22.153479+0000 2024-08-19T10:47:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:47:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:47:22.150 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:47:22.151 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:47:22.151 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 1 Processor -- start 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 1 -- start start 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient: init 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient: _add_conns ranks=[7,3,5,6,4,0,1,9,10,2,8] 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f02500a5270 0x7f02500a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient(hunting): picked mon.noname-i con 0x7f02500a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02500a6d30 0x7f02500a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient(hunting): picked mon.noname-g con 0x7f02500a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02501395b0 0x7f02501399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient(hunting): picked mon.noname-h con 0x7f02501395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:47:22.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f02500a7650 con 0x7f02500a6d30 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f02500a8710 con 0x7f02501395b0 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f02500a5a40 con 0x7f02500a5270 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f0254b1b640 10 monclient(hunting): authenticate will time out at 1491.970295s 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024e59c640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f02500a5270 msgr2=0x7f02500a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024e59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f02500a5270 0x7f02500a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02500a6d30 0x7f02500a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02501395b0 0x7f02501399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02501395b0 0x7f02501399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:53884/0 (socket says 172.21.15.89:53884) 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 1 -- 172.21.15.89:0/692918685 learned_addr learned my addr 172.21.15.89:0/692918685 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 10 monclient(hunting): get_auth_request con 0x7f02501395b0 auth_method 0 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024dd9b640 10 monclient(hunting): get_auth_request con 0x7f02500a6d30 auth_method 0 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 10 monclient(hunting): handle_auth_done global_id 6746 payload 274 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 1 -- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f02500a5270 msgr2=0x7f02500a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 1 --2- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f02500a5270 0x7f02500a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 1 -- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02500a6d30 msgr2=0x7f02500a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 1 --2- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02500a6d30 0x7f02500a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 1 monclient: found mon.noname-h 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024ed9d640 1 -- 172.21.15.89:0/692918685 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02500a81d0 con 0x7f02501395b0 2024-08-19T10:47:22.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.151+0000 7f024dd9b640 1 --2- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02500a6d30 0x7f02500a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:22.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 1 -- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02501395b0 msgr2=0x7f02501399d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 1 --2- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02501395b0 0x7f02501399d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f02480098a0 tx=0x7f024801c460 comp rx=0 tx=0).ready entity=mon.5 client_cookie=6b5c1a4e2a173aea server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 1 -- 172.21.15.89:0/692918685 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f02480b0070 con 0x7f02501395b0 2024-08-19T10:47:22.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:47:22.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 10 monclient: dump: 2024-08-19T10:47:22.159 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.159 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.159 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.159 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.159 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 10 monclient: _finish_auth 0 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:47:22.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 1 -- 172.21.15.89:0/692918685 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02480aa810 con 0x7f02501395b0 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 5 monclient: authenticate success, global_id 6746 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient: get_monmap_and_config success 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 4 set_mon_vals no callback set 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 1 -- 172.21.15.89:0/692918685 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f02480a9300 con 0x7f02501395b0 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024d59a640 10 monclient: dump: 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.161 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient: shutdown 2024-08-19T10:47:22.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 -- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02501395b0 msgr2=0x7f02501399d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 --2- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02501395b0 0x7f02501399d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f02480098a0 tx=0x7f024801c460 comp rx=0 tx=0).stop 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 -- 172.21.15.89:0/692918685 shutdown_connections 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 --2- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f02501395b0 0x7f02501399d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 --2- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02500a6d30 0x7f02500a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 --2- 172.21.15.89:0/692918685 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f02500a5270 0x7f02500a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 -- 172.21.15.89:0/692918685 >> 172.21.15.89:0/692918685 conn(0x7f0250069a80 msgr2=0x7f02500abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 -- 172.21.15.89:0/692918685 shutdown_connections 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 -- 172.21.15.89:0/692918685 wait complete. 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient: build_initial_monmap 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient: monmap: 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:47:22.163 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:22.164 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 Processor -- start 2024-08-19T10:47:22.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 -- start start 2024-08-19T10:47:22.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient: init 2024-08-19T10:47:22.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:22.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient: _add_conns ranks=[2,6,4,8,7,3,5,0,1] 2024-08-19T10:47:22.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02500a5270 0x7f02500a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient(hunting): picked mon.noname-b con 0x7f02500a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f02500a6d30 0x7f02500a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient(hunting): picked mon.noname-d con 0x7f02500a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02501395b0 0x7f025013e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient(hunting): picked mon.noname-c con 0x7f02501395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f02500a57d0 con 0x7f02500a5270 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f02500a5650 con 0x7f02501395b0 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f0250143230 con 0x7f02500a6d30 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f0254b1b640 10 monclient(hunting): authenticate will time out at 1491.978963s 2024-08-19T10:47:22.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024e59c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02500a5270 0x7f02500a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f02500a6d30 0x7f02500a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02501395b0 0x7f025013e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024e59c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02500a5270 0x7f02500a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35096/0 (socket says 172.21.15.89:35096) 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02501395b0 0x7f025013e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38086/0 (socket says 172.21.15.89:38086) 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f02500a6d30 0x7f02500a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:43590/0 (socket says 172.21.15.89:43590) 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024e59c640 1 -- 172.21.15.89:0/1456703703 learned_addr learned my addr 172.21.15.89:0/1456703703 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024e59c640 10 monclient(hunting): get_auth_request con 0x7f02500a5270 auth_method 0 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024dd9b640 10 monclient(hunting): get_auth_request con 0x7f02500a6d30 auth_method 0 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 10 monclient(hunting): get_auth_request con 0x7f02501395b0 auth_method 0 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 10 monclient(hunting): handle_auth_done global_id 44644 payload 824 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 1 -- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f02500a6d30 msgr2=0x7f02500a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f02500a6d30 0x7f02500a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 1 -- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02500a5270 msgr2=0x7f02500a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f02500a5270 0x7f02500a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 1 monclient: found mon.noname-c 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024ed9d640 1 -- 172.21.15.89:0/1456703703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0250135800 con 0x7f02501395b0 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.159+0000 7f024dd9b640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f02500a6d30 0x7f02500a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024ed9d640 1 -- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02501395b0 msgr2=0x7f025013e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024ed9d640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02501395b0 0x7f025013e430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f02480ab000 tx=0x7f0248019200 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f02480b0050 con 0x7f02501395b0 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 10 monclient: dump: 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.174 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 10 monclient: _finish_auth 0 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02480b1040 con 0x7f02501395b0 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f02480044b0 con 0x7f02501395b0 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 10 monclient: dump: 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.176 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0254b1b640 5 monclient: authenticate success, global_id 44644 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0254b1b640 10 monclient: _renew_subs 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024cd99640 4 set_mon_vals no callback set 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0254b1b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0254b1b640 1 -- 172.21.15.89:0/1456703703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025009ebf0 con 0x7f02501395b0 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0254b1b640 10 monclient: _renew_subs 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0254b1b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0254b1b640 1 -- 172.21.15.89:0/1456703703 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f02500a6a10 con 0x7f02501395b0 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0254b1b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0248004040 con 0x7f02501395b0 2024-08-19T10:47:22.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0254b1b640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013e970 0x7f025013ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0254b1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0254b1b640 1 -- 172.21.15.89:0/1456703703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0250133a80 con 0x7f025013e970 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024e59c640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013e970 0x7f025013ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024e59c640 10 monclient: get_auth_request con 0x7f025013e970 auth_method 0 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f023003bdf0 0x7f023003e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f02480bc070 con 0x7f02501395b0 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024e59c640 10 monclient(hunting): handle_auth_done global_id 6759 payload 824 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024e59c640 1 -- 172.21.15.89:0/1456703703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f025013f290 con 0x7f025013e970 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024dd9b640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f023003bdf0 0x7f023003e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.163+0000 7f024dd9b640 10 monclient: get_auth_request con 0x7f023003bdf0 auth_method 0 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f024e59c640 1 -- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013e970 msgr2=0x7f025013ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f024e59c640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013e970 0x7f025013ed50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0240007f60 tx=0x7f0240007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0240005750 con 0x7f025013e970 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f0246ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f0246ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f0246ffd640 10 monclient: dump: 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.179 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0240018070 con 0x7f025013e970 2024-08-19T10:47:22.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f0246ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f0246ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013e970 msgr2=0x7f025013ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f0246ffd640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013e970 0x7f025013ed50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0240007f60 tx=0x7f0240007f90 comp rx=0 tx=0).stop 2024-08-19T10:47:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013e970 msgr2=0x7f025013ed50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f0246ffd640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013e970 0x7f025013ed50 secure :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0240007f60 tx=0x7f0240007f90 comp rx=0 tx=0).stop 2024-08-19T10:47:22.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.167+0000 7f024dd9b640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f023003bdf0 0x7f023003e2b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0238009580 tx=0x7f0238009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013ab90 0x7f025013afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 1 -- 172.21.15.89:0/1456703703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f02500a7110 con 0x7f025013ab90 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013ab90 0x7f025013afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 10 monclient: get_auth_request con 0x7f025013ab90 auth_method 0 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 10 monclient(hunting): handle_auth_done global_id 6768 payload 824 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 1 -- 172.21.15.89:0/1456703703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0250064590 con 0x7f025013ab90 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 1 -- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013ab90 msgr2=0x7f025013afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024e59c640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013ab90 0x7f025013afa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0240007f60 tx=0x7f0240005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0240005a00 con 0x7f025013ab90 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0246ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0246ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0246ffd640 10 monclient: dump: 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.184 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9425 (secure 0 0 0) 0x7f0240005a00 con 0x7f025013ab90 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0246ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:47:22.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0246ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:22.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013ab90 msgr2=0x7f025013afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0246ffd640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013ab90 0x7f025013afa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0240007f60 tx=0x7f0240005d60 comp rx=0 tx=0).stop 2024-08-19T10:47:22.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0246ffd640 1 -- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013ab90 msgr2=0x7f025013afa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:22.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0246ffd640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f025013ab90 0x7f025013afa0 secure :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0240007f60 tx=0x7f0240005d60 comp rx=0 tx=0).stop 2024-08-19T10:47:22.186 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:47:22.186 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:22.186 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:22.186 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:47:22.186 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 90, 2024-08-19T10:47:22.186 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 571989, 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:22.187 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.188 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:47:22.190 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.191 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.192 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.193 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.195 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:47:22.198 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.201 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:47:22.203 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:47:22.204 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.206 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:47:22.207 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.208 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:47:22.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.209 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.209 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.209 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:47:22.209 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.209 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.209 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:47:22.209 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:47:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 1 -- 172.21.15.89:0/1456703703 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f023003bdf0 msgr2=0x7f023003e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f023003bdf0 0x7f023003e2b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0238009580 tx=0x7f0238009190 comp rx=0 tx=0).stop 2024-08-19T10:47:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 10 monclient: shutdown 2024-08-19T10:47:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 1 -- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02501395b0 msgr2=0x7f025013e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 1 --2- 172.21.15.89:0/1456703703 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f02501395b0 0x7f025013e430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f02480ab000 tx=0x7f0248019200 comp rx=0 tx=0).stop 2024-08-19T10:47:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024ed9d640 1 -- 172.21.15.89:0/1456703703 reap_dead start 2024-08-19T10:47:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f024ed9d640 1 -- 172.21.15.89:0/1456703703 reap_dead start 2024-08-19T10:47:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 1 -- 172.21.15.89:0/1456703703 shutdown_connections 2024-08-19T10:47:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 1 -- 172.21.15.89:0/1456703703 >> 172.21.15.89:0/1456703703 conn(0x7f0250069a80 msgr2=0x7f02501346d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 1 -- 172.21.15.89:0/1456703703 shutdown_connections 2024-08-19T10:47:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.175+0000 7f0254b1b640 1 -- 172.21.15.89:0/1456703703 wait complete. 2024-08-19T10:47:22.212 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:47:22.212 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-08-19T10:47:22.212 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:47:22.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient: get_monmap_and_config 2024-08-19T10:47:22.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient: build_initial_monmap 2024-08-19T10:47:22.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient: monmap: 2024-08-19T10:47:22.326 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:22.326 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:47:22.329951+0000 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:47:22.329951+0000 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:22.327 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:22.328 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:47:22.328 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:47:22.328 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 1 Processor -- start 2024-08-19T10:47:22.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 1 -- start start 2024-08-19T10:47:22.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient: init 2024-08-19T10:47:22.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:22.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient: _add_conns ranks=[3,5,7,8,4,6,10,1,2,0,9] 2024-08-19T10:47:22.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 0x7fb9080a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient(hunting): picked mon.noname-g con 0x7fb9080a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:47:22.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9080a7180 0x7fb9080a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient(hunting): picked mon.noname-h con 0x7fb9080a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:47:22.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb908139620 0x7fb908139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient(hunting): picked mon.noname-i con 0x7fb908139620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:47:22.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb9080a8190 con 0x7fb9080a6860 2024-08-19T10:47:22.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb9080a33d0 con 0x7fb9080a7180 2024-08-19T10:47:22.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb9080a2ec0 con 0x7fb908139620 2024-08-19T10:47:22.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:22.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90ee1b640 10 monclient(hunting): authenticate will time out at 1492.147297s 2024-08-19T10:47:22.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90d3b8640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb908139620 msgr2=0x7fb908139a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:47:22.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90d3b8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb908139620 0x7fb908139a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:22.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90cbb7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 0x7fb9080a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb907fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9080a7180 0x7fb9080a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb907fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9080a7180 0x7fb9080a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:53886/0 (socket says 172.21.15.89:53886) 2024-08-19T10:47:22.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90cbb7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 0x7fb9080a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:43622/0 (socket says 172.21.15.89:43622) 2024-08-19T10:47:22.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb907fff640 1 -- 172.21.15.89:0/2013359589 learned_addr learned my addr 172.21.15.89:0/2013359589 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb907fff640 10 monclient(hunting): get_auth_request con 0x7fb9080a7180 auth_method 0 2024-08-19T10:47:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb907fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb907fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb907fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90cbb7640 10 monclient(hunting): get_auth_request con 0x7fb9080a6860 auth_method 0 2024-08-19T10:47:22.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90cbb7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90cbb7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.327+0000 7fb90cbb7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb907fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb907fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb907fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_done global_id 35247 payload 274 2024-08-19T10:47:22.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 1 -- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb908139620 msgr2=0x7fb908139a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:47:22.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 1 --2- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb908139620 0x7fb908139a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 1 -- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9080a7180 msgr2=0x7fb9080a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:22.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 1 --2- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9080a7180 0x7fb9080a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 10 monclient: _finish_hunting 0 2024-08-19T10:47:22.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 1 monclient: found mon.noname-g 2024-08-19T10:47:22.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:22.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:47:22.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb90cbb7640 1 -- 172.21.15.89:0/2013359589 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9080a4cd0 con 0x7fb9080a6860 2024-08-19T10:47:22.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.331+0000 7fb907fff640 1 --2- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9080a7180 0x7fb9080a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:22.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90cbb7640 1 -- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 msgr2=0x7fb9080a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90cbb7640 1 --2- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 0x7fb9080a6c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009dc0 tx=0x7fb8f801c870 comp rx=0 tx=0).ready entity=mon.3 client_cookie=ce866302f7868c33 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 1 -- 172.21.15.89:0/2013359589 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb8f80ac070 con 0x7fb9080a6860 2024-08-19T10:47:22.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:47:22.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 10 monclient: dump: 2024-08-19T10:47:22.356 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.356 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.356 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.356 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.356 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.357 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.357 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.357 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.357 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.357 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.357 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.358 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.358 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.358 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.358 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.358 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 10 monclient: _finish_auth 0 2024-08-19T10:47:22.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 10 monclient: _check_auth_tickets 2024-08-19T10:47:22.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:22.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 1 -- 172.21.15.89:0/2013359589 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f80a68d0 con 0x7fb9080a6860 2024-08-19T10:47:22.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:22.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 1 -- 172.21.15.89:0/2013359589 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb8f80a7440 con 0x7fb9080a6860 2024-08-19T10:47:22.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 5 monclient: authenticate success, global_id 35247 2024-08-19T10:47:22.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient: get_monmap_and_config success 2024-08-19T10:47:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 4 set_mon_vals no callback set 2024-08-19T10:47:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:47:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb9077fe640 10 monclient: dump: 2024-08-19T10:47:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.360 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient: shutdown 2024-08-19T10:47:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 -- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 msgr2=0x7fb9080a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 --2- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 0x7fb9080a6c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009dc0 tx=0x7fb8f801c870 comp rx=0 tx=0).stop 2024-08-19T10:47:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 -- 172.21.15.89:0/2013359589 shutdown_connections 2024-08-19T10:47:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 --2- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9080a7180 0x7fb9080a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 --2- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb908139620 0x7fb908139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 --2- 172.21.15.89:0/2013359589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 0x7fb9080a6c40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 -- 172.21.15.89:0/2013359589 >> 172.21.15.89:0/2013359589 conn(0x7fb908074b40 msgr2=0x7fb9080abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:22.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 -- 172.21.15.89:0/2013359589 shutdown_connections 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 -- 172.21.15.89:0/2013359589 wait complete. 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient: build_initial_monmap 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient: monmap: 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:47:22.363 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 Processor -- start 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 -- start start 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient: init 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient: _add_conns ranks=[3,5,8,7,4,0,6,2,1] 2024-08-19T10:47:22.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 0x7fb90813eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient(hunting): picked mon.noname-g con 0x7fb9080a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9080a7180 0x7fb9080a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient(hunting): picked mon.noname-h con 0x7fb9080a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb908139620 0x7fb9080a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient(hunting): picked mon.noname-e con 0x7fb908139620 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb908143210 con 0x7fb9080a6860 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb908143090 con 0x7fb9080a7180 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb908143390 con 0x7fb908139620 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:22.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.335+0000 7fb90ee1b640 10 monclient(hunting): authenticate will time out at 1492.155936s 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb908139620 0x7fb9080a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb908139620 0x7fb9080a2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47044/0 (socket says 172.21.15.89:47044) 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 1 -- 172.21.15.89:0/778098107 learned_addr learned my addr 172.21.15.89:0/778098107 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 10 monclient(hunting): get_auth_request con 0x7fb908139620 auth_method 0 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb907fff640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9080a7180 0x7fb9080a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 0x7fb90813eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb907fff640 10 monclient(hunting): get_auth_request con 0x7fb9080a7180 auth_method 0 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb907fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb907fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb907fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 10 monclient(hunting): get_auth_request con 0x7fb9080a6860 auth_method 0 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 10 monclient(hunting): handle_auth_done global_id 6614 payload 824 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 1 -- 172.21.15.89:0/778098107 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9080a7180 msgr2=0x7fb9080a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb9080a7180 0x7fb9080a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 1 -- 172.21.15.89:0/778098107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 msgr2=0x7fb90813eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9080a6860 0x7fb90813eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 10 monclient: _finish_hunting 0 2024-08-19T10:47:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 1 monclient: found mon.noname-e 2024-08-19T10:47:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:47:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 1 -- 172.21.15.89:0/778098107 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90809f2e0 con 0x7fb908139620 2024-08-19T10:47:22.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 1 -- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb908139620 msgr2=0x7fb9080a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90d3b8640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb908139620 0x7fb9080a2600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb90000b490 tx=0x7fb90000b950 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb900017070 con 0x7fb908139620 2024-08-19T10:47:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:47:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 10 monclient: dump: 2024-08-19T10:47:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.369 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 10 monclient: _finish_auth 0 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 10 monclient: _check_auth_tickets 2024-08-19T10:47:22.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90ee1b640 5 monclient: authenticate success, global_id 6614 2024-08-19T10:47:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90ee1b640 10 monclient: _renew_subs 2024-08-19T10:47:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90ee1b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:47:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90ee1b640 1 -- 172.21.15.89:0/778098107 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90809f5c0 con 0x7fb908139620 2024-08-19T10:47:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb900003d60 con 0x7fb908139620 2024-08-19T10:47:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb900006540 con 0x7fb908139620 2024-08-19T10:47:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:47:22.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb9057fa640 10 monclient: dump: 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.372 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90ee1b640 10 monclient: _renew_subs 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90ee1b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90ee1b640 1 -- 172.21.15.89:0/778098107 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9081d1bd0 con 0x7fb908139620 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb906ffd640 4 set_mon_vals no callback set 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90ee1b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90ee1b640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809c9e0 0x7fb90809cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90ee1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90ee1b640 1 -- 172.21.15.89:0/778098107 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb9080a7560 con 0x7fb90809c9e0 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809c9e0 0x7fb90809cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 10 monclient: get_auth_request con 0x7fb90809c9e0 auth_method 0 2024-08-19T10:47:22.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.339+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb9000060d0 con 0x7fb908139620 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_done global_id 54272 payload 824 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb90cbb7640 1 -- 172.21.15.89:0/778098107 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb908064590 con 0x7fb90809c9e0 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb8f403be00 0x7fb8f403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fb90004e740 con 0x7fb908139620 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb907fff640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb8f403be00 0x7fb8f403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb907fff640 10 monclient: get_auth_request con 0x7fb8f403be00 auth_method 0 2024-08-19T10:47:22.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb90cbb7640 1 -- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809c9e0 msgr2=0x7fb90809cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb90cbb7640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809c9e0 0x7fb90809cdc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009d90 tx=0x7fb8f8019610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb8f80ac050 con 0x7fb90809c9e0 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 10 monclient: dump: 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.375 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb8f8004600 con 0x7fb90809c9e0 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809c9e0 msgr2=0x7fb90809cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809c9e0 0x7fb90809cdc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009d90 tx=0x7fb8f8019610 comp rx=0 tx=0).stop 2024-08-19T10:47:22.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809c9e0 msgr2=0x7fb90809cdc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb9057fa640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809c9e0 0x7fb90809cdc0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009d90 tx=0x7fb8f8019610 comp rx=0 tx=0).stop 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.343+0000 7fb907fff640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb8f403be00 0x7fb8f403e2c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb90809dbe0 tx=0x7fb8fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90ee1b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90ee1b640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809d300 0x7fb90813ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90ee1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90ee1b640 1 -- 172.21.15.89:0/778098107 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb9080a9540 con 0x7fb90809d300 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809d300 0x7fb90813ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 10 monclient: get_auth_request con 0x7fb90809d300 auth_method 0 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 10 monclient(hunting): handle_auth_done global_id 54281 payload 824 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 1 -- 172.21.15.89:0/778098107 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb90806cab0 con 0x7fb90809d300 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 1 -- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809d300 msgr2=0x7fb90813ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb90cbb7640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809d300 0x7fb90813ac20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb8f801c8b0 tx=0x7fb8f80a9000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb8f80c22c0 con 0x7fb90809d300 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb9057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb9057fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.351+0000 7fb9057fa640 10 monclient: dump: 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.378 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9255 (secure 0 0 0) 0x7fb8f80c22c0 con 0x7fb90809d300 2024-08-19T10:47:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb9057fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:47:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb9057fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809d300 msgr2=0x7fb90813ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb9057fa640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809d300 0x7fb90813ac20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb8f801c8b0 tx=0x7fb8f80a9000 comp rx=0 tx=0).stop 2024-08-19T10:47:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb9057fa640 1 -- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809d300 msgr2=0x7fb90813ac20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:22.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb9057fa640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb90809d300 0x7fb90813ac20 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb8f801c8b0 tx=0x7fb8f80a9000 comp rx=0 tx=0).stop 2024-08-19T10:47:22.380 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:47:22.380 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 90, 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 37747, 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:47:22.381 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.382 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.383 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:47:22.384 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.385 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:47:22.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:47:22.387 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:22.388 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.390 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:47:22.391 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:22.395 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.397 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:47:22.398 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:47:22.401 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:47:22.402 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.404 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:47:22.404 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:47:22.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90ee1b640 1 -- 172.21.15.89:0/778098107 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb8f403be00 msgr2=0x7fb8f403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90ee1b640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb8f403be00 0x7fb8f403e2c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb90809dbe0 tx=0x7fb8fc008040 comp rx=0 tx=0).stop 2024-08-19T10:47:22.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90ee1b640 10 monclient: shutdown 2024-08-19T10:47:22.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90ee1b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:22.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90ee1b640 1 -- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb908139620 msgr2=0x7fb9080a2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90d3b8640 1 -- 172.21.15.89:0/778098107 reap_dead start 2024-08-19T10:47:22.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90ee1b640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb908139620 0x7fb9080a2600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb90000b490 tx=0x7fb90000b950 comp rx=0 tx=0).stop 2024-08-19T10:47:22.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90ee1b640 1 -- 172.21.15.89:0/778098107 shutdown_connections 2024-08-19T10:47:22.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90ee1b640 1 --2- 172.21.15.89:0/778098107 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb908139620 0x7fb9080a2600 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90ee1b640 1 -- 172.21.15.89:0/778098107 >> 172.21.15.89:0/778098107 conn(0x7fb908074b40 msgr2=0x7fb9080ab3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:22.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90ee1b640 1 -- 172.21.15.89:0/778098107 shutdown_connections 2024-08-19T10:47:22.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.355+0000 7fb90ee1b640 1 -- 172.21.15.89:0/778098107 wait complete. 2024-08-19T10:47:22.406 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:47:22.406 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-08-19T10:47:22.407 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:47:22.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient: get_monmap_and_config 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient: build_initial_monmap 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient: monmap: 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:47:22.529273+0000 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:47:22.529273+0000 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:22.526 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:22.527 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:22.527 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:22.527 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:22.527 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:22.527 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:22.527 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:47:22.527 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:47:22.527 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 1 Processor -- start 2024-08-19T10:47:22.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 1 -- start start 2024-08-19T10:47:22.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient: init 2024-08-19T10:47:22.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient: _add_conns ranks=[8,1,2,3,9,7,4,6,5,10,0] 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4006c930 0x7f5b4006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient(hunting): picked mon.noname-e con 0x7f5b4006c930 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5b4006d250 addr v1:172.21.15.190:6789/0 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b40137c80 0x7f5b4013f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient(hunting): picked mon.noname-b con 0x7f5b40137c80 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b400a3070 con 0x7f5b4006d250 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f5b400a4a50 con 0x7f5b40137c80 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f5b400a3be0 con 0x7f5b4006c930 2024-08-19T10:47:22.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b47d74640 10 monclient(hunting): authenticate will time out at 1492.345947s 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b45b10640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4006c930 0x7f5b4006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b45b10640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4006c930 0x7f5b4006cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47056/0 (socket says 172.21.15.89:47056) 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b45b10640 1 -- 172.21.15.89:0/1349483595 learned_addr learned my addr 172.21.15.89:0/1349483595 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b45b10640 10 monclient(hunting): get_auth_request con 0x7f5b4006c930 auth_method 0 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 1 --2- 172.21.15.89:0/1349483595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b40137c80 0x7f5b4013f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b45b10640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b45b10640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b45b10640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 10 monclient(hunting): get_auth_request con 0x7f5b40137c80 auth_method 0 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b45b10640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b45b10640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b45b10640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 10 monclient(hunting): handle_auth_done global_id 54290 payload 274 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 1 -- 172.21.15.89:0/1349483595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4006c930 msgr2=0x7f5b4006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 1 --2- 172.21.15.89:0/1349483595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4006c930 0x7f5b4006cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 1 -- 172.21.15.89:0/1349483595 >> v1:172.21.15.190:6789/0 conn(0x7f5b4006d250 legacy=0x7f5b40137570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 10 monclient: _finish_hunting 0 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 1 monclient: found mon.noname-b 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:22.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:47:22.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.527+0000 7f5b46311640 1 -- 172.21.15.89:0/1349483595 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b400a4f00 con 0x7f5b40137c80 2024-08-19T10:47:22.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b46311640 1 -- 172.21.15.89:0/1349483595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b40137c80 msgr2=0x7f5b4013f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b46311640 1 --2- 172.21.15.89:0/1349483595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b40137c80 0x7f5b4013f7d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5b300097f0 tx=0x7f5b3001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1dca753d36c2482 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 1 -- 172.21.15.89:0/1349483595 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5b300b0070 con 0x7f5b40137c80 2024-08-19T10:47:22.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:47:22.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 10 monclient: dump: 2024-08-19T10:47:22.536 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.537 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 10 monclient: _finish_auth 0 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 10 monclient: _check_auth_tickets 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 1 -- 172.21.15.89:0/1349483595 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b300aa810 con 0x7f5b40137c80 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 1 -- 172.21.15.89:0/1349483595 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5b300a9300 con 0x7f5b40137c80 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b44b0e640 10 monclient: dump: 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.538 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.539 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 5 monclient: authenticate success, global_id 54290 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 10 monclient: get_monmap_and_config success 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 4 set_mon_vals no callback set 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 10 monclient: shutdown 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 1 -- 172.21.15.89:0/1349483595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b40137c80 msgr2=0x7f5b4013f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 1 --2- 172.21.15.89:0/1349483595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b40137c80 0x7f5b4013f7d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5b300097f0 tx=0x7f5b3001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 1 -- 172.21.15.89:0/1349483595 shutdown_connections 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 1 --2- 172.21.15.89:0/1349483595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b40137c80 0x7f5b4013f7d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 1 --2- 172.21.15.89:0/1349483595 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b4006c930 0x7f5b4006cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 1 -- 172.21.15.89:0/1349483595 >> 172.21.15.89:0/1349483595 conn(0x7f5b4006de40 msgr2=0x7f5b4006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 1 -- 172.21.15.89:0/1349483595 shutdown_connections 2024-08-19T10:47:22.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.531+0000 7f5b47d74640 1 -- 172.21.15.89:0/1349483595 wait complete. 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient: build_initial_monmap 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient: monmap: 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:22.541 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 1 Processor -- start 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 1 -- start start 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient: init 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient: _add_conns ranks=[7,2,8,6,1,3,0,5,4] 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4006c930 0x7f5b400a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient(hunting): picked mon.noname-i con 0x7f5b4006c930 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:47:22.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4006d250 0x7f5b400a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient(hunting): picked mon.noname-b con 0x7f5b4006d250 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b40137c80 0x7f5b400a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient(hunting): picked mon.noname-e con 0x7f5b40137c80 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f5b4009fdb0 con 0x7f5b4006d250 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5b4009fc30 con 0x7f5b4006c930 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f5b40142530 con 0x7f5b40137c80 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b47d74640 10 monclient(hunting): authenticate will time out at 1492.353236s 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b40137c80 0x7f5b400a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b4530f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4006d250 0x7f5b400a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b45b10640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4006c930 msgr2=0x7f5b400a02a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b40137c80 0x7f5b400a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47066/0 (socket says 172.21.15.89:47066) 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 1 -- 172.21.15.89:0/183988004 learned_addr learned my addr 172.21.15.89:0/183988004 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 10 monclient(hunting): get_auth_request con 0x7f5b40137c80 auth_method 0 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b45b10640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4006c930 0x7f5b400a02a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b4530f640 10 monclient(hunting): get_auth_request con 0x7f5b4006d250 auth_method 0 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b4530f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b4530f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b4530f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b4530f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b4530f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b4530f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 10 monclient(hunting): handle_auth_done global_id 6632 payload 824 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 1 -- 172.21.15.89:0/183988004 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4006c930 msgr2=0x7f5b400a02a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5b4006c930 0x7f5b400a02a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 1 -- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4006d250 msgr2=0x7f5b400a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5b4006d250 0x7f5b400a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 10 monclient: _finish_hunting 0 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 1 monclient: found mon.noname-e 2024-08-19T10:47:22.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:22.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:47:22.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.535+0000 7f5b46311640 1 -- 172.21.15.89:0/183988004 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b40134220 con 0x7f5b40137c80 2024-08-19T10:47:22.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b46311640 1 -- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b40137c80 msgr2=0x7f5b400a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b46311640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b40137c80 0x7f5b400a0d20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5b3001c880 tx=0x7f5b30019140 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5b300b0070 con 0x7f5b40137c80 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: dump: 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.551 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:22.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b47d74640 5 monclient: authenticate success, global_id 6632 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b300b1040 con 0x7f5b40137c80 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b47d74640 10 monclient: _renew_subs 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b47d74640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b47d74640 1 -- 172.21.15.89:0/183988004 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4013f920 con 0x7f5b40137c80 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5b300aae00 con 0x7f5b40137c80 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: dump: 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.553 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.554 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3ffff640 4 set_mon_vals no callback set 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b47d74640 10 monclient: _renew_subs 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b47d74640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b47d74640 1 -- 172.21.15.89:0/183988004 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4013ebd0 con 0x7f5b40137c80 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b47d74640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b47d74640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b400a1680 0x7f5b40133660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b47d74640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b47d74640 1 -- 172.21.15.89:0/183988004 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f5b400a61b0 con 0x7f5b400a1680 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b400a1680 0x7f5b40133660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 10 monclient: get_auth_request con 0x7f5b400a1680 auth_method 0 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 10 monclient(hunting): handle_auth_done global_id 44653 payload 824 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 1 -- 172.21.15.89:0/183988004 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b400a4330 con 0x7f5b400a1680 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5b30004240 con 0x7f5b40137c80 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5b2c03be00 0x7f5b2c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f5b300e8eb0 con 0x7f5b40137c80 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b4530f640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5b2c03be00 0x7f5b2c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b4530f640 10 monclient: get_auth_request con 0x7f5b2c03be00 auth_method 0 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 1 -- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b400a1680 msgr2=0x7f5b40133660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b45b10640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b400a1680 0x7f5b40133660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b38006c90 tx=0x7f5b3800ed40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5b38017070 con 0x7f5b400a1680 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.539+0000 7f5b3e7fc640 10 monclient: dump: 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.557 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.543+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5b3800cb30 con 0x7f5b400a1680 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.543+0000 7f5b3e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.543+0000 7f5b3e7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.543+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b400a1680 msgr2=0x7f5b40133660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.543+0000 7f5b3e7fc640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b400a1680 0x7f5b40133660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b38006c90 tx=0x7f5b3800ed40 comp rx=0 tx=0).stop 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.543+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b400a1680 msgr2=0x7f5b40133660 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:22.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.543+0000 7f5b3e7fc640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b400a1680 0x7f5b40133660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.543+0000 7f5b4530f640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5b2c03be00 0x7f5b2c03e2c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5b4013f7e0 tx=0x7f5b34003dd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b47d74640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b47d74640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4002d1d0 0x7f5b4002d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b47d74640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b47d74640 1 -- 172.21.15.89:0/183988004 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f5b4006cd10 con 0x7f5b4002d1d0 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b45b10640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4002d1d0 0x7f5b4002d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b45b10640 10 monclient: get_auth_request con 0x7f5b4002d1d0 auth_method 0 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b45b10640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b45b10640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b45b10640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b45b10640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b45b10640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b45b10640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b45b10640 10 monclient(hunting): handle_auth_done global_id 44662 payload 824 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.547+0000 7f5b45b10640 1 -- 172.21.15.89:0/183988004 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5b40064590 con 0x7f5b4002d1d0 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b45b10640 1 -- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4002d1d0 msgr2=0x7f5b4002d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b45b10640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4002d1d0 0x7f5b4002d5e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b38018000 tx=0x7f5b38004740 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5b38017030 con 0x7f5b4002d1d0 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b3e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b3e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b3e7fc640 10 monclient: dump: 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.560 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9256 (secure 0 0 0) 0x7f5b38004050 con 0x7f5b4002d1d0 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b3e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:47:22.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b3e7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:22.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4002d1d0 msgr2=0x7f5b4002d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b3e7fc640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4002d1d0 0x7f5b4002d5e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b38018000 tx=0x7f5b38004740 comp rx=0 tx=0).stop 2024-08-19T10:47:22.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b3e7fc640 1 -- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4002d1d0 msgr2=0x7f5b4002d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:22.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b3e7fc640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5b4002d1d0 0x7f5b4002d5e0 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b38018000 tx=0x7f5b38004740 comp rx=0 tx=0).stop 2024-08-19T10:47:22.562 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:47:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:47:22.562 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 90, 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 165678, 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.563 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.564 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.565 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:22.566 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.567 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.568 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:47:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:47:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.569 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.570 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:47:22.575 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.576 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.577 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.578 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.579 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.581 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.582 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.585 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:47:22.586 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:47:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b47d74640 1 -- 172.21.15.89:0/183988004 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5b2c03be00 msgr2=0x7f5b2c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b47d74640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5b2c03be00 0x7f5b2c03e2c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5b4013f7e0 tx=0x7f5b34003dd0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b47d74640 10 monclient: shutdown 2024-08-19T10:47:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b47d74640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b47d74640 1 -- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b40137c80 msgr2=0x7f5b400a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b47d74640 1 --2- 172.21.15.89:0/183988004 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5b40137c80 0x7f5b400a0d20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5b3001c880 tx=0x7f5b30019140 comp rx=0 tx=0).stop 2024-08-19T10:47:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b46311640 1 -- 172.21.15.89:0/183988004 reap_dead start 2024-08-19T10:47:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b46311640 1 -- 172.21.15.89:0/183988004 reap_dead start 2024-08-19T10:47:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b47d74640 1 -- 172.21.15.89:0/183988004 shutdown_connections 2024-08-19T10:47:22.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b47d74640 1 -- 172.21.15.89:0/183988004 >> 172.21.15.89:0/183988004 conn(0x7f5b4006de40 msgr2=0x7f5b40072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b47d74640 1 -- 172.21.15.89:0/183988004 shutdown_connections 2024-08-19T10:47:22.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.551+0000 7f5b47d74640 1 -- 172.21.15.89:0/183988004 wait complete. 2024-08-19T10:47:22.589 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:47:22.589 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-08-19T10:47:22.589 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:47:22.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient: get_monmap_and_config 2024-08-19T10:47:22.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient: build_initial_monmap 2024-08-19T10:47:22.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient: monmap: 2024-08-19T10:47:22.706 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:22.706 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:47:22.709904+0000 2024-08-19T10:47:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:47:22.709904+0000 2024-08-19T10:47:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:47:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:47:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:22.707 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:22.708 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:22.708 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:22.708 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:22.708 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:22.708 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:47:22.708 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:47:22.708 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 1 Processor -- start 2024-08-19T10:47:22.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 1 -- start start 2024-08-19T10:47:22.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient: init 2024-08-19T10:47:22.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient: _add_conns ranks=[5,0,3,1,10,2,4,9,8,6,7] 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a6860 0x7f877c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient(hunting): picked mon.noname-h con 0x7f877c0a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f877c0a7180 addr v1:172.21.15.89:6789/0 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f877c0a7560 0x7f877c139d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient(hunting): picked mon.noname-g con 0x7f877c0a7560 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f877c0a33d0 con 0x7f877c0a7180 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f877c0a8190 con 0x7f877c0a7560 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f877c0a79f0 con 0x7f877c0a6860 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f87821c4640 10 monclient(hunting): authenticate will time out at 1492.526443s 2024-08-19T10:47:22.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877affd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f877c0a7180 0x7f877c139620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48238/0 (socket says 172.21.15.89:48238) 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877affd640 1 -- 172.21.15.89:0/1698601421 learned_addr learned my addr 172.21.15.89:0/1698601421 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877bfff640 1 --2- 172.21.15.89:0/1698601421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f877c0a7560 0x7f877c139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 1 --2- 172.21.15.89:0/1698601421 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a6860 0x7f877c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877bfff640 10 monclient(hunting): get_auth_request con 0x7f877c0a7560 auth_method 0 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 10 monclient(hunting): get_auth_request con 0x7f877c0a6860 auth_method 0 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 10 monclient(hunting): handle_auth_done global_id 6773 payload 274 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 1 -- 172.21.15.89:0/1698601421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f877c0a7560 msgr2=0x7f877c139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 1 --2- 172.21.15.89:0/1698601421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f877c0a7560 0x7f877c139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 1 -- 172.21.15.89:0/1698601421 >> v1:172.21.15.89:6789/0 conn(0x7f877c0a7180 legacy=0x7f877c139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 1 monclient: found mon.noname-h 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:22.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:47:22.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877b7fe640 1 -- 172.21.15.89:0/1698601421 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c0a2ec0 con 0x7f877c0a6860 2024-08-19T10:47:22.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.707+0000 7f877bfff640 1 --2- 172.21.15.89:0/1698601421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f877c0a7560 0x7f877c139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:22.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.711+0000 7f877b7fe640 1 -- 172.21.15.89:0/1698601421 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a6860 msgr2=0x7f877c0a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877b7fe640 1 --2- 172.21.15.89:0/1698601421 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a6860 0x7f877c0a6c40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f876c0097f0 tx=0x7f876c01c3a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=c56b92271ce79c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 1 -- 172.21.15.89:0/1698601421 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f876c0b0070 con 0x7f877c0a6860 2024-08-19T10:47:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:47:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 10 monclient: dump: 2024-08-19T10:47:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.716 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 1 -- 172.21.15.89:0/1698601421 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f876c0aa810 con 0x7f877c0a6860 2024-08-19T10:47:22.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 1 -- 172.21.15.89:0/1698601421 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f876c0aae30 con 0x7f877c0a6860 2024-08-19T10:47:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:47:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877a7fc640 10 monclient: dump: 2024-08-19T10:47:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.718 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.719 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 5 monclient: authenticate success, global_id 6773 2024-08-19T10:47:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient: get_monmap_and_config success 2024-08-19T10:47:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 4 set_mon_vals no callback set 2024-08-19T10:47:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient: shutdown 2024-08-19T10:47:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 -- 172.21.15.89:0/1698601421 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a6860 msgr2=0x7f877c0a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 --2- 172.21.15.89:0/1698601421 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a6860 0x7f877c0a6c40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f876c0097f0 tx=0x7f876c01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 -- 172.21.15.89:0/1698601421 shutdown_connections 2024-08-19T10:47:22.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 --2- 172.21.15.89:0/1698601421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f877c0a7560 0x7f877c139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 --2- 172.21.15.89:0/1698601421 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a6860 0x7f877c0a6c40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 -- 172.21.15.89:0/1698601421 >> 172.21.15.89:0/1698601421 conn(0x7f877c074b40 msgr2=0x7f877c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 -- 172.21.15.89:0/1698601421 shutdown_connections 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 -- 172.21.15.89:0/1698601421 wait complete. 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient: build_initial_monmap 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient: monmap: 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.721 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:47:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:47:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:22.722 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:22.723 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:22.723 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 Processor -- start 2024-08-19T10:47:22.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 -- start start 2024-08-19T10:47:22.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient: init 2024-08-19T10:47:22.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:22.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient: _add_conns ranks=[7,5,2,8,6,4,3,1,0] 2024-08-19T10:47:22.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f877c0a6860 0x7f877c138c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient(hunting): picked mon.noname-i con 0x7f877c0a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:47:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a7180 0x7f877c1391a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient(hunting): picked mon.noname-h con 0x7f877c0a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:47:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c0a7560 0x7f877c13e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient(hunting): picked mon.noname-b con 0x7f877c0a7560 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:47:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f877c13a3f0 con 0x7f877c0a7560 2024-08-19T10:47:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f877c13a270 con 0x7f877c0a7180 2024-08-19T10:47:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f877c1440a0 con 0x7f877c0a6860 2024-08-19T10:47:22.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f87821c4640 10 monclient(hunting): authenticate will time out at 1492.535119s 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877b7fe640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f877c0a6860 msgr2=0x7f877c138c60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877b7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f877c0a6860 0x7f877c138c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c0a7560 0x7f877c13e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c0a7560 0x7f877c13e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35138/0 (socket says 172.21.15.89:35138) 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 1 -- 172.21.15.89:0/3712634840 learned_addr learned my addr 172.21.15.89:0/3712634840 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 10 monclient(hunting): get_auth_request con 0x7f877c0a7560 auth_method 0 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877affd640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a7180 0x7f877c1391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877affd640 10 monclient(hunting): get_auth_request con 0x7f877c0a7180 auth_method 0 2024-08-19T10:47:22.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 10 monclient(hunting): handle_auth_done global_id 54308 payload 824 2024-08-19T10:47:22.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 1 -- 172.21.15.89:0/3712634840 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f877c0a6860 msgr2=0x7f877c138c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:47:22.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f877c0a6860 0x7f877c138c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 1 -- 172.21.15.89:0/3712634840 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a7180 msgr2=0x7f877c1391a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a7180 0x7f877c1391a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:47:22.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 1 monclient: found mon.noname-b 2024-08-19T10:47:22.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:22.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:47:22.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877bfff640 1 -- 172.21.15.89:0/3712634840 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c13fd40 con 0x7f877c0a7560 2024-08-19T10:47:22.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.715+0000 7f877affd640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f877c0a7180 0x7f877c1391a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:47:22.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877bfff640 1 -- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c0a7560 msgr2=0x7f877c13e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877bfff640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c0a7560 0x7f877c13e250 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f877400efc0 tx=0x7f877400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8774018030 con 0x7f877c0a7560 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 10 monclient: dump: 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.733 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 10 monclient: _finish_auth 0 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 10 monclient: _check_auth_tickets 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87740148f0 con 0x7f877c0a7560 2024-08-19T10:47:22.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87740133e0 con 0x7f877c0a7560 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 10 monclient: dump: 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.735 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8779ffb640 4 set_mon_vals no callback set 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f87821c4640 5 monclient: authenticate success, global_id 54308 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f87821c4640 10 monclient: _renew_subs 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f87821c4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f87821c4640 1 -- 172.21.15.89:0/3712634840 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877c0a2710 con 0x7f877c0a7560 2024-08-19T10:47:22.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f87821c4640 10 monclient: _renew_subs 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f87821c4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f87821c4640 1 -- 172.21.15.89:0/3712634840 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f877c1238b0 con 0x7f877c0a7560 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f87821c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f87821c4640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13e790 0x7f877c13eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f87821c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f87821c4640 1 -- 172.21.15.89:0/3712634840 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f877c0ac0a0 con 0x7f877c13e790 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877b7fe640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13e790 0x7f877c13eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877b7fe640 10 monclient: get_auth_request con 0x7f877c13e790 auth_method 0 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f877401b070 con 0x7f877c0a7560 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877b7fe640 10 monclient(hunting): handle_auth_done global_id 44664 payload 824 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f877b7fe640 1 -- 172.21.15.89:0/3712634840 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f877c0a23f0 con 0x7f877c13e790 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f876803be30 0x7f876803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.719+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f8774019070 con 0x7f877c0a7560 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f877affd640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f876803be30 0x7f876803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f877affd640 10 monclient: get_auth_request con 0x7f876803be30 auth_method 0 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f877b7fe640 1 -- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13e790 msgr2=0x7f877c13eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f877b7fe640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13e790 0x7f877c13eb70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f877c0a5460 tx=0x7f876c01abe0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f876c0b9020 con 0x7f877c13e790 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f8763fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f8763fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f8763fff640 10 monclient: dump: 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.739 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f876c0b1040 con 0x7f877c13e790 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f8763fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f8763fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13e790 msgr2=0x7f877c13eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f8763fff640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13e790 0x7f877c13eb70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f877c0a5460 tx=0x7f876c01abe0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13e790 msgr2=0x7f877c13eb70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:22.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f8763fff640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13e790 0x7f877c13eb70 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f877c0a5460 tx=0x7f876c01abe0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.723+0000 7f877affd640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f876803be30 0x7f876803e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8770007fd0 tx=0x7f8770009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:47:22.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13f0b0 0x7f877c13a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 1 -- 172.21.15.89:0/3712634840 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f877c064590 con 0x7f877c13f0b0 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13f0b0 0x7f877c13a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 10 monclient: get_auth_request con 0x7f877c13f0b0 auth_method 0 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 10 monclient(hunting): handle_auth_done global_id 44673 payload 824 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 1 -- 172.21.15.89:0/3712634840 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f877c0a6c40 con 0x7f877c13f0b0 2024-08-19T10:47:22.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 1 -- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13f0b0 msgr2=0x7f877c13a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877b7fe640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13f0b0 0x7f877c13a630 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f876c01c1d0 tx=0x7f876c019140 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f876c0b9040 con 0x7f877c13f0b0 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f8763fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f8763fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f8763fff640 10 monclient: dump: 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.744 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9256 (secure 0 0 0) 0x7f876c0b5400 con 0x7f877c13f0b0 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f8763fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f8763fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13f0b0 msgr2=0x7f877c13a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f8763fff640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13f0b0 0x7f877c13a630 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f876c01c1d0 tx=0x7f876c019140 comp rx=0 tx=0).stop 2024-08-19T10:47:22.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f8763fff640 1 -- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13f0b0 msgr2=0x7f877c13a630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:22.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f8763fff640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f877c13f0b0 0x7f877c13a630 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f876c01c1d0 tx=0x7f876c019140 comp rx=0 tx=0).stop 2024-08-19T10:47:22.746 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:47:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:47:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 90, 2024-08-19T10:47:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 165858, 2024-08-19T10:47:22.746 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:47:22.747 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.748 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.749 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:47:22.750 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.751 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:47:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:47:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:22.752 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.753 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.754 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:47:22.755 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.756 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.757 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.758 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.759 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:47:22.760 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:22.761 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.763 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:47:22.764 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.766 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.767 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.768 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:47:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:47:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.769 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:47:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.770 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:47:22.770 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:47:22.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 1 -- 172.21.15.89:0/3712634840 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f876803be30 msgr2=0x7f876803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f876803be30 0x7f876803e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8770007fd0 tx=0x7f8770009040 comp rx=0 tx=0).stop 2024-08-19T10:47:22.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 10 monclient: shutdown 2024-08-19T10:47:22.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:22.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 1 -- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c0a7560 msgr2=0x7f877c13e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f877bfff640 1 -- 172.21.15.89:0/3712634840 reap_dead start 2024-08-19T10:47:22.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c0a7560 0x7f877c13e250 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f877400efc0 tx=0x7f877400c490 comp rx=0 tx=0).stop 2024-08-19T10:47:22.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 1 -- 172.21.15.89:0/3712634840 shutdown_connections 2024-08-19T10:47:22.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 1 --2- 172.21.15.89:0/3712634840 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f877c0a7560 0x7f877c13e250 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 1 -- 172.21.15.89:0/3712634840 >> 172.21.15.89:0/3712634840 conn(0x7f877c074b40 msgr2=0x7f877c141100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:22.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 1 -- 172.21.15.89:0/3712634840 shutdown_connections 2024-08-19T10:47:22.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.731+0000 7f87821c4640 1 -- 172.21.15.89:0/3712634840 wait complete. 2024-08-19T10:47:22.773 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:47:22.773 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-08-19T10:47:22.773 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:47:22.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient: get_monmap_and_config 2024-08-19T10:47:22.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient: build_initial_monmap 2024-08-19T10:47:22.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient: monmap: 2024-08-19T10:47:22.891 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:22.891 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.891 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:47:22.894656+0000 2024-08-19T10:47:22.891 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:47:22.894656+0000 2024-08-19T10:47:22.891 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:22.891 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:47:22.892 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 1 Processor -- start 2024-08-19T10:47:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 1 -- start start 2024-08-19T10:47:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient: init 2024-08-19T10:47:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient: _add_conns ranks=[7,10,3,2,4,5,1,9,6,8,0] 2024-08-19T10:47:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fad2c0a5270 0x7fad2c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient(hunting): picked mon.noname-i con 0x7fad2c0a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:47:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fad2c0a6d30 0x7fad2c0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient(hunting): picked mon.noname-f con 0x7fad2c0a6d30 addr v2:172.21.15.190:3300/0 2024-08-19T10:47:22.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c1395b0 0x7fad2c139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient(hunting): picked mon.noname-g con 0x7fad2c1395b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fad2c0a7650 con 0x7fad2c0a6d30 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fad2c0a8710 con 0x7fad2c1395b0 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fad2c0a5a40 con 0x7fad2c0a5270 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad31577640 10 monclient(hunting): authenticate will time out at 1492.711244s 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2affd640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fad2c0a5270 msgr2=0x7fad2c0a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2affd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fad2c0a5270 0x7fad2c0a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad227fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fad2c0a6d30 0x7fad2c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2b7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c1395b0 0x7fad2c139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2b7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c1395b0 0x7fad2c139a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:43642/0 (socket says 172.21.15.89:43642) 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2b7fe640 1 -- 172.21.15.89:0/1930656117 learned_addr learned my addr 172.21.15.89:0/1930656117 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2b7fe640 10 monclient(hunting): get_auth_request con 0x7fad2c1395b0 auth_method 0 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad227fc640 10 monclient(hunting): get_auth_request con 0x7fad2c0a6d30 auth_method 0 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad227fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad227fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad227fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad227fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.891+0000 7fad227fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad227fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad2b7fe640 10 monclient(hunting): handle_auth_done global_id 35274 payload 274 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad2b7fe640 1 -- 172.21.15.89:0/1930656117 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fad2c0a5270 msgr2=0x7fad2c0a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad2b7fe640 1 --2- 172.21.15.89:0/1930656117 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fad2c0a5270 0x7fad2c0a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad2b7fe640 1 -- 172.21.15.89:0/1930656117 >> v2:172.21.15.190:3300/0 conn(0x7fad2c0a6d30 msgr2=0x7fad2c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad2b7fe640 1 --2- 172.21.15.89:0/1930656117 >> v2:172.21.15.190:3300/0 conn(0x7fad2c0a6d30 0x7fad2c0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad2b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad2b7fe640 1 monclient: found mon.noname-g 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad2b7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad2b7fe640 1 -- 172.21.15.89:0/1930656117 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2c0a81d0 con 0x7fad2c1395b0 2024-08-19T10:47:22.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.895+0000 7fad227fc640 1 --2- 172.21.15.89:0/1930656117 >> v2:172.21.15.190:3300/0 conn(0x7fad2c0a6d30 0x7fad2c0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2b7fe640 1 -- 172.21.15.89:0/1930656117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c1395b0 msgr2=0x7fad2c139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2b7fe640 1 --2- 172.21.15.89:0/1930656117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c1395b0 0x7fad2c139a40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fad24009e50 tx=0x7fad2401c910 comp rx=0 tx=0).ready entity=mon.3 client_cookie=9921782de42eafea server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 1 -- 172.21.15.89:0/1930656117 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fad240090d0 con 0x7fad2c1395b0 2024-08-19T10:47:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:47:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 10 monclient: dump: 2024-08-19T10:47:22.900 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.901 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 1 -- 172.21.15.89:0/1930656117 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad240a5bc0 con 0x7fad2c1395b0 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 1 -- 172.21.15.89:0/1930656117 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fad240a5db0 con 0x7fad2c1395b0 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad2a7fc640 10 monclient: dump: 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.902 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.903 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 5 monclient: authenticate success, global_id 35274 2024-08-19T10:47:22.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 10 monclient: get_monmap_and_config success 2024-08-19T10:47:22.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 4 set_mon_vals no callback set 2024-08-19T10:47:22.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 10 monclient: shutdown 2024-08-19T10:47:22.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:22.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 1 -- 172.21.15.89:0/1930656117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c1395b0 msgr2=0x7fad2c139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 1 --2- 172.21.15.89:0/1930656117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c1395b0 0x7fad2c139a40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fad24009e50 tx=0x7fad2401c910 comp rx=0 tx=0).stop 2024-08-19T10:47:22.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 1 -- 172.21.15.89:0/1930656117 shutdown_connections 2024-08-19T10:47:22.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 1 --2- 172.21.15.89:0/1930656117 >> v2:172.21.15.190:3300/0 conn(0x7fad2c0a6d30 0x7fad2c0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 1 --2- 172.21.15.89:0/1930656117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c1395b0 0x7fad2c139a40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 1 --2- 172.21.15.89:0/1930656117 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fad2c0a5270 0x7fad2c0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 1 -- 172.21.15.89:0/1930656117 >> 172.21.15.89:0/1930656117 conn(0x7fad2c069a80 msgr2=0x7fad2c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:22.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 1 -- 172.21.15.89:0/1930656117 shutdown_connections 2024-08-19T10:47:22.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 1 -- 172.21.15.89:0/1930656117 wait complete. 2024-08-19T10:47:22.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 10 monclient: build_initial_monmap 2024-08-19T10:47:22.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 10 monclient: monmap: 2024-08-19T10:47:22.910 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:22.911 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 1 Processor -- start 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 1 -- start start 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.899+0000 7fad31577640 10 monclient: init 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient: _add_conns ranks=[3,2,6,4,1,5,0,7,8] 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c0a5270 0x7fad2c1d1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient(hunting): picked mon.noname-g con 0x7fad2c0a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad2c0a6d30 0x7fad2c0a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient(hunting): picked mon.noname-b con 0x7fad2c0a6d30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad2c1395b0 0x7fad2c0a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient(hunting): picked mon.noname-d con 0x7fad2c1395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:47:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fad2c0a5650 con 0x7fad2c0a6d30 2024-08-19T10:47:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fad2c0a4a70 con 0x7fad2c0a5270 2024-08-19T10:47:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fad2c0a57d0 con 0x7fad2c1395b0 2024-08-19T10:47:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient(hunting): authenticate will time out at 1492.720465s 2024-08-19T10:47:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad2c1395b0 0x7fad2c0a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad2c1395b0 0x7fad2c0a2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:43618/0 (socket says 172.21.15.89:43618) 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 1 -- 172.21.15.89:0/1717911406 learned_addr learned my addr 172.21.15.89:0/1717911406 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 10 monclient(hunting): get_auth_request con 0x7fad2c1395b0 auth_method 0 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad227fc640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad2c0a6d30 0x7fad2c0a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad227fc640 10 monclient(hunting): get_auth_request con 0x7fad2c0a6d30 auth_method 0 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad227fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad227fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad227fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2affd640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c0a5270 0x7fad2c1d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2affd640 10 monclient(hunting): get_auth_request con 0x7fad2c0a5270 auth_method 0 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 10 monclient(hunting): handle_auth_done global_id 44682 payload 824 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 1 -- 172.21.15.89:0/1717911406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c0a5270 msgr2=0x7fad2c1d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad2c0a5270 0x7fad2c1d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 1 -- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad2c0a6d30 msgr2=0x7fad2c0a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad2c0a6d30 0x7fad2c0a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 1 monclient: found mon.noname-d 2024-08-19T10:47:22.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 1 -- 172.21.15.89:0/1717911406 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2c09f210 con 0x7fad2c1395b0 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad227fc640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fad2c0a6d30 0x7fad2c0a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 1 -- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad2c1395b0 msgr2=0x7fad2c0a2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad2b7fe640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad2c1395b0 0x7fad2c0a2480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad24009040 tx=0x7fad240196b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fad240bc020 con 0x7fad2c1395b0 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 10 monclient: dump: 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.916 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 10 monclient: _finish_auth 0 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 10 monclient: _check_auth_tickets 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad24003dc0 con 0x7fad2c1395b0 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fad240b7450 con 0x7fad2c1395b0 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 5 monclient: authenticate success, global_id 44682 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient: _renew_subs 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 1 -- 172.21.15.89:0/1717911406 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2c0a6880 con 0x7fad2c1395b0 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad29ffb640 4 set_mon_vals no callback set 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:47:22.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad23fff640 10 monclient: dump: 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.919 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient: _renew_subs 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.903+0000 7fad31577640 1 -- 172.21.15.89:0/1717911406 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fad2c0a0450 con 0x7fad2c1395b0 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad31577640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad31577640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09cdd0 0x7fad2c09d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad31577640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad31577640 1 -- 172.21.15.89:0/1717911406 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fad2c09fef0 con 0x7fad2c09cdd0 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09cdd0 0x7fad2c09d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 10 monclient: get_auth_request con 0x7fad2c09cdd0 auth_method 0 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fad240bf020 con 0x7fad2c1395b0 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 10 monclient(hunting): handle_auth_done global_id 6641 payload 824 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 1 -- 172.21.15.89:0/1717911406 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad2c140680 con 0x7fad2c09cdd0 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fad0c03be00 0x7fad0c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fad240b3040 con 0x7fad2c1395b0 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad227fc640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fad0c03be00 0x7fad0c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad227fc640 10 monclient: get_auth_request con 0x7fad0c03be00 auth_method 0 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 1 -- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09cdd0 msgr2=0x7fad2c09d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad2affd640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09cdd0 0x7fad2c09d1b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d700 tx=0x7fad1c00dbc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fad1c003bd0 con 0x7fad2c09cdd0 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 10 monclient: dump: 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.922 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fad1c017070 con 0x7fad2c09cdd0 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09cdd0 msgr2=0x7fad2c09d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09cdd0 0x7fad2c09d1b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d700 tx=0x7fad1c00dbc0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09cdd0 msgr2=0x7fad2c09d1b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:22.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.907+0000 7fad23fff640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09cdd0 0x7fad2c09d1b0 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d700 tx=0x7fad1c00dbc0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.911+0000 7fad227fc640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fad0c03be00 0x7fad0c03e2c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fad2c09e4d0 tx=0x7fad14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad31577640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:47:22.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad31577640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09d6f0 0x7fad2c13a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:22.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad31577640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:22.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad31577640 1 -- 172.21.15.89:0/1717911406 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fad2c0a7110 con 0x7fad2c09d6f0 2024-08-19T10:47:22.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad2affd640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09d6f0 0x7fad2c13a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad2affd640 10 monclient: get_auth_request con 0x7fad2c09d6f0 auth_method 0 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad2affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad2affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad2affd640 10 monclient(hunting): handle_auth_done global_id 6650 payload 824 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.915+0000 7fad2affd640 1 -- 172.21.15.89:0/1717911406 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fad2c064590 con 0x7fad2c09d6f0 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad2affd640 1 -- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09d6f0 msgr2=0x7fad2c13a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad2affd640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09d6f0 0x7fad2c13a9e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d700 tx=0x7fad1c0059e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:22.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fad1c01cd80 con 0x7fad2c09d6f0 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad23fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad23fff640 10 monclient: dump: 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:22.927 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9423 (secure 0 0 0) 0x7fad1c01cd80 con 0x7fad2c09d6f0 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad23fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad23fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09d6f0 msgr2=0x7fad2c13a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad23fff640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09d6f0 0x7fad2c13a9e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d700 tx=0x7fad1c0059e0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad23fff640 1 -- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09d6f0 msgr2=0x7fad2c13a9e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:22.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad23fff640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fad2c09d6f0 0x7fad2c13a9e0 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d700 tx=0x7fad1c0059e0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.929 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:47:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:47:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 90, 2024-08-19T10:47:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 572713, 2024-08-19T10:47:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:47:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:47:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:47:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:47:22.929 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.930 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.931 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:47:22.932 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:47:22.933 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:22.934 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:22.938 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.939 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:22.940 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.942 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.943 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:47:22.946 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:47:22.948 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:47:22.949 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:22.950 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.951 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:22.952 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:47:22.953 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad31577640 1 -- 172.21.15.89:0/1717911406 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fad0c03be00 msgr2=0x7fad0c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad31577640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fad0c03be00 0x7fad0c03e2c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fad2c09e4d0 tx=0x7fad14008040 comp rx=0 tx=0).stop 2024-08-19T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad31577640 10 monclient: shutdown 2024-08-19T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad31577640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad31577640 1 -- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad2c1395b0 msgr2=0x7fad2c0a2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad31577640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad2c1395b0 0x7fad2c0a2480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad24009040 tx=0x7fad240196b0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad2b7fe640 1 -- 172.21.15.89:0/1717911406 reap_dead start 2024-08-19T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad31577640 1 -- 172.21.15.89:0/1717911406 shutdown_connections 2024-08-19T10:47:22.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad31577640 1 --2- 172.21.15.89:0/1717911406 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fad2c1395b0 0x7fad2c0a2480 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:22.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad31577640 1 -- 172.21.15.89:0/1717911406 >> 172.21.15.89:0/1717911406 conn(0x7fad2c069a80 msgr2=0x7fad2c0ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:22.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad31577640 1 -- 172.21.15.89:0/1717911406 shutdown_connections 2024-08-19T10:47:22.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:22.919+0000 7fad31577640 1 -- 172.21.15.89:0/1717911406 wait complete. 2024-08-19T10:47:22.955 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:47:22.955 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-08-19T10:47:22.955 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:47:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient: get_monmap_and_config 2024-08-19T10:47:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient: build_initial_monmap 2024-08-19T10:47:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient: monmap: 2024-08-19T10:47:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:47:23.073604+0000 2024-08-19T10:47:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:47:23.073604+0000 2024-08-19T10:47:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.070 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:47:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:47:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:47:23.071 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:47:23.071 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 1 Processor -- start 2024-08-19T10:47:23.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 1 -- start start 2024-08-19T10:47:23.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient: init 2024-08-19T10:47:23.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient: _add_conns ranks=[3,10,7,5,4,9,8,2,6,0,1] 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740a3d70 0x7f95740a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient(hunting): picked mon.noname-g con 0x7f95740a3d70 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95740a4150 0x7f95740a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient(hunting): picked mon.noname-f con 0x7f95740a4150 addr v2:172.21.15.190:3300/0 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f957409b040 0x7f957409b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient(hunting): picked mon.noname-i con 0x7f957409b040 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f957406f8e0 con 0x7f95740a4150 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f95740a0780 con 0x7f95740a3d70 2024-08-19T10:47:23.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f957409fb40 con 0x7f957409b040 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9579fae640 10 monclient(hunting): authenticate will time out at 1492.890586s 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9573fff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f957409b040 msgr2=0x7f957409b4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9573fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f957409b040 0x7f957409b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95740a4150 0x7f95740a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f95737fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740a3d70 0x7f95740a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95740a4150 0x7f95740a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:44526/0 (socket says 172.21.15.89:44526) 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 1 -- 172.21.15.89:0/1669626550 learned_addr learned my addr 172.21.15.89:0/1669626550 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 10 monclient(hunting): get_auth_request con 0x7f95740a4150 auth_method 0 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f95737fe640 10 monclient(hunting): get_auth_request con 0x7f95740a3d70 auth_method 0 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f95737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f95737fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f95737fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f95737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f95737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f95737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 10 monclient(hunting): handle_auth_done global_id 44578 payload 274 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 1 -- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f957409b040 msgr2=0x7f957409b4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 1 --2- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f957409b040 0x7f957409b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 1 -- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740a3d70 msgr2=0x7f95740a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:23.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 1 --2- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740a3d70 0x7f95740a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:47:23.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 1 monclient: found mon.noname-f 2024-08-19T10:47:23.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:23.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:47:23.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f9572ffd640 1 -- 172.21.15.89:0/1669626550 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957409fe90 con 0x7f95740a4150 2024-08-19T10:47:23.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.071+0000 7f95737fe640 1 --2- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740a3d70 0x7f95740a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:23.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.075+0000 7f9572ffd640 1 -- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95740a4150 msgr2=0x7f95740a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.075+0000 7f9572ffd640 1 --2- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95740a4150 0x7f95740a4570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f955c0098d0 tx=0x7f955c01c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d40271fe892678f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 1 -- 172.21.15.89:0/1669626550 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f955c0b0070 con 0x7f95740a4150 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 10 monclient: dump: 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.079 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 10 monclient: _finish_auth 0 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 10 monclient: _check_auth_tickets 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 1 -- 172.21.15.89:0/1669626550 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f955c0aa810 con 0x7f95740a4150 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 1 -- 172.21.15.89:0/1669626550 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f955c0aae30 con 0x7f95740a4150 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 5 monclient: authenticate success, global_id 44578 2024-08-19T10:47:23.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient: get_monmap_and_config success 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 4 set_mon_vals no callback set 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95727fc640 10 monclient: dump: 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.081 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient: shutdown 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 -- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95740a4150 msgr2=0x7f95740a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 --2- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95740a4150 0x7f95740a4570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f955c0098d0 tx=0x7f955c01c4b0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 -- 172.21.15.89:0/1669626550 shutdown_connections 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 --2- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95740a4150 0x7f95740a4570 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 --2- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f957409b040 0x7f957409b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 --2- 172.21.15.89:0/1669626550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95740a3d70 0x7f95740a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 -- 172.21.15.89:0/1669626550 >> 172.21.15.89:0/1669626550 conn(0x7f95740a4e60 msgr2=0x7f95740a5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:23.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 -- 172.21.15.89:0/1669626550 shutdown_connections 2024-08-19T10:47:23.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 -- 172.21.15.89:0/1669626550 wait complete. 2024-08-19T10:47:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient: build_initial_monmap 2024-08-19T10:47:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient: monmap: 2024-08-19T10:47:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:47:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:47:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:47:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:47:23.087 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 Processor -- start 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 -- start start 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient: init 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:23.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient: _add_conns ranks=[0,2,8,6,3,5,4,1,7] 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957409b040 0x7f95741cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient(hunting): picked mon.noname-a con 0x7f957409b040 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95740a3d70 0x7f9574138a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient(hunting): picked mon.noname-b con 0x7f95740a3d70 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9574138f60 0x7f95741cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient(hunting): picked mon.noname-e con 0x7f9574138f60 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9574140f30 con 0x7f957409b040 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f95740a4ab0 con 0x7f95740a3d70 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f95741410b0 con 0x7f9574138f60 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:23.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9579fae640 10 monclient(hunting): authenticate will time out at 1492.899424s 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957409b040 0x7f95741cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9573fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9574138f60 0x7f95741cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957409b040 0x7f95741cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60926/0 (socket says 172.21.15.89:60926) 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 1 -- 172.21.15.89:0/3607036469 learned_addr learned my addr 172.21.15.89:0/3607036469 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 10 monclient(hunting): get_auth_request con 0x7f957409b040 auth_method 0 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9573fff640 10 monclient(hunting): get_auth_request con 0x7f9574138f60 auth_method 0 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9573fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9573fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9573fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9572ffd640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95740a3d70 0x7f9574138a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9572ffd640 10 monclient(hunting): get_auth_request con 0x7f95740a3d70 auth_method 0 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9572ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9572ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9572ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9573fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9573fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9573fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 10 monclient(hunting): handle_auth_done global_id 6777 payload 824 2024-08-19T10:47:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 1 -- 172.21.15.89:0/3607036469 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9574138f60 msgr2=0x7f95741cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9574138f60 0x7f95741cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 1 -- 172.21.15.89:0/3607036469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95740a3d70 msgr2=0x7f9574138a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.079+0000 7f95737fe640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f95740a3d70 0x7f9574138a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f95737fe640 10 monclient: _finish_hunting 0 2024-08-19T10:47:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f95737fe640 1 monclient: found mon.noname-a 2024-08-19T10:47:23.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f95737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:23.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f95737fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f95737fe640 1 -- 172.21.15.89:0/3607036469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95740a2880 con 0x7f957409b040 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9573fff640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9574138f60 0x7f95741cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f95737fe640 1 -- 172.21.15.89:0/3607036469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957409b040 msgr2=0x7f95741cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f95737fe640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957409b040 0x7f95741cc010 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f956c009ec0 tx=0x7f956c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f956c018020 con 0x7f957409b040 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 10 monclient: dump: 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.094 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 10 monclient: _finish_auth 0 2024-08-19T10:47:23.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c014b00 con 0x7f957409b040 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9579fae640 5 monclient: authenticate success, global_id 6777 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9579fae640 10 monclient: _renew_subs 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9579fae640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9579fae640 1 -- 172.21.15.89:0/3607036469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9574131490 con 0x7f957409b040 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f956c014cb0 con 0x7f957409b040 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f956bfff640 10 monclient: dump: 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.096 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9579fae640 10 monclient: _renew_subs 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9579fae640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9579fae640 1 -- 172.21.15.89:0/3607036469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9574074310 con 0x7f957409b040 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9571ffb640 4 set_mon_vals no callback set 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9579fae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9579fae640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574130ff0 0x7f95741cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9579fae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9579fae640 1 -- 172.21.15.89:0/3607036469 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9574130bf0 con 0x7f9574130ff0 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9572ffd640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574130ff0 0x7f95741cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9572ffd640 10 monclient: get_auth_request con 0x7f9574130ff0 auth_method 0 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9572ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9572ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9572ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.083+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f956c01b020 con 0x7f957409b040 2024-08-19T10:47:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f9572ffd640 10 monclient(hunting): handle_auth_done global_id 44587 payload 824 2024-08-19T10:47:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f9572ffd640 1 -- 172.21.15.89:0/3607036469 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95740a1360 con 0x7f9574130ff0 2024-08-19T10:47:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f955803be00 0x7f955803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f956c04fe10 con 0x7f957409b040 2024-08-19T10:47:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f9573fff640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f955803be00 0x7f955803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f9573fff640 10 monclient: get_auth_request con 0x7f955803be00 auth_method 0 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f9572ffd640 1 -- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574130ff0 msgr2=0x7f95741cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f9572ffd640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574130ff0 0x7f95741cfa50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f955c0ab000 tx=0x7f955c019250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f955c0ba070 con 0x7f9574130ff0 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 10 monclient: dump: 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.102 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f9573fff640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f955803be00 0x7f955803e2c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9564003940 tx=0x7f9564003b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f955c0b1040 con 0x7f9574130ff0 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:23.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574130ff0 msgr2=0x7f95741cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574130ff0 0x7f95741cfa50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f955c0ab000 tx=0x7f955c019250 comp rx=0 tx=0).stop 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574130ff0 msgr2=0x7f95741cfa50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.087+0000 7f956bfff640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9574130ff0 0x7f95741cfa50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9579fae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9579fae640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95741cff90 0x7f957413ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9579fae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9579fae640 1 -- 172.21.15.89:0/3607036469 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f95741d0370 con 0x7f95741cff90 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9572ffd640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95741cff90 0x7f957413ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9572ffd640 10 monclient: get_auth_request con 0x7f95741cff90 auth_method 0 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9572ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9572ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9572ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9572ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9572ffd640 10 monclient(hunting): handle_auth_done global_id 44596 payload 824 2024-08-19T10:47:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.095+0000 7f9572ffd640 1 -- 172.21.15.89:0/3607036469 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f957409b4c0 con 0x7f95741cff90 2024-08-19T10:47:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9572ffd640 1 -- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95741cff90 msgr2=0x7f957413ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9572ffd640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95741cff90 0x7f957413ca40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f955c01ab60 tx=0x7f955c0a8000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f955c0b0070 con 0x7f95741cff90 2024-08-19T10:47:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f956bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f956bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:47:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f956bfff640 10 monclient: dump: 2024-08-19T10:47:23.106 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.107 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.108 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9256 (secure 0 0 0) 0x7f955c0be610 con 0x7f95741cff90 2024-08-19T10:47:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f956bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:47:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f956bfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95741cff90 msgr2=0x7f957413ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f956bfff640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95741cff90 0x7f957413ca40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f955c01ab60 tx=0x7f955c0a8000 comp rx=0 tx=0).stop 2024-08-19T10:47:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f956bfff640 1 -- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95741cff90 msgr2=0x7f957413ca40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:23.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f956bfff640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95741cff90 0x7f957413ca40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.108 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:47:23.108 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 90, 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 166226, 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:47:23.109 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:23.110 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:23.111 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:23.112 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:23.113 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:23.114 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.115 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.116 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.117 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.118 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.119 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.120 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.121 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:47:23.122 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.124 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.125 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.127 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.129 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.130 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:47:23.131 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:47:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9579fae640 1 -- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f955803be00 msgr2=0x7f955803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9579fae640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f955803be00 0x7f955803e2c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9564003940 tx=0x7f9564003b50 comp rx=0 tx=0).stop 2024-08-19T10:47:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9579fae640 10 monclient: shutdown 2024-08-19T10:47:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9579fae640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:23.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9579fae640 1 -- 172.21.15.89:0/3607036469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957409b040 msgr2=0x7f95741cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9579fae640 1 --2- 172.21.15.89:0/3607036469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f957409b040 0x7f95741cc010 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f956c009ec0 tx=0x7f956c00c490 comp rx=0 tx=0).stop 2024-08-19T10:47:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9573fff640 1 -- 172.21.15.89:0/3607036469 reap_dead start 2024-08-19T10:47:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9573fff640 1 -- 172.21.15.89:0/3607036469 reap_dead start 2024-08-19T10:47:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9579fae640 1 -- 172.21.15.89:0/3607036469 shutdown_connections 2024-08-19T10:47:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9579fae640 1 -- 172.21.15.89:0/3607036469 >> 172.21.15.89:0/3607036469 conn(0x7f95740a4e60 msgr2=0x7f957413b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9579fae640 1 -- 172.21.15.89:0/3607036469 shutdown_connections 2024-08-19T10:47:23.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.099+0000 7f9579fae640 1 -- 172.21.15.89:0/3607036469 wait complete. 2024-08-19T10:47:23.133 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:47:23.133 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-08-19T10:47:23.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:47:23.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.255+0000 7f9dbe867640 10 monclient: get_monmap_and_config 2024-08-19T10:47:23.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.255+0000 7f9dbe867640 10 monclient: build_initial_monmap 2024-08-19T10:47:23.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.255+0000 7f9dbe867640 10 monclient: monmap: 2024-08-19T10:47:23.257 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:23.257 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.257 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:47:23.260860+0000 2024-08-19T10:47:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:47:23.260860+0000 2024-08-19T10:47:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:47:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:47:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:23.258 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:23.259 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:23.259 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:23.259 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:23.259 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:23.259 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:47:23.259 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:47:23.259 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 1 Processor -- start 2024-08-19T10:47:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 1 -- start start 2024-08-19T10:47:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 10 monclient: init 2024-08-19T10:47:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 10 monclient: _add_conns ranks=[4,9,8,1,0,5,6,2,3,7,10] 2024-08-19T10:47:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9db806c930 0x7f9db80a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 10 monclient(hunting): picked mon.noname-c con 0x7f9db806c930 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:47:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9db80a3f40 0x7f9db80a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 10 monclient(hunting): picked mon.noname-a con 0x7f9db80a3f40 addr v2:172.21.15.89:3300/0 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db8139850 0x7f9db8139d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 10 monclient(hunting): picked mon.noname-e con 0x7f9db8139850 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f9db80a4860 con 0x7f9db80a3f40 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9db806cd10 con 0x7f9db806c930 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9db80a1bc0 con 0x7f9db8139850 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbe867640 10 monclient(hunting): authenticate will time out at 1493.077439s 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9db80a3f40 0x7f9db80a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db7fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9db806c930 0x7f9db80a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbce04640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db8139850 0x7f9db8139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9db80a3f40 0x7f9db80a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60934/0 (socket says 172.21.15.89:60934) 2024-08-19T10:47:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 1 -- 172.21.15.89:0/4079318102 learned_addr learned my addr 172.21.15.89:0/4079318102 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 10 monclient(hunting): get_auth_request con 0x7f9db80a3f40 auth_method 0 2024-08-19T10:47:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db7fff640 10 monclient(hunting): get_auth_request con 0x7f9db806c930 auth_method 0 2024-08-19T10:47:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbce04640 10 monclient(hunting): get_auth_request con 0x7f9db8139850 auth_method 0 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbce04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbce04640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbce04640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbce04640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbce04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbce04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 10 monclient(hunting): handle_auth_done global_id 6786 payload 274 2024-08-19T10:47:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 1 -- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db8139850 msgr2=0x7f9db8139d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 1 --2- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db8139850 0x7f9db8139d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 1 -- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9db806c930 msgr2=0x7f9db80a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 1 --2- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9db806c930 0x7f9db80a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:47:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 1 monclient: found mon.noname-a 2024-08-19T10:47:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:23.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:47:23.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db77fe640 1 -- 172.21.15.89:0/4079318102 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db80a4b60 con 0x7f9db80a3f40 2024-08-19T10:47:23.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9dbce04640 1 --2- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db8139850 0x7f9db8139d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:23.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.259+0000 7f9db7fff640 1 --2- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9db806c930 0x7f9db80a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:23.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db77fe640 1 -- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db80a3f40 msgr2=0x7f9db80a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db77fe640 1 --2- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db80a3f40 0x7f9db80a4320 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0098a0 tx=0x7f9dac01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932b3f522be9a634 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 1 -- 172.21.15.89:0/4079318102 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9dac0b0070 con 0x7f9db80a3f40 2024-08-19T10:47:23.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:47:23.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 10 monclient: dump: 2024-08-19T10:47:23.273 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.273 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.273 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.273 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.274 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 1 -- 172.21.15.89:0/4079318102 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dac0aa810 con 0x7f9db80a3f40 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 5 monclient: authenticate success, global_id 6786 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 1 -- 172.21.15.89:0/4079318102 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9dac0a9300 con 0x7f9db80a3f40 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9db6ffd640 10 monclient: dump: 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.275 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.276 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 10 monclient: get_monmap_and_config success 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 4 set_mon_vals no callback set 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 10 monclient: shutdown 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 1 -- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db80a3f40 msgr2=0x7f9db80a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 1 --2- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db80a3f40 0x7f9db80a4320 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0098a0 tx=0x7f9dac01c400 comp rx=0 tx=0).stop 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 1 -- 172.21.15.89:0/4079318102 shutdown_connections 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 1 --2- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db8139850 0x7f9db8139d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 1 --2- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9db80a3f40 0x7f9db80a4320 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 1 --2- 172.21.15.89:0/4079318102 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9db806c930 0x7f9db80a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 1 -- 172.21.15.89:0/4079318102 >> 172.21.15.89:0/4079318102 conn(0x7f9db80a9090 msgr2=0x7f9db80a94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 1 -- 172.21.15.89:0/4079318102 shutdown_connections 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 1 -- 172.21.15.89:0/4079318102 wait complete. 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 10 monclient: build_initial_monmap 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.263+0000 7f9dbe867640 10 monclient: monmap: 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.278 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 1 Processor -- start 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 1 -- start start 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 10 monclient: init 2024-08-19T10:47:23.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 10 monclient: _add_conns ranks=[8,2,1,0,7,4,5,6,3] 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db806c930 0x7f9db8138820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 10 monclient(hunting): picked mon.noname-e con 0x7f9db806c930 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9db80a3f40 0x7f9db813c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 10 monclient(hunting): picked mon.noname-b con 0x7f9db80a3f40 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9db8139850 0x7f9db813cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 10 monclient(hunting): picked mon.noname-f con 0x7f9db8139850 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9db80a0f10 con 0x7f9db8139850 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9db80a0c30 con 0x7f9db80a3f40 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9db80a0190 con 0x7f9db806c930 2024-08-19T10:47:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 10 monclient(hunting): authenticate will time out at 1493.084921s 2024-08-19T10:47:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db806c930 0x7f9db8138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db806c930 0x7f9db8138820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47126/0 (socket says 172.21.15.89:47126) 2024-08-19T10:47:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 1 -- 172.21.15.89:0/3812470961 learned_addr learned my addr 172.21.15.89:0/3812470961 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 10 monclient(hunting): get_auth_request con 0x7f9db806c930 auth_method 0 2024-08-19T10:47:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 10 monclient(hunting): handle_auth_done global_id 6677 payload 824 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 1 -- 172.21.15.89:0/3812470961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9db80a3f40 msgr2=0x7f9db813c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9db80a3f40 0x7f9db813c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 1 -- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9db8139850 msgr2=0x7f9db813cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9db8139850 0x7f9db813cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 1 monclient: found mon.noname-e 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 1 -- 172.21.15.89:0/3812470961 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db81d1f30 con 0x7f9db806c930 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 1 -- 172.21.15.89:0/3812470961 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db806c930 msgr2=0x7f9db8138820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db7fff640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db806c930 0x7f9db8138820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e980 tx=0x7f9da800ee40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9da8017070 con 0x7f9db806c930 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 10 monclient: dump: 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.283 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.284 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.284 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.284 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.284 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.284 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.284 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.284 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 10 monclient: _finish_auth 0 2024-08-19T10:47:23.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:47:23.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:23.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da800cc70 con 0x7f9db806c930 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9dbe867640 5 monclient: authenticate success, global_id 6677 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9da800ce20 con 0x7f9db806c930 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db4ff9640 10 monclient: dump: 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.285 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.267+0000 7f9db67fc640 4 set_mon_vals no callback set 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbe867640 10 monclient: _renew_subs 2024-08-19T10:47:23.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbe867640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbe867640 1 -- 172.21.15.89:0/3812470961 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db8123f00 con 0x7f9db806c930 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbe867640 10 monclient: _renew_subs 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbe867640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbe867640 1 -- 172.21.15.89:0/3812470961 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db813eae0 con 0x7f9db806c930 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbe867640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbe867640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db8138d60 0x7f9db802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbe867640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbe867640 1 -- 172.21.15.89:0/3812470961 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9db8064590 con 0x7f9db8138d60 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9da801a070 con 0x7f9db806c930 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db4ff9640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9da403be00 0x7f9da403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db77fe640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db8138d60 0x7f9db802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7f9da800f040 con 0x7f9db806c930 2024-08-19T10:47:23.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db77fe640 10 monclient: get_auth_request con 0x7f9db8138d60 auth_method 0 2024-08-19T10:47:23.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbce04640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9da403be00 0x7f9da403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9dbce04640 10 monclient: get_auth_request con 0x7f9da403be00 auth_method 0 2024-08-19T10:47:23.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db77fe640 10 monclient(hunting): handle_auth_done global_id 35301 payload 824 2024-08-19T10:47:23.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.271+0000 7f9db77fe640 1 -- 172.21.15.89:0/3812470961 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9db813fc50 con 0x7f9db8138d60 2024-08-19T10:47:23.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db77fe640 1 -- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db8138d60 msgr2=0x7f9db802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db77fe640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db8138d60 0x7f9db802d1d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0ad000 tx=0x7f9dac004580 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9dbce04640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9da403be00 0x7f9da403e2c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009bd0 tx=0x7f9db000d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9dac0b9070 con 0x7f9db8138d60 2024-08-19T10:47:23.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db4ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:47:23.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db4ff9640 10 monclient: dump: 2024-08-19T10:47:23.291 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.292 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9dac0093d0 con 0x7f9db8138d60 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db4ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db8138d60 msgr2=0x7f9db802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db4ff9640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db8138d60 0x7f9db802d1d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0ad000 tx=0x7f9dac004580 comp rx=0 tx=0).stop 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db8138d60 msgr2=0x7f9db802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.275+0000 7f9db4ff9640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db8138d60 0x7f9db802d1d0 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0ad000 tx=0x7f9dac004580 comp rx=0 tx=0).stop 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.279+0000 7f9dbe867640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.279+0000 7f9dbe867640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db802d710 0x7f9db802daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.279+0000 7f9dbe867640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.279+0000 7f9dbe867640 1 -- 172.21.15.89:0/3812470961 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9db8139e50 con 0x7f9db802d710 2024-08-19T10:47:23.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db802d710 0x7f9db802daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 10 monclient: get_auth_request con 0x7f9db802d710 auth_method 0 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 10 monclient(hunting): handle_auth_done global_id 35310 payload 824 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 1 -- 172.21.15.89:0/3812470961 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9db8072490 con 0x7f9db802d710 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 1 -- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db802d710 msgr2=0x7f9db802daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db77fe640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db802d710 0x7f9db802daf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9dac000c10 tx=0x7f9dac0a8000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9dac0b9040 con 0x7f9db802d710 2024-08-19T10:47:23.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db4ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:47:23.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.283+0000 7f9db4ff9640 10 monclient: dump: 2024-08-19T10:47:23.298 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.298 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.298 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.298 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.299 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.300 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9423 (secure 0 0 0) 0x7f9dac0bf070 con 0x7f9db802d710 2024-08-19T10:47:23.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9db4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:47:23.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9db4ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:23.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db802d710 msgr2=0x7f9db802daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9db4ff9640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db802d710 0x7f9db802daf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9dac000c10 tx=0x7f9dac0a8000 comp rx=0 tx=0).stop 2024-08-19T10:47:23.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9db4ff9640 1 -- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db802d710 msgr2=0x7f9db802daf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:23.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9db4ff9640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9db802d710 0x7f9db802daf0 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9dac000c10 tx=0x7f9dac0a8000 comp rx=0 tx=0).stop 2024-08-19T10:47:23.300 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 90, 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 288328, 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:47:23.301 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:23.302 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:23.303 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:47:23.304 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:47:23.305 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:23.306 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.307 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:47:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:47:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:47:23.308 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:47:23.309 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.310 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:47:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:47:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:47:23.312 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:47:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.315 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.317 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:23.318 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:47:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:47:23.319 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:47:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:47:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.320 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:47:23.321 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:23.322 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.323 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.324 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.325 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:47:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.326 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:47:23.326 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:47:23.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbe867640 1 -- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9da403be00 msgr2=0x7f9da403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbe867640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9da403be00 0x7f9da403e2c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009bd0 tx=0x7f9db000d040 comp rx=0 tx=0).stop 2024-08-19T10:47:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbe867640 10 monclient: shutdown 2024-08-19T10:47:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbe867640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbe867640 1 -- 172.21.15.89:0/3812470961 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db806c930 msgr2=0x7f9db8138820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbe867640 1 --2- 172.21.15.89:0/3812470961 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9db806c930 0x7f9db8138820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e980 tx=0x7f9da800ee40 comp rx=0 tx=0).stop 2024-08-19T10:47:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbce04640 1 -- 172.21.15.89:0/3812470961 reap_dead start 2024-08-19T10:47:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbce04640 1 -- 172.21.15.89:0/3812470961 reap_dead start 2024-08-19T10:47:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbe867640 1 -- 172.21.15.89:0/3812470961 shutdown_connections 2024-08-19T10:47:23.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbe867640 1 -- 172.21.15.89:0/3812470961 >> 172.21.15.89:0/3812470961 conn(0x7f9db80a9090 msgr2=0x7f9db813fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:23.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbe867640 1 -- 172.21.15.89:0/3812470961 shutdown_connections 2024-08-19T10:47:23.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.287+0000 7f9dbe867640 1 -- 172.21.15.89:0/3812470961 wait complete. 2024-08-19T10:47:23.329 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:47:23.329 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-08-19T10:47:23.329 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:47:23.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient: get_monmap_and_config 2024-08-19T10:47:23.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient: build_initial_monmap 2024-08-19T10:47:23.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient: monmap: 2024-08-19T10:47:23.446 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:23.446 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.446 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:47:23.449236+0000 2024-08-19T10:47:23.446 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:47:23.449236+0000 2024-08-19T10:47:23.446 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:23.446 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.446 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:47:23.446 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:47:23.446 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:23.447 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:23.447 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:23.447 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:23.447 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:23.447 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:23.447 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:23.447 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:47:23.447 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:47:23.447 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 1 Processor -- start 2024-08-19T10:47:23.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 1 -- start start 2024-08-19T10:47:23.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient: init 2024-08-19T10:47:23.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:23.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient: _add_conns ranks=[2,8,1,5,0,3,9,10,7,6,4] 2024-08-19T10:47:23.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc0a67d0 0x7fb2dc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient(hunting): picked mon.noname-b con 0x7fb2dc0a67d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:47:23.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb2dc0a6bb0 0x7fb2dc0a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient(hunting): picked mon.noname-e con 0x7fb2dc0a6bb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:47:23.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb2dc139590 addr v1:172.21.15.190:6789/0 2024-08-19T10:47:23.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2dc0a8250 con 0x7fb2dc139590 2024-08-19T10:47:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb2dc0a8740 con 0x7fb2dc0a67d0 2024-08-19T10:47:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb2dc0a3380 con 0x7fb2dc0a6bb0 2024-08-19T10:47:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2e09e0640 10 monclient(hunting): authenticate will time out at 1493.265882s 2024-08-19T10:47:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb2dc0a6bb0 0x7fb2dc0a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2da59c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc0a67d0 0x7fb2dc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb2dc0a6bb0 0x7fb2dc0a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47130/0 (socket says 172.21.15.89:47130) 2024-08-19T10:47:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 1 -- 172.21.15.89:0/4101033821 learned_addr learned my addr 172.21.15.89:0/4101033821 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:23.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 10 monclient(hunting): get_auth_request con 0x7fb2dc0a6bb0 auth_method 0 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2da59c640 10 monclient(hunting): get_auth_request con 0x7fb2dc0a67d0 auth_method 0 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2da59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2da59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2da59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2da59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2da59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2da59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_done global_id 6686 payload 274 2024-08-19T10:47:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 1 -- 172.21.15.89:0/4101033821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc0a67d0 msgr2=0x7fb2dc0a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 1 --2- 172.21.15.89:0/4101033821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc0a67d0 0x7fb2dc0a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 1 -- 172.21.15.89:0/4101033821 >> v1:172.21.15.190:6789/0 conn(0x7fb2dc139590 legacy=0x7fb2dc139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:47:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:47:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 1 monclient: found mon.noname-e 2024-08-19T10:47:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:47:23.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.447+0000 7fb2d9d9b640 1 -- 172.21.15.89:0/4101033821 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc0a4cc0 con 0x7fb2dc0a6bb0 2024-08-19T10:47:23.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d9d9b640 1 -- 172.21.15.89:0/4101033821 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb2dc0a6bb0 msgr2=0x7fb2dc0a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d9d9b640 1 --2- 172.21.15.89:0/4101033821 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb2dc0a6bb0 0x7fb2dc0a6f90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40098a0 tx=0x7fb2c401c4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=81858e2b485d69d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 1 -- 172.21.15.89:0/4101033821 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2c40b0070 con 0x7fb2dc0a6bb0 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 10 monclient: dump: 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.457 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.458 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.458 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.458 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.458 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.458 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.458 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.458 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.458 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 10 monclient: _finish_auth 0 2024-08-19T10:47:23.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 10 monclient: _check_auth_tickets 2024-08-19T10:47:23.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:23.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 1 -- 172.21.15.89:0/4101033821 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c40aa810 con 0x7fb2dc0a6bb0 2024-08-19T10:47:23.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2e09e0640 5 monclient: authenticate success, global_id 6686 2024-08-19T10:47:23.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2e09e0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:47:23.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:23.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 1 -- 172.21.15.89:0/4101033821 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2c40a9300 con 0x7fb2dc0a6bb0 2024-08-19T10:47:23.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:47:23.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2d959a640 10 monclient: dump: 2024-08-19T10:47:23.459 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.460 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2e09e0640 10 monclient: get_monmap_and_config success 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.451+0000 7fb2e09e0640 4 set_mon_vals no callback set 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient: shutdown 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 -- 172.21.15.89:0/4101033821 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb2dc0a6bb0 msgr2=0x7fb2dc0a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 --2- 172.21.15.89:0/4101033821 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb2dc0a6bb0 0x7fb2dc0a6f90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40098a0 tx=0x7fb2c401c4d0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 -- 172.21.15.89:0/4101033821 shutdown_connections 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 --2- 172.21.15.89:0/4101033821 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb2dc0a6bb0 0x7fb2dc0a6f90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 --2- 172.21.15.89:0/4101033821 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc0a67d0 0x7fb2dc0a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 -- 172.21.15.89:0/4101033821 >> 172.21.15.89:0/4101033821 conn(0x7fb2dc069ae0 msgr2=0x7fb2dc0abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:23.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 -- 172.21.15.89:0/4101033821 shutdown_connections 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 -- 172.21.15.89:0/4101033821 wait complete. 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient: build_initial_monmap 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient: monmap: 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:47:23.462 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 Processor -- start 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 -- start start 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient: init 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient: _reopen_session rank -1 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient: _add_conns ranks=[4,7,2,6,5,0,8,1,3] 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc0a67d0 0x7fb2dc13fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient(hunting): picked mon.noname-c con 0x7fb2dc0a67d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb2dc0a6bb0 0x7fb2dc1d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient(hunting): picked mon.noname-i con 0x7fb2dc0a6bb0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc139590 0x7fb2dc13d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient(hunting): picked mon.noname-b con 0x7fb2dc139590 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb2dc1445b0 con 0x7fb2dc139590 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb2dc144430 con 0x7fb2dc0a67d0 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb2dc144730 con 0x7fb2dc0a6bb0 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient(hunting): _renew_subs 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2e09e0640 10 monclient(hunting): authenticate will time out at 1493.274048s 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2d9d9b640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb2dc0a6bb0 msgr2=0x7fb2dc1d1c00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:47:23.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2dad9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc139590 0x7fb2dc13d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2d9d9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb2dc0a6bb0 0x7fb2dc1d1c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:47:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2dad9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc139590 0x7fb2dc13d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35194/0 (socket says 172.21.15.89:35194) 2024-08-19T10:47:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc0a67d0 0x7fb2dc13fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2dad9d640 1 -- 172.21.15.89:0/1298348527 learned_addr learned my addr 172.21.15.89:0/1298348527 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:47:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2dad9d640 10 monclient(hunting): get_auth_request con 0x7fb2dc139590 auth_method 0 2024-08-19T10:47:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2dad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2dad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2dad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 10 monclient(hunting): get_auth_request con 0x7fb2dc0a67d0 auth_method 0 2024-08-19T10:47:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2dad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2dad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2dad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 10 monclient(hunting): handle_auth_done global_id 44680 payload 824 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 1 -- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb2dc0a6bb0 msgr2=0x7fb2dc1d1c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb2dc0a6bb0 0x7fb2dc1d1c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 1 -- 172.21.15.89:0/1298348527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc139590 msgr2=0x7fb2dc13d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc139590 0x7fb2dc13d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 10 monclient: _finish_hunting 0 2024-08-19T10:47:23.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 1 monclient: found mon.noname-c 2024-08-19T10:47:23.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:47:23.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:47:23.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2da59c640 1 -- 172.21.15.89:0/1298348527 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc122aa0 con 0x7fb2dc0a67d0 2024-08-19T10:47:23.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.455+0000 7fb2dad9d640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb2dc139590 0x7fb2dc13d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2da59c640 1 -- 172.21.15.89:0/1298348527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc0a67d0 msgr2=0x7fb2dc13fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2da59c640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc0a67d0 0x7fb2dc13fb70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400efc0 tx=0x7fb2d400c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2d4018020 con 0x7fb2dc0a67d0 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 10 monclient: dump: 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.471 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:47:23.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:47:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2e09e0640 5 monclient: authenticate success, global_id 44680 2024-08-19T10:47:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d40148f0 con 0x7fb2dc0a67d0 2024-08-19T10:47:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2e09e0640 10 monclient: _renew_subs 2024-08-19T10:47:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2e09e0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:47:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2e09e0640 1 -- 172.21.15.89:0/1298348527 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc0ab350 con 0x7fb2dc0a67d0 2024-08-19T10:47:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:47:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2d40133e0 con 0x7fb2dc0a67d0 2024-08-19T10:47:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:47:23.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 10 monclient: dump: 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.474 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d3fff640 4 set_mon_vals no callback set 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2e09e0640 10 monclient: _renew_subs 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2e09e0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2e09e0640 1 -- 172.21.15.89:0/1298348527 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc140500 con 0x7fb2dc0a67d0 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2e09e0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2e09e0640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc1d2140 0x7fb2dc02d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2e09e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2e09e0640 1 -- 172.21.15.89:0/1298348527 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb2dc0a5500 con 0x7fb2dc1d2140 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb2d401b020 con 0x7fb2dc0a67d0 2024-08-19T10:47:23.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d9d9b640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc1d2140 0x7fb2dc02d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb2bc03be00 0x7fb2bc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d9d9b640 10 monclient: get_auth_request con 0x7fb2dc1d2140 auth_method 0 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(271..271 src has 150..271) ==== 7657+0+0 (secure 0 0 0) 0x7fb2d404fe30 con 0x7fb2dc0a67d0 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2dad9d640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb2bc03be00 0x7fb2bc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2dad9d640 10 monclient: get_auth_request con 0x7fb2bc03be00 auth_method 0 2024-08-19T10:47:23.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_done global_id 6791 payload 824 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.459+0000 7fb2d9d9b640 1 -- 172.21.15.89:0/1298348527 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2dc064590 con 0x7fb2dc1d2140 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d9d9b640 1 -- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc1d2140 msgr2=0x7fb2dc02d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d9d9b640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc1d2140 0x7fb2dc02d1d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40ad000 tx=0x7fb2c401ad10 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2dad9d640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb2bc03be00 0x7fb2bc03e2c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009d10 tx=0x7fb2cc0090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2c40ba020 con 0x7fb2dc1d2140 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d2ffd640 10 monclient: dump: 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.477 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb2c40aa5b0 con 0x7fb2dc1d2140 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:47:23.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc1d2140 msgr2=0x7fb2dc02d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d2ffd640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc1d2140 0x7fb2dc02d1d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40ad000 tx=0x7fb2c401ad10 comp rx=0 tx=0).stop 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc1d2140 msgr2=0x7fb2dc02d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.463+0000 7fb2d2ffd640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc1d2140 0x7fb2dc02d1d0 secure :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40ad000 tx=0x7fb2c401ad10 comp rx=0 tx=0).stop 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2e09e0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2e09e0640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc02d710 0x7fb2dc02db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2e09e0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2e09e0640 1 -- 172.21.15.89:0/1298348527 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb2dc13f010 con 0x7fb2dc02d710 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2d9d9b640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc02d710 0x7fb2dc02db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2d9d9b640 10 monclient: get_auth_request con 0x7fb2dc02d710 auth_method 0 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2d9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2d9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:47:23.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2d9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:47:23.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:47:23.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:47:23.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:47:23.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2d9d9b640 10 monclient(hunting): handle_auth_done global_id 6800 payload 824 2024-08-19T10:47:23.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.471+0000 7fb2d9d9b640 1 -- 172.21.15.89:0/1298348527 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb2dc0a0340 con 0x7fb2dc02d710 2024-08-19T10:47:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d9d9b640 1 -- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc02d710 msgr2=0x7fb2dc02db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:47:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d9d9b640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc02d710 0x7fb2dc02db20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb2c401c300 tx=0x7fb2c4019270 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:47:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb2c40c3070 con 0x7fb2dc02d710 2024-08-19T10:47:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:47:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:47:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d2ffd640 10 monclient: dump: 2024-08-19T10:47:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:47:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:47:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:47:23.481 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9590 (secure 0 0 0) 0x7fb2c40bec10 con 0x7fb2dc02d710 2024-08-19T10:47:23.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:47:23.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d2ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:47:23.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc02d710 msgr2=0x7fb2dc02db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d2ffd640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc02d710 0x7fb2dc02db20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb2c401c300 tx=0x7fb2c4019270 comp rx=0 tx=0).stop 2024-08-19T10:47:23.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d2ffd640 1 -- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc02d710 msgr2=0x7fb2dc02db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:47:23.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2d2ffd640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb2dc02d710 0x7fb2dc02db20 secure :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb2c401c300 tx=0x7fb2c4019270 comp rx=0 tx=0).stop 2024-08-19T10:47:23.483 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:47:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:47:23.483 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 90, 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 573242, 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.484 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:23.485 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:23.486 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:47:23.487 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:47:23.488 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:47:23.489 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:47:23.492 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:47:23.493 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.495 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:47:23.496 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:47:23.499 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.501 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.503 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:47:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:47:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:47:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:47:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:47:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:47:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:47:23.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:47:23.505 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:47:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:47:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:23.506 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:47:23.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:47:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:47:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:47:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:47:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:47:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:47:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:47:23.508 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:47:23.508 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:47:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2e09e0640 1 -- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb2bc03be00 msgr2=0x7fb2bc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2e09e0640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb2bc03be00 0x7fb2bc03e2c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009d10 tx=0x7fb2cc0090e0 comp rx=0 tx=0).stop 2024-08-19T10:47:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2e09e0640 10 monclient: shutdown 2024-08-19T10:47:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2e09e0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:47:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2e09e0640 1 -- 172.21.15.89:0/1298348527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc0a67d0 msgr2=0x7fb2dc13fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:47:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2e09e0640 1 --2- 172.21.15.89:0/1298348527 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb2dc0a67d0 0x7fb2dc13fb70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400efc0 tx=0x7fb2d400c490 comp rx=0 tx=0).stop 2024-08-19T10:47:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2dad9d640 1 -- 172.21.15.89:0/1298348527 reap_dead start 2024-08-19T10:47:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2dad9d640 1 -- 172.21.15.89:0/1298348527 reap_dead start 2024-08-19T10:47:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2e09e0640 1 -- 172.21.15.89:0/1298348527 shutdown_connections 2024-08-19T10:47:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2e09e0640 1 -- 172.21.15.89:0/1298348527 >> 172.21.15.89:0/1298348527 conn(0x7fb2dc069ae0 msgr2=0x7fb2dc0ab780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:47:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2e09e0640 1 -- 172.21.15.89:0/1298348527 shutdown_connections 2024-08-19T10:47:23.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:47:23.475+0000 7fb2e09e0640 1 -- 172.21.15.89:0/1298348527 wait complete. 2024-08-19T10:47:23.511 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:47:23.511 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-08-19T10:48:53.512 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-08-19T10:48:53.512 INFO:tasks.ceph.mon.i:Restarting daemon 2024-08-19T10:48:53.513 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-08-19T10:48:53.516 INFO:tasks.ceph.mon.i:Started 2024-08-19T10:48:53.517 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-08-19T10:48:53.517 INFO:tasks.ceph.mon.a:Sent signal 19 2024-08-19T10:48:53.517 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-08-19T10:48:53.517 INFO:tasks.ceph.mon.g:Sent signal 19 2024-08-19T10:48:53.517 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-08-19T10:49:08.518 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-08-19T10:49:08.519 INFO:tasks.ceph.mon.a:Sent signal 18 2024-08-19T10:49:08.519 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-08-19T10:49:08.520 INFO:tasks.ceph.mon.g:Sent signal 18 2024-08-19T10:49:08.520 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:49:08.520 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:49:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.666+0000 7fd29bf6d640 10 monclient: get_monmap_and_config 2024-08-19T10:49:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.666+0000 7fd29bf6d640 10 monclient: build_initial_monmap 2024-08-19T10:49:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.666+0000 7fd29bf6d640 10 monclient: monmap: 2024-08-19T10:49:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:08.668 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:08.672664+0000 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:08.672664+0000 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:08.669 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:08.670 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:08.670 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 1 Processor -- start 2024-08-19T10:49:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 1 -- start start 2024-08-19T10:49:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 10 monclient: init 2024-08-19T10:49:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 10 monclient: _add_conns ranks=[2,0,5,9,3,1,10,4,8,7,6] 2024-08-19T10:49:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2940a6860 0x7fd2940a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 10 monclient(hunting): picked mon.noname-b con 0x7fd2940a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:08.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd2940a7180 addr v1:172.21.15.89:6789/0 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7560 0x7fd294139d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 10 monclient(hunting): picked mon.noname-h con 0x7fd2940a7560 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2940a33d0 con 0x7fd2940a7180 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd2940a8190 con 0x7fd2940a6860 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd2940a79f0 con 0x7fd2940a7560 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29bf6d640 10 monclient(hunting): authenticate will time out at 1598.489338s 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2940a6860 0x7fd2940a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299508640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd2940a7180 0x7fd294139620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35886/0 (socket says 172.21.15.89:35886) 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299508640 1 -- 172.21.15.89:0/2173522808 learned_addr learned my addr 172.21.15.89:0/2173522808 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:08.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 10 monclient(hunting): get_auth_request con 0x7fd2940a6860 auth_method 0 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 10 monclient(hunting): handle_auth_done global_id 54353 payload 274 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29a50a640 1 --2- 172.21.15.89:0/2173522808 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7560 0x7fd294139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 1 -- 172.21.15.89:0/2173522808 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7560 msgr2=0x7fd294139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 1 --2- 172.21.15.89:0/2173522808 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7560 0x7fd294139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 1 -- 172.21.15.89:0/2173522808 >> v1:172.21.15.89:6789/0 conn(0x7fd2940a7180 legacy=0x7fd294139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 10 monclient: _finish_hunting 0 2024-08-19T10:49:08.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 1 monclient: found mon.noname-b 2024-08-19T10:49:08.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:08.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:49:08.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd299d09640 1 -- 172.21.15.89:0/2173522808 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2940a2ec0 con 0x7fd2940a6860 2024-08-19T10:49:08.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29a50a640 10 monclient: get_auth_request con 0x7fd2940a7560 auth_method 0 2024-08-19T10:49:08.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.670+0000 7fd29a50a640 1 --2- 172.21.15.89:0/2173522808 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7560 0x7fd294139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd299d09640 1 -- 172.21.15.89:0/2173522808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2940a6860 msgr2=0x7fd2940a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd299d09640 1 --2- 172.21.15.89:0/2173522808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2940a6860 0x7fd2940a6c40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2880097f0 tx=0x7fd28801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13956f58212c8d2d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 1 -- 172.21.15.89:0/2173522808 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2880b0070 con 0x7fd2940a6860 2024-08-19T10:49:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:49:08.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 10 monclient: dump: 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:08.680 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 10 monclient: _finish_auth 0 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 10 monclient: _check_auth_tickets 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 1 -- 172.21.15.89:0/2173522808 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2880aa810 con 0x7fd2940a6860 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 5 monclient: authenticate success, global_id 54353 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 1 -- 172.21.15.89:0/2173522808 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2880aae30 con 0x7fd2940a6860 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:08.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd298d07640 10 monclient: dump: 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:08.682 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 4 set_mon_vals no callback set 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 10 monclient: shutdown 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2173522808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2940a6860 msgr2=0x7fd2940a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 1 --2- 172.21.15.89:0/2173522808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2940a6860 0x7fd2940a6c40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd2880097f0 tx=0x7fd28801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2173522808 shutdown_connections 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 1 --2- 172.21.15.89:0/2173522808 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7560 0x7fd294139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 1 --2- 172.21.15.89:0/2173522808 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2940a6860 0x7fd2940a6c40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2173522808 >> 172.21.15.89:0/2173522808 conn(0x7fd294074b40 msgr2=0x7fd2940abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:08.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2173522808 shutdown_connections 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2173522808 wait complete. 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 10 monclient: build_initial_monmap 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.674+0000 7fd29bf6d640 10 monclient: monmap: 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:08.684 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 1 Processor -- start 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 1 -- start start 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 10 monclient: init 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 10 monclient: _add_conns ranks=[7,5,3,6,1,0,4,2,8] 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2940a6860 0x7fd294138f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 10 monclient(hunting): picked mon.noname-i con 0x7fd2940a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:08.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7180 0x7fd2941394d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 10 monclient(hunting): picked mon.noname-h con 0x7fd2940a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2940a7560 0x7fd29413e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 10 monclient(hunting): picked mon.noname-g con 0x7fd2940a7560 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd29413a3f0 con 0x7fd2940a7560 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd29413a270 con 0x7fd2940a7180 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd2941440a0 con 0x7fd2940a6860 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29bf6d640 10 monclient(hunting): authenticate will time out at 1598.496623s 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7180 0x7fd2941394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29a50a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2940a7560 0x7fd29413e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:08.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299d09640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2940a6860 0x7fd294138f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7180 0x7fd2941394d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:34680/0 (socket says 172.21.15.89:34680) 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 1 -- 172.21.15.89:0/2846349572 learned_addr learned my addr 172.21.15.89:0/2846349572 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 10 monclient(hunting): get_auth_request con 0x7fd2940a7180 auth_method 0 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29a50a640 10 monclient(hunting): get_auth_request con 0x7fd2940a7560 auth_method 0 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29a50a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29a50a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29a50a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299d09640 10 monclient(hunting): get_auth_request con 0x7fd2940a6860 auth_method 0 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299d09640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299d09640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299d09640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29a50a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29a50a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd29a50a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299d09640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299d09640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299d09640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 10 monclient(hunting): handle_auth_done global_id 6809 payload 824 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 1 -- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2940a6860 msgr2=0x7fd294138f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2940a6860 0x7fd294138f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 1 -- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2940a7560 msgr2=0x7fd29413e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2940a7560 0x7fd29413e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 10 monclient: _finish_hunting 0 2024-08-19T10:49:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 1 monclient: found mon.noname-h 2024-08-19T10:49:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:49:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 1 -- 172.21.15.89:0/2846349572 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2941237d0 con 0x7fd2940a7180 2024-08-19T10:49:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 1 -- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7180 msgr2=0x7fd2941394d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.678+0000 7fd299508640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7180 0x7fd2941394d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c470 tx=0x7fd29000c930 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 1 -- 172.21.15.89:0/2846349572 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd290005d00 con 0x7fd2940a7180 2024-08-19T10:49:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 10 monclient: dump: 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 10 monclient: _finish_auth 0 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 10 monclient: _check_auth_tickets 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 1 -- 172.21.15.89:0/2846349572 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2900074e0 con 0x7fd2940a7180 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 1 -- 172.21.15.89:0/2846349572 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2900076d0 con 0x7fd2940a7180 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd29bf6d640 5 monclient: authenticate success, global_id 6809 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd29bf6d640 10 monclient: _renew_subs 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd29bf6d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:49:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2846349572 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd294140510 con 0x7fd2940a7180 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 10 monclient: dump: 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:08.695 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd287fff640 4 set_mon_vals no callback set 2024-08-19T10:49:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd29bf6d640 10 monclient: _renew_subs 2024-08-19T10:49:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd29bf6d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:49:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2846349572 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2940a2710 con 0x7fd2940a7180 2024-08-19T10:49:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd29bf6d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd29bf6d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd29bf6d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:49:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2846349572 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2940a23f0 con 0x7fd2940a7180 2024-08-19T10:49:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 1 -- 172.21.15.89:0/2846349572 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd290007070 con 0x7fd2940a7180 2024-08-19T10:49:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd27803bd60 0x7fd27803e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd2867fc640 1 -- 172.21.15.89:0/2846349572 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7fd29004f7b0 con 0x7fd2940a7180 2024-08-19T10:49:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd299d09640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd27803bd60 0x7fd27803e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.682+0000 7fd299d09640 10 monclient: get_auth_request con 0x7fd27803bd60 auth_method 0 2024-08-19T10:49:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.686+0000 7fd299d09640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd27803bd60 0x7fd27803e220 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd2880ab000 tx=0x7fd2880a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.694+0000 7fd2867fc640 1 -- 172.21.15.89:0/2846349572 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd2940a23f0 con 0x7fd2940a7180 2024-08-19T10:49:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.694+0000 7fd2867fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.694+0000 7fd2867fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.926+0000 7fd29bf6d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:49:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.926+0000 7fd29bf6d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.926+0000 7fd29bf6d640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:49:08.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.926+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2846349572 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd2940a21c0 con 0x7fd2940a7180 2024-08-19T10:49:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd2867fc640 1 -- 172.21.15.89:0/2846349572 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7fd290010d10 con 0x7fd2940a7180 2024-08-19T10:49:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd2867fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:08.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd2867fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:08.929 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd27803bd60 msgr2=0x7fd27803e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd27803bd60 0x7fd27803e220 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd2880ab000 tx=0x7fd2880a6000 comp rx=0 tx=0).stop 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 10 monclient: shutdown 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7180 msgr2=0x7fd2941394d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7180 0x7fd2941394d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c470 tx=0x7fd29000c930 comp rx=0 tx=0).stop 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2846349572 shutdown_connections 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd27803bd60 0x7fd27803e220 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2940a7180 0x7fd2941394d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2940a7560 0x7fd29413e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 --2- 172.21.15.89:0/2846349572 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd2940a6860 0x7fd294138f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:08.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2846349572 >> 172.21.15.89:0/2846349572 conn(0x7fd294074b40 msgr2=0x7fd2940a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:08.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2846349572 shutdown_connections 2024-08-19T10:49:08.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:08.930+0000 7fd29bf6d640 1 -- 172.21.15.89:0/2846349572 wait complete. 2024-08-19T10:49:08.951 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:49:08.951 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:49:08.951 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:49:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.070+0000 7f3664d8c640 10 monclient: get_monmap_and_config 2024-08-19T10:49:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.070+0000 7f3664d8c640 10 monclient: build_initial_monmap 2024-08-19T10:49:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.070+0000 7f3664d8c640 10 monclient: monmap: 2024-08-19T10:49:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:09.071 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:09.076049+0000 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:09.076049+0000 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:09.072 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:09.073 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:09.073 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:09.073 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 1 Processor -- start 2024-08-19T10:49:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 1 -- start start 2024-08-19T10:49:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 10 monclient: init 2024-08-19T10:49:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 10 monclient: _add_conns ranks=[7,2,10,0,4,3,6,1,5,9,8] 2024-08-19T10:49:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f36600a3d70 0x7f36600a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 10 monclient(hunting): picked mon.noname-i con 0x7f36600a3d70 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36600a4150 0x7f36600a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 10 monclient(hunting): picked mon.noname-b con 0x7f36600a4150 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:09.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f366009b040 0x7f366009b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 10 monclient(hunting): picked mon.noname-f con 0x7f366009b040 addr v2:172.21.15.190:3300/0 2024-08-19T10:49:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f366006f8e0 con 0x7f366009b040 2024-08-19T10:49:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f36600a0780 con 0x7f36600a4150 2024-08-19T10:49:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f366009fb40 con 0x7f36600a3d70 2024-08-19T10:49:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f3664d8c640 10 monclient(hunting): authenticate will time out at 1598.893138s 2024-08-19T10:49:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36600a4150 0x7f36600a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36600a4150 0x7f36600a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54286/0 (socket says 172.21.15.89:54286) 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 1 -- 172.21.15.89:0/1594739601 learned_addr learned my addr 172.21.15.89:0/1594739601 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 10 monclient(hunting): get_auth_request con 0x7f36600a4150 auth_method 0 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365e59c640 1 --2- 172.21.15.89:0/1594739601 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f36600a3d70 0x7f36600a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 10 monclient(hunting): handle_auth_done global_id 54362 payload 274 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 1 -- 172.21.15.89:0/1594739601 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f36600a3d70 msgr2=0x7f36600a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 1 --2- 172.21.15.89:0/1594739601 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f36600a3d70 0x7f36600a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365ed9d640 1 --2- 172.21.15.89:0/1594739601 >> v2:172.21.15.190:3300/0 conn(0x7f366009b040 0x7f366009b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 1 -- 172.21.15.89:0/1594739601 >> v2:172.21.15.190:3300/0 conn(0x7f366009b040 msgr2=0x7f366009b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 1 --2- 172.21.15.89:0/1594739601 >> v2:172.21.15.190:3300/0 conn(0x7f366009b040 0x7f366009b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 10 monclient: _finish_hunting 0 2024-08-19T10:49:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 1 monclient: found mon.noname-b 2024-08-19T10:49:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:49:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365dd9b640 1 -- 172.21.15.89:0/1594739601 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366009fe90 con 0x7f36600a4150 2024-08-19T10:49:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365e59c640 10 monclient: get_auth_request con 0x7f36600a3d70 auth_method 0 2024-08-19T10:49:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365e59c640 1 --2- 172.21.15.89:0/1594739601 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f36600a3d70 0x7f36600a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365ed9d640 10 monclient: get_auth_request con 0x7f366009b040 auth_method 0 2024-08-19T10:49:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.074+0000 7f365ed9d640 1 --2- 172.21.15.89:0/1594739601 >> v2:172.21.15.190:3300/0 conn(0x7f366009b040 0x7f366009b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:09.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365dd9b640 1 -- 172.21.15.89:0/1594739601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36600a4150 msgr2=0x7f36600a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365dd9b640 1 --2- 172.21.15.89:0/1594739601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36600a4150 0x7f36600a4570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f36540098a0 tx=0x7f365401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db0088de1434dcf3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 1 -- 172.21.15.89:0/1594739601 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36540b0070 con 0x7f36600a4150 2024-08-19T10:49:09.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:49:09.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 10 monclient: dump: 2024-08-19T10:49:09.080 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.081 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 10 monclient: _finish_auth 0 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 5 monclient: authenticate success, global_id 54362 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 1 -- 172.21.15.89:0/1594739601 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36540aa810 con 0x7f36600a4150 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 4 set_mon_vals no callback set 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 1 -- 172.21.15.89:0/1594739601 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36540aae30 con 0x7f36600a4150 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f365d59a640 10 monclient: dump: 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 10 monclient: shutdown 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 1 -- 172.21.15.89:0/1594739601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36600a4150 msgr2=0x7f36600a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 1 --2- 172.21.15.89:0/1594739601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36600a4150 0x7f36600a4570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f36540098a0 tx=0x7f365401c490 comp rx=0 tx=0).stop 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 1 -- 172.21.15.89:0/1594739601 shutdown_connections 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 1 --2- 172.21.15.89:0/1594739601 >> v2:172.21.15.190:3300/0 conn(0x7f366009b040 0x7f366009b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 1 --2- 172.21.15.89:0/1594739601 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f36600a4150 0x7f36600a4570 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 1 --2- 172.21.15.89:0/1594739601 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f36600a3d70 0x7f36600a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 1 -- 172.21.15.89:0/1594739601 >> 172.21.15.89:0/1594739601 conn(0x7f36600a4e60 msgr2=0x7f36600a5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:09.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 1 -- 172.21.15.89:0/1594739601 shutdown_connections 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.078+0000 7f3664d8c640 1 -- 172.21.15.89:0/1594739601 wait complete. 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient: build_initial_monmap 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient: monmap: 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:09.085 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:09.086 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 1 Processor -- start 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 1 -- start start 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient: init 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient: _add_conns ranks=[5,0,1,3,8,7,2,4,6] 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f366009b040 0x7f366009a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient(hunting): picked mon.noname-h con 0x7f366009b040 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36600a3d70 0x7f3660073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient(hunting): picked mon.noname-a con 0x7f36600a3d70 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36600a4150 0x7f3660073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient(hunting): picked mon.noname-f con 0x7f36600a4150 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3660140f30 con 0x7f36600a3d70 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36600a4ab0 con 0x7f36600a4150 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f36601410b0 con 0x7f366009b040 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f3664d8c640 10 monclient(hunting): authenticate will time out at 1598.901446s 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36600a3d70 0x7f3660073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36600a3d70 0x7f3660073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57658/0 (socket says 172.21.15.89:57658) 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 1 -- 172.21.15.89:0/3739942588 learned_addr learned my addr 172.21.15.89:0/3739942588 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 10 monclient(hunting): get_auth_request con 0x7f36600a3d70 auth_method 0 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365e59c640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f366009b040 0x7f366009a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365ed9d640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36600a4150 0x7f3660073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365e59c640 10 monclient(hunting): get_auth_request con 0x7f366009b040 auth_method 0 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365ed9d640 10 monclient(hunting): get_auth_request con 0x7f36600a4150 auth_method 0 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 10 monclient(hunting): handle_auth_done global_id 6795 payload 824 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 1 -- 172.21.15.89:0/3739942588 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f366009b040 msgr2=0x7f366009a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f366009b040 0x7f366009a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 1 -- 172.21.15.89:0/3739942588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36600a4150 msgr2=0x7f3660073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36600a4150 0x7f3660073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 10 monclient: _finish_hunting 0 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 1 monclient: found mon.noname-a 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.082+0000 7f365dd9b640 1 -- 172.21.15.89:0/3739942588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36600a2880 con 0x7f36600a3d70 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365dd9b640 1 -- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36600a3d70 msgr2=0x7f3660073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365dd9b640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36600a3d70 0x7f3660073140 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f365401c2c0 tx=0x7f3654019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36540b0040 con 0x7f36600a3d70 2024-08-19T10:49:09.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 10 monclient: dump: 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.095 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 10 monclient: _finish_auth 0 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36540ba070 con 0x7f36600a3d70 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f3664d8c640 5 monclient: authenticate success, global_id 6795 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36540aae20 con 0x7f36600a3d70 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:09.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 10 monclient: dump: 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.097 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365cd99640 4 set_mon_vals no callback set 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f3664d8c640 10 monclient: _renew_subs 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f3664d8c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f3664d8c640 1 -- 172.21.15.89:0/3739942588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366012ef60 con 0x7f36600a3d70 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f3664d8c640 10 monclient: _renew_subs 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f3664d8c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f3664d8c640 1 -- 172.21.15.89:0/3739942588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36600a1060 con 0x7f36600a3d70 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f3664d8c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f3664d8c640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366009abc0 0x7f3660073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f3664d8c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f3664d8c640 1 -- 172.21.15.89:0/3739942588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f366012f970 con 0x7f366009abc0 2024-08-19T10:49:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365e59c640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366009abc0 0x7f3660073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365e59c640 10 monclient: get_auth_request con 0x7f366009abc0 auth_method 0 2024-08-19T10:49:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3654004040 con 0x7f36600a3d70 2024-08-19T10:49:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365e59c640 10 monclient(hunting): handle_auth_done global_id 6804 payload 824 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365e59c640 1 -- 172.21.15.89:0/3739942588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36600a0240 con 0x7f366009abc0 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f364003be00 0x7f364003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f36540e8e70 con 0x7f36600a3d70 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365ed9d640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f364003be00 0x7f364003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.086+0000 7f365ed9d640 10 monclient: get_auth_request con 0x7f364003be00 auth_method 0 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f365e59c640 1 -- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366009abc0 msgr2=0x7f3660073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f365e59c640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366009abc0 0x7f3660073bc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f365000f480 tx=0x7f365000f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3650018070 con 0x7f366009abc0 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f364effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f364effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f364effd640 10 monclient: dump: 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.100 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.101 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3650014980 con 0x7f366009abc0 2024-08-19T10:49:09.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f364effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:09.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f364effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:09.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366009abc0 msgr2=0x7f3660073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f364effd640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366009abc0 0x7f3660073bc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f365000f480 tx=0x7f365000f940 comp rx=0 tx=0).stop 2024-08-19T10:49:09.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366009abc0 msgr2=0x7f3660073bc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:09.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f364effd640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f366009abc0 0x7f3660073bc0 secure :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f365000f480 tx=0x7f365000f940 comp rx=0 tx=0).stop 2024-08-19T10:49:09.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.090+0000 7f365ed9d640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f364003be00 0x7f364003e2c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3658007fb0 tx=0x7f3658007540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f3664d8c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f3664d8c640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3660074100 0x7f3660137a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f3664d8c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f3664d8c640 1 -- 172.21.15.89:0/3739942588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3660064590 con 0x7f3660074100 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f365e59c640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3660074100 0x7f3660137a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f365e59c640 10 monclient: get_auth_request con 0x7f3660074100 auth_method 0 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f365e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f365e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f365e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f365e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f365e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f365e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f365e59c640 10 monclient(hunting): handle_auth_done global_id 6813 payload 824 2024-08-19T10:49:09.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.094+0000 7f365e59c640 1 -- 172.21.15.89:0/3739942588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f366012d760 con 0x7f3660074100 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f365e59c640 1 -- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3660074100 msgr2=0x7f3660137a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f365e59c640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3660074100 0x7f3660137a20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f365000f480 tx=0x7f3650004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f36500044a0 con 0x7f3660074100 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f364effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f364effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f364effd640 10 monclient: dump: 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.105 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9436 (secure 0 0 0) 0x7f3650004030 con 0x7f3660074100 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f364effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f364effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3660074100 msgr2=0x7f3660137a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f364effd640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3660074100 0x7f3660137a20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f365000f480 tx=0x7f3650004740 comp rx=0 tx=0).stop 2024-08-19T10:49:09.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f364effd640 1 -- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3660074100 msgr2=0x7f3660137a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:09.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f364effd640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3660074100 0x7f3660137a20 secure :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f365000f480 tx=0x7f3650004740 comp rx=0 tx=0).stop 2024-08-19T10:49:09.107 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:09.107 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:09.107 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:09.107 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:49:09.107 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:09.107 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 678911, 2024-08-19T10:49:09.107 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:09.107 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:09.108 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:09.109 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.110 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:09.111 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.112 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.113 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:09.115 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:09.116 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.117 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.118 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.119 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.120 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:09.121 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.122 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.123 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.124 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:09.126 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:09.127 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.127 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:09.127 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.127 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:09.128 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.129 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:09.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.130 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.130 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.130 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:49:09.130 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.130 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.130 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:09.130 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:09.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f3664d8c640 1 -- 172.21.15.89:0/3739942588 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f364003be00 msgr2=0x7f364003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f3664d8c640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f364003be00 0x7f364003e2c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3658007fb0 tx=0x7f3658007540 comp rx=0 tx=0).stop 2024-08-19T10:49:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f3664d8c640 10 monclient: shutdown 2024-08-19T10:49:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f3664d8c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f3664d8c640 1 -- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36600a3d70 msgr2=0x7f3660073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f3664d8c640 1 --2- 172.21.15.89:0/3739942588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36600a3d70 0x7f3660073140 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f365401c2c0 tx=0x7f3654019230 comp rx=0 tx=0).stop 2024-08-19T10:49:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f365ed9d640 1 -- 172.21.15.89:0/3739942588 reap_dead start 2024-08-19T10:49:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f365ed9d640 1 -- 172.21.15.89:0/3739942588 reap_dead start 2024-08-19T10:49:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f3664d8c640 1 -- 172.21.15.89:0/3739942588 shutdown_connections 2024-08-19T10:49:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f3664d8c640 1 -- 172.21.15.89:0/3739942588 >> 172.21.15.89:0/3739942588 conn(0x7f36600a4e60 msgr2=0x7f366013cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f3664d8c640 1 -- 172.21.15.89:0/3739942588 shutdown_connections 2024-08-19T10:49:09.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.098+0000 7f3664d8c640 1 -- 172.21.15.89:0/3739942588 wait complete. 2024-08-19T10:49:09.133 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:09.133 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:49:09.133 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:49:09.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient: get_monmap_and_config 2024-08-19T10:49:09.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient: build_initial_monmap 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient: monmap: 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:09.257235+0000 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:09.257235+0000 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:09.253 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:09.254 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:09.254 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 1 Processor -- start 2024-08-19T10:49:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 1 -- start start 2024-08-19T10:49:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient: init 2024-08-19T10:49:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient: _add_conns ranks=[9,10,6,4,7,3,0,5,8,1,2] 2024-08-19T10:49:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f68200a6860 0x7f68200a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient(hunting): picked mon.noname-a con 0x7f68200a6860 addr v2:172.21.15.89:3300/0 2024-08-19T10:49:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68200a7180 0x7f68200a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient(hunting): picked mon.noname-f con 0x7f68200a7180 addr v2:172.21.15.190:3300/0 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6820139620 0x7f6820139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient(hunting): picked mon.noname-d con 0x7f6820139620 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f68200a8190 con 0x7f68200a6860 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f68200a33d0 con 0x7f68200a7180 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f68200a2ec0 con 0x7f6820139620 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f68249c5640 10 monclient(hunting): authenticate will time out at 1599.073782s 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6820139620 0x7f6820139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681e59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f68200a6860 0x7f68200a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6820139620 0x7f6820139a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:59060/0 (socket says 172.21.15.89:59060) 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 1 -- 172.21.15.89:0/1518287291 learned_addr learned my addr 172.21.15.89:0/1518287291 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 10 monclient(hunting): get_auth_request con 0x7f6820139620 auth_method 0 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681e59c640 10 monclient(hunting): get_auth_request con 0x7f68200a6860 auth_method 0 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 10 monclient(hunting): handle_auth_done global_id 44691 payload 274 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 1 -- 172.21.15.89:0/1518287291 >> v2:172.21.15.190:3300/0 conn(0x7f68200a7180 msgr2=0x7f68200a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681dd9b640 1 --2- 172.21.15.89:0/1518287291 >> v2:172.21.15.190:3300/0 conn(0x7f68200a7180 0x7f68200a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 1 --2- 172.21.15.89:0/1518287291 >> v2:172.21.15.190:3300/0 conn(0x7f68200a7180 0x7f68200a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 1 -- 172.21.15.89:0/1518287291 >> v2:172.21.15.89:3300/0 conn(0x7f68200a6860 msgr2=0x7f68200a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 1 --2- 172.21.15.89:0/1518287291 >> v2:172.21.15.89:3300/0 conn(0x7f68200a6860 0x7f68200a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 1 monclient: found mon.noname-d 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681ed9d640 1 -- 172.21.15.89:0/1518287291 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68200a4cd0 con 0x7f6820139620 2024-08-19T10:49:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.254+0000 7f681e59c640 1 --2- 172.21.15.89:0/1518287291 >> v2:172.21.15.89:3300/0 conn(0x7f68200a6860 0x7f68200a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:09.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681ed9d640 1 -- 172.21.15.89:0/1518287291 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6820139620 msgr2=0x7f6820139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681ed9d640 1 --2- 172.21.15.89:0/1518287291 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6820139620 0x7f6820139a60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68200a9a20 tx=0x7f681801c8b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=68cf3165d22ceb97 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 1 -- 172.21.15.89:0/1518287291 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68180090d0 con 0x7f6820139620 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 10 monclient: dump: 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.264 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 10 monclient: _finish_auth 0 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 1 -- 172.21.15.89:0/1518287291 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68180a59a0 con 0x7f6820139620 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 5 monclient: authenticate success, global_id 44691 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:09.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 1 -- 172.21.15.89:0/1518287291 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68180ad3f0 con 0x7f6820139620 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 4 set_mon_vals no callback set 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f681d59a640 10 monclient: dump: 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.266 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 10 monclient: shutdown 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 1 -- 172.21.15.89:0/1518287291 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6820139620 msgr2=0x7f6820139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 1 --2- 172.21.15.89:0/1518287291 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6820139620 0x7f6820139a60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68200a9a20 tx=0x7f681801c8b0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 1 -- 172.21.15.89:0/1518287291 shutdown_connections 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 1 --2- 172.21.15.89:0/1518287291 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6820139620 0x7f6820139a60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 1 --2- 172.21.15.89:0/1518287291 >> v2:172.21.15.190:3300/0 conn(0x7f68200a7180 0x7f68200a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 1 --2- 172.21.15.89:0/1518287291 >> v2:172.21.15.89:3300/0 conn(0x7f68200a6860 0x7f68200a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 1 -- 172.21.15.89:0/1518287291 >> 172.21.15.89:0/1518287291 conn(0x7f6820074b40 msgr2=0x7f68200abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 1 -- 172.21.15.89:0/1518287291 shutdown_connections 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.258+0000 7f68249c5640 1 -- 172.21.15.89:0/1518287291 wait complete. 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient: build_initial_monmap 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient: monmap: 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 1 Processor -- start 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 1 -- start start 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient: init 2024-08-19T10:49:09.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient: _add_conns ranks=[5,3,1,4,8,6,0,7,2] 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68200a6860 0x7f6820134bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient(hunting): picked mon.noname-h con 0x7f68200a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68200a7180 0x7f682009c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient(hunting): picked mon.noname-g con 0x7f68200a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6820139620 0x7f682009cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient(hunting): picked mon.noname-f con 0x7f6820139620 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6820143210 con 0x7f6820139620 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6820143090 con 0x7f68200a7180 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6820143390 con 0x7f68200a6860 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f68249c5640 10 monclient(hunting): authenticate will time out at 1599.081015s 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68200a6860 0x7f6820134bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6820139620 0x7f682009cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681dd9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68200a7180 0x7f682009c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68200a6860 0x7f6820134bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:34700/0 (socket says 172.21.15.89:34700) 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 1 -- 172.21.15.89:0/2458065513 learned_addr learned my addr 172.21.15.89:0/2458065513 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 10 monclient(hunting): get_auth_request con 0x7f68200a6860 auth_method 0 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681ed9d640 10 monclient(hunting): get_auth_request con 0x7f6820139620 auth_method 0 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681dd9b640 10 monclient(hunting): get_auth_request con 0x7f68200a7180 auth_method 0 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 10 monclient(hunting): handle_auth_done global_id 6827 payload 824 2024-08-19T10:49:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 1 -- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68200a7180 msgr2=0x7f682009c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68200a7180 0x7f682009c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 1 -- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6820139620 msgr2=0x7f682009cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6820139620 0x7f682009cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:49:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 1 monclient: found mon.noname-h 2024-08-19T10:49:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:49:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681e59c640 1 -- 172.21.15.89:0/2458065513 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6820140540 con 0x7f68200a6860 2024-08-19T10:49:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.262+0000 7f681ed9d640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6820139620 0x7f682009cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681e59c640 1 -- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68200a6860 msgr2=0x7f6820134bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681e59c640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68200a6860 0x7f6820134bf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f681000ebe0 tx=0x7f681000c520 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6810017070 con 0x7f68200a6860 2024-08-19T10:49:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:49:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 10 monclient: dump: 2024-08-19T10:49:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 10 monclient: _finish_auth 0 2024-08-19T10:49:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f68249c5640 5 monclient: authenticate success, global_id 6827 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f681000ce40 con 0x7f68200a6860 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681cd99640 4 set_mon_vals no callback set 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68100138e0 con 0x7f68200a6860 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 10 monclient: dump: 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f68249c5640 10 monclient: _renew_subs 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f68249c5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:49:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f68249c5640 1 -- 172.21.15.89:0/2458065513 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68200a04a0 con 0x7f68200a6860 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f68249c5640 10 monclient: _renew_subs 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f68249c5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f68249c5640 1 -- 172.21.15.89:0/2458065513 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f682009e140 con 0x7f68200a6860 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f68249c5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f68249c5640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6820135130 0x7f682009d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f68249c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f68249c5640 1 -- 172.21.15.89:0/2458065513 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f682009fd90 con 0x7f6820135130 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681dd9b640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6820135130 0x7f682009d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681dd9b640 10 monclient: get_auth_request con 0x7f6820135130 auth_method 0 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681dd9b640 10 monclient(hunting): handle_auth_done global_id 54371 payload 824 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681dd9b640 1 -- 172.21.15.89:0/2458065513 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68200a7560 con 0x7f6820135130 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f681001a070 con 0x7f68200a6860 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f680003be00 0x7f680003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f681000f040 con 0x7f68200a6860 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681ed9d640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f680003be00 0x7f680003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.266+0000 7f681ed9d640 10 monclient: get_auth_request con 0x7f680003be00 auth_method 0 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f681dd9b640 1 -- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6820135130 msgr2=0x7f682009d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f681dd9b640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6820135130 0x7f682009d460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f682009e7b0 tx=0x7f681400fc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f681400fe60 con 0x7f6820135130 2024-08-19T10:49:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f680effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f680effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f680effd640 10 monclient: dump: 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f681400fe60 con 0x7f6820135130 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f680effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f680effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6820135130 msgr2=0x7f682009d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f680effd640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6820135130 0x7f682009d460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f682009e7b0 tx=0x7f681400fc70 comp rx=0 tx=0).stop 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6820135130 msgr2=0x7f682009d460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f680effd640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6820135130 0x7f682009d460 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f682009e7b0 tx=0x7f681400fc70 comp rx=0 tx=0).stop 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.270+0000 7f681ed9d640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f680003be00 0x7f680003e2c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f682006cff0 tx=0x7f68180a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f68249c5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f68249c5640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f682009d9a0 0x7f682013aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f68249c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f68249c5640 1 -- 172.21.15.89:0/2458065513 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f682009ddb0 con 0x7f682009d9a0 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f681dd9b640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f682009d9a0 0x7f682013aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f681dd9b640 10 monclient: get_auth_request con 0x7f682009d9a0 auth_method 0 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f681dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f681dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f681dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f681dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f681dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f681dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f681dd9b640 10 monclient(hunting): handle_auth_done global_id 54380 payload 824 2024-08-19T10:49:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.274+0000 7f681dd9b640 1 -- 172.21.15.89:0/2458065513 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6820064590 con 0x7f682009d9a0 2024-08-19T10:49:09.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f681dd9b640 1 -- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f682009d9a0 msgr2=0x7f682013aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f681dd9b640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f682009d9a0 0x7f682013aa80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f682009e7b0 tx=0x7f68140046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f68140066e0 con 0x7f682009d9a0 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f680effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f680effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f680effd640 10 monclient: dump: 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.287 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f68140066e0 con 0x7f682009d9a0 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f680effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f680effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f682009d9a0 msgr2=0x7f682013aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f680effd640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f682009d9a0 0x7f682013aa80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f682009e7b0 tx=0x7f68140046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f680effd640 1 -- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f682009d9a0 msgr2=0x7f682013aa80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f680effd640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f682009d9a0 0x7f682013aa80 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f682009e7b0 tx=0x7f68140046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.289 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:09.289 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:09.289 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:09.289 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:09.289 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:09.289 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 144674, 2024-08-19T10:49:09.289 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:09.289 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:09.289 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:09.290 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:09.291 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:09.293 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.294 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.310 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:09.311 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:09.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f68249c5640 1 -- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f680003be00 msgr2=0x7f680003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f68249c5640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f680003be00 0x7f680003e2c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f682006cff0 tx=0x7f68180a7070 comp rx=0 tx=0).stop 2024-08-19T10:49:09.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f68249c5640 10 monclient: shutdown 2024-08-19T10:49:09.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f68249c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:09.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f68249c5640 1 -- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68200a6860 msgr2=0x7f6820134bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f68249c5640 1 --2- 172.21.15.89:0/2458065513 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f68200a6860 0x7f6820134bf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f681000ebe0 tx=0x7f681000c520 comp rx=0 tx=0).stop 2024-08-19T10:49:09.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f681ed9d640 1 -- 172.21.15.89:0/2458065513 reap_dead start 2024-08-19T10:49:09.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f681ed9d640 1 -- 172.21.15.89:0/2458065513 reap_dead start 2024-08-19T10:49:09.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f68249c5640 1 -- 172.21.15.89:0/2458065513 shutdown_connections 2024-08-19T10:49:09.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f68249c5640 1 -- 172.21.15.89:0/2458065513 >> 172.21.15.89:0/2458065513 conn(0x7f6820074b40 msgr2=0x7f68200ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:09.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f68249c5640 1 -- 172.21.15.89:0/2458065513 shutdown_connections 2024-08-19T10:49:09.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.278+0000 7f68249c5640 1 -- 172.21.15.89:0/2458065513 wait complete. 2024-08-19T10:49:09.314 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:09.314 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:49:09.314 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:49:09.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.430+0000 7f0a9e64d640 10 monclient: get_monmap_and_config 2024-08-19T10:49:09.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.430+0000 7f0a9e64d640 10 monclient: build_initial_monmap 2024-08-19T10:49:09.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.430+0000 7f0a9e64d640 10 monclient: monmap: 2024-08-19T10:49:09.431 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:09.436129+0000 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:09.436129+0000 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:09.432 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:09.433 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:09.433 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:09.433 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 1 Processor -- start 2024-08-19T10:49:09.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 1 -- start start 2024-08-19T10:49:09.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 10 monclient: init 2024-08-19T10:49:09.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:09.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 10 monclient: _add_conns ranks=[2,5,9,10,8,7,3,0,4,1,6] 2024-08-19T10:49:09.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a980a67d0 0x7f0a980a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 10 monclient(hunting): picked mon.noname-b con 0x7f0a980a67d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0a980a6bb0 0x7f0a980a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 10 monclient(hunting): picked mon.noname-h con 0x7f0a980a6bb0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0a98139590 0x7f0a98139a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 10 monclient(hunting): picked mon.noname-a con 0x7f0a98139590 addr v2:172.21.15.89:3300/0 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f0a980a8740 con 0x7f0a98139590 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f0a980a3380 con 0x7f0a980a67d0 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0a980a8250 con 0x7f0a980a6bb0 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9e64d640 10 monclient(hunting): authenticate will time out at 1599.252765s 2024-08-19T10:49:09.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a980a67d0 0x7f0a980a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9cbea640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0a98139590 0x7f0a98139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9cbea640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0a98139590 0x7f0a98139a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57698/0 (socket says 172.21.15.89:57698) 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a980a67d0 0x7f0a980a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54326/0 (socket says 172.21.15.89:54326) 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9cbea640 1 -- 172.21.15.89:0/2540333088 learned_addr learned my addr 172.21.15.89:0/2540333088 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9cbea640 10 monclient(hunting): get_auth_request con 0x7f0a98139590 auth_method 0 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9cbea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9cbea640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9cbea640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 10 monclient(hunting): get_auth_request con 0x7f0a980a67d0 auth_method 0 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9cbea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9cbea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9cbea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a977fe640 1 --2- 172.21.15.89:0/2540333088 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0a980a6bb0 0x7f0a980a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a977fe640 10 monclient(hunting): get_auth_request con 0x7f0a980a6bb0 auth_method 0 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 10 monclient(hunting): handle_auth_done global_id 54389 payload 274 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 1 -- 172.21.15.89:0/2540333088 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0a980a6bb0 msgr2=0x7f0a980a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:09.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 1 --2- 172.21.15.89:0/2540333088 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0a980a6bb0 0x7f0a980a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 1 -- 172.21.15.89:0/2540333088 >> v2:172.21.15.89:3300/0 conn(0x7f0a98139590 msgr2=0x7f0a98139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:09.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 1 --2- 172.21.15.89:0/2540333088 >> v2:172.21.15.89:3300/0 conn(0x7f0a98139590 0x7f0a98139a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 10 monclient: _finish_hunting 0 2024-08-19T10:49:09.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 1 monclient: found mon.noname-b 2024-08-19T10:49:09.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:09.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:49:09.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a97fff640 1 -- 172.21.15.89:0/2540333088 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980a4cc0 con 0x7f0a980a67d0 2024-08-19T10:49:09.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.434+0000 7f0a9cbea640 1 --2- 172.21.15.89:0/2540333088 >> v2:172.21.15.89:3300/0 conn(0x7f0a98139590 0x7f0a98139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:09.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a97fff640 1 -- 172.21.15.89:0/2540333088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a980a67d0 msgr2=0x7f0a980a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a97fff640 1 --2- 172.21.15.89:0/2540333088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a980a67d0 0x7f0a980a5500 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0a800097c0 tx=0x7f0a8001c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a392d339ea8a3018 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 1 -- 172.21.15.89:0/2540333088 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0a800b0070 con 0x7f0a980a67d0 2024-08-19T10:49:09.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:49:09.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 10 monclient: dump: 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.446 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 10 monclient: _finish_auth 0 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 5 monclient: authenticate success, global_id 54389 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 1 -- 172.21.15.89:0/2540333088 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a800aa810 con 0x7f0a980a67d0 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 4 set_mon_vals no callback set 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 1 -- 172.21.15.89:0/2540333088 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0a800aae30 con 0x7f0a980a67d0 2024-08-19T10:49:09.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a96ffd640 10 monclient: dump: 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.448 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 10 monclient: shutdown 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2540333088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a980a67d0 msgr2=0x7f0a980a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 1 --2- 172.21.15.89:0/2540333088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a980a67d0 0x7f0a980a5500 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0a800097c0 tx=0x7f0a8001c3f0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2540333088 shutdown_connections 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 1 --2- 172.21.15.89:0/2540333088 >> v2:172.21.15.89:3300/0 conn(0x7f0a98139590 0x7f0a98139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 1 --2- 172.21.15.89:0/2540333088 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0a980a6bb0 0x7f0a980a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 1 --2- 172.21.15.89:0/2540333088 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a980a67d0 0x7f0a980a5500 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2540333088 >> 172.21.15.89:0/2540333088 conn(0x7f0a98069ae0 msgr2=0x7f0a980abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2540333088 shutdown_connections 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2540333088 wait complete. 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 10 monclient: build_initial_monmap 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 10 monclient: monmap: 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:09.450 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 1 Processor -- start 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 1 -- start start 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.438+0000 7f0a9e64d640 10 monclient: init 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient: _add_conns ranks=[6,0,7,5,4,3,2,8,1] 2024-08-19T10:49:09.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980a67d0 0x7f0a981347a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient(hunting): picked mon.noname-d con 0x7f0a980a67d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980a6bb0 0x7f0a98134ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient(hunting): picked mon.noname-a con 0x7f0a980a6bb0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a98139590 0x7f0a9809c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient(hunting): picked mon.noname-i con 0x7f0a98139590 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0a98143390 con 0x7f0a980a6bb0 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f0a98143210 con 0x7f0a980a67d0 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0a98143510 con 0x7f0a98139590 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient(hunting): authenticate will time out at 1599.260131s 2024-08-19T10:49:09.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980a6bb0 0x7f0a98134ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980a67d0 0x7f0a981347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980a6bb0 0x7f0a98134ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57710/0 (socket says 172.21.15.89:57710) 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 1 -- 172.21.15.89:0/2473270764 learned_addr learned my addr 172.21.15.89:0/2473270764 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 10 monclient(hunting): get_auth_request con 0x7f0a980a6bb0 auth_method 0 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980a67d0 0x7f0a981347a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:59074/0 (socket says 172.21.15.89:59074) 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): get_auth_request con 0x7f0a980a67d0 auth_method 0 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 10 monclient(hunting): handle_auth_done global_id 6840 payload 824 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 1 -- 172.21.15.89:0/2473270764 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a98139590 msgr2=0x7f0a9809c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a98139590 0x7f0a9809c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 1 -- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980a67d0 msgr2=0x7f0a981347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980a67d0 0x7f0a981347a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 10 monclient: _finish_hunting 0 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 1 monclient: found mon.noname-a 2024-08-19T10:49:09.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 1 -- 172.21.15.89:0/2473270764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980a0200 con 0x7f0a980a6bb0 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 1 -- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980a6bb0 msgr2=0x7f0a98134ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a977fe640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980a6bb0 0x7f0a98134ce0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0a90007ef0 tx=0x7f0a9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0a90013a00 con 0x7f0a980a6bb0 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 10 monclient: dump: 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.455 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 10 monclient: _finish_auth 0 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:49:09.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a90019070 con 0x7f0a980a6bb0 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 5 monclient: authenticate success, global_id 6840 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient: _renew_subs 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2473270764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a980a04e0 con 0x7f0a980a6bb0 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0a90013be0 con 0x7f0a980a6bb0 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a94ff9640 10 monclient: dump: 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.457 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient: _renew_subs 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a967fc640 4 set_mon_vals no callback set 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2473270764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9809e180 con 0x7f0a980a6bb0 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809d6e0 0x7f0a9802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2473270764 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f0a980a5500 con 0x7f0a9809d6e0 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809d6e0 0x7f0a9802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient: get_auth_request con 0x7f0a9809d6e0 auth_method 0 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.442+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0a90013590 con 0x7f0a980a6bb0 2024-08-19T10:49:09.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a97fff640 10 monclient(hunting): handle_auth_done global_id 44689 payload 824 2024-08-19T10:49:09.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a97fff640 1 -- 172.21.15.89:0/2473270764 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a98064590 con 0x7f0a9809d6e0 2024-08-19T10:49:09.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0a7803be30 0x7f0a7803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f0a9004fea0 con 0x7f0a980a6bb0 2024-08-19T10:49:09.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a9cbea640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0a7803be30 0x7f0a7803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a9cbea640 10 monclient: get_auth_request con 0x7f0a7803be30 auth_method 0 2024-08-19T10:49:09.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a97fff640 1 -- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809d6e0 msgr2=0x7f0a9802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a97fff640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809d6e0 0x7f0a9802d1d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0a980acce0 tx=0x7f0a80019190 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0a800b0040 con 0x7f0a9809d6e0 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 10 monclient: dump: 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.463 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0a80003910 con 0x7f0a9809d6e0 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809d6e0 msgr2=0x7f0a9802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809d6e0 0x7f0a9802d1d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0a980acce0 tx=0x7f0a80019190 comp rx=0 tx=0).stop 2024-08-19T10:49:09.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809d6e0 msgr2=0x7f0a9802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a94ff9640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809d6e0 0x7f0a9802d1d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.446+0000 7f0a9cbea640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0a7803be30 0x7f0a7803e2f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0a880099b0 tx=0x7f0a88009450 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a9e64d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a9e64d640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809dac0 0x7f0a9802d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a9e64d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2473270764 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f0a9809f910 con 0x7f0a9809dac0 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809dac0 0x7f0a9802d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 10 monclient: get_auth_request con 0x7f0a9809dac0 auth_method 0 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 10 monclient(hunting): handle_auth_done global_id 44698 payload 824 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 1 -- 172.21.15.89:0/2473270764 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0a9806ca20 con 0x7f0a9809dac0 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 1 -- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809dac0 msgr2=0x7f0a9802d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.454+0000 7f0a97fff640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809dac0 0x7f0a9802d710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c10 tx=0x7f0a800a8000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0a800b0070 con 0x7f0a9809dac0 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a94ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a94ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a94ff9640 10 monclient: dump: 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.466 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f0a800b5d30 con 0x7f0a9809dac0 2024-08-19T10:49:09.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a94ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:09.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a94ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:09.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809dac0 msgr2=0x7f0a9802d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a94ff9640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809dac0 0x7f0a9802d710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c10 tx=0x7f0a800a8000 comp rx=0 tx=0).stop 2024-08-19T10:49:09.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a94ff9640 1 -- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809dac0 msgr2=0x7f0a9802d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:09.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a94ff9640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0a9809dac0 0x7f0a9802d710 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c10 tx=0x7f0a800a8000 comp rx=0 tx=0).stop 2024-08-19T10:49:09.468 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:09.468 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:09.468 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:09.468 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:09.468 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 272585, 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.469 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.470 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.471 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:09.472 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.473 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.474 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.475 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:09.476 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.477 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.478 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:09.479 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:09.482 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.483 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.484 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:09.485 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.486 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.487 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:09.488 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.490 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:09.491 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2473270764 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0a7803be30 msgr2=0x7f0a7803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9e64d640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0a7803be30 0x7f0a7803e2f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0a880099b0 tx=0x7f0a88009450 comp rx=0 tx=0).stop 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9e64d640 10 monclient: shutdown 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9e64d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980a6bb0 msgr2=0x7f0a98134ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9e64d640 1 --2- 172.21.15.89:0/2473270764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980a6bb0 0x7f0a98134ce0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0a90007ef0 tx=0x7f0a9000e4b0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9cbea640 1 -- 172.21.15.89:0/2473270764 reap_dead start 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9cbea640 1 -- 172.21.15.89:0/2473270764 reap_dead start 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2473270764 shutdown_connections 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2473270764 >> 172.21.15.89:0/2473270764 conn(0x7f0a98069ae0 msgr2=0x7f0a98137890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2473270764 shutdown_connections 2024-08-19T10:49:09.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.458+0000 7f0a9e64d640 1 -- 172.21.15.89:0/2473270764 wait complete. 2024-08-19T10:49:09.494 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:09.494 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:49:09.494 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:49:09.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient: get_monmap_and_config 2024-08-19T10:49:09.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient: build_initial_monmap 2024-08-19T10:49:09.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient: monmap: 2024-08-19T10:49:09.613 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:09.613 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.613 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:09.617673+0000 2024-08-19T10:49:09.613 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:09.617673+0000 2024-08-19T10:49:09.613 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:09.614 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 1 Processor -- start 2024-08-19T10:49:09.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 1 -- start start 2024-08-19T10:49:09.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient: init 2024-08-19T10:49:09.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:09.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient: _add_conns ranks=[6,4,9,3,10,7,8,0,5,2,1] 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c06de10 0x7fdb7c135ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient(hunting): picked mon.noname-d con 0x7fdb7c06de10 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb7c073f00 0x7fdb7c139200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient(hunting): picked mon.noname-c con 0x7fdb7c073f00 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb7c074320 0x7fdb7c139740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient(hunting): picked mon.noname-a con 0x7fdb7c074320 addr v2:172.21.15.89:3300/0 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7c06e1f0 con 0x7fdb7c074320 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fdb7c06f190 con 0x7fdb7c073f00 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fdb7c06f470 con 0x7fdb7c06de10 2024-08-19T10:49:09.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb8223f640 10 monclient(hunting): authenticate will time out at 1599.434416s 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7affd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb7c073f00 0x7fdb7c139200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdb7c074320 0x7fdb7c139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7affd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb7c073f00 0x7fdb7c139200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58450/0 (socket says 172.21.15.89:58450) 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7affd640 1 -- 172.21.15.89:0/1180313117 learned_addr learned my addr 172.21.15.89:0/1180313117 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7affd640 10 monclient(hunting): get_auth_request con 0x7fdb7c073f00 auth_method 0 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 10 monclient(hunting): get_auth_request con 0x7fdb7c074320 auth_method 0 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7b7fe640 1 --2- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c06de10 0x7fdb7c135ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7b7fe640 10 monclient(hunting): get_auth_request con 0x7fdb7c06de10 auth_method 0 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 10 monclient(hunting): handle_auth_done global_id 6849 payload 274 2024-08-19T10:49:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 1 -- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c06de10 msgr2=0x7fdb7c135ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 1 --2- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c06de10 0x7fdb7c135ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 1 -- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb7c073f00 msgr2=0x7fdb7c139200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 1 --2- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb7c073f00 0x7fdb7c139200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:49:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 1 monclient: found mon.noname-a 2024-08-19T10:49:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7bfff640 1 -- 172.21.15.89:0/1180313117 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c06f750 con 0x7fdb7c074320 2024-08-19T10:49:09.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.614+0000 7fdb7affd640 1 --2- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb7c073f00 0x7fdb7c139200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7bfff640 1 -- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c074320 msgr2=0x7fdb7c139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7bfff640 1 --2- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c074320 0x7fdb7c139740 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0097f0 tx=0x7fdb6c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9fff2cfbd6873e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 1 -- 172.21.15.89:0/1180313117 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb6c0b0070 con 0x7fdb7c074320 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 10 monclient: dump: 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.625 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:49:09.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 1 -- 172.21.15.89:0/1180313117 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6c0aa810 con 0x7fdb7c074320 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb8223f640 5 monclient: authenticate success, global_id 6849 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb8223f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb8223f640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 1 -- 172.21.15.89:0/1180313117 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb6c0aae30 con 0x7fdb7c074320 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb8223f640 4 set_mon_vals no callback set 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.618+0000 7fdb7a7fc640 10 monclient: dump: 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.627 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient: shutdown 2024-08-19T10:49:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 -- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c074320 msgr2=0x7fdb7c139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 --2- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c074320 0x7fdb7c139740 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0097f0 tx=0x7fdb6c01c360 comp rx=0 tx=0).stop 2024-08-19T10:49:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 -- 172.21.15.89:0/1180313117 shutdown_connections 2024-08-19T10:49:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 --2- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c074320 0x7fdb7c139740 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 --2- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdb7c073f00 0x7fdb7c139200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 --2- 172.21.15.89:0/1180313117 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c06de10 0x7fdb7c135ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 -- 172.21.15.89:0/1180313117 >> 172.21.15.89:0/1180313117 conn(0x7fdb7c0aa240 msgr2=0x7fdb7c0aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 -- 172.21.15.89:0/1180313117 shutdown_connections 2024-08-19T10:49:09.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 -- 172.21.15.89:0/1180313117 wait complete. 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient: build_initial_monmap 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient: monmap: 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:09.630 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 Processor -- start 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 -- start start 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient: init 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient: _add_conns ranks=[5,0,7,8,1,6,3,4,2] 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdb7c06de10 0x7fdb7c1d1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient(hunting): picked mon.noname-h con 0x7fdb7c06de10 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c073f00 0x7fdb7c1d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient(hunting): picked mon.noname-a con 0x7fdb7c073f00 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:09.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdb7c074320 0x7fdb7c13a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient(hunting): picked mon.noname-i con 0x7fdb7c074320 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:09.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c142fc0 con 0x7fdb7c073f00 2024-08-19T10:49:09.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fdb7c13a090 con 0x7fdb7c06de10 2024-08-19T10:49:09.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fdb7c143140 con 0x7fdb7c074320 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb8223f640 10 monclient(hunting): authenticate will time out at 1599.441586s 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c073f00 0x7fdb7c1d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c073f00 0x7fdb7c1d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57738/0 (socket says 172.21.15.89:57738) 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 1 -- 172.21.15.89:0/1134251508 learned_addr learned my addr 172.21.15.89:0/1134251508 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 10 monclient(hunting): get_auth_request con 0x7fdb7c073f00 auth_method 0 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7b7fe640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdb7c06de10 0x7fdb7c1d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 10 monclient(hunting): handle_auth_done global_id 6858 payload 824 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 1 -- 172.21.15.89:0/1134251508 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdb7c074320 msgr2=0x7fdb7c13a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdb7c074320 0x7fdb7c13a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 1 -- 172.21.15.89:0/1134251508 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdb7c06de10 msgr2=0x7fdb7c1d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdb7c06de10 0x7fdb7c1d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 10 monclient: _finish_hunting 0 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 1 monclient: found mon.noname-a 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.622+0000 7fdb7affd640 1 -- 172.21.15.89:0/1134251508 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7c13d730 con 0x7fdb7c073f00 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7affd640 1 -- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c073f00 msgr2=0x7fdb7c1d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7affd640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c073f00 0x7fdb7c1d18a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400e930 tx=0x7fdb7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb74017070 con 0x7fdb7c073f00 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 10 monclient: dump: 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.635 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 10 monclient: _finish_auth 0 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 10 monclient: _check_auth_tickets 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7400cc10 con 0x7fdb7c073f00 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb8223f640 5 monclient: authenticate success, global_id 6858 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb8223f640 10 monclient: _renew_subs 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb8223f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb8223f640 1 -- 172.21.15.89:0/1134251508 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7c13da10 con 0x7fdb7c073f00 2024-08-19T10:49:09.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb7400cdc0 con 0x7fdb7c073f00 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 10 monclient: dump: 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.637 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.638 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb8223f640 10 monclient: _renew_subs 2024-08-19T10:49:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb8223f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb79ffb640 4 set_mon_vals no callback set 2024-08-19T10:49:09.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb8223f640 1 -- 172.21.15.89:0/1134251508 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7c13c4f0 con 0x7fdb7c073f00 2024-08-19T10:49:09.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb8223f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:09.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb8223f640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13a800 0x7fdb7c13abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb8223f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb8223f640 1 -- 172.21.15.89:0/1134251508 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fdb7c135ff0 con 0x7fdb7c13a800 2024-08-19T10:49:09.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7b7fe640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13a800 0x7fdb7c13abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7b7fe640 10 monclient: get_auth_request con 0x7fdb7c13a800 auth_method 0 2024-08-19T10:49:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fdb7401a070 con 0x7fdb7c073f00 2024-08-19T10:49:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdb5c03be00 0x7fdb5c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_done global_id 44718 payload 824 2024-08-19T10:49:09.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7b7fe640 1 -- 172.21.15.89:0/1134251508 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb7c064590 con 0x7fdb7c13a800 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7fdb7400f040 con 0x7fdb7c073f00 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7bfff640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdb5c03be00 0x7fdb5c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.626+0000 7fdb7bfff640 10 monclient: get_auth_request con 0x7fdb5c03be00 auth_method 0 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb7b7fe640 1 -- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13a800 msgr2=0x7fdb7c13abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb7b7fe640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13a800 0x7fdb7c13abe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400f3b0 tx=0x7fdb6400f870 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb64014a50 con 0x7fdb7c13a800 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb73fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb73fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb73fff640 10 monclient: dump: 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.644 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdb6401b070 con 0x7fdb7c13a800 2024-08-19T10:49:09.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb73fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb73fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13a800 msgr2=0x7fdb7c13abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb73fff640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13a800 0x7fdb7c13abe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400f3b0 tx=0x7fdb6400f870 comp rx=0 tx=0).stop 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13a800 msgr2=0x7fdb7c13abe0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb73fff640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13a800 0x7fdb7c13abe0 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400f3b0 tx=0x7fdb6400f870 comp rx=0 tx=0).stop 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.630+0000 7fdb7bfff640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdb5c03be00 0x7fdb5c03e2c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0097f0 tx=0x7fdb6c004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb8223f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb8223f640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13b120 0x7fdb7c0a7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb8223f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb8223f640 1 -- 172.21.15.89:0/1134251508 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fdb7c13d450 con 0x7fdb7c13b120 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb7b7fe640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13b120 0x7fdb7c0a7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb7b7fe640 10 monclient: get_auth_request con 0x7fdb7c13b120 auth_method 0 2024-08-19T10:49:09.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb7b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb7b7fe640 10 monclient(hunting): handle_auth_done global_id 44727 payload 824 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.634+0000 7fdb7b7fe640 1 -- 172.21.15.89:0/1134251508 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdb7c140130 con 0x7fdb7c13b120 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb7b7fe640 1 -- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13b120 msgr2=0x7fdb7c0a7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb7b7fe640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13b120 0x7fdb7c0a7c40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdb6401a000 tx=0x7fdb64004a50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdb64020e40 con 0x7fdb7c13b120 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb73fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb73fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:09.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb73fff640 10 monclient: dump: 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.648 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.649 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7fdb64016420 con 0x7fdb7c13b120 2024-08-19T10:49:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb73fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb73fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13b120 msgr2=0x7fdb7c0a7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb73fff640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13b120 0x7fdb7c0a7c40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdb6401a000 tx=0x7fdb64004a50 comp rx=0 tx=0).stop 2024-08-19T10:49:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb73fff640 1 -- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13b120 msgr2=0x7fdb7c0a7c40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:09.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb73fff640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdb7c13b120 0x7fdb7c0a7c40 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdb6401a000 tx=0x7fdb64004a50 comp rx=0 tx=0).stop 2024-08-19T10:49:09.649 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:09.649 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 272765, 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:09.650 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.651 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.652 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.652 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.652 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.652 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.652 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.652 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.652 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:09.652 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.653 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:09.654 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.655 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:09.657 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:09.660 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:09.661 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.663 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.664 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.666 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.667 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:09.669 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.671 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.672 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.673 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.673 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:09.673 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.673 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.673 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:09.673 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:09.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb8223f640 1 -- 172.21.15.89:0/1134251508 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdb5c03be00 msgr2=0x7fdb5c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb8223f640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdb5c03be00 0x7fdb5c03e2c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0097f0 tx=0x7fdb6c004740 comp rx=0 tx=0).stop 2024-08-19T10:49:09.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb8223f640 10 monclient: shutdown 2024-08-19T10:49:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb8223f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb8223f640 1 -- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c073f00 msgr2=0x7fdb7c1d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb7bfff640 1 -- 172.21.15.89:0/1134251508 reap_dead start 2024-08-19T10:49:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb8223f640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c073f00 0x7fdb7c1d18a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400e930 tx=0x7fdb7400edf0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb8223f640 1 -- 172.21.15.89:0/1134251508 shutdown_connections 2024-08-19T10:49:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb8223f640 1 --2- 172.21.15.89:0/1134251508 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb7c073f00 0x7fdb7c1d18a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb8223f640 1 -- 172.21.15.89:0/1134251508 >> 172.21.15.89:0/1134251508 conn(0x7fdb7c0aa240 msgr2=0x7fdb7c134660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb8223f640 1 -- 172.21.15.89:0/1134251508 shutdown_connections 2024-08-19T10:49:09.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.638+0000 7fdb8223f640 1 -- 172.21.15.89:0/1134251508 wait complete. 2024-08-19T10:49:09.675 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:09.676 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:49:09.676 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:49:09.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient: get_monmap_and_config 2024-08-19T10:49:09.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient: build_initial_monmap 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient: monmap: 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:09.798435+0000 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:09.798435+0000 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:09.794 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:09.795 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:09.795 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 1 Processor -- start 2024-08-19T10:49:09.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 1 -- start start 2024-08-19T10:49:09.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient: init 2024-08-19T10:49:09.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:09.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient: _add_conns ranks=[3,8,5,4,0,1,7,6,9,10,2] 2024-08-19T10:49:09.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b201364f0 0x7f4b201368d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient(hunting): picked mon.noname-g con 0x7f4b201364f0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20136e10 0x7f4b200a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient(hunting): picked mon.noname-e con 0x7f4b20136e10 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b200a4090 0x7f4b200a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient(hunting): picked mon.noname-h con 0x7f4b200a4090 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4b200a2330 con 0x7f4b201364f0 2024-08-19T10:49:09.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4b200a5610 con 0x7f4b200a4090 2024-08-19T10:49:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4b200a0770 con 0x7f4b20136e10 2024-08-19T10:49:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b27d0b640 10 monclient(hunting): authenticate will time out at 1599.615072s 2024-08-19T10:49:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20136e10 0x7f4b200a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20136e10 0x7f4b200a3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:46722/0 (socket says 172.21.15.89:46722) 2024-08-19T10:49:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 1 -- 172.21.15.89:0/645666300 learned_addr learned my addr 172.21.15.89:0/645666300 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 10 monclient(hunting): get_auth_request con 0x7f4b20136e10 auth_method 0 2024-08-19T10:49:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b262a8640 1 --2- 172.21.15.89:0/645666300 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b200a4090 0x7f4b200a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b262a8640 10 monclient(hunting): get_auth_request con 0x7f4b200a4090 auth_method 0 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b262a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b262a8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b262a8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b25aa7640 1 --2- 172.21.15.89:0/645666300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b201364f0 0x7f4b201368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 10 monclient(hunting): handle_auth_done global_id 6695 payload 274 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 1 -- 172.21.15.89:0/645666300 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b200a4090 msgr2=0x7f4b200a44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:09.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 1 --2- 172.21.15.89:0/645666300 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b200a4090 0x7f4b200a44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 1 -- 172.21.15.89:0/645666300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b201364f0 msgr2=0x7f4b201368d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 1 --2- 172.21.15.89:0/645666300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b201364f0 0x7f4b201368d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 10 monclient: _finish_hunting 0 2024-08-19T10:49:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 1 monclient: found mon.noname-e 2024-08-19T10:49:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b252a6640 1 -- 172.21.15.89:0/645666300 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b200a09f0 con 0x7f4b20136e10 2024-08-19T10:49:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b25aa7640 10 monclient: get_auth_request con 0x7f4b201364f0 auth_method 0 2024-08-19T10:49:09.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.794+0000 7f4b25aa7640 1 --2- 172.21.15.89:0/645666300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b201364f0 0x7f4b201368d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:09.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.798+0000 7f4b252a6640 1 -- 172.21.15.89:0/645666300 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20136e10 msgr2=0x7f4b200a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.798+0000 7f4b252a6640 1 --2- 172.21.15.89:0/645666300 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20136e10 0x7f4b200a3b50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009eb0 tx=0x7f4b1c01c8f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f4b0996b999228b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 1 -- 172.21.15.89:0/645666300 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b1c0090d0 con 0x7f4b20136e10 2024-08-19T10:49:09.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:09.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 10 monclient: dump: 2024-08-19T10:49:09.806 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.806 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.806 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.806 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.806 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.806 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.807 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.807 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.807 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.807 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.807 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.807 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.807 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.807 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.807 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.807 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 10 monclient: _finish_auth 0 2024-08-19T10:49:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 10 monclient: _check_auth_tickets 2024-08-19T10:49:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 1 -- 172.21.15.89:0/645666300 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1c0a6950 con 0x7f4b20136e10 2024-08-19T10:49:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 1 -- 172.21.15.89:0/645666300 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b1c0af480 con 0x7f4b20136e10 2024-08-19T10:49:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 5 monclient: authenticate success, global_id 6695 2024-08-19T10:49:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 4 set_mon_vals no callback set 2024-08-19T10:49:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b24aa5640 10 monclient: dump: 2024-08-19T10:49:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.809 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient: shutdown 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:09.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 -- 172.21.15.89:0/645666300 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20136e10 msgr2=0x7f4b200a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 --2- 172.21.15.89:0/645666300 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20136e10 0x7f4b200a3b50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009eb0 tx=0x7f4b1c01c8f0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 -- 172.21.15.89:0/645666300 shutdown_connections 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 --2- 172.21.15.89:0/645666300 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b200a4090 0x7f4b200a44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 --2- 172.21.15.89:0/645666300 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20136e10 0x7f4b200a3b50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 --2- 172.21.15.89:0/645666300 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b201364f0 0x7f4b201368d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 -- 172.21.15.89:0/645666300 >> 172.21.15.89:0/645666300 conn(0x7f4b200a9130 msgr2=0x7f4b200a9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 -- 172.21.15.89:0/645666300 shutdown_connections 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 -- 172.21.15.89:0/645666300 wait complete. 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient: build_initial_monmap 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient: monmap: 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:09.811 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 Processor -- start 2024-08-19T10:49:09.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 -- start start 2024-08-19T10:49:09.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient: init 2024-08-19T10:49:09.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:09.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient: _add_conns ranks=[7,5,1,8,2,4,6,3,0] 2024-08-19T10:49:09.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b200a4090 0x7f4b20135420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient(hunting): picked mon.noname-i con 0x7f4b200a4090 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:09.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b201364f0 0x7f4b2013a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient(hunting): picked mon.noname-h con 0x7f4b201364f0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:09.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b20136e10 0x7f4b2013a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient(hunting): picked mon.noname-f con 0x7f4b20136e10 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4b201371f0 con 0x7f4b20136e10 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4b200a4a30 con 0x7f4b201364f0 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4b20073a10 con 0x7f4b200a4090 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b27d0b640 10 monclient(hunting): authenticate will time out at 1599.622908s 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b201364f0 0x7f4b2013a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b201364f0 0x7f4b2013a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:34728/0 (socket says 172.21.15.89:34728) 2024-08-19T10:49:09.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 1 -- 172.21.15.89:0/2953131504 learned_addr learned my addr 172.21.15.89:0/2953131504 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 10 monclient(hunting): get_auth_request con 0x7f4b201364f0 auth_method 0 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b25aa7640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b200a4090 0x7f4b20135420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b262a8640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b20136e10 0x7f4b2013a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b262a8640 10 monclient(hunting): get_auth_request con 0x7f4b20136e10 auth_method 0 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b262a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b262a8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b262a8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b25aa7640 10 monclient(hunting): get_auth_request con 0x7f4b200a4090 auth_method 0 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b25aa7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b25aa7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b25aa7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b262a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b262a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b262a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b25aa7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b25aa7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b25aa7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 10 monclient(hunting): handle_auth_done global_id 6854 payload 824 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 1 -- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b200a4090 msgr2=0x7f4b20135420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b200a4090 0x7f4b20135420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 1 -- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b20136e10 msgr2=0x7f4b2013a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b20136e10 0x7f4b2013a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 10 monclient: _finish_hunting 0 2024-08-19T10:49:09.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 1 monclient: found mon.noname-h 2024-08-19T10:49:09.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:09.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:49:09.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.802+0000 7f4b252a6640 1 -- 172.21.15.89:0/2953131504 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b20120520 con 0x7f4b201364f0 2024-08-19T10:49:09.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b252a6640 1 -- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b201364f0 msgr2=0x7f4b2013a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b252a6640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b201364f0 0x7f4b2013a370 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c01c720 tx=0x7f4b1c019690 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b1c0be070 con 0x7f4b201364f0 2024-08-19T10:49:09.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:49:09.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 10 monclient: dump: 2024-08-19T10:49:09.822 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.822 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.823 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.823 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.823 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.823 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.823 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.823 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.823 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.823 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.823 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.824 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.824 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.824 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.824 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.824 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 10 monclient: _finish_auth 0 2024-08-19T10:49:09.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 10 monclient: _check_auth_tickets 2024-08-19T10:49:09.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:09.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1c0a6770 con 0x7f4b201364f0 2024-08-19T10:49:09.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:09.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b1c003e10 con 0x7f4b201364f0 2024-08-19T10:49:09.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:09.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b167fc640 10 monclient: dump: 2024-08-19T10:49:09.826 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.826 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.826 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.827 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.828 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.828 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.828 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.828 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b17fff640 4 set_mon_vals no callback set 2024-08-19T10:49:09.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b27d0b640 5 monclient: authenticate success, global_id 6854 2024-08-19T10:49:09.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b27d0b640 10 monclient: _renew_subs 2024-08-19T10:49:09.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b27d0b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:49:09.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b27d0b640 1 -- 172.21.15.89:0/2953131504 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b200a11d0 con 0x7f4b201364f0 2024-08-19T10:49:09.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b27d0b640 10 monclient: _renew_subs 2024-08-19T10:49:09.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b27d0b640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:49:09.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b27d0b640 1 -- 172.21.15.89:0/2953131504 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b20120dc0 con 0x7f4b201364f0 2024-08-19T10:49:09.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b27d0b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:09.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b27d0b640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b2011f5c0 0x7f4b2013adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b27d0b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b27d0b640 1 -- 172.21.15.89:0/2953131504 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4b20137370 con 0x7f4b2011f5c0 2024-08-19T10:49:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b25aa7640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b2011f5c0 0x7f4b2013adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b25aa7640 10 monclient: get_auth_request con 0x7f4b2011f5c0 auth_method 0 2024-08-19T10:49:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b25aa7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b25aa7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b25aa7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b25aa7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b25aa7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.806+0000 7f4b25aa7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b25aa7640 10 monclient(hunting): handle_auth_done global_id 6704 payload 824 2024-08-19T10:49:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b25aa7640 1 -- 172.21.15.89:0/2953131504 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b201368d0 con 0x7f4b2011f5c0 2024-08-19T10:49:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4b1c0a6d10 con 0x7f4b201364f0 2024-08-19T10:49:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b0c03bdf0 0x7f4b0c03e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f4b1c0b5030 con 0x7f4b201364f0 2024-08-19T10:49:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b262a8640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b0c03bdf0 0x7f4b0c03e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b262a8640 10 monclient: get_auth_request con 0x7f4b0c03bdf0 auth_method 0 2024-08-19T10:49:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b25aa7640 1 -- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b2011f5c0 msgr2=0x7f4b2013adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b25aa7640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b2011f5c0 0x7f4b2013adf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4b18011d90 tx=0x7f4b1800f7a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b1801e070 con 0x7f4b2011f5c0 2024-08-19T10:49:09.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 10 monclient: dump: 2024-08-19T10:49:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.833 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.834 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.836 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4b1800e7e0 con 0x7f4b2011f5c0 2024-08-19T10:49:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b2011f5c0 msgr2=0x7f4b2013adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b2011f5c0 0x7f4b2013adf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4b18011d90 tx=0x7f4b1800f7a0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b2011f5c0 msgr2=0x7f4b2013adf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b167fc640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b2011f5c0 0x7f4b2013adf0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.810+0000 7f4b262a8640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b0c03bdf0 0x7f4b0c03e2b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4b10006aa0 tx=0x7f4b10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b27d0b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b27d0b640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20135960 0x7f4b2013b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b27d0b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b27d0b640 1 -- 172.21.15.89:0/2953131504 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4b20064590 con 0x7f4b20135960 2024-08-19T10:49:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20135960 0x7f4b2013b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 10 monclient: get_auth_request con 0x7f4b20135960 auth_method 0 2024-08-19T10:49:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 10 monclient(hunting): handle_auth_done global_id 6713 payload 824 2024-08-19T10:49:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 1 -- 172.21.15.89:0/2953131504 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4b20120900 con 0x7f4b20135960 2024-08-19T10:49:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 1 -- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20135960 msgr2=0x7f4b2013b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b25aa7640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20135960 0x7f4b2013b330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4b18000c10 tx=0x7f4b180046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b1801e040 con 0x7f4b20135960 2024-08-19T10:49:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b167fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.818+0000 7f4b167fc640 10 monclient: dump: 2024-08-19T10:49:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:09.842 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:09.842 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:09.842 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:09.842 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:09.842 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:09.843 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f4b18006000 con 0x7f4b20135960 2024-08-19T10:49:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b167fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b167fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20135960 msgr2=0x7f4b2013b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b167fc640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20135960 0x7f4b2013b330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4b18000c10 tx=0x7f4b180046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b167fc640 1 -- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20135960 msgr2=0x7f4b2013b330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b167fc640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b20135960 0x7f4b2013b330 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4b18000c10 tx=0x7f4b180046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:09.845 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:09.845 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:09.845 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:09.846 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:09.846 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:09.846 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 679611, 2024-08-19T10:49:09.846 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:09.846 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:09.847 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:09.847 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:09.847 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:09.847 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:09.847 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:09.847 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:09.848 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:09.848 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:09.848 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.848 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:49:09.848 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:09.848 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:09.849 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:09.849 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.849 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.849 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.849 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.849 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.850 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.850 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.850 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.850 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.850 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.851 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.851 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.851 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:09.851 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:09.851 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.851 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.852 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.852 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.852 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.852 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.852 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.852 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.853 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.853 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.853 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.853 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.853 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:09.854 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:09.854 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:09.854 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:09.854 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:09.854 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:09.855 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:09.855 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:09.855 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:09.855 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:09.855 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:09.855 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:09.856 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:09.856 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:09.856 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:09.856 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:09.856 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:09.856 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:09.857 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:09.857 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:09.857 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:09.857 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:09.857 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:09.857 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:09.857 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:09.858 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:09.858 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:09.858 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:09.858 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.858 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:09.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.859 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:09.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.859 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:09.859 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:09.859 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.859 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.860 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:09.860 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.860 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:09.860 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.860 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.860 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.861 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:09.862 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.862 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:09.862 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.862 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.862 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:09.863 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:09.863 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.863 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.864 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.864 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:09.864 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:09.864 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.865 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.867 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:09.870 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:09.871 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.872 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:09.872 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.872 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.872 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:09.872 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:09.872 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.874 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.874 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:09.874 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.874 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.874 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.874 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:09.874 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:09.874 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.874 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.874 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.876 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:09.876 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:09.876 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:09.876 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.876 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.876 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:09.876 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:09.876 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:09.876 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:09.877 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:09.877 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.877 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.877 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:09.877 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:09.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.877 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.878 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.878 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:09.878 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.878 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.878 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:09.878 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.878 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.878 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.878 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:49:09.879 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.879 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:09.879 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:09.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:09.879 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:09.879 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:09.880 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:09.880 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:09.880 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:09.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:09.880 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:09.880 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:09.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b27d0b640 1 -- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b0c03bdf0 msgr2=0x7f4b0c03e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b27d0b640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b0c03bdf0 0x7f4b0c03e2b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4b10006aa0 tx=0x7f4b10008040 comp rx=0 tx=0).stop 2024-08-19T10:49:09.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b27d0b640 10 monclient: shutdown 2024-08-19T10:49:09.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b27d0b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:09.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b27d0b640 1 -- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b201364f0 msgr2=0x7f4b2013a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:09.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b27d0b640 1 --2- 172.21.15.89:0/2953131504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4b201364f0 0x7f4b2013a370 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c01c720 tx=0x7f4b1c019690 comp rx=0 tx=0).stop 2024-08-19T10:49:09.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b262a8640 1 -- 172.21.15.89:0/2953131504 reap_dead start 2024-08-19T10:49:09.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b262a8640 1 -- 172.21.15.89:0/2953131504 reap_dead start 2024-08-19T10:49:09.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b27d0b640 1 -- 172.21.15.89:0/2953131504 shutdown_connections 2024-08-19T10:49:09.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b27d0b640 1 -- 172.21.15.89:0/2953131504 >> 172.21.15.89:0/2953131504 conn(0x7f4b200a9130 msgr2=0x7f4b200a7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:09.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b27d0b640 1 -- 172.21.15.89:0/2953131504 shutdown_connections 2024-08-19T10:49:09.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:09.822+0000 7f4b27d0b640 1 -- 172.21.15.89:0/2953131504 wait complete. 2024-08-19T10:49:09.885 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:09.885 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:49:09.885 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:49:10.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient: get_monmap_and_config 2024-08-19T10:49:10.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient: build_initial_monmap 2024-08-19T10:49:10.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient: monmap: 2024-08-19T10:49:10.008 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:10.013168+0000 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:10.013168+0000 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:10.009 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:10.010 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:10.010 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:10.010 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 1 Processor -- start 2024-08-19T10:49:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 1 -- start start 2024-08-19T10:49:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient: init 2024-08-19T10:49:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient: _add_conns ranks=[8,1,3,6,9,10,2,7,4,0,5] 2024-08-19T10:49:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a39e0 0x7ff6280a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient(hunting): picked mon.noname-e con 0x7ff6280a39e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff6280a4300 addr v1:172.21.15.190:6789/0 2024-08-19T10:49:10.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff62813f030 0x7ff628141420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient(hunting): picked mon.noname-g con 0x7ff62813f030 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6280a4d90 con 0x7ff6280a4300 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff6280a0b60 con 0x7ff62813f030 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff6280a0e40 con 0x7ff6280a39e0 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff62dbb3640 10 monclient(hunting): authenticate will time out at 1599.829729s 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a39e0 0x7ff6280a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a39e0 0x7ff6280a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:46750/0 (socket says 172.21.15.89:46750) 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 1 -- 172.21.15.89:0/603598631 learned_addr learned my addr 172.21.15.89:0/603598631 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 10 monclient(hunting): get_auth_request con 0x7ff6280a39e0 auth_method 0 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff627fff640 1 --2- 172.21.15.89:0/603598631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff62813f030 0x7ff628141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff627fff640 10 monclient(hunting): get_auth_request con 0x7ff62813f030 auth_method 0 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff627fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff627fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 10 monclient(hunting): handle_auth_done global_id 6722 payload 274 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 1 -- 172.21.15.89:0/603598631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff62813f030 msgr2=0x7ff628141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 1 --2- 172.21.15.89:0/603598631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff62813f030 0x7ff628141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 1 -- 172.21.15.89:0/603598631 >> v1:172.21.15.190:6789/0 conn(0x7ff6280a4300 legacy=0x7ff62813d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 10 monclient: _finish_hunting 0 2024-08-19T10:49:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 1 monclient: found mon.noname-e 2024-08-19T10:49:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:10.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.010+0000 7ff6277fe640 1 -- 172.21.15.89:0/603598631 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6280a1120 con 0x7ff6280a39e0 2024-08-19T10:49:10.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6277fe640 1 -- 172.21.15.89:0/603598631 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a39e0 msgr2=0x7ff6280a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6277fe640 1 --2- 172.21.15.89:0/603598631 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a39e0 0x7ff6280a3dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6180098a0 tx=0x7ff61801c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1f410418cde45bf0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 1 -- 172.21.15.89:0/603598631 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6180b2070 con 0x7ff6280a39e0 2024-08-19T10:49:10.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 10 monclient: dump: 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.019 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 10 monclient: _finish_auth 0 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 10 monclient: _check_auth_tickets 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 1 -- 172.21.15.89:0/603598631 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6180ab7f0 con 0x7ff6280a39e0 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 1 -- 172.21.15.89:0/603598631 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6180abe10 con 0x7ff6280a39e0 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:10.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff6267fc640 10 monclient: dump: 2024-08-19T10:49:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.021 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.022 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.022 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.022 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.022 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.022 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 5 monclient: authenticate success, global_id 6722 2024-08-19T10:49:10.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:10.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 4 set_mon_vals no callback set 2024-08-19T10:49:10.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 10 monclient: shutdown 2024-08-19T10:49:10.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:10.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 1 -- 172.21.15.89:0/603598631 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a39e0 msgr2=0x7ff6280a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 1 --2- 172.21.15.89:0/603598631 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a39e0 0x7ff6280a3dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6180098a0 tx=0x7ff61801c440 comp rx=0 tx=0).stop 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 1 -- 172.21.15.89:0/603598631 shutdown_connections 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 1 --2- 172.21.15.89:0/603598631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff62813f030 0x7ff628141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 1 --2- 172.21.15.89:0/603598631 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a39e0 0x7ff6280a3dc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 1 -- 172.21.15.89:0/603598631 >> 172.21.15.89:0/603598631 conn(0x7ff6280a8f60 msgr2=0x7ff6280a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 1 -- 172.21.15.89:0/603598631 shutdown_connections 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.014+0000 7ff62dbb3640 1 -- 172.21.15.89:0/603598631 wait complete. 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient: build_initial_monmap 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient: monmap: 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:10.023 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 1 Processor -- start 2024-08-19T10:49:10.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 1 -- start start 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient: init 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient: _add_conns ranks=[0,8,5,1,6,7,4,3,2] 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6280a39e0 0x7ff628123790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient(hunting): picked mon.noname-a con 0x7ff6280a39e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a4300 0x7ff6281d1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient(hunting): picked mon.noname-e con 0x7ff6280a4300 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff62813f030 0x7ff62813cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient(hunting): picked mon.noname-h con 0x7ff62813f030 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff6280a4880 con 0x7ff6280a39e0 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff6280a4700 con 0x7ff62813f030 2024-08-19T10:49:10.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff628144180 con 0x7ff6280a4300 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff62dbb3640 10 monclient(hunting): authenticate will time out at 1599.837106s 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a4300 0x7ff6281d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff6277fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6280a39e0 0x7ff628123790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff6277fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6280a39e0 0x7ff628123790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57750/0 (socket says 172.21.15.89:57750) 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a4300 0x7ff6281d1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:46766/0 (socket says 172.21.15.89:46766) 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff6277fe640 1 -- 172.21.15.89:0/2315208250 learned_addr learned my addr 172.21.15.89:0/2315208250 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff6277fe640 10 monclient(hunting): get_auth_request con 0x7ff6280a39e0 auth_method 0 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff6277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff6277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff6277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 10 monclient(hunting): get_auth_request con 0x7ff6280a4300 auth_method 0 2024-08-19T10:49:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff627fff640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff62813f030 0x7ff62813cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff627fff640 10 monclient(hunting): get_auth_request con 0x7ff62813f030 auth_method 0 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff627fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff627fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 10 monclient(hunting): handle_auth_done global_id 6731 payload 824 2024-08-19T10:49:10.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 1 -- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff62813f030 msgr2=0x7ff62813cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff62813f030 0x7ff62813cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 1 -- 172.21.15.89:0/2315208250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6280a39e0 msgr2=0x7ff628123790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6280a39e0 0x7ff628123790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:49:10.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 1 monclient: found mon.noname-e 2024-08-19T10:49:10.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:10.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:10.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 1 -- 172.21.15.89:0/2315208250 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff628124d90 con 0x7ff6280a4300 2024-08-19T10:49:10.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff6277fe640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6280a39e0 0x7ff628123790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:10.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 1 -- 172.21.15.89:0/2315208250 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a4300 msgr2=0x7ff6281d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.018+0000 7ff626ffd640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a4300 0x7ff6281d1700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff62806d0d0 tx=0x7ff61c00c870 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff61c011c10 con 0x7ff6280a4300 2024-08-19T10:49:10.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:10.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 10 monclient: dump: 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.032 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 10 monclient: _finish_auth 0 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff62dbb3640 5 monclient: authenticate success, global_id 6731 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61c017070 con 0x7ff6280a4300 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff62dbb3640 10 monclient: _renew_subs 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff62dbb3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff62dbb3640 1 -- 172.21.15.89:0/2315208250 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff628125070 con 0x7ff6280a4300 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff61c011df0 con 0x7ff6280a4300 2024-08-19T10:49:10.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff625ffb640 4 set_mon_vals no callback set 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 10 monclient: dump: 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.034 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff62dbb3640 10 monclient: _renew_subs 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff62dbb3640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff62dbb3640 1 -- 172.21.15.89:0/2315208250 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6280a6440 con 0x7ff6280a4300 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff62dbb3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff62dbb3640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6281d1c40 0x7ff62813d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff62dbb3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff62dbb3640 1 -- 172.21.15.89:0/2315208250 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff6281c9010 con 0x7ff6281d1c40 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff61c0117a0 con 0x7ff6280a4300 2024-08-19T10:49:10.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff6277fe640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6281d1c40 0x7ff62813d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff6277fe640 10 monclient: get_auth_request con 0x7ff6281d1c40 auth_method 0 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff6277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff6277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff6277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff61403be00 0x7ff61403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7ff61c04ec80 con 0x7ff6280a4300 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff6277fe640 10 monclient(hunting): handle_auth_done global_id 44632 payload 824 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff627fff640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff61403be00 0x7ff61403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff6277fe640 1 -- 172.21.15.89:0/2315208250 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6280740b0 con 0x7ff6281d1c40 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.022+0000 7ff627fff640 10 monclient: get_auth_request con 0x7ff61403be00 auth_method 0 2024-08-19T10:49:10.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff6277fe640 1 -- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6281d1c40 msgr2=0x7ff62813d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff6277fe640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6281d1c40 0x7ff62813d170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff6180097c0 tx=0x7ff6180191e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff627fff640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff61403be00 0x7ff61403e2c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff620009ee0 tx=0x7ff62000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6180be070 con 0x7ff6281d1c40 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff60ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff60ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff60ffff640 10 monclient: dump: 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.037 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff6180b3040 con 0x7ff6281d1c40 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff60ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff60ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6281d1c40 msgr2=0x7ff62813d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff60ffff640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6281d1c40 0x7ff62813d170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff6180097c0 tx=0x7ff6180191e0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6281d1c40 msgr2=0x7ff62813d170 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:10.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.026+0000 7ff60ffff640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6281d1c40 0x7ff62813d170 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff6180097c0 tx=0x7ff6180191e0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.030+0000 7ff62dbb3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:10.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.030+0000 7ff62dbb3640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff62813d6b0 0x7ff62809c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.030+0000 7ff62dbb3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.030+0000 7ff62dbb3640 1 -- 172.21.15.89:0/2315208250 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff62813dac0 con 0x7ff62813d6b0 2024-08-19T10:49:10.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff62813d6b0 0x7ff62809c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 10 monclient: get_auth_request con 0x7ff62813d6b0 auth_method 0 2024-08-19T10:49:10.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 10 monclient(hunting): handle_auth_done global_id 44641 payload 824 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 1 -- 172.21.15.89:0/2315208250 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff628064590 con 0x7ff62813d6b0 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 1 -- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff62813d6b0 msgr2=0x7ff62809c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.034+0000 7ff6277fe640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff62813d6b0 0x7ff62809c9e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff61801c270 tx=0x7ff6180a7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6180c7070 con 0x7ff62813d6b0 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff60ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff60ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff60ffff640 10 monclient: dump: 2024-08-19T10:49:10.041 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.042 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.043 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7ff6180c0620 con 0x7ff62813d6b0 2024-08-19T10:49:10.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff60ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:10.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff60ffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:10.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff62813d6b0 msgr2=0x7ff62809c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff60ffff640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff62813d6b0 0x7ff62809c9e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff61801c270 tx=0x7ff6180a7000 comp rx=0 tx=0).stop 2024-08-19T10:49:10.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff60ffff640 1 -- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff62813d6b0 msgr2=0x7ff62809c9e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:10.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff60ffff640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff62813d6b0 0x7ff62809c9e0 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff61801c270 tx=0x7ff6180a7000 comp rx=0 tx=0).stop 2024-08-19T10:49:10.043 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:10.043 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:10.043 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 273165, 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:10.044 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.045 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.046 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:10.047 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.048 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:10.049 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.052 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.054 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.055 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.056 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.057 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.059 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.061 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.062 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.063 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.064 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.065 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.066 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.066 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:10.066 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:10.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff62dbb3640 1 -- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff61403be00 msgr2=0x7ff61403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff62dbb3640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff61403be00 0x7ff61403e2c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff620009ee0 tx=0x7ff62000b040 comp rx=0 tx=0).stop 2024-08-19T10:49:10.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff62dbb3640 10 monclient: shutdown 2024-08-19T10:49:10.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff62dbb3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:10.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff62dbb3640 1 -- 172.21.15.89:0/2315208250 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a4300 msgr2=0x7ff6281d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff62dbb3640 1 --2- 172.21.15.89:0/2315208250 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff6280a4300 0x7ff6281d1700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff62806d0d0 tx=0x7ff61c00c870 comp rx=0 tx=0).stop 2024-08-19T10:49:10.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff627fff640 1 -- 172.21.15.89:0/2315208250 reap_dead start 2024-08-19T10:49:10.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff627fff640 1 -- 172.21.15.89:0/2315208250 reap_dead start 2024-08-19T10:49:10.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff62dbb3640 1 -- 172.21.15.89:0/2315208250 shutdown_connections 2024-08-19T10:49:10.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff62dbb3640 1 -- 172.21.15.89:0/2315208250 >> 172.21.15.89:0/2315208250 conn(0x7ff6280a8f60 msgr2=0x7ff62813f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:10.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff62dbb3640 1 -- 172.21.15.89:0/2315208250 shutdown_connections 2024-08-19T10:49:10.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.038+0000 7ff62dbb3640 1 -- 172.21.15.89:0/2315208250 wait complete. 2024-08-19T10:49:10.068 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:10.068 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:49:10.069 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:49:10.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient: get_monmap_and_config 2024-08-19T10:49:10.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient: build_initial_monmap 2024-08-19T10:49:10.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient: monmap: 2024-08-19T10:49:10.187 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:10.187 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:10.185866+0000 2024-08-19T10:49:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:10.185866+0000 2024-08-19T10:49:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:10.188 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:10.189 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:10.190 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 1 Processor -- start 2024-08-19T10:49:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 1 -- start start 2024-08-19T10:49:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient: init 2024-08-19T10:49:10.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:10.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient: _add_conns ranks=[4,3,5,6,1,10,2,7,8,9,0] 2024-08-19T10:49:10.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb940a6860 0x7fcb940a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient(hunting): picked mon.noname-c con 0x7fcb940a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:10.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb940a54d0 0x7fcb940a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient(hunting): picked mon.noname-g con 0x7fcb940a54d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:10.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb940a58b0 0x7fcb940a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient(hunting): picked mon.noname-h con 0x7fcb940a58b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcb940a76c0 con 0x7fcb940a54d0 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fcb940a4c90 con 0x7fcb940a6860 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fcb94139620 con 0x7fcb940a58b0 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb9957a640 10 monclient(hunting): authenticate will time out at 1600.002991s 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb940a6860 0x7fcb940a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb940a6860 0x7fcb940a81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58452/0 (socket says 172.21.15.89:58452) 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 1 -- 172.21.15.89:0/2545549352 learned_addr learned my addr 172.21.15.89:0/2545549352 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 10 monclient(hunting): get_auth_request con 0x7fcb940a6860 auth_method 0 2024-08-19T10:49:10.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb937fe640 1 --2- 172.21.15.89:0/2545549352 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb940a58b0 0x7fcb940a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb927fc640 1 --2- 172.21.15.89:0/2545549352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb940a54d0 0x7fcb940a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb937fe640 10 monclient(hunting): get_auth_request con 0x7fcb940a58b0 auth_method 0 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb937fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb937fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_done global_id 44716 payload 274 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 1 -- 172.21.15.89:0/2545549352 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb940a58b0 msgr2=0x7fcb940a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:10.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 1 --2- 172.21.15.89:0/2545549352 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb940a58b0 0x7fcb940a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 1 -- 172.21.15.89:0/2545549352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb940a54d0 msgr2=0x7fcb940a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:10.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 1 --2- 172.21.15.89:0/2545549352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb940a54d0 0x7fcb940a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:49:10.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 1 monclient: found mon.noname-c 2024-08-19T10:49:10.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:10.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:49:10.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb92ffd640 1 -- 172.21.15.89:0/2545549352 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb94139820 con 0x7fcb940a6860 2024-08-19T10:49:10.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb927fc640 10 monclient: get_auth_request con 0x7fcb940a54d0 auth_method 0 2024-08-19T10:49:10.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.182+0000 7fcb927fc640 1 --2- 172.21.15.89:0/2545549352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb940a54d0 0x7fcb940a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:10.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb92ffd640 1 -- 172.21.15.89:0/2545549352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb940a6860 msgr2=0x7fcb940a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb92ffd640 1 --2- 172.21.15.89:0/2545549352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb940a6860 0x7fcb940a81b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009e80 tx=0x7fcb8401c9a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=6a21e53c061bac74 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 1 -- 172.21.15.89:0/2545549352 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb840ac020 con 0x7fcb940a6860 2024-08-19T10:49:10.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 10 monclient: dump: 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.201 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.202 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 10 monclient: _finish_auth 0 2024-08-19T10:49:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:49:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 1 -- 172.21.15.89:0/2545549352 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb840a68f0 con 0x7fcb940a6860 2024-08-19T10:49:10.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:10.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 1 -- 172.21.15.89:0/2545549352 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb840a7460 con 0x7fcb940a6860 2024-08-19T10:49:10.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb9957a640 5 monclient: authenticate success, global_id 44716 2024-08-19T10:49:10.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb9957a640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:10.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb9957a640 4 set_mon_vals no callback set 2024-08-19T10:49:10.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:10.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.186+0000 7fcb91ffb640 10 monclient: dump: 2024-08-19T10:49:10.203 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.204 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.205 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.205 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient: shutdown 2024-08-19T10:49:10.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:10.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 -- 172.21.15.89:0/2545549352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb940a6860 msgr2=0x7fcb940a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 --2- 172.21.15.89:0/2545549352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb940a6860 0x7fcb940a81b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009e80 tx=0x7fcb8401c9a0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 -- 172.21.15.89:0/2545549352 shutdown_connections 2024-08-19T10:49:10.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 --2- 172.21.15.89:0/2545549352 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fcb940a58b0 0x7fcb940a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 --2- 172.21.15.89:0/2545549352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb940a54d0 0x7fcb940a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 --2- 172.21.15.89:0/2545549352 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fcb940a6860 0x7fcb940a81b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 -- 172.21.15.89:0/2545549352 >> 172.21.15.89:0/2545549352 conn(0x7fcb94069a00 msgr2=0x7fcb940abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:10.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 -- 172.21.15.89:0/2545549352 shutdown_connections 2024-08-19T10:49:10.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 -- 172.21.15.89:0/2545549352 wait complete. 2024-08-19T10:49:10.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient: build_initial_monmap 2024-08-19T10:49:10.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient: monmap: 2024-08-19T10:49:10.206 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:10.206 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:10.206 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:10.207 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:10.207 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:10.207 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.207 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:10.207 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:10.207 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:10.207 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:10.207 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:10.207 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:10.207 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:10.207 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:10.208 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:10.208 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 Processor -- start 2024-08-19T10:49:10.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 -- start start 2024-08-19T10:49:10.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient: init 2024-08-19T10:49:10.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:10.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient: _add_conns ranks=[0,3,2,1,5,7,8,4,6] 2024-08-19T10:49:10.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb940a54d0 0x7fcb941d1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient(hunting): picked mon.noname-a con 0x7fcb940a54d0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:10.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb940a58b0 0x7fcb940a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient(hunting): picked mon.noname-g con 0x7fcb940a58b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:10.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb940a6860 0x7fcb940a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient(hunting): picked mon.noname-b con 0x7fcb940a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:10.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcb94143490 con 0x7fcb940a54d0 2024-08-19T10:49:10.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fcb94143310 con 0x7fcb940a6860 2024-08-19T10:49:10.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcb94143610 con 0x7fcb940a58b0 2024-08-19T10:49:10.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:10.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb9957a640 10 monclient(hunting): authenticate will time out at 1600.009983s 2024-08-19T10:49:10.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb940a54d0 0x7fcb941d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb940a54d0 0x7fcb941d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57760/0 (socket says 172.21.15.89:57760) 2024-08-19T10:49:10.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb937fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb940a6860 0x7fcb940a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 1 -- 172.21.15.89:0/459677867 learned_addr learned my addr 172.21.15.89:0/459677867 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:10.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 10 monclient(hunting): get_auth_request con 0x7fcb940a54d0 auth_method 0 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb937fe640 10 monclient(hunting): get_auth_request con 0x7fcb940a6860 auth_method 0 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb937fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb937fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb927fc640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb940a58b0 0x7fcb940a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 10 monclient(hunting): handle_auth_done global_id 6876 payload 824 2024-08-19T10:49:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 1 -- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb940a58b0 msgr2=0x7fcb940a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb940a58b0 0x7fcb940a1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 1 -- 172.21.15.89:0/459677867 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb940a6860 msgr2=0x7fcb940a20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fcb940a6860 0x7fcb940a20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:49:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 1 monclient: found mon.noname-a 2024-08-19T10:49:10.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:10.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:10.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb92ffd640 1 -- 172.21.15.89:0/459677867 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb940ab6b0 con 0x7fcb940a54d0 2024-08-19T10:49:10.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb927fc640 10 monclient: get_auth_request con 0x7fcb940a58b0 auth_method 0 2024-08-19T10:49:10.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.190+0000 7fcb927fc640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb940a58b0 0x7fcb940a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:10.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb92ffd640 1 -- 172.21.15.89:0/459677867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb940a54d0 msgr2=0x7fcb941d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb92ffd640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb940a54d0 0x7fcb941d1800 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009e20 tx=0x7fcb84019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb840ac040 con 0x7fcb940a54d0 2024-08-19T10:49:10.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 10 monclient: dump: 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.215 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.216 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:49:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:49:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb9957a640 5 monclient: authenticate success, global_id 6876 2024-08-19T10:49:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb84003bb0 con 0x7fcb940a54d0 2024-08-19T10:49:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb9957a640 10 monclient: _renew_subs 2024-08-19T10:49:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb9957a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:10.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb9957a640 1 -- 172.21.15.89:0/459677867 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb940ab310 con 0x7fcb940a54d0 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb84003d60 con 0x7fcb940a54d0 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb917fa640 4 set_mon_vals no callback set 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 10 monclient: dump: 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.217 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb9957a640 10 monclient: _renew_subs 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb9957a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb9957a640 1 -- 172.21.15.89:0/459677867 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9409ee60 con 0x7fcb940a54d0 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb9957a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:10.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb9957a640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409c9e0 0x7fcb9409cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb9957a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb9957a640 1 -- 172.21.15.89:0/459677867 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcb94135770 con 0x7fcb9409c9e0 2024-08-19T10:49:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fcb840af020 con 0x7fcb940a54d0 2024-08-19T10:49:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb927fc640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409c9e0 0x7fcb9409cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb927fc640 10 monclient: get_auth_request con 0x7fcb9409c9e0 auth_method 0 2024-08-19T10:49:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb927fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb927fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb927fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcb8003be00 0x7fcb8003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb927fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb927fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7fcb840ea0d0 con 0x7fcb940a54d0 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb927fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb937fe640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcb8003be00 0x7fcb8003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb937fe640 10 monclient: get_auth_request con 0x7fcb8003be00 auth_method 0 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb927fc640 10 monclient(hunting): handle_auth_done global_id 35346 payload 824 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.194+0000 7fcb927fc640 1 -- 172.21.15.89:0/459677867 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb940a0000 con 0x7fcb9409c9e0 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb927fc640 1 -- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409c9e0 msgr2=0x7fcb9409cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb927fc640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409c9e0 0x7fcb9409cdc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb9409e970 tx=0x7fcb8800dbf0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb937fe640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcb8003be00 0x7fcb8003e2c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcb94136cc0 tx=0x7fcb8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb88016070 con 0x7fcb9409c9e0 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb7b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:49:10.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb7b7fe640 10 monclient: dump: 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.221 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.222 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcb880088e0 con 0x7fcb9409c9e0 2024-08-19T10:49:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb7b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb7b7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409c9e0 msgr2=0x7fcb9409cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb7b7fe640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409c9e0 0x7fcb9409cdc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb9409e970 tx=0x7fcb8800dbf0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409c9e0 msgr2=0x7fcb9409cdc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:10.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.198+0000 7fcb7b7fe640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409c9e0 0x7fcb9409cdc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb9957a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb9957a640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409d300 0x7fcb9413aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb9957a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb9957a640 1 -- 172.21.15.89:0/459677867 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcb940a05c0 con 0x7fcb9409d300 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409d300 0x7fcb9413aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 10 monclient: get_auth_request con 0x7fcb9409d300 auth_method 0 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 10 monclient(hunting): handle_auth_done global_id 35355 payload 824 2024-08-19T10:49:10.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 1 -- 172.21.15.89:0/459677867 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcb94064590 con 0x7fcb9409d300 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 1 -- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409d300 msgr2=0x7fcb9413aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.206+0000 7fcb927fc640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409d300 0x7fcb9413aa90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb88008010 tx=0x7fcb88004740 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcb8801bce0 con 0x7fcb9409d300 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb7b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb7b7fe640 10 monclient: dump: 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.225 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7fcb880113c0 con 0x7fcb9409d300 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb7b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb7b7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409d300 msgr2=0x7fcb9413aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb7b7fe640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409d300 0x7fcb9413aa90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb88008010 tx=0x7fcb88004740 comp rx=0 tx=0).stop 2024-08-19T10:49:10.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb7b7fe640 1 -- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409d300 msgr2=0x7fcb9413aa90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:10.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb7b7fe640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb9409d300 0x7fcb9413aa90 secure :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb88008010 tx=0x7fcb88004740 comp rx=0 tx=0).stop 2024-08-19T10:49:10.227 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:10.227 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:10.227 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:10.227 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:10.227 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:10.227 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 395255, 2024-08-19T10:49:10.227 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:10.227 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:10.228 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:10.229 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:10.231 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.232 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:10.233 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.234 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.235 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.236 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.237 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:10.239 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:10.240 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.242 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:10.243 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.245 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.246 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.247 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.248 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.249 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.250 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:10.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.250 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:10.250 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:10.250 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:49:10.250 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.250 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.250 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.250 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:10.250 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb9957a640 1 -- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcb8003be00 msgr2=0x7fcb8003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb9957a640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcb8003be00 0x7fcb8003e2c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcb94136cc0 tx=0x7fcb8c008040 comp rx=0 tx=0).stop 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb9957a640 10 monclient: shutdown 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb9957a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb9957a640 1 -- 172.21.15.89:0/459677867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb940a54d0 msgr2=0x7fcb941d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb9957a640 1 --2- 172.21.15.89:0/459677867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb940a54d0 0x7fcb941d1800 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009e20 tx=0x7fcb84019740 comp rx=0 tx=0).stop 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb937fe640 1 -- 172.21.15.89:0/459677867 reap_dead start 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb937fe640 1 -- 172.21.15.89:0/459677867 reap_dead start 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb9957a640 1 -- 172.21.15.89:0/459677867 shutdown_connections 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb9957a640 1 -- 172.21.15.89:0/459677867 >> 172.21.15.89:0/459677867 conn(0x7fcb94069a00 msgr2=0x7fcb940abb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb9957a640 1 -- 172.21.15.89:0/459677867 shutdown_connections 2024-08-19T10:49:10.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.210+0000 7fcb9957a640 1 -- 172.21.15.89:0/459677867 wait complete. 2024-08-19T10:49:10.253 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:10.253 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:49:10.253 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:49:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient: get_monmap_and_config 2024-08-19T10:49:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient: build_initial_monmap 2024-08-19T10:49:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient: monmap: 2024-08-19T10:49:10.372 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:10.377255+0000 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:10.377255+0000 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:10.373 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:10.374 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:10.374 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:10.374 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 1 Processor -- start 2024-08-19T10:49:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 1 -- start start 2024-08-19T10:49:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient: init 2024-08-19T10:49:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient: _add_conns ranks=[2,3,0,1,10,6,7,4,5,8,9] 2024-08-19T10:49:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a3d20 0x7facdc0a4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient(hunting): picked mon.noname-b con 0x7facdc0a3d20 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facdc0a4640 0x7facdc0a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient(hunting): picked mon.noname-g con 0x7facdc0a4640 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7facdc1364f0 addr v1:172.21.15.89:6789/0 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc0a2430 con 0x7facdc1364f0 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7facdc0a0af0 con 0x7facdc0a3d20 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7facdc0a1e70 con 0x7facdc0a4640 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7face0a6f640 10 monclient(hunting): authenticate will time out at 1600.194657s 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a3d20 0x7facdc0a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a3d20 0x7facdc0a4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54342/0 (socket says 172.21.15.89:54342) 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 1 -- 172.21.15.89:0/2038849691 learned_addr learned my addr 172.21.15.89:0/2038849691 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 10 monclient(hunting): get_auth_request con 0x7facdc0a3d20 auth_method 0 2024-08-19T10:49:10.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facd9d9b640 1 --2- 172.21.15.89:0/2038849691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facdc0a4640 0x7facdc0a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facd9d9b640 10 monclient(hunting): get_auth_request con 0x7facdc0a4640 auth_method 0 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facd9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facd9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facd9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 10 monclient(hunting): handle_auth_done global_id 54407 payload 274 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 1 -- 172.21.15.89:0/2038849691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facdc0a4640 msgr2=0x7facdc0a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 1 --2- 172.21.15.89:0/2038849691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facdc0a4640 0x7facdc0a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 1 -- 172.21.15.89:0/2038849691 >> v1:172.21.15.89:6789/0 conn(0x7facdc1364f0 legacy=0x7facdc1368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 10 monclient: _finish_hunting 0 2024-08-19T10:49:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 1 monclient: found mon.noname-b 2024-08-19T10:49:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:49:10.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.374+0000 7facda59c640 1 -- 172.21.15.89:0/2038849691 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc0a1b90 con 0x7facdc0a3d20 2024-08-19T10:49:10.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facda59c640 1 -- 172.21.15.89:0/2038849691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a3d20 msgr2=0x7facdc0a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facda59c640 1 --2- 172.21.15.89:0/2038849691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a3d20 0x7facdc0a4100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faccc0098d0 tx=0x7faccc01cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aacf7a698bb1cd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 1 -- 172.21.15.89:0/2038849691 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faccc0b0070 con 0x7facdc0a3d20 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 10 monclient: dump: 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.381 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 10 monclient: _finish_auth 0 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 10 monclient: _check_auth_tickets 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 1 -- 172.21.15.89:0/2038849691 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faccc0aa810 con 0x7facdc0a3d20 2024-08-19T10:49:10.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7face0a6f640 5 monclient: authenticate success, global_id 54407 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7face0a6f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7face0a6f640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7face0a6f640 4 set_mon_vals no callback set 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 1 -- 172.21.15.89:0/2038849691 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faccc0aae30 con 0x7facdc0a3d20 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.378+0000 7facd959a640 10 monclient: dump: 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.383 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.385 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient: shutdown 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 -- 172.21.15.89:0/2038849691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a3d20 msgr2=0x7facdc0a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 --2- 172.21.15.89:0/2038849691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a3d20 0x7facdc0a4100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faccc0098d0 tx=0x7faccc01cf30 comp rx=0 tx=0).stop 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 -- 172.21.15.89:0/2038849691 shutdown_connections 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 --2- 172.21.15.89:0/2038849691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facdc0a4640 0x7facdc0a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 --2- 172.21.15.89:0/2038849691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a3d20 0x7facdc0a4100 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 -- 172.21.15.89:0/2038849691 >> 172.21.15.89:0/2038849691 conn(0x7facdc06de40 msgr2=0x7facdc06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 -- 172.21.15.89:0/2038849691 shutdown_connections 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 -- 172.21.15.89:0/2038849691 wait complete. 2024-08-19T10:49:10.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient: build_initial_monmap 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient: monmap: 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:10.387 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:10.388 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:10.388 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:10.388 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 Processor -- start 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 -- start start 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient: init 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient: _add_conns ranks=[7,2,1,3,8,5,0,6,4] 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facdc0a3d20 0x7facdc1d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient(hunting): picked mon.noname-i con 0x7facdc0a3d20 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a4640 0x7facdc1d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient(hunting): picked mon.noname-b con 0x7facdc0a4640 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facdc1364f0 0x7facdc09cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient(hunting): picked mon.noname-f con 0x7facdc1364f0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1410f0 con 0x7facdc1364f0 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7facdc140f70 con 0x7facdc0a4640 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7facdc141270 con 0x7facdc0a3d20 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7face0a6f640 10 monclient(hunting): authenticate will time out at 1600.201900s 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a4640 0x7facdc1d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a4640 0x7facdc1d1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:54350/0 (socket says 172.21.15.89:54350) 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 1 -- 172.21.15.89:0/2976544807 learned_addr learned my addr 172.21.15.89:0/2976544807 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 10 monclient(hunting): get_auth_request con 0x7facdc0a4640 auth_method 0 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 10 monclient(hunting): handle_auth_done global_id 54416 payload 824 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facda59c640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facdc0a3d20 0x7facdc1d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facdad9d640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facdc1364f0 0x7facdc09cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 1 -- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facdc0a3d20 msgr2=0x7facdc1d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facdc0a3d20 0x7facdc1d14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 1 -- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facdc1364f0 msgr2=0x7facdc09cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facdc1364f0 0x7facdc09cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 1 monclient: found mon.noname-b 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:10.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facd9d9b640 1 -- 172.21.15.89:0/2976544807 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc13cb30 con 0x7facdc0a4640 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facdad9d640 10 monclient: get_auth_request con 0x7facdc1364f0 auth_method 0 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.382+0000 7facdad9d640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facdc1364f0 0x7facdc09cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facd9d9b640 1 -- 172.21.15.89:0/2976544807 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a4640 msgr2=0x7facdc1d1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facd9d9b640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a4640 0x7facdc1d1c70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7facd000d830 tx=0x7facd000dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7facd000bd40 con 0x7facdc0a4640 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 10 monclient: dump: 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.396 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 10 monclient: _finish_auth 0 2024-08-19T10:49:10.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd0019070 con 0x7facdc0a4640 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7facd00148a0 con 0x7facdc0a4640 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 10 monclient: dump: 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.398 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccbfff640 4 set_mon_vals no callback set 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7face0a6f640 5 monclient: authenticate success, global_id 54416 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7face0a6f640 10 monclient: _renew_subs 2024-08-19T10:49:10.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7face0a6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7face0a6f640 1 -- 172.21.15.89:0/2976544807 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc13ce10 con 0x7facdc0a4640 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7face0a6f640 10 monclient: _renew_subs 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7face0a6f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7face0a6f640 1 -- 172.21.15.89:0/2976544807 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc09e670 con 0x7facdc0a4640 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7face0a6f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7face0a6f640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc1d21b0 0x7facdc09d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7face0a6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7face0a6f640 1 -- 172.21.15.89:0/2976544807 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7facdc06e630 con 0x7facdc1d21b0 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7facd0014430 con 0x7facdc0a4640 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facda59c640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc1d21b0 0x7facdc09d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facda59c640 10 monclient: get_auth_request con 0x7facdc1d21b0 auth_method 0 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7facbc03be30 0x7facbc03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facda59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facda59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facda59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7facd000f040 con 0x7facdc0a4640 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facda59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facda59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facda59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facdad9d640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7facbc03be30 0x7facbc03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facdad9d640 10 monclient: get_auth_request con 0x7facbc03be30 auth_method 0 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facda59c640 10 monclient(hunting): handle_auth_done global_id 6881 payload 824 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.386+0000 7facda59c640 1 -- 172.21.15.89:0/2976544807 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facdc0a4100 con 0x7facdc1d21b0 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7facda59c640 1 -- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc1d21b0 msgr2=0x7facdc09d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7facda59c640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc1d21b0 0x7facdc09d140 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7facdc06ea50 tx=0x7faccc019270 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7facdad9d640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7facbc03be30 0x7facbc03e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7facdc13c010 tx=0x7facd4007320 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faccc0b0070 con 0x7facdc1d21b0 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7faccaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7faccaffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7faccaffd640 10 monclient: dump: 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.402 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7faccc0b1040 con 0x7facdc1d21b0 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7faccaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7faccaffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc1d21b0 msgr2=0x7facdc09d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7faccaffd640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc1d21b0 0x7facdc09d140 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7facdc06ea50 tx=0x7faccc019270 comp rx=0 tx=0).stop 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc1d21b0 msgr2=0x7facdc09d140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:10.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.390+0000 7faccaffd640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc1d21b0 0x7facdc09d140 secure :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7facdc06ea50 tx=0x7faccc019270 comp rx=0 tx=0).stop 2024-08-19T10:49:10.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7face0a6f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:10.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7face0a6f640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc09d680 0x7facdc137500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7face0a6f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7face0a6f640 1 -- 172.21.15.89:0/2976544807 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7facdc09da60 con 0x7facdc09d680 2024-08-19T10:49:10.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7facda59c640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc09d680 0x7facdc137500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7facda59c640 10 monclient: get_auth_request con 0x7facdc09d680 auth_method 0 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7facda59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7facda59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7facda59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7facda59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7facda59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7facda59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7facda59c640 10 monclient(hunting): handle_auth_done global_id 6890 payload 824 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.398+0000 7facda59c640 1 -- 172.21.15.89:0/2976544807 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7facdc064590 con 0x7facdc09d680 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7facda59c640 1 -- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc09d680 msgr2=0x7facdc137500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7facda59c640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc09d680 0x7facdc137500 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faccc01c300 tx=0x7faccc0a4f90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faccc0c98b0 con 0x7facdc09d680 2024-08-19T10:49:10.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7faccaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7faccaffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7faccaffd640 10 monclient: dump: 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.407 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.408 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.408 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.408 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.408 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9601 (secure 0 0 0) 0x7faccc0c9440 con 0x7facdc09d680 2024-08-19T10:49:10.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7faccaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:10.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7faccaffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:10.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc09d680 msgr2=0x7facdc137500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7faccaffd640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc09d680 0x7facdc137500 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faccc01c300 tx=0x7faccc0a4f90 comp rx=0 tx=0).stop 2024-08-19T10:49:10.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7faccaffd640 1 -- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc09d680 msgr2=0x7facdc137500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:10.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7faccaffd640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7facdc09d680 0x7facdc137500 secure :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faccc01c300 tx=0x7faccc0a4f90 comp rx=0 tx=0).stop 2024-08-19T10:49:10.409 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:10.409 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:10.409 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:10.409 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:10.409 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:10.409 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 680173, 2024-08-19T10:49:10.409 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:10.409 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:10.409 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:10.409 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:10.409 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.410 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.411 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:10.412 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:10.413 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:10.415 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.416 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.417 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.418 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.419 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:10.421 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:10.422 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.424 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:10.425 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.430 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:10.431 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:10.432 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:10.433 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:10.433 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.433 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.433 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:10.433 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:10.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7face0a6f640 1 -- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7facbc03be30 msgr2=0x7facbc03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7face0a6f640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7facbc03be30 0x7facbc03e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7facdc13c010 tx=0x7facd4007320 comp rx=0 tx=0).stop 2024-08-19T10:49:10.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7face0a6f640 10 monclient: shutdown 2024-08-19T10:49:10.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7face0a6f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:10.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7face0a6f640 1 -- 172.21.15.89:0/2976544807 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a4640 msgr2=0x7facdc1d1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7face0a6f640 1 --2- 172.21.15.89:0/2976544807 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facdc0a4640 0x7facdc1d1c70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7facd000d830 tx=0x7facd000dcf0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7facdad9d640 1 -- 172.21.15.89:0/2976544807 reap_dead start 2024-08-19T10:49:10.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7facdad9d640 1 -- 172.21.15.89:0/2976544807 reap_dead start 2024-08-19T10:49:10.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7face0a6f640 1 -- 172.21.15.89:0/2976544807 shutdown_connections 2024-08-19T10:49:10.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7face0a6f640 1 -- 172.21.15.89:0/2976544807 >> 172.21.15.89:0/2976544807 conn(0x7facdc06de40 msgr2=0x7facdc0a1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:10.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7face0a6f640 1 -- 172.21.15.89:0/2976544807 shutdown_connections 2024-08-19T10:49:10.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.402+0000 7face0a6f640 1 -- 172.21.15.89:0/2976544807 wait complete. 2024-08-19T10:49:10.436 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:10.436 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:49:10.436 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:49:10.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient: get_monmap_and_config 2024-08-19T10:49:10.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient: build_initial_monmap 2024-08-19T10:49:10.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient: monmap: 2024-08-19T10:49:10.553 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:10.553 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.553 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:10.557784+0000 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:10.557784+0000 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:10.554 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:10.555 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:10.555 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:10.555 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 1 Processor -- start 2024-08-19T10:49:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 1 -- start start 2024-08-19T10:49:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient: init 2024-08-19T10:49:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient: _add_conns ranks=[7,1,0,2,5,3,10,8,9,6,4] 2024-08-19T10:49:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd880a3620 0x7fcd880a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient(hunting): picked mon.noname-i con 0x7fcd880a3620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcd8809c9e0 addr v1:172.21.15.190:6789/0 2024-08-19T10:49:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcd880a3f40 addr v1:172.21.15.89:6789/0 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd88138210 con 0x7fcd880a3f40 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd880a1790 con 0x7fcd8809c9e0 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fcd8809dab0 con 0x7fcd880a3620 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd906ab640 10 monclient(hunting): authenticate will time out at 1600.374489s 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8ec48640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcd880a3f40 0x7fcd88137b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35912/0 (socket says 172.21.15.89:35912) 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8ec48640 1 -- 172.21.15.89:0/3707968568 learned_addr learned my addr 172.21.15.89:0/3707968568 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 1 --2- 172.21.15.89:0/3707968568 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd880a3620 0x7fcd880a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 10 monclient(hunting): get_auth_request con 0x7fcd880a3620 auth_method 0 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 10 monclient(hunting): handle_auth_done global_id 64123 payload 274 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 1 -- 172.21.15.89:0/3707968568 >> v1:172.21.15.190:6789/0 conn(0x7fcd8809c9e0 legacy=0x7fcd881373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 1 -- 172.21.15.89:0/3707968568 >> v1:172.21.15.89:6789/0 conn(0x7fcd880a3f40 legacy=0x7fcd88137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 10 monclient: _finish_hunting 0 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 1 monclient: found mon.noname-i 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:49:10.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.554+0000 7fcd8e447640 1 -- 172.21.15.89:0/3707968568 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8809d1a0 con 0x7fcd880a3620 2024-08-19T10:49:10.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.558+0000 7fcd8e447640 1 -- 172.21.15.89:0/3707968568 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd880a3620 msgr2=0x7fcd880a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.558+0000 7fcd8e447640 1 --2- 172.21.15.89:0/3707968568 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd880a3620 0x7fcd880a3a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd780097f0 tx=0x7fcd7801c3a0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=8f55badc4d6937cf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 1 -- 172.21.15.89:0/3707968568 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcd780b0070 con 0x7fcd880a3620 2024-08-19T10:49:10.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 10 monclient: dump: 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.561 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 10 monclient: _finish_auth 0 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 10 monclient: _check_auth_tickets 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 1 -- 172.21.15.89:0/3707968568 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd780aa810 con 0x7fcd880a3620 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 1 -- 172.21.15.89:0/3707968568 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcd780aae30 con 0x7fcd880a3620 2024-08-19T10:49:10.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8d445640 10 monclient: dump: 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.563 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.564 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.564 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.564 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.564 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 5 monclient: authenticate success, global_id 64123 2024-08-19T10:49:10.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:10.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 4 set_mon_vals no callback set 2024-08-19T10:49:10.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient: shutdown 2024-08-19T10:49:10.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 -- 172.21.15.89:0/3707968568 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd880a3620 msgr2=0x7fcd880a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 --2- 172.21.15.89:0/3707968568 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd880a3620 0x7fcd880a3a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd780097f0 tx=0x7fcd7801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 -- 172.21.15.89:0/3707968568 shutdown_connections 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 --2- 172.21.15.89:0/3707968568 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd880a3620 0x7fcd880a3a00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 -- 172.21.15.89:0/3707968568 >> 172.21.15.89:0/3707968568 conn(0x7fcd880a8210 msgr2=0x7fcd880a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 -- 172.21.15.89:0/3707968568 shutdown_connections 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 -- 172.21.15.89:0/3707968568 wait complete. 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient: build_initial_monmap 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient: monmap: 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:10.570 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:10.571 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 Processor -- start 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 -- start start 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient: init 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient: _add_conns ranks=[6,1,8,4,3,2,0,5,7] 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcd8809c9e0 0x7fcd8811f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient(hunting): picked mon.noname-d con 0x7fcd8809c9e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd880a3620 0x7fcd881cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient(hunting): picked mon.noname-f con 0x7fcd880a3620 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd880a3f40 0x7fcd8813de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient(hunting): picked mon.noname-e con 0x7fcd880a3f40 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88142e80 con 0x7fcd880a3620 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fcd8809df40 con 0x7fcd8809c9e0 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fcd88143000 con 0x7fcd880a3f40 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd906ab640 10 monclient(hunting): authenticate will time out at 1600.383332s 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8e447640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcd8809c9e0 0x7fcd8811f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8e447640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcd8809c9e0 0x7fcd8811f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:59110/0 (socket says 172.21.15.89:59110) 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd880a3f40 0x7fcd8813de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8e447640 1 -- 172.21.15.89:0/3786353926 learned_addr learned my addr 172.21.15.89:0/3786353926 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8e447640 10 monclient(hunting): get_auth_request con 0x7fcd8809c9e0 auth_method 0 2024-08-19T10:49:10.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8e447640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8e447640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8e447640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 10 monclient(hunting): get_auth_request con 0x7fcd880a3f40 auth_method 0 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8dc46640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd880a3620 0x7fcd881cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8dc46640 10 monclient(hunting): get_auth_request con 0x7fcd880a3620 auth_method 0 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8dc46640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8dc46640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8dc46640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 10 monclient(hunting): handle_auth_done global_id 6740 payload 824 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 1 -- 172.21.15.89:0/3786353926 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcd8809c9e0 msgr2=0x7fcd8811f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcd8809c9e0 0x7fcd8811f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 1 -- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd880a3620 msgr2=0x7fcd881cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd880a3620 0x7fcd881cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 10 monclient: _finish_hunting 0 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 1 monclient: found mon.noname-e 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:10.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8ec48640 1 -- 172.21.15.89:0/3786353926 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd88120880 con 0x7fcd880a3f40 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.562+0000 7fcd8e447640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fcd8809c9e0 0x7fcd8811f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd8ec48640 1 -- 172.21.15.89:0/3786353926 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd880a3f40 msgr2=0x7fcd8813de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd8ec48640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd880a3f40 0x7fcd8813de50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000bde0 tx=0x7fcd80009ed0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcd80011b90 con 0x7fcd880a3f40 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 10 monclient: dump: 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.576 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 10 monclient: _finish_auth 0 2024-08-19T10:49:10.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd80011d70 con 0x7fcd880a3f40 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd906ab640 5 monclient: authenticate success, global_id 6740 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd906ab640 10 monclient: _renew_subs 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd906ab640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd906ab640 1 -- 172.21.15.89:0/3786353926 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8811e540 con 0x7fcd880a3f40 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcd80013870 con 0x7fcd880a3f40 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd8cc44640 4 set_mon_vals no callback set 2024-08-19T10:49:10.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 10 monclient: dump: 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.579 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd906ab640 10 monclient: _renew_subs 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd906ab640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd906ab640 1 -- 172.21.15.89:0/3786353926 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd8811fa10 con 0x7fcd880a3f40 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd906ab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd906ab640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd881cfa00 0x7fcd8813e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd906ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd906ab640 1 -- 172.21.15.89:0/3786353926 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fcd880a30f0 con 0x7fcd881cfa00 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fcd80013400 con 0x7fcd880a3f40 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd8e447640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd881cfa00 0x7fcd8813e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd8e447640 10 monclient: get_auth_request con 0x7fcd881cfa00 auth_method 0 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd8e447640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd8e447640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd8e447640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd6c03be30 0x7fcd6c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7fcd8004ffd0 con 0x7fcd880a3f40 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.566+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd8dc46640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd6c03be30 0x7fcd6c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd8e447640 10 monclient(hunting): handle_auth_done global_id 64132 payload 824 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd8e447640 1 -- 172.21.15.89:0/3786353926 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd880a3a00 con 0x7fcd881cfa00 2024-08-19T10:49:10.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd8dc46640 10 monclient: get_auth_request con 0x7fcd6c03be30 auth_method 0 2024-08-19T10:49:10.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd8e447640 1 -- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd881cfa00 msgr2=0x7fcd8813e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd8e447640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd881cfa00 0x7fcd8813e390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd7801c1d0 tx=0x7fcd78019140 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcd780b0040 con 0x7fcd881cfa00 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd7effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd7effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd7effd640 10 monclient: dump: 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.584 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd8dc46640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd6c03be30 0x7fcd6c03e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcd8813f5b0 tx=0x7fcd84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcd780aab00 con 0x7fcd881cfa00 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd7effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd7effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:10.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd881cfa00 msgr2=0x7fcd8813e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd7effd640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd881cfa00 0x7fcd8813e390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd7801c1d0 tx=0x7fcd78019140 comp rx=0 tx=0).stop 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd881cfa00 msgr2=0x7fcd8813e390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.570+0000 7fcd7effd640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd881cfa00 0x7fcd8813e390 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd7801c1d0 tx=0x7fcd78019140 comp rx=0 tx=0).stop 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd906ab640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd906ab640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd8813e8d0 0x7fcd88071a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd906ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd906ab640 1 -- 172.21.15.89:0/3786353926 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fcd8813ece0 con 0x7fcd8813e8d0 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd8e447640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd8813e8d0 0x7fcd88071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd8e447640 10 monclient: get_auth_request con 0x7fcd8813e8d0 auth_method 0 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd8e447640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd8e447640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd8e447640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd8e447640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd8e447640 10 monclient(hunting): handle_auth_done global_id 64141 payload 824 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.578+0000 7fcd8e447640 1 -- 172.21.15.89:0/3786353926 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcd88064590 con 0x7fcd8813e8d0 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd8e447640 1 -- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd8813e8d0 msgr2=0x7fcd88071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd8e447640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd8813e8d0 0x7fcd88071a40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd780ac000 tx=0x7fcd780a8000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fcd780b0040 con 0x7fcd8813e8d0 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd7effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd7effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd7effd640 10 monclient: dump: 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.587 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.589 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7fcd780b6660 con 0x7fcd8813e8d0 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd7effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd7effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd8813e8d0 msgr2=0x7fcd88071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd7effd640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd8813e8d0 0x7fcd88071a40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd780ac000 tx=0x7fcd780a8000 comp rx=0 tx=0).stop 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd7effd640 1 -- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd8813e8d0 msgr2=0x7fcd88071a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:10.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd7effd640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fcd8813e8d0 0x7fcd88071a40 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd780ac000 tx=0x7fcd780a8000 comp rx=0 tx=0).stop 2024-08-19T10:49:10.591 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:10.591 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:10.591 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:10.591 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:10.591 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:10.591 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 16979, 2024-08-19T10:49:10.591 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:10.591 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:10.591 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.592 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:10.593 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:10.595 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:10.596 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:10.598 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:10.599 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.600 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.602 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:10.603 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.604 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.606 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.607 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:10.610 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:10.611 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.612 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:10.613 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:10.614 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:10.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd906ab640 1 -- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd6c03be30 msgr2=0x7fcd6c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd906ab640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fcd6c03be30 0x7fcd6c03e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcd8813f5b0 tx=0x7fcd84009040 comp rx=0 tx=0).stop 2024-08-19T10:49:10.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd906ab640 10 monclient: shutdown 2024-08-19T10:49:10.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd906ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:10.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd906ab640 1 -- 172.21.15.89:0/3786353926 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd880a3f40 msgr2=0x7fcd8813de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd906ab640 1 --2- 172.21.15.89:0/3786353926 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fcd880a3f40 0x7fcd8813de50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000bde0 tx=0x7fcd80009ed0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd8ec48640 1 -- 172.21.15.89:0/3786353926 reap_dead start 2024-08-19T10:49:10.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd8ec48640 1 -- 172.21.15.89:0/3786353926 reap_dead start 2024-08-19T10:49:10.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd906ab640 1 -- 172.21.15.89:0/3786353926 shutdown_connections 2024-08-19T10:49:10.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd906ab640 1 -- 172.21.15.89:0/3786353926 >> 172.21.15.89:0/3786353926 conn(0x7fcd880a8210 msgr2=0x7fcd88071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:10.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd906ab640 1 -- 172.21.15.89:0/3786353926 shutdown_connections 2024-08-19T10:49:10.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.582+0000 7fcd906ab640 1 -- 172.21.15.89:0/3786353926 wait complete. 2024-08-19T10:49:10.617 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:10.617 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-08-19T10:49:10.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-08-19T10:49:10.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.734+0000 7f963e315640 10 monclient: get_monmap_and_config 2024-08-19T10:49:10.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.734+0000 7f963e315640 10 monclient: build_initial_monmap 2024-08-19T10:49:10.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.734+0000 7f963e315640 10 monclient: monmap: 2024-08-19T10:49:10.736 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:10.736 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.736 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:10.739815+0000 2024-08-19T10:49:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:10.739815+0000 2024-08-19T10:49:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:10.738 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.734+0000 7f963e315640 1 Processor -- start 2024-08-19T10:49:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.734+0000 7f963e315640 1 -- start start 2024-08-19T10:49:10.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.734+0000 7f963e315640 10 monclient: init 2024-08-19T10:49:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 10 monclient: _add_conns ranks=[8,0,6,9,3,7,5,2,4,1,10] 2024-08-19T10:49:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f96380a6860 0x7f96380a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 10 monclient(hunting): picked mon.noname-e con 0x7f96380a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f96380a7180 addr v1:172.21.15.89:6789/0 2024-08-19T10:49:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f96380a7560 0x7f9638139d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 10 monclient(hunting): picked mon.noname-d con 0x7f96380a7560 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96380a33d0 con 0x7f96380a7180 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f96380a8190 con 0x7f96380a7560 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f96380a79f0 con 0x7f96380a6860 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963e315640 10 monclient(hunting): authenticate will time out at 1600.556309s 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f96377fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f96380a7180 0x7f9638139620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35918/0 (socket says 172.21.15.89:35918) 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f9637fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f96380a6860 0x7f96380a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f96377fe640 1 -- 172.21.15.89:0/1805483461 learned_addr learned my addr 172.21.15.89:0/1805483461 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f9637fff640 10 monclient(hunting): get_auth_request con 0x7f96380a6860 auth_method 0 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f9637fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f9637fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f9637fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 1 --2- 172.21.15.89:0/1805483461 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f96380a7560 0x7f9638139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 10 monclient(hunting): get_auth_request con 0x7f96380a7560 auth_method 0 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f9637fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f9637fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f9637fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 10 monclient(hunting): handle_auth_done global_id 44745 payload 274 2024-08-19T10:49:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 1 -- 172.21.15.89:0/1805483461 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f96380a6860 msgr2=0x7f96380a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 1 --2- 172.21.15.89:0/1805483461 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f96380a6860 0x7f96380a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 1 -- 172.21.15.89:0/1805483461 >> v1:172.21.15.89:6789/0 conn(0x7f96380a7180 legacy=0x7f9638139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 10 monclient: _finish_hunting 0 2024-08-19T10:49:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 1 monclient: found mon.noname-d 2024-08-19T10:49:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:49:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f963c8b2640 1 -- 172.21.15.89:0/1805483461 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96380a2ec0 con 0x7f96380a7560 2024-08-19T10:49:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.738+0000 7f9637fff640 1 --2- 172.21.15.89:0/1805483461 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f96380a6860 0x7f96380a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963c8b2640 1 -- 172.21.15.89:0/1805483461 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f96380a7560 msgr2=0x7f9638139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963c8b2640 1 --2- 172.21.15.89:0/1805483461 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f96380a7560 0x7f9638139d30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9630009dc0 tx=0x7f963001c850 comp rx=0 tx=0).ready entity=mon.6 client_cookie=3bbbfda877254cc7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 1 -- 172.21.15.89:0/1805483461 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f96300a6960 con 0x7f96380a7560 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 10 monclient: dump: 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 10 monclient: _finish_auth 0 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 1 -- 172.21.15.89:0/1805483461 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96300a6b20 con 0x7f96380a7560 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 1 -- 172.21.15.89:0/1805483461 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f96300a6cd0 con 0x7f96380a7560 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f9636ffd640 10 monclient: dump: 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.749 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 5 monclient: authenticate success, global_id 44745 2024-08-19T10:49:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:10.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 4 set_mon_vals no callback set 2024-08-19T10:49:10.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 10 monclient: shutdown 2024-08-19T10:49:10.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:10.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 1 -- 172.21.15.89:0/1805483461 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f96380a7560 msgr2=0x7f9638139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 1 --2- 172.21.15.89:0/1805483461 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f96380a7560 0x7f9638139d30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9630009dc0 tx=0x7f963001c850 comp rx=0 tx=0).stop 2024-08-19T10:49:10.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 1 -- 172.21.15.89:0/1805483461 shutdown_connections 2024-08-19T10:49:10.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 1 --2- 172.21.15.89:0/1805483461 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f96380a7560 0x7f9638139d30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 1 --2- 172.21.15.89:0/1805483461 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f96380a6860 0x7f96380a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 1 -- 172.21.15.89:0/1805483461 >> 172.21.15.89:0/1805483461 conn(0x7f9638074b40 msgr2=0x7f96380abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 1 -- 172.21.15.89:0/1805483461 shutdown_connections 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 1 -- 172.21.15.89:0/1805483461 wait complete. 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 10 monclient: build_initial_monmap 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.742+0000 7f963e315640 10 monclient: monmap: 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.756 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 1 Processor -- start 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 1 -- start start 2024-08-19T10:49:10.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient: init 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient: _add_conns ranks=[2,4,7,6,0,1,5,8,3] 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96380a6860 0x7f96381d1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient(hunting): picked mon.noname-b con 0x7f96380a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96380a7180 0x7f96381d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient(hunting): picked mon.noname-c con 0x7f96380a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f96380a7560 0x7f963813df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient(hunting): picked mon.noname-i con 0x7f96380a7560 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f963813a3f0 con 0x7f96380a6860 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f963813a270 con 0x7f96380a7180 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f96381440a0 con 0x7f96380a7560 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient(hunting): authenticate will time out at 1600.564887s 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96377fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96380a7180 0x7f96381d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96380a6860 0x7f96381d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96377fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96380a7180 0x7f96381d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:58462/0 (socket says 172.21.15.89:58462) 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96377fe640 1 -- 172.21.15.89:0/2969238881 learned_addr learned my addr 172.21.15.89:0/2969238881 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96377fe640 10 monclient(hunting): get_auth_request con 0x7f96380a7180 auth_method 0 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96377fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96377fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 10 monclient(hunting): get_auth_request con 0x7f96380a6860 auth_method 0 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963c8b2640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f96380a7560 0x7f963813df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963c8b2640 10 monclient(hunting): get_auth_request con 0x7f96380a7560 auth_method 0 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963c8b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963c8b2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963c8b2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 10 monclient(hunting): handle_auth_done global_id 54425 payload 824 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 1 -- 172.21.15.89:0/2969238881 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f96380a7560 msgr2=0x7f963813df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f96380a7560 0x7f963813df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 1 -- 172.21.15.89:0/2969238881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96380a7180 msgr2=0x7f96381d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96380a7180 0x7f96381d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 10 monclient: _finish_hunting 0 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 1 monclient: found mon.noname-b 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 1 -- 172.21.15.89:0/2969238881 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96381404f0 con 0x7f96380a6860 2024-08-19T10:49:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96377fe640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96380a7180 0x7f96381d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 1 -- 172.21.15.89:0/2969238881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96380a6860 msgr2=0x7f96381d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9637fff640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96380a6860 0x7f96381d1360 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f96280093f0 tx=0x7f962800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 1 -- 172.21.15.89:0/2969238881 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9628004030 con 0x7f96380a6860 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 10 monclient: dump: 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 10 monclient: _finish_auth 0 2024-08-19T10:49:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 1 -- 172.21.15.89:0/2969238881 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96280041f0 con 0x7f96380a6860 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 5 monclient: authenticate success, global_id 54425 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient: _renew_subs 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f963e315640 1 -- 172.21.15.89:0/2969238881 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9638122fd0 con 0x7f96380a6860 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 1 -- 172.21.15.89:0/2969238881 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f96280043a0 con 0x7f96380a6860 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f96367fc640 4 set_mon_vals no callback set 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.746+0000 7f9634ff9640 10 monclient: dump: 2024-08-19T10:49:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:10.765 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f963e315640 10 monclient: _renew_subs 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f963e315640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f963e315640 1 -- 172.21.15.89:0/2969238881 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f96380a2570 con 0x7f96380a6860 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f9634ff9640 1 -- 172.21.15.89:0/2969238881 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f962800cb50 con 0x7f96380a6860 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f963e315640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f963e315640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f963e315640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f963e315640 1 -- 172.21.15.89:0/2969238881 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96380a2040 con 0x7f96380a6860 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f9634ff9640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f962403be00 0x7f962403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f9634ff9640 1 -- 172.21.15.89:0/2969238881 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f962800f040 con 0x7f96380a6860 2024-08-19T10:49:10.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f96377fe640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f962403be00 0x7f962403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:10.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.750+0000 7f96377fe640 10 monclient: get_auth_request con 0x7f962403be00 auth_method 0 2024-08-19T10:49:10.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.754+0000 7f96377fe640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f962403be00 0x7f962403e2c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f962c007d60 tx=0x7f962c007cb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:10.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.754+0000 7f9634ff9640 1 -- 172.21.15.89:0/2969238881 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9628017020 con 0x7f96380a6860 2024-08-19T10:49:10.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.754+0000 7f9634ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:10.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.754+0000 7f9634ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:10.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.982+0000 7f963e315640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-08-19T10:49:10.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.982+0000 7f963e315640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-08-19T10:49:10.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.982+0000 7f963e315640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:10.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.982+0000 7f963e315640 1 -- 172.21.15.89:0/2969238881 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f96380a6c40 con 0x7f96380a6860 2024-08-19T10:49:10.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.986+0000 7f9634ff9640 1 -- 172.21.15.89:0/2969238881 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f962804d5f0 con 0x7f96380a6860 2024-08-19T10:49:10.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.986+0000 7f9634ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-08-19T10:49:10.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.986+0000 7f9634ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 1 -- 172.21.15.89:0/2969238881 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f962403be00 msgr2=0x7f962403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f962403be00 0x7f962403e2c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f962c007d60 tx=0x7f962c007cb0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 10 monclient: shutdown 2024-08-19T10:49:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 1 -- 172.21.15.89:0/2969238881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96380a6860 msgr2=0x7f96381d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:10.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96380a6860 0x7f96381d1360 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f96280093f0 tx=0x7f962800ed60 comp rx=0 tx=0).stop 2024-08-19T10:49:10.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 1 -- 172.21.15.89:0/2969238881 shutdown_connections 2024-08-19T10:49:10.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f962403be00 0x7f962403e2c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f96380a7180 0x7f96381d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f96380a7560 0x7f963813df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:10.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 1 --2- 172.21.15.89:0/2969238881 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f96380a6860 0x7f96381d1360 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f96280093f0 tx=0x7f962800ed60 comp rx=0 tx=0).stop 2024-08-19T10:49:10.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.990+0000 7f963e315640 1 -- 172.21.15.89:0/2969238881 >> 172.21.15.89:0/2969238881 conn(0x7f9638074b40 msgr2=0x7f9638141090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:10.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f963e315640 1 -- 172.21.15.89:0/2969238881 shutdown_connections 2024-08-19T10:49:10.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f963e315640 1 -- 172.21.15.89:0/2969238881 wait complete. 2024-08-19T10:49:10.993 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:10.993 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=2881945743,config=2586702516,health=1509209890,kv=1101891974,logm=2732116576}) 2024-08-19T10:49:10.993 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=17,config=2,health=12,kv=12,logm=57} crc {auth=2099767220,config=2586702516,health=1509209890,kv=1101891974,logm=2729001456}) 2024-08-19T10:49:10.993 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:10.993 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=2881945743,config=2586702516,health=1509209890,kv=1101891974,logm=2732116576}) 2024-08-19T10:49:10.993 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=12,kv=11,logm=58} crc {auth=2099767220,config=2586702516,health=1509209890,kv=533376655,logm=4014309305}) 2024-08-19T10:49:10.993 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:10.993 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=2881945743,config=2586702516,health=1509209890,kv=1101891974,logm=2732116576}) 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=15,config=2,health=10,kv=12,logm=61} crc {auth=2285992429,config=2586702516,health=149096869,kv=1101891974,logm=2096913179}) 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=2881945743,config=2586702516,health=1509209890,kv=1101891974,logm=2732116576}) 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=1787939677,config=2586702516,health=1509209890,kv=1523842694,logm=2190894106}) 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=2881945743,config=2586702516,health=1509209890,kv=1101891974,logm=2732116576}) 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=4270540025,config=2586702516,health=1509209890,kv=1101891974,logm=1181465294}) 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=2881945743,config=2586702516,health=1509209890,kv=1101891974,logm=2732116576}) 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=17,config=2,health=12,kv=12,logm=57} crc {auth=2099767220,config=2586702516,health=1509209890,kv=1101891975,logm=544060610}) 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=2881945743,config=2586702516,health=1509209890,kv=1101891974,logm=2732116576}) 2024-08-19T10:49:10.994 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=16,config=2,health=12,kv=11,logm=59} crc {auth=857844147,config=2586702516,health=1509209890,kv=1523842694,logm=2094867440}) 2024-08-19T10:49:10.995 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:10.995 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=2881945743,config=2586702516,health=1509209890,kv=1101891974,logm=2732116576}) 2024-08-19T10:49:10.995 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:10.994+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=12,kv=11,logm=58} crc {auth=2099767220,config=2586702516,health=1509209890,kv=3907926731,logm=1787551656}) 2024-08-19T10:49:11.003 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3651948073}) 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4274364118}) 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3651948073}) 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1879374474}) 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3651948073}) 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2836529910}) 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3651948073}) 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1658339348}) 2024-08-19T10:49:11.004 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3651948073}) 2024-08-19T10:49:11.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2445090423}) 2024-08-19T10:49:11.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3651948073}) 2024-08-19T10:49:11.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3005341226}) 2024-08-19T10:49:11.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3651948073}) 2024-08-19T10:49:11.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2431002473}) 2024-08-19T10:49:11.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3651948073}) 2024-08-19T10:49:11.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.002+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2690389810}) 2024-08-19T10:49:11.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137654445}) 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1608760792}) 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137654445}) 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3536403720}) 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.006+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137654445}) 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2366715503}) 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137654445}) 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1689952887}) 2024-08-19T10:49:11.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137654445}) 2024-08-19T10:49:11.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2935221661}) 2024-08-19T10:49:11.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137654445}) 2024-08-19T10:49:11.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4142006961}) 2024-08-19T10:49:11.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137654445}) 2024-08-19T10:49:11.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4087364882}) 2024-08-19T10:49:11.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3137654445}) 2024-08-19T10:49:11.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3424988213}) 2024-08-19T10:49:11.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2398185733}) 2024-08-19T10:49:11.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4242077611}) 2024-08-19T10:49:11.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2398185733}) 2024-08-19T10:49:11.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2359426097}) 2024-08-19T10:49:11.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.013 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-08-19T10:49:11.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2398185733}) 2024-08-19T10:49:11.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3601392437}) 2024-08-19T10:49:11.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2398185733}) 2024-08-19T10:49:11.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2265632501}) 2024-08-19T10:49:11.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2398185733}) 2024-08-19T10:49:11.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2220466947}) 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2398185733}) 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1006549524}) 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2398185733}) 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1177360739}) 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2398185733}) 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4060805588}) 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065540431}) 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=504484830}) 2024-08-19T10:49:11.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065540431}) 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2263848465}) 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065540431}) 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2634566293}) 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.014+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065540431}) 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1765536491}) 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065540431}) 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=914368909}) 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065540431}) 2024-08-19T10:49:11.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=399991491}) 2024-08-19T10:49:11.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065540431}) 2024-08-19T10:49:11.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=908931086}) 2024-08-19T10:49:11.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065540431}) 2024-08-19T10:49:11.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3390593612}) 2024-08-19T10:49:11.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=331194881}) 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3843852043}) 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=331194881}) 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.018+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2850866456}) 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=331194881}) 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2462669961}) 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=331194881}) 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1526475673}) 2024-08-19T10:49:11.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=331194881}) 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=482915193}) 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=331194881}) 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4096723652}) 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=331194881}) 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3908996100}) 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=331194881}) 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3159866899}) 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2798458809}) 2024-08-19T10:49:11.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=333680925}) 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2798458809}) 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2825416696}) 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2798458809}) 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3577221292}) 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2798458809}) 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1685306996}) 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2798458809}) 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3489734766}) 2024-08-19T10:49:11.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2798458809}) 2024-08-19T10:49:11.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2899472851}) 2024-08-19T10:49:11.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2798458809}) 2024-08-19T10:49:11.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2610372589}) 2024-08-19T10:49:11.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2798458809}) 2024-08-19T10:49:11.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.022+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3338550894}) 2024-08-19T10:49:11.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730817601}) 2024-08-19T10:49:11.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1657271853}) 2024-08-19T10:49:11.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730817601}) 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1862711609}) 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730817601}) 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1152898368}) 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730817601}) 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.030+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2827555032}) 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730817601}) 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3553071859}) 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730817601}) 2024-08-19T10:49:11.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3726365333}) 2024-08-19T10:49:11.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730817601}) 2024-08-19T10:49:11.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4114070828}) 2024-08-19T10:49:11.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2730817601}) 2024-08-19T10:49:11.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1542864335}) 2024-08-19T10:49:11.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=798412981}) 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1575295134}) 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=798412981}) 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2287530516}) 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=798412981}) 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1006288882}) 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=798412981}) 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=411983126}) 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=798412981}) 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.034+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3377380638}) 2024-08-19T10:49:11.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=798412981}) 2024-08-19T10:49:11.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2676395923}) 2024-08-19T10:49:11.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=798412981}) 2024-08-19T10:49:11.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3512682498}) 2024-08-19T10:49:11.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=798412981}) 2024-08-19T10:49:11.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3842251056}) 2024-08-19T10:49:11.041 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.041 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803102883}) 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002027644}) 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803102883}) 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3265769888}) 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803102883}) 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1134012458}) 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803102883}) 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=630816225}) 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803102883}) 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3022147101}) 2024-08-19T10:49:11.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803102883}) 2024-08-19T10:49:11.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4216254532}) 2024-08-19T10:49:11.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803102883}) 2024-08-19T10:49:11.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3654561843}) 2024-08-19T10:49:11.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2803102883}) 2024-08-19T10:49:11.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2628265481}) 2024-08-19T10:49:11.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4284273894}) 2024-08-19T10:49:11.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3210206182}) 2024-08-19T10:49:11.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4284273894}) 2024-08-19T10:49:11.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=131181436}) 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4284273894}) 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1544122508}) 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4284273894}) 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3576939648}) 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4284273894}) 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1297987953}) 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4284273894}) 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3031814739}) 2024-08-19T10:49:11.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4284273894}) 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=642581199}) 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4284273894}) 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4237868363}) 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=657531509}) 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4085796704}) 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=657531509}) 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3800298137}) 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=657531509}) 2024-08-19T10:49:11.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=632763907}) 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=657531509}) 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3075846595}) 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=657531509}) 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2095184538}) 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=657531509}) 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=514860612}) 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=657531509}) 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2395599002}) 2024-08-19T10:49:11.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=657531509}) 2024-08-19T10:49:11.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.046+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=777365021}) 2024-08-19T10:49:11.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578498682}) 2024-08-19T10:49:11.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3244210596}) 2024-08-19T10:49:11.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578498682}) 2024-08-19T10:49:11.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=375341491}) 2024-08-19T10:49:11.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578498682}) 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2207755996}) 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578498682}) 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1068515856}) 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578498682}) 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4081269291}) 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578498682}) 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3956994659}) 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578498682}) 2024-08-19T10:49:11.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=64846604}) 2024-08-19T10:49:11.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578498682}) 2024-08-19T10:49:11.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=570821667}) 2024-08-19T10:49:11.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2314497033}) 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=725846458}) 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2314497033}) 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2198657732}) 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2314497033}) 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=207055394}) 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2314497033}) 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4013790602}) 2024-08-19T10:49:11.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2314497033}) 2024-08-19T10:49:11.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3866974825}) 2024-08-19T10:49:11.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2314497033}) 2024-08-19T10:49:11.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4244879605}) 2024-08-19T10:49:11.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2314497033}) 2024-08-19T10:49:11.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1549406590}) 2024-08-19T10:49:11.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2314497033}) 2024-08-19T10:49:11.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190732863}) 2024-08-19T10:49:11.533 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2589567921}) 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1428129270}) 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2589567921}) 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1220889616}) 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2589567921}) 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3063822030}) 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2589567921}) 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4292612727}) 2024-08-19T10:49:11.534 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.535 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2589567921}) 2024-08-19T10:49:11.535 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=851028205}) 2024-08-19T10:49:11.535 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.535 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2589567921}) 2024-08-19T10:49:11.535 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3668876186}) 2024-08-19T10:49:11.535 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.535 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2589567921}) 2024-08-19T10:49:11.535 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2123168198}) 2024-08-19T10:49:11.535 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.535 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2589567921}) 2024-08-19T10:49:11.535 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2503835126}) 2024-08-19T10:49:11.867 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65165910}) 2024-08-19T10:49:11.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1026909267}) 2024-08-19T10:49:11.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65165910}) 2024-08-19T10:49:11.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2074647439}) 2024-08-19T10:49:11.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65165910}) 2024-08-19T10:49:11.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=586047511}) 2024-08-19T10:49:11.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65165910}) 2024-08-19T10:49:11.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1926668686}) 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65165910}) 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3321172950}) 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65165910}) 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=315599111}) 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65165910}) 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3010155801}) 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65165910}) 2024-08-19T10:49:11.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=99636153}) 2024-08-19T10:49:11.872 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.872 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=871578649}) 2024-08-19T10:49:11.872 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2493977604}) 2024-08-19T10:49:11.872 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.872 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=871578649}) 2024-08-19T10:49:11.872 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=67337567}) 2024-08-19T10:49:11.872 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.872 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=871578649}) 2024-08-19T10:49:11.872 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=77577061}) 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=871578649}) 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2874679321}) 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=871578649}) 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596527055}) 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=871578649}) 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1902912583}) 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=871578649}) 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1217646183}) 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.873 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=871578649}) 2024-08-19T10:49:11.874 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3280493339}) 2024-08-19T10:49:11.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1019429318}) 2024-08-19T10:49:11.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2825415064}) 2024-08-19T10:49:11.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1019429318}) 2024-08-19T10:49:11.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1521556925}) 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1019429318}) 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2994055965}) 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1019429318}) 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2159485539}) 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1019429318}) 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=108949780}) 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1019429318}) 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3409104290}) 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1019429318}) 2024-08-19T10:49:11.878 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2168815400}) 2024-08-19T10:49:11.878 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.878 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1019429318}) 2024-08-19T10:49:11.878 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1834775599}) 2024-08-19T10:49:11.880 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3463046228}) 2024-08-19T10:49:11.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2506804965}) 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3463046228}) 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=452204477}) 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3463046228}) 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2443752871}) 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3463046228}) 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1400096854}) 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3463046228}) 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1173507585}) 2024-08-19T10:49:11.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3463046228}) 2024-08-19T10:49:11.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2701525112}) 2024-08-19T10:49:11.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3463046228}) 2024-08-19T10:49:11.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1191431341}) 2024-08-19T10:49:11.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3463046228}) 2024-08-19T10:49:11.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3810763487}) 2024-08-19T10:49:11.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603151689}) 2024-08-19T10:49:11.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2952939036}) 2024-08-19T10:49:11.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603151689}) 2024-08-19T10:49:11.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2657323406}) 2024-08-19T10:49:11.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603151689}) 2024-08-19T10:49:11.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2471389269}) 2024-08-19T10:49:11.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603151689}) 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2149686967}) 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603151689}) 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3288927879}) 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603151689}) 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2960581603}) 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603151689}) 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=963096333}) 2024-08-19T10:49:11.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603151689}) 2024-08-19T10:49:11.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1346030415}) 2024-08-19T10:49:11.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663681940}) 2024-08-19T10:49:11.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1553123452}) 2024-08-19T10:49:11.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663681940}) 2024-08-19T10:49:11.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779588287}) 2024-08-19T10:49:11.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663681940}) 2024-08-19T10:49:11.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3863236222}) 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663681940}) 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2170714370}) 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663681940}) 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1613404436}) 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663681940}) 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=852342263}) 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663681940}) 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3427364038}) 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663681940}) 2024-08-19T10:49:11.892 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2061495260}) 2024-08-19T10:49:11.896 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.896 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3448478971}) 2024-08-19T10:49:11.896 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3453386737}) 2024-08-19T10:49:11.896 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.896 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3448478971}) 2024-08-19T10:49:11.896 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1453203652}) 2024-08-19T10:49:11.896 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.896 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3448478971}) 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3196155580}) 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3448478971}) 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3729202517}) 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3448478971}) 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4259917445}) 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3448478971}) 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3192172213}) 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3448478971}) 2024-08-19T10:49:11.897 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=252667881}) 2024-08-19T10:49:11.898 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.898 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3448478971}) 2024-08-19T10:49:11.898 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=844998465}) 2024-08-19T10:49:11.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3722742487}) 2024-08-19T10:49:11.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2558954352}) 2024-08-19T10:49:11.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3722742487}) 2024-08-19T10:49:11.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3545101708}) 2024-08-19T10:49:11.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3722742487}) 2024-08-19T10:49:11.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4175392171}) 2024-08-19T10:49:11.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:11.999 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3722742487}) 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=706177068}) 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3722742487}) 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4009944010}) 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3722742487}) 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4149092063}) 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3722742487}) 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3134704323}) 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3722742487}) 2024-08-19T10:49:12.000 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:11.998+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3612391747}) 2024-08-19T10:49:12.014 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:49:12.015 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:49:12.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969054839}) 2024-08-19T10:49:12.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=165154231}) 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969054839}) 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1893686092}) 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969054839}) 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2409050588}) 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969054839}) 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1213333098}) 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969054839}) 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3893887901}) 2024-08-19T10:49:12.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969054839}) 2024-08-19T10:49:12.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2051343378}) 2024-08-19T10:49:12.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969054839}) 2024-08-19T10:49:12.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1605710796}) 2024-08-19T10:49:12.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=969054839}) 2024-08-19T10:49:12.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2957227742}) 2024-08-19T10:49:12.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient: get_monmap_and_config 2024-08-19T10:49:12.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient: build_initial_monmap 2024-08-19T10:49:12.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient: monmap: 2024-08-19T10:49:12.142 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:12.142 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:12.142 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:12.146530+0000 2024-08-19T10:49:12.142 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:12.146530+0000 2024-08-19T10:49:12.142 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:12.142 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:12.142 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:12.142 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:12.143 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:12.143 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:12.143 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:12.143 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:12.143 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:12.143 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:12.143 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:12.143 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:12.143 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:12.143 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:12.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 1 Processor -- start 2024-08-19T10:49:12.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 1 -- start start 2024-08-19T10:49:12.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient: init 2024-08-19T10:49:12.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:12.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient: _add_conns ranks=[2,9,10,0,8,7,4,6,1,3,5] 2024-08-19T10:49:12.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a6860 0x7f66040a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:12.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient(hunting): picked mon.noname-b con 0x7f66040a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:12.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f66040a54d0 0x7f66040a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:12.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient(hunting): picked mon.noname-a con 0x7f66040a54d0 addr v2:172.21.15.89:3300/0 2024-08-19T10:49:12.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f66040a58b0 0x7f66040a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient(hunting): picked mon.noname-f con 0x7f66040a58b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f66040a76c0 con 0x7f66040a54d0 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f66040a4c90 con 0x7f66040a58b0 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6604139620 con 0x7f66040a6860 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660c768640 10 monclient(hunting): authenticate will time out at 1601.963190s 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a6860 0x7f66040a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a6860 0x7f66040a81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60926/0 (socket says 172.21.15.89:60926) 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 1 -- 172.21.15.89:0/2334873993 learned_addr learned my addr 172.21.15.89:0/2334873993 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 10 monclient(hunting): get_auth_request con 0x7f66040a6860 auth_method 0 2024-08-19T10:49:12.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:12.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:12.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:12.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f6609d03640 1 --2- 172.21.15.89:0/2334873993 >> v2:172.21.15.89:3300/0 conn(0x7f66040a54d0 0x7f66040a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:12.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f6609d03640 10 monclient(hunting): get_auth_request con 0x7f66040a54d0 auth_method 0 2024-08-19T10:49:12.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f6609d03640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:12.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f6609d03640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:12.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f6609d03640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:12.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:12.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f6609d03640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f6609d03640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f6609d03640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660ad05640 1 --2- 172.21.15.89:0/2334873993 >> v2:172.21.15.190:3300/0 conn(0x7f66040a58b0 0x7f66040a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 10 monclient(hunting): handle_auth_done global_id 54434 payload 274 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 1 -- 172.21.15.89:0/2334873993 >> v2:172.21.15.190:3300/0 conn(0x7f66040a58b0 msgr2=0x7f66040a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 1 --2- 172.21.15.89:0/2334873993 >> v2:172.21.15.190:3300/0 conn(0x7f66040a58b0 0x7f66040a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 1 -- 172.21.15.89:0/2334873993 >> v2:172.21.15.89:3300/0 conn(0x7f66040a54d0 msgr2=0x7f66040a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 1 --2- 172.21.15.89:0/2334873993 >> v2:172.21.15.89:3300/0 conn(0x7f66040a54d0 0x7f66040a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 10 monclient: _finish_hunting 0 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 1 monclient: found mon.noname-b 2024-08-19T10:49:12.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:12.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:49:12.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660a504640 1 -- 172.21.15.89:0/2334873993 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66041397d0 con 0x7f66040a6860 2024-08-19T10:49:12.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660ad05640 10 monclient: get_auth_request con 0x7f66040a58b0 auth_method 0 2024-08-19T10:49:12.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f660ad05640 1 --2- 172.21.15.89:0/2334873993 >> v2:172.21.15.190:3300/0 conn(0x7f66040a58b0 0x7f66040a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:12.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.142+0000 7f6609d03640 1 --2- 172.21.15.89:0/2334873993 >> v2:172.21.15.89:3300/0 conn(0x7f66040a54d0 0x7f66040a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:12.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660a504640 1 -- 172.21.15.89:0/2334873993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a6860 msgr2=0x7f66040a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:12.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660a504640 1 --2- 172.21.15.89:0/2334873993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a6860 0x7f66040a81b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f66040acc10 tx=0x7f65fc01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a4eebcafd98f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:12.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 1 -- 172.21.15.89:0/2334873993 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65fc0b0070 con 0x7f66040a6860 2024-08-19T10:49:12.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:12.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:49:12.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 10 monclient: dump: 2024-08-19T10:49:12.155 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:12.155 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:12.155 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:12.155 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:12.155 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:12.155 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:12.155 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:12.155 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:12.155 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:12.156 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:12.156 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:12.156 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:12.156 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:12.156 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:12.156 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:12.156 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:12.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 10 monclient: _finish_auth 0 2024-08-19T10:49:12.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 10 monclient: _check_auth_tickets 2024-08-19T10:49:12.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:12.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 1 -- 172.21.15.89:0/2334873993 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65fc0a9980 con 0x7f66040a6860 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 5 monclient: authenticate success, global_id 54434 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 4 set_mon_vals no callback set 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 1 -- 172.21.15.89:0/2334873993 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65fc0aa3a0 con 0x7f66040a6860 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609502640 10 monclient: dump: 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:12.157 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:12.158 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient: shutdown 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 -- 172.21.15.89:0/2334873993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a6860 msgr2=0x7f66040a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 --2- 172.21.15.89:0/2334873993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a6860 0x7f66040a81b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f66040acc10 tx=0x7f65fc01c470 comp rx=0 tx=0).stop 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 -- 172.21.15.89:0/2334873993 shutdown_connections 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 --2- 172.21.15.89:0/2334873993 >> v2:172.21.15.190:3300/0 conn(0x7f66040a58b0 0x7f66040a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 --2- 172.21.15.89:0/2334873993 >> v2:172.21.15.89:3300/0 conn(0x7f66040a54d0 0x7f66040a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 --2- 172.21.15.89:0/2334873993 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a6860 0x7f66040a81b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 -- 172.21.15.89:0/2334873993 >> 172.21.15.89:0/2334873993 conn(0x7f6604069a00 msgr2=0x7f66040abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 -- 172.21.15.89:0/2334873993 shutdown_connections 2024-08-19T10:49:12.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 -- 172.21.15.89:0/2334873993 wait complete. 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient: build_initial_monmap 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient: monmap: 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:12.160 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:12.161 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:12.161 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:12.161 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:12.161 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:12.161 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:12.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 Processor -- start 2024-08-19T10:49:12.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 -- start start 2024-08-19T10:49:12.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient: init 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient: _add_conns ranks=[0,2,4,1,5,7,3,6,8] 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66040a54d0 0x7f660413ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient(hunting): picked mon.noname-a con 0x7f66040a54d0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a58b0 0x7f660413d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient(hunting): picked mon.noname-b con 0x7f66040a58b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f660413d8e0 0x7f66041d1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient(hunting): picked mon.noname-c con 0x7f660413d8e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6604143490 con 0x7f66040a54d0 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:12.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6604143310 con 0x7f66040a58b0 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6604143610 con 0x7f660413d8e0 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660c768640 10 monclient(hunting): authenticate will time out at 1601.971826s 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f6609d03640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a58b0 0x7f660413d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.150+0000 7f660a504640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66040a54d0 0x7f660413ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a58b0 0x7f660413d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60940/0 (socket says 172.21.15.89:60940) 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 1 -- 172.21.15.89:0/1022914114 learned_addr learned my addr 172.21.15.89:0/1022914114 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 10 monclient(hunting): get_auth_request con 0x7f66040a58b0 auth_method 0 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660ad05640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f660413d8e0 0x7f66041d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:12.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660a504640 10 monclient(hunting): get_auth_request con 0x7f66040a54d0 auth_method 0 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660a504640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660a504640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660a504640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660ad05640 10 monclient(hunting): get_auth_request con 0x7f660413d8e0 auth_method 0 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660ad05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660ad05640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660ad05640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660ad05640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660ad05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:12.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660ad05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660a504640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660a504640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660a504640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 10 monclient(hunting): handle_auth_done global_id 54443 payload 824 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 1 -- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f660413d8e0 msgr2=0x7f66041d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f660413d8e0 0x7f66041d1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 1 -- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66040a54d0 msgr2=0x7f660413ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66040a54d0 0x7f660413ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 10 monclient: _finish_hunting 0 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 1 monclient: found mon.noname-b 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:49:12.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 1 -- 172.21.15.89:0/1022914114 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660413e810 con 0x7f66040a58b0 2024-08-19T10:49:12.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660a504640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66040a54d0 0x7f660413ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:12.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660ad05640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f660413d8e0 0x7f66041d1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:12.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 1 -- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a58b0 msgr2=0x7f660413d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:12.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f6609d03640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a58b0 0x7f660413d3a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65f400ebe0 tx=0x7f65f400c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:12.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 1 -- 172.21.15.89:0/1022914114 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65f4017070 con 0x7f66040a58b0 2024-08-19T10:49:12.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:12.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:49:12.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 10 monclient: dump: 2024-08-19T10:49:12.168 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:12.168 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:12.168 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:12.168 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:12.169 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:12.169 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:12.169 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:12.169 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:12.169 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:12.169 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:12.169 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:12.169 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:12.169 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:12.169 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 10 monclient: _finish_auth 0 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 1 -- 172.21.15.89:0/1022914114 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f400ce40 con 0x7f66040a58b0 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 1 -- 172.21.15.89:0/1022914114 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65f4013d50 con 0x7f66040a58b0 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65faffd640 10 monclient: dump: 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:12.170 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:12.171 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:12.171 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:12.171 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:12.171 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:12.171 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:12.171 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:12.171 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:12.171 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660c768640 5 monclient: authenticate success, global_id 54443 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f65fbfff640 4 set_mon_vals no callback set 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660c768640 10 monclient: _renew_subs 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660c768640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660c768640 1 -- 172.21.15.89:0/1022914114 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66040aba40 con 0x7f66040a58b0 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660c768640 10 monclient: _renew_subs 2024-08-19T10:49:12.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660c768640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660c768640 1 -- 172.21.15.89:0/1022914114 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f660409d770 con 0x7f66040a58b0 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660c768640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660c768640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660c768640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.154+0000 7f660c768640 1 -- 172.21.15.89:0/1022914114 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f660413dcc0 con 0x7f66040a58b0 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.158+0000 7f6609d03640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a58b0 0x7f660413d3a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65f400ebe0 tx=0x7f65f400c520 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f65f401a070 mgrmap(e 5) 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.158+0000 7f65faffd640 1 -- 172.21.15.89:0/1022914114 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f65f401a070 con 0x7f66040a58b0 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.158+0000 7f65faffd640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f65e803be30 0x7f65e803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.158+0000 7f660a504640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f65e803be30 0x7f65e803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.158+0000 7f660a504640 10 monclient: get_auth_request con 0x7f65e803be30 auth_method 0 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.158+0000 7f660a504640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f65e803be30 0x7f65e803e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f66040a5440 tx=0x7f65fc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:12.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.162+0000 7f65faffd640 1 -- 172.21.15.89:0/1022914114 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f65f400f040 con 0x7f66040a58b0 2024-08-19T10:49:12.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338565187}) 2024-08-19T10:49:12.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1620725284}) 2024-08-19T10:49:12.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338565187}) 2024-08-19T10:49:12.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1228772956}) 2024-08-19T10:49:12.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338565187}) 2024-08-19T10:49:12.528 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2525406502}) 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338565187}) 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=106808171}) 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338565187}) 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2269345580}) 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338565187}) 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2235021335}) 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338565187}) 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3665266170}) 2024-08-19T10:49:12.529 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.530 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338565187}) 2024-08-19T10:49:12.530 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=52876375}) 2024-08-19T10:49:12.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414732891}) 2024-08-19T10:49:12.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=21825342}) 2024-08-19T10:49:12.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414732891}) 2024-08-19T10:49:12.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2503527967}) 2024-08-19T10:49:12.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414732891}) 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2112299422}) 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414732891}) 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3158920838}) 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414732891}) 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4026679590}) 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414732891}) 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=678543580}) 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414732891}) 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1335271519}) 2024-08-19T10:49:12.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414732891}) 2024-08-19T10:49:12.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=699002035}) 2024-08-19T10:49:12.569 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.570 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102460287}) 2024-08-19T10:49:12.570 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2861956433}) 2024-08-19T10:49:12.570 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.570 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102460287}) 2024-08-19T10:49:12.570 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=254849202}) 2024-08-19T10:49:12.570 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.570 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102460287}) 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1252534638}) 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102460287}) 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2116755674}) 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102460287}) 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=607070315}) 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102460287}) 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1519058289}) 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102460287}) 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3757408091}) 2024-08-19T10:49:12.571 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.572 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102460287}) 2024-08-19T10:49:12.572 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.570+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1478847092}) 2024-08-19T10:49:12.575 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.575 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2523885038}) 2024-08-19T10:49:12.576 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2279100748}) 2024-08-19T10:49:12.576 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.576 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2523885038}) 2024-08-19T10:49:12.576 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2998761530}) 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2523885038}) 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2175481256}) 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2523885038}) 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1264794063}) 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2523885038}) 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=706598520}) 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2523885038}) 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3451107290}) 2024-08-19T10:49:12.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2523885038}) 2024-08-19T10:49:12.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1849610549}) 2024-08-19T10:49:12.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2523885038}) 2024-08-19T10:49:12.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.574+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=900371325}) 2024-08-19T10:49:12.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1191515177}) 2024-08-19T10:49:12.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4191137234}) 2024-08-19T10:49:12.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1191515177}) 2024-08-19T10:49:12.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4063468647}) 2024-08-19T10:49:12.580 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1191515177}) 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1595582041}) 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1191515177}) 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1914730066}) 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1191515177}) 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3869774518}) 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1191515177}) 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3854125371}) 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1191515177}) 2024-08-19T10:49:12.581 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1541477921}) 2024-08-19T10:49:12.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1191515177}) 2024-08-19T10:49:12.582 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.578+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3474175744}) 2024-08-19T10:49:12.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=766389583}) 2024-08-19T10:49:12.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3110808860}) 2024-08-19T10:49:12.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=766389583}) 2024-08-19T10:49:12.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4171961947}) 2024-08-19T10:49:12.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=766389583}) 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3442501252}) 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=766389583}) 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=335668370}) 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=766389583}) 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3112780969}) 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=766389583}) 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=641733809}) 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=766389583}) 2024-08-19T10:49:12.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1576128780}) 2024-08-19T10:49:12.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=766389583}) 2024-08-19T10:49:12.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.582+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1572623650}) 2024-08-19T10:49:12.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.586+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.586+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603115435}) 2024-08-19T10:49:12.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.586+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=442232340}) 2024-08-19T10:49:12.589 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.586+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.586+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603115435}) 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.586+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2392607139}) 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.586+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.586+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603115435}) 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=28487901}) 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603115435}) 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3130060305}) 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603115435}) 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4178082627}) 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.590 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603115435}) 2024-08-19T10:49:12.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3328129719}) 2024-08-19T10:49:12.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603115435}) 2024-08-19T10:49:12.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1532031993}) 2024-08-19T10:49:12.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603115435}) 2024-08-19T10:49:12.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.590+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3013505065}) 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859754314}) 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3981289543}) 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859754314}) 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=867482978}) 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859754314}) 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4212329436}) 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859754314}) 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1464263610}) 2024-08-19T10:49:12.593 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859754314}) 2024-08-19T10:49:12.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3645030835}) 2024-08-19T10:49:12.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859754314}) 2024-08-19T10:49:12.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3682822524}) 2024-08-19T10:49:12.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859754314}) 2024-08-19T10:49:12.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4164022812}) 2024-08-19T10:49:12.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1859754314}) 2024-08-19T10:49:12.594 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.594+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=708652476}) 2024-08-19T10:49:12.597 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.597 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=982849840}) 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=179951765}) 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=982849840}) 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2809780320}) 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=982849840}) 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1576320916}) 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=982849840}) 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=282444744}) 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=982849840}) 2024-08-19T10:49:12.598 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1273687860}) 2024-08-19T10:49:12.599 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.599 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=982849840}) 2024-08-19T10:49:12.599 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1438374241}) 2024-08-19T10:49:12.599 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.599 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=982849840}) 2024-08-19T10:49:12.599 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2782582819}) 2024-08-19T10:49:12.599 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.599 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=982849840}) 2024-08-19T10:49:12.599 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.598+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3148396433}) 2024-08-19T10:49:12.603 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.603 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610087099}) 2024-08-19T10:49:12.603 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=604545583}) 2024-08-19T10:49:12.603 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.603 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610087099}) 2024-08-19T10:49:12.603 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1869225358}) 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610087099}) 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2822157870}) 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610087099}) 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4022359762}) 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610087099}) 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4025128257}) 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610087099}) 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3468553159}) 2024-08-19T10:49:12.604 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.605 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610087099}) 2024-08-19T10:49:12.605 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3598622994}) 2024-08-19T10:49:12.605 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.605 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1610087099}) 2024-08-19T10:49:12.605 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.602+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3957054870}) 2024-08-19T10:49:12.607 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.607 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=26808560}) 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4154158420}) 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=26808560}) 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3671205512}) 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=26808560}) 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1398369186}) 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=26808560}) 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2183523897}) 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=26808560}) 2024-08-19T10:49:12.608 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3533637100}) 2024-08-19T10:49:12.609 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.609 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=26808560}) 2024-08-19T10:49:12.609 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4128721709}) 2024-08-19T10:49:12.609 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.609 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=26808560}) 2024-08-19T10:49:12.609 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1485957766}) 2024-08-19T10:49:12.609 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.609 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=26808560}) 2024-08-19T10:49:12.609 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.606+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1561172017}) 2024-08-19T10:49:12.611 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.611 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435989400}) 2024-08-19T10:49:12.611 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1073722866}) 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435989400}) 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2589219874}) 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435989400}) 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2985936600}) 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435989400}) 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4207836545}) 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435989400}) 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=207041714}) 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.612 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435989400}) 2024-08-19T10:49:12.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4208593928}) 2024-08-19T10:49:12.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435989400}) 2024-08-19T10:49:12.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3294951685}) 2024-08-19T10:49:12.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=435989400}) 2024-08-19T10:49:12.613 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.610+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3427479216}) 2024-08-19T10:49:12.621 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.618+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.621 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.618+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836454337}) 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.618+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3371312199}) 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836454337}) 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1353075110}) 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836454337}) 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1375548313}) 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836454337}) 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1387036261}) 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836454337}) 2024-08-19T10:49:12.622 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1302895824}) 2024-08-19T10:49:12.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836454337}) 2024-08-19T10:49:12.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2937222832}) 2024-08-19T10:49:12.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836454337}) 2024-08-19T10:49:12.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2680692672}) 2024-08-19T10:49:12.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836454337}) 2024-08-19T10:49:12.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.622+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2842665958}) 2024-08-19T10:49:12.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3794281063}) 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2702968138}) 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3794281063}) 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2003305203}) 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3794281063}) 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3264702574}) 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3794281063}) 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=185279031}) 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3794281063}) 2024-08-19T10:49:12.627 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3060991963}) 2024-08-19T10:49:12.627 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.627 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3794281063}) 2024-08-19T10:49:12.627 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=595470077}) 2024-08-19T10:49:12.627 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.627 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3794281063}) 2024-08-19T10:49:12.627 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4012264738}) 2024-08-19T10:49:12.627 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.627 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3794281063}) 2024-08-19T10:49:12.627 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2445464817}) 2024-08-19T10:49:12.629 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.629 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777455405}) 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3281726945}) 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777455405}) 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1149594650}) 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777455405}) 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1393873545}) 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777455405}) 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2227335240}) 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777455405}) 2024-08-19T10:49:12.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1346534477}) 2024-08-19T10:49:12.631 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.631 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777455405}) 2024-08-19T10:49:12.631 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3148234254}) 2024-08-19T10:49:12.631 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.631 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777455405}) 2024-08-19T10:49:12.631 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1157195507}) 2024-08-19T10:49:12.631 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.631 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777455405}) 2024-08-19T10:49:12.631 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1087114489}) 2024-08-19T10:49:12.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033597454}) 2024-08-19T10:49:12.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1220882506}) 2024-08-19T10:49:12.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033597454}) 2024-08-19T10:49:12.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1738132040}) 2024-08-19T10:49:12.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033597454}) 2024-08-19T10:49:12.635 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3475697124}) 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033597454}) 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1375004431}) 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033597454}) 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1814210249}) 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033597454}) 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1632356485}) 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033597454}) 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1278475435}) 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.636 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2033597454}) 2024-08-19T10:49:12.637 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=401550210}) 2024-08-19T10:49:12.638 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=586180624}) 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2569542078}) 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=586180624}) 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2705888750}) 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=586180624}) 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2240510954}) 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=586180624}) 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2861427559}) 2024-08-19T10:49:12.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=586180624}) 2024-08-19T10:49:12.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=888527412}) 2024-08-19T10:49:12.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=586180624}) 2024-08-19T10:49:12.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3533227992}) 2024-08-19T10:49:12.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=586180624}) 2024-08-19T10:49:12.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3949963707}) 2024-08-19T10:49:12.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=586180624}) 2024-08-19T10:49:12.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232266538}) 2024-08-19T10:49:12.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.645 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3510745422}) 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3535274037}) 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3510745422}) 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4039842298}) 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3510745422}) 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2642427882}) 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3510745422}) 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1608814930}) 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3510745422}) 2024-08-19T10:49:12.646 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3547376989}) 2024-08-19T10:49:12.647 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.647 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3510745422}) 2024-08-19T10:49:12.647 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3244105268}) 2024-08-19T10:49:12.647 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.647 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3510745422}) 2024-08-19T10:49:12.647 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2967687663}) 2024-08-19T10:49:12.647 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.647 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3510745422}) 2024-08-19T10:49:12.647 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.642+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1046448361}) 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1441274125}) 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4207524513}) 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1441274125}) 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4083203647}) 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1441274125}) 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2336218778}) 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1441274125}) 2024-08-19T10:49:12.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=637743084}) 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1441274125}) 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=706622815}) 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1441274125}) 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1909480439}) 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1441274125}) 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3263421206}) 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1441274125}) 2024-08-19T10:49:12.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2586716974}) 2024-08-19T10:49:12.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592722408}) 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246749446}) 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592722408}) 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3903289643}) 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592722408}) 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1074669597}) 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592722408}) 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3882084272}) 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592722408}) 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2557671362}) 2024-08-19T10:49:12.652 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.653 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592722408}) 2024-08-19T10:49:12.653 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2582122133}) 2024-08-19T10:49:12.653 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.653 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592722408}) 2024-08-19T10:49:12.653 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2158839160}) 2024-08-19T10:49:12.653 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.653 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592722408}) 2024-08-19T10:49:12.653 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1965516232}) 2024-08-19T10:49:12.655 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.655 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3062577932}) 2024-08-19T10:49:12.655 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3541817631}) 2024-08-19T10:49:12.655 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.655 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3062577932}) 2024-08-19T10:49:12.655 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1507446441}) 2024-08-19T10:49:12.655 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.655 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3062577932}) 2024-08-19T10:49:12.655 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2438933402}) 2024-08-19T10:49:12.655 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.655 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3062577932}) 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=717818693}) 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3062577932}) 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3244635697}) 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3062577932}) 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=97595359}) 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3062577932}) 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=344636797}) 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3062577932}) 2024-08-19T10:49:12.656 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.654+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=626641696}) 2024-08-19T10:49:12.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3268117900}) 2024-08-19T10:49:12.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3418078442}) 2024-08-19T10:49:12.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3268117900}) 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2049328871}) 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3268117900}) 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3609974978}) 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3268117900}) 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1343825265}) 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3268117900}) 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1423281948}) 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3268117900}) 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=724938614}) 2024-08-19T10:49:12.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.660 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3268117900}) 2024-08-19T10:49:12.660 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1347145501}) 2024-08-19T10:49:12.660 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:12.660 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3268117900}) 2024-08-19T10:49:12.660 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:12.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=270876441}) 2024-08-19T10:49:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.986+0000 7f65faffd640 1 -- 172.21.15.89:0/1022914114 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f660413dcc0 con 0x7f66040a58b0 2024-08-19T10:49:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.986+0000 7f65faffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:12.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:12.986+0000 7f65faffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:13.011 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236006687}) 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2101750}) 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236006687}) 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2552515553}) 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236006687}) 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1377249667}) 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236006687}) 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3598919334}) 2024-08-19T10:49:13.012 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236006687}) 2024-08-19T10:49:13.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2822222362}) 2024-08-19T10:49:13.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236006687}) 2024-08-19T10:49:13.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1031428528}) 2024-08-19T10:49:13.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236006687}) 2024-08-19T10:49:13.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=110161774}) 2024-08-19T10:49:13.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236006687}) 2024-08-19T10:49:13.013 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.010+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3296225259}) 2024-08-19T10:49:13.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1077995405}) 2024-08-19T10:49:13.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2073971259}) 2024-08-19T10:49:13.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1077995405}) 2024-08-19T10:49:13.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3454548077}) 2024-08-19T10:49:13.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1077995405}) 2024-08-19T10:49:13.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2781988440}) 2024-08-19T10:49:13.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1077995405}) 2024-08-19T10:49:13.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1274795638}) 2024-08-19T10:49:13.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1077995405}) 2024-08-19T10:49:13.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2212791073}) 2024-08-19T10:49:13.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1077995405}) 2024-08-19T10:49:13.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1658777663}) 2024-08-19T10:49:13.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1077995405}) 2024-08-19T10:49:13.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4054327247}) 2024-08-19T10:49:13.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1077995405}) 2024-08-19T10:49:13.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.026+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=152535664}) 2024-08-19T10:49:13.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1945391971}) 2024-08-19T10:49:13.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=407650116}) 2024-08-19T10:49:13.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1945391971}) 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=343328468}) 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1945391971}) 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2919218026}) 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1945391971}) 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=692904}) 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1945391971}) 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1221409984}) 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1945391971}) 2024-08-19T10:49:13.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=326264227}) 2024-08-19T10:49:13.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1945391971}) 2024-08-19T10:49:13.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2213232345}) 2024-08-19T10:49:13.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1945391971}) 2024-08-19T10:49:13.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.074+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3078918103}) 2024-08-19T10:49:13.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.246+0000 7f660c768640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:49:13.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.246+0000 7f660c768640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:13.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.246+0000 7f660c768640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:13.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.246+0000 7f660c768640 1 -- 172.21.15.89:0/1022914114 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f66040a81b0 con 0x7f66040a58b0 2024-08-19T10:49:13.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f65faffd640 1 -- 172.21.15.89:0/1022914114 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7f65f4021070 con 0x7f66040a58b0 2024-08-19T10:49:13.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f65faffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:13.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f65faffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:13.249 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:49:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 -- 172.21.15.89:0/1022914114 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f65e803be30 msgr2=0x7f65e803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f65e803be30 0x7f65e803e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f66040a5440 tx=0x7f65fc0a6000 comp rx=0 tx=0).stop 2024-08-19T10:49:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 10 monclient: shutdown 2024-08-19T10:49:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 -- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a58b0 msgr2=0x7f660413d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a58b0 0x7f660413d3a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65f400ebe0 tx=0x7f65f400c520 comp rx=0 tx=0).stop 2024-08-19T10:49:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 -- 172.21.15.89:0/1022914114 shutdown_connections 2024-08-19T10:49:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f65e803be30 0x7f65e803e2f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f660413d8e0 0x7f66041d1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66040a58b0 0x7f660413d3a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 --2- 172.21.15.89:0/1022914114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66040a54d0 0x7f660413ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 -- 172.21.15.89:0/1022914114 >> 172.21.15.89:0/1022914114 conn(0x7f6604069a00 msgr2=0x7f66040abc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 -- 172.21.15.89:0/1022914114 shutdown_connections 2024-08-19T10:49:13.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.250+0000 7f660c768640 1 -- 172.21.15.89:0/1022914114 wait complete. 2024-08-19T10:49:13.269 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:49:13.269 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-08-19T10:49:13.269 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:49:13.269 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:49:13.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4285102836}) 2024-08-19T10:49:13.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=395340721}) 2024-08-19T10:49:13.384 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4285102836}) 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=655028100}) 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4285102836}) 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1393401038}) 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4285102836}) 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=539793591}) 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4285102836}) 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3153784185}) 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4285102836}) 2024-08-19T10:49:13.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3449969990}) 2024-08-19T10:49:13.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4285102836}) 2024-08-19T10:49:13.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2993388276}) 2024-08-19T10:49:13.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4285102836}) 2024-08-19T10:49:13.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1750384931}) 2024-08-19T10:49:13.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703622301}) 2024-08-19T10:49:13.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1457137194}) 2024-08-19T10:49:13.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703622301}) 2024-08-19T10:49:13.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=79958281}) 2024-08-19T10:49:13.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703622301}) 2024-08-19T10:49:13.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1027972993}) 2024-08-19T10:49:13.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703622301}) 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3443852244}) 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703622301}) 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3216963820}) 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703622301}) 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4088765964}) 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703622301}) 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=718240146}) 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2703622301}) 2024-08-19T10:49:13.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592755166}) 2024-08-19T10:49:13.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503366420}) 2024-08-19T10:49:13.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2314429596}) 2024-08-19T10:49:13.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503366420}) 2024-08-19T10:49:13.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2018599945}) 2024-08-19T10:49:13.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503366420}) 2024-08-19T10:49:13.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4034985823}) 2024-08-19T10:49:13.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503366420}) 2024-08-19T10:49:13.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=160557224}) 2024-08-19T10:49:13.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503366420}) 2024-08-19T10:49:13.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1973751548}) 2024-08-19T10:49:13.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503366420}) 2024-08-19T10:49:13.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4070900244}) 2024-08-19T10:49:13.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503366420}) 2024-08-19T10:49:13.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2462440249}) 2024-08-19T10:49:13.395 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.396 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503366420}) 2024-08-19T10:49:13.396 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2849944365}) 2024-08-19T10:49:13.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1877794769}) 2024-08-19T10:49:13.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178906037}) 2024-08-19T10:49:13.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1877794769}) 2024-08-19T10:49:13.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2979480299}) 2024-08-19T10:49:13.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1877794769}) 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3170570395}) 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1877794769}) 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4140466995}) 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1877794769}) 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=483574256}) 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1877794769}) 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2155891036}) 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1877794769}) 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1036227164}) 2024-08-19T10:49:13.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1877794769}) 2024-08-19T10:49:13.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3727049471}) 2024-08-19T10:49:13.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545830648}) 2024-08-19T10:49:13.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4120511766}) 2024-08-19T10:49:13.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545830648}) 2024-08-19T10:49:13.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1862735098}) 2024-08-19T10:49:13.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545830648}) 2024-08-19T10:49:13.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1085242898}) 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545830648}) 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=413031765}) 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545830648}) 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1524253590}) 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545830648}) 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2991124428}) 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545830648}) 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2929651062}) 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.403 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545830648}) 2024-08-19T10:49:13.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=5018412}) 2024-08-19T10:49:13.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient: get_monmap_and_config 2024-08-19T10:49:13.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient: build_initial_monmap 2024-08-19T10:49:13.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient: monmap: 2024-08-19T10:49:13.445 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:13.445 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:13.449353+0000 2024-08-19T10:49:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:13.449353+0000 2024-08-19T10:49:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:13.446 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:13.447 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 1 Processor -- start 2024-08-19T10:49:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 1 -- start start 2024-08-19T10:49:13.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient: init 2024-08-19T10:49:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient: _add_conns ranks=[0,1,3,5,2,4,8,7,9,6,10] 2024-08-19T10:49:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa3c00a6860 addr v1:172.21.15.89:6789/0 2024-08-19T10:49:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa3c00a7350 addr v1:172.21.15.190:6789/0 2024-08-19T10:49:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00a7730 0x7fa3c0141f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient(hunting): picked mon.noname-g con 0x7fa3c00a7730 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c00a33d0 con 0x7fa3c00a6860 2024-08-19T10:49:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c00a2ec0 con 0x7fa3c00a7350 2024-08-19T10:49:13.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c00a8190 con 0x7fa3c00a7730 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3c5ca9640 10 monclient(hunting): authenticate will time out at 1603.265869s 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bf7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa3c00a6860 0x7fa3c00a6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:33166/0 (socket says 172.21.15.89:33166) 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bf7fe640 1 -- 172.21.15.89:0/870443803 learned_addr learned my addr 172.21.15.89:0/870443803 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 1 --2- 172.21.15.89:0/870443803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00a7730 0x7fa3c0141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 10 monclient(hunting): get_auth_request con 0x7fa3c00a7730 auth_method 0 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 10 monclient(hunting): handle_auth_done global_id 35373 payload 274 2024-08-19T10:49:13.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 1 -- 172.21.15.89:0/870443803 >> v1:172.21.15.190:6789/0 conn(0x7fa3c00a7350 legacy=0x7fa3c013e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:13.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 1 -- 172.21.15.89:0/870443803 >> v1:172.21.15.89:6789/0 conn(0x7fa3c00a6860 legacy=0x7fa3c00a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:13.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 10 monclient: _finish_hunting 0 2024-08-19T10:49:13.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 1 monclient: found mon.noname-g 2024-08-19T10:49:13.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:13.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:49:13.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.446+0000 7fa3bffff640 1 -- 172.21.15.89:0/870443803 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c00a4cd0 con 0x7fa3c00a7730 2024-08-19T10:49:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3bffff640 1 -- 172.21.15.89:0/870443803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00a7730 msgr2=0x7fa3c0141f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3bffff640 1 --2- 172.21.15.89:0/870443803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00a7730 0x7fa3c0141f90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009dc0 tx=0x7fa3b801d2b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=9df5f9c187eaa401 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 1 -- 172.21.15.89:0/870443803 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3b80b2070 con 0x7fa3c00a7730 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 10 monclient: dump: 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.454 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 10 monclient: _finish_auth 0 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3c5ca9640 5 monclient: authenticate success, global_id 35373 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3c5ca9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 1 -- 172.21.15.89:0/870443803 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b80a5640 con 0x7fa3c00a7730 2024-08-19T10:49:13.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3c5ca9640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3c5ca9640 4 set_mon_vals no callback set 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 1 -- 172.21.15.89:0/870443803 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3b80a5ca0 con 0x7fa3c00a7730 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.450+0000 7fa3be7fc640 10 monclient: dump: 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.456 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient: shutdown 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/870443803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00a7730 msgr2=0x7fa3c0141f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 --2- 172.21.15.89:0/870443803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00a7730 0x7fa3c0141f90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009dc0 tx=0x7fa3b801d2b0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/870443803 shutdown_connections 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 --2- 172.21.15.89:0/870443803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00a7730 0x7fa3c0141f90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/870443803 >> 172.21.15.89:0/870443803 conn(0x7fa3c0074b40 msgr2=0x7fa3c00abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/870443803 shutdown_connections 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/870443803 wait complete. 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient: build_initial_monmap 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient: monmap: 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:13.458 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 Processor -- start 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 -- start start 2024-08-19T10:49:13.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient: init 2024-08-19T10:49:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient: _add_conns ranks=[2,7,0,6,3,5,4,8,1] 2024-08-19T10:49:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3c00a6860 0x7fa3c0128a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient(hunting): picked mon.noname-b con 0x7fa3c00a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa3c00a7350 0x7fa3c01408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient(hunting): picked mon.noname-i con 0x7fa3c00a7350 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:13.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00a7730 0x7fa3c01263f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient(hunting): picked mon.noname-a con 0x7fa3c00a7730 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0145080 con 0x7fa3c00a7730 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c0144f00 con 0x7fa3c00a6860 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa3c0145200 con 0x7fa3c00a7350 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3c5ca9640 10 monclient(hunting): authenticate will time out at 1603.273901s 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00a7730 0x7fa3c01263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bf7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3c00a6860 0x7fa3c0128a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00a7730 0x7fa3c01263f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60614/0 (socket says 172.21.15.89:60614) 2024-08-19T10:49:13.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 1 -- 172.21.15.89:0/1374412137 learned_addr learned my addr 172.21.15.89:0/1374412137 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bf7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3c00a6860 0x7fa3c0128a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60946/0 (socket says 172.21.15.89:60946) 2024-08-19T10:49:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bf7fe640 10 monclient(hunting): get_auth_request con 0x7fa3c00a6860 auth_method 0 2024-08-19T10:49:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 10 monclient(hunting): get_auth_request con 0x7fa3c00a7730 auth_method 0 2024-08-19T10:49:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3beffd640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa3c00a7350 0x7fa3c01408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3beffd640 10 monclient(hunting): get_auth_request con 0x7fa3c00a7350 auth_method 0 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3beffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3beffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 10 monclient(hunting): handle_auth_done global_id 6903 payload 824 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 1 -- 172.21.15.89:0/1374412137 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa3c00a7350 msgr2=0x7fa3c01408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa3c00a7350 0x7fa3c01408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 1 -- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3c00a6860 msgr2=0x7fa3c0128a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3c00a6860 0x7fa3c0128a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 10 monclient: _finish_hunting 0 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 1 monclient: found mon.noname-a 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bffff640 1 -- 172.21.15.89:0/1374412137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c012a240 con 0x7fa3c00a7730 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.454+0000 7fa3bf7fe640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa3c00a6860 0x7fa3c0128a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bffff640 1 -- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00a7730 msgr2=0x7fa3c01263f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bffff640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00a7730 0x7fa3c01263f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80b0040 tx=0x7fa3b80195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3b80b2070 con 0x7fa3c00a7730 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:13.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 10 monclient: dump: 2024-08-19T10:49:13.472 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.472 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.472 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.472 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.472 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.472 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.472 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.472 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.473 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.473 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.473 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.473 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.473 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.473 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 10 monclient: _finish_auth 0 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b80bc070 con 0x7fa3c00a7730 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3c5ca9640 5 monclient: authenticate success, global_id 6903 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3c5ca9640 10 monclient: _renew_subs 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3c5ca9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/1374412137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c00abfe0 con 0x7fa3c00a7730 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:13.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3b80a5cd0 con 0x7fa3c00a7730 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 10 monclient: dump: 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.475 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3c5ca9640 10 monclient: _renew_subs 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3c5ca9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/1374412137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c0127ee0 con 0x7fa3c00a7730 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bdffb640 4 set_mon_vals no callback set 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3c5ca9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3c5ca9640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0126930 0x7fa3c0126d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3c5ca9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/1374412137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c00a1fd0 con 0x7fa3c0126930 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bf7fe640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0126930 0x7fa3c0126d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bf7fe640 10 monclient: get_auth_request con 0x7fa3c0126930 auth_method 0 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bffff640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00a7730 0x7fa3c01263f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80b0040 tx=0x7fa3b80195f0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fa3b80e9e40 osd_map(271..271 src has 228..271) 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa3b8004040 con 0x7fa3c00a7730 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bf7fe640 10 monclient(hunting): handle_auth_done global_id 6912 payload 824 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3bf7fe640 1 -- 172.21.15.89:0/1374412137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3c00a2200 con 0x7fa3c0126930 2024-08-19T10:49:13.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3a7fff640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa3ac03be00 0x7fa3ac03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3beffd640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa3ac03be00 0x7fa3ac03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.458+0000 7fa3beffd640 10 monclient: get_auth_request con 0x7fa3ac03be00 auth_method 0 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3bf7fe640 1 -- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0126930 msgr2=0x7fa3c0126d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3bf7fe640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0126930 0x7fa3c0126d10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00072a0 tx=0x7fa3b0007760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3b0005d30 con 0x7fa3c0126930 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3a7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3a7fff640 10 monclient: dump: 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.478 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa3b000fce0 con 0x7fa3c0126930 2024-08-19T10:49:13.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3a7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3a7fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0126930 msgr2=0x7fa3c0126d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3a7fff640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0126930 0x7fa3c0126d10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00072a0 tx=0x7fa3b0007760 comp rx=0 tx=0).stop 2024-08-19T10:49:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0126930 msgr2=0x7fa3c0126d10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3a7fff640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c0126930 0x7fa3c0126d10 secure :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00072a0 tx=0x7fa3b0007760 comp rx=0 tx=0).stop 2024-08-19T10:49:13.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.462+0000 7fa3beffd640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa3ac03be00 0x7fa3ac03e2c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0127b50 tx=0x7fa3b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3c5ca9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3c5ca9640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c01224b0 0x7fa3c01228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3c5ca9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/1374412137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0127250 con 0x7fa3c01224b0 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3bf7fe640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c01224b0 0x7fa3c01228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3bf7fe640 10 monclient: get_auth_request con 0x7fa3c01224b0 auth_method 0 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3bf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3bf7fe640 10 monclient(hunting): handle_auth_done global_id 6921 payload 824 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.466+0000 7fa3bf7fe640 1 -- 172.21.15.89:0/1374412137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa3c00a6c40 con 0x7fa3c01224b0 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3bf7fe640 1 -- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c01224b0 msgr2=0x7fa3c01228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3bf7fe640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c01224b0 0x7fa3c01228c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00072a0 tx=0x7fa3b0004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa3b00064a0 con 0x7fa3c01224b0 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 10 monclient: dump: 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.483 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9436 (secure 0 0 0) 0x7fa3b0006030 con 0x7fa3c01224b0 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:13.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:13.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c01224b0 msgr2=0x7fa3c01228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c01224b0 0x7fa3c01228c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00072a0 tx=0x7fa3b0004950 comp rx=0 tx=0).stop 2024-08-19T10:49:13.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c01224b0 msgr2=0x7fa3c01228c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:13.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c01224b0 0x7fa3c01228c0 secure :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00072a0 tx=0x7fa3b0004950 comp rx=0 tx=0).stop 2024-08-19T10:49:13.485 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:13.485 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:13.485 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:13.485 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:49:13.485 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:13.485 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 683283, 2024-08-19T10:49:13.485 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:49:13.486 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:13.487 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:13.488 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:13.489 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.489 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:13.489 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:13.489 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:13.489 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:13.489 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:13.489 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:13.489 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:13.490 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:13.490 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:13.490 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:13.490 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:13.490 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:13.490 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:13.490 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:13.490 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:13.490 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:13.490 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:13.490 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:13.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.492 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.493 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.494 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:13.495 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.496 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.497 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:13.498 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:13.499 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.500 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.503 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:13.503 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:13.503 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.503 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:13.503 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:13.505 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:13.506 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:13.508 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:13.509 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:13.509 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.509 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.509 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:13.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.509 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:13.509 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:13.509 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:13.509 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.509 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.510 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:13.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.510 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:13.510 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:13.510 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:49:13.510 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.510 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.510 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:13.510 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:13.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/1374412137 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa3ac03be00 msgr2=0x7fa3ac03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3c5ca9640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa3ac03be00 0x7fa3ac03e2c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0127b50 tx=0x7fa3b4009040 comp rx=0 tx=0).stop 2024-08-19T10:49:13.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3c5ca9640 10 monclient: shutdown 2024-08-19T10:49:13.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3c5ca9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:13.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3bffff640 1 -- 172.21.15.89:0/1374412137 reap_dead start 2024-08-19T10:49:13.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00a7730 msgr2=0x7fa3c01263f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3c5ca9640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00a7730 0x7fa3c01263f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80b0040 tx=0x7fa3b80195f0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3a7fff640 1 -- 172.21.15.89:0/1374412137 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (unknown 0 0 0) 0x7fa3b80e9e40 con 0x7fa3c00a7730 2024-08-19T10:49:13.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/1374412137 shutdown_connections 2024-08-19T10:49:13.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3c5ca9640 1 --2- 172.21.15.89:0/1374412137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3c00a7730 0x7fa3c01263f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/1374412137 >> 172.21.15.89:0/1374412137 conn(0x7fa3c0074b40 msgr2=0x7fa3c009f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:13.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/1374412137 shutdown_connections 2024-08-19T10:49:13.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.470+0000 7fa3c5ca9640 1 -- 172.21.15.89:0/1374412137 wait complete. 2024-08-19T10:49:13.513 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:13.513 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:49:13.513 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:49:13.525 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931391503}) 2024-08-19T10:49:13.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3603625179}) 2024-08-19T10:49:13.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931391503}) 2024-08-19T10:49:13.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1085760597}) 2024-08-19T10:49:13.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931391503}) 2024-08-19T10:49:13.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1199068664}) 2024-08-19T10:49:13.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931391503}) 2024-08-19T10:49:13.526 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1198273312}) 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931391503}) 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3030164401}) 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931391503}) 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2445704401}) 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931391503}) 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1040714594}) 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1931391503}) 2024-08-19T10:49:13.527 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.526+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2793436165}) 2024-08-19T10:49:13.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687167085}) 2024-08-19T10:49:13.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=739473269}) 2024-08-19T10:49:13.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687167085}) 2024-08-19T10:49:13.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3125165145}) 2024-08-19T10:49:13.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687167085}) 2024-08-19T10:49:13.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1356067557}) 2024-08-19T10:49:13.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687167085}) 2024-08-19T10:49:13.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3424908748}) 2024-08-19T10:49:13.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687167085}) 2024-08-19T10:49:13.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2086291664}) 2024-08-19T10:49:13.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687167085}) 2024-08-19T10:49:13.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1764594890}) 2024-08-19T10:49:13.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687167085}) 2024-08-19T10:49:13.539 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4081768344}) 2024-08-19T10:49:13.539 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.539 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687167085}) 2024-08-19T10:49:13.539 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1319604102}) 2024-08-19T10:49:13.551 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1589140874}) 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=637754710}) 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1589140874}) 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=485414267}) 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1589140874}) 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1146373286}) 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1589140874}) 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2981003494}) 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1589140874}) 2024-08-19T10:49:13.552 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2111767669}) 2024-08-19T10:49:13.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1589140874}) 2024-08-19T10:49:13.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3202529671}) 2024-08-19T10:49:13.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1589140874}) 2024-08-19T10:49:13.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2226737390}) 2024-08-19T10:49:13.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1589140874}) 2024-08-19T10:49:13.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4275354112}) 2024-08-19T10:49:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.662+0000 7fde820d8640 10 monclient: get_monmap_and_config 2024-08-19T10:49:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.662+0000 7fde820d8640 10 monclient: build_initial_monmap 2024-08-19T10:49:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.662+0000 7fde820d8640 10 monclient: monmap: 2024-08-19T10:49:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:13.668618+0000 2024-08-19T10:49:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:13.668618+0000 2024-08-19T10:49:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:13.664 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:13.665 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:13.665 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 1 Processor -- start 2024-08-19T10:49:13.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 1 -- start start 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 10 monclient: init 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 10 monclient: _add_conns ranks=[4,3,2,0,9,6,5,8,7,10,1] 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 0x7fde7c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 10 monclient(hunting): picked mon.noname-c con 0x7fde7c0a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde7c0a7180 0x7fde7c0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 10 monclient(hunting): picked mon.noname-g con 0x7fde7c0a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c139620 0x7fde7c139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 10 monclient(hunting): picked mon.noname-b con 0x7fde7c139620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c0a8190 con 0x7fde7c139620 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c0a33d0 con 0x7fde7c0a7180 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fde7c0a2ec0 con 0x7fde7c0a6860 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde820d8640 10 monclient(hunting): authenticate will time out at 1603.484995s 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 0x7fde7c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 0x7fde7c0a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48466/0 (socket says 172.21.15.89:48466) 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 1 -- 172.21.15.89:0/938486956 learned_addr learned my addr 172.21.15.89:0/938486956 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7bfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c139620 0x7fde7c139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 10 monclient(hunting): get_auth_request con 0x7fde7c0a6860 auth_method 0 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7bfff640 10 monclient(hunting): get_auth_request con 0x7fde7c139620 auth_method 0 2024-08-19T10:49:13.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 10 monclient(hunting): handle_auth_done global_id 44743 payload 274 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7affd640 1 --2- 172.21.15.89:0/938486956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde7c0a7180 0x7fde7c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 1 -- 172.21.15.89:0/938486956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde7c0a7180 msgr2=0x7fde7c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 1 --2- 172.21.15.89:0/938486956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde7c0a7180 0x7fde7c0a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 1 -- 172.21.15.89:0/938486956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c139620 msgr2=0x7fde7c139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 1 --2- 172.21.15.89:0/938486956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c139620 0x7fde7c139a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 1 monclient: found mon.noname-c 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7b7fe640 1 -- 172.21.15.89:0/938486956 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c0a4cd0 con 0x7fde7c0a6860 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7affd640 10 monclient: get_auth_request con 0x7fde7c0a7180 auth_method 0 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7bfff640 1 --2- 172.21.15.89:0/938486956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c139620 0x7fde7c139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:13.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.666+0000 7fde7affd640 1 --2- 172.21.15.89:0/938486956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde7c0a7180 0x7fde7c0a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7b7fe640 1 -- 172.21.15.89:0/938486956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 msgr2=0x7fde7c0a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7b7fe640 1 --2- 172.21.15.89:0/938486956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 0x7fde7c0a6c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0098a0 tx=0x7fde6c01c440 comp rx=0 tx=0).ready entity=mon.4 client_cookie=e03fd6f8fd79057d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 1 -- 172.21.15.89:0/938486956 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde6c0b0070 con 0x7fde7c0a6860 2024-08-19T10:49:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:49:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 10 monclient: dump: 2024-08-19T10:49:13.678 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.679 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 1 -- 172.21.15.89:0/938486956 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6c0aa810 con 0x7fde7c0a6860 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 5 monclient: authenticate success, global_id 44743 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 1 -- 172.21.15.89:0/938486956 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde6c0a9300 con 0x7fde7c0a6860 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 4 set_mon_vals no callback set 2024-08-19T10:49:13.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde7a7fc640 10 monclient: dump: 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.681 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.682 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 10 monclient: shutdown 2024-08-19T10:49:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 1 -- 172.21.15.89:0/938486956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 msgr2=0x7fde7c0a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 1 --2- 172.21.15.89:0/938486956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 0x7fde7c0a6c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0098a0 tx=0x7fde6c01c440 comp rx=0 tx=0).stop 2024-08-19T10:49:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 1 -- 172.21.15.89:0/938486956 shutdown_connections 2024-08-19T10:49:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 1 --2- 172.21.15.89:0/938486956 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c139620 0x7fde7c139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 1 --2- 172.21.15.89:0/938486956 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde7c0a7180 0x7fde7c0a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 1 --2- 172.21.15.89:0/938486956 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 0x7fde7c0a6c40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 1 -- 172.21.15.89:0/938486956 >> 172.21.15.89:0/938486956 conn(0x7fde7c074b40 msgr2=0x7fde7c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 1 -- 172.21.15.89:0/938486956 shutdown_connections 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 1 -- 172.21.15.89:0/938486956 wait complete. 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 10 monclient: build_initial_monmap 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.670+0000 7fde820d8640 10 monclient: monmap: 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.683 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 1 Processor -- start 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 1 -- start start 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient: init 2024-08-19T10:49:13.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient: _add_conns ranks=[4,0,2,1,7,6,8,3,5] 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 0x7fde7c1d1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient(hunting): picked mon.noname-c con 0x7fde7c0a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde7c0a7180 0x7fde7c0a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient(hunting): picked mon.noname-a con 0x7fde7c0a7180 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c139620 0x7fde7c0a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient(hunting): picked mon.noname-b con 0x7fde7c139620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c143210 con 0x7fde7c0a7180 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c143090 con 0x7fde7c139620 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fde7c143390 con 0x7fde7c0a6860 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient(hunting): authenticate will time out at 1603.492561s 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 0x7fde7c1d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde7c0a7180 0x7fde7c0a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 0x7fde7c1d1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48480/0 (socket says 172.21.15.89:48480) 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 1 -- 172.21.15.89:0/1068158391 learned_addr learned my addr 172.21.15.89:0/1068158391 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 10 monclient(hunting): get_auth_request con 0x7fde7c0a6860 auth_method 0 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7bfff640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c139620 0x7fde7c0a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7affd640 10 monclient(hunting): get_auth_request con 0x7fde7c0a7180 auth_method 0 2024-08-19T10:49:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7bfff640 10 monclient(hunting): get_auth_request con 0x7fde7c139620 auth_method 0 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 10 monclient(hunting): handle_auth_done global_id 44752 payload 824 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 1 -- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c139620 msgr2=0x7fde7c0a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c139620 0x7fde7c0a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 1 -- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde7c0a7180 msgr2=0x7fde7c0a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde7c0a7180 0x7fde7c0a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 1 monclient: found mon.noname-c 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 1 -- 172.21.15.89:0/1068158391 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c0a06d0 con 0x7fde7c0a6860 2024-08-19T10:49:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 1 -- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 msgr2=0x7fde7c1d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde7b7fe640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 0x7fde7c1d1390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0ae040 tx=0x7fde6c0191e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde6c0b9070 con 0x7fde7c0a6860 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 10 monclient: dump: 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 10 monclient: _finish_auth 0 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 10 monclient: _check_auth_tickets 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 5 monclient: authenticate success, global_id 44752 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6c0044b0 con 0x7fde7c0a6860 2024-08-19T10:49:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient: _renew_subs 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 1 -- 172.21.15.89:0/1068158391 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c1d18d0 con 0x7fde7c0a6860 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde6c0b0040 con 0x7fde7c0a6860 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde63fff640 10 monclient: dump: 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient: _renew_subs 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:49:13.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde79ffb640 4 set_mon_vals no callback set 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.674+0000 7fde820d8640 1 -- 172.21.15.89:0/1068158391 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c1d1b60 con 0x7fde7c0a6860 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde820d8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde820d8640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09c9e0 0x7fde7c09cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde820d8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde820d8640 1 -- 172.21.15.89:0/1068158391 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c0a94b0 con 0x7fde7c09c9e0 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09c9e0 0x7fde7c09cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 10 monclient: get_auth_request con 0x7fde7c09c9e0 auth_method 0 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fde6c0aac70 con 0x7fde7c0a6860 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 10 monclient(hunting): handle_auth_done global_id 54479 payload 824 2024-08-19T10:49:13.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 1 -- 172.21.15.89:0/1068158391 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde7c0a0260 con 0x7fde7c09c9e0 2024-08-19T10:49:13.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fde6803bdf0 0x7fde6803e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7fde6c0e84f0 con 0x7fde7c0a6860 2024-08-19T10:49:13.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7bfff640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fde6803bdf0 0x7fde6803e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7bfff640 10 monclient: get_auth_request con 0x7fde6803bdf0 auth_method 0 2024-08-19T10:49:13.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 1 -- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09c9e0 msgr2=0x7fde7c09cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde7affd640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09c9e0 0x7fde7c09cdc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde7c13fa60 tx=0x7fde7000fce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde7000dc30 con 0x7fde7c09c9e0 2024-08-19T10:49:13.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 10 monclient: dump: 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.698 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.699 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.699 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.699 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.699 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.699 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fde7000dc30 con 0x7fde7c09c9e0 2024-08-19T10:49:13.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09c9e0 msgr2=0x7fde7c09cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09c9e0 0x7fde7c09cdc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde7c13fa60 tx=0x7fde7000fce0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09c9e0 msgr2=0x7fde7c09cdc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.678+0000 7fde63fff640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09c9e0 0x7fde7c09cdc0 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde7c13fa60 tx=0x7fde7000fce0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.682+0000 7fde7bfff640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fde6803bdf0 0x7fde6803e2b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0a9a20 tx=0x7fde7400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde820d8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:13.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde820d8640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09d300 0x7fde7c13aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde820d8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde820d8640 1 -- 172.21.15.89:0/1068158391 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c0a7560 con 0x7fde7c09d300 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde7affd640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09d300 0x7fde7c13aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde7affd640 10 monclient: get_auth_request con 0x7fde7c09d300 auth_method 0 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde7affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde7affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde7affd640 10 monclient(hunting): handle_auth_done global_id 54488 payload 824 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.686+0000 7fde7affd640 1 -- 172.21.15.89:0/1068158391 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fde7c064590 con 0x7fde7c09d300 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde7affd640 1 -- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09d300 msgr2=0x7fde7c13aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde7affd640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09d300 0x7fde7c13aa80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde7c13fa60 tx=0x7fde700046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fde70006770 con 0x7fde7c09d300 2024-08-19T10:49:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde63fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde63fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde63fff640 10 monclient: dump: 2024-08-19T10:49:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.704 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7fde70006770 con 0x7fde7c09d300 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde63fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde63fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:13.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09d300 msgr2=0x7fde7c13aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde63fff640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09d300 0x7fde7c13aa80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde7c13fa60 tx=0x7fde700046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde63fff640 1 -- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09d300 msgr2=0x7fde7c13aa80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:13.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde63fff640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fde7c09d300 0x7fde7c13aa80 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde7c13fa60 tx=0x7fde700046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.706 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:13.706 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:13.706 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:13.706 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 149086, 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:13.707 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:13.708 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:13.709 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:13.710 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:13.711 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:13.712 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.713 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:13.715 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:13.716 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.717 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.718 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:13.719 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.721 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.724 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:13.725 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:13.726 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.727 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:13.728 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:13.729 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:13.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde820d8640 1 -- 172.21.15.89:0/1068158391 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fde6803bdf0 msgr2=0x7fde6803e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde820d8640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fde6803bdf0 0x7fde6803e2b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0a9a20 tx=0x7fde7400b040 comp rx=0 tx=0).stop 2024-08-19T10:49:13.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde820d8640 10 monclient: shutdown 2024-08-19T10:49:13.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde820d8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:13.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde820d8640 1 -- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 msgr2=0x7fde7c1d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde820d8640 1 --2- 172.21.15.89:0/1068158391 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fde7c0a6860 0x7fde7c1d1390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0ae040 tx=0x7fde6c0191e0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde7bfff640 1 -- 172.21.15.89:0/1068158391 reap_dead start 2024-08-19T10:49:13.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde7bfff640 1 -- 172.21.15.89:0/1068158391 reap_dead start 2024-08-19T10:49:13.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde820d8640 1 -- 172.21.15.89:0/1068158391 shutdown_connections 2024-08-19T10:49:13.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde820d8640 1 -- 172.21.15.89:0/1068158391 >> 172.21.15.89:0/1068158391 conn(0x7fde7c074b40 msgr2=0x7fde7c0a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:13.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde820d8640 1 -- 172.21.15.89:0/1068158391 shutdown_connections 2024-08-19T10:49:13.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.690+0000 7fde820d8640 1 -- 172.21.15.89:0/1068158391 wait complete. 2024-08-19T10:49:13.732 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:13.732 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:49:13.732 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:49:13.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1717744990}) 2024-08-19T10:49:13.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2292662074}) 2024-08-19T10:49:13.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1717744990}) 2024-08-19T10:49:13.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1447888653}) 2024-08-19T10:49:13.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1717744990}) 2024-08-19T10:49:13.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3301249138}) 2024-08-19T10:49:13.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1717744990}) 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2080281809}) 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1717744990}) 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=406908108}) 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1717744990}) 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3656343298}) 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1717744990}) 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=579772643}) 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1717744990}) 2024-08-19T10:49:13.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:13.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2968218998}) 2024-08-19T10:49:13.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient: get_monmap_and_config 2024-08-19T10:49:13.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient: build_initial_monmap 2024-08-19T10:49:13.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient: monmap: 2024-08-19T10:49:13.853 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:13.853 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.853 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:13.857979+0000 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:13.857979+0000 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:13.855 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:13.855 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 1 Processor -- start 2024-08-19T10:49:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 1 -- start start 2024-08-19T10:49:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient: init 2024-08-19T10:49:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient: _add_conns ranks=[4,8,0,10,3,6,2,7,1,9,5] 2024-08-19T10:49:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64340a4b30 0x7f64340742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient(hunting): picked mon.noname-c con 0x7f64340a4b30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 0x7f64340a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient(hunting): picked mon.noname-e con 0x7f64340720c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f64340a5450 addr v1:172.21.15.89:6789/0 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6434073ea0 con 0x7f64340a5450 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6434073600 con 0x7f64340a4b30 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f64340738e0 con 0x7f64340720c0 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643c646640 10 monclient(hunting): authenticate will time out at 1603.674488s 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64340a4b30 0x7f64340742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f6439be1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 0x7f64340a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643abe3640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f64340a5450 0x7f643413d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:33178/0 (socket says 172.21.15.89:33178) 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643abe3640 1 -- 172.21.15.89:0/699007203 learned_addr learned my addr 172.21.15.89:0/699007203 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f6439be1640 10 monclient(hunting): get_auth_request con 0x7f64340720c0 auth_method 0 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f6439be1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f6439be1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f6439be1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 10 monclient(hunting): get_auth_request con 0x7f64340a4b30 auth_method 0 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f6439be1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f6439be1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f6439be1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 10 monclient(hunting): handle_auth_done global_id 44761 payload 274 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 1 -- 172.21.15.89:0/699007203 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 msgr2=0x7f64340a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 1 --2- 172.21.15.89:0/699007203 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 0x7f64340a4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 1 -- 172.21.15.89:0/699007203 >> v1:172.21.15.89:6789/0 conn(0x7f64340a5450 legacy=0x7f643413d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 10 monclient: _finish_hunting 0 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 1 monclient: found mon.noname-c 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f643a3e2640 1 -- 172.21.15.89:0/699007203 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64340a42a0 con 0x7f64340a4b30 2024-08-19T10:49:13.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.854+0000 7f6439be1640 1 --2- 172.21.15.89:0/699007203 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 0x7f64340a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:13.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.858+0000 7f643a3e2640 1 -- 172.21.15.89:0/699007203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64340a4b30 msgr2=0x7f64340742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.858+0000 7f643a3e2640 1 --2- 172.21.15.89:0/699007203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64340a4b30 0x7f64340742c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64300097f0 tx=0x7f643001c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=9c2cf15a24817f67 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.858+0000 7f64393e0640 1 -- 172.21.15.89:0/699007203 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64300b0070 con 0x7f64340a4b30 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.858+0000 7f64393e0640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f64393e0640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f64393e0640 10 monclient: dump: 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.864 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f64393e0640 10 monclient: _finish_auth 0 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f64393e0640 10 monclient: _check_auth_tickets 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f64393e0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f64393e0640 1 -- 172.21.15.89:0/699007203 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64300aa810 con 0x7f64340a4b30 2024-08-19T10:49:13.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 5 monclient: authenticate success, global_id 44761 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f64393e0640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f64393e0640 1 -- 172.21.15.89:0/699007203 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64300a9300 con 0x7f64340a4b30 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 4 set_mon_vals no callback set 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f64393e0640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f64393e0640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f64393e0640 10 monclient: dump: 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.866 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient: shutdown 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:13.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 -- 172.21.15.89:0/699007203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64340a4b30 msgr2=0x7f64340742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 --2- 172.21.15.89:0/699007203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64340a4b30 0x7f64340742c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64300097f0 tx=0x7f643001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 -- 172.21.15.89:0/699007203 shutdown_connections 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 --2- 172.21.15.89:0/699007203 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 0x7f64340a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 --2- 172.21.15.89:0/699007203 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f64340a4b30 0x7f64340742c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 -- 172.21.15.89:0/699007203 >> 172.21.15.89:0/699007203 conn(0x7f64340aa240 msgr2=0x7f64340aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 -- 172.21.15.89:0/699007203 shutdown_connections 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 -- 172.21.15.89:0/699007203 wait complete. 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient: build_initial_monmap 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient: monmap: 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:13.868 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:13.869 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:13.869 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.869 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:13.869 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:13.869 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:13.869 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:13.869 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 Processor -- start 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 -- start start 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient: init 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient: _add_conns ranks=[8,1,3,5,4,7,6,0,2] 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 0x7f64341d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient(hunting): picked mon.noname-e con 0x7f64340720c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:13.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64340a4b30 0x7f643413cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient(hunting): picked mon.noname-f con 0x7f64340a4b30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64340a5450 0x7f643413d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient(hunting): picked mon.noname-g con 0x7f64340a5450 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6434141900 con 0x7f64340a4b30 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6434141780 con 0x7f64340a5450 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6434141a80 con 0x7f64340720c0 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643c646640 10 monclient(hunting): authenticate will time out at 1603.682275s 2024-08-19T10:49:13.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 0x7f64341d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 0x7f64341d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:37910/0 (socket says 172.21.15.89:37910) 2024-08-19T10:49:13.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 1 -- 172.21.15.89:0/3329086590 learned_addr learned my addr 172.21.15.89:0/3329086590 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:13.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 10 monclient(hunting): get_auth_request con 0x7f64340720c0 auth_method 0 2024-08-19T10:49:13.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 10 monclient(hunting): handle_auth_done global_id 6767 payload 824 2024-08-19T10:49:13.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 1 -- 172.21.15.89:0/3329086590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64340a5450 msgr2=0x7f643413d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64340a5450 0x7f643413d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 1 -- 172.21.15.89:0/3329086590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64340a4b30 msgr2=0x7f643413cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64340a4b30 0x7f643413cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 10 monclient: _finish_hunting 0 2024-08-19T10:49:13.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 1 monclient: found mon.noname-e 2024-08-19T10:49:13.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:13.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:13.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.862+0000 7f643a3e2640 1 -- 172.21.15.89:0/3329086590 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64341271a0 con 0x7f64340720c0 2024-08-19T10:49:13.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643a3e2640 1 -- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 msgr2=0x7f64341d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643a3e2640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 0x7f64341d0a20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f64300097f0 tx=0x7f6430019140 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f64300b0040 con 0x7f64340720c0 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 10 monclient: dump: 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.882 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 10 monclient: _finish_auth 0 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64300044b0 con 0x7f64340720c0 2024-08-19T10:49:13.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643c646640 5 monclient: authenticate success, global_id 6767 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643c646640 10 monclient: _renew_subs 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643c646640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643c646640 1 -- 172.21.15.89:0/3329086590 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64340a27d0 con 0x7f64340720c0 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6430003a90 con 0x7f64340720c0 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f642affd640 10 monclient: dump: 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.884 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f6438bdf640 4 set_mon_vals no callback set 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643c646640 10 monclient: _renew_subs 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643c646640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:13.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643c646640 1 -- 172.21.15.89:0/3329086590 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f6434129500 con 0x7f64340720c0 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643c646640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643c646640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434127d00 0x7f6434125740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643c646640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f643c646640 1 -- 172.21.15.89:0/3329086590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6434064590 con 0x7f6434127d00 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f6439be1640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434127d00 0x7f6434125740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f6439be1640 10 monclient: get_auth_request con 0x7f6434127d00 auth_method 0 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f6439be1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f6439be1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.866+0000 7f6439be1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f6439be1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f6439be1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f6439be1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f6439be1640 10 monclient(hunting): handle_auth_done global_id 44770 payload 824 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6430003c30 con 0x7f64340720c0 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f6439be1640 1 -- 172.21.15.89:0/3329086590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64340a4f10 con 0x7f6434127d00 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f641803be00 0x7f641803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f64300e91c0 con 0x7f64340720c0 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f643abe3640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f641803be00 0x7f641803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f643abe3640 10 monclient: get_auth_request con 0x7f641803be00 auth_method 0 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f6439be1640 1 -- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434127d00 msgr2=0x7f6434125740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f6439be1640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434127d00 0x7f6434125740 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6434072050 tx=0x7f642400ec00 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6424017070 con 0x7f6434127d00 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 10 monclient: dump: 2024-08-19T10:49:13.887 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.888 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f642400cae0 con 0x7f6434127d00 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434127d00 msgr2=0x7f6434125740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434127d00 0x7f6434125740 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6434072050 tx=0x7f642400ec00 comp rx=0 tx=0).stop 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434127d00 msgr2=0x7f6434125740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f642affd640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434127d00 0x7f6434125740 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6434072050 tx=0x7f642400ec00 comp rx=0 tx=0).stop 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.870+0000 7f643abe3640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f641803be00 0x7f641803e2c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f64341299f0 tx=0x7f642c009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f643c646640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f643c646640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434125c80 0x7f6434126090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f643c646640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f643c646640 1 -- 172.21.15.89:0/3329086590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f64341265d0 con 0x7f6434125c80 2024-08-19T10:49:13.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434125c80 0x7f6434126090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 10 monclient: get_auth_request con 0x7f6434125c80 auth_method 0 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 10 monclient(hunting): handle_auth_done global_id 44779 payload 824 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 1 -- 172.21.15.89:0/3329086590 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f64341290d0 con 0x7f6434125c80 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 1 -- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434125c80 msgr2=0x7f6434126090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f6439be1640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434125c80 0x7f6434126090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6424019000 tx=0x7f64240046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6424017070 con 0x7f6434125c80 2024-08-19T10:49:13.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f642affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f642affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.878+0000 7f642affd640 10 monclient: dump: 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:13.891 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:13.892 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:13.892 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:13.892 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:13.892 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:13.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f6424006540 con 0x7f6434125c80 2024-08-19T10:49:13.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f642affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:13.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f642affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:13.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434125c80 msgr2=0x7f6434126090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f642affd640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434125c80 0x7f6434126090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6424019000 tx=0x7f64240046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f642affd640 1 -- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434125c80 msgr2=0x7f6434126090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:13.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f642affd640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6434125c80 0x7f6434126090 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6424019000 tx=0x7f64240046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 277005, 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:13.893 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:13.894 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:13.895 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:13.896 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:13.897 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.898 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.899 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:13.900 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.902 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.903 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.905 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.906 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.907 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:13.908 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.911 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:13.912 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:13.913 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.914 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:13.915 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:13.915 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:13.915 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:13.915 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:13.915 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:13.915 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:13.915 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:13.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:13.915 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:13.915 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643c646640 1 -- 172.21.15.89:0/3329086590 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f641803be00 msgr2=0x7f641803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643c646640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f641803be00 0x7f641803e2c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f64341299f0 tx=0x7f642c009210 comp rx=0 tx=0).stop 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643c646640 10 monclient: shutdown 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643c646640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643c646640 1 -- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 msgr2=0x7f64341d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643c646640 1 --2- 172.21.15.89:0/3329086590 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f64340720c0 0x7f64341d0a20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f64300097f0 tx=0x7f6430019140 comp rx=0 tx=0).stop 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643abe3640 1 -- 172.21.15.89:0/3329086590 reap_dead start 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643abe3640 1 -- 172.21.15.89:0/3329086590 reap_dead start 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643c646640 1 -- 172.21.15.89:0/3329086590 shutdown_connections 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643c646640 1 -- 172.21.15.89:0/3329086590 >> 172.21.15.89:0/3329086590 conn(0x7f64340aa240 msgr2=0x7f643413dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643c646640 1 -- 172.21.15.89:0/3329086590 shutdown_connections 2024-08-19T10:49:13.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:13.882+0000 7f643c646640 1 -- 172.21.15.89:0/3329086590 wait complete. 2024-08-19T10:49:13.918 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:13.918 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:49:13.918 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:49:14.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient: get_monmap_and_config 2024-08-19T10:49:14.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient: monmap: 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:14.037267+0000 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:14.037267+0000 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.033 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:14.034 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:14.034 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 1 Processor -- start 2024-08-19T10:49:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 1 -- start start 2024-08-19T10:49:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient: init 2024-08-19T10:49:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient: _add_conns ranks=[6,10,2,5,7,3,8,0,9,4,1] 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 0x7f4ea80a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient(hunting): picked mon.noname-d con 0x7f4ea80a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4ea80a7180 0x7f4ea80a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient(hunting): picked mon.noname-f con 0x7f4ea80a7180 addr v2:172.21.15.190:3300/0 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ea8139620 0x7f4ea8139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient(hunting): picked mon.noname-b con 0x7f4ea8139620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea80a8190 con 0x7f4ea80a7180 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea80a33d0 con 0x7f4ea8139620 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4ea80a2ec0 con 0x7f4ea80a6860 2024-08-19T10:49:14.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eafdae640 10 monclient(hunting): authenticate will time out at 1603.853900s 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 0x7f4ea80a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 0x7f4ea80a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54856/0 (socket says 172.21.15.89:54856) 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 1 -- 172.21.15.89:0/3623047756 learned_addr learned my addr 172.21.15.89:0/3623047756 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 10 monclient(hunting): get_auth_request con 0x7f4ea80a6860 auth_method 0 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4ead349640 1 --2- 172.21.15.89:0/3623047756 >> v2:172.21.15.190:3300/0 conn(0x7f4ea80a7180 0x7f4ea80a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eae34b640 1 --2- 172.21.15.89:0/3623047756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ea8139620 0x7f4ea8139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4ead349640 10 monclient(hunting): get_auth_request con 0x7f4ea80a7180 auth_method 0 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4ead349640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4ead349640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4ead349640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eae34b640 10 monclient(hunting): get_auth_request con 0x7f4ea8139620 auth_method 0 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eae34b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eae34b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eae34b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 10 monclient(hunting): handle_auth_done global_id 44754 payload 274 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 1 -- 172.21.15.89:0/3623047756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ea8139620 msgr2=0x7f4ea8139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 1 --2- 172.21.15.89:0/3623047756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ea8139620 0x7f4ea8139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 1 -- 172.21.15.89:0/3623047756 >> v2:172.21.15.190:3300/0 conn(0x7f4ea80a7180 msgr2=0x7f4ea80a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 1 --2- 172.21.15.89:0/3623047756 >> v2:172.21.15.190:3300/0 conn(0x7f4ea80a7180 0x7f4ea80a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 10 monclient: _finish_hunting 0 2024-08-19T10:49:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 1 monclient: found mon.noname-d 2024-08-19T10:49:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:49:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eadb4a640 1 -- 172.21.15.89:0/3623047756 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea80a4cd0 con 0x7f4ea80a6860 2024-08-19T10:49:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4eae34b640 1 --2- 172.21.15.89:0/3623047756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ea8139620 0x7f4ea8139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:49:14.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.034+0000 7f4ead349640 1 --2- 172.21.15.89:0/3623047756 >> v2:172.21.15.190:3300/0 conn(0x7f4ea80a7180 0x7f4ea80a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:49:14.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=93427815}) 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=824903608}) 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=93427815}) 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3421663920}) 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=93427815}) 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2103538936}) 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=93427815}) 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2209507609}) 2024-08-19T10:49:14.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=93427815}) 2024-08-19T10:49:14.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4029100873}) 2024-08-19T10:49:14.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=93427815}) 2024-08-19T10:49:14.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2622490269}) 2024-08-19T10:49:14.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=93427815}) 2024-08-19T10:49:14.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4013001523}) 2024-08-19T10:49:14.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=93427815}) 2024-08-19T10:49:14.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3570958939}) 2024-08-19T10:49:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eadb4a640 1 -- 172.21.15.89:0/3623047756 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 msgr2=0x7f4ea80a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eadb4a640 1 --2- 172.21.15.89:0/3623047756 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 0x7f4ea80a6c40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009870 tx=0x7f4e9801c420 comp rx=0 tx=0).ready entity=mon.6 client_cookie=fc149c1069d562fa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 1 -- 172.21.15.89:0/3623047756 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e980b0070 con 0x7f4ea80a6860 2024-08-19T10:49:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:49:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 10 monclient: dump: 2024-08-19T10:49:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.048 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 10 monclient: _finish_auth 0 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 10 monclient: _check_auth_tickets 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 1 -- 172.21.15.89:0/3623047756 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e980aa810 con 0x7f4ea80a6860 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eafdae640 5 monclient: authenticate success, global_id 44754 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eafdae640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eafdae640 4 set_mon_vals no callback set 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 1 -- 172.21.15.89:0/3623047756 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e980a9300 con 0x7f4ea80a6860 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.038+0000 7f4eacb48640 10 monclient: dump: 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.050 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.051 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.042+0000 7f4eafdae640 10 monclient: shutdown 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.042+0000 7f4eafdae640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.042+0000 7f4eafdae640 1 -- 172.21.15.89:0/3623047756 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 msgr2=0x7f4ea80a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.042+0000 7f4eafdae640 1 --2- 172.21.15.89:0/3623047756 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 0x7f4ea80a6c40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009870 tx=0x7f4e9801c420 comp rx=0 tx=0).stop 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.042+0000 7f4eafdae640 1 -- 172.21.15.89:0/3623047756 shutdown_connections 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.042+0000 7f4eafdae640 1 --2- 172.21.15.89:0/3623047756 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f4ea8139620 0x7f4ea8139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.042+0000 7f4eafdae640 1 --2- 172.21.15.89:0/3623047756 >> v2:172.21.15.190:3300/0 conn(0x7f4ea80a7180 0x7f4ea80a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.042+0000 7f4eafdae640 1 --2- 172.21.15.89:0/3623047756 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 0x7f4ea80a6c40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.042+0000 7f4eafdae640 1 -- 172.21.15.89:0/3623047756 >> 172.21.15.89:0/3623047756 conn(0x7f4ea8074b40 msgr2=0x7f4ea80abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 1 -- 172.21.15.89:0/3623047756 shutdown_connections 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 1 -- 172.21.15.89:0/3623047756 wait complete. 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient: monmap: 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.053 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 1 Processor -- start 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 1 -- start start 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient: init 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient: _add_conns ranks=[6,4,0,2,8,3,5,1,7] 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 0x7f4ea8134930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient(hunting): picked mon.noname-d con 0x7f4ea80a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ea80a7180 0x7f4ea8134e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient(hunting): picked mon.noname-c con 0x7f4ea80a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:14.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea8139620 0x7f4ea809cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient(hunting): picked mon.noname-a con 0x7f4ea8139620 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8143210 con 0x7f4ea8139620 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f4ea8143090 con 0x7f4ea80a7180 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4ea8143390 con 0x7f4ea80a6860 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eafdae640 10 monclient(hunting): authenticate will time out at 1603.866063s 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 0x7f4ea8134930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 0x7f4ea8134930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54868/0 (socket says 172.21.15.89:54868) 2024-08-19T10:49:14.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 1 -- 172.21.15.89:0/2100133438 learned_addr learned my addr 172.21.15.89:0/2100133438 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 10 monclient(hunting): get_auth_request con 0x7f4ea80a6860 auth_method 0 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4ead349640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ea80a7180 0x7f4ea8134e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4ead349640 10 monclient(hunting): get_auth_request con 0x7f4ea80a7180 auth_method 0 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4ead349640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4ead349640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4ead349640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4ead349640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4ead349640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4ead349640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 10 monclient(hunting): handle_auth_done global_id 44763 payload 824 2024-08-19T10:49:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 1 -- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ea80a7180 msgr2=0x7f4ea8134e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f4ea80a7180 0x7f4ea8134e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 1 -- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea8139620 msgr2=0x7f4ea809cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ea8139620 0x7f4ea809cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 10 monclient: _finish_hunting 0 2024-08-19T10:49:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 1 monclient: found mon.noname-d 2024-08-19T10:49:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:49:14.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.046+0000 7f4eadb4a640 1 -- 172.21.15.89:0/2100133438 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea80a0860 con 0x7f4ea80a6860 2024-08-19T10:49:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eadb4a640 1 -- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 msgr2=0x7f4ea8134930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eadb4a640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 0x7f4ea8134930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009870 tx=0x7f4e980191c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e980b9070 con 0x7f4ea80a6860 2024-08-19T10:49:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:49:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 10 monclient: dump: 2024-08-19T10:49:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.060 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e980044b0 con 0x7f4ea80a6860 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eafdae640 5 monclient: authenticate success, global_id 44763 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4e980b0040 con 0x7f4ea80a6860 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9ffff640 4 set_mon_vals no callback set 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 10 monclient: dump: 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.062 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.063 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eafdae640 10 monclient: _renew_subs 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eafdae640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eafdae640 1 -- 172.21.15.89:0/2100133438 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea80a2c50 con 0x7f4ea80a6860 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eafdae640 10 monclient: _renew_subs 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eafdae640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eafdae640 1 -- 172.21.15.89:0/2100133438 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea80a3d50 con 0x7f4ea80a6860 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eafdae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eafdae640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a0440 0x7f4ea809d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eafdae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eafdae640 1 -- 172.21.15.89:0/2100133438 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4ea80a6c40 con 0x7f4ea80a0440 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4ead349640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a0440 0x7f4ea809d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4ead349640 10 monclient: get_auth_request con 0x7f4ea80a0440 auth_method 0 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4ead349640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4ead349640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4ead349640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4ead349640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4ead349640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4ead349640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4ead349640 10 monclient(hunting): handle_auth_done global_id 44772 payload 824 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4ead349640 1 -- 172.21.15.89:0/2100133438 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ea8064590 con 0x7f4ea80a0440 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4e98004040 con 0x7f4ea80a6860 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e8c03be00 0x7f4e8c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f4e980e86e0 con 0x7f4ea80a6860 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eae34b640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e8c03be00 0x7f4e8c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.050+0000 7f4eae34b640 10 monclient: get_auth_request con 0x7f4e8c03be00 auth_method 0 2024-08-19T10:49:14.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4ead349640 1 -- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a0440 msgr2=0x7f4ea809d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4ead349640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a0440 0x7f4ea809d140 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4003fd0 tx=0x7f4ea4007700 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ea4014a60 con 0x7f4ea80a0440 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4e9e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4e9e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4e9e7fc640 10 monclient: dump: 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.066 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.067 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.067 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.067 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.067 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.067 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.067 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.067 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.067 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=272016597}) 2024-08-19T10:49:14.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4ea401a070 con 0x7f4ea80a0440 2024-08-19T10:49:14.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4e9e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:14.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4e9e7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:14.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a0440 msgr2=0x7f4ea809d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4e9e7fc640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a0440 0x7f4ea809d140 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4003fd0 tx=0x7f4ea4007700 comp rx=0 tx=0).stop 2024-08-19T10:49:14.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a0440 msgr2=0x7f4ea809d140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:14.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4e9e7fc640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a0440 0x7f4ea809d140 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4003fd0 tx=0x7f4ea4007700 comp rx=0 tx=0).stop 2024-08-19T10:49:14.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=89486579}) 2024-08-19T10:49:14.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=272016597}) 2024-08-19T10:49:14.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1849912269}) 2024-08-19T10:49:14.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=272016597}) 2024-08-19T10:49:14.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3388838455}) 2024-08-19T10:49:14.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=272016597}) 2024-08-19T10:49:14.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f4eae34b640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e8c03be00 0x7f4e8c03e2c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000b770 tx=0x7f4ea0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2992510563}) 2024-08-19T10:49:14.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=272016597}) 2024-08-19T10:49:14.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=55577230}) 2024-08-19T10:49:14.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=272016597}) 2024-08-19T10:49:14.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2826012424}) 2024-08-19T10:49:14.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=272016597}) 2024-08-19T10:49:14.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2134302062}) 2024-08-19T10:49:14.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=272016597}) 2024-08-19T10:49:14.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.054+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1976135256}) 2024-08-19T10:49:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4eafdae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4eafdae640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea809d680 0x7f4ea809dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4eafdae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4eafdae640 1 -- 172.21.15.89:0/2100133438 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f4ea809dff0 con 0x7f4ea809d680 2024-08-19T10:49:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4ead349640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea809d680 0x7f4ea809dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4ead349640 10 monclient: get_auth_request con 0x7f4ea809d680 auth_method 0 2024-08-19T10:49:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4ead349640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4ead349640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4ead349640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4ead349640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4ead349640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4ead349640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4ead349640 10 monclient(hunting): handle_auth_done global_id 44781 payload 824 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.062+0000 7f4ead349640 1 -- 172.21.15.89:0/2100133438 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4ea809f060 con 0x7f4ea809d680 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4ead349640 1 -- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea809d680 msgr2=0x7f4ea809dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4ead349640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea809d680 0x7f4ea809dab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4003fd0 tx=0x7f4ea4004a50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4ea4014e20 con 0x7f4ea809d680 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4e9e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4e9e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4e9e7fc640 10 monclient: dump: 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.075 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.076 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f4ea4014e20 con 0x7f4ea809d680 2024-08-19T10:49:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4e9e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4e9e7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea809d680 msgr2=0x7f4ea809dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4e9e7fc640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea809d680 0x7f4ea809dab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4003fd0 tx=0x7f4ea4004a50 comp rx=0 tx=0).stop 2024-08-19T10:49:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4e9e7fc640 1 -- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea809d680 msgr2=0x7f4ea809dab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:14.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4e9e7fc640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea809d680 0x7f4ea809dab0 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4003fd0 tx=0x7f4ea4004a50 comp rx=0 tx=0).stop 2024-08-19T10:49:14.077 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:14.077 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:14.077 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 277193, 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:14.078 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.079 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.080 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.080 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.080 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.080 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.080 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.080 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:14.080 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:14.080 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.080 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.080 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.080 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:14.081 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:14.082 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.083 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.084 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.084 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:14.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.084 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:14.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.084 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:14.084 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:14.084 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.085 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:14.086 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:14.087 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.088 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.089 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:14.091 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:14.092 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.095 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.096 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:14.096 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:14.096 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.096 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:14.096 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:14.097 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:14.098 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.099 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.100 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:49:14.102 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.103 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.103 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:14.103 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eafdae640 1 -- 172.21.15.89:0/2100133438 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e8c03be00 msgr2=0x7f4e8c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eafdae640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4e8c03be00 0x7f4e8c03e2c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000b770 tx=0x7f4ea0008040 comp rx=0 tx=0).stop 2024-08-19T10:49:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eafdae640 10 monclient: shutdown 2024-08-19T10:49:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eafdae640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eafdae640 1 -- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 msgr2=0x7f4ea8134930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eafdae640 1 --2- 172.21.15.89:0/2100133438 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f4ea80a6860 0x7f4ea8134930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009870 tx=0x7f4e980191c0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eae34b640 1 -- 172.21.15.89:0/2100133438 reap_dead start 2024-08-19T10:49:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eae34b640 1 -- 172.21.15.89:0/2100133438 reap_dead start 2024-08-19T10:49:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eafdae640 1 -- 172.21.15.89:0/2100133438 shutdown_connections 2024-08-19T10:49:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eafdae640 1 -- 172.21.15.89:0/2100133438 >> 172.21.15.89:0/2100133438 conn(0x7f4ea8074b40 msgr2=0x7f4ea80ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eafdae640 1 -- 172.21.15.89:0/2100133438 shutdown_connections 2024-08-19T10:49:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.066+0000 7f4eafdae640 1 -- 172.21.15.89:0/2100133438 wait complete. 2024-08-19T10:49:14.105 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:14.106 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:49:14.106 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:49:14.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient: get_monmap_and_config 2024-08-19T10:49:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient: monmap: 2024-08-19T10:49:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:14.225452+0000 2024-08-19T10:49:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:14.225452+0000 2024-08-19T10:49:14.221 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:14.222 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 1 Processor -- start 2024-08-19T10:49:14.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 1 -- start start 2024-08-19T10:49:14.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient: init 2024-08-19T10:49:14.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient: _add_conns ranks=[3,0,7,4,2,10,5,6,8,9,1] 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58140a6860 0x7f58140a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient(hunting): picked mon.noname-g con 0x7f58140a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f58140a7180 addr v1:172.21.15.89:6789/0 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58140a7560 0x7f5814139d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient(hunting): picked mon.noname-i con 0x7f58140a7560 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58140a33d0 con 0x7f58140a7180 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f58140a8190 con 0x7f58140a6860 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f58140a79f0 con 0x7f58140a7560 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f581a7be640 10 monclient(hunting): authenticate will time out at 1604.042013s 2024-08-19T10:49:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f58137fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f58140a7180 0x7f5814139620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:33188/0 (socket says 172.21.15.89:33188) 2024-08-19T10:49:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f58137fe640 1 -- 172.21.15.89:0/1767522960 learned_addr learned my addr 172.21.15.89:0/1767522960 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 1 --2- 172.21.15.89:0/1767522960 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58140a7560 0x7f5814139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5813fff640 1 --2- 172.21.15.89:0/1767522960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58140a6860 0x7f58140a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 10 monclient(hunting): get_auth_request con 0x7f58140a7560 auth_method 0 2024-08-19T10:49:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5813fff640 10 monclient(hunting): get_auth_request con 0x7f58140a6860 auth_method 0 2024-08-19T10:49:14.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5813fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5813fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5813fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5813fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5813fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5813fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 10 monclient(hunting): handle_auth_done global_id 64168 payload 274 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 1 -- 172.21.15.89:0/1767522960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58140a6860 msgr2=0x7f58140a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 1 --2- 172.21.15.89:0/1767522960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58140a6860 0x7f58140a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 1 -- 172.21.15.89:0/1767522960 >> v1:172.21.15.89:6789/0 conn(0x7f58140a7180 legacy=0x7f5814139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 10 monclient: _finish_hunting 0 2024-08-19T10:49:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 1 monclient: found mon.noname-i 2024-08-19T10:49:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:49:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5818d5b640 1 -- 172.21.15.89:0/1767522960 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58140a2ec0 con 0x7f58140a7560 2024-08-19T10:49:14.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.222+0000 7f5813fff640 1 --2- 172.21.15.89:0/1767522960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58140a6860 0x7f58140a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:14.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5818d5b640 1 -- 172.21.15.89:0/1767522960 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58140a7560 msgr2=0x7f5814139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5818d5b640 1 --2- 172.21.15.89:0/1767522960 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58140a7560 0x7f5814139d30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f580c0097f0 tx=0x7f580c01c3a0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=1a6a4b9087652604 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 1 -- 172.21.15.89:0/1767522960 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f580c0b0070 con 0x7f58140a7560 2024-08-19T10:49:14.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:49:14.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 10 monclient: dump: 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.233 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 10 monclient: _finish_auth 0 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f581a7be640 5 monclient: authenticate success, global_id 64168 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 1 -- 172.21.15.89:0/1767522960 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580c0aa810 con 0x7f58140a7560 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f581a7be640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f581a7be640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 1 -- 172.21.15.89:0/1767522960 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f580c0a9300 con 0x7f58140a7560 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f581a7be640 4 set_mon_vals no callback set 2024-08-19T10:49:14.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:49:14.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.226+0000 7f5812ffd640 10 monclient: dump: 2024-08-19T10:49:14.235 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.235 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.235 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.235 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.235 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.236 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient: shutdown 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 -- 172.21.15.89:0/1767522960 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58140a7560 msgr2=0x7f5814139d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 --2- 172.21.15.89:0/1767522960 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58140a7560 0x7f5814139d30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f580c0097f0 tx=0x7f580c01c3a0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 -- 172.21.15.89:0/1767522960 shutdown_connections 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 --2- 172.21.15.89:0/1767522960 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58140a7560 0x7f5814139d30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 --2- 172.21.15.89:0/1767522960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58140a6860 0x7f58140a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 -- 172.21.15.89:0/1767522960 >> 172.21.15.89:0/1767522960 conn(0x7f5814074b40 msgr2=0x7f58140abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 -- 172.21.15.89:0/1767522960 shutdown_connections 2024-08-19T10:49:14.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 -- 172.21.15.89:0/1767522960 wait complete. 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient: monmap: 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.238 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 Processor -- start 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 -- start start 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient: init 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient: _add_conns ranks=[0,5,3,8,7,1,6,2,4] 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58140a6860 0x7f5814138770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient(hunting): picked mon.noname-a con 0x7f58140a6860 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:14.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f58140a7180 0x7f5814138cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient(hunting): picked mon.noname-h con 0x7f58140a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:14.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58140a7560 0x7f581413e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient(hunting): picked mon.noname-g con 0x7f58140a7560 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:14.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f581413a3f0 con 0x7f58140a6860 2024-08-19T10:49:14.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f581413a270 con 0x7f58140a7560 2024-08-19T10:49:14.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f58141440a0 con 0x7f58140a7180 2024-08-19T10:49:14.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f581a7be640 10 monclient(hunting): authenticate will time out at 1604.050095s 2024-08-19T10:49:14.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58140a6860 0x7f5814138770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58140a6860 0x7f5814138770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60658/0 (socket says 172.21.15.89:60658) 2024-08-19T10:49:14.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 1 -- 172.21.15.89:0/2982170980 learned_addr learned my addr 172.21.15.89:0/2982170980 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:14.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 10 monclient(hunting): get_auth_request con 0x7f58140a6860 auth_method 0 2024-08-19T10:49:14.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f58137fe640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f58140a7180 0x7f5814138cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5818d5b640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58140a7560 0x7f581413e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f58137fe640 10 monclient(hunting): get_auth_request con 0x7f58140a7180 auth_method 0 2024-08-19T10:49:14.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f58137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f58137fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f58137fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 10 monclient(hunting): handle_auth_done global_id 6939 payload 824 2024-08-19T10:49:14.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 1 -- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f58140a7180 msgr2=0x7f5814138cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f58140a7180 0x7f5814138cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 1 -- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58140a7560 msgr2=0x7f581413e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58140a7560 0x7f581413e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 10 monclient: _finish_hunting 0 2024-08-19T10:49:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 1 monclient: found mon.noname-a 2024-08-19T10:49:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5813fff640 1 -- 172.21.15.89:0/2982170980 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5814123360 con 0x7f58140a6860 2024-08-19T10:49:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5818d5b640 10 monclient: get_auth_request con 0x7f58140a7560 auth_method 0 2024-08-19T10:49:14.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.230+0000 7f5818d5b640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58140a7560 0x7f581413e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:14.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5813fff640 1 -- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58140a6860 msgr2=0x7f5814138770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5813fff640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58140a6860 0x7f5814138770 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f580800ebe0 tx=0x7f580800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5808017070 con 0x7f58140a6860 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 10 monclient: dump: 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.249 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.250 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 10 monclient: _finish_auth 0 2024-08-19T10:49:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:49:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:14.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580800ce40 con 0x7f58140a6860 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5808013d50 con 0x7f58140a6860 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 10 monclient: dump: 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.251 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.252 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f58127fc640 4 set_mon_vals no callback set 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f581a7be640 5 monclient: authenticate success, global_id 6939 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f581a7be640 10 monclient: _renew_subs 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f581a7be640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f581a7be640 1 -- 172.21.15.89:0/2982170980 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5814123640 con 0x7f58140a6860 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f581a7be640 10 monclient: _renew_subs 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f581a7be640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f581a7be640 1 -- 172.21.15.89:0/2982170980 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5814123080 con 0x7f58140a6860 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f581a7be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f581a7be640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413e6f0 0x7f581413ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f581a7be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f581a7be640 1 -- 172.21.15.89:0/2982170980 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f58140a26b0 con 0x7f581413e6f0 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f58137fe640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413e6f0 0x7f581413ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f58137fe640 10 monclient: get_auth_request con 0x7f581413e6f0 auth_method 0 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f58137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f58137fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f58137fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f58137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f58137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f58137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f580801a070 con 0x7f58140a6860 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f58137fe640 10 monclient(hunting): handle_auth_done global_id 6776 payload 824 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f58137fe640 1 -- 172.21.15.89:0/2982170980 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58140a2110 con 0x7f581413e6f0 2024-08-19T10:49:14.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f580003be30 0x7f580003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f580800f040 con 0x7f58140a6860 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5818d5b640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f580003be30 0x7f580003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.234+0000 7f5818d5b640 10 monclient: get_auth_request con 0x7f580003be30 auth_method 0 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f58137fe640 1 -- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413e6f0 msgr2=0x7f581413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f58137fe640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413e6f0 0x7f581413ead0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5804005c10 tx=0x7f580400c4b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f580400ce40 con 0x7f581413e6f0 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5810ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5810ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5810ff9640 10 monclient: dump: 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.255 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5804018070 con 0x7f581413e6f0 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5810ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5810ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413e6f0 msgr2=0x7f581413ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5810ff9640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413e6f0 0x7f581413ead0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5804005c10 tx=0x7f580400c4b0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413e6f0 msgr2=0x7f581413ead0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5810ff9640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413e6f0 0x7f581413ead0 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5804005c10 tx=0x7f580400c4b0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.238+0000 7f5818d5b640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f580003be30 0x7f580003e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f580c0ad000 tx=0x7f580c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f581a7be640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f581a7be640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413f010 0x7f581413a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f581a7be640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f581a7be640 1 -- 172.21.15.89:0/2982170980 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f58140a2400 con 0x7f581413f010 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f58137fe640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413f010 0x7f581413a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f58137fe640 10 monclient: get_auth_request con 0x7f581413f010 auth_method 0 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f58137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f58137fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f58137fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f58137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f58137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f58137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f58137fe640 10 monclient(hunting): handle_auth_done global_id 6785 payload 824 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.242+0000 7f58137fe640 1 -- 172.21.15.89:0/2982170980 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5814064590 con 0x7f581413f010 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f58137fe640 1 -- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413f010 msgr2=0x7f581413a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f58137fe640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413f010 0x7f581413a630 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5804017000 tx=0x7f5804004cd0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5804007750 con 0x7f581413f010 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5810ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5810ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5810ff9640 10 monclient: dump: 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.259 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f58040072e0 con 0x7f581413f010 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5810ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5810ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413f010 msgr2=0x7f581413a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5810ff9640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413f010 0x7f581413a630 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5804017000 tx=0x7f5804004cd0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5810ff9640 1 -- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413f010 msgr2=0x7f581413a630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:14.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5810ff9640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f581413f010 0x7f581413a630 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5804017000 tx=0x7f5804004cd0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.261 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:14.261 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:14.261 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:14.261 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:14.261 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:14.261 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 684039, 2024-08-19T10:49:14.261 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:14.261 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:14.261 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:14.262 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.263 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.264 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:14.265 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.266 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:14.267 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.269 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.270 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:14.270 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.270 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.270 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:14.270 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.270 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.270 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.270 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.271 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:14.275 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:14.276 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.277 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.278 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:14.279 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:14.280 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.281 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.282 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.283 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.284 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:14.285 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:14.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f581a7be640 1 -- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f580003be30 msgr2=0x7f580003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f581a7be640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f580003be30 0x7f580003e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f580c0ad000 tx=0x7f580c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:49:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f581a7be640 10 monclient: shutdown 2024-08-19T10:49:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f581a7be640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f581a7be640 1 -- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58140a6860 msgr2=0x7f5814138770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f581a7be640 1 --2- 172.21.15.89:0/2982170980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58140a6860 0x7f5814138770 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f580800ebe0 tx=0x7f580800c520 comp rx=0 tx=0).stop 2024-08-19T10:49:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5818d5b640 1 -- 172.21.15.89:0/2982170980 reap_dead start 2024-08-19T10:49:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f5818d5b640 1 -- 172.21.15.89:0/2982170980 reap_dead start 2024-08-19T10:49:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f581a7be640 1 -- 172.21.15.89:0/2982170980 shutdown_connections 2024-08-19T10:49:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f581a7be640 1 -- 172.21.15.89:0/2982170980 >> 172.21.15.89:0/2982170980 conn(0x7f5814074b40 msgr2=0x7f5814141090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f581a7be640 1 -- 172.21.15.89:0/2982170980 shutdown_connections 2024-08-19T10:49:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.246+0000 7f581a7be640 1 -- 172.21.15.89:0/2982170980 wait complete. 2024-08-19T10:49:14.291 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:14.291 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:49:14.291 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:49:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient: get_monmap_and_config 2024-08-19T10:49:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient: monmap: 2024-08-19T10:49:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:14.413116+0000 2024-08-19T10:49:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:14.413116+0000 2024-08-19T10:49:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:14.410 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 1 Processor -- start 2024-08-19T10:49:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 1 -- start start 2024-08-19T10:49:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient: init 2024-08-19T10:49:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient: _add_conns ranks=[9,8,1,6,2,0,10,3,4,7,5] 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a700a3d00 0x7f1a700a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient(hunting): picked mon.noname-a con 0x7f1a700a3d00 addr v2:172.21.15.89:3300/0 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1a700a1a00 0x7f1a700a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient(hunting): picked mon.noname-e con 0x7f1a700a1a00 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1a7009b1d0 addr v1:172.21.15.190:6789/0 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a7006db50 con 0x7f1a7009b1d0 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f1a7009d3d0 con 0x7f1a700a3d00 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f1a7006fd00 con 0x7f1a700a1a00 2024-08-19T10:49:14.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a74cfd640 10 monclient(hunting): authenticate will time out at 1604.229809s 2024-08-19T10:49:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a700a3d00 0x7f1a700a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1a700a3d00 0x7f1a700a40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60664/0 (socket says 172.21.15.89:60664) 2024-08-19T10:49:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 1 -- 172.21.15.89:0/60782857 learned_addr learned my addr 172.21.15.89:0/60782857 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 10 monclient(hunting): get_auth_request con 0x7f1a700a3d00 auth_method 0 2024-08-19T10:49:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6dd9b640 1 --2- 172.21.15.89:0/60782857 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1a700a1a00 0x7f1a700a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6dd9b640 10 monclient(hunting): get_auth_request con 0x7f1a700a1a00 auth_method 0 2024-08-19T10:49:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 10 monclient(hunting): handle_auth_done global_id 6948 payload 274 2024-08-19T10:49:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 1 -- 172.21.15.89:0/60782857 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1a700a1a00 msgr2=0x7f1a700a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 1 --2- 172.21.15.89:0/60782857 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1a700a1a00 0x7f1a700a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 1 -- 172.21.15.89:0/60782857 >> v1:172.21.15.190:6789/0 conn(0x7f1a7009b1d0 legacy=0x7f1a7009b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:49:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 1 monclient: found mon.noname-a 2024-08-19T10:49:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6e59c640 1 -- 172.21.15.89:0/60782857 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a700a4b60 con 0x7f1a700a3d00 2024-08-19T10:49:14.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.410+0000 7f1a6dd9b640 1 --2- 172.21.15.89:0/60782857 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1a700a1a00 0x7f1a700a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6e59c640 1 -- 172.21.15.89:0/60782857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a700a3d00 msgr2=0x7f1a700a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6e59c640 1 --2- 172.21.15.89:0/60782857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a700a3d00 0x7f1a700a40e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009870 tx=0x7f1a6001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bf04545e71f90d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 1 -- 172.21.15.89:0/60782857 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1a600b0070 con 0x7f1a700a3d00 2024-08-19T10:49:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 10 monclient: dump: 2024-08-19T10:49:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.424 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 10 monclient: _finish_auth 0 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 1 -- 172.21.15.89:0/60782857 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a600aa810 con 0x7f1a700a3d00 2024-08-19T10:49:14.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 1 -- 172.21.15.89:0/60782857 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1a600a9300 con 0x7f1a700a3d00 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 5 monclient: authenticate success, global_id 6948 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 4 set_mon_vals no callback set 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a6d59a640 10 monclient: dump: 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.426 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 10 monclient: shutdown 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:14.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 1 -- 172.21.15.89:0/60782857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a700a3d00 msgr2=0x7f1a700a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 1 --2- 172.21.15.89:0/60782857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a700a3d00 0x7f1a700a40e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009870 tx=0x7f1a6001c420 comp rx=0 tx=0).stop 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 1 -- 172.21.15.89:0/60782857 shutdown_connections 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 1 --2- 172.21.15.89:0/60782857 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f1a700a1a00 0x7f1a700a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 1 --2- 172.21.15.89:0/60782857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1a700a3d00 0x7f1a700a40e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 1 -- 172.21.15.89:0/60782857 >> 172.21.15.89:0/60782857 conn(0x7f1a700a4df0 msgr2=0x7f1a700a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 1 -- 172.21.15.89:0/60782857 shutdown_connections 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.414+0000 7f1a74cfd640 1 -- 172.21.15.89:0/60782857 wait complete. 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient: monmap: 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:14.428 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 1 Processor -- start 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 1 -- start start 2024-08-19T10:49:14.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient: init 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient: _add_conns ranks=[7,5,3,8,0,4,2,6,1] 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1a7009b1d0 0x7f1a701378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient(hunting): picked mon.noname-i con 0x7f1a7009b1d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1a700a1a00 0x7f1a70137e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient(hunting): picked mon.noname-h con 0x7f1a700a1a00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a700a3d00 0x7f1a7012d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient(hunting): picked mon.noname-g con 0x7f1a700a3d00 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1a70142150 con 0x7f1a700a3d00 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1a700a5bd0 con 0x7f1a700a1a00 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f1a701422d0 con 0x7f1a7009b1d0 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a74cfd640 10 monclient(hunting): authenticate will time out at 1604.237089s 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6dd9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1a700a1a00 0x7f1a70137e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1a7009b1d0 0x7f1a701378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6ed9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a700a3d00 0x7f1a7012d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1a7009b1d0 0x7f1a701378c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:35632/0 (socket says 172.21.15.89:35632) 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 1 -- 172.21.15.89:0/1436548716 learned_addr learned my addr 172.21.15.89:0/1436548716 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 10 monclient(hunting): get_auth_request con 0x7f1a7009b1d0 auth_method 0 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6dd9b640 10 monclient(hunting): get_auth_request con 0x7f1a700a1a00 auth_method 0 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6ed9d640 10 monclient(hunting): get_auth_request con 0x7f1a700a3d00 auth_method 0 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 10 monclient(hunting): handle_auth_done global_id 64177 payload 824 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 1 -- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1a700a1a00 msgr2=0x7f1a70137e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f1a700a1a00 0x7f1a70137e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 1 -- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a700a3d00 msgr2=0x7f1a7012d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1a700a3d00 0x7f1a7012d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:49:14.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 1 monclient: found mon.noname-i 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.418+0000 7f1a6e59c640 1 -- 172.21.15.89:0/1436548716 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a70074a50 con 0x7f1a7009b1d0 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6e59c640 1 -- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1a7009b1d0 msgr2=0x7f1a701378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6e59c640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1a7009b1d0 0x7f1a701378c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1a600ae040 tx=0x7f1a600191c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1a600b9070 con 0x7f1a7009b1d0 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 10 monclient: dump: 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.434 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 10 monclient: _finish_auth 0 2024-08-19T10:49:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a600044b0 con 0x7f1a7009b1d0 2024-08-19T10:49:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a74cfd640 5 monclient: authenticate success, global_id 64177 2024-08-19T10:49:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1a600b0040 con 0x7f1a7009b1d0 2024-08-19T10:49:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:49:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6cd99640 4 set_mon_vals no callback set 2024-08-19T10:49:14.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 10 monclient: dump: 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.437 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a74cfd640 10 monclient: _renew_subs 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a74cfd640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a74cfd640 1 -- 172.21.15.89:0/1436548716 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a70130330 con 0x7f1a7009b1d0 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a74cfd640 10 monclient: _renew_subs 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a74cfd640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a74cfd640 1 -- 172.21.15.89:0/1436548716 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7012eb30 con 0x7f1a7009b1d0 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a74cfd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:14.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a74cfd640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a701d19d0 0x7f1a7012d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a74cfd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a74cfd640 1 -- 172.21.15.89:0/1436548716 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1a700a6720 con 0x7f1a701d19d0 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6dd9b640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a701d19d0 0x7f1a7012d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6dd9b640 10 monclient: get_auth_request con 0x7f1a701d19d0 auth_method 0 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1a60004040 con 0x7f1a7009b1d0 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_done global_id 44668 payload 824 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6dd9b640 1 -- 172.21.15.89:0/1436548716 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a700a61f0 con 0x7f1a701d19d0 2024-08-19T10:49:14.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1a5003be30 0x7f1a5003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f1a600e85d0 con 0x7f1a7009b1d0 2024-08-19T10:49:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6ed9d640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1a5003be30 0x7f1a5003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.422+0000 7f1a6ed9d640 10 monclient: get_auth_request con 0x7f1a5003be30 auth_method 0 2024-08-19T10:49:14.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a6dd9b640 1 -- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a701d19d0 msgr2=0x7f1a7012d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a6dd9b640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a701d19d0 0x7f1a7012d8f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a7012e640 tx=0x7f1a64011e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1a6400fe20 con 0x7f1a701d19d0 2024-08-19T10:49:14.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a5effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a5effd640 10 monclient: dump: 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.443 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1a6400fe20 con 0x7f1a701d19d0 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a5effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a5effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a6ed9d640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1a5003be30 0x7f1a5003e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1a7013eeb0 tx=0x7f1a68003d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a701d19d0 msgr2=0x7f1a7012d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a5effd640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a701d19d0 0x7f1a7012d8f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a7012e640 tx=0x7f1a64011e30 comp rx=0 tx=0).stop 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a701d19d0 msgr2=0x7f1a7012d8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.426+0000 7f1a5effd640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a701d19d0 0x7f1a7012d8f0 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a7012e640 tx=0x7f1a64011e30 comp rx=0 tx=0).stop 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a74cfd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a74cfd640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a7012de30 0x7f1a70070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a74cfd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a74cfd640 1 -- 172.21.15.89:0/1436548716 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7012e210 con 0x7f1a7012de30 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a6dd9b640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a7012de30 0x7f1a70070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a6dd9b640 10 monclient: get_auth_request con 0x7f1a7012de30 auth_method 0 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a6dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a6dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a6dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a6dd9b640 10 monclient(hunting): handle_auth_done global_id 44677 payload 824 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.434+0000 7f1a6dd9b640 1 -- 172.21.15.89:0/1436548716 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1a70068fb0 con 0x7f1a7012de30 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a6dd9b640 1 -- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a7012de30 msgr2=0x7f1a70070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a6dd9b640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a7012de30 0x7f1a70070cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a70074710 tx=0x7f1a640046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1a64004490 con 0x7f1a7012de30 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a5effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a5effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a5effd640 10 monclient: dump: 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.446 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f1a6401d400 con 0x7f1a7012de30 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a5effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a5effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a7012de30 msgr2=0x7f1a70070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a5effd640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a7012de30 0x7f1a70070cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a70074710 tx=0x7f1a640046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a5effd640 1 -- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a7012de30 msgr2=0x7f1a70070cc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:14.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a5effd640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1a7012de30 0x7f1a70070cc0 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a70074710 tx=0x7f1a640046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.448 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:14.448 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:14.448 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:14.448 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:14.448 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:14.448 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 277565, 2024-08-19T10:49:14.448 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:14.448 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:14.448 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:14.448 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:14.449 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:14.450 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.451 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:14.452 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.453 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:14.454 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.455 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.458 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:14.458 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:14.458 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.458 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.458 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.458 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:14.458 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.460 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.461 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:14.462 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:14.462 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.462 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.462 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.462 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.462 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:14.462 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.464 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.465 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:14.466 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.468 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:14.469 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.470 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.470 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:14.470 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.470 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.470 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.470 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:14.470 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:14.470 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.471 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:14.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.472 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.472 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.472 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:49:14.472 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.472 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.472 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:14.472 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a74cfd640 1 -- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1a5003be30 msgr2=0x7f1a5003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a74cfd640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1a5003be30 0x7f1a5003e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1a7013eeb0 tx=0x7f1a68003d50 comp rx=0 tx=0).stop 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a74cfd640 10 monclient: shutdown 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a74cfd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a74cfd640 1 -- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1a7009b1d0 msgr2=0x7f1a701378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a74cfd640 1 --2- 172.21.15.89:0/1436548716 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1a7009b1d0 0x7f1a701378c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1a600ae040 tx=0x7f1a600191c0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a6ed9d640 1 -- 172.21.15.89:0/1436548716 reap_dead start 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a6ed9d640 1 -- 172.21.15.89:0/1436548716 reap_dead start 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a74cfd640 1 -- 172.21.15.89:0/1436548716 shutdown_connections 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a74cfd640 1 -- 172.21.15.89:0/1436548716 >> 172.21.15.89:0/1436548716 conn(0x7f1a700a4df0 msgr2=0x7f1a7009a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a74cfd640 1 -- 172.21.15.89:0/1436548716 shutdown_connections 2024-08-19T10:49:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.438+0000 7f1a74cfd640 1 -- 172.21.15.89:0/1436548716 wait complete. 2024-08-19T10:49:14.475 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:14.475 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:49:14.475 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:49:14.536 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1208239303}) 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2235955353}) 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1208239303}) 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2418072347}) 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1208239303}) 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2537208573}) 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1208239303}) 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2633503451}) 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.537 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1208239303}) 2024-08-19T10:49:14.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=752412967}) 2024-08-19T10:49:14.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1208239303}) 2024-08-19T10:49:14.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2052668537}) 2024-08-19T10:49:14.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1208239303}) 2024-08-19T10:49:14.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=184069003}) 2024-08-19T10:49:14.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.534+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1208239303}) 2024-08-19T10:49:14.538 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.538+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3807175531}) 2024-08-19T10:49:14.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=867638862}) 2024-08-19T10:49:14.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1215214702}) 2024-08-19T10:49:14.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=867638862}) 2024-08-19T10:49:14.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=389485343}) 2024-08-19T10:49:14.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=867638862}) 2024-08-19T10:49:14.553 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3197815504}) 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=867638862}) 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2152170483}) 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=867638862}) 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3275236934}) 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=867638862}) 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3527306781}) 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=867638862}) 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=370356109}) 2024-08-19T10:49:14.554 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=867638862}) 2024-08-19T10:49:14.555 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.550+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=871421982}) 2024-08-19T10:49:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient: get_monmap_and_config 2024-08-19T10:49:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient: monmap: 2024-08-19T10:49:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.593 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:14.598145+0000 2024-08-19T10:49:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:14.598145+0000 2024-08-19T10:49:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.594 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.595 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.595 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.595 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.595 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:14.595 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:14.595 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 1 Processor -- start 2024-08-19T10:49:14.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 1 -- start start 2024-08-19T10:49:14.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient: init 2024-08-19T10:49:14.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient: _add_conns ranks=[8,1,10,3,4,6,9,0,2,5,7] 2024-08-19T10:49:14.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20d40a5270 0x7f20d40a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient(hunting): picked mon.noname-e con 0x7f20d40a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f20d40a6d30 addr v1:172.21.15.190:6789/0 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f20d40a7110 0x7f20d4139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient(hunting): picked mon.noname-f con 0x7f20d40a7110 addr v2:172.21.15.190:3300/0 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d40a8710 con 0x7f20d40a6d30 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f20d40a5a40 con 0x7f20d40a7110 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f20d40a81d0 con 0x7f20d40a5270 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20daff3640 10 monclient(hunting): authenticate will time out at 1604.415250s 2024-08-19T10:49:14.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20d40a5270 0x7f20d40a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20d40a5270 0x7f20d40a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:37950/0 (socket says 172.21.15.89:37950) 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 1 -- 172.21.15.89:0/288178729 learned_addr learned my addr 172.21.15.89:0/288178729 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 10 monclient(hunting): get_auth_request con 0x7f20d40a5270 auth_method 0 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d9590640 1 --2- 172.21.15.89:0/288178729 >> v2:172.21.15.190:3300/0 conn(0x7f20d40a7110 0x7f20d4139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d9590640 10 monclient(hunting): get_auth_request con 0x7f20d40a7110 auth_method 0 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d9590640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d9590640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d9590640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 10 monclient(hunting): handle_auth_done global_id 6803 payload 274 2024-08-19T10:49:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 1 -- 172.21.15.89:0/288178729 >> v2:172.21.15.190:3300/0 conn(0x7f20d40a7110 msgr2=0x7f20d4139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 1 --2- 172.21.15.89:0/288178729 >> v2:172.21.15.190:3300/0 conn(0x7f20d40a7110 0x7f20d4139cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 1 -- 172.21.15.89:0/288178729 >> v1:172.21.15.190:6789/0 conn(0x7f20d40a6d30 legacy=0x7f20d41395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 10 monclient: _finish_hunting 0 2024-08-19T10:49:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 1 monclient: found mon.noname-e 2024-08-19T10:49:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:14.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.594+0000 7f20d8d8f640 1 -- 172.21.15.89:0/288178729 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d40a4790 con 0x7f20d40a5270 2024-08-19T10:49:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 1 -- 172.21.15.89:0/288178729 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20d40a5270 msgr2=0x7f20d40a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 1 --2- 172.21.15.89:0/288178729 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20d40a5270 0x7f20d40a67f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009e50 tx=0x7f20c401c6e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ff836fdc9e40451d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 1 -- 172.21.15.89:0/288178729 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20c40090d0 con 0x7f20d40a5270 2024-08-19T10:49:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 10 monclient: dump: 2024-08-19T10:49:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.605 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.606 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 10 monclient: _finish_auth 0 2024-08-19T10:49:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:49:14.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 1 -- 172.21.15.89:0/288178729 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c40a5990 con 0x7f20d40a5270 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 1 -- 172.21.15.89:0/288178729 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20c40ad3f0 con 0x7f20d40a5270 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 5 monclient: authenticate success, global_id 6803 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 4 set_mon_vals no callback set 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d37fe640 10 monclient: dump: 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.607 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.608 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient: shutdown 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 -- 172.21.15.89:0/288178729 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20d40a5270 msgr2=0x7f20d40a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 --2- 172.21.15.89:0/288178729 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20d40a5270 0x7f20d40a67f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009e50 tx=0x7f20c401c6e0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 -- 172.21.15.89:0/288178729 shutdown_connections 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 --2- 172.21.15.89:0/288178729 >> v2:172.21.15.190:3300/0 conn(0x7f20d40a7110 0x7f20d4139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 --2- 172.21.15.89:0/288178729 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f20d40a5270 0x7f20d40a67f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 -- 172.21.15.89:0/288178729 >> 172.21.15.89:0/288178729 conn(0x7f20d4069a80 msgr2=0x7f20d40abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 -- 172.21.15.89:0/288178729 shutdown_connections 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 -- 172.21.15.89:0/288178729 wait complete. 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient: monmap: 2024-08-19T10:49:14.609 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.610 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 Processor -- start 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 -- start start 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient: init 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient: _add_conns ranks=[6,5,1,3,8,2,0,4,7] 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20d40a5270 0x7f20d41d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient(hunting): picked mon.noname-d con 0x7f20d40a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f20d40a6d30 0x7f20d41d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient(hunting): picked mon.noname-h con 0x7f20d40a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20d40a7110 0x7f20d413e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient(hunting): picked mon.noname-f con 0x7f20d40a7110 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f20d40a57d0 con 0x7f20d40a7110 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f20d40a5650 con 0x7f20d40a6d30 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f20d413a200 con 0x7f20d40a5270 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20daff3640 10 monclient(hunting): authenticate will time out at 1604.422926s 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20d40a5270 0x7f20d41d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20d40a5270 0x7f20d41d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:54886/0 (socket says 172.21.15.89:54886) 2024-08-19T10:49:14.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 1 -- 172.21.15.89:0/2153083863 learned_addr learned my addr 172.21.15.89:0/2153083863 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 10 monclient(hunting): get_auth_request con 0x7f20d40a5270 auth_method 0 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d9590640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20d40a7110 0x7f20d413e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d3fff640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f20d40a6d30 0x7f20d41d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d3fff640 10 monclient(hunting): get_auth_request con 0x7f20d40a6d30 auth_method 0 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d9590640 10 monclient(hunting): get_auth_request con 0x7f20d40a7110 auth_method 0 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d9590640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d9590640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d9590640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 10 monclient(hunting): handle_auth_done global_id 44790 payload 824 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 1 -- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f20d40a6d30 msgr2=0x7f20d41d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f20d40a6d30 0x7f20d41d18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 1 -- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20d40a7110 msgr2=0x7f20d413e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20d40a7110 0x7f20d413e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 10 monclient: _finish_hunting 0 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 1 monclient: found mon.noname-d 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:14.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:49:14.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.602+0000 7f20d8d8f640 1 -- 172.21.15.89:0/2153083863 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d4122f30 con 0x7f20d40a5270 2024-08-19T10:49:14.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d8d8f640 1 -- 172.21.15.89:0/2153083863 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20d40a5270 msgr2=0x7f20d41d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d8d8f640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20d40a5270 0x7f20d41d13b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f20c401cf80 tx=0x7f20c4019480 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20c4003e10 con 0x7f20d40a5270 2024-08-19T10:49:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:49:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 10 monclient: dump: 2024-08-19T10:49:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.619 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 10 monclient: _finish_auth 0 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 10 monclient: _check_auth_tickets 2024-08-19T10:49:14.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c4004040 con 0x7f20d40a5270 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20daff3640 5 monclient: authenticate success, global_id 44790 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20daff3640 10 monclient: _renew_subs 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20daff3640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20daff3640 1 -- 172.21.15.89:0/2153083863 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d4123210 con 0x7f20d40a5270 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20c4004660 con 0x7f20d40a5270 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 10 monclient: dump: 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.621 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.622 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d2ffd640 4 set_mon_vals no callback set 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20daff3640 10 monclient: _renew_subs 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20daff3640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20daff3640 1 -- 172.21.15.89:0/2153083863 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d41407d0 con 0x7f20d40a5270 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20daff3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20daff3640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d4140150 0x7f20d413e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20daff3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20daff3640 1 -- 172.21.15.89:0/2153083863 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f20d40a67f0 con 0x7f20d4140150 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f20c40be030 con 0x7f20d40a5270 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d3fff640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d4140150 0x7f20d413e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d3fff640 10 monclient: get_auth_request con 0x7f20d4140150 auth_method 0 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20c003be00 0x7f20c003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f20c40bc020 con 0x7f20d40a5270 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d9590640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20c003be00 0x7f20c003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d3fff640 10 monclient(hunting): handle_auth_done global_id 35400 payload 824 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d3fff640 1 -- 172.21.15.89:0/2153083863 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20d4064590 con 0x7f20d4140150 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.606+0000 7f20d9590640 10 monclient: get_auth_request con 0x7f20c003be00 auth_method 0 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d3fff640 1 -- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d4140150 msgr2=0x7f20d413e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d3fff640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d4140150 0x7f20d413e560 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f20d41400e0 tx=0x7f20c800e8b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d9590640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20c003be00 0x7f20c003e2c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00b950 tx=0x7f20cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20c8006c50 con 0x7f20d4140150 2024-08-19T10:49:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:49:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d17fa640 10 monclient: dump: 2024-08-19T10:49:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.625 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f20c8007450 con 0x7f20d4140150 2024-08-19T10:49:14.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d17fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d4140150 msgr2=0x7f20d413e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d17fa640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d4140150 0x7f20d413e560 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f20d41400e0 tx=0x7f20c800e8b0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d4140150 msgr2=0x7f20d413e560 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.610+0000 7f20d17fa640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d4140150 0x7f20d413e560 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20daff3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20daff3640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d413eaa0 0x7f20d413eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20daff3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20daff3640 1 -- 172.21.15.89:0/2153083863 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f20d413f3f0 con 0x7f20d413eaa0 2024-08-19T10:49:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20d3fff640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d413eaa0 0x7f20d413eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20d3fff640 10 monclient: get_auth_request con 0x7f20d413eaa0 auth_method 0 2024-08-19T10:49:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20d3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20d3fff640 10 monclient(hunting): handle_auth_done global_id 35409 payload 824 2024-08-19T10:49:14.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.618+0000 7f20d3fff640 1 -- 172.21.15.89:0/2153083863 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f20d4123450 con 0x7f20d413eaa0 2024-08-19T10:49:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d3fff640 1 -- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d413eaa0 msgr2=0x7f20d413eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d3fff640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d413eaa0 0x7f20d413eeb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f20d41400e0 tx=0x7f20c8003770 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f20c8013890 con 0x7f20d413eaa0 2024-08-19T10:49:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:49:14.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d17fa640 10 monclient: dump: 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.630 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.631 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9434 (secure 0 0 0) 0x7f20c8013420 con 0x7f20d413eaa0 2024-08-19T10:49:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d17fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d413eaa0 msgr2=0x7f20d413eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d17fa640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d413eaa0 0x7f20d413eeb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f20d41400e0 tx=0x7f20c8003770 comp rx=0 tx=0).stop 2024-08-19T10:49:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d17fa640 1 -- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d413eaa0 msgr2=0x7f20d413eeb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:14.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d17fa640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20d413eaa0 0x7f20d413eeb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.632 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 399671, 2024-08-19T10:49:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:49:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:14.636 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.637 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:14.638 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.639 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:14.640 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.641 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.642 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:14.642 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:14.642 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.642 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.642 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.642 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:14.642 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:14.642 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.643 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.644 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:14.644 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:14.644 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.644 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.644 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.644 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:14.644 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:14.644 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.645 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.646 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:14.647 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.650 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.651 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.652 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.655 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.656 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.657 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.657 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:49:14.657 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.657 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.657 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:14.657 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20daff3640 1 -- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20c003be00 msgr2=0x7f20c003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20daff3640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f20c003be00 0x7f20c003e2c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00b950 tx=0x7f20cc008040 comp rx=0 tx=0).stop 2024-08-19T10:49:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20daff3640 10 monclient: shutdown 2024-08-19T10:49:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20daff3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20daff3640 1 -- 172.21.15.89:0/2153083863 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20d40a5270 msgr2=0x7f20d41d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20daff3640 1 --2- 172.21.15.89:0/2153083863 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f20d40a5270 0x7f20d41d13b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f20c401cf80 tx=0x7f20c4019480 comp rx=0 tx=0).stop 2024-08-19T10:49:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d9590640 1 -- 172.21.15.89:0/2153083863 reap_dead start 2024-08-19T10:49:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20d9590640 1 -- 172.21.15.89:0/2153083863 reap_dead start 2024-08-19T10:49:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20daff3640 1 -- 172.21.15.89:0/2153083863 shutdown_connections 2024-08-19T10:49:14.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20daff3640 1 -- 172.21.15.89:0/2153083863 >> 172.21.15.89:0/2153083863 conn(0x7f20d4069a80 msgr2=0x7f20d40aa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:14.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20daff3640 1 -- 172.21.15.89:0/2153083863 shutdown_connections 2024-08-19T10:49:14.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.622+0000 7f20daff3640 1 -- 172.21.15.89:0/2153083863 wait complete. 2024-08-19T10:49:14.660 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:14.660 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:49:14.660 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:49:14.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient: get_monmap_and_config 2024-08-19T10:49:14.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient: monmap: 2024-08-19T10:49:14.776 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:14.781177+0000 2024-08-19T10:49:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:14.781177+0000 2024-08-19T10:49:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.777 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:14.778 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:14.778 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 1 Processor -- start 2024-08-19T10:49:14.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 1 -- start start 2024-08-19T10:49:14.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient: init 2024-08-19T10:49:14.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient: _add_conns ranks=[4,2,8,1,5,3,6,0,7,9,10] 2024-08-19T10:49:14.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74280a3d00 0x7f74280a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient(hunting): picked mon.noname-c con 0x7f74280a3d00 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:14.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 0x7f74280a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient(hunting): picked mon.noname-b con 0x7f74280a1a00 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:14.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f742809b1d0 0x7f742809b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient(hunting): picked mon.noname-e con 0x7f742809b1d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:14.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f742809d3d0 con 0x7f74280a1a00 2024-08-19T10:49:14.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f742806fd00 con 0x7f74280a3d00 2024-08-19T10:49:14.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f74280a4b60 con 0x7f742809b1d0 2024-08-19T10:49:14.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742c8c0640 10 monclient(hunting): authenticate will time out at 1604.597695s 2024-08-19T10:49:14.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74280a3d00 0x7f74280a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74280a3d00 0x7f74280a40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48518/0 (socket says 172.21.15.89:48518) 2024-08-19T10:49:14.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 1 -- 172.21.15.89:0/2899209559 learned_addr learned my addr 172.21.15.89:0/2899209559 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:14.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 10 monclient(hunting): get_auth_request con 0x7f74280a3d00 auth_method 0 2024-08-19T10:49:14.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7425d9b640 1 --2- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 0x7f74280a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7426d9d640 1 --2- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f742809b1d0 0x7f742809b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7425d9b640 10 monclient(hunting): get_auth_request con 0x7f74280a1a00 auth_method 0 2024-08-19T10:49:14.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7425d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7425d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7425d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7426d9d640 10 monclient(hunting): get_auth_request con 0x7f742809b1d0 auth_method 0 2024-08-19T10:49:14.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7426d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7426d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7426d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7425d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7425d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7425d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7426d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7426d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7426d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 10 monclient(hunting): handle_auth_done global_id 44797 payload 274 2024-08-19T10:49:14.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 1 -- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f742809b1d0 msgr2=0x7f742809b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 1 --2- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f742809b1d0 0x7f742809b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 1 -- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 msgr2=0x7f74280a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 1 --2- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 0x7f74280a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 10 monclient: _finish_hunting 0 2024-08-19T10:49:14.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 1 monclient: found mon.noname-c 2024-08-19T10:49:14.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:14.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:49:14.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f742659c640 1 -- 172.21.15.89:0/2899209559 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742806db50 con 0x7f74280a3d00 2024-08-19T10:49:14.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7426d9d640 1 --2- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f742809b1d0 0x7f742809b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:14.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.778+0000 7f7425d9b640 1 --2- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 0x7f74280a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742659c640 1 -- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74280a3d00 msgr2=0x7f74280a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742659c640 1 --2- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74280a3d00 0x7f74280a40e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7418009900 tx=0x7f741801c3c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=ecae1e1c83c10ea6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 1 -- 172.21.15.89:0/2899209559 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74180b0070 con 0x7f74280a3d00 2024-08-19T10:49:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:49:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 10 monclient: dump: 2024-08-19T10:49:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.793 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.794 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 10 monclient: _finish_auth 0 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 10 monclient: _check_auth_tickets 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 1 -- 172.21.15.89:0/2899209559 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74180aa810 con 0x7f74280a3d00 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 1 -- 172.21.15.89:0/2899209559 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74180aae30 con 0x7f74280a3d00 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 5 monclient: authenticate success, global_id 44797 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 4 set_mon_vals no callback set 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742559a640 10 monclient: dump: 2024-08-19T10:49:14.795 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.796 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 10 monclient: shutdown 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 1 -- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74280a3d00 msgr2=0x7f74280a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 1 --2- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74280a3d00 0x7f74280a40e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7418009900 tx=0x7f741801c3c0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 1 -- 172.21.15.89:0/2899209559 shutdown_connections 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 1 --2- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f742809b1d0 0x7f742809b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 1 --2- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 0x7f74280a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 1 --2- 172.21.15.89:0/2899209559 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f74280a3d00 0x7f74280a40e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 1 -- 172.21.15.89:0/2899209559 >> 172.21.15.89:0/2899209559 conn(0x7f74280a4df0 msgr2=0x7f74280a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 1 -- 172.21.15.89:0/2899209559 shutdown_connections 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.782+0000 7f742c8c0640 1 -- 172.21.15.89:0/2899209559 wait complete. 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient: monmap: 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:14.798 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:14.799 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.799 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.799 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.799 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.799 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.799 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.799 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 1 Processor -- start 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 1 -- start start 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient: init 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient: _add_conns ranks=[5,2,8,0,3,1,7,4,6] 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742809b1d0 0x7f74281cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient(hunting): picked mon.noname-h con 0x7f742809b1d0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 0x7f74281378f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient(hunting): picked mon.noname-b con 0x7f74280a1a00 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74280a3d00 0x7f7428137e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient(hunting): picked mon.noname-e con 0x7f74280a3d00 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f74280a5bd0 con 0x7f74280a1a00 2024-08-19T10:49:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f742809bf60 con 0x7f742809b1d0 2024-08-19T10:49:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7428141140 con 0x7f74280a3d00 2024-08-19T10:49:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient(hunting): authenticate will time out at 1604.604977s 2024-08-19T10:49:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7426d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74280a3d00 0x7f7428137e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 0x7f74281378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 0x7f74281378f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:32778/0 (socket says 172.21.15.89:32778) 2024-08-19T10:49:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7426d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74280a3d00 0x7f7428137e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:37962/0 (socket says 172.21.15.89:37962) 2024-08-19T10:49:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 1 -- 172.21.15.89:0/4019770128 learned_addr learned my addr 172.21.15.89:0/4019770128 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 10 monclient(hunting): get_auth_request con 0x7f74280a1a00 auth_method 0 2024-08-19T10:49:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7426d9d640 10 monclient(hunting): get_auth_request con 0x7f74280a3d00 auth_method 0 2024-08-19T10:49:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7426d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7426d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7426d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7426d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7426d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7426d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742659c640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742809b1d0 0x7f74281cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742659c640 10 monclient(hunting): get_auth_request con 0x7f742809b1d0 auth_method 0 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 10 monclient(hunting): handle_auth_done global_id 54515 payload 824 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 1 -- 172.21.15.89:0/4019770128 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74280a3d00 msgr2=0x7f7428137e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74280a3d00 0x7f7428137e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 1 -- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742809b1d0 msgr2=0x7f74281cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742809b1d0 0x7f74281cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 1 monclient: found mon.noname-b 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 1 -- 172.21.15.89:0/4019770128 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74280a6510 con 0x7f74280a1a00 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7426d9d640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f74280a3d00 0x7f7428137e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 1 -- 172.21.15.89:0/4019770128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 msgr2=0x7f74281378f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7425d9b640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 0x7f74281378f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f741c00e9b0 tx=0x7f741c00edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f741c017070 con 0x7f74280a1a00 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7416ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7416ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7416ffd640 10 monclient: dump: 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.805 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7416ffd640 10 monclient: _finish_auth 0 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7416ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7416ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741c013470 con 0x7f74280a1a00 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 5 monclient: authenticate success, global_id 54515 2024-08-19T10:49:14.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient: _renew_subs 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.786+0000 7f742c8c0640 1 -- 172.21.15.89:0/4019770128 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742813d910 con 0x7f74280a1a00 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f7416ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f741c013a90 con 0x7f74280a1a00 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f7416ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f7416ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f7416ffd640 10 monclient: dump: 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.807 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f7424d99640 4 set_mon_vals no callback set 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742c8c0640 10 monclient: _renew_subs 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742c8c0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742c8c0640 1 -- 172.21.15.89:0/4019770128 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f742812dd80 con 0x7f74280a1a00 2024-08-19T10:49:14.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742c8c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742c8c0640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813ace0 0x7f742813b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742c8c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742c8c0640 1 -- 172.21.15.89:0/4019770128 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f742812e0d0 con 0x7f742813ace0 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f741c01a070 con 0x7f74280a1a00 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742659c640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813ace0 0x7f742813b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742659c640 10 monclient: get_auth_request con 0x7f742813ace0 auth_method 0 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f7416ffd640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f740803be30 0x7f740803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f741c00f040 con 0x7f74280a1a00 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f7426d9d640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f740803be30 0x7f740803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f7426d9d640 10 monclient: get_auth_request con 0x7f740803be30 auth_method 0 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742659c640 10 monclient(hunting): handle_auth_done global_id 6935 payload 824 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.790+0000 7f742659c640 1 -- 172.21.15.89:0/4019770128 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74280a1800 con 0x7f742813ace0 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f742659c640 1 -- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813ace0 msgr2=0x7f742813b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f742659c640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813ace0 0x7f742813b0c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f74180098d0 tx=0x7f74180191c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74180ba070 con 0x7f742813ace0 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7416ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7426d9d640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f740803be30 0x7f740803e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f74281d04a0 tx=0x7f742000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7416ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7416ffd640 10 monclient: dump: 2024-08-19T10:49:14.810 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.811 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7418003830 con 0x7f742813ace0 2024-08-19T10:49:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7416ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7416ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813ace0 msgr2=0x7f742813b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7416ffd640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813ace0 0x7f742813b0c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f74180098d0 tx=0x7f74180191c0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813ace0 msgr2=0x7f742813b0c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:14.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.794+0000 7f7416ffd640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813ace0 0x7f742813b0c0 secure :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f74180098d0 tx=0x7f74180191c0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742c8c0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742c8c0640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813b600 0x7f7428099350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742c8c0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742c8c0640 1 -- 172.21.15.89:0/4019770128 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f742809b610 con 0x7f742813b600 2024-08-19T10:49:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742659c640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813b600 0x7f7428099350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742659c640 10 monclient: get_auth_request con 0x7f742813b600 auth_method 0 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.798+0000 7f742659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f742659c640 10 monclient(hunting): handle_auth_done global_id 6944 payload 824 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f742659c640 1 -- 172.21.15.89:0/4019770128 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7428064590 con 0x7f742813b600 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f742659c640 1 -- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813b600 msgr2=0x7f7428099350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f742659c640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813b600 0x7f7428099350 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f741801c1f0 tx=0x7f74180a8000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f74180b0040 con 0x7f742813b600 2024-08-19T10:49:14.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f7416ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f7416ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f7416ffd640 10 monclient: dump: 2024-08-19T10:49:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.816 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.817 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9601 (secure 0 0 0) 0x7f74180c1270 con 0x7f742813b600 2024-08-19T10:49:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f7416ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f7416ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:14.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813b600 msgr2=0x7f7428099350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f7416ffd640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813b600 0x7f7428099350 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f741801c1f0 tx=0x7f74180a8000 comp rx=0 tx=0).stop 2024-08-19T10:49:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f7416ffd640 1 -- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813b600 msgr2=0x7f7428099350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:14.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.802+0000 7f7416ffd640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f742813b600 0x7f7428099350 secure :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f741801c1f0 tx=0x7f74180a8000 comp rx=0 tx=0).stop 2024-08-19T10:49:14.818 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:14.818 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:14.818 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:14.818 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:14.818 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 684573, 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:14.819 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.820 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.821 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:14.822 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:14.823 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.824 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.825 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.826 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:14.828 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.829 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.830 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:14.830 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:14.830 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.830 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:14.830 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.831 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.832 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.833 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:14.834 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.837 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:14.838 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:14.839 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:14.840 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.841 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:14.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.842 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.842 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.842 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:14.842 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.842 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.842 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:14.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.842 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.842 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.842 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:14.843 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f742c8c0640 1 -- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f740803be30 msgr2=0x7f740803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f742c8c0640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f740803be30 0x7f740803e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f74281d04a0 tx=0x7f742000b040 comp rx=0 tx=0).stop 2024-08-19T10:49:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f742c8c0640 10 monclient: shutdown 2024-08-19T10:49:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f742c8c0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f742c8c0640 1 -- 172.21.15.89:0/4019770128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 msgr2=0x7f74281378f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f742c8c0640 1 --2- 172.21.15.89:0/4019770128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f74280a1a00 0x7f74281378f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f741c00e9b0 tx=0x7f741c00edb0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f7426d9d640 1 -- 172.21.15.89:0/4019770128 reap_dead start 2024-08-19T10:49:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f7426d9d640 1 -- 172.21.15.89:0/4019770128 reap_dead start 2024-08-19T10:49:14.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f742c8c0640 1 -- 172.21.15.89:0/4019770128 shutdown_connections 2024-08-19T10:49:14.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f742c8c0640 1 -- 172.21.15.89:0/4019770128 >> 172.21.15.89:0/4019770128 conn(0x7f74280a4df0 msgr2=0x7f7428073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:14.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f742c8c0640 1 -- 172.21.15.89:0/4019770128 shutdown_connections 2024-08-19T10:49:14.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.806+0000 7f742c8c0640 1 -- 172.21.15.89:0/4019770128 wait complete. 2024-08-19T10:49:14.846 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:14.846 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:49:14.847 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:49:14.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=310900455}) 2024-08-19T10:49:14.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2393356546}) 2024-08-19T10:49:14.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=310900455}) 2024-08-19T10:49:14.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=576678709}) 2024-08-19T10:49:14.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=310900455}) 2024-08-19T10:49:14.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2836349022}) 2024-08-19T10:49:14.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=310900455}) 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4217189217}) 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=310900455}) 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1604544152}) 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=310900455}) 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1727011686}) 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=310900455}) 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3302159897}) 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=310900455}) 2024-08-19T10:49:14.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=291638582}) 2024-08-19T10:49:14.858 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.858 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456453919}) 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=507930622}) 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456453919}) 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1321573782}) 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456453919}) 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1561178241}) 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456453919}) 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1669079257}) 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456453919}) 2024-08-19T10:49:14.860 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2243605611}) 2024-08-19T10:49:14.860 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.860 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456453919}) 2024-08-19T10:49:14.860 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2314587320}) 2024-08-19T10:49:14.860 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.860 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456453919}) 2024-08-19T10:49:14.860 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3648015612}) 2024-08-19T10:49:14.860 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.860 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456453919}) 2024-08-19T10:49:14.860 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.858+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=43673979}) 2024-08-19T10:49:14.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2596202748}) 2024-08-19T10:49:14.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=997859645}) 2024-08-19T10:49:14.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2596202748}) 2024-08-19T10:49:14.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=761155464}) 2024-08-19T10:49:14.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2596202748}) 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3918651220}) 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2596202748}) 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3472460241}) 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2596202748}) 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3881032532}) 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2596202748}) 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1765133473}) 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2596202748}) 2024-08-19T10:49:14.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4100425247}) 2024-08-19T10:49:14.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2596202748}) 2024-08-19T10:49:14.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2281566153}) 2024-08-19T10:49:14.867 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.867 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=293356062}) 2024-08-19T10:49:14.867 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2565521865}) 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=293356062}) 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=970455292}) 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=293356062}) 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=485543602}) 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=293356062}) 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3253678444}) 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=293356062}) 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4195227662}) 2024-08-19T10:49:14.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=293356062}) 2024-08-19T10:49:14.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1064945132}) 2024-08-19T10:49:14.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=293356062}) 2024-08-19T10:49:14.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4156122823}) 2024-08-19T10:49:14.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=293356062}) 2024-08-19T10:49:14.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=23914568}) 2024-08-19T10:49:14.874 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.874 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3866335797}) 2024-08-19T10:49:14.874 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1253393070}) 2024-08-19T10:49:14.874 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.874 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3866335797}) 2024-08-19T10:49:14.875 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4207863984}) 2024-08-19T10:49:14.875 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.875 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3866335797}) 2024-08-19T10:49:14.875 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4083422614}) 2024-08-19T10:49:14.875 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.875 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3866335797}) 2024-08-19T10:49:14.875 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=966788127}) 2024-08-19T10:49:14.875 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.875 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3866335797}) 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1638483143}) 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3866335797}) 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2047830347}) 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3866335797}) 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3568143074}) 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3866335797}) 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2806017158}) 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2108166055}) 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1329212110}) 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.876 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2108166055}) 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2452903933}) 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2108166055}) 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=422098335}) 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2108166055}) 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1318038043}) 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2108166055}) 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2619581997}) 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2108166055}) 2024-08-19T10:49:14.877 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1996281089}) 2024-08-19T10:49:14.878 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.878 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2108166055}) 2024-08-19T10:49:14.878 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=953974949}) 2024-08-19T10:49:14.878 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.878 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2108166055}) 2024-08-19T10:49:14.878 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1091740198}) 2024-08-19T10:49:14.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1221602044}) 2024-08-19T10:49:14.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398738490}) 2024-08-19T10:49:14.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1221602044}) 2024-08-19T10:49:14.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3927811224}) 2024-08-19T10:49:14.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1221602044}) 2024-08-19T10:49:14.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2274848752}) 2024-08-19T10:49:14.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1221602044}) 2024-08-19T10:49:14.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.874+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=532931906}) 2024-08-19T10:49:14.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1221602044}) 2024-08-19T10:49:14.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=885852642}) 2024-08-19T10:49:14.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1221602044}) 2024-08-19T10:49:14.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=28483223}) 2024-08-19T10:49:14.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1221602044}) 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3306261570}) 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1221602044}) 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3575539421}) 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821792142}) 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2765576770}) 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821792142}) 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=89590978}) 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821792142}) 2024-08-19T10:49:14.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=149173020}) 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821792142}) 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3895752792}) 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821792142}) 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=404754096}) 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821792142}) 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.878+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=132616136}) 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821792142}) 2024-08-19T10:49:14.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2795553449}) 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821792142}) 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3843453689}) 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=324283320}) 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3784319940}) 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=324283320}) 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=603340423}) 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=324283320}) 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=771678029}) 2024-08-19T10:49:14.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=324283320}) 2024-08-19T10:49:14.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1501174572}) 2024-08-19T10:49:14.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=324283320}) 2024-08-19T10:49:14.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=996895598}) 2024-08-19T10:49:14.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=324283320}) 2024-08-19T10:49:14.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4161931341}) 2024-08-19T10:49:14.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=324283320}) 2024-08-19T10:49:14.892 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1811748007}) 2024-08-19T10:49:14.892 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.882+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.892 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=324283320}) 2024-08-19T10:49:14.892 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1031692806}) 2024-08-19T10:49:14.898 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.898 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268724973}) 2024-08-19T10:49:14.899 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1816994209}) 2024-08-19T10:49:14.899 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.899 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268724973}) 2024-08-19T10:49:14.899 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3101036514}) 2024-08-19T10:49:14.899 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.899 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268724973}) 2024-08-19T10:49:14.899 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=213535447}) 2024-08-19T10:49:14.899 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.900 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268724973}) 2024-08-19T10:49:14.900 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2861429251}) 2024-08-19T10:49:14.900 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.900 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268724973}) 2024-08-19T10:49:14.900 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1916112393}) 2024-08-19T10:49:14.900 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.900 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268724973}) 2024-08-19T10:49:14.900 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=998032205}) 2024-08-19T10:49:14.900 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.900 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268724973}) 2024-08-19T10:49:14.900 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4125615696}) 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268724973}) 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.886+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2515957883}) 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1165786303}) 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=540633589}) 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1165786303}) 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=428068195}) 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1165786303}) 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=805559675}) 2024-08-19T10:49:14.901 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.902 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1165786303}) 2024-08-19T10:49:14.902 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3634823236}) 2024-08-19T10:49:14.902 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.902 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1165786303}) 2024-08-19T10:49:14.902 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2529486166}) 2024-08-19T10:49:14.902 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.902 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1165786303}) 2024-08-19T10:49:14.902 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3323453251}) 2024-08-19T10:49:14.902 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.902 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1165786303}) 2024-08-19T10:49:14.902 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3936676892}) 2024-08-19T10:49:14.903 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.890+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.903 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1165786303}) 2024-08-19T10:49:14.903 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=257990815}) 2024-08-19T10:49:14.903 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.903 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259122331}) 2024-08-19T10:49:14.903 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=136887264}) 2024-08-19T10:49:14.903 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.903 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259122331}) 2024-08-19T10:49:14.903 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2936821547}) 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259122331}) 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1101184611}) 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.894+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259122331}) 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4195180333}) 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259122331}) 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3331134462}) 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259122331}) 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3469073423}) 2024-08-19T10:49:14.904 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.905 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259122331}) 2024-08-19T10:49:14.905 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1687999679}) 2024-08-19T10:49:14.905 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.905 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4259122331}) 2024-08-19T10:49:14.905 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1671798927}) 2024-08-19T10:49:14.909 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.909 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=263445489}) 2024-08-19T10:49:14.909 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=597200249}) 2024-08-19T10:49:14.909 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.909 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=263445489}) 2024-08-19T10:49:14.909 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.898+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2806804967}) 2024-08-19T10:49:14.909 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.909 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=263445489}) 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=343893269}) 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=263445489}) 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=903554439}) 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=263445489}) 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4165881481}) 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=263445489}) 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=123581404}) 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=263445489}) 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2033417042}) 2024-08-19T10:49:14.910 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.911 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=263445489}) 2024-08-19T10:49:14.911 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1245205677}) 2024-08-19T10:49:14.911 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.911 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024304852}) 2024-08-19T10:49:14.911 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3900177536}) 2024-08-19T10:49:14.911 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.911 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024304852}) 2024-08-19T10:49:14.911 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=880565652}) 2024-08-19T10:49:14.911 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.911 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024304852}) 2024-08-19T10:49:14.911 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3636157097}) 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024304852}) 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1393283034}) 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024304852}) 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1842961393}) 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.902+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024304852}) 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2134860018}) 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024304852}) 2024-08-19T10:49:14.912 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=904920690}) 2024-08-19T10:49:14.913 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:14.913 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024304852}) 2024-08-19T10:49:14.913 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:14.906+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=549916524}) 2024-08-19T10:49:14.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.970+0000 7ff99d29a640 10 monclient: get_monmap_and_config 2024-08-19T10:49:14.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.970+0000 7ff99d29a640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient: monmap: 2024-08-19T10:49:14.973 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.973 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.973 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:14.977059+0000 2024-08-19T10:49:14.973 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:14.977059+0000 2024-08-19T10:49:14.973 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:14.974 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:14.975 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 1 Processor -- start 2024-08-19T10:49:14.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 1 -- start start 2024-08-19T10:49:14.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient: init 2024-08-19T10:49:14.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient: _add_conns ranks=[5,10,3,4,0,9,7,6,2,1,8] 2024-08-19T10:49:14.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9980a58c0 0x7ff998072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient(hunting): picked mon.noname-h con 0x7ff9980a58c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:14.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9980a5ca0 0x7ff9980a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient(hunting): picked mon.noname-f con 0x7ff9980a5ca0 addr v2:172.21.15.190:3300/0 2024-08-19T10:49:14.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff998139690 0x7ff998139b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient(hunting): picked mon.noname-g con 0x7ff998139690 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:14.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff99806dfe0 con 0x7ff9980a5ca0 2024-08-19T10:49:14.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff99806f360 con 0x7ff998139690 2024-08-19T10:49:14.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff99806f080 con 0x7ff9980a58c0 2024-08-19T10:49:14.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff99d29a640 10 monclient(hunting): authenticate will time out at 1604.793759s 2024-08-19T10:49:14.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9980a58c0 0x7ff998072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9977fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff998139690 0x7ff998139b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9967fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9980a5ca0 0x7ff9980a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9980a58c0 0x7ff998072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:49714/0 (socket says 172.21.15.89:49714) 2024-08-19T10:49:14.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 1 -- 172.21.15.89:0/1052497550 learned_addr learned my addr 172.21.15.89:0/1052497550 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:14.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 10 monclient(hunting): get_auth_request con 0x7ff9980a58c0 auth_method 0 2024-08-19T10:49:14.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9967fc640 10 monclient(hunting): get_auth_request con 0x7ff9980a5ca0 auth_method 0 2024-08-19T10:49:14.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9967fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9967fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9977fe640 10 monclient(hunting): get_auth_request con 0x7ff998139690 auth_method 0 2024-08-19T10:49:14.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9967fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9967fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9967fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff9977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 10 monclient(hunting): handle_auth_done global_id 6953 payload 274 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 1 -- 172.21.15.89:0/1052497550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff998139690 msgr2=0x7ff998139b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 1 --2- 172.21.15.89:0/1052497550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff998139690 0x7ff998139b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 1 -- 172.21.15.89:0/1052497550 >> v2:172.21.15.190:3300/0 conn(0x7ff9980a5ca0 msgr2=0x7ff9980a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:14.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 1 --2- 172.21.15.89:0/1052497550 >> v2:172.21.15.190:3300/0 conn(0x7ff9980a5ca0 0x7ff9980a6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:49:14.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 1 monclient: found mon.noname-h 2024-08-19T10:49:14.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:14.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:49:14.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.974+0000 7ff996ffd640 1 -- 172.21.15.89:0/1052497550 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99806f640 con 0x7ff9980a58c0 2024-08-19T10:49:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff996ffd640 1 -- 172.21.15.89:0/1052497550 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9980a58c0 msgr2=0x7ff998072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff996ffd640 1 --2- 172.21.15.89:0/1052497550 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9980a58c0 0x7ff998072030 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff988009870 tx=0x7ff98801c420 comp rx=0 tx=0).ready entity=mon.5 client_cookie=686b4c1d16f88abc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 1 -- 172.21.15.89:0/1052497550 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9880b2070 con 0x7ff9980a58c0 2024-08-19T10:49:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:49:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 10 monclient: dump: 2024-08-19T10:49:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.989 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.990 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 10 monclient: _finish_auth 0 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 1 -- 172.21.15.89:0/1052497550 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9880ab7f0 con 0x7ff9980a58c0 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff99d29a640 5 monclient: authenticate success, global_id 6953 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff99d29a640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff99d29a640 4 set_mon_vals no callback set 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 1 -- 172.21.15.89:0/1052497550 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9880ab9e0 con 0x7ff9980a58c0 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:14.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.978+0000 7ff995ffb640 10 monclient: dump: 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:14.992 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient: shutdown 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 -- 172.21.15.89:0/1052497550 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9980a58c0 msgr2=0x7ff998072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 --2- 172.21.15.89:0/1052497550 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9980a58c0 0x7ff998072030 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff988009870 tx=0x7ff98801c420 comp rx=0 tx=0).stop 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 -- 172.21.15.89:0/1052497550 shutdown_connections 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 --2- 172.21.15.89:0/1052497550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff998139690 0x7ff998139b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 --2- 172.21.15.89:0/1052497550 >> v2:172.21.15.190:3300/0 conn(0x7ff9980a5ca0 0x7ff9980a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 --2- 172.21.15.89:0/1052497550 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff9980a58c0 0x7ff998072030 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 -- 172.21.15.89:0/1052497550 >> 172.21.15.89:0/1052497550 conn(0x7ff9980aa4b0 msgr2=0x7ff9980a69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:14.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 -- 172.21.15.89:0/1052497550 shutdown_connections 2024-08-19T10:49:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 -- 172.21.15.89:0/1052497550 wait complete. 2024-08-19T10:49:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient: build_initial_monmap 2024-08-19T10:49:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient: monmap: 2024-08-19T10:49:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:14.994 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 Processor -- start 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 -- start start 2024-08-19T10:49:14.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient: init 2024-08-19T10:49:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient: _add_conns ranks=[8,3,4,1,2,5,7,0,6] 2024-08-19T10:49:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9980a58c0 0x7ff9981d1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient(hunting): picked mon.noname-e con 0x7ff9980a58c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9980a5ca0 0x7ff998138670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient(hunting): picked mon.noname-g con 0x7ff9980a5ca0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff998139690 0x7ff998120500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient(hunting): picked mon.noname-c con 0x7ff998139690 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff99806e860 con 0x7ff9980a5ca0 2024-08-19T10:49:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff99806e6e0 con 0x7ff998139690 2024-08-19T10:49:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff998143100 con 0x7ff9980a58c0 2024-08-19T10:49:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff99d29a640 10 monclient(hunting): authenticate will time out at 1604.802334s 2024-08-19T10:49:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9980a58c0 0x7ff9981d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9980a58c0 0x7ff9981d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:37968/0 (socket says 172.21.15.89:37968) 2024-08-19T10:49:14.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9977fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff998139690 0x7ff998120500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 1 -- 172.21.15.89:0/3337971078 learned_addr learned my addr 172.21.15.89:0/3337971078 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 10 monclient(hunting): get_auth_request con 0x7ff9980a58c0 auth_method 0 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9977fe640 10 monclient(hunting): get_auth_request con 0x7ff998139690 auth_method 0 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9967fc640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9980a5ca0 0x7ff998138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9967fc640 10 monclient(hunting): get_auth_request con 0x7ff9980a5ca0 auth_method 0 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9967fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9967fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 10 monclient(hunting): handle_auth_done global_id 6830 payload 824 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 1 -- 172.21.15.89:0/3337971078 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff998139690 msgr2=0x7ff998120500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff998139690 0x7ff998120500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 1 -- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9980a5ca0 msgr2=0x7ff998138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9980a5ca0 0x7ff998138670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:49:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 1 monclient: found mon.noname-e 2024-08-19T10:49:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff996ffd640 1 -- 172.21.15.89:0/3337971078 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff998123d30 con 0x7ff9980a58c0 2024-08-19T10:49:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.982+0000 7ff9977fe640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff998139690 0x7ff998120500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:15.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff996ffd640 1 -- 172.21.15.89:0/3337971078 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9980a58c0 msgr2=0x7ff9981d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff996ffd640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9980a58c0 0x7ff9981d1090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff9880b0040 tx=0x7ff9980a75f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9880bd070 con 0x7ff9980a58c0 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 10 monclient: dump: 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.001 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.002 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:49:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:49:15.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff99d29a640 5 monclient: authenticate success, global_id 6830 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff99d29a640 10 monclient: _renew_subs 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9880abe10 con 0x7ff9980a58c0 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff99d29a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff99d29a640 1 -- 172.21.15.89:0/3337971078 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff998124010 con 0x7ff9980a58c0 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9880b2070 con 0x7ff9980a58c0 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff994ff9640 4 set_mon_vals no callback set 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 10 monclient: dump: 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.003 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.004 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff99d29a640 10 monclient: _renew_subs 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff99d29a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff99d29a640 1 -- 172.21.15.89:0/3337971078 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7ff998139410 con 0x7ff9980a58c0 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff996ffd640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9980a58c0 0x7ff9981d1090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff9880b0040 tx=0x7ff9980a75f0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7ff9880ab740 mgrmap(e 5) 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff99d29a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff9880ab740 con 0x7ff9980a58c0 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff99d29a640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998122810 0x7ff99802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff99d29a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff99d29a640 1 -- 172.21.15.89:0/3337971078 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff998123ae0 con 0x7ff998122810 2024-08-19T10:49:15.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff97f7fe640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff98403be00 0x7ff98403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9967fc640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998122810 0x7ff99802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9967fc640 10 monclient: get_auth_request con 0x7ff998122810 auth_method 0 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9967fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9967fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9977fe640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff98403be00 0x7ff98403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9977fe640 10 monclient: get_auth_request con 0x7ff98403be00 auth_method 0 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9967fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9967fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9967fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9967fc640 10 monclient(hunting): handle_auth_done global_id 64186 payload 824 2024-08-19T10:49:15.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.986+0000 7ff9967fc640 1 -- 172.21.15.89:0/3337971078 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9980a6080 con 0x7ff998122810 2024-08-19T10:49:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff9977fe640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff98403be00 0x7ff98403e2c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff990009fd0 tx=0x7ff99000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff9967fc640 1 -- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998122810 msgr2=0x7ff99802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff9967fc640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998122810 0x7ff99802d1d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0115d0 tx=0x7ff98c011a90 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff98c00fe70 con 0x7ff998122810 2024-08-19T10:49:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff97f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff97f7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:49:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff97f7fe640 10 monclient: dump: 2024-08-19T10:49:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.009 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.010 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff98c00fa00 con 0x7ff998122810 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff97f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff97f7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998122810 msgr2=0x7ff99802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff97f7fe640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998122810 0x7ff99802d1d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0115d0 tx=0x7ff98c011a90 comp rx=0 tx=0).stop 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998122810 msgr2=0x7ff99802d1d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.990+0000 7ff97f7fe640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998122810 0x7ff99802d1d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff99d29a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff99d29a640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998138bb0 0x7ff99802d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff99d29a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff99d29a640 1 -- 172.21.15.89:0/3337971078 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff998069f50 con 0x7ff998138bb0 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff9967fc640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998138bb0 0x7ff99802d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff9967fc640 10 monclient: get_auth_request con 0x7ff998138bb0 auth_method 0 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff9967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff9967fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff9967fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff9967fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff9967fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff9967fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff9967fc640 10 monclient(hunting): handle_auth_done global_id 64195 payload 824 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:14.998+0000 7ff9967fc640 1 -- 172.21.15.89:0/3337971078 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff998064590 con 0x7ff998138bb0 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff9967fc640 1 -- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998138bb0 msgr2=0x7ff99802d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff9967fc640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998138bb0 0x7ff99802d710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0115d0 tx=0x7ff98c0046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff98c003ca0 con 0x7ff998138bb0 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 10 monclient: dump: 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.013 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.014 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9266 (secure 0 0 0) 0x7ff98c003830 con 0x7ff998138bb0 2024-08-19T10:49:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998138bb0 msgr2=0x7ff99802d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998138bb0 0x7ff99802d710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0115d0 tx=0x7ff98c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998138bb0 msgr2=0x7ff99802d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:15.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff998138bb0 0x7ff99802d710 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0115d0 tx=0x7ff98c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 98, 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 21399, 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:15.015 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:15.016 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:15.017 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:15.018 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:15.018 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:15.018 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:15.018 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:15.018 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:15.018 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:15.018 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:15.019 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.019 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:15.019 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:15.020 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:15.020 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:15.020 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:15.020 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:15.020 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:15.020 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:15.020 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:15.020 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:15.020 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:15.020 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:15.021 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:15.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:15.023 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.026 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:15.026 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.027 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:15.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.030 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:15.031 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.033 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:15.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:15.033 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.033 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.033 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:15.033 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:15.033 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:15.033 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:15.033 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:15.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.034 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:15.034 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:15.034 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:15.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:15.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.034 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:15.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:15.034 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.035 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:15.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:15.035 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.035 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.035 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:15.035 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:15.036 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:15.036 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:15.036 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:15.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.036 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:15.036 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:15.036 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:15.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:15.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:15.037 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:15.040 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:15.041 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:15.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:15.041 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:15.041 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff99d29a640 1 -- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff98403be00 msgr2=0x7ff98403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff99d29a640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff98403be00 0x7ff98403e2c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff990009fd0 tx=0x7ff99000b040 comp rx=0 tx=0).stop 2024-08-19T10:49:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff99d29a640 10 monclient: shutdown 2024-08-19T10:49:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff99d29a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff99d29a640 1 -- 172.21.15.89:0/3337971078 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9980a58c0 msgr2=0x7ff9981d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff99d29a640 1 --2- 172.21.15.89:0/3337971078 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9980a58c0 0x7ff9981d1090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff9880b0040 tx=0x7ff9980a75f0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff9977fe640 1 -- 172.21.15.89:0/3337971078 reap_dead start 2024-08-19T10:49:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff97f7fe640 1 -- 172.21.15.89:0/3337971078 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (unknown 0 0 0) 0x7ff9880e8d80 con 0x7ff9980a58c0 2024-08-19T10:49:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff9977fe640 1 -- 172.21.15.89:0/3337971078 reap_dead start 2024-08-19T10:49:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff99d29a640 1 -- 172.21.15.89:0/3337971078 shutdown_connections 2024-08-19T10:49:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff99d29a640 1 -- 172.21.15.89:0/3337971078 >> 172.21.15.89:0/3337971078 conn(0x7ff9980aa4b0 msgr2=0x7ff9980a32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff99d29a640 1 -- 172.21.15.89:0/3337971078 shutdown_connections 2024-08-19T10:49:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.002+0000 7ff99d29a640 1 -- 172.21.15.89:0/3337971078 wait complete. 2024-08-19T10:49:15.044 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:49:15.044 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-08-19T10:49:15.044 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-08-19T10:49:15.044 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-08-19T10:49:15.044 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b store 2024-08-19T10:49:15.044 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b sync_force --yes-i-really-mean-it 2024-08-19T10:49:15.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3356025821}) 2024-08-19T10:49:15.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1674961436}) 2024-08-19T10:49:15.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3356025821}) 2024-08-19T10:49:15.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2090263997}) 2024-08-19T10:49:15.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3356025821}) 2024-08-19T10:49:15.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2925810992}) 2024-08-19T10:49:15.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3356025821}) 2024-08-19T10:49:15.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=439743472}) 2024-08-19T10:49:15.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3356025821}) 2024-08-19T10:49:15.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2532604061}) 2024-08-19T10:49:15.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3356025821}) 2024-08-19T10:49:15.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4230110624}) 2024-08-19T10:49:15.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3356025821}) 2024-08-19T10:49:15.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1069029523}) 2024-08-19T10:49:15.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3356025821}) 2024-08-19T10:49:15.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.042+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2833272308}) 2024-08-19T10:49:15.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=643942712}) 2024-08-19T10:49:15.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2197648275}) 2024-08-19T10:49:15.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=643942712}) 2024-08-19T10:49:15.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3634352844}) 2024-08-19T10:49:15.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=643942712}) 2024-08-19T10:49:15.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3225625888}) 2024-08-19T10:49:15.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=643942712}) 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=720770478}) 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=643942712}) 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1378892531}) 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=643942712}) 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3709325895}) 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=643942712}) 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=850104862}) 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=643942712}) 2024-08-19T10:49:15.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.050+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2234650881}) 2024-08-19T10:49:15.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422272423}) 2024-08-19T10:49:15.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1697202680}) 2024-08-19T10:49:15.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422272423}) 2024-08-19T10:49:15.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4247476167}) 2024-08-19T10:49:15.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422272423}) 2024-08-19T10:49:15.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2028494000}) 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422272423}) 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4198767531}) 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422272423}) 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4038978038}) 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422272423}) 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=331801136}) 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422272423}) 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=953253341}) 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422272423}) 2024-08-19T10:49:15.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.090+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2672305947}) 2024-08-19T10:49:15.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.162+0000 7ffa32163640 10 monclient: get_monmap_and_config 2024-08-19T10:49:15.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.162+0000 7ffa32163640 10 monclient: build_initial_monmap 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.162+0000 7ffa32163640 10 monclient: monmap: 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:15.168317+0000 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:15.168317+0000 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:15.164 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:15.165 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:15.165 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:15.165 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:15.165 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:15.165 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 1 Processor -- start 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 1 -- start start 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 10 monclient: init 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 10 monclient: _add_conns ranks=[2,3,8,4,10,7,0,5,1,6,9] 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c0a67d0 0x7ffa2c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 10 monclient(hunting): picked mon.noname-b con 0x7ffa2c0a67d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c0a6bb0 0x7ffa2c0a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 10 monclient(hunting): picked mon.noname-g con 0x7ffa2c0a6bb0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c139590 0x7ffa2c1399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 10 monclient(hunting): picked mon.noname-e con 0x7ffa2c139590 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2c0a8740 con 0x7ffa2c0a67d0 2024-08-19T10:49:15.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2c0a3380 con 0x7ffa2c0a6bb0 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ffa2c0a8250 con 0x7ffa2c139590 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa32163640 10 monclient(hunting): authenticate will time out at 1604.985165s 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c0a67d0 0x7ffa2c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c139590 0x7ffa2c1399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c139590 0x7ffa2c1399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:37976/0 (socket says 172.21.15.89:37976) 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c0a67d0 0x7ffa2c0a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:32794/0 (socket says 172.21.15.89:32794) 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 1 -- 172.21.15.89:0/2461993298 learned_addr learned my addr 172.21.15.89:0/2461993298 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 10 monclient(hunting): get_auth_request con 0x7ffa2c139590 auth_method 0 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:15.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2b7fe640 10 monclient(hunting): get_auth_request con 0x7ffa2c0a67d0 auth_method 0 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 10 monclient(hunting): handle_auth_done global_id 6839 payload 274 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2affd640 1 --2- 172.21.15.89:0/2461993298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c0a6bb0 0x7ffa2c0a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 1 -- 172.21.15.89:0/2461993298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c0a6bb0 msgr2=0x7ffa2c0a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 1 --2- 172.21.15.89:0/2461993298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c0a6bb0 0x7ffa2c0a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 1 -- 172.21.15.89:0/2461993298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c0a67d0 msgr2=0x7ffa2c0a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 1 --2- 172.21.15.89:0/2461993298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c0a67d0 0x7ffa2c0a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:49:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 1 monclient: found mon.noname-e 2024-08-19T10:49:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2bfff640 1 -- 172.21.15.89:0/2461993298 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c0a4cc0 con 0x7ffa2c139590 2024-08-19T10:49:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2affd640 10 monclient: get_auth_request con 0x7ffa2c0a6bb0 auth_method 0 2024-08-19T10:49:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2b7fe640 1 --2- 172.21.15.89:0/2461993298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c0a67d0 0x7ffa2c0a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:15.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.166+0000 7ffa2affd640 1 --2- 172.21.15.89:0/2461993298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c0a6bb0 0x7ffa2c0a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:15.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2bfff640 1 -- 172.21.15.89:0/2461993298 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c139590 msgr2=0x7ffa2c1399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2bfff640 1 --2- 172.21.15.89:0/2461993298 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c139590 0x7ffa2c1399f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009eb0 tx=0x7ffa2401c980 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6fbec3306b07f863 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 1 -- 172.21.15.89:0/2461993298 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa240090d0 con 0x7ffa2c139590 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 10 monclient: dump: 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.177 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 5 monclient: authenticate success, global_id 6839 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:15.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 1 -- 172.21.15.89:0/2461993298 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa240a59d0 con 0x7ffa2c139590 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 1 -- 172.21.15.89:0/2461993298 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa240ad3f0 con 0x7ffa2c139590 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 4 set_mon_vals no callback set 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa2a7fc640 10 monclient: dump: 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.179 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient: shutdown 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 -- 172.21.15.89:0/2461993298 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c139590 msgr2=0x7ffa2c1399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 --2- 172.21.15.89:0/2461993298 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c139590 0x7ffa2c1399f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009eb0 tx=0x7ffa2401c980 comp rx=0 tx=0).stop 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 -- 172.21.15.89:0/2461993298 shutdown_connections 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 --2- 172.21.15.89:0/2461993298 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c139590 0x7ffa2c1399f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 --2- 172.21.15.89:0/2461993298 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c0a6bb0 0x7ffa2c0a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 --2- 172.21.15.89:0/2461993298 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c0a67d0 0x7ffa2c0a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 -- 172.21.15.89:0/2461993298 >> 172.21.15.89:0/2461993298 conn(0x7ffa2c069ae0 msgr2=0x7ffa2c0abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 -- 172.21.15.89:0/2461993298 shutdown_connections 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 -- 172.21.15.89:0/2461993298 wait complete. 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient: build_initial_monmap 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient: monmap: 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:15.181 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 Processor -- start 2024-08-19T10:49:15.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 -- start start 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient: init 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient: _add_conns ranks=[8,6,3,0,7,2,1,5,4] 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c0a67d0 0x7ffa2c1d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient(hunting): picked mon.noname-e con 0x7ffa2c0a67d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffa2c0a6bb0 0x7ffa2c1d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient(hunting): picked mon.noname-d con 0x7ffa2c0a6bb0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c139590 0x7ffa2c09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient(hunting): picked mon.noname-g con 0x7ffa2c139590 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2c143390 con 0x7ffa2c139590 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ffa2c143210 con 0x7ffa2c0a6bb0 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ffa2c143510 con 0x7ffa2c0a67d0 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.170+0000 7ffa32163640 10 monclient(hunting): authenticate will time out at 1604.991922s 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c0a67d0 0x7ffa2c1d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffa2c0a6bb0 0x7ffa2c1d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c0a67d0 0x7ffa2c1d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:37984/0 (socket says 172.21.15.89:37984) 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 1 -- 172.21.15.89:0/1430421274 learned_addr learned my addr 172.21.15.89:0/1430421274 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 10 monclient(hunting): get_auth_request con 0x7ffa2c0a67d0 auth_method 0 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): get_auth_request con 0x7ffa2c0a6bb0 auth_method 0 2024-08-19T10:49:15.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 10 monclient(hunting): handle_auth_done global_id 6848 payload 824 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 1 -- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffa2c0a6bb0 msgr2=0x7ffa2c1d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ffa2c0a6bb0 0x7ffa2c1d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 1 -- 172.21.15.89:0/1430421274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c139590 msgr2=0x7ffa2c09c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c139590 0x7ffa2c09c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 1 monclient: found mon.noname-e 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 1 -- 172.21.15.89:0/1430421274 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c0a2830 con 0x7ffa2c0a67d0 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 1 -- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c0a67d0 msgr2=0x7ffa2c1d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2b7fe640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c0a67d0 0x7ffa2c1d16f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00ec10 tx=0x7ffa1c00c520 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa1c017070 con 0x7ffa2c0a67d0 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 10 monclient: dump: 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.186 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 10 monclient: _finish_auth 0 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 10 monclient: _check_auth_tickets 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c00cdc0 con 0x7ffa2c0a67d0 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa32163640 5 monclient: authenticate success, global_id 6848 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa1c013d50 con 0x7ffa2c0a67d0 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa29ffb640 4 set_mon_vals no callback set 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 10 monclient: dump: 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.188 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa32163640 10 monclient: _renew_subs 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa32163640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa32163640 1 -- 172.21.15.89:0/1430421274 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2c09faa0 con 0x7ffa2c0a67d0 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa32163640 10 monclient: _renew_subs 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa32163640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa32163640 1 -- 172.21.15.89:0/1430421274 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2c09eb40 con 0x7ffa2c0a67d0 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa32163640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa32163640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09cf20 0x7ffa2c09d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa32163640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa32163640 1 -- 172.21.15.89:0/1430421274 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2c0a0220 con 0x7ffa2c09cf20 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09cf20 0x7ffa2c09d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient: get_auth_request con 0x7ffa2c09cf20 auth_method 0 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:15.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ffa1c01a070 con 0x7ffa2c0a67d0 2024-08-19T10:49:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 10 monclient(hunting): handle_auth_done global_id 54533 payload 824 2024-08-19T10:49:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa2affd640 1 -- 172.21.15.89:0/1430421274 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa2c0a6f90 con 0x7ffa2c09cf20 2024-08-19T10:49:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.174+0000 7ffa23fff640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffa0c03be00 0x7ffa0c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7ffa1c00f040 con 0x7ffa2c0a67d0 2024-08-19T10:49:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa2bfff640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffa0c03be00 0x7ffa0c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa2bfff640 10 monclient: get_auth_request con 0x7ffa0c03be00 auth_method 0 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa2affd640 1 -- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09cf20 msgr2=0x7ffa2c09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa2affd640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09cf20 0x7ffa2c09d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c09e650 tx=0x7ffa1400fc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa1400dc00 con 0x7ffa2c09cf20 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 10 monclient: dump: 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.194 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ffa1400dc00 con 0x7ffa2c09cf20 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:15.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09cf20 msgr2=0x7ffa2c09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09cf20 0x7ffa2c09d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c09e650 tx=0x7ffa1400fc70 comp rx=0 tx=0).stop 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09cf20 msgr2=0x7ffa2c09d300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa23fff640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09cf20 0x7ffa2c09d300 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c09e650 tx=0x7ffa1400fc70 comp rx=0 tx=0).stop 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.178+0000 7ffa2bfff640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffa0c03be00 0x7ffa0c03e2c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009e80 tx=0x7ffa24004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa32163640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa32163640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09d840 0x7ffa2c13a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa32163640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa32163640 1 -- 172.21.15.89:0/1430421274 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2c064590 con 0x7ffa2c09d840 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09d840 0x7ffa2c13a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 10 monclient: get_auth_request con 0x7ffa2c09d840 auth_method 0 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:15.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 10 monclient(hunting): handle_auth_done global_id 54542 payload 824 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 1 -- 172.21.15.89:0/1430421274 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7ffa2c06ca20 con 0x7ffa2c09d840 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 1 -- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09d840 msgr2=0x7ffa2c13a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa2affd640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09d840 0x7ffa2c13a9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c09e650 tx=0x7ffa140046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffa1400de80 con 0x7ffa2c09d840 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa23fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa23fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.186+0000 7ffa23fff640 10 monclient: dump: 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.197 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7ffa1400de80 con 0x7ffa2c09d840 2024-08-19T10:49:15.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa23fff640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:49:15.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa23fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:15.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09d840 msgr2=0x7ffa2c13a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa23fff640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09d840 0x7ffa2c13a9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c09e650 tx=0x7ffa140046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa23fff640 1 -- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09d840 msgr2=0x7ffa2c13a9c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:15.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa23fff640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffa2c09d840 0x7ffa2c13a9c0 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c09e650 tx=0x7ffa140046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.199 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:15.199 INFO:teuthology.orchestra.run.smithi089.stdout: "ret": 0, 2024-08-19T10:49:15.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-08-19T10:49:15.199 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2720804693}) 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1124474684}) 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2720804693}) 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1247129053}) 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2720804693}) 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=925830143}) 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2720804693}) 2024-08-19T10:49:15.201 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1254117684}) 2024-08-19T10:49:15.202 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.202 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2720804693}) 2024-08-19T10:49:15.202 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3351435142}) 2024-08-19T10:49:15.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa32163640 1 -- 172.21.15.89:0/1430421274 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffa0c03be00 msgr2=0x7ffa0c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa32163640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffa0c03be00 0x7ffa0c03e2c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009e80 tx=0x7ffa24004ac0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa32163640 10 monclient: shutdown 2024-08-19T10:49:15.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa32163640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:15.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa32163640 1 -- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c0a67d0 msgr2=0x7ffa2c1d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa32163640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c0a67d0 0x7ffa2c1d16f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00ec10 tx=0x7ffa1c00c520 comp rx=0 tx=0).stop 2024-08-19T10:49:15.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa2bfff640 1 -- 172.21.15.89:0/1430421274 reap_dead start 2024-08-19T10:49:15.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2720804693}) 2024-08-19T10:49:15.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2237719536}) 2024-08-19T10:49:15.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2720804693}) 2024-08-19T10:49:15.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa32163640 1 -- 172.21.15.89:0/1430421274 shutdown_connections 2024-08-19T10:49:15.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3902805907}) 2024-08-19T10:49:15.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa32163640 1 --2- 172.21.15.89:0/1430421274 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffa2c0a67d0 0x7ffa2c1d16f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa32163640 1 -- 172.21.15.89:0/1430421274 >> 172.21.15.89:0/1430421274 conn(0x7ffa2c069ae0 msgr2=0x7ffa2c0ab750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:15.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2720804693}) 2024-08-19T10:49:15.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.190+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3052995228}) 2024-08-19T10:49:15.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa32163640 1 -- 172.21.15.89:0/1430421274 shutdown_connections 2024-08-19T10:49:15.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.190+0000 7ffa32163640 1 -- 172.21.15.89:0/1430421274 wait complete. 2024-08-19T10:49:15.206 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-08-19T10:49:15.206 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-08-19T10:49:15.206 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:49:15.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82577183}) 2024-08-19T10:49:15.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3434881215}) 2024-08-19T10:49:15.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82577183}) 2024-08-19T10:49:15.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3767359934}) 2024-08-19T10:49:15.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82577183}) 2024-08-19T10:49:15.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2730034954}) 2024-08-19T10:49:15.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82577183}) 2024-08-19T10:49:15.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3267576939}) 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82577183}) 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2397307636}) 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82577183}) 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1705304325}) 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82577183}) 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=999231456}) 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82577183}) 2024-08-19T10:49:15.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:49:15.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3845596212}) 2024-08-19T10:49:15.270 INFO:tasks.ceph.mon.b:Stopped 2024-08-19T10:49:15.271 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-08-19T10:49:15.271 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:49:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient: get_monmap_and_config 2024-08-19T10:49:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient: build_initial_monmap 2024-08-19T10:49:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient: monmap: 2024-08-19T10:49:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:15.434016+0000 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:15.434016+0000 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:15.431 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 1 Processor -- start 2024-08-19T10:49:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 1 -- start start 2024-08-19T10:49:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient: init 2024-08-19T10:49:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient: _add_conns ranks=[9,1,4,8,2,5,7,0,3,6,10] 2024-08-19T10:49:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f38c406c930 0x7f38c406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient(hunting): picked mon.noname-a con 0x7f38c406c930 addr v2:172.21.15.89:3300/0 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f38c409d340 addr v1:172.21.15.190:6789/0 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38c406d250 0x7f38c4137c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient(hunting): picked mon.noname-c con 0x7f38c406d250 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38c409fbf0 con 0x7f38c409d340 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f38c40a54f0 con 0x7f38c406c930 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f38c40a3e00 con 0x7f38c406d250 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c941a640 10 monclient(hunting): authenticate will time out at 1605.250753s 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38c406d250 0x7f38c4137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c2ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f38c406c930 0x7f38c406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38c406d250 0x7f38c4137c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:48538/0 (socket says 172.21.15.89:48538) 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 1 -- 172.21.15.89:0/2013813825 learned_addr learned my addr 172.21.15.89:0/2013813825 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 10 monclient(hunting): get_auth_request con 0x7f38c406d250 auth_method 0 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c2ffd640 10 monclient(hunting): get_auth_request con 0x7f38c406c930 auth_method 0 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 10 monclient(hunting): handle_auth_done global_id 44815 payload 274 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 1 -- 172.21.15.89:0/2013813825 >> v2:172.21.15.89:3300/0 conn(0x7f38c406c930 msgr2=0x7f38c406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 1 --2- 172.21.15.89:0/2013813825 >> v2:172.21.15.89:3300/0 conn(0x7f38c406c930 0x7f38c406cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 1 -- 172.21.15.89:0/2013813825 >> v1:172.21.15.190:6789/0 conn(0x7f38c409d340 legacy=0x7f38c4137500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 1 monclient: found mon.noname-c 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c37fe640 1 -- 172.21.15.89:0/2013813825 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c40a3b30 con 0x7f38c406d250 2024-08-19T10:49:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.430+0000 7f38c2ffd640 1 --2- 172.21.15.89:0/2013813825 >> v2:172.21.15.89:3300/0 conn(0x7f38c406c930 0x7f38c406cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:15.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c37fe640 1 -- 172.21.15.89:0/2013813825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38c406d250 msgr2=0x7f38c4137c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:15.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c37fe640 1 --2- 172.21.15.89:0/2013813825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38c406d250 0x7f38c4137c10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009870 tx=0x7f38bc01c420 comp rx=0 tx=0).ready entity=mon.4 client_cookie=fd187b6a303f0209 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 1 -- 172.21.15.89:0/2013813825 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38bc0b0070 con 0x7f38c406d250 2024-08-19T10:49:15.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:49:15.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 10 monclient: dump: 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 1 -- 172.21.15.89:0/2013813825 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38bc0aa810 con 0x7f38c406d250 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 1 -- 172.21.15.89:0/2013813825 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38bc0aae30 con 0x7f38c406d250 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 5 monclient: authenticate success, global_id 44815 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 4 set_mon_vals no callback set 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c1ffb640 10 monclient: dump: 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient: shutdown 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 -- 172.21.15.89:0/2013813825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38c406d250 msgr2=0x7f38c4137c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 --2- 172.21.15.89:0/2013813825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38c406d250 0x7f38c4137c10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009870 tx=0x7f38bc01c420 comp rx=0 tx=0).stop 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 -- 172.21.15.89:0/2013813825 shutdown_connections 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 --2- 172.21.15.89:0/2013813825 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f38c406d250 0x7f38c4137c10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 --2- 172.21.15.89:0/2013813825 >> v2:172.21.15.89:3300/0 conn(0x7f38c406c930 0x7f38c406cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 -- 172.21.15.89:0/2013813825 >> 172.21.15.89:0/2013813825 conn(0x7f38c40a8f60 msgr2=0x7f38c40a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 -- 172.21.15.89:0/2013813825 shutdown_connections 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 -- 172.21.15.89:0/2013813825 wait complete. 2024-08-19T10:49:15.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient: build_initial_monmap 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient: monmap: 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:15.443 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:15.444 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:15.444 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:15.444 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 Processor -- start 2024-08-19T10:49:15.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 -- start start 2024-08-19T10:49:15.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient: init 2024-08-19T10:49:15.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient: _add_conns ranks=[1,0,2,3,4,6,7,5,8] 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38c406c930 0x7f38c4137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient(hunting): picked mon.noname-f con 0x7f38c406c930 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38c406d250 0x7f38c411fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient(hunting): picked mon.noname-a con 0x7f38c406d250 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38c409d340 0x7f38c4120250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient(hunting): picked mon.noname-b con 0x7f38c409d340 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f38c4138150 con 0x7f38c406d250 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f38c4074420 con 0x7f38c406c930 2024-08-19T10:49:15.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f38c41382d0 con 0x7f38c409d340 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c941a640 10 monclient(hunting): authenticate will time out at 1605.259105s 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c37fe640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38c409d340 msgr2=0x7f38c4120250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c37fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38c409d340 0x7f38c4120250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38c406d250 0x7f38c411fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38c406d250 0x7f38c411fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60688/0 (socket says 172.21.15.89:60688) 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 1 -- 172.21.15.89:0/1649865988 learned_addr learned my addr 172.21.15.89:0/1649865988 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 10 monclient(hunting): get_auth_request con 0x7f38c406d250 auth_method 0 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:15.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 10 monclient(hunting): handle_auth_done global_id 6966 payload 824 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 1 -- 172.21.15.89:0/1649865988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38c409d340 msgr2=0x7f38c4120250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38c409d340 0x7f38c4120250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 1 -- 172.21.15.89:0/1649865988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38c406c930 msgr2=0x7f38c4137130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38c406c930 0x7f38c4137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 10 monclient: _finish_hunting 0 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 1 monclient: found mon.noname-a 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.438+0000 7f38c27fc640 1 -- 172.21.15.89:0/1649865988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c40a82b0 con 0x7f38c406d250 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c27fc640 1 -- 172.21.15.89:0/1649865988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38c406d250 msgr2=0x7f38c411fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:15.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c27fc640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38c406d250 0x7f38c411fd10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f38b400ba80 tx=0x7f38b400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 1 -- 172.21.15.89:0/1649865988 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38b400cc10 con 0x7f38c406d250 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 10 monclient: dump: 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.451 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 1 -- 172.21.15.89:0/1649865988 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b400cdf0 con 0x7f38c406d250 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c941a640 5 monclient: authenticate success, global_id 6966 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c941a640 10 monclient: _renew_subs 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c941a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:15.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c941a640 1 -- 172.21.15.89:0/1649865988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c40a63f0 con 0x7f38c406d250 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 1 -- 172.21.15.89:0/1649865988 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f38b4014d20 con 0x7f38c406d250 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 10 monclient: dump: 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:15.453 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c941a640 10 monclient: _renew_subs 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c941a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c17fa640 4 set_mon_vals no callback set 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c941a640 1 -- 172.21.15.89:0/1649865988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c41236c0 con 0x7f38c406d250 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c941a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c941a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:15.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c941a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c941a640 1 -- 172.21.15.89:0/1649865988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c406cd10 con 0x7f38c406d250 2024-08-19T10:49:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 1 -- 172.21.15.89:0/1649865988 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f38b401b020 con 0x7f38c406d250 2024-08-19T10:49:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38a403be00 0x7f38a403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38bb7fe640 1 -- 172.21.15.89:0/1649865988 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f38b404f3e0 con 0x7f38c406d250 2024-08-19T10:49:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c2ffd640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38a403be00 0x7f38a403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.442+0000 7f38c2ffd640 10 monclient: get_auth_request con 0x7f38a403be00 auth_method 0 2024-08-19T10:49:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.446+0000 7f38c2ffd640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38a403be00 0x7f38a403e2c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009a80 tx=0x7f38ac009480 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.446+0000 7f38bb7fe640 1 -- 172.21.15.89:0/1649865988 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f38c40a63f0 con 0x7f38c406d250 2024-08-19T10:49:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.446+0000 7f38bb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:15.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.446+0000 7f38bb7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.666+0000 7f38c941a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:49:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.666+0000 7f38c941a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.666+0000 7f38c941a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.666+0000 7f38c941a640 1 -- 172.21.15.89:0/1649865988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f38c4064590 con 0x7f38c406d250 2024-08-19T10:49:15.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.666+0000 7f38bb7fe640 1 -- 172.21.15.89:0/1649865988 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3192 (secure 0 0 0) 0x7f38c40a82b0 con 0x7f38c406d250 2024-08-19T10:49:15.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.666+0000 7f38bb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:15.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.666+0000 7f38bb7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:15.668 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:49:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 -- 172.21.15.89:0/1649865988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38a403be00 msgr2=0x7f38a403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38a403be00 0x7f38a403e2c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009a80 tx=0x7f38ac009480 comp rx=0 tx=0).stop 2024-08-19T10:49:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 10 monclient: shutdown 2024-08-19T10:49:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 -- 172.21.15.89:0/1649865988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38c406d250 msgr2=0x7f38c411fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38c406d250 0x7f38c411fd10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f38b400ba80 tx=0x7f38b400bf40 comp rx=0 tx=0).stop 2024-08-19T10:49:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 -- 172.21.15.89:0/1649865988 shutdown_connections 2024-08-19T10:49:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f38a403be00 0x7f38a403e2c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f38c409d340 0x7f38c4120250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38c406d250 0x7f38c411fd10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 --2- 172.21.15.89:0/1649865988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38c406c930 0x7f38c4137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:15.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 -- 172.21.15.89:0/1649865988 >> 172.21.15.89:0/1649865988 conn(0x7f38c40a8f60 msgr2=0x7f38c40a6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:15.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 -- 172.21.15.89:0/1649865988 shutdown_connections 2024-08-19T10:49:15.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:15.670+0000 7f38c941a640 1 -- 172.21.15.89:0/1649865988 wait complete. 2024-08-19T10:49:18.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:49:18.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient: get_monmap_and_config 2024-08-19T10:49:18.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient: build_initial_monmap 2024-08-19T10:49:18.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient: monmap: 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:18.822100+0000 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:18.822100+0000 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:18.818 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:18.819 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:18.819 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:18.819 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:18.819 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:18.819 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:18.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 1 Processor -- start 2024-08-19T10:49:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 1 -- start start 2024-08-19T10:49:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient: init 2024-08-19T10:49:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient: _add_conns ranks=[7,1,2,4,3,10,5,9,6,8,0] 2024-08-19T10:49:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42140a5270 0x7f42140a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient(hunting): picked mon.noname-i con 0x7f42140a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f42140a6d30 addr v1:172.21.15.190:6789/0 2024-08-19T10:49:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42140a7110 0x7f4214139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient(hunting): picked mon.noname-b con 0x7f42140a7110 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:18.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42140a8710 con 0x7f42140a6d30 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f42140a5a40 con 0x7f42140a7110 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f42140a81d0 con 0x7f42140a5270 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f421ba2a640 10 monclient(hunting): authenticate will time out at 1608.638670s 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f4219fc7640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42140a7110 msgr2=0x7f4214139cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f4219fc7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42140a7110 0x7f4214139cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f4218fc5640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f42140a6d30 0x7f42141395b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.89:38300/0 (socket says 172.21.15.89:38300) 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f4218fc5640 1 -- 172.21.15.89:0/635225128 learned_addr learned my addr 172.21.15.89:0/635225128 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42140a5270 0x7f42140a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:18.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 10 monclient(hunting): get_auth_request con 0x7f42140a5270 auth_method 0 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 10 monclient(hunting): handle_auth_done global_id 64204 payload 274 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 1 -- 172.21.15.89:0/635225128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42140a7110 msgr2=0x7f4214139cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 1 --2- 172.21.15.89:0/635225128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42140a7110 0x7f4214139cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 1 -- 172.21.15.89:0/635225128 >> v1:172.21.15.190:6789/0 conn(0x7f42140a6d30 legacy=0x7f42141395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 10 monclient: _finish_hunting 0 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 1 monclient: found mon.noname-i 2024-08-19T10:49:18.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:18.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:49:18.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.818+0000 7f42197c6640 1 -- 172.21.15.89:0/635225128 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42140a4790 con 0x7f42140a5270 2024-08-19T10:49:18.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 1 -- 172.21.15.89:0/635225128 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42140a5270 msgr2=0x7f42140a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:18.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 1 --2- 172.21.15.89:0/635225128 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42140a5270 0x7f42140a67f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f42100097f0 tx=0x7f421001c1b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=128f7f1acb85e37f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:18.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 1 -- 172.21.15.89:0/635225128 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42100b0070 con 0x7f42140a5270 2024-08-19T10:49:18.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:18.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:49:18.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 10 monclient: dump: 2024-08-19T10:49:18.826 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:18.827 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 10 monclient: _finish_auth 0 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 1 -- 172.21.15.89:0/635225128 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42100aa810 con 0x7f42140a5270 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 1 -- 172.21.15.89:0/635225128 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42100a93e0 con 0x7f42140a5270 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f420bfff640 10 monclient: dump: 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:18.828 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:18.829 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 5 monclient: authenticate success, global_id 64204 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 4 set_mon_vals no callback set 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient: shutdown 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 -- 172.21.15.89:0/635225128 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42140a5270 msgr2=0x7f42140a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 --2- 172.21.15.89:0/635225128 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42140a5270 0x7f42140a67f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f42100097f0 tx=0x7f421001c1b0 comp rx=0 tx=0).stop 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 -- 172.21.15.89:0/635225128 shutdown_connections 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 --2- 172.21.15.89:0/635225128 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f42140a7110 0x7f4214139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 --2- 172.21.15.89:0/635225128 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f42140a5270 0x7f42140a67f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 -- 172.21.15.89:0/635225128 >> 172.21.15.89:0/635225128 conn(0x7f4214069a80 msgr2=0x7f42140abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 -- 172.21.15.89:0/635225128 shutdown_connections 2024-08-19T10:49:18.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 -- 172.21.15.89:0/635225128 wait complete. 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient: build_initial_monmap 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient: monmap: 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:18.831 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:18.832 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 Processor -- start 2024-08-19T10:49:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 -- start start 2024-08-19T10:49:18.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient: init 2024-08-19T10:49:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient: _add_conns ranks=[0,6,8,3,4,1,7,5,2] 2024-08-19T10:49:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42140a5270 0x7f4214138f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient(hunting): picked mon.noname-a con 0x7f42140a5270 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f42140a6d30 0x7f421413e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient(hunting): picked mon.noname-d con 0x7f42140a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42140a7110 0x7f421413e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient(hunting): picked mon.noname-e con 0x7f42140a7110 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f42140a57d0 con 0x7f42140a5270 2024-08-19T10:49:18.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f42140a5650 con 0x7f42140a6d30 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f421413a200 con 0x7f42140a7110 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f421ba2a640 10 monclient(hunting): authenticate will time out at 1608.647226s 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42140a5270 0x7f4214138f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4219fc7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42140a7110 0x7f421413e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4218fc5640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f42140a6d30 0x7f421413e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42140a5270 0x7f4214138f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60694/0 (socket says 172.21.15.89:60694) 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 1 -- 172.21.15.89:0/2094203062 learned_addr learned my addr 172.21.15.89:0/2094203062 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4219fc7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42140a7110 0x7f421413e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:37992/0 (socket says 172.21.15.89:37992) 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4219fc7640 10 monclient(hunting): get_auth_request con 0x7f42140a7110 auth_method 0 2024-08-19T10:49:18.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4219fc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4219fc7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4219fc7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 10 monclient(hunting): get_auth_request con 0x7f42140a5270 auth_method 0 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4218fc5640 10 monclient(hunting): get_auth_request con 0x7f42140a6d30 auth_method 0 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4218fc5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4218fc5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4218fc5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4219fc7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:18.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4219fc7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4219fc7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4218fc5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4218fc5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4218fc5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 10 monclient(hunting): handle_auth_done global_id 6975 payload 824 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 1 -- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42140a7110 msgr2=0x7f421413e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42140a7110 0x7f421413e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 1 -- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f42140a6d30 msgr2=0x7f421413e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f42140a6d30 0x7f421413e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:18.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 10 monclient: _finish_hunting 0 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 1 monclient: found mon.noname-a 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f42197c6640 1 -- 172.21.15.89:0/2094203062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4214123a40 con 0x7f42140a5270 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.826+0000 7f4219fc7640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42140a7110 0x7f421413e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f42197c6640 1 -- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42140a5270 msgr2=0x7f4214138f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f42197c6640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42140a5270 0x7f4214138f30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4210009710 tx=0x7f4210018f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 1 -- 172.21.15.89:0/2094203062 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42100b0040 con 0x7f42140a5270 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 10 monclient: dump: 2024-08-19T10:49:18.840 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:18.841 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 10 monclient: _finish_auth 0 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 1 -- 172.21.15.89:0/2094203062 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42100044b0 con 0x7f42140a5270 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 1 -- 172.21.15.89:0/2094203062 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4210003a90 con 0x7f42140a5270 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 10 monclient: dump: 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:18.842 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:18.843 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f420b7fe640 4 set_mon_vals no callback set 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f421ba2a640 5 monclient: authenticate success, global_id 6975 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f421ba2a640 10 monclient: _renew_subs 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f421ba2a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f421ba2a640 1 -- 172.21.15.89:0/2094203062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42140ab7e0 con 0x7f42140a5270 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f421ba2a640 10 monclient: _renew_subs 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f421ba2a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f421ba2a640 1 -- 172.21.15.89:0/2094203062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4214122c20 con 0x7f42140a5270 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f421ba2a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f421ba2a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f421ba2a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:18.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f421ba2a640 1 -- 172.21.15.89:0/2094203062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42140a67f0 con 0x7f42140a5270 2024-08-19T10:49:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.830+0000 7f4209ffb640 1 -- 172.21.15.89:0/2094203062 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4210003620 con 0x7f42140a5270 2024-08-19T10:49:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.834+0000 7f4209ffb640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f41f803be00 0x7f41f803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.834+0000 7f4209ffb640 1 -- 172.21.15.89:0/2094203062 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f42100e9450 con 0x7f42140a5270 2024-08-19T10:49:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.834+0000 7f4218fc5640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f41f803be00 0x7f41f803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.834+0000 7f4218fc5640 10 monclient: get_auth_request con 0x7f41f803be00 auth_method 0 2024-08-19T10:49:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.834+0000 7f4218fc5640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f41f803be00 0x7f41f803e2c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4204007ca0 tx=0x7f4204007bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.838+0000 7f4209ffb640 1 -- 172.21.15.89:0/2094203062 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f42140a8710 con 0x7f42140a5270 2024-08-19T10:49:18.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.838+0000 7f4209ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:18.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:18.838+0000 7f4209ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:19.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:49:19.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:19.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:19.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 -- 172.21.15.89:0/2094203062 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4214064590 con 0x7f42140a5270 2024-08-19T10:49:19.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f4209ffb640 1 -- 172.21.15.89:0/2094203062 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7f42100aa7e0 con 0x7f42140a5270 2024-08-19T10:49:19.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f4209ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:19.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f4209ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:19.065 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:49:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 -- 172.21.15.89:0/2094203062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f41f803be00 msgr2=0x7f41f803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f41f803be00 0x7f41f803e2c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4204007ca0 tx=0x7f4204007bf0 comp rx=0 tx=0).stop 2024-08-19T10:49:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 10 monclient: shutdown 2024-08-19T10:49:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 -- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42140a5270 msgr2=0x7f4214138f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42140a5270 0x7f4214138f30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4210009710 tx=0x7f4210018f50 comp rx=0 tx=0).stop 2024-08-19T10:49:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 -- 172.21.15.89:0/2094203062 shutdown_connections 2024-08-19T10:49:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f41f803be00 0x7f41f803e2c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f42140a7110 0x7f421413e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f42140a6d30 0x7f421413e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:19.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 --2- 172.21.15.89:0/2094203062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42140a5270 0x7f4214138f30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:19.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 -- 172.21.15.89:0/2094203062 >> 172.21.15.89:0/2094203062 conn(0x7f4214069a80 msgr2=0x7f42140aa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:19.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 -- 172.21.15.89:0/2094203062 shutdown_connections 2024-08-19T10:49:19.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:19.066+0000 7f421ba2a640 1 -- 172.21.15.89:0/2094203062 wait complete. 2024-08-19T10:49:22.084 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:49:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient: get_monmap_and_config 2024-08-19T10:49:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient: build_initial_monmap 2024-08-19T10:49:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient: monmap: 2024-08-19T10:49:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:22.214851+0000 2024-08-19T10:49:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:22.214851+0000 2024-08-19T10:49:22.210 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:22.211 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 1 Processor -- start 2024-08-19T10:49:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 1 -- start start 2024-08-19T10:49:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient: init 2024-08-19T10:49:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient: _add_conns ranks=[7,5,1,4,10,0,3,6,2,8,9] 2024-08-19T10:49:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2adc0a5270 0x7f2adc0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:22.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient(hunting): picked mon.noname-i con 0x7f2adc0a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2adc0a6d30 0x7f2adc0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient(hunting): picked mon.noname-h con 0x7f2adc0a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2adc1395b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2adc0a8710 con 0x7f2adc1395b0 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2adc0a7650 con 0x7f2adc0a6d30 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2adc0a5a40 con 0x7f2adc0a5270 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2ae2a98640 10 monclient(hunting): authenticate will time out at 1612.031606s 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.210+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2adc0a6d30 0x7f2adc0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2ae1035640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f2adc1395b0 0x7f2adc139a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.89:49586/0 (socket says 172.21.15.89:49586) 2024-08-19T10:49:22.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2adc0a6d30 0x7f2adc0a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:43870/0 (socket says 172.21.15.89:43870) 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2ae1035640 1 -- 172.21.15.89:0/2666785985 learned_addr learned my addr 172.21.15.89:0/2666785985 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2ae0834640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2adc0a5270 0x7f2adc0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 10 monclient(hunting): get_auth_request con 0x7f2adc0a6d30 auth_method 0 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2ae0834640 10 monclient(hunting): get_auth_request con 0x7f2adc0a5270 auth_method 0 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2ae0834640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2ae0834640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2ae0834640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:22.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2ae0834640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:22.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2ae0834640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:22.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2ae0834640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:22.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 10 monclient(hunting): handle_auth_done global_id 6962 payload 274 2024-08-19T10:49:22.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 1 -- 172.21.15.89:0/2666785985 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2adc0a5270 msgr2=0x7f2adc0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:22.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 1 --2- 172.21.15.89:0/2666785985 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2adc0a5270 0x7f2adc0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:22.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 1 -- 172.21.15.89:0/2666785985 >> v1:172.21.15.190:6789/0 conn(0x7f2adc1395b0 legacy=0x7f2adc139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:22.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 10 monclient: _finish_hunting 0 2024-08-19T10:49:22.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 1 monclient: found mon.noname-h 2024-08-19T10:49:22.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:22.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:49:22.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.214+0000 7f2adbfff640 1 -- 172.21.15.89:0/2666785985 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc0a81d0 con 0x7f2adc0a6d30 2024-08-19T10:49:22.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adbfff640 1 -- 172.21.15.89:0/2666785985 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2adc0a6d30 msgr2=0x7f2adc0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:22.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adbfff640 1 --2- 172.21.15.89:0/2666785985 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2adc0a6d30 0x7f2adc0a7110 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0097f0 tx=0x7f2acc01c1c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=757858ed0a4cbc3f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:22.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 1 -- 172.21.15.89:0/2666785985 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2acc0b0070 con 0x7f2adc0a6d30 2024-08-19T10:49:22.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 10 monclient: dump: 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:22.220 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 1 -- 172.21.15.89:0/2666785985 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2acc0aa810 con 0x7f2adc0a6d30 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 1 -- 172.21.15.89:0/2666785985 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2acc0a9300 con 0x7f2adc0a6d30 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:22.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2adb7fe640 10 monclient: dump: 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:22.222 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 5 monclient: authenticate success, global_id 6962 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 4 set_mon_vals no callback set 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 10 monclient: shutdown 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 1 -- 172.21.15.89:0/2666785985 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2adc0a6d30 msgr2=0x7f2adc0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 1 --2- 172.21.15.89:0/2666785985 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2adc0a6d30 0x7f2adc0a7110 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0097f0 tx=0x7f2acc01c1c0 comp rx=0 tx=0).stop 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 1 -- 172.21.15.89:0/2666785985 shutdown_connections 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 1 --2- 172.21.15.89:0/2666785985 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2adc0a6d30 0x7f2adc0a7110 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:22.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 1 --2- 172.21.15.89:0/2666785985 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2adc0a5270 0x7f2adc0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 1 -- 172.21.15.89:0/2666785985 >> 172.21.15.89:0/2666785985 conn(0x7f2adc069a80 msgr2=0x7f2adc0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 1 -- 172.21.15.89:0/2666785985 shutdown_connections 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 1 -- 172.21.15.89:0/2666785985 wait complete. 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 10 monclient: build_initial_monmap 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 10 monclient: monmap: 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:22.224 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:22.225 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:22.225 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:22.225 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:22.225 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:22.225 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:22.225 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:22.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 1 Processor -- start 2024-08-19T10:49:22.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 1 -- start start 2024-08-19T10:49:22.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.218+0000 7f2ae2a98640 10 monclient: init 2024-08-19T10:49:22.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient: _add_conns ranks=[0,3,1,5,4,8,6,2,7] 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2adc0a5270 0x7f2adc13fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient(hunting): picked mon.noname-a con 0x7f2adc0a5270 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc0a6d30 0x7f2adc1d1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient(hunting): picked mon.noname-g con 0x7f2adc0a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2adc1395b0 0x7f2adc13d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient(hunting): picked mon.noname-f con 0x7f2adc1395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0a57d0 con 0x7f2adc0a5270 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0a5650 con 0x7f2adc1395b0 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:22.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2adc144240 con 0x7f2adc0a6d30 2024-08-19T10:49:22.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:22.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient(hunting): authenticate will time out at 1612.040495s 2024-08-19T10:49:22.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2adc0a5270 0x7f2adc13fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:22.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2adc0a5270 0x7f2adc13fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55648/0 (socket says 172.21.15.89:55648) 2024-08-19T10:49:22.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 1 -- 172.21.15.89:0/3957059762 learned_addr learned my addr 172.21.15.89:0/3957059762 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:22.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 10 monclient(hunting): get_auth_request con 0x7f2adc0a5270 auth_method 0 2024-08-19T10:49:22.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:22.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:22.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2adbfff640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc0a6d30 0x7f2adc1d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae1035640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2adc1395b0 0x7f2adc13d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae1035640 10 monclient(hunting): get_auth_request con 0x7f2adc1395b0 auth_method 0 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae1035640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae1035640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae1035640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2adbfff640 10 monclient(hunting): get_auth_request con 0x7f2adc0a6d30 auth_method 0 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2adbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2adbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:22.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2adbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 10 monclient(hunting): handle_auth_done global_id 6984 payload 824 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 1 -- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc0a6d30 msgr2=0x7f2adc1d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc0a6d30 0x7f2adc1d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 1 -- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2adc1395b0 msgr2=0x7f2adc13d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2adc1395b0 0x7f2adc13d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 10 monclient: _finish_hunting 0 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 1 monclient: found mon.noname-a 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 1 -- 172.21.15.89:0/3957059762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc122ce0 con 0x7f2adc0a5270 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 1 -- 172.21.15.89:0/3957059762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2adc0a5270 msgr2=0x7f2adc13fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae0834640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2adc0a5270 0x7f2adc13fb50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000e870 tx=0x7f2ad000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 1 -- 172.21.15.89:0/3957059762 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ad0017070 con 0x7f2adc0a5270 2024-08-19T10:49:22.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 10 monclient: dump: 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:22.233 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 10 monclient: _finish_auth 0 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 10 monclient: _check_auth_tickets 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 5 monclient: authenticate success, global_id 6984 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 1 -- 172.21.15.89:0/3957059762 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad0013470 con 0x7f2adc0a5270 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:22.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 1 -- 172.21.15.89:0/3957059762 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ad0013a90 con 0x7f2adc0a5270 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ad97fa640 10 monclient: dump: 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:22.235 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:22.236 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient: _renew_subs 2024-08-19T10:49:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2adaffd640 4 set_mon_vals no callback set 2024-08-19T10:49:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 1 -- 172.21.15.89:0/3957059762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2adc0a7180 con 0x7f2adc0a5270 2024-08-19T10:49:22.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient: _renew_subs 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.222+0000 7f2ae2a98640 1 -- 172.21.15.89:0/3957059762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2adc122720 con 0x7f2adc0a5270 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.226+0000 7f2ae2a98640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.226+0000 7f2ae2a98640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.226+0000 7f2ae2a98640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.226+0000 7f2ae2a98640 1 -- 172.21.15.89:0/3957059762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2adc0a67f0 con 0x7f2adc0a5270 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.226+0000 7f2ad97fa640 1 -- 172.21.15.89:0/3957059762 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2ad001a070 con 0x7f2adc0a5270 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.226+0000 7f2ad97fa640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ac803be00 0x7f2ac803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.226+0000 7f2ad97fa640 1 -- 172.21.15.89:0/3957059762 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f2ad000f040 con 0x7f2adc0a5270 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.226+0000 7f2adbfff640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ac803be00 0x7f2ac803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.226+0000 7f2adbfff640 10 monclient: get_auth_request con 0x7f2ac803be00 auth_method 0 2024-08-19T10:49:22.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.230+0000 7f2adbfff640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ac803be00 0x7f2ac803e2c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0acc30 tx=0x7f2acc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:22.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.230+0000 7f2ad97fa640 1 -- 172.21.15.89:0/3957059762 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f2ad0021020 con 0x7f2adc0a5270 2024-08-19T10:49:22.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.230+0000 7f2ad97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:22.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.230+0000 7f2ad97fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.462+0000 7f2ae2a98640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:49:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.462+0000 7f2ae2a98640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.462+0000 7f2ae2a98640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:49:22.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.462+0000 7f2ae2a98640 1 -- 172.21.15.89:0/3957059762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2adc064590 con 0x7f2adc0a5270 2024-08-19T10:49:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.462+0000 7f2ad97fa640 1 -- 172.21.15.89:0/3957059762 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7f2ad0013e10 con 0x7f2adc0a5270 2024-08-19T10:49:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.462+0000 7f2ad97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:22.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.462+0000 7f2ad97fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:22.462 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:49:22.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 -- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ac803be00 msgr2=0x7f2ac803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ac803be00 0x7f2ac803e2c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0acc30 tx=0x7f2acc0a6000 comp rx=0 tx=0).stop 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 10 monclient: shutdown 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 -- 172.21.15.89:0/3957059762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2adc0a5270 msgr2=0x7f2adc13fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2adc0a5270 0x7f2adc13fb50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000e870 tx=0x7f2ad000ec70 comp rx=0 tx=0).stop 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 -- 172.21.15.89:0/3957059762 shutdown_connections 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ac803be00 0x7f2ac803e2c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2adc1395b0 0x7f2adc13d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc0a6d30 0x7f2adc1d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 --2- 172.21.15.89:0/3957059762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2adc0a5270 0x7f2adc13fb50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 -- 172.21.15.89:0/3957059762 >> 172.21.15.89:0/3957059762 conn(0x7f2adc069a80 msgr2=0x7f2adc0ab1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 -- 172.21.15.89:0/3957059762 shutdown_connections 2024-08-19T10:49:22.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:22.466+0000 7f2ae2a98640 1 -- 172.21.15.89:0/3957059762 wait complete. 2024-08-19T10:49:25.483 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:49:25.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient: get_monmap_and_config 2024-08-19T10:49:25.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient: build_initial_monmap 2024-08-19T10:49:25.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient: monmap: 2024-08-19T10:49:25.613 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:25.613 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:25.613 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:25.618125+0000 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:25.618125+0000 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:25.614 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:25.615 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:25.615 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:25.615 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:25.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 1 Processor -- start 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 1 -- start start 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient: init 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient: _add_conns ranks=[0,8,1,9,10,2,4,3,6,7,5] 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f98400a5270 addr v1:172.21.15.89:6789/0 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient(hunting): picked mon.noname-e con 0x7f98400a6f00 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f98400a72e0 addr v1:172.21.15.190:6789/0 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98400a8710 con 0x7f98400a5270 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98400a5a40 con 0x7f98400a72e0 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f98400a81d0 con 0x7f98400a6f00 2024-08-19T10:49:25.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f984763f640 10 monclient(hunting): authenticate will time out at 1615.434880s 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f9844bda640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f98453db640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f98400a5270 0x7f98400a67f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56348/0 (socket says 172.21.15.89:56348) 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f9844bda640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:34462/0 (socket says 172.21.15.89:34462) 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 learned_addr learned my addr 172.21.15.89:0/4017684504 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f9844bda640 10 monclient(hunting): get_auth_request con 0x7f98400a6f00 auth_method 0 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f9844bda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f9844bda640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f9844bda640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 msgr2=0x7f984013ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 msgr2=0x7f984013ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:25.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.614+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:25.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.618+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> v1:172.21.15.89:6789/0 conn(0x7f98400a5270 legacy=0x7f98400a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:25.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.618+0000 7f9845bdc640 1 -- 172.21.15.89:0/4017684504 >> v1:172.21.15.190:6789/0 conn(0x7f98400a72e0 legacy=0x7f9840140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.818+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.818+0000 7f9844bda640 10 monclient(hunting): get_auth_request con 0x7f98400a6f00 auth_method 0 2024-08-19T10:49:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.818+0000 7f9844bda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.818+0000 7f9844bda640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.818+0000 7f9844bda640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.818+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 msgr2=0x7f984013ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:49:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.818+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 msgr2=0x7f984013ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.818+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:25.818+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:49:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:26.222+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:26.222+0000 7f9844bda640 10 monclient(hunting): get_auth_request con 0x7f98400a6f00 auth_method 0 2024-08-19T10:49:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:26.222+0000 7f9844bda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:26.222+0000 7f9844bda640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:26.222+0000 7f9844bda640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:26.222+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 msgr2=0x7f984013ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:49:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:26.222+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 msgr2=0x7f984013ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:26.222+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:26.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:26.222+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:49:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:27.026+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:27.026+0000 7f9844bda640 10 monclient(hunting): get_auth_request con 0x7f98400a6f00 auth_method 0 2024-08-19T10:49:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:27.026+0000 7f9844bda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:27.026+0000 7f9844bda640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:27.026+0000 7f9844bda640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:27.026+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 msgr2=0x7f984013ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:49:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:27.026+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 msgr2=0x7f984013ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:27.026+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:27.026+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:49:28.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 10 monclient(hunting): tick 2024-08-19T10:49:28.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:49:28.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 1 monclient(hunting): continuing hunt 2024-08-19T10:49:28.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:49:28.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 msgr2=0x7f984013ea00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:49:28.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98400a6f00 0x7f984013ea00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:28.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 >> v1:172.21.15.190:6789/0 conn(0x7f98400a72e0 legacy=0x7f9840140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:28.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 >> v1:172.21.15.89:6789/0 conn(0x7f98400a5270 legacy=0x7f98400a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:28.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 10 monclient: _add_conns ranks=[7,3,5,6,10,9,4,2] 2024-08-19T10:49:28.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:28.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 10 monclient(hunting): picked mon.noname-i con 0x7f982c003930 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:28.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:28.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 10 monclient(hunting): picked mon.noname-g con 0x7f982c006470 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:28.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:28.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 10 monclient(hunting): picked mon.noname-h con 0x7f982c008ff0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:28.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:28.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f98400a81d0 con 0x7f982c006470 2024-08-19T10:49:28.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:28.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f982c00bb80 con 0x7f982c008ff0 2024-08-19T10:49:28.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:28.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f982c00bd80 con 0x7f982c003930 2024-08-19T10:49:28.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f983effd640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:28.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:28.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:28.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:28.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f98453db640 10 monclient(hunting): get_auth_request con 0x7f982c003930 auth_method 0 2024-08-19T10:49:28.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f98453db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:28.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f98453db640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:28.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f98453db640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:28.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f9844bda640 10 monclient(hunting): get_auth_request con 0x7f982c006470 auth_method 0 2024-08-19T10:49:28.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f9844bda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:28.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f9844bda640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:28.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f9844bda640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:28.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f9845bdc640 10 monclient(hunting): get_auth_request con 0x7f982c008ff0 auth_method 0 2024-08-19T10:49:28.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f9845bdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:28.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f9845bdc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:28.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.614+0000 7f9845bdc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:28.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 msgr2=0x7f982c005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:49:28.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 msgr2=0x7f982c005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:28.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:28.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f9845bdc640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 msgr2=0x7f982c00b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:49:28.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 msgr2=0x7f982c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:49:28.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f9845bdc640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 msgr2=0x7f982c00b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:28.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 msgr2=0x7f982c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:28.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:28.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:28.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:28.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:28.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.618+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9845bdc640 10 monclient(hunting): get_auth_request con 0x7f982c008ff0 auth_method 0 2024-08-19T10:49:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9845bdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9845bdc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9845bdc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f98453db640 10 monclient(hunting): get_auth_request con 0x7f982c003930 auth_method 0 2024-08-19T10:49:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f98453db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f98453db640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f98453db640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9844bda640 10 monclient(hunting): get_auth_request con 0x7f982c006470 auth_method 0 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9844bda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9844bda640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9844bda640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9845bdc640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 msgr2=0x7f982c00b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 msgr2=0x7f982c005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9845bdc640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 msgr2=0x7f982c00b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 msgr2=0x7f982c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 msgr2=0x7f982c005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 msgr2=0x7f982c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:49:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:49:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:28.822+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:49:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f98453db640 10 monclient(hunting): get_auth_request con 0x7f982c003930 auth_method 0 2024-08-19T10:49:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f98453db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f98453db640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f98453db640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9844bda640 10 monclient(hunting): get_auth_request con 0x7f982c006470 auth_method 0 2024-08-19T10:49:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9844bda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9844bda640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9844bda640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9845bdc640 10 monclient(hunting): get_auth_request con 0x7f982c008ff0 auth_method 0 2024-08-19T10:49:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9845bdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9845bdc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9845bdc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 msgr2=0x7f982c005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:49:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 msgr2=0x7f982c005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 msgr2=0x7f982c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:49:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:49:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 msgr2=0x7f982c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9845bdc640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 msgr2=0x7f982c00b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:49:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9845bdc640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 msgr2=0x7f982c00b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:49:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:29.226+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:49:30.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:30.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f98453db640 10 monclient(hunting): get_auth_request con 0x7f982c003930 auth_method 0 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f98453db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f98453db640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f98453db640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9844bda640 10 monclient(hunting): get_auth_request con 0x7f982c006470 auth_method 0 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9844bda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9844bda640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9844bda640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9845bdc640 10 monclient(hunting): get_auth_request con 0x7f982c008ff0 auth_method 0 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9845bdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9845bdc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:30.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9845bdc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 msgr2=0x7f982c005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 msgr2=0x7f982c005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 msgr2=0x7f982c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9845bdc640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 msgr2=0x7f982c00b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 msgr2=0x7f982c008900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9845bdc640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 msgr2=0x7f982c00b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:49:30.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:30.030+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:49:31.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 10 monclient(hunting): tick 2024-08-19T10:49:31.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:49:31.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 monclient(hunting): continuing hunt 2024-08-19T10:49:31.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:49:31.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 msgr2=0x7f982c005d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:49:31.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c003930 0x7f982c005d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 msgr2=0x7f982c00b440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f982c008ff0 0x7f982c00b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 msgr2=0x7f982c008900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c006470 0x7f982c008900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 10 monclient: _add_conns ranks=[9,2,6,4,10] 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f9845bdc640 1 -- 172.21.15.89:0/4017684504 reap_dead start 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 --2- 172.21.15.89:0/4017684504 >> v2:172.21.15.89:3300/0 conn(0x7f982c00c380 0x7f982c00e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 10 monclient(hunting): picked mon.noname-a con 0x7f982c00c380 addr v2:172.21.15.89:3300/0 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f982c00eda0 0x7f982c0111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 10 monclient(hunting): picked mon.noname-b con 0x7f982c00eda0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f9845bdc640 1 -- 172.21.15.89:0/4017684504 reap_dead start 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f982c003d70 0x7f982c009430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 10 monclient(hunting): picked mon.noname-d con 0x7f982c003d70 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f98400a81d0 con 0x7f982c00c380 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f982c00bb80 con 0x7f982c00eda0 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 1 -- 172.21.15.89:0/4017684504 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f982c00bd80 con 0x7f982c003d70 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f983effd640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f9844bda640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f982c00eda0 msgr2=0x7f982c0111c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.614+0000 7f9844bda640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f982c00eda0 0x7f982c0111c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> v2:172.21.15.89:3300/0 conn(0x7f982c00c380 0x7f982c00e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 10 monclient(hunting): get_auth_request con 0x7f982c00c380 auth_method 0 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f9845bdc640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f982c003d70 0x7f982c009430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f9845bdc640 10 monclient(hunting): get_auth_request con 0x7f982c003d70 auth_method 0 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f9845bdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f9845bdc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f9845bdc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f9845bdc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f9845bdc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f9845bdc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 10 monclient(hunting): handle_auth_done global_id 6993 payload 274 2024-08-19T10:49:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f982c003d70 msgr2=0x7f982c009430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f982c003d70 0x7f982c009430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f982c00eda0 msgr2=0x7f982c0111c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:49:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f982c00eda0 0x7f982c0111c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 10 monclient: _finish_hunting 0 2024-08-19T10:49:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 1 monclient: found mon.noname-a 2024-08-19T10:49:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.618+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98400a4790 con 0x7f982c00c380 2024-08-19T10:49:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f98453db640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f982c00c380 msgr2=0x7f982c00e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f98453db640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f982c00c380 0x7f982c00e770 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9838001c40 tx=0x7f983801da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5a854c8e61b17e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:31.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 1 -- 172.21.15.89:0/4017684504 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98380ac070 con 0x7f982c00c380 2024-08-19T10:49:31.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 10 monclient: dump: 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:31.626 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 10 monclient: _finish_auth 0 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 5 monclient: authenticate success, global_id 6993 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 1 -- 172.21.15.89:0/4017684504 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98380a6690 con 0x7f982c00c380 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 1 -- 172.21.15.89:0/4017684504 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98380a6cf0 con 0x7f982c00c380 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f983ffff640 10 monclient: dump: 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:31.629 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 4 set_mon_vals no callback set 2024-08-19T10:49:31.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 10 monclient: shutdown 2024-08-19T10:49:31.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 1 -- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f982c00c380 msgr2=0x7f982c00e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f982c00c380 0x7f982c00e770 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9838001c40 tx=0x7f983801da60 comp rx=0 tx=0).stop 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 1 -- 172.21.15.89:0/4017684504 shutdown_connections 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f982c003d70 0x7f982c009430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f982c00c380 0x7f982c00e770 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 1 --2- 172.21.15.89:0/4017684504 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f982c00eda0 0x7f982c0111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 1 -- 172.21.15.89:0/4017684504 >> 172.21.15.89:0/4017684504 conn(0x7f9840069a80 msgr2=0x7f98400abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 1 -- 172.21.15.89:0/4017684504 shutdown_connections 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.626+0000 7f984763f640 1 -- 172.21.15.89:0/4017684504 wait complete. 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient: build_initial_monmap 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient: monmap: 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:31.631 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:31.632 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:31.633 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:31.633 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:31.633 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:31.633 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:31.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 1 Processor -- start 2024-08-19T10:49:31.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 1 -- start start 2024-08-19T10:49:31.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient: init 2024-08-19T10:49:31.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:31.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient: _add_conns ranks=[1,7,3,5,6,8,4,2,0] 2024-08-19T10:49:31.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f982c003d70 0x7f9840138530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:31.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient(hunting): picked mon.noname-f con 0x7f982c003d70 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:31.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c00c380 0x7f9840138a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:31.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient(hunting): picked mon.noname-i con 0x7f982c00c380 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:31.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c00eda0 0x7f9840122440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:31.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient(hunting): picked mon.noname-g con 0x7f982c00eda0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:31.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:31.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f98400a57d0 con 0x7f982c003d70 2024-08-19T10:49:31.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:31.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f98400a5650 con 0x7f982c00eda0 2024-08-19T10:49:31.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:31.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9840139dc0 con 0x7f982c00c380 2024-08-19T10:49:31.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f984763f640 10 monclient(hunting): authenticate will time out at 1621.451593s 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f98453db640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f982c003d70 msgr2=0x7f9840138530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f98453db640 1 -- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f982c003d70 msgr2=0x7f9840138530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f98453db640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f982c003d70 0x7f9840138530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f9845bdc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c00eda0 0x7f9840122440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.630+0000 7f98453db640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f982c003d70 0x7f9840138530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c00c380 0x7f9840138a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c00c380 0x7f9840138a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:33412/0 (socket says 172.21.15.89:33412) 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 1 -- 172.21.15.89:0/1562559439 learned_addr learned my addr 172.21.15.89:0/1562559439 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 10 monclient(hunting): get_auth_request con 0x7f982c00c380 auth_method 0 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:31.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9845bdc640 10 monclient(hunting): get_auth_request con 0x7f982c00eda0 auth_method 0 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9845bdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9845bdc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9845bdc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9845bdc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9845bdc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9845bdc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:31.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 10 monclient(hunting): handle_auth_done global_id 64222 payload 824 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 1 -- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c00eda0 msgr2=0x7f9840122440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c00eda0 0x7f9840122440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 1 -- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f982c003d70 msgr2=0x7f9840138530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f982c003d70 0x7f9840138530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 10 monclient: _finish_hunting 0 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 1 monclient: found mon.noname-i 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 1 -- 172.21.15.89:0/1562559439 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9840123b90 con 0x7f982c00c380 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 1 -- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c00c380 msgr2=0x7f9840138a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f9844bda640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c00c380 0x7f9840138a70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9834016990 tx=0x7f9834016e50 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 1 -- 172.21.15.89:0/1562559439 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9834027070 con 0x7f982c00c380 2024-08-19T10:49:31.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 10 monclient: dump: 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:31.641 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 10 monclient: _finish_auth 0 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f984763f640 5 monclient: authenticate success, global_id 64222 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f984763f640 10 monclient: _renew_subs 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 1 -- 172.21.15.89:0/1562559439 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9834014d50 con 0x7f982c00c380 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f984763f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:49:31.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f984763f640 1 -- 172.21.15.89:0/1562559439 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9840125ef0 con 0x7f982c00c380 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 1 -- 172.21.15.89:0/1562559439 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f983400f040 con 0x7f982c00c380 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983f7fe640 4 set_mon_vals no callback set 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f983dffb640 10 monclient: dump: 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:31.643 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f984763f640 10 monclient: _renew_subs 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f984763f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.634+0000 7f984763f640 1 -- 172.21.15.89:0/1562559439 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f9840122980 con 0x7f982c00c380 2024-08-19T10:49:31.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.638+0000 7f984763f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:31.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.638+0000 7f984763f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:31.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.638+0000 7f984763f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:49:31.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.638+0000 7f984763f640 1 -- 172.21.15.89:0/1562559439 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984013a150 con 0x7f982c00c380 2024-08-19T10:49:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.638+0000 7f983dffb640 1 -- 172.21.15.89:0/1562559439 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f98340173e0 con 0x7f982c00c380 2024-08-19T10:49:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.638+0000 7f983dffb640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f982c03e260 0x7f982c040720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.638+0000 7f983dffb640 1 -- 172.21.15.89:0/1562559439 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f983401f070 con 0x7f982c00c380 2024-08-19T10:49:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.638+0000 7f98453db640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f982c03e260 0x7f982c040720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.638+0000 7f98453db640 10 monclient: get_auth_request con 0x7f982c03e260 auth_method 0 2024-08-19T10:49:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.642+0000 7f98453db640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f982c03e260 0x7f982c040720 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f982c00c290 tx=0x7f9838010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:31.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.650+0000 7f983dffb640 1 -- 172.21.15.89:0/1562559439 <== mon.7 v2:172.21.15.190:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f9834017b50 con 0x7f982c00c380 2024-08-19T10:49:31.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.650+0000 7f983dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:31.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.650+0000 7f983dffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.890+0000 7f984763f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:49:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.890+0000 7f984763f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.890+0000 7f984763f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:49:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.890+0000 7f984763f640 1 -- 172.21.15.89:0/1562559439 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9840064590 con 0x7f982c00c380 2024-08-19T10:49:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.890+0000 7f983dffb640 1 -- 172.21.15.89:0/1562559439 <== mon.7 v2:172.21.15.190:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3187 (secure 0 0 0) 0x7f983404f070 con 0x7f982c00c380 2024-08-19T10:49:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.890+0000 7f983dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:49:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.890+0000 7f983dffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:31.891 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":102,"quorum":[0,1,3,4,5,6,7,8],"quorum_names":["a","f","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 -- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f982c03e260 msgr2=0x7f982c040720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f982c03e260 0x7f982c040720 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f982c00c290 tx=0x7f9838010040 comp rx=0 tx=0).stop 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 10 monclient: shutdown 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 -- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c00c380 msgr2=0x7f9840138a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c00c380 0x7f9840138a70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9834016990 tx=0x7f9834016e50 comp rx=0 tx=0).stop 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 -- 172.21.15.89:0/1562559439 shutdown_connections 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f982c03e260 0x7f982c040720 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f982c00eda0 0x7f9840122440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f982c00c380 0x7f9840138a70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 --2- 172.21.15.89:0/1562559439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f982c003d70 0x7f9840138530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 -- 172.21.15.89:0/1562559439 >> 172.21.15.89:0/1562559439 conn(0x7f9840069a80 msgr2=0x7f98400aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 -- 172.21.15.89:0/1562559439 shutdown_connections 2024-08-19T10:49:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:31.894+0000 7f984763f640 1 -- 172.21.15.89:0/1562559439 wait complete. 2024-08-19T10:49:31.911 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-08-19T10:49:31.911 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-08-19T10:49:31.911 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:49:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.026+0000 7f790cb8a640 10 monclient: get_monmap_and_config 2024-08-19T10:49:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.026+0000 7f790cb8a640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient: monmap: 2024-08-19T10:49:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:32.032939+0000 2024-08-19T10:49:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:32.032939+0000 2024-08-19T10:49:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.028 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:32.029 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:32.029 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 1 Processor -- start 2024-08-19T10:49:32.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 1 -- start start 2024-08-19T10:49:32.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient: init 2024-08-19T10:49:32.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient: _add_conns ranks=[3,2,6,7,5,1,8,9,0,4,10] 2024-08-19T10:49:32.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79080a3620 0x7f79080a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient(hunting): picked mon.noname-g con 0x7f79080a3620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:32.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f790809c9e0 0x7f79080a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient(hunting): picked mon.noname-b con 0x7f790809c9e0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:32.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79081373f0 0x7f7908137810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient(hunting): picked mon.noname-d con 0x7f79081373f0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:32.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f790809dab0 con 0x7f790809c9e0 2024-08-19T10:49:32.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f79080a1f20 con 0x7f79080a3620 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f79080a4480 con 0x7f79081373f0 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790cb8a640 10 monclient(hunting): authenticate will time out at 1621.849482s 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7905d9b640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f790809c9e0 msgr2=0x7f79080a3f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7905d9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f790809c9e0 0x7f79080a3f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79081373f0 0x7f7908137810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79081373f0 0x7f7908137810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:44972/0 (socket says 172.21.15.89:44972) 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 1 -- 172.21.15.89:0/1993082212 learned_addr learned my addr 172.21.15.89:0/1993082212 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 10 monclient(hunting): get_auth_request con 0x7f79081373f0 auth_method 0 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790659c640 1 --2- 172.21.15.89:0/1993082212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79080a3620 0x7f79080a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790659c640 10 monclient(hunting): get_auth_request con 0x7f79080a3620 auth_method 0 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f790659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 10 monclient(hunting): handle_auth_done global_id 44826 payload 274 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 1 -- 172.21.15.89:0/1993082212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79080a3620 msgr2=0x7f79080a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 1 --2- 172.21.15.89:0/1993082212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79080a3620 0x7f79080a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 1 -- 172.21.15.89:0/1993082212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f790809c9e0 msgr2=0x7f79080a3f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:49:32.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 1 --2- 172.21.15.89:0/1993082212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f790809c9e0 0x7f79080a3f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 1 monclient: found mon.noname-d 2024-08-19T10:49:32.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:49:32.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.030+0000 7f7906d9d640 1 -- 172.21.15.89:0/1993082212 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79080a1790 con 0x7f79081373f0 2024-08-19T10:49:32.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f7906d9d640 1 -- 172.21.15.89:0/1993082212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79081373f0 msgr2=0x7f7908137810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f7906d9d640 1 --2- 172.21.15.89:0/1993082212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79081373f0 0x7f7908137810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7900009eb0 tx=0x7f790001c910 comp rx=0 tx=0).ready entity=mon.6 client_cookie=cf589374c312e3ff server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 1 -- 172.21.15.89:0/1993082212 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79000090d0 con 0x7f79081373f0 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 10 monclient: dump: 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.039 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 1 -- 172.21.15.89:0/1993082212 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79000a5a30 con 0x7f79081373f0 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 1 -- 172.21.15.89:0/1993082212 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f79000a5be0 con 0x7f79081373f0 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790559a640 10 monclient: dump: 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.041 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 5 monclient: authenticate success, global_id 44826 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 4 set_mon_vals no callback set 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 10 monclient: shutdown 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:32.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 1 -- 172.21.15.89:0/1993082212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79081373f0 msgr2=0x7f7908137810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 1 --2- 172.21.15.89:0/1993082212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79081373f0 0x7f7908137810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7900009eb0 tx=0x7f790001c910 comp rx=0 tx=0).stop 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 1 -- 172.21.15.89:0/1993082212 shutdown_connections 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 1 --2- 172.21.15.89:0/1993082212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f79081373f0 0x7f7908137810 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 1 --2- 172.21.15.89:0/1993082212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f790809c9e0 0x7f79080a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 1 --2- 172.21.15.89:0/1993082212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79080a3620 0x7f79080a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 1 -- 172.21.15.89:0/1993082212 >> 172.21.15.89:0/1993082212 conn(0x7f79080a8210 msgr2=0x7f79080a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 1 -- 172.21.15.89:0/1993082212 shutdown_connections 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 1 -- 172.21.15.89:0/1993082212 wait complete. 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.034+0000 7f790cb8a640 10 monclient: monmap: 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:32.043 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:32.044 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:32.044 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.044 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.044 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:32.044 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.045 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 1 Processor -- start 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 1 -- start start 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient: init 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient: _add_conns ranks=[0,4,2,7,3,1,8,5,6] 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790809c9e0 0x7f79081361a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient(hunting): picked mon.noname-a con 0x7f790809c9e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79080a3620 0x7f790809f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient(hunting): picked mon.noname-c con 0x7f79080a3620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081373f0 0x7f790809f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient(hunting): picked mon.noname-b con 0x7f79081373f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7908140e60 con 0x7f790809c9e0 2024-08-19T10:49:32.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f790809df40 con 0x7f79081373f0 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f7908140fe0 con 0x7f79080a3620 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient(hunting): authenticate will time out at 1621.856774s 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7906d9d640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081373f0 msgr2=0x7f790809f6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7906d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081373f0 0x7f790809f6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79080a3620 0x7f790809f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790809c9e0 0x7f79081361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790809c9e0 0x7f79081361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35106/0 (socket says 172.21.15.89:35106) 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79080a3620 0x7f790809f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44406/0 (socket says 172.21.15.89:44406) 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790659c640 1 -- 172.21.15.89:0/3110069308 learned_addr learned my addr 172.21.15.89:0/3110069308 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:32.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790659c640 10 monclient(hunting): get_auth_request con 0x7f790809c9e0 auth_method 0 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 10 monclient(hunting): get_auth_request con 0x7f79080a3620 auth_method 0 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 10 monclient(hunting): handle_auth_done global_id 44824 payload 824 2024-08-19T10:49:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 1 -- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081373f0 msgr2=0x7f790809f6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:49:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f79081373f0 0x7f790809f6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 1 -- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790809c9e0 msgr2=0x7f79081361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f790809c9e0 0x7f79081361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 1 monclient: found mon.noname-c 2024-08-19T10:49:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:49:32.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 1 -- 172.21.15.89:0/3110069308 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79081239c0 con 0x7f79080a3620 2024-08-19T10:49:32.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 1 -- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79080a3620 msgr2=0x7f790809f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7905d9b640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79080a3620 0x7f790809f180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00b400 tx=0x7f78fc00b8c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78fc016070 con 0x7f79080a3620 2024-08-19T10:49:32.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:49:32.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 10 monclient: dump: 2024-08-19T10:49:32.054 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.055 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78fc003de0 con 0x7f79080a3620 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78fc0103a0 con 0x7f79080a3620 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f78f6ffd640 10 monclient: dump: 2024-08-19T10:49:32.056 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.057 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 5 monclient: authenticate success, global_id 44824 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient: _renew_subs 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f7904d99640 4 set_mon_vals no callback set 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 1 -- 172.21.15.89:0/3110069308 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7908136be0 con 0x7f79080a3620 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient: _renew_subs 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.038+0000 7f790cb8a640 1 -- 172.21.15.89:0/3110069308 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f79080a0ac0 con 0x7f79080a3620 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790cb8a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790cb8a640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79080a0d40 0x7f790809fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790cb8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790cb8a640 1 -- 172.21.15.89:0/3110069308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f790813d6f0 con 0x7f79080a0d40 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79080a0d40 0x7f790809fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 10 monclient: get_auth_request con 0x7f79080a0d40 auth_method 0 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f78fc005070 con 0x7f79080a3620 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 10 monclient(hunting): handle_auth_done global_id 7011 payload 824 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 1 -- 172.21.15.89:0/3110069308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7908123800 con 0x7f79080a0d40 2024-08-19T10:49:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f78e803be30 0x7f78e803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f78fc04e770 con 0x7f79080a3620 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f7906d9d640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f78e803be30 0x7f78e803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f7906d9d640 10 monclient: get_auth_request con 0x7f78e803be30 auth_method 0 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 1 -- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79080a0d40 msgr2=0x7f790809fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f790659c640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79080a0d40 0x7f790809fc00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f78f8010890 tx=0x7f78f8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78f8018070 con 0x7f79080a0d40 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 10 monclient: dump: 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.060 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.061 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f78f800ecb0 con 0x7f79080a0d40 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79080a0d40 msgr2=0x7f790809fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79080a0d40 0x7f790809fc00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f78f8010890 tx=0x7f78f8010d50 comp rx=0 tx=0).stop 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79080a0d40 msgr2=0x7f790809fc00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.042+0000 7f78f6ffd640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79080a0d40 0x7f790809fc00 secure :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f78f8010890 tx=0x7f78f8010d50 comp rx=0 tx=0).stop 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.046+0000 7f7906d9d640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f78e803be30 0x7f78e803e2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7900009e80 tx=0x7f79000a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790cb8a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790cb8a640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79081366e0 0x7f79080a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790cb8a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790cb8a640 1 -- 172.21.15.89:0/3110069308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7908137810 con 0x7f79081366e0 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790659c640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79081366e0 0x7f79080a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790659c640 10 monclient: get_auth_request con 0x7f79081366e0 auth_method 0 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790659c640 10 monclient(hunting): handle_auth_done global_id 7020 payload 824 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.050+0000 7f790659c640 1 -- 172.21.15.89:0/3110069308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7908064590 con 0x7f79081366e0 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790659c640 1 -- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79081366e0 msgr2=0x7f79080a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790659c640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79081366e0 0x7f79080a0140 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f78f8010890 tx=0x7f78f80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f78f8014cc0 con 0x7f79081366e0 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f78f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f78f6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f78f6ffd640 10 monclient: dump: 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.064 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9426 (secure 0 0 0) 0x7f78f8014850 con 0x7f79081366e0 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f78f6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f78f6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79081366e0 msgr2=0x7f79080a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f78f6ffd640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79081366e0 0x7f79080a0140 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f78f8010890 tx=0x7f78f80046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f78f6ffd640 1 -- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79081366e0 msgr2=0x7f79080a0140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f78f6ffd640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f79081366e0 0x7f79080a0140 secure :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f78f8010890 tx=0x7f78f80046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.066 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:32.066 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:32.066 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:32.066 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:49:32.066 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 102, 2024-08-19T10:49:32.066 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 701867, 2024-08-19T10:49:32.066 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:32.066 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:32.068 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.069 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:32.070 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.071 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.072 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.073 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.074 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.075 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.076 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.077 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:32.078 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.081 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:32.082 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.084 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:32.085 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.086 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.086 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.086 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:32.086 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:32.086 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.086 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:32.087 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.088 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.089 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.090 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:32.090 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:32.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790cb8a640 1 -- 172.21.15.89:0/3110069308 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f78e803be30 msgr2=0x7f78e803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790cb8a640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f78e803be30 0x7f78e803e2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7900009e80 tx=0x7f79000a7070 comp rx=0 tx=0).stop 2024-08-19T10:49:32.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790cb8a640 10 monclient: shutdown 2024-08-19T10:49:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790cb8a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790cb8a640 1 -- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79080a3620 msgr2=0x7f790809f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790cb8a640 1 --2- 172.21.15.89:0/3110069308 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f79080a3620 0x7f790809f180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00b400 tx=0x7f78fc00b8c0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f7906d9d640 1 -- 172.21.15.89:0/3110069308 reap_dead start 2024-08-19T10:49:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f7906d9d640 1 -- 172.21.15.89:0/3110069308 reap_dead start 2024-08-19T10:49:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790cb8a640 1 -- 172.21.15.89:0/3110069308 shutdown_connections 2024-08-19T10:49:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790cb8a640 1 -- 172.21.15.89:0/3110069308 >> 172.21.15.89:0/3110069308 conn(0x7f79080a8210 msgr2=0x7f790813def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790cb8a640 1 -- 172.21.15.89:0/3110069308 shutdown_connections 2024-08-19T10:49:32.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.054+0000 7f790cb8a640 1 -- 172.21.15.89:0/3110069308 wait complete. 2024-08-19T10:49:32.092 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:49:32.093 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-08-19T10:49:32.093 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:49:32.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient: get_monmap_and_config 2024-08-19T10:49:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient: monmap: 2024-08-19T10:49:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:32.213629+0000 2024-08-19T10:49:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:32.213629+0000 2024-08-19T10:49:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:32.210 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 1 Processor -- start 2024-08-19T10:49:32.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 1 -- start start 2024-08-19T10:49:32.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient: init 2024-08-19T10:49:32.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient: _add_conns ranks=[0,1,4,9,3,7,5,10,8,2,6] 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f14280a5270 addr v1:172.21.15.89:6789/0 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f14280a6f00 addr v1:172.21.15.190:6789/0 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14280a7310 0x7f1428142130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient(hunting): picked mon.noname-c con 0x7f14280a7310 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14280a8710 con 0x7f14280a5270 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14280a5a40 con 0x7f14280a6f00 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f14280a81d0 con 0x7f14280a7310 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f142d7f4640 10 monclient(hunting): authenticate will time out at 1622.030153s 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14280a7310 0x7f1428142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f1426ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f14280a5270 0x7f14280a67f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57114/0 (socket says 172.21.15.89:57114) 2024-08-19T10:49:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f1426ffd640 1 -- 172.21.15.89:0/2035732365 learned_addr learned my addr 172.21.15.89:0/2035732365 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 10 monclient(hunting): get_auth_request con 0x7f14280a7310 auth_method 0 2024-08-19T10:49:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 10 monclient(hunting): handle_auth_done global_id 44833 payload 274 2024-08-19T10:49:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 1 -- 172.21.15.89:0/2035732365 >> v1:172.21.15.190:6789/0 conn(0x7f14280a6f00 legacy=0x7f142813ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 1 -- 172.21.15.89:0/2035732365 >> v1:172.21.15.89:6789/0 conn(0x7f14280a5270 legacy=0x7f14280a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 1 monclient: found mon.noname-c 2024-08-19T10:49:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:49:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.210+0000 7f14277fe640 1 -- 172.21.15.89:0/2035732365 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14280a4790 con 0x7f14280a7310 2024-08-19T10:49:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f14277fe640 1 -- 172.21.15.89:0/2035732365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14280a7310 msgr2=0x7f1428142130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f14277fe640 1 --2- 172.21.15.89:0/2035732365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14280a7310 0x7f1428142130 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1420009dc0 tx=0x7f142001c640 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4441252c10db8f54 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 1 -- 172.21.15.89:0/2035732365 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14200ac070 con 0x7f14280a7310 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 10 monclient: dump: 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 1 -- 172.21.15.89:0/2035732365 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14200a6980 con 0x7f14280a7310 2024-08-19T10:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 1 -- 172.21.15.89:0/2035732365 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f14200a7410 con 0x7f14280a7310 2024-08-19T10:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f1425ffb640 10 monclient: dump: 2024-08-19T10:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 5 monclient: authenticate success, global_id 44833 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 4 set_mon_vals no callback set 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 10 monclient: shutdown 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 1 -- 172.21.15.89:0/2035732365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14280a7310 msgr2=0x7f1428142130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 1 --2- 172.21.15.89:0/2035732365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14280a7310 0x7f1428142130 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1420009dc0 tx=0x7f142001c640 comp rx=0 tx=0).stop 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 1 -- 172.21.15.89:0/2035732365 shutdown_connections 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 1 --2- 172.21.15.89:0/2035732365 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f14280a7310 0x7f1428142130 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 1 -- 172.21.15.89:0/2035732365 >> 172.21.15.89:0/2035732365 conn(0x7f1428069a80 msgr2=0x7f14280abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 1 -- 172.21.15.89:0/2035732365 shutdown_connections 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.214+0000 7f142d7f4640 1 -- 172.21.15.89:0/2035732365 wait complete. 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient: monmap: 2024-08-19T10:49:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 1 Processor -- start 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 1 -- start start 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient: init 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient: _add_conns ranks=[5,8,1,4,3,2,7,6,0] 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f14280a5270 0x7f14281d1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient(hunting): picked mon.noname-h con 0x7f14280a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f14280a6f00 0x7f14281d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient(hunting): picked mon.noname-e con 0x7f14280a6f00 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14280a7310 0x7f14281d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient(hunting): picked mon.noname-f con 0x7f14280a7310 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1428145010 con 0x7f14280a7310 2024-08-19T10:49:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f1428144e90 con 0x7f14280a5270 2024-08-19T10:49:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f1428145190 con 0x7f14280a6f00 2024-08-19T10:49:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f142d7f4640 10 monclient(hunting): authenticate will time out at 1622.037534s 2024-08-19T10:49:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f14280a6f00 0x7f14281d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f14280a6f00 0x7f14281d15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48620/0 (socket says 172.21.15.89:48620) 2024-08-19T10:49:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 1 -- 172.21.15.89:0/2677962452 learned_addr learned my addr 172.21.15.89:0/2677962452 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 10 monclient(hunting): get_auth_request con 0x7f14280a6f00 auth_method 0 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14277fe640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14280a7310 0x7f14281d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f1426ffd640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f14280a5270 0x7f14281d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14277fe640 10 monclient(hunting): get_auth_request con 0x7f14280a7310 auth_method 0 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14277fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14277fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 10 monclient(hunting): handle_auth_done global_id 6866 payload 824 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 1 -- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f14280a5270 msgr2=0x7f14281d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f14280a5270 0x7f14281d1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 1 -- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14280a7310 msgr2=0x7f14281d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14280a7310 0x7f14281d1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 1 monclient: found mon.noname-e 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f14267fc640 1 -- 172.21.15.89:0/2677962452 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14280a2630 con 0x7f14280a6f00 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f1426ffd640 10 monclient: get_auth_request con 0x7f14280a5270 auth_method 0 2024-08-19T10:49:32.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.218+0000 7f1426ffd640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f14280a5270 0x7f14281d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:32.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f14267fc640 1 -- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f14280a6f00 msgr2=0x7f14281d15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f14267fc640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f14280a6f00 0x7f14281d15d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f141c00c480 tx=0x7f141c00c940 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f141c012cb0 con 0x7f14280a6f00 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 10 monclient: dump: 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.234 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.235 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.235 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.235 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.235 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.235 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.235 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f141c017070 con 0x7f14280a6f00 2024-08-19T10:49:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f142d7f4640 5 monclient: authenticate success, global_id 6866 2024-08-19T10:49:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f142d7f4640 10 monclient: _renew_subs 2024-08-19T10:49:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f142d7f4640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f142d7f4640 1 -- 172.21.15.89:0/2677962452 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1428129d00 con 0x7f14280a6f00 2024-08-19T10:49:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f141c0117f0 con 0x7f14280a6f00 2024-08-19T10:49:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 10 monclient: dump: 2024-08-19T10:49:32.236 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.237 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f142d7f4640 10 monclient: _renew_subs 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f14257fa640 4 set_mon_vals no callback set 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f142d7f4640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f142d7f4640 1 -- 172.21.15.89:0/2677962452 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f1428129170 con 0x7f14280a6f00 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f142d7f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f142d7f4640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126210 0x7f14281265f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f142d7f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f142d7f4640 1 -- 172.21.15.89:0/2677962452 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f14280ab4c0 con 0x7f1428126210 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f1426ffd640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126210 0x7f14281265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f1426ffd640 10 monclient: get_auth_request con 0x7f1428126210 auth_method 0 2024-08-19T10:49:32.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f1426ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f1426ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f1426ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f1426ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f1426ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f1426ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f141c011380 con 0x7f14280a6f00 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f1426ffd640 10 monclient(hunting): handle_auth_done global_id 44842 payload 824 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f1426ffd640 1 -- 172.21.15.89:0/2677962452 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f14280a67f0 con 0x7f1428126210 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f141403be00 0x7f141403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f141c04f400 con 0x7f14280a6f00 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f14277fe640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f141403be00 0x7f141403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.222+0000 7f14277fe640 10 monclient: get_auth_request con 0x7f141403be00 auth_method 0 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f1426ffd640 1 -- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126210 msgr2=0x7f14281265f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f1426ffd640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126210 0x7f14281265f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f141800c4f0 tx=0x7f141800c9b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1418015070 con 0x7f1428126210 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f140f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f140f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f140f7fe640 10 monclient: dump: 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.240 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1418011810 con 0x7f1428126210 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f140f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f140f7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:32.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126210 msgr2=0x7f14281265f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f140f7fe640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126210 0x7f14281265f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f141800c4f0 tx=0x7f141800c9b0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126210 msgr2=0x7f14281265f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f140f7fe640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126210 0x7f14281265f0 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f141800c4f0 tx=0x7f141800c9b0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.226+0000 7f14277fe640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f141403be00 0x7f141403e2c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f14280aceb0 tx=0x7f14200a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f142d7f4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f142d7f4640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126b30 0x7f1428122440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f142d7f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f142d7f4640 1 -- 172.21.15.89:0/2677962452 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f1428064590 con 0x7f1428126b30 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f1426ffd640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126b30 0x7f1428122440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f1426ffd640 10 monclient: get_auth_request con 0x7f1428126b30 auth_method 0 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f1426ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f1426ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f1426ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f1426ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f1426ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f1426ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f1426ffd640 10 monclient(hunting): handle_auth_done global_id 44851 payload 824 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.230+0000 7f1426ffd640 1 -- 172.21.15.89:0/2677962452 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f14280008e0 con 0x7f1428126b30 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f1426ffd640 1 -- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126b30 msgr2=0x7f1428122440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f1426ffd640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126b30 0x7f1428122440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f141800c4f0 tx=0x7f14180058e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1418024860 con 0x7f1428126b30 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f140f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f140f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f140f7fe640 10 monclient: dump: 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.243 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9257 (secure 0 0 0) 0x7f14180243f0 con 0x7f1428126b30 2024-08-19T10:49:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f140f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f140f7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126b30 msgr2=0x7f1428122440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f140f7fe640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126b30 0x7f1428122440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f141800c4f0 tx=0x7f14180058e0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f140f7fe640 1 -- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126b30 msgr2=0x7f1428122440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f140f7fe640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1428126b30 0x7f1428122440 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f141800c4f0 tx=0x7f14180058e0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.245 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:32.245 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:32.245 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 102, 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 295360, 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:32.246 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.247 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.248 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:32.249 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.250 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.250 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:32.250 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:32.250 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:32.250 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:32.250 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:32.250 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:32.250 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:32.250 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.250 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:32.250 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.251 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:32.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.252 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:32.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.252 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:32.252 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:32.252 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.252 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.253 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.253 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:32.253 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.253 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.253 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.253 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:32.253 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.253 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:32.254 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.256 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:32.257 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.258 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.259 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.261 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.262 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:32.262 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:32.262 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.262 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.262 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.262 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:32.262 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:32.263 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.263 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.263 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:32.263 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:32.263 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.263 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.266 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:32.269 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.269 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.269 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:32.269 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:32.269 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.269 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.269 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.269 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.269 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.270 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:32.270 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:32.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.270 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.270 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.270 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:32.270 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.270 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.270 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:32.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.271 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.271 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.271 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:32.271 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.271 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.271 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:32.271 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:32.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f142d7f4640 1 -- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f141403be00 msgr2=0x7f141403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f142d7f4640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f141403be00 0x7f141403e2c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f14280aceb0 tx=0x7f14200a7000 comp rx=0 tx=0).stop 2024-08-19T10:49:32.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f142d7f4640 10 monclient: shutdown 2024-08-19T10:49:32.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f142d7f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:32.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f142d7f4640 1 -- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f14280a6f00 msgr2=0x7f14281d15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f142d7f4640 1 --2- 172.21.15.89:0/2677962452 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f14280a6f00 0x7f14281d15d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f141c00c480 tx=0x7f141c00c940 comp rx=0 tx=0).stop 2024-08-19T10:49:32.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f14277fe640 1 -- 172.21.15.89:0/2677962452 reap_dead start 2024-08-19T10:49:32.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f14277fe640 1 -- 172.21.15.89:0/2677962452 reap_dead start 2024-08-19T10:49:32.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f142d7f4640 1 -- 172.21.15.89:0/2677962452 shutdown_connections 2024-08-19T10:49:32.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f142d7f4640 1 -- 172.21.15.89:0/2677962452 >> 172.21.15.89:0/2677962452 conn(0x7f1428069a80 msgr2=0x7f14280aaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:32.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f142d7f4640 1 -- 172.21.15.89:0/2677962452 shutdown_connections 2024-08-19T10:49:32.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.234+0000 7f142d7f4640 1 -- 172.21.15.89:0/2677962452 wait complete. 2024-08-19T10:49:32.274 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:49:32.274 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-08-19T10:49:32.275 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:49:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.386+0000 7ff7333a2640 10 monclient: get_monmap_and_config 2024-08-19T10:49:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.386+0000 7ff7333a2640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.386+0000 7ff7333a2640 10 monclient: monmap: 2024-08-19T10:49:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:32.392746+0000 2024-08-19T10:49:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:32.392746+0000 2024-08-19T10:49:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:32.388 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.389 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.389 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.389 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.389 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.389 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.389 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.389 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:32.389 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:32.389 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 1 Processor -- start 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 1 -- start start 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 10 monclient: init 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 10 monclient: _add_conns ranks=[10,4,9,2,7,3,8,5,0,1,6] 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff72c0a5270 0x7ff72c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 10 monclient(hunting): picked mon.noname-f con 0x7ff72c0a5270 addr v2:172.21.15.190:3300/0 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c0a6d30 0x7ff72c0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 10 monclient(hunting): picked mon.noname-c con 0x7ff72c0a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff72c1395b0 0x7ff72c139a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 10 monclient(hunting): picked mon.noname-a con 0x7ff72c1395b0 addr v2:172.21.15.89:3300/0 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff72c0a7650 con 0x7ff72c1395b0 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff72c0a8710 con 0x7ff72c0a5270 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff72c0a5a40 con 0x7ff72c0a6d30 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff7333a2640 10 monclient(hunting): authenticate will time out at 1622.209693s 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73093d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c0a6d30 0x7ff72c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff72c1395b0 0x7ff72c139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73093d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c0a6d30 0x7ff72c0a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44442/0 (socket says 172.21.15.89:44442) 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff72c1395b0 0x7ff72c139a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35120/0 (socket says 172.21.15.89:35120) 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73093d640 1 -- 172.21.15.89:0/3180875706 learned_addr learned my addr 172.21.15.89:0/3180875706 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73093d640 10 monclient(hunting): get_auth_request con 0x7ff72c0a6d30 auth_method 0 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73093d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73093d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73093d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 10 monclient(hunting): get_auth_request con 0x7ff72c1395b0 auth_method 0 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73113e640 1 --2- 172.21.15.89:0/3180875706 >> v2:172.21.15.190:3300/0 conn(0x7ff72c0a5270 0x7ff72c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73113e640 10 monclient(hunting): get_auth_request con 0x7ff72c0a5270 auth_method 0 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73113e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73113e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73113e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 10 monclient(hunting): handle_auth_done global_id 7029 payload 274 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 1 -- 172.21.15.89:0/3180875706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c0a6d30 msgr2=0x7ff72c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 1 --2- 172.21.15.89:0/3180875706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c0a6d30 0x7ff72c0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 1 -- 172.21.15.89:0/3180875706 >> v2:172.21.15.190:3300/0 conn(0x7ff72c0a5270 msgr2=0x7ff72c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 1 --2- 172.21.15.89:0/3180875706 >> v2:172.21.15.190:3300/0 conn(0x7ff72c0a5270 0x7ff72c0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 1 monclient: found mon.noname-a 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:32.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73193f640 1 -- 172.21.15.89:0/3180875706 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c0a81d0 con 0x7ff72c1395b0 2024-08-19T10:49:32.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73093d640 1 --2- 172.21.15.89:0/3180875706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c0a6d30 0x7ff72c0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:32.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.390+0000 7ff73113e640 1 --2- 172.21.15.89:0/3180875706 >> v2:172.21.15.190:3300/0 conn(0x7ff72c0a5270 0x7ff72c0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:49:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff73193f640 1 -- 172.21.15.89:0/3180875706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c1395b0 msgr2=0x7ff72c139a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff73193f640 1 --2- 172.21.15.89:0/3180875706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c1395b0 0x7ff72c139a10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0aceb0 tx=0x7ff71c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8146e1e8a41996db server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 1 -- 172.21.15.89:0/3180875706 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff71c0b0070 con 0x7ff72c1395b0 2024-08-19T10:49:32.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 10 monclient: dump: 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 1 -- 172.21.15.89:0/3180875706 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff71c0aa810 con 0x7ff72c1395b0 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 1 -- 172.21.15.89:0/3180875706 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff71c0a9300 con 0x7ff72c1395b0 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 5 monclient: authenticate success, global_id 7029 2024-08-19T10:49:32.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 4 set_mon_vals no callback set 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff72bfff640 10 monclient: dump: 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 10 monclient: shutdown 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 1 -- 172.21.15.89:0/3180875706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c1395b0 msgr2=0x7ff72c139a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 1 --2- 172.21.15.89:0/3180875706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c1395b0 0x7ff72c139a10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0aceb0 tx=0x7ff71c01c340 comp rx=0 tx=0).stop 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 1 -- 172.21.15.89:0/3180875706 shutdown_connections 2024-08-19T10:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 1 --2- 172.21.15.89:0/3180875706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff72c1395b0 0x7ff72c139a10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 1 --2- 172.21.15.89:0/3180875706 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff72c0a6d30 0x7ff72c0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 1 --2- 172.21.15.89:0/3180875706 >> v2:172.21.15.190:3300/0 conn(0x7ff72c0a5270 0x7ff72c0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 1 -- 172.21.15.89:0/3180875706 >> 172.21.15.89:0/3180875706 conn(0x7ff72c069a80 msgr2=0x7ff72c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 1 -- 172.21.15.89:0/3180875706 shutdown_connections 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.394+0000 7ff7333a2640 1 -- 172.21.15.89:0/3180875706 wait complete. 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient: monmap: 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.404 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 1 Processor -- start 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 1 -- start start 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient: init 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient: _add_conns ranks=[6,8,1,0,7,2,5,3,4] 2024-08-19T10:49:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c0a5270 0x7ff72c13ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient(hunting): picked mon.noname-d con 0x7ff72c0a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:49:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff72c0a6d30 0x7ff72c13cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient(hunting): picked mon.noname-e con 0x7ff72c0a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff72c1395b0 0x7ff72c13d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient(hunting): picked mon.noname-f con 0x7ff72c1395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c0a5650 con 0x7ff72c1395b0 2024-08-19T10:49:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff72c13a360 con 0x7ff72c0a5270 2024-08-19T10:49:32.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff72c0a57d0 con 0x7ff72c0a6d30 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff7333a2640 10 monclient(hunting): authenticate will time out at 1622.217733s 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c0a5270 0x7ff72c13ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73093d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff72c0a6d30 0x7ff72c13cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c0a5270 0x7ff72c13ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:44988/0 (socket says 172.21.15.89:44988) 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 1 -- 172.21.15.89:0/2921445246 learned_addr learned my addr 172.21.15.89:0/2921445246 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 10 monclient(hunting): get_auth_request con 0x7ff72c0a5270 auth_method 0 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73093d640 10 monclient(hunting): get_auth_request con 0x7ff72c0a6d30 auth_method 0 2024-08-19T10:49:32.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73093d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73093d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73093d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73193f640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff72c1395b0 0x7ff72c13d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73193f640 10 monclient(hunting): get_auth_request con 0x7ff72c1395b0 auth_method 0 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73193f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73193f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73193f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 10 monclient(hunting): handle_auth_done global_id 44835 payload 824 2024-08-19T10:49:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 1 -- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff72c0a6d30 msgr2=0x7ff72c13cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff72c0a6d30 0x7ff72c13cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 1 -- 172.21.15.89:0/2921445246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff72c1395b0 msgr2=0x7ff72c13d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff72c1395b0 0x7ff72c13d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 1 monclient: found mon.noname-d 2024-08-19T10:49:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:49:32.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.398+0000 7ff73113e640 1 -- 172.21.15.89:0/2921445246 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c135800 con 0x7ff72c0a5270 2024-08-19T10:49:32.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73113e640 1 -- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c0a5270 msgr2=0x7ff72c13ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73113e640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c0a5270 0x7ff72c13ca70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff724009fa0 tx=0x7ff72400d560 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff72401a070 con 0x7ff72c0a5270 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 10 monclient: dump: 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff724003e10 con 0x7ff72c0a5270 2024-08-19T10:49:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff7333a2640 5 monclient: authenticate success, global_id 44835 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff7333a2640 10 monclient: _renew_subs 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff7333a2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff7333a2640 1 -- 172.21.15.89:0/2921445246 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72c13e4d0 con 0x7ff72c0a5270 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff7240044a0 con 0x7ff72c0a5270 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 10 monclient: dump: 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.416 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff7333a2640 10 monclient: _renew_subs 2024-08-19T10:49:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff72b7fe640 4 set_mon_vals no callback set 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff7333a2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff7333a2640 1 -- 172.21.15.89:0/2921445246 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72c09d110 con 0x7ff72c0a5270 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff7333a2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff7333a2640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1040 0x7ff72c1d1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff7333a2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff7333a2640 1 -- 172.21.15.89:0/2921445246 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff72c09c9e0 con 0x7ff72c1d1040 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73093d640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1040 0x7ff72c1d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73093d640 10 monclient: get_auth_request con 0x7ff72c1d1040 auth_method 0 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73093d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73093d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73093d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff724004680 con 0x7ff72c0a5270 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73093d640 10 monclient(hunting): handle_auth_done global_id 44844 payload 824 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73093d640 1 -- 172.21.15.89:0/2921445246 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff72c0a7110 con 0x7ff72c1d1040 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff71803be00 0x7ff71803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7ff72404f7d0 con 0x7ff72c0a5270 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73193f640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff71803be00 0x7ff71803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.402+0000 7ff73193f640 10 monclient: get_auth_request con 0x7ff71803be00 auth_method 0 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff73093d640 1 -- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1040 msgr2=0x7ff72c1d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff73093d640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1040 0x7ff72c1d1420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff72c13eb40 tx=0x7ff720011e30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff72000fd60 con 0x7ff72c1d1040 2024-08-19T10:49:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff729ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff729ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff729ffb640 10 monclient: dump: 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.420 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff72000fd60 con 0x7ff72c1d1040 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff729ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff729ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1040 msgr2=0x7ff72c1d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff729ffb640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1040 0x7ff72c1d1420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff72c13eb40 tx=0x7ff720011e30 comp rx=0 tx=0).stop 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1040 msgr2=0x7ff72c1d1420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff729ffb640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1040 0x7ff72c1d1420 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff72c13eb40 tx=0x7ff720011e30 comp rx=0 tx=0).stop 2024-08-19T10:49:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.406+0000 7ff73193f640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff71803be00 0x7ff71803e2c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0a9960 tx=0x7ff71c0a4f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff7333a2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff7333a2640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1960 0x7ff72c132750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff7333a2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff7333a2640 1 -- 172.21.15.89:0/2921445246 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff72c09d630 con 0x7ff72c1d1960 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff73093d640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1960 0x7ff72c132750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff73093d640 10 monclient: get_auth_request con 0x7ff72c1d1960 auth_method 0 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff73093d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff73093d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff73093d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff73093d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff73093d640 10 monclient(hunting): handle_auth_done global_id 44853 payload 824 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.410+0000 7ff73093d640 1 -- 172.21.15.89:0/2921445246 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff72c064590 con 0x7ff72c1d1960 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff73093d640 1 -- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1960 msgr2=0x7ff72c132750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff73093d640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1960 0x7ff72c132750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff72c13eb40 tx=0x7ff7200046d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff7200229e0 con 0x7ff72c1d1960 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff729ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff729ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff729ffb640 10 monclient: dump: 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9257 (secure 0 0 0) 0x7ff720022570 con 0x7ff72c1d1960 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff729ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff729ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:32.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1960 msgr2=0x7ff72c132750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff729ffb640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1960 0x7ff72c132750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff72c13eb40 tx=0x7ff7200046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff729ffb640 1 -- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1960 msgr2=0x7ff72c132750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff729ffb640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c1d1960 0x7ff72c132750 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff72c13eb40 tx=0x7ff7200046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.425 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:32.425 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:32.425 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:32.425 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:32.425 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 102, 2024-08-19T10:49:32.426 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 295540, 2024-08-19T10:49:32.426 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:32.426 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:32.426 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:32.426 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:32.426 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:32.426 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:32.426 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:32.426 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:32.426 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:32.426 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.427 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.428 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.429 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.430 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:32.431 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.433 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.434 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.435 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:32.436 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.436 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.436 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:32.436 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.436 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.436 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.438 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.439 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.440 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.441 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.442 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:32.442 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.442 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.442 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.442 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:32.442 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:32.442 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.442 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.444 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.445 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:32.447 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.448 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:32.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.449 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.449 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.449 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:49:32.449 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.449 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.449 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:32.449 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff7333a2640 1 -- 172.21.15.89:0/2921445246 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff71803be00 msgr2=0x7ff71803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff7333a2640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff71803be00 0x7ff71803e2c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0a9960 tx=0x7ff71c0a4f90 comp rx=0 tx=0).stop 2024-08-19T10:49:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff7333a2640 10 monclient: shutdown 2024-08-19T10:49:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff7333a2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff7333a2640 1 -- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c0a5270 msgr2=0x7ff72c13ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff7333a2640 1 --2- 172.21.15.89:0/2921445246 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff72c0a5270 0x7ff72c13ca70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff724009fa0 tx=0x7ff72400d560 comp rx=0 tx=0).stop 2024-08-19T10:49:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff73193f640 1 -- 172.21.15.89:0/2921445246 reap_dead start 2024-08-19T10:49:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff73193f640 1 -- 172.21.15.89:0/2921445246 reap_dead start 2024-08-19T10:49:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff7333a2640 1 -- 172.21.15.89:0/2921445246 shutdown_connections 2024-08-19T10:49:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff7333a2640 1 -- 172.21.15.89:0/2921445246 >> 172.21.15.89:0/2921445246 conn(0x7ff72c069a80 msgr2=0x7ff72c0a9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff7333a2640 1 -- 172.21.15.89:0/2921445246 shutdown_connections 2024-08-19T10:49:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.414+0000 7ff7333a2640 1 -- 172.21.15.89:0/2921445246 wait complete. 2024-08-19T10:49:32.452 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:49:32.452 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-08-19T10:49:32.452 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:49:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient: get_monmap_and_config 2024-08-19T10:49:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient: monmap: 2024-08-19T10:49:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:32.577806+0000 2024-08-19T10:49:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:32.577806+0000 2024-08-19T10:49:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:32.574 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 1 Processor -- start 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 1 -- start start 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient: init 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient: _add_conns ranks=[7,9,8,2,10,6,3,0,1,5,4] 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ee40a6860 0x7f6ee40a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient(hunting): picked mon.noname-i con 0x7f6ee40a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6ee40a54d0 0x7f6ee40a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient(hunting): picked mon.noname-a con 0x7f6ee40a54d0 addr v2:172.21.15.89:3300/0 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a58b0 0x7f6ee40a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient(hunting): picked mon.noname-e con 0x7f6ee40a58b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee40a76c0 con 0x7f6ee40a54d0 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6ee40a4c90 con 0x7f6ee40a6860 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6ee4139620 con 0x7f6ee40a58b0 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6eeb109640 10 monclient(hunting): authenticate will time out at 1622.394391s 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6ee40a54d0 0x7f6ee40a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee96a6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a58b0 0x7f6ee40a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee96a6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a58b0 0x7f6ee40a7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48638/0 (socket says 172.21.15.89:48638) 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6ee40a54d0 0x7f6ee40a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35126/0 (socket says 172.21.15.89:35126) 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee96a6640 1 -- 172.21.15.89:0/1461265278 learned_addr learned my addr 172.21.15.89:0/1461265278 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee96a6640 10 monclient(hunting): get_auth_request con 0x7f6ee40a58b0 auth_method 0 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee96a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee96a6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee96a6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 10 monclient(hunting): get_auth_request con 0x7f6ee40a54d0 auth_method 0 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee96a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee96a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee96a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee8ea5640 1 --2- 172.21.15.89:0/1461265278 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ee40a6860 0x7f6ee40a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_done global_id 7038 payload 274 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 1 -- 172.21.15.89:0/1461265278 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a58b0 msgr2=0x7f6ee40a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 1 --2- 172.21.15.89:0/1461265278 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a58b0 0x7f6ee40a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 1 -- 172.21.15.89:0/1461265278 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ee40a6860 msgr2=0x7f6ee40a81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 1 --2- 172.21.15.89:0/1461265278 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ee40a6860 0x7f6ee40a81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 1 monclient: found mon.noname-a 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee3fff640 1 -- 172.21.15.89:0/1461265278 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee4139820 con 0x7f6ee40a54d0 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee96a6640 1 --2- 172.21.15.89:0/1461265278 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a58b0 0x7f6ee40a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee8ea5640 10 monclient: get_auth_request con 0x7f6ee40a6860 auth_method 0 2024-08-19T10:49:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.574+0000 7f6ee8ea5640 1 --2- 172.21.15.89:0/1461265278 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ee40a6860 0x7f6ee40a81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee3fff640 1 -- 172.21.15.89:0/1461265278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee40a54d0 msgr2=0x7f6ee40a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee3fff640 1 --2- 172.21.15.89:0/1461265278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee40a54d0 0x7f6ee40a6c40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0097c0 tx=0x7f6ecc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59947636dda5b52 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 1 -- 172.21.15.89:0/1461265278 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ecc0b0070 con 0x7f6ee40a54d0 2024-08-19T10:49:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 10 monclient: dump: 2024-08-19T10:49:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6eeb109640 5 monclient: authenticate success, global_id 7038 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 1 -- 172.21.15.89:0/1461265278 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ecc0a9980 con 0x7f6ee40a54d0 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6eeb109640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6eeb109640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6eeb109640 4 set_mon_vals no callback set 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 1 -- 172.21.15.89:0/1461265278 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ecc0aa3a0 con 0x7f6ee40a54d0 2024-08-19T10:49:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6ee37fe640 10 monclient: dump: 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6eeb109640 10 monclient: shutdown 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6eeb109640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6eeb109640 1 -- 172.21.15.89:0/1461265278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee40a54d0 msgr2=0x7f6ee40a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.578+0000 7f6eeb109640 1 --2- 172.21.15.89:0/1461265278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee40a54d0 0x7f6ee40a6c40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0097c0 tx=0x7f6ecc01c3f0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 -- 172.21.15.89:0/1461265278 shutdown_connections 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 --2- 172.21.15.89:0/1461265278 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a58b0 0x7f6ee40a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 --2- 172.21.15.89:0/1461265278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ee40a54d0 0x7f6ee40a6c40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 --2- 172.21.15.89:0/1461265278 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6ee40a6860 0x7f6ee40a81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 -- 172.21.15.89:0/1461265278 >> 172.21.15.89:0/1461265278 conn(0x7f6ee4069a00 msgr2=0x7f6ee40abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 -- 172.21.15.89:0/1461265278 shutdown_connections 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 -- 172.21.15.89:0/1461265278 wait complete. 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient: monmap: 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:32.592 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.593 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.593 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.593 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.593 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.593 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.593 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.593 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 Processor -- start 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 -- start start 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient: init 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient: _add_conns ranks=[4,5,8,0,1,2,3,6,7] 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ee40a54d0 0x7f6ee40a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient(hunting): picked mon.noname-c con 0x7f6ee40a54d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ee40a58b0 0x7f6ee40a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient(hunting): picked mon.noname-h con 0x7f6ee40a58b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a6860 0x7f6ee41d1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient(hunting): picked mon.noname-e con 0x7f6ee40a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6ee4143490 con 0x7f6ee40a54d0 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6ee4143310 con 0x7f6ee40a58b0 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6ee4143610 con 0x7f6ee40a6860 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6eeb109640 10 monclient(hunting): authenticate will time out at 1622.401540s 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ee40a54d0 0x7f6ee40a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee96a6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a6860 0x7f6ee41d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ee40a54d0 0x7f6ee40a1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44454/0 (socket says 172.21.15.89:44454) 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 1 -- 172.21.15.89:0/2621757522 learned_addr learned my addr 172.21.15.89:0/2621757522 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 10 monclient(hunting): get_auth_request con 0x7f6ee40a54d0 auth_method 0 2024-08-19T10:49:32.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee96a6640 10 monclient(hunting): get_auth_request con 0x7f6ee40a6860 auth_method 0 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee96a6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee96a6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee96a6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee96a6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee96a6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee96a6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee3fff640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ee40a58b0 0x7f6ee40a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 10 monclient(hunting): handle_auth_done global_id 44869 payload 824 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 1 -- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a6860 msgr2=0x7f6ee41d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a6860 0x7f6ee41d1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 1 -- 172.21.15.89:0/2621757522 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ee40a58b0 msgr2=0x7f6ee40a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ee40a58b0 0x7f6ee40a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 1 monclient: found mon.noname-c 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee8ea5640 1 -- 172.21.15.89:0/2621757522 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee409f500 con 0x7f6ee40a54d0 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee3fff640 10 monclient: get_auth_request con 0x7f6ee40a58b0 auth_method 0 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee3fff640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6ee40a58b0 0x7f6ee40a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.582+0000 7f6ee96a6640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee40a6860 0x7f6ee41d1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:32.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee8ea5640 1 -- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ee40a54d0 msgr2=0x7f6ee40a1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee8ea5640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ee40a54d0 0x7f6ee40a1c00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00ce00 tx=0x7f6edc00a650 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6edc00ae20 con 0x7f6ee40a54d0 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 10 monclient: dump: 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.598 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6edc00fd10 con 0x7f6ee40a54d0 2024-08-19T10:49:32.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6eeb109640 5 monclient: authenticate success, global_id 44869 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6eeb109640 10 monclient: _renew_subs 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6eeb109640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6eeb109640 1 -- 172.21.15.89:0/2621757522 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee409dfc0 con 0x7f6ee40a54d0 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6edc018870 con 0x7f6ee40a54d0 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 10 monclient: dump: 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.600 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6eeb109640 10 monclient: _renew_subs 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6eeb109640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:49:32.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee2ffd640 4 set_mon_vals no callback set 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6eeb109640 1 -- 172.21.15.89:0/2621757522 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee409ef20 con 0x7f6ee40a54d0 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6eeb109640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6eeb109640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee41d1c50 0x7f6ee41d2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6eeb109640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6eeb109640 1 -- 172.21.15.89:0/2621757522 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6ee41d2570 con 0x7f6ee41d1c50 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee3fff640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee41d1c50 0x7f6ee41d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee3fff640 10 monclient: get_auth_request con 0x7f6ee41d1c50 auth_method 0 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6edc018400 con 0x7f6ee40a54d0 2024-08-19T10:49:32.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_done global_id 6902 payload 824 2024-08-19T10:49:32.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee3fff640 1 -- 172.21.15.89:0/2621757522 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ee40a81b0 con 0x7f6ee41d1c50 2024-08-19T10:49:32.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ec803be00 0x7f6ec803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f6edc050210 con 0x7f6ee40a54d0 2024-08-19T10:49:32.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee96a6640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ec803be00 0x7f6ec803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.586+0000 7f6ee96a6640 10 monclient: get_auth_request con 0x7f6ec803be00 auth_method 0 2024-08-19T10:49:32.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee3fff640 1 -- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee41d1c50 msgr2=0x7f6ee41d2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee3fff640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee41d1c50 0x7f6ee41d2030 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0ad000 tx=0x7f6ecc019190 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ecc0b9070 con 0x7f6ee41d1c50 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee17fa640 10 monclient: dump: 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.606 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6ecc003b20 con 0x7f6ee41d1c50 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee17fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee41d1c50 msgr2=0x7f6ee41d2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee17fa640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee41d1c50 0x7f6ee41d2030 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0ad000 tx=0x7f6ecc019190 comp rx=0 tx=0).stop 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee41d1c50 msgr2=0x7f6ee41d2030 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee17fa640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee41d1c50 0x7f6ee41d2030 secure :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0ad000 tx=0x7f6ecc019190 comp rx=0 tx=0).stop 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.590+0000 7f6ee96a6640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ec803be00 0x7f6ec803e2c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009630 tx=0x7f6ed4009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6eeb109640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6eeb109640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee413ab90 0x7f6ee413afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6eeb109640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6eeb109640 1 -- 172.21.15.89:0/2621757522 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6ee4064590 con 0x7f6ee413ab90 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6ee3fff640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee413ab90 0x7f6ee413afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6ee3fff640 10 monclient: get_auth_request con 0x7f6ee413ab90 auth_method 0 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6ee3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6ee3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6ee3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6ee3fff640 10 monclient(hunting): handle_auth_done global_id 6911 payload 824 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.594+0000 7f6ee3fff640 1 -- 172.21.15.89:0/2621757522 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6ee4140650 con 0x7f6ee413ab90 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee3fff640 1 -- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee413ab90 msgr2=0x7f6ee413afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee3fff640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee413ab90 0x7f6ee413afa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009710 tx=0x7f6ecc0a8000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ecc0b0040 con 0x7f6ee413ab90 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee17fa640 10 monclient: dump: 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.609 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9424 (secure 0 0 0) 0x7f6ecc0c2070 con 0x7f6ee413ab90 2024-08-19T10:49:32.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:32.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee17fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:32.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee413ab90 msgr2=0x7f6ee413afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee17fa640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee413ab90 0x7f6ee413afa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009710 tx=0x7f6ecc0a8000 comp rx=0 tx=0).stop 2024-08-19T10:49:32.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee17fa640 1 -- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee413ab90 msgr2=0x7f6ee413afa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee17fa640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6ee413ab90 0x7f6ee413afa0 secure :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009710 tx=0x7f6ecc0a8000 comp rx=0 tx=0).stop 2024-08-19T10:49:32.611 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 102, 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 702391, 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.614 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:32.615 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.616 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:32.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.618 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:32.620 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.621 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.621 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.621 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:32.621 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:32.621 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.621 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.621 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.621 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.621 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:32.622 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:32.622 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.622 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.622 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.622 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:32.622 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.622 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.622 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.622 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:32.623 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:32.624 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.624 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.624 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.624 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:32.624 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.624 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.624 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.624 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:32.624 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:32.625 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.626 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.627 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:32.627 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:32.627 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.627 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.627 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.627 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.627 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:32.627 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.628 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.629 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.630 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.631 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.632 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.633 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.634 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.635 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:32.636 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6eeb109640 1 -- 172.21.15.89:0/2621757522 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ec803be00 msgr2=0x7f6ec803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6eeb109640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ec803be00 0x7f6ec803e2c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009630 tx=0x7f6ed4009240 comp rx=0 tx=0).stop 2024-08-19T10:49:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6eeb109640 10 monclient: shutdown 2024-08-19T10:49:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6eeb109640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6eeb109640 1 -- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ee40a54d0 msgr2=0x7f6ee40a1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6eeb109640 1 --2- 172.21.15.89:0/2621757522 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6ee40a54d0 0x7f6ee40a1c00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00ce00 tx=0x7f6edc00a650 comp rx=0 tx=0).stop 2024-08-19T10:49:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee96a6640 1 -- 172.21.15.89:0/2621757522 reap_dead start 2024-08-19T10:49:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6ee96a6640 1 -- 172.21.15.89:0/2621757522 reap_dead start 2024-08-19T10:49:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6eeb109640 1 -- 172.21.15.89:0/2621757522 shutdown_connections 2024-08-19T10:49:32.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6eeb109640 1 -- 172.21.15.89:0/2621757522 >> 172.21.15.89:0/2621757522 conn(0x7f6ee4069a00 msgr2=0x7f6ee40a9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6eeb109640 1 -- 172.21.15.89:0/2621757522 shutdown_connections 2024-08-19T10:49:32.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.598+0000 7f6eeb109640 1 -- 172.21.15.89:0/2621757522 wait complete. 2024-08-19T10:49:32.639 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:49:32.639 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-08-19T10:49:32.639 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:49:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.762+0000 7fb0dae90640 10 monclient: get_monmap_and_config 2024-08-19T10:49:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.762+0000 7fb0dae90640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.762+0000 7fb0dae90640 10 monclient: monmap: 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:32.768459+0000 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:32.768459+0000 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:32.768 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 1 Processor -- start 2024-08-19T10:49:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 1 -- start start 2024-08-19T10:49:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 10 monclient: init 2024-08-19T10:49:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 10 monclient: _add_conns ranks=[3,5,2,9,1,6,7,10,4,0,8] 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0d40a5270 0x7fb0d40a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 10 monclient(hunting): picked mon.noname-g con 0x7fb0d40a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0d40a6d30 0x7fb0d40a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 10 monclient(hunting): picked mon.noname-h con 0x7fb0d40a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d41395b0 0x7fb0d41399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 10 monclient(hunting): picked mon.noname-b con 0x7fb0d41395b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d40a7650 con 0x7fb0d41395b0 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d40a8710 con 0x7fb0d40a5270 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fb0d40a5a40 con 0x7fb0d40a6d30 2024-08-19T10:49:32.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0dae90640 10 monclient(hunting): authenticate will time out at 1622.585037s 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d942d640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d41395b0 msgr2=0x7fb0d41399d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d942d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d41395b0 0x7fb0d41399d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d8c2c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0d40a5270 0x7fb0d40a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0d40a6d30 0x7fb0d40a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0d40a6d30 0x7fb0d40a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:48242/0 (socket says 172.21.15.89:48242) 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 1 -- 172.21.15.89:0/1597679730 learned_addr learned my addr 172.21.15.89:0/1597679730 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 10 monclient(hunting): get_auth_request con 0x7fb0d40a6d30 auth_method 0 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d8c2c640 10 monclient(hunting): get_auth_request con 0x7fb0d40a5270 auth_method 0 2024-08-19T10:49:32.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d8c2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d8c2c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d8c2c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 10 monclient(hunting): handle_auth_done global_id 6971 payload 274 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 1 -- 172.21.15.89:0/1597679730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0d40a5270 msgr2=0x7fb0d40a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 1 --2- 172.21.15.89:0/1597679730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0d40a5270 0x7fb0d40a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 1 -- 172.21.15.89:0/1597679730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d41395b0 msgr2=0x7fb0d41399d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 1 --2- 172.21.15.89:0/1597679730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d41395b0 0x7fb0d41399d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 1 monclient: found mon.noname-h 2024-08-19T10:49:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:49:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.766+0000 7fb0d3fff640 1 -- 172.21.15.89:0/1597679730 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d40a81d0 con 0x7fb0d40a6d30 2024-08-19T10:49:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d3fff640 1 -- 172.21.15.89:0/1597679730 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0d40a6d30 msgr2=0x7fb0d40a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d3fff640 1 --2- 172.21.15.89:0/1597679730 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0d40a6d30 0x7fb0d40a7110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00098a0 tx=0x7fb0c001c460 comp rx=0 tx=0).ready entity=mon.5 client_cookie=f54a75528192ef17 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 1 -- 172.21.15.89:0/1597679730 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0c00b0070 con 0x7fb0d40a6d30 2024-08-19T10:49:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 10 monclient: dump: 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.777 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.778 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 1 -- 172.21.15.89:0/1597679730 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c00aa810 con 0x7fb0d40a6d30 2024-08-19T10:49:32.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 1 -- 172.21.15.89:0/1597679730 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0c00aae30 con 0x7fb0d40a6d30 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0dae90640 5 monclient: authenticate success, global_id 6971 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0dae90640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0dae90640 4 set_mon_vals no callback set 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.770+0000 7fb0d37fe640 10 monclient: dump: 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.779 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient: shutdown 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:32.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 -- 172.21.15.89:0/1597679730 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0d40a6d30 msgr2=0x7fb0d40a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 --2- 172.21.15.89:0/1597679730 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0d40a6d30 0x7fb0d40a7110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00098a0 tx=0x7fb0c001c460 comp rx=0 tx=0).stop 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 -- 172.21.15.89:0/1597679730 shutdown_connections 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 --2- 172.21.15.89:0/1597679730 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fb0d40a6d30 0x7fb0d40a7110 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 --2- 172.21.15.89:0/1597679730 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb0d41395b0 0x7fb0d41399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 --2- 172.21.15.89:0/1597679730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0d40a5270 0x7fb0d40a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 -- 172.21.15.89:0/1597679730 >> 172.21.15.89:0/1597679730 conn(0x7fb0d4069a80 msgr2=0x7fb0d40abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 -- 172.21.15.89:0/1597679730 shutdown_connections 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 -- 172.21.15.89:0/1597679730 wait complete. 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient: monmap: 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:32.781 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.782 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 Processor -- start 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 -- start start 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient: init 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient: _add_conns ranks=[0,8,4,6,1,3,7,2,5] 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0d40a5270 0x7fb0d413db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient(hunting): picked mon.noname-a con 0x7fb0d40a5270 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0d40a6d30 0x7fb0d413e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient(hunting): picked mon.noname-e con 0x7fb0d40a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d41395b0 0x7fb0d41d1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient(hunting): picked mon.noname-c con 0x7fb0d41395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d40a57d0 con 0x7fb0d40a5270 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb0d40a5650 con 0x7fb0d41395b0 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb0d4143230 con 0x7fb0d40a6d30 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0dae90640 10 monclient(hunting): authenticate will time out at 1622.594327s 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0d40a6d30 0x7fb0d413e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0d40a6d30 0x7fb0d413e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48672/0 (socket says 172.21.15.89:48672) 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 1 -- 172.21.15.89:0/1802737691 learned_addr learned my addr 172.21.15.89:0/1802737691 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d942d640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d41395b0 0x7fb0d41d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 10 monclient(hunting): get_auth_request con 0x7fb0d40a6d30 auth_method 0 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d8c2c640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0d40a5270 0x7fb0d413db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d942d640 10 monclient(hunting): get_auth_request con 0x7fb0d41395b0 auth_method 0 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d942d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d942d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d942d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d8c2c640 10 monclient(hunting): get_auth_request con 0x7fb0d40a5270 auth_method 0 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d8c2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d8c2c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d8c2c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d942d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d942d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d942d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 10 monclient(hunting): handle_auth_done global_id 6920 payload 824 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 1 -- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d41395b0 msgr2=0x7fb0d41d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d41395b0 0x7fb0d41d1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 1 -- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0d40a5270 msgr2=0x7fb0d413db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0d40a5270 0x7fb0d413db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 1 monclient: found mon.noname-e 2024-08-19T10:49:32.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:32.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d3fff640 1 -- 172.21.15.89:0/1802737691 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d409d830 con 0x7fb0d40a6d30 2024-08-19T10:49:32.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d942d640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb0d41395b0 0x7fb0d41d1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:32.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.774+0000 7fb0d8c2c640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0d40a5270 0x7fb0d413db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:49:32.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d3fff640 1 -- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0d40a6d30 msgr2=0x7fb0d413e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d3fff640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0d40a6d30 0x7fb0d413e0a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00097c0 tx=0x7fb0c0019200 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0c00b0070 con 0x7fb0d40a6d30 2024-08-19T10:49:32.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 10 monclient: dump: 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.792 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c00b1040 con 0x7fb0d40a6d30 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0dae90640 5 monclient: authenticate success, global_id 6920 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0dae90640 10 monclient: _renew_subs 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0dae90640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0dae90640 1 -- 172.21.15.89:0/1802737691 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d413f300 con 0x7fb0d40a6d30 2024-08-19T10:49:32.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0c0004240 con 0x7fb0d40a6d30 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 10 monclient: dump: 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.794 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0dae90640 10 monclient: _renew_subs 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0dae90640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d27fc640 4 set_mon_vals no callback set 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0dae90640 1 -- 172.21.15.89:0/1802737691 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d409cf90 con 0x7fb0d40a6d30 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0dae90640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0dae90640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d409d1d0 0x7fb0d41d1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0dae90640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0dae90640 1 -- 172.21.15.89:0/1802737691 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4139bd0 con 0x7fb0d409d1d0 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb0c0003620 con 0x7fb0d40a6d30 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d8c2c640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d409d1d0 0x7fb0d41d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0b803be00 0x7fb0b803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d8c2c640 10 monclient: get_auth_request con 0x7fb0d409d1d0 auth_method 0 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d8c2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d8c2c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d8c2c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7fb0c00e8bb0 con 0x7fb0d40a6d30 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d942d640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0b803be00 0x7fb0b803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d942d640 10 monclient: get_auth_request con 0x7fb0b803be00 auth_method 0 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_done global_id 44749 payload 824 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.778+0000 7fb0d8c2c640 1 -- 172.21.15.89:0/1802737691 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0d40a7110 con 0x7fb0d409d1d0 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d8c2c640 1 -- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d409d1d0 msgr2=0x7fb0d41d1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d8c2c640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d409d1d0 0x7fb0d41d1870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8011c50 tx=0x7fb0c800f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d942d640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0b803be00 0x7fb0b803e2c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc003cc0 tx=0x7fb0cc00d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0c8014ac0 con 0x7fb0d409d1d0 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d17fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d17fa640 10 monclient: dump: 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.797 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.798 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb0c8014650 con 0x7fb0d409d1d0 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d17fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d409d1d0 msgr2=0x7fb0d41d1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d17fa640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d409d1d0 0x7fb0d41d1870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8011c50 tx=0x7fb0c800f610 comp rx=0 tx=0).stop 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d409d1d0 msgr2=0x7fb0d41d1870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.782+0000 7fb0d17fa640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d409d1d0 0x7fb0d41d1870 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8011c50 tx=0x7fb0c800f610 comp rx=0 tx=0).stop 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0dae90640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0dae90640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d41d1db0 0x7fb0d4132990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0dae90640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0dae90640 1 -- 172.21.15.89:0/1802737691 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4064590 con 0x7fb0d41d1db0 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0d8c2c640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d41d1db0 0x7fb0d4132990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0d8c2c640 10 monclient: get_auth_request con 0x7fb0d41d1db0 auth_method 0 2024-08-19T10:49:32.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0d8c2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0d8c2c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0d8c2c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0d8c2c640 10 monclient(hunting): handle_auth_done global_id 44758 payload 824 2024-08-19T10:49:32.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.790+0000 7fb0d8c2c640 1 -- 172.21.15.89:0/1802737691 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb0d413fe60 con 0x7fb0d41d1db0 2024-08-19T10:49:32.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d8c2c640 1 -- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d41d1db0 msgr2=0x7fb0d4132990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d8c2c640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d41d1db0 0x7fb0d4132990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8000c10 tx=0x7fb0c80046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb0c801dda0 con 0x7fb0d41d1db0 2024-08-19T10:49:32.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d17fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:49:32.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d17fa640 10 monclient: dump: 2024-08-19T10:49:32.801 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.802 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.803 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.803 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9257 (secure 0 0 0) 0x7fb0c80273f0 con 0x7fb0d41d1db0 2024-08-19T10:49:32.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:32.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d17fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:32.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d41d1db0 msgr2=0x7fb0d4132990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d17fa640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d41d1db0 0x7fb0d4132990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8000c10 tx=0x7fb0c80046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d17fa640 1 -- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d41d1db0 msgr2=0x7fb0d4132990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d17fa640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0d41d1db0 0x7fb0d4132990 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8000c10 tx=0x7fb0c80046d0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.803 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:32.804 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:32.804 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:32.804 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:32.804 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 102, 2024-08-19T10:49:32.804 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 295925, 2024-08-19T10:49:32.804 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:32.804 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:32.804 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:32.804 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:32.804 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:32.804 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.805 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.806 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:32.807 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.808 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.809 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.812 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:32.813 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.814 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.815 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:32.816 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.818 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.819 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.820 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.821 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.822 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.823 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:32.824 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:32.825 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.826 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:32.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:32.827 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:32.827 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:32.827 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:49:32.827 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:32.827 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.827 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:32.827 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:32.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0dae90640 1 -- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0b803be00 msgr2=0x7fb0b803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0dae90640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb0b803be00 0x7fb0b803e2c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc003cc0 tx=0x7fb0cc00d040 comp rx=0 tx=0).stop 2024-08-19T10:49:32.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0dae90640 10 monclient: shutdown 2024-08-19T10:49:32.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0dae90640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:32.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0dae90640 1 -- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0d40a6d30 msgr2=0x7fb0d413e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0dae90640 1 --2- 172.21.15.89:0/1802737691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb0d40a6d30 0x7fb0d413e0a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00097c0 tx=0x7fb0c0019200 comp rx=0 tx=0).stop 2024-08-19T10:49:32.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d942d640 1 -- 172.21.15.89:0/1802737691 reap_dead start 2024-08-19T10:49:32.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0d942d640 1 -- 172.21.15.89:0/1802737691 reap_dead start 2024-08-19T10:49:32.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0dae90640 1 -- 172.21.15.89:0/1802737691 shutdown_connections 2024-08-19T10:49:32.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0dae90640 1 -- 172.21.15.89:0/1802737691 >> 172.21.15.89:0/1802737691 conn(0x7fb0d4069a80 msgr2=0x7fb0d4137b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:32.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0dae90640 1 -- 172.21.15.89:0/1802737691 shutdown_connections 2024-08-19T10:49:32.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.794+0000 7fb0dae90640 1 -- 172.21.15.89:0/1802737691 wait complete. 2024-08-19T10:49:32.830 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:49:32.830 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-08-19T10:49:32.830 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:49:32.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient: get_monmap_and_config 2024-08-19T10:49:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient: monmap: 2024-08-19T10:49:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:32.953408+0000 2024-08-19T10:49:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:32.953408+0000 2024-08-19T10:49:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:32.949 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:32.950 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 1 Processor -- start 2024-08-19T10:49:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 1 -- start start 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient: init 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient: _add_conns ranks=[3,9,1,2,0,7,8,10,5,4,6] 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87900a5270 0x7f87900a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient(hunting): picked mon.noname-g con 0x7f87900a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f87900a6d30 0x7f87900a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient(hunting): picked mon.noname-a con 0x7f87900a6d30 addr v2:172.21.15.89:3300/0 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f87901395b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87900a8710 con 0x7f87901395b0 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f87900a7650 con 0x7f87900a6d30 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f87900a5a40 con 0x7f87900a5270 2024-08-19T10:49:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f8794d24640 10 monclient(hunting): authenticate will time out at 1622.770135s 2024-08-19T10:49:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f87900a6d30 0x7f87900a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f87900a6d30 0x7f87900a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35140/0 (socket says 172.21.15.89:35140) 2024-08-19T10:49:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 1 -- 172.21.15.89:0/1416760419 learned_addr learned my addr 172.21.15.89:0/1416760419 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 10 monclient(hunting): get_auth_request con 0x7f87900a6d30 auth_method 0 2024-08-19T10:49:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878e59c640 1 --2- 172.21.15.89:0/1416760419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87900a5270 0x7f87900a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878e59c640 10 monclient(hunting): get_auth_request con 0x7f87900a5270 auth_method 0 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 10 monclient(hunting): handle_auth_done global_id 7056 payload 274 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 1 -- 172.21.15.89:0/1416760419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87900a5270 msgr2=0x7f87900a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 1 --2- 172.21.15.89:0/1416760419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87900a5270 0x7f87900a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 1 -- 172.21.15.89:0/1416760419 >> v1:172.21.15.190:6789/0 conn(0x7f87901395b0 legacy=0x7f8790139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 1 monclient: found mon.noname-a 2024-08-19T10:49:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:49:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.950+0000 7f878dd9b640 1 -- 172.21.15.89:0/1416760419 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87900a81d0 con 0x7f87900a6d30 2024-08-19T10:49:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878dd9b640 1 -- 172.21.15.89:0/1416760419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87900a6d30 msgr2=0x7f87900a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878dd9b640 1 --2- 172.21.15.89:0/1416760419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87900a6d30 0x7f87900a7110 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8780009e50 tx=0x7f878001c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911b89d0097da683 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 1 -- 172.21.15.89:0/1416760419 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87800090d0 con 0x7f87900a6d30 2024-08-19T10:49:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 10 monclient: dump: 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 1 -- 172.21.15.89:0/1416760419 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87800a59c0 con 0x7f87900a6d30 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 1 -- 172.21.15.89:0/1416760419 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87800ad3f0 con 0x7f87900a6d30 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 5 monclient: authenticate success, global_id 7056 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 4 set_mon_vals no callback set 2024-08-19T10:49:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f878d59a640 10 monclient: dump: 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 10 monclient: shutdown 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 1 -- 172.21.15.89:0/1416760419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87900a6d30 msgr2=0x7f87900a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 1 --2- 172.21.15.89:0/1416760419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87900a6d30 0x7f87900a7110 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8780009e50 tx=0x7f878001c710 comp rx=0 tx=0).stop 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 1 -- 172.21.15.89:0/1416760419 shutdown_connections 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 1 --2- 172.21.15.89:0/1416760419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87900a6d30 0x7f87900a7110 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 1 --2- 172.21.15.89:0/1416760419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87900a5270 0x7f87900a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 1 -- 172.21.15.89:0/1416760419 >> 172.21.15.89:0/1416760419 conn(0x7f8790069a80 msgr2=0x7f87900abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 1 -- 172.21.15.89:0/1416760419 shutdown_connections 2024-08-19T10:49:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.954+0000 7f8794d24640 1 -- 172.21.15.89:0/1416760419 wait complete. 2024-08-19T10:49:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient: build_initial_monmap 2024-08-19T10:49:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient: monmap: 2024-08-19T10:49:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:32.969 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 1 Processor -- start 2024-08-19T10:49:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 1 -- start start 2024-08-19T10:49:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient: init 2024-08-19T10:49:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient: _add_conns ranks=[8,1,5,0,4,6,2,7,3] 2024-08-19T10:49:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f87900a5270 0x7f879013fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient(hunting): picked mon.noname-e con 0x7f87900a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87900a6d30 0x7f8790138480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient(hunting): picked mon.noname-f con 0x7f87900a6d30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:49:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87901395b0 0x7f879013d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient(hunting): picked mon.noname-h con 0x7f87901395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f87900a57d0 con 0x7f87900a6d30 2024-08-19T10:49:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f87900a5650 con 0x7f87901395b0 2024-08-19T10:49:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f8790144240 con 0x7f87900a5270 2024-08-19T10:49:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f8794d24640 10 monclient(hunting): authenticate will time out at 1622.777323s 2024-08-19T10:49:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f87900a5270 0x7f879013fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f87900a5270 0x7f879013fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48682/0 (socket says 172.21.15.89:48682) 2024-08-19T10:49:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 1 -- 172.21.15.89:0/1202605311 learned_addr learned my addr 172.21.15.89:0/1202605311 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 10 monclient(hunting): get_auth_request con 0x7f87900a5270 auth_method 0 2024-08-19T10:49:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878ed9d640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87901395b0 0x7f879013d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878ed9d640 10 monclient(hunting): get_auth_request con 0x7f87901395b0 auth_method 0 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878dd9b640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87900a6d30 0x7f8790138480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878dd9b640 10 monclient(hunting): get_auth_request con 0x7f87900a6d30 auth_method 0 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 10 monclient(hunting): handle_auth_done global_id 6929 payload 824 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 1 -- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87901395b0 msgr2=0x7f879013d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87901395b0 0x7f879013d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 1 -- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87900a6d30 msgr2=0x7f8790138480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87900a6d30 0x7f8790138480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 1 monclient: found mon.noname-e 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.958+0000 7f878e59c640 1 -- 172.21.15.89:0/1202605311 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87901d1f90 con 0x7f87900a5270 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878e59c640 1 -- 172.21.15.89:0/1202605311 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f87900a5270 msgr2=0x7f879013fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878e59c640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f87900a5270 0x7f879013fc80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f877800d5f0 tx=0x7f877800dab0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8778012c70 con 0x7f87900a5270 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 10 monclient: dump: 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 10 monclient: _finish_auth 0 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8778019070 con 0x7f87900a5270 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8778012e30 con 0x7f87900a5270 2024-08-19T10:49:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 10 monclient: dump: 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8794d24640 5 monclient: authenticate success, global_id 6929 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8794d24640 10 monclient: _renew_subs 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8787fff640 4 set_mon_vals no callback set 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8794d24640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8794d24640 1 -- 172.21.15.89:0/1202605311 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8790122e10 con 0x7f87900a5270 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8794d24640 10 monclient: _renew_subs 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8794d24640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8794d24640 1 -- 172.21.15.89:0/1202605311 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f8790140610 con 0x7f87900a5270 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8794d24640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8794d24640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002d1d0 0x7f879002d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8794d24640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8794d24640 1 -- 172.21.15.89:0/1202605311 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f87900a67f0 con 0x7f879002d1d0 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8778012800 con 0x7f87900a5270 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878dd9b640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002d1d0 0x7f879002d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878dd9b640 10 monclient: get_auth_request con 0x7f879002d1d0 auth_method 0 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f877003bd60 0x7f877003e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f877800f040 con 0x7f87900a5270 2024-08-19T10:49:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878dd9b640 10 monclient(hunting): handle_auth_done global_id 35481 payload 824 2024-08-19T10:49:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878ed9d640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f877003bd60 0x7f877003e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878dd9b640 1 -- 172.21.15.89:0/1202605311 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8790064590 con 0x7f879002d1d0 2024-08-19T10:49:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.962+0000 7f878ed9d640 10 monclient: get_auth_request con 0x7f877003bd60 auth_method 0 2024-08-19T10:49:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f878dd9b640 1 -- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002d1d0 msgr2=0x7f879002d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f878dd9b640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002d1d0 0x7f879002d5b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f87800ae000 tx=0x7f87800194b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87800bd030 con 0x7f879002d1d0 2024-08-19T10:49:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f8786ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f8786ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:49:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f8786ffd640 10 monclient: dump: 2024-08-19T10:49:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.984 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f878ed9d640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f877003bd60 0x7f877003e220 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8788004db0 tx=0x7f8788009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8780004540 con 0x7f879002d1d0 2024-08-19T10:49:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f8786ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f8786ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002d1d0 msgr2=0x7f879002d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f8786ffd640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002d1d0 0x7f879002d5b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f87800ae000 tx=0x7f87800194b0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002d1d0 msgr2=0x7f879002d5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.966+0000 7f8786ffd640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002d1d0 0x7f879002d5b0 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f87800ae000 tx=0x7f87800194b0 comp rx=0 tx=0).stop 2024-08-19T10:49:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.970+0000 7f8794d24640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.970+0000 7f8794d24640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002daf0 0x7f879002ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.970+0000 7f8794d24640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.970+0000 7f8794d24640 1 -- 172.21.15.89:0/1202605311 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f879013f120 con 0x7f879002daf0 2024-08-19T10:49:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002daf0 0x7f879002ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 10 monclient: get_auth_request con 0x7f879002daf0 auth_method 0 2024-08-19T10:49:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:32.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:32.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 10 monclient(hunting): handle_auth_done global_id 35490 payload 824 2024-08-19T10:49:32.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 1 -- 172.21.15.89:0/1202605311 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f87900a0880 con 0x7f879002daf0 2024-08-19T10:49:32.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 1 -- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002daf0 msgr2=0x7f879002ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:32.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f878dd9b640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002daf0 0x7f879002ded0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8780000f90 tx=0x7f87800a7000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:32.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87800bd040 con 0x7f879002daf0 2024-08-19T10:49:32.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f8786ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:32.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f8786ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:49:32.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.974+0000 7f8786ffd640 10 monclient: dump: 2024-08-19T10:49:32.988 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:32.988 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:32.988 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.988 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:32.988 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:32.988 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:32.988 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:32.988 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:32.989 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:32.989 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:32.989 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:32.989 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:32.989 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:32.990 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:32.990 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:32.990 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:32.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9424 (secure 0 0 0) 0x7f87800c3a20 con 0x7f879002daf0 2024-08-19T10:49:32.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8786ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:32.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8786ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:32.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002daf0 msgr2=0x7f879002ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:32.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8786ffd640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002daf0 0x7f879002ded0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8780000f90 tx=0x7f87800a7000 comp rx=0 tx=0).stop 2024-08-19T10:49:32.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8786ffd640 1 -- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002daf0 msgr2=0x7f879002ded0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:32.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8786ffd640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f879002daf0 0x7f879002ded0 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8780000f90 tx=0x7f87800a7000 comp rx=0 tx=0).stop 2024-08-19T10:49:32.991 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 102, 2024-08-19T10:49:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 418022, 2024-08-19T10:49:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:49:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.013 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.016 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:33.017 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:33.018 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.019 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.020 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.021 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:33.022 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8794d24640 1 -- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f877003bd60 msgr2=0x7f877003e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8794d24640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f877003bd60 0x7f877003e220 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8788004db0 tx=0x7f8788009040 comp rx=0 tx=0).stop 2024-08-19T10:49:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8794d24640 10 monclient: shutdown 2024-08-19T10:49:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8794d24640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8794d24640 1 -- 172.21.15.89:0/1202605311 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f87900a5270 msgr2=0x7f879013fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8794d24640 1 --2- 172.21.15.89:0/1202605311 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f87900a5270 0x7f879013fc80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f877800d5f0 tx=0x7f877800dab0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f878ed9d640 1 -- 172.21.15.89:0/1202605311 reap_dead start 2024-08-19T10:49:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f878ed9d640 1 -- 172.21.15.89:0/1202605311 reap_dead start 2024-08-19T10:49:33.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8794d24640 1 -- 172.21.15.89:0/1202605311 shutdown_connections 2024-08-19T10:49:33.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8794d24640 1 -- 172.21.15.89:0/1202605311 >> 172.21.15.89:0/1202605311 conn(0x7f8790069a80 msgr2=0x7f87900ab180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:33.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8794d24640 1 -- 172.21.15.89:0/1202605311 shutdown_connections 2024-08-19T10:49:33.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:32.978+0000 7f8794d24640 1 -- 172.21.15.89:0/1202605311 wait complete. 2024-08-19T10:49:33.024 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:49:33.024 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-08-19T10:49:33.024 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:49:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient: get_monmap_and_config 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient: build_initial_monmap 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient: monmap: 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:33.149716+0000 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:33.149716+0000 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:33.146 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 1 Processor -- start 2024-08-19T10:49:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 1 -- start start 2024-08-19T10:49:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient: init 2024-08-19T10:49:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient: _add_conns ranks=[8,0,2,7,6,4,9,1,10,5,3] 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 0x7f0e480a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient(hunting): picked mon.noname-e con 0x7f0e4806c930 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0e480a3f40 addr v1:172.21.15.89:6789/0 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e480a4320 0x7f0e48139f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient(hunting): picked mon.noname-b con 0x7f0e480a4320 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e480a1bc0 con 0x7f0e480a3f40 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f0e4806cd10 con 0x7f0e480a4320 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f0e480a4b60 con 0x7f0e4806c930 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4e3c9640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e480a4320 msgr2=0x7f0e48139f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4e3c9640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e480a4320 0x7f0e48139f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4fe2c640 10 monclient(hunting): authenticate will time out at 1622.966484s 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 0x7f0e480a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4d3c7640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0e480a3f40 0x7f0e48139850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57124/0 (socket says 172.21.15.89:57124) 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4d3c7640 1 -- 172.21.15.89:0/1518400877 learned_addr learned my addr 172.21.15.89:0/1518400877 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 10 monclient(hunting): get_auth_request con 0x7f0e4806c930 auth_method 0 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 10 monclient(hunting): handle_auth_done global_id 6938 payload 274 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 1 -- 172.21.15.89:0/1518400877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e480a4320 msgr2=0x7f0e48139f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:49:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 1 --2- 172.21.15.89:0/1518400877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e480a4320 0x7f0e48139f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 1 -- 172.21.15.89:0/1518400877 >> v1:172.21.15.89:6789/0 conn(0x7f0e480a3f40 legacy=0x7f0e48139850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:33.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 10 monclient: _finish_hunting 0 2024-08-19T10:49:33.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 1 monclient: found mon.noname-e 2024-08-19T10:49:33.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:33.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:33.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.146+0000 7f0e4dbc8640 1 -- 172.21.15.89:0/1518400877 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e480a4de0 con 0x7f0e4806c930 2024-08-19T10:49:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4dbc8640 1 -- 172.21.15.89:0/1518400877 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 msgr2=0x7f0e480a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4dbc8640 1 --2- 172.21.15.89:0/1518400877 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 0x7f0e480a3a00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009dc0 tx=0x7f0e4401c870 comp rx=0 tx=0).ready entity=mon.8 client_cookie=dd3106288655f495 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 1 -- 172.21.15.89:0/1518400877 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e440b2070 con 0x7f0e4806c930 2024-08-19T10:49:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 10 monclient: dump: 2024-08-19T10:49:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.159 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 10 monclient: _finish_auth 0 2024-08-19T10:49:33.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 10 monclient: _check_auth_tickets 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 1 -- 172.21.15.89:0/1518400877 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e440a5a90 con 0x7f0e4806c930 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 1 -- 172.21.15.89:0/1518400877 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e440a5c40 con 0x7f0e4806c930 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4fe2c640 5 monclient: authenticate success, global_id 6938 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4fe2c640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4fe2c640 4 set_mon_vals no callback set 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4cbc6640 10 monclient: dump: 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4fe2c640 10 monclient: shutdown 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4fe2c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1518400877 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 msgr2=0x7f0e480a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4fe2c640 1 --2- 172.21.15.89:0/1518400877 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 0x7f0e480a3a00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009dc0 tx=0x7f0e4401c870 comp rx=0 tx=0).stop 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1518400877 shutdown_connections 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4fe2c640 1 --2- 172.21.15.89:0/1518400877 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e480a4320 0x7f0e48139f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4fe2c640 1 --2- 172.21.15.89:0/1518400877 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 0x7f0e480a3a00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.150+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1518400877 >> 172.21.15.89:0/1518400877 conn(0x7f0e480a9090 msgr2=0x7f0e480a94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1518400877 shutdown_connections 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1518400877 wait complete. 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient: build_initial_monmap 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient: monmap: 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:33.163 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 1 Processor -- start 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 1 -- start start 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient: init 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient: _add_conns ranks=[8,0,7,5,4,1,6,2,3] 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 0x7f0e48138f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient(hunting): picked mon.noname-e con 0x7f0e4806c930 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e480a3f40 0x7f0e481394b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient(hunting): picked mon.noname-a con 0x7f0e480a3f40 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e480a4320 0x7f0e4813d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient(hunting): picked mon.noname-i con 0x7f0e480a4320 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0e480a0190 con 0x7f0e480a3f40 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0e480a47d0 con 0x7f0e480a4320 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f0e4813a4a0 con 0x7f0e4806c930 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4fe2c640 10 monclient(hunting): authenticate will time out at 1622.973771s 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 0x7f0e48138f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4d3c7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e480a3f40 0x7f0e481394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 0x7f0e48138f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48696/0 (socket says 172.21.15.89:48696) 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 1 -- 172.21.15.89:0/1283269150 learned_addr learned my addr 172.21.15.89:0/1283269150 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 10 monclient(hunting): get_auth_request con 0x7f0e4806c930 auth_method 0 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:33.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4d3c7640 10 monclient(hunting): get_auth_request con 0x7f0e480a3f40 auth_method 0 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4d3c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4d3c7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4d3c7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4d3c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4d3c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4d3c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4e3c9640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e480a4320 0x7f0e4813d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 10 monclient(hunting): handle_auth_done global_id 6947 payload 824 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 1 -- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e480a4320 msgr2=0x7f0e4813d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e480a4320 0x7f0e4813d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 1 -- 172.21.15.89:0/1283269150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e480a3f40 msgr2=0x7f0e481394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e480a3f40 0x7f0e481394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 10 monclient: _finish_hunting 0 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 1 monclient: found mon.noname-e 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4dbc8640 1 -- 172.21.15.89:0/1283269150 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e48123b70 con 0x7f0e4806c930 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4e3c9640 10 monclient: get_auth_request con 0x7f0e480a4320 auth_method 0 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.154+0000 7f0e4e3c9640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e480a4320 0x7f0e4813d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4dbc8640 1 -- 172.21.15.89:0/1283269150 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 msgr2=0x7f0e48138f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4dbc8640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 0x7f0e48138f70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0e440b0040 tx=0x7f0e44019610 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e440b2050 con 0x7f0e4806c930 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:49:33.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 10 monclient: dump: 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.169 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e440b3040 con 0x7f0e4806c930 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e44003dc0 con 0x7f0e4806c930 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4fe2c640 5 monclient: authenticate success, global_id 6947 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4fe2c640 10 monclient: _renew_subs 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4fe2c640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1283269150 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e48123e50 con 0x7f0e4806c930 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3bfff640 4 set_mon_vals no callback set 2024-08-19T10:49:33.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 10 monclient: dump: 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.171 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4fe2c640 10 monclient: _renew_subs 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4fe2c640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1283269150 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e481399f0 con 0x7f0e4806c930 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4fe2c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4fe2c640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4813e210 0x7f0e4813e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4fe2c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1283269150 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0e48064590 con 0x7f0e4813e210 2024-08-19T10:49:33.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0e440c0030 con 0x7f0e4806c930 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4d3c7640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4813e210 0x7f0e4813e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e2c03be00 0x7f0e2c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4d3c7640 10 monclient: get_auth_request con 0x7f0e4813e210 auth_method 0 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4d3c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4d3c7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4d3c7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f0e440be020 con 0x7f0e4806c930 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4d3c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4d3c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4d3c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4e3c9640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e2c03be00 0x7f0e2c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4e3c9640 10 monclient: get_auth_request con 0x7f0e2c03be00 auth_method 0 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4d3c7640 10 monclient(hunting): handle_auth_done global_id 6989 payload 824 2024-08-19T10:49:33.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.158+0000 7f0e4d3c7640 1 -- 172.21.15.89:0/1283269150 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e480733c0 con 0x7f0e4813e210 2024-08-19T10:49:33.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e4d3c7640 1 -- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4813e210 msgr2=0x7f0e4813e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:33.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e4d3c7640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4813e210 0x7f0e4813e5f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00ea70 tx=0x7f0e3c00eb60 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:33.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e4e3c9640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e2c03be00 0x7f0e2c03e2c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0e4813d420 tx=0x7f0e40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:33.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e3c00ccb0 con 0x7f0e4813e210 2024-08-19T10:49:33.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e3a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e3a7fc640 10 monclient: dump: 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0e3c019070 con 0x7f0e4813e210 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e3a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e3a7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4813e210 msgr2=0x7f0e4813e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e3a7fc640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4813e210 0x7f0e4813e5f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00ea70 tx=0x7f0e3c00eb60 comp rx=0 tx=0).stop 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4813e210 msgr2=0x7f0e4813e5f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.162+0000 7f0e3a7fc640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4813e210 0x7f0e4813e5f0 secure :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00ea70 tx=0x7f0e3c00eb60 comp rx=0 tx=0).stop 2024-08-19T10:49:33.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4fe2c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4fe2c640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4802d1d0 0x7f0e4802d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4fe2c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1283269150 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0e480a7ba0 con 0x7f0e4802d1d0 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4d3c7640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4802d1d0 0x7f0e4802d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4d3c7640 10 monclient: get_auth_request con 0x7f0e4802d1d0 auth_method 0 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4d3c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4d3c7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4d3c7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4d3c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4d3c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4d3c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4d3c7640 10 monclient(hunting): handle_auth_done global_id 6998 payload 824 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.170+0000 7f0e4d3c7640 1 -- 172.21.15.89:0/1283269150 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0e48073910 con 0x7f0e4802d1d0 2024-08-19T10:49:33.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4d3c7640 1 -- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4802d1d0 msgr2=0x7f0e4802d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4d3c7640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4802d1d0 0x7f0e4802d5e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c007930 tx=0x7f0e3c004950 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e3c024070 con 0x7f0e4802d1d0 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e3a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e3a7fc640 10 monclient: dump: 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.180 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9591 (secure 0 0 0) 0x7f0e3c014480 con 0x7f0e4802d1d0 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e3a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e3a7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4802d1d0 msgr2=0x7f0e4802d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e3a7fc640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4802d1d0 0x7f0e4802d5e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c007930 tx=0x7f0e3c004950 comp rx=0 tx=0).stop 2024-08-19T10:49:33.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e3a7fc640 1 -- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4802d1d0 msgr2=0x7f0e4802d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:33.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e3a7fc640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e4802d1d0 0x7f0e4802d5e0 secure :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c007930 tx=0x7f0e3c004950 comp rx=0 tx=0).stop 2024-08-19T10:49:33.182 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:33.182 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:33.182 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:33.182 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:33.182 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 102, 2024-08-19T10:49:33.182 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 702949, 2024-08-19T10:49:33.182 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:33.182 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:33.183 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:33.184 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:33.185 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:33.186 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.187 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.189 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:33.189 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.189 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.189 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.189 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:33.189 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:33.189 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.189 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.189 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.190 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.191 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:33.192 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.193 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.194 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:33.195 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.197 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:33.198 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.199 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:33.201 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:33.202 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.203 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:33.204 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.205 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.206 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:33.206 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e2c03be00 msgr2=0x7f0e2c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4fe2c640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e2c03be00 0x7f0e2c03e2c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0e4813d420 tx=0x7f0e40008040 comp rx=0 tx=0).stop 2024-08-19T10:49:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4fe2c640 10 monclient: shutdown 2024-08-19T10:49:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4fe2c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1283269150 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 msgr2=0x7f0e48138f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4fe2c640 1 --2- 172.21.15.89:0/1283269150 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0e4806c930 0x7f0e48138f70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0e440b0040 tx=0x7f0e44019610 comp rx=0 tx=0).stop 2024-08-19T10:49:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4e3c9640 1 -- 172.21.15.89:0/1283269150 reap_dead start 2024-08-19T10:49:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4e3c9640 1 -- 172.21.15.89:0/1283269150 reap_dead start 2024-08-19T10:49:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1283269150 shutdown_connections 2024-08-19T10:49:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1283269150 >> 172.21.15.89:0/1283269150 conn(0x7f0e480a9090 msgr2=0x7f0e48141230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1283269150 shutdown_connections 2024-08-19T10:49:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.174+0000 7f0e4fe2c640 1 -- 172.21.15.89:0/1283269150 wait complete. 2024-08-19T10:49:33.208 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:49:33.208 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-08-19T10:49:33.209 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:49:33.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient: get_monmap_and_config 2024-08-19T10:49:33.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient: build_initial_monmap 2024-08-19T10:49:33.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient: monmap: 2024-08-19T10:49:33.332 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:33.333 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.333 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:49:33.337155+0000 2024-08-19T10:49:33.333 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:49:33.337155+0000 2024-08-19T10:49:33.333 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:33.333 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.333 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:49:33.333 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:49:33.333 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:33.333 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:33.334 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:33.334 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:33.334 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:33.334 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:33.334 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:33.334 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:49:33.334 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:49:33.334 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 1 Processor -- start 2024-08-19T10:49:33.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 1 -- start start 2024-08-19T10:49:33.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient: init 2024-08-19T10:49:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient: _add_conns ranks=[2,0,3,10,7,8,4,5,6,1,9] 2024-08-19T10:49:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63e809ea20 0x7f63e809d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient(hunting): picked mon.noname-b con 0x7f63e809ea20 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:49:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f63e809ee00 addr v1:172.21.15.89:6789/0 2024-08-19T10:49:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63e809f1e0 0x7f63e806d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient(hunting): picked mon.noname-g con 0x7f63e809f1e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:49:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e809fbc0 con 0x7f63e809ee00 2024-08-19T10:49:33.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f63e80a3dc0 con 0x7f63e809ea20 2024-08-19T10:49:33.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f63e80a51d0 con 0x7f63e809f1e0 2024-08-19T10:49:33.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:33.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63ed08a640 10 monclient(hunting): authenticate will time out at 1623.153719s 2024-08-19T10:49:33.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e6d9d640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63e809ea20 msgr2=0x7f63e809d360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:49:33.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e6d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63e809ea20 0x7f63e809d360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:33.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e659c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f63e809ee00 0x7f63e806c930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57138/0 (socket says 172.21.15.89:57138) 2024-08-19T10:49:33.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e659c640 1 -- 172.21.15.89:0/4174081949 learned_addr learned my addr 172.21.15.89:0/4174081949 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:49:33.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 1 --2- 172.21.15.89:0/4174081949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63e809f1e0 0x7f63e806d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 10 monclient(hunting): get_auth_request con 0x7f63e809f1e0 auth_method 0 2024-08-19T10:49:33.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:33.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:33.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:33.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:33.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:33.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:33.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 10 monclient(hunting): handle_auth_done global_id 35499 payload 274 2024-08-19T10:49:33.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 1 -- 172.21.15.89:0/4174081949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63e809ea20 msgr2=0x7f63e809d360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:49:33.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 1 --2- 172.21.15.89:0/4174081949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63e809ea20 0x7f63e809d360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 1 -- 172.21.15.89:0/4174081949 >> v1:172.21.15.89:6789/0 conn(0x7f63e809ee00 legacy=0x7f63e806c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:49:33.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 10 monclient: _finish_hunting 0 2024-08-19T10:49:33.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 1 monclient: found mon.noname-g 2024-08-19T10:49:33.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:33.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:49:33.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.334+0000 7f63e759e640 1 -- 172.21.15.89:0/4174081949 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e809f690 con 0x7f63e809f1e0 2024-08-19T10:49:33.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e759e640 1 -- 172.21.15.89:0/4174081949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63e809f1e0 msgr2=0x7f63e806d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:33.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e759e640 1 --2- 172.21.15.89:0/4174081949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63e809f1e0 0x7f63e806d040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009dc0 tx=0x7f63e001c900 comp rx=0 tx=0).ready entity=mon.3 client_cookie=bfcd6cb81006a8c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:33.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 1 -- 172.21.15.89:0/4174081949 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63e00b2070 con 0x7f63e809f1e0 2024-08-19T10:49:33.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:49:33.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 10 monclient: dump: 2024-08-19T10:49:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.353 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 10 monclient: _finish_auth 0 2024-08-19T10:49:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:49:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63ed08a640 5 monclient: authenticate success, global_id 35499 2024-08-19T10:49:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 1 -- 172.21.15.89:0/4174081949 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e00a5b20 con 0x7f63e809f1e0 2024-08-19T10:49:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 1 -- 172.21.15.89:0/4174081949 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63e00a5cd0 con 0x7f63e809f1e0 2024-08-19T10:49:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63ed08a640 10 monclient: get_monmap_and_config success 2024-08-19T10:49:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63ed08a640 4 set_mon_vals no callback set 2024-08-19T10:49:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:49:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.338+0000 7f63e5d9b640 10 monclient: dump: 2024-08-19T10:49:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.356 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient: shutdown 2024-08-19T10:49:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 -- 172.21.15.89:0/4174081949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63e809f1e0 msgr2=0x7f63e806d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 --2- 172.21.15.89:0/4174081949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63e809f1e0 0x7f63e806d040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009dc0 tx=0x7f63e001c900 comp rx=0 tx=0).stop 2024-08-19T10:49:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 -- 172.21.15.89:0/4174081949 shutdown_connections 2024-08-19T10:49:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 --2- 172.21.15.89:0/4174081949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63e809f1e0 0x7f63e806d040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 --2- 172.21.15.89:0/4174081949 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f63e809ea20 0x7f63e809d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 -- 172.21.15.89:0/4174081949 >> 172.21.15.89:0/4174081949 conn(0x7f63e80a8f60 msgr2=0x7f63e80a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 -- 172.21.15.89:0/4174081949 shutdown_connections 2024-08-19T10:49:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 -- 172.21.15.89:0/4174081949 wait complete. 2024-08-19T10:49:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient: build_initial_monmap 2024-08-19T10:49:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient: monmap: 2024-08-19T10:49:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:49:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:49:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:49:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:49:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:49:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:49:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:49:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:49:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:49:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:49:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:49:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:49:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:49:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:49:33.359 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 Processor -- start 2024-08-19T10:49:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 -- start start 2024-08-19T10:49:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient: init 2024-08-19T10:49:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient: _reopen_session rank -1 2024-08-19T10:49:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient: _add_conns ranks=[8,5,4,7,3,2,0,6,1] 2024-08-19T10:49:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63e809ea20 0x7f63e8122670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient(hunting): picked mon.noname-e con 0x7f63e809ea20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:49:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e809ee00 0x7f63e81394e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient(hunting): picked mon.noname-h con 0x7f63e809ee00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:49:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f63e809f1e0 0x7f63e81200f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient(hunting): picked mon.noname-c con 0x7f63e809f1e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:49:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f63e806d700 con 0x7f63e809f1e0 2024-08-19T10:49:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f63e806d580 con 0x7f63e809ee00 2024-08-19T10:49:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f63e8141d70 con 0x7f63e809ea20 2024-08-19T10:49:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient(hunting): _renew_subs 2024-08-19T10:49:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63ed08a640 10 monclient(hunting): authenticate will time out at 1623.162682s 2024-08-19T10:49:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f63e809f1e0 0x7f63e81200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f63e809f1e0 0x7f63e81200f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:44480/0 (socket says 172.21.15.89:44480) 2024-08-19T10:49:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 1 -- 172.21.15.89:0/1763145229 learned_addr learned my addr 172.21.15.89:0/1763145229 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:49:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 10 monclient(hunting): get_auth_request con 0x7f63e809f1e0 auth_method 0 2024-08-19T10:49:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e6d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63e809ea20 0x7f63e8122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e6d9d640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63e809ea20 msgr2=0x7f63e8122670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:49:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e6d9d640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63e809ea20 msgr2=0x7f63e8122670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:49:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e6d9d640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63e809ea20 0x7f63e8122670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:49:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e6d9d640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63e809ea20 0x7f63e8122670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:49:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e659c640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e809ee00 0x7f63e81394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e659c640 10 monclient(hunting): get_auth_request con 0x7f63e809ee00 auth_method 0 2024-08-19T10:49:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 10 monclient(hunting): handle_auth_done global_id 44887 payload 824 2024-08-19T10:49:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63e809ea20 msgr2=0x7f63e8122670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:49:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f63e809ea20 0x7f63e8122670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e809ee00 msgr2=0x7f63e81394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e809ee00 0x7f63e81394e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 10 monclient: _finish_hunting 0 2024-08-19T10:49:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 1 monclient: found mon.noname-c 2024-08-19T10:49:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:49:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:49:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e759e640 1 -- 172.21.15.89:0/1763145229 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e81cfb90 con 0x7f63e809f1e0 2024-08-19T10:49:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.342+0000 7f63e659c640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f63e809ee00 0x7f63e81394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:49:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e759e640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f63e809f1e0 msgr2=0x7f63e81200f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e759e640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f63e809f1e0 0x7f63e81200f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63e001d1a0 tx=0x7f63e00196a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63e00bb020 con 0x7f63e809f1e0 2024-08-19T10:49:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:49:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 10 monclient: dump: 2024-08-19T10:49:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.368 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:49:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:49:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:49:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e0003bb0 con 0x7f63e809f1e0 2024-08-19T10:49:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63ed08a640 5 monclient: authenticate success, global_id 44887 2024-08-19T10:49:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63ed08a640 10 monclient: _renew_subs 2024-08-19T10:49:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63ed08a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:49:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63ed08a640 1 -- 172.21.15.89:0/1763145229 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e8123e70 con 0x7f63e809f1e0 2024-08-19T10:49:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:49:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63e00b2040 con 0x7f63e809f1e0 2024-08-19T10:49:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:49:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 10 monclient: dump: 2024-08-19T10:49:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.371 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e559a640 4 set_mon_vals no callback set 2024-08-19T10:49:33.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63ed08a640 10 monclient: _renew_subs 2024-08-19T10:49:33.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63ed08a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:49:33.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63ed08a640 1 -- 172.21.15.89:0/1763145229 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e8121b10 con 0x7f63e809f1e0 2024-08-19T10:49:33.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63ed08a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:33.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63ed08a640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e8139a20 0x7f63e802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63ed08a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63ed08a640 1 -- 172.21.15.89:0/1763145229 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f63e809d360 con 0x7f63e8139a20 2024-08-19T10:49:33.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f63e00be020 con 0x7f63e809f1e0 2024-08-19T10:49:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e6d9d640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e8139a20 0x7f63e802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e6d9d640 10 monclient: get_auth_request con 0x7f63e8139a20 auth_method 0 2024-08-19T10:49:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63d403bd30 0x7f63d403e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(271..271 src has 228..271) ==== 7657+0+0 (secure 0 0 0) 0x7f63e00e9e60 con 0x7f63e809f1e0 2024-08-19T10:49:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e6d9d640 10 monclient(hunting): handle_auth_done global_id 64231 payload 824 2024-08-19T10:49:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e6d9d640 1 -- 172.21.15.89:0/1763145229 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63e8064590 con 0x7f63e8139a20 2024-08-19T10:49:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e659c640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63d403bd30 0x7f63d403e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.346+0000 7f63e659c640 10 monclient: get_auth_request con 0x7f63d403bd30 auth_method 0 2024-08-19T10:49:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63e6d9d640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e8139a20 msgr2=0x7f63e802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63e6d9d640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e8139a20 0x7f63e802d1d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c450 tx=0x7f63d800c910 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63d8017070 con 0x7f63e8139a20 2024-08-19T10:49:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63cf7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:49:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63cf7fe640 10 monclient: dump: 2024-08-19T10:49:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.377 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.377 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63e659c640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63d403bd30 0x7f63d403e1f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f63dc007fd0 tx=0x7f63dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:33.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f63d80039c0 con 0x7f63e8139a20 2024-08-19T10:49:33.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63cf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:49:33.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63cf7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:49:33.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e8139a20 msgr2=0x7f63e802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63cf7fe640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e8139a20 0x7f63e802d1d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c450 tx=0x7f63d800c910 comp rx=0 tx=0).stop 2024-08-19T10:49:33.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e8139a20 msgr2=0x7f63e802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:33.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.350+0000 7f63cf7fe640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e8139a20 0x7f63e802d1d0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c450 tx=0x7f63d800c910 comp rx=0 tx=0).stop 2024-08-19T10:49:33.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63ed08a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:49:33.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63ed08a640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e802d710 0x7f63e802db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:49:33.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63ed08a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:49:33.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63ed08a640 1 -- 172.21.15.89:0/1763145229 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f63e806ed40 con 0x7f63e802d710 2024-08-19T10:49:33.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63e6d9d640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e802d710 0x7f63e802db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:49:33.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63e6d9d640 10 monclient: get_auth_request con 0x7f63e802d710 auth_method 0 2024-08-19T10:49:33.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63e6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:49:33.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63e6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:49:33.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63e6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:49:33.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63e6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:49:33.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63e6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:49:33.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63e6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:49:33.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63e6d9d640 10 monclient(hunting): handle_auth_done global_id 64240 payload 824 2024-08-19T10:49:33.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.358+0000 7f63e6d9d640 1 -- 172.21.15.89:0/1763145229 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f63e81235d0 con 0x7f63e802d710 2024-08-19T10:49:33.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63e6d9d640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e802d710 msgr2=0x7f63e802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:49:33.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63e6d9d640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e802d710 0x7f63e802db20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c450 tx=0x7f63d80059e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:49:33.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f63d8017030 con 0x7f63e802d710 2024-08-19T10:49:33.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:49:33.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63cf7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:49:33.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63cf7fe640 10 monclient: dump: 2024-08-19T10:49:33.382 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:49:33.382 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:49:33.382 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.382 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:49:33.382 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:49:33.382 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:49:33.382 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:49:33.382 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:49:33.382 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:49:33.382 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:49:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:49:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:49:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:49:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:49:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:49:33.383 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:49:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9256 (secure 0 0 0) 0x7f63d8018040 con 0x7f63e802d710 2024-08-19T10:49:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63cf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:49:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63cf7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:49:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e802d710 msgr2=0x7f63e802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63cf7fe640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e802d710 0x7f63e802db20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c450 tx=0x7f63d80059e0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63cf7fe640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e802d710 msgr2=0x7f63e802db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:49:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63cf7fe640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f63e802d710 0x7f63e802db20 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c450 tx=0x7f63d80059e0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.384 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:49:33.384 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:33.384 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:33.385 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:49:33.385 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 102, 2024-08-19T10:49:33.385 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 39763, 2024-08-19T10:49:33.385 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:49:33.385 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:49:33.385 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:49:33.385 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:49:33.385 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:49:33.385 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:49:33.385 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:49:33.386 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:49:33.386 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:49:33.386 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.386 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:49:33.386 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:33.386 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:49:33.386 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:49:33.386 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:33.387 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:33.387 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:33.387 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:33.387 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:33.387 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:33.387 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:33.387 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:33.387 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:33.387 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:33.387 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:33.388 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.388 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:49:33.388 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:49:33.388 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:33.388 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:33.388 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:33.388 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:33.388 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:33.388 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:49:33.389 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:49:33.390 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.391 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:49:33.392 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:49:33.393 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.396 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:49:33.399 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:49:33.401 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.403 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.405 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:49:33.406 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:49:33.407 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:49:33.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:49:33.409 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:49:33.409 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:49:33.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63ed08a640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63d403bd30 msgr2=0x7f63d403e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63ed08a640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f63d403bd30 0x7f63d403e1f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f63dc007fd0 tx=0x7f63dc009040 comp rx=0 tx=0).stop 2024-08-19T10:49:33.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63ed08a640 10 monclient: shutdown 2024-08-19T10:49:33.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63ed08a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:49:33.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63ed08a640 1 -- 172.21.15.89:0/1763145229 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f63e809f1e0 msgr2=0x7f63e81200f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:49:33.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63ed08a640 1 --2- 172.21.15.89:0/1763145229 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f63e809f1e0 0x7f63e81200f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63e001d1a0 tx=0x7f63e00196a0 comp rx=0 tx=0).stop 2024-08-19T10:49:33.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63e759e640 1 -- 172.21.15.89:0/1763145229 reap_dead start 2024-08-19T10:49:33.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63e759e640 1 -- 172.21.15.89:0/1763145229 reap_dead start 2024-08-19T10:49:33.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63ed08a640 1 -- 172.21.15.89:0/1763145229 shutdown_connections 2024-08-19T10:49:33.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63ed08a640 1 -- 172.21.15.89:0/1763145229 >> 172.21.15.89:0/1763145229 conn(0x7f63e80a8f60 msgr2=0x7f63e8073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:49:33.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63ed08a640 1 -- 172.21.15.89:0/1763145229 shutdown_connections 2024-08-19T10:49:33.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:49:33.362+0000 7f63ed08a640 1 -- 172.21.15.89:0/1763145229 wait complete. 2024-08-19T10:49:33.411 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-08-19T10:49:33.411 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-08-19T10:51:03.412 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-08-19T10:51:03.413 INFO:tasks.ceph.mon.b:Restarting daemon 2024-08-19T10:51:03.413 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-08-19T10:51:03.417 INFO:tasks.ceph.mon.b:Started 2024-08-19T10:51:03.417 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:51:03.417 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:51:03.521 INFO:tasks.ceph.mon.b.smithi089.stderr:2024-08-19T10:51:03.521+0000 7ff4da174dc0 -1 mon.b@-1(???).log v0 log_external_backlog rewinding external_log_to from 431 -> 0 (sync_force? mon rebuild?) 2024-08-19T10:51:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient: get_monmap_and_config 2024-08-19T10:51:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient: build_initial_monmap 2024-08-19T10:51:03.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient: monmap: 2024-08-19T10:51:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:03.545876+0000 2024-08-19T10:51:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:03.545876+0000 2024-08-19T10:51:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:03.543 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 1 Processor -- start 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 1 -- start start 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient: init 2024-08-19T10:51:03.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient: _add_conns ranks=[1,10,5,9,3,2,4,6,7,8,0] 2024-08-19T10:51:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ffb1c0a3ac0 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb1c0a45b0 0x7ffb1c13ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient(hunting): picked mon.noname-f con 0x7ffb1c0a45b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 0x7ffb1c141540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient(hunting): picked mon.noname-h con 0x7ffb1c13f150 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb1c06d6b0 con 0x7ffb1c0a3ac0 2024-08-19T10:51:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ffb1c074530 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:03.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ffb1c0a0f80 con 0x7ffb1c13f150 2024-08-19T10:51:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:03.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb21ec4640 10 monclient(hunting): authenticate will time out at 1713.362153s 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1affd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb1c0a45b0 0x7ffb1c13ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 0x7ffb1c141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1affd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb1c0a45b0 0x7ffb1c13ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:45752/0 (socket says 172.21.15.89:45752) 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1affd640 1 -- 172.21.15.89:0/2542293532 learned_addr learned my addr 172.21.15.89:0/2542293532 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1affd640 10 monclient(hunting): get_auth_request con 0x7ffb1c0a45b0 auth_method 0 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 10 monclient(hunting): get_auth_request con 0x7ffb1c13f150 auth_method 0 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:03.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 10 monclient(hunting): handle_auth_done global_id 7016 payload 274 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 1 -- 172.21.15.89:0/2542293532 >> v2:172.21.15.190:3300/0 conn(0x7ffb1c0a45b0 msgr2=0x7ffb1c13ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 1 --2- 172.21.15.89:0/2542293532 >> v2:172.21.15.190:3300/0 conn(0x7ffb1c0a45b0 0x7ffb1c13ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 1 -- 172.21.15.89:0/2542293532 >> v1:172.21.15.190:6789/0 conn(0x7ffb1c0a3ac0 legacy=0x7ffb1c0a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 1 monclient: found mon.noname-h 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:51:03.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.541+0000 7ffb1bfff640 1 -- 172.21.15.89:0/2542293532 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c0a1260 con 0x7ffb1c13f150 2024-08-19T10:51:03.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.545+0000 7ffb1bfff640 1 -- 172.21.15.89:0/2542293532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 msgr2=0x7ffb1c141540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:03.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.545+0000 7ffb1bfff640 1 --2- 172.21.15.89:0/2542293532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 0x7ffb1c141540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffb140097c0 tx=0x7ffb1401c300 comp rx=0 tx=0).ready entity=mon.5 client_cookie=2eccea36e3168c34 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:03.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.545+0000 7ffb1a7fc640 1 -- 172.21.15.89:0/2542293532 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb140b0070 con 0x7ffb1c13f150 2024-08-19T10:51:03.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.545+0000 7ffb1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:03.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.545+0000 7ffb1a7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:51:03.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.545+0000 7ffb1a7fc640 10 monclient: dump: 2024-08-19T10:51:03.556 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:03.556 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:03.556 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:03.557 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 5 monclient: authenticate success, global_id 7016 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1a7fc640 1 -- 172.21.15.89:0/2542293532 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb140aa810 con 0x7ffb1c13f150 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1a7fc640 1 -- 172.21.15.89:0/2542293532 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb140a9300 con 0x7ffb1c13f150 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 4 set_mon_vals no callback set 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1a7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1a7fc640 10 monclient: dump: 2024-08-19T10:51:03.558 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:03.559 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient: shutdown 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 -- 172.21.15.89:0/2542293532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 msgr2=0x7ffb1c141540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 --2- 172.21.15.89:0/2542293532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 0x7ffb1c141540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffb140097c0 tx=0x7ffb1401c300 comp rx=0 tx=0).stop 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 -- 172.21.15.89:0/2542293532 shutdown_connections 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 --2- 172.21.15.89:0/2542293532 >> v2:172.21.15.190:3300/0 conn(0x7ffb1c0a45b0 0x7ffb1c13ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 --2- 172.21.15.89:0/2542293532 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 0x7ffb1c141540 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 -- 172.21.15.89:0/2542293532 >> 172.21.15.89:0/2542293532 conn(0x7ffb1c0a9130 msgr2=0x7ffb1c0a9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 -- 172.21.15.89:0/2542293532 shutdown_connections 2024-08-19T10:51:03.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 -- 172.21.15.89:0/2542293532 wait complete. 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient: build_initial_monmap 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient: monmap: 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:03.561 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 Processor -- start 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 -- start start 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient: init 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient: _add_conns ranks=[2,8,5,7,1,6,4,0,3] 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb1c0a3ac0 0x7ffb1c1d1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient(hunting): picked mon.noname-b con 0x7ffb1c0a3ac0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb1c0a45b0 0x7ffb1c1d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:03.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient(hunting): picked mon.noname-e con 0x7ffb1c0a45b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 0x7ffb1c136650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient(hunting): picked mon.noname-h con 0x7ffb1c13f150 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ffb1c1442a0 con 0x7ffb1c0a3ac0 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ffb1c144420 con 0x7ffb1c13f150 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ffb1c1445a0 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb21ec4640 10 monclient(hunting): authenticate will time out at 1713.370079s 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb1c0a45b0 0x7ffb1c1d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb1c0a3ac0 0x7ffb1c1d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:03.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb1c0a45b0 0x7ffb1c1d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:56550/0 (socket says 172.21.15.89:56550) 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 1 -- 172.21.15.89:0/58259478 learned_addr learned my addr 172.21.15.89:0/58259478 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 10 monclient(hunting): get_auth_request con 0x7ffb1c0a45b0 auth_method 0 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1b7fe640 10 monclient(hunting): get_auth_request con 0x7ffb1c0a3ac0 auth_method 0 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:03.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1bfff640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 0x7ffb1c136650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1b7fe640 1 -- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb1c0a3ac0 msgr2=0x7ffb1c1d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1b7fe640 1 -- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb1c0a3ac0 msgr2=0x7ffb1c1d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1b7fe640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb1c0a3ac0 0x7ffb1c1d1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1bfff640 10 monclient(hunting): get_auth_request con 0x7ffb1c13f150 auth_method 0 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1b7fe640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb1c0a3ac0 0x7ffb1c1d1710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 10 monclient(hunting): handle_auth_done global_id 6956 payload 824 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 1 -- 172.21.15.89:0/58259478 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 msgr2=0x7ffb1c136650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 0x7ffb1c136650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 1 -- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb1c0a3ac0 msgr2=0x7ffb1c1d1710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb1c0a3ac0 0x7ffb1c1d1710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:03.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 10 monclient: _finish_hunting 0 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 1 monclient: found mon.noname-e 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1affd640 1 -- 172.21.15.89:0/58259478 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c13e9b0 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.549+0000 7ffb1bfff640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 0x7ffb1c136650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb1affd640 1 -- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb1c0a45b0 msgr2=0x7ffb1c1d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb1affd640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb1c0a45b0 0x7ffb1c1d1c50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00e890 tx=0x7ffb0c00ed50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 1 -- 172.21.15.89:0/58259478 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb0c017070 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 10 monclient: dump: 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:03.566 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:03.567 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 10 monclient: _finish_auth 0 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 10 monclient: _check_auth_tickets 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 1 -- 172.21.15.89:0/58259478 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb0c013470 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb21ec4640 5 monclient: authenticate success, global_id 6956 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb21ec4640 10 monclient: _renew_subs 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb21ec4640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb21ec4640 1 -- 172.21.15.89:0/58259478 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c138140 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 1 -- 172.21.15.89:0/58259478 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ffb0c013a90 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:51:03.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 10 monclient: dump: 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:03.569 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb21ec4640 10 monclient: _renew_subs 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb21ec4640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb21ec4640 1 -- 172.21.15.89:0/58259478 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c139c00 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb19ffb640 4 set_mon_vals no callback set 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 1 -- 172.21.15.89:0/58259478 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ffb0c01a070 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb21ec4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb21ec4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb21ec4640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb21ec4640 1 -- 172.21.15.89:0/58259478 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1c138d00 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb0803be00 0x7ffb0803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:03.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb03fff640 1 -- 172.21.15.89:0/58259478 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(272..272 src has 246..272) ==== 7657+0+0 (secure 0 0 0) 0x7ffb0c00f040 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb1b7fe640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb0803be00 0x7ffb0803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:03.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.553+0000 7ffb1b7fe640 10 monclient: get_auth_request con 0x7ffb0803be00 auth_method 0 2024-08-19T10:51:03.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.557+0000 7ffb1b7fe640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb0803be00 0x7ffb0803e2c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffb10003bf0 tx=0x7ffb10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:03.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.557+0000 7ffb03fff640 1 -- 172.21.15.89:0/58259478 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ffb0c04d660 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.557+0000 7ffb03fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:03.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.557+0000 7ffb03fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:03.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.793+0000 7ffb21ec4640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:51:03.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.793+0000 7ffb21ec4640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:51:03.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.793+0000 7ffb21ec4640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:51:03.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.793+0000 7ffb21ec4640 1 -- 172.21.15.89:0/58259478 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ffb1c0a3ea0 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb03fff640 1 -- 172.21.15.89:0/58259478 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3188 (secure 0 0 0) 0x7ffb0c013d60 con 0x7ffb1c0a45b0 2024-08-19T10:51:03.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb03fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:51:03.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb03fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:03.795 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":102,"quorum":[0,1,3,4,5,6,7,8],"quorum_names":["a","f","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":93,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:51:03.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 -- 172.21.15.89:0/58259478 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb0803be00 msgr2=0x7ffb0803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:03.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb0803be00 0x7ffb0803e2c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffb10003bf0 tx=0x7ffb10009040 comp rx=0 tx=0).stop 2024-08-19T10:51:03.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 10 monclient: shutdown 2024-08-19T10:51:03.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:03.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 -- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb1c0a45b0 msgr2=0x7ffb1c1d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:03.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb1c0a45b0 0x7ffb1c1d1c50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00e890 tx=0x7ffb0c00ed50 comp rx=0 tx=0).stop 2024-08-19T10:51:03.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 -- 172.21.15.89:0/58259478 shutdown_connections 2024-08-19T10:51:03.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ffb1c13f150 0x7ffb1c136650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:03.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ffb0803be00 0x7ffb0803e2c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:03.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ffb1c0a45b0 0x7ffb1c1d1c50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:03.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 --2- 172.21.15.89:0/58259478 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ffb1c0a3ac0 0x7ffb1c1d1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:03.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 -- 172.21.15.89:0/58259478 >> 172.21.15.89:0/58259478 conn(0x7ffb1c0a9130 msgr2=0x7ffb1c1409f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:03.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 -- 172.21.15.89:0/58259478 shutdown_connections 2024-08-19T10:51:03.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:03.797+0000 7ffb21ec4640 1 -- 172.21.15.89:0/58259478 wait complete. 2024-08-19T10:51:04.721 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=110 2024-08-19T10:51:04.721 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 110 -ne 110 ]] 2024-08-19T10:51:04.721 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:51:04.721 INFO:tasks.workunit.client.0.smithi190.stderr:+ export CEPH_ARGS= 2024-08-19T10:51:04.721 INFO:tasks.workunit.client.0.smithi190.stderr:+ CEPH_ARGS= 2024-08-19T10:51:04.721 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph auth del client.foo 2024-08-19T10:51:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.834+0000 7f048da03640 10 monclient: get_monmap_and_config 2024-08-19T10:51:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.834+0000 7f048da03640 10 monclient: build_initial_monmap 2024-08-19T10:51:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.834+0000 7f048da03640 10 monclient: monmap: 2024-08-19T10:51:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:51:04.828 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:51:04.837312+0000 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:51:04.837312+0000 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:04.829 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:04.830 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:04.830 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:51:04.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.834+0000 7f048da03640 1 Processor -- start 2024-08-19T10:51:04.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.834+0000 7f048da03640 1 -- start start 2024-08-19T10:51:04.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.834+0000 7f048da03640 10 monclient: init 2024-08-19T10:51:04.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.834+0000 7f048da03640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:04.832 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 10 monclient: _add_conns ranks=[1,8,10,2,5,7,4,9,0,3,6] 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f04880a46e0 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0488074390 0x7f048813b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 10 monclient(hunting): picked mon.noname-e con 0x7f0488074390 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f048809dd90 0x7f048813dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 10 monclient(hunting): picked mon.noname-f con 0x7f048809dd90 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f048809e210 con 0x7f04880a46e0 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f048809f900 con 0x7f048809dd90 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f048809fb50 con 0x7f0488074390 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:04.833 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f048da03640 10 monclient(hunting): authenticate will time out at 1710.408162s 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f0486ffd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f04880a46e0 0x7f04880a4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:44236/0 (socket says 172.21.15.190:44236) 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f0486ffd640 1 -- 172.21.15.190:0/2507832308 learned_addr learned my addr 172.21.15.190:0/2507832308 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f04877fe640 1 --2- 172.21.15.190:0/2507832308 >> v2:172.21.15.190:3300/0 conn(0x7f048809dd90 0x7f048813dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f04877fe640 10 monclient(hunting): get_auth_request con 0x7f048809dd90 auth_method 0 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f04877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f04877fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f04877fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 1 --2- 172.21.15.190:0/2507832308 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0488074390 0x7f048813b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 10 monclient(hunting): get_auth_request con 0x7f0488074390 auth_method 0 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:04.834 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 10 monclient(hunting): handle_auth_done global_id 6965 payload 274 2024-08-19T10:51:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 1 -- 172.21.15.190:0/2507832308 >> v2:172.21.15.190:3300/0 conn(0x7f048809dd90 msgr2=0x7f048813dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 1 --2- 172.21.15.190:0/2507832308 >> v2:172.21.15.190:3300/0 conn(0x7f048809dd90 0x7f048813dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 1 -- 172.21.15.190:0/2507832308 >> v1:172.21.15.190:6789/0 conn(0x7f04880a46e0 legacy=0x7f04880a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:51:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 1 monclient: found mon.noname-e 2024-08-19T10:51:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:51:04.835 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.838+0000 7f047e7fc640 1 -- 172.21.15.190:0/2507832308 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04880a05b0 con 0x7f0488074390 2024-08-19T10:51:04.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f047e7fc640 1 -- 172.21.15.190:0/2507832308 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0488074390 msgr2=0x7f048813b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:04.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f047e7fc640 1 --2- 172.21.15.190:0/2507832308 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0488074390 0x7f048813b6d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f04700097c0 tx=0x7f047001c180 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c4e8c75a89504a21 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:04.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 1 -- 172.21.15.190:0/2507832308 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04700b0070 con 0x7f0488074390 2024-08-19T10:51:04.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:04.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:51:04.840 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 10 monclient: dump: 2024-08-19T10:51:04.841 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:51:04.841 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:04.841 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:04.841 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:04.841 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:04.841 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:51:04.841 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:04.841 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:04.841 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:04.841 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:04.841 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 10 monclient: _finish_auth 0 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 10 monclient: _check_auth_tickets 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 1 -- 172.21.15.190:0/2507832308 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04700aa810 con 0x7f0488074390 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 5 monclient: authenticate success, global_id 6965 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:04.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 1 -- 172.21.15.190:0/2507832308 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04700aae30 con 0x7f0488074390 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 4 set_mon_vals no callback set 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f04867fc640 10 monclient: dump: 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:51:04.843 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 10 monclient: shutdown 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 1 -- 172.21.15.190:0/2507832308 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0488074390 msgr2=0x7f048813b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 1 --2- 172.21.15.190:0/2507832308 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0488074390 0x7f048813b6d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f04700097c0 tx=0x7f047001c180 comp rx=0 tx=0).stop 2024-08-19T10:51:04.844 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 1 -- 172.21.15.190:0/2507832308 shutdown_connections 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 1 --2- 172.21.15.190:0/2507832308 >> v2:172.21.15.190:3300/0 conn(0x7f048809dd90 0x7f048813dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 1 --2- 172.21.15.190:0/2507832308 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0488074390 0x7f048813b6d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 1 -- 172.21.15.190:0/2507832308 >> 172.21.15.190:0/2507832308 conn(0x7f04880a92d0 msgr2=0x7f04880a57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 1 -- 172.21.15.190:0/2507832308 shutdown_connections 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 1 -- 172.21.15.190:0/2507832308 wait complete. 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 10 monclient: build_initial_monmap 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.842+0000 7f048da03640 10 monclient: monmap: 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:51:04.845 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 1 Processor -- start 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 1 -- start start 2024-08-19T10:51:04.846 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient: init 2024-08-19T10:51:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient: _add_conns ranks=[0,6,5,2,4,8,1,7,3] 2024-08-19T10:51:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0488074390 0x7f04881c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient(hunting): picked mon.noname-a con 0x7f0488074390 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:04.847 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f048809dd90 0x7f048813cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:04.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient(hunting): picked mon.noname-d con 0x7f048809dd90 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:04.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04880a46e0 0x7f048813d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:04.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient(hunting): picked mon.noname-h con 0x7f04880a46e0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:04.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:04.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0488076e30 con 0x7f0488074390 2024-08-19T10:51:04.848 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0488076cb0 con 0x7f04880a46e0 2024-08-19T10:51:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f0488076fb0 con 0x7f048809dd90 2024-08-19T10:51:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient(hunting): authenticate will time out at 1710.416619s 2024-08-19T10:51:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04880a46e0 0x7f048813d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04880a46e0 0x7f048813d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:49824/0 (socket says 172.21.15.190:49824) 2024-08-19T10:51:04.849 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 1 -- 172.21.15.190:0/1162648907 learned_addr learned my addr 172.21.15.190:0/1162648907 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:51:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 10 monclient(hunting): get_auth_request con 0x7f04880a46e0 auth_method 0 2024-08-19T10:51:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0486ffd640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0488074390 0x7f04881c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f047e7fc640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f048809dd90 0x7f048813cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:04.850 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f047e7fc640 10 monclient(hunting): get_auth_request con 0x7f048809dd90 auth_method 0 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f047e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f047e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f047e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 10 monclient(hunting): handle_auth_done global_id 7034 payload 824 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 1 -- 172.21.15.190:0/1162648907 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f048809dd90 msgr2=0x7f048813cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f048809dd90 0x7f048813cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 1 -- 172.21.15.190:0/1162648907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0488074390 msgr2=0x7f04881c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0488074390 0x7f04881c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 10 monclient: _finish_hunting 0 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 1 monclient: found mon.noname-h 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:51:04.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 1 -- 172.21.15.190:0/1162648907 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04881377e0 con 0x7f04880a46e0 2024-08-19T10:51:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0486ffd640 10 monclient: get_auth_request con 0x7f0488074390 auth_method 0 2024-08-19T10:51:04.852 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0486ffd640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0488074390 0x7f04881c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 1 -- 172.21.15.190:0/1162648907 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04880a46e0 msgr2=0x7f048813d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f04877fe640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04880a46e0 0x7f048813d190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0480007e40 tx=0x7f048000e4b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 1 -- 172.21.15.190:0/1162648907 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0480013a00 con 0x7f04880a46e0 2024-08-19T10:51:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:51:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 10 monclient: dump: 2024-08-19T10:51:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:51:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:04.854 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:04.855 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 10 monclient: _finish_auth 0 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 1 -- 172.21.15.190:0/1162648907 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0480019070 con 0x7f04880a46e0 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 1 -- 172.21.15.190:0/1162648907 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0480013c00 con 0x7f04880a46e0 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0484ff9640 10 monclient: dump: 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:04.856 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f0485ffb640 4 set_mon_vals no callback set 2024-08-19T10:51:04.857 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 5 monclient: authenticate success, global_id 7034 2024-08-19T10:51:04.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient: _renew_subs 2024-08-19T10:51:04.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:51:04.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 1 -- 172.21.15.190:0/1162648907 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04880a8790 con 0x7f04880a46e0 2024-08-19T10:51:04.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient: _renew_subs 2024-08-19T10:51:04.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:51:04.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.846+0000 7f048da03640 1 -- 172.21.15.190:0/1162648907 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f04880a18b0 con 0x7f04880a46e0 2024-08-19T10:51:04.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.850+0000 7f048da03640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:04.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.850+0000 7f048da03640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:04.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.850+0000 7f048da03640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:51:04.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.850+0000 7f048da03640 1 -- 172.21.15.190:0/1162648907 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04880a4ac0 con 0x7f04880a46e0 2024-08-19T10:51:04.858 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.850+0000 7f0484ff9640 1 -- 172.21.15.190:0/1162648907 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0480013590 con 0x7f04880a46e0 2024-08-19T10:51:04.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.850+0000 7f0484ff9640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f046803be00 0x7f046803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:04.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.850+0000 7f0484ff9640 1 -- 172.21.15.190:0/1162648907 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(272..272 src has 246..272) ==== 7657+0+0 (secure 0 0 0) 0x7f048004ff30 con 0x7f04880a46e0 2024-08-19T10:51:04.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.850+0000 7f0486ffd640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f046803be00 0x7f046803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:04.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.850+0000 7f0486ffd640 10 monclient: get_auth_request con 0x7f046803be00 auth_method 0 2024-08-19T10:51:04.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.850+0000 7f0486ffd640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f046803be00 0x7f046803e2c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f047800a850 tx=0x7f0478008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:04.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.854+0000 7f0484ff9640 1 -- 172.21.15.190:0/1162648907 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0480011260 con 0x7f04880a46e0 2024-08-19T10:51:04.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.854+0000 7f0484ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:04.859 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:04.854+0000 7f0484ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:05.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.078+0000 7f048da03640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.foo"}] 2024-08-19T10:51:05.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.078+0000 7f048da03640 10 monclient: _send_command 2 [{"prefix": "auth del", "entity": "client.foo"}] 2024-08-19T10:51:05.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.078+0000 7f048da03640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:51:05.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.078+0000 7f048da03640 1 -- 172.21.15.190:0/1162648907 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "auth del", "entity": "client.foo"} v 0) -- 0x7f0488135480 con 0x7f04880a46e0 2024-08-19T10:51:05.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.466+0000 7f0484ff9640 1 -- 172.21.15.190:0/1162648907 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.foo"}]=0 v14) ==== 80+0+0 (secure 0 0 0) 0x7f0480011440 con 0x7f04880a46e0 2024-08-19T10:51:05.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.466+0000 7f0484ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth del", "entity": "client.foo"}] 2024-08-19T10:51:05.459 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.466+0000 7f0484ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:05.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.622+0000 7f048da03640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.foo"}] 2024-08-19T10:51:05.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.622+0000 7f048da03640 10 monclient: _send_command 3 [{"prefix": "auth del", "entity": "client.foo"}] 2024-08-19T10:51:05.617 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.622+0000 7f048da03640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:51:05.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.622+0000 7f048da03640 1 -- 172.21.15.190:0/1162648907 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "auth del", "entity": "client.foo"} v 0) -- 0x7f0488051470 con 0x7f04880a46e0 2024-08-19T10:51:05.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f0484ff9640 1 -- 172.21.15.190:0/1162648907 <== mon.5 v2:172.21.15.190:3302/0 8 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.foo"}]=0 v14) ==== 80+0+0 (secure 0 0 0) 0x7f048001c360 con 0x7f04880a46e0 2024-08-19T10:51:05.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f0484ff9640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth del", "entity": "client.foo"}] 2024-08-19T10:51:05.618 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f0484ff9640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:51:05.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 -- 172.21.15.190:0/1162648907 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f046803be00 msgr2=0x7f046803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:05.621 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f046803be00 0x7f046803e2c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f047800a850 tx=0x7f0478008040 comp rx=0 tx=0).stop 2024-08-19T10:51:05.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 10 monclient: shutdown 2024-08-19T10:51:05.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:05.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 -- 172.21.15.190:0/1162648907 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04880a46e0 msgr2=0x7f048813d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:05.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04880a46e0 0x7f048813d190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0480007e40 tx=0x7f048000e4b0 comp rx=0 tx=0).stop 2024-08-19T10:51:05.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 -- 172.21.15.190:0/1162648907 shutdown_connections 2024-08-19T10:51:05.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f046803be00 0x7f046803e2c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:05.622 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04880a46e0 0x7f048813d190 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:05.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f048809dd90 0x7f048813cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:05.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 --2- 172.21.15.190:0/1162648907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0488074390 0x7f04881c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:05.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 -- 172.21.15.190:0/1162648907 >> 172.21.15.190:0/1162648907 conn(0x7f04880a92d0 msgr2=0x7f048813d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:05.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 -- 172.21.15.190:0/1162648907 shutdown_connections 2024-08-19T10:51:05.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:51:05.626+0000 7f048da03640 1 -- 172.21.15.190:0/1162648907 wait complete. 2024-08-19T10:51:05.636 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 13 2024-08-19T10:51:05.636 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status' 2024-08-19T10:51:05.636 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=13 2024-08-19T10:51:05.636 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-08-19T10:51:05.637 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-08-19T10:51:05.637 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.foo.keyring --user foo quorum_status 2024-08-19T10:51:05.741 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=13 2024-08-19T10:51:05.741 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 13 -ne 13 ]] 2024-08-19T10:51:05.741 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:51:05.741 INFO:tasks.workunit.client.0.smithi190.stderr:+ c=''\''allow command service with prefix=list, allow command quorum_status'\''' 2024-08-19T10:51:05.741 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph auth get-or-create client.bar mon '\''allow command service with prefix=list, allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.bar.keyring' 0 2024-08-19T10:51:05.741 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph auth get-or-create client.bar mon '\''allow command service with prefix=list, allow command quorum_status'\'' > /tmp/cephtest-mon-caps-madness.bar.keyring' 2024-08-19T10:51:05.741 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=0 2024-08-19T10:51:05.742 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph auth get-or-create client.bar mon ''\''allow' command service with prefix=list, allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.bar.keyring 2024-08-19T10:51:05.742 INFO:tasks.workunit.client.0.smithi190.stdout:ceph auth get-or-create client.bar mon 'allow command service with prefix=list, allow command quorum_status' > /tmp/cephtest-mon-caps-madness.bar.keyring 2024-08-19T10:51:05.742 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph auth get-or-create client.bar mon ''\''allow' command service with prefix=list, allow command 'quorum_status'\''' '>' /tmp/cephtest-mon-caps-madness.bar.keyring 2024-08-19T10:51:06.448 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=0 2024-08-19T10:51:06.449 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 0 -ne 0 ]] 2024-08-19T10:51:06.449 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:51:06.449 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 0 2024-08-19T10:51:06.449 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 2024-08-19T10:51:06.449 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=0 2024-08-19T10:51:06.449 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-08-19T10:51:06.450 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-08-19T10:51:06.450 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-08-19T10:51:06.815 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:51:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient: get_monmap_and_config 2024-08-19T10:51:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient: build_initial_monmap 2024-08-19T10:51:06.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient: monmap: 2024-08-19T10:51:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:06.950337+0000 2024-08-19T10:51:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:06.950337+0000 2024-08-19T10:51:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:06.945 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:06.946 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:06.946 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 1 Processor -- start 2024-08-19T10:51:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 1 -- start start 2024-08-19T10:51:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient: init 2024-08-19T10:51:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient: _add_conns ranks=[9,3,2,1,4,10,5,8,6,0,7] 2024-08-19T10:51:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient(hunting): picked mon.noname-a con 0x7f0ab40a67d0 addr v2:172.21.15.89:3300/0 2024-08-19T10:51:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient(hunting): picked mon.noname-g con 0x7f0ab40a6bb0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:06.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient(hunting): picked mon.noname-b con 0x7f0ab4139590 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f0ab40a8740 con 0x7f0ab40a67d0 2024-08-19T10:51:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab40a3380 con 0x7f0ab4139590 2024-08-19T10:51:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab40a8250 con 0x7f0ab40a6bb0 2024-08-19T10:51:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0aba98d640 10 monclient(hunting): authenticate will time out at 1716.767100s 2024-08-19T10:51:06.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab3fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab8f2a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab3fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36512/0 (socket says 172.21.15.89:36512) 2024-08-19T10:51:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 learned_addr learned my addr 172.21.15.89:0/1570803285 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request con 0x7f0ab40a67d0 auth_method 0 2024-08-19T10:51:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request con 0x7f0ab4139590 auth_method 0 2024-08-19T10:51:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:06.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 msgr2=0x7f0ab4139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:51:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 msgr2=0x7f0ab40a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 msgr2=0x7f0ab4139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 msgr2=0x7f0ab40a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:06.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:06.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request con 0x7f0ab40a6bb0 auth_method 0 2024-08-19T10:51:06.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:06.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:06.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:06.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 msgr2=0x7f0ab40a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:06.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 msgr2=0x7f0ab40a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:06.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:06.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:06.945+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:07.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request con 0x7f0ab4139590 auth_method 0 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request con 0x7f0ab40a67d0 auth_method 0 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 msgr2=0x7f0ab4139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 msgr2=0x7f0ab4139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:07.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 msgr2=0x7f0ab40a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 msgr2=0x7f0ab40a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request con 0x7f0ab40a6bb0 auth_method 0 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 msgr2=0x7f0ab40a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 msgr2=0x7f0ab40a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:07.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.149+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:07.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:07.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request con 0x7f0ab40a67d0 auth_method 0 2024-08-19T10:51:07.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:07.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request con 0x7f0ab4139590 auth_method 0 2024-08-19T10:51:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 msgr2=0x7f0ab40a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 msgr2=0x7f0ab40a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:07.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request con 0x7f0ab40a6bb0 auth_method 0 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 msgr2=0x7f0ab4139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 msgr2=0x7f0ab4139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 msgr2=0x7f0ab40a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 msgr2=0x7f0ab40a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:07.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:07.553+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:08.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request con 0x7f0ab4139590 auth_method 0 2024-08-19T10:51:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request con 0x7f0ab40a6bb0 auth_method 0 2024-08-19T10:51:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:08.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 msgr2=0x7f0ab4139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 msgr2=0x7f0ab4139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 msgr2=0x7f0ab40a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 msgr2=0x7f0ab40a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request con 0x7f0ab40a67d0 auth_method 0 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:08.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:08.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 msgr2=0x7f0ab40a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:51:08.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 msgr2=0x7f0ab40a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:08.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:08.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:08.357+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:51:09.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 10 monclient(hunting): tick 2024-08-19T10:51:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:51:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 monclient(hunting): continuing hunt 2024-08-19T10:51:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:51:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 msgr2=0x7f0ab40a6f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab40a6bb0 0x7f0ab40a6f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 msgr2=0x7f0ab4139a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0ab4139590 0x7f0ab4139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 -- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 msgr2=0x7f0ab40a5500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 --2- 172.21.15.89:0/1570803285 >> v2:172.21.15.89:3300/0 conn(0x7f0ab40a67d0 0x7f0ab40a5500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 10 monclient: _add_conns ranks=[7,8,1,5,4,10,0,6] 2024-08-19T10:51:09.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 10 monclient(hunting): picked mon.noname-i con 0x7f0a94003810 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 10 monclient(hunting): picked mon.noname-e con 0x7f0a940061c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0a94008d00 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 -- 172.21.15.89:0/1570803285 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ab40a4cc0 con 0x7f0a94008d00 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 -- 172.21.15.89:0/1570803285 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0ab40a8250 con 0x7f0a94003810 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 1 -- 172.21.15.89:0/1570803285 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f0a9400db90 con 0x7f0a940061c0 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab1ffb640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request con 0x7f0a940061c0 auth_method 0 2024-08-19T10:51:09.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.945+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request con 0x7f0a94003810 auth_method 0 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 msgr2=0x7f0a94008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 msgr2=0x7f0a94008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:09.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 msgr2=0x7f0a94005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:09.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 msgr2=0x7f0a94005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:09.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:09.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:09.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:09.949+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> v1:172.21.15.190:6789/0 conn(0x7f0a94008d00 legacy=0x7f0a9400b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:10.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request con 0x7f0a94003810 auth_method 0 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 msgr2=0x7f0a94005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 msgr2=0x7f0a94005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request con 0x7f0a940061c0 auth_method 0 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:10.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 msgr2=0x7f0a94008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 msgr2=0x7f0a94008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:10.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.153+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:10.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:10.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request con 0x7f0a940061c0 auth_method 0 2024-08-19T10:51:10.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:10.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:10.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:10.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 msgr2=0x7f0a94008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 msgr2=0x7f0a94008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request con 0x7f0a94003810 auth_method 0 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 msgr2=0x7f0a94005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 msgr2=0x7f0a94005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:10.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:10.557+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:11.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:11.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request con 0x7f0a940061c0 auth_method 0 2024-08-19T10:51:11.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:11.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:11.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 msgr2=0x7f0a94008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 msgr2=0x7f0a94008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request con 0x7f0a94003810 auth_method 0 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 msgr2=0x7f0a94005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 msgr2=0x7f0a94005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:11.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:11.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.361+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:51:11.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3166335172 0 0) 0x7f0ab40a4cc0 con 0x7f0a94008d00 2024-08-19T10:51:11.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): _negotiate 2024-08-19T10:51:11.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:11.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:11.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): my global_id is 44812 2024-08-19T10:51:11.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a98003530 con 0x7f0a94008d00 2024-08-19T10:51:11.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 550928455 0 0) 0x7f0a98003530 con 0x7f0a94008d00 2024-08-19T10:51:11.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 msgr2=0x7f0a94008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:11.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a940061c0 0x7f0a94008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:11.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 msgr2=0x7f0a94005c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:11.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a94003810 0x7f0a94005c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:11.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:51:11.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 monclient: found mon.noname-f-legacy 2024-08-19T10:51:11.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 reap_dead start 2024-08-19T10:51:11.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:11.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.190:6789/0 2024-08-19T10:51:11.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9400ddf0 con 0x7f0a94008d00 2024-08-19T10:51:11.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:51:11.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:51:11.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:11.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 3927028378 0 0) 0x7f0aa400f040 con 0x7f0a94008d00 2024-08-19T10:51:11.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:11.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-08-19T10:51:11.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient: dump: 2024-08-19T10:51:11.863 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:11.863 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:11.863 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:11.864 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:11.864 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:11.864 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:11.864 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:11.864 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:11.864 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:11.865 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:11.865 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:11.865 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:11.865 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:11.865 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:11.866 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:11.866 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:11.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] but i'm connected to v1:172.21.15.190:6789/0, reconnecting 2024-08-19T10:51:11.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:11.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 >> v1:172.21.15.190:6789/0 conn(0x7f0a94008d00 legacy=0x7f0a9400b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:11.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient: _add_conns ranks=[0,3,7,5,4,6,2,8,1] 2024-08-19T10:51:11.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980042e0 0x7f0a98004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:11.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): picked mon.a con 0x7f0a980042e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:11.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a98004ef0 0x7f0a980099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:11.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): picked mon.g con 0x7f0a98004ef0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:11.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a98009f00 0x7f0a9800c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:11.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): picked mon.i con 0x7f0a98009f00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:11.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:11.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab40a8250 con 0x7f0a980042e0 2024-08-19T10:51:11.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:11.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9400db90 con 0x7f0a98004ef0 2024-08-19T10:51:11.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:11.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0a9800ca70 con 0x7f0a98009f00 2024-08-19T10:51:11.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:11.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab2ffd640 10 monclient(hunting): _finish_auth 0 2024-08-19T10:51:11.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a98009f00 0x7f0a9800c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:11.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request con 0x7f0a98009f00 auth_method 0 2024-08-19T10:51:11.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:11.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:11.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:11.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.849+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a98004ef0 0x7f0a980099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:11.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request con 0x7f0a98004ef0 auth_method 0 2024-08-19T10:51:11.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:11.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:11.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:11.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:11.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:11.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:51:11.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:11.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:11.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab37fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:51:11.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980042e0 0x7f0a98004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:11.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request con 0x7f0a980042e0 auth_method 0 2024-08-19T10:51:11.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:11.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:11.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:11.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 10 monclient(hunting): handle_auth_done global_id 44812 payload 293 2024-08-19T10:51:11.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a98004ef0 msgr2=0x7f0a980099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:11.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a98004ef0 0x7f0a980099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:11.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980042e0 msgr2=0x7f0a98004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:11.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980042e0 0x7f0a98004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:11.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 10 monclient: _finish_hunting 0 2024-08-19T10:51:11.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 1 monclient: found mon.i 2024-08-19T10:51:11.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:11.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:51:11.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a98003530 con 0x7f0a98009f00 2024-08-19T10:51:11.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 10 monclient: _finish_auth 0 2024-08-19T10:51:11.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 10 monclient: _check_auth_tickets 2024-08-19T10:51:11.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab8f2a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:11.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0aba98d640 5 monclient: authenticate success, global_id 44812 2024-08-19T10:51:11.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0aba98d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:11.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.853+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980042e0 0x7f0a98004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:51:11.876 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=0 2024-08-19T10:51:11.876 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 0 -ne 0 ]] 2024-08-19T10:51:11.876 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:51:11.876 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls' 13 2024-08-19T10:51:11.876 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls' 2024-08-19T10:51:11.876 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=13 2024-08-19T10:51:11.877 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls 2024-08-19T10:51:11.878 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls 2024-08-19T10:51:11.878 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth ls 2024-08-19T10:51:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.861+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a98009f00 msgr2=0x7f0a9800c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.861+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a98009f00 0x7f0a9800c2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40a5a40 tx=0x7f0aa401d300 comp rx=0 tx=0).ready entity=mon.7 client_cookie=b18678f6a0c7de0a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0aa40b3070 con 0x7f0a98009f00 2024-08-19T10:51:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0ab2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0ab2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:51:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0ab2ffd640 10 monclient: dump: 2024-08-19T10:51:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:11.880 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:11.881 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:11.882 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa40ac780 con 0x7f0a98009f00 2024-08-19T10:51:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0ab2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0ab2ffd640 1 -- 172.21.15.89:0/1570803285 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0aa400eaf0 con 0x7f0a98009f00 2024-08-19T10:51:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0ab2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0ab2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:51:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0ab2ffd640 10 monclient: dump: 2024-08-19T10:51:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:11.882 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:11.883 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 4 set_mon_vals no callback set 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 10 monclient: shutdown 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 1 -- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a98009f00 msgr2=0x7f0a9800c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a98009f00 0x7f0a9800c2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40a5a40 tx=0x7f0aa401d300 comp rx=0 tx=0).stop 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 1 -- 172.21.15.89:0/1570803285 shutdown_connections 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0a98009f00 0x7f0a9800c2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a98004ef0 0x7f0a980099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 1 --2- 172.21.15.89:0/1570803285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a980042e0 0x7f0a98004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:11.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 1 -- 172.21.15.89:0/1570803285 >> 172.21.15.89:0/1570803285 conn(0x7f0ab4069ae0 msgr2=0x7f0ab40abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 1 -- 172.21.15.89:0/1570803285 shutdown_connections 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.873+0000 7f0aba98d640 1 -- 172.21.15.89:0/1570803285 wait complete. 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient: build_initial_monmap 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient: monmap: 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:11.885 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:11.886 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 1 Processor -- start 2024-08-19T10:51:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 1 -- start start 2024-08-19T10:51:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient: init 2024-08-19T10:51:11.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient: _add_conns ranks=[8,6,2,4,0,1,5,3,7] 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a94008d00 0x7f0ab41374e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient(hunting): picked mon.noname-e con 0x7f0a94008d00 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 0x7f0ab4137a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient(hunting): picked mon.noname-d con 0x7f0a980042e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 0x7f0ab4124080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient(hunting): picked mon.noname-b con 0x7f0a98004ef0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab40a5680 con 0x7f0a98004ef0 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f0ab40a5500 con 0x7f0a980042e0 2024-08-19T10:51:11.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f0ab40a5800 con 0x7f0a94008d00 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0aba98d640 10 monclient(hunting): authenticate will time out at 1721.698074s 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab37fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 0x7f0ab4137a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab3fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a94008d00 0x7f0ab41374e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab37fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 0x7f0ab4137a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:41046/0 (socket says 172.21.15.89:41046) 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1002861348 learned_addr learned my addr 172.21.15.89:0/1002861348 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request con 0x7f0a980042e0 auth_method 0 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request con 0x7f0a94008d00 auth_method 0 2024-08-19T10:51:11.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 msgr2=0x7f0ab4137a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:51:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 msgr2=0x7f0ab4137a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:51:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 0x7f0ab4124080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 0x7f0ab4137a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request con 0x7f0a98004ef0 auth_method 0 2024-08-19T10:51:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a94008d00 msgr2=0x7f0ab41374e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:51:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:11.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 0x7f0ab4137a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab3fff640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a94008d00 msgr2=0x7f0ab41374e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:51:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a94008d00 0x7f0ab41374e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a94008d00 0x7f0ab41374e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 msgr2=0x7f0ab4124080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:51:11.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 msgr2=0x7f0ab4124080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:51:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 0x7f0ab4124080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:11.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:11.877+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 0x7f0ab4124080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 0x7f0ab4137a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a94008d00 0x7f0ab41374e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request con 0x7f0a980042e0 auth_method 0 2024-08-19T10:51:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request con 0x7f0a94008d00 auth_method 0 2024-08-19T10:51:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 0x7f0ab4124080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request con 0x7f0a98004ef0 auth_method 0 2024-08-19T10:51:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab8f2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab8f2a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 msgr2=0x7f0ab4124080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab8f2a640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 msgr2=0x7f0ab4124080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 0x7f0ab4124080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab8f2a640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 0x7f0ab4124080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 10 monclient(hunting): handle_auth_done global_id 44880 payload 824 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a94008d00 msgr2=0x7f0ab41374e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a94008d00 0x7f0ab41374e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 msgr2=0x7f0ab4124080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:51:12.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 0x7f0ab4124080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 10 monclient: _finish_hunting 0 2024-08-19T10:51:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 1 monclient: found mon.noname-d 2024-08-19T10:51:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:51:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1002861348 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab4138780 con 0x7f0a980042e0 2024-08-19T10:51:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a94008d00 0x7f0ab41374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 msgr2=0x7f0ab4137a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab37fe640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 0x7f0ab4137a20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c019480 tx=0x7f0a9c016370 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 1 -- 172.21.15.89:0/1002861348 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0a9c00f040 con 0x7f0a980042e0 2024-08-19T10:51:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 10 monclient: dump: 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 10 monclient: _finish_auth 0 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0aba98d640 5 monclient: authenticate success, global_id 44880 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 1 -- 172.21.15.89:0/1002861348 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9c02a070 con 0x7f0a980042e0 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0aba98d640 10 monclient: _renew_subs 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0aba98d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0aba98d640 1 -- 172.21.15.89:0/1002861348 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab4137f60 con 0x7f0a980042e0 2024-08-19T10:51:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 1 -- 172.21.15.89:0/1002861348 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0a9c0044a0 con 0x7f0a980042e0 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab0ff9640 10 monclient: dump: 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.087 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0aba98d640 10 monclient: _renew_subs 2024-08-19T10:51:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0aba98d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0ab27fc640 4 set_mon_vals no callback set 2024-08-19T10:51:12.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.081+0000 7f0aba98d640 1 -- 172.21.15.89:0/1002861348 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab4125b70 con 0x7f0a980042e0 2024-08-19T10:51:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.085+0000 7f0aba98d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.085+0000 7f0aba98d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.085+0000 7f0aba98d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:12.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.085+0000 7f0aba98d640 1 -- 172.21.15.89:0/1002861348 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab40a6f90 con 0x7f0a980042e0 2024-08-19T10:51:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.085+0000 7f0ab0ff9640 1 -- 172.21.15.89:0/1002861348 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0a9c004030 con 0x7f0a980042e0 2024-08-19T10:51:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.085+0000 7f0ab0ff9640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0a94040050 0x7f0a94042510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.085+0000 7f0ab0ff9640 1 -- 172.21.15.89:0/1002861348 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(272..272 src has 246..272) ==== 7657+0+0 (secure 0 0 0) 0x7f0a9c050d90 con 0x7f0a980042e0 2024-08-19T10:51:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.085+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0a94040050 0x7f0a94042510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.085+0000 7f0ab3fff640 10 monclient: get_auth_request con 0x7f0a94040050 auth_method 0 2024-08-19T10:51:12.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.089+0000 7f0ab3fff640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0a94040050 0x7f0a94042510 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0096d0 tx=0x7f0aac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:12.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.093+0000 7f0ab0ff9640 1 -- 172.21.15.89:0/1002861348 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0a9c033020 con 0x7f0a980042e0 2024-08-19T10:51:12.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.093+0000 7f0ab0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:12.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.093+0000 7f0ab0ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:12.223 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=13 2024-08-19T10:51:12.223 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 13 -ne 13 ]] 2024-08-19T10:51:12.224 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:51:12.224 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export' 13 2024-08-19T10:51:12.224 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export' 2024-08-19T10:51:12.224 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=13 2024-08-19T10:51:12.224 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export 2024-08-19T10:51:12.224 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export 2024-08-19T10:51:12.224 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth export 2024-08-19T10:51:12.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.325+0000 7f0aba98d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:51:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.325+0000 7f0aba98d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:51:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.325+0000 7f0aba98d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.325+0000 7f0aba98d640 1 -- 172.21.15.89:0/1002861348 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0ab4064590 con 0x7f0a980042e0 2024-08-19T10:51:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.325+0000 7f0ab0ff9640 1 -- 172.21.15.89:0/1002861348 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7f0ab4138780 con 0x7f0a980042e0 2024-08-19T10:51:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.325+0000 7f0ab0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:51:12.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.325+0000 7f0ab0ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:12.327 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":110,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:51:12.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0a94040050 msgr2=0x7f0a94042510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0a94040050 0x7f0a94042510 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0096d0 tx=0x7f0aac009040 comp rx=0 tx=0).stop 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 10 monclient: shutdown 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 -- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 msgr2=0x7f0ab4137a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 0x7f0ab4137a20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c019480 tx=0x7f0a9c016370 comp rx=0 tx=0).stop 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 -- 172.21.15.89:0/1002861348 shutdown_connections 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0a98004ef0 0x7f0ab4124080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0a980042e0 0x7f0ab4137a20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0a94040050 0x7f0a94042510 secure :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0096d0 tx=0x7f0aac009040 comp rx=0 tx=0).stop 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 --2- 172.21.15.89:0/1002861348 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0a94008d00 0x7f0ab41374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 -- 172.21.15.89:0/1002861348 >> 172.21.15.89:0/1002861348 conn(0x7f0ab4069ae0 msgr2=0x7f0ab40a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 -- 172.21.15.89:0/1002861348 shutdown_connections 2024-08-19T10:51:12.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.329+0000 7f0aba98d640 1 -- 172.21.15.89:0/1002861348 wait complete. 2024-08-19T10:51:12.345 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:51:12.345 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:51:12.345 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:51:12.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient: get_monmap_and_config 2024-08-19T10:51:12.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient: build_initial_monmap 2024-08-19T10:51:12.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient: monmap: 2024-08-19T10:51:12.460 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:12.460 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.460 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:12.466276+0000 2024-08-19T10:51:12.460 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:12.466276+0000 2024-08-19T10:51:12.461 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:12.461 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.461 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:12.461 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:12.461 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:12.461 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:12.461 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:12.461 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:12.461 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:12.461 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:12.461 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:12.462 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:12.462 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:12.462 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 1 Processor -- start 2024-08-19T10:51:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 1 -- start start 2024-08-19T10:51:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient: init 2024-08-19T10:51:12.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient: _add_conns ranks=[10,6,5,4,2,8,7,0,3,9,1] 2024-08-19T10:51:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6b000a3620 0x7f6b000a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient(hunting): picked mon.noname-f con 0x7f6b000a3620 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:12.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b0009c9e0 0x7f6b000a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient(hunting): picked mon.noname-d con 0x7f6b0009c9e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b001373f0 0x7f6b00137830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient(hunting): picked mon.noname-h con 0x7f6b001373f0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6b0009dab0 con 0x7f6b000a3620 2024-08-19T10:51:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6b000a1f20 con 0x7f6b001373f0 2024-08-19T10:51:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6b000a4480 con 0x7f6b0009c9e0 2024-08-19T10:51:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6b052bd640 10 monclient(hunting): authenticate will time out at 1722.282917s 2024-08-19T10:51:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b0009c9e0 0x7f6b000a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b0009c9e0 0x7f6b000a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:41060/0 (socket says 172.21.15.89:41060) 2024-08-19T10:51:12.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 1 -- 172.21.15.89:0/14504614 learned_addr learned my addr 172.21.15.89:0/14504614 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:12.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 10 monclient(hunting): get_auth_request con 0x7f6b0009c9e0 auth_method 0 2024-08-19T10:51:12.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6aff7fe640 1 --2- 172.21.15.89:0/14504614 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b001373f0 0x7f6b00137830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6aff7fe640 10 monclient(hunting): get_auth_request con 0x7f6b001373f0 auth_method 0 2024-08-19T10:51:12.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6aff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6aff7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6aff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afeffd640 1 --2- 172.21.15.89:0/14504614 >> v2:172.21.15.190:3300/0 conn(0x7f6b000a3620 0x7f6b000a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afeffd640 10 monclient(hunting): get_auth_request con 0x7f6b000a3620 auth_method 0 2024-08-19T10:51:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_done global_id 44898 payload 274 2024-08-19T10:51:12.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 1 -- 172.21.15.89:0/14504614 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b001373f0 msgr2=0x7f6b00137830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:12.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 1 --2- 172.21.15.89:0/14504614 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b001373f0 0x7f6b00137830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 1 -- 172.21.15.89:0/14504614 >> v2:172.21.15.190:3300/0 conn(0x7f6b000a3620 msgr2=0x7f6b000a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:12.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 1 --2- 172.21.15.89:0/14504614 >> v2:172.21.15.190:3300/0 conn(0x7f6b000a3620 0x7f6b000a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:51:12.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 1 monclient: found mon.noname-d 2024-08-19T10:51:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:51:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.461+0000 7f6afe7fc640 1 -- 172.21.15.89:0/14504614 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b000a1790 con 0x7f6b0009c9e0 2024-08-19T10:51:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afe7fc640 1 -- 172.21.15.89:0/14504614 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b0009c9e0 msgr2=0x7f6b000a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afe7fc640 1 --2- 172.21.15.89:0/14504614 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b0009c9e0 0x7f6b000a3f40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009e80 tx=0x7f6ae801c8a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7dd840798982da99 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 1 -- 172.21.15.89:0/14504614 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ae80090d0 con 0x7f6b0009c9e0 2024-08-19T10:51:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:51:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 10 monclient: dump: 2024-08-19T10:51:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.489 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 10 monclient: _finish_auth 0 2024-08-19T10:51:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:51:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 1 -- 172.21.15.89:0/14504614 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae80a5b50 con 0x7f6b0009c9e0 2024-08-19T10:51:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6b052bd640 5 monclient: authenticate success, global_id 44898 2024-08-19T10:51:12.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6b052bd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:12.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:12.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6b052bd640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:12.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6b052bd640 4 set_mon_vals no callback set 2024-08-19T10:51:12.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 1 -- 172.21.15.89:0/14504614 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ae80a5d00 con 0x7f6b0009c9e0 2024-08-19T10:51:12.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.465+0000 7f6afdffb640 10 monclient: dump: 2024-08-19T10:51:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.494 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.494 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.494 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.494 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.494 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient: shutdown 2024-08-19T10:51:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 -- 172.21.15.89:0/14504614 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b0009c9e0 msgr2=0x7f6b000a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 --2- 172.21.15.89:0/14504614 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b0009c9e0 0x7f6b000a3f40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009e80 tx=0x7f6ae801c8a0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 -- 172.21.15.89:0/14504614 shutdown_connections 2024-08-19T10:51:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 --2- 172.21.15.89:0/14504614 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b001373f0 0x7f6b00137830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 --2- 172.21.15.89:0/14504614 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b0009c9e0 0x7f6b000a3f40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 --2- 172.21.15.89:0/14504614 >> v2:172.21.15.190:3300/0 conn(0x7f6b000a3620 0x7f6b000a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 -- 172.21.15.89:0/14504614 >> 172.21.15.89:0/14504614 conn(0x7f6b000a8210 msgr2=0x7f6b000a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 -- 172.21.15.89:0/14504614 shutdown_connections 2024-08-19T10:51:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 -- 172.21.15.89:0/14504614 wait complete. 2024-08-19T10:51:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient: build_initial_monmap 2024-08-19T10:51:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient: monmap: 2024-08-19T10:51:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:12.499 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 Processor -- start 2024-08-19T10:51:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 -- start start 2024-08-19T10:51:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient: init 2024-08-19T10:51:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient: _add_conns ranks=[5,1,7,6,8,3,2,4,0] 2024-08-19T10:51:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b0009c9e0 0x7f6b00135e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient(hunting): picked mon.noname-h con 0x7f6b0009c9e0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:12.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b000a3620 0x7f6b00136390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient(hunting): picked mon.noname-f con 0x7f6b000a3620 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:51:12.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b001373f0 0x7f6b0011e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient(hunting): picked mon.noname-i con 0x7f6b001373f0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:12.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0009df40 con 0x7f6b000a3620 2024-08-19T10:51:12.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6b00138180 con 0x7f6b0009c9e0 2024-08-19T10:51:12.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6b00140e60 con 0x7f6b001373f0 2024-08-19T10:51:12.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:12.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6b052bd640 10 monclient(hunting): authenticate will time out at 1722.290036s 2024-08-19T10:51:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b0009c9e0 0x7f6b00135e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b0009c9e0 0x7f6b00135e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:53188/0 (socket says 172.21.15.89:53188) 2024-08-19T10:51:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 1 -- 172.21.15.89:0/2101809610 learned_addr learned my addr 172.21.15.89:0/2101809610 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 10 monclient(hunting): get_auth_request con 0x7f6b0009c9e0 auth_method 0 2024-08-19T10:51:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6aff7fe640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b001373f0 0x7f6b0011e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afe7fc640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b000a3620 0x7f6b00136390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6aff7fe640 10 monclient(hunting): get_auth_request con 0x7f6b001373f0 auth_method 0 2024-08-19T10:51:12.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6aff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6aff7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6aff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afe7fc640 10 monclient(hunting): get_auth_request con 0x7f6b000a3620 auth_method 0 2024-08-19T10:51:12.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afe7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6aff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6aff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6aff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 10 monclient(hunting): handle_auth_done global_id 7079 payload 824 2024-08-19T10:51:12.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 1 -- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b001373f0 msgr2=0x7f6b0011e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6b001373f0 0x7f6b0011e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 1 -- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b000a3620 msgr2=0x7f6b00136390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b000a3620 0x7f6b00136390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 10 monclient: _finish_hunting 0 2024-08-19T10:51:12.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 1 monclient: found mon.noname-h 2024-08-19T10:51:12.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:51:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.469+0000 7f6afeffd640 1 -- 172.21.15.89:0/2101809610 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b000a6a90 con 0x7f6b0009c9e0 2024-08-19T10:51:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afeffd640 1 -- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b0009c9e0 msgr2=0x7f6b00135e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afeffd640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b0009c9e0 0x7f6b00135e50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6af800d830 tx=0x7f6af800dc30 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6af800bce0 con 0x7f6b0009c9e0 2024-08-19T10:51:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:51:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 10 monclient: dump: 2024-08-19T10:51:12.508 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.509 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.509 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.509 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.509 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.509 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.509 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.509 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.509 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.510 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 10 monclient: _finish_auth 0 2024-08-19T10:51:12.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6b052bd640 5 monclient: authenticate success, global_id 7079 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6b052bd640 10 monclient: _renew_subs 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af8019070 con 0x7f6b0009c9e0 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6b052bd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6b052bd640 1 -- 172.21.15.89:0/2101809610 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b00121640 con 0x7f6b0009c9e0 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6af8014430 con 0x7f6b0009c9e0 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 10 monclient: dump: 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afcff9640 4 set_mon_vals no callback set 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6b052bd640 10 monclient: _renew_subs 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6b052bd640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6b052bd640 1 -- 172.21.15.89:0/2101809610 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b0013d660 con 0x7f6b0009c9e0 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6b052bd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6b052bd640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011e7a0 0x7f6b0011eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6b052bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6b052bd640 1 -- 172.21.15.89:0/2101809610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0011fd40 con 0x7f6b0011e7a0 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afe7fc640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011e7a0 0x7f6b0011eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afe7fc640 10 monclient: get_auth_request con 0x7f6b0011e7a0 auth_method 0 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afe7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_done global_id 7092 payload 824 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6afe7fc640 1 -- 172.21.15.89:0/2101809610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b00064590 con 0x7f6b0011e7a0 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6af800b870 con 0x7f6b0009c9e0 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ae003be00 0x7f6ae003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.473+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(272..272 src has 246..272) ==== 7657+0+0 (secure 0 0 0) 0x7f6af800f040 con 0x7f6b0009c9e0 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6aff7fe640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ae003be00 0x7f6ae003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6aff7fe640 10 monclient: get_auth_request con 0x7f6ae003be00 auth_method 0 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6afe7fc640 1 -- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011e7a0 msgr2=0x7f6b0011eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6afe7fc640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011e7a0 0x7f6b0011eb80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80ae000 tx=0x7f6ae8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ae80bd070 con 0x7f6b0011e7a0 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6af77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6af77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6af77fe640 10 monclient: dump: 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6ae80a58f0 con 0x7f6b0011e7a0 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6af77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6af77fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011e7a0 msgr2=0x7f6b0011eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6af77fe640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011e7a0 0x7f6b0011eb80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80ae000 tx=0x7f6ae8019640 comp rx=0 tx=0).stop 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011e7a0 msgr2=0x7f6b0011eb80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6af77fe640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011e7a0 0x7f6b0011eb80 secure :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80ae000 tx=0x7f6ae8019640 comp rx=0 tx=0).stop 2024-08-19T10:51:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.477+0000 7f6aff7fe640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ae003be00 0x7f6ae003e2c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6af00078d0 tx=0x7f6af0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.481+0000 7f6b052bd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011f0c0 0x7f6b00138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 1 -- 172.21.15.89:0/2101809610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00137830 con 0x7f6b0011f0c0 2024-08-19T10:51:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011f0c0 0x7f6b00138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 10 monclient: get_auth_request con 0x7f6b0011f0c0 auth_method 0 2024-08-19T10:51:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 10 monclient(hunting): handle_auth_done global_id 7101 payload 824 2024-08-19T10:51:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 1 -- 172.21.15.89:0/2101809610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6b0006dc30 con 0x7f6b0011f0c0 2024-08-19T10:51:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 1 -- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011f0c0 msgr2=0x7f6b00138400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6afe7fc640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011f0c0 0x7f6b00138400 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6ae801c6d0 tx=0x7f6ae80a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ae80047b0 con 0x7f6b0011f0c0 2024-08-19T10:51:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6af77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6af77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6af77fe640 10 monclient: dump: 2024-08-19T10:51:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9604 (secure 0 0 0) 0x7f6ae80047b0 con 0x7f6b0011f0c0 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6af77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6af77fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011f0c0 msgr2=0x7f6b00138400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6af77fe640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011f0c0 0x7f6b00138400 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6ae801c6d0 tx=0x7f6ae80a7000 comp rx=0 tx=0).stop 2024-08-19T10:51:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6af77fe640 1 -- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011f0c0 msgr2=0x7f6b00138400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6af77fe640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b0011f0c0 0x7f6b00138400 secure :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6ae801c6d0 tx=0x7f6ae80a7000 comp rx=0 tx=0).stop 2024-08-19T10:51:12.526 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:12.526 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:12.526 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:12.526 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:51:12.526 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:12.526 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 802298, 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:12.527 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:12.529 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:12.530 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:12.531 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:12.531 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:12.531 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:12.531 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:12.531 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:12.531 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:12.531 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:12.532 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:12.532 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:12.532 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:12.532 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:12.532 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:12.532 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:12.532 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:12.532 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:12.532 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:12.532 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:12.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.534 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.535 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:12.536 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:12.536 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.536 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.536 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:12.536 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:12.536 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.536 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.536 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.536 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:12.537 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:12.538 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.540 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:12.541 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:12.542 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:12.544 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:12.545 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:12.546 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:12.547 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:12.549 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:12.550 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:12.551 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:51:12.552 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:12.552 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:12.552 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:12.552 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:12.552 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:12.552 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:12.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:12.552 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:12.552 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 1 -- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ae003be00 msgr2=0x7f6ae003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ae003be00 0x7f6ae003e2c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6af00078d0 tx=0x7f6af0008040 comp rx=0 tx=0).stop 2024-08-19T10:51:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 10 monclient: shutdown 2024-08-19T10:51:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 1 -- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b0009c9e0 msgr2=0x7f6b00135e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 1 --2- 172.21.15.89:0/2101809610 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b0009c9e0 0x7f6b00135e50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6af800d830 tx=0x7f6af800dc30 comp rx=0 tx=0).stop 2024-08-19T10:51:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6aff7fe640 1 -- 172.21.15.89:0/2101809610 reap_dead start 2024-08-19T10:51:12.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6aff7fe640 1 -- 172.21.15.89:0/2101809610 reap_dead start 2024-08-19T10:51:12.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 1 -- 172.21.15.89:0/2101809610 shutdown_connections 2024-08-19T10:51:12.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 1 -- 172.21.15.89:0/2101809610 >> 172.21.15.89:0/2101809610 conn(0x7f6b000a8210 msgr2=0x7f6b000a7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:12.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 1 -- 172.21.15.89:0/2101809610 shutdown_connections 2024-08-19T10:51:12.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.485+0000 7f6b052bd640 1 -- 172.21.15.89:0/2101809610 wait complete. 2024-08-19T10:51:12.555 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:12.555 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:51:12.555 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:51:12.593 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=13 2024-08-19T10:51:12.594 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 13 -ne 13 ]] 2024-08-19T10:51:12.594 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:51:12.594 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo' 13 2024-08-19T10:51:12.594 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo 2024-08-19T10:51:12.595 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo' 2024-08-19T10:51:12.595 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=13 2024-08-19T10:51:12.595 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo 2024-08-19T10:51:12.595 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar auth del client.foo 2024-08-19T10:51:12.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.677+0000 7fbf30d1e640 10 monclient: get_monmap_and_config 2024-08-19T10:51:12.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.677+0000 7fbf30d1e640 10 monclient: build_initial_monmap 2024-08-19T10:51:12.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient: monmap: 2024-08-19T10:51:12.679 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:12.679 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.679 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:12.684967+0000 2024-08-19T10:51:12.679 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:12.684967+0000 2024-08-19T10:51:12.679 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:12.679 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.679 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:12.679 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:12.680 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:12.680 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:12.680 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:12.680 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:12.680 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:12.680 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:12.680 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:12.680 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:12.680 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:12.680 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 1 Processor -- start 2024-08-19T10:51:12.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 1 -- start start 2024-08-19T10:51:12.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient: init 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient: _add_conns ranks=[9,6,5,0,3,1,7,4,10,2,8] 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbf2c0a6860 0x7fbf2c0a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient(hunting): picked mon.noname-a con 0x7fbf2c0a6860 addr v2:172.21.15.89:3300/0 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbf2c0a54d0 0x7fbf2c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient(hunting): picked mon.noname-d con 0x7fbf2c0a54d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbf2c0a58b0 0x7fbf2c0a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient(hunting): picked mon.noname-h con 0x7fbf2c0a58b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fbf2c0a76c0 con 0x7fbf2c0a6860 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fbf2c0a4c90 con 0x7fbf2c0a58b0 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fbf2c139620 con 0x7fbf2c0a54d0 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf30d1e640 10 monclient(hunting): authenticate will time out at 1722.501477s 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbf2c0a6860 0x7fbf2c0a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbf2c0a6860 0x7fbf2c0a81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36484/0 (socket says 172.21.15.89:36484) 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 1 -- 172.21.15.89:0/2605824827 learned_addr learned my addr 172.21.15.89:0/2605824827 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf29d9b640 1 --2- 172.21.15.89:0/2605824827 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbf2c0a54d0 0x7fbf2c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 10 monclient(hunting): get_auth_request con 0x7fbf2c0a6860 auth_method 0 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf29d9b640 10 monclient(hunting): get_auth_request con 0x7fbf2c0a54d0 auth_method 0 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf29d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf29d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf29d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf29d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf29d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf29d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2ad9d640 1 --2- 172.21.15.89:0/2605824827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbf2c0a58b0 0x7fbf2c0a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_done global_id 7110 payload 274 2024-08-19T10:51:12.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 1 -- 172.21.15.89:0/2605824827 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbf2c0a54d0 msgr2=0x7fbf2c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:12.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/2605824827 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbf2c0a54d0 0x7fbf2c0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 1 -- 172.21.15.89:0/2605824827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbf2c0a58b0 msgr2=0x7fbf2c0a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:12.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/2605824827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbf2c0a58b0 0x7fbf2c0a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:51:12.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 1 monclient: found mon.noname-a 2024-08-19T10:51:12.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:12.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:51:12.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2a59c640 1 -- 172.21.15.89:0/2605824827 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c139820 con 0x7fbf2c0a6860 2024-08-19T10:51:12.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2ad9d640 10 monclient: get_auth_request con 0x7fbf2c0a58b0 auth_method 0 2024-08-19T10:51:12.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf2ad9d640 1 --2- 172.21.15.89:0/2605824827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbf2c0a58b0 0x7fbf2c0a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:12.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.681+0000 7fbf29d9b640 1 --2- 172.21.15.89:0/2605824827 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbf2c0a54d0 0x7fbf2c0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:12.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2a59c640 1 -- 172.21.15.89:0/2605824827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf2c0a6860 msgr2=0x7fbf2c0a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:12.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/2605824827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf2c0a6860 0x7fbf2c0a81b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009870 tx=0x7fbf1c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880797dfb4eb9c2e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:12.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 1 -- 172.21.15.89:0/2605824827 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbf1c0b0070 con 0x7fbf2c0a6860 2024-08-19T10:51:12.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:51:12.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 10 monclient: dump: 2024-08-19T10:51:12.692 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.693 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 10 monclient: _finish_auth 0 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 10 monclient: _check_auth_tickets 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 1 -- 172.21.15.89:0/2605824827 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1c0a9980 con 0x7fbf2c0a6860 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 5 monclient: authenticate success, global_id 7110 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 1 -- 172.21.15.89:0/2605824827 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbf1c0aa3a0 con 0x7fbf2c0a6860 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 4 set_mon_vals no callback set 2024-08-19T10:51:12.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf2959a640 10 monclient: dump: 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.695 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.696 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 10 monclient: shutdown 2024-08-19T10:51:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 1 -- 172.21.15.89:0/2605824827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf2c0a6860 msgr2=0x7fbf2c0a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 1 --2- 172.21.15.89:0/2605824827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf2c0a6860 0x7fbf2c0a81b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009870 tx=0x7fbf1c01c4b0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 1 -- 172.21.15.89:0/2605824827 shutdown_connections 2024-08-19T10:51:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 1 --2- 172.21.15.89:0/2605824827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbf2c0a58b0 0x7fbf2c0a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 1 --2- 172.21.15.89:0/2605824827 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbf2c0a54d0 0x7fbf2c0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 1 --2- 172.21.15.89:0/2605824827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf2c0a6860 0x7fbf2c0a81b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 1 -- 172.21.15.89:0/2605824827 >> 172.21.15.89:0/2605824827 conn(0x7fbf2c069a00 msgr2=0x7fbf2c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 1 -- 172.21.15.89:0/2605824827 shutdown_connections 2024-08-19T10:51:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 1 -- 172.21.15.89:0/2605824827 wait complete. 2024-08-19T10:51:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 10 monclient: build_initial_monmap 2024-08-19T10:51:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 10 monclient: monmap: 2024-08-19T10:51:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:12.697 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:12.698 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.685+0000 7fbf30d1e640 1 Processor -- start 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 1 -- start start 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient: init 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient: _add_conns ranks=[3,8,2,6,1,7,0,5,4] 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf2c0a54d0 0x7fbf2c09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient(hunting): picked mon.noname-g con 0x7fbf2c0a54d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf2c0a58b0 0x7fbf2c09cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient(hunting): picked mon.noname-e con 0x7fbf2c0a58b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c0a6860 0x7fbf2c09d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient(hunting): picked mon.noname-b con 0x7fbf2c0a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c143490 con 0x7fbf2c0a6860 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c143310 con 0x7fbf2c0a54d0 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fbf2c143610 con 0x7fbf2c0a58b0 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient(hunting): authenticate will time out at 1722.508527s 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf2c0a58b0 0x7fbf2c09cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf2c0a58b0 0x7fbf2c09cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48476/0 (socket says 172.21.15.89:48476) 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 1 -- 172.21.15.89:0/1551204227 learned_addr learned my addr 172.21.15.89:0/1551204227 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 10 monclient(hunting): get_auth_request con 0x7fbf2c0a58b0 auth_method 0 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2ad9d640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c0a6860 0x7fbf2c09d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2ad9d640 10 monclient(hunting): get_auth_request con 0x7fbf2c0a6860 auth_method 0 2024-08-19T10:51:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf2c0a54d0 0x7fbf2c09c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2a59c640 10 monclient(hunting): get_auth_request con 0x7fbf2c0a54d0 auth_method 0 2024-08-19T10:51:12.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2ad9d640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c0a6860 msgr2=0x7fbf2c09d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:51:12.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2ad9d640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c0a6860 msgr2=0x7fbf2c09d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:51:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2ad9d640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c0a6860 0x7fbf2c09d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:12.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf2ad9d640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c0a6860 0x7fbf2c09d460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 10 monclient(hunting): handle_auth_done global_id 7010 payload 824 2024-08-19T10:51:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf2c0a54d0 msgr2=0x7fbf2c09c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf2c0a54d0 0x7fbf2c09c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c0a6860 msgr2=0x7fbf2c09d460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:51:12.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c0a6860 0x7fbf2c09d460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:12.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:51:12.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 1 monclient: found mon.noname-e 2024-08-19T10:51:12.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:12.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:51:12.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 1 -- 172.21.15.89:0/1551204227 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c13f3e0 con 0x7fbf2c0a58b0 2024-08-19T10:51:12.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf2c0a58b0 msgr2=0x7fbf2c09cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf29d9b640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf2c0a58b0 0x7fbf2c09cf20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000ebe0 tx=0x7fbf2000c520 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbf20017070 con 0x7fbf2c0a58b0 2024-08-19T10:51:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:51:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 10 monclient: dump: 2024-08-19T10:51:12.707 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.708 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.709 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.710 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.710 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.710 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.710 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.710 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.711 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 10 monclient: _finish_auth 0 2024-08-19T10:51:12.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 10 monclient: _check_auth_tickets 2024-08-19T10:51:12.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:12.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 5 monclient: authenticate success, global_id 7010 2024-08-19T10:51:12.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf2000ce40 con 0x7fbf2c0a58b0 2024-08-19T10:51:12.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:12.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbf20013d50 con 0x7fbf2c0a58b0 2024-08-19T10:51:12.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:12.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:51:12.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf1affd640 10 monclient: dump: 2024-08-19T10:51:12.713 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:12.713 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:12.713 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.714 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:12.714 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:12.714 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:12.714 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:12.714 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:12.714 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:12.715 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:12.715 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:12.715 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:12.715 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:12.715 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:12.716 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:12.716 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:12.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient: _renew_subs 2024-08-19T10:51:12.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:51:12.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf28d99640 4 set_mon_vals no callback set 2024-08-19T10:51:12.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 1 -- 172.21.15.89:0/1551204227 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c09ef20 con 0x7fbf2c0a58b0 2024-08-19T10:51:12.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient: _renew_subs 2024-08-19T10:51:12.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:51:12.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.689+0000 7fbf30d1e640 1 -- 172.21.15.89:0/1551204227 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c09ec40 con 0x7fbf2c0a58b0 2024-08-19T10:51:12.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf30d1e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:12.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf30d1e640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf30d1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:12.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf30d1e640 1 -- 172.21.15.89:0/1551204227 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c09ff20 con 0x7fbf2c09dce0 2024-08-19T10:51:12.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2a59c640 10 monclient: get_auth_request con 0x7fbf2c09dce0 auth_method 0 2024-08-19T10:51:12.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:12.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbf2001a070 con 0x7fbf2c0a58b0 2024-08-19T10:51:12.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2a59c640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 msgr2=0x7fbf2c1335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:51:12.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2a59c640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 msgr2=0x7fbf2c1335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:51:12.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:12.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:12.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf1affd640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf0c03be00 0x7fbf0c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:12.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(272..272 src has 246..272) ==== 7657+0+0 (secure 0 0 0) 0x7fbf2000f040 con 0x7fbf2c0a58b0 2024-08-19T10:51:12.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2ad9d640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf0c03be00 0x7fbf0c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2ad9d640 10 monclient: get_auth_request con 0x7fbf0c03be00 auth_method 0 2024-08-19T10:51:12.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.693+0000 7fbf2ad9d640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf0c03be00 0x7fbf0c03e2c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbf24007570 tx=0x7fbf24007500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:12.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.897+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:12.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.897+0000 7fbf2a59c640 10 monclient: get_auth_request con 0x7fbf2c09dce0 auth_method 0 2024-08-19T10:51:12.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.897+0000 7fbf2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:12.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.897+0000 7fbf2a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:12.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.897+0000 7fbf2a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:12.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.897+0000 7fbf2a59c640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 msgr2=0x7fbf2c1335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:51:12.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.897+0000 7fbf2a59c640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 msgr2=0x7fbf2c1335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:51:12.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.897+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:12.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:12.897+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:12.958 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=13 2024-08-19T10:51:12.959 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 13 -ne 13 ]] 2024-08-19T10:51:12.959 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:51:12.959 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump' 13 2024-08-19T10:51:12.959 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump' 2024-08-19T10:51:12.959 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=13 2024-08-19T10:51:12.959 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump 2024-08-19T10:51:12.959 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump 2024-08-19T10:51:12.960 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar osd dump 2024-08-19T10:51:13.230 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=13 2024-08-19T10:51:13.231 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 13 -ne 13 ]] 2024-08-19T10:51:13.231 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:51:13.231 INFO:tasks.workunit.client.0.smithi190.stderr:+ export CEPH_ARGS=--rados-mon-op-timeout=300 2024-08-19T10:51:13.231 INFO:tasks.workunit.client.0.smithi190.stderr:+ CEPH_ARGS=--rados-mon-op-timeout=300 2024-08-19T10:51:13.231 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump' 110 2024-08-19T10:51:13.231 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump' 2024-08-19T10:51:13.231 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=110 2024-08-19T10:51:13.231 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump 2024-08-19T10:51:13.232 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump 2024-08-19T10:51:13.232 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar pg dump 2024-08-19T10:51:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:13.301+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:13.301+0000 7fbf2a59c640 10 monclient: get_auth_request con 0x7fbf2c09dce0 auth_method 0 2024-08-19T10:51:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:13.301+0000 7fbf2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:13.301+0000 7fbf2a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:13.301+0000 7fbf2a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:13.301+0000 7fbf2a59c640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 msgr2=0x7fbf2c1335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:51:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:13.301+0000 7fbf2a59c640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 msgr2=0x7fbf2c1335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:51:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:13.301+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:13.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:13.301+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:14.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 10 monclient: get_auth_request con 0x7fbf2c09dce0 auth_method 0 2024-08-19T10:51:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_done global_id 74099 payload 824 2024-08-19T10:51:14.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 1 -- 172.21.15.89:0/1551204227 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf2c0a81b0 con 0x7fbf2c09dce0 2024-08-19T10:51:14.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 msgr2=0x7fbf2c1335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0a6330 tx=0x7fbf1c019250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbf1c0ba070 con 0x7fbf2c09dce0 2024-08-19T10:51:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf1affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf1affd640 10 monclient: dump: 2024-08-19T10:51:14.105 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.106 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.107 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.107 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbf1c003b20 con 0x7fbf2c09dce0 2024-08-19T10:51:14.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf1affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:14.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf1affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:14.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 msgr2=0x7fbf2c1335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf1affd640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0a6330 tx=0x7fbf1c019250 comp rx=0 tx=0).stop 2024-08-19T10:51:14.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 msgr2=0x7fbf2c1335c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:14.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.105+0000 7fbf1affd640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c09dce0 0x7fbf2c1335c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf30d1e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf30d1e640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c13ecd0 0x7fbf2c133b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf30d1e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf30d1e640 1 -- 172.21.15.89:0/1551204227 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c064590 con 0x7fbf2c13ecd0 2024-08-19T10:51:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c13ecd0 0x7fbf2c133b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf2a59c640 10 monclient: get_auth_request con 0x7fbf2c13ecd0 auth_method 0 2024-08-19T10:51:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf2a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf2a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf2a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.113+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.117+0000 7fbf2a59c640 10 monclient(hunting): handle_auth_done global_id 74108 payload 824 2024-08-19T10:51:14.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.117+0000 7fbf2a59c640 1 -- 172.21.15.89:0/1551204227 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbf2c09f3b0 con 0x7fbf2c13ecd0 2024-08-19T10:51:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.117+0000 7fbf2a59c640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c13ecd0 msgr2=0x7fbf2c133b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.117+0000 7fbf2a59c640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c13ecd0 0x7fbf2c133b00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c01cd50 tx=0x7fbf1c009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.117+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbf1c0b0040 con 0x7fbf2c13ecd0 2024-08-19T10:51:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.117+0000 7fbf1affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.117+0000 7fbf1affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.117+0000 7fbf1affd640 10 monclient: dump: 2024-08-19T10:51:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.119 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.117+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7fbf1c0a9e70 con 0x7fbf2c13ecd0 2024-08-19T10:51:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf1affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf1affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c13ecd0 msgr2=0x7fbf2c133b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf1affd640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c13ecd0 0x7fbf2c133b00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c01cd50 tx=0x7fbf1c009240 comp rx=0 tx=0).stop 2024-08-19T10:51:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf1affd640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c13ecd0 msgr2=0x7fbf2c133b00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf1affd640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbf2c13ecd0 0x7fbf2c133b00 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c01cd50 tx=0x7fbf1c009240 comp rx=0 tx=0).stop 2024-08-19T10:51:14.121 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:14.121 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:14.121 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:14.121 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:14.121 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:14.121 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 10615, 2024-08-19T10:51:14.121 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:14.122 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:14.123 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:14.123 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.123 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.123 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.123 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.123 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.123 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.123 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.123 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.123 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.123 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.124 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:14.125 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.126 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.127 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.129 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:14.130 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.131 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.131 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.131 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:14.131 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:14.131 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.131 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.131 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.131 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:14.132 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.135 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.138 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:14.139 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:14.140 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.142 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.143 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.144 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.145 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:14.145 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:14.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf30d1e640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf0c03be00 msgr2=0x7fbf0c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf30d1e640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbf0c03be00 0x7fbf0c03e2c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbf24007570 tx=0x7fbf24007500 comp rx=0 tx=0).stop 2024-08-19T10:51:14.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf30d1e640 10 monclient: shutdown 2024-08-19T10:51:14.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf30d1e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:14.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf30d1e640 1 -- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf2c0a58b0 msgr2=0x7fbf2c09cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf30d1e640 1 --2- 172.21.15.89:0/1551204227 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbf2c0a58b0 0x7fbf2c09cf20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000ebe0 tx=0x7fbf2000c520 comp rx=0 tx=0).stop 2024-08-19T10:51:14.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf2ad9d640 1 -- 172.21.15.89:0/1551204227 reap_dead start 2024-08-19T10:51:14.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf2ad9d640 1 -- 172.21.15.89:0/1551204227 reap_dead start 2024-08-19T10:51:14.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf30d1e640 1 -- 172.21.15.89:0/1551204227 shutdown_connections 2024-08-19T10:51:14.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf30d1e640 1 -- 172.21.15.89:0/1551204227 >> 172.21.15.89:0/1551204227 conn(0x7fbf2c069a00 msgr2=0x7fbf2c134e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:14.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf30d1e640 1 -- 172.21.15.89:0/1551204227 shutdown_connections 2024-08-19T10:51:14.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.121+0000 7fbf30d1e640 1 -- 172.21.15.89:0/1551204227 wait complete. 2024-08-19T10:51:14.147 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:14.148 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:51:14.148 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:51:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient: get_monmap_and_config 2024-08-19T10:51:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient: build_initial_monmap 2024-08-19T10:51:14.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient: monmap: 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:14.285450+0000 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:14.285450+0000 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:14.280 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:14.281 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:14.281 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:14.281 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:14.281 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:14.281 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 1 Processor -- start 2024-08-19T10:51:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 1 -- start start 2024-08-19T10:51:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient: init 2024-08-19T10:51:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient: _add_conns ranks=[2,3,5,10,0,4,9,7,6,1,8] 2024-08-19T10:51:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23040a6860 0x7f23040a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient(hunting): picked mon.noname-b con 0x7f23040a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23040a7180 0x7f23040a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient(hunting): picked mon.noname-g con 0x7f23040a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2304139620 0x7f2304139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient(hunting): picked mon.noname-h con 0x7f2304139620 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f23040a8190 con 0x7f23040a6860 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f23040a33d0 con 0x7f23040a7180 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f23040a2ec0 con 0x7f2304139620 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f230941a640 10 monclient(hunting): authenticate will time out at 1724.102028s 2024-08-19T10:51:14.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23040a6860 0x7f23040a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23040a6860 0x7f23040a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35366/0 (socket says 172.21.15.89:35366) 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 1 -- 172.21.15.89:0/3521752500 learned_addr learned my addr 172.21.15.89:0/3521752500 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 10 monclient(hunting): get_auth_request con 0x7f23040a6860 auth_method 0 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f23037fe640 1 --2- 172.21.15.89:0/3521752500 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2304139620 0x7f2304139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 10 monclient(hunting): handle_auth_done global_id 74117 payload 274 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 1 -- 172.21.15.89:0/3521752500 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2304139620 msgr2=0x7f2304139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 1 --2- 172.21.15.89:0/3521752500 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2304139620 0x7f2304139a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 1 -- 172.21.15.89:0/3521752500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23040a7180 msgr2=0x7f23040a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:14.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f23027fc640 1 --2- 172.21.15.89:0/3521752500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23040a7180 0x7f23040a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 1 --2- 172.21.15.89:0/3521752500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23040a7180 0x7f23040a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:51:14.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 1 monclient: found mon.noname-b 2024-08-19T10:51:14.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:14.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:51:14.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f2302ffd640 1 -- 172.21.15.89:0/3521752500 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23040a4cd0 con 0x7f23040a6860 2024-08-19T10:51:14.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f23027fc640 10 monclient: get_auth_request con 0x7f23040a7180 auth_method 0 2024-08-19T10:51:14.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.281+0000 7f23027fc640 1 --2- 172.21.15.89:0/3521752500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23040a7180 0x7f23040a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2302ffd640 1 -- 172.21.15.89:0/3521752500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23040a6860 msgr2=0x7f23040a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2302ffd640 1 --2- 172.21.15.89:0/3521752500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23040a6860 0x7f23040a6c40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009e80 tx=0x7f22fc01c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1135d95387dd72 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 1 -- 172.21.15.89:0/3521752500 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22fc0091b0 con 0x7f23040a6860 2024-08-19T10:51:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:51:14.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 10 monclient: dump: 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.291 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 10 monclient: _finish_auth 0 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 1 -- 172.21.15.89:0/3521752500 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22fc0a6970 con 0x7f23040a6860 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 1 -- 172.21.15.89:0/3521752500 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22fc0af480 con 0x7f23040a6860 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 5 monclient: authenticate success, global_id 74117 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 4 set_mon_vals no callback set 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:14.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f2301ffb640 10 monclient: dump: 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.293 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 10 monclient: shutdown 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 1 -- 172.21.15.89:0/3521752500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23040a6860 msgr2=0x7f23040a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 1 --2- 172.21.15.89:0/3521752500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23040a6860 0x7f23040a6c40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009e80 tx=0x7f22fc01c910 comp rx=0 tx=0).stop 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 1 -- 172.21.15.89:0/3521752500 shutdown_connections 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 1 --2- 172.21.15.89:0/3521752500 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2304139620 0x7f2304139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 1 --2- 172.21.15.89:0/3521752500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23040a7180 0x7f23040a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 1 --2- 172.21.15.89:0/3521752500 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f23040a6860 0x7f23040a6c40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 1 -- 172.21.15.89:0/3521752500 >> 172.21.15.89:0/3521752500 conn(0x7f2304074b40 msgr2=0x7f23040abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 1 -- 172.21.15.89:0/3521752500 shutdown_connections 2024-08-19T10:51:14.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.285+0000 7f230941a640 1 -- 172.21.15.89:0/3521752500 wait complete. 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient: build_initial_monmap 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient: monmap: 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:14.295 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 1 Processor -- start 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 1 -- start start 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient: init 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient: _add_conns ranks=[5,7,4,3,1,2,6,0,8] 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f23040a6860 0x7f23041349f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient(hunting): picked mon.noname-h con 0x7f23040a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f23040a7180 0x7f2304134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient(hunting): picked mon.noname-i con 0x7f23040a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2304139620 0x7f230409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient(hunting): picked mon.noname-c con 0x7f2304139620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f2304143210 con 0x7f2304139620 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2304143090 con 0x7f23040a6860 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2304143390 con 0x7f23040a7180 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f230941a640 10 monclient(hunting): authenticate will time out at 1724.109729s 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2304139620 0x7f230409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2304139620 0x7f230409c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38024/0 (socket says 172.21.15.89:38024) 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 1 -- 172.21.15.89:0/926086158 learned_addr learned my addr 172.21.15.89:0/926086158 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:14.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 10 monclient(hunting): get_auth_request con 0x7f2304139620 auth_method 0 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23027fc640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f23040a7180 0x7f2304134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f2302ffd640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f23040a6860 0x7f23041349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23027fc640 10 monclient(hunting): get_auth_request con 0x7f23040a7180 auth_method 0 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23027fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23027fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f2302ffd640 10 monclient(hunting): get_auth_request con 0x7f23040a6860 auth_method 0 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f2302ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f2302ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f2302ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 10 monclient(hunting): handle_auth_done global_id 44941 payload 824 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 1 -- 172.21.15.89:0/926086158 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f23040a7180 msgr2=0x7f2304134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f23040a7180 0x7f2304134f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 1 -- 172.21.15.89:0/926086158 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f23040a6860 msgr2=0x7f23041349f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f23040a6860 0x7f23041349f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 10 monclient: _finish_hunting 0 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 1 monclient: found mon.noname-c 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:51:14.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.289+0000 7f23037fe640 1 -- 172.21.15.89:0/926086158 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23040abbb0 con 0x7f2304139620 2024-08-19T10:51:14.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f23037fe640 1 -- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2304139620 msgr2=0x7f230409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f23037fe640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2304139620 0x7f230409c9e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c580 tx=0x7f22f400ca40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22f4015070 con 0x7f2304139620 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 10 monclient: dump: 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.306 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:51:14.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f230941a640 5 monclient: authenticate success, global_id 44941 2024-08-19T10:51:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f230941a640 10 monclient: _renew_subs 2024-08-19T10:51:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f4011c80 con 0x7f2304139620 2024-08-19T10:51:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f230941a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f230941a640 1 -- 172.21.15.89:0/926086158 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230409e100 con 0x7f2304139620 2024-08-19T10:51:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22f4011e30 con 0x7f2304139620 2024-08-19T10:51:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 10 monclient: dump: 2024-08-19T10:51:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.310 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f23017fa640 4 set_mon_vals no callback set 2024-08-19T10:51:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f230941a640 10 monclient: _renew_subs 2024-08-19T10:51:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f230941a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:14.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f230941a640 1 -- 172.21.15.89:0/926086158 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f23040a0e40 con 0x7f2304139620 2024-08-19T10:51:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f230941a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f230941a640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409d6e0 0x7f230402d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f230941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f230941a640 1 -- 172.21.15.89:0/926086158 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f23040ac030 con 0x7f230409d6e0 2024-08-19T10:51:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f2302ffd640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409d6e0 0x7f230402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f2302ffd640 10 monclient: get_auth_request con 0x7f230409d6e0 auth_method 0 2024-08-19T10:51:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f2302ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f2302ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f2302ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f22f4011810 con 0x7f2304139620 2024-08-19T10:51:14.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f2302ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f2302ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f2302ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f2302ffd640 10 monclient(hunting): handle_auth_done global_id 44950 payload 824 2024-08-19T10:51:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f2302ffd640 1 -- 172.21.15.89:0/926086158 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23040a7560 con 0x7f230409d6e0 2024-08-19T10:51:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f22f003be00 0x7f22f003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f22f404f2a0 con 0x7f2304139620 2024-08-19T10:51:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f23027fc640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f22f003be00 0x7f22f003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.293+0000 7f23027fc640 10 monclient: get_auth_request con 0x7f22f003be00 auth_method 0 2024-08-19T10:51:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f2302ffd640 1 -- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409d6e0 msgr2=0x7f230402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f2302ffd640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409d6e0 0x7f230402d1d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0ae000 tx=0x7f22fc0196b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22fc0bf070 con 0x7f230409d6e0 2024-08-19T10:51:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f22eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f22eb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f22eb7fe640 10 monclient: dump: 2024-08-19T10:51:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.313 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.314 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f22fc004040 con 0x7f230409d6e0 2024-08-19T10:51:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f22eb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f22eb7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409d6e0 msgr2=0x7f230402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f22eb7fe640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409d6e0 0x7f230402d1d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0ae000 tx=0x7f22fc0196b0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409d6e0 msgr2=0x7f230402d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f22eb7fe640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409d6e0 0x7f230402d1d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.297+0000 7f23027fc640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f22f003be00 0x7f22f003e2c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f230409e770 tx=0x7f22f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f230941a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f230941a640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409dac0 0x7f230402d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f230941a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f230941a640 1 -- 172.21.15.89:0/926086158 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f2304064590 con 0x7f230409dac0 2024-08-19T10:51:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f2302ffd640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409dac0 0x7f230402d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f2302ffd640 10 monclient: get_auth_request con 0x7f230409dac0 auth_method 0 2024-08-19T10:51:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f2302ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f2302ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f2302ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f2302ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f2302ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f2302ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f2302ffd640 10 monclient(hunting): handle_auth_done global_id 44959 payload 824 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.301+0000 7f2302ffd640 1 -- 172.21.15.89:0/926086158 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f230413c230 con 0x7f230409dac0 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f2302ffd640 1 -- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409dac0 msgr2=0x7f230402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f2302ffd640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409dac0 0x7f230402d710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f22fc01c950 tx=0x7f22fc0a9000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f22fc0bf070 con 0x7f230409dac0 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f22eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f22eb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f22eb7fe640 10 monclient: dump: 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.317 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7f22fc0c0040 con 0x7f230409dac0 2024-08-19T10:51:14.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f22eb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:14.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f22eb7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:14.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409dac0 msgr2=0x7f230402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f22eb7fe640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409dac0 0x7f230402d710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f22fc01c950 tx=0x7f22fc0a9000 comp rx=0 tx=0).stop 2024-08-19T10:51:14.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f22eb7fe640 1 -- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409dac0 msgr2=0x7f230402d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:14.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f22eb7fe640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f230409dac0 0x7f230402d710 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f22fc01c950 tx=0x7f22fc0a9000 comp rx=0 tx=0).stop 2024-08-19T10:51:14.319 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:14.319 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 397431, 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:14.320 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.321 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.322 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:14.323 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.324 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.325 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:14.327 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.328 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.329 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:14.330 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.331 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.334 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:14.336 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:14.337 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:14.338 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.339 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:14.340 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.340 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.340 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.340 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:14.340 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.340 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.340 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.340 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:14.341 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:14.341 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.341 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.341 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.341 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.341 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:14.341 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:14.341 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.341 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.341 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.342 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.343 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:14.344 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:14.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f230941a640 1 -- 172.21.15.89:0/926086158 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f22f003be00 msgr2=0x7f22f003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f230941a640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f22f003be00 0x7f22f003e2c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f230409e770 tx=0x7f22f8008040 comp rx=0 tx=0).stop 2024-08-19T10:51:14.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f230941a640 10 monclient: shutdown 2024-08-19T10:51:14.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f230941a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:14.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f230941a640 1 -- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2304139620 msgr2=0x7f230409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f23037fe640 1 -- 172.21.15.89:0/926086158 reap_dead start 2024-08-19T10:51:14.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f230941a640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2304139620 0x7f230409c9e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c580 tx=0x7f22f400ca40 comp rx=0 tx=0).stop 2024-08-19T10:51:14.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f230941a640 1 -- 172.21.15.89:0/926086158 shutdown_connections 2024-08-19T10:51:14.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f230941a640 1 --2- 172.21.15.89:0/926086158 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2304139620 0x7f230409c9e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f230941a640 1 -- 172.21.15.89:0/926086158 >> 172.21.15.89:0/926086158 conn(0x7f2304074b40 msgr2=0x7f2304137800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:14.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f230941a640 1 -- 172.21.15.89:0/926086158 shutdown_connections 2024-08-19T10:51:14.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.305+0000 7f230941a640 1 -- 172.21.15.89:0/926086158 wait complete. 2024-08-19T10:51:14.347 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:14.347 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:51:14.347 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:51:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.465+0000 7fbbb3ddf640 10 monclient: get_monmap_and_config 2024-08-19T10:51:14.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.465+0000 7fbbb3ddf640 10 monclient: build_initial_monmap 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.465+0000 7fbbb3ddf640 10 monclient: monmap: 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:14.472605+0000 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:14.472605+0000 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:14.467 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:14.468 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:14.468 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 1 Processor -- start 2024-08-19T10:51:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 1 -- start start 2024-08-19T10:51:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 10 monclient: init 2024-08-19T10:51:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 10 monclient: _add_conns ranks=[7,9,3,1,0,8,6,2,4,10,5] 2024-08-19T10:51:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbbac0a3aa0 0x7fbbac0a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 10 monclient(hunting): picked mon.noname-i con 0x7fbbac0a3aa0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a1760 0x7fbbac0a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 10 monclient(hunting): picked mon.noname-a con 0x7fbbac0a1760 addr v2:172.21.15.89:3300/0 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbac13d910 0x7fbbac13fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 10 monclient(hunting): picked mon.noname-g con 0x7fbbac13d910 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fbbac06d6d0 con 0x7fbbac0a1760 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbbac0a4900 con 0x7fbbac13d910 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fbbac0a4b90 con 0x7fbbac0a3aa0 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb3ddf640 10 monclient(hunting): authenticate will time out at 1724.289570s 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a1760 0x7fbbac0a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a1760 0x7fbbac0a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36492/0 (socket says 172.21.15.89:36492) 2024-08-19T10:51:14.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 1 -- 172.21.15.89:0/874644415 learned_addr learned my addr 172.21.15.89:0/874644415 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 10 monclient(hunting): get_auth_request con 0x7fbbac0a1760 auth_method 0 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb1b7b640 1 --2- 172.21.15.89:0/874644415 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbbac0a3aa0 0x7fbbac0a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb237c640 1 --2- 172.21.15.89:0/874644415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbac13d910 0x7fbbac13fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb1b7b640 10 monclient(hunting): get_auth_request con 0x7fbbac0a3aa0 auth_method 0 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb1b7b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb1b7b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb1b7b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb237c640 10 monclient(hunting): get_auth_request con 0x7fbbac13d910 auth_method 0 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb237c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb237c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb237c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 10 monclient(hunting): handle_auth_done global_id 7119 payload 274 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 1 -- 172.21.15.89:0/874644415 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbbac0a3aa0 msgr2=0x7fbbac0a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 1 --2- 172.21.15.89:0/874644415 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbbac0a3aa0 0x7fbbac0a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 1 -- 172.21.15.89:0/874644415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbac13d910 msgr2=0x7fbbac13fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 1 --2- 172.21.15.89:0/874644415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbac13d910 0x7fbbac13fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 10 monclient: _finish_hunting 0 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 1 monclient: found mon.noname-a 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb137a640 1 -- 172.21.15.89:0/874644415 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac0a09c0 con 0x7fbbac0a1760 2024-08-19T10:51:14.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb1b7b640 1 --2- 172.21.15.89:0/874644415 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbbac0a3aa0 0x7fbbac0a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:51:14.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.469+0000 7fbbb237c640 1 --2- 172.21.15.89:0/874644415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbac13d910 0x7fbbac13fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:51:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb137a640 1 -- 172.21.15.89:0/874644415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 msgr2=0x7fbbac0a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb137a640 1 --2- 172.21.15.89:0/874644415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 0x7fbbac0a43c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009870 tx=0x7fbb9c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdc42872db807ee server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 1 -- 172.21.15.89:0/874644415 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb9c0b0070 con 0x7fbbac0a1760 2024-08-19T10:51:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:51:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 10 monclient: dump: 2024-08-19T10:51:14.477 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.478 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 10 monclient: _finish_auth 0 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 10 monclient: _check_auth_tickets 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 1 -- 172.21.15.89:0/874644415 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb9c0aa810 con 0x7fbbac0a1760 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 5 monclient: authenticate success, global_id 7119 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 1 -- 172.21.15.89:0/874644415 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb9c0a9300 con 0x7fbbac0a1760 2024-08-19T10:51:14.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 4 set_mon_vals no callback set 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb0b79640 10 monclient: dump: 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.480 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 10 monclient: shutdown 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/874644415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 msgr2=0x7fbbac0a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 1 --2- 172.21.15.89:0/874644415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 0x7fbbac0a43c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009870 tx=0x7fbb9c01c420 comp rx=0 tx=0).stop 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/874644415 shutdown_connections 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 1 --2- 172.21.15.89:0/874644415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 0x7fbbac0a43c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 1 --2- 172.21.15.89:0/874644415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbac13d910 0x7fbbac13fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.473+0000 7fbbb3ddf640 1 --2- 172.21.15.89:0/874644415 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbbac0a3aa0 0x7fbbac0a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/874644415 >> 172.21.15.89:0/874644415 conn(0x7fbbac0a9130 msgr2=0x7fbbac0a9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/874644415 shutdown_connections 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/874644415 wait complete. 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient: build_initial_monmap 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient: monmap: 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.482 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 1 Processor -- start 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 1 -- start start 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient: init 2024-08-19T10:51:14.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient: _add_conns ranks=[0,8,4,7,3,5,1,6,2] 2024-08-19T10:51:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 0x7fbbac135c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient(hunting): picked mon.noname-a con 0x7fbbac0a1760 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbbac0a3aa0 0x7fbbac1361c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient(hunting): picked mon.noname-e con 0x7fbbac0a3aa0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbac13d910 0x7fbbac09d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient(hunting): picked mon.noname-c con 0x7fbbac13d910 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac142a60 con 0x7fbbac0a1760 2024-08-19T10:51:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbbac0a50b0 con 0x7fbbac13d910 2024-08-19T10:51:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fbbac142be0 con 0x7fbbac0a3aa0 2024-08-19T10:51:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb3ddf640 10 monclient(hunting): authenticate will time out at 1724.297637s 2024-08-19T10:51:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 0x7fbbac135c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb237c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbac13d910 0x7fbbac09d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb237c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbac13d910 0x7fbbac09d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38056/0 (socket says 172.21.15.89:38056) 2024-08-19T10:51:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 10 monclient(hunting): get_auth_request con 0x7fbbac0a1760 auth_method 0 2024-08-19T10:51:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb237c640 1 -- 172.21.15.89:0/2660284576 learned_addr learned my addr 172.21.15.89:0/2660284576 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb237c640 10 monclient(hunting): get_auth_request con 0x7fbbac13d910 auth_method 0 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb237c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb237c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb237c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb137a640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbbac0a3aa0 0x7fbbac1361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb137a640 10 monclient(hunting): get_auth_request con 0x7fbbac0a3aa0 auth_method 0 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb137a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb137a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb137a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb237c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb237c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb237c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 10 monclient(hunting): handle_auth_done global_id 7128 payload 824 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 1 -- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbbac0a3aa0 msgr2=0x7fbbac1361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbbac0a3aa0 0x7fbbac1361c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 1 -- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbac13d910 msgr2=0x7fbbac09d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbac13d910 0x7fbbac09d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 10 monclient: _finish_hunting 0 2024-08-19T10:51:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 1 monclient: found mon.noname-a 2024-08-19T10:51:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:51:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb1b7b640 1 -- 172.21.15.89:0/2660284576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac13b530 con 0x7fbbac0a1760 2024-08-19T10:51:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb237c640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbbac13d910 0x7fbbac09d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.477+0000 7fbbb137a640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbbac0a3aa0 0x7fbbac1361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb1b7b640 1 -- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 msgr2=0x7fbbac135c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb1b7b640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 0x7fbbac135c80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbba800e7e0 tx=0x7fbba800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbba8017070 con 0x7fbbac0a1760 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 10 monclient: dump: 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 10 monclient: _finish_auth 0 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:14.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba800cb40 con 0x7fbbac0a1760 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbba800f040 con 0x7fbbac0a1760 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb3ddf640 5 monclient: authenticate success, global_id 7128 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb3ddf640 10 monclient: _renew_subs 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb3ddf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/2660284576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbac09ed90 con 0x7fbbac0a1760 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba3fff640 4 set_mon_vals no callback set 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 10 monclient: dump: 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb3ddf640 10 monclient: _renew_subs 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb3ddf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/2660284576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbac139cb0 con 0x7fbbac0a1760 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb3ddf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb3ddf640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09da50 0x7fbbac09de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb3ddf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/2660284576 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fbbac13a140 con 0x7fbbac09da50 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbba801a070 con 0x7fbbac0a1760 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb137a640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09da50 0x7fbbac09de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb137a640 10 monclient: get_auth_request con 0x7fbbac09da50 auth_method 0 2024-08-19T10:51:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb137a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb137a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb137a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbb9003be30 0x7fbb9003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7fbba804f040 con 0x7fbbac0a1760 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb137a640 10 monclient(hunting): handle_auth_done global_id 44934 payload 824 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb137a640 1 -- 172.21.15.89:0/2660284576 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbbac0a3e80 con 0x7fbbac09da50 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb237c640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbb9003be30 0x7fbb9003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.481+0000 7fbbb237c640 10 monclient: get_auth_request con 0x7fbb9003be30 auth_method 0 2024-08-19T10:51:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbbb137a640 1 -- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09da50 msgr2=0x7fbbac09de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbbb137a640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09da50 0x7fbbac09de30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0ad000 tx=0x7fbb9c0191c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb9c0b0040 con 0x7fbbac09da50 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbba27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbba27fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbba27fc640 10 monclient: dump: 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbb9c0aaa40 con 0x7fbbac09da50 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbba27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbba27fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09da50 msgr2=0x7fbbac09de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbba27fc640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09da50 0x7fbbac09de30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0ad000 tx=0x7fbb9c0191c0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09da50 msgr2=0x7fbbac09de30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbba27fc640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09da50 0x7fbbac09de30 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0ad000 tx=0x7fbb9c0191c0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.485+0000 7fbbb237c640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbb9003be30 0x7fbb9003e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbbac09eb80 tx=0x7fbba4009870 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb3ddf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb3ddf640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09e370 0x7fbbac070e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb3ddf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/2660284576 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fbbac13b1b0 con 0x7fbbac09e370 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb137a640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09e370 0x7fbbac070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb137a640 10 monclient: get_auth_request con 0x7fbbac09e370 auth_method 0 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb137a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb137a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb137a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb137a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb137a640 10 monclient(hunting): handle_auth_done global_id 44943 payload 824 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.489+0000 7fbbb137a640 1 -- 172.21.15.89:0/2660284576 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbbac064590 con 0x7fbbac09e370 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb137a640 1 -- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09e370 msgr2=0x7fbbac070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb137a640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09e370 0x7fbbac070e30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c000c10 tx=0x7fbb9c0a8000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbb9c0b0040 con 0x7fbbac09e370 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbba27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbba27fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbba27fc640 10 monclient: dump: 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.503 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7fbb9c0c4070 con 0x7fbbac09e370 2024-08-19T10:51:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbba27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbba27fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09e370 msgr2=0x7fbbac070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbba27fc640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09e370 0x7fbbac070e30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c000c10 tx=0x7fbb9c0a8000 comp rx=0 tx=0).stop 2024-08-19T10:51:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbba27fc640 1 -- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09e370 msgr2=0x7fbbac070e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:14.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbba27fc640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbbac09e370 0x7fbbac070e30 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c000c10 tx=0x7fbb9c0a8000 comp rx=0 tx=0).stop 2024-08-19T10:51:14.504 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:14.504 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 397619, 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:14.505 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.506 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.507 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:14.508 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:14.509 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.510 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:14.511 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:14.514 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.516 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.517 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.518 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.518 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:14.518 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:14.518 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.518 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.518 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.518 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:14.518 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.522 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.522 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:14.522 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:14.523 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.524 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.525 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:14.526 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.527 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:14.528 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.529 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.530 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.531 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:14.532 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/2660284576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbb9003be30 msgr2=0x7fbb9003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb3ddf640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbb9003be30 0x7fbb9003e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbbac09eb80 tx=0x7fbba4009870 comp rx=0 tx=0).stop 2024-08-19T10:51:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb3ddf640 10 monclient: shutdown 2024-08-19T10:51:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb3ddf640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 msgr2=0x7fbbac135c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb3ddf640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 0x7fbbac135c80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbba800e7e0 tx=0x7fbba800eca0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb237c640 1 -- 172.21.15.89:0/2660284576 reap_dead start 2024-08-19T10:51:14.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/2660284576 shutdown_connections 2024-08-19T10:51:14.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb3ddf640 1 --2- 172.21.15.89:0/2660284576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbac0a1760 0x7fbbac135c80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/2660284576 >> 172.21.15.89:0/2660284576 conn(0x7fbbac0a9130 msgr2=0x7fbbac138b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:14.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/2660284576 shutdown_connections 2024-08-19T10:51:14.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.493+0000 7fbbb3ddf640 1 -- 172.21.15.89:0/2660284576 wait complete. 2024-08-19T10:51:14.534 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:14.534 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:51:14.534 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:51:14.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient: get_monmap_and_config 2024-08-19T10:51:14.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient: build_initial_monmap 2024-08-19T10:51:14.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient: monmap: 2024-08-19T10:51:14.651 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:14.651 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.651 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:14.657045+0000 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:14.657045+0000 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:14.652 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:14.653 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:14.653 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:14.653 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 1 Processor -- start 2024-08-19T10:51:14.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 1 -- start start 2024-08-19T10:51:14.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient: init 2024-08-19T10:51:14.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:14.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient: _add_conns ranks=[6,2,10,8,1,7,0,9,5,4,3] 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9eac0a3d70 0x7f9eac0a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient(hunting): picked mon.noname-d con 0x7f9eac0a3d70 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9eac0a4150 0x7f9eac0a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient(hunting): picked mon.noname-b con 0x7f9eac0a4150 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9eac09b040 0x7f9eac09b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient(hunting): picked mon.noname-f con 0x7f9eac09b040 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9eac06f8e0 con 0x7f9eac09b040 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9eac0a0780 con 0x7f9eac0a4150 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9eac09fb40 con 0x7f9eac0a3d70 2024-08-19T10:51:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eb1438640 10 monclient(hunting): authenticate will time out at 1724.473748s 2024-08-19T10:51:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9eac0a3d70 0x7f9eac0a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaa7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9eac0a4150 0x7f9eac0a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9eac0a3d70 0x7f9eac0a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:41098/0 (socket says 172.21.15.89:41098) 2024-08-19T10:51:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaa7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9eac0a4150 0x7f9eac0a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35378/0 (socket says 172.21.15.89:35378) 2024-08-19T10:51:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 1 -- 172.21.15.89:0/2392366152 learned_addr learned my addr 172.21.15.89:0/2392366152 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 10 monclient(hunting): get_auth_request con 0x7f9eac0a3d70 auth_method 0 2024-08-19T10:51:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaa7fc640 10 monclient(hunting): get_auth_request con 0x7f9eac0a4150 auth_method 0 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaa7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_done global_id 44952 payload 274 2024-08-19T10:51:14.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 1 -- 172.21.15.89:0/2392366152 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9eac0a4150 msgr2=0x7f9eac0a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 1 --2- 172.21.15.89:0/2392366152 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9eac0a4150 0x7f9eac0a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 1 -- 172.21.15.89:0/2392366152 >> v2:172.21.15.190:3300/0 conn(0x7f9eac09b040 msgr2=0x7f9eac09b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 1 --2- 172.21.15.89:0/2392366152 >> v2:172.21.15.190:3300/0 conn(0x7f9eac09b040 0x7f9eac09b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 10 monclient: _finish_hunting 0 2024-08-19T10:51:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 1 monclient: found mon.noname-d 2024-08-19T10:51:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:51:14.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.653+0000 7f9eaaffd640 1 -- 172.21.15.89:0/2392366152 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac09fe90 con 0x7f9eac0a3d70 2024-08-19T10:51:14.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eaaffd640 1 -- 172.21.15.89:0/2392366152 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9eac0a3d70 msgr2=0x7f9eac0a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eaaffd640 1 --2- 172.21.15.89:0/2392366152 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9eac0a3d70 0x7f9eac0a0240 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009e80 tx=0x7f9ea401c960 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b2329c2a688da7c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 1 -- 172.21.15.89:0/2392366152 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ea40090d0 con 0x7f9eac0a3d70 2024-08-19T10:51:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:51:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 10 monclient: dump: 2024-08-19T10:51:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.666 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.667 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:51:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:51:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 1 -- 172.21.15.89:0/2392366152 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea40a5b00 con 0x7f9eac0a3d70 2024-08-19T10:51:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 5 monclient: authenticate success, global_id 44952 2024-08-19T10:51:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 1 -- 172.21.15.89:0/2392366152 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ea40a5cb0 con 0x7f9eac0a3d70 2024-08-19T10:51:14.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 4 set_mon_vals no callback set 2024-08-19T10:51:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9ea9ffb640 10 monclient: dump: 2024-08-19T10:51:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.669 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 10 monclient: shutdown 2024-08-19T10:51:14.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 1 -- 172.21.15.89:0/2392366152 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9eac0a3d70 msgr2=0x7f9eac0a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 1 --2- 172.21.15.89:0/2392366152 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9eac0a3d70 0x7f9eac0a0240 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009e80 tx=0x7f9ea401c960 comp rx=0 tx=0).stop 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 1 -- 172.21.15.89:0/2392366152 shutdown_connections 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 1 --2- 172.21.15.89:0/2392366152 >> v2:172.21.15.190:3300/0 conn(0x7f9eac09b040 0x7f9eac09b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 1 --2- 172.21.15.89:0/2392366152 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9eac0a4150 0x7f9eac0a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 1 --2- 172.21.15.89:0/2392366152 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9eac0a3d70 0x7f9eac0a0240 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 1 -- 172.21.15.89:0/2392366152 >> 172.21.15.89:0/2392366152 conn(0x7f9eac0a4e60 msgr2=0x7f9eac0a5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 1 -- 172.21.15.89:0/2392366152 shutdown_connections 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.657+0000 7f9eb1438640 1 -- 172.21.15.89:0/2392366152 wait complete. 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient: build_initial_monmap 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient: monmap: 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:14.671 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:14.672 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:14.673 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 1 Processor -- start 2024-08-19T10:51:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 1 -- start start 2024-08-19T10:51:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient: init 2024-08-19T10:51:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient: _add_conns ranks=[0,4,3,2,7,1,6,5,8] 2024-08-19T10:51:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac09b040 0x7f9eac1cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient(hunting): picked mon.noname-a con 0x7f9eac09b040 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9eac0a3d70 0x7f9eac09a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient(hunting): picked mon.noname-c con 0x7f9eac0a3d70 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9eac0a4150 0x7f9eac09a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient(hunting): picked mon.noname-g con 0x7f9eac0a4150 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac140f30 con 0x7f9eac09b040 2024-08-19T10:51:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9eac0a4ab0 con 0x7f9eac0a4150 2024-08-19T10:51:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9eac1410b0 con 0x7f9eac0a3d70 2024-08-19T10:51:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eb1438640 10 monclient(hunting): authenticate will time out at 1724.481164s 2024-08-19T10:51:14.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9eac0a3d70 0x7f9eac09a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaaffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac09b040 0x7f9eac1cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9eac0a3d70 0x7f9eac09a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38062/0 (socket says 172.21.15.89:38062) 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaaffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac09b040 0x7f9eac1cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36522/0 (socket says 172.21.15.89:36522) 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 1 -- 172.21.15.89:0/146094499 learned_addr learned my addr 172.21.15.89:0/146094499 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 10 monclient(hunting): get_auth_request con 0x7f9eac0a3d70 auth_method 0 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaaffd640 10 monclient(hunting): get_auth_request con 0x7f9eac09b040 auth_method 0 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eab7fe640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9eac0a4150 0x7f9eac09a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 10 monclient(hunting): handle_auth_done global_id 44977 payload 824 2024-08-19T10:51:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 1 -- 172.21.15.89:0/146094499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9eac0a4150 msgr2=0x7f9eac09a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9eac0a4150 0x7f9eac09a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 1 -- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac09b040 msgr2=0x7f9eac1cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac09b040 0x7f9eac1cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 1 monclient: found mon.noname-c 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 1 -- 172.21.15.89:0/146094499 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac0a2880 con 0x7f9eac0a3d70 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eab7fe640 10 monclient: get_auth_request con 0x7f9eac0a4150 auth_method 0 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eab7fe640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9eac0a4150 0x7f9eac09a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaaffd640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9eac09b040 0x7f9eac1cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 1 -- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9eac0a3d70 msgr2=0x7f9eac09a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.661+0000 7f9eaa7fc640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9eac0a3d70 0x7f9eac09a030 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e8a0 tx=0x7f9ea000ed60 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ea0017070 con 0x7f9eac0a3d70 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: dump: 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.678 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: _finish_auth 0 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: _check_auth_tickets 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea0013470 con 0x7f9eac0a3d70 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eb1438640 5 monclient: authenticate success, global_id 44977 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eb1438640 10 monclient: _renew_subs 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eb1438640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:14.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eb1438640 1 -- 172.21.15.89:0/146094499 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac12d620 con 0x7f9eac0a3d70 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ea0013a90 con 0x7f9eac0a3d70 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: dump: 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.680 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.681 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eb1438640 10 monclient: _renew_subs 2024-08-19T10:51:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9ea97fa640 4 set_mon_vals no callback set 2024-08-19T10:51:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eb1438640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eb1438640 1 -- 172.21.15.89:0/146094499 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac1cf910 con 0x7f9eac0a3d70 2024-08-19T10:51:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eb1438640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eb1438640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac09aab0 0x7f9eac072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eb1438640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eb1438640 1 -- 172.21.15.89:0/146094499 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9eac09ae90 con 0x7f9eac09aab0 2024-08-19T10:51:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac09aab0 0x7f9eac072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9ea0013cf0 con 0x7f9eac0a3d70 2024-08-19T10:51:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 10 monclient: get_auth_request con 0x7f9eac09aab0 auth_method 0 2024-08-19T10:51:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e9803be00 0x7f9e9803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f9ea000f040 con 0x7f9eac0a3d70 2024-08-19T10:51:14.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_done global_id 7037 payload 824 2024-08-19T10:51:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 1 -- 172.21.15.89:0/146094499 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9eac12ef80 con 0x7f9eac09aab0 2024-08-19T10:51:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eab7fe640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e9803be00 0x7f9e9803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eab7fe640 10 monclient: get_auth_request con 0x7f9e9803be00 auth_method 0 2024-08-19T10:51:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 1 -- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac09aab0 msgr2=0x7f9eac072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9eaaffd640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac09aab0 0x7f9eac072ef0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0705d0 tx=0x7f9ea4019700 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ea4009270 con 0x7f9eac09aab0 2024-08-19T10:51:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:51:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.665+0000 7f9e937fe640 10 monclient: dump: 2024-08-19T10:51:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.687 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.688 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.669+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9ea4003dc0 con 0x7f9eac09aab0 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.669+0000 7f9e937fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.669+0000 7f9e937fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.669+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac09aab0 msgr2=0x7f9eac072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.669+0000 7f9e937fe640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac09aab0 0x7f9eac072ef0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0705d0 tx=0x7f9ea4019700 comp rx=0 tx=0).stop 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.669+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac09aab0 msgr2=0x7f9eac072ef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.669+0000 7f9e937fe640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac09aab0 0x7f9eac072ef0 secure :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0705d0 tx=0x7f9ea4019700 comp rx=0 tx=0).stop 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.669+0000 7f9eab7fe640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e9803be00 0x7f9e9803e2c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0a54b0 tx=0x7f9e9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eb1438640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eb1438640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac073430 0x7f9eac073840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eb1438640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eb1438640 1 -- 172.21.15.89:0/146094499 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9eac073d80 con 0x7f9eac073430 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eaaffd640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac073430 0x7f9eac073840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eaaffd640 10 monclient: get_auth_request con 0x7f9eac073430 auth_method 0 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eaaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eaaffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eaaffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eaaffd640 10 monclient(hunting): handle_auth_done global_id 7046 payload 824 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.673+0000 7f9eaaffd640 1 -- 172.21.15.89:0/146094499 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9eac0a0240 con 0x7f9eac073430 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eaaffd640 1 -- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac073430 msgr2=0x7f9eac073840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eaaffd640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac073430 0x7f9eac073840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009e20 tx=0x7f9ea40a7000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9ea4009270 con 0x7f9eac073430 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9e937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9e937fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9e937fe640 10 monclient: dump: 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.691 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.692 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f9ea40043e0 con 0x7f9eac073430 2024-08-19T10:51:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9e937fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9e937fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac073430 msgr2=0x7f9eac073840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9e937fe640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac073430 0x7f9eac073840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009e20 tx=0x7f9ea40a7000 comp rx=0 tx=0).stop 2024-08-19T10:51:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9e937fe640 1 -- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac073430 msgr2=0x7f9eac073840 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:14.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9e937fe640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9eac073430 0x7f9eac073840 secure :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009e20 tx=0x7f9ea40a7000 comp rx=0 tx=0).stop 2024-08-19T10:51:14.693 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:14.693 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:14.693 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:14.693 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:14.693 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:14.693 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 804470, 2024-08-19T10:51:14.693 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:14.693 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:14.694 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:14.694 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:14.694 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:14.694 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:14.694 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:14.694 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:14.694 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:14.694 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:14.694 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.695 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.696 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.696 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.696 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.696 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.696 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:14.696 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:14.696 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.696 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.696 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.696 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:14.697 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:14.698 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.699 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.700 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:14.701 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:14.702 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.703 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.704 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:14.704 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:14.704 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.704 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.704 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.704 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:14.705 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:14.706 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.707 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.708 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.709 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:14.710 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:14.711 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.712 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.713 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:14.715 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.715 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.715 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.715 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:14.715 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:14.715 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.715 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.715 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.716 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:14.716 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.716 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.716 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.716 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:14.716 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.716 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.716 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.716 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:14.717 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:14.717 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.717 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.717 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.717 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.717 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.717 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.717 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:14.717 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:14.717 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.717 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:14.718 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.719 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.719 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.719 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:14.719 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.719 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.719 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:14.719 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eb1438640 1 -- 172.21.15.89:0/146094499 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e9803be00 msgr2=0x7f9e9803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eb1438640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9e9803be00 0x7f9e9803e2c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0a54b0 tx=0x7f9e9c008040 comp rx=0 tx=0).stop 2024-08-19T10:51:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eb1438640 10 monclient: shutdown 2024-08-19T10:51:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eb1438640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eb1438640 1 -- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9eac0a3d70 msgr2=0x7f9eac09a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eb1438640 1 --2- 172.21.15.89:0/146094499 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9eac0a3d70 0x7f9eac09a030 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e8a0 tx=0x7f9ea000ed60 comp rx=0 tx=0).stop 2024-08-19T10:51:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eab7fe640 1 -- 172.21.15.89:0/146094499 reap_dead start 2024-08-19T10:51:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eab7fe640 1 -- 172.21.15.89:0/146094499 reap_dead start 2024-08-19T10:51:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eb1438640 1 -- 172.21.15.89:0/146094499 shutdown_connections 2024-08-19T10:51:14.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eb1438640 1 -- 172.21.15.89:0/146094499 >> 172.21.15.89:0/146094499 conn(0x7f9eac0a4e60 msgr2=0x7f9eac13cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:14.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eb1438640 1 -- 172.21.15.89:0/146094499 shutdown_connections 2024-08-19T10:51:14.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.677+0000 7f9eb1438640 1 -- 172.21.15.89:0/146094499 wait complete. 2024-08-19T10:51:14.722 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:14.722 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:51:14.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:51:14.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient: get_monmap_and_config 2024-08-19T10:51:14.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient: build_initial_monmap 2024-08-19T10:51:14.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient: monmap: 2024-08-19T10:51:14.843 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:14.843 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:14.849388+0000 2024-08-19T10:51:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:14.849388+0000 2024-08-19T10:51:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:14.844 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:14.845 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:14.845 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 1 Processor -- start 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 1 -- start start 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient: init 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient: _add_conns ranks=[5,3,6,8,7,4,2,1,10,9,0] 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f30a40a5270 0x7f30a40a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient(hunting): picked mon.noname-h con 0x7f30a40a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40a6d30 0x7f30a40a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient(hunting): picked mon.noname-g con 0x7f30a40a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30a41395b0 0x7f30a41399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient(hunting): picked mon.noname-d con 0x7f30a41395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:14.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f30a40a7650 con 0x7f30a40a6d30 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f30a40a8710 con 0x7f30a40a5270 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f30a40a5a40 con 0x7f30a41395b0 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a9d86640 10 monclient(hunting): authenticate will time out at 1724.665917s 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30a41395b0 0x7f30a41399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30a41395b0 0x7f30a41399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:41108/0 (socket says 172.21.15.89:41108) 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 1 -- 172.21.15.89:0/973220729 learned_addr learned my addr 172.21.15.89:0/973220729 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 10 monclient(hunting): get_auth_request con 0x7f30a41395b0 auth_method 0 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a37fe640 1 --2- 172.21.15.89:0/973220729 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f30a40a5270 0x7f30a40a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 10 monclient(hunting): handle_auth_done global_id 44961 payload 274 2024-08-19T10:51:14.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 1 -- 172.21.15.89:0/973220729 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f30a40a5270 msgr2=0x7f30a40a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:14.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 1 --2- 172.21.15.89:0/973220729 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f30a40a5270 0x7f30a40a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a2ffd640 1 --2- 172.21.15.89:0/973220729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40a6d30 0x7f30a40a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 1 -- 172.21.15.89:0/973220729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40a6d30 msgr2=0x7f30a40a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:14.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 1 --2- 172.21.15.89:0/973220729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40a6d30 0x7f30a40a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:51:14.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 1 monclient: found mon.noname-d 2024-08-19T10:51:14.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:14.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:51:14.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a3fff640 1 -- 172.21.15.89:0/973220729 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a40a81d0 con 0x7f30a41395b0 2024-08-19T10:51:14.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a37fe640 10 monclient: get_auth_request con 0x7f30a40a5270 auth_method 0 2024-08-19T10:51:14.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a37fe640 1 --2- 172.21.15.89:0/973220729 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f30a40a5270 0x7f30a40a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:14.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a2ffd640 10 monclient: get_auth_request con 0x7f30a40a6d30 auth_method 0 2024-08-19T10:51:14.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.845+0000 7f30a2ffd640 1 --2- 172.21.15.89:0/973220729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40a6d30 0x7f30a40a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a3fff640 1 -- 172.21.15.89:0/973220729 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30a41395b0 msgr2=0x7f30a41399d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a3fff640 1 --2- 172.21.15.89:0/973220729 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30a41395b0 0x7f30a41399d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f309c009e80 tx=0x7f309c01c910 comp rx=0 tx=0).ready entity=mon.6 client_cookie=8e30d3ba1677bf29 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 1 -- 172.21.15.89:0/973220729 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f309c0091b0 con 0x7f30a41395b0 2024-08-19T10:51:14.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 10 monclient: dump: 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.855 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 10 monclient: _finish_auth 0 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 1 -- 172.21.15.89:0/973220729 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f309c0a5a50 con 0x7f30a41395b0 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:14.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 1 -- 172.21.15.89:0/973220729 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f309c0a5c00 con 0x7f30a41395b0 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 5 monclient: authenticate success, global_id 44961 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 4 set_mon_vals no callback set 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a27fc640 10 monclient: dump: 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.857 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 10 monclient: shutdown 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:14.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 1 -- 172.21.15.89:0/973220729 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30a41395b0 msgr2=0x7f30a41399d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 1 --2- 172.21.15.89:0/973220729 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30a41395b0 0x7f30a41399d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f309c009e80 tx=0x7f309c01c910 comp rx=0 tx=0).stop 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 1 -- 172.21.15.89:0/973220729 shutdown_connections 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 1 --2- 172.21.15.89:0/973220729 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30a41395b0 0x7f30a41399d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 1 --2- 172.21.15.89:0/973220729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40a6d30 0x7f30a40a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 1 --2- 172.21.15.89:0/973220729 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f30a40a5270 0x7f30a40a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 1 -- 172.21.15.89:0/973220729 >> 172.21.15.89:0/973220729 conn(0x7f30a4069a80 msgr2=0x7f30a40abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 1 -- 172.21.15.89:0/973220729 shutdown_connections 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.849+0000 7f30a9d86640 1 -- 172.21.15.89:0/973220729 wait complete. 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient: build_initial_monmap 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient: monmap: 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:14.859 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:14.860 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 1 Processor -- start 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 1 -- start start 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient: init 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient: _add_conns ranks=[1,3,7,5,0,8,6,4,2] 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40a5270 0x7f30a409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient(hunting): picked mon.noname-f con 0x7f30a40a5270 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40a6d30 0x7f30a409cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient(hunting): picked mon.noname-g con 0x7f30a40a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f30a41395b0 0x7f30a409d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient(hunting): picked mon.noname-i con 0x7f30a41395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30a40a57d0 con 0x7f30a40a5270 2024-08-19T10:51:14.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f30a40a5650 con 0x7f30a40a6d30 2024-08-19T10:51:14.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f30a4143230 con 0x7f30a41395b0 2024-08-19T10:51:14.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:14.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a9d86640 10 monclient(hunting): authenticate will time out at 1724.673133s 2024-08-19T10:51:14.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40a6d30 0x7f30a409cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40a5270 0x7f30a409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f30a41395b0 0x7f30a409d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f30a41395b0 0x7f30a409d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:34404/0 (socket says 172.21.15.89:34404) 2024-08-19T10:51:14.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40a5270 0x7f30a409c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:48268/0 (socket says 172.21.15.89:48268) 2024-08-19T10:51:14.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40a6d30 0x7f30a409cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:60606/0 (socket says 172.21.15.89:60606) 2024-08-19T10:51:14.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 1 -- 172.21.15.89:0/2129701365 learned_addr learned my addr 172.21.15.89:0/2129701365 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:14.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a2ffd640 10 monclient(hunting): get_auth_request con 0x7f30a40a6d30 auth_method 0 2024-08-19T10:51:14.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 10 monclient(hunting): get_auth_request con 0x7f30a41395b0 auth_method 0 2024-08-19T10:51:14.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a37fe640 10 monclient(hunting): get_auth_request con 0x7f30a40a5270 auth_method 0 2024-08-19T10:51:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 10 monclient(hunting): handle_auth_done global_id 64303 payload 824 2024-08-19T10:51:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 1 -- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40a6d30 msgr2=0x7f30a409cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30a40a6d30 0x7f30a409cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 1 -- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40a5270 msgr2=0x7f30a409c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a40a5270 0x7f30a409c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 10 monclient: _finish_hunting 0 2024-08-19T10:51:14.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 1 monclient: found mon.noname-i 2024-08-19T10:51:14.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:14.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:51:14.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.853+0000 7f30a3fff640 1 -- 172.21.15.89:0/2129701365 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a409f4e0 con 0x7f30a41395b0 2024-08-19T10:51:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a3fff640 1 -- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f30a41395b0 msgr2=0x7f30a409d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a3fff640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f30a41395b0 0x7f30a409d460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f309c009e80 tx=0x7f309c0196b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f309c0bc020 con 0x7f30a41395b0 2024-08-19T10:51:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:51:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 10 monclient: dump: 2024-08-19T10:51:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.872 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.873 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 10 monclient: _finish_auth 0 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f309c003dc0 con 0x7f30a41395b0 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f309c0044b0 con 0x7f30a41395b0 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 10 monclient: dump: 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.874 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a1ffb640 4 set_mon_vals no callback set 2024-08-19T10:51:14.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a9d86640 5 monclient: authenticate success, global_id 64303 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a9d86640 10 monclient: _renew_subs 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a9d86640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a9d86640 1 -- 172.21.15.89:0/2129701365 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a40a6910 con 0x7f30a41395b0 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a9d86640 10 monclient: _renew_subs 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a9d86640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a9d86640 1 -- 172.21.15.89:0/2129701365 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a409f200 con 0x7f30a41395b0 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a9d86640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a9d86640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133550 0x7f30a4133930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a9d86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a9d86640 1 -- 172.21.15.89:0/2129701365 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4140650 con 0x7f30a4133550 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a37fe640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133550 0x7f30a4133930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a37fe640 10 monclient: get_auth_request con 0x7f30a4133550 auth_method 0 2024-08-19T10:51:14.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a37fe640 10 monclient(hunting): handle_auth_done global_id 44893 payload 824 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a37fe640 1 -- 172.21.15.89:0/2129701365 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30a40a7110 con 0x7f30a4133550 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f309c0bf020 con 0x7f30a41395b0 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f309003bdf0 0x7f309003e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f309c0b3040 con 0x7f30a41395b0 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a2ffd640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f309003bdf0 0x7f309003e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.857+0000 7f30a2ffd640 10 monclient: get_auth_request con 0x7f309003bdf0 auth_method 0 2024-08-19T10:51:14.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f30a37fe640 1 -- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133550 msgr2=0x7f30a4133930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f30a37fe640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133550 0x7f30a4133930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f309400f850 tx=0x7f309400fd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3094013440 con 0x7f30a4133550 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f308bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f308bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f308bfff640 10 monclient: dump: 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.878 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f30a2ffd640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f309003bdf0 0x7f309003e2b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f30a413e970 tx=0x7f3098008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f309400dbd0 con 0x7f30a4133550 2024-08-19T10:51:14.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f308bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f308bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133550 msgr2=0x7f30a4133930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f308bfff640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133550 0x7f30a4133930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f309400f850 tx=0x7f309400fd10 comp rx=0 tx=0).stop 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133550 msgr2=0x7f30a4133930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.861+0000 7f308bfff640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133550 0x7f30a4133930 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f309400f850 tx=0x7f309400fd10 comp rx=0 tx=0).stop 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a9d86640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a9d86640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133e70 0x7f30a413ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a9d86640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a9d86640 1 -- 172.21.15.89:0/2129701365 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4064590 con 0x7f30a4133e70 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133e70 0x7f30a413ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 10 monclient: get_auth_request con 0x7f30a4133e70 auth_method 0 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:14.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 10 monclient(hunting): handle_auth_done global_id 44902 payload 824 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 1 -- 172.21.15.89:0/2129701365 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f30a409ed20 con 0x7f30a4133e70 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 1 -- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133e70 msgr2=0x7f30a413ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.869+0000 7f30a37fe640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133e70 0x7f30a413ab90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f309400f850 tx=0x7f30940046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30940202d0 con 0x7f30a4133e70 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f308bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f308bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f308bfff640 10 monclient: dump: 2024-08-19T10:51:14.881 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:14.882 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:14.883 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:14.883 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:14.883 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:14.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f3094006250 con 0x7f30a4133e70 2024-08-19T10:51:14.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f308bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:14.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f308bfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:14.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133e70 msgr2=0x7f30a413ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f308bfff640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133e70 0x7f30a413ab90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f309400f850 tx=0x7f30940046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f308bfff640 1 -- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133e70 msgr2=0x7f30a413ab90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:14.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f308bfff640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30a4133e70 0x7f30a413ab90 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f309400f850 tx=0x7f30940046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.884 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:14.884 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:14.884 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:14.884 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:14.884 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:14.884 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 398003, 2024-08-19T10:51:14.884 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:14.884 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:14.884 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:14.884 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:14.884 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.885 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:14.886 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.887 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:14.888 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:14.889 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.890 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.891 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.893 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.896 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:14.896 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:14.896 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.896 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.896 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.896 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:14.896 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.897 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.898 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:14.898 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:14.898 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.898 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.898 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.898 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:14.899 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.899 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.899 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:14.899 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.899 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.899 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.899 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.900 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.901 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:14.901 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.901 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.901 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:14.901 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.901 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.901 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.902 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.903 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:14.903 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.903 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.903 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:14.903 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.903 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.903 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.904 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.905 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:14.905 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.905 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.905 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:14.905 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.905 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.905 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.905 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.906 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:14.907 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:14.907 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.907 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:14.907 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:14.907 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:14.907 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.907 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.907 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.908 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.909 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:14.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.909 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.909 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:14.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.909 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.909 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.909 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:14.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.909 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:14.909 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:14.910 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:14.910 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:14.910 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:14.910 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:14.910 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:14.910 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:14.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:14.910 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:14.910 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:14.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a9d86640 1 -- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f309003bdf0 msgr2=0x7f309003e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a9d86640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f309003bdf0 0x7f309003e2b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f30a413e970 tx=0x7f3098008040 comp rx=0 tx=0).stop 2024-08-19T10:51:14.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a9d86640 10 monclient: shutdown 2024-08-19T10:51:14.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a9d86640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:14.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a9d86640 1 -- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f30a41395b0 msgr2=0x7f30a409d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:14.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a9d86640 1 --2- 172.21.15.89:0/2129701365 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f30a41395b0 0x7f30a409d460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f309c009e80 tx=0x7f309c0196b0 comp rx=0 tx=0).stop 2024-08-19T10:51:14.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a3fff640 1 -- 172.21.15.89:0/2129701365 reap_dead start 2024-08-19T10:51:14.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a3fff640 1 -- 172.21.15.89:0/2129701365 reap_dead start 2024-08-19T10:51:14.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a9d86640 1 -- 172.21.15.89:0/2129701365 shutdown_connections 2024-08-19T10:51:14.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a9d86640 1 -- 172.21.15.89:0/2129701365 >> 172.21.15.89:0/2129701365 conn(0x7f30a4069a80 msgr2=0x7f30a40aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:14.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a9d86640 1 -- 172.21.15.89:0/2129701365 shutdown_connections 2024-08-19T10:51:14.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:14.873+0000 7f30a9d86640 1 -- 172.21.15.89:0/2129701365 wait complete. 2024-08-19T10:51:14.913 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:14.913 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:51:14.913 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:51:15.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient: get_monmap_and_config 2024-08-19T10:51:15.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient: build_initial_monmap 2024-08-19T10:51:15.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient: monmap: 2024-08-19T10:51:15.036 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:15.036 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.036 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:15.041272+0000 2024-08-19T10:51:15.036 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:15.041272+0000 2024-08-19T10:51:15.036 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:15.036 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.036 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:15.036 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:15.036 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:15.036 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:15.037 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:15.037 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:15.037 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:15.037 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:15.037 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:15.037 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:15.037 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:15.037 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 1 Processor -- start 2024-08-19T10:51:15.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 1 -- start start 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient: init 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient: _add_conns ranks=[10,3,2,9,6,8,0,7,5,1,4] 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f98bc1398b0 0x7f98bc139c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient(hunting): picked mon.noname-f con 0x7f98bc1398b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc06e090 0x7f98bc0a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient(hunting): picked mon.noname-g con 0x7f98bc06e090 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e4f0 0x7f98bc0a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient(hunting): picked mon.noname-b con 0x7f98bc06e4f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f98bc06f4e0 con 0x7f98bc1398b0 2024-08-19T10:51:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc06de80 con 0x7f98bc06e4f0 2024-08-19T10:51:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc06ef20 con 0x7f98bc06e090 2024-08-19T10:51:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c2ab6640 10 monclient(hunting): authenticate will time out at 1724.857797s 2024-08-19T10:51:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e4f0 0x7f98bc0a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e4f0 0x7f98bc0a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35380/0 (socket says 172.21.15.89:35380) 2024-08-19T10:51:15.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 1 -- 172.21.15.89:0/546634416 learned_addr learned my addr 172.21.15.89:0/546634416 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 10 monclient(hunting): get_auth_request con 0x7f98bc06e4f0 auth_method 0 2024-08-19T10:51:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c0852640 1 --2- 172.21.15.89:0/546634416 >> v2:172.21.15.190:3300/0 conn(0x7f98bc1398b0 0x7f98bc139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 10 monclient(hunting): handle_auth_done global_id 74135 payload 274 2024-08-19T10:51:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98bbfff640 1 --2- 172.21.15.89:0/546634416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc06e090 0x7f98bc0a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 1 -- 172.21.15.89:0/546634416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc06e090 msgr2=0x7f98bc0a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 1 --2- 172.21.15.89:0/546634416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc06e090 0x7f98bc0a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 1 -- 172.21.15.89:0/546634416 >> v2:172.21.15.190:3300/0 conn(0x7f98bc1398b0 msgr2=0x7f98bc139c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 1 --2- 172.21.15.89:0/546634416 >> v2:172.21.15.190:3300/0 conn(0x7f98bc1398b0 0x7f98bc139c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 10 monclient: _finish_hunting 0 2024-08-19T10:51:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 1 monclient: found mon.noname-b 2024-08-19T10:51:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:51:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c1053640 1 -- 172.21.15.89:0/546634416 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc06f200 con 0x7f98bc06e4f0 2024-08-19T10:51:15.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c0852640 10 monclient: get_auth_request con 0x7f98bc1398b0 auth_method 0 2024-08-19T10:51:15.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98bbfff640 10 monclient: get_auth_request con 0x7f98bc06e090 auth_method 0 2024-08-19T10:51:15.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98c0852640 1 --2- 172.21.15.89:0/546634416 >> v2:172.21.15.190:3300/0 conn(0x7f98bc1398b0 0x7f98bc139c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:15.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.036+0000 7f98bbfff640 1 --2- 172.21.15.89:0/546634416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc06e090 0x7f98bc0a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98c1053640 1 -- 172.21.15.89:0/546634416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e4f0 msgr2=0x7f98bc0a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98c1053640 1 --2- 172.21.15.89:0/546634416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e4f0 0x7f98bc0a4fa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009e50 tx=0x7f98b401c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60e52eebc943be35 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 1 -- 172.21.15.89:0/546634416 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98b40091b0 con 0x7f98bc06e4f0 2024-08-19T10:51:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:51:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 10 monclient: dump: 2024-08-19T10:51:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.051 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98c2ab6640 5 monclient: authenticate success, global_id 74135 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 1 -- 172.21.15.89:0/546634416 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b40a5ab0 con 0x7f98bc06e4f0 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98c2ab6640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98c2ab6640 4 set_mon_vals no callback set 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 1 -- 172.21.15.89:0/546634416 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98b40a5c60 con 0x7f98bc06e4f0 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.040+0000 7f98bb7fe640 10 monclient: dump: 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.053 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.054 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient: shutdown 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 -- 172.21.15.89:0/546634416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e4f0 msgr2=0x7f98bc0a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 --2- 172.21.15.89:0/546634416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e4f0 0x7f98bc0a4fa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009e50 tx=0x7f98b401c8f0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 -- 172.21.15.89:0/546634416 shutdown_connections 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 --2- 172.21.15.89:0/546634416 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e4f0 0x7f98bc0a4fa0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 --2- 172.21.15.89:0/546634416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc06e090 0x7f98bc0a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 --2- 172.21.15.89:0/546634416 >> v2:172.21.15.190:3300/0 conn(0x7f98bc1398b0 0x7f98bc139c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 -- 172.21.15.89:0/546634416 >> 172.21.15.89:0/546634416 conn(0x7f98bc0aa1b0 msgr2=0x7f98bc0aa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 -- 172.21.15.89:0/546634416 shutdown_connections 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 -- 172.21.15.89:0/546634416 wait complete. 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient: build_initial_monmap 2024-08-19T10:51:15.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient: monmap: 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:15.056 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 Processor -- start 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 -- start start 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient: init 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient: _add_conns ranks=[2,3,8,5,6,0,7,1,4] 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e090 0x7f98bc1d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient(hunting): picked mon.noname-b con 0x7f98bc06e090 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc06e4f0 0x7f98bc1385c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient(hunting): picked mon.noname-g con 0x7f98bc06e4f0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98bc1398b0 0x7f98bc120400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient(hunting): picked mon.noname-e con 0x7f98bc1398b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc143210 con 0x7f98bc06e090 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc13a3e0 con 0x7f98bc06e4f0 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f98bc143390 con 0x7f98bc1398b0 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c2ab6640 10 monclient(hunting): authenticate will time out at 1724.866009s 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e090 0x7f98bc1d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e090 0x7f98bc1d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35392/0 (socket says 172.21.15.89:35392) 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 1 -- 172.21.15.89:0/1102439051 learned_addr learned my addr 172.21.15.89:0/1102439051 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 10 monclient(hunting): get_auth_request con 0x7f98bc06e090 auth_method 0 2024-08-19T10:51:15.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c1053640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98bc1398b0 0x7f98bc120400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c1053640 10 monclient(hunting): get_auth_request con 0x7f98bc1398b0 auth_method 0 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c1053640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c1053640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c1053640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c1053640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c1053640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c1053640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 10 monclient(hunting): handle_auth_done global_id 74144 payload 824 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98bbfff640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc06e4f0 0x7f98bc1385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 1 -- 172.21.15.89:0/1102439051 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98bc1398b0 msgr2=0x7f98bc120400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f98bc1398b0 0x7f98bc120400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 1 -- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc06e4f0 msgr2=0x7f98bc1385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc06e4f0 0x7f98bc1385c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 10 monclient: _finish_hunting 0 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 1 monclient: found mon.noname-b 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.044+0000 7f98c0852640 1 -- 172.21.15.89:0/1102439051 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc123f60 con 0x7f98bc06e090 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c0852640 1 -- 172.21.15.89:0/1102439051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e090 msgr2=0x7f98bc1d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c0852640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e090 0x7f98bc1d13a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00c7f0 tx=0x7f98ac00ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98ac015070 con 0x7f98bc06e090 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 10 monclient: dump: 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.061 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 10 monclient: _finish_auth 0 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 10 monclient: _check_auth_tickets 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac007ca0 con 0x7f98bc06e090 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c2ab6640 5 monclient: authenticate success, global_id 74144 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c2ab6640 10 monclient: _renew_subs 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c2ab6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c2ab6640 1 -- 172.21.15.89:0/1102439051 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc139360 con 0x7f98bc06e090 2024-08-19T10:51:15.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98ac007e50 con 0x7f98bc06e090 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98baffd640 4 set_mon_vals no callback set 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 10 monclient: dump: 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.063 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c2ab6640 10 monclient: _renew_subs 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c2ab6640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c2ab6640 1 -- 172.21.15.89:0/1102439051 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc123c80 con 0x7f98bc06e090 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c2ab6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c2ab6640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc138b00 0x7f98bc120940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c2ab6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c2ab6640 1 -- 172.21.15.89:0/1102439051 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc064590 con 0x7f98bc138b00 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f98ac018070 con 0x7f98bc06e090 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98a803bfd0 0x7f98a803e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98bbfff640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc138b00 0x7f98bc120940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98bbfff640 10 monclient: get_auth_request con 0x7f98bc138b00 auth_method 0 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f98ac04f4c0 con 0x7f98bc06e090 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98bbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c1053640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98a803bfd0 0x7f98a803e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98c1053640 10 monclient: get_auth_request con 0x7f98a803bfd0 auth_method 0 2024-08-19T10:51:15.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98bbfff640 10 monclient(hunting): handle_auth_done global_id 35589 payload 824 2024-08-19T10:51:15.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.048+0000 7f98bbfff640 1 -- 172.21.15.89:0/1102439051 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98bc139c90 con 0x7f98bc138b00 2024-08-19T10:51:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98bbfff640 1 -- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc138b00 msgr2=0x7f98bc120940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98bbfff640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc138b00 0x7f98bc120940 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f98bc13f760 tx=0x7f98b000dbf0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98c1053640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98a803bfd0 0x7f98a803e490 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009e50 tx=0x7f98b40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98b0016070 con 0x7f98bc138b00 2024-08-19T10:51:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98b97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98b97fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:51:15.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98b97fa640 10 monclient: dump: 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.069 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f98b000de00 con 0x7f98bc138b00 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98b97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98b97fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc138b00 msgr2=0x7f98bc120940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98b97fa640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc138b00 0x7f98bc120940 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f98bc13f760 tx=0x7f98b000dbf0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc138b00 msgr2=0x7f98bc120940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.052+0000 7f98b97fa640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc138b00 0x7f98bc120940 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f98bc13f760 tx=0x7f98b000dbf0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98c2ab6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98c2ab6640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc120e80 0x7f98bc13a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98c2ab6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98c2ab6640 1 -- 172.21.15.89:0/1102439051 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc0a7d40 con 0x7f98bc120e80 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc120e80 0x7f98bc13a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 10 monclient: get_auth_request con 0x7f98bc120e80 auth_method 0 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 10 monclient(hunting): handle_auth_done global_id 35598 payload 824 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 1 -- 172.21.15.89:0/1102439051 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f98bc06fd90 con 0x7f98bc120e80 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 1 -- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc120e80 msgr2=0x7f98bc13a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.060+0000 7f98bbfff640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc120e80 0x7f98bc13a5a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f98b0000c10 tx=0x7f98b00046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98b0012460 con 0x7f98bc120e80 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98b97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98b97fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98b97fa640 10 monclient: dump: 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.072 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.073 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f98b0003a40 con 0x7f98bc120e80 2024-08-19T10:51:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98b97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:15.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98b97fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc120e80 msgr2=0x7f98bc13a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98b97fa640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc120e80 0x7f98bc13a5a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f98b0000c10 tx=0x7f98b00046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98b97fa640 1 -- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc120e80 msgr2=0x7f98bc13a5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:15.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98b97fa640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98bc120e80 0x7f98bc13a5a0 secure :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f98b0000c10 tx=0x7f98b00046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.074 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:15.074 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:15.074 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:15.074 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:15.074 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 520113, 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.075 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:15.076 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:15.077 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:15.078 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:15.079 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.080 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.081 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.082 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.083 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.084 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.085 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:15.086 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.087 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.088 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.089 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:15.090 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:15.091 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.092 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.092 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:15.092 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.092 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.092 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.092 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:15.092 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:15.095 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:15.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.097 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:15.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.098 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:15.098 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:15.098 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:51:15.098 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.098 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.098 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:15.098 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:15.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c2ab6640 1 -- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98a803bfd0 msgr2=0x7f98a803e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c2ab6640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98a803bfd0 0x7f98a803e490 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009e50 tx=0x7f98b40a7070 comp rx=0 tx=0).stop 2024-08-19T10:51:15.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c2ab6640 10 monclient: shutdown 2024-08-19T10:51:15.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c2ab6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:15.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c2ab6640 1 -- 172.21.15.89:0/1102439051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e090 msgr2=0x7f98bc1d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c2ab6640 1 --2- 172.21.15.89:0/1102439051 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f98bc06e090 0x7f98bc1d13a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00c7f0 tx=0x7f98ac00ccb0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c1053640 1 -- 172.21.15.89:0/1102439051 reap_dead start 2024-08-19T10:51:15.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c1053640 1 -- 172.21.15.89:0/1102439051 reap_dead start 2024-08-19T10:51:15.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c2ab6640 1 -- 172.21.15.89:0/1102439051 shutdown_connections 2024-08-19T10:51:15.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c2ab6640 1 -- 172.21.15.89:0/1102439051 >> 172.21.15.89:0/1102439051 conn(0x7f98bc0aa1b0 msgr2=0x7f98bc13fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:15.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c2ab6640 1 -- 172.21.15.89:0/1102439051 shutdown_connections 2024-08-19T10:51:15.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.064+0000 7f98c2ab6640 1 -- 172.21.15.89:0/1102439051 wait complete. 2024-08-19T10:51:15.101 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:15.101 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:51:15.101 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:51:15.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient: get_monmap_and_config 2024-08-19T10:51:15.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient: build_initial_monmap 2024-08-19T10:51:15.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient: monmap: 2024-08-19T10:51:15.215 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:15.215 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.215 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:15.221063+0000 2024-08-19T10:51:15.215 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:15.221063+0000 2024-08-19T10:51:15.215 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:15.216 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 1 Processor -- start 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 1 -- start start 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient: init 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient: _add_conns ranks=[1,10,8,0,6,7,2,5,9,4,3] 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f59d806c930 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59d80a4240 0x7f59d813e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient(hunting): picked mon.noname-f con 0x7f59d80a4240 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59d80a4660 0x7f59d8140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient(hunting): picked mon.noname-e con 0x7f59d80a4660 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d806cd10 con 0x7f59d806c930 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f59d80a3310 con 0x7f59d80a4240 2024-08-19T10:51:15.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f59d80a4d40 con 0x7f59d80a4660 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59dcf89640 10 monclient(hunting): authenticate will time out at 1725.037740s 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59d80a4660 0x7f59d8140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59d80a4660 0x7f59d8140ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48538/0 (socket says 172.21.15.89:48538) 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 1 -- 172.21.15.89:0/678779688 learned_addr learned my addr 172.21.15.89:0/678779688 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 10 monclient(hunting): get_auth_request con 0x7f59d80a4660 auth_method 0 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d5d9b640 1 --2- 172.21.15.89:0/678779688 >> v2:172.21.15.190:3300/0 conn(0x7f59d80a4240 0x7f59d813e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d5d9b640 10 monclient(hunting): get_auth_request con 0x7f59d80a4240 auth_method 0 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 10 monclient(hunting): handle_auth_done global_id 7064 payload 274 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 1 -- 172.21.15.89:0/678779688 >> v2:172.21.15.190:3300/0 conn(0x7f59d80a4240 msgr2=0x7f59d813e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 1 --2- 172.21.15.89:0/678779688 >> v2:172.21.15.190:3300/0 conn(0x7f59d80a4240 0x7f59d813e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 1 -- 172.21.15.89:0/678779688 >> v1:172.21.15.190:6789/0 conn(0x7f59d806c930 legacy=0x7f59d80a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 1 monclient: found mon.noname-e 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:51:15.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.216+0000 7f59d6d9d640 1 -- 172.21.15.89:0/678779688 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d80a08a0 con 0x7f59d80a4660 2024-08-19T10:51:15.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d6d9d640 1 -- 172.21.15.89:0/678779688 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59d80a4660 msgr2=0x7f59d8140ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d6d9d640 1 --2- 172.21.15.89:0/678779688 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59d80a4660 0x7f59d8140ee0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f59d00098a0 tx=0x7f59d001c400 comp rx=0 tx=0).ready entity=mon.8 client_cookie=bfb02dda1066e27a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 1 -- 172.21.15.89:0/678779688 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59d00b0070 con 0x7f59d80a4660 2024-08-19T10:51:15.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:51:15.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 10 monclient: dump: 2024-08-19T10:51:15.225 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.225 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.225 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 10 monclient: _finish_auth 0 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 10 monclient: _check_auth_tickets 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 1 -- 172.21.15.89:0/678779688 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d00aa810 con 0x7f59d80a4660 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 5 monclient: authenticate success, global_id 7064 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 4 set_mon_vals no callback set 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 1 -- 172.21.15.89:0/678779688 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59d00a9300 con 0x7f59d80a4660 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59d559a640 10 monclient: dump: 2024-08-19T10:51:15.227 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.228 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 10 monclient: shutdown 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 1 -- 172.21.15.89:0/678779688 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59d80a4660 msgr2=0x7f59d8140ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 1 --2- 172.21.15.89:0/678779688 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59d80a4660 0x7f59d8140ee0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f59d00098a0 tx=0x7f59d001c400 comp rx=0 tx=0).stop 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 1 -- 172.21.15.89:0/678779688 shutdown_connections 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 1 --2- 172.21.15.89:0/678779688 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59d80a4660 0x7f59d8140ee0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 1 --2- 172.21.15.89:0/678779688 >> v2:172.21.15.190:3300/0 conn(0x7f59d80a4240 0x7f59d813e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 1 -- 172.21.15.89:0/678779688 >> 172.21.15.89:0/678779688 conn(0x7f59d80a8f70 msgr2=0x7f59d80a9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 1 -- 172.21.15.89:0/678779688 shutdown_connections 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 1 -- 172.21.15.89:0/678779688 wait complete. 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 10 monclient: build_initial_monmap 2024-08-19T10:51:15.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.220+0000 7f59dcf89640 10 monclient: monmap: 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:15.230 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 1 Processor -- start 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 1 -- start start 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 10 monclient: init 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 10 monclient: _add_conns ranks=[4,7,2,6,5,8,1,3,0] 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59d806c930 0x7f59d81397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 10 monclient(hunting): picked mon.noname-c con 0x7f59d806c930 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59d80a4240 0x7f59d813e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 10 monclient(hunting): picked mon.noname-i con 0x7f59d80a4240 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59d80a4660 0x7f59d813e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 10 monclient(hunting): picked mon.noname-b con 0x7f59d80a4660 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f59d8143dc0 con 0x7f59d80a4660 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f59d8143c40 con 0x7f59d806c930 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f59d8143f40 con 0x7f59d80a4240 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59dcf89640 10 monclient(hunting): authenticate will time out at 1725.044691s 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59d806c930 0x7f59d81397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59d806c930 0x7f59d81397b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38066/0 (socket says 172.21.15.89:38066) 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d6d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59d80a4660 0x7f59d813e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 1 -- 172.21.15.89:0/2255365132 learned_addr learned my addr 172.21.15.89:0/2255365132 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 10 monclient(hunting): get_auth_request con 0x7f59d806c930 auth_method 0 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d6d9d640 10 monclient(hunting): get_auth_request con 0x7f59d80a4660 auth_method 0 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d5d9b640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59d80a4240 0x7f59d813e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d5d9b640 10 monclient(hunting): get_auth_request con 0x7f59d80a4240 auth_method 0 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 10 monclient(hunting): handle_auth_done global_id 44986 payload 824 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 1 -- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59d80a4240 msgr2=0x7f59d813e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f59d80a4240 0x7f59d813e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 1 -- 172.21.15.89:0/2255365132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59d80a4660 msgr2=0x7f59d813e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59d80a4660 0x7f59d813e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 10 monclient: _finish_hunting 0 2024-08-19T10:51:15.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 1 monclient: found mon.noname-c 2024-08-19T10:51:15.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:15.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:51:15.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 1 -- 172.21.15.89:0/2255365132 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d8140940 con 0x7f59d806c930 2024-08-19T10:51:15.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d6d9d640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59d80a4660 0x7f59d813e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:15.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 1 -- 172.21.15.89:0/2255365132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59d806c930 msgr2=0x7f59d81397b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d659c640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59d806c930 0x7f59d81397b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f59c000c480 tx=0x7f59c000c940 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59c0012cb0 con 0x7f59d806c930 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 10 monclient: dump: 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.239 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 10 monclient: _finish_auth 0 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 10 monclient: _check_auth_tickets 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c0017070 con 0x7f59d806c930 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59c00117f0 con 0x7f59d806c930 2024-08-19T10:51:15.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59ceffd640 10 monclient: dump: 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.241 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.224+0000 7f59d4d99640 4 set_mon_vals no callback set 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59dcf89640 5 monclient: authenticate success, global_id 44986 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59dcf89640 10 monclient: _renew_subs 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59dcf89640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59dcf89640 1 -- 172.21.15.89:0/2255365132 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d8123fa0 con 0x7f59d806c930 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59dcf89640 10 monclient: _renew_subs 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59dcf89640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:15.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59dcf89640 1 -- 172.21.15.89:0/2255365132 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d8073310 con 0x7f59d806c930 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59dcf89640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59dcf89640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d8139cf0 0x7f59d802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59dcf89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59dcf89640 1 -- 172.21.15.89:0/2255365132 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f59d8064590 con 0x7f59d8139cf0 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f59c0011380 con 0x7f59d806c930 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d5d9b640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d8139cf0 0x7f59d802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59ceffd640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f59b803be00 0x7f59b803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d5d9b640 10 monclient: get_auth_request con 0x7f59d8139cf0 auth_method 0 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f59c004f330 con 0x7f59d806c930 2024-08-19T10:51:15.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d6d9d640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f59b803be00 0x7f59b803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d6d9d640 10 monclient: get_auth_request con 0x7f59b803be00 auth_method 0 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d5d9b640 10 monclient(hunting): handle_auth_done global_id 7115 payload 824 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.228+0000 7f59d5d9b640 1 -- 172.21.15.89:0/2255365132 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59d80a2160 con 0x7f59d8139cf0 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59d5d9b640 1 -- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d8139cf0 msgr2=0x7f59d802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59d5d9b640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d8139cf0 0x7f59d802d1d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009ed0 tx=0x7f59c800f520 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59c8014a50 con 0x7f59d8139cf0 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59ceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59ceffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59ceffd640 10 monclient: dump: 2024-08-19T10:51:15.244 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.245 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59d6d9d640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f59b803be00 0x7f59b803e2c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f59d00ad000 tx=0x7f59d00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f59c801b070 con 0x7f59d8139cf0 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59ceffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59ceffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d8139cf0 msgr2=0x7f59d802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59ceffd640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d8139cf0 0x7f59d802d1d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009ed0 tx=0x7f59c800f520 comp rx=0 tx=0).stop 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d8139cf0 msgr2=0x7f59d802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.232+0000 7f59ceffd640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d8139cf0 0x7f59d802d1d0 secure :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009ed0 tx=0x7f59c800f520 comp rx=0 tx=0).stop 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.236+0000 7f59dcf89640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.236+0000 7f59dcf89640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d802d710 0x7f59d802db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.236+0000 7f59dcf89640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.236+0000 7f59dcf89640 1 -- 172.21.15.89:0/2255365132 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f59d813a0d0 con 0x7f59d802d710 2024-08-19T10:51:15.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d802d710 0x7f59d802db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 10 monclient: get_auth_request con 0x7f59d802d710 auth_method 0 2024-08-19T10:51:15.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 10 monclient(hunting): handle_auth_done global_id 7124 payload 824 2024-08-19T10:51:15.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 1 -- 172.21.15.89:0/2255365132 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f59d8073820 con 0x7f59d802d710 2024-08-19T10:51:15.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 1 -- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d802d710 msgr2=0x7f59d802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59d5d9b640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d802d710 0x7f59d802db20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009ed0 tx=0x7f59c8004a50 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59c8020e40 con 0x7f59d802d710 2024-08-19T10:51:15.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59ceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59ceffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.240+0000 7f59ceffd640 10 monclient: dump: 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.249 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f59c8016420 con 0x7f59d802d710 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59ceffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59ceffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d802d710 msgr2=0x7f59d802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59ceffd640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d802d710 0x7f59d802db20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009ed0 tx=0x7f59c8004a50 comp rx=0 tx=0).stop 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59ceffd640 1 -- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d802d710 msgr2=0x7f59d802db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:15.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59ceffd640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59d802d710 0x7f59d802db20 secure :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009ed0 tx=0x7f59c8004a50 comp rx=0 tx=0).stop 2024-08-19T10:51:15.251 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:15.251 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:15.251 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:15.251 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:15.251 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:15.251 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 805015, 2024-08-19T10:51:15.251 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:15.251 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:15.251 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:15.251 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:15.251 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:15.252 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:15.253 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:15.254 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:15.255 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:15.256 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.259 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:15.260 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.262 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:15.263 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:15.266 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:15.267 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.270 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.272 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.273 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.275 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:15.275 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59dcf89640 1 -- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f59b803be00 msgr2=0x7f59b803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59dcf89640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f59b803be00 0x7f59b803e2c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f59d00ad000 tx=0x7f59d00a6000 comp rx=0 tx=0).stop 2024-08-19T10:51:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59dcf89640 10 monclient: shutdown 2024-08-19T10:51:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59dcf89640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:15.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59dcf89640 1 -- 172.21.15.89:0/2255365132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59d806c930 msgr2=0x7f59d81397b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59dcf89640 1 --2- 172.21.15.89:0/2255365132 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f59d806c930 0x7f59d81397b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f59c000c480 tx=0x7f59c000c940 comp rx=0 tx=0).stop 2024-08-19T10:51:15.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59d6d9d640 1 -- 172.21.15.89:0/2255365132 reap_dead start 2024-08-19T10:51:15.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59d6d9d640 1 -- 172.21.15.89:0/2255365132 reap_dead start 2024-08-19T10:51:15.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59dcf89640 1 -- 172.21.15.89:0/2255365132 shutdown_connections 2024-08-19T10:51:15.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59dcf89640 1 -- 172.21.15.89:0/2255365132 >> 172.21.15.89:0/2255365132 conn(0x7f59d80a8f70 msgr2=0x7f59d80a7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:15.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59dcf89640 1 -- 172.21.15.89:0/2255365132 shutdown_connections 2024-08-19T10:51:15.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.244+0000 7f59dcf89640 1 -- 172.21.15.89:0/2255365132 wait complete. 2024-08-19T10:51:15.277 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:15.277 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:51:15.278 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:51:15.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient: get_monmap_and_config 2024-08-19T10:51:15.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient: build_initial_monmap 2024-08-19T10:51:15.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient: monmap: 2024-08-19T10:51:15.393 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:15.393 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.393 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:15.397843+0000 2024-08-19T10:51:15.393 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:15.397843+0000 2024-08-19T10:51:15.393 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:15.393 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.393 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:15.394 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:15.394 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 1 Processor -- start 2024-08-19T10:51:15.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 1 -- start start 2024-08-19T10:51:15.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient: init 2024-08-19T10:51:15.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:15.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient: _add_conns ranks=[7,10,5,3,1,9,2,0,6,4,8] 2024-08-19T10:51:15.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 0x7f58500a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient(hunting): picked mon.noname-i con 0x7f58500a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:15.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58500a6d30 0x7f58500a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient(hunting): picked mon.noname-f con 0x7f58500a6d30 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:15.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f58501395b0 0x7f5850139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient(hunting): picked mon.noname-h con 0x7f58501395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:15.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f58500a7650 con 0x7f58500a6d30 2024-08-19T10:51:15.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f58500a8710 con 0x7f58501395b0 2024-08-19T10:51:15.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f58500a5a40 con 0x7f58500a5270 2024-08-19T10:51:15.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:15.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5857d04640 10 monclient(hunting): authenticate will time out at 1725.214375s 2024-08-19T10:51:15.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 0x7f58500a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f58562a1640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f58501395b0 0x7f5850139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f585529f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58500a6d30 0x7f58500a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 0x7f58500a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:34428/0 (socket says 172.21.15.89:34428) 2024-08-19T10:51:15.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 1 -- 172.21.15.89:0/168630472 learned_addr learned my addr 172.21.15.89:0/168630472 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:15.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f58562a1640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f58501395b0 0x7f5850139a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:53252/0 (socket says 172.21.15.89:53252) 2024-08-19T10:51:15.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 10 monclient(hunting): get_auth_request con 0x7f58500a5270 auth_method 0 2024-08-19T10:51:15.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f58562a1640 10 monclient(hunting): get_auth_request con 0x7f58501395b0 auth_method 0 2024-08-19T10:51:15.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f58562a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f58562a1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f58562a1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f585529f640 10 monclient(hunting): get_auth_request con 0x7f58500a6d30 auth_method 0 2024-08-19T10:51:15.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f585529f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f585529f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f585529f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f58562a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f58562a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f58562a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f585529f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f585529f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f585529f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 10 monclient(hunting): handle_auth_done global_id 64321 payload 274 2024-08-19T10:51:15.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 1 -- 172.21.15.89:0/168630472 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f58501395b0 msgr2=0x7f5850139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:15.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 1 --2- 172.21.15.89:0/168630472 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f58501395b0 0x7f5850139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 1 -- 172.21.15.89:0/168630472 >> v2:172.21.15.190:3300/0 conn(0x7f58500a6d30 msgr2=0x7f58500a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:15.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 1 --2- 172.21.15.89:0/168630472 >> v2:172.21.15.190:3300/0 conn(0x7f58500a6d30 0x7f58500a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 10 monclient: _finish_hunting 0 2024-08-19T10:51:15.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 1 monclient: found mon.noname-i 2024-08-19T10:51:15.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:15.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:51:15.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.392+0000 7f5855aa0640 1 -- 172.21.15.89:0/168630472 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58500a81d0 con 0x7f58500a5270 2024-08-19T10:51:15.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5855aa0640 1 -- 172.21.15.89:0/168630472 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 msgr2=0x7f58500a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5855aa0640 1 --2- 172.21.15.89:0/168630472 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 0x7f58500a67f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f584c009870 tx=0x7f584c01c420 comp rx=0 tx=0).ready entity=mon.7 client_cookie=b45aa48284651b69 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 1 -- 172.21.15.89:0/168630472 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f584c0b1070 con 0x7f58500a5270 2024-08-19T10:51:15.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:51:15.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 10 monclient: dump: 2024-08-19T10:51:15.410 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.410 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.410 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.411 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.412 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 10 monclient: _finish_auth 0 2024-08-19T10:51:15.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 10 monclient: _check_auth_tickets 2024-08-19T10:51:15.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:15.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 1 -- 172.21.15.89:0/168630472 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f584c0ab810 con 0x7f58500a5270 2024-08-19T10:51:15.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 5 monclient: authenticate success, global_id 64321 2024-08-19T10:51:15.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:15.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:15.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 1 -- 172.21.15.89:0/168630472 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f584c0aa300 con 0x7f58500a5270 2024-08-19T10:51:15.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5854a9e640 10 monclient: dump: 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.413 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.414 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.414 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.414 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.414 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.414 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:15.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 4 set_mon_vals no callback set 2024-08-19T10:51:15.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient: shutdown 2024-08-19T10:51:15.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:15.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 -- 172.21.15.89:0/168630472 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 msgr2=0x7f58500a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 --2- 172.21.15.89:0/168630472 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 0x7f58500a67f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f584c009870 tx=0x7f584c01c420 comp rx=0 tx=0).stop 2024-08-19T10:51:15.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 -- 172.21.15.89:0/168630472 shutdown_connections 2024-08-19T10:51:15.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 --2- 172.21.15.89:0/168630472 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f58501395b0 0x7f5850139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 --2- 172.21.15.89:0/168630472 >> v2:172.21.15.190:3300/0 conn(0x7f58500a6d30 0x7f58500a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 --2- 172.21.15.89:0/168630472 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 0x7f58500a67f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 -- 172.21.15.89:0/168630472 >> 172.21.15.89:0/168630472 conn(0x7f5850069a80 msgr2=0x7f58500abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:15.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 -- 172.21.15.89:0/168630472 shutdown_connections 2024-08-19T10:51:15.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 -- 172.21.15.89:0/168630472 wait complete. 2024-08-19T10:51:15.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient: build_initial_monmap 2024-08-19T10:51:15.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient: monmap: 2024-08-19T10:51:15.415 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:15.415 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:15.416 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:15.416 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:15.416 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:15.416 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.416 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:15.416 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:15.416 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:15.416 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:15.416 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:15.416 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 Processor -- start 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 -- start start 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient: init 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient: _add_conns ranks=[7,1,6,5,0,4,3,8,2] 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 0x7f58500a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient(hunting): picked mon.noname-i con 0x7f58500a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:15.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500a6d30 0x7f58500a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient(hunting): picked mon.noname-f con 0x7f58500a6d30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:51:15.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f58501395b0 0x7f585009c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient(hunting): picked mon.noname-d con 0x7f58501395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:15.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f58500a5650 con 0x7f58500a6d30 2024-08-19T10:51:15.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f58500a4a70 con 0x7f58501395b0 2024-08-19T10:51:15.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f58500a57d0 con 0x7f58500a5270 2024-08-19T10:51:15.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:15.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5857d04640 10 monclient(hunting): authenticate will time out at 1725.222804s 2024-08-19T10:51:15.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f58501395b0 0x7f585009c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f58501395b0 0x7f585009c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:41110/0 (socket says 172.21.15.89:41110) 2024-08-19T10:51:15.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 1 -- 172.21.15.89:0/798552982 learned_addr learned my addr 172.21.15.89:0/798552982 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:15.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 10 monclient(hunting): get_auth_request con 0x7f58501395b0 auth_method 0 2024-08-19T10:51:15.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5855aa0640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 0x7f58500a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5855aa0640 10 monclient(hunting): get_auth_request con 0x7f58500a5270 auth_method 0 2024-08-19T10:51:15.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5855aa0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5855aa0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5855aa0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f585529f640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500a6d30 0x7f58500a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f585529f640 10 monclient(hunting): get_auth_request con 0x7f58500a6d30 auth_method 0 2024-08-19T10:51:15.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f585529f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f585529f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f585529f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 10 monclient(hunting): handle_auth_done global_id 44970 payload 824 2024-08-19T10:51:15.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 1 -- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 msgr2=0x7f58500a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 0x7f58500a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 1 -- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500a6d30 msgr2=0x7f58500a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58500a6d30 0x7f58500a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 10 monclient: _finish_hunting 0 2024-08-19T10:51:15.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 1 monclient: found mon.noname-d 2024-08-19T10:51:15.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:15.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:51:15.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f58562a1640 1 -- 172.21.15.89:0/798552982 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585009f0f0 con 0x7f58501395b0 2024-08-19T10:51:15.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.400+0000 7f5855aa0640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58500a5270 0x7f58500a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:51:15.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58562a1640 1 -- 172.21.15.89:0/798552982 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f58501395b0 msgr2=0x7f585009c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58562a1640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f58501395b0 0x7f585009c9e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5848009dd0 tx=0x7f584800c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5848018020 con 0x7f58501395b0 2024-08-19T10:51:15.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:51:15.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 10 monclient: dump: 2024-08-19T10:51:15.423 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.423 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.423 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.423 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.424 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.424 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.424 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.424 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.424 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.424 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.424 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.424 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.424 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.424 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.424 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 10 monclient: _finish_auth 0 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 10 monclient: _check_auth_tickets 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58480148f0 con 0x7f58501395b0 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5857d04640 5 monclient: authenticate success, global_id 44970 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5857d04640 10 monclient: _renew_subs 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5857d04640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5857d04640 1 -- 172.21.15.89:0/798552982 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5850139c40 con 0x7f58501395b0 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f58480133e0 con 0x7f58501395b0 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 10 monclient: dump: 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.426 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5847fff640 4 set_mon_vals no callback set 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5857d04640 10 monclient: _renew_subs 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5857d04640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5857d04640 1 -- 172.21.15.89:0/798552982 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f585013fb30 con 0x7f58501395b0 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5857d04640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5857d04640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58501d1600 0x7f585009cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5857d04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5857d04640 1 -- 172.21.15.89:0/798552982 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f58500a7110 con 0x7f58501d1600 2024-08-19T10:51:15.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f584801b020 con 0x7f58501395b0 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5855aa0640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58501d1600 0x7f585009cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5855aa0640 10 monclient: get_auth_request con 0x7f58501d1600 auth_method 0 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5855aa0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5855aa0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5855aa0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f583403be00 0x7f583403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f58480502f0 con 0x7f58501395b0 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5855aa0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5855aa0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5855aa0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f585529f640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f583403be00 0x7f583403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5855aa0640 10 monclient(hunting): handle_auth_done global_id 64339 payload 824 2024-08-19T10:51:15.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f5855aa0640 1 -- 172.21.15.89:0/798552982 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5850064590 con 0x7f58501d1600 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.404+0000 7f585529f640 10 monclient: get_auth_request con 0x7f583403be00 auth_method 0 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f5855aa0640 1 -- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58501d1600 msgr2=0x7f585009cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f5855aa0640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58501d1600 0x7f585009cf20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58500a34f0 tx=0x7f584c01ac60 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f584c0bb030 con 0x7f58501d1600 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f58467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f58467fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f58467fc640 10 monclient: dump: 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.429 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f585529f640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f583403be00 0x7f583403e2c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f585013f7a0 tx=0x7f5840008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f584c003ae0 con 0x7f58501d1600 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f58467fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f58467fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58501d1600 msgr2=0x7f585009cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f58467fc640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58501d1600 0x7f585009cf20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58500a34f0 tx=0x7f584c01ac60 comp rx=0 tx=0).stop 2024-08-19T10:51:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58501d1600 msgr2=0x7f585009cf20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.408+0000 7f58467fc640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f58501d1600 0x7f585009cf20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5857d04640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5857d04640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f585009d460 0x7f585013adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5857d04640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5857d04640 1 -- 172.21.15.89:0/798552982 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f585009d870 con 0x7f585009d460 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5855aa0640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f585009d460 0x7f585013adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5855aa0640 10 monclient: get_auth_request con 0x7f585009d460 auth_method 0 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5855aa0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5855aa0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5855aa0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5855aa0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5855aa0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5855aa0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5855aa0640 10 monclient(hunting): handle_auth_done global_id 64348 payload 824 2024-08-19T10:51:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.416+0000 7f5855aa0640 1 -- 172.21.15.89:0/798552982 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f585009e190 con 0x7f585009d460 2024-08-19T10:51:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5855aa0640 1 -- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f585009d460 msgr2=0x7f585013adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5855aa0640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f585009d460 0x7f585013adc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f584c01c250 tx=0x7f584c0191c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f584c0bb040 con 0x7f585009d460 2024-08-19T10:51:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58467fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:51:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58467fc640 10 monclient: dump: 2024-08-19T10:51:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.436 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7f584c0c1d10 con 0x7f585009d460 2024-08-19T10:51:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58467fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58467fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f585009d460 msgr2=0x7f585013adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58467fc640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f585009d460 0x7f585013adc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f584c01c250 tx=0x7f584c0191c0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58467fc640 1 -- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f585009d460 msgr2=0x7f585013adc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58467fc640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f585009d460 0x7f585013adc0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f584c01c250 tx=0x7f584c0191c0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.436 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 141821, 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:15.437 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:15.438 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:15.439 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:15.439 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:15.439 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:15.439 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:15.439 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:15.439 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.439 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:15.439 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:15.439 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:15.439 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:15.440 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:15.441 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:15.442 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:15.443 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.444 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.445 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:15.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:15.445 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:15.445 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.446 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:15.448 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.449 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:15.451 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.453 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.454 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:15.456 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:15.458 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:15.459 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:15.460 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:15.460 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:15.460 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:15.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:15.460 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:15.460 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5857d04640 1 -- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f583403be00 msgr2=0x7f583403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5857d04640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f583403be00 0x7f583403e2c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f585013f7a0 tx=0x7f5840008040 comp rx=0 tx=0).stop 2024-08-19T10:51:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5857d04640 10 monclient: shutdown 2024-08-19T10:51:15.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5857d04640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5857d04640 1 -- 172.21.15.89:0/798552982 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f58501395b0 msgr2=0x7f585009c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5857d04640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f58501395b0 0x7f585009c9e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5848009dd0 tx=0x7f584800c490 comp rx=0 tx=0).stop 2024-08-19T10:51:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f58562a1640 1 -- 172.21.15.89:0/798552982 reap_dead start 2024-08-19T10:51:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5857d04640 1 -- 172.21.15.89:0/798552982 shutdown_connections 2024-08-19T10:51:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5857d04640 1 --2- 172.21.15.89:0/798552982 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f58501395b0 0x7f585009c9e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5857d04640 1 -- 172.21.15.89:0/798552982 >> 172.21.15.89:0/798552982 conn(0x7f5850069a80 msgr2=0x7f5850134e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5857d04640 1 -- 172.21.15.89:0/798552982 shutdown_connections 2024-08-19T10:51:15.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.420+0000 7f5857d04640 1 -- 172.21.15.89:0/798552982 wait complete. 2024-08-19T10:51:15.462 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:15.463 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-08-19T10:51:15.463 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-08-19T10:51:15.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.580+0000 7f5c75e0a640 10 monclient: get_monmap_and_config 2024-08-19T10:51:15.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.580+0000 7f5c75e0a640 10 monclient: build_initial_monmap 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.580+0000 7f5c75e0a640 10 monclient: monmap: 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:15.587663+0000 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:15.587663+0000 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:15.582 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:15.583 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:15.583 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:15.583 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:15.583 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:15.583 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:15.583 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.580+0000 7f5c75e0a640 1 Processor -- start 2024-08-19T10:51:15.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.580+0000 7f5c75e0a640 1 -- start start 2024-08-19T10:51:15.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.580+0000 7f5c75e0a640 10 monclient: init 2024-08-19T10:51:15.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:15.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 10 monclient: _add_conns ranks=[8,10,5,3,4,9,2,6,0,7,1] 2024-08-19T10:51:15.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c700a6860 0x7f5c700a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 10 monclient(hunting): picked mon.noname-e con 0x7f5c700a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5c700a7180 0x7f5c700a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 10 monclient(hunting): picked mon.noname-f con 0x7f5c700a7180 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c70139620 0x7f5c70139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 10 monclient(hunting): picked mon.noname-h con 0x7f5c70139620 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f5c700a8190 con 0x7f5c700a7180 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f5c700a33d0 con 0x7f5c70139620 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f5c700a2ec0 con 0x7f5c700a6860 2024-08-19T10:51:15.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c75e0a640 10 monclient(hunting): authenticate will time out at 1725.404240s 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c700a6860 0x7f5c700a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c700a6860 0x7f5c700a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48546/0 (socket says 172.21.15.89:48546) 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 1 -- 172.21.15.89:0/2342677691 learned_addr learned my addr 172.21.15.89:0/2342677691 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 10 monclient(hunting): get_auth_request con 0x7f5c700a6860 auth_method 0 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6ffff640 1 --2- 172.21.15.89:0/2342677691 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c70139620 0x7f5c70139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6effd640 1 --2- 172.21.15.89:0/2342677691 >> v2:172.21.15.190:3300/0 conn(0x7f5c700a7180 0x7f5c700a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 10 monclient(hunting): handle_auth_done global_id 7073 payload 274 2024-08-19T10:51:15.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 1 -- 172.21.15.89:0/2342677691 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c70139620 msgr2=0x7f5c70139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:15.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 1 --2- 172.21.15.89:0/2342677691 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c70139620 0x7f5c70139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 1 -- 172.21.15.89:0/2342677691 >> v2:172.21.15.190:3300/0 conn(0x7f5c700a7180 msgr2=0x7f5c700a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:15.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 1 --2- 172.21.15.89:0/2342677691 >> v2:172.21.15.190:3300/0 conn(0x7f5c700a7180 0x7f5c700a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:51:15.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 1 monclient: found mon.noname-e 2024-08-19T10:51:15.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:15.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:51:15.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6f7fe640 1 -- 172.21.15.89:0/2342677691 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c700a4cd0 con 0x7f5c700a6860 2024-08-19T10:51:15.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6ffff640 10 monclient: get_auth_request con 0x7f5c70139620 auth_method 0 2024-08-19T10:51:15.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.584+0000 7f5c6ffff640 1 --2- 172.21.15.89:0/2342677691 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c70139620 0x7f5c70139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:15.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 1 -- 172.21.15.89:0/2342677691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c700a6860 msgr2=0x7f5c700a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 1 --2- 172.21.15.89:0/2342677691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c700a6860 0x7f5c700a6c40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009870 tx=0x7f5c6001c420 comp rx=0 tx=0).ready entity=mon.8 client_cookie=98815f64c938aa2c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 1 -- 172.21.15.89:0/2342677691 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c600b0070 con 0x7f5c700a6860 2024-08-19T10:51:15.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:51:15.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 10 monclient: dump: 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.593 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 1 -- 172.21.15.89:0/2342677691 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c600aa810 con 0x7f5c700a6860 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 1 -- 172.21.15.89:0/2342677691 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c600a93e0 con 0x7f5c700a6860 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 5 monclient: authenticate success, global_id 7073 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 4 set_mon_vals no callback set 2024-08-19T10:51:15.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6e7fc640 10 monclient: dump: 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient: shutdown 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 -- 172.21.15.89:0/2342677691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c700a6860 msgr2=0x7f5c700a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 --2- 172.21.15.89:0/2342677691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c700a6860 0x7f5c700a6c40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009870 tx=0x7f5c6001c420 comp rx=0 tx=0).stop 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 -- 172.21.15.89:0/2342677691 shutdown_connections 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 --2- 172.21.15.89:0/2342677691 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5c70139620 0x7f5c70139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 --2- 172.21.15.89:0/2342677691 >> v2:172.21.15.190:3300/0 conn(0x7f5c700a7180 0x7f5c700a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 --2- 172.21.15.89:0/2342677691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5c700a6860 0x7f5c700a6c40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 -- 172.21.15.89:0/2342677691 >> 172.21.15.89:0/2342677691 conn(0x7f5c70074b40 msgr2=0x7f5c700abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 -- 172.21.15.89:0/2342677691 shutdown_connections 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 -- 172.21.15.89:0/2342677691 wait complete. 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient: build_initial_monmap 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient: monmap: 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 Processor -- start 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 -- start start 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient: init 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient: _add_conns ranks=[4,2,7,8,1,0,3,6,5] 2024-08-19T10:51:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c700a6860 0x7f5c70134830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient(hunting): picked mon.noname-c con 0x7f5c700a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c700a7180 0x7f5c70134d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient(hunting): picked mon.noname-b con 0x7f5c700a7180 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c70139620 0x7f5c7009c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient(hunting): picked mon.noname-i con 0x7f5c70139620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f5c70143210 con 0x7f5c700a7180 2024-08-19T10:51:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f5c70143090 con 0x7f5c700a6860 2024-08-19T10:51:15.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5c70143390 con 0x7f5c70139620 2024-08-19T10:51:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c75e0a640 10 monclient(hunting): authenticate will time out at 1725.411455s 2024-08-19T10:51:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6effd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c700a7180 0x7f5c70134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c700a6860 0x7f5c70134830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6effd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c700a7180 0x7f5c70134d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35406/0 (socket says 172.21.15.89:35406) 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6effd640 1 -- 172.21.15.89:0/3749935239 learned_addr learned my addr 172.21.15.89:0/3749935239 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6effd640 10 monclient(hunting): get_auth_request con 0x7f5c700a7180 auth_method 0 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c700a6860 0x7f5c70134830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38068/0 (socket says 172.21.15.89:38068) 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 10 monclient(hunting): get_auth_request con 0x7f5c700a6860 auth_method 0 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6ffff640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c70139620 0x7f5c7009c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 10 monclient(hunting): handle_auth_done global_id 44995 payload 824 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 1 -- 172.21.15.89:0/3749935239 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c70139620 msgr2=0x7f5c7009c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c70139620 0x7f5c7009c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 1 -- 172.21.15.89:0/3749935239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c700a7180 msgr2=0x7f5c70134d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c700a7180 0x7f5c70134d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:51:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 1 monclient: found mon.noname-c 2024-08-19T10:51:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:51:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6f7fe640 1 -- 172.21.15.89:0/3749935239 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c700a0420 con 0x7f5c700a6860 2024-08-19T10:51:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6ffff640 10 monclient: get_auth_request con 0x7f5c70139620 auth_method 0 2024-08-19T10:51:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6effd640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c700a7180 0x7f5c70134d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.588+0000 7f5c6ffff640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c70139620 0x7f5c7009c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c6f7fe640 1 -- 172.21.15.89:0/3749935239 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c700a6860 msgr2=0x7f5c70134830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:15.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c6f7fe640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c700a6860 0x7f5c70134830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c700a3090 tx=0x7f5c600191c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 1 -- 172.21.15.89:0/3749935239 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c600b0040 con 0x7f5c700a6860 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 10 monclient: dump: 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.608 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 10 monclient: _finish_auth 0 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 10 monclient: _check_auth_tickets 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 1 -- 172.21.15.89:0/3749935239 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c600044b0 con 0x7f5c700a6860 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c75e0a640 5 monclient: authenticate success, global_id 44995 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c75e0a640 10 monclient: _renew_subs 2024-08-19T10:51:15.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c75e0a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c75e0a640 1 -- 172.21.15.89:0/3749935239 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c700ac0f0 con 0x7f5c700a6860 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 1 -- 172.21.15.89:0/3749935239 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5c600aae20 con 0x7f5c700a6860 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 10 monclient: dump: 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:15.610 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c6dffb640 4 set_mon_vals no callback set 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c75e0a640 10 monclient: _renew_subs 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c75e0a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:15.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c75e0a640 1 -- 172.21.15.89:0/3749935239 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c700a2810 con 0x7f5c700a6860 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 1 -- 172.21.15.89:0/3749935239 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5c60004040 con 0x7f5c700a6860 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c75e0a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c75e0a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c75e0a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c75e0a640 1 -- 172.21.15.89:0/3749935239 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c700a6c40 con 0x7f5c700a6860 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c5c03be00 0x7f5c5c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c57fff640 1 -- 172.21.15.89:0/3749935239 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f5c600e88d0 con 0x7f5c700a6860 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.592+0000 7f5c6effd640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c5c03be00 0x7f5c5c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.596+0000 7f5c6effd640 10 monclient: get_auth_request con 0x7f5c5c03be00 auth_method 0 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.596+0000 7f5c6effd640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c5c03be00 0x7f5c5c03e2c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5c7013fa50 tx=0x7f5c64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.600+0000 7f5c57fff640 1 -- 172.21.15.89:0/3749935239 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f5c60016130 con 0x7f5c700a6860 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.600+0000 7f5c57fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:15.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.600+0000 7f5c57fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:15.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.820+0000 7f5c75e0a640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-08-19T10:51:15.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.820+0000 7f5c75e0a640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-08-19T10:51:15.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.820+0000 7f5c75e0a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:15.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.820+0000 7f5c75e0a640 1 -- 172.21.15.89:0/3749935239 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f5c70064590 con 0x7f5c700a6860 2024-08-19T10:51:15.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.824+0000 7f5c57fff640 1 -- 172.21.15.89:0/3749935239 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f5c600e6790 con 0x7f5c700a6860 2024-08-19T10:51:15.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.824+0000 7f5c57fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-08-19T10:51:15.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.824+0000 7f5c57fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:15.828 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.828 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=12,kv=12,logm=54} crc {auth=1487050827,config=2586702516,health=3107130123,kv=1101891974,logm=3594006952}) 2024-08-19T10:51:15.828 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=664937413,config=2586702516,health=3321548992,kv=283592551,logm=694778853}) 2024-08-19T10:51:15.828 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=12,kv=12,logm=54} crc {auth=1487050827,config=2586702516,health=3107130123,kv=1101891974,logm=3594006952}) 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=20,config=2,health=11,kv=11,logm=56} crc {auth=2592062484,config=2586702516,health=3324490451,kv=1905375608,logm=825379205}) 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=12,kv=12,logm=54} crc {auth=1487050827,config=2586702516,health=3107130123,kv=1101891974,logm=3594006952}) 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=21,config=2,health=10,kv=12,logm=55} crc {auth=11656529,config=2586702516,health=2694878226,kv=1101891974,logm=1767952671}) 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=12,kv=12,logm=54} crc {auth=1487050827,config=2586702516,health=3107130123,kv=1101891974,logm=3594006952}) 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=19,config=2,health=12,kv=11,logm=56} crc {auth=248408577,config=2586702516,health=3353365297,kv=2354184577,logm=2674545046}) 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=12,kv=12,logm=54} crc {auth=1487050827,config=2586702516,health=3107130123,kv=1101891974,logm=3594006952}) 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=21,config=2,health=10,kv=12,logm=55} crc {auth=11656529,config=2586702516,health=434802332,kv=1101891974,logm=131397455}) 2024-08-19T10:51:15.829 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=12,kv=12,logm=54} crc {auth=1487050827,config=2586702516,health=3107130123,kv=1101891974,logm=3594006952}) 2024-08-19T10:51:15.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=21,config=2,health=12,kv=12,logm=53} crc {auth=11656529,config=2586702516,health=3353365297,kv=1101891974,logm=167655758}) 2024-08-19T10:51:15.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=12,kv=12,logm=54} crc {auth=1487050827,config=2586702516,health=3107130123,kv=1101891974,logm=3594006952}) 2024-08-19T10:51:15.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=21,config=2,health=12,kv=12,logm=53} crc {auth=682345387,config=2586702516,health=856284876,kv=1101891974,logm=3434794114}) 2024-08-19T10:51:15.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=12,kv=12,logm=54} crc {auth=1487050827,config=2586702516,health=3107130123,kv=1101891974,logm=3594006952}) 2024-08-19T10:51:15.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=12,kv=11,logm=55} crc {auth=3950741581,config=2586702516,health=3323212128,kv=2545447173,logm=1811701858}) 2024-08-19T10:51:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 -- 172.21.15.89:0/3749935239 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c5c03be00 msgr2=0x7f5c5c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c5c03be00 0x7f5c5c03e2c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5c7013fa50 tx=0x7f5c64008040 comp rx=0 tx=0).stop 2024-08-19T10:51:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 10 monclient: shutdown 2024-08-19T10:51:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 -- 172.21.15.89:0/3749935239 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c700a6860 msgr2=0x7f5c70134830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c700a6860 0x7f5c70134830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c700a3090 tx=0x7f5c600191c0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 -- 172.21.15.89:0/3749935239 shutdown_connections 2024-08-19T10:51:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5c5c03be00 0x7f5c5c03e2c0 secure :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5c7013fa50 tx=0x7f5c64008040 comp rx=0 tx=0).stop 2024-08-19T10:51:15.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5c70139620 0x7f5c7009c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5c700a7180 0x7f5c70134d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 --2- 172.21.15.89:0/3749935239 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5c700a6860 0x7f5c70134830 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 -- 172.21.15.89:0/3749935239 >> 172.21.15.89:0/3749935239 conn(0x7f5c70074b40 msgr2=0x7f5c700ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 -- 172.21.15.89:0/3749935239 shutdown_connections 2024-08-19T10:51:15.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f5c75e0a640 1 -- 172.21.15.89:0/3749935239 wait complete. 2024-08-19T10:51:15.833 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2502438205}) 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3480491588}) 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2502438205}) 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3432329708}) 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2502438205}) 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4174130452}) 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2502438205}) 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1689588571}) 2024-08-19T10:51:15.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2502438205}) 2024-08-19T10:51:15.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.828+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4289002784}) 2024-08-19T10:51:15.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2502438205}) 2024-08-19T10:51:15.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1910547593}) 2024-08-19T10:51:15.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2502438205}) 2024-08-19T10:51:15.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=16703086}) 2024-08-19T10:51:15.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2502438205}) 2024-08-19T10:51:15.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1950031649}) 2024-08-19T10:51:15.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219756021}) 2024-08-19T10:51:15.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=571513610}) 2024-08-19T10:51:15.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219756021}) 2024-08-19T10:51:15.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2377483324}) 2024-08-19T10:51:15.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219756021}) 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=583607722}) 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219756021}) 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.832+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2992902100}) 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219756021}) 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1163408455}) 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219756021}) 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2877736468}) 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219756021}) 2024-08-19T10:51:15.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1265456114}) 2024-08-19T10:51:15.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1219756021}) 2024-08-19T10:51:15.840 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.836+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3164763369}) 2024-08-19T10:51:15.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4077236701}) 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2361017453}) 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4077236701}) 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3227903314}) 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4077236701}) 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3497836630}) 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4077236701}) 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=224198298}) 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4077236701}) 2024-08-19T10:51:15.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4071725812}) 2024-08-19T10:51:15.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4077236701}) 2024-08-19T10:51:15.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3428914042}) 2024-08-19T10:51:15.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4077236701}) 2024-08-19T10:51:15.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3567176114}) 2024-08-19T10:51:15.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4077236701}) 2024-08-19T10:51:15.845 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.840+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=122312441}) 2024-08-19T10:51:15.850 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.850 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636233488}) 2024-08-19T10:51:15.850 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3884933574}) 2024-08-19T10:51:15.850 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.850 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636233488}) 2024-08-19T10:51:15.850 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3379653617}) 2024-08-19T10:51:15.850 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636233488}) 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=116999476}) 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636233488}) 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4055141148}) 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636233488}) 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=156954880}) 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636233488}) 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3440426391}) 2024-08-19T10:51:15.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636233488}) 2024-08-19T10:51:15.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=568464529}) 2024-08-19T10:51:15.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636233488}) 2024-08-19T10:51:15.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.844+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3646290737}) 2024-08-19T10:51:15.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043504088}) 2024-08-19T10:51:15.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1831925036}) 2024-08-19T10:51:15.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043504088}) 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2981785986}) 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043504088}) 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3993395362}) 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043504088}) 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3233473063}) 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043504088}) 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2839989441}) 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043504088}) 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3042902952}) 2024-08-19T10:51:15.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043504088}) 2024-08-19T10:51:15.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2374757882}) 2024-08-19T10:51:15.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2043504088}) 2024-08-19T10:51:15.854 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.848+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=192109665}) 2024-08-19T10:51:15.855 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-08-19T10:51:15.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1195192628}) 2024-08-19T10:51:15.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4133120531}) 2024-08-19T10:51:15.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1195192628}) 2024-08-19T10:51:15.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=176108252}) 2024-08-19T10:51:15.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1195192628}) 2024-08-19T10:51:15.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2305763970}) 2024-08-19T10:51:15.862 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1195192628}) 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3022739021}) 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1195192628}) 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4222452447}) 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1195192628}) 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2291898242}) 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1195192628}) 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3285107546}) 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1195192628}) 2024-08-19T10:51:15.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.852+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3178801350}) 2024-08-19T10:51:15.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1151634155}) 2024-08-19T10:51:15.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1224507923}) 2024-08-19T10:51:15.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1151634155}) 2024-08-19T10:51:15.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3083078113}) 2024-08-19T10:51:15.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1151634155}) 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3493894741}) 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1151634155}) 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2437995366}) 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1151634155}) 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2368855545}) 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1151634155}) 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3984937198}) 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1151634155}) 2024-08-19T10:51:15.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=259328059}) 2024-08-19T10:51:15.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1151634155}) 2024-08-19T10:51:15.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.856+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3793471357}) 2024-08-19T10:51:15.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1068005872}) 2024-08-19T10:51:15.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=141807150}) 2024-08-19T10:51:15.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1068005872}) 2024-08-19T10:51:15.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=119011266}) 2024-08-19T10:51:15.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1068005872}) 2024-08-19T10:51:15.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2448894210}) 2024-08-19T10:51:15.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1068005872}) 2024-08-19T10:51:15.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2524573110}) 2024-08-19T10:51:15.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1068005872}) 2024-08-19T10:51:15.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3586899978}) 2024-08-19T10:51:15.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.868 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1068005872}) 2024-08-19T10:51:15.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3678124844}) 2024-08-19T10:51:15.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1068005872}) 2024-08-19T10:51:15.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=249858612}) 2024-08-19T10:51:15.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1068005872}) 2024-08-19T10:51:15.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.860+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3976518613}) 2024-08-19T10:51:15.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985247813}) 2024-08-19T10:51:15.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2575650503}) 2024-08-19T10:51:15.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985247813}) 2024-08-19T10:51:15.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2392979346}) 2024-08-19T10:51:15.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985247813}) 2024-08-19T10:51:15.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3979397088}) 2024-08-19T10:51:15.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.881 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985247813}) 2024-08-19T10:51:15.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3095968508}) 2024-08-19T10:51:15.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985247813}) 2024-08-19T10:51:15.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1272315280}) 2024-08-19T10:51:15.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985247813}) 2024-08-19T10:51:15.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608983678}) 2024-08-19T10:51:15.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985247813}) 2024-08-19T10:51:15.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2140129318}) 2024-08-19T10:51:15.882 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2985247813}) 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.864+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1816709183}) 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2744491073}) 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3290917168}) 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2744491073}) 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1289663315}) 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2744491073}) 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=176930920}) 2024-08-19T10:51:15.883 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2744491073}) 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1725930336}) 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2744491073}) 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1330122295}) 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2744491073}) 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=248451836}) 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2744491073}) 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2136851028}) 2024-08-19T10:51:15.884 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.885 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2744491073}) 2024-08-19T10:51:15.885 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4239277978}) 2024-08-19T10:51:15.885 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.885 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072321142}) 2024-08-19T10:51:15.885 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1480848501}) 2024-08-19T10:51:15.885 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.868+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.885 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072321142}) 2024-08-19T10:51:15.885 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2196339359}) 2024-08-19T10:51:15.885 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.885 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072321142}) 2024-08-19T10:51:15.885 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2276041653}) 2024-08-19T10:51:15.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072321142}) 2024-08-19T10:51:15.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=904872838}) 2024-08-19T10:51:15.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072321142}) 2024-08-19T10:51:15.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1746137442}) 2024-08-19T10:51:15.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072321142}) 2024-08-19T10:51:15.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2503053646}) 2024-08-19T10:51:15.886 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072321142}) 2024-08-19T10:51:15.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=364444766}) 2024-08-19T10:51:15.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072321142}) 2024-08-19T10:51:15.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1568981582}) 2024-08-19T10:51:15.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=129455082}) 2024-08-19T10:51:15.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1042940181}) 2024-08-19T10:51:15.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.887 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=129455082}) 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825155353}) 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.872+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=129455082}) 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3944607788}) 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=129455082}) 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=317379731}) 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=129455082}) 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2625892071}) 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=129455082}) 2024-08-19T10:51:15.888 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=597734402}) 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=129455082}) 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=153807356}) 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=129455082}) 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=720468522}) 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613659463}) 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1215705956}) 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613659463}) 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3264018056}) 2024-08-19T10:51:15.889 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613659463}) 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=43479699}) 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613659463}) 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2037766670}) 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613659463}) 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1149126920}) 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613659463}) 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.876+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3138584266}) 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.890 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613659463}) 2024-08-19T10:51:15.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2630652512}) 2024-08-19T10:51:15.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2613659463}) 2024-08-19T10:51:15.891 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1607754574}) 2024-08-19T10:51:15.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3805196310}) 2024-08-19T10:51:15.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3063589552}) 2024-08-19T10:51:15.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3805196310}) 2024-08-19T10:51:15.914 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3442933317}) 2024-08-19T10:51:15.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3805196310}) 2024-08-19T10:51:15.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2409995151}) 2024-08-19T10:51:15.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3805196310}) 2024-08-19T10:51:15.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=486123847}) 2024-08-19T10:51:15.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3805196310}) 2024-08-19T10:51:15.915 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2507477754}) 2024-08-19T10:51:15.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3805196310}) 2024-08-19T10:51:15.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3238970623}) 2024-08-19T10:51:15.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3805196310}) 2024-08-19T10:51:15.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=585953286}) 2024-08-19T10:51:15.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3805196310}) 2024-08-19T10:51:15.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.880+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2724222328}) 2024-08-19T10:51:15.916 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1947023461}) 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4044782159}) 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1947023461}) 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4021449407}) 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1947023461}) 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1395583056}) 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1947023461}) 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=578760264}) 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.917 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1947023461}) 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1720121670}) 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1947023461}) 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1081196416}) 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1947023461}) 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=102311492}) 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1947023461}) 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.884+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3566183797}) 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581308736}) 2024-08-19T10:51:15.918 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3086318325}) 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581308736}) 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2124532022}) 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581308736}) 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3808938138}) 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581308736}) 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=330100244}) 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581308736}) 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3525730156}) 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.919 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581308736}) 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2178352528}) 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581308736}) 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2951504523}) 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581308736}) 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.888+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2392581935}) 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3023489688}) 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3151458689}) 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3023489688}) 2024-08-19T10:51:15.920 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1457016065}) 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3023489688}) 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2912731089}) 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3023489688}) 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1355217995}) 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3023489688}) 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3318405882}) 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3023489688}) 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2410250360}) 2024-08-19T10:51:15.921 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3023489688}) 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1693355334}) 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3023489688}) 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.892+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4214584656}) 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1731870696}) 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2924916838}) 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1731870696}) 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4061714242}) 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.922 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1731870696}) 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=137666621}) 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1731870696}) 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1080175905}) 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1731870696}) 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1122377697}) 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1731870696}) 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2715014715}) 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.923 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1731870696}) 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1899230650}) 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1731870696}) 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3589559274}) 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366774614}) 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1148743500}) 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366774614}) 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=295884620}) 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.924 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366774614}) 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3910301140}) 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366774614}) 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.896+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1644862289}) 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366774614}) 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3576213138}) 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366774614}) 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1200145222}) 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.925 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366774614}) 2024-08-19T10:51:15.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=424926974}) 2024-08-19T10:51:15.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366774614}) 2024-08-19T10:51:15.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1109767122}) 2024-08-19T10:51:15.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=211439672}) 2024-08-19T10:51:15.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1834141508}) 2024-08-19T10:51:15.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=211439672}) 2024-08-19T10:51:15.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=638363719}) 2024-08-19T10:51:15.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=211439672}) 2024-08-19T10:51:15.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2034588883}) 2024-08-19T10:51:15.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=211439672}) 2024-08-19T10:51:15.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=377220045}) 2024-08-19T10:51:15.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=211439672}) 2024-08-19T10:51:15.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=416402333}) 2024-08-19T10:51:15.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=211439672}) 2024-08-19T10:51:15.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2113874656}) 2024-08-19T10:51:15.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=211439672}) 2024-08-19T10:51:15.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=343432712}) 2024-08-19T10:51:15.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=211439672}) 2024-08-19T10:51:15.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.900+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3512773850}) 2024-08-19T10:51:15.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3198689036}) 2024-08-19T10:51:15.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1783977694}) 2024-08-19T10:51:15.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3198689036}) 2024-08-19T10:51:15.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=615179362}) 2024-08-19T10:51:15.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3198689036}) 2024-08-19T10:51:15.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=322043620}) 2024-08-19T10:51:15.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3198689036}) 2024-08-19T10:51:15.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2080170128}) 2024-08-19T10:51:15.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3198689036}) 2024-08-19T10:51:15.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2416144777}) 2024-08-19T10:51:15.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3198689036}) 2024-08-19T10:51:15.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=41079949}) 2024-08-19T10:51:15.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3198689036}) 2024-08-19T10:51:15.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=567587376}) 2024-08-19T10:51:15.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3198689036}) 2024-08-19T10:51:15.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.904+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3755532412}) 2024-08-19T10:51:15.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4129893824}) 2024-08-19T10:51:15.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3104217105}) 2024-08-19T10:51:15.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4129893824}) 2024-08-19T10:51:15.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1972530799}) 2024-08-19T10:51:15.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4129893824}) 2024-08-19T10:51:15.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3304604307}) 2024-08-19T10:51:15.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4129893824}) 2024-08-19T10:51:15.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1798170834}) 2024-08-19T10:51:15.932 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.932 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4129893824}) 2024-08-19T10:51:15.932 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2754644073}) 2024-08-19T10:51:15.932 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.932 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4129893824}) 2024-08-19T10:51:15.932 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=72154209}) 2024-08-19T10:51:15.932 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.932 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4129893824}) 2024-08-19T10:51:15.932 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1358073963}) 2024-08-19T10:51:15.932 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.932 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4129893824}) 2024-08-19T10:51:15.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.908+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3884592222}) 2024-08-19T10:51:15.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3476553791}) 2024-08-19T10:51:15.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=10159143}) 2024-08-19T10:51:15.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3476553791}) 2024-08-19T10:51:15.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1036927248}) 2024-08-19T10:51:15.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3476553791}) 2024-08-19T10:51:15.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3569295608}) 2024-08-19T10:51:15.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3476553791}) 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=979780602}) 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3476553791}) 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3704181969}) 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3476553791}) 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=461291894}) 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3476553791}) 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3239288381}) 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3476553791}) 2024-08-19T10:51:15.935 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.912+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=963373536}) 2024-08-19T10:51:15.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1901909684}) 2024-08-19T10:51:15.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=64014601}) 2024-08-19T10:51:15.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1901909684}) 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2395706554}) 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1901909684}) 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4168590359}) 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1901909684}) 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4233581650}) 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1901909684}) 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2368270356}) 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1901909684}) 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=360320991}) 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1901909684}) 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1562977827}) 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1901909684}) 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088746650}) 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448197643}) 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659852804}) 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448197643}) 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3098499608}) 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448197643}) 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1042860024}) 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448197643}) 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3355372433}) 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448197643}) 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3926617938}) 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.955 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448197643}) 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1764998915}) 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448197643}) 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2189709498}) 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448197643}) 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.916+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1795179137}) 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065396327}) 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3820684010}) 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065396327}) 2024-08-19T10:51:15.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=768371591}) 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065396327}) 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2611404907}) 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065396327}) 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=53853180}) 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065396327}) 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3403891805}) 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065396327}) 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=726508391}) 2024-08-19T10:51:15.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065396327}) 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3336943541}) 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3065396327}) 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.924+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2088534375}) 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245434245}) 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2590265238}) 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245434245}) 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3603979504}) 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245434245}) 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=228576389}) 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245434245}) 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2761348775}) 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245434245}) 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2721348932}) 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245434245}) 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=496066078}) 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245434245}) 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2727725057}) 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245434245}) 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.932+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4173993992}) 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2962948359}) 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3016466241}) 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2962948359}) 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3736069401}) 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2962948359}) 2024-08-19T10:51:15.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=471116263}) 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2962948359}) 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1191153553}) 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2962948359}) 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1477589539}) 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2962948359}) 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1019920409}) 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2962948359}) 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3454866266}) 2024-08-19T10:51:15.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2962948359}) 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2058870238}) 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852474814}) 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1338220663}) 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852474814}) 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=596954909}) 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852474814}) 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3820973189}) 2024-08-19T10:51:15.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852474814}) 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3768787106}) 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852474814}) 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1996186254}) 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852474814}) 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=197483991}) 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852474814}) 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1304992313}) 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.963 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852474814}) 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=64614638}) 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=357635891}) 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2550317994}) 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=357635891}) 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3723801189}) 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=357635891}) 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2438108564}) 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=357635891}) 2024-08-19T10:51:15.964 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=786051645}) 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=357635891}) 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3607730985}) 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=357635891}) 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1978066836}) 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=357635891}) 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=410686573}) 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=357635891}) 2024-08-19T10:51:15.965 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1143463031}) 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4168629221}) 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2701289723}) 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4168629221}) 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=687913127}) 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4168629221}) 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1510414280}) 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4168629221}) 2024-08-19T10:51:15.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1446656487}) 2024-08-19T10:51:15.970 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.970 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4168629221}) 2024-08-19T10:51:15.970 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3333457665}) 2024-08-19T10:51:15.970 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.970 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4168629221}) 2024-08-19T10:51:15.970 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=873362219}) 2024-08-19T10:51:15.970 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.970 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4168629221}) 2024-08-19T10:51:15.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=609517476}) 2024-08-19T10:51:15.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4168629221}) 2024-08-19T10:51:15.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2142374925}) 2024-08-19T10:51:15.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3112878977}) 2024-08-19T10:51:15.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3606451905}) 2024-08-19T10:51:15.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3112878977}) 2024-08-19T10:51:15.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2462001084}) 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3112878977}) 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=889443691}) 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3112878977}) 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1605717918}) 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3112878977}) 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1053822455}) 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3112878977}) 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1734790638}) 2024-08-19T10:51:15.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3112878977}) 2024-08-19T10:51:15.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4245002119}) 2024-08-19T10:51:15.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:15.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3112878977}) 2024-08-19T10:51:15.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:15.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1016424681}) 2024-08-19T10:51:16.005 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=36052354}) 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3670722020}) 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=36052354}) 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1825992155}) 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=36052354}) 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2401297777}) 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=36052354}) 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4204939676}) 2024-08-19T10:51:16.006 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.007 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=36052354}) 2024-08-19T10:51:16.007 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3601159621}) 2024-08-19T10:51:16.007 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.007 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=36052354}) 2024-08-19T10:51:16.007 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3392258041}) 2024-08-19T10:51:16.007 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.007 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=36052354}) 2024-08-19T10:51:16.007 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=180598240}) 2024-08-19T10:51:16.007 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.007 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=36052354}) 2024-08-19T10:51:16.007 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2493048278}) 2024-08-19T10:51:16.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2232082393}) 2024-08-19T10:51:16.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2032402259}) 2024-08-19T10:51:16.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2232082393}) 2024-08-19T10:51:16.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2206159903}) 2024-08-19T10:51:16.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2232082393}) 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1279063817}) 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2232082393}) 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=589250481}) 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2232082393}) 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4294601892}) 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2232082393}) 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1762094515}) 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2232082393}) 2024-08-19T10:51:16.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2638397093}) 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2232082393}) 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=493099505}) 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1551470035}) 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=372699082}) 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1551470035}) 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=424286937}) 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1551470035}) 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1225313081}) 2024-08-19T10:51:16.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1551470035}) 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1970186324}) 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1551470035}) 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1919077092}) 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1551470035}) 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3941947780}) 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1551470035}) 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3547855986}) 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.020 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1551470035}) 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.008+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=731720530}) 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2933726480}) 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=848802769}) 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2933726480}) 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=818354925}) 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2933726480}) 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2664550230}) 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2933726480}) 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=914326354}) 2024-08-19T10:51:16.021 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2933726480}) 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=13384850}) 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2933726480}) 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3206379114}) 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2933726480}) 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2102178558}) 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2933726480}) 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1653446006}) 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=269252491}) 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3573223893}) 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=269252491}) 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2501468825}) 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=269252491}) 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2777742658}) 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=269252491}) 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2136507862}) 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=269252491}) 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3663821049}) 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=269252491}) 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3069380096}) 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=269252491}) 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4155730174}) 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=269252491}) 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.012+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=541949011}) 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1226328353}) 2024-08-19T10:51:16.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=790948265}) 2024-08-19T10:51:16.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1226328353}) 2024-08-19T10:51:16.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1856593305}) 2024-08-19T10:51:16.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1226328353}) 2024-08-19T10:51:16.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=458585071}) 2024-08-19T10:51:16.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1226328353}) 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=508981988}) 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1226328353}) 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2787984223}) 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1226328353}) 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3542445582}) 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1226328353}) 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3202653038}) 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1226328353}) 2024-08-19T10:51:16.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=123328555}) 2024-08-19T10:51:16.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4145605356}) 2024-08-19T10:51:16.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2612258727}) 2024-08-19T10:51:16.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.042 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4145605356}) 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3106116036}) 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4145605356}) 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2681544498}) 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4145605356}) 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2530499462}) 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4145605356}) 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=340483224}) 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4145605356}) 2024-08-19T10:51:16.043 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2386653349}) 2024-08-19T10:51:16.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4145605356}) 2024-08-19T10:51:16.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1367341225}) 2024-08-19T10:51:16.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4145605356}) 2024-08-19T10:51:16.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.016+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2308335148}) 2024-08-19T10:51:16.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457939561}) 2024-08-19T10:51:16.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2021270450}) 2024-08-19T10:51:16.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.044 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457939561}) 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=54911395}) 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457939561}) 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3767706330}) 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457939561}) 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2488599282}) 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457939561}) 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1312604980}) 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457939561}) 2024-08-19T10:51:16.045 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4291549565}) 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457939561}) 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1985480551}) 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457939561}) 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=819032243}) 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=471264639}) 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=774624121}) 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=471264639}) 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2737182860}) 2024-08-19T10:51:16.046 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=471264639}) 2024-08-19T10:51:16.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3476372343}) 2024-08-19T10:51:16.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=471264639}) 2024-08-19T10:51:16.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=947346801}) 2024-08-19T10:51:16.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=471264639}) 2024-08-19T10:51:16.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1384517344}) 2024-08-19T10:51:16.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.047 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=471264639}) 2024-08-19T10:51:16.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2500543471}) 2024-08-19T10:51:16.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=471264639}) 2024-08-19T10:51:16.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3619409807}) 2024-08-19T10:51:16.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=471264639}) 2024-08-19T10:51:16.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4152552587}) 2024-08-19T10:51:16.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914180169}) 2024-08-19T10:51:16.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1413223332}) 2024-08-19T10:51:16.048 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914180169}) 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1704927815}) 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914180169}) 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3296291456}) 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914180169}) 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=339151468}) 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914180169}) 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2497183480}) 2024-08-19T10:51:16.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914180169}) 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3100299609}) 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914180169}) 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=974670169}) 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914180169}) 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1551885579}) 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123220080}) 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=9418392}) 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123220080}) 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=542748786}) 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123220080}) 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3516809094}) 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123220080}) 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2575536759}) 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123220080}) 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1583587803}) 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123220080}) 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2767109018}) 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123220080}) 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2212750123}) 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3123220080}) 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.024+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=111226131}) 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=529584235}) 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3008674336}) 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=529584235}) 2024-08-19T10:51:16.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4010484001}) 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=529584235}) 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1040042171}) 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=529584235}) 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2983569817}) 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=529584235}) 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1413003631}) 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=529584235}) 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=957490504}) 2024-08-19T10:51:16.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=529584235}) 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3300523612}) 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=529584235}) 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1716766723}) 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1549597116}) 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=609908122}) 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1549597116}) 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3868384339}) 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1549597116}) 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3351977944}) 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1549597116}) 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2331458493}) 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1549597116}) 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=918586930}) 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1549597116}) 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3554291384}) 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1549597116}) 2024-08-19T10:51:16.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=510741529}) 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1549597116}) 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.032+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2406863600}) 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=662334475}) 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1530794764}) 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=662334475}) 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4271753621}) 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=662334475}) 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1199145507}) 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=662334475}) 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4198039635}) 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=662334475}) 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2206954189}) 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=662334475}) 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4058082396}) 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=662334475}) 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1644877744}) 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=662334475}) 2024-08-19T10:51:16.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2657670291}) 2024-08-19T10:51:16.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1393570081}) 2024-08-19T10:51:16.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3044738699}) 2024-08-19T10:51:16.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1393570081}) 2024-08-19T10:51:16.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1616407527}) 2024-08-19T10:51:16.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1393570081}) 2024-08-19T10:51:16.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=165286205}) 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1393570081}) 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=736006452}) 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1393570081}) 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3169170770}) 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1393570081}) 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3490796650}) 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1393570081}) 2024-08-19T10:51:16.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3610019133}) 2024-08-19T10:51:16.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1393570081}) 2024-08-19T10:51:16.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3523603234}) 2024-08-19T10:51:16.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510872934}) 2024-08-19T10:51:16.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=640802490}) 2024-08-19T10:51:16.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510872934}) 2024-08-19T10:51:16.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2300909044}) 2024-08-19T10:51:16.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510872934}) 2024-08-19T10:51:16.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1867467623}) 2024-08-19T10:51:16.067 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510872934}) 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1807265408}) 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510872934}) 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1324951357}) 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510872934}) 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3958911315}) 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510872934}) 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1030743347}) 2024-08-19T10:51:16.068 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1510872934}) 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2610617954}) 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391868717}) 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1324566805}) 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391868717}) 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2646389480}) 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391868717}) 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=682657562}) 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.069 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391868717}) 2024-08-19T10:51:16.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1986517942}) 2024-08-19T10:51:16.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391868717}) 2024-08-19T10:51:16.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2326865778}) 2024-08-19T10:51:16.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391868717}) 2024-08-19T10:51:16.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2106094641}) 2024-08-19T10:51:16.070 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391868717}) 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1589597306}) 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391868717}) 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1765404551}) 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2336104478}) 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3626533719}) 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2336104478}) 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4086336508}) 2024-08-19T10:51:16.071 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2336104478}) 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1549943138}) 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2336104478}) 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3264759640}) 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2336104478}) 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3022291661}) 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2336104478}) 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1824060495}) 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.072 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2336104478}) 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4203634340}) 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2336104478}) 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3256791576}) 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328829952}) 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3145253689}) 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328829952}) 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4217179468}) 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328829952}) 2024-08-19T10:51:16.073 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2626922369}) 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328829952}) 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1580758013}) 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328829952}) 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1813914360}) 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328829952}) 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1828640963}) 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328829952}) 2024-08-19T10:51:16.074 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4156030467}) 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328829952}) 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1274723278}) 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2092648191}) 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1235655386}) 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2092648191}) 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1544329486}) 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2092648191}) 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2330430379}) 2024-08-19T10:51:16.075 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2092648191}) 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=809693785}) 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2092648191}) 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1212057637}) 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2092648191}) 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3850384278}) 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2092648191}) 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1013891988}) 2024-08-19T10:51:16.076 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2092648191}) 2024-08-19T10:51:16.077 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2383939066}) 2024-08-19T10:51:16.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3327692180}) 2024-08-19T10:51:16.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1868501525}) 2024-08-19T10:51:16.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3327692180}) 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1360403018}) 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3327692180}) 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3228550439}) 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3327692180}) 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1695473930}) 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3327692180}) 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2028780662}) 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3327692180}) 2024-08-19T10:51:16.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2418767280}) 2024-08-19T10:51:16.587 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.587 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3327692180}) 2024-08-19T10:51:16.587 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=218209881}) 2024-08-19T10:51:16.587 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.587 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3327692180}) 2024-08-19T10:51:16.587 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1612537917}) 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636683916}) 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1465259635}) 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636683916}) 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1034060316}) 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636683916}) 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2982947941}) 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636683916}) 2024-08-19T10:51:16.591 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4111537523}) 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636683916}) 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3284881493}) 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636683916}) 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1268477927}) 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636683916}) 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1432177113}) 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636683916}) 2024-08-19T10:51:16.592 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2429883977}) 2024-08-19T10:51:16.856 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:51:16.857 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2102306949}) 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1307657243}) 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2102306949}) 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2977330110}) 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2102306949}) 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=916142338}) 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2102306949}) 2024-08-19T10:51:16.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=214280595}) 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2102306949}) 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3044354097}) 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2102306949}) 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=134539710}) 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2102306949}) 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1685370119}) 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2102306949}) 2024-08-19T10:51:16.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.936+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2545587653}) 2024-08-19T10:51:16.940 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.940 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1078299773}) 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4114226765}) 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1078299773}) 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2478674934}) 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1078299773}) 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2796573305}) 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1078299773}) 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3211671684}) 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1078299773}) 2024-08-19T10:51:16.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2275444}) 2024-08-19T10:51:16.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1078299773}) 2024-08-19T10:51:16.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3459270081}) 2024-08-19T10:51:16.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1078299773}) 2024-08-19T10:51:16.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1415835867}) 2024-08-19T10:51:16.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1078299773}) 2024-08-19T10:51:16.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.940+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2727044135}) 2024-08-19T10:51:16.944 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.944 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=537411572}) 2024-08-19T10:51:16.944 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1395247860}) 2024-08-19T10:51:16.944 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.944 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=537411572}) 2024-08-19T10:51:16.944 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2169475457}) 2024-08-19T10:51:16.944 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=537411572}) 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3696052858}) 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=537411572}) 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2382404068}) 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=537411572}) 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1158346635}) 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=537411572}) 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3971705425}) 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=537411572}) 2024-08-19T10:51:16.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1897255260}) 2024-08-19T10:51:16.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=537411572}) 2024-08-19T10:51:16.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.944+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1991152872}) 2024-08-19T10:51:16.948 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.948 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273935517}) 2024-08-19T10:51:16.948 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1626577666}) 2024-08-19T10:51:16.948 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.948 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273935517}) 2024-08-19T10:51:16.948 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2681088024}) 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273935517}) 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=492245670}) 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273935517}) 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1553469299}) 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273935517}) 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=714528773}) 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273935517}) 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3414838890}) 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.949 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273935517}) 2024-08-19T10:51:16.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3674681477}) 2024-08-19T10:51:16.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273935517}) 2024-08-19T10:51:16.950 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.948+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3006891738}) 2024-08-19T10:51:16.952 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.952 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977197029}) 2024-08-19T10:51:16.952 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3333666017}) 2024-08-19T10:51:16.952 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.952 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977197029}) 2024-08-19T10:51:16.952 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3318315159}) 2024-08-19T10:51:16.952 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.952 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977197029}) 2024-08-19T10:51:16.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1244723987}) 2024-08-19T10:51:16.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977197029}) 2024-08-19T10:51:16.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3748921340}) 2024-08-19T10:51:16.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977197029}) 2024-08-19T10:51:16.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=268015481}) 2024-08-19T10:51:16.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977197029}) 2024-08-19T10:51:16.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2445794753}) 2024-08-19T10:51:16.953 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977197029}) 2024-08-19T10:51:16.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4293041823}) 2024-08-19T10:51:16.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977197029}) 2024-08-19T10:51:16.954 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.952+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1086089740}) 2024-08-19T10:51:16.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2616325795}) 2024-08-19T10:51:16.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=813496273}) 2024-08-19T10:51:16.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2616325795}) 2024-08-19T10:51:16.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3000916139}) 2024-08-19T10:51:16.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2616325795}) 2024-08-19T10:51:16.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2825552954}) 2024-08-19T10:51:16.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.959 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2616325795}) 2024-08-19T10:51:16.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3729685955}) 2024-08-19T10:51:16.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2616325795}) 2024-08-19T10:51:16.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3084791042}) 2024-08-19T10:51:16.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2616325795}) 2024-08-19T10:51:16.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=364017165}) 2024-08-19T10:51:16.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.960 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2616325795}) 2024-08-19T10:51:16.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=798389768}) 2024-08-19T10:51:16.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2616325795}) 2024-08-19T10:51:16.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.956+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=619587253}) 2024-08-19T10:51:16.966 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.966 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955298861}) 2024-08-19T10:51:16.966 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3144552775}) 2024-08-19T10:51:16.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955298861}) 2024-08-19T10:51:16.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1630993706}) 2024-08-19T10:51:16.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955298861}) 2024-08-19T10:51:16.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1189080653}) 2024-08-19T10:51:16.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955298861}) 2024-08-19T10:51:16.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3150124223}) 2024-08-19T10:51:16.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955298861}) 2024-08-19T10:51:16.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1730980564}) 2024-08-19T10:51:16.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955298861}) 2024-08-19T10:51:16.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=186516973}) 2024-08-19T10:51:16.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955298861}) 2024-08-19T10:51:16.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1392794520}) 2024-08-19T10:51:16.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3955298861}) 2024-08-19T10:51:16.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=530404595}) 2024-08-19T10:51:16.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=239938718}) 2024-08-19T10:51:16.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1061137199}) 2024-08-19T10:51:16.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=239938718}) 2024-08-19T10:51:16.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1047133092}) 2024-08-19T10:51:16.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=239938718}) 2024-08-19T10:51:16.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3073050793}) 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=239938718}) 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1455535227}) 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=239938718}) 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=366396462}) 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=239938718}) 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4096372934}) 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=239938718}) 2024-08-19T10:51:16.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2222131763}) 2024-08-19T10:51:16.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:16.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=239938718}) 2024-08-19T10:51:16.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:16.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1028635698}) 2024-08-19T10:51:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient: get_monmap_and_config 2024-08-19T10:51:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient: build_initial_monmap 2024-08-19T10:51:17.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient: monmap: 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:17.006508+0000 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:17.006508+0000 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:17.001 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:17.002 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:17.002 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:17.002 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:17.002 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:17.002 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:17.002 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 1 Processor -- start 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 1 -- start start 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient: init 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient: _add_conns ranks=[1,8,10,7,6,4,3,9,0,5,2] 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4d000a6860 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4d000a7350 0x7f4d0013e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient(hunting): picked mon.noname-e con 0x7f4d000a7350 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4d000a7730 0x7f4d00140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient(hunting): picked mon.noname-f con 0x7f4d000a7730 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d000a33d0 con 0x7f4d000a6860 2024-08-19T10:51:17.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f4d000a8190 con 0x7f4d000a7730 2024-08-19T10:51:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4d000a2ec0 con 0x7f4d000a7350 2024-08-19T10:51:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d077bd640 10 monclient(hunting): authenticate will time out at 1726.823389s 2024-08-19T10:51:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d04d58640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4d000a7350 0x7f4d0013e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d04d58640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4d000a7350 0x7f4d0013e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48564/0 (socket says 172.21.15.89:48564) 2024-08-19T10:51:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d04d58640 1 -- 172.21.15.89:0/2381199775 learned_addr learned my addr 172.21.15.89:0/2381199775 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d04d58640 10 monclient(hunting): get_auth_request con 0x7f4d000a7350 auth_method 0 2024-08-19T10:51:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d04d58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d04d58640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d04d58640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d04d58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d04d58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d05d5a640 1 --2- 172.21.15.89:0/2381199775 >> v2:172.21.15.190:3300/0 conn(0x7f4d000a7730 0x7f4d00140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d04d58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d05d5a640 10 monclient(hunting): get_auth_request con 0x7f4d000a7730 auth_method 0 2024-08-19T10:51:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d05d5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d05d5a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.000+0000 7f4d05d5a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.004+0000 7f4d04d58640 10 monclient(hunting): handle_auth_done global_id 7082 payload 274 2024-08-19T10:51:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.004+0000 7f4d04d58640 1 -- 172.21.15.89:0/2381199775 >> v2:172.21.15.190:3300/0 conn(0x7f4d000a7730 msgr2=0x7f4d00140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.004+0000 7f4d04d58640 1 --2- 172.21.15.89:0/2381199775 >> v2:172.21.15.190:3300/0 conn(0x7f4d000a7730 0x7f4d00140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.004+0000 7f4d04d58640 1 -- 172.21.15.89:0/2381199775 >> v1:172.21.15.190:6789/0 conn(0x7f4d000a6860 legacy=0x7f4d000a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.004+0000 7f4d04d58640 10 monclient: _finish_hunting 0 2024-08-19T10:51:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.004+0000 7f4d04d58640 1 monclient: found mon.noname-e 2024-08-19T10:51:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.004+0000 7f4d04d58640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.004+0000 7f4d04d58640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:51:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.004+0000 7f4d04d58640 1 -- 172.21.15.89:0/2381199775 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d000a4cd0 con 0x7f4d000a7350 2024-08-19T10:51:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.008+0000 7f4d04d58640 1 -- 172.21.15.89:0/2381199775 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4d000a7350 msgr2=0x7f4d0013e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.008+0000 7f4d04d58640 1 --2- 172.21.15.89:0/2381199775 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4d000a7350 0x7f4d0013e860 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009870 tx=0x7f4cf001ce80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=cbcc1f30dcfa78b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 1 -- 172.21.15.89:0/2381199775 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4cf00b0070 con 0x7f4d000a7350 2024-08-19T10:51:17.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:51:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 10 monclient: dump: 2024-08-19T10:51:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.015 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.016 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 10 monclient: _finish_auth 0 2024-08-19T10:51:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:51:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:17.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 1 -- 172.21.15.89:0/2381199775 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf00aa810 con 0x7f4d000a7350 2024-08-19T10:51:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 1 -- 172.21.15.89:0/2381199775 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4cf00aae30 con 0x7f4d000a7350 2024-08-19T10:51:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4d077bd640 5 monclient: authenticate success, global_id 7082 2024-08-19T10:51:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4d077bd640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4d077bd640 4 set_mon_vals no callback set 2024-08-19T10:51:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:51:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.012+0000 7f4cf7fff640 10 monclient: dump: 2024-08-19T10:51:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.017 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.018 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.019 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.019 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient: shutdown 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 -- 172.21.15.89:0/2381199775 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4d000a7350 msgr2=0x7f4d0013e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 --2- 172.21.15.89:0/2381199775 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4d000a7350 0x7f4d0013e860 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009870 tx=0x7f4cf001ce80 comp rx=0 tx=0).stop 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 -- 172.21.15.89:0/2381199775 shutdown_connections 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 --2- 172.21.15.89:0/2381199775 >> v2:172.21.15.190:3300/0 conn(0x7f4d000a7730 0x7f4d00140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 --2- 172.21.15.89:0/2381199775 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4d000a7350 0x7f4d0013e860 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 -- 172.21.15.89:0/2381199775 >> 172.21.15.89:0/2381199775 conn(0x7f4d00074b40 msgr2=0x7f4d000abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 -- 172.21.15.89:0/2381199775 shutdown_connections 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 -- 172.21.15.89:0/2381199775 wait complete. 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient: build_initial_monmap 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient: monmap: 2024-08-19T10:51:17.025 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:17.026 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 Processor -- start 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 -- start start 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient: init 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient: _add_conns ranks=[3,5,0,2,6,8,4,7,1] 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d000a6860 0x7f4d001d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient(hunting): picked mon.noname-g con 0x7f4d000a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4d000a7350 0x7f4d001d1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient(hunting): picked mon.noname-h con 0x7f4d000a7350 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:17.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d000a7730 0x7f4d001d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient(hunting): picked mon.noname-a con 0x7f4d000a7730 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00143ea0 con 0x7f4d000a7730 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4d00143d20 con 0x7f4d000a6860 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f4d00144020 con 0x7f4d000a7350 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d077bd640 10 monclient(hunting): authenticate will time out at 1726.839419s 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d000a7730 0x7f4d001d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d000a7730 0x7f4d001d19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36530/0 (socket says 172.21.15.89:36530) 2024-08-19T10:51:17.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 1 -- 172.21.15.89:0/4054356976 learned_addr learned my addr 172.21.15.89:0/4054356976 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 10 monclient(hunting): get_auth_request con 0x7f4d000a7730 auth_method 0 2024-08-19T10:51:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05559640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d000a6860 0x7f4d001d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d04d58640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4d000a7350 0x7f4d001d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05559640 10 monclient(hunting): get_auth_request con 0x7f4d000a6860 auth_method 0 2024-08-19T10:51:17.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05559640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05559640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05559640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d04d58640 10 monclient(hunting): get_auth_request con 0x7f4d000a7350 auth_method 0 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d04d58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d04d58640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d04d58640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 10 monclient(hunting): handle_auth_done global_id 7146 payload 824 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 1 -- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4d000a7350 msgr2=0x7f4d001d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4d000a7350 0x7f4d001d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 1 -- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d000a6860 msgr2=0x7f4d001d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d000a6860 0x7f4d001d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 10 monclient: _finish_hunting 0 2024-08-19T10:51:17.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 1 monclient: found mon.noname-a 2024-08-19T10:51:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:51:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.016+0000 7f4d05d5a640 1 -- 172.21.15.89:0/4054356976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d00129be0 con 0x7f4d000a7730 2024-08-19T10:51:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d05d5a640 1 -- 172.21.15.89:0/4054356976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d000a7730 msgr2=0x7f4d001d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d05d5a640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d000a7730 0x7f4d001d19b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800d560 tx=0x7f4cf800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 1 -- 172.21.15.89:0/4054356976 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4cf8012c70 con 0x7f4d000a7730 2024-08-19T10:51:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:51:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 10 monclient: dump: 2024-08-19T10:51:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.032 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.033 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.034 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:51:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:51:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d077bd640 5 monclient: authenticate success, global_id 7146 2024-08-19T10:51:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 1 -- 172.21.15.89:0/4054356976 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf8019070 con 0x7f4d000a7730 2024-08-19T10:51:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d077bd640 10 monclient: _renew_subs 2024-08-19T10:51:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d077bd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d077bd640 1 -- 172.21.15.89:0/4054356976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d00129ec0 con 0x7f4d000a7730 2024-08-19T10:51:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:17.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 1 -- 172.21.15.89:0/4054356976 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4cf80148a0 con 0x7f4d000a7730 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf77fe640 4 set_mon_vals no callback set 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 10 monclient: dump: 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.035 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d077bd640 10 monclient: _renew_subs 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d077bd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d077bd640 1 -- 172.21.15.89:0/4054356976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d00127b60 con 0x7f4d000a7730 2024-08-19T10:51:17.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d077bd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d077bd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d077bd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4d077bd640 1 -- 172.21.15.89:0/4054356976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d000a6c40 con 0x7f4d000a7730 2024-08-19T10:51:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 1 -- 172.21.15.89:0/4054356976 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4cf8014430 con 0x7f4d000a7730 2024-08-19T10:51:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.020+0000 7f4cf5ffb640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ce403be00 0x7f4ce403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.024+0000 7f4cf5ffb640 1 -- 172.21.15.89:0/4054356976 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f4cf800f040 con 0x7f4d000a7730 2024-08-19T10:51:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.024+0000 7f4d05559640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ce403be00 0x7f4ce403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.024+0000 7f4d05559640 10 monclient: get_auth_request con 0x7f4ce403be00 auth_method 0 2024-08-19T10:51:17.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.024+0000 7f4d05559640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ce403be00 0x7f4ce403e2c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005530 tx=0x7f4cfc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1497128539}) 2024-08-19T10:51:17.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=254376326}) 2024-08-19T10:51:17.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1497128539}) 2024-08-19T10:51:17.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2266924509}) 2024-08-19T10:51:17.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1497128539}) 2024-08-19T10:51:17.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3013740276}) 2024-08-19T10:51:17.040 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.040 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1497128539}) 2024-08-19T10:51:17.040 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4185377099}) 2024-08-19T10:51:17.040 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.040 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1497128539}) 2024-08-19T10:51:17.040 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3630544364}) 2024-08-19T10:51:17.040 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.040 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1497128539}) 2024-08-19T10:51:17.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f4cf5ffb640 1 -- 172.21.15.89:0/4054356976 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4cf8014a40 con 0x7f4d000a7730 2024-08-19T10:51:17.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f4cf5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:17.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f4cf5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:17.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3748085795}) 2024-08-19T10:51:17.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1497128539}) 2024-08-19T10:51:17.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=450241617}) 2024-08-19T10:51:17.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1497128539}) 2024-08-19T10:51:17.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.028+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2845759933}) 2024-08-19T10:51:17.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=163808958}) 2024-08-19T10:51:17.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1211068916}) 2024-08-19T10:51:17.052 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=163808958}) 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3432619930}) 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=163808958}) 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3477567896}) 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=163808958}) 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1650938688}) 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=163808958}) 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2037668423}) 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.053 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=163808958}) 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2634567799}) 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=163808958}) 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=194061638}) 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=163808958}) 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.036+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2285373100}) 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325673775}) 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=960726295}) 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325673775}) 2024-08-19T10:51:17.054 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=716334089}) 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325673775}) 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=92476243}) 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325673775}) 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3214333659}) 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325673775}) 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=276982161}) 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325673775}) 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1393776200}) 2024-08-19T10:51:17.055 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325673775}) 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1178265386}) 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325673775}) 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.040+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1192535948}) 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918640786}) 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1403635161}) 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918640786}) 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2314008610}) 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.056 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918640786}) 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=559171852}) 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918640786}) 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1620892201}) 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918640786}) 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1728844045}) 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918640786}) 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1693230029}) 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918640786}) 2024-08-19T10:51:17.057 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2716492546}) 2024-08-19T10:51:17.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3918640786}) 2024-08-19T10:51:17.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.044+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4081482542}) 2024-08-19T10:51:17.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523470158}) 2024-08-19T10:51:17.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=194630356}) 2024-08-19T10:51:17.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523470158}) 2024-08-19T10:51:17.058 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3728028827}) 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523470158}) 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1967330596}) 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523470158}) 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=266985123}) 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523470158}) 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3705761307}) 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523470158}) 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1472134936}) 2024-08-19T10:51:17.059 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523470158}) 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1800331157}) 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1523470158}) 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=390714598}) 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649941732}) 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1829576105}) 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649941732}) 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3981424630}) 2024-08-19T10:51:17.060 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649941732}) 2024-08-19T10:51:17.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1628023625}) 2024-08-19T10:51:17.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649941732}) 2024-08-19T10:51:17.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4044610876}) 2024-08-19T10:51:17.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649941732}) 2024-08-19T10:51:17.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=426930423}) 2024-08-19T10:51:17.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649941732}) 2024-08-19T10:51:17.061 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1576098064}) 2024-08-19T10:51:17.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649941732}) 2024-08-19T10:51:17.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=83396171}) 2024-08-19T10:51:17.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649941732}) 2024-08-19T10:51:17.062 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.048+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1197184792}) 2024-08-19T10:51:17.080 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.080 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=493670167}) 2024-08-19T10:51:17.080 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3286141769}) 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=493670167}) 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=277649070}) 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=493670167}) 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=805396223}) 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=493670167}) 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2423987491}) 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=493670167}) 2024-08-19T10:51:17.081 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2246693745}) 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=493670167}) 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=563653962}) 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=493670167}) 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=280730941}) 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=493670167}) 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.052+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=552908741}) 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228617400}) 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2993244258}) 2024-08-19T10:51:17.082 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228617400}) 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3481677984}) 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228617400}) 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3035525811}) 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228617400}) 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1385357486}) 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228617400}) 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1373616495}) 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.083 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228617400}) 2024-08-19T10:51:17.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=429754531}) 2024-08-19T10:51:17.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228617400}) 2024-08-19T10:51:17.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1879952352}) 2024-08-19T10:51:17.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2228617400}) 2024-08-19T10:51:17.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=678391620}) 2024-08-19T10:51:17.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696599014}) 2024-08-19T10:51:17.084 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=392771059}) 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696599014}) 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1761568032}) 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696599014}) 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3057598826}) 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696599014}) 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2681841869}) 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696599014}) 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2668922889}) 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.085 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696599014}) 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3156857049}) 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696599014}) 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.056+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4055492915}) 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696599014}) 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3414067206}) 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=286346089}) 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=953878776}) 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.086 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=286346089}) 2024-08-19T10:51:17.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4150042725}) 2024-08-19T10:51:17.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=286346089}) 2024-08-19T10:51:17.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2267341796}) 2024-08-19T10:51:17.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=286346089}) 2024-08-19T10:51:17.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2300874639}) 2024-08-19T10:51:17.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=286346089}) 2024-08-19T10:51:17.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3190223209}) 2024-08-19T10:51:17.087 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=286346089}) 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=486408846}) 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=286346089}) 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1632196901}) 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=286346089}) 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.060+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2256165652}) 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1380000029}) 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2097355960}) 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.088 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1380000029}) 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2660933475}) 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1380000029}) 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1220372970}) 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1380000029}) 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3686001908}) 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1380000029}) 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=986087907}) 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1380000029}) 2024-08-19T10:51:17.089 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=648791600}) 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1380000029}) 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1279488685}) 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1380000029}) 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.064+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2724989262}) 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3361670130}) 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=889380552}) 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3361670130}) 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3795374067}) 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.090 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3361670130}) 2024-08-19T10:51:17.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1857911629}) 2024-08-19T10:51:17.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3361670130}) 2024-08-19T10:51:17.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3610218329}) 2024-08-19T10:51:17.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3361670130}) 2024-08-19T10:51:17.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3604084855}) 2024-08-19T10:51:17.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3361670130}) 2024-08-19T10:51:17.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3267396364}) 2024-08-19T10:51:17.091 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3361670130}) 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=696510003}) 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3361670130}) 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3225203655}) 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.068+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695876037}) 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3048369849}) 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695876037}) 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=539628926}) 2024-08-19T10:51:17.092 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695876037}) 2024-08-19T10:51:17.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=170194716}) 2024-08-19T10:51:17.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695876037}) 2024-08-19T10:51:17.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=79251331}) 2024-08-19T10:51:17.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695876037}) 2024-08-19T10:51:17.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3817293588}) 2024-08-19T10:51:17.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695876037}) 2024-08-19T10:51:17.093 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2870788732}) 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695876037}) 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1033273619}) 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2695876037}) 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4010968803}) 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164179352}) 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2688428951}) 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164179352}) 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3893178054}) 2024-08-19T10:51:17.094 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164179352}) 2024-08-19T10:51:17.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3268571558}) 2024-08-19T10:51:17.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164179352}) 2024-08-19T10:51:17.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=802418816}) 2024-08-19T10:51:17.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164179352}) 2024-08-19T10:51:17.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2806855130}) 2024-08-19T10:51:17.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.095 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164179352}) 2024-08-19T10:51:17.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2723934799}) 2024-08-19T10:51:17.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164179352}) 2024-08-19T10:51:17.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2809192645}) 2024-08-19T10:51:17.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2164179352}) 2024-08-19T10:51:17.096 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.072+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3980665355}) 2024-08-19T10:51:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4d077bd640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:51:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4d077bd640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:51:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4d077bd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4d077bd640 1 -- 172.21.15.89:0/4054356976 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4d00064590 con 0x7f4d000a7730 2024-08-19T10:51:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4cf5ffb640 1 -- 172.21.15.89:0/4054356976 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7f4cf8013cb0 con 0x7f4d000a7730 2024-08-19T10:51:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4cf5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:51:17.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4cf5ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:17.281 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":110,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:51:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4d077bd640 1 -- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ce403be00 msgr2=0x7f4ce403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4d077bd640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ce403be00 0x7f4ce403e2c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005530 tx=0x7f4cfc009040 comp rx=0 tx=0).stop 2024-08-19T10:51:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4d077bd640 10 monclient: shutdown 2024-08-19T10:51:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4d077bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.280+0000 7f4d077bd640 1 -- 172.21.15.89:0/4054356976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d000a7730 msgr2=0x7f4d001d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.284+0000 7f4d077bd640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d000a7730 0x7f4d001d19b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800d560 tx=0x7f4cf800da20 comp rx=0 tx=0).stop 2024-08-19T10:51:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.284+0000 7f4d077bd640 1 -- 172.21.15.89:0/4054356976 shutdown_connections 2024-08-19T10:51:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.284+0000 7f4d077bd640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4ce403be00 0x7f4ce403e2c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.284+0000 7f4d077bd640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d000a7730 0x7f4d001d19b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.284+0000 7f4d077bd640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f4d000a7350 0x7f4d001d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.284+0000 7f4d077bd640 1 --2- 172.21.15.89:0/4054356976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d000a6860 0x7f4d001d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.284+0000 7f4d077bd640 1 -- 172.21.15.89:0/4054356976 >> 172.21.15.89:0/4054356976 conn(0x7f4d00074b40 msgr2=0x7f4d0009c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.284+0000 7f4d077bd640 1 -- 172.21.15.89:0/4054356976 shutdown_connections 2024-08-19T10:51:17.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.284+0000 7f4d077bd640 1 -- 172.21.15.89:0/4054356976 wait complete. 2024-08-19T10:51:17.301 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:51:17.301 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-08-19T10:51:17.301 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:51:17.302 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:51:17.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.424+0000 7f817c78f640 10 monclient: get_monmap_and_config 2024-08-19T10:51:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.424+0000 7f817c78f640 10 monclient: build_initial_monmap 2024-08-19T10:51:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient: monmap: 2024-08-19T10:51:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:17.432951+0000 2024-08-19T10:51:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:17.432951+0000 2024-08-19T10:51:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:17.427 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:17.428 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:17.428 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:17.428 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:17.428 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:17.428 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:17.428 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:17.428 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:17.428 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:17.428 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 1 Processor -- start 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 1 -- start start 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient: init 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient: _add_conns ranks=[3,5,10,4,7,6,2,0,9,1,8] 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81740a5270 0x7f81740a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient(hunting): picked mon.noname-g con 0x7f81740a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81740a6d30 0x7f81740a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient(hunting): picked mon.noname-h con 0x7f81740a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81741395b0 0x7f8174139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient(hunting): picked mon.noname-f con 0x7f81741395b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f81740a7650 con 0x7f81741395b0 2024-08-19T10:51:17.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f81740a8710 con 0x7f81740a5270 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f81740a5a40 con 0x7f81740a6d30 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817c78f640 10 monclient(hunting): authenticate will time out at 1727.249579s 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817a52b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81740a5270 0x7f81740a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817ad2c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81741395b0 0x7f8174139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81740a6d30 0x7f81740a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817ad2c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81741395b0 0x7f8174139a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:48352/0 (socket says 172.21.15.89:48352) 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81740a6d30 0x7f81740a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:53278/0 (socket says 172.21.15.89:53278) 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817ad2c640 1 -- 172.21.15.89:0/3217583234 learned_addr learned my addr 172.21.15.89:0/3217583234 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817a52b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81740a5270 0x7f81740a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:60682/0 (socket says 172.21.15.89:60682) 2024-08-19T10:51:17.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817a52b640 10 monclient(hunting): get_auth_request con 0x7f81740a5270 auth_method 0 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817a52b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817a52b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817a52b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817ad2c640 10 monclient(hunting): get_auth_request con 0x7f81741395b0 auth_method 0 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817ad2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817ad2c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817ad2c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 10 monclient(hunting): get_auth_request con 0x7f81740a6d30 auth_method 0 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817ad2c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817ad2c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817ad2c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817a52b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817a52b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817a52b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 10 monclient(hunting): handle_auth_done global_id 7151 payload 274 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 1 -- 172.21.15.89:0/3217583234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81740a5270 msgr2=0x7f81740a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 1 --2- 172.21.15.89:0/3217583234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81740a5270 0x7f81740a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 1 -- 172.21.15.89:0/3217583234 >> v2:172.21.15.190:3300/0 conn(0x7f81741395b0 msgr2=0x7f8174139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 1 --2- 172.21.15.89:0/3217583234 >> v2:172.21.15.190:3300/0 conn(0x7f81741395b0 0x7f8174139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 10 monclient: _finish_hunting 0 2024-08-19T10:51:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 1 monclient: found mon.noname-h 2024-08-19T10:51:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:51:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f8179d2a640 1 -- 172.21.15.89:0/3217583234 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81740a81d0 con 0x7f81740a6d30 2024-08-19T10:51:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817ad2c640 1 --2- 172.21.15.89:0/3217583234 >> v2:172.21.15.190:3300/0 conn(0x7f81741395b0 0x7f8174139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:17.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.428+0000 7f817a52b640 1 --2- 172.21.15.89:0/3217583234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81740a5270 0x7f81740a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.432+0000 7f8179d2a640 1 -- 172.21.15.89:0/3217583234 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81740a6d30 msgr2=0x7f81740a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.432+0000 7f8179d2a640 1 --2- 172.21.15.89:0/3217583234 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81740a6d30 0x7f81740a7110 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f81640097c0 tx=0x7f816401c360 comp rx=0 tx=0).ready entity=mon.5 client_cookie=2d8c5b88fd6a429f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 1 -- 172.21.15.89:0/3217583234 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81640b0070 con 0x7f81740a6d30 2024-08-19T10:51:17.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 10 monclient: dump: 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.439 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 10 monclient: _finish_auth 0 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 10 monclient: _check_auth_tickets 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 1 -- 172.21.15.89:0/3217583234 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81640aa810 con 0x7f81740a6d30 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 1 -- 172.21.15.89:0/3217583234 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81640a9300 con 0x7f81740a6d30 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 5 monclient: authenticate success, global_id 7151 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:17.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 4 set_mon_vals no callback set 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179529640 10 monclient: dump: 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.441 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient: shutdown 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 -- 172.21.15.89:0/3217583234 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81740a6d30 msgr2=0x7f81740a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 --2- 172.21.15.89:0/3217583234 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81740a6d30 0x7f81740a7110 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f81640097c0 tx=0x7f816401c360 comp rx=0 tx=0).stop 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 -- 172.21.15.89:0/3217583234 shutdown_connections 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 --2- 172.21.15.89:0/3217583234 >> v2:172.21.15.190:3300/0 conn(0x7f81741395b0 0x7f8174139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 --2- 172.21.15.89:0/3217583234 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f81740a6d30 0x7f81740a7110 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 --2- 172.21.15.89:0/3217583234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81740a5270 0x7f81740a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 -- 172.21.15.89:0/3217583234 >> 172.21.15.89:0/3217583234 conn(0x7f8174069a80 msgr2=0x7f81740abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 -- 172.21.15.89:0/3217583234 shutdown_connections 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 -- 172.21.15.89:0/3217583234 wait complete. 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient: build_initial_monmap 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient: monmap: 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:17.443 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 Processor -- start 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 -- start start 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient: init 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient: _add_conns ranks=[6,4,3,5,8,1,7,2,0] 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81740a5270 0x7f81740a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient(hunting): picked mon.noname-d con 0x7f81740a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81740a6d30 0x7f81740a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient(hunting): picked mon.noname-c con 0x7f81740a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81741395b0 0x7f817413e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient(hunting): picked mon.noname-g con 0x7f81741395b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f81740a5650 con 0x7f81741395b0 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f817413a350 con 0x7f81740a6d30 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f81740a57d0 con 0x7f81740a5270 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817c78f640 10 monclient(hunting): authenticate will time out at 1727.258640s 2024-08-19T10:51:17.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81740a6d30 0x7f81740a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81740a6d30 0x7f81740a2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38112/0 (socket says 172.21.15.89:38112) 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 1 -- 172.21.15.89:0/4137491750 learned_addr learned my addr 172.21.15.89:0/4137491750 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 10 monclient(hunting): get_auth_request con 0x7f81740a6d30 auth_method 0 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817a52b640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81740a5270 0x7f81740a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817a52b640 10 monclient(hunting): get_auth_request con 0x7f81740a5270 auth_method 0 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817a52b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817a52b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817a52b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817ad2c640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81741395b0 0x7f817413e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817ad2c640 10 monclient(hunting): get_auth_request con 0x7f81741395b0 auth_method 0 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817ad2c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817ad2c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817ad2c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 10 monclient(hunting): handle_auth_done global_id 45004 payload 824 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 1 -- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81740a5270 msgr2=0x7f81740a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81740a5270 0x7f81740a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 1 -- 172.21.15.89:0/4137491750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81741395b0 msgr2=0x7f817413e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81741395b0 0x7f817413e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 10 monclient: _finish_hunting 0 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 1 monclient: found mon.noname-c 2024-08-19T10:51:17.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:51:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f8179d2a640 1 -- 172.21.15.89:0/4137491750 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817409e4e0 con 0x7f81740a6d30 2024-08-19T10:51:17.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.436+0000 7f817a52b640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81740a5270 0x7f81740a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:51:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f8179d2a640 1 -- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81740a6d30 msgr2=0x7f81740a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f8179d2a640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81740a6d30 0x7f81740a2510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f81640ae040 tx=0x7f8164019100 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81640b0040 con 0x7f81740a6d30 2024-08-19T10:51:17.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 10 monclient: dump: 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.452 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 10 monclient: _finish_auth 0 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 10 monclient: _check_auth_tickets 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817c78f640 5 monclient: authenticate success, global_id 45004 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81640aae30 con 0x7f81740a6d30 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817c78f640 10 monclient: _renew_subs 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817c78f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817c78f640 1 -- 172.21.15.89:0/4137491750 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817409d2a0 con 0x7f81740a6d30 2024-08-19T10:51:17.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8164004250 con 0x7f81740a6d30 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f8178d28640 4 set_mon_vals no callback set 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 10 monclient: dump: 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.454 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817c78f640 10 monclient: _renew_subs 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817c78f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817c78f640 1 -- 172.21.15.89:0/4137491750 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f817409e7c0 con 0x7f81740a6d30 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817c78f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:17.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817c78f640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81741d14b0 0x7f817413e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817c78f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817c78f640 1 -- 172.21.15.89:0/4137491750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8174135800 con 0x7f81741d14b0 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817a52b640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81741d14b0 0x7f817413e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8164003620 con 0x7f81740a6d30 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817a52b640 10 monclient: get_auth_request con 0x7f81741d14b0 auth_method 0 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817a52b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817a52b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817a52b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817a52b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817a52b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817a52b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f815803be00 0x7f815803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817a52b640 10 monclient(hunting): handle_auth_done global_id 7155 payload 824 2024-08-19T10:51:17.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817a52b640 1 -- 172.21.15.89:0/4137491750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81740a7110 con 0x7f81741d14b0 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f81640e9210 con 0x7f81740a6d30 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817ad2c640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f815803be00 0x7f815803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.440+0000 7f817ad2c640 10 monclient: get_auth_request con 0x7f815803be00 auth_method 0 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f817a52b640 1 -- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81741d14b0 msgr2=0x7f817413e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f817a52b640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81741d14b0 0x7f817413e690 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f816c0115d0 tx=0x7f816c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f816c015460 con 0x7f81741d14b0 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f816affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f816affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f816affd640 10 monclient: dump: 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.457 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.458 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.458 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f816c00fa50 con 0x7f81741d14b0 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f816affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f816affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81741d14b0 msgr2=0x7f817413e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f816affd640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81741d14b0 0x7f817413e690 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f816c0115d0 tx=0x7f816c011a90 comp rx=0 tx=0).stop 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81741d14b0 msgr2=0x7f817413e690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f816affd640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81741d14b0 0x7f817413e690 secure :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f816c0115d0 tx=0x7f816c011a90 comp rx=0 tx=0).stop 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.444+0000 7f817ad2c640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f815803be00 0x7f815803e2c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f817413f910 tx=0x7f817000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.448+0000 7f817c78f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.448+0000 7f817c78f640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817413ebd0 0x7f8174137a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.448+0000 7f817c78f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.448+0000 7f817c78f640 1 -- 172.21.15.89:0/4137491750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8174064590 con 0x7f817413ebd0 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817413ebd0 0x7f8174137a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 10 monclient: get_auth_request con 0x7f817413ebd0 auth_method 0 2024-08-19T10:51:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 10 monclient(hunting): handle_auth_done global_id 7164 payload 824 2024-08-19T10:51:17.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 1 -- 172.21.15.89:0/4137491750 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f817413efe0 con 0x7f817413ebd0 2024-08-19T10:51:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 1 -- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817413ebd0 msgr2=0x7f8174137a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817a52b640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817413ebd0 0x7f8174137a20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f816c0115d0 tx=0x7f816c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f816c026070 con 0x7f817413ebd0 2024-08-19T10:51:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f816affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f816affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f816affd640 10 monclient: dump: 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.462 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.463 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9604 (secure 0 0 0) 0x7f816c0222d0 con 0x7f817413ebd0 2024-08-19T10:51:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f816affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f816affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817413ebd0 msgr2=0x7f8174137a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f816affd640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817413ebd0 0x7f8174137a20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f816c0115d0 tx=0x7f816c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f816affd640 1 -- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817413ebd0 msgr2=0x7f8174137a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:17.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f816affd640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f817413ebd0 0x7f8174137a20 secure :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f816c0115d0 tx=0x7f816c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 807266, 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:17.464 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:17.465 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:17.466 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:17.467 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:17.468 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:17.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:17.470 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:17.472 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.475 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:17.476 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.478 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.479 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:17.480 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.481 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.482 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:17.483 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.484 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:17.485 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.486 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:17.487 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:17.487 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:17.487 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.487 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.487 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:17.487 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:17.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817c78f640 1 -- 172.21.15.89:0/4137491750 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f815803be00 msgr2=0x7f815803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817c78f640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f815803be00 0x7f815803e2c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f817413f910 tx=0x7f817000b040 comp rx=0 tx=0).stop 2024-08-19T10:51:17.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817c78f640 10 monclient: shutdown 2024-08-19T10:51:17.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817c78f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:17.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817c78f640 1 -- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81740a6d30 msgr2=0x7f81740a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817c78f640 1 --2- 172.21.15.89:0/4137491750 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f81740a6d30 0x7f81740a2510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f81640ae040 tx=0x7f8164019100 comp rx=0 tx=0).stop 2024-08-19T10:51:17.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817ad2c640 1 -- 172.21.15.89:0/4137491750 reap_dead start 2024-08-19T10:51:17.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817ad2c640 1 -- 172.21.15.89:0/4137491750 reap_dead start 2024-08-19T10:51:17.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817c78f640 1 -- 172.21.15.89:0/4137491750 shutdown_connections 2024-08-19T10:51:17.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817c78f640 1 -- 172.21.15.89:0/4137491750 >> 172.21.15.89:0/4137491750 conn(0x7f8174069a80 msgr2=0x7f8174133de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:17.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817c78f640 1 -- 172.21.15.89:0/4137491750 shutdown_connections 2024-08-19T10:51:17.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.452+0000 7f817c78f640 1 -- 172.21.15.89:0/4137491750 wait complete. 2024-08-19T10:51:17.490 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:17.490 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:51:17.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:51:17.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1613892522}) 2024-08-19T10:51:17.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=165368092}) 2024-08-19T10:51:17.584 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1613892522}) 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1136774057}) 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1613892522}) 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2475550988}) 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1613892522}) 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3601648026}) 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1613892522}) 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1594538939}) 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.585 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1613892522}) 2024-08-19T10:51:17.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2432014390}) 2024-08-19T10:51:17.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1613892522}) 2024-08-19T10:51:17.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3139333702}) 2024-08-19T10:51:17.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1613892522}) 2024-08-19T10:51:17.586 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=534763955}) 2024-08-19T10:51:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient: get_monmap_and_config 2024-08-19T10:51:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient: build_initial_monmap 2024-08-19T10:51:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient: monmap: 2024-08-19T10:51:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:17.622100+0000 2024-08-19T10:51:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:17.622100+0000 2024-08-19T10:51:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:17.616 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:17.617 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:17.617 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 1 Processor -- start 2024-08-19T10:51:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 1 -- start start 2024-08-19T10:51:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient: init 2024-08-19T10:51:17.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient: _add_conns ranks=[5,3,2,0,6,4,7,8,9,1,10] 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f461406c930 0x7f4614139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient(hunting): picked mon.noname-h con 0x7f461406c930 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4614139a90 0x7f4614139e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient(hunting): picked mon.noname-g con 0x7f4614139a90 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461406cd10 0x7f461406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient(hunting): picked mon.noname-b con 0x7f461406cd10 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f461413a3b0 con 0x7f461406cd10 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f46140a0c10 con 0x7f4614139a90 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f46140a9170 con 0x7f461406c930 2024-08-19T10:51:17.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4618b51640 10 monclient(hunting): authenticate will time out at 1727.438637s 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461406cd10 0x7f461406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461406cd10 0x7f461406d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35428/0 (socket says 172.21.15.89:35428) 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 1 -- 172.21.15.89:0/296207612 learned_addr learned my addr 172.21.15.89:0/296207612 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 10 monclient(hunting): get_auth_request con 0x7f461406cd10 auth_method 0 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4611d9b640 1 --2- 172.21.15.89:0/296207612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4614139a90 0x7f4614139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f461259c640 1 --2- 172.21.15.89:0/296207612 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f461406c930 0x7f4614139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4611d9b640 10 monclient(hunting): get_auth_request con 0x7f4614139a90 auth_method 0 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4611d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4611d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4611d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f461259c640 10 monclient(hunting): get_auth_request con 0x7f461406c930 auth_method 0 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f461259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f461259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f461259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 10 monclient(hunting): handle_auth_done global_id 74171 payload 274 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 1 -- 172.21.15.89:0/296207612 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f461406c930 msgr2=0x7f4614139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 1 --2- 172.21.15.89:0/296207612 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f461406c930 0x7f4614139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 1 -- 172.21.15.89:0/296207612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4614139a90 msgr2=0x7f4614139e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 1 --2- 172.21.15.89:0/296207612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4614139a90 0x7f4614139e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:51:17.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 1 monclient: found mon.noname-b 2024-08-19T10:51:17.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:17.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:51:17.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.616+0000 7f4612d9d640 1 -- 172.21.15.89:0/296207612 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4614069970 con 0x7f461406cd10 2024-08-19T10:51:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f4612d9d640 1 -- 172.21.15.89:0/296207612 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461406cd10 msgr2=0x7f461406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f4612d9d640 1 --2- 172.21.15.89:0/296207612 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461406cd10 0x7f461406d150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f460c009e80 tx=0x7f460c01d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2781c142d167908b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 1 -- 172.21.15.89:0/296207612 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f460c0090d0 con 0x7f461406cd10 2024-08-19T10:51:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:51:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 10 monclient: dump: 2024-08-19T10:51:17.626 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.627 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 10 monclient: _finish_auth 0 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 10 monclient: _check_auth_tickets 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f4618b51640 5 monclient: authenticate success, global_id 74171 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f4618b51640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 1 -- 172.21.15.89:0/296207612 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460c0a5b10 con 0x7f461406cd10 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f4618b51640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f4618b51640 4 set_mon_vals no callback set 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 1 -- 172.21.15.89:0/296207612 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f460c0a5cc0 con 0x7f461406cd10 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:17.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.620+0000 7f461159a640 10 monclient: dump: 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.629 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient: shutdown 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 -- 172.21.15.89:0/296207612 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461406cd10 msgr2=0x7f461406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 --2- 172.21.15.89:0/296207612 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461406cd10 0x7f461406d150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f460c009e80 tx=0x7f460c01d3b0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 -- 172.21.15.89:0/296207612 shutdown_connections 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 --2- 172.21.15.89:0/296207612 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461406cd10 0x7f461406d150 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 --2- 172.21.15.89:0/296207612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4614139a90 0x7f4614139e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 --2- 172.21.15.89:0/296207612 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f461406c930 0x7f4614139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 -- 172.21.15.89:0/296207612 >> 172.21.15.89:0/296207612 conn(0x7f46140a83d0 msgr2=0x7f46140a87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 -- 172.21.15.89:0/296207612 shutdown_connections 2024-08-19T10:51:17.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 -- 172.21.15.89:0/296207612 wait complete. 2024-08-19T10:51:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient: build_initial_monmap 2024-08-19T10:51:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient: monmap: 2024-08-19T10:51:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:17.631 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:17.632 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:17.632 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:17.632 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:17.632 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:17.632 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:17.632 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:17.632 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:17.632 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 Processor -- start 2024-08-19T10:51:17.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 -- start start 2024-08-19T10:51:17.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient: init 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient: _add_conns ranks=[0,4,8,1,3,2,7,6,5] 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461406c930 0x7f46141388d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient(hunting): picked mon.noname-a con 0x7f461406c930 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f461406cd10 0x7f4614138e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient(hunting): picked mon.noname-c con 0x7f461406cd10 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4614139a90 0x7f461413be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient(hunting): picked mon.noname-e con 0x7f4614139a90 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f461409fa50 con 0x7f461406c930 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f461409f8d0 con 0x7f461406cd10 2024-08-19T10:51:17.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f46141431d0 con 0x7f4614139a90 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4618b51640 10 monclient(hunting): authenticate will time out at 1727.445873s 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4612d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4614139a90 0x7f461413be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461406c930 0x7f46141388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461406c930 0x7f46141388d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36556/0 (socket says 172.21.15.89:36556) 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4611d9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f461406cd10 0x7f4614138e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 1 -- 172.21.15.89:0/3319640717 learned_addr learned my addr 172.21.15.89:0/3319640717 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 10 monclient(hunting): get_auth_request con 0x7f461406c930 auth_method 0 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4612d9d640 10 monclient(hunting): get_auth_request con 0x7f4614139a90 auth_method 0 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4612d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4612d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4612d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4611d9b640 10 monclient(hunting): get_auth_request con 0x7f461406cd10 auth_method 0 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4611d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4611d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4611d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4612d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4612d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4612d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4611d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4611d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f4611d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 10 monclient(hunting): handle_auth_done global_id 7173 payload 824 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 1 -- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4614139a90 msgr2=0x7f461413be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4614139a90 0x7f461413be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 1 -- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f461406cd10 msgr2=0x7f4614138e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f461406cd10 0x7f4614138e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 10 monclient: _finish_hunting 0 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 1 monclient: found mon.noname-a 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:51:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.624+0000 7f461259c640 1 -- 172.21.15.89:0/3319640717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461413f7c0 con 0x7f461406c930 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f461259c640 1 -- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461406c930 msgr2=0x7f46141388d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f461259c640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461406c930 0x7f46141388d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f460400d5f0 tx=0x7f460400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4604012c70 con 0x7f461406c930 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 10 monclient: dump: 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.641 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 10 monclient: _finish_auth 0 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:51:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4604019070 con 0x7f461406c930 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4618b51640 5 monclient: authenticate success, global_id 7173 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4618b51640 10 monclient: _renew_subs 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4618b51640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4618b51640 1 -- 172.21.15.89:0/3319640717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461413f200 con 0x7f461406c930 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4604012e30 con 0x7f461406c930 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 10 monclient: dump: 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.643 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4618b51640 10 monclient: _renew_subs 2024-08-19T10:51:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4610d99640 4 set_mon_vals no callback set 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4618b51640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4618b51640 1 -- 172.21.15.89:0/3319640717 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4614139350 con 0x7f461406c930 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4618b51640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4618b51640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461413e2a0 0x7f461413cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4618b51640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4618b51640 1 -- 172.21.15.89:0/3319640717 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f46140a1880 con 0x7f461413e2a0 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4611d9b640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461413e2a0 0x7f461413cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4611d9b640 10 monclient: get_auth_request con 0x7f461413e2a0 auth_method 0 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4611d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4611d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4611d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4611d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4611d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4611d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4604012800 con 0x7f461406c930 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4611d9b640 10 monclient(hunting): handle_auth_done global_id 74180 payload 824 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4611d9b640 1 -- 172.21.15.89:0/3319640717 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f461406d150 con 0x7f461413e2a0 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f45f403be00 0x7f45f403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f460400f040 con 0x7f461406c930 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4612d9d640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f45f403be00 0x7f45f403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.628+0000 7f4612d9d640 10 monclient: get_auth_request con 0x7f45f403be00 auth_method 0 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4611d9b640 1 -- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461413e2a0 msgr2=0x7f461413cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4611d9b640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461413e2a0 0x7f461413cb60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f46080107f0 tx=0x7f4608010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4608018020 con 0x7f461413e2a0 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4602ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:17.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4602ffd640 10 monclient: dump: 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.647 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4608004450 con 0x7f461413e2a0 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4602ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4602ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461413e2a0 msgr2=0x7f461413cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4602ffd640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461413e2a0 0x7f461413cb60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f46080107f0 tx=0x7f4608010cb0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461413e2a0 msgr2=0x7f461413cb60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4602ffd640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461413e2a0 0x7f461413cb60 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f46080107f0 tx=0x7f4608010cb0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.632+0000 7f4612d9d640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f45f403be00 0x7f45f403e2c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f460c009e50 tx=0x7f460c004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4618b51640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4618b51640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461402d1d0 0x7f461402d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4618b51640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4618b51640 1 -- 172.21.15.89:0/3319640717 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f4614064590 con 0x7f461402d1d0 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4611d9b640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461402d1d0 0x7f461402d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4611d9b640 10 monclient: get_auth_request con 0x7f461402d1d0 auth_method 0 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4611d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4611d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4611d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4611d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4611d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4611d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4611d9b640 10 monclient(hunting): handle_auth_done global_id 74189 payload 824 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.636+0000 7f4611d9b640 1 -- 172.21.15.89:0/3319640717 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f46140734a0 con 0x7f461402d1d0 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4611d9b640 1 -- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461402d1d0 msgr2=0x7f461402d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4611d9b640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461402d1d0 0x7f461402d5e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46080107f0 tx=0x7f46080046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4608018040 con 0x7f461402d1d0 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4602ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4602ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4602ffd640 10 monclient: dump: 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.650 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f4608003620 con 0x7f461402d1d0 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4602ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4602ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461402d1d0 msgr2=0x7f461402d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4602ffd640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461402d1d0 0x7f461402d5e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46080107f0 tx=0x7f46080046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4602ffd640 1 -- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461402d1d0 msgr2=0x7f461402d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:17.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4602ffd640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f461402d1d0 0x7f461402d5e0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46080107f0 tx=0x7f46080046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.652 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 14139, 2024-08-19T10:51:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:17.652 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:17.653 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:17.654 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:17.655 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:17.656 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:17.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:17.659 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.661 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.662 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.663 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:17.664 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.666 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.667 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.668 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:17.669 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.671 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.672 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.673 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.674 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:17.674 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:17.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4618b51640 1 -- 172.21.15.89:0/3319640717 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f45f403be00 msgr2=0x7f45f403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4618b51640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f45f403be00 0x7f45f403e2c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f460c009e50 tx=0x7f460c004ac0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4618b51640 10 monclient: shutdown 2024-08-19T10:51:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4618b51640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4618b51640 1 -- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461406c930 msgr2=0x7f46141388d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4618b51640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461406c930 0x7f46141388d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f460400d5f0 tx=0x7f460400dab0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4612d9d640 1 -- 172.21.15.89:0/3319640717 reap_dead start 2024-08-19T10:51:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4618b51640 1 -- 172.21.15.89:0/3319640717 shutdown_connections 2024-08-19T10:51:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4618b51640 1 --2- 172.21.15.89:0/3319640717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f461406c930 0x7f46141388d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4618b51640 1 -- 172.21.15.89:0/3319640717 >> 172.21.15.89:0/3319640717 conn(0x7f46140a83d0 msgr2=0x7f461413fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:17.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4618b51640 1 -- 172.21.15.89:0/3319640717 shutdown_connections 2024-08-19T10:51:17.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.640+0000 7f4618b51640 1 -- 172.21.15.89:0/3319640717 wait complete. 2024-08-19T10:51:17.677 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:17.677 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:51:17.677 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:51:17.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.788+0000 7f9748873640 10 monclient: get_monmap_and_config 2024-08-19T10:51:17.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.788+0000 7f9748873640 10 monclient: build_initial_monmap 2024-08-19T10:51:17.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.788+0000 7f9748873640 10 monclient: monmap: 2024-08-19T10:51:17.790 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:17.790 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.790 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:17.796302+0000 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:17.796302+0000 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:17.791 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:17.792 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:17.792 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:17.792 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 1 Processor -- start 2024-08-19T10:51:17.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 1 -- start start 2024-08-19T10:51:17.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 10 monclient: init 2024-08-19T10:51:17.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:17.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 10 monclient: _add_conns ranks=[5,3,1,4,2,8,6,9,0,7,10] 2024-08-19T10:51:17.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97440a6860 0x7f97440a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 10 monclient(hunting): picked mon.noname-h con 0x7f97440a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:17.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97440a7180 0x7f97440a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 10 monclient(hunting): picked mon.noname-g con 0x7f97440a7180 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:17.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9744139620 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:17.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97440a33d0 con 0x7f9744139620 2024-08-19T10:51:17.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f97440a8190 con 0x7f97440a7180 2024-08-19T10:51:17.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f97440a2ec0 con 0x7f97440a6860 2024-08-19T10:51:17.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:17.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9748873640 10 monclient(hunting): authenticate will time out at 1727.612856s 2024-08-19T10:51:17.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97440a7180 0x7f97440a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9742d9d640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f9744139620 0x7f9744139a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.89:56900/0 (socket says 172.21.15.89:56900) 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9742d9d640 1 -- 172.21.15.89:0/2790437584 learned_addr learned my addr 172.21.15.89:0/2790437584 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 10 monclient(hunting): get_auth_request con 0x7f97440a7180 auth_method 0 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f974259c640 1 --2- 172.21.15.89:0/2790437584 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97440a6860 0x7f97440a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f974259c640 10 monclient(hunting): get_auth_request con 0x7f97440a6860 auth_method 0 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f974259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f974259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f974259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f974259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f974259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f974259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 10 monclient(hunting): handle_auth_done global_id 35643 payload 274 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 1 -- 172.21.15.89:0/2790437584 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97440a6860 msgr2=0x7f97440a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 1 --2- 172.21.15.89:0/2790437584 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97440a6860 0x7f97440a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 1 -- 172.21.15.89:0/2790437584 >> v1:172.21.15.190:6789/0 conn(0x7f9744139620 legacy=0x7f9744139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 1 monclient: found mon.noname-g 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:51:17.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.792+0000 7f9741d9b640 1 -- 172.21.15.89:0/2790437584 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97440a4cd0 con 0x7f97440a7180 2024-08-19T10:51:17.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9741d9b640 1 -- 172.21.15.89:0/2790437584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97440a7180 msgr2=0x7f97440a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9741d9b640 1 --2- 172.21.15.89:0/2790437584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97440a7180 0x7f97440a7560 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f973c009dc0 tx=0x7f973c01c850 comp rx=0 tx=0).ready entity=mon.3 client_cookie=e4d74cd75291f804 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 1 -- 172.21.15.89:0/2790437584 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f973c0ac070 con 0x7f97440a7180 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 10 monclient: dump: 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.809 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.810 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 10 monclient: _finish_auth 0 2024-08-19T10:51:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 10 monclient: _check_auth_tickets 2024-08-19T10:51:17.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 1 -- 172.21.15.89:0/2790437584 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973c0a68b0 con 0x7f97440a7180 2024-08-19T10:51:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 1 -- 172.21.15.89:0/2790437584 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f973c0a7420 con 0x7f97440a7180 2024-08-19T10:51:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:51:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f974159a640 10 monclient: dump: 2024-08-19T10:51:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.811 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 5 monclient: authenticate success, global_id 35643 2024-08-19T10:51:17.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:17.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 4 set_mon_vals no callback set 2024-08-19T10:51:17.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 10 monclient: shutdown 2024-08-19T10:51:17.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:17.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 1 -- 172.21.15.89:0/2790437584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97440a7180 msgr2=0x7f97440a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 1 --2- 172.21.15.89:0/2790437584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97440a7180 0x7f97440a7560 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f973c009dc0 tx=0x7f973c01c850 comp rx=0 tx=0).stop 2024-08-19T10:51:17.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 1 -- 172.21.15.89:0/2790437584 shutdown_connections 2024-08-19T10:51:17.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 1 --2- 172.21.15.89:0/2790437584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97440a7180 0x7f97440a7560 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 1 --2- 172.21.15.89:0/2790437584 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97440a6860 0x7f97440a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 1 -- 172.21.15.89:0/2790437584 >> 172.21.15.89:0/2790437584 conn(0x7f9744074b40 msgr2=0x7f97440abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 1 -- 172.21.15.89:0/2790437584 shutdown_connections 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.796+0000 7f9748873640 1 -- 172.21.15.89:0/2790437584 wait complete. 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient: build_initial_monmap 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient: monmap: 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:17.814 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:17.815 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:17.815 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:17.815 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:17.815 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:17.815 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:17.816 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:17.816 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:17.816 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 1 Processor -- start 2024-08-19T10:51:17.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 1 -- start start 2024-08-19T10:51:17.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient: init 2024-08-19T10:51:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient: _add_conns ranks=[1,5,6,0,3,4,2,7,8] 2024-08-19T10:51:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440a6860 0x7f97441d1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient(hunting): picked mon.noname-f con 0x7f97440a6860 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:51:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97440a7180 0x7f97441d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient(hunting): picked mon.noname-h con 0x7f97440a7180 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9744139620 0x7f974413d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient(hunting): picked mon.noname-d con 0x7f9744139620 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9744144430 con 0x7f97440a6860 2024-08-19T10:51:17.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f97441442b0 con 0x7f97440a7180 2024-08-19T10:51:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f97441445b0 con 0x7f9744139620 2024-08-19T10:51:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9748873640 10 monclient(hunting): authenticate will time out at 1727.621452s 2024-08-19T10:51:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9744139620 0x7f974413d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9744139620 0x7f974413d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:41138/0 (socket says 172.21.15.89:41138) 2024-08-19T10:51:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 1 -- 172.21.15.89:0/3079948622 learned_addr learned my addr 172.21.15.89:0/3079948622 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 10 monclient(hunting): get_auth_request con 0x7f9744139620 auth_method 0 2024-08-19T10:51:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f974259c640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440a6860 0x7f97441d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9741d9b640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97440a7180 0x7f97441d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f974259c640 10 monclient(hunting): get_auth_request con 0x7f97440a6860 auth_method 0 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f974259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f974259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f974259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 10 monclient(hunting): handle_auth_done global_id 44988 payload 824 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 1 -- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97440a7180 msgr2=0x7f97441d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97440a7180 0x7f97441d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 1 -- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440a6860 msgr2=0x7f97441d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97440a6860 0x7f97441d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 1 monclient: found mon.noname-d 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9742d9d640 1 -- 172.21.15.89:0/3079948622 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9744122ba0 con 0x7f9744139620 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9741d9b640 10 monclient: get_auth_request con 0x7f97440a7180 auth_method 0 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.800+0000 7f9741d9b640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f97440a7180 0x7f97441d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9742d9d640 1 -- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9744139620 msgr2=0x7f974413d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9742d9d640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9744139620 0x7f974413d110 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9734009fd0 tx=0x7f973400b870 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9734015020 con 0x7f9744139620 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 10 monclient: dump: 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.821 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.822 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.822 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.822 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 10 monclient: _finish_auth 0 2024-08-19T10:51:17.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:51:17.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:17.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9734011cb0 con 0x7f9744139620 2024-08-19T10:51:17.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9748873640 5 monclient: authenticate success, global_id 44988 2024-08-19T10:51:17.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9748873640 10 monclient: _renew_subs 2024-08-19T10:51:17.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9748873640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:17.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9748873640 1 -- 172.21.15.89:0/3079948622 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974413f0a0 con 0x7f9744139620 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9734011e60 con 0x7f9744139620 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9740d99640 4 set_mon_vals no callback set 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 10 monclient: dump: 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.823 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9748873640 10 monclient: _renew_subs 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9748873640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9748873640 1 -- 172.21.15.89:0/3079948622 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f974413f760 con 0x7f9744139620 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9748873640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:17.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9748873640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413d650 0x7f974413da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9748873640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9748873640 1 -- 172.21.15.89:0/3079948622 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f97440a6c40 con 0x7f974413d650 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f974259c640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413d650 0x7f974413da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f974259c640 10 monclient: get_auth_request con 0x7f974413d650 auth_method 0 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f974259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f974259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f974259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f974259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f974259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f974259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9734018020 con 0x7f9744139620 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f974259c640 10 monclient(hunting): handle_auth_done global_id 45022 payload 824 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f974259c640 1 -- 172.21.15.89:0/3079948622 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9744064590 con 0x7f974413d650 2024-08-19T10:51:17.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f972403be00 0x7f972403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f973404f4c0 con 0x7f9744139620 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9741d9b640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f972403be00 0x7f972403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.804+0000 7f9741d9b640 10 monclient: get_auth_request con 0x7f972403be00 auth_method 0 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f974259c640 1 -- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413d650 msgr2=0x7f974413da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f974259c640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413d650 0x7f974413da30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9738011600 tx=0x7f9738011ac0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f973800fea0 con 0x7f974413d650 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9732ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9732ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9732ffd640 10 monclient: dump: 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.826 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f973800fa30 con 0x7f974413d650 2024-08-19T10:51:17.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9732ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:17.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9732ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:17.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413d650 msgr2=0x7f974413da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9732ffd640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413d650 0x7f974413da30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9738011600 tx=0x7f9738011ac0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413d650 msgr2=0x7f974413da30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:17.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9732ffd640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413d650 0x7f974413da30 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9738011600 tx=0x7f9738011ac0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.808+0000 7f9741d9b640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f972403be00 0x7f972403e2c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f973c0ab000 tx=0x7f973c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f9748873640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:17.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f9748873640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413df70 0x7f974409ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f9748873640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f9748873640 1 -- 172.21.15.89:0/3079948622 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f97441226a0 con 0x7f974413df70 2024-08-19T10:51:17.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f974259c640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413df70 0x7f974409ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f974259c640 10 monclient: get_auth_request con 0x7f974413df70 auth_method 0 2024-08-19T10:51:17.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f974259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f974259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f974259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f974259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f974259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f974259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f974259c640 10 monclient(hunting): handle_auth_done global_id 45031 payload 824 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.812+0000 7f974259c640 1 -- 172.21.15.89:0/3079948622 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9744140330 con 0x7f974413df70 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f974259c640 1 -- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413df70 msgr2=0x7f974409ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f974259c640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413df70 0x7f974409ef70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9738011600 tx=0x7f97380046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9738003a90 con 0x7f974413df70 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9732ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9732ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:17.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9732ffd640 10 monclient: dump: 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.832 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.833 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.833 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7f9738003620 con 0x7f974413df70 2024-08-19T10:51:17.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9732ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:17.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9732ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:17.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413df70 msgr2=0x7f974409ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9732ffd640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413df70 0x7f974409ef70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9738011600 tx=0x7f97380046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9732ffd640 1 -- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413df70 msgr2=0x7f974409ef70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:17.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9732ffd640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f974413df70 0x7f974409ef70 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9738011600 tx=0x7f97380046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.833 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 400943, 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:17.834 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:17.835 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:17.836 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:17.837 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:17.838 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:17.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.840 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:17.840 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:17.840 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:17.840 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.841 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:17.841 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.841 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:17.842 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:17.842 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.842 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.842 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:17.843 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:17.845 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:17.846 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.847 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.848 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.849 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.850 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.851 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.852 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.853 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.854 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:17.855 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:17.856 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:17.857 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:17.858 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:17.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1641967710}) 2024-08-19T10:51:17.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1438034662}) 2024-08-19T10:51:17.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1641967710}) 2024-08-19T10:51:17.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=41356499}) 2024-08-19T10:51:17.859 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9748873640 1 -- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f972403be00 msgr2=0x7f972403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9748873640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f972403be00 0x7f972403e2c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f973c0ab000 tx=0x7f973c0a7000 comp rx=0 tx=0).stop 2024-08-19T10:51:17.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9748873640 10 monclient: shutdown 2024-08-19T10:51:17.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9748873640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:17.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9748873640 1 -- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9744139620 msgr2=0x7f974413d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9748873640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9744139620 0x7f974413d110 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9734009fd0 tx=0x7f973400b870 comp rx=0 tx=0).stop 2024-08-19T10:51:17.860 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1641967710}) 2024-08-19T10:51:17.861 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1740702228}) 2024-08-19T10:51:17.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9742d9d640 1 -- 172.21.15.89:0/3079948622 reap_dead start 2024-08-19T10:51:17.861 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.861 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1641967710}) 2024-08-19T10:51:17.861 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3182615281}) 2024-08-19T10:51:17.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9748873640 1 -- 172.21.15.89:0/3079948622 shutdown_connections 2024-08-19T10:51:17.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9748873640 1 --2- 172.21.15.89:0/3079948622 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9744139620 0x7f974413d110 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9748873640 1 -- 172.21.15.89:0/3079948622 >> 172.21.15.89:0/3079948622 conn(0x7f9744074b40 msgr2=0x7f97440a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:17.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1641967710}) 2024-08-19T10:51:17.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2796810744}) 2024-08-19T10:51:17.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1641967710}) 2024-08-19T10:51:17.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=837353620}) 2024-08-19T10:51:17.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1641967710}) 2024-08-19T10:51:17.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2893326374}) 2024-08-19T10:51:17.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.863 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1641967710}) 2024-08-19T10:51:17.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2095433161}) 2024-08-19T10:51:17.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9748873640 1 -- 172.21.15.89:0/3079948622 shutdown_connections 2024-08-19T10:51:17.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.816+0000 7f9748873640 1 -- 172.21.15.89:0/3079948622 wait complete. 2024-08-19T10:51:17.865 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:17.865 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:51:17.865 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:51:17.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient: get_monmap_and_config 2024-08-19T10:51:17.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient: build_initial_monmap 2024-08-19T10:51:17.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient: monmap: 2024-08-19T10:51:17.980 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:17.980 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.980 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:17.986012+0000 2024-08-19T10:51:17.980 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:17.986012+0000 2024-08-19T10:51:17.980 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:17.980 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.980 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:17.980 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:17.980 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:17.981 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:17.981 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:17.981 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:17.981 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:17.981 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:17.981 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:17.981 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:17.981 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:17.981 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 1 Processor -- start 2024-08-19T10:51:17.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 1 -- start start 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient: init 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient: _add_conns ranks=[10,0,9,2,1,3,5,8,4,7,6] 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f21340a3b30 0x7f21340a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient(hunting): picked mon.noname-f con 0x7f21340a3b30 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f21340a3f10 addr v1:172.21.15.89:6789/0 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f213413b730 0x7f213413db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient(hunting): picked mon.noname-a con 0x7f213413b730 addr v2:172.21.15.89:3300/0 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2134074170 con 0x7f21340a3f10 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f21340a08b0 con 0x7f213413b730 2024-08-19T10:51:17.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f21340a0630 con 0x7f21340a3b30 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f213a689640 10 monclient(hunting): authenticate will time out at 1727.802917s 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f212b7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f21340a3f10 0x7f21340a4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44832/0 (socket says 172.21.15.89:44832) 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f212b7fe640 1 -- 172.21.15.89:0/1980708645 learned_addr learned my addr 172.21.15.89:0/1980708645 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 1 --2- 172.21.15.89:0/1980708645 >> v2:172.21.15.190:3300/0 conn(0x7f21340a3b30 0x7f21340a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 10 monclient(hunting): get_auth_request con 0x7f21340a3b30 auth_method 0 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:17.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:17.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 10 monclient(hunting): handle_auth_done global_id 44965 payload 274 2024-08-19T10:51:17.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 1 -- 172.21.15.89:0/1980708645 >> v2:172.21.15.89:3300/0 conn(0x7f213413b730 msgr2=0x7f213413db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:17.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 1 --2- 172.21.15.89:0/1980708645 >> v2:172.21.15.89:3300/0 conn(0x7f213413b730 0x7f213413db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 1 -- 172.21.15.89:0/1980708645 >> v1:172.21.15.89:6789/0 conn(0x7f21340a3f10 legacy=0x7f21340a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:17.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 10 monclient: _finish_hunting 0 2024-08-19T10:51:17.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 1 monclient: found mon.noname-f 2024-08-19T10:51:17.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:17.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:51:17.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.980+0000 7f2133fff640 1 -- 172.21.15.89:0/1980708645 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21340a1950 con 0x7f21340a3b30 2024-08-19T10:51:17.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189250033}) 2024-08-19T10:51:17.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1175142670}) 2024-08-19T10:51:17.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189250033}) 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1997522202}) 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189250033}) 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1537198087}) 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189250033}) 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3539271932}) 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189250033}) 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3512433211}) 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189250033}) 2024-08-19T10:51:17.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3285315738}) 2024-08-19T10:51:17.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189250033}) 2024-08-19T10:51:17.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2440048897}) 2024-08-19T10:51:17.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:17.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=189250033}) 2024-08-19T10:51:17.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:17.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3979684643}) 2024-08-19T10:51:17.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f2133fff640 1 -- 172.21.15.89:0/1980708645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21340a3b30 msgr2=0x7f21340a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:17.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f2133fff640 1 --2- 172.21.15.89:0/1980708645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21340a3b30 0x7f21340a2a60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f21340a33d0 tx=0x7f211c01c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c31389fd638b74 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:17.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 1 -- 172.21.15.89:0/1980708645 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f211c0b0070 con 0x7f21340a3b30 2024-08-19T10:51:17.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 10 monclient: dump: 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.992 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 10 monclient: _finish_auth 0 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 10 monclient: _check_auth_tickets 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 1 -- 172.21.15.89:0/1980708645 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211c0aa810 con 0x7f21340a3b30 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 1 -- 172.21.15.89:0/1980708645 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f211c0aae30 con 0x7f21340a3b30 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f21337fe640 10 monclient: dump: 2024-08-19T10:51:17.993 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:17.994 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f213a689640 5 monclient: authenticate success, global_id 44965 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f213a689640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f213a689640 4 set_mon_vals no callback set 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f213a689640 10 monclient: shutdown 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f213a689640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f213a689640 1 -- 172.21.15.89:0/1980708645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21340a3b30 msgr2=0x7f21340a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.988+0000 7f213a689640 1 --2- 172.21.15.89:0/1980708645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21340a3b30 0x7f21340a2a60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f21340a33d0 tx=0x7f211c01c300 comp rx=0 tx=0).stop 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.992+0000 7f213a689640 1 -- 172.21.15.89:0/1980708645 shutdown_connections 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.992+0000 7f213a689640 1 --2- 172.21.15.89:0/1980708645 >> v2:172.21.15.89:3300/0 conn(0x7f213413b730 0x7f213413db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.992+0000 7f213a689640 1 --2- 172.21.15.89:0/1980708645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21340a3b30 0x7f21340a2a60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:17.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.992+0000 7f213a689640 1 -- 172.21.15.89:0/1980708645 >> 172.21.15.89:0/1980708645 conn(0x7f21340a8f60 msgr2=0x7f21340a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:17.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.992+0000 7f213a689640 1 -- 172.21.15.89:0/1980708645 shutdown_connections 2024-08-19T10:51:17.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 1 -- 172.21.15.89:0/1980708645 wait complete. 2024-08-19T10:51:17.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient: build_initial_monmap 2024-08-19T10:51:17.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient: monmap: 2024-08-19T10:51:17.996 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:17.997 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:17.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 1 Processor -- start 2024-08-19T10:51:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 1 -- start start 2024-08-19T10:51:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient: init 2024-08-19T10:51:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient: _add_conns ranks=[4,8,3,7,6,2,5,0,1] 2024-08-19T10:51:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21340a3b30 0x7f21341232b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient(hunting): picked mon.noname-c con 0x7f21340a3b30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f21340a3f10 0x7f213413f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient(hunting): picked mon.noname-e con 0x7f21340a3f10 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f213413b730 0x7f213413f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:17.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient(hunting): picked mon.noname-g con 0x7f213413b730 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f213413ec40 con 0x7f213413b730 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f213413eac0 con 0x7f21340a3b30 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f213413edc0 con 0x7f21340a3f10 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f213a689640 10 monclient(hunting): authenticate will time out at 1727.818572s 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f212b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f21340a3f10 0x7f213413f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f212b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f21340a3f10 0x7f213413f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48596/0 (socket says 172.21.15.89:48596) 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21340a3b30 0x7f21341232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f212b7fe640 1 -- 172.21.15.89:0/4130775947 learned_addr learned my addr 172.21.15.89:0/4130775947 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:18.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f212b7fe640 10 monclient(hunting): get_auth_request con 0x7f21340a3f10 auth_method 0 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f212b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f212b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f212b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 10 monclient(hunting): get_auth_request con 0x7f21340a3b30 auth_method 0 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 10 monclient(hunting): handle_auth_done global_id 45040 payload 824 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 1 -- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f21340a3f10 msgr2=0x7f213413f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2138c26640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f213413b730 0x7f213413f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f21340a3f10 0x7f213413f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 1 -- 172.21.15.89:0/4130775947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f213413b730 msgr2=0x7f213413f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f213413b730 0x7f213413f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 10 monclient: _finish_hunting 0 2024-08-19T10:51:18.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 1 monclient: found mon.noname-c 2024-08-19T10:51:18.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:18.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2133fff640 1 -- 172.21.15.89:0/4130775947 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21341215b0 con 0x7f21340a3b30 2024-08-19T10:51:18.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2138c26640 10 monclient: get_auth_request con 0x7f213413b730 auth_method 0 2024-08-19T10:51:18.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:17.996+0000 7f2138c26640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f213413b730 0x7f213413f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:18.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.000+0000 7f2133fff640 1 -- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21340a3b30 msgr2=0x7f21341232b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.000+0000 7f2133fff640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21340a3b30 0x7f21341232b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f211c0ad000 tx=0x7f211c0190a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f211c0b0040 con 0x7f21340a3b30 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 10 monclient: dump: 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.004 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 10 monclient: _finish_auth 0 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 10 monclient: _check_auth_tickets 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211c0b9070 con 0x7f21340a3b30 2024-08-19T10:51:18.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f213a689640 5 monclient: authenticate success, global_id 45040 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f213a689640 10 monclient: _renew_subs 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f213a689640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f213a689640 1 -- 172.21.15.89:0/4130775947 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21340a6cc0 con 0x7f21340a3b30 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f211c0044b0 con 0x7f21340a3b30 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 10 monclient: dump: 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.006 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f2132ffd640 4 set_mon_vals no callback set 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f213a689640 10 monclient: _renew_subs 2024-08-19T10:51:18.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f213a689640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f213a689640 1 -- 172.21.15.89:0/4130775947 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f21341237f0 con 0x7f21340a3b30 2024-08-19T10:51:18.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=886059150}) 2024-08-19T10:51:18.008 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=220056175}) 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=886059150}) 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2869250984}) 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=886059150}) 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3287974326}) 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=886059150}) 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1010044260}) 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=886059150}) 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2290524732}) 2024-08-19T10:51:18.009 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=886059150}) 2024-08-19T10:51:18.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2598088842}) 2024-08-19T10:51:18.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=886059150}) 2024-08-19T10:51:18.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=129558568}) 2024-08-19T10:51:18.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=886059150}) 2024-08-19T10:51:18.010 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=595355239}) 2024-08-19T10:51:18.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f211c0b3070 con 0x7f21340a3b30 2024-08-19T10:51:18.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f213a689640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f213a689640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f213413faf0 0x7f2134072e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f213a689640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f213a689640 1 -- 172.21.15.89:0/4130775947 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f213413dbc0 con 0x7f213413faf0 2024-08-19T10:51:18.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f211403be00 0x7f211403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f211c0e9130 con 0x7f21340a3b30 2024-08-19T10:51:18.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f2138c26640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f211403be00 0x7f211403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.004+0000 7f2138c26640 10 monclient: get_auth_request con 0x7f211403be00 auth_method 0 2024-08-19T10:51:18.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f213413faf0 0x7f2134072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 10 monclient: get_auth_request con 0x7f213413faf0 auth_method 0 2024-08-19T10:51:18.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 10 monclient(hunting): handle_auth_done global_id 44997 payload 824 2024-08-19T10:51:18.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 1 -- 172.21.15.89:0/4130775947 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f213413fed0 con 0x7f213413faf0 2024-08-19T10:51:18.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f2138c26640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f211403be00 0x7f211403e2c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f213411ffc0 tx=0x7f212c007360 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 1 -- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f213413faf0 msgr2=0x7f2134072e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f212b7fe640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f213413faf0 0x7f2134072e20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f212400e700 tx=0x7f212400ebc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2124019070 con 0x7f213413faf0 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f21317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f21317fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f21317fa640 10 monclient: dump: 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.015 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2124004c30 con 0x7f213413faf0 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f21317fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f21317fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f213413faf0 msgr2=0x7f2134072e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f21317fa640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f213413faf0 0x7f2134072e20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f212400e700 tx=0x7f212400ebc0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f213413faf0 msgr2=0x7f2134072e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.008+0000 7f21317fa640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f213413faf0 0x7f2134072e20 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f212400e700 tx=0x7f212400ebc0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f213a689640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:18.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f213a689640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2134073360 0x7f2134073770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f213a689640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f213a689640 1 -- 172.21.15.89:0/4130775947 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f2134064590 con 0x7f2134073360 2024-08-19T10:51:18.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2134073360 0x7f2134073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 10 monclient: get_auth_request con 0x7f2134073360 auth_method 0 2024-08-19T10:51:18.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 10 monclient(hunting): handle_auth_done global_id 45006 payload 824 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 1 -- 172.21.15.89:0/4130775947 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f21340a71f0 con 0x7f2134073360 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 1 -- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2134073360 msgr2=0x7f2134073770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.016+0000 7f212b7fe640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2134073360 0x7f2134073770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f212400e700 tx=0x7f2124004950 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2124019030 con 0x7f2134073360 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f21317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f21317fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f21317fa640 10 monclient: dump: 2024-08-19T10:51:18.020 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.021 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.022 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7f21240077b0 con 0x7f2134073360 2024-08-19T10:51:18.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f21317fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:18.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f21317fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:18.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2134073360 msgr2=0x7f2134073770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f21317fa640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2134073360 0x7f2134073770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f212400e700 tx=0x7f2124004950 comp rx=0 tx=0).stop 2024-08-19T10:51:18.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f21317fa640 1 -- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2134073360 msgr2=0x7f2134073770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f21317fa640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2134073360 0x7f2134073770 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f212400e700 tx=0x7f2124004950 comp rx=0 tx=0).stop 2024-08-19T10:51:18.022 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 401147, 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:18.023 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.024 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.025 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:18.026 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.027 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:18.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.030 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.031 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.035 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.038 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:18.041 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.042 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.044 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.045 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.046 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.046 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.046 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:18.046 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:18.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f213a689640 1 -- 172.21.15.89:0/4130775947 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f211403be00 msgr2=0x7f211403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f213a689640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f211403be00 0x7f211403e2c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f213411ffc0 tx=0x7f212c007360 comp rx=0 tx=0).stop 2024-08-19T10:51:18.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f213a689640 10 monclient: shutdown 2024-08-19T10:51:18.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f213a689640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:18.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f213a689640 1 -- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21340a3b30 msgr2=0x7f21341232b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f213a689640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21340a3b30 0x7f21341232b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f211c0ad000 tx=0x7f211c0190a0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f2138c26640 1 -- 172.21.15.89:0/4130775947 reap_dead start 2024-08-19T10:51:18.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f213a689640 1 -- 172.21.15.89:0/4130775947 shutdown_connections 2024-08-19T10:51:18.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f213a689640 1 --2- 172.21.15.89:0/4130775947 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f21340a3b30 0x7f21341232b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f213a689640 1 -- 172.21.15.89:0/4130775947 >> 172.21.15.89:0/4130775947 conn(0x7f21340a8f60 msgr2=0x7f213413bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:18.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f213a689640 1 -- 172.21.15.89:0/4130775947 shutdown_connections 2024-08-19T10:51:18.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f213a689640 1 -- 172.21.15.89:0/4130775947 wait complete. 2024-08-19T10:51:18.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3873433867}) 2024-08-19T10:51:18.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3538346679}) 2024-08-19T10:51:18.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3873433867}) 2024-08-19T10:51:18.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4196054829}) 2024-08-19T10:51:18.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.049 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3873433867}) 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2038901589}) 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3873433867}) 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=949814612}) 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3873433867}) 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3914324641}) 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3873433867}) 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1497436864}) 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3873433867}) 2024-08-19T10:51:18.050 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2365998025}) 2024-08-19T10:51:18.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3873433867}) 2024-08-19T10:51:18.051 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.020+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2263281374}) 2024-08-19T10:51:18.052 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:18.052 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:51:18.052 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:51:18.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient: get_monmap_and_config 2024-08-19T10:51:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient: build_initial_monmap 2024-08-19T10:51:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient: monmap: 2024-08-19T10:51:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:18.172994+0000 2024-08-19T10:51:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:18.172994+0000 2024-08-19T10:51:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:18.167 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:18.168 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:18.168 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:18.168 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:18.168 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:18.168 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:18.168 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:18.168 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:18.168 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:18.168 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:18.168 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 1 Processor -- start 2024-08-19T10:51:18.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 1 -- start start 2024-08-19T10:51:18.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient: init 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient: _add_conns ranks=[2,10,7,5,4,9,6,0,8,1,3] 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f3c0a5270 0x7f9f3c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient(hunting): picked mon.noname-b con 0x7f9f3c0a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f3c0a6d30 0x7f9f3c0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient(hunting): picked mon.noname-f con 0x7f9f3c0a6d30 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9f3c1395b0 0x7f9f3c139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient(hunting): picked mon.noname-i con 0x7f9f3c1395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9f3c0a7650 con 0x7f9f3c0a6d30 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c0a8710 con 0x7f9f3c0a5270 2024-08-19T10:51:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9f3c0a5a40 con 0x7f9f3c1395b0 2024-08-19T10:51:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f44347640 10 monclient(hunting): authenticate will time out at 1727.989384s 2024-08-19T10:51:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f3c0a5270 0x7f9f3c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f3c0a5270 0x7f9f3c0a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35446/0 (socket says 172.21.15.89:35446) 2024-08-19T10:51:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 1 -- 172.21.15.89:0/725638598 learned_addr learned my addr 172.21.15.89:0/725638598 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 10 monclient(hunting): get_auth_request con 0x7f9f3c0a5270 auth_method 0 2024-08-19T10:51:18.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f428e4640 1 --2- 172.21.15.89:0/725638598 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9f3c1395b0 0x7f9f3c139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f418e2640 1 --2- 172.21.15.89:0/725638598 >> v2:172.21.15.190:3300/0 conn(0x7f9f3c0a6d30 0x7f9f3c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 10 monclient(hunting): handle_auth_done global_id 74198 payload 274 2024-08-19T10:51:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 1 -- 172.21.15.89:0/725638598 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9f3c1395b0 msgr2=0x7f9f3c139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 1 --2- 172.21.15.89:0/725638598 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9f3c1395b0 0x7f9f3c139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 1 -- 172.21.15.89:0/725638598 >> v2:172.21.15.190:3300/0 conn(0x7f9f3c0a6d30 msgr2=0x7f9f3c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 1 --2- 172.21.15.89:0/725638598 >> v2:172.21.15.190:3300/0 conn(0x7f9f3c0a6d30 0x7f9f3c0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 10 monclient: _finish_hunting 0 2024-08-19T10:51:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 1 monclient: found mon.noname-b 2024-08-19T10:51:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:51:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f420e3640 1 -- 172.21.15.89:0/725638598 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c0a81d0 con 0x7f9f3c0a5270 2024-08-19T10:51:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f428e4640 10 monclient: get_auth_request con 0x7f9f3c1395b0 auth_method 0 2024-08-19T10:51:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f418e2640 10 monclient: get_auth_request con 0x7f9f3c0a6d30 auth_method 0 2024-08-19T10:51:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f428e4640 1 --2- 172.21.15.89:0/725638598 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9f3c1395b0 0x7f9f3c139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:18.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.168+0000 7f9f418e2640 1 --2- 172.21.15.89:0/725638598 >> v2:172.21.15.190:3300/0 conn(0x7f9f3c0a6d30 0x7f9f3c0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:18.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f420e3640 1 -- 172.21.15.89:0/725638598 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f3c0a5270 msgr2=0x7f9f3c0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f420e3640 1 --2- 172.21.15.89:0/725638598 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f3c0a5270 0x7f9f3c0a67f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009870 tx=0x7f9f3801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f6d67deb1de688 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 1 -- 172.21.15.89:0/725638598 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9f380b1070 con 0x7f9f3c0a5270 2024-08-19T10:51:18.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:51:18.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 10 monclient: dump: 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.180 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 10 monclient: _finish_auth 0 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 10 monclient: _check_auth_tickets 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 1 -- 172.21.15.89:0/725638598 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f380ab810 con 0x7f9f3c0a5270 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 1 -- 172.21.15.89:0/725638598 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9f380abe30 con 0x7f9f3c0a5270 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f410e1640 10 monclient: dump: 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.181 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.182 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f44347640 5 monclient: authenticate success, global_id 74198 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f44347640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.172+0000 7f9f44347640 4 set_mon_vals no callback set 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient: shutdown 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 -- 172.21.15.89:0/725638598 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f3c0a5270 msgr2=0x7f9f3c0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 --2- 172.21.15.89:0/725638598 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f3c0a5270 0x7f9f3c0a67f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009870 tx=0x7f9f3801c420 comp rx=0 tx=0).stop 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 -- 172.21.15.89:0/725638598 shutdown_connections 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 --2- 172.21.15.89:0/725638598 >> v2:172.21.15.190:3300/0 conn(0x7f9f3c0a6d30 0x7f9f3c0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 --2- 172.21.15.89:0/725638598 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9f3c1395b0 0x7f9f3c139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 --2- 172.21.15.89:0/725638598 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9f3c0a5270 0x7f9f3c0a67f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 -- 172.21.15.89:0/725638598 >> 172.21.15.89:0/725638598 conn(0x7f9f3c069a80 msgr2=0x7f9f3c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 -- 172.21.15.89:0/725638598 shutdown_connections 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 -- 172.21.15.89:0/725638598 wait complete. 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient: build_initial_monmap 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient: monmap: 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:18.184 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 Processor -- start 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 -- start start 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient: init 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient: _add_conns ranks=[6,0,1,8,4,5,7,3,2] 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9f3c0a5270 0x7f9f3c0a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient(hunting): picked mon.noname-d con 0x7f9f3c0a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f3c0a6d30 0x7f9f3c0a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient(hunting): picked mon.noname-a con 0x7f9f3c0a6d30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f3c1395b0 0x7f9f3c09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient(hunting): picked mon.noname-f con 0x7f9f3c1395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c0a5650 con 0x7f9f3c0a6d30 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c0a4a70 con 0x7f9f3c1395b0 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9f3c0a57d0 con 0x7f9f3c0a5270 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f44347640 10 monclient(hunting): authenticate will time out at 1727.998674s 2024-08-19T10:51:18.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9f3c0a5270 0x7f9f3c0a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9f3c0a5270 0x7f9f3c0a1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:41162/0 (socket says 172.21.15.89:41162) 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 1 -- 172.21.15.89:0/4044435136 learned_addr learned my addr 172.21.15.89:0/4044435136 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 10 monclient(hunting): get_auth_request con 0x7f9f3c0a5270 auth_method 0 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f418e2640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f3c0a6d30 0x7f9f3c0a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f418e2640 10 monclient(hunting): get_auth_request con 0x7f9f3c0a6d30 auth_method 0 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f418e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f418e2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f418e2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f418e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f418e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f418e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 10 monclient(hunting): handle_auth_done global_id 45015 payload 824 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f428e4640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f3c1395b0 0x7f9f3c09c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 1 -- 172.21.15.89:0/4044435136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f3c1395b0 msgr2=0x7f9f3c09c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f3c1395b0 0x7f9f3c09c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 1 -- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f3c0a6d30 msgr2=0x7f9f3c0a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f3c0a6d30 0x7f9f3c0a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 10 monclient: _finish_hunting 0 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 1 monclient: found mon.noname-d 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:18.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:51:18.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.176+0000 7f9f420e3640 1 -- 172.21.15.89:0/4044435136 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c0a26c0 con 0x7f9f3c0a5270 2024-08-19T10:51:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f420e3640 1 -- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9f3c0a5270 msgr2=0x7f9f3c0a1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f420e3640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9f3c0a5270 0x7f9f3c0a1c40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f380097c0 tx=0x7f9f380191c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9f380ba070 con 0x7f9f3c0a5270 2024-08-19T10:51:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:51:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 10 monclient: dump: 2024-08-19T10:51:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 10 monclient: _finish_auth 0 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f380abe20 con 0x7f9f3c0a5270 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9f380b1040 con 0x7f9f3c0a5270 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f44347640 5 monclient: authenticate success, global_id 45015 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f44347640 10 monclient: _renew_subs 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f44347640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f44347640 1 -- 172.21.15.89:0/4044435136 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c13f7a0 con 0x7f9f3c0a5270 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f408e0640 4 set_mon_vals no callback set 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 10 monclient: dump: 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f44347640 10 monclient: _renew_subs 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f44347640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f44347640 1 -- 172.21.15.89:0/4044435136 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c09f140 con 0x7f9f3c0a5270 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f44347640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f44347640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09f310 0x7f9f3c09cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f44347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f44347640 1 -- 172.21.15.89:0/4044435136 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9f3c0a7110 con 0x7f9f3c09f310 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9f38004250 con 0x7f9f3c0a5270 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f418e2640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09f310 0x7f9f3c09cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f418e2640 10 monclient: get_auth_request con 0x7f9f3c09f310 auth_method 0 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f418e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f418e2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f418e2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f418e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f418e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f418e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9f2003bdf0 0x7f9f2003e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f9f380e8950 con 0x7f9f3c0a5270 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f418e2640 10 monclient(hunting): handle_auth_done global_id 7109 payload 824 2024-08-19T10:51:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f418e2640 1 -- 172.21.15.89:0/4044435136 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f3c064590 con 0x7f9f3c09f310 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f428e4640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9f2003bdf0 0x7f9f2003e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.180+0000 7f9f428e4640 10 monclient: get_auth_request con 0x7f9f2003bdf0 auth_method 0 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f418e2640 1 -- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09f310 msgr2=0x7f9f3c09cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f418e2640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09f310 0x7f9f3c09cf20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0a2c20 tx=0x7f9f2c00fce0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9f2c00dc30 con 0x7f9f3c09f310 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f32ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f32ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f32ffd640 10 monclient: dump: 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9f2c00dc30 con 0x7f9f3c09f310 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f32ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f32ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09f310 msgr2=0x7f9f3c09cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f32ffd640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09f310 0x7f9f3c09cf20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0a2c20 tx=0x7f9f2c00fce0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09f310 msgr2=0x7f9f3c09cf20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f32ffd640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09f310 0x7f9f3c09cf20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.184+0000 7f9f428e4640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9f2003bdf0 0x7f9f2003e2b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9f34007130 tx=0x7f9f340083c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09df00 0x7f9f3c09d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 1 -- 172.21.15.89:0/4044435136 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9f3c1356e0 con 0x7f9f3c09df00 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09df00 0x7f9f3c09d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 10 monclient: get_auth_request con 0x7f9f3c09df00 auth_method 0 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 10 monclient(hunting): handle_auth_done global_id 7118 payload 824 2024-08-19T10:51:18.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 1 -- 172.21.15.89:0/4044435136 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9f3c06c9e0 con 0x7f9f3c09df00 2024-08-19T10:51:18.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 1 -- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09df00 msgr2=0x7f9f3c09d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f418e2640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09df00 0x7f9f3c09d460 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0a2c20 tx=0x7f9f2c0046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9f2c004490 con 0x7f9f3c09df00 2024-08-19T10:51:18.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f32ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f32ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:51:18.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f32ffd640 10 monclient: dump: 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.208 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.209 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.209 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f9f2c004020 con 0x7f9f3c09df00 2024-08-19T10:51:18.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f32ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:18.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f32ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:18.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09df00 msgr2=0x7f9f3c09d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f32ffd640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09df00 0x7f9f3c09d460 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0a2c20 tx=0x7f9f2c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f32ffd640 1 -- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09df00 msgr2=0x7f9f3c09d460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f32ffd640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9f3c09df00 0x7f9f3c09d460 secure :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0a2c20 tx=0x7f9f2c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.209 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 807986, 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:18.210 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.211 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.212 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:18.213 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:18.214 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.214 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.214 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:18.214 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:18.214 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:18.214 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:18.214 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:18.214 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:18.214 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.215 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.218 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.219 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.220 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.224 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:18.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.231 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:18.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.233 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.234 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:18.234 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 1 -- 172.21.15.89:0/4044435136 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9f2003bdf0 msgr2=0x7f9f2003e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9f2003bdf0 0x7f9f2003e2b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9f34007130 tx=0x7f9f340083c0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 10 monclient: shutdown 2024-08-19T10:51:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:18.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 1 -- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9f3c0a5270 msgr2=0x7f9f3c0a1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9f3c0a5270 0x7f9f3c0a1c40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f380097c0 tx=0x7f9f380191c0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f428e4640 1 -- 172.21.15.89:0/4044435136 reap_dead start 2024-08-19T10:51:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 1 -- 172.21.15.89:0/4044435136 shutdown_connections 2024-08-19T10:51:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 1 --2- 172.21.15.89:0/4044435136 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9f3c0a5270 0x7f9f3c0a1c40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 1 -- 172.21.15.89:0/4044435136 >> 172.21.15.89:0/4044435136 conn(0x7f9f3c069a80 msgr2=0x7f9f3c1345d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 1 -- 172.21.15.89:0/4044435136 shutdown_connections 2024-08-19T10:51:18.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.192+0000 7f9f44347640 1 -- 172.21.15.89:0/4044435136 wait complete. 2024-08-19T10:51:18.236 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:18.236 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:51:18.237 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:51:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient: get_monmap_and_config 2024-08-19T10:51:18.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient: build_initial_monmap 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient: monmap: 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:18.357696+0000 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:18.357696+0000 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:18.352 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:18.353 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:18.353 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:18.353 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:18.353 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:18.353 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:18.353 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 1 Processor -- start 2024-08-19T10:51:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 1 -- start start 2024-08-19T10:51:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient: init 2024-08-19T10:51:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient: _add_conns ranks=[1,5,10,3,7,6,8,9,2,0,4] 2024-08-19T10:51:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f59f80a6860 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:18.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59f80a8480 0x7f59f80a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient(hunting): picked mon.noname-h con 0x7f59f80a8480 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59f813e860 0x7f59f8140c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient(hunting): picked mon.noname-f con 0x7f59f813e860 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f80a8240 con 0x7f59f80a6860 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f59f80a4d50 con 0x7f59f813e860 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f59f80a2ec0 con 0x7f59f80a8480 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fe4a9640 10 monclient(hunting): authenticate will time out at 1728.174396s 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fca46640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59f813e860 0x7f59f8140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59f80a8480 0x7f59f80a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fca46640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59f813e860 0x7f59f8140c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:48406/0 (socket says 172.21.15.89:48406) 2024-08-19T10:51:18.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fca46640 1 -- 172.21.15.89:0/2040832987 learned_addr learned my addr 172.21.15.89:0/2040832987 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fca46640 10 monclient(hunting): get_auth_request con 0x7f59f813e860 auth_method 0 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fca46640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fca46640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fca46640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 10 monclient(hunting): get_auth_request con 0x7f59f80a8480 auth_method 0 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fca46640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fca46640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59fca46640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 10 monclient(hunting): handle_auth_done global_id 7178 payload 274 2024-08-19T10:51:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 1 -- 172.21.15.89:0/2040832987 >> v2:172.21.15.190:3300/0 conn(0x7f59f813e860 msgr2=0x7f59f8140c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 1 --2- 172.21.15.89:0/2040832987 >> v2:172.21.15.190:3300/0 conn(0x7f59f813e860 0x7f59f8140c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 1 -- 172.21.15.89:0/2040832987 >> v1:172.21.15.190:6789/0 conn(0x7f59f80a6860 legacy=0x7f59f80a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:51:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 1 monclient: found mon.noname-h 2024-08-19T10:51:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:51:18.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.352+0000 7f59f77fe640 1 -- 172.21.15.89:0/2040832987 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f80a4a70 con 0x7f59f80a8480 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.356+0000 7f59f77fe640 1 -- 172.21.15.89:0/2040832987 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59f80a8480 msgr2=0x7f59f80a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.356+0000 7f59f77fe640 1 --2- 172.21.15.89:0/2040832987 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59f80a8480 0x7f59f80a7350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f59e00097c0 tx=0x7f59e001c360 comp rx=0 tx=0).ready entity=mon.5 client_cookie=e1a57c04e84fe064 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 1 -- 172.21.15.89:0/2040832987 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59e00b0070 con 0x7f59f80a8480 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 10 monclient: dump: 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.360 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:18.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 1 -- 172.21.15.89:0/2040832987 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e00aa810 con 0x7f59f80a8480 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 1 -- 172.21.15.89:0/2040832987 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59e00a9300 con 0x7f59f80a8480 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 5 monclient: authenticate success, global_id 7178 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 4 set_mon_vals no callback set 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f6ffd640 10 monclient: dump: 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.362 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient: shutdown 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:18.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 -- 172.21.15.89:0/2040832987 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59f80a8480 msgr2=0x7f59f80a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 --2- 172.21.15.89:0/2040832987 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59f80a8480 0x7f59f80a7350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f59e00097c0 tx=0x7f59e001c360 comp rx=0 tx=0).stop 2024-08-19T10:51:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 -- 172.21.15.89:0/2040832987 shutdown_connections 2024-08-19T10:51:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 --2- 172.21.15.89:0/2040832987 >> v2:172.21.15.190:3300/0 conn(0x7f59f813e860 0x7f59f8140c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 --2- 172.21.15.89:0/2040832987 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f59f80a8480 0x7f59f80a7350 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 -- 172.21.15.89:0/2040832987 >> 172.21.15.89:0/2040832987 conn(0x7f59f80699d0 msgr2=0x7f59f80abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 -- 172.21.15.89:0/2040832987 shutdown_connections 2024-08-19T10:51:18.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 -- 172.21.15.89:0/2040832987 wait complete. 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient: build_initial_monmap 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient: monmap: 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:18.369 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 Processor -- start 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 -- start start 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient: init 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient: _add_conns ranks=[0,2,8,5,3,7,1,4,6] 2024-08-19T10:51:18.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59f80a6860 0x7f59f81258b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient(hunting): picked mon.noname-a con 0x7f59f80a6860 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f80a8480 0x7f59f813c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient(hunting): picked mon.noname-b con 0x7f59f80a8480 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59f813e860 0x7f59f81232e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient(hunting): picked mon.noname-e con 0x7f59f813e860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8143d40 con 0x7f59f80a6860 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f59f8143bc0 con 0x7f59f80a8480 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f59f8143ec0 con 0x7f59f813e860 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:18.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fe4a9640 10 monclient(hunting): authenticate will time out at 1728.183236s 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f80a8480 0x7f59f813c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f80a8480 0x7f59f813c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35452/0 (socket says 172.21.15.89:35452) 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 1 -- 172.21.15.89:0/905317039 learned_addr learned my addr 172.21.15.89:0/905317039 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 10 monclient(hunting): get_auth_request con 0x7f59f80a8480 auth_method 0 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f7fff640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59f80a6860 0x7f59f81258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f7fff640 10 monclient(hunting): get_auth_request con 0x7f59f80a6860 auth_method 0 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fca46640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59f813e860 0x7f59f81232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fca46640 10 monclient(hunting): get_auth_request con 0x7f59f813e860 auth_method 0 2024-08-19T10:51:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fca46640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fca46640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fca46640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fca46640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fca46640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59fca46640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 10 monclient(hunting): handle_auth_done global_id 74207 payload 824 2024-08-19T10:51:18.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 1 -- 172.21.15.89:0/905317039 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59f813e860 msgr2=0x7f59f81232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f59f813e860 0x7f59f81232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 1 -- 172.21.15.89:0/905317039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59f80a6860 msgr2=0x7f59f81258b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f59f80a6860 0x7f59f81258b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:51:18.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 1 monclient: found mon.noname-b 2024-08-19T10:51:18.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:18.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.360+0000 7f59f77fe640 1 -- 172.21.15.89:0/905317039 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f8126240 con 0x7f59f80a8480 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f77fe640 1 -- 172.21.15.89:0/905317039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f80a8480 msgr2=0x7f59f813c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f77fe640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f80a8480 0x7f59f813c720 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f59e001cc00 tx=0x7f59e0019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59e00b0040 con 0x7f59f80a8480 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 10 monclient: dump: 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 10 monclient: _finish_auth 0 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e00b9070 con 0x7f59f80a8480 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59fe4a9640 5 monclient: authenticate success, global_id 74207 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59e00044b0 con 0x7f59f80a8480 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f5ffb640 4 set_mon_vals no callback set 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 10 monclient: dump: 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.379 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59fe4a9640 10 monclient: _renew_subs 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59fe4a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59fe4a9640 1 -- 172.21.15.89:0/905317039 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f81270b0 con 0x7f59f80a8480 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59fe4a9640 10 monclient: _renew_subs 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59fe4a9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59fe4a9640 1 -- 172.21.15.89:0/905317039 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f80aafe0 con 0x7f59f80a8480 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59fe4a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59fe4a9640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802d1d0 0x7f59f802d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59fe4a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59fe4a9640 1 -- 172.21.15.89:0/905317039 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8064590 con 0x7f59f802d1d0 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.364+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f59e0004040 con 0x7f59f80a8480 2024-08-19T10:51:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f4ff9640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f59d803bdf0 0x7f59d803e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802d1d0 0x7f59f802d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f59e00e9140 con 0x7f59f80a8480 2024-08-19T10:51:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 10 monclient: get_auth_request con 0x7f59f802d1d0 auth_method 0 2024-08-19T10:51:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59fca46640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f59d803bdf0 0x7f59d803e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59fca46640 10 monclient: get_auth_request con 0x7f59d803bdf0 auth_method 0 2024-08-19T10:51:18.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 10 monclient(hunting): handle_auth_done global_id 44983 payload 824 2024-08-19T10:51:18.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 1 -- 172.21.15.89:0/905317039 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59f80a6c40 con 0x7f59f802d1d0 2024-08-19T10:51:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 1 -- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802d1d0 msgr2=0x7f59f802d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f7fff640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802d1d0 0x7f59f802d5b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f59e800f770 tx=0x7f59e800fc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59fca46640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f59d803bdf0 0x7f59d803e2b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f59f0011340 tx=0x7f59f000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59e800fe40 con 0x7f59f802d1d0 2024-08-19T10:51:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f4ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:51:18.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.368+0000 7f59f4ff9640 10 monclient: dump: 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.385 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.372+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f59e8019070 con 0x7f59f802d1d0 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.372+0000 7f59f4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.372+0000 7f59f4ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.372+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802d1d0 msgr2=0x7f59f802d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.372+0000 7f59f4ff9640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802d1d0 0x7f59f802d5b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f59e800f770 tx=0x7f59e800fc30 comp rx=0 tx=0).stop 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.372+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802d1d0 msgr2=0x7f59f802d5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.372+0000 7f59f4ff9640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802d1d0 0x7f59f802d5b0 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f59e800f770 tx=0x7f59e800fc30 comp rx=0 tx=0).stop 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59fe4a9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59fe4a9640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802daf0 0x7f59f802df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59fe4a9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59fe4a9640 1 -- 172.21.15.89:0/905317039 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59f81d1c40 con 0x7f59f802daf0 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59f7fff640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802daf0 0x7f59f802df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59f7fff640 10 monclient: get_auth_request con 0x7f59f802daf0 auth_method 0 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59f7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59f7fff640 10 monclient(hunting): handle_auth_done global_id 44992 payload 824 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.376+0000 7f59f7fff640 1 -- 172.21.15.89:0/905317039 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f59f8124d50 con 0x7f59f802daf0 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f7fff640 1 -- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802daf0 msgr2=0x7f59f802df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f7fff640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802daf0 0x7f59f802df00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f59e8016040 tx=0x7f59e8003770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f59e801ec30 con 0x7f59f802daf0 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f4ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f4ff9640 10 monclient: dump: 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.388 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f59e8006c00 con 0x7f59f802daf0 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f4ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802daf0 msgr2=0x7f59f802df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f4ff9640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802daf0 0x7f59f802df00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f59e8016040 tx=0x7f59e8003770 comp rx=0 tx=0).stop 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f4ff9640 1 -- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802daf0 msgr2=0x7f59f802df00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59f4ff9640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59f802daf0 0x7f59f802df00 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f59e8016040 tx=0x7f59e8003770 comp rx=0 tx=0).stop 2024-08-19T10:51:18.390 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:18.390 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:18.390 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:18.390 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:18.390 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:18.390 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 401511, 2024-08-19T10:51:18.390 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:18.390 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:18.390 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:18.392 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.393 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:18.394 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.395 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.397 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.399 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.400 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:18.402 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:18.402 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.402 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.402 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:18.403 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:18.404 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:18.404 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.404 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.405 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.405 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.405 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.405 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.405 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.405 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:18.405 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:18.406 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:18.406 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:18.406 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.407 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.407 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.410 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.410 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.410 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.410 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.411 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:18.411 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:18.411 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:18.411 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.412 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:18.414 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.414 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.414 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.414 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.414 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.415 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.415 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:18.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:18.415 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:18.416 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:18.416 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.416 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.416 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.416 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.418 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.419 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.419 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:18.419 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:18.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fe4a9640 1 -- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f59d803bdf0 msgr2=0x7f59d803e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fe4a9640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f59d803bdf0 0x7f59d803e2b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f59f0011340 tx=0x7f59f000b040 comp rx=0 tx=0).stop 2024-08-19T10:51:18.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fe4a9640 10 monclient: shutdown 2024-08-19T10:51:18.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fe4a9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:18.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fe4a9640 1 -- 172.21.15.89:0/905317039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f80a8480 msgr2=0x7f59f813c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fe4a9640 1 --2- 172.21.15.89:0/905317039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f59f80a8480 0x7f59f813c720 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f59e001cc00 tx=0x7f59e0019100 comp rx=0 tx=0).stop 2024-08-19T10:51:18.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fca46640 1 -- 172.21.15.89:0/905317039 reap_dead start 2024-08-19T10:51:18.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fca46640 1 -- 172.21.15.89:0/905317039 reap_dead start 2024-08-19T10:51:18.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fe4a9640 1 -- 172.21.15.89:0/905317039 shutdown_connections 2024-08-19T10:51:18.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fe4a9640 1 -- 172.21.15.89:0/905317039 >> 172.21.15.89:0/905317039 conn(0x7f59f80699d0 msgr2=0x7f59f813fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:18.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fe4a9640 1 -- 172.21.15.89:0/905317039 shutdown_connections 2024-08-19T10:51:18.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.380+0000 7f59fe4a9640 1 -- 172.21.15.89:0/905317039 wait complete. 2024-08-19T10:51:18.422 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:18.422 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:51:18.422 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:51:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.536+0000 7f4312acd640 10 monclient: get_monmap_and_config 2024-08-19T10:51:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.536+0000 7f4312acd640 10 monclient: build_initial_monmap 2024-08-19T10:51:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient: monmap: 2024-08-19T10:51:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:18.544980+0000 2024-08-19T10:51:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:18.544980+0000 2024-08-19T10:51:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.539 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:18.540 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:18.540 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 1 Processor -- start 2024-08-19T10:51:18.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 1 -- start start 2024-08-19T10:51:18.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient: init 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient: _add_conns ranks=[5,2,4,3,0,1,10,6,8,9,7] 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f430c137920 0x7f430c137d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient(hunting): picked mon.noname-h con 0x7f430c137920 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430c0743c0 0x7f430c0a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient(hunting): picked mon.noname-b con 0x7f430c0743c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 0x7f430c0a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient(hunting): picked mon.noname-c con 0x7f430c0a51a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f430c073d50 con 0x7f430c0743c0 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f430c072660 con 0x7f430c0a51a0 2024-08-19T10:51:18.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f430c0a6740 con 0x7f430c137920 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4312acd640 10 monclient(hunting): authenticate will time out at 1728.361627s 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430c0743c0 0x7f430c0a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f431106a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 0x7f430c0a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430c0743c0 0x7f430c0a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:35464/0 (socket says 172.21.15.89:35464) 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 1 -- 172.21.15.89:0/3138005193 learned_addr learned my addr 172.21.15.89:0/3138005193 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f431106a640 10 monclient(hunting): get_auth_request con 0x7f430c0a51a0 auth_method 0 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f431106a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f431106a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f431106a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 10 monclient(hunting): get_auth_request con 0x7f430c0743c0 auth_method 0 2024-08-19T10:51:18.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f431106a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f431106a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f431106a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f4310869640 1 --2- 172.21.15.89:0/3138005193 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f430c137920 0x7f430c137d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 10 monclient(hunting): handle_auth_done global_id 74216 payload 274 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 1 -- 172.21.15.89:0/3138005193 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f430c137920 msgr2=0x7f430c137d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 1 --2- 172.21.15.89:0/3138005193 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f430c137920 0x7f430c137d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 1 -- 172.21.15.89:0/3138005193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 msgr2=0x7f430c0a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 1 --2- 172.21.15.89:0/3138005193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 0x7f430c0a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:51:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 1 monclient: found mon.noname-b 2024-08-19T10:51:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:51:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f430bfff640 1 -- 172.21.15.89:0/3138005193 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c0723e0 con 0x7f430c0743c0 2024-08-19T10:51:18.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.540+0000 7f431106a640 1 --2- 172.21.15.89:0/3138005193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 0x7f430c0a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430bfff640 1 -- 172.21.15.89:0/3138005193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430c0743c0 msgr2=0x7f430c0a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430bfff640 1 --2- 172.21.15.89:0/3138005193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430c0743c0 0x7f430c0a4c60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f42f40098a0 tx=0x7f42f401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6f1b54e71a84ca0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 1 -- 172.21.15.89:0/3138005193 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42f40b1070 con 0x7f430c0743c0 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 10 monclient: dump: 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.550 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 1 -- 172.21.15.89:0/3138005193 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f40aa7b0 con 0x7f430c0743c0 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:18.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 1 -- 172.21.15.89:0/3138005193 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42f40aae10 con 0x7f430c0743c0 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f430b7fe640 10 monclient: dump: 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.552 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 5 monclient: authenticate success, global_id 74216 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 4 set_mon_vals no callback set 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 10 monclient: shutdown 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 1 -- 172.21.15.89:0/3138005193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430c0743c0 msgr2=0x7f430c0a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 1 --2- 172.21.15.89:0/3138005193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430c0743c0 0x7f430c0a4c60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f42f40098a0 tx=0x7f42f401c410 comp rx=0 tx=0).stop 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 1 -- 172.21.15.89:0/3138005193 shutdown_connections 2024-08-19T10:51:18.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 1 --2- 172.21.15.89:0/3138005193 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 0x7f430c0a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 1 --2- 172.21.15.89:0/3138005193 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f430c0743c0 0x7f430c0a4c60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 1 --2- 172.21.15.89:0/3138005193 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f430c137920 0x7f430c137d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 1 -- 172.21.15.89:0/3138005193 >> 172.21.15.89:0/3138005193 conn(0x7f430c0aa240 msgr2=0x7f430c0aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:18.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 1 -- 172.21.15.89:0/3138005193 shutdown_connections 2024-08-19T10:51:18.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.544+0000 7f4312acd640 1 -- 172.21.15.89:0/3138005193 wait complete. 2024-08-19T10:51:18.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient: build_initial_monmap 2024-08-19T10:51:18.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient: monmap: 2024-08-19T10:51:18.554 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:18.554 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:18.555 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 1 Processor -- start 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 1 -- start start 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient: init 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient: _add_conns ranks=[3,4,8,2,1,0,6,5,7] 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c0743c0 0x7f430c1d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient(hunting): picked mon.noname-g con 0x7f430c0743c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 0x7f430c1368c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient(hunting): picked mon.noname-c con 0x7f430c0a51a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f430c137920 0x7f430c136e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient(hunting): picked mon.noname-e con 0x7f430c137920 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f430c1383c0 con 0x7f430c0743c0 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f430c138240 con 0x7f430c0a51a0 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f430c141070 con 0x7f430c137920 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4312acd640 10 monclient(hunting): authenticate will time out at 1728.369189s 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 0x7f430c1368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f431106a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f430c137920 0x7f430c136e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 0x7f430c1368c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38162/0 (socket says 172.21.15.89:38162) 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 1 -- 172.21.15.89:0/848775263 learned_addr learned my addr 172.21.15.89:0/848775263 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f431106a640 10 monclient(hunting): get_auth_request con 0x7f430c137920 auth_method 0 2024-08-19T10:51:18.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f431106a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f431106a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f431106a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 10 monclient(hunting): get_auth_request con 0x7f430c0a51a0 auth_method 0 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f431106a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f431106a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f431106a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4310869640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c0743c0 0x7f430c1d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4310869640 10 monclient(hunting): get_auth_request con 0x7f430c0743c0 auth_method 0 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4310869640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4310869640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f4310869640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 10 monclient(hunting): handle_auth_done global_id 45058 payload 824 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 1 -- 172.21.15.89:0/848775263 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f430c137920 msgr2=0x7f430c136e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f430c137920 0x7f430c136e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 1 -- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c0743c0 msgr2=0x7f430c1d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c0743c0 0x7f430c1d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 1 monclient: found mon.noname-c 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:18.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 1 -- 172.21.15.89:0/848775263 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c122c00 con 0x7f430c0a51a0 2024-08-19T10:51:18.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f431106a640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f430c137920 0x7f430c136e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:18.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 1 -- 172.21.15.89:0/848775263 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 msgr2=0x7f430c1368c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.548+0000 7f430bfff640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 0x7f430c1368c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f42f40af040 tx=0x7f42f40191b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42f40ba070 con 0x7f430c0a51a0 2024-08-19T10:51:18.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:51:18.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 10 monclient: dump: 2024-08-19T10:51:18.563 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.563 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.564 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 10 monclient: _finish_auth 0 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 10 monclient: _check_auth_tickets 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f40aadd0 con 0x7f430c0a51a0 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4312acd640 5 monclient: authenticate success, global_id 45058 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4312acd640 10 monclient: _renew_subs 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4312acd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4312acd640 1 -- 172.21.15.89:0/848775263 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430c0aa620 con 0x7f430c0a51a0 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42f40b1040 con 0x7f430c0a51a0 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:18.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 10 monclient: dump: 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.566 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4312acd640 10 monclient: _renew_subs 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f430affd640 4 set_mon_vals no callback set 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4312acd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4312acd640 1 -- 172.21.15.89:0/848775263 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f430c1239e0 con 0x7f430c0a51a0 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4312acd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4312acd640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c121f60 0x7f430c120e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4312acd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4312acd640 1 -- 172.21.15.89:0/848775263 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f430c0a5600 con 0x7f430c121f60 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f42f40aa7b0 con 0x7f430c0a51a0 2024-08-19T10:51:18.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4310869640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c121f60 0x7f430c120e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4310869640 10 monclient: get_auth_request con 0x7f430c121f60 auth_method 0 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4310869640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4310869640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4310869640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42ec03be00 0x7f42ec03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4310869640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4310869640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(273..273 src has 246..273) ==== 7657+0+0 (secure 0 0 0) 0x7f42f40e88a0 con 0x7f430c0a51a0 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4310869640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f431106a640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42ec03be00 0x7f42ec03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f431106a640 10 monclient: get_auth_request con 0x7f42ec03be00 auth_method 0 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4310869640 10 monclient(hunting): handle_auth_done global_id 35661 payload 824 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.552+0000 7f4310869640 1 -- 172.21.15.89:0/848775263 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f430c064590 con 0x7f430c121f60 2024-08-19T10:51:18.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f4310869640 1 -- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c121f60 msgr2=0x7f430c120e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f4310869640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c121f60 0x7f430c120e70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00ec80 tx=0x7f42fc00c640 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f431106a640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42ec03be00 0x7f42ec03e2c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4304009580 tx=0x7f4304009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42fc017020 con 0x7f430c121f60 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f43097fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f43097fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f43097fa640 10 monclient: dump: 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.569 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f42fc003bb0 con 0x7f430c121f60 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f43097fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f43097fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:18.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c121f60 msgr2=0x7f430c120e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f43097fa640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c121f60 0x7f430c120e70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00ec80 tx=0x7f42fc00c640 comp rx=0 tx=0).stop 2024-08-19T10:51:18.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c121f60 msgr2=0x7f430c120e70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.556+0000 7f43097fa640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c121f60 0x7f430c120e70 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00ec80 tx=0x7f42fc00c640 comp rx=0 tx=0).stop 2024-08-19T10:51:18.572 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.572 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1075973324}) 2024-08-19T10:51:18.572 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1691250213}) 2024-08-19T10:51:18.572 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.572 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1075973324}) 2024-08-19T10:51:18.572 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=167945400}) 2024-08-19T10:51:18.572 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.572 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1075973324}) 2024-08-19T10:51:18.572 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3787467703}) 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1075973324}) 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2093485095}) 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1075973324}) 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2874936075}) 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1075973324}) 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2809859714}) 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1075973324}) 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2742637131}) 2024-08-19T10:51:18.573 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.574 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1075973324}) 2024-08-19T10:51:18.574 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.560+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1985113213}) 2024-08-19T10:51:18.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.564+0000 7f4312acd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:18.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.564+0000 7f4312acd640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c02d1d0 0x7f430c02d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.564+0000 7f4312acd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.564+0000 7f4312acd640 1 -- 172.21.15.89:0/848775263 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f430c0a75c0 con 0x7f430c02d1d0 2024-08-19T10:51:18.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.564+0000 7f4310869640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c02d1d0 0x7f430c02d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.564+0000 7f4310869640 10 monclient: get_auth_request con 0x7f430c02d1d0 auth_method 0 2024-08-19T10:51:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.564+0000 7f4310869640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.564+0000 7f4310869640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.564+0000 7f4310869640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.568+0000 7f4310869640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.568+0000 7f4310869640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.568+0000 7f4310869640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.568+0000 7f4310869640 10 monclient(hunting): handle_auth_done global_id 35670 payload 824 2024-08-19T10:51:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.568+0000 7f4310869640 1 -- 172.21.15.89:0/848775263 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f430c0a2aa0 con 0x7f430c02d1d0 2024-08-19T10:51:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.568+0000 7f4310869640 1 -- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c02d1d0 msgr2=0x7f430c02d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.568+0000 7f4310869640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c02d1d0 0x7f430c02d5e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f42fc018000 tx=0x7f42fc004a50 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1506023141}) 2024-08-19T10:51:18.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4175744976}) 2024-08-19T10:51:18.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1506023141}) 2024-08-19T10:51:18.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4254053417}) 2024-08-19T10:51:18.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1506023141}) 2024-08-19T10:51:18.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1048372623}) 2024-08-19T10:51:18.577 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1506023141}) 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1951729577}) 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1506023141}) 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2167571768}) 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1506023141}) 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2017845247}) 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1506023141}) 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4081359377}) 2024-08-19T10:51:18.578 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.579 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1506023141}) 2024-08-19T10:51:18.579 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2560020165}) 2024-08-19T10:51:18.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f42fc017040 con 0x7f430c02d1d0 2024-08-19T10:51:18.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f43097fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f43097fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f43097fa640 10 monclient: dump: 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.580 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.581 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f42fc01e020 con 0x7f430c02d1d0 2024-08-19T10:51:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f43097fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f43097fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c02d1d0 msgr2=0x7f430c02d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f43097fa640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c02d1d0 0x7f430c02d5e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f42fc018000 tx=0x7f42fc004a50 comp rx=0 tx=0).stop 2024-08-19T10:51:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f43097fa640 1 -- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c02d1d0 msgr2=0x7f430c02d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f43097fa640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f430c02d1d0 0x7f430c02d5e0 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f42fc018000 tx=0x7f42fc004a50 comp rx=0 tx=0).stop 2024-08-19T10:51:18.582 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:18.582 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:18.582 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:18.582 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:18.582 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:18.582 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 523621, 2024-08-19T10:51:18.582 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:18.582 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:18.582 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:18.582 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:18.582 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.583 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.584 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:18.585 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:18.586 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.587 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.590 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.591 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.593 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.594 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.595 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.596 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.598 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.599 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:18.602 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.603 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:18.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.605 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:18.606 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f4312acd640 1 -- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42ec03be00 msgr2=0x7f42ec03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f4312acd640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f42ec03be00 0x7f42ec03e2c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4304009580 tx=0x7f4304009190 comp rx=0 tx=0).stop 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f4312acd640 10 monclient: shutdown 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f4312acd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f4312acd640 1 -- 172.21.15.89:0/848775263 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 msgr2=0x7f430c1368c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f4312acd640 1 --2- 172.21.15.89:0/848775263 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f430c0a51a0 0x7f430c1368c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f42f40af040 tx=0x7f42f40191b0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f431106a640 1 -- 172.21.15.89:0/848775263 reap_dead start 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f431106a640 1 -- 172.21.15.89:0/848775263 reap_dead start 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f4312acd640 1 -- 172.21.15.89:0/848775263 shutdown_connections 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f4312acd640 1 -- 172.21.15.89:0/848775263 >> 172.21.15.89:0/848775263 conn(0x7f430c0aa240 msgr2=0x7f430c13de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f4312acd640 1 -- 172.21.15.89:0/848775263 shutdown_connections 2024-08-19T10:51:18.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.572+0000 7f4312acd640 1 -- 172.21.15.89:0/848775263 wait complete. 2024-08-19T10:51:18.610 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.610 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3240898403}) 2024-08-19T10:51:18.610 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=890412817}) 2024-08-19T10:51:18.610 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.610 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3240898403}) 2024-08-19T10:51:18.610 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1729329438}) 2024-08-19T10:51:18.610 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.610 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3240898403}) 2024-08-19T10:51:18.611 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3659285690}) 2024-08-19T10:51:18.611 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.611 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3240898403}) 2024-08-19T10:51:18.611 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=944132155}) 2024-08-19T10:51:18.611 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3240898403}) 2024-08-19T10:51:18.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=23830529}) 2024-08-19T10:51:18.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3240898403}) 2024-08-19T10:51:18.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3162158168}) 2024-08-19T10:51:18.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3240898403}) 2024-08-19T10:51:18.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3779836736}) 2024-08-19T10:51:18.614 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3240898403}) 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.584+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=581808290}) 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784712209}) 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2539724313}) 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784712209}) 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4065826982}) 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784712209}) 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3515342651}) 2024-08-19T10:51:18.615 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784712209}) 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2846664317}) 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784712209}) 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1838746640}) 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784712209}) 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=961003535}) 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784712209}) 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1094723246}) 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:18.616 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784712209}) 2024-08-19T10:51:18.617 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.588+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1951102098}) 2024-08-19T10:51:18.617 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:18.618 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:51:18.618 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:51:18.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient: get_monmap_and_config 2024-08-19T10:51:18.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient: build_initial_monmap 2024-08-19T10:51:18.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient: monmap: 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:18.741590+0000 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:18.741590+0000 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:18.736 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:18.737 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:18.737 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:18.737 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:18.737 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 1 Processor -- start 2024-08-19T10:51:18.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 1 -- start start 2024-08-19T10:51:18.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient: init 2024-08-19T10:51:18.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:18.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient: _add_conns ranks=[8,10,3,0,5,9,6,4,2,1,7] 2024-08-19T10:51:18.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406c930 0x7f70040a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient(hunting): picked mon.noname-e con 0x7f700406c930 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f700406cd10 0x7f700406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient(hunting): picked mon.noname-f con 0x7f700406cd10 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7004139550 0x7f7004139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient(hunting): picked mon.noname-g con 0x7f7004139550 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f70040a0ba0 con 0x7f700406cd10 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f70040a05e0 con 0x7f7004139550 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f70040a0300 con 0x7f700406c930 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7009330640 10 monclient(hunting): authenticate will time out at 1728.557980s 2024-08-19T10:51:18.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406c930 0x7f70040a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406c930 0x7f70040a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48652/0 (socket says 172.21.15.89:48652) 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 1 -- 172.21.15.89:0/2490090941 learned_addr learned my addr 172.21.15.89:0/2490090941 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 10 monclient(hunting): get_auth_request con 0x7f700406c930 auth_method 0 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f70037fe640 1 --2- 172.21.15.89:0/2490090941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7004139550 0x7f7004139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f70027fc640 1 --2- 172.21.15.89:0/2490090941 >> v2:172.21.15.190:3300/0 conn(0x7f700406cd10 0x7f700406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f70037fe640 10 monclient(hunting): get_auth_request con 0x7f7004139550 auth_method 0 2024-08-19T10:51:18.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f70037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f70037fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f70037fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 10 monclient(hunting): handle_auth_done global_id 7145 payload 274 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 1 -- 172.21.15.89:0/2490090941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7004139550 msgr2=0x7f7004139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 1 --2- 172.21.15.89:0/2490090941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7004139550 0x7f7004139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 1 -- 172.21.15.89:0/2490090941 >> v2:172.21.15.190:3300/0 conn(0x7f700406cd10 msgr2=0x7f700406d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 1 --2- 172.21.15.89:0/2490090941 >> v2:172.21.15.190:3300/0 conn(0x7f700406cd10 0x7f700406d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 1 monclient: found mon.noname-e 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:51:18.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.736+0000 7f7002ffd640 1 -- 172.21.15.89:0/2490090941 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70040a08c0 con 0x7f700406c930 2024-08-19T10:51:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7002ffd640 1 -- 172.21.15.89:0/2490090941 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406c930 msgr2=0x7f70040a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7002ffd640 1 --2- 172.21.15.89:0/2490090941 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406c930 0x7f70040a0fc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80098a0 tx=0x7f6ff801c400 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1a9fa0636137a439 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 1 -- 172.21.15.89:0/2490090941 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ff80b0070 con 0x7f700406c930 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 10 monclient: dump: 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.746 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 10 monclient: _finish_auth 0 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 1 -- 172.21.15.89:0/2490090941 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff80aa810 con 0x7f700406c930 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:18.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 1 -- 172.21.15.89:0/2490090941 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ff80aae30 con 0x7f700406c930 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 5 monclient: authenticate success, global_id 7145 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 4 set_mon_vals no callback set 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7001ffb640 10 monclient: dump: 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.748 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 10 monclient: shutdown 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 1 -- 172.21.15.89:0/2490090941 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406c930 msgr2=0x7f70040a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 1 --2- 172.21.15.89:0/2490090941 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406c930 0x7f70040a0fc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80098a0 tx=0x7f6ff801c400 comp rx=0 tx=0).stop 2024-08-19T10:51:18.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 1 -- 172.21.15.89:0/2490090941 shutdown_connections 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 1 --2- 172.21.15.89:0/2490090941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7004139550 0x7f7004139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 1 --2- 172.21.15.89:0/2490090941 >> v2:172.21.15.190:3300/0 conn(0x7f700406cd10 0x7f700406d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 1 --2- 172.21.15.89:0/2490090941 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406c930 0x7f70040a0fc0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 1 -- 172.21.15.89:0/2490090941 >> 172.21.15.89:0/2490090941 conn(0x7f70040a83d0 msgr2=0x7f70040a87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 1 -- 172.21.15.89:0/2490090941 shutdown_connections 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.740+0000 7f7009330640 1 -- 172.21.15.89:0/2490090941 wait complete. 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient: build_initial_monmap 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient: monmap: 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:18.750 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 1 Processor -- start 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 1 -- start start 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient: init 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:18.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient: _add_conns ranks=[6,8,4,5,2,1,0,3,7] 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f700406c930 0x7f70041387a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient(hunting): picked mon.noname-d con 0x7f700406c930 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406cd10 0x7f7004138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient(hunting): picked mon.noname-e con 0x7f700406cd10 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7004139550 0x7f700413bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient(hunting): picked mon.noname-c con 0x7f7004139550 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f700406d630 con 0x7f7004139550 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f70041431d0 con 0x7f700406c930 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7004077650 con 0x7f700406cd10 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:18.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7009330640 10 monclient(hunting): authenticate will time out at 1728.565148s 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70027fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406cd10 0x7f7004138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7004139550 0x7f700413bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7002ffd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f700406c930 0x7f70041387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7004139550 0x7f700413bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38172/0 (socket says 172.21.15.89:38172) 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 1 -- 172.21.15.89:0/740957964 learned_addr learned my addr 172.21.15.89:0/740957964 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 10 monclient(hunting): get_auth_request con 0x7f7004139550 auth_method 0 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70027fc640 10 monclient(hunting): get_auth_request con 0x7f700406cd10 auth_method 0 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70027fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70027fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7002ffd640 10 monclient(hunting): get_auth_request con 0x7f700406c930 auth_method 0 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7002ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7002ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7002ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70027fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70027fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70027fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 10 monclient(hunting): handle_auth_done global_id 45067 payload 824 2024-08-19T10:51:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 1 -- 172.21.15.89:0/740957964 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406cd10 msgr2=0x7f7004138ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f700406cd10 0x7f7004138ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 1 -- 172.21.15.89:0/740957964 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f700406c930 msgr2=0x7f70041387a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f700406c930 0x7f70041387a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 10 monclient: _finish_hunting 0 2024-08-19T10:51:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 1 monclient: found mon.noname-c 2024-08-19T10:51:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.744+0000 7f70037fe640 1 -- 172.21.15.89:0/740957964 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700413f690 con 0x7f7004139550 2024-08-19T10:51:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f70037fe640 1 -- 172.21.15.89:0/740957964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7004139550 msgr2=0x7f700413bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f70037fe640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7004139550 0x7f700413bd10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc007e60 tx=0x7f6ffc00e4b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ffc014890 con 0x7f7004139550 2024-08-19T10:51:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:51:18.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 10 monclient: dump: 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.759 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ffc019070 con 0x7f7004139550 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7009330640 5 monclient: authenticate success, global_id 45067 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7009330640 10 monclient: _renew_subs 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7009330640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7009330640 1 -- 172.21.15.89:0/740957964 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70040a1210 con 0x7f7004139550 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ffc013380 con 0x7f7004139550 2024-08-19T10:51:18.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 10 monclient: dump: 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.761 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f70017fa640 4 set_mon_vals no callback set 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7009330640 10 monclient: _renew_subs 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7009330640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7009330640 1 -- 172.21.15.89:0/740957964 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f700409afe0 con 0x7f7004139550 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7009330640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7009330640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413c250 0x7f700413c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7009330640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7009330640 1 -- 172.21.15.89:0/740957964 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f700406d7b0 con 0x7f700413c250 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6ffc014420 con 0x7f7004139550 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ff003be00 0x7f6ff003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7002ffd640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413c250 0x7f700413c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f6ffc04fcc0 con 0x7f7004139550 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7002ffd640 10 monclient: get_auth_request con 0x7f700413c250 auth_method 0 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7002ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7002ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7002ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f70027fc640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ff003be00 0x7f6ff003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f70027fc640 10 monclient: get_auth_request con 0x7f6ff003be00 auth_method 0 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7002ffd640 10 monclient(hunting): handle_auth_done global_id 7187 payload 824 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.748+0000 7f7002ffd640 1 -- 172.21.15.89:0/740957964 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f700406d0f0 con 0x7f700413c250 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f70027fc640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ff003be00 0x7f6ff003e2c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f700413dc80 tx=0x7f6ff4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f7002ffd640 1 -- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413c250 msgr2=0x7f700413c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f7002ffd640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413c250 0x7f700413c630 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f70040695c0 tx=0x7f6ff80191a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ff80ba070 con 0x7f700413c250 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f6feb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f6feb7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f6feb7fe640 10 monclient: dump: 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.764 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6ff80b1040 con 0x7f700413c250 2024-08-19T10:51:18.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f6feb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f6feb7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:18.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413c250 msgr2=0x7f700413c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f6feb7fe640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413c250 0x7f700413c630 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f70040695c0 tx=0x7f6ff80191a0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413c250 msgr2=0x7f700413c630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.752+0000 7f6feb7fe640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413c250 0x7f700413c630 secure :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f70040695c0 tx=0x7f6ff80191a0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7009330640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:18.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7009330640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413cb70 0x7f7004071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7009330640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7009330640 1 -- 172.21.15.89:0/740957964 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f700413f0d0 con 0x7f700413cb70 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413cb70 0x7f7004071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 10 monclient: get_auth_request con 0x7f700413cb70 auth_method 0 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 10 monclient(hunting): handle_auth_done global_id 7196 payload 824 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 1 -- 172.21.15.89:0/740957964 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7004064590 con 0x7f700413cb70 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 1 -- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413cb70 msgr2=0x7f7004071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.760+0000 7f7002ffd640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413cb70 0x7f7004071d90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6ff801c230 tx=0x7f6ff80a8000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6ff80b0070 con 0x7f700413cb70 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f6feb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f6feb7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f6feb7fe640 10 monclient: dump: 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.769 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f6ff80c8410 con 0x7f700413cb70 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f6feb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f6feb7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413cb70 msgr2=0x7f7004071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f6feb7fe640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413cb70 0x7f7004071d90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6ff801c230 tx=0x7f6ff80a8000 comp rx=0 tx=0).stop 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f6feb7fe640 1 -- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413cb70 msgr2=0x7f7004071d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f6feb7fe640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f700413cb70 0x7f7004071d90 secure :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6ff801c230 tx=0x7f6ff80a8000 comp rx=0 tx=0).stop 2024-08-19T10:51:18.771 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:18.771 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:18.771 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:18.771 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:18.771 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:18.771 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 808539, 2024-08-19T10:51:18.771 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:18.771 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:18.771 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:18.771 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:18.771 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.772 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.773 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:18.774 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:18.775 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.776 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.777 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.779 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.780 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:18.781 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.782 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:18.782 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.782 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.782 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.782 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.782 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.786 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.787 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:18.788 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.789 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.789 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:18.789 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.789 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:18.789 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.789 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.790 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.791 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.793 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.793 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:18.793 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.793 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.793 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.793 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.793 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.793 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.793 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.794 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:51:18.795 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.796 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.796 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:18.796 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:18.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f7009330640 1 -- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ff003be00 msgr2=0x7f6ff003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f7009330640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6ff003be00 0x7f6ff003e2c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f700413dc80 tx=0x7f6ff4008040 comp rx=0 tx=0).stop 2024-08-19T10:51:18.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f7009330640 10 monclient: shutdown 2024-08-19T10:51:18.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f7009330640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:18.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f7009330640 1 -- 172.21.15.89:0/740957964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7004139550 msgr2=0x7f700413bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f7009330640 1 --2- 172.21.15.89:0/740957964 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7004139550 0x7f700413bd10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc007e60 tx=0x7f6ffc00e4b0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f70037fe640 1 -- 172.21.15.89:0/740957964 reap_dead start 2024-08-19T10:51:18.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f70037fe640 1 -- 172.21.15.89:0/740957964 reap_dead start 2024-08-19T10:51:18.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f7009330640 1 -- 172.21.15.89:0/740957964 shutdown_connections 2024-08-19T10:51:18.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f7009330640 1 -- 172.21.15.89:0/740957964 >> 172.21.15.89:0/740957964 conn(0x7f70040a83d0 msgr2=0x7f70040a2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:18.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f7009330640 1 -- 172.21.15.89:0/740957964 shutdown_connections 2024-08-19T10:51:18.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.764+0000 7f7009330640 1 -- 172.21.15.89:0/740957964 wait complete. 2024-08-19T10:51:18.798 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:18.798 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:51:18.799 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:51:18.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient: get_monmap_and_config 2024-08-19T10:51:18.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient: build_initial_monmap 2024-08-19T10:51:18.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient: monmap: 2024-08-19T10:51:18.915 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:18.915 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.915 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:18.921037+0000 2024-08-19T10:51:18.915 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:18.921037+0000 2024-08-19T10:51:18.916 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:18.916 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.916 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:18.916 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:18.916 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:18.916 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:18.916 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:18.916 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:18.916 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:18.917 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:18.917 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:18.917 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:18.917 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:18.917 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 1 Processor -- start 2024-08-19T10:51:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 1 -- start start 2024-08-19T10:51:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient: init 2024-08-19T10:51:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient: _add_conns ranks=[4,1,8,5,3,6,10,0,7,9,2] 2024-08-19T10:51:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa84806de10 0x7fa848073f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient(hunting): picked mon.noname-c con 0x7fa84806de10 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa848074440 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa848139910 0x7fa8481415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient(hunting): picked mon.noname-e con 0x7fa848139910 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa848136730 con 0x7fa848074440 2024-08-19T10:51:18.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa848135c80 con 0x7fa84806de10 2024-08-19T10:51:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa84806e1f0 con 0x7fa848139910 2024-08-19T10:51:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84ca1b640 10 monclient(hunting): authenticate will time out at 1728.737579s 2024-08-19T10:51:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa84806de10 0x7fa848073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa846d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa848139910 0x7fa8481415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa846d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa848139910 0x7fa8481415a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:48674/0 (socket says 172.21.15.89:48674) 2024-08-19T10:51:18.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa84806de10 0x7fa848073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38182/0 (socket says 172.21.15.89:38182) 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa846d9d640 1 -- 172.21.15.89:0/1780759664 learned_addr learned my addr 172.21.15.89:0/1780759664 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa846d9d640 10 monclient(hunting): get_auth_request con 0x7fa848139910 auth_method 0 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa846d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa846d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa846d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 10 monclient(hunting): get_auth_request con 0x7fa84806de10 auth_method 0 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa846d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa846d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa846d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 10 monclient(hunting): handle_auth_done global_id 45076 payload 274 2024-08-19T10:51:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 1 -- 172.21.15.89:0/1780759664 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa848139910 msgr2=0x7fa8481415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 1 --2- 172.21.15.89:0/1780759664 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa848139910 0x7fa8481415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 1 -- 172.21.15.89:0/1780759664 >> v1:172.21.15.190:6789/0 conn(0x7fa848074440 legacy=0x7fa848139200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 10 monclient: _finish_hunting 0 2024-08-19T10:51:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 1 monclient: found mon.noname-c 2024-08-19T10:51:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa84659c640 1 -- 172.21.15.89:0/1780759664 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84806f190 con 0x7fa84806de10 2024-08-19T10:51:18.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.916+0000 7fa846d9d640 1 --2- 172.21.15.89:0/1780759664 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa848139910 0x7fa8481415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:18.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84659c640 1 -- 172.21.15.89:0/1780759664 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa84806de10 msgr2=0x7fa848073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84659c640 1 --2- 172.21.15.89:0/1780759664 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa84806de10 0x7fa848073f00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8300097f0 tx=0x7fa83001c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d411e98ef7b51a5b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 1 -- 172.21.15.89:0/1780759664 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8300b0070 con 0x7fa84806de10 2024-08-19T10:51:18.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:51:18.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 10 monclient: dump: 2024-08-19T10:51:18.930 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.930 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.931 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 10 monclient: _finish_auth 0 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 10 monclient: _check_auth_tickets 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 5 monclient: authenticate success, global_id 45076 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 1 -- 172.21.15.89:0/1780759664 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8300aa810 con 0x7fa84806de10 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 4 set_mon_vals no callback set 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 1 -- 172.21.15.89:0/1780759664 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8300a4e60 con 0x7fa84806de10 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84559a640 10 monclient: dump: 2024-08-19T10:51:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.933 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.934 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.935 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.935 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.935 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.935 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.935 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.935 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 10 monclient: shutdown 2024-08-19T10:51:18.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:18.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 1 -- 172.21.15.89:0/1780759664 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa84806de10 msgr2=0x7fa848073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 1 --2- 172.21.15.89:0/1780759664 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa84806de10 0x7fa848073f00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8300097f0 tx=0x7fa83001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 1 -- 172.21.15.89:0/1780759664 shutdown_connections 2024-08-19T10:51:18.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 1 --2- 172.21.15.89:0/1780759664 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa848139910 0x7fa8481415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.936 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 1 --2- 172.21.15.89:0/1780759664 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa84806de10 0x7fa848073f00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 1 -- 172.21.15.89:0/1780759664 >> 172.21.15.89:0/1780759664 conn(0x7fa8480aa240 msgr2=0x7fa8480aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:18.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 1 -- 172.21.15.89:0/1780759664 shutdown_connections 2024-08-19T10:51:18.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 1 -- 172.21.15.89:0/1780759664 wait complete. 2024-08-19T10:51:18.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 10 monclient: build_initial_monmap 2024-08-19T10:51:18.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.920+0000 7fa84ca1b640 10 monclient: monmap: 2024-08-19T10:51:18.938 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:18.938 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:18.938 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:18.938 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:18.938 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:18.938 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.938 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:18.938 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:18.938 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:18.938 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:18.938 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:18.939 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:18.939 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:18.939 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:18.939 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:18.939 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 1 Processor -- start 2024-08-19T10:51:18.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 1 -- start start 2024-08-19T10:51:18.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient: init 2024-08-19T10:51:18.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient: _add_conns ranks=[3,7,4,8,6,5,1,0,2] 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa84806de10 0x7fa84812fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient(hunting): picked mon.noname-g con 0x7fa84806de10 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848074440 0x7fa8481307a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient(hunting): picked mon.noname-i con 0x7fa848074440 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa848139910 0x7fa848130ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient(hunting): picked mon.noname-c con 0x7fa848139910 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa8480a5e80 con 0x7fa84806de10 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa8480a5d00 con 0x7fa848139910 2024-08-19T10:51:18.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa848144300 con 0x7fa848074440 2024-08-19T10:51:18.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:18.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient(hunting): authenticate will time out at 1728.744960s 2024-08-19T10:51:18.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa848139910 0x7fa848130ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa848139910 0x7fa848130ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:38190/0 (socket says 172.21.15.89:38190) 2024-08-19T10:51:18.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 1 -- 172.21.15.89:0/2914093583 learned_addr learned my addr 172.21.15.89:0/2914093583 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:18.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 10 monclient(hunting): get_auth_request con 0x7fa848139910 auth_method 0 2024-08-19T10:51:18.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa845d9b640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848074440 0x7fa8481307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa845d9b640 10 monclient(hunting): get_auth_request con 0x7fa848074440 auth_method 0 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa845d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa845d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa845d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 10 monclient(hunting): handle_auth_done global_id 45085 payload 824 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 1 -- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848074440 msgr2=0x7fa8481307a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848074440 0x7fa8481307a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84659c640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa84806de10 0x7fa84812fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 1 -- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa84806de10 msgr2=0x7fa84812fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa84806de10 0x7fa84812fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:51:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 1 monclient: found mon.noname-c 2024-08-19T10:51:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 1 -- 172.21.15.89:0/2914093583 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84813a8d0 con 0x7fa848139910 2024-08-19T10:51:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 1 -- 172.21.15.89:0/2914093583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa848139910 msgr2=0x7fa848130ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa846d9d640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa848139910 0x7fa848130ce0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa84000efc0 tx=0x7fa84000c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa84000cdf0 con 0x7fa848139910 2024-08-19T10:51:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa83effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa83effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:51:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa83effd640 10 monclient: dump: 2024-08-19T10:51:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.948 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.949 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa83effd640 10 monclient: _finish_auth 0 2024-08-19T10:51:18.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa83effd640 10 monclient: _check_auth_tickets 2024-08-19T10:51:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa83effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8400148f0 con 0x7fa848139910 2024-08-19T10:51:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 5 monclient: authenticate success, global_id 45085 2024-08-19T10:51:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient: _renew_subs 2024-08-19T10:51:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa84ca1b640 1 -- 172.21.15.89:0/2914093583 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84813abb0 con 0x7fa848139910 2024-08-19T10:51:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.924+0000 7fa83effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8400073f0 con 0x7fa848139910 2024-08-19T10:51:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa83effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa83effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:51:18.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa844d99640 4 set_mon_vals no callback set 2024-08-19T10:51:18.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa83effd640 10 monclient: dump: 2024-08-19T10:51:18.951 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.951 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.951 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.951 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.951 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.951 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.951 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.951 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.951 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84ca1b640 10 monclient: _renew_subs 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84ca1b640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84ca1b640 1 -- 172.21.15.89:0/2914093583 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fa848139190 con 0x7fa848139910 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84ca1b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84ca1b640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848130110 0x7fa848131220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84ca1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84ca1b640 1 -- 172.21.15.89:0/2914093583 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa8481414c0 con 0x7fa848130110 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa84001b070 con 0x7fa848139910 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84659c640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848130110 0x7fa848131220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84659c640 10 monclient: get_auth_request con 0x7fa848130110 auth_method 0 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa83effd640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa82803be00 0x7fa82803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fa840019020 con 0x7fa848139910 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84659c640 10 monclient(hunting): handle_auth_done global_id 64366 payload 824 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa84659c640 1 -- 172.21.15.89:0/2914093583 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa848064590 con 0x7fa848130110 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa845d9b640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa82803be00 0x7fa82803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.928+0000 7fa845d9b640 10 monclient: get_auth_request con 0x7fa82803be00 auth_method 0 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa84659c640 1 -- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848130110 msgr2=0x7fa848131220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa84659c640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848130110 0x7fa848131220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8300ad000 tx=0x7fa830019140 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8300b9070 con 0x7fa848130110 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa83effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa83effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa83effd640 10 monclient: dump: 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.954 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa845d9b640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa82803be00 0x7fa82803e2c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa8480a4430 tx=0x7fa838009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa8300a4e60 con 0x7fa848130110 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa83effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa83effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848130110 msgr2=0x7fa848131220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa83effd640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848130110 0x7fa848131220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8300ad000 tx=0x7fa830019140 comp rx=0 tx=0).stop 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848130110 msgr2=0x7fa848131220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.932+0000 7fa83effd640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848130110 0x7fa848131220 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8300ad000 tx=0x7fa830019140 comp rx=0 tx=0).stop 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.936+0000 7fa84ca1b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.936+0000 7fa84ca1b640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848131760 0x7fa848136dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.936+0000 7fa84ca1b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.936+0000 7fa84ca1b640 1 -- 172.21.15.89:0/2914093583 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa848140a20 con 0x7fa848131760 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848131760 0x7fa848136dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 10 monclient: get_auth_request con 0x7fa848131760 auth_method 0 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 10 monclient(hunting): handle_auth_done global_id 64375 payload 824 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 1 -- 172.21.15.89:0/2914093583 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa848131b40 con 0x7fa848131760 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 1 -- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848131760 msgr2=0x7fa848136dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa84659c640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848131760 0x7fa848136dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa83001c1d0 tx=0x7fa8300a8000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa8300b0040 con 0x7fa848131760 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa83effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa83effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.940+0000 7fa83effd640 10 monclient: dump: 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:18.958 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:18.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7fa8300c2070 con 0x7fa848131760 2024-08-19T10:51:18.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa83effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:18.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa83effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:18.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848131760 msgr2=0x7fa848136dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa83effd640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848131760 0x7fa848136dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa83001c1d0 tx=0x7fa8300a8000 comp rx=0 tx=0).stop 2024-08-19T10:51:18.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa83effd640 1 -- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848131760 msgr2=0x7fa848136dd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:18.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa83effd640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa848131760 0x7fa848136dd0 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa83001c1d0 tx=0x7fa8300a8000 comp rx=0 tx=0).stop 2024-08-19T10:51:18.959 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:18.959 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:18.959 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:18.960 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:18.960 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 110, 2024-08-19T10:51:18.960 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 145341, 2024-08-19T10:51:18.960 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:18.960 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:18.960 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:18.960 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:18.960 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 7, 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.961 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.962 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.962 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.962 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.962 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.962 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.962 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.962 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.962 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.962 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.963 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:18.964 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:18.965 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:18.966 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:18.966 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:18.966 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:18.966 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:18.966 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.966 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:18.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.966 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:18.966 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.966 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:18.966 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.968 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.968 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.968 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.968 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:18.968 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.968 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.969 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.969 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.969 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:18.969 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:18.969 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.970 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.970 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.970 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.970 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:18.970 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.971 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.971 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.971 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:18.971 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.971 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.971 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.971 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.972 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.972 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:18.972 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.972 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.972 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.973 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:18.973 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:18.973 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.973 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.974 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.974 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:18.974 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.974 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.975 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.975 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:18.975 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.975 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.975 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.976 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.976 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:18.976 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.976 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.976 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.977 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:18.977 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:18.977 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.977 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.978 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.978 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:18.978 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.979 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.979 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:18.979 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.979 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.979 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.980 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.980 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:18.980 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.980 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.980 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.981 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:18.981 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:18.981 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.981 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.982 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.982 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:18.982 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:18.983 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.983 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.983 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.984 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.984 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:18.984 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.984 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.984 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.984 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:18.985 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:18.986 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.989 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.990 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.991 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:18.992 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:18.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:18.993 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:18.993 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa84ca1b640 1 -- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa82803be00 msgr2=0x7fa82803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa84ca1b640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa82803be00 0x7fa82803e2c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa8480a4430 tx=0x7fa838009040 comp rx=0 tx=0).stop 2024-08-19T10:51:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa84ca1b640 10 monclient: shutdown 2024-08-19T10:51:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa84ca1b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa84ca1b640 1 -- 172.21.15.89:0/2914093583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa848139910 msgr2=0x7fa848130ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa84ca1b640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa848139910 0x7fa848130ce0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa84000efc0 tx=0x7fa84000c490 comp rx=0 tx=0).stop 2024-08-19T10:51:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa846d9d640 1 -- 172.21.15.89:0/2914093583 reap_dead start 2024-08-19T10:51:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa84ca1b640 1 -- 172.21.15.89:0/2914093583 shutdown_connections 2024-08-19T10:51:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa84ca1b640 1 --2- 172.21.15.89:0/2914093583 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa848139910 0x7fa848130ce0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa84ca1b640 1 -- 172.21.15.89:0/2914093583 >> 172.21.15.89:0/2914093583 conn(0x7fa8480aa240 msgr2=0x7fa8480732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa84ca1b640 1 -- 172.21.15.89:0/2914093583 shutdown_connections 2024-08-19T10:51:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:18.944+0000 7fa84ca1b640 1 -- 172.21.15.89:0/2914093583 wait complete. 2024-08-19T10:51:18.996 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:51:18.996 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['e', 'h', 'c'] 2024-08-19T10:51:18.996 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a'] 2024-08-19T10:51:18.996 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-08-19T10:51:18.996 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-08-19T10:51:18.997 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-08-19T10:51:18.997 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:51:19.014 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.014 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3478679172}) 2024-08-19T10:51:19.014 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=353476574}) 2024-08-19T10:51:19.014 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.014 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3478679172}) 2024-08-19T10:51:19.014 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1695560984}) 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3478679172}) 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3977752684}) 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3478679172}) 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2876564683}) 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3478679172}) 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2815555154}) 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3478679172}) 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=120082490}) 2024-08-19T10:51:19.015 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3478679172}) 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2674717300}) 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3478679172}) 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.960+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1454765318}) 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4281154045}) 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2843122777}) 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4281154045}) 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2609343564}) 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.016 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4281154045}) 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1705217925}) 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4281154045}) 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4260751585}) 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4281154045}) 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2644380011}) 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4281154045}) 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1541616554}) 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4281154045}) 2024-08-19T10:51:19.017 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1265414838}) 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4281154045}) 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.964+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1593770639}) 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328028780}) 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3375211240}) 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328028780}) 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=498297328}) 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328028780}) 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1051770749}) 2024-08-19T10:51:19.018 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.019 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328028780}) 2024-08-19T10:51:19.022 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3689941891}) 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328028780}) 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2038091003}) 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328028780}) 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2419589933}) 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328028780}) 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3016049771}) 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1328028780}) 2024-08-19T10:51:19.023 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.968+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3796418734}) 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2303774022}) 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=854611102}) 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2303774022}) 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=137871295}) 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2303774022}) 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=813929015}) 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2303774022}) 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3207719080}) 2024-08-19T10:51:19.024 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2303774022}) 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=483181039}) 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2303774022}) 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2615292277}) 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2303774022}) 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2369049845}) 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2303774022}) 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2562226318}) 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.025 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746894457}) 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1923245524}) 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746894457}) 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1085640593}) 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746894457}) 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3629459198}) 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746894457}) 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3524372578}) 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.026 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746894457}) 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2100727664}) 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746894457}) 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1678080470}) 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746894457}) 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=703138388}) 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3746894457}) 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.972+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3567424308}) 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751653942}) 2024-08-19T10:51:19.027 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=769882231}) 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751653942}) 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3972068269}) 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751653942}) 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3976198363}) 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751653942}) 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=651275487}) 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751653942}) 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2964005960}) 2024-08-19T10:51:19.028 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751653942}) 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=755099447}) 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751653942}) 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3795934320}) 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751653942}) 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.976+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3189935417}) 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707571338}) 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=794239165}) 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.029 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707571338}) 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2931602814}) 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707571338}) 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2985308909}) 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707571338}) 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2489464418}) 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707571338}) 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=626743248}) 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.030 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707571338}) 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4264967628}) 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707571338}) 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1060170118}) 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707571338}) 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=847073202}) 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3304617437}) 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1046469498}) 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3304617437}) 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3316022765}) 2024-08-19T10:51:19.031 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3304617437}) 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2423458051}) 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3304617437}) 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2556260675}) 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3304617437}) 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1960009567}) 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3304617437}) 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1070675614}) 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.032 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3304617437}) 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1128290237}) 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3304617437}) 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.980+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3175597975}) 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1157645838}) 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3012383429}) 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1157645838}) 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1324127623}) 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1157645838}) 2024-08-19T10:51:19.033 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1965738531}) 2024-08-19T10:51:19.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1157645838}) 2024-08-19T10:51:19.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3071279019}) 2024-08-19T10:51:19.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1157645838}) 2024-08-19T10:51:19.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=492478039}) 2024-08-19T10:51:19.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1157645838}) 2024-08-19T10:51:19.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2108960313}) 2024-08-19T10:51:19.034 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1157645838}) 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=64490851}) 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1157645838}) 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.984+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=607451099}) 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4221573750}) 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1930506272}) 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4221573750}) 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3507240572}) 2024-08-19T10:51:19.035 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4221573750}) 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1937074379}) 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4221573750}) 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1596145594}) 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4221573750}) 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2962561358}) 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4221573750}) 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3480600709}) 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.036 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4221573750}) 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1998200774}) 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4221573750}) 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:18.988+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=796508158}) 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2786895221}) 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3759082267}) 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2786895221}) 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4108426074}) 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2786895221}) 2024-08-19T10:51:19.037 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=597447376}) 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2786895221}) 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2054611263}) 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2786895221}) 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1427192536}) 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2786895221}) 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1106526450}) 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2786895221}) 2024-08-19T10:51:19.038 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2553190541}) 2024-08-19T10:51:19.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:51:19.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2786895221}) 2024-08-19T10:51:19.039 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:51:19.000+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2071705382}) 2024-08-19T10:51:19.040 INFO:tasks.ceph.mon.e:Stopped 2024-08-19T10:51:19.040 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h 2024-08-19T10:51:19.040 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-08-19T10:51:19.040 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-08-19T10:51:19.040 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:51:19.070 INFO:tasks.ceph.mon.h:Stopped 2024-08-19T10:51:19.070 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-08-19T10:51:19.070 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-08-19T10:51:19.070 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-08-19T10:51:19.070 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:51:19.103 INFO:tasks.ceph.mon.c:Stopped 2024-08-19T10:51:19.103 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-08-19T10:51:19.103 INFO:tasks.ceph.mon.a:Sent signal 19 2024-08-19T10:51:19.103 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-08-19T10:51:34.104 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-08-19T10:51:34.104 INFO:tasks.ceph.mon.a:Sent signal 18 2024-08-19T10:51:34.104 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-08-19T10:51:34.105 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:51:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient: get_monmap_and_config 2024-08-19T10:51:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient: build_initial_monmap 2024-08-19T10:51:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient: monmap: 2024-08-19T10:51:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:34.238313+0000 2024-08-19T10:51:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:34.238313+0000 2024-08-19T10:51:34.232 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:34.233 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:34.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 1 Processor -- start 2024-08-19T10:51:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 1 -- start start 2024-08-19T10:51:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient: init 2024-08-19T10:51:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient: _add_conns ranks=[8,7,10,2,4,6,0,5,1,3,9] 2024-08-19T10:51:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55380a5270 0x7f55380a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient(hunting): picked mon.noname-e con 0x7f55380a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient(hunting): picked mon.noname-i con 0x7f55380a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient(hunting): picked mon.noname-f con 0x7f55381395b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:34.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f55380a7650 con 0x7f55381395b0 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f55380a8710 con 0x7f55380a6d30 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f55380a5a40 con 0x7f55380a5270 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f553e1d5640 10 monclient(hunting): authenticate will time out at 1744.054837s 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f55377fe640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55380a5270 msgr2=0x7f55380a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f55377fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55380a5270 0x7f55380a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5536ffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5537fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:34.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5536ffd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:60086/0 (socket says 172.21.15.89:60086) 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 learned_addr learned my addr 172.21.15.89:0/3249549828 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5536ffd640 10 monclient(hunting): get_auth_request con 0x7f55380a6d30 auth_method 0 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5536ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5536ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5536ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5537fff640 10 monclient(hunting): get_auth_request con 0x7f55381395b0 auth_method 0 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5537fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5537fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 msgr2=0x7f5538139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 msgr2=0x7f55380a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:34.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 msgr2=0x7f5538139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:34.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 msgr2=0x7f55380a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:34.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:34.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:34.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:34.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.236+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:34.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55380a5270 msgr2=0x7f55380a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:51:34.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55380a5270 0x7f55380a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5537fff640 10 monclient(hunting): get_auth_request con 0x7f55381395b0 auth_method 0 2024-08-19T10:51:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5537fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5537fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5536ffd640 10 monclient(hunting): get_auth_request con 0x7f55380a6d30 auth_method 0 2024-08-19T10:51:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5536ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:34.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5536ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5536ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 msgr2=0x7f5538139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 msgr2=0x7f5538139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 msgr2=0x7f55380a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 msgr2=0x7f55380a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:34.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.440+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:34.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55380a5270 msgr2=0x7f55380a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55380a5270 0x7f55380a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5537fff640 10 monclient(hunting): get_auth_request con 0x7f55381395b0 auth_method 0 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5537fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5537fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5536ffd640 10 monclient(hunting): get_auth_request con 0x7f55380a6d30 auth_method 0 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5536ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5536ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5536ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 msgr2=0x7f5538139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:34.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 msgr2=0x7f5538139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:34.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:34.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:34.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 msgr2=0x7f55380a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:34.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 msgr2=0x7f55380a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:34.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:34.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:34.844+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:35.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55380a5270 msgr2=0x7f55380a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:51:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55380a5270 0x7f55380a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:51:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5537fff640 10 monclient(hunting): get_auth_request con 0x7f55381395b0 auth_method 0 2024-08-19T10:51:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5537fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5537fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5536ffd640 10 monclient(hunting): get_auth_request con 0x7f55380a6d30 auth_method 0 2024-08-19T10:51:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5536ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:35.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5536ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:35.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5536ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:35.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 msgr2=0x7f5538139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:35.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 msgr2=0x7f5538139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:35.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:35.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 msgr2=0x7f55380a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:35.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 msgr2=0x7f55380a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:35.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:35.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:51:35.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:35.648+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:51:37.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 10 monclient(hunting): tick 2024-08-19T10:51:37.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:51:37.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 monclient(hunting): continuing hunt 2024-08-19T10:51:37.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:51:37.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55380a5270 msgr2=0x7f55380a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f55380a5270 0x7f55380a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 msgr2=0x7f55380a7110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f55380a6d30 0x7f55380a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 msgr2=0x7f5538139a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.190:3300/0 conn(0x7f55381395b0 0x7f5538139a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 10 monclient: _add_conns ranks=[2,3,9,0,5,4,6,1] 2024-08-19T10:51:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 10 monclient(hunting): picked mon.noname-b con 0x7f5524003820 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 10 monclient(hunting): picked mon.noname-g con 0x7f5524006230 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:37.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 10 monclient(hunting): picked mon.noname-a con 0x7f5524008d90 addr v2:172.21.15.89:3300/0 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f55380a7650 con 0x7f5524008d90 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f55380a8710 con 0x7f5524003820 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f55380a5a40 con 0x7f5524006230 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55357fa640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55377fe640 10 monclient(hunting): get_auth_request con 0x7f5524003820 auth_method 0 2024-08-19T10:51:37.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55377fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55377fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5537fff640 10 monclient(hunting): get_auth_request con 0x7f5524008d90 auth_method 0 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5537fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5537fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5536ffd640 10 monclient(hunting): get_auth_request con 0x7f5524006230 auth_method 0 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5536ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5536ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5536ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 msgr2=0x7f552400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:51:37.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 msgr2=0x7f552400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:37.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:37.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 msgr2=0x7f5524005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:37.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 msgr2=0x7f5524005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:37.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.236+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:37.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.240+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:37.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.240+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 msgr2=0x7f55240086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:37.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.240+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 msgr2=0x7f55240086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:37.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.240+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:37.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.240+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:37.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.240+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:37.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f55377fe640 10 monclient(hunting): get_auth_request con 0x7f5524003820 auth_method 0 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f55377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f55377fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f55377fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5537fff640 10 monclient(hunting): get_auth_request con 0x7f5524008d90 auth_method 0 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5537fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5537fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 msgr2=0x7f5524005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:37.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 msgr2=0x7f5524005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5536ffd640 10 monclient(hunting): get_auth_request con 0x7f5524006230 auth_method 0 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5536ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5536ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5536ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 msgr2=0x7f552400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 msgr2=0x7f552400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 msgr2=0x7f55240086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:51:37.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 msgr2=0x7f55240086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:37.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:37.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.444+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:51:37.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:37.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:37.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5537fff640 10 monclient(hunting): get_auth_request con 0x7f5524008d90 auth_method 0 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5537fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5537fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f55377fe640 10 monclient(hunting): get_auth_request con 0x7f5524003820 auth_method 0 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f55377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f55377fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f55377fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 msgr2=0x7f552400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 msgr2=0x7f552400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5536ffd640 10 monclient(hunting): get_auth_request con 0x7f5524006230 auth_method 0 2024-08-19T10:51:37.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5536ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5536ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5536ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 msgr2=0x7f5524005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 msgr2=0x7f5524005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 msgr2=0x7f55240086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:51:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 msgr2=0x7f55240086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:37.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:37.848+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:51:38.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:38.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5537fff640 10 monclient(hunting): get_auth_request con 0x7f5524008d90 auth_method 0 2024-08-19T10:51:38.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:38.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5537fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:38.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5537fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:38.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:38.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f55377fe640 10 monclient(hunting): get_auth_request con 0x7f5524003820 auth_method 0 2024-08-19T10:51:38.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f55377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f55377fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f55377fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 msgr2=0x7f552400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 msgr2=0x7f552400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5536ffd640 10 monclient(hunting): get_auth_request con 0x7f5524006230 auth_method 0 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5536ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5536ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5536ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 msgr2=0x7f5524005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:51:38.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 msgr2=0x7f5524005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:38.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:38.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:51:38.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 msgr2=0x7f55240086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:51:38.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 msgr2=0x7f55240086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:51:38.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:51:38.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:38.652+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:51:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 10 monclient(hunting): tick 2024-08-19T10:51:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:51:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 monclient(hunting): continuing hunt 2024-08-19T10:51:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:51:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 msgr2=0x7f55240086c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5524006230 0x7f55240086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 msgr2=0x7f5524005c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003820 0x7f5524005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 msgr2=0x7f552400b250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> v2:172.21.15.89:3300/0 conn(0x7f5524008d90 0x7f552400b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 10 monclient: _add_conns ranks=[5,6,4,0,1] 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 reap_dead start 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f552400bdf0 0x7f552400e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 10 monclient(hunting): picked mon.noname-h con 0x7f552400bdf0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f552400e7c0 0x7f5524010bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 10 monclient(hunting): picked mon.noname-d con 0x7f552400e7c0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 reap_dead start 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5524003c20 0x7f55240066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 10 monclient(hunting): picked mon.noname-c con 0x7f5524003c20 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f55380a7650 con 0x7f5524003c20 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f55380a8710 con 0x7f552400bdf0 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 1 -- 172.21.15.89:0/3249549828 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f55380a5a40 con 0x7f552400e7c0 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55357fa640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55377fe640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f552400bdf0 msgr2=0x7f552400e1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3302/0 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f5537fff640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5524003c20 msgr2=0x7f55240066b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.236+0000 7f55377fe640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f552400bdf0 0x7f552400e1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5537fff640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5524003c20 0x7f55240066b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f552400e7c0 0x7f5524010bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 10 monclient(hunting): get_auth_request con 0x7f552400e7c0 auth_method 0 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 10 monclient(hunting): handle_auth_done global_id 45033 payload 274 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f552400bdf0 msgr2=0x7f552400e1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f552400bdf0 0x7f552400e1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5524003c20 msgr2=0x7f55240066b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5524003c20 0x7f55240066b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:51:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 1 monclient: found mon.noname-d 2024-08-19T10:51:40.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:40.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:51:40.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.240+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55380a81d0 con 0x7f552400e7c0 2024-08-19T10:51:40.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f5536ffd640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f552400e7c0 msgr2=0x7f5524010bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f5536ffd640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f552400e7c0 0x7f5524010bb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f552c016a20 tx=0x7f552c01de70 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e84a8eac4c930220 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 1 -- 172.21.15.89:0/3249549828 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f552c0b3070 con 0x7f552400e7c0 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 10 monclient: dump: 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.244 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.245 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.245 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.245 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.245 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.245 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.245 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 10 monclient: _finish_auth 0 2024-08-19T10:51:40.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 10 monclient: _check_auth_tickets 2024-08-19T10:51:40.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:40.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 1 -- 172.21.15.89:0/3249549828 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552c0173e0 con 0x7f552400e7c0 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 1 -- 172.21.15.89:0/3249549828 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f552c00f040 con 0x7f552400e7c0 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 5 monclient: authenticate success, global_id 45033 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 4 set_mon_vals no callback set 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f55367fc640 10 monclient: dump: 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.246 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.247 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 10 monclient: shutdown 2024-08-19T10:51:40.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:40.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 1 -- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f552400e7c0 msgr2=0x7f5524010bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f552400e7c0 0x7f5524010bb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f552c016a20 tx=0x7f552c01de70 comp rx=0 tx=0).stop 2024-08-19T10:51:40.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 1 -- 172.21.15.89:0/3249549828 shutdown_connections 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5524003c20 0x7f55240066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f552400e7c0 0x7f5524010bb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 1 --2- 172.21.15.89:0/3249549828 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f552400bdf0 0x7f552400e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 1 -- 172.21.15.89:0/3249549828 >> 172.21.15.89:0/3249549828 conn(0x7f5538069a80 msgr2=0x7f55380abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 1 -- 172.21.15.89:0/3249549828 shutdown_connections 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.248+0000 7f553e1d5640 1 -- 172.21.15.89:0/3249549828 wait complete. 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient: build_initial_monmap 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient: monmap: 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:40.251 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 1 Processor -- start 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 1 -- start start 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient: init 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient: _add_conns ranks=[2,4,7,5,3,8,1,0,6] 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003c20 0x7f5538127100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient(hunting): picked mon.noname-b con 0x7f5524003c20 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552400bdf0 0x7f5538124c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient(hunting): picked mon.noname-c con 0x7f552400bdf0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f552400e7c0 0x7f5538125160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient(hunting): picked mon.noname-i con 0x7f552400e7c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f55380a57d0 con 0x7f5524003c20 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f55380a5650 con 0x7f552400bdf0 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f553813a1d0 con 0x7f552400e7c0 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f553e1d5640 10 monclient(hunting): authenticate will time out at 1750.069423s 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f5536ffd640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552400bdf0 msgr2=0x7f5538124c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f5536ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552400bdf0 0x7f5538124c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003c20 0x7f5538127100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003c20 0x7f5538127100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49368/0 (socket says 172.21.15.89:49368) 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 1 -- 172.21.15.89:0/590817476 learned_addr learned my addr 172.21.15.89:0/590817476 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 10 monclient(hunting): get_auth_request con 0x7f5524003c20 auth_method 0 2024-08-19T10:51:40.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f5537fff640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f552400e7c0 0x7f5538125160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f5537fff640 10 monclient(hunting): get_auth_request con 0x7f552400e7c0 auth_method 0 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f5537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f5537fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f5537fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 10 monclient(hunting): handle_auth_done global_id 74225 payload 824 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 1 -- 172.21.15.89:0/590817476 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f552400e7c0 msgr2=0x7f5538125160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f552400e7c0 0x7f5538125160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 1 -- 172.21.15.89:0/590817476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552400bdf0 msgr2=0x7f5538124c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:51:40.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552400bdf0 0x7f5538124c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 10 monclient: _finish_hunting 0 2024-08-19T10:51:40.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 1 monclient: found mon.noname-b 2024-08-19T10:51:40.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:40.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:51:40.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.252+0000 7f55377fe640 1 -- 172.21.15.89:0/590817476 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5538128340 con 0x7f5524003c20 2024-08-19T10:51:40.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f55377fe640 1 -- 172.21.15.89:0/590817476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003c20 msgr2=0x7f5538127100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f55377fe640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003c20 0x7f5538127100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f552800bb80 tx=0x7f5528005480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 1 -- 172.21.15.89:0/590817476 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5528017a70 con 0x7f5524003c20 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 10 monclient: dump: 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.259 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 10 monclient: _finish_auth 0 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f553e1d5640 5 monclient: authenticate success, global_id 74225 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 1 -- 172.21.15.89:0/590817476 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5528017c30 con 0x7f5524003c20 2024-08-19T10:51:40.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f553e1d5640 10 monclient: _renew_subs 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f553e1d5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f553e1d5640 1 -- 172.21.15.89:0/590817476 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5538128620 con 0x7f5524003c20 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 1 -- 172.21.15.89:0/590817476 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5528017de0 con 0x7f5524003c20 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 10 monclient: dump: 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.261 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f5535ffb640 4 set_mon_vals no callback set 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f553e1d5640 10 monclient: _renew_subs 2024-08-19T10:51:40.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f553e1d5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f553e1d5640 1 -- 172.21.15.89:0/590817476 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f55381265a0 con 0x7f5524003c20 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f553e1d5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f553e1d5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f553e1d5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f553e1d5640 1 -- 172.21.15.89:0/590817476 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553813c9f0 con 0x7f5524003c20 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 1 -- 172.21.15.89:0/590817476 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5528017600 con 0x7f5524003c20 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f552403df60 0x7f5524040420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f551ffff640 1 -- 172.21.15.89:0/590817476 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f5528014040 con 0x7f5524003c20 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f5536ffd640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f552403df60 0x7f5524040420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.256+0000 7f5536ffd640 10 monclient: get_auth_request con 0x7f552403df60 auth_method 0 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.260+0000 7f5536ffd640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f552403df60 0x7f5524040420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f552c009100 tx=0x7f552c0a8070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.260+0000 7f551ffff640 1 -- 172.21.15.89:0/590817476 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f5528020e00 con 0x7f5524003c20 2024-08-19T10:51:40.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.260+0000 7f551ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:40.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.260+0000 7f551ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:40.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.496+0000 7f553e1d5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:51:40.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.496+0000 7f553e1d5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:51:40.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.496+0000 7f553e1d5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:51:40.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.496+0000 7f553e1d5640 1 -- 172.21.15.89:0/590817476 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5538064590 con 0x7f5524003c20 2024-08-19T10:51:40.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f551ffff640 1 -- 172.21.15.89:0/590817476 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3175 (secure 0 0 0) 0x7f552804e6a0 con 0x7f5524003c20 2024-08-19T10:51:40.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f551ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:51:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f551ffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:40.495 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":116,"quorum":[0,1,2,3,6,7],"quorum_names":["a","f","b","g","d","i"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:51:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 -- 172.21.15.89:0/590817476 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f552403df60 msgr2=0x7f5524040420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f552403df60 0x7f5524040420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f552c009100 tx=0x7f552c0a8070 comp rx=0 tx=0).stop 2024-08-19T10:51:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 10 monclient: shutdown 2024-08-19T10:51:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 -- 172.21.15.89:0/590817476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003c20 msgr2=0x7f5538127100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003c20 0x7f5538127100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f552800bb80 tx=0x7f5528005480 comp rx=0 tx=0).stop 2024-08-19T10:51:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 -- 172.21.15.89:0/590817476 shutdown_connections 2024-08-19T10:51:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f552403df60 0x7f5524040420 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f552400bdf0 0x7f5538124c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f552400e7c0 0x7f5538125160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 --2- 172.21.15.89:0/590817476 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5524003c20 0x7f5538127100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 -- 172.21.15.89:0/590817476 >> 172.21.15.89:0/590817476 conn(0x7f5538069a80 msgr2=0x7f553809da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 -- 172.21.15.89:0/590817476 shutdown_connections 2024-08-19T10:51:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.500+0000 7f553e1d5640 1 -- 172.21.15.89:0/590817476 wait complete. 2024-08-19T10:51:40.514 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-08-19T10:51:40.514 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon a 2024-08-19T10:51:40.514 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:51:40.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient: get_monmap_and_config 2024-08-19T10:51:40.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient: build_initial_monmap 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient: monmap: 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:40.637199+0000 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:40.637199+0000 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:40.631 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:40.632 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:40.632 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 1 Processor -- start 2024-08-19T10:51:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 1 -- start start 2024-08-19T10:51:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient: init 2024-08-19T10:51:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:40.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient: _add_conns ranks=[9,6,1,5,3,0,8,2,10,7,4] 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f08fc0a5270 0x7f08fc0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient(hunting): picked mon.noname-a con 0x7f08fc0a5270 addr v2:172.21.15.89:3300/0 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 0x7f08fc0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient(hunting): picked mon.noname-d con 0x7f08fc0a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f08fc1395b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08fc0a8710 con 0x7f08fc1395b0 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f08fc0a7650 con 0x7f08fc0a5270 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f08fc0a5a40 con 0x7f08fc0a6d30 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f090276f640 10 monclient(hunting): authenticate will time out at 1750.453771s 2024-08-19T10:51:40.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 0x7f08fc0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 0x7f08fc0a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:32964/0 (socket says 172.21.15.89:32964) 2024-08-19T10:51:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 1 -- 172.21.15.89:0/3818759610 learned_addr learned my addr 172.21.15.89:0/3818759610 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 10 monclient(hunting): get_auth_request con 0x7f08fc0a6d30 auth_method 0 2024-08-19T10:51:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fbfff640 1 --2- 172.21.15.89:0/3818759610 >> v2:172.21.15.89:3300/0 conn(0x7f08fc0a5270 0x7f08fc0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fbfff640 10 monclient(hunting): get_auth_request con 0x7f08fc0a5270 auth_method 0 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 10 monclient(hunting): handle_auth_done global_id 45042 payload 274 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 1 -- 172.21.15.89:0/3818759610 >> v2:172.21.15.89:3300/0 conn(0x7f08fc0a5270 msgr2=0x7f08fc0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 1 --2- 172.21.15.89:0/3818759610 >> v2:172.21.15.89:3300/0 conn(0x7f08fc0a5270 0x7f08fc0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 1 -- 172.21.15.89:0/3818759610 >> v1:172.21.15.190:6789/0 conn(0x7f08fc1395b0 legacy=0x7f08fc139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 1 monclient: found mon.noname-d 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fb7fe640 1 -- 172.21.15.89:0/3818759610 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc0a81d0 con 0x7f08fc0a6d30 2024-08-19T10:51:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.636+0000 7f08fbfff640 1 --2- 172.21.15.89:0/3818759610 >> v2:172.21.15.89:3300/0 conn(0x7f08fc0a5270 0x7f08fc0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:51:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 1 -- 172.21.15.89:0/3818759610 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 msgr2=0x7f08fc0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 1 --2- 172.21.15.89:0/3818759610 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 0x7f08fc0a7110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009e50 tx=0x7f08f401c710 comp rx=0 tx=0).ready entity=mon.6 client_cookie=80d137524bcb4361 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 1 -- 172.21.15.89:0/3818759610 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08f40090d0 con 0x7f08fc0a6d30 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 10 monclient: dump: 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 10 monclient: _finish_auth 0 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 10 monclient: _check_auth_tickets 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 1 -- 172.21.15.89:0/3818759610 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f40a59c0 con 0x7f08fc0a6d30 2024-08-19T10:51:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 1 -- 172.21.15.89:0/3818759610 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08f40ad3f0 con 0x7f08fc0a6d30 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 5 monclient: authenticate success, global_id 45042 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 4 set_mon_vals no callback set 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08faffd640 10 monclient: dump: 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.642 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient: shutdown 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 -- 172.21.15.89:0/3818759610 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 msgr2=0x7f08fc0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 --2- 172.21.15.89:0/3818759610 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 0x7f08fc0a7110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009e50 tx=0x7f08f401c710 comp rx=0 tx=0).stop 2024-08-19T10:51:40.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 -- 172.21.15.89:0/3818759610 shutdown_connections 2024-08-19T10:51:40.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 --2- 172.21.15.89:0/3818759610 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 0x7f08fc0a7110 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 --2- 172.21.15.89:0/3818759610 >> v2:172.21.15.89:3300/0 conn(0x7f08fc0a5270 0x7f08fc0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 -- 172.21.15.89:0/3818759610 >> 172.21.15.89:0/3818759610 conn(0x7f08fc069a80 msgr2=0x7f08fc0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:40.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 -- 172.21.15.89:0/3818759610 shutdown_connections 2024-08-19T10:51:40.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 -- 172.21.15.89:0/3818759610 wait complete. 2024-08-19T10:51:40.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient: build_initial_monmap 2024-08-19T10:51:40.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient: monmap: 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 Processor -- start 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 -- start start 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient: init 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient: _add_conns ranks=[3,6,4,7,8,5,2,1,0] 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc0a5270 0x7f08fc13fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient(hunting): picked mon.noname-g con 0x7f08fc0a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 0x7f08fc1d1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient(hunting): picked mon.noname-d con 0x7f08fc0a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08fc1395b0 0x7f08fc13d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient(hunting): picked mon.noname-c con 0x7f08fc1395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc0a57d0 con 0x7f08fc0a5270 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f08fc0a5650 con 0x7f08fc1395b0 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f08fc144240 con 0x7f08fc0a6d30 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f090276f640 10 monclient(hunting): authenticate will time out at 1750.462766s 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f0900d0c640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08fc1395b0 msgr2=0x7f08fc13d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f0900d0c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08fc1395b0 0x7f08fc13d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 0x7f08fc1d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 0x7f08fc1d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:32966/0 (socket says 172.21.15.89:32966) 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 1 -- 172.21.15.89:0/403234910 learned_addr learned my addr 172.21.15.89:0/403234910 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 10 monclient(hunting): get_auth_request con 0x7f08fc0a6d30 auth_method 0 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fbfff640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc0a5270 0x7f08fc13fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fbfff640 10 monclient(hunting): get_auth_request con 0x7f08fc0a5270 auth_method 0 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 10 monclient(hunting): handle_auth_done global_id 45051 payload 824 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 1 -- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08fc1395b0 msgr2=0x7f08fc13d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f08fc1395b0 0x7f08fc13d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 1 -- 172.21.15.89:0/403234910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc0a5270 msgr2=0x7f08fc13fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc0a5270 0x7f08fc13fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 1 monclient: found mon.noname-d 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.644+0000 7f08fb7fe640 1 -- 172.21.15.89:0/403234910 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc140560 con 0x7f08fc0a6d30 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fb7fe640 1 -- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 msgr2=0x7f08fc1d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fb7fe640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 0x7f08fc1d1790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f08f401c540 tx=0x7f08f40194b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08f40bc020 con 0x7f08fc0a6d30 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 10 monclient: dump: 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 10 monclient: _finish_auth 0 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f090276f640 5 monclient: authenticate success, global_id 45051 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f4003dc0 con 0x7f08fc0a6d30 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f090276f640 10 monclient: _renew_subs 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f090276f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f090276f640 1 -- 172.21.15.89:0/403234910 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc140bd0 con 0x7f08fc0a6d30 2024-08-19T10:51:40.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08f40b8470 con 0x7f08fc0a6d30 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 10 monclient: dump: 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.657 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f9ffb640 4 set_mon_vals no callback set 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f090276f640 10 monclient: _renew_subs 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f090276f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f090276f640 1 -- 172.21.15.89:0/403234910 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc0a72c0 con 0x7f08fc0a6d30 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f090276f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f090276f640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc1d1cd0 0x7f08fc02d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f090276f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f090276f640 1 -- 172.21.15.89:0/403234910 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc0a2660 con 0x7f08fc1d1cd0 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fbfff640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc1d1cd0 0x7f08fc02d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fbfff640 10 monclient: get_auth_request con 0x7f08fc1d1cd0 auth_method 0 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f08f40bf020 con 0x7f08fc0a6d30 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fbfff640 10 monclient(hunting): handle_auth_done global_id 7209 payload 824 2024-08-19T10:51:40.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08dc03be00 0x7f08dc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08fbfff640 1 -- 172.21.15.89:0/403234910 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08fc0a67f0 con 0x7f08fc1d1cd0 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f08f40ea640 con 0x7f08fc0a6d30 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f0900d0c640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08dc03be00 0x7f08dc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.648+0000 7f0900d0c640 10 monclient: get_auth_request con 0x7f08dc03be00 auth_method 0 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08fbfff640 1 -- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc1d1cd0 msgr2=0x7f08fc02d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08fbfff640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc1d1cd0 0x7f08fc02d1d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00f440 tx=0x7f08ec00f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08ec019030 con 0x7f08fc1d1cd0 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08f8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08f8ff9640 10 monclient: dump: 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.660 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f08ec014880 con 0x7f08fc1d1cd0 2024-08-19T10:51:40.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08f8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:40.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08f8ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:40.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc1d1cd0 msgr2=0x7f08fc02d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08f8ff9640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc1d1cd0 0x7f08fc02d1d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00f440 tx=0x7f08ec00f900 comp rx=0 tx=0).stop 2024-08-19T10:51:40.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc1d1cd0 msgr2=0x7f08fc02d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:40.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f08f8ff9640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc1d1cd0 0x7f08fc02d1d0 secure :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00f440 tx=0x7f08ec00f900 comp rx=0 tx=0).stop 2024-08-19T10:51:40.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.652+0000 7f0900d0c640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08dc03be00 0x7f08dc03e2c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f08e4003780 tx=0x7f08e40079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.656+0000 7f090276f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:40.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc02d710 0x7f08fc02db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 1 -- 172.21.15.89:0/403234910 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc064590 con 0x7f08fc02d710 2024-08-19T10:51:40.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc02d710 0x7f08fc02db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 10 monclient: get_auth_request con 0x7f08fc02d710 auth_method 0 2024-08-19T10:51:40.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 10 monclient(hunting): handle_auth_done global_id 7218 payload 824 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 1 -- 172.21.15.89:0/403234910 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f08fc0a05d0 con 0x7f08fc02d710 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 1 -- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc02d710 msgr2=0x7f08fc02db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08fbfff640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc02d710 0x7f08fc02db20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f08ec019000 tx=0x7f08ec0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f08ec015a90 con 0x7f08fc02d710 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08f8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08f8ff9640 10 monclient: dump: 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.665 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9571 (secure 0 0 0) 0x7f08ec015620 con 0x7f08fc02d710 2024-08-19T10:51:40.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08f8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:40.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08f8ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:40.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc02d710 msgr2=0x7f08fc02db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08f8ff9640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc02d710 0x7f08fc02db20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f08ec019000 tx=0x7f08ec0046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08f8ff9640 1 -- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc02d710 msgr2=0x7f08fc02db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:40.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f08f8ff9640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f08fc02d710 0x7f08fc02db20 secure :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f08ec019000 tx=0x7f08ec0046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.667 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:40.667 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:40.667 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:40.667 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:51:40.667 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 116, 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 830473, 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: 7 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:40.668 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:40.669 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:40.671 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:40.672 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.673 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:40.674 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:40.675 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.677 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:40.680 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.681 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.683 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.684 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.685 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:40.686 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.688 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.689 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:51:40.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.691 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:40.691 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:40.691 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:40.691 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.691 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.691 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:40.691 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:40.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 1 -- 172.21.15.89:0/403234910 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08dc03be00 msgr2=0x7f08dc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f08dc03be00 0x7f08dc03e2c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f08e4003780 tx=0x7f08e40079e0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 10 monclient: shutdown 2024-08-19T10:51:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 1 -- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 msgr2=0x7f08fc1d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 1 --2- 172.21.15.89:0/403234910 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f08fc0a6d30 0x7f08fc1d1790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f08f401c540 tx=0x7f08f40194b0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f0900d0c640 1 -- 172.21.15.89:0/403234910 reap_dead start 2024-08-19T10:51:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f0900d0c640 1 -- 172.21.15.89:0/403234910 reap_dead start 2024-08-19T10:51:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 1 -- 172.21.15.89:0/403234910 shutdown_connections 2024-08-19T10:51:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 1 -- 172.21.15.89:0/403234910 >> 172.21.15.89:0/403234910 conn(0x7f08fc069a80 msgr2=0x7f08fc0ab0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 1 -- 172.21.15.89:0/403234910 shutdown_connections 2024-08-19T10:51:40.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.660+0000 7f090276f640 1 -- 172.21.15.89:0/403234910 wait complete. 2024-08-19T10:51:40.694 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:51:40.694 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-08-19T10:51:40.694 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:51:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.808+0000 7fc5c9467640 10 monclient: get_monmap_and_config 2024-08-19T10:51:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.808+0000 7fc5c9467640 10 monclient: build_initial_monmap 2024-08-19T10:51:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.808+0000 7fc5c9467640 10 monclient: monmap: 2024-08-19T10:51:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:40.806 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:40.812722+0000 2024-08-19T10:51:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:40.812722+0000 2024-08-19T10:51:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:40.807 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:40.808 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:40.808 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 1 Processor -- start 2024-08-19T10:51:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 1 -- start start 2024-08-19T10:51:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 10 monclient: init 2024-08-19T10:51:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 10 monclient: _add_conns ranks=[8,1,5,7,9,6,0,10,3,4,2] 2024-08-19T10:51:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc5c40a5270 0x7fc5c40a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 10 monclient(hunting): picked mon.noname-e con 0x7fc5c40a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:40.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc5c40a6d30 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc5c40a7110 0x7fc5c4139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 10 monclient(hunting): picked mon.noname-h con 0x7fc5c40a7110 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c40a8710 con 0x7fc5c40a6d30 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc5c40a5a40 con 0x7fc5c40a7110 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc5c40a81d0 con 0x7fc5c40a5270 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c2ffd640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc5c40a5270 msgr2=0x7fc5c40a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c9467640 10 monclient(hunting): authenticate will time out at 1750.629299s 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c2ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc5c40a5270 0x7fc5c40a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c37fe640 1 -- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc5c40a7110 msgr2=0x7fc5c4139cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3302/0 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c37fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc5c40a7110 0x7fc5c4139cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c27fc640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fc5c40a6d30 0x7fc5c41395b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.89:38222/0 (socket says 172.21.15.89:38222) 2024-08-19T10:51:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.812+0000 7fc5c27fc640 1 -- 172.21.15.89:0/2779851279 learned_addr learned my addr 172.21.15.89:0/2779851279 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:51:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 1 -- 172.21.15.89:0/2779851279 >> v1:172.21.15.190:6789/0 conn(0x7fc5c40a6d30 legacy=0x7fc5c41395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1889405489 0 0) 0x7fc5b4001460 con 0x7fc5c40a6d30 2024-08-19T10:51:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): _negotiate 2024-08-19T10:51:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): my global_id is 45001 2024-08-19T10:51:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5b0003530 con 0x7fc5c40a6d30 2024-08-19T10:51:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1716226603 0 0) 0x7fc5b0003530 con 0x7fc5c40a6d30 2024-08-19T10:51:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc5c40a5270 msgr2=0x7fc5c40a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc5c40a5270 0x7fc5c40a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc5c40a7110 msgr2=0x7fc5c4139cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc5c40a7110 0x7fc5c4139cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient: _finish_hunting 0 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 monclient: found mon.noname-f-legacy 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.190:6789/0 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c40a4790 con 0x7fc5c40a6d30 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 3927028378 0 0) 0x7fc5b40048b0 con 0x7fc5c40a6d30 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c9467640 5 monclient: authenticate success, global_id 45001 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c9467640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient: dump: 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] but i'm connected to v1:172.21.15.190:6789/0, reconnecting 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 >> v1:172.21.15.190:6789/0 conn(0x7fc5c40a6d30 legacy=0x7fc5c41395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient: _add_conns ranks=[2,0,4,8,7,1,5,6,3] 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004240 0x7fc5b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): picked mon.b con 0x7fc5b0004240 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5b0004f30 0x7fc5b00099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): picked mon.a con 0x7fc5b0004f30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5b0009f00 0x7fc5b000c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): picked mon.c con 0x7fc5b0009f00 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c40a5a40 con 0x7fc5b0004f30 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c40a81d0 con 0x7fc5b0004240 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc5b000cb90 con 0x7fc5b0009f00 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c1ffb640 10 monclient(hunting): _finish_auth 0 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c9467640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c37fe640 1 -- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5b0009f00 msgr2=0x7fc5b000c2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c37fe640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5b0009f00 0x7fc5b000c2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5b0004f30 0x7fc5b00099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient(hunting): get_auth_request con 0x7fc5b0004f30 auth_method 0 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c2ffd640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004240 0x7fc5b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c2ffd640 10 monclient(hunting): get_auth_request con 0x7fc5b0004240 auth_method 0 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_done global_id 45001 payload 293 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 1 -- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5b0009f00 msgr2=0x7fc5b000c2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc5b0009f00 0x7fc5b000c2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 1 -- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004240 msgr2=0x7fc5b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004240 0x7fc5b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient: _finish_hunting 0 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 1 monclient: found mon.a 2024-08-19T10:51:40.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:40.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:40.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 1 -- 172.21.15.89:0/2779851279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b0003530 con 0x7fc5b0004f30 2024-08-19T10:51:40.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c37fe640 1 -- 172.21.15.89:0/2779851279 reap_dead start 2024-08-19T10:51:40.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient: _finish_auth 0 2024-08-19T10:51:40.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:51:40.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.816+0000 7fc5c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:40.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c27fc640 1 -- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5b0004f30 msgr2=0x7fc5b00099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c27fc640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5b0004f30 0x7fc5b00099c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40016b0 tx=0x7fc5b401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4508b651e9d1b577 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5b4008cc0 con 0x7fc5b0004f30 2024-08-19T10:51:40.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:40.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c1ffb640 10 monclient: dump: 2024-08-19T10:51:40.829 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.829 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.829 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.829 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.829 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:40.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b4005d50 con 0x7fc5b0004f30 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c1ffb640 1 -- 172.21.15.89:0/2779851279 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5b4008620 con 0x7fc5b0004f30 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 4 set_mon_vals no callback set 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c1ffb640 10 monclient: dump: 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.831 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 10 monclient: shutdown 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:40.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 1 -- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5b0004f30 msgr2=0x7fc5b00099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5b0004f30 0x7fc5b00099c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40016b0 tx=0x7fc5b401cab0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 1 -- 172.21.15.89:0/2779851279 shutdown_connections 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 1 --2- 172.21.15.89:0/2779851279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5b0004f30 0x7fc5b00099c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 1 -- 172.21.15.89:0/2779851279 >> 172.21.15.89:0/2779851279 conn(0x7fc5c4069a80 msgr2=0x7fc5c40abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 1 -- 172.21.15.89:0/2779851279 shutdown_connections 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 1 -- 172.21.15.89:0/2779851279 wait complete. 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 10 monclient: build_initial_monmap 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 10 monclient: monmap: 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.833 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.820+0000 7fc5c9467640 1 Processor -- start 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 1 -- start start 2024-08-19T10:51:40.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient: init 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient: _add_conns ranks=[2,1,7,8,4,3,0,6,5] 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004f30 0x7fc5c4138650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient(hunting): picked mon.noname-b con 0x7fc5b0004f30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5c41d1360 0x7fc5c4138b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient(hunting): picked mon.noname-f con 0x7fc5c41d1360 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5c4122440 0x7fc5c41228a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient(hunting): picked mon.noname-i con 0x7fc5c4122440 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c40a57d0 con 0x7fc5c41d1360 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c40a5650 con 0x7fc5b0004f30 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc5c4139dc0 con 0x7fc5c4122440 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient(hunting): authenticate will time out at 1750.640610s 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004f30 0x7fc5c4138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004f30 0x7fc5c4138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:49386/0 (socket says 172.21.15.89:49386) 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 -- 172.21.15.89:0/2220139357 learned_addr learned my addr 172.21.15.89:0/2220139357 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 10 monclient(hunting): get_auth_request con 0x7fc5b0004f30 auth_method 0 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:40.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c27fc640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5c41d1360 0x7fc5c4138b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c37fe640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5c4122440 0x7fc5c41228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 10 monclient(hunting): handle_auth_done global_id 74234 payload 824 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 -- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5c4122440 msgr2=0x7fc5c41228a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc5c4122440 0x7fc5c41228a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 -- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5c41d1360 msgr2=0x7fc5c4138b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5c41d1360 0x7fc5c4138b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 monclient: found mon.noname-b 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 -- 172.21.15.89:0/2220139357 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c4123cb0 con 0x7fc5b0004f30 2024-08-19T10:51:40.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c27fc640 10 monclient: get_auth_request con 0x7fc5c41d1360 auth_method 0 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c27fc640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5c41d1360 0x7fc5c4138b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 -- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004f30 msgr2=0x7fc5c4138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004f30 0x7fc5c4138650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0096c0 tx=0x7fc5ac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c2ffd640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004f30 0x7fc5c4138650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0096c0 tx=0x7fc5ac009b80 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fc5ac00edd0 config(0 keys) 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5ac016020 con 0x7fc5b0004f30 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5bb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5bb7fe640 10 monclient: dump: 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.838 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5bb7fe640 10 monclient: _finish_auth 0 2024-08-19T10:51:40.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5bb7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5bb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ac00edd0 con 0x7fc5b0004f30 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 5 monclient: authenticate success, global_id 74234 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5bb7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient: _renew_subs 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 1 -- 172.21.15.89:0/2220139357 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c4125770 con 0x7fc5b0004f30 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c17fa640 4 set_mon_vals no callback set 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient: _renew_subs 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.824+0000 7fc5c9467640 1 -- 172.21.15.89:0/2220139357 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c4122de0 con 0x7fc5b0004f30 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c9467640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c9467640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c4123070 0x7fc5c4123490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c9467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c9467640 1 -- 172.21.15.89:0/2220139357 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c41254f0 con 0x7fc5c4123070 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c4123070 0x7fc5c4123490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 10 monclient: get_auth_request con 0x7fc5c4123070 auth_method 0 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_done global_id 74243 payload 824 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 1 -- 172.21.15.89:0/2220139357 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5c413a150 con 0x7fc5c4123070 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5ac012860 con 0x7fc5b0004f30 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 10 monclient: dump: 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.842 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc5ac019020 con 0x7fc5b0004f30 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc5a403be50 0x7fc5a403e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c37fe640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc5a403be50 0x7fc5a403e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c37fe640 10 monclient: get_auth_request con 0x7fc5a403be50 auth_method 0 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 1 -- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c4123070 msgr2=0x7fc5c4123490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5c27fc640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c4123070 0x7fc5c4123490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40a9000 tx=0x7fc5b40198f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5b4005e60 con 0x7fc5c4123070 2024-08-19T10:51:40.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 10 monclient: dump: 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.844 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc5b40b5030 con 0x7fc5c4123070 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c4123070 msgr2=0x7fc5c4123490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c4123070 0x7fc5c4123490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40a9000 tx=0x7fc5b40198f0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c4123070 msgr2=0x7fc5c4123490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.828+0000 7fc5bb7fe640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c4123070 0x7fc5c4123490 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40a9000 tx=0x7fc5b40198f0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.832+0000 7fc5c37fe640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc5a403be50 0x7fc5a403e310 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40a86e0 tx=0x7fc5bc0071c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c9467640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:40.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c9467640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c409d5f0 0x7fc5c409da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c9467640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c9467640 1 -- 172.21.15.89:0/2220139357 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c4064590 con 0x7fc5c409d5f0 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c409d5f0 0x7fc5c409da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 10 monclient: get_auth_request con 0x7fc5c409d5f0 auth_method 0 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 10 monclient(hunting): handle_auth_done global_id 74252 payload 824 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 1 -- 172.21.15.89:0/2220139357 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc5c4125c20 con 0x7fc5c409d5f0 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 1 -- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c409d5f0 msgr2=0x7fc5c409da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5c27fc640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c409d5f0 0x7fc5c409da20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40016b0 tx=0x7fc5b0005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:40.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc5b4008620 con 0x7fc5c409d5f0 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5bb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5bb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.836+0000 7fc5bb7fe640 10 monclient: dump: 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.849 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9234 (secure 0 0 0) 0x7fc5b40081b0 con 0x7fc5c409d5f0 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5bb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5bb7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c409d5f0 msgr2=0x7fc5c409da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5bb7fe640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c409d5f0 0x7fc5c409da20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40016b0 tx=0x7fc5b0005840 comp rx=0 tx=0).stop 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c409d5f0 msgr2=0x7fc5c409da20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:40.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5bb7fe640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5c409d5f0 0x7fc5c409da20 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40016b0 tx=0x7fc5b0005840 comp rx=0 tx=0).stop 2024-08-19T10:51:40.851 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 116, 2024-08-19T10:51:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 37335, 2024-08-19T10:51:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:40.851 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: 7 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:40.852 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:40.853 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:40.855 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.856 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.857 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.858 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:40.859 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.862 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.863 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.864 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:40.867 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:40.869 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.871 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 4 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:40.872 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:40.873 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:40.873 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:40.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c9467640 1 -- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc5a403be50 msgr2=0x7fc5a403e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c9467640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc5a403be50 0x7fc5a403e310 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40a86e0 tx=0x7fc5bc0071c0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c9467640 10 monclient: shutdown 2024-08-19T10:51:40.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c9467640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:40.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c9467640 1 -- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004f30 msgr2=0x7fc5c4138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c9467640 1 --2- 172.21.15.89:0/2220139357 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fc5b0004f30 0x7fc5c4138650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0096c0 tx=0x7fc5ac009b80 comp rx=0 tx=0).stop 2024-08-19T10:51:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c37fe640 1 -- 172.21.15.89:0/2220139357 reap_dead start 2024-08-19T10:51:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5bb7fe640 1 -- 172.21.15.89:0/2220139357 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (unknown 0 0 0) 0x7fc5ac00f040 con 0x7fc5b0004f30 2024-08-19T10:51:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c37fe640 1 -- 172.21.15.89:0/2220139357 reap_dead start 2024-08-19T10:51:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c9467640 1 -- 172.21.15.89:0/2220139357 shutdown_connections 2024-08-19T10:51:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c9467640 1 -- 172.21.15.89:0/2220139357 >> 172.21.15.89:0/2220139357 conn(0x7fc5c4069a80 msgr2=0x7fc5c41c4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c9467640 1 -- 172.21.15.89:0/2220139357 shutdown_connections 2024-08-19T10:51:40.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.840+0000 7fc5c9467640 1 -- 172.21.15.89:0/2220139357 wait complete. 2024-08-19T10:51:40.875 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:51:40.875 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-08-19T10:51:40.876 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:51:40.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.984+0000 7f11acc99640 10 monclient: get_monmap_and_config 2024-08-19T10:51:40.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.984+0000 7f11acc99640 10 monclient: build_initial_monmap 2024-08-19T10:51:40.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.984+0000 7f11acc99640 10 monclient: monmap: 2024-08-19T10:51:40.982 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:40.982 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:40.988644+0000 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:40.988644+0000 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:40.983 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:40.984 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:40.984 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:40.984 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 1 Processor -- start 2024-08-19T10:51:40.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 1 -- start start 2024-08-19T10:51:40.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 10 monclient: init 2024-08-19T10:51:40.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:40.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 10 monclient: _add_conns ranks=[4,0,5,1,7,8,9,10,6,2,3] 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a80a3620 0x7f11a80a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 10 monclient(hunting): picked mon.noname-c con 0x7f11a80a3620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f11a809c9e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f11a80a3f40 0x7f11a8137b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 10 monclient(hunting): picked mon.noname-h con 0x7f11a80a3f40 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a80a1790 con 0x7f11a809c9e0 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f11a809dab0 con 0x7f11a80a3620 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f11a80a1f20 con 0x7f11a80a3f40 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11acc99640 10 monclient(hunting): authenticate will time out at 1750.805260s 2024-08-19T10:51:40.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a659c640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a80a3620 msgr2=0x7f11a80a3a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:51:40.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a659c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a80a3620 0x7f11a80a3a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:40.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a5d9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f11a809c9e0 0x7f11a81373f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:35926/0 (socket says 172.21.15.89:35926) 2024-08-19T10:51:40.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a5d9b640 1 -- 172.21.15.89:0/2083343341 learned_addr learned my addr 172.21.15.89:0/2083343341 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:51:40.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a6d9d640 1 -- 172.21.15.89:0/2083343341 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f11a80a3f40 msgr2=0x7f11a8137b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3302/0 2024-08-19T10:51:40.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a6d9d640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f11a80a3f40 0x7f11a8137b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:40.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a5d9b640 1 -- 172.21.15.89:0/2083343341 >> v1:172.21.15.89:6789/0 conn(0x7f11a809c9e0 legacy=0x7f11a81373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:40.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1100826757 0 0) 0x7f11a80a1790 con 0x7f11a809c9e0 2024-08-19T10:51:40.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a559a640 10 monclient(hunting): _negotiate 2024-08-19T10:51:40.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a559a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a559a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:40.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a559a640 10 monclient(hunting): my global_id is 7227 2024-08-19T10:51:40.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.988+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1190003530 con 0x7f11a809c9e0 2024-08-19T10:51:40.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 794171820 0 0) 0x7f1190003530 con 0x7f11a809c9e0 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f11a80a3f40 msgr2=0x7f11a8137b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f11a80a3f40 0x7f11a8137b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a80a3620 msgr2=0x7f11a80a3a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f11a80a3620 0x7f11a80a3a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient: _finish_hunting 0 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 monclient: found mon.noname-a-legacy 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.89:6789/0 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a809d1a0 con 0x7f11a809c9e0 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient: _finish_auth 0 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient: _check_auth_tickets 2024-08-19T10:51:40.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 3927028378 0 0) 0x7f1198001ce0 con 0x7f11a809c9e0 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11acc99640 5 monclient: authenticate success, global_id 7227 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11acc99640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient: dump: 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:40.992 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] but i'm connected to v1:172.21.15.89:6789/0, reconnecting 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 >> v1:172.21.15.89:6789/0 conn(0x7f11a809c9e0 legacy=0x7f11a81373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient: _add_conns ranks=[1,6,0,2,4,5,3,7,8] 2024-08-19T10:51:40.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1190004240 0x7f1190004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient(hunting): picked mon.f con 0x7f1190004240 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1190004f30 0x7f11900099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient(hunting): picked mon.d con 0x7f1190004f30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1190009f00 0x7f119000c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient(hunting): picked mon.a con 0x7f1190009f00 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f11a809dab0 con 0x7f1190009f00 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f119000cb60 con 0x7f1190004240 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f119000cd10 con 0x7f1190004f30 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:40.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a559a640 10 monclient(hunting): _finish_auth 0 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11acc99640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a6d9d640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1190009f00 0x7f119000c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1190004f30 0x7f11900099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a6d9d640 10 monclient(hunting): get_auth_request con 0x7f1190009f00 auth_method 0 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient(hunting): get_auth_request con 0x7f1190004f30 auth_method 0 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:40.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a659c640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1190004240 0x7f1190004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient(hunting): handle_auth_done global_id 7227 payload 293 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 1 -- 172.21.15.89:0/2083343341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1190004240 msgr2=0x7f1190004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1190004240 0x7f1190004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 1 -- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1190009f00 msgr2=0x7f119000c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1190009f00 0x7f119000c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 1 monclient: found mon.d 2024-08-19T10:51:40.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:40.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:51:40.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 1 -- 172.21.15.89:0/2083343341 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1190003530 con 0x7f1190004f30 2024-08-19T10:51:40.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient: _finish_auth 0 2024-08-19T10:51:40.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:51:40.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:40.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a6d9d640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1190009f00 0x7f119000c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:40.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.992+0000 7f11a6d9d640 1 -- 172.21.15.89:0/2083343341 reap_dead start 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 1 -- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1190004f30 msgr2=0x7f11900099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1190004f30 0x7f11900099c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f11980014e0 tx=0x7f119801caf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ab1f05e9d701305c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1198008d50 con 0x7f1190004f30 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a559a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a559a640 10 monclient: dump: 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.004 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1198005d50 con 0x7f1190004f30 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 4 set_mon_vals no callback set 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a559a640 1 -- 172.21.15.89:0/2083343341 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1198008620 con 0x7f1190004f30 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a559a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a559a640 10 monclient: dump: 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient: shutdown 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 -- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1190004f30 msgr2=0x7f11900099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1190004f30 0x7f11900099c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f11980014e0 tx=0x7f119801caf0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 -- 172.21.15.89:0/2083343341 shutdown_connections 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 --2- 172.21.15.89:0/2083343341 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1190004f30 0x7f11900099c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 -- 172.21.15.89:0/2083343341 >> 172.21.15.89:0/2083343341 conn(0x7f11a80a8210 msgr2=0x7f11a80a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 -- 172.21.15.89:0/2083343341 shutdown_connections 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 -- 172.21.15.89:0/2083343341 wait complete. 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient: build_initial_monmap 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient: monmap: 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.008 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 Processor -- start 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 -- start start 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient: init 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:41.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient: _add_conns ranks=[4,0,2,5,8,1,3,6,7] 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1190004f30 0x7f11a8136480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient(hunting): picked mon.noname-c con 0x7f1190004f30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a81cec20 0x7f11a81369c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient(hunting): picked mon.noname-a con 0x7f11a81cec20 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a8120090 0x7f11a81204f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient(hunting): picked mon.noname-b con 0x7f11a8120090 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8137c00 con 0x7f11a81cec20 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f11a809df40 con 0x7f11a8120090 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a659c640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1190004f30 msgr2=0x7f11a8136480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:51:41.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f11a8137d80 con 0x7f1190004f30 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11acc99640 10 monclient(hunting): authenticate will time out at 1750.815331s 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a659c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1190004f30 0x7f11a8136480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a81cec20 0x7f11a81369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a6d9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a8120090 0x7f11a81204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a81cec20 0x7f11a81369c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35362/0 (socket says 172.21.15.89:35362) 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 1 -- 172.21.15.89:0/1949853595 learned_addr learned my addr 172.21.15.89:0/1949853595 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 10 monclient(hunting): get_auth_request con 0x7f11a81cec20 auth_method 0 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a6d9d640 10 monclient(hunting): get_auth_request con 0x7f11a8120090 auth_method 0 2024-08-19T10:51:41.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 10 monclient(hunting): handle_auth_done global_id 7236 payload 824 2024-08-19T10:51:41.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 1 -- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1190004f30 msgr2=0x7f11a8136480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f1190004f30 0x7f11a8136480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 1 -- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a8120090 msgr2=0x7f11a81204f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a8120090 0x7f11a81204f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 1 monclient: found mon.noname-a 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a5d9b640 1 -- 172.21.15.89:0/1949853595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a80a2ef0 con 0x7f11a81cec20 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:40.996+0000 7f11a6d9d640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f11a8120090 0x7f11a81204f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a5d9b640 1 -- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a81cec20 msgr2=0x7f11a81369c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a5d9b640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a81cec20 0x7f11a81369c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1198000c10 tx=0x7f1190005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1198001cc0 con 0x7f11a81cec20 2024-08-19T10:51:41.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 10 monclient: dump: 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.014 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 10 monclient: _finish_auth 0 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11980b2030 con 0x7f11a81cec20 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11acc99640 5 monclient: authenticate success, global_id 7236 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11acc99640 10 monclient: _renew_subs 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11acc99640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11acc99640 1 -- 172.21.15.89:0/1949853595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a8121ae0 con 0x7f11a81cec20 2024-08-19T10:51:41.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1198008620 con 0x7f11a81cec20 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 10 monclient: dump: 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.016 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11acc99640 10 monclient: _renew_subs 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a4d99640 4 set_mon_vals no callback set 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11acc99640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11acc99640 1 -- 172.21.15.89:0/1949853595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a8123880 con 0x7f11a81cec20 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11acc99640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11acc99640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a8120a30 0x7f11a8120e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11acc99640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11acc99640 1 -- 172.21.15.89:0/1949853595 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f11a8138170 con 0x7f11a8120a30 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a659c640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a8120a30 0x7f11a8120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a659c640 10 monclient: get_auth_request con 0x7f11a8120a30 auth_method 0 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f11980081b0 con 0x7f11a81cec20 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a659c640 10 monclient(hunting): handle_auth_done global_id 45060 payload 824 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a659c640 1 -- 172.21.15.89:0/1949853595 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11a8064590 con 0x7f11a8120a30 2024-08-19T10:51:41.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f118803be30 0x7f118803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f11980e8ea0 con 0x7f11a81cec20 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a6d9d640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f118803be30 0x7f118803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.000+0000 7f11a6d9d640 10 monclient: get_auth_request con 0x7f118803be30 auth_method 0 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f11a659c640 1 -- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a8120a30 msgr2=0x7f11a8120e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f11a659c640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a8120a30 0x7f11a8120e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f119c0038a0 tx=0x7f119c00c950 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f119c003de0 con 0x7f11a8120a30 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f1196ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f1196ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f1196ffd640 10 monclient: dump: 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.019 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.020 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f119c016070 con 0x7f11a8120a30 2024-08-19T10:51:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f1196ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f1196ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a8120a30 msgr2=0x7f11a8120e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f1196ffd640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a8120a30 0x7f11a8120e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f119c0038a0 tx=0x7f119c00c950 comp rx=0 tx=0).stop 2024-08-19T10:51:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a8120a30 msgr2=0x7f11a8120e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f1196ffd640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a8120a30 0x7f11a8120e10 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f119c0038a0 tx=0x7f119c00c950 comp rx=0 tx=0).stop 2024-08-19T10:51:41.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.004+0000 7f11a6d9d640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f118803be30 0x7f118803e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f11a80a5080 tx=0x7f11a0009520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.008+0000 7f11acc99640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:41.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.008+0000 7f11acc99640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a80712f0 0x7f11a80716d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.008+0000 7f11acc99640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.008+0000 7f11acc99640 1 -- 172.21.15.89:0/1949853595 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f11a806dc30 con 0x7f11a80712f0 2024-08-19T10:51:41.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.008+0000 7f11a659c640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a80712f0 0x7f11a80716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.008+0000 7f11a659c640 10 monclient: get_auth_request con 0x7f11a80712f0 auth_method 0 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.008+0000 7f11a659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.008+0000 7f11a659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.008+0000 7f11a659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11a659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11a659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11a659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11a659c640 10 monclient(hunting): handle_auth_done global_id 45069 payload 824 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11a659c640 1 -- 172.21.15.89:0/1949853595 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f11a8121350 con 0x7f11a80712f0 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11a659c640 1 -- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a80712f0 msgr2=0x7f11a80716d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11a659c640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a80712f0 0x7f11a80716d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f119c0038a0 tx=0x7f119c005a90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f119c011410 con 0x7f11a80712f0 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f1196ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f1196ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f1196ffd640 10 monclient: dump: 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.025 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9402 (secure 0 0 0) 0x7f119c005490 con 0x7f11a80712f0 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f1196ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f1196ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a80712f0 msgr2=0x7f11a80716d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f1196ffd640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a80712f0 0x7f11a80716d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f119c0038a0 tx=0x7f119c005a90 comp rx=0 tx=0).stop 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f1196ffd640 1 -- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a80712f0 msgr2=0x7f11a80716d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:41.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f1196ffd640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f11a80712f0 0x7f11a80716d0 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f119c0038a0 tx=0x7f119c005a90 comp rx=0 tx=0).stop 2024-08-19T10:51:41.027 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 116, 2024-08-19T10:51:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 424139, 2024-08-19T10:51:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:41.027 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: 7 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.028 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.029 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:41.030 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.031 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:41.032 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:41.033 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:41.034 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:41.036 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:41.037 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.038 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:41.040 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.042 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.043 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.044 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.045 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.046 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.047 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:41.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.049 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.050 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:41.050 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:41.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11acc99640 1 -- 172.21.15.89:0/1949853595 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f118803be30 msgr2=0x7f118803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11acc99640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f118803be30 0x7f118803e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f11a80a5080 tx=0x7f11a0009520 comp rx=0 tx=0).stop 2024-08-19T10:51:41.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11acc99640 10 monclient: shutdown 2024-08-19T10:51:41.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11acc99640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:41.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11acc99640 1 -- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a81cec20 msgr2=0x7f11a81369c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11a6d9d640 1 -- 172.21.15.89:0/1949853595 reap_dead start 2024-08-19T10:51:41.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11acc99640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a81cec20 0x7f11a81369c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1198000c10 tx=0x7f1190005840 comp rx=0 tx=0).stop 2024-08-19T10:51:41.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11acc99640 1 -- 172.21.15.89:0/1949853595 shutdown_connections 2024-08-19T10:51:41.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11acc99640 1 --2- 172.21.15.89:0/1949853595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11a81cec20 0x7f11a81369c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11acc99640 1 -- 172.21.15.89:0/1949853595 >> 172.21.15.89:0/1949853595 conn(0x7f11a80a8210 msgr2=0x7f11a80707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:41.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11acc99640 1 -- 172.21.15.89:0/1949853595 shutdown_connections 2024-08-19T10:51:41.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.012+0000 7f11acc99640 1 -- 172.21.15.89:0/1949853595 wait complete. 2024-08-19T10:51:41.052 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:51:41.052 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon f 2024-08-19T10:51:41.053 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:51:41.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient: get_monmap_and_config 2024-08-19T10:51:41.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient: build_initial_monmap 2024-08-19T10:51:41.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient: monmap: 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:41.177936+0000 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:41.177936+0000 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:41.172 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:41.173 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:41.173 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 1 Processor -- start 2024-08-19T10:51:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 1 -- start start 2024-08-19T10:51:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient: init 2024-08-19T10:51:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient: _add_conns ranks=[4,10,1,8,7,6,3,5,0,9,2] 2024-08-19T10:51:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb040a6860 0x7fbb040a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient(hunting): picked mon.noname-c con 0x7fbb040a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbb040a7180 0x7fbb040a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient(hunting): picked mon.noname-f con 0x7fbb040a7180 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbb04139620 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb040a33d0 con 0x7fbb04139620 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbb040a8190 con 0x7fbb040a7180 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbb040a2ec0 con 0x7fbb040a6860 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb08f00640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb040a6860 msgr2=0x7fbb040a6c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb0b164640 10 monclient(hunting): authenticate will time out at 1750.994676s 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb08f00640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb040a6860 0x7fbb040a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb03fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbb040a7180 0x7fbb040a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb03fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbb040a7180 0x7fbb040a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:50422/0 (socket says 172.21.15.89:50422) 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb03fff640 1 -- 172.21.15.89:0/3328944046 learned_addr learned my addr 172.21.15.89:0/3328944046 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:41.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb03fff640 10 monclient(hunting): get_auth_request con 0x7fbb040a7180 auth_method 0 2024-08-19T10:51:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb03fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb03fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb03fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb03fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb03fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.176+0000 7fbb03fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.180+0000 7fbb09701640 1 -- 172.21.15.89:0/3328944046 >> v1:172.21.15.190:6789/0 conn(0x7fbb04139620 legacy=0x7fbb04139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.180+0000 7fbb03fff640 10 monclient(hunting): handle_auth_done global_id 45010 payload 274 2024-08-19T10:51:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.180+0000 7fbb03fff640 1 -- 172.21.15.89:0/3328944046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb040a6860 msgr2=0x7fbb040a6c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.180+0000 7fbb03fff640 1 --2- 172.21.15.89:0/3328944046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb040a6860 0x7fbb040a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.180+0000 7fbb03fff640 1 -- 172.21.15.89:0/3328944046 >> v1:172.21.15.190:6789/0 conn(0x7fbb04139620 legacy=0x7fbb04139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.180+0000 7fbb03fff640 10 monclient: _finish_hunting 0 2024-08-19T10:51:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.180+0000 7fbb03fff640 1 monclient: found mon.noname-f 2024-08-19T10:51:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.180+0000 7fbb03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.180+0000 7fbb03fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:51:41.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.180+0000 7fbb03fff640 1 -- 172.21.15.89:0/3328944046 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb040a4cd0 con 0x7fbb040a7180 2024-08-19T10:51:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb03fff640 1 -- 172.21.15.89:0/3328944046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb040a7180 msgr2=0x7fbb040a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb03fff640 1 --2- 172.21.15.89:0/3328944046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb040a7180 0x7fbb040a7560 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00098f0 tx=0x7fbaf001c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e226f3f551f0fc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 1 -- 172.21.15.89:0/3328944046 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbaf00b0070 con 0x7fbb040a7180 2024-08-19T10:51:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:51:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 10 monclient: dump: 2024-08-19T10:51:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.181 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.182 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 10 monclient: _finish_auth 0 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 10 monclient: _check_auth_tickets 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 1 -- 172.21.15.89:0/3328944046 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf00aa810 con 0x7fbb040a7180 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 1 -- 172.21.15.89:0/3328944046 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbaf00a93e0 con 0x7fbb040a7180 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb037fe640 10 monclient: dump: 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.183 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.184 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 5 monclient: authenticate success, global_id 45010 2024-08-19T10:51:41.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:41.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 4 set_mon_vals no callback set 2024-08-19T10:51:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 10 monclient: shutdown 2024-08-19T10:51:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:41.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 1 -- 172.21.15.89:0/3328944046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb040a7180 msgr2=0x7fbb040a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 1 --2- 172.21.15.89:0/3328944046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb040a7180 0x7fbb040a7560 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00098f0 tx=0x7fbaf001c4c0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 1 -- 172.21.15.89:0/3328944046 shutdown_connections 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 1 --2- 172.21.15.89:0/3328944046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb040a7180 0x7fbb040a7560 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 1 --2- 172.21.15.89:0/3328944046 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb040a6860 0x7fbb040a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 1 -- 172.21.15.89:0/3328944046 >> 172.21.15.89:0/3328944046 conn(0x7fbb04074b40 msgr2=0x7fbb040abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 1 -- 172.21.15.89:0/3328944046 shutdown_connections 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.184+0000 7fbb0b164640 1 -- 172.21.15.89:0/3328944046 wait complete. 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient: build_initial_monmap 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient: monmap: 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:41.190 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:41.191 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 1 Processor -- start 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 1 -- start start 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient: init 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient: _add_conns ranks=[4,1,0,7,6,8,5,2,3] 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb040a6860 0x7fbb041d1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient(hunting): picked mon.noname-c con 0x7fbb040a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb040a7180 0x7fbb041d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient(hunting): picked mon.noname-f con 0x7fbb040a7180 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb04139620 0x7fbb0413d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient(hunting): picked mon.noname-a con 0x7fbb04139620 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb08f00640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb040a6860 msgr2=0x7fbb041d1780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:51:41.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04144430 con 0x7fbb04139620 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041442b0 con 0x7fbb040a7180 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbb041445b0 con 0x7fbb040a6860 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb08f00640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb040a6860 0x7fbb041d1780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb0b164640 10 monclient(hunting): authenticate will time out at 1751.005512s 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb04139620 0x7fbb0413d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb04139620 0x7fbb0413d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37446/0 (socket says 172.21.15.89:37446) 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 1 -- 172.21.15.89:0/1865289281 learned_addr learned my addr 172.21.15.89:0/1865289281 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 10 monclient(hunting): get_auth_request con 0x7fbb04139620 auth_method 0 2024-08-19T10:51:41.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb03fff640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb040a7180 0x7fbb041d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 10 monclient(hunting): handle_auth_done global_id 7245 payload 824 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 1 -- 172.21.15.89:0/1865289281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb040a6860 msgr2=0x7fbb041d1780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb040a6860 0x7fbb041d1780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 1 -- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb040a7180 msgr2=0x7fbb041d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb040a7180 0x7fbb041d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 10 monclient: _finish_hunting 0 2024-08-19T10:51:41.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 1 monclient: found mon.noname-a 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb09701640 1 -- 172.21.15.89:0/1865289281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb04122cc0 con 0x7fbb04139620 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb03fff640 10 monclient: get_auth_request con 0x7fbb040a7180 auth_method 0 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.188+0000 7fbb03fff640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb040a7180 0x7fbb041d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb09701640 1 -- 172.21.15.89:0/1865289281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb04139620 msgr2=0x7fbb0413d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb09701640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb04139620 0x7fbb0413d340 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c730 tx=0x7fbafc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbafc017070 con 0x7fbb04139620 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 10 monclient: dump: 2024-08-19T10:51:41.195 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.196 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 10 monclient: _finish_auth 0 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 10 monclient: _check_auth_tickets 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc00ce40 con 0x7fbb04139620 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbafc007070 con 0x7fbb04139620 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb0b164640 5 monclient: authenticate success, global_id 7245 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb02ffd640 4 set_mon_vals no callback set 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:51:41.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 10 monclient: dump: 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.198 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb0b164640 10 monclient: _renew_subs 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb0b164640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb0b164640 1 -- 172.21.15.89:0/1865289281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0413f5d0 con 0x7fbb04139620 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb0b164640 10 monclient: _renew_subs 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb0b164640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb0b164640 1 -- 172.21.15.89:0/1865289281 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb041229e0 con 0x7fbb04139620 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb0b164640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb0b164640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413d880 0x7fbb0413dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb0b164640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb0b164640 1 -- 172.21.15.89:0/1865289281 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb040a6c40 con 0x7fbb0413d880 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbafc007250 con 0x7fbb04139620 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbae803be00 0x7fbae803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fbafc04eac0 con 0x7fbb04139620 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb08f00640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413d880 0x7fbb0413dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb08f00640 10 monclient: get_auth_request con 0x7fbb0413d880 auth_method 0 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb08f00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb08f00640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb08f00640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb03fff640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbae803be00 0x7fbae803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb08f00640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb08f00640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb08f00640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb03fff640 10 monclient: get_auth_request con 0x7fbae803be00 auth_method 0 2024-08-19T10:51:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb08f00640 10 monclient(hunting): handle_auth_done global_id 45028 payload 824 2024-08-19T10:51:41.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.192+0000 7fbb08f00640 1 -- 172.21.15.89:0/1865289281 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb04064590 con 0x7fbb0413d880 2024-08-19T10:51:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb03fff640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbae803be00 0x7fbae803e2c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00ad000 tx=0x7fbaf00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb08f00640 1 -- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413d880 msgr2=0x7fbb0413dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb08f00640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413d880 0x7fbb0413dc60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8003990 tx=0x7fbaf800b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbaf8016070 con 0x7fbb0413d880 2024-08-19T10:51:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb017fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb017fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:51:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb017fa640 10 monclient: dump: 2024-08-19T10:51:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.203 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.204 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbaf8005070 con 0x7fbb0413d880 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb017fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb017fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413d880 msgr2=0x7fbb0413dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb017fa640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413d880 0x7fbb0413dc60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8003990 tx=0x7fbaf800b810 comp rx=0 tx=0).stop 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413d880 msgr2=0x7fbb0413dc60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.196+0000 7fbb017fa640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413d880 0x7fbb0413dc60 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8003990 tx=0x7fbaf800b810 comp rx=0 tx=0).stop 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.200+0000 7fbb0b164640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.200+0000 7fbb0b164640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413e1a0 0x7fbb0409fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.200+0000 7fbb0b164640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.200+0000 7fbb0b164640 1 -- 172.21.15.89:0/1865289281 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0413f850 con 0x7fbb0413e1a0 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413e1a0 0x7fbb0409fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 10 monclient: get_auth_request con 0x7fbb0413e1a0 auth_method 0 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 10 monclient(hunting): handle_auth_done global_id 45037 payload 824 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 1 -- 172.21.15.89:0/1865289281 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbb0406cab0 con 0x7fbb0413e1a0 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 1 -- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413e1a0 msgr2=0x7fbb0409fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.204+0000 7fbb08f00640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413e1a0 0x7fbb0409fbe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8003990 tx=0x7fbaf8003800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbaf8016030 con 0x7fbb0413e1a0 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb017fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb017fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb017fa640 10 monclient: dump: 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.207 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.208 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9402 (secure 0 0 0) 0x7fbaf8019ab0 con 0x7fbb0413e1a0 2024-08-19T10:51:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb017fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb017fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413e1a0 msgr2=0x7fbb0409fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb017fa640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413e1a0 0x7fbb0409fbe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8003990 tx=0x7fbaf8003800 comp rx=0 tx=0).stop 2024-08-19T10:51:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb017fa640 1 -- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413e1a0 msgr2=0x7fbb0409fbe0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:41.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb017fa640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0413e1a0 0x7fbb0409fbe0 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8003990 tx=0x7fbaf8003800 comp rx=0 tx=0).stop 2024-08-19T10:51:41.209 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:41.209 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:41.209 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:41.209 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:41.209 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 116, 2024-08-19T10:51:41.209 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 424330, 2024-08-19T10:51:41.209 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:41.209 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:41.209 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:41.209 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:41.209 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: 7 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.210 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.211 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:41.212 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:41.213 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:41.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.215 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.216 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.217 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:41.218 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.221 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:41.224 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:41.226 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:41.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:41.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:41.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.232 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:41.232 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:41.232 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:41.232 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.232 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.232 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:41.232 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb0b164640 1 -- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbae803be00 msgr2=0x7fbae803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb0b164640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbae803be00 0x7fbae803e2c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00ad000 tx=0x7fbaf00a6000 comp rx=0 tx=0).stop 2024-08-19T10:51:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb0b164640 10 monclient: shutdown 2024-08-19T10:51:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb0b164640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb0b164640 1 -- 172.21.15.89:0/1865289281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb04139620 msgr2=0x7fbb0413d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb0b164640 1 --2- 172.21.15.89:0/1865289281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb04139620 0x7fbb0413d340 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c730 tx=0x7fbafc00cbf0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb09701640 1 -- 172.21.15.89:0/1865289281 reap_dead start 2024-08-19T10:51:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb09701640 1 -- 172.21.15.89:0/1865289281 reap_dead start 2024-08-19T10:51:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb0b164640 1 -- 172.21.15.89:0/1865289281 shutdown_connections 2024-08-19T10:51:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb0b164640 1 -- 172.21.15.89:0/1865289281 >> 172.21.15.89:0/1865289281 conn(0x7fbb04074b40 msgr2=0x7fbb040a9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb0b164640 1 -- 172.21.15.89:0/1865289281 shutdown_connections 2024-08-19T10:51:41.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.208+0000 7fbb0b164640 1 -- 172.21.15.89:0/1865289281 wait complete. 2024-08-19T10:51:41.235 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:51:41.235 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-08-19T10:51:41.235 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:51:41.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient: get_monmap_and_config 2024-08-19T10:51:41.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient: build_initial_monmap 2024-08-19T10:51:41.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient: monmap: 2024-08-19T10:51:41.347 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:41.353572+0000 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:41.353572+0000 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:41.348 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:41.349 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:41.349 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:41.349 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:41.349 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:41.349 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 1 Processor -- start 2024-08-19T10:51:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 1 -- start start 2024-08-19T10:51:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient: init 2024-08-19T10:51:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient: _add_conns ranks=[5,8,0,7,1,10,6,9,2,3,4] 2024-08-19T10:51:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faed00a5270 0x7faed00a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient(hunting): picked mon.noname-h con 0x7faed00a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:51:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faed00a6d30 0x7faed00a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient(hunting): picked mon.noname-e con 0x7faed00a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faed01395b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:51:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faed00a8710 con 0x7faed01395b0 2024-08-19T10:51:41.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7faed00a7650 con 0x7faed00a5270 2024-08-19T10:51:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7faed00a5a40 con 0x7faed00a6d30 2024-08-19T10:51:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed7915640 10 monclient(hunting): authenticate will time out at 1751.169976s 2024-08-19T10:51:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed4eb0640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faed00a6d30 msgr2=0x7faed00a7110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:51:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed4eb0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faed00a6d30 0x7faed00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed56b1640 1 -- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faed00a5270 msgr2=0x7faed00a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3302/0 2024-08-19T10:51:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed56b1640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faed00a5270 0x7faed00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed5eb2640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7faed01395b0 0x7faed0139a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34538/0 (socket says 172.21.15.89:34538) 2024-08-19T10:51:41.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.352+0000 7faed5eb2640 1 -- 172.21.15.89:0/2214864264 learned_addr learned my addr 172.21.15.89:0/2214864264 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:51:41.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed5eb2640 1 -- 172.21.15.89:0/2214864264 >> v1:172.21.15.89:6789/0 conn(0x7faed01395b0 legacy=0x7faed0139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2816754102 0 0) 0x7faec80017e0 con 0x7faed01395b0 2024-08-19T10:51:41.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): _negotiate 2024-08-19T10:51:41.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): my global_id is 7254 2024-08-19T10:51:41.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faebc003530 con 0x7faed01395b0 2024-08-19T10:51:41.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 488223546 0 0) 0x7faebc003530 con 0x7faed01395b0 2024-08-19T10:51:41.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faed00a6d30 msgr2=0x7faed00a7110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:41.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faed00a6d30 0x7faed00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faed00a5270 msgr2=0x7faed00a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faed00a5270 0x7faed00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:51:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 monclient: found mon.noname-a-legacy 2024-08-19T10:51:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.89:6789/0 2024-08-19T10:51:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed00a81d0 con 0x7faed01395b0 2024-08-19T10:51:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient: _finish_auth 0 2024-08-19T10:51:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:51:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 3927028378 0 0) 0x7faec8004c30 con 0x7faed01395b0 2024-08-19T10:51:41.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-08-19T10:51:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient: dump: 2024-08-19T10:51:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.359 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] but i'm connected to v1:172.21.15.89:6789/0, reconnecting 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 >> v1:172.21.15.89:6789/0 conn(0x7faed01395b0 legacy=0x7faed0139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient: _add_conns ranks=[1,8,7,5,6,0,2,4,3] 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faebc004210 0x7faebc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): picked mon.f con 0x7faebc004210 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:51:41.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faed00a6d30 0x7faebc004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): picked mon.e con 0x7faed00a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faebc0099c0 0x7faebc00bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed4eb0640 1 -- 172.21.15.89:0/2214864264 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faed00a6d30 msgr2=0x7faebc004f80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): picked mon.i con 0x7faebc0099c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faed00a7650 con 0x7faebc004210 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7faed00a5a40 con 0x7faebc0099c0 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed4eb0640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faed00a6d30 0x7faebc004f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7faebc00c590 con 0x7faed00a6d30 2024-08-19T10:51:41.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faec7fff640 10 monclient(hunting): _finish_auth 0 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faebc004210 0x7faebc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed5eb2640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faebc0099c0 0x7faebc00bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient(hunting): get_auth_request con 0x7faebc004210 auth_method 0 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed5eb2640 10 monclient(hunting): get_auth_request con 0x7faebc0099c0 auth_method 0 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed5eb2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed5eb2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed5eb2640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:51:41.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed5eb2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed5eb2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed5eb2640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient(hunting): handle_auth_done global_id 7254 payload 293 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 1 -- 172.21.15.89:0/2214864264 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faed00a6d30 msgr2=0x7faebc004f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faed00a6d30 0x7faebc004f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 1 -- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faebc0099c0 msgr2=0x7faebc00bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faebc0099c0 0x7faebc00bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient: _finish_hunting 0 2024-08-19T10:51:41.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 1 monclient: found mon.f 2024-08-19T10:51:41.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:41.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:51:41.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 1 -- 172.21.15.89:0/2214864264 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faebc003530 con 0x7faebc004210 2024-08-19T10:51:41.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient: _finish_auth 0 2024-08-19T10:51:41.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 10 monclient: _check_auth_tickets 2024-08-19T10:51:41.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed56b1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:41.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed7915640 5 monclient: authenticate success, global_id 7254 2024-08-19T10:51:41.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.356+0000 7faed7915640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:41.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed56b1640 1 -- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faebc004210 msgr2=0x7faebc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed56b1640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faebc004210 0x7faebc004810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faecc003940 tx=0x7faecc01c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd6b6f4e5b06bb64 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faecc009520 con 0x7faebc004210 2024-08-19T10:51:41.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faec7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faec7fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:51:41.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faec7fff640 10 monclient: dump: 2024-08-19T10:51:41.372 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.372 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.372 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.372 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.372 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faecc0039c0 con 0x7faebc004210 2024-08-19T10:51:41.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:51:41.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faec7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:41.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed56b1640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faebc004210 0x7faebc004810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faecc003940 tx=0x7faecc01c420 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7faecc005070 mon_map magic: 0 2024-08-19T10:51:41.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:41.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 4 set_mon_vals no callback set 2024-08-19T10:51:41.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 10 monclient: shutdown 2024-08-19T10:51:41.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:41.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 1 -- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faebc004210 msgr2=0x7faebc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 1 --2- 172.21.15.89:0/2214864264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faebc004210 0x7faebc004810 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faecc003940 tx=0x7faecc01c420 comp rx=0 tx=0).stop 2024-08-19T10:51:41.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed5eb2640 1 -- 172.21.15.89:0/2214864264 reap_dead start 2024-08-19T10:51:41.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faec7fff640 1 -- 172.21.15.89:0/2214864264 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 0 0 0) 0x7faecc005070 con 0x7faebc004210 2024-08-19T10:51:41.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faec7fff640 10 monclient: discarding stray monitor message mon_map magic: 0 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 1 -- 172.21.15.89:0/2214864264 shutdown_connections 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 1 -- 172.21.15.89:0/2214864264 >> 172.21.15.89:0/2214864264 conn(0x7faed0069a80 msgr2=0x7faed00abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 1 -- 172.21.15.89:0/2214864264 shutdown_connections 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 1 -- 172.21.15.89:0/2214864264 wait complete. 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.360+0000 7faed7915640 10 monclient: build_initial_monmap 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient: monmap: 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.375 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 1 Processor -- start 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 1 -- start start 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient: init 2024-08-19T10:51:41.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient: _add_conns ranks=[8,7,4,2,5,6,3,0,1] 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faebc004210 0x7faed01d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient(hunting): picked mon.noname-e con 0x7faebc004210 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faed01377a0 0x7faed0137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient(hunting): picked mon.noname-i con 0x7faed01377a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faed00a0bf0 0x7faed00a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient(hunting): picked mon.noname-c con 0x7faed00a0bf0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7faed00a57d0 con 0x7faed00a0bf0 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7faed00a5650 con 0x7faed01377a0 2024-08-19T10:51:41.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7faed0142230 con 0x7faebc004210 2024-08-19T10:51:41.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:41.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed7915640 10 monclient(hunting): authenticate will time out at 1751.180929s 2024-08-19T10:51:41.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed5eb2640 1 -- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faed00a0bf0 msgr2=0x7faed00a1030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3302/0 2024-08-19T10:51:41.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed56b1640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faebc004210 msgr2=0x7faed01d13e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:51:41.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed5eb2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faed00a0bf0 0x7faed00a1030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:41.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed56b1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faebc004210 0x7faed01d13e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:41.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faed01377a0 0x7faed0137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faed01377a0 0x7faed0137bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:35068/0 (socket says 172.21.15.89:35068) 2024-08-19T10:51:41.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 1 -- 172.21.15.89:0/3186662369 learned_addr learned my addr 172.21.15.89:0/3186662369 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 10 monclient(hunting): get_auth_request con 0x7faed01377a0 auth_method 0 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 10 monclient(hunting): handle_auth_done global_id 64393 payload 824 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 1 -- 172.21.15.89:0/3186662369 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faebc004210 msgr2=0x7faed01d13e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7faebc004210 0x7faed01d13e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 1 -- 172.21.15.89:0/3186662369 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faed00a0bf0 msgr2=0x7faed00a1030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:51:41.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7faed00a0bf0 0x7faed00a1030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 10 monclient: _finish_hunting 0 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 1 monclient: found mon.noname-i 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.364+0000 7faed4eb0640 1 -- 172.21.15.89:0/3186662369 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed01224f0 con 0x7faed01377a0 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed4eb0640 1 -- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faed01377a0 msgr2=0x7faed0137bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed4eb0640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faed01377a0 0x7faed0137bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faec000b580 tx=0x7faec000ba40 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faec0016070 con 0x7faed01377a0 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 10 monclient: dump: 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.380 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.381 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faec0003de0 con 0x7faed01377a0 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed7915640 5 monclient: authenticate success, global_id 64393 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faec000f3e0 con 0x7faed01377a0 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec77fe640 4 set_mon_vals no callback set 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 10 monclient: dump: 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.382 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.383 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed7915640 10 monclient: _renew_subs 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed7915640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed7915640 1 -- 172.21.15.89:0/3186662369 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed0123e90 con 0x7faed01377a0 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed7915640 10 monclient: _renew_subs 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed7915640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed7915640 1 -- 172.21.15.89:0/3186662369 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7faed0122970 con 0x7faed01377a0 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed7915640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed7915640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed0123a60 0x7faed00a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed7915640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed7915640 1 -- 172.21.15.89:0/3186662369 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faed00a1ab0 con 0x7faed0123a60 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed56b1640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed0123a60 0x7faed00a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed56b1640 10 monclient: get_auth_request con 0x7faed0123a60 auth_method 0 2024-08-19T10:51:41.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed56b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed56b1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed56b1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed56b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed56b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed56b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 <== mon.7 v2:172.21.15.190:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7faec00050d0 con 0x7faed01377a0 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7faeb403be00 0x7faeb403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 <== mon.7 v2:172.21.15.190:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7faec004ec90 con 0x7faed01377a0 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed56b1640 10 monclient(hunting): handle_auth_done global_id 35697 payload 824 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed56b1640 1 -- 172.21.15.89:0/3186662369 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faed00a67f0 con 0x7faed0123a60 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed5eb2640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7faeb403be00 0x7faeb403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.368+0000 7faed5eb2640 10 monclient: get_auth_request con 0x7faeb403be00 auth_method 0 2024-08-19T10:51:41.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faed56b1640 1 -- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed0123a60 msgr2=0x7faed00a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faed56b1640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed0123a60 0x7faed00a1570 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faecc008000 tx=0x7faebc005840 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faecc0bb070 con 0x7faed0123a60 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faec5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faec5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faec5ffb640 10 monclient: dump: 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.386 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faed5eb2640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7faeb403be00 0x7faeb403e2c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faed00a34f0 tx=0x7faec80020c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7faecc0050c0 con 0x7faed0123a60 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faec5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faec5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:41.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed0123a60 msgr2=0x7faed00a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faec5ffb640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed0123a60 0x7faed00a1570 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faecc008000 tx=0x7faebc005840 comp rx=0 tx=0).stop 2024-08-19T10:51:41.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed0123a60 msgr2=0x7faed00a1570 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:41.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.372+0000 7faec5ffb640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed0123a60 0x7faed00a1570 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.376+0000 7faed7915640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:41.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.376+0000 7faed7915640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed009c9e0 0x7faed009cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.376+0000 7faed7915640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.376+0000 7faed7915640 1 -- 172.21.15.89:0/3186662369 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faed0064590 con 0x7faed009c9e0 2024-08-19T10:51:41.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed009c9e0 0x7faed009cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 10 monclient: get_auth_request con 0x7faed009c9e0 auth_method 0 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 10 monclient(hunting): handle_auth_done global_id 35706 payload 824 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 1 -- 172.21.15.89:0/3186662369 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faed0140a20 con 0x7faed009c9e0 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 1 -- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed009c9e0 msgr2=0x7faed009cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.380+0000 7faed56b1640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed009c9e0 0x7faed009cdf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faecc01c250 tx=0x7faecc0090e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faecc0c1070 con 0x7faed009c9e0 2024-08-19T10:51:41.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faec5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faec5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faec5ffb640 10 monclient: dump: 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.391 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9402 (secure 0 0 0) 0x7faecc0acc30 con 0x7faed009c9e0 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faec5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faec5ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed009c9e0 msgr2=0x7faed009cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faec5ffb640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed009c9e0 0x7faed009cdf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faecc01c250 tx=0x7faecc0090e0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faec5ffb640 1 -- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed009c9e0 msgr2=0x7faed009cdf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:41.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faec5ffb640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faed009c9e0 0x7faed009cdf0 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faecc01c250 tx=0x7faecc0090e0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.393 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:41.393 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:41.393 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:41.393 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:41.393 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 116, 2024-08-19T10:51:41.393 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 546424, 2024-08-19T10:51:41.393 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:41.393 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:41.393 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: 7 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.394 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.395 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:41.396 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:41.397 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.398 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.400 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.403 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:41.403 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:41.403 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.403 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:41.403 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.403 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.405 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.408 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:41.409 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:41.412 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:41.413 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:41.415 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:41.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.417 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:41.417 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:41.417 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:51:41.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.417 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.417 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:41.417 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:41.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed7915640 1 -- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7faeb403be00 msgr2=0x7faeb403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed7915640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7faeb403be00 0x7faeb403e2c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faed00a34f0 tx=0x7faec80020c0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed7915640 10 monclient: shutdown 2024-08-19T10:51:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed7915640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed7915640 1 -- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faed01377a0 msgr2=0x7faed0137bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed7915640 1 --2- 172.21.15.89:0/3186662369 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faed01377a0 0x7faed0137bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faec000b580 tx=0x7faec000ba40 comp rx=0 tx=0).stop 2024-08-19T10:51:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed5eb2640 1 -- 172.21.15.89:0/3186662369 reap_dead start 2024-08-19T10:51:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed5eb2640 1 -- 172.21.15.89:0/3186662369 reap_dead start 2024-08-19T10:51:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed7915640 1 -- 172.21.15.89:0/3186662369 shutdown_connections 2024-08-19T10:51:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed7915640 1 -- 172.21.15.89:0/3186662369 >> 172.21.15.89:0/3186662369 conn(0x7faed0069a80 msgr2=0x7faed013b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed7915640 1 -- 172.21.15.89:0/3186662369 shutdown_connections 2024-08-19T10:51:41.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.384+0000 7faed7915640 1 -- 172.21.15.89:0/3186662369 wait complete. 2024-08-19T10:51:41.420 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:51:41.420 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-08-19T10:51:41.420 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:51:41.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient: get_monmap_and_config 2024-08-19T10:51:41.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient: build_initial_monmap 2024-08-19T10:51:41.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient: monmap: 2024-08-19T10:51:41.535 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:41.535 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.535 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:51:41.541617+0000 2024-08-19T10:51:41.535 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:51:41.541617+0000 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:51:41.536 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:51:41.537 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 1 Processor -- start 2024-08-19T10:51:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 1 -- start start 2024-08-19T10:51:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient: init 2024-08-19T10:51:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:41.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient: _add_conns ranks=[3,10,1,0,4,2,8,9,7,6,5] 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b680a5270 0x7f4b680a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient(hunting): picked mon.noname-g con 0x7f4b680a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4b680a6d30 0x7f4b680a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient(hunting): picked mon.noname-f con 0x7f4b680a6d30 addr v2:172.21.15.190:3300/0 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4b681395b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b680a8710 con 0x7f4b681395b0 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f4b680a7650 con 0x7f4b680a6d30 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4b680a5a40 con 0x7f4b680a5270 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:41.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6eb3b640 10 monclient(hunting): authenticate will time out at 1751.358228s 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b67fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4b680a6d30 0x7f4b680a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b680a5270 0x7f4b680a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b67fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4b680a6d30 0x7f4b680a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:49518/0 (socket says 172.21.15.89:49518) 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b67fff640 1 -- 172.21.15.89:0/158248112 learned_addr learned my addr 172.21.15.89:0/158248112 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b67fff640 10 monclient(hunting): get_auth_request con 0x7f4b680a6d30 auth_method 0 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b67fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b67fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 10 monclient(hunting): get_auth_request con 0x7f4b680a5270 auth_method 0 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 10 monclient(hunting): handle_auth_done global_id 35715 payload 274 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 1 -- 172.21.15.89:0/158248112 >> v2:172.21.15.190:3300/0 conn(0x7f4b680a6d30 msgr2=0x7f4b680a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 1 --2- 172.21.15.89:0/158248112 >> v2:172.21.15.190:3300/0 conn(0x7f4b680a6d30 0x7f4b680a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 1 -- 172.21.15.89:0/158248112 >> v1:172.21.15.190:6789/0 conn(0x7f4b681395b0 legacy=0x7f4b68139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 10 monclient: _finish_hunting 0 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 1 monclient: found mon.noname-g 2024-08-19T10:51:41.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:41.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:51:41.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.540+0000 7f4b6c8d7640 1 -- 172.21.15.89:0/158248112 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b680a81d0 con 0x7f4b680a5270 2024-08-19T10:51:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.544+0000 7f4b6c8d7640 1 -- 172.21.15.89:0/158248112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b680a5270 msgr2=0x7f4b680a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.544+0000 7f4b6c8d7640 1 --2- 172.21.15.89:0/158248112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b680a5270 0x7f4b680a67f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009d90 tx=0x7f4b6001c650 comp rx=0 tx=0).ready entity=mon.3 client_cookie=1c0e886e8c0459df server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 1 -- 172.21.15.89:0/158248112 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b600ac070 con 0x7f4b680a5270 2024-08-19T10:51:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:51:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 10 monclient: dump: 2024-08-19T10:51:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.547 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 10 monclient: _finish_auth 0 2024-08-19T10:51:41.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 10 monclient: _check_auth_tickets 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 1 -- 172.21.15.89:0/158248112 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b600a6910 con 0x7f4b680a5270 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 1 -- 172.21.15.89:0/158248112 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b600a7480 con 0x7f4b680a5270 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b677fe640 10 monclient: dump: 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.549 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 5 monclient: authenticate success, global_id 35715 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient: get_monmap_and_config success 2024-08-19T10:51:41.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 4 set_mon_vals no callback set 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient: shutdown 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/158248112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b680a5270 msgr2=0x7f4b680a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 --2- 172.21.15.89:0/158248112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b680a5270 0x7f4b680a67f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009d90 tx=0x7f4b6001c650 comp rx=0 tx=0).stop 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/158248112 shutdown_connections 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 --2- 172.21.15.89:0/158248112 >> v2:172.21.15.190:3300/0 conn(0x7f4b680a6d30 0x7f4b680a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 --2- 172.21.15.89:0/158248112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b680a5270 0x7f4b680a67f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/158248112 >> 172.21.15.89:0/158248112 conn(0x7f4b68069a80 msgr2=0x7f4b680abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/158248112 shutdown_connections 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/158248112 wait complete. 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient: build_initial_monmap 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient: monmap: 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:51:41.551 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:51:41.552 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 Processor -- start 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 -- start start 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient: init 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient: _reopen_session rank -1 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient: _add_conns ranks=[1,3,8,2,4,5,7,0,6] 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b680a5270 0x7f4b6813fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient(hunting): picked mon.noname-f con 0x7f4b680a5270 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b680a6d30 0x7f4b681384c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient(hunting): picked mon.noname-g con 0x7f4b680a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b681395b0 0x7f4b6813d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient(hunting): picked mon.noname-e con 0x7f4b681395b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:51:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4b680a57d0 con 0x7f4b680a5270 2024-08-19T10:51:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4b680a5650 con 0x7f4b680a6d30 2024-08-19T10:51:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4b68144240 con 0x7f4b681395b0 2024-08-19T10:51:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient(hunting): _renew_subs 2024-08-19T10:51:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6eb3b640 10 monclient(hunting): authenticate will time out at 1751.366838s 2024-08-19T10:51:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6d0d8640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b681395b0 msgr2=0x7f4b6813d7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:51:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6d0d8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b681395b0 0x7f4b6813d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:51:41.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b680a5270 0x7f4b6813fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b67fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b680a6d30 0x7f4b681384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b680a5270 0x7f4b6813fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:49528/0 (socket says 172.21.15.89:49528) 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 1 -- 172.21.15.89:0/2455228921 learned_addr learned my addr 172.21.15.89:0/2455228921 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 10 monclient(hunting): get_auth_request con 0x7f4b680a5270 auth_method 0 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b67fff640 10 monclient(hunting): get_auth_request con 0x7f4b680a6d30 auth_method 0 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b67fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b67fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 10 monclient(hunting): handle_auth_done global_id 45055 payload 824 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 1 -- 172.21.15.89:0/2455228921 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b681395b0 msgr2=0x7f4b6813d7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f4b681395b0 0x7f4b6813d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 1 -- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b680a6d30 msgr2=0x7f4b681384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b680a6d30 0x7f4b681384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 10 monclient: _finish_hunting 0 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 1 monclient: found mon.noname-f 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:51:41.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:51:41.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.548+0000 7f4b6c8d7640 1 -- 172.21.15.89:0/2455228921 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b680a24f0 con 0x7f4b680a5270 2024-08-19T10:51:41.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6c8d7640 1 -- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b680a5270 msgr2=0x7f4b6813fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6c8d7640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b680a5270 0x7f4b6813fcb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4b6001c480 tx=0x7f4b600193f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b600ac070 con 0x7f4b680a5270 2024-08-19T10:51:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:51:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 10 monclient: dump: 2024-08-19T10:51:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.561 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 10 monclient: _finish_auth 0 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 10 monclient: _check_auth_tickets 2024-08-19T10:51:41.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b60003dc0 con 0x7f4b680a5270 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b600a7480 con 0x7f4b680a5270 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6eb3b640 5 monclient: authenticate success, global_id 45055 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6eb3b640 10 monclient: _renew_subs 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b667fc640 4 set_mon_vals no callback set 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6eb3b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/2455228921 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b68140640 con 0x7f4b680a5270 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b657fa640 10 monclient: dump: 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.563 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.564 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6eb3b640 10 monclient: _renew_subs 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6eb3b640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/2455228921 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b68122e50 con 0x7f4b680a5270 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6eb3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6eb3b640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b681c90d0 0x7f4b6802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6eb3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.552+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/2455228921 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4b68122ad0 con 0x7f4b681c90d0 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b681c90d0 0x7f4b6802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 10 monclient: get_auth_request con 0x7f4b681c90d0 auth_method 0 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f4b600af070 con 0x7f4b680a5270 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 10 monclient(hunting): handle_auth_done global_id 64402 payload 824 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 1 -- 172.21.15.89:0/2455228921 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b680a67f0 con 0x7f4b681c90d0 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b657fa640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b4c03be00 0x7f4b4c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f4b600e9ef0 con 0x7f4b680a5270 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b6d0d8640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b4c03be00 0x7f4b4c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b6d0d8640 10 monclient: get_auth_request con 0x7f4b4c03be00 auth_method 0 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 1 -- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b681c90d0 msgr2=0x7f4b6802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b67fff640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b681c90d0 0x7f4b6802d1d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b6813f7c0 tx=0x7f4b50011d60 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b5000fe50 con 0x7f4b681c90d0 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b657fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b657fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.556+0000 7f4b657fa640 10 monclient: dump: 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.567 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.560+0000 7f4b6d0d8640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b4c03be00 0x7f4b4c03e2c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4b58003780 tx=0x7f4b580079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.560+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4b5000fe50 con 0x7f4b681c90d0 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.560+0000 7f4b657fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.560+0000 7f4b657fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:51:41.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.560+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b681c90d0 msgr2=0x7f4b6802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.560+0000 7f4b657fa640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b681c90d0 0x7f4b6802d1d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b6813f7c0 tx=0x7f4b50011d60 comp rx=0 tx=0).stop 2024-08-19T10:51:41.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.560+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b681c90d0 msgr2=0x7f4b6802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:41.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.560+0000 7f4b657fa640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b681c90d0 0x7f4b6802d1d0 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b6813f7c0 tx=0x7f4b50011d60 comp rx=0 tx=0).stop 2024-08-19T10:51:41.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b6eb3b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:51:41.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b6eb3b640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b6802d710 0x7f4b6802db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:51:41.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b6eb3b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:51:41.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/2455228921 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f4b68064590 con 0x7f4b6802d710 2024-08-19T10:51:41.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b67fff640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b6802d710 0x7f4b6802db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b67fff640 10 monclient: get_auth_request con 0x7f4b6802d710 auth_method 0 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b67fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b67fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b67fff640 10 monclient(hunting): handle_auth_done global_id 64411 payload 824 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.564+0000 7f4b67fff640 1 -- 172.21.15.89:0/2455228921 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4b680a06a0 con 0x7f4b6802d710 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b67fff640 1 -- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b6802d710 msgr2=0x7f4b6802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b67fff640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b6802d710 0x7f4b6802db20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b6813f7c0 tx=0x7f4b500046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:51:41.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4b500148b0 con 0x7f4b6802d710 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b657fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b657fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b657fa640 10 monclient: dump: 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:51:41.572 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:51:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:51:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:51:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:51:41.573 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:51:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9235 (secure 0 0 0) 0x7f4b500273a0 con 0x7f4b6802d710 2024-08-19T10:51:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b657fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:51:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b657fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:51:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b6802d710 msgr2=0x7f4b6802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b657fa640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b6802d710 0x7f4b6802db20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b6813f7c0 tx=0x7f4b500046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b657fa640 1 -- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b6802d710 msgr2=0x7f4b6802db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:51:41.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b657fa640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4b6802d710 0x7f4b6802db20 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b6813f7c0 tx=0x7f4b500046d0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.574 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:51:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:51:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 116, 2024-08-19T10:51:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 167965, 2024-08-19T10:51:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:51:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:51:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:51:41.574 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: 7 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.575 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.576 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:51:41.577 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:51:41.578 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:51:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:51:41.582 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:51:41.586 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:51:41.587 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.588 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.590 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:51:41.591 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:51:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:51:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.592 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:51:41.593 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:51:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:51:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:51:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:51:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:51:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:51:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:51:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:51:41.595 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:51:41.596 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:51:41.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:51:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:51:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:51:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:51:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:51:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:51:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:51:41.598 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:51:41.598 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:51:41.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b4c03be00 msgr2=0x7f4b4c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6eb3b640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f4b4c03be00 0x7f4b4c03e2c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4b58003780 tx=0x7f4b580079e0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6eb3b640 10 monclient: shutdown 2024-08-19T10:51:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6eb3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:51:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b680a5270 msgr2=0x7f4b6813fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:51:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6eb3b640 1 --2- 172.21.15.89:0/2455228921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b680a5270 0x7f4b6813fcb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4b6001c480 tx=0x7f4b600193f0 comp rx=0 tx=0).stop 2024-08-19T10:51:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6d0d8640 1 -- 172.21.15.89:0/2455228921 reap_dead start 2024-08-19T10:51:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6d0d8640 1 -- 172.21.15.89:0/2455228921 reap_dead start 2024-08-19T10:51:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/2455228921 shutdown_connections 2024-08-19T10:51:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/2455228921 >> 172.21.15.89:0/2455228921 conn(0x7f4b68069a80 msgr2=0x7f4b680ab1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:51:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/2455228921 shutdown_connections 2024-08-19T10:51:41.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:51:41.568+0000 7f4b6eb3b640 1 -- 172.21.15.89:0/2455228921 wait complete. 2024-08-19T10:51:41.601 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:51:41.601 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-08-19T10:53:11.602 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-08-19T10:53:11.602 INFO:tasks.ceph.mon.e:Restarting daemon 2024-08-19T10:53:11.603 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-08-19T10:53:11.607 INFO:tasks.ceph.mon.e:Started 2024-08-19T10:53:11.607 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.h 2024-08-19T10:53:11.607 INFO:tasks.ceph.mon.h:Restarting daemon 2024-08-19T10:53:11.607 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-08-19T10:53:11.610 INFO:tasks.ceph.mon.h:Started 2024-08-19T10:53:11.610 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-08-19T10:53:11.610 INFO:tasks.ceph.mon.c:Restarting daemon 2024-08-19T10:53:11.610 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-08-19T10:53:11.613 INFO:tasks.ceph.mon.c:Started 2024-08-19T10:53:11.613 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-08-19T10:53:11.613 INFO:tasks.ceph.mon.a:Sent signal 19 2024-08-19T10:53:11.613 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-08-19T10:53:26.615 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-08-19T10:53:26.616 INFO:tasks.ceph.mon.a:Sent signal 18 2024-08-19T10:53:26.616 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:53:26.616 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:53:26.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient: get_monmap_and_config 2024-08-19T10:53:26.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient: build_initial_monmap 2024-08-19T10:53:26.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient: monmap: 2024-08-19T10:53:26.753 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:26.753 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:26.753 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:26.759450+0000 2024-08-19T10:53:26.754 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:26.759450+0000 2024-08-19T10:53:26.754 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:26.754 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:26.754 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:26.754 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:26.754 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:26.754 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:26.754 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:26.754 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:26.755 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:26.755 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:26.755 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:26.755 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:26.755 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:26.755 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:26.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 1 Processor -- start 2024-08-19T10:53:26.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 1 -- start start 2024-08-19T10:53:26.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient: init 2024-08-19T10:53:26.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:26.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient: _add_conns ranks=[10,3,5,1,0,9,8,6,2,7,4] 2024-08-19T10:53:26.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04f40a3620 0x7f04f40a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:26.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient(hunting): picked mon.noname-f con 0x7f04f40a3620 addr v2:172.21.15.190:3300/0 2024-08-19T10:53:26.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f409c9e0 0x7f04f40a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:26.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient(hunting): picked mon.noname-g con 0x7f04f409c9e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:26.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f41373f0 0x7f04f4137830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:26.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient(hunting): picked mon.noname-h con 0x7f04f41373f0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f04f409dab0 con 0x7f04f40a3620 2024-08-19T10:53:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f04f40a1f20 con 0x7f04f409c9e0 2024-08-19T10:53:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f04f40a4480 con 0x7f04f41373f0 2024-08-19T10:53:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:26.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.754+0000 7f04f9643640 10 monclient(hunting): authenticate will time out at 1856.575925s 2024-08-19T10:53:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f409c9e0 0x7f04f40a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f37fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f41373f0 0x7f04f4137830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f2ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04f40a3620 0x7f04f40a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f37fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f41373f0 0x7f04f4137830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:36648/0 (socket says 172.21.15.89:36648) 2024-08-19T10:53:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f409c9e0 0x7f04f40a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:35178/0 (socket says 172.21.15.89:35178) 2024-08-19T10:53:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f37fe640 1 -- 172.21.15.89:0/161027372 learned_addr learned my addr 172.21.15.89:0/161027372 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f37fe640 10 monclient(hunting): get_auth_request con 0x7f04f41373f0 auth_method 0 2024-08-19T10:53:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:26.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 10 monclient(hunting): get_auth_request con 0x7f04f409c9e0 auth_method 0 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f2ffd640 10 monclient(hunting): get_auth_request con 0x7f04f40a3620 auth_method 0 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:26.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f37fe640 1 -- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f41373f0 msgr2=0x7f04f4137830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f37fe640 1 -- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f41373f0 msgr2=0x7f04f4137830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f37fe640 1 --2- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f41373f0 0x7f04f4137830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f37fe640 1 --2- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f41373f0 0x7f04f4137830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 10 monclient(hunting): handle_auth_done global_id 35733 payload 274 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 1 -- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f41373f0 msgr2=0x7f04f4137830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 1 --2- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f41373f0 0x7f04f4137830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 1 -- 172.21.15.89:0/161027372 >> v2:172.21.15.190:3300/0 conn(0x7f04f40a3620 msgr2=0x7f04f40a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 1 --2- 172.21.15.89:0/161027372 >> v2:172.21.15.190:3300/0 conn(0x7f04f40a3620 0x7f04f40a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 10 monclient: _finish_hunting 0 2024-08-19T10:53:26.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 1 monclient: found mon.noname-g 2024-08-19T10:53:26.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:26.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:53:26.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.758+0000 7f04f27fc640 1 -- 172.21.15.89:0/161027372 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f40a1790 con 0x7f04f409c9e0 2024-08-19T10:53:26.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f27fc640 1 -- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f409c9e0 msgr2=0x7f04f40a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:26.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f27fc640 1 --2- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f409c9e0 0x7f04f40a3f40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f04e80097f0 tx=0x7f04e801c330 comp rx=0 tx=0).ready entity=mon.3 client_cookie=f5d0ec26773fc4e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:26.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 1 -- 172.21.15.89:0/161027372 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04e80b0070 con 0x7f04f409c9e0 2024-08-19T10:53:26.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 10 monclient: dump: 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:26.774 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 10 monclient: _finish_auth 0 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 1 -- 172.21.15.89:0/161027372 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e80aa810 con 0x7f04f409c9e0 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 1 -- 172.21.15.89:0/161027372 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04e80a9300 con 0x7f04f409c9e0 2024-08-19T10:53:26.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f1ffb640 10 monclient: dump: 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:26.776 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:26.777 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 5 monclient: authenticate success, global_id 35733 2024-08-19T10:53:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 4 set_mon_vals no callback set 2024-08-19T10:53:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 10 monclient: shutdown 2024-08-19T10:53:26.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 1 -- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f409c9e0 msgr2=0x7f04f40a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 1 --2- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f409c9e0 0x7f04f40a3f40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f04e80097f0 tx=0x7f04e801c330 comp rx=0 tx=0).stop 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 1 -- 172.21.15.89:0/161027372 shutdown_connections 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 1 --2- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f41373f0 0x7f04f4137830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 1 --2- 172.21.15.89:0/161027372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04f409c9e0 0x7f04f40a3f40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 1 --2- 172.21.15.89:0/161027372 >> v2:172.21.15.190:3300/0 conn(0x7f04f40a3620 0x7f04f40a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 1 -- 172.21.15.89:0/161027372 >> 172.21.15.89:0/161027372 conn(0x7f04f40a8210 msgr2=0x7f04f40a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 1 -- 172.21.15.89:0/161027372 shutdown_connections 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 1 -- 172.21.15.89:0/161027372 wait complete. 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 10 monclient: build_initial_monmap 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 10 monclient: monmap: 2024-08-19T10:53:26.778 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:26.779 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.762+0000 7f04f9643640 1 Processor -- start 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 1 -- start start 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 10 monclient: init 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 10 monclient: _add_conns ranks=[5,7,8,1,0,3,6,4,2] 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f409c9e0 0x7f04f41d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 10 monclient(hunting): picked mon.noname-h con 0x7f04f409c9e0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f04f40a3620 0x7f04f4136540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:26.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 10 monclient(hunting): picked mon.noname-i con 0x7f04f40a3620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f04f41373f0 0x7f04f409f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 10 monclient(hunting): picked mon.noname-e con 0x7f04f41373f0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f04f409df40 con 0x7f04f409c9e0 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f04f4138180 con 0x7f04f40a3620 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f04f4140e60 con 0x7f04f41373f0 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f9643640 10 monclient(hunting): authenticate will time out at 1856.584617s 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f37fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f04f41373f0 0x7f04f409f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f37fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f04f41373f0 0x7f04f409f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:55306/0 (socket says 172.21.15.89:55306) 2024-08-19T10:53:26.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f37fe640 1 -- 172.21.15.89:0/3177009397 learned_addr learned my addr 172.21.15.89:0/3177009397 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f37fe640 10 monclient(hunting): get_auth_request con 0x7f04f41373f0 auth_method 0 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f37fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f37fe640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f04f41373f0 msgr2=0x7f04f409f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f2ffd640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f409c9e0 0x7f04f41d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f37fe640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f04f41373f0 msgr2=0x7f04f409f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f37fe640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f04f41373f0 0x7f04f409f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f04f40a3620 0x7f04f4136540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f37fe640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f04f41373f0 0x7f04f409f180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f2ffd640 10 monclient(hunting): get_auth_request con 0x7f04f409c9e0 auth_method 0 2024-08-19T10:53:26.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 10 monclient(hunting): get_auth_request con 0x7f04f40a3620 auth_method 0 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f2ffd640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f409c9e0 msgr2=0x7f04f41d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f2ffd640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f409c9e0 msgr2=0x7f04f41d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f2ffd640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f409c9e0 0x7f04f41d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:26.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f2ffd640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f409c9e0 0x7f04f41d0f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 10 monclient(hunting): handle_auth_done global_id 64420 payload 824 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f04f41373f0 msgr2=0x7f04f409f180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f04f41373f0 0x7f04f409f180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f409c9e0 msgr2=0x7f04f41d0f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f04f409c9e0 0x7f04f41d0f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 10 monclient: _finish_hunting 0 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 1 monclient: found mon.noname-i 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.190:3303/0 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.766+0000 7f04f27fc640 1 -- 172.21.15.89:0/3177009397 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f40a0fb0 con 0x7f04f40a3620 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f04f40a3620 msgr2=0x7f04f4136540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:26.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f04f40a3620 0x7f04f4136540 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f04e80ab000 tx=0x7f04e80190d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04e80b0040 con 0x7f04f40a3620 2024-08-19T10:53:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-08-19T10:53:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: dump: 2024-08-19T10:53:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:26.785 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:26.786 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:26.787 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: _finish_auth 0 2024-08-19T10:53:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:53:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f9643640 5 monclient: authenticate success, global_id 64420 2024-08-19T10:53:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f9643640 10 monclient: _renew_subs 2024-08-19T10:53:26.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f9643640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:53:26.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f9643640 1 -- 172.21.15.89:0/3177009397 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f40a6bb0 con 0x7f04f40a3620 2024-08-19T10:53:26.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e8004040 con 0x7f04f40a3620 2024-08-19T10:53:26.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:26.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f0ff9640 4 set_mon_vals no callback set 2024-08-19T10:53:26.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f9643640 10 monclient: _renew_subs 2024-08-19T10:53:26.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f9643640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:53:26.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f9643640 1 -- 172.21.15.89:0/3177009397 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f4123c40 con 0x7f04f40a3620 2024-08-19T10:53:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04e8003620 con 0x7f04f40a3620 2024-08-19T10:53:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:53:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: dump: 2024-08-19T10:53:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:26.789 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:26.790 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:26.790 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f04f40a3620 msgr2=0x7f04f4136540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f04f40a3620 msgr2=0x7f04f4136540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f04f40a3620 0x7f04f4136540 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f04e80ab000 tx=0x7f04e80190d0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f04f40a3620 0x7f04f4136540 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f04e80ab000 tx=0x7f04e80190d0 comp rx=0 tx=0).stop 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: ms_handle_reset current mon [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f04f40a3620 msgr2=0x7f04f4136540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f04f40a3620 0x7f04f4136540 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient: _add_conns ranks=[4,3,0,7,2,6,5,1,8] 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04e0004d60 0x7f04e0007150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient(hunting): picked mon.c con 0x7f04e0004d60 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:26.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04e00078a0 0x7f04e0009cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient(hunting): picked mon.g con 0x7f04e00078a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04e000a780 0x7f04e000cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient(hunting): picked mon.a con 0x7f04e000a780 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f04e000d7c0 con 0x7f04e000a780 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f04e000d970 con 0x7f04e00078a0 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f04e000db20 con 0x7f04e0004d60 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04db7fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04e0004d60 0x7f04e0007150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:26.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient(hunting): get_auth_request con 0x7f04e0004d60 auth_method 0 2024-08-19T10:53:26.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:26.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:26.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:26.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f37fe640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04e000a780 0x7f04e000cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:26.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f37fe640 10 monclient(hunting): get_auth_request con 0x7f04e000a780 auth_method 0 2024-08-19T10:53:26.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:26.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f37fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:26.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f37fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:26.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f9643640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f9643640 10 monclient(hunting): _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04e00078a0 0x7f04e0009cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 10 monclient(hunting): get_auth_request con 0x7f04e00078a0 auth_method 0 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f37fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:53:26.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f27fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient(hunting): handle_auth_done global_id 64420 payload 293 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04e00078a0 msgr2=0x7f04e0009cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04e00078a0 0x7f04e0009cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04e000a780 msgr2=0x7f04e000cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04e000a780 0x7f04e000cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 1 monclient: found mon.c 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f37fe640 1 -- 172.21.15.89:0/3177009397 reap_dead start 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:26.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 1 -- 172.21.15.89:0/3177009397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) -- 0x7f04e000dea0 con 0x7f04e0004d60 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 1 -- 172.21.15.89:0/3177009397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f4122720 con 0x7f04e0004d60 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 1 -- 172.21.15.89:0/3177009397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e4011260 con 0x7f04e0004d60 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04e0004d60 msgr2=0x7f04e0007150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.770+0000 7f04f2ffd640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04e0004d60 0x7f04e0007150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f04e80aa7e0 tx=0x7f04e80a6000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f04e4019070 con 0x7f04e0004d60 2024-08-19T10:53:26.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04db7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04db7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04db7fe640 10 monclient: dump: 2024-08-19T10:53:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:26.797 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e40158d0 con 0x7f04e0004d60 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04db7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04f17fa640 4 set_mon_vals no callback set 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f04e4015460 con 0x7f04e0004d60 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04db7fe640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04e0040be0 0x7f04e00430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:26.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 <== mon.4 v2:172.21.15.89:3302/0 4 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f04e404e150 con 0x7f04e0004d60 2024-08-19T10:53:26.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04f27fc640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04e0040be0 0x7f04e00430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:26.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.774+0000 7f04f27fc640 10 monclient: get_auth_request con 0x7f04e0040be0 auth_method 0 2024-08-19T10:53:26.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.778+0000 7f04f27fc640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04e0040be0 0x7f04e00430a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f04f406fdf0 tx=0x7f04e8008070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:26.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.778+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 <== mon.4 v2:172.21.15.89:3302/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f04e4013c40 con 0x7f04e0004d60 2024-08-19T10:53:26.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.778+0000 7f04db7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:26.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.778+0000 7f04db7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:26.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.786+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f04e404b060 con 0x7f04e0004d60 2024-08-19T10:53:26.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:26.786+0000 7f04db7fe640 10 monclient: handle_mon_command_ack 1 not found 2024-08-19T10:53:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.014+0000 7f04f9643640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:53:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.014+0000 7f04f9643640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.014+0000 7f04f9643640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.014+0000 7f04f9643640 1 -- 172.21.15.89:0/3177009397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f04f413d880 con 0x7f04e0004d60 2024-08-19T10:53:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.014+0000 7f04db7fe640 1 -- 172.21.15.89:0/3177009397 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7f04e404b3e0 con 0x7f04e0004d60 2024-08-19T10:53:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.014+0000 7f04db7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.014+0000 7f04db7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:27.015 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":124,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:53:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04e0040be0 msgr2=0x7f04e00430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04e0040be0 0x7f04e00430a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f04f406fdf0 tx=0x7f04e8008070 comp rx=0 tx=0).stop 2024-08-19T10:53:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 10 monclient: shutdown 2024-08-19T10:53:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 1 -- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04e0004d60 msgr2=0x7f04e0007150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04e0004d60 0x7f04e0007150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f04e80aa7e0 tx=0x7f04e80a6000 comp rx=0 tx=0).stop 2024-08-19T10:53:27.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 1 -- 172.21.15.89:0/3177009397 shutdown_connections 2024-08-19T10:53:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f04e0040be0 0x7f04e00430a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 1 --2- 172.21.15.89:0/3177009397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f04e0004d60 0x7f04e0007150 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 1 -- 172.21.15.89:0/3177009397 >> 172.21.15.89:0/3177009397 conn(0x7f04f40a8210 msgr2=0x7f04f4072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 1 -- 172.21.15.89:0/3177009397 shutdown_connections 2024-08-19T10:53:27.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.018+0000 7f04f9643640 1 -- 172.21.15.89:0/3177009397 wait complete. 2024-08-19T10:53:27.031 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:53:27.032 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:53:27.032 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:53:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient: get_monmap_and_config 2024-08-19T10:53:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient: build_initial_monmap 2024-08-19T10:53:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient: monmap: 2024-08-19T10:53:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:27.157279+0000 2024-08-19T10:53:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:27.157279+0000 2024-08-19T10:53:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.151 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:27.152 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:27.152 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 1 Processor -- start 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 1 -- start start 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient: init 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient: _add_conns ranks=[4,7,3,6,8,1,5,2,10,0,9] 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a6860 0x7fb4000a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient(hunting): picked mon.noname-c con 0x7fb4000a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb4000a7180 0x7fb4000a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient(hunting): picked mon.noname-i con 0x7fb4000a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb400139620 0x7fb400139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient(hunting): picked mon.noname-g con 0x7fb400139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:27.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb4000a8190 con 0x7fb400139620 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb4000a33d0 con 0x7fb4000a6860 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb4000a2ec0 con 0x7fb4000a7180 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb4048c9640 10 monclient(hunting): authenticate will time out at 1856.974082s 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fe59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a6860 0x7fb4000a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fe59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a6860 0x7fb4000a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49004/0 (socket says 172.21.15.89:49004) 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fe59c640 1 -- 172.21.15.89:0/934815492 learned_addr learned my addr 172.21.15.89:0/934815492 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fe59c640 10 monclient(hunting): get_auth_request con 0x7fb4000a6860 auth_method 0 2024-08-19T10:53:27.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fe59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fe59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fe59c640 1 -- 172.21.15.89:0/934815492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a6860 msgr2=0x7fb4000a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fe59c640 1 -- 172.21.15.89:0/934815492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a6860 msgr2=0x7fb4000a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fe59c640 1 --2- 172.21.15.89:0/934815492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a6860 0x7fb4000a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fe59c640 1 --2- 172.21.15.89:0/934815492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a6860 0x7fb4000a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fdd9b640 1 --2- 172.21.15.89:0/934815492 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb4000a7180 0x7fb4000a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 1 --2- 172.21.15.89:0/934815492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb400139620 0x7fb400139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 10 monclient(hunting): get_auth_request con 0x7fb400139620 auth_method 0 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fdd9b640 10 monclient(hunting): get_auth_request con 0x7fb4000a7180 auth_method 0 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fdd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 10 monclient(hunting): handle_auth_done global_id 35742 payload 274 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 1 -- 172.21.15.89:0/934815492 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb4000a7180 msgr2=0x7fb4000a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:27.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 1 --2- 172.21.15.89:0/934815492 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb4000a7180 0x7fb4000a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 1 -- 172.21.15.89:0/934815492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a6860 msgr2=0x7fb4000a6c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 1 --2- 172.21.15.89:0/934815492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a6860 0x7fb4000a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:53:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 1 monclient: found mon.noname-g 2024-08-19T10:53:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:53:27.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.154+0000 7fb3fed9d640 1 -- 172.21.15.89:0/934815492 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4000a4cd0 con 0x7fb400139620 2024-08-19T10:53:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.158+0000 7fb3fed9d640 1 -- 172.21.15.89:0/934815492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb400139620 msgr2=0x7fb400139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.158+0000 7fb3fed9d640 1 --2- 172.21.15.89:0/934815492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb400139620 0x7fb400139a60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00097f0 tx=0x7fb3f001c3a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=8be74eb94e27c786 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.158+0000 7fb3fd59a640 1 -- 172.21.15.89:0/934815492 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3f00b0070 con 0x7fb400139620 2024-08-19T10:53:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.158+0000 7fb3fd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fd59a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:53:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fd59a640 10 monclient: dump: 2024-08-19T10:53:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.169 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fd59a640 10 monclient: _finish_auth 0 2024-08-19T10:53:27.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fd59a640 10 monclient: _check_auth_tickets 2024-08-19T10:53:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 5 monclient: authenticate success, global_id 35742 2024-08-19T10:53:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fd59a640 1 -- 172.21.15.89:0/934815492 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f00aa3a0 con 0x7fb400139620 2024-08-19T10:53:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fd59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 4 set_mon_vals no callback set 2024-08-19T10:53:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fd59a640 1 -- 172.21.15.89:0/934815492 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3f00aaa40 con 0x7fb400139620 2024-08-19T10:53:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fd59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fd59a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:53:27.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fd59a640 10 monclient: dump: 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.172 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.173 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.173 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.173 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient: shutdown 2024-08-19T10:53:27.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 -- 172.21.15.89:0/934815492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb400139620 msgr2=0x7fb400139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 --2- 172.21.15.89:0/934815492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb400139620 0x7fb400139a60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00097f0 tx=0x7fb3f001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 -- 172.21.15.89:0/934815492 shutdown_connections 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 --2- 172.21.15.89:0/934815492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb400139620 0x7fb400139a60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 --2- 172.21.15.89:0/934815492 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb4000a7180 0x7fb4000a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 --2- 172.21.15.89:0/934815492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a6860 0x7fb4000a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 -- 172.21.15.89:0/934815492 >> 172.21.15.89:0/934815492 conn(0x7fb400074b40 msgr2=0x7fb4000abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 -- 172.21.15.89:0/934815492 shutdown_connections 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 -- 172.21.15.89:0/934815492 wait complete. 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient: build_initial_monmap 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient: monmap: 2024-08-19T10:53:27.174 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:27.175 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 Processor -- start 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 -- start start 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient: init 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient: _add_conns ranks=[8,4,2,7,6,0,3,1,5] 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4000a6860 0x7fb4001d1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient(hunting): picked mon.noname-e con 0x7fb4000a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a7180 0x7fb4001d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient(hunting): picked mon.noname-c con 0x7fb4000a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb400139620 0x7fb40009c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient(hunting): picked mon.noname-b con 0x7fb400139620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:27.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fb400143210 con 0x7fb400139620 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb400143090 con 0x7fb4000a7180 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fb400143390 con 0x7fb4000a6860 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb4048c9640 10 monclient(hunting): authenticate will time out at 1856.982315s 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fe59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4000a6860 0x7fb4001d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fdd9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a7180 0x7fb4001d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fe59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4000a6860 0x7fb4001d1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:55308/0 (socket says 172.21.15.89:55308) 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb400139620 0x7fb40009c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fe59c640 1 -- 172.21.15.89:0/1512250372 learned_addr learned my addr 172.21.15.89:0/1512250372 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:27.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fdd9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a7180 0x7fb4001d1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49016/0 (socket says 172.21.15.89:49016) 2024-08-19T10:53:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fe59c640 10 monclient(hunting): get_auth_request con 0x7fb4000a6860 auth_method 0 2024-08-19T10:53:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fe59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fe59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 10 monclient(hunting): get_auth_request con 0x7fb400139620 auth_method 0 2024-08-19T10:53:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fdd9b640 10 monclient(hunting): get_auth_request con 0x7fb4000a7180 auth_method 0 2024-08-19T10:53:27.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fdd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fe59c640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4000a6860 msgr2=0x7fb4001d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fe59c640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4000a6860 msgr2=0x7fb4001d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fe59c640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4000a6860 0x7fb4001d1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fdd9b640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a7180 msgr2=0x7fb4001d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fdd9b640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a7180 msgr2=0x7fb4001d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fdd9b640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a7180 0x7fb4001d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fe59c640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4000a6860 0x7fb4001d1660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fdd9b640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a7180 0x7fb4001d1ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 10 monclient(hunting): handle_auth_done global_id 74270 payload 824 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4000a6860 msgr2=0x7fb4001d1660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fb4000a6860 0x7fb4001d1660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a7180 msgr2=0x7fb4001d1ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb4000a7180 0x7fb4001d1ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 1 monclient: found mon.noname-b 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.162+0000 7fb3fed9d640 1 -- 172.21.15.89:0/1512250372 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb400140520 con 0x7fb400139620 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fed9d640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb400139620 msgr2=0x7fb40009c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fed9d640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb400139620 0x7fb40009c9e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00ab000 tx=0x7fb3f0019140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3f00b0050 con 0x7fb400139620 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 10 monclient: dump: 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb4048c9640 5 monclient: authenticate success, global_id 74270 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f00bc070 con 0x7fb400139620 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb4048c9640 10 monclient: _renew_subs 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb4048c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:53:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb4048c9640 1 -- 172.21.15.89:0/1512250372 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4000a2710 con 0x7fb400139620 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3f00aa9c0 con 0x7fb400139620 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 10 monclient: dump: 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.183 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f7fff640 4 set_mon_vals no callback set 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb4048c9640 10 monclient: _renew_subs 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb4048c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb4048c9640 1 -- 172.21.15.89:0/1512250372 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40009fc60 con 0x7fb400139620 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb4048c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:27.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb4048c9640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4000a0c00 0x7fb40009cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb4048c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb4048c9640 1 -- 172.21.15.89:0/1512250372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb40009fea0 con 0x7fb4000a0c00 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fe59c640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4000a0c00 0x7fb40009cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fe59c640 10 monclient: get_auth_request con 0x7fb4000a0c00 auth_method 0 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fe59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fe59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb3f00aaba0 con 0x7fb400139620 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fe59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fe59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fe59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fe59c640 10 monclient(hunting): handle_auth_done global_id 7263 payload 824 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fe59c640 1 -- 172.21.15.89:0/1512250372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4000a7560 con 0x7fb4000a0c00 2024-08-19T10:53:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3e403be00 0x7fb3e403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fb3f00e9b90 con 0x7fb400139620 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fdd9b640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3e403be00 0x7fb3e403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.166+0000 7fb3fdd9b640 10 monclient: get_auth_request con 0x7fb3e403be00 auth_method 0 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3fe59c640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4000a0c00 msgr2=0x7fb40009cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3fe59c640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4000a0c00 0x7fb40009cf20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009fd0 tx=0x7fb3e8010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3e801a030 con 0x7fb4000a0c00 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3f6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3f6ffd640 10 monclient: dump: 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.187 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb3e800ebe0 con 0x7fb4000a0c00 2024-08-19T10:53:27.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3f6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:27.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3f6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:27.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4000a0c00 msgr2=0x7fb40009cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3f6ffd640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4000a0c00 0x7fb40009cf20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009fd0 tx=0x7fb3e8010c20 comp rx=0 tx=0).stop 2024-08-19T10:53:27.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4000a0c00 msgr2=0x7fb40009cf20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:27.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3f6ffd640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4000a0c00 0x7fb40009cf20 secure :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009fd0 tx=0x7fb3e8010c20 comp rx=0 tx=0).stop 2024-08-19T10:53:27.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.170+0000 7fb3fdd9b640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3e403be00 0x7fb3e403e2c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb40013f610 tx=0x7fb3f800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb4048c9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb4048c9640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40009d460 0x7fb40009d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb4048c9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb4048c9640 1 -- 172.21.15.89:0/1512250372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb40009ddb0 con 0x7fb40009d460 2024-08-19T10:53:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb3fe59c640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40009d460 0x7fb40009d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb3fe59c640 10 monclient: get_auth_request con 0x7fb40009d460 auth_method 0 2024-08-19T10:53:27.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb3fe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb3fe59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb3fe59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb3fe59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb3fe59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb3fe59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb3fe59c640 10 monclient(hunting): handle_auth_done global_id 7272 payload 824 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.174+0000 7fb3fe59c640 1 -- 172.21.15.89:0/1512250372 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb400064590 con 0x7fb40009d460 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3fe59c640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40009d460 msgr2=0x7fb40009d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3fe59c640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40009d460 0x7fb40009d870 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009fd0 tx=0x7fb3e80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb3e800f040 con 0x7fb40009d460 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3f6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3f6ffd640 10 monclient: dump: 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.192 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.193 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9604 (secure 0 0 0) 0x7fb3e8004020 con 0x7fb40009d460 2024-08-19T10:53:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3f6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3f6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40009d460 msgr2=0x7fb40009d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3f6ffd640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40009d460 0x7fb40009d870 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009fd0 tx=0x7fb3e80046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3f6ffd640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40009d460 msgr2=0x7fb40009d870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:27.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3f6ffd640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb40009d460 0x7fb40009d870 secure :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009fd0 tx=0x7fb3e80046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.193 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:27.193 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 936991, 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:27.194 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.195 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.196 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:27.197 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.198 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.199 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.199 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.199 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.199 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.199 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.199 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.199 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.199 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:27.201 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.202 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.203 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.204 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.206 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:27.207 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.208 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:27.209 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.210 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.211 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.213 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.214 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.215 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:27.216 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.217 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:27.218 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:27.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb4048c9640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3e403be00 msgr2=0x7fb3e403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb4048c9640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb3e403be00 0x7fb3e403e2c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb40013f610 tx=0x7fb3f800b040 comp rx=0 tx=0).stop 2024-08-19T10:53:27.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb4048c9640 10 monclient: shutdown 2024-08-19T10:53:27.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb4048c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:27.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb4048c9640 1 -- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb400139620 msgr2=0x7fb40009c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb4048c9640 1 --2- 172.21.15.89:0/1512250372 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fb400139620 0x7fb40009c9e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00ab000 tx=0x7fb3f0019140 comp rx=0 tx=0).stop 2024-08-19T10:53:27.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3fed9d640 1 -- 172.21.15.89:0/1512250372 reap_dead start 2024-08-19T10:53:27.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb3fed9d640 1 -- 172.21.15.89:0/1512250372 reap_dead start 2024-08-19T10:53:27.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb4048c9640 1 -- 172.21.15.89:0/1512250372 shutdown_connections 2024-08-19T10:53:27.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb4048c9640 1 -- 172.21.15.89:0/1512250372 >> 172.21.15.89:0/1512250372 conn(0x7fb400074b40 msgr2=0x7fb4000ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:27.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb4048c9640 1 -- 172.21.15.89:0/1512250372 shutdown_connections 2024-08-19T10:53:27.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.178+0000 7fb4048c9640 1 -- 172.21.15.89:0/1512250372 wait complete. 2024-08-19T10:53:27.220 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:27.221 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:53:27.221 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:53:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient: get_monmap_and_config 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient: build_initial_monmap 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient: monmap: 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:27.341024+0000 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:27.341024+0000 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:27.336 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 1 Processor -- start 2024-08-19T10:53:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 1 -- start start 2024-08-19T10:53:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient: init 2024-08-19T10:53:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient: _add_conns ranks=[5,1,3,2,4,8,9,6,7,10,0] 2024-08-19T10:53:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41ec0a3d20 0x7f41ec0a4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient(hunting): picked mon.noname-h con 0x7f41ec0a3d20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f41ec0a4640 addr v1:172.21.15.190:6789/0 2024-08-19T10:53:27.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41ec0a4a20 0x7f41ec136c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient(hunting): picked mon.noname-g con 0x7f41ec0a4a20 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41ec0a2430 con 0x7f41ec0a4640 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f41ec0a0af0 con 0x7f41ec0a4a20 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f41ec0a1e70 con 0x7f41ec0a3d20 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f33ae640 10 monclient(hunting): authenticate will time out at 1857.157653s 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f114a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41ec0a3d20 0x7f41ec0a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f114a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41ec0a3d20 0x7f41ec0a4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:36670/0 (socket says 172.21.15.89:36670) 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41ec0a4a20 0x7f41ec136c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f114a640 1 -- 172.21.15.89:0/2798635735 learned_addr learned my addr 172.21.15.89:0/2798635735 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f114a640 10 monclient(hunting): get_auth_request con 0x7f41ec0a3d20 auth_method 0 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f114a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f114a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f114a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f0949640 1 --1- 172.21.15.89:0/2798635735 >> v1:172.21.15.190:6789/0 conn(0x7f41ec0a4640 0x7f41ec1364f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.89:53474/0 (socket says 172.21.15.89:53474) 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 10 monclient(hunting): get_auth_request con 0x7f41ec0a4a20 auth_method 0 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f114a640 1 -- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41ec0a3d20 msgr2=0x7f41ec0a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f114a640 1 -- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41ec0a3d20 msgr2=0x7f41ec0a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f114a640 1 --2- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41ec0a3d20 0x7f41ec0a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f114a640 1 --2- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41ec0a3d20 0x7f41ec0a4100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 10 monclient(hunting): handle_auth_done global_id 35751 payload 274 2024-08-19T10:53:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 1 -- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41ec0a3d20 msgr2=0x7f41ec0a4100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 1 --2- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41ec0a3d20 0x7f41ec0a4100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 1 -- 172.21.15.89:0/2798635735 >> v1:172.21.15.190:6789/0 conn(0x7f41ec0a4640 legacy=0x7f41ec1364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 10 monclient: _finish_hunting 0 2024-08-19T10:53:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 1 monclient: found mon.noname-g 2024-08-19T10:53:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:53:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.338+0000 7f41f194b640 1 -- 172.21.15.89:0/2798635735 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec0a1b90 con 0x7f41ec0a4a20 2024-08-19T10:53:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.342+0000 7f41f194b640 1 -- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41ec0a4a20 msgr2=0x7f41ec136c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.342+0000 7f41f194b640 1 --2- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41ec0a4a20 0x7f41ec136c00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097f0 tx=0x7f41dc01c3e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=942c513c8263d485 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.342+0000 7f41ebfff640 1 -- 172.21.15.89:0/2798635735 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f41dc0b2070 con 0x7f41ec0a4a20 2024-08-19T10:53:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.342+0000 7f41ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.342+0000 7f41ebfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:53:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.342+0000 7f41ebfff640 10 monclient: dump: 2024-08-19T10:53:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41ebfff640 10 monclient: _finish_auth 0 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41ebfff640 10 monclient: _check_auth_tickets 2024-08-19T10:53:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41ebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41ebfff640 1 -- 172.21.15.89:0/2798635735 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41dc0ab7f0 con 0x7f41ec0a4a20 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 5 monclient: authenticate success, global_id 35751 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41ebfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41ebfff640 1 -- 172.21.15.89:0/2798635735 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f41dc0abe10 con 0x7f41ec0a4a20 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 4 set_mon_vals no callback set 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41ebfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41ebfff640 10 monclient: dump: 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.348 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient: shutdown 2024-08-19T10:53:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 -- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41ec0a4a20 msgr2=0x7f41ec136c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 --2- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41ec0a4a20 0x7f41ec136c00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097f0 tx=0x7f41dc01c3e0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 -- 172.21.15.89:0/2798635735 shutdown_connections 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 --2- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41ec0a4a20 0x7f41ec136c00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 --2- 172.21.15.89:0/2798635735 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f41ec0a3d20 0x7f41ec0a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 -- 172.21.15.89:0/2798635735 >> 172.21.15.89:0/2798635735 conn(0x7f41ec06de40 msgr2=0x7f41ec06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 -- 172.21.15.89:0/2798635735 shutdown_connections 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 -- 172.21.15.89:0/2798635735 wait complete. 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient: build_initial_monmap 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient: monmap: 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 Processor -- start 2024-08-19T10:53:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 -- start start 2024-08-19T10:53:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient: init 2024-08-19T10:53:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient: _add_conns ranks=[4,0,7,5,2,1,6,3,8] 2024-08-19T10:53:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41ec0a3d20 0x7f41ec139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient(hunting): picked mon.noname-c con 0x7f41ec0a3d20 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41ec0a4640 0x7f41ec13f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient(hunting): picked mon.noname-a con 0x7f41ec0a4640 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f41ec0a4a20 0x7f41ec13fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient(hunting): picked mon.noname-i con 0x7f41ec0a4a20 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec09fee0 con 0x7f41ec0a4640 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f41ec09fd60 con 0x7f41ec0a3d20 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f41ec137140 con 0x7f41ec0a4a20 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f33ae640 10 monclient(hunting): authenticate will time out at 1857.166213s 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f114a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41ec0a3d20 0x7f41ec139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f114a640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41ec0a3d20 0x7f41ec139ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49032/0 (socket says 172.21.15.89:49032) 2024-08-19T10:53:27.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f114a640 1 -- 172.21.15.89:0/4261450803 learned_addr learned my addr 172.21.15.89:0/4261450803 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f114a640 10 monclient(hunting): get_auth_request con 0x7f41ec0a3d20 auth_method 0 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f114a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f114a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f114a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41ec0a4640 0x7f41ec13f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 10 monclient(hunting): get_auth_request con 0x7f41ec0a4640 auth_method 0 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f114a640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41ec0a3d20 msgr2=0x7f41ec139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f114a640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41ec0a3d20 msgr2=0x7f41ec139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f114a640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41ec0a3d20 0x7f41ec139ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f114a640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41ec0a3d20 0x7f41ec139ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f194b640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f41ec0a4a20 0x7f41ec13fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f194b640 10 monclient(hunting): get_auth_request con 0x7f41ec0a4a20 auth_method 0 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f194b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f194b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f194b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 10 monclient(hunting): handle_auth_done global_id 7281 payload 824 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f41ec0a4a20 msgr2=0x7f41ec13fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f41ec0a4a20 0x7f41ec13fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41ec0a3d20 msgr2=0x7f41ec139ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f41ec0a3d20 0x7f41ec139ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 10 monclient: _finish_hunting 0 2024-08-19T10:53:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 1 monclient: found mon.noname-a 2024-08-19T10:53:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:27.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.346+0000 7f41f0949640 1 -- 172.21.15.89:0/4261450803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec13cd30 con 0x7f41ec0a4640 2024-08-19T10:53:27.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f0949640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41ec0a4640 msgr2=0x7f41ec13f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f0949640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41ec0a4640 0x7f41ec13f760 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f41e000c470 tx=0x7f41e0005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f41e0017070 con 0x7f41ec0a4640 2024-08-19T10:53:27.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 10 monclient: dump: 2024-08-19T10:53:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.363 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.363 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.363 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.363 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.363 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.363 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.363 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:53:27.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e00074f0 con 0x7f41ec0a4640 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f41e00076a0 con 0x7f41ec0a4640 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 10 monclient: dump: 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.365 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.365 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.365 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.365 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.365 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.365 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.365 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.365 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.365 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41eb7fe640 4 set_mon_vals no callback set 2024-08-19T10:53:27.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f33ae640 5 monclient: authenticate success, global_id 7281 2024-08-19T10:53:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f33ae640 10 monclient: _renew_subs 2024-08-19T10:53:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f33ae640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f33ae640 1 -- 172.21.15.89:0/4261450803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ec13ad90 con 0x7f41ec0a4640 2024-08-19T10:53:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f33ae640 10 monclient: _renew_subs 2024-08-19T10:53:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f33ae640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f33ae640 1 -- 172.21.15.89:0/4261450803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ec0a6fd0 con 0x7f41ec0a4640 2024-08-19T10:53:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f33ae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f33ae640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec02e350 0x7f41ec1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f33ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f33ae640 1 -- 172.21.15.89:0/4261450803 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f41ec139870 con 0x7f41ec02e350 2024-08-19T10:53:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f114a640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec02e350 0x7f41ec1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f114a640 10 monclient: get_auth_request con 0x7f41ec02e350 auth_method 0 2024-08-19T10:53:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f114a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f114a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f114a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f114a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f114a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f114a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f0949640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41ec0a4640 0x7f41ec13f760 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f41e000c470 tx=0x7f41e0005790 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f41e004e8e0 osd_map(274..274 src has 246..274) 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f41e0005820 con 0x7f41ec0a4640 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f114a640 10 monclient(hunting): handle_auth_done global_id 74279 payload 824 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f114a640 1 -- 172.21.15.89:0/4261450803 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41ec13d800 con 0x7f41ec02e350 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41e9ffb640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f41d803be30 0x7f41d803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f194b640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f41d803be30 0x7f41d803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.350+0000 7f41f194b640 10 monclient: get_auth_request con 0x7f41d803be30 auth_method 0 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41f114a640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec02e350 msgr2=0x7f41ec1401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41f114a640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec02e350 0x7f41ec1401e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41e4007d00 tx=0x7f41e400f520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f41e4014a50 con 0x7f41ec02e350 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41e9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:27.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41e9ffb640 10 monclient: dump: 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.369 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.370 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.370 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f41e40145e0 con 0x7f41ec02e350 2024-08-19T10:53:27.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41e9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:27.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41e9ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:27.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec02e350 msgr2=0x7f41ec1401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41e9ffb640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec02e350 0x7f41ec1401e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41e4007d00 tx=0x7f41e400f520 comp rx=0 tx=0).stop 2024-08-19T10:53:27.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec02e350 msgr2=0x7f41ec1401e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:27.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41e9ffb640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec02e350 0x7f41ec1401e0 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41e4007d00 tx=0x7f41e400f520 comp rx=0 tx=0).stop 2024-08-19T10:53:27.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.354+0000 7f41f194b640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f41d803be30 0x7f41d803e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097f0 tx=0x7f41dc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f33ae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:27.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f33ae640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec063b30 0x7f41ec063f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f33ae640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f33ae640 1 -- 172.21.15.89:0/4261450803 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f41ec02e730 con 0x7f41ec063b30 2024-08-19T10:53:27.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f114a640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec063b30 0x7f41ec063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f114a640 10 monclient: get_auth_request con 0x7f41ec063b30 auth_method 0 2024-08-19T10:53:27.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f114a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f114a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f114a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f114a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f114a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.358+0000 7f41f114a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f114a640 10 monclient(hunting): handle_auth_done global_id 74288 payload 824 2024-08-19T10:53:27.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f114a640 1 -- 172.21.15.89:0/4261450803 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f41ec0a6b30 con 0x7f41ec063b30 2024-08-19T10:53:27.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f114a640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec063b30 msgr2=0x7f41ec063f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f114a640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec063b30 0x7f41ec063f40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f41e4019000 tx=0x7f41e4004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f41e4014e40 con 0x7f41ec063b30 2024-08-19T10:53:27.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 10 monclient: dump: 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.375 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.376 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.376 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.376 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.376 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7f41e4014e40 con 0x7f41ec063b30 2024-08-19T10:53:27.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:27.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:27.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec063b30 msgr2=0x7f41ec063f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec063b30 0x7f41ec063f40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f41e4019000 tx=0x7f41e4004a50 comp rx=0 tx=0).stop 2024-08-19T10:53:27.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec063b30 msgr2=0x7f41ec063f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:27.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f41ec063b30 0x7f41ec063f40 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f41e4019000 tx=0x7f41e4004a50 comp rx=0 tx=0).stop 2024-08-19T10:53:27.377 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 143861, 2024-08-19T10:53:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:27.377 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:27.378 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.379 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.383 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:27.384 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:27.385 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.387 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.388 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.389 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:27.390 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.391 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:27.393 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:27.394 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.395 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:27.397 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.400 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:27.400 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:27.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.400 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:27.400 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:27.400 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:27.400 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:53:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.401 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:27.401 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f33ae640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f41d803be30 msgr2=0x7f41d803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f33ae640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f41d803be30 0x7f41d803e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097f0 tx=0x7f41dc0a7000 comp rx=0 tx=0).stop 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f33ae640 10 monclient: shutdown 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f33ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f33ae640 1 -- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41ec0a4640 msgr2=0x7f41ec13f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f33ae640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41ec0a4640 0x7f41ec13f760 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f41e000c470 tx=0x7f41e0005790 comp rx=0 tx=0).stop 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f194b640 1 -- 172.21.15.89:0/4261450803 reap_dead start 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41e9ffb640 1 -- 172.21.15.89:0/4261450803 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (unknown 0 0 0) 0x7f41e004e8e0 con 0x7f41ec0a4640 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f33ae640 1 -- 172.21.15.89:0/4261450803 shutdown_connections 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f33ae640 1 --2- 172.21.15.89:0/4261450803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41ec0a4640 0x7f41ec13f760 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f33ae640 1 -- 172.21.15.89:0/4261450803 >> 172.21.15.89:0/4261450803 conn(0x7f41ec06de40 msgr2=0x7f41ec13da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f33ae640 1 -- 172.21.15.89:0/4261450803 shutdown_connections 2024-08-19T10:53:27.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.362+0000 7f41f33ae640 1 -- 172.21.15.89:0/4261450803 wait complete. 2024-08-19T10:53:27.404 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:27.404 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:53:27.404 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:53:27.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient: get_monmap_and_config 2024-08-19T10:53:27.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient: build_initial_monmap 2024-08-19T10:53:27.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient: monmap: 2024-08-19T10:53:27.526 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:27.526 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.526 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:27.529140+0000 2024-08-19T10:53:27.526 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:27.529140+0000 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:27.527 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 1 Processor -- start 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 1 -- start start 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient: init 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient: _add_conns ranks=[4,7,3,10,5,6,1,8,2,9,0] 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd320137920 0x7fd320137d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient(hunting): picked mon.noname-c con 0x7fd320137920 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd3200743c0 0x7fd3200a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient(hunting): picked mon.noname-i con 0x7fd3200743c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200a51a0 0x7fd3200a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient(hunting): picked mon.noname-g con 0x7fd3200a51a0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd320073d50 con 0x7fd3200a51a0 2024-08-19T10:53:27.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd320072660 con 0x7fd320137920 2024-08-19T10:53:27.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd3200a6740 con 0x7fd3200743c0 2024-08-19T10:53:27.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:27.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd328085640 10 monclient(hunting): authenticate will time out at 1857.345756s 2024-08-19T10:53:27.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325e21640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd320137920 0x7fd320137d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325e21640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd320137920 0x7fd320137d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49040/0 (socket says 172.21.15.89:49040) 2024-08-19T10:53:27.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325e21640 1 -- 172.21.15.89:0/477358727 learned_addr learned my addr 172.21.15.89:0/477358727 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:27.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325e21640 10 monclient(hunting): get_auth_request con 0x7fd320137920 auth_method 0 2024-08-19T10:53:27.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325e21640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325e21640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325e21640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325e21640 1 -- 172.21.15.89:0/477358727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd320137920 msgr2=0x7fd320137d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325e21640 1 -- 172.21.15.89:0/477358727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd320137920 msgr2=0x7fd320137d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325e21640 1 --2- 172.21.15.89:0/477358727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd320137920 0x7fd320137d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325e21640 1 --2- 172.21.15.89:0/477358727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd320137920 0x7fd320137d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325620640 1 --2- 172.21.15.89:0/477358727 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd3200743c0 0x7fd3200a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 1 --2- 172.21.15.89:0/477358727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200a51a0 0x7fd3200a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 10 monclient(hunting): get_auth_request con 0x7fd3200a51a0 auth_method 0 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325620640 10 monclient(hunting): get_auth_request con 0x7fd3200743c0 auth_method 0 2024-08-19T10:53:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325620640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325620640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325620640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325620640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325620640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd325620640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 10 monclient(hunting): handle_auth_done global_id 35760 payload 274 2024-08-19T10:53:27.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 1 -- 172.21.15.89:0/477358727 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd3200743c0 msgr2=0x7fd3200a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:27.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 1 --2- 172.21.15.89:0/477358727 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd3200743c0 0x7fd3200a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 1 -- 172.21.15.89:0/477358727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd320137920 msgr2=0x7fd320137d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:27.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 1 --2- 172.21.15.89:0/477358727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd320137920 0x7fd320137d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 10 monclient: _finish_hunting 0 2024-08-19T10:53:27.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 1 monclient: found mon.noname-g 2024-08-19T10:53:27.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:27.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:53:27.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.526+0000 7fd326622640 1 -- 172.21.15.89:0/477358727 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3200723e0 con 0x7fd3200a51a0 2024-08-19T10:53:27.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd326622640 1 -- 172.21.15.89:0/477358727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200a51a0 msgr2=0x7fd3200a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd326622640 1 --2- 172.21.15.89:0/477358727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200a51a0 0x7fd3200a5600 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0097f0 tx=0x7fd31c01c370 comp rx=0 tx=0).ready entity=mon.3 client_cookie=4e5eb986a052cb5d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 1 -- 172.21.15.89:0/477358727 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd31c0b0070 con 0x7fd3200a51a0 2024-08-19T10:53:27.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:53:27.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 10 monclient: dump: 2024-08-19T10:53:27.536 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.536 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.536 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.537 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 10 monclient: _finish_auth 0 2024-08-19T10:53:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 10 monclient: _check_auth_tickets 2024-08-19T10:53:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 1 -- 172.21.15.89:0/477358727 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c0aa810 con 0x7fd3200a51a0 2024-08-19T10:53:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd328085640 5 monclient: authenticate success, global_id 35760 2024-08-19T10:53:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd328085640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd328085640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd328085640 4 set_mon_vals no callback set 2024-08-19T10:53:27.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 1 -- 172.21.15.89:0/477358727 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd31c0a9300 con 0x7fd3200a51a0 2024-08-19T10:53:27.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:53:27.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.530+0000 7fd324e1f640 10 monclient: dump: 2024-08-19T10:53:27.539 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.539 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.539 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.539 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.539 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.539 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.539 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.539 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient: shutdown 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 -- 172.21.15.89:0/477358727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200a51a0 msgr2=0x7fd3200a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 --2- 172.21.15.89:0/477358727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200a51a0 0x7fd3200a5600 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0097f0 tx=0x7fd31c01c370 comp rx=0 tx=0).stop 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 -- 172.21.15.89:0/477358727 shutdown_connections 2024-08-19T10:53:27.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 --2- 172.21.15.89:0/477358727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200a51a0 0x7fd3200a5600 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 --2- 172.21.15.89:0/477358727 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd3200743c0 0x7fd3200a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 --2- 172.21.15.89:0/477358727 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd320137920 0x7fd320137d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 -- 172.21.15.89:0/477358727 >> 172.21.15.89:0/477358727 conn(0x7fd3200aa240 msgr2=0x7fd3200aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 -- 172.21.15.89:0/477358727 shutdown_connections 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 -- 172.21.15.89:0/477358727 wait complete. 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient: build_initial_monmap 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient: monmap: 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:27.541 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 Processor -- start 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 -- start start 2024-08-19T10:53:27.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient: init 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient: _add_conns ranks=[0,4,5,7,2,3,1,8,6] 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3200743c0 0x7fd320136050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient(hunting): picked mon.noname-a con 0x7fd3200743c0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3200a51a0 0x7fd320136590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient(hunting): picked mon.noname-c con 0x7fd3200a51a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd320137920 0x7fd32013b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient(hunting): picked mon.noname-h con 0x7fd320137920 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd3201383c0 con 0x7fd3200743c0 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd320138240 con 0x7fd3200a51a0 2024-08-19T10:53:27.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd320141070 con 0x7fd320137920 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd328085640 10 monclient(hunting): authenticate will time out at 1857.354595s 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325620640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3200a51a0 0x7fd320136590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325620640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3200a51a0 0x7fd320136590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49056/0 (socket says 172.21.15.89:49056) 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325620640 1 -- 172.21.15.89:0/4068572537 learned_addr learned my addr 172.21.15.89:0/4068572537 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325620640 10 monclient(hunting): get_auth_request con 0x7fd3200a51a0 auth_method 0 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325620640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325620640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325620640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3200743c0 0x7fd320136050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 10 monclient(hunting): get_auth_request con 0x7fd3200743c0 auth_method 0 2024-08-19T10:53:27.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325620640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3200a51a0 msgr2=0x7fd320136590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:53:27.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325620640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3200a51a0 msgr2=0x7fd320136590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:53:27.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325620640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3200a51a0 0x7fd320136590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:27.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325620640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3200a51a0 0x7fd320136590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:27.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 10 monclient(hunting): handle_auth_done global_id 7290 payload 824 2024-08-19T10:53:27.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd320137920 msgr2=0x7fd32013b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd320137920 0x7fd32013b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3200a51a0 msgr2=0x7fd320136590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd3200a51a0 0x7fd320136590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 10 monclient: _finish_hunting 0 2024-08-19T10:53:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 1 monclient: found mon.noname-a 2024-08-19T10:53:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.534+0000 7fd325e21640 1 -- 172.21.15.89:0/4068572537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd320120890 con 0x7fd3200743c0 2024-08-19T10:53:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd325e21640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3200743c0 msgr2=0x7fd320136050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd325e21640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3200743c0 0x7fd320136050 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd318007af0 tx=0x7fd31800d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd31801a070 con 0x7fd3200743c0 2024-08-19T10:53:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 10 monclient: dump: 2024-08-19T10:53:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 10 monclient: _finish_auth 0 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 10 monclient: _check_auth_tickets 2024-08-19T10:53:27.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31800dd20 con 0x7fd3200743c0 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd318014c80 con 0x7fd3200743c0 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd328085640 5 monclient: authenticate success, global_id 7290 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd328085640 10 monclient: _renew_subs 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd328085640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd328085640 1 -- 172.21.15.89:0/4068572537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd320121e50 con 0x7fd3200743c0 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd317fff640 4 set_mon_vals no callback set 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 10 monclient: dump: 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.551 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd328085640 10 monclient: _renew_subs 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd328085640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd328085640 1 -- 172.21.15.89:0/4068572537 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd320121b70 con 0x7fd3200743c0 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd328085640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd328085640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 0x7fd32013c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd328085640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd328085640 1 -- 172.21.15.89:0/4068572537 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd3200a3160 con 0x7fd32013c370 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd3180143a0 con 0x7fd3200743c0 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd325620640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 0x7fd32013c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd325620640 10 monclient: get_auth_request con 0x7fd32013c370 auth_method 0 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd325620640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd325620640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd325620640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd325620640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 msgr2=0x7fd32013c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:53:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd325620640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 msgr2=0x7fd32013c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:53:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd325620640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 0x7fd32013c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd325620640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 0x7fd32013c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd30403be30 0x7fd30403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.538+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fd31804f2d0 con 0x7fd3200743c0 2024-08-19T10:53:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.542+0000 7fd326622640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd30403be30 0x7fd30403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.542+0000 7fd326622640 10 monclient: get_auth_request con 0x7fd30403be30 auth_method 0 2024-08-19T10:53:27.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.542+0000 7fd326622640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd30403be30 0x7fd30403e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0ad000 tx=0x7fd31c004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 0x7fd32013c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 10 monclient: get_auth_request con 0x7fd32013c370 auth_method 0 2024-08-19T10:53:27.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:27.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 10 monclient(hunting): handle_auth_done global_id 84109 payload 824 2024-08-19T10:53:27.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 1 -- 172.21.15.89:0/4068572537 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3200a5600 con 0x7fd32013c370 2024-08-19T10:53:27.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 msgr2=0x7fd32013c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd325620640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 0x7fd32013c750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd32013cce0 tx=0x7fd310013d10 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd31001c070 con 0x7fd32013c370 2024-08-19T10:53:27.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd3167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd3167fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.742+0000 7fd3167fc640 10 monclient: dump: 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.743 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.744 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.744 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.744 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.744 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.744 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.744 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.746+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd310009e20 con 0x7fd32013c370 2024-08-19T10:53:27.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.746+0000 7fd3167fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:27.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.746+0000 7fd3167fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:27.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.746+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 msgr2=0x7fd32013c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.746+0000 7fd3167fc640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 0x7fd32013c750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd32013cce0 tx=0x7fd310013d10 comp rx=0 tx=0).stop 2024-08-19T10:53:27.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.746+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 msgr2=0x7fd32013c750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:27.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.746+0000 7fd3167fc640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32013c370 0x7fd32013c750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd328085640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:27.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd328085640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32002d1d0 0x7fd32002d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd328085640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd328085640 1 -- 172.21.15.89:0/4068572537 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fd320064590 con 0x7fd32002d1d0 2024-08-19T10:53:27.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd325620640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32002d1d0 0x7fd32002d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd325620640 10 monclient: get_auth_request con 0x7fd32002d1d0 auth_method 0 2024-08-19T10:53:27.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd325620640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd325620640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd325620640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd325620640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd325620640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd325620640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd325620640 10 monclient(hunting): handle_auth_done global_id 84118 payload 824 2024-08-19T10:53:27.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.754+0000 7fd325620640 1 -- 172.21.15.89:0/4068572537 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd3200a0100 con 0x7fd32002d1d0 2024-08-19T10:53:27.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd325620640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32002d1d0 msgr2=0x7fd32002d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd325620640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32002d1d0 0x7fd32002d5e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd32013cce0 tx=0x7fd310004740 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd310011e60 con 0x7fd32002d1d0 2024-08-19T10:53:27.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd3167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd3167fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:27.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd3167fc640 10 monclient: dump: 2024-08-19T10:53:27.755 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.755 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.756 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.756 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.756 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.756 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.756 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.756 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.756 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.756 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.756 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.756 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.757 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.757 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.757 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.757 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7fd310011e60 con 0x7fd32002d1d0 2024-08-19T10:53:27.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd3167fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:27.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd3167fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:27.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32002d1d0 msgr2=0x7fd32002d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd3167fc640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32002d1d0 0x7fd32002d5e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd32013cce0 tx=0x7fd310004740 comp rx=0 tx=0).stop 2024-08-19T10:53:27.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd3167fc640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32002d1d0 msgr2=0x7fd32002d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:27.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd3167fc640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fd32002d1d0 0x7fd32002d5e0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd32013cce0 tx=0x7fd310004740 comp rx=0 tx=0).stop 2024-08-19T10:53:27.758 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:27.758 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:27.758 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:27.758 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:27.758 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:27.758 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 16059, 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:53:27.759 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.760 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.761 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:27.762 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.763 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.764 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.765 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.766 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.767 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.769 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:27.770 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.771 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.772 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:27.773 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:27.774 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.775 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.776 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.777 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.778 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.779 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.780 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.781 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.782 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.783 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:27.783 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:27.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd328085640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd30403be30 msgr2=0x7fd30403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd328085640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd30403be30 0x7fd30403e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0ad000 tx=0x7fd31c004740 comp rx=0 tx=0).stop 2024-08-19T10:53:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd328085640 10 monclient: shutdown 2024-08-19T10:53:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd328085640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd328085640 1 -- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3200743c0 msgr2=0x7fd320136050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd328085640 1 --2- 172.21.15.89:0/4068572537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3200743c0 0x7fd320136050 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd318007af0 tx=0x7fd31800d0d0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd326622640 1 -- 172.21.15.89:0/4068572537 reap_dead start 2024-08-19T10:53:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd326622640 1 -- 172.21.15.89:0/4068572537 reap_dead start 2024-08-19T10:53:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd328085640 1 -- 172.21.15.89:0/4068572537 shutdown_connections 2024-08-19T10:53:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd328085640 1 -- 172.21.15.89:0/4068572537 >> 172.21.15.89:0/4068572537 conn(0x7fd3200aa240 msgr2=0x7fd3200a1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd328085640 1 -- 172.21.15.89:0/4068572537 shutdown_connections 2024-08-19T10:53:27.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.758+0000 7fd328085640 1 -- 172.21.15.89:0/4068572537 wait complete. 2024-08-19T10:53:27.786 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:27.786 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:53:27.786 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:53:27.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient: get_monmap_and_config 2024-08-19T10:53:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient: build_initial_monmap 2024-08-19T10:53:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient: monmap: 2024-08-19T10:53:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.916 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:27.921191+0000 2024-08-19T10:53:27.917 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:27.921191+0000 2024-08-19T10:53:27.917 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:27.917 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.917 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:27.917 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:27.917 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:27.919 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 1 Processor -- start 2024-08-19T10:53:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 1 -- start start 2024-08-19T10:53:27.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient: init 2024-08-19T10:53:27.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:27.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient: _add_conns ranks=[1,4,10,6,2,7,3,0,5,8,9] 2024-08-19T10:53:27.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f30980a6860 addr v1:172.21.15.190:6789/0 2024-08-19T10:53:27.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30980a7350 0x7f309813e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient(hunting): picked mon.noname-c con 0x7f30980a7350 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:27.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f30980a7730 0x7f3098140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient(hunting): picked mon.noname-f con 0x7f30980a7730 addr v2:172.21.15.190:3300/0 2024-08-19T10:53:27.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30980a33d0 con 0x7f30980a6860 2024-08-19T10:53:27.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f30980a8190 con 0x7f30980a7730 2024-08-19T10:53:27.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f30980a2ec0 con 0x7f30980a7350 2024-08-19T10:53:27.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:27.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f309e122640 10 monclient(hunting): authenticate will time out at 1857.737904s 2024-08-19T10:53:27.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30980a7350 0x7f309813e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30980a7350 0x7f309813e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49084/0 (socket says 172.21.15.89:49084) 2024-08-19T10:53:27.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 1 -- 172.21.15.89:0/1993995444 learned_addr learned my addr 172.21.15.89:0/1993995444 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:27.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 10 monclient(hunting): get_auth_request con 0x7f30980a7350 auth_method 0 2024-08-19T10:53:27.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3097fff640 1 --2- 172.21.15.89:0/1993995444 >> v2:172.21.15.190:3300/0 conn(0x7f30980a7730 0x7f3098140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3097fff640 10 monclient(hunting): get_auth_request con 0x7f30980a7730 auth_method 0 2024-08-19T10:53:27.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3097fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3097fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3097fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 10 monclient(hunting): handle_auth_done global_id 84127 payload 274 2024-08-19T10:53:27.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 1 -- 172.21.15.89:0/1993995444 >> v2:172.21.15.190:3300/0 conn(0x7f30980a7730 msgr2=0x7f3098140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:27.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 1 --2- 172.21.15.89:0/1993995444 >> v2:172.21.15.190:3300/0 conn(0x7f30980a7730 0x7f3098140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 1 -- 172.21.15.89:0/1993995444 >> v1:172.21.15.190:6789/0 conn(0x7f30980a6860 legacy=0x7f30980a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:27.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:53:27.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 1 monclient: found mon.noname-c 2024-08-19T10:53:27.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:27.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:53:27.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.918+0000 7f3096ffd640 1 -- 172.21.15.89:0/1993995444 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30980a4cd0 con 0x7f30980a7350 2024-08-19T10:53:27.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f3096ffd640 1 -- 172.21.15.89:0/1993995444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30980a7350 msgr2=0x7f309813e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f3096ffd640 1 --2- 172.21.15.89:0/1993995444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30980a7350 0x7f309813e860 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3088009e50 tx=0x7f308801d350 comp rx=0 tx=0).ready entity=mon.4 client_cookie=69e6f0d54a005171 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 1 -- 172.21.15.89:0/1993995444 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30880090d0 con 0x7f30980a7350 2024-08-19T10:53:27.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:53:27.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 10 monclient: dump: 2024-08-19T10:53:27.948 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.948 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.948 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.948 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.948 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.948 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 10 monclient: _finish_auth 0 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 10 monclient: _check_auth_tickets 2024-08-19T10:53:27.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f309e122640 5 monclient: authenticate success, global_id 84127 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f309e122640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 1 -- 172.21.15.89:0/1993995444 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30880a5640 con 0x7f30980a7350 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 1 -- 172.21.15.89:0/1993995444 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3088009270 con 0x7f30980a7350 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f30967fc640 10 monclient: dump: 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.950 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f309e122640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.922+0000 7f309e122640 4 set_mon_vals no callback set 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient: shutdown 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 -- 172.21.15.89:0/1993995444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30980a7350 msgr2=0x7f309813e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 --2- 172.21.15.89:0/1993995444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30980a7350 0x7f309813e860 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3088009e50 tx=0x7f308801d350 comp rx=0 tx=0).stop 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 -- 172.21.15.89:0/1993995444 shutdown_connections 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 --2- 172.21.15.89:0/1993995444 >> v2:172.21.15.190:3300/0 conn(0x7f30980a7730 0x7f3098140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 --2- 172.21.15.89:0/1993995444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f30980a7350 0x7f309813e860 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 -- 172.21.15.89:0/1993995444 >> 172.21.15.89:0/1993995444 conn(0x7f3098074b40 msgr2=0x7f30980abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 -- 172.21.15.89:0/1993995444 shutdown_connections 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 -- 172.21.15.89:0/1993995444 wait complete. 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient: build_initial_monmap 2024-08-19T10:53:27.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient: monmap: 2024-08-19T10:53:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:27.953 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 Processor -- start 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 -- start start 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient: init 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient: _add_conns ranks=[2,0,5,7,1,6,4,3,8] 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30980a6860 0x7f3098128940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient(hunting): picked mon.noname-b con 0x7f30980a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:27.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30980a7350 0x7f309813f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient(hunting): picked mon.noname-a con 0x7f30980a7350 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f30980a7730 0x7f3098126370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient(hunting): picked mon.noname-h con 0x7f30980a7730 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3098143ea0 con 0x7f30980a7350 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3098143d20 con 0x7f30980a6860 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f3098144020 con 0x7f30980a7730 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f309e122640 10 monclient(hunting): authenticate will time out at 1857.746220s 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30980a7350 0x7f309813f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f30977fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30980a6860 0x7f3098128940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30980a7350 0x7f309813f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39910/0 (socket says 172.21.15.89:39910) 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 1 -- 172.21.15.89:0/1352508998 learned_addr learned my addr 172.21.15.89:0/1352508998 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f30977fe640 10 monclient(hunting): get_auth_request con 0x7f30980a6860 auth_method 0 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f30977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f30977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f30977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 10 monclient(hunting): get_auth_request con 0x7f30980a7350 auth_method 0 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f30977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f30977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f30977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3097fff640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f30980a7730 0x7f3098126370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3097fff640 10 monclient(hunting): get_auth_request con 0x7f30980a7730 auth_method 0 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3097fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3097fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3097fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 10 monclient(hunting): handle_auth_done global_id 7299 payload 824 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 1 -- 172.21.15.89:0/1352508998 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f30980a7730 msgr2=0x7f3098126370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f30980a7730 0x7f3098126370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 1 -- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30980a6860 msgr2=0x7f3098128940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f30980a6860 0x7f3098128940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:53:27.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 1 monclient: found mon.noname-a 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.926+0000 7f3096ffd640 1 -- 172.21.15.89:0/1352508998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30981292d0 con 0x7f30980a7350 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f3096ffd640 1 -- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30980a7350 msgr2=0x7f309813f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f3096ffd640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30980a7350 0x7f309813f7b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f308801d190 tx=0x7f3088019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3088009210 con 0x7f30980a7350 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 10 monclient: dump: 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.958 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 10 monclient: _finish_auth 0 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30880a5c10 con 0x7f30980a7350 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f30880a5dc0 con 0x7f30980a7350 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f309e122640 5 monclient: authenticate success, global_id 7299 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f309e122640 10 monclient: _renew_subs 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f309e122640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f309e122640 1 -- 172.21.15.89:0/1352508998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309812a140 con 0x7f30980a7350 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f30957fa640 4 set_mon_vals no callback set 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:27.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 10 monclient: dump: 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.961 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f309e122640 10 monclient: _renew_subs 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f309e122640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f309e122640 1 -- 172.21.15.89:0/1352508998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30981409c0 con 0x7f30980a7350 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f309e122640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f309e122640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981268b0 0x7f3098126c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f309e122640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f309e122640 1 -- 172.21.15.89:0/1352508998 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f30981271d0 con 0x7f30981268b0 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f30977fe640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981268b0 0x7f3098126c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f30977fe640 10 monclient: get_auth_request con 0x7f30981268b0 auth_method 0 2024-08-19T10:53:27.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f30977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f30977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f30977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f30977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f30977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f30977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f30977fe640 10 monclient(hunting): handle_auth_done global_id 45078 payload 824 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3088004040 con 0x7f30980a7350 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f30977fe640 1 -- 172.21.15.89:0/1352508998 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30980a9420 con 0x7f30981268b0 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f307803be30 0x7f307803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f30880e9970 con 0x7f30980a7350 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f3097fff640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f307803be30 0x7f307803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.930+0000 7f3097fff640 10 monclient: get_auth_request con 0x7f307803be30 auth_method 0 2024-08-19T10:53:27.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f30977fe640 1 -- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981268b0 msgr2=0x7f3098126c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f30977fe640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981268b0 0x7f3098126c90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f308000f770 tx=0x7f308000fc30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f308000fe40 con 0x7f30981268b0 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f308ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f308ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f308ffff640 10 monclient: dump: 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.964 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3080019070 con 0x7f30981268b0 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f308ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f308ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981268b0 msgr2=0x7f3098126c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f308ffff640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981268b0 0x7f3098126c90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f308000f770 tx=0x7f308000fc30 comp rx=0 tx=0).stop 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981268b0 msgr2=0x7f3098126c90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f308ffff640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981268b0 0x7f3098126c90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.934+0000 7f3097fff640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f307803be30 0x7f307803e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f309000d8d0 tx=0x7f3090008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f309e122640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f309e122640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981224b0 0x7f30981228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f309e122640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f309e122640 1 -- 172.21.15.89:0/1352508998 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f30980a6c40 con 0x7f30981224b0 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f30977fe640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981224b0 0x7f30981228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f30977fe640 10 monclient: get_auth_request con 0x7f30981224b0 auth_method 0 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f30977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f30977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f30977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:27.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f30977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:27.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f30977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:27.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f30977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:27.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f30977fe640 10 monclient(hunting): handle_auth_done global_id 45087 payload 824 2024-08-19T10:53:27.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.938+0000 7f30977fe640 1 -- 172.21.15.89:0/1352508998 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3098064590 con 0x7f30981224b0 2024-08-19T10:53:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f30977fe640 1 -- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981224b0 msgr2=0x7f30981228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f30977fe640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981224b0 0x7f30981228c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f308000f770 tx=0x7f3080003770 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f308001ec30 con 0x7f30981224b0 2024-08-19T10:53:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f308ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f308ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f308ffff640 10 monclient: dump: 2024-08-19T10:53:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:27.969 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:27.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f308001e7c0 con 0x7f30981224b0 2024-08-19T10:53:27.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f308ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:27.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f308ffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:27.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981224b0 msgr2=0x7f30981228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f308ffff640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981224b0 0x7f30981228c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f308000f770 tx=0x7f3080003770 comp rx=0 tx=0).stop 2024-08-19T10:53:27.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f308ffff640 1 -- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981224b0 msgr2=0x7f30981228c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:27.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f308ffff640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f30981224b0 0x7f30981228c0 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f308000f770 tx=0x7f3080003770 comp rx=0 tx=0).stop 2024-08-19T10:53:27.970 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 531068, 2024-08-19T10:53:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:27.971 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.972 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.973 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:27.974 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:27.975 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.976 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.977 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.978 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.980 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:27.981 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.982 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.983 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.984 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.985 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:27.986 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:27.987 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:27.990 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:27.992 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:53:27.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:27.994 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:27.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:27.995 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:27.995 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:27.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f309e122640 1 -- 172.21.15.89:0/1352508998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f307803be30 msgr2=0x7f307803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f309e122640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f307803be30 0x7f307803e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f309000d8d0 tx=0x7f3090008040 comp rx=0 tx=0).stop 2024-08-19T10:53:27.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f309e122640 10 monclient: shutdown 2024-08-19T10:53:27.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f309e122640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:27.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f309e122640 1 -- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30980a7350 msgr2=0x7f309813f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:27.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f309e122640 1 --2- 172.21.15.89:0/1352508998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30980a7350 0x7f309813f7b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f308801d190 tx=0x7f3088019690 comp rx=0 tx=0).stop 2024-08-19T10:53:27.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f3097fff640 1 -- 172.21.15.89:0/1352508998 reap_dead start 2024-08-19T10:53:27.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f3097fff640 1 -- 172.21.15.89:0/1352508998 reap_dead start 2024-08-19T10:53:27.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f309e122640 1 -- 172.21.15.89:0/1352508998 shutdown_connections 2024-08-19T10:53:27.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f309e122640 1 -- 172.21.15.89:0/1352508998 >> 172.21.15.89:0/1352508998 conn(0x7f3098074b40 msgr2=0x7f30980ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:27.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f309e122640 1 -- 172.21.15.89:0/1352508998 shutdown_connections 2024-08-19T10:53:27.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:27.942+0000 7f309e122640 1 -- 172.21.15.89:0/1352508998 wait complete. 2024-08-19T10:53:27.997 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:27.997 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:53:27.998 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:53:28.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient: get_monmap_and_config 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient: build_initial_monmap 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient: monmap: 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:28.121071+0000 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:28.121071+0000 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:28.115 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:28.116 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:28.116 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:28.116 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:28.116 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:28.116 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:28.116 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:28.116 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 1 Processor -- start 2024-08-19T10:53:28.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 1 -- start start 2024-08-19T10:53:28.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient: init 2024-08-19T10:53:28.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient: _add_conns ranks=[10,9,1,8,3,5,6,0,2,4,7] 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f768809b8e0 0x7f768809bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient(hunting): picked mon.noname-f con 0x7f768809b8e0 addr v2:172.21.15.190:3300/0 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f768809ddd0 0x7f76880a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient(hunting): picked mon.noname-a con 0x7f768809ddd0 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f76880a4240 addr v1:172.21.15.190:6789/0 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f768806db50 con 0x7f76880a4240 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f768809d760 con 0x7f768809ddd0 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f768809d9b0 con 0x7f768809b8e0 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f7690538640 10 monclient(hunting): authenticate will time out at 1857.937617s 2024-08-19T10:53:28.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f768809ddd0 0x7f76880a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f768809ddd0 0x7f76880a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39916/0 (socket says 172.21.15.89:39916) 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 1 -- 172.21.15.89:0/138510144 learned_addr learned my addr 172.21.15.89:0/138510144 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 10 monclient(hunting): get_auth_request con 0x7f768809ddd0 auth_method 0 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768e2d4640 1 --2- 172.21.15.89:0/138510144 >> v2:172.21.15.190:3300/0 conn(0x7f768809b8e0 0x7f768809bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768e2d4640 10 monclient(hunting): get_auth_request con 0x7f768809b8e0 auth_method 0 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768e2d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768e2d4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768e2d4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 10 monclient(hunting): handle_auth_done global_id 7308 payload 274 2024-08-19T10:53:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 1 -- 172.21.15.89:0/138510144 >> v2:172.21.15.190:3300/0 conn(0x7f768809b8e0 msgr2=0x7f768809bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 1 --2- 172.21.15.89:0/138510144 >> v2:172.21.15.190:3300/0 conn(0x7f768809b8e0 0x7f768809bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 1 -- 172.21.15.89:0/138510144 >> v1:172.21.15.190:6789/0 conn(0x7f76880a4240 legacy=0x7f76881404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 10 monclient: _finish_hunting 0 2024-08-19T10:53:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 1 monclient: found mon.noname-a 2024-08-19T10:53:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:28.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.118+0000 7f768dad3640 1 -- 172.21.15.89:0/138510144 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768806fd00 con 0x7f768809ddd0 2024-08-19T10:53:28.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768dad3640 1 -- 172.21.15.89:0/138510144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768809ddd0 msgr2=0x7f76880a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768dad3640 1 --2- 172.21.15.89:0/138510144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768809ddd0 0x7f76880a3d00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f76880705b0 tx=0x7f768401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed6ae0ce525901 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 1 -- 172.21.15.89:0/138510144 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76840b0070 con 0x7f768809ddd0 2024-08-19T10:53:28.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:28.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 10 monclient: dump: 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 10 monclient: _finish_auth 0 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 10 monclient: _check_auth_tickets 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 1 -- 172.21.15.89:0/138510144 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76840aa810 con 0x7f768809ddd0 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 1 -- 172.21.15.89:0/138510144 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76840a9300 con 0x7f768809ddd0 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f768d2d2640 10 monclient: dump: 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 5 monclient: authenticate success, global_id 7308 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 4 set_mon_vals no callback set 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 10 monclient: shutdown 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 1 -- 172.21.15.89:0/138510144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768809ddd0 msgr2=0x7f76880a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 1 --2- 172.21.15.89:0/138510144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768809ddd0 0x7f76880a3d00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f76880705b0 tx=0x7f768401c3c0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 1 -- 172.21.15.89:0/138510144 shutdown_connections 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 1 --2- 172.21.15.89:0/138510144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768809ddd0 0x7f76880a3d00 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 1 --2- 172.21.15.89:0/138510144 >> v2:172.21.15.190:3300/0 conn(0x7f768809b8e0 0x7f768809bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 1 -- 172.21.15.89:0/138510144 >> 172.21.15.89:0/138510144 conn(0x7f76880a4df0 msgr2=0x7f76880a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 1 -- 172.21.15.89:0/138510144 shutdown_connections 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 1 -- 172.21.15.89:0/138510144 wait complete. 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 10 monclient: build_initial_monmap 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.122+0000 7f7690538640 10 monclient: monmap: 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 1 Processor -- start 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 1 -- start start 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient: init 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient: _add_conns ranks=[6,8,1,7,2,4,5,3,0] 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f768809b8e0 0x7f76881d1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient(hunting): picked mon.noname-d con 0x7f768809b8e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f768809ddd0 0x7f768809a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient(hunting): picked mon.noname-e con 0x7f768809ddd0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76880a4240 0x7f768809ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient(hunting): picked mon.noname-f con 0x7f76880a4240 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7688069300 con 0x7f76880a4240 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f76880a4a80 con 0x7f768809b8e0 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f76881443f0 con 0x7f768809ddd0 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient(hunting): authenticate will time out at 1857.944944s 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f768809ddd0 0x7f768809a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f768809ddd0 0x7f768809a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:55322/0 (socket says 172.21.15.89:55322) 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768e2d4640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f768809b8e0 0x7f76881d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 1 -- 172.21.15.89:0/4246727545 learned_addr learned my addr 172.21.15.89:0/4246727545 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 10 monclient(hunting): get_auth_request con 0x7f768809ddd0 auth_method 0 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768e2d4640 10 monclient(hunting): get_auth_request con 0x7f768809b8e0 auth_method 0 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768e2d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768e2d4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768e2d4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768e2d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768e2d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768e2d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 10 monclient(hunting): handle_auth_done global_id 84113 payload 824 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768ead5640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76880a4240 0x7f768809ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 1 -- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f768809b8e0 msgr2=0x7f76881d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f768809b8e0 0x7f76881d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 1 -- 172.21.15.89:0/4246727545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76880a4240 msgr2=0x7f768809ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76880a4240 0x7f768809ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 10 monclient: _finish_hunting 0 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 1 monclient: found mon.noname-e 2024-08-19T10:53:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:53:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 1 -- 172.21.15.89:0/4246727545 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7688092ac0 con 0x7f768809ddd0 2024-08-19T10:53:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768ead5640 10 monclient: get_auth_request con 0x7f76880a4240 auth_method 0 2024-08-19T10:53:28.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768ead5640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76880a4240 0x7f768809ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:53:28.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 1 -- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f768809ddd0 msgr2=0x7f768809a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768dad3640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f768809ddd0 0x7f768809a8f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76840ae040 tx=0x7f7684019160 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76840b9070 con 0x7f768809ddd0 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 10 monclient: dump: 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.140 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 10 monclient: _finish_auth 0 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76840044b0 con 0x7f768809ddd0 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 5 monclient: authenticate success, global_id 84113 2024-08-19T10:53:28.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient: _renew_subs 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f7690538640 1 -- 172.21.15.89:0/4246727545 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76880a39d0 con 0x7f768809ddd0 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f76840b0040 con 0x7f768809ddd0 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f767effd640 10 monclient: dump: 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.142 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f7690538640 10 monclient: _renew_subs 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f7690538640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f7690538640 1 -- 172.21.15.89:0/4246727545 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f7688093360 con 0x7f768809ddd0 2024-08-19T10:53:28.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.126+0000 7f768cad1640 4 set_mon_vals no callback set 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f7690538640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f7690538640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072020 0x7f7688072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f7690538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f7690538640 1 -- 172.21.15.89:0/4246727545 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7688064590 con 0x7f7688072020 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072020 0x7f7688072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 10 monclient: get_auth_request con 0x7f7688072020 auth_method 0 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7684004040 con 0x7f768809ddd0 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 10 monclient(hunting): handle_auth_done global_id 84122 payload 824 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 1 -- 172.21.15.89:0/4246727545 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76880a3f00 con 0x7f7688072020 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f766c03be00 0x7f766c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f76840e8950 con 0x7f768809ddd0 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768ead5640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f766c03be00 0x7f766c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768ead5640 10 monclient: get_auth_request con 0x7f766c03be00 auth_method 0 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 1 -- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072020 msgr2=0x7f7688072400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f768e2d4640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072020 0x7f7688072400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7678010780 tx=0x7f7678010c40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7678018020 con 0x7f7688072020 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 10 monclient: dump: 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.145 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.146 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7678004450 con 0x7f7688072020 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072020 msgr2=0x7f7688072400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072020 0x7f7688072400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7678010780 tx=0x7f7678010c40 comp rx=0 tx=0).stop 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072020 msgr2=0x7f7688072400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.130+0000 7f767effd640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072020 0x7f7688072400 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7678010780 tx=0x7f7678010c40 comp rx=0 tx=0).stop 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.134+0000 7f768ead5640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f766c03be00 0x7f766c03e2c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7680007e10 tx=0x7f7680007320 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f7690538640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f7690538640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072940 0x7f768813ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f7690538640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f7690538640 1 -- 172.21.15.89:0/4246727545 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f768809b370 con 0x7f7688072940 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f768e2d4640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072940 0x7f768813ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f768e2d4640 10 monclient: get_auth_request con 0x7f7688072940 auth_method 0 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f768e2d4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f768e2d4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f768e2d4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f768e2d4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f768e2d4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f768e2d4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f768e2d4640 10 monclient(hunting): handle_auth_done global_id 84131 payload 824 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.138+0000 7f768e2d4640 1 -- 172.21.15.89:0/4246727545 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f768809bcc0 con 0x7f7688072940 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f768e2d4640 1 -- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072940 msgr2=0x7f768813ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f768e2d4640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072940 0x7f768813ec00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7678019000 tx=0x7f76780046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7678018040 con 0x7f7688072940 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f767effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f767effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f767effd640 10 monclient: dump: 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.149 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.150 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.150 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.150 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.150 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f7678003620 con 0x7f7688072940 2024-08-19T10:53:28.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f767effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:28.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f767effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:28.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072940 msgr2=0x7f768813ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f767effd640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072940 0x7f768813ec00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7678019000 tx=0x7f76780046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f767effd640 1 -- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072940 msgr2=0x7f768813ec00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:28.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f767effd640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7688072940 0x7f768813ec00 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7678019000 tx=0x7f76780046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 16447, 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:28.151 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.152 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.153 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:28.154 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:28.155 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.156 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.157 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.158 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.159 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.160 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:28.161 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.163 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.164 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:28.165 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.167 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:28.168 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.171 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:28.172 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.174 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:28.174 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f7690538640 1 -- 172.21.15.89:0/4246727545 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f766c03be00 msgr2=0x7f766c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f7690538640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f766c03be00 0x7f766c03e2c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7680007e10 tx=0x7f7680007320 comp rx=0 tx=0).stop 2024-08-19T10:53:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f7690538640 10 monclient: shutdown 2024-08-19T10:53:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f7690538640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:28.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f7690538640 1 -- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f768809ddd0 msgr2=0x7f768809a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f7690538640 1 --2- 172.21.15.89:0/4246727545 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f768809ddd0 0x7f768809a8f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76840ae040 tx=0x7f7684019160 comp rx=0 tx=0).stop 2024-08-19T10:53:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f768ead5640 1 -- 172.21.15.89:0/4246727545 reap_dead start 2024-08-19T10:53:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f768ead5640 1 -- 172.21.15.89:0/4246727545 reap_dead start 2024-08-19T10:53:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f7690538640 1 -- 172.21.15.89:0/4246727545 shutdown_connections 2024-08-19T10:53:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f7690538640 1 -- 172.21.15.89:0/4246727545 >> 172.21.15.89:0/4246727545 conn(0x7f76880a4df0 msgr2=0x7f768813d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f7690538640 1 -- 172.21.15.89:0/4246727545 shutdown_connections 2024-08-19T10:53:28.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.142+0000 7f7690538640 1 -- 172.21.15.89:0/4246727545 wait complete. 2024-08-19T10:53:28.176 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:28.176 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:53:28.177 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:53:28.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient: get_monmap_and_config 2024-08-19T10:53:28.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient: build_initial_monmap 2024-08-19T10:53:28.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient: monmap: 2024-08-19T10:53:28.291 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:28.291 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.291 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:28.297551+0000 2024-08-19T10:53:28.291 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:28.297551+0000 2024-08-19T10:53:28.291 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:28.292 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:28.293 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:28.293 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:28.293 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 1 Processor -- start 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 1 -- start start 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient: init 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient: _add_conns ranks=[0,3,6,5,9,4,8,2,10,1,7] 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fca080a56b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca080a1660 0x7fca080a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient(hunting): picked mon.noname-g con 0x7fca080a1660 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca08072250 0x7fca0813e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient(hunting): picked mon.noname-d con 0x7fca08072250 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca08074010 con 0x7fca080a56b0 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fca08072010 con 0x7fca080a1660 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fca08073d30 con 0x7fca08072250 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0f7b9640 10 monclient(hunting): authenticate will time out at 1858.114102s 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0d555640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fca080a56b0 0x7fca080a5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60274/0 (socket says 172.21.15.89:60274) 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0d555640 1 -- 172.21.15.89:0/4254879851 learned_addr learned my addr 172.21.15.89:0/4254879851 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 1 --2- 172.21.15.89:0/4254879851 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca08072250 0x7fca0813e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 10 monclient(hunting): get_auth_request con 0x7fca08072250 auth_method 0 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0cd54640 1 --2- 172.21.15.89:0/4254879851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca080a1660 0x7fca080a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0cd54640 10 monclient(hunting): get_auth_request con 0x7fca080a1660 auth_method 0 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0cd54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0cd54640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0cd54640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 10 monclient(hunting): handle_auth_done global_id 45105 payload 274 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 1 -- 172.21.15.89:0/4254879851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca080a1660 msgr2=0x7fca080a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 1 --2- 172.21.15.89:0/4254879851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca080a1660 0x7fca080a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 1 -- 172.21.15.89:0/4254879851 >> v1:172.21.15.89:6789/0 conn(0x7fca080a56b0 legacy=0x7fca080a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 10 monclient: _finish_hunting 0 2024-08-19T10:53:28.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 1 monclient: found mon.noname-d 2024-08-19T10:53:28.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:28.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:28.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.294+0000 7fca0dd56640 1 -- 172.21.15.89:0/4254879851 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca080a71a0 con 0x7fca08072250 2024-08-19T10:53:28.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0dd56640 1 -- 172.21.15.89:0/4254879851 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca08072250 msgr2=0x7fca0813e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0dd56640 1 --2- 172.21.15.89:0/4254879851 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca08072250 0x7fca0813e6b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fca00009dc0 tx=0x7fca0001c850 comp rx=0 tx=0).ready entity=mon.6 client_cookie=aba26ae639a955ba server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 1 -- 172.21.15.89:0/4254879851 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca000b2070 con 0x7fca08072250 2024-08-19T10:53:28.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:28.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 10 monclient: dump: 2024-08-19T10:53:28.302 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.302 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.303 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 10 monclient: _finish_auth 0 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 10 monclient: _check_auth_tickets 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 1 -- 172.21.15.89:0/4254879851 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca000a5cd0 con 0x7fca08072250 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 5 monclient: authenticate success, global_id 45105 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 1 -- 172.21.15.89:0/4254879851 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca000ab7b0 con 0x7fca08072250 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 4 set_mon_vals no callback set 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fc9fffff640 10 monclient: dump: 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.306 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.306 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 10 monclient: shutdown 2024-08-19T10:53:28.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:28.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 1 -- 172.21.15.89:0/4254879851 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca08072250 msgr2=0x7fca0813e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 1 --2- 172.21.15.89:0/4254879851 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca08072250 0x7fca0813e6b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fca00009dc0 tx=0x7fca0001c850 comp rx=0 tx=0).stop 2024-08-19T10:53:28.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 1 -- 172.21.15.89:0/4254879851 shutdown_connections 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 1 --2- 172.21.15.89:0/4254879851 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca08072250 0x7fca0813e6b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 1 --2- 172.21.15.89:0/4254879851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca080a1660 0x7fca080a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 1 -- 172.21.15.89:0/4254879851 >> 172.21.15.89:0/4254879851 conn(0x7fca080aa2a0 msgr2=0x7fca080a67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 1 -- 172.21.15.89:0/4254879851 shutdown_connections 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.298+0000 7fca0f7b9640 1 -- 172.21.15.89:0/4254879851 wait complete. 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient: build_initial_monmap 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient: monmap: 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 1 Processor -- start 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 1 -- start start 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient: init 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient: _add_conns ranks=[6,0,7,8,3,5,1,2,4] 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca080a56b0 0x7fca0806da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient(hunting): picked mon.noname-d con 0x7fca080a56b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca0813ddf0 0x7fca0806df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient(hunting): picked mon.noname-a con 0x7fca0813ddf0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca0806e4c0 0x7fca08132610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient(hunting): picked mon.noname-i con 0x7fca0806e4c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca081417a0 con 0x7fca0813ddf0 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fca08141620 con 0x7fca080a56b0 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fca08141920 con 0x7fca0806e4c0 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0f7b9640 10 monclient(hunting): authenticate will time out at 1858.121732s 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0cd54640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca0813ddf0 0x7fca0806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca080a56b0 0x7fca0806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0cd54640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca0813ddf0 0x7fca0806df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39926/0 (socket says 172.21.15.89:39926) 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca080a56b0 0x7fca0806da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:60876/0 (socket says 172.21.15.89:60876) 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0cd54640 1 -- 172.21.15.89:0/3015821730 learned_addr learned my addr 172.21.15.89:0/3015821730 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0cd54640 10 monclient(hunting): get_auth_request con 0x7fca0813ddf0 auth_method 0 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0cd54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0cd54640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0cd54640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 10 monclient(hunting): get_auth_request con 0x7fca080a56b0 auth_method 0 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0cd54640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0cd54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0cd54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0dd56640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca0806e4c0 0x7fca08132610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 10 monclient(hunting): handle_auth_done global_id 45114 payload 824 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 1 -- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca0806e4c0 msgr2=0x7fca08132610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fca0806e4c0 0x7fca08132610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 1 -- 172.21.15.89:0/3015821730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca0813ddf0 msgr2=0x7fca0806df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca0813ddf0 0x7fca0806df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 10 monclient: _finish_hunting 0 2024-08-19T10:53:28.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 1 monclient: found mon.noname-d 2024-08-19T10:53:28.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:28.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:28.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.302+0000 7fca0d555640 1 -- 172.21.15.89:0/3015821730 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0813e8e0 con 0x7fca080a56b0 2024-08-19T10:53:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0d555640 1 -- 172.21.15.89:0/3015821730 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca080a56b0 msgr2=0x7fca0806da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0d555640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca080a56b0 0x7fca0806da40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fca0400b4c0 tx=0x7fca0400b980 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca04015020 con 0x7fca080a56b0 2024-08-19T10:53:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 10 monclient: dump: 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 10 monclient: _finish_auth 0 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0f7b9640 5 monclient: authenticate success, global_id 45114 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca04011cb0 con 0x7fca080a56b0 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0f7b9640 10 monclient: _renew_subs 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0f7b9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0f7b9640 1 -- 172.21.15.89:0/3015821730 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0813cb20 con 0x7fca080a56b0 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fca04011e60 con 0x7fca080a56b0 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9feffd640 4 set_mon_vals no callback set 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 10 monclient: dump: 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0f7b9640 10 monclient: _renew_subs 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0f7b9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0f7b9640 1 -- 172.21.15.89:0/3015821730 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0813e610 con 0x7fca080a56b0 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0f7b9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0f7b9640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca08072250 0x7fca08132b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0f7b9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0f7b9640 1 -- 172.21.15.89:0/3015821730 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca080a5a90 con 0x7fca08072250 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fca04018020 con 0x7fca080a56b0 2024-08-19T10:53:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc9ec03be30 0x7fc9ec03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0cd54640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca08072250 0x7fca08132b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fca0404f150 con 0x7fca080a56b0 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0cd54640 10 monclient: get_auth_request con 0x7fca08072250 auth_method 0 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0cd54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0cd54640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0cd54640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0cd54640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0cd54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0cd54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0dd56640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc9ec03be30 0x7fc9ec03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0dd56640 10 monclient: get_auth_request con 0x7fc9ec03be30 auth_method 0 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0cd54640 10 monclient(hunting): handle_auth_done global_id 45091 payload 824 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.306+0000 7fca0cd54640 1 -- 172.21.15.89:0/3015821730 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca08064590 con 0x7fca08072250 2024-08-19T10:53:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fca0dd56640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc9ec03be30 0x7fc9ec03e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fca08074630 tx=0x7fca000a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fca0cd54640 1 -- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca08072250 msgr2=0x7fca08132b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fca0cd54640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca08072250 0x7fca08132b50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8006e90 tx=0x7fc9f8010cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9f80044a0 con 0x7fca08072250 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fc9fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fc9fdffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fc9fdffb640 10 monclient: dump: 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc9f800ebf0 con 0x7fca08072250 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fc9fdffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fc9fdffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca08072250 msgr2=0x7fca08132b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fc9fdffb640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca08072250 0x7fca08132b50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8006e90 tx=0x7fc9f8010cf0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca08072250 msgr2=0x7fca08132b50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.310+0000 7fc9fdffb640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca08072250 0x7fca08132b50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0f7b9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0f7b9640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca080a15f0 0x7fca08133090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0f7b9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0f7b9640 1 -- 172.21.15.89:0/3015821730 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca0806fd40 con 0x7fca080a15f0 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca080a15f0 0x7fca08133090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 10 monclient: get_auth_request con 0x7fca080a15f0 auth_method 0 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 10 monclient(hunting): handle_auth_done global_id 45100 payload 824 2024-08-19T10:53:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 1 -- 172.21.15.89:0/3015821730 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fca0813d170 con 0x7fca080a15f0 2024-08-19T10:53:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 1 -- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca080a15f0 msgr2=0x7fca08133090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.318+0000 7fca0cd54640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca080a15f0 0x7fca08133090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8019000 tx=0x7fc9f80046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc9f8003d50 con 0x7fca080a15f0 2024-08-19T10:53:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fc9fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fc9fdffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:53:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fc9fdffb640 10 monclient: dump: 2024-08-19T10:53:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.326 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7fc9f8003d50 con 0x7fca080a15f0 2024-08-19T10:53:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fc9fdffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fc9fdffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca080a15f0 msgr2=0x7fca08133090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fc9fdffb640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca080a15f0 0x7fca08133090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8019000 tx=0x7fc9f80046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fc9fdffb640 1 -- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca080a15f0 msgr2=0x7fca08133090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fc9fdffb640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca080a15f0 0x7fca08133090 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8019000 tx=0x7fc9f80046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.327 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:28.327 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:28.327 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:28.327 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 531447, 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:28.328 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.329 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.330 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:28.331 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.332 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:28.333 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.334 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:28.336 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.337 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.340 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:28.341 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.342 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.343 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.344 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.345 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:28.346 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.347 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.349 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.350 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:28.350 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:28.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0f7b9640 1 -- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc9ec03be30 msgr2=0x7fc9ec03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0f7b9640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc9ec03be30 0x7fc9ec03e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fca08074630 tx=0x7fca000a7070 comp rx=0 tx=0).stop 2024-08-19T10:53:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0f7b9640 10 monclient: shutdown 2024-08-19T10:53:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0f7b9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0f7b9640 1 -- 172.21.15.89:0/3015821730 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca080a56b0 msgr2=0x7fca0806da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0f7b9640 1 --2- 172.21.15.89:0/3015821730 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fca080a56b0 0x7fca0806da40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fca0400b4c0 tx=0x7fca0400b980 comp rx=0 tx=0).stop 2024-08-19T10:53:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0dd56640 1 -- 172.21.15.89:0/3015821730 reap_dead start 2024-08-19T10:53:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0dd56640 1 -- 172.21.15.89:0/3015821730 reap_dead start 2024-08-19T10:53:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0f7b9640 1 -- 172.21.15.89:0/3015821730 shutdown_connections 2024-08-19T10:53:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0f7b9640 1 -- 172.21.15.89:0/3015821730 >> 172.21.15.89:0/3015821730 conn(0x7fca080aa2a0 msgr2=0x7fca080a9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0f7b9640 1 -- 172.21.15.89:0/3015821730 shutdown_connections 2024-08-19T10:53:28.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.322+0000 7fca0f7b9640 1 -- 172.21.15.89:0/3015821730 wait complete. 2024-08-19T10:53:28.353 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:28.353 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:53:28.353 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:53:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient: get_monmap_and_config 2024-08-19T10:53:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient: build_initial_monmap 2024-08-19T10:53:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient: monmap: 2024-08-19T10:53:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:28.477000+0000 2024-08-19T10:53:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:28.477000+0000 2024-08-19T10:53:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:28.471 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 1 Processor -- start 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 1 -- start start 2024-08-19T10:53:28.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient: init 2024-08-19T10:53:28.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:28.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient: _add_conns ranks=[3,6,10,5,7,2,4,8,9,0,1] 2024-08-19T10:53:28.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd40a56b0 0x7f6bd40a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient(hunting): picked mon.noname-g con 0x7f6bd40a56b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bd40a1660 0x7f6bd40a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient(hunting): picked mon.noname-d con 0x7f6bd40a1660 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6bd4072250 0x7f6bd4137460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient(hunting): picked mon.noname-f con 0x7f6bd4072250 addr v2:172.21.15.190:3300/0 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd40a6510 con 0x7f6bd4072250 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd4074010 con 0x7f6bd40a56b0 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6bd4072010 con 0x7f6bd40a1660 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd8d4f640 10 monclient(hunting): authenticate will time out at 1858.293463s 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bd40a1660 0x7f6bd40a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bd40a1660 0x7f6bd40a5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:60880/0 (socket says 172.21.15.89:60880) 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 1 -- 172.21.15.89:0/1195339774 learned_addr learned my addr 172.21.15.89:0/1195339774 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 10 monclient(hunting): get_auth_request con 0x7f6bd40a1660 auth_method 0 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd259c640 1 --2- 172.21.15.89:0/1195339774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd40a56b0 0x7f6bd40a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd2d9d640 1 --2- 172.21.15.89:0/1195339774 >> v2:172.21.15.190:3300/0 conn(0x7f6bd4072250 0x7f6bd4137460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd2d9d640 10 monclient(hunting): get_auth_request con 0x7f6bd4072250 auth_method 0 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd2d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd259c640 10 monclient(hunting): get_auth_request con 0x7f6bd40a56b0 auth_method 0 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 10 monclient(hunting): handle_auth_done global_id 45123 payload 274 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 1 -- 172.21.15.89:0/1195339774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd40a56b0 msgr2=0x7f6bd40a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 1 --2- 172.21.15.89:0/1195339774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd40a56b0 0x7f6bd40a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 1 -- 172.21.15.89:0/1195339774 >> v2:172.21.15.190:3300/0 conn(0x7f6bd4072250 msgr2=0x7f6bd4137460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 1 --2- 172.21.15.89:0/1195339774 >> v2:172.21.15.190:3300/0 conn(0x7f6bd4072250 0x7f6bd4137460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 1 monclient: found mon.noname-d 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:28.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.474+0000 7f6bd1d9b640 1 -- 172.21.15.89:0/1195339774 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd4073d30 con 0x7f6bd40a1660 2024-08-19T10:53:28.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd1d9b640 1 -- 172.21.15.89:0/1195339774 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bd40a1660 msgr2=0x7f6bd40a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd1d9b640 1 --2- 172.21.15.89:0/1195339774 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bd40a1660 0x7f6bd40a5fd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009870 tx=0x7f6bc801c190 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ac7c2903e4e3a546 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 1 -- 172.21.15.89:0/1195339774 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6bc80b0070 con 0x7f6bd40a1660 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 10 monclient: dump: 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.482 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 10 monclient: _finish_auth 0 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 10 monclient: _check_auth_tickets 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 1 -- 172.21.15.89:0/1195339774 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc80aa810 con 0x7f6bd40a1660 2024-08-19T10:53:28.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 5 monclient: authenticate success, global_id 45123 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 1 -- 172.21.15.89:0/1195339774 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6bc80aae30 con 0x7f6bd40a1660 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 4 set_mon_vals no callback set 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd159a640 10 monclient: dump: 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.484 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 10 monclient: shutdown 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1195339774 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bd40a1660 msgr2=0x7f6bd40a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 1 --2- 172.21.15.89:0/1195339774 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bd40a1660 0x7f6bd40a5fd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009870 tx=0x7f6bc801c190 comp rx=0 tx=0).stop 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1195339774 shutdown_connections 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 1 --2- 172.21.15.89:0/1195339774 >> v2:172.21.15.190:3300/0 conn(0x7f6bd4072250 0x7f6bd4137460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 1 --2- 172.21.15.89:0/1195339774 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6bd40a1660 0x7f6bd40a5fd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 1 --2- 172.21.15.89:0/1195339774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd40a56b0 0x7f6bd40a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1195339774 >> 172.21.15.89:0/1195339774 conn(0x7f6bd40aa2a0 msgr2=0x7f6bd40a67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1195339774 shutdown_connections 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1195339774 wait complete. 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 10 monclient: build_initial_monmap 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.478+0000 7f6bd8d4f640 10 monclient: monmap: 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:28.486 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 1 Processor -- start 2024-08-19T10:53:28.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 1 -- start start 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 10 monclient: init 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 10 monclient: _add_conns ranks=[1,5,7,6,8,3,0,2,4] 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bd40a56b0 0x7f6bd41d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 10 monclient(hunting): picked mon.noname-f con 0x7f6bd40a56b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6bd413d430 0x7f6bd4136820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 10 monclient(hunting): picked mon.noname-h con 0x7f6bd413d430 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bd4136d60 0x7f6bd406da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 10 monclient(hunting): picked mon.noname-i con 0x7f6bd4136d60 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd4141260 con 0x7f6bd40a56b0 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6bd41410e0 con 0x7f6bd413d430 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f6bd41413e0 con 0x7f6bd4136d60 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd8d4f640 10 monclient(hunting): authenticate will time out at 1858.300653s 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bd40a56b0 0x7f6bd41d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd2d9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bd4136d60 0x7f6bd406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6bd413d430 0x7f6bd4136820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6bd413d430 0x7f6bd4136820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:36696/0 (socket says 172.21.15.89:36696) 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 1 -- 172.21.15.89:0/1715822773 learned_addr learned my addr 172.21.15.89:0/1715822773 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 10 monclient(hunting): get_auth_request con 0x7f6bd413d430 auth_method 0 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd259c640 10 monclient(hunting): get_auth_request con 0x7f6bd40a56b0 auth_method 0 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd2d9d640 10 monclient(hunting): get_auth_request con 0x7f6bd4136d60 auth_method 0 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd2d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 10 monclient(hunting): handle_auth_done global_id 84119 payload 824 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 1 -- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bd4136d60 msgr2=0x7f6bd406da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f6bd4136d60 0x7f6bd406da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 1 -- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bd40a56b0 msgr2=0x7f6bd41d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bd40a56b0 0x7f6bd41d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 1 monclient: found mon.noname-h 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:53:28.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.482+0000 7f6bd1d9b640 1 -- 172.21.15.89:0/1715822773 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd413d010 con 0x7f6bd413d430 2024-08-19T10:53:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd1d9b640 1 -- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6bd413d430 msgr2=0x7f6bd4136820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd1d9b640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6bd413d430 0x7f6bd4136820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80097c0 tx=0x7f6bc8018f90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6bc80ba070 con 0x7f6bd413d430 2024-08-19T10:53:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:53:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 10 monclient: dump: 2024-08-19T10:53:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.496 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc80b1040 con 0x7f6bd413d430 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd8d4f640 5 monclient: authenticate success, global_id 84119 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd8d4f640 10 monclient: _renew_subs 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd8d4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1715822773 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd406f460 con 0x7f6bd413d430 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6bc80b0070 con 0x7f6bd413d430 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd0d99640 4 set_mon_vals no callback set 2024-08-19T10:53:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 10 monclient: dump: 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd8d4f640 10 monclient: _renew_subs 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd8d4f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1715822773 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd413b810 con 0x7f6bd413d430 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd8d4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd8d4f640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd4072250 0x7f6bd406df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd8d4f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1715822773 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd40a9760 con 0x7f6bd4072250 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd259c640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd4072250 0x7f6bd406df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd259c640 10 monclient: get_auth_request con 0x7f6bd4072250 auth_method 0 2024-08-19T10:53:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd259c640 10 monclient(hunting): handle_auth_done global_id 35787 payload 824 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd259c640 1 -- 172.21.15.89:0/1715822773 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bd4064590 con 0x7f6bd4072250 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6bc8003620 con 0x7f6bd413d430 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6bb403bdf0 0x7f6bb403e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f6bc80e8b40 con 0x7f6bd413d430 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd2d9d640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6bb403bdf0 0x7f6bb403e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.486+0000 7f6bd2d9d640 10 monclient: get_auth_request con 0x7f6bb403bdf0 auth_method 0 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bd259c640 1 -- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd4072250 msgr2=0x7f6bd406df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bd259c640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd4072250 0x7f6bd406df80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4010820 tx=0x7f6bc4010ce0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6bc4018020 con 0x7f6bd4072250 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bc2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bc2ffd640 10 monclient: dump: 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6bc4004450 con 0x7f6bd4072250 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bc2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bc2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd4072250 msgr2=0x7f6bd406df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bc2ffd640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd4072250 0x7f6bd406df80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4010820 tx=0x7f6bc4010ce0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd4072250 msgr2=0x7f6bd406df80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bc2ffd640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd4072250 0x7f6bd406df80 secure :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4010820 tx=0x7f6bc4010ce0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.490+0000 7f6bd2d9d640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6bb403bdf0 0x7f6bb403e2b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc003ab0 tx=0x7f6bcc009820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.494+0000 7f6bd8d4f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.494+0000 7f6bd8d4f640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd413cae0 0x7f6bd406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.494+0000 7f6bd8d4f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.494+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1715822773 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd4137460 con 0x7f6bd413cae0 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd413cae0 0x7f6bd406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 10 monclient: get_auth_request con 0x7f6bd413cae0 auth_method 0 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 10 monclient(hunting): handle_auth_done global_id 35796 payload 824 2024-08-19T10:53:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 1 -- 172.21.15.89:0/1715822773 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6bd406fd40 con 0x7f6bd413cae0 2024-08-19T10:53:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 1 -- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd413cae0 msgr2=0x7f6bd406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bd259c640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd413cae0 0x7f6bd406e4c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4019000 tx=0x7f6bc40046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6bc4018040 con 0x7f6bd413cae0 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bc2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.498+0000 7f6bc2ffd640 10 monclient: dump: 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f6bc4003620 con 0x7f6bd413cae0 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bc2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bc2ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd413cae0 msgr2=0x7f6bd406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bc2ffd640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd413cae0 0x7f6bd406e4c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4019000 tx=0x7f6bc40046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bc2ffd640 1 -- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd413cae0 msgr2=0x7f6bd406e4c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bc2ffd640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6bd413cae0 0x7f6bd406e4c0 secure :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4019000 tx=0x7f6bc40046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.507 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:28.507 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:28.507 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:28.507 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:28.507 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:28.507 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 653545, 2024-08-19T10:53:28.507 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:28.507 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:28.508 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:28.509 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.510 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:28.511 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.512 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:28.513 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.514 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.515 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:28.516 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.518 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.519 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.520 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:28.521 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.522 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.523 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.524 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.525 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:28.526 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.527 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:28.528 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.529 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:28.530 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:28.530 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:53:28.530 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.530 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.530 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:28.530 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:28.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6bb403bdf0 msgr2=0x7f6bb403e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd8d4f640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6bb403bdf0 0x7f6bb403e2b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc003ab0 tx=0x7f6bcc009820 comp rx=0 tx=0).stop 2024-08-19T10:53:28.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd8d4f640 10 monclient: shutdown 2024-08-19T10:53:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd8d4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6bd413d430 msgr2=0x7f6bd4136820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd8d4f640 1 --2- 172.21.15.89:0/1715822773 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6bd413d430 0x7f6bd4136820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80097c0 tx=0x7f6bc8018f90 comp rx=0 tx=0).stop 2024-08-19T10:53:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd2d9d640 1 -- 172.21.15.89:0/1715822773 reap_dead start 2024-08-19T10:53:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd2d9d640 1 -- 172.21.15.89:0/1715822773 reap_dead start 2024-08-19T10:53:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1715822773 shutdown_connections 2024-08-19T10:53:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1715822773 >> 172.21.15.89:0/1715822773 conn(0x7f6bd40aa2a0 msgr2=0x7f6bd40a2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1715822773 shutdown_connections 2024-08-19T10:53:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.502+0000 7f6bd8d4f640 1 -- 172.21.15.89:0/1715822773 wait complete. 2024-08-19T10:53:28.532 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:28.533 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:53:28.533 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:53:28.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.650+0000 7f7b1665f640 10 monclient: get_monmap_and_config 2024-08-19T10:53:28.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.650+0000 7f7b1665f640 10 monclient: build_initial_monmap 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 10 monclient: monmap: 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:28.656930+0000 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:28.656930+0000 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:28.651 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:28.652 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:28.652 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:28.652 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:28.652 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:28.652 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:28.652 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 1 Processor -- start 2024-08-19T10:53:28.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 1 -- start start 2024-08-19T10:53:28.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 10 monclient: init 2024-08-19T10:53:28.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:28.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 10 monclient: _add_conns ranks=[0,9,5,2,8,1,4,6,3,10,7] 2024-08-19T10:53:28.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7b100a6860 addr v1:172.21.15.89:6789/0 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7b100a7350 0x7f7b1013e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 10 monclient(hunting): picked mon.noname-a con 0x7f7b100a7350 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b100a7730 0x7f7b10140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 10 monclient(hunting): picked mon.noname-h con 0x7f7b100a7730 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b100a33d0 con 0x7f7b100a6860 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f7b100a8190 con 0x7f7b100a7350 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7b100a2ec0 con 0x7f7b100a7730 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b1665f640 10 monclient(hunting): authenticate will time out at 1858.473507s 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b077fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7b100a7350 0x7f7b1013e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b077fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7b100a7350 0x7f7b1013e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39940/0 (socket says 172.21.15.89:39940) 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b077fe640 1 -- 172.21.15.89:0/1223589545 learned_addr learned my addr 172.21.15.89:0/1223589545 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b077fe640 10 monclient(hunting): get_auth_request con 0x7f7b100a7350 auth_method 0 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b077fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b077fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 1 --2- 172.21.15.89:0/1223589545 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b100a7730 0x7f7b10140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 10 monclient(hunting): get_auth_request con 0x7f7b100a7730 auth_method 0 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 10 monclient(hunting): handle_auth_done global_id 84128 payload 274 2024-08-19T10:53:28.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 1 -- 172.21.15.89:0/1223589545 >> v2:172.21.15.89:3300/0 conn(0x7f7b100a7350 msgr2=0x7f7b1013e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:28.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 1 --2- 172.21.15.89:0/1223589545 >> v2:172.21.15.89:3300/0 conn(0x7f7b100a7350 0x7f7b1013e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 1 -- 172.21.15.89:0/1223589545 >> v1:172.21.15.89:6789/0 conn(0x7f7b100a6860 legacy=0x7f7b100a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:28.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 10 monclient: _finish_hunting 0 2024-08-19T10:53:28.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 1 monclient: found mon.noname-h 2024-08-19T10:53:28.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:28.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:53:28.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.654+0000 7f7b14bfc640 1 -- 172.21.15.89:0/1223589545 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b100a4cd0 con 0x7f7b100a7730 2024-08-19T10:53:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b14bfc640 1 -- 172.21.15.89:0/1223589545 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b100a7730 msgr2=0x7f7b10140db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b14bfc640 1 --2- 172.21.15.89:0/1223589545 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b100a7730 0x7f7b10140db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009d90 tx=0x7f7b0801d290 comp rx=0 tx=0).ready entity=mon.5 client_cookie=75460614bb5c419c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 1 -- 172.21.15.89:0/1223589545 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7b080b2070 con 0x7f7b100a7730 2024-08-19T10:53:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:53:28.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 10 monclient: dump: 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 1 -- 172.21.15.89:0/1223589545 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b080a59f0 con 0x7f7b100a7730 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 1 -- 172.21.15.89:0/1223589545 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7b080a5be0 con 0x7f7b100a7730 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b1665f640 5 monclient: authenticate success, global_id 84128 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b1665f640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b1665f640 4 set_mon_vals no callback set 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.658+0000 7f7b0f7fe640 10 monclient: dump: 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.662 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient: shutdown 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 -- 172.21.15.89:0/1223589545 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b100a7730 msgr2=0x7f7b10140db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 --2- 172.21.15.89:0/1223589545 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b100a7730 0x7f7b10140db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009d90 tx=0x7f7b0801d290 comp rx=0 tx=0).stop 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 -- 172.21.15.89:0/1223589545 shutdown_connections 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 --2- 172.21.15.89:0/1223589545 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b100a7730 0x7f7b10140db0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 --2- 172.21.15.89:0/1223589545 >> v2:172.21.15.89:3300/0 conn(0x7f7b100a7350 0x7f7b1013e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 -- 172.21.15.89:0/1223589545 >> 172.21.15.89:0/1223589545 conn(0x7f7b10074b40 msgr2=0x7f7b100abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 -- 172.21.15.89:0/1223589545 shutdown_connections 2024-08-19T10:53:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 -- 172.21.15.89:0/1223589545 wait complete. 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient: build_initial_monmap 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient: monmap: 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:28.665 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:28.665 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:28.665 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 Processor -- start 2024-08-19T10:53:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 -- start start 2024-08-19T10:53:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient: init 2024-08-19T10:53:28.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient: _add_conns ranks=[6,5,1,7,0,3,4,8,2] 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b100a6860 0x7f7b101288e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient(hunting): picked mon.noname-d con 0x7f7b100a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b100a7350 0x7f7b1013f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient(hunting): picked mon.noname-h con 0x7f7b100a7350 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b100a7730 0x7f7b10126290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient(hunting): picked mon.noname-f con 0x7f7b100a7730 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10143ea0 con 0x7f7b100a7730 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7b10143d20 con 0x7f7b100a7350 2024-08-19T10:53:28.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7b10144020 con 0x7f7b100a6860 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b1665f640 10 monclient(hunting): authenticate will time out at 1858.482258s 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b100a6860 0x7f7b101288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b100a6860 0x7f7b101288e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:60886/0 (socket says 172.21.15.89:60886) 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 1 -- 172.21.15.89:0/3401079830 learned_addr learned my addr 172.21.15.89:0/3401079830 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 10 monclient(hunting): get_auth_request con 0x7f7b100a6860 auth_method 0 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b077fe640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b100a7350 0x7f7b1013f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b14bfc640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b100a7730 0x7f7b10126290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b077fe640 10 monclient(hunting): get_auth_request con 0x7f7b100a7350 auth_method 0 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b077fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b077fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 10 monclient(hunting): handle_auth_done global_id 45132 payload 824 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b100a7350 msgr2=0x7f7b1013f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b100a7350 0x7f7b1013f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b100a7730 msgr2=0x7f7b10126290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b100a7730 0x7f7b10126290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 1 monclient: found mon.noname-d 2024-08-19T10:53:28.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b0ffff640 1 -- 172.21.15.89:0/3401079830 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1012a0e0 con 0x7f7b100a6860 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b14bfc640 10 monclient: get_auth_request con 0x7f7b100a7730 auth_method 0 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.662+0000 7f7b14bfc640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b100a7730 0x7f7b10126290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0ffff640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b100a6860 msgr2=0x7f7b101288e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0ffff640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b100a6860 0x7f7b101288e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000c450 tx=0x7f7b0000c910 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7b00012cb0 con 0x7f7b100a6860 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 10 monclient: dump: 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.673 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:53:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b00017070 con 0x7f7b100a6860 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b1665f640 5 monclient: authenticate success, global_id 45132 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b1665f640 10 monclient: _renew_subs 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b1665f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b1665f640 1 -- 172.21.15.89:0/3401079830 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b10140a30 con 0x7f7b100a6860 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7b000117f0 con 0x7f7b100a6860 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 10 monclient: dump: 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.676 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b1665f640 10 monclient: _renew_subs 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b1665f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0effd640 4 set_mon_vals no callback set 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b1665f640 1 -- 172.21.15.89:0/3401079830 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b10129270 con 0x7f7b100a6860 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b1665f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b1665f640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 0x7f7b10126bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b1665f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b1665f640 1 -- 172.21.15.89:0/3401079830 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7b10129a80 con 0x7f7b101267d0 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7b00011380 con 0x7f7b100a6860 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7af003be00 0x7f7af003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 0x7f7b10126bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 10 monclient: get_auth_request con 0x7f7b101267d0 auth_method 0 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f7b0004f360 con 0x7f7b100a6860 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b14bfc640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7af003be00 0x7f7af003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b14bfc640 10 monclient: get_auth_request con 0x7f7af003be00 auth_method 0 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 10 monclient(hunting): handle_auth_done global_id 84146 payload 824 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 1 -- 172.21.15.89:0/3401079830 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b101270f0 con 0x7f7b101267d0 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 msgr2=0x7f7b10126bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 msgr2=0x7f7b10126bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 0x7f7b10126bb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f7b101283f0 tx=0x7f7af8011a60 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.666+0000 7f7b077fe640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 0x7f7b10126bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.670+0000 7f7b14bfc640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7af003be00 0x7f7af003e2c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7b1006cff0 tx=0x7f7b08004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.870+0000 7f7b077fe640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 0x7f7b10126bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.870+0000 7f7b077fe640 10 monclient: get_auth_request con 0x7f7b101267d0 auth_method 0 2024-08-19T10:53:28.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.870+0000 7f7b077fe640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.870+0000 7f7b077fe640 10 monclient: _init_auth method 2 2024-08-19T10:53:28.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.870+0000 7f7b077fe640 10 monclient: _init_auth already have auth, reseting 2024-08-19T10:53:28.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.870+0000 7f7b077fe640 10 monclient: handle_auth_reply_more payload 9 2024-08-19T10:53:28.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.870+0000 7f7b077fe640 10 monclient: handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.870+0000 7f7b077fe640 10 monclient: handle_auth_reply_more responding with 132 bytes 2024-08-19T10:53:28.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.870+0000 7f7b077fe640 10 monclient: handle_auth_done global_id 84146 payload 293 2024-08-19T10:53:28.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b077fe640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 msgr2=0x7f7b10126bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b077fe640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 0x7f7b10126bb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7af8006cd0 tx=0x7f7af8008040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7af8023030 con 0x7f7b101267d0 2024-08-19T10:53:28.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b0d7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b0d7fa640 10 monclient: dump: 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.872 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.873 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7af800fe40 con 0x7f7b101267d0 2024-08-19T10:53:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b0d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b0d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:28.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 msgr2=0x7f7b10126bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b0d7fa640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 0x7f7b10126bb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7af8006cd0 tx=0x7f7af8008040 comp rx=0 tx=0).stop 2024-08-19T10:53:28.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 msgr2=0x7f7b10126bb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:28.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.874+0000 7f7b0d7fa640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101267d0 0x7f7b10126bb0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7af8006cd0 tx=0x7f7af8008040 comp rx=0 tx=0).stop 2024-08-19T10:53:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b1665f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b1665f640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101224b0 0x7f7b101228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b1665f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b1665f640 1 -- 172.21.15.89:0/3401079830 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7b100a6c40 con 0x7f7b101224b0 2024-08-19T10:53:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b077fe640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101224b0 0x7f7b101228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b077fe640 10 monclient: get_auth_request con 0x7f7b101224b0 auth_method 0 2024-08-19T10:53:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b077fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b077fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b077fe640 10 monclient(hunting): handle_auth_done global_id 84155 payload 824 2024-08-19T10:53:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.882+0000 7f7b077fe640 1 -- 172.21.15.89:0/3401079830 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7b10064590 con 0x7f7b101224b0 2024-08-19T10:53:28.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b077fe640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101224b0 msgr2=0x7f7b101228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:28.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b077fe640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101224b0 0x7f7b101228c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7af8023000 tx=0x7f7af8004740 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7af800f610 con 0x7f7b101224b0 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b0d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b0d7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b0d7fa640 10 monclient: dump: 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f7af800f570 con 0x7f7b101224b0 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b0d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b0d7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101224b0 msgr2=0x7f7b101228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b0d7fa640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101224b0 0x7f7b101228c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7af8023000 tx=0x7f7af8004740 comp rx=0 tx=0).stop 2024-08-19T10:53:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b0d7fa640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101224b0 msgr2=0x7f7b101228c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b0d7fa640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7b101224b0 0x7f7b101228c0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7af8023000 tx=0x7f7af8004740 comp rx=0 tx=0).stop 2024-08-19T10:53:28.887 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:28.887 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:28.887 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:28.887 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:28.887 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 17179, 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:53:28.888 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.889 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.890 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:28.891 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:28.892 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.895 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.896 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.897 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:28.898 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.899 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.900 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:28.901 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.902 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.903 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.904 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.905 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:28.906 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:28.908 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:28.909 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:28.910 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:28.910 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:28.910 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:28.910 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:28.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:28.910 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:28.910 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:28.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b1665f640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7af003be00 msgr2=0x7f7af003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b1665f640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7af003be00 0x7f7af003e2c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7b1006cff0 tx=0x7f7b08004ac0 comp rx=0 tx=0).stop 2024-08-19T10:53:28.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b1665f640 10 monclient: shutdown 2024-08-19T10:53:28.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b1665f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:28.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b1665f640 1 -- 172.21.15.89:0/3401079830 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b100a6860 msgr2=0x7f7b101288e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:28.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b1665f640 1 --2- 172.21.15.89:0/3401079830 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7b100a6860 0x7f7b101288e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000c450 tx=0x7f7b0000c910 comp rx=0 tx=0).stop 2024-08-19T10:53:28.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b14bfc640 1 -- 172.21.15.89:0/3401079830 reap_dead start 2024-08-19T10:53:28.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b14bfc640 1 -- 172.21.15.89:0/3401079830 reap_dead start 2024-08-19T10:53:28.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b1665f640 1 -- 172.21.15.89:0/3401079830 shutdown_connections 2024-08-19T10:53:28.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b1665f640 1 -- 172.21.15.89:0/3401079830 >> 172.21.15.89:0/3401079830 conn(0x7f7b10074b40 msgr2=0x7f7b1009c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:28.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b1665f640 1 -- 172.21.15.89:0/3401079830 shutdown_connections 2024-08-19T10:53:28.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:28.886+0000 7f7b1665f640 1 -- 172.21.15.89:0/3401079830 wait complete. 2024-08-19T10:53:28.913 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:28.914 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:53:28.914 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:53:29.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient: get_monmap_and_config 2024-08-19T10:53:29.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient: build_initial_monmap 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient: monmap: 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:29.049846+0000 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:29.049846+0000 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:29.045 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 1 Processor -- start 2024-08-19T10:53:29.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 1 -- start start 2024-08-19T10:53:29.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient: init 2024-08-19T10:53:29.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient: _add_conns ranks=[3,9,10,4,0,6,2,7,8,5,1] 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8c00a5270 0x7ff8c00a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient(hunting): picked mon.noname-g con 0x7ff8c00a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8c00a6d30 0x7ff8c00a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient(hunting): picked mon.noname-a con 0x7ff8c00a6d30 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff8c01395b0 0x7ff8c0139a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient(hunting): picked mon.noname-f con 0x7ff8c01395b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff8c00a7650 con 0x7ff8c00a6d30 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff8c00a8710 con 0x7ff8c01395b0 2024-08-19T10:53:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c00a5a40 con 0x7ff8c00a5270 2024-08-19T10:53:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8c54ab640 10 monclient(hunting): authenticate will time out at 1858.866598s 2024-08-19T10:53:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8c00a6d30 0x7ff8c00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff8c00a6d30 0x7ff8c00a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39948/0 (socket says 172.21.15.89:39948) 2024-08-19T10:53:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 1 -- 172.21.15.89:0/3235512634 learned_addr learned my addr 172.21.15.89:0/3235512634 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 10 monclient(hunting): get_auth_request con 0x7ff8c00a6d30 auth_method 0 2024-08-19T10:53:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 10 monclient(hunting): handle_auth_done global_id 7335 payload 274 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8beffd640 1 --2- 172.21.15.89:0/3235512634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8c00a5270 0x7ff8c00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8bf7fe640 1 --2- 172.21.15.89:0/3235512634 >> v2:172.21.15.190:3300/0 conn(0x7ff8c01395b0 0x7ff8c0139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 1 -- 172.21.15.89:0/3235512634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8c00a5270 msgr2=0x7ff8c00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 1 --2- 172.21.15.89:0/3235512634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8c00a5270 0x7ff8c00a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 1 -- 172.21.15.89:0/3235512634 >> v2:172.21.15.190:3300/0 conn(0x7ff8c01395b0 msgr2=0x7ff8c0139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 1 --2- 172.21.15.89:0/3235512634 >> v2:172.21.15.190:3300/0 conn(0x7ff8c01395b0 0x7ff8c0139a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 1 monclient: found mon.noname-a 2024-08-19T10:53:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:29.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:29.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8be7fc640 1 -- 172.21.15.89:0/3235512634 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c00a81d0 con 0x7ff8c00a6d30 2024-08-19T10:53:29.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8bf7fe640 10 monclient: get_auth_request con 0x7ff8c01395b0 auth_method 0 2024-08-19T10:53:29.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.046+0000 7ff8bf7fe640 1 --2- 172.21.15.89:0/3235512634 >> v2:172.21.15.190:3300/0 conn(0x7ff8c01395b0 0x7ff8c0139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:53:29.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8be7fc640 1 -- 172.21.15.89:0/3235512634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c00a6d30 msgr2=0x7ff8c00a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:29.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8be7fc640 1 --2- 172.21.15.89:0/3235512634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c00a6d30 0x7ff8c00a7110 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00a9960 tx=0x7ff8b001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a85434b9c032b6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:29.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 1 -- 172.21.15.89:0/3235512634 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b00b1070 con 0x7ff8c00a6d30 2024-08-19T10:53:29.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:29.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:29.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 10 monclient: dump: 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:29.055 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:29.056 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 10 monclient: _finish_auth 0 2024-08-19T10:53:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:53:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 1 -- 172.21.15.89:0/3235512634 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b00ab810 con 0x7ff8c00a6d30 2024-08-19T10:53:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8c54ab640 5 monclient: authenticate success, global_id 7335 2024-08-19T10:53:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8c54ab640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 1 -- 172.21.15.89:0/3235512634 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b00aa3e0 con 0x7ff8c00a6d30 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8c54ab640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8c54ab640 4 set_mon_vals no callback set 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.050+0000 7ff8bdffb640 10 monclient: dump: 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:29.057 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient: shutdown 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 -- 172.21.15.89:0/3235512634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c00a6d30 msgr2=0x7ff8c00a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 --2- 172.21.15.89:0/3235512634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c00a6d30 0x7ff8c00a7110 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00a9960 tx=0x7ff8b001c3e0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 -- 172.21.15.89:0/3235512634 shutdown_connections 2024-08-19T10:53:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 --2- 172.21.15.89:0/3235512634 >> v2:172.21.15.190:3300/0 conn(0x7ff8c01395b0 0x7ff8c0139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 --2- 172.21.15.89:0/3235512634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c00a6d30 0x7ff8c00a7110 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 --2- 172.21.15.89:0/3235512634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8c00a5270 0x7ff8c00a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 -- 172.21.15.89:0/3235512634 >> 172.21.15.89:0/3235512634 conn(0x7ff8c0069a80 msgr2=0x7ff8c00abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 -- 172.21.15.89:0/3235512634 shutdown_connections 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 -- 172.21.15.89:0/3235512634 wait complete. 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient: build_initial_monmap 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient: monmap: 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 Processor -- start 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 -- start start 2024-08-19T10:53:29.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient: init 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient: _add_conns ranks=[5,2,7,3,8,4,1,0,6] 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff8c00a5270 0x7ff8c0133550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient(hunting): picked mon.noname-h con 0x7ff8c00a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8c00a6d30 0x7ff8c0133a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient(hunting): picked mon.noname-b con 0x7ff8c00a6d30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c01395b0 0x7ff8c0133fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient(hunting): picked mon.noname-i con 0x7ff8c01395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c00a5650 con 0x7ff8c00a6d30 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff8c013a360 con 0x7ff8c00a5270 2024-08-19T10:53:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff8c00a57d0 con 0x7ff8c01395b0 2024-08-19T10:53:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8c54ab640 10 monclient(hunting): authenticate will time out at 1858.874055s 2024-08-19T10:53:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8c00a6d30 0x7ff8c0133a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8c00a6d30 0x7ff8c0133a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:34388/0 (socket says 172.21.15.89:34388) 2024-08-19T10:53:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 1 -- 172.21.15.89:0/2126939454 learned_addr learned my addr 172.21.15.89:0/2126939454 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 10 monclient(hunting): get_auth_request con 0x7ff8c00a6d30 auth_method 0 2024-08-19T10:53:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8beffd640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff8c00a5270 0x7ff8c0133550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 10 monclient(hunting): handle_auth_done global_id 74306 payload 824 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8bf7fe640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c01395b0 0x7ff8c0133fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 1 -- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c01395b0 msgr2=0x7ff8c0133fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c01395b0 0x7ff8c0133fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 1 -- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff8c00a5270 msgr2=0x7ff8c0133550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff8c00a5270 0x7ff8c0133550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:53:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 1 monclient: found mon.noname-b 2024-08-19T10:53:29.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:29.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:53:29.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8be7fc640 1 -- 172.21.15.89:0/2126939454 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c0139c10 con 0x7ff8c00a6d30 2024-08-19T10:53:29.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8beffd640 10 monclient: get_auth_request con 0x7ff8c00a5270 auth_method 0 2024-08-19T10:53:29.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.054+0000 7ff8beffd640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff8c00a5270 0x7ff8c0133550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:53:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8be7fc640 1 -- 172.21.15.89:0/2126939454 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8c00a6d30 msgr2=0x7ff8c0133a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8be7fc640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8c00a6d30 0x7ff8c0133a90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff8b001cc80 tx=0x7ff8b0019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b00b1040 con 0x7ff8c00a6d30 2024-08-19T10:53:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:53:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 10 monclient: dump: 2024-08-19T10:53:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:29.069 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 10 monclient: _finish_auth 0 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b00ba070 con 0x7ff8c00a6d30 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8c54ab640 5 monclient: authenticate success, global_id 74306 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8c54ab640 10 monclient: _renew_subs 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8c54ab640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8c54ab640 1 -- 172.21.15.89:0/2126939454 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c00a0110 con 0x7ff8c00a6d30 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b0004250 con 0x7ff8c00a6d30 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 10 monclient: dump: 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:29.072 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8bcff9640 4 set_mon_vals no callback set 2024-08-19T10:53:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8c54ab640 10 monclient: _renew_subs 2024-08-19T10:53:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8c54ab640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:53:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8c54ab640 1 -- 172.21.15.89:0/2126939454 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c009f400 con 0x7ff8c00a6d30 2024-08-19T10:53:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8c54ab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8c54ab640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0137a20 0x7ff8c0137e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8c54ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8c54ab640 1 -- 172.21.15.89:0/2126939454 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff8c00a6a20 con 0x7ff8c0137a20 2024-08-19T10:53:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff8b0003620 con 0x7ff8c00a6d30 2024-08-19T10:53:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff8ac03be00 0x7ff8ac03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7ff8b00e9750 con 0x7ff8c00a6d30 2024-08-19T10:53:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8beffd640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0137a20 0x7ff8c0137e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8beffd640 10 monclient: get_auth_request con 0x7ff8c0137a20 auth_method 0 2024-08-19T10:53:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8beffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8beffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8bf7fe640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff8ac03be00 0x7ff8ac03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8bf7fe640 10 monclient: get_auth_request con 0x7ff8ac03be00 auth_method 0 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8beffd640 10 monclient(hunting): handle_auth_done global_id 64465 payload 824 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.058+0000 7ff8beffd640 1 -- 172.21.15.89:0/2126939454 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8c00a7110 con 0x7ff8c0137a20 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8beffd640 1 -- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0137a20 msgr2=0x7ff8c0137e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8beffd640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0137a20 0x7ff8c0137e00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400d560 tx=0x7ff8b400da20 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8bf7fe640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff8ac03be00 0x7ff8ac03e2c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8007130 tx=0x7ff8b80083c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b4016070 con 0x7ff8c0137a20 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8a77fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8a77fe640 10 monclient: dump: 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff8b4012960 con 0x7ff8c0137a20 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8a77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8a77fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0137a20 msgr2=0x7ff8c0137e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8a77fe640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0137a20 0x7ff8c0137e00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400d560 tx=0x7ff8b400da20 comp rx=0 tx=0).stop 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0137a20 msgr2=0x7ff8c0137e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.062+0000 7ff8a77fe640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0137a20 0x7ff8c0137e00 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400d560 tx=0x7ff8b400da20 comp rx=0 tx=0).stop 2024-08-19T10:53:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8c54ab640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8c54ab640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0138340 0x7ff8c009c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8c54ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8c54ab640 1 -- 172.21.15.89:0/2126939454 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff8c0064590 con 0x7ff8c0138340 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8beffd640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0138340 0x7ff8c009c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8beffd640 10 monclient: get_auth_request con 0x7ff8c0138340 auth_method 0 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8beffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8beffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8beffd640 10 monclient(hunting): handle_auth_done global_id 64474 payload 824 2024-08-19T10:53:29.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.070+0000 7ff8beffd640 1 -- 172.21.15.89:0/2126939454 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff8c006c9e0 con 0x7ff8c0138340 2024-08-19T10:53:29.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8beffd640 1 -- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0138340 msgr2=0x7ff8c009c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:29.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8beffd640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0138340 0x7ff8c009c9e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400d560 tx=0x7ff8b4004740 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8b4011830 con 0x7ff8c0138340 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8a77fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8a77fe640 10 monclient: dump: 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7ff8b40113c0 con 0x7ff8c0138340 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8a77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8a77fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0138340 msgr2=0x7ff8c009c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8a77fe640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0138340 0x7ff8c009c9e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400d560 tx=0x7ff8b4004740 comp rx=0 tx=0).stop 2024-08-19T10:53:29.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8a77fe640 1 -- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0138340 msgr2=0x7ff8c009c9e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:29.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8a77fe640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8c0138340 0x7ff8c009c9e0 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400d560 tx=0x7ff8b4004740 comp rx=0 tx=0).stop 2024-08-19T10:53:29.081 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:29.081 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:29.081 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:29.081 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:29.081 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:29.081 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 275473, 2024-08-19T10:53:29.081 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:29.081 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:29.082 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:29.083 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:29.084 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:29.085 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:29.086 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:29.087 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.088 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:29.089 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.090 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:29.091 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:29.092 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:29.093 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:29.094 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:29.096 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.097 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:29.098 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:29.099 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:29.100 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:29.101 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.102 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:29.104 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:29.105 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:29.105 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:29.105 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:29.105 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:29.105 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:29.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8c54ab640 1 -- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff8ac03be00 msgr2=0x7ff8ac03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8c54ab640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff8ac03be00 0x7ff8ac03e2c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8007130 tx=0x7ff8b80083c0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8c54ab640 10 monclient: shutdown 2024-08-19T10:53:29.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8c54ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:29.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8c54ab640 1 -- 172.21.15.89:0/2126939454 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8c00a6d30 msgr2=0x7ff8c0133a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8c54ab640 1 --2- 172.21.15.89:0/2126939454 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff8c00a6d30 0x7ff8c0133a90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff8b001cc80 tx=0x7ff8b0019180 comp rx=0 tx=0).stop 2024-08-19T10:53:29.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8bf7fe640 1 -- 172.21.15.89:0/2126939454 reap_dead start 2024-08-19T10:53:29.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8bf7fe640 1 -- 172.21.15.89:0/2126939454 reap_dead start 2024-08-19T10:53:29.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8c54ab640 1 -- 172.21.15.89:0/2126939454 shutdown_connections 2024-08-19T10:53:29.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8c54ab640 1 -- 172.21.15.89:0/2126939454 >> 172.21.15.89:0/2126939454 conn(0x7ff8c0069a80 msgr2=0x7ff8c00ab130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:29.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8c54ab640 1 -- 172.21.15.89:0/2126939454 shutdown_connections 2024-08-19T10:53:29.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.074+0000 7ff8c54ab640 1 -- 172.21.15.89:0/2126939454 wait complete. 2024-08-19T10:53:29.108 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:29.108 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-08-19T10:53:29.108 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-08-19T10:53:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.218+0000 7f7df740f640 10 monclient: get_monmap_and_config 2024-08-19T10:53:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.218+0000 7f7df740f640 10 monclient: build_initial_monmap 2024-08-19T10:53:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient: monmap: 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:29.224951+0000 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:29.224951+0000 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:29.219 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:29.220 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:29.220 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:29.220 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:29.220 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 1 Processor -- start 2024-08-19T10:53:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 1 -- start start 2024-08-19T10:53:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient: init 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient: _add_conns ranks=[10,8,2,4,0,3,9,6,1,5,7] 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7df01398b0 0x7f7df0139c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient(hunting): picked mon.noname-f con 0x7f7df01398b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 0x7f7df00a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient(hunting): picked mon.noname-e con 0x7f7df006e090 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df006e4f0 0x7f7df00a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient(hunting): picked mon.noname-b con 0x7f7df006e4f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7df006f4e0 con 0x7f7df01398b0 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7df006de80 con 0x7f7df006e4f0 2024-08-19T10:53:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7df006ef20 con 0x7f7df006e090 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df740f640 10 monclient(hunting): authenticate will time out at 1859.041259s 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 0x7f7df00a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 0x7f7df00a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:55356/0 (socket says 172.21.15.89:55356) 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df59ac640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df006e4f0 0x7f7df00a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 -- 172.21.15.89:0/708886922 learned_addr learned my addr 172.21.15.89:0/708886922 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 10 monclient(hunting): get_auth_request con 0x7f7df006e090 auth_method 0 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df59ac640 10 monclient(hunting): get_auth_request con 0x7f7df006e4f0 auth_method 0 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df59ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df59ac640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df59ac640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df59ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df59ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df59ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df51ab640 1 --2- 172.21.15.89:0/708886922 >> v2:172.21.15.190:3300/0 conn(0x7f7df01398b0 0x7f7df0139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df51ab640 10 monclient(hunting): get_auth_request con 0x7f7df01398b0 auth_method 0 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df51ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df51ab640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df51ab640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:29.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 10 monclient(hunting): handle_auth_done global_id 84140 payload 274 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 -- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df006e4f0 msgr2=0x7f7df00a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df006e4f0 0x7f7df00a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 -- 172.21.15.89:0/708886922 >> v2:172.21.15.190:3300/0 conn(0x7f7df01398b0 msgr2=0x7f7df0139c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 --2- 172.21.15.89:0/708886922 >> v2:172.21.15.190:3300/0 conn(0x7f7df01398b0 0x7f7df0139c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 10 monclient: _finish_hunting 0 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 monclient: found mon.noname-e 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 -- 172.21.15.89:0/708886922 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df006f200 con 0x7f7df006e090 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df59ac640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7df006e4f0 0x7f7df00a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 -- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 msgr2=0x7f7df00a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 -- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 msgr2=0x7f7df00a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 0x7f7df00a4a60 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f7dec0098a0 tx=0x7f7dec01c400 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:29.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.222+0000 7f7df49aa640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 0x7f7df00a4a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:29.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 0x7f7df00a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 10 monclient: get_auth_request con 0x7f7df006e090 auth_method 0 2024-08-19T10:53:29.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 0 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 0x7f7df00a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-08-19T10:53:29.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 0x7f7df00a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 10 monclient: ms_handle_reset current mon [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:29.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:29.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 1 -- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 msgr2=0x7f7df00a4a60 unknown :-1 s=STATE_CLOSED l=0).mark_down 2024-08-19T10:53:29.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df006e090 0x7f7df00a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 10 monclient: _add_conns ranks=[9,7,3,5,1,6,4,0] 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 1 --2- 172.21.15.89:0/708886922 >> v2:172.21.15.89:3300/0 conn(0x7f7ddc003640 0x7f7ddc005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 10 monclient(hunting): picked mon.noname-a con 0x7f7ddc003640 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ddc0062b0 0x7f7ddc008740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 10 monclient(hunting): picked mon.noname-i con 0x7f7ddc0062b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ddc009110 0x7f7ddc00b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 10 monclient(hunting): picked mon.noname-g con 0x7f7ddc009110 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 1 -- 172.21.15.89:0/708886922 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f7ddc00bf50 con 0x7f7ddc003640 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 1 -- 172.21.15.89:0/708886922 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7ddc00c0d0 con 0x7f7ddc009110 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 1 -- 172.21.15.89:0/708886922 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7ddc00c2d0 con 0x7f7ddc0062b0 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7de7fff640 10 monclient(hunting): ms_handle_reset stray mon [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 1 --2- 172.21.15.89:0/708886922 >> v2:172.21.15.89:3300/0 conn(0x7f7ddc003640 0x7f7ddc005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 10 monclient(hunting): get_auth_request con 0x7f7ddc003640 auth_method 0 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ddc0062b0 0x7f7ddc008740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df59ac640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ddc009110 0x7f7ddc00b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 10 monclient(hunting): get_auth_request con 0x7f7ddc0062b0 auth_method 0 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df59ac640 10 monclient(hunting): get_auth_request con 0x7f7ddc009110 auth_method 0 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df59ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df59ac640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df59ac640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df49aa640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df59ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df59ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df59ac640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 10 monclient(hunting): handle_auth_done global_id 84140 payload 293 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 1 -- 172.21.15.89:0/708886922 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ddc0062b0 msgr2=0x7f7ddc008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7ddc0062b0 0x7f7ddc008740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 1 -- 172.21.15.89:0/708886922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ddc009110 msgr2=0x7f7ddc00b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ddc009110 0x7f7ddc00b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 10 monclient: _finish_hunting 0 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 1 monclient: found mon.noname-a 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df59ac640 1 -- 172.21.15.89:0/708886922 reap_dead start 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.426+0000 7f7df51ab640 1 -- 172.21.15.89:0/708886922 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc00c590 con 0x7f7ddc003640 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df51ab640 1 -- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 msgr2=0x7f7ddc005a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df51ab640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 0x7f7ddc005a90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7de000ba00 tx=0x7f7dec009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434ebf15408c46b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 1 -- 172.21.15.89:0/708886922 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7de000b600 con 0x7f7ddc003640 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 10 monclient: dump: 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:29.433 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 10 monclient: _finish_auth 0 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 1 -- 172.21.15.89:0/708886922 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de0019070 con 0x7f7ddc003640 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 5 monclient: authenticate success, global_id 84140 2024-08-19T10:53:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 4 set_mon_vals no callback set 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 1 -- 172.21.15.89:0/708886922 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7de000de60 con 0x7f7ddc003640 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7de7fff640 10 monclient: dump: 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:29.436 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 10 monclient: shutdown 2024-08-19T10:53:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 1 -- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 msgr2=0x7f7ddc005a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 0x7f7ddc005a90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7de000ba00 tx=0x7f7dec009240 comp rx=0 tx=0).stop 2024-08-19T10:53:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 1 -- 172.21.15.89:0/708886922 shutdown_connections 2024-08-19T10:53:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 1 --2- 172.21.15.89:0/708886922 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 0x7f7ddc005a90 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 1 -- 172.21.15.89:0/708886922 >> 172.21.15.89:0/708886922 conn(0x7f7df00aa1b0 msgr2=0x7f7df00aa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 1 -- 172.21.15.89:0/708886922 shutdown_connections 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.430+0000 7f7df740f640 1 -- 172.21.15.89:0/708886922 wait complete. 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient: build_initial_monmap 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient: monmap: 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 1 Processor -- start 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 1 -- start start 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient: init 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient: _add_conns ranks=[0,8,5,3,1,7,6,2,4] 2024-08-19T10:53:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 0x7f7df0120b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient(hunting): picked mon.noname-a con 0x7f7ddc003640 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df0121080 0x7f7df01214c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient(hunting): picked mon.noname-e con 0x7f7df0121080 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7df01d5dd0 0x7f7df01d81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient(hunting): picked mon.noname-h con 0x7f7df01d5dd0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7df013a020 con 0x7f7ddc003640 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7df0139ea0 con 0x7f7df01d5dd0 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f7df013a3e0 con 0x7f7df0121080 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df740f640 10 monclient(hunting): authenticate will time out at 1859.254065s 2024-08-19T10:53:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 0x7f7df0120b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 0x7f7df0120b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39960/0 (socket says 172.21.15.89:39960) 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 1 -- 172.21.15.89:0/2737582524 learned_addr learned my addr 172.21.15.89:0/2737582524 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 10 monclient(hunting): get_auth_request con 0x7f7ddc003640 auth_method 0 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df49aa640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df0121080 0x7f7df01214c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df49aa640 10 monclient(hunting): get_auth_request con 0x7f7df0121080 auth_method 0 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df49aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df49aa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df49aa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df59ac640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7df01d5dd0 0x7f7df01d81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df49aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df49aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df49aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df59ac640 10 monclient(hunting): get_auth_request con 0x7f7df01d5dd0 auth_method 0 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df59ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df59ac640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df59ac640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 10 monclient(hunting): handle_auth_done global_id 7344 payload 824 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 1 -- 172.21.15.89:0/2737582524 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df0121080 msgr2=0x7f7df01214c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df0121080 0x7f7df01214c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 1 -- 172.21.15.89:0/2737582524 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7df01d5dd0 msgr2=0x7f7df01d81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7df01d5dd0 0x7f7df01d81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 10 monclient: _finish_hunting 0 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 1 monclient: found mon.noname-a 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.434+0000 7f7df51ab640 1 -- 172.21.15.89:0/2737582524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df0124960 con 0x7f7ddc003640 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df51ab640 1 -- 172.21.15.89:0/2737582524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 msgr2=0x7f7df0120b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df51ab640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 0x7f7df0120b40 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7de000b950 tx=0x7f7de0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 1 -- 172.21.15.89:0/2737582524 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7de0004460 con 0x7f7ddc003640 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 10 monclient: dump: 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 10 monclient: _finish_auth 0 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 1 -- 172.21.15.89:0/2737582524 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de0003a90 con 0x7f7ddc003640 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df740f640 5 monclient: authenticate success, global_id 7344 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df740f640 10 monclient: _renew_subs 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df740f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df740f640 1 -- 172.21.15.89:0/2737582524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df00a39a0 con 0x7f7ddc003640 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 1 -- 172.21.15.89:0/2737582524 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7de0003c40 con 0x7f7ddc003640 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de77fe640 4 set_mon_vals no callback set 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 10 monclient: dump: 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df740f640 10 monclient: _renew_subs 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df740f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df740f640 1 -- 172.21.15.89:0/2737582524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df0123af0 con 0x7f7ddc003640 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 1 -- 172.21.15.89:0/2737582524 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7de0026020 con 0x7f7ddc003640 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df740f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df740f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df740f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df740f640 1 -- 172.21.15.89:0/2737582524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df00a7d60 con 0x7f7ddc003640 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd403be30 0x7f7dd403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:29.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7de5ffb640 1 -- 172.21.15.89:0/2737582524 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f7de000f040 con 0x7f7ddc003640 2024-08-19T10:53:29.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df49aa640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd403be30 0x7f7dd403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:29.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.438+0000 7f7df49aa640 10 monclient: get_auth_request con 0x7f7dd403be30 auth_method 0 2024-08-19T10:53:29.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.442+0000 7f7df49aa640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd403be30 0x7f7dd403e2f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7df00a0ee0 tx=0x7f7dec0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:29.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.442+0000 7f7de5ffb640 1 -- 172.21.15.89:0/2737582524 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7de004ed10 con 0x7f7ddc003640 2024-08-19T10:53:29.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.442+0000 7f7de5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:29.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.442+0000 7f7de5ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:29.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.674+0000 7f7df740f640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-08-19T10:53:29.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.674+0000 7f7df740f640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-08-19T10:53:29.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.674+0000 7f7df740f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:29.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.678+0000 7f7df740f640 1 -- 172.21.15.89:0/2737582524 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f7df00a4ae0 con 0x7f7ddc003640 2024-08-19T10:53:29.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.678+0000 7f7de5ffb640 1 -- 172.21.15.89:0/2737582524 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f7de00123e0 con 0x7f7ddc003640 2024-08-19T10:53:29.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.678+0000 7f7de5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-08-19T10:53:29.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.678+0000 7f7de5ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:29.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 -- 172.21.15.89:0/2737582524 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd403be30 msgr2=0x7f7dd403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd403be30 0x7f7dd403e2f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7df00a0ee0 tx=0x7f7dec0a6000 comp rx=0 tx=0).stop 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 10 monclient: shutdown 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 -- 172.21.15.89:0/2737582524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 msgr2=0x7f7df0120b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 0x7f7df0120b40 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7de000b950 tx=0x7f7de0004740 comp rx=0 tx=0).stop 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 -- 172.21.15.89:0/2737582524 shutdown_connections 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7df01d5dd0 0x7f7df01d81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7dd403be30 0x7f7dd403e2f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f7df0121080 0x7f7df01214c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 --2- 172.21.15.89:0/2737582524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ddc003640 0x7f7df0120b40 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 -- 172.21.15.89:0/2737582524 >> 172.21.15.89:0/2737582524 conn(0x7f7df00aa1b0 msgr2=0x7f7df013a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 -- 172.21.15.89:0/2737582524 shutdown_connections 2024-08-19T10:53:29.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:29.682+0000 7f7df740f640 1 -- 172.21.15.89:0/2737582524 wait complete. 2024-08-19T10:53:29.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=1,health=12,kv=12,logm=53} crc {auth=3374177568,config=1228700967,health=1877213006,kv=1101891974,logm=193087501}) 2024-08-19T10:53:29.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=2,health=11,kv=11,logm=53} crc {auth=3932802887,config=2586702516,health=388374834,kv=1523842694,logm=2861545154}) 2024-08-19T10:53:29.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=1,health=12,kv=12,logm=53} crc {auth=3374177568,config=1228700967,health=1877213006,kv=1101891974,logm=193087501}) 2024-08-19T10:53:29.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=24,config=2,health=12,kv=10,logm=52} crc {auth=4229509833,config=2586702516,health=1877213006,kv=1744794472,logm=2691519600}) 2024-08-19T10:53:29.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=1,health=12,kv=12,logm=53} crc {auth=3374177568,config=1228700967,health=1877213006,kv=1101891974,logm=193087501}) 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=23,config=2,health=12,kv=12,logm=51} crc {auth=2184240602,config=2586702516,health=1877213006,kv=1101891974,logm=3096453800}) 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=1,health=12,kv=12,logm=53} crc {auth=3374177568,config=1228700967,health=1877213006,kv=1101891974,logm=193087501}) 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=4229509833,config=2586702516,health=1877213007,kv=1550707372,logm=1662634245}) 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=1,health=12,kv=12,logm=53} crc {auth=3374177568,config=1228700967,health=1877213006,kv=1101891974,logm=193087501}) 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=23,config=2,health=11,kv=12,logm=52} crc {auth=2449671545,config=2586702516,health=233346546,kv=1101891974,logm=2084208246}) 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=1,health=12,kv=12,logm=53} crc {auth=3374177568,config=1228700967,health=1877213006,kv=1101891974,logm=193087501}) 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=22,config=2,health=11,kv=11,logm=54} crc {auth=2346114771,config=2586702516,health=3727025094,kv=2303807255,logm=2858460154}) 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=1,health=12,kv=12,logm=53} crc {auth=3374177568,config=1228700967,health=1877213006,kv=1101891974,logm=193087501}) 2024-08-19T10:53:29.687 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=21,config=2,health=12,kv=12,logm=53} crc {auth=3066957173,config=2586702516,health=2601151155,kv=1101891974,logm=436826032}) 2024-08-19T10:53:29.687 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.687 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=1,health=12,kv=12,logm=53} crc {auth=3374177568,config=1228700967,health=1877213006,kv=1101891974,logm=193087501}) 2024-08-19T10:53:29.687 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=22,config=2,health=11,kv=12,logm=53} crc {auth=1752526036,config=2586702516,health=376220456,kv=1101891974,logm=4258698594}) 2024-08-19T10:53:29.691 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.691 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2385540747}) 2024-08-19T10:53:29.691 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=565748073}) 2024-08-19T10:53:29.691 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.691 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2385540747}) 2024-08-19T10:53:29.691 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1864298441}) 2024-08-19T10:53:29.691 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.691 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2385540747}) 2024-08-19T10:53:29.691 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=212317999}) 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2385540747}) 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2819902164}) 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2385540747}) 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1160859089}) 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2385540747}) 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1909214143}) 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2385540747}) 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1178297100}) 2024-08-19T10:53:29.692 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.693 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2385540747}) 2024-08-19T10:53:29.693 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1712903512}) 2024-08-19T10:53:29.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1409079707}) 2024-08-19T10:53:29.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2350843253}) 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1409079707}) 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2356207283}) 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1409079707}) 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1414798272}) 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1409079707}) 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1522930883}) 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1409079707}) 2024-08-19T10:53:29.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3397804266}) 2024-08-19T10:53:29.698 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.698 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1409079707}) 2024-08-19T10:53:29.698 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3123911642}) 2024-08-19T10:53:29.698 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.698 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1409079707}) 2024-08-19T10:53:29.698 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1637843302}) 2024-08-19T10:53:29.698 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.698 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1409079707}) 2024-08-19T10:53:29.698 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3832360269}) 2024-08-19T10:53:29.699 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-08-19T10:53:29.703 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.703 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3538696963}) 2024-08-19T10:53:29.703 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2527493995}) 2024-08-19T10:53:29.703 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.703 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3538696963}) 2024-08-19T10:53:29.703 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2860127806}) 2024-08-19T10:53:29.703 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.703 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3538696963}) 2024-08-19T10:53:29.703 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2110715352}) 2024-08-19T10:53:29.703 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.703 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3538696963}) 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=403183210}) 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3538696963}) 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=364840076}) 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3538696963}) 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3359368870}) 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3538696963}) 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4175147553}) 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3538696963}) 2024-08-19T10:53:29.704 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.702+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1665851116}) 2024-08-19T10:53:29.707 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.707 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3184189727}) 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2421461389}) 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3184189727}) 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=14464234}) 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3184189727}) 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1812053194}) 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3184189727}) 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1372804581}) 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.708 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3184189727}) 2024-08-19T10:53:29.709 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3335149222}) 2024-08-19T10:53:29.709 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.709 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3184189727}) 2024-08-19T10:53:29.709 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4158450459}) 2024-08-19T10:53:29.709 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.709 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3184189727}) 2024-08-19T10:53:29.709 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2685922032}) 2024-08-19T10:53:29.709 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.709 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3184189727}) 2024-08-19T10:53:29.709 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.706+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=960523325}) 2024-08-19T10:53:29.715 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.715 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3693871208}) 2024-08-19T10:53:29.715 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3784437206}) 2024-08-19T10:53:29.715 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.715 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3693871208}) 2024-08-19T10:53:29.715 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2328957600}) 2024-08-19T10:53:29.715 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.715 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3693871208}) 2024-08-19T10:53:29.715 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2505656544}) 2024-08-19T10:53:29.715 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3693871208}) 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4266110457}) 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3693871208}) 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4255697768}) 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3693871208}) 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=360898851}) 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3693871208}) 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3976403996}) 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.716 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3693871208}) 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.710+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2695476333}) 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2866826128}) 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=639488562}) 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2866826128}) 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=115342456}) 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2866826128}) 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1058771743}) 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2866826128}) 2024-08-19T10:53:29.717 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2580633851}) 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2866826128}) 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2160109304}) 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2866826128}) 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1817278224}) 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2866826128}) 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4063529065}) 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2866826128}) 2024-08-19T10:53:29.718 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.714+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2502082338}) 2024-08-19T10:53:29.723 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.723 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=459309759}) 2024-08-19T10:53:29.723 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=70858141}) 2024-08-19T10:53:29.723 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.723 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=459309759}) 2024-08-19T10:53:29.723 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2049311849}) 2024-08-19T10:53:29.723 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.723 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=459309759}) 2024-08-19T10:53:29.723 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=466997090}) 2024-08-19T10:53:29.723 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=459309759}) 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=454535282}) 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=459309759}) 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1881465646}) 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=459309759}) 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1409949023}) 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=459309759}) 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4172626603}) 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=459309759}) 2024-08-19T10:53:29.724 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.718+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1359828866}) 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1103304534}) 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3473109078}) 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1103304534}) 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1611357157}) 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1103304534}) 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1887689949}) 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1103304534}) 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=721836059}) 2024-08-19T10:53:29.725 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.726 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1103304534}) 2024-08-19T10:53:29.726 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4198855751}) 2024-08-19T10:53:29.726 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.726 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1103304534}) 2024-08-19T10:53:29.726 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2318426688}) 2024-08-19T10:53:29.726 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.726 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1103304534}) 2024-08-19T10:53:29.726 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3932797524}) 2024-08-19T10:53:29.726 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.726 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1103304534}) 2024-08-19T10:53:29.726 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3647334416}) 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2587094176}) 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2236975572}) 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2587094176}) 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3085613981}) 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2587094176}) 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=236873784}) 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2587094176}) 2024-08-19T10:53:29.733 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3550058499}) 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2587094176}) 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3173138895}) 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2587094176}) 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4145598263}) 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2587094176}) 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3320850473}) 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2587094176}) 2024-08-19T10:53:29.734 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.726+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=29642060}) 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2791393603}) 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3301703118}) 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2791393603}) 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1744688254}) 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2791393603}) 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1404745000}) 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2791393603}) 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2871046584}) 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.735 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2791393603}) 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3646099448}) 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2791393603}) 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3952857043}) 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2791393603}) 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3128418406}) 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2791393603}) 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.730+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1204427081}) 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=469870316}) 2024-08-19T10:53:29.736 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4280765922}) 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=469870316}) 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1911425745}) 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=469870316}) 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2014444761}) 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=469870316}) 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3199795966}) 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=469870316}) 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2936699513}) 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.737 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=469870316}) 2024-08-19T10:53:29.738 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1584164977}) 2024-08-19T10:53:29.738 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.738 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=469870316}) 2024-08-19T10:53:29.738 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4229732555}) 2024-08-19T10:53:29.738 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.738 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=469870316}) 2024-08-19T10:53:29.738 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.734+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1291216676}) 2024-08-19T10:53:29.744 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.744 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545067125}) 2024-08-19T10:53:29.744 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3864671349}) 2024-08-19T10:53:29.744 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.744 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545067125}) 2024-08-19T10:53:29.744 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3922828523}) 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545067125}) 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1922366802}) 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545067125}) 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=621159088}) 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545067125}) 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2721548812}) 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545067125}) 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3463040418}) 2024-08-19T10:53:29.745 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545067125}) 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=85623673}) 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2545067125}) 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1535699828}) 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4165680886}) 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3764914007}) 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4165680886}) 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4194722022}) 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.746 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4165680886}) 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1363649695}) 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4165680886}) 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=995800713}) 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4165680886}) 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3138825210}) 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4165680886}) 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=444616944}) 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4165680886}) 2024-08-19T10:53:29.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=271238309}) 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4165680886}) 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1106567684}) 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722495446}) 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2609024005}) 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722495446}) 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1735064070}) 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722495446}) 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4125243247}) 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722495446}) 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2984097713}) 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722495446}) 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2663045078}) 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722495446}) 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=485303554}) 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722495446}) 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3221555470}) 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722495446}) 2024-08-19T10:53:29.749 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2927961960}) 2024-08-19T10:53:29.755 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.755 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=112443279}) 2024-08-19T10:53:29.755 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2855859401}) 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=112443279}) 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=349862438}) 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.746+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=112443279}) 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=698152889}) 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=112443279}) 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1223473195}) 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=112443279}) 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3227843423}) 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.756 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=112443279}) 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2274835917}) 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=112443279}) 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4038616312}) 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=112443279}) 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2967682330}) 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751827825}) 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4217093639}) 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751827825}) 2024-08-19T10:53:29.757 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3027235752}) 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751827825}) 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1469329273}) 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751827825}) 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1932637475}) 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751827825}) 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2097953908}) 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751827825}) 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1546313847}) 2024-08-19T10:53:29.758 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.750+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751827825}) 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1818155226}) 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1751827825}) 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1170632660}) 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982879763}) 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3772765483}) 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982879763}) 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3046299455}) 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.759 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982879763}) 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1994762474}) 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982879763}) 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=610905394}) 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982879763}) 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=385236616}) 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982879763}) 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1725868076}) 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982879763}) 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3394651876}) 2024-08-19T10:53:29.760 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.761 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982879763}) 2024-08-19T10:53:29.761 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=201588860}) 2024-08-19T10:53:29.766 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.766 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639515555}) 2024-08-19T10:53:29.766 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2308845257}) 2024-08-19T10:53:29.766 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.766 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639515555}) 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2872483528}) 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639515555}) 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3480849943}) 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639515555}) 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1346317095}) 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639515555}) 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3524142395}) 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639515555}) 2024-08-19T10:53:29.767 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4027380010}) 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639515555}) 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4072117549}) 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2639515555}) 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.758+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4180048042}) 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813157568}) 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1347631745}) 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813157568}) 2024-08-19T10:53:29.768 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4195079905}) 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813157568}) 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1242756015}) 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813157568}) 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3802989571}) 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813157568}) 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3518624659}) 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813157568}) 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=870492555}) 2024-08-19T10:53:29.769 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813157568}) 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1940601950}) 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3813157568}) 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=9609202}) 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555604698}) 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4121971578}) 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555604698}) 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=485056207}) 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555604698}) 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2538584342}) 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555604698}) 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2345091108}) 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555604698}) 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2696128284}) 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555604698}) 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1500046674}) 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555604698}) 2024-08-19T10:53:29.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=604772988}) 2024-08-19T10:53:29.772 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.772 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555604698}) 2024-08-19T10:53:29.772 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.766+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2418961963}) 2024-08-19T10:53:29.778 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.778 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311713092}) 2024-08-19T10:53:29.778 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2839714017}) 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311713092}) 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=610920637}) 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311713092}) 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1770462787}) 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311713092}) 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1023380164}) 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311713092}) 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3521846451}) 2024-08-19T10:53:29.779 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311713092}) 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3148082655}) 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311713092}) 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1594844028}) 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311713092}) 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3714072713}) 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1033446485}) 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4087312881}) 2024-08-19T10:53:29.780 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1033446485}) 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2888708257}) 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1033446485}) 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4019983400}) 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1033446485}) 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3500601928}) 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1033446485}) 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1508642458}) 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1033446485}) 2024-08-19T10:53:29.781 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3465269438}) 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1033446485}) 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3429385617}) 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1033446485}) 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.770+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=416521644}) 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2123235890}) 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1780544969}) 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2123235890}) 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=292873317}) 2024-08-19T10:53:29.782 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2123235890}) 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3888142720}) 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2123235890}) 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=651314458}) 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2123235890}) 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3577630832}) 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2123235890}) 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2081563770}) 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.783 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2123235890}) 2024-08-19T10:53:29.784 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=525545541}) 2024-08-19T10:53:29.784 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.784 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2123235890}) 2024-08-19T10:53:29.784 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.774+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683809072}) 2024-08-19T10:53:29.790 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.790 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623044358}) 2024-08-19T10:53:29.790 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4114619604}) 2024-08-19T10:53:29.790 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.790 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623044358}) 2024-08-19T10:53:29.790 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1082388631}) 2024-08-19T10:53:29.790 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.790 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623044358}) 2024-08-19T10:53:29.790 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3755493486}) 2024-08-19T10:53:29.790 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.790 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623044358}) 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=570754253}) 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623044358}) 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1396875504}) 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623044358}) 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3609845202}) 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623044358}) 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3297617086}) 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3623044358}) 2024-08-19T10:53:29.791 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.782+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2841065755}) 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2584987430}) 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3324666613}) 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2584987430}) 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=183337402}) 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2584987430}) 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=769148803}) 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2584987430}) 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=511816316}) 2024-08-19T10:53:29.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2584987430}) 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=112066803}) 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2584987430}) 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=465500511}) 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2584987430}) 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2168417746}) 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2584987430}) 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1810305924}) 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1770741447}) 2024-08-19T10:53:29.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2272164837}) 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1770741447}) 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1469485704}) 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1770741447}) 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1313090617}) 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1770741447}) 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4125480581}) 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1770741447}) 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=332494931}) 2024-08-19T10:53:29.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.795 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1770741447}) 2024-08-19T10:53:29.795 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1968987284}) 2024-08-19T10:53:29.795 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.795 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1770741447}) 2024-08-19T10:53:29.795 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2686643370}) 2024-08-19T10:53:29.795 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.795 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1770741447}) 2024-08-19T10:53:29.795 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.790+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1244971832}) 2024-08-19T10:53:29.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3505571003}) 2024-08-19T10:53:29.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1440572869}) 2024-08-19T10:53:29.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3505571003}) 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3900511906}) 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3505571003}) 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4026371555}) 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3505571003}) 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3646383961}) 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3505571003}) 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1913235615}) 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3505571003}) 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1745883683}) 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3505571003}) 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3933255037}) 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3505571003}) 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3150608257}) 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=653792276}) 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1247392014}) 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.803 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=653792276}) 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=533685338}) 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=653792276}) 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3762803311}) 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=653792276}) 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3237867236}) 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=653792276}) 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2497717861}) 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=653792276}) 2024-08-19T10:53:29.804 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1427454124}) 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=653792276}) 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3111953222}) 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=653792276}) 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3886886716}) 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153226953}) 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3915231461}) 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153226953}) 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3263847453}) 2024-08-19T10:53:29.805 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153226953}) 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1328309149}) 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153226953}) 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2902289846}) 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153226953}) 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2318859676}) 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153226953}) 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2651299045}) 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.806 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153226953}) 2024-08-19T10:53:29.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1169560619}) 2024-08-19T10:53:29.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:29.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2153226953}) 2024-08-19T10:53:29.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:29.798+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2020096511}) 2024-08-19T10:53:30.315 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.315 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391344003}) 2024-08-19T10:53:30.315 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2287375835}) 2024-08-19T10:53:30.315 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.315 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391344003}) 2024-08-19T10:53:30.315 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2580037397}) 2024-08-19T10:53:30.315 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.315 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391344003}) 2024-08-19T10:53:30.316 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1304960081}) 2024-08-19T10:53:30.316 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.316 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391344003}) 2024-08-19T10:53:30.316 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=603675694}) 2024-08-19T10:53:30.316 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.316 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391344003}) 2024-08-19T10:53:30.316 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1978157199}) 2024-08-19T10:53:30.316 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.316 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391344003}) 2024-08-19T10:53:30.316 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4289770117}) 2024-08-19T10:53:30.317 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.317 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391344003}) 2024-08-19T10:53:30.317 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2247779648}) 2024-08-19T10:53:30.317 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.317 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391344003}) 2024-08-19T10:53:30.317 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.314+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2179815235}) 2024-08-19T10:53:30.319 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.319 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863884017}) 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3627796835}) 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863884017}) 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3976188963}) 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863884017}) 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=34425420}) 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863884017}) 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1905906763}) 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.320 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863884017}) 2024-08-19T10:53:30.321 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3013366306}) 2024-08-19T10:53:30.321 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.321 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863884017}) 2024-08-19T10:53:30.321 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1741926621}) 2024-08-19T10:53:30.321 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.321 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863884017}) 2024-08-19T10:53:30.321 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1740931169}) 2024-08-19T10:53:30.321 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.321 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863884017}) 2024-08-19T10:53:30.321 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.318+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3823134053}) 2024-08-19T10:53:30.333 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.333 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2479608506}) 2024-08-19T10:53:30.333 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=14599477}) 2024-08-19T10:53:30.333 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.333 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2479608506}) 2024-08-19T10:53:30.333 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1390251479}) 2024-08-19T10:53:30.333 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.333 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2479608506}) 2024-08-19T10:53:30.333 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3604532768}) 2024-08-19T10:53:30.333 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2479608506}) 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1445948421}) 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2479608506}) 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3188551421}) 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2479608506}) 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3257389000}) 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2479608506}) 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1534988004}) 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2479608506}) 2024-08-19T10:53:30.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=72555874}) 2024-08-19T10:53:30.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=322896577}) 2024-08-19T10:53:30.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2739457562}) 2024-08-19T10:53:30.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=322896577}) 2024-08-19T10:53:30.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2795781531}) 2024-08-19T10:53:30.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=322896577}) 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1464774005}) 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=322896577}) 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2466464984}) 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=322896577}) 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2286109041}) 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=322896577}) 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3791023888}) 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=322896577}) 2024-08-19T10:53:30.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=400887732}) 2024-08-19T10:53:30.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=322896577}) 2024-08-19T10:53:30.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2635987941}) 2024-08-19T10:53:30.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935620241}) 2024-08-19T10:53:30.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2799587651}) 2024-08-19T10:53:30.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935620241}) 2024-08-19T10:53:30.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=55956098}) 2024-08-19T10:53:30.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935620241}) 2024-08-19T10:53:30.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1960034985}) 2024-08-19T10:53:30.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935620241}) 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3807535338}) 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935620241}) 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=575447914}) 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935620241}) 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2139524249}) 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935620241}) 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3681513994}) 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.343 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935620241}) 2024-08-19T10:53:30.344 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4237987753}) 2024-08-19T10:53:30.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3635993629}) 2024-08-19T10:53:30.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=561712494}) 2024-08-19T10:53:30.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3635993629}) 2024-08-19T10:53:30.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=415396916}) 2024-08-19T10:53:30.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3635993629}) 2024-08-19T10:53:30.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3061910798}) 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3635993629}) 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3446051904}) 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3635993629}) 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2444377897}) 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3635993629}) 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=735958201}) 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3635993629}) 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1020670066}) 2024-08-19T10:53:30.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.350 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3635993629}) 2024-08-19T10:53:30.350 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.350+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2753373536}) 2024-08-19T10:53:30.352 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3954178694}) 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2295757242}) 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3954178694}) 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3769924057}) 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3954178694}) 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1912891483}) 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3954178694}) 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3748736918}) 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.353 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3954178694}) 2024-08-19T10:53:30.354 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1422138891}) 2024-08-19T10:53:30.354 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.354 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3954178694}) 2024-08-19T10:53:30.354 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=30303706}) 2024-08-19T10:53:30.354 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.354 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3954178694}) 2024-08-19T10:53:30.354 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=277375477}) 2024-08-19T10:53:30.354 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.354 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3954178694}) 2024-08-19T10:53:30.354 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.354+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3426511111}) 2024-08-19T10:53:30.356 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.356 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1840168060}) 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4193365012}) 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1840168060}) 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2298743218}) 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1840168060}) 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1372597210}) 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1840168060}) 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1631301095}) 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1840168060}) 2024-08-19T10:53:30.357 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1129752794}) 2024-08-19T10:53:30.358 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.358 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1840168060}) 2024-08-19T10:53:30.358 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4207421973}) 2024-08-19T10:53:30.358 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.358 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1840168060}) 2024-08-19T10:53:30.358 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1822809721}) 2024-08-19T10:53:30.358 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.358 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1840168060}) 2024-08-19T10:53:30.358 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.358+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3765326422}) 2024-08-19T10:53:30.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1261780701}) 2024-08-19T10:53:30.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1815614380}) 2024-08-19T10:53:30.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1261780701}) 2024-08-19T10:53:30.361 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2361763688}) 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1261780701}) 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1759362699}) 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1261780701}) 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=924521294}) 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1261780701}) 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1483011999}) 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1261780701}) 2024-08-19T10:53:30.362 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2045792917}) 2024-08-19T10:53:30.363 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.363 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1261780701}) 2024-08-19T10:53:30.363 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2689542624}) 2024-08-19T10:53:30.363 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.363 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1261780701}) 2024-08-19T10:53:30.363 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.362+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=311115509}) 2024-08-19T10:53:30.365 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966088500}) 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2504354697}) 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966088500}) 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=822540949}) 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966088500}) 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2261499157}) 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966088500}) 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3621829648}) 2024-08-19T10:53:30.366 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.367 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966088500}) 2024-08-19T10:53:30.367 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3511787279}) 2024-08-19T10:53:30.367 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.367 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966088500}) 2024-08-19T10:53:30.367 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3803272638}) 2024-08-19T10:53:30.367 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.367 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966088500}) 2024-08-19T10:53:30.367 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1380351592}) 2024-08-19T10:53:30.367 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.367 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966088500}) 2024-08-19T10:53:30.367 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.366+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4085562101}) 2024-08-19T10:53:30.370 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.370 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914545259}) 2024-08-19T10:53:30.370 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2257698615}) 2024-08-19T10:53:30.370 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.370 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914545259}) 2024-08-19T10:53:30.370 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3073625586}) 2024-08-19T10:53:30.370 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.370 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914545259}) 2024-08-19T10:53:30.370 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=711350892}) 2024-08-19T10:53:30.370 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914545259}) 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=227626002}) 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914545259}) 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=703753727}) 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914545259}) 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=594503168}) 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914545259}) 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4038254889}) 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=914545259}) 2024-08-19T10:53:30.371 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.370+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1580976220}) 2024-08-19T10:53:30.373 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.373 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594497772}) 2024-08-19T10:53:30.373 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2622763518}) 2024-08-19T10:53:30.373 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.373 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594497772}) 2024-08-19T10:53:30.374 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2620864195}) 2024-08-19T10:53:30.374 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.374 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594497772}) 2024-08-19T10:53:30.374 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3836771835}) 2024-08-19T10:53:30.374 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.374 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594497772}) 2024-08-19T10:53:30.374 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1565572611}) 2024-08-19T10:53:30.374 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.374 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594497772}) 2024-08-19T10:53:30.374 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=377311773}) 2024-08-19T10:53:30.375 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.375 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594497772}) 2024-08-19T10:53:30.375 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3219704830}) 2024-08-19T10:53:30.375 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.375 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594497772}) 2024-08-19T10:53:30.375 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4061723243}) 2024-08-19T10:53:30.375 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.375 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594497772}) 2024-08-19T10:53:30.375 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.374+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1975746631}) 2024-08-19T10:53:30.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727292991}) 2024-08-19T10:53:30.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2625353855}) 2024-08-19T10:53:30.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727292991}) 2024-08-19T10:53:30.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2483796375}) 2024-08-19T10:53:30.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727292991}) 2024-08-19T10:53:30.377 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3410233247}) 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727292991}) 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=616977970}) 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727292991}) 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=557580315}) 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727292991}) 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1050517688}) 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727292991}) 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1065202819}) 2024-08-19T10:53:30.378 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.379 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727292991}) 2024-08-19T10:53:30.379 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.378+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1300571738}) 2024-08-19T10:53:30.381 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.381 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384761428}) 2024-08-19T10:53:30.381 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2619281055}) 2024-08-19T10:53:30.381 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.381 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384761428}) 2024-08-19T10:53:30.381 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1403808351}) 2024-08-19T10:53:30.381 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384761428}) 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3785623298}) 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384761428}) 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3782618547}) 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384761428}) 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=897089790}) 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384761428}) 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3433562381}) 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.382 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384761428}) 2024-08-19T10:53:30.383 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=253504900}) 2024-08-19T10:53:30.383 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.383 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384761428}) 2024-08-19T10:53:30.383 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.382+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3548870882}) 2024-08-19T10:53:30.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=334476606}) 2024-08-19T10:53:30.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3325167078}) 2024-08-19T10:53:30.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=334476606}) 2024-08-19T10:53:30.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2758083298}) 2024-08-19T10:53:30.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.385 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=334476606}) 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2241605599}) 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=334476606}) 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3077215661}) 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=334476606}) 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1515257989}) 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=334476606}) 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=845827665}) 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=334476606}) 2024-08-19T10:53:30.386 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=238905494}) 2024-08-19T10:53:30.387 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.387 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=334476606}) 2024-08-19T10:53:30.387 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.386+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4256983009}) 2024-08-19T10:53:30.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3376815673}) 2024-08-19T10:53:30.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1747333248}) 2024-08-19T10:53:30.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3376815673}) 2024-08-19T10:53:30.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1281570720}) 2024-08-19T10:53:30.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.389 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3376815673}) 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1868428750}) 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3376815673}) 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2617779021}) 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3376815673}) 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3528096930}) 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3376815673}) 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2402864203}) 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3376815673}) 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1691066672}) 2024-08-19T10:53:30.390 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.391 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3376815673}) 2024-08-19T10:53:30.391 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.390+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2187935651}) 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939178473}) 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1893141971}) 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939178473}) 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417062066}) 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939178473}) 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3042124569}) 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939178473}) 2024-08-19T10:53:30.393 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3217135581}) 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939178473}) 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=936818556}) 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939178473}) 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2792107406}) 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939178473}) 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1183427266}) 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939178473}) 2024-08-19T10:53:30.394 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.394+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4250793208}) 2024-08-19T10:53:30.397 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.397 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545686809}) 2024-08-19T10:53:30.397 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=198694380}) 2024-08-19T10:53:30.397 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545686809}) 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2913964109}) 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545686809}) 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2052267379}) 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545686809}) 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3390152412}) 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545686809}) 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3313893045}) 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545686809}) 2024-08-19T10:53:30.398 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3798366393}) 2024-08-19T10:53:30.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545686809}) 2024-08-19T10:53:30.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4140505875}) 2024-08-19T10:53:30.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545686809}) 2024-08-19T10:53:30.399 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.398+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3538051815}) 2024-08-19T10:53:30.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524570509}) 2024-08-19T10:53:30.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2257039325}) 2024-08-19T10:53:30.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.400 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524570509}) 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4076560743}) 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524570509}) 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1886420464}) 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524570509}) 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3673702829}) 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524570509}) 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3886823008}) 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524570509}) 2024-08-19T10:53:30.401 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1400885624}) 2024-08-19T10:53:30.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524570509}) 2024-08-19T10:53:30.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2655698840}) 2024-08-19T10:53:30.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=524570509}) 2024-08-19T10:53:30.402 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3163559784}) 2024-08-19T10:53:30.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2604652668}) 2024-08-19T10:53:30.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3711303030}) 2024-08-19T10:53:30.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2604652668}) 2024-08-19T10:53:30.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4218327458}) 2024-08-19T10:53:30.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2604652668}) 2024-08-19T10:53:30.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3120442561}) 2024-08-19T10:53:30.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2604652668}) 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1477713054}) 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2604652668}) 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2058333805}) 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2604652668}) 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=615565526}) 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2604652668}) 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3268372426}) 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2604652668}) 2024-08-19T10:53:30.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.406+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1240950226}) 2024-08-19T10:53:30.408 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.408 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=604334572}) 2024-08-19T10:53:30.408 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3221179009}) 2024-08-19T10:53:30.408 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.408 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=604334572}) 2024-08-19T10:53:30.408 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=237942625}) 2024-08-19T10:53:30.408 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.408 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=604334572}) 2024-08-19T10:53:30.408 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=994939899}) 2024-08-19T10:53:30.408 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.408 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=604334572}) 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1756948976}) 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=604334572}) 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=191774717}) 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=604334572}) 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2791285207}) 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=604334572}) 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3427897018}) 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=604334572}) 2024-08-19T10:53:30.409 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.410+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3656881057}) 2024-08-19T10:53:30.411 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.411 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=923993233}) 2024-08-19T10:53:30.411 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=78843412}) 2024-08-19T10:53:30.411 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.411 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=923993233}) 2024-08-19T10:53:30.411 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3277938122}) 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=923993233}) 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3288113701}) 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=923993233}) 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1227357708}) 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=923993233}) 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3611482972}) 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=923993233}) 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1054683279}) 2024-08-19T10:53:30.412 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.413 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=923993233}) 2024-08-19T10:53:30.413 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4072857781}) 2024-08-19T10:53:30.413 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.413 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=923993233}) 2024-08-19T10:53:30.413 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.414+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3998199301}) 2024-08-19T10:53:30.415 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.415 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=582162178}) 2024-08-19T10:53:30.415 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2674266425}) 2024-08-19T10:53:30.415 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.415 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=582162178}) 2024-08-19T10:53:30.415 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1438725926}) 2024-08-19T10:53:30.415 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.415 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=582162178}) 2024-08-19T10:53:30.415 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3744384895}) 2024-08-19T10:53:30.415 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=582162178}) 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3330657048}) 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=582162178}) 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3827682911}) 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=582162178}) 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3024075858}) 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=582162178}) 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1039260806}) 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=582162178}) 2024-08-19T10:53:30.416 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=355087765}) 2024-08-19T10:53:30.418 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.418 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=132344761}) 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=712397382}) 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=132344761}) 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3412939030}) 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=132344761}) 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2212206021}) 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=132344761}) 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3670658331}) 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=132344761}) 2024-08-19T10:53:30.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=750853764}) 2024-08-19T10:53:30.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=132344761}) 2024-08-19T10:53:30.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3388785609}) 2024-08-19T10:53:30.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=132344761}) 2024-08-19T10:53:30.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3507527229}) 2024-08-19T10:53:30.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=132344761}) 2024-08-19T10:53:30.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3223939388}) 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=353454622}) 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1015874618}) 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=353454622}) 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=929799629}) 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=353454622}) 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=739197443}) 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=353454622}) 2024-08-19T10:53:30.423 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1533143559}) 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=353454622}) 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1494005329}) 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=353454622}) 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2094104590}) 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=353454622}) 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3813620483}) 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=353454622}) 2024-08-19T10:53:30.424 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.422+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1224413833}) 2024-08-19T10:53:30.426 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.426 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3684033599}) 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1774139927}) 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3684033599}) 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2600617661}) 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3684033599}) 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=943784589}) 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3684033599}) 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1920999445}) 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3684033599}) 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4062395382}) 2024-08-19T10:53:30.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3684033599}) 2024-08-19T10:53:30.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3899828857}) 2024-08-19T10:53:30.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3684033599}) 2024-08-19T10:53:30.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2367164138}) 2024-08-19T10:53:30.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3684033599}) 2024-08-19T10:53:30.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2939720962}) 2024-08-19T10:53:30.430 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.430 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483540093}) 2024-08-19T10:53:30.430 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2441153532}) 2024-08-19T10:53:30.430 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.430 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483540093}) 2024-08-19T10:53:30.430 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3163941504}) 2024-08-19T10:53:30.430 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.430 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483540093}) 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2451598809}) 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483540093}) 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3228206872}) 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483540093}) 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=238925361}) 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483540093}) 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=657757105}) 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483540093}) 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1831134056}) 2024-08-19T10:53:30.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483540093}) 2024-08-19T10:53:30.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2271062699}) 2024-08-19T10:53:30.433 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1684036763}) 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1029364005}) 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1684036763}) 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=883560551}) 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1684036763}) 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3226332010}) 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1684036763}) 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1205163691}) 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.434 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1684036763}) 2024-08-19T10:53:30.435 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=144604644}) 2024-08-19T10:53:30.435 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.435 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1684036763}) 2024-08-19T10:53:30.435 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2733121572}) 2024-08-19T10:53:30.435 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.435 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1684036763}) 2024-08-19T10:53:30.435 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2770139909}) 2024-08-19T10:53:30.435 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.435 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1684036763}) 2024-08-19T10:53:30.435 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.434+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2056840061}) 2024-08-19T10:53:30.437 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.437 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336478354}) 2024-08-19T10:53:30.437 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3961903652}) 2024-08-19T10:53:30.437 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.437 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336478354}) 2024-08-19T10:53:30.437 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1903066279}) 2024-08-19T10:53:30.437 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.437 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336478354}) 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1908774365}) 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336478354}) 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4008315111}) 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336478354}) 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=347956723}) 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336478354}) 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1560534564}) 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336478354}) 2024-08-19T10:53:30.438 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=257669031}) 2024-08-19T10:53:30.439 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.439 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336478354}) 2024-08-19T10:53:30.439 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3661717125}) 2024-08-19T10:53:30.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139787056}) 2024-08-19T10:53:30.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=739704920}) 2024-08-19T10:53:30.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139787056}) 2024-08-19T10:53:30.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3174670706}) 2024-08-19T10:53:30.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139787056}) 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2417394005}) 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139787056}) 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2715974902}) 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139787056}) 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1921620337}) 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139787056}) 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2574059598}) 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139787056}) 2024-08-19T10:53:30.443 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2114841938}) 2024-08-19T10:53:30.443 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.443 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139787056}) 2024-08-19T10:53:30.443 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=408136581}) 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1034320018}) 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3362251237}) 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1034320018}) 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=750482905}) 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1034320018}) 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3353611774}) 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1034320018}) 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=986676113}) 2024-08-19T10:53:30.445 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1034320018}) 2024-08-19T10:53:30.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4158270114}) 2024-08-19T10:53:30.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1034320018}) 2024-08-19T10:53:30.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3983539327}) 2024-08-19T10:53:30.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1034320018}) 2024-08-19T10:53:30.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3123330420}) 2024-08-19T10:53:30.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1034320018}) 2024-08-19T10:53:30.446 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1067015762}) 2024-08-19T10:53:30.448 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.448 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1882185004}) 2024-08-19T10:53:30.448 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4097492199}) 2024-08-19T10:53:30.448 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.448 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1882185004}) 2024-08-19T10:53:30.448 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2863088141}) 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1882185004}) 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2763592330}) 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1882185004}) 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3304169153}) 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1882185004}) 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3635784792}) 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1882185004}) 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1887232937}) 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.449 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1882185004}) 2024-08-19T10:53:30.450 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3240015140}) 2024-08-19T10:53:30.450 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.450 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1882185004}) 2024-08-19T10:53:30.450 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=228856062}) 2024-08-19T10:53:30.453 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.453 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3668896658}) 2024-08-19T10:53:30.453 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=562449755}) 2024-08-19T10:53:30.453 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.453 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3668896658}) 2024-08-19T10:53:30.453 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1462453474}) 2024-08-19T10:53:30.453 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.453 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3668896658}) 2024-08-19T10:53:30.453 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1586965949}) 2024-08-19T10:53:30.453 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3668896658}) 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4011996498}) 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3668896658}) 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1413940050}) 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3668896658}) 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=275043655}) 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3668896658}) 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4102978194}) 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.454 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3668896658}) 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3170502746}) 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592600572}) 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1965527222}) 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592600572}) 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=851017143}) 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592600572}) 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=837111515}) 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592600572}) 2024-08-19T10:53:30.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3173855437}) 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592600572}) 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1779691362}) 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592600572}) 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4066705447}) 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592600572}) 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1816636159}) 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592600572}) 2024-08-19T10:53:30.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.454+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1145649909}) 2024-08-19T10:53:30.647 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.648 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014679895}) 2024-08-19T10:53:30.648 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=522414455}) 2024-08-19T10:53:30.648 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.648 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014679895}) 2024-08-19T10:53:30.648 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3199545143}) 2024-08-19T10:53:30.648 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.648 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014679895}) 2024-08-19T10:53:30.648 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=966152342}) 2024-08-19T10:53:30.648 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.648 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.646+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014679895}) 2024-08-19T10:53:30.648 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1193422890}) 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014679895}) 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=515415957}) 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014679895}) 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1772195951}) 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014679895}) 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2542424735}) 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014679895}) 2024-08-19T10:53:30.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1928673241}) 2024-08-19T10:53:30.688 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.688 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2667920000}) 2024-08-19T10:53:30.688 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=774589178}) 2024-08-19T10:53:30.688 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2667920000}) 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2933313644}) 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2667920000}) 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3476939848}) 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2667920000}) 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3501282223}) 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2667920000}) 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=460495706}) 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2667920000}) 2024-08-19T10:53:30.689 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3371915034}) 2024-08-19T10:53:30.690 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.690 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2667920000}) 2024-08-19T10:53:30.690 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1999600438}) 2024-08-19T10:53:30.690 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.690 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2667920000}) 2024-08-19T10:53:30.690 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.690+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3646405980}) 2024-08-19T10:53:30.700 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:53:30.701 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:53:30.787 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.787 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765059279}) 2024-08-19T10:53:30.787 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=535683257}) 2024-08-19T10:53:30.787 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765059279}) 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2926050047}) 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765059279}) 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4199041492}) 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765059279}) 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1926851289}) 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765059279}) 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3461310842}) 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765059279}) 2024-08-19T10:53:30.788 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1811681865}) 2024-08-19T10:53:30.789 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.789 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765059279}) 2024-08-19T10:53:30.789 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2480733459}) 2024-08-19T10:53:30.789 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:30.789 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2765059279}) 2024-08-19T10:53:30.789 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:30.786+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4027350335}) 2024-08-19T10:53:30.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient: get_monmap_and_config 2024-08-19T10:53:30.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient: build_initial_monmap 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient: monmap: 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:30.846898+0000 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:30.846898+0000 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:30.841 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:30.842 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:30.842 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:30.842 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:30.842 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:30.842 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:30.842 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:30.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 1 Processor -- start 2024-08-19T10:53:30.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 1 -- start start 2024-08-19T10:53:30.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient: init 2024-08-19T10:53:30.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:30.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient: _add_conns ranks=[10,4,7,5,1,2,6,9,0,8,3] 2024-08-19T10:53:30.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5d5c0a6860 0x7f5d5c0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:30.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient(hunting): picked mon.noname-f con 0x7f5d5c0a6860 addr v2:172.21.15.190:3300/0 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a7180 0x7f5d5c0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient(hunting): picked mon.noname-c con 0x7f5d5c0a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5d5c139620 0x7f5d5c139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient(hunting): picked mon.noname-i con 0x7f5d5c139620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f5d5c0a8190 con 0x7f5d5c0a6860 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f5d5c0a33d0 con 0x7f5d5c0a7180 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5d5c0a2ec0 con 0x7f5d5c139620 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d62a52640 10 monclient(hunting): authenticate will time out at 1860.663413s 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a7180 0x7f5d5c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:30.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a7180 0x7f5d5c0a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49106/0 (socket says 172.21.15.89:49106) 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 1 -- 172.21.15.89:0/4157991540 learned_addr learned my addr 172.21.15.89:0/4157991540 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 10 monclient(hunting): get_auth_request con 0x7f5d5c0a7180 auth_method 0 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5bfff640 1 --2- 172.21.15.89:0/4157991540 >> v2:172.21.15.190:3300/0 conn(0x7f5d5c0a6860 0x7f5d5c0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5bfff640 10 monclient(hunting): get_auth_request con 0x7f5d5c0a6860 auth_method 0 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:30.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d60fef640 1 --2- 172.21.15.89:0/4157991540 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5d5c139620 0x7f5d5c139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 10 monclient(hunting): handle_auth_done global_id 84136 payload 274 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 1 -- 172.21.15.89:0/4157991540 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5d5c139620 msgr2=0x7f5d5c139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 1 --2- 172.21.15.89:0/4157991540 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5d5c139620 0x7f5d5c139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 1 -- 172.21.15.89:0/4157991540 >> v2:172.21.15.190:3300/0 conn(0x7f5d5c0a6860 msgr2=0x7f5d5c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 1 --2- 172.21.15.89:0/4157991540 >> v2:172.21.15.190:3300/0 conn(0x7f5d5c0a6860 0x7f5d5c0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 1 monclient: found mon.noname-c 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d5b7fe640 1 -- 172.21.15.89:0/4157991540 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c0a4cd0 con 0x7f5d5c0a7180 2024-08-19T10:53:30.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d60fef640 10 monclient: get_auth_request con 0x7f5d5c139620 auth_method 0 2024-08-19T10:53:30.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.842+0000 7f5d60fef640 1 --2- 172.21.15.89:0/4157991540 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5d5c139620 0x7f5d5c139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:53:30.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5b7fe640 1 -- 172.21.15.89:0/4157991540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a7180 msgr2=0x7f5d5c0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:30.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5b7fe640 1 --2- 172.21.15.89:0/4157991540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a7180 0x7f5d5c0a7560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009870 tx=0x7f5d5001c420 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a29c0be45b33cfe4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:30.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 1 -- 172.21.15.89:0/4157991540 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d500b0070 con 0x7f5d5c0a7180 2024-08-19T10:53:30.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:30.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:53:30.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 10 monclient: dump: 2024-08-19T10:53:30.851 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:30.851 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:30.852 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 10 monclient: _finish_auth 0 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 1 -- 172.21.15.89:0/4157991540 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d500aa810 con 0x7f5d5c0a7180 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 5 monclient: authenticate success, global_id 84136 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 1 -- 172.21.15.89:0/4157991540 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d500a9300 con 0x7f5d5c0a7180 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:30.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 4 set_mon_vals no callback set 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5affd640 10 monclient: dump: 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:30.854 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient: shutdown 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 -- 172.21.15.89:0/4157991540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a7180 msgr2=0x7f5d5c0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 --2- 172.21.15.89:0/4157991540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a7180 0x7f5d5c0a7560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009870 tx=0x7f5d5001c420 comp rx=0 tx=0).stop 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 -- 172.21.15.89:0/4157991540 shutdown_connections 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 --2- 172.21.15.89:0/4157991540 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5d5c139620 0x7f5d5c139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:30.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 --2- 172.21.15.89:0/4157991540 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a7180 0x7f5d5c0a7560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 --2- 172.21.15.89:0/4157991540 >> v2:172.21.15.190:3300/0 conn(0x7f5d5c0a6860 0x7f5d5c0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 -- 172.21.15.89:0/4157991540 >> 172.21.15.89:0/4157991540 conn(0x7f5d5c074b40 msgr2=0x7f5d5c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 -- 172.21.15.89:0/4157991540 shutdown_connections 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 -- 172.21.15.89:0/4157991540 wait complete. 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient: build_initial_monmap 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient: monmap: 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:30.856 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 Processor -- start 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 -- start start 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient: init 2024-08-19T10:53:30.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient: _add_conns ranks=[4,7,2,6,0,8,1,3,5] 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a6860 0x7f5d5c1d1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient(hunting): picked mon.noname-c con 0x7f5d5c0a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5d5c0a7180 0x7f5d5c0a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient(hunting): picked mon.noname-i con 0x7f5d5c0a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5c139620 0x7f5d5c0a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient(hunting): picked mon.noname-b con 0x7f5d5c139620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5c143210 con 0x7f5d5c139620 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f5d5c143090 con 0x7f5d5c0a6860 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:30.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f5d5c143390 con 0x7f5d5c0a7180 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d62a52640 10 monclient(hunting): authenticate will time out at 1860.670517s 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a6860 0x7f5d5c1d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a6860 0x7f5d5c1d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:49122/0 (socket says 172.21.15.89:49122) 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 1 -- 172.21.15.89:0/4244899006 learned_addr learned my addr 172.21.15.89:0/4244899006 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 10 monclient(hunting): get_auth_request con 0x7f5d5c0a6860 auth_method 0 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d60fef640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5c139620 0x7f5d5c0a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d60fef640 10 monclient(hunting): get_auth_request con 0x7f5d5c139620 auth_method 0 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d60fef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d60fef640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:30.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d60fef640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d60fef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d60fef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d60fef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5b7fe640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5d5c0a7180 0x7f5d5c0a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5b7fe640 10 monclient(hunting): get_auth_request con 0x7f5d5c0a7180 auth_method 0 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 10 monclient(hunting): handle_auth_done global_id 84145 payload 824 2024-08-19T10:53:30.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 1 -- 172.21.15.89:0/4244899006 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5d5c0a7180 msgr2=0x7f5d5c0a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:30.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5d5c0a7180 0x7f5d5c0a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:30.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 1 -- 172.21.15.89:0/4244899006 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5c139620 msgr2=0x7f5d5c0a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:30.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5c139620 0x7f5d5c0a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:30.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:53:30.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 1 monclient: found mon.noname-c 2024-08-19T10:53:30.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:30.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:53:30.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.850+0000 7f5d5bfff640 1 -- 172.21.15.89:0/4244899006 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c0a2540 con 0x7f5d5c0a6860 2024-08-19T10:53:30.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d5bfff640 1 -- 172.21.15.89:0/4244899006 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a6860 msgr2=0x7f5d5c1d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:30.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d5bfff640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a6860 0x7f5d5c1d1580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00d800 tx=0x7f5d4c00dcc0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:30.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 1 -- 172.21.15.89:0/4244899006 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d4c00bd40 con 0x7f5d5c0a6860 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 10 monclient: dump: 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:30.864 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 10 monclient: _finish_auth 0 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d62a52640 5 monclient: authenticate success, global_id 84145 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 1 -- 172.21.15.89:0/4244899006 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4c019070 con 0x7f5d5c0a6860 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d62a52640 10 monclient: _renew_subs 2024-08-19T10:53:30.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d62a52640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d62a52640 1 -- 172.21.15.89:0/4244899006 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5c09e230 con 0x7f5d5c0a6860 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 1 -- 172.21.15.89:0/4244899006 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5d4c0148a0 con 0x7f5d5c0a6860 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d5a7fc640 4 set_mon_vals no callback set 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 10 monclient: dump: 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:30.866 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d62a52640 10 monclient: _renew_subs 2024-08-19T10:53:30.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d62a52640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d62a52640 1 -- 172.21.15.89:0/4244899006 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5c09fcf0 con 0x7f5d5c0a6860 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d62a52640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d62a52640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d62a52640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d62a52640 1 -- 172.21.15.89:0/4244899006 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5c0ac270 con 0x7f5d5c0a6860 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 1 -- 172.21.15.89:0/4244899006 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5d4c014430 con 0x7f5d5c0a6860 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d4803be00 0x7f5d4803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d58ff9640 1 -- 172.21.15.89:0/4244899006 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f5d4c00f040 con 0x7f5d5c0a6860 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d5b7fe640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d4803be00 0x7f5d4803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.854+0000 7f5d5b7fe640 10 monclient: get_auth_request con 0x7f5d4803be00 auth_method 0 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.858+0000 7f5d5b7fe640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d4803be00 0x7f5d4803e2c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c06cff0 tx=0x7f5d500a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.858+0000 7f5d58ff9640 1 -- 172.21.15.89:0/4244899006 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f5d5c0ac270 con 0x7f5d5c0a6860 2024-08-19T10:53:30.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.858+0000 7f5d58ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:30.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:30.858+0000 7f5d58ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:31.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:53:31.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:31.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:31.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 -- 172.21.15.89:0/4244899006 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5d5c0a6c40 con 0x7f5d5c0a6860 2024-08-19T10:53:31.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d58ff9640 1 -- 172.21.15.89:0/4244899006 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7f5d4c022020 con 0x7f5d5c0a6860 2024-08-19T10:53:31.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d58ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:31.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d58ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:31.096 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":124,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:53:31.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 -- 172.21.15.89:0/4244899006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d4803be00 msgr2=0x7f5d4803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d4803be00 0x7f5d4803e2c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c06cff0 tx=0x7f5d500a6000 comp rx=0 tx=0).stop 2024-08-19T10:53:31.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 10 monclient: shutdown 2024-08-19T10:53:31.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:31.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 -- 172.21.15.89:0/4244899006 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a6860 msgr2=0x7f5d5c1d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a6860 0x7f5d5c1d1580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00d800 tx=0x7f5d4c00dcc0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 -- 172.21.15.89:0/4244899006 shutdown_connections 2024-08-19T10:53:31.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5d4803be00 0x7f5d4803e2c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5d5c139620 0x7f5d5c0a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f5d5c0a7180 0x7f5d5c0a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 --2- 172.21.15.89:0/4244899006 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5d5c0a6860 0x7f5d5c1d1580 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 -- 172.21.15.89:0/4244899006 >> 172.21.15.89:0/4244899006 conn(0x7f5d5c074b40 msgr2=0x7f5d5c0ab310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:31.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 -- 172.21.15.89:0/4244899006 shutdown_connections 2024-08-19T10:53:31.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.098+0000 7f5d62a52640 1 -- 172.21.15.89:0/4244899006 wait complete. 2024-08-19T10:53:31.113 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:53:31.114 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-08-19T10:53:31.114 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:53:31.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:53:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.238+0000 7f2e2750e640 10 monclient: get_monmap_and_config 2024-08-19T10:53:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.238+0000 7f2e2750e640 10 monclient: build_initial_monmap 2024-08-19T10:53:31.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient: monmap: 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:31.244927+0000 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:31.244927+0000 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:31.239 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:31.240 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:31.240 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 1 Processor -- start 2024-08-19T10:53:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 1 -- start start 2024-08-19T10:53:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient: init 2024-08-19T10:53:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient: _add_conns ranks=[10,3,6,8,2,7,4,9,5,1,0] 2024-08-19T10:53:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2e200a5270 0x7f2e200a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient(hunting): picked mon.noname-f con 0x7f2e200a5270 addr v2:172.21.15.190:3300/0 2024-08-19T10:53:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e200a6d30 0x7f2e200a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient(hunting): picked mon.noname-g con 0x7f2e200a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:31.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e201395b0 0x7f2e20139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient(hunting): picked mon.noname-d con 0x7f2e201395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f2e200a7650 con 0x7f2e200a5270 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2e200a8710 con 0x7f2e200a6d30 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f2e200a5a40 con 0x7f2e201395b0 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e2750e640 10 monclient(hunting): authenticate will time out at 1861.061512s 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e201395b0 0x7f2e20139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e201395b0 0x7f2e20139a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46638/0 (socket says 172.21.15.89:46638) 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e252aa640 1 --2- 172.21.15.89:0/151025960 >> v2:172.21.15.190:3300/0 conn(0x7f2e200a5270 0x7f2e200a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 1 -- 172.21.15.89:0/151025960 learned_addr learned my addr 172.21.15.89:0/151025960 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 10 monclient(hunting): get_auth_request con 0x7f2e201395b0 auth_method 0 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e252aa640 10 monclient(hunting): get_auth_request con 0x7f2e200a5270 auth_method 0 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e252aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e252aa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e252aa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e24aa9640 1 --2- 172.21.15.89:0/151025960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e200a6d30 0x7f2e200a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e24aa9640 10 monclient(hunting): get_auth_request con 0x7f2e200a6d30 auth_method 0 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e24aa9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e24aa9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e24aa9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e252aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e252aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e252aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 10 monclient(hunting): handle_auth_done global_id 45141 payload 274 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 1 -- 172.21.15.89:0/151025960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e200a6d30 msgr2=0x7f2e200a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 1 --2- 172.21.15.89:0/151025960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e200a6d30 0x7f2e200a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 1 -- 172.21.15.89:0/151025960 >> v2:172.21.15.190:3300/0 conn(0x7f2e200a5270 msgr2=0x7f2e200a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 1 --2- 172.21.15.89:0/151025960 >> v2:172.21.15.190:3300/0 conn(0x7f2e200a5270 0x7f2e200a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 10 monclient: _finish_hunting 0 2024-08-19T10:53:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 1 monclient: found mon.noname-d 2024-08-19T10:53:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:31.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.242+0000 7f2e25aab640 1 -- 172.21.15.89:0/151025960 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e200a81d0 con 0x7f2e201395b0 2024-08-19T10:53:31.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e25aab640 1 -- 172.21.15.89:0/151025960 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e201395b0 msgr2=0x7f2e20139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e25aab640 1 --2- 172.21.15.89:0/151025960 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e201395b0 0x7f2e20139a40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009e50 tx=0x7f2e1801c8f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e8f28f61758017c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 1 -- 172.21.15.89:0/151025960 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e180090d0 con 0x7f2e201395b0 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 10 monclient: dump: 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.250 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 10 monclient: _finish_auth 0 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 10 monclient: _check_auth_tickets 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 1 -- 172.21.15.89:0/151025960 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e180a5ba0 con 0x7f2e201395b0 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 1 -- 172.21.15.89:0/151025960 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e180a5d50 con 0x7f2e201395b0 2024-08-19T10:53:31.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 5 monclient: authenticate success, global_id 45141 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 4 set_mon_vals no callback set 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e17fff640 10 monclient: dump: 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.252 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 10 monclient: shutdown 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 1 -- 172.21.15.89:0/151025960 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e201395b0 msgr2=0x7f2e20139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 1 --2- 172.21.15.89:0/151025960 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e201395b0 0x7f2e20139a40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009e50 tx=0x7f2e1801c8f0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 1 -- 172.21.15.89:0/151025960 shutdown_connections 2024-08-19T10:53:31.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 1 --2- 172.21.15.89:0/151025960 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e201395b0 0x7f2e20139a40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 1 --2- 172.21.15.89:0/151025960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e200a6d30 0x7f2e200a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 1 --2- 172.21.15.89:0/151025960 >> v2:172.21.15.190:3300/0 conn(0x7f2e200a5270 0x7f2e200a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 1 -- 172.21.15.89:0/151025960 >> 172.21.15.89:0/151025960 conn(0x7f2e20069a80 msgr2=0x7f2e200abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 1 -- 172.21.15.89:0/151025960 shutdown_connections 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.246+0000 7f2e2750e640 1 -- 172.21.15.89:0/151025960 wait complete. 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient: build_initial_monmap 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient: monmap: 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:31.254 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 1 Processor -- start 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 1 -- start start 2024-08-19T10:53:31.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient: init 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient: _add_conns ranks=[6,0,8,2,5,3,4,1,7] 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e200a5270 0x7f2e20134b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient(hunting): picked mon.noname-d con 0x7f2e200a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e200a6d30 0x7f2e2009c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient(hunting): picked mon.noname-a con 0x7f2e200a6d30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2e201395b0 0x7f2e2009cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient(hunting): picked mon.noname-e con 0x7f2e201395b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2e200a5650 con 0x7f2e200a6d30 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f2e200a4a70 con 0x7f2e200a5270 2024-08-19T10:53:31.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2e200a57d0 con 0x7f2e201395b0 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e2750e640 10 monclient(hunting): authenticate will time out at 1861.069358s 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e200a5270 0x7f2e20134b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e200a5270 0x7f2e20134b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46650/0 (socket says 172.21.15.89:46650) 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 1 -- 172.21.15.89:0/784188674 learned_addr learned my addr 172.21.15.89:0/784188674 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 10 monclient(hunting): get_auth_request con 0x7f2e200a5270 auth_method 0 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e24aa9640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e200a6d30 0x7f2e2009c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e25aab640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2e201395b0 0x7f2e2009cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e24aa9640 10 monclient(hunting): get_auth_request con 0x7f2e200a6d30 auth_method 0 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e24aa9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e24aa9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e24aa9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e25aab640 10 monclient(hunting): get_auth_request con 0x7f2e201395b0 auth_method 0 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e25aab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e25aab640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e25aab640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e25aab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e25aab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e25aab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 10 monclient(hunting): handle_auth_done global_id 45150 payload 824 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 1 -- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2e201395b0 msgr2=0x7f2e2009cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2e201395b0 0x7f2e2009cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 1 -- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e200a6d30 msgr2=0x7f2e2009c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e200a6d30 0x7f2e2009c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 10 monclient: _finish_hunting 0 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 1 monclient: found mon.noname-d 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:31.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e252aa640 1 -- 172.21.15.89:0/784188674 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e20135600 con 0x7f2e200a5270 2024-08-19T10:53:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e25aab640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2e201395b0 0x7f2e2009cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:31.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.250+0000 7f2e24aa9640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e200a6d30 0x7f2e2009c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:31.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e252aa640 1 -- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e200a5270 msgr2=0x7f2e20134b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e252aa640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e200a5270 0x7f2e20134b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00e870 tx=0x7f2e1c00ed30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e1c017070 con 0x7f2e200a5270 2024-08-19T10:53:31.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:31.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 10 monclient: dump: 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.267 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 10 monclient: _finish_auth 0 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1c00ee60 con 0x7f2e200a5270 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e2750e640 5 monclient: authenticate success, global_id 45150 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e2750e640 10 monclient: _renew_subs 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e2750e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e2750e640 1 -- 172.21.15.89:0/784188674 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e200a05b0 con 0x7f2e200a5270 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e1c0138e0 con 0x7f2e200a5270 2024-08-19T10:53:31.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 10 monclient: dump: 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.269 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e177fe640 4 set_mon_vals no callback set 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e2750e640 10 monclient: _renew_subs 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e2750e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e2750e640 1 -- 172.21.15.89:0/784188674 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e200a02d0 con 0x7f2e200a5270 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e2750e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e2750e640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e201350c0 0x7f2e2009d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e2750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e2750e640 1 -- 172.21.15.89:0/784188674 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2e200ac010 con 0x7f2e201350c0 2024-08-19T10:53:31.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e201350c0 0x7f2e2009d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 10 monclient: get_auth_request con 0x7f2e201350c0 auth_method 0 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2e1c01a070 con 0x7f2e200a5270 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_done global_id 7362 payload 824 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 1 -- 172.21.15.89:0/784188674 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e200a7110 con 0x7f2e201350c0 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2e0403be00 0x7f2e0403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f2e1c00f040 con 0x7f2e200a5270 2024-08-19T10:53:31.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e25aab640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2e0403be00 0x7f2e0403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e25aab640 10 monclient: get_auth_request con 0x7f2e0403be00 auth_method 0 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 1 -- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e201350c0 msgr2=0x7f2e2009d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.254+0000 7f2e24aa9640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e201350c0 0x7f2e2009d460 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2e2009e8c0 tx=0x7f2e1000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e10014ac0 con 0x7f2e201350c0 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e15ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e15ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e15ffb640 10 monclient: dump: 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.272 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2e10014650 con 0x7f2e201350c0 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e15ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:31.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e15ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e201350c0 msgr2=0x7f2e2009d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e15ffb640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e201350c0 0x7f2e2009d460 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2e2009e8c0 tx=0x7f2e1000f610 comp rx=0 tx=0).stop 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e201350c0 msgr2=0x7f2e2009d460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e15ffb640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e201350c0 0x7f2e2009d460 secure :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2e2009e8c0 tx=0x7f2e1000f610 comp rx=0 tx=0).stop 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.258+0000 7f2e25aab640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2e0403be00 0x7f2e0403e2c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2e180ab000 tx=0x7f2e180a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e2750e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e2750e640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2009d9a0 0x7f2e2013ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e2750e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e2750e640 1 -- 172.21.15.89:0/784188674 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2e20064590 con 0x7f2e2009d9a0 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e24aa9640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2009d9a0 0x7f2e2013ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e24aa9640 10 monclient: get_auth_request con 0x7f2e2009d9a0 auth_method 0 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e24aa9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e24aa9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e24aa9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e24aa9640 10 monclient(hunting): handle_auth_done global_id 7371 payload 824 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.262+0000 7f2e24aa9640 1 -- 172.21.15.89:0/784188674 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2e2009ddb0 con 0x7f2e2009d9a0 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e24aa9640 1 -- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2009d9a0 msgr2=0x7f2e2013ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e24aa9640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2009d9a0 0x7f2e2013ac20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2e2009e8c0 tx=0x7f2e100046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2e1002b020 con 0x7f2e2009d9a0 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e15ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e15ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e15ffb640 10 monclient: dump: 2024-08-19T10:53:31.275 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.276 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.277 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9604 (secure 0 0 0) 0x7f2e100133a0 con 0x7f2e2009d9a0 2024-08-19T10:53:31.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e15ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:31.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e15ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:31.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2009d9a0 msgr2=0x7f2e2013ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e15ffb640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2009d9a0 0x7f2e2013ac20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2e2009e8c0 tx=0x7f2e100046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e15ffb640 1 -- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2009d9a0 msgr2=0x7f2e2013ac20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:31.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e15ffb640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e2009d9a0 0x7f2e2013ac20 secure :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2e2009e8c0 tx=0x7f2e100046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.278 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:31.278 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:31.278 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:31.278 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:53:31.278 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:31.278 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 941079, 2024-08-19T10:53:31.278 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:31.278 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:31.278 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:31.278 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:31.278 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.279 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:31.280 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.281 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.281 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.281 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.281 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.281 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.281 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.281 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.282 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.282 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.282 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.282 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.282 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:31.282 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:31.282 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:31.283 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.284 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.285 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.286 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.287 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.288 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:31.289 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:31.290 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.292 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:31.293 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.295 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.296 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.297 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.299 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:31.300 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:31.301 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:31.302 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:31.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e2750e640 1 -- 172.21.15.89:0/784188674 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2e0403be00 msgr2=0x7f2e0403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e2750e640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2e0403be00 0x7f2e0403e2c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2e180ab000 tx=0x7f2e180a7070 comp rx=0 tx=0).stop 2024-08-19T10:53:31.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e2750e640 10 monclient: shutdown 2024-08-19T10:53:31.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e2750e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:31.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e2750e640 1 -- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e200a5270 msgr2=0x7f2e20134b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e2750e640 1 --2- 172.21.15.89:0/784188674 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2e200a5270 0x7f2e20134b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00e870 tx=0x7f2e1c00ed30 comp rx=0 tx=0).stop 2024-08-19T10:53:31.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e25aab640 1 -- 172.21.15.89:0/784188674 reap_dead start 2024-08-19T10:53:31.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e25aab640 1 -- 172.21.15.89:0/784188674 reap_dead start 2024-08-19T10:53:31.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e2750e640 1 -- 172.21.15.89:0/784188674 shutdown_connections 2024-08-19T10:53:31.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e2750e640 1 -- 172.21.15.89:0/784188674 >> 172.21.15.89:0/784188674 conn(0x7f2e20069a80 msgr2=0x7f2e200ab2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:31.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e2750e640 1 -- 172.21.15.89:0/784188674 shutdown_connections 2024-08-19T10:53:31.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.266+0000 7f2e2750e640 1 -- 172.21.15.89:0/784188674 wait complete. 2024-08-19T10:53:31.305 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:31.305 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:53:31.305 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:53:31.327 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.327 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729097593}) 2024-08-19T10:53:31.327 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1034663662}) 2024-08-19T10:53:31.327 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.327 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729097593}) 2024-08-19T10:53:31.327 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4179319051}) 2024-08-19T10:53:31.327 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.327 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729097593}) 2024-08-19T10:53:31.327 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=814725451}) 2024-08-19T10:53:31.327 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729097593}) 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=516799212}) 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729097593}) 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4101513301}) 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729097593}) 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2100846406}) 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729097593}) 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2053584486}) 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.328 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1729097593}) 2024-08-19T10:53:31.329 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2037314180}) 2024-08-19T10:53:31.330 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.330 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256553481}) 2024-08-19T10:53:31.330 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237598401}) 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256553481}) 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3220647914}) 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256553481}) 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3153734756}) 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256553481}) 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3829895562}) 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256553481}) 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2274004351}) 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.331 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256553481}) 2024-08-19T10:53:31.332 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1426824621}) 2024-08-19T10:53:31.332 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.332 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256553481}) 2024-08-19T10:53:31.332 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4042801232}) 2024-08-19T10:53:31.332 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.332 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256553481}) 2024-08-19T10:53:31.332 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.330+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3819469579}) 2024-08-19T10:53:31.333 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256887036}) 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2290061051}) 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256887036}) 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=506301146}) 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256887036}) 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4204450599}) 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256887036}) 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2189102904}) 2024-08-19T10:53:31.334 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256887036}) 2024-08-19T10:53:31.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3109561306}) 2024-08-19T10:53:31.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256887036}) 2024-08-19T10:53:31.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1870761351}) 2024-08-19T10:53:31.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256887036}) 2024-08-19T10:53:31.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=50773252}) 2024-08-19T10:53:31.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3256887036}) 2024-08-19T10:53:31.335 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.334+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2898258227}) 2024-08-19T10:53:31.337 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.337 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3007060941}) 2024-08-19T10:53:31.337 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2257318444}) 2024-08-19T10:53:31.337 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.337 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3007060941}) 2024-08-19T10:53:31.337 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1604145100}) 2024-08-19T10:53:31.337 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.337 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3007060941}) 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1443459220}) 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3007060941}) 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2436707914}) 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3007060941}) 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2853135760}) 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3007060941}) 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=416526637}) 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.338 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3007060941}) 2024-08-19T10:53:31.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3310886771}) 2024-08-19T10:53:31.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3007060941}) 2024-08-19T10:53:31.339 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.338+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1817156227}) 2024-08-19T10:53:31.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1118542575}) 2024-08-19T10:53:31.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2561150877}) 2024-08-19T10:53:31.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1118542575}) 2024-08-19T10:53:31.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=728759521}) 2024-08-19T10:53:31.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.340 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1118542575}) 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2577793546}) 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1118542575}) 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=153318853}) 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1118542575}) 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4030221733}) 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1118542575}) 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3903720418}) 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1118542575}) 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3167121768}) 2024-08-19T10:53:31.341 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1118542575}) 2024-08-19T10:53:31.342 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4131260368}) 2024-08-19T10:53:31.344 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.344 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3393197473}) 2024-08-19T10:53:31.344 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=439174388}) 2024-08-19T10:53:31.344 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.344 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3393197473}) 2024-08-19T10:53:31.344 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=434761495}) 2024-08-19T10:53:31.344 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.344 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3393197473}) 2024-08-19T10:53:31.344 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1607798250}) 2024-08-19T10:53:31.344 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3393197473}) 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3820110965}) 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3393197473}) 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=826090455}) 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3393197473}) 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4064284186}) 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3393197473}) 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2028401913}) 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.345 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3393197473}) 2024-08-19T10:53:31.346 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.342+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=994060389}) 2024-08-19T10:53:31.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784327100}) 2024-08-19T10:53:31.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4089884525}) 2024-08-19T10:53:31.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784327100}) 2024-08-19T10:53:31.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2697781141}) 2024-08-19T10:53:31.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784327100}) 2024-08-19T10:53:31.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1808565495}) 2024-08-19T10:53:31.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.348 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784327100}) 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3507416040}) 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784327100}) 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4047674595}) 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784327100}) 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1788525299}) 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784327100}) 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1658517679}) 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784327100}) 2024-08-19T10:53:31.349 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.346+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3422020773}) 2024-08-19T10:53:31.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient: get_monmap_and_config 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient: build_initial_monmap 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient: monmap: 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:31.433167+0000 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:31.433167+0000 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:31.427 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:31.428 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:31.428 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 1 Processor -- start 2024-08-19T10:53:31.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 1 -- start start 2024-08-19T10:53:31.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient: init 2024-08-19T10:53:31.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:31.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient: _add_conns ranks=[0,4,9,1,8,10,6,3,7,5,2] 2024-08-19T10:53:31.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdd140a67d0 addr v1:172.21.15.89:6789/0 2024-08-19T10:53:31.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a72c0 0x7fdd140a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient(hunting): picked mon.noname-c con 0x7fdd140a72c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fdd1413e9e0 0x7fdd14140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient(hunting): picked mon.noname-a con 0x7fdd1413e9e0 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd140a8250 con 0x7fdd140a67d0 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fdd140a8740 con 0x7fdd1413e9e0 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fdd140a3380 con 0x7fdd140a72c0 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd1b757640 10 monclient(hunting): authenticate will time out at 1861.249728s 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd194f3640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fdd140a67d0 0x7fdd140a6bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43872/0 (socket says 172.21.15.89:43872) 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd194f3640 1 -- 172.21.15.89:0/1340976482 learned_addr learned my addr 172.21.15.89:0/1340976482 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd19cf4640 1 --2- 172.21.15.89:0/1340976482 >> v2:172.21.15.89:3300/0 conn(0x7fdd1413e9e0 0x7fdd14140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 1 --2- 172.21.15.89:0/1340976482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a72c0 0x7fdd140a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd19cf4640 10 monclient(hunting): get_auth_request con 0x7fdd1413e9e0 auth_method 0 2024-08-19T10:53:31.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd19cf4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd19cf4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd19cf4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 10 monclient(hunting): get_auth_request con 0x7fdd140a72c0 auth_method 0 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_done global_id 84154 payload 274 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 1 -- 172.21.15.89:0/1340976482 >> v2:172.21.15.89:3300/0 conn(0x7fdd1413e9e0 msgr2=0x7fdd14140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 1 --2- 172.21.15.89:0/1340976482 >> v2:172.21.15.89:3300/0 conn(0x7fdd1413e9e0 0x7fdd14140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 1 -- 172.21.15.89:0/1340976482 >> v1:172.21.15.89:6789/0 conn(0x7fdd140a67d0 legacy=0x7fdd140a6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:31.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 10 monclient: _finish_hunting 0 2024-08-19T10:53:31.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 1 monclient: found mon.noname-c 2024-08-19T10:53:31.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:31.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:53:31.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.430+0000 7fdd18cf2640 1 -- 172.21.15.89:0/1340976482 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd140a4cc0 con 0x7fdd140a72c0 2024-08-19T10:53:31.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd18cf2640 1 -- 172.21.15.89:0/1340976482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a72c0 msgr2=0x7fdd140a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd18cf2640 1 --2- 172.21.15.89:0/1340976482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a72c0 0x7fdd140a5500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd040097c0 tx=0x7fdd0401c3f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=b533fef7fc8362bf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 1 -- 172.21.15.89:0/1340976482 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd040b0070 con 0x7fdd140a72c0 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 10 monclient: dump: 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.437 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 10 monclient: _finish_auth 0 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 1 -- 172.21.15.89:0/1340976482 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd040aa810 con 0x7fdd140a72c0 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 5 monclient: authenticate success, global_id 84154 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:31.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 1 -- 172.21.15.89:0/1340976482 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd040a9300 con 0x7fdd140a72c0 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 4 set_mon_vals no callback set 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd0bfff640 10 monclient: dump: 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.439 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 10 monclient: shutdown 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 1 -- 172.21.15.89:0/1340976482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a72c0 msgr2=0x7fdd140a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 1 --2- 172.21.15.89:0/1340976482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a72c0 0x7fdd140a5500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd040097c0 tx=0x7fdd0401c3f0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 1 -- 172.21.15.89:0/1340976482 shutdown_connections 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 1 --2- 172.21.15.89:0/1340976482 >> v2:172.21.15.89:3300/0 conn(0x7fdd1413e9e0 0x7fdd14140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 1 --2- 172.21.15.89:0/1340976482 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a72c0 0x7fdd140a5500 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 1 -- 172.21.15.89:0/1340976482 >> 172.21.15.89:0/1340976482 conn(0x7fdd14069ae0 msgr2=0x7fdd140abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 1 -- 172.21.15.89:0/1340976482 shutdown_connections 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.434+0000 7fdd1b757640 1 -- 172.21.15.89:0/1340976482 wait complete. 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient: build_initial_monmap 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient: monmap: 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.441 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 1 Processor -- start 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 1 -- start start 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient: init 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:31.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient: _add_conns ranks=[4,3,8,2,5,6,1,0,7] 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a67d0 0x7fdd14125930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient(hunting): picked mon.noname-c con 0x7fdd140a67d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd140a72c0 0x7fdd141d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient(hunting): picked mon.noname-g con 0x7fdd140a72c0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd1413e9e0 0x7fdd14123360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient(hunting): picked mon.noname-e con 0x7fdd1413e9e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdd14143ec0 con 0x7fdd140a72c0 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fdd14143d40 con 0x7fdd140a67d0 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fdd14144040 con 0x7fdd1413e9e0 2024-08-19T10:53:31.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd1b757640 10 monclient(hunting): authenticate will time out at 1861.257191s 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a67d0 0x7fdd14125930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a67d0 0x7fdd14125930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:52988/0 (socket says 172.21.15.89:52988) 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 1 -- 172.21.15.89:0/486107691 learned_addr learned my addr 172.21.15.89:0/486107691 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 10 monclient(hunting): get_auth_request con 0x7fdd140a67d0 auth_method 0 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd19cf4640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd1413e9e0 0x7fdd14123360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd19cf4640 10 monclient(hunting): get_auth_request con 0x7fdd1413e9e0 auth_method 0 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd19cf4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd19cf4640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd19cf4640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd19cf4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd19cf4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd19cf4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 10 monclient(hunting): handle_auth_done global_id 84163 payload 824 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 1 -- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd1413e9e0 msgr2=0x7fdd14123360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd18cf2640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd140a72c0 0x7fdd141d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd1413e9e0 0x7fdd14123360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 1 -- 172.21.15.89:0/486107691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd140a72c0 msgr2=0x7fdd141d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd140a72c0 0x7fdd141d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 10 monclient: _finish_hunting 0 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 1 monclient: found mon.noname-c 2024-08-19T10:53:31.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:31.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:53:31.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 1 -- 172.21.15.89:0/486107691 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd14126e50 con 0x7fdd140a67d0 2024-08-19T10:53:31.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 1 -- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a67d0 msgr2=0x7fdd14125930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.438+0000 7fdd194f3640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a67d0 0x7fdd14125930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000c4a0 tx=0x7fdd1000c960 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd10017070 con 0x7fdd140a67d0 2024-08-19T10:53:31.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: dump: 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.450 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: _finish_auth 0 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd1b757640 5 monclient: authenticate success, global_id 84163 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd10005d00 con 0x7fdd140a67d0 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd1b757640 10 monclient: _renew_subs 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd1b757640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd1b757640 1 -- 172.21.15.89:0/486107691 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1413dbf0 con 0x7fdd140a67d0 2024-08-19T10:53:31.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd100074e0 con 0x7fdd140a67d0 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: dump: 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd1b757640 10 monclient: _renew_subs 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd0b7fe640 4 set_mon_vals no callback set 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd1b757640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd1b757640 1 -- 172.21.15.89:0/486107691 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd140ac290 con 0x7fdd140a67d0 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd1b757640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd1b757640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd141d1b00 0x7fdd1402d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd1b757640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd1b757640 1 -- 172.21.15.89:0/486107691 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fdd141c8ed0 con 0x7fdd141d1b00 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd141d1b00 0x7fdd1402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fdd10007070 con 0x7fdd140a67d0 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 10 monclient: get_auth_request con 0x7fdd141d1b00 auth_method 0 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdcf803bd30 0x7fdcf803e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_done global_id 74333 payload 824 2024-08-19T10:53:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 1 -- 172.21.15.89:0/486107691 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd140a5500 con 0x7fdd141d1b00 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fdd1004e990 con 0x7fdd140a67d0 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd19cf4640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdcf803bd30 0x7fdcf803e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd19cf4640 10 monclient: get_auth_request con 0x7fdcf803bd30 auth_method 0 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 1 -- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd141d1b00 msgr2=0x7fdd1402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd18cf2640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd141d1b00 0x7fdd1402d1d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd040ad000 tx=0x7fdd04019190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd040b9070 con 0x7fdd141d1b00 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.442+0000 7fdd09ffb640 10 monclient: dump: 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.455 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.446+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdd04003b20 con 0x7fdd141d1b00 2024-08-19T10:53:31.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.446+0000 7fdd09ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.446+0000 7fdd09ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.446+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd141d1b00 msgr2=0x7fdd1402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.446+0000 7fdd09ffb640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd141d1b00 0x7fdd1402d1d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd040ad000 tx=0x7fdd04019190 comp rx=0 tx=0).stop 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.446+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd141d1b00 msgr2=0x7fdd1402d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.446+0000 7fdd09ffb640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd141d1b00 0x7fdd1402d1d0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd040ad000 tx=0x7fdd04019190 comp rx=0 tx=0).stop 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.446+0000 7fdd19cf4640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdcf803bd30 0x7fdcf803e1f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c003ab0 tx=0x7fdd0c009780 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd1b757640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd1b757640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd1402d710 0x7fdd1402db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd1b757640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd1b757640 1 -- 172.21.15.89:0/486107691 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fdd14064590 con 0x7fdd1402d710 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd18cf2640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd1402d710 0x7fdd1402db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd18cf2640 10 monclient: get_auth_request con 0x7fdd1402d710 auth_method 0 2024-08-19T10:53:31.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd18cf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd18cf2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd18cf2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd18cf2640 10 monclient(hunting): handle_auth_done global_id 74342 payload 824 2024-08-19T10:53:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.450+0000 7fdd18cf2640 1 -- 172.21.15.89:0/486107691 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdd141262c0 con 0x7fdd1402d710 2024-08-19T10:53:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd18cf2640 1 -- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd1402d710 msgr2=0x7fdd1402db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd18cf2640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd1402d710 0x7fdd1402db20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd040097c0 tx=0x7fdd040a8000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd040b0040 con 0x7fdd1402d710 2024-08-19T10:53:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd09ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd09ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd09ffb640 10 monclient: dump: 2024-08-19T10:53:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7fdd040c2070 con 0x7fdd1402d710 2024-08-19T10:53:31.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd09ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:31.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd09ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:31.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd1402d710 msgr2=0x7fdd1402db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd09ffb640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd1402d710 0x7fdd1402db20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd040097c0 tx=0x7fdd040a8000 comp rx=0 tx=0).stop 2024-08-19T10:53:31.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd09ffb640 1 -- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd1402d710 msgr2=0x7fdd1402db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:31.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd09ffb640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd1402d710 0x7fdd1402db20 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd040097c0 tx=0x7fdd040a8000 comp rx=0 tx=0).stop 2024-08-19T10:53:31.461 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:31.461 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:31.461 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:31.461 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 147952, 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:31.462 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.463 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.464 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:31.465 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.466 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.467 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.469 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.469 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.469 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.469 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.469 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.469 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:31.471 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.473 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:31.474 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:31.477 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.478 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.478 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.478 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:31.478 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:31.478 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.478 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.478 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.479 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:31.480 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:31.483 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:31.484 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.485 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:31.487 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:31.487 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:53:31.487 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.487 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.487 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:31.487 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:31.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd1b757640 1 -- 172.21.15.89:0/486107691 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdcf803bd30 msgr2=0x7fdcf803e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd1b757640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdcf803bd30 0x7fdcf803e1f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c003ab0 tx=0x7fdd0c009780 comp rx=0 tx=0).stop 2024-08-19T10:53:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd1b757640 10 monclient: shutdown 2024-08-19T10:53:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd1b757640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd1b757640 1 -- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a67d0 msgr2=0x7fdd14125930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd1b757640 1 --2- 172.21.15.89:0/486107691 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdd140a67d0 0x7fdd14125930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000c4a0 tx=0x7fdd1000c960 comp rx=0 tx=0).stop 2024-08-19T10:53:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd19cf4640 1 -- 172.21.15.89:0/486107691 reap_dead start 2024-08-19T10:53:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd19cf4640 1 -- 172.21.15.89:0/486107691 reap_dead start 2024-08-19T10:53:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd1b757640 1 -- 172.21.15.89:0/486107691 shutdown_connections 2024-08-19T10:53:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd1b757640 1 -- 172.21.15.89:0/486107691 >> 172.21.15.89:0/486107691 conn(0x7fdd14069ae0 msgr2=0x7fdd140aad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd1b757640 1 -- 172.21.15.89:0/486107691 shutdown_connections 2024-08-19T10:53:31.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.454+0000 7fdd1b757640 1 -- 172.21.15.89:0/486107691 wait complete. 2024-08-19T10:53:31.490 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:31.490 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:53:31.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:53:31.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325192164}) 2024-08-19T10:53:31.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=282441235}) 2024-08-19T10:53:31.623 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325192164}) 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=257556396}) 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325192164}) 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2901548387}) 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325192164}) 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3344761047}) 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325192164}) 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2950427763}) 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.624 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325192164}) 2024-08-19T10:53:31.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1721024053}) 2024-08-19T10:53:31.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325192164}) 2024-08-19T10:53:31.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3631972896}) 2024-08-19T10:53:31.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325192164}) 2024-08-19T10:53:31.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4093794262}) 2024-08-19T10:53:31.638 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.638 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125677632}) 2024-08-19T10:53:31.638 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1255963073}) 2024-08-19T10:53:31.638 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.634+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.638 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125677632}) 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3498941320}) 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125677632}) 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2376021379}) 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125677632}) 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=577341750}) 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125677632}) 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1859936865}) 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125677632}) 2024-08-19T10:53:31.639 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4250032118}) 2024-08-19T10:53:31.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125677632}) 2024-08-19T10:53:31.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2251382180}) 2024-08-19T10:53:31.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=125677632}) 2024-08-19T10:53:31.640 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.638+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3487489764}) 2024-08-19T10:53:31.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=54036541}) 2024-08-19T10:53:31.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3121271209}) 2024-08-19T10:53:31.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=54036541}) 2024-08-19T10:53:31.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4166340300}) 2024-08-19T10:53:31.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.657 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=54036541}) 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3433668828}) 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=54036541}) 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3897116455}) 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=54036541}) 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3437840572}) 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=54036541}) 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=59434978}) 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=54036541}) 2024-08-19T10:53:31.658 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2569416969}) 2024-08-19T10:53:31.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=54036541}) 2024-08-19T10:53:31.659 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.658+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1902162129}) 2024-08-19T10:53:31.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.682+0000 7f576325f640 10 monclient: get_monmap_and_config 2024-08-19T10:53:31.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.682+0000 7f576325f640 10 monclient: build_initial_monmap 2024-08-19T10:53:31.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.682+0000 7f576325f640 10 monclient: monmap: 2024-08-19T10:53:31.682 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:31.682 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.682 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:31.688209+0000 2024-08-19T10:53:31.682 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:31.688209+0000 2024-08-19T10:53:31.682 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:31.682 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.682 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:31.683 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:31.683 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:31.683 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:31.683 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:31.683 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:31.683 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:31.683 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:31.683 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:31.683 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:31.683 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:31.684 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 1 Processor -- start 2024-08-19T10:53:31.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 1 -- start start 2024-08-19T10:53:31.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 10 monclient: init 2024-08-19T10:53:31.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:31.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 10 monclient: _add_conns ranks=[8,6,1,3,4,5,7,9,10,2,0] 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f575c06c930 0x7f575c09d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 10 monclient(hunting): picked mon.noname-e con 0x7f575c06c930 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 0x7f575c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 10 monclient(hunting): picked mon.noname-d con 0x7f575c06cd10 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f575c137650 addr v1:172.21.15.190:6789/0 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f575c0a31b0 con 0x7f575c137650 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f575c0a3ae0 con 0x7f575c06cd10 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f575c0a3ce0 con 0x7f575c06c930 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f576325f640 10 monclient(hunting): authenticate will time out at 1861.504866s 2024-08-19T10:53:31.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 0x7f575c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 0x7f575c06d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46666/0 (socket says 172.21.15.89:46666) 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 1 -- 172.21.15.89:0/833956994 learned_addr learned my addr 172.21.15.89:0/833956994 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 10 monclient(hunting): get_auth_request con 0x7f575c06cd10 auth_method 0 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f5760ffb640 1 --2- 172.21.15.89:0/833956994 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f575c06c930 0x7f575c09d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f5760ffb640 10 monclient(hunting): get_auth_request con 0x7f575c06c930 auth_method 0 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f5760ffb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f5760ffb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f5760ffb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 10 monclient(hunting): handle_auth_done global_id 45159 payload 274 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 1 -- 172.21.15.89:0/833956994 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f575c06c930 msgr2=0x7f575c09d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 1 --2- 172.21.15.89:0/833956994 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f575c06c930 0x7f575c09d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 1 -- 172.21.15.89:0/833956994 >> v1:172.21.15.190:6789/0 conn(0x7f575c137650 legacy=0x7f575c137a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 1 monclient: found mon.noname-d 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:31.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.686+0000 7f575bfff640 1 -- 172.21.15.89:0/833956994 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c0a3f60 con 0x7f575c06cd10 2024-08-19T10:53:31.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575bfff640 1 -- 172.21.15.89:0/833956994 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 msgr2=0x7f575c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575bfff640 1 --2- 172.21.15.89:0/833956994 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 0x7f575c06d0f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f57480098a0 tx=0x7f574801c440 comp rx=0 tx=0).ready entity=mon.6 client_cookie=92782f7cd70d12eb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 1 -- 172.21.15.89:0/833956994 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f57480b0070 con 0x7f575c06cd10 2024-08-19T10:53:31.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:31.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 10 monclient: dump: 2024-08-19T10:53:31.693 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.693 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.693 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.694 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 1 -- 172.21.15.89:0/833956994 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57480aa810 con 0x7f575c06cd10 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 5 monclient: authenticate success, global_id 45159 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 1 -- 172.21.15.89:0/833956994 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f57480a9300 con 0x7f575c06cd10 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f575b7fe640 10 monclient: dump: 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.695 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.696 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 4 set_mon_vals no callback set 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 10 monclient: shutdown 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 1 -- 172.21.15.89:0/833956994 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 msgr2=0x7f575c06d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 1 --2- 172.21.15.89:0/833956994 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 0x7f575c06d0f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f57480098a0 tx=0x7f574801c440 comp rx=0 tx=0).stop 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 1 -- 172.21.15.89:0/833956994 shutdown_connections 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 1 --2- 172.21.15.89:0/833956994 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 0x7f575c06d0f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 1 --2- 172.21.15.89:0/833956994 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f575c06c930 0x7f575c09d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 1 -- 172.21.15.89:0/833956994 >> 172.21.15.89:0/833956994 conn(0x7f575c06de40 msgr2=0x7f575c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 1 -- 172.21.15.89:0/833956994 shutdown_connections 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 1 -- 172.21.15.89:0/833956994 wait complete. 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 10 monclient: build_initial_monmap 2024-08-19T10:53:31.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.690+0000 7f576325f640 10 monclient: monmap: 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:31.698 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 1 Processor -- start 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 1 -- start start 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 10 monclient: init 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 10 monclient: _add_conns ranks=[0,6,2,1,8,4,3,7,5] 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f575c06c930 0x7f575c1360c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 10 monclient(hunting): picked mon.noname-a con 0x7f575c06c930 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 0x7f575c136600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 10 monclient(hunting): picked mon.noname-d con 0x7f575c06cd10 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:31.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f575c137650 0x7f575c0a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 10 monclient(hunting): picked mon.noname-b con 0x7f575c137650 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1424d0 con 0x7f575c06c930 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f575c142350 con 0x7f575c137650 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f575c142650 con 0x7f575c06cd10 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f576325f640 10 monclient(hunting): authenticate will time out at 1861.513215s 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 0x7f575c136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f5760ffb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f575c06c930 0x7f575c1360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f5760ffb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f575c06c930 0x7f575c1360c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52990/0 (socket says 172.21.15.89:52990) 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 0x7f575c136600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46670/0 (socket says 172.21.15.89:46670) 2024-08-19T10:53:31.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f5760ffb640 1 -- 172.21.15.89:0/746730515 learned_addr learned my addr 172.21.15.89:0/746730515 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f5760ffb640 10 monclient(hunting): get_auth_request con 0x7f575c06c930 auth_method 0 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f5760ffb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f5760ffb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f5760ffb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 10 monclient(hunting): get_auth_request con 0x7f575c06cd10 auth_method 0 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f57617fc640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f575c137650 0x7f575c0a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f57617fc640 10 monclient(hunting): get_auth_request con 0x7f575c137650 auth_method 0 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f57617fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f57617fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f57617fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f57617fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f57617fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f57617fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 10 monclient(hunting): handle_auth_done global_id 45168 payload 824 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 1 -- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f575c137650 msgr2=0x7f575c0a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f575c137650 0x7f575c0a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 1 -- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f575c06c930 msgr2=0x7f575c1360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f575c06c930 0x7f575c1360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 1 monclient: found mon.noname-d 2024-08-19T10:53:31.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:31.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:31.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f575bfff640 1 -- 172.21.15.89:0/746730515 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c1371f0 con 0x7f575c06cd10 2024-08-19T10:53:31.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f57617fc640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f575c137650 0x7f575c0a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:31.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.694+0000 7f5760ffb640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f575c06c930 0x7f575c1360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:31.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f575bfff640 1 -- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 msgr2=0x7f575c136600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f575bfff640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 0x7f575c136600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f57480ab000 tx=0x7f57480191e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f57480b0040 con 0x7f575c06cd10 2024-08-19T10:53:31.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:31.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 10 monclient: dump: 2024-08-19T10:53:31.707 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.707 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.707 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.707 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.707 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.707 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.707 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 10 monclient: _finish_auth 0 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 10 monclient: _check_auth_tickets 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f576325f640 5 monclient: authenticate success, global_id 45168 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f576325f640 10 monclient: _renew_subs 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5748004040 con 0x7f575c06cd10 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f576325f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f576325f640 1 -- 172.21.15.89:0/746730515 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575c13eed0 con 0x7f575c06cd10 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5748003620 con 0x7f575c06cd10 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 10 monclient: dump: 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.709 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f575affd640 4 set_mon_vals no callback set 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f576325f640 10 monclient: _renew_subs 2024-08-19T10:53:31.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f576325f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f576325f640 1 -- 172.21.15.89:0/746730515 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f575c13e910 con 0x7f575c06cd10 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f576325f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f576325f640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c063b30 0x7f575c063f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f576325f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f576325f640 1 -- 172.21.15.89:0/746730515 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f575c09eda0 con 0x7f575c063b30 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c063b30 0x7f575c063f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 10 monclient: get_auth_request con 0x7f575c063b30 auth_method 0 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f57480bb060 con 0x7f575c06cd10 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 10 monclient(hunting): handle_auth_done global_id 84172 payload 824 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 1 -- 172.21.15.89:0/746730515 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f575c09d4f0 con 0x7f575c063b30 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f574003be00 0x7f574003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f57480e8450 con 0x7f575c06cd10 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57617fc640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f574003be00 0x7f574003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f57617fc640 10 monclient: get_auth_request con 0x7f574003be00 auth_method 0 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 1 -- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c063b30 msgr2=0x7f575c063f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.698+0000 7f5760ffb640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c063b30 0x7f575c063f10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5750009ec0 tx=0x7f575000f610 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5750014ac0 con 0x7f575c063b30 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57597fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57597fa640 10 monclient: dump: 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.712 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5750014650 con 0x7f575c063b30 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57597fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57597fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c063b30 msgr2=0x7f575c063f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57597fa640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c063b30 0x7f575c063f10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5750009ec0 tx=0x7f575000f610 comp rx=0 tx=0).stop 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c063b30 msgr2=0x7f575c063f10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57597fa640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c063b30 0x7f575c063f10 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5750009ec0 tx=0x7f575000f610 comp rx=0 tx=0).stop 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.702+0000 7f57617fc640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f574003be00 0x7f574003e2c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5754009fd0 tx=0x7f575400d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f576325f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f576325f640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c02d1d0 0x7f575c02d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f576325f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f576325f640 1 -- 172.21.15.89:0/746730515 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f575c064590 con 0x7f575c02d1d0 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f5760ffb640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c02d1d0 0x7f575c02d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f5760ffb640 10 monclient: get_auth_request con 0x7f575c02d1d0 auth_method 0 2024-08-19T10:53:31.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f5760ffb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f5760ffb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f5760ffb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f5760ffb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f5760ffb640 10 monclient(hunting): handle_auth_done global_id 84181 payload 824 2024-08-19T10:53:31.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.706+0000 7f5760ffb640 1 -- 172.21.15.89:0/746730515 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f575c06e3e0 con 0x7f575c02d1d0 2024-08-19T10:53:31.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f5760ffb640 1 -- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c02d1d0 msgr2=0x7f575c02d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f5760ffb640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c02d1d0 0x7f575c02d5b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5750010010 tx=0x7f57500046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5750004490 con 0x7f575c02d1d0 2024-08-19T10:53:31.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57597fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57597fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57597fa640 10 monclient: dump: 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.717 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.718 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.718 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.718 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.718 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f575001dc80 con 0x7f575c02d1d0 2024-08-19T10:53:31.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57597fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:31.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57597fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:31.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c02d1d0 msgr2=0x7f575c02d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57597fa640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c02d1d0 0x7f575c02d5b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5750010010 tx=0x7f57500046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57597fa640 1 -- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c02d1d0 msgr2=0x7f575c02d5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:31.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57597fa640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f575c02d1d0 0x7f575c02d5b0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5750010010 tx=0x7f57500046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 20011, 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:31.719 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:53:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.720 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:31.721 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.722 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:31.723 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.724 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:31.725 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.726 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.727 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:31.728 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.729 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.730 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:31.731 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.732 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.733 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.734 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.735 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:31.736 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:31.739 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:31.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:31.741 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:31.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f576325f640 1 -- 172.21.15.89:0/746730515 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f574003be00 msgr2=0x7f574003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f576325f640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f574003be00 0x7f574003e2c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5754009fd0 tx=0x7f575400d040 comp rx=0 tx=0).stop 2024-08-19T10:53:31.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f576325f640 10 monclient: shutdown 2024-08-19T10:53:31.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f576325f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:31.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f576325f640 1 -- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 msgr2=0x7f575c136600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f576325f640 1 --2- 172.21.15.89:0/746730515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f575c06cd10 0x7f575c136600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f57480ab000 tx=0x7f57480191e0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57617fc640 1 -- 172.21.15.89:0/746730515 reap_dead start 2024-08-19T10:53:31.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f57617fc640 1 -- 172.21.15.89:0/746730515 reap_dead start 2024-08-19T10:53:31.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f576325f640 1 -- 172.21.15.89:0/746730515 shutdown_connections 2024-08-19T10:53:31.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f576325f640 1 -- 172.21.15.89:0/746730515 >> 172.21.15.89:0/746730515 conn(0x7f575c06de40 msgr2=0x7f575c0a76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:31.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f576325f640 1 -- 172.21.15.89:0/746730515 shutdown_connections 2024-08-19T10:53:31.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.710+0000 7f576325f640 1 -- 172.21.15.89:0/746730515 wait complete. 2024-08-19T10:53:31.744 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:31.744 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:53:31.744 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:53:31.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255280987}) 2024-08-19T10:53:31.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3640423990}) 2024-08-19T10:53:31.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255280987}) 2024-08-19T10:53:31.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1116905101}) 2024-08-19T10:53:31.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255280987}) 2024-08-19T10:53:31.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1249277111}) 2024-08-19T10:53:31.747 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255280987}) 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.738+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3356957784}) 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255280987}) 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4261097648}) 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255280987}) 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2904431977}) 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255280987}) 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=29331135}) 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2255280987}) 2024-08-19T10:53:31.748 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.742+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=821084263}) 2024-08-19T10:53:31.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2780970624}) 2024-08-19T10:53:31.752 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=602043545}) 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2780970624}) 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1255279459}) 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2780970624}) 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1586589988}) 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2780970624}) 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1808370050}) 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2780970624}) 2024-08-19T10:53:31.753 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1619635478}) 2024-08-19T10:53:31.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2780970624}) 2024-08-19T10:53:31.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3893589181}) 2024-08-19T10:53:31.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2780970624}) 2024-08-19T10:53:31.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3803213561}) 2024-08-19T10:53:31.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2780970624}) 2024-08-19T10:53:31.754 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.754+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3159584074}) 2024-08-19T10:53:31.792 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520854741}) 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3351040767}) 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520854741}) 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3093780278}) 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520854741}) 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2513675921}) 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520854741}) 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=298570900}) 2024-08-19T10:53:31.793 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520854741}) 2024-08-19T10:53:31.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=286044153}) 2024-08-19T10:53:31.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520854741}) 2024-08-19T10:53:31.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3610519180}) 2024-08-19T10:53:31.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520854741}) 2024-08-19T10:53:31.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=7020828}) 2024-08-19T10:53:31.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:31.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1520854741}) 2024-08-19T10:53:31.794 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:31.794+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2158641330}) 2024-08-19T10:53:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.890+0000 7f576e0dc640 10 monclient: get_monmap_and_config 2024-08-19T10:53:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.890+0000 7f576e0dc640 10 monclient: build_initial_monmap 2024-08-19T10:53:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.890+0000 7f576e0dc640 10 monclient: monmap: 2024-08-19T10:53:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:31.896596+0000 2024-08-19T10:53:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:31.896596+0000 2024-08-19T10:53:31.890 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:31.891 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 1 Processor -- start 2024-08-19T10:53:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 1 -- start start 2024-08-19T10:53:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 10 monclient: init 2024-08-19T10:53:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 10 monclient: _add_conns ranks=[10,8,2,1,9,7,3,5,6,0,4] 2024-08-19T10:53:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f57680a6860 0x7f57680a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 10 monclient(hunting): picked mon.noname-f con 0x7f57680a6860 addr v2:172.21.15.190:3300/0 2024-08-19T10:53:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f57680a7180 0x7f57680a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 10 monclient(hunting): picked mon.noname-e con 0x7f57680a7180 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:31.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5768139620 0x7f5768139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 10 monclient(hunting): picked mon.noname-b con 0x7f5768139620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f57680a8190 con 0x7f57680a6860 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f57680a33d0 con 0x7f5768139620 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f57680a2ec0 con 0x7f57680a7180 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f576e0dc640 10 monclient(hunting): authenticate will time out at 1861.712968s 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f57680a7180 0x7f57680a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f57680a7180 0x7f57680a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:52932/0 (socket says 172.21.15.89:52932) 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5767fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5768139620 0x7f5768139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 1 -- 172.21.15.89:0/3943307578 learned_addr learned my addr 172.21.15.89:0/3943307578 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 10 monclient(hunting): get_auth_request con 0x7f57680a7180 auth_method 0 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5767fff640 10 monclient(hunting): get_auth_request con 0x7f5768139620 auth_method 0 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5767fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5767fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5767fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5767fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5767fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5767fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f57677fe640 1 --2- 172.21.15.89:0/3943307578 >> v2:172.21.15.190:3300/0 conn(0x7f57680a6860 0x7f57680a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 10 monclient(hunting): handle_auth_done global_id 84185 payload 274 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 1 -- 172.21.15.89:0/3943307578 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5768139620 msgr2=0x7f5768139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 1 --2- 172.21.15.89:0/3943307578 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5768139620 0x7f5768139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 1 -- 172.21.15.89:0/3943307578 >> v2:172.21.15.190:3300/0 conn(0x7f57680a6860 msgr2=0x7f57680a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 1 --2- 172.21.15.89:0/3943307578 >> v2:172.21.15.190:3300/0 conn(0x7f57680a6860 0x7f57680a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 1 monclient: found mon.noname-e 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f5766ffd640 1 -- 172.21.15.89:0/3943307578 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57680a4cd0 con 0x7f57680a7180 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f57677fe640 10 monclient: get_auth_request con 0x7f57680a6860 auth_method 0 2024-08-19T10:53:31.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.894+0000 7f57677fe640 1 --2- 172.21.15.89:0/3943307578 >> v2:172.21.15.190:3300/0 conn(0x7f57680a6860 0x7f57680a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:53:31.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f5766ffd640 1 -- 172.21.15.89:0/3943307578 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f57680a7180 msgr2=0x7f57680a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f5766ffd640 1 --2- 172.21.15.89:0/3943307578 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f57680a7180 0x7f57680a7560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f575c009870 tx=0x7f575c01c420 comp rx=0 tx=0).ready entity=mon.8 client_cookie=7c6267dccf99e4b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 1 -- 172.21.15.89:0/3943307578 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f575c0b0070 con 0x7f57680a7180 2024-08-19T10:53:31.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 10 monclient: dump: 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.904 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 10 monclient: _finish_auth 0 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 10 monclient: _check_auth_tickets 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 1 -- 172.21.15.89:0/3943307578 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f575c0aa810 con 0x7f57680a7180 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 5 monclient: authenticate success, global_id 84185 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 1 -- 172.21.15.89:0/3943307578 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f575c0aae30 con 0x7f57680a7180 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 4 set_mon_vals no callback set 2024-08-19T10:53:31.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f57667fc640 10 monclient: dump: 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.906 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 10 monclient: shutdown 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 1 -- 172.21.15.89:0/3943307578 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f57680a7180 msgr2=0x7f57680a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 1 --2- 172.21.15.89:0/3943307578 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f57680a7180 0x7f57680a7560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f575c009870 tx=0x7f575c01c420 comp rx=0 tx=0).stop 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 1 -- 172.21.15.89:0/3943307578 shutdown_connections 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 1 --2- 172.21.15.89:0/3943307578 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f5768139620 0x7f5768139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 1 --2- 172.21.15.89:0/3943307578 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f57680a7180 0x7f57680a7560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 1 --2- 172.21.15.89:0/3943307578 >> v2:172.21.15.190:3300/0 conn(0x7f57680a6860 0x7f57680a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 1 -- 172.21.15.89:0/3943307578 >> 172.21.15.89:0/3943307578 conn(0x7f5768074b40 msgr2=0x7f57680abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 1 -- 172.21.15.89:0/3943307578 shutdown_connections 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 1 -- 172.21.15.89:0/3943307578 wait complete. 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 10 monclient: build_initial_monmap 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 10 monclient: monmap: 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:31.908 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 1 Processor -- start 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 1 -- start start 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.898+0000 7f576e0dc640 10 monclient: init 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient: _add_conns ranks=[3,6,8,2,1,4,5,0,7] 2024-08-19T10:53:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57680a6860 0x7f57681d1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient(hunting): picked mon.noname-g con 0x7f57680a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57680a7180 0x7f57680a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient(hunting): picked mon.noname-d con 0x7f57680a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5768139620 0x7f57680a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient(hunting): picked mon.noname-e con 0x7f5768139620 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5768143210 con 0x7f57680a6860 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f5768143090 con 0x7f57680a7180 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f5768143390 con 0x7f5768139620 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:31.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient(hunting): authenticate will time out at 1861.720283s 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5766ffd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57680a7180 0x7f57680a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5768139620 0x7f57680a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5766ffd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57680a7180 0x7f57680a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46684/0 (socket says 172.21.15.89:46684) 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5768139620 0x7f57680a2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:52948/0 (socket says 172.21.15.89:52948) 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5766ffd640 1 -- 172.21.15.89:0/1502312708 learned_addr learned my addr 172.21.15.89:0/1502312708 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5766ffd640 10 monclient(hunting): get_auth_request con 0x7f57680a7180 auth_method 0 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5766ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5766ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5766ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 10 monclient(hunting): get_auth_request con 0x7f5768139620 auth_method 0 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5766ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5766ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5766ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f57677fe640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57680a6860 0x7f57681d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 10 monclient(hunting): handle_auth_done global_id 84194 payload 824 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 1 -- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57680a7180 msgr2=0x7f57680a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f57680a7180 0x7f57680a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 1 -- 172.21.15.89:0/1502312708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57680a6860 msgr2=0x7f57681d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57680a6860 0x7f57681d1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 10 monclient: _finish_hunting 0 2024-08-19T10:53:31.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 1 monclient: found mon.noname-e 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 1 -- 172.21.15.89:0/1502312708 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5768135700 con 0x7f5768139620 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f57677fe640 10 monclient: get_auth_request con 0x7f57680a6860 auth_method 0 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f57677fe640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57680a6860 0x7f57681d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 1 -- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5768139620 msgr2=0x7f57680a2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5767fff640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5768139620 0x7f57680a2230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f576000e7f0 tx=0x7f576000ecb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5760017070 con 0x7f5768139620 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 10 monclient: dump: 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.913 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.914 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 10 monclient: _finish_auth 0 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f576000cdb0 con 0x7f5768139620 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f57600138e0 con 0x7f5768139620 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 5 monclient: authenticate success, global_id 84194 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient: _renew_subs 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 1 -- 172.21.15.89:0/1502312708 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57680ac110 con 0x7f5768139620 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:53:31.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f5765ffb640 4 set_mon_vals no callback set 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f574ffff640 10 monclient: dump: 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.916 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient: _renew_subs 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 1 -- 172.21.15.89:0/1502312708 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f57681d18d0 con 0x7f5768139620 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809cc00 0x7f576809cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.902+0000 7f576e0dc640 1 -- 172.21.15.89:0/1502312708 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f57680a0160 con 0x7f576809cc00 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809cc00 0x7f576809cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 10 monclient: get_auth_request con 0x7f576809cc00 auth_method 0 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f576001a070 con 0x7f5768139620 2024-08-19T10:53:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 10 monclient(hunting): handle_auth_done global_id 45186 payload 824 2024-08-19T10:53:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 1 -- 172.21.15.89:0/1502312708 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57680a3d50 con 0x7f576809cc00 2024-08-19T10:53:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f575403be00 0x7f575403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f576000f040 con 0x7f5768139620 2024-08-19T10:53:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f5766ffd640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f575403be00 0x7f575403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f5766ffd640 10 monclient: get_auth_request con 0x7f575403be00 auth_method 0 2024-08-19T10:53:31.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 1 -- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809cc00 msgr2=0x7f576809cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f57677fe640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809cc00 0x7f576809cfe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5758007f30 tx=0x7f575800b840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f575801a070 con 0x7f576809cc00 2024-08-19T10:53:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 10 monclient: dump: 2024-08-19T10:53:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.921 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5758008800 con 0x7f576809cc00 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:31.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809cc00 msgr2=0x7f576809cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809cc00 0x7f576809cfe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5758007f30 tx=0x7f575800b840 comp rx=0 tx=0).stop 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809cc00 msgr2=0x7f576809cfe0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f574ffff640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809cc00 0x7f576809cfe0 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5758007f30 tx=0x7f575800b840 comp rx=0 tx=0).stop 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.906+0000 7f5766ffd640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f575403be00 0x7f575403e2c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f575c01ccc0 tx=0x7f575c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f576e0dc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f576e0dc640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809d520 0x7f576813ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f576e0dc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f576e0dc640 1 -- 172.21.15.89:0/1502312708 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f57680a6c40 con 0x7f576809d520 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809d520 0x7f576813ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 10 monclient: get_auth_request con 0x7f576809d520 auth_method 0 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:31.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 10 monclient(hunting): handle_auth_done global_id 45195 payload 824 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 1 -- 172.21.15.89:0/1502312708 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5768064590 con 0x7f576809d520 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 1 -- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809d520 msgr2=0x7f576813ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f57677fe640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809d520 0x7f576813ab20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f575800f010 tx=0x7f5758004a50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f575801a040 con 0x7f576809d520 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.914+0000 7f574ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f574ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f574ffff640 10 monclient: dump: 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:31.924 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:31.925 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:31.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f5758015920 con 0x7f576809d520 2024-08-19T10:53:31.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f574ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:31.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f574ffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:31.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809d520 msgr2=0x7f576813ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f574ffff640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809d520 0x7f576813ab20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f575800f010 tx=0x7f5758004a50 comp rx=0 tx=0).stop 2024-08-19T10:53:31.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f574ffff640 1 -- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809d520 msgr2=0x7f576813ab20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:31.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f574ffff640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f576809d520 0x7f576813ab20 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f575800f010 tx=0x7f5758004a50 comp rx=0 tx=0).stop 2024-08-19T10:53:31.926 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:31.926 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 535044, 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:31.927 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.928 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.929 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:31.930 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:31.931 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:31.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:31.933 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:31.934 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.936 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:31.937 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.941 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.942 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.943 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.944 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.945 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.946 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:31.947 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.948 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:31.949 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:31.950 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:31.950 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:31.950 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:31.950 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:31.950 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:31.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:31.950 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:31.950 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:31.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f576e0dc640 1 -- 172.21.15.89:0/1502312708 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f575403be00 msgr2=0x7f575403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f576e0dc640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f575403be00 0x7f575403e2c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f575c01ccc0 tx=0x7f575c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:53:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f576e0dc640 10 monclient: shutdown 2024-08-19T10:53:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f576e0dc640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f576e0dc640 1 -- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5768139620 msgr2=0x7f57680a2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f576e0dc640 1 --2- 172.21.15.89:0/1502312708 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5768139620 0x7f57680a2230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f576000e7f0 tx=0x7f576000ecb0 comp rx=0 tx=0).stop 2024-08-19T10:53:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f5767fff640 1 -- 172.21.15.89:0/1502312708 reap_dead start 2024-08-19T10:53:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f5767fff640 1 -- 172.21.15.89:0/1502312708 reap_dead start 2024-08-19T10:53:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f576e0dc640 1 -- 172.21.15.89:0/1502312708 shutdown_connections 2024-08-19T10:53:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f576e0dc640 1 -- 172.21.15.89:0/1502312708 >> 172.21.15.89:0/1502312708 conn(0x7f5768074b40 msgr2=0x7f57680ab520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f576e0dc640 1 -- 172.21.15.89:0/1502312708 shutdown_connections 2024-08-19T10:53:31.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:31.918+0000 7f576e0dc640 1 -- 172.21.15.89:0/1502312708 wait complete. 2024-08-19T10:53:31.953 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:31.953 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:53:31.953 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:53:32.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient: get_monmap_and_config 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient: build_initial_monmap 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient: monmap: 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:32.125924+0000 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:32.125924+0000 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:32.120 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:32.121 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:32.121 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:32.121 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:32.121 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:32.121 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:32.121 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:32.121 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 1 Processor -- start 2024-08-19T10:53:32.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 1 -- start start 2024-08-19T10:53:32.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient: init 2024-08-19T10:53:32.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:32.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient: _add_conns ranks=[10,7,6,5,1,0,8,3,2,9,4] 2024-08-19T10:53:32.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f92700a5270 0x7f92700a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient(hunting): picked mon.noname-f con 0x7f92700a5270 addr v2:172.21.15.190:3300/0 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f92700a6d30 0x7f92700a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient(hunting): picked mon.noname-i con 0x7f92700a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92701395b0 0x7f9270139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient(hunting): picked mon.noname-d con 0x7f92701395b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f92700a7650 con 0x7f92700a5270 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f92700a8710 con 0x7f92701395b0 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f92700a5a40 con 0x7f92700a6d30 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f92783ba640 10 monclient(hunting): authenticate will time out at 1861.942465s 2024-08-19T10:53:32.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92701395b0 0x7f9270139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92701395b0 0x7f9270139a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46698/0 (socket says 172.21.15.89:46698) 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 1 -- 172.21.15.89:0/1230465776 learned_addr learned my addr 172.21.15.89:0/1230465776 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 10 monclient(hunting): get_auth_request con 0x7f92701395b0 auth_method 0 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276156640 1 --2- 172.21.15.89:0/1230465776 >> v2:172.21.15.190:3300/0 conn(0x7f92700a5270 0x7f92700a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9275955640 1 --2- 172.21.15.89:0/1230465776 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f92700a6d30 0x7f92700a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276156640 10 monclient(hunting): get_auth_request con 0x7f92700a5270 auth_method 0 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276156640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276156640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276156640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9275955640 10 monclient(hunting): get_auth_request con 0x7f92700a6d30 auth_method 0 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9275955640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9275955640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9275955640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 10 monclient(hunting): handle_auth_done global_id 45204 payload 274 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 1 -- 172.21.15.89:0/1230465776 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f92700a6d30 msgr2=0x7f92700a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 1 --2- 172.21.15.89:0/1230465776 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f92700a6d30 0x7f92700a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 1 -- 172.21.15.89:0/1230465776 >> v2:172.21.15.190:3300/0 conn(0x7f92700a5270 msgr2=0x7f92700a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 1 --2- 172.21.15.89:0/1230465776 >> v2:172.21.15.190:3300/0 conn(0x7f92700a5270 0x7f92700a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 10 monclient: _finish_hunting 0 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 1 monclient: found mon.noname-d 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:32.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276957640 1 -- 172.21.15.89:0/1230465776 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92700a81d0 con 0x7f92701395b0 2024-08-19T10:53:32.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.122+0000 7f9276156640 1 --2- 172.21.15.89:0/1230465776 >> v2:172.21.15.190:3300/0 conn(0x7f92700a5270 0x7f92700a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:53:32.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276957640 1 -- 172.21.15.89:0/1230465776 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92701395b0 msgr2=0x7f9270139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276957640 1 --2- 172.21.15.89:0/1230465776 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92701395b0 0x7f9270139a40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9268009870 tx=0x7f926801c420 comp rx=0 tx=0).ready entity=mon.6 client_cookie=2cfe82acbe70bcd0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 1 -- 172.21.15.89:0/1230465776 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f92680b1070 con 0x7f92701395b0 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 10 monclient: dump: 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.129 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 10 monclient: _finish_auth 0 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 10 monclient: _check_auth_tickets 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 5 monclient: authenticate success, global_id 45204 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 1 -- 172.21.15.89:0/1230465776 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92680ab810 con 0x7f92701395b0 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 1 -- 172.21.15.89:0/1230465776 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f92680ab3a0 con 0x7f92701395b0 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 4 set_mon_vals no callback set 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275154640 10 monclient: dump: 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.131 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient: shutdown 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:32.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 -- 172.21.15.89:0/1230465776 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92701395b0 msgr2=0x7f9270139a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 --2- 172.21.15.89:0/1230465776 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92701395b0 0x7f9270139a40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9268009870 tx=0x7f926801c420 comp rx=0 tx=0).stop 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 -- 172.21.15.89:0/1230465776 shutdown_connections 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 --2- 172.21.15.89:0/1230465776 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f92701395b0 0x7f9270139a40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 --2- 172.21.15.89:0/1230465776 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f92700a6d30 0x7f92700a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 --2- 172.21.15.89:0/1230465776 >> v2:172.21.15.190:3300/0 conn(0x7f92700a5270 0x7f92700a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 -- 172.21.15.89:0/1230465776 >> 172.21.15.89:0/1230465776 conn(0x7f9270069a80 msgr2=0x7f92700abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 -- 172.21.15.89:0/1230465776 shutdown_connections 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 -- 172.21.15.89:0/1230465776 wait complete. 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient: build_initial_monmap 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient: monmap: 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:32.133 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 Processor -- start 2024-08-19T10:53:32.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 -- start start 2024-08-19T10:53:32.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient: init 2024-08-19T10:53:32.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:32.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient: _add_conns ranks=[2,0,1,5,7,3,6,8,4] 2024-08-19T10:53:32.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92700a5270 0x7f92700a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient(hunting): picked mon.noname-b con 0x7f92700a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:32.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92700a6d30 0x7f927013e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient(hunting): picked mon.noname-a con 0x7f92700a6d30 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:32.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92701395b0 0x7f927013eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient(hunting): picked mon.noname-f con 0x7f92701395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f92700a5650 con 0x7f92700a6d30 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f92700a4a70 con 0x7f92701395b0 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f92700a57d0 con 0x7f92700a5270 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f92783ba640 10 monclient(hunting): authenticate will time out at 1861.951058s 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92700a5270 0x7f92700a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92700a5270 0x7f92700a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59606/0 (socket says 172.21.15.89:59606) 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 1 -- 172.21.15.89:0/3606675338 learned_addr learned my addr 172.21.15.89:0/3606675338 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 10 monclient(hunting): get_auth_request con 0x7f92700a5270 auth_method 0 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275955640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92700a6d30 0x7f927013e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275955640 10 monclient(hunting): get_auth_request con 0x7f92700a6d30 auth_method 0 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275955640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275955640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275955640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275955640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275955640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275955640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276957640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92701395b0 0x7f927013eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276957640 10 monclient(hunting): get_auth_request con 0x7f92701395b0 auth_method 0 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276957640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276957640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276957640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 10 monclient(hunting): handle_auth_done global_id 74369 payload 824 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 1 -- 172.21.15.89:0/3606675338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92701395b0 msgr2=0x7f927013eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92701395b0 0x7f927013eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 1 -- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92700a6d30 msgr2=0x7f927013e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92700a6d30 0x7f927013e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 10 monclient: _finish_hunting 0 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 1 monclient: found mon.noname-b 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:53:32.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9276156640 1 -- 172.21.15.89:0/3606675338 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92700aac00 con 0x7f92700a5270 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.130+0000 7f9275955640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92700a6d30 0x7f927013e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9276156640 1 -- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92700a5270 msgr2=0x7f92700a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9276156640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92700a5270 0x7f92700a1610 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f926c009fa0 tx=0x7f926c00d560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f926c01a070 con 0x7f92700a5270 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 10 monclient: dump: 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.144 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 10 monclient: _finish_auth 0 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:32.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f926c003e10 con 0x7f92700a5270 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f926c0044a0 con 0x7f92700a5270 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 10 monclient: dump: 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.146 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9274953640 4 set_mon_vals no callback set 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f92783ba640 5 monclient: authenticate success, global_id 74369 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f92783ba640 10 monclient: _renew_subs 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f92783ba640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f92783ba640 1 -- 172.21.15.89:0/3606675338 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927009edf0 con 0x7f92700a5270 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f92783ba640 10 monclient: _renew_subs 2024-08-19T10:53:32.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f92783ba640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f92783ba640 1 -- 172.21.15.89:0/3606675338 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9270133a80 con 0x7f92700a5270 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f92783ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f92783ba640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f927009d8d0 0x7f927013f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f92783ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f92783ba640 1 -- 172.21.15.89:0/3606675338 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f927009e890 con 0x7f927009d8d0 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9275955640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f927009d8d0 0x7f927013f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9275955640 10 monclient: get_auth_request con 0x7f927009d8d0 auth_method 0 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9275955640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9275955640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9275955640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9275955640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9275955640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9275955640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f926c004680 con 0x7f92700a5270 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9275955640 10 monclient(hunting): handle_auth_done global_id 84203 payload 824 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9275955640 1 -- 172.21.15.89:0/3606675338 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92700a7110 con 0x7f927009d8d0 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f925403be00 0x7f925403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f926c04f7d0 con 0x7f92700a5270 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.134+0000 7f9276957640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f925403be00 0x7f925403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9276957640 10 monclient: get_auth_request con 0x7f925403be00 auth_method 0 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9275955640 1 -- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f927009d8d0 msgr2=0x7f927013f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9275955640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f927009d8d0 0x7f927013f060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f927009c9e0 tx=0x7f9260007f70 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9260018020 con 0x7f927009d8d0 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9266ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9266ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9266ffd640 10 monclient: dump: 2024-08-19T10:53:32.149 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.150 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.151 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.151 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f926000e9d0 con 0x7f927009d8d0 2024-08-19T10:53:32.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9266ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:32.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9266ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:32.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f927009d8d0 msgr2=0x7f927013f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9266ffd640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f927009d8d0 0x7f927013f060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f927009c9e0 tx=0x7f9260007f70 comp rx=0 tx=0).stop 2024-08-19T10:53:32.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f927009d8d0 msgr2=0x7f927013f060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:32.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9266ffd640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f927009d8d0 0x7f927013f060 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.138+0000 7f9276957640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f925403be00 0x7f925403e2c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f92700aceb0 tx=0x7f92680a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f92783ba640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:32.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f92783ba640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9270137a20 0x7f9270137e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f92783ba640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f92783ba640 1 -- 172.21.15.89:0/3606675338 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9270064590 con 0x7f9270137a20 2024-08-19T10:53:32.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9270137a20 0x7f9270137e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 10 monclient: get_auth_request con 0x7f9270137a20 auth_method 0 2024-08-19T10:53:32.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 10 monclient(hunting): handle_auth_done global_id 84212 payload 824 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 1 -- 172.21.15.89:0/3606675338 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f927009cd70 con 0x7f9270137a20 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 1 -- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9270137a20 msgr2=0x7f9270137e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9275955640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9270137a20 0x7f9270137e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f927013fce0 tx=0x7f92600046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9260018040 con 0x7f9270137a20 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9266ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9266ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9266ffd640 10 monclient: dump: 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.154 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.155 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f9260014470 con 0x7f9270137a20 2024-08-19T10:53:32.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9266ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:32.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9266ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:32.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9270137a20 msgr2=0x7f9270137e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9266ffd640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9270137a20 0x7f9270137e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f927013fce0 tx=0x7f92600046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9266ffd640 1 -- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9270137a20 msgr2=0x7f9270137e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:32.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.146+0000 7f9266ffd640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9270137a20 0x7f9270137e30 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f927013fce0 tx=0x7f92600046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.156 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:32.156 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:32.156 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:32.156 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 20451, 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:32.157 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.158 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.159 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:32.160 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.161 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:32.162 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.163 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:32.164 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.165 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.166 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:32.167 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.169 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.170 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.171 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:32.172 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.174 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.175 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.176 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.177 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:32.178 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.179 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.179 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:32.179 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f92783ba640 1 -- 172.21.15.89:0/3606675338 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f925403be00 msgr2=0x7f925403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f92783ba640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f925403be00 0x7f925403e2c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f92700aceb0 tx=0x7f92680a7000 comp rx=0 tx=0).stop 2024-08-19T10:53:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f92783ba640 10 monclient: shutdown 2024-08-19T10:53:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f92783ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f92783ba640 1 -- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92700a5270 msgr2=0x7f92700a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f92783ba640 1 --2- 172.21.15.89:0/3606675338 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f92700a5270 0x7f92700a1610 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f926c009fa0 tx=0x7f926c00d560 comp rx=0 tx=0).stop 2024-08-19T10:53:32.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f9276957640 1 -- 172.21.15.89:0/3606675338 reap_dead start 2024-08-19T10:53:32.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f9276957640 1 -- 172.21.15.89:0/3606675338 reap_dead start 2024-08-19T10:53:32.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f92783ba640 1 -- 172.21.15.89:0/3606675338 shutdown_connections 2024-08-19T10:53:32.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f92783ba640 1 -- 172.21.15.89:0/3606675338 >> 172.21.15.89:0/3606675338 conn(0x7f9270069a80 msgr2=0x7f927013ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:32.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f92783ba640 1 -- 172.21.15.89:0/3606675338 shutdown_connections 2024-08-19T10:53:32.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.150+0000 7f92783ba640 1 -- 172.21.15.89:0/3606675338 wait complete. 2024-08-19T10:53:32.181 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:32.181 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:53:32.182 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:53:32.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=857382309}) 2024-08-19T10:53:32.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2843638326}) 2024-08-19T10:53:32.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=857382309}) 2024-08-19T10:53:32.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3369206159}) 2024-08-19T10:53:32.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=857382309}) 2024-08-19T10:53:32.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3901513101}) 2024-08-19T10:53:32.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.188 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=857382309}) 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2409394661}) 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=857382309}) 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3159685920}) 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=857382309}) 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3690092739}) 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=857382309}) 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1717498248}) 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=857382309}) 2024-08-19T10:53:32.189 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.170+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=749752462}) 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464077894}) 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1043788404}) 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464077894}) 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4195355489}) 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464077894}) 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3347810227}) 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464077894}) 2024-08-19T10:53:32.190 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=442778974}) 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464077894}) 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=350621817}) 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464077894}) 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1096091014}) 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464077894}) 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2407505386}) 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1464077894}) 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.174+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1427632455}) 2024-08-19T10:53:32.191 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2051184399}) 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=482363541}) 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2051184399}) 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2165857902}) 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2051184399}) 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3866726049}) 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2051184399}) 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2223181503}) 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2051184399}) 2024-08-19T10:53:32.192 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1506493010}) 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2051184399}) 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2562068082}) 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2051184399}) 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3861661941}) 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2051184399}) 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.178+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1001154903}) 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4060109263}) 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3182712901}) 2024-08-19T10:53:32.193 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4060109263}) 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2920294351}) 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4060109263}) 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1504371903}) 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4060109263}) 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3420374659}) 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4060109263}) 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1986319217}) 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.194 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4060109263}) 2024-08-19T10:53:32.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2159748984}) 2024-08-19T10:53:32.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4060109263}) 2024-08-19T10:53:32.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1874427710}) 2024-08-19T10:53:32.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4060109263}) 2024-08-19T10:53:32.195 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.182+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2068190514}) 2024-08-19T10:53:32.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2754078805}) 2024-08-19T10:53:32.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=759654666}) 2024-08-19T10:53:32.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.196 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2754078805}) 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2249941729}) 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2754078805}) 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4073847021}) 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2754078805}) 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3182119886}) 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2754078805}) 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1931306220}) 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2754078805}) 2024-08-19T10:53:32.197 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3894275181}) 2024-08-19T10:53:32.198 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.198 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2754078805}) 2024-08-19T10:53:32.198 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.194+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=167407178}) 2024-08-19T10:53:32.198 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.198 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2754078805}) 2024-08-19T10:53:32.198 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1751344381}) 2024-08-19T10:53:32.202 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.202 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117271194}) 2024-08-19T10:53:32.202 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2361920897}) 2024-08-19T10:53:32.202 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.202 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117271194}) 2024-08-19T10:53:32.202 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3931326308}) 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117271194}) 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2800082344}) 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117271194}) 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2640351423}) 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117271194}) 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3986128233}) 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117271194}) 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1411585492}) 2024-08-19T10:53:32.203 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117271194}) 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1352714147}) 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2117271194}) 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.198+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2190867249}) 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2110381977}) 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4026302738}) 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2110381977}) 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4182204428}) 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.204 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2110381977}) 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2320377207}) 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2110381977}) 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=317698722}) 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2110381977}) 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3988380604}) 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2110381977}) 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=735831818}) 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2110381977}) 2024-08-19T10:53:32.205 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3026270131}) 2024-08-19T10:53:32.206 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.206 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2110381977}) 2024-08-19T10:53:32.206 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.202+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4187798422}) 2024-08-19T10:53:32.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944656080}) 2024-08-19T10:53:32.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1015154629}) 2024-08-19T10:53:32.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944656080}) 2024-08-19T10:53:32.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=213057432}) 2024-08-19T10:53:32.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944656080}) 2024-08-19T10:53:32.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4097989234}) 2024-08-19T10:53:32.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.209 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944656080}) 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2235460044}) 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944656080}) 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1472239054}) 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944656080}) 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1307602529}) 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944656080}) 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2003719839}) 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944656080}) 2024-08-19T10:53:32.210 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.206+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3498670736}) 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2814378589}) 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3511506283}) 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2814378589}) 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1054536456}) 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2814378589}) 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1967888765}) 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2814378589}) 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1790096044}) 2024-08-19T10:53:32.211 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.212 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2814378589}) 2024-08-19T10:53:32.212 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3865890543}) 2024-08-19T10:53:32.212 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.212 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2814378589}) 2024-08-19T10:53:32.212 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2285671124}) 2024-08-19T10:53:32.212 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.212 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2814378589}) 2024-08-19T10:53:32.212 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2246916466}) 2024-08-19T10:53:32.212 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.212 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2814378589}) 2024-08-19T10:53:32.212 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.210+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4145290590}) 2024-08-19T10:53:32.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549414389}) 2024-08-19T10:53:32.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4045219888}) 2024-08-19T10:53:32.219 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549414389}) 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2869321597}) 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549414389}) 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1713123562}) 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549414389}) 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=994532300}) 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549414389}) 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3541049137}) 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.220 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549414389}) 2024-08-19T10:53:32.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=37930406}) 2024-08-19T10:53:32.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549414389}) 2024-08-19T10:53:32.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2207129319}) 2024-08-19T10:53:32.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3549414389}) 2024-08-19T10:53:32.221 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.222+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2596525175}) 2024-08-19T10:53:32.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083529666}) 2024-08-19T10:53:32.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2690322790}) 2024-08-19T10:53:32.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083529666}) 2024-08-19T10:53:32.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2657561495}) 2024-08-19T10:53:32.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083529666}) 2024-08-19T10:53:32.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3850514515}) 2024-08-19T10:53:32.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.223 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083529666}) 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3566094604}) 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083529666}) 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1439645436}) 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083529666}) 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1180521672}) 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083529666}) 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=438793361}) 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083529666}) 2024-08-19T10:53:32.224 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=466445178}) 2024-08-19T10:53:32.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.226 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=862464386}) 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187685646}) 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=862464386}) 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3566047219}) 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=862464386}) 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3621290009}) 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=862464386}) 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4045734328}) 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=862464386}) 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=395048952}) 2024-08-19T10:53:32.227 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=862464386}) 2024-08-19T10:53:32.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2423374319}) 2024-08-19T10:53:32.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=862464386}) 2024-08-19T10:53:32.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3240550771}) 2024-08-19T10:53:32.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=862464386}) 2024-08-19T10:53:32.228 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.226+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=841514803}) 2024-08-19T10:53:32.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1013300235}) 2024-08-19T10:53:32.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3366952739}) 2024-08-19T10:53:32.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1013300235}) 2024-08-19T10:53:32.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1806158819}) 2024-08-19T10:53:32.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1013300235}) 2024-08-19T10:53:32.230 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2537966110}) 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1013300235}) 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2113204029}) 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1013300235}) 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=485259487}) 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1013300235}) 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4072741877}) 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1013300235}) 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=73574605}) 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.231 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1013300235}) 2024-08-19T10:53:32.232 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.230+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1795700500}) 2024-08-19T10:53:32.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852898107}) 2024-08-19T10:53:32.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2862183664}) 2024-08-19T10:53:32.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852898107}) 2024-08-19T10:53:32.235 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2663229777}) 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852898107}) 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=802823054}) 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852898107}) 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3475583259}) 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852898107}) 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1325322751}) 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852898107}) 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=92242077}) 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852898107}) 2024-08-19T10:53:32.236 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1431281042}) 2024-08-19T10:53:32.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1852898107}) 2024-08-19T10:53:32.237 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.234+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3974974308}) 2024-08-19T10:53:32.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122803023}) 2024-08-19T10:53:32.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1075988485}) 2024-08-19T10:53:32.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122803023}) 2024-08-19T10:53:32.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=605713187}) 2024-08-19T10:53:32.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122803023}) 2024-08-19T10:53:32.238 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1269071373}) 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122803023}) 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=682204219}) 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122803023}) 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2544305562}) 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122803023}) 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=636178810}) 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122803023}) 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3955749241}) 2024-08-19T10:53:32.239 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2122803023}) 2024-08-19T10:53:32.240 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.238+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4082872112}) 2024-08-19T10:53:32.242 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.242 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=834127017}) 2024-08-19T10:53:32.242 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4270128773}) 2024-08-19T10:53:32.242 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.242 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=834127017}) 2024-08-19T10:53:32.242 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3190930181}) 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=834127017}) 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1640124663}) 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=834127017}) 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1530020279}) 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=834127017}) 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=404402434}) 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=834127017}) 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1672553443}) 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.243 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=834127017}) 2024-08-19T10:53:32.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3745898226}) 2024-08-19T10:53:32.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=834127017}) 2024-08-19T10:53:32.244 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.242+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3893898942}) 2024-08-19T10:53:32.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716019439}) 2024-08-19T10:53:32.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=176843005}) 2024-08-19T10:53:32.246 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716019439}) 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1264142277}) 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716019439}) 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1733810784}) 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716019439}) 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2134990881}) 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716019439}) 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1136523681}) 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716019439}) 2024-08-19T10:53:32.247 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1701141735}) 2024-08-19T10:53:32.248 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.248 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716019439}) 2024-08-19T10:53:32.248 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1670853918}) 2024-08-19T10:53:32.248 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.248 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716019439}) 2024-08-19T10:53:32.248 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.246+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3365853414}) 2024-08-19T10:53:32.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3266577977}) 2024-08-19T10:53:32.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3345625495}) 2024-08-19T10:53:32.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3266577977}) 2024-08-19T10:53:32.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4038378898}) 2024-08-19T10:53:32.250 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3266577977}) 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3288505202}) 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3266577977}) 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2504060968}) 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3266577977}) 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1002604663}) 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3266577977}) 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1933414833}) 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.251 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3266577977}) 2024-08-19T10:53:32.252 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3853558193}) 2024-08-19T10:53:32.252 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.252 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3266577977}) 2024-08-19T10:53:32.252 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.250+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3456433500}) 2024-08-19T10:53:32.253 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.253 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2284075119}) 2024-08-19T10:53:32.253 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2595451468}) 2024-08-19T10:53:32.253 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.253 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2284075119}) 2024-08-19T10:53:32.253 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=640265708}) 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2284075119}) 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3164207151}) 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2284075119}) 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2427218518}) 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2284075119}) 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3385716247}) 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2284075119}) 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1096794753}) 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2284075119}) 2024-08-19T10:53:32.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2467097861}) 2024-08-19T10:53:32.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2284075119}) 2024-08-19T10:53:32.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3601151337}) 2024-08-19T10:53:32.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=57784008}) 2024-08-19T10:53:32.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1801898979}) 2024-08-19T10:53:32.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=57784008}) 2024-08-19T10:53:32.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3025853894}) 2024-08-19T10:53:32.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.257 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=57784008}) 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4185474377}) 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=57784008}) 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3275199373}) 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=57784008}) 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2097479304}) 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=57784008}) 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2397366689}) 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=57784008}) 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2709741859}) 2024-08-19T10:53:32.258 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=57784008}) 2024-08-19T10:53:32.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1446169929}) 2024-08-19T10:53:32.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2762920074}) 2024-08-19T10:53:32.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3020648742}) 2024-08-19T10:53:32.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2762920074}) 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=245806114}) 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2762920074}) 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=592858305}) 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2762920074}) 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1274734278}) 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2762920074}) 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1147525320}) 2024-08-19T10:53:32.261 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.262 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2762920074}) 2024-08-19T10:53:32.262 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1695562125}) 2024-08-19T10:53:32.262 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.262 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2762920074}) 2024-08-19T10:53:32.262 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1484048638}) 2024-08-19T10:53:32.262 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.262 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2762920074}) 2024-08-19T10:53:32.262 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3066738387}) 2024-08-19T10:53:32.266 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.266 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3418607411}) 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3329943100}) 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3418607411}) 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=301770744}) 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3418607411}) 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2143229887}) 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3418607411}) 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3040789970}) 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3418607411}) 2024-08-19T10:53:32.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=905348841}) 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3418607411}) 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3555245828}) 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3418607411}) 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2050578400}) 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3418607411}) 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3693313218}) 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=243432399}) 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2473877186}) 2024-08-19T10:53:32.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=243432399}) 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2922701783}) 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=243432399}) 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1462837429}) 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=243432399}) 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3555653833}) 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=243432399}) 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1734885207}) 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=243432399}) 2024-08-19T10:53:32.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1767729584}) 2024-08-19T10:53:32.270 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.270 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=243432399}) 2024-08-19T10:53:32.270 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2887902647}) 2024-08-19T10:53:32.270 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.270 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=243432399}) 2024-08-19T10:53:32.270 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4155500464}) 2024-08-19T10:53:32.276 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.276 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3328541479}) 2024-08-19T10:53:32.276 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3968340172}) 2024-08-19T10:53:32.276 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.276 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3328541479}) 2024-08-19T10:53:32.276 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2123518716}) 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3328541479}) 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2376758427}) 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3328541479}) 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1539428374}) 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3328541479}) 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3657549145}) 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3328541479}) 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3966839077}) 2024-08-19T10:53:32.277 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3328541479}) 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2827353863}) 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3328541479}) 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2498837090}) 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=169525553}) 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=977496693}) 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=169525553}) 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=315860651}) 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.278 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=169525553}) 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4072520359}) 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=169525553}) 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3293279616}) 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=169525553}) 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=315095176}) 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=169525553}) 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1747320992}) 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=169525553}) 2024-08-19T10:53:32.279 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3020772831}) 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=169525553}) 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1689906352}) 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1225246580}) 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1081960617}) 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1225246580}) 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1579939675}) 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1225246580}) 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=91701100}) 2024-08-19T10:53:32.280 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1225246580}) 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3158834825}) 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1225246580}) 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1075963805}) 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1225246580}) 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1329277179}) 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1225246580}) 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1700761959}) 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.281 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1225246580}) 2024-08-19T10:53:32.282 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1812261483}) 2024-08-19T10:53:32.294 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.294 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785549317}) 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2298310445}) 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785549317}) 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3253522422}) 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785549317}) 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2155164833}) 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785549317}) 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2767976073}) 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785549317}) 2024-08-19T10:53:32.295 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3281424290}) 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785549317}) 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1811316610}) 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785549317}) 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=561396248}) 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785549317}) 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.278+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=812773086}) 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2554543411}) 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=953412478}) 2024-08-19T10:53:32.296 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2554543411}) 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2884849511}) 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2554543411}) 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3449485659}) 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2554543411}) 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1272304109}) 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2554543411}) 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=674525200}) 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.297 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2554543411}) 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2858911634}) 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2554543411}) 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2456533697}) 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2554543411}) 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.282+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3698131694}) 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1929086591}) 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1626789763}) 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1929086591}) 2024-08-19T10:53:32.298 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=546422229}) 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1929086591}) 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1272768905}) 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1929086591}) 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3934561467}) 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1929086591}) 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1758007129}) 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1929086591}) 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=652533556}) 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.299 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1929086591}) 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1273802509}) 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1929086591}) 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.286+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1719309364}) 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1379059605}) 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2425871102}) 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1379059605}) 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3116835572}) 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1379059605}) 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=858947433}) 2024-08-19T10:53:32.300 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1379059605}) 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3703183822}) 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1379059605}) 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4064769128}) 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1379059605}) 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3279732256}) 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1379059605}) 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3318721671}) 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.301 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1379059605}) 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.290+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1742446752}) 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1668568968}) 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3995888301}) 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1668568968}) 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1945033015}) 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1668568968}) 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=428188826}) 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1668568968}) 2024-08-19T10:53:32.302 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1306159480}) 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1668568968}) 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3857188416}) 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1668568968}) 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1768676697}) 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1668568968}) 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1243002973}) 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1668568968}) 2024-08-19T10:53:32.303 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.294+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1720229124}) 2024-08-19T10:53:32.309 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.309 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3029753345}) 2024-08-19T10:53:32.309 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1598039434}) 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3029753345}) 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2475387961}) 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3029753345}) 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=136353319}) 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3029753345}) 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2023218969}) 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3029753345}) 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2112932691}) 2024-08-19T10:53:32.310 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3029753345}) 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1324933333}) 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3029753345}) 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1177171519}) 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3029753345}) 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.302+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140755678}) 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=15092956}) 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3704849490}) 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=15092956}) 2024-08-19T10:53:32.311 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3558817110}) 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=15092956}) 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3882123521}) 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=15092956}) 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3202932893}) 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=15092956}) 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=780389576}) 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=15092956}) 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4189685558}) 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=15092956}) 2024-08-19T10:53:32.312 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2465110744}) 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=15092956}) 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.306+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1119711227}) 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514326092}) 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=585913647}) 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514326092}) 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3569022741}) 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514326092}) 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=604970776}) 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514326092}) 2024-08-19T10:53:32.313 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2183632457}) 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514326092}) 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1809134105}) 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514326092}) 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2809523518}) 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514326092}) 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1499491073}) 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514326092}) 2024-08-19T10:53:32.314 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.310+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2244390959}) 2024-08-19T10:53:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.326+0000 7f2adc3f4640 10 monclient: get_monmap_and_config 2024-08-19T10:53:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.326+0000 7f2adc3f4640 10 monclient: build_initial_monmap 2024-08-19T10:53:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.326+0000 7f2adc3f4640 10 monclient: monmap: 2024-08-19T10:53:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:32.331844+0000 2024-08-19T10:53:32.325 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:32.331844+0000 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:32.326 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.326+0000 7f2adc3f4640 1 Processor -- start 2024-08-19T10:53:32.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.326+0000 7f2adc3f4640 1 -- start start 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.326+0000 7f2adc3f4640 10 monclient: init 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 10 monclient: _add_conns ranks=[6,7,9,2,10,8,1,4,3,0,5] 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ad40a6a20 0x7f2ad40a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 10 monclient(hunting): picked mon.noname-d con 0x7f2ad40a6a20 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2ad40a55d0 0x7f2ad40a7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 10 monclient(hunting): picked mon.noname-i con 0x7f2ad40a55d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ad413d800 0x7f2ad413fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 10 monclient(hunting): picked mon.noname-a con 0x7f2ad413d800 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad40a34f0 con 0x7f2ad413d800 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f2ad40a7880 con 0x7f2ad40a6a20 2024-08-19T10:53:32.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2ad40a4ab0 con 0x7f2ad40a55d0 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2adc3f4640 10 monclient(hunting): authenticate will time out at 1862.148336s 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada190640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ad40a6a20 0x7f2ad40a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ad413d800 0x7f2ad413fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2ad413d800 0x7f2ad413fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53020/0 (socket says 172.21.15.89:53020) 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 1 -- 172.21.15.89:0/2656844812 learned_addr learned my addr 172.21.15.89:0/2656844812 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 10 monclient(hunting): get_auth_request con 0x7f2ad413d800 auth_method 0 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada190640 1 --2- 172.21.15.89:0/2656844812 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ad40a6a20 0x7f2ad40a6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46710/0 (socket says 172.21.15.89:46710) 2024-08-19T10:53:32.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada190640 10 monclient(hunting): get_auth_request con 0x7f2ad40a6a20 auth_method 0 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada190640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada190640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada190640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada190640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada190640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada190640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ad998f640 1 --2- 172.21.15.89:0/2656844812 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2ad40a55d0 0x7f2ad40a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 10 monclient(hunting): handle_auth_done global_id 7407 payload 274 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 1 -- 172.21.15.89:0/2656844812 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2ad40a55d0 msgr2=0x7f2ad40a7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 1 --2- 172.21.15.89:0/2656844812 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2ad40a55d0 0x7f2ad40a7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 1 -- 172.21.15.89:0/2656844812 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ad40a6a20 msgr2=0x7f2ad40a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:32.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 1 --2- 172.21.15.89:0/2656844812 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ad40a6a20 0x7f2ad40a6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 10 monclient: _finish_hunting 0 2024-08-19T10:53:32.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 1 monclient: found mon.noname-a 2024-08-19T10:53:32.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:32.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:32.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada991640 1 -- 172.21.15.89:0/2656844812 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad40a3d90 con 0x7f2ad413d800 2024-08-19T10:53:32.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.330+0000 7f2ada190640 1 --2- 172.21.15.89:0/2656844812 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ad40a6a20 0x7f2ad40a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:32.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada991640 1 -- 172.21.15.89:0/2656844812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad413d800 msgr2=0x7f2ad413fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada991640 1 --2- 172.21.15.89:0/2656844812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad413d800 0x7f2ad413fbf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009e50 tx=0x7f2ac401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8531e53288b76c44 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 1 -- 172.21.15.89:0/2656844812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ac40090d0 con 0x7f2ad413d800 2024-08-19T10:53:32.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:32.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 10 monclient: dump: 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.336 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 10 monclient: _finish_auth 0 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 10 monclient: _check_auth_tickets 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 1 -- 172.21.15.89:0/2656844812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac40a5a90 con 0x7f2ad413d800 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 5 monclient: authenticate success, global_id 7407 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 1 -- 172.21.15.89:0/2656844812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ac40a5c40 con 0x7f2ad413d800 2024-08-19T10:53:32.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 4 set_mon_vals no callback set 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad918e640 10 monclient: dump: 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.338 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient: shutdown 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 -- 172.21.15.89:0/2656844812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad413d800 msgr2=0x7f2ad413fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 --2- 172.21.15.89:0/2656844812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad413d800 0x7f2ad413fbf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009e50 tx=0x7f2ac401c8f0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 -- 172.21.15.89:0/2656844812 shutdown_connections 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 --2- 172.21.15.89:0/2656844812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad413d800 0x7f2ad413fbf0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 --2- 172.21.15.89:0/2656844812 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2ad40a55d0 0x7f2ad40a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 --2- 172.21.15.89:0/2656844812 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ad40a6a20 0x7f2ad40a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 -- 172.21.15.89:0/2656844812 >> 172.21.15.89:0/2656844812 conn(0x7f2ad4074bf0 msgr2=0x7f2ad40ac020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 -- 172.21.15.89:0/2656844812 shutdown_connections 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 -- 172.21.15.89:0/2656844812 wait complete. 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient: build_initial_monmap 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient: monmap: 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:32.340 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 Processor -- start 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 -- start start 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient: init 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient: _add_conns ranks=[0,5,4,7,6,1,8,3,2] 2024-08-19T10:53:32.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad40a55d0 0x7f2ad41387f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient(hunting): picked mon.noname-a con 0x7f2ad40a55d0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2ad40a6a20 0x7f2ad40a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient(hunting): picked mon.noname-h con 0x7f2ad40a6a20 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ad413d800 0x7f2ad40a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient(hunting): picked mon.noname-c con 0x7f2ad413d800 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad4142ad0 con 0x7f2ad40a55d0 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f2ad4142950 con 0x7f2ad413d800 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2ad4142c50 con 0x7f2ad40a6a20 2024-08-19T10:53:32.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2adc3f4640 10 monclient(hunting): authenticate will time out at 1862.155447s 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad40a55d0 0x7f2ad41387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad40a55d0 0x7f2ad41387f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53030/0 (socket says 172.21.15.89:53030) 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada991640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ad413d800 0x7f2ad40a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 1 -- 172.21.15.89:0/424811572 learned_addr learned my addr 172.21.15.89:0/424811572 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 10 monclient(hunting): get_auth_request con 0x7f2ad40a55d0 auth_method 0 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada991640 10 monclient(hunting): get_auth_request con 0x7f2ad413d800 auth_method 0 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada991640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada991640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada991640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada991640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada991640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada991640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ad998f640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2ad40a6a20 0x7f2ad40a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 10 monclient(hunting): handle_auth_done global_id 7416 payload 824 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 1 -- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2ad40a6a20 msgr2=0x7f2ad40a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2ad40a6a20 0x7f2ad40a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 1 -- 172.21.15.89:0/424811572 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ad413d800 msgr2=0x7f2ad40a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ad413d800 0x7f2ad40a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 10 monclient: _finish_hunting 0 2024-08-19T10:53:32.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 1 monclient: found mon.noname-a 2024-08-19T10:53:32.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:32.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:32.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada190640 1 -- 172.21.15.89:0/424811572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad40a2430 con 0x7f2ad40a55d0 2024-08-19T10:53:32.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.334+0000 7f2ada991640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2ad413d800 0x7f2ad40a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:32.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ada190640 1 -- 172.21.15.89:0/424811572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad40a55d0 msgr2=0x7f2ad41387f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ada190640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad40a55d0 0x7f2ad41387f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00ec10 tx=0x7f2acc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2acc017070 con 0x7f2ad40a55d0 2024-08-19T10:53:32.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 10 monclient: dump: 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.349 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2adc3f4640 5 monclient: authenticate success, global_id 7416 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2acc00ce40 con 0x7f2ad40a55d0 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2adc3f4640 10 monclient: _renew_subs 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2adc3f4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2adc3f4640 1 -- 172.21.15.89:0/424811572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad4139870 con 0x7f2ad40a55d0 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:32.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2acc013d50 con 0x7f2ad40a55d0 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad898d640 4 set_mon_vals no callback set 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 10 monclient: dump: 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.351 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2adc3f4640 10 monclient: _renew_subs 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2adc3f4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2adc3f4640 1 -- 172.21.15.89:0/424811572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad4123ed0 con 0x7f2ad40a55d0 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2adc3f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2adc3f4640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad4138d30 0x7f2ad40a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2adc3f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2adc3f4640 1 -- 172.21.15.89:0/424811572 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad413f930 con 0x7f2ad4138d30 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2acc01a070 con 0x7f2ad40a55d0 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ac003be00 0x7f2ac003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad998f640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad4138d30 0x7f2ad40a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad998f640 10 monclient: get_auth_request con 0x7f2ad4138d30 auth_method 0 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad998f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad998f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f2acc00f040 con 0x7f2ad40a55d0 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad998f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad998f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad998f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ad998f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.338+0000 7f2ada991640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ac003be00 0x7f2ac003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ada991640 10 monclient: get_auth_request con 0x7f2ac003be00 auth_method 0 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad998f640 10 monclient(hunting): handle_auth_done global_id 45172 payload 824 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad998f640 1 -- 172.21.15.89:0/424811572 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ad413f400 con 0x7f2ad4138d30 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad998f640 1 -- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad4138d30 msgr2=0x7f2ad40a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad998f640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad4138d30 0x7f2ad40a15d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40ace90 tx=0x7f2ac800d9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ada991640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ac003be00 0x7f2ac003e2c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40acf60 tx=0x7f2ac40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ac8016070 con 0x7f2ad4138d30 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad2ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad2ffd640 10 monclient: dump: 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.354 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2ac8012960 con 0x7f2ad4138d30 2024-08-19T10:53:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad4138d30 msgr2=0x7f2ad40a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad2ffd640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad4138d30 0x7f2ad40a15d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40ace90 tx=0x7f2ac800d9e0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.342+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad4138d30 msgr2=0x7f2ad40a15d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.346+0000 7f2ad2ffd640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad4138d30 0x7f2ad40a15d0 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40ace90 tx=0x7f2ac800d9e0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2adc3f4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2adc3f4640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad40a1b10 0x7f2ad409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2adc3f4640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2adc3f4640 1 -- 172.21.15.89:0/424811572 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad4123aa0 con 0x7f2ad40a1b10 2024-08-19T10:53:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2ad998f640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad40a1b10 0x7f2ad409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2ad998f640 10 monclient: get_auth_request con 0x7f2ad40a1b10 auth_method 0 2024-08-19T10:53:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2ad998f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2ad998f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2ad998f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2ad998f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2ad998f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2ad998f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2ad998f640 10 monclient(hunting): handle_auth_done global_id 45181 payload 824 2024-08-19T10:53:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.350+0000 7f2ad998f640 1 -- 172.21.15.89:0/424811572 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2ad40a6e00 con 0x7f2ad40a1b10 2024-08-19T10:53:32.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad998f640 1 -- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad40a1b10 msgr2=0x7f2ad409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad998f640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad40a1b10 0x7f2ad409c9e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40ace90 tx=0x7f2ac8004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2ac801ae20 con 0x7f2ad40a1b10 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad2ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad2ffd640 10 monclient: dump: 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.359 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f2ac801ae20 con 0x7f2ad40a1b10 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad2ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad40a1b10 msgr2=0x7f2ad409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad2ffd640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad40a1b10 0x7f2ad409c9e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40ace90 tx=0x7f2ac8004740 comp rx=0 tx=0).stop 2024-08-19T10:53:32.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad2ffd640 1 -- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad40a1b10 msgr2=0x7f2ad409c9e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ad2ffd640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ad40a1b10 0x7f2ad409c9e0 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40ace90 tx=0x7f2ac8004740 comp rx=0 tx=0).stop 2024-08-19T10:53:32.361 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:32.361 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:32.361 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:32.361 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:32.361 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:32.361 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 535483, 2024-08-19T10:53:32.361 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:32.361 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:32.362 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.363 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.364 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:32.365 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.366 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.367 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:32.368 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.369 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.370 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.371 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:32.372 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.373 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:32.374 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.376 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:32.377 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.379 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.380 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.381 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:32.382 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:32.383 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:32.384 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:32.385 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.385 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.385 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:32.385 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2adc3f4640 1 -- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ac003be00 msgr2=0x7f2ac003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2adc3f4640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ac003be00 0x7f2ac003e2c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40acf60 tx=0x7f2ac40a7070 comp rx=0 tx=0).stop 2024-08-19T10:53:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2adc3f4640 10 monclient: shutdown 2024-08-19T10:53:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2adc3f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2adc3f4640 1 -- 172.21.15.89:0/424811572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad40a55d0 msgr2=0x7f2ad41387f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2adc3f4640 1 --2- 172.21.15.89:0/424811572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ad40a55d0 0x7f2ad41387f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00ec10 tx=0x7f2acc00c520 comp rx=0 tx=0).stop 2024-08-19T10:53:32.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ada991640 1 -- 172.21.15.89:0/424811572 reap_dead start 2024-08-19T10:53:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2ada991640 1 -- 172.21.15.89:0/424811572 reap_dead start 2024-08-19T10:53:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2adc3f4640 1 -- 172.21.15.89:0/424811572 shutdown_connections 2024-08-19T10:53:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2adc3f4640 1 -- 172.21.15.89:0/424811572 >> 172.21.15.89:0/424811572 conn(0x7f2ad4074bf0 msgr2=0x7f2ad40abdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2adc3f4640 1 -- 172.21.15.89:0/424811572 shutdown_connections 2024-08-19T10:53:32.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.354+0000 7f2adc3f4640 1 -- 172.21.15.89:0/424811572 wait complete. 2024-08-19T10:53:32.387 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:32.387 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:53:32.387 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:53:32.493 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.493 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1592111567}) 2024-08-19T10:53:32.493 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1854537637}) 2024-08-19T10:53:32.493 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1592111567}) 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1871429781}) 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1592111567}) 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3228378086}) 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1592111567}) 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=157005499}) 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1592111567}) 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2771580938}) 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.494 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1592111567}) 2024-08-19T10:53:32.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=633771192}) 2024-08-19T10:53:32.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1592111567}) 2024-08-19T10:53:32.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1408867603}) 2024-08-19T10:53:32.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1592111567}) 2024-08-19T10:53:32.495 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.494+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=558061044}) 2024-08-19T10:53:32.497 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.497 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839675346}) 2024-08-19T10:53:32.497 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956588519}) 2024-08-19T10:53:32.497 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.497 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839675346}) 2024-08-19T10:53:32.497 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3603461482}) 2024-08-19T10:53:32.497 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.497 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839675346}) 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4206801253}) 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839675346}) 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2404225926}) 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839675346}) 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1120608031}) 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839675346}) 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2345485383}) 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839675346}) 2024-08-19T10:53:32.498 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=44185047}) 2024-08-19T10:53:32.499 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.499 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839675346}) 2024-08-19T10:53:32.499 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.498+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2512452745}) 2024-08-19T10:53:32.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient: get_monmap_and_config 2024-08-19T10:53:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient: build_initial_monmap 2024-08-19T10:53:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient: monmap: 2024-08-19T10:53:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:32.561127+0000 2024-08-19T10:53:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:32.561127+0000 2024-08-19T10:53:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:32.555 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:32.556 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:32.556 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 1 Processor -- start 2024-08-19T10:53:32.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 1 -- start start 2024-08-19T10:53:32.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient: init 2024-08-19T10:53:32.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:32.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient: _add_conns ranks=[8,6,5,0,10,4,2,3,7,1,9] 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd900a58c0 0x7fdd90072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient(hunting): picked mon.noname-e con 0x7fdd900a58c0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdd900a5ca0 0x7fdd900a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient(hunting): picked mon.noname-d con 0x7fdd900a5ca0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd90139690 0x7fdd90139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient(hunting): picked mon.noname-h con 0x7fdd90139690 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fdd9006f360 con 0x7fdd90139690 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fdd9006f080 con 0x7fdd900a5ca0 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fdd9006f640 con 0x7fdd900a58c0 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd94c07640 10 monclient(hunting): authenticate will time out at 1862.377675s 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8dd9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdd900a5ca0 0x7fdd900a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd900a58c0 0x7fdd90072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8dd9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdd900a5ca0 0x7fdd900a6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46726/0 (socket says 172.21.15.89:46726) 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd900a58c0 0x7fdd90072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:52990/0 (socket says 172.21.15.89:52990) 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8dd9b640 1 -- 172.21.15.89:0/1028027262 learned_addr learned my addr 172.21.15.89:0/1028027262 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8dd9b640 10 monclient(hunting): get_auth_request con 0x7fdd900a5ca0 auth_method 0 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 10 monclient(hunting): get_auth_request con 0x7fdd900a58c0 auth_method 0 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8ed9d640 1 --2- 172.21.15.89:0/1028027262 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd90139690 0x7fdd90139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_done global_id 84221 payload 274 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 1 -- 172.21.15.89:0/1028027262 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdd900a5ca0 msgr2=0x7fdd900a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:32.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 1 --2- 172.21.15.89:0/1028027262 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdd900a5ca0 0x7fdd900a6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 1 -- 172.21.15.89:0/1028027262 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd90139690 msgr2=0x7fdd90139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:32.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 1 --2- 172.21.15.89:0/1028027262 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd90139690 0x7fdd90139ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:53:32.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 1 monclient: found mon.noname-e 2024-08-19T10:53:32.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:32.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:53:32.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8e59c640 1 -- 172.21.15.89:0/1028027262 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd900a73e0 con 0x7fdd900a58c0 2024-08-19T10:53:32.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.558+0000 7fdd8dd9b640 1 --2- 172.21.15.89:0/1028027262 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdd900a5ca0 0x7fdd900a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8e59c640 1 -- 172.21.15.89:0/1028027262 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd900a58c0 msgr2=0x7fdd90072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8e59c640 1 --2- 172.21.15.89:0/1028027262 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd900a58c0 0x7fdd90072030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009df0 tx=0x7fdd8801c8e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f9de002654439111 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 1 -- 172.21.15.89:0/1028027262 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd880a6940 con 0x7fdd900a58c0 2024-08-19T10:53:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:53:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 10 monclient: dump: 2024-08-19T10:53:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.567 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.568 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 10 monclient: _finish_auth 0 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 5 monclient: authenticate success, global_id 84221 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 1 -- 172.21.15.89:0/1028027262 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd880a6b00 con 0x7fdd900a58c0 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 4 set_mon_vals no callback set 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 1 -- 172.21.15.89:0/1028027262 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd880a6cb0 con 0x7fdd900a58c0 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd8d59a640 10 monclient: dump: 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.569 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.570 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 10 monclient: shutdown 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 1 -- 172.21.15.89:0/1028027262 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd900a58c0 msgr2=0x7fdd90072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 1 --2- 172.21.15.89:0/1028027262 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd900a58c0 0x7fdd90072030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009df0 tx=0x7fdd8801c8e0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 1 -- 172.21.15.89:0/1028027262 shutdown_connections 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 1 --2- 172.21.15.89:0/1028027262 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdd90139690 0x7fdd90139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 1 --2- 172.21.15.89:0/1028027262 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fdd900a5ca0 0x7fdd900a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 1 --2- 172.21.15.89:0/1028027262 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fdd900a58c0 0x7fdd90072030 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 1 -- 172.21.15.89:0/1028027262 >> 172.21.15.89:0/1028027262 conn(0x7fdd900aa4b0 msgr2=0x7fdd900a69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 1 -- 172.21.15.89:0/1028027262 shutdown_connections 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 1 -- 172.21.15.89:0/1028027262 wait complete. 2024-08-19T10:53:32.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 10 monclient: build_initial_monmap 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.562+0000 7fdd94c07640 10 monclient: monmap: 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:32.572 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 1 Processor -- start 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 1 -- start start 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient: init 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient: _add_conns ranks=[7,2,3,8,1,5,6,0,4] 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdd900a58c0 0x7fdd90136510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient(hunting): picked mon.noname-i con 0x7fdd900a58c0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd900a5ca0 0x7fdd90136a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient(hunting): picked mon.noname-b con 0x7fdd900a5ca0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:32.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd90139690 0x7fdd9013d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient(hunting): picked mon.noname-g con 0x7fdd90139690 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fdd90143100 con 0x7fdd900a5ca0 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdd90143280 con 0x7fdd90139690 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fdd90143400 con 0x7fdd900a58c0 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient(hunting): authenticate will time out at 1862.384757s 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd900a5ca0 0x7fdd90136a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd900a5ca0 0x7fdd90136a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59610/0 (socket says 172.21.15.89:59610) 2024-08-19T10:53:32.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 1 -- 172.21.15.89:0/2137081039 learned_addr learned my addr 172.21.15.89:0/2137081039 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 10 monclient(hunting): get_auth_request con 0x7fdd900a5ca0 auth_method 0 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8e59c640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdd900a58c0 0x7fdd90136510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8ed9d640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd90139690 0x7fdd9013d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 10 monclient(hunting): handle_auth_done global_id 74378 payload 824 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 1 -- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdd900a58c0 msgr2=0x7fdd90136510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdd900a58c0 0x7fdd90136510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 1 -- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd90139690 msgr2=0x7fdd9013d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd90139690 0x7fdd9013d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 10 monclient: _finish_hunting 0 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 1 monclient: found mon.noname-b 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 1 -- 172.21.15.89:0/2137081039 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9013f110 con 0x7fdd900a5ca0 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8e59c640 10 monclient: get_auth_request con 0x7fdd900a58c0 auth_method 0 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8e59c640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fdd900a58c0 0x7fdd90136510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8ed9d640 10 monclient: get_auth_request con 0x7fdd90139690 auth_method 0 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8ed9d640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd90139690 0x7fdd9013d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 1 -- 172.21.15.89:0/2137081039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd900a5ca0 msgr2=0x7fdd90136a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8dd9b640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd900a5ca0 0x7fdd90136a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000e9d0 tx=0x7fdd8000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd80017070 con 0x7fdd900a5ca0 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 10 monclient: dump: 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.577 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 10 monclient: _finish_auth 0 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8000cc70 con 0x7fdd900a5ca0 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd8000ce20 con 0x7fdd900a5ca0 2024-08-19T10:53:32.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 5 monclient: authenticate success, global_id 74378 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient: _renew_subs 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 1 -- 172.21.15.89:0/2137081039 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9013fa00 con 0x7fdd900a5ca0 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd8cd99640 4 set_mon_vals no callback set 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd86ffd640 10 monclient: dump: 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.579 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient: _renew_subs 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.566+0000 7fdd94c07640 1 -- 172.21.15.89:0/2137081039 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd900a7d60 con 0x7fdd900a5ca0 2024-08-19T10:53:32.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd94c07640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd94c07640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013dc30 0x7fdd9013e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd94c07640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd94c07640 1 -- 172.21.15.89:0/2137081039 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9006e8f0 con 0x7fdd9013dc30 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013dc30 0x7fdd9013e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fdd8001a070 con 0x7fdd900a5ca0 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 10 monclient: get_auth_request con 0x7fdd9013dc30 auth_method 0 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd86ffd640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdd7803be00 0x7fdd7803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fdd8000f040 con 0x7fdd900a5ca0 2024-08-19T10:53:32.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_done global_id 35814 payload 824 2024-08-19T10:53:32.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 1 -- 172.21.15.89:0/2137081039 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd900a6080 con 0x7fdd9013dc30 2024-08-19T10:53:32.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8ed9d640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdd7803be00 0x7fdd7803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8ed9d640 10 monclient: get_auth_request con 0x7fdd7803be00 auth_method 0 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 1 -- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013dc30 msgr2=0x7fdd9013e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.570+0000 7fdd8e59c640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013dc30 0x7fdd9013e010 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdd880af000 tx=0x7fdd88019680 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd880bd070 con 0x7fdd9013dc30 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd86ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd86ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd86ffd640 10 monclient: dump: 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.585 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd8ed9d640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdd7803be00 0x7fdd7803e2c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdd90122470 tx=0x7fdd7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdd88004540 con 0x7fdd9013dc30 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd86ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:32.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd86ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013dc30 msgr2=0x7fdd9013e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd86ffd640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013dc30 0x7fdd9013e010 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdd880af000 tx=0x7fdd88019680 comp rx=0 tx=0).stop 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013dc30 msgr2=0x7fdd9013e010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.574+0000 7fdd86ffd640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013dc30 0x7fdd9013e010 secure :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdd880af000 tx=0x7fdd88019680 comp rx=0 tx=0).stop 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd94c07640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd94c07640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013e550 0x7fdd90137880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd94c07640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd94c07640 1 -- 172.21.15.89:0/2137081039 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdd90069f50 con 0x7fdd9013e550 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd8e59c640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013e550 0x7fdd90137880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd8e59c640 10 monclient: get_auth_request con 0x7fdd9013e550 auth_method 0 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd8e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd8e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd8e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.578+0000 7fdd8e59c640 10 monclient(hunting): handle_auth_done global_id 35823 payload 824 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd8e59c640 1 -- 172.21.15.89:0/2137081039 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdd9006f920 con 0x7fdd9013e550 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd8e59c640 1 -- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013e550 msgr2=0x7fdd90137880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd8e59c640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013e550 0x7fdd90137880 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdd880b0040 tx=0x7fdd880a9000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdd880b9ac0 con 0x7fdd9013e550 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd86ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd86ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd86ffd640 10 monclient: dump: 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.588 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7fdd880ae420 con 0x7fdd9013e550 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd86ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd86ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:32.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013e550 msgr2=0x7fdd90137880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd86ffd640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013e550 0x7fdd90137880 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdd880b0040 tx=0x7fdd880a9000 comp rx=0 tx=0).stop 2024-08-19T10:53:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd86ffd640 1 -- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013e550 msgr2=0x7fdd90137880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:32.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.582+0000 7fdd86ffd640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd9013e550 0x7fdd90137880 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.590 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:32.590 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:32.590 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:32.590 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:32.590 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:32.590 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 657629, 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:32.591 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.592 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.593 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.595 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.596 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.597 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:32.599 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:32.602 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.604 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.605 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:32.607 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.610 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:32.611 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.612 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:53:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.613 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:32.613 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:32.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd94c07640 1 -- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdd7803be00 msgr2=0x7fdd7803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd94c07640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdd7803be00 0x7fdd7803e2c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdd90122470 tx=0x7fdd7c008040 comp rx=0 tx=0).stop 2024-08-19T10:53:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd94c07640 10 monclient: shutdown 2024-08-19T10:53:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd94c07640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd94c07640 1 -- 172.21.15.89:0/2137081039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd900a5ca0 msgr2=0x7fdd90136a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd94c07640 1 --2- 172.21.15.89:0/2137081039 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdd900a5ca0 0x7fdd90136a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000e9d0 tx=0x7fdd8000ee90 comp rx=0 tx=0).stop 2024-08-19T10:53:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd8ed9d640 1 -- 172.21.15.89:0/2137081039 reap_dead start 2024-08-19T10:53:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd8ed9d640 1 -- 172.21.15.89:0/2137081039 reap_dead start 2024-08-19T10:53:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd94c07640 1 -- 172.21.15.89:0/2137081039 shutdown_connections 2024-08-19T10:53:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd94c07640 1 -- 172.21.15.89:0/2137081039 >> 172.21.15.89:0/2137081039 conn(0x7fdd900aa4b0 msgr2=0x7fdd900a8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd94c07640 1 -- 172.21.15.89:0/2137081039 shutdown_connections 2024-08-19T10:53:32.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.586+0000 7fdd94c07640 1 -- 172.21.15.89:0/2137081039 wait complete. 2024-08-19T10:53:32.616 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:32.616 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:53:32.616 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:53:32.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707662331}) 2024-08-19T10:53:32.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1582902798}) 2024-08-19T10:53:32.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707662331}) 2024-08-19T10:53:32.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3609460390}) 2024-08-19T10:53:32.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707662331}) 2024-08-19T10:53:32.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3968935356}) 2024-08-19T10:53:32.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.625 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707662331}) 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2031219486}) 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707662331}) 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11259371}) 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707662331}) 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4092072655}) 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707662331}) 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2859658381}) 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.626 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707662331}) 2024-08-19T10:53:32.627 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.626+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=910751379}) 2024-08-19T10:53:32.629 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.629 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=770219674}) 2024-08-19T10:53:32.629 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=451472334}) 2024-08-19T10:53:32.629 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.629 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=770219674}) 2024-08-19T10:53:32.629 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2447190006}) 2024-08-19T10:53:32.629 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.629 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=770219674}) 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1907858649}) 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=770219674}) 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1465501701}) 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=770219674}) 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=180953969}) 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=770219674}) 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=954536389}) 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=770219674}) 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3438225470}) 2024-08-19T10:53:32.630 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.631 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=770219674}) 2024-08-19T10:53:32.631 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.630+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=281895932}) 2024-08-19T10:53:32.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient: get_monmap_and_config 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient: build_initial_monmap 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient: monmap: 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:32.741082+0000 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:32.741082+0000 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:32.735 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:32.736 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:32.736 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:32.736 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:32.736 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:32.736 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:32.736 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 1 Processor -- start 2024-08-19T10:53:32.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 1 -- start start 2024-08-19T10:53:32.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient: init 2024-08-19T10:53:32.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:32.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient: _add_conns ranks=[5,8,6,1,7,10,3,2,4,0,9] 2024-08-19T10:53:32.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f73740a3a80 0x7f73740a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient(hunting): picked mon.noname-h con 0x7f73740a3a80 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:32.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73740a02a0 0x7f73740a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient(hunting): picked mon.noname-e con 0x7f73740a02a0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f73741398f0 0x7f7374139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient(hunting): picked mon.noname-d con 0x7f73741398f0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7374074550 con 0x7f73740a3a80 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f73740a4ed0 con 0x7f73741398f0 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f73740a48e0 con 0x7f73740a02a0 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737894d640 10 monclient(hunting): authenticate will time out at 1862.557808s 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f73741398f0 0x7f7374139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7371d9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73740a02a0 0x7f73740a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f73741398f0 0x7f7374139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46742/0 (socket says 172.21.15.89:46742) 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 1 -- 172.21.15.89:0/2377047523 learned_addr learned my addr 172.21.15.89:0/2377047523 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 10 monclient(hunting): get_auth_request con 0x7f73741398f0 auth_method 0 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7371d9b640 10 monclient(hunting): get_auth_request con 0x7f73740a02a0 auth_method 0 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7371d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7371d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7371d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737259c640 1 --2- 172.21.15.89:0/2377047523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f73740a3a80 0x7f73740a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7371d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7371d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7371d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737259c640 10 monclient(hunting): get_auth_request con 0x7f73740a3a80 auth_method 0 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f737259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 10 monclient(hunting): handle_auth_done global_id 45231 payload 274 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 1 -- 172.21.15.89:0/2377047523 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73740a02a0 msgr2=0x7f73740a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 1 --2- 172.21.15.89:0/2377047523 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73740a02a0 0x7f73740a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 1 -- 172.21.15.89:0/2377047523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f73740a3a80 msgr2=0x7f73740a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 1 --2- 172.21.15.89:0/2377047523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f73740a3a80 0x7f73740a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:53:32.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 1 monclient: found mon.noname-d 2024-08-19T10:53:32.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:32.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:32.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7372d9d640 1 -- 172.21.15.89:0/2377047523 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73740a1300 con 0x7f73741398f0 2024-08-19T10:53:32.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.738+0000 7f7371d9b640 1 --2- 172.21.15.89:0/2377047523 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73740a02a0 0x7f73740a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:32.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f7372d9d640 1 -- 172.21.15.89:0/2377047523 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f73741398f0 msgr2=0x7f7374139cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f7372d9d640 1 --2- 172.21.15.89:0/2377047523 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f73741398f0 0x7f7374139cf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f73640098d0 tx=0x7f736401c440 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a939f88d14f4ba88 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 1 -- 172.21.15.89:0/2377047523 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73640b0070 con 0x7f73741398f0 2024-08-19T10:53:32.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:32.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 10 monclient: dump: 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.746 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 10 monclient: _finish_auth 0 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 10 monclient: _check_auth_tickets 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 1 -- 172.21.15.89:0/2377047523 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73640aa810 con 0x7f73741398f0 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 1 -- 172.21.15.89:0/2377047523 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73640a9300 con 0x7f73741398f0 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 5 monclient: authenticate success, global_id 45231 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:32.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 4 set_mon_vals no callback set 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737159a640 10 monclient: dump: 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.748 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 10 monclient: shutdown 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 1 -- 172.21.15.89:0/2377047523 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f73741398f0 msgr2=0x7f7374139cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 1 --2- 172.21.15.89:0/2377047523 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f73741398f0 0x7f7374139cf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f73640098d0 tx=0x7f736401c440 comp rx=0 tx=0).stop 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 1 -- 172.21.15.89:0/2377047523 shutdown_connections 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 1 --2- 172.21.15.89:0/2377047523 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f73741398f0 0x7f7374139cf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 1 --2- 172.21.15.89:0/2377047523 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f73740a02a0 0x7f73740a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 1 --2- 172.21.15.89:0/2377047523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f73740a3a80 0x7f73740a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 1 -- 172.21.15.89:0/2377047523 >> 172.21.15.89:0/2377047523 conn(0x7f737406de40 msgr2=0x7f737406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 1 -- 172.21.15.89:0/2377047523 shutdown_connections 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.742+0000 7f737894d640 1 -- 172.21.15.89:0/2377047523 wait complete. 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient: build_initial_monmap 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient: monmap: 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:32.750 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 1 Processor -- start 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 1 -- start start 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient: init 2024-08-19T10:53:32.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient: _add_conns ranks=[0,1,4,7,6,5,2,3,8] 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73740a02a0 0x7f7374138420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient(hunting): picked mon.noname-a con 0x7f73740a02a0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73740a3a80 0x7f7374138960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient(hunting): picked mon.noname-f con 0x7f73740a3a80 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73741398f0 0x7f737413c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient(hunting): picked mon.noname-c con 0x7f73741398f0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7374143360 con 0x7f73740a02a0 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f737413a640 con 0x7f73740a3a80 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f73741434e0 con 0x7f73741398f0 2024-08-19T10:53:32.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737894d640 10 monclient(hunting): authenticate will time out at 1862.565104s 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73740a02a0 0x7f7374138420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7372d9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73741398f0 0x7f737413c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73740a02a0 0x7f7374138420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53044/0 (socket says 172.21.15.89:53044) 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 1 -- 172.21.15.89:0/1684712209 learned_addr learned my addr 172.21.15.89:0/1684712209 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 10 monclient(hunting): get_auth_request con 0x7f73740a02a0 auth_method 0 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7372d9d640 10 monclient(hunting): get_auth_request con 0x7f73741398f0 auth_method 0 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7372d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7372d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7372d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7372d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7372d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7372d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7371d9b640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73740a3a80 0x7f7374138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7371d9b640 10 monclient(hunting): get_auth_request con 0x7f73740a3a80 auth_method 0 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7371d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7371d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7371d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 10 monclient(hunting): handle_auth_done global_id 7425 payload 824 2024-08-19T10:53:32.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 1 -- 172.21.15.89:0/1684712209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73741398f0 msgr2=0x7f737413c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73741398f0 0x7f737413c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 1 -- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73740a3a80 msgr2=0x7f7374138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73740a3a80 0x7f7374138960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 10 monclient: _finish_hunting 0 2024-08-19T10:53:32.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 1 monclient: found mon.noname-a 2024-08-19T10:53:32.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:32.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:32.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 1 -- 172.21.15.89:0/1684712209 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7374123d30 con 0x7f73740a02a0 2024-08-19T10:53:32.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7372d9d640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f73741398f0 0x7f737413c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:32.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 1 -- 172.21.15.89:0/1684712209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73740a02a0 msgr2=0x7f7374138420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f737259c640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73740a02a0 0x7f7374138420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f736c007f20 tx=0x7f736c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.746+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f736c013a00 con 0x7f73740a02a0 2024-08-19T10:53:32.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:32.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 10 monclient: dump: 2024-08-19T10:53:32.759 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.759 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.759 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 10 monclient: _finish_auth 0 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f736c019070 con 0x7f73740a02a0 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737894d640 5 monclient: authenticate success, global_id 7425 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737894d640 10 monclient: _renew_subs 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737894d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737894d640 1 -- 172.21.15.89:0/1684712209 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7374124010 con 0x7f73740a02a0 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f736c013c00 con 0x7f73740a02a0 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 10 monclient: dump: 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.761 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.762 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7370d99640 4 set_mon_vals no callback set 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737894d640 10 monclient: _renew_subs 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737894d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737894d640 1 -- 172.21.15.89:0/1684712209 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73740a8a50 con 0x7f73740a02a0 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737894d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737894d640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737413d610 0x7f737413d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737894d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737894d640 1 -- 172.21.15.89:0/1684712209 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f737413fb60 con 0x7f737413d610 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f737259c640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73740a02a0 0x7f7374138420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f736c007f20 tx=0x7f736c00e570 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f736c04fbb0 osd_map(274..274 src has 246..274) 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f736c013590 con 0x7f73740a02a0 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7371d9b640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737413d610 0x7f737413d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7371d9b640 10 monclient: get_auth_request con 0x7f737413d610 auth_method 0 2024-08-19T10:53:32.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7371d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7371d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7371d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7362ffd640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f735403be00 0x7f735403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7371d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7371d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7371d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7372d9d640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f735403be00 0x7f735403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7371d9b640 10 monclient(hunting): handle_auth_done global_id 84182 payload 824 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7371d9b640 1 -- 172.21.15.89:0/1684712209 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73740a6fd0 con 0x7f737413d610 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.750+0000 7f7372d9d640 10 monclient: get_auth_request con 0x7f735403be00 auth_method 0 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7371d9b640 1 -- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737413d610 msgr2=0x7f737413d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7371d9b640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737413d610 0x7f737413d9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f737413c820 tx=0x7f7368011a60 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7372d9d640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f735403be00 0x7f735403e2c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f73640098a0 tx=0x7f73640a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f736800fe20 con 0x7f737413d610 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7362ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7362ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7362ffd640 10 monclient: dump: 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.765 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f736800f9b0 con 0x7f737413d610 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7362ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7362ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737413d610 msgr2=0x7f737413d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7362ffd640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737413d610 0x7f737413d9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f737413c820 tx=0x7f7368011a60 comp rx=0 tx=0).stop 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737413d610 msgr2=0x7f737413d9f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:32.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.754+0000 7f7362ffd640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737413d610 0x7f737413d9f0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f737413c820 tx=0x7f7368011a60 comp rx=0 tx=0).stop 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.758+0000 7f737894d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.758+0000 7f737894d640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737402d1d0 0x7f737402d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.758+0000 7f737894d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.758+0000 7f737894d640 1 -- 172.21.15.89:0/1684712209 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7374064590 con 0x7f737402d1d0 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737402d1d0 0x7f737402d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 10 monclient: get_auth_request con 0x7f737402d1d0 auth_method 0 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 10 monclient(hunting): handle_auth_done global_id 84191 payload 824 2024-08-19T10:53:32.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 1 -- 172.21.15.89:0/1684712209 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7374124250 con 0x7f737402d1d0 2024-08-19T10:53:32.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=528847697}) 2024-08-19T10:53:32.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=713455623}) 2024-08-19T10:53:32.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=528847697}) 2024-08-19T10:53:32.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=191213939}) 2024-08-19T10:53:32.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=528847697}) 2024-08-19T10:53:32.770 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4073958750}) 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=528847697}) 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=546057118}) 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=528847697}) 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=66635624}) 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=528847697}) 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=320329387}) 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=528847697}) 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1091049763}) 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.771 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=528847697}) 2024-08-19T10:53:32.772 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3103234957}) 2024-08-19T10:53:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 1 -- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737402d1d0 msgr2=0x7f737402d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7371d9b640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737402d1d0 0x7f737402d5e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f737413c820 tx=0x7f73680046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f73680064b0 con 0x7f737402d1d0 2024-08-19T10:53:32.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7362ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7362ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.762+0000 7f7362ffd640 10 monclient: dump: 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.773 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f7368005eb0 con 0x7f737402d1d0 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f7362ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f7362ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737402d1d0 msgr2=0x7f737402d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f7362ffd640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737402d1d0 0x7f737402d5e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f737413c820 tx=0x7f73680046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737402d1d0 msgr2=0x7f737402d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:32.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f7362ffd640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f737402d1d0 0x7f737402d5e0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f737413c820 tx=0x7f73680046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.775 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:32.775 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:32.775 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:32.775 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:32.775 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:32.775 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 21059, 2024-08-19T10:53:32.775 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:32.775 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:32.775 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:32.775 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.776 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.777 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.778 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:32.779 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.780 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:32.781 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.782 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.783 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:32.784 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.785 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.786 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:32.787 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:32.788 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.790 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:32.791 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:32.793 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.794 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.795 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:32.796 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.797 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.798 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:32.798 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f737894d640 1 -- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f735403be00 msgr2=0x7f735403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f737894d640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f735403be00 0x7f735403e2c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f73640098a0 tx=0x7f73640a6000 comp rx=0 tx=0).stop 2024-08-19T10:53:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f737894d640 10 monclient: shutdown 2024-08-19T10:53:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f737894d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f737894d640 1 -- 172.21.15.89:0/1684712209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73740a02a0 msgr2=0x7f7374138420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f737894d640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73740a02a0 0x7f7374138420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f736c007f20 tx=0x7f736c00e570 comp rx=0 tx=0).stop 2024-08-19T10:53:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.766+0000 7f7372d9d640 1 -- 172.21.15.89:0/1684712209 reap_dead start 2024-08-19T10:53:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.770+0000 7f7362ffd640 1 -- 172.21.15.89:0/1684712209 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (unknown 0 0 0) 0x7f736c04fbb0 con 0x7f73740a02a0 2024-08-19T10:53:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.770+0000 7f737894d640 1 -- 172.21.15.89:0/1684712209 shutdown_connections 2024-08-19T10:53:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.770+0000 7f737894d640 1 --2- 172.21.15.89:0/1684712209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73740a02a0 0x7f7374138420 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.770+0000 7f737894d640 1 -- 172.21.15.89:0/1684712209 >> 172.21.15.89:0/1684712209 conn(0x7f737406de40 msgr2=0x7f737413fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.770+0000 7f737894d640 1 -- 172.21.15.89:0/1684712209 shutdown_connections 2024-08-19T10:53:32.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.770+0000 7f737894d640 1 -- 172.21.15.89:0/1684712209 wait complete. 2024-08-19T10:53:32.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3511402617}) 2024-08-19T10:53:32.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3700196439}) 2024-08-19T10:53:32.800 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3511402617}) 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=422328493}) 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3511402617}) 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2597241183}) 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3511402617}) 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4215797407}) 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3511402617}) 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4164895014}) 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.801 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3511402617}) 2024-08-19T10:53:32.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2809599950}) 2024-08-19T10:53:32.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3511402617}) 2024-08-19T10:53:32.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1951141788}) 2024-08-19T10:53:32.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3511402617}) 2024-08-19T10:53:32.802 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.778+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1272553397}) 2024-08-19T10:53:32.803 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:32.803 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:53:32.803 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:53:32.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545076720}) 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=157978719}) 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545076720}) 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=528968291}) 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545076720}) 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=820052572}) 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545076720}) 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=58913070}) 2024-08-19T10:53:32.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545076720}) 2024-08-19T10:53:32.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3249256423}) 2024-08-19T10:53:32.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545076720}) 2024-08-19T10:53:32.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=671834290}) 2024-08-19T10:53:32.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545076720}) 2024-08-19T10:53:32.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=48328413}) 2024-08-19T10:53:32.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:32.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545076720}) 2024-08-19T10:53:32.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:32.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3707095906}) 2024-08-19T10:53:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.950+0000 7fd820379640 10 monclient: get_monmap_and_config 2024-08-19T10:53:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.950+0000 7fd820379640 10 monclient: build_initial_monmap 2024-08-19T10:53:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.950+0000 7fd820379640 10 monclient: monmap: 2024-08-19T10:53:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:32.950 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:32.956741+0000 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:32.956741+0000 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:32.951 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:32.952 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:32.952 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 1 Processor -- start 2024-08-19T10:53:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 1 -- start start 2024-08-19T10:53:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 10 monclient: init 2024-08-19T10:53:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 10 monclient: _add_conns ranks=[6,8,0,5,3,10,9,7,2,1,4] 2024-08-19T10:53:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a6860 0x7fd8180a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 10 monclient(hunting): picked mon.noname-d con 0x7fd8180a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a7180 0x7fd8180a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 10 monclient(hunting): picked mon.noname-e con 0x7fd8180a7180 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd818139620 addr v1:172.21.15.89:6789/0 2024-08-19T10:53:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8180a33d0 con 0x7fd818139620 2024-08-19T10:53:32.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd8180a8190 con 0x7fd8180a6860 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd8180a2ec0 con 0x7fd8180a7180 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd820379640 10 monclient(hunting): authenticate will time out at 1862.773356s 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e115640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a6860 0x7fd8180a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a7180 0x7fd8180a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e916640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd818139620 0x7fd818139a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43880/0 (socket says 172.21.15.89:43880) 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e115640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a6860 0x7fd8180a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46756/0 (socket says 172.21.15.89:46756) 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e916640 1 -- 172.21.15.89:0/2950632053 learned_addr learned my addr 172.21.15.89:0/2950632053 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e115640 10 monclient(hunting): get_auth_request con 0x7fd8180a6860 auth_method 0 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e115640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e115640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e115640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 10 monclient(hunting): get_auth_request con 0x7fd8180a7180 auth_method 0 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 10 monclient(hunting): handle_auth_done global_id 84239 payload 274 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 1 -- 172.21.15.89:0/2950632053 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a6860 msgr2=0x7fd8180a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:32.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 1 --2- 172.21.15.89:0/2950632053 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a6860 0x7fd8180a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 1 -- 172.21.15.89:0/2950632053 >> v1:172.21.15.89:6789/0 conn(0x7fd818139620 legacy=0x7fd818139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:32.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 10 monclient: _finish_hunting 0 2024-08-19T10:53:32.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 1 monclient: found mon.noname-e 2024-08-19T10:53:32.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:32.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:53:32.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81d914640 1 -- 172.21.15.89:0/2950632053 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8180a4cd0 con 0x7fd8180a7180 2024-08-19T10:53:32.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.954+0000 7fd81e115640 1 --2- 172.21.15.89:0/2950632053 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a6860 0x7fd8180a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:32.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d914640 1 -- 172.21.15.89:0/2950632053 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a7180 msgr2=0x7fd8180a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d914640 1 --2- 172.21.15.89:0/2950632053 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a7180 0x7fd8180a7560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd8080097f0 tx=0x7fd80801c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5b973f8b5f3e255f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 1 -- 172.21.15.89:0/2950632053 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8080b0070 con 0x7fd8180a7180 2024-08-19T10:53:32.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:53:32.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 10 monclient: dump: 2024-08-19T10:53:32.961 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.961 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.961 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.961 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.961 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 10 monclient: _finish_auth 0 2024-08-19T10:53:32.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 10 monclient: _check_auth_tickets 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 1 -- 172.21.15.89:0/2950632053 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8080aa810 con 0x7fd8180a7180 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 1 -- 172.21.15.89:0/2950632053 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8080a9300 con 0x7fd8180a7180 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 5 monclient: authenticate success, global_id 84239 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 4 set_mon_vals no callback set 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd81d113640 10 monclient: dump: 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.963 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.964 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 10 monclient: shutdown 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 1 -- 172.21.15.89:0/2950632053 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a7180 msgr2=0x7fd8180a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 1 --2- 172.21.15.89:0/2950632053 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a7180 0x7fd8180a7560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd8080097f0 tx=0x7fd80801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 1 -- 172.21.15.89:0/2950632053 shutdown_connections 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 1 --2- 172.21.15.89:0/2950632053 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a7180 0x7fd8180a7560 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 1 --2- 172.21.15.89:0/2950632053 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a6860 0x7fd8180a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 1 -- 172.21.15.89:0/2950632053 >> 172.21.15.89:0/2950632053 conn(0x7fd818074b40 msgr2=0x7fd8180abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 1 -- 172.21.15.89:0/2950632053 shutdown_connections 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 1 -- 172.21.15.89:0/2950632053 wait complete. 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 10 monclient: build_initial_monmap 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 10 monclient: monmap: 2024-08-19T10:53:32.965 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.958+0000 7fd820379640 1 Processor -- start 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 1 -- start start 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient: init 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient: _add_conns ranks=[8,6,5,7,4,0,1,2,3] 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a6860 0x7fd8181d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient(hunting): picked mon.noname-e con 0x7fd8180a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a7180 0x7fd8181d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient(hunting): picked mon.noname-d con 0x7fd8180a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd818139620 0x7fd8181d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient(hunting): picked mon.noname-h con 0x7fd818139620 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd818144430 con 0x7fd818139620 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd8181442b0 con 0x7fd8180a7180 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd8181445b0 con 0x7fd8180a6860 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient(hunting): authenticate will time out at 1862.780517s 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e115640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a6860 0x7fd8181d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a7180 0x7fd8181d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e115640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a6860 0x7fd8181d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:53012/0 (socket says 172.21.15.89:53012) 2024-08-19T10:53:32.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e115640 1 -- 172.21.15.89:0/3906791153 learned_addr learned my addr 172.21.15.89:0/3906791153 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e115640 10 monclient(hunting): get_auth_request con 0x7fd8180a6860 auth_method 0 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e115640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e115640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e115640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 10 monclient(hunting): get_auth_request con 0x7fd8180a7180 auth_method 0 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e916640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd818139620 0x7fd8181d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e916640 10 monclient(hunting): get_auth_request con 0x7fd818139620 auth_method 0 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e916640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e916640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e916640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 10 monclient(hunting): handle_auth_done global_id 45249 payload 824 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 1 -- 172.21.15.89:0/3906791153 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a6860 msgr2=0x7fd8181d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a6860 0x7fd8181d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 1 -- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd818139620 msgr2=0x7fd8181d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd818139620 0x7fd8181d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 10 monclient: _finish_hunting 0 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 1 monclient: found mon.noname-d 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 1 -- 172.21.15.89:0/3906791153 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81813ec90 con 0x7fd8180a7180 2024-08-19T10:53:32.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81e115640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd8180a6860 0x7fd8181d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 1 -- 172.21.15.89:0/3906791153 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a7180 msgr2=0x7fd8181d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81d914640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a7180 0x7fd8181d1a00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd8080ab000 tx=0x7fd808019140 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd8080b0040 con 0x7fd8180a7180 2024-08-19T10:53:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 10 monclient: dump: 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.975 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 10 monclient: _finish_auth 0 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8080044b0 con 0x7fd8180a7180 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 5 monclient: authenticate success, global_id 45249 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient: _renew_subs 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 1 -- 172.21.15.89:0/3906791153 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8180aae40 con 0x7fd8180a7180 2024-08-19T10:53:32.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd808003a90 con 0x7fd8180a7180 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd80effd640 10 monclient: dump: 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.977 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd81c912640 4 set_mon_vals no callback set 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient: _renew_subs 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.962+0000 7fd820379640 1 -- 172.21.15.89:0/3906791153 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7fd818122c10 con 0x7fd8180a7180 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd820379640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:32.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd820379640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813d330 0x7fd81813d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd820379640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd820379640 1 -- 172.21.15.89:0/3906791153 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd8180ab300 con 0x7fd81813d330 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81d914640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a7180 0x7fd8181d1a00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd8080ab000 tx=0x7fd808019140 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fd8080b9020 osd_map(274..274 src has 246..274) 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd8080bb030 con 0x7fd8180a7180 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813d330 0x7fd81813d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 10 monclient: get_auth_request con 0x7fd81813d330 auth_method 0 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd80effd640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7fc03be30 0x7fd7fc03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e916640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7fc03be30 0x7fd7fc03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e916640 10 monclient: get_auth_request con 0x7fd7fc03be30 auth_method 0 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 10 monclient(hunting): handle_auth_done global_id 64501 payload 824 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 1 -- 172.21.15.89:0/3906791153 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8180a6c40 con 0x7fd81813d330 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 1 -- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813d330 msgr2=0x7fd81813d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd81e115640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813d330 0x7fd81813d710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ef0 tx=0x7fd81400f610 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd814014cd0 con 0x7fd81813d330 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd80effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd80effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.966+0000 7fd80effd640 10 monclient: dump: 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.980 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.981 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.970+0000 7fd81e916640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7fc03be30 0x7fd7fc03e2f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd81813f2b0 tx=0x7fd810009480 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.970+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd814014860 con 0x7fd81813d330 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.970+0000 7fd80effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.970+0000 7fd80effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.970+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813d330 msgr2=0x7fd81813d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.970+0000 7fd80effd640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813d330 0x7fd81813d710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ef0 tx=0x7fd81400f610 comp rx=0 tx=0).stop 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.970+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813d330 msgr2=0x7fd81813d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.970+0000 7fd80effd640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813d330 0x7fd81813d710 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ef0 tx=0x7fd81400f610 comp rx=0 tx=0).stop 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd820379640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd820379640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813dc50 0x7fd81809ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd820379640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd820379640 1 -- 172.21.15.89:0/3906791153 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd818064590 con 0x7fd81813dc50 2024-08-19T10:53:32.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd81e115640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813dc50 0x7fd81809ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd81e115640 10 monclient: get_auth_request con 0x7fd81813dc50 auth_method 0 2024-08-19T10:53:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd81e115640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd81e115640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd81e115640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.974+0000 7fd81e115640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.978+0000 7fd81e115640 10 monclient(hunting): handle_auth_done global_id 64510 payload 824 2024-08-19T10:53:32.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.978+0000 7fd81e115640 1 -- 172.21.15.89:0/3906791153 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd81806cab0 con 0x7fd81813dc50 2024-08-19T10:53:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.978+0000 7fd81e115640 1 -- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813dc50 msgr2=0x7fd81809ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.978+0000 7fd81e115640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813dc50 0x7fd81809ef50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ef0 tx=0x7fd8140046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.978+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd814004490 con 0x7fd81813dc50 2024-08-19T10:53:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.978+0000 7fd80effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.978+0000 7fd80effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:53:32.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.978+0000 7fd80effd640 10 monclient: dump: 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:32.985 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:32.986 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7fd81402b480 con 0x7fd81813dc50 2024-08-19T10:53:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd80effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd80effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813dc50 msgr2=0x7fd81809ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd80effd640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813dc50 0x7fd81809ef50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ef0 tx=0x7fd8140046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813dc50 msgr2=0x7fd81809ef50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:32.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd80effd640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd81813dc50 0x7fd81809ef50 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ef0 tx=0x7fd8140046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:32.986 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:32.987 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:32.987 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:32.987 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:32.987 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 124, 2024-08-19T10:53:32.987 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 279377, 2024-08-19T10:53:32.987 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:32.987 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:32.987 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:32.987 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:32.987 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:53:32.987 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.988 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.989 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:32.990 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:32.991 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:32.992 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.995 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:32.996 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.998 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:32.999 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:33.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:33.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:33.002 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:33.003 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:33.004 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:33.006 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:33.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:33.008 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:33.009 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:33.010 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:33.010 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:33.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd820379640 1 -- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7fc03be30 msgr2=0x7fd7fc03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd820379640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd7fc03be30 0x7fd7fc03e2f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd81813f2b0 tx=0x7fd810009480 comp rx=0 tx=0).stop 2024-08-19T10:53:33.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd820379640 10 monclient: shutdown 2024-08-19T10:53:33.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd820379640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:33.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd820379640 1 -- 172.21.15.89:0/3906791153 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a7180 msgr2=0x7fd8181d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd820379640 1 --2- 172.21.15.89:0/3906791153 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd8180a7180 0x7fd8181d1a00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd8080ab000 tx=0x7fd808019140 comp rx=0 tx=0).stop 2024-08-19T10:53:33.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd81e916640 1 -- 172.21.15.89:0/3906791153 reap_dead start 2024-08-19T10:53:33.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd80effd640 1 -- 172.21.15.89:0/3906791153 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (unknown 0 0 0) 0x7fd8080b9020 con 0x7fd8180a7180 2024-08-19T10:53:33.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd81e916640 1 -- 172.21.15.89:0/3906791153 reap_dead start 2024-08-19T10:53:33.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd820379640 1 -- 172.21.15.89:0/3906791153 shutdown_connections 2024-08-19T10:53:33.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd820379640 1 -- 172.21.15.89:0/3906791153 >> 172.21.15.89:0/3906791153 conn(0x7fd818074b40 msgr2=0x7fd8180a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:33.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd820379640 1 -- 172.21.15.89:0/3906791153 shutdown_connections 2024-08-19T10:53:33.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:32.982+0000 7fd820379640 1 -- 172.21.15.89:0/3906791153 wait complete. 2024-08-19T10:53:33.013 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:53:33.013 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['e', 'i', 'g'] 2024-08-19T10:53:33.013 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-08-19T10:53:33.013 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-08-19T10:53:33.013 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e store 2024-08-19T10:53:33.013 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e sync_force --yes-i-really-mean-it 2024-08-19T10:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient: get_monmap_and_config 2024-08-19T10:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient: build_initial_monmap 2024-08-19T10:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient: monmap: 2024-08-19T10:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:33.135189+0000 2024-08-19T10:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:33.135189+0000 2024-08-19T10:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.129 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:33.130 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 1 Processor -- start 2024-08-19T10:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 1 -- start start 2024-08-19T10:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient: init 2024-08-19T10:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient: _add_conns ranks=[2,1,0,6,4,9,3,5,8,7,10] 2024-08-19T10:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9be80a6860 0x7f9be80a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient(hunting): picked mon.noname-b con 0x7f9be80a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9be80a7180 addr v1:172.21.15.190:6789/0 2024-08-19T10:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9be80a7560 addr v1:172.21.15.89:6789/0 2024-08-19T10:53:33.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9be80a33d0 con 0x7f9be80a7560 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9be813a440 con 0x7f9be80a7180 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9be80a8190 con 0x7f9be80a6860 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9bec8de640 10 monclient(hunting): authenticate will time out at 1862.951739s 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9be659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9be80a6860 0x7f9be80a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9be659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9be80a6860 0x7f9be80a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:59622/0 (socket says 172.21.15.89:59622) 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9be659c640 1 -- 172.21.15.89:0/995253234 learned_addr learned my addr 172.21.15.89:0/995253234 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9be659c640 10 monclient(hunting): get_auth_request con 0x7f9be80a6860 auth_method 0 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9be6d9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9be80a7560 0x7f9be8139d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43886/0 (socket says 172.21.15.89:43886) 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9be659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9be659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.130+0000 7f9be659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.134+0000 7f9be659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.134+0000 7f9be659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.134+0000 7f9be659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.134+0000 7f9be659c640 10 monclient(hunting): handle_auth_done global_id 74387 payload 274 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.134+0000 7f9be659c640 1 -- 172.21.15.89:0/995253234 >> v1:172.21.15.190:6789/0 conn(0x7f9be80a7180 legacy=0x7f9be8139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.134+0000 7f9be659c640 1 -- 172.21.15.89:0/995253234 >> v1:172.21.15.89:6789/0 conn(0x7f9be80a7560 legacy=0x7f9be8139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.134+0000 7f9be659c640 10 monclient: _finish_hunting 0 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.134+0000 7f9be659c640 1 monclient: found mon.noname-b 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.134+0000 7f9be659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.134+0000 7f9be659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:53:33.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.134+0000 7f9be659c640 1 -- 172.21.15.89:0/995253234 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be80a2ec0 con 0x7f9be80a6860 2024-08-19T10:53:33.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 1 -- 172.21.15.89:0/995253234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9be80a6860 msgr2=0x7f9be80a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 1 --2- 172.21.15.89:0/995253234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9be80a6860 0x7f9be80a6c40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80098a0 tx=0x7f9bd801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d4631dbe1ce5d46 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 1 -- 172.21.15.89:0/995253234 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9bd80b0070 con 0x7f9be80a6860 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 10 monclient: dump: 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.140 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 10 monclient: _finish_auth 0 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 10 monclient: _check_auth_tickets 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:33.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 1 -- 172.21.15.89:0/995253234 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd80aa810 con 0x7f9be80a6860 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 1 -- 172.21.15.89:0/995253234 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9bd80aae30 con 0x7f9be80a6860 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be559a640 10 monclient: dump: 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.142 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 5 monclient: authenticate success, global_id 74387 2024-08-19T10:53:33.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 4 set_mon_vals no callback set 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient: shutdown 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 -- 172.21.15.89:0/995253234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9be80a6860 msgr2=0x7f9be80a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 --2- 172.21.15.89:0/995253234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9be80a6860 0x7f9be80a6c40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80098a0 tx=0x7f9bd801c4d0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 -- 172.21.15.89:0/995253234 shutdown_connections 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 --2- 172.21.15.89:0/995253234 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9be80a6860 0x7f9be80a6c40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 -- 172.21.15.89:0/995253234 >> 172.21.15.89:0/995253234 conn(0x7f9be8074b40 msgr2=0x7f9be80abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 -- 172.21.15.89:0/995253234 shutdown_connections 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 -- 172.21.15.89:0/995253234 wait complete. 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient: build_initial_monmap 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient: monmap: 2024-08-19T10:53:33.144 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 Processor -- start 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 -- start start 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient: init 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient: _add_conns ranks=[6,1,8,7,5,0,2,3,4] 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 0x7f9be81406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient(hunting): picked mon.noname-d con 0x7f9be80a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9be80a7180 0x7f9be81d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient(hunting): picked mon.noname-f con 0x7f9be80a7180 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be80a7560 0x7f9be81d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient(hunting): picked mon.noname-e con 0x7f9be80a7560 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:33.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9be81454c0 con 0x7f9be80a7180 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9be80a79f0 con 0x7f9be80a6860 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9be8145640 con 0x7f9be80a7560 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9bec8de640 10 monclient(hunting): authenticate will time out at 1862.959436s 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 0x7f9be81406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be6d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be80a7560 0x7f9be81d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 0x7f9be81406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46778/0 (socket says 172.21.15.89:46778) 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 1 -- 172.21.15.89:0/495124380 learned_addr learned my addr 172.21.15.89:0/495124380 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:33.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 10 monclient(hunting): get_auth_request con 0x7f9be80a6860 auth_method 0 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be6d9d640 10 monclient(hunting): get_auth_request con 0x7f9be80a7560 auth_method 0 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:33.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be5d9b640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9be80a7180 0x7f9be81d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be5d9b640 10 monclient(hunting): get_auth_request con 0x7f9be80a7180 auth_method 0 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 10 monclient(hunting): handle_auth_done global_id 45258 payload 824 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be80a7560 msgr2=0x7f9be81d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be80a7560 0x7f9be81d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9be80a7180 msgr2=0x7f9be81d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9be80a7180 0x7f9be81d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 10 monclient: _finish_hunting 0 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 1 monclient: found mon.noname-d 2024-08-19T10:53:33.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.138+0000 7f9be659c640 1 -- 172.21.15.89:0/495124380 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be80a9060 con 0x7f9be80a6860 2024-08-19T10:53:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.142+0000 7f9be6d9d640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be80a7560 0x7f9be81d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.142+0000 7f9be659c640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 msgr2=0x7f9be81406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:53:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.142+0000 7f9be659c640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 msgr2=0x7f9be81406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:53:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.142+0000 7f9be659c640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 0x7f9be81406d0 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f9bd801cd70 tx=0x7f9bd8019270 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:33.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.142+0000 7f9be659c640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 0x7f9be81406d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:33.253 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1855176481}) 2024-08-19T10:53:33.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=204823497}) 2024-08-19T10:53:33.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1855176481}) 2024-08-19T10:53:33.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=844517252}) 2024-08-19T10:53:33.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1855176481}) 2024-08-19T10:53:33.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1502822814}) 2024-08-19T10:53:33.254 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1855176481}) 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1106944418}) 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1855176481}) 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2829108037}) 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1855176481}) 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=864474741}) 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1855176481}) 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4074592451}) 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1855176481}) 2024-08-19T10:53:33.255 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.254+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2680878548}) 2024-08-19T10:53:33.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2706657334}) 2024-08-19T10:53:33.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=285412605}) 2024-08-19T10:53:33.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2706657334}) 2024-08-19T10:53:33.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3618802784}) 2024-08-19T10:53:33.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2706657334}) 2024-08-19T10:53:33.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3168315302}) 2024-08-19T10:53:33.259 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2706657334}) 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1249971628}) 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2706657334}) 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1165732660}) 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2706657334}) 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=178144808}) 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2706657334}) 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=681927906}) 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2706657334}) 2024-08-19T10:53:33.260 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.258+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3582163539}) 2024-08-19T10:53:33.263 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.263 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230597234}) 2024-08-19T10:53:33.263 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4055659934}) 2024-08-19T10:53:33.263 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.263 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230597234}) 2024-08-19T10:53:33.263 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=868365095}) 2024-08-19T10:53:33.263 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230597234}) 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4181927906}) 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230597234}) 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3337175365}) 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.262+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230597234}) 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3470529429}) 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230597234}) 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3252837809}) 2024-08-19T10:53:33.264 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.265 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230597234}) 2024-08-19T10:53:33.265 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2583707106}) 2024-08-19T10:53:33.265 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.265 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230597234}) 2024-08-19T10:53:33.265 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.266+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4013902337}) 2024-08-19T10:53:33.267 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474672923}) 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3187108095}) 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474672923}) 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1362504461}) 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474672923}) 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3959957622}) 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474672923}) 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2864187652}) 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.268 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474672923}) 2024-08-19T10:53:33.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1618160794}) 2024-08-19T10:53:33.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474672923}) 2024-08-19T10:53:33.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1522071831}) 2024-08-19T10:53:33.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474672923}) 2024-08-19T10:53:33.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2238638971}) 2024-08-19T10:53:33.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474672923}) 2024-08-19T10:53:33.269 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.270+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2538457257}) 2024-08-19T10:53:33.272 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.272 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2650609448}) 2024-08-19T10:53:33.272 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2958536821}) 2024-08-19T10:53:33.272 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.272 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2650609448}) 2024-08-19T10:53:33.272 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3588143195}) 2024-08-19T10:53:33.272 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.272 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2650609448}) 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1843705988}) 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2650609448}) 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=38186260}) 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2650609448}) 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=829890647}) 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2650609448}) 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=537314611}) 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2650609448}) 2024-08-19T10:53:33.273 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3720976452}) 2024-08-19T10:53:33.274 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:53:33.274 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2650609448}) 2024-08-19T10:53:33.274 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:53:33.274+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=332182345}) 2024-08-19T10:53:33.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 0x7f9be81406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 10 monclient: get_auth_request con 0x7f9be80a6860 auth_method 0 2024-08-19T10:53:33.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 0 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 0x7f9be81406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-08-19T10:53:33.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 0x7f9be81406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 10 monclient: ms_handle_reset current mon [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:33.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:33.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 msgr2=0x7f9be81406d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:33.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9be80a6860 0x7f9be81406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 10 monclient: _add_conns ranks=[0,4,7,5,2,3] 2024-08-19T10:53:33.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc80035e0 0x7f9bc8005a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f9bc80035e0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:33.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bc80065d0 0x7f9bc8008a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f9bc80065d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9bc8009640 0x7f9bc800bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 10 monclient(hunting): picked mon.noname-i con 0x7f9bc8009640 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc800c6f0 con 0x7f9bc80035e0 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9bc800c870 con 0x7f9bc80065d0 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9bc800ca70 con 0x7f9bc8009640 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9bd6ffd640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc80035e0 0x7f9bc8005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be5d9b640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bc80065d0 0x7f9bc8008a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 10 monclient(hunting): get_auth_request con 0x7f9bc80035e0 auth_method 0 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be5d9b640 10 monclient(hunting): get_auth_request con 0x7f9bc80065d0 auth_method 0 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be5d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:33.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be6d9d640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9bc8009640 0x7f9bc800bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be6d9d640 10 monclient(hunting): get_auth_request con 0x7f9bc8009640 auth_method 0 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be6d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 10 monclient(hunting): handle_auth_done global_id 45258 payload 293 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9bc8009640 msgr2=0x7f9bc800bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9bc8009640 0x7f9bc800bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bc80065d0 msgr2=0x7f9bc8008a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9bc80065d0 0x7f9bc8008a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 10 monclient: _finish_hunting 0 2024-08-19T10:53:33.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 1 monclient: found mon.noname-a 2024-08-19T10:53:33.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:33.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:33.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be659c640 1 -- 172.21.15.89:0/495124380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc800cd30 con 0x7f9bc80035e0 2024-08-19T10:53:33.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be6d9d640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9bc8009640 0x7f9bc800bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:53:33.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.346+0000 7f9be6d9d640 1 -- 172.21.15.89:0/495124380 reap_dead start 2024-08-19T10:53:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9be659c640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc80035e0 msgr2=0x7f9bc8005a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9be659c640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc80035e0 0x7f9bc8005a30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80ae040 tx=0x7f9bd80a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:33.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9bd80b0070 con 0x7f9bc80035e0 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 10 monclient: dump: 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:33.353 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bec8de640 5 monclient: authenticate success, global_id 45258 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd8000e50 con 0x7f9bc80035e0 2024-08-19T10:53:33.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bec8de640 10 monclient: _renew_subs 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bec8de640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bec8de640 1 -- 172.21.15.89:0/495124380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be80a2670 con 0x7f9bc80035e0 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9bd80042d0 con 0x7f9bc80035e0 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd7fff640 4 set_mon_vals no callback set 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 10 monclient: dump: 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.355 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bec8de640 10 monclient: _renew_subs 2024-08-19T10:53:33.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bec8de640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bec8de640 1 -- 172.21.15.89:0/495124380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be813fb70 con 0x7f9bc80035e0 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bec8de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bec8de640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81406d0 0x7f9be81d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bec8de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bec8de640 1 -- 172.21.15.89:0/495124380 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9be81229a0 con 0x7f9be81406d0 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9bd8003620 con 0x7f9bc80035e0 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9be5d9b640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81406d0 0x7f9be81d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9be5d9b640 10 monclient: get_auth_request con 0x7f9be81406d0 auth_method 0 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9be5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9be5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9be5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9be5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:33.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9be5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.350+0000 7f9be5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9bc8040590 0x7f9bc8042a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9be5d9b640 10 monclient(hunting): handle_auth_done global_id 84266 payload 824 2024-08-19T10:53:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f9bd80e9a10 con 0x7f9bc80035e0 2024-08-19T10:53:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9be5d9b640 1 -- 172.21.15.89:0/495124380 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9be8064590 con 0x7f9be81406d0 2024-08-19T10:53:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9be6d9d640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9bc8040590 0x7f9bc8042a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9be6d9d640 10 monclient: get_auth_request con 0x7f9bc8040590 auth_method 0 2024-08-19T10:53:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9be5d9b640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81406d0 msgr2=0x7f9be81d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9be5d9b640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81406d0 0x7f9be81d19e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9be81401e0 tx=0x7f9bdc011a50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9bdc019020 con 0x7f9be81406d0 2024-08-19T10:53:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:53:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 10 monclient: dump: 2024-08-19T10:53:33.359 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:33.360 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:33.361 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9bdc006cc0 con 0x7f9be81406d0 2024-08-19T10:53:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81406d0 msgr2=0x7f9be81d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81406d0 0x7f9be81d19e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9be81401e0 tx=0x7f9bdc011a50 comp rx=0 tx=0).stop 2024-08-19T10:53:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81406d0 msgr2=0x7f9be81d19e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9bd6ffd640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81406d0 0x7f9be81d19e0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9be81401e0 tx=0x7f9bdc011a50 comp rx=0 tx=0).stop 2024-08-19T10:53:33.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.354+0000 7f9be6d9d640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9bc8040590 0x7f9bc8042a50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80098a0 tx=0x7f9be0009610 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9bec8de640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:53:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9bec8de640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81d1f20 0x7f9be813e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9bec8de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9bec8de640 1 -- 172.21.15.89:0/495124380 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9be80a95b0 con 0x7f9be81d1f20 2024-08-19T10:53:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9be5d9b640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81d1f20 0x7f9be813e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9be5d9b640 10 monclient: get_auth_request con 0x7f9be81d1f20 auth_method 0 2024-08-19T10:53:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9be5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9be5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9be5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9be5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9be5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9be5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9be5d9b640 10 monclient(hunting): handle_auth_done global_id 84275 payload 824 2024-08-19T10:53:33.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.362+0000 7f9be5d9b640 1 -- 172.21.15.89:0/495124380 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7f9be8140ab0 con 0x7f9be81d1f20 2024-08-19T10:53:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9be5d9b640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81d1f20 msgr2=0x7f9be813e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9be5d9b640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81d1f20 0x7f9be813e110 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9be81401e0 tx=0x7f9bdc003700 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9bdc019040 con 0x7f9be81d1f20 2024-08-19T10:53:33.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bd6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bd6ffd640 10 monclient: dump: 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:33.365 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:33.366 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7f9bdc015470 con 0x7f9be81d1f20 2024-08-19T10:53:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bd6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:53:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bd6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81d1f20 msgr2=0x7f9be813e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bd6ffd640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81d1f20 0x7f9be813e110 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9be81401e0 tx=0x7f9bdc003700 comp rx=0 tx=0).stop 2024-08-19T10:53:33.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bd6ffd640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81d1f20 msgr2=0x7f9be813e110 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:33.367 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:33.367 INFO:teuthology.orchestra.run.smithi089.stdout: "ret": 0, 2024-08-19T10:53:33.367 INFO:teuthology.orchestra.run.smithi089.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-08-19T10:53:33.367 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bd6ffd640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81d1f20 0x7f9be813e110 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9bc8040590 msgr2=0x7f9bc8042a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9bc8040590 0x7f9bc8042a50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80098a0 tx=0x7f9be0009610 comp rx=0 tx=0).stop 2024-08-19T10:53:33.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 10 monclient: shutdown 2024-08-19T10:53:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 -- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc80035e0 msgr2=0x7f9bc8005a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc80035e0 0x7f9bc8005a30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80ae040 tx=0x7f9bd80a6000 comp rx=0 tx=0).stop 2024-08-19T10:53:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 -- 172.21.15.89:0/495124380 shutdown_connections 2024-08-19T10:53:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9bc8040590 0x7f9bc8042a50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc80035e0 0x7f9bc8005a30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81406d0 0x7f9be81d19e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 --2- 172.21.15.89:0/495124380 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f9be81d1f20 0x7f9be813e110 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 -- 172.21.15.89:0/495124380 >> 172.21.15.89:0/495124380 conn(0x7f9be8074b40 msgr2=0x7f9be809d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 -- 172.21.15.89:0/495124380 shutdown_connections 2024-08-19T10:53:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.366+0000 7f9bec8de640 1 -- 172.21.15.89:0/495124380 wait complete. 2024-08-19T10:53:33.386 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-08-19T10:53:33.386 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-08-19T10:53:33.386 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:53:33.401 INFO:tasks.ceph.mon.e:Stopped 2024-08-19T10:53:33.401 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-08-19T10:53:33.401 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-08-19T10:53:33.401 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-08-19T10:53:33.402 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:53:33.428 INFO:tasks.ceph.mon.i:Stopped 2024-08-19T10:53:33.428 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-08-19T10:53:33.428 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-08-19T10:53:33.428 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-08-19T10:53:33.428 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:53:33.502 INFO:tasks.ceph.mon.g:Stopped 2024-08-19T10:53:33.503 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-08-19T10:53:33.503 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:53:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient: get_monmap_and_config 2024-08-19T10:53:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient: build_initial_monmap 2024-08-19T10:53:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient: monmap: 2024-08-19T10:53:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:33.628 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:33.634283+0000 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:33.634283+0000 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:33.629 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:33.630 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:33.630 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:33.630 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:33.630 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 1 Processor -- start 2024-08-19T10:53:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 1 -- start start 2024-08-19T10:53:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient: init 2024-08-19T10:53:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient: _add_conns ranks=[0,9,4,7,5,6,8,10,3,1,2] 2024-08-19T10:53:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff00006c930 addr v1:172.21.15.89:6789/0 2024-08-19T10:53:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff000074320 0x7ff00006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient(hunting): picked mon.noname-a con 0x7ff000074320 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff00013c740 0x7ff00013eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient(hunting): picked mon.noname-c con 0x7ff00013c740 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:33.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0000a3050 con 0x7ff00006c930 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff0000a51a0 con 0x7ff000074320 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff0000a53f0 con 0x7ff00013c740 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7ff005caf640 10 monclient(hunting): authenticate will time out at 1863.451578s 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feffffff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff00013c740 0x7ff00013eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feff7fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff000074320 0x7ff00006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feffffff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff00013c740 0x7ff00013eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53038/0 (socket says 172.21.15.89:53038) 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feffffff640 1 -- 172.21.15.89:0/4050077684 learned_addr learned my addr 172.21.15.89:0/4050077684 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feff7fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff000074320 0x7ff00006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53076/0 (socket says 172.21.15.89:53076) 2024-08-19T10:53:33.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feffffff640 10 monclient(hunting): get_auth_request con 0x7ff00013c740 auth_method 0 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feffffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feffffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feffffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feff7fff640 10 monclient(hunting): get_auth_request con 0x7ff000074320 auth_method 0 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feff7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feff7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feff7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feff7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feff7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feff7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feffffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feffffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:33.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.630+0000 7feffffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:33.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.634+0000 7feff7fff640 10 monclient(hunting): handle_auth_done global_id 7434 payload 274 2024-08-19T10:53:33.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.634+0000 7feff7fff640 1 -- 172.21.15.89:0/4050077684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff00013c740 msgr2=0x7ff00013eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:33.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.634+0000 7feff7fff640 1 --2- 172.21.15.89:0/4050077684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff00013c740 0x7ff00013eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.634+0000 7feff7fff640 1 -- 172.21.15.89:0/4050077684 >> v1:172.21.15.89:6789/0 conn(0x7ff00006c930 legacy=0x7ff00006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:33.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.634+0000 7feff7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:53:33.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.634+0000 7feff7fff640 1 monclient: found mon.noname-a 2024-08-19T10:53:33.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.634+0000 7feff7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:33.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.634+0000 7feff7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:33.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.634+0000 7feff7fff640 1 -- 172.21.15.89:0/4050077684 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0000a49a0 con 0x7ff000074320 2024-08-19T10:53:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feff7fff640 1 -- 172.21.15.89:0/4050077684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff000074320 msgr2=0x7ff00006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feff7fff640 1 --2- 172.21.15.89:0/4050077684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff000074320 0x7ff00006d420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fefe80097c0 tx=0x7fefe801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed30bab87fde92f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 1 -- 172.21.15.89:0/4050077684 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fefe80b0070 con 0x7ff000074320 2024-08-19T10:53:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 10 monclient: dump: 2024-08-19T10:53:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.641 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 10 monclient: _finish_auth 0 2024-08-19T10:53:33.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 1 -- 172.21.15.89:0/4050077684 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe80aa810 con 0x7ff000074320 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 1 -- 172.21.15.89:0/4050077684 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fefe80a9300 con 0x7ff000074320 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 5 monclient: authenticate success, global_id 7434 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 4 set_mon_vals no callback set 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffeffd640 10 monclient: dump: 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:33.643 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:33.644 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient: shutdown 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 -- 172.21.15.89:0/4050077684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff000074320 msgr2=0x7ff00006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 --2- 172.21.15.89:0/4050077684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff000074320 0x7ff00006d420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fefe80097c0 tx=0x7fefe801c360 comp rx=0 tx=0).stop 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 -- 172.21.15.89:0/4050077684 shutdown_connections 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 --2- 172.21.15.89:0/4050077684 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff00013c740 0x7ff00013eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 --2- 172.21.15.89:0/4050077684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff000074320 0x7ff00006d420 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 -- 172.21.15.89:0/4050077684 >> 172.21.15.89:0/4050077684 conn(0x7ff0000a8f60 msgr2=0x7ff0000a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 -- 172.21.15.89:0/4050077684 shutdown_connections 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 -- 172.21.15.89:0/4050077684 wait complete. 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient: build_initial_monmap 2024-08-19T10:53:33.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient: monmap: 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:33.646 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 Processor -- start 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 -- start start 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient: init 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient: _add_conns ranks=[8,5,0,4,6,7,1,2,3] 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff00006c930 0x7ff0001d1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient(hunting): picked mon.noname-e con 0x7ff00006c930 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff000074320 0x7ff00013acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient(hunting): picked mon.noname-h con 0x7ff000074320 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:33.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff00013c740 0x7ff00013b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient(hunting): picked mon.noname-a con 0x7ff00013c740 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff000141a10 con 0x7ff00013c740 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff000141890 con 0x7ff000074320 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff000141b90 con 0x7ff00006c930 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7ff005caf640 10 monclient(hunting): authenticate will time out at 1863.458762s 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7fefff7fe640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff00006c930 msgr2=0x7ff0001d1100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7fefff7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff00006c930 0x7ff0001d1100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff00013c740 0x7ff00013b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff00013c740 0x7ff00013b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53088/0 (socket says 172.21.15.89:53088) 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 1 -- 172.21.15.89:0/601264956 learned_addr learned my addr 172.21.15.89:0/601264956 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 10 monclient(hunting): get_auth_request con 0x7ff00013c740 auth_method 0 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 10 monclient(hunting): handle_auth_done global_id 7443 payload 824 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 1 -- 172.21.15.89:0/601264956 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff00006c930 msgr2=0x7ff0001d1100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff00006c930 0x7ff0001d1100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 1 -- 172.21.15.89:0/601264956 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff000074320 msgr2=0x7ff00013acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff000074320 0x7ff00013acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 10 monclient: _finish_hunting 0 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 1 monclient: found mon.noname-a 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.638+0000 7feffffff640 1 -- 172.21.15.89:0/601264956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff000125cb0 con 0x7ff00013c740 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffffff640 1 -- 172.21.15.89:0/601264956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff00013c740 msgr2=0x7ff00013b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffffff640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff00013c740 0x7ff00013b1f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7feff800ec10 tx=0x7feff800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 1 -- 172.21.15.89:0/601264956 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7feff80039a0 con 0x7ff00013c740 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 10 monclient: dump: 2024-08-19T10:53:33.650 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:33.651 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 10 monclient: _finish_auth 0 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 10 monclient: _check_auth_tickets 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 1 -- 172.21.15.89:0/601264956 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff8003b60 con 0x7ff00013c740 2024-08-19T10:53:33.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 1 -- 172.21.15.89:0/601264956 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7feff8003d10 con 0x7ff00013c740 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 10 monclient: dump: 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:33.653 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:33.654 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:33.654 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:33.654 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:33.654 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:33.654 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:33.654 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:33.654 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:33.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffdffb640 4 set_mon_vals no callback set 2024-08-19T10:53:33.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7ff005caf640 5 monclient: authenticate success, global_id 7443 2024-08-19T10:53:33.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7ff005caf640 10 monclient: _renew_subs 2024-08-19T10:53:33.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7ff005caf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:33.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7ff005caf640 1 -- 172.21.15.89:0/601264956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00009ee80 con 0x7ff00013c740 2024-08-19T10:53:33.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7ff005caf640 10 monclient: _renew_subs 2024-08-19T10:53:33.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7ff005caf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:33.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7ff005caf640 1 -- 172.21.15.89:0/601264956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff000127760 con 0x7ff00013c740 2024-08-19T10:53:33.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7ff005caf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:33.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7ff005caf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:33.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7ff005caf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:33.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7ff005caf640 1 -- 172.21.15.89:0/601264956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00009f0b0 con 0x7ff00013c740 2024-08-19T10:53:33.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 1 -- 172.21.15.89:0/601264956 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7feff8012050 con 0x7ff00013c740 2024-08-19T10:53:33.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefe003be00 0x7fefe003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:33.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7feffcff9640 1 -- 172.21.15.89:0/601264956 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7feff804fbc0 con 0x7ff00013c740 2024-08-19T10:53:33.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7fefff7fe640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefe003be00 0x7fefe003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:33.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.642+0000 7fefff7fe640 10 monclient: get_auth_request con 0x7fefe003be00 auth_method 0 2024-08-19T10:53:33.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.646+0000 7fefff7fe640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefe003be00 0x7fefe003e2c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7feff0003780 tx=0x7feff0007a60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:33.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.646+0000 7feffcff9640 1 -- 172.21.15.89:0/601264956 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff00009f0b0 con 0x7ff00013c740 2024-08-19T10:53:33.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.646+0000 7feffcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:33.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.646+0000 7feffcff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:33.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.882+0000 7ff005caf640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:53:33.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.882+0000 7ff005caf640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:33.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.882+0000 7ff005caf640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:33.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.882+0000 7ff005caf640 1 -- 172.21.15.89:0/601264956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff00006cd10 con 0x7ff00013c740 2024-08-19T10:53:33.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7feffcff9640 1 -- 172.21.15.89:0/601264956 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7feff8022020 con 0x7ff00013c740 2024-08-19T10:53:33.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7feffcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:33.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7feffcff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:33.884 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":124,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:53:33.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 -- 172.21.15.89:0/601264956 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefe003be00 msgr2=0x7fefe003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefe003be00 0x7fefe003e2c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7feff0003780 tx=0x7feff0007a60 comp rx=0 tx=0).stop 2024-08-19T10:53:33.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 10 monclient: shutdown 2024-08-19T10:53:33.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:33.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 -- 172.21.15.89:0/601264956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff00013c740 msgr2=0x7ff00013b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:33.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff00013c740 0x7ff00013b1f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7feff800ec10 tx=0x7feff800c490 comp rx=0 tx=0).stop 2024-08-19T10:53:33.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 -- 172.21.15.89:0/601264956 shutdown_connections 2024-08-19T10:53:33.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff00013c740 0x7ff00013b1f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff000074320 0x7ff00013acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fefe003be00 0x7fefe003e2c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 --2- 172.21.15.89:0/601264956 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff00006c930 0x7ff0001d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:33.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 -- 172.21.15.89:0/601264956 >> 172.21.15.89:0/601264956 conn(0x7ff0000a8f60 msgr2=0x7ff0000a82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:33.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 -- 172.21.15.89:0/601264956 shutdown_connections 2024-08-19T10:53:33.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:33.886+0000 7ff005caf640 1 -- 172.21.15.89:0/601264956 wait complete. 2024-08-19T10:53:36.904 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:53:37.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient: get_monmap_and_config 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient: build_initial_monmap 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient: monmap: 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:37.042009+0000 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:37.042009+0000 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:37.036 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:37.037 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:37.037 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:37.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 1 Processor -- start 2024-08-19T10:53:37.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 1 -- start start 2024-08-19T10:53:37.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient: init 2024-08-19T10:53:37.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:37.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient: _add_conns ranks=[4,8,5,0,3,2,6,10,7,1,9] 2024-08-19T10:53:37.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2b80a6860 0x7ff2b80a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:37.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient(hunting): picked mon.noname-c con 0x7ff2b80a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:37.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2b80a7180 0x7ff2b80a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient(hunting): picked mon.noname-e con 0x7ff2b80a7180 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff2b8139620 0x7ff2b8139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient(hunting): picked mon.noname-h con 0x7ff2b8139620 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff2b80a8190 con 0x7ff2b80a6860 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff2b80a33d0 con 0x7ff2b8139620 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff2b80a2ec0 con 0x7ff2b80a7180 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2be8cf640 10 monclient(hunting): authenticate will time out at 1866.858678s 2024-08-19T10:53:37.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b77fe640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2b80a7180 msgr2=0x7ff2b80a7560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b77fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2b80a7180 0x7ff2b80a7560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2b80a6860 0x7ff2b80a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2b80a6860 0x7ff2b80a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:53040/0 (socket says 172.21.15.89:53040) 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 1 -- 172.21.15.89:0/1995878523 learned_addr learned my addr 172.21.15.89:0/1995878523 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 10 monclient(hunting): get_auth_request con 0x7ff2b80a6860 auth_method 0 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:37.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2bce6c640 1 --2- 172.21.15.89:0/1995878523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff2b8139620 0x7ff2b8139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2bce6c640 10 monclient(hunting): get_auth_request con 0x7ff2b8139620 auth_method 0 2024-08-19T10:53:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2bce6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2bce6c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2bce6c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 10 monclient(hunting): handle_auth_done global_id 84217 payload 274 2024-08-19T10:53:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 1 -- 172.21.15.89:0/1995878523 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2b80a7180 msgr2=0x7ff2b80a7560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 1 --2- 172.21.15.89:0/1995878523 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2b80a7180 0x7ff2b80a7560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 1 -- 172.21.15.89:0/1995878523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff2b8139620 msgr2=0x7ff2b8139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 1 --2- 172.21.15.89:0/1995878523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff2b8139620 0x7ff2b8139a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:53:37.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 1 monclient: found mon.noname-c 2024-08-19T10:53:37.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:37.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:53:37.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.037+0000 7ff2b7fff640 1 -- 172.21.15.89:0/1995878523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b80a4cd0 con 0x7ff2b80a6860 2024-08-19T10:53:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b7fff640 1 -- 172.21.15.89:0/1995878523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2b80a6860 msgr2=0x7ff2b80a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b7fff640 1 --2- 172.21.15.89:0/1995878523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2b80a6860 0x7ff2b80a6c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80098a0 tx=0x7ff2a801c460 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a60303520794a32e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 1 -- 172.21.15.89:0/1995878523 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2a80b0070 con 0x7ff2b80a6860 2024-08-19T10:53:37.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 10 monclient: dump: 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:37.049 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:37.050 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:53:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 1 -- 172.21.15.89:0/1995878523 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a80aa810 con 0x7ff2b80a6860 2024-08-19T10:53:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2be8cf640 5 monclient: authenticate success, global_id 84217 2024-08-19T10:53:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2be8cf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:37.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 1 -- 172.21.15.89:0/1995878523 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2a80a9300 con 0x7ff2b80a6860 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2b6ffd640 10 monclient: dump: 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:37.051 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2be8cf640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.041+0000 7ff2be8cf640 4 set_mon_vals no callback set 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient: shutdown 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1995878523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2b80a6860 msgr2=0x7ff2b80a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 --2- 172.21.15.89:0/1995878523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2b80a6860 0x7ff2b80a6c40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80098a0 tx=0x7ff2a801c460 comp rx=0 tx=0).stop 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1995878523 shutdown_connections 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 --2- 172.21.15.89:0/1995878523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff2b8139620 0x7ff2b8139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 --2- 172.21.15.89:0/1995878523 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff2b80a7180 0x7ff2b80a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:37.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 --2- 172.21.15.89:0/1995878523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff2b80a6860 0x7ff2b80a6c40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1995878523 >> 172.21.15.89:0/1995878523 conn(0x7ff2b8074b40 msgr2=0x7ff2b80abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1995878523 shutdown_connections 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1995878523 wait complete. 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient: build_initial_monmap 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient: monmap: 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:37.053 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 Processor -- start 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 -- start start 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient: init 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient: _add_conns ranks=[3,6,0,1,8,2,4,7,5] 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2b80a6860 0x7ff2b8134e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:37.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient(hunting): picked mon.noname-g con 0x7ff2b80a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b80a7180 0x7ff2b809c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient(hunting): picked mon.noname-d con 0x7ff2b80a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b8139620 0x7ff2b809cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient(hunting): picked mon.noname-a con 0x7ff2b8139620 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b8143210 con 0x7ff2b8139620 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b8143090 con 0x7ff2b80a6860 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff2b8143390 con 0x7ff2b80a7180 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2be8cf640 10 monclient(hunting): authenticate will time out at 1866.865982s 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b80a7180 0x7ff2b809c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b7fff640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2b80a6860 msgr2=0x7ff2b8134e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:53:37.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b80a7180 0x7ff2b809c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:46796/0 (socket says 172.21.15.89:46796) 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2bce6c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b8139620 0x7ff2b809cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 1 -- 172.21.15.89:0/1980734540 learned_addr learned my addr 172.21.15.89:0/1980734540 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 10 monclient(hunting): get_auth_request con 0x7ff2b80a7180 auth_method 0 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b7fff640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2b80a6860 0x7ff2b8134e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2bce6c640 10 monclient(hunting): get_auth_request con 0x7ff2b8139620 auth_method 0 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2bce6c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2bce6c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2bce6c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:37.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2bce6c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2bce6c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2bce6c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 10 monclient(hunting): handle_auth_done global_id 45267 payload 824 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 1 -- 172.21.15.89:0/1980734540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2b80a6860 msgr2=0x7ff2b8134e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2b80a6860 0x7ff2b8134e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 1 -- 172.21.15.89:0/1980734540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b8139620 msgr2=0x7ff2b809cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b8139620 0x7ff2b809cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 1 monclient: found mon.noname-d 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2b77fe640 1 -- 172.21.15.89:0/1980734540 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b809e130 con 0x7ff2b80a7180 2024-08-19T10:53:37.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.045+0000 7ff2bce6c640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b8139620 0x7ff2b809cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b77fe640 1 -- 172.21.15.89:0/1980734540 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b80a7180 msgr2=0x7ff2b809c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b77fe640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b80a7180 0x7ff2b809c9e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b550 tx=0x7ff2ac00ba10 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 1 -- 172.21.15.89:0/1980734540 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2ac0092d0 con 0x7ff2b80a7180 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 10 monclient: dump: 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:37.058 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 10 monclient: _finish_auth 0 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 1 -- 172.21.15.89:0/1980734540 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2ac015070 con 0x7ff2b80a7180 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2be8cf640 5 monclient: authenticate success, global_id 45267 2024-08-19T10:53:37.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2be8cf640 10 monclient: _renew_subs 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2be8cf640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1980734540 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b809fc50 con 0x7ff2b80a7180 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 1 -- 172.21.15.89:0/1980734540 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2ac003e00 con 0x7ff2b80a7180 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 10 monclient: dump: 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:37.060 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b5ffb640 4 set_mon_vals no callback set 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2be8cf640 10 monclient: _renew_subs 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2be8cf640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1980734540 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b80a01b0 con 0x7ff2b80a7180 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2be8cf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2be8cf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2be8cf640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:37.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1980734540 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b80a7560 con 0x7ff2b80a7180 2024-08-19T10:53:37.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 1 -- 172.21.15.89:0/1980734540 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff2ac0050f0 con 0x7ff2b80a7180 2024-08-19T10:53:37.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff2a403be00 0x7ff2a403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:37.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b4ff9640 1 -- 172.21.15.89:0/1980734540 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7ff2ac04f360 con 0x7ff2b80a7180 2024-08-19T10:53:37.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b7fff640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff2a403be00 0x7ff2a403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:37.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.049+0000 7ff2b7fff640 10 monclient: get_auth_request con 0x7ff2a403be00 auth_method 0 2024-08-19T10:53:37.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.053+0000 7ff2b7fff640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff2a403be00 0x7ff2a403e2c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009870 tx=0x7ff2a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:37.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.053+0000 7ff2b4ff9640 1 -- 172.21.15.89:0/1980734540 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff2b809fc50 con 0x7ff2b80a7180 2024-08-19T10:53:37.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.053+0000 7ff2b4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:37.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.053+0000 7ff2b4ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:37.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.277+0000 7ff2be8cf640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:53:37.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.277+0000 7ff2be8cf640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:37.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.277+0000 7ff2be8cf640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:37.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.277+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1980734540 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff2b8064590 con 0x7ff2b80a7180 2024-08-19T10:53:37.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.277+0000 7ff2b4ff9640 1 -- 172.21.15.89:0/1980734540 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3194 (secure 0 0 0) 0x7ff2ac010500 con 0x7ff2b80a7180 2024-08-19T10:53:37.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.277+0000 7ff2b4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:37.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.277+0000 7ff2b4ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:37.278 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":124,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:53:37.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1980734540 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff2a403be00 msgr2=0x7ff2a403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:37.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff2a403be00 0x7ff2a403e2c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009870 tx=0x7ff2a8008040 comp rx=0 tx=0).stop 2024-08-19T10:53:37.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 10 monclient: shutdown 2024-08-19T10:53:37.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:37.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1980734540 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b80a7180 msgr2=0x7ff2b809c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:37.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b80a7180 0x7ff2b809c9e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b550 tx=0x7ff2ac00ba10 comp rx=0 tx=0).stop 2024-08-19T10:53:37.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1980734540 shutdown_connections 2024-08-19T10:53:37.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff2a403be00 0x7ff2a403e2c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:37.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b8139620 0x7ff2b809cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:37.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b80a7180 0x7ff2b809c9e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:37.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 --2- 172.21.15.89:0/1980734540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2b80a6860 0x7ff2b8134e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:37.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1980734540 >> 172.21.15.89:0/1980734540 conn(0x7ff2b8074b40 msgr2=0x7ff2b80a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:37.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1980734540 shutdown_connections 2024-08-19T10:53:37.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:37.281+0000 7ff2be8cf640 1 -- 172.21.15.89:0/1980734540 wait complete. 2024-08-19T10:53:40.296 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:53:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient: get_monmap_and_config 2024-08-19T10:53:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient: build_initial_monmap 2024-08-19T10:53:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient: monmap: 2024-08-19T10:53:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:40.430538+0000 2024-08-19T10:53:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:40.430538+0000 2024-08-19T10:53:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:40.424 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:40.425 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:40.426 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 1 Processor -- start 2024-08-19T10:53:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 1 -- start start 2024-08-19T10:53:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient: init 2024-08-19T10:53:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient: _add_conns ranks=[9,7,8,3,1,0,6,5,2,10,4] 2024-08-19T10:53:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47900a5270 0x7f47900a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:40.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient(hunting): picked mon.noname-a con 0x7f47900a5270 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f47900a6d30 0x7f47900a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient(hunting): picked mon.noname-i con 0x7f47900a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47901395b0 0x7f4790139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient(hunting): picked mon.noname-e con 0x7f47901395b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f47900a7650 con 0x7f47900a5270 2024-08-19T10:53:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f47900a8710 con 0x7f47900a6d30 2024-08-19T10:53:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:40.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f47900a5a40 con 0x7f47901395b0 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f4794f2f640 10 monclient(hunting): authenticate will time out at 1870.247119s 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478ed9d640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47901395b0 msgr2=0x7f4790139a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478ed9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47901395b0 0x7f4790139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47900a5270 0x7f47900a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f47900a5270 0x7f47900a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53108/0 (socket says 172.21.15.89:53108) 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 1 -- 172.21.15.89:0/568250788 learned_addr learned my addr 172.21.15.89:0/568250788 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 10 monclient(hunting): get_auth_request con 0x7f47900a5270 auth_method 0 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:40.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478dd9b640 1 -- 172.21.15.89:0/568250788 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f47900a6d30 msgr2=0x7f47900a7110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478dd9b640 1 --2- 172.21.15.89:0/568250788 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f47900a6d30 0x7f47900a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 10 monclient(hunting): handle_auth_done global_id 7461 payload 274 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 1 -- 172.21.15.89:0/568250788 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47901395b0 msgr2=0x7f4790139a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 1 --2- 172.21.15.89:0/568250788 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47901395b0 0x7f4790139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 1 -- 172.21.15.89:0/568250788 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f47900a6d30 msgr2=0x7f47900a7110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 1 --2- 172.21.15.89:0/568250788 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f47900a6d30 0x7f47900a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 1 monclient: found mon.noname-a 2024-08-19T10:53:40.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:40.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:40.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.425+0000 7f478e59c640 1 -- 172.21.15.89:0/568250788 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47900a81d0 con 0x7f47900a5270 2024-08-19T10:53:40.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478e59c640 1 -- 172.21.15.89:0/568250788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 msgr2=0x7f47900a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478e59c640 1 --2- 172.21.15.89:0/568250788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 0x7f47900a67f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4780009870 tx=0x7f478001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9cd39c639fa969 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 1 -- 172.21.15.89:0/568250788 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f47800b1070 con 0x7f47900a5270 2024-08-19T10:53:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:40.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 10 monclient: dump: 2024-08-19T10:53:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:40.439 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:40.440 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:40.440 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:40.440 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:40.440 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:40.440 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:40.441 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:40.441 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:40.441 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:40.441 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:40.441 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:40.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 10 monclient: _finish_auth 0 2024-08-19T10:53:40.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:53:40.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:40.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f4794f2f640 5 monclient: authenticate success, global_id 7461 2024-08-19T10:53:40.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 1 -- 172.21.15.89:0/568250788 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47800a8050 con 0x7f47900a5270 2024-08-19T10:53:40.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f4794f2f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:40.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:40.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f4794f2f640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:40.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 1 -- 172.21.15.89:0/568250788 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f47800abcc0 con 0x7f47900a5270 2024-08-19T10:53:40.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f4794f2f640 4 set_mon_vals no callback set 2024-08-19T10:53:40.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:40.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:40.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.429+0000 7f478d59a640 10 monclient: dump: 2024-08-19T10:53:40.444 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:40.444 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:40.444 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:40.445 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:40.445 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:40.445 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:40.445 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:40.445 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:40.445 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:40.446 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:40.446 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:40.446 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:40.446 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:40.446 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:40.447 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:40.447 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:40.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient: shutdown 2024-08-19T10:53:40.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:40.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 -- 172.21.15.89:0/568250788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 msgr2=0x7f47900a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:40.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 --2- 172.21.15.89:0/568250788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 0x7f47900a67f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4780009870 tx=0x7f478001c460 comp rx=0 tx=0).stop 2024-08-19T10:53:40.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 -- 172.21.15.89:0/568250788 shutdown_connections 2024-08-19T10:53:40.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 --2- 172.21.15.89:0/568250788 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47901395b0 0x7f4790139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:40.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 --2- 172.21.15.89:0/568250788 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f47900a6d30 0x7f47900a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:40.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 --2- 172.21.15.89:0/568250788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 0x7f47900a67f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:40.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 -- 172.21.15.89:0/568250788 >> 172.21.15.89:0/568250788 conn(0x7f4790069a80 msgr2=0x7f47900abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:40.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 -- 172.21.15.89:0/568250788 shutdown_connections 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 -- 172.21.15.89:0/568250788 wait complete. 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient: build_initial_monmap 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient: monmap: 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:40.450 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 Processor -- start 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 -- start start 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient: init 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient: _add_conns ranks=[0,8,1,3,7,4,5,2,6] 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 0x7f47901d1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:40.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient(hunting): picked mon.noname-a con 0x7f47900a5270 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47900a6d30 0x7f47900a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient(hunting): picked mon.noname-e con 0x7f47900a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47901395b0 0x7f47900a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient(hunting): picked mon.noname-f con 0x7f47901395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f47900a5650 con 0x7f47900a5270 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f47900a4a70 con 0x7f47901395b0 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f47900a57d0 con 0x7f47900a6d30 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f4794f2f640 10 monclient(hunting): authenticate will time out at 1870.254086s 2024-08-19T10:53:40.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478dd9b640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47900a6d30 msgr2=0x7f47900a1f60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478dd9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47900a6d30 0x7f47900a1f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 0x7f47901d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 0x7f47901d1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53114/0 (socket says 172.21.15.89:53114) 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 1 -- 172.21.15.89:0/2048428006 learned_addr learned my addr 172.21.15.89:0/2048428006 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 10 monclient(hunting): get_auth_request con 0x7f47900a5270 auth_method 0 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 10 monclient(hunting): handle_auth_done global_id 7470 payload 824 2024-08-19T10:53:40.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 1 -- 172.21.15.89:0/2048428006 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47900a6d30 msgr2=0x7f47900a1f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47900a6d30 0x7f47900a1f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 1 -- 172.21.15.89:0/2048428006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47901395b0 msgr2=0x7f47900a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47901395b0 0x7f47900a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 1 monclient: found mon.noname-a 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.433+0000 7f478e59c640 1 -- 172.21.15.89:0/2048428006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479013ed00 con 0x7f47900a5270 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f478e59c640 1 -- 172.21.15.89:0/2048428006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 msgr2=0x7f47901d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f478e59c640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 0x7f47901d1350 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f478001c290 tx=0x7f4780019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 1 -- 172.21.15.89:0/2048428006 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f47800b1040 con 0x7f47900a5270 2024-08-19T10:53:40.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 10 monclient: dump: 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:40.455 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 10 monclient: _finish_auth 0 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 1 -- 172.21.15.89:0/2048428006 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4780003be0 con 0x7f47900a5270 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f4794f2f640 5 monclient: authenticate success, global_id 7470 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f4794f2f640 10 monclient: _renew_subs 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f4794f2f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f4794f2f640 1 -- 172.21.15.89:0/2048428006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4790139d40 con 0x7f47900a5270 2024-08-19T10:53:40.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 1 -- 172.21.15.89:0/2048428006 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f4780003d90 con 0x7f47900a5270 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477ffff640 4 set_mon_vals no callback set 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 10 monclient: dump: 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:40.457 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f4794f2f640 10 monclient: _renew_subs 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f4794f2f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f4794f2f640 1 -- 172.21.15.89:0/2048428006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479009d820 con 0x7f47900a5270 2024-08-19T10:53:40.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f4794f2f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:40.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f4794f2f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:40.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f4794f2f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:40.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f4794f2f640 1 -- 172.21.15.89:0/2048428006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47901d1890 con 0x7f47900a5270 2024-08-19T10:53:40.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f478e59c640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 0x7f47901d1350 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f478001c290 tx=0x7f4780019200 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f47800ab3e0 mgrmap(e 5) 2024-08-19T10:53:40.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 1 -- 172.21.15.89:0/2048428006 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f47800ab3e0 con 0x7f47900a5270 2024-08-19T10:53:40.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f477effd640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f477003be00 0x7f477003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:40.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f478dd9b640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f477003be00 0x7f477003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:40.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.437+0000 7f478dd9b640 10 monclient: get_auth_request con 0x7f477003be00 auth_method 0 2024-08-19T10:53:40.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.441+0000 7f478dd9b640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f477003be00 0x7f477003e2c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4784006fd0 tx=0x7f4784006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:40.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:40.441+0000 7f477effd640 1 -- 172.21.15.89:0/2048428006 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f47800e8c50 con 0x7f47900a5270 2024-08-19T10:53:41.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.061+0000 7f477effd640 1 -- 172.21.15.89:0/2048428006 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f4790139d40 con 0x7f47900a5270 2024-08-19T10:53:41.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.061+0000 7f477effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:41.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.061+0000 7f477effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:41.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.309+0000 7f4794f2f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:53:41.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.309+0000 7f4794f2f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:41.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.309+0000 7f4794f2f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:41.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.309+0000 7f4794f2f640 1 -- 172.21.15.89:0/2048428006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f47900a7110 con 0x7f47900a5270 2024-08-19T10:53:41.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.309+0000 7f477effd640 1 -- 172.21.15.89:0/2048428006 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3194 (secure 0 0 0) 0x7f4780016170 con 0x7f47900a5270 2024-08-19T10:53:41.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.309+0000 7f477effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:41.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.309+0000 7f477effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:41.310 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":124,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:53:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 -- 172.21.15.89:0/2048428006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f477003be00 msgr2=0x7f477003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f477003be00 0x7f477003e2c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4784006fd0 tx=0x7f4784006d40 comp rx=0 tx=0).stop 2024-08-19T10:53:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 10 monclient: shutdown 2024-08-19T10:53:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 -- 172.21.15.89:0/2048428006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 msgr2=0x7f47901d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 0x7f47901d1350 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f478001c290 tx=0x7f4780019200 comp rx=0 tx=0).stop 2024-08-19T10:53:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 -- 172.21.15.89:0/2048428006 shutdown_connections 2024-08-19T10:53:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47901395b0 0x7f47900a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f477003be00 0x7f477003e2c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f47900a6d30 0x7f47900a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:41.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 --2- 172.21.15.89:0/2048428006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47900a5270 0x7f47901d1350 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:41.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 -- 172.21.15.89:0/2048428006 >> 172.21.15.89:0/2048428006 conn(0x7f4790069a80 msgr2=0x7f4790132e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:41.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 -- 172.21.15.89:0/2048428006 shutdown_connections 2024-08-19T10:53:41.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:41.313+0000 7f4794f2f640 1 -- 172.21.15.89:0/2048428006 wait complete. 2024-08-19T10:53:44.330 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:53:44.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient: get_monmap_and_config 2024-08-19T10:53:44.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient: build_initial_monmap 2024-08-19T10:53:44.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient: monmap: 2024-08-19T10:53:44.460 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:44.460 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:44.460 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:44.466299+0000 2024-08-19T10:53:44.460 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:44.466299+0000 2024-08-19T10:53:44.460 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:44.460 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:44.460 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:44.461 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:44.461 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:44.461 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:44.461 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:44.461 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:44.461 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:44.461 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:44.461 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:44.461 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:44.461 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:44.461 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:44.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 1 Processor -- start 2024-08-19T10:53:44.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 1 -- start start 2024-08-19T10:53:44.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient: init 2024-08-19T10:53:44.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:44.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient: _add_conns ranks=[6,9,7,8,10,3,4,1,2,5,0] 2024-08-19T10:53:44.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:44.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient(hunting): picked mon.noname-d con 0x7ff4d00a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:44.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:44.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient(hunting): picked mon.noname-a con 0x7ff4d00a6d30 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:44.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4d01395b0 0x7ff4d0139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient(hunting): picked mon.noname-i con 0x7ff4d01395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d00a7650 con 0x7ff4d00a6d30 2024-08-19T10:53:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff4d00a8710 con 0x7ff4d00a5270 2024-08-19T10:53:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff4d00a5a40 con 0x7ff4d01395b0 2024-08-19T10:53:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4d590c640 10 monclient(hunting): authenticate will time out at 1874.283114s 2024-08-19T10:53:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:44.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4cf7fe640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4d01395b0 msgr2=0x7ff4d0139a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:53:44.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ce7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:44.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4cf7fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4d01395b0 0x7ff4d0139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:44.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:39156/0 (socket says 172.21.15.89:39156) 2024-08-19T10:53:44.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ce7fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56598/0 (socket says 172.21.15.89:56598) 2024-08-19T10:53:44.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 learned_addr learned my addr 172.21.15.89:0/2624974769 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:44.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request con 0x7ff4d00a5270 auth_method 0 2024-08-19T10:53:44.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:44.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ceffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:44.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:44.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request con 0x7ff4d00a6d30 auth_method 0 2024-08-19T10:53:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 msgr2=0x7ff4d00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:53:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 msgr2=0x7ff4d00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ce7fc640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 msgr2=0x7ff4d00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:53:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ce7fc640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 msgr2=0x7ff4d00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:44.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.461+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:44.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4d01395b0 msgr2=0x7ff4d0139a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:53:44.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4d01395b0 0x7ff4d0139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:53:44.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:44.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request con 0x7ff4d00a6d30 auth_method 0 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request con 0x7ff4d00a5270 auth_method 0 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ceffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ceffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ce7fc640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 msgr2=0x7ff4d00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ce7fc640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 msgr2=0x7ff4d00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:44.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:53:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 msgr2=0x7ff4d00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:53:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 msgr2=0x7ff4d00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:44.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:44.665+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:53:45.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:45.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request con 0x7ff4d00a6d30 auth_method 0 2024-08-19T10:53:45.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:45.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:45.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:45.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:45.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4d01395b0 msgr2=0x7ff4d0139a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request con 0x7ff4d00a5270 auth_method 0 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ceffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ceffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ce7fc640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 msgr2=0x7ff4d00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4d01395b0 0x7ff4d0139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ce7fc640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 msgr2=0x7ff4d00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 msgr2=0x7ff4d00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 msgr2=0x7ff4d00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:45.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.069+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:53:45.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4d01395b0 msgr2=0x7ff4d0139a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:53:45.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4d01395b0 0x7ff4d0139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:53:45.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:45.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request con 0x7ff4d00a6d30 auth_method 0 2024-08-19T10:53:45.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:45.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:45.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:45.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:45.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request con 0x7ff4d00a5270 auth_method 0 2024-08-19T10:53:45.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:45.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ceffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:45.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ceffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:45.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ce7fc640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 msgr2=0x7ff4d00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:53:45.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ce7fc640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 msgr2=0x7ff4d00a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:45.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:45.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 msgr2=0x7ff4d00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:53:45.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 msgr2=0x7ff4d00a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:45.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:45.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:53:45.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:45.873+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:53:47.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 10 monclient(hunting): tick 2024-08-19T10:53:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:53:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 monclient(hunting): continuing hunt 2024-08-19T10:53:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:53:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4d01395b0 msgr2=0x7ff4d0139a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4d01395b0 0x7ff4d0139a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 msgr2=0x7ff4d00a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff4d00a5270 0x7ff4d00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 msgr2=0x7ff4d00a7110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.89:3300/0 conn(0x7ff4d00a6d30 0x7ff4d00a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 10 monclient: _add_conns ranks=[10,1,5,8,2,3,0,4] 2024-08-19T10:53:47.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 0x7ff4bc005c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 10 monclient(hunting): picked mon.noname-f con 0x7ff4bc003820 addr v2:172.21.15.190:3300/0 2024-08-19T10:53:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff4bc006260 addr v1:172.21.15.190:6789/0 2024-08-19T10:53:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 0x7ff4bc00c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 10 monclient(hunting): picked mon.noname-h con 0x7ff4bc009f90 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 -- 172.21.15.89:0/2624974769 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4d00a81d0 con 0x7ff4bc006260 2024-08-19T10:53:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 -- 172.21.15.89:0/2624974769 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d00a7650 con 0x7ff4bc003820 2024-08-19T10:53:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 1 -- 172.21.15.89:0/2624974769 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff4d00a8710 con 0x7ff4bc009f90 2024-08-19T10:53:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ccff9640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:47.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 0x7ff4bc00c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.461+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 0x7ff4bc005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4cf7fe640 10 monclient(hunting): get_auth_request con 0x7ff4bc009f90 auth_method 0 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4cf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request con 0x7ff4bc003820 auth_method 0 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4ceffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 msgr2=0x7ff4bc00c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 msgr2=0x7ff4bc00c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 0x7ff4bc00c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:47.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 msgr2=0x7ff4bc005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:53:47.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 msgr2=0x7ff4bc005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:47.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 0x7ff4bc005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:47.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 0x7ff4bc005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:47.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 0x7ff4bc00c450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:47.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.465+0000 7ff4ce7fc640 1 -- 172.21.15.89:0/2624974769 >> v1:172.21.15.190:6789/0 conn(0x7ff4bc006260 legacy=0x7ff4bc008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:47.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 0x7ff4bc005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:47.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 0x7ff4bc00c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request con 0x7ff4bc003820 auth_method 0 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4ceffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4ceffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4cf7fe640 10 monclient(hunting): get_auth_request con 0x7ff4bc009f90 auth_method 0 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4cf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4cf7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 msgr2=0x7ff4bc005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 msgr2=0x7ff4bc005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 0x7ff4bc005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 msgr2=0x7ff4bc00c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:53:47.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 msgr2=0x7ff4bc00c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:53:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 0x7ff4bc00c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:53:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 0x7ff4bc005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:53:47.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.669+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 0x7ff4bc00c450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:53:47.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 191662698 0 0) 0x7ff4d00a81d0 con 0x7ff4bc006260 2024-08-19T10:53:47.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): _negotiate 2024-08-19T10:53:47.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:47.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:47.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): my global_id is 45208 2024-08-19T10:53:47.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b8003530 con 0x7ff4bc006260 2024-08-19T10:53:47.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 296313101 0 0) 0x7ff4b8003530 con 0x7ff4bc006260 2024-08-19T10:53:47.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 msgr2=0x7ff4bc00c450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:47.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4bc009f90 0x7ff4bc00c450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 msgr2=0x7ff4bc005c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:47.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 --2- 172.21.15.89:0/2624974769 >> v2:172.21.15.190:3300/0 conn(0x7ff4bc003820 0x7ff4bc005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient: _finish_hunting 0 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 monclient: found mon.noname-f-legacy 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 reap_dead start 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.190:6789/0 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc00dbf0 con 0x7ff4bc006260 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient: _finish_auth 0 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 3927028378 0 0) 0x7ff4c000f040 con 0x7ff4bc006260 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-08-19T10:53:47.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient: dump: 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:47.878 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] but i'm connected to v1:172.21.15.190:6789/0, reconnecting 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 >> v1:172.21.15.190:6789/0 conn(0x7ff4bc006260 legacy=0x7ff4bc008720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient: _add_conns ranks=[7,1,5,2,3,8,6,0,4] 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4b8004210 0x7ff4b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): picked mon.i con 0x7ff4b8004210 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b8004e20 0x7ff4b80052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): picked mon.f con 0x7ff4b8004e20 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 0x7ff4b800c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:47.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): picked mon.h con 0x7ff4b8009c00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d00a7650 con 0x7ff4b8004e20 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff4d00a8710 con 0x7ff4b8009c00 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff4b800c9a0 con 0x7ff4b8004210 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cdffb640 10 monclient(hunting): _finish_auth 0 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4ceffd640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4b8004210 msgr2=0x7ff4b8004810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4b8004210 0x7ff4b8004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b8004e20 0x7ff4b80052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:47.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 0x7ff4b800c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cf7fe640 10 monclient(hunting): get_auth_request con 0x7ff4b8009c00 auth_method 0 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4cf7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request con 0x7ff4b8004e20 auth_method 0 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.873+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4ce7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:47.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4ce7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4ce7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 10 monclient(hunting): handle_auth_done global_id 45208 payload 293 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4b8004210 msgr2=0x7ff4b8004810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4b8004210 0x7ff4b8004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b8004e20 msgr2=0x7ff4b80052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b8004e20 0x7ff4b80052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 1 monclient: found mon.h 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b8003530 con 0x7ff4b8009c00 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 10 monclient: _finish_auth 0 2024-08-19T10:53:47.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:53:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4d590c640 5 monclient: authenticate success, global_id 45208 2024-08-19T10:53:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4d590c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.877+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b8004e20 0x7ff4b80052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cf7fe640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 msgr2=0x7ff4b800c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:47.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 0x7ff4b800c090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00a88c0 tx=0x7ff4c801d470 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a76396f8590f51d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4c80b1070 con 0x7ff4b8009c00 2024-08-19T10:53:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cdffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:53:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cdffb640 10 monclient: dump: 2024-08-19T10:53:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:47.884 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:47.885 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c80ab810 con 0x7ff4b8009c00 2024-08-19T10:53:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 4 set_mon_vals no callback set 2024-08-19T10:53:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cdffb640 1 -- 172.21.15.89:0/2624974769 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4c8007bc0 con 0x7ff4b8009c00 2024-08-19T10:53:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cdffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:53:47.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4cdffb640 10 monclient: dump: 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:47.887 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:47.888 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:47.888 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:47.888 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 10 monclient: shutdown 2024-08-19T10:53:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 1 -- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 msgr2=0x7ff4b800c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 0x7ff4b800c090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00a88c0 tx=0x7ff4c801d470 comp rx=0 tx=0).stop 2024-08-19T10:53:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 1 -- 172.21.15.89:0/2624974769 shutdown_connections 2024-08-19T10:53:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b8004e20 0x7ff4b80052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff4b8004210 0x7ff4b8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 1 --2- 172.21.15.89:0/2624974769 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 0x7ff4b800c090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 1 -- 172.21.15.89:0/2624974769 >> 172.21.15.89:0/2624974769 conn(0x7ff4d0069a80 msgr2=0x7ff4d00abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:47.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 1 -- 172.21.15.89:0/2624974769 shutdown_connections 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.885+0000 7ff4d590c640 1 -- 172.21.15.89:0/2624974769 wait complete. 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient: build_initial_monmap 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient: monmap: 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:47.896 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 1 Processor -- start 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 1 -- start start 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient: init 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient: _add_conns ranks=[0,1,5,2,6,3,8,7,4] 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4b8004210 0x7ff4d0122440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient(hunting): picked mon.noname-a con 0x7ff4b8004210 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:47.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b8004e20 0x7ff4d0122980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient(hunting): picked mon.noname-f con 0x7ff4b8004e20 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 0x7ff4d0122ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient(hunting): picked mon.noname-h con 0x7ff4b8009c00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d00a57d0 con 0x7ff4b8004210 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d00a5650 con 0x7ff4b8004e20 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff4d013a1d0 con 0x7ff4b8009c00 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4d590c640 10 monclient(hunting): authenticate will time out at 1877.709769s 2024-08-19T10:53:47.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4b8004210 0x7ff4d0122440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4b8004210 0x7ff4d0122440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56638/0 (socket says 172.21.15.89:56638) 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 1 -- 172.21.15.89:0/3377706765 learned_addr learned my addr 172.21.15.89:0/3377706765 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request con 0x7ff4b8004210 auth_method 0 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4cf7fe640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 0x7ff4d0122ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b8004e20 0x7ff4d0122980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4cf7fe640 10 monclient(hunting): get_auth_request con 0x7ff4b8009c00 auth_method 0 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4cf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:47.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request con 0x7ff4b8004e20 auth_method 0 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ce7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ce7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ce7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ce7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:47.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 10 monclient(hunting): handle_auth_done global_id 7479 payload 824 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 1 -- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 msgr2=0x7ff4d0122ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 0x7ff4d0122ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 1 -- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b8004e20 msgr2=0x7ff4d0122980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b8004e20 0x7ff4d0122980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 10 monclient: _finish_hunting 0 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 1 monclient: found mon.noname-a 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.889+0000 7ff4ceffd640 1 -- 172.21.15.89:0/3377706765 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d00ab1a0 con 0x7ff4b8004210 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4ceffd640 1 -- 172.21.15.89:0/3377706765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4b8004210 msgr2=0x7ff4d0122440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4ceffd640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4b8004210 0x7ff4d0122440 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4018860 tx=0x7ff4c4018d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 1 -- 172.21.15.89:0/3377706765 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4c4025070 con 0x7ff4b8004210 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:47.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 10 monclient: dump: 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:47.902 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 10 monclient: _finish_auth 0 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 1 -- 172.21.15.89:0/3377706765 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c4014d50 con 0x7ff4b8004210 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4d590c640 5 monclient: authenticate success, global_id 7479 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 1 -- 172.21.15.89:0/3377706765 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff4c400f040 con 0x7ff4b8004210 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4d590c640 10 monclient: _renew_subs 2024-08-19T10:53:47.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4d590c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4d590c640 1 -- 172.21.15.89:0/3377706765 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d00a1c00 con 0x7ff4b8004210 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4ccff9640 4 set_mon_vals no callback set 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 10 monclient: dump: 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:47.904 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4d590c640 10 monclient: _renew_subs 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4d590c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4d590c640 1 -- 172.21.15.89:0/3377706765 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d0138500 con 0x7ff4b8004210 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 1 -- 172.21.15.89:0/3377706765 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff4c4028360 con 0x7ff4b8004210 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4d590c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4d590c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4d590c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:47.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4d590c640 1 -- 172.21.15.89:0/3377706765 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d013c9f0 con 0x7ff4b8004210 2024-08-19T10:53:47.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4bc03cba0 0x7ff4bc03f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:47.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4b7fff640 1 -- 172.21.15.89:0/3377706765 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7ff4c404fee0 con 0x7ff4b8004210 2024-08-19T10:53:47.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4bc03cba0 0x7ff4bc03f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:47.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.893+0000 7ff4ce7fc640 10 monclient: get_auth_request con 0x7ff4bc03cba0 auth_method 0 2024-08-19T10:53:47.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.897+0000 7ff4ce7fc640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4bc03cba0 0x7ff4bc03f060 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00a1870 tx=0x7ff4c0010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:47.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.897+0000 7ff4b7fff640 1 -- 172.21.15.89:0/3377706765 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff4c401d070 con 0x7ff4b8004210 2024-08-19T10:53:47.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.897+0000 7ff4b7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:47.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:47.897+0000 7ff4b7fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:48.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.133+0000 7ff4d590c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:53:48.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.133+0000 7ff4d590c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:48.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.133+0000 7ff4d590c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:48.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.133+0000 7ff4d590c640 1 -- 172.21.15.89:0/3377706765 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff4d0064590 con 0x7ff4b8004210 2024-08-19T10:53:48.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.133+0000 7ff4b7fff640 1 -- 172.21.15.89:0/3377706765 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3175 (secure 0 0 0) 0x7ff4c404dda0 con 0x7ff4b8004210 2024-08-19T10:53:48.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.133+0000 7ff4b7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:53:48.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.133+0000 7ff4b7fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:48.134 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":128,"quorum":[0,1,2,4,5,6],"quorum_names":["a","f","b","c","h","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:53:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 -- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4bc03cba0 msgr2=0x7ff4bc03f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4bc03cba0 0x7ff4bc03f060 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00a1870 tx=0x7ff4c0010040 comp rx=0 tx=0).stop 2024-08-19T10:53:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 10 monclient: shutdown 2024-08-19T10:53:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 -- 172.21.15.89:0/3377706765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4b8004210 msgr2=0x7ff4d0122440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4b8004210 0x7ff4d0122440 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4018860 tx=0x7ff4c4018d20 comp rx=0 tx=0).stop 2024-08-19T10:53:48.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 -- 172.21.15.89:0/3377706765 shutdown_connections 2024-08-19T10:53:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff4bc03cba0 0x7ff4bc03f060 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff4b8009c00 0x7ff4d0122ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4b8004e20 0x7ff4d0122980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 --2- 172.21.15.89:0/3377706765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4b8004210 0x7ff4d0122440 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 -- 172.21.15.89:0/3377706765 >> 172.21.15.89:0/3377706765 conn(0x7ff4d0069a80 msgr2=0x7ff4d00a9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 -- 172.21.15.89:0/3377706765 shutdown_connections 2024-08-19T10:53:48.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.137+0000 7ff4d590c640 1 -- 172.21.15.89:0/3377706765 wait complete. 2024-08-19T10:53:48.154 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-08-19T10:53:48.154 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon a 2024-08-19T10:53:48.154 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:53:48.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.273+0000 7f249fbef640 10 monclient: get_monmap_and_config 2024-08-19T10:53:48.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.273+0000 7f249fbef640 10 monclient: build_initial_monmap 2024-08-19T10:53:48.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.273+0000 7f249fbef640 10 monclient: monmap: 2024-08-19T10:53:48.274 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:48.280542+0000 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:48.280542+0000 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:48.275 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:48.276 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:48.276 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:48.276 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 1 Processor -- start 2024-08-19T10:53:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 1 -- start start 2024-08-19T10:53:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 10 monclient: init 2024-08-19T10:53:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 10 monclient: _add_conns ranks=[2,5,8,4,6,10,1,7,9,0,3] 2024-08-19T10:53:48.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24980a39f0 0x7f24980a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 10 monclient(hunting): picked mon.noname-b con 0x7f24980a39f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2498074670 0x7f24980a4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 10 monclient(hunting): picked mon.noname-h con 0x7f2498074670 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f249813a7d0 0x7f249813cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 10 monclient(hunting): picked mon.noname-e con 0x7f249813a7d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f249806d550 con 0x7f24980a39f0 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f24980a4850 con 0x7f2498074670 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f249809e030 con 0x7f249813a7d0 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:48.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249fbef640 10 monclient(hunting): authenticate will time out at 1878.097517s 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249e18c640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f249813a7d0 msgr2=0x7f249813cbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249e18c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f249813a7d0 0x7f249813cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24980a39f0 0x7f24980a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24980a39f0 0x7f24980a3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43560/0 (socket says 172.21.15.89:43560) 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 1 -- 172.21.15.89:0/1677610053 learned_addr learned my addr 172.21.15.89:0/1677610053 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 10 monclient(hunting): get_auth_request con 0x7f24980a39f0 auth_method 0 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d18a640 1 --2- 172.21.15.89:0/1677610053 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2498074670 0x7f24980a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d18a640 10 monclient(hunting): get_auth_request con 0x7f2498074670 auth_method 0 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d18a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d18a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d18a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 10 monclient(hunting): handle_auth_done global_id 74396 payload 274 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 1 -- 172.21.15.89:0/1677610053 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f249813a7d0 msgr2=0x7f249813cbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 1 --2- 172.21.15.89:0/1677610053 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f249813a7d0 0x7f249813cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 1 -- 172.21.15.89:0/1677610053 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2498074670 msgr2=0x7f24980a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 1 --2- 172.21.15.89:0/1677610053 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2498074670 0x7f24980a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 10 monclient: _finish_hunting 0 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 1 monclient: found mon.noname-b 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d98b640 1 -- 172.21.15.89:0/1677610053 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249809dcb0 con 0x7f24980a39f0 2024-08-19T10:53:48.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.277+0000 7f249d18a640 1 --2- 172.21.15.89:0/1677610053 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2498074670 0x7f24980a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:53:48.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 1 -- 172.21.15.89:0/1677610053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24980a39f0 msgr2=0x7f24980a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 1 --2- 172.21.15.89:0/1677610053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24980a39f0 0x7f24980a3dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f249400bb40 tx=0x7f249401eb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38596d188cadcfae server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 1 -- 172.21.15.89:0/1677610053 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24940b4070 con 0x7f24980a39f0 2024-08-19T10:53:48.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 10 monclient: dump: 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.284 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 10 monclient: _finish_auth 0 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 10 monclient: _check_auth_tickets 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 1 -- 172.21.15.89:0/1677610053 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24940a7c60 con 0x7f24980a39f0 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 1 -- 172.21.15.89:0/1677610053 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24940a7e10 con 0x7f24980a39f0 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 5 monclient: authenticate success, global_id 74396 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:48.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 4 set_mon_vals no callback set 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249c989640 10 monclient: dump: 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.286 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient: shutdown 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 -- 172.21.15.89:0/1677610053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24980a39f0 msgr2=0x7f24980a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 --2- 172.21.15.89:0/1677610053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24980a39f0 0x7f24980a3dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f249400bb40 tx=0x7f249401eb00 comp rx=0 tx=0).stop 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 -- 172.21.15.89:0/1677610053 shutdown_connections 2024-08-19T10:53:48.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 --2- 172.21.15.89:0/1677610053 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f249813a7d0 0x7f249813cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 --2- 172.21.15.89:0/1677610053 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2498074670 0x7f24980a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 --2- 172.21.15.89:0/1677610053 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f24980a39f0 0x7f24980a3dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 -- 172.21.15.89:0/1677610053 >> 172.21.15.89:0/1677610053 conn(0x7f24980a50e0 msgr2=0x7f249806c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:48.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 -- 172.21.15.89:0/1677610053 shutdown_connections 2024-08-19T10:53:48.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 -- 172.21.15.89:0/1677610053 wait complete. 2024-08-19T10:53:48.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient: build_initial_monmap 2024-08-19T10:53:48.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient: monmap: 2024-08-19T10:53:48.293 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:48.294 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 Processor -- start 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 -- start start 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient: init 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient: _add_conns ranks=[4,0,7,2,1,5,6,8,3] 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2498074670 0x7f2498070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient(hunting): picked mon.noname-c con 0x7f2498074670 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24980a39f0 0x7f24980710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient(hunting): picked mon.noname-a con 0x7f24980a39f0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f249813a7d0 0x7f2498071630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient(hunting): picked mon.noname-i con 0x7f249813a7d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f249809d1f0 con 0x7f24980a39f0 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f24980a00d0 con 0x7f2498074670 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f249809d370 con 0x7f249813a7d0 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249fbef640 10 monclient(hunting): authenticate will time out at 1878.106867s 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2498074670 0x7f2498070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d18a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24980a39f0 0x7f24980710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249e18c640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f249813a7d0 msgr2=0x7f2498071630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:53:48.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2498074670 0x7f2498070bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51376/0 (socket says 172.21.15.89:51376) 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 1 -- 172.21.15.89:0/2284966444 learned_addr learned my addr 172.21.15.89:0/2284966444 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 10 monclient(hunting): get_auth_request con 0x7f2498074670 auth_method 0 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249e18c640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f249813a7d0 0x7f2498071630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d18a640 10 monclient(hunting): get_auth_request con 0x7f24980a39f0 auth_method 0 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d18a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d18a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d18a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d18a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d18a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d18a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 10 monclient(hunting): handle_auth_done global_id 84226 payload 824 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 1 -- 172.21.15.89:0/2284966444 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f249813a7d0 msgr2=0x7f2498071630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f249813a7d0 0x7f2498071630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 1 -- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24980a39f0 msgr2=0x7f24980710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24980a39f0 0x7f24980710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 10 monclient: _finish_hunting 0 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 1 monclient: found mon.noname-c 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:48.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d98b640 1 -- 172.21.15.89:0/2284966444 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249813cbd0 con 0x7f2498074670 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.285+0000 7f249d18a640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24980a39f0 0x7f24980710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d98b640 1 -- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2498074670 msgr2=0x7f2498070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d98b640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2498074670 0x7f2498070bb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f249401f3a0 tx=0x7f249401b8a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24940bd020 con 0x7f2498074670 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 10 monclient: dump: 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.299 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2494004410 con 0x7f2498074670 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24940b4040 con 0x7f2498074670 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249fbef640 5 monclient: authenticate success, global_id 84226 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 10 monclient: dump: 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.301 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249fbef640 10 monclient: _renew_subs 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248bfff640 4 set_mon_vals no callback set 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249fbef640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249fbef640 1 -- 172.21.15.89:0/2284966444 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2498073a00 con 0x7f2498074670 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249fbef640 10 monclient: _renew_subs 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249fbef640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249fbef640 1 -- 172.21.15.89:0/2284966444 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f249811eb80 con 0x7f2498074670 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249fbef640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249fbef640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813b1a0 0x7f249813b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249fbef640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249fbef640 1 -- 172.21.15.89:0/2284966444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2498133940 con 0x7f249813b1a0 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f24940c0020 con 0x7f2498074670 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d18a640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813b1a0 0x7f249813b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d18a640 10 monclient: get_auth_request con 0x7f249813b1a0 auth_method 0 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d18a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d18a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d18a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d18a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d18a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d18a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f247c03be30 0x7f247c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d18a640 10 monclient(hunting): handle_auth_done global_id 7497 payload 824 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249d18a640 1 -- 172.21.15.89:0/2284966444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24980a6360 con 0x7f249813b1a0 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f24940ebec0 con 0x7f2498074670 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249e18c640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f247c03be30 0x7f247c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.289+0000 7f249e18c640 10 monclient: get_auth_request con 0x7f247c03be30 auth_method 0 2024-08-19T10:53:48.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f249d18a640 1 -- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813b1a0 msgr2=0x7f249813b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f249d18a640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813b1a0 0x7f249813b580 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2498072310 tx=0x7f248c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f248c019070 con 0x7f249813b1a0 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f248a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f248a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f248a7fc640 10 monclient: dump: 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.305 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f248c00fe30 con 0x7f249813b1a0 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f248a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f248a7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:48.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813b1a0 msgr2=0x7f249813b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f248a7fc640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813b1a0 0x7f249813b580 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2498072310 tx=0x7f248c011e60 comp rx=0 tx=0).stop 2024-08-19T10:53:48.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813b1a0 msgr2=0x7f249813b580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:48.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f248a7fc640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813b1a0 0x7f249813b580 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.293+0000 7f249e18c640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f247c03be30 0x7f247c03e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2490003cc0 tx=0x7f24900091f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.297+0000 7f249fbef640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.297+0000 7f249fbef640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813bac0 0x7f24981d6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.297+0000 7f249fbef640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.297+0000 7f249fbef640 1 -- 172.21.15.89:0/2284966444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24980a5e50 con 0x7f249813bac0 2024-08-19T10:53:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813bac0 0x7f24981d6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 10 monclient: get_auth_request con 0x7f249813bac0 auth_method 0 2024-08-19T10:53:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 10 monclient(hunting): handle_auth_done global_id 7506 payload 824 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 1 -- 172.21.15.89:0/2284966444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f24980a6060 con 0x7f249813bac0 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 1 -- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813bac0 msgr2=0x7f24981d6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249d18a640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813bac0 0x7f24981d6f00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2498072310 tx=0x7f248c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f248c019040 con 0x7f249813bac0 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f248a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f248a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f248a7fc640 10 monclient: dump: 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.310 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.311 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.311 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9571 (secure 0 0 0) 0x7f248c004020 con 0x7f249813bac0 2024-08-19T10:53:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f248a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f248a7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813bac0 msgr2=0x7f24981d6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f248a7fc640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813bac0 0x7f24981d6f00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2498072310 tx=0x7f248c0046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f248a7fc640 1 -- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813bac0 msgr2=0x7f24981d6f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:48.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f248a7fc640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f249813bac0 0x7f24981d6f00 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.312 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:48.312 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:48.312 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 128, 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 958114, 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: 6 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:53:48.313 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.314 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.315 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:48.316 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.317 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.318 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.320 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.321 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.322 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.322 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:48.322 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:48.322 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.322 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.322 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.322 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.322 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:48.322 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:48.324 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.325 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:48.327 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.328 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:48.329 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.330 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.331 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.332 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:48.333 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.334 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.335 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.336 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:53:48.336 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.336 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:48.336 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:48.336 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:48.336 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.336 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.336 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:48.336 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249fbef640 1 -- 172.21.15.89:0/2284966444 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f247c03be30 msgr2=0x7f247c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249fbef640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f247c03be30 0x7f247c03e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2490003cc0 tx=0x7f24900091f0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249fbef640 10 monclient: shutdown 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249fbef640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249fbef640 1 -- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2498074670 msgr2=0x7f2498070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249fbef640 1 --2- 172.21.15.89:0/2284966444 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2498074670 0x7f2498070bb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f249401f3a0 tx=0x7f249401b8a0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249e18c640 1 -- 172.21.15.89:0/2284966444 reap_dead start 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249e18c640 1 -- 172.21.15.89:0/2284966444 reap_dead start 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249fbef640 1 -- 172.21.15.89:0/2284966444 shutdown_connections 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249fbef640 1 -- 172.21.15.89:0/2284966444 >> 172.21.15.89:0/2284966444 conn(0x7f24980a50e0 msgr2=0x7f24980a0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249fbef640 1 -- 172.21.15.89:0/2284966444 shutdown_connections 2024-08-19T10:53:48.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.301+0000 7f249fbef640 1 -- 172.21.15.89:0/2284966444 wait complete. 2024-08-19T10:53:48.339 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:53:48.339 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-08-19T10:53:48.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:53:48.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.453+0000 7f990ba97640 10 monclient: get_monmap_and_config 2024-08-19T10:53:48.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.453+0000 7f990ba97640 10 monclient: build_initial_monmap 2024-08-19T10:53:48.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.453+0000 7f990ba97640 10 monclient: monmap: 2024-08-19T10:53:48.453 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:48.453 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.453 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:48.459548+0000 2024-08-19T10:53:48.453 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:48.459548+0000 2024-08-19T10:53:48.453 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:48.453 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:48.454 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.453+0000 7f990ba97640 1 Processor -- start 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.453+0000 7f990ba97640 1 -- start start 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.453+0000 7f990ba97640 10 monclient: init 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 10 monclient: _add_conns ranks=[6,9,3,8,2,1,5,7,4,10,0] 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99040a6860 0x7f99040a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 10 monclient(hunting): picked mon.noname-d con 0x7f99040a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99040a6c40 0x7f99040a7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 10 monclient(hunting): picked mon.noname-a con 0x7f99040a6c40 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9904139620 0x7f9904139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 10 monclient(hunting): picked mon.noname-g con 0x7f9904139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:48.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f99040a61a0 con 0x7f99040a6c40 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f99040a81b0 con 0x7f9904139620 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f99040a4c90 con 0x7f99040a6860 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990ba97640 10 monclient(hunting): authenticate will time out at 1878.276161s 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909833640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99040a6860 0x7f99040a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f99040a6c40 0x7f99040a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990a034640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9904139620 msgr2=0x7f9904139ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909833640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99040a6860 0x7f99040a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:39190/0 (socket says 172.21.15.89:39190) 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909833640 1 -- 172.21.15.89:0/2852757156 learned_addr learned my addr 172.21.15.89:0/2852757156 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:48.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909833640 10 monclient(hunting): get_auth_request con 0x7f99040a6860 auth_method 0 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909833640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909833640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909833640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f990a034640 1 --2- 172.21.15.89:0/2852757156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9904139620 0x7f9904139ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 10 monclient(hunting): get_auth_request con 0x7f99040a6c40 auth_method 0 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909833640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909833640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909833640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 10 monclient(hunting): handle_auth_done global_id 7515 payload 274 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 1 -- 172.21.15.89:0/2852757156 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99040a6860 msgr2=0x7f99040a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 1 --2- 172.21.15.89:0/2852757156 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99040a6860 0x7f99040a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 1 -- 172.21.15.89:0/2852757156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9904139620 msgr2=0x7f9904139ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 1 --2- 172.21.15.89:0/2852757156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9904139620 0x7f9904139ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 10 monclient: _finish_hunting 0 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 1 monclient: found mon.noname-a 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909032640 1 -- 172.21.15.89:0/2852757156 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99040a85c0 con 0x7f99040a6c40 2024-08-19T10:53:48.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.457+0000 7f9909833640 1 --2- 172.21.15.89:0/2852757156 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99040a6860 0x7f99040a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:48.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9909032640 1 -- 172.21.15.89:0/2852757156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99040a6c40 msgr2=0x7f99040a7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9909032640 1 --2- 172.21.15.89:0/2852757156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99040a6c40 0x7f99040a7020 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009d90 tx=0x7f98fc01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9a16a6be2270ad server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 1 -- 172.21.15.89:0/2852757156 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98fc0ac070 con 0x7f99040a6c40 2024-08-19T10:53:48.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:48.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 10 monclient: dump: 2024-08-19T10:53:48.464 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.464 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.464 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 10 monclient: _finish_auth 0 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 10 monclient: _check_auth_tickets 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 5 monclient: authenticate success, global_id 7515 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 1 -- 172.21.15.89:0/2852757156 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98fc0a6910 con 0x7f99040a6c40 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 4 set_mon_vals no callback set 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 1 -- 172.21.15.89:0/2852757156 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98fc0a7480 con 0x7f99040a6c40 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9908831640 10 monclient: dump: 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.466 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.467 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient: shutdown 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 -- 172.21.15.89:0/2852757156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99040a6c40 msgr2=0x7f99040a7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 --2- 172.21.15.89:0/2852757156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99040a6c40 0x7f99040a7020 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009d90 tx=0x7f98fc01c850 comp rx=0 tx=0).stop 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 -- 172.21.15.89:0/2852757156 shutdown_connections 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 --2- 172.21.15.89:0/2852757156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9904139620 0x7f9904139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 --2- 172.21.15.89:0/2852757156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99040a6c40 0x7f99040a7020 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 --2- 172.21.15.89:0/2852757156 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f99040a6860 0x7f99040a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 -- 172.21.15.89:0/2852757156 >> 172.21.15.89:0/2852757156 conn(0x7f99040699d0 msgr2=0x7f99040abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 -- 172.21.15.89:0/2852757156 shutdown_connections 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 -- 172.21.15.89:0/2852757156 wait complete. 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient: build_initial_monmap 2024-08-19T10:53:48.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient: monmap: 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 Processor -- start 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 -- start start 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient: init 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient: _add_conns ranks=[5,2,6,8,4,7,1,3,0] 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99040a6860 0x7f99041345d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient(hunting): picked mon.noname-h con 0x7f99040a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99040a6c40 0x7f9904134b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient(hunting): picked mon.noname-b con 0x7f99040a6c40 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9904139620 0x7f9904135050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient(hunting): picked mon.noname-d con 0x7f9904139620 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9904143210 con 0x7f99040a6c40 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f9904143090 con 0x7f99040a6860 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f9904143390 con 0x7f9904139620 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990ba97640 10 monclient(hunting): authenticate will time out at 1878.283729s 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9909032640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99040a6c40 0x7f9904134b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990a034640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9904139620 0x7f9904135050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9909032640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99040a6c40 0x7f9904134b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43566/0 (socket says 172.21.15.89:43566) 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9909032640 1 -- 172.21.15.89:0/961027815 learned_addr learned my addr 172.21.15.89:0/961027815 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:48.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9909032640 10 monclient(hunting): get_auth_request con 0x7f99040a6c40 auth_method 0 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9909032640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9909032640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f9909032640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990a034640 10 monclient(hunting): get_auth_request con 0x7f9904139620 auth_method 0 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990a034640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990a034640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.461+0000 7f990a034640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909032640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909032640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909032640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99040a6860 0x7f99041345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 10 monclient(hunting): get_auth_request con 0x7f99040a6860 auth_method 0 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 10 monclient(hunting): handle_auth_done global_id 45285 payload 824 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 1 -- 172.21.15.89:0/961027815 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99040a6860 msgr2=0x7f99041345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f99040a6860 0x7f99041345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 1 -- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99040a6c40 msgr2=0x7f9904134b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f99040a6c40 0x7f9904134b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 10 monclient: _finish_hunting 0 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 1 monclient: found mon.noname-d 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:48.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:48.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 1 -- 172.21.15.89:0/961027815 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990409edd0 con 0x7f9904139620 2024-08-19T10:53:48.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 1 -- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9904139620 msgr2=0x7f9904135050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990a034640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9904139620 0x7f9904135050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f98f400b410 tx=0x7f98f400b8d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98f4017070 con 0x7f9904139620 2024-08-19T10:53:48.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:48.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 10 monclient: dump: 2024-08-19T10:53:48.478 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.479 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 10 monclient: _finish_auth 0 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 10 monclient: _check_auth_tickets 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990ba97640 5 monclient: authenticate success, global_id 45285 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990ba97640 10 monclient: _renew_subs 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990ba97640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f4003d90 con 0x7f9904139620 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990ba97640 1 -- 172.21.15.89:0/961027815 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990409ff70 con 0x7f9904139620 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98f4006540 con 0x7f9904139620 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:48.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9903fff640 4 set_mon_vals no callback set 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 10 monclient: dump: 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.481 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990ba97640 10 monclient: _renew_subs 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990ba97640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990ba97640 1 -- 172.21.15.89:0/961027815 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f990409e1d0 con 0x7f9904139620 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990ba97640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990ba97640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413aa90 0x7f990413ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990ba97640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f990ba97640 1 -- 172.21.15.89:0/961027815 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f99040a2a20 con 0x7f990413aa90 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f98f40060d0 con 0x7f9904139620 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413aa90 0x7f990413ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 10 monclient: get_auth_request con 0x7f990413aa90 auth_method 0 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.465+0000 7f9909833640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98e403be00 0x7f98e403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f9909833640 10 monclient(hunting): handle_auth_done global_id 74414 payload 824 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f9909833640 1 -- 172.21.15.89:0/961027815 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99040a7020 con 0x7f990413aa90 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f98f404e630 con 0x7f9904139620 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f9909032640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98e403be00 0x7f98e403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f9909032640 10 monclient: get_auth_request con 0x7f98e403be00 auth_method 0 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f9909833640 1 -- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413aa90 msgr2=0x7f990413ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f9909833640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413aa90 0x7f990413ae70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f98ec011600 tx=0x7f98ec011ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98ec00fe90 con 0x7f990413aa90 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 10 monclient: dump: 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.484 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f98ec00fa20 con 0x7f990413aa90 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413aa90 msgr2=0x7f990413ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413aa90 0x7f990413ae70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f98ec011600 tx=0x7f98ec011ac0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413aa90 msgr2=0x7f990413ae70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f99027fc640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413aa90 0x7f990413ae70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.469+0000 7f9909032640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98e403be00 0x7f98e403e2c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f98fc01c890 tx=0x7f98fc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f990ba97640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f990ba97640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413b3b0 0x7f990409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f990ba97640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f990ba97640 1 -- 172.21.15.89:0/961027815 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9904064590 con 0x7f990413b3b0 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413b3b0 0x7f990409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 10 monclient: get_auth_request con 0x7f990413b3b0 auth_method 0 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 10 monclient(hunting): handle_auth_done global_id 74423 payload 824 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 1 -- 172.21.15.89:0/961027815 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f990406ca20 con 0x7f990413b3b0 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 1 -- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413b3b0 msgr2=0x7f990409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f9909833640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413b3b0 0x7f990409c9e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98ec011600 tx=0x7f98ec0046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98ec022570 con 0x7f990413b3b0 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f99027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f99027fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f99027fc640 10 monclient: dump: 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.487 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.488 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.489 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9235 (secure 0 0 0) 0x7f98ec0064d0 con 0x7f990413b3b0 2024-08-19T10:53:48.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f99027fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:48.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f99027fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:48.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413b3b0 msgr2=0x7f990409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f99027fc640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413b3b0 0x7f990409c9e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98ec011600 tx=0x7f98ec0046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f99027fc640 1 -- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413b3b0 msgr2=0x7f990409c9e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:48.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.477+0000 7f99027fc640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f990413b3b0 0x7f990409c9e0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f98ec011600 tx=0x7f98ec0046d0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.489 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:48.489 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 128, 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 164976, 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: 6 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.490 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.491 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.492 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:48.493 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.494 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:48.495 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.496 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.498 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.500 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.501 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.502 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:48.503 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:48.504 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:48.505 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.506 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:48.507 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:48.508 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.509 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.511 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.512 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.512 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:48.512 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:48.512 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.512 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:48.512 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:48.512 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:48.512 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.512 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.513 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:48.513 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.513 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:48.513 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:48.513 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:53:48.513 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.513 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.513 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:48.513 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:48.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990ba97640 1 -- 172.21.15.89:0/961027815 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98e403be00 msgr2=0x7f98e403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990ba97640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f98e403be00 0x7f98e403e2c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f98fc01c890 tx=0x7f98fc0a7000 comp rx=0 tx=0).stop 2024-08-19T10:53:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990ba97640 10 monclient: shutdown 2024-08-19T10:53:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990ba97640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990ba97640 1 -- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9904139620 msgr2=0x7f9904135050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990ba97640 1 --2- 172.21.15.89:0/961027815 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f9904139620 0x7f9904135050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f98f400b410 tx=0x7f98f400b8d0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990a034640 1 -- 172.21.15.89:0/961027815 reap_dead start 2024-08-19T10:53:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990a034640 1 -- 172.21.15.89:0/961027815 reap_dead start 2024-08-19T10:53:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990ba97640 1 -- 172.21.15.89:0/961027815 shutdown_connections 2024-08-19T10:53:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990ba97640 1 -- 172.21.15.89:0/961027815 >> 172.21.15.89:0/961027815 conn(0x7f99040699d0 msgr2=0x7f99040ab3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990ba97640 1 -- 172.21.15.89:0/961027815 shutdown_connections 2024-08-19T10:53:48.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.481+0000 7f990ba97640 1 -- 172.21.15.89:0/961027815 wait complete. 2024-08-19T10:53:48.516 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:53:48.516 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon c 2024-08-19T10:53:48.516 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:53:48.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient: get_monmap_and_config 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient: build_initial_monmap 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient: monmap: 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:48.641035+0000 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:48.641035+0000 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:48.635 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:48.636 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:48.636 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:48.636 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:48.636 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:48.636 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:48.636 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:48.636 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 1 Processor -- start 2024-08-19T10:53:48.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 1 -- start start 2024-08-19T10:53:48.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient: init 2024-08-19T10:53:48.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient: _add_conns ranks=[3,8,2,7,4,0,5,9,6,10,1] 2024-08-19T10:53:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77680a3b30 0x7f77680a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient(hunting): picked mon.noname-g con 0x7f77680a3b30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77680a3f10 0x7f77680a4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient(hunting): picked mon.noname-e con 0x7f77680a3f10 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77681364f0 0x7f7768136970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient(hunting): picked mon.noname-b con 0x7f77681364f0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f77680a08b0 con 0x7f77681364f0 2024-08-19T10:53:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7768074170 con 0x7f77680a3b30 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f77680a0630 con 0x7f77680a3f10 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776e34e640 10 monclient(hunting): authenticate will time out at 1878.457613s 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f77677fe640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77680a3f10 msgr2=0x7f77680a4330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f77677fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77680a3f10 0x7f77680a4330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f7767fff640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77680a3b30 msgr2=0x7f77680a2a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f7767fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77680a3b30 0x7f77680a2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77681364f0 0x7f7768136970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77681364f0 0x7f7768136970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:43580/0 (socket says 172.21.15.89:43580) 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 1 -- 172.21.15.89:0/364364984 learned_addr learned my addr 172.21.15.89:0/364364984 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 10 monclient(hunting): get_auth_request con 0x7f77681364f0 auth_method 0 2024-08-19T10:53:48.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 10 monclient(hunting): handle_auth_done global_id 74432 payload 274 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 1 -- 172.21.15.89:0/364364984 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77680a3f10 msgr2=0x7f77680a4330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 1 --2- 172.21.15.89:0/364364984 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77680a3f10 0x7f77680a4330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 1 -- 172.21.15.89:0/364364984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77680a3b30 msgr2=0x7f77680a2a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 1 --2- 172.21.15.89:0/364364984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77680a3b30 0x7f77680a2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 10 monclient: _finish_hunting 0 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 1 monclient: found mon.noname-b 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:48.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:53:48.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.637+0000 7f776c8eb640 1 -- 172.21.15.89:0/364364984 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77680a1950 con 0x7f77681364f0 2024-08-19T10:53:48.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776c8eb640 1 -- 172.21.15.89:0/364364984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77681364f0 msgr2=0x7f7768136970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776c8eb640 1 --2- 172.21.15.89:0/364364984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77681364f0 0x7f7768136970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f77580098a0 tx=0x7f775801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a72d04735c4cdae9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 1 -- 172.21.15.89:0/364364984 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77580b0070 con 0x7f77681364f0 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 10 monclient: dump: 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.646 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 10 monclient: _finish_auth 0 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 1 -- 172.21.15.89:0/364364984 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77580aa810 con 0x7f77681364f0 2024-08-19T10:53:48.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 1 -- 172.21.15.89:0/364364984 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77580aae30 con 0x7f77681364f0 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 5 monclient: authenticate success, global_id 74432 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 4 set_mon_vals no callback set 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f7766ffd640 10 monclient: dump: 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.648 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 10 monclient: shutdown 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:48.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 1 -- 172.21.15.89:0/364364984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77681364f0 msgr2=0x7f7768136970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 1 --2- 172.21.15.89:0/364364984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77681364f0 0x7f7768136970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f77580098a0 tx=0x7f775801c480 comp rx=0 tx=0).stop 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 1 -- 172.21.15.89:0/364364984 shutdown_connections 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 1 --2- 172.21.15.89:0/364364984 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77680a3f10 0x7f77680a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 1 --2- 172.21.15.89:0/364364984 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77681364f0 0x7f7768136970 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 1 --2- 172.21.15.89:0/364364984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77680a3b30 0x7f77680a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 1 -- 172.21.15.89:0/364364984 >> 172.21.15.89:0/364364984 conn(0x7f77680a8f60 msgr2=0x7f77680a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 1 -- 172.21.15.89:0/364364984 shutdown_connections 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.641+0000 7f776e34e640 1 -- 172.21.15.89:0/364364984 wait complete. 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient: build_initial_monmap 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient: monmap: 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:48.650 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 1 Processor -- start 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 1 -- start start 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient: init 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient: _add_conns ranks=[8,3,1,5,2,6,4,0,7] 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77680a3b30 0x7f7768134e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient(hunting): picked mon.noname-e con 0x7f77680a3b30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77680a3f10 0x7f77681353b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient(hunting): picked mon.noname-g con 0x7f77680a3f10 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77681364f0 0x7f77681358f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient(hunting): picked mon.noname-f con 0x7f77681364f0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f776813ff60 con 0x7f77681364f0 2024-08-19T10:53:48.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f77680a4870 con 0x7f77680a3f10 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f77681400e0 con 0x7f77680a3b30 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776e34e640 10 monclient(hunting): authenticate will time out at 1878.465087s 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f7767fff640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77680a3b30 msgr2=0x7f7768134e70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f7767fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77680a3b30 0x7f7768134e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f77677fe640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77680a3f10 msgr2=0x7f77681353b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f77677fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77680a3f10 0x7f77681353b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77681364f0 0x7f77681358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77681364f0 0x7f77681358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:55402/0 (socket says 172.21.15.89:55402) 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 1 -- 172.21.15.89:0/663656080 learned_addr learned my addr 172.21.15.89:0/663656080 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:48.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 10 monclient(hunting): get_auth_request con 0x7f77681364f0 auth_method 0 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 10 monclient(hunting): handle_auth_done global_id 45226 payload 824 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 1 -- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77680a3b30 msgr2=0x7f7768134e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77680a3b30 0x7f7768134e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 1 -- 172.21.15.89:0/663656080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77680a3f10 msgr2=0x7f77681353b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77680a3f10 0x7f77681353b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 10 monclient: _finish_hunting 0 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 1 monclient: found mon.noname-f 2024-08-19T10:53:48.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:53:48.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.645+0000 7f776c8eb640 1 -- 172.21.15.89:0/663656080 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7768136980 con 0x7f77681364f0 2024-08-19T10:53:48.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776c8eb640 1 -- 172.21.15.89:0/663656080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77681364f0 msgr2=0x7f77681358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776c8eb640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77681364f0 0x7f77681358f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f77580097c0 tx=0x7f7758019220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77580b0040 con 0x7f77681364f0 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 10 monclient: dump: 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 10 monclient: _finish_auth 0 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77580044b0 con 0x7f77681364f0 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776e34e640 5 monclient: authenticate success, global_id 45226 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776e34e640 10 monclient: _renew_subs 2024-08-19T10:53:48.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776e34e640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776e34e640 1 -- 172.21.15.89:0/663656080 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77680a9560 con 0x7f77681364f0 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7758003620 con 0x7f77681364f0 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f77667fc640 4 set_mon_vals no callback set 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 10 monclient: dump: 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.661 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776e34e640 10 monclient: _renew_subs 2024-08-19T10:53:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776e34e640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776e34e640 1 -- 172.21.15.89:0/663656080 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7768121e40 con 0x7f77681364f0 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776e34e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776e34e640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137500 0x7f77681378e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776e34e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f776e34e640 1 -- 172.21.15.89:0/663656080 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f7768121ac0 con 0x7f7768137500 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7767fff640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137500 0x7f77681378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7767fff640 10 monclient: get_auth_request con 0x7f7768137500 auth_method 0 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7767fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7767fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7767fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7767fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7767fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7767fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7767fff640 10 monclient(hunting): handle_auth_done global_id 84235 payload 824 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7767fff640 1 -- 172.21.15.89:0/663656080 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77680a4330 con 0x7f7768137500 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7758004040 con 0x7f77681364f0 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f774803be00 0x7f774803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f77580e9290 con 0x7f77681364f0 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f77677fe640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f774803be00 0x7f774803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.649+0000 7f77677fe640 10 monclient: get_auth_request con 0x7f774803be00 auth_method 0 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7767fff640 1 -- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137500 msgr2=0x7f77681378e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7767fff640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137500 0x7f77681378e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f776000bcc0 tx=0x7f7760009f30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77600057d0 con 0x7f7768137500 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7764ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7764ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7764ff9640 10 monclient: dump: 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.665 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.666 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.666 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.666 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7760005360 con 0x7f7768137500 2024-08-19T10:53:48.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7764ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:48.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7764ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:48.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137500 msgr2=0x7f77681378e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7764ff9640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137500 0x7f77681378e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f776000bcc0 tx=0x7f7760009f30 comp rx=0 tx=0).stop 2024-08-19T10:53:48.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137500 msgr2=0x7f77681378e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:48.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f7764ff9640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137500 0x7f77681378e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.653+0000 7f77677fe640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f774803be00 0x7f774803e2c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f776813b410 tx=0x7f7750006d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f776e34e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f776e34e640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137e20 0x7f776811f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f776e34e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f776e34e640 1 -- 172.21.15.89:0/663656080 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f7768064590 con 0x7f7768137e20 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f7767fff640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137e20 0x7f776811f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f7767fff640 10 monclient: get_auth_request con 0x7f7768137e20 auth_method 0 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f7767fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f7767fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f7767fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f7767fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f7767fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f7767fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f7767fff640 10 monclient(hunting): handle_auth_done global_id 84244 payload 824 2024-08-19T10:53:48.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.657+0000 7f7767fff640 1 -- 172.21.15.89:0/663656080 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f776806ed40 con 0x7f7768137e20 2024-08-19T10:53:48.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7767fff640 1 -- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137e20 msgr2=0x7f776811f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7767fff640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137e20 0x7f776811f380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f776000bcc0 tx=0x7f7760005dd0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7760019040 con 0x7f7768137e20 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7764ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7764ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7764ff9640 10 monclient: dump: 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.669 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9234 (secure 0 0 0) 0x7f776001a8e0 con 0x7f7768137e20 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7764ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7764ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137e20 msgr2=0x7f776811f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7764ff9640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137e20 0x7f776811f380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f776000bcc0 tx=0x7f7760005dd0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7764ff9640 1 -- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137e20 msgr2=0x7f776811f380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:48.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f7764ff9640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7768137e20 0x7f776811f380 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f776000bcc0 tx=0x7f7760005dd0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.671 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:48.671 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:48.671 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:48.671 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:48.671 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 128, 2024-08-19T10:53:48.671 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 36963, 2024-08-19T10:53:48.671 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:48.671 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:48.671 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: 6 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.672 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.673 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:48.674 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:48.675 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.676 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.678 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:48.679 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.680 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.681 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.682 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.683 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.685 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:48.686 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:48.687 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.688 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.689 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:48.690 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:48.691 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.692 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:48.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.694 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:48.694 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:48.694 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:53:48.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.694 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.694 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:48.694 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:48.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776e34e640 1 -- 172.21.15.89:0/663656080 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f774803be00 msgr2=0x7f774803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776e34e640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f774803be00 0x7f774803e2c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f776813b410 tx=0x7f7750006d10 comp rx=0 tx=0).stop 2024-08-19T10:53:48.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776e34e640 10 monclient: shutdown 2024-08-19T10:53:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776e34e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776e34e640 1 -- 172.21.15.89:0/663656080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77681364f0 msgr2=0x7f77681358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776e34e640 1 --2- 172.21.15.89:0/663656080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77681364f0 0x7f77681358f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f77580097c0 tx=0x7f7758019220 comp rx=0 tx=0).stop 2024-08-19T10:53:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776c8eb640 1 -- 172.21.15.89:0/663656080 reap_dead start 2024-08-19T10:53:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776c8eb640 1 -- 172.21.15.89:0/663656080 reap_dead start 2024-08-19T10:53:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776e34e640 1 -- 172.21.15.89:0/663656080 shutdown_connections 2024-08-19T10:53:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776e34e640 1 -- 172.21.15.89:0/663656080 >> 172.21.15.89:0/663656080 conn(0x7f77680a8f60 msgr2=0x7f77680a7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776e34e640 1 -- 172.21.15.89:0/663656080 shutdown_connections 2024-08-19T10:53:48.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.661+0000 7f776e34e640 1 -- 172.21.15.89:0/663656080 wait complete. 2024-08-19T10:53:48.697 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:53:48.697 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-08-19T10:53:48.697 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:53:48.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.813+0000 7f65339b7640 10 monclient: get_monmap_and_config 2024-08-19T10:53:48.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.813+0000 7f65339b7640 10 monclient: build_initial_monmap 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 10 monclient: monmap: 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:48.820967+0000 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:48.820967+0000 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:48.815 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:48.816 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:48.816 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:48.816 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:48.816 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:48.816 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:48.816 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 1 Processor -- start 2024-08-19T10:53:48.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 1 -- start start 2024-08-19T10:53:48.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 10 monclient: init 2024-08-19T10:53:48.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:48.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 10 monclient: _add_conns ranks=[7,1,5,2,9,10,6,8,3,0,4] 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f652c0a5270 0x7f652c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 10 monclient(hunting): picked mon.noname-i con 0x7f652c0a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f652c0a6d30 addr v1:172.21.15.190:6789/0 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f652c0a7110 0x7f652c139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 10 monclient(hunting): picked mon.noname-h con 0x7f652c0a7110 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f652c0a8710 con 0x7f652c0a6d30 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f652c0a5a40 con 0x7f652c0a7110 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f652c0a81d0 con 0x7f652c0a5270 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f65339b7640 10 monclient(hunting): authenticate will time out at 1878.637588s 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531753640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f652c0a5270 msgr2=0x7f652c0a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:53:48.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531753640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f652c0a5270 0x7f652c0a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f652c0a7110 0x7f652c139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6530f52640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f652c0a6d30 0x7f652c1395b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.89:34042/0 (socket says 172.21.15.89:34042) 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6530f52640 1 -- 172.21.15.89:0/1718481827 learned_addr learned my addr 172.21.15.89:0/1718481827 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 10 monclient(hunting): get_auth_request con 0x7f652c0a7110 auth_method 0 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 10 monclient(hunting): handle_auth_done global_id 84245 payload 274 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 1 -- 172.21.15.89:0/1718481827 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f652c0a5270 msgr2=0x7f652c0a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:53:48.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 1 --2- 172.21.15.89:0/1718481827 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f652c0a5270 0x7f652c0a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 1 -- 172.21.15.89:0/1718481827 >> v1:172.21.15.190:6789/0 conn(0x7f652c0a6d30 legacy=0x7f652c1395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:48.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 10 monclient: _finish_hunting 0 2024-08-19T10:53:48.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 1 monclient: found mon.noname-h 2024-08-19T10:53:48.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:48.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:53:48.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.817+0000 7f6531f54640 1 -- 172.21.15.89:0/1718481827 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c0a4790 con 0x7f652c0a7110 2024-08-19T10:53:48.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6531f54640 1 -- 172.21.15.89:0/1718481827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f652c0a7110 msgr2=0x7f652c139cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6531f54640 1 --2- 172.21.15.89:0/1718481827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f652c0a7110 0x7f652c139cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6524009dc0 tx=0x7f652401c660 comp rx=0 tx=0).ready entity=mon.5 client_cookie=110645a72174ea63 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 1 -- 172.21.15.89:0/1718481827 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65240ac070 con 0x7f652c0a7110 2024-08-19T10:53:48.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:53:48.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 10 monclient: dump: 2024-08-19T10:53:48.823 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.823 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.823 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.823 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.823 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 10 monclient: _finish_auth 0 2024-08-19T10:53:48.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 10 monclient: _check_auth_tickets 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 1 -- 172.21.15.89:0/1718481827 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65240a69a0 con 0x7f652c0a7110 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 1 -- 172.21.15.89:0/1718481827 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65240a7430 con 0x7f652c0a7110 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f6523fff640 10 monclient: dump: 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.825 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f65339b7640 5 monclient: authenticate success, global_id 84245 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f65339b7640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:48.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.821+0000 7f65339b7640 4 set_mon_vals no callback set 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient: shutdown 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 -- 172.21.15.89:0/1718481827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f652c0a7110 msgr2=0x7f652c139cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 --2- 172.21.15.89:0/1718481827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f652c0a7110 0x7f652c139cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6524009dc0 tx=0x7f652401c660 comp rx=0 tx=0).stop 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 -- 172.21.15.89:0/1718481827 shutdown_connections 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 --2- 172.21.15.89:0/1718481827 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f652c0a7110 0x7f652c139cc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 --2- 172.21.15.89:0/1718481827 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f652c0a5270 0x7f652c0a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 -- 172.21.15.89:0/1718481827 >> 172.21.15.89:0/1718481827 conn(0x7f652c069a80 msgr2=0x7f652c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 -- 172.21.15.89:0/1718481827 shutdown_connections 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 -- 172.21.15.89:0/1718481827 wait complete. 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient: build_initial_monmap 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient: monmap: 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:48.827 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:48.828 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 Processor -- start 2024-08-19T10:53:48.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 -- start start 2024-08-19T10:53:48.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient: init 2024-08-19T10:53:48.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:48.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient: _add_conns ranks=[2,3,4,6,1,7,8,0,5] 2024-08-19T10:53:48.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652c0a5270 0x7f652c13de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient(hunting): picked mon.noname-b con 0x7f652c0a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f652c0a6d30 0x7f652c13e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient(hunting): picked mon.noname-g con 0x7f652c0a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f652c0a7110 0x7f652c13e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient(hunting): picked mon.noname-c con 0x7f652c0a7110 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f652c0a57d0 con 0x7f652c0a5270 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f652c0a5650 con 0x7f652c0a6d30 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f652c13a200 con 0x7f652c0a7110 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f65339b7640 10 monclient(hunting): authenticate will time out at 1878.645849s 2024-08-19T10:53:48.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6530f52640 1 -- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f652c0a6d30 msgr2=0x7f652c13e380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3301/0 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6530f52640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f652c0a6d30 0x7f652c13e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f652c0a7110 0x7f652c13e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f652c0a7110 0x7f652c13e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51416/0 (socket says 172.21.15.89:51416) 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 1 -- 172.21.15.89:0/665351397 learned_addr learned my addr 172.21.15.89:0/665351397 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 10 monclient(hunting): get_auth_request con 0x7f652c0a7110 auth_method 0 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531753640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652c0a5270 0x7f652c13de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531753640 10 monclient(hunting): get_auth_request con 0x7f652c0a5270 auth_method 0 2024-08-19T10:53:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531753640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531753640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531753640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531753640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531753640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531753640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 10 monclient(hunting): handle_auth_done global_id 84253 payload 824 2024-08-19T10:53:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 1 -- 172.21.15.89:0/665351397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f652c0a6d30 msgr2=0x7f652c13e380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f652c0a6d30 0x7f652c13e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 1 -- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652c0a5270 msgr2=0x7f652c13de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f652c0a5270 0x7f652c13de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 10 monclient: _finish_hunting 0 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 1 monclient: found mon.noname-c 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.825+0000 7f6531f54640 1 -- 172.21.15.89:0/665351397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c123640 con 0x7f652c0a7110 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531f54640 1 -- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f652c0a7110 msgr2=0x7f652c13e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531f54640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f652c0a7110 0x7f652c13e8c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6524098160 tx=0x7f6524019400 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65240ac070 con 0x7f652c0a7110 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 10 monclient: dump: 2024-08-19T10:53:48.838 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.839 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 10 monclient: _finish_auth 0 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f65339b7640 5 monclient: authenticate success, global_id 84253 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6524003dc0 con 0x7f652c0a7110 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f65240b1450 con 0x7f652c0a7110 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:48.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6522ffd640 4 set_mon_vals no callback set 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 10 monclient: dump: 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.841 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f65339b7640 10 monclient: _renew_subs 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f65339b7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f65339b7640 1 -- 172.21.15.89:0/665351397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652c0ab4e0 con 0x7f652c0a7110 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f65339b7640 10 monclient: _renew_subs 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f65339b7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f65339b7640 1 -- 172.21.15.89:0/665351397 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f652c13f8b0 con 0x7f652c0a7110 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f65339b7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f65339b7640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a03b0 0x7f652c0a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f65339b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f65339b7640 1 -- 172.21.15.89:0/665351397 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f652c123ad0 con 0x7f652c0a03b0 2024-08-19T10:53:48.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531753640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a03b0 0x7f652c0a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531753640 10 monclient: get_auth_request con 0x7f652c0a03b0 auth_method 0 2024-08-19T10:53:48.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531753640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531753640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531753640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f65240af070 con 0x7f652c0a7110 2024-08-19T10:53:48.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531753640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531753640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531753640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f651003be00 0x7f651003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531753640 10 monclient(hunting): handle_auth_done global_id 45294 payload 824 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6531753640 1 -- 172.21.15.89:0/665351397 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f652c0a67f0 con 0x7f652c0a03b0 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f65240b8070 con 0x7f652c0a7110 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6530f52640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f651003be00 0x7f651003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.829+0000 7f6530f52640 10 monclient: get_auth_request con 0x7f651003be00 auth_method 0 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6531753640 1 -- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a03b0 msgr2=0x7f652c0a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6531753640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a03b0 0x7f652c0a0790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6528003fd0 tx=0x7f6528009a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6528016070 con 0x7f652c0a03b0 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6521ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6521ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6521ffb640 10 monclient: dump: 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f652800e970 con 0x7f652c0a03b0 2024-08-19T10:53:48.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6521ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:48.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6521ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:48.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a03b0 msgr2=0x7f652c0a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6521ffb640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a03b0 0x7f652c0a0790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6528003fd0 tx=0x7f6528009a40 comp rx=0 tx=0).stop 2024-08-19T10:53:48.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a03b0 msgr2=0x7f652c0a0790 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:48.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6521ffb640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a03b0 0x7f652c0a0790 secure :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6528003fd0 tx=0x7f6528009a40 comp rx=0 tx=0).stop 2024-08-19T10:53:48.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.833+0000 7f6530f52640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f651003be00 0x7f651003e2c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f652c13f540 tx=0x7f651c007b30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f65339b7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:48.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f65339b7640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a0cd0 0x7f652c1db020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f65339b7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f65339b7640 1 -- 172.21.15.89:0/665351397 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f652c064590 con 0x7f652c0a0cd0 2024-08-19T10:53:48.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f6531753640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a0cd0 0x7f652c1db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f6531753640 10 monclient: get_auth_request con 0x7f652c0a0cd0 auth_method 0 2024-08-19T10:53:48.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f6531753640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f6531753640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f6531753640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f6531753640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f6531753640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f6531753640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f6531753640 10 monclient(hunting): handle_auth_done global_id 45303 payload 824 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.837+0000 7f6531753640 1 -- 172.21.15.89:0/665351397 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f652c139fb0 con 0x7f652c0a0cd0 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.841+0000 7f6531753640 1 -- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a0cd0 msgr2=0x7f652c1db020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.841+0000 7f6531753640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a0cd0 0x7f652c1db020 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6528003fd0 tx=0x7f65280059e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.841+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6528016070 con 0x7f652c0a0cd0 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.841+0000 7f6521ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.841+0000 7f6521ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:48.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.841+0000 7f6521ffb640 10 monclient: dump: 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:48.850 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:48.851 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:48.851 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9402 (secure 0 0 0) 0x7f6528006590 con 0x7f652c0a0cd0 2024-08-19T10:53:48.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f6521ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:48.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f6521ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:48.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a0cd0 msgr2=0x7f652c1db020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f6521ffb640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a0cd0 0x7f652c1db020 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6528003fd0 tx=0x7f65280059e0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f6521ffb640 1 -- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a0cd0 msgr2=0x7f652c1db020 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:48.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f6521ffb640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f652c0a0cd0 0x7f652c1db020 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:48.851 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 128, 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 551968, 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: 6 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:48.868 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:48.871 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:48.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:48.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:48.874 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:48.874 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:48.874 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:53:48.874 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:48.874 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:48.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:48.874 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:48.874 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:48.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f65339b7640 1 -- 172.21.15.89:0/665351397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f651003be00 msgr2=0x7f651003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f65339b7640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f651003be00 0x7f651003e2c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f652c13f540 tx=0x7f651c007b30 comp rx=0 tx=0).stop 2024-08-19T10:53:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f65339b7640 10 monclient: shutdown 2024-08-19T10:53:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f65339b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f65339b7640 1 -- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f652c0a7110 msgr2=0x7f652c13e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f65339b7640 1 --2- 172.21.15.89:0/665351397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f652c0a7110 0x7f652c13e8c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6524098160 tx=0x7f6524019400 comp rx=0 tx=0).stop 2024-08-19T10:53:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f6531f54640 1 -- 172.21.15.89:0/665351397 reap_dead start 2024-08-19T10:53:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f6531f54640 1 -- 172.21.15.89:0/665351397 reap_dead start 2024-08-19T10:53:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f65339b7640 1 -- 172.21.15.89:0/665351397 shutdown_connections 2024-08-19T10:53:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f65339b7640 1 -- 172.21.15.89:0/665351397 >> 172.21.15.89:0/665351397 conn(0x7f652c069a80 msgr2=0x7f652c0a24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f65339b7640 1 -- 172.21.15.89:0/665351397 shutdown_connections 2024-08-19T10:53:48.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.845+0000 7f65339b7640 1 -- 172.21.15.89:0/665351397 wait complete. 2024-08-19T10:53:48.877 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:53:48.877 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon f 2024-08-19T10:53:48.877 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:53:48.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient: get_monmap_and_config 2024-08-19T10:53:48.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient: build_initial_monmap 2024-08-19T10:53:48.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient: monmap: 2024-08-19T10:53:48.991 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:48.991 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:48.991 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:48.997598+0000 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:48.997598+0000 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:48.992 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:48.993 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:48.993 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:48.993 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:48.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 1 Processor -- start 2024-08-19T10:53:48.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 1 -- start start 2024-08-19T10:53:48.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient: init 2024-08-19T10:53:48.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:48.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient: _add_conns ranks=[9,6,4,8,0,5,10,3,2,7,1] 2024-08-19T10:53:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f00f809b8e0 0x7f00f809bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient(hunting): picked mon.noname-a con 0x7f00f809b8e0 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00f809ddd0 0x7f00f80a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient(hunting): picked mon.noname-d con 0x7f00f809ddd0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 0x7f00f80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient(hunting): picked mon.noname-c con 0x7f00f80a4240 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f00f809d760 con 0x7f00f809b8e0 2024-08-19T10:53:48.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f00f809d9b0 con 0x7f00f80a4240 2024-08-19T10:53:48.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:48.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f00f806fd00 con 0x7f00f809ddd0 2024-08-19T10:53:48.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:48.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00fd147640 10 monclient(hunting): authenticate will time out at 1878.814058s 2024-08-19T10:53:48.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00f809ddd0 0x7f00f80a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f759e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 0x7f00f80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f6d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f00f809b8e0 0x7f00f809bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:48.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f759e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 0x7f00f80a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51430/0 (socket says 172.21.15.89:51430) 2024-08-19T10:53:48.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f6d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f00f809b8e0 0x7f00f809bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56680/0 (socket says 172.21.15.89:56680) 2024-08-19T10:53:48.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f759e640 1 -- 172.21.15.89:0/944123264 learned_addr learned my addr 172.21.15.89:0/944123264 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:48.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00f809ddd0 0x7f00f80a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:39232/0 (socket says 172.21.15.89:39232) 2024-08-19T10:53:48.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 10 monclient(hunting): get_auth_request con 0x7f00f809ddd0 auth_method 0 2024-08-19T10:53:48.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f759e640 10 monclient(hunting): get_auth_request con 0x7f00f80a4240 auth_method 0 2024-08-19T10:53:48.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f759e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f759e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f6d9d640 10 monclient(hunting): get_auth_request con 0x7f00f809b8e0 auth_method 0 2024-08-19T10:53:48.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:48.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:48.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:48.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:48.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:48.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:48.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:49.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 10 monclient(hunting): handle_auth_done global_id 45312 payload 274 2024-08-19T10:53:49.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 1 -- 172.21.15.89:0/944123264 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 msgr2=0x7f00f80a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:49.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 1 --2- 172.21.15.89:0/944123264 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 0x7f00f80a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 1 -- 172.21.15.89:0/944123264 >> v2:172.21.15.89:3300/0 conn(0x7f00f809b8e0 msgr2=0x7f00f809bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:49.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 1 --2- 172.21.15.89:0/944123264 >> v2:172.21.15.89:3300/0 conn(0x7f00f809b8e0 0x7f00f809bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 10 monclient: _finish_hunting 0 2024-08-19T10:53:49.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 1 monclient: found mon.noname-d 2024-08-19T10:53:49.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:49.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:53:49.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.993+0000 7f00f659c640 1 -- 172.21.15.89:0/944123264 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f806db50 con 0x7f00f809ddd0 2024-08-19T10:53:49.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f659c640 1 -- 172.21.15.89:0/944123264 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00f809ddd0 msgr2=0x7f00f80a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:49.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f659c640 1 --2- 172.21.15.89:0/944123264 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00f809ddd0 0x7f00f80a3d00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0097c0 tx=0x7f00ec01c360 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e60f4f98d62bc9b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:49.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 1 -- 172.21.15.89:0/944123264 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00ec0b0070 con 0x7f00f809ddd0 2024-08-19T10:53:49.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:53:49.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 10 monclient: dump: 2024-08-19T10:53:49.007 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.007 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.007 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.007 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.007 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.007 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.007 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.008 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.008 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.008 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.008 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.008 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.008 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.008 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.008 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.008 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 10 monclient: _finish_auth 0 2024-08-19T10:53:49.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:53:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 5 monclient: authenticate success, global_id 45312 2024-08-19T10:53:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:53:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 1 -- 172.21.15.89:0/944123264 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ec0aa810 con 0x7f00f809ddd0 2024-08-19T10:53:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 4 set_mon_vals no callback set 2024-08-19T10:53:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 1 -- 172.21.15.89:0/944123264 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00ec0a4e60 con 0x7f00f809ddd0 2024-08-19T10:53:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:53:49.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00f5d9b640 10 monclient: dump: 2024-08-19T10:53:49.010 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.010 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.010 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.010 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.010 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.010 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.010 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.010 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.010 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 10 monclient: shutdown 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 1 -- 172.21.15.89:0/944123264 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00f809ddd0 msgr2=0x7f00f80a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 1 --2- 172.21.15.89:0/944123264 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00f809ddd0 0x7f00f80a3d00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0097c0 tx=0x7f00ec01c360 comp rx=0 tx=0).stop 2024-08-19T10:53:49.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 1 -- 172.21.15.89:0/944123264 shutdown_connections 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 1 --2- 172.21.15.89:0/944123264 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 0x7f00f80a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 1 --2- 172.21.15.89:0/944123264 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f00f809ddd0 0x7f00f80a3d00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 1 --2- 172.21.15.89:0/944123264 >> v2:172.21.15.89:3300/0 conn(0x7f00f809b8e0 0x7f00f809bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 1 -- 172.21.15.89:0/944123264 >> 172.21.15.89:0/944123264 conn(0x7f00f80a4df0 msgr2=0x7f00f80a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 1 -- 172.21.15.89:0/944123264 shutdown_connections 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:48.997+0000 7f00fd147640 1 -- 172.21.15.89:0/944123264 wait complete. 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient: build_initial_monmap 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient: monmap: 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:49.012 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:49.013 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 1 Processor -- start 2024-08-19T10:53:49.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 1 -- start start 2024-08-19T10:53:49.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient: init 2024-08-19T10:53:49.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:49.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient: _add_conns ranks=[8,0,4,2,5,1,7,6,3] 2024-08-19T10:53:49.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00f809b8e0 0x7f00f8092840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient(hunting): picked mon.noname-e con 0x7f00f809b8e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:53:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f809ddd0 0x7f00f8092d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient(hunting): picked mon.noname-a con 0x7f00f809ddd0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:53:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 0x7f00f81d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient(hunting): picked mon.noname-c con 0x7f00f80a4240 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f00f8069300 con 0x7f00f809ddd0 2024-08-19T10:53:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f00f80a5c00 con 0x7f00f80a4240 2024-08-19T10:53:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f00f8142cd0 con 0x7f00f809b8e0 2024-08-19T10:53:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00fd147640 10 monclient(hunting): authenticate will time out at 1878.821194s 2024-08-19T10:53:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f6d9d640 1 -- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00f809b8e0 msgr2=0x7f00f8092840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3304/0 2024-08-19T10:53:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f6d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00f809b8e0 0x7f00f8092840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f759e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 0x7f00f81d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f809ddd0 0x7f00f8092d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f759e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 0x7f00f81d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51434/0 (socket says 172.21.15.89:51434) 2024-08-19T10:53:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f759e640 1 -- 172.21.15.89:0/3383007822 learned_addr learned my addr 172.21.15.89:0/3383007822 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 10 monclient(hunting): get_auth_request con 0x7f00f809ddd0 auth_method 0 2024-08-19T10:53:49.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f759e640 10 monclient(hunting): get_auth_request con 0x7f00f80a4240 auth_method 0 2024-08-19T10:53:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f759e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f759e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:49.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 10 monclient(hunting): handle_auth_done global_id 7533 payload 824 2024-08-19T10:53:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 1 -- 172.21.15.89:0/3383007822 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00f809b8e0 msgr2=0x7f00f8092840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f00f809b8e0 0x7f00f8092840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 1 -- 172.21.15.89:0/3383007822 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 msgr2=0x7f00f81d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 0x7f00f81d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 10 monclient: _finish_hunting 0 2024-08-19T10:53:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 1 monclient: found mon.noname-a 2024-08-19T10:53:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:49.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 1 -- 172.21.15.89:0/3383007822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f80a3440 con 0x7f00f809ddd0 2024-08-19T10:53:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f759e640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f00f80a4240 0x7f00f81d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 1 -- 172.21.15.89:0/3383007822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f809ddd0 msgr2=0x7f00f8092d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.001+0000 7f00f659c640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f809ddd0 0x7f00f8092d80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f00f809c230 tx=0x7f00ec019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00ec0b0040 con 0x7f00f809ddd0 2024-08-19T10:53:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 10 monclient: dump: 2024-08-19T10:53:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.019 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.020 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 10 monclient: _finish_auth 0 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ec0044b0 con 0x7f00f809ddd0 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00fd147640 5 monclient: authenticate success, global_id 7533 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00fd147640 10 monclient: _renew_subs 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00fd147640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00fd147640 1 -- 172.21.15.89:0/3383007822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f813c6a0 con 0x7f00f809ddd0 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00ec0aabd0 con 0x7f00f809ddd0 2024-08-19T10:53:49.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 10 monclient: dump: 2024-08-19T10:53:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.022 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.023 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00fd147640 10 monclient: _renew_subs 2024-08-19T10:53:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00fd147640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:53:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00fd147640 1 -- 172.21.15.89:0/3383007822 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f8095110 con 0x7f00f809ddd0 2024-08-19T10:53:49.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f4d99640 4 set_mon_vals no callback set 2024-08-19T10:53:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00fd147640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00fd147640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f81d14b0 0x7f00f81d1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00fd147640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00fd147640 1 -- 172.21.15.89:0/3383007822 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00f8094920 con 0x7f00f81d14b0 2024-08-19T10:53:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f6d9d640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f81d14b0 0x7f00f81d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f6d9d640 10 monclient: get_auth_request con 0x7f00f81d14b0 auth_method 0 2024-08-19T10:53:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:49.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f00ec004040 con 0x7f00f809ddd0 2024-08-19T10:53:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f6d9d640 10 monclient(hunting): handle_auth_done global_id 45235 payload 824 2024-08-19T10:53:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f00e403be00 0x7f00e403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f6d9d640 1 -- 172.21.15.89:0/3383007822 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f00f81d1dd0 con 0x7f00f81d14b0 2024-08-19T10:53:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f00ec0b9070 con 0x7f00f809ddd0 2024-08-19T10:53:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f759e640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f00e403be00 0x7f00e403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.005+0000 7f00f759e640 10 monclient: get_auth_request con 0x7f00e403be00 auth_method 0 2024-08-19T10:53:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00f6d9d640 1 -- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f81d14b0 msgr2=0x7f00f81d1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:49.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00f6d9d640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f81d14b0 0x7f00f81d1890 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009510 tx=0x7f00e800ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:49.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00e8017070 con 0x7f00f81d14b0 2024-08-19T10:53:49.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00df7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00df7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:53:49.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00df7fe640 10 monclient: dump: 2024-08-19T10:53:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.029 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.030 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00f759e640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f00e403be00 0x7f00e403e2c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009580 tx=0x7f00f000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f00e800cb90 con 0x7f00f81d14b0 2024-08-19T10:53:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00df7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00df7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f81d14b0 msgr2=0x7f00f81d1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00df7fe640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f81d14b0 0x7f00f81d1890 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009510 tx=0x7f00e800ec70 comp rx=0 tx=0).stop 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f81d14b0 msgr2=0x7f00f81d1890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.009+0000 7f00df7fe640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f81d14b0 0x7f00f81d1890 secure :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009510 tx=0x7f00e800ec70 comp rx=0 tx=0).stop 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.013+0000 7f00fd147640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.013+0000 7f00fd147640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f8070cc0 0x7f00f80710d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.013+0000 7f00fd147640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.013+0000 7f00fd147640 1 -- 172.21.15.89:0/3383007822 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00f8064590 con 0x7f00f8070cc0 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f8070cc0 0x7f00f80710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 10 monclient: get_auth_request con 0x7f00f8070cc0 auth_method 0 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:49.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:49.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:49.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 10 monclient(hunting): handle_auth_done global_id 45244 payload 824 2024-08-19T10:53:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 1 -- 172.21.15.89:0/3383007822 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f00f80a3d00 con 0x7f00f8070cc0 2024-08-19T10:53:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 1 -- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f8070cc0 msgr2=0x7f00f80710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00f6d9d640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f8070cc0 0x7f00f80710d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f00e8019000 tx=0x7f00e8004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f00e8017070 con 0x7f00f8070cc0 2024-08-19T10:53:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.017+0000 7f00df7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00df7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:53:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00df7fe640 10 monclient: dump: 2024-08-19T10:53:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.034 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9402 (secure 0 0 0) 0x7f00e80066e0 con 0x7f00f8070cc0 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00df7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00df7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f8070cc0 msgr2=0x7f00f80710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00df7fe640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f8070cc0 0x7f00f80710d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f00e8019000 tx=0x7f00e8004740 comp rx=0 tx=0).stop 2024-08-19T10:53:49.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00df7fe640 1 -- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f8070cc0 msgr2=0x7f00f80710d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:49.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00df7fe640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00f8070cc0 0x7f00f80710d0 secure :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f00e8019000 tx=0x7f00e8004740 comp rx=0 tx=0).stop 2024-08-19T10:53:49.036 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:49.036 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:49.036 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:49.036 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:49.036 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 128, 2024-08-19T10:53:49.037 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 552147, 2024-08-19T10:53:49.037 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:49.037 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:49.037 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:49.037 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:49.037 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:49.037 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:49.037 INFO:teuthology.orchestra.run.smithi089.stdout: 6 2024-08-19T10:53:49.037 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:49.037 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:53:49.038 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:49.038 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:49.038 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:49.038 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:49.038 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:49.038 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:49.038 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:49.038 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:49.038 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:49.038 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:49.039 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:49.039 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:49.039 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:49.039 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:49.039 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:49.039 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:49.039 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:49.039 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:49.039 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:49.039 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:49.039 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:49.040 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:49.040 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:49.040 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:49.040 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:49.040 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:49.040 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:49.040 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:49.040 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.040 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:49.041 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:49.041 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:49.041 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:49.041 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:49.041 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:49.041 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:49.041 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:49.041 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:49.041 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:49.041 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:49.041 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:49.042 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:49.042 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:49.042 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:49.042 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:49.042 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:49.042 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:49.042 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:49.042 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:49.042 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:49.042 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:49.042 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:49.043 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:49.043 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:49.043 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:49.043 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:49.043 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:49.043 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:49.043 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:49.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.043 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:49.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.044 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:49.044 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:49.044 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.044 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.044 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.044 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.045 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:49.045 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.045 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.045 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:49.045 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.045 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.045 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.046 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:49.046 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:49.046 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.046 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.046 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.046 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.046 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:49.046 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:49.046 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.047 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.047 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:49.047 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.047 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.047 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:49.047 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.047 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.048 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.048 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:49.048 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:49.048 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.048 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.048 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.049 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:49.049 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:49.049 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.049 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.049 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.049 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:49.049 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.049 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.050 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:49.050 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.050 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.050 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.050 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:49.050 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:49.050 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.050 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.050 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.051 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:49.051 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:49.051 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.051 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:49.051 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.051 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.052 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.052 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.052 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:49.052 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.052 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.052 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.052 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:49.052 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:49.052 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.052 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.053 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.053 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:49.053 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:49.053 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.053 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.053 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.053 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:49.053 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.054 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.054 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.054 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:49.054 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.054 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.054 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.054 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:49.054 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:49.055 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.055 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.055 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.055 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.055 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.055 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:49.055 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:49.055 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.055 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.055 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.056 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:49.057 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:49.058 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:49.058 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.059 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.059 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.059 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:49.059 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:49.059 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.059 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.059 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.059 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:49.060 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:49.060 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.060 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.060 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:49.060 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.060 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.060 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.061 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:49.062 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:49.062 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.062 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.062 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.062 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.062 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:49.062 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.062 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.062 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.063 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:49.063 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.063 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.063 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.063 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:49.063 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:49.063 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.063 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.063 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.063 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.064 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.064 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.064 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:49.064 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:49.064 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.064 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:49.064 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:49.064 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:49.064 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.064 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:49.064 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:53:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 4 2024-08-19T10:53:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:49.066 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.066 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.066 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.066 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:49.066 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:49.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00fd147640 1 -- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f00e403be00 msgr2=0x7f00e403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00fd147640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f00e403be00 0x7f00e403e2c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009580 tx=0x7f00f000b040 comp rx=0 tx=0).stop 2024-08-19T10:53:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00fd147640 10 monclient: shutdown 2024-08-19T10:53:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00fd147640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00fd147640 1 -- 172.21.15.89:0/3383007822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f809ddd0 msgr2=0x7f00f8092d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00fd147640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f809ddd0 0x7f00f8092d80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f00f809c230 tx=0x7f00ec019100 comp rx=0 tx=0).stop 2024-08-19T10:53:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00f759e640 1 -- 172.21.15.89:0/3383007822 reap_dead start 2024-08-19T10:53:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00fd147640 1 -- 172.21.15.89:0/3383007822 shutdown_connections 2024-08-19T10:53:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00fd147640 1 --2- 172.21.15.89:0/3383007822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f00f809ddd0 0x7f00f8092d80 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00fd147640 1 -- 172.21.15.89:0/3383007822 >> 172.21.15.89:0/3383007822 conn(0x7f00f80a4df0 msgr2=0x7f00f80a0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:49.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00fd147640 1 -- 172.21.15.89:0/3383007822 shutdown_connections 2024-08-19T10:53:49.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.021+0000 7f00fd147640 1 -- 172.21.15.89:0/3383007822 wait complete. 2024-08-19T10:53:49.069 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:53:49.069 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-08-19T10:53:49.069 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:53:49.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.185+0000 7fa2eab0c640 10 monclient: get_monmap_and_config 2024-08-19T10:53:49.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.185+0000 7fa2eab0c640 10 monclient: build_initial_monmap 2024-08-19T10:53:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 10 monclient: monmap: 2024-08-19T10:53:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:53:49.192916+0000 2024-08-19T10:53:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:53:49.192916+0000 2024-08-19T10:53:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:53:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:53:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:49.187 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:49.188 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:49.188 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:49.188 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:49.188 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:49.188 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:49.188 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:53:49.188 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:53:49.188 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 1 Processor -- start 2024-08-19T10:53:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 1 -- start start 2024-08-19T10:53:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 10 monclient: init 2024-08-19T10:53:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 10 monclient: _add_conns ranks=[9,4,0,5,3,8,2,1,6,7,10] 2024-08-19T10:53:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa2e4137750 0x7fa2e4137b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 10 monclient(hunting): picked mon.noname-a con 0x7fa2e4137750 addr v2:172.21.15.89:3300/0 2024-08-19T10:53:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e4138070 0x7fa2e40a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 10 monclient(hunting): picked mon.noname-c con 0x7fa2e4138070 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:49.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa2e40a51b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:53:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e4072620 con 0x7fa2e40a51b0 2024-08-19T10:53:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e40739a0 con 0x7fa2e4137750 2024-08-19T10:53:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa2e4073f60 con 0x7fa2e4138070 2024-08-19T10:53:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2eab0c640 10 monclient(hunting): authenticate will time out at 1879.009624s 2024-08-19T10:53:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa2e4137750 0x7fa2e4137b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e90a9640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa2e40a51b0 0x7fa2e40a5640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:46754/0 (socket says 172.21.15.89:46754) 2024-08-19T10:53:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e90a9640 1 -- 172.21.15.89:0/3793432620 learned_addr learned my addr 172.21.15.89:0/3793432620 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:53:49.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e3fff640 1 --2- 172.21.15.89:0/3793432620 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e4138070 0x7fa2e40a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 10 monclient(hunting): get_auth_request con 0x7fa2e4137750 auth_method 0 2024-08-19T10:53:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e3fff640 10 monclient(hunting): get_auth_request con 0x7fa2e4138070 auth_method 0 2024-08-19T10:53:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:49.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 10 monclient(hunting): handle_auth_done global_id 7542 payload 274 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 1 -- 172.21.15.89:0/3793432620 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e4138070 msgr2=0x7fa2e40a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 1 --2- 172.21.15.89:0/3793432620 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e4138070 0x7fa2e40a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 1 -- 172.21.15.89:0/3793432620 >> v1:172.21.15.89:6789/0 conn(0x7fa2e40a51b0 legacy=0x7fa2e40a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 10 monclient: _finish_hunting 0 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 1 monclient: found mon.noname-a 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:53:49.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e88a8640 1 -- 172.21.15.89:0/3793432620 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e40736c0 con 0x7fa2e4137750 2024-08-19T10:53:49.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.189+0000 7fa2e3fff640 1 --2- 172.21.15.89:0/3793432620 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e4138070 0x7fa2e40a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:49.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e88a8640 1 -- 172.21.15.89:0/3793432620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2e4137750 msgr2=0x7fa2e4137b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e88a8640 1 --2- 172.21.15.89:0/3793432620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2e4137750 0x7fa2e4137b30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40097c0 tx=0x7fa2d401c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f544549adebf2a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 1 -- 172.21.15.89:0/3793432620 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa2d40b0070 con 0x7fa2e4137750 2024-08-19T10:53:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:53:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 10 monclient: dump: 2024-08-19T10:53:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.201 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.202 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 10 monclient: _finish_auth 0 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 1 -- 172.21.15.89:0/3793432620 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d40aa810 con 0x7fa2e4137750 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 1 -- 172.21.15.89:0/3793432620 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa2d40aae30 con 0x7fa2e4137750 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 5 monclient: authenticate success, global_id 7542 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 10 monclient: get_monmap_and_config success 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 4 set_mon_vals no callback set 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2e37fe640 10 monclient: dump: 2024-08-19T10:53:49.203 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.204 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 10 monclient: shutdown 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 1 -- 172.21.15.89:0/3793432620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2e4137750 msgr2=0x7fa2e4137b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 1 --2- 172.21.15.89:0/3793432620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2e4137750 0x7fa2e4137b30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40097c0 tx=0x7fa2d401c180 comp rx=0 tx=0).stop 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 1 -- 172.21.15.89:0/3793432620 shutdown_connections 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 1 --2- 172.21.15.89:0/3793432620 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e4138070 0x7fa2e40a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 1 --2- 172.21.15.89:0/3793432620 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2e4137750 0x7fa2e4137b30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 1 -- 172.21.15.89:0/3793432620 >> 172.21.15.89:0/3793432620 conn(0x7fa2e40aa0a0 msgr2=0x7fa2e40aa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:49.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 1 -- 172.21.15.89:0/3793432620 shutdown_connections 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.193+0000 7fa2eab0c640 1 -- 172.21.15.89:0/3793432620 wait complete. 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient: build_initial_monmap 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient: monmap: 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:53:49.206 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:53:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:53:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:53:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:53:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:53:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:53:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:53:49.207 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 1 Processor -- start 2024-08-19T10:53:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 1 -- start start 2024-08-19T10:53:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient: init 2024-08-19T10:53:49.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient: _reopen_session rank -1 2024-08-19T10:53:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient: _add_conns ranks=[4,7,6,5,8,0,3,2,1] 2024-08-19T10:53:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e40a51b0 0x7fa2e4136450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient(hunting): picked mon.noname-c con 0x7fa2e40a51b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:53:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa2e4137750 0x7fa2e413b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient(hunting): picked mon.noname-i con 0x7fa2e4137750 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:53:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa2e4138070 0x7fa2e413b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient(hunting): picked mon.noname-d con 0x7fa2e4138070 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:53:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fa2e4142060 con 0x7fa2e40a51b0 2024-08-19T10:53:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fa2e4141ee0 con 0x7fa2e4138070 2024-08-19T10:53:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fa2e41421e0 con 0x7fa2e4137750 2024-08-19T10:53:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient(hunting): _renew_subs 2024-08-19T10:53:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient(hunting): authenticate will time out at 1879.017066s 2024-08-19T10:53:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e40a51b0 0x7fa2e4136450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e90a9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa2e4138070 0x7fa2e413b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e40a51b0 0x7fa2e4136450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:51446/0 (socket says 172.21.15.89:51446) 2024-08-19T10:53:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 1 -- 172.21.15.89:0/4231753739 learned_addr learned my addr 172.21.15.89:0/4231753739 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:53:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 10 monclient(hunting): get_auth_request con 0x7fa2e40a51b0 auth_method 0 2024-08-19T10:53:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:49.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e3fff640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa2e4137750 msgr2=0x7fa2e413b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:53:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e3fff640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa2e4137750 0x7fa2e413b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:53:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e90a9640 10 monclient(hunting): get_auth_request con 0x7fa2e4138070 auth_method 0 2024-08-19T10:53:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e90a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e90a9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e90a9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:49.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e90a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e90a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e90a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 10 monclient(hunting): handle_auth_done global_id 84289 payload 824 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa2e4137750 msgr2=0x7fa2e413b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fa2e4137750 0x7fa2e413b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa2e4138070 msgr2=0x7fa2e413b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa2e4138070 0x7fa2e413b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 10 monclient: _finish_hunting 0 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 1 monclient: found mon.noname-c 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:53:49.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 1 -- 172.21.15.89:0/4231753739 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e40a8cd0 con 0x7fa2e40a51b0 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e90a9640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fa2e4138070 0x7fa2e413b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e40a51b0 msgr2=0x7fa2e4136450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e88a8640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e40a51b0 0x7fa2e4136450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2d401bfb0 tx=0x7fa2d4018f20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa2d40b0040 con 0x7fa2e40a51b0 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e17fa640 10 monclient: dump: 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.212 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.213 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.213 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e17fa640 10 monclient: _finish_auth 0 2024-08-19T10:53:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e17fa640 10 monclient: _check_auth_tickets 2024-08-19T10:53:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:53:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d40b9070 con 0x7fa2e40a51b0 2024-08-19T10:53:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 5 monclient: authenticate success, global_id 84289 2024-08-19T10:53:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient: _renew_subs 2024-08-19T10:53:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.197+0000 7fa2eab0c640 1 -- 172.21.15.89:0/4231753739 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e4120890 con 0x7fa2e40a51b0 2024-08-19T10:53:49.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e17fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:53:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa2d40aae60 con 0x7fa2e40a51b0 2024-08-19T10:53:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:53:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e2ffd640 4 set_mon_vals no callback set 2024-08-19T10:53:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e17fa640 10 monclient: dump: 2024-08-19T10:53:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.215 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.216 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.216 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2eab0c640 10 monclient: _renew_subs 2024-08-19T10:53:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2eab0c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:53:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2eab0c640 1 -- 172.21.15.89:0/4231753739 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e413db80 con 0x7fa2e40a51b0 2024-08-19T10:53:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2eab0c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2eab0c640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e4136990 0x7fa2e413bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2eab0c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2eab0c640 1 -- 172.21.15.89:0/4231753739 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa2e413ce90 con 0x7fa2e4136990 2024-08-19T10:53:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa2d40b3070 con 0x7fa2e40a51b0 2024-08-19T10:53:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e17fa640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa2d003be00 0x7fa2d003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e3fff640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e4136990 0x7fa2e413bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e3fff640 10 monclient: get_auth_request con 0x7fa2e4136990 auth_method 0 2024-08-19T10:53:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fa2d40e8fb0 con 0x7fa2e40a51b0 2024-08-19T10:53:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e90a9640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa2d003be00 0x7fa2d003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e90a9640 10 monclient: get_auth_request con 0x7fa2d003be00 auth_method 0 2024-08-19T10:53:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_done global_id 84254 payload 824 2024-08-19T10:53:49.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.201+0000 7fa2e3fff640 1 -- 172.21.15.89:0/4231753739 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa2e41202f0 con 0x7fa2e4136990 2024-08-19T10:53:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e3fff640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e4136990 msgr2=0x7fa2e413bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e3fff640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e4136990 0x7fa2e413bf10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa2e413b3a0 tx=0x7fa2d800ec00 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa2d8017070 con 0x7fa2e4136990 2024-08-19T10:53:49.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e17fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:53:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e17fa640 10 monclient: dump: 2024-08-19T10:53:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.222 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e90a9640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa2d003be00 0x7fa2d003e2c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc005c10 tx=0x7fa2dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa2d800c980 con 0x7fa2e4136990 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e17fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:53:49.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e4136990 msgr2=0x7fa2e413bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e17fa640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e4136990 0x7fa2e413bf10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa2e413b3a0 tx=0x7fa2d800ec00 comp rx=0 tx=0).stop 2024-08-19T10:53:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e4136990 msgr2=0x7fa2e413bf10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.205+0000 7fa2e17fa640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e4136990 0x7fa2e413bf10 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa2e413b3a0 tx=0x7fa2d800ec00 comp rx=0 tx=0).stop 2024-08-19T10:53:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2eab0c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:53:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2eab0c640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e413c450 0x7fa2e409e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:53:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2eab0c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:53:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2eab0c640 1 -- 172.21.15.89:0/4231753739 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fa2e4120f60 con 0x7fa2e413c450 2024-08-19T10:53:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2e3fff640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e413c450 0x7fa2e409e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:53:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2e3fff640 10 monclient: get_auth_request con 0x7fa2e413c450 auth_method 0 2024-08-19T10:53:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:53:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2e3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:53:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:53:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:53:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:53:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:53:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.209+0000 7fa2e3fff640 10 monclient(hunting): handle_auth_done global_id 84263 payload 824 2024-08-19T10:53:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.213+0000 7fa2e3fff640 1 -- 172.21.15.89:0/4231753739 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa2e413e6e0 con 0x7fa2e413c450 2024-08-19T10:53:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.213+0000 7fa2e3fff640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e413c450 msgr2=0x7fa2e409e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:53:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.213+0000 7fa2e3fff640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e413c450 0x7fa2e409e9f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009fd0 tx=0x7fa2d8004740 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:53:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.213+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa2d8017070 con 0x7fa2e413c450 2024-08-19T10:53:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.213+0000 7fa2e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:53:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.213+0000 7fa2e17fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.213+0000 7fa2e17fa640 10 monclient: dump: 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:53:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:53:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:53:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:53:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:53:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:53:49.227 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:53:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9234 (secure 0 0 0) 0x7fa2d8000e50 con 0x7fa2e413c450 2024-08-19T10:53:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2e17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:53:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2e17fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:53:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e413c450 msgr2=0x7fa2e409e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2e17fa640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e413c450 0x7fa2e409e9f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009fd0 tx=0x7fa2d8004740 comp rx=0 tx=0).stop 2024-08-19T10:53:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2e17fa640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e413c450 msgr2=0x7fa2e409e9f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:53:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2e17fa640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fa2e413c450 0x7fa2e409e9f0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009fd0 tx=0x7fa2d8004740 comp rx=0 tx=0).stop 2024-08-19T10:53:49.228 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:53:49.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:49.228 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:49.228 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:53:49.228 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 128, 2024-08-19T10:53:49.228 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 37507, 2024-08-19T10:53:49.228 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:53:49.229 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:53:49.229 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:53:49.229 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:53:49.229 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:53:49.229 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:53:49.229 INFO:teuthology.orchestra.run.smithi089.stdout: 6 2024-08-19T10:53:49.229 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:49.229 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 1, 2024-08-19T10:53:49.229 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:49.229 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:53:49.229 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:53:49.230 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:49.230 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:49.230 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:49.230 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:49.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:49.230 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:49.230 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:49.230 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:49.230 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:49.230 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:49.231 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:49.231 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:49.231 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:53:49.231 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:53:49.231 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:49.231 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:49.231 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:49.231 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:49.231 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:49.231 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:49.231 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:49.232 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:49.232 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:49.232 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:49.232 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:49.232 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.232 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:53:49.232 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:53:49.232 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:53:49.232 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:53:49.232 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:53:49.233 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:53:49.233 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:49.233 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:53:49.233 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:53:49.233 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:53:49.233 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:53:49.233 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:53:49.233 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:53:49.233 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:53:49.233 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:53:49.234 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:53:49.234 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:53:49.234 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:53:49.234 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:53:49.234 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:53:49.234 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:53:49.234 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:53:49.234 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:53:49.234 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:53:49.234 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:53:49.234 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:53:49.235 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:53:49.235 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:53:49.235 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:49.235 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:53:49.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.235 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:53:49.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.235 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:53:49.235 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:53:49.235 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.237 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:53:49.237 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:53:49.237 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.237 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.237 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.237 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:53:49.237 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:53:49.237 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:53:49.238 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:53:49.238 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.238 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.239 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:53:49.239 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:53:49.239 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.239 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.239 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.239 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:53:49.239 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:53:49.240 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:53:49.240 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:53:49.240 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:53:49.241 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:53:49.242 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.242 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.242 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.242 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:53:49.242 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.242 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.242 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:53:49.242 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:53:49.243 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:53:49.244 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.244 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.244 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.244 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:53:49.244 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.244 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.244 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:53:49.244 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.245 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.245 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.245 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:53:49.245 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:53:49.245 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.245 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.245 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.245 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.245 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:53:49.246 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:53:49.247 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:53:49.248 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.248 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.248 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.248 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:53:49.248 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.248 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.248 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:53:49.248 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:53:49.249 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:53:49.250 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.250 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.250 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.250 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:53:49.250 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.250 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.250 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.250 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:53:49.251 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.251 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.251 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.251 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:53:49.251 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:53:49.251 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.251 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.251 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.251 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:53:49.252 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:53:49.253 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:53:49.253 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.253 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.253 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:53:49.253 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:53:49.253 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:53:49.253 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:53:49.253 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:53:49.253 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.253 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.254 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:53:49.254 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:53:49.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.254 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:49.254 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:49.254 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:49.254 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.254 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:53:49.254 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:53:49.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:53:49.255 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:53:49.255 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:53:49.255 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:53:49.255 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:53:49.255 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:53:49.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:53:49.255 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:53:49.255 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:53:49.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2eab0c640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa2d003be00 msgr2=0x7fa2d003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2eab0c640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa2d003be00 0x7fa2d003e2c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc005c10 tx=0x7fa2dc009040 comp rx=0 tx=0).stop 2024-08-19T10:53:49.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2eab0c640 10 monclient: shutdown 2024-08-19T10:53:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2eab0c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:53:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2eab0c640 1 -- 172.21.15.89:0/4231753739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e40a51b0 msgr2=0x7fa2e4136450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:53:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2eab0c640 1 --2- 172.21.15.89:0/4231753739 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fa2e40a51b0 0x7fa2e4136450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2d401bfb0 tx=0x7fa2d4018f20 comp rx=0 tx=0).stop 2024-08-19T10:53:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2e90a9640 1 -- 172.21.15.89:0/4231753739 reap_dead start 2024-08-19T10:53:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2e90a9640 1 -- 172.21.15.89:0/4231753739 reap_dead start 2024-08-19T10:53:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2eab0c640 1 -- 172.21.15.89:0/4231753739 shutdown_connections 2024-08-19T10:53:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2eab0c640 1 -- 172.21.15.89:0/4231753739 >> 172.21.15.89:0/4231753739 conn(0x7fa2e40aa0a0 msgr2=0x7fa2e40a9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:53:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2eab0c640 1 -- 172.21.15.89:0/4231753739 shutdown_connections 2024-08-19T10:53:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:53:49.217+0000 7fa2eab0c640 1 -- 172.21.15.89:0/4231753739 wait complete. 2024-08-19T10:53:49.257 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-08-19T10:53:49.258 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-08-19T10:55:19.259 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-08-19T10:55:19.260 INFO:tasks.ceph.mon.e:Restarting daemon 2024-08-19T10:55:19.260 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-08-19T10:55:19.264 INFO:tasks.ceph.mon.e:Started 2024-08-19T10:55:19.264 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-08-19T10:55:19.264 INFO:tasks.ceph.mon.i:Restarting daemon 2024-08-19T10:55:19.264 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-08-19T10:55:19.267 INFO:tasks.ceph.mon.i:Started 2024-08-19T10:55:19.267 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-08-19T10:55:19.267 INFO:tasks.ceph.mon.g:Restarting daemon 2024-08-19T10:55:19.267 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-08-19T10:55:19.270 INFO:tasks.ceph.mon.g:Started 2024-08-19T10:55:19.270 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:55:19.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:55:19.376 INFO:tasks.ceph.mon.e.smithi089.stderr:2024-08-19T10:55:19.379+0000 7f3a8b7afdc0 -1 mon.e@-1(???).log v0 log_external_backlog rewinding external_log_to from 546 -> 0 (sync_force? mon rebuild?) 2024-08-19T10:55:19.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.395+0000 7fc76ef65640 10 monclient: get_monmap_and_config 2024-08-19T10:55:19.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.395+0000 7fc76ef65640 10 monclient: build_initial_monmap 2024-08-19T10:55:19.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.395+0000 7fc76ef65640 10 monclient: monmap: 2024-08-19T10:55:19.395 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:19.395 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:19.395 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:19.400502+0000 2024-08-19T10:55:19.395 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:19.400502+0000 2024-08-19T10:55:19.395 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:19.395 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:19.395 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:19.395 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:19.395 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:19.396 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:19.396 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:19.396 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:19.396 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:19.396 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:19.396 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:19.396 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:19.396 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:19.396 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:19.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 1 Processor -- start 2024-08-19T10:55:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 1 -- start start 2024-08-19T10:55:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 10 monclient: init 2024-08-19T10:55:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 10 monclient: _add_conns ranks=[7,10,9,5,4,1,0,2,8,3,6] 2024-08-19T10:55:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc76809b8e0 0x7fc76809bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 10 monclient(hunting): picked mon.noname-i con 0x7fc76809b8e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:19.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc76809ddd0 0x7fc7680a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 10 monclient(hunting): picked mon.noname-f con 0x7fc76809ddd0 addr v2:172.21.15.190:3300/0 2024-08-19T10:55:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc7680a4240 0x7fc7680a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 10 monclient(hunting): picked mon.noname-a con 0x7fc7680a4240 addr v2:172.21.15.89:3300/0 2024-08-19T10:55:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fc76809d760 con 0x7fc7680a4240 2024-08-19T10:55:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc76809d9b0 con 0x7fc76809ddd0 2024-08-19T10:55:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:19.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fc76806fd00 con 0x7fc76809b8e0 2024-08-19T10:55:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76ef65640 10 monclient(hunting): authenticate will time out at 1969.216712s 2024-08-19T10:55:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc7680a4240 0x7fc7680a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc7680a4240 0x7fc7680a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54684/0 (socket says 172.21.15.89:54684) 2024-08-19T10:55:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 1 -- 172.21.15.89:0/1803126554 learned_addr learned my addr 172.21.15.89:0/1803126554 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 10 monclient(hunting): get_auth_request con 0x7fc7680a4240 auth_method 0 2024-08-19T10:55:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:19.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc75ffff640 1 --2- 172.21.15.89:0/1803126554 >> v2:172.21.15.190:3300/0 conn(0x7fc76809ddd0 0x7fc7680a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76cd01640 1 --2- 172.21.15.89:0/1803126554 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc76809b8e0 0x7fc76809bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc75ffff640 10 monclient(hunting): get_auth_request con 0x7fc76809ddd0 auth_method 0 2024-08-19T10:55:19.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc75ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc75ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc75ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 10 monclient(hunting): handle_auth_done global_id 7551 payload 274 2024-08-19T10:55:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 1 -- 172.21.15.89:0/1803126554 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc76809b8e0 msgr2=0x7fc76809bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 1 --2- 172.21.15.89:0/1803126554 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc76809b8e0 0x7fc76809bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 1 -- 172.21.15.89:0/1803126554 >> v2:172.21.15.190:3300/0 conn(0x7fc76809ddd0 msgr2=0x7fc7680a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 1 --2- 172.21.15.89:0/1803126554 >> v2:172.21.15.190:3300/0 conn(0x7fc76809ddd0 0x7fc7680a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 10 monclient: _finish_hunting 0 2024-08-19T10:55:19.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 1 monclient: found mon.noname-a 2024-08-19T10:55:19.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:19.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:55:19.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76d502640 1 -- 172.21.15.89:0/1803126554 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76806db50 con 0x7fc7680a4240 2024-08-19T10:55:19.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76cd01640 10 monclient: get_auth_request con 0x7fc76809b8e0 auth_method 0 2024-08-19T10:55:19.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.399+0000 7fc76cd01640 1 --2- 172.21.15.89:0/1803126554 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc76809b8e0 0x7fc76809bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:55:19.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 1 -- 172.21.15.89:0/1803126554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7680a4240 msgr2=0x7fc7680a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:19.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 1 --2- 172.21.15.89:0/1803126554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7680a4240 0x7fc7680a46c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc7680a5b70 tx=0x7fc76001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efd530a76babec0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:19.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 1 -- 172.21.15.89:0/1803126554 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc7600090d0 con 0x7fc7680a4240 2024-08-19T10:55:19.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:55:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 10 monclient: dump: 2024-08-19T10:55:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:19.411 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 1 -- 172.21.15.89:0/1803126554 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7600a5b70 con 0x7fc7680a4240 2024-08-19T10:55:19.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 5 monclient: authenticate success, global_id 7551 2024-08-19T10:55:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 4 set_mon_vals no callback set 2024-08-19T10:55:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 1 -- 172.21.15.89:0/1803126554 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc7600a5d20 con 0x7fc7680a4240 2024-08-19T10:55:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75f7fe640 10 monclient: dump: 2024-08-19T10:55:19.413 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:19.414 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient: shutdown 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 -- 172.21.15.89:0/1803126554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7680a4240 msgr2=0x7fc7680a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 --2- 172.21.15.89:0/1803126554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7680a4240 0x7fc7680a46c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc7680a5b70 tx=0x7fc76001c880 comp rx=0 tx=0).stop 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 -- 172.21.15.89:0/1803126554 shutdown_connections 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 --2- 172.21.15.89:0/1803126554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc7680a4240 0x7fc7680a46c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 --2- 172.21.15.89:0/1803126554 >> v2:172.21.15.190:3300/0 conn(0x7fc76809ddd0 0x7fc7680a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 --2- 172.21.15.89:0/1803126554 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fc76809b8e0 0x7fc76809bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 -- 172.21.15.89:0/1803126554 >> 172.21.15.89:0/1803126554 conn(0x7fc7680a4df0 msgr2=0x7fc7680a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 -- 172.21.15.89:0/1803126554 shutdown_connections 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 -- 172.21.15.89:0/1803126554 wait complete. 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient: build_initial_monmap 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient: monmap: 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:19.416 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 Processor -- start 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 -- start start 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient: init 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient: _add_conns ranks=[5,4,8,1,0,2,7,3,6] 2024-08-19T10:55:19.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc76809b8e0 0x7fc76809a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient(hunting): picked mon.noname-h con 0x7fc76809b8e0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc76809ddd0 0x7fc76809a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient(hunting): picked mon.noname-c con 0x7fc76809ddd0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc7680a4240 0x7fc768094e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient(hunting): picked mon.noname-e con 0x7fc7680a4240 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fc768069300 con 0x7fc76809ddd0 2024-08-19T10:55:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fc7680a5c00 con 0x7fc76809b8e0 2024-08-19T10:55:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:19.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fc768142cd0 con 0x7fc7680a4240 2024-08-19T10:55:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76ef65640 10 monclient(hunting): authenticate will time out at 1969.223692s 2024-08-19T10:55:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc7680a4240 0x7fc768094e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75ffff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc76809ddd0 0x7fc76809a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc7680a4240 0x7fc768094e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:47498/0 (socket says 172.21.15.89:47498) 2024-08-19T10:55:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 1 -- 172.21.15.89:0/986790530 learned_addr learned my addr 172.21.15.89:0/986790530 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 10 monclient(hunting): get_auth_request con 0x7fc7680a4240 auth_method 0 2024-08-19T10:55:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:19.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75ffff640 10 monclient(hunting): get_auth_request con 0x7fc76809ddd0 auth_method 0 2024-08-19T10:55:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76cd01640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc76809b8e0 0x7fc76809a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76cd01640 10 monclient(hunting): get_auth_request con 0x7fc76809b8e0 auth_method 0 2024-08-19T10:55:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76cd01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76cd01640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76cd01640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 1 -- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc7680a4240 msgr2=0x7fc768094e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:55:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 1 -- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc7680a4240 msgr2=0x7fc768094e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:55:19.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc7680a4240 0x7fc768094e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc76d502640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc7680a4240 0x7fc768094e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.403+0000 7fc75ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76cd01640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76cd01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76cd01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 10 monclient(hunting): handle_auth_done global_id 84298 payload 824 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 1 -- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc7680a4240 msgr2=0x7fc768094e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc7680a4240 0x7fc768094e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 1 -- 172.21.15.89:0/986790530 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc76809b8e0 msgr2=0x7fc76809a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc76809b8e0 0x7fc76809a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:55:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 1 monclient: found mon.noname-c 2024-08-19T10:55:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 1 -- 172.21.15.89:0/986790530 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7680a5360 con 0x7fc76809ddd0 2024-08-19T10:55:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 1 -- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc76809ddd0 msgr2=0x7fc76809a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75ffff640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc76809ddd0 0x7fc76809a6c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc75800d5f0 tx=0x7fc75800dab0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 1 -- 172.21.15.89:0/986790530 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc758012c70 con 0x7fc76809ddd0 2024-08-19T10:55:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 10 monclient: dump: 2024-08-19T10:55:19.422 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:19.423 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:19.424 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:19.424 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:19.424 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:19.424 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:19.424 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:19.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:55:19.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:55:19.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:19.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 1 -- 172.21.15.89:0/986790530 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc758019070 con 0x7fc76809ddd0 2024-08-19T10:55:19.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:19.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 1 -- 172.21.15.89:0/986790530 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fc7580148a0 con 0x7fc76809ddd0 2024-08-19T10:55:19.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76ef65640 5 monclient: authenticate success, global_id 84298 2024-08-19T10:55:19.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76ef65640 10 monclient: _renew_subs 2024-08-19T10:55:19.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76ef65640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:19.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76ef65640 1 -- 172.21.15.89:0/986790530 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc768098970 con 0x7fc76809ddd0 2024-08-19T10:55:19.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:19.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75effd640 4 set_mon_vals no callback set 2024-08-19T10:55:19.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:19.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 10 monclient: dump: 2024-08-19T10:55:19.426 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:19.426 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:19.426 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:19.426 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:19.426 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:19.426 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:19.426 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:19.426 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:19.427 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76ef65640 10 monclient: _renew_subs 2024-08-19T10:55:19.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76ef65640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:19.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76ef65640 1 -- 172.21.15.89:0/986790530 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7680a3440 con 0x7fc76809ddd0 2024-08-19T10:55:19.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76ef65640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:19.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76ef65640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:19.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76ef65640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:19.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 1 -- 172.21.15.89:0/986790530 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fc758014430 con 0x7fc76809ddd0 2024-08-19T10:55:19.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc76ef65640 1 -- 172.21.15.89:0/986790530 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76809ac00 con 0x7fc76809ddd0 2024-08-19T10:55:19.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc75403be30 0x7fc75403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:19.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.407+0000 7fc75d7fa640 1 -- 172.21.15.89:0/986790530 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fc75800f040 con 0x7fc76809ddd0 2024-08-19T10:55:19.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.411+0000 7fc76cd01640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc75403be30 0x7fc75403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:19.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.411+0000 7fc76cd01640 10 monclient: get_auth_request con 0x7fc75403be30 auth_method 0 2024-08-19T10:55:19.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.411+0000 7fc76cd01640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc75403be30 0x7fc75403e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc76400ab60 tx=0x7fc764008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:19.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.415+0000 7fc75d7fa640 1 -- 172.21.15.89:0/986790530 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fc7680a5360 con 0x7fc76809ddd0 2024-08-19T10:55:19.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.415+0000 7fc75d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:19.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.415+0000 7fc75d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:19.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:55:19.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:19.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:19.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 1 -- 172.21.15.89:0/986790530 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc7680a3d00 con 0x7fc76809ddd0 2024-08-19T10:55:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc75d7fa640 1 -- 172.21.15.89:0/986790530 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3176 (secure 0 0 0) 0x7fc75804d980 con 0x7fc76809ddd0 2024-08-19T10:55:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc75d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:19.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc75d7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:19.648 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":128,"quorum":[0,1,2,4,5,6],"quorum_names":["a","f","b","c","h","d"],"quorum_leader_name":"a","quorum_age":91,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:55:19.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 1 -- 172.21.15.89:0/986790530 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc75403be30 msgr2=0x7fc75403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:19.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc75403be30 0x7fc75403e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc76400ab60 tx=0x7fc764008040 comp rx=0 tx=0).stop 2024-08-19T10:55:19.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 10 monclient: shutdown 2024-08-19T10:55:19.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:19.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 1 -- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc76809ddd0 msgr2=0x7fc76809a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:19.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc76809ddd0 0x7fc76809a6c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc75800d5f0 tx=0x7fc75800dab0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 1 -- 172.21.15.89:0/986790530 shutdown_connections 2024-08-19T10:55:19.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fc75403be30 0x7fc75403e2f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fc7680a4240 0x7fc768094e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fc76809ddd0 0x7fc76809a6c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 1 --2- 172.21.15.89:0/986790530 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fc76809b8e0 0x7fc76809a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:19.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.651+0000 7fc76ef65640 1 -- 172.21.15.89:0/986790530 >> 172.21.15.89:0/986790530 conn(0x7fc7680a4df0 msgr2=0x7fc7680a0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:19.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.655+0000 7fc76ef65640 1 -- 172.21.15.89:0/986790530 shutdown_connections 2024-08-19T10:55:19.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:19.655+0000 7fc76ef65640 1 -- 172.21.15.89:0/986790530 wait complete. 2024-08-19T10:55:22.668 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:55:22.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient: get_monmap_and_config 2024-08-19T10:55:22.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient: build_initial_monmap 2024-08-19T10:55:22.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient: monmap: 2024-08-19T10:55:22.795 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:22.795 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:22.795 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:22.801229+0000 2024-08-19T10:55:22.795 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:22.801229+0000 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:22.796 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:22.797 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:22.797 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 1 Processor -- start 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 1 -- start start 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient: init 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient: _add_conns ranks=[7,4,6,1,8,5,3,10,9,2,0] 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient(hunting): picked mon.noname-i con 0x7f09b009b390 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient(hunting): picked mon.noname-c con 0x7f09b009b770 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient(hunting): picked mon.noname-d con 0x7f09b009bbb0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:22.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f09b009c900 con 0x7f09b009b770 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f09b009d9a0 con 0x7f09b009bbb0 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f09b009dc80 con 0x7f09b009b390 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b6769640 10 monclient(hunting): authenticate will time out at 1972.618527s 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09a77fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09a77fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:55408/0 (socket says 172.21.15.89:55408) 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09a77fe640 1 -- 172.21.15.89:0/2035673296 learned_addr learned my addr 172.21.15.89:0/2035673296 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09a77fe640 10 monclient(hunting): get_auth_request con 0x7f09b009b770 auth_method 0 2024-08-19T10:55:22.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09a77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b4d06640 10 monclient(hunting): get_auth_request con 0x7f09b009bbb0 auth_method 0 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09a77fe640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 msgr2=0x7f09b00a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b4d06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b4d06640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b4d06640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09a77fe640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 msgr2=0x7f09b00a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09affff640 10 monclient(hunting): get_auth_request con 0x7f09b009b390 auth_method 0 2024-08-19T10:55:22.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b4d06640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 msgr2=0x7f09b00a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b4d06640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 msgr2=0x7f09b00a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 msgr2=0x7f09b009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 msgr2=0x7f09b009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:22.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:22.799+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09a77fe640 10 monclient(hunting): get_auth_request con 0x7f09b009b770 auth_method 0 2024-08-19T10:55:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09a77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09a77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:23.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09b4d06640 10 monclient(hunting): get_auth_request con 0x7f09b009bbb0 auth_method 0 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09b4d06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09b4d06640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09b4d06640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09a77fe640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 msgr2=0x7f09b00a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09a77fe640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 msgr2=0x7f09b00a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09affff640 10 monclient(hunting): get_auth_request con 0x7f09b009b390 auth_method 0 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:23.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09b4d06640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 msgr2=0x7f09b00a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09b4d06640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 msgr2=0x7f09b00a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 msgr2=0x7f09b009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 msgr2=0x7f09b009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:23.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.003+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:23.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09a77fe640 10 monclient(hunting): get_auth_request con 0x7f09b009b770 auth_method 0 2024-08-19T10:55:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09a77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09a77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09b4d06640 10 monclient(hunting): get_auth_request con 0x7f09b009bbb0 auth_method 0 2024-08-19T10:55:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09b4d06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09b4d06640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09b4d06640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:23.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09a77fe640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 msgr2=0x7f09b00a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09a77fe640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 msgr2=0x7f09b00a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09affff640 10 monclient(hunting): get_auth_request con 0x7f09b009b390 auth_method 0 2024-08-19T10:55:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 msgr2=0x7f09b009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 msgr2=0x7f09b009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:23.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:23.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09b4d06640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 msgr2=0x7f09b00a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:23.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09b4d06640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 msgr2=0x7f09b00a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:23.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:23.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:23.407+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:24.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09a77fe640 10 monclient(hunting): get_auth_request con 0x7f09b009b770 auth_method 0 2024-08-19T10:55:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09a77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09a77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09b4d06640 10 monclient(hunting): get_auth_request con 0x7f09b009bbb0 auth_method 0 2024-08-19T10:55:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09b4d06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09b4d06640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09b4d06640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:24.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09affff640 10 monclient(hunting): get_auth_request con 0x7f09b009b390 auth_method 0 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09b4d06640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 msgr2=0x7f09b00a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09a77fe640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 msgr2=0x7f09b00a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09b4d06640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 msgr2=0x7f09b00a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09a77fe640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 msgr2=0x7f09b00a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 msgr2=0x7f09b009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 msgr2=0x7f09b009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:24.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:55:24.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:24.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:55:24.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:24.210+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:55:25.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 10 monclient(hunting): tick 2024-08-19T10:55:25.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:55:25.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 monclient(hunting): continuing hunt 2024-08-19T10:55:25.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:55:25.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 msgr2=0x7f09b009ec20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:25.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b009b390 0x7f09b009ec20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 msgr2=0x7f09b00a4240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f09b009bbb0 0x7f09b00a4240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 msgr2=0x7f09b00a3d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f09b009b770 0x7f09b00a3d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 10 monclient: _add_conns ranks=[9,8,2,0,1,10,3,5] 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 --2- 172.21.15.89:0/2035673296 >> v2:172.21.15.89:3300/0 conn(0x7f099c003870 0x7f099c005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 10 monclient(hunting): picked mon.noname-a con 0x7f099c003870 addr v2:172.21.15.89:3300/0 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f099c006280 0x7f099c008710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 10 monclient(hunting): picked mon.noname-e con 0x7f099c006280 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f099c008d70 0x7f099c00b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 10 monclient(hunting): picked mon.noname-b con 0x7f099c008d70 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 -- 172.21.15.89:0/2035673296 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f09b009c900 con 0x7f099c003870 2024-08-19T10:55:25.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 -- 172.21.15.89:0/2035673296 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f09b009d9a0 con 0x7f099c008d70 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 1 -- 172.21.15.89:0/2035673296 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f09b009dc80 con 0x7f099c006280 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09ae7fc640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> v2:172.21.15.89:3300/0 conn(0x7f099c003870 0x7f099c005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 10 monclient(hunting): get_auth_request con 0x7f099c003870 auth_method 0 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f099c006280 0x7f099c008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09b4d06640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f099c008d70 0x7f099c00b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09b4d06640 10 monclient(hunting): get_auth_request con 0x7f099c008d70 auth_method 0 2024-08-19T10:55:25.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09b4d06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09b4d06640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09b4d06640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09a77fe640 10 monclient(hunting): get_auth_request con 0x7f099c006280 auth_method 0 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09a77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09a77fe640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f099c006280 msgr2=0x7f099c008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09a77fe640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f099c006280 msgr2=0x7f099c008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09b4d06640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:25.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09b4d06640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f099c006280 0x7f099c008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09b4d06640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09a77fe640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f099c006280 0x7f099c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 10 monclient(hunting): handle_auth_done global_id 7560 payload 274 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f099c006280 msgr2=0x7f099c008710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f099c006280 0x7f099c008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.798+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f099c008d70 msgr2=0x7f099c00b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.802+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f099c008d70 0x7f099c00b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.802+0000 7f09affff640 10 monclient: _finish_hunting 0 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.802+0000 7f09affff640 1 monclient: found mon.noname-a 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.802+0000 7f09affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:25.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.802+0000 7f09affff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:55:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.802+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b006db50 con 0x7f099c003870 2024-08-19T10:55:25.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.802+0000 7f09b4d06640 1 -- 172.21.15.89:0/2035673296 reap_dead start 2024-08-19T10:55:25.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.806+0000 7f09affff640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f099c003870 msgr2=0x7f099c005ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:25.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.806+0000 7f09affff640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f099c003870 0x7f099c005ca0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f09a0000ee0 tx=0x7f09a001dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ad00b45361af63 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:25.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 1 -- 172.21.15.89:0/2035673296 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09a00b3070 con 0x7f099c003870 2024-08-19T10:55:25.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:25.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 10 monclient: dump: 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:25.808 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 10 monclient: _finish_auth 0 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 1 -- 172.21.15.89:0/2035673296 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a00ac780 con 0x7f099c003870 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 5 monclient: authenticate success, global_id 7560 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 1 -- 172.21.15.89:0/2035673296 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f09a000f040 con 0x7f099c003870 2024-08-19T10:55:25.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 4 set_mon_vals no callback set 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09af7fe640 10 monclient: dump: 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:25.810 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:25.811 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:25.811 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:25.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 10 monclient: shutdown 2024-08-19T10:55:25.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:25.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 1 -- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f099c003870 msgr2=0x7f099c005ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:25.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f099c003870 0x7f099c005ca0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f09a0000ee0 tx=0x7f09a001dac0 comp rx=0 tx=0).stop 2024-08-19T10:55:25.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 1 -- 172.21.15.89:0/2035673296 shutdown_connections 2024-08-19T10:55:25.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 1 --2- 172.21.15.89:0/2035673296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f099c003870 0x7f099c005ca0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:25.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 1 -- 172.21.15.89:0/2035673296 >> 172.21.15.89:0/2035673296 conn(0x7f09b00a4df0 msgr2=0x7f09b00a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:25.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 1 -- 172.21.15.89:0/2035673296 shutdown_connections 2024-08-19T10:55:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 1 -- 172.21.15.89:0/2035673296 wait complete. 2024-08-19T10:55:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 10 monclient: build_initial_monmap 2024-08-19T10:55:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 10 monclient: monmap: 2024-08-19T10:55:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:25.816 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 1 Processor -- start 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 1 -- start start 2024-08-19T10:55:25.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.810+0000 7f09b6769640 10 monclient: init 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient: _add_conns ranks=[7,0,3,6,1,2,5,4,8] 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b0135dd0 0x7f09b013dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient(hunting): picked mon.noname-i con 0x7f09b0135dd0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09b013e280 0x7f09b013e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient(hunting): picked mon.noname-a con 0x7f09b013e280 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b01d5c90 0x7f09b01d8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient(hunting): picked mon.noname-g con 0x7f09b01d5c90 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f09b00a3d00 con 0x7f09b013e280 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:25.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f09b00a5bd0 con 0x7f09b01d5c90 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f09b00a3e80 con 0x7f09b0135dd0 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient(hunting): authenticate will time out at 1975.632383s 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09b013e280 0x7f09b013e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09b013e280 0x7f09b013e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40638/0 (socket says 172.21.15.89:40638) 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 1 -- 172.21.15.89:0/802783971 learned_addr learned my addr 172.21.15.89:0/802783971 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 10 monclient(hunting): get_auth_request con 0x7f09b013e280 auth_method 0 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:25.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09affff640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b0135dd0 0x7f09b013dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09affff640 10 monclient(hunting): get_auth_request con 0x7f09b0135dd0 auth_method 0 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09affff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09affff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b4d06640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b01d5c90 0x7f09b01d8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b4d06640 10 monclient(hunting): get_auth_request con 0x7f09b01d5c90 auth_method 0 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b4d06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b4d06640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b4d06640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 10 monclient(hunting): handle_auth_done global_id 7569 payload 824 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 1 -- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b0135dd0 msgr2=0x7f09b013dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:25.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b0135dd0 0x7f09b013dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 1 -- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b01d5c90 msgr2=0x7f09b01d8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b01d5c90 0x7f09b01d8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 1 monclient: found mon.noname-a 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 1 -- 172.21.15.89:0/802783971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b013d620 con 0x7f09b013e280 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 1 -- 172.21.15.89:0/802783971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09b013e280 msgr2=0x7f09b013e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09a77fe640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09b013e280 0x7f09b013e660 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0998016730 tx=0x7f0998016bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 1 -- 172.21.15.89:0/802783971 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f099800f040 con 0x7f09b013e280 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:25.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 10 monclient: dump: 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:25.822 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:25.823 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 10 monclient: _finish_auth 0 2024-08-19T10:55:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:55:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 1 -- 172.21.15.89:0/802783971 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0998016e40 con 0x7f09b013e280 2024-08-19T10:55:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 5 monclient: authenticate success, global_id 7569 2024-08-19T10:55:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient: _renew_subs 2024-08-19T10:55:25.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 1 -- 172.21.15.89:0/802783971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b013eba0 con 0x7f09b013e280 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 1 -- 172.21.15.89:0/802783971 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0998018d30 con 0x7f09b013e280 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09aeffd640 4 set_mon_vals no callback set 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09ad7fa640 10 monclient: dump: 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:25.824 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient: _renew_subs 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:55:25.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.814+0000 7f09b6769640 1 -- 172.21.15.89:0/802783971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b013b190 con 0x7f09b013e280 2024-08-19T10:55:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.818+0000 7f09b6769640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.818+0000 7f09b6769640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.818+0000 7f09b6769640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:55:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.818+0000 7f09ad7fa640 1 -- 172.21.15.89:0/802783971 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0998020020 con 0x7f09b013e280 2024-08-19T10:55:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.818+0000 7f09b6769640 1 -- 172.21.15.89:0/802783971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b00a4140 con 0x7f09b013e280 2024-08-19T10:55:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.818+0000 7f09ad7fa640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f099c03cff0 0x7f099c03f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.818+0000 7f09ad7fa640 1 -- 172.21.15.89:0/802783971 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f0998050020 con 0x7f09b013e280 2024-08-19T10:55:25.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.818+0000 7f09affff640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f099c03cff0 0x7f099c03f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.818+0000 7f09affff640 10 monclient: get_auth_request con 0x7f099c03cff0 auth_method 0 2024-08-19T10:55:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.822+0000 7f09affff640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f099c03cff0 0x7f099c03f4b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f09a00ad000 tx=0x7f09a0004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.822+0000 7f09ad7fa640 1 -- 172.21.15.89:0/802783971 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f09b013eba0 con 0x7f09b013e280 2024-08-19T10:55:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.822+0000 7f09ad7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:25.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:25.822+0000 7f09ad7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:26.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.066+0000 7f09b6769640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:55:26.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.066+0000 7f09b6769640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:26.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.066+0000 7f09b6769640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:55:26.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.066+0000 7f09b6769640 1 -- 172.21.15.89:0/802783971 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f09b01361b0 con 0x7f09b013e280 2024-08-19T10:55:26.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09ad7fa640 1 -- 172.21.15.89:0/802783971 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3175 (secure 0 0 0) 0x7f09b013d620 con 0x7f09b013e280 2024-08-19T10:55:26.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09ad7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:26.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09ad7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:26.073 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":132,"quorum":[0,1,2,4,5,6],"quorum_names":["a","f","b","c","h","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:55:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 -- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f099c03cff0 msgr2=0x7f099c03f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f099c03cff0 0x7f099c03f4b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f09a00ad000 tx=0x7f09a0004740 comp rx=0 tx=0).stop 2024-08-19T10:55:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 10 monclient: shutdown 2024-08-19T10:55:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 -- 172.21.15.89:0/802783971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09b013e280 msgr2=0x7f09b013e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:26.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09b013e280 0x7f09b013e660 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0998016730 tx=0x7f0998016bf0 comp rx=0 tx=0).stop 2024-08-19T10:55:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 -- 172.21.15.89:0/802783971 shutdown_connections 2024-08-19T10:55:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f099c03cff0 0x7f099c03f4b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f09b013e280 0x7f09b013e660 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:26.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b01d5c90 0x7f09b01d8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 --2- 172.21.15.89:0/802783971 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f09b0135dd0 0x7f09b013dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 -- 172.21.15.89:0/802783971 >> 172.21.15.89:0/802783971 conn(0x7f09b00a4df0 msgr2=0x7f09b009a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 -- 172.21.15.89:0/802783971 shutdown_connections 2024-08-19T10:55:26.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:26.074+0000 7f09b6769640 1 -- 172.21.15.89:0/802783971 wait complete. 2024-08-19T10:55:29.091 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:55:29.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient: get_monmap_and_config 2024-08-19T10:55:29.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient: build_initial_monmap 2024-08-19T10:55:29.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient: monmap: 2024-08-19T10:55:29.217 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:29.217 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:29.217 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:29.222875+0000 2024-08-19T10:55:29.217 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:29.222875+0000 2024-08-19T10:55:29.217 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:29.217 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:29.218 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:29.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 1 Processor -- start 2024-08-19T10:55:29.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 1 -- start start 2024-08-19T10:55:29.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient: init 2024-08-19T10:55:29.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:29.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient: _add_conns ranks=[9,7,0,3,2,1,10,8,6,5,4] 2024-08-19T10:55:29.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:29.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient(hunting): picked mon.noname-a con 0x7f7f7c0a5270 addr v2:172.21.15.89:3300/0 2024-08-19T10:55:29.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:29.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient(hunting): picked mon.noname-i con 0x7f7f7c0a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:29.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7f7c1395b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:55:29.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f7c0a8710 con 0x7f7f7c1395b0 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f7f7c0a7650 con 0x7f7f7c0a5270 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7f7c0a5a40 con 0x7f7f7c0a6d30 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f82b91640 10 monclient(hunting): authenticate will time out at 1979.039611s 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f8092d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f8112e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7f7c1395b0 0x7f7f7c139a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50792/0 (socket says 172.21.15.89:50792) 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f8112e640 1 -- 172.21.15.89:0/1508455954 learned_addr learned my addr 172.21.15.89:0/1508455954 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f8092d640 10 monclient(hunting): get_auth_request con 0x7f7f7c0a5270 auth_method 0 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f8092d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f8092d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:29.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f8092d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f8092d640 1 -- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 msgr2=0x7f7f7c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f8092d640 1 -- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 msgr2=0x7f7f7c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.218+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.222+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.222+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request con 0x7f7f7c0a6d30 auth_method 0 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.222+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.222+0000 7f7f7bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.222+0000 7f7f7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.222+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f7c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.222+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f7c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:29.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.222+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.222+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:29.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.222+0000 7f7f8112e640 1 -- 172.21.15.89:0/1508455954 >> v1:172.21.15.89:6789/0 conn(0x7f7f7c1395b0 legacy=0x7f7f7c139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:29.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.422+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.422+0000 7f7f8092d640 10 monclient(hunting): get_auth_request con 0x7f7f7c0a5270 auth_method 0 2024-08-19T10:55:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.422+0000 7f7f8092d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.422+0000 7f7f8092d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.422+0000 7f7f8092d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.422+0000 7f7f8092d640 1 -- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 msgr2=0x7f7f7c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.422+0000 7f7f8092d640 1 -- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 msgr2=0x7f7f7c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.422+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:29.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.422+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:29.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.426+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:29.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.426+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request con 0x7f7f7c0a6d30 auth_method 0 2024-08-19T10:55:29.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.426+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:29.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.426+0000 7f7f7bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:29.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.426+0000 7f7f7bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:29.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.426+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f7c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:29.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.426+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f7c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:29.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.426+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:29.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.426+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:29.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.826+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:29.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.826+0000 7f7f8092d640 10 monclient(hunting): get_auth_request con 0x7f7f7c0a5270 auth_method 0 2024-08-19T10:55:29.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.826+0000 7f7f8092d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:29.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.826+0000 7f7f8092d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:29.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.826+0000 7f7f8092d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:29.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.826+0000 7f7f8092d640 1 -- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 msgr2=0x7f7f7c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:29.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.826+0000 7f7f8092d640 1 -- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 msgr2=0x7f7f7c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:29.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.826+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:29.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.826+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:29.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.830+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:29.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.830+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request con 0x7f7f7c0a6d30 auth_method 0 2024-08-19T10:55:29.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.830+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:29.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.830+0000 7f7f7bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:29.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.830+0000 7f7f7bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:29.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.830+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f7c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:29.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.830+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f7c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:29.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.830+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:29.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:29.830+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:30.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.630+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:30.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.630+0000 7f7f8092d640 10 monclient(hunting): get_auth_request con 0x7f7f7c0a5270 auth_method 0 2024-08-19T10:55:30.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.630+0000 7f7f8092d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:30.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.630+0000 7f7f8092d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:30.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.630+0000 7f7f8092d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:30.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.630+0000 7f7f8092d640 1 -- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 msgr2=0x7f7f7c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:30.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.630+0000 7f7f8092d640 1 -- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 msgr2=0x7f7f7c0a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:30.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.630+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:30.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.630+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:55:30.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.634+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:30.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.634+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request con 0x7f7f7c0a6d30 auth_method 0 2024-08-19T10:55:30.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.634+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:30.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.634+0000 7f7f7bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:30.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.634+0000 7f7f7bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:30.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.634+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f7c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:30.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.634+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f7c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:30.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.634+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:30.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:30.634+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:55:31.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.806+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3592800488 0 0) 0x7f7f6c001460 con 0x7f7f7c1395b0 2024-08-19T10:55:31.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.806+0000 7f7f7b7fe640 10 monclient(hunting): _negotiate 2024-08-19T10:55:31.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.806+0000 7f7f7b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:31.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.806+0000 7f7f7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:31.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.806+0000 7f7f7b7fe640 10 monclient(hunting): my global_id is 7578 2024-08-19T10:55:31.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.806+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f68003530 con 0x7f7f7c1395b0 2024-08-19T10:55:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2299207841 0 0) 0x7f7f68003530 con 0x7f7f7c1395b0 2024-08-19T10:55:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f7c0a7110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f7c0a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 msgr2=0x7f7f7c0a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 --2- 172.21.15.89:0/1508455954 >> v2:172.21.15.89:3300/0 conn(0x7f7f7c0a5270 0x7f7f7c0a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 monclient: found mon.noname-a-legacy 2024-08-19T10:55:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.89:6789/0 2024-08-19T10:55:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c0a81d0 con 0x7f7f7c1395b0 2024-08-19T10:55:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 3927028378 0 0) 0x7f7f6c0048b0 con 0x7f7f7c1395b0 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient: dump: 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] but i'm connected to v1:172.21.15.89:6789/0, reconnecting 2024-08-19T10:55:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 >> v1:172.21.15.89:6789/0 conn(0x7f7f7c1395b0 legacy=0x7f7f7c139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient: _add_conns ranks=[1,7,0,5,6,3,4,2,8] 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680042e0 0x7f7f68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient(hunting): picked mon.f con 0x7f7f680042e0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f68004f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient(hunting): picked mon.i con 0x7f7f7c0a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f680099c0 0x7f7f6800bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient(hunting): picked mon.a con 0x7f7f680099c0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0a7650 con 0x7f7f680099c0 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0a5a40 con 0x7f7f680042e0 2024-08-19T10:55:31.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7f6800c480 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7b7fe640 10 monclient(hunting): _finish_auth 0 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f68004f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8112e640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f680099c0 0x7f7f6800bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8092d640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680042e0 0x7f7f68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request con 0x7f7f7c0a6d30 auth_method 0 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8092d640 10 monclient(hunting): get_auth_request con 0x7f7f680042e0 auth_method 0 2024-08-19T10:55:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8092d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8092d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8092d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8112e640 10 monclient(hunting): get_auth_request con 0x7f7f680099c0 auth_method 0 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8112e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8112e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8112e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8092d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8092d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8092d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:55:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8112e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8112e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f8112e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient(hunting): handle_auth_done global_id 7578 payload 293 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680042e0 msgr2=0x7f7f68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680042e0 0x7f7f68004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f680099c0 msgr2=0x7f7f6800bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f680099c0 0x7f7f6800bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 1 monclient: found mon.i 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.190:3303/0 2024-08-19T10:55:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f68003530 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient: _finish_auth 0 2024-08-19T10:55:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:55:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f7bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f82b91640 5 monclient: authenticate success, global_id 7578 2024-08-19T10:55:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.810+0000 7f7f82b91640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7bfff640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f68004f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f68004f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f640099c0 tx=0x7f7f6401d420 comp rx=0 tx=0).ready entity=mon.7 client_cookie=af462e0f93ddfff5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7f640b3070 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:55:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7b7fe640 10 monclient: dump: 2024-08-19T10:55:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 <== mon.7 v2:172.21.15.190:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f640ac780 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7b7fe640 1 -- 172.21.15.89:0/1508455954 <== mon.7 v2:172.21.15.190:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7f64009640 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7b7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f7b7fe640 10 monclient: dump: 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f82b91640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f82b91640 4 set_mon_vals no callback set 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f82b91640 10 monclient: shutdown 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f82b91640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f82b91640 1 -- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f68004f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.818+0000 7f7f82b91640 1 --2- 172.21.15.89:0/1508455954 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7f7c0a6d30 0x7f7f68004f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f640099c0 tx=0x7f7f6401d420 comp rx=0 tx=0).stop 2024-08-19T10:55:31.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f8112e640 1 -- 172.21.15.89:0/1508455954 reap_dead start 2024-08-19T10:55:31.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 1 -- 172.21.15.89:0/1508455954 shutdown_connections 2024-08-19T10:55:31.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 1 -- 172.21.15.89:0/1508455954 >> 172.21.15.89:0/1508455954 conn(0x7f7f7c069a80 msgr2=0x7f7f7c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:31.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 1 -- 172.21.15.89:0/1508455954 shutdown_connections 2024-08-19T10:55:31.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 1 -- 172.21.15.89:0/1508455954 wait complete. 2024-08-19T10:55:31.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 10 monclient: build_initial_monmap 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 10 monclient: monmap: 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 1 Processor -- start 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 1 -- start start 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 10 monclient: init 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 10 monclient: _add_conns ranks=[2,6,1,8,7,5,0,4,3] 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7c0a6d30 0x7f7f7c1d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 10 monclient(hunting): picked mon.noname-b con 0x7f7f7c0a6d30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.822+0000 7f7f82b91640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7f7c137730 0x7f7f7c137b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 10 monclient(hunting): picked mon.noname-d con 0x7f7f7c137730 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0a0cf0 0x7f7f7c0a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 10 monclient(hunting): picked mon.noname-f con 0x7f7f7c0a0cf0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0a57d0 con 0x7f7f7c0a0cf0 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c0a5650 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7f7c142230 con 0x7f7f7c137730 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 10 monclient(hunting): authenticate will time out at 1981.644121s 2024-08-19T10:55:31.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7c0a6d30 0x7f7f7c1d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f7bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7f7c137730 0x7f7f7c137b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7c0a6d30 0x7f7f7c1d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60810/0 (socket says 172.21.15.89:60810) 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 1 -- 172.21.15.89:0/3399752241 learned_addr learned my addr 172.21.15.89:0/3399752241 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f7bfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7f7c137730 0x7f7f7c137b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36396/0 (socket says 172.21.15.89:36396) 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 10 monclient(hunting): get_auth_request con 0x7f7f7c0a6d30 auth_method 0 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request con 0x7f7f7c137730 auth_method 0 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f7bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:31.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8112e640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0a0cf0 0x7f7f7c0a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8112e640 10 monclient(hunting): get_auth_request con 0x7f7f7c0a0cf0 auth_method 0 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8112e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8112e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8112e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 10 monclient(hunting): handle_auth_done global_id 74459 payload 824 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 1 -- 172.21.15.89:0/3399752241 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7f7c137730 msgr2=0x7f7f7c137b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7f7c137730 0x7f7f7c137b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:31.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 1 -- 172.21.15.89:0/3399752241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0a0cf0 msgr2=0x7f7f7c0a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0a0cf0 0x7f7f7c0a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 10 monclient: _finish_hunting 0 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 1 monclient: found mon.noname-b 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 1 -- 172.21.15.89:0/3399752241 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c0aad10 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 1 -- 172.21.15.89:0/3399752241 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f7c1d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f8092d640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7c0a6d30 0x7f7f7c1d13e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7f74017bd0 tx=0x7f7f74015730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 1 -- 172.21.15.89:0/3399752241 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7f7400f040 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:55:31.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 10 monclient: dump: 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:31.835 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 10 monclient: _finish_auth 0 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 10 monclient: _check_auth_tickets 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 5 monclient: authenticate success, global_id 74459 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 1 -- 172.21.15.89:0/3399752241 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f74017cc0 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 10 monclient: _renew_subs 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 1 -- 172.21.15.89:0/3399752241 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7c1241e0 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:31.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 1 -- 172.21.15.89:0/3399752241 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7f74017e70 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f7affd640 4 set_mon_vals no callback set 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f797fa640 10 monclient: dump: 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:31.837 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 10 monclient: _renew_subs 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.826+0000 7f7f82b91640 1 -- 172.21.15.89:0/3399752241 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f7c0a2660 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.830+0000 7f7f797fa640 1 -- 172.21.15.89:0/3399752241 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7f74004020 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.830+0000 7f7f82b91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:31.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.830+0000 7f7f82b91640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:31.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.830+0000 7f7f82b91640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:55:31.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.830+0000 7f7f797fa640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f5c03be00 0x7f7f5c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:31.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.830+0000 7f7f82b91640 1 -- 172.21.15.89:0/3399752241 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f7c064590 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.830+0000 7f7f797fa640 1 -- 172.21.15.89:0/3399752241 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f7f740503a0 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.830+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f5c03be00 0x7f7f5c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:31.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.830+0000 7f7f7bfff640 10 monclient: get_auth_request con 0x7f7f5c03be00 auth_method 0 2024-08-19T10:55:31.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.834+0000 7f7f7bfff640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f5c03be00 0x7f7f5c03e2c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7f640ad000 tx=0x7f7f64004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:31.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.834+0000 7f7f797fa640 1 -- 172.21.15.89:0/3399752241 <== mon.2 v2:172.21.15.89:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7f7404e260 con 0x7f7f7c0a6d30 2024-08-19T10:55:31.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.834+0000 7f7f797fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:31.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:31.834+0000 7f7f797fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:32.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.074+0000 7f7f82b91640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:55:32.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.074+0000 7f7f82b91640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:32.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.074+0000 7f7f82b91640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:55:32.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.074+0000 7f7f82b91640 1 -- 172.21.15.89:0/3399752241 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7f7c06c9e0 con 0x7f7f7c0a6d30 2024-08-19T10:55:32.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.682+0000 7f7f797fa640 1 -- 172.21.15.89:0/3399752241 <== mon.2 v2:172.21.15.89:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7f7f740500d0 con 0x7f7f7c0a6d30 2024-08-19T10:55:32.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.682+0000 7f7f797fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:32.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.682+0000 7f7f797fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:32.683 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":140,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:55:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 -- 172.21.15.89:0/3399752241 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f5c03be00 msgr2=0x7f7f5c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f5c03be00 0x7f7f5c03e2c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7f640ad000 tx=0x7f7f64004ac0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 10 monclient: shutdown 2024-08-19T10:55:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 -- 172.21.15.89:0/3399752241 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7c0a6d30 msgr2=0x7f7f7c1d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7c0a6d30 0x7f7f7c1d13e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7f74017bd0 tx=0x7f7f74015730 comp rx=0 tx=0).stop 2024-08-19T10:55:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 -- 172.21.15.89:0/3399752241 shutdown_connections 2024-08-19T10:55:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7f5c03be00 0x7f7f5c03e2c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0a0cf0 0x7f7f7c0a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f7f7c137730 0x7f7f7c137b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 --2- 172.21.15.89:0/3399752241 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7f7c0a6d30 0x7f7f7c1d13e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 -- 172.21.15.89:0/3399752241 >> 172.21.15.89:0/3399752241 conn(0x7f7f7c069a80 msgr2=0x7f7f7c0aa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 -- 172.21.15.89:0/3399752241 shutdown_connections 2024-08-19T10:55:32.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.686+0000 7f7f82b91640 1 -- 172.21.15.89:0/3399752241 wait complete. 2024-08-19T10:55:32.709 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:55:32.709 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:55:32.709 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:55:32.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient: get_monmap_and_config 2024-08-19T10:55:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient: build_initial_monmap 2024-08-19T10:55:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient: monmap: 2024-08-19T10:55:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:32.845557+0000 2024-08-19T10:55:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:32.845557+0000 2024-08-19T10:55:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:32.840 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:32.841 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:32.841 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:32.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 1 Processor -- start 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 1 -- start start 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient: init 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient: _add_conns ranks=[2,7,4,0,5,3,10,6,9,8,1] 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff670139880 0x7ff670139c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient(hunting): picked mon.noname-b con 0x7ff670139880 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff67006e4f0 0x7ff6700a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient(hunting): picked mon.noname-i con 0x7ff67006e4f0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff67013a1a0 0x7ff6700a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient(hunting): picked mon.noname-c con 0x7ff67013a1a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff67006de80 con 0x7ff670139880 2024-08-19T10:55:32.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff67006f200 con 0x7ff67013a1a0 2024-08-19T10:55:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff67006ef20 con 0x7ff67006e4f0 2024-08-19T10:55:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff675f8d640 10 monclient(hunting): authenticate will time out at 1982.662163s 2024-08-19T10:55:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff670139880 0x7ff670139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff67013a1a0 0x7ff6700a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff670139880 0x7ff670139c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60820/0 (socket says 172.21.15.89:60820) 2024-08-19T10:55:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66f7fe640 1 -- 172.21.15.89:0/81468544 learned_addr learned my addr 172.21.15.89:0/81468544 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:32.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66f7fe640 10 monclient(hunting): get_auth_request con 0x7ff670139880 auth_method 0 2024-08-19T10:55:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 10 monclient(hunting): get_auth_request con 0x7ff67013a1a0 auth_method 0 2024-08-19T10:55:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:32.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66effd640 1 --2- 172.21.15.89:0/81468544 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff67006e4f0 0x7ff6700a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 10 monclient(hunting): handle_auth_done global_id 84307 payload 274 2024-08-19T10:55:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 1 -- 172.21.15.89:0/81468544 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff67006e4f0 msgr2=0x7ff6700a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 1 --2- 172.21.15.89:0/81468544 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff67006e4f0 0x7ff6700a4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 1 -- 172.21.15.89:0/81468544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff670139880 msgr2=0x7ff670139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 1 --2- 172.21.15.89:0/81468544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff670139880 0x7ff670139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:55:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 1 monclient: found mon.noname-c 2024-08-19T10:55:32.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66ffff640 1 -- 172.21.15.89:0/81468544 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67006e090 con 0x7ff67013a1a0 2024-08-19T10:55:32.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.842+0000 7ff66f7fe640 1 --2- 172.21.15.89:0/81468544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff670139880 0x7ff670139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66ffff640 1 -- 172.21.15.89:0/81468544 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff67013a1a0 msgr2=0x7ff6700a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66ffff640 1 --2- 172.21.15.89:0/81468544 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff67013a1a0 0x7ff6700a4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff668009dc0 tx=0x7ff66801c850 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a735febef6fa18eb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 1 -- 172.21.15.89:0/81468544 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6680b2070 con 0x7ff67013a1a0 2024-08-19T10:55:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 10 monclient: dump: 2024-08-19T10:55:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:55:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 5 monclient: authenticate success, global_id 84307 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 1 -- 172.21.15.89:0/81468544 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6680a59b0 con 0x7ff67013a1a0 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 4 set_mon_vals no callback set 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 1 -- 172.21.15.89:0/81468544 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6680a5ba0 con 0x7ff67013a1a0 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff66e7fc640 10 monclient: dump: 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:32.858 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 10 monclient: shutdown 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 1 -- 172.21.15.89:0/81468544 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff67013a1a0 msgr2=0x7ff6700a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 1 --2- 172.21.15.89:0/81468544 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff67013a1a0 0x7ff6700a4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff668009dc0 tx=0x7ff66801c850 comp rx=0 tx=0).stop 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 1 -- 172.21.15.89:0/81468544 shutdown_connections 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 1 --2- 172.21.15.89:0/81468544 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff67013a1a0 0x7ff6700a4f80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 1 --2- 172.21.15.89:0/81468544 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff67006e4f0 0x7ff6700a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 1 --2- 172.21.15.89:0/81468544 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff670139880 0x7ff670139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 1 -- 172.21.15.89:0/81468544 >> 172.21.15.89:0/81468544 conn(0x7ff6700aa1b0 msgr2=0x7ff6700aa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 1 -- 172.21.15.89:0/81468544 shutdown_connections 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.846+0000 7ff675f8d640 1 -- 172.21.15.89:0/81468544 wait complete. 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient: build_initial_monmap 2024-08-19T10:55:32.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient: monmap: 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 1 Processor -- start 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 1 -- start start 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient: init 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient: _add_conns ranks=[3,0,6,2,4,8,5,1,7] 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67006e4f0 0x7ff6701d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient(hunting): picked mon.noname-g con 0x7ff67006e4f0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff670139880 0x7ff670136700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient(hunting): picked mon.noname-a con 0x7ff670139880 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff67013a1a0 0x7ff670136c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient(hunting): picked mon.noname-d con 0x7ff67013a1a0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff670143390 con 0x7ff670139880 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff670143210 con 0x7ff67006e4f0 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff670143510 con 0x7ff67013a1a0 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff675f8d640 10 monclient(hunting): authenticate will time out at 1982.669085s 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66effd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff670139880 0x7ff670136700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff67013a1a0 0x7ff670136c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff67013a1a0 0x7ff670136c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36408/0 (socket says 172.21.15.89:36408) 2024-08-19T10:55:32.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 1 -- 172.21.15.89:0/4089342104 learned_addr learned my addr 172.21.15.89:0/4089342104 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 10 monclient(hunting): get_auth_request con 0x7ff67013a1a0 auth_method 0 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66effd640 10 monclient(hunting): get_auth_request con 0x7ff670139880 auth_method 0 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:32.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 10 monclient(hunting): handle_auth_done global_id 45339 payload 824 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 1 -- 172.21.15.89:0/4089342104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67006e4f0 msgr2=0x7ff6701d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66f7fe640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67006e4f0 0x7ff6701d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff67006e4f0 0x7ff6701d16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 1 -- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff670139880 msgr2=0x7ff670136700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff670139880 0x7ff670136700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 1 monclient: found mon.noname-d 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:55:32.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 1 -- 172.21.15.89:0/4089342104 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6701221a0 con 0x7ff67013a1a0 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 1 -- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff67013a1a0 msgr2=0x7ff670136c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff66ffff640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff67013a1a0 0x7ff670136c40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff66801c680 tx=0x7ff6680195f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff6680b2050 con 0x7ff67013a1a0 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff657fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff657fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff657fff640 10 monclient: dump: 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:32.865 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff657fff640 10 monclient: _finish_auth 0 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff657fff640 10 monclient: _check_auth_tickets 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff657fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff675f8d640 5 monclient: authenticate success, global_id 45339 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.850+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6680be070 con 0x7ff67013a1a0 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff668003dc0 con 0x7ff67013a1a0 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 10 monclient: dump: 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:32.867 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66dffb640 4 set_mon_vals no callback set 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff675f8d640 10 monclient: _renew_subs 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff675f8d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff675f8d640 1 -- 172.21.15.89:0/4089342104 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67013f900 con 0x7ff67013a1a0 2024-08-19T10:55:32.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff675f8d640 10 monclient: _renew_subs 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff675f8d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff675f8d640 1 -- 172.21.15.89:0/4089342104 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6701209a0 con 0x7ff67013a1a0 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff675f8d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff675f8d640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e010 0x7ff67002d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff675f8d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff675f8d640 1 -- 172.21.15.89:0/4089342104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff670064590 con 0x7ff67013e010 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e010 0x7ff67002d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 10 monclient: get_auth_request con 0x7ff67013e010 auth_method 0 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:32.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff668004040 con 0x7ff67013a1a0 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 10 monclient(hunting): handle_auth_done global_id 7596 payload 824 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 1 -- 172.21.15.89:0/4089342104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6700a8070 con 0x7ff67013e010 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff65c03be30 0x7ff65c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7ff6680ea0b0 con 0x7ff67013a1a0 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66effd640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff65c03be30 0x7ff65c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66effd640 10 monclient: get_auth_request con 0x7ff65c03be30 auth_method 0 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 1 -- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e010 msgr2=0x7ff67002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff66f7fe640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e010 0x7ff67002d1d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff6600107c0 tx=0x7ff660010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff66000f040 con 0x7ff67013e010 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:32.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.854+0000 7ff657fff640 10 monclient: dump: 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:32.871 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:32.872 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.858+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff660010e90 con 0x7ff67013e010 2024-08-19T10:55:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.858+0000 7ff657fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.858+0000 7ff657fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.858+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e010 msgr2=0x7ff67002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.858+0000 7ff657fff640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e010 0x7ff67002d1d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff6600107c0 tx=0x7ff660010c80 comp rx=0 tx=0).stop 2024-08-19T10:55:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.858+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e010 msgr2=0x7ff67002d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:32.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.858+0000 7ff657fff640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e010 0x7ff67002d1d0 secure :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff6600107c0 tx=0x7ff660010c80 comp rx=0 tx=0).stop 2024-08-19T10:55:32.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.858+0000 7ff66effd640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff65c03be30 0x7ff65c03e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff67013efb0 tx=0x7ff664008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:32.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff675f8d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:32.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff675f8d640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e3f0 0x7ff67002d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:32.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff675f8d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff675f8d640 1 -- 172.21.15.89:0/4089342104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff6700a7bb0 con 0x7ff67013e3f0 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff66f7fe640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e3f0 0x7ff67002d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff66f7fe640 10 monclient: get_auth_request con 0x7ff67013e3f0 auth_method 0 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff66f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff66f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff66f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff66f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff66f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff66f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff66f7fe640 10 monclient(hunting): handle_auth_done global_id 7605 payload 824 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.862+0000 7ff66f7fe640 1 -- 172.21.15.89:0/4089342104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff670138e90 con 0x7ff67013e3f0 2024-08-19T10:55:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff66f7fe640 1 -- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e3f0 msgr2=0x7ff67002d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff66f7fe640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e3f0 0x7ff67002d710 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff6600107c0 tx=0x7ff6600046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff660021570 con 0x7ff67013e3f0 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff657fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff657fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff657fff640 10 monclient: dump: 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9438 (secure 0 0 0) 0x7ff66001cab0 con 0x7ff67013e3f0 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff657fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff657fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e3f0 msgr2=0x7ff67002d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff657fff640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e3f0 0x7ff67002d710 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff6600107c0 tx=0x7ff6600046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff657fff640 1 -- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e3f0 msgr2=0x7ff67002d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff657fff640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff67013e3f0 0x7ff67002d710 secure :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff6600107c0 tx=0x7ff6600046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.878 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:55:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 1062679, 2024-08-19T10:55:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:55:32.902 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:32.902 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:32.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:32.902 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:32.902 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:32.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff675f8d640 1 -- 172.21.15.89:0/4089342104 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff65c03be30 msgr2=0x7ff65c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:32.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff675f8d640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff65c03be30 0x7ff65c03e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff67013efb0 tx=0x7ff664008040 comp rx=0 tx=0).stop 2024-08-19T10:55:32.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff675f8d640 10 monclient: shutdown 2024-08-19T10:55:32.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff675f8d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:32.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff675f8d640 1 -- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff67013a1a0 msgr2=0x7ff670136c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:32.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff675f8d640 1 --2- 172.21.15.89:0/4089342104 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff67013a1a0 0x7ff670136c40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff66801c680 tx=0x7ff6680195f0 comp rx=0 tx=0).stop 2024-08-19T10:55:32.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff66ffff640 1 -- 172.21.15.89:0/4089342104 reap_dead start 2024-08-19T10:55:32.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff66ffff640 1 -- 172.21.15.89:0/4089342104 reap_dead start 2024-08-19T10:55:32.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff675f8d640 1 -- 172.21.15.89:0/4089342104 shutdown_connections 2024-08-19T10:55:32.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff675f8d640 1 -- 172.21.15.89:0/4089342104 >> 172.21.15.89:0/4089342104 conn(0x7ff6700aa1b0 msgr2=0x7ff67013fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:32.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff675f8d640 1 -- 172.21.15.89:0/4089342104 shutdown_connections 2024-08-19T10:55:32.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:32.866+0000 7ff675f8d640 1 -- 172.21.15.89:0/4089342104 wait complete. 2024-08-19T10:55:32.904 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:32.904 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:55:32.905 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:55:33.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.014+0000 7f197ea30640 10 monclient: get_monmap_and_config 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.014+0000 7f197ea30640 10 monclient: build_initial_monmap 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.014+0000 7f197ea30640 10 monclient: monmap: 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:33.020344+0000 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:33.020344+0000 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:33.015 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:33.016 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:33.016 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:33.016 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:33.016 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:33.016 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:33.016 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 1 Processor -- start 2024-08-19T10:55:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 1 -- start start 2024-08-19T10:55:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 10 monclient: init 2024-08-19T10:55:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 10 monclient: _add_conns ranks=[0,4,1,7,9,3,2,6,10,5,8] 2024-08-19T10:55:33.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f19780a67d0 addr v1:172.21.15.89:6789/0 2024-08-19T10:55:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 0x7f19780a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 10 monclient(hunting): picked mon.noname-c con 0x7f19780a72c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f197813e9e0 addr v1:172.21.15.190:6789/0 2024-08-19T10:55:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19780a8250 con 0x7f19780a67d0 2024-08-19T10:55:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19780a4cc0 con 0x7f197813e9e0 2024-08-19T10:55:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f19780a8740 con 0x7f19780a72c0 2024-08-19T10:55:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f197ea30640 10 monclient(hunting): authenticate will time out at 1982.836946s 2024-08-19T10:55:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f1977fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f19780a67d0 0x7f19780a6bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49416/0 (socket says 172.21.15.89:49416) 2024-08-19T10:55:33.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 0x7f19780a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f1977fff640 1 -- 172.21.15.89:0/4281027735 learned_addr learned my addr 172.21.15.89:0/4281027735 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 10 monclient(hunting): get_auth_request con 0x7f19780a72c0 auth_method 0 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 10 monclient(hunting): handle_auth_done global_id 84316 payload 274 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 1 -- 172.21.15.89:0/4281027735 >> v1:172.21.15.190:6789/0 conn(0x7f197813e9e0 legacy=0x7f1978140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 1 -- 172.21.15.89:0/4281027735 >> v1:172.21.15.89:6789/0 conn(0x7f19780a67d0 legacy=0x7f19780a6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:33.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 1 monclient: found mon.noname-c 2024-08-19T10:55:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:33.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.018+0000 7f19777fe640 1 -- 172.21.15.89:0/4281027735 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19780a6140 con 0x7f19780a72c0 2024-08-19T10:55:33.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f19777fe640 1 -- 172.21.15.89:0/4281027735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 msgr2=0x7f19780a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f19777fe640 1 --2- 172.21.15.89:0/4281027735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 0x7f19780a5500 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f196c0097f0 tx=0x7f196c01c430 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8ea234cdbbeac9c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 1 -- 172.21.15.89:0/4281027735 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f196c0b0070 con 0x7f19780a72c0 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 10 monclient: dump: 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.025 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 10 monclient: _finish_auth 0 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 5 monclient: authenticate success, global_id 84316 2024-08-19T10:55:33.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 1 -- 172.21.15.89:0/4281027735 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c0a9a60 con 0x7f19780a72c0 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 4 set_mon_vals no callback set 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 1 -- 172.21.15.89:0/4281027735 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f196c0a9c10 con 0x7f19780a72c0 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f1976ffd640 10 monclient: dump: 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.027 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 10 monclient: shutdown 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:33.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 1 -- 172.21.15.89:0/4281027735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 msgr2=0x7f19780a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 1 --2- 172.21.15.89:0/4281027735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 0x7f19780a5500 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f196c0097f0 tx=0x7f196c01c430 comp rx=0 tx=0).stop 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 1 -- 172.21.15.89:0/4281027735 shutdown_connections 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 1 --2- 172.21.15.89:0/4281027735 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 0x7f19780a5500 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 1 -- 172.21.15.89:0/4281027735 >> 172.21.15.89:0/4281027735 conn(0x7f1978069ae0 msgr2=0x7f19780abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 1 -- 172.21.15.89:0/4281027735 shutdown_connections 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 1 -- 172.21.15.89:0/4281027735 wait complete. 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 10 monclient: build_initial_monmap 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 10 monclient: monmap: 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:33.029 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 1 Processor -- start 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 1 -- start start 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.022+0000 7f197ea30640 10 monclient: init 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient: _add_conns ranks=[3,4,6,1,7,5,0,2,8] 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19780a67d0 0x7f1978124840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient(hunting): picked mon.noname-g con 0x7f19780a67d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 0x7f19781d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient(hunting): picked mon.noname-c con 0x7f19780a72c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f197813e9e0 0x7f1978122420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient(hunting): picked mon.noname-d con 0x7f197813e9e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1978144f20 con 0x7f19780a67d0 2024-08-19T10:55:33.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f19781450a0 con 0x7f19780a72c0 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1978145220 con 0x7f197813e9e0 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient(hunting): authenticate will time out at 1982.844225s 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19777fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 0x7f19781d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f197813e9e0 0x7f1978122420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19777fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 0x7f19781d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:35080/0 (socket says 172.21.15.89:35080) 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 10 monclient(hunting): get_auth_request con 0x7f197813e9e0 auth_method 0 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19777fe640 1 -- 172.21.15.89:0/1306673693 learned_addr learned my addr 172.21.15.89:0/1306673693 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19777fe640 10 monclient(hunting): get_auth_request con 0x7f19780a72c0 auth_method 0 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19777fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19777fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1977fff640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19780a67d0 0x7f1978124840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 10 monclient(hunting): handle_auth_done global_id 45348 payload 824 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 1 -- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 msgr2=0x7f19781d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 0x7f19781d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 1 -- 172.21.15.89:0/1306673693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19780a67d0 msgr2=0x7f1978124840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19780a67d0 0x7f1978124840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 10 monclient: _finish_hunting 0 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 1 monclient: found mon.noname-d 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 1 -- 172.21.15.89:0/1306673693 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1978125d60 con 0x7f197813e9e0 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1977fff640 10 monclient: get_auth_request con 0x7f19780a67d0 auth_method 0 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1977fff640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19780a67d0 0x7f1978124840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:55:33.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19777fe640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f19780a72c0 0x7f19781d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:33.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 1 -- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f197813e9e0 msgr2=0x7f1978122420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197cfcd640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f197813e9e0 0x7f1978122420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f197000b3e0 tx=0x7f197000b8a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1970015020 con 0x7f197813e9e0 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 10 monclient: dump: 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.040 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.041 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 10 monclient: _finish_auth 0 2024-08-19T10:55:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:55:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1970011cb0 con 0x7f197813e9e0 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1970011e60 con 0x7f197813e9e0 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 5 monclient: authenticate success, global_id 45348 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient: _renew_subs 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 1 -- 172.21.15.89:0/1306673693 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19781420a0 con 0x7f197813e9e0 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f19767fc640 4 set_mon_vals no callback set 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f1974ff9640 10 monclient: dump: 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient: _renew_subs 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.026+0000 7f197ea30640 1 -- 172.21.15.89:0/1306673693 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f1978123ce0 con 0x7f197813e9e0 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f197ea30640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f197ea30640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f197813d7f0 0x7f1978122960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f197ea30640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f197ea30640 1 -- 172.21.15.89:0/1306673693 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f19780a3360 con 0x7f197813d7f0 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f197813d7f0 0x7f1978122960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 10 monclient: get_auth_request con 0x7f197813d7f0 auth_method 0 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1970018020 con 0x7f197813e9e0 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 10 monclient(hunting): handle_auth_done global_id 74477 payload 824 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 1 -- 172.21.15.89:0/1306673693 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19780a5500 con 0x7f197813d7f0 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f196403be00 0x7f196403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f197004f000 con 0x7f197813e9e0 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f19777fe640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f196403be00 0x7f196403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f19777fe640 10 monclient: get_auth_request con 0x7f196403be00 auth_method 0 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 1 -- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f197813d7f0 msgr2=0x7f1978122960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1977fff640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f197813d7f0 0x7f1978122960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f196800dbb0 tx=0x7f196800b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f196800be30 con 0x7f197813d7f0 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 10 monclient: dump: 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.046 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1968017070 con 0x7f197813d7f0 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f197813d7f0 msgr2=0x7f1978122960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f197813d7f0 0x7f1978122960 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f196800dbb0 tx=0x7f196800b4a0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f197813d7f0 msgr2=0x7f1978122960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.030+0000 7f1974ff9640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f197813d7f0 0x7f1978122960 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f196800dbb0 tx=0x7f196800b4a0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.034+0000 7f19777fe640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f196403be00 0x7f196403e2c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f196c0097f0 tx=0x7f196c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f197ea30640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f197ea30640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19781257a0 0x7f1978122ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f197ea30640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f197ea30640 1 -- 172.21.15.89:0/1306673693 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1978125f70 con 0x7f19781257a0 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f1977fff640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19781257a0 0x7f1978122ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f1977fff640 10 monclient: get_auth_request con 0x7f19781257a0 auth_method 0 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f1977fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f1977fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f1977fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f1977fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f1977fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f1977fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f1977fff640 10 monclient(hunting): handle_auth_done global_id 74486 payload 824 2024-08-19T10:55:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.038+0000 7f1977fff640 1 -- 172.21.15.89:0/1306673693 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1978064590 con 0x7f19781257a0 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1977fff640 1 -- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19781257a0 msgr2=0x7f1978122ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1977fff640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19781257a0 0x7f1978122ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f196800dbb0 tx=0x7f1968004cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1968012e40 con 0x7f19781257a0 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1974ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1974ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1974ff9640 10 monclient: dump: 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7f19680129d0 con 0x7f19781257a0 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1974ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1974ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19781257a0 msgr2=0x7f1978122ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1974ff9640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19781257a0 0x7f1978122ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f196800dbb0 tx=0x7f1968004cd0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1974ff9640 1 -- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19781257a0 msgr2=0x7f1978122ea0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:33.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f1974ff9640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f19781257a0 0x7f1978122ea0 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f196800dbb0 tx=0x7f1968004cd0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.051 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:33.051 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:33.051 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:33.051 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:33.051 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:33.051 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 269541, 2024-08-19T10:55:33.051 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:33.051 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:33.051 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:33.052 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:33.053 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:33.055 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:33.056 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.058 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.059 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:33.060 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.060 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.060 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:33.060 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:33.060 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.060 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.060 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.061 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.062 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.063 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.064 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.065 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:33.066 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.067 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.068 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:33.069 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:33.070 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.071 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.072 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.073 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.074 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:33.075 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:33.075 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:55:33.075 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.075 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.075 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.075 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:33.075 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:33.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197ea30640 1 -- 172.21.15.89:0/1306673693 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f196403be00 msgr2=0x7f196403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197ea30640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f196403be00 0x7f196403e2c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f196c0097f0 tx=0x7f196c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:55:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197ea30640 10 monclient: shutdown 2024-08-19T10:55:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197ea30640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197ea30640 1 -- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f197813e9e0 msgr2=0x7f1978122420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197cfcd640 1 -- 172.21.15.89:0/1306673693 reap_dead start 2024-08-19T10:55:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197ea30640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f197813e9e0 0x7f1978122420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f197000b3e0 tx=0x7f197000b8a0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197ea30640 1 -- 172.21.15.89:0/1306673693 shutdown_connections 2024-08-19T10:55:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197ea30640 1 --2- 172.21.15.89:0/1306673693 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f197813e9e0 0x7f1978122420 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197ea30640 1 -- 172.21.15.89:0/1306673693 >> 172.21.15.89:0/1306673693 conn(0x7f1978069ae0 msgr2=0x7f19780aab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197ea30640 1 -- 172.21.15.89:0/1306673693 shutdown_connections 2024-08-19T10:55:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.042+0000 7f197ea30640 1 -- 172.21.15.89:0/1306673693 wait complete. 2024-08-19T10:55:33.078 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:33.078 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:55:33.078 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:55:33.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient: get_monmap_and_config 2024-08-19T10:55:33.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient: build_initial_monmap 2024-08-19T10:55:33.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient: monmap: 2024-08-19T10:55:33.191 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:33.191 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:33.197210+0000 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:33.197210+0000 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:33.192 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:33.193 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:33.193 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:33.193 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 1 Processor -- start 2024-08-19T10:55:33.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 1 -- start start 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient: init 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient: _add_conns ranks=[10,9,0,3,2,4,8,6,1,7,5] 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f20880a6860 0x7f20880a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient(hunting): picked mon.noname-f con 0x7f20880a6860 addr v2:172.21.15.190:3300/0 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20880a7180 0x7f20880a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient(hunting): picked mon.noname-a con 0x7f20880a7180 addr v2:172.21.15.89:3300/0 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2088139620 addr v1:172.21.15.89:6789/0 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20880a33d0 con 0x7f2088139620 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f20880a8190 con 0x7f20880a7180 2024-08-19T10:55:33.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f20880a2ec0 con 0x7f20880a6860 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208ccdb640 10 monclient(hunting): authenticate will time out at 1983.013757s 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f20880a7180 0x7f20880a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2086d9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2088139620 0x7f2088139a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49422/0 (socket says 172.21.15.89:49422) 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2086d9d640 1 -- 172.21.15.89:0/1864936951 learned_addr learned my addr 172.21.15.89:0/1864936951 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 10 monclient(hunting): get_auth_request con 0x7f20880a7180 auth_method 0 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208659c640 1 --2- 172.21.15.89:0/1864936951 >> v2:172.21.15.190:3300/0 conn(0x7f20880a6860 0x7f20880a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208659c640 10 monclient(hunting): get_auth_request con 0x7f20880a6860 auth_method 0 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f208659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 10 monclient(hunting): handle_auth_done global_id 7614 payload 274 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 1 -- 172.21.15.89:0/1864936951 >> v2:172.21.15.190:3300/0 conn(0x7f20880a6860 msgr2=0x7f20880a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 1 --2- 172.21.15.89:0/1864936951 >> v2:172.21.15.190:3300/0 conn(0x7f20880a6860 0x7f20880a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 1 -- 172.21.15.89:0/1864936951 >> v1:172.21.15.89:6789/0 conn(0x7f2088139620 legacy=0x7f2088139a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 1 monclient: found mon.noname-a 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:55:33.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.194+0000 7f2085d9b640 1 -- 172.21.15.89:0/1864936951 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20880a4cd0 con 0x7f20880a7180 2024-08-19T10:55:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f2085d9b640 1 -- 172.21.15.89:0/1864936951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20880a7180 msgr2=0x7f20880a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f2085d9b640 1 --2- 172.21.15.89:0/1864936951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20880a7180 0x7f20880a7560 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f208806cff0 tx=0x7f207c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68518750dfd143aa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 1 -- 172.21.15.89:0/1864936951 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f207c0b0070 con 0x7f20880a7180 2024-08-19T10:55:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:55:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 10 monclient: dump: 2024-08-19T10:55:33.201 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.202 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 10 monclient: _finish_auth 0 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 10 monclient: _check_auth_tickets 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 1 -- 172.21.15.89:0/1864936951 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207c0aa810 con 0x7f20880a7180 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 5 monclient: authenticate success, global_id 7614 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 1 -- 172.21.15.89:0/1864936951 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f207c0a9300 con 0x7f20880a7180 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 4 set_mon_vals no callback set 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208559a640 10 monclient: dump: 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.204 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 10 monclient: shutdown 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 1 -- 172.21.15.89:0/1864936951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20880a7180 msgr2=0x7f20880a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 1 --2- 172.21.15.89:0/1864936951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20880a7180 0x7f20880a7560 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f208806cff0 tx=0x7f207c01c340 comp rx=0 tx=0).stop 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 1 -- 172.21.15.89:0/1864936951 shutdown_connections 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 1 --2- 172.21.15.89:0/1864936951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20880a7180 0x7f20880a7560 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 1 --2- 172.21.15.89:0/1864936951 >> v2:172.21.15.190:3300/0 conn(0x7f20880a6860 0x7f20880a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 1 -- 172.21.15.89:0/1864936951 >> 172.21.15.89:0/1864936951 conn(0x7f2088074b40 msgr2=0x7f20880abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 1 -- 172.21.15.89:0/1864936951 shutdown_connections 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.198+0000 7f208ccdb640 1 -- 172.21.15.89:0/1864936951 wait complete. 2024-08-19T10:55:33.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient: build_initial_monmap 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient: monmap: 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:33.206 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 1 Processor -- start 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 1 -- start start 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient: init 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient: _add_conns ranks=[2,4,3,7,5,0,6,1,8] 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20880a6860 0x7f20881d1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient(hunting): picked mon.noname-b con 0x7f20880a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20880a7180 0x7f20881d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient(hunting): picked mon.noname-c con 0x7f20880a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:33.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2088139620 0x7f208813d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient(hunting): picked mon.noname-g con 0x7f2088139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f2088144430 con 0x7f20880a6860 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f20881442b0 con 0x7f2088139620 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f20881445b0 con 0x7f20880a7180 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208ccdb640 10 monclient(hunting): authenticate will time out at 1983.020967s 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20880a6860 0x7f20881d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20880a7180 0x7f20881d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20880a6860 0x7f20881d1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60844/0 (socket says 172.21.15.89:60844) 2024-08-19T10:55:33.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208659c640 1 -- 172.21.15.89:0/1294445889 learned_addr learned my addr 172.21.15.89:0/1294445889 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 10 monclient(hunting): get_auth_request con 0x7f20880a7180 auth_method 0 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208659c640 10 monclient(hunting): get_auth_request con 0x7f20880a6860 auth_method 0 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2086d9d640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2088139620 0x7f208813d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 10 monclient(hunting): handle_auth_done global_id 84334 payload 824 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 1 -- 172.21.15.89:0/1294445889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2088139620 msgr2=0x7f208813d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2088139620 0x7f208813d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 1 -- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20880a6860 msgr2=0x7f20881d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20880a6860 0x7f20881d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 1 monclient: found mon.noname-c 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 1 -- 172.21.15.89:0/1294445889 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2088122830 con 0x7f20880a7180 2024-08-19T10:55:33.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2086d9d640 10 monclient: get_auth_request con 0x7f2088139620 auth_method 0 2024-08-19T10:55:33.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f208659c640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f20880a6860 0x7f20881d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:33.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2086d9d640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2088139620 0x7f208813d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:55:33.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 1 -- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20880a7180 msgr2=0x7f20881d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2085d9b640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20880a7180 0x7f20881d1da0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f207c0ab000 tx=0x7f207c0190e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f207c0b0040 con 0x7f20880a7180 2024-08-19T10:55:33.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 10 monclient: dump: 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.215 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 10 monclient: _finish_auth 0 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207c0044b0 con 0x7f20880a7180 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f207c003a90 con 0x7f20880a7180 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2076ffd640 10 monclient: dump: 2024-08-19T10:55:33.216 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.217 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.202+0000 7f2084d99640 4 set_mon_vals no callback set 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208ccdb640 5 monclient: authenticate success, global_id 84334 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208ccdb640 10 monclient: _renew_subs 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208ccdb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208ccdb640 1 -- 172.21.15.89:0/1294445889 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2088122b10 con 0x7f20880a7180 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208ccdb640 10 monclient: _renew_subs 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208ccdb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208ccdb640 1 -- 172.21.15.89:0/1294445889 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f20880ab1c0 con 0x7f20880a7180 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208ccdb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208ccdb640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813df10 0x7f208802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208ccdb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208ccdb640 1 -- 172.21.15.89:0/1294445889 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f20880a6c40 con 0x7f208813df10 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208659c640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813df10 0x7f208802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208659c640 10 monclient: get_auth_request con 0x7f208813df10 auth_method 0 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f207c003c50 con 0x7f20880a7180 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208659c640 10 monclient(hunting): handle_auth_done global_id 84343 payload 824 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f208659c640 1 -- 172.21.15.89:0/1294445889 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2088064590 con 0x7f208813df10 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f2076ffd640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f206803be30 0x7f206803e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f207c0e80e0 con 0x7f20880a7180 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f2086d9d640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f206803be30 0x7f206803e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.206+0000 7f2086d9d640 10 monclient: get_auth_request con 0x7f206803be30 auth_method 0 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f208659c640 1 -- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813df10 msgr2=0x7f208802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f208659c640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813df10 0x7f208802d1d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2078010cb0 tx=0x7f207800e5a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2078018070 con 0x7f208813df10 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2076ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2076ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2076ffd640 10 monclient: dump: 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.220 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2078014640 con 0x7f208813df10 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2076ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2076ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:33.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813df10 msgr2=0x7f208802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2076ffd640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813df10 0x7f208802d1d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2078010cb0 tx=0x7f207800e5a0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813df10 msgr2=0x7f208802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:33.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2076ffd640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813df10 0x7f208802d1d0 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2078010cb0 tx=0x7f207800e5a0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.210+0000 7f2086d9d640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f206803be30 0x7f206803e2f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f208813eec0 tx=0x7f2080008310 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208ccdb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:33.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208ccdb640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813e2f0 0x7f208802d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208ccdb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208ccdb640 1 -- 172.21.15.89:0/1294445889 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f20881402a0 con 0x7f208813e2f0 2024-08-19T10:55:33.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208659c640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813e2f0 0x7f208802d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208659c640 10 monclient: get_auth_request con 0x7f208813e2f0 auth_method 0 2024-08-19T10:55:33.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208659c640 10 monclient(hunting): handle_auth_done global_id 84352 payload 824 2024-08-19T10:55:33.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.214+0000 7f208659c640 1 -- 172.21.15.89:0/1294445889 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f20880a05d0 con 0x7f208813e2f0 2024-08-19T10:55:33.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208659c640 1 -- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813e2f0 msgr2=0x7f208802d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208659c640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813e2f0 0x7f208802d710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f207801a000 tx=0x7f20780046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2078004030 con 0x7f208813e2f0 2024-08-19T10:55:33.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2076ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2076ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:33.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2076ffd640 10 monclient: dump: 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.225 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.226 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.226 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.226 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7f2078004030 con 0x7f208813e2f0 2024-08-19T10:55:33.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2076ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:33.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2076ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:33.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813e2f0 msgr2=0x7f208802d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2076ffd640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813e2f0 0x7f208802d710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f207801a000 tx=0x7f20780046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2076ffd640 1 -- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813e2f0 msgr2=0x7f208802d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:33.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2076ffd640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f208813e2f0 0x7f208802d710 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f207801a000 tx=0x7f20780046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.227 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:33.227 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:33.227 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:33.227 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:33.227 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:33.227 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 141520, 2024-08-19T10:55:33.227 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:33.227 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:33.227 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:33.227 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:33.227 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:33.228 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:33.229 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:33.231 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:33.232 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:33.234 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.235 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.235 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.235 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:33.235 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:33.235 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.236 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:33.236 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.236 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:33.237 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:33.240 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:33.243 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:33.244 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.245 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:33.247 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.248 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.249 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:33.250 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.251 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:33.251 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:33.251 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:55:33.251 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.251 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.251 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:33.251 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:33.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208ccdb640 1 -- 172.21.15.89:0/1294445889 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f206803be30 msgr2=0x7f206803e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208ccdb640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f206803be30 0x7f206803e2f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f208813eec0 tx=0x7f2080008310 comp rx=0 tx=0).stop 2024-08-19T10:55:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208ccdb640 10 monclient: shutdown 2024-08-19T10:55:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208ccdb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208ccdb640 1 -- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20880a7180 msgr2=0x7f20881d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208ccdb640 1 --2- 172.21.15.89:0/1294445889 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f20880a7180 0x7f20881d1da0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f207c0ab000 tx=0x7f207c0190e0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2086d9d640 1 -- 172.21.15.89:0/1294445889 reap_dead start 2024-08-19T10:55:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f2086d9d640 1 -- 172.21.15.89:0/1294445889 reap_dead start 2024-08-19T10:55:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208ccdb640 1 -- 172.21.15.89:0/1294445889 shutdown_connections 2024-08-19T10:55:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208ccdb640 1 -- 172.21.15.89:0/1294445889 >> 172.21.15.89:0/1294445889 conn(0x7f2088074b40 msgr2=0x7f2088140fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208ccdb640 1 -- 172.21.15.89:0/1294445889 shutdown_connections 2024-08-19T10:55:33.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.218+0000 7f208ccdb640 1 -- 172.21.15.89:0/1294445889 wait complete. 2024-08-19T10:55:33.254 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:33.254 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:55:33.254 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:55:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient: get_monmap_and_config 2024-08-19T10:55:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient: build_initial_monmap 2024-08-19T10:55:33.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient: monmap: 2024-08-19T10:55:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:33.373864+0000 2024-08-19T10:55:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:33.373864+0000 2024-08-19T10:55:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.370 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:33.371 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:33.371 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 1 Processor -- start 2024-08-19T10:55:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 1 -- start start 2024-08-19T10:55:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient: init 2024-08-19T10:55:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient: _add_conns ranks=[10,5,9,4,3,8,6,1,0,2,7] 2024-08-19T10:55:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f34f8132dc0 0x7f34f81331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient(hunting): picked mon.noname-f con 0x7f34f8132dc0 addr v2:172.21.15.190:3300/0 2024-08-19T10:55:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34f806da40 0x7f34f8135f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient(hunting): picked mon.noname-h con 0x7f34f806da40 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f34f81336e0 0x7f34f8136460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient(hunting): picked mon.noname-a con 0x7f34f81336e0 addr v2:172.21.15.89:3300/0 2024-08-19T10:55:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f34f80a1640 con 0x7f34f81336e0 2024-08-19T10:55:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f34f80a1920 con 0x7f34f8132dc0 2024-08-19T10:55:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f34f80a1c00 con 0x7f34f806da40 2024-08-19T10:55:33.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34fdccd640 10 monclient(hunting): authenticate will time out at 1983.191174s 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f34f81336e0 0x7f34f8136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f34f81336e0 0x7f34f8136460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49236/0 (socket says 172.21.15.89:49236) 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 1 -- 172.21.15.89:0/144652225 learned_addr learned my addr 172.21.15.89:0/144652225 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 10 monclient(hunting): get_auth_request con 0x7f34f81336e0 auth_method 0 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 10 monclient(hunting): handle_auth_done global_id 7623 payload 274 2024-08-19T10:55:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 1 -- 172.21.15.89:0/144652225 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34f806da40 msgr2=0x7f34f8135f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 1 --2- 172.21.15.89:0/144652225 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34f806da40 0x7f34f8135f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 1 -- 172.21.15.89:0/144652225 >> v2:172.21.15.190:3300/0 conn(0x7f34f8132dc0 msgr2=0x7f34f81331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 1 --2- 172.21.15.89:0/144652225 >> v2:172.21.15.190:3300/0 conn(0x7f34f8132dc0 0x7f34f81331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:55:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 1 monclient: found mon.noname-a 2024-08-19T10:55:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:55:33.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.370+0000 7f34f7fff640 1 -- 172.21.15.89:0/144652225 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f8069650 con 0x7f34f81336e0 2024-08-19T10:55:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.374+0000 7f34f7fff640 1 -- 172.21.15.89:0/144652225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34f81336e0 msgr2=0x7f34f8136460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.374+0000 7f34f7fff640 1 --2- 172.21.15.89:0/144652225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34f81336e0 0x7f34f8136460 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009870 tx=0x7f34f001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7409fcd5a746157c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 1 -- 172.21.15.89:0/144652225 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34f00b0070 con 0x7f34f81336e0 2024-08-19T10:55:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:55:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 10 monclient: dump: 2024-08-19T10:55:33.383 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.384 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 10 monclient: _finish_auth 0 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 1 -- 172.21.15.89:0/144652225 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f00aa810 con 0x7f34f81336e0 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 5 monclient: authenticate success, global_id 7623 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:33.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 4 set_mon_vals no callback set 2024-08-19T10:55:33.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 1 -- 172.21.15.89:0/144652225 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34f00aae30 con 0x7f34f81336e0 2024-08-19T10:55:33.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:33.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f67fc640 10 monclient: dump: 2024-08-19T10:55:33.386 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.386 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.386 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.386 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.386 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.386 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.386 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient: shutdown 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:33.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 -- 172.21.15.89:0/144652225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34f81336e0 msgr2=0x7f34f8136460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 --2- 172.21.15.89:0/144652225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34f81336e0 0x7f34f8136460 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009870 tx=0x7f34f001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 -- 172.21.15.89:0/144652225 shutdown_connections 2024-08-19T10:55:33.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 --2- 172.21.15.89:0/144652225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34f81336e0 0x7f34f8136460 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 --2- 172.21.15.89:0/144652225 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f34f806da40 0x7f34f8135f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 --2- 172.21.15.89:0/144652225 >> v2:172.21.15.190:3300/0 conn(0x7f34f8132dc0 0x7f34f81331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 -- 172.21.15.89:0/144652225 >> 172.21.15.89:0/144652225 conn(0x7f34f80a83d0 msgr2=0x7f34f80a87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:33.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 -- 172.21.15.89:0/144652225 shutdown_connections 2024-08-19T10:55:33.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 -- 172.21.15.89:0/144652225 wait complete. 2024-08-19T10:55:33.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient: build_initial_monmap 2024-08-19T10:55:33.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient: monmap: 2024-08-19T10:55:33.388 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:33.389 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:33.390 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:33.390 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 Processor -- start 2024-08-19T10:55:33.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 -- start start 2024-08-19T10:55:33.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient: init 2024-08-19T10:55:33.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:33.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient: _add_conns ranks=[2,7,8,1,0,3,5,4,6] 2024-08-19T10:55:33.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34f806da40 0x7f34f80983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient(hunting): picked mon.noname-b con 0x7f34f806da40 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:33.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34f8132dc0 0x7f34f8098910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient(hunting): picked mon.noname-i con 0x7f34f8132dc0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34f81336e0 0x7f34f81cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient(hunting): picked mon.noname-e con 0x7f34f81336e0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f34f813fc80 con 0x7f34f806da40 2024-08-19T10:55:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f34f813fe00 con 0x7f34f8132dc0 2024-08-19T10:55:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f34f813ff80 con 0x7f34f81336e0 2024-08-19T10:55:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34fdccd640 10 monclient(hunting): authenticate will time out at 1983.198838s 2024-08-19T10:55:33.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34f806da40 0x7f34f80983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34f806da40 0x7f34f80983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60860/0 (socket says 172.21.15.89:60860) 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34f81336e0 0x7f34f81cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 1 -- 172.21.15.89:0/1071334344 learned_addr learned my addr 172.21.15.89:0/1071334344 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 10 monclient(hunting): get_auth_request con 0x7f34f806da40 auth_method 0 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f7fff640 10 monclient(hunting): get_auth_request con 0x7f34f81336e0 auth_method 0 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f7fff640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34f81336e0 msgr2=0x7f34f81cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-08-19T10:55:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f7fff640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34f81336e0 msgr2=0x7f34f81cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:55:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f7fff640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34f81336e0 0x7f34f81cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f7fff640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34f81336e0 0x7f34f81cf210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f6ffd640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34f8132dc0 0x7f34f8098910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 10 monclient(hunting): handle_auth_done global_id 74504 payload 824 2024-08-19T10:55:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34f81336e0 msgr2=0x7f34f81cf210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:55:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f34f81336e0 0x7f34f81cf210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34f8132dc0 msgr2=0x7f34f8098910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f34f8132dc0 0x7f34f8098910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 1 monclient: found mon.noname-b 2024-08-19T10:55:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:55:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.378+0000 7f34f77fe640 1 -- 172.21.15.89:0/1071334344 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f8098e50 con 0x7f34f806da40 2024-08-19T10:55:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f77fe640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34f806da40 msgr2=0x7f34f80983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f77fe640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34f806da40 0x7f34f80983d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f34e800ba50 tx=0x7f34e800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34e8019070 con 0x7f34f806da40 2024-08-19T10:55:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:55:33.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 10 monclient: dump: 2024-08-19T10:55:33.395 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.395 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.395 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.395 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.395 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.395 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.395 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.395 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.395 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.395 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 10 monclient: _finish_auth 0 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 10 monclient: _check_auth_tickets 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e800cbf0 con 0x7f34f806da40 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34fdccd640 5 monclient: authenticate success, global_id 74504 2024-08-19T10:55:33.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34fdccd640 10 monclient: _renew_subs 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34fdccd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34fdccd640 1 -- 172.21.15.89:0/1071334344 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f81302e0 con 0x7f34f806da40 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34e800cda0 con 0x7f34f806da40 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 10 monclient: dump: 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.397 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.398 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.398 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.398 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.398 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.398 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.398 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.398 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.398 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.398 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.398 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.398 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34fdccd640 10 monclient: _renew_subs 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34fdccd640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34fdccd640 1 -- 172.21.15.89:0/1071334344 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f80a4a40 con 0x7f34f806da40 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f5ffb640 4 set_mon_vals no callback set 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34fdccd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34fdccd640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f80a5480 0x7f34f802d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34fdccd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34fdccd640 1 -- 172.21.15.89:0/1071334344 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f34f8132c30 con 0x7f34f80a5480 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f6ffd640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f80a5480 0x7f34f802d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f6ffd640 10 monclient: get_auth_request con 0x7f34f80a5480 auth_method 0 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f34e8007370 con 0x7f34f806da40 2024-08-19T10:55:33.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f6ffd640 10 monclient(hunting): handle_auth_done global_id 45357 payload 824 2024-08-19T10:55:33.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f6ffd640 1 -- 172.21.15.89:0/1071334344 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34f81331a0 con 0x7f34f80a5480 2024-08-19T10:55:33.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f34e403be30 0x7f34e403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f34e804f280 con 0x7f34f806da40 2024-08-19T10:55:33.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f7fff640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f34e403be30 0x7f34e403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.382+0000 7f34f7fff640 10 monclient: get_auth_request con 0x7f34e403be30 auth_method 0 2024-08-19T10:55:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34f6ffd640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f80a5480 msgr2=0x7f34f802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34f6ffd640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f80a5480 0x7f34f802d1d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f34f8130020 tx=0x7f34ec00c950 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34ec015070 con 0x7f34f80a5480 2024-08-19T10:55:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34dffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34dffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34dffff640 10 monclient: dump: 2024-08-19T10:55:33.403 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.404 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.405 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f34ec011810 con 0x7f34f80a5480 2024-08-19T10:55:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34dffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34dffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f80a5480 msgr2=0x7f34f802d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34dffff640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f80a5480 0x7f34f802d1d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f34f8130020 tx=0x7f34ec00c950 comp rx=0 tx=0).stop 2024-08-19T10:55:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f80a5480 msgr2=0x7f34f802d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34dffff640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f80a5480 0x7f34f802d1d0 secure :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f34f8130020 tx=0x7f34ec00c950 comp rx=0 tx=0).stop 2024-08-19T10:55:33.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.386+0000 7f34f7fff640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f34e403be30 0x7f34e403e2f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f34f00ab000 tx=0x7f34f00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.390+0000 7f34fdccd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.390+0000 7f34fdccd640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f802d710 0x7f34f802db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.390+0000 7f34fdccd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.390+0000 7f34fdccd640 1 -- 172.21.15.89:0/1071334344 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f34f8064590 con 0x7f34f802d710 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.390+0000 7f34f6ffd640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f802d710 0x7f34f802db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.390+0000 7f34f6ffd640 10 monclient: get_auth_request con 0x7f34f802d710 auth_method 0 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34f6ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34f6ffd640 10 monclient(hunting): handle_auth_done global_id 45366 payload 824 2024-08-19T10:55:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34f6ffd640 1 -- 172.21.15.89:0/1071334344 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f34f80a04c0 con 0x7f34f802d710 2024-08-19T10:55:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34f6ffd640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f802d710 msgr2=0x7f34f802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34f6ffd640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f802d710 0x7f34f802db20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f34f802e890 tx=0x7f34ec0047e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34ec028020 con 0x7f34f802d710 2024-08-19T10:55:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34dffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34dffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34dffff640 10 monclient: dump: 2024-08-19T10:55:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.407 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.408 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.409 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f34ec023350 con 0x7f34f802d710 2024-08-19T10:55:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34dffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34dffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f802d710 msgr2=0x7f34f802db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34dffff640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f802d710 0x7f34f802db20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f34f802e890 tx=0x7f34ec0047e0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34dffff640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f802d710 msgr2=0x7f34f802db20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:33.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34dffff640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f34f802d710 0x7f34f802db20 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f34f802e890 tx=0x7f34ec0047e0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.410 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:33.410 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:33.410 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:33.410 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:33.410 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:33.410 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 656521, 2024-08-19T10:55:33.410 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:33.410 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:33.410 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:33.410 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:33.411 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:33.411 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:33.411 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:33.411 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:33.411 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:33.411 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:33.411 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.411 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 0, 2024-08-19T10:55:33.411 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:33.411 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:33.412 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:33.412 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:33.412 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:33.412 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:33.412 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:33.412 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:33.412 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:33.412 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:33.412 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:33.412 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:33.413 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:33.413 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:33.413 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.413 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:33.413 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:33.413 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:33.413 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:33.413 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:33.413 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:33.413 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:33.414 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:33.414 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:33.414 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:33.414 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:33.414 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:33.414 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:33.414 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.414 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:33.414 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:33.414 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:33.415 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:33.415 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:33.415 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:33.415 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:33.415 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:33.415 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:33.415 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:33.415 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:33.415 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:33.415 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:33.415 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:33.416 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:33.416 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:33.416 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:33.416 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:33.416 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:33.416 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:33.416 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:33.416 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:33.416 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:33.416 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:33.416 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:33.417 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:33.417 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:33.417 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:33.417 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.417 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:33.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.417 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:33.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.417 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:33.417 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:33.417 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:33.418 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:33.418 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.419 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.419 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:33.419 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:33.419 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.419 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.419 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.419 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:33.420 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:33.420 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.420 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.420 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.420 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:33.420 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.420 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.420 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:33.421 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.421 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.421 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.421 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:33.421 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:33.421 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.421 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.421 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.422 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:33.422 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:33.422 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.422 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.422 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.422 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:33.422 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.423 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.423 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:33.423 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.423 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.423 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.423 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:33.423 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:33.423 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.423 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.423 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.424 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:33.424 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:33.424 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.424 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.425 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:33.425 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.425 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.425 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:33.425 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.425 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.425 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.426 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:33.426 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:33.426 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.426 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.426 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.426 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:33.426 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:33.426 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.426 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.427 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.427 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:33.427 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.427 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.427 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:33.427 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.427 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.427 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.428 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.429 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.429 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:33.429 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.429 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.429 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:33.429 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.429 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.429 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.430 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:33.430 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:33.430 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.430 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.430 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.430 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:33.430 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:33.430 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.431 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.432 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:33.432 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:33.432 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.432 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.432 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.432 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:33.432 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:33.432 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.433 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.433 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.433 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:33.433 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.433 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.433 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:33.433 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.433 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:33.434 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:33.435 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:33.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.435 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:33.435 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:33.435 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.435 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:33.435 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:33.435 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:33.435 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:33.436 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:33.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.437 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:33.437 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:33.437 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:33.437 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.437 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.437 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:55:33.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.437 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:33.438 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:33.438 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:33.438 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.438 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:33.438 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:33.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:33.438 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:33.438 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:33.438 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:55:33.438 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:33.438 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:33.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:33.439 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:33.439 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:33.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34fdccd640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f34e403be30 msgr2=0x7f34e403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34fdccd640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f34e403be30 0x7f34e403e2f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f34f00ab000 tx=0x7f34f00a6000 comp rx=0 tx=0).stop 2024-08-19T10:55:33.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34fdccd640 10 monclient: shutdown 2024-08-19T10:55:33.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34fdccd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:33.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34fdccd640 1 -- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34f806da40 msgr2=0x7f34f80983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34fdccd640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34f806da40 0x7f34f80983d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f34e800ba50 tx=0x7f34e800bf10 comp rx=0 tx=0).stop 2024-08-19T10:55:33.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34f7fff640 1 -- 172.21.15.89:0/1071334344 reap_dead start 2024-08-19T10:55:33.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34fdccd640 1 -- 172.21.15.89:0/1071334344 shutdown_connections 2024-08-19T10:55:33.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34fdccd640 1 --2- 172.21.15.89:0/1071334344 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f34f806da40 0x7f34f80983d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34fdccd640 1 -- 172.21.15.89:0/1071334344 >> 172.21.15.89:0/1071334344 conn(0x7f34f80a83d0 msgr2=0x7f34f809b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:33.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34fdccd640 1 -- 172.21.15.89:0/1071334344 shutdown_connections 2024-08-19T10:55:33.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.394+0000 7f34fdccd640 1 -- 172.21.15.89:0/1071334344 wait complete. 2024-08-19T10:55:33.442 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:33.442 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:55:33.442 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:55:33.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient: get_monmap_and_config 2024-08-19T10:55:33.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient: build_initial_monmap 2024-08-19T10:55:33.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient: monmap: 2024-08-19T10:55:33.560 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:33.560 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.560 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:33.565764+0000 2024-08-19T10:55:33.560 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:33.565764+0000 2024-08-19T10:55:33.560 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:33.560 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.560 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:33.561 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:33.561 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:33.561 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:33.561 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:33.561 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:33.561 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:33.561 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:33.561 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:33.561 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:33.561 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:33.561 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 1 Processor -- start 2024-08-19T10:55:33.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 1 -- start start 2024-08-19T10:55:33.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient: init 2024-08-19T10:55:33.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:33.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient: _add_conns ranks=[10,6,2,3,1,9,5,4,7,0,8] 2024-08-19T10:55:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0c4006c930 0x7f0c4009d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient(hunting): picked mon.noname-f con 0x7f0c4006c930 addr v2:172.21.15.190:3300/0 2024-08-19T10:55:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006cd10 0x7f0c4006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient(hunting): picked mon.noname-d con 0x7f0c4006cd10 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c40137500 0x7f0c40137980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient(hunting): picked mon.noname-b con 0x7f0c40137500 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0c4009cb80 con 0x7f0c4006c930 2024-08-19T10:55:33.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f0c400a3d90 con 0x7f0c40137500 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f0c4009fe80 con 0x7f0c4006cd10 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c4727b640 10 monclient(hunting): authenticate will time out at 1983.382521s 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45818640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c40137500 0x7f0c40137980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006cd10 0x7f0c4006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006cd10 0x7f0c4006d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36454/0 (socket says 172.21.15.89:36454) 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45818640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c40137500 0x7f0c40137980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60866/0 (socket says 172.21.15.89:60866) 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 1 -- 172.21.15.89:0/826775377 learned_addr learned my addr 172.21.15.89:0/826775377 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 10 monclient(hunting): get_auth_request con 0x7f0c4006cd10 auth_method 0 2024-08-19T10:55:33.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45818640 10 monclient(hunting): get_auth_request con 0x7f0c40137500 auth_method 0 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45818640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45818640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45818640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45818640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45818640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45818640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45017640 1 --2- 172.21.15.89:0/826775377 >> v2:172.21.15.190:3300/0 conn(0x7f0c4006c930 0x7f0c4009d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45017640 10 monclient(hunting): get_auth_request con 0x7f0c4006c930 auth_method 0 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45017640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45017640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45017640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 10 monclient(hunting): handle_auth_done global_id 45375 payload 274 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 1 -- 172.21.15.89:0/826775377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c40137500 msgr2=0x7f0c40137980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 1 --2- 172.21.15.89:0/826775377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c40137500 0x7f0c40137980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 1 -- 172.21.15.89:0/826775377 >> v2:172.21.15.190:3300/0 conn(0x7f0c4006c930 msgr2=0x7f0c4009d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 1 --2- 172.21.15.89:0/826775377 >> v2:172.21.15.190:3300/0 conn(0x7f0c4006c930 0x7f0c4009d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 10 monclient: _finish_hunting 0 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 1 monclient: found mon.noname-d 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:55:33.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c44816640 1 -- 172.21.15.89:0/826775377 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c400a2df0 con 0x7f0c4006cd10 2024-08-19T10:55:33.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.562+0000 7f0c45818640 1 --2- 172.21.15.89:0/826775377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c40137500 0x7f0c40137980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c44816640 1 -- 172.21.15.89:0/826775377 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006cd10 msgr2=0x7f0c4006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c44816640 1 --2- 172.21.15.89:0/826775377 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006cd10 0x7f0c4006d0f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0c300098a0 tx=0x7f0c3001c400 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a10c3c9cd0904d1f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 1 -- 172.21.15.89:0/826775377 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c300b0070 con 0x7f0c4006cd10 2024-08-19T10:55:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:55:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 10 monclient: dump: 2024-08-19T10:55:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.574 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 10 monclient: _finish_auth 0 2024-08-19T10:55:33.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 10 monclient: _check_auth_tickets 2024-08-19T10:55:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 1 -- 172.21.15.89:0/826775377 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c300aa810 con 0x7f0c4006cd10 2024-08-19T10:55:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c4727b640 5 monclient: authenticate success, global_id 45375 2024-08-19T10:55:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c4727b640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c4727b640 4 set_mon_vals no callback set 2024-08-19T10:55:33.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 1 -- 172.21.15.89:0/826775377 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c300a9300 con 0x7f0c4006cd10 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.566+0000 7f0c37fff640 10 monclient: dump: 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.578 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient: shutdown 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 -- 172.21.15.89:0/826775377 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006cd10 msgr2=0x7f0c4006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 --2- 172.21.15.89:0/826775377 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006cd10 0x7f0c4006d0f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0c300098a0 tx=0x7f0c3001c400 comp rx=0 tx=0).stop 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 -- 172.21.15.89:0/826775377 shutdown_connections 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 --2- 172.21.15.89:0/826775377 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0c40137500 0x7f0c40137980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 --2- 172.21.15.89:0/826775377 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006cd10 0x7f0c4006d0f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 --2- 172.21.15.89:0/826775377 >> v2:172.21.15.190:3300/0 conn(0x7f0c4006c930 0x7f0c4009d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 -- 172.21.15.89:0/826775377 >> 172.21.15.89:0/826775377 conn(0x7f0c4006de40 msgr2=0x7f0c4006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 -- 172.21.15.89:0/826775377 shutdown_connections 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 -- 172.21.15.89:0/826775377 wait complete. 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient: build_initial_monmap 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient: monmap: 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:33.580 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 Processor -- start 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 -- start start 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient: init 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient: _add_conns ranks=[6,0,5,8,1,2,3,7,4] 2024-08-19T10:55:33.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006c930 0x7f0c401cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient(hunting): picked mon.noname-d con 0x7f0c4006c930 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c4006cd10 0x7f0c40134660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient(hunting): picked mon.noname-a con 0x7f0c4006cd10 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0c40137500 0x7f0c4013a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient(hunting): picked mon.noname-h con 0x7f0c40137500 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4006d630 con 0x7f0c4006cd10 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0c40140fe0 con 0x7f0c40137500 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f0c4006e220 con 0x7f0c4006c930 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:33.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c4727b640 10 monclient(hunting): authenticate will time out at 1983.389635s 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006c930 0x7f0c401cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c44816640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c4006cd10 0x7f0c40134660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006c930 0x7f0c401cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36468/0 (socket says 172.21.15.89:36468) 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 1 -- 172.21.15.89:0/2932173628 learned_addr learned my addr 172.21.15.89:0/2932173628 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 10 monclient(hunting): get_auth_request con 0x7f0c4006c930 auth_method 0 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c44816640 10 monclient(hunting): get_auth_request con 0x7f0c4006cd10 auth_method 0 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c44816640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c44816640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c44816640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 10 monclient(hunting): handle_auth_done global_id 45384 payload 824 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45818640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0c40137500 0x7f0c4013a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0c40137500 msgr2=0x7f0c4013a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0c40137500 0x7f0c4013a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c4006cd10 msgr2=0x7f0c40134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c4006cd10 0x7f0c40134660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 10 monclient: _finish_hunting 0 2024-08-19T10:55:33.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 1 monclient: found mon.noname-d 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45017640 1 -- 172.21.15.89:0/2932173628 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4011f760 con 0x7f0c4006c930 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45818640 10 monclient: get_auth_request con 0x7f0c40137500 auth_method 0 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c4006cd10 0x7f0c40134660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.570+0000 7f0c45818640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0c40137500 0x7f0c4013a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c45017640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006c930 msgr2=0x7f0c401cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c45017640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006c930 0x7f0c401cf0e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0c38007f20 tx=0x7f0c3800e570 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c38013a00 con 0x7f0c4006c930 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 10 monclient: dump: 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.585 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.586 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 10 monclient: _finish_auth 0 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c38019070 con 0x7f0c4006c930 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c4727b640 5 monclient: authenticate success, global_id 45384 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c4727b640 10 monclient: _renew_subs 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c4727b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c4727b640 1 -- 172.21.15.89:0/2932173628 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4013c1b0 con 0x7f0c4006c930 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c38013c00 con 0x7f0c4006c930 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:33.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 10 monclient: dump: 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:33.588 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c377fe640 4 set_mon_vals no callback set 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c4727b640 10 monclient: _renew_subs 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c4727b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c4727b640 1 -- 172.21.15.89:0/2932173628 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4013cd10 con 0x7f0c4006c930 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c4727b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c4727b640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c4727b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c4727b640 1 -- 172.21.15.89:0/2932173628 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f0c4006d7b0 con 0x7f0c40134ba0 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c44816640 10 monclient: get_auth_request con 0x7f0c40134ba0 auth_method 0 2024-08-19T10:55:33.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c44816640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c44816640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c44816640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c44816640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 msgr2=0x7f0c4013ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:55:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c44816640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 msgr2=0x7f0c4013ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:55:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0c38013590 con 0x7f0c4006c930 2024-08-19T10:55:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0c2403be00 0x7f0c2403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f0c3804fc90 con 0x7f0c4006c930 2024-08-19T10:55:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c45818640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0c2403be00 0x7f0c2403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.574+0000 7f0c45818640 10 monclient: get_auth_request con 0x7f0c2403be00 auth_method 0 2024-08-19T10:55:33.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.578+0000 7f0c45818640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0c2403be00 0x7f0c2403e2c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0c4011fb10 tx=0x7f0c3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:33.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.778+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.778+0000 7f0c44816640 10 monclient: get_auth_request con 0x7f0c40134ba0 auth_method 0 2024-08-19T10:55:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.778+0000 7f0c44816640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.778+0000 7f0c44816640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.778+0000 7f0c44816640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.778+0000 7f0c44816640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 msgr2=0x7f0c4013ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:55:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.778+0000 7f0c44816640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 msgr2=0x7f0c4013ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:55:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.778+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:33.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:33.778+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.182+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.182+0000 7f0c44816640 10 monclient: get_auth_request con 0x7f0c40134ba0 auth_method 0 2024-08-19T10:55:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.182+0000 7f0c44816640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.182+0000 7f0c44816640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:34.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.182+0000 7f0c44816640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.182+0000 7f0c44816640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 msgr2=0x7f0c4013ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:55:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.182+0000 7f0c44816640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 msgr2=0x7f0c4013ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:55:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.182+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:34.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.182+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:34.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:34.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 10 monclient: get_auth_request con 0x7f0c40134ba0 auth_method 0 2024-08-19T10:55:34.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:34.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:34.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:34.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:34.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:34.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:34.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 10 monclient(hunting): handle_auth_done global_id 114110 payload 824 2024-08-19T10:55:34.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 1 -- 172.21.15.89:0/2932173628 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c4006e700 con 0x7f0c40134ba0 2024-08-19T10:55:34.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 msgr2=0x7f0c4013ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:34.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0c300ad000 tx=0x7f0c300191a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:34.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c300ba070 con 0x7f0c40134ba0 2024-08-19T10:55:34.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:34.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c35ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:55:34.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c35ffb640 10 monclient: dump: 2024-08-19T10:55:34.985 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:34.985 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:34.985 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:34.986 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:34.986 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:34.986 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:34.986 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:34.986 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:34.986 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:34.986 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:34.986 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:34.986 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:34.986 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:34.986 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:34.987 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:34.987 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:34.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0c300aa960 con 0x7f0c40134ba0 2024-08-19T10:55:34.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c35ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:34.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c35ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:34.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 msgr2=0x7f0c4013ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:34.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c35ffb640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0c300ad000 tx=0x7f0c300191a0 comp rx=0 tx=0).stop 2024-08-19T10:55:34.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 msgr2=0x7f0c4013ac40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:34.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.986+0000 7f0c35ffb640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c40134ba0 0x7f0c4013ac40 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0c300ad000 tx=0x7f0c300191a0 comp rx=0 tx=0).stop 2024-08-19T10:55:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c4727b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c4727b640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c4013b180 0x7f0c401356f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c4727b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c4727b640 1 -- 172.21.15.89:0/2932173628 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f0c40064590 con 0x7f0c4013b180 2024-08-19T10:55:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c4013b180 0x7f0c401356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c44816640 10 monclient: get_auth_request con 0x7f0c4013b180 auth_method 0 2024-08-19T10:55:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c44816640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c44816640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c44816640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:34.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:34.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:34.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c44816640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:34.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c44816640 10 monclient(hunting): handle_auth_done global_id 114119 payload 824 2024-08-19T10:55:34.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.994+0000 7f0c44816640 1 -- 172.21.15.89:0/2932173628 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0c4013b590 con 0x7f0c4013b180 2024-08-19T10:55:34.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c44816640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c4013b180 msgr2=0x7f0c401356f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:34.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c44816640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c4013b180 0x7f0c401356f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c300097c0 tx=0x7f0c30009240 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:34.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0c300b0040 con 0x7f0c4013b180 2024-08-19T10:55:34.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c35ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:34.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c35ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:55:34.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c35ffb640 10 monclient: dump: 2024-08-19T10:55:34.997 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:34.997 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:34.998 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:34.999 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:34.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f0c30003e10 con 0x7f0c4013b180 2024-08-19T10:55:34.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c35ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:34.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c35ffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:34.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c4013b180 msgr2=0x7f0c401356f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:34.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c35ffb640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c4013b180 0x7f0c401356f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c300097c0 tx=0x7f0c30009240 comp rx=0 tx=0).stop 2024-08-19T10:55:34.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c35ffb640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c4013b180 msgr2=0x7f0c401356f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:34.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c35ffb640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f0c4013b180 0x7f0c401356f0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c300097c0 tx=0x7f0c30009240 comp rx=0 tx=0).stop 2024-08-19T10:55:34.999 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:34.999 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:34.999 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 15647, 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:35.000 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.001 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.002 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:35.003 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.004 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:35.005 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:35.006 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:35.007 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.008 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.009 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.009 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:35.009 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.009 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.009 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.010 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.011 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:35.013 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:35.016 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:35.017 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.018 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.019 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.020 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.022 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:35.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.024 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:35.024 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:35.024 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:35.024 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.024 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.024 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:35.024 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:35.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c4727b640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0c2403be00 msgr2=0x7f0c2403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c4727b640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0c2403be00 0x7f0c2403e2c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0c4011fb10 tx=0x7f0c3c008040 comp rx=0 tx=0).stop 2024-08-19T10:55:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c4727b640 10 monclient: shutdown 2024-08-19T10:55:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c4727b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c4727b640 1 -- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006c930 msgr2=0x7f0c401cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c4727b640 1 --2- 172.21.15.89:0/2932173628 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f0c4006c930 0x7f0c401cf0e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0c38007f20 tx=0x7f0c3800e570 comp rx=0 tx=0).stop 2024-08-19T10:55:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c45818640 1 -- 172.21.15.89:0/2932173628 reap_dead start 2024-08-19T10:55:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c45818640 1 -- 172.21.15.89:0/2932173628 reap_dead start 2024-08-19T10:55:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c4727b640 1 -- 172.21.15.89:0/2932173628 shutdown_connections 2024-08-19T10:55:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c4727b640 1 -- 172.21.15.89:0/2932173628 >> 172.21.15.89:0/2932173628 conn(0x7f0c4006de40 msgr2=0x7f0c400a7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c4727b640 1 -- 172.21.15.89:0/2932173628 shutdown_connections 2024-08-19T10:55:35.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:34.998+0000 7f0c4727b640 1 -- 172.21.15.89:0/2932173628 wait complete. 2024-08-19T10:55:35.027 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:35.027 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:55:35.027 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:55:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient: get_monmap_and_config 2024-08-19T10:55:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient: build_initial_monmap 2024-08-19T10:55:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient: monmap: 2024-08-19T10:55:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:35.161062+0000 2024-08-19T10:55:35.155 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:35.161062+0000 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:35.156 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 1 Processor -- start 2024-08-19T10:55:35.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 1 -- start start 2024-08-19T10:55:35.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient: init 2024-08-19T10:55:35.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient: _add_conns ranks=[2,0,1,4,5,8,10,3,7,6,9] 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8190139940 0x7f8190139d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient(hunting): picked mon.noname-b con 0x7f8190139940 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f819006e1b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f81900a5490 addr v1:172.21.15.190:6789/0 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f819006f8b0 con 0x7f819006e1b0 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81900a66f0 con 0x7f81900a5490 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f819006df70 con 0x7f8190139940 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f8195bcb640 10 monclient(hunting): authenticate will time out at 1984.977614s 2024-08-19T10:55:35.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818effd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f819006e1b0 0x7f81900a4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49424/0 (socket says 172.21.15.89:49424) 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8190139940 0x7f8190139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818effd640 1 -- 172.21.15.89:0/3448472333 learned_addr learned my addr 172.21.15.89:0/3448472333 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 10 monclient(hunting): get_auth_request con 0x7f8190139940 auth_method 0 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 10 monclient(hunting): handle_auth_done global_id 74522 payload 274 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 1 -- 172.21.15.89:0/3448472333 >> v1:172.21.15.190:6789/0 conn(0x7f81900a5490 legacy=0x7f81900a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:35.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 1 -- 172.21.15.89:0/3448472333 >> v1:172.21.15.89:6789/0 conn(0x7f819006e1b0 legacy=0x7f81900a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:35.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:35.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 1 monclient: found mon.noname-b 2024-08-19T10:55:35.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:35.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:55:35.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.158+0000 7f818f7fe640 1 -- 172.21.15.89:0/3448472333 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819006f010 con 0x7f8190139940 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818f7fe640 1 -- 172.21.15.89:0/3448472333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8190139940 msgr2=0x7f8190139d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818f7fe640 1 --2- 172.21.15.89:0/3448472333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8190139940 0x7f8190139d20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f81880097f0 tx=0x7f818801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=508f6abe2d156e27 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 1 -- 172.21.15.89:0/3448472333 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81880b0070 con 0x7f8190139940 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 10 monclient: dump: 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.163 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:55:35.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 5 monclient: authenticate success, global_id 74522 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 1 -- 172.21.15.89:0/3448472333 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81880aa810 con 0x7f8190139940 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 1 -- 172.21.15.89:0/3448472333 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81880aae30 con 0x7f8190139940 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 4 set_mon_vals no callback set 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f818e7fc640 10 monclient: dump: 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.165 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.166 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 10 monclient: shutdown 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 1 -- 172.21.15.89:0/3448472333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8190139940 msgr2=0x7f8190139d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 1 --2- 172.21.15.89:0/3448472333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8190139940 0x7f8190139d20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f81880097f0 tx=0x7f818801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 1 -- 172.21.15.89:0/3448472333 shutdown_connections 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 1 --2- 172.21.15.89:0/3448472333 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f8190139940 0x7f8190139d20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 1 -- 172.21.15.89:0/3448472333 >> 172.21.15.89:0/3448472333 conn(0x7f81900aa240 msgr2=0x7f81900aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 1 -- 172.21.15.89:0/3448472333 shutdown_connections 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 1 -- 172.21.15.89:0/3448472333 wait complete. 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 10 monclient: build_initial_monmap 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.162+0000 7f8195bcb640 10 monclient: monmap: 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:35.167 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 1 Processor -- start 2024-08-19T10:55:35.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 1 -- start start 2024-08-19T10:55:35.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient: init 2024-08-19T10:55:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient: _add_conns ranks=[3,6,8,2,5,1,4,7,0] 2024-08-19T10:55:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f819006e1b0 0x7f81901d1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient(hunting): picked mon.noname-g con 0x7f819006e1b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81900a5490 0x7f81901d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient(hunting): picked mon.noname-d con 0x7f81900a5490 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8190139940 0x7f8190140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient(hunting): picked mon.noname-e con 0x7f8190139940 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f81901454c0 con 0x7f819006e1b0 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f819013a470 con 0x7f81900a5490 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f8190145640 con 0x7f8190139940 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient(hunting): authenticate will time out at 1984.984728s 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8190139940 0x7f8190140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818effd640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81900a5490 0x7f81901d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8190139940 0x7f8190140320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:59846/0 (socket says 172.21.15.89:59846) 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818effd640 10 monclient(hunting): get_auth_request con 0x7f81900a5490 auth_method 0 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 1 -- 172.21.15.89:0/1982219929 learned_addr learned my addr 172.21.15.89:0/1982219929 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 10 monclient(hunting): get_auth_request con 0x7f8190139940 auth_method 0 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818f7fe640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f819006e1b0 0x7f81901d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818f7fe640 10 monclient(hunting): get_auth_request con 0x7f819006e1b0 auth_method 0 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 10 monclient(hunting): handle_auth_done global_id 114128 payload 824 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81900a5490 msgr2=0x7f81901d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81900a5490 0x7f81901d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f819006e1b0 msgr2=0x7f81901d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f819006e1b0 0x7f81901d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:55:35.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 1 monclient: found mon.noname-e 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 1 -- 172.21.15.89:0/1982219929 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81900a8120 con 0x7f8190139940 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818effd640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f81900a5490 0x7f81901d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8190139940 msgr2=0x7f8190140320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818ffff640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8190139940 0x7f8190140320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f818000c410 tx=0x7f818000c8d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8180017070 con 0x7f8190139940 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 10 monclient: dump: 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.177 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.178 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 10 monclient: _finish_auth 0 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 10 monclient: _check_auth_tickets 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8180003a70 con 0x7f8190139940 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 5 monclient: authenticate success, global_id 114128 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient: _renew_subs 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 1 -- 172.21.15.89:0/1982219929 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8190141cd0 con 0x7f8190139940 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8180003c60 con 0x7f8190139940 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:55:35.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8187fff640 10 monclient: dump: 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.180 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f818dffb640 4 set_mon_vals no callback set 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient: _renew_subs 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.166+0000 7f8195bcb640 1 -- 172.21.15.89:0/1982219929 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f81901213b0 con 0x7f8190139940 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f8195bcb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f8195bcb640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 0x7f8190140c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f8195bcb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f8195bcb640 1 -- 172.21.15.89:0/1982219929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f81900aa8a0 con 0x7f8190140860 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8180003600 con 0x7f8190139940 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f818f7fe640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 0x7f8190140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f818f7fe640 10 monclient: get_auth_request con 0x7f8190140860 auth_method 0 2024-08-19T10:55:35.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f818f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f818f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f818f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f818f7fe640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 msgr2=0x7f8190140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-08-19T10:55:35.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f818f7fe640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 msgr2=0x7f8190140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:55:35.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f818f7fe640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 0x7f8190140c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:35.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f8187fff640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f817403be00 0x7f817403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f818f7fe640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 0x7f8190140c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:35.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f818004e870 con 0x7f8190139940 2024-08-19T10:55:35.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f818effd640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f817403be00 0x7f817403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.170+0000 7f818effd640 10 monclient: get_auth_request con 0x7f817403be00 auth_method 0 2024-08-19T10:55:35.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.174+0000 7f818effd640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f817403be00 0x7f817403e2c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8190120970 tx=0x7f8178009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.374+0000 7f818f7fe640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 0x7f8190140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.374+0000 7f818f7fe640 10 monclient: get_auth_request con 0x7f8190140860 auth_method 0 2024-08-19T10:55:35.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.374+0000 7f818f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.374+0000 7f818f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.374+0000 7f818f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:35.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.374+0000 7f818f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.374+0000 7f818f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.374+0000 7f818f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.374+0000 7f818f7fe640 10 monclient(hunting): handle_auth_done global_id 45289 payload 824 2024-08-19T10:55:35.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.374+0000 7f818f7fe640 1 -- 172.21.15.89:0/1982219929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8190139d20 con 0x7f8190140860 2024-08-19T10:55:35.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f818f7fe640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 msgr2=0x7f8190140c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f818f7fe640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 0x7f8190140c40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f81901398b0 tx=0x7f8188019140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81880b0040 con 0x7f8190140860 2024-08-19T10:55:35.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f8187fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f8187fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:55:35.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f8187fff640 10 monclient: dump: 2024-08-19T10:55:35.378 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.378 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.378 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.378 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.379 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.380 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.381 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.381 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f81880039d0 con 0x7f8190140860 2024-08-19T10:55:35.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f8187fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:35.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f8187fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:35.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 msgr2=0x7f8190140c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f8187fff640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 0x7f8190140c40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f81901398b0 tx=0x7f8188019140 comp rx=0 tx=0).stop 2024-08-19T10:55:35.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 msgr2=0x7f8190140c40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:35.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.378+0000 7f8187fff640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190140860 0x7f8190140c40 secure :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f81901398b0 tx=0x7f8188019140 comp rx=0 tx=0).stop 2024-08-19T10:55:35.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f8195bcb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:35.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f8195bcb640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190141180 0x7f819013c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f8195bcb640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f8195bcb640 1 -- 172.21.15.89:0/1982219929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8190064590 con 0x7f8190141180 2024-08-19T10:55:35.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f818f7fe640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190141180 0x7f819013c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f818f7fe640 10 monclient: get_auth_request con 0x7f8190141180 auth_method 0 2024-08-19T10:55:35.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f818f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f818f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f818f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f818f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f818f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f818f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f818f7fe640 10 monclient(hunting): handle_auth_done global_id 45298 payload 824 2024-08-19T10:55:35.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.386+0000 7f818f7fe640 1 -- 172.21.15.89:0/1982219929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8190121770 con 0x7f8190141180 2024-08-19T10:55:35.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.390+0000 7f818f7fe640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190141180 msgr2=0x7f819013c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.390+0000 7f818f7fe640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190141180 0x7f819013c600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8188000c10 tx=0x7f8188009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.390+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f81880b0050 con 0x7f8190141180 2024-08-19T10:55:35.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.390+0000 7f8187fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.390+0000 7f8187fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:55:35.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.390+0000 7f8187fff640 10 monclient: dump: 2024-08-19T10:55:35.393 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.393 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.393 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.394 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.394 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.394 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.394 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.394 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.394 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.395 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.395 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.395 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.395 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.395 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.396 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.390+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f81880b5d10 con 0x7f8190141180 2024-08-19T10:55:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.390+0000 7f8187fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:35.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.390+0000 7f8187fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.390+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190141180 msgr2=0x7f819013c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8187fff640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190141180 0x7f819013c600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8188000c10 tx=0x7f8188009190 comp rx=0 tx=0).stop 2024-08-19T10:55:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8187fff640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190141180 msgr2=0x7f819013c600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:35.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8187fff640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8190141180 0x7f819013c600 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8188000c10 tx=0x7f8188009190 comp rx=0 tx=0).stop 2024-08-19T10:55:35.398 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:35.398 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 658520, 2024-08-19T10:55:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:35.399 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:35.400 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:35.401 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.402 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:35.403 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.404 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.405 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.409 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.411 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:35.412 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.413 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.415 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:35.416 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.417 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:35.420 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.423 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.424 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.426 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.427 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:35.428 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:35.429 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.430 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:35.431 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:35.432 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:35.433 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:35.434 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:35.435 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:35.436 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.437 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:35.438 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:35.439 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:35.441 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:55:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.442 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 4 2024-08-19T10:55:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:35.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.445 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:35.445 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8195bcb640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f817403be00 msgr2=0x7f817403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8195bcb640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f817403be00 0x7f817403e2c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8190120970 tx=0x7f8178009040 comp rx=0 tx=0).stop 2024-08-19T10:55:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8195bcb640 10 monclient: shutdown 2024-08-19T10:55:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8195bcb640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8195bcb640 1 -- 172.21.15.89:0/1982219929 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8190139940 msgr2=0x7f8190140320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8195bcb640 1 --2- 172.21.15.89:0/1982219929 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f8190139940 0x7f8190140320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f818000c410 tx=0x7f818000c8d0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f818ffff640 1 -- 172.21.15.89:0/1982219929 reap_dead start 2024-08-19T10:55:35.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f818ffff640 1 -- 172.21.15.89:0/1982219929 reap_dead start 2024-08-19T10:55:35.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8195bcb640 1 -- 172.21.15.89:0/1982219929 shutdown_connections 2024-08-19T10:55:35.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8195bcb640 1 -- 172.21.15.89:0/1982219929 >> 172.21.15.89:0/1982219929 conn(0x7f81900aa240 msgr2=0x7f81900a12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:35.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8195bcb640 1 -- 172.21.15.89:0/1982219929 shutdown_connections 2024-08-19T10:55:35.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.394+0000 7f8195bcb640 1 -- 172.21.15.89:0/1982219929 wait complete. 2024-08-19T10:55:35.450 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:35.450 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:55:35.450 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:55:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient: get_monmap_and_config 2024-08-19T10:55:35.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient: build_initial_monmap 2024-08-19T10:55:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient: monmap: 2024-08-19T10:55:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:35.581174+0000 2024-08-19T10:55:35.576 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:35.581174+0000 2024-08-19T10:55:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:35.577 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:35.578 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:35.578 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 1 Processor -- start 2024-08-19T10:55:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 1 -- start start 2024-08-19T10:55:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient: init 2024-08-19T10:55:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient: _add_conns ranks=[3,6,2,1,9,7,0,8,10,5,4] 2024-08-19T10:55:35.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c09ea20 0x7f342c09d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient(hunting): picked mon.noname-g con 0x7f342c09ea20 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f342c09ee00 0x7f342c09f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient(hunting): picked mon.noname-d con 0x7f342c09ee00 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f342c06c930 0x7f342c06cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient(hunting): picked mon.noname-b con 0x7f342c06c930 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f342c0a3dc0 con 0x7f342c06c930 2024-08-19T10:55:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f342c0a51d0 con 0x7f342c09ea20 2024-08-19T10:55:35.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f342c09fbc0 con 0x7f342c09ee00 2024-08-19T10:55:35.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:35.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3434525640 10 monclient(hunting): authenticate will time out at 1985.397866s 2024-08-19T10:55:35.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3431ac0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f342c09ee00 0x7f342c09f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3431ac0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f342c09ee00 0x7f342c09f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36474/0 (socket says 172.21.15.89:36474) 2024-08-19T10:55:35.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f342c06c930 0x7f342c06cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3431ac0640 1 -- 172.21.15.89:0/3749145047 learned_addr learned my addr 172.21.15.89:0/3749145047 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:35.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3431ac0640 10 monclient(hunting): get_auth_request con 0x7f342c09ee00 auth_method 0 2024-08-19T10:55:35.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3431ac0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3431ac0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3431ac0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 10 monclient(hunting): get_auth_request con 0x7f342c06c930 auth_method 0 2024-08-19T10:55:35.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f34322c1640 1 --2- 172.21.15.89:0/3749145047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c09ea20 0x7f342c09d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f34322c1640 10 monclient(hunting): get_auth_request con 0x7f342c09ea20 auth_method 0 2024-08-19T10:55:35.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f34322c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f34322c1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f34322c1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 10 monclient(hunting): handle_auth_done global_id 74531 payload 274 2024-08-19T10:55:35.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 1 -- 172.21.15.89:0/3749145047 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f342c09ee00 msgr2=0x7f342c09f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:35.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 1 --2- 172.21.15.89:0/3749145047 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f342c09ee00 0x7f342c09f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 1 -- 172.21.15.89:0/3749145047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c09ea20 msgr2=0x7f342c09d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:35.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 1 --2- 172.21.15.89:0/3749145047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c09ea20 0x7f342c09d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 10 monclient: _finish_hunting 0 2024-08-19T10:55:35.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 1 monclient: found mon.noname-b 2024-08-19T10:55:35.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:35.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:55:35.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.578+0000 7f3432ac2640 1 -- 172.21.15.89:0/3749145047 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c0a3b40 con 0x7f342c06c930 2024-08-19T10:55:35.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3432ac2640 1 -- 172.21.15.89:0/3749145047 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f342c06c930 msgr2=0x7f342c06cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3432ac2640 1 --2- 172.21.15.89:0/3749145047 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f342c06c930 0x7f342c06cd70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3424009df0 tx=0x7f342401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aebf6f6f795f052b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 1 -- 172.21.15.89:0/3749145047 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34240a6940 con 0x7f342c06c930 2024-08-19T10:55:35.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:55:35.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 10 monclient: dump: 2024-08-19T10:55:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.595 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.596 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 10 monclient: _finish_auth 0 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 10 monclient: _check_auth_tickets 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 1 -- 172.21.15.89:0/3749145047 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34240a6b00 con 0x7f342c06c930 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 5 monclient: authenticate success, global_id 74531 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 1 -- 172.21.15.89:0/3749145047 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34240a6cb0 con 0x7f342c06c930 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 4 set_mon_vals no callback set 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f34312bf640 10 monclient: dump: 2024-08-19T10:55:35.597 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.598 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.599 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 10 monclient: shutdown 2024-08-19T10:55:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 1 -- 172.21.15.89:0/3749145047 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f342c06c930 msgr2=0x7f342c06cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 1 --2- 172.21.15.89:0/3749145047 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f342c06c930 0x7f342c06cd70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3424009df0 tx=0x7f342401c8e0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 1 -- 172.21.15.89:0/3749145047 shutdown_connections 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 1 --2- 172.21.15.89:0/3749145047 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f342c09ee00 0x7f342c09f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 1 --2- 172.21.15.89:0/3749145047 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f342c06c930 0x7f342c06cd70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 1 --2- 172.21.15.89:0/3749145047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c09ea20 0x7f342c09d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 1 -- 172.21.15.89:0/3749145047 >> 172.21.15.89:0/3749145047 conn(0x7f342c0a8f60 msgr2=0x7f342c0a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 1 -- 172.21.15.89:0/3749145047 shutdown_connections 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.582+0000 7f3434525640 1 -- 172.21.15.89:0/3749145047 wait complete. 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient: build_initial_monmap 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient: monmap: 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:35.600 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:35.601 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:35.602 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 1 Processor -- start 2024-08-19T10:55:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 1 -- start start 2024-08-19T10:55:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient: init 2024-08-19T10:55:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient: _add_conns ranks=[0,8,5,3,4,2,1,7,6] 2024-08-19T10:55:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f342c06c930 0x7f342c123850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient(hunting): picked mon.noname-a con 0x7f342c06c930 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:55:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f342c09ea20 0x7f342c121740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient(hunting): picked mon.noname-e con 0x7f342c09ea20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f342c09ee00 0x7f342c1396b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient(hunting): picked mon.noname-h con 0x7f342c09ee00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f342c140ee0 con 0x7f342c06c930 2024-08-19T10:55:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f342c140d60 con 0x7f342c09ee00 2024-08-19T10:55:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f342c141060 con 0x7f342c09ea20 2024-08-19T10:55:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3434525640 10 monclient(hunting): authenticate will time out at 1985.405152s 2024-08-19T10:55:35.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f342c06c930 0x7f342c123850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3431ac0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f342c09ea20 0x7f342c121740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f342c06c930 0x7f342c123850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49258/0 (socket says 172.21.15.89:49258) 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 1 -- 172.21.15.89:0/272911090 learned_addr learned my addr 172.21.15.89:0/272911090 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 10 monclient(hunting): get_auth_request con 0x7f342c06c930 auth_method 0 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3431ac0640 10 monclient(hunting): get_auth_request con 0x7f342c09ea20 auth_method 0 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3431ac0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3431ac0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3431ac0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 10 monclient(hunting): handle_auth_done global_id 7641 payload 824 2024-08-19T10:55:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 1 -- 172.21.15.89:0/272911090 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f342c09ea20 msgr2=0x7f342c121740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f342c09ea20 0x7f342c121740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f3432ac2640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f342c09ee00 0x7f342c1396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 1 -- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f342c09ee00 msgr2=0x7f342c1396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f342c09ee00 0x7f342c1396b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 10 monclient: _finish_hunting 0 2024-08-19T10:55:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 1 monclient: found mon.noname-a 2024-08-19T10:55:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:55:35.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 1 -- 172.21.15.89:0/272911090 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c0a8a80 con 0x7f342c06c930 2024-08-19T10:55:35.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 1 -- 172.21.15.89:0/272911090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f342c06c930 msgr2=0x7f342c123850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.586+0000 7f34322c1640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f342c06c930 0x7f342c123850 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d800 tx=0x7f341c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f341c00bd40 con 0x7f342c06c930 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 10 monclient: dump: 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.612 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.613 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 10 monclient: _finish_auth 0 2024-08-19T10:55:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:55:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341c019070 con 0x7f342c06c930 2024-08-19T10:55:35.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3434525640 5 monclient: authenticate success, global_id 7641 2024-08-19T10:55:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3434525640 10 monclient: _renew_subs 2024-08-19T10:55:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3434525640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:55:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3434525640 1 -- 172.21.15.89:0/272911090 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c125050 con 0x7f342c06c930 2024-08-19T10:55:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f341c0148a0 con 0x7f342c06c930 2024-08-19T10:55:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:35.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3430abe640 4 set_mon_vals no callback set 2024-08-19T10:55:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 10 monclient: dump: 2024-08-19T10:55:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.615 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.616 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.616 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3434525640 10 monclient: _renew_subs 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3434525640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3434525640 1 -- 172.21.15.89:0/272911090 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1247b0 con 0x7f342c06c930 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3434525640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3434525640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c139bf0 0x7f342c1cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3434525640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3434525640 1 -- 172.21.15.89:0/272911090 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f342c0a9660 con 0x7f342c139bf0 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f341c014430 con 0x7f342c06c930 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3431ac0640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c139bf0 0x7f342c1cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f341003be00 0x7f341003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3431ac0640 10 monclient: get_auth_request con 0x7f342c139bf0 auth_method 0 2024-08-19T10:55:35.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3431ac0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3431ac0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3431ac0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f341c00f040 con 0x7f342c06c930 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3432ac2640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f341003be00 0x7f341003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3432ac2640 10 monclient: get_auth_request con 0x7f341003be00 auth_method 0 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3431ac0640 10 monclient(hunting): handle_auth_done global_id 114123 payload 824 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.590+0000 7f3431ac0640 1 -- 172.21.15.89:0/272911090 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f342c124b00 con 0x7f342c139bf0 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3431ac0640 1 -- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c139bf0 msgr2=0x7f342c1cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3431ac0640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c139bf0 0x7f342c1cf8b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f342c123360 tx=0x7f3428011c40 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3432ac2640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f341003be00 0x7f341003e2c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f342c0a0060 tx=0x7f34240a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34280044a0 con 0x7f342c139bf0 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3422ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3422ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3422ffd640 10 monclient: dump: 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.619 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.620 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.621 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.621 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.622 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f342800fc80 con 0x7f342c139bf0 2024-08-19T10:55:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3422ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3422ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c139bf0 msgr2=0x7f342c1cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3422ffd640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c139bf0 0x7f342c1cf8b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f342c123360 tx=0x7f3428011c40 comp rx=0 tx=0).stop 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c139bf0 msgr2=0x7f342c1cf8b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.594+0000 7f3422ffd640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c139bf0 0x7f342c1cf8b0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f342c123360 tx=0x7f3428011c40 comp rx=0 tx=0).stop 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.598+0000 7f3434525640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.598+0000 7f3434525640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c02d1d0 0x7f342c02d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.598+0000 7f3434525640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.598+0000 7f3434525640 1 -- 172.21.15.89:0/272911090 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f342c09d360 con 0x7f342c02d1d0 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c02d1d0 0x7f342c02d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 10 monclient: get_auth_request con 0x7f342c02d1d0 auth_method 0 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 10 monclient(hunting): handle_auth_done global_id 114132 payload 824 2024-08-19T10:55:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 1 -- 172.21.15.89:0/272911090 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f342c064590 con 0x7f342c02d1d0 2024-08-19T10:55:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 1 -- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c02d1d0 msgr2=0x7f342c02d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3431ac0640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c02d1d0 0x7f342c02d5e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f342c123360 tx=0x7f34280046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f34280148b0 con 0x7f342c02d1d0 2024-08-19T10:55:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3422ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3422ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.602+0000 7f3422ffd640 10 monclient: dump: 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.625 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.626 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f342801d3f0 con 0x7f342c02d1d0 2024-08-19T10:55:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3422ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3422ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c02d1d0 msgr2=0x7f342c02d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3422ffd640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c02d1d0 0x7f342c02d5e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f342c123360 tx=0x7f34280046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3422ffd640 1 -- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c02d1d0 msgr2=0x7f342c02d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:35.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3422ffd640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f342c02d1d0 0x7f342c02d5e0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f342c123360 tx=0x7f34280046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.627 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:35.627 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:35.627 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:35.627 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:35.627 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:35.627 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 16255, 2024-08-19T10:55:35.628 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:35.628 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:35.628 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:35.628 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:35.628 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:35.628 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:35.628 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:35.628 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:35.628 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:35.629 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:35.629 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.629 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 2, 2024-08-19T10:55:35.629 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:35.629 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:35.629 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:35.630 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.630 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.630 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.630 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.630 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.630 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.631 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.631 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.631 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.631 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.631 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.631 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.632 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:35.632 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:35.632 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.632 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.632 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.633 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.633 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.633 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.633 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.633 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.633 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.634 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.634 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.634 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:35.634 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:35.634 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:35.635 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:35.635 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:35.635 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:35.635 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:35.635 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:35.635 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:35.636 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:35.636 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:35.636 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:35.636 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:35.636 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:35.637 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:35.637 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:35.637 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:35.637 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.637 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.637 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.638 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.638 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.638 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.638 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.638 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.639 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.639 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.639 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.639 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.639 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:35.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.640 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:35.640 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.640 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:35.640 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:35.640 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.641 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.641 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.641 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:35.641 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.642 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.642 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:35.642 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.642 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.643 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.643 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:35.643 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:35.643 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.643 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.644 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.644 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:35.644 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:35.644 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.644 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.645 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.645 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:35.645 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.646 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.646 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:35.646 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.646 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.646 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.647 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.648 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.649 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.650 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.651 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:35.651 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:35.651 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.651 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.651 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.651 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:35.651 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:35.653 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:35.654 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.656 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.657 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.659 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.660 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:35.661 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:35.661 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:35.661 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.661 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.661 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:35.661 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3434525640 1 -- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f341003be00 msgr2=0x7f341003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3434525640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f341003be00 0x7f341003e2c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f342c0a0060 tx=0x7f34240a7000 comp rx=0 tx=0).stop 2024-08-19T10:55:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3434525640 10 monclient: shutdown 2024-08-19T10:55:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3434525640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3434525640 1 -- 172.21.15.89:0/272911090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f342c06c930 msgr2=0x7f342c123850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3434525640 1 --2- 172.21.15.89:0/272911090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f342c06c930 0x7f342c123850 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d800 tx=0x7f341c00dcc0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3432ac2640 1 -- 172.21.15.89:0/272911090 reap_dead start 2024-08-19T10:55:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3432ac2640 1 -- 172.21.15.89:0/272911090 reap_dead start 2024-08-19T10:55:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3434525640 1 -- 172.21.15.89:0/272911090 shutdown_connections 2024-08-19T10:55:35.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3434525640 1 -- 172.21.15.89:0/272911090 >> 172.21.15.89:0/272911090 conn(0x7f342c0a8f60 msgr2=0x7f342c073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:35.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3434525640 1 -- 172.21.15.89:0/272911090 shutdown_connections 2024-08-19T10:55:35.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.606+0000 7f3434525640 1 -- 172.21.15.89:0/272911090 wait complete. 2024-08-19T10:55:35.664 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:35.664 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:55:35.664 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:55:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.782+0000 7f2d9d614640 10 monclient: get_monmap_and_config 2024-08-19T10:55:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.782+0000 7f2d9d614640 10 monclient: build_initial_monmap 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.782+0000 7f2d9d614640 10 monclient: monmap: 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:35.788405+0000 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:35.788405+0000 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:35.784 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 1 Processor -- start 2024-08-19T10:55:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 1 -- start start 2024-08-19T10:55:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 10 monclient: init 2024-08-19T10:55:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 10 monclient: _add_conns ranks=[7,0,5,8,2,1,6,3,4,9,10] 2024-08-19T10:55:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d9809b620 0x7f2d9809ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 10 monclient(hunting): picked mon.noname-i con 0x7f2d9809b620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2d9809dd60 addr v1:172.21.15.89:6789/0 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980a4410 0x7f2d981414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 10 monclient(hunting): picked mon.noname-h con 0x7f2d980a4410 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d9809d660 con 0x7f2d9809dd60 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2d9806db50 con 0x7f2d980a4410 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2d9809d9b0 con 0x7f2d9809b620 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d9d614640 10 monclient(hunting): authenticate will time out at 1985.605137s 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d8ffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2d9809dd60 0x7f2d980a3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49432/0 (socket says 172.21.15.89:49432) 2024-08-19T10:55:35.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d8ffff640 1 -- 172.21.15.89:0/2106188144 learned_addr learned my addr 172.21.15.89:0/2106188144 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:55:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 1 --2- 172.21.15.89:0/2106188144 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980a4410 0x7f2d981414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d96ffd640 1 --2- 172.21.15.89:0/2106188144 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d9809b620 0x7f2d9809ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 10 monclient(hunting): get_auth_request con 0x7f2d980a4410 auth_method 0 2024-08-19T10:55:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d96ffd640 10 monclient(hunting): get_auth_request con 0x7f2d9809b620 auth_method 0 2024-08-19T10:55:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d96ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d96ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d96ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.790 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 10 monclient(hunting): handle_auth_done global_id 84281 payload 274 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 1 -- 172.21.15.89:0/2106188144 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d9809b620 msgr2=0x7f2d9809ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 1 --2- 172.21.15.89:0/2106188144 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d9809b620 0x7f2d9809ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 1 -- 172.21.15.89:0/2106188144 >> v1:172.21.15.89:6789/0 conn(0x7f2d9809dd60 legacy=0x7f2d980a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 1 monclient: found mon.noname-h 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:55:35.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.786+0000 7f2d977fe640 1 -- 172.21.15.89:0/2106188144 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d98069360 con 0x7f2d980a4410 2024-08-19T10:55:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d977fe640 1 -- 172.21.15.89:0/2106188144 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980a4410 msgr2=0x7f2d981414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d977fe640 1 --2- 172.21.15.89:0/2106188144 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980a4410 0x7f2d981414c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009dc0 tx=0x7f2d9001c850 comp rx=0 tx=0).ready entity=mon.5 client_cookie=9320c42ccbde1e78 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 1 -- 172.21.15.89:0/2106188144 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d900ac070 con 0x7f2d980a4410 2024-08-19T10:55:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:55:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 10 monclient: dump: 2024-08-19T10:55:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.798 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.799 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 10 monclient: _finish_auth 0 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 10 monclient: _check_auth_tickets 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 5 monclient: authenticate success, global_id 84281 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 1 -- 172.21.15.89:0/2106188144 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d900a68c0 con 0x7f2d980a4410 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 4 set_mon_vals no callback set 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 1 -- 172.21.15.89:0/2106188144 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d900a6af0 con 0x7f2d980a4410 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d967fc640 10 monclient: dump: 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.801 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 10 monclient: shutdown 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 1 -- 172.21.15.89:0/2106188144 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980a4410 msgr2=0x7f2d981414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 1 --2- 172.21.15.89:0/2106188144 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980a4410 0x7f2d981414c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009dc0 tx=0x7f2d9001c850 comp rx=0 tx=0).stop 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 1 -- 172.21.15.89:0/2106188144 shutdown_connections 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 1 --2- 172.21.15.89:0/2106188144 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980a4410 0x7f2d981414c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 1 --2- 172.21.15.89:0/2106188144 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d9809b620 0x7f2d9809ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 1 -- 172.21.15.89:0/2106188144 >> 172.21.15.89:0/2106188144 conn(0x7f2d980a4df0 msgr2=0x7f2d980a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 1 -- 172.21.15.89:0/2106188144 shutdown_connections 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.790+0000 7f2d9d614640 1 -- 172.21.15.89:0/2106188144 wait complete. 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient: build_initial_monmap 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient: monmap: 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:35.803 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 1 Processor -- start 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 1 -- start start 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient: init 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient: _add_conns ranks=[7,8,4,0,2,5,3,1,6] 2024-08-19T10:55:35.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d9809b620 0x7f2d981d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient(hunting): picked mon.noname-i con 0x7f2d9809b620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d9809dd60 0x7f2d981d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient(hunting): picked mon.noname-e con 0x7f2d9809dd60 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d980a4410 0x7f2d98092380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient(hunting): picked mon.noname-c con 0x7f2d980a4410 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f2d981443a0 con 0x7f2d980a4410 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2d98144220 con 0x7f2d9809b620 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2d98144520 con 0x7f2d9809dd60 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:35.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d9d614640 10 monclient(hunting): authenticate will time out at 1985.613547s 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d8ffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d9809dd60 0x7f2d981d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d980a4410 0x7f2d98092380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d8ffff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d9809dd60 0x7f2d981d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:59858/0 (socket says 172.21.15.89:59858) 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d8ffff640 1 -- 172.21.15.89:0/4045135648 learned_addr learned my addr 172.21.15.89:0/4045135648 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d8ffff640 10 monclient(hunting): get_auth_request con 0x7f2d9809dd60 auth_method 0 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d8ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d8ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 10 monclient(hunting): get_auth_request con 0x7f2d980a4410 auth_method 0 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d96ffd640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d9809b620 0x7f2d981d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d8ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d8ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d8ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d96ffd640 10 monclient(hunting): get_auth_request con 0x7f2d9809b620 auth_method 0 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d96ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d96ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d96ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 10 monclient(hunting): handle_auth_done global_id 84361 payload 824 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 1 -- 172.21.15.89:0/4045135648 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d9809dd60 msgr2=0x7f2d981d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d9809dd60 0x7f2d981d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 1 -- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d9809b620 msgr2=0x7f2d981d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2d9809b620 0x7f2d981d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 1 monclient: found mon.noname-c 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d977fe640 1 -- 172.21.15.89:0/4045135648 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d98095870 con 0x7f2d980a4410 2024-08-19T10:55:35.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.794+0000 7f2d8ffff640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2d9809dd60 0x7f2d981d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d977fe640 1 -- 172.21.15.89:0/4045135648 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d980a4410 msgr2=0x7f2d98092380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d977fe640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d980a4410 0x7f2d98092380 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d900ab000 tx=0x7f2d900195f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d900bf070 con 0x7f2d980a4410 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 10 monclient: dump: 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.809 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d900a6e40 con 0x7f2d980a4410 2024-08-19T10:55:35.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d900ac050 con 0x7f2d980a4410 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 10 monclient: dump: 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.811 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d95ffb640 4 set_mon_vals no callback set 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d9d614640 5 monclient: authenticate success, global_id 84361 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d9d614640 10 monclient: _renew_subs 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d9d614640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d9d614640 1 -- 172.21.15.89:0/4045135648 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d98093a50 con 0x7f2d980a4410 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d9d614640 10 monclient: _renew_subs 2024-08-19T10:55:35.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d9d614640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d9d614640 1 -- 172.21.15.89:0/4045135648 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d980a7c80 con 0x7f2d980a4410 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d9d614640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d9d614640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980928c0 0x7f2d98092ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d9d614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d9d614640 1 -- 172.21.15.89:0/4045135648 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2d980a1b30 con 0x7f2d980928c0 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d977fe640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d980a4410 0x7f2d98092380 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d900ab000 tx=0x7f2d900195f0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f2d900af070 mgrmap(e 5) 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2d900af070 con 0x7f2d980a4410 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d96ffd640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980928c0 0x7f2d98092ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d96ffd640 10 monclient: get_auth_request con 0x7f2d980928c0 auth_method 0 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d96ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d96ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d96ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8f7fe640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2d7803be00 0x7f2d7803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8ffff640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2d7803be00 0x7f2d7803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d8ffff640 10 monclient: get_auth_request con 0x7f2d7803be00 auth_method 0 2024-08-19T10:55:35.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_done global_id 84290 payload 824 2024-08-19T10:55:35.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.798+0000 7f2d96ffd640 1 -- 172.21.15.89:0/4045135648 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d980a7770 con 0x7f2d980928c0 2024-08-19T10:55:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d96ffd640 1 -- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980928c0 msgr2=0x7f2d98092ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d96ffd640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980928c0 0x7f2d98092ca0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d880119d0 tx=0x7f2d88011e90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d88019070 con 0x7f2d980928c0 2024-08-19T10:55:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:55:35.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8f7fe640 10 monclient: dump: 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.817 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8ffff640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2d7803be00 0x7f2d7803e2c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2d80007d60 tx=0x7f2d80007cb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2d8800f950 con 0x7f2d980928c0 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8f7fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980928c0 msgr2=0x7f2d98092ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8f7fe640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980928c0 0x7f2d98092ca0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d880119d0 tx=0x7f2d88011e90 comp rx=0 tx=0).stop 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980928c0 msgr2=0x7f2d98092ca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:35.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.802+0000 7f2d8f7fe640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980928c0 0x7f2d98092ca0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d880119d0 tx=0x7f2d88011e90 comp rx=0 tx=0).stop 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d9d614640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d9d614640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980931e0 0x7f2d981406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d9d614640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d9d614640 1 -- 172.21.15.89:0/4045135648 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2d9809ba00 con 0x7f2d980931e0 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980931e0 0x7f2d981406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 10 monclient: get_auth_request con 0x7f2d980931e0 auth_method 0 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_done global_id 84299 payload 824 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 1 -- 172.21.15.89:0/4045135648 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2d98064590 con 0x7f2d980931e0 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 1 -- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980931e0 msgr2=0x7f2d981406e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.810+0000 7f2d96ffd640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980931e0 0x7f2d981406e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d880096c0 tx=0x7f2d880046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2d88019040 con 0x7f2d980931e0 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 10 monclient: dump: 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.820 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f2d88003e10 con 0x7f2d980931e0 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:35.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:35.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980931e0 msgr2=0x7f2d981406e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980931e0 0x7f2d981406e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d880096c0 tx=0x7f2d880046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980931e0 msgr2=0x7f2d981406e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:35.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2d980931e0 0x7f2d981406e0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d880096c0 tx=0x7f2d880046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.822 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:35.822 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:35.822 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:35.822 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 144108, 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:35.823 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.824 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.825 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:35.826 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:35.827 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.830 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:35.831 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:35.835 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.835 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.835 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:35.835 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:35.835 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.835 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.835 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:35.837 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.837 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.837 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:35.837 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:35.837 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.837 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.837 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.838 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:35.838 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:35.838 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:35.838 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.842 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:35.843 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.844 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:35.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.846 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:35.847 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:55:35.847 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:35.847 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:35.847 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:35.847 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:35.847 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:35.847 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:35.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:35.847 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:35.847 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d9d614640 1 -- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2d7803be00 msgr2=0x7f2d7803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d9d614640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2d7803be00 0x7f2d7803e2c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2d80007d60 tx=0x7f2d80007cb0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d9d614640 10 monclient: shutdown 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d9d614640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d9d614640 1 -- 172.21.15.89:0/4045135648 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d980a4410 msgr2=0x7f2d98092380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d9d614640 1 --2- 172.21.15.89:0/4045135648 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f2d980a4410 0x7f2d98092380 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d900ab000 tx=0x7f2d900195f0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d977fe640 1 -- 172.21.15.89:0/4045135648 reap_dead start 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d977fe640 1 -- 172.21.15.89:0/4045135648 reap_dead start 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d8f7fe640 1 -- 172.21.15.89:0/4045135648 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (unknown 0 0 0) 0x7f2d900b7070 con 0x7f2d980a4410 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d9d614640 1 -- 172.21.15.89:0/4045135648 shutdown_connections 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d9d614640 1 -- 172.21.15.89:0/4045135648 >> 172.21.15.89:0/4045135648 conn(0x7f2d980a4df0 msgr2=0x7f2d980a0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d9d614640 1 -- 172.21.15.89:0/4045135648 shutdown_connections 2024-08-19T10:55:35.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.814+0000 7f2d9d614640 1 -- 172.21.15.89:0/4045135648 wait complete. 2024-08-19T10:55:35.850 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:35.850 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:55:35.850 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:55:35.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient: get_monmap_and_config 2024-08-19T10:55:35.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient: build_initial_monmap 2024-08-19T10:55:35.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient: monmap: 2024-08-19T10:55:35.968 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:35.968 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.968 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:35.973575+0000 2024-08-19T10:55:35.968 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:35.973575+0000 2024-08-19T10:55:35.968 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:35.968 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 1 Processor -- start 2024-08-19T10:55:35.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 1 -- start start 2024-08-19T10:55:35.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient: init 2024-08-19T10:55:35.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:35.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient: _add_conns ranks=[1,0,4,10,5,6,8,3,7,9,2] 2024-08-19T10:55:35.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0e700a39c0 addr v1:172.21.15.190:6789/0 2024-08-19T10:55:35.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0e700a44b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:55:35.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e70140040 0x7f0e70142430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient(hunting): picked mon.noname-c con 0x7f0e70140040 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:35.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e700a4d30 con 0x7f0e700a44b0 2024-08-19T10:55:35.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e700a4ab0 con 0x7f0e700a39c0 2024-08-19T10:55:35.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f0e700a02a0 con 0x7f0e70140040 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e75281640 10 monclient(hunting): authenticate will time out at 1985.790149s 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6e7fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0e700a44b0 0x7f0e7013e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49442/0 (socket says 172.21.15.89:49442) 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6e7fc640 1 -- 172.21.15.89:0/3865881255 learned_addr learned my addr 172.21.15.89:0/3865881255 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 1 --2- 172.21.15.89:0/3865881255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e70140040 0x7f0e70142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 10 monclient(hunting): get_auth_request con 0x7f0e70140040 auth_method 0 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 10 monclient(hunting): handle_auth_done global_id 84370 payload 274 2024-08-19T10:55:35.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 1 -- 172.21.15.89:0/3865881255 >> v1:172.21.15.190:6789/0 conn(0x7f0e700a39c0 legacy=0x7f0e700a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:35.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 1 -- 172.21.15.89:0/3865881255 >> v1:172.21.15.89:6789/0 conn(0x7f0e700a44b0 legacy=0x7f0e7013e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:35.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:35.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 1 monclient: found mon.noname-c 2024-08-19T10:55:35.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:35.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:35.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.970+0000 7f0e6f7fe640 1 -- 172.21.15.89:0/3865881255 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e700a0910 con 0x7f0e70140040 2024-08-19T10:55:35.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6f7fe640 1 -- 172.21.15.89:0/3865881255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e70140040 msgr2=0x7f0e70142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6f7fe640 1 --2- 172.21.15.89:0/3865881255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e70140040 0x7f0e70142430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097f0 tx=0x7f0e6801c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c3de739d737a5d6e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 1 -- 172.21.15.89:0/3865881255 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e680b0070 con 0x7f0e70140040 2024-08-19T10:55:35.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 10 monclient: dump: 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.977 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 10 monclient: _finish_auth 0 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 1 -- 172.21.15.89:0/3865881255 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e680aa810 con 0x7f0e70140040 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 1 -- 172.21.15.89:0/3865881255 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e680aae30 con 0x7f0e70140040 2024-08-19T10:55:35.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e75281640 5 monclient: authenticate success, global_id 84370 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e75281640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e75281640 4 set_mon_vals no callback set 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.974+0000 7f0e6dffb640 10 monclient: dump: 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient: shutdown 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 -- 172.21.15.89:0/3865881255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e70140040 msgr2=0x7f0e70142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 --2- 172.21.15.89:0/3865881255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e70140040 0x7f0e70142430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097f0 tx=0x7f0e6801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 -- 172.21.15.89:0/3865881255 shutdown_connections 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 --2- 172.21.15.89:0/3865881255 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f0e70140040 0x7f0e70142430 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 -- 172.21.15.89:0/3865881255 >> 172.21.15.89:0/3865881255 conn(0x7f0e700a8f60 msgr2=0x7f0e700a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 -- 172.21.15.89:0/3865881255 shutdown_connections 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 -- 172.21.15.89:0/3865881255 wait complete. 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient: build_initial_monmap 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient: monmap: 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:35.981 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 Processor -- start 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 -- start start 2024-08-19T10:55:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient: init 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient: _add_conns ranks=[2,3,5,0,7,4,1,8,6] 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e700a39c0 0x7f0e7013a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient(hunting): picked mon.noname-b con 0x7f0e700a39c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e700a44b0 0x7f0e7013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient(hunting): picked mon.noname-g con 0x7f0e700a44b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e70140040 0x7f0e7013cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient(hunting): picked mon.noname-h con 0x7f0e70140040 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f0e70074520 con 0x7f0e700a39c0 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0e700743a0 con 0x7f0e700a44b0 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f0e701455a0 con 0x7f0e70140040 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e75281640 10 monclient(hunting): authenticate will time out at 1985.798232s 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e700a39c0 0x7f0e7013a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e700a39c0 0x7f0e7013a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60878/0 (socket says 172.21.15.89:60878) 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 1 -- 172.21.15.89:0/4118733150 learned_addr learned my addr 172.21.15.89:0/4118733150 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 10 monclient(hunting): get_auth_request con 0x7f0e700a39c0 auth_method 0 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6f7fe640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e70140040 0x7f0e7013cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 10 monclient(hunting): handle_auth_done global_id 74540 payload 824 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 1 -- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e70140040 msgr2=0x7f0e7013cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e70140040 0x7f0e7013cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6e7fc640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e700a44b0 0x7f0e7013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 1 -- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e700a44b0 msgr2=0x7f0e7013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e700a44b0 0x7f0e7013a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 10 monclient: _finish_hunting 0 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 1 monclient: found mon.noname-b 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6effd640 1 -- 172.21.15.89:0/4118733150 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e70123c50 con 0x7f0e700a39c0 2024-08-19T10:55:35.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6f7fe640 10 monclient: get_auth_request con 0x7f0e70140040 auth_method 0 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.978+0000 7f0e6f7fe640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f0e70140040 0x7f0e7013cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6effd640 1 -- 172.21.15.89:0/4118733150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e700a39c0 msgr2=0x7f0e7013a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6effd640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e700a39c0 0x7f0e7013a360 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000c7c0 tx=0x7f0e6000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e60016020 con 0x7f0e700a39c0 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 10 monclient: dump: 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.991 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 10 monclient: _finish_auth 0 2024-08-19T10:55:35.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 10 monclient: _check_auth_tickets 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e60012420 con 0x7f0e700a39c0 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e60012a80 con 0x7f0e700a39c0 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 10 monclient: dump: 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.993 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e75281640 5 monclient: authenticate success, global_id 74540 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6cff9640 4 set_mon_vals no callback set 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e75281640 10 monclient: _renew_subs 2024-08-19T10:55:35.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e75281640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e75281640 1 -- 172.21.15.89:0/4118733150 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e70141e30 con 0x7f0e700a39c0 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e75281640 10 monclient: _renew_subs 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e75281640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e75281640 1 -- 172.21.15.89:0/4118733150 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e70124210 con 0x7f0e700a39c0 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e75281640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e75281640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013d1a0 0x7f0e7013d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e75281640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e75281640 1 -- 172.21.15.89:0/4118733150 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0e70064590 con 0x7f0e7013d1a0 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0e60019020 con 0x7f0e700a39c0 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e5003be00 0x7f0e5003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f0e6004f720 con 0x7f0e700a39c0 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6e7fc640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013d1a0 0x7f0e7013d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6e7fc640 10 monclient: get_auth_request con 0x7f0e7013d1a0 auth_method 0 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6f7fe640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e5003be00 0x7f0e5003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6f7fe640 10 monclient: get_auth_request con 0x7f0e5003be00 auth_method 0 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6e7fc640 10 monclient(hunting): handle_auth_done global_id 114127 payload 824 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.982+0000 7f0e6e7fc640 1 -- 172.21.15.89:0/4118733150 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e700a3da0 con 0x7f0e7013d1a0 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e6e7fc640 1 -- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013d1a0 msgr2=0x7f0e7013d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e6e7fc640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013d1a0 0x7f0e7013d580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0e7013e3c0 tx=0x7f0e5800d880 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e6f7fe640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e5003be00 0x7f0e5003e2c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0e680ab000 tx=0x7f0e680a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:35.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e58018070 con 0x7f0e7013d1a0 2024-08-19T10:55:35.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:35.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e677fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:55:35.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e677fe640 10 monclient: dump: 2024-08-19T10:55:35.997 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:35.997 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:35.997 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.997 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:35.997 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:35.997 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0e58005d00 con 0x7f0e7013d1a0 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e677fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:35.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e677fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:35.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013d1a0 msgr2=0x7f0e7013d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:35.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e677fe640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013d1a0 0x7f0e7013d580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0e7013e3c0 tx=0x7f0e5800d880 comp rx=0 tx=0).stop 2024-08-19T10:55:35.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013d1a0 msgr2=0x7f0e7013d580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:35.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.986+0000 7f0e677fe640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013d1a0 0x7f0e7013d580 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e75281640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:36.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e75281640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013dac0 0x7f0e70070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:36.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e75281640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:36.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e75281640 1 -- 172.21.15.89:0/4118733150 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f0e70123ff0 con 0x7f0e7013dac0 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e6e7fc640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013dac0 0x7f0e70070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e6e7fc640 10 monclient: get_auth_request con 0x7f0e7013dac0 auth_method 0 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e6e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e6e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e6e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e6e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e6e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e6e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e6e7fc640 10 monclient(hunting): handle_auth_done global_id 114136 payload 824 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.994+0000 7f0e6e7fc640 1 -- 172.21.15.89:0/4118733150 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0e7013efa0 con 0x7f0e7013dac0 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e6e7fc640 1 -- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013dac0 msgr2=0x7f0e70070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e6e7fc640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013dac0 0x7f0e70070e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e7013e3c0 tx=0x7f0e58004950 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:36.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0e58018030 con 0x7f0e7013dac0 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e677fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e677fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e677fe640 10 monclient: dump: 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:36.002 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f0e5801bad0 con 0x7f0e7013dac0 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e677fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e677fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013dac0 msgr2=0x7f0e70070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e677fe640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013dac0 0x7f0e70070e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e7013e3c0 tx=0x7f0e58004950 comp rx=0 tx=0).stop 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e677fe640 1 -- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013dac0 msgr2=0x7f0e70070e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:36.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e677fe640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f0e7013dac0 0x7f0e70070e10 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e7013e3c0 tx=0x7f0e58004950 comp rx=0 tx=0).stop 2024-08-19T10:55:36.004 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:36.004 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:36.004 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:36.004 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:36.004 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:36.004 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 16647, 2024-08-19T10:55:36.004 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:36.004 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:36.004 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 3, 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:36.005 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:36.006 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:36.006 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:36.006 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:36.006 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:36.006 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:36.006 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:36.006 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:36.006 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:36.006 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:36.006 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:36.006 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:36.007 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:36.008 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:36.009 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.010 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:36.011 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.012 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.013 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.014 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:36.016 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:36.017 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:36.018 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.020 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:36.021 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:36.023 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.024 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:36.026 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:36.027 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:36.027 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:36.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:36.027 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:36.027 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e75281640 1 -- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e5003be00 msgr2=0x7f0e5003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e75281640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f0e5003be00 0x7f0e5003e2c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0e680ab000 tx=0x7f0e680a6000 comp rx=0 tx=0).stop 2024-08-19T10:55:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e75281640 10 monclient: shutdown 2024-08-19T10:55:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e75281640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:36.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e75281640 1 -- 172.21.15.89:0/4118733150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e700a39c0 msgr2=0x7f0e7013a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e75281640 1 --2- 172.21.15.89:0/4118733150 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f0e700a39c0 0x7f0e7013a360 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000c7c0 tx=0x7f0e6000cc80 comp rx=0 tx=0).stop 2024-08-19T10:55:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e6f7fe640 1 -- 172.21.15.89:0/4118733150 reap_dead start 2024-08-19T10:55:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e6f7fe640 1 -- 172.21.15.89:0/4118733150 reap_dead start 2024-08-19T10:55:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e75281640 1 -- 172.21.15.89:0/4118733150 shutdown_connections 2024-08-19T10:55:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e75281640 1 -- 172.21.15.89:0/4118733150 >> 172.21.15.89:0/4118733150 conn(0x7f0e700a8f60 msgr2=0x7f0e70140d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e75281640 1 -- 172.21.15.89:0/4118733150 shutdown_connections 2024-08-19T10:55:36.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:35.998+0000 7f0e75281640 1 -- 172.21.15.89:0/4118733150 wait complete. 2024-08-19T10:55:36.029 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:36.029 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-08-19T10:55:36.030 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-08-19T10:55:36.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.146+0000 7f70fccd3640 10 monclient: get_monmap_and_config 2024-08-19T10:55:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.146+0000 7f70fccd3640 10 monclient: build_initial_monmap 2024-08-19T10:55:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.146+0000 7f70fccd3640 10 monclient: monmap: 2024-08-19T10:55:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:36.151753+0000 2024-08-19T10:55:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:36.151753+0000 2024-08-19T10:55:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:36.146 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:36.147 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:36.147 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:36.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.146+0000 7f70fccd3640 1 Processor -- start 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.146+0000 7f70fccd3640 1 -- start start 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.146+0000 7f70fccd3640 10 monclient: init 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 10 monclient: _add_conns ranks=[2,7,8,5,10,4,0,3,1,9,6] 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f70f80a5270 0x7f70f80a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 10 monclient(hunting): picked mon.noname-b con 0x7f70f80a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a6d30 0x7f70f80a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 10 monclient(hunting): picked mon.noname-i con 0x7f70f80a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f70f81395b0 0x7f70f81399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 10 monclient(hunting): picked mon.noname-e con 0x7f70f81395b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:36.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f70f80a7650 con 0x7f70f80a5270 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f70f80a8710 con 0x7f70f80a6d30 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f70f80a5a40 con 0x7f70f81395b0 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70fccd3640 10 monclient(hunting): authenticate will time out at 1985.968310s 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f70f80a5270 0x7f70f80a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f6d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f70f81395b0 0x7f70f81399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f70f80a5270 0x7f70f80a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60884/0 (socket says 172.21.15.89:60884) 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f6d9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f70f81395b0 0x7f70f81399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:59870/0 (socket says 172.21.15.89:59870) 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 1 -- 172.21.15.89:0/1325885179 learned_addr learned my addr 172.21.15.89:0/1325885179 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 10 monclient(hunting): get_auth_request con 0x7f70f80a5270 auth_method 0 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:36.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f6d9d640 10 monclient(hunting): get_auth_request con 0x7f70f81395b0 auth_method 0 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f5d9b640 1 --2- 172.21.15.89:0/1325885179 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a6d30 0x7f70f80a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:36.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f5d9b640 10 monclient(hunting): get_auth_request con 0x7f70f80a6d30 auth_method 0 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 10 monclient(hunting): handle_auth_done global_id 74549 payload 274 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 1 -- 172.21.15.89:0/1325885179 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f70f81395b0 msgr2=0x7f70f81399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 1 --2- 172.21.15.89:0/1325885179 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f70f81395b0 0x7f70f81399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 1 -- 172.21.15.89:0/1325885179 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a6d30 msgr2=0x7f70f80a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 1 --2- 172.21.15.89:0/1325885179 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a6d30 0x7f70f80a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 10 monclient: _finish_hunting 0 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 1 monclient: found mon.noname-b 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f659c640 1 -- 172.21.15.89:0/1325885179 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f80a81d0 con 0x7f70f80a5270 2024-08-19T10:55:36.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.150+0000 7f70f6d9d640 1 --2- 172.21.15.89:0/1325885179 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f70f81395b0 0x7f70f81399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f659c640 1 -- 172.21.15.89:0/1325885179 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f70f80a5270 msgr2=0x7f70f80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f659c640 1 --2- 172.21.15.89:0/1325885179 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f70f80a5270 0x7f70f80a67f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f70e80098a0 tx=0x7f70e801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3301675d764d7125 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 1 -- 172.21.15.89:0/1325885179 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70e80b0070 con 0x7f70f80a5270 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 10 monclient: dump: 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:36.158 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 10 monclient: _finish_auth 0 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 10 monclient: _check_auth_tickets 2024-08-19T10:55:36.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 1 -- 172.21.15.89:0/1325885179 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e80aa810 con 0x7f70f80a5270 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 5 monclient: authenticate success, global_id 74549 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 4 set_mon_vals no callback set 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 1 -- 172.21.15.89:0/1325885179 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70e80a9300 con 0x7f70f80a5270 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70f559a640 10 monclient: dump: 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:36.160 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient: shutdown 2024-08-19T10:55:36.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 -- 172.21.15.89:0/1325885179 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f70f80a5270 msgr2=0x7f70f80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 --2- 172.21.15.89:0/1325885179 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f70f80a5270 0x7f70f80a67f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f70e80098a0 tx=0x7f70e801c440 comp rx=0 tx=0).stop 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 -- 172.21.15.89:0/1325885179 shutdown_connections 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 --2- 172.21.15.89:0/1325885179 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f70f81395b0 0x7f70f81399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 --2- 172.21.15.89:0/1325885179 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a6d30 0x7f70f80a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 --2- 172.21.15.89:0/1325885179 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f70f80a5270 0x7f70f80a67f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 -- 172.21.15.89:0/1325885179 >> 172.21.15.89:0/1325885179 conn(0x7f70f8069a80 msgr2=0x7f70f80abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 -- 172.21.15.89:0/1325885179 shutdown_connections 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 -- 172.21.15.89:0/1325885179 wait complete. 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient: build_initial_monmap 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient: monmap: 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:36.162 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:36.163 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 Processor -- start 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 -- start start 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient: init 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient: _add_conns ranks=[7,3,5,0,2,1,8,4,6] 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a5270 0x7f70f813dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient(hunting): picked mon.noname-i con 0x7f70f80a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f80a6d30 0x7f70f813e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient(hunting): picked mon.noname-g con 0x7f70f80a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f70f81395b0 0x7f70f813e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient(hunting): picked mon.noname-h con 0x7f70f81395b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:36.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f70f80a57d0 con 0x7f70f80a6d30 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f70f80a5650 con 0x7f70f81395b0 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f70f8143230 con 0x7f70f80a5270 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.154+0000 7f70fccd3640 10 monclient(hunting): authenticate will time out at 1985.975829s 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f659c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a5270 0x7f70f813dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f80a6d30 0x7f70f813e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f659c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a5270 0x7f70f813dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3303/0 says I am v2:172.21.15.89:47640/0 (socket says 172.21.15.89:47640) 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f659c640 1 -- 172.21.15.89:0/3350623736 learned_addr learned my addr 172.21.15.89:0/3350623736 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f6d9d640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f70f81395b0 0x7f70f813e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f659c640 10 monclient(hunting): get_auth_request con 0x7f70f80a5270 auth_method 0 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:36.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f659c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f659c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 10 monclient(hunting): get_auth_request con 0x7f70f80a6d30 auth_method 0 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f6d9d640 10 monclient(hunting): get_auth_request con 0x7f70f81395b0 auth_method 0 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f6d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:36.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 10 monclient(hunting): handle_auth_done global_id 114141 payload 824 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 1 -- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a5270 msgr2=0x7f70f813dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a5270 0x7f70f813dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 1 -- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f70f81395b0 msgr2=0x7f70f813e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f70f81395b0 0x7f70f813e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 1 monclient: found mon.noname-g 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:36.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:55:36.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 1 -- 172.21.15.89:0/3350623736 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f809e670 con 0x7f70f80a6d30 2024-08-19T10:55:36.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f659c640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a5270 0x7f70f813dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:36.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 1 -- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f80a6d30 msgr2=0x7f70f813e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:36.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f5d9b640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f80a6d30 0x7f70f813e270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00d620 tx=0x7f70ec00dae0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:36.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 1 -- 172.21.15.89:0/3350623736 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70ec00bca0 con 0x7f70f80a6d30 2024-08-19T10:55:36.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:36.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:55:36.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 10 monclient: dump: 2024-08-19T10:55:36.170 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:36.170 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:36.170 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:36.170 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:36.170 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:36.170 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:36.170 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70fccd3640 5 monclient: authenticate success, global_id 114141 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70fccd3640 10 monclient: _renew_subs 2024-08-19T10:55:36.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 1 -- 172.21.15.89:0/3350623736 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70ec019070 con 0x7f70f80a6d30 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70fccd3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70fccd3640 1 -- 172.21.15.89:0/3350623736 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f809e950 con 0x7f70f80a6d30 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 1 -- 172.21.15.89:0/3350623736 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70ec0148a0 con 0x7f70f80a6d30 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70e6ffd640 10 monclient: dump: 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:36.172 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70f4d99640 4 set_mon_vals no callback set 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70fccd3640 10 monclient: _renew_subs 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70fccd3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:55:36.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.158+0000 7f70fccd3640 1 -- 172.21.15.89:0/3350623736 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f809e390 con 0x7f70f80a6d30 2024-08-19T10:55:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.162+0000 7f70fccd3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.162+0000 7f70fccd3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.162+0000 7f70fccd3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:55:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.162+0000 7f70fccd3640 1 -- 172.21.15.89:0/3350623736 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f8140710 con 0x7f70f80a6d30 2024-08-19T10:55:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.162+0000 7f70e6ffd640 1 -- 172.21.15.89:0/3350623736 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f70ec014430 con 0x7f70f80a6d30 2024-08-19T10:55:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.162+0000 7f70e6ffd640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f70d803be00 0x7f70d803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.162+0000 7f70e6ffd640 1 -- 172.21.15.89:0/3350623736 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f70ec00f040 con 0x7f70f80a6d30 2024-08-19T10:55:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.162+0000 7f70f659c640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f70d803be00 0x7f70d803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.162+0000 7f70f659c640 10 monclient: get_auth_request con 0x7f70d803be00 auth_method 0 2024-08-19T10:55:36.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.166+0000 7f70f659c640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f70d803be00 0x7f70d803e2c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f70e801c270 tx=0x7f70e80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:36.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.174+0000 7f70e6ffd640 1 -- 172.21.15.89:0/3350623736 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f70f8140710 con 0x7f70f80a6d30 2024-08-19T10:55:36.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.174+0000 7f70e6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:36.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.174+0000 7f70e6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.398+0000 7f70fccd3640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-08-19T10:55:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.398+0000 7f70fccd3640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-08-19T10:55:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.398+0000 7f70fccd3640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:55:36.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.398+0000 7f70fccd3640 1 -- 172.21.15.89:0/3350623736 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f70f80a67f0 con 0x7f70f80a6d30 2024-08-19T10:55:36.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=11,kv=10,logm=49} crc {auth=674992314,config=2586702516,health=1045639550,kv=1960968631,logm=857970863}) 2024-08-19T10:55:36.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=29,config=2,health=11,kv=12,logm=46} crc {auth=2661113809,config=2586702516,health=1861899969,kv=1101891974,logm=3372495088}) 2024-08-19T10:55:36.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=11,kv=10,logm=49} crc {auth=674992314,config=2586702516,health=1045639550,kv=1960968631,logm=857970863}) 2024-08-19T10:55:36.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=26,config=2,health=12,kv=12,logm=48} crc {auth=1518104242,config=2586702516,health=655881190,kv=1101891975,logm=1777914469}) 2024-08-19T10:55:36.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=11,kv=10,logm=49} crc {auth=674992314,config=2586702516,health=1045639550,kv=1960968631,logm=857970863}) 2024-08-19T10:55:36.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=29,config=1,health=11,kv=11,logm=48} crc {auth=3289261740,config=1747058506,health=1909376855,kv=1101891974,logm=1350290820}) 2024-08-19T10:55:36.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.404 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=11,kv=10,logm=49} crc {auth=674992314,config=2586702516,health=1045639550,kv=1960968631,logm=857970863}) 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=28,config=2,health=12,kv=12,logm=46} crc {auth=3391113936,config=2586702516,health=655881190,kv=1101891974,logm=3437946461}) 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=11,kv=10,logm=49} crc {auth=674992314,config=2586702516,health=1045639550,kv=1960968631,logm=857970863}) 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=28,config=2,health=12,kv=12,logm=46} crc {auth=1503393566,config=2586702516,health=655881190,kv=3153366146,logm=3132664687}) 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=11,kv=10,logm=49} crc {auth=674992314,config=2586702516,health=1045639550,kv=1960968631,logm=857970863}) 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=27,config=2,health=11,kv=12,logm=48} crc {auth=2918613082,config=2586702516,health=1909376855,kv=1101891974,logm=142897387}) 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=11,kv=10,logm=49} crc {auth=674992314,config=2586702516,health=1045639550,kv=1960968631,logm=857970863}) 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=27,config=2,health=12,kv=12,logm=47} crc {auth=323914395,config=2586702516,health=655881190,kv=1101891974,logm=3568672155}) 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.405 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=11,kv=10,logm=49} crc {auth=674992314,config=2586702516,health=1045639550,kv=1960968631,logm=857970863}) 2024-08-19T10:55:36.406 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.402+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=29,config=2,health=12,kv=11,logm=46} crc {auth=3289261740,config=2586702516,health=655881190,kv=1905375608,logm=492495655}) 2024-08-19T10:55:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70e6ffd640 1 -- 172.21.15.89:0/3350623736 <== mon.3 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f70ec022020 con 0x7f70f80a6d30 2024-08-19T10:55:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70e6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-08-19T10:55:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70e6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 -- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f70d803be00 msgr2=0x7f70d803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f70d803be00 0x7f70d803e2c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f70e801c270 tx=0x7f70e80a6000 comp rx=0 tx=0).stop 2024-08-19T10:55:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 10 monclient: shutdown 2024-08-19T10:55:36.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 -- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f80a6d30 msgr2=0x7f70f813e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f80a6d30 0x7f70f813e270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00d620 tx=0x7f70ec00dae0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 -- 172.21.15.89:0/3350623736 shutdown_connections 2024-08-19T10:55:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f70d803be00 0x7f70d803e2c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f70f81395b0 0x7f70f813e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70f80a6d30 0x7f70f813e270 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 --2- 172.21.15.89:0/3350623736 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f70f80a5270 0x7f70f813dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 -- 172.21.15.89:0/3350623736 >> 172.21.15.89:0/3350623736 conn(0x7f70f8069a80 msgr2=0x7f70f80ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 -- 172.21.15.89:0/3350623736 shutdown_connections 2024-08-19T10:55:36.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:36.406+0000 7f70fccd3640 1 -- 172.21.15.89:0/3350623736 wait complete. 2024-08-19T10:55:36.418 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100757722}) 2024-08-19T10:55:36.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1844958016}) 2024-08-19T10:55:36.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100757722}) 2024-08-19T10:55:36.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3547517903}) 2024-08-19T10:55:36.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100757722}) 2024-08-19T10:55:36.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3798925844}) 2024-08-19T10:55:36.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.419 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100757722}) 2024-08-19T10:55:36.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=658594955}) 2024-08-19T10:55:36.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100757722}) 2024-08-19T10:55:36.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=381683703}) 2024-08-19T10:55:36.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100757722}) 2024-08-19T10:55:36.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1496388515}) 2024-08-19T10:55:36.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100757722}) 2024-08-19T10:55:36.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3236936270}) 2024-08-19T10:55:36.420 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.421 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100757722}) 2024-08-19T10:55:36.421 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.418+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=534383821}) 2024-08-19T10:55:36.423 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-08-19T10:55:36.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1430003780}) 2024-08-19T10:55:36.427 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1772438735}) 2024-08-19T10:55:36.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1430003780}) 2024-08-19T10:55:36.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=611811865}) 2024-08-19T10:55:36.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1430003780}) 2024-08-19T10:55:36.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=447762124}) 2024-08-19T10:55:36.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1430003780}) 2024-08-19T10:55:36.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2077949074}) 2024-08-19T10:55:36.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.428 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1430003780}) 2024-08-19T10:55:36.429 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1964735810}) 2024-08-19T10:55:36.429 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.429 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1430003780}) 2024-08-19T10:55:36.429 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4276324867}) 2024-08-19T10:55:36.429 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.429 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1430003780}) 2024-08-19T10:55:36.429 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3659431926}) 2024-08-19T10:55:36.429 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.429 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1430003780}) 2024-08-19T10:55:36.429 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.426+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399996870}) 2024-08-19T10:55:36.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=156910078}) 2024-08-19T10:55:36.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3396576282}) 2024-08-19T10:55:36.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=156910078}) 2024-08-19T10:55:36.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3503120564}) 2024-08-19T10:55:36.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=156910078}) 2024-08-19T10:55:36.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2737203115}) 2024-08-19T10:55:36.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=156910078}) 2024-08-19T10:55:36.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3017744188}) 2024-08-19T10:55:36.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=156910078}) 2024-08-19T10:55:36.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=477334472}) 2024-08-19T10:55:36.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=156910078}) 2024-08-19T10:55:36.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2007719601}) 2024-08-19T10:55:36.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=156910078}) 2024-08-19T10:55:36.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2777932334}) 2024-08-19T10:55:36.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=156910078}) 2024-08-19T10:55:36.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.670+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3930966485}) 2024-08-19T10:55:36.695 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.695 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=183689392}) 2024-08-19T10:55:36.695 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=337828305}) 2024-08-19T10:55:36.695 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.695 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=183689392}) 2024-08-19T10:55:36.695 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=100499621}) 2024-08-19T10:55:36.695 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=183689392}) 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=274274095}) 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=183689392}) 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4221824978}) 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=183689392}) 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2688922744}) 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=183689392}) 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3676906916}) 2024-08-19T10:55:36.696 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=183689392}) 2024-08-19T10:55:36.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3138219596}) 2024-08-19T10:55:36.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.694+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=183689392}) 2024-08-19T10:55:36.697 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.698+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=197378980}) 2024-08-19T10:55:36.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3003694126}) 2024-08-19T10:55:36.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=170212061}) 2024-08-19T10:55:36.807 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3003694126}) 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3430609409}) 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3003694126}) 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=788335485}) 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3003694126}) 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=627987413}) 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3003694126}) 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=242543715}) 2024-08-19T10:55:36.808 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3003694126}) 2024-08-19T10:55:36.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2002759880}) 2024-08-19T10:55:36.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3003694126}) 2024-08-19T10:55:36.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=25005971}) 2024-08-19T10:55:36.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3003694126}) 2024-08-19T10:55:36.809 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.806+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3557141763}) 2024-08-19T10:55:36.811 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.811 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046835941}) 2024-08-19T10:55:36.811 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=962105000}) 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046835941}) 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1525631932}) 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046835941}) 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=411676436}) 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046835941}) 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2491100951}) 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046835941}) 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=361412866}) 2024-08-19T10:55:36.812 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046835941}) 2024-08-19T10:55:36.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3658345063}) 2024-08-19T10:55:36.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046835941}) 2024-08-19T10:55:36.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2270203003}) 2024-08-19T10:55:36.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1046835941}) 2024-08-19T10:55:36.813 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.810+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1870927394}) 2024-08-19T10:55:36.816 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.816 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1677343414}) 2024-08-19T10:55:36.816 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2135059792}) 2024-08-19T10:55:36.816 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.816 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1677343414}) 2024-08-19T10:55:36.816 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=112910784}) 2024-08-19T10:55:36.816 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.816 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1677343414}) 2024-08-19T10:55:36.816 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=945308192}) 2024-08-19T10:55:36.816 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1677343414}) 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2538802497}) 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1677343414}) 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1702540599}) 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1677343414}) 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2952267700}) 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1677343414}) 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=501247642}) 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1677343414}) 2024-08-19T10:55:36.817 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3198132260}) 2024-08-19T10:55:36.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1025008294}) 2024-08-19T10:55:36.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4157420468}) 2024-08-19T10:55:36.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1025008294}) 2024-08-19T10:55:36.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1411612314}) 2024-08-19T10:55:36.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1025008294}) 2024-08-19T10:55:36.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1248512606}) 2024-08-19T10:55:36.820 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1025008294}) 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3836152065}) 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1025008294}) 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3361673487}) 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1025008294}) 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1492055115}) 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1025008294}) 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2079893682}) 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.821 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1025008294}) 2024-08-19T10:55:36.822 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31807821}) 2024-08-19T10:55:36.823 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.823 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4238193263}) 2024-08-19T10:55:36.823 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3030037989}) 2024-08-19T10:55:36.823 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.823 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4238193263}) 2024-08-19T10:55:36.823 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1822967046}) 2024-08-19T10:55:36.823 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.823 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4238193263}) 2024-08-19T10:55:36.823 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2473251894}) 2024-08-19T10:55:36.823 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4238193263}) 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1407348230}) 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4238193263}) 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3334429997}) 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4238193263}) 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=531329127}) 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4238193263}) 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=482775897}) 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4238193263}) 2024-08-19T10:55:36.825 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.822+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2550391374}) 2024-08-19T10:55:36.826 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.826 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759324718}) 2024-08-19T10:55:36.826 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2953160817}) 2024-08-19T10:55:36.826 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.826 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759324718}) 2024-08-19T10:55:36.826 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2771476439}) 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759324718}) 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4251665249}) 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759324718}) 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=308312535}) 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759324718}) 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3033432985}) 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759324718}) 2024-08-19T10:55:36.827 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=547333236}) 2024-08-19T10:55:36.828 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.828 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759324718}) 2024-08-19T10:55:36.828 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3093963109}) 2024-08-19T10:55:36.828 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.828 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759324718}) 2024-08-19T10:55:36.828 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.826+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4092292946}) 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167998335}) 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3525062031}) 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167998335}) 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1558930742}) 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167998335}) 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2687626385}) 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167998335}) 2024-08-19T10:55:36.830 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1543643257}) 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167998335}) 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1481968120}) 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167998335}) 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=63745269}) 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167998335}) 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4112222414}) 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=167998335}) 2024-08-19T10:55:36.831 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.830+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3177930343}) 2024-08-19T10:55:36.833 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.833 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3294970444}) 2024-08-19T10:55:36.833 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1598515658}) 2024-08-19T10:55:36.833 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.833 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3294970444}) 2024-08-19T10:55:36.833 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3808063350}) 2024-08-19T10:55:36.833 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.833 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3294970444}) 2024-08-19T10:55:36.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=256866342}) 2024-08-19T10:55:36.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3294970444}) 2024-08-19T10:55:36.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2754473956}) 2024-08-19T10:55:36.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3294970444}) 2024-08-19T10:55:36.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4149025068}) 2024-08-19T10:55:36.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3294970444}) 2024-08-19T10:55:36.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2189183578}) 2024-08-19T10:55:36.834 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3294970444}) 2024-08-19T10:55:36.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1492044647}) 2024-08-19T10:55:36.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3294970444}) 2024-08-19T10:55:36.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.834+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3153490518}) 2024-08-19T10:55:36.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=472093955}) 2024-08-19T10:55:36.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3997342290}) 2024-08-19T10:55:36.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=472093955}) 2024-08-19T10:55:36.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=292074002}) 2024-08-19T10:55:36.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=472093955}) 2024-08-19T10:55:36.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1125099979}) 2024-08-19T10:55:36.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=472093955}) 2024-08-19T10:55:36.838 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1001491390}) 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=472093955}) 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=224820150}) 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=472093955}) 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1766727321}) 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=472093955}) 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3859180597}) 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=472093955}) 2024-08-19T10:55:36.839 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.838+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=386590938}) 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3946994834}) 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2630611278}) 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3946994834}) 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1521328191}) 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3946994834}) 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1917324569}) 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3946994834}) 2024-08-19T10:55:36.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1909300940}) 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3946994834}) 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=600192747}) 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3946994834}) 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1152219678}) 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3946994834}) 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3758973661}) 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3946994834}) 2024-08-19T10:55:36.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.842+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1522110139}) 2024-08-19T10:55:36.846 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.846 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2684725305}) 2024-08-19T10:55:36.846 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2239280967}) 2024-08-19T10:55:36.846 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.846 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2684725305}) 2024-08-19T10:55:36.846 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2396732443}) 2024-08-19T10:55:36.846 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2684725305}) 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2266674714}) 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2684725305}) 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=314531313}) 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2684725305}) 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3056573865}) 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2684725305}) 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1707980973}) 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.847 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2684725305}) 2024-08-19T10:55:36.848 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1150621474}) 2024-08-19T10:55:36.848 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.849 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2684725305}) 2024-08-19T10:55:36.849 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.846+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=511409159}) 2024-08-19T10:55:36.851 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=424454986}) 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1828862483}) 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=424454986}) 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3487610527}) 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=424454986}) 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=929586405}) 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=424454986}) 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3541510493}) 2024-08-19T10:55:36.852 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=424454986}) 2024-08-19T10:55:36.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3031662583}) 2024-08-19T10:55:36.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=424454986}) 2024-08-19T10:55:36.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2738963371}) 2024-08-19T10:55:36.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=424454986}) 2024-08-19T10:55:36.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1169336226}) 2024-08-19T10:55:36.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=424454986}) 2024-08-19T10:55:36.853 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.850+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1098431535}) 2024-08-19T10:55:36.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.855 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809980628}) 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2815696310}) 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809980628}) 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2669420151}) 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809980628}) 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1627613480}) 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809980628}) 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3602020581}) 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809980628}) 2024-08-19T10:55:36.856 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=531460677}) 2024-08-19T10:55:36.857 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.857 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809980628}) 2024-08-19T10:55:36.857 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2212450813}) 2024-08-19T10:55:36.857 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.857 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809980628}) 2024-08-19T10:55:36.857 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1503092809}) 2024-08-19T10:55:36.857 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.857 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809980628}) 2024-08-19T10:55:36.857 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.854+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2194444996}) 2024-08-19T10:55:36.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.862+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827434612}) 2024-08-19T10:55:36.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3616564483}) 2024-08-19T10:55:36.864 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827434612}) 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3500451389}) 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827434612}) 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4138633202}) 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827434612}) 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3826559345}) 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827434612}) 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4041264527}) 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827434612}) 2024-08-19T10:55:36.865 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=125181915}) 2024-08-19T10:55:36.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827434612}) 2024-08-19T10:55:36.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=705074424}) 2024-08-19T10:55:36.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827434612}) 2024-08-19T10:55:36.866 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.866+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2573420869}) 2024-08-19T10:55:36.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=749482178}) 2024-08-19T10:55:36.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1524459127}) 2024-08-19T10:55:36.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=749482178}) 2024-08-19T10:55:36.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=535084473}) 2024-08-19T10:55:36.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=749482178}) 2024-08-19T10:55:36.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=354859232}) 2024-08-19T10:55:36.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.869 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=749482178}) 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1876096360}) 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=749482178}) 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2258573713}) 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=749482178}) 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2481555554}) 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=749482178}) 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2857594818}) 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=749482178}) 2024-08-19T10:55:36.870 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:36.870+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3047420432}) 2024-08-19T10:55:37.424 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-08-19T10:55:37.424 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:55:37.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692977025}) 2024-08-19T10:55:37.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1895304844}) 2024-08-19T10:55:37.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692977025}) 2024-08-19T10:55:37.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4283752473}) 2024-08-19T10:55:37.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692977025}) 2024-08-19T10:55:37.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1324549345}) 2024-08-19T10:55:37.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.431 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692977025}) 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2166901048}) 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692977025}) 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=731407191}) 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692977025}) 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2808929043}) 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692977025}) 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2653869953}) 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2692977025}) 2024-08-19T10:55:37.432 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.430+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3216044329}) 2024-08-19T10:55:37.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3331445549}) 2024-08-19T10:55:37.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=915814025}) 2024-08-19T10:55:37.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3331445549}) 2024-08-19T10:55:37.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=724704374}) 2024-08-19T10:55:37.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3331445549}) 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1988751651}) 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3331445549}) 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2448382889}) 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3331445549}) 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3720220220}) 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3331445549}) 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3729595659}) 2024-08-19T10:55:37.458 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.459 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3331445549}) 2024-08-19T10:55:37.459 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=991571738}) 2024-08-19T10:55:37.459 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.459 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3331445549}) 2024-08-19T10:55:37.459 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3576984529}) 2024-08-19T10:55:37.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient: get_monmap_and_config 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient: build_initial_monmap 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient: monmap: 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:37.558558+0000 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:37.558558+0000 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:37.553 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:37.554 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:37.554 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:37.554 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:37.554 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:37.554 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:37.554 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:37.554 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:37.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 1 Processor -- start 2024-08-19T10:55:37.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 1 -- start start 2024-08-19T10:55:37.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient: init 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient: _add_conns ranks=[4,1,2,7,8,10,0,6,3,5,9] 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 0x7fdc9c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient(hunting): picked mon.noname-c con 0x7fdc9c0a3b30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fdc9c0a3f10 addr v1:172.21.15.190:6789/0 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c13b730 0x7fdc9c13db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient(hunting): picked mon.noname-b con 0x7fdc9c13b730 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc9c074170 con 0x7fdc9c0a3f10 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fdc9c0a08b0 con 0x7fdc9c13b730 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fdc9c0a0630 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdca191c640 10 monclient(hunting): authenticate will time out at 1987.375019s 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c13b730 0x7fdc9c13db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9affd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 0x7fdc9c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c13b730 0x7fdc9c13db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60890/0 (socket says 172.21.15.89:60890) 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 1 -- 172.21.15.89:0/443042293 learned_addr learned my addr 172.21.15.89:0/443042293 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 10 monclient(hunting): get_auth_request con 0x7fdc9c13b730 auth_method 0 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9affd640 10 monclient(hunting): get_auth_request con 0x7fdc9c0a3b30 auth_method 0 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:37.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 10 monclient(hunting): handle_auth_done global_id 74558 payload 274 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 1 -- 172.21.15.89:0/443042293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 msgr2=0x7fdc9c0a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 1 --2- 172.21.15.89:0/443042293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 0x7fdc9c0a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 1 -- 172.21.15.89:0/443042293 >> v1:172.21.15.190:6789/0 conn(0x7fdc9c0a3f10 legacy=0x7fdc9c0a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 1 monclient: found mon.noname-b 2024-08-19T10:55:37.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:37.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:55:37.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9b7fe640 1 -- 172.21.15.89:0/443042293 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9c0a1950 con 0x7fdc9c13b730 2024-08-19T10:55:37.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.554+0000 7fdc9affd640 1 --2- 172.21.15.89:0/443042293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 0x7fdc9c0a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:37.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc9b7fe640 1 -- 172.21.15.89:0/443042293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c13b730 msgr2=0x7fdc9c13db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:37.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc9b7fe640 1 --2- 172.21.15.89:0/443042293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c13b730 0x7fdc9c13db20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009870 tx=0x7fdc8c01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19389e84437bfb53 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:37.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc99ffb640 1 -- 172.21.15.89:0/443042293 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc8c0b0070 con 0x7fdc9c13b730 2024-08-19T10:55:37.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc99ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:37.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc99ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:55:37.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc99ffb640 10 monclient: dump: 2024-08-19T10:55:37.566 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:37.566 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:37.566 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:37.566 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:37.566 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc99ffb640 10 monclient: _finish_auth 0 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc99ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:55:37.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc99ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdca191c640 5 monclient: authenticate success, global_id 74558 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc99ffb640 1 -- 172.21.15.89:0/443042293 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8c0aa810 con 0x7fdc9c13b730 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdca191c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc99ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdca191c640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdca191c640 4 set_mon_vals no callback set 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc99ffb640 1 -- 172.21.15.89:0/443042293 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc8c0a9300 con 0x7fdc9c13b730 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.558+0000 7fdc99ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc99ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc99ffb640 10 monclient: dump: 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:37.568 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:37.569 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient: shutdown 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 -- 172.21.15.89:0/443042293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c13b730 msgr2=0x7fdc9c13db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 --2- 172.21.15.89:0/443042293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c13b730 0x7fdc9c13db20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009870 tx=0x7fdc8c01c440 comp rx=0 tx=0).stop 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 -- 172.21.15.89:0/443042293 shutdown_connections 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 --2- 172.21.15.89:0/443042293 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fdc9c13b730 0x7fdc9c13db20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 --2- 172.21.15.89:0/443042293 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 0x7fdc9c0a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 -- 172.21.15.89:0/443042293 >> 172.21.15.89:0/443042293 conn(0x7fdc9c0a8f60 msgr2=0x7fdc9c0a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 -- 172.21.15.89:0/443042293 shutdown_connections 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 -- 172.21.15.89:0/443042293 wait complete. 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient: build_initial_monmap 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient: monmap: 2024-08-19T10:55:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 Processor -- start 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 -- start start 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient: init 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient: _add_conns ranks=[4,5,3,1,7,0,2,8,6] 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 0x7fdc9c06c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient(hunting): picked mon.noname-c con 0x7fdc9c0a3b30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdc9c0a3f10 0x7fdc9c139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient(hunting): picked mon.noname-h con 0x7fdc9c0a3f10 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc9c13b730 0x7fdc9c139e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient(hunting): picked mon.noname-g con 0x7fdc9c13b730 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdc9c140c10 con 0x7fdc9c13b730 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fdc9c140a90 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fdc9c140d90 con 0x7fdc9c0a3f10 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdca191c640 10 monclient(hunting): authenticate will time out at 1987.382102s 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 0x7fdc9c06c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 0x7fdc9c06c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:35202/0 (socket says 172.21.15.89:35202) 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 1 -- 172.21.15.89:0/2082665741 learned_addr learned my addr 172.21.15.89:0/2082665741 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 10 monclient(hunting): get_auth_request con 0x7fdc9c0a3b30 auth_method 0 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9a7fc640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdc9c0a3f10 0x7fdc9c139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9b7fe640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc9c13b730 0x7fdc9c139e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9a7fc640 10 monclient(hunting): get_auth_request con 0x7fdc9c0a3f10 auth_method 0 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9a7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9b7fe640 10 monclient(hunting): get_auth_request con 0x7fdc9c13b730 auth_method 0 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 10 monclient(hunting): handle_auth_done global_id 84388 payload 824 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 1 -- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdc9c0a3f10 msgr2=0x7fdc9c139920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdc9c0a3f10 0x7fdc9c139920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 1 -- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc9c13b730 msgr2=0x7fdc9c139e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc9c13b730 0x7fdc9c139e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 10 monclient: _finish_hunting 0 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 1 monclient: found mon.noname-c 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.562+0000 7fdc9affd640 1 -- 172.21.15.89:0/2082665741 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9c127020 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc9affd640 1 -- 172.21.15.89:0/2082665741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 msgr2=0x7fdc9c06c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc9affd640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 0x7fdc9c06c930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400efc0 tx=0x7fdc9400c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:37.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 1 -- 172.21.15.89:0/2082665741 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc94018020 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 10 monclient: dump: 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:37.576 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 10 monclient: _finish_auth 0 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 10 monclient: _check_auth_tickets 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 1 -- 172.21.15.89:0/2082665741 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc94014b00 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdca191c640 5 monclient: authenticate success, global_id 84388 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdca191c640 10 monclient: _renew_subs 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdca191c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:37.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdca191c640 1 -- 172.21.15.89:0/2082665741 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9c127300 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 1 -- 172.21.15.89:0/2082665741 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fdc94014cb0 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc997fa640 4 set_mon_vals no callback set 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 10 monclient: dump: 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:37.578 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdca191c640 10 monclient: _renew_subs 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdca191c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdca191c640 1 -- 172.21.15.89:0/2082665741 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9c0a7c40 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdca191c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:37.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdca191c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:37.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdca191c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:37.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdca191c640 1 -- 172.21.15.89:0/2082665741 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc9c0a2a60 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 1 -- 172.21.15.89:0/2082665741 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fdc9401b020 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdc7c03be30 0x7fdc7c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:37.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc93fff640 1 -- 172.21.15.89:0/2082665741 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fdc9404fe70 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc9a7fc640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdc7c03be30 0x7fdc7c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:37.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.566+0000 7fdc9a7fc640 10 monclient: get_auth_request con 0x7fdc7c03be30 auth_method 0 2024-08-19T10:55:37.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.570+0000 7fdc9a7fc640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdc7c03be30 0x7fdc7c03e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c06cf60 tx=0x7fdc84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:37.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.570+0000 7fdc93fff640 1 -- 172.21.15.89:0/2082665741 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fdc94010040 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.570+0000 7fdc93fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:37.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.570+0000 7fdc93fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:37.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207151433}) 2024-08-19T10:55:37.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1619879210}) 2024-08-19T10:55:37.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.649 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207151433}) 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3008860028}) 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207151433}) 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3148074313}) 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207151433}) 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4150425772}) 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207151433}) 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4259161055}) 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207151433}) 2024-08-19T10:55:37.650 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2387989701}) 2024-08-19T10:55:37.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207151433}) 2024-08-19T10:55:37.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=215617221}) 2024-08-19T10:55:37.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207151433}) 2024-08-19T10:55:37.651 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=842574154}) 2024-08-19T10:55:37.667 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.667 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488129109}) 2024-08-19T10:55:37.667 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3783321214}) 2024-08-19T10:55:37.667 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.667 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488129109}) 2024-08-19T10:55:37.667 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=355309415}) 2024-08-19T10:55:37.667 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488129109}) 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3109780461}) 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488129109}) 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2069219404}) 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488129109}) 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=790393092}) 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488129109}) 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4075737495}) 2024-08-19T10:55:37.668 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488129109}) 2024-08-19T10:55:37.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1639675704}) 2024-08-19T10:55:37.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488129109}) 2024-08-19T10:55:37.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1984706749}) 2024-08-19T10:55:37.675 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.674+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.675 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.674+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935864678}) 2024-08-19T10:55:37.675 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.674+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2937129396}) 2024-08-19T10:55:37.675 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.674+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.675 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.674+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935864678}) 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.674+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3177212778}) 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.674+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935864678}) 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1986252021}) 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935864678}) 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3882875677}) 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935864678}) 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4022773467}) 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935864678}) 2024-08-19T10:55:37.676 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2637565006}) 2024-08-19T10:55:37.677 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.677 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935864678}) 2024-08-19T10:55:37.677 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1774502339}) 2024-08-19T10:55:37.677 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.677 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1935864678}) 2024-08-19T10:55:37.677 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.678+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1370701920}) 2024-08-19T10:55:37.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=141433703}) 2024-08-19T10:55:37.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3798780707}) 2024-08-19T10:55:37.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=141433703}) 2024-08-19T10:55:37.685 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1810204177}) 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=141433703}) 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1456018669}) 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=141433703}) 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3929776064}) 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=141433703}) 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=89831734}) 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=141433703}) 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3321015914}) 2024-08-19T10:55:37.686 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.687 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=141433703}) 2024-08-19T10:55:37.687 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2656180260}) 2024-08-19T10:55:37.687 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.687 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=141433703}) 2024-08-19T10:55:37.687 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.686+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2298980241}) 2024-08-19T10:55:37.720 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.720 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=363965594}) 2024-08-19T10:55:37.720 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3044555111}) 2024-08-19T10:55:37.720 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.720 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=363965594}) 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2925798155}) 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=363965594}) 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1236737922}) 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=363965594}) 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=84177579}) 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=363965594}) 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1172822224}) 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.721 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=363965594}) 2024-08-19T10:55:37.722 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4049084202}) 2024-08-19T10:55:37.722 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.722 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=363965594}) 2024-08-19T10:55:37.722 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3642648631}) 2024-08-19T10:55:37.722 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:37.722 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=363965594}) 2024-08-19T10:55:37.722 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:37.722+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=963786540}) 2024-08-19T10:55:37.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.814+0000 7fdca191c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:55:37.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.814+0000 7fdca191c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.814+0000 7fdca191c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.814+0000 7fdca191c640 1 -- 172.21.15.89:0/2082665741 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdc9c064590 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdc93fff640 1 -- 172.21.15.89:0/2082665741 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7fdc94012340 con 0x7fdc9c0a3b30 2024-08-19T10:55:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdc93fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:37.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdc93fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:37.815 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":140,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:55:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 -- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdc7c03be30 msgr2=0x7fdc7c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdc7c03be30 0x7fdc7c03e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c06cf60 tx=0x7fdc84009040 comp rx=0 tx=0).stop 2024-08-19T10:55:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 10 monclient: shutdown 2024-08-19T10:55:37.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 -- 172.21.15.89:0/2082665741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 msgr2=0x7fdc9c06c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 0x7fdc9c06c930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400efc0 tx=0x7fdc9400c490 comp rx=0 tx=0).stop 2024-08-19T10:55:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 -- 172.21.15.89:0/2082665741 shutdown_connections 2024-08-19T10:55:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fdc7c03be30 0x7fdc7c03e2f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc9c13b730 0x7fdc9c139e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fdc9c0a3f10 0x7fdc9c139920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 --2- 172.21.15.89:0/2082665741 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fdc9c0a3b30 0x7fdc9c06c930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 -- 172.21.15.89:0/2082665741 >> 172.21.15.89:0/2082665741 conn(0x7fdc9c0a8f60 msgr2=0x7fdc9c13c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 -- 172.21.15.89:0/2082665741 shutdown_connections 2024-08-19T10:55:37.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:37.818+0000 7fdca191c640 1 -- 172.21.15.89:0/2082665741 wait complete. 2024-08-19T10:55:37.834 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-08-19T10:55:37.834 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-08-19T10:55:37.834 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-08-19T10:55:37.834 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:55:38.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient: get_monmap_and_config 2024-08-19T10:55:38.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient: build_initial_monmap 2024-08-19T10:55:38.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient: monmap: 2024-08-19T10:55:38.000 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:38.000 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.000 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:38.005979+0000 2024-08-19T10:55:38.000 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:38.005979+0000 2024-08-19T10:55:38.000 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:38.001 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 1 Processor -- start 2024-08-19T10:55:38.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 1 -- start start 2024-08-19T10:55:38.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient: init 2024-08-19T10:55:38.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:38.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient: _add_conns ranks=[1,9,3,0,7,2,10,6,4,8,5] 2024-08-19T10:55:38.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f24b40a3d00 addr v1:172.21.15.190:6789/0 2024-08-19T10:55:38.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f24b40a47f0 0x7f24b413c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient(hunting): picked mon.noname-a con 0x7f24b40a47f0 addr v2:172.21.15.89:3300/0 2024-08-19T10:55:38.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b413cde0 0x7f24b413f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient(hunting): picked mon.noname-g con 0x7f24b413cde0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:38.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24b409ca30 con 0x7f24b40a3d00 2024-08-19T10:55:38.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f24b406db50 con 0x7f24b40a47f0 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f24b40a1e50 con 0x7f24b413cde0 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bc7a0640 10 monclient(hunting): authenticate will time out at 1987.822564s 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f24b40a47f0 0x7f24b413c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f24b40a47f0 0x7f24b413c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49286/0 (socket says 172.21.15.89:49286) 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 1 -- 172.21.15.89:0/4132993348 learned_addr learned my addr 172.21.15.89:0/4132993348 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 10 monclient(hunting): get_auth_request con 0x7f24b40a47f0 auth_method 0 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bad3d640 1 --2- 172.21.15.89:0/4132993348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b413cde0 0x7f24b413f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bad3d640 10 monclient(hunting): get_auth_request con 0x7f24b413cde0 auth_method 0 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bad3d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bad3d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bad3d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bad3d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bad3d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24bad3d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 10 monclient(hunting): handle_auth_done global_id 7650 payload 274 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 1 -- 172.21.15.89:0/4132993348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b413cde0 msgr2=0x7f24b413f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 1 --2- 172.21.15.89:0/4132993348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b413cde0 0x7f24b413f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 1 -- 172.21.15.89:0/4132993348 >> v1:172.21.15.190:6789/0 conn(0x7f24b40a3d00 legacy=0x7f24b40a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 10 monclient: _finish_hunting 0 2024-08-19T10:55:38.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 1 monclient: found mon.noname-a 2024-08-19T10:55:38.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:38.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:55:38.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.002+0000 7f24b9d3b640 1 -- 172.21.15.89:0/4132993348 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b40a27d0 con 0x7f24b40a47f0 2024-08-19T10:55:38.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b9d3b640 1 -- 172.21.15.89:0/4132993348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b40a47f0 msgr2=0x7f24b413c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b9d3b640 1 --2- 172.21.15.89:0/4132993348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b40a47f0 0x7f24b413c8a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009e50 tx=0x7f24a801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4fac5df43d5275 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 1 -- 172.21.15.89:0/4132993348 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24a80090d0 con 0x7f24b40a47f0 2024-08-19T10:55:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:55:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 10 monclient: dump: 2024-08-19T10:55:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.012 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 10 monclient: _finish_auth 0 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 10 monclient: _check_auth_tickets 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:38.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 5 monclient: authenticate success, global_id 7650 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 1 -- 172.21.15.89:0/4132993348 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a80a5ab0 con 0x7f24b40a47f0 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 4 set_mon_vals no callback set 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 1 -- 172.21.15.89:0/4132993348 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24a80a5c60 con 0x7f24b40a47f0 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b953a640 10 monclient: dump: 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.014 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.015 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient: shutdown 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 -- 172.21.15.89:0/4132993348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b40a47f0 msgr2=0x7f24b413c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 --2- 172.21.15.89:0/4132993348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b40a47f0 0x7f24b413c8a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009e50 tx=0x7f24a801c8f0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 -- 172.21.15.89:0/4132993348 shutdown_connections 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 --2- 172.21.15.89:0/4132993348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24b413cde0 0x7f24b413f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 --2- 172.21.15.89:0/4132993348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b40a47f0 0x7f24b413c8a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 -- 172.21.15.89:0/4132993348 >> 172.21.15.89:0/4132993348 conn(0x7f24b40a4df0 msgr2=0x7f24b40a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 -- 172.21.15.89:0/4132993348 shutdown_connections 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 -- 172.21.15.89:0/4132993348 wait complete. 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient: build_initial_monmap 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient: monmap: 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:38.016 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:38.017 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:38.018 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 Processor -- start 2024-08-19T10:55:38.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 -- start start 2024-08-19T10:55:38.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient: init 2024-08-19T10:55:38.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:38.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient: _add_conns ranks=[5,6,1,8,3,0,2,4,7] 2024-08-19T10:55:38.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f24b40a3d00 0x7f24b41cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient(hunting): picked mon.noname-h con 0x7f24b40a3d00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:38.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f24b40a47f0 0x7f24b41cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient(hunting): picked mon.noname-d con 0x7f24b40a47f0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:38.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b413cde0 0x7f24b41cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient(hunting): picked mon.noname-f con 0x7f24b413cde0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24b40a1b80 con 0x7f24b413cde0 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f24b40a1a00 con 0x7f24b40a3d00 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f24b4141f30 con 0x7f24b40a47f0 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24bc7a0640 10 monclient(hunting): authenticate will time out at 1987.831711s 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b9d3b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f24b40a47f0 0x7f24b41cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b9d3b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f24b40a47f0 0x7f24b41cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36480/0 (socket says 172.21.15.89:36480) 2024-08-19T10:55:38.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b9d3b640 1 -- 172.21.15.89:0/2220091248 learned_addr learned my addr 172.21.15.89:0/2220091248 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b9d3b640 10 monclient(hunting): get_auth_request con 0x7f24b40a47f0 auth_method 0 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b9d3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b9d3b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b9d3b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b9d3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.010+0000 7f24b9d3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f24b40a3d00 0x7f24b41cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bad3d640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b413cde0 0x7f24b41cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 10 monclient(hunting): get_auth_request con 0x7f24b40a3d00 auth_method 0 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 10 monclient(hunting): handle_auth_done global_id 45411 payload 824 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 1 -- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f24b40a3d00 msgr2=0x7f24b41cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f24b40a3d00 0x7f24b41cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 1 -- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b413cde0 msgr2=0x7f24b41cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b413cde0 0x7f24b41cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 10 monclient: _finish_hunting 0 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 1 monclient: found mon.noname-d 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 1 -- 172.21.15.89:0/2220091248 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b412fbc0 con 0x7f24b40a47f0 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bad3d640 10 monclient: get_auth_request con 0x7f24b413cde0 auth_method 0 2024-08-19T10:55:38.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bad3d640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24b413cde0 0x7f24b41cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:55:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 1 -- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f24b40a47f0 msgr2=0x7f24b41cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b9d3b640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f24b40a47f0 0x7f24b41cf460 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f24a801c720 tx=0x7f24a8019690 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24a80bb020 con 0x7f24b40a47f0 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 10 monclient: dump: 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.026 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a8003dc0 con 0x7f24b40a47f0 2024-08-19T10:55:38.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bc7a0640 5 monclient: authenticate success, global_id 45411 2024-08-19T10:55:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bc7a0640 10 monclient: _renew_subs 2024-08-19T10:55:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bc7a0640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:55:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bc7a0640 1 -- 172.21.15.89:0/2220091248 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b40a36d0 con 0x7f24b40a47f0 2024-08-19T10:55:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24a80b6440 con 0x7f24b40a47f0 2024-08-19T10:55:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 10 monclient: dump: 2024-08-19T10:55:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.028 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.029 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bc7a0640 10 monclient: _renew_subs 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b8d39640 4 set_mon_vals no callback set 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bc7a0640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bc7a0640 1 -- 172.21.15.89:0/2220091248 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b412f8e0 con 0x7f24b40a47f0 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bc7a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bc7a0640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138670 0x7f24b4138a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bc7a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24bc7a0640 1 -- 172.21.15.89:0/2220091248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24b413cae0 con 0x7f24b4138670 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138670 0x7f24b4138a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 10 monclient: get_auth_request con 0x7f24b4138670 auth_method 0 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f24a80be020 con 0x7f24b40a47f0 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.014+0000 7f24ba53c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24ba53c640 10 monclient(hunting): handle_auth_done global_id 7659 payload 824 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24ba53c640 1 -- 172.21.15.89:0/2220091248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24b40a40e0 con 0x7f24b4138670 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f24a003be00 0x7f24a003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f24a80ea2c0 con 0x7f24b40a47f0 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24bad3d640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f24a003be00 0x7f24a003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24bad3d640 10 monclient: get_auth_request con 0x7f24a003be00 auth_method 0 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24ba53c640 1 -- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138670 msgr2=0x7f24b4138a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24ba53c640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138670 0x7f24b4138a50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f24a400e450 tx=0x7f24a400e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24a4006c50 con 0x7f24b4138670 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 10 monclient: dump: 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.032 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f24a4007450 con 0x7f24b4138670 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138670 msgr2=0x7f24b4138a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138670 0x7f24b4138a50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f24a400e450 tx=0x7f24a400e910 comp rx=0 tx=0).stop 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138670 msgr2=0x7f24b4138a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:38.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24b2ffd640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138670 0x7f24b4138a50 secure :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f24a400e450 tx=0x7f24a400e910 comp rx=0 tx=0).stop 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.018+0000 7f24bad3d640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f24a003be00 0x7f24a003e2c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f24b413c790 tx=0x7f24ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24bc7a0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24bc7a0640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138f90 0x7f24b412d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24bc7a0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24bc7a0640 1 -- 172.21.15.89:0/2220091248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4064590 con 0x7f24b4138f90 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138f90 0x7f24b412d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 10 monclient: get_auth_request con 0x7f24b4138f90 auth_method 0 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 10 monclient(hunting): handle_auth_done global_id 7668 payload 824 2024-08-19T10:55:38.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 1 -- 172.21.15.89:0/2220091248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f24b406e580 con 0x7f24b4138f90 2024-08-19T10:55:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 1 -- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138f90 msgr2=0x7f24b412d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24ba53c640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138f90 0x7f24b412d3b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000c10 tx=0x7f24a4003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f24a4013890 con 0x7f24b4138f90 2024-08-19T10:55:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24b2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.026+0000 7f24b2ffd640 10 monclient: dump: 2024-08-19T10:55:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.036 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9438 (secure 0 0 0) 0x7f24a4013420 con 0x7f24b4138f90 2024-08-19T10:55:38.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24b2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:38.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24b2ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:38.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138f90 msgr2=0x7f24b412d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24b2ffd640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138f90 0x7f24b412d3b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000c10 tx=0x7f24a4003770 comp rx=0 tx=0).stop 2024-08-19T10:55:38.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24b2ffd640 1 -- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138f90 msgr2=0x7f24b412d3b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:38.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24b2ffd640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24b4138f90 0x7f24b412d3b0 secure :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000c10 tx=0x7f24a4003770 comp rx=0 tx=0).stop 2024-08-19T10:55:38.038 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:38.039 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:38.039 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:38.039 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:55:38.039 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:38.039 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 1067839, 2024-08-19T10:55:38.039 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:38.039 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:38.039 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:38.039 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:38.039 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.040 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:38.041 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.042 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:38.043 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.044 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.045 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:38.046 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.046 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.046 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.046 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:38.046 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.046 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.046 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.046 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:38.046 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:38.052 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.053 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.054 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:38.055 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.059 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.060 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.061 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.062 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:38.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.063 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.063 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.063 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:55:38.063 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.063 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.063 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:38.063 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:38.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bc7a0640 1 -- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f24a003be00 msgr2=0x7f24a003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bc7a0640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f24a003be00 0x7f24a003e2c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f24b413c790 tx=0x7f24ac008040 comp rx=0 tx=0).stop 2024-08-19T10:55:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bc7a0640 10 monclient: shutdown 2024-08-19T10:55:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bc7a0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bc7a0640 1 -- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f24b40a47f0 msgr2=0x7f24b41cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bc7a0640 1 --2- 172.21.15.89:0/2220091248 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f24b40a47f0 0x7f24b41cf460 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f24a801c720 tx=0x7f24a8019690 comp rx=0 tx=0).stop 2024-08-19T10:55:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bad3d640 1 -- 172.21.15.89:0/2220091248 reap_dead start 2024-08-19T10:55:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bad3d640 1 -- 172.21.15.89:0/2220091248 reap_dead start 2024-08-19T10:55:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bc7a0640 1 -- 172.21.15.89:0/2220091248 shutdown_connections 2024-08-19T10:55:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bc7a0640 1 -- 172.21.15.89:0/2220091248 >> 172.21.15.89:0/2220091248 conn(0x7f24b40a4df0 msgr2=0x7f24b413d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bc7a0640 1 -- 172.21.15.89:0/2220091248 shutdown_connections 2024-08-19T10:55:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.030+0000 7f24bc7a0640 1 -- 172.21.15.89:0/2220091248 wait complete. 2024-08-19T10:55:38.066 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:38.066 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-08-19T10:55:38.066 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-08-19T10:55:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.182+0000 7f40d464f640 10 monclient: get_monmap_and_config 2024-08-19T10:55:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.182+0000 7f40d464f640 10 monclient: build_initial_monmap 2024-08-19T10:55:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.182+0000 7f40d464f640 10 monclient: monmap: 2024-08-19T10:55:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:38.188797+0000 2024-08-19T10:55:38.183 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:38.188797+0000 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:38.184 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:38.185 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 1 Processor -- start 2024-08-19T10:55:38.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 1 -- start start 2024-08-19T10:55:38.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 10 monclient: init 2024-08-19T10:55:38.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 10 monclient: _add_conns ranks=[7,6,8,5,10,2,0,4,1,9,3] 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f40cc0a6860 0x7f40cc0a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 10 monclient(hunting): picked mon.noname-i con 0x7f40cc0a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a7180 0x7f40cc0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 10 monclient(hunting): picked mon.noname-d con 0x7f40cc0a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc139620 0x7f40cc139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 10 monclient(hunting): picked mon.noname-e con 0x7f40cc139620 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f40cc0a8190 con 0x7f40cc0a7180 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f40cc0a33d0 con 0x7f40cc0a6860 2024-08-19T10:55:38.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f40cc0a2ec0 con 0x7f40cc139620 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d464f640 10 monclient(hunting): authenticate will time out at 1988.005688s 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d2bec640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc139620 0x7f40cc139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a7180 0x7f40cc0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d2bec640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc139620 0x7f40cc139a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:59890/0 (socket says 172.21.15.89:59890) 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d2bec640 1 -- 172.21.15.89:0/3832094266 learned_addr learned my addr 172.21.15.89:0/3832094266 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d2bec640 10 monclient(hunting): get_auth_request con 0x7f40cc139620 auth_method 0 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d2bec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d2bec640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d2bec640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 10 monclient(hunting): get_auth_request con 0x7f40cc0a7180 auth_method 0 2024-08-19T10:55:38.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d2bec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d2bec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d2bec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d23eb640 1 --2- 172.21.15.89:0/3832094266 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f40cc0a6860 0x7f40cc0a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d23eb640 10 monclient(hunting): get_auth_request con 0x7f40cc0a6860 auth_method 0 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d23eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d23eb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d23eb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 10 monclient(hunting): handle_auth_done global_id 45420 payload 274 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 1 -- 172.21.15.89:0/3832094266 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc139620 msgr2=0x7f40cc139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 1 --2- 172.21.15.89:0/3832094266 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc139620 0x7f40cc139a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 1 -- 172.21.15.89:0/3832094266 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f40cc0a6860 msgr2=0x7f40cc0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 1 --2- 172.21.15.89:0/3832094266 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f40cc0a6860 0x7f40cc0a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 10 monclient: _finish_hunting 0 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 1 monclient: found mon.noname-d 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d1bea640 1 -- 172.21.15.89:0/3832094266 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc0a4cd0 con 0x7f40cc0a7180 2024-08-19T10:55:38.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.186+0000 7f40d2bec640 1 --2- 172.21.15.89:0/3832094266 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc139620 0x7f40cc139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:38.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d1bea640 1 -- 172.21.15.89:0/3832094266 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a7180 msgr2=0x7f40cc0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d1bea640 1 --2- 172.21.15.89:0/3832094266 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a7180 0x7f40cc0a7560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009dc0 tx=0x7f40c401c850 comp rx=0 tx=0).ready entity=mon.6 client_cookie=1a3a30575b8f1c59 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 1 -- 172.21.15.89:0/3832094266 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40c40ac070 con 0x7f40cc0a7180 2024-08-19T10:55:38.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:55:38.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 10 monclient: dump: 2024-08-19T10:55:38.197 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.198 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 10 monclient: _finish_auth 0 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 10 monclient: _check_auth_tickets 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 1 -- 172.21.15.89:0/3832094266 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c40a68b0 con 0x7f40cc0a7180 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 1 -- 172.21.15.89:0/3832094266 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40c40a7420 con 0x7f40cc0a7180 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 5 monclient: authenticate success, global_id 45420 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 4 set_mon_vals no callback set 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:38.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d13e9640 10 monclient: dump: 2024-08-19T10:55:38.200 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.200 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.200 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.200 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.200 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.200 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.200 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.200 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.200 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.200 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 10 monclient: shutdown 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 1 -- 172.21.15.89:0/3832094266 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a7180 msgr2=0x7f40cc0a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 1 --2- 172.21.15.89:0/3832094266 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a7180 0x7f40cc0a7560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009dc0 tx=0x7f40c401c850 comp rx=0 tx=0).stop 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 1 -- 172.21.15.89:0/3832094266 shutdown_connections 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 1 --2- 172.21.15.89:0/3832094266 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc139620 0x7f40cc139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 1 --2- 172.21.15.89:0/3832094266 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f40cc0a7180 0x7f40cc0a7560 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 1 --2- 172.21.15.89:0/3832094266 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f40cc0a6860 0x7f40cc0a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 1 -- 172.21.15.89:0/3832094266 >> 172.21.15.89:0/3832094266 conn(0x7f40cc074b40 msgr2=0x7f40cc0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 1 -- 172.21.15.89:0/3832094266 shutdown_connections 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 1 -- 172.21.15.89:0/3832094266 wait complete. 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 10 monclient: build_initial_monmap 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.190+0000 7f40d464f640 10 monclient: monmap: 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:38.202 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.203 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:38.203 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:38.203 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:38.203 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:38.203 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:38.203 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:38.203 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:38.203 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:38.203 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:38.203 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 1 Processor -- start 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 1 -- start start 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient: init 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient: _add_conns ranks=[8,2,4,3,5,7,0,6,1] 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc0a6860 0x7f40cc1348c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient(hunting): picked mon.noname-e con 0x7f40cc0a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc0a7180 0x7f40cc134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient(hunting): picked mon.noname-b con 0x7f40cc0a7180 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40cc139620 0x7f40cc09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient(hunting): picked mon.noname-c con 0x7f40cc139620 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc143210 con 0x7f40cc0a7180 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f40cc143090 con 0x7f40cc139620 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f40cc143390 con 0x7f40cc0a6860 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient(hunting): authenticate will time out at 1988.012859s 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc0a6860 0x7f40cc1348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc0a6860 0x7f40cc1348c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:59902/0 (socket says 172.21.15.89:59902) 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d2bec640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40cc139620 0x7f40cc09c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 1 -- 172.21.15.89:0/3216558533 learned_addr learned my addr 172.21.15.89:0/3216558533 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 10 monclient(hunting): get_auth_request con 0x7f40cc0a6860 auth_method 0 2024-08-19T10:55:38.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d1bea640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc0a7180 0x7f40cc134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d2bec640 10 monclient(hunting): get_auth_request con 0x7f40cc139620 auth_method 0 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d2bec640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d2bec640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d2bec640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d1bea640 10 monclient(hunting): get_auth_request con 0x7f40cc0a7180 auth_method 0 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d1bea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d1bea640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d1bea640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d2bec640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d2bec640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d2bec640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 10 monclient(hunting): handle_auth_done global_id 114173 payload 824 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 1 -- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40cc139620 msgr2=0x7f40cc09c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f40cc139620 0x7f40cc09c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 1 -- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc0a7180 msgr2=0x7f40cc134e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc0a7180 0x7f40cc134e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 10 monclient: _finish_hunting 0 2024-08-19T10:55:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 1 monclient: found mon.noname-e 2024-08-19T10:55:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:55:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 1 -- 172.21.15.89:0/3216558533 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc0a0380 con 0x7f40cc0a6860 2024-08-19T10:55:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 1 -- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc0a6860 msgr2=0x7f40cc1348c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d23eb640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc0a6860 0x7f40cc1348c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f40c800d830 tx=0x7f40c800dcf0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40c800bd40 con 0x7f40cc0a6860 2024-08-19T10:55:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:55:38.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 10 monclient: dump: 2024-08-19T10:55:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.209 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.210 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.210 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.210 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.210 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.210 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.210 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.210 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.210 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.210 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.210 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 10 monclient: _finish_auth 0 2024-08-19T10:55:38.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:55:38.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:38.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 5 monclient: authenticate success, global_id 114173 2024-08-19T10:55:38.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient: _renew_subs 2024-08-19T10:55:38.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c8019070 con 0x7f40cc0a6860 2024-08-19T10:55:38.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:55:38.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 1 -- 172.21.15.89:0/3216558533 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc0a8f60 con 0x7f40cc0a6860 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40c80148a0 con 0x7f40cc0a6860 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d0be8640 4 set_mon_vals no callback set 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40c2ffd640 10 monclient: dump: 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.212 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.213 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.213 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.213 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.213 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.213 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.213 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.213 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.213 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.213 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.194+0000 7f40d464f640 10 monclient: _renew_subs 2024-08-19T10:55:38.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d464f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:55:38.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d464f640 1 -- 172.21.15.89:0/3216558533 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc09fae0 con 0x7f40cc0a6860 2024-08-19T10:55:38.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d464f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:38.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d464f640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09cf20 0x7f40cc09d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d464f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d464f640 1 -- 172.21.15.89:0/3216558533 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc0a0db0 con 0x7f40cc09cf20 2024-08-19T10:55:38.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d23eb640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc0a6860 0x7f40cc1348c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f40c800d830 tx=0x7f40c800dcf0 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f40c8014430 mgrmap(e 5) 2024-08-19T10:55:38.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09cf20 0x7f40cc09d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 10 monclient: get_auth_request con 0x7f40cc09cf20 auth_method 0 2024-08-19T10:55:38.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f40c8014430 con 0x7f40cc0a6860 2024-08-19T10:55:38.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 10 monclient(hunting): handle_auth_done global_id 74576 payload 824 2024-08-19T10:55:38.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f40b803be00 0x7f40b803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 1 -- 172.21.15.89:0/3216558533 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40cc0a7560 con 0x7f40cc09cf20 2024-08-19T10:55:38.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d2bec640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f40b803be00 0x7f40b803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d2bec640 10 monclient: get_auth_request con 0x7f40b803be00 auth_method 0 2024-08-19T10:55:38.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 1 -- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09cf20 msgr2=0x7f40cc09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40d1bea640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09cf20 0x7f40cc09d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f40c40ae000 tx=0x7f40c40195f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40c40ac050 con 0x7f40cc09cf20 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 10 monclient: dump: 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.219 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f40c4004250 con 0x7f40cc09cf20 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09cf20 msgr2=0x7f40cc09d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09cf20 0x7f40cc09d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f40c40ae000 tx=0x7f40c40195f0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09cf20 msgr2=0x7f40cc09d300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:38.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.198+0000 7f40c2ffd640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09cf20 0x7f40cc09d300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.202+0000 7f40d2bec640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f40b803be00 0x7f40b803e2c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0096b0 tx=0x7f40bc009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d464f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d464f640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09d840 0x7f40cc13a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d464f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d464f640 1 -- 172.21.15.89:0/3216558533 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc09ff80 con 0x7f40cc09d840 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d1bea640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09d840 0x7f40cc13a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d1bea640 10 monclient: get_auth_request con 0x7f40cc09d840 auth_method 0 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d1bea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d1bea640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d1bea640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d1bea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d1bea640 10 monclient(hunting): handle_auth_done global_id 74585 payload 824 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.206+0000 7f40d1bea640 1 -- 172.21.15.89:0/3216558533 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f40cc064590 con 0x7f40cc09d840 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d1bea640 1 -- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09d840 msgr2=0x7f40cc13a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d1bea640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09d840 0x7f40cc13a970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f40cc13a8e0 tx=0x7f40c40a9000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f40c40b1ad0 con 0x7f40cc09d840 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 10 monclient: dump: 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.222 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7f40c40a6440 con 0x7f40cc09d840 2024-08-19T10:55:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09d840 msgr2=0x7f40cc13a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09d840 0x7f40cc13a970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f40cc13a8e0 tx=0x7f40c40a9000 comp rx=0 tx=0).stop 2024-08-19T10:55:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09d840 msgr2=0x7f40cc13a970 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:38.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f40cc09d840 0x7f40cc13a970 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f40cc13a8e0 tx=0x7f40c40a9000 comp rx=0 tx=0).stop 2024-08-19T10:55:38.224 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:38.224 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:38.224 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:38.224 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 274709, 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:38.225 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.226 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.227 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:38.228 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.229 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.230 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.231 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.232 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.236 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.237 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.237 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:38.237 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.237 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.237 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.237 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:38.237 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.237 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.237 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.238 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:38.239 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.240 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.241 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.243 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.245 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.246 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.247 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:38.248 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d464f640 1 -- 172.21.15.89:0/3216558533 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f40b803be00 msgr2=0x7f40b803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d464f640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f40b803be00 0x7f40b803e2c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0096b0 tx=0x7f40bc009240 comp rx=0 tx=0).stop 2024-08-19T10:55:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d464f640 10 monclient: shutdown 2024-08-19T10:55:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d464f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d464f640 1 -- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc0a6860 msgr2=0x7f40cc1348c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d464f640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc0a6860 0x7f40cc1348c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f40c800d830 tx=0x7f40c800dcf0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d2bec640 1 -- 172.21.15.89:0/3216558533 reap_dead start 2024-08-19T10:55:38.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40c2ffd640 1 -- 172.21.15.89:0/3216558533 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (unknown 0 0 0) 0x7f40c800f040 con 0x7f40cc0a6860 2024-08-19T10:55:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d464f640 1 -- 172.21.15.89:0/3216558533 shutdown_connections 2024-08-19T10:55:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d464f640 1 --2- 172.21.15.89:0/3216558533 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f40cc0a6860 0x7f40cc1348c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d464f640 1 -- 172.21.15.89:0/3216558533 >> 172.21.15.89:0/3216558533 conn(0x7f40cc074b40 msgr2=0x7f40cc0a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d464f640 1 -- 172.21.15.89:0/3216558533 shutdown_connections 2024-08-19T10:55:38.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.210+0000 7f40d464f640 1 -- 172.21.15.89:0/3216558533 wait complete. 2024-08-19T10:55:38.251 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:38.251 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-08-19T10:55:38.252 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:55:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient: get_monmap_and_config 2024-08-19T10:55:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient: build_initial_monmap 2024-08-19T10:55:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient: monmap: 2024-08-19T10:55:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.367 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:38.373090+0000 2024-08-19T10:55:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:38.373090+0000 2024-08-19T10:55:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:38.368 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:38.369 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:38.369 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 1 Processor -- start 2024-08-19T10:55:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 1 -- start start 2024-08-19T10:55:38.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient: init 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient: _add_conns ranks=[8,7,2,10,4,5,0,9,6,1,3] 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8400a6860 0x7ff8400a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient(hunting): picked mon.noname-e con 0x7ff8400a6860 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8400a7180 0x7ff8400a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient(hunting): picked mon.noname-i con 0x7ff8400a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff840139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient(hunting): picked mon.noname-b con 0x7ff840139620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff8400a8190 con 0x7ff840139620 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff8400a33d0 con 0x7ff8400a7180 2024-08-19T10:55:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff8400a2ec0 con 0x7ff8400a6860 2024-08-19T10:55:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff8452ea640 10 monclient(hunting): authenticate will time out at 1988.189693s 2024-08-19T10:55:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff840139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83effd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8400a6860 0x7ff8400a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff840139a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60912/0 (socket says 172.21.15.89:60912) 2024-08-19T10:55:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 1 -- 172.21.15.89:0/2018439498 learned_addr learned my addr 172.21.15.89:0/2018439498 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 10 monclient(hunting): get_auth_request con 0x7ff840139620 auth_method 0 2024-08-19T10:55:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83effd640 10 monclient(hunting): get_auth_request con 0x7ff8400a6860 auth_method 0 2024-08-19T10:55:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83e7fc640 1 --2- 172.21.15.89:0/2018439498 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8400a7180 0x7ff8400a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83e7fc640 10 monclient(hunting): get_auth_request con 0x7ff8400a7180 auth_method 0 2024-08-19T10:55:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 10 monclient(hunting): handle_auth_done global_id 74594 payload 274 2024-08-19T10:55:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 1 -- 172.21.15.89:0/2018439498 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8400a6860 msgr2=0x7ff8400a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 1 --2- 172.21.15.89:0/2018439498 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8400a6860 0x7ff8400a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 1 -- 172.21.15.89:0/2018439498 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8400a7180 msgr2=0x7ff8400a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 1 --2- 172.21.15.89:0/2018439498 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8400a7180 0x7ff8400a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:38.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 1 monclient: found mon.noname-b 2024-08-19T10:55:38.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:38.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:55:38.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83f7fe640 1 -- 172.21.15.89:0/2018439498 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8400a4cd0 con 0x7ff840139620 2024-08-19T10:55:38.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.370+0000 7ff83effd640 1 --2- 172.21.15.89:0/2018439498 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8400a6860 0x7ff8400a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83f7fe640 1 -- 172.21.15.89:0/2018439498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 msgr2=0x7ff840139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83f7fe640 1 --2- 172.21.15.89:0/2018439498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff840139a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff838009e80 tx=0x7ff83801c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d7a4845188cbbe server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 1 -- 172.21.15.89:0/2018439498 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8380090d0 con 0x7ff840139620 2024-08-19T10:55:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:55:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 10 monclient: dump: 2024-08-19T10:55:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.382 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.383 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 10 monclient: _finish_auth 0 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 10 monclient: _check_auth_tickets 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 1 -- 172.21.15.89:0/2018439498 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8380a6970 con 0x7ff840139620 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 1 -- 172.21.15.89:0/2018439498 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8380af480 con 0x7ff840139620 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 5 monclient: authenticate success, global_id 74594 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 4 set_mon_vals no callback set 2024-08-19T10:55:38.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff83dffb640 10 monclient: dump: 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.385 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 10 monclient: shutdown 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 1 -- 172.21.15.89:0/2018439498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 msgr2=0x7ff840139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 1 --2- 172.21.15.89:0/2018439498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff840139a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff838009e80 tx=0x7ff83801c910 comp rx=0 tx=0).stop 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 1 -- 172.21.15.89:0/2018439498 shutdown_connections 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 1 --2- 172.21.15.89:0/2018439498 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff840139a60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 1 --2- 172.21.15.89:0/2018439498 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff8400a7180 0x7ff8400a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 1 --2- 172.21.15.89:0/2018439498 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff8400a6860 0x7ff8400a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 1 -- 172.21.15.89:0/2018439498 >> 172.21.15.89:0/2018439498 conn(0x7ff840074b40 msgr2=0x7ff8400abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 1 -- 172.21.15.89:0/2018439498 shutdown_connections 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.374+0000 7ff8452ea640 1 -- 172.21.15.89:0/2018439498 wait complete. 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient: build_initial_monmap 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient: monmap: 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 1 Processor -- start 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 1 -- start start 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient: init 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient: _add_conns ranks=[5,6,2,4,0,8,7,3,1] 2024-08-19T10:55:38.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff8400a6860 0x7ff840134930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient(hunting): picked mon.noname-h con 0x7ff8400a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff8400a7180 0x7ff840134e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient(hunting): picked mon.noname-d con 0x7ff8400a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff84009c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient(hunting): picked mon.noname-b con 0x7ff840139620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff840143210 con 0x7ff840139620 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff840143090 con 0x7ff8400a6860 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff840143390 con 0x7ff8400a7180 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:38.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8452ea640 10 monclient(hunting): authenticate will time out at 1988.197009s 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83e7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff8400a7180 0x7ff840134e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff84009c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83e7fc640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff8400a7180 0x7ff840134e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36510/0 (socket says 172.21.15.89:36510) 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff84009c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60926/0 (socket says 172.21.15.89:60926) 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83e7fc640 1 -- 172.21.15.89:0/518739935 learned_addr learned my addr 172.21.15.89:0/518739935 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83e7fc640 10 monclient(hunting): get_auth_request con 0x7ff8400a7180 auth_method 0 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 10 monclient(hunting): get_auth_request con 0x7ff840139620 auth_method 0 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 10 monclient(hunting): handle_auth_done global_id 74603 payload 824 2024-08-19T10:55:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83effd640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff8400a6860 0x7ff840134930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 1 -- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff8400a7180 msgr2=0x7ff840134e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff8400a7180 0x7ff840134e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 1 -- 172.21.15.89:0/518739935 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff8400a6860 msgr2=0x7ff840134930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff8400a6860 0x7ff840134930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 1 monclient: found mon.noname-b 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 1 -- 172.21.15.89:0/518739935 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84009fc30 con 0x7ff840139620 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83effd640 10 monclient: get_auth_request con 0x7ff8400a6860 auth_method 0 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83effd640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff8400a6860 0x7ff840134930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 1 -- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 msgr2=0x7ff84009c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff83f7fe640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff84009c9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff8380ab000 tx=0x7ff8380196b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff8380039a0 con 0x7ff840139620 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8277fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.378+0000 7ff8277fe640 10 monclient: dump: 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.393 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 10 monclient: _finish_auth 0 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 10 monclient: _check_auth_tickets 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff838003b80 con 0x7ff840139620 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8452ea640 5 monclient: authenticate success, global_id 74603 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8452ea640 10 monclient: _renew_subs 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8452ea640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8452ea640 1 -- 172.21.15.89:0/518739935 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84009ff10 con 0x7ff840139620 2024-08-19T10:55:38.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff838003d30 con 0x7ff840139620 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 10 monclient: dump: 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.395 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8452ea640 10 monclient: _renew_subs 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8452ea640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.89:3301/0 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8452ea640 1 -- 172.21.15.89:0/518739935 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84009ecd0 con 0x7ff840139620 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83d7fa640 4 set_mon_vals no callback set 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8452ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8452ea640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8400a0f00 0x7ff84009d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8452ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8452ea640 1 -- 172.21.15.89:0/518739935 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff84009e1f0 con 0x7ff8400a0f00 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8400a0f00 0x7ff84009d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 10 monclient: get_auth_request con 0x7ff8400a0f00 auth_method 0 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 <== mon.2 v2:172.21.15.89:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff8380a6920 con 0x7ff840139620 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 10 monclient(hunting): handle_auth_done global_id 84406 payload 824 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 1 -- 172.21.15.89:0/518739935 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8400a9490 con 0x7ff8400a0f00 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff82c03be00 0x7ff82c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 <== mon.2 v2:172.21.15.89:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7ff8380b5030 con 0x7ff840139620 2024-08-19T10:55:38.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83e7fc640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff82c03be00 0x7ff82c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83e7fc640 10 monclient: get_auth_request con 0x7ff82c03be00 auth_method 0 2024-08-19T10:55:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 1 -- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8400a0f00 msgr2=0x7ff84009d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff83effd640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8400a0f00 0x7ff84009d6e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff834010790 tx=0x7ff834010c50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff83400f040 con 0x7ff8400a0f00 2024-08-19T10:55:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 10 monclient: dump: 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.401 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff83401c400 con 0x7ff8400a0f00 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8400a0f00 msgr2=0x7ff84009d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.382+0000 7ff8277fe640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8400a0f00 0x7ff84009d6e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff834010790 tx=0x7ff834010c50 comp rx=0 tx=0).stop 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.386+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8400a0f00 msgr2=0x7ff84009d6e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.386+0000 7ff8277fe640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff8400a0f00 0x7ff84009d6e0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff834010790 tx=0x7ff834010c50 comp rx=0 tx=0).stop 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.386+0000 7ff83e7fc640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff82c03be00 0x7ff82c03e2c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff84013f700 tx=0x7ff830008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff8452ea640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:38.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff8452ea640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff84002d1d0 0x7ff84002d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff8452ea640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff8452ea640 1 -- 172.21.15.89:0/518739935 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff8400a00e0 con 0x7ff84002d1d0 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff83effd640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff84002d1d0 0x7ff84002d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff83effd640 10 monclient: get_auth_request con 0x7ff84002d1d0 auth_method 0 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff83effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff83effd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff83effd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff83effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff83effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff83effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff83effd640 10 monclient(hunting): handle_auth_done global_id 84415 payload 824 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.390+0000 7ff83effd640 1 -- 172.21.15.89:0/518739935 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff840064590 con 0x7ff84002d1d0 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff83effd640 1 -- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff84002d1d0 msgr2=0x7ff84002d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff83effd640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff84002d1d0 0x7ff84002d5e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff834010790 tx=0x7ff8340046d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff834005c30 con 0x7ff84002d1d0 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8277fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8277fe640 10 monclient: dump: 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.404 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9268 (secure 0 0 0) 0x7ff834029400 con 0x7ff84002d1d0 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8277fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8277fe640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff84002d1d0 msgr2=0x7ff84002d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8277fe640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff84002d1d0 0x7ff84002d5e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff834010790 tx=0x7ff8340046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8277fe640 1 -- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff84002d1d0 msgr2=0x7ff84002d5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:38.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8277fe640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff84002d1d0 0x7ff84002d5e0 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff834010790 tx=0x7ff8340046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.406 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:38.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:38.406 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:38.406 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:38.406 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:38.406 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 146696, 2024-08-19T10:55:38.406 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:38.406 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:38.406 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:38.406 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.407 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.408 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.409 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:38.410 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.411 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.412 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:38.413 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.414 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.415 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:38.416 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.419 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:38.420 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.422 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.423 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:38.425 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.426 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.427 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:38.428 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8452ea640 1 -- 172.21.15.89:0/518739935 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff82c03be00 msgr2=0x7ff82c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8452ea640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff82c03be00 0x7ff82c03e2c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff84013f700 tx=0x7ff830008040 comp rx=0 tx=0).stop 2024-08-19T10:55:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8452ea640 10 monclient: shutdown 2024-08-19T10:55:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8452ea640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8452ea640 1 -- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 msgr2=0x7ff84009c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8452ea640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff84009c9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff8380ab000 tx=0x7ff8380196b0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff83f7fe640 1 -- 172.21.15.89:0/518739935 reap_dead start 2024-08-19T10:55:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8452ea640 1 -- 172.21.15.89:0/518739935 shutdown_connections 2024-08-19T10:55:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8452ea640 1 --2- 172.21.15.89:0/518739935 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff840139620 0x7ff84009c9e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8452ea640 1 -- 172.21.15.89:0/518739935 >> 172.21.15.89:0/518739935 conn(0x7ff840074b40 msgr2=0x7ff840138910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:38.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8452ea640 1 -- 172.21.15.89:0/518739935 shutdown_connections 2024-08-19T10:55:38.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.394+0000 7ff8452ea640 1 -- 172.21.15.89:0/518739935 wait complete. 2024-08-19T10:55:38.431 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:38.431 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-08-19T10:55:38.431 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:55:38.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100500178}) 2024-08-19T10:55:38.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1805928581}) 2024-08-19T10:55:38.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100500178}) 2024-08-19T10:55:38.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=64620298}) 2024-08-19T10:55:38.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.438+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.440 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100500178}) 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3409314791}) 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100500178}) 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=917757789}) 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100500178}) 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1059117781}) 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100500178}) 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=92984448}) 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.441 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100500178}) 2024-08-19T10:55:38.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3637012023}) 2024-08-19T10:55:38.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100500178}) 2024-08-19T10:55:38.442 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.442+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1549845098}) 2024-08-19T10:55:38.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.455 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2782439361}) 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2047660189}) 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2782439361}) 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4141810541}) 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2782439361}) 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1547308240}) 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2782439361}) 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4225136790}) 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2782439361}) 2024-08-19T10:55:38.456 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1585559838}) 2024-08-19T10:55:38.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2782439361}) 2024-08-19T10:55:38.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2699476092}) 2024-08-19T10:55:38.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2782439361}) 2024-08-19T10:55:38.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4138560227}) 2024-08-19T10:55:38.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2782439361}) 2024-08-19T10:55:38.457 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.458+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3498297164}) 2024-08-19T10:55:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient: get_monmap_and_config 2024-08-19T10:55:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient: build_initial_monmap 2024-08-19T10:55:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient: monmap: 2024-08-19T10:55:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:38.557091+0000 2024-08-19T10:55:38.551 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:38.557091+0000 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:38.552 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:38.553 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 1 Processor -- start 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 1 -- start start 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient: init 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient: _add_conns ranks=[3,9,5,2,8,0,7,1,10,6,4] 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeb006c930 0x7faeb006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient(hunting): picked mon.noname-g con 0x7faeb006c930 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faeb009d340 0x7faeb006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient(hunting): picked mon.noname-a con 0x7faeb009d340 addr v2:172.21.15.89:3300/0 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faeb0137500 0x7faeb0137960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient(hunting): picked mon.noname-h con 0x7faeb0137500 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:38.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7faeb00a54f0 con 0x7faeb009d340 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faeb00a3e00 con 0x7faeb006c930 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7faeb006d790 con 0x7faeb0137500 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb814e640 10 monclient(hunting): authenticate will time out at 1988.373615s 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faeb009d340 0x7faeb006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7faeb009d340 0x7faeb006d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49314/0 (socket says 172.21.15.89:49314) 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 1 -- 172.21.15.89:0/3494580187 learned_addr learned my addr 172.21.15.89:0/3494580187 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 10 monclient(hunting): get_auth_request con 0x7faeb009d340 auth_method 0 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb5eea640 1 --2- 172.21.15.89:0/3494580187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeb006c930 0x7faeb006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb66eb640 1 --2- 172.21.15.89:0/3494580187 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faeb0137500 0x7faeb0137960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb66eb640 10 monclient(hunting): get_auth_request con 0x7faeb0137500 auth_method 0 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb66eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb66eb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb66eb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb5eea640 10 monclient(hunting): get_auth_request con 0x7faeb006c930 auth_method 0 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb5eea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb5eea640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb5eea640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb66eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb66eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb66eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 10 monclient(hunting): handle_auth_done global_id 7677 payload 274 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 1 -- 172.21.15.89:0/3494580187 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faeb0137500 msgr2=0x7faeb0137960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 1 --2- 172.21.15.89:0/3494580187 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faeb0137500 0x7faeb0137960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 1 -- 172.21.15.89:0/3494580187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeb006c930 msgr2=0x7faeb006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 1 --2- 172.21.15.89:0/3494580187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeb006c930 0x7faeb006cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 10 monclient: _finish_hunting 0 2024-08-19T10:55:38.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 1 monclient: found mon.noname-a 2024-08-19T10:55:38.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:38.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:55:38.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.554+0000 7faeb56e9640 1 -- 172.21.15.89:0/3494580187 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb009fbf0 con 0x7faeb009d340 2024-08-19T10:55:38.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb56e9640 1 -- 172.21.15.89:0/3494580187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faeb009d340 msgr2=0x7faeb006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb56e9640 1 --2- 172.21.15.89:0/3494580187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faeb009d340 0x7faeb006d250 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7faea8009e50 tx=0x7faea801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98188f83888c2e11 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 1 -- 172.21.15.89:0/3494580187 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faea80090d0 con 0x7faeb009d340 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 10 monclient: dump: 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.563 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 10 monclient: _finish_auth 0 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 10 monclient: _check_auth_tickets 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 1 -- 172.21.15.89:0/3494580187 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea80a5a90 con 0x7faeb009d340 2024-08-19T10:55:38.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 5 monclient: authenticate success, global_id 7677 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 4 set_mon_vals no callback set 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 1 -- 172.21.15.89:0/3494580187 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faea80a5c40 con 0x7faeb009d340 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb4ee8640 10 monclient: dump: 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.565 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 10 monclient: shutdown 2024-08-19T10:55:38.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 1 -- 172.21.15.89:0/3494580187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faeb009d340 msgr2=0x7faeb006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 1 --2- 172.21.15.89:0/3494580187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faeb009d340 0x7faeb006d250 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7faea8009e50 tx=0x7faea801c8f0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 1 -- 172.21.15.89:0/3494580187 shutdown_connections 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 1 --2- 172.21.15.89:0/3494580187 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7faeb0137500 0x7faeb0137960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 1 --2- 172.21.15.89:0/3494580187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faeb009d340 0x7faeb006d250 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 1 --2- 172.21.15.89:0/3494580187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeb006c930 0x7faeb006cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 1 -- 172.21.15.89:0/3494580187 >> 172.21.15.89:0/3494580187 conn(0x7faeb00a8f60 msgr2=0x7faeb00a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 1 -- 172.21.15.89:0/3494580187 shutdown_connections 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.558+0000 7faeb814e640 1 -- 172.21.15.89:0/3494580187 wait complete. 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient: build_initial_monmap 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient: monmap: 2024-08-19T10:55:38.567 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:38.568 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 1 Processor -- start 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 1 -- start start 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient: init 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient: _add_conns ranks=[7,6,2,5,0,8,3,4,1] 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faeb006c930 0x7faeb013c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient(hunting): picked mon.noname-i con 0x7faeb006c930 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb009d340 0x7faeb01364a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient(hunting): picked mon.noname-d con 0x7faeb009d340 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faeb0137500 0x7faeb011f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient(hunting): picked mon.noname-b con 0x7faeb0137500 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:38.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7faeb0074420 con 0x7faeb0137500 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7faeb01382b0 con 0x7faeb009d340 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7faeb0140f70 con 0x7faeb006c930 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb814e640 10 monclient(hunting): authenticate will time out at 1988.381367s 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb009d340 0x7faeb01364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb66eb640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faeb0137500 0x7faeb011f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb009d340 0x7faeb01364a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36514/0 (socket says 172.21.15.89:36514) 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb66eb640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faeb0137500 0x7faeb011f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60928/0 (socket says 172.21.15.89:60928) 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 1 -- 172.21.15.89:0/1194086713 learned_addr learned my addr 172.21.15.89:0/1194086713 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:38.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 10 monclient(hunting): get_auth_request con 0x7faeb009d340 auth_method 0 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb66eb640 10 monclient(hunting): get_auth_request con 0x7faeb0137500 auth_method 0 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb66eb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb66eb640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb66eb640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb5eea640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faeb006c930 0x7faeb013c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb5eea640 10 monclient(hunting): get_auth_request con 0x7faeb006c930 auth_method 0 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb5eea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb5eea640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb5eea640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb66eb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb66eb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb66eb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 10 monclient(hunting): handle_auth_done global_id 45438 payload 824 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 1 -- 172.21.15.89:0/1194086713 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faeb006c930 msgr2=0x7faeb013c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7faeb006c930 0x7faeb013c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 1 -- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faeb0137500 msgr2=0x7faeb011f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faeb0137500 0x7faeb011f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 10 monclient: _finish_hunting 0 2024-08-19T10:55:38.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 1 monclient: found mon.noname-d 2024-08-19T10:55:38.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:38.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:55:38.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb56e9640 1 -- 172.21.15.89:0/1194086713 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb013d890 con 0x7faeb009d340 2024-08-19T10:55:38.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.562+0000 7faeb66eb640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7faeb0137500 0x7faeb011f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb56e9640 1 -- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb009d340 msgr2=0x7faeb01364a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb56e9640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb009d340 0x7faeb01364a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faea801d190 tx=0x7faea8019690 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faea80bc020 con 0x7faeb009d340 2024-08-19T10:55:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:55:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 10 monclient: dump: 2024-08-19T10:55:38.577 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.578 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 10 monclient: _finish_auth 0 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea8003e10 con 0x7faeb009d340 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faea80044b0 con 0x7faeb009d340 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb814e640 5 monclient: authenticate success, global_id 45438 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb814e640 10 monclient: _renew_subs 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb814e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea7fff640 4 set_mon_vals no callback set 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb814e640 1 -- 172.21.15.89:0/1194086713 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb0121e60 con 0x7faeb009d340 2024-08-19T10:55:38.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 10 monclient: dump: 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.580 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb814e640 10 monclient: _renew_subs 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb814e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb814e640 1 -- 172.21.15.89:0/1194086713 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb013cfe0 con 0x7faeb009d340 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb814e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb814e640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb01369e0 0x7faeb011f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb814e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb814e640 1 -- 172.21.15.89:0/1194086713 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7faeb009e9f0 con 0x7faeb01369e0 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb5eea640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb01369e0 0x7faeb011f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb5eea640 10 monclient: get_auth_request con 0x7faeb01369e0 auth_method 0 2024-08-19T10:55:38.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb5eea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb5eea640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb5eea640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7faea80bf020 con 0x7faeb009d340 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb5eea640 10 monclient(hunting): handle_auth_done global_id 45447 payload 824 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb5eea640 1 -- 172.21.15.89:0/1194086713 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faeb0064590 con 0x7faeb01369e0 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fae9403be00 0x7fae9403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7faea80ea3e0 con 0x7faeb009d340 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb66eb640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fae9403be00 0x7fae9403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.566+0000 7faeb66eb640 10 monclient: get_auth_request con 0x7fae9403be00 auth_method 0 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faeb5eea640 1 -- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb01369e0 msgr2=0x7faeb011f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faeb5eea640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb01369e0 0x7faeb011f8c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faeac0119d0 tx=0x7faeac011e90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faeac019070 con 0x7faeb01369e0 2024-08-19T10:55:38.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faea67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faea67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:38.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faea67fc640 10 monclient: dump: 2024-08-19T10:55:38.583 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.583 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.583 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.583 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7faeac00f950 con 0x7faeb01369e0 2024-08-19T10:55:38.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faea67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faea67fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb01369e0 msgr2=0x7faeb011f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faea67fc640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb01369e0 0x7faeb011f8c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faeac0119d0 tx=0x7faeac011e90 comp rx=0 tx=0).stop 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb01369e0 msgr2=0x7faeb011f8c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faea67fc640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb01369e0 0x7faeb011f8c0 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faeac0119d0 tx=0x7faeac011e90 comp rx=0 tx=0).stop 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.570+0000 7faeb66eb640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fae9403be00 0x7fae9403e2c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7faeb009f680 tx=0x7faea0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb814e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb814e640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb011fe00 0x7faeb0072e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb814e640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb814e640 1 -- 172.21.15.89:0/1194086713 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7faeb0120210 con 0x7faeb011fe00 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb5eea640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb011fe00 0x7faeb0072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb5eea640 10 monclient: get_auth_request con 0x7faeb011fe00 auth_method 0 2024-08-19T10:55:38.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb5eea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb5eea640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb5eea640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb5eea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb5eea640 10 monclient(hunting): handle_auth_done global_id 45456 payload 824 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.574+0000 7faeb5eea640 1 -- 172.21.15.89:0/1194086713 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faeb0137910 con 0x7faeb011fe00 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb5eea640 1 -- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb011fe00 msgr2=0x7faeb0072e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb5eea640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb011fe00 0x7faeb0072e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faeac0119d0 tx=0x7faeac0046d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7faeac019040 con 0x7faeb011fe00 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faea67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faea67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:38.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faea67fc640 10 monclient: dump: 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.587 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.588 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.588 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7faeac01e450 con 0x7faeb011fe00 2024-08-19T10:55:38.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faea67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:38.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faea67fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:38.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb011fe00 msgr2=0x7faeb0072e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faea67fc640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb011fe00 0x7faeb0072e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faeac0119d0 tx=0x7faeac0046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faea67fc640 1 -- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb011fe00 msgr2=0x7faeb0072e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:38.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faea67fc640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb011fe00 0x7faeb0072e20 secure :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faeac0119d0 tx=0x7faeac0046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.588 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:38.588 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 661705, 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:38.589 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.590 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.591 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:38.592 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.593 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:38.594 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:38.595 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.596 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:38.597 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:38.598 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.599 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:38.601 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.602 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.603 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:38.604 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.607 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:38.608 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.610 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.611 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 3 2024-08-19T10:55:38.612 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.612 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.612 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:38.612 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:38.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb814e640 1 -- 172.21.15.89:0/1194086713 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fae9403be00 msgr2=0x7fae9403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb814e640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fae9403be00 0x7fae9403e2c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7faeb009f680 tx=0x7faea0009190 comp rx=0 tx=0).stop 2024-08-19T10:55:38.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb814e640 10 monclient: shutdown 2024-08-19T10:55:38.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb814e640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:38.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb814e640 1 -- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb009d340 msgr2=0x7faeb01364a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb814e640 1 --2- 172.21.15.89:0/1194086713 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7faeb009d340 0x7faeb01364a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faea801d190 tx=0x7faea8019690 comp rx=0 tx=0).stop 2024-08-19T10:55:38.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb66eb640 1 -- 172.21.15.89:0/1194086713 reap_dead start 2024-08-19T10:55:38.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb66eb640 1 -- 172.21.15.89:0/1194086713 reap_dead start 2024-08-19T10:55:38.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb814e640 1 -- 172.21.15.89:0/1194086713 shutdown_connections 2024-08-19T10:55:38.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb814e640 1 -- 172.21.15.89:0/1194086713 >> 172.21.15.89:0/1194086713 conn(0x7faeb00a8f60 msgr2=0x7faeb009ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:38.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb814e640 1 -- 172.21.15.89:0/1194086713 shutdown_connections 2024-08-19T10:55:38.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.578+0000 7faeb814e640 1 -- 172.21.15.89:0/1194086713 wait complete. 2024-08-19T10:55:38.615 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:38.615 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-08-19T10:55:38.615 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:55:38.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient: get_monmap_and_config 2024-08-19T10:55:38.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient: build_initial_monmap 2024-08-19T10:55:38.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient: monmap: 2024-08-19T10:55:38.776 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:38.776 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.776 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:38.781578+0000 2024-08-19T10:55:38.776 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:38.781578+0000 2024-08-19T10:55:38.777 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:38.777 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.777 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:38.777 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:38.777 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:38.777 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:38.778 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:38.778 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:38.778 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:38.778 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:38.778 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:38.778 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:38.779 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:38.779 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 1 Processor -- start 2024-08-19T10:55:38.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 1 -- start start 2024-08-19T10:55:38.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient: init 2024-08-19T10:55:38.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:38.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient: _add_conns ranks=[5,1,0,8,6,3,10,7,4,9,2] 2024-08-19T10:55:38.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2ce00a5270 0x7f2ce00a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient(hunting): picked mon.noname-h con 0x7f2ce00a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:38.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2ce00a6d30 addr v1:172.21.15.190:6789/0 2024-08-19T10:55:38.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2ce00a7110 addr v1:172.21.15.89:6789/0 2024-08-19T10:55:38.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ce00a8710 con 0x7f2ce00a7110 2024-08-19T10:55:38.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ce013a3d0 con 0x7f2ce00a6d30 2024-08-19T10:55:38.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2ce00a5a40 con 0x7f2ce00a5270 2024-08-19T10:55:38.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:38.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2ce5be0640 10 monclient(hunting): authenticate will time out at 1988.598661s 2024-08-19T10:55:38.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2ce00a7110 0x7f2ce0139cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49450/0 (socket says 172.21.15.89:49450) 2024-08-19T10:55:38.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdffff640 1 -- 172.21.15.89:0/3103007396 learned_addr learned my addr 172.21.15.89:0/3103007396 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 1 --2- 172.21.15.89:0/3103007396 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2ce00a5270 0x7f2ce00a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 10 monclient(hunting): get_auth_request con 0x7f2ce00a5270 auth_method 0 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 10 monclient(hunting): handle_auth_done global_id 84344 payload 274 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 1 -- 172.21.15.89:0/3103007396 >> v1:172.21.15.190:6789/0 conn(0x7f2ce00a6d30 legacy=0x7f2ce01395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 1 -- 172.21.15.89:0/3103007396 >> v1:172.21.15.89:6789/0 conn(0x7f2ce00a7110 legacy=0x7f2ce0139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:38.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 1 monclient: found mon.noname-h 2024-08-19T10:55:38.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:38.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:55:38.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.778+0000 7f2cdf7fe640 1 -- 172.21.15.89:0/3103007396 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce00a81d0 con 0x7f2ce00a5270 2024-08-19T10:55:38.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 1 -- 172.21.15.89:0/3103007396 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2ce00a5270 msgr2=0x7f2ce00a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 1 --2- 172.21.15.89:0/3103007396 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2ce00a5270 0x7f2ce00a67f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00097f0 tx=0x7f2cd001c1c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=5c2abca502e45bd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 1 -- 172.21.15.89:0/3103007396 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2cd00b0070 con 0x7f2ce00a5270 2024-08-19T10:55:38.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:55:38.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 10 monclient: dump: 2024-08-19T10:55:38.792 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.792 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.793 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.793 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.793 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.793 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.793 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.793 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.793 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.793 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.793 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.793 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.793 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.794 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.794 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.794 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 10 monclient: _finish_auth 0 2024-08-19T10:55:38.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:55:38.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:38.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 1 -- 172.21.15.89:0/3103007396 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd00aa810 con 0x7f2ce00a5270 2024-08-19T10:55:38.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:38.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 1 -- 172.21.15.89:0/3103007396 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2cd00a93e0 con 0x7f2ce00a5270 2024-08-19T10:55:38.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 5 monclient: authenticate success, global_id 84344 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 4 set_mon_vals no callback set 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cde7fc640 10 monclient: dump: 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.795 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.796 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.796 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.796 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.796 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.796 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.796 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.796 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.796 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient: shutdown 2024-08-19T10:55:38.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:38.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 -- 172.21.15.89:0/3103007396 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2ce00a5270 msgr2=0x7f2ce00a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 --2- 172.21.15.89:0/3103007396 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2ce00a5270 0x7f2ce00a67f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00097f0 tx=0x7f2cd001c1c0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 -- 172.21.15.89:0/3103007396 shutdown_connections 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 --2- 172.21.15.89:0/3103007396 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f2ce00a5270 0x7f2ce00a67f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 -- 172.21.15.89:0/3103007396 >> 172.21.15.89:0/3103007396 conn(0x7f2ce0069a80 msgr2=0x7f2ce00abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 -- 172.21.15.89:0/3103007396 shutdown_connections 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 -- 172.21.15.89:0/3103007396 wait complete. 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient: build_initial_monmap 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient: monmap: 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:38.797 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.798 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:38.798 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:38.798 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:38.798 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:38.798 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:38.798 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:38.798 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:38.798 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 Processor -- start 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 -- start start 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient: init 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient: _add_conns ranks=[8,3,6,1,2,4,0,5,7] 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce00a5270 0x7f2ce0138b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient(hunting): picked mon.noname-e con 0x7f2ce00a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce00a6d30 0x7f2ce01390b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient(hunting): picked mon.noname-g con 0x7f2ce00a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ce00a7110 0x7f2ce013f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient(hunting): picked mon.noname-d con 0x7f2ce00a7110 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce00a5650 con 0x7f2ce00a6d30 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f2ce00a7570 con 0x7f2ce00a7110 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2ce00a57d0 con 0x7f2ce00a5270 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2ce5be0640 10 monclient(hunting): authenticate will time out at 1988.607655s 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce00a5270 0x7f2ce0138b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce00a5270 0x7f2ce0138b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:59916/0 (socket says 172.21.15.89:59916) 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 1 -- 172.21.15.89:0/1944288838 learned_addr learned my addr 172.21.15.89:0/1944288838 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:38.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 10 monclient(hunting): get_auth_request con 0x7f2ce00a5270 auth_method 0 2024-08-19T10:55:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdffff640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ce00a7110 0x7f2ce013f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdffff640 10 monclient(hunting): get_auth_request con 0x7f2ce00a7110 auth_method 0 2024-08-19T10:55:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.786+0000 7f2cdffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce00a6d30 0x7f2ce01390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient(hunting): get_auth_request con 0x7f2ce00a6d30 auth_method 0 2024-08-19T10:55:38.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 10 monclient(hunting): handle_auth_done global_id 114191 payload 824 2024-08-19T10:55:38.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 1 -- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ce00a7110 msgr2=0x7f2ce013f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ce00a7110 0x7f2ce013f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 1 -- 172.21.15.89:0/1944288838 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce00a6d30 msgr2=0x7f2ce01390b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ce00a6d30 0x7f2ce01390b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:38.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 1 monclient: found mon.noname-e 2024-08-19T10:55:38.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:38.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:55:38.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 1 -- 172.21.15.89:0/1944288838 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce0141d30 con 0x7f2ce00a5270 2024-08-19T10:55:38.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdffff640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f2ce00a7110 0x7f2ce013f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:38.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 1 -- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce00a5270 msgr2=0x7f2ce0138b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdf7fe640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce00a5270 0x7f2ce0138b70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0000c10 tx=0x7f2cd0018f60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2cd00b0040 con 0x7f2ce00a5270 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 10 monclient: dump: 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.804 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.805 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.805 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.805 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.805 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.805 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.805 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.805 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.805 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 10 monclient: _finish_auth 0 2024-08-19T10:55:38.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 10 monclient: _check_auth_tickets 2024-08-19T10:55:38.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd00b9070 con 0x7f2ce00a5270 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2cd0004250 con 0x7f2ce00a5270 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2ce5be0640 5 monclient: authenticate success, global_id 114191 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2ce5be0640 10 monclient: _renew_subs 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2ce5be0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2ce5be0640 1 -- 172.21.15.89:0/1944288838 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce0122b30 con 0x7f2ce00a5270 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cddffb640 4 set_mon_vals no callback set 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 10 monclient: dump: 2024-08-19T10:55:38.806 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.807 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2ce5be0640 10 monclient: _renew_subs 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2ce5be0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2ce5be0640 1 -- 172.21.15.89:0/1944288838 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce0122850 con 0x7f2ce00a5270 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2ce5be0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2ce5be0640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce013f8b0 0x7f2ce013fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2ce5be0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2ce5be0640 1 -- 172.21.15.89:0/1944288838 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2ce00a26c0 con 0x7f2ce013f8b0 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce013f8b0 0x7f2ce013fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient: get_auth_request con 0x7f2ce013f8b0 auth_method 0 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2cd00b3070 con 0x7f2ce00a5270 2024-08-19T10:55:38.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 10 monclient(hunting): handle_auth_done global_id 114200 payload 824 2024-08-19T10:55:38.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.790+0000 7f2cdeffd640 1 -- 172.21.15.89:0/1944288838 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ce00a67f0 con 0x7f2ce013f8b0 2024-08-19T10:55:38.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ccc03be00 0x7f2ccc03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f2cd00e9250 con 0x7f2ce00a5270 2024-08-19T10:55:38.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cdffff640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ccc03be00 0x7f2ccc03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cdffff640 10 monclient: get_auth_request con 0x7f2ccc03be00 auth_method 0 2024-08-19T10:55:38.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cdeffd640 1 -- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce013f8b0 msgr2=0x7f2ce013fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cdeffd640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce013f8b0 0x7f2ce013fc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400e420 tx=0x7f2cd400e8e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2cd4006c50 con 0x7f2ce013f8b0 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 10 monclient: dump: 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.812 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.813 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.813 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.813 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.813 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.813 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.813 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.813 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.813 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.813 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2cd4007450 con 0x7f2ce013f8b0 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce013f8b0 msgr2=0x7f2ce013fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce013f8b0 0x7f2ce013fc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400e420 tx=0x7f2cd400e8e0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce013f8b0 msgr2=0x7f2ce013fc90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cc7fff640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce013f8b0 0x7f2ce013fc90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.794+0000 7f2cdffff640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ccc03be00 0x7f2ccc03e2c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8005c10 tx=0x7f2cd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2ce5be0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2ce5be0640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce01401d0 0x7f2ce013a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2ce5be0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2ce5be0640 1 -- 172.21.15.89:0/1944288838 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2ce0064590 con 0x7f2ce01401d0 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce01401d0 0x7f2ce013a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 10 monclient: get_auth_request con 0x7f2ce01401d0 auth_method 0 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 10 monclient(hunting): handle_auth_done global_id 114209 payload 824 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 1 -- 172.21.15.89:0/1944288838 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2ce01424c0 con 0x7f2ce01401d0 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 1 -- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce01401d0 msgr2=0x7f2ce013a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cdeffd640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce01401d0 0x7f2ce013a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400e420 tx=0x7f2cd4003770 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2cd4028430 con 0x7f2ce01401d0 2024-08-19T10:55:38.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cc7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cc7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:55:38.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cc7fff640 10 monclient: dump: 2024-08-19T10:55:38.816 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.816 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.816 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.816 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.816 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.816 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.816 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f2cd401eb80 con 0x7f2ce01401d0 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cc7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cc7fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:38.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce01401d0 msgr2=0x7f2ce013a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cc7fff640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce01401d0 0x7f2ce013a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400e420 tx=0x7f2cd4003770 comp rx=0 tx=0).stop 2024-08-19T10:55:38.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cc7fff640 1 -- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce01401d0 msgr2=0x7f2ce013a7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:38.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.802+0000 7f2cc7fff640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce01401d0 0x7f2ce013a7a0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400e420 tx=0x7f2cd4003770 comp rx=0 tx=0).stop 2024-08-19T10:55:38.818 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:38.818 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:38.818 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:38.818 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:38.818 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:38.818 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 19451, 2024-08-19T10:55:38.819 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:38.819 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:38.819 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:38.819 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:38.819 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:38.819 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:38.819 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:38.819 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:38.819 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:38.819 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:38.820 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.820 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:55:38.820 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:38.820 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:38.820 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:38.820 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.820 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.820 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.820 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.820 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.820 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.821 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:38.822 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:38.823 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:38.823 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:38.823 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:38.823 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:38.823 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:38.823 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:38.823 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:38.823 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:38.823 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:38.823 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:38.824 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.826 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.827 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:38.827 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:38.827 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.827 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.827 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:38.827 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.827 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.828 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.829 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.830 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.832 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:38.833 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:38.834 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.834 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.834 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.834 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.834 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:38.834 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:38.834 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.834 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.834 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.835 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:38.836 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.837 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.838 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:38.839 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.840 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.841 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:38.842 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2ce5be0640 1 -- 172.21.15.89:0/1944288838 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ccc03be00 msgr2=0x7f2ccc03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2ce5be0640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f2ccc03be00 0x7f2ccc03e2c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8005c10 tx=0x7f2cd8009040 comp rx=0 tx=0).stop 2024-08-19T10:55:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2ce5be0640 10 monclient: shutdown 2024-08-19T10:55:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2ce5be0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2ce5be0640 1 -- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce00a5270 msgr2=0x7f2ce0138b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2ce5be0640 1 --2- 172.21.15.89:0/1944288838 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f2ce00a5270 0x7f2ce0138b70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0000c10 tx=0x7f2cd0018f60 comp rx=0 tx=0).stop 2024-08-19T10:55:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2cdffff640 1 -- 172.21.15.89:0/1944288838 reap_dead start 2024-08-19T10:55:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2cdffff640 1 -- 172.21.15.89:0/1944288838 reap_dead start 2024-08-19T10:55:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2ce5be0640 1 -- 172.21.15.89:0/1944288838 shutdown_connections 2024-08-19T10:55:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2ce5be0640 1 -- 172.21.15.89:0/1944288838 >> 172.21.15.89:0/1944288838 conn(0x7f2ce0069a80 msgr2=0x7f2ce009cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2ce5be0640 1 -- 172.21.15.89:0/1944288838 shutdown_connections 2024-08-19T10:55:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.806+0000 7f2ce5be0640 1 -- 172.21.15.89:0/1944288838 wait complete. 2024-08-19T10:55:38.845 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:38.846 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-08-19T10:55:38.846 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:55:38.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=975124240}) 2024-08-19T10:55:38.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=200824157}) 2024-08-19T10:55:38.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=975124240}) 2024-08-19T10:55:38.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=394371091}) 2024-08-19T10:55:38.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.926 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=975124240}) 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4145493538}) 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=975124240}) 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3862651840}) 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=975124240}) 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3108956798}) 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=975124240}) 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1152245137}) 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=975124240}) 2024-08-19T10:55:38.927 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4068494888}) 2024-08-19T10:55:38.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=975124240}) 2024-08-19T10:55:38.928 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.926+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1320457178}) 2024-08-19T10:55:38.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890446356}) 2024-08-19T10:55:38.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=598802860}) 2024-08-19T10:55:38.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890446356}) 2024-08-19T10:55:38.929 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3117128327}) 2024-08-19T10:55:38.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890446356}) 2024-08-19T10:55:38.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1813348957}) 2024-08-19T10:55:38.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890446356}) 2024-08-19T10:55:38.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4025135902}) 2024-08-19T10:55:38.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890446356}) 2024-08-19T10:55:38.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2721775968}) 2024-08-19T10:55:38.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.930 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890446356}) 2024-08-19T10:55:38.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2509648975}) 2024-08-19T10:55:38.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890446356}) 2024-08-19T10:55:38.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2004005253}) 2024-08-19T10:55:38.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890446356}) 2024-08-19T10:55:38.931 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.930+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2471301}) 2024-08-19T10:55:38.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763655261}) 2024-08-19T10:55:38.933 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1978680073}) 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763655261}) 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2430698430}) 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763655261}) 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1012890268}) 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763655261}) 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2280513194}) 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763655261}) 2024-08-19T10:55:38.934 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=503858552}) 2024-08-19T10:55:38.935 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.935 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763655261}) 2024-08-19T10:55:38.935 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=868799409}) 2024-08-19T10:55:38.935 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.935 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763655261}) 2024-08-19T10:55:38.935 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2895380135}) 2024-08-19T10:55:38.935 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.935 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2763655261}) 2024-08-19T10:55:38.935 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.934+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3837338546}) 2024-08-19T10:55:38.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1127661872}) 2024-08-19T10:55:38.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1898063184}) 2024-08-19T10:55:38.937 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1127661872}) 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2290838959}) 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1127661872}) 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4088902602}) 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1127661872}) 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4183251712}) 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1127661872}) 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=30179495}) 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.938 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1127661872}) 2024-08-19T10:55:38.939 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=623759767}) 2024-08-19T10:55:38.939 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.939 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1127661872}) 2024-08-19T10:55:38.939 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3390531022}) 2024-08-19T10:55:38.939 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.939 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1127661872}) 2024-08-19T10:55:38.939 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.938+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1512424691}) 2024-08-19T10:55:38.940 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.940 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1932613683}) 2024-08-19T10:55:38.940 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3004159161}) 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1932613683}) 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2457652850}) 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1932613683}) 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3895515069}) 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1932613683}) 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2225721386}) 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1932613683}) 2024-08-19T10:55:38.941 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3308052663}) 2024-08-19T10:55:38.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1932613683}) 2024-08-19T10:55:38.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=659431874}) 2024-08-19T10:55:38.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1932613683}) 2024-08-19T10:55:38.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2304712375}) 2024-08-19T10:55:38.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1932613683}) 2024-08-19T10:55:38.942 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.942+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3261798694}) 2024-08-19T10:55:38.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1256053410}) 2024-08-19T10:55:38.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=393671437}) 2024-08-19T10:55:38.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1256053410}) 2024-08-19T10:55:38.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3204410180}) 2024-08-19T10:55:38.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1256053410}) 2024-08-19T10:55:38.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2387048951}) 2024-08-19T10:55:38.945 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1256053410}) 2024-08-19T10:55:38.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3398252045}) 2024-08-19T10:55:38.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1256053410}) 2024-08-19T10:55:38.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2288162308}) 2024-08-19T10:55:38.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1256053410}) 2024-08-19T10:55:38.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1096464668}) 2024-08-19T10:55:38.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.946 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1256053410}) 2024-08-19T10:55:38.947 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1531429625}) 2024-08-19T10:55:38.947 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.947 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1256053410}) 2024-08-19T10:55:38.947 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.946+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=681370648}) 2024-08-19T10:55:38.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.954+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.954+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2494127412}) 2024-08-19T10:55:38.956 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.954+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=436245160}) 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.954+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.954+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2494127412}) 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.954+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1318906681}) 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.954+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.954+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2494127412}) 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1342323121}) 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2494127412}) 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=994484708}) 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2494127412}) 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2362582313}) 2024-08-19T10:55:38.957 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2494127412}) 2024-08-19T10:55:38.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1015176759}) 2024-08-19T10:55:38.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2494127412}) 2024-08-19T10:55:38.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3735192839}) 2024-08-19T10:55:38.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2494127412}) 2024-08-19T10:55:38.958 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.958+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4066447690}) 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2062164521}) 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=252397730}) 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2062164521}) 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2513776696}) 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2062164521}) 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2187173936}) 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2062164521}) 2024-08-19T10:55:38.961 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1404430868}) 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2062164521}) 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3186083603}) 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2062164521}) 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4090233623}) 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2062164521}) 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2032493703}) 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2062164521}) 2024-08-19T10:55:38.962 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.962+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3957890215}) 2024-08-19T10:55:38.966 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.966 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2104210142}) 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4011880862}) 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2104210142}) 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3865835909}) 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2104210142}) 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1162185217}) 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2104210142}) 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=976228703}) 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.967 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2104210142}) 2024-08-19T10:55:38.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2183499746}) 2024-08-19T10:55:38.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2104210142}) 2024-08-19T10:55:38.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2571728937}) 2024-08-19T10:55:38.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2104210142}) 2024-08-19T10:55:38.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3991015756}) 2024-08-19T10:55:38.968 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2104210142}) 2024-08-19T10:55:38.969 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.966+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2758493977}) 2024-08-19T10:55:38.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385106855}) 2024-08-19T10:55:38.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4056892844}) 2024-08-19T10:55:38.971 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385106855}) 2024-08-19T10:55:38.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1863431717}) 2024-08-19T10:55:38.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385106855}) 2024-08-19T10:55:38.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1550592173}) 2024-08-19T10:55:38.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385106855}) 2024-08-19T10:55:38.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1713154209}) 2024-08-19T10:55:38.972 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385106855}) 2024-08-19T10:55:38.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2959066759}) 2024-08-19T10:55:38.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385106855}) 2024-08-19T10:55:38.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1367497163}) 2024-08-19T10:55:38.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385106855}) 2024-08-19T10:55:38.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=79176238}) 2024-08-19T10:55:38.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385106855}) 2024-08-19T10:55:38.973 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.970+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1797746327}) 2024-08-19T10:55:38.974 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.974 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=445997695}) 2024-08-19T10:55:38.974 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1749765533}) 2024-08-19T10:55:38.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f7086ea6640 10 monclient: get_monmap_and_config 2024-08-19T10:55:38.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f7086ea6640 10 monclient: build_initial_monmap 2024-08-19T10:55:38.975 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f7086ea6640 10 monclient: monmap: 2024-08-19T10:55:38.975 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:38.975 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.975 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:38.979029+0000 2024-08-19T10:55:38.975 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:38.979029+0000 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:38.976 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.979 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=445997695}) 2024-08-19T10:55:38.979 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3349844144}) 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=445997695}) 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1981781146}) 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=445997695}) 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=700155483}) 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=445997695}) 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1437921667}) 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=445997695}) 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=978391981}) 2024-08-19T10:55:38.980 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=445997695}) 2024-08-19T10:55:38.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2533992488}) 2024-08-19T10:55:38.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=445997695}) 2024-08-19T10:55:38.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=814306619}) 2024-08-19T10:55:38.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f7086ea6640 1 Processor -- start 2024-08-19T10:55:38.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f7086ea6640 1 -- start start 2024-08-19T10:55:38.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.974+0000 7f7086ea6640 10 monclient: init 2024-08-19T10:55:38.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:38.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 10 monclient: _add_conns ranks=[0,4,1,8,3,7,2,10,6,5,9] 2024-08-19T10:55:38.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f70800a3bd0 addr v1:172.21.15.89:6789/0 2024-08-19T10:55:38.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70800a46c0 0x7f708013eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:38.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 10 monclient(hunting): picked mon.noname-c con 0x7f70800a46c0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:38.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f708013f070 addr v1:172.21.15.190:6789/0 2024-08-19T10:55:38.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f708006d680 con 0x7f70800a3bd0 2024-08-19T10:55:38.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70800a1d90 con 0x7f708013f070 2024-08-19T10:55:38.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f70800a1450 con 0x7f70800a46c0 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7086ea6640 10 monclient(hunting): authenticate will time out at 1988.796288s 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7084c42640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f70800a3bd0 0x7f70800a3fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49452/0 (socket says 172.21.15.89:49452) 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f7084c42640 1 -- 172.21.15.89:0/771068284 learned_addr learned my addr 172.21.15.89:0/771068284 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 1 --2- 172.21.15.89:0/771068284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70800a46c0 0x7f708013eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 10 monclient(hunting): get_auth_request con 0x7f70800a46c0 auth_method 0 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:38.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:38.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:38.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 10 monclient(hunting): handle_auth_done global_id 84424 payload 274 2024-08-19T10:55:38.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 1 -- 172.21.15.89:0/771068284 >> v1:172.21.15.190:6789/0 conn(0x7f708013f070 legacy=0x7f7080141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 1 -- 172.21.15.89:0/771068284 >> v1:172.21.15.89:6789/0 conn(0x7f70800a3bd0 legacy=0x7f70800a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:38.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:55:38.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 1 monclient: found mon.noname-c 2024-08-19T10:55:38.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:38.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:38.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f707ffff640 1 -- 172.21.15.89:0/771068284 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70800a4d20 con 0x7f70800a46c0 2024-08-19T10:55:38.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4074072984}) 2024-08-19T10:55:38.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1154630740}) 2024-08-19T10:55:38.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4074072984}) 2024-08-19T10:55:38.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3721033431}) 2024-08-19T10:55:38.988 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4074072984}) 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3468433688}) 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4074072984}) 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3932380874}) 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4074072984}) 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=966005847}) 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4074072984}) 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3230383214}) 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.989 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4074072984}) 2024-08-19T10:55:38.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1508670589}) 2024-08-19T10:55:38.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:38.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4074072984}) 2024-08-19T10:55:38.990 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:38.978+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2042784175}) 2024-08-19T10:55:38.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707ffff640 1 -- 172.21.15.89:0/771068284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70800a46c0 msgr2=0x7f708013eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:38.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707ffff640 1 --2- 172.21.15.89:0/771068284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70800a46c0 0x7f708013eb30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f70800a0860 tx=0x7f707001c360 comp rx=0 tx=0).ready entity=mon.4 client_cookie=e8da1dd26c531755 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:38.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 1 -- 172.21.15.89:0/771068284 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70700b0070 con 0x7f70800a46c0 2024-08-19T10:55:38.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:38.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 10 monclient: dump: 2024-08-19T10:55:38.996 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.996 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.996 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.996 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.996 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.996 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.996 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.996 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 1 -- 172.21.15.89:0/771068284 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70700aa810 con 0x7f70800a46c0 2024-08-19T10:55:38.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 5 monclient: authenticate success, global_id 84424 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 1 -- 172.21.15.89:0/771068284 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70700aae30 con 0x7f70800a46c0 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 4 set_mon_vals no callback set 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f707f7fe640 10 monclient: dump: 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:38.998 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 10 monclient: shutdown 2024-08-19T10:55:38.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 -- 172.21.15.89:0/771068284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70800a46c0 msgr2=0x7f708013eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 --2- 172.21.15.89:0/771068284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70800a46c0 0x7f708013eb30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f70800a0860 tx=0x7f707001c360 comp rx=0 tx=0).stop 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 -- 172.21.15.89:0/771068284 shutdown_connections 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 --2- 172.21.15.89:0/771068284 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f70800a46c0 0x7f708013eb30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 -- 172.21.15.89:0/771068284 >> 172.21.15.89:0/771068284 conn(0x7f708006de40 msgr2=0x7f708006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 -- 172.21.15.89:0/771068284 shutdown_connections 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 -- 172.21.15.89:0/771068284 wait complete. 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 10 monclient: build_initial_monmap 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 10 monclient: monmap: 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:39.000 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 Processor -- start 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 -- start start 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 10 monclient: init 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 10 monclient: _add_conns ranks=[0,5,6,8,3,4,1,2,7] 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70800a3bd0 0x7f7080125a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 10 monclient(hunting): picked mon.noname-a con 0x7f70800a3bd0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f70800a46c0 0x7f70801d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 10 monclient(hunting): picked mon.noname-h con 0x7f70800a46c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f708013f070 0x7f7080123790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.982+0000 7f7086ea6640 10 monclient(hunting): picked mon.noname-d con 0x7f708013f070 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7086ea6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7086ea6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f70800a51e0 con 0x7f70800a3bd0 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7086ea6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7086ea6640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f70800a5510 con 0x7f70800a46c0 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7086ea6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7086ea6640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7080142480 con 0x7f708013f070 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7086ea6640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7086ea6640 10 monclient(hunting): authenticate will time out at 1988.804036s 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70800a3bd0 0x7f7080125a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70800a3bd0 0x7f7080125a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49328/0 (socket says 172.21.15.89:49328) 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 1 -- 172.21.15.89:0/3505894738 learned_addr learned my addr 172.21.15.89:0/3505894738 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 10 monclient(hunting): get_auth_request con 0x7f70800a3bd0 auth_method 0 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7085443640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f708013f070 0x7f7080123790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7085443640 10 monclient(hunting): get_auth_request con 0x7f708013f070 auth_method 0 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7085443640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7085443640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7085443640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707ffff640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f70800a46c0 0x7f70801d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7085443640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7085443640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7085443640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707ffff640 10 monclient(hunting): get_auth_request con 0x7f70800a46c0 auth_method 0 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 10 monclient(hunting): handle_auth_done global_id 7686 payload 824 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 1 -- 172.21.15.89:0/3505894738 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f708013f070 msgr2=0x7f7080123790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f708013f070 0x7f7080123790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 1 -- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f70800a46c0 msgr2=0x7f70801d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f70800a46c0 0x7f70801d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 10 monclient: _finish_hunting 0 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 1 monclient: found mon.noname-a 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 1 -- 172.21.15.89:0/3505894738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7080126fa0 con 0x7f70800a3bd0 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 1 -- 172.21.15.89:0/3505894738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70800a3bd0 msgr2=0x7f7080125a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7084c42640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70800a3bd0 0x7f7080125a80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f707400c470 tx=0x7f707400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7074005d00 con 0x7f70800a3bd0 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707d7fa640 10 monclient: dump: 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.006 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7086ea6640 5 monclient: authenticate success, global_id 7686 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f7086ea6640 10 monclient: _renew_subs 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.986+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70740074e0 con 0x7f70800a3bd0 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f7086ea6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f7086ea6640 1 -- 172.21.15.89:0/3505894738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7080127280 con 0x7f70800a3bd0 2024-08-19T10:55:39.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70740076d0 con 0x7f70800a3bd0 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707d7fa640 10 monclient: dump: 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.008 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707effd640 4 set_mon_vals no callback set 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f7086ea6640 10 monclient: _renew_subs 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f7086ea6640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f7086ea6640 1 -- 172.21.15.89:0/3505894738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708013ea30 con 0x7f70800a3bd0 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f7086ea6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f7086ea6640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70801d20e0 0x7f7080123cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f7086ea6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f7086ea6640 1 -- 172.21.15.89:0/3505894738 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f70800a5360 con 0x7f70801d20e0 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707ffff640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70801d20e0 0x7f7080123cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707ffff640 10 monclient: get_auth_request con 0x7f70801d20e0 auth_method 0 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7074007070 con 0x7f70800a3bd0 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707d7fa640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f706c03be00 0x7f706c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f707404f320 con 0x7f70800a3bd0 2024-08-19T10:55:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707ffff640 10 monclient(hunting): handle_auth_done global_id 45307 payload 824 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f707ffff640 1 -- 172.21.15.89:0/3505894738 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70800a3fb0 con 0x7f70801d20e0 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f7085443640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f706c03be00 0x7f706c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.990+0000 7f7085443640 10 monclient: get_auth_request con 0x7f706c03be00 auth_method 0 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707ffff640 1 -- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70801d20e0 msgr2=0x7f7080123cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707ffff640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70801d20e0 0x7f7080123cd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f708006da10 tx=0x7f7070019100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f7085443640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f706c03be00 0x7f706c03e2c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7078009f60 tx=0x7f707800d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70700b9070 con 0x7f70801d20e0 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707d7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707d7fa640 10 monclient: dump: 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.011 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.012 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.013 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7070003910 con 0x7f70801d20e0 2024-08-19T10:55:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70801d20e0 msgr2=0x7f7080123cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707d7fa640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70801d20e0 0x7f7080123cd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f708006da10 tx=0x7f7070019100 comp rx=0 tx=0).stop 2024-08-19T10:55:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70801d20e0 msgr2=0x7f7080123cd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:39.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:38.994+0000 7f707d7fa640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70801d20e0 0x7f7080123cd0 secure :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f708006da10 tx=0x7f7070019100 comp rx=0 tx=0).stop 2024-08-19T10:55:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f7086ea6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f7086ea6640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7080124210 0x7f7080140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f7086ea6640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f7086ea6640 1 -- 172.21.15.89:0/3505894738 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7080064590 con 0x7f7080124210 2024-08-19T10:55:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f707ffff640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7080124210 0x7f7080140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f707ffff640 10 monclient: get_auth_request con 0x7f7080124210 auth_method 0 2024-08-19T10:55:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f707ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f707ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f707ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f707ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f707ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f707ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f707ffff640 10 monclient(hunting): handle_auth_done global_id 45316 payload 824 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.002+0000 7f707ffff640 1 -- 172.21.15.89:0/3505894738 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f70801245f0 con 0x7f7080124210 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707ffff640 1 -- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7080124210 msgr2=0x7f7080140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707ffff640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7080124210 0x7f7080140530 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7070009710 tx=0x7f70700a8000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f70700b9070 con 0x7f7080124210 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707d7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707d7fa640 10 monclient: dump: 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.016 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.017 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.017 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.017 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.017 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.017 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.017 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.017 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.017 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.017 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f70700b5e30 con 0x7f7080124210 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707d7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7080124210 msgr2=0x7f7080140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707d7fa640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7080124210 0x7f7080140530 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7070009710 tx=0x7f70700a8000 comp rx=0 tx=0).stop 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707d7fa640 1 -- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7080124210 msgr2=0x7f7080140530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:39.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f707d7fa640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7080124210 0x7f7080140530 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7070009710 tx=0x7f70700a8000 comp rx=0 tx=0).stop 2024-08-19T10:55:39.019 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:39.019 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:39.019 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:39.019 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:39.019 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:39.019 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 662136, 2024-08-19T10:55:39.019 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:39.019 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:39.019 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:39.019 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.020 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:39.021 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:39.022 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:39.023 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.024 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.025 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.027 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.028 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:39.029 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.030 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.031 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:39.033 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.034 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.035 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.036 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.037 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.038 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.040 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 4 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.041 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:39.042 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:39.042 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:39.042 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.042 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.042 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:39.042 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:39.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7086ea6640 1 -- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f706c03be00 msgr2=0x7f706c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7086ea6640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f706c03be00 0x7f706c03e2c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7078009f60 tx=0x7f707800d040 comp rx=0 tx=0).stop 2024-08-19T10:55:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7086ea6640 10 monclient: shutdown 2024-08-19T10:55:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7086ea6640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7086ea6640 1 -- 172.21.15.89:0/3505894738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70800a3bd0 msgr2=0x7f7080125a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7086ea6640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70800a3bd0 0x7f7080125a80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f707400c470 tx=0x7f707400c930 comp rx=0 tx=0).stop 2024-08-19T10:55:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7085443640 1 -- 172.21.15.89:0/3505894738 reap_dead start 2024-08-19T10:55:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7086ea6640 1 -- 172.21.15.89:0/3505894738 shutdown_connections 2024-08-19T10:55:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7086ea6640 1 --2- 172.21.15.89:0/3505894738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70800a3bd0 0x7f7080125a80 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7086ea6640 1 -- 172.21.15.89:0/3505894738 >> 172.21.15.89:0/3505894738 conn(0x7f708006de40 msgr2=0x7f708013f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7086ea6640 1 -- 172.21.15.89:0/3505894738 shutdown_connections 2024-08-19T10:55:39.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.006+0000 7f7086ea6640 1 -- 172.21.15.89:0/3505894738 wait complete. 2024-08-19T10:55:39.045 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:39.045 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-08-19T10:55:39.045 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:55:39.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient: get_monmap_and_config 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient: build_initial_monmap 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient: monmap: 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:39.217656+0000 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:39.217656+0000 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:39.212 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:39.213 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:39.213 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:39.213 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:39.213 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:39.213 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:39.213 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:39.213 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:39.213 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 1 Processor -- start 2024-08-19T10:55:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 1 -- start start 2024-08-19T10:55:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient: init 2024-08-19T10:55:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient: _add_conns ranks=[5,4,0,2,8,6,9,7,3,10,1] 2024-08-19T10:55:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6dd40a5270 0x7f6dd40a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient(hunting): picked mon.noname-h con 0x7f6dd40a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:39.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 0x7f6dd40a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient(hunting): picked mon.noname-c con 0x7f6dd40a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6dd41395b0 addr v1:172.21.15.89:6789/0 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dd40a8710 con 0x7f6dd41395b0 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6dd40a7650 con 0x7f6dd40a6d30 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6dd40a5a40 con 0x7f6dd40a5270 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6ddbf83640 10 monclient(hunting): authenticate will time out at 1989.034198s 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 0x7f6dd40a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dda520640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6dd41395b0 0x7f6dd4139a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:49468/0 (socket says 172.21.15.89:49468) 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dda520640 1 -- 172.21.15.89:0/2283897716 learned_addr learned my addr 172.21.15.89:0/2283897716 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 10 monclient(hunting): get_auth_request con 0x7f6dd40a6d30 auth_method 0 2024-08-19T10:55:39.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 10 monclient(hunting): handle_auth_done global_id 84433 payload 274 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd9d1f640 1 --2- 172.21.15.89:0/2283897716 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6dd40a5270 0x7f6dd40a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 1 -- 172.21.15.89:0/2283897716 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6dd40a5270 msgr2=0x7f6dd40a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 1 --2- 172.21.15.89:0/2283897716 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6dd40a5270 0x7f6dd40a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 1 -- 172.21.15.89:0/2283897716 >> v1:172.21.15.89:6789/0 conn(0x7f6dd41395b0 legacy=0x7f6dd4139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 10 monclient: _finish_hunting 0 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 1 monclient: found mon.noname-c 2024-08-19T10:55:39.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:39.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:39.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.214+0000 7f6dd951e640 1 -- 172.21.15.89:0/2283897716 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd40a81d0 con 0x7f6dd40a6d30 2024-08-19T10:55:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd951e640 1 -- 172.21.15.89:0/2283897716 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 msgr2=0x7f6dd40a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd951e640 1 --2- 172.21.15.89:0/2283897716 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 0x7f6dd40a7110 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40097f0 tx=0x7f6dc401c1c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=fcde9e6f03706e9a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 1 -- 172.21.15.89:0/2283897716 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dc40b0070 con 0x7f6dd40a6d30 2024-08-19T10:55:39.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 10 monclient: dump: 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.223 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 10 monclient: _finish_auth 0 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 10 monclient: _check_auth_tickets 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 5 monclient: authenticate success, global_id 84433 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 1 -- 172.21.15.89:0/2283897716 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc40aa810 con 0x7f6dd40a6d30 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:39.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 4 set_mon_vals no callback set 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 1 -- 172.21.15.89:0/2283897716 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dc40a93e0 con 0x7f6dd40a6d30 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6dd8d1d640 10 monclient: dump: 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.225 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 10 monclient: shutdown 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2283897716 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 msgr2=0x7f6dd40a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 1 --2- 172.21.15.89:0/2283897716 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 0x7f6dd40a7110 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40097f0 tx=0x7f6dc401c1c0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2283897716 shutdown_connections 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 1 --2- 172.21.15.89:0/2283897716 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 0x7f6dd40a7110 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 1 --2- 172.21.15.89:0/2283897716 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6dd40a5270 0x7f6dd40a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2283897716 >> 172.21.15.89:0/2283897716 conn(0x7f6dd4069a80 msgr2=0x7f6dd40abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2283897716 shutdown_connections 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.218+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2283897716 wait complete. 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient: build_initial_monmap 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient: monmap: 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:39.227 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 1 Processor -- start 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 1 -- start start 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient: init 2024-08-19T10:55:39.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient: _add_conns ranks=[6,4,1,0,2,5,7,8,3] 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6dd40a5270 0x7f6dd413fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient(hunting): picked mon.noname-d con 0x7f6dd40a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 0x7f6dd41d1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient(hunting): picked mon.noname-c con 0x7f6dd40a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dd41395b0 0x7f6dd413d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient(hunting): picked mon.noname-f con 0x7f6dd41395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd40a57d0 con 0x7f6dd41395b0 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6dd40a5650 con 0x7f6dd40a6d30 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6dd4144240 con 0x7f6dd40a5270 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6ddbf83640 10 monclient(hunting): authenticate will time out at 1989.041373s 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 0x7f6dd41d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd9d1f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6dd40a5270 0x7f6dd413fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 0x7f6dd41d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:35258/0 (socket says 172.21.15.89:35258) 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd9d1f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6dd40a5270 0x7f6dd413fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36564/0 (socket says 172.21.15.89:36564) 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 1 -- 172.21.15.89:0/2774257958 learned_addr learned my addr 172.21.15.89:0/2774257958 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 10 monclient(hunting): get_auth_request con 0x7f6dd40a6d30 auth_method 0 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd9d1f640 10 monclient(hunting): get_auth_request con 0x7f6dd40a5270 auth_method 0 2024-08-19T10:55:39.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd9d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd9d1f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd9d1f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd9d1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd9d1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd9d1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dda520640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dd41395b0 0x7f6dd413d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dda520640 10 monclient(hunting): get_auth_request con 0x7f6dd41395b0 auth_method 0 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dda520640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dda520640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dda520640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 10 monclient(hunting): handle_auth_done global_id 84442 payload 824 2024-08-19T10:55:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 1 -- 172.21.15.89:0/2774257958 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6dd40a5270 msgr2=0x7f6dd413fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6dd40a5270 0x7f6dd413fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 1 -- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dd41395b0 msgr2=0x7f6dd413d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dd41395b0 0x7f6dd413d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 10 monclient: _finish_hunting 0 2024-08-19T10:55:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 1 monclient: found mon.noname-c 2024-08-19T10:55:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd951e640 1 -- 172.21.15.89:0/2774257958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd4122ce0 con 0x7f6dd40a6d30 2024-08-19T10:55:39.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.222+0000 7f6dd9d1f640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6dd40a5270 0x7f6dd413fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd951e640 1 -- 172.21.15.89:0/2774257958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 msgr2=0x7f6dd41d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd951e640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 0x7f6dd41d1830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6dc401bff0 tx=0x7f6dc4018f60 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dc40b0040 con 0x7f6dd40a6d30 2024-08-19T10:55:39.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 10 monclient: dump: 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.237 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 10 monclient: _finish_auth 0 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc40b9070 con 0x7f6dd40a6d30 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dc4004250 con 0x7f6dd40a6d30 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:39.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 10 monclient: dump: 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.239 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6ddbf83640 5 monclient: authenticate success, global_id 84442 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6ddbf83640 10 monclient: _renew_subs 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6ddbf83640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2774257958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd40a7180 con 0x7f6dd40a6d30 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dcb7fe640 4 set_mon_vals no callback set 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6ddbf83640 10 monclient: _renew_subs 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6ddbf83640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2774257958 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd4122720 con 0x7f6dd40a6d30 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6ddbf83640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6ddbf83640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41d1d70 0x7f6dd402d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6ddbf83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2774257958 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd41c90d0 con 0x7f6dd41d1d70 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6dc4003620 con 0x7f6dd40a6d30 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd9d1f640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41d1d70 0x7f6dd402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd9d1f640 10 monclient: get_auth_request con 0x7f6dd41d1d70 auth_method 0 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd9d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd9d1f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd9d1f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6db803be00 0x7f6db803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f6dc40e9610 con 0x7f6dd40a6d30 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd9d1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd9d1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd9d1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dda520640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6db803be00 0x7f6db803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dda520640 10 monclient: get_auth_request con 0x7f6db803be00 auth_method 0 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd9d1f640 10 monclient(hunting): handle_auth_done global_id 114186 payload 824 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.226+0000 7f6dd9d1f640 1 -- 172.21.15.89:0/2774257958 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6dd4140ad0 con 0x7f6dd41d1d70 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dd9d1f640 1 -- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41d1d70 msgr2=0x7f6dd402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dd9d1f640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41d1d70 0x7f6dd402d1d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000fc70 tx=0x7f6dd000d590 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dda520640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6db803be00 0x7f6db803e2c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6dd413f0a0 tx=0x7f6dcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dd000fe80 con 0x7f6dd41d1d70 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dca7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dca7fc640 10 monclient: dump: 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.242 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6dd000fe80 con 0x7f6dd41d1d70 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dca7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dca7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41d1d70 msgr2=0x7f6dd402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dca7fc640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41d1d70 0x7f6dd402d1d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000fc70 tx=0x7f6dd000d590 comp rx=0 tx=0).stop 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41d1d70 msgr2=0x7f6dd402d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.230+0000 7f6dca7fc640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41d1d70 0x7f6dd402d1d0 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000fc70 tx=0x7f6dd000d590 comp rx=0 tx=0).stop 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.234+0000 7f6ddbf83640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.234+0000 7f6ddbf83640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41228f0 0x7f6dd402d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6ddbf83640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2774257958 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd40a2550 con 0x7f6dd41228f0 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41228f0 0x7f6dd402d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 10 monclient: get_auth_request con 0x7f6dd41228f0 auth_method 0 2024-08-19T10:55:39.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 10 monclient(hunting): handle_auth_done global_id 114195 payload 824 2024-08-19T10:55:39.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 1 -- 172.21.15.89:0/2774257958 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6dd40a67f0 con 0x7f6dd41228f0 2024-08-19T10:55:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 1 -- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41228f0 msgr2=0x7f6dd402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.238+0000 7f6dd9d1f640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41228f0 0x7f6dd402d710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000fc70 tx=0x7f6dd00046d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6dd0020bf0 con 0x7f6dd41228f0 2024-08-19T10:55:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dca7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:55:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dca7fc640 10 monclient: dump: 2024-08-19T10:55:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.246 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f6dd00293b0 con 0x7f6dd41228f0 2024-08-19T10:55:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dca7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dca7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41228f0 msgr2=0x7f6dd402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dca7fc640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41228f0 0x7f6dd402d710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000fc70 tx=0x7f6dd00046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dca7fc640 1 -- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41228f0 msgr2=0x7f6dd402d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:39.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dca7fc640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6dd41228f0 0x7f6dd402d710 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000fc70 tx=0x7f6dd00046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.248 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:39.248 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:39.248 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:39.248 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 19891, 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.249 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.250 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.251 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:39.252 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.253 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:39.254 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.256 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:39.257 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:39.258 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.259 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.271 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.272 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:39.272 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:39.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6db803be00 msgr2=0x7f6db803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6ddbf83640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6db803be00 0x7f6db803e2c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6dd413f0a0 tx=0x7f6dcc009040 comp rx=0 tx=0).stop 2024-08-19T10:55:39.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6ddbf83640 10 monclient: shutdown 2024-08-19T10:55:39.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6ddbf83640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:39.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2774257958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 msgr2=0x7f6dd41d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6ddbf83640 1 --2- 172.21.15.89:0/2774257958 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6dd40a6d30 0x7f6dd41d1830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6dc401bff0 tx=0x7f6dc4018f60 comp rx=0 tx=0).stop 2024-08-19T10:55:39.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dda520640 1 -- 172.21.15.89:0/2774257958 reap_dead start 2024-08-19T10:55:39.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6dda520640 1 -- 172.21.15.89:0/2774257958 reap_dead start 2024-08-19T10:55:39.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2774257958 shutdown_connections 2024-08-19T10:55:39.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2774257958 >> 172.21.15.89:0/2774257958 conn(0x7f6dd4069a80 msgr2=0x7f6dd40ab1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:39.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2774257958 shutdown_connections 2024-08-19T10:55:39.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.242+0000 7f6ddbf83640 1 -- 172.21.15.89:0/2774257958 wait complete. 2024-08-19T10:55:39.275 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:39.275 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-08-19T10:55:39.275 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:55:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient: get_monmap_and_config 2024-08-19T10:55:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient: build_initial_monmap 2024-08-19T10:55:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient: monmap: 2024-08-19T10:55:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:39.394133+0000 2024-08-19T10:55:39.388 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:39.394133+0000 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:39.389 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:39.390 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:39.390 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 1 Processor -- start 2024-08-19T10:55:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 1 -- start start 2024-08-19T10:55:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient: init 2024-08-19T10:55:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:39.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient: _add_conns ranks=[5,3,10,6,8,9,4,7,1,0,2] 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f32140a5270 0x7f32140a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient(hunting): picked mon.noname-h con 0x7f32140a5270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32140a6d30 0x7f32140a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient(hunting): picked mon.noname-g con 0x7f32140a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f32141395b0 0x7f3214139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient(hunting): picked mon.noname-f con 0x7f32141395b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f32140a7650 con 0x7f32141395b0 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f32140a8710 con 0x7f32140a6d30 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f32140a5a40 con 0x7f32140a5270 2024-08-19T10:55:39.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321a2de640 10 monclient(hunting): authenticate will time out at 1989.210851s 2024-08-19T10:55:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32140a6d30 0x7f32140a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f3213fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f32140a5270 0x7f32140a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321887b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f32141395b0 0x7f3214139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32140a6d30 0x7f32140a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:45430/0 (socket says 172.21.15.89:45430) 2024-08-19T10:55:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f3213fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f32140a5270 0x7f32140a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:57768/0 (socket says 172.21.15.89:57768) 2024-08-19T10:55:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321887b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f32141395b0 0x7f3214139a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:38424/0 (socket says 172.21.15.89:38424) 2024-08-19T10:55:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 1 -- 172.21.15.89:0/4011821152 learned_addr learned my addr 172.21.15.89:0/4011821152 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321887b640 10 monclient(hunting): get_auth_request con 0x7f32141395b0 auth_method 0 2024-08-19T10:55:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321887b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321887b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321887b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 10 monclient(hunting): get_auth_request con 0x7f32140a6d30 auth_method 0 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f3213fff640 10 monclient(hunting): get_auth_request con 0x7f32140a5270 auth_method 0 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f3213fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f3213fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f3213fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321887b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321887b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321887b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f3213fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f3213fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f3213fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 10 monclient(hunting): handle_auth_done global_id 114204 payload 274 2024-08-19T10:55:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 1 -- 172.21.15.89:0/4011821152 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f32140a5270 msgr2=0x7f32140a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 1 --2- 172.21.15.89:0/4011821152 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f32140a5270 0x7f32140a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 1 -- 172.21.15.89:0/4011821152 >> v2:172.21.15.190:3300/0 conn(0x7f32141395b0 msgr2=0x7f3214139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 1 --2- 172.21.15.89:0/4011821152 >> v2:172.21.15.190:3300/0 conn(0x7f32141395b0 0x7f3214139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 1 monclient: found mon.noname-g 2024-08-19T10:55:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:55:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f32137fe640 1 -- 172.21.15.89:0/4011821152 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32140a81d0 con 0x7f32140a6d30 2024-08-19T10:55:39.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.390+0000 7f321887b640 1 --2- 172.21.15.89:0/4011821152 >> v2:172.21.15.190:3300/0 conn(0x7f32141395b0 0x7f3214139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 1 -- 172.21.15.89:0/4011821152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32140a6d30 msgr2=0x7f32140a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 1 --2- 172.21.15.89:0/4011821152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32140a6d30 0x7f32140a7110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3204009e50 tx=0x7f320401c8f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3e318309d05ec1fc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 1 -- 172.21.15.89:0/4011821152 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32040090d0 con 0x7f32140a6d30 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 10 monclient: dump: 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.404 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 10 monclient: _finish_auth 0 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 5 monclient: authenticate success, global_id 114204 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:39.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 1 -- 172.21.15.89:0/4011821152 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32040a5ba0 con 0x7f32140a6d30 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 1 -- 172.21.15.89:0/4011821152 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32040a5d90 con 0x7f32140a6d30 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 4 set_mon_vals no callback set 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3212ffd640 10 monclient: dump: 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.406 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient: shutdown 2024-08-19T10:55:39.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 -- 172.21.15.89:0/4011821152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32140a6d30 msgr2=0x7f32140a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 --2- 172.21.15.89:0/4011821152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32140a6d30 0x7f32140a7110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3204009e50 tx=0x7f320401c8f0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 -- 172.21.15.89:0/4011821152 shutdown_connections 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 --2- 172.21.15.89:0/4011821152 >> v2:172.21.15.190:3300/0 conn(0x7f32141395b0 0x7f3214139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 --2- 172.21.15.89:0/4011821152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32140a6d30 0x7f32140a7110 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 --2- 172.21.15.89:0/4011821152 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f32140a5270 0x7f32140a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 -- 172.21.15.89:0/4011821152 >> 172.21.15.89:0/4011821152 conn(0x7f3214069a80 msgr2=0x7f32140abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 -- 172.21.15.89:0/4011821152 shutdown_connections 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 -- 172.21.15.89:0/4011821152 wait complete. 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient: build_initial_monmap 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient: monmap: 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:39.408 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:39.409 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 Processor -- start 2024-08-19T10:55:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 -- start start 2024-08-19T10:55:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient: init 2024-08-19T10:55:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient: _add_conns ranks=[1,4,7,2,5,3,6,8,0] 2024-08-19T10:55:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32140a5270 0x7f3214133550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient(hunting): picked mon.noname-f con 0x7f32140a5270 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:55:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32140a6d30 0x7f3214133a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient(hunting): picked mon.noname-c con 0x7f32140a6d30 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:39.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32141395b0 0x7f3214133fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient(hunting): picked mon.noname-i con 0x7f32141395b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32140a5650 con 0x7f32140a5270 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f321413a350 con 0x7f32140a6d30 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f32140a57d0 con 0x7f32141395b0 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321a2de640 10 monclient(hunting): authenticate will time out at 1989.219079s 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32140a6d30 0x7f3214133a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32140a6d30 0x7f3214133a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:35272/0 (socket says 172.21.15.89:35272) 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 1 -- 172.21.15.89:0/2268009492 learned_addr learned my addr 172.21.15.89:0/2268009492 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:39.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 10 monclient(hunting): get_auth_request con 0x7f32140a6d30 auth_method 0 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3213fff640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32140a5270 0x7f3214133550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321887b640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32141395b0 0x7f3214133fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3213fff640 10 monclient(hunting): get_auth_request con 0x7f32140a5270 auth_method 0 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3213fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3213fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f3213fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 10 monclient(hunting): handle_auth_done global_id 84451 payload 824 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 1 -- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32141395b0 msgr2=0x7f3214133fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32141395b0 0x7f3214133fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 1 -- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32140a5270 msgr2=0x7f3214133550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32140a5270 0x7f3214133550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 1 monclient: found mon.noname-c 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f32137fe640 1 -- 172.21.15.89:0/2268009492 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321409f680 con 0x7f32140a6d30 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321887b640 10 monclient: get_auth_request con 0x7f32141395b0 auth_method 0 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.398+0000 7f321887b640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f32141395b0 0x7f3214133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f32137fe640 1 -- 172.21.15.89:0/2268009492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32140a6d30 msgr2=0x7f3214133a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f32137fe640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32140a6d30 0x7f3214133a90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f320401c720 tx=0x7f3204019690 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f32040bc020 con 0x7f32140a6d30 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 10 monclient: dump: 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.414 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 10 monclient: _finish_auth 0 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3204003bb0 con 0x7f32140a6d30 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:39.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3204003d60 con 0x7f32140a6d30 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 10 monclient: dump: 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.416 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.417 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3211ffb640 4 set_mon_vals no callback set 2024-08-19T10:55:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f321a2de640 5 monclient: authenticate success, global_id 84451 2024-08-19T10:55:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f321a2de640 10 monclient: _renew_subs 2024-08-19T10:55:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f321a2de640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:39.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f321a2de640 1 -- 172.21.15.89:0/2268009492 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321409f960 con 0x7f32140a6d30 2024-08-19T10:55:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f321a2de640 10 monclient: _renew_subs 2024-08-19T10:55:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f321a2de640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f321a2de640 1 -- 172.21.15.89:0/2268009492 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f3214135720 con 0x7f32140a6d30 2024-08-19T10:55:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f321a2de640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f321a2de640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d1a0 0x7f321402d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f321a2de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f321a2de640 1 -- 172.21.15.89:0/2268009492 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f321409e160 con 0x7f321409d1a0 2024-08-19T10:55:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f32040bf020 con 0x7f32140a6d30 2024-08-19T10:55:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3213fff640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d1a0 0x7f321402d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3213fff640 10 monclient: get_auth_request con 0x7f321409d1a0 auth_method 0 2024-08-19T10:55:39.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3213fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3213fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3213fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.402+0000 7f3210ff9640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31f403bd60 0x7f31f403e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3213fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3213fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f32040ea6e0 con 0x7f32140a6d30 2024-08-19T10:55:39.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3213fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f321887b640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31f403bd60 0x7f31f403e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3213fff640 10 monclient(hunting): handle_auth_done global_id 84371 payload 824 2024-08-19T10:55:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3213fff640 1 -- 172.21.15.89:0/2268009492 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32140a72a0 con 0x7f321409d1a0 2024-08-19T10:55:39.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f321887b640 10 monclient: get_auth_request con 0x7f31f403bd60 auth_method 0 2024-08-19T10:55:39.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3213fff640 1 -- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d1a0 msgr2=0x7f321402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3213fff640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d1a0 0x7f321402d1d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f31fc011510 tx=0x7f31fc0119d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31fc0044a0 con 0x7f321409d1a0 2024-08-19T10:55:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:55:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 10 monclient: dump: 2024-08-19T10:55:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.423 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f321887b640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31f403bd60 0x7f31f403e220 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f321413f490 tx=0x7f320c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.424 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f31fc004030 con 0x7f321409d1a0 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d1a0 msgr2=0x7f321402d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d1a0 0x7f321402d1d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f31fc011510 tx=0x7f31fc0119d0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d1a0 msgr2=0x7f321402d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.406+0000 7f3210ff9640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d1a0 0x7f321402d1d0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f31fc011510 tx=0x7f31fc0119d0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f321a2de640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f321a2de640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d580 0x7f321402d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f321a2de640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f321a2de640 1 -- 172.21.15.89:0/2268009492 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f32140a67f0 con 0x7f321409d580 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f3213fff640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d580 0x7f321402d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f3213fff640 10 monclient: get_auth_request con 0x7f321409d580 auth_method 0 2024-08-19T10:55:39.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f3213fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f3213fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f3213fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f3213fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f3213fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f3213fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f3213fff640 10 monclient(hunting): handle_auth_done global_id 84380 payload 824 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.414+0000 7f3213fff640 1 -- 172.21.15.89:0/2268009492 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3214064590 con 0x7f321409d580 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3213fff640 1 -- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d580 msgr2=0x7f321402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3213fff640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d580 0x7f321402d710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f31fc011510 tx=0x7f31fc0046d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31fc003a90 con 0x7f321409d580 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3210ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3210ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3210ff9640 10 monclient: dump: 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.427 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9435 (secure 0 0 0) 0x7f31fc003620 con 0x7f321409d580 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3210ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3210ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d580 msgr2=0x7f321402d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3210ff9640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d580 0x7f321402d710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f31fc011510 tx=0x7f31fc0046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3210ff9640 1 -- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d580 msgr2=0x7f321402d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:39.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f3210ff9640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f321409d580 0x7f321402d710 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f31fc011510 tx=0x7f31fc0046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.429 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 147712, 2024-08-19T10:55:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:55:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:39.454 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321a2de640 1 -- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31f403bd60 msgr2=0x7f31f403e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321a2de640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31f403bd60 0x7f31f403e220 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f321413f490 tx=0x7f320c008040 comp rx=0 tx=0).stop 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321a2de640 10 monclient: shutdown 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321a2de640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321a2de640 1 -- 172.21.15.89:0/2268009492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32140a6d30 msgr2=0x7f3214133a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321a2de640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32140a6d30 0x7f3214133a90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f320401c720 tx=0x7f3204019690 comp rx=0 tx=0).stop 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321887b640 1 -- 172.21.15.89:0/2268009492 reap_dead start 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321a2de640 1 -- 172.21.15.89:0/2268009492 shutdown_connections 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321a2de640 1 --2- 172.21.15.89:0/2268009492 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f32140a6d30 0x7f3214133a90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321a2de640 1 -- 172.21.15.89:0/2268009492 >> 172.21.15.89:0/2268009492 conn(0x7f3214069a80 msgr2=0x7f32140ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321a2de640 1 -- 172.21.15.89:0/2268009492 shutdown_connections 2024-08-19T10:55:39.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.418+0000 7f321a2de640 1 -- 172.21.15.89:0/2268009492 wait complete. 2024-08-19T10:55:39.457 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:39.457 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-08-19T10:55:39.457 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-08-19T10:55:39.461 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.461 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257232037}) 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3859370622}) 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257232037}) 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3231959416}) 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257232037}) 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4096902759}) 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257232037}) 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=349229431}) 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.462 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257232037}) 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=825066630}) 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257232037}) 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1016874654}) 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257232037}) 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3892287158}) 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1257232037}) 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.446+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1508741309}) 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2597657008}) 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4199493608}) 2024-08-19T10:55:39.463 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2597657008}) 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3507262955}) 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2597657008}) 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1470800715}) 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2597657008}) 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2491606893}) 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2597657008}) 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2531727902}) 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2597657008}) 2024-08-19T10:55:39.464 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1580537379}) 2024-08-19T10:55:39.465 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.465 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2597657008}) 2024-08-19T10:55:39.465 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=229866955}) 2024-08-19T10:55:39.465 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.465 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2597657008}) 2024-08-19T10:55:39.465 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.450+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3555622754}) 2024-08-19T10:55:39.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.472 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=9944608}) 2024-08-19T10:55:39.473 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1210236267}) 2024-08-19T10:55:39.473 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.473 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=9944608}) 2024-08-19T10:55:39.473 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1058267691}) 2024-08-19T10:55:39.473 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.473 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=9944608}) 2024-08-19T10:55:39.474 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2470283266}) 2024-08-19T10:55:39.474 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.474 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=9944608}) 2024-08-19T10:55:39.474 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2665299682}) 2024-08-19T10:55:39.474 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.474 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=9944608}) 2024-08-19T10:55:39.475 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3956558430}) 2024-08-19T10:55:39.475 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.475 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=9944608}) 2024-08-19T10:55:39.475 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2367078699}) 2024-08-19T10:55:39.475 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.475 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=9944608}) 2024-08-19T10:55:39.475 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=483507263}) 2024-08-19T10:55:39.476 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.476 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=9944608}) 2024-08-19T10:55:39.476 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.474+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1049968015}) 2024-08-19T10:55:39.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient: get_monmap_and_config 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient: build_initial_monmap 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient: monmap: 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:39.585675+0000 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:39.585675+0000 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:39.580 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:39.581 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:39.581 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 1 Processor -- start 2024-08-19T10:55:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 1 -- start start 2024-08-19T10:55:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient: init 2024-08-19T10:55:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient: _add_conns ranks=[2,10,3,0,9,4,7,1,6,8,5] 2024-08-19T10:55:39.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b38069300 0x7f9b3812d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient(hunting): picked mon.noname-b con 0x7f9b38069300 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:39.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9b3812d9d0 0x7f9b3812ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient(hunting): picked mon.noname-f con 0x7f9b3812d9d0 addr v2:172.21.15.190:3300/0 2024-08-19T10:55:39.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b38133b30 0x7f9b38133fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient(hunting): picked mon.noname-g con 0x7f9b38133b30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:39.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9b380696e0 con 0x7f9b3812d9d0 2024-08-19T10:55:39.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f9b380a27d0 con 0x7f9b38069300 2024-08-19T10:55:39.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3809c3d0 con 0x7f9b38133b30 2024-08-19T10:55:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3ff78640 10 monclient(hunting): authenticate will time out at 1989.402202s 2024-08-19T10:55:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b38069300 0x7f9b3812d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b38069300 0x7f9b3812d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3301/0 says I am v2:172.21.15.89:60934/0 (socket says 172.21.15.89:60934) 2024-08-19T10:55:39.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 1 -- 172.21.15.89:0/3465982529 learned_addr learned my addr 172.21.15.89:0/3465982529 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 10 monclient(hunting): get_auth_request con 0x7f9b38069300 auth_method 0 2024-08-19T10:55:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3e515640 1 --2- 172.21.15.89:0/3465982529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b38133b30 0x7f9b38133fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3d513640 1 --2- 172.21.15.89:0/3465982529 >> v2:172.21.15.190:3300/0 conn(0x7f9b3812d9d0 0x7f9b3812ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3e515640 10 monclient(hunting): get_auth_request con 0x7f9b38133b30 auth_method 0 2024-08-19T10:55:39.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3e515640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3e515640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3e515640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3d513640 10 monclient(hunting): get_auth_request con 0x7f9b3812d9d0 auth_method 0 2024-08-19T10:55:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3d513640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3d513640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3d513640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 10 monclient(hunting): handle_auth_done global_id 74621 payload 274 2024-08-19T10:55:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 1 -- 172.21.15.89:0/3465982529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b38133b30 msgr2=0x7f9b38133fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 1 --2- 172.21.15.89:0/3465982529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b38133b30 0x7f9b38133fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 1 -- 172.21.15.89:0/3465982529 >> v2:172.21.15.190:3300/0 conn(0x7f9b3812d9d0 msgr2=0x7f9b3812ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:39.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 1 --2- 172.21.15.89:0/3465982529 >> v2:172.21.15.190:3300/0 conn(0x7f9b3812d9d0 0x7f9b3812ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 10 monclient: _finish_hunting 0 2024-08-19T10:55:39.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 1 monclient: found mon.noname-b 2024-08-19T10:55:39.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:39.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.89:3301/0 2024-08-19T10:55:39.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.582+0000 7f9b3dd14640 1 -- 172.21.15.89:0/3465982529 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3809c6b0 con 0x7f9b38069300 2024-08-19T10:55:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3dd14640 1 -- 172.21.15.89:0/3465982529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b38069300 msgr2=0x7f9b3812d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3dd14640 1 --2- 172.21.15.89:0/3465982529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b38069300 0x7f9b3812d490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9b340098a0 tx=0x7f9b3401c140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ac32325f7a1ae6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 1 -- 172.21.15.89:0/3465982529 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b340b0070 con 0x7f9b38069300 2024-08-19T10:55:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-08-19T10:55:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 10 monclient: dump: 2024-08-19T10:55:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.611 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.612 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 10 monclient: _finish_auth 0 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 10 monclient: _check_auth_tickets 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 1 -- 172.21.15.89:0/3465982529 <== mon.2 v2:172.21.15.89:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b340a9980 con 0x7f9b38069300 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 1 -- 172.21.15.89:0/3465982529 <== mon.2 v2:172.21.15.89:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b340aa3a0 con 0x7f9b38069300 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3ff78640 5 monclient: authenticate success, global_id 74621 2024-08-19T10:55:39.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3ff78640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3ff78640 4 set_mon_vals no callback set 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.586+0000 7f9b3cd12640 10 monclient: dump: 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.614 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient: shutdown 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 -- 172.21.15.89:0/3465982529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b38069300 msgr2=0x7f9b3812d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 --2- 172.21.15.89:0/3465982529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b38069300 0x7f9b3812d490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9b340098a0 tx=0x7f9b3401c140 comp rx=0 tx=0).stop 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 -- 172.21.15.89:0/3465982529 shutdown_connections 2024-08-19T10:55:39.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 --2- 172.21.15.89:0/3465982529 >> v2:172.21.15.190:3300/0 conn(0x7f9b3812d9d0 0x7f9b3812ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 --2- 172.21.15.89:0/3465982529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b38133b30 0x7f9b38133fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 --2- 172.21.15.89:0/3465982529 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f9b38069300 0x7f9b3812d490 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 -- 172.21.15.89:0/3465982529 >> 172.21.15.89:0/3465982529 conn(0x7f9b380a83d0 msgr2=0x7f9b380a87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 -- 172.21.15.89:0/3465982529 shutdown_connections 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 -- 172.21.15.89:0/3465982529 wait complete. 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient: build_initial_monmap 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient: monmap: 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.616 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 Processor -- start 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 -- start start 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient: init 2024-08-19T10:55:39.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient: _add_conns ranks=[4,7,5,1,3,2,6,8,0] 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b38069300 0x7f9b3809b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient(hunting): picked mon.noname-c con 0x7f9b38069300 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b3812d9d0 0x7f9b3809b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient(hunting): picked mon.noname-i con 0x7f9b3812d9d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b38133b30 0x7f9b38139070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient(hunting): picked mon.noname-h con 0x7f9b38133b30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f9b38140fd0 con 0x7f9b38069300 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f9b38140e50 con 0x7f9b38133b30 2024-08-19T10:55:39.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9b38141150 con 0x7f9b3812d9d0 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3ff78640 10 monclient(hunting): authenticate will time out at 1989.410612s 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b38069300 0x7f9b3809b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b38069300 0x7f9b3809b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:35274/0 (socket says 172.21.15.89:35274) 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 1 -- 172.21.15.89:0/602551991 learned_addr learned my addr 172.21.15.89:0/602551991 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 10 monclient(hunting): get_auth_request con 0x7f9b38069300 auth_method 0 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 10 monclient(hunting): handle_auth_done global_id 84460 payload 824 2024-08-19T10:55:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3e515640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b38133b30 0x7f9b38139070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 1 -- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b3812d9d0 msgr2=0x7f9b3809b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b3812d9d0 0x7f9b3809b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 1 -- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b38133b30 msgr2=0x7f9b38139070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f9b38133b30 0x7f9b38139070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 10 monclient: _finish_hunting 0 2024-08-19T10:55:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 1 monclient: found mon.noname-c 2024-08-19T10:55:39.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.590+0000 7f9b3dd14640 1 -- 172.21.15.89:0/602551991 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3813be20 con 0x7f9b38069300 2024-08-19T10:55:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3dd14640 1 -- 172.21.15.89:0/602551991 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b38069300 msgr2=0x7f9b3809b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3dd14640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b38069300 0x7f9b3809b390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b340ae040 tx=0x7f9b34018ee0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b340b9070 con 0x7f9b38069300 2024-08-19T10:55:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 10 monclient: dump: 2024-08-19T10:55:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.621 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.622 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.622 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.622 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.622 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.622 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.622 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.622 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.623 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.624 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 10 monclient: _finish_auth 0 2024-08-19T10:55:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:55:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b34004250 con 0x7f9b38069300 2024-08-19T10:55:39.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3ff78640 5 monclient: authenticate success, global_id 84460 2024-08-19T10:55:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3ff78640 10 monclient: _renew_subs 2024-08-19T10:55:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3ff78640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3ff78640 1 -- 172.21.15.89:0/602551991 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b3813df00 con 0x7f9b38069300 2024-08-19T10:55:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b340b0040 con 0x7f9b38069300 2024-08-19T10:55:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2ffff640 4 set_mon_vals no callback set 2024-08-19T10:55:39.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 10 monclient: dump: 2024-08-19T10:55:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.626 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3ff78640 10 monclient: _renew_subs 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3ff78640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3ff78640 1 -- 172.21.15.89:0/602551991 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3813bb40 con 0x7f9b38069300 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3ff78640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3ff78640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b381395b0 0x7f9b38139990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3ff78640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3ff78640 1 -- 172.21.15.89:0/602551991 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9b3813c790 con 0x7f9b381395b0 2024-08-19T10:55:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9b34003620 con 0x7f9b38069300 2024-08-19T10:55:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3d513640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b381395b0 0x7f9b38139990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3d513640 10 monclient: get_auth_request con 0x7f9b381395b0 auth_method 0 2024-08-19T10:55:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3d513640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3d513640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3d513640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9b1c03be30 0x7f9b1c03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f9b340e8990 con 0x7f9b38069300 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3d513640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3d513640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3d513640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3e515640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9b1c03be30 0x7f9b1c03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3e515640 10 monclient: get_auth_request con 0x7f9b1c03be30 auth_method 0 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3d513640 10 monclient(hunting): handle_auth_done global_id 114190 payload 824 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.594+0000 7f9b3d513640 1 -- 172.21.15.89:0/602551991 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b3806d080 con 0x7f9b381395b0 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b3d513640 1 -- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b381395b0 msgr2=0x7f9b38139990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b3d513640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b381395b0 0x7f9b38139990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b381307e0 tx=0x7f9b2800d960 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b3e515640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9b1c03be30 0x7f9b1c03e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009d90 tx=0x7f9b3000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b28012c70 con 0x7f9b381395b0 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b2e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b2e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:55:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b2e7fc640 10 monclient: dump: 2024-08-19T10:55:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.630 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.631 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9b28019070 con 0x7f9b381395b0 2024-08-19T10:55:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b2e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:39.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b2e7fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b381395b0 msgr2=0x7f9b38139990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b2e7fc640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b381395b0 0x7f9b38139990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b381307e0 tx=0x7f9b2800d960 comp rx=0 tx=0).stop 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b381395b0 msgr2=0x7f9b38139990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.598+0000 7f9b2e7fc640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b381395b0 0x7f9b38139990 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b381307e0 tx=0x7f9b2800d960 comp rx=0 tx=0).stop 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3ff78640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3ff78640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b3812e4a0 0x7f9b3812e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3ff78640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3ff78640 1 -- 172.21.15.89:0/602551991 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9b38139ed0 con 0x7f9b3812e4a0 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3d513640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b3812e4a0 0x7f9b3812e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3d513640 10 monclient: get_auth_request con 0x7f9b3812e4a0 auth_method 0 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3d513640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3d513640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3d513640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3d513640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3d513640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3d513640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3d513640 10 monclient(hunting): handle_auth_done global_id 114199 payload 824 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.606+0000 7f9b3d513640 1 -- 172.21.15.89:0/602551991 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9b3813b710 con 0x7f9b3812e4a0 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3d513640 1 -- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b3812e4a0 msgr2=0x7f9b3812e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3d513640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b3812e4a0 0x7f9b3812e8b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b28018000 tx=0x7f9b280046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9b280288d0 con 0x7f9b3812e4a0 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b2e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b2e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b2e7fc640 10 monclient: dump: 2024-08-19T10:55:39.633 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.634 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.635 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9267 (secure 0 0 0) 0x7f9b28014430 con 0x7f9b3812e4a0 2024-08-19T10:55:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b2e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:55:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b2e7fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b3812e4a0 msgr2=0x7f9b3812e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b2e7fc640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b3812e4a0 0x7f9b3812e8b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b28018000 tx=0x7f9b280046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b2e7fc640 1 -- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b3812e4a0 msgr2=0x7f9b3812e8b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:39.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b2e7fc640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f9b3812e4a0 0x7f9b3812e8b0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b28018000 tx=0x7f9b280046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.636 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:55:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 140, 2024-08-19T10:55:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 20259, 2024-08-19T10:55:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:55:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:55:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:55:39.636 INFO:teuthology.orchestra.run.smithi089.stdout: 2, 2024-08-19T10:55:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:55:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:55:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:55:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:55:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: 7, 2024-08-19T10:55:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:55:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 6, 2024-08-19T10:55:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:55:39.637 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.638 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.639 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:55:39.640 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:55:39.641 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:55:39.642 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.644 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:55:39.645 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.646 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:55:39.647 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.647 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.647 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:55:39.647 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.647 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.647 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:55:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:55:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:55:39.648 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.649 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:55:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:55:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.651 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:55:39.652 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.653 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.654 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:55:39.655 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:55:39.656 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:55:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:55:39.658 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:55:39.659 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:39.660 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:55:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:55:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:55:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:55:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:55:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:55:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:55:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:55:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:55:39.661 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:55:39.661 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:39.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3ff78640 1 -- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9b1c03be30 msgr2=0x7f9b1c03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3ff78640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f9b1c03be30 0x7f9b1c03e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009d90 tx=0x7f9b3000b040 comp rx=0 tx=0).stop 2024-08-19T10:55:39.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3ff78640 10 monclient: shutdown 2024-08-19T10:55:39.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3ff78640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:39.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3ff78640 1 -- 172.21.15.89:0/602551991 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b38069300 msgr2=0x7f9b3809b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3ff78640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b38069300 0x7f9b3809b390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b340ae040 tx=0x7f9b34018ee0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3e515640 1 -- 172.21.15.89:0/602551991 reap_dead start 2024-08-19T10:55:39.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3ff78640 1 -- 172.21.15.89:0/602551991 shutdown_connections 2024-08-19T10:55:39.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3ff78640 1 --2- 172.21.15.89:0/602551991 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f9b38069300 0x7f9b3809b390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3ff78640 1 -- 172.21.15.89:0/602551991 >> 172.21.15.89:0/602551991 conn(0x7f9b380a83d0 msgr2=0x7f9b3813d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:39.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3ff78640 1 -- 172.21.15.89:0/602551991 shutdown_connections 2024-08-19T10:55:39.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.610+0000 7f9b3ff78640 1 -- 172.21.15.89:0/602551991 wait complete. 2024-08-19T10:55:39.664 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-08-19T10:55:39.665 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b', 'i'] 2024-08-19T10:55:39.665 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-08-19T10:55:39.665 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-08-19T10:55:39.665 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b store 2024-08-19T10:55:39.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b sync_force --yes-i-really-mean-it 2024-08-19T10:55:39.669 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2932897731}) 2024-08-19T10:55:39.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2385237400}) 2024-08-19T10:55:39.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2932897731}) 2024-08-19T10:55:39.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2333800607}) 2024-08-19T10:55:39.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.670 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2932897731}) 2024-08-19T10:55:39.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1822129435}) 2024-08-19T10:55:39.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2932897731}) 2024-08-19T10:55:39.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3044864050}) 2024-08-19T10:55:39.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2932897731}) 2024-08-19T10:55:39.671 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3416346311}) 2024-08-19T10:55:39.672 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.672 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2932897731}) 2024-08-19T10:55:39.672 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=417462917}) 2024-08-19T10:55:39.672 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.672 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2932897731}) 2024-08-19T10:55:39.672 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=751105217}) 2024-08-19T10:55:39.672 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.672 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2932897731}) 2024-08-19T10:55:39.672 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.650+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4162414417}) 2024-08-19T10:55:39.673 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.673 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1818978624}) 2024-08-19T10:55:39.673 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3298385360}) 2024-08-19T10:55:39.673 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.673 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1818978624}) 2024-08-19T10:55:39.673 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=594493142}) 2024-08-19T10:55:39.673 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.673 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1818978624}) 2024-08-19T10:55:39.673 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=880424691}) 2024-08-19T10:55:39.673 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.673 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1818978624}) 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=809818340}) 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1818978624}) 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3474760285}) 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1818978624}) 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1227443828}) 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1818978624}) 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=834836379}) 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.674 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1818978624}) 2024-08-19T10:55:39.675 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.666+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1258031535}) 2024-08-19T10:55:39.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient: get_monmap_and_config 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient: build_initial_monmap 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient: monmap: 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:39.806664+0000 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:39.806664+0000 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:39.801 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:39.802 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:39.802 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 1 Processor -- start 2024-08-19T10:55:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 1 -- start start 2024-08-19T10:55:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient: init 2024-08-19T10:55:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient: _add_conns ranks=[8,10,9,4,2,1,7,0,6,5,3] 2024-08-19T10:55:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 0x7fa69c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient(hunting): picked mon.noname-e con 0x7fa69c0a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa69c0a6d30 0x7fa69c0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient(hunting): picked mon.noname-f con 0x7fa69c0a6d30 addr v2:172.21.15.190:3300/0 2024-08-19T10:55:39.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa69c1395b0 0x7fa69c139a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient(hunting): picked mon.noname-a con 0x7fa69c1395b0 addr v2:172.21.15.89:3300/0 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fa69c0a7650 con 0x7fa69c1395b0 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa69c0a8710 con 0x7fa69c0a6d30 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa69c0a5a40 con 0x7fa69c0a5270 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa6a213f640 10 monclient(hunting): authenticate will time out at 1989.623378s 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 0x7fa69c0a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 0x7fa69c0a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:59940/0 (socket says 172.21.15.89:59940) 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69b7fe640 1 -- 172.21.15.89:0/3923964611 learned_addr learned my addr 172.21.15.89:0/3923964611 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:39.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69b7fe640 10 monclient(hunting): get_auth_request con 0x7fa69c0a5270 auth_method 0 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69bfff640 1 --2- 172.21.15.89:0/3923964611 >> v2:172.21.15.89:3300/0 conn(0x7fa69c1395b0 0x7fa69c139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69bfff640 10 monclient(hunting): get_auth_request con 0x7fa69c1395b0 auth_method 0 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69affd640 1 --2- 172.21.15.89:0/3923964611 >> v2:172.21.15.190:3300/0 conn(0x7fa69c0a6d30 0x7fa69c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69affd640 10 monclient(hunting): get_auth_request con 0x7fa69c0a6d30 auth_method 0 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.802+0000 7fa69affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.806+0000 7fa69b7fe640 10 monclient(hunting): handle_auth_done global_id 114218 payload 274 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.806+0000 7fa69b7fe640 1 -- 172.21.15.89:0/3923964611 >> v2:172.21.15.190:3300/0 conn(0x7fa69c0a6d30 msgr2=0x7fa69c0a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.806+0000 7fa69b7fe640 1 --2- 172.21.15.89:0/3923964611 >> v2:172.21.15.190:3300/0 conn(0x7fa69c0a6d30 0x7fa69c0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.806+0000 7fa69b7fe640 1 -- 172.21.15.89:0/3923964611 >> v2:172.21.15.89:3300/0 conn(0x7fa69c1395b0 msgr2=0x7fa69c139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.806+0000 7fa69b7fe640 1 --2- 172.21.15.89:0/3923964611 >> v2:172.21.15.89:3300/0 conn(0x7fa69c1395b0 0x7fa69c139a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.806+0000 7fa69b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:39.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.806+0000 7fa69b7fe640 1 monclient: found mon.noname-e 2024-08-19T10:55:39.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.806+0000 7fa69b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:39.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.806+0000 7fa69b7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:55:39.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.806+0000 7fa69b7fe640 1 -- 172.21.15.89:0/3923964611 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69c0a81d0 con 0x7fa69c0a5270 2024-08-19T10:55:39.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.806+0000 7fa69bfff640 1 --2- 172.21.15.89:0/3923964611 >> v2:172.21.15.89:3300/0 conn(0x7fa69c1395b0 0x7fa69c139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:39.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 1 -- 172.21.15.89:0/3923964611 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 msgr2=0x7fa69c0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 1 --2- 172.21.15.89:0/3923964611 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 0x7fa69c0a67f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0a34f0 tx=0x7fa68c01c3e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1ca87fe902299366 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 1 -- 172.21.15.89:0/3923964611 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa68c0b1070 con 0x7fa69c0a5270 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 10 monclient: dump: 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.813 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 5 monclient: authenticate success, global_id 114218 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 1 -- 172.21.15.89:0/3923964611 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68c0ab810 con 0x7fa69c0a5270 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:39.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 1 -- 172.21.15.89:0/3923964611 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa68c0aa300 con 0x7fa69c0a5270 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69a7fc640 10 monclient: dump: 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.815 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 4 set_mon_vals no callback set 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient: shutdown 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 -- 172.21.15.89:0/3923964611 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 msgr2=0x7fa69c0a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 --2- 172.21.15.89:0/3923964611 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 0x7fa69c0a67f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0a34f0 tx=0x7fa68c01c3e0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 -- 172.21.15.89:0/3923964611 shutdown_connections 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 --2- 172.21.15.89:0/3923964611 >> v2:172.21.15.89:3300/0 conn(0x7fa69c1395b0 0x7fa69c139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 --2- 172.21.15.89:0/3923964611 >> v2:172.21.15.190:3300/0 conn(0x7fa69c0a6d30 0x7fa69c0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 --2- 172.21.15.89:0/3923964611 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 0x7fa69c0a67f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 -- 172.21.15.89:0/3923964611 >> 172.21.15.89:0/3923964611 conn(0x7fa69c069a80 msgr2=0x7fa69c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 -- 172.21.15.89:0/3923964611 shutdown_connections 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 -- 172.21.15.89:0/3923964611 wait complete. 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient: build_initial_monmap 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient: monmap: 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:39.817 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 Processor -- start 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 -- start start 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient: init 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:39.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient: _add_conns ranks=[8,3,1,0,4,6,2,7,5] 2024-08-19T10:55:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 0x7fa69c0a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient(hunting): picked mon.noname-e con 0x7fa69c0a5270 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa69c0a6d30 0x7fa69c0a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient(hunting): picked mon.noname-g con 0x7fa69c0a6d30 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa69c1395b0 0x7fa69c09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient(hunting): picked mon.noname-f con 0x7fa69c1395b0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:55:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c0a5650 con 0x7fa69c1395b0 2024-08-19T10:55:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa69c13a360 con 0x7fa69c0a6d30 2024-08-19T10:55:39.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fa69c0a57d0 con 0x7fa69c0a5270 2024-08-19T10:55:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa6a213f640 10 monclient(hunting): authenticate will time out at 1989.630936s 2024-08-19T10:55:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 0x7fa69c0a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 0x7fa69c0a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:59952/0 (socket says 172.21.15.89:59952) 2024-08-19T10:55:39.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 1 -- 172.21.15.89:0/3814396076 learned_addr learned my addr 172.21.15.89:0/3814396076 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 10 monclient(hunting): get_auth_request con 0x7fa69c0a5270 auth_method 0 2024-08-19T10:55:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 10 monclient(hunting): handle_auth_done global_id 114227 payload 824 2024-08-19T10:55:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 1 -- 172.21.15.89:0/3814396076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa69c0a6d30 msgr2=0x7fa69c0a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69bfff640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa69c1395b0 0x7fa69c09c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69affd640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa69c0a6d30 0x7fa69c0a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa69c0a6d30 0x7fa69c0a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 1 -- 172.21.15.89:0/3814396076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa69c1395b0 msgr2=0x7fa69c09c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa69c1395b0 0x7fa69c09c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 1 monclient: found mon.noname-e 2024-08-19T10:55:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:55:39.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.810+0000 7fa69b7fe640 1 -- 172.21.15.89:0/3814396076 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69c1d1600 con 0x7fa69c0a5270 2024-08-19T10:55:39.823 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1082505958}) 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=454260463}) 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1082505958}) 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3197939692}) 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1082505958}) 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=60149986}) 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1082505958}) 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=121600292}) 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1082505958}) 2024-08-19T10:55:39.824 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1133503303}) 2024-08-19T10:55:39.825 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.825 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1082505958}) 2024-08-19T10:55:39.825 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2096402627}) 2024-08-19T10:55:39.825 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.825 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1082505958}) 2024-08-19T10:55:39.825 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3592323962}) 2024-08-19T10:55:39.825 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.825 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1082505958}) 2024-08-19T10:55:39.825 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.814+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2819782000}) 2024-08-19T10:55:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69b7fe640 1 -- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 msgr2=0x7fa69c0a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69b7fe640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 0x7fa69c0a1ea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa68c01cc80 tx=0x7fa68c019180 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa68c0b1040 con 0x7fa69c0a5270 2024-08-19T10:55:39.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 10 monclient: dump: 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.828 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 10 monclient: _finish_auth 0 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 10 monclient: _check_auth_tickets 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68c0ba070 con 0x7fa69c0a5270 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa6a213f640 5 monclient: authenticate success, global_id 114227 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa6a213f640 10 monclient: _renew_subs 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa6a213f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa6a213f640 1 -- 172.21.15.89:0/3814396076 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69c09f680 con 0x7fa69c0a5270 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:39.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa68c0abe30 con 0x7fa69c0a5270 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 10 monclient: dump: 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.830 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa699ffb640 4 set_mon_vals no callback set 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa6a213f640 10 monclient: _renew_subs 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa6a213f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa6a213f640 1 -- 172.21.15.89:0/3814396076 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69c135720 con 0x7fa69c0a5270 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa6a213f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa6a213f640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c13f2d0 0x7fa69c09cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa6a213f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa6a213f640 1 -- 172.21.15.89:0/3814396076 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa69c0a6980 con 0x7fa69c13f2d0 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69affd640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c13f2d0 0x7fa69c09cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69affd640 10 monclient: get_auth_request con 0x7fa69c13f2d0 auth_method 0 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fa68c004250 con 0x7fa69c0a5270 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69affd640 10 monclient(hunting): handle_auth_done global_id 74630 payload 824 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa68803be00 0x7fa68803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa69affd640 1 -- 172.21.15.89:0/3814396076 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa69c1d2080 con 0x7fa69c13f2d0 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.814+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fa68c0e9370 con 0x7fa69c0a5270 2024-08-19T10:55:39.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa69bfff640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa68803be00 0x7fa68803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.833 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.833 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489830941}) 2024-08-19T10:55:39.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa69bfff640 10 monclient: get_auth_request con 0x7fa68803be00 auth_method 0 2024-08-19T10:55:39.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=471728309}) 2024-08-19T10:55:39.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489830941}) 2024-08-19T10:55:39.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3144982554}) 2024-08-19T10:55:39.835 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489830941}) 2024-08-19T10:55:39.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3114822241}) 2024-08-19T10:55:39.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489830941}) 2024-08-19T10:55:39.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1616705651}) 2024-08-19T10:55:39.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489830941}) 2024-08-19T10:55:39.836 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3752956941}) 2024-08-19T10:55:39.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489830941}) 2024-08-19T10:55:39.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3239031375}) 2024-08-19T10:55:39.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489830941}) 2024-08-19T10:55:39.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3908011113}) 2024-08-19T10:55:39.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3489830941}) 2024-08-19T10:55:39.837 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2336902421}) 2024-08-19T10:55:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa69affd640 1 -- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c13f2d0 msgr2=0x7fa69c09cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa69affd640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c13f2d0 0x7fa69c09cf20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa69c139580 tx=0x7fa69000e990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa690017070 con 0x7fa69c13f2d0 2024-08-19T10:55:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa683fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa683fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa683fff640 10 monclient: dump: 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.840 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa690013700 con 0x7fa69c13f2d0 2024-08-19T10:55:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa683fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa683fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c13f2d0 msgr2=0x7fa69c09cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa683fff640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c13f2d0 0x7fa69c09cf20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa69c139580 tx=0x7fa69000e990 comp rx=0 tx=0).stop 2024-08-19T10:55:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c13f2d0 msgr2=0x7fa69c09cf20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa683fff640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c13f2d0 0x7fa69c09cf20 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa69c139580 tx=0x7fa69000e990 comp rx=0 tx=0).stop 2024-08-19T10:55:39.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1390563282}) 2024-08-19T10:55:39.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=341570642}) 2024-08-19T10:55:39.841 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1390563282}) 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=470935353}) 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1390563282}) 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3176047432}) 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1390563282}) 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1653249332}) 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1390563282}) 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3875650702}) 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.842 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1390563282}) 2024-08-19T10:55:39.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4212537142}) 2024-08-19T10:55:39.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1390563282}) 2024-08-19T10:55:39.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2467393852}) 2024-08-19T10:55:39.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.843 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1390563282}) 2024-08-19T10:55:39.844 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.818+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1725287241}) 2024-08-19T10:55:39.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.818+0000 7fa69bfff640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa68803be00 0x7fa68803e2c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa69c13fd40 tx=0x7fa69400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa6a213f640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:55:39.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa6a213f640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c09d460 0x7fa69c137e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:39.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa6a213f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:39.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa6a213f640 1 -- 172.21.15.89:0/3814396076 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fa69c09d840 con 0x7fa69c09d460 2024-08-19T10:55:39.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa69affd640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c09d460 0x7fa69c137e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa69affd640 10 monclient: get_auth_request con 0x7fa69c09d460 auth_method 0 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa69affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa69affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa69affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa69affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa69affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa69affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa69affd640 10 monclient(hunting): handle_auth_done global_id 74639 payload 824 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.826+0000 7fa69affd640 1 -- 172.21.15.89:0/3814396076 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7fa69c139ba0 con 0x7fa69c09d460 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa69affd640 1 -- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c09d460 msgr2=0x7fa69c137e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa69affd640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c09d460 0x7fa69c137e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa69c074dd0 tx=0x7fa690004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 <== mon.2 v2:172.21.15.89:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fa69002c080 con 0x7fa69c09d460 2024-08-19T10:55:39.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa683fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa683fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa683fff640 10 monclient: dump: 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:39.849 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:39.850 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:39.850 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:39.850 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:39.850 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:39.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 <== mon.2 v2:172.21.15.89:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7fa690012370 con 0x7fa69c09d460 2024-08-19T10:55:39.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa683fff640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:55:39.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa683fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:39.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c09d460 msgr2=0x7fa69c137e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa683fff640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c09d460 0x7fa69c137e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa69c074dd0 tx=0x7fa690004740 comp rx=0 tx=0).stop 2024-08-19T10:55:39.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa683fff640 1 -- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c09d460 msgr2=0x7fa69c137e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:39.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa683fff640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fa69c09d460 0x7fa69c137e30 secure :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa69c074dd0 tx=0x7fa690004740 comp rx=0 tx=0).stop 2024-08-19T10:55:39.851 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:39.851 INFO:teuthology.orchestra.run.smithi089.stdout: "ret": 0, 2024-08-19T10:55:39.851 INFO:teuthology.orchestra.run.smithi089.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-08-19T10:55:39.851 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:39.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa6a213f640 1 -- 172.21.15.89:0/3814396076 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa68803be00 msgr2=0x7fa68803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa6a213f640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fa68803be00 0x7fa68803e2c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa69c13fd40 tx=0x7fa69400b040 comp rx=0 tx=0).stop 2024-08-19T10:55:39.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa6a213f640 10 monclient: shutdown 2024-08-19T10:55:39.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa6a213f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:39.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa6a213f640 1 -- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 msgr2=0x7fa69c0a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:39.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa6a213f640 1 --2- 172.21.15.89:0/3814396076 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fa69c0a5270 0x7fa69c0a1ea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa68c01cc80 tx=0x7fa68c019180 comp rx=0 tx=0).stop 2024-08-19T10:55:39.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa69bfff640 1 -- 172.21.15.89:0/3814396076 reap_dead start 2024-08-19T10:55:39.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa69bfff640 1 -- 172.21.15.89:0/3814396076 reap_dead start 2024-08-19T10:55:39.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa6a213f640 1 -- 172.21.15.89:0/3814396076 shutdown_connections 2024-08-19T10:55:39.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa6a213f640 1 -- 172.21.15.89:0/3814396076 >> 172.21.15.89:0/3814396076 conn(0x7fa69c069a80 msgr2=0x7fa69c0ab360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:39.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa6a213f640 1 -- 172.21.15.89:0/3814396076 shutdown_connections 2024-08-19T10:55:39.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:39.830+0000 7fa6a213f640 1 -- 172.21.15.89:0/3814396076 wait complete. 2024-08-19T10:55:39.853 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-08-19T10:55:39.854 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-08-19T10:55:39.854 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:55:39.867 INFO:tasks.ceph.mon.b:Stopped 2024-08-19T10:55:39.867 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-08-19T10:55:39.867 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i store 2024-08-19T10:55:39.867 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i sync_force --yes-i-really-mean-it 2024-08-19T10:55:39.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=491961879}) 2024-08-19T10:55:39.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2620018592}) 2024-08-19T10:55:39.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=491961879}) 2024-08-19T10:55:39.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2809465925}) 2024-08-19T10:55:39.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=491961879}) 2024-08-19T10:55:39.981 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=211927927}) 2024-08-19T10:55:39.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=491961879}) 2024-08-19T10:55:39.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=787119864}) 2024-08-19T10:55:39.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=491961879}) 2024-08-19T10:55:39.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1025187935}) 2024-08-19T10:55:39.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=491961879}) 2024-08-19T10:55:39.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=992655336}) 2024-08-19T10:55:39.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.982 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=491961879}) 2024-08-19T10:55:39.983 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=491743442}) 2024-08-19T10:55:39.983 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-08-19T10:55:39.983 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=491961879}) 2024-08-19T10:55:39.983 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:55:39.982+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1617904761}) 2024-08-19T10:55:40.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.034+0000 7f9334c60640 10 monclient: get_monmap_and_config 2024-08-19T10:55:40.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.034+0000 7f9334c60640 10 monclient: build_initial_monmap 2024-08-19T10:55:40.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.034+0000 7f9334c60640 10 monclient: monmap: 2024-08-19T10:55:40.033 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:40.033 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:40.039176+0000 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:40.039176+0000 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:40.034 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:40.035 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:40.035 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:40.035 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.034+0000 7f9334c60640 1 Processor -- start 2024-08-19T10:55:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.034+0000 7f9334c60640 1 -- start start 2024-08-19T10:55:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.034+0000 7f9334c60640 10 monclient: init 2024-08-19T10:55:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 10 monclient: _add_conns ranks=[6,7,4,9,5,1,10,8,3,0,2] 2024-08-19T10:55:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93300a5270 0x7f93300a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 10 monclient(hunting): picked mon.noname-d con 0x7f93300a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93300a6d30 0x7f93300a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 10 monclient(hunting): picked mon.noname-i con 0x7f93300a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93301395b0 0x7f93301399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 10 monclient(hunting): picked mon.noname-c con 0x7f93301395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f93300a7650 con 0x7f93301395b0 2024-08-19T10:55:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f93300a8710 con 0x7f93300a5270 2024-08-19T10:55:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f93300a5a40 con 0x7f93300a6d30 2024-08-19T10:55:40.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:40.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f9334c60640 10 monclient(hunting): authenticate will time out at 1989.856270s 2024-08-19T10:55:40.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93300a5270 0x7f93300a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932ed9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93301395b0 0x7f93301399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932ed9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93301395b0 0x7f93301399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:35290/0 (socket says 172.21.15.89:35290) 2024-08-19T10:55:40.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93300a5270 0x7f93300a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:36580/0 (socket says 172.21.15.89:36580) 2024-08-19T10:55:40.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932ed9d640 1 -- 172.21.15.89:0/94961695 learned_addr learned my addr 172.21.15.89:0/94961695 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:40.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932ed9d640 10 monclient(hunting): get_auth_request con 0x7f93301395b0 auth_method 0 2024-08-19T10:55:40.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 10 monclient(hunting): get_auth_request con 0x7f93300a5270 auth_method 0 2024-08-19T10:55:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:40.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:40.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:40.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:40.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:40.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:40.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932dd9b640 1 --2- 172.21.15.89:0/94961695 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93300a6d30 0x7f93300a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932dd9b640 10 monclient(hunting): get_auth_request con 0x7f93300a6d30 auth_method 0 2024-08-19T10:55:40.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:40.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 10 monclient(hunting): handle_auth_done global_id 45492 payload 274 2024-08-19T10:55:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 1 -- 172.21.15.89:0/94961695 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93300a6d30 msgr2=0x7f93300a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 1 --2- 172.21.15.89:0/94961695 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93300a6d30 0x7f93300a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 1 -- 172.21.15.89:0/94961695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93301395b0 msgr2=0x7f93301399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 1 --2- 172.21.15.89:0/94961695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93301395b0 0x7f93301399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:55:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 1 monclient: found mon.noname-d 2024-08-19T10:55:40.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:40.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:55:40.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932e59c640 1 -- 172.21.15.89:0/94961695 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93300a81d0 con 0x7f93300a5270 2024-08-19T10:55:40.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.038+0000 7f932ed9d640 1 --2- 172.21.15.89:0/94961695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93301395b0 0x7f93301399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:40.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 1 -- 172.21.15.89:0/94961695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93300a5270 msgr2=0x7f93300a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:40.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 1 --2- 172.21.15.89:0/94961695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93300a5270 0x7f93300a67f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f93200097f0 tx=0x7f932001c3a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=91f5baf96fb819e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 1 -- 172.21.15.89:0/94961695 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93200b0070 con 0x7f93300a5270 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 10 monclient: dump: 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:40.053 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 10 monclient: _finish_auth 0 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 1 -- 172.21.15.89:0/94961695 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93200aa810 con 0x7f93300a5270 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:40.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 1 -- 172.21.15.89:0/94961695 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93200a9300 con 0x7f93300a5270 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 5 monclient: authenticate success, global_id 45492 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 4 set_mon_vals no callback set 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932d59a640 10 monclient: dump: 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.055 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient: shutdown 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 -- 172.21.15.89:0/94961695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93300a5270 msgr2=0x7f93300a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:40.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 --2- 172.21.15.89:0/94961695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93300a5270 0x7f93300a67f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f93200097f0 tx=0x7f932001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 -- 172.21.15.89:0/94961695 shutdown_connections 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 --2- 172.21.15.89:0/94961695 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93301395b0 0x7f93301399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 --2- 172.21.15.89:0/94961695 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93300a6d30 0x7f93300a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 --2- 172.21.15.89:0/94961695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f93300a5270 0x7f93300a67f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 -- 172.21.15.89:0/94961695 >> 172.21.15.89:0/94961695 conn(0x7f9330069a80 msgr2=0x7f93300abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 -- 172.21.15.89:0/94961695 shutdown_connections 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 -- 172.21.15.89:0/94961695 wait complete. 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient: build_initial_monmap 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient: monmap: 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:40.057 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 Processor -- start 2024-08-19T10:55:40.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 -- start start 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient: init 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient: _add_conns ranks=[4,8,3,5,6,7,1,2,0] 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93300a5270 0x7f933013dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient(hunting): picked mon.noname-c con 0x7f93300a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f93300a6d30 0x7f933013e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient(hunting): picked mon.noname-e con 0x7f93300a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93301395b0 0x7f933013e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient(hunting): picked mon.noname-g con 0x7f93301395b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f93300a57d0 con 0x7f93301395b0 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f93300a5650 con 0x7f93300a5270 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f9330143230 con 0x7f93300a6d30 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f9334c60640 10 monclient(hunting): authenticate will time out at 1989.863501s 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93300a5270 0x7f933013dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932dd9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f93300a6d30 0x7f933013e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93300a5270 0x7f933013dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:35296/0 (socket says 172.21.15.89:35296) 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 1 -- 172.21.15.89:0/1316470928 learned_addr learned my addr 172.21.15.89:0/1316470928 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 10 monclient(hunting): get_auth_request con 0x7f93300a5270 auth_method 0 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:40.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:40.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932dd9b640 10 monclient(hunting): get_auth_request con 0x7f93300a6d30 auth_method 0 2024-08-19T10:55:40.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:40.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:40.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:40.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:40.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:40.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:40.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:40.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:40.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:40.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932ed9d640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93301395b0 0x7f933013e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 10 monclient(hunting): handle_auth_done global_id 84478 payload 824 2024-08-19T10:55:40.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 1 -- 172.21.15.89:0/1316470928 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f93300a6d30 msgr2=0x7f933013e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:40.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f93300a6d30 0x7f933013e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 1 -- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93301395b0 msgr2=0x7f933013e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:40.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93301395b0 0x7f933013e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 1 monclient: found mon.noname-c 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.042+0000 7f932e59c640 1 -- 172.21.15.89:0/1316470928 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933009e8c0 con 0x7f93300a5270 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f932e59c640 1 -- 172.21.15.89:0/1316470928 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93300a5270 msgr2=0x7f933013dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f932e59c640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93300a5270 0x7f933013dea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f932001c1d0 tx=0x7f9320019140 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f93200b0040 con 0x7f93300a5270 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 10 monclient: dump: 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.063 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:40.064 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 10 monclient: _finish_auth 0 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 10 monclient: _check_auth_tickets 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93200044b0 con 0x7f93300a5270 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9320003a90 con 0x7f93300a5270 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 10 monclient: dump: 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.065 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f932cd99640 4 set_mon_vals no callback set 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f9334c60640 5 monclient: authenticate success, global_id 84478 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f9334c60640 10 monclient: _renew_subs 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f9334c60640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f9334c60640 1 -- 172.21.15.89:0/1316470928 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93300a7210 con 0x7f93300a5270 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f9334c60640 10 monclient: _renew_subs 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f9334c60640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f9334c60640 1 -- 172.21.15.89:0/1316470928 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f933009e300 con 0x7f93300a5270 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f9334c60640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f9334c60640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1330 0x7f93301d1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f9334c60640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f9334c60640 1 -- 172.21.15.89:0/1316470928 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f933009d0c0 con 0x7f93301d1330 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9320003cd0 con 0x7f93300a5270 2024-08-19T10:55:40.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f932dd9b640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1330 0x7f93301d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f931003bd30 0x7f931003e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f932dd9b640 10 monclient: get_auth_request con 0x7f93301d1330 auth_method 0 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f932dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f932dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f932dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f93200e9000 con 0x7f93300a5270 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f932dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f932dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.046+0000 7f932dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f932ed9d640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f931003bd30 0x7f931003e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f932ed9d640 10 monclient: get_auth_request con 0x7f931003bd30 auth_method 0 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f932dd9b640 10 monclient(hunting): handle_auth_done global_id 114217 payload 824 2024-08-19T10:55:40.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f932dd9b640 1 -- 172.21.15.89:0/1316470928 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93300a67f0 con 0x7f93301d1330 2024-08-19T10:55:40.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f932dd9b640 1 -- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1330 msgr2=0x7f93301d1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:40.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f932dd9b640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1330 0x7f93301d1710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f933013f5d0 tx=0x7f9324010c80 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:40.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f932ed9d640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f931003bd30 0x7f931003e1f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9328009c90 tx=0x7f932800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:40.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9324018020 con 0x7f93301d1330 2024-08-19T10:55:40.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f931effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:40.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f931effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:55:40.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f931effd640 10 monclient: dump: 2024-08-19T10:55:40.069 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:40.069 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.069 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.069 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9324004450 con 0x7f93301d1330 2024-08-19T10:55:40.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f931effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:40.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f931effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:40.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1330 msgr2=0x7f93301d1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:40.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f931effd640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1330 0x7f93301d1710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f933013f5d0 tx=0x7f9324010c80 comp rx=0 tx=0).stop 2024-08-19T10:55:40.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1330 msgr2=0x7f93301d1710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:40.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.050+0000 7f931effd640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1330 0x7f93301d1710 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f933013f5d0 tx=0x7f9324010c80 comp rx=0 tx=0).stop 2024-08-19T10:55:40.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f9334c60640 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:55:40.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f9334c60640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1c50 0x7f9330133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f9334c60640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f9334c60640 1 -- 172.21.15.89:0/1316470928 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f9330064590 con 0x7f93301d1c50 2024-08-19T10:55:40.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f932dd9b640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1c50 0x7f9330133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f932dd9b640 10 monclient: get_auth_request con 0x7f93301d1c50 auth_method 0 2024-08-19T10:55:40.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f932dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:40.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f932dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:40.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f932dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:40.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f932dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:40.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f932dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f932dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f932dd9b640 10 monclient(hunting): handle_auth_done global_id 114226 payload 824 2024-08-19T10:55:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.058+0000 7f932dd9b640 1 -- 172.21.15.89:0/1316470928 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) -- 0x7f93300a2880 con 0x7f93301d1c50 2024-08-19T10:55:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f932dd9b640 1 -- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1c50 msgr2=0x7f9330133760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f932dd9b640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1c50 0x7f9330133760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9324019000 tx=0x7f93240046d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 <== mon.7 v2:172.21.15.190:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9324018040 con 0x7f93301d1c50 2024-08-19T10:55:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f931effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f931effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-08-19T10:55:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f931effd640 10 monclient: dump: 2024-08-19T10:55:40.074 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:40.075 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:40.076 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 <== mon.7 v2:172.21.15.190:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+83 (secure 0 0 0) 0x7f9324003620 con 0x7f93301d1c50 2024-08-19T10:55:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f931effd640 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-08-19T10:55:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f931effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1c50 msgr2=0x7f9330133760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f931effd640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1c50 0x7f9330133760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9324019000 tx=0x7f93240046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f931effd640 1 -- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1c50 msgr2=0x7f9330133760 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:55:40.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.062+0000 7f931effd640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f93301d1c50 0x7f9330133760 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9324019000 tx=0x7f93240046d0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.076 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:55:40.077 INFO:teuthology.orchestra.run.smithi089.stdout: "ret": 0, 2024-08-19T10:55:40.077 INFO:teuthology.orchestra.run.smithi089.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-08-19T10:55:40.077 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:55:40.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f9334c60640 1 -- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f931003bd30 msgr2=0x7f931003e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f9334c60640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f931003bd30 0x7f931003e1f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9328009c90 tx=0x7f932800b040 comp rx=0 tx=0).stop 2024-08-19T10:55:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f9334c60640 10 monclient: shutdown 2024-08-19T10:55:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f9334c60640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f9334c60640 1 -- 172.21.15.89:0/1316470928 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93300a5270 msgr2=0x7f933013dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f9334c60640 1 --2- 172.21.15.89:0/1316470928 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f93300a5270 0x7f933013dea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f932001c1d0 tx=0x7f9320019140 comp rx=0 tx=0).stop 2024-08-19T10:55:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f932ed9d640 1 -- 172.21.15.89:0/1316470928 reap_dead start 2024-08-19T10:55:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f932ed9d640 1 -- 172.21.15.89:0/1316470928 reap_dead start 2024-08-19T10:55:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f9334c60640 1 -- 172.21.15.89:0/1316470928 shutdown_connections 2024-08-19T10:55:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f9334c60640 1 -- 172.21.15.89:0/1316470928 >> 172.21.15.89:0/1316470928 conn(0x7f9330069a80 msgr2=0x7f93300ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f9334c60640 1 -- 172.21.15.89:0/1316470928 shutdown_connections 2024-08-19T10:55:40.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.066+0000 7f9334c60640 1 -- 172.21.15.89:0/1316470928 wait complete. 2024-08-19T10:55:40.079 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-08-19T10:55:40.080 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-08-19T10:55:40.080 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:55:40.098 INFO:tasks.ceph.mon.i:Stopped 2024-08-19T10:55:40.099 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-08-19T10:55:40.099 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:55:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient: get_monmap_and_config 2024-08-19T10:55:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient: build_initial_monmap 2024-08-19T10:55:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient: monmap: 2024-08-19T10:55:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:40.218120+0000 2024-08-19T10:55:40.212 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:40.218120+0000 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:40.213 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:40.214 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 1 Processor -- start 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 1 -- start start 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient: init 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient: _add_conns ranks=[1,7,4,2,6,5,3,0,9,8,10] 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7e7406c930 addr v1:172.21.15.190:6789/0 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e740a2e60 0x7f7e7413c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient(hunting): picked mon.noname-i con 0x7f7e740a2e60 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7409d540 0x7f7e7413eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient(hunting): picked mon.noname-c con 0x7f7e7409d540 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e740a4000 con 0x7f7e7406c930 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f7e7409cbf0 con 0x7f7e7409d540 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7e740a3d60 con 0x7f7e740a2e60 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e7b0d5640 10 monclient(hunting): authenticate will time out at 1990.035466s 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7409d540 0x7f7e7413eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e6bfff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e740a2e60 msgr2=0x7f7e7413c950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e6bfff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e740a2e60 0x7f7e7413c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7409d540 0x7f7e7413eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:35324/0 (socket says 172.21.15.89:35324) 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 1 -- 172.21.15.89:0/1066379254 learned_addr learned my addr 172.21.15.89:0/1066379254 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 10 monclient(hunting): get_auth_request con 0x7f7e7409d540 auth_method 0 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:40.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 10 monclient(hunting): handle_auth_done global_id 84487 payload 274 2024-08-19T10:55:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 1 -- 172.21.15.89:0/1066379254 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e740a2e60 msgr2=0x7f7e7413c950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 1 --2- 172.21.15.89:0/1066379254 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e740a2e60 0x7f7e7413c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 1 -- 172.21.15.89:0/1066379254 >> v1:172.21.15.190:6789/0 conn(0x7f7e7406c930 legacy=0x7f7e7406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 10 monclient: _finish_hunting 0 2024-08-19T10:55:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 1 monclient: found mon.noname-c 2024-08-19T10:55:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:40.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.214+0000 7f7e79672640 1 -- 172.21.15.89:0/1066379254 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e740a27d0 con 0x7f7e7409d540 2024-08-19T10:55:40.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e79672640 1 -- 172.21.15.89:0/1066379254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7409d540 msgr2=0x7f7e7413eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:40.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e79672640 1 --2- 172.21.15.89:0/1066379254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7409d540 0x7f7e7413eea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e80 tx=0x7f7e6c01c930 comp rx=0 tx=0).ready entity=mon.4 client_cookie=565c9345562081c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:40.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 1 -- 172.21.15.89:0/1066379254 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e6c0a5a90 con 0x7f7e7409d540 2024-08-19T10:55:40.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:40.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:40.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 10 monclient: dump: 2024-08-19T10:55:40.221 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:40.221 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.221 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.221 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.221 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:40.221 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.221 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:40.221 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:40.221 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 10 monclient: _finish_auth 0 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 1 -- 172.21.15.89:0/1066379254 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e6c0a5c50 con 0x7f7e7409d540 2024-08-19T10:55:40.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 5 monclient: authenticate success, global_id 84487 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 1 -- 172.21.15.89:0/1066379254 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e6c0a5e00 con 0x7f7e7409d540 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6b7fe640 10 monclient: dump: 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:40.223 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 4 set_mon_vals no callback set 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient: shutdown 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/1066379254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7409d540 msgr2=0x7f7e7413eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:40.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 --2- 172.21.15.89:0/1066379254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7409d540 0x7f7e7413eea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e80 tx=0x7f7e6c01c930 comp rx=0 tx=0).stop 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/1066379254 shutdown_connections 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 --2- 172.21.15.89:0/1066379254 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f7e7409d540 0x7f7e7413eea0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 --2- 172.21.15.89:0/1066379254 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e740a2e60 0x7f7e7413c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/1066379254 >> 172.21.15.89:0/1066379254 conn(0x7f7e7406de40 msgr2=0x7f7e7406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/1066379254 shutdown_connections 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/1066379254 wait complete. 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient: build_initial_monmap 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient: monmap: 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:40.225 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:40.226 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.226 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:40.226 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:40.226 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:40.226 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:40.226 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:40.226 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:40.226 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:40.226 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:40.226 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:40.226 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 Processor -- start 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 -- start start 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient: init 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient: _add_conns ranks=[7,5,2,6,1,4,3,8,0] 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e7406c930 0x7f7e74137940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient(hunting): picked mon.noname-i con 0x7f7e7406c930 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7e7409d540 0x7f7e74120430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient(hunting): picked mon.noname-h con 0x7f7e7409d540 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e740a2e60 0x7f7e74120970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient(hunting): picked mon.noname-b con 0x7f7e740a2e60 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f7e74141f90 con 0x7f7e740a2e60 2024-08-19T10:55:40.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f7e74141e10 con 0x7f7e7409d540 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f7e74142110 con 0x7f7e7406c930 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e7b0d5640 10 monclient(hunting): authenticate will time out at 1990.042601s 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e79672640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e740a2e60 msgr2=0x7f7e74120970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e79672640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e740a2e60 0x7f7e74120970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e78e71640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e7406c930 msgr2=0x7f7e74137940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e78e71640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e7406c930 0x7f7e74137940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7e7409d540 0x7f7e74120430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7e7409d540 0x7f7e74120430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:57804/0 (socket says 172.21.15.89:57804) 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 1 -- 172.21.15.89:0/2624857818 learned_addr learned my addr 172.21.15.89:0/2624857818 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 10 monclient(hunting): get_auth_request con 0x7f7e7409d540 auth_method 0 2024-08-19T10:55:40.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 10 monclient(hunting): handle_auth_done global_id 84389 payload 824 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 1 -- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e7406c930 msgr2=0x7f7e74137940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e7406c930 0x7f7e74137940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 1 -- 172.21.15.89:0/2624857818 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e740a2e60 msgr2=0x7f7e74120970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e740a2e60 0x7f7e74120970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 10 monclient: _finish_hunting 0 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 1 monclient: found mon.noname-h 2024-08-19T10:55:40.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.222+0000 7f7e6bfff640 1 -- 172.21.15.89:0/2624857818 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e740a5f80 con 0x7f7e7409d540 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e6bfff640 1 -- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7e7409d540 msgr2=0x7f7e74120430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e6bfff640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7e7409d540 0x7f7e74120430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400b560 tx=0x7f7e6400ba20 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e697fa640 1 -- 172.21.15.89:0/2624857818 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e64013020 con 0x7f7e7409d540 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e697fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e697fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e697fa640 10 monclient: dump: 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:40.234 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e697fa640 10 monclient: _finish_auth 0 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e697fa640 10 monclient: _check_auth_tickets 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:40.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e697fa640 1 -- 172.21.15.89:0/2624857818 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e64003de0 con 0x7f7e7409d540 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e697fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e6bfff640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7e7409d540 0x7f7e74120430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400b560 tx=0x7f7e6400ba20 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7f7e640054e0 mon_map magic: 0 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e7b0d5640 5 monclient: authenticate success, global_id 84389 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e6a7fc640 4 set_mon_vals no callback set 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e7b0d5640 10 monclient: _renew_subs 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e7b0d5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/2624857818 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7413eed0 con 0x7f7e7409d540 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e7b0d5640 10 monclient: _renew_subs 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e7b0d5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/2624857818 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e74123ed0 con 0x7f7e7409d540 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e7b0d5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e7b0d5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e7b0d5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:55:40.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/2624857818 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e740a61b0 con 0x7f7e7409d540 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e697fa640 1 -- 172.21.15.89:0/2624857818 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7e640054e0 con 0x7f7e7409d540 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.226+0000 7f7e697fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.230+0000 7f7e697fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.230+0000 7f7e697fa640 10 monclient: dump: 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:40.237 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.230+0000 7f7e697fa640 1 -- 172.21.15.89:0/2624857818 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f7e64016020 con 0x7f7e7409d540 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.230+0000 7f7e697fa640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e6003be80 0x7f7e6003e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.230+0000 7f7e78e71640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e6003be80 0x7f7e6003e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.230+0000 7f7e78e71640 10 monclient: get_auth_request con 0x7f7e6003be80 auth_method 0 2024-08-19T10:55:40.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.230+0000 7f7e78e71640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e6003be80 0x7f7e6003e340 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7e70003730 tx=0x7f7e70007a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:40.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.242+0000 7f7e697fa640 1 -- 172.21.15.89:0/2624857818 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f7e6404fef0 con 0x7f7e7409d540 2024-08-19T10:55:40.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.850+0000 7f7e697fa640 1 -- 172.21.15.89:0/2624857818 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f7e740a61b0 con 0x7f7e7409d540 2024-08-19T10:55:40.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.850+0000 7f7e697fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:40.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:40.850+0000 7f7e697fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:41.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.082+0000 7f7e7b0d5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:55:41.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.082+0000 7f7e7b0d5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:41.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.082+0000 7f7e7b0d5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:55:41.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.082+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/2624857818 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7e74064590 con 0x7f7e7409d540 2024-08-19T10:55:41.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.082+0000 7f7e697fa640 1 -- 172.21.15.89:0/2624857818 <== mon.5 v2:172.21.15.190:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3193 (secure 0 0 0) 0x7f7e6401d020 con 0x7f7e7409d540 2024-08-19T10:55:41.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.082+0000 7f7e697fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:41.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.082+0000 7f7e697fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:41.081 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":140,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:55:41.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e6003be80 msgr2=0x7f7e6003e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e6003be80 0x7f7e6003e340 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7e70003730 tx=0x7f7e70007a20 comp rx=0 tx=0).stop 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 10 monclient: shutdown 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7e7409d540 msgr2=0x7f7e74120430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7e7409d540 0x7f7e74120430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400b560 tx=0x7f7e6400ba20 comp rx=0 tx=0).stop 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/2624857818 shutdown_connections 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f7e6003be80 0x7f7e6003e340 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f7e7409d540 0x7f7e74120430 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f7e740a2e60 0x7f7e74120970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 --2- 172.21.15.89:0/2624857818 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f7e7406c930 0x7f7e74137940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/2624857818 >> 172.21.15.89:0/2624857818 conn(0x7f7e7406de40 msgr2=0x7f7e740a7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:41.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/2624857818 shutdown_connections 2024-08-19T10:55:41.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:41.086+0000 7f7e7b0d5640 1 -- 172.21.15.89:0/2624857818 wait complete. 2024-08-19T10:55:44.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:55:44.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient: get_monmap_and_config 2024-08-19T10:55:44.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient: build_initial_monmap 2024-08-19T10:55:44.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient: monmap: 2024-08-19T10:55:44.229 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:44.229 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:44.229 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:44.234781+0000 2024-08-19T10:55:44.229 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:44.234781+0000 2024-08-19T10:55:44.229 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:44.229 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:44.229 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:44.230 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:44.230 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:44.230 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:44.230 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:44.230 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:44.230 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:44.230 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:44.230 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:44.230 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:44.230 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:44.230 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:44.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 1 Processor -- start 2024-08-19T10:55:44.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 1 -- start start 2024-08-19T10:55:44.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient: init 2024-08-19T10:55:44.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:44.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient: _add_conns ranks=[1,7,10,4,5,6,0,8,9,2,3] 2024-08-19T10:55:44.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb7e40a5270 addr v1:172.21.15.190:6789/0 2024-08-19T10:55:44.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7e40a6f00 0x7fb7e413ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:44.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient(hunting): picked mon.noname-i con 0x7fb7e40a6f00 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:44.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7e40a72e0 0x7fb7e4140f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:44.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient(hunting): picked mon.noname-f con 0x7fb7e40a72e0 addr v2:172.21.15.190:3300/0 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e40a8710 con 0x7fb7e40a5270 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e40a5a40 con 0x7fb7e40a72e0 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fb7e40a81d0 con 0x7fb7e40a6f00 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7eaf62640 10 monclient(hunting): authenticate will time out at 1994.051437s 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7e3fff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7e40a6f00 msgr2=0x7fb7e413ea00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7e40a6f00 0x7fb7e413ea00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7e94ff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7e40a72e0 0x7fb7e4140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7e94ff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7e40a72e0 0x7fb7e4140f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:59046/0 (socket says 172.21.15.89:59046) 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7e94ff640 1 -- 172.21.15.89:0/772303764 learned_addr learned my addr 172.21.15.89:0/772303764 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:44.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7e94ff640 10 monclient(hunting): get_auth_request con 0x7fb7e40a72e0 auth_method 0 2024-08-19T10:55:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7e94ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7e94ff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.230+0000 7fb7e94ff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:44.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e8cfe640 1 -- 172.21.15.89:0/772303764 >> v1:172.21.15.190:6789/0 conn(0x7fb7e40a5270 legacy=0x7fb7e40a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 10 monclient(hunting): handle_auth_done global_id 45370 payload 274 2024-08-19T10:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 1 -- 172.21.15.89:0/772303764 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7e40a6f00 msgr2=0x7fb7e413ea00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 1 --2- 172.21.15.89:0/772303764 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7e40a6f00 0x7fb7e413ea00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 1 -- 172.21.15.89:0/772303764 >> v1:172.21.15.190:6789/0 conn(0x7fb7e40a5270 legacy=0x7fb7e40a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 10 monclient: _finish_hunting 0 2024-08-19T10:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 1 monclient: found mon.noname-f 2024-08-19T10:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:55:44.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.234+0000 7fb7e94ff640 1 -- 172.21.15.89:0/772303764 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e40a4790 con 0x7fb7e40a72e0 2024-08-19T10:55:44.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.238+0000 7fb7e94ff640 1 -- 172.21.15.89:0/772303764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e40a72e0 msgr2=0x7fb7e4140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:44.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.238+0000 7fb7e94ff640 1 --2- 172.21.15.89:0/772303764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e40a72e0 0x7fb7e4140f50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0098f0 tx=0x7fb7dc01c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4194bfe71333f7a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:44.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.238+0000 7fb7e37fe640 1 -- 172.21.15.89:0/772303764 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb7dc0aa940 con 0x7fb7e40a72e0 2024-08-19T10:55:44.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.238+0000 7fb7e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e37fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e37fe640 10 monclient: dump: 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:44.240 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e37fe640 10 monclient: _finish_auth 0 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e37fe640 10 monclient: _check_auth_tickets 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e37fe640 1 -- 172.21.15.89:0/772303764 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc0af070 con 0x7fb7e40a72e0 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e37fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e37fe640 1 -- 172.21.15.89:0/772303764 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb7dc0a9350 con 0x7fb7e40a72e0 2024-08-19T10:55:44.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 5 monclient: authenticate success, global_id 45370 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 4 set_mon_vals no callback set 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e37fe640 10 monclient: dump: 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:44.242 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient: shutdown 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:44.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 -- 172.21.15.89:0/772303764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e40a72e0 msgr2=0x7fb7e4140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:44.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 --2- 172.21.15.89:0/772303764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e40a72e0 0x7fb7e4140f50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0098f0 tx=0x7fb7dc01c490 comp rx=0 tx=0).stop 2024-08-19T10:55:44.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 -- 172.21.15.89:0/772303764 shutdown_connections 2024-08-19T10:55:44.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 --2- 172.21.15.89:0/772303764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e40a72e0 0x7fb7e4140f50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:44.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 --2- 172.21.15.89:0/772303764 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fb7e40a6f00 0x7fb7e413ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:44.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 -- 172.21.15.89:0/772303764 >> 172.21.15.89:0/772303764 conn(0x7fb7e4069a80 msgr2=0x7fb7e40abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:44.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 -- 172.21.15.89:0/772303764 shutdown_connections 2024-08-19T10:55:44.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 -- 172.21.15.89:0/772303764 wait complete. 2024-08-19T10:55:44.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient: build_initial_monmap 2024-08-19T10:55:44.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient: monmap: 2024-08-19T10:55:44.248 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:44.249 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 Processor -- start 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 -- start start 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient: init 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient: _add_conns ranks=[4,1,6,3,2,7,0,5,8] 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7e40a5270 0x7fb7e4128780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient(hunting): picked mon.noname-c con 0x7fb7e40a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e40a6f00 0x7fb7e41d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient(hunting): picked mon.noname-f con 0x7fb7e40a6f00 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb7e40a72e0 0x7fb7e41d1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:44.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient(hunting): picked mon.noname-d con 0x7fb7e40a72e0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e4143e30 con 0x7fb7e40a6f00 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fb7e4143cb0 con 0x7fb7e40a5270 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fb7e4143fb0 con 0x7fb7e40a72e0 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7eaf62640 10 monclient(hunting): authenticate will time out at 1994.062242s 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7e40a5270 0x7fb7e4128780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7e40a5270 0x7fb7e4128780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39910/0 (socket says 172.21.15.89:39910) 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 1 -- 172.21.15.89:0/1972348333 learned_addr learned my addr 172.21.15.89:0/1972348333 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 10 monclient(hunting): get_auth_request con 0x7fb7e40a5270 auth_method 0 2024-08-19T10:55:44.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e94ff640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb7e40a72e0 0x7fb7e41d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e94ff640 10 monclient(hunting): get_auth_request con 0x7fb7e40a72e0 auth_method 0 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e94ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e94ff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e94ff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e94ff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e94ff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:44.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e94ff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e3fff640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e40a6f00 0x7fb7e41d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e3fff640 10 monclient(hunting): get_auth_request con 0x7fb7e40a6f00 auth_method 0 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e3fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 10 monclient(hunting): handle_auth_done global_id 84496 payload 824 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 1 -- 172.21.15.89:0/1972348333 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb7e40a72e0 msgr2=0x7fb7e41d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb7e40a72e0 0x7fb7e41d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 1 -- 172.21.15.89:0/1972348333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e40a6f00 msgr2=0x7fb7e41d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e40a6f00 0x7fb7e41d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 1 monclient: found mon.noname-c 2024-08-19T10:55:44.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e8cfe640 1 -- 172.21.15.89:0/1972348333 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e4129110 con 0x7fb7e40a5270 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.242+0000 7fb7e94ff640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb7e40a72e0 0x7fb7e41d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e8cfe640 1 -- 172.21.15.89:0/1972348333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7e40a5270 msgr2=0x7fb7e4128780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e8cfe640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7e40a5270 0x7fb7e4128780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400c7a0 tx=0x7fb7d400cc60 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 1 -- 172.21.15.89:0/1972348333 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb7d4017070 con 0x7fb7e40a5270 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 10 monclient: dump: 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:44.254 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:44.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 10 monclient: _finish_auth 0 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 10 monclient: _check_auth_tickets 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7eaf62640 5 monclient: authenticate success, global_id 84496 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 1 -- 172.21.15.89:0/1972348333 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d400aab0 con 0x7fb7e40a5270 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7eaf62640 10 monclient: _renew_subs 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7eaf62640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7eaf62640 1 -- 172.21.15.89:0/1972348333 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e41296e0 con 0x7fb7e40a5270 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 1 -- 172.21.15.89:0/1972348333 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fb7d400ac60 con 0x7fb7e40a5270 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e2ffd640 4 set_mon_vals no callback set 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 10 monclient: dump: 2024-08-19T10:55:44.256 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:44.257 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7eaf62640 10 monclient: _renew_subs 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7eaf62640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7eaf62640 1 -- 172.21.15.89:0/1972348333 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e4127c20 con 0x7fb7e40a5270 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7eaf62640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7eaf62640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7eaf62640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7eaf62640 1 -- 172.21.15.89:0/1972348333 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e40a67f0 con 0x7fb7e40a5270 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 1 -- 172.21.15.89:0/1972348333 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fb7d400a640 con 0x7fb7e40a5270 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb7c403be00 0x7fb7c403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e17fa640 1 -- 172.21.15.89:0/1972348333 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fb7d404ea20 con 0x7fb7e40a5270 2024-08-19T10:55:44.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e3fff640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb7c403be00 0x7fb7c403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:44.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.246+0000 7fb7e3fff640 10 monclient: get_auth_request con 0x7fb7c403be00 auth_method 0 2024-08-19T10:55:44.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.250+0000 7fb7e3fff640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb7c403be00 0x7fb7c403e2c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc006fd0 tx=0x7fb7cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:44.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.250+0000 7fb7e17fa640 1 -- 172.21.15.89:0/1972348333 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fb7e41296e0 con 0x7fb7e40a5270 2024-08-19T10:55:44.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.250+0000 7fb7e17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:44.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.250+0000 7fb7e17fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:44.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:55:44.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:44.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:44.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 -- 172.21.15.89:0/1972348333 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb7e4064590 con 0x7fb7e40a5270 2024-08-19T10:55:44.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7e17fa640 1 -- 172.21.15.89:0/1972348333 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3194 (secure 0 0 0) 0x7fb7e40a8710 con 0x7fb7e40a5270 2024-08-19T10:55:44.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7e17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:44.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7e17fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:44.476 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":140,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:55:44.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 -- 172.21.15.89:0/1972348333 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb7c403be00 msgr2=0x7fb7c403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:44.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb7c403be00 0x7fb7c403e2c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc006fd0 tx=0x7fb7cc008040 comp rx=0 tx=0).stop 2024-08-19T10:55:44.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 10 monclient: shutdown 2024-08-19T10:55:44.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 -- 172.21.15.89:0/1972348333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7e40a5270 msgr2=0x7fb7e4128780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7e40a5270 0x7fb7e4128780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400c7a0 tx=0x7fb7d400cc60 comp rx=0 tx=0).stop 2024-08-19T10:55:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 -- 172.21.15.89:0/1972348333 shutdown_connections 2024-08-19T10:55:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fb7c403be00 0x7fb7c403e2c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fb7e40a72e0 0x7fb7e41d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e40a6f00 0x7fb7e41d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 --2- 172.21.15.89:0/1972348333 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fb7e40a5270 0x7fb7e4128780 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 -- 172.21.15.89:0/1972348333 >> 172.21.15.89:0/1972348333 conn(0x7fb7e4069a80 msgr2=0x7fb7e409f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 -- 172.21.15.89:0/1972348333 shutdown_connections 2024-08-19T10:55:44.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:44.478+0000 7fb7eaf62640 1 -- 172.21.15.89:0/1972348333 wait complete. 2024-08-19T10:55:47.494 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:55:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.626+0000 7f0761fa0640 10 monclient: get_monmap_and_config 2024-08-19T10:55:47.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.626+0000 7f0761fa0640 10 monclient: build_initial_monmap 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.626+0000 7f0761fa0640 10 monclient: monmap: 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:47.631495+0000 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:47.631495+0000 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:47.626 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:47.627 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:47.627 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:47.627 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:47.627 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:47.627 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:47.627 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:47.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.626+0000 7f0761fa0640 1 Processor -- start 2024-08-19T10:55:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.626+0000 7f0761fa0640 1 -- start start 2024-08-19T10:55:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.626+0000 7f0761fa0640 10 monclient: init 2024-08-19T10:55:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.626+0000 7f0761fa0640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.626+0000 7f0761fa0640 10 monclient: _add_conns ranks=[2,9,3,0,5,8,1,6,4,10,7] 2024-08-19T10:55:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.626+0000 7f0761fa0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f075c0a5270 0x7f075c0a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.626+0000 7f0761fa0640 10 monclient(hunting): picked mon.noname-b con 0x7f075c0a5270 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:55:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.626+0000 7f0761fa0640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 0x7f075c0a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f0761fa0640 10 monclient(hunting): picked mon.noname-a con 0x7f075c0a6d30 addr v2:172.21.15.89:3300/0 2024-08-19T10:55:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f0761fa0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f075c1395b0 0x7f075c139a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f0761fa0640 10 monclient(hunting): picked mon.noname-g con 0x7f075c1395b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:47.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f0761fa0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f0761fa0640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f075c0a7650 con 0x7f075c0a6d30 2024-08-19T10:55:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f0761fa0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f0761fa0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f075c0a8710 con 0x7f075c0a5270 2024-08-19T10:55:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f0761fa0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f0761fa0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f075c0a5a40 con 0x7f075c1395b0 2024-08-19T10:55:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f0761fa0640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f0761fa0640 10 monclient(hunting): authenticate will time out at 1997.448056s 2024-08-19T10:55:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075b7fe640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f075c0a5270 msgr2=0x7f075c0a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:55:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f075c0a5270 0x7f075c0a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:47.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 0x7f075c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 0x7f075c0a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60398/0 (socket says 172.21.15.89:60398) 2024-08-19T10:55:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 -- 172.21.15.89:0/3825814904 learned_addr learned my addr 172.21.15.89:0/3825814904 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 10 monclient(hunting): get_auth_request con 0x7f075c0a6d30 auth_method 0 2024-08-19T10:55:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:47.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075bfff640 1 --2- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f075c1395b0 0x7f075c139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075bfff640 10 monclient(hunting): get_auth_request con 0x7f075c1395b0 auth_method 0 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 10 monclient(hunting): handle_auth_done global_id 7704 payload 274 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 -- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f075c1395b0 msgr2=0x7f075c139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 --2- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f075c1395b0 0x7f075c139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 -- 172.21.15.89:0/3825814904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f075c0a5270 msgr2=0x7f075c0a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 --2- 172.21.15.89:0/3825814904 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f075c0a5270 0x7f075c0a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 10 monclient: _finish_hunting 0 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 monclient: found mon.noname-a 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:47.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:55:47.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 -- 172.21.15.89:0/3825814904 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c0a81d0 con 0x7f075c0a6d30 2024-08-19T10:55:47.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 -- 172.21.15.89:0/3825814904 >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 msgr2=0x7f075c0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:47.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 -- 172.21.15.89:0/3825814904 >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 msgr2=0x7f075c0a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:47.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 --2- 172.21.15.89:0/3825814904 >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 0x7f075c0a7110 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f0744009870 tx=0x7f074401c440 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:47.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.630+0000 7f075affd640 1 --2- 172.21.15.89:0/3825814904 >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 0x7f075c0a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:47.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075affd640 1 --2- 172.21.15.89:0/3825814904 >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 0x7f075c0a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:47.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075affd640 10 monclient: get_auth_request con 0x7f075c0a6d30 auth_method 0 2024-08-19T10:55:47.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075affd640 0 --2- 172.21.15.89:0/3825814904 >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 0x7f075c0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075affd640 1 --2- 172.21.15.89:0/3825814904 >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 0x7f075c0a7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 10 monclient: ms_handle_reset current mon v2:172.21.15.89:3300/0 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 1 -- 172.21.15.89:0/3825814904 >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 msgr2=0x7f075c0a7110 unknown :-1 s=STATE_CLOSED l=0).mark_down 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 1 --2- 172.21.15.89:0/3825814904 >> v2:172.21.15.89:3300/0 conn(0x7f075c0a6d30 0x7f075c0a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 10 monclient: _add_conns ranks=[10,7,1,8,5,6,0,4] 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 1 --2- 172.21.15.89:0/3825814904 >> v2:172.21.15.190:3300/0 conn(0x7f0748003640 0x7f0748005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 10 monclient(hunting): picked mon.noname-f con 0x7f0748003640 addr v2:172.21.15.190:3300/0 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 1 --2- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f07480062b0 0x7f0748008740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 10 monclient(hunting): picked mon.noname-i con 0x7f07480062b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0748009090 addr v1:172.21.15.190:6789/0 2024-08-19T10:55:47.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 1 -- 172.21.15.89:0/3825814904 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f074800d040 con 0x7f0748009090 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 1 -- 172.21.15.89:0/3825814904 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f074800e2b0 con 0x7f0748003640 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 1 -- 172.21.15.89:0/3825814904 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f074800e480 con 0x7f07480062b0 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075a7fc640 10 monclient(hunting): ms_handle_reset stray mon v2:172.21.15.89:3300/0 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075affd640 1 -- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f07480062b0 msgr2=0x7f0748008740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075affd640 1 --2- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f07480062b0 0x7f0748008740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 1 --2- 172.21.15.89:0/3825814904 >> v2:172.21.15.190:3300/0 conn(0x7f0748003640 0x7f0748005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 10 monclient(hunting): get_auth_request con 0x7f0748003640 auth_method 0 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:47.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:55:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075bfff640 1 -- 172.21.15.89:0/3825814904 >> v1:172.21.15.190:6789/0 conn(0x7f0748009090 legacy=0x7f074800b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:47.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 10 monclient(hunting): handle_auth_done global_id 7704 payload 293 2024-08-19T10:55:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 1 -- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f07480062b0 msgr2=0x7f0748008740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 1 --2- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f07480062b0 0x7f0748008740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 1 -- 172.21.15.89:0/3825814904 >> v1:172.21.15.190:6789/0 conn(0x7f0748009090 legacy=0x7f074800b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:55:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 1 monclient: found mon.noname-f 2024-08-19T10:55:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:55:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075b7fe640 1 -- 172.21.15.89:0/3825814904 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074800e740 con 0x7f0748003640 2024-08-19T10:55:47.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.834+0000 7f075bfff640 1 -- 172.21.15.89:0/3825814904 reap_dead start 2024-08-19T10:55:47.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075b7fe640 1 -- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0748003640 msgr2=0x7f0748005a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:47.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075b7fe640 1 --2- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0748003640 0x7f0748005a90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0754003d10 tx=0x7f0744009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cba67b8d8f5c80c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 1 -- 172.21.15.89:0/3825814904 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0754009a30 con 0x7f0748003640 2024-08-19T10:55:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:55:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 10 monclient: dump: 2024-08-19T10:55:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:47.837 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:47.838 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:47.839 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 10 monclient: _finish_auth 0 2024-08-19T10:55:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:55:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 1 -- 172.21.15.89:0/3825814904 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07540097f0 con 0x7f0748003640 2024-08-19T10:55:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 1 -- 172.21.15.89:0/3825814904 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0754015400 con 0x7f0748003640 2024-08-19T10:55:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:47.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:55:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f075a7fc640 10 monclient: dump: 2024-08-19T10:55:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:47.840 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:47.841 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 5 monclient: authenticate success, global_id 7704 2024-08-19T10:55:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 10 monclient: get_monmap_and_config success 2024-08-19T10:55:47.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 4 set_mon_vals no callback set 2024-08-19T10:55:47.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 10 monclient: shutdown 2024-08-19T10:55:47.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 1 -- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0748003640 msgr2=0x7f0748005a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 1 --2- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0748003640 0x7f0748005a90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0754003d10 tx=0x7f0744009240 comp rx=0 tx=0).stop 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 1 -- 172.21.15.89:0/3825814904 shutdown_connections 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 1 --2- 172.21.15.89:0/3825814904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0748003640 0x7f0748005a90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 1 -- 172.21.15.89:0/3825814904 >> 172.21.15.89:0/3825814904 conn(0x7f075c069a80 msgr2=0x7f075c0abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 1 -- 172.21.15.89:0/3825814904 shutdown_connections 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.838+0000 7f0761fa0640 1 -- 172.21.15.89:0/3825814904 wait complete. 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient: build_initial_monmap 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient: monmap: 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:55:47.846 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:47.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 1 Processor -- start 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 1 -- start start 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient: init 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient: _add_conns ranks=[0,4,1,6,5,3,2,7,8] 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748003640 0x7f075c1d1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient(hunting): picked mon.noname-a con 0x7f0748003640 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f075c0a20d0 0x7f075c0a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient(hunting): picked mon.noname-c con 0x7f075c0a20d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f075c13d250 0x7f075c13d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient(hunting): picked mon.noname-f con 0x7f075c13d250 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:47.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f075c0a5650 con 0x7f0748003640 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f075c0a4a70 con 0x7f075c13d250 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f075c0a57d0 con 0x7f075c0a20d0 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f0761fa0640 10 monclient(hunting): authenticate will time out at 1997.662196s 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f075c0a20d0 0x7f075c0a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748003640 0x7f075c1d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f075c0a20d0 0x7f075c0a2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:39924/0 (socket says 172.21.15.89:39924) 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 1 -- 172.21.15.89:0/4278314626 learned_addr learned my addr 172.21.15.89:0/4278314626 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 10 monclient(hunting): get_auth_request con 0x7f075c0a20d0 auth_method 0 2024-08-19T10:55:47.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075b7fe640 10 monclient(hunting): get_auth_request con 0x7f0748003640 auth_method 0 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:55:47.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075bfff640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f075c13d250 0x7f075c13d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075bfff640 10 monclient(hunting): get_auth_request con 0x7f075c13d250 auth_method 0 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 10 monclient(hunting): handle_auth_done global_id 84505 payload 824 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 1 -- 172.21.15.89:0/4278314626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f075c13d250 msgr2=0x7f075c13d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f075c13d250 0x7f075c13d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 1 -- 172.21.15.89:0/4278314626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748003640 msgr2=0x7f075c1d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748003640 0x7f075c1d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 10 monclient: _finish_hunting 0 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 1 monclient: found mon.noname-c 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:55:47.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.842+0000 7f075affd640 1 -- 172.21.15.89:0/4278314626 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c09db70 con 0x7f075c0a20d0 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f075affd640 1 -- 172.21.15.89:0/4278314626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f075c0a20d0 msgr2=0x7f075c0a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f075affd640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f075c0a20d0 0x7f075c0a2510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f07440ad7f0 tx=0x7f07440191e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 1 -- 172.21.15.89:0/4278314626 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f07440044a0 con 0x7f075c0a20d0 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 10 monclient: dump: 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:47.852 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 10 monclient: _finish_auth 0 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 10 monclient: _check_auth_tickets 2024-08-19T10:55:47.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 1 -- 172.21.15.89:0/4278314626 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07440addf0 con 0x7f075c0a20d0 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0761fa0640 5 monclient: authenticate success, global_id 84505 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 1 -- 172.21.15.89:0/4278314626 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f0744003a90 con 0x7f075c0a20d0 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0761fa0640 10 monclient: _renew_subs 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0761fa0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0761fa0640 1 -- 172.21.15.89:0/4278314626 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c13ec10 con 0x7f075c0a20d0 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0759ffb640 4 set_mon_vals no callback set 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 10 monclient: dump: 2024-08-19T10:55:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:55:47.855 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0761fa0640 10 monclient: _renew_subs 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0761fa0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0761fa0640 1 -- 172.21.15.89:0/4278314626 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c13f770 con 0x7f075c0a20d0 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0761fa0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0761fa0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0761fa0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 1 -- 172.21.15.89:0/4278314626 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f0744003cb0 con 0x7f075c0a20d0 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0761fa0640 1 -- 172.21.15.89:0/4278314626 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075c0a2a50 con 0x7f075c0a20d0 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f073c03be00 0x7f073c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:47.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f0753fff640 1 -- 172.21.15.89:0/4278314626 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f07440e9160 con 0x7f075c0a20d0 2024-08-19T10:55:47.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f075b7fe640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f073c03be00 0x7f073c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:47.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.846+0000 7f075b7fe640 10 monclient: get_auth_request con 0x7f073c03be00 auth_method 0 2024-08-19T10:55:47.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.850+0000 7f075b7fe640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f073c03be00 0x7f073c03e2c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0748003470 tx=0x7f0754005000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:55:47.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.850+0000 7f0753fff640 1 -- 172.21.15.89:0/4278314626 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f0744016150 con 0x7f075c0a20d0 2024-08-19T10:55:47.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.850+0000 7f0753fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:55:47.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:47.850+0000 7f0753fff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:55:48.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.078+0000 7f0761fa0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:55:48.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.078+0000 7f0761fa0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:48.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.078+0000 7f0761fa0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:55:48.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.078+0000 7f0761fa0640 1 -- 172.21.15.89:0/4278314626 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f075c0a67f0 con 0x7f075c0a20d0 2024-08-19T10:55:48.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0753fff640 1 -- 172.21.15.89:0/4278314626 <== mon.4 v2:172.21.15.89:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3194 (secure 0 0 0) 0x7f07440e7020 con 0x7f075c0a20d0 2024-08-19T10:55:48.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0753fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:55:48.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0753fff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:55:48.080 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":140,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:55:48.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 -- 172.21.15.89:0/4278314626 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f073c03be00 msgr2=0x7f073c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:48.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f073c03be00 0x7f073c03e2c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0748003470 tx=0x7f0754005000 comp rx=0 tx=0).stop 2024-08-19T10:55:48.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 10 monclient: shutdown 2024-08-19T10:55:48.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:55:48.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 -- 172.21.15.89:0/4278314626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f075c0a20d0 msgr2=0x7f075c0a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:48.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f075c0a20d0 0x7f075c0a2510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f07440ad7f0 tx=0x7f07440191e0 comp rx=0 tx=0).stop 2024-08-19T10:55:48.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 -- 172.21.15.89:0/4278314626 shutdown_connections 2024-08-19T10:55:48.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f073c03be00 0x7f073c03e2c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:48.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f075c13d250 0x7f075c13d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:48.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f075c0a20d0 0x7f075c0a2510 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:48.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 --2- 172.21.15.89:0/4278314626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0748003640 0x7f075c1d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:48.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 -- 172.21.15.89:0/4278314626 >> 172.21.15.89:0/4278314626 conn(0x7f075c069a80 msgr2=0x7f075c134760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:55:48.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 -- 172.21.15.89:0/4278314626 shutdown_connections 2024-08-19T10:55:48.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:48.082+0000 7f0761fa0640 1 -- 172.21.15.89:0/4278314626 wait complete. 2024-08-19T10:55:51.098 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-08-19T10:55:51.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.226+0000 7fbbb08f8640 10 monclient: get_monmap_and_config 2024-08-19T10:55:51.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.226+0000 7fbbb08f8640 10 monclient: build_initial_monmap 2024-08-19T10:55:51.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.226+0000 7fbbb08f8640 10 monclient: monmap: 2024-08-19T10:55:51.225 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:55:51.225 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:55:51.225 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:55:51.231125+0000 2024-08-19T10:55:51.225 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:55:51.231125+0000 2024-08-19T10:55:51.226 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:55:51.226 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:55:51.226 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:55:51.226 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:55:51.226 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:55:51.226 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:55:51.226 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:55:51.227 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:55:51.227 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:55:51.227 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:55:51.227 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:55:51.227 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:55:51.227 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:55:51.227 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:55:51.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.226+0000 7fbbb08f8640 1 Processor -- start 2024-08-19T10:55:51.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.226+0000 7fbbb08f8640 1 -- start start 2024-08-19T10:55:51.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.226+0000 7fbbb08f8640 10 monclient: init 2024-08-19T10:55:51.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 10 monclient: _reopen_session rank -1 2024-08-19T10:55:51.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 10 monclient: _add_conns ranks=[0,9,1,8,4,3,10,7,2,5,6] 2024-08-19T10:55:51.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbbac06c930 addr v1:172.21.15.89:6789/0 2024-08-19T10:55:51.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:51.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 10 monclient(hunting): picked mon.noname-a con 0x7fbbac0a4240 addr v2:172.21.15.89:3300/0 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbbac0a4620 addr v1:172.21.15.190:6789/0 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbac0743e0 con 0x7fbbac06c930 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbac06d620 con 0x7fbbac0a4620 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fbbac0a12d0 con 0x7fbbac0a4240 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbb08f8640 10 monclient(hunting): authenticate will time out at 2001.048478s 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbba9d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbaa59c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbbac06c930 0x7fbbac0a3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:34924/0 (socket says 172.21.15.89:34924) 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 learned_addr learned my addr 172.21.15.89:0/426346212 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbbac0a4240 auth_method 0 2024-08-19T10:55:51.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:51.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:51.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbba9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:51.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 msgr2=0x7fbbac13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:51.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 msgr2=0x7fbbac13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:51.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:51.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:51.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> v1:172.21.15.89:6789/0 conn(0x7fbbac06c930 legacy=0x7fbbac0a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:51.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.230+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> v1:172.21.15.190:6789/0 conn(0x7fbbac0a4620 legacy=0x7fbbac140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:55:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.434+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.434+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbbac0a4240 auth_method 0 2024-08-19T10:55:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.434+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.434+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.434+0000 7fbba9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.434+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 msgr2=0x7fbbac13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.434+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 msgr2=0x7fbbac13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.434+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:51.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.434+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.838+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.838+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbbac0a4240 auth_method 0 2024-08-19T10:55:51.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.838+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:51.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.838+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:51.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.838+0000 7fbba9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:51.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.838+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 msgr2=0x7fbbac13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:51.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.838+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 msgr2=0x7fbbac13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:51.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.838+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:51.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:51.838+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:52.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:52.642+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:52.642+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbbac0a4240 auth_method 0 2024-08-19T10:55:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:52.642+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:52.642+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:52.642+0000 7fbba9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:52.642+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 msgr2=0x7fbbac13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:52.642+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 msgr2=0x7fbbac13e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:52.642+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:52.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:52.642+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:55:54.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 10 monclient(hunting): tick 2024-08-19T10:55:54.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:55:54.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 1 monclient(hunting): continuing hunt 2024-08-19T10:55:54.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:55:54.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 msgr2=0x7fbbac13e990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:54.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbbac0a4240 0x7fbbac13e990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:54.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 >> v1:172.21.15.190:6789/0 conn(0x7fbbac0a4620 legacy=0x7fbbac140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:54.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 >> v1:172.21.15.89:6789/0 conn(0x7fbbac06c930 legacy=0x7fbbac0a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:55:54.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 10 monclient: _add_conns ranks=[10,4,8,2,6,3,5,7] 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 10 monclient(hunting): picked mon.noname-f con 0x7fbb8c003900 addr v2:172.21.15.190:3300/0 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 10 monclient(hunting): picked mon.noname-c con 0x7fbb8c0063a0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 10 monclient(hunting): picked mon.noname-e con 0x7fbb8c008f40 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbbac0a12d0 con 0x7fbb8c003900 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbb8c00bb40 con 0x7fbb8c0063a0 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:54.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fbb8c00bd40 con 0x7fbb8c008f40 2024-08-19T10:55:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbb9bfff640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request con 0x7fbb8c003900 auth_method 0 2024-08-19T10:55:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:54.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbb8c0063a0 auth_method 0 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbba9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request con 0x7fbb8c008f40 auth_method 0 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 msgr2=0x7fbb8c005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 msgr2=0x7fbb8c005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 msgr2=0x7fbb8c0087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 msgr2=0x7fbb8c0087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:54.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:54.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:54.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:54.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:54.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:54.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:54.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.230+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbb8c0063a0 auth_method 0 2024-08-19T10:55:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbba9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request con 0x7fbb8c003900 auth_method 0 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaa59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request con 0x7fbb8c008f40 auth_method 0 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 msgr2=0x7fbb8c0087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 msgr2=0x7fbb8c0087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 msgr2=0x7fbb8c005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:54.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 msgr2=0x7fbb8c005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:54.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:54.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:54.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:54.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:54.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:54.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.434+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbb8c0063a0 auth_method 0 2024-08-19T10:55:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbba9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request con 0x7fbb8c008f40 auth_method 0 2024-08-19T10:55:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:54.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request con 0x7fbb8c003900 auth_method 0 2024-08-19T10:55:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaa59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 msgr2=0x7fbb8c0087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 msgr2=0x7fbb8c0087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:54.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 msgr2=0x7fbb8c005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 msgr2=0x7fbb8c005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:54.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:54.838+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:55.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:55.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request con 0x7fbb8c008f40 auth_method 0 2024-08-19T10:55:55.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:55.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:55.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbb8c0063a0 auth_method 0 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbba9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request con 0x7fbb8c003900 auth_method 0 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:55.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:55.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaa59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:55.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 msgr2=0x7fbb8c0087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:55.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 msgr2=0x7fbb8c0087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:55.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:55.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 msgr2=0x7fbb8c005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:55.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 msgr2=0x7fbb8c005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:55.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:55:55.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:55.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:55.642+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:55:57.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 10 monclient(hunting): tick 2024-08-19T10:55:57.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:55:57.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 monclient(hunting): continuing hunt 2024-08-19T10:55:57.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:55:57.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:57.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c008f40 0x7fbb8c00b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:57.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 msgr2=0x7fbb8c0087f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:57.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0063a0 0x7fbb8c0087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:57.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 msgr2=0x7fbb8c005d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c003900 0x7fbb8c005d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 reap_dead start 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 10 monclient: _add_conns ranks=[3,6,5,2,7] 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 10 monclient(hunting): picked mon.noname-g con 0x7fbb8c0067e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 10 monclient(hunting): picked mon.noname-d con 0x7fbb8c006c60 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 10 monclient(hunting): picked mon.noname-h con 0x7fbb8c007270 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbbac0a12d0 con 0x7fbb8c0067e0 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fbb8c00bb40 con 0x7fbb8c007270 2024-08-19T10:55:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fbb8c00bd40 con 0x7fbb8c006c60 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbb9bfff640 10 monclient(hunting): _renew_subs 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbb8c006c60 auth_method 0 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbba9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request con 0x7fbb8c0067e0 auth_method 0 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request con 0x7fbb8c007270 auth_method 0 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 msgr2=0x7fbb8c005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 msgr2=0x7fbb8c005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 msgr2=0x7fbb8c0076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:57.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 msgr2=0x7fbb8c0076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:57.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:57.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:57.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.230+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbb8c006c60 auth_method 0 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbba9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 msgr2=0x7fbb8c005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 msgr2=0x7fbb8c005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request con 0x7fbb8c007270 auth_method 0 2024-08-19T10:55:57.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request con 0x7fbb8c0067e0 auth_method 0 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaa59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 msgr2=0x7fbb8c0076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 msgr2=0x7fbb8c0076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:57.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:57.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:57.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.434+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:55:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbb8c006c60 auth_method 0 2024-08-19T10:55:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbba9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 msgr2=0x7fbb8c005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 msgr2=0x7fbb8c005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:57.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request con 0x7fbb8c0067e0 auth_method 0 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaa59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request con 0x7fbb8c007270 auth_method 0 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:57.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:57.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 msgr2=0x7fbb8c0076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:57.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 msgr2=0x7fbb8c0076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:57.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:57.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:57.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:57.838+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:55:58.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request con 0x7fbb8c0067e0 auth_method 0 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaa59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request con 0x7fbb8c007270 auth_method 0 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaad9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:58.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 msgr2=0x7fbb8c0076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 msgr2=0x7fbb8c0076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbb8c006c60 auth_method 0 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbba9d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 msgr2=0x7fbb8c005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 msgr2=0x7fbb8c005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-19T10:55:58.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:55:58.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:55:58.642+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:56:00.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 10 monclient(hunting): tick 2024-08-19T10:56:00.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:56:00.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 1 monclient(hunting): continuing hunt 2024-08-19T10:56:00.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:56:00.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 msgr2=0x7fbb8c005cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:00.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbb8c006c60 0x7fbb8c005cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 msgr2=0x7fbb8c0076f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbb8c007270 0x7fbb8c0076f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 msgr2=0x7fbb8c00b400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb8c0067e0 0x7fbb8c00b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 10 monclient: _add_conns ranks=[7,2] 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0080a0 0x7fbb8c008480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 10 monclient(hunting): picked mon.noname-i con 0x7fbb8c0080a0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c010390 0x7fbb8c012780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 10 monclient(hunting): picked mon.noname-b con 0x7fbb8c010390 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fbbac0a12d0 con 0x7fbb8c010390 2024-08-19T10:56:00.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:00.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fbb8c00bb40 con 0x7fbb8c0080a0 2024-08-19T10:56:00.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbb9bfff640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:00.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c010390 msgr2=0x7fbb8c012780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:00.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c010390 0x7fbb8c012780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:00.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0080a0 msgr2=0x7fbb8c008480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:00.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.230+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0080a0 0x7fbb8c008480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:00.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.434+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c010390 msgr2=0x7fbb8c012780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:00.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.434+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c010390 0x7fbb8c012780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:56:00.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.434+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0080a0 msgr2=0x7fbb8c008480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:00.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.434+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0080a0 0x7fbb8c008480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T10:56:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.838+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c010390 msgr2=0x7fbb8c012780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.838+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c010390 0x7fbb8c012780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:56:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.838+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0080a0 msgr2=0x7fbb8c008480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:00.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:00.838+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0080a0 0x7fbb8c008480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T10:56:01.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:01.642+0000 7fbbaa59c640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0080a0 msgr2=0x7fbb8c008480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:01.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:01.642+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0080a0 0x7fbb8c008480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:56:01.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:01.642+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c010390 msgr2=0x7fbb8c012780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:01.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:01.642+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c010390 0x7fbb8c012780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T10:56:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 10 monclient(hunting): tick 2024-08-19T10:56:03.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 10 monclient(hunting): _check_auth_tickets 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 1 monclient(hunting): continuing hunt 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 10 monclient(hunting): _reopen_session rank -1 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0080a0 msgr2=0x7fbb8c008480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0080a0 0x7fbb8c008480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c010390 msgr2=0x7fbb8c012780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbb8c010390 0x7fbb8c012780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaad9d640 1 -- 172.21.15.89:0/426346212 reap_dead start 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 10 monclient: _add_conns ranks=[4,9,10,3,6,1,0,5,2,7,8] 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0067e0 0x7fbb8c005cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 10 monclient(hunting): picked mon.noname-c con 0x7fbb8c0067e0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbb8c006c60 0x7fbb8c00b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 10 monclient(hunting): picked mon.noname-a con 0x7fbb8c006c60 addr v2:172.21.15.89:3300/0 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c007270 0x7fbb8c012780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 10 monclient(hunting): picked mon.noname-f con 0x7fbb8c007270 addr v2:172.21.15.190:3300/0 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fbbac0a12d0 con 0x7fbb8c006c60 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbb8c00bb40 con 0x7fbb8c007270 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 1 -- 172.21.15.89:0/426346212 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbb8c00bd40 con 0x7fbb8c0067e0 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbb9bfff640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0067e0 0x7fbb8c005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.89:3300/0 conn(0x7fbb8c006c60 0x7fbb8c00b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request con 0x7fbb8c0067e0 auth_method 0 2024-08-19T10:56:03.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaa59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaa59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbb8c006c60 auth_method 0 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaad9d640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c007270 0x7fbb8c012780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request con 0x7fbb8c007270 auth_method 0 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbbaad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 10 monclient(hunting): handle_auth_done global_id 7722 payload 274 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0067e0 msgr2=0x7fbb8c005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0067e0 0x7fbb8c005cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c007270 msgr2=0x7fbb8c012780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c007270 0x7fbb8c012780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:56:03.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 1 monclient: found mon.noname-a 2024-08-19T10:56:03.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:03.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:56:03.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.229+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac0a4e30 con 0x7fbb8c006c60 2024-08-19T10:56:03.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba9d9b640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c006c60 msgr2=0x7fbb8c00b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:03.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c006c60 0x7fbb8c00b400 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c008be0 tx=0x7fbba001b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b98440844df16c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 1 -- 172.21.15.89:0/426346212 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbba00ae070 con 0x7fbb8c006c60 2024-08-19T10:56:03.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:56:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 10 monclient: dump: 2024-08-19T10:56:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.242 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 10 monclient: _finish_auth 0 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 10 monclient: _check_auth_tickets 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 1 -- 172.21.15.89:0/426346212 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba00a88e0 con 0x7fbb8c006c60 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbbb08f8640 5 monclient: authenticate success, global_id 7722 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 1 -- 172.21.15.89:0/426346212 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbba000f040 con 0x7fbb8c006c60 2024-08-19T10:56:03.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbba959a640 10 monclient: dump: 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.244 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.245 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.245 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.245 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.245 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.245 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbbb08f8640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:03.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.237+0000 7fbbb08f8640 4 set_mon_vals no callback set 2024-08-19T10:56:03.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient: shutdown 2024-08-19T10:56:03.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:03.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 -- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c006c60 msgr2=0x7fbb8c00b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c006c60 0x7fbb8c00b400 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c008be0 tx=0x7fbba001b470 comp rx=0 tx=0).stop 2024-08-19T10:56:03.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 -- 172.21.15.89:0/426346212 shutdown_connections 2024-08-19T10:56:03.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 --2- 172.21.15.89:0/426346212 >> v2:172.21.15.190:3300/0 conn(0x7fbb8c007270 0x7fbb8c012780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb8c006c60 0x7fbb8c00b400 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 --2- 172.21.15.89:0/426346212 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c0067e0 0x7fbb8c005cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 -- 172.21.15.89:0/426346212 >> 172.21.15.89:0/426346212 conn(0x7fbbac06de40 msgr2=0x7fbbac06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 -- 172.21.15.89:0/426346212 shutdown_connections 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 -- 172.21.15.89:0/426346212 wait complete. 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient: build_initial_monmap 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient: monmap: 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:03.251 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 Processor -- start 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 -- start start 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient: init 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:03.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient: _add_conns ranks=[7,8,4,2,1,3,5,6,0] 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0067e0 0x7fbbac1271a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient(hunting): picked mon.noname-i con 0x7fbb8c0067e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c006c60 0x7fbbac1276e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient(hunting): picked mon.noname-e con 0x7fbb8c006c60 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c007270 0x7fbbac127c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient(hunting): picked mon.noname-c con 0x7fbb8c007270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbbac0a40c0 con 0x7fbb8c007270 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fbbac0a3f40 con 0x7fbb8c0067e0 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fbbac13c180 con 0x7fbb8c006c60 2024-08-19T10:56:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbb08f8640 10 monclient(hunting): authenticate will time out at 2013.063680s 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbaa59c640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0067e0 msgr2=0x7fbbac1271a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbba9d9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c006c60 0x7fbbac1276e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbaad9d640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c007270 0x7fbbac127c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbbaa59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0067e0 0x7fbbac1271a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbba9d9b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c006c60 0x7fbbac1276e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:51964/0 (socket says 172.21.15.89:51964) 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbba9d9b640 1 -- 172.21.15.89:0/2425124562 learned_addr learned my addr 172.21.15.89:0/2425124562 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request con 0x7fbb8c006c60 auth_method 0 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbba9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbba9d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.241+0000 7fbba9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request con 0x7fbb8c007270 auth_method 0 2024-08-19T10:56:03.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbaad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbaad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbaad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbaad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbaad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbaad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 10 monclient(hunting): handle_auth_done global_id 114245 payload 824 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 1 -- 172.21.15.89:0/2425124562 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0067e0 msgr2=0x7fbbac1271a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0067e0 0x7fbbac1271a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 1 -- 172.21.15.89:0/2425124562 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c007270 msgr2=0x7fbbac127c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c007270 0x7fbbac127c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 1 monclient: found mon.noname-e 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 1 -- 172.21.15.89:0/2425124562 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac0a22f0 con 0x7fbb8c006c60 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 1 -- 172.21.15.89:0/2425124562 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c006c60 msgr2=0x7fbbac1276e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba9d9b640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c006c60 0x7fbbac1276e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbba0000c10 tx=0x7fbba0008110 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 1 -- 172.21.15.89:0/2425124562 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbba00ae070 con 0x7fbb8c006c60 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 10 monclient: dump: 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.256 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.257 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 10 monclient: _finish_auth 0 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 1 -- 172.21.15.89:0/2425124562 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba00a8ca0 con 0x7fbb8c006c60 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbb08f8640 5 monclient: authenticate success, global_id 114245 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbb08f8640 10 monclient: _renew_subs 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbb08f8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbb08f8640 1 -- 172.21.15.89:0/2425124562 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbac12aba0 con 0x7fbb8c006c60 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:03.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 1 -- 172.21.15.89:0/2425124562 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbba000f040 con 0x7fbb8c006c60 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbb9affd640 10 monclient: dump: 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.259 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbb08f8640 10 monclient: _renew_subs 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbb08f8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbba8d99640 4 set_mon_vals no callback set 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.245+0000 7fbbb08f8640 1 -- 172.21.15.89:0/2425124562 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbac12b330 con 0x7fbb8c006c60 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.249+0000 7fbb9affd640 1 -- 172.21.15.89:0/2425124562 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbba00b1070 con 0x7fbb8c006c60 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.249+0000 7fbbb08f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.249+0000 7fbbb08f8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:03.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.249+0000 7fbbb08f8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.249+0000 7fbbb08f8640 1 -- 172.21.15.89:0/2425124562 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbac09f700 con 0x7fbb8c006c60 2024-08-19T10:56:03.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.249+0000 7fbb9affd640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbb8c03ee90 0x7fbb8c041350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.249+0000 7fbb9affd640 1 -- 172.21.15.89:0/2425124562 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fbba00eb1c0 con 0x7fbb8c006c60 2024-08-19T10:56:03.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.249+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbb8c03ee90 0x7fbb8c041350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.249+0000 7fbbaa59c640 10 monclient: get_auth_request con 0x7fbb8c03ee90 auth_method 0 2024-08-19T10:56:03.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.253+0000 7fbbaa59c640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbb8c03ee90 0x7fbb8c041350 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009010 tx=0x7fbb9c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.253+0000 7fbb9affd640 1 -- 172.21.15.89:0/2425124562 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fbbac0a22f0 con 0x7fbb8c006c60 2024-08-19T10:56:03.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.253+0000 7fbb9affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:03.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.253+0000 7fbb9affd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:03.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.497+0000 7fbbb08f8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-08-19T10:56:03.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.497+0000 7fbbb08f8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-08-19T10:56:03.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.497+0000 7fbbb08f8640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.497+0000 7fbbb08f8640 1 -- 172.21.15.89:0/2425124562 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbbac13c300 con 0x7fbb8c006c60 2024-08-19T10:56:03.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.497+0000 7fbb9affd640 1 -- 172.21.15.89:0/2425124562 <== mon.8 v2:172.21.15.89:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+3181 (secure 0 0 0) 0x7fbba000b140 con 0x7fbb8c006c60 2024-08-19T10:56:03.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.497+0000 7fbb9affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-08-19T10:56:03.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.497+0000 7fbb9affd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:03.497 INFO:teuthology.orchestra.run.smithi089.stdout:{"election_epoch":146,"quorum":[0,1,3,4,5,6,8],"quorum_names":["a","f","g","c","h","d","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","modified":"2024-08-19T10:37:22.945188Z","created":"2024-08-19T10:37:22.945188Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3301","nonce":0},{"type":"v1","addr":"172.21.15.89:6790","nonce":0}]},"addr":"172.21.15.89:6790/0","public_addr":"172.21.15.89:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3301","nonce":0},{"type":"v1","addr":"172.21.15.190:6790","nonce":0}]},"addr":"172.21.15.190:6790/0","public_addr":"172.21.15.190:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3302","nonce":0},{"type":"v1","addr":"172.21.15.89:6791","nonce":0}]},"addr":"172.21.15.89:6791/0","public_addr":"172.21.15.89:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3302","nonce":0},{"type":"v1","addr":"172.21.15.190:6791","nonce":0}]},"addr":"172.21.15.190:6791/0","public_addr":"172.21.15.190:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3303","nonce":0},{"type":"v1","addr":"172.21.15.89:6792","nonce":0}]},"addr":"172.21.15.89:6792/0","public_addr":"172.21.15.89:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3303","nonce":0},{"type":"v1","addr":"172.21.15.190:6792","nonce":0}]},"addr":"172.21.15.190:6792/0","public_addr":"172.21.15.190:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3304","nonce":0},{"type":"v1","addr":"172.21.15.89:6793","nonce":0}]},"addr":"172.21.15.89:6793/0","public_addr":"172.21.15.89:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-08-19T10:56:03.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 -- 172.21.15.89:0/2425124562 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbb8c03ee90 msgr2=0x7fbb8c041350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbb8c03ee90 0x7fbb8c041350 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009010 tx=0x7fbb9c01c040 comp rx=0 tx=0).stop 2024-08-19T10:56:03.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 10 monclient: shutdown 2024-08-19T10:56:03.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:03.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 -- 172.21.15.89:0/2425124562 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c006c60 msgr2=0x7fbbac1276e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c006c60 0x7fbbac1276e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbba0000c10 tx=0x7fbba0008110 comp rx=0 tx=0).stop 2024-08-19T10:56:03.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 -- 172.21.15.89:0/2425124562 shutdown_connections 2024-08-19T10:56:03.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbb8c007270 0x7fbbac127c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbb8c03ee90 0x7fbb8c041350 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbb8c006c60 0x7fbbac1276e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 --2- 172.21.15.89:0/2425124562 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fbb8c0067e0 0x7fbbac1271a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 -- 172.21.15.89:0/2425124562 >> 172.21.15.89:0/2425124562 conn(0x7fbbac06de40 msgr2=0x7fbbac0a73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:03.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 -- 172.21.15.89:0/2425124562 shutdown_connections 2024-08-19T10:56:03.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.501+0000 7fbbb08f8640 1 -- 172.21.15.89:0/2425124562 wait complete. 2024-08-19T10:56:03.516 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-08-19T10:56:03.516 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-08-19T10:56:03.517 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-08-19T10:56:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.637+0000 7fee14990640 10 monclient: get_monmap_and_config 2024-08-19T10:56:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.637+0000 7fee14990640 10 monclient: build_initial_monmap 2024-08-19T10:56:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.637+0000 7fee14990640 10 monclient: monmap: 2024-08-19T10:56:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:03.644885+0000 2024-08-19T10:56:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:03.644885+0000 2024-08-19T10:56:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.639 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:03.640 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:03.640 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 1 Processor -- start 2024-08-19T10:56:03.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 1 -- start start 2024-08-19T10:56:03.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 10 monclient: init 2024-08-19T10:56:03.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:03.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 10 monclient: _add_conns ranks=[1,7,8,3,2,10,6,0,9,5,4] 2024-08-19T10:56:03.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fee1009b650 addr v1:172.21.15.190:6789/0 2024-08-19T10:56:03.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee1006c930 0x7fee1009ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 10 monclient(hunting): picked mon.noname-i con 0x7fee1006c930 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee10135e00 0x7fee1013da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 10 monclient(hunting): picked mon.noname-e con 0x7fee10135e00 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee100a81b0 con 0x7fee1009b650 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fee100a0b50 con 0x7fee1006c930 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fee100a0e30 con 0x7fee10135e00 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee14990640 10 monclient(hunting): authenticate will time out at 2013.461403s 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee10135e00 0x7fee1013da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0dd9b640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee1006c930 msgr2=0x7fee1009ba30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0dd9b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee1006c930 0x7fee1009ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee10135e00 0x7fee1013da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:51978/0 (socket says 172.21.15.89:51978) 2024-08-19T10:56:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 1 -- 172.21.15.89:0/3997471842 learned_addr learned my addr 172.21.15.89:0/3997471842 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 10 monclient(hunting): get_auth_request con 0x7fee10135e00 auth_method 0 2024-08-19T10:56:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 10 monclient(hunting): handle_auth_done global_id 114254 payload 274 2024-08-19T10:56:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 1 -- 172.21.15.89:0/3997471842 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee1006c930 msgr2=0x7fee1009ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 1 --2- 172.21.15.89:0/3997471842 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee1006c930 0x7fee1009ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 1 -- 172.21.15.89:0/3997471842 >> v1:172.21.15.190:6789/0 conn(0x7fee1009b650 legacy=0x7fee101356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:56:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 1 monclient: found mon.noname-e 2024-08-19T10:56:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.641+0000 7fee0ed9d640 1 -- 172.21.15.89:0/3997471842 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee100a1bd0 con 0x7fee10135e00 2024-08-19T10:56:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0ed9d640 1 -- 172.21.15.89:0/3997471842 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee10135e00 msgr2=0x7fee1013da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0ed9d640 1 --2- 172.21.15.89:0/3997471842 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee10135e00 0x7fee1013da30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee08009e80 tx=0x7fee0801c9c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e8f4f96d3276a199 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 1 -- 172.21.15.89:0/3997471842 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee080091b0 con 0x7fee10135e00 2024-08-19T10:56:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 10 monclient: dump: 2024-08-19T10:56:03.655 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 10 monclient: _finish_auth 0 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 5 monclient: authenticate success, global_id 114254 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 1 -- 172.21.15.89:0/3997471842 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee080a5a10 con 0x7fee10135e00 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 1 -- 172.21.15.89:0/3997471842 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee080a5bc0 con 0x7fee10135e00 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 4 set_mon_vals no callback set 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee0d59a640 10 monclient: dump: 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 10 monclient: shutdown 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 1 -- 172.21.15.89:0/3997471842 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee10135e00 msgr2=0x7fee1013da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 1 --2- 172.21.15.89:0/3997471842 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee10135e00 0x7fee1013da30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee08009e80 tx=0x7fee0801c9c0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 1 -- 172.21.15.89:0/3997471842 shutdown_connections 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 1 --2- 172.21.15.89:0/3997471842 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee10135e00 0x7fee1013da30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 1 --2- 172.21.15.89:0/3997471842 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fee1006c930 0x7fee1009ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 1 -- 172.21.15.89:0/3997471842 >> 172.21.15.89:0/3997471842 conn(0x7fee100a83d0 msgr2=0x7fee100a87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 1 -- 172.21.15.89:0/3997471842 shutdown_connections 2024-08-19T10:56:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.645+0000 7fee14990640 1 -- 172.21.15.89:0/3997471842 wait complete. 2024-08-19T10:56:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient: build_initial_monmap 2024-08-19T10:56:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient: monmap: 2024-08-19T10:56:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 1 Processor -- start 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 1 -- start start 2024-08-19T10:56:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient: init 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient: _add_conns ranks=[8,5,6,4,0,1,3,7,2] 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee1006c930 0x7fee101cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient(hunting): picked mon.noname-e con 0x7fee1006c930 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee1009b650 0x7fee101cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient(hunting): picked mon.noname-h con 0x7fee1009b650 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fee10135e00 0x7fee101cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient(hunting): picked mon.noname-d con 0x7fee10135e00 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fee10069690 con 0x7fee1009b650 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fee10069510 con 0x7fee10135e00 2024-08-19T10:56:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fee10069810 con 0x7fee1006c930 2024-08-19T10:56:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee14990640 10 monclient(hunting): authenticate will time out at 2013.469460s 2024-08-19T10:56:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee1006c930 0x7fee101cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee1006c930 0x7fee101cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:51984/0 (socket says 172.21.15.89:51984) 2024-08-19T10:56:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 1 -- 172.21.15.89:0/1730663789 learned_addr learned my addr 172.21.15.89:0/1730663789 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 10 monclient(hunting): get_auth_request con 0x7fee1006c930 auth_method 0 2024-08-19T10:56:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0ed9d640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fee10135e00 0x7fee101cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0ed9d640 10 monclient(hunting): get_auth_request con 0x7fee10135e00 auth_method 0 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 10 monclient(hunting): handle_auth_done global_id 114263 payload 824 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0dd9b640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee1009b650 0x7fee101cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 1 -- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fee10135e00 msgr2=0x7fee101cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fee10135e00 0x7fee101cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 1 -- 172.21.15.89:0/1730663789 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee1009b650 msgr2=0x7fee101cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee1009b650 0x7fee101cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 1 monclient: found mon.noname-e 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0e59c640 1 -- 172.21.15.89:0/1730663789 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee100909e0 con 0x7fee1006c930 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0dd9b640 10 monclient: get_auth_request con 0x7fee1009b650 auth_method 0 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0ed9d640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fee10135e00 0x7fee101cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.649+0000 7fee0dd9b640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fee1009b650 0x7fee101cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:03.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0e59c640 1 -- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee1006c930 msgr2=0x7fee101cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:03.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0e59c640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee1006c930 0x7fee101cf540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c480 tx=0x7fee0000c940 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee00012cb0 con 0x7fee1006c930 2024-08-19T10:56:03.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:03.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 10 monclient: dump: 2024-08-19T10:56:03.666 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.666 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.666 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 10 monclient: _finish_auth 0 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee00017070 con 0x7fee1006c930 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee000117f0 con 0x7fee1006c930 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 10 monclient: dump: 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.669 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.670 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee14990640 5 monclient: authenticate success, global_id 114263 2024-08-19T10:56:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee14990640 10 monclient: _renew_subs 2024-08-19T10:56:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0cd99640 4 set_mon_vals no callback set 2024-08-19T10:56:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee14990640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee14990640 1 -- 172.21.15.89:0/1730663789 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1013dac0 con 0x7fee1006c930 2024-08-19T10:56:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee14990640 10 monclient: _renew_subs 2024-08-19T10:56:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee14990640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee14990640 1 -- 172.21.15.89:0/1730663789 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7fee10090700 con 0x7fee1006c930 2024-08-19T10:56:03.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee14990640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee14990640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008df70 0x7fee1008e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee14990640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee14990640 1 -- 172.21.15.89:0/1730663789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee10091330 con 0x7fee1008df70 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008df70 0x7fee1008e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 10 monclient: get_auth_request con 0x7fee1008df70 auth_method 0 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fee00011380 con 0x7fee1006c930 2024-08-19T10:56:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 10 monclient(hunting): handle_auth_done global_id 7731 payload 824 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 1 -- 172.21.15.89:0/1730663789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee1009ba30 con 0x7fee1008df70 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fedf003be00 0x7fedf003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fee0004f380 con 0x7fee1006c930 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0ed9d640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fedf003be00 0x7fedf003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0ed9d640 10 monclient: get_auth_request con 0x7fedf003be00 auth_method 0 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 1 -- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008df70 msgr2=0x7fee1008e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.653+0000 7fee0dd9b640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008df70 0x7fee1008e350 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fee100a2a90 tx=0x7fee0400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee04003bc0 con 0x7fee1008df70 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fedfeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fedfeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fedfeffd640 10 monclient: dump: 2024-08-19T10:56:03.672 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fee04016070 con 0x7fee1008df70 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fedfeffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fedfeffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008df70 msgr2=0x7fee1008e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fedfeffd640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008df70 0x7fee1008e350 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fee100a2a90 tx=0x7fee0400cb80 comp rx=0 tx=0).stop 2024-08-19T10:56:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008df70 msgr2=0x7fee1008e350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fedfeffd640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008df70 0x7fee1008e350 secure :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fee100a2a90 tx=0x7fee0400cb80 comp rx=0 tx=0).stop 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.657+0000 7fee0ed9d640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fedf003be00 0x7fedf003e2c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fee100a5ee0 tx=0x7fee080a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee14990640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee14990640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008e890 0x7fee1009c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee14990640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee14990640 1 -- 172.21.15.89:0/1730663789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee100902d0 con 0x7fee1008e890 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee0dd9b640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008e890 0x7fee1009c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee0dd9b640 10 monclient: get_auth_request con 0x7fee1008e890 auth_method 0 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee0dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee0dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee0dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee0dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee0dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee0dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee0dd9b640 10 monclient(hunting): handle_auth_done global_id 7740 payload 824 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.661+0000 7fee0dd9b640 1 -- 172.21.15.89:0/1730663789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fee10064590 con 0x7fee1008e890 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee0dd9b640 1 -- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008e890 msgr2=0x7fee1009c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee0dd9b640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008e890 0x7fee1009c280 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fee100a2a90 tx=0x7fee040059d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fee04005750 con 0x7fee1008e890 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fedfeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fedfeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fedfeffd640 10 monclient: dump: 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 <== mon.0 v2:172.21.15.89:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9416 (secure 0 0 0) 0x7fee040052e0 con 0x7fee1008e890 2024-08-19T10:56:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fedfeffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:56:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fedfeffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008e890 msgr2=0x7fee1009c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fedfeffd640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008e890 0x7fee1009c280 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fee100a2a90 tx=0x7fee040059d0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fedfeffd640 1 -- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008e890 msgr2=0x7fee1009c280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fedfeffd640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee1008e890 0x7fee1009c280 secure :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fee100a2a90 tx=0x7fee040059d0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.680 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:56:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:56:03.680 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "leader", 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 146, 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 1093479, 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:03.681 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:03.682 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:03.683 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:03.684 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:03.685 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:56:03.686 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.689 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.690 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.691 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:56:03.692 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.693 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.694 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:56:03.695 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.697 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.698 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.699 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.700 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:56:03.701 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:03.702 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:03.703 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:56:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.704 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:56:03.704 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:03.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee14990640 1 -- 172.21.15.89:0/1730663789 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fedf003be00 msgr2=0x7fedf003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee14990640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fedf003be00 0x7fedf003e2c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fee100a5ee0 tx=0x7fee080a7070 comp rx=0 tx=0).stop 2024-08-19T10:56:03.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee14990640 10 monclient: shutdown 2024-08-19T10:56:03.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee14990640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:03.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee14990640 1 -- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee1006c930 msgr2=0x7fee101cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee14990640 1 --2- 172.21.15.89:0/1730663789 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fee1006c930 0x7fee101cf540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c480 tx=0x7fee0000c940 comp rx=0 tx=0).stop 2024-08-19T10:56:03.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee0ed9d640 1 -- 172.21.15.89:0/1730663789 reap_dead start 2024-08-19T10:56:03.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee0ed9d640 1 -- 172.21.15.89:0/1730663789 reap_dead start 2024-08-19T10:56:03.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee14990640 1 -- 172.21.15.89:0/1730663789 shutdown_connections 2024-08-19T10:56:03.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee14990640 1 -- 172.21.15.89:0/1730663789 >> 172.21.15.89:0/1730663789 conn(0x7fee100a83d0 msgr2=0x7fee1009a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:03.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee14990640 1 -- 172.21.15.89:0/1730663789 shutdown_connections 2024-08-19T10:56:03.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.665+0000 7fee14990640 1 -- 172.21.15.89:0/1730663789 wait complete. 2024-08-19T10:56:03.706 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:56:03.707 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-08-19T10:56:03.707 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-08-19T10:56:03.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.821+0000 7f3f8ad15640 10 monclient: get_monmap_and_config 2024-08-19T10:56:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.821+0000 7f3f8ad15640 10 monclient: build_initial_monmap 2024-08-19T10:56:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.821+0000 7f3f8ad15640 10 monclient: monmap: 2024-08-19T10:56:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:03.827667+0000 2024-08-19T10:56:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:03.827667+0000 2024-08-19T10:56:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:03.822 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:03.823 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:03.823 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 1 Processor -- start 2024-08-19T10:56:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 1 -- start start 2024-08-19T10:56:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 10 monclient: init 2024-08-19T10:56:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 10 monclient: _add_conns ranks=[10,2,1,5,4,8,0,6,9,3,7] 2024-08-19T10:56:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3f840a5270 0x7f3f840a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 10 monclient(hunting): picked mon.noname-f con 0x7f3f840a5270 addr v2:172.21.15.190:3300/0 2024-08-19T10:56:03.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f840a6d30 0x7f3f840a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 10 monclient(hunting): picked mon.noname-b con 0x7f3f840a6d30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3f841395b0 addr v1:172.21.15.190:6789/0 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f840a8710 con 0x7f3f841395b0 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f3f840a7650 con 0x7f3f840a5270 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3f840a5a40 con 0x7f3f840a6d30 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f8ad15640 10 monclient(hunting): authenticate will time out at 2013.645003s 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f83fff640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f840a6d30 msgr2=0x7f3f840a7110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f83fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f840a6d30 0x7f3f840a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3f840a5270 0x7f3f840a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3f840a5270 0x7f3f840a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:34094/0 (socket says 172.21.15.89:34094) 2024-08-19T10:56:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 1 -- 172.21.15.89:0/3512038490 learned_addr learned my addr 172.21.15.89:0/3512038490 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 10 monclient(hunting): get_auth_request con 0x7f3f840a5270 auth_method 0 2024-08-19T10:56:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f892b2640 1 -- 172.21.15.89:0/3512038490 >> v1:172.21.15.190:6789/0 conn(0x7f3f841395b0 legacy=0x7f3f84139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_done global_id 45406 payload 274 2024-08-19T10:56:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 1 -- 172.21.15.89:0/3512038490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f840a6d30 msgr2=0x7f3f840a7110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 1 --2- 172.21.15.89:0/3512038490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f840a6d30 0x7f3f840a7110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 1 -- 172.21.15.89:0/3512038490 >> v1:172.21.15.190:6789/0 conn(0x7f3f841395b0 legacy=0x7f3f84139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 10 monclient: _finish_hunting 0 2024-08-19T10:56:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 1 monclient: found mon.noname-f 2024-08-19T10:56:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:03.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:03.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.825+0000 7f3f88ab1640 1 -- 172.21.15.89:0/3512038490 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f840a81d0 con 0x7f3f840a5270 2024-08-19T10:56:03.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f88ab1640 1 -- 172.21.15.89:0/3512038490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f840a5270 msgr2=0x7f3f840a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:03.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f88ab1640 1 --2- 172.21.15.89:0/3512038490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f840a5270 0x7f3f840a67f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3f780098f0 tx=0x7f3f7801c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e6fa3b96d0ea796 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 1 -- 172.21.15.89:0/3512038490 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f780aa970 con 0x7f3f840a5270 2024-08-19T10:56:03.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:03.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 10 monclient: dump: 2024-08-19T10:56:03.832 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.833 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 10 monclient: _finish_auth 0 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 10 monclient: _check_auth_tickets 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 1 -- 172.21.15.89:0/3512038490 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f780af070 con 0x7f3f840a5270 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 1 -- 172.21.15.89:0/3512038490 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f780a9380 con 0x7f3f840a5270 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 5 monclient: authenticate success, global_id 45406 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 4 set_mon_vals no callback set 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:03.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f837fe640 10 monclient: dump: 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.835 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient: shutdown 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3512038490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f840a5270 msgr2=0x7f3f840a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 --2- 172.21.15.89:0/3512038490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f840a5270 0x7f3f840a67f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3f780098f0 tx=0x7f3f7801c4c0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3512038490 shutdown_connections 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 --2- 172.21.15.89:0/3512038490 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3f840a6d30 0x7f3f840a7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 --2- 172.21.15.89:0/3512038490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f840a5270 0x7f3f840a67f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3512038490 >> 172.21.15.89:0/3512038490 conn(0x7f3f84069a80 msgr2=0x7f3f840abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3512038490 shutdown_connections 2024-08-19T10:56:03.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3512038490 wait complete. 2024-08-19T10:56:03.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient: build_initial_monmap 2024-08-19T10:56:03.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient: monmap: 2024-08-19T10:56:03.841 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:03.841 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:03.841 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:03.841 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:03.841 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:03.841 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.841 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:03.841 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:03.841 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 Processor -- start 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 -- start start 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient: init 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient: _add_conns ranks=[3,7,8,2,0,5,6,1,4] 2024-08-19T10:56:03.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f840a5270 0x7f3f84138010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient(hunting): picked mon.noname-g con 0x7f3f840a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:03.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3f840a6d30 0x7f3f84138550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient(hunting): picked mon.noname-i con 0x7f3f840a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:03.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f841395b0 0x7f3f8413d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient(hunting): picked mon.noname-e con 0x7f3f841395b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:03.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3f840a57d0 con 0x7f3f840a5270 2024-08-19T10:56:03.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f3f840a5650 con 0x7f3f840a6d30 2024-08-19T10:56:03.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3f84144240 con 0x7f3f841395b0 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f8ad15640 10 monclient(hunting): authenticate will time out at 2013.655138s 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f83fff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3f840a6d30 msgr2=0x7f3f84138550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f841395b0 0x7f3f8413d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f83fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3f840a6d30 0x7f3f84138550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f841395b0 0x7f3f8413d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:52000/0 (socket says 172.21.15.89:52000) 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 1 -- 172.21.15.89:0/3239630970 learned_addr learned my addr 172.21.15.89:0/3239630970 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 10 monclient(hunting): get_auth_request con 0x7f3f841395b0 auth_method 0 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f88ab1640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f840a5270 0x7f3f84138010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f88ab1640 10 monclient(hunting): get_auth_request con 0x7f3f840a5270 auth_method 0 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f88ab1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f88ab1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f88ab1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 10 monclient(hunting): handle_auth_done global_id 114272 payload 824 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 1 -- 172.21.15.89:0/3239630970 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3f840a6d30 msgr2=0x7f3f84138550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3f840a6d30 0x7f3f84138550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 1 -- 172.21.15.89:0/3239630970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f840a5270 msgr2=0x7f3f84138010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f840a5270 0x7f3f84138010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 10 monclient: _finish_hunting 0 2024-08-19T10:56:03.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 1 monclient: found mon.noname-e 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.833+0000 7f3f892b2640 1 -- 172.21.15.89:0/3239630970 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f841404d0 con 0x7f3f841395b0 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.837+0000 7f3f892b2640 1 -- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f841395b0 msgr2=0x7f3f8413d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.837+0000 7f3f892b2640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f841395b0 0x7f3f8413d4f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00c7a0 tx=0x7f3f7c00cc60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.837+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f7c017070 con 0x7f3f841395b0 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: dump: 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.846 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.847 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: _finish_auth 0 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: _check_auth_tickets 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f7c00aab0 con 0x7f3f841395b0 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f7c00ac60 con 0x7f3f841395b0 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: dump: 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.848 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f82ffd640 4 set_mon_vals no callback set 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f8ad15640 5 monclient: authenticate success, global_id 114272 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f8ad15640 10 monclient: _renew_subs 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f8ad15640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3239630970 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f840a2530 con 0x7f3f841395b0 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f8ad15640 10 monclient: _renew_subs 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f8ad15640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3239630970 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8413efe0 con 0x7f3f841395b0 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f8ad15640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f8ad15640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f84122960 0x7f3f8413da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f8ad15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3239630970 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3f84138a90 con 0x7f3f84122960 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f84122960 0x7f3f8413da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 10 monclient: get_auth_request con 0x7f3f84122960 auth_method 0 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3f7c00a640 con 0x7f3f841395b0 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_done global_id 84532 payload 824 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 1 -- 172.21.15.89:0/3239630970 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f840a67f0 con 0x7f3f84122960 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f6803be00 0x7f3f6803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f3f7c04ea50 con 0x7f3f841395b0 2024-08-19T10:56:03.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f83fff640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f6803be00 0x7f3f6803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f83fff640 10 monclient: get_auth_request con 0x7f3f6803be00 auth_method 0 2024-08-19T10:56:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 1 -- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f84122960 msgr2=0x7f3f8413da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f88ab1640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f84122960 0x7f3f8413da30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3f840a9960 tx=0x7f3f78019260 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f780044f0 con 0x7f3f84122960 2024-08-19T10:56:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:56:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.841+0000 7f3f817fa640 10 monclient: dump: 2024-08-19T10:56:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.854 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.845+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3f780b8070 con 0x7f3f84122960 2024-08-19T10:56:03.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.845+0000 7f3f817fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.845+0000 7f3f817fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.845+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f84122960 msgr2=0x7f3f8413da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.845+0000 7f3f817fa640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f84122960 0x7f3f8413da30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3f840a9960 tx=0x7f3f78019260 comp rx=0 tx=0).stop 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.845+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f84122960 msgr2=0x7f3f8413da30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.845+0000 7f3f817fa640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f84122960 0x7f3f8413da30 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3f840a9960 tx=0x7f3f78019260 comp rx=0 tx=0).stop 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.845+0000 7f3f83fff640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f6803be00 0x7f3f6803e2c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f70006fd0 tx=0x7f3f70006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f8ad15640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f8ad15640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f8413df70 0x7f3f8413e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f8ad15640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3239630970 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3f84064590 con 0x7f3f8413df70 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f88ab1640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f8413df70 0x7f3f8413e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f88ab1640 10 monclient: get_auth_request con 0x7f3f8413df70 auth_method 0 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f88ab1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:03.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f88ab1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f88ab1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f88ab1640 10 monclient(hunting): handle_auth_done global_id 84541 payload 824 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.849+0000 7f3f88ab1640 1 -- 172.21.15.89:0/3239630970 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3f84122d40 con 0x7f3f8413df70 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f88ab1640 1 -- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f8413df70 msgr2=0x7f3f8413e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f88ab1640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f8413df70 0x7f3f8413e380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3f780098f0 tx=0x7f3f780a9000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3f780c29e0 con 0x7f3f8413df70 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f817fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f817fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f817fa640 10 monclient: dump: 2024-08-19T10:56:03.857 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:03.858 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:03.859 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:03.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 <== mon.4 v2:172.21.15.89:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9246 (secure 0 0 0) 0x7f3f780af040 con 0x7f3f8413df70 2024-08-19T10:56:03.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f817fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:56:03.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f817fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:03.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f8413df70 msgr2=0x7f3f8413e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f817fa640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f8413df70 0x7f3f8413e380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3f780098f0 tx=0x7f3f780a9000 comp rx=0 tx=0).stop 2024-08-19T10:56:03.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f817fa640 1 -- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f8413df70 msgr2=0x7f3f8413e380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:03.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f817fa640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3f8413df70 0x7f3f8413e380 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3f780098f0 tx=0x7f3f780a9000 comp rx=0 tx=0).stop 2024-08-19T10:56:03.859 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:03.859 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 146, 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 172155, 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:56:03.860 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:03.861 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:03.862 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:56:03.863 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:03.864 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.865 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:56:03.866 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:56:03.867 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.868 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.869 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:56:03.870 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.871 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.872 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.873 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.874 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.875 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.876 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.877 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.878 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:03.879 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:03.880 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:56:03.881 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:03.882 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:03.882 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:03.882 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:56:03.882 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:03.882 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:03.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:03.882 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:56:03.882 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:03.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3239630970 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f6803be00 msgr2=0x7f3f6803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f8ad15640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3f6803be00 0x7f3f6803e2c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f70006fd0 tx=0x7f3f70006d40 comp rx=0 tx=0).stop 2024-08-19T10:56:03.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f8ad15640 10 monclient: shutdown 2024-08-19T10:56:03.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f8ad15640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:03.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f841395b0 msgr2=0x7f3f8413d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:03.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f8ad15640 1 --2- 172.21.15.89:0/3239630970 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3f841395b0 0x7f3f8413d4f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00c7a0 tx=0x7f3f7c00cc60 comp rx=0 tx=0).stop 2024-08-19T10:56:03.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f892b2640 1 -- 172.21.15.89:0/3239630970 reap_dead start 2024-08-19T10:56:03.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f892b2640 1 -- 172.21.15.89:0/3239630970 reap_dead start 2024-08-19T10:56:03.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3239630970 shutdown_connections 2024-08-19T10:56:03.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3239630970 >> 172.21.15.89:0/3239630970 conn(0x7f3f84069a80 msgr2=0x7f3f840a9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:03.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3239630970 shutdown_connections 2024-08-19T10:56:03.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:03.853+0000 7f3f8ad15640 1 -- 172.21.15.89:0/3239630970 wait complete. 2024-08-19T10:56:03.885 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:56:03.885 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-08-19T10:56:03.885 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-08-19T10:56:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient: get_monmap_and_config 2024-08-19T10:56:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient: build_initial_monmap 2024-08-19T10:56:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient: monmap: 2024-08-19T10:56:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:04.009932+0000 2024-08-19T10:56:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:04.009932+0000 2024-08-19T10:56:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:04.004 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:04.005 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 1 Processor -- start 2024-08-19T10:56:04.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 1 -- start start 2024-08-19T10:56:04.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient: init 2024-08-19T10:56:04.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient: _add_conns ranks=[8,9,7,5,3,0,6,2,10,4,1] 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77700a3620 0x7f77700a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient(hunting): picked mon.noname-e con 0x7f77700a3620 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f777009c9e0 0x7f77700a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient(hunting): picked mon.noname-a con 0x7f777009c9e0 addr v2:172.21.15.89:3300/0 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f77701373f0 0x7f7770137830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient(hunting): picked mon.noname-i con 0x7f77701373f0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f777009dab0 con 0x7f777009c9e0 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f77700a1f20 con 0x7f77701373f0 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f77700a4480 con 0x7f77700a3620 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f7774a5b640 10 monclient(hunting): authenticate will time out at 2013.826517s 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776dd9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f777009c9e0 0x7f77700a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77700a3620 0x7f77700a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776dd9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f777009c9e0 0x7f77700a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33902/0 (socket says 172.21.15.89:33902) 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776dd9b640 1 -- 172.21.15.89:0/3486518536 learned_addr learned my addr 172.21.15.89:0/3486518536 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776dd9b640 10 monclient(hunting): get_auth_request con 0x7f777009c9e0 auth_method 0 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776ed9d640 1 -- 172.21.15.89:0/3486518536 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f77701373f0 msgr2=0x7f7770137830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 10 monclient(hunting): get_auth_request con 0x7f77700a3620 auth_method 0 2024-08-19T10:56:04.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776ed9d640 1 --2- 172.21.15.89:0/3486518536 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f77701373f0 0x7f7770137830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 10 monclient(hunting): handle_auth_done global_id 114281 payload 274 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 1 -- 172.21.15.89:0/3486518536 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f77701373f0 msgr2=0x7f7770137830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 1 --2- 172.21.15.89:0/3486518536 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f77701373f0 0x7f7770137830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 1 -- 172.21.15.89:0/3486518536 >> v2:172.21.15.89:3300/0 conn(0x7f777009c9e0 msgr2=0x7f77700a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 1 --2- 172.21.15.89:0/3486518536 >> v2:172.21.15.89:3300/0 conn(0x7f777009c9e0 0x7f77700a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 10 monclient: _finish_hunting 0 2024-08-19T10:56:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 1 monclient: found mon.noname-e 2024-08-19T10:56:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776e59c640 1 -- 172.21.15.89:0/3486518536 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77700a1790 con 0x7f77700a3620 2024-08-19T10:56:04.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.005+0000 7f776dd9b640 1 --2- 172.21.15.89:0/3486518536 >> v2:172.21.15.89:3300/0 conn(0x7f777009c9e0 0x7f77700a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:04.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776e59c640 1 -- 172.21.15.89:0/3486518536 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77700a3620 msgr2=0x7f77700a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776e59c640 1 --2- 172.21.15.89:0/3486518536 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77700a3620 0x7f77700a3a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7768009dc0 tx=0x7f776801c800 comp rx=0 tx=0).ready entity=mon.8 client_cookie=4a74c9b1c8f974c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 1 -- 172.21.15.89:0/3486518536 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77680b2070 con 0x7f77700a3620 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 10 monclient: dump: 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.018 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 10 monclient: _finish_auth 0 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 1 -- 172.21.15.89:0/3486518536 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77680a5ab0 con 0x7f77700a3620 2024-08-19T10:56:04.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f7774a5b640 5 monclient: authenticate success, global_id 114281 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f7774a5b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f7774a5b640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f7774a5b640 4 set_mon_vals no callback set 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 1 -- 172.21.15.89:0/3486518536 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77680a5ca0 con 0x7f77700a3620 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.009+0000 7f776d59a640 10 monclient: dump: 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.020 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient: shutdown 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:04.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 -- 172.21.15.89:0/3486518536 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77700a3620 msgr2=0x7f77700a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 --2- 172.21.15.89:0/3486518536 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77700a3620 0x7f77700a3a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7768009dc0 tx=0x7f776801c800 comp rx=0 tx=0).stop 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 -- 172.21.15.89:0/3486518536 shutdown_connections 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 --2- 172.21.15.89:0/3486518536 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f77701373f0 0x7f7770137830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 --2- 172.21.15.89:0/3486518536 >> v2:172.21.15.89:3300/0 conn(0x7f777009c9e0 0x7f77700a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 --2- 172.21.15.89:0/3486518536 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f77700a3620 0x7f77700a3a00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 -- 172.21.15.89:0/3486518536 >> 172.21.15.89:0/3486518536 conn(0x7f77700a8210 msgr2=0x7f77700a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 -- 172.21.15.89:0/3486518536 shutdown_connections 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 -- 172.21.15.89:0/3486518536 wait complete. 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient: build_initial_monmap 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient: monmap: 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:04.022 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:04.023 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 Processor -- start 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 -- start start 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient: init 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient: _add_conns ranks=[3,2,6,1,5,8,7,0,4] 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f777009c9e0 0x7f77701cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient(hunting): picked mon.noname-g con 0x7f777009c9e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77700a3620 0x7f7770136470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient(hunting): picked mon.noname-b con 0x7f77700a3620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701373f0 0x7f777011e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient(hunting): picked mon.noname-d con 0x7f77701373f0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f777009df40 con 0x7f77700a3620 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7770138180 con 0x7f777009c9e0 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f7770140e60 con 0x7f77701373f0 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f7774a5b640 10 monclient(hunting): authenticate will time out at 2013.833621s 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776dd9b640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77700a3620 msgr2=0x7f7770136470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776dd9b640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77700a3620 0x7f7770136470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701373f0 0x7f777011e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701373f0 0x7f777011e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:39960/0 (socket says 172.21.15.89:39960) 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 1 -- 172.21.15.89:0/2479830460 learned_addr learned my addr 172.21.15.89:0/2479830460 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 10 monclient(hunting): get_auth_request con 0x7f77701373f0 auth_method 0 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776e59c640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f777009c9e0 0x7f77701cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 10 monclient(hunting): handle_auth_done global_id 45519 payload 824 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f777009c9e0 msgr2=0x7f77701cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f777009c9e0 0x7f77701cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77700a3620 msgr2=0x7f7770136470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f77700a3620 0x7f7770136470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 1 monclient: found mon.noname-d 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:56:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.013+0000 7f776ed9d640 1 -- 172.21.15.89:0/2479830460 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7770121e30 con 0x7f77701373f0 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776ed9d640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701373f0 msgr2=0x7f777011e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776ed9d640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701373f0 0x7f777011e260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f776000b410 tx=0x7f7760009f90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7760017070 con 0x7f77701373f0 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 10 monclient: dump: 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.027 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 10 monclient: _finish_auth 0 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f7774a5b640 5 monclient: authenticate success, global_id 45519 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7760003d90 con 0x7f77701373f0 2024-08-19T10:56:04.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f7774a5b640 10 monclient: _renew_subs 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f7774a5b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f7774a5b640 1 -- 172.21.15.89:0/2479830460 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77701203b0 con 0x7f77701373f0 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f7760006540 con 0x7f77701373f0 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776cd99640 4 set_mon_vals no callback set 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 10 monclient: dump: 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.029 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f7774a5b640 10 monclient: _renew_subs 2024-08-19T10:56:04.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f7774a5b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f7774a5b640 1 -- 172.21.15.89:0/2479830460 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f7770121b50 con 0x7f77701373f0 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f7774a5b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f7774a5b640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701369b0 0x7f777011e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f7774a5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f7774a5b640 1 -- 172.21.15.89:0/2479830460 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f777013da20 con 0x7f77701369b0 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776e59c640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701369b0 0x7f777011e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776e59c640 10 monclient: get_auth_request con 0x7f77701369b0 auth_method 0 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f77600060d0 con 0x7f77701373f0 2024-08-19T10:56:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776e59c640 10 monclient(hunting): handle_auth_done global_id 45528 payload 824 2024-08-19T10:56:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776e59c640 1 -- 172.21.15.89:0/2479830460 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7770137830 con 0x7f77701369b0 2024-08-19T10:56:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f775003be00 0x7f775003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f776004e7c0 con 0x7f77701373f0 2024-08-19T10:56:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776dd9b640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f775003be00 0x7f775003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776dd9b640 10 monclient: get_auth_request con 0x7f775003be00 auth_method 0 2024-08-19T10:56:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776dd9b640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f775003be00 msgr2=0x7f775003e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-08-19T10:56:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776dd9b640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f775003be00 msgr2=0x7f775003e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T10:56:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776dd9b640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f775003be00 0x7f775003e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T10:56:04.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.017+0000 7f776dd9b640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f775003be00 0x7f775003e2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:04.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f776e59c640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701369b0 msgr2=0x7f777011e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f776e59c640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701369b0 0x7f777011e7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f77680af000 tx=0x7f77680195a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77680bc020 con 0x7f77701369b0 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f775effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f775effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f775effd640 10 monclient: dump: 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.035 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7768004040 con 0x7f77701369b0 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f775effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f775effd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701369b0 msgr2=0x7f777011e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f775effd640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701369b0 0x7f777011e7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f77680af000 tx=0x7f77680195a0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701369b0 msgr2=0x7f777011e7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.021+0000 7f775effd640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701369b0 0x7f777011e7a0 secure :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f77680af000 tx=0x7f77680195a0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f7774a5b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f7774a5b640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f777011ece0 0x7f7770138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f7774a5b640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f7774a5b640 1 -- 172.21.15.89:0/2479830460 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f777011f0f0 con 0x7f777011ece0 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f776e59c640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f777011ece0 0x7f7770138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f776e59c640 10 monclient: get_auth_request con 0x7f777011ece0 auth_method 0 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f776e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f776e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f776e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f776e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f776e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f776e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f776e59c640 10 monclient(hunting): handle_auth_done global_id 45537 payload 824 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.025+0000 7f776e59c640 1 -- 172.21.15.89:0/2479830460 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f77700a6e00 con 0x7f777011ece0 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f776e59c640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f777011ece0 msgr2=0x7f7770138400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f776e59c640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f777011ece0 0x7f7770138400 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f77680b0040 tx=0x7f77680a7000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f77680bc050 con 0x7f777011ece0 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f775effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f775effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f775effd640 10 monclient: dump: 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.038 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 <== mon.6 v2:172.21.15.89:3303/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9413 (secure 0 0 0) 0x7f77680041f0 con 0x7f777011ece0 2024-08-19T10:56:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f775effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:56:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f775effd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f777011ece0 msgr2=0x7f7770138400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f775effd640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f777011ece0 0x7f7770138400 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f77680b0040 tx=0x7f77680a7000 comp rx=0 tx=0).stop 2024-08-19T10:56:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f775effd640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f777011ece0 msgr2=0x7f7770138400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:04.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f775effd640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f777011ece0 0x7f7770138400 secure :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f77680b0040 tx=0x7f77680a7000 comp rx=0 tx=0).stop 2024-08-19T10:56:04.040 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:04.040 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:56:04.040 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 146, 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 687156, 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.041 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.042 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.043 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:56:04.044 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.045 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:56:04.046 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.047 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.048 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:56:04.049 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:56:04.050 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.051 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.052 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.053 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.054 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.055 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.056 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:56:04.057 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.058 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.059 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.062 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:56:04.063 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.064 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.065 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.066 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.067 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.067 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 4 2024-08-19T10:56:04.067 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.067 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.067 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.067 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:56:04.067 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:04.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f7774a5b640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f775003be00 msgr2=0x7f775003e2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f7774a5b640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f775003be00 0x7f775003e2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f7774a5b640 10 monclient: shutdown 2024-08-19T10:56:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f7774a5b640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f7774a5b640 1 -- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701373f0 msgr2=0x7f777011e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f7774a5b640 1 --2- 172.21.15.89:0/2479830460 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f77701373f0 0x7f777011e260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f776000b410 tx=0x7f7760009f90 comp rx=0 tx=0).stop 2024-08-19T10:56:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f776ed9d640 1 -- 172.21.15.89:0/2479830460 reap_dead start 2024-08-19T10:56:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f776ed9d640 1 -- 172.21.15.89:0/2479830460 reap_dead start 2024-08-19T10:56:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f7774a5b640 1 -- 172.21.15.89:0/2479830460 shutdown_connections 2024-08-19T10:56:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f7774a5b640 1 -- 172.21.15.89:0/2479830460 >> 172.21.15.89:0/2479830460 conn(0x7f77700a8210 msgr2=0x7f7770072e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f7774a5b640 1 -- 172.21.15.89:0/2479830460 shutdown_connections 2024-08-19T10:56:04.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.029+0000 7f7774a5b640 1 -- 172.21.15.89:0/2479830460 wait complete. 2024-08-19T10:56:04.070 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:56:04.070 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-08-19T10:56:04.070 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-08-19T10:56:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.177+0000 7f6a78ad9640 10 monclient: get_monmap_and_config 2024-08-19T10:56:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.177+0000 7f6a78ad9640 10 monclient: build_initial_monmap 2024-08-19T10:56:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.177+0000 7f6a78ad9640 10 monclient: monmap: 2024-08-19T10:56:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:04.184058+0000 2024-08-19T10:56:04.178 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:04.184058+0000 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:04.179 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 1 Processor -- start 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 1 -- start start 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 10 monclient: init 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 10 monclient: _add_conns ranks=[4,6,3,5,7,9,0,10,1,8,2] 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a740a6860 0x7f6a740a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 10 monclient(hunting): picked mon.noname-c con 0x7f6a740a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6a740a7180 0x7f6a740a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 10 monclient(hunting): picked mon.noname-d con 0x7f6a740a7180 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a74139620 0x7f6a74139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 10 monclient(hunting): picked mon.noname-g con 0x7f6a74139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:04.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6a740a8190 con 0x7f6a74139620 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f6a740a33d0 con 0x7f6a740a6860 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6a740a2ec0 con 0x7f6a740a7180 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a78ad9640 10 monclient(hunting): authenticate will time out at 2014.000966s 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6a740a7180 0x7f6a740a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a7259c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a740a6860 0x7f6a740a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a7259c640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a740a6860 0x7f6a740a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:33608/0 (socket says 172.21.15.89:33608) 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a7259c640 1 -- 172.21.15.89:0/5058397 learned_addr learned my addr 172.21.15.89:0/5058397 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a7259c640 10 monclient(hunting): get_auth_request con 0x7f6a740a6860 auth_method 0 2024-08-19T10:56:04.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a7259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a7259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a7259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 10 monclient(hunting): get_auth_request con 0x7f6a740a7180 auth_method 0 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a7259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a7259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a7259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 10 monclient(hunting): handle_auth_done global_id 45546 payload 274 2024-08-19T10:56:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a72d9d640 1 --2- 172.21.15.89:0/5058397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a74139620 0x7f6a74139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 1 -- 172.21.15.89:0/5058397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a740a6860 msgr2=0x7f6a740a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 1 --2- 172.21.15.89:0/5058397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a740a6860 0x7f6a740a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 1 -- 172.21.15.89:0/5058397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a74139620 msgr2=0x7f6a74139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 1 --2- 172.21.15.89:0/5058397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a74139620 0x7f6a74139a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:56:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 1 monclient: found mon.noname-d 2024-08-19T10:56:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:56:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a71d9b640 1 -- 172.21.15.89:0/5058397 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a740a4cd0 con 0x7f6a740a7180 2024-08-19T10:56:04.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.181+0000 7f6a7259c640 1 --2- 172.21.15.89:0/5058397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a740a6860 0x7f6a740a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:04.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a71d9b640 1 -- 172.21.15.89:0/5058397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6a740a7180 msgr2=0x7f6a740a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a71d9b640 1 --2- 172.21.15.89:0/5058397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6a740a7180 0x7f6a740a7560 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6a680097f0 tx=0x7f6a6801c3a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a6369095f7c5450b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 1 -- 172.21.15.89:0/5058397 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a680b0070 con 0x7f6a740a7180 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 10 monclient: dump: 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.189 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 10 monclient: _finish_auth 0 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 10 monclient: _check_auth_tickets 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 1 -- 172.21.15.89:0/5058397 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a680aa810 con 0x7f6a740a7180 2024-08-19T10:56:04.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 5 monclient: authenticate success, global_id 45546 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 1 -- 172.21.15.89:0/5058397 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a680a9300 con 0x7f6a740a7180 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 4 set_mon_vals no callback set 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a7159a640 10 monclient: dump: 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.191 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient: shutdown 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:04.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 -- 172.21.15.89:0/5058397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6a740a7180 msgr2=0x7f6a740a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 --2- 172.21.15.89:0/5058397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6a740a7180 0x7f6a740a7560 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6a680097f0 tx=0x7f6a6801c3a0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 -- 172.21.15.89:0/5058397 shutdown_connections 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 --2- 172.21.15.89:0/5058397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a74139620 0x7f6a74139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 --2- 172.21.15.89:0/5058397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6a740a7180 0x7f6a740a7560 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 --2- 172.21.15.89:0/5058397 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f6a740a6860 0x7f6a740a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 -- 172.21.15.89:0/5058397 >> 172.21.15.89:0/5058397 conn(0x7f6a74074b40 msgr2=0x7f6a740abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 -- 172.21.15.89:0/5058397 shutdown_connections 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 -- 172.21.15.89:0/5058397 wait complete. 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient: build_initial_monmap 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient: monmap: 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:04.193 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 Processor -- start 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 -- start start 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient: init 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient: _add_conns ranks=[3,1,0,6,4,8,2,7,5] 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a740a6860 0x7f6a74134990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient(hunting): picked mon.noname-g con 0x7f6a740a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a740a7180 0x7f6a74134ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient(hunting): picked mon.noname-f con 0x7f6a740a7180 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a74139620 0x7f6a7409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient(hunting): picked mon.noname-a con 0x7f6a74139620 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74143210 con 0x7f6a74139620 2024-08-19T10:56:04.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74143090 con 0x7f6a740a7180 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6a74143390 con 0x7f6a740a6860 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a78ad9640 10 monclient(hunting): authenticate will time out at 2014.007726s 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a72d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a74139620 0x7f6a7409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a72d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a74139620 0x7f6a7409c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33910/0 (socket says 172.21.15.89:33910) 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a72d9d640 1 -- 172.21.15.89:0/250183746 learned_addr learned my addr 172.21.15.89:0/250183746 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a72d9d640 10 monclient(hunting): get_auth_request con 0x7f6a74139620 auth_method 0 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a72d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a72d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a72d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a72d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a72d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.185+0000 7f6a72d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a7259c640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a740a6860 0x7f6a74134990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 10 monclient(hunting): handle_auth_done global_id 7758 payload 824 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a71d9b640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a740a7180 0x7f6a74134ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 1 -- 172.21.15.89:0/250183746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a740a6860 msgr2=0x7f6a74134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a740a6860 0x7f6a74134990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 1 -- 172.21.15.89:0/250183746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a740a7180 msgr2=0x7f6a74134ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a740a7180 0x7f6a74134ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 1 monclient: found mon.noname-a 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:56:04.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 1 -- 172.21.15.89:0/250183746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a740a8f60 con 0x7f6a74139620 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 1 -- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a74139620 msgr2=0x7f6a7409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a72d9d640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a74139620 0x7f6a7409c9e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00ccb0 tx=0x7f6a6c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a6c00ce60 con 0x7f6a74139620 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 10 monclient: dump: 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.202 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 10 monclient: _finish_auth 0 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:04.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6c00fa70 con 0x7f6a74139620 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a78ad9640 5 monclient: authenticate success, global_id 7758 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a78ad9640 10 monclient: _renew_subs 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a78ad9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a78ad9640 1 -- 172.21.15.89:0/250183746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7409e0f0 con 0x7f6a74139620 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a6c00fc20 con 0x7f6a74139620 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 10 monclient: dump: 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.204 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.205 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a70d99640 4 set_mon_vals no callback set 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a78ad9640 10 monclient: _renew_subs 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a78ad9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a78ad9640 1 -- 172.21.15.89:0/250183746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a7409fbb0 con 0x7f6a74139620 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a78ad9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a78ad9640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409cf20 0x7f6a7409d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a78ad9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a78ad9640 1 -- 172.21.15.89:0/250183746 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6a740a7560 con 0x7f6a7409cf20 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a7259c640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409cf20 0x7f6a7409d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a7259c640 10 monclient: get_auth_request con 0x7f6a7409cf20 auth_method 0 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a7259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a7259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a7259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a7259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a7259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a7259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6a6c00f600 con 0x7f6a74139620 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a7259c640 10 monclient(hunting): handle_auth_done global_id 114290 payload 824 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a7259c640 1 -- 172.21.15.89:0/250183746 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a74064590 con 0x7f6a7409cf20 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a5403be00 0x7f6a5403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.189+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f6a6c009030 con 0x7f6a74139620 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a71d9b640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a5403be00 0x7f6a5403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a71d9b640 10 monclient: get_auth_request con 0x7f6a5403be00 auth_method 0 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a7259c640 1 -- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409cf20 msgr2=0x7f6a7409d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a7259c640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409cf20 0x7f6a7409d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6a640107c0 tx=0x7f6a64010bc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a6400f040 con 0x7f6a7409cf20 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a62ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a62ffd640 10 monclient: dump: 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.208 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6a6401c400 con 0x7f6a7409cf20 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a62ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a62ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409cf20 msgr2=0x7f6a7409d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a62ffd640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409cf20 0x7f6a7409d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6a640107c0 tx=0x7f6a64010bc0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409cf20 msgr2=0x7f6a7409d300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:04.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a62ffd640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409cf20 0x7f6a7409d300 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6a640107c0 tx=0x7f6a64010bc0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.193+0000 7f6a71d9b640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a5403be00 0x7f6a5403e2c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6a680097c0 tx=0x7f6a680a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409d840 0x7f6a7413aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 1 -- 172.21.15.89:0/250183746 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f6a7409ec50 con 0x7f6a7409d840 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409d840 0x7f6a7413aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 10 monclient: get_auth_request con 0x7f6a7409d840 auth_method 0 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 10 monclient(hunting): handle_auth_done global_id 114299 payload 824 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 1 -- 172.21.15.89:0/250183746 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6a7406cab0 con 0x7f6a7409d840 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 1 -- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409d840 msgr2=0x7f6a7413aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a7259c640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409d840 0x7f6a7413aa20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6a640107c0 tx=0x7f6a640046d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6a640212c0 con 0x7f6a7409d840 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a62ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a62ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a62ffd640 10 monclient: dump: 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.211 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 <== mon.8 v2:172.21.15.89:3304/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9245 (secure 0 0 0) 0x7f6a640212c0 con 0x7f6a7409d840 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a62ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:56:04.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a62ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409d840 msgr2=0x7f6a7413aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a62ffd640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409d840 0x7f6a7413aa20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6a640107c0 tx=0x7f6a640046d0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a62ffd640 1 -- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409d840 msgr2=0x7f6a7413aa20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:04.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a62ffd640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f6a7409d840 0x7f6a7413aa20 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6a640107c0 tx=0x7f6a640046d0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.213 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:04.213 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:56:04.213 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:56:04.213 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:56:04.213 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 146, 2024-08-19T10:56:04.213 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 44850, 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:56:04.214 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.215 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:56:04.217 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.218 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.219 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.220 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.222 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.223 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.224 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:56:04.225 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:56:04.226 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.227 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.228 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:56:04.229 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:56:04.231 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.232 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:56:04.233 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:56:04.234 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 2 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:56:04.235 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 1 -- 172.21.15.89:0/250183746 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a5403be00 msgr2=0x7f6a5403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6a5403be00 0x7f6a5403e2c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6a680097c0 tx=0x7f6a680a6000 comp rx=0 tx=0).stop 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 10 monclient: shutdown 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 1 -- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a74139620 msgr2=0x7f6a7409c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 1 --2- 172.21.15.89:0/250183746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a74139620 0x7f6a7409c9e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00ccb0 tx=0x7f6a6c00a5c0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a72d9d640 1 -- 172.21.15.89:0/250183746 reap_dead start 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a72d9d640 1 -- 172.21.15.89:0/250183746 reap_dead start 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 1 -- 172.21.15.89:0/250183746 shutdown_connections 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 1 -- 172.21.15.89:0/250183746 >> 172.21.15.89:0/250183746 conn(0x7f6a74074b40 msgr2=0x7f6a740a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 1 -- 172.21.15.89:0/250183746 shutdown_connections 2024-08-19T10:56:04.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.201+0000 7f6a78ad9640 1 -- 172.21.15.89:0/250183746 wait complete. 2024-08-19T10:56:04.238 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:56:04.238 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-08-19T10:56:04.238 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-08-19T10:56:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient: get_monmap_and_config 2024-08-19T10:56:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient: build_initial_monmap 2024-08-19T10:56:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient: monmap: 2024-08-19T10:56:04.351 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:04.357207+0000 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:04.357207+0000 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:04.353 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:04.353 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:04.353 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:04.353 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 1 Processor -- start 2024-08-19T10:56:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 1 -- start start 2024-08-19T10:56:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient: init 2024-08-19T10:56:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient: _add_conns ranks=[3,0,10,1,9,8,2,6,5,7,4] 2024-08-19T10:56:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e00a3620 0x7f31e00a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient(hunting): picked mon.noname-g con 0x7f31e00a3620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f31e009c9e0 addr v1:172.21.15.89:6789/0 2024-08-19T10:56:04.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31e00a3f40 0x7f31e0137b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient(hunting): picked mon.noname-f con 0x7f31e00a3f40 addr v2:172.21.15.190:3300/0 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e00a1790 con 0x7f31e009c9e0 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f31e009dab0 con 0x7f31e00a3f40 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f31e00a1f20 con 0x7f31e00a3620 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e844a640 10 monclient(hunting): authenticate will time out at 2014.173554s 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e59e5640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f31e009c9e0 0x7f31e01373f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60922/0 (socket says 172.21.15.89:60922) 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e59e5640 1 -- 172.21.15.89:0/3929983312 learned_addr learned my addr 172.21.15.89:0/3929983312 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 1 --2- 172.21.15.89:0/3929983312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e00a3620 0x7f31e00a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 10 monclient(hunting): get_auth_request con 0x7f31e00a3620 auth_method 0 2024-08-19T10:56:04.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e69e7640 1 --2- 172.21.15.89:0/3929983312 >> v2:172.21.15.190:3300/0 conn(0x7f31e00a3f40 0x7f31e0137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e69e7640 10 monclient(hunting): get_auth_request con 0x7f31e00a3f40 auth_method 0 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e69e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e69e7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e69e7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e69e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e69e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e69e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 10 monclient(hunting): handle_auth_done global_id 114240 payload 274 2024-08-19T10:56:04.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 1 -- 172.21.15.89:0/3929983312 >> v2:172.21.15.190:3300/0 conn(0x7f31e00a3f40 msgr2=0x7f31e0137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:04.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 1 --2- 172.21.15.89:0/3929983312 >> v2:172.21.15.190:3300/0 conn(0x7f31e00a3f40 0x7f31e0137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 1 -- 172.21.15.89:0/3929983312 >> v1:172.21.15.89:6789/0 conn(0x7f31e009c9e0 legacy=0x7f31e01373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:04.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 10 monclient: _finish_hunting 0 2024-08-19T10:56:04.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 1 monclient: found mon.noname-g 2024-08-19T10:56:04.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:04.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:56:04.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.353+0000 7f31e61e6640 1 -- 172.21.15.89:0/3929983312 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e009d1a0 con 0x7f31e00a3620 2024-08-19T10:56:04.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e61e6640 1 -- 172.21.15.89:0/3929983312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e00a3620 msgr2=0x7f31e00a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e61e6640 1 --2- 172.21.15.89:0/3929983312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e00a3620 0x7f31e00a3a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009d90 tx=0x7f31dc01c800 comp rx=0 tx=0).ready entity=mon.3 client_cookie=493e8ee02f333e7a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 1 -- 172.21.15.89:0/3929983312 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31dc0b2070 con 0x7f31e00a3620 2024-08-19T10:56:04.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:56:04.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 10 monclient: dump: 2024-08-19T10:56:04.362 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.362 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.362 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.362 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.362 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 10 monclient: _finish_auth 0 2024-08-19T10:56:04.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 10 monclient: _check_auth_tickets 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e844a640 5 monclient: authenticate success, global_id 114240 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e844a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 1 -- 172.21.15.89:0/3929983312 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31dc0a5a90 con 0x7f31e00a3620 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e844a640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e844a640 4 set_mon_vals no callback set 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 1 -- 172.21.15.89:0/3929983312 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31dc0a5c80 con 0x7f31e00a3620 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.357+0000 7f31e51e4640 10 monclient: dump: 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.364 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.365 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient: shutdown 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 -- 172.21.15.89:0/3929983312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e00a3620 msgr2=0x7f31e00a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 --2- 172.21.15.89:0/3929983312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e00a3620 0x7f31e00a3a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009d90 tx=0x7f31dc01c800 comp rx=0 tx=0).stop 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 -- 172.21.15.89:0/3929983312 shutdown_connections 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 --2- 172.21.15.89:0/3929983312 >> v2:172.21.15.190:3300/0 conn(0x7f31e00a3f40 0x7f31e0137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 --2- 172.21.15.89:0/3929983312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e00a3620 0x7f31e00a3a00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 -- 172.21.15.89:0/3929983312 >> 172.21.15.89:0/3929983312 conn(0x7f31e00a8210 msgr2=0x7f31e00a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 -- 172.21.15.89:0/3929983312 shutdown_connections 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 -- 172.21.15.89:0/3929983312 wait complete. 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient: build_initial_monmap 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient: monmap: 2024-08-19T10:56:04.366 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:04.367 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 Processor -- start 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 -- start start 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient: init 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient: _add_conns ranks=[3,2,5,0,7,6,8,1,4] 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e009c9e0 0x7f31e013df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient(hunting): picked mon.noname-g con 0x7f31e009c9e0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31e00a3620 0x7f31e01370f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient(hunting): picked mon.noname-b con 0x7f31e00a3620 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f31e00a3f40 0x7f31e0137630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient(hunting): picked mon.noname-h con 0x7f31e00a3f40 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:04.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f31e0138040 con 0x7f31e00a3620 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f31e00a4350 con 0x7f31e009c9e0 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f31e01381c0 con 0x7f31e00a3f40 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e844a640 10 monclient(hunting): authenticate will time out at 2014.181702s 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e59e5640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31e00a3620 msgr2=0x7f31e01370f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e59e5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31e00a3620 0x7f31e01370f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e69e7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f31e00a3f40 0x7f31e0137630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e009c9e0 0x7f31e013df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e009c9e0 0x7f31e013df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:55294/0 (socket says 172.21.15.89:55294) 2024-08-19T10:56:04.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 1 -- 172.21.15.89:0/753302823 learned_addr learned my addr 172.21.15.89:0/753302823 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 10 monclient(hunting): get_auth_request con 0x7f31e009c9e0 auth_method 0 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e69e7640 10 monclient(hunting): get_auth_request con 0x7f31e00a3f40 auth_method 0 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e69e7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e69e7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e69e7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e69e7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e69e7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e69e7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 10 monclient(hunting): handle_auth_done global_id 114249 payload 824 2024-08-19T10:56:04.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 1 -- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f31e00a3f40 msgr2=0x7f31e0137630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f31e00a3f40 0x7f31e0137630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 1 -- 172.21.15.89:0/753302823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31e00a3620 msgr2=0x7f31e01370f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:04.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f31e00a3620 0x7f31e01370f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 10 monclient: _finish_hunting 0 2024-08-19T10:56:04.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 1 monclient: found mon.noname-g 2024-08-19T10:56:04.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:04.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:56:04.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.361+0000 7f31e61e6640 1 -- 172.21.15.89:0/753302823 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e0073380 con 0x7f31e009c9e0 2024-08-19T10:56:04.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31e61e6640 1 -- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e009c9e0 msgr2=0x7f31e013df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31e61e6640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e009c9e0 0x7f31e013df70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f31dc01d0a0 tx=0x7f31dc0195a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31dc0b2050 con 0x7f31e009c9e0 2024-08-19T10:56:04.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:56:04.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 10 monclient: dump: 2024-08-19T10:56:04.374 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.374 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.375 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 10 monclient: _finish_auth 0 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31dc0be070 con 0x7f31e009c9e0 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31dc003dc0 con 0x7f31e009c9e0 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31d6ffd640 10 monclient: dump: 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.376 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31e844a640 5 monclient: authenticate success, global_id 114249 2024-08-19T10:56:04.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31e844a640 10 monclient: _renew_subs 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31e49e3640 4 set_mon_vals no callback set 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31e844a640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31e844a640 1 -- 172.21.15.89:0/753302823 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e01237e0 con 0x7f31e009c9e0 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31e844a640 10 monclient: _renew_subs 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31e844a640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.365+0000 7f31e844a640 1 -- 172.21.15.89:0/753302823 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e00a0d60 con 0x7f31e009c9e0 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e844a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e844a640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009f940 0x7f31e002d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e844a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e844a640 1 -- 172.21.15.89:0/753302823 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f31e00738b0 con 0x7f31e009f940 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009f940 0x7f31e002d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 10 monclient: get_auth_request con 0x7f31e009f940 auth_method 0 2024-08-19T10:56:04.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 10 monclient(hunting): handle_auth_done global_id 45424 payload 824 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 1 -- 172.21.15.89:0/753302823 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31e00735b0 con 0x7f31e009f940 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f31dc004040 con 0x7f31e009c9e0 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31d6ffd640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31c403be30 0x7f31c403e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f31dc0ea550 con 0x7f31e009c9e0 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e69e7640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31c403be30 0x7f31c403e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e69e7640 10 monclient: get_auth_request con 0x7f31c403be30 auth_method 0 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 1 -- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009f940 msgr2=0x7f31e002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.369+0000 7f31e59e5640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009f940 0x7f31e002d1d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f31e013de80 tx=0x7f31d800c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31e69e7640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31c403be30 0x7f31c403e2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009ac0 tx=0x7f31d0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31d8017070 con 0x7f31e009f940 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31d6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31d6ffd640 10 monclient: dump: 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.380 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f31d8005dc0 con 0x7f31e009f940 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31d6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31d6ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009f940 msgr2=0x7f31e002d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31d6ffd640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009f940 0x7f31e002d1d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f31e013de80 tx=0x7f31d800c8e0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009f940 msgr2=0x7f31e002d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:04.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.373+0000 7f31d6ffd640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009f940 0x7f31e002d1d0 secure :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f31e013de80 tx=0x7f31d800c8e0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e844a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e844a640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009fd20 0x7f31e002d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e844a640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e844a640 1 -- 172.21.15.89:0/753302823 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0064590 con 0x7f31e009fd20 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e59e5640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009fd20 0x7f31e002d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e59e5640 10 monclient: get_auth_request con 0x7f31e009fd20 auth_method 0 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e59e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e59e5640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e59e5640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e59e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e59e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e59e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e59e5640 10 monclient(hunting): handle_auth_done global_id 45433 payload 824 2024-08-19T10:56:04.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.381+0000 7f31e59e5640 1 -- 172.21.15.89:0/753302823 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f31e013ab20 con 0x7f31e009fd20 2024-08-19T10:56:04.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e59e5640 1 -- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009fd20 msgr2=0x7f31e002d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e59e5640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009fd20 0x7f31e002d710 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f31e013e5d0 tx=0x7f31d80049e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f31d8017040 con 0x7f31e009fd20 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31d6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31d6ffd640 10 monclient: dump: 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.385 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 <== mon.1 v2:172.21.15.190:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9413 (secure 0 0 0) 0x7f31d8022410 con 0x7f31e009fd20 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31d6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31d6ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009fd20 msgr2=0x7f31e002d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31d6ffd640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009fd20 0x7f31e002d710 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f31e013e5d0 tx=0x7f31d80049e0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31d6ffd640 1 -- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009fd20 msgr2=0x7f31e002d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:04.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31d6ffd640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e009fd20 0x7f31e002d710 secure :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f31e013e5d0 tx=0x7f31d80049e0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.387 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:04.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:56:04.387 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:56:04.387 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:56:04.387 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 146, 2024-08-19T10:56:04.387 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 687516, 2024-08-19T10:56:04.387 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:56:04.387 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 4, 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.388 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.389 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:56:04.390 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:56:04.391 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:56:04.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.393 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:56:04.394 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.395 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.396 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:56:04.397 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.399 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.400 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.401 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:56:04.402 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.404 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:56:04.405 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.406 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.407 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:04.408 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: "osd": [ 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 4 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.409 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.410 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:56:04.410 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:04.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e844a640 1 -- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31c403be30 msgr2=0x7f31c403e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e844a640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f31c403be30 0x7f31c403e2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009ac0 tx=0x7f31d0009190 comp rx=0 tx=0).stop 2024-08-19T10:56:04.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e844a640 10 monclient: shutdown 2024-08-19T10:56:04.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e844a640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:04.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e844a640 1 -- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e009c9e0 msgr2=0x7f31e013df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e844a640 1 --2- 172.21.15.89:0/753302823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e009c9e0 0x7f31e013df70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f31dc01d0a0 tx=0x7f31dc0195a0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e69e7640 1 -- 172.21.15.89:0/753302823 reap_dead start 2024-08-19T10:56:04.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e69e7640 1 -- 172.21.15.89:0/753302823 reap_dead start 2024-08-19T10:56:04.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e844a640 1 -- 172.21.15.89:0/753302823 shutdown_connections 2024-08-19T10:56:04.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e844a640 1 -- 172.21.15.89:0/753302823 >> 172.21.15.89:0/753302823 conn(0x7f31e00a8210 msgr2=0x7f31e0072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:04.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e844a640 1 -- 172.21.15.89:0/753302823 shutdown_connections 2024-08-19T10:56:04.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.385+0000 7f31e844a640 1 -- 172.21.15.89:0/753302823 wait complete. 2024-08-19T10:56:04.412 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:56:04.412 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-08-19T10:56:04.413 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-08-19T10:56:04.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient: get_monmap_and_config 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient: build_initial_monmap 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient: monmap: 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:04.533547+0000 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:04.533547+0000 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:04.528 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:04.529 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:04.529 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 1 Processor -- start 2024-08-19T10:56:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 1 -- start start 2024-08-19T10:56:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient: init 2024-08-19T10:56:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:04.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient: _add_conns ranks=[6,4,8,9,3,0,7,1,10,2,5] 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f988c0a67d0 0x7f988c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient(hunting): picked mon.noname-d con 0x7f988c0a67d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988c0a6bb0 0x7f988c0a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient(hunting): picked mon.noname-c con 0x7f988c0a6bb0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c139590 0x7f988c1399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient(hunting): picked mon.noname-e con 0x7f988c139590 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f988c0a8740 con 0x7f988c0a6bb0 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f988c0a3380 con 0x7f988c0a67d0 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f988c0a8250 con 0x7f988c139590 2024-08-19T10:56:04.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f98920f1640 10 monclient(hunting): authenticate will time out at 2014.350239s 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988c0a6bb0 0x7f988c0a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c139590 0x7f988c1399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988b7fe640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f988c0a67d0 0x7f988c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988bfff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c139590 0x7f988c1399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:52042/0 (socket says 172.21.15.89:52042) 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988bfff640 1 -- 172.21.15.89:0/924432515 learned_addr learned my addr 172.21.15.89:0/924432515 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 10 monclient(hunting): get_auth_request con 0x7f988c0a6bb0 auth_method 0 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988b7fe640 10 monclient(hunting): get_auth_request con 0x7f988c0a67d0 auth_method 0 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988bfff640 10 monclient(hunting): get_auth_request con 0x7f988c139590 auth_method 0 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 10 monclient(hunting): handle_auth_done global_id 84559 payload 274 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 1 -- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c139590 msgr2=0x7f988c1399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 1 --2- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c139590 0x7f988c1399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 1 -- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f988c0a67d0 msgr2=0x7f988c0a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 1 --2- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f988c0a67d0 0x7f988c0a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 10 monclient: _finish_hunting 0 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 1 monclient: found mon.noname-c 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f9883fff640 1 -- 172.21.15.89:0/924432515 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988c0a4cc0 con 0x7f988c0a6bb0 2024-08-19T10:56:04.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.529+0000 7f988b7fe640 1 --2- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f988c0a67d0 0x7f988c0a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f9883fff640 1 -- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988c0a6bb0 msgr2=0x7f988c0a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f9883fff640 1 --2- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988c0a6bb0 0x7f988c0a6f90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f98740097f0 tx=0x7f987401c430 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8b2deb7bbac89e81 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 1 -- 172.21.15.89:0/924432515 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98740b0070 con 0x7f988c0a6bb0 2024-08-19T10:56:04.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 10 monclient: dump: 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.542 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 10 monclient: _finish_auth 0 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 5 monclient: authenticate success, global_id 84559 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 1 -- 172.21.15.89:0/924432515 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98740aa810 con 0x7f988c0a6bb0 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:04.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 4 set_mon_vals no callback set 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 1 -- 172.21.15.89:0/924432515 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98740aae30 con 0x7f988c0a6bb0 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f988affd640 10 monclient: dump: 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.544 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 10 monclient: shutdown 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 1 -- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988c0a6bb0 msgr2=0x7f988c0a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 1 --2- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988c0a6bb0 0x7f988c0a6f90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f98740097f0 tx=0x7f987401c430 comp rx=0 tx=0).stop 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 1 -- 172.21.15.89:0/924432515 shutdown_connections 2024-08-19T10:56:04.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 1 --2- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c139590 0x7f988c1399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 1 --2- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988c0a6bb0 0x7f988c0a6f90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 1 --2- 172.21.15.89:0/924432515 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f988c0a67d0 0x7f988c0a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.533+0000 7f98920f1640 1 -- 172.21.15.89:0/924432515 >> 172.21.15.89:0/924432515 conn(0x7f988c069ae0 msgr2=0x7f988c0abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 1 -- 172.21.15.89:0/924432515 shutdown_connections 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 1 -- 172.21.15.89:0/924432515 wait complete. 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient: build_initial_monmap 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient: monmap: 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.546 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 1 Processor -- start 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 1 -- start start 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient: init 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:04.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient: _add_conns ranks=[2,8,4,5,6,0,3,7,1] 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988c0a67d0 0x7f988c134a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient(hunting): picked mon.noname-b con 0x7f988c0a67d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c0a6bb0 0x7f988c134f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient(hunting): picked mon.noname-e con 0x7f988c0a6bb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988c139590 0x7f988c09c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient(hunting): picked mon.noname-c con 0x7f988c139590 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f988c143390 con 0x7f988c0a67d0 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f988c143210 con 0x7f988c139590 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f988c143510 con 0x7f988c0a6bb0 2024-08-19T10:56:04.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f988b7fe640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988c0a67d0 msgr2=0x7f988c134a00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f98920f1640 10 monclient(hunting): authenticate will time out at 2014.357578s 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f988b7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988c0a67d0 0x7f988c134a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c0a6bb0 0x7f988c134f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f988bfff640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988c139590 0x7f988c09c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c0a6bb0 0x7f988c134f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:52044/0 (socket says 172.21.15.89:52044) 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 1 -- 172.21.15.89:0/1644253880 learned_addr learned my addr 172.21.15.89:0/1644253880 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f988bfff640 10 monclient(hunting): get_auth_request con 0x7f988c139590 auth_method 0 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f988bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f988bfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f988bfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 10 monclient(hunting): get_auth_request con 0x7f988c0a6bb0 auth_method 0 2024-08-19T10:56:04.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f988bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f988bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f988bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 10 monclient(hunting): handle_auth_done global_id 114317 payload 824 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 1 -- 172.21.15.89:0/1644253880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988c139590 msgr2=0x7f988c09c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f988c139590 0x7f988c09c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 1 -- 172.21.15.89:0/1644253880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988c0a67d0 msgr2=0x7f988c134a00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:04.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f988c0a67d0 0x7f988c134a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 10 monclient: _finish_hunting 0 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 1 monclient: found mon.noname-e 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.537+0000 7f9883fff640 1 -- 172.21.15.89:0/1644253880 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988c0a2a20 con 0x7f988c0a6bb0 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9883fff640 1 -- 172.21.15.89:0/1644253880 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c0a6bb0 msgr2=0x7f988c134f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9883fff640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c0a6bb0 0x7f988c134f40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f987401c260 tx=0x7f98740191d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98740b0040 con 0x7f988c0a6bb0 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 10 monclient: dump: 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.551 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 10 monclient: _finish_auth 0 2024-08-19T10:56:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:56:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98740b9070 con 0x7f988c0a6bb0 2024-08-19T10:56:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f98740044b0 con 0x7f988c0a6bb0 2024-08-19T10:56:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 10 monclient: dump: 2024-08-19T10:56:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.553 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.554 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988a7fc640 4 set_mon_vals no callback set 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f98920f1640 5 monclient: authenticate success, global_id 114317 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f98920f1640 10 monclient: _renew_subs 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f98920f1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f98920f1640 1 -- 172.21.15.89:0/1644253880 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988c09e2d0 con 0x7f988c0a6bb0 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f98920f1640 10 monclient: _renew_subs 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f98920f1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f98920f1640 1 -- 172.21.15.89:0/1644253880 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f988c09ee30 con 0x7f988c0a6bb0 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f98920f1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f98920f1640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09d6e0 0x7f988c02d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f98920f1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f98920f1640 1 -- 172.21.15.89:0/1644253880 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f988c0a5500 con 0x7f988c09d6e0 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9874004040 con 0x7f988c0a6bb0 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988b7fe640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09d6e0 0x7f988c02d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988b7fe640 10 monclient: get_auth_request con 0x7f988c09d6e0 auth_method 0 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f986c03be00 0x7f986c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f98740e9130 con 0x7f988c0a6bb0 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988bfff640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f986c03be00 0x7f986c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988bfff640 10 monclient: get_auth_request con 0x7f986c03be00 auth_method 0 2024-08-19T10:56:04.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988b7fe640 10 monclient(hunting): handle_auth_done global_id 114258 payload 824 2024-08-19T10:56:04.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.541+0000 7f988b7fe640 1 -- 172.21.15.89:0/1644253880 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f988c064590 con 0x7f988c09d6e0 2024-08-19T10:56:04.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f988b7fe640 1 -- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09d6e0 msgr2=0x7f988c02d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f988b7fe640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09d6e0 0x7f988c02d1d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f987c0096a0 tx=0x7f987c00ec00 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f988bfff640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f986c03be00 0x7f986c03e2c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f988c09e070 tx=0x7f988400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f987c017070 con 0x7f988c09d6e0 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f9888ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f9888ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f9888ff9640 10 monclient: dump: 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.560 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f987c00cb60 con 0x7f988c09d6e0 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f9888ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f9888ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09d6e0 msgr2=0x7f988c02d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f9888ff9640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09d6e0 0x7f988c02d1d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f987c0096a0 tx=0x7f987c00ec00 comp rx=0 tx=0).stop 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09d6e0 msgr2=0x7f988c02d1d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.545+0000 7f9888ff9640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09d6e0 0x7f988c02d1d0 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f987c0096a0 tx=0x7f987c00ec00 comp rx=0 tx=0).stop 2024-08-19T10:56:04.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.549+0000 7f98920f1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.549+0000 7f98920f1640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09dac0 0x7f988c02d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.549+0000 7f98920f1640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.549+0000 7f98920f1640 1 -- 172.21.15.89:0/1644253880 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f988c0a7020 con 0x7f988c09dac0 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09dac0 0x7f988c02d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 10 monclient: get_auth_request con 0x7f988c09dac0 auth_method 0 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 10 monclient(hunting): handle_auth_done global_id 114267 payload 824 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 1 -- 172.21.15.89:0/1644253880 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f988c06ca20 con 0x7f988c09dac0 2024-08-19T10:56:04.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 1 -- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09dac0 msgr2=0x7f988c02d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f988b7fe640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09dac0 0x7f988c02d710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f987c019000 tx=0x7f987c004740 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f987c017070 con 0x7f988c09dac0 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f9888ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f9888ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.553+0000 7f9888ff9640 10 monclient: dump: 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.563 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 <== mon.3 v2:172.21.15.190:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9245 (secure 0 0 0) 0x7f987c006650 con 0x7f988c09dac0 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f9888ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f9888ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09dac0 msgr2=0x7f988c02d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f9888ff9640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09dac0 0x7f988c02d710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f987c019000 tx=0x7f987c004740 comp rx=0 tx=0).stop 2024-08-19T10:56:04.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f9888ff9640 1 -- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09dac0 msgr2=0x7f988c02d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:04.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f9888ff9640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f988c09dac0 0x7f988c02d710 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f987c019000 tx=0x7f987c004740 comp rx=0 tx=0).stop 2024-08-19T10:56:04.565 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:04.565 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:56:04.565 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:56:04.565 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:56:04.565 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 146, 2024-08-19T10:56:04.565 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 45207, 2024-08-19T10:56:04.565 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:56:04.565 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.566 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:56:04.567 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:56:04.568 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:56:04.569 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.570 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.572 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.573 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:56:04.574 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.575 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:56:04.576 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.577 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.578 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:56:04.579 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.580 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:56:04.582 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.584 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:56:04.585 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:56:04.586 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.587 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:56:04.588 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f98920f1640 1 -- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f986c03be00 msgr2=0x7f986c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f98920f1640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f986c03be00 0x7f986c03e2c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f988c09e070 tx=0x7f988400b040 comp rx=0 tx=0).stop 2024-08-19T10:56:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f98920f1640 10 monclient: shutdown 2024-08-19T10:56:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f98920f1640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f98920f1640 1 -- 172.21.15.89:0/1644253880 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c0a6bb0 msgr2=0x7f988c134f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f98920f1640 1 --2- 172.21.15.89:0/1644253880 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f988c0a6bb0 0x7f988c134f40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f987401c260 tx=0x7f98740191d0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f988bfff640 1 -- 172.21.15.89:0/1644253880 reap_dead start 2024-08-19T10:56:04.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f988bfff640 1 -- 172.21.15.89:0/1644253880 reap_dead start 2024-08-19T10:56:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f98920f1640 1 -- 172.21.15.89:0/1644253880 shutdown_connections 2024-08-19T10:56:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f98920f1640 1 -- 172.21.15.89:0/1644253880 >> 172.21.15.89:0/1644253880 conn(0x7f988c069ae0 msgr2=0x7f988c138a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f98920f1640 1 -- 172.21.15.89:0/1644253880 shutdown_connections 2024-08-19T10:56:04.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.557+0000 7f98920f1640 1 -- 172.21.15.89:0/1644253880 wait complete. 2024-08-19T10:56:04.590 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:56:04.590 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-08-19T10:56:04.590 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-08-19T10:56:04.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient: get_monmap_and_config 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient: build_initial_monmap 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient: monmap: 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:04.701646+0000 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:04.701646+0000 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:04.696 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:04.697 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:04.697 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:04.697 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:04.697 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:04.697 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:04.697 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:04.697 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:04.697 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 1 Processor -- start 2024-08-19T10:56:04.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 1 -- start start 2024-08-19T10:56:04.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient: init 2024-08-19T10:56:04.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient: _add_conns ranks=[4,8,0,9,10,6,5,3,7,2,1] 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c80a6860 0x7ff0c80a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient(hunting): picked mon.noname-c con 0x7ff0c80a6860 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 0x7ff0c80a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient(hunting): picked mon.noname-e con 0x7ff0c80a7180 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff0c8139620 addr v1:172.21.15.89:6789/0 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c80a33d0 con 0x7ff0c8139620 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7ff0c80a8190 con 0x7ff0c80a6860 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff0c80a2ec0 con 0x7ff0c80a7180 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:04.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cf121640 10 monclient(hunting): authenticate will time out at 2014.518261s 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0ccebd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c80a6860 0x7ff0c80a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 0x7ff0c80a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cd6be640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff0c8139620 0x7ff0c8139a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60932/0 (socket says 172.21.15.89:60932) 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0cd6be640 1 -- 172.21.15.89:0/591805917 learned_addr learned my addr 172.21.15.89:0/591805917 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 10 monclient(hunting): get_auth_request con 0x7ff0c80a7180 auth_method 0 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0ccebd640 10 monclient(hunting): get_auth_request con 0x7ff0c80a6860 auth_method 0 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0ccebd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0ccebd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0ccebd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0ccebd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0ccebd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0ccebd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_done global_id 114326 payload 274 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 1 -- 172.21.15.89:0/591805917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c80a6860 msgr2=0x7ff0c80a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 1 --2- 172.21.15.89:0/591805917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c80a6860 0x7ff0c80a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 1 -- 172.21.15.89:0/591805917 >> v1:172.21.15.89:6789/0 conn(0x7ff0c8139620 legacy=0x7ff0c8139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 1 monclient: found mon.noname-e 2024-08-19T10:56:04.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0c7fff640 1 -- 172.21.15.89:0/591805917 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c80a4cd0 con 0x7ff0c80a7180 2024-08-19T10:56:04.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.697+0000 7ff0ccebd640 1 --2- 172.21.15.89:0/591805917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c80a6860 0x7ff0c80a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c7fff640 1 -- 172.21.15.89:0/591805917 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 msgr2=0x7ff0c80a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c7fff640 1 --2- 172.21.15.89:0/591805917 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 0x7ff0c80a7560 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009dc0 tx=0x7ff0b001c850 comp rx=0 tx=0).ready entity=mon.8 client_cookie=cc478a39286c8181 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 1 -- 172.21.15.89:0/591805917 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0b00ac070 con 0x7ff0c80a7180 2024-08-19T10:56:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 10 monclient: dump: 2024-08-19T10:56:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.707 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.708 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 10 monclient: _finish_auth 0 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 1 -- 172.21.15.89:0/591805917 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b00a68b0 con 0x7ff0c80a7180 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 5 monclient: authenticate success, global_id 114326 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 1 -- 172.21.15.89:0/591805917 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0b00a7420 con 0x7ff0c80a7180 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 4 set_mon_vals no callback set 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0c77fe640 10 monclient: dump: 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.710 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 10 monclient: shutdown 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 1 -- 172.21.15.89:0/591805917 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 msgr2=0x7ff0c80a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 1 --2- 172.21.15.89:0/591805917 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 0x7ff0c80a7560 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009dc0 tx=0x7ff0b001c850 comp rx=0 tx=0).stop 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 1 -- 172.21.15.89:0/591805917 shutdown_connections 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 1 --2- 172.21.15.89:0/591805917 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 0x7ff0c80a7560 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 1 --2- 172.21.15.89:0/591805917 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7ff0c80a6860 0x7ff0c80a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 1 -- 172.21.15.89:0/591805917 >> 172.21.15.89:0/591805917 conn(0x7ff0c8074b40 msgr2=0x7ff0c80abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:04.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 1 -- 172.21.15.89:0/591805917 shutdown_connections 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.701+0000 7ff0cf121640 1 -- 172.21.15.89:0/591805917 wait complete. 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient: build_initial_monmap 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient: monmap: 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:04.712 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 1 Processor -- start 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 1 -- start start 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient: init 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient: _add_conns ranks=[0,8,1,3,4,6,2,5,7] 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c80a6860 0x7ff0c8138010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient(hunting): picked mon.noname-a con 0x7ff0c80a6860 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:56:04.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 0x7ff0c8138550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient(hunting): picked mon.noname-e con 0x7ff0c80a7180 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0c8139620 0x7ff0c813d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient(hunting): picked mon.noname-f con 0x7ff0c8139620 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c8144430 con 0x7ff0c80a6860 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c81442b0 con 0x7ff0c8139620 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff0c81445b0 con 0x7ff0c80a7180 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient(hunting): authenticate will time out at 2014.525088s 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c80a6860 0x7ff0c8138010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c7fff640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 0x7ff0c8138550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c80a6860 0x7ff0c8138010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33926/0 (socket says 172.21.15.89:33926) 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 1 -- 172.21.15.89:0/4190810649 learned_addr learned my addr 172.21.15.89:0/4190810649 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 10 monclient(hunting): get_auth_request con 0x7ff0c80a6860 auth_method 0 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c7fff640 10 monclient(hunting): get_auth_request con 0x7ff0c80a7180 auth_method 0 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 10 monclient(hunting): handle_auth_done global_id 7767 payload 824 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 1 -- 172.21.15.89:0/4190810649 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 msgr2=0x7ff0c8138550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 0x7ff0c8138550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 1 -- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0c8139620 msgr2=0x7ff0c813d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0c8139620 0x7ff0c813d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 10 monclient: _finish_hunting 0 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 1 monclient: found mon.noname-a 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 1 -- 172.21.15.89:0/4190810649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c8122690 con 0x7ff0c80a6860 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c7fff640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff0c80a7180 0x7ff0c8138550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:04.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 1 -- 172.21.15.89:0/4190810649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c80a6860 msgr2=0x7ff0c8138010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0ccebd640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c80a6860 0x7ff0c8138010 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000ebe0 tx=0x7ff0c000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0c0017070 con 0x7ff0c80a6860 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c57fa640 10 monclient: dump: 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.717 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c57fa640 10 monclient: _finish_auth 0 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c000ce40 con 0x7ff0c80a6860 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 5 monclient: authenticate success, global_id 7767 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient: _renew_subs 2024-08-19T10:56:04.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.705+0000 7ff0cf121640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0cf121640 1 -- 172.21.15.89:0/4190810649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c8122970 con 0x7ff0c80a6860 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0c0013d50 con 0x7ff0c80a6860 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c57fa640 10 monclient: dump: 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.719 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0cf121640 10 monclient: _renew_subs 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c6ffd640 4 set_mon_vals no callback set 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0cf121640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:04.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0cf121640 1 -- 172.21.15.89:0/4190810649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c80ab250 con 0x7ff0c80a6860 2024-08-19T10:56:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0cf121640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0cf121640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813da00 0x7ff0c813dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0cf121640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0cf121640 1 -- 172.21.15.89:0/4190810649 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff0c80a6c40 con 0x7ff0c813da00 2024-08-19T10:56:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff0c001a070 con 0x7ff0c80a6860 2024-08-19T10:56:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c7fff640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813da00 0x7ff0c813dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c7fff640 10 monclient: get_auth_request con 0x7ff0c813da00 auth_method 0 2024-08-19T10:56:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c57fa640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff0a803be00 0x7ff0a803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7ff0c000f040 con 0x7ff0c80a6860 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0cd6be640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff0a803be00 0x7ff0a803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0cd6be640 10 monclient: get_auth_request con 0x7ff0a803be00 auth_method 0 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_done global_id 84407 payload 824 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.709+0000 7ff0c7fff640 1 -- 172.21.15.89:0/4190810649 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0c8064590 con 0x7ff0c813da00 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c7fff640 1 -- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813da00 msgr2=0x7ff0c813dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c7fff640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813da00 0x7ff0c813dde0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80a9a20 tx=0x7ff0b00195f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0cd6be640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff0a803be00 0x7ff0a803e2c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8138e70 tx=0x7ff0b8006be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0b00ac050 con 0x7ff0c813da00 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c57fa640 10 monclient: dump: 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.726 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff0b0004600 con 0x7ff0c813da00 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813da00 msgr2=0x7ff0c813dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c57fa640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813da00 0x7ff0c813dde0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80a9a20 tx=0x7ff0b00195f0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813da00 msgr2=0x7ff0c813dde0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.713+0000 7ff0c57fa640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813da00 0x7ff0c813dde0 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80a9a20 tx=0x7ff0b00195f0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.717+0000 7ff0cf121640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.717+0000 7ff0cf121640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813e320 0x7ff0c809eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.717+0000 7ff0cf121640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.717+0000 7ff0cf121640 1 -- 172.21.15.89:0/4190810649 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7ff0c813fda0 con 0x7ff0c813e320 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813e320 0x7ff0c809eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 10 monclient: get_auth_request con 0x7ff0c813e320 auth_method 0 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 10 monclient(hunting): handle_auth_done global_id 84416 payload 824 2024-08-19T10:56:04.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 1 -- 172.21.15.89:0/4190810649 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff0c813ef70 con 0x7ff0c813e320 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 1 -- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813e320 msgr2=0x7ff0c809eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c7fff640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813e320 0x7ff0c809eec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009d60 tx=0x7ff0b00a9000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff0b00b1ce0 con 0x7ff0c813e320 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.721+0000 7ff0c57fa640 10 monclient: dump: 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:04.729 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 <== mon.5 v2:172.21.15.190:3302/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9413 (secure 0 0 0) 0x7ff0b00a6440 con 0x7ff0c813e320 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0c57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0c57fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813e320 msgr2=0x7ff0c809eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0c57fa640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813e320 0x7ff0c809eec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009d60 tx=0x7ff0b00a9000 comp rx=0 tx=0).stop 2024-08-19T10:56:04.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0c57fa640 1 -- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813e320 msgr2=0x7ff0c809eec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-08-19T10:56:04.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0c57fa640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7ff0c813e320 0x7ff0c809eec0 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009d60 tx=0x7ff0b00a9000 comp rx=0 tx=0).stop 2024-08-19T10:56:04.731 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:04.731 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:56:04.731 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:56:04.731 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "peon", 2024-08-19T10:56:04.731 INFO:teuthology.orchestra.run.smithi089.stdout: "election_epoch": 146, 2024-08-19T10:56:04.731 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 173020, 2024-08-19T10:56:04.731 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum": [ 2024-08-19T10:56:04.731 INFO:teuthology.orchestra.run.smithi089.stdout: 0, 2024-08-19T10:56:04.731 INFO:teuthology.orchestra.run.smithi089.stdout: 1, 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: 3, 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: 4, 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: 5, 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: 6, 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: 8 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_age": 5, 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: "required_con": "2450521708007866372", 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: "required_mon": [ 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.732 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_con": "4540701547738038271", 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "quorum_mon": [ 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.733 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: "outside_quorum": [], 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: "extra_probe_peers": [], 2024-08-19T10:56:04.734 INFO:teuthology.orchestra.run.smithi089.stdout: "sync_provider": [], 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "monmap": { 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 1, 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "fsid": "b6e37c79-20e9-449f-b89e-c2165d4f4bed", 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "modified": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "created": "2024-08-19T10:37:22.945188Z", 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release": 19, 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "min_mon_release_name": "squid", 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "election_strategy": 1, 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "disallowed_leaders: ": "", 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false, 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "tiebreaker_mon": "", 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "removed_ranks: ": "", 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "features": { 2024-08-19T10:56:04.735 INFO:teuthology.orchestra.run.smithi089.stdout: "persistent": [ 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "kraken", 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "luminous", 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "mimic", 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap-prune", 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "nautilus", 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus", 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific", 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "elector-pinging", 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy", 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "reef", 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "squid" 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: "optional": [] 2024-08-19T10:56:04.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: "mons": [ 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 0, 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "a", 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3300", 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.737 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789", 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6789/0", 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6789/0", 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 1, 2024-08-19T10:56:04.738 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "f", 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3300", 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789", 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6789/0", 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6789/0", 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 2, 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "b", 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.740 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3301", 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790", 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6790/0", 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6790/0", 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.741 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 3, 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "g", 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3301", 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790", 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6790/0", 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6790/0", 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 4, 2024-08-19T10:56:04.743 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "c", 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3302", 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791", 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.744 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6791/0", 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6791/0", 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 5, 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "h", 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.745 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3302", 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791", 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6791/0", 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6791/0", 2024-08-19T10:56:04.746 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 6, 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "d", 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3303", 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.747 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792", 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6792/0", 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6792/0", 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.748 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 7, 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "i", 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:3303", 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792", 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.749 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.190:6792/0", 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.190:6792/0", 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: "rank": 8, 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "e", 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addrs": { 2024-08-19T10:56:04.750 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:3304", 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793", 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 0 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.751 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6793/0", 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: "public_addr": "172.21.15.89:6793/0", 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: "priority": 0, 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: "weight": 0, 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_location": "{}" 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: "feature_map": { 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: "mon": [ 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:04.752 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: "client": [ 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr": [ 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: "features": "0x3f03cffffffdffff", 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: "release": "squid", 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 1 2024-08-19T10:56:04.753 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-08-19T10:56:04.754 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-08-19T10:56:04.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-08-19T10:56:04.754 INFO:teuthology.orchestra.run.smithi089.stdout: "stretch_mode": false 2024-08-19T10:56:04.754 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:04.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cf121640 1 -- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff0a803be00 msgr2=0x7ff0a803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cf121640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff0a803be00 0x7ff0a803e2c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8138e70 tx=0x7ff0b8006be0 comp rx=0 tx=0).stop 2024-08-19T10:56:04.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cf121640 10 monclient: shutdown 2024-08-19T10:56:04.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cf121640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cf121640 1 -- 172.21.15.89:0/4190810649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c80a6860 msgr2=0x7ff0c8138010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cf121640 1 --2- 172.21.15.89:0/4190810649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0c80a6860 0x7ff0c8138010 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000ebe0 tx=0x7ff0c000c520 comp rx=0 tx=0).stop 2024-08-19T10:56:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cd6be640 1 -- 172.21.15.89:0/4190810649 reap_dead start 2024-08-19T10:56:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cd6be640 1 -- 172.21.15.89:0/4190810649 reap_dead start 2024-08-19T10:56:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cf121640 1 -- 172.21.15.89:0/4190810649 shutdown_connections 2024-08-19T10:56:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cf121640 1 -- 172.21.15.89:0/4190810649 >> 172.21.15.89:0/4190810649 conn(0x7ff0c8074b40 msgr2=0x7ff0c80a2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cf121640 1 -- 172.21.15.89:0/4190810649 shutdown_connections 2024-08-19T10:56:04.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:04.725+0000 7ff0cf121640 1 -- 172.21.15.89:0/4190810649 wait complete. 2024-08-19T10:56:04.756 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-08-19T10:56:04.757 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-08-19T10:56:13.507 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=110 2024-08-19T10:56:13.507 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 110 -ne 110 ]] 2024-08-19T10:56:13.507 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:56:13.507 INFO:tasks.workunit.client.0.smithi190.stderr:+ export CEPH_ARGS= 2024-08-19T10:56:13.507 INFO:tasks.workunit.client.0.smithi190.stderr:+ CEPH_ARGS= 2024-08-19T10:56:13.507 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph auth del client.bar 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient: get_monmap_and_config 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient: build_initial_monmap 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient: monmap: 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:56:13.628682+0000 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:56:13.628682+0000 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:13.619 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:13.620 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:13.620 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 1 Processor -- start 2024-08-19T10:56:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 1 -- start start 2024-08-19T10:56:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient: init 2024-08-19T10:56:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient: _add_conns ranks=[10,7,6,3,4,5,9,1,0,8,2] 2024-08-19T10:56:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7080a4880 0x7fd7080a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:13.623 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient(hunting): picked mon.noname-f con 0x7fd7080a4880 addr v2:172.21.15.190:3300/0 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd708077010 0x7fd7080a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient(hunting): picked mon.noname-i con 0x7fd708077010 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd708077440 0x7fd708139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient(hunting): picked mon.noname-d con 0x7fd708077440 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd70809e400 con 0x7fd7080a4880 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd70809ddb0 con 0x7fd708077440 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7fd7080a56e0 con 0x7fd708077010 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70f7ab640 10 monclient(hunting): authenticate will time out at 2019.198644s 2024-08-19T10:56:13.624 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70cd46640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd708077010 msgr2=0x7fd7080a51a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70cd46640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd708077010 0x7fd7080a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7080a4880 0x7fd7080a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd7080a4880 0x7fd7080a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46994/0 (socket says 172.21.15.190:46994) 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 1 -- 172.21.15.190:0/2243326695 learned_addr learned my addr 172.21.15.190:0/2243326695 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 10 monclient(hunting): get_auth_request con 0x7fd7080a4880 auth_method 0 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 10 monclient(hunting): handle_auth_done global_id 45442 payload 274 2024-08-19T10:56:13.625 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 1 -- 172.21.15.190:0/2243326695 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd708077010 msgr2=0x7fd7080a51a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 1 --2- 172.21.15.190:0/2243326695 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd708077010 0x7fd7080a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 1 -- 172.21.15.190:0/2243326695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd708077440 msgr2=0x7fd708139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 1 --2- 172.21.15.190:0/2243326695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd708077440 0x7fd708139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 10 monclient: _finish_hunting 0 2024-08-19T10:56:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 1 monclient: found mon.noname-f 2024-08-19T10:56:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:13.626 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.625+0000 7fd70d547640 1 -- 172.21.15.190:0/2243326695 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70809dad0 con 0x7fd7080a4880 2024-08-19T10:56:13.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd70d547640 1 -- 172.21.15.190:0/2243326695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7080a4880 msgr2=0x7fd7080a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:13.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd70d547640 1 --2- 172.21.15.190:0/2243326695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7080a4880 0x7fd7080a4c60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd704009e50 tx=0x7fd70401c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=998419c0f95f9dcd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:13.630 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 1 -- 172.21.15.190:0/2243326695 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7040ac020 con 0x7fd7080a4880 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 10 monclient: dump: 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:13.631 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 10 monclient: _finish_auth 0 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 10 monclient: _check_auth_tickets 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 1 -- 172.21.15.190:0/2243326695 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7040091f0 con 0x7fd7080a4880 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 1 -- 172.21.15.190:0/2243326695 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7040a6d70 con 0x7fd7080a4880 2024-08-19T10:56:13.632 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd70f7ab640 5 monclient: authenticate success, global_id 45442 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd70f7ab640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd70f7ab640 4 set_mon_vals no callback set 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.629+0000 7fd6fffff640 10 monclient: dump: 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:13.633 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient: shutdown 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 -- 172.21.15.190:0/2243326695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7080a4880 msgr2=0x7fd7080a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 --2- 172.21.15.190:0/2243326695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7080a4880 0x7fd7080a4c60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd704009e50 tx=0x7fd70401c620 comp rx=0 tx=0).stop 2024-08-19T10:56:13.634 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 -- 172.21.15.190:0/2243326695 shutdown_connections 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 --2- 172.21.15.190:0/2243326695 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd708077440 0x7fd708139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 --2- 172.21.15.190:0/2243326695 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7fd708077010 0x7fd7080a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 --2- 172.21.15.190:0/2243326695 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7080a4880 0x7fd7080a4c60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 -- 172.21.15.190:0/2243326695 >> 172.21.15.190:0/2243326695 conn(0x7fd7080a9470 msgr2=0x7fd7080a5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 -- 172.21.15.190:0/2243326695 shutdown_connections 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 -- 172.21.15.190:0/2243326695 wait complete. 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient: build_initial_monmap 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient: monmap: 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:56:13.635 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 Processor -- start 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 -- start start 2024-08-19T10:56:13.636 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient: init 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient: _add_conns ranks=[0,2,6,1,4,5,8,3,7] 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd708077010 0x7fd7081d1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient(hunting): picked mon.noname-a con 0x7fd708077010 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd708077440 0x7fd708135a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient(hunting): picked mon.noname-b con 0x7fd708077440 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7080a4880 0x7fd70813dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient(hunting): picked mon.noname-d con 0x7fd7080a4880 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd708142ab0 con 0x7fd708077010 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd708142930 con 0x7fd708077440 2024-08-19T10:56:13.637 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fd708142c30 con 0x7fd7080a4880 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70f7ab640 10 monclient(hunting): authenticate will time out at 2019.205845s 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70cd46640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd708077440 msgr2=0x7fd708135a00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70cd46640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd708077440 0x7fd708135a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd708077010 0x7fd7081d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70dd48640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7080a4880 0x7fd70813dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70dd48640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7080a4880 0x7fd70813dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:51326/0 (socket says 172.21.15.190:51326) 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70dd48640 1 -- 172.21.15.190:0/4026439030 learned_addr learned my addr 172.21.15.190:0/4026439030 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 10 monclient(hunting): get_auth_request con 0x7fd708077010 auth_method 0 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:13.638 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70dd48640 10 monclient(hunting): get_auth_request con 0x7fd7080a4880 auth_method 0 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70dd48640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70dd48640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70dd48640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70dd48640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70dd48640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70dd48640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:13.639 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 10 monclient(hunting): handle_auth_done global_id 7776 payload 824 2024-08-19T10:56:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 1 -- 172.21.15.190:0/4026439030 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7080a4880 msgr2=0x7fd70813dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7080a4880 0x7fd70813dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 1 -- 172.21.15.190:0/4026439030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd708077440 msgr2=0x7fd708135a00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd708077440 0x7fd708135a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 10 monclient: _finish_hunting 0 2024-08-19T10:56:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 1 monclient: found mon.noname-a 2024-08-19T10:56:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:56:13.640 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.633+0000 7fd70d547640 1 -- 172.21.15.190:0/4026439030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7080a7f30 con 0x7fd708077010 2024-08-19T10:56:13.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70d547640 1 -- 172.21.15.190:0/4026439030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd708077010 msgr2=0x7fd7081d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:13.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70d547640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd708077010 0x7fd7081d1060 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd7040ad000 tx=0x7fd7040193c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:13.642 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 1 -- 172.21.15.190:0/4026439030 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7040ac040 con 0x7fd708077010 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 10 monclient: dump: 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:13.643 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 10 monclient: _finish_auth 0 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 10 monclient: _check_auth_tickets 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 1 -- 172.21.15.190:0/4026439030 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7040a6490 con 0x7fd708077010 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 1 -- 172.21.15.190:0/4026439030 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd7040a6ab0 con 0x7fd708077010 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70f7ab640 5 monclient: authenticate success, global_id 7776 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70f7ab640 10 monclient: _renew_subs 2024-08-19T10:56:13.644 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70f7ab640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6feffd640 4 set_mon_vals no callback set 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70f7ab640 1 -- 172.21.15.190:0/4026439030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd708120de0 con 0x7fd708077010 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 10 monclient: dump: 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:13.645 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70f7ab640 10 monclient: _renew_subs 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70f7ab640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70f7ab640 1 -- 172.21.15.190:0/4026439030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70813f650 con 0x7fd708077010 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70f7ab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70f7ab640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:13.646 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70f7ab640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:13.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd70f7ab640 1 -- 172.21.15.190:0/4026439030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd708139660 con 0x7fd708077010 2024-08-19T10:56:13.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.637+0000 7fd6fdffb640 1 -- 172.21.15.190:0/4026439030 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd7040b9020 con 0x7fd708077010 2024-08-19T10:56:13.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.641+0000 7fd6fdffb640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6ec03be30 0x7fd6ec03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:13.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.641+0000 7fd6fdffb640 1 -- 172.21.15.190:0/4026439030 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fd7040e9880 con 0x7fd708077010 2024-08-19T10:56:13.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.641+0000 7fd70cd46640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6ec03be30 0x7fd6ec03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:13.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.641+0000 7fd70cd46640 10 monclient: get_auth_request con 0x7fd6ec03be30 auth_method 0 2024-08-19T10:56:13.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.641+0000 7fd70cd46640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6ec03be30 0x7fd6ec03e2f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd70813f3a0 tx=0x7fd6f8006d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:13.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.641+0000 7fd6fdffb640 1 -- 172.21.15.190:0/4026439030 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:56:13.647 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.653+0000 7fd6fdffb640 1 -- 172.21.15.190:0/4026439030 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd7040c1020 con 0x7fd708077010 2024-08-19T10:56:13.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.653+0000 7fd6fdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:13.648 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.653+0000 7fd6fdffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:13.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.885+0000 7fd70f7ab640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bar"}] 2024-08-19T10:56:13.879 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.885+0000 7fd70f7ab640 10 monclient: _send_command 2 [{"prefix": "auth del", "entity": "client.bar"}] 2024-08-19T10:56:13.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.885+0000 7fd70f7ab640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:13.880 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.885+0000 7fd70f7ab640 1 -- 172.21.15.190:0/4026439030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bar"} v 0) -- 0x7fd708051470 con 0x7fd708077010 2024-08-19T10:56:13.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.889+0000 7fd6fdffb640 1 -- 172.21.15.190:0/4026439030 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bar"}]=0 v24) ==== 80+0+0 (secure 0 0 0) 0x7fd7080a7f30 con 0x7fd708077010 2024-08-19T10:56:13.885 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.889+0000 7fd6fdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth del", "entity": "client.bar"}] 2024-08-19T10:56:13.886 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:13.889+0000 7fd6fdffb640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:14.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.057+0000 7fd70f7ab640 10 start_mon_command cmd=[{"prefix": "auth del", "entity": "client.bar"}] 2024-08-19T10:56:14.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.057+0000 7fd70f7ab640 10 monclient: _send_command 3 [{"prefix": "auth del", "entity": "client.bar"}] 2024-08-19T10:56:14.051 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.057+0000 7fd70f7ab640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:14.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.057+0000 7fd70f7ab640 1 -- 172.21.15.190:0/4026439030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bar"} v 0) -- 0x7fd70811fba0 con 0x7fd708077010 2024-08-19T10:56:14.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.057+0000 7fd6fdffb640 1 -- 172.21.15.190:0/4026439030 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bar"}]=0 v24) ==== 80+0+0 (secure 0 0 0) 0x7fd704016620 con 0x7fd708077010 2024-08-19T10:56:14.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.057+0000 7fd6fdffb640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth del", "entity": "client.bar"}] 2024-08-19T10:56:14.052 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.057+0000 7fd6fdffb640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:56:14.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 -- 172.21.15.190:0/4026439030 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6ec03be30 msgr2=0x7fd6ec03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:14.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6ec03be30 0x7fd6ec03e2f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd70813f3a0 tx=0x7fd6f8006d10 comp rx=0 tx=0).stop 2024-08-19T10:56:14.054 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 10 monclient: shutdown 2024-08-19T10:56:14.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:14.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 -- 172.21.15.190:0/4026439030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd708077010 msgr2=0x7fd7081d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:14.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd708077010 0x7fd7081d1060 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd7040ad000 tx=0x7fd7040193c0 comp rx=0 tx=0).stop 2024-08-19T10:56:14.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 -- 172.21.15.190:0/4026439030 shutdown_connections 2024-08-19T10:56:14.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd6ec03be30 0x7fd6ec03e2f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:14.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fd7080a4880 0x7fd70813dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:14.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd708077440 0x7fd708135a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:14.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 --2- 172.21.15.190:0/4026439030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd708077010 0x7fd7081d1060 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:14.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 -- 172.21.15.190:0/4026439030 >> 172.21.15.190:0/4026439030 conn(0x7fd7080a9470 msgr2=0x7fd7080a6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:14.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 -- 172.21.15.190:0/4026439030 shutdown_connections 2024-08-19T10:56:14.055 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:14.061+0000 7fd70f7ab640 1 -- 172.21.15.190:0/4026439030 wait complete. 2024-08-19T10:56:14.070 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 13 2024-08-19T10:56:14.070 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status' 2024-08-19T10:56:14.070 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=13 2024-08-19T10:56:14.071 INFO:tasks.workunit.client.0.smithi190.stdout:ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-08-19T10:56:14.071 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-08-19T10:56:14.071 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph -k /tmp/cephtest-mon-caps-madness.bar.keyring --user bar quorum_status 2024-08-19T10:56:20.189 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=13 2024-08-19T10:56:20.190 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 13 -ne 13 ]] 2024-08-19T10:56:20.190 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:56:20.190 INFO:tasks.workunit.client.0.smithi190.stderr:+ rm /tmp/cephtest-mon-caps-madness.bazar.keyring /tmp/cephtest-mon-caps-madness.foo.keyring /tmp/cephtest-mon-caps-madness.bar.keyring 2024-08-19T10:56:20.191 INFO:tasks.workunit.client.0.smithi190.stderr:+ cat 2024-08-19T10:56:20.191 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph auth import -i - 2024-08-19T10:56:20.305 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.309+0000 7ff97e136640 10 monclient: get_monmap_and_config 2024-08-19T10:56:20.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.309+0000 7ff97e136640 10 monclient: build_initial_monmap 2024-08-19T10:56:20.306 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.309+0000 7ff97e136640 10 monclient: monmap: 2024-08-19T10:56:20.306 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:56:20.306 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:20.306 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:56:20.315467+0000 2024-08-19T10:56:20.306 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:56:20.315467+0000 2024-08-19T10:56:20.306 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:20.306 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:20.307 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:20.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 1 Processor -- start 2024-08-19T10:56:20.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 1 -- start start 2024-08-19T10:56:20.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 10 monclient: init 2024-08-19T10:56:20.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:20.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 10 monclient: _add_conns ranks=[6,10,2,9,1,0,3,7,5,4,8] 2024-08-19T10:56:20.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff9780a50d0 0x7ff9780a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:20.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 10 monclient(hunting): picked mon.noname-d con 0x7ff9780a50d0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:20.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9780a54b0 0x7ff9780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:20.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 10 monclient(hunting): picked mon.noname-f con 0x7ff9780a54b0 addr v2:172.21.15.190:3300/0 2024-08-19T10:56:20.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff9780a6a60 0x7ff9780a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:20.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 10 monclient(hunting): picked mon.noname-b con 0x7ff9780a6a60 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff9780a3ea0 con 0x7ff9780a54b0 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff9780a9680 con 0x7ff9780a6a60 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff9780a9900 con 0x7ff9780a50d0 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff97e136640 10 monclient(hunting): authenticate will time out at 2025.885351s 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff977fff640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff9780a6a60 msgr2=0x7ff9780a6ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff977fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff9780a6a60 0x7ff9780a6ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9780a54b0 0x7ff9780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9780a54b0 0x7ff9780a58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:37084/0 (socket says 172.21.15.190:37084) 2024-08-19T10:56:20.311 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 1 -- 172.21.15.190:0/2259051342 learned_addr learned my addr 172.21.15.190:0/2259051342 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 10 monclient(hunting): get_auth_request con 0x7ff9780a54b0 auth_method 0 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff9777fe640 1 --2- 172.21.15.190:0/2259051342 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff9780a50d0 0x7ff9780a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff9777fe640 10 monclient(hunting): get_auth_request con 0x7ff9780a50d0 auth_method 0 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff9777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff9777fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff9777fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 10 monclient(hunting): handle_auth_done global_id 45469 payload 274 2024-08-19T10:56:20.312 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 1 -- 172.21.15.190:0/2259051342 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff9780a50d0 msgr2=0x7ff9780a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:20.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 1 --2- 172.21.15.190:0/2259051342 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff9780a50d0 0x7ff9780a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:20.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 1 -- 172.21.15.190:0/2259051342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff9780a6a60 msgr2=0x7ff9780a6ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:20.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 1 --2- 172.21.15.190:0/2259051342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff9780a6a60 0x7ff9780a6ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:20.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:56:20.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 1 monclient: found mon.noname-f 2024-08-19T10:56:20.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:20.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:20.313 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.313+0000 7ff976ffd640 1 -- 172.21.15.190:0/2259051342 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9780a7c00 con 0x7ff9780a54b0 2024-08-19T10:56:20.316 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff976ffd640 1 -- 172.21.15.190:0/2259051342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a54b0 msgr2=0x7ff9780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff976ffd640 1 --2- 172.21.15.190:0/2259051342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a54b0 0x7ff9780a58f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009870 tx=0x7ff96c01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bbcb1dff3925f09 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 1 -- 172.21.15.190:0/2259051342 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff96c0b0070 con 0x7ff9780a54b0 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 10 monclient: dump: 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:20.317 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 10 monclient: _finish_auth 0 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 10 monclient: _check_auth_tickets 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:20.318 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 1 -- 172.21.15.190:0/2259051342 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96c0aa810 con 0x7ff9780a54b0 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 1 -- 172.21.15.190:0/2259051342 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff96c0aae30 con 0x7ff9780a54b0 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff9767fc640 10 monclient: dump: 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:20.319 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff97e136640 5 monclient: authenticate success, global_id 45469 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff97e136640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff97e136640 4 set_mon_vals no callback set 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.317+0000 7ff97e136640 10 monclient: shutdown 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:20.320 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 -- 172.21.15.190:0/2259051342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a54b0 msgr2=0x7ff9780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 --2- 172.21.15.190:0/2259051342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a54b0 0x7ff9780a58f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009870 tx=0x7ff96c01c440 comp rx=0 tx=0).stop 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 -- 172.21.15.190:0/2259051342 shutdown_connections 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 --2- 172.21.15.190:0/2259051342 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff9780a6a60 0x7ff9780a6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 --2- 172.21.15.190:0/2259051342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a54b0 0x7ff9780a58f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 --2- 172.21.15.190:0/2259051342 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff9780a50d0 0x7ff9780a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 -- 172.21.15.190:0/2259051342 >> 172.21.15.190:0/2259051342 conn(0x7ff9780ac6c0 msgr2=0x7ff9780a8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 -- 172.21.15.190:0/2259051342 shutdown_connections 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 -- 172.21.15.190:0/2259051342 wait complete. 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient: build_initial_monmap 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient: monmap: 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:20.321 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:20.322 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 Processor -- start 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 -- start start 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient: init 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient: _add_conns ranks=[3,8,1,4,7,6,2,5,0] 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9780a50d0 0x7ff97813b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient(hunting): picked mon.noname-g con 0x7ff9780a50d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9780a54b0 0x7ff97813b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient(hunting): picked mon.noname-e con 0x7ff9780a54b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a6a60 0x7ff978123450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient(hunting): picked mon.noname-f con 0x7ff9780a6a60 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff978143080 con 0x7ff9780a6a60 2024-08-19T10:56:20.323 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff9780743a0 con 0x7ff9780a50d0 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7ff978143200 con 0x7ff9780a54b0 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff97e136640 10 monclient(hunting): authenticate will time out at 2025.893692s 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff9777fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9780a50d0 0x7ff97813b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a6a60 0x7ff978123450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff9777fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9780a50d0 0x7ff97813b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:39352/0 (socket says 172.21.15.190:39352) 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff9777fe640 1 -- 172.21.15.190:0/3325502312 learned_addr learned my addr 172.21.15.190:0/3325502312 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 10 monclient(hunting): get_auth_request con 0x7ff9780a6a60 auth_method 0 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:20.324 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff9777fe640 10 monclient(hunting): get_auth_request con 0x7ff9780a50d0 auth_method 0 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff9777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff9777fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff9777fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff9777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff9777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff9777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff976ffd640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9780a54b0 0x7ff97813b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 10 monclient(hunting): handle_auth_done global_id 45478 payload 824 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 1 -- 172.21.15.190:0/3325502312 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9780a54b0 msgr2=0x7ff97813b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:20.325 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9780a54b0 0x7ff97813b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:20.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 1 -- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9780a50d0 msgr2=0x7ff97813b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:20.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9780a50d0 0x7ff97813b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:20.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 10 monclient: _finish_hunting 0 2024-08-19T10:56:20.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 1 monclient: found mon.noname-f 2024-08-19T10:56:20.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:20.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:20.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff977fff640 1 -- 172.21.15.190:0/3325502312 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff978127180 con 0x7ff9780a6a60 2024-08-19T10:56:20.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff976ffd640 10 monclient: get_auth_request con 0x7ff9780a54b0 auth_method 0 2024-08-19T10:56:20.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff976ffd640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9780a54b0 0x7ff97813b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:20.326 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.321+0000 7ff9777fe640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9780a50d0 0x7ff97813b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:20.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff977fff640 1 -- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a6a60 msgr2=0x7ff978123450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:20.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff977fff640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a6a60 0x7ff978123450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff97000bdc0 tx=0x7ff970009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:20.328 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 1 -- 172.21.15.190:0/3325502312 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff970006960 con 0x7ff9780a6a60 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 10 monclient: dump: 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:20.329 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 10 monclient: _finish_auth 0 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 1 -- 172.21.15.190:0/3325502312 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff970006b60 con 0x7ff9780a6a60 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:20.330 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 1 -- 172.21.15.190:0/3325502312 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff9700158b0 con 0x7ff9780a6a60 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff97e136640 5 monclient: authenticate success, global_id 45478 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff97e136640 10 monclient: _renew_subs 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff97e136640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff97e136640 1 -- 172.21.15.190:0/3325502312 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff978124e20 con 0x7ff9780a6a60 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 10 monclient: dump: 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:20.331 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff975ffb640 4 set_mon_vals no callback set 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff97e136640 10 monclient: _renew_subs 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff97e136640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:20.332 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff97e136640 1 -- 172.21.15.190:0/3325502312 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97813c580 con 0x7ff9780a6a60 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff97e136640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff97e136640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff97e136640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff97e136640 1 -- 172.21.15.190:0/3325502312 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9780a58f0 con 0x7ff9780a6a60 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 1 -- 172.21.15.190:0/3325502312 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff970015440 con 0x7ff9780a6a60 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff96403be00 0x7ff96403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff95ffff640 1 -- 172.21.15.190:0/3325502312 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7ff970050460 con 0x7ff9780a6a60 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff9777fe640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff96403be00 0x7ff96403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.325+0000 7ff9777fe640 10 monclient: get_auth_request con 0x7ff96403be00 auth_method 0 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.329+0000 7ff9777fe640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff96403be00 0x7ff96403e2c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff968007f00 tx=0x7ff968008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.329+0000 7ff95ffff640 1 -- 172.21.15.190:0/3325502312 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff970012330 con 0x7ff9780a6a60 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.329+0000 7ff95ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:20.333 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.329+0000 7ff95ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:20.546 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.553+0000 7ff97e136640 10 start_mon_command cmd=[{"prefix": "auth import"}] 2024-08-19T10:56:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.553+0000 7ff97e136640 10 monclient: _send_command 2 [{"prefix": "auth import"}] 2024-08-19T10:56:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.553+0000 7ff97e136640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:20.547 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.553+0000 7ff97e136640 1 -- 172.21.15.190:0/3325502312 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "auth import"} v 0) -- 0x7ff978051470 con 0x7ff9780a6a60 2024-08-19T10:56:20.548 INFO:tasks.ceph.mon.a.smithi089.stderr:2024-08-19T10:56:20.549+0000 7f16cc065640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 auth entities have invalid capabilities (AUTH_BAD_CAPS) 2024-08-19T10:56:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.561+0000 7ff95ffff640 1 -- 172.21.15.190:0/3325502312 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "auth import"}]=0 v25) ==== 59+0+0 (secure 0 0 0) 0x7ff970006780 con 0x7ff9780a6a60 2024-08-19T10:56:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.561+0000 7ff95ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth import"}] 2024-08-19T10:56:20.554 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.561+0000 7ff95ffff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:20.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.721+0000 7ff97e136640 10 start_mon_command cmd=[{"prefix": "auth import"}] 2024-08-19T10:56:20.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.721+0000 7ff97e136640 10 monclient: _send_command 3 [{"prefix": "auth import"}] 2024-08-19T10:56:20.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.721+0000 7ff97e136640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:20.717 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:20.721+0000 7ff97e136640 1 -- 172.21.15.190:0/3325502312 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "auth import"} v 0) -- 0x7ff97806cb60 con 0x7ff9780a6a60 2024-08-19T10:56:21.429 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.433+0000 7ff95ffff640 1 -- 172.21.15.190:0/3325502312 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "auth import"}]=0 v26) ==== 59+0+0 (secure 0 0 0) 0x7ff970013d90 con 0x7ff9780a6a60 2024-08-19T10:56:21.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.433+0000 7ff95ffff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth import"}] 2024-08-19T10:56:21.430 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.433+0000 7ff95ffff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:56:21.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 -- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff96403be00 msgr2=0x7ff96403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:21.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff96403be00 0x7ff96403e2c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff968007f00 tx=0x7ff968008040 comp rx=0 tx=0).stop 2024-08-19T10:56:21.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 10 monclient: shutdown 2024-08-19T10:56:21.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:21.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 -- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a6a60 msgr2=0x7ff978123450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:21.434 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a6a60 0x7ff978123450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff97000bdc0 tx=0x7ff970009eb0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 -- 172.21.15.190:0/3325502312 shutdown_connections 2024-08-19T10:56:21.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9780a6a60 0x7ff978123450 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff96403be00 0x7ff96403e2c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7ff9780a54b0 0x7ff97813b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 --2- 172.21.15.190:0/3325502312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9780a50d0 0x7ff97813b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 -- 172.21.15.190:0/3325502312 >> 172.21.15.190:0/3325502312 conn(0x7ff9780ac6c0 msgr2=0x7ff9780ab270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:21.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 -- 172.21.15.190:0/3325502312 shutdown_connections 2024-08-19T10:56:21.435 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.441+0000 7ff97e136640 1 -- 172.21.15.190:0/3325502312 wait complete. 2024-08-19T10:56:21.456 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph health 2024-08-19T10:56:21.456 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep AUTH_BAD_CAP 2024-08-19T10:56:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient: get_monmap_and_config 2024-08-19T10:56:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient: build_initial_monmap 2024-08-19T10:56:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient: monmap: 2024-08-19T10:56:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:56:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:56:21.575746+0000 2024-08-19T10:56:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:56:21.575746+0000 2024-08-19T10:56:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:21.566 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:21.567 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 1 Processor -- start 2024-08-19T10:56:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 1 -- start start 2024-08-19T10:56:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient: init 2024-08-19T10:56:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient: _add_conns ranks=[8,5,7,3,2,0,1,10,4,9,6] 2024-08-19T10:56:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25700a3b30 0x7f25700a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient(hunting): picked mon.noname-e con 0x7f25700a3b30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f25700a4450 0x7f25701375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:21.570 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient(hunting): picked mon.noname-h con 0x7f25700a4450 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 0x7f2570137f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient(hunting): picked mon.noname-i con 0x7f2570137af0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2570074300 con 0x7f25700a4450 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f25700a0e30 con 0x7f2570137af0 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f25700779f0 con 0x7f25700a3b30 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f25770f5640 10 monclient(hunting): authenticate will time out at 2027.145833s 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f2575692640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 msgr2=0x7f2570137f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:21.571 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f2575692640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 0x7f2570137f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f25700a4450 0x7f25701375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f25700a4450 0x7f25701375b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:42556/0 (socket says 172.21.15.190:42556) 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 1 -- 172.21.15.190:0/3258078115 learned_addr learned my addr 172.21.15.190:0/3258078115 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 10 monclient(hunting): get_auth_request con 0x7f25700a4450 auth_method 0 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f2574e91640 1 --2- 172.21.15.190:0/3258078115 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25700a3b30 0x7f25700a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f2574e91640 10 monclient(hunting): get_auth_request con 0x7f25700a3b30 auth_method 0 2024-08-19T10:56:21.572 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f2574e91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f2574e91640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f2574e91640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 10 monclient(hunting): handle_auth_done global_id 84434 payload 274 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 1 -- 172.21.15.190:0/3258078115 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25700a3b30 msgr2=0x7f25700a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 1 --2- 172.21.15.190:0/3258078115 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25700a3b30 0x7f25700a3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 1 -- 172.21.15.190:0/3258078115 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 msgr2=0x7f2570137f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 1 --2- 172.21.15.190:0/3258078115 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 0x7f2570137f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 10 monclient: _finish_hunting 0 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 1 monclient: found mon.noname-h 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:56:21.573 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.573+0000 7f256ffff640 1 -- 172.21.15.190:0/3258078115 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25700a0590 con 0x7f25700a4450 2024-08-19T10:56:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256ffff640 1 -- 172.21.15.190:0/3258078115 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f25700a4450 msgr2=0x7f25701375b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256ffff640 1 --2- 172.21.15.190:0/3258078115 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f25700a4450 0x7f25701375b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2558009e80 tx=0x7f255801c930 comp rx=0 tx=0).ready entity=mon.5 client_cookie=dd966f488871b83d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 1 -- 172.21.15.190:0/3258078115 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f25580090d0 con 0x7f25700a4450 2024-08-19T10:56:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:21.577 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 10 monclient: dump: 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:21.578 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 10 monclient: _finish_auth 0 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 10 monclient: _check_auth_tickets 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 1 -- 172.21.15.190:0/3258078115 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25580a5980 con 0x7f25700a4450 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f25770f5640 5 monclient: authenticate success, global_id 84434 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f25770f5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:21.579 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f25770f5640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 1 -- 172.21.15.190:0/3258078115 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f25580ad3f0 con 0x7f25700a4450 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f25770f5640 4 set_mon_vals no callback set 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.577+0000 7f256f7fe640 10 monclient: dump: 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:21.580 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient: shutdown 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 -- 172.21.15.190:0/3258078115 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f25700a4450 msgr2=0x7f25701375b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 --2- 172.21.15.190:0/3258078115 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f25700a4450 0x7f25701375b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2558009e80 tx=0x7f255801c930 comp rx=0 tx=0).stop 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 -- 172.21.15.190:0/3258078115 shutdown_connections 2024-08-19T10:56:21.581 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 --2- 172.21.15.190:0/3258078115 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 0x7f2570137f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 --2- 172.21.15.190:0/3258078115 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f25700a4450 0x7f25701375b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 --2- 172.21.15.190:0/3258078115 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25700a3b30 0x7f25700a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 -- 172.21.15.190:0/3258078115 >> 172.21.15.190:0/3258078115 conn(0x7f25700a91e0 msgr2=0x7f25700a95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 -- 172.21.15.190:0/3258078115 shutdown_connections 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 -- 172.21.15.190:0/3258078115 wait complete. 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient: build_initial_monmap 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient: monmap: 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:56:21.582 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 Processor -- start 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 -- start start 2024-08-19T10:56:21.583 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient: init 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient: _add_conns ranks=[1,8,7,4,2,6,3,0,5] 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25700a3b30 0x7f257013d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient(hunting): picked mon.noname-f con 0x7f25700a3b30 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25700a4450 0x7f25701cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient(hunting): picked mon.noname-e con 0x7f25700a4450 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 0x7f25701cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient(hunting): picked mon.noname-i con 0x7f2570137af0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2570076f40 con 0x7f25700a3b30 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:21.584 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f25700a31a0 con 0x7f2570137af0 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f25701c6140 con 0x7f25700a4450 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f25770f5640 10 monclient(hunting): authenticate will time out at 2027.153982s 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2575692640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 msgr2=0x7f25701cfc60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2575692640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 0x7f25701cfc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25700a3b30 0x7f257013d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25700a3b30 0x7f257013d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:37108/0 (socket says 172.21.15.190:37108) 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 1 -- 172.21.15.190:0/1094149317 learned_addr learned my addr 172.21.15.190:0/1094149317 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 10 monclient(hunting): get_auth_request con 0x7f25700a3b30 auth_method 0 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:21.585 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f256ffff640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25700a4450 0x7f25701cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 10 monclient(hunting): handle_auth_done global_id 45487 payload 824 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 1 -- 172.21.15.190:0/1094149317 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25700a4450 msgr2=0x7f25701cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25700a4450 0x7f25701cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 1 -- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 msgr2=0x7f25701cfc60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 0x7f25701cfc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 10 monclient: _finish_hunting 0 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 1 monclient: found mon.noname-f 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:21.586 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:21.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f2574e91640 1 -- 172.21.15.190:0/1094149317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2570121960 con 0x7f25700a3b30 2024-08-19T10:56:21.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f256ffff640 10 monclient: get_auth_request con 0x7f25700a4450 auth_method 0 2024-08-19T10:56:21.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.581+0000 7f256ffff640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25700a4450 0x7f25701cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f2574e91640 1 -- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25700a3b30 msgr2=0x7f257013d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f2574e91640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25700a3b30 0x7f257013d320 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f256800d800 tx=0x7f256800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 1 -- 172.21.15.190:0/1094149317 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f256800bce0 con 0x7f25700a3b30 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 10 monclient: dump: 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:21.589 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 10 monclient: _finish_auth 0 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 10 monclient: _check_auth_tickets 2024-08-19T10:56:21.590 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 1 -- 172.21.15.190:0/1094149317 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2568019070 con 0x7f25700a3b30 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 1 -- 172.21.15.190:0/1094149317 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f25680148a0 con 0x7f25700a3b30 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f25770f5640 5 monclient: authenticate success, global_id 45487 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f25770f5640 10 monclient: _renew_subs 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f25770f5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f25770f5640 1 -- 172.21.15.190:0/1094149317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2570121c40 con 0x7f25700a3b30 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256e7fc640 4 set_mon_vals no callback set 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 10 monclient: dump: 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:21.591 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:21.592 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f25770f5640 10 monclient: _renew_subs 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f25770f5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f25770f5640 1 -- 172.21.15.190:0/1094149317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25700a86b0 con 0x7f25700a3b30 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f25770f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f25770f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f25770f5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f25770f5640 1 -- 172.21.15.190:0/1094149317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257006d850 con 0x7f25700a3b30 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 1 -- 172.21.15.190:0/1094149317 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2568014430 con 0x7f25700a3b30 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f255403be00 0x7f255403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256d7fa640 1 -- 172.21.15.190:0/1094149317 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f256800f040 con 0x7f25700a3b30 2024-08-19T10:56:21.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256ffff640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f255403be00 0x7f255403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:21.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.585+0000 7f256ffff640 10 monclient: get_auth_request con 0x7f255403be00 auth_method 0 2024-08-19T10:56:21.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.589+0000 7f256ffff640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f255403be00 0x7f255403e2c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2558009e50 tx=0x7f25580a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:21.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.589+0000 7f256d7fa640 1 -- 172.21.15.190:0/1094149317 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f2568011e20 con 0x7f25700a3b30 2024-08-19T10:56:21.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.589+0000 7f256d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:21.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.589+0000 7f256d7fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:21.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.821+0000 7f25770f5640 10 start_mon_command cmd=[{"prefix": "health"}] 2024-08-19T10:56:21.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.821+0000 7f25770f5640 10 monclient: _send_command 2 [{"prefix": "health"}] 2024-08-19T10:56:21.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.821+0000 7f25770f5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:21.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.821+0000 7f25770f5640 1 -- 172.21.15.190:0/1094149317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "health"} v 0) -- 0x7f25701375b0 con 0x7f25700a3b30 2024-08-19T10:56:21.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.821+0000 7f256d7fa640 1 -- 172.21.15.190:0/1094149317 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "health"}]=0 v0) ==== 54+0+136 (secure 0 0 0) 0x7f256801dce0 con 0x7f25700a3b30 2024-08-19T10:56:21.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.821+0000 7f256d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health"}] 2024-08-19T10:56:21.815 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.821+0000 7f256d7fa640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:21.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 10 start_mon_command cmd=[{"prefix": "health"}] 2024-08-19T10:56:21.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 10 monclient: _send_command 3 [{"prefix": "health"}] 2024-08-19T10:56:21.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:21.942 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 -- 172.21.15.190:0/1094149317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "health"} v 0) -- 0x7f2570051470 con 0x7f25700a3b30 2024-08-19T10:56:21.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f256d7fa640 1 -- 172.21.15.190:0/1094149317 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "health"}]=0 v0) ==== 54+0+136 (secure 0 0 0) 0x7f2568014a40 con 0x7f25700a3b30 2024-08-19T10:56:21.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f256d7fa640 10 monclient: handle_mon_command_ack 3 [{"prefix": "health"}] 2024-08-19T10:56:21.943 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f256d7fa640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:56:21.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 -- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f255403be00 msgr2=0x7f255403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:21.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f255403be00 0x7f255403e2c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2558009e50 tx=0x7f25580a7070 comp rx=0 tx=0).stop 2024-08-19T10:56:21.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 10 monclient: shutdown 2024-08-19T10:56:21.945 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:21.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 -- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25700a3b30 msgr2=0x7f257013d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:21.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25700a3b30 0x7f257013d320 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f256800d800 tx=0x7f256800dcc0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 -- 172.21.15.190:0/1094149317 shutdown_connections 2024-08-19T10:56:21.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f2570137af0 0x7f25701cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f255403be00 0x7f255403e2c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25700a4450 0x7f25701cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 --2- 172.21.15.190:0/1094149317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25700a3b30 0x7f257013d320 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:21.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 -- 172.21.15.190:0/1094149317 >> 172.21.15.190:0/1094149317 conn(0x7f25700a91e0 msgr2=0x7f257006fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:21.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 -- 172.21.15.190:0/1094149317 shutdown_connections 2024-08-19T10:56:21.946 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:21.949+0000 7f25770f5640 1 -- 172.21.15.190:0/1094149317 wait complete. 2024-08-19T10:56:21.960 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph health detail 2024-08-19T10:56:21.960 INFO:tasks.workunit.client.0.smithi190.stderr:+ grep client.bad 2024-08-19T10:56:22.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.073+0000 7f6ba4346640 10 monclient: get_monmap_and_config 2024-08-19T10:56:22.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.073+0000 7f6ba4346640 10 monclient: build_initial_monmap 2024-08-19T10:56:22.069 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.073+0000 7f6ba4346640 10 monclient: monmap: 2024-08-19T10:56:22.069 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:56:22.070 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:22.070 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:56:22.078614+0000 2024-08-19T10:56:22.070 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:56:22.078614+0000 2024-08-19T10:56:22.070 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:22.070 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.070 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:22.070 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:22.070 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:22.070 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:22.070 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:22.071 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:22.071 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:22.071 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:22.071 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:22.071 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:22.071 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:22.071 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.073+0000 7f6ba4346640 1 Processor -- start 2024-08-19T10:56:22.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.073+0000 7f6ba4346640 1 -- start start 2024-08-19T10:56:22.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.073+0000 7f6ba4346640 10 monclient: init 2024-08-19T10:56:22.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:22.073 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 10 monclient: _add_conns ranks=[5,2,6,3,7,1,4,8,10,0,9] 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b9c0a3f00 0x7f6b9c0a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 10 monclient(hunting): picked mon.noname-h con 0x7f6b9c0a3f00 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b9c0742c0 0x7f6b9c0a4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 10 monclient(hunting): picked mon.noname-b con 0x7f6b9c0742c0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b9c069680 0x7f6b9c09b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 10 monclient(hunting): picked mon.noname-d con 0x7f6b9c069680 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9c074f60 con 0x7f6b9c0742c0 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f6b9c0a5f50 con 0x7f6b9c0a3f00 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.074 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6b9c0a4d60 con 0x7f6b9c069680 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba4346640 10 monclient(hunting): authenticate will time out at 2027.648483s 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b9c0a3f00 0x7f6b9c0a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b9c0a3f00 0x7f6b9c0a42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.190:42568/0 (socket says 172.21.15.190:42568) 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 1 -- 172.21.15.190:0/3066758062 learned_addr learned my addr 172.21.15.190:0/3066758062 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 10 monclient(hunting): get_auth_request con 0x7f6b9c0a3f00 auth_method 0 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba18e1640 1 -- 172.21.15.190:0/3066758062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b9c0742c0 msgr2=0x7f6b9c0a4820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:22.075 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba18e1640 1 --2- 172.21.15.190:0/3066758062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b9c0742c0 0x7f6b9c0a4820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 10 monclient(hunting): handle_auth_done global_id 84443 payload 274 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba28e3640 1 --2- 172.21.15.190:0/3066758062 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b9c069680 0x7f6b9c09b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 1 -- 172.21.15.190:0/3066758062 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b9c069680 msgr2=0x7f6b9c09b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 1 --2- 172.21.15.190:0/3066758062 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b9c069680 0x7f6b9c09b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 1 -- 172.21.15.190:0/3066758062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b9c0742c0 msgr2=0x7f6b9c0a4820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 1 --2- 172.21.15.190:0/3066758062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b9c0742c0 0x7f6b9c0a4820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 10 monclient: _finish_hunting 0 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 1 monclient: found mon.noname-h 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:56:22.076 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.077+0000 7f6ba20e2640 1 -- 172.21.15.190:0/3066758062 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c09c690 con 0x7f6b9c0a3f00 2024-08-19T10:56:22.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba20e2640 1 -- 172.21.15.190:0/3066758062 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b9c0a3f00 msgr2=0x7f6b9c0a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:22.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba20e2640 1 --2- 172.21.15.190:0/3066758062 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b9c0a3f00 0x7f6b9c0a42e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098d0 tx=0x7f6b9801c400 comp rx=0 tx=0).ready entity=mon.5 client_cookie=57810382d0186d39 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:22.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 1 -- 172.21.15.190:0/3066758062 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b980b0070 con 0x7f6b9c0a3f00 2024-08-19T10:56:22.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:22.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:56:22.081 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 10 monclient: dump: 2024-08-19T10:56:22.081 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:22.081 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:22.081 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.081 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 10 monclient: _finish_auth 0 2024-08-19T10:56:22.082 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 10 monclient: _check_auth_tickets 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 1 -- 172.21.15.190:0/3066758062 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b980a9a60 con 0x7f6b9c0a3f00 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 5 monclient: authenticate success, global_id 84443 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 1 -- 172.21.15.190:0/3066758062 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b980a9c10 con 0x7f6b9c0a3f00 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 4 set_mon_vals no callback set 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba10e0640 10 monclient: dump: 2024-08-19T10:56:22.083 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:22.084 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient: shutdown 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 -- 172.21.15.190:0/3066758062 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b9c0a3f00 msgr2=0x7f6b9c0a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 --2- 172.21.15.190:0/3066758062 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b9c0a3f00 0x7f6b9c0a42e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098d0 tx=0x7f6b9801c400 comp rx=0 tx=0).stop 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 -- 172.21.15.190:0/3066758062 shutdown_connections 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 --2- 172.21.15.190:0/3066758062 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b9c069680 0x7f6b9c09b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 --2- 172.21.15.190:0/3066758062 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f6b9c0742c0 0x7f6b9c0a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 --2- 172.21.15.190:0/3066758062 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f6b9c0a3f00 0x7f6b9c0a42e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 -- 172.21.15.190:0/3066758062 >> 172.21.15.190:0/3066758062 conn(0x7f6b9c0a4ff0 msgr2=0x7f6b9c0a53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 -- 172.21.15.190:0/3066758062 shutdown_connections 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 -- 172.21.15.190:0/3066758062 wait complete. 2024-08-19T10:56:22.085 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient: build_initial_monmap 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient: monmap: 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:22.086 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 Processor -- start 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 -- start start 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient: init 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient: _add_conns ranks=[0,1,6,4,8,3,7,5,2] 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b9c069680 0x7f6b9c13dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient(hunting): picked mon.noname-a con 0x7f6b9c069680 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b9c0742c0 0x7f6b9c13e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.087 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient(hunting): picked mon.noname-f con 0x7f6b9c0742c0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b9c0a3f00 0x7f6b9c1d1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient(hunting): picked mon.noname-d con 0x7f6b9c0a3f00 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c143470 con 0x7f6b9c069680 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1432f0 con 0x7f6b9c0742c0 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f6b9c1435f0 con 0x7f6b9c0a3f00 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba4346640 10 monclient(hunting): authenticate will time out at 2027.655554s 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba18e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b9c0742c0 0x7f6b9c13e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba18e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b9c0742c0 0x7f6b9c13e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:37124/0 (socket says 172.21.15.190:37124) 2024-08-19T10:56:22.088 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba18e1640 1 -- 172.21.15.190:0/4249788917 learned_addr learned my addr 172.21.15.190:0/4249788917 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba18e1640 10 monclient(hunting): get_auth_request con 0x7f6b9c0742c0 auth_method 0 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba18e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba18e1640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba18e1640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba18e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba18e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba18e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba20e2640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b9c069680 0x7f6b9c13dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba18e1640 10 monclient(hunting): handle_auth_done global_id 45496 payload 824 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.081+0000 7f6ba28e3640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b9c0a3f00 0x7f6b9c1d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba18e1640 1 -- 172.21.15.190:0/4249788917 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b9c0a3f00 msgr2=0x7f6b9c1d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba18e1640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b9c0a3f00 0x7f6b9c1d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.089 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba18e1640 1 -- 172.21.15.190:0/4249788917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b9c069680 msgr2=0x7f6b9c13dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba18e1640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b9c069680 0x7f6b9c13dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba18e1640 10 monclient: _finish_hunting 0 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba18e1640 1 monclient: found mon.noname-f 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba18e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba18e1640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba18e1640 1 -- 172.21.15.190:0/4249788917 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c12f070 con 0x7f6b9c0742c0 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba18e1640 1 -- 172.21.15.190:0/4249788917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b9c0742c0 msgr2=0x7f6b9c13e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba18e1640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b9c0742c0 0x7f6b9c13e4f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00b580 tx=0x7f6b8c00ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 1 -- 172.21.15.190:0/4249788917 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b8c013020 con 0x7f6b9c0742c0 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 10 monclient: dump: 2024-08-19T10:56:22.090 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:22.091 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 10 monclient: _finish_auth 0 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 1 -- 172.21.15.190:0/4249788917 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8c003dd0 con 0x7f6b9c0742c0 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 1 -- 172.21.15.190:0/4249788917 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f6b8c00f400 con 0x7f6b9c0742c0 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba4346640 5 monclient: authenticate success, global_id 45496 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba4346640 10 monclient: _renew_subs 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba4346640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba4346640 1 -- 172.21.15.190:0/4249788917 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c12e010 con 0x7f6b9c0742c0 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b93fff640 4 set_mon_vals no callback set 2024-08-19T10:56:22.092 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 10 monclient: dump: 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:22.093 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba4346640 10 monclient: _renew_subs 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba4346640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba4346640 1 -- 172.21.15.190:0/4249788917 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c13fda0 con 0x7f6b9c0742c0 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba4346640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba4346640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba4346640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6ba4346640 1 -- 172.21.15.190:0/4249788917 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9c051470 con 0x7f6b9c0742c0 2024-08-19T10:56:22.094 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 1 -- 172.21.15.190:0/4249788917 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f6b8c016020 con 0x7f6b9c0742c0 2024-08-19T10:56:22.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b8003be00 0x7f6b8003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.095 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.085+0000 7f6b92ffd640 1 -- 172.21.15.190:0/4249788917 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f6b8c04f3a0 con 0x7f6b9c0742c0 2024-08-19T10:56:22.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.089+0000 7f6ba20e2640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b8003be00 0x7f6b8003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:22.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.089+0000 7f6ba20e2640 10 monclient: get_auth_request con 0x7f6b8003be00 auth_method 0 2024-08-19T10:56:22.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.089+0000 7f6ba20e2640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b8003be00 0x7f6b8003e2c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098d0 tx=0x7f6b980a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:22.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.093+0000 7f6b92ffd640 1 -- 172.21.15.190:0/4249788917 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f6b8c01d020 con 0x7f6b9c0742c0 2024-08-19T10:56:22.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.093+0000 7f6b92ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:22.096 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.093+0000 7f6b92ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:22.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.313+0000 7f6ba4346640 10 start_mon_command cmd=[{"prefix": "health", "detail": "detail"}] 2024-08-19T10:56:22.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.313+0000 7f6ba4346640 10 monclient: _send_command 2 [{"prefix": "health", "detail": "detail"}] 2024-08-19T10:56:22.309 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.313+0000 7f6ba4346640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:22.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.313+0000 7f6ba4346640 1 -- 172.21.15.190:0/4249788917 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f6b9c09b080 con 0x7f6b9c0742c0 2024-08-19T10:56:22.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.317+0000 7f6b92ffd640 1 -- 172.21.15.190:0/4249788917 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+1324 (secure 0 0 0) 0x7f6b8c005590 con 0x7f6b9c0742c0 2024-08-19T10:56:22.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.317+0000 7f6b92ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "detail": "detail"}] 2024-08-19T10:56:22.310 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.317+0000 7f6b92ffd640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:22.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.465+0000 7f6ba4346640 10 start_mon_command cmd=[{"prefix": "health", "detail": "detail"}] 2024-08-19T10:56:22.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.465+0000 7f6ba4346640 10 monclient: _send_command 3 [{"prefix": "health", "detail": "detail"}] 2024-08-19T10:56:22.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.465+0000 7f6ba4346640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:22.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.465+0000 7f6ba4346640 1 -- 172.21.15.190:0/4249788917 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f6b9c0008e0 con 0x7f6b9c0742c0 2024-08-19T10:56:22.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6b92ffd640 1 -- 172.21.15.190:0/4249788917 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+1324 (secure 0 0 0) 0x7f6b8c00e360 con 0x7f6b9c0742c0 2024-08-19T10:56:22.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6b92ffd640 10 monclient: handle_mon_command_ack 3 [{"prefix": "health", "detail": "detail"}] 2024-08-19T10:56:22.462 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6b92ffd640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:56:22.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 -- 172.21.15.190:0/4249788917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b8003be00 msgr2=0x7f6b8003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:22.464 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b8003be00 0x7f6b8003e2c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098d0 tx=0x7f6b980a6000 comp rx=0 tx=0).stop 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 10 monclient: shutdown 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 -- 172.21.15.190:0/4249788917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b9c0742c0 msgr2=0x7f6b9c13e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b9c0742c0 0x7f6b9c13e4f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00b580 tx=0x7f6b8c00ba40 comp rx=0 tx=0).stop 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 -- 172.21.15.190:0/4249788917 shutdown_connections 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f6b8003be00 0x7f6b8003e2c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f6b9c0a3f00 0x7f6b9c1d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b9c0742c0 0x7f6b9c13e4f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 --2- 172.21.15.190:0/4249788917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6b9c069680 0x7f6b9c13dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 -- 172.21.15.190:0/4249788917 >> 172.21.15.190:0/4249788917 conn(0x7f6b9c0a4ff0 msgr2=0x7f6b9c09cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 -- 172.21.15.190:0/4249788917 shutdown_connections 2024-08-19T10:56:22.465 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.469+0000 7f6ba4346640 1 -- 172.21.15.190:0/4249788917 wait complete. 2024-08-19T10:56:22.478 INFO:tasks.workunit.client.0.smithi190.stdout: client.bad mds capability parse failed, stopped at 'also does not parse' of 'also does not parse' 2024-08-19T10:56:22.478 INFO:tasks.workunit.client.0.smithi190.stdout: client.bad mon capability parse failed, stopped at 'this is wrong' of 'this is wrong' 2024-08-19T10:56:22.478 INFO:tasks.workunit.client.0.smithi190.stdout: client.bad osd capability parse failed, stopped at 'does not parse' of 'does not parse' 2024-08-19T10:56:22.478 INFO:tasks.workunit.client.0.smithi190.stdout: client.bad mds capability parse failed, stopped at 'also does not parse' of 'also does not parse' 2024-08-19T10:56:22.478 INFO:tasks.workunit.client.0.smithi190.stdout: client.bad mon capability parse failed, stopped at 'this is wrong' of 'this is wrong' 2024-08-19T10:56:22.479 INFO:tasks.workunit.client.0.smithi190.stdout: client.bad osd capability parse failed, stopped at 'does not parse' of 'does not parse' 2024-08-19T10:56:22.479 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph auth rm client.bad 2024-08-19T10:56:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient: get_monmap_and_config 2024-08-19T10:56:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient: build_initial_monmap 2024-08-19T10:56:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient: monmap: 2024-08-19T10:56:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:56:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:56:22.596761+0000 2024-08-19T10:56:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:56:22.596761+0000 2024-08-19T10:56:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:22.587 INFO:tasks.workunit.client.0.smithi190.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:22.588 INFO:tasks.workunit.client.0.smithi190.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:22.588 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 1 Processor -- start 2024-08-19T10:56:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 1 -- start start 2024-08-19T10:56:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient: init 2024-08-19T10:56:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient: _add_conns ranks=[2,6,7,5,10,0,3,8,1,4,9] 2024-08-19T10:56:22.591 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a50d0 0x7f1ac40a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient(hunting): picked mon.noname-b con 0x7f1ac40a50d0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ac40a54b0 0x7f1ac40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient(hunting): picked mon.noname-d con 0x7f1ac40a54b0 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a6a60 0x7f1ac40a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient(hunting): picked mon.noname-i con 0x7f1ac40a6a60 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac40a3ea0 con 0x7f1ac40a50d0 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f1ac40a9680 con 0x7f1ac40a54b0 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f1ac40a9900 con 0x7f1ac40a6a60 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:22.592 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac9182640 10 monclient(hunting): authenticate will time out at 2028.167272s 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac359e640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a6a60 msgr2=0x7f1ac40a6ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac359e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a6a60 0x7f1ac40a6ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1aba59c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ac40a54b0 0x7f1ac40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1aba59c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ac40a54b0 0x7f1ac40a58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.190:35292/0 (socket says 172.21.15.190:35292) 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1aba59c640 1 -- 172.21.15.190:0/2217957770 learned_addr learned my addr 172.21.15.190:0/2217957770 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1aba59c640 10 monclient(hunting): get_auth_request con 0x7f1ac40a54b0 auth_method 0 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1aba59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1aba59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1aba59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac2d9d640 1 -- 172.21.15.190:0/2217957770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a50d0 msgr2=0x7f1ac40a4880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1aba59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1aba59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1ac2d9d640 1 --2- 172.21.15.190:0/2217957770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a50d0 0x7f1ac40a4880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:22.593 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.593+0000 7f1aba59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.597+0000 7f1aba59c640 10 monclient(hunting): handle_auth_done global_id 45591 payload 274 2024-08-19T10:56:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.597+0000 7f1aba59c640 1 -- 172.21.15.190:0/2217957770 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a6a60 msgr2=0x7f1ac40a6ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.597+0000 7f1aba59c640 1 --2- 172.21.15.190:0/2217957770 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a6a60 0x7f1ac40a6ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.597+0000 7f1aba59c640 1 -- 172.21.15.190:0/2217957770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a50d0 msgr2=0x7f1ac40a4880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.597+0000 7f1aba59c640 1 --2- 172.21.15.190:0/2217957770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a50d0 0x7f1ac40a4880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.597+0000 7f1aba59c640 10 monclient: _finish_hunting 0 2024-08-19T10:56:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.597+0000 7f1aba59c640 1 monclient: found mon.noname-d 2024-08-19T10:56:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.597+0000 7f1aba59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.597+0000 7f1aba59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:56:22.594 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.597+0000 7f1aba59c640 1 -- 172.21.15.190:0/2217957770 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac40a7c00 con 0x7f1ac40a54b0 2024-08-19T10:56:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1aba59c640 1 -- 172.21.15.190:0/2217957770 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ac40a54b0 msgr2=0x7f1ac40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1aba59c640 1 --2- 172.21.15.190:0/2217957770 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ac40a54b0 0x7f1ac40a58f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0098d0 tx=0x7f1aac01cee0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=5a4b895f4122f753 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 1 -- 172.21.15.190:0/2217957770 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1aac0b0070 con 0x7f1ac40a54b0 2024-08-19T10:56:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:56:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 10 monclient: dump: 2024-08-19T10:56:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:22.595 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:22.596 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 10 monclient: _finish_auth 0 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 10 monclient: _check_auth_tickets 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 1 -- 172.21.15.190:0/2217957770 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac0aa810 con 0x7f1ac40a54b0 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 1 -- 172.21.15.190:0/2217957770 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1aac0a93e0 con 0x7f1ac40a54b0 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac259c640 10 monclient: dump: 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:22.597 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.598 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 5 monclient: authenticate success, global_id 45591 2024-08-19T10:56:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:22.599 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 4 set_mon_vals no callback set 2024-08-19T10:56:22.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient: shutdown 2024-08-19T10:56:22.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:22.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 -- 172.21.15.190:0/2217957770 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ac40a54b0 msgr2=0x7f1ac40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:22.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 --2- 172.21.15.190:0/2217957770 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ac40a54b0 0x7f1ac40a58f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0098d0 tx=0x7f1aac01cee0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 -- 172.21.15.190:0/2217957770 shutdown_connections 2024-08-19T10:56:22.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 --2- 172.21.15.190:0/2217957770 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f1ac40a54b0 0x7f1ac40a58f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 --2- 172.21.15.190:0/2217957770 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a6a60 0x7f1ac40a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 --2- 172.21.15.190:0/2217957770 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a50d0 0x7f1ac40a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 -- 172.21.15.190:0/2217957770 >> 172.21.15.190:0/2217957770 conn(0x7f1ac40ac6c0 msgr2=0x7f1ac40a8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:22.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 -- 172.21.15.190:0/2217957770 shutdown_connections 2024-08-19T10:56:22.604 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 -- 172.21.15.190:0/2217957770 wait complete. 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient: build_initial_monmap 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient: monmap: 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 0 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 0.000000 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:created 0.000000 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:22.605 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:22.606 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:22.606 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:22.606 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:22.606 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:22.606 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 Processor -- start 2024-08-19T10:56:22.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 -- start start 2024-08-19T10:56:22.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient: init 2024-08-19T10:56:22.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:22.606 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient: _add_conns ranks=[1,7,2,4,6,5,0,8,3] 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac40a50d0 0x7f1ac41d1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient(hunting): picked mon.noname-f con 0x7f1ac40a50d0 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a54b0 0x7f1ac413bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient(hunting): picked mon.noname-i con 0x7f1ac40a54b0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a6a60 0x7f1ac41238c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient(hunting): picked mon.noname-b con 0x7f1ac40a6a60 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4143080 con 0x7f1ac40a50d0 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac40743a0 con 0x7f1ac40a6a60 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:22.607 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f1ac4143200 con 0x7f1ac40a54b0 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1ac9182640 10 monclient(hunting): authenticate will time out at 2028.175950s 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1aba59c640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a54b0 msgr2=0x7f1ac413bcf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.601+0000 7f1aba59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a54b0 0x7f1ac413bcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac40a50d0 0x7f1ac41d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac40a50d0 0x7f1ac41d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:37128/0 (socket says 172.21.15.190:37128) 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 1 -- 172.21.15.190:0/3993834649 learned_addr learned my addr 172.21.15.190:0/3993834649 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 10 monclient(hunting): get_auth_request con 0x7f1ac40a50d0 auth_method 0 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac359e640 1 -- 172.21.15.190:0/3993834649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a6a60 msgr2=0x7f1ac41238c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:22.608 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac359e640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a6a60 0x7f1ac41238c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 10 monclient(hunting): handle_auth_done global_id 45505 payload 824 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 1 -- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a54b0 msgr2=0x7f1ac413bcf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a54b0 0x7f1ac413bcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 1 -- 172.21.15.190:0/3993834649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a6a60 msgr2=0x7f1ac41238c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a6a60 0x7f1ac41238c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 10 monclient: _finish_hunting 0 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 1 monclient: found mon.noname-f 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:22.609 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 1 -- 172.21.15.190:0/3993834649 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac4125330 con 0x7f1ac40a50d0 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 1 -- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac40a50d0 msgr2=0x7f1ac41d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac2d9d640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac40a50d0 0x7f1ac41d1510 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400b530 tx=0x7f1ab400b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 1 -- 172.21.15.190:0/3993834649 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ab4013020 con 0x7f1ac40a50d0 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 10 monclient: dump: 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.610 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 10 monclient: _finish_auth 0 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:22.611 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac9182640 5 monclient: authenticate success, global_id 45505 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac9182640 10 monclient: _renew_subs 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 1 -- 172.21.15.190:0/3993834649 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab4003de0 con 0x7f1ac40a50d0 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac9182640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac9182640 1 -- 172.21.15.190:0/3993834649 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac40a2750 con 0x7f1ac40a50d0 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 1 -- 172.21.15.190:0/3993834649 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f1ab400f400 con 0x7f1ac40a50d0 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac159a640 4 set_mon_vals no callback set 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 10 monclient: dump: 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:epoch 1 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.612 INFO:tasks.workunit.client.0.smithi190.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:election_strategy: 1 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr: 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac9182640 10 monclient: _renew_subs 2024-08-19T10:56:22.613 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac9182640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:22.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac9182640 1 -- 172.21.15.190:0/3993834649 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac4126df0 con 0x7f1ac40a50d0 2024-08-19T10:56:22.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac9182640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:22.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac9182640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:22.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac9182640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:22.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1ac9182640 1 -- 172.21.15.190:0/3993834649 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac40a86c0 con 0x7f1ac40a50d0 2024-08-19T10:56:22.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.605+0000 7f1abbfff640 1 -- 172.21.15.190:0/3993834649 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f1ab4016020 con 0x7f1ac40a50d0 2024-08-19T10:56:22.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.609+0000 7f1abbfff640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1aa403be00 0x7f1aa403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:22.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.609+0000 7f1abbfff640 1 -- 172.21.15.190:0/3993834649 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f1ab404f2d0 con 0x7f1ac40a50d0 2024-08-19T10:56:22.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.609+0000 7f1aba59c640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1aa403be00 0x7f1aa403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:22.614 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.609+0000 7f1aba59c640 10 monclient: get_auth_request con 0x7f1aa403be00 auth_method 0 2024-08-19T10:56:22.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.609+0000 7f1aba59c640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1aa403be00 0x7f1aa403e2c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0098a0 tx=0x7f1aac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:22.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.613+0000 7f1abbfff640 1 -- 172.21.15.190:0/3993834649 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f1ac40a86c0 con 0x7f1ac40a50d0 2024-08-19T10:56:22.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.613+0000 7f1abbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:22.615 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.613+0000 7f1abbfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:22.842 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.849+0000 7f1ac9182640 10 start_mon_command cmd=[{"prefix": "auth rm", "entity": "client.bad"}] 2024-08-19T10:56:22.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.849+0000 7f1ac9182640 10 monclient: _send_command 2 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-08-19T10:56:22.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.849+0000 7f1ac9182640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:22.843 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.849+0000 7f1ac9182640 1 -- 172.21.15.190:0/3993834649 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.bad"} v 0) -- 0x7f1ac40a58f0 con 0x7f1ac40a50d0 2024-08-19T10:56:22.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.857+0000 7f1abbfff640 1 -- 172.21.15.190:0/3993834649 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.bad"}]=0 v27) ==== 79+0+0 (secure 0 0 0) 0x7f1ab401d020 con 0x7f1ac40a50d0 2024-08-19T10:56:22.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.857+0000 7f1abbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-08-19T10:56:22.851 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:22.857+0000 7f1abbfff640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:23.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.009+0000 7f1ac9182640 10 start_mon_command cmd=[{"prefix": "auth rm", "entity": "client.bad"}] 2024-08-19T10:56:23.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.009+0000 7f1ac9182640 10 monclient: _send_command 3 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-08-19T10:56:23.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.009+0000 7f1ac9182640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:23.004 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.009+0000 7f1ac9182640 1 -- 172.21.15.190:0/3993834649 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.bad"} v 0) -- 0x7f1ac4051470 con 0x7f1ac40a50d0 2024-08-19T10:56:23.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1abbfff640 1 -- 172.21.15.190:0/3993834649 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.bad"}]=0 v27) ==== 79+0+0 (secure 0 0 0) 0x7f1ab4005580 con 0x7f1ac40a50d0 2024-08-19T10:56:23.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1abbfff640 10 monclient: handle_mon_command_ack 3 [{"prefix": "auth rm", "entity": "client.bad"}] 2024-08-19T10:56:23.006 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1abbfff640 10 monclient: _finish_command 3 = system:0 2024-08-19T10:56:23.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 -- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1aa403be00 msgr2=0x7f1aa403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:23.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1aa403be00 0x7f1aa403e2c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0098a0 tx=0x7f1aac008040 comp rx=0 tx=0).stop 2024-08-19T10:56:23.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 10 monclient: shutdown 2024-08-19T10:56:23.008 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:23.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 -- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac40a50d0 msgr2=0x7f1ac41d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:23.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac40a50d0 0x7f1ac41d1510 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400b530 tx=0x7f1ab400b9f0 comp rx=0 tx=0).stop 2024-08-19T10:56:23.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 -- 172.21.15.190:0/3993834649 shutdown_connections 2024-08-19T10:56:23.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f1aa403be00 0x7f1aa403e2c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:23.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f1ac40a6a60 0x7f1ac41238c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:23.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f1ac40a54b0 0x7f1ac413bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:23.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 --2- 172.21.15.190:0/3993834649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ac40a50d0 0x7f1ac41d1510 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:23.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 -- 172.21.15.190:0/3993834649 >> 172.21.15.190:0/3993834649 conn(0x7f1ac40ac6c0 msgr2=0x7f1ac413d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:23.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 -- 172.21.15.190:0/3993834649 shutdown_connections 2024-08-19T10:56:23.009 INFO:tasks.workunit.client.0.smithi190.stderr:2024-08-19T10:56:23.013+0000 7f1ac9182640 1 -- 172.21.15.190:0/3993834649 wait complete. 2024-08-19T10:56:23.022 INFO:tasks.workunit.client.0.smithi190.stderr:+ expect 'ceph auth health | grep AUTH_BAD_CAP' 1 2024-08-19T10:56:23.023 INFO:tasks.workunit.client.0.smithi190.stderr:+ cmd='ceph auth health | grep AUTH_BAD_CAP' 2024-08-19T10:56:23.023 INFO:tasks.workunit.client.0.smithi190.stderr:+ expected_ret=1 2024-08-19T10:56:23.023 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo ceph auth health '|' grep AUTH_BAD_CAP 2024-08-19T10:56:23.023 INFO:tasks.workunit.client.0.smithi190.stdout:ceph auth health | grep AUTH_BAD_CAP 2024-08-19T10:56:23.023 INFO:tasks.workunit.client.0.smithi190.stderr:+ eval ceph auth health '|' grep AUTH_BAD_CAP 2024-08-19T10:56:23.368 INFO:tasks.workunit.client.0.smithi190.stderr:+ ret=1 2024-08-19T10:56:23.368 INFO:tasks.workunit.client.0.smithi190.stderr:+ [[ 1 -ne 1 ]] 2024-08-19T10:56:23.368 INFO:tasks.workunit.client.0.smithi190.stderr:+ return 0 2024-08-19T10:56:23.368 INFO:tasks.workunit.client.0.smithi190.stdout:OK 2024-08-19T10:56:23.369 INFO:tasks.workunit.client.0.smithi190.stderr:+ echo OK 2024-08-19T10:56:23.370 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-19T10:56:23.370 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-19T10:56:23.425 INFO:tasks.workunit:Stopping ['mon/pool_ops.sh', 'mon/crush_ops.sh', 'mon/osd.sh', 'mon/caps.sh'] on client.0... 2024-08-19T10:56:23.425 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-19T10:56:23.787 DEBUG:teuthology.parallel:result is None 2024-08-19T10:56:23.788 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-08-19T10:56:23.801 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-08-19T10:56:23.801 DEBUG:teuthology.orchestra.run.smithi190:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-08-19T10:56:23.860 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-08-19T10:56:23.861 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-08-19T10:56:23.870 INFO:tasks.mon_thrash:joining mon_thrasher 2024-08-19T10:56:23.871 ERROR:teuthology.run_tasks:Manager failed: mon_thrash Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa/tasks/mon_thrash.py", line 466, in task thrash_proc.stop_and_join() File "/home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa/tasks/mon_thrash.py", line 178, in stop_and_join return self.join() File "/home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa/tasks/mon_thrash.py", line 170, in join self.stopping.set() AttributeError: 'bool' object has no attribute 'set' 2024-08-19T10:56:23.872 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-08-19T10:56:23.903 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-19T10:56:23.904 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T10:56:24.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient: get_monmap_and_config 2024-08-19T10:56:24.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient: build_initial_monmap 2024-08-19T10:56:24.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient: monmap: 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:24.042194+0000 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:24.042194+0000 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:24.037 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:24.038 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:24.038 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:24.038 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:24.038 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 1 Processor -- start 2024-08-19T10:56:24.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 1 -- start start 2024-08-19T10:56:24.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient: init 2024-08-19T10:56:24.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:24.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient: _add_conns ranks=[6,1,3,10,7,2,8,5,0,9,4] 2024-08-19T10:56:24.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f955c0a6860 0x7f955c0a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient(hunting): picked mon.noname-d con 0x7f955c0a6860 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:24.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f955c0a54d0 addr v1:172.21.15.190:6789/0 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955c0a58b0 0x7f955c1408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient(hunting): picked mon.noname-g con 0x7f955c0a58b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f955c0a4c90 con 0x7f955c0a54d0 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f955c0a7350 con 0x7f955c0a58b0 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f955c0a7550 con 0x7f955c0a6860 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f9560906640 10 monclient(hunting): authenticate will time out at 2033.858663s 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f955c0a6860 0x7f955c0a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f955c0a6860 0x7f955c0a81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:44152/0 (socket says 172.21.15.89:44152) 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 1 -- 172.21.15.89:0/1835885583 learned_addr learned my addr 172.21.15.89:0/1835885583 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:24.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 10 monclient(hunting): get_auth_request con 0x7f955c0a6860 auth_method 0 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955ad9d640 1 --2- 172.21.15.89:0/1835885583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955c0a58b0 0x7f955c1408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955ad9d640 10 monclient(hunting): get_auth_request con 0x7f955c0a58b0 auth_method 0 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955ad9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 10 monclient(hunting): handle_auth_done global_id 45600 payload 274 2024-08-19T10:56:24.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 1 -- 172.21.15.89:0/1835885583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955c0a58b0 msgr2=0x7f955c1408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 1 --2- 172.21.15.89:0/1835885583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955c0a58b0 0x7f955c1408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 1 -- 172.21.15.89:0/1835885583 >> v1:172.21.15.190:6789/0 conn(0x7f955c0a54d0 legacy=0x7f955c0a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 10 monclient: _finish_hunting 0 2024-08-19T10:56:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 1 monclient: found mon.noname-d 2024-08-19T10:56:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:56:24.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.037+0000 7f955a59c640 1 -- 172.21.15.89:0/1835885583 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c141e30 con 0x7f955c0a6860 2024-08-19T10:56:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f955a59c640 1 -- 172.21.15.89:0/1835885583 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f955c0a6860 msgr2=0x7f955c0a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f955a59c640 1 --2- 172.21.15.89:0/1835885583 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f955c0a6860 0x7f955c0a81b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f954c0098a0 tx=0x7f954c01c4d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b5731d4d5b47e837 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 1 -- 172.21.15.89:0/1835885583 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f954c0b0070 con 0x7f955c0a6860 2024-08-19T10:56:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:56:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 10 monclient: dump: 2024-08-19T10:56:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.049 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.050 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 10 monclient: _finish_auth 0 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 10 monclient: _check_auth_tickets 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 1 -- 172.21.15.89:0/1835885583 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954c0a9980 con 0x7f955c0a6860 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9560906640 5 monclient: authenticate success, global_id 45600 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9560906640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9560906640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 1 -- 172.21.15.89:0/1835885583 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f954c0aa3a0 con 0x7f955c0a6860 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9560906640 4 set_mon_vals no callback set 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:24.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.041+0000 7f9559d9b640 10 monclient: dump: 2024-08-19T10:56:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.052 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient: shutdown 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 -- 172.21.15.89:0/1835885583 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f955c0a6860 msgr2=0x7f955c0a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 --2- 172.21.15.89:0/1835885583 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f955c0a6860 0x7f955c0a81b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f954c0098a0 tx=0x7f954c01c4d0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 -- 172.21.15.89:0/1835885583 shutdown_connections 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 --2- 172.21.15.89:0/1835885583 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955c0a58b0 0x7f955c1408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 --2- 172.21.15.89:0/1835885583 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f955c0a6860 0x7f955c0a81b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 -- 172.21.15.89:0/1835885583 >> 172.21.15.89:0/1835885583 conn(0x7f955c069a00 msgr2=0x7f955c0abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 -- 172.21.15.89:0/1835885583 shutdown_connections 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 -- 172.21.15.89:0/1835885583 wait complete. 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient: build_initial_monmap 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient: monmap: 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 Processor -- start 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 -- start start 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient: init 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient: _add_conns ranks=[3,5,7,4,2,0,6,8,1] 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955c0a54d0 0x7f955c13a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient(hunting): picked mon.noname-g con 0x7f955c0a54d0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f955c0a58b0 0x7f955c13abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient(hunting): picked mon.noname-h con 0x7f955c0a58b0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f955c0a6860 0x7f955c13b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient(hunting): picked mon.noname-i con 0x7f955c0a6860 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f955c1419c0 con 0x7f955c0a54d0 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f955c141840 con 0x7f955c0a58b0 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f955c141b40 con 0x7f955c0a6860 2024-08-19T10:56:24.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9560906640 10 monclient(hunting): authenticate will time out at 2033.866427s 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f955ad9d640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f955c0a6860 msgr2=0x7f955c13b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f955ad9d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f955c0a6860 0x7f955c13b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f955c0a58b0 0x7f955c13abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f955a59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955c0a54d0 0x7f955c13a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f955c0a58b0 0x7f955c13abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:44794/0 (socket says 172.21.15.89:44794) 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 1 -- 172.21.15.89:0/2409157936 learned_addr learned my addr 172.21.15.89:0/2409157936 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 10 monclient(hunting): get_auth_request con 0x7f955c0a58b0 auth_method 0 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f955a59c640 10 monclient(hunting): get_auth_request con 0x7f955c0a54d0 auth_method 0 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f955a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:24.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f955a59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f955a59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f955a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f955a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f955a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 10 monclient(hunting): handle_auth_done global_id 84452 payload 824 2024-08-19T10:56:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 1 -- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f955c0a6860 msgr2=0x7f955c13b120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f955c0a6860 0x7f955c13b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 1 -- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955c0a54d0 msgr2=0x7f955c13a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955c0a54d0 0x7f955c13a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 10 monclient: _finish_hunting 0 2024-08-19T10:56:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 1 monclient: found mon.noname-h 2024-08-19T10:56:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:56:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.045+0000 7f9551d9b640 1 -- 172.21.15.89:0/2409157936 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c13cf00 con 0x7f955c0a58b0 2024-08-19T10:56:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9551d9b640 1 -- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f955c0a58b0 msgr2=0x7f955c13abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:24.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9551d9b640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f955c0a58b0 0x7f955c13abe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f954400c480 tx=0x7f954400c940 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 1 -- 172.21.15.89:0/2409157936 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9544012cb0 con 0x7f955c0a58b0 2024-08-19T10:56:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 10 monclient: dump: 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.062 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 10 monclient: _finish_auth 0 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 10 monclient: _check_auth_tickets 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 1 -- 172.21.15.89:0/2409157936 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9544017070 con 0x7f955c0a58b0 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 1 -- 172.21.15.89:0/2409157936 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f9544011380 con 0x7f955c0a58b0 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:56:24.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f95537fe640 10 monclient: dump: 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.064 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9558d99640 4 set_mon_vals no callback set 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9560906640 5 monclient: authenticate success, global_id 84452 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9560906640 10 monclient: _renew_subs 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9560906640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9560906640 1 -- 172.21.15.89:0/2409157936 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c140ac0 con 0x7f955c0a58b0 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9560906640 10 monclient: _renew_subs 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9560906640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9560906640 1 -- 172.21.15.89:0/2409157936 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c13def0 con 0x7f955c0a58b0 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9560906640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9560906640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:24.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9560906640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.190:3302/0 2024-08-19T10:56:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.049+0000 7f9560906640 1 -- 172.21.15.89:0/2409157936 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955c0a9470 con 0x7f955c0a58b0 2024-08-19T10:56:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.053+0000 7f95537fe640 1 -- 172.21.15.89:0/2409157936 <== mon.5 v2:172.21.15.190:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f9544012840 con 0x7f955c0a58b0 2024-08-19T10:56:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.053+0000 7f95537fe640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f953c03be00 0x7f953c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.053+0000 7f95537fe640 1 -- 172.21.15.89:0/2409157936 <== mon.5 v2:172.21.15.190:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f954404fec0 con 0x7f955c0a58b0 2024-08-19T10:56:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.053+0000 7f955a59c640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f953c03be00 0x7f953c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.053+0000 7f955a59c640 10 monclient: get_auth_request con 0x7f953c03be00 auth_method 0 2024-08-19T10:56:24.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.057+0000 7f955a59c640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f953c03be00 0x7f953c03e2c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f955c0a5440 tx=0x7f954c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:24.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.065+0000 7f95537fe640 1 -- 172.21.15.89:0/2409157936 <== mon.5 v2:172.21.15.190:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f955c0a9470 con 0x7f955c0a58b0 2024-08-19T10:56:24.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.065+0000 7f95537fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:24.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.065+0000 7f95537fe640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:24.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.233+0000 7f9560906640 1 -- 172.21.15.89:0/2409157936 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f955c0a1ad0 con 0x7f953c03be00 2024-08-19T10:56:24.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.237+0000 7f95537fe640 1 -- 172.21.15.89:0/2409157936 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+78517 (secure 0 0 0) 0x7f955c0a1ad0 con 0x7f953c03be00 2024-08-19T10:56:24.238 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:56:24.238 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-08-19T10:56:24.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 -- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f953c03be00 msgr2=0x7f953c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f953c03be00 0x7f953c03e2c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f955c0a5440 tx=0x7f954c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 10 monclient: shutdown 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 -- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f955c0a58b0 msgr2=0x7f955c13abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f955c0a58b0 0x7f955c13abe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f954400c480 tx=0x7f954400c940 comp rx=0 tx=0).stop 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 -- 172.21.15.89:0/2409157936 shutdown_connections 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f953c03be00 0x7f953c03e2c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f955c0a58b0 0x7f955c13abe0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f955c0a6860 0x7f955c13b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 --2- 172.21.15.89:0/2409157936 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955c0a54d0 0x7f955c13a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 -- 172.21.15.89:0/2409157936 >> 172.21.15.89:0/2409157936 conn(0x7f955c069a00 msgr2=0x7f955c135d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 -- 172.21.15.89:0/2409157936 shutdown_connections 2024-08-19T10:56:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.241+0000 7f9560906640 1 -- 172.21.15.89:0/2409157936 wait complete. 2024-08-19T10:56:24.256 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":817,"stamp":"2024-08-19T10:56:22.774248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":198040,"kb_used_data":10936,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033000,"statfs":{"total":579820584960,"available":579617792000,"internally_reserved":0,"allocated":11198464,"data_stored":8850040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":191486624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003588"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:09.851422+0000","last_change":"2024-08-19T10:43:42.769648+0000","last_active":"2024-08-19T10:54:09.851422+0000","last_peered":"2024-08-19T10:54:09.851422+0000","last_clean":"2024-08-19T10:54:09.851422+0000","last_became_active":"2024-08-19T10:43:42.769057+0000","last_became_peered":"2024-08-19T10:43:42.769057+0000","last_unstale":"2024-08-19T10:54:09.851422+0000","last_undegraded":"2024-08-19T10:54:09.851422+0000","last_fullsized":"2024-08-19T10:54:09.851422+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:59:18.923960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.531111+0000","last_change":"2024-08-19T10:43:42.776790+0000","last_active":"2024-08-19T10:52:24.531111+0000","last_peered":"2024-08-19T10:52:24.531111+0000","last_clean":"2024-08-19T10:52:24.531111+0000","last_became_active":"2024-08-19T10:43:42.775930+0000","last_became_peered":"2024-08-19T10:43:42.775930+0000","last_unstale":"2024-08-19T10:52:24.531111+0000","last_undegraded":"2024-08-19T10:52:24.531111+0000","last_fullsized":"2024-08-19T10:52:24.531111+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:00:49.264074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:14.852152+0000","last_change":"2024-08-19T10:43:42.776699+0000","last_active":"2024-08-19T10:54:14.852152+0000","last_peered":"2024-08-19T10:54:14.852152+0000","last_clean":"2024-08-19T10:54:14.852152+0000","last_became_active":"2024-08-19T10:43:42.775384+0000","last_became_peered":"2024-08-19T10:43:42.775384+0000","last_unstale":"2024-08-19T10:54:14.852152+0000","last_undegraded":"2024-08-19T10:54:14.852152+0000","last_fullsized":"2024-08-19T10:54:14.852152+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:11:44.821797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.531052+0000","last_change":"2024-08-19T10:43:42.770075+0000","last_active":"2024-08-19T10:52:24.531052+0000","last_peered":"2024-08-19T10:52:24.531052+0000","last_clean":"2024-08-19T10:52:24.531052+0000","last_became_active":"2024-08-19T10:43:42.769225+0000","last_became_peered":"2024-08-19T10:43:42.769225+0000","last_unstale":"2024-08-19T10:52:24.531052+0000","last_undegraded":"2024-08-19T10:52:24.531052+0000","last_fullsized":"2024-08-19T10:52:24.531052+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:14:39.665839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":198,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.534076+0000","last_change":"2024-08-19T10:43:42.771832+0000","last_active":"2024-08-19T10:52:24.534076+0000","last_peered":"2024-08-19T10:52:24.534076+0000","last_clean":"2024-08-19T10:52:24.534076+0000","last_became_active":"2024-08-19T10:43:42.770907+0000","last_became_peered":"2024-08-19T10:43:42.770907+0000","last_unstale":"2024-08-19T10:52:24.534076+0000","last_undegraded":"2024-08-19T10:52:24.534076+0000","last_fullsized":"2024-08-19T10:52:24.534076+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:51:29.432515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":202,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:15.416700+0000","last_change":"2024-08-19T10:43:42.789929+0000","last_active":"2024-08-19T10:54:15.416700+0000","last_peered":"2024-08-19T10:54:15.416700+0000","last_clean":"2024-08-19T10:54:15.416700+0000","last_became_active":"2024-08-19T10:43:42.789563+0000","last_became_peered":"2024-08-19T10:43:42.789563+0000","last_unstale":"2024-08-19T10:54:15.416700+0000","last_undegraded":"2024-08-19T10:54:15.416700+0000","last_fullsized":"2024-08-19T10:54:15.416700+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:51:07.959247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":309,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:05.773040+0000","last_change":"2024-08-19T10:43:42.788997+0000","last_active":"2024-08-19T10:53:05.773040+0000","last_peered":"2024-08-19T10:53:05.773040+0000","last_clean":"2024-08-19T10:53:05.773040+0000","last_became_active":"2024-08-19T10:43:42.784411+0000","last_became_peered":"2024-08-19T10:43:42.784411+0000","last_unstale":"2024-08-19T10:53:05.773040+0000","last_undegraded":"2024-08-19T10:53:05.773040+0000","last_fullsized":"2024-08-19T10:53:05.773040+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:44:43.826178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421871+0000","last_change":"2024-08-19T10:43:42.785782+0000","last_active":"2024-08-19T10:54:10.421871+0000","last_peered":"2024-08-19T10:54:10.421871+0000","last_clean":"2024-08-19T10:54:10.421871+0000","last_became_active":"2024-08-19T10:43:42.785474+0000","last_became_peered":"2024-08-19T10:43:42.785474+0000","last_unstale":"2024-08-19T10:54:10.421871+0000","last_undegraded":"2024-08-19T10:54:10.421871+0000","last_fullsized":"2024-08-19T10:54:10.421871+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:37:33.828996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.413864+0000","last_change":"2024-08-19T10:43:42.786904+0000","last_active":"2024-08-19T10:53:49.413864+0000","last_peered":"2024-08-19T10:53:49.413864+0000","last_clean":"2024-08-19T10:53:49.413864+0000","last_became_active":"2024-08-19T10:43:42.786027+0000","last_became_peered":"2024-08-19T10:43:42.786027+0000","last_unstale":"2024-08-19T10:53:49.413864+0000","last_undegraded":"2024-08-19T10:53:49.413864+0000","last_fullsized":"2024-08-19T10:53:49.413864+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:18:23.651707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":316,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:40.769323+0000","last_change":"2024-08-19T10:43:42.785108+0000","last_active":"2024-08-19T10:52:40.769323+0000","last_peered":"2024-08-19T10:52:40.769323+0000","last_clean":"2024-08-19T10:52:40.769323+0000","last_became_active":"2024-08-19T10:43:42.784268+0000","last_became_peered":"2024-08-19T10:43:42.784268+0000","last_unstale":"2024-08-19T10:52:40.769323+0000","last_undegraded":"2024-08-19T10:52:40.769323+0000","last_fullsized":"2024-08-19T10:52:40.769323+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:40:36.868319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":318,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:40.769280+0000","last_change":"2024-08-19T10:43:42.784728+0000","last_active":"2024-08-19T10:52:40.769280+0000","last_peered":"2024-08-19T10:52:40.769280+0000","last_clean":"2024-08-19T10:52:40.769280+0000","last_became_active":"2024-08-19T10:43:42.783709+0000","last_became_peered":"2024-08-19T10:43:42.783709+0000","last_unstale":"2024-08-19T10:52:40.769280+0000","last_undegraded":"2024-08-19T10:52:40.769280+0000","last_fullsized":"2024-08-19T10:52:40.769280+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:30:15.771986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421853+0000","last_change":"2024-08-19T10:43:42.767704+0000","last_active":"2024-08-19T10:54:10.421853+0000","last_peered":"2024-08-19T10:54:10.421853+0000","last_clean":"2024-08-19T10:54:10.421853+0000","last_became_active":"2024-08-19T10:43:42.767225+0000","last_became_peered":"2024-08-19T10:43:42.767225+0000","last_unstale":"2024-08-19T10:54:10.421853+0000","last_undegraded":"2024-08-19T10:54:10.421853+0000","last_fullsized":"2024-08-19T10:54:10.421853+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:29:35.198111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.415852+0000","last_change":"2024-08-19T10:43:43.768263+0000","last_active":"2024-08-19T10:54:10.415852+0000","last_peered":"2024-08-19T10:54:10.415852+0000","last_clean":"2024-08-19T10:54:10.415852+0000","last_became_active":"2024-08-19T10:43:43.767866+0000","last_became_peered":"2024-08-19T10:43:43.767866+0000","last_unstale":"2024-08-19T10:54:10.415852+0000","last_undegraded":"2024-08-19T10:54:10.415852+0000","last_fullsized":"2024-08-19T10:54:10.415852+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:12:13.038948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":205,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421793+0000","last_change":"2024-08-19T10:43:43.763907+0000","last_active":"2024-08-19T10:54:10.421793+0000","last_peered":"2024-08-19T10:54:10.421793+0000","last_clean":"2024-08-19T10:54:10.421793+0000","last_became_active":"2024-08-19T10:43:43.763033+0000","last_became_peered":"2024-08-19T10:43:43.763033+0000","last_unstale":"2024-08-19T10:54:10.421793+0000","last_undegraded":"2024-08-19T10:54:10.421793+0000","last_fullsized":"2024-08-19T10:54:10.421793+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:15:08.848730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":235,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:54.965756+0000","last_change":"2024-08-19T10:43:43.773879+0000","last_active":"2024-08-19T10:52:54.965756+0000","last_peered":"2024-08-19T10:52:54.965756+0000","last_clean":"2024-08-19T10:52:54.965756+0000","last_became_active":"2024-08-19T10:43:43.773098+0000","last_became_peered":"2024-08-19T10:43:43.773098+0000","last_unstale":"2024-08-19T10:52:54.965756+0000","last_undegraded":"2024-08-19T10:52:54.965756+0000","last_fullsized":"2024-08-19T10:52:54.965756+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:05:31.013999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":202,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:50.412824+0000","last_change":"2024-08-19T10:43:42.776431+0000","last_active":"2024-08-19T10:53:50.412824+0000","last_peered":"2024-08-19T10:53:50.412824+0000","last_clean":"2024-08-19T10:53:50.412824+0000","last_became_active":"2024-08-19T10:43:42.776033+0000","last_became_peered":"2024-08-19T10:43:42.776033+0000","last_unstale":"2024-08-19T10:53:50.412824+0000","last_undegraded":"2024-08-19T10:53:50.412824+0000","last_fullsized":"2024-08-19T10:53:50.412824+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:10:47.358711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.423024+0000","last_change":"2024-08-19T10:43:42.807228+0000","last_active":"2024-08-19T10:53:49.423024+0000","last_peered":"2024-08-19T10:53:49.423024+0000","last_clean":"2024-08-19T10:53:49.423024+0000","last_became_active":"2024-08-19T10:43:42.775224+0000","last_became_peered":"2024-08-19T10:43:42.775224+0000","last_unstale":"2024-08-19T10:53:49.423024+0000","last_undegraded":"2024-08-19T10:53:49.423024+0000","last_fullsized":"2024-08-19T10:53:49.423024+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_clean_scrub_stamp":"2024-08-19T10:38:04.611102+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:21:32.163214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":491,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:49.537692+0000","last_change":"2024-08-19T10:43:42.811458+0000","last_active":"2024-08-19T10:52:49.537692+0000","last_peered":"2024-08-19T10:52:49.537692+0000","last_clean":"2024-08-19T10:52:49.537692+0000","last_became_active":"2024-08-19T10:43:42.777758+0000","last_became_peered":"2024-08-19T10:43:42.777758+0000","last_unstale":"2024-08-19T10:52:49.537692+0000","last_undegraded":"2024-08-19T10:52:49.537692+0000","last_fullsized":"2024-08-19T10:52:49.537692+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:46:54.701371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002082918,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":200,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.413826+0000","last_change":"2024-08-19T10:43:42.776830+0000","last_active":"2024-08-19T10:53:49.413826+0000","last_peered":"2024-08-19T10:53:49.413826+0000","last_clean":"2024-08-19T10:53:49.413826+0000","last_became_active":"2024-08-19T10:43:42.775836+0000","last_became_peered":"2024-08-19T10:43:42.775836+0000","last_unstale":"2024-08-19T10:53:49.413826+0000","last_undegraded":"2024-08-19T10:53:49.413826+0000","last_fullsized":"2024-08-19T10:53:49.413826+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:59:14.529046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010100350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421830+0000","last_change":"2024-08-19T10:43:42.767491+0000","last_active":"2024-08-19T10:54:10.421830+0000","last_peered":"2024-08-19T10:54:10.421830+0000","last_clean":"2024-08-19T10:54:10.421830+0000","last_became_active":"2024-08-19T10:43:42.767132+0000","last_became_peered":"2024-08-19T10:43:42.767132+0000","last_unstale":"2024-08-19T10:54:10.421830+0000","last_undegraded":"2024-08-19T10:54:10.421830+0000","last_fullsized":"2024-08-19T10:54:10.421830+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:04:13.207849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001410375,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":544,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:50.770839+0000","last_change":"2024-08-19T10:43:42.796170+0000","last_active":"2024-08-19T10:52:50.770839+0000","last_peered":"2024-08-19T10:52:50.770839+0000","last_clean":"2024-08-19T10:52:50.770839+0000","last_became_active":"2024-08-19T10:43:42.790684+0000","last_became_peered":"2024-08-19T10:43:42.790684+0000","last_unstale":"2024-08-19T10:52:50.770839+0000","last_undegraded":"2024-08-19T10:52:50.770839+0000","last_fullsized":"2024-08-19T10:52:50.770839+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:48:05.328122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001115312,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":515,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:50.531232+0000","last_change":"2024-08-19T10:43:42.769655+0000","last_active":"2024-08-19T10:52:50.531232+0000","last_peered":"2024-08-19T10:52:50.531232+0000","last_clean":"2024-08-19T10:52:50.531232+0000","last_became_active":"2024-08-19T10:43:42.769275+0000","last_became_peered":"2024-08-19T10:43:42.769275+0000","last_unstale":"2024-08-19T10:52:50.531232+0000","last_undegraded":"2024-08-19T10:52:50.531232+0000","last_fullsized":"2024-08-19T10:52:50.531232+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:23:16.606513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010645120000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":198,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:14.426772+0000","last_change":"2024-08-19T10:43:42.771283+0000","last_active":"2024-08-19T10:54:14.426772+0000","last_peered":"2024-08-19T10:54:14.426772+0000","last_clean":"2024-08-19T10:54:14.426772+0000","last_became_active":"2024-08-19T10:43:42.770419+0000","last_became_peered":"2024-08-19T10:43:42.770419+0000","last_unstale":"2024-08-19T10:54:14.426772+0000","last_undegraded":"2024-08-19T10:54:14.426772+0000","last_fullsized":"2024-08-19T10:54:14.426772+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:39:44.981500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003718322,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.961415+0000","last_change":"2024-08-19T10:43:42.784789+0000","last_active":"2024-08-19T10:52:24.961415+0000","last_peered":"2024-08-19T10:52:24.961415+0000","last_clean":"2024-08-19T10:52:24.961415+0000","last_became_active":"2024-08-19T10:43:42.784451+0000","last_became_peered":"2024-08-19T10:43:42.784451+0000","last_unstale":"2024-08-19T10:52:24.961415+0000","last_undegraded":"2024-08-19T10:52:24.961415+0000","last_fullsized":"2024-08-19T10:52:24.961415+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:19:58.579737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081542699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":480,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:24.419480+0000","last_change":"2024-08-19T10:43:43.885027+0000","last_active":"2024-08-19T10:53:24.419480+0000","last_peered":"2024-08-19T10:53:24.419480+0000","last_clean":"2024-08-19T10:53:24.419480+0000","last_became_active":"2024-08-19T10:43:43.884192+0000","last_became_peered":"2024-08-19T10:43:43.884192+0000","last_unstale":"2024-08-19T10:53:24.419480+0000","last_undegraded":"2024-08-19T10:53:24.419480+0000","last_fullsized":"2024-08-19T10:53:24.419480+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:52:36.680781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021153309999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":6},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607774,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1868579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607773,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1278230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607776,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1278211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607778,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1278230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1278211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640480,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1868579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T10:56:24.258 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T10:56:24.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient: get_monmap_and_config 2024-08-19T10:56:24.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient: build_initial_monmap 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient: monmap: 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:24.377429+0000 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:24.377429+0000 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:24.372 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:24.373 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:24.373 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 1 Processor -- start 2024-08-19T10:56:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 1 -- start start 2024-08-19T10:56:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient: init 2024-08-19T10:56:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient: _add_conns ranks=[1,9,6,2,3,7,10,5,4,0,8] 2024-08-19T10:56:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5a3c09ea20 addr v1:172.21.15.190:6789/0 2024-08-19T10:56:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a3c09d360 0x7f5a3c13c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient(hunting): picked mon.noname-a con 0x7f5a3c09d360 addr v2:172.21.15.89:3300/0 2024-08-19T10:56:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5a3c13ca70 0x7f5a3c13ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient(hunting): picked mon.noname-d con 0x7f5a3c13ca70 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a3c09fbc0 con 0x7f5a3c09ea20 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f5a3c0a3dc0 con 0x7f5a3c09d360 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f5a3c0a51d0 con 0x7f5a3c13ca70 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a44794640 10 monclient(hunting): authenticate will time out at 2034.194750s 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a41d2f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a3c09d360 0x7f5a3c13c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5a3c13ca70 0x7f5a3c13ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a41d2f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5a3c09d360 0x7f5a3c13c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59238/0 (socket says 172.21.15.89:59238) 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5a3c13ca70 0x7f5a3c13ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:44166/0 (socket says 172.21.15.89:44166) 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a41d2f640 1 -- 172.21.15.89:0/1131579740 learned_addr learned my addr 172.21.15.89:0/1131579740 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:24.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a41d2f640 10 monclient(hunting): get_auth_request con 0x7f5a3c09d360 auth_method 0 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a41d2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a41d2f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a41d2f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 10 monclient(hunting): get_auth_request con 0x7f5a3c13ca70 auth_method 0 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a41d2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:24.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a41d2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a41d2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 10 monclient(hunting): handle_auth_done global_id 45609 payload 274 2024-08-19T10:56:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 1 -- 172.21.15.89:0/1131579740 >> v2:172.21.15.89:3300/0 conn(0x7f5a3c09d360 msgr2=0x7f5a3c13c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 1 --2- 172.21.15.89:0/1131579740 >> v2:172.21.15.89:3300/0 conn(0x7f5a3c09d360 0x7f5a3c13c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 1 -- 172.21.15.89:0/1131579740 >> v1:172.21.15.190:6789/0 conn(0x7f5a3c09ea20 legacy=0x7f5a3c09ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 10 monclient: _finish_hunting 0 2024-08-19T10:56:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 1 monclient: found mon.noname-d 2024-08-19T10:56:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:56:24.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.373+0000 7f5a42d31640 1 -- 172.21.15.89:0/1131579740 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c0a3b40 con 0x7f5a3c13ca70 2024-08-19T10:56:24.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a42d31640 1 -- 172.21.15.89:0/1131579740 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5a3c13ca70 msgr2=0x7f5a3c13ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:24.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a42d31640 1 --2- 172.21.15.89:0/1131579740 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5a3c13ca70 0x7f5a3c13ee60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009d90 tx=0x7f5a3401c920 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f4cb8585682bb653 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:24.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 1 -- 172.21.15.89:0/1131579740 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5a340b2070 con 0x7f5a3c13ca70 2024-08-19T10:56:24.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:56:24.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 10 monclient: dump: 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.383 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 10 monclient: _finish_auth 0 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 10 monclient: _check_auth_tickets 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 1 -- 172.21.15.89:0/1131579740 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a340a5bf0 con 0x7f5a3c13ca70 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a44794640 5 monclient: authenticate success, global_id 45609 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a44794640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a44794640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 1 -- 172.21.15.89:0/1131579740 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5a340a5de0 con 0x7f5a3c13ca70 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a44794640 4 set_mon_vals no callback set 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.377+0000 7f5a4152e640 10 monclient: dump: 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.385 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient: shutdown 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 -- 172.21.15.89:0/1131579740 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5a3c13ca70 msgr2=0x7f5a3c13ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 --2- 172.21.15.89:0/1131579740 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5a3c13ca70 0x7f5a3c13ee60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009d90 tx=0x7f5a3401c920 comp rx=0 tx=0).stop 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 -- 172.21.15.89:0/1131579740 shutdown_connections 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 --2- 172.21.15.89:0/1131579740 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f5a3c13ca70 0x7f5a3c13ee60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 --2- 172.21.15.89:0/1131579740 >> v2:172.21.15.89:3300/0 conn(0x7f5a3c09d360 0x7f5a3c13c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 -- 172.21.15.89:0/1131579740 >> 172.21.15.89:0/1131579740 conn(0x7f5a3c0a8f60 msgr2=0x7f5a3c0a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 -- 172.21.15.89:0/1131579740 shutdown_connections 2024-08-19T10:56:24.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 -- 172.21.15.89:0/1131579740 wait complete. 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient: build_initial_monmap 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient: monmap: 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:24.387 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 Processor -- start 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 -- start start 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient: init 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient: _add_conns ranks=[0,8,5,3,7,4,6,2,1] 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c09d360 0x7f5a3c1371b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient(hunting): picked mon.noname-a con 0x7f5a3c09d360 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5a3c09ea20 0x7f5a3c1376f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient(hunting): picked mon.noname-e con 0x7f5a3c09ea20 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:24.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5a3c13ca70 0x7f5a3c11f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient(hunting): picked mon.noname-h con 0x7f5a3c13ca70 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c09f690 con 0x7f5a3c09d360 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f5a3c09f510 con 0x7f5a3c13ca70 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f5a3c141bc0 con 0x7f5a3c09ea20 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a44794640 10 monclient(hunting): authenticate will time out at 2034.201867s 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c09d360 0x7f5a3c1371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a41d2f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5a3c09ea20 0x7f5a3c1376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c09d360 0x7f5a3c1371b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59254/0 (socket says 172.21.15.89:59254) 2024-08-19T10:56:24.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 1 -- 172.21.15.89:0/3169726046 learned_addr learned my addr 172.21.15.89:0/3169726046 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 10 monclient(hunting): get_auth_request con 0x7f5a3c09d360 auth_method 0 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a41d2f640 10 monclient(hunting): get_auth_request con 0x7f5a3c09ea20 auth_method 0 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a41d2f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a41d2f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a41d2f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a41d2f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a41d2f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:24.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a41d2f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 10 monclient(hunting): handle_auth_done global_id 7812 payload 824 2024-08-19T10:56:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 1 -- 172.21.15.89:0/3169726046 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5a3c09ea20 msgr2=0x7f5a3c1376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5a3c09ea20 0x7f5a3c1376f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 1 -- 172.21.15.89:0/3169726046 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5a3c13ca70 msgr2=0x7f5a3c11f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5a3c13ca70 0x7f5a3c11f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 10 monclient: _finish_hunting 0 2024-08-19T10:56:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 1 monclient: found mon.noname-a 2024-08-19T10:56:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:56:24.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.381+0000 7f5a42530640 1 -- 172.21.15.89:0/3169726046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c06d540 con 0x7f5a3c09d360 2024-08-19T10:56:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a42530640 1 -- 172.21.15.89:0/3169726046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c09d360 msgr2=0x7f5a3c1371b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a42530640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c09d360 0x7f5a3c1371b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c480 tx=0x7f5a3800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 1 -- 172.21.15.89:0/3169726046 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5a38011c10 con 0x7f5a3c09d360 2024-08-19T10:56:24.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 10 monclient: dump: 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.394 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 10 monclient: _finish_auth 0 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 1 -- 172.21.15.89:0/3169726046 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a38017070 con 0x7f5a3c09d360 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 1 -- 172.21.15.89:0/3169726046 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5a38011df0 con 0x7f5a3c09d360 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:56:24.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 10 monclient: dump: 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.396 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a33fff640 4 set_mon_vals no callback set 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a44794640 5 monclient: authenticate success, global_id 7812 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a44794640 10 monclient: _renew_subs 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a44794640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a44794640 1 -- 172.21.15.89:0/3169726046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3c06d080 con 0x7f5a3c09d360 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a44794640 10 monclient: _renew_subs 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a44794640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a44794640 1 -- 172.21.15.89:0/3169726046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3c121e10 con 0x7f5a3c09d360 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a44794640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a44794640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a44794640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:24.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a44794640 1 -- 172.21.15.89:0/3169726046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3c09ee00 con 0x7f5a3c09d360 2024-08-19T10:56:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 1 -- 172.21.15.89:0/3169726046 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f5a380117a0 con 0x7f5a3c09d360 2024-08-19T10:56:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5a2003bd30 0x7f5a2003e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a32ffd640 1 -- 172.21.15.89:0/3169726046 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f5a3804f310 con 0x7f5a3c09d360 2024-08-19T10:56:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a41d2f640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5a2003bd30 0x7f5a2003e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.385+0000 7f5a41d2f640 10 monclient: get_auth_request con 0x7f5a2003bd30 auth_method 0 2024-08-19T10:56:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.389+0000 7f5a41d2f640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5a2003bd30 0x7f5a2003e1f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009b50 tx=0x7f5a2c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.389+0000 7f5a32ffd640 1 -- 172.21.15.89:0/3169726046 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f5a38010cf0 con 0x7f5a3c09d360 2024-08-19T10:56:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.389+0000 7f5a32ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:24.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.389+0000 7f5a32ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:24.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.557+0000 7f5a44794640 1 -- 172.21.15.89:0/3169726046 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a3c064590 con 0x7f5a2003bd30 2024-08-19T10:56:24.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.561+0000 7f5a32ffd640 1 -- 172.21.15.89:0/3169726046 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+78517 (secure 0 0 0) 0x7f5a3c064590 con 0x7f5a2003bd30 2024-08-19T10:56:24.562 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:56:24.563 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-08-19T10:56:24.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 -- 172.21.15.89:0/3169726046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5a2003bd30 msgr2=0x7f5a2003e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5a2003bd30 0x7f5a2003e1f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009b50 tx=0x7f5a2c009240 comp rx=0 tx=0).stop 2024-08-19T10:56:24.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 10 monclient: shutdown 2024-08-19T10:56:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 -- 172.21.15.89:0/3169726046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c09d360 msgr2=0x7f5a3c1371b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c09d360 0x7f5a3c1371b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c480 tx=0x7f5a3800c940 comp rx=0 tx=0).stop 2024-08-19T10:56:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 -- 172.21.15.89:0/3169726046 shutdown_connections 2024-08-19T10:56:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5a3c13ca70 0x7f5a3c11f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f5a2003bd30 0x7f5a2003e1f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f5a3c09ea20 0x7f5a3c1376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 --2- 172.21.15.89:0/3169726046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a3c09d360 0x7f5a3c1371b0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 -- 172.21.15.89:0/3169726046 >> 172.21.15.89:0/3169726046 conn(0x7f5a3c0a8f60 msgr2=0x7f5a3c0a6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 -- 172.21.15.89:0/3169726046 shutdown_connections 2024-08-19T10:56:24.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.565+0000 7f5a44794640 1 -- 172.21.15.89:0/3169726046 wait complete. 2024-08-19T10:56:24.580 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":817,"stamp":"2024-08-19T10:56:22.774248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":198040,"kb_used_data":10936,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033000,"statfs":{"total":579820584960,"available":579617792000,"internally_reserved":0,"allocated":11198464,"data_stored":8850040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":191486624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003588"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:09.851422+0000","last_change":"2024-08-19T10:43:42.769648+0000","last_active":"2024-08-19T10:54:09.851422+0000","last_peered":"2024-08-19T10:54:09.851422+0000","last_clean":"2024-08-19T10:54:09.851422+0000","last_became_active":"2024-08-19T10:43:42.769057+0000","last_became_peered":"2024-08-19T10:43:42.769057+0000","last_unstale":"2024-08-19T10:54:09.851422+0000","last_undegraded":"2024-08-19T10:54:09.851422+0000","last_fullsized":"2024-08-19T10:54:09.851422+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:59:18.923960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.531111+0000","last_change":"2024-08-19T10:43:42.776790+0000","last_active":"2024-08-19T10:52:24.531111+0000","last_peered":"2024-08-19T10:52:24.531111+0000","last_clean":"2024-08-19T10:52:24.531111+0000","last_became_active":"2024-08-19T10:43:42.775930+0000","last_became_peered":"2024-08-19T10:43:42.775930+0000","last_unstale":"2024-08-19T10:52:24.531111+0000","last_undegraded":"2024-08-19T10:52:24.531111+0000","last_fullsized":"2024-08-19T10:52:24.531111+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:00:49.264074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:14.852152+0000","last_change":"2024-08-19T10:43:42.776699+0000","last_active":"2024-08-19T10:54:14.852152+0000","last_peered":"2024-08-19T10:54:14.852152+0000","last_clean":"2024-08-19T10:54:14.852152+0000","last_became_active":"2024-08-19T10:43:42.775384+0000","last_became_peered":"2024-08-19T10:43:42.775384+0000","last_unstale":"2024-08-19T10:54:14.852152+0000","last_undegraded":"2024-08-19T10:54:14.852152+0000","last_fullsized":"2024-08-19T10:54:14.852152+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:11:44.821797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.531052+0000","last_change":"2024-08-19T10:43:42.770075+0000","last_active":"2024-08-19T10:52:24.531052+0000","last_peered":"2024-08-19T10:52:24.531052+0000","last_clean":"2024-08-19T10:52:24.531052+0000","last_became_active":"2024-08-19T10:43:42.769225+0000","last_became_peered":"2024-08-19T10:43:42.769225+0000","last_unstale":"2024-08-19T10:52:24.531052+0000","last_undegraded":"2024-08-19T10:52:24.531052+0000","last_fullsized":"2024-08-19T10:52:24.531052+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:14:39.665839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":198,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.534076+0000","last_change":"2024-08-19T10:43:42.771832+0000","last_active":"2024-08-19T10:52:24.534076+0000","last_peered":"2024-08-19T10:52:24.534076+0000","last_clean":"2024-08-19T10:52:24.534076+0000","last_became_active":"2024-08-19T10:43:42.770907+0000","last_became_peered":"2024-08-19T10:43:42.770907+0000","last_unstale":"2024-08-19T10:52:24.534076+0000","last_undegraded":"2024-08-19T10:52:24.534076+0000","last_fullsized":"2024-08-19T10:52:24.534076+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:51:29.432515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":202,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:15.416700+0000","last_change":"2024-08-19T10:43:42.789929+0000","last_active":"2024-08-19T10:54:15.416700+0000","last_peered":"2024-08-19T10:54:15.416700+0000","last_clean":"2024-08-19T10:54:15.416700+0000","last_became_active":"2024-08-19T10:43:42.789563+0000","last_became_peered":"2024-08-19T10:43:42.789563+0000","last_unstale":"2024-08-19T10:54:15.416700+0000","last_undegraded":"2024-08-19T10:54:15.416700+0000","last_fullsized":"2024-08-19T10:54:15.416700+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:51:07.959247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":309,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:05.773040+0000","last_change":"2024-08-19T10:43:42.788997+0000","last_active":"2024-08-19T10:53:05.773040+0000","last_peered":"2024-08-19T10:53:05.773040+0000","last_clean":"2024-08-19T10:53:05.773040+0000","last_became_active":"2024-08-19T10:43:42.784411+0000","last_became_peered":"2024-08-19T10:43:42.784411+0000","last_unstale":"2024-08-19T10:53:05.773040+0000","last_undegraded":"2024-08-19T10:53:05.773040+0000","last_fullsized":"2024-08-19T10:53:05.773040+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:44:43.826178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421871+0000","last_change":"2024-08-19T10:43:42.785782+0000","last_active":"2024-08-19T10:54:10.421871+0000","last_peered":"2024-08-19T10:54:10.421871+0000","last_clean":"2024-08-19T10:54:10.421871+0000","last_became_active":"2024-08-19T10:43:42.785474+0000","last_became_peered":"2024-08-19T10:43:42.785474+0000","last_unstale":"2024-08-19T10:54:10.421871+0000","last_undegraded":"2024-08-19T10:54:10.421871+0000","last_fullsized":"2024-08-19T10:54:10.421871+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:37:33.828996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.413864+0000","last_change":"2024-08-19T10:43:42.786904+0000","last_active":"2024-08-19T10:53:49.413864+0000","last_peered":"2024-08-19T10:53:49.413864+0000","last_clean":"2024-08-19T10:53:49.413864+0000","last_became_active":"2024-08-19T10:43:42.786027+0000","last_became_peered":"2024-08-19T10:43:42.786027+0000","last_unstale":"2024-08-19T10:53:49.413864+0000","last_undegraded":"2024-08-19T10:53:49.413864+0000","last_fullsized":"2024-08-19T10:53:49.413864+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:18:23.651707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":316,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:40.769323+0000","last_change":"2024-08-19T10:43:42.785108+0000","last_active":"2024-08-19T10:52:40.769323+0000","last_peered":"2024-08-19T10:52:40.769323+0000","last_clean":"2024-08-19T10:52:40.769323+0000","last_became_active":"2024-08-19T10:43:42.784268+0000","last_became_peered":"2024-08-19T10:43:42.784268+0000","last_unstale":"2024-08-19T10:52:40.769323+0000","last_undegraded":"2024-08-19T10:52:40.769323+0000","last_fullsized":"2024-08-19T10:52:40.769323+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:40:36.868319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":318,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:40.769280+0000","last_change":"2024-08-19T10:43:42.784728+0000","last_active":"2024-08-19T10:52:40.769280+0000","last_peered":"2024-08-19T10:52:40.769280+0000","last_clean":"2024-08-19T10:52:40.769280+0000","last_became_active":"2024-08-19T10:43:42.783709+0000","last_became_peered":"2024-08-19T10:43:42.783709+0000","last_unstale":"2024-08-19T10:52:40.769280+0000","last_undegraded":"2024-08-19T10:52:40.769280+0000","last_fullsized":"2024-08-19T10:52:40.769280+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:30:15.771986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421853+0000","last_change":"2024-08-19T10:43:42.767704+0000","last_active":"2024-08-19T10:54:10.421853+0000","last_peered":"2024-08-19T10:54:10.421853+0000","last_clean":"2024-08-19T10:54:10.421853+0000","last_became_active":"2024-08-19T10:43:42.767225+0000","last_became_peered":"2024-08-19T10:43:42.767225+0000","last_unstale":"2024-08-19T10:54:10.421853+0000","last_undegraded":"2024-08-19T10:54:10.421853+0000","last_fullsized":"2024-08-19T10:54:10.421853+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:29:35.198111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.415852+0000","last_change":"2024-08-19T10:43:43.768263+0000","last_active":"2024-08-19T10:54:10.415852+0000","last_peered":"2024-08-19T10:54:10.415852+0000","last_clean":"2024-08-19T10:54:10.415852+0000","last_became_active":"2024-08-19T10:43:43.767866+0000","last_became_peered":"2024-08-19T10:43:43.767866+0000","last_unstale":"2024-08-19T10:54:10.415852+0000","last_undegraded":"2024-08-19T10:54:10.415852+0000","last_fullsized":"2024-08-19T10:54:10.415852+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:12:13.038948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":205,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421793+0000","last_change":"2024-08-19T10:43:43.763907+0000","last_active":"2024-08-19T10:54:10.421793+0000","last_peered":"2024-08-19T10:54:10.421793+0000","last_clean":"2024-08-19T10:54:10.421793+0000","last_became_active":"2024-08-19T10:43:43.763033+0000","last_became_peered":"2024-08-19T10:43:43.763033+0000","last_unstale":"2024-08-19T10:54:10.421793+0000","last_undegraded":"2024-08-19T10:54:10.421793+0000","last_fullsized":"2024-08-19T10:54:10.421793+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:15:08.848730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":235,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:54.965756+0000","last_change":"2024-08-19T10:43:43.773879+0000","last_active":"2024-08-19T10:52:54.965756+0000","last_peered":"2024-08-19T10:52:54.965756+0000","last_clean":"2024-08-19T10:52:54.965756+0000","last_became_active":"2024-08-19T10:43:43.773098+0000","last_became_peered":"2024-08-19T10:43:43.773098+0000","last_unstale":"2024-08-19T10:52:54.965756+0000","last_undegraded":"2024-08-19T10:52:54.965756+0000","last_fullsized":"2024-08-19T10:52:54.965756+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:05:31.013999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":202,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:50.412824+0000","last_change":"2024-08-19T10:43:42.776431+0000","last_active":"2024-08-19T10:53:50.412824+0000","last_peered":"2024-08-19T10:53:50.412824+0000","last_clean":"2024-08-19T10:53:50.412824+0000","last_became_active":"2024-08-19T10:43:42.776033+0000","last_became_peered":"2024-08-19T10:43:42.776033+0000","last_unstale":"2024-08-19T10:53:50.412824+0000","last_undegraded":"2024-08-19T10:53:50.412824+0000","last_fullsized":"2024-08-19T10:53:50.412824+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:10:47.358711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.423024+0000","last_change":"2024-08-19T10:43:42.807228+0000","last_active":"2024-08-19T10:53:49.423024+0000","last_peered":"2024-08-19T10:53:49.423024+0000","last_clean":"2024-08-19T10:53:49.423024+0000","last_became_active":"2024-08-19T10:43:42.775224+0000","last_became_peered":"2024-08-19T10:43:42.775224+0000","last_unstale":"2024-08-19T10:53:49.423024+0000","last_undegraded":"2024-08-19T10:53:49.423024+0000","last_fullsized":"2024-08-19T10:53:49.423024+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_clean_scrub_stamp":"2024-08-19T10:38:04.611102+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:21:32.163214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":491,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:49.537692+0000","last_change":"2024-08-19T10:43:42.811458+0000","last_active":"2024-08-19T10:52:49.537692+0000","last_peered":"2024-08-19T10:52:49.537692+0000","last_clean":"2024-08-19T10:52:49.537692+0000","last_became_active":"2024-08-19T10:43:42.777758+0000","last_became_peered":"2024-08-19T10:43:42.777758+0000","last_unstale":"2024-08-19T10:52:49.537692+0000","last_undegraded":"2024-08-19T10:52:49.537692+0000","last_fullsized":"2024-08-19T10:52:49.537692+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:46:54.701371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002082918,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":200,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.413826+0000","last_change":"2024-08-19T10:43:42.776830+0000","last_active":"2024-08-19T10:53:49.413826+0000","last_peered":"2024-08-19T10:53:49.413826+0000","last_clean":"2024-08-19T10:53:49.413826+0000","last_became_active":"2024-08-19T10:43:42.775836+0000","last_became_peered":"2024-08-19T10:43:42.775836+0000","last_unstale":"2024-08-19T10:53:49.413826+0000","last_undegraded":"2024-08-19T10:53:49.413826+0000","last_fullsized":"2024-08-19T10:53:49.413826+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:59:14.529046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010100350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421830+0000","last_change":"2024-08-19T10:43:42.767491+0000","last_active":"2024-08-19T10:54:10.421830+0000","last_peered":"2024-08-19T10:54:10.421830+0000","last_clean":"2024-08-19T10:54:10.421830+0000","last_became_active":"2024-08-19T10:43:42.767132+0000","last_became_peered":"2024-08-19T10:43:42.767132+0000","last_unstale":"2024-08-19T10:54:10.421830+0000","last_undegraded":"2024-08-19T10:54:10.421830+0000","last_fullsized":"2024-08-19T10:54:10.421830+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:04:13.207849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001410375,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":544,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:50.770839+0000","last_change":"2024-08-19T10:43:42.796170+0000","last_active":"2024-08-19T10:52:50.770839+0000","last_peered":"2024-08-19T10:52:50.770839+0000","last_clean":"2024-08-19T10:52:50.770839+0000","last_became_active":"2024-08-19T10:43:42.790684+0000","last_became_peered":"2024-08-19T10:43:42.790684+0000","last_unstale":"2024-08-19T10:52:50.770839+0000","last_undegraded":"2024-08-19T10:52:50.770839+0000","last_fullsized":"2024-08-19T10:52:50.770839+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:48:05.328122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001115312,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":515,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:50.531232+0000","last_change":"2024-08-19T10:43:42.769655+0000","last_active":"2024-08-19T10:52:50.531232+0000","last_peered":"2024-08-19T10:52:50.531232+0000","last_clean":"2024-08-19T10:52:50.531232+0000","last_became_active":"2024-08-19T10:43:42.769275+0000","last_became_peered":"2024-08-19T10:43:42.769275+0000","last_unstale":"2024-08-19T10:52:50.531232+0000","last_undegraded":"2024-08-19T10:52:50.531232+0000","last_fullsized":"2024-08-19T10:52:50.531232+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:23:16.606513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010645120000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":198,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:14.426772+0000","last_change":"2024-08-19T10:43:42.771283+0000","last_active":"2024-08-19T10:54:14.426772+0000","last_peered":"2024-08-19T10:54:14.426772+0000","last_clean":"2024-08-19T10:54:14.426772+0000","last_became_active":"2024-08-19T10:43:42.770419+0000","last_became_peered":"2024-08-19T10:43:42.770419+0000","last_unstale":"2024-08-19T10:54:14.426772+0000","last_undegraded":"2024-08-19T10:54:14.426772+0000","last_fullsized":"2024-08-19T10:54:14.426772+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:39:44.981500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003718322,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.961415+0000","last_change":"2024-08-19T10:43:42.784789+0000","last_active":"2024-08-19T10:52:24.961415+0000","last_peered":"2024-08-19T10:52:24.961415+0000","last_clean":"2024-08-19T10:52:24.961415+0000","last_became_active":"2024-08-19T10:43:42.784451+0000","last_became_peered":"2024-08-19T10:43:42.784451+0000","last_unstale":"2024-08-19T10:52:24.961415+0000","last_undegraded":"2024-08-19T10:52:24.961415+0000","last_fullsized":"2024-08-19T10:52:24.961415+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:19:58.579737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081542699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":480,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:24.419480+0000","last_change":"2024-08-19T10:43:43.885027+0000","last_active":"2024-08-19T10:53:24.419480+0000","last_peered":"2024-08-19T10:53:24.419480+0000","last_clean":"2024-08-19T10:53:24.419480+0000","last_became_active":"2024-08-19T10:43:43.884192+0000","last_became_peered":"2024-08-19T10:43:43.884192+0000","last_unstale":"2024-08-19T10:53:24.419480+0000","last_undegraded":"2024-08-19T10:53:24.419480+0000","last_fullsized":"2024-08-19T10:53:24.419480+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:52:36.680781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021153309999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":6},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607774,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1868579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607773,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1278230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607776,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1278211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607778,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1278230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640478,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1278211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640480,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1868579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T10:56:24.581 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-19T10:56:24.582 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T10:56:24.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.685+0000 7f2549ed7640 10 monclient: get_monmap_and_config 2024-08-19T10:56:24.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.685+0000 7f2549ed7640 10 monclient: build_initial_monmap 2024-08-19T10:56:24.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.685+0000 7f2549ed7640 10 monclient: monmap: 2024-08-19T10:56:24.686 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:24.692334+0000 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:24.692334+0000 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:24.687 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:24.688 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:24.688 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:24.688 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:24.688 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 1 Processor -- start 2024-08-19T10:56:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 1 -- start start 2024-08-19T10:56:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 10 monclient: init 2024-08-19T10:56:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 10 monclient: _add_conns ranks=[7,8,3,6,5,0,10,9,4,1,2] 2024-08-19T10:56:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f254406c930 0x7f25440a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 10 monclient(hunting): picked mon.noname-i con 0x7f254406c930 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25440a4050 0x7f25440a4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 10 monclient(hunting): picked mon.noname-e con 0x7f25440a4050 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254413d980 0x7f254413fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 10 monclient(hunting): picked mon.noname-g con 0x7f254413d980 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f25440a1240 con 0x7f254413d980 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f2544074400 con 0x7f254406c930 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f254406d650 con 0x7f25440a4050 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2549ed7640 10 monclient(hunting): authenticate will time out at 2034.509243s 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25440a4050 0x7f25440a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25440a4050 0x7f25440a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50238/0 (socket says 172.21.15.89:50238) 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f25437fe640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f254406c930 msgr2=0x7f25440a3b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 1 -- 172.21.15.89:0/1374765168 learned_addr learned my addr 172.21.15.89:0/1374765168 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:24.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 10 monclient(hunting): get_auth_request con 0x7f25440a4050 auth_method 0 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f25437fe640 1 --2- 172.21.15.89:0/1374765168 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f254406c930 0x7f25440a3b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f25427fc640 1 -- 172.21.15.89:0/1374765168 DispatchQueue::entry inject delay of 0.002000 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2543fff640 1 --2- 172.21.15.89:0/1374765168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254413d980 0x7f254413fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 10 monclient(hunting): handle_auth_done global_id 114371 payload 274 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 1 -- 172.21.15.89:0/1374765168 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f254406c930 msgr2=0x7f25440a3b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 1 --2- 172.21.15.89:0/1374765168 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f254406c930 0x7f25440a3b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 1 -- 172.21.15.89:0/1374765168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254413d980 msgr2=0x7f254413fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:24.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 1 --2- 172.21.15.89:0/1374765168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254413d980 0x7f254413fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:56:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 1 monclient: found mon.noname-e 2024-08-19T10:56:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2542ffd640 1 -- 172.21.15.89:0/1374765168 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25440a4df0 con 0x7f25440a4050 2024-08-19T10:56:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2543fff640 10 monclient: get_auth_request con 0x7f254413d980 auth_method 0 2024-08-19T10:56:24.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.689+0000 7f2543fff640 1 --2- 172.21.15.89:0/1374765168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254413d980 0x7f254413fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:24.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2542ffd640 1 -- 172.21.15.89:0/1374765168 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25440a4050 msgr2=0x7f25440a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:24.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2542ffd640 1 --2- 172.21.15.89:0/1374765168 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25440a4050 0x7f25440a4430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2538009900 tx=0x7f253801c440 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f0df41ecdf491eb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:24.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 1 -- 172.21.15.89:0/1374765168 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f25380b0070 con 0x7f25440a4050 2024-08-19T10:56:24.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:24.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 10 monclient: dump: 2024-08-19T10:56:24.697 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.697 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.697 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.697 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.698 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.698 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.698 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.698 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.698 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.698 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.698 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.698 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.698 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.698 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.699 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.699 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 10 monclient: _finish_auth 0 2024-08-19T10:56:24.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 10 monclient: _check_auth_tickets 2024-08-19T10:56:24.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:24.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 1 -- 172.21.15.89:0/1374765168 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25380aa810 con 0x7f25440a4050 2024-08-19T10:56:24.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:24.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 1 -- 172.21.15.89:0/1374765168 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f25380aae30 con 0x7f25440a4050 2024-08-19T10:56:24.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 5 monclient: authenticate success, global_id 114371 2024-08-19T10:56:24.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:24.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 4 set_mon_vals no callback set 2024-08-19T10:56:24.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:24.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f25427fc640 10 monclient: dump: 2024-08-19T10:56:24.700 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.700 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.700 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.700 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.700 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.700 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.701 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.701 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.701 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.701 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.701 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.701 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.701 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.701 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.701 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.701 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 10 monclient: shutdown 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 1 -- 172.21.15.89:0/1374765168 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25440a4050 msgr2=0x7f25440a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 1 --2- 172.21.15.89:0/1374765168 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25440a4050 0x7f25440a4430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2538009900 tx=0x7f253801c440 comp rx=0 tx=0).stop 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 1 -- 172.21.15.89:0/1374765168 shutdown_connections 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 1 --2- 172.21.15.89:0/1374765168 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f25440a4050 0x7f25440a4430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 1 --2- 172.21.15.89:0/1374765168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f254413d980 0x7f254413fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 1 --2- 172.21.15.89:0/1374765168 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f254406c930 0x7f25440a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 1 -- 172.21.15.89:0/1374765168 >> 172.21.15.89:0/1374765168 conn(0x7f254406de40 msgr2=0x7f254406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 1 -- 172.21.15.89:0/1374765168 shutdown_connections 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 1 -- 172.21.15.89:0/1374765168 wait complete. 2024-08-19T10:56:24.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 10 monclient: build_initial_monmap 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 10 monclient: monmap: 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:24.703 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 1 Processor -- start 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 1 -- start start 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.693+0000 7f2549ed7640 10 monclient: init 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient: _add_conns ranks=[8,3,5,1,2,4,0,7,6] 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f254406c930 0x7f25441d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient(hunting): picked mon.noname-e con 0x7f254406c930 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25440a4050 0x7f25441389e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient(hunting): picked mon.noname-g con 0x7f25440a4050 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f254413d980 0x7f254413a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient(hunting): picked mon.noname-h con 0x7f254413d980 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2544142c50 con 0x7f25440a4050 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f2544142ad0 con 0x7f254413d980 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f2544142dd0 con 0x7f254406c930 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient(hunting): authenticate will time out at 2034.516092s 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f254406c930 0x7f25441d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f254406c930 0x7f25441d0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50242/0 (socket says 172.21.15.89:50242) 2024-08-19T10:56:24.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 1 -- 172.21.15.89:0/3001087872 learned_addr learned my addr 172.21.15.89:0/3001087872 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 10 monclient(hunting): get_auth_request con 0x7f254406c930 auth_method 0 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2543fff640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f254413d980 0x7f254413a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2542ffd640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25440a4050 0x7f25441389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 10 monclient(hunting): handle_auth_done global_id 114380 payload 824 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 1 -- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f254413d980 msgr2=0x7f254413a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f254413d980 0x7f254413a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 1 -- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25440a4050 msgr2=0x7f25441389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25440a4050 0x7f25441389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 10 monclient: _finish_hunting 0 2024-08-19T10:56:24.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 1 monclient: found mon.noname-e 2024-08-19T10:56:24.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:24.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:24.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 1 -- 172.21.15.89:0/3001087872 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2544124380 con 0x7f254406c930 2024-08-19T10:56:24.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2542ffd640 10 monclient: get_auth_request con 0x7f25440a4050 auth_method 0 2024-08-19T10:56:24.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2542ffd640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25440a4050 0x7f25441389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:24.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 1 -- 172.21.15.89:0/3001087872 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f254406c930 msgr2=0x7f25441d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:24.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f25437fe640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f254406c930 0x7f25441d0d60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f253400b530 tx=0x7f253400b9f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:24.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 1 -- 172.21.15.89:0/3001087872 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f2534013020 con 0x7f254406c930 2024-08-19T10:56:24.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:24.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 10 monclient: dump: 2024-08-19T10:56:24.710 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.710 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.710 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.711 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 10 monclient: _finish_auth 0 2024-08-19T10:56:24.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 10 monclient: _check_auth_tickets 2024-08-19T10:56:24.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:24.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 1 -- 172.21.15.89:0/3001087872 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2534003de0 con 0x7f254406c930 2024-08-19T10:56:24.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:24.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 1 -- 172.21.15.89:0/3001087872 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f253400f400 con 0x7f254406c930 2024-08-19T10:56:24.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 10 monclient: dump: 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:24.714 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2541ffb640 4 set_mon_vals no callback set 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 5 monclient: authenticate success, global_id 114380 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient: _renew_subs 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 1 -- 172.21.15.89:0/3001087872 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254406e4d0 con 0x7f254406c930 2024-08-19T10:56:24.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient: _renew_subs 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 1 -- 172.21.15.89:0/3001087872 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f25441240a0 con 0x7f254406c930 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f2549ed7640 1 -- 172.21.15.89:0/3001087872 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25440a4430 con 0x7f254406c930 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.697+0000 7f252bfff640 1 -- 172.21.15.89:0/3001087872 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f2534016020 con 0x7f254406c930 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.701+0000 7f252bfff640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f253003be30 0x7f253003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.701+0000 7f252bfff640 1 -- 172.21.15.89:0/3001087872 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f253404f310 con 0x7f254406c930 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.701+0000 7f2542ffd640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f253003be30 0x7f253003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.701+0000 7f2542ffd640 10 monclient: get_auth_request con 0x7f253003be30 auth_method 0 2024-08-19T10:56:24.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.701+0000 7f2542ffd640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f253003be30 0x7f253003e2f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f25380ab000 tx=0x7f25380a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:24.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.705+0000 7f252bfff640 1 -- 172.21.15.89:0/3001087872 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f25440a4430 con 0x7f254406c930 2024-08-19T10:56:24.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.705+0000 7f252bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:24.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.705+0000 7f252bfff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:24.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.857+0000 7f2549ed7640 1 -- 172.21.15.89:0/3001087872 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25440a2120 con 0x7f253003be30 2024-08-19T10:56:24.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.861+0000 7f252bfff640 1 -- 172.21.15.89:0/3001087872 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+78517 (secure 0 0 0) 0x7f25440a2120 con 0x7f253003be30 2024-08-19T10:56:24.861 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:56:24.862 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-08-19T10:56:24.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.861+0000 7f2549ed7640 1 -- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f253003be30 msgr2=0x7f253003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.861+0000 7f2549ed7640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f253003be30 0x7f253003e2f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f25380ab000 tx=0x7f25380a6000 comp rx=0 tx=0).stop 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 10 monclient: shutdown 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 1 -- 172.21.15.89:0/3001087872 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f254406c930 msgr2=0x7f25441d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f254406c930 0x7f25441d0d60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f253400b530 tx=0x7f253400b9f0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 1 -- 172.21.15.89:0/3001087872 shutdown_connections 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f254413d980 0x7f254413a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25440a4050 0x7f25441389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f253003be30 0x7f253003e2f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 1 --2- 172.21.15.89:0/3001087872 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f254406c930 0x7f25441d0d60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 1 -- 172.21.15.89:0/3001087872 >> 172.21.15.89:0/3001087872 conn(0x7f254406de40 msgr2=0x7f2544072cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:24.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 1 -- 172.21.15.89:0/3001087872 shutdown_connections 2024-08-19T10:56:24.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:24.865+0000 7f2549ed7640 1 -- 172.21.15.89:0/3001087872 wait complete. 2024-08-19T10:56:24.878 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":818,"stamp":"2024-08-19T10:56:24.774837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":198040,"kb_used_data":10936,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033000,"statfs":{"total":579820584960,"available":579617792000,"internally_reserved":0,"allocated":11198464,"data_stored":8850040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":191486624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003489"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:09.851422+0000","last_change":"2024-08-19T10:43:42.769648+0000","last_active":"2024-08-19T10:54:09.851422+0000","last_peered":"2024-08-19T10:54:09.851422+0000","last_clean":"2024-08-19T10:54:09.851422+0000","last_became_active":"2024-08-19T10:43:42.769057+0000","last_became_peered":"2024-08-19T10:43:42.769057+0000","last_unstale":"2024-08-19T10:54:09.851422+0000","last_undegraded":"2024-08-19T10:54:09.851422+0000","last_fullsized":"2024-08-19T10:54:09.851422+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:59:18.923960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.531111+0000","last_change":"2024-08-19T10:43:42.776790+0000","last_active":"2024-08-19T10:52:24.531111+0000","last_peered":"2024-08-19T10:52:24.531111+0000","last_clean":"2024-08-19T10:52:24.531111+0000","last_became_active":"2024-08-19T10:43:42.775930+0000","last_became_peered":"2024-08-19T10:43:42.775930+0000","last_unstale":"2024-08-19T10:52:24.531111+0000","last_undegraded":"2024-08-19T10:52:24.531111+0000","last_fullsized":"2024-08-19T10:52:24.531111+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:00:49.264074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:14.852152+0000","last_change":"2024-08-19T10:43:42.776699+0000","last_active":"2024-08-19T10:54:14.852152+0000","last_peered":"2024-08-19T10:54:14.852152+0000","last_clean":"2024-08-19T10:54:14.852152+0000","last_became_active":"2024-08-19T10:43:42.775384+0000","last_became_peered":"2024-08-19T10:43:42.775384+0000","last_unstale":"2024-08-19T10:54:14.852152+0000","last_undegraded":"2024-08-19T10:54:14.852152+0000","last_fullsized":"2024-08-19T10:54:14.852152+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:11:44.821797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.531052+0000","last_change":"2024-08-19T10:43:42.770075+0000","last_active":"2024-08-19T10:52:24.531052+0000","last_peered":"2024-08-19T10:52:24.531052+0000","last_clean":"2024-08-19T10:52:24.531052+0000","last_became_active":"2024-08-19T10:43:42.769225+0000","last_became_peered":"2024-08-19T10:43:42.769225+0000","last_unstale":"2024-08-19T10:52:24.531052+0000","last_undegraded":"2024-08-19T10:52:24.531052+0000","last_fullsized":"2024-08-19T10:52:24.531052+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:14:39.665839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":198,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.534076+0000","last_change":"2024-08-19T10:43:42.771832+0000","last_active":"2024-08-19T10:52:24.534076+0000","last_peered":"2024-08-19T10:52:24.534076+0000","last_clean":"2024-08-19T10:52:24.534076+0000","last_became_active":"2024-08-19T10:43:42.770907+0000","last_became_peered":"2024-08-19T10:43:42.770907+0000","last_unstale":"2024-08-19T10:52:24.534076+0000","last_undegraded":"2024-08-19T10:52:24.534076+0000","last_fullsized":"2024-08-19T10:52:24.534076+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:51:29.432515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":202,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:15.416700+0000","last_change":"2024-08-19T10:43:42.789929+0000","last_active":"2024-08-19T10:54:15.416700+0000","last_peered":"2024-08-19T10:54:15.416700+0000","last_clean":"2024-08-19T10:54:15.416700+0000","last_became_active":"2024-08-19T10:43:42.789563+0000","last_became_peered":"2024-08-19T10:43:42.789563+0000","last_unstale":"2024-08-19T10:54:15.416700+0000","last_undegraded":"2024-08-19T10:54:15.416700+0000","last_fullsized":"2024-08-19T10:54:15.416700+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:51:07.959247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":309,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:05.773040+0000","last_change":"2024-08-19T10:43:42.788997+0000","last_active":"2024-08-19T10:53:05.773040+0000","last_peered":"2024-08-19T10:53:05.773040+0000","last_clean":"2024-08-19T10:53:05.773040+0000","last_became_active":"2024-08-19T10:43:42.784411+0000","last_became_peered":"2024-08-19T10:43:42.784411+0000","last_unstale":"2024-08-19T10:53:05.773040+0000","last_undegraded":"2024-08-19T10:53:05.773040+0000","last_fullsized":"2024-08-19T10:53:05.773040+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:44:43.826178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421871+0000","last_change":"2024-08-19T10:43:42.785782+0000","last_active":"2024-08-19T10:54:10.421871+0000","last_peered":"2024-08-19T10:54:10.421871+0000","last_clean":"2024-08-19T10:54:10.421871+0000","last_became_active":"2024-08-19T10:43:42.785474+0000","last_became_peered":"2024-08-19T10:43:42.785474+0000","last_unstale":"2024-08-19T10:54:10.421871+0000","last_undegraded":"2024-08-19T10:54:10.421871+0000","last_fullsized":"2024-08-19T10:54:10.421871+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:37:33.828996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.413864+0000","last_change":"2024-08-19T10:43:42.786904+0000","last_active":"2024-08-19T10:53:49.413864+0000","last_peered":"2024-08-19T10:53:49.413864+0000","last_clean":"2024-08-19T10:53:49.413864+0000","last_became_active":"2024-08-19T10:43:42.786027+0000","last_became_peered":"2024-08-19T10:43:42.786027+0000","last_unstale":"2024-08-19T10:53:49.413864+0000","last_undegraded":"2024-08-19T10:53:49.413864+0000","last_fullsized":"2024-08-19T10:53:49.413864+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:18:23.651707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":316,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:40.769323+0000","last_change":"2024-08-19T10:43:42.785108+0000","last_active":"2024-08-19T10:52:40.769323+0000","last_peered":"2024-08-19T10:52:40.769323+0000","last_clean":"2024-08-19T10:52:40.769323+0000","last_became_active":"2024-08-19T10:43:42.784268+0000","last_became_peered":"2024-08-19T10:43:42.784268+0000","last_unstale":"2024-08-19T10:52:40.769323+0000","last_undegraded":"2024-08-19T10:52:40.769323+0000","last_fullsized":"2024-08-19T10:52:40.769323+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:40:36.868319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":318,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:40.769280+0000","last_change":"2024-08-19T10:43:42.784728+0000","last_active":"2024-08-19T10:52:40.769280+0000","last_peered":"2024-08-19T10:52:40.769280+0000","last_clean":"2024-08-19T10:52:40.769280+0000","last_became_active":"2024-08-19T10:43:42.783709+0000","last_became_peered":"2024-08-19T10:43:42.783709+0000","last_unstale":"2024-08-19T10:52:40.769280+0000","last_undegraded":"2024-08-19T10:52:40.769280+0000","last_fullsized":"2024-08-19T10:52:40.769280+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:30:15.771986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421853+0000","last_change":"2024-08-19T10:43:42.767704+0000","last_active":"2024-08-19T10:54:10.421853+0000","last_peered":"2024-08-19T10:54:10.421853+0000","last_clean":"2024-08-19T10:54:10.421853+0000","last_became_active":"2024-08-19T10:43:42.767225+0000","last_became_peered":"2024-08-19T10:43:42.767225+0000","last_unstale":"2024-08-19T10:54:10.421853+0000","last_undegraded":"2024-08-19T10:54:10.421853+0000","last_fullsized":"2024-08-19T10:54:10.421853+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:29:35.198111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.415852+0000","last_change":"2024-08-19T10:43:43.768263+0000","last_active":"2024-08-19T10:54:10.415852+0000","last_peered":"2024-08-19T10:54:10.415852+0000","last_clean":"2024-08-19T10:54:10.415852+0000","last_became_active":"2024-08-19T10:43:43.767866+0000","last_became_peered":"2024-08-19T10:43:43.767866+0000","last_unstale":"2024-08-19T10:54:10.415852+0000","last_undegraded":"2024-08-19T10:54:10.415852+0000","last_fullsized":"2024-08-19T10:54:10.415852+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:12:13.038948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":205,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421793+0000","last_change":"2024-08-19T10:43:43.763907+0000","last_active":"2024-08-19T10:54:10.421793+0000","last_peered":"2024-08-19T10:54:10.421793+0000","last_clean":"2024-08-19T10:54:10.421793+0000","last_became_active":"2024-08-19T10:43:43.763033+0000","last_became_peered":"2024-08-19T10:43:43.763033+0000","last_unstale":"2024-08-19T10:54:10.421793+0000","last_undegraded":"2024-08-19T10:54:10.421793+0000","last_fullsized":"2024-08-19T10:54:10.421793+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:15:08.848730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":235,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:54.965756+0000","last_change":"2024-08-19T10:43:43.773879+0000","last_active":"2024-08-19T10:52:54.965756+0000","last_peered":"2024-08-19T10:52:54.965756+0000","last_clean":"2024-08-19T10:52:54.965756+0000","last_became_active":"2024-08-19T10:43:43.773098+0000","last_became_peered":"2024-08-19T10:43:43.773098+0000","last_unstale":"2024-08-19T10:52:54.965756+0000","last_undegraded":"2024-08-19T10:52:54.965756+0000","last_fullsized":"2024-08-19T10:52:54.965756+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:05:31.013999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":202,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:50.412824+0000","last_change":"2024-08-19T10:43:42.776431+0000","last_active":"2024-08-19T10:53:50.412824+0000","last_peered":"2024-08-19T10:53:50.412824+0000","last_clean":"2024-08-19T10:53:50.412824+0000","last_became_active":"2024-08-19T10:43:42.776033+0000","last_became_peered":"2024-08-19T10:43:42.776033+0000","last_unstale":"2024-08-19T10:53:50.412824+0000","last_undegraded":"2024-08-19T10:53:50.412824+0000","last_fullsized":"2024-08-19T10:53:50.412824+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:10:47.358711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.423024+0000","last_change":"2024-08-19T10:43:42.807228+0000","last_active":"2024-08-19T10:53:49.423024+0000","last_peered":"2024-08-19T10:53:49.423024+0000","last_clean":"2024-08-19T10:53:49.423024+0000","last_became_active":"2024-08-19T10:43:42.775224+0000","last_became_peered":"2024-08-19T10:43:42.775224+0000","last_unstale":"2024-08-19T10:53:49.423024+0000","last_undegraded":"2024-08-19T10:53:49.423024+0000","last_fullsized":"2024-08-19T10:53:49.423024+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_clean_scrub_stamp":"2024-08-19T10:38:04.611102+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:21:32.163214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":491,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:49.537692+0000","last_change":"2024-08-19T10:43:42.811458+0000","last_active":"2024-08-19T10:52:49.537692+0000","last_peered":"2024-08-19T10:52:49.537692+0000","last_clean":"2024-08-19T10:52:49.537692+0000","last_became_active":"2024-08-19T10:43:42.777758+0000","last_became_peered":"2024-08-19T10:43:42.777758+0000","last_unstale":"2024-08-19T10:52:49.537692+0000","last_undegraded":"2024-08-19T10:52:49.537692+0000","last_fullsized":"2024-08-19T10:52:49.537692+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:46:54.701371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002082918,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":200,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.413826+0000","last_change":"2024-08-19T10:43:42.776830+0000","last_active":"2024-08-19T10:53:49.413826+0000","last_peered":"2024-08-19T10:53:49.413826+0000","last_clean":"2024-08-19T10:53:49.413826+0000","last_became_active":"2024-08-19T10:43:42.775836+0000","last_became_peered":"2024-08-19T10:43:42.775836+0000","last_unstale":"2024-08-19T10:53:49.413826+0000","last_undegraded":"2024-08-19T10:53:49.413826+0000","last_fullsized":"2024-08-19T10:53:49.413826+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:59:14.529046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010100350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421830+0000","last_change":"2024-08-19T10:43:42.767491+0000","last_active":"2024-08-19T10:54:10.421830+0000","last_peered":"2024-08-19T10:54:10.421830+0000","last_clean":"2024-08-19T10:54:10.421830+0000","last_became_active":"2024-08-19T10:43:42.767132+0000","last_became_peered":"2024-08-19T10:43:42.767132+0000","last_unstale":"2024-08-19T10:54:10.421830+0000","last_undegraded":"2024-08-19T10:54:10.421830+0000","last_fullsized":"2024-08-19T10:54:10.421830+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:04:13.207849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001410375,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":544,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:50.770839+0000","last_change":"2024-08-19T10:43:42.796170+0000","last_active":"2024-08-19T10:52:50.770839+0000","last_peered":"2024-08-19T10:52:50.770839+0000","last_clean":"2024-08-19T10:52:50.770839+0000","last_became_active":"2024-08-19T10:43:42.790684+0000","last_became_peered":"2024-08-19T10:43:42.790684+0000","last_unstale":"2024-08-19T10:52:50.770839+0000","last_undegraded":"2024-08-19T10:52:50.770839+0000","last_fullsized":"2024-08-19T10:52:50.770839+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:48:05.328122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001115312,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":515,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:50.531232+0000","last_change":"2024-08-19T10:43:42.769655+0000","last_active":"2024-08-19T10:52:50.531232+0000","last_peered":"2024-08-19T10:52:50.531232+0000","last_clean":"2024-08-19T10:52:50.531232+0000","last_became_active":"2024-08-19T10:43:42.769275+0000","last_became_peered":"2024-08-19T10:43:42.769275+0000","last_unstale":"2024-08-19T10:52:50.531232+0000","last_undegraded":"2024-08-19T10:52:50.531232+0000","last_fullsized":"2024-08-19T10:52:50.531232+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:23:16.606513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010645120000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":198,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:14.426772+0000","last_change":"2024-08-19T10:43:42.771283+0000","last_active":"2024-08-19T10:54:14.426772+0000","last_peered":"2024-08-19T10:54:14.426772+0000","last_clean":"2024-08-19T10:54:14.426772+0000","last_became_active":"2024-08-19T10:43:42.770419+0000","last_became_peered":"2024-08-19T10:43:42.770419+0000","last_unstale":"2024-08-19T10:54:14.426772+0000","last_undegraded":"2024-08-19T10:54:14.426772+0000","last_fullsized":"2024-08-19T10:54:14.426772+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:39:44.981500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003718322,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.961415+0000","last_change":"2024-08-19T10:43:42.784789+0000","last_active":"2024-08-19T10:52:24.961415+0000","last_peered":"2024-08-19T10:52:24.961415+0000","last_clean":"2024-08-19T10:52:24.961415+0000","last_became_active":"2024-08-19T10:43:42.784451+0000","last_became_peered":"2024-08-19T10:43:42.784451+0000","last_unstale":"2024-08-19T10:52:24.961415+0000","last_undegraded":"2024-08-19T10:52:24.961415+0000","last_fullsized":"2024-08-19T10:52:24.961415+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:19:58.579737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081542699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":480,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:24.419480+0000","last_change":"2024-08-19T10:43:43.885027+0000","last_active":"2024-08-19T10:53:24.419480+0000","last_peered":"2024-08-19T10:53:24.419480+0000","last_clean":"2024-08-19T10:53:24.419480+0000","last_became_active":"2024-08-19T10:43:43.884192+0000","last_became_peered":"2024-08-19T10:43:43.884192+0000","last_unstale":"2024-08-19T10:53:24.419480+0000","last_undegraded":"2024-08-19T10:53:24.419480+0000","last_fullsized":"2024-08-19T10:53:24.419480+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:52:36.680781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021153309999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":6},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607774,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1868579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607773,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1278230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607776,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1278211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607778,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1278230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640479,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1278211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640481,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1868579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T10:56:24.880 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T10:56:25.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient: get_monmap_and_config 2024-08-19T10:56:25.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient: build_initial_monmap 2024-08-19T10:56:25.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient: monmap: 2024-08-19T10:56:25.000 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:25.000 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:25.000 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:25.005895+0000 2024-08-19T10:56:25.000 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:25.005895+0000 2024-08-19T10:56:25.000 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:25.000 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:25.000 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:25.001 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:25.001 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:25.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 1 Processor -- start 2024-08-19T10:56:25.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 1 -- start start 2024-08-19T10:56:25.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient: init 2024-08-19T10:56:25.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:25.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient: _add_conns ranks=[7,8,4,9,0,2,3,10,6,1,5] 2024-08-19T10:56:25.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a5270 0x7f3dc00a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:25.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient(hunting): picked mon.noname-i con 0x7f3dc00a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:25.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3dc00a6d30 0x7f3dc00a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:25.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient(hunting): picked mon.noname-e con 0x7f3dc00a6d30 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:25.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3dc01395b0 0x7f3dc01399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:25.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient(hunting): picked mon.noname-c con 0x7f3dc01395b0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:25.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:25.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3dc00a7650 con 0x7f3dc01395b0 2024-08-19T10:56:25.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:25.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f3dc00a8710 con 0x7f3dc00a5270 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3dc00a5a40 con 0x7f3dc00a6d30 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc7b93640 10 monclient(hunting): authenticate will time out at 2034.822543s 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc6130640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3dc01395b0 0x7f3dc01399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3dc00a6d30 0x7f3dc00a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc592f640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a5270 msgr2=0x7f3dc00a67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc6130640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3dc01395b0 0x7f3dc01399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45906/0 (socket says 172.21.15.89:45906) 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3dc00a6d30 0x7f3dc00a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50250/0 (socket says 172.21.15.89:50250) 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc6130640 1 -- 172.21.15.89:0/861915051 learned_addr learned my addr 172.21.15.89:0/861915051 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc6130640 10 monclient(hunting): get_auth_request con 0x7f3dc01395b0 auth_method 0 2024-08-19T10:56:25.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc6130640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc6130640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc6130640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc592f640 1 --2- 172.21.15.89:0/861915051 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a5270 0x7f3dc00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 10 monclient(hunting): get_auth_request con 0x7f3dc00a6d30 auth_method 0 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc6130640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc6130640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:25.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc6130640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:25.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 10 monclient(hunting): handle_auth_done global_id 114389 payload 274 2024-08-19T10:56:25.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 1 -- 172.21.15.89:0/861915051 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a5270 msgr2=0x7f3dc00a67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:25.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 1 --2- 172.21.15.89:0/861915051 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a5270 0x7f3dc00a67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:25.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 1 -- 172.21.15.89:0/861915051 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3dc01395b0 msgr2=0x7f3dc01399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:25.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 1 --2- 172.21.15.89:0/861915051 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3dc01395b0 0x7f3dc01399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:25.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 10 monclient: _finish_hunting 0 2024-08-19T10:56:25.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 1 monclient: found mon.noname-e 2024-08-19T10:56:25.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:25.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:25.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc512e640 1 -- 172.21.15.89:0/861915051 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc00a81d0 con 0x7f3dc00a6d30 2024-08-19T10:56:25.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.001+0000 7f3dc6130640 1 --2- 172.21.15.89:0/861915051 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3dc01395b0 0x7f3dc01399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:25.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc512e640 1 -- 172.21.15.89:0/861915051 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3dc00a6d30 msgr2=0x7f3dc00a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:25.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc512e640 1 --2- 172.21.15.89:0/861915051 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3dc00a6d30 0x7f3dc00a7110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009dc0 tx=0x7f3dbc01c870 comp rx=0 tx=0).ready entity=mon.8 client_cookie=bf754785db831050 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:25.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 1 -- 172.21.15.89:0/861915051 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3dbc0b2070 con 0x7f3dc00a6d30 2024-08-19T10:56:25.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:25.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:25.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 10 monclient: dump: 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:25.013 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 10 monclient: _finish_auth 0 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 10 monclient: _check_auth_tickets 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 1 -- 172.21.15.89:0/861915051 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dbc0a59b0 con 0x7f3dc00a6d30 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc7b93640 5 monclient: authenticate success, global_id 114389 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc7b93640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc7b93640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 1 -- 172.21.15.89:0/861915051 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3dbc0a5ba0 con 0x7f3dc00a6d30 2024-08-19T10:56:25.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc7b93640 4 set_mon_vals no callback set 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.005+0000 7f3dc492d640 10 monclient: dump: 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:25.015 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient: shutdown 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 -- 172.21.15.89:0/861915051 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3dc00a6d30 msgr2=0x7f3dc00a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 --2- 172.21.15.89:0/861915051 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3dc00a6d30 0x7f3dc00a7110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009dc0 tx=0x7f3dbc01c870 comp rx=0 tx=0).stop 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 -- 172.21.15.89:0/861915051 shutdown_connections 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 --2- 172.21.15.89:0/861915051 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3dc01395b0 0x7f3dc01399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 --2- 172.21.15.89:0/861915051 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3dc00a6d30 0x7f3dc00a7110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 --2- 172.21.15.89:0/861915051 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a5270 0x7f3dc00a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:25.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 -- 172.21.15.89:0/861915051 >> 172.21.15.89:0/861915051 conn(0x7f3dc0069a80 msgr2=0x7f3dc00abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 -- 172.21.15.89:0/861915051 shutdown_connections 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 -- 172.21.15.89:0/861915051 wait complete. 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient: build_initial_monmap 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient: monmap: 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:25.017 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 Processor -- start 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 -- start start 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient: init 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient: _add_conns ranks=[6,7,3,1,0,4,8,5,2] 2024-08-19T10:56:25.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3dc00a5270 0x7f3dc01338a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient(hunting): picked mon.noname-d con 0x7f3dc00a5270 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a6d30 0x7f3dc0133de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient(hunting): picked mon.noname-i con 0x7f3dc00a6d30 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3dc01395b0 0x7f3dc009c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient(hunting): picked mon.noname-g con 0x7f3dc01395b0 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc00a57d0 con 0x7f3dc01395b0 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f3dc00a5650 con 0x7f3dc00a5270 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f3dc0143230 con 0x7f3dc00a6d30 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:25.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc7b93640 10 monclient(hunting): authenticate will time out at 2034.829930s 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3dc00a5270 0x7f3dc01338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc512e640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a6d30 msgr2=0x7f3dc0133de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc512e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a6d30 0x7f3dc0133de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3dc00a5270 0x7f3dc01338a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:44178/0 (socket says 172.21.15.89:44178) 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 1 -- 172.21.15.89:0/3325995899 learned_addr learned my addr 172.21.15.89:0/3325995899 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 10 monclient(hunting): get_auth_request con 0x7f3dc00a5270 auth_method 0 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:25.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 10 monclient(hunting): handle_auth_done global_id 45618 payload 824 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc6130640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3dc01395b0 0x7f3dc009c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 1 -- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a6d30 msgr2=0x7f3dc0133de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a6d30 0x7f3dc0133de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 1 -- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3dc01395b0 msgr2=0x7f3dc009c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3dc01395b0 0x7f3dc009c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 10 monclient: _finish_hunting 0 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 1 monclient: found mon.noname-d 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.009+0000 7f3dc592f640 1 -- 172.21.15.89:0/3325995899 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc013f600 con 0x7f3dc00a5270 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc592f640 1 -- 172.21.15.89:0/3325995899 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3dc00a5270 msgr2=0x7f3dc01338a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc592f640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3dc00a5270 0x7f3dc01338a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b560 tx=0x7f3db000ba20 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 1 -- 172.21.15.89:0/3325995899 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3db0013020 con 0x7f3dc00a5270 2024-08-19T10:56:25.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 10 monclient: dump: 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:25.022 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 10 monclient: _finish_auth 0 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 1 -- 172.21.15.89:0/3325995899 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3db0003de0 con 0x7f3dc00a5270 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc7b93640 5 monclient: authenticate success, global_id 45618 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc7b93640 10 monclient: _renew_subs 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc7b93640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc7b93640 1 -- 172.21.15.89:0/3325995899 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc00a7230 con 0x7f3dc00a5270 2024-08-19T10:56:25.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 1 -- 172.21.15.89:0/3325995899 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3db000f460 con 0x7f3dc00a5270 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 10 monclient: dump: 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:25.024 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:25.025 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db77fe640 4 set_mon_vals no callback set 2024-08-19T10:56:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc7b93640 10 monclient: _renew_subs 2024-08-19T10:56:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc7b93640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:25.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc7b93640 1 -- 172.21.15.89:0/3325995899 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc009ee40 con 0x7f3dc00a5270 2024-08-19T10:56:25.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc7b93640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:25.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc7b93640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:25.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc7b93640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:25.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc7b93640 1 -- 172.21.15.89:0/3325995899 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc00a67f0 con 0x7f3dc00a5270 2024-08-19T10:56:25.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 1 -- 172.21.15.89:0/3325995899 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3db0016020 con 0x7f3dc00a5270 2024-08-19T10:56:25.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3da403be00 0x7f3da403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:25.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3db67fc640 1 -- 172.21.15.89:0/3325995899 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f3db004f440 con 0x7f3dc00a5270 2024-08-19T10:56:25.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc512e640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3da403be00 0x7f3da403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:25.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.013+0000 7f3dc512e640 10 monclient: get_auth_request con 0x7f3da403be00 auth_method 0 2024-08-19T10:56:25.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.017+0000 7f3dc512e640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3da403be00 0x7f3da403e2c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009d90 tx=0x7f3dbc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:25.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.017+0000 7f3db67fc640 1 -- 172.21.15.89:0/3325995899 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3db001d020 con 0x7f3dc00a5270 2024-08-19T10:56:25.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.017+0000 7f3db67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:25.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.017+0000 7f3db67fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:25.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.189+0000 7f3dc7b93640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:56:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.189+0000 7f3dc7b93640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:56:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.189+0000 7f3dc7b93640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:25.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.189+0000 7f3dc7b93640 1 -- 172.21.15.89:0/3325995899 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3dc0064590 con 0x7f3dc00a5270 2024-08-19T10:56:25.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.193+0000 7f3db67fc640 1 -- 172.21.15.89:0/3325995899 <== mon.6 v2:172.21.15.89:3303/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v274) ==== 74+0+22037 (secure 0 0 0) 0x7f3db0005300 con 0x7f3dc00a5270 2024-08-19T10:56:25.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.193+0000 7f3db67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-08-19T10:56:25.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.193+0000 7f3db67fc640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:25.193 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:56:25.194 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":274,"fsid":"b6e37c79-20e9-449f-b89e-c2165d4f4bed","created":"2024-08-19T10:37:50.906618+0000","modified":"2024-08-19T10:51:18.565091+0000","last_up_change":"2024-08-19T10:38:04.611102+0000","last_in_change":"2024-08-19T10:51:18.565091+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":183,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":15,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T10:38:04.494964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-19T10:38:11.600504+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"fooset","create_time":"2024-08-19T10:42:11.215358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"barset","create_time":"2024-08-19T10:42:11.998164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2be543b2-eb94-4907-8f21-6e91b7dd9733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6817","nonce":2476057681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6819","nonce":2476057681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6823","nonce":2476057681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2476057681},{"type":"v1","addr":"172.21.15.89:6821","nonce":2476057681}]},"public_addr":"172.21.15.89:6817/2476057681","cluster_addr":"172.21.15.89:6819/2476057681","heartbeat_back_addr":"172.21.15.89:6823/2476057681","heartbeat_front_addr":"172.21.15.89:6821/2476057681","state":["exists","up"]},{"osd":1,"uuid":"224c1713-25c8-4a48-b16e-8929fcb6424b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6801","nonce":375502586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6803","nonce":375502586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6807","nonce":375502586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":375502586},{"type":"v1","addr":"172.21.15.89:6805","nonce":375502586}]},"public_addr":"172.21.15.89:6801/375502586","cluster_addr":"172.21.15.89:6803/375502586","heartbeat_back_addr":"172.21.15.89:6807/375502586","heartbeat_front_addr":"172.21.15.89:6805/375502586","state":["exists","up"]},{"osd":2,"uuid":"5c4ce4c7-8a93-4987-9ef4-0c9153d8d95d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6809","nonce":1060450292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6811","nonce":1060450292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6815","nonce":1060450292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1060450292},{"type":"v1","addr":"172.21.15.89:6813","nonce":1060450292}]},"public_addr":"172.21.15.89:6809/1060450292","cluster_addr":"172.21.15.89:6811/1060450292","heartbeat_back_addr":"172.21.15.89:6815/1060450292","heartbeat_front_addr":"172.21.15.89:6813/1060450292","state":["exists","up"]},{"osd":3,"uuid":"2dcaafe0-5837-410b-ae99-c7561ef6b150","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6811","nonce":2751619171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6813","nonce":2751619171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6817","nonce":2751619171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2751619171},{"type":"v1","addr":"172.21.15.190:6815","nonce":2751619171}]},"public_addr":"172.21.15.190:6811/2751619171","cluster_addr":"172.21.15.190:6813/2751619171","heartbeat_back_addr":"172.21.15.190:6817/2751619171","heartbeat_front_addr":"172.21.15.190:6815/2751619171","state":["exists","up"]},{"osd":4,"uuid":"5619c403-27fd-48de-8cee-c5e84289cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6819","nonce":1174298329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6821","nonce":1174298329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6825","nonce":1174298329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1174298329},{"type":"v1","addr":"172.21.15.190:6823","nonce":1174298329}]},"public_addr":"172.21.15.190:6819/1174298329","cluster_addr":"172.21.15.190:6821/1174298329","heartbeat_back_addr":"172.21.15.190:6825/1174298329","heartbeat_front_addr":"172.21.15.190:6823/1174298329","state":["exists","up"]},{"osd":5,"uuid":"82ab73a3-1212-4b0d-ab05-aa9c01dd5586","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6803","nonce":2434772607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6805","nonce":2434772607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6809","nonce":2434772607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2434772607},{"type":"v1","addr":"172.21.15.190:6807","nonce":2434772607}]},"public_addr":"172.21.15.190:6803/2434772607","cluster_addr":"172.21.15.190:6805/2434772607","heartbeat_back_addr":"172.21.15.190:6809/2434772607","heartbeat_front_addr":"172.21.15.190:6807/2434772607","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["autoout","exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["autoout","exists","new"]},{"osd":8,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["autoout","exists","new"]},{"osd":9,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"81c92c7b-73dd-42d5-ae6d-11d899af5fe3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.511833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:01.832703+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.351820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.269178+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.511490+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T10:38:02.178520+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":65536,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":65536,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":65536,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"ec-foo-ssd":{"crush-device-class":"ssd","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T10:56:25.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.193+0000 7f3dc7b93640 1 -- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3da403be00 msgr2=0x7f3da403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:25.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3da403be00 0x7f3da403e2c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009d90 tx=0x7f3dbc0a7070 comp rx=0 tx=0).stop 2024-08-19T10:56:25.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 10 monclient: shutdown 2024-08-19T10:56:25.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:25.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 1 -- 172.21.15.89:0/3325995899 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3dc00a5270 msgr2=0x7f3dc01338a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:25.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3dc00a5270 0x7f3dc01338a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b560 tx=0x7f3db000ba20 comp rx=0 tx=0).stop 2024-08-19T10:56:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 1 -- 172.21.15.89:0/3325995899 shutdown_connections 2024-08-19T10:56:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3da403be00 0x7f3da403e2c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3dc01395b0 0x7f3dc009c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3dc00a6d30 0x7f3dc0133de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 1 --2- 172.21.15.89:0/3325995899 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f3dc00a5270 0x7f3dc01338a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 1 -- 172.21.15.89:0/3325995899 >> 172.21.15.89:0/3325995899 conn(0x7f3dc0069a80 msgr2=0x7f3dc01345d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 1 -- 172.21.15.89:0/3325995899 shutdown_connections 2024-08-19T10:56:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:25.197+0000 7f3dc7b93640 1 -- 172.21.15.89:0/3325995899 wait complete. 2024-08-19T10:56:26.213 INFO:tasks.ceph:Scrubbing osd.0 2024-08-19T10:56:26.214 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-08-19T10:56:26.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient: get_monmap_and_config 2024-08-19T10:56:26.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient: build_initial_monmap 2024-08-19T10:56:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient: monmap: 2024-08-19T10:56:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:26.354374+0000 2024-08-19T10:56:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:26.354374+0000 2024-08-19T10:56:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:26.349 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:26.350 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:26.350 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 1 Processor -- start 2024-08-19T10:56:26.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 1 -- start start 2024-08-19T10:56:26.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient: init 2024-08-19T10:56:26.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient: _add_conns ranks=[7,1,6,5,2,9,10,3,8,0,4] 2024-08-19T10:56:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facd01397d0 0x7facd0139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient(hunting): picked mon.noname-i con 0x7facd01397d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7facd006e750 addr v1:172.21.15.190:6789/0 2024-08-19T10:56:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7facd00a5d50 0x7facd0141670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient(hunting): picked mon.noname-d con 0x7facd00a5d50 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd006f0f0 con 0x7facd006e750 2024-08-19T10:56:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7facd006e050 con 0x7facd00a5d50 2024-08-19T10:56:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7facd006f3d0 con 0x7facd01397d0 2024-08-19T10:56:26.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:26.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd648d640 10 monclient(hunting): authenticate will time out at 2036.171573s 2024-08-19T10:56:26.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7faccffff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facd01397d0 msgr2=0x7facd0139bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:26.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7faccffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facd01397d0 0x7facd0139bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:26.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd4a2a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7facd00a5d50 0x7facd0141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd4a2a640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7facd00a5d50 0x7facd0141670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:44184/0 (socket says 172.21.15.89:44184) 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd4a2a640 1 -- 172.21.15.89:0/690334251 learned_addr learned my addr 172.21.15.89:0/690334251 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd4a2a640 10 monclient(hunting): get_auth_request con 0x7facd00a5d50 auth_method 0 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd4a2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd4a2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.349+0000 7facd4a2a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 10 monclient(hunting): handle_auth_done global_id 45627 payload 274 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 1 -- 172.21.15.89:0/690334251 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facd01397d0 msgr2=0x7facd0139bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 1 --2- 172.21.15.89:0/690334251 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facd01397d0 0x7facd0139bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 1 -- 172.21.15.89:0/690334251 >> v1:172.21.15.190:6789/0 conn(0x7facd006e750 legacy=0x7facd00a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:26.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 10 monclient: _finish_hunting 0 2024-08-19T10:56:26.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 1 monclient: found mon.noname-d 2024-08-19T10:56:26.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:26.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:56:26.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.353+0000 7facd4a2a640 1 -- 172.21.15.89:0/690334251 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd00a7190 con 0x7facd00a5d50 2024-08-19T10:56:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 1 -- 172.21.15.89:0/690334251 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7facd00a5d50 msgr2=0x7facd0141670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 1 --2- 172.21.15.89:0/690334251 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7facd00a5d50 0x7facd0141670 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7facc8009e80 tx=0x7facc801c930 comp rx=0 tx=0).ready entity=mon.6 client_cookie=9ddbc4c51739dbd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 1 -- 172.21.15.89:0/690334251 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7facc80091b0 con 0x7facd00a5d50 2024-08-19T10:56:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:56:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 10 monclient: dump: 2024-08-19T10:56:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.360 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 10 monclient: _finish_auth 0 2024-08-19T10:56:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 1 -- 172.21.15.89:0/690334251 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc80a5a90 con 0x7facd00a5d50 2024-08-19T10:56:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 1 -- 172.21.15.89:0/690334251 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7facc80a5c40 con 0x7facd00a5d50 2024-08-19T10:56:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 5 monclient: authenticate success, global_id 45627 2024-08-19T10:56:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 4 set_mon_vals no callback set 2024-08-19T10:56:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:26.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facceffd640 10 monclient: dump: 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.363 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient: shutdown 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 -- 172.21.15.89:0/690334251 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7facd00a5d50 msgr2=0x7facd0141670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 --2- 172.21.15.89:0/690334251 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7facd00a5d50 0x7facd0141670 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7facc8009e80 tx=0x7facc801c930 comp rx=0 tx=0).stop 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 -- 172.21.15.89:0/690334251 shutdown_connections 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 --2- 172.21.15.89:0/690334251 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7facd00a5d50 0x7facd0141670 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 --2- 172.21.15.89:0/690334251 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facd01397d0 0x7facd0139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 -- 172.21.15.89:0/690334251 >> 172.21.15.89:0/690334251 conn(0x7facd00aa230 msgr2=0x7facd00a6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 -- 172.21.15.89:0/690334251 shutdown_connections 2024-08-19T10:56:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 -- 172.21.15.89:0/690334251 wait complete. 2024-08-19T10:56:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient: build_initial_monmap 2024-08-19T10:56:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient: monmap: 2024-08-19T10:56:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:26.365 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:26.366 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 Processor -- start 2024-08-19T10:56:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 -- start start 2024-08-19T10:56:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient: init 2024-08-19T10:56:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient: _add_conns ranks=[7,2,8,5,4,3,0,1,6] 2024-08-19T10:56:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facd006e750 0x7facd0140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient(hunting): picked mon.noname-i con 0x7facd006e750 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facd00a5d50 0x7facd0137840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient(hunting): picked mon.noname-b con 0x7facd00a5d50 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7facd01397d0 0x7facd0137d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient(hunting): picked mon.noname-e con 0x7facd01397d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7facd01443d0 con 0x7facd00a5d50 2024-08-19T10:56:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7facd0144550 con 0x7facd006e750 2024-08-19T10:56:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7facd01446d0 con 0x7facd01397d0 2024-08-19T10:56:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd648d640 10 monclient(hunting): authenticate will time out at 2036.178950s 2024-08-19T10:56:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7faccf7fe640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facd00a5d50 msgr2=0x7facd0137840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7faccf7fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facd00a5d50 0x7facd0137840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:26.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7faccffff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facd006e750 msgr2=0x7facd0140080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7facd01397d0 0x7facd0137d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7faccffff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facd006e750 0x7facd0140080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7facd01397d0 0x7facd0137d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50258/0 (socket says 172.21.15.89:50258) 2024-08-19T10:56:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 1 -- 172.21.15.89:0/1006303633 learned_addr learned my addr 172.21.15.89:0/1006303633 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 10 monclient(hunting): get_auth_request con 0x7facd01397d0 auth_method 0 2024-08-19T10:56:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:26.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 10 monclient(hunting): handle_auth_done global_id 114398 payload 824 2024-08-19T10:56:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 1 -- 172.21.15.89:0/1006303633 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facd006e750 msgr2=0x7facd0140080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7facd006e750 0x7facd0140080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 1 -- 172.21.15.89:0/1006303633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facd00a5d50 msgr2=0x7facd0137840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7facd00a5d50 0x7facd0137840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 10 monclient: _finish_hunting 0 2024-08-19T10:56:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 1 monclient: found mon.noname-e 2024-08-19T10:56:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:26.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.357+0000 7facd4a2a640 1 -- 172.21.15.89:0/1006303633 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd0121d00 con 0x7facd01397d0 2024-08-19T10:56:26.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facd4a2a640 1 -- 172.21.15.89:0/1006303633 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7facd01397d0 msgr2=0x7facd0137d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:26.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facd4a2a640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7facd01397d0 0x7facd0137d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7facc801ce10 tx=0x7facc80196d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:26.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 1 -- 172.21.15.89:0/1006303633 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7facc80bc020 con 0x7facd01397d0 2024-08-19T10:56:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 10 monclient: dump: 2024-08-19T10:56:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.374 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.375 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.376 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 10 monclient: _finish_auth 0 2024-08-19T10:56:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 10 monclient: _check_auth_tickets 2024-08-19T10:56:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 1 -- 172.21.15.89:0/1006303633 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc8003bb0 con 0x7facd01397d0 2024-08-19T10:56:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facd648d640 5 monclient: authenticate success, global_id 114398 2024-08-19T10:56:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facd648d640 10 monclient: _renew_subs 2024-08-19T10:56:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facd648d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facd648d640 1 -- 172.21.15.89:0/1006303633 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd0140aa0 con 0x7facd01397d0 2024-08-19T10:56:26.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 1 -- 172.21.15.89:0/1006303633 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7facc8003d60 con 0x7facd01397d0 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 10 monclient: dump: 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.377 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.378 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facce7fc640 4 set_mon_vals no callback set 2024-08-19T10:56:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facd648d640 10 monclient: _renew_subs 2024-08-19T10:56:26.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facd648d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facd648d640 1 -- 172.21.15.89:0/1006303633 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7facd0122b70 con 0x7facd01397d0 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facd648d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facd648d640 1 -- 172.21.15.89:0/1006303633 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_get_version(what=osdmap handle=1) -- 0x7facd0141520 con 0x7facd01397d0 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 1 -- 172.21.15.89:0/1006303633 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7facc80bf020 con 0x7facd01397d0 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7facbc03be30 0x7facbc03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 1 -- 172.21.15.89:0/1006303633 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7facc80ea440 con 0x7facd01397d0 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x7facbc044ea0 0x7facbc047300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 1 -- 172.21.15.89:0/1006303633 --> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facc80020c0 con 0x7facbc044ea0 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 1 -- 172.21.15.89:0/1006303633 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_get_version_reply(handle=1 version=274) ==== 24+0+0 (secure 0 0 0) 0x7facc80f1050 con 0x7facd01397d0 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7facccff9640 10 monclient: handle_get_version_reply finishing 1 version 274 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7faccf7fe640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x7facbc044ea0 0x7facbc047300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7faccf7fe640 10 monclient: get_auth_request con 0x7facbc044ea0 auth_method 0 2024-08-19T10:56:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7faccffff640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7facbc03be30 0x7facbc03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.361+0000 7faccffff640 10 monclient: get_auth_request con 0x7facbc03be30 auth_method 0 2024-08-19T10:56:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.365+0000 7faccffff640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7facbc03be30 0x7facbc03e2f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7facc0006fd0 tx=0x7facc0006d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.365+0000 7faccf7fe640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x7facbc044ea0 0x7facbc047300 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.369+0000 7facccff9640 1 -- 172.21.15.89:0/1006303633 <== osd.0 v2:172.21.15.89:6816/2476057681 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facc80020c0 con 0x7facbc044ea0 2024-08-19T10:56:26.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 1 -- 172.21.15.89:0/1006303633 --> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7facd00a5830 con 0x7facbc044ea0 2024-08-19T10:56:26.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facccff9640 1 -- 172.21.15.89:0/1006303633 <== osd.0 v2:172.21.15.89:6816/2476057681 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7facd00a5830 con 0x7facbc044ea0 2024-08-19T10:56:26.381 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:26.381 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T10:56:26.381 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:26.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 1 -- 172.21.15.89:0/1006303633 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x7facbc044ea0 msgr2=0x7facbc047300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.89:6816/2476057681,v1:172.21.15.89:6817/2476057681] conn(0x7facbc044ea0 0x7facbc047300 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 1 -- 172.21.15.89:0/1006303633 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7facbc03be30 msgr2=0x7facbc03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7facbc03be30 0x7facbc03e2f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7facc0006fd0 tx=0x7facc0006d40 comp rx=0 tx=0).stop 2024-08-19T10:56:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 10 monclient: shutdown 2024-08-19T10:56:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 1 -- 172.21.15.89:0/1006303633 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7facd01397d0 msgr2=0x7facd0137d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 1 --2- 172.21.15.89:0/1006303633 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7facd01397d0 0x7facd0137d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7facc801ce10 tx=0x7facc80196d0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd4a2a640 1 -- 172.21.15.89:0/1006303633 reap_dead start 2024-08-19T10:56:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 1 -- 172.21.15.89:0/1006303633 shutdown_connections 2024-08-19T10:56:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 1 -- 172.21.15.89:0/1006303633 >> 172.21.15.89:0/1006303633 conn(0x7facd00aa230 msgr2=0x7facd00a7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 1 -- 172.21.15.89:0/1006303633 shutdown_connections 2024-08-19T10:56:26.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.381+0000 7facd648d640 1 -- 172.21.15.89:0/1006303633 wait complete. 2024-08-19T10:56:26.397 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-08-19T10:56:26.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient: get_monmap_and_config 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient: build_initial_monmap 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient: monmap: 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:26.517526+0000 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:26.517526+0000 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:26.512 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:26.513 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:26.513 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:26.513 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:26.513 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:26.513 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:26.513 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:26.513 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:26.513 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 1 Processor -- start 2024-08-19T10:56:26.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 1 -- start start 2024-08-19T10:56:26.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient: init 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient: _add_conns ranks=[8,0,9,4,10,6,1,2,7,3,5] 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4d809eb60 0x7fd4d809d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient(hunting): picked mon.noname-e con 0x7fd4d809eb60 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd4d809ef40 addr v1:172.21.15.89:6789/0 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd4d809f380 0x7fd4d806d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient(hunting): picked mon.noname-a con 0x7fd4d809f380 addr v2:172.21.15.89:3300/0 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d80a3b20 con 0x7fd4d809ef40 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d80a3da0 con 0x7fd4d809f380 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fd4d809fc30 con 0x7fd4d809eb60 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4ddbda640 10 monclient(hunting): authenticate will time out at 2036.334477s 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d6ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd4d809ef40 0x7fd4d806c930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54340/0 (socket says 172.21.15.89:54340) 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4d809eb60 0x7fd4d809d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d6ffd640 1 -- 172.21.15.89:0/3211286927 learned_addr learned my addr 172.21.15.89:0/3211286927 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 1 --2- 172.21.15.89:0/3211286927 >> v2:172.21.15.89:3300/0 conn(0x7fd4d809f380 0x7fd4d806d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d77fe640 10 monclient(hunting): get_auth_request con 0x7fd4d809eb60 auth_method 0 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d77fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 10 monclient(hunting): get_auth_request con 0x7fd4d809f380 auth_method 0 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:26.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 10 monclient(hunting): handle_auth_done global_id 7821 payload 274 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 1 -- 172.21.15.89:0/3211286927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4d809eb60 msgr2=0x7fd4d809d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 1 --2- 172.21.15.89:0/3211286927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4d809eb60 0x7fd4d809d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 1 -- 172.21.15.89:0/3211286927 >> v1:172.21.15.89:6789/0 conn(0x7fd4d809ef40 legacy=0x7fd4d806c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 1 monclient: found mon.noname-a 2024-08-19T10:56:26.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:26.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:56:26.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d7fff640 1 -- 172.21.15.89:0/3211286927 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d80a4e40 con 0x7fd4d809f380 2024-08-19T10:56:26.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.513+0000 7fd4d77fe640 1 --2- 172.21.15.89:0/3211286927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4d809eb60 0x7fd4d809d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:26.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d7fff640 1 -- 172.21.15.89:0/3211286927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 msgr2=0x7fd4d806d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:26.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d7fff640 1 --2- 172.21.15.89:0/3211286927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 0x7fd4d806d040 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009d90 tx=0x7fd4d001c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283a2262e067462e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:26.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 1 -- 172.21.15.89:0/3211286927 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd4d00ac070 con 0x7fd4d809f380 2024-08-19T10:56:26.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:56:26.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 10 monclient: dump: 2024-08-19T10:56:26.522 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.522 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.522 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.522 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 10 monclient: _finish_auth 0 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 1 -- 172.21.15.89:0/3211286927 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d00a68f0 con 0x7fd4d809f380 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 1 -- 172.21.15.89:0/3211286927 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd4d00a7460 con 0x7fd4d809f380 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 5 monclient: authenticate success, global_id 7821 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 4 set_mon_vals no callback set 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4d67fc640 10 monclient: dump: 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.524 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 10 monclient: shutdown 2024-08-19T10:56:26.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 1 -- 172.21.15.89:0/3211286927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 msgr2=0x7fd4d806d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 1 --2- 172.21.15.89:0/3211286927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 0x7fd4d806d040 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009d90 tx=0x7fd4d001c830 comp rx=0 tx=0).stop 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 1 -- 172.21.15.89:0/3211286927 shutdown_connections 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 1 --2- 172.21.15.89:0/3211286927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 0x7fd4d806d040 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 1 --2- 172.21.15.89:0/3211286927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fd4d809eb60 0x7fd4d809d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 1 -- 172.21.15.89:0/3211286927 >> 172.21.15.89:0/3211286927 conn(0x7fd4d80a8f60 msgr2=0x7fd4d80a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 1 -- 172.21.15.89:0/3211286927 shutdown_connections 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.517+0000 7fd4ddbda640 1 -- 172.21.15.89:0/3211286927 wait complete. 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient: build_initial_monmap 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient: monmap: 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:26.526 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 1 Processor -- start 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 1 -- start start 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient: init 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient: _add_conns ranks=[2,1,0,4,7,6,8,5,3] 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4d809eb60 0x7fd4d81397a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient(hunting): picked mon.noname-b con 0x7fd4d809eb60 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4d809ef40 0x7fd4d8139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient(hunting): picked mon.noname-f con 0x7fd4d809ef40 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 0x7fd4d80a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient(hunting): picked mon.noname-a con 0x7fd4d809f380 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d8141d70 con 0x7fd4d809f380 2024-08-19T10:56:26.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d806d790 con 0x7fd4d809ef40 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d8141ef0 con 0x7fd4d809eb60 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4ddbda640 10 monclient(hunting): authenticate will time out at 2036.341342s 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d77fe640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4d809eb60 msgr2=0x7fd4d81397a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4d809eb60 0x7fd4d81397a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 0x7fd4d80a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 0x7fd4d80a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59278/0 (socket says 172.21.15.89:59278) 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 1 -- 172.21.15.89:0/2527294066 learned_addr learned my addr 172.21.15.89:0/2527294066 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 10 monclient(hunting): get_auth_request con 0x7fd4d809f380 auth_method 0 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:26.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 10 monclient(hunting): handle_auth_done global_id 7830 payload 824 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 1 -- 172.21.15.89:0/2527294066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4d809eb60 msgr2=0x7fd4d81397a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4d809eb60 0x7fd4d81397a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 1 -- 172.21.15.89:0/2527294066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4d809ef40 msgr2=0x7fd4d8139ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4d809ef40 0x7fd4d8139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 10 monclient: _finish_hunting 0 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 1 monclient: found mon.noname-a 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:26.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:56:26.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.521+0000 7fd4d7fff640 1 -- 172.21.15.89:0/2527294066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d80a1aa0 con 0x7fd4d809f380 2024-08-19T10:56:26.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4d7fff640 1 -- 172.21.15.89:0/2527294066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 msgr2=0x7fd4d80a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4d7fff640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 0x7fd4d80a02a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd4d001c660 tx=0x7fd4d00195d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 1 -- 172.21.15.89:0/2527294066 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd4d00ac050 con 0x7fd4d809f380 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 10 monclient: dump: 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.533 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 10 monclient: _finish_auth 0 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 10 monclient: _check_auth_tickets 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4ddbda640 5 monclient: authenticate success, global_id 7830 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 1 -- 172.21.15.89:0/2527294066 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d0003bb0 con 0x7fd4d809f380 2024-08-19T10:56:26.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4ddbda640 10 monclient: _renew_subs 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4ddbda640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4ddbda640 1 -- 172.21.15.89:0/2527294066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d8126170 con 0x7fd4d809f380 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 1 -- 172.21.15.89:0/2527294066 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd4d0003d60 con 0x7fd4d809f380 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4d5ffb640 4 set_mon_vals no callback set 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 10 monclient: dump: 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.535 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4ddbda640 10 monclient: _renew_subs 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4ddbda640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4ddbda640 1 -- 172.21.15.89:0/2527294066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d80a6bc0 con 0x7fd4d809f380 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4ddbda640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4ddbda640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4ddbda640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4ddbda640 1 -- 172.21.15.89:0/2527294066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d80a8960 con 0x7fd4d809f380 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 1 -- 172.21.15.89:0/2527294066 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd4d00c1020 con 0x7fd4d809f380 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4b803be00 0x7fd4b803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4cffff640 1 -- 172.21.15.89:0/2527294066 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fd4d00b7070 con 0x7fd4d809f380 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4d77fe640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4b803be00 0x7fd4b803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.525+0000 7fd4d77fe640 10 monclient: get_auth_request con 0x7fd4b803be00 auth_method 0 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.529+0000 7fd4d77fe640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4b803be00 0x7fd4b803e2c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009580 tx=0x7fd4c8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.529+0000 7fd4cffff640 1 -- 172.21.15.89:0/2527294066 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd4d80a8960 con 0x7fd4d809f380 2024-08-19T10:56:26.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.529+0000 7fd4cffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:26.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.529+0000 7fd4cffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:26.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.709+0000 7fd4ddbda640 1 -- 172.21.15.89:0/2527294066 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fd4d8125b10 con 0x7fd4b803be00 2024-08-19T10:56:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.713+0000 7fd4cffff640 1 -- 172.21.15.89:0/2527294066 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd4d8125b10 con 0x7fd4b803be00 2024-08-19T10:56:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 0 to deep-scrub 2024-08-19T10:56:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.713+0000 7fd4ddbda640 1 -- 172.21.15.89:0/2527294066 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4b803be00 msgr2=0x7fd4b803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4b803be00 0x7fd4b803e2c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009580 tx=0x7fd4c8009190 comp rx=0 tx=0).stop 2024-08-19T10:56:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 10 monclient: shutdown 2024-08-19T10:56:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 1 -- 172.21.15.89:0/2527294066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 msgr2=0x7fd4d80a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 0x7fd4d80a02a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd4d001c660 tx=0x7fd4d00195d0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 1 -- 172.21.15.89:0/2527294066 shutdown_connections 2024-08-19T10:56:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd4b803be00 0x7fd4b803e2c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4d809f380 0x7fd4d80a02a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4d809ef40 0x7fd4d8139ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 1 --2- 172.21.15.89:0/2527294066 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd4d809eb60 0x7fd4d81397a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 1 -- 172.21.15.89:0/2527294066 >> 172.21.15.89:0/2527294066 conn(0x7fd4d80a8f60 msgr2=0x7fd4d8073690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:26.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 1 -- 172.21.15.89:0/2527294066 shutdown_connections 2024-08-19T10:56:26.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.717+0000 7fd4ddbda640 1 -- 172.21.15.89:0/2527294066 wait complete. 2024-08-19T10:56:26.742 INFO:tasks.ceph:Scrubbing osd.1 2024-08-19T10:56:26.742 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-08-19T10:56:26.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient: get_monmap_and_config 2024-08-19T10:56:26.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient: build_initial_monmap 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient: monmap: 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:26.861557+0000 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:26.861557+0000 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:26.856 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:26.857 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:26.857 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:26.857 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:26.857 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:26.857 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:26.857 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 1 Processor -- start 2024-08-19T10:56:26.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 1 -- start start 2024-08-19T10:56:26.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient: init 2024-08-19T10:56:26.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient: _add_conns ranks=[5,3,2,7,1,0,10,9,4,8,6] 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87780a6e40 0x7f87781373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient(hunting): picked mon.noname-h con 0x7f87780a6e40 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8778137930 0x7f8778137d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient(hunting): picked mon.noname-g con 0x7f8778137930 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87780a5640 0x7f87780a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient(hunting): picked mon.noname-b con 0x7f87780a5640 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f87780a7220 con 0x7f87780a5640 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f87780a3fc0 con 0x7f8778137930 2024-08-19T10:56:26.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f87780723b0 con 0x7f87780a6e40 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877e8aa640 10 monclient(hunting): authenticate will time out at 2036.677750s 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877ce47640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87780a5640 msgr2=0x7f87780a5a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f877ce47640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87780a5640 0x7f87780a5a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87780a6e40 0x7f87781373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87780a6e40 0x7f87781373f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:44822/0 (socket says 172.21.15.89:44822) 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 1 -- 172.21.15.89:0/2610366114 learned_addr learned my addr 172.21.15.89:0/2610366114 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 10 monclient(hunting): get_auth_request con 0x7f87780a6e40 auth_method 0 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f87777fe640 1 --2- 172.21.15.89:0/2610366114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8778137930 0x7f8778137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f87777fe640 10 monclient(hunting): get_auth_request con 0x7f8778137930 auth_method 0 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f87777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f87777fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f87777fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f87777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f87777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f87777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 10 monclient(hunting): handle_auth_done global_id 84461 payload 274 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 1 -- 172.21.15.89:0/2610366114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8778137930 msgr2=0x7f8778137d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 1 --2- 172.21.15.89:0/2610366114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8778137930 0x7f8778137d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 1 -- 172.21.15.89:0/2610366114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87780a5640 msgr2=0x7f87780a5a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:26.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 1 --2- 172.21.15.89:0/2610366114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87780a5640 0x7f87780a5a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 10 monclient: _finish_hunting 0 2024-08-19T10:56:26.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 1 monclient: found mon.noname-h 2024-08-19T10:56:26.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:26.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.190:3302/0 2024-08-19T10:56:26.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.857+0000 7f8777fff640 1 -- 172.21.15.89:0/2610366114 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87780a17c0 con 0x7f87780a6e40 2024-08-19T10:56:26.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8777fff640 1 -- 172.21.15.89:0/2610366114 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87780a6e40 msgr2=0x7f87781373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:26.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8777fff640 1 --2- 172.21.15.89:0/2610366114 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87780a6e40 0x7f87781373f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f87680098d0 tx=0x7f876801c430 comp rx=0 tx=0).ready entity=mon.5 client_cookie=b1718e49e6a5c298 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:26.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 1 -- 172.21.15.89:0/2610366114 <== mon.5 v2:172.21.15.190:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f87680b0070 con 0x7f87780a6e40 2024-08-19T10:56:26.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-08-19T10:56:26.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 10 monclient: dump: 2024-08-19T10:56:26.870 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.870 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.870 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.870 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.871 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.871 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.871 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.871 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.871 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.871 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 10 monclient: _finish_auth 0 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 1 -- 172.21.15.89:0/2610366114 <== mon.5 v2:172.21.15.190:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87680044a0 con 0x7f87780a6e40 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 5 monclient: authenticate success, global_id 84461 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:26.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 4 set_mon_vals no callback set 2024-08-19T10:56:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 1 -- 172.21.15.89:0/2610366114 <== mon.5 v2:172.21.15.190:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8768003620 con 0x7f87780a6e40 2024-08-19T10:56:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-08-19T10:56:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f8776ffd640 10 monclient: dump: 2024-08-19T10:56:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.873 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 10 monclient: shutdown 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 1 -- 172.21.15.89:0/2610366114 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87780a6e40 msgr2=0x7f87781373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 1 --2- 172.21.15.89:0/2610366114 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87780a6e40 0x7f87781373f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f87680098d0 tx=0x7f876801c430 comp rx=0 tx=0).stop 2024-08-19T10:56:26.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 1 -- 172.21.15.89:0/2610366114 shutdown_connections 2024-08-19T10:56:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 1 --2- 172.21.15.89:0/2610366114 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f87780a5640 0x7f87780a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 1 --2- 172.21.15.89:0/2610366114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8778137930 0x7f8778137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 1 --2- 172.21.15.89:0/2610366114 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f87780a6e40 0x7f87781373f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 1 -- 172.21.15.89:0/2610366114 >> 172.21.15.89:0/2610366114 conn(0x7f87780aa230 msgr2=0x7f87780a6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 1 -- 172.21.15.89:0/2610366114 shutdown_connections 2024-08-19T10:56:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.861+0000 7f877e8aa640 1 -- 172.21.15.89:0/2610366114 wait complete. 2024-08-19T10:56:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient: build_initial_monmap 2024-08-19T10:56:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient: monmap: 2024-08-19T10:56:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:26.875 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:26.876 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 1 Processor -- start 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 1 -- start start 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient: init 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient: _add_conns ranks=[1,0,4,8,5,3,6,2,7] 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87780a5640 0x7f87781cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient(hunting): picked mon.noname-f con 0x7f87780a5640 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87780a6e40 0x7f87781364b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient(hunting): picked mon.noname-a con 0x7f87780a6e40 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8778137930 0x7f877813b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient(hunting): picked mon.noname-c con 0x7f8778137930 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:26.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8778074390 con 0x7f87780a6e40 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f87780a63d0 con 0x7f87780a5640 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f8778141070 con 0x7f8778137930 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877e8aa640 10 monclient(hunting): authenticate will time out at 2036.685270s 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8778137930 0x7f877813b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f87777fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87780a6e40 0x7f87781364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f87777fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87780a6e40 0x7f87781364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59294/0 (socket says 172.21.15.89:59294) 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8778137930 0x7f877813b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45910/0 (socket says 172.21.15.89:45910) 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f87777fe640 1 -- 172.21.15.89:0/653497081 learned_addr learned my addr 172.21.15.89:0/653497081 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:26.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f87777fe640 10 monclient(hunting): get_auth_request con 0x7f87780a6e40 auth_method 0 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f87777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f87777fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f87777fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 10 monclient(hunting): get_auth_request con 0x7f8778137930 auth_method 0 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f87777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f87777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f87777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:26.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f8777fff640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87780a5640 0x7f87781cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f8777fff640 10 monclient(hunting): get_auth_request con 0x7f87780a5640 auth_method 0 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f8777fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f8777fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f8777fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 10 monclient(hunting): handle_auth_done global_id 84595 payload 824 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 1 -- 172.21.15.89:0/653497081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87780a5640 msgr2=0x7f87781cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87780a5640 0x7f87781cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 1 -- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87780a6e40 msgr2=0x7f87781364b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87780a6e40 0x7f87781364b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 10 monclient: _finish_hunting 0 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 1 monclient: found mon.noname-c 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:26.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f877ce47640 1 -- 172.21.15.89:0/653497081 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8778121e50 con 0x7f8778137930 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.865+0000 7f87777fe640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87780a6e40 0x7f87781364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f877ce47640 1 -- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8778137930 msgr2=0x7f877813b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f877ce47640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8778137930 0x7f877813b5c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f877000bdd0 tx=0x7f8770009ed0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 1 -- 172.21.15.89:0/653497081 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f877000ce00 con 0x7f8778137930 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 10 monclient: dump: 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.881 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 10 monclient: _finish_auth 0 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:26.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 1 -- 172.21.15.89:0/653497081 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8770010030 con 0x7f8778137930 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f877e8aa640 5 monclient: authenticate success, global_id 84595 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 1 -- 172.21.15.89:0/653497081 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f8770006a30 con 0x7f8778137930 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f877e8aa640 10 monclient: _renew_subs 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f877e8aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f877e8aa640 1 -- 172.21.15.89:0/653497081 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877813d0b0 con 0x7f8778137930 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 10 monclient: dump: 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:26.883 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:26.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f87767fc640 4 set_mon_vals no callback set 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f877e8aa640 10 monclient: _renew_subs 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f877e8aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f877e8aa640 1 -- 172.21.15.89:0/653497081 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f8778121890 con 0x7f8778137930 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f877e8aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f877e8aa640 1 -- 172.21.15.89:0/653497081 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f87780a8b40 con 0x7f8778137930 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 1 -- 172.21.15.89:0/653497081 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f8770003d10 con 0x7f8778137930 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f876403be00 0x7f876403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 1 -- 172.21.15.89:0/653497081 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f877004f6f0 con 0x7f8778137930 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x7f8764044ef0 0x7f8764047350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 1 -- 172.21.15.89:0/653497081 --> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8764047a20 con 0x7f8764044ef0 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 1 -- 172.21.15.89:0/653497081 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_get_version_reply(handle=1 version=274) ==== 24+0+0 (secure 0 0 0) 0x7f8770056050 con 0x7f8778137930 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8774ff9640 10 monclient: handle_get_version_reply finishing 1 version 274 2024-08-19T10:56:26.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f87777fe640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x7f8764044ef0 0x7f8764047350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f87777fe640 10 monclient: get_auth_request con 0x7f8764044ef0 auth_method 0 2024-08-19T10:56:26.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8777fff640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f876403be00 0x7f876403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:26.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.869+0000 7f8777fff640 10 monclient: get_auth_request con 0x7f876403be00 auth_method 0 2024-08-19T10:56:26.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.873+0000 7f8777fff640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f876403be00 0x7f876403e2c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f87680ad000 tx=0x7f8768009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:26.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.873+0000 7f87777fe640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x7f8764044ef0 0x7f8764047350 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:26.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.873+0000 7f8774ff9640 1 -- 172.21.15.89:0/653497081 <== osd.1 v2:172.21.15.89:6800/375502586 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8764047a20 con 0x7f8764044ef0 2024-08-19T10:56:26.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.885+0000 7f877e8aa640 1 -- 172.21.15.89:0/653497081 --> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8778136c40 con 0x7f8764044ef0 2024-08-19T10:56:26.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.885+0000 7f8774ff9640 1 -- 172.21.15.89:0/653497081 <== osd.1 v2:172.21.15.89:6800/375502586 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8778136c40 con 0x7f8764044ef0 2024-08-19T10:56:26.887 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:26.887 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T10:56:26.887 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 1 -- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x7f8764044ef0 msgr2=0x7f8764047350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:6800/375502586,v1:172.21.15.89:6801/375502586] conn(0x7f8764044ef0 0x7f8764047350 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 1 -- 172.21.15.89:0/653497081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f876403be00 msgr2=0x7f876403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f876403be00 0x7f876403e2c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f87680ad000 tx=0x7f8768009040 comp rx=0 tx=0).stop 2024-08-19T10:56:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 10 monclient: shutdown 2024-08-19T10:56:26.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 1 -- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8778137930 msgr2=0x7f877813b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 1 --2- 172.21.15.89:0/653497081 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f8778137930 0x7f877813b5c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f877000bdd0 tx=0x7f8770009ed0 comp rx=0 tx=0).stop 2024-08-19T10:56:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877ce47640 1 -- 172.21.15.89:0/653497081 reap_dead start 2024-08-19T10:56:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 1 -- 172.21.15.89:0/653497081 shutdown_connections 2024-08-19T10:56:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 1 -- 172.21.15.89:0/653497081 >> 172.21.15.89:0/653497081 conn(0x7f87780aa230 msgr2=0x7f87780a1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 1 -- 172.21.15.89:0/653497081 shutdown_connections 2024-08-19T10:56:26.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:26.889+0000 7f877e8aa640 1 -- 172.21.15.89:0/653497081 wait complete. 2024-08-19T10:56:26.902 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-08-19T10:56:27.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient: get_monmap_and_config 2024-08-19T10:56:27.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient: build_initial_monmap 2024-08-19T10:56:27.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient: monmap: 2024-08-19T10:56:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:27.013267+0000 2024-08-19T10:56:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:27.013267+0000 2024-08-19T10:56:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:27.008 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:27.009 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:27.009 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 1 Processor -- start 2024-08-19T10:56:27.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 1 -- start start 2024-08-19T10:56:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient: init 2024-08-19T10:56:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient: _add_conns ranks=[2,1,10,0,9,3,6,5,7,8,4] 2024-08-19T10:56:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2b806c930 0x7ff2b806cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient(hunting): picked mon.noname-b con 0x7ff2b806c930 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff2b809d340 addr v1:172.21.15.190:6789/0 2024-08-19T10:56:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2b806d250 0x7ff2b8137c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient(hunting): picked mon.noname-f con 0x7ff2b806d250 addr v2:172.21.15.190:3300/0 2024-08-19T10:56:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b809fbf0 con 0x7ff2b809d340 2024-08-19T10:56:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b80a54f0 con 0x7ff2b806d250 2024-08-19T10:56:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b80a3e00 con 0x7ff2b806c930 2024-08-19T10:56:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2befb7640 10 monclient(hunting): authenticate will time out at 2036.829726s 2024-08-19T10:56:27.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bcd53640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2b806c930 msgr2=0x7ff2b806cd10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bcd53640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2b806c930 0x7ff2b806cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2b806d250 0x7ff2b8137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2b7fff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff2b809d340 0x7ff2b8137500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.89:60848/0 (socket says 172.21.15.89:60848) 2024-08-19T10:56:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2b7fff640 1 -- 172.21.15.89:0/126079456 learned_addr learned my addr 172.21.15.89:0/126079456 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:56:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 10 monclient(hunting): get_auth_request con 0x7ff2b806d250 auth_method 0 2024-08-19T10:56:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 10 monclient(hunting): handle_auth_done global_id 45532 payload 274 2024-08-19T10:56:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 1 -- 172.21.15.89:0/126079456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2b806c930 msgr2=0x7ff2b806cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 1 --2- 172.21.15.89:0/126079456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2b806c930 0x7ff2b806cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 1 -- 172.21.15.89:0/126079456 >> v1:172.21.15.190:6789/0 conn(0x7ff2b809d340 legacy=0x7ff2b8137500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 10 monclient: _finish_hunting 0 2024-08-19T10:56:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 1 monclient: found mon.noname-f 2024-08-19T10:56:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:27.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.009+0000 7ff2bd554640 1 -- 172.21.15.89:0/126079456 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b80a3b30 con 0x7ff2b806d250 2024-08-19T10:56:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2bd554640 1 -- 172.21.15.89:0/126079456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b806d250 msgr2=0x7ff2b8137c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2bd554640 1 --2- 172.21.15.89:0/126079456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b806d250 0x7ff2b8137c10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009d90 tx=0x7ff2b001c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d6ab6bd1c83a90 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 1 -- 172.21.15.89:0/126079456 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2b00b2070 con 0x7ff2b806d250 2024-08-19T10:56:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 10 monclient: dump: 2024-08-19T10:56:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.022 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.023 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 10 monclient: _finish_auth 0 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 10 monclient: _check_auth_tickets 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 1 -- 172.21.15.89:0/126079456 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b00a5ab0 con 0x7ff2b806d250 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 1 -- 172.21.15.89:0/126079456 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2b00a5c60 con 0x7ff2b806d250 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2b77fe640 10 monclient: dump: 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.024 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2befb7640 5 monclient: authenticate success, global_id 45532 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2befb7640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.013+0000 7ff2befb7640 4 set_mon_vals no callback set 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient: shutdown 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 -- 172.21.15.89:0/126079456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b806d250 msgr2=0x7ff2b8137c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 --2- 172.21.15.89:0/126079456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b806d250 0x7ff2b8137c10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009d90 tx=0x7ff2b001c850 comp rx=0 tx=0).stop 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 -- 172.21.15.89:0/126079456 shutdown_connections 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 --2- 172.21.15.89:0/126079456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b806d250 0x7ff2b8137c10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 --2- 172.21.15.89:0/126079456 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7ff2b806c930 0x7ff2b806cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 -- 172.21.15.89:0/126079456 >> 172.21.15.89:0/126079456 conn(0x7ff2b80a8f60 msgr2=0x7ff2b80a9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 -- 172.21.15.89:0/126079456 shutdown_connections 2024-08-19T10:56:27.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 -- 172.21.15.89:0/126079456 wait complete. 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient: build_initial_monmap 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient: monmap: 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:27.027 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 Processor -- start 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 -- start start 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient: init 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient: _add_conns ranks=[0,7,6,4,5,2,8,1,3] 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b806c930 0x7ff2b8137140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient(hunting): picked mon.noname-a con 0x7ff2b806c930 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2b806d250 0x7ff2b8137680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient(hunting): picked mon.noname-i con 0x7ff2b806d250 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b809d340 0x7ff2b813cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient(hunting): picked mon.noname-d con 0x7ff2b809d340 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b8138150 con 0x7ff2b806c930 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7ff2b8074420 con 0x7ff2b809d340 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7ff2b81382d0 con 0x7ff2b806d250 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2befb7640 10 monclient(hunting): authenticate will time out at 2036.837939s 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2b7fff640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2b806d250 msgr2=0x7ff2b8137680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b809d340 0x7ff2b813cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2b7fff640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2b806d250 0x7ff2b8137680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bcd53640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b806c930 0x7ff2b8137140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b809d340 0x7ff2b813cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:44198/0 (socket says 172.21.15.89:44198) 2024-08-19T10:56:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 1 -- 172.21.15.89:0/2973895720 learned_addr learned my addr 172.21.15.89:0/2973895720 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 10 monclient(hunting): get_auth_request con 0x7ff2b809d340 auth_method 0 2024-08-19T10:56:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bcd53640 10 monclient(hunting): get_auth_request con 0x7ff2b806c930 auth_method 0 2024-08-19T10:56:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bcd53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bcd53640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bcd53640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bcd53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bcd53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bcd53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 10 monclient(hunting): handle_auth_done global_id 45636 payload 824 2024-08-19T10:56:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 1 -- 172.21.15.89:0/2973895720 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2b806d250 msgr2=0x7ff2b8137680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:27.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2b806d250 0x7ff2b8137680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 1 -- 172.21.15.89:0/2973895720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b806c930 msgr2=0x7ff2b8137140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b806c930 0x7ff2b8137140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 10 monclient: _finish_hunting 0 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 1 monclient: found mon.noname-d 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bd554640 1 -- 172.21.15.89:0/2973895720 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b80a8710 con 0x7ff2b809d340 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.017+0000 7ff2bcd53640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b806c930 0x7ff2b8137140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2bd554640 1 -- 172.21.15.89:0/2973895720 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b809d340 msgr2=0x7ff2b813cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2bd554640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b809d340 0x7ff2b813cf40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff2b001af00 tx=0x7ff2b00195f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 1 -- 172.21.15.89:0/2973895720 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2b00bb030 con 0x7ff2b809d340 2024-08-19T10:56:27.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:56:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 10 monclient: dump: 2024-08-19T10:56:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.033 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 10 monclient: _finish_auth 0 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 10 monclient: _check_auth_tickets 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2befb7640 5 monclient: authenticate success, global_id 45636 2024-08-19T10:56:27.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 1 -- 172.21.15.89:0/2973895720 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b0003c70 con 0x7ff2b809d340 2024-08-19T10:56:27.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2befb7640 10 monclient: _renew_subs 2024-08-19T10:56:27.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2befb7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:27.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2befb7640 1 -- 172.21.15.89:0/2973895720 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b80a6310 con 0x7ff2b809d340 2024-08-19T10:56:27.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:27.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 1 -- 172.21.15.89:0/2973895720 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7ff2b00b2040 con 0x7ff2b809d340 2024-08-19T10:56:27.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b67fc640 4 set_mon_vals no callback set 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 10 monclient: dump: 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.036 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.037 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.037 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.037 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.037 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.037 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.037 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2befb7640 10 monclient: _renew_subs 2024-08-19T10:56:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2befb7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2befb7640 1 -- 172.21.15.89:0/2973895720 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b80a0fa0 con 0x7ff2b809d340 2024-08-19T10:56:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2befb7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2befb7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2befb7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2befb7640 1 -- 172.21.15.89:0/2973895720 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b80a1f00 con 0x7ff2b809d340 2024-08-19T10:56:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 1 -- 172.21.15.89:0/2973895720 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7ff2b00be070 con 0x7ff2b809d340 2024-08-19T10:56:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff29803be60 0x7ff29803e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2b57fa640 1 -- 172.21.15.89:0/2973895720 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7ff2b00bc070 con 0x7ff2b809d340 2024-08-19T10:56:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2bcd53640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff29803be60 0x7ff29803e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.021+0000 7ff2bcd53640 10 monclient: get_auth_request con 0x7ff29803be60 auth_method 0 2024-08-19T10:56:27.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.025+0000 7ff2bcd53640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff29803be60 0x7ff29803e320 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0003990 tx=0x7ff2a0009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.025+0000 7ff2b57fa640 1 -- 172.21.15.89:0/2973895720 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7ff2b80a1f00 con 0x7ff2b809d340 2024-08-19T10:56:27.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.025+0000 7ff2b57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:27.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.025+0000 7ff2b57fa640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:27.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.181+0000 7ff2befb7640 1 -- 172.21.15.89:0/2973895720 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7ff2b80a22f0 con 0x7ff29803be60 2024-08-19T10:56:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.181+0000 7ff2b57fa640 1 -- 172.21.15.89:0/2973895720 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff2a0005a50 con 0x7ff29803be60 2024-08-19T10:56:27.182 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 1 to deep-scrub 2024-08-19T10:56:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 -- 172.21.15.89:0/2973895720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff29803be60 msgr2=0x7ff29803e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff29803be60 0x7ff29803e320 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0003990 tx=0x7ff2a0009110 comp rx=0 tx=0).stop 2024-08-19T10:56:27.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 10 monclient: shutdown 2024-08-19T10:56:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 -- 172.21.15.89:0/2973895720 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b809d340 msgr2=0x7ff2b813cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b809d340 0x7ff2b813cf40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff2b001af00 tx=0x7ff2b00195f0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 -- 172.21.15.89:0/2973895720 shutdown_connections 2024-08-19T10:56:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7ff29803be60 0x7ff29803e320 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7ff2b809d340 0x7ff2b813cf40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7ff2b806d250 0x7ff2b8137680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 --2- 172.21.15.89:0/2973895720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff2b806c930 0x7ff2b8137140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 -- 172.21.15.89:0/2973895720 >> 172.21.15.89:0/2973895720 conn(0x7ff2b80a8f60 msgr2=0x7ff2b813ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 -- 172.21.15.89:0/2973895720 shutdown_connections 2024-08-19T10:56:27.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.185+0000 7ff2befb7640 1 -- 172.21.15.89:0/2973895720 wait complete. 2024-08-19T10:56:27.201 INFO:tasks.ceph:Scrubbing osd.2 2024-08-19T10:56:27.201 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-08-19T10:56:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.317+0000 7f482e5c5640 10 monclient: get_monmap_and_config 2024-08-19T10:56:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.317+0000 7f482e5c5640 10 monclient: build_initial_monmap 2024-08-19T10:56:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.317+0000 7f482e5c5640 10 monclient: monmap: 2024-08-19T10:56:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:27.324849+0000 2024-08-19T10:56:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:27.324849+0000 2024-08-19T10:56:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.319 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:27.320 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:27.320 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 1 Processor -- start 2024-08-19T10:56:27.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 1 -- start start 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 10 monclient: init 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 10 monclient: _add_conns ranks=[2,9,3,0,6,10,5,1,4,8,7] 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48280a6860 0x7f48280a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 10 monclient(hunting): picked mon.noname-b con 0x7f48280a6860 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48280a7180 0x7f48280a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 10 monclient(hunting): picked mon.noname-a con 0x7f48280a7180 addr v2:172.21.15.89:3300/0 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4828139620 0x7f4828139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 10 monclient(hunting): picked mon.noname-g con 0x7f4828139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f48280a2f30 con 0x7f48280a7180 2024-08-19T10:56:27.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f48280a33d0 con 0x7f48280a6860 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f48280a5550 con 0x7f4828139620 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482e5c5640 10 monclient(hunting): authenticate will time out at 2037.141427s 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f4827fff640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48280a6860 msgr2=0x7f48280a6c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f4827fff640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48280a6860 0x7f48280a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48280a7180 0x7f48280a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f48280a7180 0x7f48280a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59312/0 (socket says 172.21.15.89:59312) 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 1 -- 172.21.15.89:0/1302464388 learned_addr learned my addr 172.21.15.89:0/1302464388 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 10 monclient(hunting): get_auth_request con 0x7f48280a7180 auth_method 0 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482cb62640 1 --2- 172.21.15.89:0/1302464388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4828139620 0x7f4828139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 10 monclient(hunting): handle_auth_done global_id 7857 payload 274 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 1 -- 172.21.15.89:0/1302464388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4828139620 msgr2=0x7f4828139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 1 --2- 172.21.15.89:0/1302464388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4828139620 0x7f4828139a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 1 -- 172.21.15.89:0/1302464388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48280a6860 msgr2=0x7f48280a6c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 1 --2- 172.21.15.89:0/1302464388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48280a6860 0x7f48280a6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 1 monclient: found mon.noname-a 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:27.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:56:27.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f481f7fe640 1 -- 172.21.15.89:0/1302464388 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48280a4cd0 con 0x7f48280a7180 2024-08-19T10:56:27.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482cb62640 10 monclient: get_auth_request con 0x7f4828139620 auth_method 0 2024-08-19T10:56:27.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.321+0000 7f482cb62640 1 --2- 172.21.15.89:0/1302464388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4828139620 0x7f4828139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:27.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f481f7fe640 1 -- 172.21.15.89:0/1302464388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48280a7180 msgr2=0x7f48280a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:27.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f481f7fe640 1 --2- 172.21.15.89:0/1302464388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48280a7180 0x7f48280a7560 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f48100098a0 tx=0x7f481001ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6eef6d3a9649e1e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 1 -- 172.21.15.89:0/1302464388 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48100b2070 con 0x7f48280a7180 2024-08-19T10:56:27.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:56:27.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 10 monclient: dump: 2024-08-19T10:56:27.329 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.330 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 10 monclient: _finish_auth 0 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 10 monclient: _check_auth_tickets 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 1 -- 172.21.15.89:0/1302464388 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48100ab7b0 con 0x7f48280a7180 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 1 -- 172.21.15.89:0/1302464388 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48100abe10 con 0x7f48280a7180 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 5 monclient: authenticate success, global_id 7857 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 4 set_mon_vals no callback set 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:56:27.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f48277fe640 10 monclient: dump: 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.332 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 10 monclient: shutdown 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 1 -- 172.21.15.89:0/1302464388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48280a7180 msgr2=0x7f48280a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 1 --2- 172.21.15.89:0/1302464388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48280a7180 0x7f48280a7560 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f48100098a0 tx=0x7f481001ce50 comp rx=0 tx=0).stop 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 1 -- 172.21.15.89:0/1302464388 shutdown_connections 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 1 --2- 172.21.15.89:0/1302464388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48280a7180 0x7f48280a7560 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 1 --2- 172.21.15.89:0/1302464388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4828139620 0x7f4828139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 1 --2- 172.21.15.89:0/1302464388 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f48280a6860 0x7f48280a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 1 -- 172.21.15.89:0/1302464388 >> 172.21.15.89:0/1302464388 conn(0x7f4828069a40 msgr2=0x7f48280abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 1 -- 172.21.15.89:0/1302464388 shutdown_connections 2024-08-19T10:56:27.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 1 -- 172.21.15.89:0/1302464388 wait complete. 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 10 monclient: build_initial_monmap 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.325+0000 7f482e5c5640 10 monclient: monmap: 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:27.334 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 1 Processor -- start 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 1 -- start start 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient: init 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient: _add_conns ranks=[1,8,7,3,2,5,0,6,4] 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48280a6860 0x7f482813ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient(hunting): picked mon.noname-f con 0x7f48280a6860 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48280a7180 0x7f48280a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient(hunting): picked mon.noname-e con 0x7f48280a7180 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:27.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4828139620 0x7f48281d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient(hunting): picked mon.noname-i con 0x7f4828139620 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4828143090 con 0x7f48280a6860 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f48280a8560 con 0x7f4828139620 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f4828143210 con 0x7f48280a7180 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient(hunting): authenticate will time out at 2037.148604s 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48280a7180 0x7f48280a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482cb62640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4828139620 msgr2=0x7f48281d14b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:27.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48280a7180 0x7f48280a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50278/0 (socket says 172.21.15.89:50278) 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 1 -- 172.21.15.89:0/3773036894 learned_addr learned my addr 172.21.15.89:0/3773036894 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482cb62640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4828139620 0x7f48281d14b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 10 monclient(hunting): get_auth_request con 0x7f48280a7180 auth_method 0 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f4827fff640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48280a6860 0x7f482813ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 10 monclient(hunting): handle_auth_done global_id 114416 payload 824 2024-08-19T10:56:27.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 1 -- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4828139620 msgr2=0x7f48281d14b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f4828139620 0x7f48281d14b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 1 -- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48280a6860 msgr2=0x7f482813ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48280a6860 0x7f482813ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 10 monclient: _finish_hunting 0 2024-08-19T10:56:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 1 monclient: found mon.noname-e 2024-08-19T10:56:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 1 -- 172.21.15.89:0/3773036894 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482813f680 con 0x7f48280a7180 2024-08-19T10:56:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f4827fff640 10 monclient: get_auth_request con 0x7f48280a6860 auth_method 0 2024-08-19T10:56:27.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f4827fff640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48280a6860 0x7f482813ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 1 -- 172.21.15.89:0/3773036894 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48280a7180 msgr2=0x7f48280a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f481f7fe640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48280a7180 0x7f48280a2020 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f48100b0040 tx=0x7f4810019190 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 1 -- 172.21.15.89:0/3773036894 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48100bb070 con 0x7f48280a7180 2024-08-19T10:56:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 10 monclient: dump: 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.343 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 10 monclient: _finish_auth 0 2024-08-19T10:56:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 10 monclient: _check_auth_tickets 2024-08-19T10:56:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 1 -- 172.21.15.89:0/3773036894 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48100abdd0 con 0x7f48280a7180 2024-08-19T10:56:27.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 1 -- 172.21.15.89:0/3773036894 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f48100b2040 con 0x7f48280a7180 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 5 monclient: authenticate success, global_id 114416 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient: _renew_subs 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 1 -- 172.21.15.89:0/3773036894 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4828139cd0 con 0x7f48280a7180 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f4826ffd640 4 set_mon_vals no callback set 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f48257fa640 10 monclient: dump: 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.344 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.346 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient: _renew_subs 2024-08-19T10:56:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.329+0000 7f482e5c5640 1 -- 172.21.15.89:0/3773036894 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f482809f010 con 0x7f48280a7180 2024-08-19T10:56:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f482e5c5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f482e5c5640 1 -- 172.21.15.89:0/3773036894 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48280a7560 con 0x7f48280a7180 2024-08-19T10:56:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f48257fa640 1 -- 172.21.15.89:0/3773036894 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f48100ab7b0 con 0x7f48280a7180 2024-08-19T10:56:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f48257fa640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f480803bdf0 0x7f480803e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f48257fa640 1 -- 172.21.15.89:0/3773036894 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f48100e8b40 con 0x7f48280a7180 2024-08-19T10:56:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f48257fa640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] conn(0x7f4808044e60 0x7f48080472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f48257fa640 1 -- 172.21.15.89:0/3773036894 --> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48100b3e50 con 0x7f4808044e60 2024-08-19T10:56:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f48257fa640 1 -- 172.21.15.89:0/3773036894 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_get_version_reply(handle=1 version=274) ==== 24+0+0 (secure 0 0 0) 0x7f48100eb320 con 0x7f48280a7180 2024-08-19T10:56:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f48257fa640 10 monclient: handle_get_version_reply finishing 1 version 274 2024-08-19T10:56:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f482cb62640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] conn(0x7f4808044e60 0x7f48080472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f482cb62640 10 monclient: get_auth_request con 0x7f4808044e60 auth_method 0 2024-08-19T10:56:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f4827fff640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f480803bdf0 0x7f480803e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f4827fff640 10 monclient: get_auth_request con 0x7f480803bdf0 auth_method 0 2024-08-19T10:56:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.333+0000 7f4827fff640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f480803bdf0 0x7f480803e2b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4818003990 tx=0x7f4818008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.337+0000 7f482cb62640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] conn(0x7f4808044e60 0x7f48080472c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.337+0000 7f48257fa640 1 -- 172.21.15.89:0/3773036894 <== osd.2 v2:172.21.15.89:6808/1060450292 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48100b3e50 con 0x7f4808044e60 2024-08-19T10:56:27.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 1 -- 172.21.15.89:0/3773036894 --> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4828077450 con 0x7f4808044e60 2024-08-19T10:56:27.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f48257fa640 1 -- 172.21.15.89:0/3773036894 <== osd.2 v2:172.21.15.89:6808/1060450292 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4828077450 con 0x7f4808044e60 2024-08-19T10:56:27.350 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:27.350 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T10:56:27.350 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 1 -- 172.21.15.89:0/3773036894 >> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] conn(0x7f4808044e60 msgr2=0x7f48080472c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.89:6808/1060450292,v1:172.21.15.89:6809/1060450292] conn(0x7f4808044e60 0x7f48080472c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 1 -- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f480803bdf0 msgr2=0x7f480803e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f480803bdf0 0x7f480803e2b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4818003990 tx=0x7f4818008040 comp rx=0 tx=0).stop 2024-08-19T10:56:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 10 monclient: shutdown 2024-08-19T10:56:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 1 -- 172.21.15.89:0/3773036894 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48280a7180 msgr2=0x7f48280a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 1 --2- 172.21.15.89:0/3773036894 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f48280a7180 0x7f48280a2020 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f48100b0040 tx=0x7f4810019190 comp rx=0 tx=0).stop 2024-08-19T10:56:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482cb62640 1 -- 172.21.15.89:0/3773036894 reap_dead start 2024-08-19T10:56:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 1 -- 172.21.15.89:0/3773036894 shutdown_connections 2024-08-19T10:56:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 1 -- 172.21.15.89:0/3773036894 >> 172.21.15.89:0/3773036894 conn(0x7f4828069a40 msgr2=0x7f48280ab2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.349+0000 7f482e5c5640 1 -- 172.21.15.89:0/3773036894 shutdown_connections 2024-08-19T10:56:27.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.353+0000 7f482e5c5640 1 -- 172.21.15.89:0/3773036894 wait complete. 2024-08-19T10:56:27.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-08-19T10:56:27.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient: get_monmap_and_config 2024-08-19T10:56:27.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient: build_initial_monmap 2024-08-19T10:56:27.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient: monmap: 2024-08-19T10:56:27.480 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:27.480 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.480 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:27.485929+0000 2024-08-19T10:56:27.480 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:27.485929+0000 2024-08-19T10:56:27.480 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:27.480 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.480 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:27.481 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:27.481 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:27.481 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:27.481 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:27.481 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:27.481 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:27.481 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:27.481 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:27.481 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:27.481 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:27.481 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 1 Processor -- start 2024-08-19T10:56:27.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 1 -- start start 2024-08-19T10:56:27.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient: init 2024-08-19T10:56:27.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient: _add_conns ranks=[3,8,0,6,5,4,9,1,2,10,7] 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf00a6860 0x7fbaf00a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient(hunting): picked mon.noname-g con 0x7fbaf00a6860 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbaf00a7180 0x7fbaf00a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient(hunting): picked mon.noname-e con 0x7fbaf00a7180 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbaf0139620 addr v1:172.21.15.89:6789/0 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaf00a33d0 con 0x7fbaf0139620 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf00a8190 con 0x7fbaf00a6860 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7fbaf00a2ec0 con 0x7fbaf00a7180 2024-08-19T10:56:27.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf7ebd640 10 monclient(hunting): authenticate will time out at 2037.302504s 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf645a640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbaf0139620 0x7fbaf0139a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54344/0 (socket says 172.21.15.89:54344) 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf645a640 1 -- 172.21.15.89:0/2624222258 learned_addr learned my addr 172.21.15.89:0/2624222258 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbaf00a7180 0x7fbaf00a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 10 monclient(hunting): get_auth_request con 0x7fbaf00a7180 auth_method 0 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5c59640 1 --2- 172.21.15.89:0/2624222258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf00a6860 0x7fbaf00a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 10 monclient(hunting): handle_auth_done global_id 114425 payload 274 2024-08-19T10:56:27.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 1 -- 172.21.15.89:0/2624222258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf00a6860 msgr2=0x7fbaf00a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:27.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 1 --2- 172.21.15.89:0/2624222258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf00a6860 0x7fbaf00a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 1 -- 172.21.15.89:0/2624222258 >> v1:172.21.15.89:6789/0 conn(0x7fbaf0139620 legacy=0x7fbaf0139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:27.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 10 monclient: _finish_hunting 0 2024-08-19T10:56:27.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 1 monclient: found mon.noname-e 2024-08-19T10:56:27.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:27.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:27.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5458640 1 -- 172.21.15.89:0/2624222258 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf00a4cd0 con 0x7fbaf00a7180 2024-08-19T10:56:27.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5c59640 10 monclient: get_auth_request con 0x7fbaf00a6860 auth_method 0 2024-08-19T10:56:27.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.481+0000 7fbaf5c59640 1 --2- 172.21.15.89:0/2624222258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf00a6860 0x7fbaf00a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:27.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.485+0000 7fbaf5458640 1 -- 172.21.15.89:0/2624222258 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbaf00a7180 msgr2=0x7fbaf00a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:27.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.485+0000 7fbaf5458640 1 --2- 172.21.15.89:0/2624222258 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbaf00a7180 0x7fbaf00a7560 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbae00097f0 tx=0x7fbae001c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=af256d3c47ba49d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.485+0000 7fbaf4c57640 1 -- 172.21.15.89:0/2624222258 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbae00b0070 con 0x7fbaf00a7180 2024-08-19T10:56:27.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.485+0000 7fbaf4c57640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf4c57640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf4c57640 10 monclient: dump: 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.489 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf4c57640 10 monclient: _finish_auth 0 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf4c57640 10 monclient: _check_auth_tickets 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf4c57640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf4c57640 1 -- 172.21.15.89:0/2624222258 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae00aa810 con 0x7fbaf00a7180 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf4c57640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf4c57640 1 -- 172.21.15.89:0/2624222258 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbae00a93e0 con 0x7fbaf00a7180 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 5 monclient: authenticate success, global_id 114425 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:27.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 4 set_mon_vals no callback set 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf4c57640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf4c57640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf4c57640 10 monclient: dump: 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.491 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient: shutdown 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/2624222258 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbaf00a7180 msgr2=0x7fbaf00a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 --2- 172.21.15.89:0/2624222258 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbaf00a7180 0x7fbaf00a7560 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbae00097f0 tx=0x7fbae001c3a0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/2624222258 shutdown_connections 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 --2- 172.21.15.89:0/2624222258 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7fbaf00a7180 0x7fbaf00a7560 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 --2- 172.21.15.89:0/2624222258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf00a6860 0x7fbaf00a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/2624222258 >> 172.21.15.89:0/2624222258 conn(0x7fbaf0074b40 msgr2=0x7fbaf00abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/2624222258 shutdown_connections 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/2624222258 wait complete. 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient: build_initial_monmap 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient: monmap: 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:27.493 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:27.494 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:27.494 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:27.494 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:27.494 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:27.494 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:27.494 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 Processor -- start 2024-08-19T10:56:27.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 -- start start 2024-08-19T10:56:27.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient: init 2024-08-19T10:56:27.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:27.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient: _add_conns ranks=[5,4,3,6,8,0,1,7,2] 2024-08-19T10:56:27.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbaf00a6860 0x7fbaf01d1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient(hunting): picked mon.noname-h con 0x7fbaf00a6860 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:27.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbaf00a7180 0x7fbaf01d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient(hunting): picked mon.noname-c con 0x7fbaf00a7180 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:27.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf0139620 0x7fbaf013d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient(hunting): picked mon.noname-g con 0x7fbaf0139620 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:27.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf0144430 con 0x7fbaf0139620 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbaf01442b0 con 0x7fbaf00a7180 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fbaf01445b0 con 0x7fbaf00a6860 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf7ebd640 10 monclient(hunting): authenticate will time out at 2037.310262s 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbaf00a7180 0x7fbaf01d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbaf00a7180 0x7fbaf01d1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45920/0 (socket says 172.21.15.89:45920) 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 1 -- 172.21.15.89:0/3174475523 learned_addr learned my addr 172.21.15.89:0/3174475523 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 10 monclient(hunting): get_auth_request con 0x7fbaf00a7180 auth_method 0 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5c59640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbaf00a6860 0x7fbaf01d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5c59640 10 monclient(hunting): get_auth_request con 0x7fbaf00a6860 auth_method 0 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5c59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5c59640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5c59640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf645a640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf0139620 0x7fbaf013d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 10 monclient(hunting): handle_auth_done global_id 84604 payload 824 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 1 -- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbaf00a6860 msgr2=0x7fbaf01d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbaf00a6860 0x7fbaf01d1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 1 -- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf0139620 msgr2=0x7fbaf013d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf0139620 0x7fbaf013d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 10 monclient: _finish_hunting 0 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 1 monclient: found mon.noname-c 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf5458640 1 -- 172.21.15.89:0/3174475523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf0122790 con 0x7fbaf00a7180 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf645a640 10 monclient: get_auth_request con 0x7fbaf0139620 auth_method 0 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.489+0000 7fbaf645a640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf0139620 0x7fbaf013d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf5458640 1 -- 172.21.15.89:0/3174475523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbaf00a7180 msgr2=0x7fbaf01d1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf5458640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbaf00a7180 0x7fbaf01d1c90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbae001c1d0 tx=0x7fbae0019140 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 1 -- 172.21.15.89:0/3174475523 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbae00b0040 con 0x7fbaf00a7180 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 10 monclient: dump: 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.502 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 10 monclient: _finish_auth 0 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 10 monclient: _check_auth_tickets 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf7ebd640 5 monclient: authenticate success, global_id 84604 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 1 -- 172.21.15.89:0/3174475523 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae00b9070 con 0x7fbaf00a7180 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 1 -- 172.21.15.89:0/3174475523 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbae00044b0 con 0x7fbaf00a7180 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf7ebd640 10 monclient: _renew_subs 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf7ebd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:27.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/3174475523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf0122a70 con 0x7fbaf00a7180 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae7fff640 4 set_mon_vals no callback set 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 10 monclient: dump: 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.504 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf7ebd640 10 monclient: _renew_subs 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf7ebd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/3174475523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf01224b0 con 0x7fbaf00a7180 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf7ebd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf7ebd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf7ebd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:27.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/3174475523 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf00a6c40 con 0x7fbaf00a7180 2024-08-19T10:56:27.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 1 -- 172.21.15.89:0/3174475523 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbae0004040 con 0x7fbaf00a7180 2024-08-19T10:56:27.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbad403be00 0x7fbad403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbae67fc640 1 -- 172.21.15.89:0/3174475523 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fbae00e93b0 con 0x7fbaf00a7180 2024-08-19T10:56:27.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf5c59640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbad403be00 0x7fbad403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.493+0000 7fbaf5c59640 10 monclient: get_auth_request con 0x7fbad403be00 auth_method 0 2024-08-19T10:56:27.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.497+0000 7fbaf5c59640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbad403be00 0x7fbad403e2c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00a9a0 tx=0x7fbaec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.497+0000 7fbae67fc640 1 -- 172.21.15.89:0/3174475523 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fbae00160b0 con 0x7fbaf00a7180 2024-08-19T10:56:27.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.497+0000 7fbae67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:27.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.497+0000 7fbae67fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:27.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.669+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/3174475523 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fbaf013ef70 con 0x7fbad403be00 2024-08-19T10:56:27.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.669+0000 7fbae67fc640 1 -- 172.21.15.89:0/3174475523 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbaf013ef70 con 0x7fbad403be00 2024-08-19T10:56:27.671 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 2 to deep-scrub 2024-08-19T10:56:27.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbad403be00 msgr2=0x7fbad403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbad403be00 0x7fbad403e2c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00a9a0 tx=0x7fbaec008040 comp rx=0 tx=0).stop 2024-08-19T10:56:27.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 10 monclient: shutdown 2024-08-19T10:56:27.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:27.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/3174475523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbaf00a7180 msgr2=0x7fbaf01d1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbaf00a7180 0x7fbaf01d1c90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbae001c1d0 tx=0x7fbae0019140 comp rx=0 tx=0).stop 2024-08-19T10:56:27.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/3174475523 shutdown_connections 2024-08-19T10:56:27.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbad403be00 0x7fbad403e2c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaf0139620 0x7fbaf013d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbaf00a7180 0x7fbaf01d1c90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 --2- 172.21.15.89:0/3174475523 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbaf00a6860 0x7fbaf01d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/3174475523 >> 172.21.15.89:0/3174475523 conn(0x7fbaf0074b40 msgr2=0x7fbaf00a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:27.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/3174475523 shutdown_connections 2024-08-19T10:56:27.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.673+0000 7fbaf7ebd640 1 -- 172.21.15.89:0/3174475523 wait complete. 2024-08-19T10:56:27.690 INFO:tasks.ceph:Scrubbing osd.3 2024-08-19T10:56:27.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-08-19T10:56:27.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient: get_monmap_and_config 2024-08-19T10:56:27.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient: build_initial_monmap 2024-08-19T10:56:27.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient: monmap: 2024-08-19T10:56:27.808 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:27.808 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.808 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:27.813623+0000 2024-08-19T10:56:27.808 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:27.813623+0000 2024-08-19T10:56:27.808 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:27.808 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.808 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:27.808 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:27.809 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:27.809 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:27.809 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:27.809 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:27.809 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:27.809 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:27.809 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:27.809 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:27.809 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:27.809 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 1 Processor -- start 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 1 -- start start 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient: init 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient: _add_conns ranks=[7,4,8,9,1,3,0,5,10,2,6] 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7eff500a67d0 0x7eff501397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient(hunting): picked mon.noname-i con 0x7eff500a67d0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff50139d20 0x7eff5013a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient(hunting): picked mon.noname-c con 0x7eff50139d20 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7eff500a6bb0 0x7eff500a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient(hunting): picked mon.noname-e con 0x7eff500a6bb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7eff5013a640 con 0x7eff50139d20 2024-08-19T10:56:27.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7eff500a4d10 con 0x7eff500a67d0 2024-08-19T10:56:27.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7eff500a82a0 con 0x7eff500a6bb0 2024-08-19T10:56:27.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:27.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff54c7f640 10 monclient(hunting): authenticate will time out at 2037.630162s 2024-08-19T10:56:27.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4e59c640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7eff500a67d0 msgr2=0x7eff501397e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:27.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff50139d20 0x7eff5013a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4e59c640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7eff500a67d0 0x7eff501397e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:27.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4ed9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7eff500a6bb0 0x7eff500a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4ed9d640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7eff500a6bb0 0x7eff500a6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50312/0 (socket says 172.21.15.89:50312) 2024-08-19T10:56:27.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff50139d20 0x7eff5013a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45934/0 (socket says 172.21.15.89:45934) 2024-08-19T10:56:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4ed9d640 1 -- 172.21.15.89:0/3187954927 learned_addr learned my addr 172.21.15.89:0/3187954927 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4ed9d640 10 monclient(hunting): get_auth_request con 0x7eff500a6bb0 auth_method 0 2024-08-19T10:56:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 10 monclient(hunting): get_auth_request con 0x7eff50139d20 auth_method 0 2024-08-19T10:56:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 10 monclient(hunting): handle_auth_done global_id 84613 payload 274 2024-08-19T10:56:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 1 -- 172.21.15.89:0/3187954927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7eff500a6bb0 msgr2=0x7eff500a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 1 --2- 172.21.15.89:0/3187954927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7eff500a6bb0 0x7eff500a6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 1 -- 172.21.15.89:0/3187954927 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7eff500a67d0 msgr2=0x7eff501397e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 1 --2- 172.21.15.89:0/3187954927 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7eff500a67d0 0x7eff501397e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 10 monclient: _finish_hunting 0 2024-08-19T10:56:27.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 1 monclient: found mon.noname-c 2024-08-19T10:56:27.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:27.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:56:27.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4dd9b640 1 -- 172.21.15.89:0/3187954927 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff500a2eb0 con 0x7eff50139d20 2024-08-19T10:56:27.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.809+0000 7eff4ed9d640 1 --2- 172.21.15.89:0/3187954927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7eff500a6bb0 0x7eff500a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4dd9b640 1 -- 172.21.15.89:0/3187954927 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff50139d20 msgr2=0x7eff5013a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4dd9b640 1 --2- 172.21.15.89:0/3187954927 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff50139d20 0x7eff5013a100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7eff40009dc0 tx=0x7eff4001d2a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=daa9f7c12eed1338 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 1 -- 172.21.15.89:0/3187954927 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff400b2070 con 0x7eff50139d20 2024-08-19T10:56:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:56:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 10 monclient: dump: 2024-08-19T10:56:27.824 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.825 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 10 monclient: _finish_auth 0 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 10 monclient: _check_auth_tickets 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 5 monclient: authenticate success, global_id 84613 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 1 -- 172.21.15.89:0/3187954927 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff400a5ab0 con 0x7eff50139d20 2024-08-19T10:56:27.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 4 set_mon_vals no callback set 2024-08-19T10:56:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 1 -- 172.21.15.89:0/3187954927 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff400a5c60 con 0x7eff50139d20 2024-08-19T10:56:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:56:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff4d59a640 10 monclient: dump: 2024-08-19T10:56:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.827 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.829 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.830 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 10 monclient: shutdown 2024-08-19T10:56:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 1 -- 172.21.15.89:0/3187954927 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff50139d20 msgr2=0x7eff5013a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 1 --2- 172.21.15.89:0/3187954927 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff50139d20 0x7eff5013a100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7eff40009dc0 tx=0x7eff4001d2a0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 1 -- 172.21.15.89:0/3187954927 shutdown_connections 2024-08-19T10:56:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 1 --2- 172.21.15.89:0/3187954927 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7eff500a6bb0 0x7eff500a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 1 --2- 172.21.15.89:0/3187954927 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7eff50139d20 0x7eff5013a100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.813+0000 7eff54c7f640 1 --2- 172.21.15.89:0/3187954927 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7eff500a67d0 0x7eff501397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 1 -- 172.21.15.89:0/3187954927 >> 172.21.15.89:0/3187954927 conn(0x7eff500abdd0 msgr2=0x7eff500ac1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 1 -- 172.21.15.89:0/3187954927 shutdown_connections 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 1 -- 172.21.15.89:0/3187954927 wait complete. 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient: build_initial_monmap 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient: monmap: 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:27.831 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 1 Processor -- start 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 1 -- start start 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient: init 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient: _add_conns ranks=[8,0,6,1,4,7,5,2,3] 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7eff500a67d0 0x7eff501361f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient(hunting): picked mon.noname-e con 0x7eff500a67d0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff500a6bb0 0x7eff50136730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient(hunting): picked mon.noname-a con 0x7eff500a6bb0 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7eff50139d20 0x7eff5009e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient(hunting): picked mon.noname-d con 0x7eff50139d20 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7eff501432e0 con 0x7eff500a6bb0 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7eff500a7b50 con 0x7eff50139d20 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7eff50143460 con 0x7eff500a67d0 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff54c7f640 10 monclient(hunting): authenticate will time out at 2037.638131s 2024-08-19T10:56:27.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7eff50139d20 0x7eff5009e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4e59c640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7eff500a67d0 0x7eff501361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7eff50139d20 0x7eff5009e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:44206/0 (socket says 172.21.15.89:44206) 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 1 -- 172.21.15.89:0/387844397 learned_addr learned my addr 172.21.15.89:0/387844397 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 10 monclient(hunting): get_auth_request con 0x7eff50139d20 auth_method 0 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4e59c640 10 monclient(hunting): get_auth_request con 0x7eff500a67d0 auth_method 0 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4e59c640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4e59c640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4dd9b640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff500a6bb0 0x7eff50136730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4dd9b640 10 monclient(hunting): get_auth_request con 0x7eff500a6bb0 auth_method 0 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4dd9b640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 10 monclient(hunting): handle_auth_done global_id 45645 payload 824 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 1 -- 172.21.15.89:0/387844397 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7eff500a67d0 msgr2=0x7eff501361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7eff500a67d0 0x7eff501361f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 1 -- 172.21.15.89:0/387844397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff500a6bb0 msgr2=0x7eff50136730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff500a6bb0 0x7eff50136730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 10 monclient: _finish_hunting 0 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 1 monclient: found mon.noname-d 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.817+0000 7eff4ed9d640 1 -- 172.21.15.89:0/387844397 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff500a23d0 con 0x7eff50139d20 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff4ed9d640 1 -- 172.21.15.89:0/387844397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7eff50139d20 msgr2=0x7eff5009e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff4ed9d640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7eff50139d20 0x7eff5009e860 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e930 tx=0x7eff4800edf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 1 -- 172.21.15.89:0/387844397 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff48017070 con 0x7eff50139d20 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 10 monclient: dump: 2024-08-19T10:56:27.836 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.837 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.838 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.838 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.838 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 10 monclient: _finish_auth 0 2024-08-19T10:56:27.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:27.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:27.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 1 -- 172.21.15.89:0/387844397 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4800cc10 con 0x7eff50139d20 2024-08-19T10:56:27.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:27.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 1 -- 172.21.15.89:0/387844397 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7eff4800cdc0 con 0x7eff50139d20 2024-08-19T10:56:27.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:27.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 10 monclient: dump: 2024-08-19T10:56:27.840 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.840 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.840 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.840 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.840 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.840 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.840 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.840 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.840 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.841 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff4cd99640 4 set_mon_vals no callback set 2024-08-19T10:56:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff54c7f640 5 monclient: authenticate success, global_id 45645 2024-08-19T10:56:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff54c7f640 10 monclient: _renew_subs 2024-08-19T10:56:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff54c7f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff54c7f640 1 -- 172.21.15.89:0/387844397 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff500ac2d0 con 0x7eff50139d20 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff54c7f640 10 monclient: _renew_subs 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff54c7f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff54c7f640 1 -- 172.21.15.89:0/387844397 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7eff501372e0 con 0x7eff50139d20 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff54c7f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff54c7f640 1 -- 172.21.15.89:0/387844397 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff500a6ff0 con 0x7eff50139d20 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 1 -- 172.21.15.89:0/387844397 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7eff4801a070 con 0x7eff50139d20 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7eff3003be00 0x7eff3003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 1 -- 172.21.15.89:0/387844397 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7eff4800f040 con 0x7eff50139d20 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] conn(0x7eff30044ef0 0x7eff30047350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 1 -- 172.21.15.89:0/387844397 --> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff48021ec0 con 0x7eff30044ef0 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 1 -- 172.21.15.89:0/387844397 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_get_version_reply(handle=1 version=274) ==== 24+0+0 (secure 0 0 0) 0x7eff48056050 con 0x7eff50139d20 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff3effd640 10 monclient: handle_get_version_reply finishing 1 version 274 2024-08-19T10:56:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff4dd9b640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] conn(0x7eff30044ef0 0x7eff30047350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff4e59c640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7eff3003be00 0x7eff3003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff4dd9b640 10 monclient: get_auth_request con 0x7eff30044ef0 auth_method 0 2024-08-19T10:56:27.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.821+0000 7eff4e59c640 10 monclient: get_auth_request con 0x7eff3003be00 auth_method 0 2024-08-19T10:56:27.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.825+0000 7eff4e59c640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7eff3003be00 0x7eff3003e2c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7eff440093d0 tx=0x7eff44009360 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.829+0000 7eff4dd9b640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] conn(0x7eff30044ef0 0x7eff30047350 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.829+0000 7eff3effd640 1 -- 172.21.15.89:0/387844397 <== osd.3 v2:172.21.15.190:6810/2751619171 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff48021ec0 con 0x7eff30044ef0 2024-08-19T10:56:27.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 1 -- 172.21.15.89:0/387844397 --> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7eff50064590 con 0x7eff30044ef0 2024-08-19T10:56:27.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff3effd640 1 -- 172.21.15.89:0/387844397 <== osd.3 v2:172.21.15.190:6810/2751619171 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7eff50064590 con 0x7eff30044ef0 2024-08-19T10:56:27.844 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:27.844 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T10:56:27.844 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 1 -- 172.21.15.89:0/387844397 >> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] conn(0x7eff30044ef0 msgr2=0x7eff30047350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.190:6810/2751619171,v1:172.21.15.190:6811/2751619171] conn(0x7eff30044ef0 0x7eff30047350 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 1 -- 172.21.15.89:0/387844397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7eff3003be00 msgr2=0x7eff3003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7eff3003be00 0x7eff3003e2c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7eff440093d0 tx=0x7eff44009360 comp rx=0 tx=0).stop 2024-08-19T10:56:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 10 monclient: shutdown 2024-08-19T10:56:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 1 -- 172.21.15.89:0/387844397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7eff50139d20 msgr2=0x7eff5009e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 1 --2- 172.21.15.89:0/387844397 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7eff50139d20 0x7eff5009e860 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7eff4800e930 tx=0x7eff4800edf0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff4ed9d640 1 -- 172.21.15.89:0/387844397 reap_dead start 2024-08-19T10:56:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 1 -- 172.21.15.89:0/387844397 shutdown_connections 2024-08-19T10:56:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 1 -- 172.21.15.89:0/387844397 >> 172.21.15.89:0/387844397 conn(0x7eff500abdd0 msgr2=0x7eff500a8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 1 -- 172.21.15.89:0/387844397 shutdown_connections 2024-08-19T10:56:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.841+0000 7eff54c7f640 1 -- 172.21.15.89:0/387844397 wait complete. 2024-08-19T10:56:27.857 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-08-19T10:56:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient: get_monmap_and_config 2024-08-19T10:56:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient: build_initial_monmap 2024-08-19T10:56:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient: monmap: 2024-08-19T10:56:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:27.973902+0000 2024-08-19T10:56:27.968 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:27.973902+0000 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:27.969 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:27.970 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:27.970 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 1 Processor -- start 2024-08-19T10:56:27.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 1 -- start start 2024-08-19T10:56:27.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient: init 2024-08-19T10:56:27.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:27.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient: _add_conns ranks=[0,7,1,3,6,2,10,5,9,4,8] 2024-08-19T10:56:27.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f66dc06e050 addr v1:172.21.15.89:6789/0 2024-08-19T10:56:27.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc073ee0 0x7f66dc13e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient(hunting): picked mon.noname-i con 0x7f66dc073ee0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:27.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f66dc074320 addr v1:172.21.15.190:6789/0 2024-08-19T10:56:27.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66dc136980 con 0x7f66dc06e050 2024-08-19T10:56:27.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66dc06f190 con 0x7f66dc074320 2024-08-19T10:56:27.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f66dc06f470 con 0x7f66dc073ee0 2024-08-19T10:56:27.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:27.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66e2637640 10 monclient(hunting): authenticate will time out at 2037.790501s 2024-08-19T10:56:27.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66dbfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f66dc06e050 0x7f66dc136270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54358/0 (socket says 172.21.15.89:54358) 2024-08-19T10:56:27.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66dbfff640 1 -- 172.21.15.89:0/2564201094 learned_addr learned my addr 172.21.15.89:0/2564201094 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:56:27.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66db7fe640 1 -- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc073ee0 msgr2=0x7f66dc13e4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:27.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.969+0000 7f66db7fe640 1 --2- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc073ee0 0x7f66dc13e4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:27.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 1 -- 172.21.15.89:0/2564201094 >> v1:172.21.15.89:6789/0 conn(0x7f66dc06e050 legacy=0x7f66dc136270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:27.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029144678 0 0) 0x7f66dc136980 con 0x7f66dc06e050 2024-08-19T10:56:27.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): _negotiate 2024-08-19T10:56:27.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:27.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): my global_id is 7875 2024-08-19T10:56:27.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66c8003530 con 0x7f66dc06e050 2024-08-19T10:56:27.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66e0bd4640 1 -- 172.21.15.89:0/2564201094 >> v1:172.21.15.190:6789/0 conn(0x7f66dc074320 legacy=0x7f66dc140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:27.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 <== mon.0 v1:172.21.15.89:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1483878993 0 0) 0x7f66c8003530 con 0x7f66dc06e050 2024-08-19T10:56:27.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc073ee0 msgr2=0x7f66dc13e4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:27.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 --2- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc073ee0 0x7f66dc13e4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 >> v1:172.21.15.190:6789/0 conn(0x7f66dc074320 legacy=0x7f66dc140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient: _finish_hunting 0 2024-08-19T10:56:27.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 monclient: found mon.noname-a-legacy 2024-08-19T10:56:27.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:27.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.89:6789/0 2024-08-19T10:56:27.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 --> v1:172.21.15.89:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc06f750 con 0x7f66dc06e050 2024-08-19T10:56:27.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient: _finish_auth 0 2024-08-19T10:56:27.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:27.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:27.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 <== mon.0 v1:172.21.15.89:6789/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (unknown 3927028378 0 0) 0x7f66cc001ce0 con 0x7f66dc06e050 2024-08-19T10:56:27.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-08-19T10:56:27.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient: dump: 2024-08-19T10:56:27.984 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.984 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.985 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.985 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.985 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.985 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.985 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.985 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.986 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.986 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.986 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.986 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.986 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.986 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.987 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.987 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] but i'm connected to v1:172.21.15.89:6789/0, reconnecting 2024-08-19T10:56:27.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:27.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 >> v1:172.21.15.89:6789/0 conn(0x7f66dc06e050 legacy=0x7f66dc136270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:27.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient: _add_conns ranks=[6,7,5,3,8,0,4,1,2] 2024-08-19T10:56:27.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 --2- 172.21.15.89:0/2564201094 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 0x7f66c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): picked mon.d con 0x7f66c8004240 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:27.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 --2- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc073ee0 0x7f66c8004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): picked mon.i con 0x7f66dc073ee0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:27.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 --2- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66c80099c0 0x7f66c800bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): picked mon.h con 0x7f66c80099c0 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f66dc06f470 con 0x7f66c80099c0 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f66c800c590 con 0x7f66c8004240 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f66c800c740 con 0x7f66dc073ee0 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66daffd640 10 monclient(hunting): _finish_auth 0 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66db7fe640 1 -- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc073ee0 msgr2=0x7f66c8004fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66db7fe640 1 --2- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc073ee0 0x7f66c8004fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 1 --2- 172.21.15.89:0/2564201094 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 0x7f66c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:27.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient(hunting): get_auth_request con 0x7f66c8004240 auth_method 0 2024-08-19T10:56:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-08-19T10:56:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-08-19T10:56:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient(hunting): handle_auth_done global_id 7875 payload 293 2024-08-19T10:56:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 1 -- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc073ee0 msgr2=0x7f66c8004fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 1 --2- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc073ee0 0x7f66c8004fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 1 -- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66c80099c0 msgr2=0x7f66c800bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:27.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 1 --2- 172.21.15.89:0/2564201094 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f66c80099c0 0x7f66c800bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:27.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient: _finish_hunting 0 2024-08-19T10:56:27.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 1 monclient: found mon.d 2024-08-19T10:56:27.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:27.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:27.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 1 -- 172.21.15.89:0/2564201094 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c8003530 con 0x7f66c8004240 2024-08-19T10:56:27.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient: _finish_auth 0 2024-08-19T10:56:27.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 10 monclient: _check_auth_tickets 2024-08-19T10:56:27.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:27.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66e2637640 5 monclient: authenticate success, global_id 7875 2024-08-19T10:56:27.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.973+0000 7f66e2637640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:27.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66dbfff640 1 -- 172.21.15.89:0/2564201094 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 msgr2=0x7f66c8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:27.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66dbfff640 1 --2- 172.21.15.89:0/2564201094 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 0x7f66c8004810 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0014e0 tx=0x7f66cc01cb20 comp rx=0 tx=0).ready entity=mon.6 client_cookie=37affb8aa697939a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:27.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66cc008ca0 con 0x7f66c8004240 2024-08-19T10:56:27.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66daffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:27.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66daffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66daffd640 10 monclient: dump: 2024-08-19T10:56:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:27.998 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66cc005d50 con 0x7f66c8004240 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66daffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 4 set_mon_vals no callback set 2024-08-19T10:56:27.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66daffd640 1 -- 172.21.15.89:0/2564201094 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66cc008620 con 0x7f66c8004240 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66daffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66daffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66daffd640 10 monclient: dump: 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.000 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 10 monclient: shutdown 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 1 -- 172.21.15.89:0/2564201094 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 msgr2=0x7f66c8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 1 --2- 172.21.15.89:0/2564201094 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 0x7f66c8004810 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0014e0 tx=0x7f66cc01cb20 comp rx=0 tx=0).stop 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e0bd4640 1 -- 172.21.15.89:0/2564201094 reap_dead start 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 1 -- 172.21.15.89:0/2564201094 shutdown_connections 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 1 -- 172.21.15.89:0/2564201094 >> 172.21.15.89:0/2564201094 conn(0x7f66dc0aa240 msgr2=0x7f66dc0aa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 1 -- 172.21.15.89:0/2564201094 shutdown_connections 2024-08-19T10:56:28.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 1 -- 172.21.15.89:0/2564201094 wait complete. 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 10 monclient: build_initial_monmap 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 10 monclient: monmap: 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:28.002 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:28.003 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 1 Processor -- start 2024-08-19T10:56:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 1 -- start start 2024-08-19T10:56:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.977+0000 7f66e2637640 10 monclient: init 2024-08-19T10:56:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:28.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient: _add_conns ranks=[6,7,2,5,0,3,1,4,8] 2024-08-19T10:56:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 0x7f66dc1d1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient(hunting): picked mon.noname-d con 0x7f66c8004240 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc13b3e0 0x7f66dc13b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient(hunting): picked mon.noname-i con 0x7f66dc13b3e0 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66dc0a1850 0x7f66dc0a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient(hunting): picked mon.noname-b con 0x7f66dc0a1850 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:28.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f66dc142cc0 con 0x7f66dc0a1850 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7f66dc142b40 con 0x7f66c8004240 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f66dc142e40 con 0x7f66dc13b3e0 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient(hunting): authenticate will time out at 2037.800293s 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e0bd4640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66dc0a1850 msgr2=0x7f66dc0a1cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e0bd4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66dc0a1850 0x7f66dc0a1cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 0x7f66dc1d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 0x7f66dc1d1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:44216/0 (socket says 172.21.15.89:44216) 2024-08-19T10:56:28.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 1 -- 172.21.15.89:0/1631838699 learned_addr learned my addr 172.21.15.89:0/1631838699 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:28.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 10 monclient(hunting): get_auth_request con 0x7f66c8004240 auth_method 0 2024-08-19T10:56:28.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66db7fe640 1 -- 172.21.15.89:0/1631838699 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc13b3e0 msgr2=0x7f66dc13b800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:28.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66db7fe640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc13b3e0 0x7f66dc13b800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:28.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 10 monclient(hunting): handle_auth_done global_id 45654 payload 824 2024-08-19T10:56:28.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 1 -- 172.21.15.89:0/1631838699 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc13b3e0 msgr2=0x7f66dc13b800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:28.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc13b3e0 0x7f66dc13b800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 1 -- 172.21.15.89:0/1631838699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66dc0a1850 msgr2=0x7f66dc0a1cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:28.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66dc0a1850 0x7f66dc0a1cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 10 monclient: _finish_hunting 0 2024-08-19T10:56:28.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 1 monclient: found mon.noname-d 2024-08-19T10:56:28.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:28.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:56:28.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 1 -- 172.21.15.89:0/1631838699 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc1419e0 con 0x7f66c8004240 2024-08-19T10:56:28.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 1 -- 172.21.15.89:0/1631838699 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 msgr2=0x7f66dc1d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:28.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66dbfff640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 0x7f66dc1d1230 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f66cc000eb0 tx=0x7f66c8005840 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 1 -- 172.21.15.89:0/1631838699 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66cc008ca0 con 0x7f66c8004240 2024-08-19T10:56:28.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:56:28.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 10 monclient: dump: 2024-08-19T10:56:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.009 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.010 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 10 monclient: _finish_auth 0 2024-08-19T10:56:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:56:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 5 monclient: authenticate success, global_id 45654 2024-08-19T10:56:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient: _renew_subs 2024-08-19T10:56:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 1 -- 172.21.15.89:0/1631838699 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66cc001cc0 con 0x7f66c8004240 2024-08-19T10:56:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 1 -- 172.21.15.89:0/1631838699 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66dc141ee0 con 0x7f66c8004240 2024-08-19T10:56:28.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 1 -- 172.21.15.89:0/1631838699 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f66cc0a5ad0 con 0x7f66c8004240 2024-08-19T10:56:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66da7fc640 4 set_mon_vals no callback set 2024-08-19T10:56:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66d8ff9640 10 monclient: dump: 2024-08-19T10:56:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.012 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.013 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.014 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient: _renew_subs 2024-08-19T10:56:28.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:28.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.981+0000 7f66e2637640 1 -- 172.21.15.89:0/1631838699 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({osdmap=0}) -- 0x7f66dc137e90 con 0x7f66c8004240 2024-08-19T10:56:28.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.985+0000 7f66d8ff9640 1 -- 172.21.15.89:0/1631838699 <== mon.6 v2:172.21.15.89:3303/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f66cc0a5c70 con 0x7f66c8004240 2024-08-19T10:56:28.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.985+0000 7f66d8ff9640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f66bc03be30 0x7f66bc03e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.985+0000 7f66d8ff9640 1 -- 172.21.15.89:0/1631838699 <== mon.6 v2:172.21.15.89:3303/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f66cc0ed060 con 0x7f66c8004240 2024-08-19T10:56:28.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.985+0000 7f66e2637640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:28.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.985+0000 7f66e2637640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:28.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.985+0000 7f66e2637640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.89:3303/0 2024-08-19T10:56:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.985+0000 7f66e2637640 1 -- 172.21.15.89:0/1631838699 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66dc134260 con 0x7f66c8004240 2024-08-19T10:56:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.985+0000 7f66db7fe640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f66bc03be30 0x7f66bc03e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.985+0000 7f66db7fe640 10 monclient: get_auth_request con 0x7f66bc03be30 auth_method 0 2024-08-19T10:56:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.985+0000 7f66db7fe640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f66bc03be30 0x7f66bc03e2f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0a37a0 tx=0x7f66d4009290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.989+0000 7f66d8ff9640 1 -- 172.21.15.89:0/1631838699 <== mon.6 v2:172.21.15.89:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f66dc134260 con 0x7f66c8004240 2024-08-19T10:56:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.989+0000 7f66d8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:28.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:27.989+0000 7f66d8ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:28.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.161+0000 7f66e2637640 1 -- 172.21.15.89:0/1631838699 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f66dc136270 con 0x7f66bc03be30 2024-08-19T10:56:28.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.161+0000 7f66d8ff9640 1 -- 172.21.15.89:0/1631838699 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f66dc136270 con 0x7f66bc03be30 2024-08-19T10:56:28.161 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 3 to deep-scrub 2024-08-19T10:56:28.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 -- 172.21.15.89:0/1631838699 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f66bc03be30 msgr2=0x7f66bc03e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f66bc03be30 0x7f66bc03e2f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0a37a0 tx=0x7f66d4009290 comp rx=0 tx=0).stop 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 10 monclient: shutdown 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 -- 172.21.15.89:0/1631838699 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 msgr2=0x7f66dc1d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 0x7f66dc1d1230 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f66cc000eb0 tx=0x7f66c8005840 comp rx=0 tx=0).stop 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 -- 172.21.15.89:0/1631838699 shutdown_connections 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f66bc03be30 0x7f66bc03e2f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f66dc0a1850 0x7f66dc0a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f66dc13b3e0 0x7f66dc13b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 --2- 172.21.15.89:0/1631838699 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7f66c8004240 0x7f66dc1d1230 secure :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f66cc000eb0 tx=0x7f66c8005840 comp rx=0 tx=0).stop 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 -- 172.21.15.89:0/1631838699 >> 172.21.15.89:0/1631838699 conn(0x7f66dc0aa240 msgr2=0x7f66dc0a8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:28.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 -- 172.21.15.89:0/1631838699 shutdown_connections 2024-08-19T10:56:28.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.165+0000 7f66e2637640 1 -- 172.21.15.89:0/1631838699 wait complete. 2024-08-19T10:56:28.180 INFO:tasks.ceph:Scrubbing osd.4 2024-08-19T10:56:28.180 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-08-19T10:56:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient: get_monmap_and_config 2024-08-19T10:56:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient: build_initial_monmap 2024-08-19T10:56:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient: monmap: 2024-08-19T10:56:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:28.309114+0000 2024-08-19T10:56:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:28.309114+0000 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:28.305 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 1 Processor -- start 2024-08-19T10:56:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 1 -- start start 2024-08-19T10:56:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient: init 2024-08-19T10:56:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient: _add_conns ranks=[3,5,10,1,2,9,0,6,7,4,8] 2024-08-19T10:56:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806c930 0x7efde80712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient(hunting): picked mon.noname-g con 0x7efde806c930 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efde806cd10 0x7efde806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient(hunting): picked mon.noname-h con 0x7efde806cd10 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efde8137460 0x7efde81378c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient(hunting): picked mon.noname-f con 0x7efde8137460 addr v2:172.21.15.190:3300/0 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7efde80a3da0 con 0x7efde8137460 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7efde80a4e40 con 0x7efde806c930 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7efde80a5120 con 0x7efde806cd10 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdefe1c640 10 monclient(hunting): authenticate will time out at 2038.125589s 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdee3b9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efde8137460 0x7efde81378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806c930 0x7efde80712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efded3b7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efde806cd10 0x7efde806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806c930 0x7efde80712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:54642/0 (socket says 172.21.15.89:54642) 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdee3b9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efde8137460 0x7efde81378c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:56692/0 (socket says 172.21.15.89:56692) 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 1 -- 172.21.15.89:0/3126171842 learned_addr learned my addr 172.21.15.89:0/3126171842 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 10 monclient(hunting): get_auth_request con 0x7efde806c930 auth_method 0 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdee3b9640 10 monclient(hunting): get_auth_request con 0x7efde8137460 auth_method 0 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdee3b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdee3b9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdee3b9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efded3b7640 10 monclient(hunting): get_auth_request con 0x7efde806cd10 auth_method 0 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efded3b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efded3b7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efded3b7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdee3b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdee3b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdee3b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efded3b7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efded3b7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efded3b7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 10 monclient(hunting): handle_auth_done global_id 114321 payload 274 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 1 -- 172.21.15.89:0/3126171842 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efde806cd10 msgr2=0x7efde806d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 1 --2- 172.21.15.89:0/3126171842 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efde806cd10 0x7efde806d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 1 -- 172.21.15.89:0/3126171842 >> v2:172.21.15.190:3300/0 conn(0x7efde8137460 msgr2=0x7efde81378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 1 --2- 172.21.15.89:0/3126171842 >> v2:172.21.15.190:3300/0 conn(0x7efde8137460 0x7efde81378c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 10 monclient: _finish_hunting 0 2024-08-19T10:56:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 1 monclient: found mon.noname-g 2024-08-19T10:56:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:56:28.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.305+0000 7efdedbb8640 1 -- 172.21.15.89:0/3126171842 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde80a5400 con 0x7efde806c930 2024-08-19T10:56:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdedbb8640 1 -- 172.21.15.89:0/3126171842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806c930 msgr2=0x7efde80712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdedbb8640 1 --2- 172.21.15.89:0/3126171842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806c930 0x7efde80712a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efde0009870 tx=0x7efde001c4b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=7e30e34afe72de5d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 1 -- 172.21.15.89:0/3126171842 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efde00ab8c0 con 0x7efde806c930 2024-08-19T10:56:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:56:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 10 monclient: dump: 2024-08-19T10:56:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.315 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 10 monclient: _finish_auth 0 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 10 monclient: _check_auth_tickets 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 1 -- 172.21.15.89:0/3126171842 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde00b0070 con 0x7efde806c930 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 1 -- 172.21.15.89:0/3126171842 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efde00aa3b0 con 0x7efde806c930 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdecbb6640 10 monclient: dump: 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.317 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdefe1c640 5 monclient: authenticate success, global_id 114321 2024-08-19T10:56:28.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdefe1c640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.309+0000 7efdefe1c640 4 set_mon_vals no callback set 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient: shutdown 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 -- 172.21.15.89:0/3126171842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806c930 msgr2=0x7efde80712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 --2- 172.21.15.89:0/3126171842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806c930 0x7efde80712a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efde0009870 tx=0x7efde001c4b0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 -- 172.21.15.89:0/3126171842 shutdown_connections 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 --2- 172.21.15.89:0/3126171842 >> v2:172.21.15.190:3300/0 conn(0x7efde8137460 0x7efde81378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 --2- 172.21.15.89:0/3126171842 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7efde806cd10 0x7efde806d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 --2- 172.21.15.89:0/3126171842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806c930 0x7efde80712a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 -- 172.21.15.89:0/3126171842 >> 172.21.15.89:0/3126171842 conn(0x7efde80a9090 msgr2=0x7efde80a94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 -- 172.21.15.89:0/3126171842 shutdown_connections 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 -- 172.21.15.89:0/3126171842 wait complete. 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient: build_initial_monmap 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient: monmap: 2024-08-19T10:56:28.319 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:28.320 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 Processor -- start 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 -- start start 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient: init 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient: _add_conns ranks=[0,3,6,5,4,1,7,2,8] 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efde806c930 0x7efde81cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient(hunting): picked mon.noname-a con 0x7efde806c930 addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806cd10 0x7efde81345d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient(hunting): picked mon.noname-g con 0x7efde806cd10 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efde8137460 0x7efde813a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient(hunting): picked mon.noname-d con 0x7efde8137460 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:28.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efde806d7b0 con 0x7efde806c930 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7efde806d630 con 0x7efde806cd10 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7efde8140ed0 con 0x7efde8137460 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdefe1c640 10 monclient(hunting): authenticate will time out at 2038.134143s 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efde806c930 0x7efde81cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdee3b9640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efde8137460 0x7efde813a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efde806c930 0x7efde81cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59336/0 (socket says 172.21.15.89:59336) 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 1 -- 172.21.15.89:0/3739733126 learned_addr learned my addr 172.21.15.89:0/3739733126 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 10 monclient(hunting): get_auth_request con 0x7efde806c930 auth_method 0 2024-08-19T10:56:28.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdee3b9640 10 monclient(hunting): get_auth_request con 0x7efde8137460 auth_method 0 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdee3b9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdee3b9640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdee3b9640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdee3b9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdee3b9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdee3b9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efded3b7640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806cd10 0x7efde81345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efded3b7640 10 monclient(hunting): get_auth_request con 0x7efde806cd10 auth_method 0 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efded3b7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efded3b7640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efded3b7640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 10 monclient(hunting): handle_auth_done global_id 7884 payload 824 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 1 -- 172.21.15.89:0/3739733126 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efde8137460 msgr2=0x7efde813a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efde8137460 0x7efde813a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 1 -- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806cd10 msgr2=0x7efde81345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806cd10 0x7efde81345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 10 monclient: _finish_hunting 0 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 1 monclient: found mon.noname-a 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:28.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.89:3300/0 2024-08-19T10:56:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdedbb8640 1 -- 172.21.15.89:0/3739733126 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde813c100 con 0x7efde806c930 2024-08-19T10:56:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efdee3b9640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7efde8137460 0x7efde813a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:28.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.313+0000 7efded3b7640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde806cd10 0x7efde81345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T10:56:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdedbb8640 1 -- 172.21.15.89:0/3739733126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efde806c930 msgr2=0x7efde81cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdedbb8640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efde806c930 0x7efde81cf0e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7efde00ad040 tx=0x7efde0019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 1 -- 172.21.15.89:0/3739733126 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efde00ab450 con 0x7efde806c930 2024-08-19T10:56:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 10 monclient: dump: 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.328 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 10 monclient: _finish_auth 0 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 10 monclient: _check_auth_tickets 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 1 -- 172.21.15.89:0/3739733126 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde00ab650 con 0x7efde806c930 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdefe1c640 5 monclient: authenticate success, global_id 7884 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdefe1c640 10 monclient: _renew_subs 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdefe1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdefe1c640 1 -- 172.21.15.89:0/3739733126 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde80a9850 con 0x7efde806c930 2024-08-19T10:56:28.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 1 -- 172.21.15.89:0/3739733126 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7efde00abc70 con 0x7efde806c930 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 10 monclient: dump: 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.330 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efddffff640 4 set_mon_vals no callback set 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdefe1c640 10 monclient: _renew_subs 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdefe1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdefe1c640 1 -- 172.21.15.89:0/3739733126 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde811f7d0 con 0x7efde806c930 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdefe1c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.89:3300/0 2024-08-19T10:56:28.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdefe1c640 1 -- 172.21.15.89:0/3739733126 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efde806d0f0 con 0x7efde806c930 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 1 -- 172.21.15.89:0/3739733126 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7efde00ab800 con 0x7efde806c930 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efdd403be00 0x7efdd403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 1 -- 172.21.15.89:0/3739733126 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7efde00e9d30 con 0x7efde806c930 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] conn(0x7efdd4044ef0 0x7efdd4047350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 1 -- 172.21.15.89:0/3739733126 --> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efde00c5ec0 con 0x7efdd4044ef0 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 1 -- 172.21.15.89:0/3739733126 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=274) ==== 24+0+0 (secure 0 0 0) 0x7efde00ec300 con 0x7efde806c930 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdde7fc640 10 monclient: handle_get_version_reply finishing 1 version 274 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdee3b9640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] conn(0x7efdd4044ef0 0x7efdd4047350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efded3b7640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efdd403be00 0x7efdd403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efdee3b9640 10 monclient: get_auth_request con 0x7efdd4044ef0 auth_method 0 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.317+0000 7efded3b7640 10 monclient: get_auth_request con 0x7efdd403be00 auth_method 0 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.321+0000 7efded3b7640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efdd403be00 0x7efdd403e2c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efde813bd70 tx=0x7efde400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.321+0000 7efdee3b9640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] conn(0x7efdd4044ef0 0x7efdd4047350 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.325+0000 7efdde7fc640 1 -- 172.21.15.89:0/3739733126 <== osd.4 v2:172.21.15.190:6818/1174298329 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efde00c5ec0 con 0x7efdd4044ef0 2024-08-19T10:56:28.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.337+0000 7efdefe1c640 1 -- 172.21.15.89:0/3739733126 --> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efde8064590 con 0x7efdd4044ef0 2024-08-19T10:56:28.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.337+0000 7efdde7fc640 1 -- 172.21.15.89:0/3739733126 <== osd.4 v2:172.21.15.190:6818/1174298329 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efde8064590 con 0x7efdd4044ef0 2024-08-19T10:56:28.338 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:28.339 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T10:56:28.339 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.337+0000 7efdefe1c640 1 -- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] conn(0x7efdd4044ef0 msgr2=0x7efdd4047350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.337+0000 7efdefe1c640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:6818/1174298329,v1:172.21.15.190:6819/1174298329] conn(0x7efdd4044ef0 0x7efdd4047350 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.341+0000 7efdefe1c640 1 -- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efdd403be00 msgr2=0x7efdd403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.341+0000 7efdefe1c640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7efdd403be00 0x7efdd403e2c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efde813bd70 tx=0x7efde400b040 comp rx=0 tx=0).stop 2024-08-19T10:56:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.341+0000 7efdefe1c640 10 monclient: shutdown 2024-08-19T10:56:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.341+0000 7efdefe1c640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.341+0000 7efdefe1c640 1 -- 172.21.15.89:0/3739733126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efde806c930 msgr2=0x7efde81cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.341+0000 7efdefe1c640 1 --2- 172.21.15.89:0/3739733126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efde806c930 0x7efde81cf0e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7efde00ad040 tx=0x7efde0019250 comp rx=0 tx=0).stop 2024-08-19T10:56:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.341+0000 7efdee3b9640 1 -- 172.21.15.89:0/3739733126 reap_dead start 2024-08-19T10:56:28.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.341+0000 7efdefe1c640 1 -- 172.21.15.89:0/3739733126 shutdown_connections 2024-08-19T10:56:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.341+0000 7efdefe1c640 1 -- 172.21.15.89:0/3739733126 >> 172.21.15.89:0/3739733126 conn(0x7efde80a9090 msgr2=0x7efde80a4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.341+0000 7efdefe1c640 1 -- 172.21.15.89:0/3739733126 shutdown_connections 2024-08-19T10:56:28.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.341+0000 7efdefe1c640 1 -- 172.21.15.89:0/3739733126 wait complete. 2024-08-19T10:56:28.355 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-08-19T10:56:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.473+0000 7f3c6d985640 10 monclient: get_monmap_and_config 2024-08-19T10:56:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.473+0000 7f3c6d985640 10 monclient: build_initial_monmap 2024-08-19T10:56:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.473+0000 7f3c6d985640 10 monclient: monmap: 2024-08-19T10:56:28.475 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:28.480526+0000 2024-08-19T10:56:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:28.480526+0000 2024-08-19T10:56:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.476 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:28.477 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:28.478 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:28.478 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:28.478 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:28.478 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:28.478 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:28.479 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 1 Processor -- start 2024-08-19T10:56:28.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 1 -- start start 2024-08-19T10:56:28.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 10 monclient: init 2024-08-19T10:56:28.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:28.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 10 monclient: _add_conns ranks=[10,8,0,3,1,6,7,4,2,5,9] 2024-08-19T10:56:28.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3c680a67d0 0x7f3c680a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 10 monclient(hunting): picked mon.noname-f con 0x7f3c680a67d0 addr v2:172.21.15.190:3300/0 2024-08-19T10:56:28.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c680a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 10 monclient(hunting): picked mon.noname-e con 0x7f3c680a6bb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:28.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3c68139590 addr v1:172.21.15.89:6789/0 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c680a8250 con 0x7f3c68139590 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f3c680a8740 con 0x7f3c680a67d0 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3c680a3380 con 0x7f3c680a6bb0 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c6d985640 10 monclient(hunting): authenticate will time out at 2038.297374s 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c680a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c677fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3c68139590 0x7f3c68139a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:54364/0 (socket says 172.21.15.89:54364) 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c680a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50336/0 (socket says 172.21.15.89:50336) 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c677fe640 1 -- 172.21.15.89:0/3505616962 learned_addr learned my addr 172.21.15.89:0/3505616962 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:56:28.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 10 monclient(hunting): get_auth_request con 0x7f3c680a6bb0 auth_method 0 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 10 monclient(hunting): handle_auth_done global_id 114452 payload 274 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c66ffd640 1 --2- 172.21.15.89:0/3505616962 >> v2:172.21.15.190:3300/0 conn(0x7f3c680a67d0 0x7f3c680a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 1 -- 172.21.15.89:0/3505616962 >> v2:172.21.15.190:3300/0 conn(0x7f3c680a67d0 msgr2=0x7f3c680a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 1 --2- 172.21.15.89:0/3505616962 >> v2:172.21.15.190:3300/0 conn(0x7f3c680a67d0 0x7f3c680a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 1 -- 172.21.15.89:0/3505616962 >> v1:172.21.15.89:6789/0 conn(0x7f3c68139590 legacy=0x7f3c68139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 10 monclient: _finish_hunting 0 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 1 monclient: found mon.noname-e 2024-08-19T10:56:28.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.477+0000 7f3c667fc640 1 -- 172.21.15.89:0/3505616962 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c680a4cc0 con 0x7f3c680a6bb0 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c667fc640 1 -- 172.21.15.89:0/3505616962 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 msgr2=0x7f3c680a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c667fc640 1 --2- 172.21.15.89:0/3505616962 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c680a6f90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c0097c0 tx=0x7f3c5c01c3f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9f3a7632716470f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 1 -- 172.21.15.89:0/3505616962 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c5c0b0070 con 0x7f3c680a6bb0 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 10 monclient: dump: 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.496 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 10 monclient: _finish_auth 0 2024-08-19T10:56:28.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 5 monclient: authenticate success, global_id 114452 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 1 -- 172.21.15.89:0/3505616962 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5c0aa810 con 0x7f3c680a6bb0 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 4 set_mon_vals no callback set 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 1 -- 172.21.15.89:0/3505616962 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c5c0a93e0 con 0x7f3c680a6bb0 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c65ffb640 10 monclient: dump: 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.498 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.499 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 10 monclient: shutdown 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 1 -- 172.21.15.89:0/3505616962 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 msgr2=0x7f3c680a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 1 --2- 172.21.15.89:0/3505616962 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c680a6f90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c0097c0 tx=0x7f3c5c01c3f0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 1 -- 172.21.15.89:0/3505616962 shutdown_connections 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 1 --2- 172.21.15.89:0/3505616962 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c680a6f90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 1 --2- 172.21.15.89:0/3505616962 >> v2:172.21.15.190:3300/0 conn(0x7f3c680a67d0 0x7f3c680a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 1 -- 172.21.15.89:0/3505616962 >> 172.21.15.89:0/3505616962 conn(0x7f3c68069ae0 msgr2=0x7f3c680abe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 1 -- 172.21.15.89:0/3505616962 shutdown_connections 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 1 -- 172.21.15.89:0/3505616962 wait complete. 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 10 monclient: build_initial_monmap 2024-08-19T10:56:28.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 10 monclient: monmap: 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:28.501 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 1 Processor -- start 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 1 -- start start 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.481+0000 7f3c6d985640 10 monclient: init 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient: _add_conns ranks=[4,8,5,6,2,7,3,0,1] 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c680a67d0 0x7f3c681d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient(hunting): picked mon.noname-c con 0x7f3c680a67d0 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c681d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient(hunting): picked mon.noname-e con 0x7f3c680a6bb0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:28.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3c68139590 0x7f3c681d2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient(hunting): picked mon.noname-h con 0x7f3c68139590 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f3c681445b0 con 0x7f3c680a67d0 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f3c68144430 con 0x7f3c68139590 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3c68144730 con 0x7f3c680a6bb0 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient(hunting): authenticate will time out at 2038.304453s 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c681d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c66ffd640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c680a67d0 0x7f3c681d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c681d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:50344/0 (socket says 172.21.15.89:50344) 2024-08-19T10:56:28.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 1 -- 172.21.15.89:0/3335413430 learned_addr learned my addr 172.21.15.89:0/3335413430 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 10 monclient(hunting): get_auth_request con 0x7f3c680a6bb0 auth_method 0 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c66ffd640 10 monclient(hunting): get_auth_request con 0x7f3c680a67d0 auth_method 0 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c66ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c66ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c66ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c66ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c66ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c66ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 10 monclient(hunting): handle_auth_done global_id 114461 payload 824 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 1 -- 172.21.15.89:0/3335413430 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3c68139590 msgr2=0x7f3c681d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3c68139590 0x7f3c681d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 1 -- 172.21.15.89:0/3335413430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c680a67d0 msgr2=0x7f3c681d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c680a67d0 0x7f3c681d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 10 monclient: _finish_hunting 0 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 1 monclient: found mon.noname-e 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 1 -- 172.21.15.89:0/3335413430 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c681409e0 con 0x7f3c680a6bb0 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 1 -- 172.21.15.89:0/3335413430 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 msgr2=0x7f3c681d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:28.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c667fc640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c681d1bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c01cc90 tx=0x7f3c5c019190 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 1 -- 172.21.15.89:0/3335413430 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c5c0b0040 con 0x7f3c680a6bb0 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 10 monclient: dump: 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.506 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 10 monclient: _finish_auth 0 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 1 -- 172.21.15.89:0/3335413430 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5c0b9070 con 0x7f3c680a6bb0 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:28.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 1 -- 172.21.15.89:0/3335413430 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3c5c0044b0 con 0x7f3c680a6bb0 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c4ffff640 10 monclient: dump: 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.508 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c64ff9640 4 set_mon_vals no callback set 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 5 monclient: authenticate success, global_id 114461 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient: _renew_subs 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 1 -- 172.21.15.89:0/3335413430 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c680a2380 con 0x7f3c680a6bb0 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient: _renew_subs 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:28.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.485+0000 7f3c6d985640 1 -- 172.21.15.89:0/3335413430 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c680a0b20 con 0x7f3c680a6bb0 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.489+0000 7f3c6d985640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.489+0000 7f3c6d985640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.489+0000 7f3c6d985640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.89:3304/0 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.489+0000 7f3c6d985640 1 -- 172.21.15.89:0/3335413430 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c680a3f00 con 0x7f3c680a6bb0 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.489+0000 7f3c4ffff640 1 -- 172.21.15.89:0/3335413430 <== mon.8 v2:172.21.15.89:3304/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3c5c003620 con 0x7f3c680a6bb0 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.489+0000 7f3c4ffff640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c5403be00 0x7f3c5403e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.489+0000 7f3c4ffff640 1 -- 172.21.15.89:0/3335413430 <== mon.8 v2:172.21.15.89:3304/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f3c5c0a4e00 con 0x7f3c680a6bb0 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.489+0000 7f3c66ffd640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c5403be00 0x7f3c5403e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.489+0000 7f3c66ffd640 10 monclient: get_auth_request con 0x7f3c5403be00 auth_method 0 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.493+0000 7f3c66ffd640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c5403be00 0x7f3c5403e2c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800aae0 tx=0x7f3c58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.493+0000 7f3c4ffff640 1 -- 172.21.15.89:0/3335413430 <== mon.8 v2:172.21.15.89:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3c5c016070 con 0x7f3c680a6bb0 2024-08-19T10:56:28.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.493+0000 7f3c4ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:28.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.493+0000 7f3c4ffff640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 -- 172.21.15.89:0/3335413430 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f3c680a5500 con 0x7f3c5403be00 2024-08-19T10:56:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c4ffff640 1 -- 172.21.15.89:0/3335413430 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3c680a5500 con 0x7f3c5403be00 2024-08-19T10:56:28.660 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 4 to deep-scrub 2024-08-19T10:56:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 -- 172.21.15.89:0/3335413430 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c5403be00 msgr2=0x7f3c5403e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c5403be00 0x7f3c5403e2c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800aae0 tx=0x7f3c58008040 comp rx=0 tx=0).stop 2024-08-19T10:56:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 10 monclient: shutdown 2024-08-19T10:56:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 -- 172.21.15.89:0/3335413430 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 msgr2=0x7f3c681d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c681d1bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c01cc90 tx=0x7f3c5c019190 comp rx=0 tx=0).stop 2024-08-19T10:56:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 -- 172.21.15.89:0/3335413430 shutdown_connections 2024-08-19T10:56:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f3c68139590 0x7f3c681d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3c5403be00 0x7f3c5403e2c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3c680a6bb0 0x7f3c681d1bf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 --2- 172.21.15.89:0/3335413430 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f3c680a67d0 0x7f3c681d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 -- 172.21.15.89:0/3335413430 >> 172.21.15.89:0/3335413430 conn(0x7f3c68069ae0 msgr2=0x7f3c68140c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.661+0000 7f3c6d985640 1 -- 172.21.15.89:0/3335413430 shutdown_connections 2024-08-19T10:56:28.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.665+0000 7f3c6d985640 1 -- 172.21.15.89:0/3335413430 wait complete. 2024-08-19T10:56:28.679 INFO:tasks.ceph:Scrubbing osd.5 2024-08-19T10:56:28.679 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-08-19T10:56:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.793+0000 7f534d4f0640 10 monclient: get_monmap_and_config 2024-08-19T10:56:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.793+0000 7f534d4f0640 10 monclient: build_initial_monmap 2024-08-19T10:56:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.793+0000 7f534d4f0640 10 monclient: monmap: 2024-08-19T10:56:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:28.800830+0000 2024-08-19T10:56:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:28.800830+0000 2024-08-19T10:56:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:28.795 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:28.796 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:28.796 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 1 Processor -- start 2024-08-19T10:56:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 1 -- start start 2024-08-19T10:56:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 10 monclient: init 2024-08-19T10:56:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:28.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 10 monclient: _add_conns ranks=[10,2,5,1,7,3,0,4,8,6,9] 2024-08-19T10:56:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53480a3620 0x7f53480a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 10 monclient(hunting): picked mon.noname-f con 0x7f53480a3620 addr v2:172.21.15.190:3300/0 2024-08-19T10:56:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53480a3f40 0x7f53481363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 10 monclient(hunting): picked mon.noname-b con 0x7f53480a3f40 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5348136920 0x7f5348136da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 10 monclient(hunting): picked mon.noname-h con 0x7f5348136920 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f534809fc80 con 0x7f53480a3620 2024-08-19T10:56:28.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f534809fed0 con 0x7f53480a3f40 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f534809ec30 con 0x7f5348136920 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f534d4f0640 10 monclient(hunting): authenticate will time out at 2038.617126s 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f533e7fc640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53480a3f40 msgr2=0x7f53481363e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f533e7fc640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53480a3f40 0x7f53481363e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53480a3620 0x7f53480a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f53477fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5348136920 0x7f5348136da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53480a3620 0x7f53480a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:56704/0 (socket says 172.21.15.89:56704) 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f53477fe640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5348136920 0x7f5348136da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3302/0 says I am v2:172.21.15.89:44882/0 (socket says 172.21.15.89:44882) 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 1 -- 172.21.15.89:0/237066486 learned_addr learned my addr 172.21.15.89:0/237066486 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:28.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 10 monclient(hunting): get_auth_request con 0x7f53480a3620 auth_method 0 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f53477fe640 10 monclient(hunting): get_auth_request con 0x7f5348136920 auth_method 0 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f53477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f53477fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f53477fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f53477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f53477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f53477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 10 monclient(hunting): handle_auth_done global_id 45559 payload 274 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 1 -- 172.21.15.89:0/237066486 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5348136920 msgr2=0x7f5348136da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 1 --2- 172.21.15.89:0/237066486 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5348136920 0x7f5348136da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 1 -- 172.21.15.89:0/237066486 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53480a3f40 msgr2=0x7f53481363e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 1 --2- 172.21.15.89:0/237066486 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53480a3f40 0x7f53481363e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 10 monclient: _finish_hunting 0 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 1 monclient: found mon.noname-f 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f5346ffd640 1 -- 172.21.15.89:0/237066486 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534809f180 con 0x7f53480a3620 2024-08-19T10:56:28.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.797+0000 7f53477fe640 1 --2- 172.21.15.89:0/237066486 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5348136920 0x7f5348136da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:28.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f5346ffd640 1 -- 172.21.15.89:0/237066486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53480a3620 msgr2=0x7f53480a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:28.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f5346ffd640 1 --2- 172.21.15.89:0/237066486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53480a3620 0x7f53480a3a00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f533800bb40 tx=0x7f533801eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f75382d9ba1d30d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 1 -- 172.21.15.89:0/237066486 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53380b4070 con 0x7f53480a3620 2024-08-19T10:56:28.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 10 monclient: dump: 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.812 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 10 monclient: _finish_auth 0 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 10 monclient: _check_auth_tickets 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 5 monclient: authenticate success, global_id 45559 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 1 -- 172.21.15.89:0/237066486 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53380a7ba0 con 0x7f53480a3620 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:28.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 4 set_mon_vals no callback set 2024-08-19T10:56:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 1 -- 172.21.15.89:0/237066486 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f53380a7d90 con 0x7f53480a3620 2024-08-19T10:56:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f53467fc640 10 monclient: dump: 2024-08-19T10:56:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.814 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 10 monclient: shutdown 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 1 -- 172.21.15.89:0/237066486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53480a3620 msgr2=0x7f53480a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 1 --2- 172.21.15.89:0/237066486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53480a3620 0x7f53480a3a00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f533800bb40 tx=0x7f533801eaf0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 1 -- 172.21.15.89:0/237066486 shutdown_connections 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 1 --2- 172.21.15.89:0/237066486 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f5348136920 0x7f5348136da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 1 --2- 172.21.15.89:0/237066486 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53480a3f40 0x7f53481363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 1 --2- 172.21.15.89:0/237066486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53480a3620 0x7f53480a3a00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 1 -- 172.21.15.89:0/237066486 >> 172.21.15.89:0/237066486 conn(0x7f53480a4710 msgr2=0x7f53480a4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 1 -- 172.21.15.89:0/237066486 shutdown_connections 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.801+0000 7f534d4f0640 1 -- 172.21.15.89:0/237066486 wait complete. 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient: build_initial_monmap 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient: monmap: 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:28.816 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 1 Processor -- start 2024-08-19T10:56:28.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 1 -- start start 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient: init 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient: _add_conns ranks=[2,4,5,3,7,6,0,8,1] 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53480a3f40 0x7f53481cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient(hunting): picked mon.noname-b con 0x7f53480a3f40 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5348136920 0x7f53481cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient(hunting): picked mon.noname-c con 0x7f5348136920 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f53481cde60 0x7f534813af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient(hunting): picked mon.noname-h con 0x7f53481cde60 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f534809e430 con 0x7f53480a3f40 2024-08-19T10:56:28.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f53480a5550 con 0x7f5348136920 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7f53480a4320 con 0x7f53481cde60 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f534d4f0640 10 monclient(hunting): authenticate will time out at 2038.625175s 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f5346ffd640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53480a3f40 msgr2=0x7f53481cd3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f5346ffd640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53480a3f40 0x7f53481cd3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5348136920 0x7f53481cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5348136920 0x7f53481cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45952/0 (socket says 172.21.15.89:45952) 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 1 -- 172.21.15.89:0/3773338407 learned_addr learned my addr 172.21.15.89:0/3773338407 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 10 monclient(hunting): get_auth_request con 0x7f5348136920 auth_method 0 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f53477fe640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f53481cde60 0x7f534813af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 10 monclient(hunting): handle_auth_done global_id 84631 payload 824 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 1 -- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f53481cde60 msgr2=0x7f534813af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f53481cde60 0x7f534813af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 1 -- 172.21.15.89:0/3773338407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53480a3f40 msgr2=0x7f53481cd3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f53480a3f40 0x7f53481cd3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 10 monclient: _finish_hunting 0 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 1 monclient: found mon.noname-c 2024-08-19T10:56:28.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 1 -- 172.21.15.89:0/3773338407 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5348120580 con 0x7f5348136920 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f53477fe640 10 monclient: get_auth_request con 0x7f53481cde60 auth_method 0 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f53477fe640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7f53481cde60 0x7f534813af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 1 -- 172.21.15.89:0/3773338407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5348136920 msgr2=0x7f53481cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.805+0000 7f533e7fc640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5348136920 0x7f53481cd920 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f533000b550 tx=0x7f533000ba10 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 1 -- 172.21.15.89:0/3773338407 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5330003de0 con 0x7f5348136920 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 10 monclient: dump: 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.821 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 10 monclient: _finish_auth 0 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 10 monclient: _check_auth_tickets 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f534d4f0640 5 monclient: authenticate success, global_id 84631 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 1 -- 172.21.15.89:0/3773338407 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5330015070 con 0x7f5348136920 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f534d4f0640 10 monclient: _renew_subs 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f534d4f0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f534d4f0640 1 -- 172.21.15.89:0/3773338407 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5348120860 con 0x7f5348136920 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 1 -- 172.21.15.89:0/3773338407 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f5330010480 con 0x7f5348136920 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f53457fa640 4 set_mon_vals no callback set 2024-08-19T10:56:28.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 10 monclient: dump: 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.824 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f534d4f0640 10 monclient: _renew_subs 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f534d4f0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f534d4f0640 1 -- 172.21.15.89:0/3773338407 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f534811f9f0 con 0x7f5348136920 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f534d4f0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f534d4f0640 1 -- 172.21.15.89:0/3773338407 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53481363e0 con 0x7f5348136920 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 1 -- 172.21.15.89:0/3773338407 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f53300050f0 con 0x7f5348136920 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f532803be00 0x7f532803e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 1 -- 172.21.15.89:0/3773338407 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f533004f4c0 con 0x7f5348136920 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] conn(0x7f5328044e70 0x7f53280472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 1 -- 172.21.15.89:0/3773338407 --> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53280479a0 con 0x7f5328044e70 2024-08-19T10:56:28.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 1 -- 172.21.15.89:0/3773338407 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_get_version_reply(handle=1 version=274) ==== 24+0+0 (secure 0 0 0) 0x7f5330056050 con 0x7f5348136920 2024-08-19T10:56:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f533ffff640 10 monclient: handle_get_version_reply finishing 1 version 274 2024-08-19T10:56:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f5346ffd640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f532803be00 0x7f532803e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f53477fe640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] conn(0x7f5328044e70 0x7f53280472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f5346ffd640 10 monclient: get_auth_request con 0x7f532803be00 auth_method 0 2024-08-19T10:56:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.809+0000 7f53477fe640 10 monclient: get_auth_request con 0x7f5328044e70 auth_method 0 2024-08-19T10:56:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.813+0000 7f5346ffd640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f532803be00 0x7f532803e2c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f533800bb40 tx=0x7f53380a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.813+0000 7f53477fe640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] conn(0x7f5328044e70 0x7f53280472d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.817+0000 7f533ffff640 1 -- 172.21.15.89:0/3773338407 <== osd.5 v2:172.21.15.190:6802/2434772607 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53280479a0 con 0x7f5328044e70 2024-08-19T10:56:28.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 1 -- 172.21.15.89:0/3773338407 --> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f53480a7220 con 0x7f5328044e70 2024-08-19T10:56:28.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f533ffff640 1 -- 172.21.15.89:0/3773338407 <== osd.5 v2:172.21.15.190:6802/2434772607 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f53480a7220 con 0x7f5328044e70 2024-08-19T10:56:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-08-19T10:56:28.828 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-19T10:56:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-08-19T10:56:28.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 1 -- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] conn(0x7f5328044e70 msgr2=0x7f53280472d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:6802/2434772607,v1:172.21.15.190:6803/2434772607] conn(0x7f5328044e70 0x7f53280472d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 1 -- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f532803be00 msgr2=0x7f532803e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f532803be00 0x7f532803e2c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f533800bb40 tx=0x7f53380a9070 comp rx=0 tx=0).stop 2024-08-19T10:56:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 10 monclient: shutdown 2024-08-19T10:56:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 1 -- 172.21.15.89:0/3773338407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5348136920 msgr2=0x7f53481cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 1 --2- 172.21.15.89:0/3773338407 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f5348136920 0x7f53481cd920 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f533000b550 tx=0x7f533000ba10 comp rx=0 tx=0).stop 2024-08-19T10:56:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f53477fe640 1 -- 172.21.15.89:0/3773338407 reap_dead start 2024-08-19T10:56:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 1 -- 172.21.15.89:0/3773338407 shutdown_connections 2024-08-19T10:56:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 1 -- 172.21.15.89:0/3773338407 >> 172.21.15.89:0/3773338407 conn(0x7f53480a4710 msgr2=0x7f53480a65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 1 -- 172.21.15.89:0/3773338407 shutdown_connections 2024-08-19T10:56:28.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.829+0000 7f534d4f0640 1 -- 172.21.15.89:0/3773338407 wait complete. 2024-08-19T10:56:28.843 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-08-19T10:56:28.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient: get_monmap_and_config 2024-08-19T10:56:28.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient: build_initial_monmap 2024-08-19T10:56:28.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient: monmap: 2024-08-19T10:56:28.948 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:28.948 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.948 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:28.953616+0000 2024-08-19T10:56:28.948 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:28.953616+0000 2024-08-19T10:56:28.948 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:28.948 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.949 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:28.949 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:28.949 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:28.949 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:28.949 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:28.949 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:28.949 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:28.950 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:28.950 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:28.950 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:28.951 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:28.952 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 1 Processor -- start 2024-08-19T10:56:28.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 1 -- start start 2024-08-19T10:56:28.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient: init 2024-08-19T10:56:28.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:28.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient: _add_conns ranks=[3,6,2,4,8,10,5,9,7,1,0] 2024-08-19T10:56:28.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe300a5270 0x7fbe300a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient(hunting): picked mon.noname-g con 0x7fbe300a5270 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe300a6d30 0x7fbe300a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient(hunting): picked mon.noname-d con 0x7fbe300a6d30 addr [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] 2024-08-19T10:56:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 0x7fbe301399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient(hunting): picked mon.noname-b con 0x7fbe301395b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fbe300a7650 con 0x7fbe301395b0 2024-08-19T10:56:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbe300a8710 con 0x7fbe300a5270 2024-08-19T10:56:28.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 1 -- --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_getmap magic: 0 -- 0x7fbe300a5a40 con 0x7fbe300a6d30 2024-08-19T10:56:28.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:28.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe385a7640 10 monclient(hunting): authenticate will time out at 2038.770199s 2024-08-19T10:56:28.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe36b44640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 msgr2=0x7fbe301399d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:28.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe36b44640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 0x7fbe301399d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:28.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe300a6d30 0x7fbe300a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 1 --2- >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe300a6d30 0x7fbe300a7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3303/0 says I am v2:172.21.15.89:44232/0 (socket says 172.21.15.89:44232) 2024-08-19T10:56:28.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 1 -- 172.21.15.89:0/3781154335 learned_addr learned my addr 172.21.15.89:0/3781154335 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:28.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 10 monclient(hunting): get_auth_request con 0x7fbe300a6d30 auth_method 0 2024-08-19T10:56:28.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe36343640 1 --2- 172.21.15.89:0/3781154335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe300a5270 0x7fbe300a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 10 monclient(hunting): handle_auth_done global_id 45672 payload 274 2024-08-19T10:56:28.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 1 -- 172.21.15.89:0/3781154335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe300a5270 msgr2=0x7fbe300a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:28.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 1 --2- 172.21.15.89:0/3781154335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe300a5270 0x7fbe300a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 1 -- 172.21.15.89:0/3781154335 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 msgr2=0x7fbe301399d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T10:56:28.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 1 --2- 172.21.15.89:0/3781154335 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 0x7fbe301399d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 10 monclient: _finish_hunting 0 2024-08-19T10:56:28.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 1 monclient: found mon.noname-d 2024-08-19T10:56:28.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:28.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.89:3303/0 2024-08-19T10:56:28.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe35b42640 1 -- 172.21.15.89:0/3781154335 --> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe300a81d0 con 0x7fbe300a6d30 2024-08-19T10:56:28.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe36343640 10 monclient: get_auth_request con 0x7fbe300a5270 auth_method 0 2024-08-19T10:56:28.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.949+0000 7fbe36343640 1 --2- 172.21.15.89:0/3781154335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe300a5270 0x7fbe300a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:28.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35b42640 1 -- 172.21.15.89:0/3781154335 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe300a6d30 msgr2=0x7fbe300a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:28.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35b42640 1 --2- 172.21.15.89:0/3781154335 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe300a6d30 0x7fbe300a7110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbe200098a0 tx=0x7fbe2001c460 comp rx=0 tx=0).ready entity=mon.6 client_cookie=72a6bcb076c29a5e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 1 -- 172.21.15.89:0/3781154335 <== mon.6 v2:172.21.15.89:3303/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbe200b0070 con 0x7fbe300a6d30 2024-08-19T10:56:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-08-19T10:56:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 10 monclient: dump: 2024-08-19T10:56:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.980 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.981 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.981 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.981 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.981 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.981 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.981 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.982 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.982 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 10 monclient: _finish_auth 0 2024-08-19T10:56:28.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 10 monclient: _check_auth_tickets 2024-08-19T10:56:28.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:28.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 5 monclient: authenticate success, global_id 45672 2024-08-19T10:56:28.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:28.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 1 -- 172.21.15.89:0/3781154335 <== mon.6 v2:172.21.15.89:3303/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe200aa810 con 0x7fbe300a6d30 2024-08-19T10:56:28.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:28.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:28.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 4 set_mon_vals no callback set 2024-08-19T10:56:28.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 1 -- 172.21.15.89:0/3781154335 <== mon.6 v2:172.21.15.89:3303/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbe200aae30 con 0x7fbe300a6d30 2024-08-19T10:56:28.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-08-19T10:56:28.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe35341640 10 monclient: dump: 2024-08-19T10:56:28.984 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.985 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 10 monclient: shutdown 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 1 -- 172.21.15.89:0/3781154335 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe300a6d30 msgr2=0x7fbe300a7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 1 --2- 172.21.15.89:0/3781154335 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe300a6d30 0x7fbe300a7110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbe200098a0 tx=0x7fbe2001c460 comp rx=0 tx=0).stop 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 1 -- 172.21.15.89:0/3781154335 shutdown_connections 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 1 --2- 172.21.15.89:0/3781154335 >> [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] conn(0x7fbe300a6d30 0x7fbe300a7110 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 1 --2- 172.21.15.89:0/3781154335 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 0x7fbe301399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 1 --2- 172.21.15.89:0/3781154335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe300a5270 0x7fbe300a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 1 -- 172.21.15.89:0/3781154335 >> 172.21.15.89:0/3781154335 conn(0x7fbe30069a80 msgr2=0x7fbe300abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 1 -- 172.21.15.89:0/3781154335 shutdown_connections 2024-08-19T10:56:28.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.953+0000 7fbe385a7640 1 -- 172.21.15.89:0/3781154335 wait complete. 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient: build_initial_monmap 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient: monmap: 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:28.987 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 1 Processor -- start 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 1 -- start start 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient: init 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient: _add_conns ranks=[4,5,2,1,7,6,3,0,8] 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe300a5270 0x7fbe301d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient(hunting): picked mon.noname-c con 0x7fbe300a5270 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbe300a6d30 0x7fbe300a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient(hunting): picked mon.noname-h con 0x7fbe300a6d30 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:28.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 0x7fbe300a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient(hunting): picked mon.noname-b con 0x7fbe301395b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fbe300a57d0 con 0x7fbe301395b0 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7fbe300a5650 con 0x7fbe300a5270 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fbe30143230 con 0x7fbe300a6d30 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe385a7640 10 monclient(hunting): authenticate will time out at 2038.777227s 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36b44640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 msgr2=0x7fbe300a2480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36b44640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 0x7fbe300a2480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe300a5270 0x7fbe301d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe300a5270 0x7fbe301d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45956/0 (socket says 172.21.15.89:45956) 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 1 -- 172.21.15.89:0/1637933097 learned_addr learned my addr 172.21.15.89:0/1637933097 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 10 monclient(hunting): get_auth_request con 0x7fbe300a5270 auth_method 0 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 10 monclient(hunting): handle_auth_done global_id 84640 payload 824 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe35b42640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbe300a6d30 0x7fbe300a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 1 -- 172.21.15.89:0/1637933097 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbe300a6d30 msgr2=0x7fbe300a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbe300a6d30 0x7fbe300a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 1 -- 172.21.15.89:0/1637933097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 msgr2=0x7fbe300a2480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 0x7fbe300a2480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 10 monclient: _finish_hunting 0 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 1 monclient: found mon.noname-c 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 1 -- 172.21.15.89:0/1637933097 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3009d700 con 0x7fbe300a5270 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 1 -- 172.21.15.89:0/1637933097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe300a5270 msgr2=0x7fbe301d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.957+0000 7fbe36343640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe300a5270 0x7fbe301d16a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800ce00 tx=0x7fbe2800a650 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe36343640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe300a5270 0x7fbe301d16a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800ce00 tx=0x7fbe2800a650 comp rx=0 tx=0).queue_received will delay after 1.72406e+09 on 0x7fbe2800fd10 config(0 keys) 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 1 -- 172.21.15.89:0/1637933097 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbe2800ae20 con 0x7fbe300a5270 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 10 monclient: dump: 2024-08-19T10:56:28.991 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.992 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 10 monclient: _finish_auth 0 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 1 -- 172.21.15.89:0/1637933097 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe2800fd10 con 0x7fbe300a5270 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe385a7640 5 monclient: authenticate success, global_id 84640 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe385a7640 10 monclient: _renew_subs 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe385a7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe385a7640 1 -- 172.21.15.89:0/1637933097 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe300a29c0 con 0x7fbe300a5270 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe34b40640 4 set_mon_vals no callback set 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe385a7640 10 monclient: _renew_subs 2024-08-19T10:56:28.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe385a7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe385a7640 1 -- 172.21.15.89:0/1637933097 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3009d420 con 0x7fbe300a5270 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe24ff9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe24ff9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe24ff9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.89:3302/0 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 1 -- 172.21.15.89:0/1637933097 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fbe28018870 con 0x7fbe300a5270 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe24ff9640 1 -- 172.21.15.89:0/1637933097 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe0c005090 con 0x7fbe300a5270 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 10 monclient: dump: 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:28.994 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:28.995 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:28.995 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:28.995 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:28.995 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:28.995 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:28.995 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:28.995 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:28.995 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:28.995 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:28.995 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:28.996 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:28.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 1 -- 172.21.15.89:0/1637933097 <== mon.4 v2:172.21.15.89:3302/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fbe2800f8a0 con 0x7fbe300a5270 2024-08-19T10:56:28.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe26ffd640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe1403be50 0x7fbe1403e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:28.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe35b42640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe1403be50 0x7fbe1403e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:28.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.961+0000 7fbe35b42640 10 monclient: get_auth_request con 0x7fbe1403be50 auth_method 0 2024-08-19T10:56:28.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.965+0000 7fbe35b42640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe1403be50 0x7fbe1403e310 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbe200ab000 tx=0x7fbe200a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:28.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:28.965+0000 7fbe26ffd640 1 -- 172.21.15.89:0/1637933097 <== mon.4 v2:172.21.15.89:3302/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fbe280508a0 con 0x7fbe300a5270 2024-08-19T10:56:29.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.585+0000 7fbe26ffd640 1 -- 172.21.15.89:0/1637933097 <== mon.4 v2:172.21.15.89:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fbe3009d700 con 0x7fbe300a5270 2024-08-19T10:56:29.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.585+0000 7fbe26ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:29.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.585+0000 7fbe26ffd640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:29.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 1 -- 172.21.15.89:0/1637933097 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fbe0c000bd0 con 0x7fbe1403be50 2024-08-19T10:56:29.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe26ffd640 1 -- 172.21.15.89:0/1637933097 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbe0c000bd0 con 0x7fbe1403be50 2024-08-19T10:56:29.764 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 5 to deep-scrub 2024-08-19T10:56:29.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 1 -- 172.21.15.89:0/1637933097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe1403be50 msgr2=0x7fbe1403e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe1403be50 0x7fbe1403e310 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbe200ab000 tx=0x7fbe200a6000 comp rx=0 tx=0).stop 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 10 monclient: shutdown 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 1 -- 172.21.15.89:0/1637933097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe300a5270 msgr2=0x7fbe301d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe300a5270 0x7fbe301d16a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800ce00 tx=0x7fbe2800a650 comp rx=0 tx=0).stop 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 1 -- 172.21.15.89:0/1637933097 shutdown_connections 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fbe1403be50 0x7fbe1403e310 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fbe301395b0 0x7fbe300a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fbe300a6d30 0x7fbe300a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 1 --2- 172.21.15.89:0/1637933097 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7fbe300a5270 0x7fbe301d16a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.765+0000 7fbe24ff9640 1 -- 172.21.15.89:0/1637933097 >> 172.21.15.89:0/1637933097 conn(0x7fbe30069a80 msgr2=0x7fbe30137b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:29.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.769+0000 7fbe24ff9640 1 -- 172.21.15.89:0/1637933097 shutdown_connections 2024-08-19T10:56:29.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.769+0000 7fbe24ff9640 1 -- 172.21.15.89:0/1637933097 wait complete. 2024-08-19T10:56:29.783 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T10:56:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.897+0000 7f46ebbd2640 10 monclient: get_monmap_and_config 2024-08-19T10:56:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.897+0000 7f46ebbd2640 10 monclient: build_initial_monmap 2024-08-19T10:56:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.897+0000 7f46ebbd2640 10 monclient: monmap: 2024-08-19T10:56:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:29.903773+0000 2024-08-19T10:56:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:29.903773+0000 2024-08-19T10:56:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:29.898 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:29.899 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:29.899 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:29.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.897+0000 7f46ebbd2640 1 Processor -- start 2024-08-19T10:56:29.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.897+0000 7f46ebbd2640 1 -- start start 2024-08-19T10:56:29.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.897+0000 7f46ebbd2640 10 monclient: init 2024-08-19T10:56:29.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 10 monclient: _add_conns ranks=[9,0,4,8,2,1,7,5,10,3,6] 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46e40a5270 0x7f46e40a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 10 monclient(hunting): picked mon.noname-a con 0x7f46e40a5270 addr v2:172.21.15.89:3300/0 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f46e40a6d30 addr v1:172.21.15.89:6789/0 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46e40a7140 0x7f46e4139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 10 monclient(hunting): picked mon.noname-c con 0x7f46e40a7140 addr [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e40a8710 con 0x7f46e40a6d30 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f46e40a5a40 con 0x7f46e40a5270 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 1 -- --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_getmap magic: 0 -- 0x7f46e40a81d0 con 0x7f46e40a7140 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:29.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ebbd2640 10 monclient(hunting): authenticate will time out at 2039.720556s 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46e40a7140 0x7f46e4139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46e996e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f46e40a5270 0x7f46e40a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 1 --2- >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46e40a7140 0x7f46e4139cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3302/0 says I am v2:172.21.15.89:45970/0 (socket says 172.21.15.89:45970) 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 1 -- 172.21.15.89:0/1245316922 learned_addr learned my addr 172.21.15.89:0/1245316922 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 10 monclient(hunting): get_auth_request con 0x7f46e40a7140 auth_method 0 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46e996e640 10 monclient(hunting): get_auth_request con 0x7f46e40a5270 auth_method 0 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46e996e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46e996e640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:29.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46e996e640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46e996e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46e996e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46e996e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 10 monclient(hunting): handle_auth_done global_id 84649 payload 274 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 1 -- 172.21.15.89:0/1245316922 >> v2:172.21.15.89:3300/0 conn(0x7f46e40a5270 msgr2=0x7f46e40a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 1 --2- 172.21.15.89:0/1245316922 >> v2:172.21.15.89:3300/0 conn(0x7f46e40a5270 0x7f46e40a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 1 -- 172.21.15.89:0/1245316922 >> v1:172.21.15.89:6789/0 conn(0x7f46e40a6d30 legacy=0x7f46e41395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 10 monclient: _finish_hunting 0 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 1 monclient: found mon.noname-c 2024-08-19T10:56:29.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:29.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.89:3302/0 2024-08-19T10:56:29.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46ea16f640 1 -- 172.21.15.89:0/1245316922 --> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e40a4790 con 0x7f46e40a7140 2024-08-19T10:56:29.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.901+0000 7f46e996e640 1 --2- 172.21.15.89:0/1245316922 >> v2:172.21.15.89:3300/0 conn(0x7f46e40a5270 0x7f46e40a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:29.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ea16f640 1 -- 172.21.15.89:0/1245316922 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46e40a7140 msgr2=0x7f46e4139cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:29.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ea16f640 1 --2- 172.21.15.89:0/1245316922 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46e40a7140 0x7f46e4139cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009870 tx=0x7f46d401c210 comp rx=0 tx=0).ready entity=mon.4 client_cookie=5bff6dce4d078c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:29.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 1 -- 172.21.15.89:0/1245316922 <== mon.4 v2:172.21.15.89:3302/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46d40b0070 con 0x7f46e40a7140 2024-08-19T10:56:29.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:29.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-08-19T10:56:29.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 10 monclient: dump: 2024-08-19T10:56:29.909 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:29.909 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:29.909 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:29.909 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:29.909 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:29.909 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:29.909 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:29.909 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:29.909 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:29.910 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:29.910 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:29.910 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:29.910 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:29.910 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:29.910 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:29.910 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:29.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 10 monclient: _finish_auth 0 2024-08-19T10:56:29.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 10 monclient: _check_auth_tickets 2024-08-19T10:56:29.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 5 monclient: authenticate success, global_id 84649 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 1 -- 172.21.15.89:0/1245316922 <== mon.4 v2:172.21.15.89:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d40aa810 con 0x7f46e40a7140 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 4 set_mon_vals no callback set 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 1 -- 172.21.15.89:0/1245316922 <== mon.4 v2:172.21.15.89:3302/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46d40a9300 con 0x7f46e40a7140 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46e896c640 10 monclient: dump: 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:29.911 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:29.912 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:29.912 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:29.912 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:29.912 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:29.912 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:29.912 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:29.912 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:29.912 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:29.912 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 10 monclient: shutdown 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1245316922 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46e40a7140 msgr2=0x7f46e4139cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 1 --2- 172.21.15.89:0/1245316922 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46e40a7140 0x7f46e4139cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009870 tx=0x7f46d401c210 comp rx=0 tx=0).stop 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1245316922 shutdown_connections 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 1 --2- 172.21.15.89:0/1245316922 >> [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] conn(0x7f46e40a7140 0x7f46e4139cc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 1 --2- 172.21.15.89:0/1245316922 >> v2:172.21.15.89:3300/0 conn(0x7f46e40a5270 0x7f46e40a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:29.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1245316922 >> 172.21.15.89:0/1245316922 conn(0x7f46e4069a80 msgr2=0x7f46e40abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1245316922 shutdown_connections 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1245316922 wait complete. 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 10 monclient: build_initial_monmap 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 10 monclient: monmap: 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:29.914 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 1 Processor -- start 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 1 -- start start 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 10 monclient: init 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 10 monclient: _add_conns ranks=[7,2,3,8,4,0,6,1,5] 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f46e40a5270 0x7f46e41388c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:29.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.905+0000 7f46ebbd2640 10 monclient(hunting): picked mon.noname-i con 0x7f46e40a5270 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:29.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46e40a6d30 0x7f46e4138e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:29.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 10 monclient(hunting): picked mon.noname-b con 0x7f46e40a6d30 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:29.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46e40a7140 0x7f46e413e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:29.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 10 monclient(hunting): picked mon.noname-g con 0x7f46e40a7140 addr [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] 2024-08-19T10:56:29.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:29.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f46e40a57d0 con 0x7f46e40a6d30 2024-08-19T10:56:29.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:29.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f46e40a5650 con 0x7f46e40a7140 2024-08-19T10:56:29.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f46e413a200 con 0x7f46e40a5270 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46e916d640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46e40a6d30 msgr2=0x7f46e4138e00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 10 monclient(hunting): authenticate will time out at 2039.728097s 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46e916d640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46e40a6d30 0x7f46e4138e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46e996e640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f46e40a5270 msgr2=0x7f46e41388c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46e996e640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f46e40a5270 0x7f46e41388c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46e40a7140 0x7f46e413e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46e40a7140 0x7f46e413e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.89:54658/0 (socket says 172.21.15.89:54658) 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 1 -- 172.21.15.89:0/1082895593 learned_addr learned my addr 172.21.15.89:0/1082895593 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 10 monclient(hunting): get_auth_request con 0x7f46e40a7140 auth_method 0 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:29.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 10 monclient(hunting): handle_auth_done global_id 114339 payload 824 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 1 -- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f46e40a5270 msgr2=0x7f46e41388c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f46e40a5270 0x7f46e41388c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 1 -- 172.21.15.89:0/1082895593 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46e40a6d30 msgr2=0x7f46e4138e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46e40a6d30 0x7f46e4138e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 10 monclient: _finish_hunting 0 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 1 monclient: found mon.noname-g 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:29.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.190:3301/0 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 1 -- 172.21.15.89:0/1082895593 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e40a9360 con 0x7f46e40a7140 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 1 -- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46e40a7140 msgr2=0x7f46e413e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ea16f640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46e40a7140 0x7f46e413e280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f46d401cab0 tx=0x7f46d4018fb0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46e27fc640 1 -- 172.21.15.89:0/1082895593 <== mon.3 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46d40b9070 con 0x7f46e40a7140 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46e27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46e27fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46e27fc640 10 monclient: dump: 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:29.924 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46e27fc640 10 monclient: _finish_auth 0 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46e27fc640 10 monclient: _check_auth_tickets 2024-08-19T10:56:29.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46e27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 5 monclient: authenticate success, global_id 114339 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 10 monclient: _renew_subs 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.909+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1082895593 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e4123320 con 0x7f46e40a7140 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46ebbd2640 10 monclient: _renew_subs 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46ebbd2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1082895593 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e41404d0 con 0x7f46e40a7140 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e27fc640 1 -- 172.21.15.89:0/1082895593 <== mon.3 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d4004040 con 0x7f46e40a7140 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e27fc640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e27fc640 1 -- 172.21.15.89:0/1082895593 <== mon.3 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f46d40b0040 con 0x7f46e40a7140 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:29.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e27fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e3fff640 4 set_mon_vals no callback set 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e27fc640 10 monclient: dump: 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:29.927 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46ebbd2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46ebbd2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46ebbd2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.190:3301/0 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1082895593 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c4005090 con 0x7f46e40a7140 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e27fc640 1 -- 172.21.15.89:0/1082895593 <== mon.3 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f46d40041f0 con 0x7f46e40a7140 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e27fc640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f46d003be00 0x7f46d003e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e27fc640 1 -- 172.21.15.89:0/1082895593 <== mon.3 v2:172.21.15.190:3301/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f46d40e89e0 con 0x7f46e40a7140 2024-08-19T10:56:29.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e996e640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f46d003be00 0x7f46d003e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:29.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.913+0000 7f46e996e640 10 monclient: get_auth_request con 0x7f46d003be00 auth_method 0 2024-08-19T10:56:29.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.917+0000 7f46e996e640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f46d003be00 0x7f46d003e2c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009c00 tx=0x7f46dc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:29.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.925+0000 7f46e27fc640 1 -- 172.21.15.89:0/1082895593 <== mon.3 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f46e40a9360 con 0x7f46e40a7140 2024-08-19T10:56:29.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.925+0000 7f46e27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:29.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:29.925+0000 7f46e27fc640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:30.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.081+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1082895593 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46c4000bd0 con 0x7f46d003be00 2024-08-19T10:56:30.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.085+0000 7f46e27fc640 1 -- 172.21.15.89:0/1082895593 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+78517 (secure 0 0 0) 0x7f46c4000bd0 con 0x7f46d003be00 2024-08-19T10:56:30.085 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:56:30.086 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-08-19T10:56:30.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f46d003be00 msgr2=0x7f46d003e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:30.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f46d003be00 0x7f46d003e2c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009c00 tx=0x7f46dc00b040 comp rx=0 tx=0).stop 2024-08-19T10:56:30.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 10 monclient: shutdown 2024-08-19T10:56:30.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:30.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46e40a7140 msgr2=0x7f46e413e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:30.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46e40a7140 0x7f46e413e280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f46d401cab0 tx=0x7f46d4018fb0 comp rx=0 tx=0).stop 2024-08-19T10:56:30.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1082895593 shutdown_connections 2024-08-19T10:56:30.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f46d003be00 0x7f46d003e2c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:30.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46e40a7140 0x7f46e413e280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:30.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f46e40a6d30 0x7f46e4138e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:30.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 --2- 172.21.15.89:0/1082895593 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f46e40a5270 0x7f46e41388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:30.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1082895593 >> 172.21.15.89:0/1082895593 conn(0x7f46e4069a80 msgr2=0x7f46e4141230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:30.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1082895593 shutdown_connections 2024-08-19T10:56:30.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:30.089+0000 7f46ebbd2640 1 -- 172.21.15.89:0/1082895593 wait complete. 2024-08-19T10:56:30.103 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":820,"stamp":"2024-08-19T10:56:28.776118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":198040,"kb_used_data":10936,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033000,"statfs":{"total":579820584960,"available":579617792000,"internally_reserved":0,"allocated":11198464,"data_stored":8850040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":191486624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003570"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:09.851422+0000","last_change":"2024-08-19T10:43:42.769648+0000","last_active":"2024-08-19T10:54:09.851422+0000","last_peered":"2024-08-19T10:54:09.851422+0000","last_clean":"2024-08-19T10:54:09.851422+0000","last_became_active":"2024-08-19T10:43:42.769057+0000","last_became_peered":"2024-08-19T10:43:42.769057+0000","last_unstale":"2024-08-19T10:54:09.851422+0000","last_undegraded":"2024-08-19T10:54:09.851422+0000","last_fullsized":"2024-08-19T10:54:09.851422+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:59:18.923960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.531111+0000","last_change":"2024-08-19T10:43:42.776790+0000","last_active":"2024-08-19T10:52:24.531111+0000","last_peered":"2024-08-19T10:52:24.531111+0000","last_clean":"2024-08-19T10:52:24.531111+0000","last_became_active":"2024-08-19T10:43:42.775930+0000","last_became_peered":"2024-08-19T10:43:42.775930+0000","last_unstale":"2024-08-19T10:52:24.531111+0000","last_undegraded":"2024-08-19T10:52:24.531111+0000","last_fullsized":"2024-08-19T10:52:24.531111+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:00:49.264074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:14.852152+0000","last_change":"2024-08-19T10:43:42.776699+0000","last_active":"2024-08-19T10:54:14.852152+0000","last_peered":"2024-08-19T10:54:14.852152+0000","last_clean":"2024-08-19T10:54:14.852152+0000","last_became_active":"2024-08-19T10:43:42.775384+0000","last_became_peered":"2024-08-19T10:43:42.775384+0000","last_unstale":"2024-08-19T10:54:14.852152+0000","last_undegraded":"2024-08-19T10:54:14.852152+0000","last_fullsized":"2024-08-19T10:54:14.852152+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:11:44.821797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.531052+0000","last_change":"2024-08-19T10:43:42.770075+0000","last_active":"2024-08-19T10:52:24.531052+0000","last_peered":"2024-08-19T10:52:24.531052+0000","last_clean":"2024-08-19T10:52:24.531052+0000","last_became_active":"2024-08-19T10:43:42.769225+0000","last_became_peered":"2024-08-19T10:43:42.769225+0000","last_unstale":"2024-08-19T10:52:24.531052+0000","last_undegraded":"2024-08-19T10:52:24.531052+0000","last_fullsized":"2024-08-19T10:52:24.531052+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:14:39.665839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":198,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.534076+0000","last_change":"2024-08-19T10:43:42.771832+0000","last_active":"2024-08-19T10:52:24.534076+0000","last_peered":"2024-08-19T10:52:24.534076+0000","last_clean":"2024-08-19T10:52:24.534076+0000","last_became_active":"2024-08-19T10:43:42.770907+0000","last_became_peered":"2024-08-19T10:43:42.770907+0000","last_unstale":"2024-08-19T10:52:24.534076+0000","last_undegraded":"2024-08-19T10:52:24.534076+0000","last_fullsized":"2024-08-19T10:52:24.534076+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:51:29.432515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":202,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:15.416700+0000","last_change":"2024-08-19T10:43:42.789929+0000","last_active":"2024-08-19T10:54:15.416700+0000","last_peered":"2024-08-19T10:54:15.416700+0000","last_clean":"2024-08-19T10:54:15.416700+0000","last_became_active":"2024-08-19T10:43:42.789563+0000","last_became_peered":"2024-08-19T10:43:42.789563+0000","last_unstale":"2024-08-19T10:54:15.416700+0000","last_undegraded":"2024-08-19T10:54:15.416700+0000","last_fullsized":"2024-08-19T10:54:15.416700+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:51:07.959247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":309,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:05.773040+0000","last_change":"2024-08-19T10:43:42.788997+0000","last_active":"2024-08-19T10:53:05.773040+0000","last_peered":"2024-08-19T10:53:05.773040+0000","last_clean":"2024-08-19T10:53:05.773040+0000","last_became_active":"2024-08-19T10:43:42.784411+0000","last_became_peered":"2024-08-19T10:43:42.784411+0000","last_unstale":"2024-08-19T10:53:05.773040+0000","last_undegraded":"2024-08-19T10:53:05.773040+0000","last_fullsized":"2024-08-19T10:53:05.773040+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:44:43.826178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421871+0000","last_change":"2024-08-19T10:43:42.785782+0000","last_active":"2024-08-19T10:54:10.421871+0000","last_peered":"2024-08-19T10:54:10.421871+0000","last_clean":"2024-08-19T10:54:10.421871+0000","last_became_active":"2024-08-19T10:43:42.785474+0000","last_became_peered":"2024-08-19T10:43:42.785474+0000","last_unstale":"2024-08-19T10:54:10.421871+0000","last_undegraded":"2024-08-19T10:54:10.421871+0000","last_fullsized":"2024-08-19T10:54:10.421871+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:37:33.828996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.413864+0000","last_change":"2024-08-19T10:43:42.786904+0000","last_active":"2024-08-19T10:53:49.413864+0000","last_peered":"2024-08-19T10:53:49.413864+0000","last_clean":"2024-08-19T10:53:49.413864+0000","last_became_active":"2024-08-19T10:43:42.786027+0000","last_became_peered":"2024-08-19T10:43:42.786027+0000","last_unstale":"2024-08-19T10:53:49.413864+0000","last_undegraded":"2024-08-19T10:53:49.413864+0000","last_fullsized":"2024-08-19T10:53:49.413864+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:18:23.651707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":316,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:40.769323+0000","last_change":"2024-08-19T10:43:42.785108+0000","last_active":"2024-08-19T10:52:40.769323+0000","last_peered":"2024-08-19T10:52:40.769323+0000","last_clean":"2024-08-19T10:52:40.769323+0000","last_became_active":"2024-08-19T10:43:42.784268+0000","last_became_peered":"2024-08-19T10:43:42.784268+0000","last_unstale":"2024-08-19T10:52:40.769323+0000","last_undegraded":"2024-08-19T10:52:40.769323+0000","last_fullsized":"2024-08-19T10:52:40.769323+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:40:36.868319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":318,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:40.769280+0000","last_change":"2024-08-19T10:43:42.784728+0000","last_active":"2024-08-19T10:52:40.769280+0000","last_peered":"2024-08-19T10:52:40.769280+0000","last_clean":"2024-08-19T10:52:40.769280+0000","last_became_active":"2024-08-19T10:43:42.783709+0000","last_became_peered":"2024-08-19T10:43:42.783709+0000","last_unstale":"2024-08-19T10:52:40.769280+0000","last_undegraded":"2024-08-19T10:52:40.769280+0000","last_fullsized":"2024-08-19T10:52:40.769280+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:30:15.771986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421853+0000","last_change":"2024-08-19T10:43:42.767704+0000","last_active":"2024-08-19T10:54:10.421853+0000","last_peered":"2024-08-19T10:54:10.421853+0000","last_clean":"2024-08-19T10:54:10.421853+0000","last_became_active":"2024-08-19T10:43:42.767225+0000","last_became_peered":"2024-08-19T10:43:42.767225+0000","last_unstale":"2024-08-19T10:54:10.421853+0000","last_undegraded":"2024-08-19T10:54:10.421853+0000","last_fullsized":"2024-08-19T10:54:10.421853+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:29:35.198111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":201,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.415852+0000","last_change":"2024-08-19T10:43:43.768263+0000","last_active":"2024-08-19T10:54:10.415852+0000","last_peered":"2024-08-19T10:54:10.415852+0000","last_clean":"2024-08-19T10:54:10.415852+0000","last_became_active":"2024-08-19T10:43:43.767866+0000","last_became_peered":"2024-08-19T10:43:43.767866+0000","last_unstale":"2024-08-19T10:54:10.415852+0000","last_undegraded":"2024-08-19T10:54:10.415852+0000","last_fullsized":"2024-08-19T10:54:10.415852+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:12:13.038948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":205,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421793+0000","last_change":"2024-08-19T10:43:43.763907+0000","last_active":"2024-08-19T10:54:10.421793+0000","last_peered":"2024-08-19T10:54:10.421793+0000","last_clean":"2024-08-19T10:54:10.421793+0000","last_became_active":"2024-08-19T10:43:43.763033+0000","last_became_peered":"2024-08-19T10:43:43.763033+0000","last_unstale":"2024-08-19T10:54:10.421793+0000","last_undegraded":"2024-08-19T10:54:10.421793+0000","last_fullsized":"2024-08-19T10:54:10.421793+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:15:08.848730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":235,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:54.965756+0000","last_change":"2024-08-19T10:43:43.773879+0000","last_active":"2024-08-19T10:52:54.965756+0000","last_peered":"2024-08-19T10:52:54.965756+0000","last_clean":"2024-08-19T10:52:54.965756+0000","last_became_active":"2024-08-19T10:43:43.773098+0000","last_became_peered":"2024-08-19T10:43:43.773098+0000","last_unstale":"2024-08-19T10:52:54.965756+0000","last_undegraded":"2024-08-19T10:52:54.965756+0000","last_fullsized":"2024-08-19T10:52:54.965756+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:12.561633+0000","last_clean_scrub_stamp":"2024-08-19T10:42:12.561633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:05:31.013999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":202,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:50.412824+0000","last_change":"2024-08-19T10:43:42.776431+0000","last_active":"2024-08-19T10:53:50.412824+0000","last_peered":"2024-08-19T10:53:50.412824+0000","last_clean":"2024-08-19T10:53:50.412824+0000","last_became_active":"2024-08-19T10:43:42.776033+0000","last_became_peered":"2024-08-19T10:43:42.776033+0000","last_unstale":"2024-08-19T10:53:50.412824+0000","last_undegraded":"2024-08-19T10:53:50.412824+0000","last_fullsized":"2024-08-19T10:53:50.412824+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:42:11.547982+0000","last_clean_scrub_stamp":"2024-08-19T10:42:11.547982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:10:47.358711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.423024+0000","last_change":"2024-08-19T10:43:42.807228+0000","last_active":"2024-08-19T10:53:49.423024+0000","last_peered":"2024-08-19T10:53:49.423024+0000","last_clean":"2024-08-19T10:53:49.423024+0000","last_became_active":"2024-08-19T10:43:42.775224+0000","last_became_peered":"2024-08-19T10:43:42.775224+0000","last_unstale":"2024-08-19T10:53:49.423024+0000","last_undegraded":"2024-08-19T10:53:49.423024+0000","last_fullsized":"2024-08-19T10:53:49.423024+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:04.611102+0000","last_clean_scrub_stamp":"2024-08-19T10:38:04.611102+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:21:32.163214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":491,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:49.537692+0000","last_change":"2024-08-19T10:43:42.811458+0000","last_active":"2024-08-19T10:52:49.537692+0000","last_peered":"2024-08-19T10:52:49.537692+0000","last_clean":"2024-08-19T10:52:49.537692+0000","last_became_active":"2024-08-19T10:43:42.777758+0000","last_became_peered":"2024-08-19T10:43:42.777758+0000","last_unstale":"2024-08-19T10:52:49.537692+0000","last_undegraded":"2024-08-19T10:52:49.537692+0000","last_fullsized":"2024-08-19T10:52:49.537692+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:46:54.701371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002082918,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":200,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:49.413826+0000","last_change":"2024-08-19T10:43:42.776830+0000","last_active":"2024-08-19T10:53:49.413826+0000","last_peered":"2024-08-19T10:53:49.413826+0000","last_clean":"2024-08-19T10:53:49.413826+0000","last_became_active":"2024-08-19T10:43:42.775836+0000","last_became_peered":"2024-08-19T10:43:42.775836+0000","last_unstale":"2024-08-19T10:53:49.413826+0000","last_undegraded":"2024-08-19T10:53:49.413826+0000","last_fullsized":"2024-08-19T10:53:49.413826+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:59:14.529046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010100350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:10.421830+0000","last_change":"2024-08-19T10:43:42.767491+0000","last_active":"2024-08-19T10:54:10.421830+0000","last_peered":"2024-08-19T10:54:10.421830+0000","last_clean":"2024-08-19T10:54:10.421830+0000","last_became_active":"2024-08-19T10:43:42.767132+0000","last_became_peered":"2024-08-19T10:43:42.767132+0000","last_unstale":"2024-08-19T10:54:10.421830+0000","last_undegraded":"2024-08-19T10:54:10.421830+0000","last_fullsized":"2024-08-19T10:54:10.421830+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:04:13.207849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001410375,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":544,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:50.770839+0000","last_change":"2024-08-19T10:43:42.796170+0000","last_active":"2024-08-19T10:52:50.770839+0000","last_peered":"2024-08-19T10:52:50.770839+0000","last_clean":"2024-08-19T10:52:50.770839+0000","last_became_active":"2024-08-19T10:43:42.790684+0000","last_became_peered":"2024-08-19T10:43:42.790684+0000","last_unstale":"2024-08-19T10:52:50.770839+0000","last_undegraded":"2024-08-19T10:52:50.770839+0000","last_fullsized":"2024-08-19T10:52:50.770839+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:48:05.328122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001115312,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":515,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:50.531232+0000","last_change":"2024-08-19T10:43:42.769655+0000","last_active":"2024-08-19T10:52:50.531232+0000","last_peered":"2024-08-19T10:52:50.531232+0000","last_clean":"2024-08-19T10:52:50.531232+0000","last_became_active":"2024-08-19T10:43:42.769275+0000","last_became_peered":"2024-08-19T10:43:42.769275+0000","last_unstale":"2024-08-19T10:52:50.531232+0000","last_undegraded":"2024-08-19T10:52:50.531232+0000","last_fullsized":"2024-08-19T10:52:50.531232+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:23:16.606513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010645120000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":198,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:54:14.426772+0000","last_change":"2024-08-19T10:43:42.771283+0000","last_active":"2024-08-19T10:54:14.426772+0000","last_peered":"2024-08-19T10:54:14.426772+0000","last_clean":"2024-08-19T10:54:14.426772+0000","last_became_active":"2024-08-19T10:43:42.770419+0000","last_became_peered":"2024-08-19T10:43:42.770419+0000","last_unstale":"2024-08-19T10:54:14.426772+0000","last_undegraded":"2024-08-19T10:54:14.426772+0000","last_fullsized":"2024-08-19T10:54:14.426772+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:39:44.981500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003718322,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":199,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:52:24.961415+0000","last_change":"2024-08-19T10:43:42.784789+0000","last_active":"2024-08-19T10:52:24.961415+0000","last_peered":"2024-08-19T10:52:24.961415+0000","last_clean":"2024-08-19T10:52:24.961415+0000","last_became_active":"2024-08-19T10:43:42.784451+0000","last_became_peered":"2024-08-19T10:43:42.784451+0000","last_unstale":"2024-08-19T10:52:24.961415+0000","last_undegraded":"2024-08-19T10:52:24.961415+0000","last_fullsized":"2024-08-19T10:52:24.961415+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:19:58.579737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081542699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":480,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:53:24.419480+0000","last_change":"2024-08-19T10:43:43.885027+0000","last_active":"2024-08-19T10:53:24.419480+0000","last_peered":"2024-08-19T10:53:24.419480+0000","last_clean":"2024-08-19T10:53:24.419480+0000","last_became_active":"2024-08-19T10:43:43.884192+0000","last_became_peered":"2024-08-19T10:43:43.884192+0000","last_unstale":"2024-08-19T10:53:24.419480+0000","last_undegraded":"2024-08-19T10:53:24.419480+0000","last_fullsized":"2024-08-19T10:53:24.419480+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:38:11.775089+0000","last_clean_scrub_stamp":"2024-08-19T10:38:11.775089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:52:36.680781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021153309999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":6},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607775,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1868579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607774,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1278230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607777,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1278211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607779,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1278230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640479,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1278211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640481,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1868579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T10:56:30.108 INFO:tasks.ceph:pgid 10.4 last_scrub_stamp 2024-08-19T10:42:12.561633+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=12, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.108 INFO:tasks.ceph:pgid 9.7 last_scrub_stamp 2024-08-19T10:42:11.547982+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.108 INFO:tasks.ceph:pgid 10.5 last_scrub_stamp 2024-08-19T10:42:12.561633+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=12, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.109 INFO:tasks.ceph:pgid 9.6 last_scrub_stamp 2024-08-19T10:42:11.547982+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.109 INFO:tasks.ceph:pgid 10.6 last_scrub_stamp 2024-08-19T10:42:12.561633+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=12, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.109 INFO:tasks.ceph:pgid 9.5 last_scrub_stamp 2024-08-19T10:42:11.547982+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.109 INFO:tasks.ceph:pgid 10.7 last_scrub_stamp 2024-08-19T10:42:12.561633+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=12, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.109 INFO:tasks.ceph:pgid 9.4 last_scrub_stamp 2024-08-19T10:42:11.547982+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.109 INFO:tasks.ceph:pgid 10.0 last_scrub_stamp 2024-08-19T10:42:12.561633+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=12, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.109 INFO:tasks.ceph:pgid 9.3 last_scrub_stamp 2024-08-19T10:42:11.547982+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.109 INFO:tasks.ceph:pgid 10.1 last_scrub_stamp 2024-08-19T10:42:12.561633+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=12, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.109 INFO:tasks.ceph:pgid 9.2 last_scrub_stamp 2024-08-19T10:42:11.547982+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.110 INFO:tasks.ceph:pgid 10.2 last_scrub_stamp 2024-08-19T10:42:12.561633+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=12, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.110 INFO:tasks.ceph:pgid 9.1 last_scrub_stamp 2024-08-19T10:42:11.547982+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.110 INFO:tasks.ceph:pgid 10.3 last_scrub_stamp 2024-08-19T10:42:12.561633+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=12, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.110 INFO:tasks.ceph:pgid 9.0 last_scrub_stamp 2024-08-19T10:42:11.547982+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=42, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.110 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-08-19T10:38:04.611102+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=38, tm_sec=4, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.110 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-08-19T10:38:11.775089+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=38, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.110 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-08-19T10:38:11.775089+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=38, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.110 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-08-19T10:38:11.775089+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=38, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.110 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-08-19T10:38:11.775089+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=38, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.111 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-08-19T10:38:11.775089+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=38, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.111 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-08-19T10:38:11.775089+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=38, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.111 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-08-19T10:38:11.775089+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=38, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.111 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-08-19T10:38:11.775089+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=38, tm_sec=11, tm_wday=0, tm_yday=232, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=19, tm_hour=10, tm_min=56, tm_sec=25, tm_wday=0, tm_yday=232, tm_isdst=0) 2024-08-19T10:56:30.111 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-19T10:56:50.112 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T10:56:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient: get_monmap_and_config 2024-08-19T10:56:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient: build_initial_monmap 2024-08-19T10:56:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient: monmap: 2024-08-19T10:56:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:50.239834+0000 2024-08-19T10:56:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:50.239834+0000 2024-08-19T10:56:50.234 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:50.235 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 1 Processor -- start 2024-08-19T10:56:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 1 -- start start 2024-08-19T10:56:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient: init 2024-08-19T10:56:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient: _add_conns ranks=[10,5,0,1,3,9,2,8,4,7,6] 2024-08-19T10:56:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2b80a3620 0x7fd2b80a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient(hunting): picked mon.noname-f con 0x7fd2b80a3620 addr v2:172.21.15.190:3300/0 2024-08-19T10:56:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b80a3f40 0x7fd2b81363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient(hunting): picked mon.noname-h con 0x7fd2b80a3f40 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd2b8136920 addr v1:172.21.15.89:6789/0 2024-08-19T10:56:50.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b80a0640 con 0x7fd2b8136920 2024-08-19T10:56:50.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:50.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd2b809f390 con 0x7fd2b80a3620 2024-08-19T10:56:50.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:50.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd2b809f670 con 0x7fd2b80a3f40 2024-08-19T10:56:50.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:50.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bf7ec640 10 monclient(hunting): authenticate will time out at 2060.056692s 2024-08-19T10:56:50.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bdd89640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd2b8136920 0x7fd2b813d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:50972/0 (socket says 172.21.15.89:50972) 2024-08-19T10:56:50.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bdd89640 1 -- 172.21.15.89:0/2116623541 learned_addr learned my addr 172.21.15.89:0/2116623541 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-08-19T10:56:50.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 1 --2- 172.21.15.89:0/2116623541 >> v2:172.21.15.190:3300/0 conn(0x7fd2b80a3620 0x7fd2b80a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bcd87640 1 --2- 172.21.15.89:0/2116623541 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b80a3f40 0x7fd2b81363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bcd87640 10 monclient(hunting): get_auth_request con 0x7fd2b80a3f40 auth_method 0 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bcd87640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bcd87640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bcd87640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 10 monclient(hunting): get_auth_request con 0x7fd2b80a3620 auth_method 0 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bcd87640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bcd87640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bcd87640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:50.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 10 monclient(hunting): handle_auth_done global_id 45568 payload 274 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 1 -- 172.21.15.89:0/2116623541 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b80a3f40 msgr2=0x7fd2b81363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 1 --2- 172.21.15.89:0/2116623541 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b80a3f40 0x7fd2b81363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 1 -- 172.21.15.89:0/2116623541 >> v1:172.21.15.89:6789/0 conn(0x7fd2b8136920 legacy=0x7fd2b813d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 10 monclient: _finish_hunting 0 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 1 monclient: found mon.noname-f 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bd588640 1 -- 172.21.15.89:0/2116623541 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b809e170 con 0x7fd2b80a3620 2024-08-19T10:56:50.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.236+0000 7fd2bcd87640 1 --2- 172.21.15.89:0/2116623541 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b80a3f40 0x7fd2b81363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bd588640 1 -- 172.21.15.89:0/2116623541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 msgr2=0x7fd2b80a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bd588640 1 --2- 172.21.15.89:0/2116623541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 0x7fd2b80a3a00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0097c0 tx=0x7fd2ac01ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=643283154faf5217 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 1 -- 172.21.15.89:0/2116623541 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2ac0b0070 con 0x7fd2b80a3620 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 10 monclient: dump: 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:50.243 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 10 monclient: _finish_auth 0 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 10 monclient: _check_auth_tickets 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 1 -- 172.21.15.89:0/2116623541 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac0aa810 con 0x7fd2b80a3620 2024-08-19T10:56:50.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 1 -- 172.21.15.89:0/2116623541 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2ac0a9300 con 0x7fd2b80a3620 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 5 monclient: authenticate success, global_id 45568 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 4 set_mon_vals no callback set 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2abfff640 10 monclient: dump: 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.245 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:50.246 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 10 monclient: shutdown 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2116623541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 msgr2=0x7fd2b80a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 1 --2- 172.21.15.89:0/2116623541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 0x7fd2b80a3a00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0097c0 tx=0x7fd2ac01ce50 comp rx=0 tx=0).stop 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2116623541 shutdown_connections 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 1 --2- 172.21.15.89:0/2116623541 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b80a3f40 0x7fd2b81363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 1 --2- 172.21.15.89:0/2116623541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 0x7fd2b80a3a00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2116623541 >> 172.21.15.89:0/2116623541 conn(0x7fd2b80a8210 msgr2=0x7fd2b80a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2116623541 shutdown_connections 2024-08-19T10:56:50.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.240+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2116623541 wait complete. 2024-08-19T10:56:50.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient: build_initial_monmap 2024-08-19T10:56:50.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient: monmap: 2024-08-19T10:56:50.252 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:50.252 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:50.252 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:50.252 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:50.252 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:50.252 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.252 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:50.252 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:50.252 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 1 Processor -- start 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 1 -- start start 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient: init 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient: _add_conns ranks=[1,2,5,4,8,6,7,0,3] 2024-08-19T10:56:50.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 0x7fd2b8131650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient(hunting): picked mon.noname-f con 0x7fd2b80a3620 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2b80a3f40 0x7fd2b8131b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient(hunting): picked mon.noname-b con 0x7fd2b80a3f40 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b8136920 0x7fd2b81373f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient(hunting): picked mon.noname-h con 0x7fd2b8136920 addr [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b809df50 con 0x7fd2b80a3620 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b809ddd0 con 0x7fd2b80a3f40 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 1 -- --> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] -- mon_getmap magic: 0 -- 0x7fd2b80a1f00 con 0x7fd2b8136920 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:50.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bf7ec640 10 monclient(hunting): authenticate will time out at 2060.065726s 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bcd87640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2b80a3f40 msgr2=0x7fd2b8131b90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bcd87640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2b80a3f40 0x7fd2b8131b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 0x7fd2b8131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bdd89640 1 --2- >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b8136920 0x7fd2b81373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 0x7fd2b8131650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:37476/0 (socket says 172.21.15.89:37476) 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 1 -- 172.21.15.89:0/2473227880 learned_addr learned my addr 172.21.15.89:0/2473227880 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 10 monclient(hunting): get_auth_request con 0x7fd2b80a3620 auth_method 0 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bdd89640 10 monclient(hunting): get_auth_request con 0x7fd2b8136920 auth_method 0 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bdd89640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:50.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bdd89640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bdd89640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bdd89640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bdd89640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bdd89640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 10 monclient(hunting): handle_auth_done global_id 45577 payload 824 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 1 -- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b8136920 msgr2=0x7fd2b81373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b8136920 0x7fd2b81373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 1 -- 172.21.15.89:0/2473227880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2b80a3f40 msgr2=0x7fd2b8131b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2b80a3f40 0x7fd2b8131b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 10 monclient: _finish_hunting 0 2024-08-19T10:56:50.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 1 monclient: found mon.noname-f 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bd588640 1 -- 172.21.15.89:0/2473227880 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b813ea60 con 0x7fd2b80a3620 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.244+0000 7fd2bdd89640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b8136920 0x7fd2b81373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bd588640 1 -- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 msgr2=0x7fd2b8131650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bd588640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 0x7fd2b8131650 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac01cc90 tx=0x7fd2ac019190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 1 -- 172.21.15.89:0/2473227880 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2ac0b0040 con 0x7fd2b80a3620 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 10 monclient: dump: 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:50.257 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:50.258 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 10 monclient: _finish_auth 0 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 10 monclient: _check_auth_tickets 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 1 -- 172.21.15.89:0/2473227880 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac0b9070 con 0x7fd2b80a3620 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bf7ec640 5 monclient: authenticate success, global_id 45577 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bf7ec640 10 monclient: _renew_subs 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bf7ec640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2473227880 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b8138b40 con 0x7fd2b80a3620 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 1 -- 172.21.15.89:0/2473227880 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7fd2ac0044b0 con 0x7fd2b80a3620 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:50.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 10 monclient: dump: 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:50.260 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2ab7fe640 4 set_mon_vals no callback set 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bf7ec640 10 monclient: _renew_subs 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bf7ec640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2473227880 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b813a600 con 0x7fd2b80a3620 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bf7ec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bf7ec640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bf7ec640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2473227880 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b80a3a00 con 0x7fd2b80a3620 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 1 -- 172.21.15.89:0/2473227880 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7fd2ac004040 con 0x7fd2b80a3620 2024-08-19T10:56:50.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd29c03be00 0x7fd29c03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.248+0000 7fd2a9ffb640 1 -- 172.21.15.89:0/2473227880 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7fd2ac0e95f0 con 0x7fd2b80a3620 2024-08-19T10:56:50.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.252+0000 7fd2bcd87640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd29c03be00 0x7fd29c03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:50.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.252+0000 7fd2bcd87640 10 monclient: get_auth_request con 0x7fd29c03be00 auth_method 0 2024-08-19T10:56:50.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.252+0000 7fd2bcd87640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd29c03be00 0x7fd29c03e2c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009580 tx=0x7fd2b4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:50.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.264+0000 7fd2a9ffb640 1 -- 172.21.15.89:0/2473227880 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7fd2ac016070 con 0x7fd2b80a3620 2024-08-19T10:56:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.264+0000 7fd2a9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:50.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.264+0000 7fd2a9ffb640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:50.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.428+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2473227880 --> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2b813deb0 con 0x7fd29c03be00 2024-08-19T10:56:50.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2a9ffb640 1 -- 172.21.15.89:0/2473227880 <== mgr.4105 v2:172.21.15.190:6800/3725341825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+78534 (secure 0 0 0) 0x7fd2b400e8e0 con 0x7fd29c03be00 2024-08-19T10:56:50.433 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-08-19T10:56:50.434 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-08-19T10:56:50.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd29c03be00 msgr2=0x7fd29c03e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:50.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd29c03be00 0x7fd29c03e2c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009580 tx=0x7fd2b4009190 comp rx=0 tx=0).stop 2024-08-19T10:56:50.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 10 monclient: shutdown 2024-08-19T10:56:50.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:50.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 msgr2=0x7fd2b8131650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:50.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 0x7fd2b8131650 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac01cc90 tx=0x7fd2ac019190 comp rx=0 tx=0).stop 2024-08-19T10:56:50.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2473227880 shutdown_connections 2024-08-19T10:56:50.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7fd29c03be00 0x7fd29c03e2c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] conn(0x7fd2b8136920 0x7fd2b81373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7fd2b80a3f40 0x7fd2b8131b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 --2- 172.21.15.89:0/2473227880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2b80a3620 0x7fd2b8131650 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2473227880 >> 172.21.15.89:0/2473227880 conn(0x7fd2b80a8210 msgr2=0x7fd2b809c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:50.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2473227880 shutdown_connections 2024-08-19T10:56:50.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.432+0000 7fd2bf7ec640 1 -- 172.21.15.89:0/2473227880 wait complete. 2024-08-19T10:56:50.451 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":830,"stamp":"2024-08-19T10:56:48.782561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1771161,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":198040,"kb_used_data":10936,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566033000,"statfs":{"total":579820584960,"available":579617792000,"internally_reserved":0,"allocated":11198464,"data_stored":8850040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":191486624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003965"},"pg_stats":[{"pgid":"10.4","version":"0'0","reported_seq":210,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:30.773995+0000","last_change":"2024-08-19T10:56:30.774115+0000","last_active":"2024-08-19T10:56:30.773995+0000","last_peered":"2024-08-19T10:56:30.773995+0000","last_clean":"2024-08-19T10:56:30.773995+0000","last_became_active":"2024-08-19T10:43:42.769057+0000","last_became_peered":"2024-08-19T10:43:42.769057+0000","last_unstale":"2024-08-19T10:56:30.773995+0000","last_undegraded":"2024-08-19T10:56:30.773995+0000","last_fullsized":"2024-08-19T10:56:30.773995+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:30.773775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:30.773775+0000","last_clean_scrub_stamp":"2024-08-19T10:56:30.773775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:05:55.713481+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":209,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:31.080289+0000","last_change":"2024-08-19T10:56:31.080289+0000","last_active":"2024-08-19T10:56:31.080289+0000","last_peered":"2024-08-19T10:56:31.080289+0000","last_clean":"2024-08-19T10:56:31.080289+0000","last_became_active":"2024-08-19T10:43:42.775930+0000","last_became_peered":"2024-08-19T10:43:42.775930+0000","last_unstale":"2024-08-19T10:56:31.080289+0000","last_undegraded":"2024-08-19T10:56:31.080289+0000","last_fullsized":"2024-08-19T10:56:31.080289+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:31.080031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:31.080031+0000","last_clean_scrub_stamp":"2024-08-19T10:56:31.080031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:57:49.473063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":210,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:31.805912+0000","last_change":"2024-08-19T10:56:31.806035+0000","last_active":"2024-08-19T10:56:31.805912+0000","last_peered":"2024-08-19T10:56:31.805912+0000","last_clean":"2024-08-19T10:56:31.805912+0000","last_became_active":"2024-08-19T10:43:42.775384+0000","last_became_peered":"2024-08-19T10:43:42.775384+0000","last_unstale":"2024-08-19T10:56:31.805912+0000","last_undegraded":"2024-08-19T10:56:31.805912+0000","last_fullsized":"2024-08-19T10:56:31.805912+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:31.805699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:31.805699+0000","last_clean_scrub_stamp":"2024-08-19T10:56:31.805699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:42:05.123705+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":209,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:28.045848+0000","last_change":"2024-08-19T10:56:28.045848+0000","last_active":"2024-08-19T10:56:28.045848+0000","last_peered":"2024-08-19T10:56:28.045848+0000","last_clean":"2024-08-19T10:56:28.045848+0000","last_became_active":"2024-08-19T10:43:42.769225+0000","last_became_peered":"2024-08-19T10:43:42.769225+0000","last_unstale":"2024-08-19T10:56:28.045848+0000","last_undegraded":"2024-08-19T10:56:28.045848+0000","last_fullsized":"2024-08-19T10:56:28.045848+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:28.045600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:28.045600+0000","last_clean_scrub_stamp":"2024-08-19T10:56:28.045600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:44:12.557392+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":208,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:32.050578+0000","last_change":"2024-08-19T10:56:32.050578+0000","last_active":"2024-08-19T10:56:32.050578+0000","last_peered":"2024-08-19T10:56:32.050578+0000","last_clean":"2024-08-19T10:56:32.050578+0000","last_became_active":"2024-08-19T10:43:42.770907+0000","last_became_peered":"2024-08-19T10:43:42.770907+0000","last_unstale":"2024-08-19T10:56:32.050578+0000","last_undegraded":"2024-08-19T10:56:32.050578+0000","last_fullsized":"2024-08-19T10:56:32.050578+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:32.050316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:32.050316+0000","last_clean_scrub_stamp":"2024-08-19T10:56:32.050316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:54:00.980370+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":211,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:31.595915+0000","last_change":"2024-08-19T10:56:31.595975+0000","last_active":"2024-08-19T10:56:31.595915+0000","last_peered":"2024-08-19T10:56:31.595915+0000","last_clean":"2024-08-19T10:56:31.595915+0000","last_became_active":"2024-08-19T10:43:42.789563+0000","last_became_peered":"2024-08-19T10:43:42.789563+0000","last_unstale":"2024-08-19T10:56:31.595915+0000","last_undegraded":"2024-08-19T10:56:31.595915+0000","last_fullsized":"2024-08-19T10:56:31.595915+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:31.595733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:31.595733+0000","last_clean_scrub_stamp":"2024-08-19T10:56:31.595733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:47:28.227705+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":318,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:30.738741+0000","last_change":"2024-08-19T10:56:30.738844+0000","last_active":"2024-08-19T10:56:30.738741+0000","last_peered":"2024-08-19T10:56:30.738741+0000","last_clean":"2024-08-19T10:56:30.738741+0000","last_became_active":"2024-08-19T10:43:42.784411+0000","last_became_peered":"2024-08-19T10:43:42.784411+0000","last_unstale":"2024-08-19T10:56:30.738741+0000","last_undegraded":"2024-08-19T10:56:30.738741+0000","last_fullsized":"2024-08-19T10:56:30.738741+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:30.738521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:30.738521+0000","last_clean_scrub_stamp":"2024-08-19T10:56:30.738521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T18:23:40.775565+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":216,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:30.377571+0000","last_change":"2024-08-19T10:56:30.377632+0000","last_active":"2024-08-19T10:56:30.377571+0000","last_peered":"2024-08-19T10:56:30.377571+0000","last_clean":"2024-08-19T10:56:30.377571+0000","last_became_active":"2024-08-19T10:43:42.785474+0000","last_became_peered":"2024-08-19T10:43:42.785474+0000","last_unstale":"2024-08-19T10:56:30.377571+0000","last_undegraded":"2024-08-19T10:56:30.377571+0000","last_fullsized":"2024-08-19T10:56:30.377571+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:30.377386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:30.377386+0000","last_clean_scrub_stamp":"2024-08-19T10:56:30.377386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T16:03:53.748047+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":210,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:30.122696+0000","last_change":"2024-08-19T10:56:30.122786+0000","last_active":"2024-08-19T10:56:30.122696+0000","last_peered":"2024-08-19T10:56:30.122696+0000","last_clean":"2024-08-19T10:56:30.122696+0000","last_became_active":"2024-08-19T10:43:42.786027+0000","last_became_peered":"2024-08-19T10:43:42.786027+0000","last_unstale":"2024-08-19T10:56:30.122696+0000","last_undegraded":"2024-08-19T10:56:30.122696+0000","last_fullsized":"2024-08-19T10:56:30.122696+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:30.122454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:30.122454+0000","last_clean_scrub_stamp":"2024-08-19T10:56:30.122454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T17:18:43.890969+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":325,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:29.731203+0000","last_change":"2024-08-19T10:56:29.731242+0000","last_active":"2024-08-19T10:56:29.731203+0000","last_peered":"2024-08-19T10:56:29.731203+0000","last_clean":"2024-08-19T10:56:29.731203+0000","last_became_active":"2024-08-19T10:43:42.784268+0000","last_became_peered":"2024-08-19T10:43:42.784268+0000","last_unstale":"2024-08-19T10:56:29.731203+0000","last_undegraded":"2024-08-19T10:56:29.731203+0000","last_fullsized":"2024-08-19T10:56:29.731203+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:29.731085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:29.731085+0000","last_clean_scrub_stamp":"2024-08-19T10:56:29.731085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:34:54.708931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":327,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:28.681897+0000","last_change":"2024-08-19T10:56:28.681933+0000","last_active":"2024-08-19T10:56:28.681897+0000","last_peered":"2024-08-19T10:56:28.681897+0000","last_clean":"2024-08-19T10:56:28.681897+0000","last_became_active":"2024-08-19T10:43:42.783709+0000","last_became_peered":"2024-08-19T10:43:42.783709+0000","last_unstale":"2024-08-19T10:56:28.681897+0000","last_undegraded":"2024-08-19T10:56:28.681897+0000","last_fullsized":"2024-08-19T10:56:28.681897+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:28.681777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:28.681777+0000","last_clean_scrub_stamp":"2024-08-19T10:56:28.681777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:41:51.301715+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":216,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:28.341841+0000","last_change":"2024-08-19T10:56:28.341900+0000","last_active":"2024-08-19T10:56:28.341841+0000","last_peered":"2024-08-19T10:56:28.341841+0000","last_clean":"2024-08-19T10:56:28.341841+0000","last_became_active":"2024-08-19T10:43:42.767225+0000","last_became_peered":"2024-08-19T10:43:42.767225+0000","last_unstale":"2024-08-19T10:56:28.341841+0000","last_undegraded":"2024-08-19T10:56:28.341841+0000","last_fullsized":"2024-08-19T10:56:28.341841+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:28.341626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:28.341626+0000","last_clean_scrub_stamp":"2024-08-19T10:56:28.341626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:55:01.683694+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":210,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:33.557033+0000","last_change":"2024-08-19T10:56:33.557095+0000","last_active":"2024-08-19T10:56:33.557033+0000","last_peered":"2024-08-19T10:56:33.557033+0000","last_clean":"2024-08-19T10:56:33.557033+0000","last_became_active":"2024-08-19T10:43:43.767866+0000","last_became_peered":"2024-08-19T10:43:43.767866+0000","last_unstale":"2024-08-19T10:56:33.557033+0000","last_undegraded":"2024-08-19T10:56:33.557033+0000","last_fullsized":"2024-08-19T10:56:33.557033+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:33.556849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:33.556849+0000","last_clean_scrub_stamp":"2024-08-19T10:56:33.556849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T21:38:43.767411+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":214,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:31.376251+0000","last_change":"2024-08-19T10:56:31.376309+0000","last_active":"2024-08-19T10:56:31.376251+0000","last_peered":"2024-08-19T10:56:31.376251+0000","last_clean":"2024-08-19T10:56:31.376251+0000","last_became_active":"2024-08-19T10:43:43.763033+0000","last_became_peered":"2024-08-19T10:43:43.763033+0000","last_unstale":"2024-08-19T10:56:31.376251+0000","last_undegraded":"2024-08-19T10:56:31.376251+0000","last_fullsized":"2024-08-19T10:56:31.376251+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:31.376049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:31.376049+0000","last_clean_scrub_stamp":"2024-08-19T10:56:31.376049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T14:15:56.842716+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":244,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:29.763476+0000","last_change":"2024-08-19T10:56:29.763552+0000","last_active":"2024-08-19T10:56:29.763476+0000","last_peered":"2024-08-19T10:56:29.763476+0000","last_clean":"2024-08-19T10:56:29.763476+0000","last_became_active":"2024-08-19T10:43:43.773098+0000","last_became_peered":"2024-08-19T10:43:43.773098+0000","last_unstale":"2024-08-19T10:56:29.763476+0000","last_undegraded":"2024-08-19T10:56:29.763476+0000","last_fullsized":"2024-08-19T10:56:29.763476+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:29.763272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:29.763272+0000","last_clean_scrub_stamp":"2024-08-19T10:56:29.763272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:32:30.919778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":211,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:32.567502+0000","last_change":"2024-08-19T10:56:32.567629+0000","last_active":"2024-08-19T10:56:32.567502+0000","last_peered":"2024-08-19T10:56:32.567502+0000","last_clean":"2024-08-19T10:56:32.567502+0000","last_became_active":"2024-08-19T10:43:42.776033+0000","last_became_peered":"2024-08-19T10:43:42.776033+0000","last_unstale":"2024-08-19T10:56:32.567502+0000","last_undegraded":"2024-08-19T10:56:32.567502+0000","last_fullsized":"2024-08-19T10:56:32.567502+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:32.567270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:32.567270+0000","last_clean_scrub_stamp":"2024-08-19T10:56:32.567270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:28:03.709410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"274'74","reported_seq":218,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:30.053632+0000","last_change":"2024-08-19T10:56:30.053760+0000","last_active":"2024-08-19T10:56:30.053632+0000","last_peered":"2024-08-19T10:56:30.053632+0000","last_clean":"2024-08-19T10:56:30.053632+0000","last_became_active":"2024-08-19T10:43:42.775224+0000","last_became_peered":"2024-08-19T10:43:42.775224+0000","last_unstale":"2024-08-19T10:56:30.053632+0000","last_undegraded":"2024-08-19T10:56:30.053632+0000","last_fullsized":"2024-08-19T10:56:30.053632+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'74","last_scrub_stamp":"2024-08-19T10:56:30.053415+0000","last_deep_scrub":"274'74","last_deep_scrub_stamp":"2024-08-19T10:56:30.053415+0000","last_clean_scrub_stamp":"2024-08-19T10:56:30.053415+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:18:28.758361+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"274'2","reported_seq":501,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:28.073624+0000","last_change":"2024-08-19T10:56:28.073752+0000","last_active":"2024-08-19T10:56:28.073624+0000","last_peered":"2024-08-19T10:56:28.073624+0000","last_clean":"2024-08-19T10:56:28.073624+0000","last_became_active":"2024-08-19T10:43:42.777758+0000","last_became_peered":"2024-08-19T10:43:42.777758+0000","last_unstale":"2024-08-19T10:56:28.073624+0000","last_undegraded":"2024-08-19T10:56:28.073624+0000","last_fullsized":"2024-08-19T10:56:28.073624+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'2","last_scrub_stamp":"2024-08-19T10:56:28.073410+0000","last_deep_scrub":"274'2","last_deep_scrub_stamp":"2024-08-19T10:56:28.073410+0000","last_clean_scrub_stamp":"2024-08-19T10:56:28.073410+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:59:23.499519+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.002082918,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":209,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:29.075538+0000","last_change":"2024-08-19T10:56:29.075648+0000","last_active":"2024-08-19T10:56:29.075538+0000","last_peered":"2024-08-19T10:56:29.075538+0000","last_clean":"2024-08-19T10:56:29.075538+0000","last_became_active":"2024-08-19T10:43:42.775836+0000","last_became_peered":"2024-08-19T10:43:42.775836+0000","last_unstale":"2024-08-19T10:56:29.075538+0000","last_undegraded":"2024-08-19T10:56:29.075538+0000","last_fullsized":"2024-08-19T10:56:29.075538+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:29.075259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:29.075259+0000","last_clean_scrub_stamp":"2024-08-19T10:56:29.075259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T19:04:17.705937+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010100350000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:29.340595+0000","last_change":"2024-08-19T10:56:29.340652+0000","last_active":"2024-08-19T10:56:29.340595+0000","last_peered":"2024-08-19T10:56:29.340595+0000","last_clean":"2024-08-19T10:56:29.340595+0000","last_became_active":"2024-08-19T10:43:42.767132+0000","last_became_peered":"2024-08-19T10:43:42.767132+0000","last_unstale":"2024-08-19T10:56:29.340595+0000","last_undegraded":"2024-08-19T10:56:29.340595+0000","last_fullsized":"2024-08-19T10:56:29.340595+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:29.340417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:29.340417+0000","last_clean_scrub_stamp":"2024-08-19T10:56:29.340417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:22:58.355528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001410375,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"274'3","reported_seq":554,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:27.638654+0000","last_change":"2024-08-19T10:56:27.638779+0000","last_active":"2024-08-19T10:56:27.638654+0000","last_peered":"2024-08-19T10:56:27.638654+0000","last_clean":"2024-08-19T10:56:27.638654+0000","last_became_active":"2024-08-19T10:43:42.790684+0000","last_became_peered":"2024-08-19T10:43:42.790684+0000","last_unstale":"2024-08-19T10:56:27.638654+0000","last_undegraded":"2024-08-19T10:56:27.638654+0000","last_fullsized":"2024-08-19T10:56:27.638654+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"274'3","last_scrub_stamp":"2024-08-19T10:56:27.638448+0000","last_deep_scrub":"274'3","last_deep_scrub_stamp":"2024-08-19T10:56:27.638448+0000","last_clean_scrub_stamp":"2024-08-19T10:56:27.638448+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T22:06:20.703680+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001115312,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":524,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:30.597008+0000","last_change":"2024-08-19T10:56:30.597072+0000","last_active":"2024-08-19T10:56:30.597008+0000","last_peered":"2024-08-19T10:56:30.597008+0000","last_clean":"2024-08-19T10:56:30.597008+0000","last_became_active":"2024-08-19T10:43:42.769275+0000","last_became_peered":"2024-08-19T10:43:42.769275+0000","last_unstale":"2024-08-19T10:56:30.597008+0000","last_undegraded":"2024-08-19T10:56:30.597008+0000","last_fullsized":"2024-08-19T10:56:30.597008+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:30.596829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:30.596829+0000","last_clean_scrub_stamp":"2024-08-19T10:56:30.596829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:05:03.896850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010645120000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":207,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:29.088491+0000","last_change":"2024-08-19T10:56:29.088548+0000","last_active":"2024-08-19T10:56:29.088491+0000","last_peered":"2024-08-19T10:56:29.088491+0000","last_clean":"2024-08-19T10:56:29.088491+0000","last_became_active":"2024-08-19T10:43:42.770419+0000","last_became_peered":"2024-08-19T10:43:42.770419+0000","last_unstale":"2024-08-19T10:56:29.088491+0000","last_undegraded":"2024-08-19T10:56:29.088491+0000","last_fullsized":"2024-08-19T10:56:29.088491+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:29.088324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:29.088324+0000","last_clean_scrub_stamp":"2024-08-19T10:56:29.088324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T20:34:37.950397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003718322,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":209,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:28.792992+0000","last_change":"2024-08-19T10:56:28.792992+0000","last_active":"2024-08-19T10:56:28.792992+0000","last_peered":"2024-08-19T10:56:28.792992+0000","last_clean":"2024-08-19T10:56:28.792992+0000","last_became_active":"2024-08-19T10:43:42.784451+0000","last_became_peered":"2024-08-19T10:43:42.784451+0000","last_unstale":"2024-08-19T10:56:28.792992+0000","last_undegraded":"2024-08-19T10:56:28.792992+0000","last_fullsized":"2024-08-19T10:56:28.792992+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:28.792741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:28.792741+0000","last_clean_scrub_stamp":"2024-08-19T10:56:28.792741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T15:05:03.883624+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00081542699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":489,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-08-19T10:56:31.039657+0000","last_change":"2024-08-19T10:56:31.039729+0000","last_active":"2024-08-19T10:56:31.039657+0000","last_peered":"2024-08-19T10:56:31.039657+0000","last_clean":"2024-08-19T10:56:31.039657+0000","last_became_active":"2024-08-19T10:43:43.884192+0000","last_became_peered":"2024-08-19T10:43:43.884192+0000","last_unstale":"2024-08-19T10:56:31.039657+0000","last_undegraded":"2024-08-19T10:56:31.039657+0000","last_fullsized":"2024-08-19T10:56:31.039657+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T10:56:31.039469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T10:56:31.039469+0000","last_clean_scrub_stamp":"2024-08-19T10:56:31.039469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:44:48.171541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021153309999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":6},{"poolid":9,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607779,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1868579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607778,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1278230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607781,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1278211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607783,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1278230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640483,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":1278211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640485,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1868579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T10:56:50.454 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-08-19T10:56:50.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient: get_monmap_and_config 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient: build_initial_monmap 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient: monmap: 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:56:50.580609+0000 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:56:50.580609+0000 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:0: v1:172.21.15.89:6789/0 mon.noname-a-legacy 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:1: v1:172.21.15.190:6789/0 mon.noname-f-legacy 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:50.575 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:50.576 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:50.576 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:50.576 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:50.576 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:50.576 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:50.576 INFO:teuthology.orchestra.run.smithi089.stderr:9: v2:172.21.15.89:3300/0 mon.noname-a 2024-08-19T10:56:50.576 INFO:teuthology.orchestra.run.smithi089.stderr:10: v2:172.21.15.190:3300/0 mon.noname-f 2024-08-19T10:56:50.576 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 1 Processor -- start 2024-08-19T10:56:50.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 1 -- start start 2024-08-19T10:56:50.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient: init 2024-08-19T10:56:50.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:50.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient: _add_conns ranks=[1,8,10,3,2,4,0,9,7,5,6] 2024-08-19T10:56:50.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3a780a6860 addr v1:172.21.15.190:6789/0 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3a780a81b0 0x7f3a780a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient(hunting): picked mon.noname-e con 0x7f3a780a81b0 addr [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3a780a7180 0x7f3a781408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient(hunting): picked mon.noname-f con 0x7f3a780a7180 addr v2:172.21.15.190:3300/0 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a780a4c90 con 0x7f3a780a6860 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f3a780a7630 con 0x7f3a780a7180 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 1 -- --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_getmap magic: 0 -- 0x7f3a780a8590 con 0x7f3a780a81b0 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7e707640 10 monclient(hunting): authenticate will time out at 2060.397169s 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3a780a81b0 0x7f3a780a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:50.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 1 --2- >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3a780a81b0 0x7f3a780a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3304/0 says I am v2:172.21.15.89:35124/0 (socket says 172.21.15.89:35124) 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 1 -- 172.21.15.89:0/2503689385 learned_addr learned my addr 172.21.15.89:0/2503689385 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 10 monclient(hunting): get_auth_request con 0x7f3a780a81b0 auth_method 0 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7cca4640 1 --2- 172.21.15.89:0/2503689385 >> v2:172.21.15.190:3300/0 conn(0x7f3a780a7180 0x7f3a781408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 10 monclient(hunting): handle_auth_done global_id 114470 payload 274 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 1 -- 172.21.15.89:0/2503689385 >> v2:172.21.15.190:3300/0 conn(0x7f3a780a7180 msgr2=0x7f3a781408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 1 --2- 172.21.15.89:0/2503689385 >> v2:172.21.15.190:3300/0 conn(0x7f3a780a7180 0x7f3a781408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 1 -- 172.21.15.89:0/2503689385 >> v1:172.21.15.190:6789/0 conn(0x7f3a780a6860 legacy=0x7f3a780a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T10:56:50.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 10 monclient: _finish_hunting 0 2024-08-19T10:56:50.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 1 monclient: found mon.noname-e 2024-08-19T10:56:50.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:50.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.89:3304/0 2024-08-19T10:56:50.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a777fe640 1 -- 172.21.15.89:0/2503689385 --> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a78141e30 con 0x7f3a780a81b0 2024-08-19T10:56:50.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7cca4640 10 monclient: get_auth_request con 0x7f3a780a7180 auth_method 0 2024-08-19T10:56:50.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.576+0000 7f3a7cca4640 1 --2- 172.21.15.89:0/2503689385 >> v2:172.21.15.190:3300/0 conn(0x7f3a780a7180 0x7f3a781408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T10:56:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a777fe640 1 -- 172.21.15.89:0/2503689385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3a780a81b0 msgr2=0x7f3a780a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a777fe640 1 --2- 172.21.15.89:0/2503689385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3a780a81b0 0x7f3a780a6c40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009870 tx=0x7f3a6c01c4b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6d6dcdd05210e843 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 1 -- 172.21.15.89:0/2503689385 <== mon.8 v2:172.21.15.89:3304/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3a6c0b0070 con 0x7f3a780a81b0 2024-08-19T10:56:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-08-19T10:56:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 10 monclient: dump: 2024-08-19T10:56:50.585 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:50.586 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 10 monclient: _finish_auth 0 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 10 monclient: _check_auth_tickets 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 5 monclient: authenticate success, global_id 114470 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 1 -- 172.21.15.89:0/2503689385 <== mon.8 v2:172.21.15.89:3304/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a6c0a9a60 con 0x7f3a780a81b0 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 10 monclient: get_monmap_and_config success 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 1 -- 172.21.15.89:0/2503689385 <== mon.8 v2:172.21.15.89:3304/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3a6c0a9c50 con 0x7f3a780a81b0 2024-08-19T10:56:50.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 4 set_mon_vals no callback set 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a76ffd640 10 monclient: dump: 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:50.588 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 10 monclient: shutdown 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 1 -- 172.21.15.89:0/2503689385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3a780a81b0 msgr2=0x7f3a780a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 1 --2- 172.21.15.89:0/2503689385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3a780a81b0 0x7f3a780a6c40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009870 tx=0x7f3a6c01c4b0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 1 -- 172.21.15.89:0/2503689385 shutdown_connections 2024-08-19T10:56:50.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 1 --2- 172.21.15.89:0/2503689385 >> v2:172.21.15.190:3300/0 conn(0x7f3a780a7180 0x7f3a781408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 1 --2- 172.21.15.89:0/2503689385 >> [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] conn(0x7f3a780a81b0 0x7f3a780a6c40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 1 -- 172.21.15.89:0/2503689385 >> 172.21.15.89:0/2503689385 conn(0x7f3a78069a00 msgr2=0x7f3a780abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 1 -- 172.21.15.89:0/2503689385 shutdown_connections 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 1 -- 172.21.15.89:0/2503689385 wait complete. 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 10 monclient: build_initial_monmap 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.580+0000 7f3a7e707640 10 monclient: monmap: 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 0 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 0.000000 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:created 0.000000 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 0 (unknown) 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.590 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.noname-a 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.noname-f 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.noname-b 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.noname-g 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.noname-c 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.noname-h 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.noname-d 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.noname-i 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.noname-e 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 1 Processor -- start 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 1 -- start start 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 10 monclient: init 2024-08-19T10:56:50.591 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 10 monclient: _reopen_session rank -1 2024-08-19T10:56:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 10 monclient: _add_conns ranks=[1,7,2,8,0,5,3,6,4] 2024-08-19T10:56:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a780a6860 0x7f3a781d1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 10 monclient(hunting): picked mon.noname-f con 0x7f3a780a6860 addr [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T10:56:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3a780a7180 0x7f3a781d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 10 monclient(hunting): picked mon.noname-i con 0x7f3a780a7180 addr [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] 2024-08-19T10:56:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a780a81b0 0x7f3a781375d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 10 monclient(hunting): picked mon.noname-b con 0x7f3a780a81b0 addr [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] 2024-08-19T10:56:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3a78141a00 con 0x7f3a780a6860 2024-08-19T10:56:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:50.592 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 1 -- --> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] -- mon_getmap magic: 0 -- 0x7f3a78141880 con 0x7f3a780a81b0 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 10 monclient(hunting): start opening mon connection 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 1 -- --> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] -- mon_getmap magic: 0 -- 0x7f3a78141b80 con 0x7f3a780a7180 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 10 monclient(hunting): _renew_subs 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7e707640 10 monclient(hunting): authenticate will time out at 2060.404786s 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7cca4640 1 -- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a780a81b0 msgr2=0x7f3a781375d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:3301/0 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a7cca4640 1 --2- >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a780a81b0 0x7f3a781375d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a777fe640 1 -- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3a780a7180 msgr2=0x7f3a781d15b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.190:3303/0 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a777fe640 1 --2- >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3a780a7180 0x7f3a781d15b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a780a6860 0x7f3a781d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a780a6860 0x7f3a781d1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.89:37500/0 (socket says 172.21.15.89:37500) 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 1 -- 172.21.15.89:0/4118871379 learned_addr learned my addr 172.21.15.89:0/4118871379 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 10 monclient(hunting): get_auth_request con 0x7f3a780a6860 auth_method 0 2024-08-19T10:56:50.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 10 monclient(hunting): _init_auth method 2 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 10 monclient(hunting): _init_auth creating new auth 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 10 monclient(hunting): handle_auth_done global_id 45586 payload 824 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 1 -- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3a780a7180 msgr2=0x7f3a781d15b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3a780a7180 0x7f3a781d15b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 1 -- 172.21.15.89:0/4118871379 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a780a81b0 msgr2=0x7f3a781375d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a780a81b0 0x7f3a781375d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 10 monclient: _finish_hunting 0 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 1 monclient: found mon.noname-f 2024-08-19T10:56:50.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-08-19T10:56:50.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.190:3300/0 2024-08-19T10:56:50.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.584+0000 7f3a77fff640 1 -- 172.21.15.89:0/4118871379 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7813a830 con 0x7f3a780a6860 2024-08-19T10:56:50.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a77fff640 1 -- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a780a6860 msgr2=0x7f3a781d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-08-19T10:56:50.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a77fff640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a780a6860 0x7f3a781d1070 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800b410 tx=0x7f3a6800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:50.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 1 -- 172.21.15.89:0/4118871379 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3a68015020 con 0x7f3a780a6860 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 10 monclient: dump: 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:50.597 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 10 monclient: _finish_auth 0 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 10 monclient: _check_auth_tickets 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 1 -- 172.21.15.89:0/4118871379 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a68011cb0 con 0x7f3a780a6860 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 10 monclient: handle_config config(0 keys) 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 1 -- 172.21.15.89:0/4118871379 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 1037+0+0 (secure 0 0 0) 0x7f3a68011e60 con 0x7f3a780a6860 2024-08-19T10:56:50.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 10 monclient: dump: 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:epoch 1 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:fsid b6e37c79-20e9-449f-b89e-c2165d4f4bed 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:last_changed 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:created 2024-08-19T10:37:22.945188+0000 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:min_mon_release 19 (squid) 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:election_strategy: 1 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.f 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:2: [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] mon.b 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:3: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.g 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:4: [v2:172.21.15.89:3302/0,v1:172.21.15.89:6791/0] mon.c 2024-08-19T10:56:50.599 INFO:teuthology.orchestra.run.smithi089.stderr:5: [v2:172.21.15.190:3302/0,v1:172.21.15.190:6791/0] mon.h 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:6: [v2:172.21.15.89:3303/0,v1:172.21.15.89:6792/0] mon.d 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:7: [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] mon.i 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:8: [v2:172.21.15.89:3304/0,v1:172.21.15.89:6793/0] mon.e 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a767fc640 4 set_mon_vals no callback set 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a7e707640 5 monclient: authenticate success, global_id 45586 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a7e707640 10 monclient: _renew_subs 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a7e707640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a7e707640 1 -- 172.21.15.89:0/4118871379 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7813ab10 con 0x7f3a780a6860 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a7e707640 10 monclient: _renew_subs 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a7e707640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a7e707640 1 -- 172.21.15.89:0/4118871379 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7813bab0 con 0x7f3a780a6860 2024-08-19T10:56:50.600 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a7e707640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a7e707640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a7e707640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a7e707640 1 -- 172.21.15.89:0/4118871379 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a780a6c40 con 0x7f3a780a6860 2024-08-19T10:56:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 1 -- 172.21.15.89:0/4118871379 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 47793+0+0 (secure 0 0 0) 0x7f3a68018020 con 0x7f3a780a6860 2024-08-19T10:56:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3a6403bd30 0x7f3a6403e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T10:56:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a74ff9640 1 -- 172.21.15.89:0/4118871379 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(274..274 src has 246..274) ==== 7657+0+0 (secure 0 0 0) 0x7f3a6804f880 con 0x7f3a780a6860 2024-08-19T10:56:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a777fe640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3a6403bd30 0x7f3a6403e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T10:56:50.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.588+0000 7f3a777fe640 10 monclient: get_auth_request con 0x7f3a6403bd30 auth_method 0 2024-08-19T10:56:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.592+0000 7f3a777fe640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3a6403bd30 0x7f3a6403e1f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3a780a6330 tx=0x7f3a6c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T10:56:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.604+0000 7f3a74ff9640 1 -- 172.21.15.89:0/4118871379 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186915 (secure 0 0 0) 0x7f3a6801f070 con 0x7f3a780a6860 2024-08-19T10:56:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.604+0000 7f3a74ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-08-19T10:56:50.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.604+0000 7f3a74ff9640 10 monclient: _finish_command 1 = system:0 2024-08-19T10:56:50.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.776+0000 7f3a7e707640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-08-19T10:56:50.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.776+0000 7f3a7e707640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-08-19T10:56:50.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.776+0000 7f3a7e707640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.190:3300/0 2024-08-19T10:56:50.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.776+0000 7f3a7e707640 1 -- 172.21.15.89:0/4118871379 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f3a78064590 con 0x7f3a780a6860 2024-08-19T10:56:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.784+0000 7f3a74ff9640 1 -- 172.21.15.89:0/4118871379 <== mon.1 v2:172.21.15.190:3300/0 7 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3a68011860 con 0x7f3a780a6860 2024-08-19T10:56:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.784+0000 7f3a74ff9640 10 monclient: handle_config config(1 keys) 2024-08-19T10:56:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.784+0000 7f3a767fc640 4 set_mon_vals no callback set 2024-08-19T10:56:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.784+0000 7f3a767fc640 10 set_mon_vals mon_health_to_clog = false 2024-08-19T10:56:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.784+0000 7f3a74ff9640 1 -- 172.21.15.89:0/4118871379 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f3a6804c790 con 0x7f3a780a6860 2024-08-19T10:56:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.784+0000 7f3a74ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-08-19T10:56:50.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.784+0000 7f3a74ff9640 10 monclient: _finish_command 2 = system:0 2024-08-19T10:56:50.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 -- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3a6403bd30 msgr2=0x7f3a6403e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3a6403bd30 0x7f3a6403e1f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3a780a6330 tx=0x7f3a6c0a6000 comp rx=0 tx=0).stop 2024-08-19T10:56:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 10 monclient: shutdown 2024-08-19T10:56:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 20 monclient: shutdown discarding 0 pending message(s) 2024-08-19T10:56:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 -- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a780a6860 msgr2=0x7f3a781d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T10:56:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a780a6860 0x7f3a781d1070 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800b410 tx=0x7f3a6800b8d0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 -- 172.21.15.89:0/4118871379 shutdown_connections 2024-08-19T10:56:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:6800/3725341825,v1:172.21.15.190:6801/3725341825] conn(0x7f3a6403bd30 0x7f3a6403e1f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.89:3301/0,v1:172.21.15.89:6790/0] conn(0x7f3a780a81b0 0x7f3a781375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:3303/0,v1:172.21.15.190:6792/0] conn(0x7f3a780a7180 0x7f3a781d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 --2- 172.21.15.89:0/4118871379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a780a6860 0x7f3a781d1070 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T10:56:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 -- 172.21.15.89:0/4118871379 >> 172.21.15.89:0/4118871379 conn(0x7f3a78069a00 msgr2=0x7f3a780abc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T10:56:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 -- 172.21.15.89:0/4118871379 shutdown_connections 2024-08-19T10:56:50.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-08-19T10:56:50.788+0000 7f3a7e707640 1 -- 172.21.15.89:0/4118871379 wait complete. 2024-08-19T10:56:50.802 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa/tasks/ceph.py", line 1980, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa/tasks/mon_thrash.py", line 466, in task thrash_proc.stop_and_join() File "/home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa/tasks/mon_thrash.py", line 178, in stop_and_join return self.join() File "/home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa/tasks/mon_thrash.py", line 170, in join self.stopping.set() AttributeError: 'bool' object has no attribute 'set' 2024-08-19T10:56:50.803 INFO:teuthology.misc:Shutting down mds daemons... 2024-08-19T10:56:50.803 INFO:teuthology.misc:Shutting down osd daemons... 2024-08-19T10:56:50.803 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-08-19T10:56:50.803 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:50.870 INFO:tasks.ceph.osd.0:Stopped 2024-08-19T10:56:50.871 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-08-19T10:56:50.871 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:50.929 INFO:tasks.ceph.osd.1:Stopped 2024-08-19T10:56:50.929 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-08-19T10:56:50.929 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:50.990 INFO:tasks.ceph.osd.2:Stopped 2024-08-19T10:56:50.990 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-08-19T10:56:50.990 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:51.061 INFO:tasks.ceph.osd.3:Stopped 2024-08-19T10:56:51.061 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-08-19T10:56:51.061 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:51.142 INFO:tasks.ceph.osd.4:Stopped 2024-08-19T10:56:51.142 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-08-19T10:56:51.142 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:51.216 INFO:tasks.ceph.osd.5:Stopped 2024-08-19T10:56:51.216 INFO:teuthology.misc:Shutting down mgr daemons... 2024-08-19T10:56:51.217 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-08-19T10:56:51.217 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:51.246 INFO:tasks.ceph.mgr.x:Stopped 2024-08-19T10:56:51.247 INFO:teuthology.misc:Shutting down mon daemons... 2024-08-19T10:56:51.247 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-08-19T10:56:51.247 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:51.285 INFO:tasks.ceph.mon.a:Stopped 2024-08-19T10:56:51.286 ERROR:tasks.ceph.mon.b:tried to stop a non-running daemon 2024-08-19T10:56:51.286 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-08-19T10:56:51.286 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:51.352 INFO:tasks.ceph.mon.c:Stopped 2024-08-19T10:56:51.352 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-08-19T10:56:51.352 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:51.424 INFO:tasks.ceph.mon.d:Stopped 2024-08-19T10:56:51.424 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-08-19T10:56:51.424 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:51.492 INFO:tasks.ceph.mon.e:Stopped 2024-08-19T10:56:51.492 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-08-19T10:56:51.492 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:51.524 INFO:tasks.ceph.mon.f:Stopped 2024-08-19T10:56:51.524 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-08-19T10:56:51.524 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:51.587 INFO:tasks.ceph.mon.g:Stopped 2024-08-19T10:56:51.587 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-08-19T10:56:51.587 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T10:56:51.650 INFO:tasks.ceph.mon.h:Stopped 2024-08-19T10:56:51.650 ERROR:tasks.ceph.mon.i:tried to stop a non-running daemon 2024-08-19T10:56:51.651 INFO:tasks.ceph:Checking cluster log for badness... 2024-08-19T10:56:51.651 DEBUG:teuthology.orchestra.run.smithi089:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v ScrubResult | egrep -v 'scrub mismatch' | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(AUTH_BAD_CAPS\)' | head -n 1 2024-08-19T10:56:51.652 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.d is failed for ~0s 2024-08-19T10:56:51.652 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.e is failed for ~0s 2024-08-19T10:56:51.652 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.f is failed for ~0s 2024-08-19T10:56:51.652 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.g is failed for ~0s 2024-08-19T10:56:51.652 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.h is failed for ~0s 2024-08-19T10:56:51.684 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi089.front.sepia.ceph.com 2024-08-19T10:56:51.684 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-08-19T10:56:51.939 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi089.front.sepia.ceph.com 2024-08-19T10:56:51.939 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-08-19T10:56:52.118 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi089.front.sepia.ceph.com 2024-08-19T10:56:52.119 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-08-19T10:56:52.251 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi190.front.sepia.ceph.com 2024-08-19T10:56:52.251 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-08-19T10:56:52.430 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi190.front.sepia.ceph.com 2024-08-19T10:56:52.430 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-08-19T10:56:52.652 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi190.front.sepia.ceph.com 2024-08-19T10:56:52.653 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-08-19T10:56:52.833 INFO:tasks.ceph:Archiving mon data... 2024-08-19T10:56:52.835 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/data/mon.a.tgz 2024-08-19T10:56:52.835 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-08-19T10:56:52.840 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.Rd63SpoNPb 2024-08-19T10:56:52.841 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Rd63SpoNPb 2024-08-19T10:56:53.049 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.Rd63SpoNPb 2024-08-19T10:56:53.111 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.Rd63SpoNPb is 990KB 2024-08-19T10:56:53.177 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.Rd63SpoNPb 2024-08-19T10:56:53.183 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/data/mon.b.tgz 2024-08-19T10:56:53.183 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-08-19T10:56:53.231 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.ULe5Mid4h5 2024-08-19T10:56:53.231 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ULe5Mid4h5 2024-08-19T10:56:53.449 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.ULe5Mid4h5 2024-08-19T10:56:53.512 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.ULe5Mid4h5 is 1012KB 2024-08-19T10:56:53.578 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.ULe5Mid4h5 2024-08-19T10:56:53.583 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/data/mon.c.tgz 2024-08-19T10:56:53.584 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-08-19T10:56:53.631 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.c9ZWxToNSP 2024-08-19T10:56:53.632 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.c9ZWxToNSP 2024-08-19T10:56:53.839 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.c9ZWxToNSP 2024-08-19T10:56:53.902 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.c9ZWxToNSP is 982KB 2024-08-19T10:56:53.970 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.c9ZWxToNSP 2024-08-19T10:56:53.975 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-d to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/data/mon.d.tgz 2024-08-19T10:56:53.975 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-08-19T10:56:54.023 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.WDFX02zV6l 2024-08-19T10:56:54.024 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-d -- . > /tmp/tmp.WDFX02zV6l 2024-08-19T10:56:54.236 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.WDFX02zV6l 2024-08-19T10:56:54.299 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.WDFX02zV6l is 989KB 2024-08-19T10:56:54.365 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.WDFX02zV6l 2024-08-19T10:56:54.371 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-e to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/data/mon.e.tgz 2024-08-19T10:56:54.371 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-08-19T10:56:54.419 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.rTmObBm5Eh 2024-08-19T10:56:54.419 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-e -- . > /tmp/tmp.rTmObBm5Eh 2024-08-19T10:56:54.626 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.rTmObBm5Eh 2024-08-19T10:56:54.691 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.rTmObBm5Eh is 979KB 2024-08-19T10:56:54.753 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.rTmObBm5Eh 2024-08-19T10:56:54.758 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-f to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/data/mon.f.tgz 2024-08-19T10:56:54.758 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-08-19T10:56:54.764 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.xcoy8QWwkV 2024-08-19T10:56:54.765 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-f -- . > /tmp/tmp.xcoy8QWwkV 2024-08-19T10:56:54.971 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.xcoy8QWwkV 2024-08-19T10:56:55.036 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.xcoy8QWwkV is 986KB 2024-08-19T10:56:55.288 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.xcoy8QWwkV 2024-08-19T10:56:55.294 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-g to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/data/mon.g.tgz 2024-08-19T10:56:55.294 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-08-19T10:56:55.342 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.BSrv0JjmJp 2024-08-19T10:56:55.343 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-g -- . > /tmp/tmp.BSrv0JjmJp 2024-08-19T10:56:55.558 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.BSrv0JjmJp 2024-08-19T10:56:55.622 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.BSrv0JjmJp is 979KB 2024-08-19T10:56:55.692 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.BSrv0JjmJp 2024-08-19T10:56:55.697 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-h to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/data/mon.h.tgz 2024-08-19T10:56:55.697 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-08-19T10:56:55.746 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.hXUGX6zEBH 2024-08-19T10:56:55.746 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-h -- . > /tmp/tmp.hXUGX6zEBH 2024-08-19T10:56:55.964 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.hXUGX6zEBH 2024-08-19T10:56:56.030 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.hXUGX6zEBH is 982KB 2024-08-19T10:56:56.100 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.hXUGX6zEBH 2024-08-19T10:56:56.105 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-i to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/data/mon.i.tgz 2024-08-19T10:56:56.105 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-08-19T10:56:56.154 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.63yswV9kGh 2024-08-19T10:56:56.155 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-i -- . > /tmp/tmp.63yswV9kGh 2024-08-19T10:56:56.375 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.63yswV9kGh 2024-08-19T10:56:56.442 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.63yswV9kGh is 1006KB 2024-08-19T10:56:56.552 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.63yswV9kGh 2024-08-19T10:56:56.558 INFO:tasks.ceph:Cleaning ceph cluster... 2024-08-19T10:56:56.559 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-19T10:56:56.561 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-19T10:56:56.619 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-08-19T10:56:56.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.d has been restored 2024-08-19T10:56:56.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.e has been restored 2024-08-19T10:56:56.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.f has been restored 2024-08-19T10:56:56.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.g has been restored 2024-08-19T10:56:56.653 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.h has been restored 2024-08-19T10:56:56.662 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-08-19T10:56:56.662 INFO:tasks.ceph:Archiving crash dumps... 2024-08-19T10:56:56.664 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/remote/smithi089/crash 2024-08-19T10:56:56.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-19T10:56:56.677 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/remote/smithi190/crash 2024-08-19T10:56:56.678 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-19T10:56:56.718 INFO:tasks.ceph:Compressing logs... 2024-08-19T10:56:56.718 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-19T10:56:56.724 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-19T10:56:56.754 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29919.log 2024-08-19T10:56:56.755 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37422.log 2024-08-19T10:56:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36576.log 2024-08-19T10:56:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37422.log: gzip -5 --verbose -- 89.9% /var/log/ceph/ceph-client.admin.29841.log -- replaced with /var/log/ceph/ceph-client.admin.29919.log.gz 2024-08-19T10:56:56.756 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30640.log 2024-08-19T10:56:56.756 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.37422.log.gz 2024-08-19T10:56:56.756 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31507.log 2024-08-19T10:56:56.756 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36576.log.gz 2024-08-19T10:56:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31247.log 2024-08-19T10:56:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31507.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.29841.log.gz 2024-08-19T10:56:56.759 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36334.log 2024-08-19T10:56:56.759 INFO:teuthology.orchestra.run.smithi089.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.30640.log.gz 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31247.log: gzip 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31507.log.gz 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.29366.log 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29711.log 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.31247.log.gz 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34485.log 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29711.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36334.log.gz 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.29366.log.gz 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33965.log 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34485.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29711.log.gz 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35839.log 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36255.log 2024-08-19T10:56:56.760 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.34485.log.gz 2024-08-19T10:56:56.765 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31715.log 2024-08-19T10:56:56.765 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33965.log.gz 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31056.log 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31715.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.35839.log.gz 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31767.log 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36255.log.gz 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35335.log 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.31715.log.gz 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31663.log 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31767.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31056.log.gz 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35813.log 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32794.log 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31767.log.gz 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.35335.log.gz 2024-08-19T10:56:56.766 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35813.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.31663.log.gz 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35943.log 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32794.log: gzip -5 --verbose -- 90.6% /var/log/ceph/ceph-client.admin.31975.log -- replaced with /var/log/ceph/ceph-client.admin.35813.log.gz 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36516.log 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31975.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32794.log.gz 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29607.log 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36516.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.35943.log.gz 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25099.log 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.31975.log.gz 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29607.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.36516.log.gz 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.25099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37942.log 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29607.log.gz 2024-08-19T10:56:56.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36125.log 2024-08-19T10:56:56.777 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37942.log: gzip -5 --verbose 91.5% -- -- replaced with /var/log/ceph/ceph-client.admin.25099.log.gz /var/log/ceph/ceph-client.admin.34173.log 2024-08-19T10:56:56.777 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:56.777 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9%/var/log/ceph/ceph-client.admin.36125.log: -- replaced with /var/log/ceph/ceph-client.admin.37942.log.gz 2024-08-19T10:56:56.777 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36151.log 2024-08-19T10:56:56.777 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33859.log 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.36125.log.gz 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38019.log 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.34173.log.gz 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36360.log 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.36151.log.gz 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.38019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35543.log 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33859.log.gz 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35022.log 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38019.log.gz 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31403.log 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35543.log: /var/log/ceph/ceph-client.admin.35022.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36360.log.gz 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36203.log 2024-08-19T10:56:56.778 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31403.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.35543.log.gz 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36229.log 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.35022.log.gz 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33286.log 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36229.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.31403.log.gz 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37266.log 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.36203.log.gz 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34303.log 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.36229.log.gz 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35179.log 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.33286.log.gz 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36307.log 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35179.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37266.log.gz 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.34303.log.gz 2024-08-19T10:56:56.779 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34563.log 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35179.log.gz 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30796.log 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36307.log: /var/log/ceph/ceph-client.admin.34563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30588.log 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32349.log 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30796.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36307.log.gz 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34563.log.gz 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32167.log 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32349.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.30796.log.gz 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29113.log 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30588.log.gz 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32167.log: 90.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.32349.log.gz 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37995.log 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33002.log 2024-08-19T10:56:56.780 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29113.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.32167.log.gz 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35309.log 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33002.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.29113.log.gz 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37552.log 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37995.log.gz 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33002.log.gz 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30432.log 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37162.log 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37552.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35309.log.gz 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35127.log 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37162.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37552.log.gz 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.30432.log.gz 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30078.log 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30562.log 2024-08-19T10:56:56.781 INFO:teuthology.orchestra.run.smithi089.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.37162.log.gz 2024-08-19T10:56:56.791 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35127.log: /var/log/ceph/ceph-client.admin.30078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32950.log 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5/var/log/ceph/ceph-client.admin.30562.log: --verbose -- /var/log/ceph/ceph-client.admin.31325.log 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.35127.log.gz 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34996.log 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.30078.log.gz 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30562.log.gz 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29017.log 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.32950.log.gz 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.31325.log.gz 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34225.log 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34996.log: gzip/var/log/ceph/ceph-client.admin.29017.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34589.log 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36099.log 2024-08-19T10:56:56.792 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.34996.log.gz 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34589.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29017.log.gz 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36490.log 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34225.log.gz 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34589.log.gz 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32027.log 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36412.log 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36490.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.36099.log.gz 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32027.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.32027.log.gz 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:gzip 89.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.36490.log.gz --verbose 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr: -- /var/log/ceph/ceph-client.admin.37968.log 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36412.log: /var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34459.log 2024-08-19T10:56:56.793 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37058.log 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.36412.log.gz 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34459.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.37968.log.gz /var/log/ceph/ceph-client.admin.37240.log 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36668.log 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34355.log 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37240.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.37058.log.gz 90.6% 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34459.log.gz 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33494.log 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34355.log: gzip -5 --verbose -- 90.4% /var/log/ceph/ceph-client.admin.34969.log -- replaced with /var/log/ceph/ceph-client.admin.37240.log.gz 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.36668.log.gz 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30484.log 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33494.log: /var/log/ceph/ceph-client.admin.34969.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.34355.log.gz 2024-08-19T10:56:56.794 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32479.log 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30484.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33494.log.gz 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.34969.log.gz 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29893.log 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.a.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32479.log.gz 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.30484.log.gz 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37318.log 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30380.log 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29893.log: /var/log/ceph/ceph-client.admin.37318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36798.log 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4%/var/log/ceph/ceph-client.admin.30380.log: -- replaced with /var/log/ceph/ceph-client.admin.29893.log.gz 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32661.log 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37318.log.gz 2024-08-19T10:56:56.795 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31221.log 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36798.log: /var/log/ceph/ceph-client.admin.32661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32061.log 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.30380.log.gz 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34615.log 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.32661.log.gz 91.1% -- replaced with /var/log/ceph/ceph-client.admin.36798.log.gz 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35735.log 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32061.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31221.log.gz 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31559.log 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.e.log 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35735.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.32061.log.gz 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32872.log 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33129.log 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr: 90.8% 91.9% -- replaced with /var/log/ceph/ceph-client.admin.34615.log.gz 2024-08-19T10:56:56.796 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.31559.log.gz 2024-08-19T10:56:56.797 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35735.log.gz 2024-08-19T10:56:56.797 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36746.log 2024-08-19T10:56:56.935 INFO:teuthology.orchestra.run.smithi089.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.32872.log.gz 2024-08-19T10:56:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29363.log 2024-08-19T10:56:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33155.log 2024-08-19T10:56:56.935 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36746.log: /var/log/ceph/ceph-client.admin.29363.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37578.log 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33129.log.gz 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33155.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.36746.log.gz 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25334.log 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29363.log.gz 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37578.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33155.log.gz 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25407.log 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37578.log.gz 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.25334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30822.log 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.25334.log.gz 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.25407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30132.log 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35361.log 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30822.log: /var/log/ceph/ceph-client.admin.30132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30614.log 2024-08-19T10:56:56.936 INFO:teuthology.orchestra.run.smithi089.stderr: 90.3%/var/log/ceph/ceph-client.admin.35361.log: -- replaced with /var/log/ceph/ceph-client.admin.25407.log.gz 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.30822.log.gz 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.30132.log.gz 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.35361.log.gz 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33468.log 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36902.log 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31194.log 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.30614.log.gz/var/log/ceph/ceph-client.admin.36902.log: 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34511.log 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33468.log.gz 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37396.log 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36902.log.gz 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31194.log: /var/log/ceph/ceph-client.admin.34511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37890.log 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30458.log 2024-08-19T10:56:56.937 INFO:teuthology.orchestra.run.smithi089.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31194.log.gz 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.34511.log.gz 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37292.log 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.30458.log: -- /var/log/ceph/ceph-client.admin.36954.log 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37396.log.gz 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37890.log.gz 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37292.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.30458.log.gz 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37292.log.gz 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37630.log 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33650.log 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29659.log 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1%/var/log/ceph/ceph-client.admin.33650.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.36954.log.gz 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36980.log 2024-08-19T10:56:56.938 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37630.log.gz 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.33650.log.gz 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36464.log 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36980.log: /var/log/ceph/ceph-client.admin.36438.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30874.log 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29659.log.gz 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36980.log.gz 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33991.log 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gz 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30874.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.36464.log.gz 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33807.log 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35205.log 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33991.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30874.log.gz/var/log/ceph/ceph-client.admin.33807.log: 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31030.log 2024-08-19T10:56:56.939 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36281.log 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33991.log.gz 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1%/var/log/ceph/ceph-client.admin.35205.log: -- replaced with /var/log/ceph/ceph-client.admin.33807.log.gz 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31030.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35205.log.gz 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.31030.log.gz 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31004.log 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30718.log 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31273.log 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35865.log 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.36281.log.gz 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.31004.log.gz 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31273.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.30718.log.gz 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.31273.log.gz 2024-08-19T10:56:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31481.log 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37188.log 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33598.log 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.35865.log.gz 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31689.log 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.31481.log.gzgzip 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30236.log 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.37188.log.gz 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33598.log: /var/log/ceph/ceph-client.admin.31689.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33598.log.gz 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.31689.log.gz 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32846.log 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34719.log 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32453.log 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34719.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.30236.log.gz 2024-08-19T10:56:56.941 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35891.log 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.32846.log.gz 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31455.log 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32453.log: /var/log/ceph/ceph-client.admin.35891.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.34719.log.gz 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32453.log.gz 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35891.log.gz 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37760.log 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30926.log 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32219.log 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.31455.log.gz 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29789.log 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37760.log.gz 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32924.log 2024-08-19T10:56:56.942 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30926.log.gz/var/log/ceph/ceph-client.admin.32219.log: 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29196.log 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29737.log 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.32219.log.gz/var/log/ceph/ceph-client.admin.32924.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.29789.log.gz 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29196.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.32924.log.gz 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29196.log.gz 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29165.log 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34641.log 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33338.log 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29737.log.gz 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29165.log: /var/log/ceph/ceph-client.admin.34641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33442.log 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36642.log 2024-08-19T10:56:56.943 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29165.log.gz 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.34641.log.gz 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33442.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.33338.log.gz 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33442.log.gz 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33312.log 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35569.log 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29633.log 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30354.log 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.36642.log.gz 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33312.log.gz 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29633.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.35569.log.gz 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29633.log.gz 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30262.log 2024-08-19T10:56:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29087.log 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30328.log 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.30354.log.gz 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29087.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.30262.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31108.log 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29087.log.gz/var/log/ceph/ceph-client.admin.30328.log: 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37864.log 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31108.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.30328.log.gz 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36850.log 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36177.log 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37864.log: /var/log/ceph/ceph-client.admin.36850.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31108.log.gz 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29211.log 2024-08-19T10:56:56.945 INFO:teuthology.orchestra.run.smithi089.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.36850.log.gz 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37864.log.gz 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36177.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.36177.log.gz 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32687.log 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34251.log 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37812.log 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29211.log.gz 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33755.log 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.32687.log.gz 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37812.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34251.log.gz 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.d.log 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37812.log.gz 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35257.log 2024-08-19T10:56:56.946 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34381.log 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.d.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.33755.log.gz 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35257.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.35257.log.gz 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29202.log 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34095.log 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29202.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34381.log.gz 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29202.log.gz 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37006.log 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33676.log 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37006.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.34095.log.gz 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.37006.log.gz 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32820.log 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36772.log 2024-08-19T10:56:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32820.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.33676.log.gz 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.32820.log.gz 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34329.log 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36928.log 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34329.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36772.log.gz 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34329.log.gz 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30158.log 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37708.log 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30158.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36928.log.gz 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.30158.log.gz 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34537.log 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32765.log 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34537.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37708.log.gz 2024-08-19T10:56:56.948 INFO:teuthology.orchestra.run.smithi089.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34537.log.gz 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33028.log 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37032.log 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33028.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32765.log.gz 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33028.log.gz 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34043.log 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31741.log 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34043.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.37032.log.gz 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.34043.log.gz 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33885.log 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31793.log 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33885.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31741.log.gz 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.33885.log.gz 2024-08-19T10:56:56.949 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35621.log 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33390.log 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35621.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.31793.log.gz 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.35621.log.gz 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37786.log 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29207.log 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37786.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33390.log.gz 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37786.log.gz 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37136.log 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31533.log 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37136.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29207.log.gz 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.37136.log.gz 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34407.log 2024-08-19T10:56:56.950 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33572.log 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34407.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31533.log.gz 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34407.log.gz 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34852.log 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36073.log 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34852.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.33572.log.gz 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34852.log.gz 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30510.log 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29209.log 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30510.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.36073.log.gz 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.30510.log.gz 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35439.log 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31923.log 2024-08-19T10:56:56.951 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35439.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29209.log.gz 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.35439.log.gz 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30952.log 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34823.log 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30952.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.31923.log.gz 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.30952.log.gz 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33702.log 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34823.log.gz 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28943.log 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33702.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.33702.log.gz 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35101.log 2024-08-19T10:56:56.952 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.28943.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28943.log.gz 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37344.log 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35101.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.35101.log.gz 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30106.log 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37344.log: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.37344.log.gz 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35465.log 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30106.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.30106.log.gz 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31351.log 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35465.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.35465.log.gz 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34771.log 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31351.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.31351.log.gz 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33260.log 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34771.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.34771.log.gz 2024-08-19T10:56:56.953 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32375.log 2024-08-19T10:56:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33260.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33260.log.gz 2024-08-19T10:56:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31585.log 2024-08-19T10:56:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32375.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.32375.log.gz 2024-08-19T10:56:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34943.log 2024-08-19T10:56:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31585.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.31585.log.gz 2024-08-19T10:56:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31082.log 2024-08-19T10:56:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34943.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.34943.log.gz 2024-08-19T10:56:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31949.log 2024-08-19T10:56:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31082.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.31082.log.gz 2024-08-19T10:56:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33624.log 2024-08-19T10:56:56.954 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31949.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31949.log.gz 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33182.log 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33624.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.33624.log.gz 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31819.log 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33182.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33182.log.gz 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36021.log 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31819.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31819.log.gz 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33781.log 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36021.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36021.log.gz 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29347.log 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33781.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33781.log.gz 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33729.log 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29347.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29347.log.gz 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32505.log 2024-08-19T10:56:56.955 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33729.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.33729.log.gz 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32271.log 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32505.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.32505.log.gz 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33833.log 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32271.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.32271.log.gz 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34199.log 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33833.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33833.log.gz 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34199.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34199.log.gz 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35231.log 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33364.log 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35231.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.35231.log.gz 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29971.log 2024-08-19T10:56:56.956 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33364.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33364.log.gz 2024-08-19T10:56:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37370.log 2024-08-19T10:56:56.958 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29971.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.29971.log.gz 2024-08-19T10:56:56.966 INFO:teuthology.orchestra.run.smithi089.stderr: 92.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34667.log 2024-08-19T10:56:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37370.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37370.log.gz 2024-08-19T10:56:56.967 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-08-19T10:56:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30666.log 2024-08-19T10:56:56.967 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34667.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34667.log.gz 2024-08-19T10:56:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35048.log 2024-08-19T10:56:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30666.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.30666.log.gz 2024-08-19T10:56:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37500.log 2024-08-19T10:56:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35048.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.35048.log.gz 2024-08-19T10:56:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32583.log 2024-08-19T10:56:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37500.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.37500.log.gz 2024-08-19T10:56:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31299.log 2024-08-19T10:56:56.977 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32583.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.32583.log.gz 2024-08-19T10:56:56.977 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37084.log 2024-08-19T10:56:56.977 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31299.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.31299.log.gz 2024-08-19T10:56:56.977 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29529.log 2024-08-19T10:56:56.977 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37084.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37084.log.gz 2024-08-19T10:56:56.977 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35413.log 2024-08-19T10:56:56.987 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29529.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29529.log.gz 2024-08-19T10:56:56.987 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37474.log 2024-08-19T10:56:56.987 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35413.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.35413.log.gz 2024-08-19T10:56:56.988 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35075.log 2024-08-19T10:56:56.988 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37474.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37474.log.gz 2024-08-19T10:56:56.988 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33911.log 2024-08-19T10:56:56.988 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35075.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.35075.log.gz 2024-08-19T10:56:56.988 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34147.log 2024-08-19T10:56:56.988 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33911.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33911.log.gz 2024-08-19T10:56:56.988 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31160.log 2024-08-19T10:56:56.988 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34147.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.34147.log.gz 2024-08-19T10:56:56.988 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34069.log 2024-08-19T10:56:56.988 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31160.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.31160.log.gz 2024-08-19T10:56:56.988 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29364.log 2024-08-19T10:56:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34069.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.34069.log.gz 2024-08-19T10:56:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32976.log 2024-08-19T10:56:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29364.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29364.log.gz 2024-08-19T10:56:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31845.log 2024-08-19T10:56:56.997 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30591.log 2024-08-19T10:56:56.997 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36331.log 2024-08-19T10:56:56.997 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34090.log 2024-08-19T10:56:56.997 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30591.log: /var/log/ceph/ceph-client.admin.36331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35681.log 2024-08-19T10:56:56.997 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36715.log 2024-08-19T10:56:56.997 INFO:teuthology.orchestra.run.smithi190.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36331.log.gz 2024-08-19T10:56:56.997 INFO:teuthology.orchestra.run.smithi190.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34090.log.gz 2024-08-19T10:56:56.997 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30591.log.gz 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33417.log 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35681.log: /var/log/ceph/ceph-client.admin.36715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32594.log 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35441.log 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33417.log: /var/log/ceph/ceph-client.admin.32594.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36715.log.gz 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35681.log.gz 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33417.log.gz 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.32594.log.gz 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31352.log 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr:gzip/var/log/ceph/ceph-client.admin.35441.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30957.log 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bazar.37138.log 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33587.log 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35441.log.gz 2024-08-19T10:56:56.998 INFO:teuthology.orchestra.run.smithi190.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31352.log.gz 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.bazar.37138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30567.log 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30957.log.gz 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33587.log: 83.9% -- replaced with /var/log/ceph/ceph-client.bazar.37138.log.gz 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30388.log 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33587.log.gz 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31155.log 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33965.log 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30388.log: /var/log/ceph/ceph-client.admin.31155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32474.log 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30567.log.gz 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30388.log.gz 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33965.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31155.log.gz 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32356.log 2024-08-19T10:56:56.999 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32474.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33965.log.gz 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31400.log 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32090.log 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32356.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32474.log.gz 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34959.log 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32090.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32356.log.gz 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31400.log.gz 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36787.log 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30035.log 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32090.log.gz 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37383.log 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34959.log.gz 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31822.log 2024-08-19T10:56:57.000 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36787.log.gz 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32428.log 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30035.log.gz 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31634.log 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37383.log.gz 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30717.log 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31822.log.gz 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36915.log 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35103.log 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32428.log.gz 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31634.log.gz 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32406.log 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30717.log.gz 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.f.log 2024-08-19T10:56:57.001 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35103.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36915.log.gz 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36595.log 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31774.log 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32406.log.gz 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35103.log.gz 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25304.log 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31990.log 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36595.log.gz 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.25304.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.31774.log.gz 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32064.log 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36547.log 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.25304.log.gz 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31302.log 2024-08-19T10:56:57.002 INFO:teuthology.orchestra.run.smithi190.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31990.log.gz 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.32064.log.gz 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37442.log 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35729.log 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31302.log: /var/log/ceph/ceph-client.bar.37442.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36547.log.gz 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31586.log 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35729.log: 89.1% -- replaced with /var/log/ceph/ceph-client.bar.37442.log.gz 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32929.log 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31302.log.gz 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32212.log 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35729.log.gz 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31586.log: /var/log/ceph/ceph-client.admin.32929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36037.log 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32212.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31586.log.gz 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32929.log.gz 2024-08-19T10:56:57.003 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34911.log 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32212.log.gz 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36307.log 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32737.log 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0%/var/log/ceph/ceph-client.admin.36307.log: -- replaced with /var/log/ceph/ceph-client.admin.36037.log.gz 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36062.log 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32737.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36307.log.gz 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35297.log 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34911.log.gz 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32809.log 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36062.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32737.log.gz 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31918.log 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32809.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36062.log.gz 2024-08-19T10:56:57.004 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35297.log.gz 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33197.log 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32809.log.gz 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31846.log 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36571.log 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31918.log.gz 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32238.log 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33197.log.gz 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33989.log 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31846.log.gz 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30081.log 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30410.log 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32238.log.gz 2024-08-19T10:56:57.005 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31057.log 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33989.log.gz 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30432.log 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30081.log.gz 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.30410.log.gz 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33611.log 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36187.log 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30432.log: /var/log/ceph/ceph-client.admin.33611.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.31057.log.gz 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35127.log 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36187.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33611.log.gz 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30432.log.gz 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31726.log 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34522.log 2024-08-19T10:56:57.006 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35127.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36187.log.gz 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.37289.log 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35127.log.gz 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr: 89.7%/var/log/ceph/ceph-client.admin.34522.log: -- replaced with /var/log/ceph/ceph-client.admin.31726.log.gz 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31327.log 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.foo.37289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35825.log 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.foo.37289.log.gz 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31327.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34522.log.gz 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30813.log 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37149.log 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35825.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31327.log.gz 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32857.log 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37149.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30813.log.gz 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35825.log.gz 2024-08-19T10:56:57.007 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.37311.log 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37554.log 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32857.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37149.log.gz 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.foo.37311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35321.log 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.bar.37554.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.32857.log.gzgzip 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33027.log 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33491.log 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33635.log 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr: 93.1% 89.3% -- replaced with /var/log/ceph/ceph-client.foo.37311.log.gz -- replaced with /var/log/ceph/ceph-client.admin.35321.log.gz 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37466.log 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.bar.37554.log.gz 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33027.log.gz 2024-08-19T10:56:57.008 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35753.log 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33491.log.gz 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.bar.37466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30310.log 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33635.log.gz 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.bar.37466.log.gz 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33823.log 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35753.log: /var/log/ceph/ceph-client.admin.30310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33681.log 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34549.log 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30310.log.gz 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35753.log.gz 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33823.log.gz 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34261.log 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33681.log: /var/log/ceph/ceph-client.admin.34549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34653.log 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34261.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33681.log.gz 2024-08-19T10:56:57.009 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34497.log 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34549.log.gz 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34261.log.gz 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36136.log 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36451.log 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34497.log: /var/log/ceph/ceph-client.admin.36136.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34653.log.gz -5 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37418.log 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36451.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34497.log.gz 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32979.log 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37418.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36136.log.gz 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32619.log 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36451.log.gz 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37418.log.gz 2024-08-19T10:56:57.010 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36619.log 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37114.log 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32619.log: /var/log/ceph/ceph-client.admin.36619.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32979.log.gz 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31610.log 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32619.log.gz 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36619.log.gz 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35705.log 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30011.log 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35705.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37114.log.gz 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33775.log 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31610.log.gz 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31450.log 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30519.log 2024-08-19T10:56:57.011 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33775.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35705.log.gz 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31450.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.30011.log.gz 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34983.log 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35965.log 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31450.log.gz 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30519.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33775.log.gz 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33003.log 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29877.log 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34983.log.gz 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30519.log.gz 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33003.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35965.log.gz 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35633.log 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 89.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.33003.log.gz -- 2024-08-19T10:56:57.012 INFO:teuthology.orchestra.run.smithi190.stderr: /var/log/ceph/ceph-client.admin.34734.log 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.29877.log: /var/log/ceph/ceph-client.admin.35633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33515.log 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34734.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29877.log.gz 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31798.log 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35633.log.gz 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33051.log 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34734.log.gz 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33075.log 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31798.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33515.log.gz 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35249.log 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33075.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33051.log.gz 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34213.log 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31798.log.gz 2024-08-19T10:56:57.013 INFO:teuthology.orchestra.run.smithi190.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33075.log.gz 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31180.log 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35249.log: /var/log/ceph/ceph-client.admin.34213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36475.log 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29828.log 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34213.log.gz 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35249.log.gz 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37798.log 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31180.log.gz 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33393.log 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.29828.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36475.log.gz 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37734.log 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25383.log 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33393.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37798.log.gz 2024-08-19T10:56:57.014 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29828.log.gz 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.bar.37734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34887.log 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.25383.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33393.log.gz 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.bar.37734.log.gz 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32038.log 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.34887.log: -- /var/log/ceph/ceph-client.admin.36088.log 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.25383.log.gz 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37710.log 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36088.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34887.log.gz 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33563.log 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32038.log.gz 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34015.log 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36088.log.gz 2024-08-19T10:56:57.015 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37710.log.gz 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31870.log 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30126.log 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31870.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33563.log.gz 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34015.log.gz 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37532.log 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31870.log.gz 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31476.log 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.bar.37532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35609.log 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31476.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.30126.log.gz 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30543.log 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.bar.37532.log.gz 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31476.log.gz 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32496.log 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr:gzip/var/log/ceph/ceph-client.admin.30543.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33848.log 2024-08-19T10:56:57.016 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36283.log 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35609.log.gz 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30933.log 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30543.log.gz 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32496.log.gz 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36283.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33848.log.gz 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33659.log 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30663.log 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30933.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36283.log.gz 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29965.log 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30663.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33659.log.gz 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36891.log 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30933.log.gz 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30663.log.gz 2024-08-19T10:56:57.017 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.29965.log: /var/log/ceph/ceph-client.admin.36891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34140.log 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29965.log.gz 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36891.log.gz 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29779.log 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36835.log 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.5.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34140.log.gz 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.29779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33293.log 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29779.log.gz 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37510.log 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34237.log 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36427.log 2024-08-19T10:56:57.018 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.bar.37510.log: 89.1% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36835.log.gz 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34237.log: -- replaced with /var/log/ceph/ceph-client.admin.33293.log.gzgzip 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.25328.log 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.bar.37510.log.gz 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34376.log 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34237.log.gz 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.37197.log 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.25328.log: /var/log/ceph/ceph-client.admin.34376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33149.log 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36427.log.gz 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.foo.37197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33099.log 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34376.log.gz 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.25328.log.gz 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30639.log 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.foo.37197.log.gz 2024-08-19T10:56:57.019 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33099.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33149.log.gz 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34707.log 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32905.log 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30639.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33099.log.gz 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30765.log 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34328.log 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32905.log: 89.2% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30639.log.gz 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34707.log.gz/var/log/ceph/ceph-client.admin.30765.log: 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32569.log 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32905.log.gz 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36499.log 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30765.log.gz 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32667.log 2024-08-19T10:56:57.020 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32569.log: /var/log/ceph/ceph-client.admin.36499.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35585.log 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34328.log.gz 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33465.log 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr: 88.8% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36499.log.gz 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32569.log.gz 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36763.log 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35585.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32667.log.gz 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30789.log 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33465.log: /var/log/ceph/ceph-client.admin.36763.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35585.log.gz 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35273.log 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35175.log 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30789.log: -- replaced with /var/log/ceph/ceph-client.admin.36763.log.gz 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33465.log.gz 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32691.log 2024-08-19T10:56:57.021 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32116.log 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30789.log.gz 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32691.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35273.log.gz 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29989.log 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35175.log.gz 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.37407.log/var/log/ceph/ceph-client.admin.32116.log: 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.32691.log.gz 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31204.log 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.29989.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32116.log.gz 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31252.log 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.foo.37407.log: gzip -5 86.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.foo.37407.log.gz 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr: /var/log/ceph/ceph-client.admin.32332.log 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31204.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.29989.log.gz 2024-08-19T10:56:57.022 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33539.log 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31204.log.gz 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33439.log 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31252.log.gz 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32332.log: /var/log/ceph/ceph-client.admin.33539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33171.log 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33439.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32332.log.gz 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.i.log 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33539.log.gz 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33439.log.gz 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35989.log 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33171.log: gzip/var/log/ceph/ceph-mon.i.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31131.log 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35989.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.g.log 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr: 88.7%/var/log/ceph/ceph-client.admin.31131.log: -- replaced with /var/log/ceph/ceph-client.admin.33171.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35031.log 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:56:57.023 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35989.log.gz 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.g.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31131.log.gz 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32520.log 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30280.log 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36211.log 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35031.log.gz 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30280.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32520.log.gz 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.bar.37488.log 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34352.log 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.30280.log.gz 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.bar.37488.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36211.log.gz 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32452.log 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.bar.37488.log.gz 2024-08-19T10:56:57.024 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36643.log 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36163.log 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1%/var/log/ceph/ceph-client.admin.32452.log: -- replaced with /var/log/ceph/ceph-client.admin.34352.log.gz 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34065.log 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32452.log.gz 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36163.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36643.log.gz 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30057.log 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36163.log.gz 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31228.log 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34065.log.gz/var/log/ceph/ceph-client.admin.30057.log: 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30102.log 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31228.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30057.log.gz 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31228.log.gz 2024-08-19T10:56:57.025 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35079.log 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35151.log 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr: 89.3%/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.30102.log.gz/var/log/ceph/ceph-client.admin.35079.log: 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33245.log 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35079.log.gz 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35151.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35151.log.gz 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37823.log 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35369.log 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37823.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33245.log.gz 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37823.log.gz 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35850.log 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30909.log 2024-08-19T10:56:57.026 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35850.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35369.log.gz 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34629.log 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35850.log.gz 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30909.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30909.log.gz 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.37221.log 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34629.log.gz 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36112.log 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.foo.37221.log: 89.0% -- replaced with /var/log/ceph/ceph-client.foo.37221.log.gz 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35465.log 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36112.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36112.log.gz 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36963.log 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35465.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35465.log.gz 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29921.log 2024-08-19T10:56:57.027 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36963.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36963.log.gz 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36691.log 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.29921.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29921.log.gz 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34758.log 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36691.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36691.log.gz 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30475.log 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34758.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34758.log.gz 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34860.log 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30475.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30475.log.gz 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36811.log 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34860.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34860.log.gz 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34424.log 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36811.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36811.log.gz 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34189.log 2024-08-19T10:56:57.028 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34424.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.34424.log.gz 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32188.log 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34189.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34189.log.gz 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31942.log 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32188.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32188.log.gz 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30861.log 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31942.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31942.log.gz 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29943.log 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30861.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30861.log.gz 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34578.log 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.29943.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29943.log.gz 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33729.log 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34578.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34578.log.gz 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.h.log 2024-08-19T10:56:57.029 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33729.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33729.log.gz 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36235.log 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.h.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36403.log 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36235.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36235.log.gz 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29803.log 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36403.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36403.log.gz 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36355.log 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.29803.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29803.log.gz 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35200.log 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36355.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36355.log.gz 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32953.log 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35200.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35200.log.gz 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr: 93.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-19T10:56:57.030 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33943.log 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32953.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32953.log.gz 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32282.log 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33943.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33943.log.gz 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37847.log 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32282.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32282.log.gz 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31894.log 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37847.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37847.log.gz 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30693.log 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31894.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31894.log.gz 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34039.log 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30693.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30693.log.gz 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31006.log 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34039.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34039.log.gz 2024-08-19T10:56:57.031 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34473.log 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31006.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31006.log.gz 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36013.log 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34473.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34473.log.gz 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31704.log 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36013.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36013.log.gz 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32260.log 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31704.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31704.log.gz 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32544.log 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32260.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32260.log.gz 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32544.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32544.log.gz 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34782.log 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35513.log 2024-08-19T10:56:57.032 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34782.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34782.log.gz 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32976.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.32976.log.gz 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36386.log 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31845.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31845.log.gz 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35153.log 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36386.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.36386.log.gz 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32713.log 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35153.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.35153.log.gz 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30210.log 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32713.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.32713.log.gz 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30770.log 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30210.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.30210.log.gz 2024-08-19T10:56:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35917.log 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30770.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.30770.log.gz 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32323.log 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35917.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.35917.log.gz 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29358.log 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32323.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.32323.log.gz 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34797.log 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29358.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29358.log.gz 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35283.log 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34797.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.34797.log.gz 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32427.log 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35283.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35283.log.gz 2024-08-19T10:56:57.046 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-19T10:56:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32427.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.32427.log.gz 2024-08-19T10:56:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29367.log 2024-08-19T10:56:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33208.log 2024-08-19T10:56:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29367.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29367.log.gz 2024-08-19T10:56:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36824.log 2024-08-19T10:56:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33208.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.33208.log.gz 2024-08-19T10:56:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35491.log 2024-08-19T10:56:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36824.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36824.log.gz 2024-08-19T10:56:57.051 INFO:teuthology.orchestra.run.smithi089.stderr: 96.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-19T10:56:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37656.log 2024-08-19T10:56:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35491.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.35491.log.gz 2024-08-19T10:56:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30744.log 2024-08-19T10:56:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37656.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37656.log.gz 2024-08-19T10:56:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29042.log 2024-08-19T10:56:57.051 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30744.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.30744.log.gz 2024-08-19T10:56:57.052 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37682.log 2024-08-19T10:56:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29042.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29042.log.gz 2024-08-19T10:56:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30052.log 2024-08-19T10:56:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37682.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37682.log.gz 2024-08-19T10:56:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21409.log 2024-08-19T10:56:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30052.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30052.log.gz 2024-08-19T10:56:57.056 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30692.log 2024-08-19T10:56:57.057 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.tmp-client.admin.21409.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21409.log.gz 2024-08-19T10:56:57.057 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32531.log 2024-08-19T10:56:57.059 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30692.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.30692.log.gz 2024-08-19T10:56:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29555.log 2024-08-19T10:56:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32531.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.32531.log.gz 2024-08-19T10:56:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35995.log 2024-08-19T10:56:57.071 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29555.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29555.log.gz 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31429.log 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35995.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.35995.log.gz 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29867.log 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31429.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.31429.log.gz 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37734.log 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29867.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.29867.log.gz 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31134.log 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37734.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37734.log.gz 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37448.log 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31134.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.31134.log.gz 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34121.log 2024-08-19T10:56:57.072 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37448.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.37448.log.gz 2024-08-19T10:56:57.073 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37110.log 2024-08-19T10:56:57.073 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34121.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.34121.log.gz 2024-08-19T10:56:57.073 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32401.log 2024-08-19T10:56:57.075 INFO:teuthology.orchestra.run.smithi190.stderr: 96.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35007.log 2024-08-19T10:56:57.075 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35513.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35513.log.gz 2024-08-19T10:56:57.075 INFO:teuthology.orchestra.run.smithi190.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-19T10:56:57.075 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35489.log 2024-08-19T10:56:57.075 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35007.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35007.log.gz 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31376.log 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35489.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35489.log.gz 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30498.log 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31376.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31376.log.gz 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35417.log 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30498.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30498.log.gz 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32881.log 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35417.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35417.log.gz 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30741.log 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32881.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32881.log.gz 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32643.log 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30741.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30741.log.gz 2024-08-19T10:56:57.076 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35941.log 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32643.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32643.log.gz 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36987.log 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35941.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35941.log.gz 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37173.log 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36987.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36987.log.gz 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33921.log 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37173.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37173.log.gz 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.37245.log 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33921.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33921.log.gz 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32306.log 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.foo.37245.log: 88.1% -- replaced with /var/log/ceph/ceph-client.foo.37245.log.gz 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36739.log 2024-08-19T10:56:57.077 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32306.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32306.log.gz 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34114.log 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36739.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36739.log.gz 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31658.log 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34114.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34114.log.gz 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33341.log 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31658.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31658.log.gz 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31083.log 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33341.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33341.log.gz 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33367.log 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31083.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31083.log.gz 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34164.log 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33367.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33367.log.gz 2024-08-19T10:56:57.078 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35345.log 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34164.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34164.log.gz 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32014.log 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35345.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35345.log.gz 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33799.log 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32014.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32014.log.gz 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30615.log 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33799.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33799.log.gz 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33221.log 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30615.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30615.log.gz 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35224.log 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33221.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33221.log.gz 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36667.log 2024-08-19T10:56:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35224.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35224.log.gz 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37083.log 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36667.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36667.log.gz 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30454.log 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37083.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37083.log.gz 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34806.log 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30454.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30454.log.gz 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37059.log 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34806.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34806.log.gz 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34400.log 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37059.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37059.log.gz 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34605.log 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34400.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34400.log.gz 2024-08-19T10:56:57.080 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33753.log 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34605.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34605.log.gz 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35777.log 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33753.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33753.log.gz 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.foo.37267.log 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35777.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35777.log.gz 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37035.log 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.foo.37267.log: 88.8% -- replaced with /var/log/ceph/ceph-client.foo.37267.log.gz 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33873.log 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37035.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37035.log.gz 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34680.log 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33873.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33873.log.gz 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35537.log 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34680.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34680.log.gz 2024-08-19T10:56:57.081 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31107.log 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35537.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35537.log.gz 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32785.log 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31107.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31107.log.gz 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30885.log 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32785.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32785.log.gz 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36523.log 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30885.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30885.log.gz 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35055.log 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36523.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36523.log.gz 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30224.log 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35055.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35055.log.gz 2024-08-19T10:56:57.082 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35393.log 2024-08-19T10:56:57.083 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30224.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30224.log.gz 2024-08-19T10:56:57.083 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31966.log 2024-08-19T10:56:57.083 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35393.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35393.log.gz 2024-08-19T10:56:57.083 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30174.log 2024-08-19T10:56:57.083 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31966.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31966.log.gz 2024-08-19T10:56:57.092 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30837.log 2024-08-19T10:56:57.092 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30174.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30174.log.gz 2024-08-19T10:56:57.092 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29899.log 2024-08-19T10:56:57.092 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30837.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30837.log.gz 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37749.log 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.29899.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29899.log.gz 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33705.log 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37749.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37749.log.gz 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35801.log 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33705.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33705.log.gz 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33317.log 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35801.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35801.log.gz 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37011.log 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33317.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33317.log.gz 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34833.log 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37011.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37011.log.gz 2024-08-19T10:56:57.093 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34449.log 2024-08-19T10:56:57.094 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34833.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34833.log.gz 2024-08-19T10:56:57.094 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-08-19T10:56:57.094 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34449.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34449.log.gz 2024-08-19T10:56:57.094 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32142.log 2024-08-19T10:56:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37110.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37110.log.gz 2024-08-19T10:56:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35761.log 2024-08-19T10:56:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32401.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.32401.log.gz 2024-08-19T10:56:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33416.log 2024-08-19T10:56:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35761.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.35761.log.gz 2024-08-19T10:56:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34017.log 2024-08-19T10:56:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33416.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33416.log.gz 2024-08-19T10:56:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33520.log 2024-08-19T10:56:57.097 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34017.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.34017.log.gz 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29581.log 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33520.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.33520.log.gz 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29503.log 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29581.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29581.log.gz 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32001.log 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29503.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29503.log.gz 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31637.log 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32001.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.32001.log.gz 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31611.log 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31637.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.31637.log.gz 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35969.log 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31611.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.31611.log.gz 2024-08-19T10:56:57.098 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34693.log 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35969.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.35969.log.gz 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32898.log 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34693.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.34693.log.gz 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30406.log 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32898.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.32898.log.gz 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29945.log 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30406.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.30406.log.gz 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32297.log 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29945.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.29945.log.gz 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33937.log 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32297.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.32297.log.gz 2024-08-19T10:56:57.099 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35595.log 2024-08-19T10:56:57.101 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33937.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.33937.log.gz 2024-08-19T10:56:57.101 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29139.log 2024-08-19T10:56:57.101 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35595.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35595.log.gz 2024-08-19T10:56:57.101 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30536.log 2024-08-19T10:56:57.101 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29139.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29139.log.gz 2024-08-19T10:56:57.102 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37526.log 2024-08-19T10:56:57.103 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30536.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.30536.log.gz 2024-08-19T10:56:57.103 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-08-19T10:56:57.104 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37526.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37526.log.gz 2024-08-19T10:56:57.104 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32193.log 2024-08-19T10:56:57.105 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30254.log 2024-08-19T10:56:57.106 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32142.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32142.log.gz 2024-08-19T10:56:57.121 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32380.log 2024-08-19T10:56:57.122 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30254.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30254.log.gz 2024-08-19T10:56:57.123 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33234.log 2024-08-19T10:56:57.123 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29755.log 2024-08-19T10:56:57.124 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32193.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32193.log.gz 2024-08-19T10:56:57.124 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32380.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32380.log.gz 2024-08-19T10:56:57.137 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30198.log 2024-08-19T10:56:57.138 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.29755.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.29755.log.gz 2024-08-19T10:56:57.139 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35787.log 2024-08-19T10:56:57.140 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33234.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.33234.log.gz 2024-08-19T10:56:57.140 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33546.log 2024-08-19T10:56:57.142 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35787.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.35787.log.gz 2024-08-19T10:56:57.145 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36859.log 2024-08-19T10:56:57.146 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30198.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30198.log.gz 2024-08-19T10:56:57.151 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30023.log 2024-08-19T10:56:57.152 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33546.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33546.log.gz 2024-08-19T10:56:57.153 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25352.log 2024-08-19T10:56:57.154 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36859.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36859.log.gz 2024-08-19T10:56:57.161 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30150.log 2024-08-19T10:56:57.162 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.25352.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.25352.log.gz 2024-08-19T10:56:57.163 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37838.log 2024-08-19T10:56:57.163 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30023.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30023.log.gz 2024-08-19T10:56:57.169 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31750.log 2024-08-19T10:56:57.170 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30150.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30150.log.gz 2024-08-19T10:56:57.171 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34433.log 2024-08-19T10:56:57.172 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37838.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37838.log.gz 2024-08-19T10:56:57.177 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36379.log 2024-08-19T10:56:57.178 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31750.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31750.log.gz 2024-08-19T10:56:57.179 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36047.log 2024-08-19T10:56:57.179 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34433.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.34433.log.gz 2024-08-19T10:56:57.187 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32609.log 2024-08-19T10:56:57.187 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36047.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.36047.log.gz 2024-08-19T10:56:57.189 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36939.log 2024-08-19T10:56:57.190 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36379.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36379.log.gz 2024-08-19T10:56:57.195 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37916.log 2024-08-19T10:56:57.195 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32609.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.32609.log.gz 2024-08-19T10:56:57.197 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32164.log 2024-08-19T10:56:57.198 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36939.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36939.log.gz 2024-08-19T10:56:57.203 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30302.log 2024-08-19T10:56:57.203 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37916.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37916.log.gz 2024-08-19T10:56:57.206 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35561.log 2024-08-19T10:56:57.206 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32164.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32164.log.gz 2024-08-19T10:56:57.211 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36720.log 2024-08-19T10:56:57.212 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30302.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.30302.log.gz 2024-08-19T10:56:57.213 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32833.log 2024-08-19T10:56:57.214 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35561.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35561.log.gz 2024-08-19T10:56:57.220 INFO:teuthology.orchestra.run.smithi190.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-08-19T10:56:57.221 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35874.log 2024-08-19T10:56:57.222 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32833.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32833.log.gz 2024-08-19T10:56:57.223 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29852.log 2024-08-19T10:56:57.223 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35874.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35874.log.gz 2024-08-19T10:56:57.224 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33125.log 2024-08-19T10:56:57.225 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.29852.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29852.log.gz 2024-08-19T10:56:57.225 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30981.log 2024-08-19T10:56:57.226 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33125.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33125.log.gz 2024-08-19T10:56:57.227 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33897.log 2024-08-19T10:56:57.227 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30981.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30981.log.gz 2024-08-19T10:56:57.228 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25280.log 2024-08-19T10:56:57.229 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33897.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33897.log.gz 2024-08-19T10:56:57.229 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32761.log 2024-08-19T10:56:57.230 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.25280.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.25280.log.gz 2024-08-19T10:56:57.231 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34935.log 2024-08-19T10:56:57.231 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35517.log 2024-08-19T10:56:57.232 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36720.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36720.log.gz 2024-08-19T10:56:57.232 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32761.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32761.log.gz 2024-08-19T10:56:57.232 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31680.log 2024-08-19T10:56:57.233 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.34935.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.34935.log.gz 2024-08-19T10:56:57.234 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35657.log 2024-08-19T10:56:57.234 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31680.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31680.log.gz 2024-08-19T10:56:57.235 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30366.log 2024-08-19T10:56:57.236 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.35657.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35657.log.gz 2024-08-19T10:56:57.236 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37773.log 2024-08-19T10:56:57.237 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30366.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30366.log.gz 2024-08-19T10:56:57.238 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32715.log 2024-08-19T10:56:57.239 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.37773.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37773.log.gz 2024-08-19T10:56:57.239 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33064.log 2024-08-19T10:56:57.239 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31277.log 2024-08-19T10:56:57.240 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35517.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.35517.log.gz 2024-08-19T10:56:57.241 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.32715.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32715.log.gz 2024-08-19T10:56:57.241 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36259.log 2024-08-19T10:56:57.241 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31277.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.31277.log.gz 2024-08-19T10:56:57.242 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25408.log 2024-08-19T10:56:57.243 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36259.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36259.log.gz 2024-08-19T10:56:57.243 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31424.log 2024-08-19T10:56:57.244 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34277.log 2024-08-19T10:56:57.244 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29685.log 2024-08-19T10:56:57.245 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.25408.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.25408.log.gz 2024-08-19T10:56:57.245 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31032.log 2024-08-19T10:56:57.245 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.33064.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33064.log.gz 2024-08-19T10:56:57.246 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31424.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31424.log.gz 2024-08-19T10:56:57.246 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33269.log 2024-08-19T10:56:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.31032.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31032.log.gz 2024-08-19T10:56:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30336.log 2024-08-19T10:56:57.248 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.33269.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33269.log.gz 2024-08-19T10:56:57.249 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.30336.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.30336.log.gz 2024-08-19T10:56:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30978.log 2024-08-19T10:56:57.259 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29685.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29685.log.gz 2024-08-19T10:56:57.267 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34277.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.34277.log.gz 2024-08-19T10:56:57.275 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31871.log 2024-08-19T10:56:57.276 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30978.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.30978.log.gz 2024-08-19T10:56:57.299 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37214.log 2024-08-19T10:56:57.300 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31871.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.31871.log.gz 2024-08-19T10:56:57.311 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37604.log 2024-08-19T10:56:57.312 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37214.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.37214.log.gz 2024-08-19T10:56:57.323 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32245.log 2024-08-19T10:56:57.324 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.37604.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37604.log.gz 2024-08-19T10:56:57.339 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30848.log 2024-08-19T10:56:57.340 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32245.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.32245.log.gz 2024-08-19T10:56:57.351 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35387.log 2024-08-19T10:56:57.352 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30848.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.30848.log.gz 2024-08-19T10:56:57.367 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30184.log 2024-08-19T10:56:57.368 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.35387.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.35387.log.gz 2024-08-19T10:56:57.379 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32635.log 2024-08-19T10:56:57.380 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30184.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.30184.log.gz 2024-08-19T10:56:57.383 INFO:teuthology.orchestra.run.smithi190.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.i.log.gz 2024-08-19T10:56:57.395 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29210.log 2024-08-19T10:56:57.396 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32635.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.32635.log.gz 2024-08-19T10:56:57.396 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32557.log 2024-08-19T10:56:57.397 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29210.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29210.log.gz 2024-08-19T10:56:57.398 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36542.log 2024-08-19T10:56:57.399 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32557.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.32557.log.gz 2024-08-19T10:56:57.399 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34917.log 2024-08-19T10:56:57.400 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36542.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.36542.log.gz 2024-08-19T10:56:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36876.log 2024-08-19T10:56:57.401 INFO:teuthology.orchestra.run.smithi190.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.f.log.gz 2024-08-19T10:56:57.402 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34917.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.34917.log.gz 2024-08-19T10:56:57.402 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29067.log 2024-08-19T10:56:57.403 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36876.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.36876.log.gz 2024-08-19T10:56:57.404 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36694.log 2024-08-19T10:56:57.405 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29067.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29067.log.gz 2024-08-19T10:56:57.405 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32739.log 2024-08-19T10:56:57.406 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.36694.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.36694.log.gz 2024-08-19T10:56:57.407 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31897.log 2024-08-19T10:56:57.408 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.32739.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32739.log.gz 2024-08-19T10:56:57.408 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29997.log 2024-08-19T10:56:57.409 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31897.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.31897.log.gz 2024-08-19T10:56:57.410 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29763.log 2024-08-19T10:56:57.411 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29997.log: 92.0% -- replaced with /var/log/ceph/ceph-mon.e.log.gz 2024-08-19T10:56:57.411 INFO:teuthology.orchestra.run.smithi089.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.29997.log.gz 2024-08-19T10:56:57.411 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31377.log 2024-08-19T10:56:57.412 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30900.log 2024-08-19T10:56:57.412 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29763.log: /var/log/ceph/ceph-client.admin.31377.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.29763.log.gz 2024-08-19T10:56:57.413 INFO:teuthology.orchestra.run.smithi089.stderr: 90.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34745.log 2024-08-19T10:56:57.413 INFO:teuthology.orchestra.run.smithi089.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31377.log.gz 2024-08-19T10:56:57.413 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30900.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30900.log.gz 2024-08-19T10:56:57.414 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29815.log 2024-08-19T10:56:57.415 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.34745.log: /var/log/ceph/ceph-client.admin.29815.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.34745.log.gz 2024-08-19T10:56:57.415 INFO:teuthology.orchestra.run.smithi089.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.29815.log.gz 2024-08-19T10:56:57.481 INFO:teuthology.orchestra.run.smithi190.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.g.log.gz 2024-08-19T10:56:57.503 INFO:teuthology.orchestra.run.smithi089.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-08-19T10:56:57.506 INFO:teuthology.orchestra.run.smithi089.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.d.log.gz 2024-08-19T10:56:57.617 INFO:teuthology.orchestra.run.smithi190.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.h.log.gz 2024-08-19T10:56:57.621 INFO:teuthology.orchestra.run.smithi089.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-08-19T10:56:57.968 INFO:teuthology.orchestra.run.smithi089.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-08-19T10:56:58.027 INFO:teuthology.orchestra.run.smithi089.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-08-19T10:56:58.247 INFO:teuthology.orchestra.run.smithi190.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-08-19T10:56:58.283 INFO:teuthology.orchestra.run.smithi190.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-08-19T10:56:58.295 INFO:teuthology.orchestra.run.smithi089.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-08-19T10:56:58.477 INFO:teuthology.orchestra.run.smithi190.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-08-19T10:56:58.479 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T10:56:58.479 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m1.706s 2024-08-19T10:56:58.479 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m7.232s 2024-08-19T10:56:58.479 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.337s 2024-08-19T10:56:58.522 INFO:teuthology.orchestra.run.smithi089.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-08-19T10:56:58.523 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-08-19T10:56:58.523 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m1.797s 2024-08-19T10:56:58.523 INFO:teuthology.orchestra.run.smithi089.stderr:user 0m8.202s 2024-08-19T10:56:58.523 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m0.435s 2024-08-19T10:56:58.525 INFO:tasks.ceph:Archiving logs... 2024-08-19T10:56:58.525 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/remote/smithi089/log 2024-08-19T10:56:58.526 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T10:56:59.772 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/remote/smithi190/log 2024-08-19T10:56:59.773 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T10:57:00.999 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-19T10:57:01.009 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/task/install/__init__.py", line 641, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa/tasks/mon_thrash.py", line 466, in task thrash_proc.stop_and_join() File "/home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa/tasks/mon_thrash.py", line 178, in stop_and_join return self.join() File "/home/teuthworker/src/git.ceph.com_ceph_964adda23da9f7b057cb34fdc0b82aeb7ed65324/qa/tasks/mon_thrash.py", line 170, in join self.stopping.set() AttributeError: 'bool' object has no attribute 'set' 2024-08-19T10:57:01.009 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-19T10:57:01.010 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T10:57:01.012 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T10:57:01.093 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-19T10:57:01.102 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-19T10:57:01.102 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T10:57:01.105 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T10:57:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: remote refid st t when poll reach delay offset jitter 2024-08-19T10:57:01.117 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================== 2024-08-19T10:57:01.117 INFO:teuthology.orchestra.run.smithi089.stdout:+hv01.front.sepi 67.205.162.81 3 u 38 64 377 0.093 -6.819 0.750 2024-08-19T10:57:01.117 INFO:teuthology.orchestra.run.smithi089.stdout:+hv02.front.sepi 63.231.80.2 3 u 44 64 377 0.094 -4.647 1.073 2024-08-19T10:57:01.117 INFO:teuthology.orchestra.run.smithi089.stdout:*hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.109 -6.840 0.979 2024-08-19T10:57:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-08-19T10:57:01.119 INFO:teuthology.orchestra.run.smithi190.stdout: remote refid st t when poll reach delay offset jitter 2024-08-19T10:57:01.119 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================== 2024-08-19T10:57:01.120 INFO:teuthology.orchestra.run.smithi190.stdout:*hv01.front.sepi 67.205.162.81 3 u 45 64 377 0.109 -9.957 1.136 2024-08-19T10:57:01.120 INFO:teuthology.orchestra.run.smithi190.stdout:+hv02.front.sepi 63.231.80.2 3 u 41 64 377 0.076 -8.305 0.984 2024-08-19T10:57:01.120 INFO:teuthology.orchestra.run.smithi190.stdout:+hv03.front.sepi 74.6.168.72 3 u 40 64 377 0.099 -10.118 1.536 2024-08-19T10:57:01.120 INFO:teuthology.orchestra.run.smithi190.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-08-19T10:57:01.121 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-19T10:57:01.129 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-19T10:57:01.129 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-19T10:57:01.137 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-19T10:57:01.145 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-19T10:57:01.154 INFO:teuthology.task.internal:Duration was 1740.992241 seconds 2024-08-19T10:57:01.154 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-19T10:57:01.163 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-19T10:57:01.163 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T10:57:01.166 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T10:57:01.204 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-19T10:57:01.204 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2024-08-19T10:57:01.204 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T10:57:01.252 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-08-19T10:57:01.252 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T10:57:01.263 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-19T10:57:01.263 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T10:57:01.300 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T10:57:01.331 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-19T10:57:01.331 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T10:57:01.372 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T10:57:01.433 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-19T10:57:01.444 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-19T10:57:01.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T10:57:01.456 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T10:57:01.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-19T10:57:01.477 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T10:57:01.500 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T10:57:01.508 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2024-08-19T10:57:01.520 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-08-19T10:57:01.570 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T10:57:01.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T10:57:01.616 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T10:57:01.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T10:57:01.621 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-19T10:57:01.631 INFO:teuthology.task.internal:Transferring archived files... 2024-08-19T10:57:01.632 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/remote/smithi089 2024-08-19T10:57:01.632 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T10:57:01.681 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_20:00:17-rados-main-distro-default-smithi/7859373/remote/smithi190 2024-08-19T10:57:01.681 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T10:57:01.701 INFO:teuthology.task.internal:Removing archive directory... 2024-08-19T10:57:01.701 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-19T10:57:01.720 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-19T10:57:01.746 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-19T10:57:01.756 INFO:teuthology.task.internal:Not uploading archives. 2024-08-19T10:57:01.756 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-19T10:57:01.768 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-19T10:57:01.768 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T10:57:01.770 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T10:57:01.774 INFO:teuthology.orchestra.run.smithi089.stdout: 658636 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 10:57 /home/ubuntu/cephtest 2024-08-19T10:57:01.794 INFO:teuthology.orchestra.run.smithi190.stdout: 658465 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 10:57 /home/ubuntu/cephtest 2024-08-19T10:57:01.795 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-19T10:57:01.805 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-19T10:57:01.815 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: AttributeError: 'bool' object has no attribute 'set' 2024-08-19T10:57:01.818 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-bitmap} rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_mon_workunits} duration: 1740.9922411441803 failure_reason: '''bool'' object has no attribute ''set''' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-08-19T10:57:01.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-19T10:57:01.957 INFO:teuthology.run:FAIL